The Audit That Isn't Ready

It's a Thursday afternoon. The chief audit executive is presenting the annual risk assessment to the audit committee. She walks through cybersecurity controls, SOX compliance, vendor risk management. Solid marks across the board.

Then a director asks: "What about the AI systems we're deploying across claims processing and underwriting?"

She pauses. "Those fall under our existing IT controls framework. Change management, access controls, data integrity checks—all covered."

The director presses: "But who's testing whether the outputs are accurate? Whether they're biased? Whether they've drifted since deployment?"

Silence.

She's not incompetent. Her team is experienced, credentialed, and thorough. They audit what they were built to audit. The problem is that AI systems fail in ways her audit program was never designed to detect.

What I Keep Seeing

I've trained over 60,000 professionals across governance, compliance, and AI. Some of the sharpest people in the room are audit and compliance leaders. They know controls. They know evidence. They know how to hold an organization accountable.

But when I run a session on AI governance and ask a simple question—"How would you audit an AI system that's making lending recommendations?"—the room splits.

Half immediately reach for what they know: access controls, change management logs, vendor risk assessments, data classification. Good instincts. Real controls. But controls that wrap around the system without ever looking at what the system produces.

The other half pauses. They know something is different here but can't name it. They'll ask: "What do you mean by 'audit the output'? We audit the process."

That's the gap in one sentence. They audit the process. AI risk lives in the output.

When I push further—"What if the model approves 94% of applicants from one zip code and 60% from another, and nobody's checked?"—the room goes quiet. Not because they don't care. Because they don't have a method for it. Nobody handed them a workpaper template for demographic disparity analysis. Nobody trained them on drift detection. Nobody told them that a model that passed testing six months ago might be producing different results today.

These are skilled professionals being asked to assure something their entire career didn't prepare them for. And most of them know it.

The Pattern

Traditional audit asks a specific set of questions: Who had access? Was the change approved? Did the control execute? Is there an audit trail?

AI governance requires a different set entirely: Did the model's outputs degrade after deployment? Did a subgroup experience systematically worse outcomes? Can the organization explain a specific decision to a regulator, a customer, or a court? Can a human meaningfully override the system—and is there evidence that they do?

That's not a training gap. It's a control-design gap. The audit function is certifying the controls around the system while missing the failures produced by the system itself.

And the profession knows it. AuditBoard's 2026 Focus on the Future survey found that "fewer than 30% of internal audit leaders feel confident auditing AI-related risks." 63% of surveyed organizations haven't defined formal risk appetites or governance frameworks for AI. The people responsible for assurance are telling you, on the record, that they aren't ready.

Meanwhile, the EU AI Act's main high-risk obligations take effect August 2, 2026—risk management, technical documentation, human oversight, post-market monitoring, incident reporting. These aren't principles. They're operational requirements. And most audit functions don't have the methods, tools, or people to evaluate any of them.

What Happens When the Audit Isn't Ready

New York City passed one of the most specific AI accountability laws in the country. Local Law 144 requires annual bias audits for automated hiring tools, public posting of audit summaries, and notices to candidates. On paper, this is exactly the kind of structured assurance framework governance advocates have been asking for.

In December 2025, the New York State Comptroller audited the enforcement. The results were damning.

The agency responsible for enforcement—the Department of Consumer and Worker Protection—reviewed 32 companies and found a single instance of non-compliance. The Comptroller's auditors reviewed the same 32 companies and found at least 17 instances of potential non-compliance. 75% of test complaint calls to the city's 311 hotline were misrouted and never reached the responsible department. The agency hadn't conducted additional public outreach since May 2023.

The law existed. The mandate existed. The audit function behind it was underpowered—lacking rigorous technical methods, consistent complaint handling, and the capacity to evaluate substance rather than check boxes.

This is the pattern at the organizational level too. The audit plan exists. The AI systems are nominally "covered." But the coverage is IT general controls applied to something that doesn't fail like IT. And the gap stays invisible until a regulator, a lawsuit, or an incident forces someone to look at what the system actually produced.

That's already happening. Eightfold AI was sued in January 2026 for generating applicant scores without informing candidates or giving them a way to challenge inaccuracies—an assurance failure around transparency and contestability, not access controls. Workday's AI screening tools are facing a nationwide collective action after a federal judge in March 2026 allowed age-discrimination claims to proceed. In both cases, a traditional audit would have found the controls around the system functioning as designed. The liability came from what the system did, not how it was managed.

What Needs to Change

The solution isn't to blame your audit team. It's to recognize that AI assurance is a different discipline—and staff, scope, and measure it accordingly.

Separate AI systems on the audit plan. Stop treating them as a subset of IT general controls. AI systems that touch customers, pricing, underwriting, hiring, or material decisions need their own risk assessment, their own test plan, and their own evidence requirements. The question isn't "are access controls in place?" It's "are the outputs fair, accurate, explainable, and stable?"

Add outcome testing to the methodology. This means testing model outputs across demographic groups, monitoring for drift over time, and validating that the system performs in production the way it performed in testing. If your audit team doesn't know how to do this, that's the skill gap to close first—not a reason to skip it.

Define what "evidence" means for AI. Traditional evidence is an access log, a change ticket, a sign-off. AI evidence includes model documentation, test results across subgroups, drift monitoring reports, explainability artifacts, override logs, and records showing that human oversight actually changed outcomes. If you can't produce this package, you can't defend the system to a regulator. And assembling it from scattered screenshots and spreadsheets the week before fieldwork isn't a strategy.

Build the capability or bring it in. The IAPP found that 77% of organizations are working on AI governance but only 1.5% said they won't need more staff. Half of current AI governance professionals sit in compliance, legal, or privacy—functions absorbing AI oversight without the model-risk and technical assurance skills the work now demands. You either invest in upskilling your existing audit team, hire specialists, or engage external expertise. Hoping the current team can stretch isn't a strategy.

Put a clock on it. August 2, 2026 isn't a future deadline—it's four months away. If your organization deploys high-risk AI systems in the EU or serves EU customers, the compliance obligations are specific and enforceable. Even outside the EU, the direction is clear: regulators and courts are already asking for evidence that AI systems are governed, tested, and auditable. The organizations that build this capability before the first enforcement action will be in a fundamentally different position than those scrambling after it.

The Real Test

Ask your chief audit executive three questions this week.

Which AI systems in customer-facing or material workflows are on the audit plan—and what specifically is being tested beyond access controls and change management?

Does the team have the skills and methods to evaluate model output quality, bias, drift, and explainability? If not, when does that capability arrive?

If a regulator asked tomorrow for evidence that your AI systems are governed, monitored, and auditable—could you produce it?

If the first answer is "we haven't scoped AI systems separately," you have a coverage gap. If the second is "we're working on it," you have a timing problem. If the third is "not quickly," you have an evidence problem that a traditional audit won't solve.

The Bottom Line

Most organizations don't have an AI audit problem. They have an AI assurance problem.

The audit function was built for a world where systems did what they were programmed to do. AI systems do what they were trained to do—which isn't always the same thing, and isn't always stable over time.

If your assurance model can't tell the difference, neither can your board.

P.S. This is the problem we built AssuranceOps to solve. Most teams have controls and policies—what they don't have is structured, defensible evidence that maps to what AI auditors and regulators actually need. We started with SOC 2 evidence operations. Now we're expanding into AI assurance—use case inventories, testing records, oversight packs, and monitoring evidence. If you want to see where your current audit coverage stands against AI assurance requirements, book a 20-minute diagnostic at https://assuranceops.com/ai-assurance