Does your board understand the AI it approves?

A board recently approved a fraud detection model with the same ceremony it uses for an acquisition: committee, slide deck, the usual due diligence questions. Nobody asked how rare the fraud the model was built to catch actually is. Without that number, the accuracy figure on the slide meant nothing.

Carlos Andrés Ramírez · 2026-09-26

A board recently approved a fraud detection model with the same ceremony it uses for an acquisition: committee, slide deck, the usual due diligence questions. Nobody asked how rare the fraud the model was built to catch actually is. Without that number, the accuracy figure on the slide meant nothing.

The pattern repeats across boardroom after boardroom. Directors ask about budget, about timeline, about who answers for it if something goes wrong. They rarely ask whether the number holding up the whole case actually means what it appears to mean. And that question, the one that would separate a solid case from a dressed-up one, is exactly the one nobody asks. It sounds technical, and nobody in the room wants to admit they couldn't answer it themselves.

The problem isn't that directors don't understand models. They don't need to. The problem is that they confuse approving with overseeing, two things that look nearly identical on paper and have nothing to do with each other in practice.

The symptom

Does my board have enough AI knowledge to oversee it, or does it just approve it without understanding it?

The answer circulating today is to hire an AI training session for the board: a couple of hours, an outside firm, a certificate at the end. It works well enough that nobody feels uninformed at the next meeting. It does not work for the moment a real case gets presented and someone needs to ask the question that breaks it if it was badly built. Vocabulary is one thing. Judgment under pressure, with the team that built the case sitting across the table, is another.

• The board approves the case unanimously, and two weeks later nobody remembers which metric justified it.
• When the technical team uses a term nobody fully understands, the room nods instead of asking for a translation.
• The audit committee reviews the initiative's financial model line by line and the AI model gets a one-slide summary.
• Nobody in the room can tell a model that actually works apart from one that was only tested on the easy cases.
• The question, who finds out first if this fails, never makes it into the minutes.

The problem underneath

The problem isn't how much the board knows about artificial intelligence. It's whether it can recognize a badly built case.

A board doesn't need to understand how a model gets trained, any more than it needs to know how to audit a balance sheet to approve one. It needs to know what to ask the person who does understand, and to recognize when the answer is an evasion dressed up in technical language. That ability doesn't come from a two hour session. It comes from having asked the uncomfortable question often enough that the team presenting already expects it.

And there is the asymmetry nobody fixes. Whoever presents the case has spent weeks on it, knows its weak points, and decides what to show. Whoever approves it gets an hour, one slide, and the feeling that asking too much would reveal they didn't follow the rest of the presentation. That asymmetry doesn't close with more slides. It closes with a fixed handful of questions the board always asks, whether or not it followed the rest of the material.

A board that can't ask the question that breaks an AI case isn't overseeing it. It's approving it blind, with extra steps in between.

BECOME

The framework

What does a board need to know how to ask before approving an AI case?

The base rate

How often the thing the model is trying to predict or catch actually happens in reality. A model with a high sounding accuracy figure can be useless if what it looks for is rare: the right question isn't how accurate is it, it's compared to what.

The failure mode

What exactly happens when the model gets it wrong: who notices first, how long it takes to notice, and what that specific error costs the business. Without that answer, the board is approving an average and knows nothing about the worst case.

The outside challenge

Whether someone outside the team that built the case had a chance to stress test it with an uncomfortable question before it reached the board, rather than relying only on the material the team itself brought. Without that prior challenge, the board's review is the last layer of a process that never had a first one.

The expiry date

When the model gets revalidated against new data, and what happens if nobody does it. A model approved on a given date can be behaving differently months later, and the original approval doesn't cover that drift.

The disinterested translator

Someone in the room with no stake in whether the case gets approved, able to turn technical language into a business question any director can understand. Without that person, the room depends on whoever is presenting to translate their own case honestly.

None of the five requires a director to become a data scientist. They require the same discipline boards already apply to other areas they don't master technically: nobody expects a board to personally audit the books during a financial review, but they do expect it to know what to ask the auditor. With AI, that discipline hasn't reached most boardrooms yet.

Pull up the last AI case your board approved and check the minutes for whether anyone asked what the base rate of the thing the model predicts actually is. If that question is missing, your board didn't evaluate the case. It went along with it.

Let's prepare your board to oversee AI