AI agent governance: who in your company knows what the machine did yesterday?
Two thirds of French companies use generative AI. Almost none watch their agents. Here is AI agent governance in four lines.
A few days ago, in the office of the head of a mid-sized industrial group, the conversation went off the rails in thirty seconds. I asked how many artificial intelligence tools were running in his company. He answered two, maybe three. His finance director, sitting next to him, counted seven. A silence.
Then the sentence I hear more and more often: “Hold on, what exactly does that one have access to?”
Nobody in the room had the answer. And that is exactly what AI agent governance is: the ability to answer that question, on a Tuesday morning, without calling a supplier.
The paradox: everyone uses it, almost nobody watches it
The figures came out this month and they are unambiguous. According to the Banque de France survey published on 3 September 2026, covering some 7,000 companies between late February and early April, 67 % of French companies with at least 20 employees say they use generative AI tools. The rate climbs to 83 % above a thousand employees, and already stands at 64 % among the smallest in the sample.
Adoption has happened. It happened without you, without a budget, without a steering committee.
But the same survey spells out two things few commentators picked up. First, that this use remains “mainly limited or experimental”. Second, that it is concentrated in support and cross-cutting functions: administration, sales. In other words, AI came in through the service entrance, and stayed there.
That is where the subject changes. Because between an employee asking an assistant to rewrite an email and an agent that reads your databases, drafts a client reply and sends it, there is not a difference of degree. There is a change of nature. The first produces a draft someone reviews. The second takes a decision.
Written governance protects nothing
I long believed the problem was a technical lag. It is not. It is organisational, and the Wavestone benchmark on AI security puts a number on it with almost embarrassing clarity.
In their panel, 87 % of organisations had defined elements of AI governance. The façade holds. But only 7 % had the expertise needed to face the risks that come with it. On incident response, measured maturity falls to 9 %, with fewer than one organisation in three having even begun to integrate AI scenarios into its response plan.
One detail stopped me longer than the others. In that same panel, 71 % of companies do collect their application logs. But only 13 % send them to their monitoring centre. Translation: the trace exists, nobody reads it.
The 2026 edition of the benchmark, covering more than two hundred organisations, confirms the mechanism. Three quarters have policies for secure AI use. Roughly one in ten has put in place technical protections against the attacks specific to these systems.
On one side, a signed charter. On the other, nothing behind it.
Why your brain lowers its guard in front of a machine
Here I change hats. Because what is at play is not only a procedural gap: it is a reflex of the brain, and it has been documented for twenty-five years.
Researchers call it automation bias. Linda Skitka, Kathleen Mosier and Mark Burdick dissected it in a series of papers published in the International Journal of Human-Computer Studies. They distinguish two families of error. Errors of omission: you fail to react to a problem because the machine did not flag it. And errors of commission: you follow the machine’s recommendation when other, more reliable information says the opposite.
Their conclusion about the origin of the second family is the sentence I quote most often in sessions. They result from a combination of two things: not going to check the available information, and believing in the superiority of the automated tool’s judgement.
Raja Parasuraman and Dietrich Manzey extended the work in Human Factors in 2010, with a finding that should give pause to any leader counting on their teams’ vigilance. Complacency towards automation appears above all when people are doing several things at once, it is found in novices and experts alike, and it is not corrected by repetition.
Read that sentence again while thinking about your leadership team. Competent people, overloaded, handling four subjects in parallel. That is the most exposed population, not the best protected.
Your brain is not lazy, it is economical. Faced with an answer that arrives fast, well written and self-assured, it does what it always does: it saves the effort of verification. It is exactly the same mechanism I described about companies that depend too much on their leader, with one difference. When you delegate to a human, you keep a doubt. When you delegate to a machine, the doubt goes out.
AI agent governance fits in four lines
The good news: this is not about writing a manual. In the companies I work with, one page per agent is enough, and it has four entries.
A perimeter. Which data, which mailboxes, which folders does this agent have access to? The question is not technical, it is about your assets. Your meeting notes, your tender responses, your sales scripts are capital you spent fifteen years building.
A ceiling. What can it trigger on its own, and above what amount, what volume or what level of sensitivity must it stop and ask a human? An agent without a ceiling is not a productive tool, it is unbounded exposure.
A log. Who can say, this morning, what that agent did yesterday? If the answer is nobody, you do not have a tool, you have a blind spot working in your name.
A stop button, with a person’s name next to it. Not a function, not a department: a name. And the time within which that person can actually stop everything.
If you keep only one, keep the fourth. It reveals all the others, and it is almost always the one missing.
What actually works: naming someone accountable, not buying a tool
I kept the best for last, because it is the most useful result in this whole literature and the least known among company leaders.
Skitka and her colleagues did not only measure automation bias. They looked for what reduces it. And what reduces it is neither training, nor a better tool, nor a warning on the screen. It is accountability: making participants answerable for their performance or for the accuracy of their decisions brought the bias rates down.
That is why I consider AI agent governance a leadership subject rather than an IT one. A technical manager can install a log. They cannot decide who in the company will answer for what the machine did. Only the person at the top can say that sentence.
It is also the difference between companies that get something out of AI and those that stack up subscriptions. In an earlier piece I explained why so few mid-sized companies genuinely succeed at adopting AI. A year on, the dividing line has moved. It no longer separates those who have AI from those who do not. It separates those who know what their agents do from those who do not.
A word on timing, to finish. The transparency obligations of the European regulation on artificial intelligence have applied since 2 August 2026, with additional time until 2 December for machine-readable marking of systems already in service. I do not put much faith in fear of penalties as a driver. But I do believe in the virtue of a date: by the end of the year, you will need to be able to say which of your company’s content was produced by a machine. You may as well start by knowing which machines are running.
Do the exercise this week. Get three people together, list your agents, and fill in the four lines for each. If you get stuck on the fourth, you have just found your project for the quarter.
If you want to look together at what is actually running in your company and what it is worth, I am happy to spend thirty minutes on it.
Frequently asked questions
What is AI agent governance in a company?
It is the set of rules defining what an artificial intelligence agent may see, decide, trigger and spend, together with the trace it leaves and the person entitled to stop it. It differs from an AI usage policy, which governs employees: here you are governing a system that acts alone between two human checkpoints. In practice it comes down to four things: an access perimeter, an action ceiling, a readable log and a named person with a stop button.
Does a 50-person company really need governance for its AI agents?
Yes, and for a reason that is not regulatory. An agent plugged into a mailbox, a CRM or a pricing spreadsheet acts on real decisions, with or without a written procedure. In a smaller company the risk is in fact more concentrated than in a large group: there is no substantial IT department and no internal control function to catch a mistake. The right scale is not a hundred-page manual, it is one page per agent.
How do I find out whether my teams already use AI agents without my knowing?
Three checks are enough for a first pass. Look at software subscriptions paid on a company card outside the IT budget. Ask which tools are connected to your email and your shared file space. Then ask the question in a team meeting without looking for a culprit: undeclared use surfaces when it stops being punishable. Most leaders discover three to five tools they did not know about.
What does European regulation say about content produced by an AI?
Article 50 of the European regulation on artificial intelligence sets transparency obligations applicable since 2 August 2026: clearly informing a person who is talking to a machine, and making synthetic content identifiable. Systems already on the market before that date have additional time, until 2 December 2026, to satisfy the machine-readable marking obligation. For a company leader the useful deadline is not the penalty: it is the date by which you must be able to say which of your company's content was produced by a machine.