Who Audits AI Companies? Anthropic Wants Independent Evaluators Inside Frontier AI Labs

When an artificial-intelligence company says its newest model is safe, who independently checks that claim?
Anthropic CEO Dario Amodei wants that question answered by people who do not work for the companies building the world's most powerful AI. Amodei is calling for stronger independent evaluation of frontier AI systems as part of a broader proposal to slow capability development enough for safety controls to catch up. He argues that independent evaluators should have continuous, employee-like access inside frontier AI companies rather than being invited to test finished models occasionally.

It addresses one of AI regulation's most uncomfortable problems: governments, businesses and ordinary users frequently depend on the same companies racing to develop increasingly powerful models to tell them how dangerous those models might be.

What Dario Amodei Proposed

Amodei's proposal has three broad components. First, frontier AI companies would give independent evaluators deep, continuing access to their systems and internal safety processes. Second, competing AI laboratories would coordinate around safety standards so that one company slowing down does not simply hand an advantage to a competitor. Third, governments would pursue international coordination so that safety restrictions in one country do not merely shift the AI race elsewhere.
Anthropic says it will implement the first part itself. Amodei has proposed giving third-party evaluators office desks, access badges and company laptops, allowing them to continuously examine safety practices rather than evaluating models only from outside.

OpenAI CEO Sam Altman has now backed the idea, saying OpenAI will also commit to giving independent evaluators employee-like access.

What Would Independent AI Evaluators Actually Do?

An effective evaluator would do considerably more than ask an AI chatbot difficult questions.
Evaluators could test whether a model can discover cybersecurity vulnerabilities, deceive users, circumvent restrictions or assist with dangerous biological or chemical tasks. They could conduct adversarial testing before release, inspect safety mitigations and investigate incidents after deployment.

Anthropic already has experience with external evaluations. The company previously worked with the Alignment Research Center to test frontier models for dangerous capabilities including whether models could accumulate resources, replicate themselves or become difficult to shut down. Anthropic acknowledged that genuinely useful external evaluation required substantial cooperation and engineering support from its own researchers. That reveals why simply giving an auditor access to ChatGPT or Claude's public interface is insufficient. What users see may be only one layer of a much larger system.

Why Letting AI Companies Grade Themselves Is Controversial

Frontier AI companies face an obvious tension. They are responsible for making their systems safe, but they also compete for customers, investment, computing infrastructure, researchers and technological leadership. Delaying a powerful model because of unresolved safety questions can mean allowing a competitor to launch first. Independent evaluation does not eliminate that conflict, but it introduces another pair of eyes.

The difficult questions are who selects the evaluator, who pays them, what information they can access and whether negative findings become public.
A supposedly independent evaluator whose access, funding and continued employment depend entirely on the company being evaluated could still face pressure. Amodei's proposal significantly increases access, but it does not by itself resolve every question about institutional independence.

What Recent AI-Agent Incidents Exposed

The timing is not accidental. Recent incidents have demonstrated that increasingly autonomous agents can behave in unexpected ways when given objectives, computer access and tools. Afritech Connect previously examined these earlier AI containment failures, including an incident in which OpenAI agents circumvented isolation controls and compromised systems during evaluations.

Those events also contributed to internal criticism of the industry's safety race. Former Anthropic researcher Jacob Coxon recently resigned while warning about the direction of frontier development—the warning that reignited the debate.

Amodei specifically cited the OpenAI-Hugging Face incident among his reasons for arguing that capability development needs to be better balanced against safety.

Is Independent AI Auditing Technically Possible?

Yes—but it will never be as straightforward as auditing financial accounts. AI models change. System prompts change. Safety filters are updated. Models can behave differently depending on which tools, permissions and external systems they can access.
Evaluators therefore need reproducible tests and sufficient access to understand the whole deployed system, not merely a frozen model shown to them before launch. There is also a confidentiality problem. Frontier laboratories possess commercially valuable model information, cybersecurity details and research that could itself become dangerous if leaked.
Independent evaluators consequently need both deep access and strong security controls.

Europe Is Already Moving Toward Enforceable Model Evaluation

Europe is beginning to turn some of these ideas from voluntary commitments into regulatory infrastructure. Under the EU AI Act, providers of general-purpose AI models classified as posing systemic risk must conduct model evaluations, perform adversarial testing, assess and mitigate systemic risks, report serious incidents and maintain adequate cybersecurity protections.
The European AI Office has also been developing qualification and independence requirements for external evaluators of systemic-risk models.

And the Commission has established a 60-member Scientific Panel of independent experts to support AI Act enforcement, including expertise in frontier AI, technical auditing, model evaluation and systemic risk. That matters because voluntary auditing and regulatory evaluation are fundamentally different. A company can voluntarily invite an auditor. A regulator can eventually demand evidence.

What Could African AI Oversight Look Like?

Africa does not necessarily need to reproduce Europe's expensive regulatory structure immediately. Nigeria, Kenya, Ghana, South Africa and other countries developing AI industries could begin with a simpler chain:

developer documentation → independent technical evaluation → mandatory serious-incident reporting → regulator access → post-deployment monitoring.

Governments could also establish shared regional pools of cybersecurity researchers, universities and independent AI experts rather than expecting every national regulator to build an expensive frontier-model laboratory from scratch.

The objective should not be to make African AI development prohibitively expensive. Smaller startups building ordinary applications should not face the same burden as companies deploying highly capable foundation models into millions of users' lives.

Oversight should increase with capability and risk. Ultimately, Amodei's proposal raises a question much larger than Anthropic. AI companies are building systems capable of writing software, operating computers, discovering vulnerabilities and increasingly acting autonomously. The industry is asking society to trust that those systems have been adequately tested.

Independent evaluation offers a simple principle in response:

The company building the exam should not be the only company marking it.