Anthropic Lets Outside Watchdog Inside Its Walls

Anthropic has started letting an outside nonprofit roam its offices with the same access as an employee, a significant step in the AI industry's halting effort to prove it can police itself.

The company announced Thursday that METR, a nonprofit that evaluates frontier AI systems for risk, now has a team working inside Anthropic's offices with access to its hardware and internal systems. Anthropic is calling it "employee-like access," and it's meant to let METR assess the dangers of Claude models more thoroughly than an arm's-length audit would allow.

METR isn't new to this work. The nonprofit has already evaluated models from OpenAI, Google, and Meta, making it one of the few independent groups with a track record across multiple top labs. What's new is the depth of access Anthropic is granting: a resident presence rather than a periodic check-in.

The announcement is the first concrete move in a broader push by AI labs to bring in third-party evaluators as a check against the risks that increasingly capable systems could pose to the economy, cybersecurity, and potentially more existential concerns. That push gained unusual momentum in September, when Anthropic CEO Dario Amodei proposed embedding external auditors inside leading labs and found an unlikely ally: OpenAI's Sam Altman and SpaceX's Elon Musk, both normally happy to spar with Amodei, publicly backed the idea.

Thursday's rollout turns that rare moment of agreement into an actual practice, at least at Anthropic. OpenAI has not yet detailed how or when it will bring in its own evaluators, even though Altman endorsed the concept.

Why Anthropic is doing this now

In its announcement, Anthropic framed the move as a response to its own models getting more powerful and harder to evaluate. "As models have become more powerful, they've required stronger safeguards and more challenging risk assessment," the company said. It also acknowledged the limits of transparency: "We want to be publicly accountable for the decisions we're making. But we can't publicly share all the information relevant to our safety work."

That tension sits at the center of the whole arrangement. Anthropic argues that publishing everything about its safety testing could hand useful information to malicious actors or rival labs, so instead of full disclosure, it's opting for a trusted insider who can vouch for what's happening behind closed doors without broadcasting the details publicly.

To explain the model, Anthropic pointed to bank supervision, where regulators station resident examiners inside the largest banks rather than relying solely on periodic external audits. The comparison is meant to signal permanence and depth: this isn't a consultant dropping in for a scheduled review, but a standing presence with ongoing visibility into the company's systems.

The skepticism

Not everyone is convinced that labs grading their own homework, even with an embedded outsider, solves the underlying problem. Critics have pointed to the accounting industry as a cautionary tale, where auditors who are paid by the companies they evaluate have sometimes faced conflicts of interest that undermined the independence of their findings. The worry is that an evaluator with ongoing access and a working relationship with a lab could end up softer on that lab than true independence would require.

The other critique is more structural: that voluntary arrangements between AI companies and nonprofits are no substitute for government regulation with actual enforcement teeth. Embedding an evaluator is a choice Anthropic made and can presumably unmake; a regulatory requirement would not carry that same flexibility.

Anthropic's response, in effect, is that the bank-supervision model already shows how private-sector risk can be monitored closely without requiring every detail to become public, and that this kind of access is better than the status quo even if it's not the same as binding oversight.

What to watch

The real test will be what METR's embedded access actually produces: whether its findings lead to changes in how Anthropic deploys or restricts its models, and whether any of that becomes visible to the public even in summary form. Also worth watching is whether other labs follow through on the September consensus. Altman said Amodei's proposal was a good idea, but OpenAI has yet to say what its own version of external access would look like, or when it might start.

For now, Anthropic is the only major lab that's turned the talk about third-party oversight into something an outside group can actually walk into the building and use.