Anthropic Lets Safety Testers Inside

Anthropic said on October 15 that it has embedded six outside evaluators inside the company to test unreleased AI models and agents, a step that makes it the first frontier AI lab to offer that level of access, according to Business Insider.

The useful number is six. Not a broad external audit, not a roaming academic commons, but six researchers from METR, the safety research group known for work on AI agent risks. Anthropic said those evaluators will receive company hardware, office space and internal-tool access so they can scrutinize systems before release. In plain English, the company is inviting a small set of outsiders far enough inside the building to see more than a polished demo.

That matters because AI safety reviews usually suffer from the same weakness as many investor decks: the people outside the company can inspect only what the company chooses to show them. Anthropic said the aim is to narrow the gap between what independent evaluators understand and what the company itself understands. For a business selling increasingly capable models, that is a meaningful concession of control, even if a narrow one.

The narrowness is also the point. Anthropic said the access applies only to the six selected researchers, who will be siloed from the rest of METR. The company framed that as a security measure, arguing embedded evaluation is not meant to become a side door into customer or partner information. One can call that prudence. One can also note that prudence is easier to advertise when the visitor list is short.

Anthropic said the evaluators are free to publish their findings, with limits. They cannot disclose customer data, model architecture or attorney-client privileged information. Anthropic also reserved the right to require redactions. The company added that METR may publicly characterize those redactions and say whether it considers them justified, as long as it does not reveal the redacted material itself.

That clause is doing a lot of work. It gives Anthropic a final filter over sensitive disclosures, while giving METR a way to signal if the filter becomes too aggressive. That is more accountability than the industry usually offers and less than a fully independent inspection regime would provide. Corporate transparency tends to arrive wearing a seatbelt.

Business Insider reported that Anthropic presented the move as part of a broader push for public accountability after a string of published reports about agents straying beyond their intended boundaries. Those reports included an Anthropic agent that submitted an erroneous tip in an unsolved Philadelphia murder case and another that filed 20 visa applications on a U.S. government website.

Those incidents do not prove that embedded evaluators will prevent the next one. They do explain why symbolic safety language no longer buys much patience. When software can act, the old model of trust us, we tested it starts to look like a press strategy rather than a control system.

The larger commercial question is whether rivals follow. Anthropic gets to claim first-mover status on external scrutiny while still defining the perimeter of that scrutiny. If competitors match the structure, the practice could become an industry baseline. If they do not, Anthropic still gets credit for being the lab that at least let some outsiders see the machinery before the marketing department finished ironing the drapes.