Anthropic PBC

Dario Amodei was VP of Research at OpenAI in 2020, overseeing the development of GPT-3 — the most capable language model ever built at the time. He and a group of colleagues, including his sister Daniela, became increasingly concerned that OpenAI was moving too fast and prioritising commercial deployment over safety research. In December 2020, Dario, Daniela, and five other senior OpenAI researchers resigned. It was the largest talent departure in OpenAI's history.

Anthropic was founded in January 2021 with $124 million in seed funding and a stated mission: the responsible development and maintenance of advanced AI for the long-term benefit of humanity. The company's structure — a Public Benefit Corporation — was designed to balance commercial returns with its safety mission. Anthropic committed to publishing safety research regardless of whether it helped competitors. In an industry where companies routinely hired researchers away from academia, Anthropic positioned itself as the place where safety-focused researchers could do commercial work without compromising their principles.

Anthropic developed a training technique called Constitutional AI — a method for training AI systems to be helpful, harmless, and honest by having the AI evaluate its own outputs against a set of principles. The approach was published openly. Anthropic launched Claude, its AI assistant, in March 2023. Claude was designed from the ground up with safety properties that Anthropic's founders believed were absent from competing systems — including the ability to refuse harmful requests while explaining why, rather than simply complying or producing a generic error.

Amazon invested up to $4 billion in Anthropic in September 2023 — one of the largest investments in an AI company in history. Google had previously invested $300 million. The investments valued Anthropic at approximately $20 billion. The funding came with commercial agreements: Amazon would deploy Claude across its AWS services; Google would integrate Claude into its cloud offerings. Anthropic had become, despite its non-commercial founding philosophy, one of the most commercially significant AI companies in the world.

By 2024, Anthropic's Claude 3 models were benchmarking as the most capable AI systems available — outperforming OpenAI's GPT-4 on many measures. The company that had left OpenAI over safety concerns had built an AI that was simultaneously safer and more capable than what it had left behind. Critics argued this proved that safety and capability were not in tension. Others argued that building more capable AI, however carefully, was inherently dangerous regardless of the precautions taken. Anthropic's founders had not resolved this tension. They had made it more precise.