Particle.news
Download on the App Store

Anthropic Warns AI Is Beginning to Build Itself and Urges a Verifiable Global Pause

The company says a coordinated slowdown would give regulators and researchers time to study risks from recursive self‑improvement under rules that can be independently checked.

Overview

  • Anthropic, which published a detailed report on Thursday, disclosed internal data showing its Claude models now write more than 80% of merged production code and that engineers are shipping roughly eight times as much code per quarter as before 2025.
  • The report warns that trends point toward recursive self‑improvement, a stage where AI could design and train its own successors and make meaningful human control harder.
  • Anthropic called for an option to slow or temporarily pause frontier model development only if multiple well‑resourced labs in several countries agree to stop under independently verifiable conditions.
  • Observers and rivals have questioned the timing and motives of the proposal after Anthropic filed confidential IPO paperwork and closed recent funding, and reporting has also tied company engineers to NSA work on the Mythos model, raising trust and dual‑use concerns.
  • The company says verification is hard because training runs can be concealed, which creates enforcement and incentive problems that policymakers must address even as U.S. officials introduce voluntary pre‑release review measures and labs race for compute and market positions.