Anthropic has filed its IPO prospectus, dedicating 94 pages to risk factors.

The filing highlights existential risks tied to recursive self-improvement, where AI systems automate R&D beyond human evaluation and control.

The dangers to humanity

Several pages of the documents are devoted to risk factors. They describe scenarios in which advanced AI systems could seriously endanger the survival of humanity. The company links these risks to a process in which humans hand over the research and development of new AI models to the systems themselves, gradually removing humans from the process.

According to the filing, these risks will increase as systems take over research and technology, to the point where progress can outpace humans’ ability to evaluate and protect new models.…