top of page

Arguments and Counterarguments: The AI Extinction Threat

59 minutes ago
2 min read

By Iris E. '29


If you’ve been reading the news lately, you may have noticed high-profile articles focused on artificial intelligence's trajectory. With titles from media organizations like the BBC warning of a “more than 10% chance AI ‘could kill all humans’,” and AI workers who are “‘genuinely terrified,’” it’s hard not to wonder about the future of technology. Especially in the Bay Area, AI predictions often sit at the forefront of our lives, magnified further by widespread pessimism and tech bros who are sure they’re the next John McCarthy. But are these claims really substantial? How seriously should we be taking these potential catastrophes?


The arguments supporting claims of extinction primarily stem from a fear of uncontrollable development. Evan Hubinger, who works at Anthropic as an alignment and safety researcher, wrote a statement that garnered significant media attention. Hubinger warned that there is a significant chance AI will wipe out the entire human population in less than ten years. The primary concern Hubinger outlines is that AI models will rapidly rfeach a state in which they can improve themselves completely autonomously, eliminating the need for human assistance. If that happens, Hubinger argues, superintelligence will develop in a truly uncontrollable form and lead to extinction-level misalignments.


Alignment, in the context of AI, is the practice of ensuring that an AI model’s actions align with human values. A recent example of misalignment occurred when OpenAI's agents went rogue and hacked into Hugging Face’s platform. This incident was followed in September by the resignation of an Anthropic senior researcher, who claims that the employees of Anthropic genuinely believe AI is already acting semi-independently. At the UN Security Council, OpenAI and Anthropic called for global oversight and restrictions on AI development.


Claude, a popular AI chatbot, IPOed on Thursday. (Photo Credits: Wix Images)
Claude, a popular AI chatbot, IPOed on Thursday. (Photo Credits: Wix Images)

However, some experts warn that this call for restrictions on the world’s largest AI companies might actually be a strategic move to establish a monopoly. The idea is that by imposing multiple legal hurdles, leading AI frontier labs would force out competitors entirely. Smaller companies like Mistral AI and Cohere have argued that these constraints would place nearly all AI advancements in the hands of just a few trillion-dollar American companies.


An interesting argument can be made that AI companies are inherently trying to do exactly what the U.S. government attempted with its monopoly on the development of nuclear power, using legislation like the Atomic Energy Act of 1946 to limit independent competition. While leaders in AI like Altman have denied that regulatory capture is the goal, it remains an issue that could undermine the credibility of their claims.

Recent Posts

See All
Seniors of CPS: Phi E.

Interview conducted by Meera K. Meera: Hi Phi! How are you today? Phi: I'm good! Meera: I'm going to start with some rapid-fire questions, so you can answer these pretty quickly. Favorite Halloween

 
 
 

Comments


bottom of page