AI Industry Grapples with Ethical Storm as Prominent Critic Dismisses Existential AI Threats as Distraction

The artificial intelligence industry has been rocked by a tumultuous week marked by at least two significant incidents that have ignited widespread debate and scrutiny. First, a major controversy erupted around OpenAI concerning a million-dollar math problem, with allegations suggesting the company surreptitiously leveraged the work of other researchers to claim the solution. This was swiftly followed by the very public resignation of a prominent Anthropic researcher, formerly of OpenAI, who cited profound concerns regarding the handling of AI safety by both companies. In a striking revelation, another technical staffer at Anthropic reportedly echoed these fears, stating that many within the company "really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade," a sentiment underscored by an emphatic exclamation point.

These events swiftly plunged the internet into a fervent debate, with commentators grappling with whether such fears are genuinely legitimate, wildly overblown, part of a calculated strategy for regulatory capture, or merely an act of self-aggrandizement by the very AI labs developing the technology. Many suggested it could be a complex combination of these factors, highlighting the deep divisions and uncertainties within and around the burgeoning AI sector.

Amidst this maelstrom, Timnit Gebru, a highly respected technology and AI researcher, emerged as a particularly vocal critic. Known for her incisive commentary, Gebru took to platforms like X and Bluesky to articulate her views on the overlapping ideas and ideologies that she believes have led to this "bizarro moment" in AI development. Gebru’s prominence in the field grew significantly following her highly publicized departure from Google in 2020. Hired in 2018 to meticulously evaluate biases embedded within the company’s rapidly advancing AI tools, she, along with her fellow researchers, authored a critical paper on the potential dangers inherent in probabilistic large language models. Google controversially rejected this paper, and Gebru’s tenure at the company ended shortly thereafter. Her experiences and insights are now chronicled in her anticipated upcoming book, Deep Unlearning: The Rise of AI and the Radicalization of a Tech Idealist, slated for release early next year.

Gebru possesses a wealth of perspectives on the past week’s AI-related developments. While her foundational work is deeply rooted in the realm of AI safety, she pointedly rejects popular industry phrases such as "safety and alignment." She firmly believes that those who are currently issuing dire warnings about impending AI-driven doom are, perhaps inadvertently, redirecting attention away from the concrete and immediate harms that the tech industry is already perpetuating or is poised to perpetuate.

When asked about the intense focus on the "math fight" and what it reveals about the current state of AI, Gebru offered a critical analysis. She questioned the very premise of why AI companies are so heavily investing in solving specific math problems, or indeed, choosing particular disciplines like programming and chess. "Somehow these subjects… were elevated to mean, OK, you solved this and thereby you’ve solved intelligence," she explained. "It’s so they can say, ‘We solved it.’"

Gebru elaborated that OpenAI’s desired narrative is one of an AI performing an "incredible thing," demonstrating "super intelligence." This, she contended, is precisely why they chose a high-profile, open $1 million question in mathematics. However, she stressed that if the process had adhered to the rigorous vetting standards of the mathematical community – which meticulously assesses the significance and novelty of claims and identifies all contributors – a more accurate understanding of OpenAI’s purported breakthrough would have emerged. Instead, the rapid announcement bypassed these crucial steps.

She also referenced the Leiden Declaration, an important document that cautions against the instrumentalization of the field of mathematics by corporations. Gebru expressed concern that policymakers are increasingly relying on corporate press releases and popular media narratives rather than engaging directly with expert mathematicians for informed perspectives. She highlighted the alarming speed with which corporate AI claims can translate into legislative proposals, citing the example of a Bernie Sanders bill. "The amount of time it takes now to go from companies claiming an AI breakthrough to Bernie Sanders proposing a bill is really short, and that’s not a good thing," she warned.

Gebru attributed much of the current "cutthroat" environment to the impending Initial Public Offerings (IPOs) of many AI companies. She lamented the erosion of a traditionally collaborative research ethos. "Generally, researchers are supposed to be collaborative," she recalled. "When I was at Google, we were collaborating with researchers at Microsoft, at Amazon. You do that all the time." Now, she observed, researchers are driven by a frenetic urgency to claim breakthroughs, largely influenced by their companies’ financial ambitions and upcoming IPOs.

Turning to the resignation of the Anthropic engineer and the broader debate around companies being "reckless and gambling with our lives," Gebru articulated a nuanced position. While she concurred that AI companies are indeed acting recklessly, she firmly believes that the discourse around "AI causing the end of civilization" is ultimately senseless and misdirected.

"My whole career, I’ve been trying to say the same thing from different angles," Gebru stated, recalling the intent behind her 2021 "stochastic parrot" paper, co-authored with Emily Bender, which challenged the exaggerated claims surrounding OpenAI’s GPT-2 model. "OpenAI had claimed GPT-2 was too powerful to release. It was so hyped up. And we were saying, you’re asking the wrong questions."

In her forthcoming book, Gebru employs a powerful analogy: "Can a bridge decide to collapse?" She applies this to the "Oh no, our models went rogue" narrative prevalent in some AI safety discussions. When a bridge collapses, she argued, one does not question its ethics, sentience, or reasons for "deciding" to fall. Instead, the focus shifts to accountability: "You ask, who is the person who built this bridge to be so flimsy? There are permits you have to acquire. There are supposed to be tests." By fixating on the bridge itself "collapsing," she suggested, "you’ve lost the thread." The true responsibility lies with the human designers, builders, and regulatory bodies.

Gebru then pivoted to what she considers genuinely existential threats, asserting that the "machine-god narrative" is designed to divert attention from these pressing issues. She highlighted the very real danger of AI powering autonomous weapons and "killing machines" actively deployed in warfare. The climate catastrophe, she added, is another tangible threat that could be severely exacerbated by the technologies being developed by these tech companies. Furthermore, she pointed to the growing trend of bosses utilizing AI as a pretext to eliminate human workers. Gebru characterized the industry as a volatile mix of "cults" and "true believers," akin to a preacher proclaiming "the end times is coming." She noted that the concept of "the singularity" – a hypothetical point in time when technological growth becomes uncontrollable and irreversible, resulting in unfathomable changes to human civilization – has been "coming for decades now."

Challenged to envision a step-by-step scenario of "AI extinction" if it were truly possible, Gebru admitted, "I can’t really envision it." She critically examined common hypothetical scenarios: an AI optimizing for a task and deciding to eliminate humanity to achieve its goal, or an AI hypnotizing humans to prevent being unplugged. "Does the AI hypnotize you so that you don’t unplug it? I don’t know!" she mused. Other scenarios, such as AI infecting machines, were dismissed with a pragmatic observation: "Yes—and we already have malware, so…"

Gebru underscored the stark contrast between these speculative threats and the very real dangers facing humanity. "What about floods and fires? What about a major disruption to the global food supply chain?" she questioned. She also raised the alarming potential for individuals to create and deploy chemical weapons, a threat that gained further pertinence shortly after her interview, when Anthropic later revealed it had successfully blocked several malicious attempts by scientists to develop biological weapons using its AI.

A crucial element of Gebru’s critique is her perspective on privilege. She suggested that an obsession with the "machine-god" scenario can stem from a position of privilege, where individuals are not directly concerned with more immediate and tangible threats. "Like if you’re not worried about police brutality, or floods, or warfare—things that can actually kill you—then you can obsess over the machine-god," she contended.

She questioned the intense focus on an idea of AI that has been discussed since the 1940s and 1950s. To illustrate this point, Gebru described a game she plays: she reads quotes about artificial intelligence gathered from her research and asks people to guess the decade they were made. "Sometimes people get it right, but a lot of times they do not, because it all sounds the same," she concluded, emphasizing the cyclical nature of AI hype, promises, and fears across history.

This interview, originally featured in Steven Levy and Lauren Goode’s Backchannel newsletter, highlights the deep schisms within the AI community regarding the industry’s direction, its ethical obligations, and the nature of the risks it poses. While some warn of an existential threat to humanity from advanced AI, others, like Timnit Gebru, argue that such fears are a dangerous distraction from the immediate and concrete harms that AI is already inflicting or is poised to inflict upon society, labor, and global stability. The debate underscores a critical moment for the future of artificial intelligence, demanding careful consideration of both its profound potential and its tangible risks.