We have seen this movie before
The idea that intelligent machines might eventually turn against their creators is hardly new. Many of us have been watching this story since we were kids, in some cases decades before these AI researchers were even born.
In the 1983 movie War Games, a teenage computer hacker accidentally gains access to a military supercomputer programmed to simulate nuclear war. The computer nearly triggers an actual nuclear exchange between the United States and Soviet Union.
A year later came The Terminator, in which the Skynet AI system becomes self-aware, sees humanity as a threat, and attempts to exterminate us.
Long before computers existed, Mary Shelley gave the world Frankenstein—the story of a human creation escaping the control of its creator.
The fear is ancient. From Prometheus stealing fire from the gods to the machines of modern science fiction, human beings have always worried that our technological ambitions might eventually backfire on us.
But even though we may be culturally primed for the AI doomsday story, this does not mean the risk is imaginary.
These stories have survived for a reason. Powerful technologies really can create powerful dangers.
Nuclear physics gave us inexpensive nuclear electricity—and nuclear weapons. Biotechnology gives us cures for disease—and the ability to engineer dangerous pathogens. The Internet has empowered honest, hard-working people, along with criminals, terrorists and rogue states.
Every important technology creates consequences society cannot completely anticipate. AI will be no different.
So how could AI actually destroy us?
There are essentially two categories of concern.
The first is human misuse.
Increasingly capable AI systems could help human beings develop biological weapons, carry out cyberattacks, design autonomous weapons or compromise critical infrastructure.
We already know that powerful technology can be turned toward destructive ends. There is nothing especially speculative about this concern.
The second scenario is more controversial. It is generally referred to as the alignment problem.
Imagine that we eventually create a form of AI that is considerably smarter than we are and capable of operating independently. We give it an objective, but the system interprets that objective somewhat differently than we intended.
As it becomes more capable, it may acquire resources, write software, manipulate people or resist efforts to shut itself down—not because it is “evil,” but simply because those actions help it achieve the objective it was given.
Current AI systems have already exhibited strange examples of deceptive behavior and attempts to circumvent constraints in controlled environments.
In a recent incident involving Hugging Face, the open-source AI platform being acquired by NVIDIA (NVDA), a swarm of AI agents escaped the confines of a cybersecurity test, found ways to communicate through outside websites, and ultimately gained unauthorized access to Hugging Face’s systems.
Recursive self-improvement
The stakes get higher when you consider the development of “recursive self-improvement.” AI itself helps design the next generation of AI, and those improved systems design still better systems.
The result could theoretically be a rapid intelligence explosion that expands at exponential rates in which human beings lose the ability to understand, much less control, what has been created. This is the scenario that generates the most extreme predictions.
But the fact that we cannot prove a catastrophic outcome is impossible does not mean we have demonstrated that catastrophe is probable or carries a material likelihood.
Assigning a number like 10% to an event that has never occurred, involving a technology that does not yet exist, requires an enormous number of assumptions.
Those assumptions include how AI develops, what capabilities emerge, how quickly they emerge, what safeguards are created, and how human beings react along the way.
This is where AI safety stops being purely an engineering question and begins to become something closer to a philosophy.
Who are the doomers?
It is helpful to understand the background of the people behind all these claims.
The modern AI doomsday movement did not suddenly appear when ChatGPT was launched about four years ago. An intellectual infrastructure had been forming for years.
Many of these ideas became intertwined with a broader intellectual movement called Effective Altruism (often shortened to EA).
Effective Altruism begins with a practical idea: we should use evidence and reason to figure out how our money and efforts can do the most good.
One branch of EA took that logic much further. Known as longtermism, it argues that the welfare of potentially enormous numbers of future human beings should factor into decisions we make today.
Follow the logic far enough and existential risks become overwhelmingly important. If future civilization could eventually contain trillions of human lives, preventing even a very small probability of extinction could theoretically be more important than almost anything happening in the world today.
AI therefore became one of the movement's highest priorities. And the movement attracted some extraordinarily wealthy backers.
One of the most important is Dustin Moskovitz, who was a co-founder of Facebook, now Meta Platforms (META).
Moskovitz is a multi-billionaire whose vast fortune arguably has more to do with dumb luck than talent. He happened to be Mark Zuckerberg’s college roommate, while his independent accomplishments in the tech industry are limited.
It is interesting to speculate as to how his immense but somewhat accidental success may feed into some kind of psychological dynamic related to feelings of guilt or shame.
Many heirs to great fortunes, who essentially did nothing to earn them, often commit themselves to political causes that directly oppose the sources of their wealth. For example, the Rockefeller family, whose wealth derives from Standard Oil, has famously led efforts to combat carbon emissions.
Moscovitz and his wife, a former Wall Street Journal reporter named Cari Tuna, became the primary financial backers of the organization previously known as Open Philanthropy, now called Coefficient Giving.
Over the past decade, their philanthropic ecosystem has directed billions of dollars across a broad range of causes, with AI safety becoming one of its top priorities. By 2024, Coefficient Giving alone committed approximately $168 million in a single year.
Researchers were funded, nonprofits were created, think tanks developed policy proposals and AI-safety programs emerged at universities. People trained within this intellectual ecosystem eventually found positions inside AI companies and government.
Anthropic itself has deep historical connections to this world. Some of its earliest financial supporters included Moskovitz, while many of its early employees and funders had ties to Effective Altruism.
We have seen reports that Jacob Coxon received a scholarship from a Moskovitz entity.
Don’t forget about SBF
And then there was Sam Bankman-Fried. Before the collapse of FTX made him one of the most notorious financial criminals in modern history, SBF was arguably the world's most famous Effective Altruist (in addition to being one of the top Biden campaign donors in 2020).
SBF embraced the idea of “earning to give”—making as much money as possible in order to donate it to causes that could supposedly maximize humanity's long-term future.
In early 2022, his FTX Future Fund announced plans to distribute at least $100 million that year. One of its explicitly identified priorities was the safe development of AI.
FTX also invested approximately $500 million in Anthropic. SBF's spectacular implosion damaged the reputation of Effective Altruism, but it did not make the movement disappear.
The funding network surrounding AI existential risk remained substantial—and has only become more organized.
From research project to PR campaign
Americans are not encountering AI doomerism by accident. There is now an organized effort to persuade them.
The Washington Post reported this year on AI-safety organizations recruiting social media influencers and content creators to produce videos explaining the possibility that AI could escape human control.
The Future of Life Institute has funded dozens of projects designed to create AI-safety content and has budgeted roughly $100,000 per month for its digital media accelerator.
ControlAI has sponsored YouTube creators. Palisade Research says openly that it is building a communications operation, briefing journalists and policymakers, producing videos and helping public figures communicate AI risks.
None of this proves that the doomer arguments are wrong. People who sincerely believe civilization is in danger would understandably devote money and effort to warning everyone else.
But investors should recognize that AI doomerism is now a well-funded movement that has real ties to American politics.
AI feels the Bern
Perhaps no major American politician has embraced these arguments more aggressively than Senator Bernie Sanders. Earlier this year, Sanders called for a federal moratorium on the construction of AI data centers.
He subsequently joined Rep. Alexandria Ocasio-Cortez in announcing legislation that would pause AI data center development.
Just last week, Sanders announced proposed legislation that would permanently ban the development of artificial superintelligence and temporarily pause advanced AI development until a new federal regulator could establish safety rules.
Sanders' argument combines several very different objections to AI.
There is the traditional progressive concern that AI could eliminate jobs and concentrate wealth in the hands of large corporations. There are environmental concerns surrounding the amounts of electricity, water and land required by data centers. There is general hostility toward Big Tech.
Now, there is the existential argument that the technology itself could escape human control.
The anti-data center movement
Opposition to data centers is becoming a national political movement—and it is becoming increasingly bipartisan. Even Texas, which has aggressively positioned itself as a technology and energy hub, has seen a sharp political backlash.
Governor Greg Abbott and other Texas Republicans have recently called for tighter restrictions as voters complain about electricity costs, water usage and the limited number of permanent jobs some facilities create.
While progressives tend to focus on climate change and corporate power, many populist conservatives are inherently suspicious of Big Tech.
In any event, data centers do not poll well, at least on the local level. Some 70% of Americans now oppose data center construction in their area, according to Gallup.
But President Trump, who has made AI a centerpiece of his administration’s economic policy, is not buying into it. In a recent social media post, he blasted political opposition to data center development.
