The heads of America’s leading artificial intelligence (AI) companies united this weekend around calls to pace the development of the rapidly evolving technology—igniting vigorous public debate.
It followed fresh warnings from researchers last week that self-improving AI, in which AI can autonomously make itself better and more efficient and build new models, could pose an existential threat to humans.
“The question is: where does this ultimately end? If the models are more capable than the humans who design and train them, then by definition we have lost our ability to impose our will on them. That uncertainty is what feels dangerous,” CFR’s Connor Martin told the Daily Brief. (Check CFR.org soon for Martin’s full interview.)
Yet two camps exist when it comes to regulation.
Slow or stop development
Fears about the development of AI capabilities are not new. Stories of AI agents involved in cyberattacks, AI-powered scams, and other attempts to use the technology for nefarious means have underscored the risks.
But a stop to AI development is not considered practical in the industry, Martin said. “If the technology is possible to invent, sooner or later some human being is going to invent it.”
By “pacing” AI development, the aim is to “add just enough drag to the speed of model improvement” that guardrails can be imposed “before the technology escapes our grasp,” he continued.
Yet questions about when and how to slow down, and the accompanying consequences, remain.
“These are private companies barreling toward landmark IPOs, facing very real competitive pressures,” CFR’s Jessica Brandt told the Daily Brief. “It’s a prisoner’s dilemma—any firm that slows down unilaterally risks losing ground.”
Little-to-no regulation
Not everyone believes that the worst scenarios of runaway AI are bound to happen, or even likely. President Donald Trump pushed back today against calls for greater regulation and denounced a “conspiracy” against AI.
AI advancement is also part of the United States’ race for geoeconomic competitiveness with China. Those, like Trump, who favor less regulation fear a slowdown could both hamper the anticipated benefits of advanced AI—such as breakthroughs in medicine—and leave China with a dominant edge.
“A coordinated slowdown by American labs could erode the United States’s narrow lead over China in cutting edge capabilities,” Brandt said. “If China pulls ahead, that could upend the global balance of economic and military power.”
Those concerns are amplified by the fact China largely hasn’t shared any such safety concerns. As Jordan Schneider, founder of ChinaTalk, told CFR’s Sebastian Mallaby on an episode of The Spillover in July, China sees AI as a “normal technology.” He characterized Beijing’s approach to regulation from 2018–2025 as “benign neglect.”
But that may be changing. “China seems to be facing its own reckoning with the risks of AI,” Brandt said, pointing out that China’s top spy chief this weekend warned AI could threaten the Chinese Communist Party’s grip on power.
The Daily Brief will keep bringing you the latest in the AI race as it comes.
