Doom Debates!

Bestselling author Robert Wright and I discuss this week’s stock market jitters, Google’s new interactive world model, and why I sometimes spend $1,000 in a day on AI agents. Plus:
The Binance hack;
An AI audit of two star economists;
Anthropic’s financial prospects;
And whether AI could crack the math behind encryption.
Enjoy another edition of the Nonzero x Doom Debates collab!
Watch on YouTube: https://www.youtube.com/watch?v=wkKpPeMVVX8
Timestamps
00:00:00 — Teaser
00:00:50 — Nonzero Doom returns
00:03:07 — A foom in slow motion
00:07:47 — Is AI winter canceled forever?
00:10:33 — Will Google’s Genie 4 lure us into the Matrix?
00:16:54 — Liron’s bull case for Google stock
00:24:34 — Does Google have big-company disease?
00:28:28 — Could a slowdown actually help Anthropic and OpenAI?
00:30:12 — Was OpenAI’s “pause” just PR?
00:40:01 — Social science judgment day is coming
00:47:12 — Why even AI’s upsides could tear society apart
00:51:12 — Does OpenAI’s alignment approach improve on Asimov?
00:55:06 — Sam and Satya pivot on alignment
00:58:34 — Crypto and the AI hacking threat
01:03:42 — Could AI crack the math behind encryption?
01:08:52 — Liron’s $1,000-a-day AI coding habit
01:15:23 — Explaining the missing productivity boost
01:17:26 — Does Anthropic’s S-1 sink the AI bubble thesis?
Links
ROBERT WRIGHT & NONZERO
Nonzero’s post for this episode — “Is an AI Bubble Bursting?”
Subscribe to The NonZero Newsletter
The God Test — Bob’s 2026 book on AI
Join NonZero’s Discord server
Nonzero on X (@NonzeroPods)
THE WEEK IN AI
“Anthropic files S-1 with the SEC for proposed initial public offering” — the IPO filing Liron calls the final nail in the coffin of the Ed Zitron camp
Where’s Your Ed At — Ed Zitron’s newsletter, home of the AI-bubble bear case
“Introducing Genie 4” — Google DeepMind’s real-time interactive world model
“Gemini 4 Argon: Built to solve your hardest problems” — Google’s new frontier training run, the “lukewarm” release
“Gemini 4 Carbon: our next frontier model” — the follow-up Google announced two weeks later
“Google Gemini co-lead Noam Shazeer leaves for OpenAI” — CNBC on the talent Google has bled
“Errors in the Research of Emmanuel Saez and Gabriel Zucman” — Tyler Cowen’s Marginal Revolution post on the AI audit: 138 papers, 2,035 problems
Boaz Barak: “We organized the Model Spec around five core values” — the OpenAI alignment update Liron compares to Asimov’s three laws
“An Updated Model Spec for More Capable and Beneficial AI” — OpenAI’s announcement
“Sam Altman says AI means we can no longer just patch all the bugs” — TNW on the Stanford GSB talk Liron quotes on alignment and pacing
“Satya Nadella says he was wrong to think AI alignment would come with scale” — TNW on his 20VC interview: “I used to always think that alignment will just fall out. I no longer believe that.”
“OpenAI Takes Initial Steps To Address Its Alignment Problems” — Zvi Mowshowitz on OpenAI’s training “pause”
2026 Binance hack — Wikipedia’s summary of the theft Bob and Liron debate
“$2 Million+ Drained From Polymarket Traders in New Supply Chain Attack” — SecurityWeek on the hack Liron walks through
“PHP blames supply chain security biz Chainguard for putting huge chunks of the web at risk” — The Register on “the PHP thing last week”
ALSO MENTIONED
The Hanson-Yudkowsky AI-Foom Debate — LessWrong wiki on the 2008 debate where “AI Go Foom” originated
“The AI Revolution: Our Immortality or Extinction” — Tim Urban’s Wait But Why post, where the intelligence train blows right past the station
PauseAI US — Holly Elmore’s organization, plugged at the close of the episode
DOOM DEBATES
Doom Debates on YouTube
Join the Doom Debates Discord
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe

Cybersecurity expert and entrepreneur Zack Korman is an increasingly prominent nondoomer voice on Twitter. Naturally, with AI now breaking out of sandboxes and hacking corporate systems, people want to hear what cybersecurity experts have to say. So I’m thrilled that Zack was brave enough to face me on Doom Debates!
Zack was reading Nick Bostrom at Oxford and worrying about AI doom as far back as 2016, but his views have since shifted. In a recent post, he writes: “This focus on AI causing human extinction is going to get people killed.” Now he expects security failures to give humanity plenty of opportunities to learn, fight back, and stop risky superintelligence development if necessary.
We argue about whether sandboxing and AI monitoring are hopeless strategies, whether competing superintelligences would keep each other in check, whether to expect instrumental convergence, and more.
Watch on YouTube: https://youtu.be/QJTeZfx1ZLk
Timestamps
00:00:00 — Cold Open
00:01:15 — Introducing Zack Korman
00:03:41 — From Law to AI Security
00:09:17 — Why Zack Stopped Being a Doomer
00:12:03 — What’s Your P(Doom)?™
00:21:00 — Why Zack Supports AI Safety
00:25:54 — Believing in Superintelligence
00:30:06 — AI Without Tools
00:39:42 — Warning Shots and Time to React
00:46:08 — A Country of Geniuses in a Datacenter
00:55:55 — Can We Sandbox AI?
01:05:58 — Monitoring and Layers of Defense
01:14:49 — Riding the Doom Train™
01:22:23 — The Case for Accelerating
01:24:46 — AI CEOs and the Transfer of Power
01:34:19 — Who Presses the Stop Button?
01:41:38 — Humans, Bacteria, and the Limits of Power
01:48:30 — When Would Zack Pause AI?
01:56:13 — Will Different AIs Keep Each Other in Check?
02:06:11 — Can Progress Turn Against Us?
02:08:35 — The Ideological Turing Test
02:14:58 — Closing Thoughts
Links
Zack Korman on X
Zack Korman’s website
Zack Korman: “AI doomers are furious”
Zack Korman: “AI doomers are anti-security”
Zack Korman: “3 reasons sandboxing won’t secure your AI agent”
Zack Korman: “This focus on AI causing human extinction is going to get people killed.”
METR’s investigation of the OpenAI–Hugging Face incident
The PHP Foundation: “Today’s PHP Security Release, and How We’re Strengthening PHP Security”
Nick Bostrom: Superintelligence
Center for AI Safety: Statement on AI Risk
Eliezer Yudkowsky and Nate Soares: If Anyone Builds It, Everyone Dies
AI 2027
Zvi Mowshowitz: “The Three AI Pills”
PauseAI
Eliezer Yudkowsky: “Value is Fragile”
Aaron Scher at MIRI
Doom Debates: Robin Hanson vs. Liron Shapira — Is Near-Term Extinction From AGI Plausible?
Doom Debates: OpenAI’s BOMBSHELL Hack — Swarms of Agents, Zero-Day Exploits, & Misaligned AI
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe

Physicists Jeremie and Edouard Harris, co-founders of Gladstone AI, are preparing for an “AI Chernobyl” warning shot.
If that forces Trump and Xi to negotiate an AI slowdown, the Harris brothers want to ensure the US has the enforcement tools to make the deal stick. More specifically, they want verification technologies developed and vetted before the crisis hits, alongside enforcement capabilities to deter cheating.
Yet when I bring up Eliezer Yudkowsky’s writing on the possibility of airstrikes on rogue data centers as a last-resort treaty deterrent, they push back. They also don’t share my enthusiasm for PauseAI, expressing uncertainty about whether the movement has been net beneficial.
Jeremie and Edouard have earned a ton of credibility:
• They redirected their careers around the AI threat as far back as 2015, stepping away from their Y Combinator startup before ChatGPT even launched and brought AI into the mainstream.
• They’ve built relationships with senior government decision-makers without a prior national security background.
• They’ve demonstrated a talent for public communication in their excellent Joe Rogan appearances.
In this episode, they bring a level of multi-step strategic thinking that I rarely explore. But I can’t help wondering whether a catastrophe 50 times worse than the Hugging Face incident would let world leaders move much faster on a global AI pause treaty than they expect.
Let’s see if they can convince me that an emergency US–China agreement is our best path to a safer world.

Watch on YouTube: https://youtu.be/4jmFcE4WZ7M
Timestamps
00:00:00 — Cold Open
00:00:53 — Introducing Jeremie and Edouard Harris
00:02:16 — The GPT-3 Wake-Up Call
00:03:59 — AI Risk Outside the EA Community
00:08:49 — The Harris Brothers’ Backgrounds
00:16:37 — Yudkowsky and Many-Worlds Quantum Mechanics
00:22:00 — SharpestMinds and Leaving Academia
00:25:04 — Briefing the US Government
00:33:01 — How Gladstone AI Pays the Bills
00:35:00 — The Trump–Xi Phone Call
00:39:29 — Pacing Isn’t Pausing
00:42:45 — Defining AI Chernobyl
00:44:07 — Verifying China’s AI Slowdown
00:47:10 — Espionage and Lab Security
00:54:10 — Offensive Operations and Deterrence
00:57:47 — The US–China AI Lead
01:05:27 — Why a Treaty Won’t Happen
01:08:01 — Red Lines and Enforcement
01:16:25 — Warning Shots and Extinction
01:21:26 — What’s Your P(Doom)?™
01:24:39 — PauseAI and Movement Politics
01:29:16 — Wrap-Up
Links
• Their treaty analysis: “Arms control negotiators think there won’t be a U.S.-China AI treaty”
• The Endgame — the Harris brothers’ Substack
• Gladstone AI
• Their 2024 government-commissioned AI action plan
• Their 2025 report: America’s Superintelligence Project
• Eliezer Yudkowsky’s 2023 TIME essay advocating a global shutdown
• Joe Rogan Experience #2156 — Jeremie and Edouard Harris, May 2024
• Joe Rogan Experience #2311 — Jeremie and Edouard Harris, April 2025
• PauseAI US
• PauseAI Global
• Join the Doom Debates Discord
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe

Dr. Holly Elmore is the founder and executive director of PauseAI US and holds a Harvard PhD in evolutionary biology. She’s back, and this time she’s taking on none other than Eliezer Yudkowsky!
She argues his public behavior is alienating the people PauseAI US needs to reach, but I wonder if she’s asking too much of the man who laid the foundation for the AI safety movement.
We also clash over Yudkowsky’s coherent extrapolated volition. She thinks the theory doesn’t properly factor in humanity’s consent, but I argue that’s already embedded into the CEV framework.
Holly and I dive into the rationalists’ involvement in a controversial kink scene, John Wentworth’s defunding news, transhumanism, pacing the frontier, and “Holly’s Basilisk”—her case for sending AI lab employees to jail.
It’s another great episode with one of the most controversial—and thoughtful—thinkers in AI safety. Enjoy!
Watch on YouTube: https://www.youtube.com/watch?v=Jn8mS3kC-SU
Timestamps
00:00:00 — Cold Open
00:01:09 — Holly Elmore and PauseAI’s Political Breakthrough
00:07:36 — Pause AI vs. “Pacing the Frontier”
00:14:17 — Holly’s Case Against Eliezer Yudkowsky
00:16:20 — Coherent Extrapolated Volition and Pivotal Acts
00:31:19 — Brain Copies, Consent, and Self-Determination
00:36:56 — Transhumanism, Dyson Swarms, and Humanity’s Future
00:45:36 — The Missing Democratic Checkpoint
00:51:04 — Yudkowsky’s Role in Accelerating AI
01:02:48 — Genius, Arrogance, and Rationalist Culture
01:10:14 — The Aella Photo and Holly’s Breaking Point
01:20:21 — John Wentworth’s Grant Controversy
01:28:17 — PauseAI US’s Rules and Red Lines
01:32:05 — Poe’s Law and the Cost of Sarcasm
01:36:18 — Holly’s Case for Jailing AI Developers
Links
Donate to PauseAI US
Holly on X
Holly’s Substack
Pacing the Frontier letter
Jacob Coxon’s resignation post
PauseAI leadership disendorsement letter, reproduced on LessWrong
Holly on movement infighting — No, the Judean People’s Front should not just get along with the People’s Front of Judea
Coherent Extrapolated Volition — Eliezer Yudkowsky (2004 PDF)
Pivotal act — Eliezer Yudkowsky (Arbital mirror)
Genes did misalignment first — Holly Elmore
Holly’s reaction to the Yudkowsky/Aella photo
On Kink & Status in the Rationalist Community — John Wentworth (October 8 response)
Holly’s basilisk
Are We a Circular Firing Squad? — previous Holly Elmore episode
Let’s Call Out the AI Doom Enablers — previous Holly Elmore episode
Join the Doom Debates Discord
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe

Sarah Hastings-Woodhouse is an AI safety writer and researcher who went down the AI doom rabbit hole in 2023 and never came back up. After a year as digital communications officer at the UK AI Security Institute (AISI), she’s gone independent to speak her mind unfiltered.
Read Sarah’s Substack: https://longerramblings.substack.com
Join PauseAI UK’s March Against AI Extinction in London on November 21: https://luma.com/pauseai-nov26
Watch on YouTube: https://www.youtube.com/watch?v=TDbHCKwuMrk
Timestamps
00:00:00 — Cold Open
00:00:41 — Introducing Sarah Hastings-Woodhouse
00:02:06 — How a Humanities Grad Got AGI-Pilled
00:08:40 — Whose Timelines Do You Trust?
00:11:32 — Joining the UK AI Security Institute
00:15:47 — What AISI Actually Does
00:19:23 — Verifying Anthropic’s Mythos Claims
00:21:59 — Leaving Government to Speak Freely
00:23:22 — What’s Your P(Doom)?™
00:25:37 — Yudkowsky, EA, and Keeping Your Identity Small
00:32:08 — Is Anthropic Doing Enough to Call for a Slowdown?
00:35:56 — Can You Do AI Safety from Second Place?
00:39:20 — Hugging Face and the Vibe Shift
00:44:02 — Is Everything Weird Yet?
00:48:27 — Prevent Recursive Self-Improvement, Don’t Ban It
00:52:28 — Build the Speed Dial, Not the Pause Button
00:56:54 — Was Calling for a Pause in 2023 a Mistake?
01:01:22 — Turning Up the Fear Dial
01:04:23 — Donation Drive
01:05:11 — PauseAI UK’s March Against AI Extinction
Links
SARAH HASTINGS-WOODHOUSE
Sarah’s Substack
Sarah on X (@littIeramblings)
Consistently Candid — Sarah’s podcast
“What is recursive self-improvement and what would it mean to ban it?” — Sarah’s RSI post (Oct 1, 2026)
“Is everything weird yet?” (Aug 30, 2026)
“What the hell just happened?” — on AI safety’s viral moment (Sep 18, 2026)
“We have been warned” — the Hugging Face hack as a warning shot (Jul 27, 2026)
“The pause-shaped hole in ‘Policy on the AI Exponential’” — is Anthropic doing enough to call for a slowdown? (Jun 19, 2026)
Sarah’s 2023 thread on stumbling into the world of AI doom (“I’m angry because I love the world”)
“The other night I successfully AGI-pilled a man on a first date”
“Welcome, and also I’m sorry” — Sarah’s note to AI safety newcomers (Sep 11, 2026)
“‘We’re all f*cked and it’s totally hopeless’ is almost as bad of a meme to be spreading” (Feb 12, 2025)
REFERENCED IN THE EPISODE
“Our evaluation of Claude Mythos Preview’s cyber capabilities” — AISI (Apr 13, 2026)
AISI Frontier AI Trends Report (Dec 2025)
“The levers of political persuasion with conversational AI” — AISI Human Influence team
US Center for AI Standards and Innovation (CAISI) — NIST
“Geoffrey Hinton tells us why he’s now scared of the tech he helped build” — MIT Technology Review (May 2023)
“We’re All Gonna Die with Eliezer Yudkowsky” — Bankless (2023)
“GPT-2 As Step Toward General Intelligence” — Scott Alexander, Slate Star Codex (Feb 2019)
“AGI Ruin: A List of Lethalities” — Eliezer Yudkowsky
“Keep Your Identity Small” — Paul Graham
Pacing the Frontier — the open letter signed by 1,300+ frontier lab employees
“You can’t do safety from second place” — Sarah Heck, Anthropic’s head of public policy (Sep 29, 2026)
Ezra Klein’s op-ed calling for a ban on recursive self-improvement — The New York Times (Sep 20, 2026)
“Banning self-recursive improvement in AI models?” — Tyler Cowen, Marginal Revolution (Sep 22, 2026)
PauseAI UK
March Against AI Extinction — PauseAI UK, London, November 21, 2026
PAST DOOM DEBATES EPISODES MENTIONED
Top Economist Sees AI Doom Coming — Dr. Peter Berezin, BCA Research
Will AI Kill Everyone by 2050? Debate with Dr. Casey Hart (Ontology Explained)
DOOM DEBATES
Donate to Doom Debates
Join the Doom Debates Discord
Transcript, links, and show notes: https://doomdebates.com/p/ai-has-become-profoundly-weird
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe