Doom Debates!

Adam Khoja is a top AI forecaster who led the 2023 Center for AI Safety statement that shattered the Overton window on AI extinction risk. We cover his background, Mutual Assured AI Malfunction (MAIM), his new paper on AI betrayal, and whether Yudkowsky’s theoretical alignment research was a dead end.
Then Adam makes the case that an international AI slowdown is within reach today. All it takes is US and Chinese auditors inside each other’s AI labs. It worked for nuclear weapons, so why couldn’t it work for data centers?
Adam puts his P(Doom) at 40%, right next to my 50%. The real disagreement is how we get out of this: theory or empirics, MIRI or the labs. Enjoy the ride.
Watch on YouTube: https://www.youtube.com/watch?v=QqESBXuo6EI
Timestamps
00:00:00 — Cold Open
00:00:36 — Introducing Adam Khoja
00:02:45 — Leading the Statement on AI Risk as a Sophomore
00:10:17 — The Statement Leaked on Manifold
00:15:11 — Mutual Assured AI Malfunction (MAIM)
00:24:58 — Is Frontier AI Harder to Hide Than a Nuke?
00:32:09 — The AI Deterrence Escalation Ladder
00:36:10 — What’s Your P(Doom)?™
00:38:01 — Where Adam Departs from Yudkowsky
00:42:30 — Liron Explains Intellidynamics
00:47:26 — Neats vs. Scruffies in Deep Learning
00:55:27 — AI Deterrence by Betrayal
01:01:52 — Subversion vs. Overt Co-option
01:05:34 — Could the Government Seize the Labs’ AI?
01:07:29 — The Offense-Defense Balance of AI Security
01:12:46 — An International AI Slowdown Is Ready
01:15:51 — Does Adam Support PauseAI?
01:16:37 — “We’re All Already Spying on Each Other”
01:19:28 — Airstrikes on Rogue Data Centers
01:24:23 — Safety Research During a Slowdown
01:27:54 — Governance Over Technical Research
01:31:14 — Join the Center for AI Safety
Links
Adam Khoja (personal site) — https://adamkhoja.com/
Adam Khoja's Substack — https://adamkhoja.substack.com/
Adam's July 2023 Manifold market — "Will OpenAI's Superalignment project produce a significant breakthrough in alignment research before 2027?" — https://manifold.markets/AdamK/will-openais-superalignment-project
Center for AI Safety — careers / job board — https://safe.ai/careers
Statement on AI Risk (Center for AI Safety, May 2023) — the one-sentence statement Adam project-led, with full signatory list — https://safe.ai/work/statement-on-ai-extinction-risk
"Superintelligence Strategy" — Dan Hendrycks, Eric Schmidt & Alexandr Wang (Mutual Assured AI Malfunction / MAIM) — https://www.nationalsecurity.ai/
"AI Deterrence by Betrayal" — Adam Khoja, Aiden Kim et al. (CAIS, 2026) — https://www.aibetrayal.com/
"An International AI Slowdown Is Ready Whenever Politicians Are" — Adam Khoja, AI Frontiers — https://newsletter.ai-frontiers.org/p/an-international-ai-slowdown-is-ready
Pause Giant AI Experiments: An Open Letter (Future of Life Institute, March 2023) — the "Pause letter" that preceded the CAIS Statement — https://futureoflife.org/open-letter/pause-giant-ai-experiments/
Introducing Superalignment (OpenAI, July 2023) — Ilya Sutskever & Jan Leike's four-year goal — https://openai.com/index/introducing-superalignment/
Pacing the Frontier — the 2026 letter signed by 1,100+ frontier-lab employees — https://www.pacingthefrontier.com/
Why Iran targeted Amazon data centers (The Conversation) — the precedent Adam cites for strikes on compute — https://theconversation.com/why-iran-targeted-amazon-data-centers-and-what-that-does-and-doesnt-change-about-warfare-278642
Anthropic says Trump admin has lifted export controls on Claude Fable 5 and Mythos 5 (CNBC) — https://www.cnbc.com/2026/06/30/anthropic-says-trump-admin-has-lifted-export-controls-on-claude-fable-5-and-mythos-5.html
Mark Zuckerberg — "Personal Superintelligence" — https://www.meta.com/superintelligence/
Resolution — the theory-plus-empirics alignment org Adam is excited about — https://resolution.org/
Rationality: From AI to Zombies — Eliezer Yudkowsky's Sequences — https://www.readthesequences.com/
Robin Hanson — Futarchy: Vote Values, But Bet Beliefs (prediction markets as decision processes) — https://mason.gmu.edu/~rhanson/futarchy.html
The OpenAI–Hugging Face Incident — original Black Hat USA 2026 talk — https://www.youtube.com/watch?v=87DyyMV0kCY
OpenAI's Model Just ATTACKED Them — the Hugging Face hack breakdown — https://www.youtube.com/watch?v=RczYubQzXbI
Robin Hanson vs. Liron Shapira: Is Near-Term Extinction From AGI Plausible? — https://www.youtube.com/watch?v=dTQb6N3_zu8
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe

Sam Altman keeps insisting that AI progress is going better than the doomers predicted and that even superintelligence may not change the world as radically as people think. I react to his latest interview and explain why I think that calm, reassuring framing badly downplays the danger we're actually in.
I go through the interview line by line: Sam's "frame control", his framing of AI as normal technology, his "pro-human" branding, the liberty-vs-safety pivot, and his victory lap on AI safety — all while his own AI just went rogue.
Don't let them pull the Overton window backward.
Watch on YouTube: https://www.youtube.com/watch?v=yb9lNGHVycs
Timestamps
0:00 Teaser
0:47 Why I’m reacting to Sam’s interview
3:17 Sam Altman’s “frame control”
5:51 Framing AI as normal technology
9:50 Let’s watch Sam do it
10:34 “The world… won’t be that different” with superintelligence
12:25 This is gaslighting
12:30 Sam acknowledges loss of control
15:16 His other big risk: centralized power
19:44 Sam’s “pro-human” framing
25:33 Conflating AI critics with anti-human views
30:13 “Liberty vs. safety”
33:50 Sam vs. the “doomers”
36:53 Is alignment really an “unsolvable problem”?
38:00 Sam says the doomers predicted wrong
46:00 “The crazy bad predictions… have not happened”
47:06 Sam’s lean-startup theory of AI safety
50:26 Taking a victory lap on AI safety
55:43 “Disconnecting yourself from reality”
59:11 What would actually make OpenAI slow down?
1:03:19 Sam compares AI safety to aviation safety
1:07:30 Charging into the fog
1:12:50 The “missing mood” around AI extinction
1:14:51 Richard Ngo on Sam Altman’s “earnestness field”
1:19:01 Don’t let them pull the Overton window backward
Links
Sam Altman on David Senra — “Sam Altman on Building OpenAI & Betting on the Impossible” (the interview I react to) — https://www.youtube.com/watch?v=kG8AoExkX40
Richard Ngo — What Just Happened? Pragmatism and Pessimization (the “earnestness field” post) — https://www.lesswrong.com/posts/yaz8nx4ogZmiqHzt7/what-just-happened-pragmatism-and-pessimization
Richard Ngo — What Just Happened? A Retrospective of AI Alignment (start of the series) — https://www.lesswrong.com/posts/9RL9MuGZjzm4q3gKG/what-just-happened-a-retrospective-of-ai-alignment
Eliezer Yudkowsky & Nate Soares — If Anyone Builds It, Everyone Dies — https://www.amazon.com/dp/0316595640
Eliezer Yudkowsky — Coherent Extrapolated Volition — https://www.lesswrong.com/w/coherent-extrapolated-volition
Doom Debates: Dario Amodei BUNGLES Another Essay — MIRI’s Harlan Stewart Reacts — https://www.youtube.com/watch?v=aCYVVzza7A0
Doom Debates: OpenAI’s Bombshell Hack — Swarms of Agents — https://www.youtube.com/watch?v=RczYubQzXbI
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe

Come with me into the world of PlzDontKillUs, a bold new AI x-risk communication accelerator program that just finished its first cohort.
PlzDontKillUs took the influencer-house formula and moved it to Berkeley, giving creators personal mentorship from AI safety experts like Eliezer Yudkowsky and Nate Soares. With 57 people making daily videos for a month, the program racked up over 100 million views.
Find out how Aella, an independent sex researcher, and Ronny Fernandez, a generalist at Lightcone Infrastructure, teamed up to run the largest ever creator bootcamp focused on AI x-risk reduction.
Then meet three of the program’s top creators: Avalon Warren, Avisha NessAiver, and Josh Thor.
We do a post-mortem on PDKU and ask: Was it effective? What are creators taking from it, and do they have complaints? And does doomerism really boost view counts? PlzDoEnjoyOur special episode on PlzDontKillUs!

Watch on YouTube: https://www.youtube.com/watch?v=aPsjMCG7Ic8

Timestamps
00:00:00 — Cold Open
00:01:19 — Introducing PlzDontKillUs & Aella
00:09:33 — Aella Being Early to AI Doom
00:14:45 — The Origin of PlzDontKillUs
00:19:08 — How It Went
00:25:40 — Complaints with Prize $$
00:29:33 — Aella, What's Your P(Doom)?™
00:31:14 — Transhumanism & AI Girlfriends
00:34:38 — Aella Rides the Doom Train™
00:38:08 — Introducing Ronny Fernandez
00:41:21 — Running Lighthaven
00:47:18 — Life Inside PDKU
00:52:35 — Smashing the Overton Window
00:59:27 — Ronny, What's Your P(Doom)?™
01:05:41 — Brangus Lore, No Cringe, No Disgust
01:16:57 — Introducing Avisha NessAiver
01:17:44 — How He Makes Viral Hits
01:22:16 — Award-Winning Explainer Vid
01:26:51 — Avisha, What's Your P(Doom)?™
01:32:11 — Introducing Avalon Warren
01:35:49 — Normalizing AI Doom
01:37:53 — Avalon, What's Your P(Doom)?™
01:39:27 — Introducing Josh Thor
01:41:36 — Josh, What's Your P(Doom)?™
01:44:34 — Josh's Hits: Cover Song & Soares Collab
01:47:50 — Josh's Critiques of PDKU
01:52:05 — Cold-Emailing a Politician
01:54:07 — Donation Drive
01:55:01 — Outro: What Are You Fighting For?
Links
PlzDontKillUs (applications open for round two) — https://pleasedontkillus.com
Aella on X — https://x.com/Aella_Girl
Aella's Substack (Knowingless) — https://aella.substack.com
Ronny Fernandez's Substack (Brangus) — https://ratorthodox.substack.com
Ronny Fernandez on X — https://x.com/RatOrthodox
Avisha NessAiver — Distilled Science on Instagram — https://www.instagram.com/distilledscience/
Distilled Science (site) — https://distilledscience.xyz
Avalon Warren on Instagram — https://www.instagram.com/avalonwarrenn/
Josh Thor on X — https://x.com/joshthor9
Josh Thor — "Last Year Alive" ("Last Friday Night" AI apocalypse parody, ft. Eliezer Yudkowsky and a Nate Soares sax solo) — https://www.youtube.com/watch?v=9fYIm72GqrE
Tyler Alterman — "What are you fighting for? AI safety professionals answer" (the closing montage) — https://www.youtube.com/watch?v=9sN1VfVAp6o
Nate Soares — "AI Danger But You Don't Understand Full Sentences" (caveman video, script by Josh Thor) — https://www.instagram.com/p/DboGRisxC2E/
Avisha's AI water-use misinformation debunk (8.5M views) — https://www.instagram.com/p/DbXV6j9k8JE/
Avisha's award-winning "clearest AI x-risk explainer" — https://www.instagram.com/p/DbZeRT4gIq0/
Avisha's viral "Tingles and Touch" oxytocin video — https://www.instagram.com/p/DIxMLxkuhCl/
Avalon — "if the AI apocalypse is coming, might as well dress cute" — https://www.instagram.com/p/DbUi6yGCS9M/
Avalon — warrior-princess ballroom dress video (3.6M views) — https://www.tiktok.com/@avalonwarren/video/6892040140248632582
Serialsevens — the anime "If Anyone Builds It" video — https://www.instagram.com/serialsevens/
Kenorlands — PDKU creator clip featured in the episode — https://www.instagram.com/kenorlands/
Evie (@0xgingergirl) — "first month in SF" video — https://x.com/0xgingergirl
White House press briefing, March 30, 2023 — Peter Doocy asks about "literally everyone on Earth will die" and Karine Jean-Pierre laughs — https://bidenwhitehouse.archives.gov/briefing-room/press-briefings/2023/03/30/press-briefing-by-press-secretary-karine-jean-pierre-22/
White House press briefing, May 30, 2023 — the AI extinction statement gets a serious answer — https://bidenwhitehouse.archives.gov/briefing-room/press-briefings/2023/05/30/press-briefing-by-press-secretary-karine-jean-pierre-and-office-of-management-and-budget-director-shalanda-young/
Wes & Dylan Join Doom Debates — Violent Robots, Eliezer Yudkowsky, & Who Has the HIGHEST P(Doom)?! — https://www.youtube.com/watch?v=Fy6gz41-6gc
Aella — "My attempts to sensemake AI risk" (Aug 2022) — https://aella.substack.com/p/my-attempts-to-sensemake-ai-risk
If Anyone Builds It, Everyone Dies — https://ifanyonebuildsit.com

---
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe

In April 2023, weeks after GPT-4 launched, when many were saying it was just a "stochastic parrot", I went on AdQuick's Madvertising podcast hosted by Adam Singer and warned that GPT was "about to be the strongest hacker the world has ever known".
Watch that conversation, which also covered the Web3 bubble and AI doom more broadly, and judge for yourself how my claims are holding up.
Watch on YouTube: https://www.youtube.com/watch?v=M6gNwZWY-T8
Timestamps
00:00:00 — Adam Introduces Liron
00:02:03 — Liron's P(Doom) Poll
00:03:24 — Web3 Postmortem
00:05:05 — Axie Infinity and Hollow Abstractions
00:07:54 — Brand Marketers' Metaverse FOMO
00:13:33 — Near-Term AI: Elevator to Heaven
00:16:46 — AI at Relationship Hero
00:18:25 — From Chatbots to Bricking the Universe
00:20:17 — The Passive Income Doom Scenario
00:22:58 — 50 Orders of Magnitude Smarter
00:26:10 — Accelerationism and Transhumanism
00:28:51 — Why There's No Off Button
00:31:00 — Can't We Hard-Code Safeguards?
00:33:13 — CRISPR, Nukes, and Fusion
00:35:40 — AI Doom vs. Climate Change
00:39:10 — Nanotechnology Endgame
00:41:45 — AI Attackers vs. AI Defenders
00:43:28 — The Ethics of AI Persuasion
00:46:41 — The Blue Sky Scenario
00:49:04 — Will AI Kill Craft?
00:51:13 — Stuck Culture and Hollywood Sequels
00:54:50 — The Pause Letter: Don't Train GPT-5
00:59:00 — Worldcoin and Proof of Humanity
01:00:44 — "You Probably Won't Die of Old Age"
01:01:52 — Wrap-Up
Original episode: https://madvertising.buzzsprout.com/2118560/episodes/12603815-liron-shapira-web-3-mania-cybersecurity-how-ai-could-brick-the-universe-madvertising-6
Links
ADAM SINGER & MADVERTISING
Adam Singer on X: https://x.com/AdamSinger
Madvertising by AdQuick (the original show): https://madvertising.buzzsprout.com/
LIRON'S PROJECTS MENTIONED
Bloated MVP (Liron's blog on hollow abstractions): https://www.bloatedmvp.com/
REFERENCED IN THE EPISODE
"Pause Giant AI Experiments" open letter (FLI, March 2023): https://futureoflife.org/open-letter/pause-giant-ai-experiments/
Eliezer Yudkowsky — "Pausing AI Developments Isn't Enough. We Need to Shut it All Down" (TIME, March 2023): https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/
Rob Miles AI Safety on YouTube: https://www.youtube.com/c/robertmilesai
Engines of Creation by Eric Drexler: https://www.amazon.com/Engines-Creation-Nanotechnology-Library-Science/dp/0385199732
---
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe

Michael Vassar used to work closely with Eliezer Yudkowsky as President of the Singularity Institute for Artificial Intelligence, the organization that later became the Machine Intelligence Research Institute (MIRI).
Even though his P(Doom) is in the tens of percents by 2050, he says the people trying to pause AI are "bad people”. He hopes this episode convinces me to stop being a fearmonger… but I claim the general public needs to hurry up and get more scared.
All aboard the Doom Train! 🚂
Watch on YouTube: https://www.youtube.com/watch?v=4GbFng6htaQ

Timestamps
00:00:00 — Cold Open
00:01:26 — Introducing Michael Vassar
00:02:39 — Shorting Boeing at the Start of COVID
00:05:41 — What's Your P(Doom)™?
00:08:11 — The Magic Dial Thought Experiment
00:10:25 — "A Treaty Between Demis and Dario"
00:13:33 — AI CEOs 20 Years Ago
00:15:59 — Lessons from Peter Thiel
00:17:47 — Riding the Doom Train™
00:19:41 — Will the Human Brain Stay Relevant?
00:22:21 — When Does a Singleton Form?
00:28:35 — Hugging Face Hack
00:30:43 — Mistake Theory vs. Conflict Theory
00:33:18 — Debating the Orthogonality Thesis
00:36:38 — Maximally Evil AI Scenario
00:40:28 — Is the AI Development Process Safe?
00:42:05 — "PauseAI People Are Bad People"
00:44:44 — Michael Psychoanalyzes Liron
00:49:39 — How Liron Can Redeem Himself
00:55:10 — Michael Critiques Eliezer Yudkowsky
00:59:11 — Is Doomerism Bad Epistemology?
01:02:03 — Do Serious People Believe in Doom?
01:04:45 — Should Liron Stop Fearmongering?
01:07:16 — Political Violence Won't Help
01:11:04 — The Data Center Airstrike Scenario
01:13:34 — Could Every Nation Sign a Pause Treaty?
01:17:21 — What "Governance" Actually Means
01:26:12 — Michael's History with Rationalists
01:33:43 — Are We in a Computer Simulation?
01:38:43 — Wrap-Up
Links
Michael Vassar on X — https://x.com/HiFromMichaelV
The anti-anti-normativity sequence on Natural Hazard (Michael's recommended reading) — https://naturalhazard.xyz/ben_jess_sarah_starter_pack
Michael on DemystifySci #366 — "1000 Year Plan to Crush Thought" — https://www.youtube.com/watch?v=W17Cx21in9I
Michael on The Vance Crowe Podcast #303 — Good vs. Evil — https://open.spotify.com/episode/2RzlQDSwxGbjloRKqCh1xg
Michael on Clearer Thinking with Spencer Greenberg — Preference Falsification and Postmodernism — https://podcast.clearerthinking.org/episode/028/michael-vassar-preference-falsification-and-postmodernism/
"The Survival of Noise Traders in Financial Markets" — De Long, Shleifer, Summers & Waldmann — https://www.nber.org/papers/w2715
Scott Alexander — Conflict vs. Mistake — https://slatestarcodex.com/2018/01/24/conflict-vs-mistake/
Scott Alexander — Kolmogorov Complicity and the Parable of Lightning — https://slatestarcodex.com/2017/10/23/kolmogorov-complicity-and-the-parable-of-lightning/
Scott Alexander — Samsara — https://slatestarcodex.com/2019/11/04/samsara/
Eliezer Yudkowsky — Meta-Honesty: Firming Up Honesty Around Its Edge-Cases — https://www.lesswrong.com/posts/xdwbX9pFEr7Pomaxv/meta-honesty-firming-up-honesty-around-its-edge-cases
Zvi Mowshowitz — Book Review: Going Infinite — https://thezvi.substack.com/p/book-review-going-infinite
Jaan Tallinn — "Why Now? A Quest in Metaphysics" (Singularity Summit 2012) — https://vimeo.com/54718573
Paul Graham — "Five Founders" (the Sam Altman essay) — https://paulgraham.com/5founders.html
Truthful AI (Owain Evans' safety org) — https://truthful.ai
METR — Measuring AI Ability to Complete Long Tasks (the time-horizon graph) — https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/
Statement on AI Risk (Center for AI Safety, 2023) — https://safe.ai/work/statement-on-ai-risk
Vitalik Buterin — Will "d/acc" Protect Humanity from Superintelligent AI? — https://lironshapira.substack.com/p/debate-with-vitalik-buterin-will
Holly Elmore — These Effective Altruists Betrayed Me — https://www.youtube.com/watch?v=mFFObQ6vnLU
Noah Smith — Top Economist Says P(Doom) Is 0.1% (the "AI drugs" claim) — https://www.youtube.com/watch?v=AwmJ-OnK2I4
---
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe