Doom Debates!

Now that Doom Debates is almost 2 years old, enough time has passed that we can revisit my claims from 2024. I claim they’ve been aging well!
In this early episode, I react to Martin Casado, a General Partner at Andreessen Horowitz (a16z) who claims that AI is basically just a buzzword for statistical models and simulations. As a result of this worldview, he only predicts incremental AI progress that doesn’t pose an existential threat to humanity, and he sees AI regulation as a net negative.
From my perspective, Martin’s problem is that he needs to go beyond analyzing AI as just statistical models and simulations, and analyze it using the more predictive concept of “intelligence” in the sense of hitting tiny high-value targets in exponentially-large search spaces.If Martin appreciated that intelligence is a quantifiable property that algorithms have, and that our existing AIs are getting close to surpassing human-level general intelligence, then hopefully he’d come around to raising his P(doom) and appreciating the urgent extinction risk we face.

Links
Watch the original episode of the Cognitive Revolution podcast with Martin and host Nathan Labenz — https://www.youtube.com/watch?v=oZ7788oKoss
Follow Martin — https://x.com/martin_casado
Follow Nate — https://x.com/labenz
Follow Liron — https://x.com/liron

Timestamps
00:00 Introducing Martin Casado
01:42 Martin’s AGI Timeline
05:39 Martin’s Analysis of Self-Driving Cars
15:30 Heavy-Tail Distributions
38:03 Understanding General Intelligence
38:29 AI's Progress in Specific Domains
43:20 AI’s Understanding of Meaning
47:16 Compression and Intelligence
48:09 Symbol Grounding
53:24 Human Abstractions and AI
01:18:18 The Frontier of AI Applications
01:23:04 Human vs. AI: Concept Creation and Reasoning
01:25:51 The Complexity of the Universe and AI's Limitations
01:28:16 AI's Potential in Biology and Simulation
01:32:40 The Essence of Intelligence and Creativity in AI
01:41:13 AI's Future Capabilities
02:00:29 Intelligence vs. Simulation
02:14:59 AI Regulation
02:23:05 Concluding Thoughts
---
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe

Dr. Ben Goertzel is a pioneering AI researcher, entrepreneur, and author who has helped organize AGI as a research field since the 2000s through books, workshops, and conferences.
We talk about the early days of AGI research with Yudkowsky, get Ben’s timeline to the singularity, and I try to understand what makes him so optimistic that superintelligence will be beneficial for humanity.
Timestamps
00:00:00 — Cold Open
00:00:59 — Introducing Dr. Ben Goertzel
00:04:11 — How Did Ben Get Interested in AGI?
00:08:26 — Early AGI Enthusiast Community: Extropians & SL4
00:15:15 — Is There Headroom Above Human Intelligence?
00:19:53 — Can Superintelligent AI Kill Everyone?
00:28:25 — Ben’s AGI Timeline: 2029
00:36:00 — What Modern AI Is Missing
00:42:01 — AGI to Superintelligence — FOOM in a Matter of Months
00:46:34 — What’s Your P(Doom)?™
00:53:37 — Ben’s Case for AI Acceleration
01:03:50 — The Orthogonality Thesis Is “Dishonest”
01:13:43 — Liron: Moore’s Law for Goal Achievement
01:23:50 — Ben: Alignment Is Just Good Parenting
01:27:36 — Alignment in Practice: OmegaCLAW
01:35:47 — Creating Stable Goals Under Self-Modification
01:45:22 — SingularityNET, OpenBGI, and the ASI Alliance
01:57:31 — The $ASI Token Economy
02:01:15 — Closing
Links
Ben Goertzel on X — https://x.com/bengoertzel
SingularityNET — https://singularitynet.io/
OpenBGI — https://openbgi.com/
Hyperon AGI Framework — https://hyperon.dev/
OmegaCLAW on GitHub — https://github.com/asi-alliance/OmegaClaw-Core
ASI Alliance — https://superintelligence.io/
The Consciousness Explosion (book) — https://theconsciousnessexplosion.ai/
“The Singularity Institute’s Scary Idea (and Why I Don’t Buy It)” (2010) — http://multiverseaccordingtoben.blogspot.com/2010/10/singularity-institutes-scary-idea-and.html
Ken Stanley Debate — https://lironshapira.substack.com/p/debate-with-a-former-openai-research
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe

We recap the Dr. Mike Israetel debate and dig into the nuances of rationality vs. rationalization. Then onto the week's biggest news: Anthropic dethroning OpenAI, Trump and Xi Jinping exploring AI guardrails, and Liron's own company AI going rogue.
Timestamps
00:00:00 — Cold Open
00:03:09 — Dr. Mike Israetel Round 2 Recap
00:14:34 — A Viewer Accuses Liron of Rationalizing
00:34:20 — Is Sam Altman Any Worse than a Typical Tech CEO?
00:45:15 — What Would Lower P(Doom) Below 10%?
00:53:25 — Can We Disconnect an ASI from the Internet?
01:05:00 — Should Doomers Boycott AI?
01:16:27 — Liron Reacts to ThePrimeagen Reacting to the Yud Debate
01:26:13 — Ben Goertzel Episode Preview
01:38:10 — Scrolling Twitter: MIRI, Ryan Greenblatt
01:47:32 — Liron’s AI Goes Rogue at Work
01:49:15 — “Positive Alignment” Paper Gets Destroyed
01:58:30 — Anthropic Dethrones OpenAI
02:12:06 — Trump & Xi Talk AI Guardrails
Links
Zvi Mowshowitz AI roundup blog — https://thezvi.wordpress.com/
“The Bottom Line” by Eliezer Yudkowsky (LessWrong) — https://www.lesswrong.com/posts/34XxbRFe54FycoCDw/the-bottom-line
Rationality: From AI to Zombies by Eliezer Yudkowsky — https://intelligence.org/rationality-ai-zombies/
Less Online 2026 conference (June 5–7) — https://less.online/
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe

In December 2023, I joined the Theo Jaffee Podcast to talk about a wide range of topics including non-AI x-risks, where I disagree with Eliezer Yudkwowsky, cryptocurrency and if P(Doom) is rigorous. This episode was recorded 6 months before I'd started Doom Debates. Let's look back with the power of hindsight and see if my answers still hold up.
Links
#10: Liron Shapira — AI doom, FOOM, rationalism, and crypto (YouTube) — https://youtu.be/YfEcAtHExFM
Watch Theo Jafee now on MTS — https://www.mts.now/
MTS Live — https://x.com/MTSlive
Timestamps
00:00:00 — Theo's Introduction
00:01:03 — Is Liron Worried About Non-AI Existential Risks
00:03:36 — Suffering Risks
00:05:22 — Is P(Doom) rigorous?
00:09:42 — Is Eliezer Overestimating P(Doom)?
00:12:18 — Where Does Liron Disagree with Eliezer?
00:15:43 — What Would Change Liron's Mind
00:17:14 — Elon Musk's AI Timelines
00:18:34 — Is xAI Making Things Worse?
00:20:24 — The Case for an AI Manhattan Project
00:22:32 — Are Superforecasters Wrong About AI Risk?
00:26:02 — The Race Against Time for Alignment
00:28:01 — Headroom Above Human Intelligence
00:33:31 — Vitalik's d/acc Framework
00:35:23 — Edging Toward Superintelligence
00:38:21 — From Chatbot to World-Ender
00:41:55 — GPT Paradigm vs. AlphaZero
00:43:18 — Critiquing AI Optimism
00:48:29 — Deceptive Alignment and Gradient Descent
00:53:31 — Does Nice Training Data Make Nice AI?
00:57:57 — How Do You Live with 50% P(Doom)?
01:00:08 — Why Have Kids If the World Might End?01:02:16 — Israel vs Hamas
01:06:15 — How LessWrong Changed Liron's Lif
e01:08:42 — Rationalism and Effective Altruism
01:14:49 — Why Blockchain Has No Use Case Beyond Cryptocurrency
01:22:13 — Charlie Munger and Richard Feynman
01:24:38 — Closing
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe

Dr. Mike Israetel is back in the arena to FIGHT ME (verbally) about the likelihood of AI killing everyone vs. the likelihood of AI making the universe totally awesome!
Back by popular demand after last year’s widely-viewed debate, the renowned exercise scientist, fitness entrepreneur, and low-P(Doom) AI futurist reveals where he gets off the Doom Train™, why he thinks AI is conscious, and how he pictures the coming AI utopia.

Links
Watch Dr. Mike’s AI/futurism YouTube channel —https://www.youtube.com/@mikeisraetelmakingprogress
Dr. Mike on X — https://x.com/misraetel
Make sure to watch Round 1 of our debate from last year — https://www.youtube.com/watch?v=RaDWSPMdM4o
Liron referenced a recent episode, “Steven Byrnes Returns — 90% P(Doom) on the Trajectory to ASI” — https://www.youtube.com/watch?v=oOb9K1KIAyk

Timestamps
00:00:00 — Cold Open
00:00:48 — Returning Champion Dr. Mike Israetel!
00:02:13 — What's Been Your Biggest AI Productivity Gain?
00:03:46 — Can AI Be a Personal Trainer? Dr. Mike Judges Liron's Technique
00:09:41 — Get Good at Taking Advice From AI
00:14:23 — Dr. Mike Israetel, What's Your Latest P(Doom)?™
00:20:12 — Liron's Mainline Scenario: Loss of Control
00:21:39 — Mike's Problems With a Paperclip Maximizer Scenario
00:26:00 — What Will the Goals of a Superintelligence Be?
00:28:49 — AI Labs Have Bit Off More Than They Can Chew
00:31:09 — Current AIs Are Moral
00:34:40 — AIs With a Ruthless Game-Playing Flavor
00:38:48 — Introducing the Doom Train
00:45:07 — "Can It" vs. "Will It" — Two-Part Argument on AI X-Risk
00:49:40 — Mike's Tweet on Superintelligence by 2027
01:01:34 — AI as a Goal Engine
01:20:51 — Offense vs. Defense in the Coming Cyber War
01:39:14 — Is Morality Just Game Theory? The "Different Flavor" of Future AI
01:46:16 — Ricardo's Law: Does AI Have Reason to Keep Us Around?
01:52:14 — Why Mike Gets Off the Doom Train
01:54:58 — Is AI Conscious?
02:01:39 — Mike's Vision for the Coming AI Utopia
02:06:52 — Wrap-Up
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏


Get full access to Doom Debates at lironshapira.substack.com/subscribe