(2024-06-10) Zvim On Dwarkeshs Podcast With Leopold Aschenbrenner

Zvi Mowshowitz: On Dwarkesh Patel's Podcast with Leopold Aschenbrenner. The central discussion was the theses of his paper, Situational Awareness, which I offered quotes from earlier, with a focus on the consequences of AGI rather than whether AGI will happen soon. There are also a variety of other topics. (2024-06-01) Leopold Aschenbrenner's AGI Situational Awareness Paper

While I have strong disagreements with Leopold, only some of which I detail here, and I especially believe he is dangerously wrong and overly optimistic about alignment, existential risks and loss of control in ways that are highly load bearing, causing potential sign errors in interventions, and also I worry that the new AGI fund may make our situation worse rather than better, I want to most of all say: Thank you. Leopold has shown great courage. He stands up for what he believes in even at great personal cost.

By writing down, in great detail, what he actually believes, he allows us to compare notes and arguments, and to move forward. This is The Way. I have often said I need better critics. This is a better critic. A worthy opponent.

The Trillion Dollar Cluster

AGI 2028: The Return of History

In 2023, Leopold could start to feel the AGI, see the training clusters that would be built, the rough algorithms it would use and so on. Expects rest of the world to feel it soon.

Espionage & American AI Supremacy

Geopolitical Implications of AI

State-Led vs. Private-Led AI

Skipping Sections

Intelligence Explosion

This is a super frustrating

Dwarkesh’s Immigration Story

AGI Investment Fund: Leopold is launching one. Why? The post-AGI period will be important and there is a lot of money to be made.

Lessons From WW2

Changing your mind is really important.

I see lots of the proposals by many on alignment, including Leopold here and in his paper, as being naive and not making sense and not reflecting the underlying realities, so there you go.