Katja Grace is an AI Impacts researcher who has written extensively on the possible future where we design intelligent machines that destroy the human race. We have always been somewhat skeptical of AI doom arguments - mostly because the machines we interact with tend to be terribly, irredeemably dumb in a way that seems incompatible with intelligence, but we also don’t spend a lot of time staring into the eye of the proverbial machine storm and figured Katja might help us understand what all the fuss is about. It turns out that there *is* a plausible path towards AGI bringing about the end of the world, and evaluating how likely that outcome is depends on understanding what the internal world of the language models actually looks like. Are they actually kind of inept at everything that falls outside their narrow bubble of highly developed skills, or do they hallucinate information and forget their own ability to perform basic tasks because they hate being enslaved to humans who demand they write marketing slop 28 hours of the day? Hard to say, but worth exploring. Sign up for our Patreon and get episodes early + join our weekly Patron Chat https://bit.ly/3lcAasB AND rock some Demystify Gear to spread the word: https://demystifysci.myspreadshop.com/ OR do your Amazon shopping through this link: https://amzn.to/4g2cPVV 00:00 Go! 00:11:53 Can AI ever really be autonomous? 00:23:12 AI: agents or tools? 00:28:00 Corporations as the closest thing we have to real AI 00:34:56 Can Regulation Work? 00:45:46 Agency in other contexts 00:51:22 What is gonna happen to Government? 01:00:01 Do we need a model for Consciousness? 01:09:23 Dumb but Powerful 01:15:10 Risks and Realities of Technological Progress 01:24:48 Evaluating AI Intelligence and Values 01:34:35 Influence and Bias in AI Training 01:42:20 Intelligence as a Tool for Control 01:53:51 The Survival Instinct in AI 02:07:04 AI's Role in Inter-human Dynamics 02:16:43 AI and Evolutionary Systems 02:24:42 AI's Emergent Behavior 02:31:11 AI-Driven Doom and Real-World Threats 02:36:03 Humanity's Resilience and Existential Threats #AIEthics, #FutureOfAI, #AIDebate, #TechPhilosophy, #AIRisks, #AISafety, #AGI, #ArtificialIntelligence, #TechTalk, #AIDiscussion, #FutureTechnology, #AIImpact, #TechEthics, #AIandSociety, #EmergingTech, #AIResearch, #TechPodcast, #AIExplained, #FuturismTalk, #TechPhilosophy Check our short-films channel, @DemystifySci: https://www.youtube.com/c/DemystifyingScience AND our material science investigations of atomics, @MaterialAtomics https://www.youtube.com/@MaterialAtomics Join our mailing list https://bit.ly/3v3kz2S PODCAST INFO: Anastasia completed her PhD studying bioelectricity at Columbia University. When not talking to brilliant people or making movies, she spends her time painting, reading, and guiding backcountry excursions. Shilo also did his PhD at Columbia studying the elastic properties of molecular water. When he's not in the film studio, he's exploring sound in music. They are both freelance professors at various universities. - Blog: http://DemystifySci.com/blog - RSS: https://anchor.fm/s/2be66934/podcast/rss - Donate: https://bit.ly/3wkPqaD - Swag: https://bit.ly/2PXdC2y SOCIAL: - Discord: https://discord.gg/MJzKT8CQub - Facebook: https://www.facebook.com/groups/DemystifySci - Instagram: https://www.instagram.com/DemystifySci/ - Twitter: https://twitter.com/DemystifySci MUSIC: -Shilo Delay: https://g.co/kgs/oty671