Emmett Shear, founder of Twitch and former OpenAI interim CEO, challenges the fundamental assumptions driving AGI development. In this conversation with Erik Torenberg and Séb Krier, Shear argues that the entire "control and steering" paradigm for AI alignment is fatally flawed. Instead, he proposes "organic alignment" - teaching AI systems to genuinely care about humans the way we naturally do. The discussion explores why treating AGI as a tool rather than a potential being could be catastrophic, how current chatbots act as "narcissistic mirrors," and why the only sustainable path forward is creating AI that can say no to harmful requests. Shear shares his technical approach through multi-agent simulations at his new company Softmax, and offers a surprisingly hopeful vision of humans and AI as collaborative teammates - if we can get the alignment right. Timecodes: 00:00 - “If it's a machine, it's a tool. And if it's a being, it's a slave” 01:01 - Alignment Takes an Argument: The Hidden Assumptions in "Aligned AI" 02:26 - Alignment as Process, Not Destination 04:23 - Morality as Ongoing Learning, Not Fixed Rules 08:09 - Most AI Alignment Is Actually Steering (Or Slavery) 09:01 - The Dangerous Assumption: "We're Making Beings, But They Don't Count" 14:37 - Goal Inference and Theory of Mind 23:01 - The Foundation of Care 24:41 - Why Most AI Labs Focus on Steering and Control 27:35 - The Only Good Outcome: A Being That Actually Cares About Us 32:42 - The Substrate Question: Does Silicon vs. Carbon Matter? 51:24 - The Only Sustainable Form of Alignment 54:55 - AI Chatbots and Social Dynamics 59:50 - AI Futures: Tools, Beings, and Society 1:01:54 - Visions for a Good AI Future Resources: Follow Emmett on X: https://x.com/eshear Follow Séb on X: https://x.com/sebkrier Follow Erik on X: https://x.com/eriktorenberg Stay Updated: If you enjoyed this episode, be sure to like, subscribe, and share with your friends! Find a16z on X: https://x.com/a16z Find a16z on LinkedIn: https://www.linkedin.com/company/a16z Listen to the a16z Podcast on Spotify: https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX Listen to the a16z Podcast on Apple Podcasts: https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711 Follow our host: https://x.com/eriktorenberg Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details, please see a16z.com/disclosures.