Loading
Episode
801: Merged LLMs Are Smaller And More Capable — with Arcee AI's Mark McQuade and Charles Goddard
Views
1K
0.1× the show’s average
Episode1K
Show avg17K
Published
2y ago
Jul 16, 2024
Length
1h 16m
show avg 1h 09m
From the show
Chapters
#LLMs #ModelMerging #SmallLanguageModels Merged LLMs are the future, and we’re exploring how with Mark McQuade and Charles Goddard from Arcee AI on this episode with @JonKrohnLearns. Learn how to combine multiple LLMs without adding bulk, train more efficiently, and dive into different expert approaches. Discover how smaller models can outperform larger ones and leverage open-source projects for big enterprise wins. This episode is packed with must-know insights for data scientists and ML engineers. Don’t miss out! Interested in sponsoring a SuperDataScience Podcast episode? Email natalie@superdatascience.com for sponsorship information. In this episode you will learn: Additional materials: https://www.superdatascience.com/801
- 0:00Introduction
- 2:55Explanation of Charles' job title: Chief of Frontier Research
- 3:10Model Merging Technology combining multiple LLMs without increasing size
- 13:31Using MergeKit for model merging
- 21:21Evolutionary Model Merging using evolutionary algorithms
- 26:28Commercial applications and success stories
- 36:31Comparison of Mixture of Experts (MoE) vs. Mixture of Agents
- 52:33Spectrum Project for efficient training by targeting specific modules
- 59:50Future of Small Language Models (SLMs) and their advantages
Related
Talks about similar things
808: In Case You Missed It in July 2024 — with Jon Krohn (@JonKrohnLearns)
356 viewsSuper Data Science: ML & AI Podcast with Jon Krohn·Aug 9, 2024
707: Vicuña, Gorilla, Chatbot Arena and Socially Beneficial LLMs — with Prof. Joey Gonzalez
809 viewsSuper Data Science: ML & AI Podcast with Jon Krohn·Aug 22, 2023
Why Would AI Companies Want to Slow Down?
28K viewsa16z·Sep 18, 2026
How Decagon Runs 90% of Its Agents on Open-Source Models
14K viewsa16z·Jul 31, 2026
Grab’s $235 Million Profit, BYD Takes Over in Singapore, Toyota Lays Off Staff
51K viewsAsia Tech Podcast Official·Aug 6, 2026
XPeng’s $900 Million Bet on Robots and AI, Alibaba's Qwen launches Qwen3.8-Flash AI
48K viewsAsia Tech Podcast Official·Aug 27, 2026