Support us! https://www.patreon.com/mlst Sara Hooker is an exceptionally talented and accomplished leader and research scientist in the field of machine learning. She is the founder of Cohere For AI, a non-profit research lab that seeks to solve complex machine learning problems. She is passionate about creating more points of entry into machine learning research and has dedicated her efforts to understanding how progress in this field can be translated into reliable and accessible machine learning in the real-world. Sara is also the co-founder of the Trustworthy ML Initiative, a forum and seminar series related to Trustworthy ML. She is on the advisory board of Patterns and is an active member of the MLC research group, which has a focus on making participation in machine learning research more accessible. Before starting Cohere For AI, Sara worked as a research scientist at Google Brain. She has written several influential research papers, including "The Hardware Lottery", "The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation", "Moving Beyond “Algorithmic Bias is a Data Problem”" and "Characterizing and Mitigating Bias in Compact Models". In addition to her research work, Sara is also the founder of the local Bay Area non-profit Delta Analytics, which works with non-profits and communities all over the world to build technical capacity and empower others to use data. She regularly gives tutorials on machine learning fundamentals, interpretability, model compression and deep neural networks and is dedicated to collaborating with independent researchers around the world. Sara Hooker is famous for writing a paper introducing the concept of the 'hardware lottery', in which the success of a research idea is determined not by its inherent superiority, but by its compatibility with available software and hardware. She argued that choices about software and hardware have had a substantial impact in deciding the outcomes of early computer science history, and that with the increasing heterogeneity of the hardware landscape, gains from advances in computing may become increasingly disparate. Sara proposed that an interim goal should be to create better feedback mechanisms for researchers to understand how their algorithms interact with the hardware they use. She suggested that domain-specific languages, auto-tuning of algorithmic parameters, and better profiling tools may help to alleviate this issue, as well as provide researchers with more informed opinions about how hardware and software should progress. Ultimately, Sara encouraged researchers to be mindful of the implications of the hardware lottery, as it could mean that progress on some research directions is further obstructed. If you want to learn more about that paper, watch our previous interview with Sara. TOC: [00:00:00] Intro [00:02:53] Interpretability / Fairness [00:35:29] LLMs Find Sara: https://www.sarahooker.me/ https://twitter.com/sarahookr MLST Discord: https://discord.gg/aNPkGUQtc5 Pod: https://anchor.fm/machinelearningstreettalk/episodes/92---SARA-HOOKER---Fairness--Interpretability--Language-Models-e1skba2 References; A Tale Of Two Long Tails [Daniel D’souza, Zach Nussbaum, Chirag Agarwal, Sara Hooker] https://arxiv.org/pdf/2107.13098.pdf When less is more: Simplifying inputs aids neural network understanding [Robin Tibor Schirrmeister, Rosanne Liu, Sara Hooker, Tonio Ball] https://arxiv.org/pdf/2201.05610.pdf Mysteries of mode collapse [janus/LW] https://www.lesswrong.com/posts/t9svvNPNmFf5Qa3TA/mysteries-of-mode-collapse The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation https://arxiv.org/abs/2110.03036 Moving beyond “algorithmic bias is a data problem” [Sara Hooker] https://www.sciencedirect.com/science/article/pii/S2666389921000611 Characterising Bias in Compressed Models [Hooker et al] https://arxiv.org/abs/2010.03058 December 2021 CACM: The Hardware Lottery [we used clips of Sara from this video] https://www.youtube.com/watch?v=POozdlZAyQc