MLST Discord! https://discord.gg/aNPkGUQtc5 Patreon: https://www.patreon.com/mlst Pod: https://anchor.fm/machinelearningstreettalk/episodes/101-DR--WALID-SABA---Extrapolation--Vagueness-and-Behaviorism-e1up2er We had a discussion with Dr. Walid Saba about whether or not MLP neural networks can extrapolate outside of the training support, and what it means to extrapolate in a vector space. Then we discussed the concept of vagueness in cognitive science, for example, what does it mean to be "rich" or what is a "pile of sand"? Finally we discussed behaviourism and the reward is enough hypothesis. Panel: Dr. Tim Scarfe, Dr. Keith Duggar References: A Spline Theory of Deep Networks [Balestriero] https://proceedings.mlr.press/v80/balestriero18b/balestriero18b.pdf The animation we showed of the spline theory was created by Ahmed Imtiaz Humayun (https://twitter.com/imtiazprio) and we will be showing an interview with Imtiaz and Randall very soon! [00:00:00] Interpolation vs Extrapolation [00:23:41] Type 1 Type 2 generalisation and compositionality / Fodor / Systematicity [00:31:21] Keith's brain teaser [00:35:55] Neural turing machines / discrete vs continuous / learnability Note that we deleted the final third of the show because it needed some editing for tone, but we went on to discuss: Vagueness Concepts grounded in language