logo
episode-header-image
Mar 2021
54m 2s

Can Language Models Be Too Big? 🦜 with ...

Sam Charrington
About this episode

Today we’re joined by Emily M. Bender, Professor at the University of Washington, and AI Researcher, Margaret Mitchell. 

Emily and Meg, as well as Timnit Gebru and Angelina McMillan-Major, are co-authors on the paper On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜. As most of you undoubtedly know by now, there has been much controversy surrounding, and fallout from, this paper. In this conversation, our main priority was to focus on the message of the paper itself. We spend some time discussing the historical context for the paper, then turn to the goals of the paper, discussing the many reasons why the ever-growing datasets and models are not necessarily the direction we should be going. 

We explore the cost of these training datasets, both literal and environmental, as well as the bias implications of these models, and of course the perpetual debate about responsibility when building and deploying ML systems. Finally, we discuss the thin line between AI hype and useful AI systems, and the importance of doing pre-mortems to truly flesh out any issues you could potentially come across prior to building models, and much much more. 

The complete show notes for this episode can be found at twimlai.com/go/467.

Up next
Yesterday
Multimodal AI Models on Apple Silicon with MLX with Prince Canuma - #744
Today, we're joined by Prince Canuma, an ML engineer and open-source developer focused on optimizing AI inference on Apple Silicon devices. Prince shares his journey to becoming one of the most prolific contributors to Apple’s MLX ecosystem, having published over 1,000 models and ... Show More
1h 10m
Aug 19
Genie 3: A New Frontier for World Models with Jack Parker-Holder and Shlomi Fruchter - #743
Today, we're joined by Jack Parker-Holder and Shlomi Fruchter, researchers at Google DeepMind, to discuss the recent release of Genie 3, a model capable of generating “playable” virtual worlds. We dig into the evolution of the Genie project and review the current model’s scaled-u ... Show More
1h 1m
Aug 12
Closing the Loop Between AI Training and Inference with Lin Qiao - #742
In this episode, we're joined by Lin Qiao, CEO and co-founder of Fireworks AI. Drawing on key lessons from her time building PyTorch, Lin shares her perspective on the modern generative AI development lifecycle. She explains why aligning training and inference systems is essentia ... Show More
1h 1m
Recommended Episodes
Feb 2023
In Machines We Trust: The AI in the newsroom
We asked ChatGPT to summarize this episode and this is what it wrote: "In the episode, the host discussed the increasing use of AI language models like ChatGPT in newsrooms. The host explained that ChatGPT, a large language model developed by OpenAI, is being used to automate tas ... Show More
18m 13s
Dec 2022
Ethical AI
In this episode of High Theory, Alex Hanna talks with Nathan Kim about Ethical AI. Their conversation is part of our High Theory in STEM series, which tackles topics in science, technology, engineering, and medicine from a highly theoretical perspective. In this episode, Alex hel ... Show More
22m 39s
Jul 2023
Moore’s law in peril and the future of computing
Demand for computer power continues to soar, but can the hardware keep up? 
1h 1m
Oct 2023
Txt
In this episode of High Theory, Matthew Kirschenbaum talks about txt, or text. Not texting, or textbooks, but text as a form of data that is feeding large language models. Will the world end in fire, flood, or text?In the full interview, Matthew recommended Tim Maughan’s novel In ... Show More
19m 54s
Aug 2022
Babbage: How AI cracked biology’s biggest problem
DeepMind’s artificial-intelligence system AlphaFold has predicted the three-dimensional shape of almost all known proteins. The company’s boss Demis Hassabis tells us how the AI was able to solve what was, for decades, biology’s grand challenge. Plus, Gilead Amit, The Economist’s ... Show More
34m 34s
Aug 2023
20VC: Spending $2M to Train a Single AI Model: What Matters More; Model Size or Data Size | Hallucinations: Feature or Bug | Will Everyone Have an AI Friend in the Future & Raising $150M from a16z wit
Noam Shazeer is the co-founder and CEO of Character.AI, a full-stack AI computing platform that gives people access to their own flexible superintelligence. A renowned computer scientist and researcher, Shazeer is one of the foremost experts in artificial intelligence (AI) and na ... Show More
35m 9s
Jan 2024
This AI just figured out geometry — is this a step towards artificial reasoning?
In this episode:0:55 The AI that deduces solutions to complex maths problemsResearchers at Google Deepmind have developed an AI that can solve International Mathematical Olympiad-level geometry problems, something previous AIs have struggled with. They provided the system with a ... Show More
32m 24s
Mar 2023
Babbage: Is GPT-4 the dawn of true artificial intelligence?
OpenAI's ChatGPT, an advanced chatbot, has taken the world by storm, amassing over 100 million monthly active users and exhibiting unprecedented capabilities. From crafting essays and fiction to designing websites and writing code, you’d be forgiven for thinking there’s little it ... Show More
43m 22s
Jan 2023
24. Artificial Intelligence: What Is It? What Is It Not? (feat. Susan Farrell, Principal UX Researcher at mmhmm.app)
The term artificial intelligence, AI, is having a bit of a boom, with the explosion in popularity of tools like ChatGPT, Lensa, DALL•E 2, and many others. The praises of AI have been equally met with skepticism and criticism, with cautionary tales about AI information quality, pl ... Show More
35m 35s