logo
episode-header-image
Aug 2024
1h 12m

#199 – Nathan Calvin on California’s AI ...

Rob, Luisa, and the 80,000 Hours team
About this episode

"I do think that there is a really significant sentiment among parts of the opposition that it’s not really just that this bill itself is that bad or extreme — when you really drill into it, it feels like one of those things where you read it and it’s like, 'This is the thing that everyone is screaming about?' I think it’s a pretty modest bill in a lot of ways, but I think part of what they are thinking is that this is the first step to shutting down AI development. Or that if California does this, then lots of other states are going to do it, and we need to really slam the door shut on model-level regulation or else they’re just going to keep going.

"I think that is like a lot of what the sentiment here is: it’s less about, in some ways, the details of this specific bill, and more about the sense that they want this to stop here, and they’re worried that if they give an inch that there will continue to be other things in the future. And I don’t think that is going to be tolerable to the public in the long run. I think it’s a bad choice, but I think that is the calculus that they are making." —Nathan Calvin

In today’s episode, host Luisa Rodriguez speaks to Nathan Calvin — senior policy counsel at the Center for AI Safety Action Fund — about the new AI safety bill in California, SB 1047, which he’s helped shape as it’s moved through the state legislature.

Links to learn more, highlights, and full transcript.

They cover:

  • What’s actually in SB 1047, and which AI models it would apply to.
  • The most common objections to the bill — including how it could affect competition, startups, open source models, and US national security — and which of these objections Nathan thinks hold water.
  • What Nathan sees as the biggest misunderstandings about the bill that get in the way of good public discourse about it.
  • Why some AI companies are opposed to SB 1047, despite claiming that they want the industry to be regulated.
  • How the bill is different from Biden’s executive order on AI and voluntary commitments made by AI companies.
  • Why California is taking state-level action rather than waiting for federal regulation.
  • How state-level regulations can be hugely impactful at national and global scales, and how listeners could get involved in state-level work to make a real difference on lots of pressing problems.
  • And plenty more.

Chapters:

  • Cold open (00:00:00)
  • Luisa's intro (00:00:57)
  • The interview begins (00:02:30)
  • What risks from AI does SB 1047 try to address? (00:03:10)
  • Supporters and critics of the bill (00:11:03)
  • Misunderstandings about the bill (00:24:07)
  • Competition, open source, and liability concerns (00:30:56)
  • Model size thresholds (00:46:24)
  • How is SB 1047 different from the executive order? (00:55:36)
  • Objections Nathan is sympathetic to (00:58:31)
  • Current status of the bill (01:02:57)
  • How can listeners get involved in work like this? (01:05:00)
  • Luisa's outro (01:11:52)

Producer and editor: Keiran Harris
Audio engineering by Ben Cordell, Milo McGuire, Simon Monsour, and Dominic Armstrong
Additional content editing: Katy Moore and Luisa Rodriguez
Transcriptions: Katy Moore

Up next
Jul 15
Rebuilding after apocalypse: What 13 experts say about bouncing back
What happens when civilisation faces its greatest tests?This compilation brings together insights from researchers, defence experts, philosophers, and policymakers on humanity’s ability to survive and recover from catastrophic events. From nuclear winter and electromagnetic pulse ... Show More
4h 26m
Jul 8
#220 – Ryan Greenblatt on the 4 most likely ways for AI to take over, and the case for and against AGI in <8 years
Ryan Greenblatt — lead author on the explosive paper “Alignment faking in large language models” and chief scientist at Redwood Research — thinks there’s a 25% chance that within four years, AI will be able to do everything needed to run an AI company, from writing code to design ... Show More
2h 50m
Jun 24
#219 – Toby Ord on graphs AI companies would prefer you didn't (fully) understand
The era of making AI smarter just by making it bigger is ending. But that doesn’t mean progress is slowing down — far from it. AI models continue to get much more powerful, just using very different methods, and those underlying technical changes force a big rethink of what comin ... Show More
2h 48m
Recommended Episodes
Jan 2025
With OpenAI seeking profits, activist seeks payback to the public
A battle is brewing over the restructuring of OpenAI, the creator of pioneering artificial intelligence chatbot ChatGPT. It was founded as a nonprofit in 2015 with the goal of developing AI to benefit humanity, not investors. But advanced AI requires massive processing power, whi ... Show More
15m 21s
Oct 2024
OpenAI’s Path to Become a For-Profit Company Is Complicated
OpenAI plans to convert from a non-profit to a for-profit organization, a complex move that is rarely done. WSJ reporter Theo Francis joins host Zoe Thomas to discuss the hurdles that OpenAI will face and the possible reasons for the change. Plus, a controversial bill to regulate ... Show More
13m 16s
Oct 2024
20VC: Why Founder Mode is Dangerous & Could Encourage Bad Behaviour | Why Fundraising is a Waste of Time & OKRs are BS | Why Angel Investing is Bad for Founders to Do and the VC Model is on it's Last
Zach Perret is the CEO and Co-Founder of Plaid, a technology platform reshaping financial services. To date, Zach has raised over $734M for Plaid from the likes of NEA, Spark, GV, Coatue and a16z, to name a few. Today, thousands of companies including the largest fintechs, severa ... Show More
50m 53s
May 2023
Do fundraisers understand the nuances of corporate giving?
How many prospective funders agree to meet as a simple courtesy only to pass up the opportunity that has been presented to them? What if a better understanding of how corporations go about their decision-making processes could reduce the fundraiser’s workload and increase the lik ... Show More
43m 54s
Mar 2025
Nonprofits navigate Trump’s drastic funding cuts, with The Chronicle of Philanthropy CEO Stacy Palmer
President Trump’s dramatic cuts to U.S. government grants are destabilizing every corner of the non-profit sector, leaving organizations scrambling to adapt. Stacy Palmer, CEO of the Chronicle of Philanthropy, explores the executive order’s impact on both the public and private s ... Show More
27m 37s
Apr 2025
Purpose Isn’t Found, It’s Built with Aaron Hurst
In episode 230 of The Business Development Podcast, Kelly Kennedy is joined by Aaron Hurst, bestselling author of The Purpose Economy and a pioneer in the field of meaningful work. Together, they explore the idea that purpose isn’t something we find, but something we intentionall ... Show More
1h 6m
Dec 2024
Fixing Education in America: What's Stopping Us?
Over half of Americans live in childcare deserts, while 90% of brain development happens before the age of five. All the while, education and childcare remain among the most resistant sectors to technological change. Billions of dollars have been spent, but outcomes continue to l ... Show More
38m 54s
Dec 2024
Sam Altman's Equity Dilemma
In this episode of the AI Chat podcast, host Jaeden Schaefer discusses the controversies surrounding OpenAI, particularly focusing on Sam Altman's claims about his equity in the company. The conversation delves into Altman's testimony before Congress, the significant valuation gr ... Show More
14m 4s
Nov 2024
ChatGPT’s First Victim + The Department of Government Efficiency (DOGE)
Scott and Ed open the show by discussing Spotify and Disney’s earnings, a gambling company’s strong third quarter results, and Elliot Management’s activist investment in Honeywell. Then Scott breaks down how Chegg allowed ChatGPT to take its business to the woodshed and why he th ... Show More
54m 57s
Jul 2024
Pump and Dump Schemes are Now Totally Legal
Send us a textA Texas District Judge Andrew S. Hanen has dismissed all charges against seven social-media influencers the SEC and Justice Department had accused of perpetrating a “stock manipulation scheme” on Twitter and Discord, ruling that the prosecution failed to state an of ... Show More
20m 27s