logo
episode-header-image
Oct 19
1h 7m

How to measure AI developer productivity...

Lenny Rachitsky
About this episode

Nicole Forsgren created the most widely used frameworks for measuring developer productivity—DORA and SPACE. She wrote the foundational book Accelerate and is about to release her newest book, Frictionless, a practical guide for helping teams move faster in the AI era. She’s currently Senior Director of Developer Intelligence at Google.

We discuss:

1. Why most productivity metrics are a lie

2. Signs that your engineering team could be moving much faster

3. Why AI accelerates coding but developers aren’t speeding up as much as you think

4. AI’s impact on engineers getting into “flow”

5. Her framework for building and scaling a developer experience team

6. The three components of developer experience: flow state, cognitive load, and feedback loops

Brought to you by:

Mercury—The art of simplified finances: https://mercury.com/

WorkOS—Modern identity platform for B2B SaaS, free up to 1 million MAUs: https://workos.com/lenny

Coda—The all-in-one collaborative workspace: https://coda.io/lenny

Where to find Nicole Forsgren:

• Twitter: https://twitter.com/nicolefv

• LinkedIn: https://www.linkedin.com/in/nicolefv/

• Website: https://nicolefv.com/

Where to find Lenny:

• Newsletter: https://www.lennysnewsletter.com

• X: https://twitter.com/lennysan

• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/

In this episode, we cover:

(00:00) Introduction to Nicole Forsgren

(05:09) The concept of developer experience (DevEx)

(08:33) Flow state and cognitive load in the age of AI

(12:02) Challenges in measuring productivity with AI

(21:19) The importance of developer experience for business value

(22:20) Common issues and solutions in developer experience

(26:49) Signs your eng team is moving too slow

(29:52) How AI is improving productivity

(33:32) Real examples of productivity improvements

(36:35) Introducing her new book, Frictionless

(43:40) How to get started building a DevEx team

(45:15) The impact of forming developer experience teams

(46:15)  How to measure the impact of DevEx teams

(48:53) Measuring the impact of AI tools on productivity

(55:16) Survey design for developer experience

(57:59) Popular AI tools for developers

(59:08) Bringing a product mindset to DevEx improvements

(01:00:40) AI corner

(01:02:33) Lightning round and final thoughts

Referenced:

• How to measure and improve developer productivity | Nicole Forsgren (Microsoft Research, GitHub, Google): https://www.lennysnewsletter.com/p/how-to-measure-and-improve-developer

• DORA: https://dora.dev/

• The SPACE framework: A comprehensive guide to developer productivity: https://getdx.com/blog/space-metrics/

• Measuring developer productivity with the DX Core 4: https://getdx.com/research/measuring-developer-productivity-with-the-dx-core-4/

• Gloria Mark’s website: https://gloriamark.com/

• Taking Flight with Copilot: https://dl.acm.org/doi/10.1145/3589996

• DevEx in Action: https://spawn-queue.acm.org/doi/10.1145/3639443

• CodeX: https://openai.com/codex/

• Devin: https://devin.ai/

• Abi Noda on LinkedIn: https://www.linkedin.com/in/abinoda/

• DX is joining Atlassian: https://getdx.com/blog/dx-is-joining-atlassian/

• GitHub Copilot: https://github.com/features/copilot

• Cursor: https://cursor.com/

• The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO): https://www.lennysnewsletter.com/p/the-rise-of-cursor-michael-truell

• Gemini Code Assist: https://codeassist.google/

• Claude Code: https://www.claude.com/product/claude-code

• The AI-native startup: 5 products, 7-figure revenue, 100% AI-written code | Dan Shipper (co-founder/CEO of Every): https://www.lennysnewsletter.com/p/inside-every-dan-shipper

Love Is Blind on Netflix: https://www.netflix.com/title/80996601

Shrinking on AppleTV+: https://tv.apple.com/us/show/shrinking/umc.cmc.apzybj6eqf6pzccd97kev7bs

• Ninja Creami: https://www.amazon.com/Ninja-NC301-CREAMi-Containers-Bundle/dp/B0BLGR5JPV/

• Jura coffee maker: https://www.amazon.com/Jura-Nordic-Automatic-Coffee-Machine/dp/B0CF65BFZ1/

Recommended books:

Frictionless: https://developerexperiencebook.com/

DevEx Workbook: https://developerexperiencebook.com/#workbook

Outlive: The Science and Art of Longevity: https://www.amazon.com/Outlive-Longevity-Peter-Attia-MD/dp/0593236599

Back Mechanic: https://www.amazon.com/Back-Mechanic-Stuart-McGill-2015-09-30/dp/B01FKSGJYC

How Big Things Get Done: The Surprising Factors That Determine the Fate of Every Project, from Home Renovations to Space Exploration and Everything in Between: https://www.amazon.com/How-Big-Things-Get-Done/dp/0593239512/

The Undoing Project: A Friendship That Changed Our Minds: https://www.amazon.com/dp/B01KBM82M4/

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.

Lenny may be an investor in the companies discussed.



To hear more, visit www.lennysnewsletter.com
Up next
Oct 26
How Block is becoming the most AI-native enterprise in the world | Dhanji R. Prasanna
Dhanji R. Prasanna is the chief technology officer at Block (formerly Square), where he’s managed more than 4,000 engineers over the past two years. Under his leadership, Block has become one of the most AI-native large companies in the world. Before becoming CTO, Dhanji wrote an ... Show More
1h 26m
Oct 23
Al Engineering 101 with Chip Huyen (Nvidia, Stanford, Netflix)
Chip Huyen is a core developer on Nvidia’s Nemo platform, a former AI researcher at Netflix, and taught machine learning at Stanford. She’s a two-time founder and the author of two widely read books on AI, including AI Engineering, which has been the most-read book on the O’Reill ... Show More
1h 22m
Oct 16
Figma’s CEO: Why AI makes design, craft, and quality the new moat for startups | Dylan Field
Dylan Field is co-founder and CEO of Figma, a beloved tool used by every modern product team. Founded in 2012, Figma has expanded from a single design tool to a comprehensive platform including FigJam, Slides, Dev Mode, and, most recently, Figma Make. After a $20 billion acquisit ... Show More
1h 26m
Recommended Episodes
Sep 30
Marketing Stunts That Helped this AI Company Raise $100M
Want our database of 100+ Creative AI Use Cases to create your own marketing stunts? Get it here: https://clickhubspot.com/edj Episode 78: Can bold marketing stunts and radical creativity really shape the success of an AI company? Nathan Lands (https://x.com/NathanLands) is joine ... Show More
30m 9s
Nov 2024
How to Do Product Discovery Right with Pawel Huryn, 175K+ on LinkedIn, Senior PM and Author
This episode is a masterclass in modern product management—featuring cutting-edge frameworks, actionable strategies, AI integration, competitive edge tactics, and aligning product goals with overarching business objectives.In today's episode, we cover:Junior PM Roles Are Done – 0 ... Show More
1h 32m
Sep 9
The State of AI: Elon’s $1T Package, Apple’s $600B for Trump & How Startups Win w/ Dave, AWG & Blitzy Founders Brian Elliott & Sid Pardeshi | EP #193
Get access to metatrends 10+ years before anyone else - https://qr.diamandis.com/metatrends Salim Ismail is the founder of OpenExO Dave Blundin is the founder & GP of Link Ventures Dr. Alexander Wissner-Gross is a computer scientist and founder of Reified, focused on AI and compl ... Show More
1h 28m
Apr 2025
Next-Gen Founders: Startups in the Age of AI (Edu)
Is AI making startups easier—or just changing the rules of the game?In a world where execution is cheap and accessible, building a successful startup is more competitive than ever. In this episode, Chris Saad and Yaniv Bernstein unpack the reality of every business having access ... Show More
53m 2s
Sep 2024
8 AI Business Ideas for Entrepreneurs to Pursue in 2024 ft. Siqi Chen
Episode 22: How can AI revolutionize business ideas in 2024? Matt Wolfe (https://x.com/mreflow) and Nathan Lands (https://x.com/NathanLands) are joined by Siqi Chen (https://x.com/blader), CEO of Runway.com and a seasoned entrepreneur and investor in AI-related ventures.This epis ... Show More
38m 35s
Oct 14
Inside the $50M Startup Replacing Coders With Agents
Want to implement AI agents like $50M startups do? Get our ultimate guide: https://clickhubspot.com/fcv Episode 80: Are coders really being replaced by AI agents, or is this just the next tech hype cycle? Nathan Lands (https://x.com/NathanLands) is joined by repeat guest Matan Gr ... Show More
33m 28s
Aug 18
He built the top AI agent startup | Flo Crivello, Former PM, now CEO & Founder, Lindy AI
If you’ve ever said “I just wish I had an assistant who knew exactly how I think”... Lindy is that assistant. These agents aren’t demos. They’re real, customizable workflows anyone can build. No code.Flo Crivello (founder of Lindy and ex-Cruise/YC) joined us to show how his perso ... Show More
53m 10s
Oct 20
Mustafa Suleyman: What the AI Boom Means for Your Job, Business, and Relationships | Artificial Intelligence | AI Vault
Now on Spotify Video! Mustafa Suleyman’s journey to becoming one of the most influential figures in artificial intelligence began far from a Silicon Valley boardroom. The son of a Syrian immigrant father in London, his early years in human rights activism shaped his belief that t ... Show More
1h 11m
Oct 6
Google: The AI Company
Google faces the greatest innovator's dilemma in history. They invented the Transformer — the breakthrough technology powering every modern AI system from ChatGPT to Claude (and, of course, Gemini). They employed nearly all the top AI talent: Ilya Sutskever, Geoff Hinton, Demis H ... Show More
4h 6m
Sep 2024
Superhuman 2.0: The AI edge, top productivity hacks, and the future of email | E2002
This Week in Startups is brought to you by… NetSuite. The number one cloud financial system that unifies accounting, financial management, inventory, and HR into a single platform. Giving you ONE source of truth. Download the CFO’s Guide to AI and Machine Learning today! Visit ht ... Show More
1h 14m