logo
episode-header-image
Mar 2025
16m 16s

#105 AI UX is Broken – How Do We Measure...

PATRICIA REINERS
About this episode

In this episode, we’re tackling a huge question: How do we measure a “good” AI experience?

AI is shaping everything—from hiring decisions to medical diagnoses, from content recommendations to self-driving cars. But here’s the issue: we don’t even have clear UX standards for AI.

💡 If we don’t measure UX in AI properly, we risk building products that are functional but untrustworthy, accurate but unethical, powerful but frustrating to use.

We dive into:

〜 Why traditional UX metrics fail in AI-driven products
〜 The Amazon hiring AI scandal—how biased algorithms create bad UX
〜 The problem of explainability—why we often don’t understand AI decisions
〜 The future of AI UX standards—should there be a universal AI UX certification?

📌 Core Question: Can we trust AI to deliver great user experiences if we don’t even know how to measure them?

Let’s explore what’s missing in AI UX—and what needs to change.


Other resources

💻 ✨Waiting list for the AI Academy: 6-week Bootcamp to Learn How to Master AI as a Designer

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Get notified when doors are opening again⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠


👉 0€ AI mini Training for Designers 👈

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Learn 3 strategies to leverage gen AI as a designer in just 30 min⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠


💌 Newsletter

⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Free Weekly UX insights and updates: UX Newsletter⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠


Follow the Future of UX:

🎨 Linkedin: ⁠Future of UX Podcast⁠

📸 Instagram: ⁠Future of UX Podcast⁠

🍏 Apple Podcast: ⁠Future of UX⁠

💚 Spotify: ⁠Future of UX⁠



Find Patricia on Socials

📸 ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Instagram @ux.patricia⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

🎨 ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Linkedin Patricia Reiners⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

🐦 ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Twitter @ux_Patricia⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

📝 ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠My UX Design Blog ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

🎥 ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Youtube ⁠⁠⁠⁠

Up next
Nov 20
#133 What Hiring Managers Want in 2026 (and What They Don’t)
<p>Design roles are shifting fast and AI is speeding things up. In this episode of <em>Future of UX</em>, we dig into what’s really changing in the UX world, which roles are emerging, and how you can stay ahead in a landscape shaped by automation, agents, and AI tools.</p><p><br> ... Show More
25 m
Nov 13
#132 Understand AI Agents in 20 Minutes
<p>AI Agents are reshaping how we interact with technology — from simple assistants to fully autonomous systems that <em>think and act</em> on their own.<br />In this episode, Patricia breaks down what AI agents really are, how they differ from chatbots, what goes into building t ... Show More
22m 45s
Nov 6
#131 Quarterly AI & UX Update: Vibe-Coding, AI Design Ecosystems & the Rise of Agent Browsers
This episode mixes a few personal updates — balancing work, creativity, and life — with the top three AI trends every designer should know about this season:Vibe-Coding Goes Mainstream: How design teams are using emotional prototyping and AI-generated “vibes” to test concepts and ... Show More
27m 17s
Recommended Episodes
May 2025
50. Responsible AI Use for Research Analysis (feat. Alexander Knoll, Co-Founder of Condens.io)
AI can do more than it’s ever done… but there’s a lot of unfounded hype, especially when it comes to user research. When should you delegate tasks to AI? And when should you insist on keeping the human in the loop? In this episode, Therese Fessenden sits down with Alexander Knoll ... Show More
41m 15s
Jul 2025
Measuring the impact of AI on software engineering – with Laura Tacho
Supported by Our Partners•⁠ Statsig ⁠ — ⁠ The unified platform for flags, analytics, experiments, and more.• Graphite — The AI developer productivity platform.—There’s no shortage of bold claims about AI and developer productivity, but how do you separate signal from noise?In thi ... Show More
1h 11m
Nov 11
Inside Harvey AI’s $8 billion AI lawyer app, PLUS How OpenRouter unites the LLMs | E2207
Register for Founder University Japan’s Kickoff! https://luma.com/cm0x90mkToday’s show:Find out why AI is perfectly suited to legal tasks… despite being too fast for “billable hours”On today’s TWiST, Alex takes a deep dive into LLM Law with Harvey AI co-founder/president Gabe Per ... Show More
1h 3m
Aug 15
Measuring AI code assistants and agents with the AI Measurement Framework
In this episode of Engineering Enablement, DX CTO Laura Tacho and CEO Abi Noda break down how to measure developer productivity in the age of AI using DX’s AI Measurement Framework. Drawing on research with industry leaders, vendors, and hundreds of organizations, they explain ho ... Show More
41m 14s
Sep 26
53. Why You're Not Getting Job Interviews (feat. Hang Xu, Design Recruiter)
What does it really take to get hired in design today? In this episode, design recruiter and former product designer Hang Xu shares hard truths about what’s changed in hiring, why portfolios miss the mark, and how candidates can better align with what employers actually want. He ... Show More
50m 17s
Sep 8
68: Using AI at Work Without Being Replaced with Steff Vanhaverbeke
<p>Chris Daigle sits down with Steff Vanhaverbeke, author of Being Replaced: The Five Human Skills AI Cannot Automate, to explore how professionals and businesses can thrive in an AI-powered world. Steff introduces her Cognitive Agility Framework, highlighting the uniquely human ... Show More
53m 13s
Jan 2025
Navigating the AI Landscape: Google's Strategy
<p> In this episode, Jaeden Schafer discusses Google's current position in the AI landscape, focusing on the significance of the Gemini project for the company's future. He highlights the urgency expressed by CEO Sundar Pichai regarding the need for rapid innovation and adaptatio ... Show More
7m 46s
Feb 2025
OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic)
<p><strong>Karina Nguyen </strong>leads research at OpenAI, where she’s been pivotal in developing groundbreaking products like Canvas, Tasks, and the o1 language model. Before OpenAI, Karina was at Anthropic, where she led post-training and evaluation work for Claude 3 models, c ... Show More
1h 14m
Sep 7
How AI is reshaping the product role | Oji and Ezinne Udezue
Ezinne and Oji Udezue have over 50 years of combined product leadership experience at Microsoft, Twitter, Atlassian, WP Engine, Typeform, and Calendly. They’ve witnessed every major shift in product management, and, despite their seniority, they’re taking beginner AI courses and ... Show More
1h 18m
Apr 2025
Co-Intelligence — Using AI to Think Better, Create More, and Live Smarter
<p>The era of artificially intelligent large language models is upon us and isn't going away. Rather, AI tools like ChatGPT are only going to get better and better and affect more and more areas of human life.</p><p>If you haven't yet felt both amazed and unsettled by these techn ... Show More
57m 46s