Has AI benchmarking reached its limit, and what do we have to fill this gap? Sinan Ozdemir speaks to Jon Krohn about the lack of transparency in training data and the necessity of human-led quality assurance to detect AI hallucinations, when and why to be skeptical of AI benchmarks, and the future of benchmarking agentic and multimodal models.
Additional ... Show More
Oct 7
929: Dragon Hatchling: The Missing Link Between Transformers and the Brain, with Adrian Kosowski
Breaking news: Jon Krohn welcomes Adrian Kosowski to the show to talk about the groundbreaking research happening at Pathway. Adrian and his team demonstrate how they have brought attention in AI closer to the way the brain functions, creating, in essence, a “massively parallel s ... Show More
1h 14m
Oct 3
928: The “Lethal Trifecta”: Can AI Agents Ever Be Safe?
Prompt injections, malicious code, and AI agents: In this week’s Five-Minute Friday, Jon Krohn looks into the current security weaknesses found in AI systems. A structural vulnerability that The Economist dubs a “lethal trifecta” could cause havoc for AI users, unless we take the ... Show More
5m 55s
Sep 30
927: Automating Code Review with AI, feat. CodeRabbit’s David Loker
Earlier this year, David Loker joined CodeRabbit as their Director of AI. As more people come to write code with the help of large language models, David believes CodeRabbit will become a helpful assistant for code reviewing and pull requests. He tells Jon Krohn how CodeRabbit as ... Show More
1h 19m
Aug 2024
Metrics Driven Development
How do you systematically measure, optimize, and improve the performance of LLM applications (like those powered by RAG or tool use)? Ragas is an open source effort that has been trying to answer this question comprehensively, and they are promoting a “Metrics Driven Development” ... Show More
42m 12s
Nov 2024
The Future of AI: Predictions and Realities
In this episode, Jaeden Schafer discusses the current challenges and developments in the AI industry, particularly focusing on the limitations faced by major players like OpenAI and Anthropic. The conversation explores the anticipated improvements in AI models, the predictions fo ... Show More
18m 14s
Apr 2024
Measuring The Speed of AI Through Benchmarks
David Kanter, Executive Director at MLCommons, discusses the work they’re doing with MLPerf Benchmarks, creating the world’s first industry standard approach to measuring AI speed and safety. He also shares ways they’re testing AI and LLMs for harm, to measure—and, over time, red ... Show More
31m 45s
Jul 2023
AI Today Podcast: How AI is Transforming Insurance, Interview with Connor Atchison, Wisedocs
AI is proving transformational in every industry, including long established industries, and insurance is no exception. AI is able to optimize underwriting processes, enable more personalized insurance offerings, enhance the overall customer experience, as well as help with proce ... Show More
30m 19s
Feb 2025
Grok 3: The New AI Challenger
In this episode, Jaeden discusses the launch of Grok 3, the latest AI model from X AI, highlighting its capabilities, training methods, and performance benchmarks compared to competitors like OpenAI's ChatGPT. He shares personal experiences using Grok 3, including its reasoni ... Show More
16m 45s
Feb 2025
The Future of Data Engineering: AI, LLMs, and Automation
Summary
In this episode of the Data Engineering Podcast Gleb Mezhanskiy, CEO and co-founder of DataFold, talks about the intersection of AI and data engineering. He discusses the challenges and opportunities of integrating AI into data engineering, particularly using large langua ... Show More
59m 39s