About this episode
Aug 27
Specialized AI for Data Engineers: Inside Astronomer’s Otto
Summary In this episode Yetunde Dada discusses Otto, Astronomer’s AI agent for Airflow, and the broader challenge of making agentic tooling actually useful for data engineers. She explored why generic coding assistants often fall short in data workflows, how Otto adds the missing ... Show More
45m 30s
Aug 2
Why Multi-Agent Systems Need Shared State, Graph Semantics, and Governance
Summary In this episode Ragnor Comerford talks about OmniGraph, a lakehouse-native graph storage layer designed around the needs of agentic systems. He explores how graphs are primarily a semantic model for representing the world, rather than just a specialized engine for travers ... Show More
1h 2m
Jul 6
Building the Context Flywheel for AI Data Agents
Summary In this episode Prukalpa Sankar, co-founder of Atlan, talks about what it takes to build a “context flywheel” for AI agents in data-intensive organizations. She explained why model intelligence alone isn’t enough to make AI useful in production, and how real performance d ... Show More
1 h
Jan 2024
Fivetran COO Unravels Enterprise Data Movement
Automating the collection of dispersed, divergent and disjointed pools of enterprise data on to a single repository to drive analytics and build applications remains complex. In this edition of Bloomberg Intelligence’s Tech Disruptors podcast, Fivetran cofounder and COO Taylor Br ... Show More
40m 15s
Feb 2025
How Can GenAI Make Analytics More Accessible to Product Teams? (with Mario Ciabarra)
<p>Whether you prefer the term data-driven, or data-informed, or data-dazzled, it doesn't matter—today's tech cannot survive without high quality data sets AND the tools to use them effectively. But we also can't afford to think about data as the responsibility of ... Show More
27m 46s
May 2022
How to Link Data to Business Outcomes
<p><span style="font-weight: 400;">Every business likes to claim that it is "data-driven" or at least "data-informed," but too often, that's not the way things actually work. Data is relegated to an IT function, siloed and, in some cases, boils down to simply producing more repor ... Show More
17m 32s
Sep 2024
Data for Dummies: A Crash Course for Non-Technical PMs (with Mo Hallaba)
<p>In today's data-driven landscape, organizations often find themselves drowning in a sea of data, yet struggling to glean actionable insights from it. Many companies are eager to label themselves as data-centric, but the reality is that not everyone is equally adept at int ... Show More
23m 23s
Oct 2025
Too much data, not enough insight
Companies have access to more of their data than ever before. But using it to inform real-time decision-making remains a challenge. Chip Kleinheksel talks to Skyworks Solutions’ CIO, Satya Jayadev, and Deloitte's Som Suresh, about how leading companies are turning data into reven ... Show More
31m 37s
Nov 2022
How can You Become a Data Hero -- Aron Clymer //Data Clymer
<p>Aron Clymer, Founder and CEO of Data Clymer, talks about data and analytics. Nowadays, companies are running their entire firm on 20 to sometimes even 100 different SaaS applications. If these companies truly want to understand their customers and everything they’re doing, the ... Show More
17m 5s
May 2019
263: Communicating Data
In this episode of the SuperDataScience Podcast, I chat with Eoin Murray, the founder of Kyso.io, a platform where you can blog about your data science projects using tools such as Jupyter notebooks. You will learn what the platform means for data scientists and how you can use i ... Show More
1h 1m
Aug 2025
Amperity Reimagines Data and Developer Workflows with AI - Ep. 271
Derek Slager, co-founder and CTO of Amperity, explores how agentic AI and vibe coding are reshaping enterprise data management and the developer experience on the NVIDIA AI Podcast. Hear how Amperity’s platform unifies customer data, powers advanced analytics, and brings conversa ... Show More
36m 40s
May 2022
How to Use Data in the PM Role by LinkedIn Product Leader
<p>LinkedIn Product Leader <a href='https://productschool.com/product-leaders/Ming-Chang?utm_campaign=202205-mkt_podcast_episode-ming_chang&utm_medium=social&utm_source=buzzsprout&utm_term=b2c_current_pm'>Ming Chang</a> believes that one of the best PM superpowers is ... Show More
32m 2s
Dec 2023
LLMs for Data Analysis
In this episode, we are joined by Amir Netz, a Technical Fellow at Microsoft and the CTO of Microsoft Fabric. He discusses how companies can use Microsoft's latest tools for business intelligence. Amir started by discussing how business intelligence has progressed in relevance ov ... Show More
29 m
Summary
Every organization needs to be able to use data to answer questions about their business. The trouble is that the data is usually spread across a wide and shifting array of systems, from databases to dashboards. The other challenge is that even if you do find the information you are seeking, there might not be enough context available to determine how to use it or what it means. Castor is building a data discovery platform aimed at solving this problem, allowing you to search for and document details about everything from a database column to a business intelligence dashboard. In this episode CTO Amaury Dumoulin shares his perspective on the complexity of letting everyone in the company find answers to their questions and how Castor is designed to help.
Announcements
- Hello and welcome to the Data Engineering Podcast, the show about modern data management
- You listen to this show to learn about all of the latest tools, patterns, and practices that power data engineering projects across every domain. Now there’s a book that captures the foundational lessons and principles that underly everything that you hear about here. I’m happy to announce I collected wisdom from the community to help you in your journey as a data engineer and worked with O’Reilly to publish it as 97 Things Every Data Engineer Should Know. Go to dataengineeringpodcast.com/97things today to get your copy!
- When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With their managed Kubernetes platform it’s now even easier to deploy and scale your workflows, or try out the latest Helm charts from tools like Pulsar and Pachyderm. With simple pricing, fast networking, object storage, and worldwide data centers, you’ve got everything you need to run a bulletproof data platform. Go to dataengineeringpodcast.com/linode today and get a $100 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
- Are you bored with writing scripts to move data into SaaS tools like Salesforce, Marketo, or Facebook Ads? Hightouch is the easiest way to sync data into the platforms that your business teams rely on. The data you’re looking for is already in your data warehouse and BI tools. Connect your warehouse to Hightouch, paste a SQL query, and use their visual mapper to specify how data should appear in your SaaS systems. No more scripts, just SQL. Supercharge your business teams with customer data using Hightouch for Reverse ETL today. Get started for free at dataengineeringpodcast.com/hightouch.
- Have you ever had to develop ad-hoc solutions for security, privacy, and compliance requirements? Are you spending too much of your engineering resources on creating database views, configuring database permissions, and manually granting and revoking access to sensitive data? Satori has built the first DataSecOps Platform that streamlines data access and security. Satori’s DataSecOps automates data access controls, permissions, and masking for all major data platforms such as Snowflake, Redshift and SQL Server and even delegates data access management to business users, helping you move your organization from default data access to need-to-know access. Go to dataengineeringpodcast.com/satori today and get a $5K credit for your next Satori subscription.
- Your host is Tobias Macey and today I’m interviewing Amaury Dumoulin about Castor, a managed platform for easy data cataloging and discovery
Interview
- Introduction
- How did you get involved in the area of data management?
- Can you describe what Castor is and the story behind it?
- The market for data catalogues is nascent but growing fast. What are the broad categories for the different products and projects in the space?
- What do you see as the core features that are required to be competitive?
- In what ways has that changed in the past 1 – 2 years?
- What are the opportunities for innovation and differentiation in the data catalog/discovery ecosystem?
- How do you characterize your current position in the market?
- Who are the target users of Castor?
- Can you describe the technical architecture and implementation of the Castor platform?
- How have the goals and design changed since you first began working on it?
- Can you talk through the workflow of getting Castor set up in an organization and onboarding the users?
- What are the design elements and platform features that allow for serving the various roles and stakeholders in an organization?
- What are the organizational benefits that you have seen from users adopting Castor or other data discovery/catalog systems?
- What are the most interesting, innovative, or unexpected ways that you have seen Castor used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on Castor?
- When is Castor the wrong choice?
- What do you have planned for the future of Castor?
Contact Info
Parting Question
- From your perspective, what is the biggest gap in the tooling or technology for data management today?
Links
The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA
Support Data Engineering Podcast
