logo
episode-header-image
Mar 2024
58m 25s

#454: Data Pipelines with Dagster

MICHAEL KENNEDY
About this episode
Do you have data that you pull from external sources or is generated and appears at your digital doorstep? I bet that data needs processed, filtered, transformed, distributed, and much more. One of the biggest tools to create these data pipelines with Python is Dagster. And we are fortunate to have Pedram Navid on the show this episode. Pedram is the Head of ... Show More
Up next
Jun 17
#552: Astral joins OpenAI
OpenAI just acquired Astral, the company behind uv, Ruff, and ty. And if your first thought was "wait, is uv toast?", you are not alone. But here's the twist Charlie Marsh shared with me: he thinks they may ship more open source at OpenAI than they ever did at Astral. On this epi ... Show More
1h 5m
Jun 11
#551: Stroll Down Startup Lane - 2026
If you've ever been to PyCon, you know one of the best parts of the expo hall is Startup Row, a stretch of booths where early-stage companies built on Python show off what they're creating. But only attendees get to walk that lane, so let's bring it to everyone. In this episode, ... Show More
1h 48m
May 30
#550: AI Contributions and Maintainer Load in Open Source
You wake up, brew the coffee, open GitHub, and there it is. Another pull request on your open source project. Thirteen thousand lines added. No issue filed first. No discussion. Just "here, please review this for me." Over the past year, GitHub activity has spiked roughly twelve ... Show More
1h 2m
Recommended Episodes
Sep 2021
Massively Parallel Data Processing In Python Without The Effort Using Bodo
<div class="wp-block-jetpack-markdown"><h2>Summary</h2> <p>Python has beome the de facto language for working with data. That has brought with it a number of challenges having to do with the speed and scalability of working with large volumes of information.There have been many ... Show More
1h 4m
Feb 2023
Shorten the distance between production data and insight
Modern networked applications generate a lot of data, and every business wants to make the most of that data. Most of the time, that means moving production data through some transformation process to get it ready for the analytics process. But what if you could have in-app analy ... Show More
20m 27s
Mar 2024
Ship Smarter Not Harder With Declarative And Collaborative Data Orchestration On Dagster+
<h2>Summary</h2> <p>A core differentiator of Dagster in the ecosystem of data orchestration is their focus on software defined assets as a means of building declarative workflows. With their launch of Dagster+ as the redesigned commercial companion to the open source project t ... Show More
55m 40s
Apr 2024
Establish A Single Source Of Truth For Your Data Consumers With A Semantic Layer
<h2>Summary</h2> <p>Maintaining a single source of truth for your data is the biggest challenge in data engineering. Different roles and tasks in the business need their own ways to access and analyze the data in the organization. In order to enable this use case, while mainta ... Show More
56m 23s
Oct 2023
#628: Data on EKS
Organizations use their data to make better decisions and build innovative experiences for their customers. With the exponential growth in data, and the rapid pace of innovation in machine learning (ML), there is a growing need to build modern data applications that are agile and ... Show More
20m 56s
Feb 2024
G: The World's Smartest Animal
<p>This episode begins with a rant. This rant, in particular, comes from Dan Engber - a science writer who loves animals but despises animal intelligence research. Dan told us that so much of the way we study animals involves tests that we think show a human is smart ... not the ... Show More
50m 20s
Jan 2024
Introducing On This Day in Working Class History: A new daily podcast from WCH
Introducing a brand-new daily podcast from the team at WCH. On This Day in Working Class History will be a brief reminder each morning of our collective struggles for a better world which have taken place on this date in history.<br />Launching on 1 February on a trial basis, eac ... Show More
2m 32s
Feb 2024
Episode 108 - Diving into Amazon Q Builder with Clare Liguori
🚀 Dive into the world of AI with Morgan Willis, Principal Cloud Technologist for AWS, as she interviews Clare Liguori, a Senior Principal Software Engineer at AWS and one of the visionaries behind Amazon Q. Discover the secrets behind this groundbreaking Generative AI conversati ... Show More
48m 6s