#454: Data Pipelines with Dagster

Today we’re turning tiny tips into big wins. Khuyen Tran, creator of CodeCut.ai, has shipped hundreds of bite-size Python and data science snippets across four years. We dig into open-source tools you can use right now, cleaner workflows, and why notebooks and scripts don’t have ... Show More

English is now an API. Our apps read untrusted text; they follow instructions hidden in plain sight, and sometimes they turn that text into action. If you connect a model to tools or let it read documents from the wild, you have created a brand new attack surface. In this episode ... Show More

A couple years ago, Charlie Marsh lit a fire under Python tooling with Ruff and then uv. Today he’s back with something on the other side of that coin: pyx. Pyx isn’t a PyPI replacement. Think server, not just index. It mirrors PyPI, plays fine with pip or uv, and aims to make in ... Show More

Summary Python has beome the de facto language for working with data. That has brought with it a number of challenges having to do with the speed and scalability of working with large volumes of information.There have been many projects and strategies for overcoming these challen ... Show More

Modern networked applications generate a lot of data, and every business wants to make the most of that data. Most of the time, that means moving production data through some transformation process to get it ready for the analytics process. But what if you could have in-app analy ... Show More

Summary A core differentiator of Dagster in the ecosystem of data orchestration is their focus on software defined assets as a means of building declarative workflows. With their launch of Dagster+ as the redesigned commercial companion to the open source project they are investi ... Show More

Summary Maintaining a single source of truth for your data is the biggest challenge in data engineering. Different roles and tasks in the business need their own ways to access and analyze the data in the organization. In order to enable this use case, while maintaining a single ... Show More

Organizations use their data to make better decisions and build innovative experiences for their customers. With the exponential growth in data, and the rapid pace of innovation in machine learning (ML), there is a growing need to build modern data applications that are agile and ... Show More

This episode begins with a rant. This rant, in particular, comes from Dan Engber - a science writer who loves animals but despises animal intelligence research. Dan told us that so much of the way we study animals involves tests that we think show a human is smart ... not the ani ... Show More

The Transformation Ground Control podcast covers a number of topics important to digital and business transformation. This episode covers the following topics and interviews: Microsoft’s New Direction with Copilot, Q&A (Darian Chwialkowski, Third Stage Consulting) Data Management ... Show More

Introducing a brand-new daily podcast from the team at WCH. On This Day in Working Class History will be a brief reminder each morning of our collective struggles for a better world which have taken place on this date in history. Launching on 1 February on a trial basis, each epi ... Show More

🚀 Dive into the world of AI with Morgan Willis, Principal Cloud Technologist for AWS, as she interviews Clare Liguori, a Senior Principal Software Engineer at AWS and one of the visionaries behind Amazon Q. Discover the secrets behind this groundbreaking Generative AI conversati ... Show More