About this episode
Apr 7
The AI-First Data Engineer: 10–50x Productivity and What Changes Next
Summary In this episode, I sit down with Gleb Mezhanskiy, CEO and co-founder of Datafold, to explore how agentic AI is reshaping data engineering. We unpack the leap from chat-assisted coding to truly agentic workflows where AI not only writes SQL and dbt models but also executes ... Show More
59m 24s
Mar 29
Treat Metering Like Finance: Building Data Platforms for Consumption Economics
Summary In this episode Himant Goyal, Senior Product Manager at Salesforce, talks about how data platform investments enable reliable, accurate metering for consumption-based business models. Himant explains why consumption turns operations into a real-time optimization problem s ... Show More
50m 19s
Mar 22
Beyond the PDF: Rowan Cockett on Reproducible, Composable Science
Summary In this episode Rowan Cockett, co-founder and CEO of CurveNote and co-founder of the Continuous Science Foundation, talks about building data systems that make scientific research reproducible, reusable, and easier to communicate. He digs into the sociotechnical roots of ... Show More
42m 40s
Aug 2023
2476: ThoughtSpot - How AI Analytics is Redefining Business Intelligence
<p>In the rapidly evolving world of data analytics, staying ahead of the curve is essential. Today on Tech Talks Daily, I'm thrilled to have Sumeet Arora from ThoughtSpot to walk us through their game-changing announcements. ThoughtSpot is already renowned for its advanced analyt ... Show More
33m 55s
Mar 2022
Mining the Golden Age of Data with Tableau’s CEO & President Mark Nelson
<p><a href="https://www.linkedin.com/in/markthomasnelson/">Mark Nelson</a> is the President and CEO of <a href="https://www.tableau.com/">Tableau</a>, a company dedicated to democratizing analytics and putting data back in the hands of consumers. But while this digital pioneer ma ... Show More
36m 32s
Sep 2023
2503: LogicMonitor - Data Observability with Taggart Matthiesen
<p>Today, I dive into the complex and often nebulous world of data observability with a leading expert in the field, Taggart Matthiesen, the Chief Product Officer at LogicMonitor. With an impressive career trajectory that includes pivotal roles at Lyft, Twitter, and Salesforce, T ... Show More
22m 7s
Sep 2021
From Different Leadership Vantage Points: Data Drives Value but is Driven by Values
<p>One way to think about data is that it is like rain, and it is pouring outside. Imagine c-suite executives running around in a parking lot with huge buckets trying to capture as much as they can. Afterward, they return to the office, analyze the data, and then decide what to d ... Show More
51m 50s
Jun 2021
Buying and Selling Homes Algorithmically with Opendoor’s VP of Research and Data Science, Kushal Chakrabarti
<p>For many people, the process of buying and selling a home will undoubtedly be the most difficult decisions they will make in their lifetime. Is the price you’re paying for your home fair? Is the price you’re selling your home for an adequate sale price? For a long time, realto ... Show More
32m 26s
Mar 2022
Bayesian Machine Learning with Ravin Kumar (Ep. 191)
<p>This is one episode where passion for math, statistics and computers are merged.
I have a very interesting conversation with Ravin, data scientist at Google where he uses data to inform decisions.</p>
<p>He has previously worked at Sweetgreen, designing systems that would b ... Show More
31m 12s
Nov 2021
Time Plus Data Equals Efficiency with Paul Dix, the Founder and CTO of InfluxData and the Creator of InfluxDB
<p>If the topic of databases is brought up to certain people, their eyes may gloss over. But if that happened, that would be because they just don’t know the awesome power of databases. Data can be valuable but only if it is contextualized, and time is an extremely relevant aspec ... Show More
36m 4s
Dec 2020
The Algorithms that Bring you Style with Stitch Fix’s Director of Data Science, Tatsiana Maskalevich
<p>The old saying, “look good, feel good,'' fits Stitch Fix perfectly. The direct-to-consumer, online personal styling service has boomed due to its ability to not only match consumers with trendy and comfortable clothes, but to make it a personalized experience for each buyer.</ ... Show More
52m 39s
Summary
Business intelligence is often equated with a collection of dashboards that show various charts and graphs representing data for an organization. What is overlooked in that characterization is the level of complexity and effort that are required to collect and present that information, and the opportunities for providing those insights in other contexts. In this episode Telmo Silva explains how he co-founded ClicData to bring full featured business intelligence and reporting to every organization without having to build and maintain that capability on their own. This is a great conversation about the technical and organizational operations involved in building a comprehensive business intelligence system and the current state of the market.
Announcements
- Hello and welcome to the Data Engineering Podcast, the show about modern data management
- When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With their managed Kubernetes platform it’s now even easier to deploy and scale your workflows, or try out the latest Helm charts from tools like Pulsar and Pachyderm. With simple pricing, fast networking, object storage, and worldwide data centers, you’ve got everything you need to run a bulletproof data platform. Go to dataengineeringpodcast.com/linode today and get a $100 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
- Modern Data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days. Datafold helps Data teams gain visibility and confidence in the quality of their analytical data through data profiling, column-level lineage and intelligent anomaly detection. Datafold also helps automate regression testing of ETL code with its Data Diff feature that instantly shows how a change in ETL or BI code affects the produced data, both on a statistical level and down to individual rows and values. Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows. Go to dataengineeringpodcast.com/datafold today to start a 30-day trial of Datafold.
- Atlan is a collaborative workspace for data-driven teams, like Github for engineering or Figma for design teams. By acting as a virtual hub for data assets ranging from tables and dashboards to SQL snippets & code, Atlan enables teams to create a single source of truth for all their data assets, and collaborate across the modern data stack through deep integrations with tools like Snowflake, Slack, Looker and more. Go to dataengineeringpodcast.com/atlan today and sign up for a free trial. If you’re a data engineering podcast listener, you get credits worth $3000 on an annual subscription
- Your host is Tobias Macey and today I’m interviewing Telmo Silva about ClicData,
Interview
- Introduction
- How did you get involved in the area of data management?
- Can you describe what ClicData is and the story behind it?
- How would you characterize the current state of the market for business intelligence?
- What are the systems/capabilities that are required to run a full-featured BI system?
- What are the challenges that businesses face in developing in-house capacity for business intelligence?
- Can you describe how the ClicData platform is architected?
- How has it changed or evolved since you first began working on it?
- How are you approaching schema design and evolution in the storage layer?
- How do you handle questions of data security/privacy/regulations given that you are storing the information on behalf of the business?
- In your work with clients what are some of the challenges that businesses are facing when attempting to answer questions and gain insights from their data in a repeatable fashion?
- What are some strategies that you have found useful for structuring schemas or dashboards to make iterative exploration of data effective?
- What are the most interesting, innovative, or unexpected ways that you have seen ClicData used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on ClicData?
- When is ClicData the wrong choice?
- What do you have planned for the future of ClicData?
Contact Info
Parting Question
- From your perspective, what is the biggest gap in the tooling or technology for data management today?
Closing Announcements
- Thank you for listening! Don’t forget to check out our other show, Podcast.__init__ to learn about the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com) with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers
Links
The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA
Support Data Engineering Podcast
