logo
episode-header-image
May 2022
1 h

Take A Deep Dive On How Code Completion ...

Tobias Macey
About this episode

Summary

Most developers have encountered code completion systems and rely on them as part of their daily work. They allow you to stay in the flow of programming, but have you ever stopped to think about how they work? In this episode Meredydd Luff takes us behind the scenes to dig into the mechanics of code completion engines and how you can customize them to fit your particular use case.

Announcements

  • Hello and welcome to Podcast.__init__, the podcast about Python’s role in data and science.
  • When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With the launch of their managed Kubernetes platform it’s easy to get started with the next generation of deployment and scaling, powered by the battle tested Linode platform, including simple pricing, node balancers, 40Gbit networking, dedicated CPU and GPU instances, and worldwide data centers. Go to pythonpodcast.com/linode and get a $100 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
  • Your host as usual is Tobias Macey and today I’m interviewing Meredydd Luff about how code completion works and what it takes to build your own

Interview

  • Introductions
  • How did you get introduced to Python?
  • Most programmers are familiar with the idea of code completion, but can you just give the elevator pitch to get us all on the same page?
  • You gave a presentation recently at PyCon about how to build a code completion system. What was your approach to identifying what fundamental concepts needed to be addressed and how to fit that lesson into the available time?
  • In the presentation you mentioned that you had built a more full-featured completion engine into Anvil. Can you describe what possessed you to build your own code completion tool?
    • What are the core components required to build a completion engine?
    • What are the benefits that can be realized by customizing the completion engine for a given language or task?
  • Can you describe the feature set and implementation details of the full-fledged completion engine that is available in Anvil?
  • Beyond the toy example, there are a number of considerations to address if you want to make the completion engine "production grade". Can you talk through some of the obvious edge cases and how to solve for them? (e.g. handling parsing of incomplete code)
  • What are the inputs that you use to build up the list of candidate tokens for completion?
  • Once you have a functioning baseline for offering completions, what are some of the signals that you hook into for ranking suggestions?
  • In your presentation you leaned on the machinery available in the Python standard library. What are some of the ways that you might think about generalizing across languages vs. coupling to a given language?
  • What design/architectural advice do you have for compartmentalizing logic in a full-featured completion engine?
  • What are some of the complexities that become a factor when you are trying to scale across an entire code base?
  • Beyond just being able to parse and process a body of code, there is also the question of integrating with the development environment. What are some of the challenges that get introduced when trying to access the appropriate set(s) of files and code through the editor interface(s)?
  • What are the most interesting, innovative, or unexpected ways that you have seen code completion applied to developer experience?
  • What are the most interesting, unexpected, or challenging lessons that you have learned while working on code completion for Anvil?
  • When is code completion more effort than it’s worth?
  • What do you have planned for the future of the Anvil code completion functionality?

Keep In Touch

Picks

Closing Announcements

  • Thank you for listening! Don’t forget to check out our other show, the Data Engineering Podcast for the latest on modern data management.
  • Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
  • If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@podcastinit.com) with your story.
  • To help other people find the show please leave a review on iTunes and tell your friends and co-workers

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Up next
Dec 2022
Update Your Model's View Of The World In Real Time With Streaming Machine Learning Using River
Preamble This is a cross-over episode from our new show The Machine Learning Podcast, the show about going from idea to production with machine learning. Summary The majority of machine learning projects that you read about or work on are built around batch processes. The model i ... Show More
1h 16m
Dec 2022
Declarative Machine Learning For High Performance Deep Learning Models With Predibase
Preamble This is a cross-over episode from our new show The Machine Learning Podcast, the show about going from idea to production with machine learning. Summary Deep learning is a revolutionary category of machine learning that accelerates our ability to build powerful inference ... Show More
59m 22s
Nov 2022
Build Better Machine Learning Models With Confidence By Adding Validation With Deepchecks
Preamble This is a cross-over episode from our new show The Machine Learning Podcast, the show about going from idea to production with machine learning. Summary Machine learning has the potential to transform industries and revolutionize business capabilities, but only if the mo ... Show More
47m 37s
Recommended Episodes
Feb 2025
#495: OSMnx: Python and OpenStreetMap
On this episode, I’m joined by Dr. Jeff Boeing, an assistant professor at the University of Southern California whose research spans urban planning, spatial analysis, and data science. We explore why OpenStreetMap is such a powerful source of global map data—and how Jeff’s Python ... Show More
1h 1m
Sep 2021
An Exploration Of The Data Engineering Requirements For Bioinformatics
Summary Biology has been gaining a lot of attention in recent years, even before the pandemic. As an outgrowth of that popularity, a new field has grown up that pairs statistics and compuational analysis with scientific research, namely bioinformatics. This brings with it a uniqu ... Show More
55m 10s
May 2022
Insights And Advice On Building A Data Lake Platform From Someone Who Learned The Hard Way
Summary Designing a data platform is a complex and iterative undertaking which requires accounting for many conflicting needs. Designing a platform that relies on a data lake as its central architectural tenet adds additional layers of difficulty. Srivatsan Sridharan has had the ... Show More
58m 11s
Mar 2021
Data Quality Management For The Whole Team With Soda Data
Summary Data quality is on the top of everyone’s mind recently, but getting it right is as challenging as ever. One of the contributing factors is the number of people who are involved in the process and the potential impact on the business if something goes wrong. In this episod ... Show More
58 m
Aug 2024
The Evolution of DataOps: Insights from DataKitchen's CEO
Summary In this episode of the Data Engineering Podcast, host Tobias Macey welcomes back Chris Berg, CEO of DataKitchen, to discuss his ongoing mission to simplify the lives of data engineers. Chris explains the challenges faced by data engineers, such as constant system failures ... Show More
53m 30s
Feb 2025
The Future of Data Engineering: AI, LLMs, and Automation
Summary In this episode of the Data Engineering Podcast Gleb Mezhanskiy, CEO and co-founder of DataFold, talks about the intersection of AI and data engineering. He discusses the challenges and opportunities of integrating AI into data engineering, particularly using large langua ... Show More
59m 39s
Feb 2024
Using Trino And Iceberg As The Foundation Of Your Data Lakehouse
Summary A data lakehouse is intended to combine the benefits of data lakes (cost effective, scalable storage and compute) and data warehouses (user friendly SQL interface). Multiple open source projects and vendors have been working together to make this vision a reality. In this ... Show More
58m 46s
Aug 2018
258: A Foot in the Door
This week, we debut the new show format! First, Marshall formally introduces himself, and we answer a listener's question about how to get their foot in the UX door. Then we cover a few headlines, fight about stock vs. third-party apps, and share a couple cool things. If you have ... Show More
38m 51s
Aug 2019
Building Tools And Platforms For Data Analytics
Summary Data engineers are responsible for building tools and platforms to power the workflows of other members of the business. Each group of users has their own set of requirements for the way that they access and interact with those platforms depending on the insights they are ... Show More
48m 7s
Dec 2024
The Art of Database Selection and Evolution
Summary In this episode of the Data Engineering Podcast Sam Kleinman talks about the pivotal role of databases in software engineering. Sam shares his journey into the world of data and discusses the complexities of database selection, highlighting the trade-offs between differen ... Show More
59m 56s