logo
episode-header-image
Dec 2017
20m 37s

[MINI] Parallel Algorithms

Kyle Polich
About this episode

When computers became commodity hardware and storage became incredibly cheap, we entered the era of so-call "big" data. Most definitions of big data will include something about not being able to process all the data on a single machine. Distributed computing is required for such large datasets.

Getting an algorithm to run on data spread out over a variety of different machines introduced new challenges for designing large-scale systems. First, there are concerns about the best strategy for spreading that data over many machines in an orderly fashion. Resolving ambiguity or disagreements across sources is sometimes required.

This episode discusses how such algorithms related to the complexity class NC.

Up next
Jul 6
The Network Diversion Problem
In this episode, Professor Pål Grønås Drange from the University of Bergen, introduces the field of Parameterized Complexity - a powerful framework for tackling hard computational problems by focusing on specific structural aspects of the input. This framework allows researchers ... Show More
46m 14s
Jun 28
Complex Dynamic in Networks
In this episode, we learn why simply analyzing the structure of a network is not enough, and how the dynamics - the actual mechanisms of interaction between components - can drastically change how information or influence spreads. Our guest, Professor Baruch Barzel of Bar-Ilan Un ... Show More
56 m
Jun 22
Github Network Analysis
In this episode we'll discuss how to use Github data as a network to extract insights about teamwork. Our guest, Gabriel Ramirez, manager of the notifications team at GitHub, will show how to apply network analysis to better understand and improve collaboration within his enginee ... Show More
36m 46s
Recommended Episodes
Jul 2020
A Reality Check on AI-Driven Medical Assistants
The data science and artificial intelligence community has made amazing strides in the past few years to algorithmically automate portions of the healthcare process. This episode looks at two computer vision algorithms, one that diagnoses diabetic retinopathy and another that cla ... Show More
14 m
Oct 2023
Quantum algorithms make clever use of noisy hardware
While quantum computers show great promise for the future, today’s processors are small and noisy – and this makes it very difficult to do meaningful quantum calculations right now. To address this problem, researchers are developing clever quantum algorithms that make the most o ... Show More
33m 16s
Sep 2018
Data Engineering
If you’re a data scientist, you know how important it is to keep your data orderly, clean, moving smoothly between different systems, well-documented… there’s a ton of work that goes into building and maintaining databases and data pipelines. This job, that of owner and maintaine ... Show More
16m 22s
Jul 2023
Tech and AI: 8. The Algorithm
At its simplest, an algorithm is a sequence of step-by-step instructions designed to give a result. They are the building blocks of every computer program and are there to ensure every digital device gives the right results on request. For example, when we type a search query int ... Show More
13m 54s
Sep 2020
Marcello La Rocca on Algorithms and Data Structures
Q McCallum (Senior Content Advisor at Formulatedby, the company behind Data Science Salon) met up with Marcello La Rocca, someone who compiled his extensive knowledge of algorithms into a rather hefty book on the topic.  In this episode, the author of Algorithms and Data Structur ... Show More
1h 2m
Jul 2018
#29 Machine Learning & Data Science at Github
Omoju Miller, a Senior Machine Learning Data Scientist with Github, speaks with Hugo about the role of data science in product development at github, what it means to “use computation to build products to solve real-life decision making, practical challenges” and what building da ... Show More
59m 23s
Jan 2022
Making Agile work for data science
Data scientists and engineers don’t always play well together. Data scientists will plan out a solution, carefully build models, test them in notebooks, then throw that solution over the wall to engineering. Implementing that solution can take months.Historically, the data scienc ... Show More
20m 52s
Feb 2021
The promise of quantum computers | Matt Langione
What if microparticles could help us solve the world's biggest problems in a matter of minutes? That's the promise -- and magic -- of quantum computers, says Matt Langione. Speaking next to IBM's quantum computer, he explains how these machines solve complex challenges like devel ... Show More
12m 49s
May 2020
How Important are algorithm and data structures in backend engineering?
Algorithms & Data Structures are critical to Backend Engineering however it really depends on what kind of application and infrastructure you are building. In this video I want to go through the following   1 Backend Engineers are two types - Integrating Existing Backend  - Core ... Show More
13m 29s