logo
episode-header-image
Dec 2017
20m 37s

[MINI] Parallel Algorithms

Kyle Polich
About this episode

When computers became commodity hardware and storage became incredibly cheap, we entered the era of so-call "big" data. Most definitions of big data will include something about not being able to process all the data on a single machine. Distributed computing is required for such large datasets.

Getting an algorithm to run on data spread out over a variety of different machines introduced new challenges for designing large-scale systems. First, there are concerns about the best strategy for spreading that data over many machines in an orderly fashion. Resolving ambiguity or disagreements across sources is sometimes required.

This episode discusses how such algorithms related to the complexity class NC.

Up next
Nov 23
Designing Recommender Systems for Digital Humanities
<p>In this episode of Data Skeptic, we explore the fascinating intersection of recommender systems and digital humanities with guest Florian Atzenhofer-Baumgartner, a PhD student at Graz University of Technology. Florian is working on <a href= "http://monasterium.net/">Monasteriu ... Show More
36m 48s
Nov 13
DataRec Library for Reproducible in Recommend Systems
<p>In this episode of Data Skeptic's Recommender Systems series, host Kyle Polich explores DataRec, a new Python library designed to bring reproducibility and standardization to recommender systems research. Guest Alberto Carlo Maria Mancino, a postdoc researcher from Politecnico ... Show More
32m 48s
Nov 5
Shilling Attacks on Recommender Systems
In this episode of Data Skeptic's Recommender Systems series, Kyle sits down with Aditya Chichani, a senior machine learning engineer at Walmart, to explore the darker side of recommendation algorithms. The conversation centers on shilling attacks—a form of manipulation where mal ... Show More
34m 48s
Recommended Episodes
Jul 2020
A Reality Check on AI-Driven Medical Assistants
The data science and artificial intelligence community has made amazing strides in the past few years to algorithmically automate portions of the healthcare process. This episode looks at two computer vision algorithms, one that diagnoses diabetic retinopathy and another that cla ... Show More
14 m
Sep 2018
Data Engineering
If you’re a data scientist, you know how important it is to keep your data orderly, clean, moving smoothly between different systems, well-documented… there’s a ton of work that goes into building and maintaining databases and data pipelines. This job, that of owner and maintaine ... Show More
16m 22s
Jul 2023
Tech and AI: 8. The Algorithm
At its simplest, an algorithm is a sequence of step-by-step instructions designed to give a result. They are the building blocks of every computer program and are there to ensure every digital device gives the right results on request. For example, when we type a search query int ... Show More
13m 54s
Sep 2020
Marcello La Rocca on Algorithms and Data Structures
<p><a href="https://twitter.com/qethanm" target="_blank">Q McCallum</a> (Senior Content Advisor at <a href="https://formulated.by/" target="_blank">Formulatedby</a>, the company behind <a href="https://www.datascience.salon/" target="_blank">Data Science Salon</a>) met up with <a ... Show More
1h 2m
Jan 2022
Making Agile work for data science
<p>Data scientists and engineers don’t always play well together. Data scientists will plan out a solution, carefully build models, test them in notebooks, then throw that solution over the wall to engineering. Implementing that solution can take months.</p><p>Historically, the d ... Show More
20m 52s
Feb 2021
The promise of quantum computers | Matt Langione
<p>What if microparticles could help us solve the world's biggest problems in a matter of minutes? That's the promise -- and magic -- of quantum computers, says Matt Langione. Speaking next to IBM's quantum computer, he explains how these machines solve complex challenges like de ... Show More
12m 49s
May 2020
How Important are algorithm and data structures in backend engineering?
<p>Algorithms &amp; Data Structures are critical to Backend Engineering however it really depends on what kind of application and infrastructure you are building. In this video I want to go through the following &nbsp;&nbsp;1 Backend Engineers are two types - Integrating Existing ... Show More
13m 29s