episode-header-image

Apr 2020

21m 19s

When should you shard your database?

About this episode

Database Application level sharding is the process of splitting a table into multiple database instances in order to distribute the load. However, Sharding a database is an expensive operation (maintainability and overhead) and I suggest you do that only when you absolutely need to. That means when your single instance database can no longer serve queries with minimum latency. So I suggest you monitor that and only shard if necessary.. I rather do replication make master / backup and make requests read from replica than Sharding just because it is easier.. good question... nice idea for a video

5:00 Partition horizontally

7:45 Replication (Master/backup) Scale reads

11:00 Scale writes by region

12:30 Sharding

16:40 Sharding in YouTube

Up next

CPU and Kernel Page Faults

<p>Page faults occurs when the process tries to access a memory that isn’t backed by a physical page kernel raises a fault which loads a page. It happens on first access, stack expansion, COW, swap and much more. However it comes with a cost. </p><p><br /></p><p>In this episode o ... Show More

Amazon US-EAST-1 Outage in Details

On October 19 2025 AWS experienced an outage that lasted over a day, 10 days later we finally got the root cause analysis and we know exactly what caused the DNS to fail0:00 Summary 5:30 How did Dynamo lost its DNS?13:41 EC2 Errors 16:16 Network Load Balancer ErrorsRCA here https ... Show More

Graceful shutdown in HTTP

There are cases where the backend may need to close the connection to prevent unexpected situations, prevent bad actors or simply just free up resources. Closing a connection gracefully allows clients and backends to clean up and finish any pending requests. In this episode of th ... Show More

Recommended Episodes

Postgres Replication at speed with PeerDB's Sai Srirampur

<p>Join Scott Hanselman as he sits down with Sai Srirampur, as they explore the intricacies of Postgres Replication and how it can be turbocharged using PeerDB. In the realm of databases, Postgres Replication is like a well-choreographed dance between servers. Sai takes us behind ... Show More

Easier Stream Processing On Kafka With ksqlDB

<div class="wp-block-jetpack-markdown"><h2>Summary</h2> <p>Building applications on top of unbounded event streams is a complex endeavor, requiring careful integration of multiple disparate systems that were engineered in isolation. The ksqlDB project was created to address this ... Show More

Episode 507: Stephen Margheim on SQLite in Ruby on Rails Applications

If you start a new Rails project today, you’re probably going to pick Postgres for your database. And if you have a ten year old Rails app, you might be using MySQL. But Rails ships with support for SQLite, a lightweight SQL database. Can that lightweight database be used to make ... Show More

#420: Database Consistency & Isolation for Python Devs

See the full show notes for this episode on the website at <a href="https://talkpython.fm/420">talkpython.fm/420</a>

Accelerating ML Training And Delivery With In-Database Machine Learning

<div class="wp-block-jetpack-markdown"><h2>Summary</h2> <p>When you build a machine learning model, the first step is always to load your data. Typically this means downloading files from object storage, or querying a database. To speed up the process, why not build the model in ... Show More

S17:E9 - What are some database architectures and their use cases (Kyle Bernhardy)

In this episode, we talk about database architectures and some of their use cases, with Kyle Bernhardy, CTO of HarperDB. Kyle talks about what a database is, different types of databases, and when you might want to use one type of database over another. Show Links DevDiscuss (spo ... Show More

Writing data is easy. You take in the information and put it away for future use. It’s remembering exactly what you wrote and where you put it that’s the challenge. Just like having to look for your keys as you try to rush out the door, getting that data quickly makes all the dif ... Show More

Listen to millions of songs and podcasts on Anghami