episode-header-image

Apr 2020

21m 19s

When should you shard your database?

About this episode

Database Application level sharding is the process of splitting a table into multiple database instances in order to distribute the load. However, Sharding a database is an expensive operation (maintainability and overhead) and I suggest you do that only when you absolutely need to. That means when your single instance database can no longer serve queries with minimum latency. So I suggest you monitor that and only shard if necessary.. I rather do replication make master / backup and make requests read from replica than Sharding just because it is easier.. good question... nice idea for a video

5:00 Partition horizontally

7:45 Replication (Master/backup) Scale reads

11:00 Scale writes by region

12:30 Sharding

16:40 Sharding in YouTube

--- Support this podcast: https://anchor.fm/hnasr/support

Up next

kTLS - Kernel level TLS

Fundamentals of Operating Systems Course https://oscourse.winktls is brilliant.TLS encryption/decryption often happens in userland. While TCP lives in the kernel. With ktls, userland can hand the keys to the kernel and the kernel does crypto. When calling write, the kernel encryp ... Show More

The beauty of the CPU

If you are bored of contemporary topics of AI and need a breather, I invite you to join me to explore a mundane, fundamental and earthy topic.The CPU.A reading of my substack article https://hnasr.substack.com/p/the-beauty-of-the-cpu

Sequential Scans in Postgres just got faster

This new PostgreSQL 17 feature is game changer. They know can combine IOs when performing sequential scan. Grab my database coursehttps://courses.husseinnasser.com

Recommended Episodes

Postgres Replication at speed with PeerDB's Sai Srirampur

Join Scott Hanselman as he sits down with Sai Srirampur, as they explore the intricacies of Postgres Replication and how it can be turbocharged using PeerDB. In the realm of databases, Postgres Replication is like a well-choreographed dance between servers. Sai takes us behind th ... Show More

Easier Stream Processing On Kafka With ksqlDB

Summary Building applications on top of unbounded event streams is a complex endeavor, requiring careful integration of multiple disparate systems that were engineered in isolation. The ksqlDB project was created to address this state of affairs by building a unified layer on top ... Show More

Episode 507: Stephen Margheim on SQLite in Ruby on Rails Applications

If you start a new Rails project today, you’re probably going to pick Postgres for your database. And if you have a ten year old Rails app, you might be using MySQL. But Rails ships with support for SQLite, a lightweight SQL database. Can that lightweight database be used to make ... Show More

#420: Database Consistency & Isolation for Python Devs

See the full show notes for this episode on the website at talkpython.fm/420

Accelerating ML Training And Delivery With In-Database Machine Learning

Summary When you build a machine learning model, the first step is always to load your data. Typically this means downloading files from object storage, or querying a database. To speed up the process, why not build the model inside the database so that you don’t have to move the ... Show More

S17:E9 - What are some database architectures and their use cases (Kyle Bernhardy)

In this episode, we talk about database architectures and some of their use cases, with Kyle Bernhardy, CTO of HarperDB. Kyle talks about what a database is, different types of databases, and when you might want to use one type of database over another. Show Links DevDiscuss (spo ... Show More

Writing data is easy. You take in the information and put it away for future use. It’s remembering exactly what you wrote and where you put it that’s the challenge. Just like having to look for your keys as you try to rush out the door, getting that data quickly makes all the dif ... Show More

Listen to millions of songs and podcasts on Anghami