logo
episode-header-image
Feb 2023
34m 4s

Postgres Architecture | The Backend Engi...

Hussein Nasser
About this episode

Creating a listener on the backend application that accepts connections is simple. You listen on an address-port pair, connection attempts to that address and port will get added to an accept queue; The application accepts connections from the queue and start reading the data stream sent on the connection.

However, what part of your application does the accepting and what part does the reading and what part does the execution? You can architect your application in many ways based on your use cases. I have a medium post just exploring the different options.

In this video I explore the PostgreSQL process architecture in details. Please note that the information here is derived from both the Postgres doc and code. Discussions about scalability and performance are solely based on my opinions.

0:00 Intro

1:30 Overview

3:30 Postgres MVCC

5:30 Processes vs Threads

7:40 Postmaster Process

8:00 Backend Processes

13:30 Shared Buffers

14:52 Background Workers

17:18 Auxiliary Processes

17:45 Background Writer

22:30 Checkpointer

23:40 Logger

24:06 Autovacuum Launcher and Workers

25:30 WAL Processes

28:53 Startup Process

Read full article

https://medium.com/@hnasr/postgresql-process-architecture-f21e16459907

Up next
Nov 24
CPU and Kernel Page Faults
<p>Page faults occurs when the process tries to access a memory that isn’t backed by a physical page kernel raises a fault which loads a page. It happens on first access, stack expansion, COW, swap and much more. However it comes with a cost. </p><p><br /></p><p>In this episode o ... Show More
48m 37s
Oct 31
Amazon US-EAST-1 Outage in Details
On October 19 2025 AWS experienced an outage that lasted over a day, 10 days later we finally got the root cause analysis and we know exactly what caused the DNS to fail0:00 Summary 5:30 How did Dynamo lost its DNS?13:41 EC2 Errors 16:16 Network Load Balancer ErrorsRCA here https ... Show More
24m 26s
Oct 17
Graceful shutdown in HTTP
There are cases where the backend may need to close the connection to prevent unexpected situations, prevent bad actors or simply just free up resources. Closing a connection gracefully allows clients and backends to clean up and finish any pending requests. In this episode of th ... Show More
25m 49s
Recommended Episodes
Mar 2020
Easier Stream Processing On Kafka With ksqlDB
<div class="wp-block-jetpack-markdown"><h2>Summary</h2> <p>Building applications on top of unbounded event streams is a complex endeavor, requiring careful integration of multiple disparate systems that were engineered in isolation. The ksqlDB project was created to address this ... Show More
43m 36s
Apr 2022
Postgres.js
Rasmus Porsager created Postgres.js –the fastest full-featured PostgreSQL client for Node.js and Deno. Today he joins Jerod for a deep-dive on Postgres, why he created this open source library, and how you can use it to build pg-backed JavaScript applications. Discuss on Changelo ... Show More
50m 6s
Apr 2022
Postgres.js (JS Party #221)
Rasmus Porsager created Postgres.js –the fastest full-featured PostgreSQL client for Node.js and Deno. Today he joins Jerod for a deep-dive on Postgres, why he created this open source library, and how you can use it to build pg-backed JavaScript applications. 
50m 6s
Sep 2023
Episode 486: High Performance PostgreSQL for Rails with Andrew Atkinson
If you love taking about databases, this is the episode for you. Ahead of the launch of his new book, High Performance PostgreSQL for Rails, Andrew Atkinson joined the show with special guest co-host, Pat Bair, to talk about why he wrote a book, why he focused on PostgreSQL and h ... Show More
31m 29s
Feb 2024
Episode 507: Stephen Margheim on SQLite in Ruby on Rails Applications
If you start a new Rails project today, you’re probably going to pick Postgres for your database. And if you have a ten year old Rails app, you might be using MySQL. But Rails ships with support for SQLite, a lightweight SQL database. Can that lightweight database be used to make ... Show More
31m 21s
Aug 2022
Tech job interview support group
Struggling through the tech job interview process? We feel you! On this episode, Amal, Nick & Amelia get together to discuss the various ways the interview process disappoints, share their own interview stories, and suggest ways we can improve the process for everyone. Discuss on ... Show More
1h 4m
Jun 2021
Accelerating ML Training And Delivery With In-Database Machine Learning
<div class="wp-block-jetpack-markdown"><h2>Summary</h2> <p>When you build a machine learning model, the first step is always to load your data. Typically this means downloading files from object storage, or querying a database. To speed up the process, why not build the model in ... Show More
1h 5m
Oct 2023
166: Speedy Database Queries with Lukas Fittl
<p>- pganalyze: <a href="https://pganalyze.com/">https://pganalyze.com/</a></p><p>- Weekly series "5mins of Postgres": <a href="https://www.youtube.com/channel/UCDV_1Dz2Ixgl1nT_3DUZVFw">https://www.youtube.com/channel/UCDV_1Dz2Ixgl1nT_3DUZVFw</a></p><p>- How Postgres chooses whic ... Show More
1h 12m
Dec 2019
Building The Materialize Engine For Interactive Streaming Analytics In SQL
<div class="wp-block-jetpack-markdown"><h2>Summary</h2> <p>Transactional databases used in applications are optimized for fast reads and writes with relatively simple queries on a small number of records. Data warehouses are optimized for batched writes and complex analytical qu ... Show More
48m 7s
Nov 2021
Exploring Processing Patterns For Streaming Data Integration In Your Data Lake
<div class="wp-block-jetpack-markdown"><h2>Summary</h2> <p>One of the perennial challenges posed by data lakes is how to keep them up to date as new data is collected. With the improvements in streaming engines it is now possible to perform all of your data integration in near r ... Show More
52m 53s