Part 188: Change Data Capture (CDC): WAL Streaming & Debezium

Streaming every database insert, update, and delete into real-time event streams with zero application overhead.

1. Why Polling Fails at Scale

Polling queries (SELECT * FROM orders WHERE updated_at > :last_poll) cause severe CPU overhead and miss deleted records entirely. Change Data Capture (CDC) reads changes directly from the database transaction Write-Ahead Log (WAL / Binlog) in real-time!

2. PostgreSQL Logical Replication for CDC

-- 1. Configure postgresql.conf:
-- wal_level = logical
-- max_replication_slots = 10

-- 2. Create publication for CDC consumer:
CREATE PUBLICATION cdc_publication FOR TABLE orders, customers;

-- 3. Create replication slot:
SELECT pg_create_logical_replication_slot('debezium_slot', 'pgoutput');

3. CDC Event Structure (Debezium JSON)

When an update runs, CDC produces an event with full before and after state:

{
  "op": "u",
  "ts_ms": 1757788800000,
  "before": { "id": 105, "status": "pending", "total": 49.99 },
  "after":  { "id": 105, "status": "shipped", "total": 49.99 }
}

Key Takeaways

• CDC creates an event-driven architecture directly from your database WAL without double-writing in application code.

In Part 189, we explore The Transactional Outbox Pattern with SQL!