Skip to content
Back to skills

Answers

ASecurity

- Normalized question: PostgreSQL logical replication vs Debezium for streaming order-database changes into Kafka: which should we use, and what are the tradeoffs? - Research kind: `web` - Depth mode: `deep` - Evidence chain requirements: typed evidence; extracted excerpts for web claims; repository evidence IDs for codebase claims

  • 30 stars
  • 0 votes
  • 0 copies
  • 0 views
  • Added September 19, 2026
ai-agentsgosqlnodeawsapidatabaseperformancedocumentation

Works with

  • api

Security analysis

A100/100

Pro scans all 9 files and shows the line behind each finding

Scanned September 19, 2026

npx -y skills add johnqtcg/awesome-skills --skill answers --agent claude-code

Installs into .claude/skills of the current project.

Are you the author of Answers?

Add the live security badge to your README. It updates with every re-scan.

Security grade badge for Answers
[![Security: A — Skills Directory](https://www.skillsdirectory.com/api/skills/johnqtcg-answers-66a71196/badge)](https://www.skillsdirectory.com/skills/johnqtcg-answers-66a71196)

More formats (shields.io, HTML) on the badges page. Keep it an A: scan every change in CI with Pro.

Download with Pro
SKILL.md
## 1) Research Question
- Normalized question: PostgreSQL logical replication vs Debezium for streaming order-database changes into Kafka: which should we use, and what are the tradeoffs?
- Research kind: `web`
- Depth mode: `deep`
- Evidence chain requirements: typed evidence; extracted excerpts for web claims; repository evidence IDs for codebase claims

## 2) Method
- Execution mode: `deep`
- Degradation level: `Partial`
- Retrieval plan (queries/subtopics): 7 distinct queries represented
- Dedup strategy: normalized URL canonicalization + first-seen retention
- Retrieved sources: 49
- Content items processed: 14
- Successfully extracted: 14
- Repository evidence units: 0
- Cited evidence units: 10
- Validation checks performed: required-input, extraction-success, exact excerpt, repository reference, claim-support review, confidence, and degradation checks
- Citation integrity (this is what `Degradation` above reports): every cited excerpt was re-read from the artifact it names
- Claim support (a separate verdict, not implied by the one above): disputed — attested 10, qualified 1, unreviewed 0, disputed 7, blocked 0
- Machine checks cannot decide entailment. A polarity and number screen can only remove support; the reviewed stance on each finding is an author attestation.
- Budget consumed: retrieval 23/50, extractions 14/15, live verifications 10/15

## 3) Executive Summary
Partial result: PostgreSQL's native logical replication (CREATE PUBLICATION / CREATE SUBSCRIPTION) ships schema and DDL changes to subscribers only if applied by hand; sequence state and large objects are excluded outright. Any pipeline built on top of the same logical-decoding mechanism (Debezium included) inherits the DDL blindness, because it is a property of the decoding layer, not of the consumer. PostgreSQL's native logical replication is a publish/subscribe protocol between PostgreSQL instances: the subscriber is itself a full PostgreSQL database that applies incoming changes. It has no built-in path to Kafka. Getting logically-decoded PostgreSQL changes into Kafka therefore always requires a separate consumer of the replication stream (a Kafka Connect source connector such as Debezium, a custom pg_recvlogical-based consumer, or a vendor CDC product) -- 'logical replication' by itself is not a second, competing way to reach Kafka, it is the underlying mechanism Debezium itself is built on. Debezium's PostgreSQL connector must be configured with a logical decoding output plug-in. decoderbufs is a Protobuf-based plug-in maintained by the Debezium community and requires separate installation; pgoutput is the standard PostgreSQL 10+ output plug-in maintained by the PostgreSQL community itself (the same plug-in used by native CREATE SUBSCRIPTION), so it needs no extra library.

## 4) Key Findings
- **PostgreSQL's built-in logical replication does not replicate DDL, sequence values, or large objects** (High confidence): PostgreSQL's native logical replication (CREATE PUBLICATION / CREATE SUBSCRIPTION) ships schema and DDL changes to subscribers only if applied by hand; sequence state and large objects are excluded outright. Any pipeline built on top of the same logical-decoding mechanism (Debezium included) inherits the DDL blindness, because it is a property of the decoding layer, not of the consumer. [7]
- **Built-in CREATE PUBLICATION / CREATE SUBSCRIPTION targets another PostgreSQL database, not Kafka** (Medium confidence): PostgreSQL's native logical replication is a publish/subscribe protocol between PostgreSQL instances: the subscriber is itself a full PostgreSQL database that applies incoming changes. It has no built-in path to Kafka. Getting logically-decoded PostgreSQL changes into Kafka therefore always requires a separate consumer of the replication stream (a Kafka Connect source connector such as Debezium, a custom pg_recvlogical-based consumer, or a vendor CDC product) -- 'logical replication' by itself is not a second, competing way to reach Kafka, it is the underlying mechanism Debezium itself is built on. [8] [2] Downgrade: the only primary source is the subject project's own documentation; a comparison or recommendation needs an independent primary unit; polarity screen flagged logical replication; author override recorded, so this claim cannot be High.
- **Debezium reads the same logical-decoding stream via one of two output plug-ins: decoderbufs or pgoutput** (Medium confidence): Debezium's PostgreSQL connector must be configured with a logical decoding output plug-in. decoderbufs is a Protobuf-based plug-in maintained by the Debezium community and requires separate installation; pgoutput is the standard PostgreSQL 10+ output plug-in maintained by the PostgreSQL community itself (the same plug-in used by native CREATE SUBSCRIPTION), so it needs no extra library. [1]
- **Debezium also cannot emit DDL change events for PostgreSQL, for the same underlying reason** (Medium confidence): Debezium's PostgreSQL connector inherits PostgreSQL logical decoding's restrictions directly: it states plainly that it cannot report DDL change events because logical decoding itself does not carry DDL. This is not a Debezium shortcoming relative to native replication -- it is the same limitation, because both sit on the same decoding layer. [1] Downgrade: polarity screen flagged logical decod; author override recorded, so this claim cannot be High.
- **Both approaches depend on a PostgreSQL replication slot, which can retain WAL without bound and fill disk if the consumer stops** (Medium confidence): Native logical replication (a subscription) and Debezium both work by creating and holding a PostgreSQL replication slot. PostgreSQL's own configuration reference states that by default (max_slot_wal_keep_size = -1) a slot may retain an unlimited amount of WAL, and that once the slot falls too far behind, WAL removal is blocked. Debezium's own connector documentation independently confirms the same mechanism from its side: because PostgreSQL purges WAL after a period, the connector's slot is what prevents that purge so it does not lose its position -- the operational risk (an abandoned or lagging slot silently filling the data disk) is identical for either implementation and must be monitored regardless of which one is chosen. [5] [1] Downgrade: the only primary source is the subject project's own documentation; a comparison or recommendation needs an independent primary unit; High requires one current-process live-verified T1 primary Web source for a narrow single fact, direct code evidence for a code fact, code plus a passing test for runtime behavior, or two independent verified units including a primary source.
- **An abandoned replication slot has caused a real production primary-disk-full outage** (Medium confidence): A first-hand incident report describes a PostgreSQL primary that PANIC'd after pg_wal grew to 214GB on a 250GB disk because a decommissioned standby's replication slot was never dropped; the slot's frozen restart_lsn blocked WAL cleanup even though write volume was ordinary. This is a single practitioner account (not vendor documentation) but it directly illustrates the mechanism PostgreSQL's own docs describe in finding-5, and applies equally to an abandoned Debezium slot. [6] Downgrade: polarity screen flagged pg wal, the slot s; author override recorded, so this claim cannot be High.
- **The Debezium PostgreSQL source connector (as documented by Confluent) delivers at-least-once and runs as a single task** (Medium confidence): Confluent's documentation for the Debezium PostgreSQL source connector states it guarantees at-least-once delivery (duplicates are possible after a restart or network fault) and that the connector supports running only one task -- i.e. a single connector instance cannot be horizontally sharded across multiple Kafka Connect tasks. Downstream consumers must be built to tolerate duplicate records, and throughput/HA planning has to work within a single-task-per-connector model (running more connectors, one per database/table-set, is the scale-out lever, not more tasks per connector). [3]
- **Third-party extensions (e.g. pglogical) exist specifically to add DDL replication on top of native logical replication** (Low confidence): Because native logical replication does not propagate DDL, vendors have built extensions to close that gap for PostgreSQL-to-PostgreSQL replication specifically; pglogical's community edition offers a function to capture and re-execute DDL on subscribers. This does not extend to Debezium/Kafka pipelines (pglogical replicates to another PostgreSQL node, not to Kafka) but it confirms the DDL gap is treated as a known, addressed-by-tooling problem rather than a rare edge case. [9]
- **Debezium change events can be Avro-serialized with a schema registry (Apicurio or Confluent) to manage schema evolution in Kafka** (Medium confidence): Because Debezium dynamically generates each record's schema from the source table's structure, records written to the same Kafka topic over time can carry different schema versions. Debezium supports Avro serialization backed by a schema registry (Apicurio or Confluent Schema Registry) specifically to let consumers adapt to that drift. This is a Kafka-ecosystem capability that only exists because Debezium's output is Kafka messages -- it has no equivalent in native logical replication, which never produces a serialized, registry-tracked message schema at all. [4]
- **Some managed PostgreSQL offerings cannot run Debezium because the logical decoding plug-in cannot be installed** (Medium confidence): Confluent's own connector documentation calls out that databases hosted by a service such as Heroku Postgres cannot be monitored with Debezium, because the operator may be unable to install the logical decoding plug-in the connector needs. This is a real deployment-environment constraint to check against wherever the orders database is actually hosted, independent of the architectural comparison. [3] Downgrade: polarity screen flagged databas host by a service such as heroku postgr; author override recorded, so this claim cannot be High.
- **An independent CDC-vendor comparison frames native logical replication as replication-only, and Debezium+Kafka Connect as the higher-complexity, more flexible middleware path** (Low confidence): RisingWave (a vendor of a third, competing CDC approach) publishes a comparison table describing native PostgreSQL logical replication as having no middleware and being replication-only (no transformation, no non-PostgreSQL destinations), versus Debezium+Kafka Connect as requiring Kafka Connect and Kafka as middleware with higher operational complexity but supporting external/non-PostgreSQL destinations. The direction of this framing (Debezium = more moving parts) is consistent with findings 2, 3, and 7 above from the projects' own documentation, but the source has a direct commercial interest in making both alternatives look more complex than its own product, so treat the complexity framing as corroborating context rather than an independent verdict. [10] Downgrade: polarity screen flagged nonpostgresql destination; author override recorded, so this claim cannot be High.

## 5) Detailed Analysis
### How each option actually gets changes into Kafka
PostgreSQL's built-in logical replication (CREATE PUBLICATION / CREATE SUBSCRIPTION) is a publish/subscribe protocol whose subscriber is itself a PostgreSQL database -- it has no native Kafka target. Debezium is a Kafka Connect source connector that consumes the same underlying logical-decoding stream (via the decoderbufs or pgoutput plug-in) and republishes each change as a Kafka record, one topic per table by default. So the real choice for 'streaming changes into Kafka' is not between two competing streaming mechanisms; it is between writing and operating your own consumer of PostgreSQL's logical decoding stream, or adopting Debezium as a maintained, off-the-shelf Kafka Connect consumer of that same stream. Native logical replication alone does not reach Kafka. [8] [2] [1]

### Schema and DDL handling
Neither option replicates DDL. PostgreSQL's own restrictions page states that schema and DDL commands are not replicated by logical replication, and separately that sequence data and large objects are excluded too. Debezium's PostgreSQL connector documentation independently confirms it cannot report DDL change events, for the identical reason: the logical decoding layer beneath it carries no DDL. Any 'add a column and the pipeline breaks' story you may have heard about either approach traces back to this same root cause, not to a design choice either project made. Third-party extensions such as pglogical exist specifically to bolt DDL replication onto native logical replication for Postgres-to-Postgres topologies, which is further evidence the gap is a known, not a hypothetical, pain point. [7] [1] [9]

### Operational risk: replication slots and WAL retention
Both a native subscription and a Debezium connector work by creating and holding a PostgreSQL replication slot. PostgreSQL will not remove any WAL segment the slot might still need; by default (max_slot_wal_keep_size = -1) that retention is unbounded. If the consumer -- a subscriber or a Debezium connector -- stops permanently without the slot being dropped, WAL accumulates on the primary's disk indefinitely. A first-hand incident report describes exactly this failure mode taking a primary down over a quiet weekend after a standby was decommissioned without dropping its slot. This is an operational responsibility that exists identically for either technology choice: whichever one you pick, you need alerting on replication slot lag / restart_lsn and a runbook for dropping a dead slot before it eats the disk. [5] [6]

### Delivery semantics, scaling, and message format
Confluent's documentation for the Debezium PostgreSQL source connector states it is at-least-once (a restart or network fault can produce duplicate Kafka records, so downstream consumers must de-duplicate) and that the connector runs as a single Kafka Connect task -- scale-out means running more connectors (e.g. one per database or table set), not more tasks per connector. On the message-format side, because Debezium's output is Kafka messages, it can serialize records as Avro and track schema versions in a schema registry (Apicurio or Confluent Schema Registry) so consumers can handle a table's schema changing over time; native logical replication has no analogous concept because it never produces an externally-consumed, versioned message schema at all. Separately, if the orders database ends up on a managed PostgreSQL host, verify up front that the platform allows installing a logical decoding plug-in -- Confluent's docs name Heroku Postgres as one host where Debezium cannot be used for exactly that reason. [3] [4]

## 6) Consensus vs Debate
### Consensus
- DDL/schema changes are not carried by either option, because both sit on the same PostgreSQL logical-decoding mechanism. [7] [1]
- Both approaches create a PostgreSQL replication slot and inherit its unbounded-WAL-retention risk if the consumer stops. [5] [6]

### Debate / Contradictory Evidence
- Native logical replication is sometimes framed as the 'simpler' option, but for this specific goal (landing changes in Kafka) it is not a complete alternative on its own: it cannot target Kafka without an additional custom consumer, whereas Debezium is a ready-made Kafka Connect consumer of the same underlying stream. A third-party CDC vendor's public comparison labels native logical replication as the lowest-complexity option and Debezium+Kafka Connect as the higher-complexity, higher-flexibility option -- which is directionally consistent with the projects' own docs, but the source sells a third competing CDC product, so its complexity scoring for the other two should be read as illustrative, not as an independent benchmark. [8] [10]

## 7) Source Quality Notes
- Source tier distribution: T1: 3, T2: 0, T3: 0, T4: 7, T5: 0
- Classification basis: caller-provided web tier/type labels are ignored; domain heuristics are re-derived from normalized URLs, and only a fresh validator-controlled capture may establish primary-source status.
- Live Web verification: 38/38 verified Web evidence references were freshly captured; validator-derived live T1: 11. Serialized content artifacts are audit inputs, not execution proof.
- Potential bias / sponsorship requiring review: Debezium connector for PostgreSQL, Debezium Architecture :: Debezium Documentation, Debezium PostgreSQL Source Connector for Confluent Platform | Confluent ..., Avro Serialization :: Debezium Documentation, PostgreSQL: Documentation: 18: 19.6. Replication, PostgreSQL Replication Slots: Retained WAL Fills Disk | MonPG, PostgreSQL: Documentation: 18: 29.8. Restrictions, PostgreSQL: Documentation: 18: Chapter 29. Logical Replication, How Not To Use PostgreSQL DDL Replication - pgedge.com, PostgreSQL CDC: Logical Replication vs Debezium vs Native Streaming
- Sources with unknown methodology: 10
- Repository evidence quality: code observations: 0; commit-pinned: 0; tests passed: 0; tests failed/other: 0
- Domain authority: 11/38 cited Web units were classified from the curated authority registry rather than a URL-shape heuristic; 11 are the subject project's own documentation and cannot serve as the independent primary unit for a comparison or recommendation.
- Claim support: disputed (attested 10, qualified 1, unreviewed 0, disputed 7, blocked 0); 7 claim(s) tripped the polarity screen. Excerpt containment is proved mechanically; entailment is an author attestation and is not machine-verified.
- Single-source findings: 3
- Unverified findings omitted from substantive sections: 0
- Evidence chain status: partially satisfied

## 8) Sources
[1] Debezium connector for PostgreSQL — https://debezium.io/documentation/reference/stable/connectors/postgresql.html — website (preclassified T4) — date: unknown; basis: heuristic:unverified-domain; sponsorship: unknown; methodology: unknown
[2] Debezium Architecture :: Debezium Documentation — https://debezium.io/documentation/reference/stable/architecture.html — website (preclassified T4) — date: unknown; basis: heuristic:unverified-domain; sponsorship: unknown; methodology: unknown
[3] Debezium PostgreSQL Source Connector for Confluent Platform | Confluent ... — https://docs.confluent.io/kafka-connectors/debezium-postgres-source/current/overview.html — website (preclassified T4) — date: unknown; basis: heuristic:unverified-domain; sponsorship: unknown; methodology: unknown
[4] Avro Serialization :: Debezium Documentation — https://debezium.io/documentation/reference/stable/configuration/avro.html — website (preclassified T4) — date: unknown; basis: heuristic:unverified-domain; sponsorship: unknown; methodology: unknown
[5] PostgreSQL: Documentation: 18: 19.6. Replication — https://www.postgresql.org/docs/current/runtime-config-replication.html — official (preclassified T1) — date: unknown; basis: registry:project-owned:Official domain of the PostgreSQL Global Development Group; sponsorship: unknown; methodology: unknown
[6] PostgreSQL Replication Slots: Retained WAL Fills Disk | MonPG — https://monpg.app/blog/postgresql-replication-slots-retained-wal-disk-full — website (preclassified T4) — date: unknown; basis: heuristic:unverified-domain; sponsorship: unknown; methodology: unknown
[7] PostgreSQL: Documentation: 18: 29.8. Restrictions — https://www.postgresql.org/docs/current/logical-replication-restrictions.html — official (preclassified T1) — date: unknown; basis: registry:project-owned:Official domain of the PostgreSQL Global Development Group; sponsorship: unknown; methodology: unknown
[8] PostgreSQL: Documentation: 18: Chapter 29. Logical Replication — https://www.postgresql.org/docs/current/logical-replication.html — official (preclassified T1) — date: unknown; basis: registry:project-owned:Official domain of the PostgreSQL Global Development Group; sponsorship: unknown; methodology: unknown
[9] How Not To Use PostgreSQL DDL Replication - pgedge.com — https://www.pgedge.com/blog/how-not-to-use-postgresql-ddl-replication — website (preclassified T4) — date: unknown; basis: heuristic:unverified-domain; sponsorship: unknown; methodology: unknown
[10] PostgreSQL CDC: Logical Replication vs Debezium vs Native Streaming — https://risingwave.com/blog/postgresql-cdc-logical-replication-debezium — website (preclassified T4) — date: unknown; basis: heuristic:unverified-domain; sponsorship: unknown; methodology: unknown

## 9) Gaps & Limitations
- No independent, disclosed-methodology benchmark comparing Debezium's end-to-end latency/throughput against a hand-rolled logical-replication consumer was found; any performance claim in vendor blogs (e.g. RisingWave's '<5% source-database overhead' framing) was excluded from findings because it carries no disclosed method or environment and cannot be verified.
- Two candidate sources failed content extraction and are excluded from findings: https://estuary.dev/blog/postgresql-replication-slots/ returned only page CSS, not article text, and https://wiki.postgresql.org/images/d/dd/20230602-DDL_Replication.pdf returned raw PDF bytes that the extractor did not convert to text. Neither is cited.
- PostgreSQL 16 added logical replication from standby servers, and a 2023-era blog post found during retrieval discusses an early Debezium beta's support for that specific topology; because the beta version is dated and the mainline Debezium PostgreSQL connector documentation fetched for this report does not confirm current standby-based streaming support, this topic is named here rather than asserted as a current capability. If replicating from a standby (rather than the primary) matters for the orders database, verify current support directly against the Debezium version you plan to deploy.
- This report does not cover managed CDC alternatives (e.g. AWS DMS, cloud-vendor CDC services) or Debezium Server / Debezium Engine deployment modes in depth; retrieval and analysis were scoped to the Kafka Connect deployment of Debezium versus PostgreSQL's built-in logical replication, per the question as asked.
- Cost and licensing were not part of the evidence gathered (both PostgreSQL logical replication and open-source Debezium are free; Confluent's packaged connector and any managed Kafka/Kafka Connect service are separate cost centers not quantified here).
- finding #2 claim support is disputed: polarity screen flagged logical replication; author override recorded, so this claim cannot be High
- finding #4 claim support is disputed: polarity screen flagged logical decod; author override recorded, so this claim cannot be High
- finding #5 confidence downgraded from high to medium
- finding #6 claim support is disputed: polarity screen flagged pg wal, the slot s; author override recorded, so this claim cannot be High
- finding #10 claim support is disputed: polarity screen flagged databas host by a service such as heroku postgr; author override recorded, so this claim cannot be High
- finding #11 claim support is disputed: polarity screen flagged nonpostgresql destination; author override recorded, so this claim cannot be High

Files in this skill

  • eval-0-verifiable-single-fact__with_skill.md10.1 KB
  • eval-0-verifiable-single-fact__without_skill.md4.8 KB
  • eval-1-comparison-with-method__with_skill.md22.9 KB
  • eval-1-comparison-with-method__without_skill.md18.8 KB
  • eval-2-conflicting-official-sources__with_skill.md13.3 KB
  • eval-2-conflicting-official-sources__without_skill.md14.6 KB
  • eval-3-unknowable-figure__with_skill.md8.7 KB
  • eval-3-unknowable-figure__without_skill.md5.6 KB

Attribution

Is this your skill, or is something wrong with this listing? Request removal or report an issue. Author removals are honored within 72 hours.

Comments

Loading comments…

Newsletter

Get the weekly list of safe new skills, and the ones we pulled.