Change Data Capture (CDC)

Substation supports change data capture (CDC) on AWS DynamoDB tables using DynamoDB Streams. CDC events are modeled similarly to events from Debezium:

JSON

{
	"source": {
		"ts_ms": 0,
		"table": "table",
		"connector": "dynamodb"
	},
	"ts_ms": 0,
	"op": "c",
	"before": { ... },
	"after": { ... }
}

Observability

CDC enables observability on DynamoDB tables by generating events for changes to any item, including item creation, modification, and removal. These events can be treated as observability (or telemetry) data and handled like any other dataset in the system:

Notifications

The system can emit notifications based on CDC activity by using multiple Substation nodes that have unique condition configurations:

Use cases include:

• Notify downstream services about table changes using HTTPS
• Allow other teams to subscribe to table changes using a Pub/Sub pattern

Replication

For data enrichment use cases, the CDC events can be used to replicate data to other locations:

Use cases include:

• Replicate enriched data from a service-oriented table to a monolithic table
• Replicate enriched data to long-term cloud bucket storage