Data & AI

Kappa Architecture Explained

Data & AI Pipelines diagram · made in Skemio

Kappa architecture diagram showing an immutable event stream feeding a single stream processing engine that outputs directly to serving storage.
Kappa architecture diagram showing an immutable event stream feeding a single stream processing engine that outputs directly to serving storage.
Open it in SkemioMake your own

What it shows

  • Eliminates the dual codebase problem by treating everything as a stream.
  • Reprocessing history simply means resetting consumer offsets to timestamp zero.
  • A single stream engine (Flink/Kafka Streams) handles real-time and backfills.

The takeaway

Kappa treats batch processing as a subset of streaming: all data is an infinite, replayable log.

More data & ai pipelines diagrams