System Design

Real-Time Metrics with Kafka and ClickHouse

System Design diagram · made in Skemio

System architecture diagram illustrating telemetry events flowing through Kafka, processed by stream workers, stored in ClickHouse, and queried by Grafana.
System architecture diagram illustrating telemetry events flowing through Kafka, processed by stream workers, stored in ClickHouse, and queried by Grafana.
Open it in SkemioMake your own

What it shows

  • Kafka partitions decouple high-velocity ingestion from analytical storage.
  • Stream workers aggregate metrics into micro-batches for efficient I/O.
  • ReplacingMergeTree deduplicates data into fast columnar compressed blocks.

The takeaway

Buffer high-frequency writes with Kafka and leverage columnar engines for sub-second analytical queries.

More system design diagrams