Apache Iceberg Metadata Architecture

Data & AI Pipelines diagram · made in Skemio

Tree-structured diagram of Apache Iceberg metadata showing Catalog pointing to Metadata File, pointing to Manifest List, referencing Manifest Files and Parquet files.

Open it in Skemio Make your own

What it shows

Iceberg catalog stores an atomic pointer to the active metadata file.
Manifest lists track snapshots and partition summaries without file scans.
Manifest files hold file-level metrics enabling rapid partition pruning.

The takeaway

Iceberg isolates queries from raw file trees using hierarchical, atomic snapshot metadata.

#ApacheIceberg
#DataEngineering
#DataLake
#SystemDesign
#Architecture

More data & ai pipelines diagrams