Iceberg as streaming storage — commit interval vs small files

4 h
requires STH-03

From your world. Your Iceberg tables: the small-files problem you already fight with compaction, now caused by streaming commit frequency

Practice quiz with instant feedback. For a scored run that counts toward your local progress, use uv run de quiz STH-04.

2A team sets a 1-second checkpoint interval to make an Iceberg table 'as fresh as possible'. A day later, Trino queries are slow. What is the most likely cause? AIceberg cannot handle streaming writes BFrequent commits produced a huge number of small data files and snapshots, so query planning and reads got expensive CThe watermark stalled DTrino does not support Iceberg v2

3A streaming job writes to an Iceberg sink. It runs forever and never commits a snapshot. The source is a datagen (unbounded) table with no checkpointing. What fixes it for a one-shot backfill? AAdd more TaskManagers BUse a bounded source (for example a filesystem / debezium-json file) so the job finishes and fires its final commit CSwitch to batch SELECT COUNT(*) DDisable the primary key

6You mirror a high-churn OLTP table (mostly updates to the same keys) onto the lake. Why might Apache Paimon be a better fit than Iceberg v2 for this feed? APaimon does not support primary keys BPaimon is an LSM primary-key store with built-in compaction designed for high-rate upserts, which keeps read amplification bounded. The delete files and read-side merge of Iceberg v2 grow with churn CIceberg cannot store updates at all DPaimon avoids writing any files

7Which statements about commit cadence on a Flink → Iceberg table are correct? (choose all) (select all that apply) AA shorter checkpoint interval improves freshness but increases the number of snapshots and small files BRunning the same total ingest as one job over a batched input produces fewer snapshots than running it as many small per-micro-batch jobs CA longer checkpoint interval produces larger, fewer files but a more stale table DCompaction removes the need to ever checkpoint

8Both Iceberg and Paimon land on object storage and commit per checkpoint. What is the key architectural difference in how they handle updates? AThere is none. They are identical BIceberg v2 uses merge-on-read with delete files. Paimon uses an LSM tree with primary-key buckets and background compaction CPaimon only supports batch writes DIceberg stores data in a database, not files