Foundations of distributed data · FND-06
Storage formats
- 3 h
- requires FND-01
From your world. Why Athena charges by scanned bytes; row-group pruning is why a WHERE on a sorted column is cheap
Explain to an analyst who sees a big Athena bill:
"Why does the same query cost far more on CSV than on Parquet, and how does a
WHEREon a sorted column make Athena read only a fraction of the file?"
Walk through row groups, column chunks, and footer min/max stats.
Rubric (self-check — aim for 5/6)
- Contrasts row vs columnar access patterns. Says why columnar reads fewer bytes.
- Names the layout of Parquet: row groups → column chunks → pages, with stats in the footer.
- Explains row-group pruning with the
min/maxof each group (skip when the range cannot match). - States that pruning needs sorting/clustering. Random order prunes nothing.
- Connects to Athena bytes-scanned billing and column projection as the top cost lever.
- Mentions that Iceberg manifest column metrics / sort order do file-level pruning.