Apache Spark · SPK-07
Partitioning, bucketing, and the small-file problem
- 6 h
- requires SPK-06
From your world. Redshift DISTKEY / SORTKEY; Athena/Iceberg partitioning and file sizing
- Hit a target output file countSPK-07/01-target-file-count
- Reduce files after a filter without a shuffleSPK-07/02-coalesce-no-shuffle
- Partition the output by a low-cardinality keySPK-07/03-partition-by-layout
Run it locally
uv run de start SPK-07
uv run de test