DE Mastery GitHub

Track · FND · ~20 h

Foundations of distributed data

Why this track exists: Spark, Kafka and Flink are three answers to the same few problems: partitioning, replication, ordering, time and failure. If you learn those problems once, each tool becomes a set of design choices instead of an API to memorize.

Lab profile: none (pure Python) plus core for FND-07.

Interview drills: "How would you design for exactly-once?", "Explain watermarks to a PM", "Why columnar?"

Modules

  1. 01 Why distributed? 2 h · 1 exercise
  2. 02 Partitioning & shuffles 3 h · 1 exercise
  3. 03 Replication & consensus 3 h · 1 exercise
  4. 04 Time 3 h · 1 exercise
  5. 05 Delivery semantics 3 h · 1 exercise
  6. 06 Storage formats 3 h · 1 exercise
  7. 07 Back-of-envelope estimation 3 h · 1 exercise