DE Mastery GitHub

Apache Flink · Build your own · FLK-BYO

Build your own mini stream processor

  • 10 h
  • 8 stages

Build the real thing from scratch, one stage at a time. Each stage has its own tests, and you move on when it passes.

  1. 01Operator graph + push-based execution
  2. 02Keyed partitioning across workers
  3. 03Event time + watermarks (min across inputs)
  4. 04Tumbling/sliding windows with triggers
  5. 05Keyed state + timers (process functions)
  6. 06Checkpoint barriers (Chandy-Lamport / ABS)
  7. 07Recovery from a checkpoint with a replayable source
  8. 08Transactional (2PC) sink — end-to-end exactly-once
Run it locally
uv run de start FLK-BYO/01
uv run de test