Apache Flink · FLK-04
Flink SQL & dynamic tables
- 6 h
- requires FLK-02
From your world. your dbt models — the same SELECTs, but as continuously-updating dynamic tables
Prompt A. "Explain the stream–table duality and the four RowKinds (+I, -U, +U, -D). Walk through
the exact changelog that an unbounded SELECT k, COUNT(*) FROM t GROUP BY k emits when the rows
(k='x'), (k='y'), (k='x') arrive."
Prompt B. "Explain the difference between an append-only query and an updating query. Give an example of each. Say which Kafka sink each one needs and why."
Rubric — a strong explanation hits these (5–7):
- A Flink SQL query produces a dynamic table (a result that updates continuously). The changes of this table are a changelog stream.
- Names the RowKinds: +I insert, -U retract old, +U new value, -D delete.
- Traces the GROUP BY changelog: +I(x,1), +I(y,1), then -U(x,1) +U(x,2) when the second x arrives.
- Append-only = only +I, and past output never changes (filter/project, window TVF). Updating = -U/+U/-D, and past output is revised (unbounded GROUP BY, latest-per-key).
- Append-only → plain
kafkasink. Updating →upsert-kafkakeyed by the grain (null = delete). - The final materialized state after all input equals the batch answer over the same events.
- (Bonus) A window TVF aggregation is append-only (one row per closed window), unlike an unbounded GROUP BY.