DE Mastery GitHub

Build your own mini table format · Stage 2 of 7

Snapshots & time travel

Every append already wrote a new metadata version. Now expose that history as snapshots, and let a reader travel back to any past snapshot. This works only because data files are immutable: an old snapshot still points at files that were never overwritten.

Implement

  • A Snapshot object (or namedtuple-like) exposing at least snapshot_id, version, timestamp_ms, and manifest.
  • table.snapshots() -> list[Snapshot] — every committed snapshot, oldest first, including the empty create snapshot.
  • table.current_snapshot() -> Snapshot — the latest.
  • table.scan(snapshot_id=<id>) — read the table as of that snapshot, i.e. only the data files that snapshot's manifest listed. With no snapshot_id, scan the current snapshot (stage-1 behavior).

Why it matters

Time travel and auditing (SOX!) fall straight out of immutable data + a chain of metadata versions. A reader that captured snapshot_id keeps seeing exactly those rows no matter how many appends happen afterward.

Keep stage-1 tests green: scan() with no arguments is still "latest".

Hints

Hint 1

A snapshot is just a parsed v<N>.json. snapshots() = read v1..vLatest in order; each carries snapshot_id, version, timestamp_ms, manifest.

Hint 2

Give each commit a monotonically increasing snapshot_id and a timestamp_ms. append returns that id.

Hint 3

scan(snapshot_id=k): find the snapshot whose snapshot_id == k and read ONLY its manifest's files. No snapshot_id => current snapshot. Old data files are never overwritten, so this just works.

Run it locally
uv run de start LKH-BYO/02-snapshots-time-travel
uv run de test