Build your own mini Kafka · Stage 3 of 8

Produce: RecordBatch v2 + CRC32C

Implement Produce (api 0). The client sends a serialized RecordBatch v2 per partition; you append it to the log and reply with the base offset you assigned.

Assign the current log-end offset as the batch's base; stamp it into the batch header (first 8 bytes) before writing.
The batch header carries a CRC32C (Castagnoli) over the bytes after the crc field — the tests re-check it, so your stored bytes must stay valid.
Advance the log-end offset by the record count.

Tests: a real producer gets contiguous offsets (0,1,2,...), a second produce continues the offsets, and the stored batch is magic=2 with a valid CRC32C.

Hints

Hint 1

A Produce request (api 0, flexible from v9) carries, per partition, a 'records' blob that is already a serialized RecordBatch v2. Append that blob to the active segment and reply with the base offset you assigned.

Hint 2

RecordBatch v2 layout: baseOffset int64, batchLength int32, partitionLeaderEpoch int32, magic int8 (=2), crc uint32, then attributes/lastOffsetDelta/timestamps/... The CRC is CRC32C (Castagnoli, poly 0x82F63B78) over every byte AFTER the crc field.

Hint 3

Before writing to disk, stamp the batch's baseOffset (the first 8 bytes) with the offset you assigned (the current log-end offset). Advance the log-end offset by the record count (bytes 57..60 of the batch).

Run it locally

uv run de start KFK-BYO/03-produce-recordbatch-crc
uv run de test