File Persistence and Serialization

Storing a table in memory is convenient during a session, but you lose everything when the program exits. Databases persist data to disk. The simplest format is direct serialization: write the row array to a file as raw bytes.

SQLite uses a B-Tree structure with variable-sized pages; we'll use a simpler approach: a header with metadata, followed by all rows packed end-to-end.

[Header: magic (4 bytes) | version (1 byte) | count (uint32) | next_id (uint32)]
[Row 1: id | name (256 bytes) | age]
[Row 2: id | name (256 bytes) | age]
...

Why a Header at All?

A file on disk is just bytes — it carries no type information. Six months from now, something will hand your load function the wrong file: a truncated copy, a file from an older build, a JPEG some script renamed to .db. The header is how your code notices before it does damage.

• The magic number (e.g., 0xCAFEBABE) is a cheap sanity check: if the first four bytes don't match, this is not our file, stop immediately. Nearly every binary format starts this way — %PDF, \x7fELF, PNG's 8-byte signature. SQLite files begin with the 16-byte string "SQLite format 3\0".
• The version byte buys you the right to change the format later. If you add a column in version 2, version-1 code sees version == 2 and bails out gracefully instead of interpreting new-format bytes with old-format offsets — which wouldn't error, it would silently load garbage. Rejecting loudly beats corrupting quietly, every time.
• count tells the loader how much to read, and doubles as a corruption check: a header claiming 1000 rows in a file with space for 3 means the file was truncated.

The general principle: make the file self-describing. Your in-memory struct can change every compile; the bytes you've written to users' disks are forever.

Struct Padding: The Silent Format-Breaker

We're going to serialize with fwrite(t->rows, sizeof(Person), count, f) — writing the structs exactly as they sit in memory. That only works because Person has no padding: 4 + 256 + 4 bytes, all 4-byte-aligned. The compiler is free to insert invisible padding bytes between fields (and after the last one) to satisfy alignment, and those bytes are uninitialized garbage that would go straight into your file. Worse, padding differs between compilers and architectures, so a file written on one machine might not parse on another.

This is why serious formats don't dump structs: they serialize field by field into an explicitly laid-out buffer (SQLite's record format, Protocol Buffers, etc.). We accept the struct-dump shortcut because our layout is padding-free and we control both writer and reader — but if you ever add a uint8_t active field to Person, three padding bytes appear and your file format has silently changed. When in doubt: _Static_assert(sizeof(Person) == 264, "Person layout changed — bump DB_VERSION");

Endianness Gotcha

When you write uint32_t count = 42 with fwrite, the bytes on disk depend on your CPU's endianness: little-endian (x86, ARM) writes 2A 00 00 00, while big-endian (PowerPC, network byte order) writes 00 00 00 2A. If you write on little-endian and read on big-endian, the value is backwards — 42 becomes 704,643,072.

Real databases pick one order and stick to it: SQLite stores all multi-byte integers big-endian regardless of the host CPU, converting on every read and write, which is why an SQLite file copied between any two machines just works. For this course, we'll assume all reads and writes happen on the same machine, so you can use native byte order and fwrite directly. The magic number gives you partial protection anyway: read on a wrong-endian machine, 0xCAFEBABE comes back as 0xBEBAFECA and the load is rejected.

Serialization Strategy

To load: read the header to learn how many rows exist, allocate space for them, then read that many rows into the array. To save: write the header, then all rows. This is O(n) in table size for each operation — every save rewrites the entire file even if one row changed.

Why is that a problem at scale? A 10 GB table where you update one 264-byte row would rewrite 10 GB. Real databases fix this with page-oriented storage: the file is an array of fixed-size pages, each page holds some rows, and a write only touches the pages containing changed rows. That's also why B-Trees (later lesson) are the universal database structure — they're designed so every operation touches O(log n) pages. Combined with write-ahead logging (append changes to a log, apply them to pages lazily), a one-row update costs a few KB of I/O regardless of table size. We'll keep whole-file rewrites — correct, just not incremental — and make them crash-safe in the next lesson.

One more error-handling discipline this challenge forces on you: every fread/fwrite return value must be checked. Disks fill up, files get truncated. A load that ignores a short read returns a table full of uninitialized memory — the worst kind of bug, because it usually works.