Durability — What "Saved" Actually Means

Your table_save returns 0 and you trust the data is on disk. Here's the uncomfortable truth: it probably isn't yet. If the machine loses power one second after table_save returns, there's a good chance your file is gone, empty, or half-written. This lesson is about the D in ACID — durability — and it's where a toy database starts becoming a real one.

The Write Path: Four Layers of Lies

When you call fwrite, the bytes travel through a stack of buffers, and each layer reports success as soon as it has handed off to the next:

your buffer
   → stdio buffer        (in your process; flushed by fflush/fclose)
   → kernel page cache   (in RAM; flushed by the OS "eventually", or by fsync)
   → drive write cache   (RAM on the disk itself; flushed by a cache-flush command)
   → the actual medium   (platter / NAND — the only layer that survives power loss)

• fwrite usually copies into the stdio buffer and returns. The kernel hasn't even seen the data.
• fflush (or fclose) pushes it to the kernel page cache. Now it survives your process crashing — but the page cache is RAM. The kernel writes dirty pages back on its own schedule, typically within ~30 seconds. Power loss in that window: data gone, fwrite and fclose both reported success.
• fsync(fd) tells the kernel: block me until this file's data is on stable storage, including telling the drive to flush its own cache. This is the only call in the stack that means "durable".

Why the layering? Performance. RAM is ~1000× faster than storage, and batching many small writes into few large ones is the single biggest I/O optimization there is. The page cache makes 99% of programs faster and is the right default. Databases are the 1%: when a database says "committed", that's a promise about power loss, so databases call fsync at every commit point — and pay for it. An fsync costs ~milliseconds on an SSD (~tens on spinning rust), which is why real databases group commit: batch several transactions' writes, fsync once, then report them all committed.

Crash Windows: Why Overwrite-In-Place Corrupts

Durability isn't just "did the bytes arrive" — it's what state is the file in if we crash halfway. Look at our current save:

FILE *f = fopen(path, "wb");   /* truncates the file to 0 bytes! */
/* ... write header, write rows ... */

The instant fopen(path, "wb") returns, the old data is already destroyed. For the entire duration of the write, the file on disk is some prefix of the new data: empty, header-only, or half the rows. Crash anywhere in that window and you've lost both the old version and the new one. Note this failure needs no power loss — the process being killed (OOM killer, Ctrl-C, a bug) is enough.

It gets subtler: the kernel and the drive may write your data back in any order. Writing bytes 0–4095 then 4096–8191 does not mean they reach the platter in that order. A crash can leave the second page written and the first one old — a file that's interleaved old/new garbage. This is called a torn write, and it's why "I'll just be careful about write order" doesn't work without explicit barriers (fsync is also an ordering barrier).

The Fix: Write-New-Then-Rename

POSIX gives us one operation with exactly the right property: rename(old, new) is atomic. If new already exists, it is replaced, and any observer — including one that crashes and reboots — sees either the old file or the new file, never a mixture, never a missing file. Filesystems implement this as a single metadata update in their journal.

That gives the classic durable-save recipe, used by everything from text editors to package managers:

1. Write the complete new contents to a temporary file in the same directory (data.db.tmp). Crash here? The real file is untouched; the leftover .tmp is litter, not damage.
2. fflush the stdio buffer, then fsync(fileno(f)) — the new bytes are now durable, but under the temp name. (Order matters: fsync before rename, or you can crash into a durable rename of a non-durable file — an empty data.db.)
3. fclose, then rename("data.db.tmp", "data.db") — the atomic switch.
4. (Full rigor: fsync the directory too, so the rename itself — a directory entry update — is durable. We'll mention it, not require it.)

Failure handling falls out naturally: any error before the rename → delete the temp file and report failure; the previous database is still intact. This either/or property is atomicity meeting durability, and it's precisely what SQLite's rollback journal and WAL achieve at page granularity: never put the only copy of your data into a state you couldn't recover from. Our whole-file version costs O(n) per save; SQLite's journaled version costs O(changed pages) — same guarantee, better price.

A C Portability Note: Feature-Test Macros

fileno is a POSIX function, not ISO C (fsync itself has needed no such guard on glibc since 2.16, but fileno still does). We compile with -std=c17, which asks libc for strict standard C — and glibc responds by hiding POSIX declarations it doesn't have to expose. The fix is to declare which API level you want, before any #include:

#define _POSIX_C_SOURCE 200809L   /* "give me POSIX.1-2008" */
#include <stdio.h>
#include <unistd.h>               /* fsync */

This is worth knowing beyond this course: it's why a program can compile fine with -std=gnu17 and break with -std=c17. The starter code has the define in place.