// lesson: election-timing
Election Timing in a Deterministic World
Raft's liveness hinges on timing (ยง5.6, "Timing and availability"). Followers expect a heartbeat from the leader well before their election timeout expires; the paper's rule of thumb is
// broadcastTime << electionTimeout << MTBF
If every server used the same election timeout, they would all time out
together, all become candidates together, split the vote, and repeat forever.
Raft's fix is charmingly low-tech: randomized election timeouts (e.g.
150โ300 ms). Whoever draws the shortest timeout usually wins the election
before anyone else even wakes up. The paper's own experiments (ยง9.3,
"Performance") back this recommendation: shrinking the timeout range brings
faster failover, but push it too low (well under the broadcast time) and
leaders start missing their own heartbeat deadline before followers'
timeouts fire, causing unnecessary elections and lower availability โ which
is exactly the broadcastTime << electionTimeout half of the inequality
above. 150โ300 ms is the paper's own conservative recommendation, chosen to
avoid that failure mode while still detecting a crashed leader quickly.
Time as data
Randomness and wall clocks are poison for tests. Production-grade Raft
implementations (etcd's raft library is the famous example) therefore model
time as an integer: the node has no timer at all, just a Tick() method the
host calls at a fixed cadence, plus counters:
ElectionElapsedโ ticks since the last heartbeat/vote-grant; when it reachesElectionTimeout, the election fires.HeartbeatElapsedโ ticks since the leader last broadcast; when it reachesHeartbeatTimeout, the leader sends heartbeats.
The randomized timeout becomes plain data: something outside the node picks
ElectionTimeout (randomly in production, explicitly in tests) and stores it
in the struct. A node with ElectionTimeout = 4 deterministically loses
patience before one with ElectionTimeout = 6 โ which is exactly how we will
stage elections on purpose in the final challenge.
Nodes emit, they do not send
The second determinism trick: a node never talks to a network. Stepping a node
returns the messages it wants sent and lets the caller deliver them. The
node stays a pure state machine; the messy parts (sockets, retries, delays,
partitions) live outside and can be simulated. In this challenge Tick returns
a []Message; peers are listed in Peers, and to keep everything reproducible
you must emit messages in Peers order.
What can a tick emit?
- A follower or candidate whose
ElectionElapsedreachesElectionTimeoutstarts an election exactly as in lesson 1 โ and emits aMsgRequestVoteto every peer, carrying itsLastLogIndex/LastLogTermfor the ยง5.4.1 check. - A leader whose
HeartbeatElapsedreachesHeartbeatTimeoutemits aMsgHeartbeat(an empty AppendEntries) to every peer, carrying itsLeaderCommitso followers can advance their commit index. - Everything else: no messages, just a counter bumped.
Leaders never run an election timer โ receiving valid heartbeats is the follower's job, sending them is the leader's.
โบ Drive a Node with Ticks
35 ptsImplement Tick(n *Node) []Message, one clock tick for one node.
Rules:
- Leader: increment
HeartbeatElapsed. If it is still belowHeartbeatTimeout, return no messages. Otherwise reset it to 0 and return oneMsgHeartbeatper peer, inPeersorder, withTerm = CurrentTermandLeaderCommit = CommitIndex. Leaders never touchElectionElapsed. - Follower or candidate: increment
ElectionElapsed. If it is still belowElectionTimeout, return no messages. Otherwise the election fires: resetElectionElapsedto 0, becomeCandidate, incrementCurrentTerm, setVotedFor = n.IDandVotesGranted = 1. If that is already a majority of the cluster (len(Peers)+1servers), becomeLeaderimmediately and return no messages. Otherwise return oneMsgRequestVoteper peer, inPeersorder, withTerm = CurrentTermand the node's last log index and term (1-based; empty log โ 0 and 0).
Set From and To on every message. Fields that do not apply to a message
type stay zero.
Log in to submit a solution and earn points.