Reference
File state machine
The 7-state `file_locations` shard lifecycle for Path C.
The Relay's file_locations table is the single source of truth for the
lifecycle of each (file_id, version_number, shard_index, node_id) shard.
Every shard transitions through the fixed sequence below; the states are the
only legal values of the status column.
States
CREATED ──► UPLOADING ──► RELAY_BUFFERED ──► NODE_RECEIVING ──► NODE_VERIFIED ──► NODE_STORED ──► RELAY_CLEANUP| State | Meaning | Owner of the bytes | Terminal? |
|---|---|---|---|
CREATED | Shard slot reserved (schema default); unused by the HTTP upload path | — | no |
UPLOADING | Client is mid-POST /buffer/upload; transient reservation before commit | client | no |
RELAY_BUFFERED | Bytes committed to the Relay buffer (buffer_id set); awaiting node pickup | Relay | no |
NODE_RECEIVING | Node consumed the single-use fetch token; bytes in flight to the node | node (transfer) | no |
NODE_VERIFIED | Node acked verified; Relay copy still on disk until cleanup | node | no |
NODE_STORED | Node owns the shard; Relay buffer_id cleared and file deleted | node | yes |
RELAY_CLEANUP | Shard abandoned (TTL expiry, crashed upload, failed pickup) | — | yes |
The transitions that Shift-state by acks/sweeps:
RELAY_BUFFERED → NODE_RECEIVING: node callsGET /buffer/fetchwith a valid single-use token bound to that node plus itsX-Nodus-*request signature (handler/buffer_fetch.go, guarded by a status check so a second fetch is409; a token redeemed by a different node is403).NODE_RECEIVING → NODE_VERIFIED → NODE_STORED: node ackshard_ack status="verified"(handler/ws.gohandleShardAckVerified).NODE_RECEIVING → RELAY_BUFFERED: node ackstatus="failed"; the buffer file is kept for redelivery (handleShardAckFailed).UPLOADING → RELAY_CLEANUP: TTL sweep, crashed client (>1 h grace, seebuffer/buffer.gosweepOrphanedUploading).RELAY_BUFFERED → RELAY_CLEANUP: TTL sweep runs down the buffer's lease (default 72 h,cfg.BufferTTL).NODE_RECEIVING/NODE_VERIFIED → RELAY_BUFFERED: recovery sweep reverts shards whose node vanished mid-transfer (>30 min grace,sweepStuckInTransit).
Recovery / redelivery
- On (re)connect and
register, the Relay scans allRELAY_BUFFEREDshards for the node and re-notifies viapending_notifywith a fresh fetch token (handler/ws.gocheckAndDeliverPendingShards). Tokens are re-issued on every reconnect because a token minted while the node was offline may expire. - A node crash between fetch and ack is self-healing: the recovery sweep returns
the shard to
RELAY_BUFFEREDand the next reconnect delivers it again.
Invariants
- Terminal states (
NODE_STORED,RELAY_CLEANUP) are never left. - A
buffer_idis set exactly while the Relay owns the bytes (RELAY_BUFFERED, in-transit states) and cleared the moment custody passes to the node (NODE_STORED) or the file is reclaimed (RELAY_CLEANUP). - Duplicate/stale acks are rejected:
verified/failedonly apply fromNODE_RECEIVING, and the fetch endpoint refuses a second token for the same row.