Transfer Manager spec

Shared contract for the TypeScript client-side (packages/transfer-manager) and Rust node-side (services/storage-node/src/transfer/) implementations. Both sides honor the same path enum, state machine, timeout table, and cache semantics — the spec is the source of truth, not shared code.

Transfer PathsCopy Anchor Link

Four paths, tried in order. Paths A/B share a WebRTC DataChannel transport but differ in how the signaling channel is established.

Per-Shard State MachineCopy Anchor Link

PENDING ──► ATTEMPTING(path) ──► SUCCEEDED
                  │
                  ├─► (retry same path with backoff)
                  │
                  └─► FAILED_PERMANENT (all paths exhausted)

States:

• PENDING — shard transfer not yet started.
• ATTEMPTING(path) — actively trying a specific transfer path.
• SUCCEEDED — shard successfully transferred and verified.
• FAILED_PERMANENT — all paths exhausted; shard transfer will not succeed without operator intervention or new information.

Fallback SequenceCopy Anchor Link

1. Path A: Local signaling → fresh mDNS discovery → WebRTC negotiation → DataChannel transfer
   └─ failure →
2. Path B: Relay signaling → WS SDP exchange → WebRTC negotiation → DataChannel transfer
   └─ failure →
3. Path C: Buffer-and-Relay → POST /buffer/upload → relay stores → node fetches → relay deletes
   └─ relay unavailable →
4. Path D: Local persistent queue → store shard locally → retry on connectivity restoration

Failure types per stageCopy Anchor Link

Each stage in paths A/B has two distinct failure modes:

1. Signaling failure — couldn't establish the signaling channel at all (mDNS timeout, HTTP error, WS disconnect). Skip straight to next signaling method.

2. WebRTC negotiation failure — signaling succeeded (SDP offer/answer exchanged) but the DataChannel never opened within the timeout window. Also falls through; this is the case the ~4s benchmark targets.

Path cache behaviorCopy Anchor Link

On cache hit for node_id:

1. Attempt the cached path first (using fresh mDNS if LOCAL_SIGNALING — never cache the IP, only the method).
2. On success → update cache with same path and fresh timestamp.
3. On failure → evict cache entry immediately, then fall through the remaining chain. Never retry a path that was just confirmed broken.

On cache miss:

• Start from path A and walk the full chain.

Timeout TableCopy Anchor Link

All values configurable via TransferConfig. Defaults below; tune after benchmarking (see scripts/benchmark-webrtc/).

Backoff & RetryCopy Anchor Link

Exponential backoff with jitter applied between retries of the same stage before falling through to the next path.

Formula:

delay = base_ms * 2^attempt + random(0, jitter_ms)

Jitter bound nuance: the TS side samples [0, jitter) (exclusive upper bound); the Rust side samples [0, jitter] (inclusive) because rand::gen_range(0..0) would panic when jitter is configured to 0. The ≤1ms difference is immaterial to the scheduling contract.

Default constants:

• backoff_base_ms: 500
• backoff_jitter_ms: 300
• max_retries_per_stage: 2

After 2 failed attempts at a stage, fall through to the next path without further retries.

ConcurrencyCopy Anchor Link

Both implementations use a bounded concurrency pool for per-shard transfers.

• Default max_concurrency: 4
• Excess transfers queued FIFO and picked up as slots free
• Each queued transfer runs its own independent state machine (not a single global state machine processing one shard at a time)

Path Cache PersistenceCopy Anchor Link

Rust node-sideCopy Anchor Link

Columns on trusted_nodes:

• last_successful_path TEXT — enum value of the last successful path
• last_success_at TEXT — ISO 8601 timestamp of last successful transfer

Read/write in transfer/cache.rs. Updated on successful transfer; evicted immediately on failure of a cached path.

TypeScript client-sideCopy Anchor Link

In-memory Map<string, { path, lastSuccessAt }> with a generic PathCache interface. Phase 14 plugs in IndexedDB persistence for the web (apps/web/lib/transfer/path-cache.ts, IndexedDBPathCache); SQLite for mobile remains a later backend. The interface is:

interface PathCache {
  get(nodeId: string): PathCacheEntry | undefined;
  set(nodeId: string, path: TransferPath): void;
  evict(nodeId: string): void;
}

Local Persistent Queue (Path D)Copy Anchor Link

Generic interface with platform-specific backends:

interface LocalQueue {
  enqueue(item: QueueItem): void;
  dequeue(): QueueItem | undefined;
  peek(): QueueItem | undefined;
  remove(transferId: string): void;
  onConnectivityRestored(callback: () => void): void;
  notifyConnectivityRestored(): void;
  readonly size: number;
}

Default implementation: MemoryLocalQueue (in-process, not persisted). Phase 14 adds IndexedDBLocalQueue (apps/web/lib/transfer/local-queue.ts) for web; SQLiteLocalQueue for mobile remains a later backend.

Retry scheduling is triggered on connectivity restoration or Relay reachability — not a blind timer.

Re-fetch-from-peer (§21 Repair)Copy Anchor Link

When reconciliation marks an object DEGRADED, the repair path:

1. Query trusted_nodes for all peers.
2. Order: path-cache hits first, then by last_success_at descending.
3. Submit one fetch_shard(peer, object_id) per DEGRADED object to the best-candidate peer only (v1: no N×M fan-out — the executor has no peer-fallback loop, and failed repairs are retried by the next scan).
4. On success: BLAKE3-verify the received bytes against object_id, then atomically write (<data_dir>/objects/<ab>/<object_id>, temp-file + rename) and flip storage_objects DEGRADED → STORED.
5. On permanent failure: leave DEGRADED; retry on next reconciliation scan.

object_id (the BLAKE3 content hash) is threaded through ShardTransferRequest and TransferResult on both sides — it is both the fetch target and the key under which restored bytes are verified and written.

Node→node Path A transport (v1)Copy Anchor Link

Client-initiated Path A still uses live mDNS discovery + WebRTC (/nodus/webrtc/*), but the node→node repair backhaul uses plain authenticated HTTP on the same advertised port instead of WebRTC: the peer's WebRtcManager only accepts inbound offers from paired devices, not other nodes. The v1 repair Path A is:

• GET /nodus/shard/{object_id} served by the receiving node.
• Stateless authentication via X-Nodus-* headers on the request:
  • X-Nodus-Node-Id: the requester's hex node id.
  • X-Nodus-Timestamp: unix epoch milliseconds.
  • X-Nodus-Signature: hex Ed25519 signature over "{node_id}:{object_id}:{timestamp_ms}".
• The receiving node checks the caller is in trusted_nodes, the timestamp is within ±60 s, and the signature verifies against the caller's stored public key. Non-2xx/404 responses surface as a failed attempt; the reconcile loop hash-verifies the payload before restoring. Paths B/C/D remain honest failures for node→node repair (see transfer/node_attempter.rs).