018 — Rehydrate the server from a device

Objective

Implements flow 8 of docs/p2p-architecture.md: the owner's "the original source can go offline, the video stays reachable from devices". After this plan, when /api/streams fails at the source (removed/private video, yt-dlp failing) AND the server has no ready copy AND a device holding a verified cid of that video is online with sharing on:

the server sends that ONE device {type:'upload-request', videoId, cid} over /ws/p2p (at most once per video per 10 min) and answers 503 { ok:false, restoring:true, error:'…try again in a minute.' };
the device uploads the file through intake with restore: true; intake accepts a known cid again ONLY when the server no longer holds those bytes, re-hashes + re-validates, and adopts it into the media cache — the next play is served from the server again.

Pre-tested: rehydrator + restore intake (bun:test hub 4/4, intake 5/5) and the whole loop in Chromium (plans/harness/rehydrate-*.js: first request → restoring:true, device uploads, server adopts the exact cid).

Context the executor must NOT rediscover

Patches (apply in order):
- plans/patches/018-server-rehydrate.diff — server/p2p-hub.js gains createRehydrator({ p2pDb, hub, hasServerCopy, enabled, now }); server/p2p-intake.js gains the restore body flag and a serverHasCid(cid) dep; tests for both.
- plans/patches/018-client-restore.diff — frontend/p2p-client.js answers upload-request (only when sharing is on, only for the exact cid it holds, one at a time) and contribute() accepts restore.
server/server.js /api/streams handler ends (~line 623):
```
  } catch (err) {
    return c.json({ ok: false, error: err.message }, 500);
  }
});
```
(this catch covers the yt-dlp resolve path; the cached-copy path returned earlier).
Plans 013–016 created in server.js: fileForCid(cid), const p2p = registerP2pRoutes(…), const p2pHub = createP2pHub(…), the /api/p2p/holders route, and const intake = registerIntakeRoutes(app, { cfg: P2P, p2pDb, gate: p2p.gate, requireDevice: p2p.requireDevice, admitFile, validateMedia: …, adopt: … });
media.getReady(videoId) resolves the ready row or null.

Steps

From the repo root (STOP on failure):

git apply plans/patches/018-server-rehydrate.diff
git apply plans/patches/018-client-restore.diff

server/server.js — change the import from ./p2p-hub.js to import { createP2pHub, holdersPayload, createRehydrator } from './p2p-hub.js';
server/server.js — in the registerIntakeRoutes(app, { … }) options add serverHasCid: async (cid) => !!(await fileForCid(cid)),

server/server.js — directly after the const intake = registerIntakeRoutes(…); statement add:

// Source gone + server copy evicted → ask one online holder to send it back
// through intake (docs/p2p-architecture.md flow 8).
const p2pRehydrate = createRehydrator({
  p2pDb, hub: p2pHub, enabled: () => P2P.enabled,
  hasServerCopy: async (id) => !!(await media.getReady(id).catch(() => null)),
});

server/server.js /api/streams — replace the final catch block shown in Context with:

  } catch (err) {
    // The source failed. If a device holds a verified copy, ask it to send one
    // to the server so the video comes back (P2P flow 8).
    let restoring = false;
    try { restoring = await p2pRehydrate(videoId); } catch { /* best effort */ }
    if (restoring) {
      return c.json({
        ok: false, restoring: true,
        error: 'This video is unavailable at the source — a device that has it is sending a copy to the server. Try again in a minute.',
      }, 503);
    }
    return c.json({ ok: false, error: err.message }, 500);
  }

(p2pRehydrate is declared further down the file; it is only called at request time, after startup.)

Out of scope / do NOT touch

The client error UI (the existing toast + Retry + plan-017 peer button already cover it).
Do not ask more than one device per video per 10 minutes; do not upload without sharing on.

Verification

cd /home/user/ytplayer/server && bun install >/dev/null 2>&1
bun test ./p2p-hub.test.js 2>&1 | tail -3
bun test --timeout 60000 ./p2p-intake.test.js 2>&1 | tail -3
bun run test 2>&1 | grep -E "^ *[0-9]+ (pass|fail)"
bun build server.js --target=bun --outdir=/tmp/ytp-check >/dev/null && echo SERVER_OK
cd .. && node --check frontend/p2p-client.js && echo FRONT_OK
cd server
bun ../plans/harness/rehydrate-server.js >/tmp/ytp018.log 2>&1 & SRV=$!; sleep 3
cd ../plans/harness && (npm ls playwright >/dev/null 2>&1 || npm i --no-save playwright >/dev/null 2>&1); timeout 120 node rehydrate-check.mjs
kill $SRV; true

Expected: hub 4 pass; intake 5 pass; every file 0 fail; SERVER_OK; FRONT_OK; browser JSON {"accepted":1,"first":{"ok":false,"restoring":true,"error":"restoring from a device"},"adopted":[["goneAAAAAAA",true]],"second":{"ok":false,"error":"source unavailable"}} (the harness's stand-in /api/streams has no media cache, so the second answer is expected).

Report format (executor: follow exactly)

Output ONLY the following, no other prose:

git diff (unified) of all changes.
Raw output of the Verification commands.
Findings: — max 10 lines.

Do not commit. Do not push. Do not touch files outside the Steps.

Execution log

Executor: in-session Agent (haiku). Attempts: 1. Fix rounds: 0.
Orchestrator re-ran Verification: p2p-hub.js, p2p-hub.test.js, p2p-intake.js, p2p-intake.test.js, p2p-client.js byte-identical to the pre-tested versions; hub tests 4 pass, intake tests 5 pass; all 15 server test files 0 fail; SERVER_OK; frontend syntax ok; browser check in Chromium reproduced the expected JSON (first /api/streams -> restoring: true, holder uploaded, server adopted the exact cid). No leftover processes.
Executor Findings (verbatim): All patches applied successfully without conflicts. All 4 plan steps executed: (1) both git patches applied, (2) import modified to include createRehydrator, (3) serverHasCid option added to registerIntakeRoutes, (4) p2pRehydrate instance created with correct parameters, (5) /api/streams catch block replaced with rehydration logic. Hub tests: 4 pass. Intake tests: 5 pass. All server tests pass (0 fail across all suites). Browser test confirms complete rehydration flow: device detects request, uploads file, server adopts it into cache. Expected JSON output matched exactly.

6.7 KiB Raw Permalink Blame History Unescape Escape