Files
ytplayer/plans/done/018-server-rehydrate-from-peer-4fb8bd.md

127 lines
6.7 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
id: 018-server-rehydrate-from-peer-4fb8bd
title: Restore an evicted server copy from an online holder
created: 2026-09-29
depends_on: [017-peer-transfer-1faaa7]
est_files: 6
---
# 018 — Rehydrate the server from a device
## Objective
Implements flow 8 of `docs/p2p-architecture.md`: the owner's "the original source can go
offline, the video stays reachable from devices". After this plan, when `/api/streams`
fails at the source (removed/private video, yt-dlp failing) AND the server has no ready
copy AND a device holding a verified cid of that video is online with sharing on:
- the server sends that ONE device `{type:'upload-request', videoId, cid}` over `/ws/p2p`
(at most once per video per 10 min) and answers
`503 { ok:false, restoring:true, error:'…try again in a minute.' }`;
- the device uploads the file through intake with `restore: true`; intake accepts a known
cid again ONLY when the server no longer holds those bytes, re-hashes + re-validates,
and adopts it into the media cache — the next play is served from the server again.
Pre-tested: rehydrator + restore intake (`bun:test` hub 4/4, intake 5/5) and the whole loop in
Chromium (`plans/harness/rehydrate-*.js`: first request → `restoring:true`, device uploads,
server adopts the exact cid).
## Context the executor must NOT rediscover
- Patches (apply in order):
- `plans/patches/018-server-rehydrate.diff` — `server/p2p-hub.js` gains
`createRehydrator({ p2pDb, hub, hasServerCopy, enabled, now })`; `server/p2p-intake.js` gains the
`restore` body flag and a `serverHasCid(cid)` dep; tests for both.
- `plans/patches/018-client-restore.diff` — `frontend/p2p-client.js` answers `upload-request`
(only when sharing is on, only for the exact cid it holds, one at a time) and `contribute()`
accepts `restore`.
- `server/server.js` `/api/streams` handler ends (~line 623):
```js
} catch (err) {
return c.json({ ok: false, error: err.message }, 500);
}
});
```
(this catch covers the yt-dlp resolve path; the cached-copy path returned earlier).
- Plans 013–016 created in server.js: `fileForCid(cid)`, `const p2p = registerP2pRoutes(…)`,
`const p2pHub = createP2pHub(…)`, the `/api/p2p/holders` route, and
`const intake = registerIntakeRoutes(app, { cfg: P2P, p2pDb, gate: p2p.gate, requireDevice: p2p.requireDevice, admitFile, validateMedia: …, adopt: … });`
- `media.getReady(videoId)` resolves the ready row or null.
## Steps
1. From the repo root (STOP on failure):
```bash
git apply plans/patches/018-server-rehydrate.diff
git apply plans/patches/018-client-restore.diff
```
2. `server/server.js` — change the import from `./p2p-hub.js` to
`import { createP2pHub, holdersPayload, createRehydrator } from './p2p-hub.js';`
3. `server/server.js` — in the `registerIntakeRoutes(app, { … })` options add
`serverHasCid: async (cid) => !!(await fileForCid(cid)),`
4. `server/server.js` — directly after the `const intake = registerIntakeRoutes(…);` statement add:
```js
// Source gone + server copy evicted → ask one online holder to send it back
// through intake (docs/p2p-architecture.md flow 8).
const p2pRehydrate = createRehydrator({
p2pDb, hub: p2pHub, enabled: () => P2P.enabled,
hasServerCopy: async (id) => !!(await media.getReady(id).catch(() => null)),
});
```
5. `server/server.js` `/api/streams` — replace the final catch block shown in Context with:
```js
} catch (err) {
// The source failed. If a device holds a verified copy, ask it to send one
// to the server so the video comes back (P2P flow 8).
let restoring = false;
try { restoring = await p2pRehydrate(videoId); } catch { /* best effort */ }
if (restoring) {
return c.json({
ok: false, restoring: true,
error: 'This video is unavailable at the source — a device that has it is sending a copy to the server. Try again in a minute.',
}, 503);
}
return c.json({ ok: false, error: err.message }, 500);
}
```
(`p2pRehydrate` is declared further down the file; it is only called at request time, after startup.)
## Out of scope / do NOT touch
- The client error UI (the existing toast + Retry + plan-017 peer button already cover it).
- Do not ask more than one device per video per 10 minutes; do not upload without sharing on.
## Verification
```bash
cd /home/user/ytplayer/server && bun install >/dev/null 2>&1
bun test ./p2p-hub.test.js 2>&1 | tail -3
bun test --timeout 60000 ./p2p-intake.test.js 2>&1 | tail -3
bun run test 2>&1 | grep -E "^ *[0-9]+ (pass|fail)"
bun build server.js --target=bun --outdir=/tmp/ytp-check >/dev/null && echo SERVER_OK
cd .. && node --check frontend/p2p-client.js && echo FRONT_OK
cd server
bun ../plans/harness/rehydrate-server.js >/tmp/ytp018.log 2>&1 & SRV=$!; sleep 3
cd ../plans/harness && (npm ls playwright >/dev/null 2>&1 || npm i --no-save playwright >/dev/null 2>&1); timeout 120 node rehydrate-check.mjs
kill $SRV; true
```
Expected: hub `4 pass`; intake `5 pass`; every file `0 fail`; `SERVER_OK`; `FRONT_OK`; browser JSON
`{"accepted":1,"first":{"ok":false,"restoring":true,"error":"restoring from a device"},"adopted":[["goneAAAAAAA",true]],"second":{"ok":false,"error":"source unavailable"}}`
(the harness's stand-in `/api/streams` has no media cache, so the second answer is expected).
## Report format (executor: follow exactly)
Output ONLY the following, no other prose:
1. `git diff` (unified) of all changes.
2. Raw output of the Verification commands.
3. `Findings:` — max 10 lines.
Do not commit. Do not push. Do not touch files outside the Steps.
## Execution log
- Executor: in-session Agent (haiku). Attempts: 1. Fix rounds: 0.
- Orchestrator re-ran Verification: `p2p-hub.js`, `p2p-hub.test.js`, `p2p-intake.js`, `p2p-intake.test.js`, `p2p-client.js` byte-identical to the pre-tested versions; hub tests 4 pass, intake tests 5 pass; all 15 server test files 0 fail; `SERVER_OK`; frontend syntax ok; browser check in Chromium reproduced the expected JSON (first `/api/streams` -> `restoring: true`, holder uploaded, server adopted the exact cid). No leftover processes.
- Executor Findings (verbatim): All patches applied successfully without conflicts. All 4 plan steps executed: (1) both git patches applied, (2) import modified to include createRehydrator, (3) serverHasCid option added to registerIntakeRoutes, (4) p2pRehydrate instance created with correct parameters, (5) /api/streams catch block replaced with rehydration logic. Hub tests: 4 pass. Intake tests: 5 pass. All server tests pass (0 fail across all suites). Browser test confirms complete rehydration flow: device detects request, uploads file, server adopts it into cache. Expected JSON output matched exactly.