josh/ytplayer

Fork 0

Files

Claude 8d57a654c1 Let a device hand a file to the server for hashing and validation

2026-09-30 07:50:38 +00:00

8.8 KiB

Raw Blame History

id, title, created, depends_on, est_files

title

created

depends_on

est_files

016-intake-and-server-verification-cfe031

Let a device hand a file to the server for hashing and validation

2026-09-29

015-availability-ui-and-settings-3b9397

016 — Intake: server-side verification of device files

Objective

Implements flow 7 of docs/p2p-architecture.md — the owner's rule "a file must first be downloaded by the server and checked before its hash is added to the server DB". After this plan:

POST /api/p2p/intake (device auth) opens a 30-min ticket; PUT /api/p2p/intake/:ticket streams the bytes into P2P_INTAKE_DIR (never served). The SERVER hashes while writing, a claimed cid must match, validateMedia() must pass, then admitFile() (malware scan only when P2P_MALWARE_SCAN=1, off by default). Only then does the cid enter p2p_content (origin 'intake'), and the uploader becomes a holder (trust 'challenged').
If the server has no copy of that video, the file is adopted into the media cache byte-for-byte (media.adoptFile, new) so it can be streamed again. Otherwise deleted.
Settings → Sharing shows "Verify & share N saved videos" for saves whose hash the server doesn't know yet (old saves, fallback-path saves); one tap uploads them one by one. Nothing uploads automatically in this plan.

Pre-tested: intake 4/4 (valid, hash-mismatch, truncated, non-media, size limits, ticket reuse, scan-infected), media-cache 27/27 (incl. adoptFile), and the browser flow (unknown → contribute → accepted) in Chromium.

Context the executor must NOT rediscover

Patches (apply in this order from the repo root):
- plans/patches/016-p2p-intake-new.diff → new server/p2p-intake.js (registerIntakeRoutes) + server/p2p-intake.test.js
- plans/patches/016-media-cache-adopt.diff → adoptFile(id, src, { sha256, probe, meta }) in server/media-cache.js + a test
- plans/patches/016-client-intake.diff → P2PClient.contribute(videoId, { cid, title, channel }), P2PClient.unknownVideos() in frontend/p2p-client.js; OPFS.getFileObject(videoId) in frontend/opfs.js
server/server.js imports { createMediaCache, HIGH, LOW } from './media-cache.js'; FFMPEG const exists; ffprobe path is process.env.FFPROBE_PATH || 'ffprobe' (used in createMediaCache).
Plan 013/014 added in server.js: const p2p = registerP2pRoutes(app, …) (exposes gate, requireDevice) and const p2pHub = createP2pHub(…). Plan 009 imported admitFile, P2P, * as p2pDb.

Plan 015 added to renderSettings() the Sharing group ending with:

       <div class="set-row">
         <span>This device<small id="p2pDeviceInfo">…</small></span>
         <span id="p2pHoldingCount" class="set-stat">…</span>
       </div>
     </div>

and an (async () => { const info = $('p2pDeviceInfo'); … cnt.textContent = …; })(); block.

Settings buttons use class="btn". videoById(id) (app.js ~7301) returns the known video object (title/channel) or undefined. toast(msg) shows a toast.

Steps

Apply the three patches (STOP and report on any failure):

git apply plans/patches/016-p2p-intake-new.diff
git apply plans/patches/016-media-cache-adopt.diff
git apply plans/patches/016-client-intake.diff

server/package.json "test" script — append && bun test --timeout 60000 ./p2p-intake.test.js.

server/server.js: a. change the media-cache import to import { createMediaCache, HIGH, LOW, validateMedia } from './media-cache.js'; b. add import { registerIntakeRoutes } from './p2p-intake.js'; c. directly after the /api/p2p/holders route (plan 014) add:

// Device → server intake: the server hashes, validates (and scans when
// P2P_MALWARE_SCAN=1) before a cid is admitted. docs/p2p-architecture.md flow 7.
const intake = registerIntakeRoutes(app, {
  cfg: P2P, p2pDb, gate: p2p.gate, requireDevice: p2p.requireDevice, admitFile,
  validateMedia: (path, expected, opts) => validateMedia(path, expected,
    { ...opts, ffmpeg: FFMPEG, ffprobe: process.env.FFPROBE_PATH || 'ffprobe' }),
  adopt: (videoId, path, info) => media.adoptFile(videoId, path, info),
});

docker-compose.yml — under the P2P env lines from plan 008 add the commented lines:

   # P2P_INTAKE_DIR: "/app/data/p2p-intake"   # quarantine for device uploads (never served)
   # P2P_INTAKE_MAX_BYTES: "3221225472"       # 3 GiB

frontend/app.js renderSettings template — directly BEFORE the closing </div> of the Sharing group (i.e. after the This device row), insert:

      <div class="set-row hidden" id="p2pContributeRow">
        <span>
          Saved videos the server can't verify yet
          <small>Uploading lets the server check them (hash + media validation) so other devices can get them. Uses your upload bandwidth.</small>
        </span>
        <button id="p2pContributeBtn" class="btn">Verify &amp; share</button>
      </div>

frontend/app.js renderSettings — inside the plan-015 (async () => { … })(); block, after the cnt.textContent = …; line, add:

    const unknown = window.P2PClient ? await window.P2PClient.unknownVideos() : [];
    const row = $('p2pContributeRow');
    const btn = $('p2pContributeBtn');
    if (row && btn && unknown.length) {
      row.classList.remove('hidden');
      btn.textContent = `Verify & share ${unknown.length} saved video${unknown.length === 1 ? '' : 's'}`;
      btn.onclick = async () => {
        btn.disabled = true;
        let ok = 0;
        for (let i = 0; i < unknown.length; i++) {
          const v = videoById(unknown[i]) || {};
          btn.textContent = `Uploading ${i + 1} of ${unknown.length}…`;
          const r = await window.P2PClient.contribute(unknown[i], { title: v.title || '', channel: v.channel || '' });
          if (r && r.ok) ok++;
        }
        toast(`Verified ${ok} of ${unknown.length} saved video${unknown.length === 1 ? '' : 's'}`);
        btn.disabled = false;
        row.classList.add('hidden');
      };
    }

Out of scope / do NOT touch

No automatic uploads (plan 018 adds server-requested ones). No UI for the admin (plan 019).
Never serve files from P2P_INTAKE_DIR; never add a /api/p2p/intake GET.

Verification

cd /home/user/ytplayer/server && bun install >/dev/null 2>&1
which ffmpeg ffprobe || echo "NO FFMPEG — install it (apt-get install -y ffmpeg); intake/media tests need it"
bun test --timeout 60000 ./p2p-intake.test.js 2>&1 | tail -4
bun test --timeout 60000 ./media-cache.test.js 2>&1 | tail -4
bun run test 2>&1 | grep -E "^ *[0-9]+ (pass|fail)"
bun build server.js --target=bun --outdir=/tmp/ytp-check >/dev/null && echo SERVER_OK
cd .. && node --check frontend/app.js frontend/p2p-client.js frontend/opfs.js && echo FRONT_OK
cd server
bun ../plans/harness/intake-server.js >/tmp/ytp016.log 2>&1 & SRV=$!; sleep 3
cd ../plans/harness && (npm ls playwright >/dev/null 2>&1 || npm i --no-save playwright >/dev/null 2>&1); timeout 90 node intake-check.mjs
kill $SRV; true

Expected: intake 4 pass; media-cache 27 pass; every file 0 fail; SERVER_OK; FRONT_OK; browser JSON {"firstUnknown":1,"unknown":["upAAAAAAAA9"],"contribute":{"ok":true,"adopted":false,"cidOk":true},"secondAccepted":1,"secondUnknown":0}.

Report format (executor: follow exactly)

Output ONLY the following, no other prose:

git diff (unified) of all changes.
Raw output of the Verification commands.
Findings: — max 10 lines.

Do not commit. Do not push. Do not touch files outside the Steps.

Execution log

Executor: in-session Agent (haiku). Attempts: 1. Fix rounds: 0.
Orchestrator re-ran Verification: p2p-intake.js, p2p-intake.test.js, media-cache.js, media-cache.test.js, p2p-client.js, opfs.js byte-identical to the pre-tested 016 versions; intake tests 4 pass; media-cache 27 pass; all 14 server test files 0 fail; SERVER_OK; frontend syntax ok; browser check in Chromium reproduced the expected JSON (unknown file -> contribute -> accepted, cid valid, second report accepted 1 / unknown 0). No leftover processes.
Executor Findings (verbatim): All three patches applied without errors. server/package.json updated to run p2p-intake.test.js. server/server.js updated with validateMedia import and registerIntakeRoutes import and intake route initialization. frontend/app.js updated with HTML and JS for contribute button. docker-compose.yml updated with commented P2P_INTAKE_DIR and P2P_INTAKE_MAX_BYTES. All 4 expected intake tests passed, all 27 media-cache tests passed, full test suite passed with 0 failures across all test files. Server build successful. Frontend syntax check passed. Browser test output matches expected JSON exactly.

8.8 KiB Raw Blame History