Let a device hand a file to the server for hashing and validation
This commit is contained in:
Claude
@@ -1,150 +0,0 @@
|
||||
---
|
||||
id: 016-intake-and-server-verification-cfe031
|
||||
title: Let a device hand a file to the server for hashing and validation
|
||||
created: 2026-09-29
|
||||
depends_on: [015-availability-ui-and-settings-3b9397]
|
||||
est_files: 9
|
||||
---
|
||||
|
||||
# 016 — Intake: server-side verification of device files
|
||||
|
||||
## Objective
|
||||
|
||||
Implements flow 7 of `docs/p2p-architecture.md` — the owner's rule "a file must first
|
||||
be downloaded by the server and checked before its hash is added to the server DB".
|
||||
After this plan:
|
||||
- `POST /api/p2p/intake` (device auth) opens a 30-min ticket; `PUT /api/p2p/intake/:ticket`
|
||||
streams the bytes into `P2P_INTAKE_DIR` (never served). The SERVER hashes while writing,
|
||||
a claimed cid must match, `validateMedia()` must pass, then `admitFile()` (malware scan
|
||||
only when `P2P_MALWARE_SCAN=1`, off by default). Only then does the cid enter
|
||||
`p2p_content` (`origin 'intake'`), and the uploader becomes a holder (`trust 'challenged'`).
|
||||
- If the server has no copy of that video, the file is adopted into the media cache
|
||||
byte-for-byte (`media.adoptFile`, new) so it can be streamed again. Otherwise deleted.
|
||||
- Settings → Sharing shows "Verify & share N saved videos" for saves whose hash the server
|
||||
doesn't know yet (old saves, fallback-path saves); one tap uploads them one by one.
|
||||
Nothing uploads automatically in this plan.
|
||||
|
||||
Pre-tested: intake 4/4 (valid, hash-mismatch, truncated, non-media, size limits, ticket
|
||||
reuse, scan-infected), media-cache 27/27 (incl. `adoptFile`), and the browser flow
|
||||
(unknown → contribute → accepted) in Chromium.
|
||||
|
||||
## Context the executor must NOT rediscover
|
||||
|
||||
- Patches (apply in this order from the repo root):
|
||||
- `plans/patches/016-p2p-intake-new.diff` → new `server/p2p-intake.js` (`registerIntakeRoutes`) + `server/p2p-intake.test.js`
|
||||
- `plans/patches/016-media-cache-adopt.diff` → `adoptFile(id, src, { sha256, probe, meta })` in `server/media-cache.js` + a test
|
||||
- `plans/patches/016-client-intake.diff` → `P2PClient.contribute(videoId, { cid, title, channel })`,
|
||||
`P2PClient.unknownVideos()` in `frontend/p2p-client.js`; `OPFS.getFileObject(videoId)` in `frontend/opfs.js`
|
||||
- `server/server.js` imports `{ createMediaCache, HIGH, LOW } from './media-cache.js'`; `FFMPEG`
|
||||
const exists; ffprobe path is `process.env.FFPROBE_PATH || 'ffprobe'` (used in createMediaCache).
|
||||
- Plan 013/014 added in server.js: `const p2p = registerP2pRoutes(app, …)` (exposes `gate`,
|
||||
`requireDevice`) and `const p2pHub = createP2pHub(…)`. Plan 009 imported `admitFile`, `P2P`, `* as p2pDb`.
|
||||
- Plan 015 added to `renderSettings()` the Sharing group ending with:
|
||||
```html
|
||||
<div class="set-row">
|
||||
<span>This device<small id="p2pDeviceInfo">…</small></span>
|
||||
<span id="p2pHoldingCount" class="set-stat">…</span>
|
||||
</div>
|
||||
</div>
|
||||
```
|
||||
and an `(async () => { const info = $('p2pDeviceInfo'); … cnt.textContent = …; })();` block.
|
||||
- Settings buttons use `class="btn"`. `videoById(id)` (app.js ~7301) returns the known video object
|
||||
(title/channel) or undefined. `toast(msg)` shows a toast.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Apply the three patches (STOP and report on any failure):
|
||||
```bash
|
||||
git apply plans/patches/016-p2p-intake-new.diff
|
||||
git apply plans/patches/016-media-cache-adopt.diff
|
||||
git apply plans/patches/016-client-intake.diff
|
||||
```
|
||||
2. `server/package.json` "test" script — append ` && bun test --timeout 60000 ./p2p-intake.test.js`.
|
||||
3. `server/server.js`:
|
||||
a. change the media-cache import to `import { createMediaCache, HIGH, LOW, validateMedia } from './media-cache.js';`
|
||||
b. add `import { registerIntakeRoutes } from './p2p-intake.js';`
|
||||
c. directly after the `/api/p2p/holders` route (plan 014) add:
|
||||
```js
|
||||
// Device → server intake: the server hashes, validates (and scans when
|
||||
// P2P_MALWARE_SCAN=1) before a cid is admitted. docs/p2p-architecture.md flow 7.
|
||||
const intake = registerIntakeRoutes(app, {
|
||||
cfg: P2P, p2pDb, gate: p2p.gate, requireDevice: p2p.requireDevice, admitFile,
|
||||
validateMedia: (path, expected, opts) => validateMedia(path, expected,
|
||||
{ ...opts, ffmpeg: FFMPEG, ffprobe: process.env.FFPROBE_PATH || 'ffprobe' }),
|
||||
adopt: (videoId, path, info) => media.adoptFile(videoId, path, info),
|
||||
});
|
||||
```
|
||||
4. `docker-compose.yml` — under the P2P env lines from plan 008 add the commented lines:
|
||||
```yaml
|
||||
# P2P_INTAKE_DIR: "/app/data/p2p-intake" # quarantine for device uploads (never served)
|
||||
# P2P_INTAKE_MAX_BYTES: "3221225472" # 3 GiB
|
||||
```
|
||||
5. `frontend/app.js` `renderSettings` template — directly BEFORE the closing `</div>` of the
|
||||
Sharing group (i.e. after the `This device` row), insert:
|
||||
```html
|
||||
<div class="set-row hidden" id="p2pContributeRow">
|
||||
<span>
|
||||
Saved videos the server can't verify yet
|
||||
<small>Uploading lets the server check them (hash + media validation) so other devices can get them. Uses your upload bandwidth.</small>
|
||||
</span>
|
||||
<button id="p2pContributeBtn" class="btn">Verify & share</button>
|
||||
</div>
|
||||
```
|
||||
6. `frontend/app.js` `renderSettings` — inside the plan-015 `(async () => { … })();` block, after the
|
||||
`cnt.textContent = …;` line, add:
|
||||
```js
|
||||
const unknown = window.P2PClient ? await window.P2PClient.unknownVideos() : [];
|
||||
const row = $('p2pContributeRow');
|
||||
const btn = $('p2pContributeBtn');
|
||||
if (row && btn && unknown.length) {
|
||||
row.classList.remove('hidden');
|
||||
btn.textContent = `Verify & share ${unknown.length} saved video${unknown.length === 1 ? '' : 's'}`;
|
||||
btn.onclick = async () => {
|
||||
btn.disabled = true;
|
||||
let ok = 0;
|
||||
for (let i = 0; i < unknown.length; i++) {
|
||||
const v = videoById(unknown[i]) || {};
|
||||
btn.textContent = `Uploading ${i + 1} of ${unknown.length}…`;
|
||||
const r = await window.P2PClient.contribute(unknown[i], { title: v.title || '', channel: v.channel || '' });
|
||||
if (r && r.ok) ok++;
|
||||
}
|
||||
toast(`Verified ${ok} of ${unknown.length} saved video${unknown.length === 1 ? '' : 's'}`);
|
||||
btn.disabled = false;
|
||||
row.classList.add('hidden');
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
## Out of scope / do NOT touch
|
||||
|
||||
- No automatic uploads (plan 018 adds server-requested ones). No UI for the admin (plan 019).
|
||||
- Never serve files from `P2P_INTAKE_DIR`; never add a `/api/p2p/intake` GET.
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
cd /home/user/ytplayer/server && bun install >/dev/null 2>&1
|
||||
which ffmpeg ffprobe || echo "NO FFMPEG — install it (apt-get install -y ffmpeg); intake/media tests need it"
|
||||
bun test --timeout 60000 ./p2p-intake.test.js 2>&1 | tail -4
|
||||
bun test --timeout 60000 ./media-cache.test.js 2>&1 | tail -4
|
||||
bun run test 2>&1 | grep -E "^ *[0-9]+ (pass|fail)"
|
||||
bun build server.js --target=bun --outdir=/tmp/ytp-check >/dev/null && echo SERVER_OK
|
||||
cd .. && node --check frontend/app.js frontend/p2p-client.js frontend/opfs.js && echo FRONT_OK
|
||||
cd server
|
||||
bun ../plans/harness/intake-server.js >/tmp/ytp016.log 2>&1 & SRV=$!; sleep 3
|
||||
cd ../plans/harness && (npm ls playwright >/dev/null 2>&1 || npm i --no-save playwright >/dev/null 2>&1); timeout 90 node intake-check.mjs
|
||||
kill $SRV; true
|
||||
```
|
||||
|
||||
Expected: intake `4 pass`; media-cache `27 pass`; every file `0 fail`; `SERVER_OK`; `FRONT_OK`;
|
||||
browser JSON `{"firstUnknown":1,"unknown":["upAAAAAAAA9"],"contribute":{"ok":true,"adopted":false,"cidOk":true},"secondAccepted":1,"secondUnknown":0}`.
|
||||
|
||||
## Report format (executor: follow exactly)
|
||||
|
||||
Output ONLY the following, no other prose:
|
||||
|
||||
1. `git diff` (unified) of all changes.
|
||||
2. Raw output of the Verification commands.
|
||||
3. `Findings:` — max 10 lines.
|
||||
|
||||
Do not commit. Do not push. Do not touch files outside the Steps.
|
||||
Reference in New Issue
Block a user