plan: start 012-device-file-registry-288d55

2026-09-30 07:27:31 +00:00
parent b56be586ee
commit 8b7b2a62bd
2 changed files with 1 additions and 1 deletions
--- a/plans/active/012-device-file-registry-288d55.md
+++ b/plans/active/012-device-file-registry-288d55.md
@@ -0,0 +1,215 @@
+---
+id: 012-device-file-registry-288d55
+title: Add the on-device IndexedDB file registry and hash saves while downloading
+created: 2026-09-29
+depends_on: [009-server-content-hash-186e7f, 011-browser-sha256-e1793d]
+est_files: 6
+---
+
+# 012 — On-device file registry + hash while saving
+
+## Objective
+
+Implements flow 2 of `docs/p2p-architecture.md`. After this plan:
+- `frontend/device-db.js` (`window.DeviceDB`) keeps one IndexedDB record per saved
+  video: `{ videoId, cid, size, savedAt, lastCheckedAt, state }`.
+- The OPFS download worker hashes bytes as it writes them. If the server sent
+  `X-Content-SHA256` (plan 009) and the hash differs, the save FAILS and the partial
+  file is deleted ("integrity check failed").
+- `preload()` records the file (`state: 'verified'` when the hashes matched,
+  `'unverified'` when the server sent none, `'unhashed'` for the main-thread fallback);
+  deleting / clearing saved videos removes the records.
+
+Pre-tested in Chromium with the harness in `plans/harness/` (good hash → saved and
+`verified`; wrong hash → rejected, no file left; no header → saved with a local hash).
+
+## Context the executor must NOT rediscover
+
+- `plans/patches/012-opfs-hash.diff` patches `frontend/opfs-worker.js` (importScripts
+  `/sha256.js`, hash in the write loop, compare with `x-content-sha256`, post
+  `{ type:'done', ext, sha256, expectedSha, size }`) and `frontend/opfs.js`
+  (`downloadVideo` resolves `{ ok:true, sha256, expectedSha, size }`).
+- `frontend/app.js:157-198` `async function opfsDownload(videoId, { mux = false } = {})` — worker path:
+  ```js
+    if (typeof window.OPFS.downloadVideo === 'function' && typeof Worker !== 'undefined') {
+      const w = await window.OPFS.downloadVideo(videoId, url);
+      if (w.ok) return { ok: true, cached: true };
+      workerError = w.error || null;
+    }
+  ```
+- `frontend/app.js:258-268`:
+  ```js
+  async function opfsDelete(videoId) {
+    if (!window.OPFS || !window.OPFS.isSupported()) return { ok: true };
+    try { await window.OPFS.deleteVideo(videoId); } catch { /* ignore */ }
+    return { ok: true };
+  }
+
+  async function opfsClear() {
+    if (!window.OPFS || !window.OPFS.isSupported()) return { ok: true };
+    try { await window.OPFS.clearAll(); } catch { /* ignore */ }
+    return { ok: true };
+  }
+  ```
+- `frontend/app.js` `async function preload(video, …)` (~line 1237), success branch:
+  ```js
+      const res = await API.cacheDownload(id, { mux });
+      if (res && res.ok && res.cached) {
+        cachedIds.add(id);
+        cacheMutations++;
+        warmThumb(thumbUrlFor(id, video));
+  ```
+- Script order in `frontend/index.html` after plan 011: `… stats-core.js, sha256.js, async-guard.js, sw-update.js, app.js`.
+- `frontend/sw.js` `SHELL` after plan 011 contains `'/sha256.js',`.
+
+## Steps
+
+1. Create `frontend/device-db.js` — copy VERBATIM from the Appendix.
+2. From the repo root: `git apply plans/patches/012-opfs-hash.diff` (STOP and report on failure).
+3. `frontend/index.html` — add `    <script src="device-db.js"></script>` directly after the `sha256.js` line.
+4. `frontend/sw.js` `SHELL` — add `'/device-db.js',` directly after `'/sha256.js',`.
+5. `frontend/app.js` `opfsDownload` — the line `if (w.ok) return { ok: true, cached: true };` appears
+   TWICE in app.js; change ONLY the first one (inside `async function opfsDownload`, ~line 180),
+   NOT the one inside `opfsDownloadEdited` (edited cuts are never shared). Change it to:
+   ```js
+       if (w.ok) return { ok: true, cached: true, sha256: w.sha256 || null, expectedSha: w.expectedSha || null, size: w.size || 0 };
+   ```
+   (The main-thread fallback below keeps returning `{ ok: true, cached: true }` — no hash.)
+6. `frontend/app.js` — directly ABOVE `async function preload(video, …)` add:
+   ```js
+   // This device's record of what it holds and each file's content id
+   // (docs/p2p-architecture.md flow 2). The P2P client reports these.
+   function recordDeviceFile(id, res) {
+     if (!WEB || !window.DeviceDB) return;
+     const cid = res && window.Sha256 && window.Sha256.isHex(res.sha256) ? res.sha256 : null;
+     const state = cid ? (res.expectedSha === cid ? 'verified' : 'unverified') : 'unhashed';
+     const now = Date.now();
+     window.DeviceDB.putFile({ videoId: id, cid, size: (res && res.size) || 0, savedAt: now, lastCheckedAt: now, state })
+       .then(() => { if (window.P2PClient) window.P2PClient.changed(); })
+       .catch(() => {});
+   }
+   ```
+7. `frontend/app.js` `preload` success branch — after `cacheMutations++;` add
+   `recordDeviceFile(id, res);`
+8. `frontend/app.js` `opfsDelete` — after the `try { await window.OPFS.deleteVideo(videoId); } …` line add:
+   ```js
+     if (window.DeviceDB) { await window.DeviceDB.deleteFile(videoId); if (window.P2PClient) window.P2PClient.changed(); }
+   ```
+   `opfsClear` — after its `try { await window.OPFS.clearAll(); } …` line add:
+   ```js
+     if (window.DeviceDB) { await window.DeviceDB.clear(); if (window.P2PClient) window.P2PClient.changed(); }
+   ```
+
+## Out of scope / do NOT touch
+
+- `opfsDownloadEdited` (edited cuts are device-only, never shared).
+- No server calls here (plan 013 reports holdings). Do not hash old files here (plan 013).
+- Do not migrate `_ytpdata` / playlists to IndexedDB.
+
+## Verification
+
+```bash
+cd /home/user/ytplayer && node --check frontend/app.js frontend/opfs.js frontend/opfs-worker.js frontend/device-db.js && echo SYNTAX_OK
+node --test frontend/*.test.js 2>&1 | grep -E "^# (pass|fail)"
+grep -c "device-db.js" frontend/index.html frontend/sw.js
+# Browser check (Chromium). Skip with a note in Findings if no Chromium/playwright is available.
+cd plans/harness && (npm ls playwright >/dev/null 2>&1 || npm i --no-save playwright >/dev/null 2>&1)
+bun device-db-server.js >/tmp/ytp012.log 2>&1 & SRV=$!; sleep 2; timeout 60 node device-db-check.mjs; kill $SRV; true
+```
+
+Expected: `SYNTAX_OK`; `# fail 0`; grep `1` each; the browser check prints JSON containing
+`"shaOk":true,"goodExpected":true`, `"bad":{"ok":false,"error":"integrity check failed (content hash mismatch)"}`,
+`"list":["good","nohash"]`, `"rec":"verified"`, `"after":0`.
+
+## Report format (executor: follow exactly)
+
+Output ONLY the following, no other prose:
+
+1. `git diff` (unified) of all changes.
+2. Raw output of the Verification commands.
+3. `Findings:` — max 10 lines.
+
+Do not commit. Do not push. Do not touch files outside the Steps.
+
+---
+
+## Appendix — frontend/device-db.js
+
+```js
+/* ============================================================================
+ * device-db.js — this device's own file registry (IndexedDB "ytp-device")
+ *
+ * One record per saved video, next to the bytes in OPFS:
+ *   { videoId, cid, size, savedAt, lastCheckedAt, state }
+ *     cid    SHA-256 of the stored file (P2P content id) or null
+ *     state  'verified'   hash computed on save and equal to the server's
+ *            'unverified' hash computed, but the server sent none to compare
+ *            'unhashed'   saved before hashing existed / main-thread fallback
+ * The P2P client (p2p-client.js) reads this to report holdings; OPFS stays the
+ * source of truth for what is playable (a record without a file is ignored).
+ * Every call resolves (null / [] on failure) — private windows can block IDB.
+ * See docs/p2p-architecture.md.
+ * ========================================================================== */
+(function () {
+  'use strict';
+
+  const DB_NAME = 'ytp-device';
+  const VERSION = 1;
+  let _open = null;
+
+  function open() {
+    if (!_open) {
+      _open = new Promise((resolve, reject) => {
+        const r = indexedDB.open(DB_NAME, VERSION);
+        r.onupgradeneeded = () => {
+          const d = r.result;
+          if (!d.objectStoreNames.contains('files')) {
+            const s = d.createObjectStore('files', { keyPath: 'videoId' });
+            s.createIndex('cid', 'cid', { unique: false });
+          }
+        };
+        r.onsuccess = () => resolve(r.result);
+        r.onerror = () => reject(r.error);
+        r.onblocked = () => reject(new Error('device-db blocked'));
+      });
+      _open.catch(() => { _open = null; });
+    }
+    return _open;
+  }
+
+  const done = (req) => new Promise((resolve, reject) => {
+    req.onsuccess = () => resolve(req.result);
+    req.onerror = () => reject(req.error);
+  });
+
+  async function store(mode) {
+    const d = await open();
+    return d.transaction('files', mode).objectStore('files');
+  }
+
+  async function safe(fn, fallback) {
+    try { return await fn(); } catch { return fallback; }
+  }
+
+  window.DeviceDB = {
+    isSupported: () => typeof indexedDB !== 'undefined',
+    putFile: (rec) => safe(async () => {
+      if (!rec || !rec.videoId) return null;
+      await done((await store('readwrite')).put({
+        videoId: String(rec.videoId),
+        cid: rec.cid || null,
+        size: Number(rec.size) || 0,
+        savedAt: Number(rec.savedAt) || Date.now(),
+        lastCheckedAt: Number(rec.lastCheckedAt) || Date.now(),
+        state: rec.state || 'unhashed',
+      }));
+      return true;
+    }, null),
+    getFile: (videoId) => safe(async () => (await done((await store('readonly')).get(String(videoId)))) || null, null),
+    getByCid: (cid) => safe(async () => (await done((await store('readonly')).index('cid').get(String(cid)))) || null, null),
+    listFiles: () => safe(async () => (await done((await store('readonly')).getAll())) || [], []),
+    deleteFile: (videoId) => safe(async () => { await done((await store('readwrite')).delete(String(videoId))); return true; }, null),
+    clear: () => safe(async () => { await done((await store('readwrite')).clear()); return true; }, null),
+  };
+}());
+```