ytplayer/plans/done/010-views-and-retention-d0c6ca.md

id, title, created, depends_on, est_files

id	title	created	depends_on	est_files
010-views-and-retention-d0c6ca	Count views and evict server copies by retention criteria before LRU	2026-09-29	009-server-content-hash-186e7f	5

010 — Views + retention-ordered eviction

Objective

Implements flow 9 of docs/p2p-architecture.md. After this plan:

• Every play (GET /api/streams) and save (GET /api/download/:id) counts one view in video_views (deduped per client + video for 30 min; warm-ups don't count).
• When the media cache needs room it evicts copies that are neither top (≥ P2P_KEEP_MIN_VIEWS views in P2P_KEEP_DAYS) nor recent (played within P2P_KEEP_RECENT_DAYS) first — fewest views, then oldest — and only then qualifying ones by LRU. Budget, disk guard and 10-min play protection are unchanged.
• Eviction never touches p2p_content / holders (the catalog keeps growing).

The query + media-cache change were tested ahead of time (retention test 1/1, media-cache 26/26).

Context the executor must NOT rediscover

• plans/patches/010-retention-eviction.diff patches server/media-cache.js (makeRoom uses db.listMediaEvictionOrder() when present, else db.listMediaLru()) and adds one test to server/media-cache.test.js. It applies on top of plan 009's patches.
• server/p2p-db.js (plan 008) has dayKey, rowsOf, addView(videoId, now).
• server/server.js createMediaCache({ dir: MEDIA_DIR, db: { getMedia, upsertMedia, deleteMedia, listMedia, listMediaLru, touchMedia, mediaStats }, … (~line 983).
• app.get('/api/streams', async (c) => { (~line 532): first lines validate videoId and return 400 when empty. app.get('/api/download/:videoId', async (c) => { (~line 1311) does the same.
• Plan 009 imported P2P and * as p2pDb in server.js.

Steps

1. server/p2p-db.js — append at the end of the file:

   
   // ---- retention (plan 010) -------------------------------------------------------
   // Server copies in the order they should be evicted when the cache needs room:
   // first the ones that are neither "top" (≥ keepMinViews views in keepDays) nor
   // "recent" (played within keepRecentDays) — fewest views, then oldest — and
   // only then the qualifying ones, least recently played first.
   export async function listMediaEvictionOrder({ now, keepMinViews, keepDays, keepRecentDays }) {
     const r = await db.execute({
       sql: `SELECT m.video_id, m.size, m.last_access,
                    COALESCE((SELECT SUM(v.n) FROM video_views v
                               WHERE v.video_id = m.video_id AND v.day >= ?), 0) AS views
               FROM media_cache m WHERE m.status = 'ready'`,
       args: [dayKey(now - keepDays * 86400_000)],
     });
     const recentCut = now - keepRecentDays * 86400_000;
     const rows = rowsOf(r).map((x) => ({
       ...x, views: Number(x.views) || 0,
       qualifies: (Number(x.views) || 0) >= keepMinViews || Number(x.last_access) >= recentCut,
     }));
     rows.sort((a, b) => (a.qualifies - b.qualifies)
       || (a.qualifies ? 0 : a.views - b.views)
       || (a.last_access - b.last_access));
     return rows;
   }
   

2. Create server/p2p-retention.test.js — copy VERBATIM from the Appendix.
3. server/package.json "test" script — append && bun test ./p2p-retention.test.js.
4. From the repo root: git apply plans/patches/010-retention-eviction.diff (STOP and report on failure).
5. server/server.js createMediaCache({ … db: { … } … — replace the db: line with:

     db: {
       getMedia, upsertMedia, deleteMedia, listMedia, listMediaLru, touchMedia, mediaStats,
       // Retention (docs/p2p-architecture.md flow 9): cold copies go before popular ones.
       listMediaEvictionOrder: () => p2pDb.listMediaEvictionOrder({
         now: Date.now(), keepMinViews: P2P.keepMinViews, keepDays: P2P.keepDays, keepRecentDays: P2P.keepRecentDays,
       }),
     },
   

6. server/server.js — directly above app.get('/api/streams', …) (and above the plan-005 warm route if present) add:

   // One view per client per video per 30 min (a play and its save count once).
   const viewSeen = new Map(); // `${who}|${id}` -> ms
   function countView(c, videoId) {
     const who = (c.req.header('x-forwarded-for') || '').split(',')[0].trim() || c.req.query('fp') || 'local';
     const key = who + '|' + videoId;
     const now = Date.now();
     if (now - (viewSeen.get(key) || 0) < 30 * 60_000) return;
     viewSeen.set(key, now);
     if (viewSeen.size > 20000) viewSeen.clear();
     p2pDb.addView(videoId, now).catch(() => {});
   }
   

7. In /api/streams, directly after if (!videoId) return c.json({ ok: false, error: 'missing videoId' }, 400); add countView(c, videoId);. Do the same in /api/download/:videoId after its missing videoId guard.

Out of scope / do NOT touch

• MEDIA_CACHE_MAX_BYTES, EVICT_PROTECT_MS, the disk guard, listMediaLru itself.
• Do not delete server files on a timer — eviction stays budget-driven.

Verification

cd /home/user/ytplayer/server && bun install >/dev/null 2>&1
bun test ./p2p-retention.test.js 2>&1 | tail -4
bun test --timeout 60000 ./media-cache.test.js 2>&1 | tail -4
bun run test 2>&1 | grep -E "^ *[0-9]+ (pass|fail)"
bun build server.js --target=bun --outdir=/tmp/ytp-check >/dev/null && echo SERVER_OK
grep -c "countView(c, videoId)" server.js

Expected: 1 pass; media-cache 26 pass 0 fail; every file 0 fail; SERVER_OK; grep 3 (the function definition plus the two calls).

Report format (executor: follow exactly)

Output ONLY the following, no other prose:

1. git diff (unified) of all changes.
2. Raw output of the Verification commands.
3. Findings: — max 10 lines.

Do not commit. Do not push. Do not touch files outside the Steps.

---

Appendix — server/p2p-retention.test.js

import { test, expect, beforeAll } from 'bun:test';
import { mkdtempSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';

const root = mkdtempSync(join(tmpdir(), 'ytp-retention-test-'));
process.env.DB_PATH = join(root, 'test.db');
const dbmod = await import('./db.js');
const P = await import('./p2p-db.js');

const DAY = 86400_000;
const NOW = Date.UTC(2026, 8, 29, 12);
const opts = { now: NOW, keepMinViews: 3, keepDays: 30, keepRecentDays: 14 };

beforeAll(async () => {
  await dbmod.initDb();
  await P.initP2pSchema();
  const add = (id, lastAccessDaysAgo) => dbmod.upsertMedia(id, { status: 'ready', size: 100, last_access: NOW - lastAccessDaysAgo * DAY });
  await add('topOld00001', 60);   // 5 views in window → top, but old
  await add('recent00001', 2);    // 0 views, played 2 days ago → recent
  await add('cold0000001', 40);   // 1 view in window, old → evict first-ish
  await add('cold0000002', 20);   // 0 views, 20 days → evict first
  await add('cold0000003', 50);   // 0 views, 50 days → evict first (older than cold2)
  for (let i = 0; i < 5; i++) await P.addView('topOld00001', NOW - 5 * DAY);
  await P.addView('cold0000001', NOW - 3 * DAY);
  await P.addView('cold0000002', NOW - 45 * DAY); // outside the 30-day window
});

test('non-qualifying copies go first (fewest views, then oldest); qualifying ones by LRU', async () => {
  const order = (await P.listMediaEvictionOrder(opts)).map((r) => [r.video_id, r.qualifies]);
  expect(order).toEqual([
    ['cold0000003', false],
    ['cold0000002', false],
    ['cold0000001', false],
    ['topOld00001', true],
    ['recent00001', true],
  ]);
});

Execution log

• Executor: in-session Agent (haiku). Attempts: 1. Fix rounds: 0 (plan text corrected by orchestrator).
• Orchestrator re-ran Verification: media-cache.js, media-cache.test.js, p2p-retention.test.js byte-identical to the pre-tested versions; retention test 1 pass; media-cache 26 pass; all 10 server test files 0 fail; SERVER_OK.
• Plan flaw fixed: the verification grep countView(c, videoId) also matches the function definition, so the correct expected count is 3, not 2 (the executor reported this correctly).
• Executor Findings (verbatim): Plan verification target was "grep 2" but actual count is 3: function definition on line 606 (function countView(c, videoId)) plus 2 calls match the grep pattern. All functionality correct. p2p-retention test: 1 pass. media-cache tests: 26 pass, 0 fail. All test suites: 0 fail across all files. Server build: SERVER_OK. countView calls: grep result 3.