ytplayer/plans/done/010-views-and-retention-d0c6ca.md

---
id: 010-views-and-retention-d0c6ca
title: Count views and evict server copies by retention criteria before LRU
created: 2026-09-29
depends_on: [009-server-content-hash-186e7f]
est_files: 5
---

# 010 — Views + retention-ordered eviction

## Objective

Implements flow 9 of `docs/p2p-architecture.md`. After this plan:
- Every play (`GET /api/streams`) and save (`GET /api/download/:id`) counts one view in
  `video_views` (deduped per client + video for 30 min; warm-ups don't count).
- When the media cache needs room it evicts copies that are neither **top**
  (≥ `P2P_KEEP_MIN_VIEWS` views in `P2P_KEEP_DAYS`) nor **recent** (played within
  `P2P_KEEP_RECENT_DAYS`) first — fewest views, then oldest — and only then qualifying
  ones by LRU. Budget, disk guard and 10-min play protection are unchanged.
- Eviction never touches `p2p_content` / holders (the catalog keeps growing).

The query + media-cache change were tested ahead of time (retention test 1/1,
media-cache 26/26).

## Context the executor must NOT rediscover

- `plans/patches/010-retention-eviction.diff` patches `server/media-cache.js` (`makeRoom` uses
  `db.listMediaEvictionOrder()` when present, else `db.listMediaLru()`) and adds one test to
  `server/media-cache.test.js`. It applies on top of plan 009's patches.
- `server/p2p-db.js` (plan 008) has `dayKey`, `rowsOf`, `addView(videoId, now)`.
- `server/server.js` `createMediaCache({ dir: MEDIA_DIR, db: { getMedia, upsertMedia, deleteMedia, listMedia, listMediaLru, touchMedia, mediaStats }, …` (~line 983).
- `app.get('/api/streams', async (c) => {` (~line 532): first lines validate `videoId` and
  return 400 when empty. `app.get('/api/download/:videoId', async (c) => {` (~line 1311) does the same.
- Plan 009 imported `P2P` and `* as p2pDb` in server.js.

## Steps

1. `server/p2p-db.js` — append at the end of the file:
   ```js

   // ---- retention (plan 010) -------------------------------------------------------
   // Server copies in the order they should be evicted when the cache needs room:
   // first the ones that are neither "top" (≥ keepMinViews views in keepDays) nor
   // "recent" (played within keepRecentDays) — fewest views, then oldest — and
   // only then the qualifying ones, least recently played first.
   export async function listMediaEvictionOrder({ now, keepMinViews, keepDays, keepRecentDays }) {
     const r = await db.execute({
       sql: `SELECT m.video_id, m.size, m.last_access,
                    COALESCE((SELECT SUM(v.n) FROM video_views v
                               WHERE v.video_id = m.video_id AND v.day >= ?), 0) AS views
               FROM media_cache m WHERE m.status = 'ready'`,
       args: [dayKey(now - keepDays * 86400_000)],
     });
     const recentCut = now - keepRecentDays * 86400_000;
     const rows = rowsOf(r).map((x) => ({
       ...x, views: Number(x.views) || 0,
       qualifies: (Number(x.views) || 0) >= keepMinViews || Number(x.last_access) >= recentCut,
     }));
     rows.sort((a, b) => (a.qualifies - b.qualifies)
       || (a.qualifies ? 0 : a.views - b.views)
       || (a.last_access - b.last_access));
     return rows;
   }
   ```
2. Create `server/p2p-retention.test.js` — copy VERBATIM from the Appendix.
3. `server/package.json` "test" script — append ` && bun test ./p2p-retention.test.js`.
4. From the repo root: `git apply plans/patches/010-retention-eviction.diff` (STOP and report on failure).
5. `server/server.js` `createMediaCache({ … db: { … } …` — replace the `db:` line with:
   ```js
     db: {
       getMedia, upsertMedia, deleteMedia, listMedia, listMediaLru, touchMedia, mediaStats,
       // Retention (docs/p2p-architecture.md flow 9): cold copies go before popular ones.
       listMediaEvictionOrder: () => p2pDb.listMediaEvictionOrder({
         now: Date.now(), keepMinViews: P2P.keepMinViews, keepDays: P2P.keepDays, keepRecentDays: P2P.keepRecentDays,
       }),
     },
   ```
6. `server/server.js` — directly above `app.get('/api/streams', …)` (and above the plan-005
   warm route if present) add:
   ```js
   // One view per client per video per 30 min (a play and its save count once).
   const viewSeen = new Map(); // `${who}|${id}` -> ms
   function countView(c, videoId) {
     const who = (c.req.header('x-forwarded-for') || '').split(',')[0].trim() || c.req.query('fp') || 'local';
     const key = who + '|' + videoId;
     const now = Date.now();
     if (now - (viewSeen.get(key) || 0) < 30 * 60_000) return;
     viewSeen.set(key, now);
     if (viewSeen.size > 20000) viewSeen.clear();
     p2pDb.addView(videoId, now).catch(() => {});
   }
   ```
7. In `/api/streams`, directly after `if (!videoId) return c.json({ ok: false, error: 'missing videoId' }, 400);`
   add `countView(c, videoId);`. Do the same in `/api/download/:videoId` after its
   `missing videoId` guard.

## Out of scope / do NOT touch

- `MEDIA_CACHE_MAX_BYTES`, `EVICT_PROTECT_MS`, the disk guard, `listMediaLru` itself.
- Do not delete server files on a timer — eviction stays budget-driven.

## Verification

```bash
cd /home/user/ytplayer/server && bun install >/dev/null 2>&1
bun test ./p2p-retention.test.js 2>&1 | tail -4
bun test --timeout 60000 ./media-cache.test.js 2>&1 | tail -4
bun run test 2>&1 | grep -E "^ *[0-9]+ (pass|fail)"
bun build server.js --target=bun --outdir=/tmp/ytp-check >/dev/null && echo SERVER_OK
grep -c "countView(c, videoId)" server.js
```

Expected: `1 pass`; media-cache `26 pass 0 fail`; every file `0 fail`; `SERVER_OK`; grep `3` (the function definition plus the two calls).

## Report format (executor: follow exactly)

Output ONLY the following, no other prose:

1. `git diff` (unified) of all changes.
2. Raw output of the Verification commands.
3. `Findings:` — max 10 lines.

Do not commit. Do not push. Do not touch files outside the Steps.

---

## Appendix — server/p2p-retention.test.js

```js
import { test, expect, beforeAll } from 'bun:test';
import { mkdtempSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';

const root = mkdtempSync(join(tmpdir(), 'ytp-retention-test-'));
process.env.DB_PATH = join(root, 'test.db');
const dbmod = await import('./db.js');
const P = await import('./p2p-db.js');

const DAY = 86400_000;
const NOW = Date.UTC(2026, 8, 29, 12);
const opts = { now: NOW, keepMinViews: 3, keepDays: 30, keepRecentDays: 14 };

beforeAll(async () => {
  await dbmod.initDb();
  await P.initP2pSchema();
  const add = (id, lastAccessDaysAgo) => dbmod.upsertMedia(id, { status: 'ready', size: 100, last_access: NOW - lastAccessDaysAgo * DAY });
  await add('topOld00001', 60);   // 5 views in window → top, but old
  await add('recent00001', 2);    // 0 views, played 2 days ago → recent
  await add('cold0000001', 40);   // 1 view in window, old → evict first-ish
  await add('cold0000002', 20);   // 0 views, 20 days → evict first
  await add('cold0000003', 50);   // 0 views, 50 days → evict first (older than cold2)
  for (let i = 0; i < 5; i++) await P.addView('topOld00001', NOW - 5 * DAY);
  await P.addView('cold0000001', NOW - 3 * DAY);
  await P.addView('cold0000002', NOW - 45 * DAY); // outside the 30-day window
});

test('non-qualifying copies go first (fewest views, then oldest); qualifying ones by LRU', async () => {
  const order = (await P.listMediaEvictionOrder(opts)).map((r) => [r.video_id, r.qualifies]);
  expect(order).toEqual([
    ['cold0000003', false],
    ['cold0000002', false],
    ['cold0000001', false],
    ['topOld00001', true],
    ['recent00001', true],
  ]);
});
```

## Execution log

- Executor: in-session Agent (haiku). Attempts: 1. Fix rounds: 0 (plan text corrected by orchestrator).
- Orchestrator re-ran Verification: `media-cache.js`, `media-cache.test.js`, `p2p-retention.test.js` byte-identical to the pre-tested versions; retention test 1 pass; media-cache 26 pass; all 10 server test files 0 fail; `SERVER_OK`.
- Plan flaw fixed: the verification grep `countView(c, videoId)` also matches the function definition, so the correct expected count is 3, not 2 (the executor reported this correctly).
- Executor Findings (verbatim): Plan verification target was "grep 2" but actual count is 3: function definition on line 606 (function countView(c, videoId)) plus 2 calls match the grep pattern. All functionality correct. p2p-retention test: 1 pass. media-cache tests: 26 pass, 0 fail. All test suites: 0 fail across all files. Server build: SERVER_OK. countView calls: grep result 3.