josh/ytplayer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

plan: start 010-views-and-retention-d0c6ca

Browse Source
This commit is contained in:
Claude
2026-09-30 07:18:52 +00:00
parent ea56d286b3
commit f12ca87131
2 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

168
plans/queue/010-views-and-retention-d0c6ca.md
View File

@@ -1,168 +0,0 @@
---
id: 010-views-and-retention-d0c6ca
title: Count views and evict server copies by retention criteria before LRU
created: 2026-09-29
depends_on: [009-server-content-hash-186e7f]
est_files: 5
---
# 010 — Views + retention-ordered eviction
## Objective
Implements flow 9 of `docs/p2p-architecture.md`. After this plan:
- Every play (`GET /api/streams`) and save (`GET /api/download/:id`) counts one view in
`video_views` (deduped per client + video for 30 min; warm-ups don't count).
- When the media cache needs room it evicts copies that are neither **top**
(≥ `P2P_KEEP_MIN_VIEWS` views in `P2P_KEEP_DAYS`) nor **recent** (played within
`P2P_KEEP_RECENT_DAYS`) first — fewest views, then oldest — and only then qualifying
ones by LRU. Budget, disk guard and 10-min play protection are unchanged.
- Eviction never touches `p2p_content` / holders (the catalog keeps growing).
The query + media-cache change were tested ahead of time (retention test 1/1,
media-cache 26/26).
## Context the executor must NOT rediscover
- `plans/patches/010-retention-eviction.diff` patches `server/media-cache.js` (`makeRoom` uses
`db.listMediaEvictionOrder()` when present, else `db.listMediaLru()`) and adds one test to
`server/media-cache.test.js`. It applies on top of plan 009's patches.
- `server/p2p-db.js` (plan 008) has `dayKey`, `rowsOf`, `addView(videoId, now)`.
- `server/server.js` `createMediaCache({ dir: MEDIA_DIR, db: { getMedia, upsertMedia, deleteMedia, listMedia, listMediaLru, touchMedia, mediaStats }, …` (~line 983).
- `app.get('/api/streams', async (c) => {` (~line 532): first lines validate `videoId` and
return 400 when empty. `app.get('/api/download/:videoId', async (c) => {` (~line 1311) does the same.
- Plan 009 imported `P2P` and `* as p2pDb` in server.js.
## Steps
1. `server/p2p-db.js` — append at the end of the file:
```js
// ---- retention (plan 010) -------------------------------------------------------
// Server copies in the order they should be evicted when the cache needs room:
// first the ones that are neither "top" (≥ keepMinViews views in keepDays) nor
// "recent" (played within keepRecentDays) — fewest views, then oldest — and
// only then the qualifying ones, least recently played first.
export async function listMediaEvictionOrder({ now, keepMinViews, keepDays, keepRecentDays }) {
const r = await db.execute({
sql: `SELECT m.video_id, m.size, m.last_access,
COALESCE((SELECT SUM(v.n) FROM video_views v
WHERE v.video_id = m.video_id AND v.day >= ?), 0) AS views
FROM media_cache m WHERE m.status = 'ready'`,
args: [dayKey(now - keepDays * 86400_000)],
});
const recentCut = now - keepRecentDays * 86400_000;
const rows = rowsOf(r).map((x) => ({
...x, views: Number(x.views) || 0,
qualifies: (Number(x.views) || 0) >= keepMinViews || Number(x.last_access) >= recentCut,
}));
rows.sort((a, b) => (a.qualifies - b.qualifies)
|| (a.qualifies ? 0 : a.views - b.views)
|| (a.last_access - b.last_access));
return rows;
}
```
2. Create `server/p2p-retention.test.js` — copy VERBATIM from the Appendix.
3. `server/package.json` "test" script — append ` && bun test ./p2p-retention.test.js`.
4. From the repo root: `git apply plans/patches/010-retention-eviction.diff` (STOP and report on failure).
5. `server/server.js` `createMediaCache({ … db: { … } …` — replace the `db:` line with:
```js
db: {
getMedia, upsertMedia, deleteMedia, listMedia, listMediaLru, touchMedia, mediaStats,
// Retention (docs/p2p-architecture.md flow 9): cold copies go before popular ones.
listMediaEvictionOrder: () => p2pDb.listMediaEvictionOrder({
now: Date.now(), keepMinViews: P2P.keepMinViews, keepDays: P2P.keepDays, keepRecentDays: P2P.keepRecentDays,
}),
},
```
6. `server/server.js` — directly above `app.get('/api/streams', …)` (and above the plan-005
warm route if present) add:
```js
// One view per client per video per 30 min (a play and its save count once).
const viewSeen = new Map(); // `${who}|${id}` -> ms
function countView(c, videoId) {
const who = (c.req.header('x-forwarded-for') || '').split(',')[0].trim() || c.req.query('fp') || 'local';
const key = who + '|' + videoId;
const now = Date.now();
if (now - (viewSeen.get(key) || 0) < 30 * 60_000) return;
viewSeen.set(key, now);
if (viewSeen.size > 20000) viewSeen.clear();
p2pDb.addView(videoId, now).catch(() => {});
}
```
7. In `/api/streams`, directly after `if (!videoId) return c.json({ ok: false, error: 'missing videoId' }, 400);`
add `countView(c, videoId);`. Do the same in `/api/download/:videoId` after its
`missing videoId` guard.
## Out of scope / do NOT touch
- `MEDIA_CACHE_MAX_BYTES`, `EVICT_PROTECT_MS`, the disk guard, `listMediaLru` itself.
- Do not delete server files on a timer — eviction stays budget-driven.
## Verification
```bash
cd /home/user/ytplayer/server && bun install >/dev/null 2>&1
bun test ./p2p-retention.test.js 2>&1 | tail -4
bun test --timeout 60000 ./media-cache.test.js 2>&1 | tail -4
bun run test 2>&1 | grep -E "^ *[0-9]+ (pass|fail)"
bun build server.js --target=bun --outdir=/tmp/ytp-check >/dev/null && echo SERVER_OK
grep -c "countView(c, videoId)" server.js
```
Expected: `1 pass`; media-cache `26 pass 0 fail`; every file `0 fail`; `SERVER_OK`; grep `2`.
## Report format (executor: follow exactly)
Output ONLY the following, no other prose:
1. `git diff` (unified) of all changes.
2. Raw output of the Verification commands.
3. `Findings:` — max 10 lines.
Do not commit. Do not push. Do not touch files outside the Steps.
---
## Appendix — server/p2p-retention.test.js
```js
import { test, expect, beforeAll } from 'bun:test';
import { mkdtempSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
const root = mkdtempSync(join(tmpdir(), 'ytp-retention-test-'));
process.env.DB_PATH = join(root, 'test.db');
const dbmod = await import('./db.js');
const P = await import('./p2p-db.js');
const DAY = 86400_000;
const NOW = Date.UTC(2026, 8, 29, 12);
const opts = { now: NOW, keepMinViews: 3, keepDays: 30, keepRecentDays: 14 };
beforeAll(async () => {
await dbmod.initDb();
await P.initP2pSchema();
const add = (id, lastAccessDaysAgo) => dbmod.upsertMedia(id, { status: 'ready', size: 100, last_access: NOW - lastAccessDaysAgo * DAY });
await add('topOld00001', 60); // 5 views in window → top, but old
await add('recent00001', 2); // 0 views, played 2 days ago → recent
await add('cold0000001', 40); // 1 view in window, old → evict first-ish
await add('cold0000002', 20); // 0 views, 20 days → evict first
await add('cold0000003', 50); // 0 views, 50 days → evict first (older than cold2)
for (let i = 0; i < 5; i++) await P.addView('topOld00001', NOW - 5 * DAY);
await P.addView('cold0000001', NOW - 3 * DAY);
await P.addView('cold0000002', NOW - 45 * DAY); // outside the 30-day window
});
test('non-qualifying copies go first (fewest views, then oldest); qualifying ones by LRU', async () => {
const order = (await P.listMediaEvictionOrder(opts)).map((r) => [r.video_id, r.qualifies]);
expect(order).toEqual([
['cold0000003', false],
['cold0000002', false],
['cold0000001', false],
['topOld00001', true],
['recent00001', true],
]);
});
```