---
id: 002-compress-and-etag-shell-bd459c
title: Serve the app shell gzip/brotli-compressed with ETags
created: 2026-09-29
depends_on: [001-perf-timing-marks-105acc]
est_files: 2
---

# 002 — Serve the app shell gzip/brotli-compressed with ETags

## Objective

Measured on prod: `app.js` is sent as 426 244 bytes with no `content-encoding`
and no `ETag` (2.8 s download at ~150 KB/s). gzip -9 makes it 117 527 bytes;
the whole shell drops from 618 KB to ~158 KB. After this plan:
- Every static text file under `./public` (`.js .css .html .json .webmanifest .svg`)
  is served compressed (`br` preferred, else `gzip`) when the client accepts it,
  with `Vary: Accept-Encoding`.
- Every static file carries a strong `ETag` and `If-None-Match` returns `304`.
- `index.html` (build-stamped at request time) and `/sw.js` (BUILD_TAG-injected) are
  also compressed + ETagged using the text they actually send.
- `Cache-Control` values stay EXACTLY as they are today (`no-cache` for the shell,
  `no-store` for sw.js) — CLAUDE.md "Update-flow architecture" depends on that.

## Context the executor must NOT rediscover

`server/server.js:1895-1910` today:

```js
function indexHtml(c) {
  if (_indexSource === null) {
    try { _indexSource = readFileSync('./public/index.html', 'utf8'); }
    catch { return c.text('index.html not found', 404); }
  }
  return c.html(_indexSource.replace('__BUILD_TAG__', BUILD_TAG), 200, { 'Cache-Control': 'no-cache' });
}
app.get('/', indexHtml);
app.get('/index.html', indexHtml);
...
app.use('/*', serveStatic({ root: './public', onFound: (_path, c) => { c.header('Cache-Control', 'no-cache'); } }));
// SPA fallback — return index.html for any unmatched path
app.get('/*', indexHtml);
```

`/sw.js` handler at ~line 1860 ends with `return c.text(src, 200, { ...no-store headers })`.

Bun provides `Bun.gzipSync(buf, { level: 9 })`. Brotli: `import { brotliCompressSync, constants } from 'node:zlib'`
(works in Bun). `createHash` is already imported from `node:crypto` at the top.

**Never compress** `/api/play`, `/api/media/*`, `/api/download/*` (Range/binary) —
they are not served by `serveStatic`, so a static-only middleware cannot touch them.

## Steps

1. `server/server.js` — add near the top imports:
   `import { brotliCompressSync, constants as zlibConstants } from 'node:zlib';`
2. `server/server.js` — directly ABOVE `function indexHtml(c) {` add this helper block:
   ```js
   // Compressed + ETagged static text. The shell is ~620 KB raw / ~160 KB gzip
   // and every byte crosses the slow VPS→homelab link, so compress once per
   // file content and keep it in memory. ETag = sha256 of the RAW bytes, so a
   // `no-cache` revalidation costs a 304 instead of the whole file.
   const COMPRESSIBLE = /\.(js|css|html|json|webmanifest|svg|txt)$/i;
   const compressedCache = new Map(); // key -> { etag, raw, gz, br, type }
   function compressedEntry(key, raw, type) {
     let e = compressedCache.get(key);
     const etag = '"' + createHash('sha256').update(raw).digest('hex').slice(0, 32) + '"';
     if (e && e.etag === etag) return e;
     e = {
       etag, raw, type,
       gz: Bun.gzipSync(raw, { level: 9 }),
       br: brotliCompressSync(raw, { params: { [zlibConstants.BROTLI_PARAM_QUALITY]: 11 } }),
     };
     compressedCache.set(key, e);
     return e;
   }
   function sendCompressed(c, e, cacheControl) {
     const headers = { 'Content-Type': e.type, 'Cache-Control': cacheControl, ETag: e.etag, Vary: 'Accept-Encoding' };
     const inm = c.req.header('if-none-match') || '';
     if (inm.split(',').map((s) => s.trim()).includes(e.etag)) return new Response(null, { status: 304, headers });
     const ae = c.req.header('accept-encoding') || '';
     if (/\bbr\b/.test(ae)) return new Response(e.br, { headers: { ...headers, 'Content-Encoding': 'br' } });
     if (/\bgzip\b/.test(ae)) return new Response(e.gz, { headers: { ...headers, 'Content-Encoding': 'gzip' } });
     return new Response(e.raw, { headers });
   }
   const MIME = { js: 'text/javascript; charset=utf-8', css: 'text/css; charset=utf-8', html: 'text/html; charset=utf-8',
     json: 'application/json', webmanifest: 'application/manifest+json', svg: 'image/svg+xml', txt: 'text/plain; charset=utf-8' };
   ```
3. `server/server.js` — change `indexHtml` so its return line becomes:
   ```js
   const html = _indexSource.replace('__BUILD_TAG__', BUILD_TAG);
   return sendCompressed(c, compressedEntry('index.html', Buffer.from(html), MIME.html), 'no-cache');
   ```
4. `server/server.js` — in the `/sw.js` handler, replace its final return, which is exactly:
   ```js
     return c.text(src, 200, {
       'Content-Type': 'application/javascript; charset=utf-8',
       'Cache-Control': 'no-store, no-cache, must-revalidate',
     });
   ```
   with:
   ```js
     return sendCompressed(c, compressedEntry('sw.js', Buffer.from(src), MIME.js), 'no-store, no-cache, must-revalidate');
   ```
   (`text/javascript` is a valid service-worker MIME type.) Dry-run result: app.js 426 244 → 93 717 bytes (br).
5. `server/server.js` — directly BEFORE the `app.use('/*', serveStatic(...))` line add:
   ```js
   app.get('/*', async (c, next) => {
     const p = decodeURIComponent(new URL(c.req.url).pathname);
     if (!COMPRESSIBLE.test(p) || p.includes('..') || p.startsWith('/api/')) return next();
     const file = Bun.file('./public' + p);
     if (!(await file.exists())) return next();
     const raw = Buffer.from(await file.arrayBuffer());
     const ext = p.slice(p.lastIndexOf('.') + 1).toLowerCase();
     return sendCompressed(c, compressedEntry(p, raw, MIME[ext] || 'application/octet-stream'), 'no-cache');
   });
   ```
   (`/sw.js`, `/` and `/index.html` are registered earlier and win; this only covers the rest.)

## Out of scope / do NOT touch

- `frontend/sw.js`, `frontend/sw-update.js`: no changes. The SW fetches shell files with
  `cache: 'reload'` and `?__ytpfresh=` — query strings don't affect the pathname match, fine.
- Do not change any `Cache-Control` value. Do not add `hono/compress` globally (it would
  hit Range media responses).
- Do not touch `computeBuildTag`.

## Verification

```bash
cd /home/user/ytplayer/server && [ -e public ] || ln -s ../frontend public
PORT=3999 bun server.js > /tmp/ytp002.log 2>&1 & SRV=$!; sleep 4
curl -s -o /dev/null -D - -H 'Accept-Encoding: gzip, br' http://localhost:3999/app.js | grep -iE 'content-encoding|etag|cache-control|vary'
ET=$(curl -s -D - -o /dev/null http://localhost:3999/app.js | grep -i '^etag' | cut -d' ' -f2 | tr -d '\r')
curl -s -o /dev/null -w '%{http_code}\n' -H "If-None-Match: $ET" http://localhost:3999/app.js
curl -s -H 'Accept-Encoding: gzip' --compressed http://localhost:3999/ | grep -c 'ytp-build'
curl -s -o /dev/null -D - -H 'Accept-Encoding: gzip' http://localhost:3999/sw.js | grep -iE 'content-encoding|cache-control'
curl -s --compressed http://localhost:3999/sw.js | grep -c "__BUILD_TAG__ !== 'undefined'" ; true
kill $SRV; true
cd /home/user/ytplayer && node --test frontend/*.test.js 2>&1 | tail -3
```

Expected: `content-encoding: br`, an `etag`, `cache-control: no-cache`, `vary: Accept-Encoding`;
the If-None-Match request prints `304`; index grep prints `1`; sw.js shows `content-encoding: gzip`
and its original no-store cache-control; the last grep prints `0` (tag was injected); tests `fail 0`.

## Report format (executor: follow exactly)

Output ONLY the following, no other prose:

1. `git diff` (unified) of all changes.
2. Raw output of the Verification commands.
3. `Findings:` — max 10 lines: surprises, deviations from the steps, anything
   skipped and why.

Do not commit. Do not push. Do not touch files outside the Steps.

## Execution log

- Executor: in-session Agent (haiku). Attempts: 1. Fix rounds: 0.
- Orchestrator re-ran Verification on a fresh port: `Content-Encoding: br`, ETag, `Vary`, 304 on If-None-Match, index stamped (`ytp-build` 1), sw.js gzip + original no-store `Cache-Control`, BUILD_TAG injected (0 placeholders), SPA fallback 200, 52 frontend tests pass. app.js 426 244 -> 93 759 bytes.
- Executor Findings (verbatim): All 5 verification tests pass. All unit tests pass. Changes follow the plan exactly: added brotli/zlib imports, implemented compressedEntry/sendCompressed helpers with ETag and dual-compression support (br preferred, fallback to gzip), updated indexHtml and sw.js handlers to use sendCompressed, and added middleware for compressing all text files under ./public. Cache-Control headers preserved exactly as specified. No changes made to sw.js, sw-update.js, or computeBuildTag.
- Orchestrator note: the executor left a background server on port 3999; killed. Cosmetic: the helper block sits between the old indexHtml comment and `let _indexSource`.