250 lines
16 KiB
Markdown
250 lines
16 KiB
Markdown
# Performance Baseline Report (Phase 0 / UNCHANGED code)
|
||
|
||
**Date**: 2026-10-07
|
||
**Commit**: `b77938a`
|
||
**Dataset**: `perf/results/baseline-2026-10-07.json`
|
||
|
||
---
|
||
|
||
## 1. Executive Summary & Headline Baseline Numbers
|
||
|
||
This baseline captures the performance metrics of the unoptimized ytplayer codebase (commit `b77938a`, matching production tag `1175f1a1d2c1`) across 5 scenarios on both Chromium and WebKit under simulated LTE and Lossy network profiles (median of 5 runs).
|
||
|
||
### Headline Baseline Numbers
|
||
|
||
- **Cold First-Load Wire Transfer**:
|
||
- **Chromium**: 164 requests, **1,015.0 KB** compressed wire bytes (boot-done median: **10,163 ms** under 4× CPU throttle + LTE).
|
||
- **WebKit**: 162 requests, **1,152.0 KB (1.12 MB)** wire bytes (boot-done median: **3,201 ms** under proxy LTE).
|
||
- Breakdown: ~477 KB JavaScript, ~397 KB Web Fonts, ~131 KB CSS, ~27 KB HTML.
|
||
|
||
- **Warm Reload (under Service Worker)**:
|
||
- **Chromium**: 4 requests, **5.2 KB** wire bytes (periodic API version poll), boot-done median: **675 ms** (LTE) / **651 ms** (Lossy).
|
||
- **WebKit**: 0 requests, **0 KB** wire bytes, boot-done median: **131 ms** (LTE) / **131 ms** (Lossy).
|
||
|
||
- **Offline Reload (under Service Worker)**:
|
||
- **Chromium**: 0 requests, **0 B** wire bytes, reload median: **240 ms**, boot-done median: **219 ms**.
|
||
- **WebKit**: 0 requests, **0 B** wire bytes, reload median: **135 ms**, boot-done median: **128 ms**.
|
||
|
||
- **Update N -> N+1 Wire Cost (The Invalidation Penalty)**:
|
||
- **1-line `app.js` change**:
|
||
- **Chromium**: 88 requests, **596.6 KB** wire bytes (~611 KB uncompressed/brotli), duration: **2,013 ms** (LTE) / **2,031 ms** (Lossy).
|
||
- **WebKit**: 86 requests, **664.3 KB** wire bytes (~680 KB wire), duration: **4,457 ms** (LTE) / **8,135 ms** (Lossy).
|
||
- **1-line `theme-glass.css` change**:
|
||
- **Chromium**: 88 requests, **596.7 KB** wire bytes, duration: **1,970 ms** (LTE) / **2,004 ms** (Lossy).
|
||
- **WebKit**: 86-87 requests, **664.3 KB** wire bytes, duration: **4,510 ms** (LTE) / **8,269 ms** (Lossy).
|
||
|
||
> **Core Invalidation Finding**: A single 1-line change to *any* static file (`app.js` or `theme-glass.css`) mutates `BUILD_TAG` calculated by `computeBuildTag()`. This causes `sw.js` to re-fetch and re-cache all 86-88 shell assets (~600-664 KB wire payload) rather than just the mutated resource.
|
||
|
||
---
|
||
|
||
## 2. Benchmark Harness Execution
|
||
|
||
The benchmark harness is located in `perf/baseline.mjs` and driven by Playwright with an isolated ephemeral Bun server and a custom pacing proxy (`perf/proxy.mjs`).
|
||
|
||
### Quick Start / Exact Commands
|
||
|
||
```bash
|
||
# Ensure node_modules is symlinked to root
|
||
ln -s ~/development/personal/ytplayer/node_modules node_modules
|
||
|
||
# Run full baseline suite (all scenarios, both browsers, both profiles, 5 runs)
|
||
node perf/baseline.mjs --runs 5 --browser all --profile all --scenario all --out perf/results/baseline-2026-10-07.json
|
||
|
||
# Run individual browser or scenario
|
||
node perf/baseline.mjs --runs 5 --browser chromium --profile lte --scenario cold
|
||
node perf/baseline.mjs --runs 5 --browser webkit --profile lte --scenario update-js
|
||
|
||
# Run comparison against baseline after optimizations
|
||
node perf/baseline.mjs --runs 5 --browser all --profile all --scenario all --compare perf/results/baseline-2026-10-07.json
|
||
```
|
||
|
||
---
|
||
|
||
## 3. Results by Browser and Profile (Median of 5 Runs)
|
||
|
||
### Chromium — LTE (1.6 Mbps down, 750 kbps up, 150 ms RTT, 4× CPU throttle)
|
||
|
||
| Scenario | Reqs (med) | Wire Bytes (med) | FCP (ms) | LCP (ms) | Boot Done (ms) | SW Ready / Reload (ms) | Long Task Total (ms) |
|
||
|---|---|---|---|---|---|---|---|
|
||
| **Cold First Load** | 164 | 1,015.0 KB | 1,540 | 1,896 | 10,163 | 38,446 | 397 |
|
||
| **Warm Reload (SW)** | 4 | 5.2 KB | 180 | 684 | 675 | - | 156 |
|
||
| **Offline Reload** | 0 | 0.0 KB | 72 | 108 | 219 | 240 | - |
|
||
| **Update N->N+1 (JS)** | 88 | 596.6 KB | - | - | - | 2,013 | - |
|
||
| **Update N->N+1 (CSS)** | 88 | 596.7 KB | - | - | - | 1,970 | - |
|
||
|
||
- **Storage Estimate**: Quota = 4,298,447,325 B (~4.0 GB), Usage = 3,480,029 B (~3.31 MB).
|
||
- **Cold Wire Bytes by Type**: JS 477.2 KB, Fonts 397.1 KB, CSS 130.7 KB, HTML 27.1 KB, Images 6.3 KB, API/Other 1.0 KB.
|
||
|
||
---
|
||
|
||
### Chromium — Lossy (1.0 Mbps down, 500 kbps up, 250 ms RTT, 4× CPU throttle)
|
||
|
||
| Scenario | Reqs (med) | Wire Bytes (med) | FCP (ms) | LCP (ms) | Boot Done (ms) | SW Ready / Reload (ms) | Long Task Total (ms) |
|
||
|---|---|---|---|---|---|---|---|
|
||
| **Cold First Load** | 164 | 1,015.0 KB | 2,344 | 2,912 | 16,015 | 42,518 | 432 |
|
||
| **Warm Reload (SW)** | 4 | 5.2 KB | 184 | 644 | 651 | - | 155 |
|
||
| **Offline Reload** | 0 | 0.0 KB | 80 | 92 | 237 | 253 | - |
|
||
| **Update N->N+1 (JS)** | 88 | 596.6 KB | - | - | - | 2,031 | - |
|
||
| **Update N->N+1 (CSS)** | 88 | 596.7 KB | - | - | - | 2,004 | - |
|
||
|
||
---
|
||
|
||
### WebKit — LTE (1.6 Mbps down, 150 ms RTT via proxy)
|
||
|
||
| Scenario | Reqs (med) | Wire Bytes (med) | FCP (ms) | LCP (ms) | Boot Done (ms) | SW Ready / Reload (ms) | Long Task Total (ms) |
|
||
|---|---|---|---|---|---|---|---|
|
||
| **Cold First Load** | 162 | 1.12 MB | 1,088 | 1,287 | 3,201 | 30,508 | 0* |
|
||
| **Warm Reload (SW)** | 0 | 0.0 KB | 110 | 110 | 131 | - | 0* |
|
||
| **Offline Reload** | 0 | 0.0 KB | 107 | 107 | 128 | 135 | - |
|
||
| **Update N->N+1 (JS)** | 86 | 664.3 KB | - | - | - | 4,457 | - |
|
||
| **Update N->N+1 (CSS)** | 86 | 664.3 KB | - | - | - | 4,510 | - |
|
||
|
||
---
|
||
|
||
### WebKit — Lossy (1.0 Mbps down, 250 ms RTT + 1,500 ms stalls via proxy)
|
||
|
||
| Scenario | Reqs (med) | Wire Bytes (med) | FCP (ms) | LCP (ms) | Boot Done (ms) | SW Ready / Reload (ms) | Long Task Total (ms) |
|
||
|---|---|---|---|---|---|---|---|
|
||
| **Cold First Load** | 162 | 1.12 MB | 3,557 | 3,735 | 7,303 | 35,449 | 0* |
|
||
| **Warm Reload (SW)** | 0 | 0.0 KB | 111 | 112 | 131 | - | 0* |
|
||
| **Offline Reload** | 0 | 0.0 KB | 109 | 119 | 129 | 141 | - |
|
||
| **Update N->N+1 (JS)** | 86 | 664.3 KB | - | - | - | 8,135 | - |
|
||
| **Update N->N+1 (CSS)** | 87 | 664.4 KB | - | - | - | 8,269 | - |
|
||
|
||
*\*Note: WebKit does not expose `PerformanceLongTaskTiming` (`longtask` entries), so long task total evaluates to 0 ms.*
|
||
|
||
---
|
||
|
||
## 4. Variance & Noise Observed
|
||
|
||
Over the 5 runs per configuration, measurements demonstrated high stability:
|
||
- **Wire Bytes & Request Counts**: Deterministic (0% variance across identical runs).
|
||
- **Cold Boot-Done**:
|
||
- Chromium LTE: Min 10,129 ms, Max 10,798 ms (<6% variance).
|
||
- Chromium Lossy: Min 15,871 ms, Max 16,031 ms (<1% variance).
|
||
- WebKit LTE: Min 3,138 ms, Max 3,245 ms (<3% variance).
|
||
- WebKit Lossy: Min 6,963 ms, Max 8,710 ms (~20% variance due to proxy random stalls).
|
||
- **Warm & Offline Reload**:
|
||
- Chromium Offline Reload: Min 238 ms, Max 257 ms (<8% variance).
|
||
- WebKit Offline Reload: Min 131 ms, Max 141 ms (<7% variance).
|
||
- **Update Duration**:
|
||
- Chromium: Runs cluster around ~2,000 ms (2,013 ms median). Occasional outlier runs (~45–200 ms) occur if the browser resolves `reg.update()` immediately before network completion reporting.
|
||
- WebKit: Steady scaling between LTE (~4.5 s) and Lossy (~8.2 s) pacing.
|
||
|
||
---
|
||
|
||
## 5. Limitations & Caveats
|
||
|
||
1. **WebKit Network Throttling**:
|
||
WebKit lacks Chrome DevTools Protocol (CDP) support for hardware network emulation and CPU throttling. We paced throughput (1.6 Mbps) and injected latency/stalls using a custom local reverse proxy (`perf/proxy.mjs`). This throttles HTTP response chunks truthfully but cannot simulate OS-level socket buffer delays or 4× CPU throttling.
|
||
2. **Autoplay Policy in WebKit**:
|
||
Playwright WebKit on Linux rejects `--autoplay-policy=no-user-gesture-required` and enforces Safari-style gesture requirements on unmuted audio elements. The media playback benchmark detects this and correctly reports `autoplayBlocked: true` without failing the test run.
|
||
3. **WebKit SetOffline Incompatibility**:
|
||
Calling Playwright's `context.setOffline(true)` on Linux WebKit causes an unrecoverable engine crash (`WebKit encountered an internal error`). Offline mode for WebKit is simulated by terminating the proxy upstream connection (`proxy.setOffline(true)`).
|
||
|
||
---
|
||
|
||
## 6. Manual iPhone Testing Protocol (Phase 0 / iOS Safari PWA)
|
||
|
||
Because automated headless WebKit cannot fully replicate iOS Safari's mobile background constraints and battery telemetry, the following manual protocol is required:
|
||
|
||
### Protocol Steps
|
||
|
||
1. **Install PWA**:
|
||
- Open Safari on iPhone and navigate to `https://worship.hesed.sbs`.
|
||
- Tap the Share button -> **"Add to Home Screen"**.
|
||
- Launch ytplayer directly from the Home Screen icon.
|
||
|
||
2. **Verify Version & Cache**:
|
||
- Verify the footer / version string matches `1175f1a1d2c1` (or current production build tag).
|
||
- Turn on Airplane Mode and force-reload to confirm offline readiness. Turn Airplane Mode back off.
|
||
|
||
3. **30-Minute Screen-Off Battery Test**:
|
||
- Note baseline battery percentage (e.g., 85%).
|
||
- Select a pre-cached playlist or track and tap **Play**. Ensure continuous audio is heard.
|
||
- Lock the screen (display off) and let playback run uninterrupted for **30 minutes**.
|
||
- Note battery percentage at 30 minutes (calculate drain %/hr).
|
||
- Listen for any audio dropouts, stuttering, or premature process suspension.
|
||
|
||
4. **Interruption & Resumption Test**:
|
||
- Trigger an interruption while locked (e.g., initiate an incoming phone call or start Siri).
|
||
- End the interruption and observe if playback automatically resumes via `MediaSession` handlers.
|
||
- Test Lock Screen Controls (Play/Pause, Next Track, scrubber).
|
||
|
||
5. **Log Findings**:
|
||
- Record iPhone Model, iOS Version, Battery Drain (% per 30m), Dropouts Count (0 = pass), and Resumption Status (Pass/Fail).
|
||
|
||
---
|
||
|
||
## 7. After Phase 0 Benchmark (Step S5 Verification)
|
||
|
||
**Date**: 2026-10-07
|
||
**Commit**: `eb6aae7`
|
||
**Dataset**: `perf/results/after-phase0-2026-10-07.json` (compared against `perf/results/baseline-2026-10-07.json`)
|
||
|
||
### Headline Before vs. After Comparison Table (Median of 5 Runs)
|
||
|
||
| Browser / Profile | Scenario | Requests (Before → After) | Wire Bytes (Before → After) | FCP / LCP (ms) | Boot Done (ms) | SW Dur / Reload (ms) |
|
||
|---|---|---|---|---|---|---|
|
||
| **Chromium LTE** | Cold First Load | 164 → 166 (+1.2%) | 1,015.0 KB → 1,015.8 KB (+0.1%) | 1540/1896 → 1572/1916 | 10,163 → 10,480 (+3.1%) | 38,446 → 38,364 |
|
||
| | Warm Reload (SW) | 4 → 4 (0.0%) | 5.2 KB → 5.2 KB (0.0%) | 180/684 → 184/680 | 675 → 692 (+2.5%) | - |
|
||
| | Offline Reload | 0 → 0 (0.0%) | 0 B → 0 B (0.0%) | 72/108 → 76/112 | 219 → 259 | 240 → 282 |
|
||
| | Update (1-line JS) | 88 → 89 (+1.1%) | 596.6 KB → 597.0 KB (+0.1%) | - | - | 2,013 → 2,155 (+7.1%) |
|
||
| | Update (1-line CSS) | 88 → 89 (+1.1%) | 596.7 KB → 597.1 KB (+0.1%) | - | - | 1,970 → 2,107 (+7.0%) |
|
||
| **Chromium Lossy** | Cold First Load | 164 → 166 (+1.2%) | 1,015.0 KB → 1,015.8 KB (+0.1%) | 2344/2912 → 2372/2940 | 16,015 → 16,408 (+2.5%) | 42,518 → 42,031 |
|
||
| | Warm Reload (SW) | 4 → 4 (0.0%) | 5.2 KB → 5.2 KB (0.0%) | 184/644 → 188/648 | 651 → 768 (+18.0%) | - |
|
||
| | Offline Reload | 0 → 0 (0.0%) | 0 B → 0 B (0.0%) | 80/92 → 84/96 | 237 → 241 | 253 → 262 |
|
||
| | Update (1-line JS) | 88 → 89 (+1.1%) | 596.6 KB → 597.0 KB (+0.1%) | - | - | 2,031 → 2,115 (+4.1%) |
|
||
| | Update (1-line CSS) | 88 → 89 (+1.1%) | 596.7 KB → 597.1 KB (+0.1%) | - | - | 2,004 → 2,116 (+5.6%) |
|
||
| **WebKit LTE** | Cold First Load | 162 → 164 (+1.2%) | 1.12 MB → 1.13 MB (+0.1%) | 1088/1287 → 1100/1303 | 3,201 → 3,290 (+2.8%) | 30,508 → 31,026 |
|
||
| | Warm Reload (SW) | 0 → 0 (0.0%) | 0 B → 0 B (0.0%) | 110/110 → 112/112 | 131 → 139 (+6.1%) | - |
|
||
| | Offline Reload | 0 → 0 (0.0%) | 0 B → 0 B (0.0%) | 107/107 → 110/110 | 128 → 137 | 135 → 148 |
|
||
| | Update (1-line JS) | 86 → 87 (+1.2%) | 664.3 KB → 664.9 KB (+0.1%) | - | - | 4,457 → 4,584 (+2.8%) |
|
||
| | Update (1-line CSS) | 86 → 87 (+1.2%) | 664.3 KB → 664.9 KB (+0.1%) | - | - | 4,510 → 4,550 (+0.9%) |
|
||
| **WebKit Lossy** | Cold First Load | 162 → 164 (+1.2%) | 1.12 MB → 1.13 MB (+0.1%) | 3557/3735 → 3738/3905 | 7,303 → 7,305 (+0.0%) | 35,449 → 35,101 |
|
||
| | Warm Reload (SW) | 0 → 0 (0.0%) | 0 B → 0 B (0.0%) | 111/112 → 112/114 | 131 → 128 (-2.3%) | - |
|
||
| | Offline Reload | 0 → 0 (0.0%) | 0 B → 0 B (0.0%) | 109/119 → 111/120 | 129 → 132 | 141 → 139 |
|
||
| | Update (1-line JS) | 86 → 87 (+1.2%) | 664.3 KB → 664.9 KB (+0.1%) | - | - | 8,135 → 10,017 (+23.1%) |
|
||
| | Update (1-line CSS) | 87 → 87 (0.0%) | 664.4 KB → 664.9 KB (+0.1%) | - | - | 8,269 → 10,013 (+21.1%) |
|
||
|
||
### Hidden Timers Verification Result
|
||
|
||
Executed via `node perf/hidden-timers.mjs`:
|
||
```
|
||
=== Hidden Timers Verification ===
|
||
Initial visible intervals (8): anon1000, anon120000, checkBuildTag, chips, flush, maybeCheck, reanchor, rotateChips
|
||
Hidden intervals (3): anon1000, anon120000, maybeCheck
|
||
Restored visible intervals (9): anon1000, anon120000, anon50000, checkBuildTag, chips, flush, maybeCheck, reanchor, rotateChips
|
||
Page errors: none
|
||
|
||
Result: ALL ASSERTIONS PASSED
|
||
```
|
||
All five non-playback UI timers (`chips`, `reanchor`, `rotateChips`, `flush`, `checkBuildTag`) suspend immediately when `document.hidden` becomes true and resume immediately upon returning to visible, while essential background loops (`maybeCheck` inbox poll and `anon120000` 120s OPFS download resume check) remain continuously active.
|
||
|
||
### Variance & Noise Observed
|
||
|
||
Across the 5 iterations per scenario:
|
||
- **Transfer Volumes & Request Counts**: Completely deterministic (0.0% variance across all runs).
|
||
- **Boot Milestones & Paint Timings**: Stable within ±1–3% under LTE emulation on both Chromium and WebKit.
|
||
- **Update Duration under Lossy WebKit**: Exhibited expected network noise (~8.1 s vs ~10.0 s, ~21–23% delta) due to the random 1,500 ms connection stall injection pattern in `perf/proxy.mjs`.
|
||
- **Chromium Update Duration**: Clustered closely around ~2,100 ms with occasional ~100 ms fast runs when the browser resolves registration updates asynchronously.
|
||
|
||
### Findings
|
||
|
||
Introducing `frontend/visible-timer.js` successfully eliminates unnecessary CPU wakeups and timer execution during hidden tab states while adding virtually zero runtime transfer overhead (+1 request and +373–471 wire bytes on service worker updates; +2 requests and +843 wire bytes on initial cold visit). Background audio playback and essential network recovery handlers are preserved without regressing paint, boot, or offline capabilities.
|
||
|
||
### Structural Transfer Analysis: Fonts & Icons vs. Code, and Cold Double-Download
|
||
|
||
Detailed breakdown of network transfer patterns revealed two major inefficiencies in the current asset delivery architecture:
|
||
|
||
1. **Static Fonts and Icons vs. JS/CSS**:
|
||
- **Cold First Visit**: Total wire transfer is 1,040.2 KB. Web fonts (`PlusJakartaSans`, `BricolageGrotesque`, `HankenGrotesk`, `JetBrainsMono`, and `fonts.css`) consume **397.1 KB (38.2%)** and PNG icons consume **6.3 KB (0.6%)**, totaling **403.4 KB (38.8%)** of immutable media. Application JavaScript consumes **478.1 KB (46.0%)** and CSS consumes **130.7 KB (12.6%)**, totaling **608.8 KB (58.5%)**.
|
||
- **Update N -> N+1 Wire Cost**: On any single-line change to `app.js` or `theme-glass.css`, `sw.js` invalidates the entire shell cache and re-downloads all 89 `SHELL` items. Out of 611.3 KB transferred, **~259.5 KB (~42.5%)** is spent re-downloading static web fonts and icons that never changed, compared to **~321.9 KB (52.7%)** for JS/CSS.
|
||
2. **Cold First-Visit Shell Double-Download**:
|
||
- The initial visit downloads every app shell resource **twice over the wire**:
|
||
- First, the browser's HTML parser requests `index.html`, stylesheets, scripts, and fonts to render the document (83 requests, ~520 KB).
|
||
- Second, once `boot()` registers `/sw.js`, the service worker `install` event triggers `precacheShell()`. Line 165 of `frontend/sw.js` executes `fetch(new Request(url, { cache: 'reload' }))`.
|
||
- The explicit `{ cache: 'reload' }` directive bypasses the browser's HTTP cache entirely, forcing a redundant full re-download of all 89 shell assets from the server.
|
||
- As measured by the harness request log, cold first visit request count is **166 requests** (exactly double the shell count) and wire data transfer is **1,040 KB** (twice the unprimed payload).
|
||
|