Record phase 6 measurements and compatibility verification
This commit is contained in:
Jonathan Sykes
168
plans/phase6-verification.md
Normal file
168
plans/phase6-verification.md
Normal file
@@ -0,0 +1,168 @@
|
||||
# Phase 6 verification
|
||||
|
||||
Final product under test: `4b1f520` (Phase 5 parent `21c1273`). No production requests.
|
||||
All historical source worktrees contain matching server and frontend revisions.
|
||||
The complete diagnostic waterfalls and five-run acceptance samples are committed
|
||||
under `perf/results/phase6-*.json`. KB in these tables means 1,000 bytes.
|
||||
|
||||
## Cause and controlled trials
|
||||
|
||||
See [phase6-investigation.md](phase6-investigation.md) for the three-run tip
|
||||
comparisons, adjacent-commit bisect, rejected cache/handoff prototypes and fixes.
|
||||
The regressing commit is `5d6a154`: LTE FCP 1,032 → 1,615 ms immediately across
|
||||
that cut. Phase 2/3/5 medians are 1,044/1,646/1,736 ms. Phase 4 extraction did
|
||||
not introduce the paint regression.
|
||||
|
||||
Representative WebKit/LTE sample 2, proxy times relative to tracking start:
|
||||
|
||||
| Request | Phase 5 start/end ms | Phase 6 start/end ms |
|
||||
|---|---:|---:|
|
||||
| lazy.js | 250/417 | 250/418 |
|
||||
| styles.css | 251/570 | 251/570 |
|
||||
| theme-classic.css | 1540/1711 | 250/421 |
|
||||
| page app.js | 1524/2273 | 1552/2297 |
|
||||
| first sw.js | 2337/2533 | 2367/2564 |
|
||||
| worker app.js | 2903/3651 | absent |
|
||||
| FCP | 1736 | 905 |
|
||||
|
||||
The stylesheet becomes available before the parser applies it. The worker starts
|
||||
after paint. Default fetch caching was already in the modern synchronizer;
|
||||
force-cache and a page-to-worker HTTP-cache handoff both still transferred two
|
||||
app bodies in WebKit. A single page-owned response, verified and cached before
|
||||
boot registers the worker, removes the redundant app transfer. Controlled pages
|
||||
and other engines retain native external scripts; no window cache/hash lookup
|
||||
is added to those paths. Each retained stale app requires all shell contracts
|
||||
to match, enforced by the worker.
|
||||
|
||||
## Final measurements
|
||||
|
||||
Exact final command (120 samples: both engines, both profiles, six scenarios,
|
||||
five fresh samples each):
|
||||
|
||||
```sh
|
||||
node perf/baseline.mjs --runs 5 --browser all --profile all --scenario all --assert-cold --out perf/results/phase6-final-2026-10-08.json
|
||||
```
|
||||
|
||||
Final app.js positive-byte responses are exactly one in all 20 cold samples.
|
||||
The earlier separate cold/regression datasets used 9408ee0 and are diagnostics,
|
||||
not the final acceptance build. `phase6-native-auto-priority-2026-10-08.json`
|
||||
combines that rejected intermediate sweep. Final core contract is 2: the final
|
||||
section-rail script now owns the loader; a stale pre-packing section script would
|
||||
not load app.js. No Phase 5 SHELL file was deleted. app.js and sw-update.js remain
|
||||
byte-for-byte unchanged.
|
||||
|
||||
Actual final command output excerpts:
|
||||
|
||||
```text
|
||||
Chromium/LTE: run 4/5... done (653267 bytes, 115 reqs, fcp=1396ms, boot=6478ms)
|
||||
Chromium/lossy: run 2/5... done (653267 bytes, 115 reqs, fcp=1972ms, boot=10292ms)
|
||||
WebKit/LTE: run 2/5... done (983326 bytes, 158 reqs, fcp=905ms, boot=2335ms)
|
||||
WebKit/lossy: run 2/5... done (990152 bytes, 161 reqs, fcp=2748ms, boot=5046ms)
|
||||
```
|
||||
|
||||
WebKit FCP targets (≤1300/3800 ms) pass. Contemporary Phase 2 three-run medians:
|
||||
1044/4045 ms; its lossy samples are 4045/3218/5297, all retained. Original baseline
|
||||
WebKit cold wire is 1,179,624 bytes. Final LTE/lossy wire is 983,326/990,152 bytes
|
||||
(16.6%/16.1% lower). The final cold set transfers app.js once in both engines.
|
||||
|
||||
|
||||
## Complete final comparison
|
||||
|
||||
Five-run medians. Each cell is Phase 5 → Phase 6; bytes are exact body wire bytes.
|
||||
|
||||
| Browser/profile | Cold requests | Cold bytes | FCP ms | LCP ms | Boot ms | Long tasks ms |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
| chromium/lte | 115 → 115 | 651012 → 653267 | 1356 → 1388 | 1356 → 1388 | 7043 → 6478 | 278 → 304 |
|
||||
| chromium/lossy | 115 → 115 | 651096 → 653267 | 2052 → 1976 | 2052 → 1976 | 11342 → 10314 | 295 → 297 |
|
||||
| webkit/lte | 159 → 158 | 1097048 → 983326 | 1739 → 905 | 1810 → 982 | 2315 → 2335 | 0 → 0 |
|
||||
| webkit/lossy | 162 → 161 | 1105402 → 990152 | 4303 → 2748 | 4371 → 2821 | 5573 → 5046 | 0 → 0 |
|
||||
|
||||
WebKit does not expose the Long Tasks API; its zero fields mean unmeasured.
|
||||
|
||||
| Browser/profile | Warm boot ms | Offline boot ms | Offline reload ms | Core update ms/bytes | CSS update ms/bytes | Feature update ms/bytes |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
| chromium/lte | 410 → 418 | 185 → 188 | 202 → 206 | 745/125795 → 872/126186 | 1380/35363 → 1388/35760 | 1378/33408 → 1392/33762 |
|
||||
| chromium/lossy | 403 → 442 | 189 → 194 | 204 → 207 | 747/125795 → 805/126186 | 1377/35363 → 1383/35760 | 1375/33408 → 1386/33762 |
|
||||
| webkit/lte | 83 → 88 | 86 → 88 | 100 → 102 | 1890/147259 → 2015/147626 | 1455/45659 → 1467/46254 | 1455/43318 → 1380/41860 |
|
||||
| webkit/lossy | 85 → 89 | 81 → 85 | 95 → 98 | 4192/147259 → 2934/147626 | 3647/43705 → 3687/44211 | 3644/41364 → 3693/41860 |
|
||||
|
||||
## Limits and exact unmet parity targets
|
||||
|
||||
Strict zero regression versus the recorded Phase 5 run is NOT achieved.
|
||||
Chromium/LTE FCP is 1388 versus 1356 ms (+32); cold bytes 653267 versus 651012
|
||||
(+2255, 0.35%); long tasks 304 versus 278 ms. The verified loader, CSP permission,
|
||||
contract checks and metadata add code/bytes; parser insertion also changes task
|
||||
boundaries. The whole timing delta has not been causally isolated. Three-run
|
||||
contemporary Phase 5 LTE medians were FCP 1368, boot 7090 and long tasks 294 ms.
|
||||
Final Chrome boot improves to 6478/10314 ms (Phase 5 7043/11342).
|
||||
|
||||
Warm/offline/update deltas in the complete table remain explicit. Contemporary
|
||||
five-run Phase 5 warm boots were Chrome 426/446 and WebKit 88/88 ms; final values
|
||||
418/442 and 88/89 ms. Contemporary Phase 5 core-update times were Chrome 869/799
|
||||
and WebKit 1955/2776 ms; final 872/805 and 2015/2934 ms. Metadata adds roughly
|
||||
0.4 KB to core updates. Periodic lossy stalls depend on request order; timing
|
||||
comparisons cannot establish exact parity. The extra CPU/cache work is small,
|
||||
but zero cost was not proved, and the positive timing deltas are not dismissed.
|
||||
|
||||
Every final update passed assertChangedAssets: only app.js, theme-glass.css or
|
||||
presenter.js is downloaded as application code, with index/API/worker metadata;
|
||||
fonts/icons are not re-downloaded. Whole CSS-update wire remains above 30 KB,
|
||||
a pre-existing Phase 5 gap. Readiness and completion remain separately measured.
|
||||
|
||||
Real iPhone Safari/PWA code-cache behavior, OPFS quota pressure, forced cache
|
||||
eviction and uninterrupted locked-screen/background audio for 30 minutes were
|
||||
not verified. The historical media fixture is invalid for autoplay: these are
|
||||
home-launch metrics, not media-start measurements. No playback code changed.
|
||||
Other selected layouts may prefetch 4079 unused classic CSS bytes; that preload
|
||||
never applies, and the selected stylesheet retains its original cascade order.
|
||||
|
||||
## Compatibility verification on the final product
|
||||
|
||||
All commands below ran on 4b1f520, build 31617662d2f0:
|
||||
|
||||
```sh
|
||||
node perf/migration.mjs --browser all --profile unthrottled --out perf/results/phase6-migration-unthrottled-2026-10-08.json
|
||||
node perf/migration.mjs --browser all --profile lossy --out perf/results/phase6-migration-lossy-2026-10-08.json
|
||||
node perf/hidden-timers.mjs
|
||||
node perf/lazy.mjs --browser all --out perf/results/phase6-lazy-2026-10-08.json
|
||||
node perf/seams.mjs --browser all
|
||||
ASSET_SYNC=0 node perf/baseline.mjs --runs 1 --browser all --profile lte --scenario offline --out perf/results/phase6-sync-rollback-2026-10-08.json
|
||||
ASSET_HASHING=0 node perf/baseline.mjs --runs 1 --browser all --profile lte --scenario offline --out perf/results/phase6-hashing-rollback-2026-10-08.json
|
||||
node --test frontend/*.test.js
|
||||
node --test perf/measurement.test.mjs
|
||||
(cd server && bun install && PATH="$PWD/../perf/.tmp/test-venv/bin:$PATH" bun run test)
|
||||
```
|
||||
|
||||
Actual output excerpts:
|
||||
|
||||
```text
|
||||
Migration unthrottled: chromium banners=1 updateReloads=1 cachedFiles=122 userInitiatedOnly=true offline=true playbackGuard=true evictionRepair=true
|
||||
Migration unthrottled: webkit banners=1 updateReloads=1 cachedFiles=122 userInitiatedOnly=true offline=true playbackGuard=true evictionRepair=true
|
||||
Migration lossy: chromium durationMs=25233 banners=1 updateReloads=1 legacyURLs=21 userInitiatedOnly=true offline=true
|
||||
Migration lossy: webkit durationMs=22860 banners=1 updateReloads=1 legacyURLs=21 userInitiatedOnly=true offline=true
|
||||
Hidden timers: Page errors: none; Result: ALL ASSERTIONS PASSED
|
||||
Lazy: chromium PASS; webkit PASS
|
||||
Seams: chromium checked=[shell,presenter,remote,party,share,stats] offlineNetworkRequests=0 errors=[]
|
||||
Seams: webkit checked=[shell,presenter,remote,party,share,stats] offlineNetworkRequests=0 errors=[]
|
||||
Frontend: tests 223; pass 223; fail 0; skipped 0
|
||||
Measurement: tests 3; pass 3; fail 0
|
||||
Server: sum of per-file output: 184 pass; 0 fail
|
||||
```
|
||||
|
||||
Both rollback modes launched offline with zero network in Chromium and WebKit.
|
||||
The archived b77938a fixture was rebuilt by the migration script each time.
|
||||
Legacy reload remains user-initiated; its pre-existing Refresh UI timer exception
|
||||
is unchanged. New-page playback guards, old-client readiness, one banner, old
|
||||
plain/?v=oldTag/?__ytpfresh URLs, already-open old tabs and cached offline reload
|
||||
remain covered. Four expected WebKit disconnected API transport errors per
|
||||
migration were recorded and matched to failed API requests; no script errors.
|
||||
The seam harness's historical Phase 4 artifact was restored after saving the
|
||||
new output as phase6-seams-2026-10-08.json.
|
||||
|
||||
Self-review re-read the complete product diff and master §§3.2/3.3/6/6b.
|
||||
No frontend import was added to server code; no original shell file was removed;
|
||||
no auto skipWaiting, live stateful re-evaluation, playback edit, push or deploy.
|
||||
Cache responses are exact-key checked; stale bodies under a new URL are no-store
|
||||
and carry no decoded transport encoding. Core contract 2 blocks loader-free N-1
|
||||
packing, while same-contract app fallback and executed feature pinning remain
|
||||
covered by both-engine smoke. Failed quota writes never delete saved music.
|
||||
Reference in New Issue
Block a user