Explain save failures and keep download diagnostics available
This commit is contained in:
Jonathan Sykes
109
docs/youtube-save-unavailable.md
Normal file
109
docs/youtube-save-unavailable.md
Normal file
@@ -0,0 +1,109 @@
|
||||
# Rock Medley save investigation
|
||||
|
||||
Video: `wZzRoXymOUU` — **Rock Medley**, Petra - Topic, 605 seconds.
|
||||
Investigation date: 2026-10-03. Main was pulled with `--ff-only` before changes.
|
||||
No production shell, credentials, cookies, vault, deployment or proxy was used.
|
||||
|
||||
## Evidence
|
||||
|
||||
YouTube's public oEmbed endpoint returns HTTP 200, the correct title/artist and
|
||||
thumbnail. This confirms the ID, but not playback permission. The watch page's
|
||||
initial player response on this development machine returns:
|
||||
|
||||
```json
|
||||
{"status":"UNPLAYABLE","reason":"Video unavailable"}
|
||||
```
|
||||
|
||||
The server's metadata invocation (`-J --no-warnings URL`) and the exact cache
|
||||
save invocation fail with `ERROR: [youtube] wZzRoXymOUU: Video unavailable`.
|
||||
Providing Bun as the JS runtime mirrors the Docker image's `/etc/yt-dlp.conf`.
|
||||
The error occurs during player extraction, before selecting/downloading a
|
||||
format. The server uses a ladder of selectors, not a hard-coded format ID.
|
||||
|
||||
| Local extractor | Default result |
|
||||
| --- | --- |
|
||||
| PATH yt-dlp 2026.07.04 | Video unavailable |
|
||||
| Repository binary 2026.08.19 | Video unavailable |
|
||||
| Downloaded nightly 2026.09.27.232945 | Video unavailable |
|
||||
|
||||
With 2026.08.19, explicit `web_embedded`, `web_safari`, `tv_simply`, `android_vr`,
|
||||
`mweb` and `web_music` clients all fail for this song too. A control video
|
||||
(`0gfX0dFLaBc`) succeeds from the same machine/runtime. This is video-specific
|
||||
upstream playability refusal, not a demonstrated stale-extractor, missing-format
|
||||
or broken-JS-runtime problem. It is **not exclusively reproducible on prod**.
|
||||
The generic response does not establish region, IP, account or licensing as the
|
||||
specific cause. Cookies or PO tokens were not requested or tested.
|
||||
|
||||
Upstream release notes: [2026.08.19](https://github.com/yt-dlp/yt-dlp/releases/tag/2026.08.19).
|
||||
[yt-dlp documentation](https://github.com/yt-dlp/yt-dlp#dependencies) describes
|
||||
runtime dependencies; no dependency update resolved this local reproduction.
|
||||
The Dockerfile is unchanged because an update alone is not a proven remedy.
|
||||
Its existing release-download layer is cached by Docker, so a future intentional
|
||||
extractor update must rebuild that layer, rather than assume new app code updates
|
||||
it automatically.
|
||||
|
||||
## Changes
|
||||
|
||||
1. Shared extraction now tries a bounded client ladder for generic unavailable
|
||||
and missing-format errors, as well as bot checks. This covers both metadata
|
||||
probes and actual saves. Explicit private/member/age/country/removal errors,
|
||||
local disk errors, cancellation and explicit client overrides do not trigger
|
||||
that ladder. The final diagnostic preserves the original failure and lists
|
||||
attempted clients; it does not mislabel the song as deleted.
|
||||
2. Explicit Retry POSTs `/api/download/:id/retry` before preparing a device save.
|
||||
It clears stale preparation/stream hints and calls the normal cache with
|
||||
`force:true` to bypass a **failed row's** cooldown. It does not delete ready
|
||||
media or duplicate running jobs. USB-volume, free-space/budget, duration and
|
||||
validation rules still belong to the existing cache. Repeated failed retries
|
||||
are limited to one per song per 15 seconds. Normal automatic resumes do not
|
||||
force a retry. Manual Retry follows the same new client ladder.
|
||||
3. Downloads, Settings → Downloads & storage, and save-failure toasts use plain
|
||||
language. Expandable “Technical details” retains the raw tool error as text,
|
||||
never HTML. Controls remain usable in both themes at 390/1440 px.
|
||||
|
||||
## Remaining reviewer check
|
||||
|
||||
**These changes do not demonstrate a successful download of Rock Medley.**
|
||||
The local default and all tested clients still receive YouTube's refusal.
|
||||
To establish the production-specific reason, the reviewer should compare:
|
||||
|
||||
- Actual yt-dlp version and the invocation below inside the deployed container.
|
||||
- Logged-out YouTube playback on the homelab's outbound network versus the
|
||||
working phone network; repeat with the phone on that same network if possible.
|
||||
- If only authenticated playback works, confirm whether the video needs account
|
||||
access before considering the existing optional read-only cookies mechanism.
|
||||
Do not assume a PO token is required without extractor diagnostics saying so.
|
||||
- If the failure tracks a network/region, use a permitted outbound network where
|
||||
the video is available, or a permitted alternate source/upload/device copy.
|
||||
Merely changing a region hint does not prove actual playback eligibility.
|
||||
|
||||
No network, cookies, token, Docker or deployment workaround was guessed or
|
||||
silently enabled. After any confirmed environment correction, Retry now starts
|
||||
a genuine fresh attempt instead of just returning the old backoff failure.
|
||||
|
||||
```sh
|
||||
# Metadata probe, same as the server (Docker config supplies the Bun runtime)
|
||||
yt-dlp -J --no-warnings 'https://www.youtube.com/watch?v=wZzRoXymOUU'
|
||||
|
||||
# Cache save, same selector/merge/rate args as the server's defaults
|
||||
yt-dlp 'https://www.youtube.com/watch?v=wZzRoXymOUU' --no-warnings --no-playlist \
|
||||
-f 'bv*[height<=720][vcodec^=avc1]+ba[ext=m4a]/bv*[height<=720][vcodec^=avc1]+ba/b[ext=mp4][vcodec^=avc1]/bv*[height<=720]+ba/b[ext=mp4]/b' \
|
||||
--merge-output-format mp4 --limit-rate 2M -o /tmp/rock-medley-check.mp4
|
||||
```
|
||||
|
||||
Tests: `node --test frontend/*.test.js`; app syntax check; unbundled server build;
|
||||
`bun test server/ytdlp-resilience.test.js server/download-retry.test.js
|
||||
server/media-cache.test.js`; and
|
||||
`npx playwright test --config playwright.download-retry.config.js`.
|
||||
The existing pool tests additionally need an importable yt-dlp zipapp:
|
||||
`YTDLP_PATH=/path/to/zipapp bun test server/ytdlp-pool.test.js`. The installed
|
||||
PATH executable here is a Python entry-point script, not an importable zipapp;
|
||||
providing the downloaded nightly zipapp makes all pool tests pass without any
|
||||
production code or environment changes.
|
||||
|
||||
Final validation: **118 frontend unit tests, 44 server tests and 13 browser cases
|
||||
pass**, plus the app syntax check and server build. Browser cases cover the real
|
||||
Retry controls in both Downloads and Settings, normal resumes, friendly failures,
|
||||
44 px details controls, safe raw text and overflow at phone/desktop widths.
|
||||
A real iPhone Safari/PWA save should still be checked after deployment, especially
|
||||
pause/resume of a partial copy and retry after a network change.
|
||||
Reference in New Issue
Block a user