305 lines
22 KiB
Markdown
305 lines
22 KiB
Markdown
# ytplayer — Feature Ideas & Enhancement Roadmap
|
||
|
||
> **Status:** Research & Proposed Roadmap
|
||
> **Target:** Web PWA, Tauri (Windows), zero-native (macOS / Linux)
|
||
> **Date:** October 2026
|
||
|
||
---
|
||
|
||
## 1. Context & Architecture Baseline
|
||
|
||
`ytplayer` is an ad-free, privacy-preserving media engine with a dual identity:
|
||
1. **Ad-Free Media Player & Offline PWA**: Built with vanilla JS on the frontend and Bun + Hono on the backend (libsql SQLite database). It uses `yt-dlp` and InnerTube for direct stream extraction without Google accounts, and caches media locally (OPFS in browser, single-file H.264/HEVC faststart on server, WebRTC P2P mesh between devices).
|
||
2. **Live Worship & Stage Performance Tool**: Equipped with **Service Mode** (wake-locked setlist view, auto-scaling lyric fonts, per-song arrangement notes), synchronized LRC lyrics with section/cue tags, and a **Presenter screen** (`/?present=<code>`) with QR pairing.
|
||
|
||
This document synthesizes high-value feature opportunities gathered from research into modern YouTube frontends (FreeTube, ReVanced, SmartTube, Namida), modern music players (Plexamp, Spotube), and live stage tools (Planning Center Music Stand, OnSong, MultiTracks Playback).
|
||
|
||
---
|
||
|
||
## 2. Feature Proposals by Domain
|
||
|
||
### Domain 1: Smart YouTube & Playback Experience
|
||
|
||
#### 1.1 SponsorBlock & Non-Music Segment Skipping
|
||
* **The Concept**: Automatically skip or visually mark sponsored segments, self-promos, interaction reminders, and non-music sections.
|
||
* **Why it matters**: Music videos frequently feature skits, prolonged silent pauses, or spoken intros. Skipping `music_offtopic` ensures the music starts immediately.
|
||
* **Technical Path**:
|
||
* Free, unauthenticated REST API: `GET https://sponsor.ajay.app/api/skipSegments?videoID=<id>&categories=["sponsor","intro","outro","music_offtopic","selfpromo"]`.
|
||
* Highlight segments as color-coded spans over the seek bar (`#timeline` / Glass Stage scrubber).
|
||
* During the playback tick in `frontend/app.js`, if `currentTime` enters a skippable segment and auto-skip is enabled in Settings, seek to the segment end and show an "Unskip" toast.
|
||
* Server media cache can optionally apply `--sponsorblock-mark` during ffmpeg intake.
|
||
|
||
#### 1.2 Automatic YouTube Chapter Extraction & Interactive Scrubber
|
||
* **The Concept**: Automatically parse video chapters from YouTube metadata and display chapter splits on the progress bar.
|
||
* **Why it matters**: Currently, chapters are only created manually. Full albums, sermons, conference talks, and live sets already have timestamps in video descriptions.
|
||
* **Technical Path**:
|
||
* In `server/server.js` (`/api/streams`), expose `chapters: [{ start_time, end_time, title }]` directly from InnerTube or `yt-dlp`.
|
||
* Render tick marks / gaps on the progress scrubber for each chapter boundary.
|
||
* Display the current chapter title next to the timestamp.
|
||
* Add keyboard shortcuts: `[` (previous chapter) and `]` (next chapter).
|
||
|
||
#### 1.3 Return YouTube Dislike (RYD) & Quality Rating
|
||
* **The Concept**: Show community like/dislike ratios and ratings for videos.
|
||
* **Why it matters**: Helps identify misleading arrangements, clickbait, poor audio rips, or broken videos before playback.
|
||
* **Technical Path**:
|
||
* Query `https://returnyoutubedislikeapi.com/votes?videoId=<id>` (public, CORS-enabled).
|
||
* Display a discreet like ratio bar or percentage badge on the video card and now-playing details.
|
||
|
||
#### 1.4 On-Screen Subtitles & Closed Captions (CC) Overlay
|
||
* **The Concept**: Render synchronized captions directly over the video playback stage.
|
||
* **Why it matters**: `ytplayer` currently only uses captions for transcript search or importing into lyrics. Spoken sermons, tutorials, foreign tracks, and non-musical videos need readable on-screen subtitles.
|
||
* **Technical Path**:
|
||
* Add a `CC` button on the player controls.
|
||
* Fetch timed lines from `/api/notes/:id/captions`.
|
||
* Render an HTML overlay synchronized to `Player.master.currentTime` with settings for font size, background opacity, and positioning.
|
||
|
||
#### 1.5 Silence Skipping (Talk & Sermon Mode)
|
||
* **The Concept**: Automatically speed up or skip past silent gaps in spoken-word media.
|
||
* **Why it matters**: Long sermons, podcasts, and talks often have pauses; skipping silence saves 10–20% of listening time without altering speech tone.
|
||
* **Technical Path**:
|
||
* Connect an `AnalyserNode` or `AudioWorklet` in the Web Audio graph.
|
||
* If audio energy (RMS) stays below -45 dB for >800 ms, ramp `playbackRate` to 2.5× until speech resumes.
|
||
|
||
---
|
||
|
||
### Domain 2: Audiophile & Music Player Polish
|
||
|
||
#### 2.1 EBU R128 / LUFS Loudness Normalization
|
||
* **The Concept**: Eliminate jarring volume jumps between tracks in playlists and queues.
|
||
* **Why it matters**: YouTube master volumes vary widely (-24 LUFS to -8 LUFS). Normalization ensures consistent listening levels.
|
||
* **Technical Path**:
|
||
* When generating waveform peaks in `server/media-cache.js`, run ffmpeg's `ebur128` filter once to measure integrated loudness (LUFS).
|
||
* Store `lufs` in the database and pass it via `/api/streams` and `/api/media/:id`.
|
||
* In `frontend/app.js`, apply a Web Audio `GainNode` to normalize tracks to a standard -14 LUFS target.
|
||
|
||
#### 2.2 Real-Time Pitch Shifting & Independent Tempo Adjustment
|
||
* **The Concept**: Transpose playback key (±6 semitones) without altering tempo, or slow down tempo without changing pitch.
|
||
* **Why it matters**: Singers and worship teams often need to adapt songs to a band's vocal key. Instrumentalists need to slow down fast sections to learn them.
|
||
* **Technical Path**:
|
||
* Use an AudioWorklet running a Phase Vocoder or `SoundTouchJS` DSP processor.
|
||
* Provide stepper controls for semitone transposition and tempo percentage (0.5×–1.5×).
|
||
|
||
#### 2.3 Ambient Glow / Reactive Lighting Mode
|
||
* **The Concept**: A soft dynamic glow behind the video player that mirrors current video colors (similar to YouTube Ambient Mode or Apple Music dynamic art).
|
||
* **Technical Path**:
|
||
* Render video frames onto an offscreen 16×16 `<canvas>` at ~10 FPS.
|
||
* Display a blurred canvas (`filter: blur(50px)`) directly behind the player container with smooth CSS opacity.
|
||
* Auto-disable when `settings.perfMode` is active or under `prefers-reduced-motion`.
|
||
|
||
#### 2.4 Smart Autoplay Radio / Endless Mix
|
||
* **The Concept**: Keep playing related songs when the active playlist or queue concludes.
|
||
* **Technical Path**:
|
||
* Fetch candidate related tracks via InnerTube `next` continuity tokens.
|
||
* Re-rank candidates using the client's local `StatsTrack` (`playCount`, listening duration, and skips) to favor familiar favorites while avoiding immediate repeats.
|
||
|
||
#### 2.5 Audio Stem Separation & Vocal Attenuator (Karaoke / Rehearsal Mode)
|
||
* **The Concept**: Mute or attenuate lead vocals in real time (`🎤 Vocal: 100% ────● 0%`), or isolate the vocal track for harmony practice.
|
||
* **Why it matters**: Gives singers clean backing tracks for rehearsal and live events even when official minus-one recordings do not exist.
|
||
* **Technical Path**:
|
||
* Client-side: Run `demucs-onnx` with WebGPU acceleration to process saved OPFS tracks offline.
|
||
* Server-side: Provide `GET /api/media/:id/stems` (caching `.vocals.m4a` and `.instrumental.m4a` alongside the existing `.m4a` audio sidecar in `server/media-cache.js`).
|
||
* Frontend: Two synchronized audio elements or Web Audio `GainNode` faders for seamless real-time blending.
|
||
|
||
#### 2.6 10-Band Graphic Equalizer & Spatial Hall Simulation
|
||
* **The Concept**: Upgrade the existing 5-band filter in `EQ` (`frontend/app.js`) to a full 10-band graphic equalizer with acoustic presets and spatial audio widening.
|
||
* **Technical Path**:
|
||
* 10 `BiquadFilterNode` bands (31Hz, 62Hz, 125Hz, 250Hz, 500Hz, 1kHz, 2kHz, 4kHz, 8kHz, 16kHz).
|
||
* Presets: *Vocal Clarity (speech presence boost), Acoustic Warmth, Worship Stage, Bass Punch, and Flat*.
|
||
* Spatial stereo widener using a Haas-effect micro-delay and `StereoPannerNode` for an immersive headphone soundstage.
|
||
|
||
---
|
||
|
||
### Domain 3: Live Worship, Band & Stage Performance Engine
|
||
|
||
#### 3.1 Interactive ChordPro Sheets & Live Transposition
|
||
* **The Concept**: Display chords inline or above lyrics with 1-tap live transposition and capo calculation.
|
||
* **Why it matters**: Replaces the need for external music stand apps (Planning Center Music Stand, OnSong) on stage.
|
||
* **Technical Path**:
|
||
* Extend `frontend/lyrics-core.js` to parse ChordPro directives (`[G]`, `[D/F#]`, `{key: G}`).
|
||
* Add a Transpose controller (+1 / -1 semitone, Capo selector) that dynamically recalculates chord symbols in real time.
|
||
* Add a view toggle: "Lyrics Only" vs. "Chords & Lyrics".
|
||
|
||
#### 3.2 Bluetooth Foot Pedal / Page-Turner Support
|
||
* **The Concept**: Hands-free navigation for musicians on stage using standard Bluetooth foot pedals (AirTurn, PageFlip, Donner, Stomp).
|
||
* **Why it matters**: Musicians cannot remove their hands from instruments while playing.
|
||
* **Technical Path**:
|
||
* Standard foot pedals map to keyboard keys: `PageDown`, `PageUp`, `ArrowUp`, `ArrowDown`.
|
||
* Extend the global `keydown` listener in `frontend/app.js` to handle pedal events in Service Mode:
|
||
* `PageDown` / `PageUp`: Scroll down / up by one lyric section.
|
||
* Configurable actions: Next/Previous track, trigger A-B loop, or jump to Chorus.
|
||
|
||
#### 3.3 Confidence Monitor & Stage Clock (Presenter Mode)
|
||
* **The Concept**: A high-visibility stage display mode for the Presenter screen (`/?present=<code>`).
|
||
* **Why it matters**: Singers and worship leaders need glanceable cues for upcoming lyrics, section names, and remaining time.
|
||
* **Technical Path**:
|
||
* In `Presenter` (`frontend/app.js`), provide a **Confidence Monitor** layout:
|
||
* **Current Line**: Extra-large, high-contrast text.
|
||
* **Next Line Preview**: Smaller, dimmed text directly underneath.
|
||
* **Section Badge**: Prominent label (e.g. `# CHORUS 2`).
|
||
* **Stage Clock & Timer**: Current wall clock plus elapsed/remaining song timer.
|
||
|
||
#### 3.4 Broadcast Lower-Third Overlay for OBS & Live Streams
|
||
* **The Concept**: A dedicated live stream output URL (`/?present=<code>&mode=lowerthird`) designed to be added as a transparent Browser Source in OBS, vMix, or ATEM.
|
||
* **Why it matters**: Enables churches and streamers to overlay synchronized lower-third lyrics onto their video feed without separate presentation software.
|
||
* **Technical Path**:
|
||
* Transparent background (`background: transparent;`).
|
||
* Auto-updating 2-line broadcast graphic with smooth slide/fade transitions and drop-shadows.
|
||
* Driven in real-time by the host or phone remote via the existing WebSocket relay in `server/remote.js`.
|
||
|
||
#### 3.5 Section Quick-Jump Pad
|
||
* **The Concept**: A grid of buttons representing song sections (`[Intro]`, `[Verse 1]`, `[Chorus]`, `[Bridge]`, `[Outro]`).
|
||
* **Why it matters**: Worship teams often repeat choruses or spontaneously transition between sections during live services.
|
||
* **Technical Path**:
|
||
* Parse `# Section` headers from synced lyrics in `frontend/lyrics-core.js`.
|
||
* Render an arrangement button strip in Service Mode.
|
||
* Tapping a section immediately seeks playback to that section's start timestamp.
|
||
|
||
#### 3.6 Hardware MIDI Controller Integration (Web MIDI API)
|
||
* **The Concept**: Direct plug-and-play support for hardware USB/Bluetooth MIDI pedalboards (Morningstar MC3/MC6, Looptimus, Behringer FCB1010) without native drivers.
|
||
* **Technical Path**:
|
||
* Use the browser `navigator.requestMIDIAccess()` API.
|
||
* Map MIDI Note-On and CC messages to player actions (Play/Pause, Next Track, Chorus Repeat, Tap Tempo, Mute).
|
||
|
||
#### 3.7 Planning Center Services (PCO) & SongSelect Setlist Importer
|
||
* **The Concept**: One-click setlist and chord chart importing from Planning Center Services and CCLI SongSelect.
|
||
* **Technical Path**:
|
||
* Import setlist text or CSV files from PCO.
|
||
* Auto-match titles with YouTube media, attaching keys (`@ Key E`) and chord sheets directly to playlist entries.
|
||
|
||
#### 3.8 Bilingual & Phonetic Lyrics (Romanization / Transliteration)
|
||
* **The Concept**: Simultaneous dual-language or phonetic pronunciation lyrics (e.g. Mandarin with Pinyin, Spanish/English).
|
||
* **Technical Path**:
|
||
* Extend `frontend/lyrics-core.js` to parse secondary translation lines.
|
||
* Display translations or romanized cues in smaller muted text under the primary lyric line.
|
||
|
||
#### 3.9 Visual Metronome & Count-In Click
|
||
* **The Concept**: Visual flash or gentle audio click count-in before track playback begins.
|
||
* **Why it matters**: Enables the band to start playing on beat before the recorded track introduces audio.
|
||
* **Technical Path**:
|
||
* When `@ 70 BPM` metadata is present in song notes, offer an optional 4-beat visual flash on the stage frame or Web Audio click before playback starts.
|
||
|
||
---
|
||
|
||
### Domain 4: Interactive Piano Transcription, Sheet Music & Practice Engine
|
||
|
||
#### 4.1 Automatic Audio-to-Piano Transcription (Polyphonic Audio to Score)
|
||
* **The Concept**: Transcribe piano melodies and chords from recorded audio into polyphonic note events (MIDI / MusicXML).
|
||
* **Why it matters**: Allows musicians, vocalists, and students to learn the exact piano parts played in any YouTube song or uploaded track.
|
||
* **Technical Path**:
|
||
* **Option A (Client-Side WASM / WebGPU)**: Run Spotify's **Basic Pitch** model (`@spotify/basic-pitch`) in the browser via TensorFlow.js or ONNX Runtime Web. It performs polyphonic pitch prediction directly on decoded audio frames without server compute.
|
||
* **Option B (Server-Side Worker, like `lyrics-worker`)**: Run ByteDance's `piano_transcription_inference` (High-Resolution Piano Transcription on PyTorch/ONNX) inside a container worker. Saves `<id>.notes.json` / `<id>.mid` to the cache directory.
|
||
* Output data shape: Array of note events `[{ pitch: 60, start: 1.25, end: 1.85, velocity: 84 }]`.
|
||
|
||
#### 4.2 Sheet Music Generation & Real-Time Scrolling Score
|
||
* **The Concept**: Convert transcribed note events into standard grand staff sheet music (Treble and Bass clefs) rendered on screen.
|
||
* **Technical Path**:
|
||
* Render sheet notation on an HTML5 `<canvas>` or `<svg>` using **VexFlow** or **OpenSheetMusicDisplay (OSMD)**.
|
||
* Display a vertical playhead cursor that scrolls synchronously with `Player.master.currentTime`.
|
||
* Support instant transposition: adjusting the song key shifts the musical notation and key signature on the fly.
|
||
|
||
#### 4.3 Interactive Virtual Piano & Falling Notes (Synthesia Style)
|
||
* **The Concept**: A responsive 88-key or 61-key virtual piano keyboard that visualizes which keys to press in sync with playback.
|
||
* **Visual Modes**:
|
||
* **Falling Notes Mode**: Vertical roll above the keyboard where color-coded note bars fall downward toward the keys (Treble notes in Amber/Coral, Bass notes in Cyan/Blue). When a bar hits the key line, the corresponding piano key illuminates and depresses.
|
||
* **Sheet Music + Lit Keys Mode**: Traditional score at the top, illuminated keyboard at the bottom highlighting active notes in real time.
|
||
* **Technical Path**:
|
||
* Canvas-rendered piano keyboard with responsive key sizing that fits desktop, tablet, and mobile orientations.
|
||
* Note rendering loop tied to `requestAnimationFrame` and driven by the current playback timestamp.
|
||
|
||
#### 4.4 Queued Song & Section Learning Mode
|
||
* **The Concept**: A dedicated learning queue and practice loop system that guides users step-by-step through mastering a song.
|
||
* **Features**:
|
||
* **Queue Practice Sessions**: Queue an entire song, specific sections (e.g. `[Intro]`, `[Verse 1]`, `[Chorus]`), or arbitrary A-B loop bars into the practice queue.
|
||
* **Tempo Slowdown (Pitch-Preserved)**: Slow down playback to 50%, 65%, 75%, or 90% speed without altering key.
|
||
* **Speed Ramp Auto-Trainer**: Start a looped section at 60% speed; each time the user completes the loop, speed automatically increases by +5% until 100% full tempo is achieved.
|
||
* **Loop Pause Interval**: A configurable 1–2 beat pause between repetitions to allow players to reset their hand position.
|
||
|
||
#### 4.5 Interactive "Wait-for-Note" Practice via Hardware MIDI Input
|
||
* **The Concept**: Connect any USB or Bluetooth digital piano / MIDI keyboard to the computer or tablet. The player pauses and waits for the user to press the correct key(s) before advancing!
|
||
* **Technical Path**:
|
||
* Listen to incoming MIDI `noteon` events via the Web MIDI API (`navigator.requestMIDIAccess()`).
|
||
* Compare the user's played MIDI note number against active notes in the transcription dataset.
|
||
* If the note is correct, turn the key green and resume playback to the next note event. If incorrect, turn the key red and provide visual guidance on the virtual keyboard.
|
||
|
||
#### 4.6 Technical Feasibility Analysis
|
||
|
||
| Component | Technical Feasibility | Resource Cost & Constraints | Recommendation |
|
||
|:---|:---|:---|:---|
|
||
| **Audio-to-MIDI Transcription** | **High (Proven)** | Basic Pitch: ~15 MB model file, runs on client WebGPU/CPU in ~0.3×–0.8× real time. ByteDance: higher accuracy, requires ~1.5 GB PyTorch model on server. | Use **Basic Pitch (WASM/ONNX)** on client for instant preview; offload to homelab server worker for full-song high-fidelity scores. |
|
||
| **Piano Roll / Virtual Keyboard** | **High (Straightforward)** | Pure HTML5 Canvas rendering. Minimal CPU overhead (~60 FPS with <5% CPU usage). | Built natively into frontend without heavy external dependencies. |
|
||
| **Sheet Music Engraving** | **High (Proven)** | **VexFlow** (~180 KB minified) renders standard grand staff notation cleanly from JSON notes. | Dynamic import (`import()`) of VexFlow only when the user opens Sheet View. |
|
||
| **Interactive MIDI Feedback** | **High (Zero-latency)** | Web MIDI API is supported natively in Chrome, Edge, and Opera (and via polyfill in Firefox/Safari). Latency is sub-5ms. | Native browser event listeners listening to MIDI input. |
|
||
|
||
---
|
||
|
||
### Domain 5: Collaboration, Windowing & Mobile Experience
|
||
|
||
#### 5.1 Document Picture-in-Picture (Always-On-Top Lyrics & Chords)
|
||
* **The Concept**: An always-on-top floating desktop window containing interactive synchronized lyrics, chords, and transport controls.
|
||
* **Technical Path**:
|
||
* Use the Chrome/Edge `documentPictureInPicture.requestWindow({ width: 420, height: 500 })` API.
|
||
* Moves the active `#stageLyrics` or ChordPro view into the floating window with full DOM interactivity.
|
||
|
||
#### 5.2 Voice Search & Driving Car Mode
|
||
* **The Concept**: A hands-free, distraction-free interface for driving or kitchen listening.
|
||
* **Features**:
|
||
* **Voice Search**: Microphone button on the search bar powered by `webkitSpeechRecognition` for real-time speech-to-query search.
|
||
* **Car Mode**: One-tap toggle for a simplified layout with giant 100px touch buttons (Play/Pause, Next, Previous, 15s Rewind) and disabled scroll lists.
|
||
|
||
#### 5.3 Collaborative Setlists & Playlists
|
||
* **The Concept**: Multiple devices editing a single playlist simultaneously with live sync.
|
||
* **Technical Path**:
|
||
* Extend playlist sync in `server/db.js` with shared write keys.
|
||
* Broadcast item reordering and song additions over the existing WebSocket hub in `server/party.js`.
|
||
|
||
#### 5.4 Watch Party DJ Mode & Upvote Queue
|
||
* **The Concept**: Allow guests in a watch party (`/ws/party`) to submit song requests that the group can upvote.
|
||
* **Technical Path**:
|
||
* Add a "Suggested Queue" tab in the watch party drawer.
|
||
* Upvotes dynamically sort songs; when current playback finishes, the highest-voted song plays next.
|
||
|
||
---
|
||
|
||
### Domain 6: Library, Offline & Sync Innovations
|
||
|
||
#### 6.1 Visual Download Manager & Storage Dashboard
|
||
* **The Concept**: A unified management dashboard in Settings for offline media.
|
||
* **Technical Path**:
|
||
* Show active download tasks with real-time progress bars.
|
||
* List storage usage grouped by playlist, audio vs. video format, and OPFS vs. server cache.
|
||
* Add controls: "Pause All", "Resume All", and "Retry Failed".
|
||
|
||
#### 6.2 Smart Auto-Offline Favorites
|
||
* **The Concept**: Keep a user's most essential songs available offline automatically.
|
||
* **Technical Path**:
|
||
* Setting: *"Keep Top 25 Most-Played Songs Offline"* or *"Keep Active Setlist Offline"*.
|
||
* Automated background routine checks local `playCount` and triggers cache downloads for missing items during idle time.
|
||
|
||
#### 6.3 Self-Contained Offline Playlist Bundling (Zip Export)
|
||
* **The Concept**: Export a full offline bundle containing playlist JSON plus the saved audio/video files for complete air-gapped device migration.
|
||
|
||
---
|
||
|
||
## 3. Implementation Matrix & Recommended Phases
|
||
|
||
| Phase | Feature | Effort | Prerequisites & Reused Modules |
|
||
|:---|:---|:---:|:---|
|
||
| **Phase 1: Quick Wins** | **SponsorBlock & Non-Music Skip** | Low | Free SponsorBlock API, playback tick in `frontend/app.js` |
|
||
| | **Automatic YouTube Chapter Ticks** | Low | InnerTube metadata in `server/server.js`, timeline scrubber |
|
||
| | **Bluetooth Foot Pedal Shortcuts** | Low | Keydown event listener in `frontend/app.js` |
|
||
| | **Section Quick-Jump Pad** | Low | Lyric section parser in `frontend/lyrics-core.js` |
|
||
| | **Voice Search (SpeechRecognition)** | Low | Web Speech API, search input handler |
|
||
| **Phase 2: Stage & Audio** | **EBU R128 LUFS Normalization** | Medium | ffmpeg `ebur128` in `server/media-cache.js`, Web Audio GainNode |
|
||
| | **ChordPro Display & Transpose** | Medium | `frontend/lyrics-core.js`, chord parser |
|
||
| | **Confidence Monitor Stage View** | Medium | `Presenter` & `fitLyricLines` in `frontend/app.js` |
|
||
| | **OBS Lower-Third Overlay** | Low-Med | `server/remote.js` WebSocket relay, transparent route |
|
||
| | **On-Screen Subtitles (CC)** | Medium | `/api/notes/:id/captions`, overlay renderer |
|
||
| **Phase 3: Piano & Interactive Learning** | **Interactive Piano Roll & Key Visualizer** | Medium | HTML5 Canvas keyboard renderer, note event synchronizer |
|
||
| | **Audio-to-MIDI Transcription** | Medium-High | Spotify Basic Pitch (WASM) / ByteDance transcription worker |
|
||
| | **Sheet Music View (VexFlow)** | Medium | VexFlow dynamic import, MusicXML / MIDI generator |
|
||
| | **Wait-for-Note Practice (Web MIDI)**| Medium | Web MIDI API (`navigator.requestMIDIAccess`), practice queue |
|
||
| **Phase 4: Advanced Engine** | **Audio Stem Separation (Karaoke)** | High | ONNX Runtime Web / server Demucs |
|
||
| | **Pitch Shift & Tempo Worklet** | High | AudioWorklet / `SoundTouchJS` DSP |
|
||
| | **Document Picture-in-Picture** | Low-Med | Chrome Document PiP API |
|
||
| | **Collaborative Setlists** | High | WebSocket hub in `server/party.js` |
|