← product

Downstream Soundtracks — handoff (start here to build v0)

✅ v0 BUILT 2026-07-15 — migration 489, PR (this session). What landed: the downstream_soundtracks table; the pure core (moodForPhase/pickForMood, downstream-soundtracks-core.ts, 10/10 tested); the service (getSoundtrackForPhase reuses importFromUrl); /api/downstream/soundtracks/*; the glasses looper (apps/glasses/src/soundtrack.ts) that loops the bed and ducks under every nudge (duckFor wired into speakNudge), with nextReplayDelayMs making it correct whether playAudio resolves at end or immediately; voice control ("play/stop music"); and a dashboard "🎧 Soundtracks" panel (import + mood + preview). The one open call is still the on-device MIX-vs-INTERRUPT test below — the code handles both. Remaining after that: v1 mood playlists, v2 shared/synced group beds. The plan below is preserved as the build record.

Written 2026-07-15. Everything a fresh session needs to build the in-ear soundtrack layer (task 1180ea1a, DOWNSTREAM_AUDIO_IDEAS.md §1). The audio-source decision is made and — the happy surprise — the hard part (Suno ingestion → durable storage) is already built for the EmberKiln Studio. This doc is the reuse map + the one real unknown to verify on-device.

What it is (one paragraph)

Stream a soundtrack — a focus/mood bed, or a shared track piped to a group — through the Downstream Glasses speaker, ducking automatically under JQ so the music never talks over your own voice. It rides the same session.audio.playAudio HTTPS-MP3 path the forward-voice nudge already uses, so the transport is free; the new work is continuity (looping / next-track) and mixing (duck on nudge).

Decision: audio source = Suno, via the EXISTING ingestion

Curate tracks in Suno, ingest into our system. Do not build a new ingester — reuse the studio one, which already solves the durability gotcha (Suno rotates its CDN URLs, so we re-host).

Reusable service — apps/backend/src/services/creator-audio-library.ts:

  • importFromUrl({...}) — paste a Suno share page (suno.com/song/<id>) or a direct cdn1.suno.ai/*.mp3; it scrapes the embedded CDN URL and re-uploads to our storage so the asset survives Suno's URL rotation. This is the whole ballgame.
  • storeUpload({...}) — multipart MP3 upload (drag-drop from disk).
  • listLibraryForUser / archiveLibraryTrack / renameLibraryTrack / setLibraryTrackTags.
  • Table creator_audio_library (migrations 298 + 300-tags); 25 MB cap.
  • Route shape to copy: apps/backend/src/routes/studio.ts /api/studio/audio-library/* (list / upload / import / rename / archive).
  • Frontend reference UI: apps/frontend/src/app/studio/audio-library/page.tsx.

Licensing box to tick before anything public: Suno's commercial-use rights depend on the plan (paid tiers grant them; free doesn't). Fine for personal/dev now.

(The Graphene background music the user liked is the same lineage — proven aesthetic.)

Proposed v0 build

  1. Storage — reuse importFromUrl / storeUpload for the heavy lifting. For the soundtrack references, EITHER tag creator_audio_library rows (downstream_soundtrack) OR add a thin downstream_soundtracks table. Recommend the small table — the §3 ultradian tie-in wants Downstream-specific metadata (mood, maybe a phase affinity) that doesn't belong on the studio library. Suggested cols: id, user_id (nullable → curated/global), audio_url (the re-hosted MP3), title, mood (focus/calm/energize/ambient), loopable bool, source_url.
  2. Backend — /api/downstream/soundtracks (list + import-via-importFromUrl + upload). A getSoundtrackForPhase(userId) that picks a track by mood matching the current rhythm phase (call getRhythm — peak→energize, trough→calm) is the bridge to §3's "adaptive audio."
  3. Glasses player — a loop in apps/glasses/src/session.ts: fetch a track URL, session.audio.playAudio({ audioUrl, volume }) at low volume, replay on completion. Duck on speakNudge: drop soundtrack volume (or pause) → play the nudge → restore.
  4. App control — a toggle + track/mood picker on /dashboard/downstream.

Phasing: v0 = one looping track ducked under nudges; v1 = mood playlists + voice control + phase-adaptive selection (§3); v2 = shared/synced group soundtrack (room- scoped via the JQ Bridge graph).

The one real unknown — VERIFY ON-DEVICE (Mentra Live)

session.audio.playAudio({ audioUrl, volume }) — note volume already works (apps/glasses/src/watch.ts:99 plays a cue at volume: 0.4). Two behaviors decide the ducking strategy and are NOT known from the code — test on the actual glasses:

  1. Does a second playAudio MIX with the first, or INTERRUPT it?
    • Mixes → duck = keep the soundtrack playing at low volume, play the nudge on top. Ideal.
    • Interrupts → duck = pause/stop the soundtrack, play nudge, then restart the loop (accept a seam). Build the pause/resume path.
  2. Does playAudio resolve when the track ENDS (so a loop = await → replay), or immediately (needs a duration timer to know when to replay)?

Everything else is standard; these two are the only "built blind" bits — mirror how the immersive/inspiration VR rooms handle hosted-MP3 playback if useful, but the glasses audio API is the source of truth.

File map / where things plug in

  • Ingestion (reuse): apps/backend/src/services/creator-audio-library.ts (importFromUrl, storeUpload)
  • Route shape (copy): apps/backend/src/routes/studio.ts → /api/studio/audio-library/*
  • Glasses session + duck point: apps/glasses/src/session.ts (speakNudge, attachJqSession), volume proof at apps/glasses/src/watch.ts:99
  • Nudge audio the soundtrack must duck under: sayInUserVoice → apps/backend/src/services/downstream-nudge.ts
  • Phase for adaptive selection (§3 bridge): getRhythm in apps/backend/src/services/downstream.ts
  • Product spec: DOWNSTREAM_AUDIO_IDEAS.md §1 (+ §3 for the adaptive tie-in)

Landmines / conventions

  • main is protected — squash-merge via PR only; migration-gated PRs need the SQL applied to prod first (see CLAUDE.md).
  • gh pr edit --body is blocked on this repo — set the PR body at create time.
  • Frontend preflight: cd apps/frontend && npm run check before a PR.
  • Backend tests: put the deterministic core (mood→phase mapping, loop/duck state machine) behind a pure function + a jest test, per the downstream-*/levers-core pattern (NODE_OPTIONS=--experimental-vm-modules npx jest <name>).
  • New cost-bearing cron? none needed for v0 (playback is client/glasses-driven).
  • Keep docs in sync — INDEX.md + this doc + DOWNSTREAM_AUDIO_IDEAS.md §1 status.
DOWNSTREAM SOUNDTRACKS HANDOFF — Docs | HiveJournal