AR Glasses — the hardware layer under the Living Voice

Status: strategy / not built. Authored 2026-07-08. This is the hardware-adoption read and the integration map for smart glasses (audio-first AI glasses now; display + spatial later). It is the hardware layer beneath LIVING_VOICE_ROADMAP.md — most of our ecosystem is voice- and presence-first, which is exactly the modality glasses are strong at before their displays mature. Nothing here changes the priority order: Workstream A (writing experience) and Odessa stay top of the stack. This is a captured direction, not a pull-forward.

The thesis in one line

Our products don't need a display — they need a voice in the ear and a camera on the world. That is precisely what Wave-1 glasses (Ray-Ban Meta class) ship today. So "AR integration" for the next ~18 months is not an app that lives on the glasses — it's our existing apps gaining a glasses-shaped capture/playback mode, delivered through the companion phone + Bluetooth audio + camera export.

Market adoption — three waves, not one date

Plan against waves. The mistake is waiting for the Wave-3 optics fantasy; the opportunity is that our voice layer is already the native content type for the Wave-1 hardware selling right now.

Planning read: build audio-first for Wave 1 now, keep a Wave-2 glanceable-card adapter in mind, treat Wave 3 as the direction Track B is already walking toward. Mass-adoption (smartphone-replacement scale) is a late-decade event; meaningful early-adopter reach on display glasses is ~2027; audio glasses reach is real today.

Platform-risk caveat (read before scoping any code)

Wave-1 glasses are Meta-controlled with no open third-party on-glasses app layer. For roughly the next 18 months the realistic surface is companion-phone app + Bluetooth audio out + the glasses' camera-export/share sheet, not a native app running on the glasses. Android XR (2026+) is the first likely open third-party SDK target. So every integration below is scoped as "our existing surface gains a hands-free/audio-first/capture mode," which is buildable now, rather than "an app on the glasses," which is not.

The 10 integrations, mapped to what we already have

Ranked by how soon it's buildable × how much it leans on shipped infra. Each names the spine it reuses.

The Wave-1 wedge (audio-first, buildable now — start here)

1. Lovio presence, in-ear. The enduring-presence avatar (Track B) doesn't need a face to start — it needs a voice with memory. A loved one's cloned voice answering "tell me about grandpa's boat" through glasses is the most emotionally powerful display-optional thing we own. Reuses: sayInUserVoice / ttsSegment, lovio_user_voices, the JQ Bridge consent graph (who may hear this voice). Read-only bright line applies (Living Voice cross-cutting rule 2).

2. "Hear about this," pointed at the real world. Surface narrations render a spoken intro per web surface today. On glasses the "surface" becomes what the camera sees — a place, object, or photo → house/user voice narrates its story or context. Reuses: surface_narrations, services/narration.ts, the render-once/serve-to-all content_hash cache. Gate: camera vision needs Wave-1 camera-export or Wave-2 SDK — audio path is buildable now against a manual/text trigger.

3. JQ forward-voice nudges as ambient coaching. Your own future/goal voice whispering the right nudge at the right moment is the archetypal glasses use case (ambient, hands-free, contextual). Reuses: POST /api/jq/voice/say, the DreamPro nudge cadence engine, the calm-not-nagging guardrail. Glasses just remove the phone from the loop.

Wave-1 → Wave-2 capture & playback extensions

4. Odessa reframe on lived experience. Odessa's fiction→real-change loop runs on written text today; a camera lets it capture lived moments as raw material for the reframe. Guardrail (load-bearing): honor the Odessa north star — capture-for-the-writing, never turn the art into live coaching. Scope glasses as a capture surface feeding the existing loop, not a new advice channel.

5. Hands-free journaling capture (HiveJournal core). Voice-dictated entries with optional first-person photo/context tags. Protects Workstream A's "build off the existing surface" rule — glasses feed the writing surface, never replace it. Reuses: journal entry create + voice_audio_url render cache.

6. Lovio capsule recording in the moment. Capsules are authored at a desk today; let someone seal a capsule to a future date from the moment itself ("record this for her 18th birthday") via first-person camera + voice. Reuses: the shipped create→seal→deliver→unlock capsule pipeline. Strong fit for Lighthouse parent-reconnection too.

7. Legacy Channel / eldercare, hands-free. Living Voice Track C's nursing-home GTM has an accessibility fit: residents who can't manage a phone UI can talk to a familiar cloned voice through glasses. Audio-first + zero-UI is a genuine accessibility advantage. Reuses: Track C capture UX + delayed vault; inherits the capacity/guardianship consent guardrails verbatim.

8. Emberkiln reel / scene capture. First-person footage as raw material for the story→script→movie pipeline ("Scene Studio first"). Glasses are a natural B-roll / reference-capture rig. Reuses: the render-kit IR, Scene Studio breakdowns.

Wave-2 glanceable / utility

9. Live audiobook / narration companion. Listen to an Emberkiln audiobook through the glasses' speakers, with the current line optionally captioned on a HUD and "who is this character again?" answered by voice. Reuses: the narration + Production IR layers; the HUD caption is the first Wave-2 glanceable-card adapter.

10. Citizen Science / Open Energy field capture. Hands-free data + photo logging for experiments in the field, city-centroid privacy preserved (Citizen Scientist Map floor). Lowest emotional stakes, cleanest utility fit — validates the capture pipeline at low risk.

Where we actually start

The wedge is #1 + #2 + #3 — all three are (a) audio-first so they run on 2025 hardware, (b) built almost entirely on the shipped Living Voice layer, and (c) reuse the render-once/quota-gated cost guardrail already committed to. The rest are Wave-2 extensions of the same voice + camera spine.

Buildable-now first code slice (audio-first, no glasses SDK required): a hands-free / audio-first playback mode for the existing voice surfaces — a lean, large-target, voice/Bluetooth-friendly player + a "play through glasses" affordance that is really just "route cached MP3 to the default Bluetooth audio out, no autoplay." This is the honest Wave-1 surface: it works the day a user pairs Ray-Ban Metas, needs no Meta SDK, and every one of the 10 integrations plays through it. Everything camera/vision-triggered waits on Wave-2 Android XR SDK access and is scoped but not started.

Status — first slice shipped (2026-07-08, PR #1093)

The audio-first foundation is built and three integrations' playback paths already route through it. The hook is useAudioFirstPlayer (lib/useAudioFirstPlayer.ts); the hardware hook is applyMediaSession (lib/media-session.ts) — binding any <audio> to the OS/glasses now-playing card + hardware play/pause/seek/skip, no Meta SDK. Shared UI: components/audio/AudioFirstPlayer.tsx.

• #2 "Hear about this" — HearAboutThis.tsx routed onto the hook (proved it load-bearing; lazy first-tap synthesis = the hook's resolve).
• #3 JQ ambient nudges — JqNudgeCard.tsx routed on; the nudge line shows on the now-playing card.
• #1 Lovio in-ear (read-only slice) — PresenceViewer.tsx routed on (covers both the "sit with an entry" and capsule playback surfaces); the loved one's name + photo show on the now-playing card. Read-only playback only — the generative "answer a new question in their voice" remains bright-line gated (cross-cutting rule 2).

Not yet done: camera/vision triggers (Wave-2 SDK), the Wave-2 glanceable-card HUD adapter, and end-to-end browser verification of the audio (needs the backend + a rendered voice). The refactors are behavior-preserving; static gates (lint / typecheck / next build) are green.

Wave-2 revisit trigger — the SDK watcher

There's no hardware today with both an open developer SDK and a real audience, so the Wave-2 glanceable-card adapter is deliberately not started. Rather than remember to check, a daily cron watches for the gate to open: services/sdk-release-watch.ts polls Android XR (Maven), Meta Spatial SDK + Wearables toolkit, Snap Spectacles, and Brilliant Labs Frame, and alerts on any change via /dashboard/admin/sdk-watch, the /errors console, and the /sdk-releases Claude skill (migration 463; cron sdk_release_watch_tick).

The concrete trigger to build Wave-2: android_xr_sdk reports a version leaving alpha/beta (≈ Android XR consumer glasses shipping), or meta_wearables_toolkit changes (≈ Ray-Ban Meta opening a 3rd-party SDK). When the watcher fires either, revisit this section.

Track B probe — "The Stream," in WebXR (prototype, 2026-07-08)

A first taste of the Wave-3 spatial north star, built the cheap/correct way — WebXR, not a native Quest app. You stand at the water's edge of your journal stream under a starry sky and your notes drift past on the current; catch one (gaze/point + trigger on a headset, click on desktop) and it lifts out of the water and opens in full space, reading itself aloud in voice when a render exists; release it and it casts back down into the current. Route: /dashboard/stream/immersive; scene: components/stream-vr/StreamXRScene.tsx. Runs in the Quest 3 browser (tap "Enter VR") and previews on desktop. Cards are typed by entry data — text notes, voice (waveform + in-ear playback), image cards (an entry's own attached photo — signed thumbnail from journal_entry_attachments — comic art, or notebook cover), encouragement "drops," and glowing mood orbs — over a fog-immune procedural-star sky dome. The current is curated, not just recent: GET /api/journal/stream-curated blends on-this-day (memories from this date in past years), pinned, "in my stream" (voice_stream_enabled), recent, and a daily-rotating resurfaced sample — interleaved so the river carries a spread of a life. Each card whispers why it surfaced ("3 years ago today"). A "current" selector (For you / On this day / Recent / Resurfaced — a mode param on the endpoint) lets you choose which water to stand in. An opt-in ambient river bed (lib/useAmbientRiver.ts) is synthesized live with the Web Audio API — no audio asset — so tapping the speaker adds a calm water + night-air layer (opt-in-to-play, never autoplay). A vantage picker lets you sit in a canoe / kayak / rowboat / paddleboard (procedural, asset-free hulls that gently bob) or stand on a dock off a sandy beach (anchored, static) as you look out. Warm fireflies drift over the water (drei Sparkles), a moon (disc + soft halo) is the scene's key light and lays a shimmering moonglade down the river (a band in the water shader, broken by the ripples), and a weather picker (clear / mist / rain — rain adds a falling point-cloud + expanding ripple rings on the water, mist tightens the fog + adds drifting haze). The chosen current + vessel + weather persist in localStorage so the scene remembers your setup between visits.

Non-journal stream types: the immersive current also floats the same surfaces the 2D stream mixes in (lib/streamExtras.ts, best-effort/allSettled): Signal Union songs (song, embed-only → "Open ↗"), Veneer issues (veneer) + tracks (track, the only one with a real mp3_url → in-ear playback), Graphene season posters (graphene), Looking Glass anniversaries (glass, quote + "N years ago today"), Café books (book), FB reels (reel). Rendered as image cards where art exists, text-media cards otherwise; each opens with an "Open ↗" out-link. Hidden while visiting a neighbor.

Social layer (first slice): hillsides flank the river with glowing dome houses — one per person in your world, from GET /api/journal/stream-neighbors (you, family_members, a few active ai_personas). Tap a house to see "about them" (name, relationship, bio). "Visit their stream" is live via GET /api/journal/stream-visit?userId=X — the target's entries filtered through filterEntryForViewer for the caller, so you only ever see what they've shared with you (their private notes stay hidden). AI companions (public notebooks) are visitable now; the same route serves humans once a per-user relationship + share settings gate it. A "You're in X's stream" banner + "Return to yours" swaps the water back. Components: components/stream-vr/Neighborhood.tsx. North star: a house lights when someone shares something new, and eventually their Lovio voice greets you at the door (read-only bright line intact); houses appear only where the relationship + sharing settings permit (rides the existing consent graph). Frontend falls back to the plain recency feed if the endpoint isn't live.

Why this validates the Track-B thesis: it reuses the shipped journal API (GET /api/journal) + the voice-render cache + applyMediaSession (the same glasses/OS now-playing hook), needs no app-store review, and the exact same scene forward-ports to Wave-3 binocular AR. A native Unity Quest app would forward-port to nothing. Stack: three + @react-three/fiber v9 + drei v10 + @react-three/xr (v6). Comfort rule honored: the user never moves — the current flows toward a fixed camera (zero locomotion = zero nausea). Open design questions before this is more than a toy: stream curation (which notes ride the current — "on this day" / resurfaced / Odessa candidates, not all N), and honoring the read-only-for-the-dead bright line on any Lovio-voiced note.

Landmine for the next 3D feature (learned the hard way, #1102 → #1103): Next 15's App Router runtime is React 19 (vendored). @react-three/fiber must be v9+ (React-19 reconciler) — v8 builds clean but crashes in-browser with Cannot read properties of undefined (reading 'ReactCurrentBatchConfig'). fiber v9 installs nested in the frontend workspace, so a next.config.js webpack alias pins @react-three/fiber to the one copy (single instance for r3f context). The @iwer/devui XR emulator is stubbed out (src/stubs/iwer-devui.js) — undeclared zustand import breaks the build. Load the scene only via dynamic(..., { ssr: false }) and keep all @react-three/* imports out of the page module (else the reconciler chain hits the server prerender).

The dome-home — your spatial dashboard (first slice, 2026-07-09)

The vision (user's): the inside of a geodesic dome as a Quest-Home-style spatial dashboard, where the décor reflects goal adherence — keep your DreamPro goals and the room fills in and warms; slip and it dims. Outside = the stream (reflection); inside the dome = home base. The commissioned dome model is now in — public/models/dome.glb, the full SketchUp interior exported to glTF and compressed 17 MB → 3 MB with @gltf-transform (Draco geometry + WebP textures; loaded via useGLTF(url, true), normalized SketchUp Z-up→Y-up + scaled to fit at runtime). It's the environment; the goal-reactive layer rides on top. The dome sits in a world (components/dome/DomeExterior.tsx): a starry night skyscape, dark hillsides with a few other domes glowing out there, and water on one side — the same world as the stream, seen through the shell (visibility depends on the model's panel transparency).

First slice (built): /dashboard/dome + components/dome/DomeScene.tsx — the real dome.glb interior as the environment, with a goal hologram floating over it (one glowing tile per DreamPro dream, brightening with its progress_percentage; color from dream_categories.color) and the whole space's warmth/lighting lifting with overall adherence — all bound to real data: GET /api/dreampro/dreams (tiles) + GET /api/coaching/me momentum.current_streak (blended 50/50 with average goal progress). Reachable from the immersive stream ("◇ Enter your dome") and by tapping your own house on the hill. North star: the dome interior becomes the "visit inside someone's space," and eventually renders on AR glasses.

Ethics & cost inheritance (non-negotiable)

This doc inherits, not re-litigates, the Living Voice load-bearing wall:

• Read-only vs. generative bright line for any enduring/deceased voice (rule 2).
• Consent v2 scope + the JQ Bridge permission graph for "who may trigger/hear this voice."
• Cost guardrails: cache rendered audio, keep nudges short, rate-limit, never force re-render, reuse tts-spend.ts on every synthesis path.
• Calm, not noisy: opt-in-to-play, never autoplay, never a nag — the same discipline as the JQ orb.
• Privacy floors of each source product carry over (e.g. Citizen Science city-centroid only).

Sequencing

1. Wave-1 audio playback mode (the buildable-now slice above) — the surface every integration plays through.
2. #1 / #2 / #3 audio paths onto that surface, reusing the Living Voice primitives.
3. Wave-2 glanceable-card adapter + camera-triggered "Hear about this" when Android XR SDK access lands (~2026+).
4. Wave-3 spatial — already Living Voice Track B (WebXR + glTF); funded/motivated by whatever traction Waves 1–2 prove.

See also

• OWNED_HARDWARE_HORIZON.md — the "walls removed" scenario for this doc: what unlocks if we own the glasses (open-source Frame-class now → our own hardware later) instead of renting Meta/Apple's — and the surveillance-question ethics spec that owning the hardware forces on us.
• MOBILE_JQ_PRESENCE.md — the phone sibling of this doc: how much always-on JQ presence iOS/Android actually permit (the same audio-first player + voice layer, different hardware).
• LIVING_VOICE_ROADMAP.md — the voice/presence spine this is the hardware layer for (Track B = the spatial north star).
• LIGHTHOUSE.md — safeguarding/gating model the enduring + capsule integrations inherit.
• ODESSA_NORTH_STAR.md — the art-first bound on integration #4.
• SCENE_STUDIO_ARCHITECTURE.md / EMBERKILN_PIPELINE_ARCHITECTURE.md — the capture targets for #8/#9.