Production IR — design proposal (for sign-off)

Status: first slice shipped (#829). Choices below are decided (see §8). This is the design for Step 3 of the unified pipeline. Steps 1–2 (the render kit: providers, ffmpeg primitives, composeStillVideo, loudnorm recipe) shipped in #820–#827. Step 3 introduces the Production IR — the shared intermediate contract between content and renderers — and the source→Work router that feeds it.

Shipped so far:

• render-kit/ir/ — the types (ProductionIR / Work meta / Section / Segment / CastMember), the pure assembleSeasonIR + assembleEntryIR assemblers, and buildProductionIR (the season + journal_entry loaders). (#829, #831)
• Print — the first renderer as an IR projection: fetchSeasonForPrint reads the Work, projectSeasonIRToPrint reproduces the old fetch byte-for-byte. (#829)
• Reel (source + render-half) — the journal_entry Work source; draftReelScript reads the entry via the IR and builds its prompt from the Work (byte-identical, golden-tested). (#831) The render-half now also goes through the IR: reelScriptToSegments (render-kit/ir/reel.ts) projects the drafted script onto 2 dialogue segments + a reel cast, and renderReelAudio resolves each segment's voice from the cast by speaker key and TTS-renders the segments in order — byte-equivalent by construction (same lines/voices/order), golden-tested. Captions/timings stay reel-specific (the logged impedance).
• Screenplay — screenplay-ir.ts: screenplayProjectToIR (scenes → Sections)
  • assembleFountainFromIR; assembleFountain now assembles the Fountain doc through the IR, byte-for-byte (golden-tested). The screenplay source is a season's scene re-segmentation, built from the screenplay project's own loader.

All behavior-preserving, no works table.

The headline decision up front, because it shapes everything: the IR is a derived, in-memory contract — NOT a new database table. story_seasons is already the de-facto Work hub. We build the IR from the existing tables and have renderers consume it; we do not migrate data into a works table. That keeps this a behavior-preserving refactor in the same spirit as Steps 1–2, not a risky data migration. (Why, and when that might change, in §6.)

1. Goal & non-goals

Goal. One canonical, segmented representation of a creative work that every renderer (audiobook, scene/video, screenplay, print, reel) reads from — so "one story, every form" becomes a real engine instead of five surfaces that each re-query the database and re-derive structure. The dogfood target: "Al, a trope" is one Work → the scene recipe makes the video, the audiobook recipe makes the VO, the reel recipe makes the vertical cut — all from the same source.

Non-goals (explicit, to bound scope):

• Not a works table or a data migration. The IR is assembled on demand.
• Not a mega-orchestrator. Renderers stay separate; they just take the IR as input instead of raw table rows. (Per the architecture doc: "the win is shared stages + a shared intermediate, not a shared orchestrator.")
• Not a forced lowest-common-denominator. Surfaces keep their rich per-surface fields via typed extension bags (§4) — the IR core carries only the spine that's genuinely shared.
• Not a rewrite of the ingest routes. The router wraps the existing ingest paths; it doesn't replace manuscript.ts, notebooks.ts, etc.

2. Ground truth — the shapes already converge

This is the evidence the IR is real and not invented. Each surface's current intermediate (verbatim field names from the code/migrations):

The shared spine, read across the rows above: an ordered list of segments, each carrying text, an optional speaker/voice, an optional visual intent, and an optional duration. Everything else is a per-surface elaboration of that spine. That's the IR.

Two more facts that make this cheap:

1. story_seasons is already the hub. Audiobook, print, screenplay, and scene studio all root on story_seasons → story_episodes (chapter_prose). Only the reel roots elsewhere (journal_entries). So 4 of 5 surfaces already share a source-of-truth; the IR formalizes what's already true.
2. The cast/voice model is already shared. season_character_voices (character_key, character_name, voice_id, voice_direction) is consumed by both audiobook and scene studio. The IR's cast block is a rename, not an invention.

3. The three-level shape

The IR is a small tree: Work → Section → Segment, plus a flat Cast.

Work                      (≈ story_seasons, or a journal_entry for a reel)
├── meta: title, author, premise/logline, genre, source lineage
├── cast: CastMember[]    (≈ season_character_voices, shared today)
└── sections: Section[]   (≈ story_episodes / chapters / scenes)
    └── segments: Segment[]   (≈ scene_shots / audio_segments / fountain elements)

• Work — the whole piece. Maps to a story_seasons row (narrative) or wraps a single journal_entry (reel). Carries title/author/premise + source lineage.
• Section — an ordered chapter/scene-group. Maps to story_episodes (episode_number, title, chapter_prose, dramatic_role). A reel has one synthetic section; a screenplay scene is a section.
• Segment — the atomic renderable unit. The shared spine: text + optional speaker + optional visual + optional duration. Maps to a scene_shot, an season_audio_segment/VoiceSpan, a FountainElement, or a reel line.
• CastMember — a named voice/character: key, name, description, voice_id, voice_direction, reference_image_url.

4. Proposed types (the contract)

Strawman TypeScript — the thing to critique. Core fields are the shared spine; per-surface richness lives in typed extension bags (render_hints) so we never force camera_motion or adaptation_decision into the core.

// render-kit/ir/types.ts  (proposed)

export interface ProductionIR {
  /** Stable id of the underlying source (season id, or `entry:<id>` for reels). */
  work_id: string
  source: WorkSource              // lineage — how this Work was ingested (§5)
  meta: WorkMeta
  cast: CastMember[]
  sections: Section[]
}

export interface WorkMeta {
  title: string
  author: string | null
  /** premise (novel) or logline (screenplay) — same slot, surface picks the label. */
  premise: string | null
  genre: string | null            // 'drama' | 'mystery' | … (story_seasons.genre)
  poster_url?: string | null
}

export interface CastMember {
  key: string                     // season_character_voices.character_key; 'narrator' reserved
  name: string                    // display name
  description?: string | null     // scene_projects.characters[].description
  voice_id?: string | null        // ElevenLabs voice id
  voice_direction?: string | null // 'default' | 'whispered' | 'emotional' | …
  reference_image_url?: string | null
}

export interface Section {
  idx: number                     // order; episode_number for chapters
  title: string | null
  /** opening | inciting_incident | rising | midpoint | crisis | climax | resolution */
  role?: string | null            // story_episodes.dramatic_role / scene beat_role
  /** Raw prose for this section, when it exists pre-segmentation (chapter_prose). */
  prose?: string | null
  segments: Segment[]
}

export type SegmentKind = 'narration' | 'dialogue' | 'action' | 'heading' | 'transition'

export interface Segment {
  idx: number
  kind: SegmentKind
  /** The words. Narration prose, a spoken line, an action line, a slug line. */
  text: string
  /** Cast key of who speaks/acts. null = narrator/none. */
  speaker?: string | null
  /** Image-generation intent for the visual surfaces. */
  visual_prompt?: string | null
  /** Intended on-screen/spoken seconds, when known (scene_shots.duration_s). */
  duration_s?: number | null
  /** Per-surface elaboration that doesn't belong in the shared spine. */
  render_hints?: RenderHints
}

/** Typed, optional, surface-scoped. A renderer reads only its own bag. */
export interface RenderHints {
  scene?: {                       // scene_shots extras
    shot_type?: string
    transition?: 'cut' | 'fade' | 'dissolve'
    camera_motion?: string
    trim_start_s?: number
    trim_end_s?: number
  }
  audio?: {                       // season_audio_segments extras
    mood?: string | null
    intensity?: number | null
    pace?: 'slow' | 'medium' | 'fast' | null
    silence_after_ms?: number
  }
  screenplay?: {                  // FountainElement / ScreenplayScene extras
    fountain_type?: 'Scene Heading' | 'Action' | 'Character' | 'Parenthetical' | 'Dialogue' | 'Transition'
    int_ext?: string | null
    location?: string | null
    time_of_day?: string | null
  }
  reel?: { caption?: string }     // caption_lines mapping
}

Why extension bags and not a fat flat segment? Because the surfaces diverge hard below the spine (lip-sync versions, pulse layouts, fidelity rubrics). A flat segment with every field becomes a lie — most fields null for most surfaces. A typed bag keeps the core honest and lets a renderer pattern-match only what it owns. This is the same "recipes stay thin" discipline as the render kit.

5. The source→Work router

One function, buildProductionIR(source): Promise<ProductionIR>, that wraps the existing ingest reality (it does not replace it). The sources we have today:

export type WorkSource =
  | { kind: 'season'; season_id: string }        // story_seasons → 4 surfaces
  | { kind: 'journal_entry'; entry_id: string }  // journal_entries → reel
  // future: { kind: 'manuscript'; upload_id } / { kind: 'paste'; text } resolve
  // to a season first via the existing manuscript.ts ingest, then load as 'season'.

• season — load story_seasons + story_episodes (+ season_character_voices
  • scene_projects/scene_shots when present) → assemble Work/Sections/Segments. This is a read model: a set of SELECTs + a pure assembler. No writes.
• journal_entry — wrap one entry as a one-section Work (the reel case). The reel's two-voice script becomes 2 dialogue segments (speaker: 'user' / speaker: 'odessa') under a synthetic cast.
• manuscript / paste — already resolve to a story_seasons row via manuscript.ts (create-season, port-to-graphene). So they're not new IR sources; they're existing ingest that produces a season, which then loads as kind: 'season'. The router doesn't duplicate that logic.

Renderers become projections of the IR (the payoff):

Note every renderer's existing output stages are unchanged — they already exist and are partly shared via the render kit. The IR only changes their input: from "query my own tables" to "read the Work."

6. Why a derived contract, not a table (and when that flips)

Now: derived. Building the IR in memory from existing tables means:

• Zero migration risk; nothing to backfill; no dual-write window.
• Each surface can adopt the IR independently (swap its input loader) and we can diff old-vs-new output per surface — same safety model as Steps 1–2.
• If the IR shape is wrong, we change a TypeScript type, not a live schema.

Later (maybe): persist. A works/segments table earns its place only when we need something a derived view can't give cheaply — e.g. editing the IR directly (a unified editor that isn't "edit the season then re-derive"), cross-surface caching of an expensive segmentation, or provenance (one immutable record of "this Work produced these N artifacts"). None of those are needed for the dogfood. We design the types now so a future table is a serialization of the same shape — but we don't build it until a feature demands it.

7. Adoption path (incremental, lowest-risk first)

1. Land the types + the season loader (render-kit/ir/), with a golden test that asserts a known season assembles to an expected IR. No renderer changes yet — pure addition.
2. Re-express ONE renderer as an IR projection — print first. Print is the safest: deterministic, already a clean PrintChapter[] shape, output is a PDF we can byte-compare. generateSeasonPrintAssets reads the IR instead of fetchSeasonForPrint. Prove identical output.
3. Then reel — wrap the journal-entry source, render via composeStillVideo. This is the dogfood-relevant one ("Al, a trope" as a Work).
4. Then audiobook / scene / screenplay, one at a time, each diffed against current output before merge.
5. Revisit persistence (§6) only if an editor/caching/provenance feature lands.

Each step is one PR, behavior-preserving, golden-tested where deterministic — the exact cadence of #820–#827.

8. Decisions (resolved at sign-off)

1. Segment granularity for prose — BOTH. The IR carries Section.prose (lazy) AND Section.segments (eager); a Section may be "unsegmented" (prose set, segments empty) and that's valid. Lazy renderers (audiobook/print) read prose; eager ones (scene/screenplay) read segments. ✅ shipped in the types.
2. Reel as a Work — YES, but adopt it second. Print proves the contract first (deterministic, diffable); the reel (journal_entry source → synthetic 2-segment Work) lands next as the "Al, a trope" dogfood.
3. Loader is pure read — YES. The loader reflects what exists and never triggers generation; renderers generate what's missing (e.g. scene breakdown).
4. Naming — ProductionIR / Work meta / Section / Segment / CastMember. Section is the neutral superset of chapter/scene. ✅
5. Home — render-kit/ir/. Same consolidation effort as the rest of the kit. ✅

9. Next steps (durable — survives session switch)

State as of #832. Prod validation of what's merged is pending — see EMBERKILN_PRODUCTION_IR_TEST_GUIDE.md (the loaders run prod queries with code-inferred columns; MCP ≠ prod, so they've never hit the real schema). Run that first.

Done (merged, behavior-preserving, golden-tested)

• IR types + season/journal_entry loaders (#829, #831)
• Print — full source→artifact projection (#829)
• Reel — source side — journal_entry Work + draft prompt via IR (#831)
• Screenplay — Fountain assembly via IR (#832)
• Reel — render-half — reelScriptToSegments (render-kit/ir/reel.ts) projects the drafted script onto 2 dialogue segments + a reel cast; renderReelAudio resolves each segment's voice from the cast by speaker key and TTS-renders the segments in order. Byte-equivalent by construction, golden-tested. The first behavioral surface through the IR — pending the rendered-reel eyeball (the "Al, a trope" dogfood). Captions/timings stay reel-specific.
• Voice resolution (forward) — resolveVoice(segment, cast, fallback) (render-kit/ir/voice.ts): cast key → concrete { voice_id, voice_direction }, direct-match → narrator → fallback → throw. Golden-tested; the reel uses it.
• Audiobook IR assembler (reverse mapping) — audiobookScriptToIR (render-kit/ir/audiobook.ts): season_audio_segments rows → eager Segments with speaker reverse-mapped from each row's voice_id (narrator owns ties; orphan/null → speaker: null), segment_type + per-row voice_direction + episode_number preserved in render_hints.audio. PURE + additive; golden- tested incl. the resolveVoice round-trip. New audiobook WorkSource kind. buildCast extracted + shared with the season loader.
• Audiobook render — journal-mode — buildProductionIR({kind:'audiobook'}) (build.ts buildAudiobookIR) loads the season + story_cast personas + season_character_voices + the audio rows, and builds a cast that covers every row voice (coverRowVoices: narrator + personas + character voices + a synthetic member per leftover voice → zero orphans). renderSeasonAudio resolves each segment's voice through that IR (== the row's own voice by construction → byte-equivalent), with a raw-voice fallback if the IR build fails. The journal-mode trap this solves: journal voices come from story_cast, NOT season_character_voices — a cast missing them would reverse-map persona rows to the narrator and mis-voice every line. Cache / row id / pulse_layout / voice_direction + dialogue span-splitting stay render-side. Pending a rendered-audiobook eyeball.
• Audiobook render — novel-mode — renderChapterAudio (season-novel-chapter-audio.ts) adopts the same IR voice resolution as journal-mode (same buildAudiobookIR + coverRowVoices + raw fallback). Novel-mode dialogue voices come from season_character_voices (which the cast already covered), so it's the cleaner fit. Byte-equivalent; both render paths now route voices through the IR. Pending the same rendered-chapter eyeball.
• Scene Studio assembler (eager segments) — sceneProjectToIR (render-kit/ir/scene.ts): scene_shots → eager Segments with visual_prompt
  • duration_s as core fields, dialogue → text, speaker normalized to a cast key, and shot_type / transition / camera_motion / trim / place_key in render_hints.scene (the first real use of that bag). Characters → visual-only cast (description + reference image, no voice). PURE + additive (no render touched); golden-tested. New scene WorkSource kind (place_key added to RenderHints.scene). The render adoption (scene-studio.ts reads the IR) is the follow-up.

Remaining — and why they're NOT done yet

The clean print/screenplay pattern is exhausted; the reel render-half, the audiobook assembler, AND both audiobook render paths (journal + novel-chapter) are in — every audio renderer now routes voices through the IR. What's left:

1. Scene Studio render adoption — the pure sceneProjectToIR assembler is ✅ shipped (eager segments + render_hints.scene). What remains is pointing the scene render (scene-studio.ts keyframe/clip/animatic pipeline) at the IR instead of raw scene_shots — behavioral, needs a rendered-scene eyeball. Render-side concerns (keyframe_url/clip_url/audio_url caches, *_versions, the per-shot generation) stay out of the IR, same discipline as the audiobook.
2. intent_pulse pulse_layout — the multi-voice pulse blob is still NOT modeled in the IR (only segment_type: 'intent_pulse' is preserved; the render reads pulse_layout from the raw row and renders it unchanged via the IR-skipping branch). Model it (a hint, or a typed pulse extension) only when a renderer needs the pulse to flow through the IR.

Render-side concerns that intentionally stay OUT of the IR (not "remaining work" — settled): render-time dialogue span-splitting (splitProseByVoice re-splits prose into TTS spans by detecting attribution at render time — the IR resolves the segment's base voice, the render span-splits as before); cache state (audio_url/duration_seconds) + music/announcement config; and the fact that there's no deterministic source→artifact seam for audio (TTS is non-deterministic) — so audio adoptions are byte-equivalent by construction (same inputs to the same TTS calls), not byte-comparable artifacts.

Suggested order for a future session

1. Run the prod test guide; fix any loader column mismatches. (Still pending — the loaders have not yet hit the prod schema; independent of the reel render-half, which works on the in-memory drafted script.)
2. Reel render-half — ✅ shipped (render-kit/ir/reel.ts + renderReelAudio); awaiting the rendered-reel eyeball before this is considered fully validated.
3. Audiobook: ✅ fully shipped — forward resolution (resolveVoice), reverse- mapping assembler (audiobookScriptToIR), AND both render paths (renderSeasonAudio journal-mode + renderChapterAudio novel-mode) route voices through the IR. Pending only the rendered-audiobook eyeball.
4. Scene Studio eager segments (render_hints.scene) — the next target.
5. intent_pulse pulse_layout modeling — when a renderer needs it.

Net

The IR's hypothesis is validated: one Work → Section → Segment contract held across a PDF book, a reel draft, a Fountain screenplay, the reel's two-voice audio render, AND both audiobook render paths (journal + novel). Every renderer now reads voices/text/order through the shared contract. The remaining surfaces (Scene Studio eager segments, intent_pulse modeling) are incremental, not foundational — the shape is proven across all five forms.