ai

HiveJournal Architecture

Tour of the monorepo, the deployment topology, and how data flows between layers. If you've never touched this codebase before, read this once.

Monorepo Layout

hivejournal-2026/
├── apps/
│   ├── frontend/         # Next.js 15 (App Router) — main web app
│   ├── backend/          # Express + TypeScript — API server
│   ├── mobile/           # React Native + Expo — mobile companion
│   └── chrome-extension/ # Browser extension for quick capture
├── packages/
│   └── shared/           # Shared types/utils across apps
├── supabase/
│   └── migrations/       # Numbered SQL migrations (001 → 047 currently)
├── scripts/              # One-off TypeScript scripts (imports, seeds)
└── docs/                 # Setup guides, runbooks, this AI docs subdirectory
    └── ai/               # ← you are here

Tech Stack

LayerTechNotes
Frontend frameworkNext.js 15 (App Router)'use client' is the norm — most pages are client components wrapped in server-component layouts that export metadata
UI stylingTailwind CSS + CSS modulesCSS modules used heavily in /open-energy/* for the night-sky aesthetic; rest of the app is mostly Tailwind
Backend frameworkExpress + TypeScriptCompiled with tsc, run with node dist/index.js
DatabaseSupabase (PostgreSQL)All persistence. Row-Level Security on every user-data table.
AuthSupabase AuthJWT bearer tokens. Backend middleware verifies via supabase.auth.getUser(token)
StorageSupabase StorageBuckets for avatars, entry attachments, tone packs, generated images
AIOpenAI (GPT-4 + DALL-E)Used in chat, note analysis, image generation
EmailResendTransactional + auth emails
PaymentsStripeSubscriptions
MobileReact Native 0.81 + Expo 54Shares packages/shared types
CronVercel CronEndpoints in apps/frontend/src/app/api/cron/, secret-protected
Hosting (frontend)VercelAuto-deploy on push to main
Hosting (backend)RailwayUses nixpacks.toml to override install command (legacy peer deps)

How a request flows

                         ┌────────────────────────────┐
                         │  Browser / Mobile Client   │
                         └────────────────┬───────────┘
                                          │
                            (1) Bearer token from Supabase Auth
                                          │
                                          ▼
                         ┌────────────────────────────┐
                         │  Express backend (Railway) │
                         │  apps/backend              │
                         └────────────────┬───────────┘
                                          │
                            (2) authenticateUser middleware
                                          │
                                          ▼
                         ┌────────────────────────────┐
                         │  Route handler              │
                         │  apps/backend/src/routes/* │
                         └────────────────┬───────────┘
                                          │
                            (3) supabase.from(...) using
                                service-role client
                                          │
                                          ▼
                         ┌────────────────────────────┐
                         │  Supabase Postgres         │
                         │  RLS bypassed at this layer│
                         └────────────────────────────┘

The backend uses the service role key, so it bypasses RLS. Authorization is enforced manually in route handlers via the authenticateUser middleware (which sets req.user) and explicit checks like isSuperAdmin(supabase, req.user.id) for admin endpoints.

The frontend can also call Supabase directly using the anon key (for client-side reads where RLS is sufficient). This pattern is used for the dashboard's real-time entry list, theme settings, etc.

Concrete example — what an entry write looks like end-to-end:

flowchart LR
    A[Composer<br/>/dashboard/entries/new] -->|POST /api/journal| B[apiRequest&lpar;&rpar;<br/>+ bearer token]
    B --> C[authenticateUser<br/>middleware]
    C --> D[routes/journal.ts]
    D -->|service-role insert| E[(journal_entries<br/>RLS bypassed)]
    D -.->|on error| F[recordServiceError&lpar;&rpar;]
    F --> G[(service_errors<br/>→ admin/errors)]

Multi-step pipelines follow the same request-handler shape but chain into backend services that call out to ElevenLabs / OpenAI between writes. The canonical example is the port-to-Graphene pipeline that takes a notebook of journal entries and turns it into a narrated season on the public slate:

flowchart TD
    A[Notebook entries] -->|POST /api/notebooks/:id/<br/>port-to-graphene| B[port route]
    B --> C[(story_seasons)]
    B --> D[(story_episodes<br/>chapter_prose)]
    D --> E[generateNovelScript]
    E --> F[(season_audio_segments)]
    F --> G[renderSeasonAudio<br/>ElevenLabs TTS]
    G --> H[(season_publishings<br/>+ chapter_audio_url)]
    H --> I[PublishStrip UI]
    I --> J[/graphene slate]

Three ways the frontend talks to data

  1. apiRequest() helper (apps/frontend/src/lib/api.ts) — wraps fetch() to call the Express backend with the auth token. Used for most CRUD operations on journal entries, notebooks, goals, etc.

  2. Supabase client direct (apps/frontend/src/lib/supabase.ts) — used in places where RLS is enough, like fetching the user's own entries on the dashboard, theme settings, profile.

  3. Public open endpoints/api/notebooks/:id/public, /api/open-energy/* (most), /api/og/* — no auth required, return only public-flagged data.

  4. Experiment calculatorsPOST /api/dreampro/calculate, GET /api/dreampro/calculators, POST /api/dreampro/optimize — public, no auth, deterministic computation. The calculator engine at apps/backend/src/services/experiment-calculators/ is pure math (no DB reads), so these endpoints are stateless and cacheable. Structured measurement data flows via the auth-gated PATCH /api/dreampro/replications/:id/measurements back into dream_replications.measured_values JSONB, where GET /api/dreampro/templates/:id/measurement-stats aggregates it for the community results view.

There's also an implicit fifth pattern — polling for long-running renders. Audio rendering can take minutes per chapter, so the SeasonClient re-fetches season state on an interval and stops once every chapter has a chapter_audio_url. PublishStrip uses this shape to drive live-progress feedback without holding open a long socket:

sequenceDiagram
    participant U as SeasonClient
    participant API as GET /api/story-seasons/:id
    participant DB as story_episodes
    U->>API: fetch season state
    API->>DB: SELECT episodes[].chapter_audio_url
    DB-->>API: rendered count (N of total)
    API-->>U: { episodes, all_rendered: false }
    Note over U: setTimeout 8s
    U->>API: re-poll
    API-->>U: all_rendered: true → stop

Frontend layout structure

apps/frontend/src/app/
├── layout.tsx                  # Root layout — Navbar wrapper, theme provider, analytics
├── not-found.tsx               # App Router 404 (server component, no context deps)
├── page.tsx                    # Public landing
├── auth/                       # Sign-in, sign-up, password reset flows
├── dashboard/                  # Authenticated routes
│   ├── layout.tsx              # Wraps dashboard pages with DashboardLayout
│   ├── page.tsx                # Main dashboard (entry feed)
│   ├── entries/                # Journal entry CRUD
│   ├── notebooks/              # Notebook CRUD
│   ├── goals/                  # Accountability goals
│   ├── dreampro/               # Dream Pro AI goal breakdown
│   ├── workout-window/         # Activity tracking gamification
│   ├── jq-bridge/              # User-to-user connections
│   ├── stream/                 # Visual feed view
│   ├── settings/               # Profile & preferences
│   ├── subscription/           # Stripe billing
│   └── admin/                  # Super-admin tools
│       ├── page.tsx            # 5300-line single-file admin (legacy)
│       └── open-energy/        # Open Energy admin (clean sub-page pattern)
├── open-energy/                # Public Open Energy section
│   ├── layout.tsx              # OG metadata + PWA manifest reference
│   ├── manifest.ts             # PWA manifest for add-to-home-screen
│   ├── page.tsx                # Landing
│   ├── experiments/            # Constellation pathway
│   └── contributors/           # Leaderboard
├── api/
│   ├── og/                     # OG image generation routes (@vercel/og)
│   └── cron/                   # Vercel cron endpoints
└── pages/                      # Pages Router fallback for _error.tsx only
    └── _error.tsx              # Custom error page (no useContext deps)

Backend layout structure

apps/backend/src/
├── index.ts                    # Express app + Supabase client + route mounting
├── middleware/
│   └── auth.ts                 # authenticateUser — JWT verify + req.user
├── database/
│   ├── visibility.ts           # Visibility filter helpers
│   ├── roles.ts                # isSuperAdmin, isOrgOwner, etc.
│   ├── types.ts                # TypeScript types matching Supabase tables
│   └── linked-accounts-recursive.sql
├── routes/                     # One file per feature area
│   ├── journal.ts              # Entry CRUD
│   ├── notebooks.ts            # Notebook CRUD + public endpoint
│   ├── auth.ts                 # Custom auth helpers
│   ├── user.ts                 # Profile management
│   ├── ai.ts                   # OpenAI proxies (chat, image gen, analysis)
│   ├── chat.ts                 # Chatbot conversation persistence
│   ├── goals.ts                # Accountability goals
│   ├── dreampro.ts             # Dream Pro
│   ├── workout-window.ts       # Activity tracking
│   ├── jq-bridge.ts            # User connections
│   ├── drops.ts                # Currency
│   ├── encouragement-drops.ts  # Themed micro-drops
│   ├── tone-packs.ts           # Tone pack marketplace
│   ├── satisfaction.ts         # Quick satisfaction logging
│   ├── sleep.ts                # Sleep tracking
│   ├── actions.ts              # Action notes
│   ├── health.ts               # Wellness actions + recommendations
│   ├── spotify.ts              # OAuth + sync
│   ├── email.ts                # Resend transactional
│   ├── payment.ts              # Stripe
│   ├── covers.ts               # Notebook covers
│   ├── stream-backgrounds.ts   # Stream backgrounds
│   ├── organizations.ts        # Org CRUD
│   ├── teams.ts                # Team CRUD
│   ├── linked-accounts.ts      # Multi-account aggregation
│   ├── admin.ts                # Super-admin user mgmt
│   ├── admin-import.ts         # Bulk imports
│   ├── admin-conversion.ts     # Currency conversion rates
│   ├── firebase-import.ts      # Legacy Firebase data
│   ├── mongodb-import.ts       # Legacy MongoDB data
│   ├── quotes.ts               # ZenQuotes API proxy
│   ├── health.ts               # Health actions
│   └── open-energy.ts          # Open Energy: progress, replications,
│                               #   builds, contributors, activity, OG, admin seed
├── services/                   # Per-feature business logic
│   ├── workout-window/
│   ├── dreampro/
│   ├── chat/
│   └── encouragement-drops/
└── utils/
    ├── displayName.ts          # Pseudonym generation
    └── tokenTracking.ts        # AI usage tracking

Database conventions

  • Migrations are numbered sequentially (001_*.sql047_*.sql). Always pick the next available number.
  • Every user-data table has:
    • id UUID PRIMARY KEY DEFAULT gen_random_uuid()
    • user_id UUID NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE
    • created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
    • updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
    • A trigger calling set_updated_at() (helper defined in many migrations)
    • RLS enabled with policies that check user_id = auth.uid() (or visibility helpers for sharing)
  • Aggregate stats are exposed via VIEWS granted to anon and authenticated (e.g., open_energy_replication_stats, open_energy_experiment_stats)
  • Seed/test data uses an is_seed BOOLEAN NOT NULL DEFAULT FALSE column so it can be cleanly distinguished and wiped via admin tools

Visibility model

There are two layers of access control:

  1. Database (RLS): enforces ownership and basic role checks
  2. Application (visibility helpers): filters content based on visibility levels (user, team, org, public)

The visibility levels exist on both the entry and its parent notebook. Helpers in apps/backend/src/database/visibility.ts compute effective visibility (entry overrides notebook).

flowchart TD
    A[Read request<br/>for an entry] --> B{entry.metadata_<br/>visibility set?}
    B -->|yes| C[Use entry visibility]
    B -->|no| D{notebook.metadata_<br/>visibility set?}
    D -->|yes| E[Use notebook visibility]
    D -->|no| F[Fall through to<br/>'owner only']
    C --> G[Apply public/team/<br/>org/user filter]
    E --> G
    F --> G

For public-by-design content (Open Energy notebook, share links), the visibility is set to 'public' at insert time and the backend exposes dedicated unauthenticated endpoints that filter on this.

Deployment topology

                           ┌──────────────────────┐
                           │   GitHub: main       │
                           └──────────┬───────────┘
                                      │ push
                  ┌───────────────────┼───────────────────┐
                  │                   │                   │
                  ▼                   ▼                   ▼
         ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
         │ Vercel          │ │ Railway         │ │ Supabase        │
         │ apps/frontend   │ │ apps/backend    │ │ Postgres + Auth │
         │ Next.js         │ │ Express         │ │ + Storage       │
         └─────────────────┘ └─────────────────┘ └─────────────────┘
  • Vercel auto-deploys apps/frontend on every push to main. Uses npm install --legacy-peer-deps.
  • Railway auto-deploys apps/backend on every push to main. Uses nixpacks.toml to force npm ci --legacy-peer-deps.
  • Supabase is a managed service. Migrations are applied manually via the SQL editor (see docs/PRODUCTION_MIGRATIONS.md).
  • Cron jobs are defined in Vercel and call /api/cron/* routes inside apps/frontend. The frontend cron routes typically forward to the backend or do their own work.

Key environment variables

Frontend (.env.local or Vercel):

  • NEXT_PUBLIC_SUPABASE_URL
  • NEXT_PUBLIC_SUPABASE_ANON_KEY
  • NEXT_PUBLIC_API_URL — backend base URL (Railway in prod)
  • NEXT_PUBLIC_SITE_URL — used in OG cards, canonical URLs
  • NEXT_PUBLIC_PATENTS_NOTEBOOK_ID — notebook UUID for the patents collection
  • NEXT_PUBLIC_EXPERIMENTS_NOTEBOOK_ID — notebook UUID for the experiments collection
  • NEXT_PUBLIC_GA_MEASUREMENT_ID, NEXT_PUBLIC_FB_PIXEL_ID — analytics
  • NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY

Backend (.env or Railway):

  • SUPABASE_URL
  • SUPABASE_SERVICE_ROLE_KEY — bypasses RLS
  • OPENAI_API_KEY — chat, image gen, note analysis
  • STRIPE_SECRET_KEY
  • STRIPE_PRICE_GRAPHENE_PLUS / STRIPE_PRICE_GRAPHENE_PLUS_ANNUAL — Graphene+ ($5/mo + $48/yr) recurring price IDs from the Stripe dashboard
  • STRIPE_PRICE_WRITE_CAFE_PRO — write.cafe Pro ($3/mo) recurring price ID from the Stripe dashboard. When unset, the cafe-pro checkout endpoint 503s and the modal shows a "not yet configured" notice
  • STRIPE_COUPON_RETENTION — coupon ID for the in-app cancellation flow's retention offer (e.g. 100% off for 1 month). When unset, the offer step is skipped server-side
  • RESEND_API_KEY
  • ELEVENLABS_API_KEY — TTS for season audio + short-stories audio
  • SPOTIFY_CLIENT_ID / SPOTIFY_CLIENT_SECRET
  • ALLOWED_ORIGINS — comma-separated list of allowed CORS origins
  • Cron secrets: WORKOUT_WINDOW_CRON_SECRET, CHAT_CRON_SECRET, DROPS_CRON_SECRET, etc.

Known landmines

These are real issues encountered in this codebase, documented so you don't hit them:

  • Prod-only DB objects drift silently: a handle_new_user trigger that auto-created a profiles row on signup lived ONLY in the prod database (dashboard-created, never committed as a migration). It stopped firing ~2026-05-21 and ~18 days of signups got no profile → profiles 404s + GET /api/chat/settings 500s, invisible because nothing in the repo described the trigger. Fixed by codifying it in migration 352 + a backfill. Lesson: any trigger/function/RLS the app depends on must live in a committed migration — if a prod object isn't in supabase/migrations/, treat it as load-bearing-but-undocumented and commit it. If you see fresh-user 404/500s on profile reads, check this trigger exists on prod first.

  • Reader deep-link params: /seasons/[id]/read honors ?episode=<n> (episode-window filter) and the #chapter-<n> anchor (auto-scroll) — NOT ?ep=<n>. The chapter-drop notification email shipped with ?ep= and silently landed followers at the top of the season; it now uses #chapter-<n>. Use the anchor for "jump to this chapter" links.

  • useContext null during static generation: client components that use React context (theme, auth) will crash the build if rendered without their provider. The fix is to render children directly during SSR and only mount the provider after useEffect (see ThemeProviderWrapper.tsx). The Pages Router _error.tsx exists for the same reason — it has zero context deps.

  • Next.js 15 strict types on useSearchParams()/useParams(): both can return null. Solution: append ! non-null assertion at the call site since they never return null in mounted client components.

  • legacy-peer-deps everywhere: react-native@0.81 demands react@19 but the rest of the workspace uses react@18. Vercel and Railway both need this flag. nixpacks.toml handles Railway; Vercel is configured via its own install command.

  • 5300-line dashboard/admin/page.tsx: legacy single-file admin. Don't add new tabs there — instead create a new sub-page like dashboard/admin/open-energy/ and link to it from the existing tab row.

  • Large hooks dep arrays: many existing pages have eslint warnings about missing dependencies. Don't try to "fix" all of them at once — they're often intentional, and changing them risks infinite re-render loops.

  • Promise.resolve(supabaseQuery).then(...) doesn't actually execute the query: Supabase query builders are PromiseLike but not real Promises; wrapping them in Promise.resolve() returns the builder, not the result. Always either await q directly or do await (async () => await q)() if you need a Promise for Promise.race (e.g. timeout wrappers). Encountered in auto-universe-batches's reliableUpdate helper — the bug stranded every batch silently with rows stuck in generating.

  • Per-:id ownership guard 401s public POST endpoints unless explicitly allowlisted (story-seasons.ts:225): the router.param('id', ...) middleware on the story-seasons router runs before any route handler and 401s ALL anonymous mutations (no bearer token + POST/PATCH/DELETE). Routes that are PUBLIC-BY-DESIGN (engagement beacons, listener feedback, comments, letters, timelines branch state) need to be added to the PUBLIC_MUTATION_PATTERNS regex allowlist or they'll 401 silently for signed-out visitors. Pre-fix, 7 endpoints were broken site-wide: /track-engagement, /comments, /feedback, /letters, /listener-rating, /episodes/:n/listen-event, all /timelines/* — entire categories of analytics + reader feedback were quietly dropping. Allowlisted entries still gate on is_published_to_graphene || owner_user_id IS NULL so private drafts don't leak existence.

  • .single() masks PostgREST column-doesn't-exist errors as "Season not found": Supabase JS .single() returns { data: null, error: <real error> } when the SELECT references a missing column. Code that only checks if (!season) throw new Error('Season not found') swallows the actual schema error. Convention going forward: also check if (seasonErr) throw new Error(...seasonErr.message). Encountered today when a SELECT including the new craft_framework column ran against a DB where migration 262 hadn't applied yet — every chapter in bulk-improve failed with "Season not found" while the actual problem was the column. Fixed in season-novel-chapter.ts:generateChapterRaw + generateChapter.

  • LLM-output field names colliding with schema-constrained columns (auto-show-in-universe.ts): the show-concept prompt asked the LLM for mode: 'novel' | 'serial' and the code passed the response straight to story_seasons.mode — but the schema check constraint (migration 208) only accepts 'journal' | 'novel' | 'timelines'. The "serial" concept actually belongs to story_seasons.narrative_mode ('bounded' | 'serialized', migration 155). When the LLM returned 'serial' the insert 500'd with a check-constraint violation. Two near-identical-looking fields with different vocabularies = easy to conflate in a prompt. Fix: map LLM output explicitly — 'novel'/'serial' from the LLM goes to narrative_mode ('bounded'/'serialized'); mode is hardcoded to 'novel' for auto-shows (none of them are journal or timelines). General lesson: when a prompt asks for a value that matches a column name, double-check the schema's allowed values before passing through.

  • Railway build phase must stay defined in nixpacks.toml — never rely on auto-detection, and never add a root "build" script: nixpacks.toml customizes the install phase; nixpacks' auto-detected build phase runs npm run build at the repo root, which has no "build" script (only build:frontend/build:backend/build:shared) → the whole Railway deploy dies with npm error Missing script: "build" (happened 2026-07-21, fixed by adding [phases.build]cd apps/backend && npm run build). Do NOT "fix" it by adding a root build script — the root vercel.json sets the frontend's buildCommand to npm run build, so a root build script makes Vercel build the backend instead of the Next app. The build fix belongs in nixpacks.toml (Railway-only; Vercel never reads it). Symptom is the same silent-prod-lag as the devDeps landmine below: a failing Railway build leaves the previous deploy running, so merged backend PRs sit undeployed while Vercel keeps shipping the frontend.

  • Railway omits devDependencies — build-required @types/* must live in dependencies (apps/backend/package.json): Railway's npm install runs prod-only, so any @types/* package the tsc build needs must sit in dependencies (that's why @types/express, @types/node, @types/multer, even typescript itself are there); devDependencies is only for test-time types (@types/jest, @types/supertest — the build's tsconfig excludes tests). @types/web-push landed in devDependencies in #1267 and every Railway deploy from #1270 to #1284 failed with TS7016: Could not find a declaration file for module 'web-push' — ~14 merged PRs sat undeployed for ~12h while Vercel kept shipping the frontend, so prod showed pages whose APIs 404'd. Fixed in #1284. Quick diagnosis for "new API 404s in prod but the page exists": curl -s -o /dev/null -w "%{http_code}" https://hivejournalbackend-production.up.railway.app/api/<route> — 401 means the route is deployed (auth-gated), 404 means the running build predates it; then check Railway → backend service → Deployments for the actual build error.

  • A prepare: tsc hook in a workspace package must pin "types" in its tsconfig, or it fails the deploy npm ci (packages/shared/tsconfig.json): packages/shared builds via a prepare script (tsc) so its gitignored dist regenerates on install. During the Railway deploy's npm ci --legacy-peer-deps, that prepare runs with the whole hoisted monorepo node_modules in scope — and with no "types" set, tsc auto-includes every @types/* it finds, including backend-only ones (better-sqlite3, cookiejar) whose type files it can't resolve in the shared package's context → error TS2688: Cannot find type definition file for '…'npm ci exits 2 → the whole deploy fails. This broke every backend deploy from the code-split (2026-07-26) to #1657 (~2 days), freezing prod on the last good build while ~a dozen backend PRs (persona-QS, posture routes, migrations 595/596) sat undeployed — same silent-prod-lag as the two landmines above. Fix: "types": [] in the platform-agnostic package's tsconfig (it needs no ambient types; add ["node"] only if it uses node globals). Applies to any workspace package with a prepare/build tsc hook. Local tsc --noEmit per-app does NOT catch it — only a fresh npm ci (or the deploy) runs the prepare hook in the hoisted-@types context.

  • Diagnosing a failed Railway deploy: the CLI hides it — use the dashboard log export. railway logs only tails the active (currently-serving) deployment, so on a failed deploy it streams the old healthy build's runtime logs (crons firing, canary green) — which looks like "everything's fine" and is deeply misleading. railway logs -d <failed-id> / --build frequently returns empty for a failed deployment, and can even hand back a stale cached build log (check the docker image.created timestamp — if it predates the failure, it's not the real build). The authoritative source is the Railway dashboard → service → Deployments → the failed one → "View Logs" → Download (a logs.<ts>.json array of {message,timestamp,severity}); grep it for severity:error / npm error / Build Failed. That JSON is what finally surfaced the TS2688 above. Also: railway redeploy reuses the latest deployment's commit — after a merge, if git auto-deploy didn't fire (it's flaky on squash-merges), redeploy just retries the OLD (broken) commit; use railway up --detach from the repo root to deploy the current local main, or confirm the new commit hash actually became latestDeployment. No healthcheck is configured, so Railway marks a deploy FAILED on either a build failure (npm ci exit≠0) or a process crash/exit (ON_FAILURE ×10) — build failures are the common case and the ones the CLI hides.

  • Email lives on auth.users, not profiles: there is no profiles.email column. There is also no profiles.name — the display-name fields are full_name (in auth.users.user_metadata) plus the display_name metadata key. Code that does profiles.select('email, name, ...') errors at runtime (column profiles.email does not exist) but silently — Resend-backed crons swallow it as "0 candidates this tick" and never warn. Use the getAuthUserEmailsByIds() helper to resolve id → { email, display_name, full_name } via paginated auth.admin.listUsers. Five known callsites fixed in this sweep: daily-prompt-email, weekly-recap-email, reader-digest-email, lovio-reminders, lovio-delivery. Surfaced by the Infra Monitor's daily brief flagging daily_prompt_email_tick erroring every 60 min.

New patterns introduced this cycle

These are reusable patterns that future features should follow rather than reinvent:

  • Self-logging error capture — backend catch blocks call recordServiceError({ service, context, error, metadata, severity? }) from services/service-errors.ts; the frontend has <ClientErrorReporter /> (mounted via ClientOnlyComponents) that hooks window.error + unhandledrejection and an explicit reportClientError() helper. Both write to the same service_errors table; the frontend events tag service='frontend' with two-layer noise filtering + server-side throttling. Triage UI at /dashboard/admin/errors plus per-group resolve / mute (24h / 7d / forever) controls.

  • Daily-snapshot + sparkline + WoW delta pattern — for any "is this metric going up?" view, capture a daily row in a *_snapshots table via a heartbeat-tracked cron (monetization_snapshot_tick in apps/backend/src/index.ts is the reference impl) with a UPSERT-on-date for idempotency. On first deploy, detect empty table and backfill from source data so the chart is non-empty immediately. Frontend <SparklinePanel> is inline-SVG (no chart-lib dep). Headline metric cards thread delta / deltaFormat props for "+N 7d" pills computed from history[length-8] vs history[length-1], suppressed when there's <8 days of data.

  • Self-hosted funnel events with PostHog dual-writemonetization_funnel_events mirrors PostHog's funnel for locked_impression / modal_opened / checkout_started / subscribed. Frontend trackProductEvent() in lib/analytics.ts writes to both. Terminal stage (subscribed) is server-side-only (Stripe webhook) so the conversion-rate denominator can't be inflated by a malicious client. Read endpoint aggregates server-side — no PostHog API dependency. Funnel events thread experiment_key + variant_key through metadata so any A/B can split conversion by variant for free.

  • Sticky A/B variant assignmentservices/experiments.ts does weighted random pinning per (experiment, identity) where identity is anon_id (PostHog distinct_id, pre-auth) preferring user_id (post-auth). Append-only — re-bucketing would invalidate the funnel. Frontend useExperimentVariant() hook caches the assignment in localStorage for instant subsequent reads; getCachedExperimentVariant() is the sync helper for event-metadata threading at click time.

  • Per-universe canon refresh (orthogonal to platform-version refresh) — when a universe's hardcoded constants change (new canonical character, expanded prompt directive, new commercial library entry), seasons in that universe need to be re-run through the latest constants to actually reflect the change in prose. The naive approach — bump CURRENT_PLATFORM_VERSION — marks every season across every universe as "behind" and triggers cross-universe noise. Instead: UniverseRefreshBanner is an opt-in per-universe refresh that calls /improve-all-behind with a new force_all=true body flag (added to improveAllBehindChapters opts) to bypass the platform-version filter. Three-step flow: bible bootstrap (force=true) → optional chapter re-plan (new regenerateChapterPlanForSeason — locks chapter count, updates outlines in place) → chapter prose regen against the refreshed plans + bible. Apply this pattern whenever a universe (or any other source-of-truth constant) gets a meaningful canon update.

  • State-aware "refresh available" banners via timestamp comparison — for any "is this thing on the latest version?" UI question, attach a canon_revision ISO timestamp to the universe/feature config and compare against the entity's created_at (or updated_at). When entity.created_at >= canon_revision, render a calm green "✓ on current" success state; when older, render the action CTA with a last_changes summary so the user knows what they'd be picking up. Avoids both extremes: nagging banners on freshly-created entities AND staleness creeping in unnoticed for old ones. The same component handles both states with the comparison gating which branch renders. Used today on Deep Cut (canon_revision='2026-05-17T20:00:00Z') and Turing Logs (canon_revision='2025-01-01T00:00:00Z' = far-past so all existing seasons stay green until canon changes).

  • End-to-end "+ auto-generate" flow (autoGenerateShowInUniverse) — when a "+ new X" affordance is meant to feel like a finished product (not a stub), structure the flow as: SYNCHRONOUS portion ≤30s for the parts the user needs to see immediately (concept + plan + insert → return the entity_id), then FIRE-AND-FORGET background sequence wrapped in startEvent/tracker.complete/failWithError so the existing pipeline-progress UI on the destination page lights up automatically (no new frontend wiring needed). Chain dependent steps inside a single async IIFE so failures propagate cleanly; bail subsequent steps if a critical earlier step fails. Surface the cost + step list in the confirm dialog so the user knows what they're spending. On the destination page, the existing pipeline-event polling does the rest. Reference: clicking "+ new show" on /graphene now produces a fully-playable show with audio in ~90-120s for ~$0.15-0.20.

  • Multi-plan subscriptions table — single subscriptions table discriminated by plan_key (graphene_plus, write_cafe_pro, future writer_studio_*). Each plan has a sibling helper file (graphene-subscription.ts, write-cafe-pro.ts) exposing an is<Tier>() predicate + get<Tier>Subscription() reader. The Stripe webhook's priceToPlanKey() maps env-var-configured price IDs to plan keys; the customer.subscription.* handlers stamp the row generically. Cancellation + retention flow (subscription_cancellations) is plan-agnostic and feeds the Why-people-cancel admin panel.

Where new things go

Adding...Goes in...
New backend endpointapps/backend/src/routes/<feature>.ts, mount in index.ts
New database tableNew numbered migration in supabase/migrations/ + apply manually
New page/routeapps/frontend/src/app/<route>/page.tsx (+ layout.tsx for metadata)
New OG cardapps/frontend/src/app/api/og/<route>/route.tsx (edge runtime)
New cron taskapps/frontend/src/app/api/cron/<feature>/<task>/route.ts + secret env var. MUST also call recordHeartbeat() from system-heartbeats.ts so it shows on /dashboard/admin/system-health — see docs/reference/RECURRING_TASKS.md
New backend setInterval loopapps/backend/src/index.ts — same heartbeat requirement as Vercel crons
New shared typepackages/shared/src/types/
New seed/import scriptscripts/<purpose>.ts, runnable via npx tsx scripts/<file>.ts
New AI feature docdocs/ai/features/<feature>.md and link from INDEX.md
New setup runbookdocs/<FEATURE>_SETUP.md (top-level docs/)

Future direction — unified production pipeline

The EmberKiln content surfaces (audiobook, Scene Studio, Maxell Reels, screenplay, print) currently each re-implement the same source → plan → render → assemble → distribute pipeline over the same providers (ElevenLabs, Replicate / Kling / Flux, ffmpeg), with duplicated cost metering + assembly. The north-star is a DRY refactor — a shared render kit of metered/cached stage adapters, a canonical Production IR, a source→Work ingest router, and thin per-surface recipes — so "one story, every form" is one engine, not five. See docs/product/EMBERKILN_PIPELINE_ARCHITECTURE.md. Not started; the safe entry point is extracting the render kit (behavior- preserving), then converging Reels onto the Scene Studio engine.

ARCHITECTURE — Docs | HiveJournal