A new top-level section building the metabolism concept: the track
supplies the food field, event impulses and per-section laws; the seed
picks the species (a rule set, not a parameter vector). Same song + seed
=> bit-identical frames; different songs => different worlds.
- analysis: single FFT pass -> per-frame feature table; score (onsets
folded into world geometry via voice ring + pitch-class sector), wells
(food placed by the harmony), graph (band correlations -> interaction
matrix), laws (per-section regimes)
- sim: typed-array world, 3 castes (builder/orbit, forager/commute,
spark/orbit-wide) with causality rings, persistence trails,
consequence field; fully deterministic
- lens: software framebuffer renderer (PNG in node, ImageData in
browser) + event-reactive camera (cuts on activity change, alternating
shot sizes, drift-and-push)
- metrics: coupling x residual fitness (not per-frame descriptors) plus
a grown-structure fingerprint; report + strip tools; 15 tests green
- DESIGN.md captures the concept, invariants and build order
3 reference test signals (drone/runner/dusk) grow measurably distinct
worlds while seeds of one song stay in the same biome.
One album must not render the same dungeon twice. dungeonTheme() scores
the measured summary per theme — pace terms gate on tempoConfidence so
doubled detectors never read fast, crypt is penalised for blazing
masters — plus small seeded jitter so ambiguous songs vary across seeds
while decided races stay decided. The plan carries the theme name;
dressing mix, planting/timber/puddle densities follow it, and the facade
builds all materials (stone/floor multipliers, torch + light color, wood,
fog tint) from the same record.
Calibrated bank spread is 4/4/4/4/1 (windswept holds runner); corners
stable across seeds; plan reshaping, facade wiring, and murk-vs-clear
gates covered by tests.
New VenueGenerator 'dungeon' archetype. The floor plan is generated first
on a 44x44 grid (rejection-sampled rooms, Kruskal MST + loop edges,
L-shaped corridors, BFS-verified connectivity); walls are merged runs
built exactly where open meets solid, so the shell has no holes, and
the walk route is a closed travelling tour through every room.
The camera either walks first-person (hero hidden to avoid clipping) or
plants on a per-room watch cell with line-of-sight to the route and
tracks the actor. Enemies (walker/structure/swarm ActorSpecs from the
track's own summary) lurk, stand sentry, dart and drift; loot caches
pause the analytic odometer, which can hold or advance but never retreat.
Includes dungeon-debug.html (2D map + timeline stills), 14 map tests
plus facade camera tests, and the portcullis-holds-for-the-whole-party
regression test.
Dropped files ignored pins while the URL still claimed them, so the census venue replaced the demo's pinned room. Drop loads now inherit venueSel/viz from the query, and stale bank addressing (song/seed/frame/warm) is cleared on drop while view settings stay.
Forced actor slots on like the billboard (sparse setups hid them), figures normalized to landmark scale against houses, crew-like emissive so they read at night. They were present but effectively invisible: gated, unlit, person-scale at the frame edge.
Two seeded slots on ~40% of builds hold figures of the track's own cast, grown on first update like the crew (identity arrives via update), normalized to person-scale with feet planted via measured bounds, palette-recolored per frame, glancing around analytically. Static for the whole video: they stream with their slots and never pop. Verified variance across seeds, pose and planting.
Full-width nose fascia with inset grille mouth, chrome slats and bezel-ringed lamps seated into it; rear fascia carrying the taillight bar plus diffuser fins. Patch-up: front side panels, rear quarters, floor pan and taller skirts sealing every daylight gap; lamps/grille/mirrors attached to body faces, all volumes interpenetrating. Verified front/rear/side, lint + tests green.
Mirrored env azimuth rotated 90 degrees in both sky and paint mappings (shared constant): fold lines sit fore/aft instead of mirroring the world at side views. Same render also completes the body patch-up: front side panels, rear quarters, floor pan, taller skirts sealing the doors, lamps/grille seated into the bumper, mirrors rooted in the wall. Verified on the 10x3 grid: reflections track the viz per cell, no seam, no holes at card scale.
Shared sky dome gains an optional viz overlay (off by default; chamber/stage untouched): the running video mixed over the gradient, masked to the upper sky so the horizon stays fog-matched. Same mirrored-sphere projection as the bodywork reflection, so paint and sky agree. Ride binds its vizTarget and holds 0.55.
Two rows of two like a real car (2 riders: 2-seater; more: 4-seater with one seat left empty); riders sit on cushions against backrests with headrests, driver ahead of the wheel. Long painted doors with handles, full-width dashboard with dial strip, center console with shifter glow, torus steering wheel on a column. Fender arches fattened. Rng cadence preserved per rider so downstream draws don't move.
Hood, trunk, bumpers, side skirts, torus-arch fenders, grille, headlamps, taillight bar, mirrors, rear spoiler, exhausts. Fixed layout in palette paint (painted panels join the viz reflection feed); dark trim stays matte. Verified front/side/rear: lamps on, viz pattern on the hood, arches over wheels.
The wraparound band read as bent-glass weirdness. Replaced with environmental spherical projection: chassis and cabin materials sample the shared viz texture by reflected view direction (onBeforeCompile injection, since this three version compiles 2D envMaps as cube samplers), added to emissive so palette bodywork color is retained. Verified on the 10x3 grid: patterning tracks the viz per cell while base colors hold.
The dash radio and lucky billboards hid the viz, so it gets a 200-degree glass band with the same feed, floated above the body's measured highest point (procedural bodies vary per track; fixed glass gets buried, measured) on cabin masts. Verified: 504px at strong contrast on a rear-side card, present by construction on all orbit angles.
The dash radio (0.6x0.35) and lucky billboards were the only viz surfaces, so pinned viz never read on Ride cards. A raked windshield plane across the cabin front carries the same viz texture feed at 4x the area, front-hemisphere only like real glass, with A-pillars seating it. Verified front/rear/side in the orientation rig and viz-bound in full composite.
Every rendered still links to the app frame that produced it (song + seed + frame + venue/viz pins + probe warm-up, new window), via shared src/ui/preview.js. Supporting work, each measured: app honors ?frame/?venueSel/?viz/?warm= with pre-build pins (Show.applyPins) and primes deep-link loads; shared pinViz/pinVenueShell paths for gallery, filmstrip and app; probes prime and carry their warm-up; bank probes analyse the bank PCM and file decode is fixed-rate with bit-exact WAV parsing so app and probes see identical samples; materials built mid-update compile synchronously (needsCompile). Round trip verified bit-identical gallery-versus-app.
Ride + corridor showed no corridor: with a room present the viz plays on its screens (radio/window/wall), so a dark model viz on the Ride's dash radio is invisible, and the shared rig was filming it through the room camera grammar meant for another world (directCamera rooms now keep the subject grammar + viz framing). Battery pins switched to combos that visibly read (Chamber + Kaleido, Stage + Assembly); scenario text states the visibility rule so a dead pin can't hide again.
New shared component src/ui/preview.js (appFrameUrl + previewLink): song + seed + frame + venue/venueSel deep-links. App honors ?frame= (warm seek on bank-song load) and ?venueSel= via shared pinVenueShell helper (src/venues/pin.js), which filmstrip and venue-gallery now also use. Filmstrip figures and venue-gallery cards wrap canvases; gallery cells stay unlinked (isolated scene probes, not reproducible in-app).
Collects every manual testing entry point into one library (src/debug/battery.js) rendered from the live registries: all 17 bank songs with app/+venue/strip links, all 70 scenes grouped by family with gate/wide links, all 3 venues, 9 curated param and scenario combos, and the graduated lab pages with their headless runners. debug.html renders it; new songs/scenes/venues join free.
Actors and objects change only at camera cuts. int/bool/discrete
params skip drift, lookahead and reactivity; discrete slowAxis steps
on the section-held journey. Venues fade instead of toggling on
interpolated popMul; roadside density is arrangement-only.
DESIGN.md collects the project's design decisions; AGENTS.md points
agents at it and the build-visualizer loop.
Headcount split (1-3 actors by population, rest junk): actors take the
first indices so popMul gating never hides the cast while thinning junk.
Each figure is the track's monolith (shared part geometries, own palette
material, distinct entries per figure, feet planted via bbox) and walks
a closed analytic loop - stand at furniture A facing it, meet at the
centre facing its partner (a lone figure watches the window instead),
stand at furniture B facing it - on a slow seeded unsynced period.
Smootherstep legs rest at both ends, so figures pause, look, then move;
facing blends toward the next leg's target for smooth turns. Positions
first for all figures, then yaws off same-frame positions; nothing
integrates, so seeks match playback. Figures breathe brighter than junk
to read against their own room in monochrome palettes.
Verified: 24/24 tests, scene lint clean, re-render bit-identical,
chamber strips centre 0.09/0.12 (pre-existing flat), drone 0.26->0.57,
elegy 0.61->0.59, agit unchanged.
Tables, chairs, cupboards, shelves with seeded book rows, potted plants
and box stacks built from primitives, dealt from a shuffled deck per
(seed, arrangement) so the first slots always land distinct types. Same
spiral + repulsion placement with per-item footprint radii; front clamp
pulled to z<=-1.5 for furniture scale so close cameras never wear a
table as a lens bar. Static (junk doesn't bob) with the existing
story-driven emissive breathe; re-hang on arrangement jumps moves
everything under the camera cut. popKey drops the actor dependence.
Verified: 24/24 tests, scene lint clean, same-frame re-render
bit-identical, rooms read as furnished across centre/drone/elegy.
Chamber-forced strips: elegy direction 0.26->0.61; centre stays
pre-existing-flat (0.11->0.09, dark song, same early arrangement);
drone agit unchanged (viz-driven).
driveRideCamera checked the passerby/billboard plants after the generic
chase branch, which returns on every planned shot — both plants were dead
code (camera orbited, only the car drive-through worked). Check them first.
Planner fixes for the now-live plants:
- a planted section keeps the plant on its first cue only; later cues fall
back to the screen/room rotation (a whole section planted is a stuck
camera with the car looping through one frame)
- venueShotAt holds plant position too, not just the look
- no consecutive planted sections in either order; the section-level
history never forms streaks for single-cue sections, so gate on the
previous section directly (draw-first, rng-neutral elsewhere)
- passerby plant back/up, aimed between approach and pass point so the
whoosh does not swallow the frame
Verified: billboard holds centre-frame with the viz live while the car
crosses mid-cue; passerby plant static +/- handheld; tests 24/24, scene
lint clean, standard gates unchanged vs HEAD (P9/P12 + census
centre/drone fail pre-existing).
Hard-cut cues now obey a single rule: the colour jumps with the image
(palette blend lerp skipped, even under blend-mode directors — the 2-frame
image ramp stays for flash safety) and both cameras must land somewhere
new, never hold static.
- Viz gaze relocates unconditionally on hard cuts (was +0.3 chance)
- Venue marks force fresh size (≠ previous) and flip look target
screen↔room on hard-cut cues, landed instantly via venueShotAt
- Motive yields to grammar: a cutaway needn't face the action
Verified on lattice (2 hard cuts): gaze jumps, venue flips size+target
with 1.4/2.6-unit moves. Phases 8/11 fast gates 30/30.
Camera architecture: a module with directCamera:true drives its own rig
inside update() (analytic f(timeline, seed, framing, venueState)); the
shared rig is ignored for that layer only. ModelLayer prefers own camera,
Show skips shared injection. Ride migrates its full grammar over
(traverse, orbit, car tracking, jumps, clamp); Compositor ride branches
removed. Chamber/stage unchanged on the shared path.
Sidewalk shot: new ride-only 'passerby' plan target (breaths only, never
climax, never twice running) plants a static roadside camera; the module
drives the car +12 to -14 on driveT with the world frozen. Verified
far-pass-away through the Show path; lint + 24 tests green.
The corner title plate should stay clean — the tweak is already
confirmed via the Apply dialog and a toast. Appending to the fileName
made every LLM-tuned export carry an ugly suffix.
Co-Authored-By: Claude <noreply@anthropic.com>
- spec.js: round all songSpec + extended summary/tempo/normalization to
r2 (2dp) so high-precision 0.6607… stops wasting tokens; hoist r2/r3
above specForPrompt to fix ReferenceError; FULL control wording kept.
- thumbs.js: border detection — first/last + 0.5s of any section start
→ 64×64 (borders), interior → 32×32 (between). Back-compat shim for
old {size:N} callers. Returns size+role per frame for attachFrames.
- vite.config.js: raise sanitizeBodyForLog to 64k for __llm so 8×
vision frames aren't truncated (was 16384, only 2 images logged).
Also log reasoningEffort via providerOptions.
- llmTweak.js: set providerOptions reasoningEffort:'low' for both
streamObject and generateObject fallback to cut 2459 → faster
streaming progress, temperature 0.7 kept for variety.
Verification: build+lint:scenes green, tests 18/18.
Co-Authored-By: Claude <noreply@anthropic.com>
The dev proxy buffered the entire gateway response via
arrayBuffer() before forwarding, which defeats streamObject —
no partials arrive until the full completion. Now event-stream
responses are streamed chunk-by-chunk via the upstream ReadableStream,
so the UI's "LLM streaming…" toast and console partials update live.
Non-streaming JSON is still buffered as before.
Co-Authored-By: Claude <noreply@anthropic.com>
- Streaming: requestSongTweak now uses streamObject when a progress
callback is provided (via handleLlmTweak), emitting partial {rationale,
edits} and updating toast/console as text arrives; falls back to
generateObject if gateway doesn't support streaming. Models always
called via provider.chat(modelId) / /v1/chat/completions, the correct
path for meta/muse-spark-1.3-contributor via gateway.
- Rich context: spec.js gains FULL control wording (any amount/arrangement/
bpm allowed), and detailed payload — trackSummary with bandBalance,
tempo, normalization (per-feature lo/hi), sectionsDetailed (energy/
flux/centroid/flatness/width/slope/endEnergy/duration), stats
(kinds, avg/max energy, bars). LLM told You have FULL control.
- Low-res vision: new src/llm/thumbs.js — captureThumbnails(show, {8,32})
samples section midpoints, warms up state, renders via Show, box-averages
down to 32×32 JPEG data URLs, restores timeline. main.js now captures 8
frames with two animation frames of pause, then passes them as {type:
image, image: dataUrl} parts in the user message (llmTweak.js
attachFramesToMessages).
- Proxy logging (no bearer): vite.config.js llmGatewayProxy now logs
sanitized requests/responses to logs/llm-proxy.log (JSON lines, 1MB
rotation, redacts apiKey/authorization from path/body/headers) so the
coding agent can inspect prompts without re-calling. .gitignore adds
logs/.
Co-Authored-By: Claude <noreply@anthropic.com>
Tune appeared to do nothing — the 400's cause was buried, in-flight was
near-invisible, and dialog only appeared on success. Make every stage
observable and cheap to diagnose.
- src/llm/llmTweak.js (+ spec.js): provider.chat(model) + providerOptions
strictJsonSchema:false fix (correct placement), edits nullable so strict
requires all 7 keys; clamped, budget-aware. Now logs [llm] fetch target,
messages/spec, raw vs clamped edits, rationale/confidence/usage,
elapsed time, and diff summary. NO_EDITS refunded with console.warn.
- src/main.js: immediate OK toast "Contacting LLM (model)…" on click,
console.group for handleLlmTweak start, post-result toast with
rationale snippet + editsSummary before dialog, preview synthesis
logged, catch logs stack once with toast.
- vite.config.js: /__llm dev proxy already in place from prior fix;
retained for CORS-free dev.
Co-Authored-By: Claude <noreply@anthropic.com>
Previous attempt set `strictJsonSchema:false` on createOpenAI()
(factory ignores unknown keys), so wire was still strict:true and
sparse edits {brightness:0.3} were rejected: Missing 'arrangement'.
- src/llm/llmTweak.js: use provider.chat(modelId) (/chat/completions,
correct for Meta via gateway, not /responses) and pass
`providerOptions:{ openai:{ strictJsonSchema:false } }` on
generateObject where the SDK actually reads it.
- src/llm/spec.js: edits fields nullable instead of optional — strict
requires every key in `required` when `additionalProperties:false`;
nullable makes required = all 7 (model emits null for unchanged),
clampEdits already strips nulls so NO_EDITS still works. Prompt now
instructs "use null for unchanged, example { bpm:null,... }".
- test/llm.test.js: minimal valid result now all-null with one edit.
Co-Authored-By: Claude <noreply@anthropic.com>
The Responses API via Vercel AI Gateway requires `required` to
include every key when `strict:true`. Our TweakEditsSchema has 7
optional song knobs, so sparse edits (e.g. only {brightness: 0.3})
were rejected with 400: Missing 'arrangement'.
- src/llm/llmTweak.js: createOpenAI({ strictJsonSchema: false }) so
the gateway accepts optional properties; keeps zod validation +
clampEdits as the real guardrail.
- src/llm/spec.js: drop .strict() on TweakEditsSchema — extra keys
are stripped rather than erroring, more robust against model drift.
Co-Authored-By: Claude <noreply@anthropic.com>
Direct browser fetch to https://ai-gateway.vercel.sh/v1 is blocked
(No Access-Control-Allow-Origin). In dev the AI SDK now calls
/__llm/v1 via same-origin Vite middleware (llmGatewayProxy), which
forwards server-side to the real gateway — no CORS. Production still
calls the gateway directly (deployments needing CORS-free prod should
add a serverless /api/llm proxy).
- vite.config.js: new llmGatewayProxy middleware for /__llm/* —
handles OPTIONS preflight, forwards via Node fetch to
x-llm-gateway (fallback Vercel), mirrors status/headers +
access-control-allow-origin, preserves query/body.
- src/llm/llmTweak.js: in DEV use baseURL /__llm/v1 + proxyFetch
that injects x-llm-gateway; prod keeps cfg.gatewayUrl.
Co-Authored-By: Claude <noreply@anthropic.com>
Secrets must never be baked into the static build. Stop reading
VITE_AI_API_KEY from env — key comes only from localStorage via the
⚙️ config dialog (flow-state:llm). Gateway URL / model may still come
from env as non-secret defaults.
- src/llm/config.js: DEFAULTS model → meta/muse-spark-1.3-contributor
(cheap, per user request), readEnv no longer touches apiKey,
getLlmConfig resolves key only from stored.apiKey.
- .env.example: drop VITE_AI_API_KEY=, document browser-only key flow.
- src/main.js: dialog copy now says key is browser-only, never baked.
Build + lint:scenes + tests still green. No VITE_AI_API_KEY string
remains in source.
Co-Authored-By: Claude <noreply@anthropic.com>
Add a ✨ tune button that lets an LLM inspect the proposed
visualization and tweak the proc-song knobs so the next generation
looks better. Edits only the 7 songbank params (bpm, brightness,
noise, noiseColour, dynamics, transition, arrangement) — never the
Look — then re-synthesizes and re-generates locally for preview.
- ai + @ai-sdk/openai + zod (library per plan) against the Vercel AI
Gateway (OpenAI-compatible, configurable via
VITE_AI_GATEWAY_URL/MODEL/API_KEY, localStorage override,
.env.example).
- Structured output (TweakResultSchema) + clampEdits enforces ranges;
prompt ≤ ~2.5k tokens, max 2-3 knobs ±0.15 / bpm ±8.
- Budget caps: 3/load + 10/session (sessionStorage), checked before
network, NO_EDITS refunded, offline/auth/rate-limit mapped.
- Prompt split into src/llm/spec.js (pure, node-testable) + llmTweak.js
(gateway + digest) to keep `node --test` off the Vite JSON path.
- UI: ⚙️ gateway/model/key dialog (localStorage flow-state:llm),
confirm(prompt) with hint, diff modal with Apply/Dismiss; Apply
synthesizes → FeatureTrack → generateLook → Show.useTrack.
- Tests: 7 new LLM cases (clamp, schema, prompt, budget), 18/18 pass.
Build + lint:scenes green.
Co-Authored-By: Claude <noreply@anthropic.com>
Whole-screen flashing/coherent motion is now a gated comfort signal
rather than a hunt-and-tweak. Three-arm agitation (flash/3,
tile-motion/0.32, structure/0.12) over a 1s floating window, worst
window wins — engine/agitation.js.
Policy centralised in look/comfort.js: director comfort appetite,
per-scene agitationOf proxy, agitationWeight lean and stackAgitation.
Canvas beat/flux capped in Layer.resolveParams (0.15/0.25, spike→smooth)
so one contract change tames every full-frame scene; lint warns above
the cap.
Selection now filters before casting (look/scene-filter.js, 12+ ranged
metrics from checks/scene-metrics.js, SCHEMA 7) and validates after
stacking (look/composition-validator.js) with a deterministic retune:
drop overlay → swap quiet composable → reselect filtered shot. Fix
validator attempt-3 ground-layer bug that crashed ?song=dusk
(LookGenerator now passes module, not layer).
Eight canvas shaders window their beat/flux (focusField + per-seed hash
mask) and report the scaled amount to Layer so the cap stays authoritative:
scan-tear, gate-corridor, analog-wow, block-mosh, metaballs,
pitch-shatter, scale-mosaic, tide-rings.
Checks: scene-gate comfort gate (agitation ≤ 0.70), filmstrip/gallery
meters + per-scene drawer, metadata SCHEMA 7 with ranged metrics and
fingerprint over agitation/comfort sources. main.js surfaces
COMPOSITION_REFUSED as a user-facing error.
Lint: 114 files clean, 71 scenes ok. All 17 songs generate looks.
Build passes. metadata.json refresh deferred to gallery → Refresh
metadata (SCHEMA 7 writer in vite.config).
Co-Authored-By: Claude <noreply@anthropic.com>
probeExportCodec returned codec/muxerCodec but not config, so the
pre-probed pick used by the software-encoding warning path passed
undefined to VideoEncoder.configure, which Chrome rejects as "not of
type VideoEncoderConfig". Include config in the probe result and thread
it through exportSegment (test render was still reprobing and ignoring
the pick).
Also prefer the browser-negotiated config from isConfigSupported when
available — it is guaranteed valid for configure (covers new required
fields), whereas our minimal config had started failing validation on
Chrome despite isConfigSupported reporting "supported".
Co-Authored-By: Claude <noreply@anthropic.com>
Firefox on Linux exposes VideoEncoder but reports every avc1.* config as
unsupported (OpenH264 is decode-only), so pickVideoConfig returned null and
export threw "no supported H.264 configuration found". Probe VP9/AV1 families
after H.264 and pick the first profile that isConfigSupported and does not
reorder (B-frame) — same 12-frame ordering test. Map the chosen family to
mp4-muxer's video codec (avc/vp9/av1).
Before any rendering, probe the codec for the chosen preset and show a
modal when we fell off H.264 onto a software VP9/AV1 path — that's what the
user felt as "super slow". The modal names the codec, explains render vs
encode cost, suggests Chromium/Chrome for hardware H.264 (or 720p/lower
preset), and offers Continue anyway / Cancel (Esc/backdrop also cancel) plus
"Don't warn again this session". Injected pick is threaded through
Exporter.export({ videoPick }) and exportSegment so we don't probe twice.
Co-Authored-By: Claude <noreply@anthropic.com>
Model stages render with alpha:0 where no mesh covers (transparent
clear) and carry dark material colours. Lumakey keys on luma
(dot(src,0.212…)), so a dim mesh keyed to ~0 and vanished — Pylon
Field 3D / Synthwave Corridor looked black. Use 'normal' for
kind:'model' so the alpha is the mask; fragment over dark ground
stays covered. Per-stage fog/light tuning deferred to a director
param (LookGenerator) per review — not hand-tuned per stage.
Co-Authored-By: Claude <noreply@anthropic.com>
Two Stage-A model stages behind the same infra as Assembly:
Pylon Field 3D (structural, kind:model, actor:structure) — chorus
stacked legs + protagonist crowns on the pylon-grid perspective
grid, transparent compositing via ModelLayer, analytic f(t,seed).
Synthwave Corridor (structural, kind:model, actor:vehicle) — grid
verges with streaming chorus passers + monolith hero on the road,
verge math preserved from synthwave-run horizon/roadHalf calc.
Registered in scenes/registry.js; consumes:['form','ink','staging'],
traits:['shape','space','camera','style'] so trait/consumes gates
stay testable (reads personality in update()). Grid texture falls
back to DataTexture headless so lint/build stay green outside a
browser.
Gates: lint 110 clean / 70 literals / 71 scenes; vite build 301 modules.
Co-Authored-By: Claude <noreply@anthropic.com>
Dejvino