music-video-gen/flow-state/HOWTO-visualizers.md

How to build a visualizer (scene)

Start here: npm run new:scene -- "My Scene" --family=glitch --traits=camera,style

That writes the module, registers it, and leaves a skeleton that already passes every gate. Then write the scene() body, npm run lint:scenes, and open checks.html?scene=My%20Scene. The rest of this file is the reference.

There is also a repo skill — .claude/skills/build-visualizer/ — which is the same procedure in the form an agent will follow.

A scene is a fragment shader plus a params block — nothing else. Uniform binding, the generated UI sliders, seeded per-track sampling, arc drift and the phase-gate checks are all derived from the schema. There is no per-scene wiring.

Reference: src/scenes/shader/kaleido-tunnel.js (simple), block-mosh.js (feedback/glitch), scan-tear.js (beat-quantised), metaballs.js (loops). Read these before writing anything.

The shape of a scene

export const myScene = {
    name: 'My Scene',
    family: 'glitch',              // flow | organic | minimal | structural | geometric | glitch
    kind: 'fragment',
    traits: ['camera', 'style'],   // which personality traits it honours (see below)

    params: {
        density: { type: 'float',  range: [0, 1],  default: 0.5, uniform: 'u_density', bias: 'density' },
        speed:   { type: 'float',  range: [0.1, 2], default: 0.5, uniform: 'u_speed', rate: true },
        palette: { type: 'palette', count: 4 },
    },

    reactive: {
        density: { feature: 'bandLow', amount: 0.3, response: 'smooth' },
    },

    shader: `
vec4 scene(vec2 uv, vec2 p) {
    float t = u_time * u_speed + u_seed;
    return vec4(palRamp(fbm(p * 4.0 + t, 4)), 1.0);
}
`,
};

The scaffolder writes all of that for you, including the registry import and entry, and bakes name-derived constants into the skeleton field so two freshly scaffolded scenes are not identical to each other. To do it by hand: write the file and add the import + MODULES entry in src/scenes/registry.js.

The shader contract

You write one function: vec4 scene(vec2 uv, vec2 p).

• uv is 0..1 across the frame; p is centred, aspect-corrected, ~-1..1 on the short axis. Work in these, never pixels — scale pixel-sized things by u_pixelScale so a 720p preview matches a 4K export.
• The preamble is injected for you. Never declare main(), u_resolution, vUv, or any contract uniform yourself.

Uniforms available (see engine/shader-contract.js):

• Frame: u_time, u_frame, u_progress, u_seed, u_resolution, u_aspect, u_pixelScale, u_opacity.
• Audio, filled per frame from the FeatureTrack: u_loudness, u_rms, u_bandSub/low/mid/high/air, u_flux, u_centroid, u_flatness, u_width, u_beat, u_beatPhase, u_barPhase, u_phrasePhase, u_sectionProgress, u_sectionEnergy, u_buildSlope.
• Personality (u_sigSides, u_sigDrift, u_sigLine, …) — see below.
• Feedback: u_prev sampler, read with prev(vec2 uv) (returns the previous frame's colour; black if none). This is what makes trails/smear/datamosh work for free.

Helpers (already in the preamble):

• Palette: pal(int), palRamp(float) — always use these, or the look can't recolour the scene.
• Noise: hash11/12/22, vnoise, fbm, curl, rot, kaleido.
• Personality: sigShape/sigForm, sigCamera, sigFolded, sigEdge, sigGrain, sigHorizonY, sigAir.
• sat(x) = clamp(x, 0, 1).

Param schema fields

• type: float | int | bool | vec2 | palette.
• range [min, max] required for numerics; default required unless you want range[0] (prefer explicit defaults).
• uniform: the GLSL name. Must be u_* and must not collide with the contract (see "traps").
• bias: which track-character axis nudges sampling — energy, density, motion. This is how a loud drop gets denser scenes without the scene knowing about audio.
• rate: true: mandatory if the shader multiplies u_time by this param.
• reactive: { feature, amount, response }. response: linear (default) | spike | smooth | inverse.

Personality traits (traits)

The look generator builds each track on a signature of 1-2 traits, and a scene that doesn't honour all of them is never cast in that track — so the library intentionally shrinks per track. Only declare what you genuinely use:

trait	what to call	lint evidence
shape	sigShape / sigForm	a sigShape/sigForm( call
camera	sigCamera(p)	a sigCamera( call
space	sigHorizonY / sigAir	those calls or u_sigHorizon/Depth/Wash
style	sigEdge / sigGrain / sigFolded, or u_sigLine/Soft/Texture/Fold	the calls / those uniforms

The lint greps your shader and fails a declared trait with no evidence. Declaring [] (none) is valid.

Traps that have actually bitten here

1. Anything multiplying u_time must be rate: true. Phase is elapsed × rate; modulating a rate jumps the phase by elapsed × Δrate — a minute in a small wobble throws the image several units between frames, which measured as strobing at 2× the accessibility limit. So don't react a rate param, and add a bounded term instead: u_time * u_speed + u_bandLow * 2.0 is fine; u_time * (u_speed + u_bandLow) is not.
2. Never reuse a contract uniform name (u_width, u_time, u_seed, …). It's a GLSL redefinition error; the only symptom is a black frame.
3. Don't modulate whole-frame luminance on the beat. A per-kick min→max→min cycle is exactly what the WCAG 2.3.1 / Harding 3-flashes-per-second ceiling bans. Pulse a small local term (per-block tint) instead, and use smooth responses on loud things. If it's glitchy, quantise it — `floor(u_barPhase
   • n) + floor(t * k) * nmakes corruption step on the grid instead of crawling, which both reads better and stays below the flash rate (seescan-tear.js, block-mosh.js`).
4. Determinism is absolute. No Math.random(), performance.now(), Date.now(), new Date() — use hash*/fbm for variation, and let time flow through u_time/u_frame only. The lint greps for these.
5. Set a base image. A scene that only reads prev() is black for the first frames and fragile under seek. Generate your own field underneath the effect.
6. Don't hardcode saturated vec3(r,g,b) literals when you declared a palette — the lint flags more than two.

Families

Chosen by section kind in the arc driver — a breakdown never lands on a strobing glitch scene. The library is at seven per family (42 scenes). Check the current spread before adding another:

node -e "import('./src/scenes/registry.js').then(({scenes})=>{const b={};for(const m of scenes)(b[m.family]??=[]).push(m.name);console.log(b)})"

Depth matters more than it looks: the casting rule in look/Personality.js disqualifies scenes that do not honour the track's signature traits, so the pool a given track draws from is smaller than the library. Thin traits (space, shape) are worth more than thin families.

Verify

Three rungs, each about ten times cheaper than the next. Climb them in order.

npm run lint:scenes   # ~1s, no browser

Static gates: schema and shader agreeing both ways, determinism grep, rate flags, a declared trait with no evidence in the source, a dead camera (p = sigCamera(p) and then nothing reads p), prev() with no base image, and loops with a large bound and no early break. A loop whose cost is genuinely fixed — sampling a curve at a set resolution — can say so with a // lint: fixed-cost comment just above it.

http://localhost:5180/checks.html?scene=My%20Scene

The per-scene acceptance battery for one scene: schema, renders, animates, deterministic, distinct from every other scene, param sweep, flash rate, and one line per declared trait proving the image actually responds to it. Ten lines and a verdict — this is the loop to stay in while writing.

http://localhost:5180/checks.html?slow=1

Everything. Phases 2, 5 and 7 iterate the registry so a new scene is covered automatically; Phases 8-10 cover how the look generator uses it. Run this once before committing.