music-video-gen/flow-state/HOWTO-visualizers.md

How to build a visualizer (scene)

A scene is a fragment shader plus a params block — nothing else. Uniform binding, the generated UI sliders, seeded per-track sampling, arc drift and the phase-gate checks are all derived from the schema. There is no per-scene wiring.

Reference: src/scenes/shader/kaleido-tunnel.js (simple), block-mosh.js (feedback/glitch), scan-tear.js (beat-quantised), metaballs.js (loops). Read these before writing anything.

The shape of a scene

export const myScene = {
    name: 'My Scene',
    family: 'glitch',              // flow | organic | minimal | structural | geometric | glitch
    kind: 'fragment',
    traits: ['camera', 'style'],   // which personality traits it honours (see below)

    params: {
        density: { type: 'float',  range: [0, 1],  default: 0.5, uniform: 'u_density', bias: 'density' },
        speed:   { type: 'float',  range: [0.1, 2], default: 0.5, uniform: 'u_speed', rate: true },
        palette: { type: 'palette', count: 4 },
    },

    reactive: {
        density: { feature: 'bandLow', amount: 0.3, response: 'smooth' },
    },

    shader: `
vec4 scene(vec2 uv, vec2 p) {
    float t = u_time * u_speed + u_seed;
    return vec4(palRamp(fbm(p * 4.0 + t, 4)), 1.0);
}
`,
};

Register it in src/scenes/registry.js (import + push into MODULES). Done.

The shader contract

You write one function: vec4 scene(vec2 uv, vec2 p).

• uv is 0..1 across the frame; p is centred, aspect-corrected, ~-1..1 on the short axis. Work in these, never pixels — scale pixel-sized things by u_pixelScale so a 720p preview matches a 4K export.
• The preamble is injected for you. Never declare main(), u_resolution, vUv, or any contract uniform yourself.

Uniforms available (see engine/shader-contract.js):

• Frame: u_time, u_frame, u_progress, u_seed, u_resolution, u_aspect, u_pixelScale, u_opacity.
• Audio, filled per frame from the FeatureTrack: u_loudness, u_rms, u_bandSub/low/mid/high/air, u_flux, u_centroid, u_flatness, u_width, u_beat, u_beatPhase, u_barPhase, u_phrasePhase, u_sectionProgress, u_sectionEnergy, u_buildSlope.
• Personality (u_sigSides, u_sigDrift, u_sigLine, …) — see below.
• Feedback: u_prev sampler, read with prev(vec2 uv) (returns the previous frame's colour; black if none). This is what makes trails/smear/datamosh work for free.

Helpers (already in the preamble):

• Palette: pal(int), palRamp(float) — always use these, or the look can't recolour the scene.
• Noise: hash11/12/22, vnoise, fbm, curl, rot, kaleido.
• Personality: sigShape/sigForm, sigCamera, sigFolded, sigEdge, sigGrain, sigHorizonY, sigAir.
• sat(x) = clamp(x, 0, 1).

Param schema fields

• type: float | int | bool | vec2 | palette.
• range [min, max] required for numerics; default required unless you want range[0] (prefer explicit defaults).
• uniform: the GLSL name. Must be u_* and must not collide with the contract (see "traps").
• bias: which track-character axis nudges sampling — energy, density, motion. This is how a loud drop gets denser scenes without the scene knowing about audio.
• rate: true: mandatory if the shader multiplies u_time by this param.
• reactive: { feature, amount, response }. response: linear (default) | spike | smooth | inverse.

Personality traits (traits)

The look generator builds each track on a signature of 1-2 traits, and a scene that doesn't honour all of them is never cast in that track — so the library intentionally shrinks per track. Only declare what you genuinely use:

trait	what to call	lint evidence
shape	sigShape / sigForm	a sigShape/sigForm( call
camera	sigCamera(p)	a sigCamera( call
space	sigHorizonY / sigAir	those calls or u_sigHorizon/Depth/Wash
style	sigEdge / sigGrain / sigFolded, or u_sigLine/Soft/Texture/Fold	the calls / those uniforms

The lint greps your shader and fails a declared trait with no evidence. Declaring [] (none) is valid.

Traps that have actually bitten here

1. Anything multiplying u_time must be rate: true. Phase is elapsed × rate; modulating a rate jumps the phase by elapsed × Δrate — a minute in a small wobble throws the image several units between frames, which measured as strobing at 2× the accessibility limit. So don't react a rate param, and add a bounded term instead: u_time * u_speed + u_bandLow * 2.0 is fine; u_time * (u_speed + u_bandLow) is not.
2. Never reuse a contract uniform name (u_width, u_time, u_seed, …). It's a GLSL redefinition error; the only symptom is a black frame.
3. Don't modulate whole-frame luminance on the beat. A per-kick min→max→min cycle is exactly what the WCAG 2.3.1 / Harding 3-flashes-per-second ceiling bans. Pulse a small local term (per-block tint) instead, and use smooth responses on loud things. If it's glitchy, quantise it — `floor(u_barPhase
   • n) + floor(t * k) * nmakes corruption step on the grid instead of crawling, which both reads better and stays below the flash rate (seescan-tear.js, block-mosh.js`).
4. Determinism is absolute. No Math.random(), performance.now(), Date.now(), new Date() — use hash*/fbm for variation, and let time flow through u_time/u_frame only. The lint greps for these.
5. Set a base image. A scene that only reads prev() is black for the first frames and fragile under seek. Generate your own field underneath the effect.
6. Don't hardcode saturated vec3(r,g,b) literals when you declared a palette — the lint flags more than two.

Families

Chosen by section kind in the arc driver — a breakdown never lands on a strobing glitch scene. Current counts (aim: 4-6 each): flow 2, organic 3, minimal 3, structural 2, geometric 3, glitch 3. The thin families are the best place to contribute next.

Verify

npm run lint:scenes   # schema/shader agreement both ways + determinism grep
npm test              # audio pipeline (unaffected, but cheap)

Then the GPU gates in the browser at http://localhost:5180/checks.html: Phase 2 (param range sweep — no NaN/black/white), Phase 5 (flash-rate sweep), Phase 7 (per-scene acceptance: distinctness, liveness, animation, determinism, 4K budget). These iterate the registry, so a new scene is covered automatically once registered.