music-video-gen/flow-state/HOWTO-visualizers.md
Dejvino 44e1826fcb Six more visualizers, built through the scaffolder
Seven per family now, 42 scenes. Each one exists because of the second
line of its header comment — what makes it different from the scenes it
sits beside — since "no two scenes render the same image" is a gate with a
numeric floor:

  Smoke Column (flow)        a plume with a source, a body and a
                             dissipating head. Everything moves one way and
                             widens; an isotropic field has no up.
  Cell Divide (organic)      a partition, not objects. Every pixel belongs
                             to a cell, boundaries are hard, no background.
  Eclipse Field (minimal)    the occluder nearly fills the frame and is not
                             the subject — the corona around it is. The
                             library's one high-contrast minimal scene.
  Girder Lattice (structural) the only scene whose subject is ABOVE the
                             camera; perspective converges downward.
  Quasicrystal (geometric)   five plane waves at incommensurate angles, so
                             the pattern has local symmetry and no tile,
                             no cell and no centre.
  Time Smear (glitch)        nothing is displaced. Each band shows the same
                             image at a different age — a slit-scan built
                             from one feedback buffer by giving each band
                             its own persistence.

The scaffolder produced six skeletons that passed lint and every gate
before a line of shader was written, and all six passed their per-scene
gate first time once written. That is what it was for.

Two real faults, both found by gates rather than by eye:

Cell Divide coloured each pixel by its nearest seed, and exactly on a tie
which seed is nearest comes down to the last bit of a distance. Two renders
disagreed by a whole palette step: 6/255 against a ceiling of 1, and it
also broke Phase 6's preview/export parity because that look casts it.
Blending the nearest tint with the runner-up across the membrane makes the
two answers agree in the limit — and reads better, as membranes rather than
cuts.

The lint's own "prev() with no base image" rule fired on a scene whose
comment explained why it does NOT rely on prev(). Rules that ask "does the
code do X" now run against a comment-stripped copy; the fixed-cost opt-out
still reads the original, since it is a comment.

Phase 6's parity tolerance goes from 1/255 to 2. Measured in order:
unprimed, the first pass differed on frames 0/2/3 — fixed earlier by
priming. Primed and isolated: 0/40 at delta 0, three times over. Primed,
range-warmed and run at the end of the full suite: three frames at delta 2.
The warm-up stays because it is correct, but the residue is GPU load, not
logic, and it is the same 1-2/255 Phases 5 and 7 already account for. A
real divergence scores in the tens.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 00:14:20 +02:00

186 lines
8.2 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# How to build a visualizer (scene)
> **Start here:** `npm run new:scene -- "My Scene" --family=glitch --traits=camera,style`
>
> That writes the module, registers it, and leaves a skeleton that already passes
> every gate. Then write the `scene()` body, `npm run lint:scenes`, and open
> `checks.html?scene=My%20Scene`. The rest of this file is the reference.
>
> There is also a repo skill — `.claude/skills/build-visualizer/` — which is the
> same procedure in the form an agent will follow.
A scene is **a fragment shader plus a params block** — nothing else. Uniform
binding, the generated UI sliders, seeded per-track sampling, arc drift and the
phase-gate checks are all derived from the schema. There is no per-scene wiring.
Reference: `src/scenes/shader/kaleido-tunnel.js` (simple), `block-mosh.js`
(feedback/glitch), `scan-tear.js` (beat-quantised), `metaballs.js` (loops).
Read these before writing anything.
## The shape of a scene
```js
export const myScene = {
name: 'My Scene',
family: 'glitch', // flow | organic | minimal | structural | geometric | glitch
kind: 'fragment',
traits: ['camera', 'style'], // which personality traits it honours (see below)
params: {
density: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_density', bias: 'density' },
speed: { type: 'float', range: [0.1, 2], default: 0.5, uniform: 'u_speed', rate: true },
palette: { type: 'palette', count: 4 },
},
reactive: {
density: { feature: 'bandLow', amount: 0.3, response: 'smooth' },
},
shader: `
vec4 scene(vec2 uv, vec2 p) {
float t = u_time * u_speed + u_seed;
return vec4(palRamp(fbm(p * 4.0 + t, 4)), 1.0);
}
`,
};
```
The scaffolder writes all of that for you, including the registry import and
entry, and bakes name-derived constants into the skeleton field so two freshly
scaffolded scenes are not identical to each other. To do it by hand: write the
file and add the import + `MODULES` entry in `src/scenes/registry.js`.
## The shader contract
You write one function: `vec4 scene(vec2 uv, vec2 p)`.
- `uv` is 0..1 across the frame; `p` is centred, aspect-corrected, ~-1..1 on the
short axis. **Work in these, never pixels** — scale pixel-sized things by
`u_pixelScale` so a 720p preview matches a 4K export.
- The preamble is injected for you. Never declare `main()`, `u_resolution`,
`vUv`, or any contract uniform yourself.
**Uniforms available** (see `engine/shader-contract.js`):
- Frame: `u_time`, `u_frame`, `u_progress`, `u_seed`, `u_resolution`, `u_aspect`,
`u_pixelScale`, `u_opacity`.
- Audio, filled per frame from the FeatureTrack: `u_loudness`, `u_rms`,
`u_bandSub/low/mid/high/air`, `u_flux`, `u_centroid`, `u_flatness`, `u_width`,
`u_beat`, `u_beatPhase`, `u_barPhase`, `u_phrasePhase`, `u_sectionProgress`,
`u_sectionEnergy`, `u_buildSlope`.
- Personality (`u_sigSides`, `u_sigDrift`, `u_sigLine`, …) — see below.
- Feedback: `u_prev` sampler, read with `prev(vec2 uv)` (returns the previous
frame's colour; black if none). This is what makes trails/smear/datamosh work
for free.
**Helpers** (already in the preamble):
- Palette: `pal(int)`, `palRamp(float)` — always use these, or the look can't
recolour the scene.
- Noise: `hash11/12/22`, `vnoise`, `fbm`, `curl`, `rot`, `kaleido`.
- Personality: `sigShape`/`sigForm`, `sigCamera`, `sigFolded`, `sigEdge`,
`sigGrain`, `sigHorizonY`, `sigAir`.
- `sat(x)` = `clamp(x, 0, 1)`.
## Param schema fields
- `type`: `float` | `int` | `bool` | `vec2` | `palette`.
- `range` `[min, max]` required for numerics; `default` required unless you want
range[0] (prefer explicit defaults).
- `uniform`: the GLSL name. Must be `u_*` and must not collide with the contract
(see "traps").
- `bias`: which track-character axis nudges sampling — `energy`, `density`,
`motion`. This is how a loud drop gets denser scenes without the scene knowing
about audio.
- `rate: true`: **mandatory if the shader multiplies `u_time` by this param.**
- `reactive`: `{ feature, amount, response }`. `response`: `linear` (default) |
`spike` | `smooth` | `inverse`.
## Personality traits (`traits`)
The look generator builds each track on a signature of 1-2 traits, and a scene
that doesn't honour **all** of them is never cast in that track — so the library
intentionally shrinks per track. Only declare what you genuinely use:
| trait | what to call | lint evidence |
|---|---|---|
| `shape` | `sigShape` / `sigForm` | a `sigShape`/`sigForm(` call |
| `camera` | `sigCamera(p)` | a `sigCamera(` call |
| `space` | `sigHorizonY` / `sigAir` | those calls or `u_sigHorizon/Depth/Wash` |
| `style` | `sigEdge` / `sigGrain` / `sigFolded`, or `u_sigLine/Soft/Texture/Fold` | the calls / those uniforms |
The lint greps your shader and fails a declared trait with no evidence. Declaring
`[]` (none) is valid.
## Traps that have actually bitten here
1. **Anything multiplying `u_time` must be `rate: true`.** Phase is
`elapsed × rate`; modulating a rate jumps the phase by `elapsed × Δrate` — a
minute in a small wobble throws the image several units between frames, which
measured as strobing at 2× the accessibility limit. So don't react a rate
param, and add a *bounded* term instead: `u_time * u_speed + u_bandLow * 2.0`
is fine; `u_time * (u_speed + u_bandLow)` is not.
2. **Never reuse a contract uniform name** (`u_width`, `u_time`, `u_seed`, …).
It's a GLSL redefinition error; the only symptom is a black frame.
3. **Don't modulate whole-frame luminance on the beat.** A per-kick min→max→min
cycle is exactly what the WCAG 2.3.1 / Harding 3-flashes-per-second ceiling
bans. Pulse a *small local* term (per-block tint) instead, and use `smooth`
responses on loud things. If it's glitchy, **quantise it**`floor(u_barPhase
* n) + floor(t * k) * n` makes corruption step on the grid instead of
crawling, which both reads better and stays below the flash rate (see
`scan-tear.js`, `block-mosh.js`).
4. **Determinism is absolute.** No `Math.random()`, `performance.now()`,
`Date.now()`, `new Date()` — use `hash*`/`fbm` for variation, and let time flow
through `u_time`/`u_frame` only. The lint greps for these.
5. **Set a base image.** A scene that only reads `prev()` is black for the first
frames and fragile under seek. Generate your own field underneath the effect.
6. Don't hardcode saturated `vec3(r,g,b)` literals when you declared a palette —
the lint flags more than two.
## Families
Chosen by section kind in the arc driver — a breakdown never lands on a strobing
glitch scene. The library is at **seven per family** (42 scenes). Check the
current spread before adding another:
```bash
node -e "import('./src/scenes/registry.js').then(({scenes})=>{const b={};for(const m of scenes)(b[m.family]??=[]).push(m.name);console.log(b)})"
```
Depth matters more than it looks: the casting rule in `look/Personality.js`
disqualifies scenes that do not honour the track's signature traits, so the pool
a given track draws from is smaller than the library. Thin traits (`space`,
`shape`) are worth more than thin families.
## Verify
Three rungs, each about ten times cheaper than the next. Climb them in order.
```bash
npm run lint:scenes # ~1s, no browser
```
Static gates: schema and shader agreeing both ways, determinism grep, rate flags,
a declared trait with no evidence in the source, a **dead camera**
(`p = sigCamera(p)` and then nothing reads `p`), `prev()` with no base image, and
loops with a large bound and no early break. A loop whose cost is genuinely fixed
— sampling a curve at a set resolution — can say so with a `// lint: fixed-cost`
comment just above it.
```
http://localhost:5180/checks.html?scene=My%20Scene
```
The per-scene acceptance battery for one scene: schema, renders, animates,
deterministic, distinct from every other scene, param sweep, flash rate, and one
line per declared trait proving the image actually responds to it. Ten lines and
a verdict — this is the loop to stay in while writing.
```
http://localhost:5180/checks.html?slow=1
```
Everything. Phases 2, 5 and 7 iterate the registry so a new scene is covered
automatically; Phases 8-10 cover how the look generator uses it. Run this once
before committing.