10eeddcf6d
HOWTO-visualizers.md gets a scene working. Nothing said how to make one look different from song to song, which is the thing the library is worst at and the thing every measurement this epic has been about. Written as evidence rather than advice: each claim carries the number behind it, so a later change can contradict it. Several already contradict things believed earlier in the same week. It records the failures at least as carefully as the wins, because they were more informative and each looked obviously right beforehand — effects as parameters inflating a score without changing a picture, concentrating disturbance instead of adding it, the roster-size theory that a direct sweep found to be nothing, and reading a form as a metric rather than drawing it. It also documents what the descriptor cannot see, since half of "why is my score low" is there: brightness, colour, rotation, quality, and layout for anything that fills the frame. And it is explicit about which numbers to trust. Direct render comparisons have a noise floor of zero; aggregate ratios swing enough to have produced four withdrawn conclusions in one epic. Ongoing by design, with the open questions listed: coupling has never moved off zero, fixed-geometry scenes have no known route to variety, and the 0.04 bar was set against scores that were depressed by a measurement bug. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
205 lines
8.9 KiB
Markdown
205 lines
8.9 KiB
Markdown
# How to build a visualizer (scene)
|
||
|
||
> **Start here:** `npm run new:scene -- "My Scene" --family=glitch --traits=camera,style`
|
||
>
|
||
> That writes the module, registers it, and leaves a skeleton that already passes
|
||
> every gate. Then write the `scene()` body, `npm run lint:scenes`, and open
|
||
> `checks.html?scene=My%20Scene`. The rest of this file is the reference.
|
||
>
|
||
> There is also a repo skill — `.claude/skills/build-visualizer/` — which is the
|
||
> same procedure in the form an agent will follow.
|
||
|
||
A scene is **a fragment shader plus a params block** — nothing else. Uniform
|
||
binding, the generated UI sliders, seeded per-track sampling, arc drift and the
|
||
phase-gate checks are all derived from the schema. There is no per-scene wiring.
|
||
|
||
Reference: `src/scenes/shader/kaleido-tunnel.js` (simple), `block-mosh.js`
|
||
(feedback/glitch), `scan-tear.js` (beat-quantised), `metaballs.js` (loops).
|
||
Read these before writing anything.
|
||
|
||
## The shape of a scene
|
||
|
||
```js
|
||
export const myScene = {
|
||
name: 'My Scene',
|
||
family: 'glitch', // flow | organic | minimal | structural | geometric | glitch
|
||
kind: 'fragment',
|
||
traits: ['camera', 'style'], // which personality traits it honours (see below)
|
||
|
||
params: {
|
||
density: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_density', bias: 'density' },
|
||
speed: { type: 'float', range: [0.1, 2], default: 0.5, uniform: 'u_speed', rate: true },
|
||
palette: { type: 'palette', count: 4 },
|
||
},
|
||
|
||
reactive: {
|
||
density: { feature: 'bandLow', amount: 0.3, response: 'smooth' },
|
||
},
|
||
|
||
shader: `
|
||
vec4 scene(vec2 uv, vec2 p) {
|
||
float t = u_time * u_speed + u_seed;
|
||
return vec4(palRamp(fbm(p * 4.0 + t, 4)), 1.0);
|
||
}
|
||
`,
|
||
};
|
||
```
|
||
|
||
The scaffolder writes all of that for you, including the registry import and
|
||
entry, and bakes name-derived constants into the skeleton field so two freshly
|
||
scaffolded scenes are not identical to each other. To do it by hand: write the
|
||
file and add the import + `MODULES` entry in `src/scenes/registry.js`.
|
||
|
||
## The shader contract
|
||
|
||
You write one function: `vec4 scene(vec2 uv, vec2 p)`.
|
||
|
||
- `uv` is 0..1 across the frame; `p` is centred, aspect-corrected, ~-1..1 on the
|
||
short axis. **Work in these, never pixels** — scale pixel-sized things by
|
||
`u_pixelScale` so a 720p preview matches a 4K export.
|
||
- The preamble is injected for you. Never declare `main()`, `u_resolution`,
|
||
`vUv`, or any contract uniform yourself.
|
||
|
||
**Uniforms available** (see `engine/shader-contract.js`):
|
||
|
||
- Frame: `u_time`, `u_frame`, `u_progress`, `u_seed`, `u_resolution`, `u_aspect`,
|
||
`u_pixelScale`, `u_opacity`.
|
||
- Audio, filled per frame from the FeatureTrack: `u_loudness`, `u_rms`,
|
||
`u_bandSub/low/mid/high/air`, `u_flux`, `u_centroid`, `u_flatness`, `u_width`,
|
||
`u_beat`, `u_beatPhase`, `u_barPhase`, `u_phrasePhase`, `u_sectionProgress`,
|
||
`u_sectionEnergy`, `u_buildSlope`.
|
||
- Personality (`u_sigSides`, `u_sigDrift`, `u_sigLine`, …) — see below.
|
||
- Feedback: `u_prev` sampler, read with `prev(vec2 uv)` (returns the previous
|
||
frame's colour; black if none). This is what makes trails/smear/datamosh work
|
||
for free.
|
||
|
||
**Helpers** (already in the preamble):
|
||
|
||
- Palette: `pal(int)`, `palRamp(float)` — always use these, or the look can't
|
||
recolour the scene.
|
||
- Noise: `hash11/12/22`, `vnoise`, `fbm`, `curl`, `rot`, `kaleido`.
|
||
- Personality: `sigShape`/`sigForm`, `sigCamera`, `sigFolded`, `sigEdge`,
|
||
`sigGrain`, `sigHorizonY`, `sigAir`.
|
||
- `sat(x)` = `clamp(x, 0, 1)`.
|
||
|
||
## Param schema fields
|
||
|
||
- `type`: `float` | `int` | `bool` | `vec2` | `palette`.
|
||
- `range` `[min, max]` required for numerics; `default` required unless you want
|
||
range[0] (prefer explicit defaults).
|
||
- `uniform`: the GLSL name. Must be `u_*` and must not collide with the contract
|
||
(see "traps").
|
||
- `bias`: which track-character axis nudges sampling — `energy`, `density`,
|
||
`motion`. This is how a loud drop gets denser scenes without the scene knowing
|
||
about audio.
|
||
- `rate: true`: **mandatory if the shader multiplies `u_time` by this param.**
|
||
- `reactive`: `{ feature, amount, response }`. `response`: `linear` (default) |
|
||
`spike` | `smooth` | `inverse`.
|
||
|
||
## Personality traits (`traits`)
|
||
|
||
The look generator builds each track on a signature of 1-2 traits, and a scene
|
||
that doesn't honour **all** of them is never cast in that track — so the library
|
||
intentionally shrinks per track. Only declare what you genuinely use:
|
||
|
||
| trait | what to call | lint evidence |
|
||
|---|---|---|
|
||
| `shape` | `sigShape` / `sigForm` | a `sigShape`/`sigForm(` call |
|
||
| `camera` | `sigCamera(p)` | a `sigCamera(` call |
|
||
| `space` | `sigHorizonY` / `sigAir` | those calls or `u_sigHorizon/Depth/Wash` |
|
||
| `style` | `sigEdge` / `sigGrain` / `sigFolded`, or `u_sigLine/Soft/Texture/Fold` | the calls / those uniforms |
|
||
|
||
The lint greps your shader and fails a declared trait with no evidence. Declaring
|
||
`[]` (none) is valid.
|
||
|
||
### `texture` — how much surface grain your scene accepts
|
||
|
||
Optional module field, `0..2`, default `1`. It scales `u_sigTexture` — and so
|
||
every `sigGrain(uv)` in your shader — for this scene only. Set it to `0` if your
|
||
scene is crisp line work that grain only furs up, or to something under 1 if it
|
||
should be dusted rather than dirty. It has nothing to do with the *grade's*
|
||
grain, which is a per-track treatment (see `look/grain.js`) and is off entirely
|
||
for most tracks.
|
||
|
||
## Traps that have actually bitten here
|
||
|
||
1. **Anything multiplying `u_time` must be `rate: true`.** Phase is
|
||
`elapsed × rate`; modulating a rate jumps the phase by `elapsed × Δrate` — a
|
||
minute in a small wobble throws the image several units between frames, which
|
||
measured as strobing at 2× the accessibility limit. So don't react a rate
|
||
param, and add a *bounded* term instead: `u_time * u_speed + u_bandLow * 2.0`
|
||
is fine; `u_time * (u_speed + u_bandLow)` is not.
|
||
2. **Never reuse a contract uniform name** (`u_width`, `u_time`, `u_seed`, …).
|
||
It's a GLSL redefinition error; the only symptom is a black frame.
|
||
3. **Don't modulate whole-frame luminance on the beat.** A per-kick min→max→min
|
||
cycle is exactly what the WCAG 2.3.1 / Harding 3-flashes-per-second ceiling
|
||
bans. Pulse a *small local* term (per-block tint) instead, and use `smooth`
|
||
responses on loud things. If it's glitchy, **quantise it** — `floor(u_barPhase
|
||
* n) + floor(t * k) * n` makes corruption step on the grid instead of
|
||
crawling, which both reads better and stays below the flash rate (see
|
||
`scan-tear.js`, `block-mosh.js`).
|
||
4. **Determinism is absolute.** No `Math.random()`, `performance.now()`,
|
||
`Date.now()`, `new Date()` — use `hash*`/`fbm` for variation, and let time flow
|
||
through `u_time`/`u_frame` only. The lint greps for these.
|
||
5. **Set a base image.** A scene that only reads `prev()` is black for the first
|
||
frames and fragile under seek. Generate your own field underneath the effect.
|
||
6. Don't hardcode saturated `vec3(r,g,b)` literals when you declared a palette —
|
||
the lint flags more than two.
|
||
|
||
## Families
|
||
|
||
Chosen by section kind in the arc driver — a breakdown never lands on a strobing
|
||
glitch scene. The library is at **seven per family** (42 scenes). Check the
|
||
current spread before adding another:
|
||
|
||
```bash
|
||
node -e "import('./src/scenes/registry.js').then(({scenes})=>{const b={};for(const m of scenes)(b[m.family]??=[]).push(m.name);console.log(b)})"
|
||
```
|
||
|
||
Depth matters more than it looks: the casting rule in `look/Personality.js`
|
||
disqualifies scenes that do not honour the track's signature traits, so the pool
|
||
a given track draws from is smaller than the library. Thin traits (`space`,
|
||
`shape`) are worth more than thin families.
|
||
|
||
## Verify
|
||
|
||
Three rungs, each about ten times cheaper than the next. Climb them in order.
|
||
|
||
```bash
|
||
npm run lint:scenes # ~1s, no browser
|
||
```
|
||
|
||
Static gates: schema and shader agreeing both ways, determinism grep, rate flags,
|
||
a declared trait with no evidence in the source, a **dead camera**
|
||
(`p = sigCamera(p)` and then nothing reads `p`), `prev()` with no base image, and
|
||
loops with a large bound and no early break. A loop whose cost is genuinely fixed
|
||
— sampling a curve at a set resolution — can say so with a `// lint: fixed-cost`
|
||
comment just above it.
|
||
|
||
```
|
||
http://localhost:5180/checks.html?scene=My%20Scene
|
||
```
|
||
|
||
The per-scene acceptance battery for one scene: schema, renders, animates,
|
||
deterministic, distinct from every other scene, param sweep, flash rate, and one
|
||
line per declared trait proving the image actually responds to it. Ten lines and
|
||
a verdict — this is the loop to stay in while writing.
|
||
|
||
```
|
||
http://localhost:5180/checks.html?slow=1
|
||
```
|
||
|
||
Everything. Phases 2, 5 and 7 iterate the registry so a new scene is covered
|
||
automatically; Phases 8-10 cover how the look generator uses it. Run this once
|
||
before committing.
|
||
|
||
---
|
||
|
||
## Making it VARY
|
||
|
||
This document gets a scene working. Making it look different from one song to
|
||
the next is a separate skill with its own measurements, its own failures worth
|
||
not repeating, and its own gate — the gallery, with a minimum score of 0.04.
|
||
|
||
See `HOWTO-variety.md`.
|