14d1204e82
Grain was in every video. It was added twice unconditionally — every scene called sigGrain, and the grade added its own on top — so the only thing that varied between two tracks was how much of it there was. That makes grain the renderer's fingerprint rather than a decision about one video. It is now described rather than dialled (look/grain.js): a mode (off / constant / swell / sections / transient), a cell size in pixels, a refresh rate in frames, a mask (uniform, shadows, highlights, edges, bands) and a chroma amount. Roughly 45% of tracks get none at all. The non-constant modes carry a per-frame envelope computed in Show._postAt from frame and features only, so preview and export still agree. Scene-side grain is gated the same way, and a module can decline it outright with `texture: 0` — crisp line work should stay crisp. The post tab grew a real grain block so any of it can be forced per track. Slow songs got fast scenes. `motion` bias was mostly section energy with tempo as a small correction, so a 70bpm track's drop asked for nearly as much speed as a 150bpm one. Motion is now tempo-dominated, and every `rate: true` param is additionally scaled by a per-track rateScale — measured, 84bpm now samples its rate params at 0.276 of range against 148bpm's 0.571. Parameter sampling also commits harder: extremity starts at 0.45 rather than 0.25 and shapes the draw more aggressively. This was first pushed to 0.82 and backed off to 0.72, because the gates caught the overshoot — seeds began collapsing onto the same range ends and a sparse scene sampled at its low end rendered effectively black. Fallout worth recording: turning the default grade grain off exposed two scenes that were never really animating. Dust Chamber and Eclipse Field passed the Phase 7 movement gate only because per-pixel noise was moving underneath them; both now breathe on their own fixed clock, and Dust Chamber needed a brightness floor as well. Four scenes (Classic Wave, Silk Ribbon, Kaleido Tunnel, Slow Orb) express the style trait ONLY through grain and so cannot opt out yet; they hold a reduced share at 0.35 pending real edge and softness response. Phase 3's look-space check now measures its closest pair relative to image brightness, the same correction Phase 10 already documents for sparse scenes: the absolute number was being propped up by grain rather than by look-space width. Five new Phase 10 checks cover grain distribution, treatment variety, envelope range, the texture opt-out, and tempo. 93/93 pass. Two transport bugs, both stale state surfacing in the UI: - Loading a track replaces audio.src, which stops playback silently, so state.playing stayed true and the play button stayed on pause — the first click after a track change only flipped the flag back. Added stopPlayback(). - The controls row wrapped mid-song because two readouts that change length while playing were sized by their content: the clock crossing ten minutes and the section label whenever a scene name is long or a crossfade appears. The clock is now fixed-width with tabular figures and the section label is the row's only flexible item, laying out at zero width and ellipsising into whatever space is left. The two spacers competed with it for that space and are gone; its own text-align does their job. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
195 lines
8.6 KiB
Markdown
195 lines
8.6 KiB
Markdown
# How to build a visualizer (scene)
|
||
|
||
> **Start here:** `npm run new:scene -- "My Scene" --family=glitch --traits=camera,style`
|
||
>
|
||
> That writes the module, registers it, and leaves a skeleton that already passes
|
||
> every gate. Then write the `scene()` body, `npm run lint:scenes`, and open
|
||
> `checks.html?scene=My%20Scene`. The rest of this file is the reference.
|
||
>
|
||
> There is also a repo skill — `.claude/skills/build-visualizer/` — which is the
|
||
> same procedure in the form an agent will follow.
|
||
|
||
A scene is **a fragment shader plus a params block** — nothing else. Uniform
|
||
binding, the generated UI sliders, seeded per-track sampling, arc drift and the
|
||
phase-gate checks are all derived from the schema. There is no per-scene wiring.
|
||
|
||
Reference: `src/scenes/shader/kaleido-tunnel.js` (simple), `block-mosh.js`
|
||
(feedback/glitch), `scan-tear.js` (beat-quantised), `metaballs.js` (loops).
|
||
Read these before writing anything.
|
||
|
||
## The shape of a scene
|
||
|
||
```js
|
||
export const myScene = {
|
||
name: 'My Scene',
|
||
family: 'glitch', // flow | organic | minimal | structural | geometric | glitch
|
||
kind: 'fragment',
|
||
traits: ['camera', 'style'], // which personality traits it honours (see below)
|
||
|
||
params: {
|
||
density: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_density', bias: 'density' },
|
||
speed: { type: 'float', range: [0.1, 2], default: 0.5, uniform: 'u_speed', rate: true },
|
||
palette: { type: 'palette', count: 4 },
|
||
},
|
||
|
||
reactive: {
|
||
density: { feature: 'bandLow', amount: 0.3, response: 'smooth' },
|
||
},
|
||
|
||
shader: `
|
||
vec4 scene(vec2 uv, vec2 p) {
|
||
float t = u_time * u_speed + u_seed;
|
||
return vec4(palRamp(fbm(p * 4.0 + t, 4)), 1.0);
|
||
}
|
||
`,
|
||
};
|
||
```
|
||
|
||
The scaffolder writes all of that for you, including the registry import and
|
||
entry, and bakes name-derived constants into the skeleton field so two freshly
|
||
scaffolded scenes are not identical to each other. To do it by hand: write the
|
||
file and add the import + `MODULES` entry in `src/scenes/registry.js`.
|
||
|
||
## The shader contract
|
||
|
||
You write one function: `vec4 scene(vec2 uv, vec2 p)`.
|
||
|
||
- `uv` is 0..1 across the frame; `p` is centred, aspect-corrected, ~-1..1 on the
|
||
short axis. **Work in these, never pixels** — scale pixel-sized things by
|
||
`u_pixelScale` so a 720p preview matches a 4K export.
|
||
- The preamble is injected for you. Never declare `main()`, `u_resolution`,
|
||
`vUv`, or any contract uniform yourself.
|
||
|
||
**Uniforms available** (see `engine/shader-contract.js`):
|
||
|
||
- Frame: `u_time`, `u_frame`, `u_progress`, `u_seed`, `u_resolution`, `u_aspect`,
|
||
`u_pixelScale`, `u_opacity`.
|
||
- Audio, filled per frame from the FeatureTrack: `u_loudness`, `u_rms`,
|
||
`u_bandSub/low/mid/high/air`, `u_flux`, `u_centroid`, `u_flatness`, `u_width`,
|
||
`u_beat`, `u_beatPhase`, `u_barPhase`, `u_phrasePhase`, `u_sectionProgress`,
|
||
`u_sectionEnergy`, `u_buildSlope`.
|
||
- Personality (`u_sigSides`, `u_sigDrift`, `u_sigLine`, …) — see below.
|
||
- Feedback: `u_prev` sampler, read with `prev(vec2 uv)` (returns the previous
|
||
frame's colour; black if none). This is what makes trails/smear/datamosh work
|
||
for free.
|
||
|
||
**Helpers** (already in the preamble):
|
||
|
||
- Palette: `pal(int)`, `palRamp(float)` — always use these, or the look can't
|
||
recolour the scene.
|
||
- Noise: `hash11/12/22`, `vnoise`, `fbm`, `curl`, `rot`, `kaleido`.
|
||
- Personality: `sigShape`/`sigForm`, `sigCamera`, `sigFolded`, `sigEdge`,
|
||
`sigGrain`, `sigHorizonY`, `sigAir`.
|
||
- `sat(x)` = `clamp(x, 0, 1)`.
|
||
|
||
## Param schema fields
|
||
|
||
- `type`: `float` | `int` | `bool` | `vec2` | `palette`.
|
||
- `range` `[min, max]` required for numerics; `default` required unless you want
|
||
range[0] (prefer explicit defaults).
|
||
- `uniform`: the GLSL name. Must be `u_*` and must not collide with the contract
|
||
(see "traps").
|
||
- `bias`: which track-character axis nudges sampling — `energy`, `density`,
|
||
`motion`. This is how a loud drop gets denser scenes without the scene knowing
|
||
about audio.
|
||
- `rate: true`: **mandatory if the shader multiplies `u_time` by this param.**
|
||
- `reactive`: `{ feature, amount, response }`. `response`: `linear` (default) |
|
||
`spike` | `smooth` | `inverse`.
|
||
|
||
## Personality traits (`traits`)
|
||
|
||
The look generator builds each track on a signature of 1-2 traits, and a scene
|
||
that doesn't honour **all** of them is never cast in that track — so the library
|
||
intentionally shrinks per track. Only declare what you genuinely use:
|
||
|
||
| trait | what to call | lint evidence |
|
||
|---|---|---|
|
||
| `shape` | `sigShape` / `sigForm` | a `sigShape`/`sigForm(` call |
|
||
| `camera` | `sigCamera(p)` | a `sigCamera(` call |
|
||
| `space` | `sigHorizonY` / `sigAir` | those calls or `u_sigHorizon/Depth/Wash` |
|
||
| `style` | `sigEdge` / `sigGrain` / `sigFolded`, or `u_sigLine/Soft/Texture/Fold` | the calls / those uniforms |
|
||
|
||
The lint greps your shader and fails a declared trait with no evidence. Declaring
|
||
`[]` (none) is valid.
|
||
|
||
### `texture` — how much surface grain your scene accepts
|
||
|
||
Optional module field, `0..2`, default `1`. It scales `u_sigTexture` — and so
|
||
every `sigGrain(uv)` in your shader — for this scene only. Set it to `0` if your
|
||
scene is crisp line work that grain only furs up, or to something under 1 if it
|
||
should be dusted rather than dirty. It has nothing to do with the *grade's*
|
||
grain, which is a per-track treatment (see `look/grain.js`) and is off entirely
|
||
for most tracks.
|
||
|
||
## Traps that have actually bitten here
|
||
|
||
1. **Anything multiplying `u_time` must be `rate: true`.** Phase is
|
||
`elapsed × rate`; modulating a rate jumps the phase by `elapsed × Δrate` — a
|
||
minute in a small wobble throws the image several units between frames, which
|
||
measured as strobing at 2× the accessibility limit. So don't react a rate
|
||
param, and add a *bounded* term instead: `u_time * u_speed + u_bandLow * 2.0`
|
||
is fine; `u_time * (u_speed + u_bandLow)` is not.
|
||
2. **Never reuse a contract uniform name** (`u_width`, `u_time`, `u_seed`, …).
|
||
It's a GLSL redefinition error; the only symptom is a black frame.
|
||
3. **Don't modulate whole-frame luminance on the beat.** A per-kick min→max→min
|
||
cycle is exactly what the WCAG 2.3.1 / Harding 3-flashes-per-second ceiling
|
||
bans. Pulse a *small local* term (per-block tint) instead, and use `smooth`
|
||
responses on loud things. If it's glitchy, **quantise it** — `floor(u_barPhase
|
||
* n) + floor(t * k) * n` makes corruption step on the grid instead of
|
||
crawling, which both reads better and stays below the flash rate (see
|
||
`scan-tear.js`, `block-mosh.js`).
|
||
4. **Determinism is absolute.** No `Math.random()`, `performance.now()`,
|
||
`Date.now()`, `new Date()` — use `hash*`/`fbm` for variation, and let time flow
|
||
through `u_time`/`u_frame` only. The lint greps for these.
|
||
5. **Set a base image.** A scene that only reads `prev()` is black for the first
|
||
frames and fragile under seek. Generate your own field underneath the effect.
|
||
6. Don't hardcode saturated `vec3(r,g,b)` literals when you declared a palette —
|
||
the lint flags more than two.
|
||
|
||
## Families
|
||
|
||
Chosen by section kind in the arc driver — a breakdown never lands on a strobing
|
||
glitch scene. The library is at **seven per family** (42 scenes). Check the
|
||
current spread before adding another:
|
||
|
||
```bash
|
||
node -e "import('./src/scenes/registry.js').then(({scenes})=>{const b={};for(const m of scenes)(b[m.family]??=[]).push(m.name);console.log(b)})"
|
||
```
|
||
|
||
Depth matters more than it looks: the casting rule in `look/Personality.js`
|
||
disqualifies scenes that do not honour the track's signature traits, so the pool
|
||
a given track draws from is smaller than the library. Thin traits (`space`,
|
||
`shape`) are worth more than thin families.
|
||
|
||
## Verify
|
||
|
||
Three rungs, each about ten times cheaper than the next. Climb them in order.
|
||
|
||
```bash
|
||
npm run lint:scenes # ~1s, no browser
|
||
```
|
||
|
||
Static gates: schema and shader agreeing both ways, determinism grep, rate flags,
|
||
a declared trait with no evidence in the source, a **dead camera**
|
||
(`p = sigCamera(p)` and then nothing reads `p`), `prev()` with no base image, and
|
||
loops with a large bound and no early break. A loop whose cost is genuinely fixed
|
||
— sampling a curve at a set resolution — can say so with a `// lint: fixed-cost`
|
||
comment just above it.
|
||
|
||
```
|
||
http://localhost:5180/checks.html?scene=My%20Scene
|
||
```
|
||
|
||
The per-scene acceptance battery for one scene: schema, renders, animates,
|
||
deterministic, distinct from every other scene, param sweep, flash rate, and one
|
||
line per declared trait proving the image actually responds to it. Ten lines and
|
||
a verdict — this is the loop to stay in while writing.
|
||
|
||
```
|
||
http://localhost:5180/checks.html?slow=1
|
||
```
|
||
|
||
Everything. Phases 2, 5 and 7 iterate the registry so a new scene is covered
|
||
automatically; Phases 8-10 cover how the look generator uses it. Run this once
|
||
before committing.
|