diff --git a/flow-state/HOWTO-variety.md b/flow-state/HOWTO-variety.md new file mode 100644 index 0000000..4f3a224 --- /dev/null +++ b/flow-state/HOWTO-variety.md @@ -0,0 +1,249 @@ +# How to build a visualizer that varies + +`HOWTO-visualizers.md` covers making a scene that *works*. This one covers making +a scene that looks different from one song to the next — a separate skill, and +the one the library is currently worst at. + +Everything here is a measurement rather than an opinion, and the numbers are +quoted so a future change can contradict them. Several already contradict things +believed earlier in the same week. Living document: add what you learn, and +delete what stops being true. + +--- + +## The one-line version + +> A scene varies when the SONG can change what is on screen. It does not vary +> because its parameters moved. + +Parameters mostly change the same picture. Content changes the picture. + +--- + +## How to know if you succeeded + +``` +gallery.html your scene, six times, on six songs — with a score +checks.html?scene=X the per-scene gate, including `consumes:` lines +``` + +The gallery score is the mean structural distance between a scene's own six +frames, on the same descriptor the variety harness uses. **Minimum bar: 0.04** +(`MIN_VARIETY` in `checks/gallery.js`), drawn as a red line. Below it a scene is +the same picture wherever it appears, and because it is cast into many songs +that sameness leaks between them — the viewer recognises the shot rather than +the song. + +Some reference points, measured: + +``` +0.219 Droste Feedback varies a lot +0.181 Spectrum Sculpture +0.049 Isometric Blocks after work — see below +0.039 Apollonian Gasket beautiful, and the same six times +0.031 Moiré Grid the same picture in six palettes +``` + +--- + +## What the descriptor can and cannot see + +Design against this, because half of "why is my score low" is here. + +**It is blind to:** + +- **Brightness and contrast.** Frames are standardised to zero mean and unit + variance before anything is measured. A scene that only gets brighter has not + changed. +- **Colour.** Measured, reported, and excluded from the score. Six palettes + cannot disguise one image, which was the entire point. +- **Rotation**, on the scale, orientation and texture blocks. +- **Quality.** It measures change between songs, nothing else. Apollonian Gasket + scores 0.039 and looks great. Both facts are true and neither implies the + other. + +**It sees:** + +| block | what it measures | how to move it | +|---|---|---| +| `scale` | feature size — fine grain vs large forms | let the song set element size (`stageScale()`) | +| `orient` | grid vs radial vs stripes, rotation-blind | break or vary a regular lattice | +| `layout` | where in the frame structure sits | move the subject, change what is empty | +| `texture` | element count, sparsity, mirror and radial symmetry | vary how many things there are | +| `motion` | what moves and where, not how much | vary the KIND of movement, not the rate | + +**A known blind spot.** `layout` is nearly useless for full-frame fields. Voronoi +Shatter measures 0.004 there no matter what changes, because edge-to-edge cells +occupy the same frame however they fall. That is honest — there genuinely is no +arrangement — but it means a whole family is judged on four blocks instead of +five. If your scene fills the frame corner to corner, expect to earn your score +on `orient` and `texture`. + +--- + +## What has actually worked + +Ordered by measured effect. + +### 1. Draw the song's content instead of your own + +The largest single lever. Take `consumes: ['cast', 'ink', 'staging']` and draw +`castMain`/`castChorus` where you would have drawn your own primitive. Recipe in +`MIGRATION.md`. + +Measured: identity is worth 117% of what the container is worth +(`checks.html?decompose=1`) — swapping the song's cast under a fixed scene moves +the picture more than swapping the scene under a fixed cast. A migrated library +scene also carries the identity better than a stage written from scratch to +carry it, which was a surprise and is why the whole library was migrated rather +than replaced. + +### 2. Let the song decide element SIZE + +`stageScale()`. Not the size of your features relative to each other — the size +of the whole vocabulary. A song of six huge forms and a song of four hundred +tiny ones are different videos before anything else is decided. + +Measured: adding `elementScale` to the identity moved stages from ++0.0111 ±0.0043 to +0.0143 ±0.0008, and the `scale` block from 0.033 to 0.041. + +### 3. Displace a regular lattice + +If your scene is a grid, let the song push things off it. Bounded — past about +half a cell the structure the grid was providing stops reading. + +Measured: Isometric Blocks 0.034 → 0.049, with `orient` nearly doubling from +0.093 to 0.164 as the rigid lattice softened. + +### 4. Give a full-frame field somewhere to be about + +`focusWarp(p, amount)` and `focusField(p)`. The song picks one to three focal +points; tile in the warped coordinate so cells crowd toward them or pull away. + +Measured: Voronoi Shatter 0.0463 → 0.0588. **Note what moved:** `orient` 0.140 → +0.194, while `layout` went 0.004 → 0.007, i.e. nowhere. Warping where cells sit +changes what they look like without changing where the frame's energy is. The +gain is real; the stated reason for it was wrong. + +--- + +## What has measurably NOT worked + +Recorded because the failures were more informative than the wins, and because +each of these looked obviously right beforehand. + +### Effects as parameters + +Forty-nine scenes had their own `glow`; every one had its own grain. An effect +knob sampled per song makes a scene look varied across parameter draws while its +structure never moves — it inflates the score without changing the picture, and +it fights the grade, which already does bloom, grain, chroma and vignette with +an envelope the scene cannot see. + +**Do not add glow, bloom, grain, haze, chroma or trails to a scene.** The post +chain owns them. `npm run lint:scenes` does not catch this yet; reviewers should. + +### Concentrating disturbance instead of adding it + +Isometric Blocks, twice. Spending the scatter budget near the focal points and +calming the rest measured 0.049 → 0.0374 — most of the field went back onto the +rigid lattice and took the orientation variety with it. Making the focus +additive instead recovered nothing: 0.0365. + +Two plausible diagnoses, both wrong. The lesson is narrow and worth keeping: +**a focal point has to be something the field gains, not something the rest of it +pays for** — and even that framing did not rescue it, so the real cause is still +unknown. + +### Making the roster smaller + +A whole afternoon went into the theory that a track drawing on fewer scenes +would look more like itself. Swept directly across pool sizes 4, 8, 16 and 32 +over twelve songs with three draws each: differences of 0.003 to 0.007 against a +run-to-run noise of ±0.003 to ±0.005. **Nothing.** The theory came from an +experiment whose arms differed in two ways at once. + +### Reading a form as a metric rather than drawing it + +Spectrum Sculpture used `sigShape` as a radial distance rather than as a subject. +Migrating it to `castMain` rendered pure black, because the cast carries notches +and a hollow and an annulus used as a radius turns a sculpture inside out. If +your scene consults a form's geometry rather than drawing it, it wants +`sigShape`, and the cast is not for you. + +--- + +## Scene shapes, and how hard each is to vary + +- **Element-placing** (`metaballs`, `floating-geometry`, `firefly-drift`) — + easiest. Take the cast, place on `stageNode`, keep your motion. Nearly all the + wins above are this shape. +- **Single figure** (`slow-orb`, `eclipse-field`) — take the cast and + `stageScale()`. Composition is yours to vary; most such scenes centre their + subject and never move it, which is free `layout` left on the table. +- **Full-frame field** (`voronoi-shatter`, `plasma-bloom`, `curl-flow`) — hard. + `layout` is unavailable to you. Earn it on `orient` and `texture`, and consider + a focus. +- **Fixed geometry** (`truchet-fold`, `quasicrystal`, `apollonian-gasket`) — + hardest, and currently unsolved. The image *is* the maths, so the cast cannot + be pasted on top of it. The open idea is to make the cast the repeating UNIT + the tiling is built from, which is a real rewrite per scene rather than a + recipe step. Nobody has tried it yet. +- **Treatment** (`analog-wow`, `scan-tear`, `halftone-misprint`) — these are + effects wearing a scene's clothes. They score 0.031–0.040 and their whole + migration was one posterisation. They probably belong in the identity's + effects register rather than competing for screen time as subjects. + +--- + +## Trusting the numbers + +This harness has two classes of measurement and only one is safe to steer by. + +**Direct render comparisons** — the gallery score, `checks.html?decompose=1`, the +per-scene gate — compare rendered frames to each other. Noise floor of exactly +zero. Trust these. + +**Aggregate ratios** — chiefly `separation`, which divides one small difference +by another — swing wildly. Three runs of the same post-migration measurement gave +0.31, 0.56 and 0.07. **Four conclusions were drawn and withdrawn during this epic +for exactly that reason.** Anything under about 0.01 of spread needs +`&repeats=3` before it is believed, and a difference that changes sign with the +sample size is not a difference. + +Two more traps, both of which have bitten: + +- **A permanently-zero block looks like a property of your scene.** The gallery + built four of the descriptor's five blocks for weeks; `motion` came back 0.000 + for every scene and depressed every score by a fifth. If a block is identical + across many unrelated scenes, suspect the instrument. +- **Arms that differ in more than one way cannot be read as if they differed in + one.** This produced the roster-size result above, and it was believed for a + day. + +--- + +## Checklist for a new or reworked scene + +1. `consumes` declared, and the `consumes:` lines pass in + `checks.html?scene=`. A declared artifact that is ignored is worse than + one not declared, because the casting code believes it. +2. No glow, bloom, grain, haze, chroma or trail parameters. +3. Element size goes through `stageScale()`. +4. Placement goes through `stageNode()` unless the composition IS the scene. +5. Gallery score above 0.04, checked before and after. +6. The rest of the per-scene battery still passes — especially `distinct`. The + more scenes share a cast, the easier it is to become one of your neighbours. + +--- + +## Open questions + +- **Coupling is zero and has never moved.** Whether a song *looks* different in + proportion to how it *sounds* different measures -0.02 ±0.03. Every change in + this epic left it there. Nobody has attacked it directly. +- **Fixed-geometry scenes** have no known route to variety. +- **The `layout` blindness** for full-frame fields would need a new descriptor + block measuring cell statistics rather than where structure sits. +- **The 0.04 bar** is a starting point, set where the flat cluster sat — and set + against scores that were depressed by the motion bug. It deserves re-deriving. diff --git a/flow-state/HOWTO-visualizers.md b/flow-state/HOWTO-visualizers.md index 2e83bc2..1dfae79 100644 --- a/flow-state/HOWTO-visualizers.md +++ b/flow-state/HOWTO-visualizers.md @@ -192,3 +192,13 @@ http://localhost:5180/checks.html?slow=1 Everything. Phases 2, 5 and 7 iterate the registry so a new scene is covered automatically; Phases 8-10 cover how the look generator uses it. Run this once before committing. + +--- + +## Making it VARY + +This document gets a scene working. Making it look different from one song to +the next is a separate skill with its own measurements, its own failures worth +not repeating, and its own gate — the gallery, with a minimum score of 0.04. + +See `HOWTO-variety.md`. diff --git a/flow-state/debug.html b/flow-state/debug.html index e2175a2..a8efd7e 100644 --- a/flow-state/debug.html +++ b/flow-state/debug.html @@ -50,7 +50,8 @@

Every visualizer, six times, on six different songs' content — cast, ink, lattice, palette and parameters all varying. Sorted least-varied first, so the scenes that always look the same come to the top.

-
~3 min · the one to open when a scene feels familiar
+
~3 min · the one to open when a scene feels familiar · + HOWTO-variety.md for how to fix what it finds

the app