music-video-gen/flow-state/README.md

# flow-state

Ambient/EDM music video generator. Drop in a track, get a full-length, non-story,
music-reactive video. No sourced footage — every frame is generated, and the whole
look is derived from the audio.

```bash
npm install
npm run dev          # http://localhost:5180
```

Drop an audio file onto the page (mp3, flac, wav, ogg). Analysis takes a second or
two, then the video is ready to preview and export.

## How it works

The track is decoded and analysed **before the first frame renders**, into a table
with one row per video frame: band energies, onset flux, spectral centroid and
flatness, a phase-locked beat grid, section boundaries, and lookahead fields.

Nothing reads a live `AnalyserNode`. Realtime preview maps `audio.currentTime` to a
frame index; export counts frames. Both read the same rows, so **what you preview is
what you export** — the exporter has no render path of its own.

Analysing the whole track up front also buys the thing a causal analyser cannot do:
a build can *anticipate* its drop and arrive at the transition already at full
tension, instead of reacting once the drop has landed.

## Working with it

| | |
|---|---|
| `space` | play / pause |
| `←` `→` | previous / next section boundary |
| `L` | loop the current section |
| `D` | debug HUD |
| `O` | toggle the corner title plate |
| `,` `.` | step one frame |

**test render** exports 20 seconds around the playhead at full export quality. Use
it before committing to a full render.

**reroll** re-seeds the whole track; **reroll section** changes only the section
under the playhead; **lock** protects a section from further rerolls. Every
parameter the generator chose is exposed under the *scene* tab and can be edited
live.

The **click track** button (look tab) mixes an audible click onto the detected beat
grid. If the clicks don't sit on the beat, tempo detection is wrong and everything
downstream inherits it — check this first when a track looks off.

## Checks

```bash
npm test             # audio pipeline against synthetic ground truth
npm run lint:scenes  # determinism grep + scene schema/shader agreement
```

`http://localhost:5180/checks.html` runs the GPU gates for every phase. Add
`?slow=1` for the full suite, `?phase=5` for one phase.

### Seed variety test

Every other phase asks whether one video is correct. Phase 12 asks whether two
videos are *different* — the one failure the rest of the suite cannot see, since
a generator that ignores its seed passes determinism and liveness perfectly.

```
http://localhost:5180/checks.html?variety=1
```

Frames are reduced to a structural descriptor that is deliberately blind to
brightness, contrast, hue and rotation, and sensitive to feature scale,
orientation structure, composition, texture statistics and motion. Colour is
measured but never counted — its job is to expose the case where two seeds
differ only in palette. The score sits between two references the same
instrument produced: the **floor** is how far one video travels from itself
across its own sections, the **ceiling** is the same pipeline with every layer
recast at random. Two checks gate the instrument itself before any number from
it is trusted.

The report also sweeps **all** visualizations in the library, every pair, and
names the structural twins — different scenes that are the same look in
different colours, which the per-scene `distinct` gate cannot catch because it
compares raw pixels.

## Adding a scene

Quick walkthrough: `HOWTO-visualizers.md`.

A scene is a shader plus a params block. Everything else — uniform binding, UI
controls, seeded per-track sampling, arc automation — is derived from the schema.

```js
export const myScene = {
    name: 'My Scene',
    family: 'organic',          // flow organic minimal structural geometric glitch
    kind: 'fragment',
    params: {
        density: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_density', bias: 'density' },
        speed:   { type: 'float', range: [0.1, 2], default: 0.5, uniform: 'u_speed', rate: true },
        palette: { type: 'palette', count: 4 },
    },
    reactive: {
        density: { feature: 'bandLow', amount: 0.3 },
    },
    shader: `
vec4 scene(vec2 uv, vec2 p) {
    float t = u_time * u_speed + u_seed;
    return vec4(palRamp(fbm(p * 4.0 + t, 4)), 1.0);
}
`,
};
```

Register it in `src/scenes/registry.js`, then run `npm run lint:scenes` and the
Phase 7 checks. Three rules the linter enforces, each of which has already caused a
real bug here:

- **Anything multiplying `u_time` must be `rate: true`.** Phase is `elapsed × rate`,
  so modulating a rate jumps the phase by `elapsed × delta` — a minute in, a small
  wobble throws the image several whole units between frames. It measured as
  strobing at twice the accessibility limit.
- **Don't reuse a contract uniform name** (`u_width`, `u_time`, `u_seed`, …). It's a
  GLSL redefinition error, and the only symptom is a black frame.
- **Use `pal()` / `palRamp()`**, not hardcoded colours, or the look generator can't
  recolour the scene.

Scenes that composite over a background rather than being one declare
`role: 'accent'`.

## Layout

```
src/
  audio/      decode, STFT analysis, tempo, segmentation, FeatureTrack, click track
  engine/     Timeline, Renderer, Layer, Compositor, passes, seeded rng, flash safety
  look/       palette (OKLCH), LookGenerator, ArcDriver
  params/     declarative schema, validation, serialisation
  scenes/     the library — shader/ and layers3d/
  export/     WebCodecs exporter
  ui/         preview surface
  checks/     phase gates, run from checks.html
```

`PLAN.md` has the full design and the reasoning behind each gate.

Forked from `party-stage` by copying what was useful, then fully detached — there
are no imports across the directory boundary in either direction.