# flow-state

Ambient/EDM music video generator. Drop in a track, get a full-length, non-story,
music-reactive video. No sourced footage — every frame is generated, and the whole
look is derived from the audio.

```bash
npm install
npm run dev          # http://localhost:5180
```

Drop an audio file onto the page (mp3, flac, wav, ogg). Analysis takes a second or
two, then the video is ready to preview and export.

## How it works

The track is decoded and analysed **before the first frame renders**, into a table
with one row per video frame: band energies, onset flux, spectral centroid and
flatness, a phase-locked beat grid, section boundaries, and lookahead fields.

Nothing reads a live `AnalyserNode`. Realtime preview maps `audio.currentTime` to a
frame index; export counts frames. Both read the same rows, so **what you preview is
what you export** — the exporter has no render path of its own.

Analysing the whole track up front also buys the thing a causal analyser cannot do:
a build can *anticipate* its drop and arrive at the transition already at full
tension, instead of reacting once the drop has landed.

## The story

Everything above happens at a moment. On top of it the track gets a **story**: a
plot chosen per song, four **moments** found in the audio (arrival, turn, climax,
resolution), and a position for every section — *which* of the four drops this
is, not just that it is a drop.

That position moves what the video does. The kind's anchor scene is reserved for
the climax rather than opening every section of its kind; the dials ratchet with
each recurrence; the cutting rate follows tension and the resolution holds; the
song's cast arrives over the video instead of being fully stated in the first
shot; and under a *return* plot the outro re-casts what the intro opened on.

The story never overrides the song. Its moments are read off the section
energies, and its effect on parameter bias is bounded so it can decide which
drop this is and never whether a breakdown is one. `D` shows the current plot,
act and journey; the timeline marks the moments. `EPIC-4.md` has the design.


## Working with it

| | |
|---|---|
| `space` | play / pause |
| `←` `→` | previous / next section boundary |
| `L` | loop the current section |
| `D` | debug HUD |
| `O` | toggle the corner title plate |
| `,` `.` | step one frame |

**test render** exports 20 seconds around the playhead at full export quality. Use
it before committing to a full render.

**reroll** re-seeds the whole track; **reroll section** changes only the section
under the playhead; **lock** protects a section from further rerolls. Every
parameter the generator chose is exposed under the *scene* tab and can be edited
live.

The **metronome** button (look tab) mixes an audible click onto the detected beat
grid. If the clicks don't sit on the beat, tempo detection is wrong and everything
downstream inherits it — check this first when a track looks off.

## Checks

```bash
npm test             # audio pipeline against synthetic ground truth
npm run lint:scenes  # determinism grep + scene schema/shader agreement
```

`http://localhost:5180/filmstrip.html` renders every song in the bank as one frame
every 30 seconds, so a video that is busy and going nowhere is visible as a row of
interchangeable stills. `http://localhost:5180/checks.html` runs the GPU gates for
every phase. Add
`?slow=1` for the full suite, `?phase=5` for one phase.

### Seed variety test

Every other phase asks whether one video is correct. Phase 12 asks whether two
videos are *different* — the one failure the rest of the suite cannot see, since
a generator that ignores its seed passes determinism and liveness perfectly.

```
http://localhost:5180/checks.html?variety=1
```

Frames are reduced to a structural descriptor that is deliberately blind to
brightness, contrast, hue and rotation, and sensitive to feature scale,
orientation structure, composition, texture statistics and motion. Colour is
measured but never counted — its job is to expose the case where two seeds
differ only in palette. The score sits between two references the same
instrument produced: the **floor** is how far one video travels from itself
across its own sections, the **ceiling** is the same pipeline with every layer
recast at random. Two checks gate the instrument itself before any number from
it is trusted.

The report also prints **direction**: the rank correlation between how far apart two
probes are in time and how far apart they look. `drift` cannot tell a video that
travels from one that merely keeps changing, and a video with a story raises it on
purpose — so read the two together. A floor that rises *with* direction is an arc; a
floor that rises without one is a shuffle.

The report also sweeps **all** visualizations in the library, every pair, and
names the structural twins — different scenes that are the same look in
different colours, which the per-scene `distinct` gate cannot catch because it
compares raw pixels.

## Adding a scene

Quick walkthrough: `HOWTO-visualizers.md`.

A scene is a shader plus a params block. Everything else — uniform binding, UI
controls, seeded per-track sampling, arc automation — is derived from the schema.

```js
export const myScene = {
    name: 'My Scene',
    family: 'organic',          // flow organic minimal structural geometric glitch
    kind: 'fragment',
    params: {
        density: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_density', bias: 'density' },
        speed:   { type: 'float', range: [0.1, 2], default: 0.5, uniform: 'u_speed', rate: true },
        palette: { type: 'palette', count: 4 },
    },
    reactive: {
        density: { feature: 'bandLow', amount: 0.3 },
    },
    shader: `
vec4 scene(vec2 uv, vec2 p) {
    float t = u_time * u_speed + u_seed;
    return vec4(palRamp(fbm(p * 4.0 + t, 4)), 1.0);
}
`,
};
```

Register it in `src/scenes/registry.js`, then run `npm run lint:scenes` and the
Phase 7 checks. Three rules the linter enforces, each of which has already caused a
real bug here:

- **Anything multiplying `u_time` must be `rate: true`.** Phase is `elapsed × rate`,
  so modulating a rate jumps the phase by `elapsed × delta` — a minute in, a small
  wobble throws the image several whole units between frames. It measured as
  strobing at twice the accessibility limit.
- **Don't reuse a contract uniform name** (`u_width`, `u_time`, `u_seed`, …). It's a
  GLSL redefinition error, and the only symptom is a black frame.
- **Use `pal()` / `palRamp()`**, not hardcoded colours, or the look generator can't
  recolour the scene.

Scenes that composite over a background rather than being one declare
`role: 'accent'`.

## Layout

```
src/
  audio/      decode, STFT analysis, tempo, segmentation, FeatureTrack, metronome
  engine/     Timeline, Renderer, Layer, Compositor, passes, seeded rng, flash safety
  look/       palette (OKLCH), LookGenerator, ArcDriver, Story
  params/     declarative schema, validation, serialisation
  scenes/     the library — shader/ and layers3d/
  export/     WebCodecs exporter
  ui/         preview surface
  checks/     phase gates, run from checks.html
```

`PLAN.md` has the full design and the reasoning behind each gate.

Forked from `party-stage` by copying what was useful, then fully detached — there
are no imports across the directory boundary in either direction.