9d15c3cf49
Ten new scenes, with 'minimal' first: the family was empty, so intros and breakdowns fell through to flow/organic and every track opened at full density. Quiet sections now land on a restful family 48/48 times across 24 seeds, 28 of them minimal. New: Horizon Lines, Spectrum Sculpture, Slow Orb (minimal); Curl Flow (flow); Plasma Bloom, Metaballs (organic); Kaleido Tunnel, Moiré Grid (geometric); Ridge Terrain (structural); Scan Tear (glitch). Four real bugs, three of which the existing gates could not have caught: 1. SHADER PROGRAMS LINK ASYNCHRONOUSLY. three.js uses KHR_parallel_shader_compile, so draws against an unlinked program render wrong. The heaviest scene had its first TEN frames differ from every later render of the same frames. Preview hides this entirely; export renders each frame once, so those frames would ship broken. Added Engine.prime() — WebGLRenderer.compile() plus a discarded warm frame — and the exporter now primes before encoding. Rendering a throwaway frame and reading it back is NOT sufficient; measured, it left 3-5 frames wrong. 2. Moiré Grid declared a param on u_width, which the shader contract already uses for stereo width. GLSL redefinition, and the only symptom was a black frame. Lint now rejects any param uniform colliding with the contract. 3. Spectrum Sculpture strobed at 4 flashes/s. Two causes: rotation measured in turns meant bar-crossing frequency was bars x rate (82 bars put a slow-looking 0.12 turns/s at 10 Hz), and hard band-tier boundaries made every bar switch band simultaneously. Rotation is now in segment units so the rate IS the crossing frequency, bands interpolate, and the range is capped where the flash meter measures zero. 4. Particle Field was being chosen as a primary background despite being mostly empty by design. Scenes now declare role: 'accent'; those are never primary and are judged on variance rather than luminance. Three checks were themselves wrong and were rebuilt: mean-distance metrics unfairly fail sparse scenes for being tasteful rather than static, so "animates" and "no duplicates" now use max channel delta. PLAN.md §1 gains two refinements: programs must be primed before the first frame, and even same-machine the heaviest shaders vary by one LSB under differing GPU load — so the per-scene criterion is max delta <= 1 rather than an identical hash. A real bug scores in the tens there. Full suite 67/67 across all seven phases. Worst 4K frame 3.8ms, worst flash rate 0/s, worst determinism delta 1/255. Adds README.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
122 lines
4.5 KiB
Markdown
122 lines
4.5 KiB
Markdown
# flow-state
|
||
|
||
Ambient/EDM music video generator. Drop in a track, get a full-length, non-story,
|
||
music-reactive video. No sourced footage — every frame is generated, and the whole
|
||
look is derived from the audio.
|
||
|
||
```bash
|
||
npm install
|
||
npm run dev # http://localhost:5180
|
||
```
|
||
|
||
Drop an audio file onto the page (mp3, flac, wav, ogg). Analysis takes a second or
|
||
two, then the video is ready to preview and export.
|
||
|
||
## How it works
|
||
|
||
The track is decoded and analysed **before the first frame renders**, into a table
|
||
with one row per video frame: band energies, onset flux, spectral centroid and
|
||
flatness, a phase-locked beat grid, section boundaries, and lookahead fields.
|
||
|
||
Nothing reads a live `AnalyserNode`. Realtime preview maps `audio.currentTime` to a
|
||
frame index; export counts frames. Both read the same rows, so **what you preview is
|
||
what you export** — the exporter has no render path of its own.
|
||
|
||
Analysing the whole track up front also buys the thing a causal analyser cannot do:
|
||
a build can *anticipate* its drop and arrive at the transition already at full
|
||
tension, instead of reacting once the drop has landed.
|
||
|
||
## Working with it
|
||
|
||
| | |
|
||
|---|---|
|
||
| `space` | play / pause |
|
||
| `←` `→` | previous / next section boundary |
|
||
| `L` | loop the current section |
|
||
| `D` | debug HUD |
|
||
| `,` `.` | step one frame |
|
||
|
||
**test render** exports 20 seconds around the playhead at full export quality. Use
|
||
it before committing to a full render.
|
||
|
||
**reroll** re-seeds the whole track; **reroll section** changes only the section
|
||
under the playhead; **lock** protects a section from further rerolls. Every
|
||
parameter the generator chose is exposed under the *scene* tab and can be edited
|
||
live.
|
||
|
||
The **click track** button (look tab) mixes an audible click onto the detected beat
|
||
grid. If the clicks don't sit on the beat, tempo detection is wrong and everything
|
||
downstream inherits it — check this first when a track looks off.
|
||
|
||
## Checks
|
||
|
||
```bash
|
||
npm test # audio pipeline against synthetic ground truth
|
||
npm run lint:scenes # determinism grep + scene schema/shader agreement
|
||
```
|
||
|
||
`http://localhost:5180/checks.html` runs the GPU gates for every phase. Add
|
||
`?slow=1` for the full suite, `?phase=5` for one phase.
|
||
|
||
## Adding a scene
|
||
|
||
A scene is a shader plus a params block. Everything else — uniform binding, UI
|
||
controls, seeded per-track sampling, arc automation — is derived from the schema.
|
||
|
||
```js
|
||
export const myScene = {
|
||
name: 'My Scene',
|
||
family: 'organic', // flow organic minimal structural geometric glitch
|
||
kind: 'fragment',
|
||
params: {
|
||
density: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_density', bias: 'density' },
|
||
speed: { type: 'float', range: [0.1, 2], default: 0.5, uniform: 'u_speed', rate: true },
|
||
palette: { type: 'palette', count: 4 },
|
||
},
|
||
reactive: {
|
||
density: { feature: 'bandLow', amount: 0.3 },
|
||
},
|
||
shader: `
|
||
vec4 scene(vec2 uv, vec2 p) {
|
||
float t = u_time * u_speed + u_seed;
|
||
return vec4(palRamp(fbm(p * 4.0 + t, 4)), 1.0);
|
||
}
|
||
`,
|
||
};
|
||
```
|
||
|
||
Register it in `src/scenes/registry.js`, then run `npm run lint:scenes` and the
|
||
Phase 7 checks. Three rules the linter enforces, each of which has already caused a
|
||
real bug here:
|
||
|
||
- **Anything multiplying `u_time` must be `rate: true`.** Phase is `elapsed × rate`,
|
||
so modulating a rate jumps the phase by `elapsed × delta` — a minute in, a small
|
||
wobble throws the image several whole units between frames. It measured as
|
||
strobing at twice the accessibility limit.
|
||
- **Don't reuse a contract uniform name** (`u_width`, `u_time`, `u_seed`, …). It's a
|
||
GLSL redefinition error, and the only symptom is a black frame.
|
||
- **Use `pal()` / `palRamp()`**, not hardcoded colours, or the look generator can't
|
||
recolour the scene.
|
||
|
||
Scenes that composite over a background rather than being one declare
|
||
`role: 'accent'`.
|
||
|
||
## Layout
|
||
|
||
```
|
||
src/
|
||
audio/ decode, STFT analysis, tempo, segmentation, FeatureTrack, click track
|
||
engine/ Timeline, Renderer, Layer, Compositor, passes, seeded rng, flash safety
|
||
look/ palette (OKLCH), LookGenerator, ArcDriver
|
||
params/ declarative schema, validation, serialisation
|
||
scenes/ the library — shader/ and layers3d/
|
||
export/ WebCodecs exporter
|
||
ui/ preview surface
|
||
checks/ phase gates, run from checks.html
|
||
```
|
||
|
||
`PLAN.md` has the full design and the reasoning behind each gate.
|
||
|
||
Forked from `party-stage` by copying what was useful, then fully detached — there
|
||
are no imports across the directory boundary in either direction.
|