Whole-track analysis into a frame-indexed FeatureTrack. Nothing reads a
live AnalyserNode: realtime preview maps currentTime to a frame index,
export counts frames, both read the same rows.
- fft.js: radix-2 with precomputed tables, allocated once per track
- analyze.js: STFT at hop 1/60s with CENTRED windows (a window that starts
at the timestamp reports energy arriving up to 23ms later, which reads as
visuals lagging the music). Energy features normalised against the track's
own percentiles; absolute stats kept in summary for the look generator.
- tempo.js: autocorrelation + grid F-measure, beat grid, downbeats
- segment.js: self-similarity novelty, boundaries snapped to the bar grid
- FeatureTrack: assembles everything, plus the lookahead fields. buildSlope
rises through the bars leading into a higher-energy section, so a build can
ramp into the drop rather than react after it lands.
- clicktrack.js: mixes clicks onto the detected grid for validation by ear
Three real bugs found and fixed by the tests:
- 174 BPM read as 87. Mean-energy-per-beat scores a half-tempo grid
identically to the true one; only an F-measure penalises the missed
onsets via recall.
- 90 BPM read as 180. Offbeat hi-hats make a double-tempo grid score
perfectly on both precision and recall, so the grid is now interpreted
metrically afterwards: a systematic strong/weak alternation means the
real beat is every other grid point.
- Beat grid drifted ~30ms over 30s from integer-frame offsets. Onset peaks
are now parabolically interpolated and the grid least-squares fitted.
Gates: 11/11 node tests against synthetic ground truth (tempo within 2%
across 90-174 BPM, beat alignment under half a frame, segmentation within
2s of a known boundary, graceful on silence, 6-minute analysis in 1.0s);
5/5 browser checks including audio-driven vs fixed-step frame parity.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Engine core: Timeline (fixed dt, audio-mastered in realtime), seeded Rng,
Renderer, Layer/ShaderLayer/SceneLayer, Compositor with blend modes,
feedback and post chain.
Shader scenes are compiled against a fixed uniform contract and define only
`vec4 scene(vec2 uv, vec2 p)`, so adding a scene costs a shader plus a
params block. Deep Nebula ported from party-stage as the first one.
Gate passes, 7/7 in checks.html:
- 300 frames rendered twice are bit-identical
- a fresh Engine reproduces the same frames
- simulated dropped frames change nothing (proves dt is fixed)
- seek matches sequential playback
- 320x180 vs 1280x720 agree within 0.010 (limit 0.06)
- seeded rng reproducible, forked streams independent
- compositor reset clears feedback history
Static gates: no wall-clock or unseeded randomness in deterministic
directories; scene schemas and shader sources agree in both directions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Design document for a new generator that produces full-length, non-story
music videos from a track alone, with no sourced footage.
Core decisions:
- Deterministic engine: audio is analyzed up front into a frame-indexed
feature table, so realtime preview and offline export render from the
same code path and produce the same frames.
- Whole-song analysis enables lookahead, so builds can anticipate drops.
- Scenes declare their parameters in a schema, which drives uniform
binding, generated UI, seeded per-track variation and arc automation.
- Hybrid compositor: fragment-shader and three.js layers in one stack.
- Preview is a first-class surface with section jumping, live param
editing and segment test-renders.
- Forked from party-stage by copy, then fully detached.
Includes per-phase gates and the validation tooling they depend on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
