music-video-gen/metabolism

metabolism

Songs as substrate. A music video generator whose outcome is grown, not mixed: the track is not a parameter set the renderer modulates — it is the world an agent ecology grows on. Same song + same seed ⇒ the same world, bit for bit, frame for frame, preview identical to export.

 song (PCM) ──► analyse ──► score · field · graph · laws ──► species ──► world ──► camera ──► frames

The full concept and rationale live in DESIGN.md — read it before touching the code.

---

The idea in one paragraph

Most visualisers modulate one fixed stage: the song turns knobs, and every track ends up presenting the same toy in different clothes. Here the song places the food (events land where the track's harmonies point), the laws (each section changes the rules at its boundary), and the species (the seed picks a rule set — built from tempo, noise, brightness and band correlations — not a parameter vector). Two songs differ like two habitats differ, not like two skins. drone grows a slow tonal weaver with heavy, lasting ground markings; runner grows a fast noisy swarm with jittered motion and thin ephemeral trails; dusk grows a crystalline, slow, over-clustered world where the sparks reject the builders.

Invariants

Everything is load-bearing to keep these true:

• Persistence — agents and their trails last seconds to tens of seconds.
• Causality — every visible event has an audible cause (hits are rings on the ground) and a visible one.
• Consequence — the field is eaten, the trails accumulate; state at t is made of state at t−1, never re-rolled.
• Determinism — no Math.random, no wall clock; tools and browser share the same Run, so preview provably equals export.

---

Quick start

Requires Node 20+ (for node --test) and your usual package manager.

npm install

# tests: analysis, determinism, world invariants, metrics sanity
npm test

# filmstrips: 12 frames per song, tiled
#   (needs the flow-state test bank WAVs at ../flow-state/test/songs)
npm run strip                 # drone, runner, dusk
npm run strip -- --songs drone --cols 6 --rows 2 --out out
npm run strip -- --songs drone,runner,dusk --combined 1   # comparison sheet

# report: coupling × residual fitness + fingerprint divergence across songs/seeds
npm run report                 # prints a table, writes out/report.md + report.json
npm run report -- --songs drone,runner,dusk --seeds 999,12345,777777

# prepare: analyse songs once, so the demo loads instantly instead of
# re-analysing the WAV in the browser (falls back to live analysis if absent)
npm run prepare               # writes out/prepared/<song>-<seed>.json

Outputs land in out/ (gitignored).

The demo

npm run dev            # open http://localhost:5173 and pick a song

A browser demo that plays the songs off the actual pipeline. The page serves the bank from ../flow-state/test/songs (via vite.config.js — no copies), lists all 17 tracks with the species each one grows, and plays the chosen track while the world steps to it:

• select a song → it fetches the WAV, decodes it, and grows the world to frame 0 before letting you in. Run npm run prepare once and the browser loads the pre-computed analysis instead of re-analysing, so viewing is fast (it falls back to live analysis when an artifact is missing);
• play / pause (or space), seek (or ←/→ ±5 s) — the audio clock only chooses which frame the world should have reached, so seeking is fast forward; a backward seek re-grows from frame 0;
• seed (the exact world), res (640×360 realtime default, up to 1280×720) and volume; changes live without breaking playback;
• the seek bar shows section ticks (the laws), the status line shows the current law, shot size/zoom and live world stats (population, field, trail);
• the page accent wears the track's palette hue, and the URL carries ?song=&seed=&res= so a film is a link.

Determinism is the point: the page renders through the same Run as the tools — same song, same seed, same frame ⇒ the same pixels (tested bit-identical against strip output, including the prepared-artifact path). npm run build bundles it; npm run preview serves the built page from dist/. The world behind the Run is the macro ecology of EVOLUTION.md M1 — population fields, not entities; see EVOLUTION.md.

Files: index.html, vite.config.js (song-bank + prepared-artifact plugin), src/demo/* (main.js page wiring, player.js Run→canvas, audio.js WebAudio clock, catalog.js metadata, style.css).

Layout

DESIGN.md            concept, invariants, build order (end-goal)
EVOLUTION.md         course-corrected roadmap (M1 done, M2–M4 ahead)
index.html           the demo page
vite.config.js       song-bank + prepared-artifact plugin (dev + build)
src/
  audio/             WAV decode, radix-2 FFT, feature table, sectioning, onset score
  song/              events (onsets → geometry), species (genome), rules.js (the
                     per-song rule program + section operator regimes),
                     tribe.js (the per-tribe behaviour VM: senses, memory,
                     actions, branching — compiled from the song), palette,
                     song assembly, serialize.js (prepare cache)
  sim/               wells (harmony→food) and world.js (the macro field-world:
                     128² density, 5 substeps/frame, executes the rule program
                     and the tribe programs)
  render/            software framebuffer: trail composer, density glow + speck
                     swarms (the "billions"), PNG encoder
  camera.js          wide event-reactive shots, shot-size rhythm, drift-and-push
  film.js            Run: the single wire-up of sim → camera → framebuffer
  metrics.js         coupling, residual, fitness, fingerprint, divergence
  rng.js             seeded PRNGs (the only randomness in the repo)
  demo/              player page: main.js wiring, player.js (Run→canvas),
                     audio.js (WebAudio clock), catalog.js, style.css
tools/
  load.js            WAV → analysis → song  (node-only)
  prepare.js         pre-computes analysis → out/prepared/<song>-<seed>.json
  strip.js           filmstrip renderer
  report.js          system-level evaluation → markdown table
test/                node:test suites (incl. prepared-artifact bit-identity)

References

tools/* load the synthetic reference tracks from ../flow-state/test/songs (drone, runner, dusk) — three 120 s test signals chosen to cover the axes the species reads: dark/tonal/beatless, bright/noisy/fast, darkest/all-tonal/mid-tempo. Drop any WAV there and point --songs at it; the species will derive from whatever the track actually says.

Status

M1, M2 and M2.6 of EVOLUTION.md are done: the entity world is a macro, population-field ecology (density, substeps, coarse-grid forces, speck rendering); the rules that run it are a declared, compiled rule program (per-caste rule entries, interaction rules, per-section operator regimes, the event→response mapping); and the tribes are small deterministic programs — sensing, memory, actions and branching, compiled by the song and executed on the decision grid every substep. Songs grow balanced, mixed societies (block mix 0.86–0.95) with structured geography, and the opening is a genesis-settled world, not a blank plate. Everything is deterministic and bit-identical across the demo, the tools and the prepared-artifact path (28 tests green). It is not yet a film — the camera still cuts rather than travels (M3) and nothing is muxed to a video file (M4).

Auditing what it grew

The report's coupling × residual fitness is the honest score: coupling is how much the ecology answers the track; residual is the internal dynamics the track does not explain. A pure visualiser scores ~0 (no residual); a random walk scores ~0 (no coupling). The fingerprint (occupancy, clustering, trail fill, class mix) tells you whether two songs grew different worlds without anyone having to opine about it.