music-video-gen/metabolism/EVOLUTION.md

evolution — from a petri dish to a world

Course correction after watching the demo, agreed with the code:

What works. The song→analysis→events→wells→laws front-end genuinely connects to the sim (coupling/residual and fingerprint divergence are real). The determinism discipline is intact and now proven bit-for-bit in the browser. The demo shell and the tools share one Run.

What failed. The world behind the Run. All songs grow the same three castes on the same 128² dish with the same opening and the same laws-as-multipliers. Statistically different, structurally identical. Two skins, not two habitats. And the camera watches it at the microscopic scale, so nothing larger ever has room to appear.

The correction is one idea: the world must be a macro ecology — population density, not individual dots — governed by structural rules the song compiles to, watched by a camera that travels instead of cutting.

---

Milestones

M0 — agreement ✔

Representation: population fields. Rules: a scoped rule program (compiled per song, deterministic). Plan: this document. DESIGN.md updated to the new end-goal.

M1 — the macro world: density, substeps, geography ✔

The entity sim is replaced by a population-field sim.

• World: FIELD=128 integrated density grid (rendered as RENDER_FIELD=256 speck texture); layers: food, three caste populations, three tinted trail fields, wells, a static seeded jitter pair.
• Time: SUBSTEPS=5 sim steps per rendered frame (the old world: 1), so local interactions propagate into coherent macro patterns.
• Physics: forces are integrated on a coarse 64² grid (2×2 block sums — long-range coupling at a fraction of the cost), then advected on the fine grid with a square-root-free L1 axis split. Reaction: eat → logistic growth with a total-density gate → deposit; starvation on empty ground. Events still splat food and bloom the voice that spoke; rings land on strong hits.
• Home basins: each caste anchors to its own wells (the field-world "builder orbits its well"), so colonies persist as continents instead of melting into a uniform haze.
• Look: trails compose the geography; density renders as a glow plus deterministic speck swarms — "billions of little guys" is texture, at zero entity cost.
• Opening: the dish wakes into the song's own first bars.

Outcome: at 60 s the population is still geographically structured (8-block CV 1.5–1.9 vs ~0.1 before), songs and seeds diverge at a glance in-browser, openings differ per song, realtime holds at 640×360 (960×540 on reasonable hardware), and the test suite is green including a prepared-artifact bit-identity check. M2 then replaced the multiplier laws with the rule program; the camera's travel grammar is M3.

M2 — the rule program: structure, not knobs ✔

The laws-as-multipliers are replaced by a scoped rule program (src/song/rules.js) — declarative per-song rules:

• castes are rule entries: speed / seek / dispersion / cap / growth / eat / deposit, plus growth rules — mutualism (a caste breeds fastest among the others, so mixed crowds beat monochrome blobs) and a social floor (sparks only bloom in a luminous crowd);
• interaction rules per caste pair (the signed band-correlation graph);
• per-section operator regimes — a breakdown drives disperse≈0 and deposit high (the world settles and inscribes); a drop drives home≈0 and turbulence+spawn high (the flood). Structural, scoped by the track's own character (noisy → turbulence up, tonal → deposit up);
• the event→response mapping: which voice feeds which caste, and how much of every hit splashes to the others;
• memory (trail/field half-lives, diffusion) and a global carrying capacity so one song cannot flood while another starves;
• compiled once per song, deterministically — the program is part of the song, rebuilt bit-identically by the prepared-artifact path, and v3's rule search now searches programs, not floats.

Outcome: 23 tests green, including: program compilation is deterministic; different songs get structurally different programs; regimes are structurally distinct by kind (a growth=0 edit visibly halts the world — the program is executed, not decoration); genesis pre-simulates generations so frame 0 is already an ecosystem, with the amount tuned per song; the opening is seeded where the song's own hits landed. Mixing is a program rule: castes co-mingle in one shared geography (drone 0.93, dusk 0.78, runner 0.48 block-mix; was ~0.02–0.15), the lens holds the whole world (zoom ~0.9–1.6), and divergence between songs grew to 16–40× where seeds stay ~0.01–0.05.

M2.5 — song-driven life, more generations, higher altitude ✔

• The song spawns: colonies are no longer placed at a pre-chosen set of spots. Every hit lands food and births life there (the voice first and loudest, splashes to the others); the drop regime visibly floods the ground with new colonies where the track hits.
• Genesis: the world pre-simulates several generations before frame 0 (the amount comes from the song — intro length, onset density, pulse), so the opening is an old, settled ecosystem, never a blank plate, and the film's history starts clean at frame 0.
• Altitude: the lens sits higher (zoom ~0.6–1.5), holding the whole map — different cultures and the roads and fronts between them — in frame.

M2.6 — the tribes are programs ✔

The M2 rule program steered how hard behaviours ran. Tribes go deeper: each tribe is a small deterministic behaviour program (src/song/tribe.js) — a tiny VM with

• sensing (food, own/other densities, trail, law, hunger, rim-edge, time),
• memory (four persistent registers, smoothed by the track's pulse),
• actions (move, excrete, eat, hit, spawn),
• basic branching (if hunger > threshold → seek food, else hold-and-ink),

compiled by the song: the band-correlation matrix signs who a tribe approaches vs flees, flatness sets how much it branches on gradients, onset density sets its spawning, the seed jitters thresholds. The world no longer executes a hand-tuned force formula on the coarse decision grid — it runs what each tribe's program decides, every substep. Surprising behaviour is possible precisely because the programs branch: a starving tribe behaves differently from a fed one, and the threshold is musical.

Outcome: the three societies are now balanced and mixed (drone 0.86 / runner 0.87 / dusk 0.95 block-mix; populations 40–76 k, castes near-equal), the geography holds (population CV 2.2–2.8). New tests pin the VM: programs compile deterministically and differ by song (restless/social/hoarding/breeding archetypes selected from the stats); branching changes behaviour; registers persist as memory; a no-op program visibly changes the film. Realtime holds at 640×360 (tribe runs add ~4 ms/frame at 960×540 in node).

Honest limit: the compiler's vocabulary is still a small archetype family, so the structural divergence between songs is modest (cross ≈ 0.16–0.18 vs within-seed ≈ 0.02–0.04, ~4–7×) even though the films look different. Broadening the instruction grammar (more branch shapes, richer memory, more action archetypes) is the natural next step for this layer.

M2.7 — one species, many minds: strains & allegiances (design agreed; not yet built)

A further correction, agreed before building anything on M2.6: drop tribes entirely. Not three castes, not k tribes of fixed membership. The picture is one population of specialised microorganisms building a bigger organism — and then several such macro-organisms sharing a map, sometimes fighting, sometimes cooperating. Membership is a property of each organism, and it can change over time.

Wells keep the correction from the previous draft — small, weak, many (8–16), born where a pitch class enters and holds, drifting with its harmony, dying when it leaves, all on a precompiled deterministic timeline — because wells are where bodies collide: everyone needs them, so a well is the natural front, its birth a land-rush, its death a retreat.

The population is one density field with two live properties:

• strain mix (personality). The song decides how many minds it needs — m strains from the number of independent voices the mix resolves into, bounded by arrangement depth, jittered by the seed — and compiles a LittleVM program per strain from the grammar: senses (food, density, trail, law, hunger, rim, well proximity and liveliness, body presence), registers as memory, real branching, actions (move, graze, excrete, hit, spawn, follow, patrol). Density is partitioned by strain; each strain advects by its own program. And the mix evolves by local selection: strains that eat and grow where they stand propagate, proportions shift district by district — specialisation is visible and develops over the film.
• allegiance (the body). A composition field over bodies (the song opens 2–4; a cap of ~8 keeps colour legible), advected frozen-in with the population flux. Same body = cooperation: shared trails, reinforcement, complementary strains building one structure. Different bodies = contest where presences overlap: hit-actions spend density to push the other body's ground and convert it, scaled by the song's harmony — consonant, locked voices breed truce and symbiosis; dissonant, decorrelated voices breed skirmish and siege. Allegiance changes by rule, never by fiat:
  • conversion — a won front converts the locals it overruns;
  • defection — a body whose presence collapses (starved, outflanked) releases its members to the neighbouring body;
  • fission — a coherent splinter cut off from its body, hit by a strong onset, founds a new body with a new colour;
  • merger — two bodies whose members fully intermix under a cooperative regime become one.

The song sets the diplomacy. Consonance and the section regimes map to war and peace: drops are expansion wars, breakdowns are truces and mutual building, an outro is a settling of borders. The body-to-body relations come from the music, not a constant.

Exit: two songs with the same pulse but different arrangement depth compile different strain counts and different diplomacies; at least one allegiance change (conversion, defection, fission or merger) is legible on screen per film; strain proportions measurably drift across the film (local selection actually runs); a well's birth, drift and death are each legible; the world's typed-array layers, trail tints and palette generalise from 3 fixed castes to the strain/body fields; the prepared-artifact path still rebuilds bit-identical timelines, programs and opening state.

M3 — the lens: travel, don't cut

Replace the cut-to-a-point grammar with a move grammar: track (follow a moving front), reveal (pan along a coastline), establish (hold wide and slow). Cuts are reserved for musical punctuation — strong onsets in drop/build. Because the world is macro, a pan teaches the map instead of disorienting.

Exit: a 120 s render where a single front can be followed across the world without losing bearings; cuts read as beats, not jumps.

M4 — the film

Mux frames + PCM to video. No longer the bottleneck: the preview pipeline and the lens exist; once M1–M3 make the world worth exporting, muxing is plumbing.

---

The prepare step (pre-calculation)

Viewing should be fast because the expensive work happens ahead of time, once:

1. tools/prepare.js runs the WAV → analysis → compile pipeline in node and writes a prepared song (out/prepared/<song>-<seed>.json): the compiled rule program and every analysis structure the world, camera, and metrics need — but not the raw PCM.
2. The browser fetches the prepared song instead of analysing the WAV. The WAV is still fetched, but only to feed the WebAudio clock (decoded straight to a buffer; analysis skipped).
3. src/song/serialize.js is the single serialize/deserialize for both sides — the browser's deserializeSong must rebuild a song that is bit-identical to one built by analysing the WAV, so prepare can never drift from the live path. That equality is a test.

This is exactly the DESIGN promise inverted and kept: the pipeline still defines the world; prepare only pays its cost once, off the viewing path.

Invariants, restated for the field world

• Determinism: same song + same seed + same frame ⇒ same pixel. No Math.random, no wall clock, no per-substep RNG. The static jitter noise field is generated once from the seeded RNG at construction.
• Causality: events still place food and rings where the track put them; a section boundary still changes the rules at the boundary only.
• Persistence: population fields carry their history; trails accumulate in the same three tinted layers the renderer already composes.
• Consequence: everything is a pure update rule over the previous substep; nothing is re-rolled.