music-video-gen/metabolism/EVOLUTION.md

evolution — from a petri dish to a world

Course correction after watching the demo, agreed with the code:

What works. The song→analysis→events→wells→laws front-end genuinely connects to the sim (coupling/residual and fingerprint divergence are real). The determinism discipline is intact and now proven bit-for-bit in the browser. The demo shell and the tools share one Run.

What failed. The world behind the Run. All songs grow the same three castes on the same 128² dish with the same opening and the same laws-as-multipliers. Statistically different, structurally identical. Two skins, not two habitats. And the camera watches it at the microscopic scale, so nothing larger ever has room to appear.

The correction is one idea: the world must be a macro ecology — population density, not individual dots — governed by structural rules the song compiles to, watched by a camera that travels instead of cutting.

---

Milestones

M0 — agreement ✔

Representation: population fields. Rules: a scoped rule program (compiled per song, deterministic). Plan: this document. DESIGN.md updated to the new end-goal.

M1 — the macro world: density, substeps, geography ✔

The entity sim is replaced by a population-field sim.

• World: FIELD=128 integrated density grid (rendered as RENDER_FIELD=256 speck texture); layers: food, three caste populations, three tinted trail fields, wells, a static seeded jitter pair.
• Time: SUBSTEPS=5 sim steps per rendered frame (the old world: 1), so local interactions propagate into coherent macro patterns.
• Physics: forces are integrated on a coarse 64² grid (2×2 block sums — long-range coupling at a fraction of the cost), then advected on the fine grid with a square-root-free L1 axis split. Reaction: eat → logistic growth with a total-density gate → deposit; starvation on empty ground. Events still splat food and bloom the voice that spoke; rings land on strong hits.
• Home basins: each caste anchors to its own wells (the field-world "builder orbits its well"), so colonies persist as continents instead of melting into a uniform haze.
• Look: trails compose the geography; density renders as a glow plus deterministic speck swarms — "billions of little guys" is texture, at zero entity cost.
• Opening: the dish wakes into the song's own first bars.

Outcome: at 60 s the population is still geographically structured (8-block CV 1.5–1.9 vs ~0.1 before), songs and seeds diverge at a glance in-browser, openings differ per song, realtime holds at 640×360 (960×540 on reasonable hardware), and the test suite is green including a prepared-artifact bit-identity check. M2 then replaced the multiplier laws with the rule program; the camera's travel grammar is M3.

M2 — the rule program: structure, not knobs ✔

The laws-as-multipliers are replaced by a scoped rule program (src/song/rules.js) — declarative per-song rules:

• castes are rule entries: speed / seek / dispersion / cap / growth / eat / deposit, plus growth rules — mutualism (a caste breeds fastest among the others, so mixed crowds beat monochrome blobs) and a social floor (sparks only bloom in a luminous crowd);
• interaction rules per caste pair (the signed band-correlation graph);
• per-section operator regimes — a breakdown drives disperse≈0 and deposit high (the world settles and inscribes); a drop drives home≈0 and turbulence+spawn high (the flood). Structural, scoped by the track's own character (noisy → turbulence up, tonal → deposit up);
• the event→response mapping: which voice feeds which caste, and how much of every hit splashes to the others;
• memory (trail/field half-lives, diffusion) and a global carrying capacity so one song cannot flood while another starves;
• compiled once per song, deterministically — the program is part of the song, rebuilt bit-identically by the prepared-artifact path, and v3's rule search now searches programs, not floats.

Outcome: 23 tests green, including: program compilation is deterministic; different songs get structurally different programs; regimes are structurally distinct by kind (a growth=0 edit visibly halts the world — the program is executed, not decoration); genesis pre-simulates generations so frame 0 is already an ecosystem, with the amount tuned per song; the opening is seeded where the song's own hits landed. Mixing is a program rule: castes co-mingle in one shared geography (drone 0.93, dusk 0.78, runner 0.48 block-mix; was ~0.02–0.15), the lens holds the whole world (zoom ~0.9–1.6), and divergence between songs grew to 16–40× where seeds stay ~0.01–0.05.

M2.5 — song-driven life, more generations, higher altitude ✔

• The song spawns: colonies are no longer placed at a pre-chosen set of spots. Every hit lands food and births life there (the voice first and loudest, splashes to the others); the drop regime visibly floods the ground with new colonies where the track hits.
• Genesis: the world pre-simulates several generations before frame 0 (the amount comes from the song — intro length, onset density, pulse), so the opening is an old, settled ecosystem, never a blank plate, and the film's history starts clean at frame 0.
• Altitude: the lens sits higher (zoom ~0.6–1.5), holding the whole map — different cultures and the roads and fronts between them — in frame.

M2.6 — the tribes are programs ✔

The M2 rule program steered how hard behaviours ran. Tribes go deeper: each tribe is a small deterministic behaviour program (src/song/tribe.js) — a tiny VM with

• sensing (food, own/other densities, trail, law, hunger, rim-edge, time),
• memory (four persistent registers, smoothed by the track's pulse),
• actions (move, excrete, eat, hit, spawn),
• basic branching (if hunger > threshold → seek food, else hold-and-ink),

compiled by the song: the band-correlation matrix signs who a tribe approaches vs flees, flatness sets how much it branches on gradients, onset density sets its spawning, the seed jitters thresholds. The world no longer executes a hand-tuned force formula on the coarse decision grid — it runs what each tribe's program decides, every substep. Surprising behaviour is possible precisely because the programs branch: a starving tribe behaves differently from a fed one, and the threshold is musical.

Outcome: the three societies are now balanced and mixed (drone 0.86 / runner 0.87 / dusk 0.95 block-mix; populations 40–76 k, castes near-equal), the geography holds (population CV 2.2–2.8). New tests pin the VM: programs compile deterministically and differ by song (restless/social/hoarding/breeding archetypes selected from the stats); branching changes behaviour; registers persist as memory; a no-op program visibly changes the film. Realtime holds at 640×360 (tribe runs add ~4 ms/frame at 960×540 in node).

Honest limit: the compiler's vocabulary is still a small archetype family, so the structural divergence between songs is modest (cross ≈ 0.16–0.18 vs within-seed ≈ 0.02–0.04, ~4–7×) even though the films look different. Broadening the instruction grammar (more branch shapes, richer memory, more action archetypes) is the natural next step for this layer.

M2.7 — one species, many minds: strains & allegiances ✔

Built — with one correction agreed mid-build, after watching the first films:

Allegiance does not change sides. The original draft had fronts converting, collapsed bodies leaking their members, splinters seceding and friendly bodies merging. On screen that read as one thing: every colour blurring into every other within a minute — ugly, and it erased the very polities the mechanic was meant to create. So the fluid politics were removed: bodies are founded once and persist; war kills, it does not convert; there is no defection and no merger. What keeps the cast plural is colonisation — a blessed settlement on virgin ground, far from every body core, plants a new flag. And the renderer now paints each cell in its DOMINANT body's colour, so territories read as territories.

What was built:

• strains (src/song/strains.js) — the song decides how many minds it needs (2–5, from onset density, an independent spectral voice, texture noisiness, seed jitter) and compiles a LittleVM program per strain from motif roles: builder, grazer, scout, sentinel, breeder. Senses include the body's presence (KIN/FOE), the living wells, and the section's PEACE — a program can branch on war or truce. The world runs local selection: strains that thrive where they stand propagate.
• living wells (src/sim/wells.js) — a precompiled deterministic timeline of small, weak, numerous wells (4+ at open, ≤12 alive): born where a pitch class enters and holds, drifting along its harmony's compass, starving when it leaves. A birth is a land-rush; a death a retreat.
• bodies & diplomacy — allegiance is a concentration field advected frozen-in with the population flux (Σ bodies = density exactly, a test). Diplomacy = the track's consonance bent by the section's peace operator; hostile fronts bleed the loser (casualties, never conversion). Colonisation plants new bodies on blessed virgin ground.
• a food-limited economy — the old world was accidentally cap-limited: the gate floor let life grow on bare ground, so the dish filled to a uniform saturated carpet and stopped moving. Now the gate is nearly shut without food and starvation dissolves colonies off bare ground: the population tracks its wells, blooms on them, and must keep moving to keep eating.

Outcome: 39 tests green, including the allegiance invariant (no negatives, composition = population), war-kills-not-converts (a war world ends poorer than its truce twin, with each body's mass still in its own heartland), colonisation, strain-mix drift, well determinism/drift/land-rush. Realtime holds at 640×360 (~12 ms/frame sim+render). Songs still diverge ≫ seeds on the fingerprint (classMix now sorted — the seed permutes the casting, the song decides its shape).

M2.8 — rock: the world gains an inert resource ✔

The films were alive but weightless: nothing on the map was stuff. Rock is the correction — an inert resource with mass, presence and consequence:

• the field (world.rock) — a rock height per cell; above 0.55 the cell is impassable (advection aims mass at stone and the mass stays, piling against the wall; diffusion only crosses open edges). Food does not recharge through rock: capped ground is barren until someone quarries it open.
• the song's weather — the map is born with a deterministic scatter of stone (7–15 blobs, from the seed). Strong low/mid hits DROP rock where they land (and crush what they land on); strong high hits SHATTER it. The section regimes gained rockfall and erode operators: drops rain stone, breakdowns weather it away.
• the masons — builders gained two VM actions: dig (a slow bite out of the rock, which becomes carried burden — and grinding yields soil, a little food, right where they dug) and build (the carried burden rises as a wall where the ground is clear). The carry lives in per-block registers of the world, so stone genuinely travels: quarries open, roads run from them, walls rise where the colony loiters.
• livelier organisms — slower metabolisms (longer lives off food), faster strains, softer bare-ground starvation: agents travel farther between wells, meet more, and work more.

Outcome: 44 tests green, with new pins: the deterministic scatter; the song's rockfall outpacing a mute sky; a stone wall holding a crowd west of it for 10 s straight; quarries opening with soil and burden appearing; carried rock rising as walls. The strips read it directly: grey masses the colonies grow around, fronts grinding along them.

M2.9 — flora, gardens, walls: a world of workable things ✔

After M2.8 the films had stone but nothing to DO with it, and the ecology still lived on wells alone. This iteration added the interactable layer — items with feedback loops, each needing its own LittleVM instructions, hence real specialisation:

• aimed work — the VM gained an AIM (two pseudo-actions setting a heading; it defaults to the direction of travel), and every work action lands at the block the agent is FACING, never beneath its feet: masons carve the face of the wall ahead, farmers tend the bed they walk toward, strikers hit where the enemy stands. The work-senses (rock, garden) look where the last aim pointed — sight follows intent.
• gardens — the mason TILLS fresh stone into a cultivated bed (brought to maturity one cell at a time; a dusting of fertility would exhaust as fast as it is laid). A bed yields food slowly for a while, then the spent soil sets back into rock: agriculture slowly eats the world's stone, and the song's rockfalls replenish it. Digging itself yields NO food — stone is for building; food comes from wells, flora, events and gardens.
• walls — the carried burden rises as a wall (pooled per block, mortared along a rounded footprint). Walls block movement, so they defend; enemies strike at their aim target and SAP the stone at the front — sieges happen, and unmaintained walls weather away. The masons are never done.
• farmers — a new strain role: it walks the field rows and TENDS the beds (a tended bed yields up to threefold), opens new beds into fresh stone, and commutes along the garden gradient. Work is foraging: the hungry branches of mason and husbandman work the ground they aim at, because that is where their food comes from.
• flora — the world's pioneers, living in ISLANDS scattered at birth (a dozen small patches; they can never spread beyond their own seed ground). Each stand is food — grazed down by hungry crowds, never wiped out (seed-rain), regrowing slowly. Its vigour tracks the world's famine: a starving or dwindling world blooms with flora and is seeded back to life; a strong, well-fed world grazes it back to a whisper. Rendered as speckles — a few apple trees and bushes here and there, never a garden of Eden.

Balance notes carved into the tests: the ecology is food-limited (the gate is nearly shut on bare ground; colonies that outgrow their ground starve back and wander); rock has a budget that survives a whole film (calm songs keep sparse stone, percussive ones accumulate walls); agriculture consumes stone instead of minting it. 44 tests green; realtime holds at 640×360.

The decision layer on the GPU (analysis, first slice landed)

The profile of a heavy frame (field 128, ~40 000 organisms planted) says:

layer	share
the strain VMs (runProgram)	~35%
senses/gradients/action plumbing (runStrains)	~20%
field passes (block sums, reaction, substep)	~16%
advection (+allegiance, burden)	~7%
camera + render	~10%

Three moves, in the order they pay:

1. Think slower than you move (landed). The programs ran at 150 Hz — every substep — though a mind's decisions barely change in 1/30 s. The world now thinks every vmEvery-th substep (default 2: ~15 thoughts/s) and coasts on its last decision between thoughts; blocks with no population think not at all. Measured on the heavy world (41 k planted, sim only): 25.0 → 20.3 ms/frame at ×2, 17.7 at ×3 — with statistically identical ecologies (38.6 k vs 38.7 k final). The tests pinned the rest.
2. The GPU (first slice landed, experimental). The decision layer is ideal GPU work: m uniform programs over ~4 000 independent blocks — lockstep, only the data differs. src/sim/gpu.js holds a WebGPU compute path for exactly that piece: senses/grads packed on the CPU, the flat bytecode uploaded per strain, registers resident on the device, one dispatch per strain, the action buffer read back per substep. tools/gpu-check.js (browser-side) validates the GPU result against the CPU interpreter and times both. Status: written against the WebGPU API, awaiting a device to run on; NOT yet wired into World.step — that wiring needs an async stepper (the readback is asynchronous), which the demo can adopt and the exporter need not.
3. The rest of the field (block sums, advection, reaction) is the next and larger GPU slice — element-wise and stencil passes over the fields, with the population/politics staying on the CPU. That is what unlocks field 256 and fatter casts.

Determinism policy. GPU float ordering differs from the CPU's, so a GPU-accelerated preview may drift slightly from a CPU-exported film. The policy: the prepared-artifact/export pipeline stays CPU and remains the canonical pixel source; the GPU path is a preview accelerator whose world is the same world to within float noise. If preview/export bit-identity is ever demanded, the exporter must run on the same backend as the preview — a node WebGPU binding (e.g. a wgpu native addon) is the door for that, and it is explicitly out of scope until the film needs it.

M3 — the lens: travel, don't cut

Replace the cut-to-a-point grammar with a move grammar: track (follow a moving front), reveal (pan along a coastline), establish (hold wide and slow). Cuts are reserved for musical punctuation — strong onsets in drop/build. Because the world is macro, a pan teaches the map instead of disorienting.

Exit: a 120 s render where a single front can be followed across the world without losing bearings; cuts read as beats, not jumps.

M4 — the film

Mux frames + PCM to video. No longer the bottleneck: the preview pipeline and the lens exist; once M1–M3 make the world worth exporting, muxing is plumbing.

---

The prepare step (pre-calculation)

Viewing should be fast because the expensive work happens ahead of time, once:

1. tools/prepare.js runs the WAV → analysis → compile pipeline in node and writes a prepared song (out/prepared/<song>-<seed>.json): the compiled rule program and every analysis structure the world, camera, and metrics need — but not the raw PCM.
2. The browser fetches the prepared song instead of analysing the WAV. The WAV is still fetched, but only to feed the WebAudio clock (decoded straight to a buffer; analysis skipped).
3. src/song/serialize.js is the single serialize/deserialize for both sides — the browser's deserializeSong must rebuild a song that is bit-identical to one built by analysing the WAV, so prepare can never drift from the live path. That equality is a test.

This is exactly the DESIGN promise inverted and kept: the pipeline still defines the world; prepare only pays its cost once, off the viewing path.

Invariants, restated for the field world

• Determinism: same song + same seed + same frame ⇒ same pixel. No Math.random, no wall clock, no per-substep RNG. The static jitter noise field is generated once from the seeded RNG at construction.
• Causality: events still place food and rings where the track put them; a section boundary still changes the rules at the boundary only.
• Persistence: population fields carry their history; trails accumulate in the same three tinted layers the renderer already composes.
• Consequence: everything is a pure update rule over the previous substep; nothing is re-rolled.