music-video-gen/metabolism/EVOLUTION.md

# evolution — from a petri dish to a world

Course correction after watching the demo, agreed with the code:

**What works.** The song→analysis→events→wells→laws front-end genuinely connects
to the sim (coupling/residual and fingerprint divergence are real). The
determinism discipline is intact and now proven bit-for-bit in the browser.
The demo shell and the tools share one `Run`.

**What failed.** The *world behind the Run*. All songs grow the same three
castes on the same 128² dish with the same opening and the same laws-as-multipliers.
Statistically different, structurally identical. Two skins, not two habitats.
And the camera watches it at the microscopic scale, so nothing larger ever has
room to appear.

**The correction** is one idea: the world must be a *macro* ecology — population
density, not individual dots — governed by *structural rules* the song compiles
to, watched by a camera that *travels* instead of cutting.

---

## Milestones

### M0 — agreement ✔
Representation: **population fields**. Rules: **a scoped rule program** (compiled
per song, deterministic). Plan: this document. `DESIGN.md` updated to the new
end-goal.

### M1 — the macro world: density, substeps, geography ✔
The entity sim is replaced by a population-field sim.

- **World**: `FIELD=128` integrated density grid (rendered as `RENDER_FIELD=256`
  speck texture); layers: food, three caste populations, three tinted trail
  fields, wells, a static seeded jitter pair.
- **Time**: `SUBSTEPS=5` sim steps per rendered frame (the old world: 1), so
  local interactions propagate into coherent macro patterns.
- **Physics**: forces are integrated on a coarse 64² grid (2×2 block sums —
  long-range coupling at a fraction of the cost), then advected on the fine
  grid with a square-root-free L1 axis split. Reaction: eat → logistic growth
  with a total-density gate → deposit; starvation on empty ground. Events still
  splat food and bloom the voice that spoke; rings land on strong hits.
- **Home basins**: each caste anchors to its own wells (the field-world
  "builder orbits its well"), so colonies persist as continents instead of
  melting into a uniform haze.
- **Look**: trails compose the geography; density renders as a glow plus
  deterministic speck swarms — "billions of little guys" is texture, at zero
  entity cost.
- **Opening**: the dish wakes into the song's own first bars.

*Outcome:* at 60 s the population is still geographically structured (8-block
CV 1.5–1.9 vs ~0.1 before), songs and seeds diverge at a glance in-browser,
openings differ per song, realtime holds at 640×360 (960×540 on reasonable
hardware), and the test suite is green including a prepared-artifact
bit-identity check. *M2 then replaced the multiplier laws with the rule
program; the camera's travel grammar is M3.*

### M2 — the rule program: structure, not knobs ✔
The laws-as-multipliers are replaced by a **scoped rule program**
(`src/song/rules.js`) — declarative per-song rules:

- **castes** are rule entries: speed / seek / dispersion / cap / growth / eat /
  deposit, plus *growth rules* — mutualism (a caste breeds fastest among the
  others, so mixed crowds beat monochrome blobs) and a social floor (sparks
  only bloom in a luminous crowd);
- **interaction rules** per caste pair (the signed band-correlation graph);
- per-section **operator regimes** — a breakdown drives disperse≈0 and
  deposit high (the world settles and inscribes); a drop drives home≈0 and
  turbulence+spawn high (the flood). Structural, scoped by the track's own
  character (noisy → turbulence up, tonal → deposit up);
- the **event→response** mapping: which voice feeds which caste, and how much
  of every hit splashes to the others;
- **memory** (trail/field half-lives, diffusion) and a **global carrying
  capacity** so one song cannot flood while another starves;
- compiled once per song, deterministically — the program is part of the song,
  rebuilt bit-identically by the prepared-artifact path, and v3's rule search
  now searches *programs*, not floats.

*Outcome:* 23 tests green, including: program compilation is deterministic;
different songs get structurally different programs; regimes are structurally
distinct by kind (a growth=0 edit visibly halts the world — the program is
executed, not decoration); genesis pre-simulates generations so frame 0 is
already an ecosystem, with the amount tuned per song; the opening is seeded
where the song's own hits landed. Mixing is a program rule: castes co-mingle
in one shared geography (drone 0.93, dusk 0.78, runner 0.48 block-mix; was
~0.02–0.15), the lens holds the whole world (zoom ~0.9–1.6), and divergence
between songs grew to 16–40× where seeds stay ~0.01–0.05.

**M2.5 — song-driven life, more generations, higher altitude ✔**
- *The song spawns:* colonies are no longer placed at a pre-chosen set of
  spots. Every hit lands food *and* births life there (the voice first and
  loudest, splashes to the others); the drop regime visibly floods the ground
  with new colonies where the track hits.
- *Genesis:* the world pre-simulates several generations before frame 0 (the
  amount comes from the song — intro length, onset density, pulse), so the
  opening is an old, settled ecosystem, never a blank plate, and the film's
  history starts clean at frame 0.
- *Altitude:* the lens sits higher (zoom ~0.6–1.5), holding the whole map —
  different cultures and the roads and fronts between them — in frame.

### M2.6 — the tribes are programs ✔
The M2 rule program steered *how hard* behaviours ran. Tribes go deeper: each
tribe is a small **deterministic behaviour program** (`src/song/tribe.js`) — a
tiny VM with

- **sensing** (food, own/other densities, trail, law, hunger, rim-edge, time),
- **memory** (four persistent registers, smoothed by the track's pulse),
- **actions** (move, excrete, eat, hit, spawn),
- **basic branching** (if hunger > threshold → seek food, else hold-and-ink),

compiled **by the song**: the band-correlation matrix signs who a tribe
approaches vs flees, flatness sets how much it branches on gradients, onset
density sets its spawning, the seed jitters thresholds. The world no longer
executes a hand-tuned force formula on the coarse decision grid — it runs what
each tribe's program decides, every substep. Surprising behaviour is *possible*
precisely because the programs branch: a starving tribe behaves differently
from a fed one, and the threshold is musical.

*Outcome:* the three societies are now balanced and mixed (drone 0.86 /
runner 0.87 / dusk 0.95 block-mix; populations 40–76 k, castes near-equal), the
geography holds (population CV 2.2–2.8). New tests pin the VM: programs compile
deterministically and differ by song (restless/social/hoarding/breeding
archetypes selected from the stats); branching changes behaviour; registers
persist as memory; a no-op program visibly changes the film. Realtime holds at
640×360 (tribe runs add ~4 ms/frame at 960×540 in node).

*Honest limit:* the compiler's vocabulary is still a small archetype family, so
the *structural* divergence between songs is modest (cross ≈ 0.16–0.18 vs
within-seed ≈ 0.02–0.04, ~4–7×) even though the films look different. Broadening
the instruction grammar (more branch shapes, richer memory, more action
archetypes) is the natural next step for this layer.

### M2.7 — one species, many minds: strains & allegiances ✔

Built — with one correction agreed mid-build, after watching the first films:

> **Allegiance does not change sides.** The original draft had fronts
> converting, collapsed bodies leaking their members, splinters seceding and
> friendly bodies merging. On screen that read as one thing: every colour
> blurring into every other within a minute — ugly, and it erased the very
> polities the mechanic was meant to create. So the fluid politics were
> removed: bodies are founded once and persist; war kills, it does not
> convert; there is no defection and no merger. What keeps the cast plural is
> **colonisation** — a blessed settlement on virgin ground, far from every
> body core, plants a new flag. And the renderer now paints each cell in its
> DOMINANT body's colour, so territories read as territories.

What was built:

- **strains** (`src/song/strains.js`) — the song decides how many minds it
  needs (2–5, from onset density, an independent spectral voice, texture
  noisiness, seed jitter) and compiles a LittleVM program per strain from
  motif roles: builder, grazer, scout, sentinel, breeder. Senses include the
  body's presence (KIN/FOE), the living wells, and the section's PEACE — a
  program can branch on war or truce. The world runs local selection:
  strains that thrive where they stand propagate.
- **living wells** (`src/sim/wells.js`) — a precompiled deterministic
  timeline of small, weak, numerous wells (4+ at open, ≤12 alive): born where
  a pitch class enters and holds, drifting along its harmony's compass,
  starving when it leaves. A birth is a land-rush; a death a retreat.
- **bodies & diplomacy** — allegiance is a concentration field advected
  frozen-in with the population flux (Σ bodies = density exactly, a test).
  Diplomacy = the track's consonance bent by the section's peace operator;
  hostile fronts bleed the loser (casualties, never conversion). Colonisation
  plants new bodies on blessed virgin ground.
- **a food-limited economy** — the old world was accidentally
  cap-limited: the gate floor let life grow on bare ground, so the dish
  filled to a uniform saturated carpet and stopped moving. Now the gate is
  nearly shut without food and starvation dissolves colonies off bare
  ground: the population tracks its wells, blooms on them, and must keep
  moving to keep eating.

*Outcome:* 39 tests green, including the allegiance invariant (no negatives,
composition = population), war-kills-not-converts (a war world ends poorer
than its truce twin, with each body's mass still in its own heartland),
colonisation, strain-mix drift, well determinism/drift/land-rush. Realtime
holds at 640×360 (~12 ms/frame sim+render). Songs still diverge ≫ seeds on
the fingerprint (classMix now sorted — the seed permutes the casting, the
song decides its shape).

### M2.8 — rock: the world gains an inert resource ✔

The films were alive but weightless: nothing on the map was *stuff*. Rock is
the correction — an inert resource with mass, presence and consequence:

- **the field** (`world.rock`) — a rock height per cell; above 0.55 the cell
  is impassable (advection aims mass at stone and the mass stays, piling
  against the wall; diffusion only crosses open edges). Food does not recharge
  through rock: capped ground is barren until someone quarries it open.
- **the song's weather** — the map is born with a deterministic scatter of
  stone (7–15 blobs, from the seed). Strong low/mid hits DROP rock where they
  land (and crush what they land on); strong high hits SHATTER it. The
  section regimes gained `rockfall` and `erode` operators: drops rain stone,
  breakdowns weather it away.
- **the masons** — builders gained two VM actions: **dig** (a slow bite out
  of the rock, which becomes carried burden — and grinding yields soil, a
  little food, right where they dug) and **build** (the carried burden rises
  as a wall where the ground is clear). The carry lives in per-block registers
  of the world, so stone genuinely travels: quarries open, roads run from
  them, walls rise where the colony loiters.
- **livelier organisms** — slower metabolisms (longer lives off food), faster
  strains, softer bare-ground starvation: agents travel farther between
  wells, meet more, and work more.

*Outcome:* 44 tests green, with new pins: the deterministic scatter; the
song's rockfall outpacing a mute sky; a stone wall holding a crowd west of it
for 10 s straight; quarries opening with soil and burden appearing; carried
rock rising as walls. The strips read it directly: grey masses the colonies
grow around, fronts grinding along them.

### M2.9 — flora, gardens, walls: a world of workable things ✔

After M2.8 the films had stone but nothing to DO with it, and the ecology
still lived on wells alone. This iteration added the interactable layer —
items with feedback loops, each needing its own LittleVM instructions, hence
real specialisation:

- **aimed work** — the VM gained an AIM (two pseudo-actions setting a heading;
  it defaults to the direction of travel), and every work action lands at the
  block the agent is FACING, never beneath its feet: masons carve the face of
  the wall ahead, farmers tend the bed they walk toward, strikers hit where
  the enemy stands. The work-senses (rock, garden) look where the last aim
  pointed — sight follows intent.
- **gardens** — the mason TILLS fresh stone into a cultivated bed (brought to
  maturity one cell at a time; a dusting of fertility would exhaust as fast
  as it is laid). A bed yields food slowly for a while, then the spent soil
  sets back into rock: agriculture slowly eats the world's stone, and the
  song's rockfalls replenish it. Digging itself yields NO food — stone is for
  building; food comes from wells, flora, events and gardens.
- **walls** — the carried burden rises as a wall (pooled per block, mortared
  along a rounded footprint). Walls block movement, so they defend; enemies
  strike at their aim target and SAP the stone at the front — sieges happen,
  and unmaintained walls weather away. The masons are never done.
- **farmers** — a new strain role: it walks the field rows and TENDS the beds
  (a tended bed yields up to threefold), opens new beds into fresh stone, and
  commutes along the garden gradient. Work is foraging: the hungry branches
  of mason and husbandman work the ground they aim at, because that is where
  their food comes from.
- **flora** — the world's pioneers, living in ISLANDS scattered at birth
  (a dozen small patches; they can never spread beyond their own seed
  ground). Each stand is food — grazed down by hungry crowds, never wiped out
  (seed-rain), regrowing slowly. Its vigour tracks the world's famine: a
  starving or dwindling world blooms with flora and is seeded back to life;
  a strong, well-fed world grazes it back to a whisper. Rendered as speckles
  — a few apple trees and bushes here and there, never a garden of Eden.

*Balance notes carved into the tests:* the ecology is food-limited (the gate
is nearly shut on bare ground; colonies that outgrow their ground starve back
and wander); rock has a budget that survives a whole film (calm songs keep
sparse stone, percussive ones accumulate walls); agriculture consumes stone
instead of minting it. 44 tests green; realtime holds at 640×360.

### The decision layer on the GPU (analysis, first slice landed)

The profile of a heavy frame (field 128, ~40 000 organisms planted) says:

| layer | share |
|---|---|
| the strain VMs (`runProgram`) | ~35% |
| senses/gradients/action plumbing (`runStrains`) | ~20% |
| field passes (block sums, reaction, `substep`) | ~16% |
| advection (+allegiance, burden) | ~7% |
| camera + render | ~10% |

Three moves, in the order they pay:

1. **Think slower than you move (landed).** The programs ran at 150 Hz —
   every substep — though a mind's decisions barely change in 1/30 s. The
   world now thinks every `vmEvery`-th substep (default 2: ~15 thoughts/s)
   and coasts on its last decision between thoughts; blocks with no
   population think not at all. Measured on the heavy world
   (41 k planted, sim only): 25.0 → 20.3 ms/frame at ×2, 17.7 at ×3 — with
   statistically identical ecologies (38.6 k vs 38.7 k final). The tests
   pinned the rest.
2. **The GPU (first slice landed, experimental).** The decision layer is
   ideal GPU work: m uniform programs over ~4 000 independent blocks —
   lockstep, only the data differs. `src/sim/gpu.js` holds a WebGPU compute
   path for exactly that piece: senses/grads packed on the CPU, the flat
   bytecode uploaded per strain, registers resident on the device, one
   dispatch per strain, the action buffer read back per substep.
   `tools/gpu-check.js` (browser-side) validates the GPU result against the
   CPU interpreter and times both. Status: written against the WebGPU API,
   awaiting a device to run on; NOT yet wired into `World.step` — that
   wiring needs an async stepper (the readback is asynchronous), which the
   demo can adopt and the exporter need not.
3. **The rest of the field** (block sums, advection, reaction) is the next
   and larger GPU slice — element-wise and stencil passes over the fields,
   with the population/politics staying on the CPU. That is what unlocks
   field 256 and fatter casts.

**Determinism policy.** GPU float ordering differs from the CPU's, so a
GPU-accelerated preview may drift slightly from a CPU-exported film. The
policy: the prepared-artifact/export pipeline stays CPU and remains the
canonical pixel source; the GPU path is a preview accelerator whose world
is the same world to within float noise. If preview/export bit-identity is
ever demanded, the exporter must run on the same backend as the preview —
a node WebGPU binding (e.g. a wgpu native addon) is the door for that, and
it is explicitly out of scope until the film needs it.

### M3 — the lens: travel, don't cut
Replace the cut-to-a-point grammar with a **move grammar**: *track* (follow a
moving front), *reveal* (pan along a coastline), *establish* (hold wide and
slow). Cuts are reserved for musical punctuation — strong onsets in drop/build.
Because the world is macro, a pan teaches the map instead of disorienting.

*Exit: a 120 s render where a single front can be followed across the world
without losing bearings; cuts read as beats, not jumps.*

### M4 — the film
Mux frames + PCM to video. No longer the bottleneck: the preview pipeline and
the lens exist; once M1–M3 make the world worth exporting, muxing is plumbing.

---

## The prepare step (pre-calculation)

Viewing should be fast because the *expensive* work happens ahead of time, once:

1. `tools/prepare.js` runs the WAV → analysis → compile pipeline in node and
   writes a **prepared song** (`out/prepared/<song>-<seed>.json`): the compiled
   rule program and every analysis structure the world, camera, and metrics
   need — but not the raw PCM.
2. The browser fetches the prepared song instead of analysing the WAV. The WAV
   is still fetched, but only to feed the WebAudio clock (decoded straight to a
   buffer; analysis skipped).
3. `src/song/serialize.js` is the single serialize/deserialize for both sides —
   the browser's `deserializeSong` must rebuild a song that is *bit-identical*
   to one built by analysing the WAV, so prepare can never drift from the live
   path. That equality is a test.

This is exactly the DESIGN promise inverted and kept: the pipeline still defines
the world; prepare only pays its cost once, off the viewing path.

## Invariants, restated for the field world

- **Determinism**: same song + same seed + same frame ⇒ same pixel. No
  `Math.random`, no wall clock, no per-substep RNG. The static jitter noise
  field is generated once from the seeded RNG at construction.
- **Causality**: events still place food and rings where the track put them;
  a section boundary still changes the rules at the boundary only.
- **Persistence**: population fields carry their history; trails accumulate in
  the same three tinted layers the renderer already composes.
- **Consequence**: everything is a pure update rule over the previous substep;
  nothing is re-rolled.