236 lines
13 KiB
Markdown
236 lines
13 KiB
Markdown
# Epic 2 — from texture generator to something worth watching
|
||
|
||
PLAN.md took this from nothing to a deterministic engine with a 42-scene library, a
|
||
production-design layer, and an editor. Epic 2 is the answer to a different question, asked
|
||
after watching the result rather than after building it:
|
||
|
||
> Is it varied enough per track? Interesting enough to hold five minutes? Pleasant enough to
|
||
> watch for long? Is there a unifying factor? Are we reusing the same visuals between tracks?
|
||
> Are the visuals active enough?
|
||
|
||
The honest answers were: partly, no, mixed, yes, yes, and no in both directions. This document
|
||
is what those answers turn into.
|
||
|
||
---
|
||
|
||
## 1. What was measured
|
||
|
||
Everything below is a number off the real system, not an impression. The measurements are
|
||
reproducible from the check harness and several have become gates (§4).
|
||
|
||
**Shot rhythm is a metronome.** A five-minute track at 90 BPM gets sixteen shots of 19.3,
|
||
18.7, 18.7, 18.7 … 18.7, 19.3 seconds. At 124 BPM it is 15.5/15.5/13.6 repeating. `planShots`
|
||
picks a bar count *once per section* and then divides the section evenly, so every cut inside
|
||
a section lands on the same pulse. Over five minutes that is the most fatiguing edit rhythm
|
||
available.
|
||
|
||
**The palette is frozen for the whole track.** One `look.palette`, pushed to every layer of
|
||
every section for the entire runtime. Colour is the strongest perceptual variable the system
|
||
has and it never moves.
|
||
|
||
**39% of the library does not develop.** Rendering every scene at generated parameters and
|
||
comparing frames 0.5 s / 5 s / 30 s / 2 min apart, normalised for brightness:
|
||
|
||
| behaviour | count | examples (0.5 s → 2 min) |
|
||
|---|---|---|
|
||
| churn — changes as much in half a second as in two minutes | 11 | Curl Flow 0.81 → 0.75, Moiré Grid 1.42 → 1.40, Circuit Bloom 2.86 → 2.83 |
|
||
| nearly static | 5 | Pylon Grid 0.15 at 2 min, Dust Chamber 0.39, Salt Flat 0.64 |
|
||
| genuinely evolving | ~25 | Flora 0.29 → 3.25, Silk Ribbon 0.26 → 2.68, Slow Orb 0.09 → 1.21 |
|
||
|
||
Churn is the more interesting failure. Those scenes are violently animated and read as
|
||
*static*, because motion without structure is texture: the eye adapts in about two seconds and
|
||
then there is nothing left to find. "Active enough" fails at both ends of the range.
|
||
|
||
**The most-cast scenes are among the least developing.** Across twelve tracks, four scenes
|
||
appeared in ten to twelve of them — Dust Chamber, Slow Orb, Salt Flat, Eclipse Field — and two
|
||
of those are in the nearly-static bucket. Twenty-nine of forty-two scenes appeared in none of
|
||
the twelve.
|
||
|
||
**Every track makes the same genre decisions.** `FAMILY_BY_KIND` is a module constant: an
|
||
intro is always minimal/flow/organic, a drop always geometric/glitch/structural, and intro and
|
||
outro are literally the same list. Cross-track sameness is baked into a table.
|
||
|
||
**A five-minute track shows five or six distinct images.**
|
||
|
||
**The production-design layer works and is not in scope for this epic.** Changing only the
|
||
personality moves the image 0.846; changing only the parameters 0.882; changing only the
|
||
palette 0.467. The unifying mechanism is as strong as the variation mechanism. Leave it alone.
|
||
|
||
---
|
||
|
||
## 2. The diagnosis
|
||
|
||
Two separate problems, and it matters that they are separate:
|
||
|
||
**The system has no arc longer than fifteen seconds.** It has a per-frame arc (reactivity), a
|
||
per-shot arc (cuts), and a per-section arc (drift), and then it stops. Nothing is built to pay
|
||
off at minute four. Held attention over five minutes is a property of *structure over
|
||
minutes*, and there is currently no mechanism that operates on that timescale.
|
||
|
||
**Every scene is the same shot.** A locked-off, full-frame, abstract texture. There is no
|
||
scale, no framing, no composition — no wide against close, no negative space, no subject the
|
||
eye can track across a cut. Music-video attention is held by framing changes at least as much
|
||
as by content changes, and the vocabulary for that does not exist here at all.
|
||
|
||
The first is a set of bugs in disguise. The second is a ceiling.
|
||
|
||
---
|
||
|
||
## 3. The work
|
||
|
||
Five workstreams, ordered by impact per unit of effort. Each is independently shippable and
|
||
independently valuable; nothing here is a prerequisite for anything below it except where
|
||
noted.
|
||
|
||
### 3.1 Break the cut metronome
|
||
|
||
Shot lengths must vary *within* a section. The target is a phrase shape rather than a pulse: a
|
||
long hold, two quick ones, a long hold. Cuts still land on downbeats and still respect the
|
||
floor and ceiling — this changes where the cuts are, not the rules they obey.
|
||
|
||
The mechanism is a per-section **rhythm pattern**: a small set of bar-length sequences (`[8,
|
||
8, 4, 4, 16]` and friends) chosen by seed and energy, walked in order and repeated as needed,
|
||
instead of one bar count divided evenly. A section then reads as edited rather than as
|
||
metered.
|
||
|
||
Two things to preserve, both load-bearing and both easy to break here: cuts must stay on
|
||
phrase lines, and `MIN_SHOT_SECONDS` / `MAX_SHOT_SECONDS` remain hard.
|
||
|
||
### 3.2 A per-track director
|
||
|
||
`FAMILY_BY_KIND` becomes one of several mappings, chosen per track at look-generation time.
|
||
Each mapping is a coherent point of view about what a song looks like — one that answers a
|
||
drop with geometry, one that answers it with glitch, one that stays organic throughout, one
|
||
that treats the intro as structural rather than minimal.
|
||
|
||
This is a small change that directly attacks the largest source of cross-track sameness, and
|
||
it makes the twenty-nine unused scenes reachable without touching the library or the casting
|
||
weights.
|
||
|
||
### 3.3 Let the palette move
|
||
|
||
Per-section colour movement, driven by the arc that already exists: a hue rotation, a
|
||
saturation shift, or a scheme change into a drop. The track keeps one identity — the movement
|
||
is *from* the track's palette, not a replacement for it — so this does not fight §3.5 of
|
||
PLAN.md or the personality layer.
|
||
|
||
Constraint: the contrast floor the Phase 3 palette gate enforces has to hold at every point in
|
||
the movement, not just at the endpoints.
|
||
|
||
### 3.4 Give the churn scenes a slow axis
|
||
|
||
Each of the eleven needs at least one parameter on a multi-minute ramp, so that the frame at
|
||
three minutes is not the frame at thirty seconds. This is per-scene work and it is mechanical,
|
||
but it is the only thing that fixes the specific failure of "animated and yet static".
|
||
|
||
The five nearly-static scenes get the opposite treatment, and Phase 7's movement gate already
|
||
describes what "enough" means.
|
||
|
||
**Status after the first pass: mechanism delivered, most of the scene work still open.**
|
||
|
||
The mechanism exists and is proven. `ArcDriver._slowAxisFor` walks a declared param across the
|
||
whole track, monotonically — the existing drift is a 20-70 second LFO, and an LFO returns,
|
||
which is precisely why ten cycles of it over five minutes reads as static.
|
||
|
||
Two things were learned the hard way and are worth not relearning:
|
||
|
||
*Which param you move decides everything.* The first version chose at random from everything
|
||
eligible and measured as doing **nothing whatsoever** — identical structural change with the
|
||
axis applied and with it disabled. Sweeping Moiré Grid's `width` moves its time-averaged
|
||
structure by 0.110 and its `offset` by 0.002; a random draw finds the second kind almost every
|
||
time. Hence `slowAxis: true` as a declaration rather than a heuristic.
|
||
|
||
*Single-frame distance cannot measure this.* A churning scene's consecutive frames are already
|
||
~0.6 apart, so every pair of its frames scores the same whether the structure moved or not.
|
||
The gate measures **ten-second time-averaged** frames, which cancels the churn and leaves the
|
||
structure. The window was measured: at one second, Moiré Grid's frozen-parameter control still
|
||
reads 0.024; at ten it reads 0.0095 while the axis-driven signal stays at 0.037.
|
||
|
||
Measured across ten candidate scenes (ratio of axis-driven structural change to what the scene
|
||
does on its own):
|
||
|
||
| works | Moiré Grid 3.93× · Gate Corridor 2.88× · Truchet Fold 1.40× |
|
||
|---|---|
|
||
| **no parameter helps** | Curl Flow 1.31× · Signal Decay 1.21× · Circuit Bloom 1.10× |
|
||
| **already develops; never the problem** | Firefly Drift · Vortex Drift · Kaleido Tunnel · Plasma Bloom |
|
||
|
||
The middle row is the remaining work, and it is **not** mechanical: those scenes have no
|
||
parameter that changes their structure, so they need shader changes that introduce one.
|
||
Parameter automation cannot substitute for structure a scene does not have.
|
||
|
||
### 3.5 A framing layer
|
||
|
||
The one that raises the ceiling rather than the floor. A shared zoom / crop / scale envelope
|
||
sitting above the scenes, so the same image can be played as a wide and as a close, and so a
|
||
cut can change the framing without changing the subject.
|
||
|
||
This is deliberately last: it is the largest change, it interacts with resolution
|
||
independence (§1 of PLAN.md) and with the feedback buffer, and the four items above will have
|
||
changed what it needs to do.
|
||
|
||
**Status after the first pass: shipped, deliberately simple.**
|
||
|
||
A shot now carries a size and a recentre, applied inside `sigCamera` in scene coordinates —
|
||
rendered close rather than magnified, which is what keeps it resolution-independent and free at
|
||
4K. `look/framing.js` owns the vocabulary: three shot sizes with measured headroom (a close-up
|
||
past ~2.2 is a blurry wide, a wide past ~0.55 is a speck), and a per-track style (`locked` /
|
||
`gentle` / `edited`) that decides how far sizes travel and how often a cut also changes the shot.
|
||
The arc driver plans one framing per cue, walking the cuts in order so the whole video gets a
|
||
consistent hand and a seek finds the same framing as playback. The gates hold it honest: cuts
|
||
actually change size, sizes stay in headroom, and two drivers over one look agree on every shot.
|
||
|
||
What this pass does not do, and the reason it is "simple": the framing is constant within a shot.
|
||
A zoom that moves during a shot is a separate device and would fight the drift LFO and the slow
|
||
axis, both of which already own continuous motion. Scenes built outside the shader contract (the
|
||
single 3D layer) are not framed yet. Both are the obvious next steps and neither is required for
|
||
a working project.
|
||
|
||
---
|
||
|
||
## 4. Validation
|
||
|
||
The rule from PLAN.md §11 stands: variety is a property of a population, not of one render, so
|
||
every gate here samples many tracks and asks about a spread.
|
||
|
||
| workstream | gate |
|
||
|---|---|
|
||
| 3.1 rhythm | Within one section, shot lengths must have real spread (no section where every shot is within a few percent of the mean). Cuts still land on downbeats; floor and ceiling still hold. |
|
||
| 3.2 director | Across a population, the same section kind must draw from more than one family set. Every scene in the library must be reachable by some director. No director may leave a kind with too few scenes to build a roster from. |
|
||
| 3.3 palette | Contrast floor holds at every sampled point of the movement, not just the endpoints. Colour actually moves across a track — measurably, not decoratively. |
|
||
| 3.4 slow axis | Every scene's two-minute frame distance must exceed its half-second distance by a real margin. This is the churn number from §1, promoted to a gate. |
|
||
| 3.5 framing | Resolution independence survives (the existing dual-resolution diff). Framing changes are visible. Determinism holds. |
|
||
|
||
Two failure modes are worth naming in advance, because both have already happened once in this
|
||
project's history:
|
||
|
||
- **A gate that measures the wrong thing.** The Phase 3 look-space check was propped up by
|
||
grain until grain was removed. Any new gate here should be sanity-checked by deliberately
|
||
breaking the thing it is supposed to catch.
|
||
- **A change that a gate rewards and a viewer does not.** Cut variety that is *random* rather
|
||
than *phrased* would pass 3.1's gate and look worse. The gates are necessary and not
|
||
sufficient; the end-to-end watch stays the real test.
|
||
|
||
---
|
||
|
||
## 5. Sequencing
|
||
|
||
1. **3.1 rhythm** — biggest felt change, smallest diff, no dependencies.
|
||
2. **3.2 director** — small, orthogonal, attacks cross-track sameness directly.
|
||
3. **3.3 palette** — medium, touches the arc driver.
|
||
4. **3.4 slow axis** — large but mechanical, eleven-plus scenes, one at a time.
|
||
5. **3.5 framing** — largest, and best attempted once the other four have landed.
|
||
|
||
Each ships as its own commit with its gate green.
|
||
|
||
---
|
||
|
||
## 6. What this epic does not do
|
||
|
||
- **It does not touch the personality layer.** Measured, it works.
|
||
- **It does not grow the library.** Twenty-nine scenes are currently unreachable; making them
|
||
reachable (3.2) is worth more than adding a forty-third.
|
||
- **It does not add audio analysis.** Every mechanism here is driven by features that already
|
||
exist in the FeatureTrack.
|
||
- **It does not relax determinism.** Every rule in PLAN.md §1 still applies, and the seek /
|
||
export / repeat gates are non-negotiable.
|