e6f5a2f0d5
Revisiting the framing layer turned up that it was not reaching four of the forty-two scenes, and that none of its gates could have told us. Those gates check the PLAN — cue sizes, the distribution of shot sizes across a population, headroom, seek determinism — and a plan that never reaches the image passes every one of them. Rendered at wide, normal and close, Scan Tear, Pylon Grid, Pitch Shatter and the 3D Particle Field came back byte-identical at every size. The cause was a category error in the first implementation. Framing was applied inside sigCamera, which is gated on the `camera` personality trait — so a scene that declined the track's drift and sway silently declined the shot size as well. That gating is right for a TRAIT and wrong for framing, which is not one: framing is where the camera is standing for this shot, and no scene should be exempt from it because of an unrelated art-direction decision. - Framing now lives in the shader epilogue, applied to the coordinate every fragment scene is handed, so honouring it is not optional. uv is left unframed on purpose: it is screen space, and prev() and sigGrain belong to the output image rather than to the scene being filmed. - Scan Tear and Pitch Shatter build their image from uv deliberately — a signal artefact happens to the signal, not to the world behind it. They now slice on raw uv and build the field they displace from a new framedUv(p), so the tear stays locked to the frame while the imagery behind it is filmed wide or close. - Particle Field receives framing in update() and honours it as a camera dolly, which is what framing literally is when a layer has a real camera. Distance divided by scale, matching the fragment path where the coordinate is divided by it. New gate renders instead of inspecting: 42 of 42 scenes now respond to framing, weakest Ridge Terrain at 0.22 of its own brightness, against a 0.05 floor. Also drops a stale comment on Layer.setFraming that still claimed sigCamera applied it. 105/105 checks pass including the slow set. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
268 lines
15 KiB
Markdown
268 lines
15 KiB
Markdown
# Epic 2 — from texture generator to something worth watching
|
||
|
||
PLAN.md took this from nothing to a deterministic engine with a 42-scene library, a
|
||
production-design layer, and an editor. Epic 2 is the answer to a different question, asked
|
||
after watching the result rather than after building it:
|
||
|
||
> Is it varied enough per track? Interesting enough to hold five minutes? Pleasant enough to
|
||
> watch for long? Is there a unifying factor? Are we reusing the same visuals between tracks?
|
||
> Are the visuals active enough?
|
||
|
||
The honest answers were: partly, no, mixed, yes, yes, and no in both directions. This document
|
||
is what those answers turn into.
|
||
|
||
---
|
||
|
||
## 1. What was measured
|
||
|
||
Everything below is a number off the real system, not an impression. The measurements are
|
||
reproducible from the check harness and several have become gates (§4).
|
||
|
||
**Shot rhythm is a metronome.** A five-minute track at 90 BPM gets sixteen shots of 19.3,
|
||
18.7, 18.7, 18.7 … 18.7, 19.3 seconds. At 124 BPM it is 15.5/15.5/13.6 repeating. `planShots`
|
||
picks a bar count *once per section* and then divides the section evenly, so every cut inside
|
||
a section lands on the same pulse. Over five minutes that is the most fatiguing edit rhythm
|
||
available.
|
||
|
||
**The palette is frozen for the whole track.** One `look.palette`, pushed to every layer of
|
||
every section for the entire runtime. Colour is the strongest perceptual variable the system
|
||
has and it never moves.
|
||
|
||
**39% of the library does not develop.** Rendering every scene at generated parameters and
|
||
comparing frames 0.5 s / 5 s / 30 s / 2 min apart, normalised for brightness:
|
||
|
||
| behaviour | count | examples (0.5 s → 2 min) |
|
||
|---|---|---|
|
||
| churn — changes as much in half a second as in two minutes | 11 | Curl Flow 0.81 → 0.75, Moiré Grid 1.42 → 1.40, Circuit Bloom 2.86 → 2.83 |
|
||
| nearly static | 5 | Pylon Grid 0.15 at 2 min, Dust Chamber 0.39, Salt Flat 0.64 |
|
||
| genuinely evolving | ~25 | Flora 0.29 → 3.25, Silk Ribbon 0.26 → 2.68, Slow Orb 0.09 → 1.21 |
|
||
|
||
Churn is the more interesting failure. Those scenes are violently animated and read as
|
||
*static*, because motion without structure is texture: the eye adapts in about two seconds and
|
||
then there is nothing left to find. "Active enough" fails at both ends of the range.
|
||
|
||
**The most-cast scenes are among the least developing.** Across twelve tracks, four scenes
|
||
appeared in ten to twelve of them — Dust Chamber, Slow Orb, Salt Flat, Eclipse Field — and two
|
||
of those are in the nearly-static bucket. Twenty-nine of forty-two scenes appeared in none of
|
||
the twelve.
|
||
|
||
**Every track makes the same genre decisions.** `FAMILY_BY_KIND` is a module constant: an
|
||
intro is always minimal/flow/organic, a drop always geometric/glitch/structural, and intro and
|
||
outro are literally the same list. Cross-track sameness is baked into a table.
|
||
|
||
**A five-minute track shows five or six distinct images.**
|
||
|
||
**The production-design layer works and is not in scope for this epic.** Changing only the
|
||
personality moves the image 0.846; changing only the parameters 0.882; changing only the
|
||
palette 0.467. The unifying mechanism is as strong as the variation mechanism. Leave it alone.
|
||
|
||
---
|
||
|
||
## 2. The diagnosis
|
||
|
||
Two separate problems, and it matters that they are separate:
|
||
|
||
**The system has no arc longer than fifteen seconds.** It has a per-frame arc (reactivity), a
|
||
per-shot arc (cuts), and a per-section arc (drift), and then it stops. Nothing is built to pay
|
||
off at minute four. Held attention over five minutes is a property of *structure over
|
||
minutes*, and there is currently no mechanism that operates on that timescale.
|
||
|
||
**Every scene is the same shot.** A locked-off, full-frame, abstract texture. There is no
|
||
scale, no framing, no composition — no wide against close, no negative space, no subject the
|
||
eye can track across a cut. Music-video attention is held by framing changes at least as much
|
||
as by content changes, and the vocabulary for that does not exist here at all.
|
||
|
||
The first is a set of bugs in disguise. The second is a ceiling.
|
||
|
||
---
|
||
|
||
## 3. The work
|
||
|
||
Five workstreams, ordered by impact per unit of effort. Each is independently shippable and
|
||
independently valuable; nothing here is a prerequisite for anything below it except where
|
||
noted.
|
||
|
||
### 3.1 Break the cut metronome
|
||
|
||
Shot lengths must vary *within* a section. The target is a phrase shape rather than a pulse: a
|
||
long hold, two quick ones, a long hold. Cuts still land on downbeats and still respect the
|
||
floor and ceiling — this changes where the cuts are, not the rules they obey.
|
||
|
||
The mechanism is a per-section **rhythm pattern**: a small set of bar-length sequences (`[8,
|
||
8, 4, 4, 16]` and friends) chosen by seed and energy, walked in order and repeated as needed,
|
||
instead of one bar count divided evenly. A section then reads as edited rather than as
|
||
metered.
|
||
|
||
Two things to preserve, both load-bearing and both easy to break here: cuts must stay on
|
||
phrase lines, and `MIN_SHOT_SECONDS` / `MAX_SHOT_SECONDS` remain hard.
|
||
|
||
### 3.2 A per-track director
|
||
|
||
`FAMILY_BY_KIND` becomes one of several mappings, chosen per track at look-generation time.
|
||
Each mapping is a coherent point of view about what a song looks like — one that answers a
|
||
drop with geometry, one that answers it with glitch, one that stays organic throughout, one
|
||
that treats the intro as structural rather than minimal.
|
||
|
||
This is a small change that directly attacks the largest source of cross-track sameness, and
|
||
it makes the twenty-nine unused scenes reachable without touching the library or the casting
|
||
weights.
|
||
|
||
### 3.3 Let the palette move
|
||
|
||
Per-section colour movement, driven by the arc that already exists: a hue rotation, a
|
||
saturation shift, or a scheme change into a drop. The track keeps one identity — the movement
|
||
is *from* the track's palette, not a replacement for it — so this does not fight §3.5 of
|
||
PLAN.md or the personality layer.
|
||
|
||
Constraint: the contrast floor the Phase 3 palette gate enforces has to hold at every point in
|
||
the movement, not just at the endpoints.
|
||
|
||
### 3.4 Give the churn scenes a slow axis
|
||
|
||
Each of the eleven needs at least one parameter on a multi-minute ramp, so that the frame at
|
||
three minutes is not the frame at thirty seconds. This is per-scene work and it is mechanical,
|
||
but it is the only thing that fixes the specific failure of "animated and yet static".
|
||
|
||
The five nearly-static scenes get the opposite treatment, and Phase 7's movement gate already
|
||
describes what "enough" means.
|
||
|
||
**Status after the first pass: mechanism delivered, most of the scene work still open.**
|
||
|
||
The mechanism exists and is proven. `ArcDriver._slowAxisFor` walks a declared param across the
|
||
whole track, monotonically — the existing drift is a 20-70 second LFO, and an LFO returns,
|
||
which is precisely why ten cycles of it over five minutes reads as static.
|
||
|
||
Two things were learned the hard way and are worth not relearning:
|
||
|
||
*Which param you move decides everything.* The first version chose at random from everything
|
||
eligible and measured as doing **nothing whatsoever** — identical structural change with the
|
||
axis applied and with it disabled. Sweeping Moiré Grid's `width` moves its time-averaged
|
||
structure by 0.110 and its `offset` by 0.002; a random draw finds the second kind almost every
|
||
time. Hence `slowAxis: true` as a declaration rather than a heuristic.
|
||
|
||
*Single-frame distance cannot measure this.* A churning scene's consecutive frames are already
|
||
~0.6 apart, so every pair of its frames scores the same whether the structure moved or not.
|
||
The gate measures **ten-second time-averaged** frames, which cancels the churn and leaves the
|
||
structure. The window was measured: at one second, Moiré Grid's frozen-parameter control still
|
||
reads 0.024; at ten it reads 0.0095 while the axis-driven signal stays at 0.037.
|
||
|
||
Measured across ten candidate scenes (ratio of axis-driven structural change to what the scene
|
||
does on its own):
|
||
|
||
| works | Moiré Grid 3.93× · Gate Corridor 2.88× · Truchet Fold 1.40× |
|
||
|---|---|
|
||
| **no parameter helps** | Curl Flow 1.31× · Signal Decay 1.21× · Circuit Bloom 1.10× |
|
||
| **already develops; never the problem** | Firefly Drift · Vortex Drift · Kaleido Tunnel · Plasma Bloom |
|
||
|
||
The middle row is the remaining work, and it is **not** mechanical: those scenes have no
|
||
parameter that changes their structure, so they need shader changes that introduce one.
|
||
Parameter automation cannot substitute for structure a scene does not have.
|
||
|
||
**Closed by side quest 2** (SIDE-QUESTS.md §2). All three got the structure they were missing,
|
||
and the measurement was corrected in the process — the ratios above were produced by an
|
||
experiment that compared two different points in the SONG, so its denominator was full of audio
|
||
reactivity and it was really asking "does the axis move this scene more than the music does".
|
||
Circuit Bloom failed at 0.92× while its mean luminance moved 5.6× across the axis. The gate now
|
||
holds time constant and varies only the axis, against the metric's own noise floor. Current
|
||
figures: Moiré Grid 28.7× · Gate Corridor 14.3× · Truchet Fold 3.3× · Signal Decay 1.9× ·
|
||
Curl Flow 1.7× · Circuit Bloom 1.4×, against a 1.25× bar, with the axis disabled reading 0.00×.
|
||
|
||
### 3.5 A framing layer
|
||
|
||
The one that raises the ceiling rather than the floor. A shared zoom / crop / scale envelope
|
||
sitting above the scenes, so the same image can be played as a wide and as a close, and so a
|
||
cut can change the framing without changing the subject.
|
||
|
||
This is deliberately last: it is the largest change, it interacts with resolution
|
||
independence (§1 of PLAN.md) and with the feedback buffer, and the four items above will have
|
||
changed what it needs to do.
|
||
|
||
**Status after the first pass: shipped, deliberately simple.**
|
||
|
||
A shot now carries a size and a recentre, applied inside `sigCamera` in scene coordinates —
|
||
rendered close rather than magnified, which is what keeps it resolution-independent and free at
|
||
4K. `look/framing.js` owns the vocabulary: three shot sizes with measured headroom (a close-up
|
||
past ~2.2 is a blurry wide, a wide past ~0.55 is a speck), and a per-track style (`locked` /
|
||
`gentle` / `edited`) that decides how far sizes travel and how often a cut also changes the shot.
|
||
The arc driver plans one framing per cue, walking the cuts in order so the whole video gets a
|
||
consistent hand and a seek finds the same framing as playback. The gates hold it honest: cuts
|
||
actually change size, sizes stay in headroom, and two drivers over one look agree on every shot.
|
||
|
||
What this pass does not do, and the reason it is "simple": the framing is constant within a shot.
|
||
A zoom that moves during a shot is a separate device and would fight the drift LFO and the slow
|
||
axis, both of which already own continuous motion.
|
||
|
||
**Revisited, and it was not reaching four scenes.**
|
||
|
||
The gates above check the PLAN — cue sizes, distribution, headroom, determinism — and a plan
|
||
that never reaches the image passes every one of them. It did. Framing was applied inside
|
||
`sigCamera`, which is gated on the `camera` personality trait, so a scene that declined the
|
||
track's drift silently declined the shot size too. That is correct for a trait and wrong for
|
||
framing, which is not one: it is where the camera is standing, and no scene should be exempt
|
||
from it because of an unrelated art-direction decision. Rendered, Scan Tear, Pylon Grid, Pitch
|
||
Shatter and the 3D layer were identical at every size, and nothing noticed.
|
||
|
||
Three fixes:
|
||
|
||
- Framing moved out of `sigCamera` and into the shader epilogue, applied to the coordinate every
|
||
fragment scene is handed. `uv` is deliberately left unframed — it is screen space, and `prev()`
|
||
and `sigGrain` belong to the output image rather than to the scene being filmed.
|
||
- Screen-space glitch scenes (Scan Tear, Pitch Shatter) build their image from `uv` on purpose:
|
||
a signal artefact happens to the signal, not to the world behind it. They now slice on raw
|
||
`uv` and build the field they displace from `framedUv(p)`, so the tear stays locked to the
|
||
frame while the imagery behind it is filmed wide or close.
|
||
- The 3D layer receives `framing` in `update` and honours it as an actual camera dolly, which is
|
||
what framing literally is when a scene has a real camera.
|
||
|
||
And a gate that renders rather than inspecting the plan: **42 of 42 scenes now respond to
|
||
framing**, weakest Ridge Terrain at 0.22 of its own brightness.
|
||
|
||
---
|
||
|
||
## 4. Validation
|
||
|
||
The rule from PLAN.md §11 stands: variety is a property of a population, not of one render, so
|
||
every gate here samples many tracks and asks about a spread.
|
||
|
||
| workstream | gate |
|
||
|---|---|
|
||
| 3.1 rhythm | Within one section, shot lengths must have real spread (no section where every shot is within a few percent of the mean). Cuts still land on downbeats; floor and ceiling still hold. |
|
||
| 3.2 director | Across a population, the same section kind must draw from more than one family set. Every scene in the library must be reachable by some director. No director may leave a kind with too few scenes to build a roster from. |
|
||
| 3.3 palette | Contrast floor holds at every sampled point of the movement, not just the endpoints. Colour actually moves across a track — measurably, not decoratively. |
|
||
| 3.4 slow axis | Every scene's two-minute frame distance must exceed its half-second distance by a real margin. This is the churn number from §1, promoted to a gate. |
|
||
| 3.5 framing | Resolution independence survives (the existing dual-resolution diff). Framing changes are visible. Determinism holds. |
|
||
|
||
Two failure modes are worth naming in advance, because both have already happened once in this
|
||
project's history:
|
||
|
||
- **A gate that measures the wrong thing.** The Phase 3 look-space check was propped up by
|
||
grain until grain was removed. Any new gate here should be sanity-checked by deliberately
|
||
breaking the thing it is supposed to catch.
|
||
- **A change that a gate rewards and a viewer does not.** Cut variety that is *random* rather
|
||
than *phrased* would pass 3.1's gate and look worse. The gates are necessary and not
|
||
sufficient; the end-to-end watch stays the real test.
|
||
|
||
---
|
||
|
||
## 5. Sequencing
|
||
|
||
1. **3.1 rhythm** — biggest felt change, smallest diff, no dependencies.
|
||
2. **3.2 director** — small, orthogonal, attacks cross-track sameness directly.
|
||
3. **3.3 palette** — medium, touches the arc driver.
|
||
4. **3.4 slow axis** — large but mechanical, eleven-plus scenes, one at a time.
|
||
5. **3.5 framing** — largest, and best attempted once the other four have landed.
|
||
|
||
Each ships as its own commit with its gate green.
|
||
|
||
---
|
||
|
||
## 6. What this epic does not do
|
||
|
||
- **It does not touch the personality layer.** Measured, it works.
|
||
- **It does not grow the library.** Twenty-nine scenes are currently unreachable; making them
|
||
reachable (3.2) is worth more than adding a forty-third.
|
||
- **It does not add audio analysis.** Every mechanism here is driven by features that already
|
||
exist in the FeatureTrack.
|
||
- **It does not relax determinism.** Every rule in PLAN.md §1 still applies, and the seek /
|
||
export / repeat gates are non-negotiable.
|