Every shot stands on something, and the frame has two ends
A section used to be one scene, and two thirds of the library is composable —
sparse by design, elements ON something. Cast as backgrounds anyway, they left
9 of 40 sampled frames under 20% painted, the darkest at 0.3%: a minute and a
half of a few bright things on black, invisible to every gate because every
gate on the stack was a limit rather than a floor.
Every section now stands on a GROUND: a canvas that fills the frame, cast per
section kind so a shot cut changes the shot and not the world. When the shot
fills the frame itself it IS the ground — two canvases stacked is two pictures
fighting. Above that, a coverage BUDGET: director appetite times the section's
energy times where the story is, capped at two frames' worth of material.
The measured facts move into the repo. scenes/metadata.json is generated from
the gallery — coverage as a shot, coverage as a bed, variety, the structural
profile — tracked in git, stamped with a fingerprint of the scenes and the
metric definitions, and refreshed from gallery.html. `surface` is derived from
it rather than declared; nine scenes claimed `canvas` while painting under a
third of the frame, and declaring it is now a lint error. The generator weights
every layering choice by measured structural distance, because family labels
and the render disagree: two `geometric` scenes can be 0.31 apart and a `flow`
and an `organic` scene 0.04.
The gallery's 0.1 red line is gone. It was right when a section was one scene
and wrong now — nineteen scenes were failing a bar for being consistent, which
is a virtue in an ingredient.
Chasing the numbers turned up four real faults:
* A scene that reads prev() cannot be a ground. It returns the whole
composited frame including the layers above it, so a datamosh under a shot
is eating it: the render stopped reproducing from a seek and two WebGL
contexts diverged by 91/255 against a tolerance of 4.
* Screen was the wrong operator for a shot over a bed. It lightens, so a
median quarter of every frame clipped to paper and whole sections rendered
100% white. Replaced by a lumakey — the shot's brightness is its alpha.
* Feedback was an accumulator: a still image settled at 2.3x its own
brightness. Fine over black, fatal over a filled ground. Normalised at 0.6,
plus a highlight shoulder so the top rolls off instead of clipping.
* useTrack never prewarmed, so a fresh Show's first frame differed from every
later render of it — the export-breaking hazard Compositor.prime documents.
Blazing is a decision now, not a side effect: directors declare an appetite for
it, a section must be loud and late in the story to earn one, and quiet kinds
never do. The ceiling gate matches that — a hard cap per section, and no more
than a fifth of them hot at all.
Rendered across twelve videos, middle of every section:
painted 51% mean, darkest 0.3% -> 87% mean, darkest 43%
clipped white 24% median, worst 100% -> 1% mean, worst 30%
separation 0.10 -> 0.44
Seven scenes can ground a section — five geometric, two organic — so every
quiet section of every video stands on one of two beds. That is the library's
largest hole and it is scene work: there is no minimal or flow canvas that
fills half the frame without reading prev().
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Dejvino
parent
7d151be6e1
commit
89e05459c0
@ -394,3 +394,191 @@ do it once the numbers were understood, and they now are.
|
||||
written — direction clears the arcless reference by 0.26, which is hard to explain without
|
||||
a story — but it remains the only thing that would attribute the floor rise (0.1697 →
|
||||
0.2130) to this work rather than to everything else that changed alongside it.
|
||||
|
||||
---
|
||||
|
||||
## 14. Composition — the frame has a floor and a ceiling
|
||||
|
||||
Not part of the original plan. It comes out of the same place §12 did: the layer stack was
|
||||
being asked to carry the video, and half of what it was stacking was nothing.
|
||||
|
||||
**The floor.** Two thirds of the library is `composable` — sparse by design, elements ON
|
||||
something. `canBackground` let almost all of it anchor a section anyway, so a section's
|
||||
bottom layer was regularly a scene painting 2% of the frame with black behind it. Measured
|
||||
across six videos, sampling the middle of every section: **9 of 40 frames were under 20%
|
||||
painted, the darkest at 0.3%**. That is a minute and a half of a few bright things on black,
|
||||
and no gate could see it, because every gate on the stack was a limit rather than a floor.
|
||||
|
||||
Every section now stands on a GROUND: a canvas painting at least half the frame, cast per
|
||||
section kind so a section's shot cuts change the shot and not the world. When the shot is
|
||||
itself a full canvas it *is* the ground — two canvases stacked is two pictures fighting. The
|
||||
same measurement now reads **mean 92% painted, darkest 63%, nothing under 30%**, and phase 12
|
||||
gates it in the image, not in the spec.
|
||||
|
||||
**The ceiling.** Layering had a rate but no budget, so how full a frame got was a die roll
|
||||
that knew nothing about the song. A stack now has a coverage budget — director appetite
|
||||
(`crowd`, 1.05 brutalist to 1.7 corrupt), times the section's energy and density, times where
|
||||
the story is — capped at **200%**, two frames' worth of material. The ground and the shot are
|
||||
paid for first; the budget governs what may be stacked on top, and the odds of an overlay
|
||||
fall off as the headroom does rather than only at the wall.
|
||||
|
||||
**What it cost, and what it bought.** Same instrument, same six seeds:
|
||||
|
||||
```
|
||||
baseline with grounds
|
||||
separation 0.10 0.44
|
||||
direction 0.06 0.21 (arcless reference 0.02 → 0.08)
|
||||
library 80% 97% of what the library can express
|
||||
scale 42% 56%
|
||||
motion 68% 96%
|
||||
texture 74% 90%
|
||||
```
|
||||
|
||||
The ceiling *fell* (0.2334 → 0.2031) while observed rose, so part of that separation gain is
|
||||
the reference arm coming down: videos that share a ground vocabulary are less unalike even
|
||||
when they share no shots. The honest reading is that the floor rose 0.009, direction tripled,
|
||||
and the ceiling moved toward the floor — the numbers are up, and not all of the rise is
|
||||
signal.
|
||||
|
||||
**A scene that reads the previous frame cannot be a ground.** `prev()` returns the whole
|
||||
composited frame, *including whatever is layered on top of this scene*, so a datamosh under a
|
||||
shot is not grounding it, it is eating it. Measured the moment one became a bed: the render
|
||||
stopped reproducing from a seek, and two WebGL contexts diverged by 91/255 against a ceiling
|
||||
of 4 — phase 4 caught both. `readsHistory` is derived from the shader source rather than
|
||||
declared, and costs the ground pool six of twenty scenes, five of them glitch.
|
||||
|
||||
`scenes/coverage.js` is the measured coverage of every scene, pasted back out of the phase 12
|
||||
gate, which re-measures and fails on drift over a tenth of a frame. A new scene has no entry
|
||||
and therefore cannot ground anything until someone renders it.
|
||||
|
||||
### 14.1 The measurements moved into the repo
|
||||
|
||||
Everything in §14 depends on a number — how much of the frame a scene paints — and the
|
||||
first version of it was a table pasted into a source file by hand. That is the same mistake
|
||||
`surface` already was: a measured fact written down by a person, correct on the day and
|
||||
wrong after the next shader edit.
|
||||
|
||||
**`src/scenes/metadata.json`** is now generated, tracked in git, and holds every measured
|
||||
fact about every visualizer: coverage as a shot, coverage as a bed, the variety score, the
|
||||
per-block scores, and the mean structural profile. Nothing in it is typed by a human.
|
||||
`surface` is derived from it (canvas at 50%, composable below) and declaring `surface:` in a
|
||||
scene file is now a lint error.
|
||||
|
||||
It is refreshed from **gallery.html → refresh metadata**, which re-renders the library and
|
||||
writes the file through a dev-only endpoint in `vite.config.js`. The file carries a
|
||||
fingerprint of everything that can move a number in it — the scenes, the shader contract,
|
||||
the identities and palettes they are handed, the descriptor definitions — and phase 12 fails
|
||||
when that stops matching. A stale metadata file does not produce a stale report, it produces
|
||||
wrong videos, so it has to be detectable.
|
||||
|
||||
**Measuring at the wrong bias measures nothing.** Coverage is mostly a function of a scene's
|
||||
parameters, and the first measurement took one number per scene at the busiest section of
|
||||
six songs. Metaballs measures 69% there and painted **6%** as an intro's ground. Three
|
||||
things came out of chasing that, in order of how much each was worth:
|
||||
|
||||
* A bed is drawn with a **moderate hand**. `extremity` pushes parameters toward the ends of
|
||||
their ranges, and half of what decides coverage is a parameter with no `bias` key at all —
|
||||
Metaballs' `threshold` — so extremity is the only thing moving it and one end is an empty
|
||||
frame. Grounds sample at `extremity * 0.2`. The extremes belong to the shot.
|
||||
* Energy and density are **floors** for a ground, not reductions. Calming the bed by
|
||||
lowering energy is how to empty it: scenes bias their fill against `energy` as often as
|
||||
against `density`.
|
||||
* Coverage is measured **twice**, once as a shot and once as a bed, at the same bias and
|
||||
temperament the generator will use. `GROUND_BIAS` and `groundTemperamentFrom` live in
|
||||
`scenes/surface.js` and are imported by both the generator and the measuring pass, so the
|
||||
two cannot drift apart.
|
||||
|
||||
Rendered result across six videos, sampling the middle of every section: **mean 87% painted,
|
||||
darkest 30%**, against 51%/0.3% before any of this.
|
||||
|
||||
**Eleven scenes can ground a section** — five organic, five geometric, one structural. There
|
||||
is no `minimal`, `flow` or `glitch` canvas in the library that paints half the frame without
|
||||
reading `prev()`, so every quiet section in every video stands on one of five organic beds.
|
||||
That is the largest remaining hole and it is library work, not generator work: the fix is
|
||||
writing dense, self-contained canvases in the three thin families.
|
||||
|
||||
### 14.2 The red line is gone
|
||||
|
||||
`MIN_VARIETY` — 0.1, drawn across the gallery, "below this a scene is the same picture
|
||||
wherever it appears" — was right when a section was one scene and is wrong now. A section is
|
||||
a ground, a shot on it and sometimes a pass over that, so what a viewer sees is a
|
||||
combination, and a scene that is reliably itself is a good ingredient in one. Nineteen
|
||||
scenes were failing a bar for being consistent.
|
||||
|
||||
What replaced it is a level up, and it is the reason the profiles are in the metadata: the
|
||||
generator weights every layering choice by **structural distance** — how unalike two scenes
|
||||
measured, on the same descriptor the variety harness compares videos with. Family labels and
|
||||
the render disagree often enough to matter; two `geometric` scenes can be 0.31 apart and a
|
||||
`flow` and an `organic` scene 0.04, and stacking the second pair is one picture at double
|
||||
density. Measured over 436 stacked pairs: **mean distance 0.268, zero near-twins**.
|
||||
|
||||
### 14.3 The other end of the frame
|
||||
|
||||
The floor had a ceiling missing. Measured across twelve videos, sampling the middle of every
|
||||
section: **a median 24% of every frame was clipped to pure white, and whole sections
|
||||
rendered at 100%.** The videos were washed out, and no gate could see it because every guard
|
||||
on the frame asked whether there was enough in it.
|
||||
|
||||
Three causes, all introduced or exposed by §14, in the order they were found:
|
||||
|
||||
**The shot was screened over its ground.** Screen is a lightening operator — correct for a
|
||||
few bright elements over a bed, and over a filled canvas it drives everything toward white.
|
||||
Replaced with a **lumakey** blend (`passes.js`): the shot's own brightness is its alpha, so
|
||||
it replaces the ground where it paints and leaves it where it does not, keeping its own
|
||||
colour instead of adding it to the bed's. Median clipping 24% → 11%.
|
||||
|
||||
**The feedback loop was an accumulator.** `cur + hist * decay * amount` settles a still
|
||||
image at `1/(1 - decay*amount)` times its drawn brightness — **2.3x** at the settings the
|
||||
generator hands out. Survivable when a frame was a few bright things on black; fatal the
|
||||
moment every section stood on a filled ground. Turning feedback off took a blown frame from
|
||||
100% to 34% mean luminance, which is the whole diagnosis in one number. It now divides by
|
||||
the same gain, at 0.6 rather than 1.0: full normalisation took the lift out along with the
|
||||
blowout — the median frame fell from 95% painted to 71% and fourteen sections dropped
|
||||
through the black-frame floor.
|
||||
|
||||
**Nothing rolled off.** The grade clipped. A **highlight shoulder** now compresses everything
|
||||
above 0.75 toward but never to 1. Clipped white is not brightness, it is missing information:
|
||||
every difference inside it has been deleted.
|
||||
|
||||
Rendered, after all three — 74 sections, twelve videos:
|
||||
|
||||
```
|
||||
before after
|
||||
painted 51% mean 88% mean, darkest 43% (was darkest 0.3%)
|
||||
clipped white 24% median 0% median, worst 21% (was worst 100%)
|
||||
luminance 77% median 42% median, worst 80%
|
||||
```
|
||||
|
||||
The gate is now two-ended: `composition · a rendered section is neither black nor blown out`.
|
||||
|
||||
**A ground has to survive the song's identity.** Chasing the dark end turned up the same
|
||||
class of error as §14.1: a ground measured at 61% painted **2%** in one particular video,
|
||||
because that song's ink treatment is `hollow` — outlines, no fill. Two changes came out of
|
||||
it. The world is now drawn solid and the song's hand is kept for the subject
|
||||
(`groundPersonalityFrom`), and eligibility takes the **worst** identity into account as well
|
||||
as the mean: a ground fills the frame on average and never vanishes. Holding the worst case
|
||||
to the full 50% would leave *four* castable beds in the library, which is a worse video than
|
||||
an occasionally dim intro.
|
||||
|
||||
**Seven scenes can ground a section** — five geometric, two organic. Every quiet section of
|
||||
every video stands on one of two beds. Restated from §14.1 because it got worse, not better:
|
||||
this is the library's largest hole.
|
||||
|
||||
### 14.4 Blazing on purpose
|
||||
|
||||
Screening the shot over its ground and letting the two brightnesses sum is a real effect — a
|
||||
drop that goes to paper for eight bars reads as the song peaking. The mistake in §14.3 was
|
||||
not the effect, it was that the effect was the **default**: every section did it, so a median
|
||||
quarter of every frame in every video was clipped, and nothing about that said "peak".
|
||||
|
||||
It is a decision now. Each director declares a `blaze` appetite — brutalist 0.05 (mass does
|
||||
not glow), corrupt 0.5 (overload is the subject) — and a section still has to earn one: loud,
|
||||
and late in the story. Quiet kinds never blaze, because a breakdown that goes white is not a
|
||||
choice, it is a bug with a rationale. The default composite is the lumakey, which keeps the
|
||||
shot's own colour.
|
||||
|
||||
The ceiling gate matches the intent rather than banning brightness. Per section, only a hard
|
||||
cap — past about half the frame at full white there is no picture left to read. Across the
|
||||
population, **no more than a fifth of sections may be hot at all**, which is the number that
|
||||
actually distinguishes a director choosing to peak from a pipeline with no headroom.
|
||||
Measured now: **1 of 74 sections blazing**, worst 30% clipped.
|
||||
|
||||
@ -28,16 +28,23 @@ checks.html?scene=X the per-scene gate, including `consumes:` lines
|
||||
```
|
||||
|
||||
The gallery score is the mean structural distance between a scene's own six
|
||||
frames, on the same descriptor the variety harness uses. **Minimum bar: 0.1**
|
||||
(`MIN_VARIETY` in `checks/gallery.js`), drawn as a red line. Below it a scene is
|
||||
the same picture wherever it appears, and because it is cast into many songs
|
||||
that sameness leaks between them — the viewer recognises the shot rather than
|
||||
the song.
|
||||
frames, on the same descriptor the variety harness uses. High means the song
|
||||
changes this scene a lot; low means it looks like itself wherever it is cast.
|
||||
|
||||
Most of the library does not clear it. That is deliberate: a bar set where the
|
||||
work already is measures nothing.
|
||||
**There is no bar.** There used to be one — 0.1, drawn across the gallery as a
|
||||
red line — and it was removed when composition arrived. A section is no longer
|
||||
one scene: it is a ground, a shot standing on it, and sometimes a pass over
|
||||
that, so what a viewer sees is a COMBINATION. A scene that is reliably itself is
|
||||
a perfectly good ingredient in one, and the old line failed it for being
|
||||
consistent.
|
||||
|
||||
Some reference points, measured:
|
||||
What replaced it is a level up. The interesting quantity is how unalike the
|
||||
scenes in one stack are, and the generator now weights its choices by exactly
|
||||
that — see `structuralDistance` in `scenes/surface.js`. Your scene does not have
|
||||
to be varied on its own. It has to be unlike the things it will be stacked with,
|
||||
and that is measured for you.
|
||||
|
||||
Some reference points, measured across six songs:
|
||||
|
||||
```
|
||||
0.324 Droste Feedback varies a lot
|
||||
@ -47,6 +54,29 @@ Some reference points, measured:
|
||||
0.047 Moiré Grid after a subject; 0.031 before
|
||||
```
|
||||
|
||||
### The measured metadata
|
||||
|
||||
`src/scenes/metadata.json` is generated, tracked in git, and holds every
|
||||
measured fact about every scene: coverage, variety, per-block scores and the
|
||||
structural profile the distance above is computed from. Nothing in it is
|
||||
hand-written, and two decisions read it:
|
||||
|
||||
* **`surface`** — canvas at 50% coverage or more, composable below. It used to
|
||||
be declared per scene and it drifted; nine scenes claimed `canvas` while
|
||||
painting under a third of the frame. Declaring `surface:` in a scene file is
|
||||
now an error.
|
||||
* **which scenes may be a GROUND** — a canvas that does not call `prev()`, fills the frame
|
||||
when sampled as a bed, and does not vanish under any of the six identities. Coverage is
|
||||
measured twice, once as a shot and once as a bed, because a scene paints wildly different
|
||||
amounts at different parameters: Metaballs is 69% as a drop's shot and 6% as an intro's
|
||||
ground.
|
||||
|
||||
Refresh it from **gallery.html → refresh metadata** after changing a shader or a
|
||||
metric. The file carries a fingerprint of the scenes, the shader contract, the
|
||||
identities and palettes, and the descriptor definitions; when that stops
|
||||
matching, phase 12 fails and tells you to re-measure. The write goes through a
|
||||
dev-only endpoint in `vite.config.js`, so it only works under `npm run dev`.
|
||||
|
||||
---
|
||||
|
||||
## What the descriptor can and cannot see
|
||||
|
||||
@ -126,6 +126,7 @@
|
||||
<span class="spinner"></span>
|
||||
<span id="libctl">
|
||||
<button id="rebuild">rebuild</button>
|
||||
<button id="refresh" title="re-measure the library and write src/scenes/metadata.json — the git-tracked numbers the generator composes with">refresh metadata</button>
|
||||
<select id="sort">
|
||||
<option value="variety">sort: least varied first</option>
|
||||
<option value="name">sort: by name</option>
|
||||
@ -152,7 +153,10 @@
|
||||
<div id="out"></div>
|
||||
|
||||
<script type="module">
|
||||
import { buildGallery, galleryContexts, THUMB, MIN_VARIETY, blit } from '/src/checks/gallery.js';
|
||||
import { buildGallery, galleryContexts, THUMB, blit } from '/src/checks/gallery.js';
|
||||
import { measureLibrary, metadataDrift, writeMetadata, metadataIsFresh } from '/src/checks/metadata.js';
|
||||
// A gallery showing numbers the generator is not using is a trap, so say so.
|
||||
const STALE_NOTE = metadataIsFresh() ? '' : ' · metadata is STALE — press refresh';
|
||||
import {
|
||||
sourceFingerprint, loadGallery, saveGallery, clearGallery,
|
||||
pixelsToBlob, blobToCanvas,
|
||||
@ -167,13 +171,16 @@ const status = document.getElementById('status');
|
||||
const sortSel = document.getElementById('sort');
|
||||
const progress = document.querySelector('#progress i');
|
||||
const rebuildBtn = document.getElementById('rebuild');
|
||||
const refreshBtn = document.getElementById('refresh');
|
||||
|
||||
let rows = []; // { name, family, role, variety, byBlock, blobs, error }
|
||||
let contexts = [];
|
||||
|
||||
// Graded against the bar rather than against taste: below it is a failure, and
|
||||
// the band just above it is where a scene is passing by too little to trust.
|
||||
const grade = (v, err) => (err ? 'broken' : v < MIN_VARIETY ? 'flat' : v < MIN_VARIETY * 2 ? 'thin' : 'good');
|
||||
// There is no bar any more — see checks/gallery.js for why the 0.1 red line
|
||||
// went. What is left is a broken/working distinction and a relative shading, so
|
||||
// the eye can still find the flattest scenes in a sorted list without anyone
|
||||
// pretending a threshold decides whether a scene is allowed to exist.
|
||||
const grade = (v, err) => (err ? 'broken' : v < 0.08 ? 'thin' : 'good');
|
||||
|
||||
function draw() {
|
||||
const mode = sortSel.value;
|
||||
@ -184,17 +191,7 @@ function draw() {
|
||||
: a.variety - b.variety));
|
||||
|
||||
out.innerHTML = '';
|
||||
let drewCut = false;
|
||||
for (const row of sorted) {
|
||||
// One line across the whole gallery, where the sort crosses the bar.
|
||||
// Everything above it repeats itself between songs.
|
||||
if (mode === 'variety' && !drewCut && row.variety >= MIN_VARIETY) {
|
||||
drewCut = true;
|
||||
const cut = document.createElement('div');
|
||||
cut.className = 'cutline';
|
||||
cut.innerHTML = `<span>minimum variety ${MIN_VARIETY.toFixed(3)} — everything above repeats itself between songs</span>`;
|
||||
out.appendChild(cut);
|
||||
}
|
||||
const el = document.createElement('div');
|
||||
el.className = `row ${grade(row.variety, row.error)}`;
|
||||
const blocks = Object.entries(row.byBlock || {})
|
||||
@ -203,9 +200,9 @@ function draw() {
|
||||
<div class="head">
|
||||
<span class="name"><a href="?scene=${encodeURIComponent(row.name)}" title="open this one on a wide grid of songs and seeds">${row.name}</a></span>
|
||||
<span class="fam">${row.family}${row.role === 'accent' ? ' · accent' : ''}</span>
|
||||
<span class="surf ${row.surface}" title="paints ${((row.coverage || 0) * 100).toFixed(0)}% of the frame">${row.surface || '?'} ${((row.coverage || 0) * 100).toFixed(0)}%</span>
|
||||
<span class="surf ${surfaceFor(row)}" title="paints ${((row.coverage || 0) * 100).toFixed(0)}% of the frame across the six songs — this is what decides the label">${surfaceFor(row)} ${((row.coverage || 0) * 100).toFixed(0)}%</span>
|
||||
<span class="score">
|
||||
<span class="meter"><i style="width:${Math.min(100, row.variety * 500)}%"></i><b style="left:${MIN_VARIETY * 500}%" title="minimum ${MIN_VARIETY}"></b></span>
|
||||
<span class="meter"><i style="width:${Math.min(100, row.variety * 500)}%"></i></span>
|
||||
${row.variety.toFixed(3)}
|
||||
</span>
|
||||
<span class="blocks">${row.error ? `<span class="err">${row.error}</span>` : blocks}</span>
|
||||
@ -231,13 +228,18 @@ function draw() {
|
||||
|
||||
sortSel.addEventListener('change', draw);
|
||||
|
||||
// The label a scene WOULD carry from this build's numbers, which is not
|
||||
// necessarily the one the checked-in metadata gives it — that is the whole
|
||||
// point of showing it here.
|
||||
const surfaceFor = (row) => ((row.coverage || 0) >= 0.5 ? 'canvas' : 'composable');
|
||||
|
||||
function summarise(built) {
|
||||
const flat = rows.filter((r) => r.variety < MIN_VARIETY && !r.error).length;
|
||||
const broken = rows.filter((r) => r.error).length;
|
||||
return `${rows.length} scenes · ${flat} below the ${MIN_VARIETY} bar` +
|
||||
const canvases = rows.filter((r) => surfaceFor(r) === 'canvas').length;
|
||||
return `${rows.length} scenes · ${canvases} canvas / ${rows.length - canvases} composable` +
|
||||
(broken ? ` · ${broken} BROKEN` : '') +
|
||||
` · ${rows.filter((r) => (r.coverage || 0) < 0.3).length} paint under 30% of the frame` +
|
||||
(built ? ` · built ${new Date(built).toLocaleString()}` : '');
|
||||
(built ? ` · built ${new Date(built).toLocaleString()}` : '') + STALE_NOTE;
|
||||
}
|
||||
|
||||
async function build(fingerprint) {
|
||||
@ -432,14 +434,14 @@ async function renderSweep() {
|
||||
|
||||
// Three numbers rather than one: which AXIS is flat is the thing the
|
||||
// library gallery cannot tell you, and it decides what to go and fix.
|
||||
const weak = (v) => (v < MIN_VARIETY ? ' weak' : '');
|
||||
const weak = (v) => (v < 0.08 ? ' weak' : '');
|
||||
const twin = result.closest
|
||||
? `${cells[result.closest.a].name} ≈ ${cells[result.closest.b].name} at ${result.closest.distance.toFixed(3)}`
|
||||
: '—';
|
||||
sweepInfo.innerHTML = `
|
||||
<div class="axes">
|
||||
<div class="${weak(result.variety)}">overall <b>${result.variety.toFixed(3)}</b>
|
||||
<small>every cell against every other · bar ${MIN_VARIETY}</small></div>
|
||||
<small>every cell against every other</small></div>
|
||||
<div class="${weak(result.bySeed)}">across seeds <b>${result.bySeed.toFixed(3)}</b>
|
||||
<small>same song, different draw — low means the scene ignores the identity</small></div>
|
||||
<div class="${weak(result.bySong)}">across songs <b>${result.bySong.toFixed(3)}</b>
|
||||
@ -512,6 +514,43 @@ async function main() {
|
||||
await build(fingerprint);
|
||||
}
|
||||
|
||||
// ---- metadata refresh ---------------------------------------------------
|
||||
// The gallery is where the library gets measured, so it is where the measured
|
||||
// facts are written back. The numbers go to src/scenes/metadata.json — tracked
|
||||
// in git, read by the look generator, and stamped with a fingerprint of the
|
||||
// scenes and the metric definitions so a stale file is detectable rather than
|
||||
// merely old. See src/checks/metadata.js.
|
||||
refreshBtn.addEventListener('click', async () => {
|
||||
refreshBtn.disabled = true;
|
||||
rebuildBtn.disabled = true;
|
||||
bar.classList.add('building');
|
||||
const started = Date.now();
|
||||
try {
|
||||
const fresh = measureLibrary({
|
||||
onScene: (done, total, name) => {
|
||||
status.textContent = `measuring ${done}/${total} · ${name}`;
|
||||
progress.style.width = `${(done / total) * 100}%`;
|
||||
},
|
||||
});
|
||||
const { moved, gone } = metadataDrift(fresh);
|
||||
await writeMetadata(fresh);
|
||||
const changes = moved.slice(0, 6).map((m) => `${m.name} ${m.note}`).join(' · ');
|
||||
status.textContent =
|
||||
`metadata written in ${((Date.now() - started) / 1000).toFixed(0)}s · ` +
|
||||
`${Object.keys(fresh.scenes).length} scenes · ` +
|
||||
(moved.length ? `${moved.length} moved — ${changes}` : 'nothing moved') +
|
||||
(gone.length ? ` · dropped ${gone.join(', ')}` : '');
|
||||
} catch (err) {
|
||||
status.textContent = `metadata refresh failed: ${err.message} ` +
|
||||
'(the write endpoint only exists under npm run dev)';
|
||||
} finally {
|
||||
bar.classList.remove('building');
|
||||
refreshBtn.disabled = false;
|
||||
rebuildBtn.disabled = false;
|
||||
progress.style.width = '0%';
|
||||
}
|
||||
});
|
||||
|
||||
rebuildBtn.addEventListener('click', async () => {
|
||||
await clearGallery();
|
||||
await build(sourceFingerprint());
|
||||
|
||||
@ -6,6 +6,7 @@ import { generateLook, rerollLook, rerollSection } from './look/LookGenerator.js
|
||||
import { ArcDriver } from './look/ArcDriver.js';
|
||||
import { grainEnvelope } from './look/grain.js';
|
||||
import { hashSamples } from './engine/rng.js';
|
||||
import { subjectOf } from './look/stack.js';
|
||||
|
||||
/**
|
||||
* The opening and closing fade to black. Exported because anything SAMPLING a
|
||||
@ -45,8 +46,12 @@ export class Show {
|
||||
* Decode, analyse, and generate a look. `onProgress(stage, fraction)` is
|
||||
* called throughout; analysis is CPU-bound and will block the main thread
|
||||
* for a second or two on a long track.
|
||||
*
|
||||
* `seed` replaces the audio-derived one. Only the debug pages pass it, and
|
||||
* only so that watching a song reproduces the look they measured — the
|
||||
* filmstrip seeds off the song's name, not off its samples.
|
||||
*/
|
||||
async load(file, onProgress = null) {
|
||||
async load(file, onProgress = null, { seed = null } = {}) {
|
||||
const report = (stage, p) => onProgress && onProgress(stage, p);
|
||||
|
||||
report('decoding', 0);
|
||||
@ -64,7 +69,9 @@ export class Show {
|
||||
|
||||
report('look', 0.97);
|
||||
const samples = monoSamples(audioBuffer);
|
||||
this.setLook(generateLook(this.track, { samples }));
|
||||
this.setLook(seed === null
|
||||
? generateLook(this.track, { samples })
|
||||
: generateLook(this.track, { seed }));
|
||||
|
||||
this.engine.timeline.setDuration(this.track.duration);
|
||||
this.engine.setFeatureProvider(featureProviderFor(this.track));
|
||||
@ -76,12 +83,22 @@ export class Show {
|
||||
return this;
|
||||
}
|
||||
|
||||
/** Attach an already-analysed track. Used by the check harness and by tests. */
|
||||
/**
|
||||
* Attach an already-analysed track. Used by the check harness and by tests.
|
||||
*
|
||||
* Prewarms, like every other path that installs a look. It was the one that
|
||||
* did not, and that was invisible while most sections were a single layer:
|
||||
* with a ground under every shot there are always at least two programs to
|
||||
* link, and the first frame a fresh Show rendered came out different from
|
||||
* every later render of it — the exact hazard Compositor.prime documents,
|
||||
* caught by the phase 4 first-render check.
|
||||
*/
|
||||
useTrack(track, look) {
|
||||
this.track = track;
|
||||
this.engine.timeline.setDuration(track.duration);
|
||||
this.engine.setFeatureProvider(featureProviderFor(track));
|
||||
this.setLook(look || generateLook(track, { seed: 1 }));
|
||||
this.prewarm();
|
||||
return this;
|
||||
}
|
||||
|
||||
@ -116,7 +133,7 @@ export class Show {
|
||||
setSectionParam(sectionIndex, name, value) {
|
||||
const section = this.look.sections[sectionIndex];
|
||||
if (!section) return;
|
||||
section.layers[0].params[name] = value;
|
||||
subjectOf(section.layers).params[name] = value;
|
||||
this._lastLayers = null;
|
||||
}
|
||||
|
||||
|
||||
21
flow-state/src/checks/blit.js
Normal file
21
flow-state/src/checks/blit.js
Normal file
@ -0,0 +1,21 @@
|
||||
/**
|
||||
* WebGL readback into a canvas, the right way up.
|
||||
*
|
||||
* Its own module because two debug pages need it and neither should pull in the
|
||||
* other's dependencies to get it. It lived in checks/gallery.js, and importing
|
||||
* it from the filmstrip dragged the entire gallery — the engine, the song bank,
|
||||
* the look generator, the whole scene library — into a page that wanted ten
|
||||
* lines of pixel copying.
|
||||
*/
|
||||
export function blit(canvas, pixels, width, height) {
|
||||
canvas.width = width;
|
||||
canvas.height = height;
|
||||
const ctx = canvas.getContext('2d');
|
||||
const image = ctx.createImageData(width, height);
|
||||
const row = width * 4;
|
||||
for (let y = 0; y < height; y++) {
|
||||
const src = (height - 1 - y) * row;
|
||||
image.data.set(pixels.subarray(src, src + row), y * row);
|
||||
}
|
||||
ctx.putImageData(image, 0, 0);
|
||||
}
|
||||
@ -16,8 +16,9 @@
|
||||
// looking for the repetitive ones is exactly the job a sort order should do.
|
||||
|
||||
import { Engine } from '../engine/Engine.js';
|
||||
import { blit } from './blit.js';
|
||||
import { scenes } from '../scenes/registry.js';
|
||||
import { sampleValues, surfaceOf } from '../params/schema.js';
|
||||
import { sampleValues } from '../params/schema.js';
|
||||
import { Rng, hashString } from '../engine/rng.js';
|
||||
import { featureProviderFor } from '../audio/FeatureTrack.js';
|
||||
import { songBank } from '../audio/songbank.js';
|
||||
@ -29,28 +30,23 @@ import { descriptorDistance, STRUCTURAL } from './variety/signature.js';
|
||||
|
||||
const THUMB = { width: 256, height: 144 };
|
||||
|
||||
/**
|
||||
* The floor a visualizer has to clear: how different its own six frames must be
|
||||
* from each other, across six songs' identities.
|
||||
/*
|
||||
* There used to be a MIN_VARIETY floor here — 0.1, drawn as a red line across
|
||||
* the gallery — on the theory that a scene which looks the same in every song
|
||||
* leaks that sameness between videos.
|
||||
*
|
||||
* Below this a scene is the same picture wherever it appears, and because it is
|
||||
* cast into many songs that repetition leaks between them — the viewer who
|
||||
* watches two videos recognises the shot rather than the song. It is a property
|
||||
* of the scene, not of the generator, so it is the scene that has to be lifted.
|
||||
* That was true when a section was ONE scene. It is not any more: a section is
|
||||
* a ground, a shot over it and sometimes a pass over that, so what a viewer
|
||||
* sees is a combination, and a scene that is reliably itself is a perfectly
|
||||
* good ingredient in one. Held against a floor, such scenes were failing for
|
||||
* being consistent.
|
||||
*
|
||||
* 0.1, and deliberately a target rather than a description of where the library
|
||||
* currently sits. Most of it does not clear this — of the scenes measured after
|
||||
* the region block went in, only Droste Feedback at 0.324 is comfortably over,
|
||||
* with Plasma Bloom at 0.082 and Voronoi Shatter at 0.062 still short. A bar set
|
||||
* where the work already is measures nothing.
|
||||
*
|
||||
* It has moved twice for instrument reasons rather than taste — the motion block
|
||||
* was reading zero for the whole library, and adding the region block made the
|
||||
* total a mean over six rather than five — and both raised every score. Expect
|
||||
* to re-read it off a fresh gallery whenever the descriptor changes, rather than
|
||||
* carrying the old number forward.
|
||||
* The score is still measured, still reported, and still what the gallery sorts
|
||||
* by — it is genuinely the right question to ask about a scene you are working
|
||||
* on. It is no longer a bar anything has to clear, and the interesting quantity
|
||||
* moved up a level: how unalike the scenes in one stack are. That lives in
|
||||
* scenes/metadata.json and is read by the look generator.
|
||||
*/
|
||||
export const MIN_VARIETY = 0.1;
|
||||
|
||||
/**
|
||||
* Six contexts, one per song: everything a scene is handed when it is cast.
|
||||
@ -78,20 +74,6 @@ export function galleryContexts(count = 6) {
|
||||
});
|
||||
}
|
||||
|
||||
/** Bottom-up WebGL pixels into a canvas the right way up. */
|
||||
function blit(canvas, pixels, width, height) {
|
||||
canvas.width = width;
|
||||
canvas.height = height;
|
||||
const ctx = canvas.getContext('2d');
|
||||
const image = ctx.createImageData(width, height);
|
||||
const row = width * 4;
|
||||
for (let y = 0; y < height; y++) {
|
||||
const src = (height - 1 - y) * row;
|
||||
image.data.set(pixels.subarray(src, src + row), y * row);
|
||||
}
|
||||
ctx.putImageData(image, 0, 0);
|
||||
}
|
||||
|
||||
/**
|
||||
* Render one scene across every context.
|
||||
*
|
||||
@ -175,9 +157,14 @@ export function renderScene(engine, module, contexts) {
|
||||
|
||||
return {
|
||||
thumbs,
|
||||
// The six per-context descriptors, so a caller can average them into a
|
||||
// profile rather than re-render the library to get one. See
|
||||
// checks/metadata.js — this function is the only place the scenes are
|
||||
// rendered under real identities, and everything measured about a scene
|
||||
// comes out of here.
|
||||
descriptors,
|
||||
variety: pairs ? total / pairs : 0,
|
||||
coverage: covered,
|
||||
surface: surfaceOf(module),
|
||||
byBlock,
|
||||
error: dead
|
||||
? `renders nothing — luminance ${lum.toFixed(4)}, variance ${variance.toFixed(4)}. ` +
|
||||
|
||||
217
flow-state/src/checks/metadata.js
Normal file
217
flow-state/src/checks/metadata.js
Normal file
@ -0,0 +1,217 @@
|
||||
// Measuring every visualizer, and writing the answers back into the repo.
|
||||
//
|
||||
// The library has always had two kinds of fact about a scene. Declared ones —
|
||||
// family, traits, `consumes` — which say what the scene is FOR, and measured
|
||||
// ones, which say what it actually does when rendered. Declared facts belong in
|
||||
// the scene file. Measured facts do not: hand-written, they drift the moment a
|
||||
// shader changes, and nine scenes declaring `surface: 'canvas'` while painting
|
||||
// under a third of the frame is what that drift looks like.
|
||||
//
|
||||
// So the measured half lives in scenes/metadata.json, generated from here,
|
||||
// tracked in git, and stamped with a fingerprint of everything that could
|
||||
// change it. When the fingerprint stops matching, the numbers are stale and the
|
||||
// phase 12 gate says so — the file is a cache of a render, and a cache nobody
|
||||
// can tell is stale is worse than no cache.
|
||||
//
|
||||
// The measurement is the GALLERY's: six songs, sampled parameters, real
|
||||
// identities and palettes — a scene as it is actually cast, not as it renders
|
||||
// at default parameters. The difference is not academic. Salt Flat paints 65%
|
||||
// of the frame at defaults and 32% across six real songs, and the generator
|
||||
// chooses grounds with this number.
|
||||
|
||||
import { Engine } from '../engine/Engine.js';
|
||||
import { scenes } from '../scenes/registry.js';
|
||||
import { galleryContexts, renderScene, THUMB } from './gallery.js';
|
||||
import { readsHistory } from '../params/schema.js';
|
||||
import { GROUND_BIAS, groundTemperamentFrom, groundPersonalityFrom } from '../scenes/surface.js';
|
||||
import metadata from '../scenes/metadata.json';
|
||||
|
||||
/**
|
||||
* What invalidates the measurements.
|
||||
*
|
||||
* Deliberately NOT every file under src/, which is what the gallery cache
|
||||
* fingerprints: that changes when the UI changes, and it would mark the
|
||||
* metadata stale for edits that cannot move a single number. What can move one
|
||||
* is the scenes themselves, the contract they are compiled against, the
|
||||
* identities and palettes they are handed, and the metric definitions — so
|
||||
* those, and nothing else.
|
||||
*
|
||||
* Globbed rather than listed wherever a whole directory qualifies, because the
|
||||
* file that invalidates a measurement is exactly the one nobody remembers to
|
||||
* add to a list.
|
||||
*/
|
||||
const SOURCES = {
|
||||
...import.meta.glob('/src/scenes/**/*.js', { query: '?raw', import: 'default', eager: true }),
|
||||
...import.meta.glob('/src/checks/variety/descriptors.js', { query: '?raw', import: 'default', eager: true }),
|
||||
...import.meta.glob('/src/checks/variety/signature.js', { query: '?raw', import: 'default', eager: true }),
|
||||
...import.meta.glob('/src/checks/gallery.js', { query: '?raw', import: 'default', eager: true }),
|
||||
...import.meta.glob('/src/checks/metadata.js', { query: '?raw', import: 'default', eager: true }),
|
||||
...import.meta.glob('/src/engine/shader-contract.js', { query: '?raw', import: 'default', eager: true }),
|
||||
...import.meta.glob('/src/look/Identity.js', { query: '?raw', import: 'default', eager: true }),
|
||||
...import.meta.glob('/src/look/Personality.js', { query: '?raw', import: 'default', eager: true }),
|
||||
...import.meta.glob('/src/look/palette.js', { query: '?raw', import: 'default', eager: true }),
|
||||
...import.meta.glob('/src/audio/songbank.js', { query: '?raw', import: 'default', eager: true }),
|
||||
};
|
||||
|
||||
/** The version of the measurement itself. Bump to force a refresh of everything. */
|
||||
export const SCHEMA = 4;
|
||||
|
||||
export function metricsFingerprint() {
|
||||
let h = 0x811c9dc5 >>> 0;
|
||||
const mix = (str) => {
|
||||
for (let i = 0; i < str.length; i++) {
|
||||
h ^= str.charCodeAt(i);
|
||||
h = Math.imul(h, 0x01000193) >>> 0;
|
||||
}
|
||||
};
|
||||
mix(`schema:${SCHEMA}`);
|
||||
for (const path of Object.keys(SOURCES).sort()) {
|
||||
mix(path);
|
||||
mix(SOURCES[path]);
|
||||
}
|
||||
return h.toString(16).padStart(8, '0');
|
||||
}
|
||||
|
||||
/** Whether the checked-in metadata was measured from the code that is here now. */
|
||||
export function metadataIsFresh() {
|
||||
return metadata.fingerprint === metricsFingerprint();
|
||||
}
|
||||
|
||||
/** Anything a viewer would read as painted rather than as backdrop. */
|
||||
function litFraction(pixels) {
|
||||
let lit = 0;
|
||||
for (let i = 0; i < pixels.length; i += 4) {
|
||||
if (pixels[i] + pixels[i + 1] + pixels[i + 2] > 90) lit++;
|
||||
}
|
||||
return lit / (pixels.length / 4);
|
||||
}
|
||||
|
||||
const round = (x, places = 4) => Number(x.toFixed(places));
|
||||
|
||||
/** Mean of the six per-context descriptors, block by block. */
|
||||
function meanProfile(descriptors) {
|
||||
const out = {};
|
||||
for (const block of Object.keys(descriptors[0])) {
|
||||
const length = descriptors[0][block].length;
|
||||
const acc = new Array(length).fill(0);
|
||||
for (const d of descriptors) {
|
||||
for (let i = 0; i < length; i++) acc[i] += d[block][i] / descriptors.length;
|
||||
}
|
||||
out[block] = acc.map((v) => round(v));
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Measure the whole library.
|
||||
*
|
||||
* Per scene: how much frame it paints, how much it changes between songs, and
|
||||
* its mean structural profile — the descriptor the variety harness compares
|
||||
* videos with, averaged over the six renders. The profile is what makes this
|
||||
* more than a list of numbers: two profiles can be compared, so the generator
|
||||
* can ask whether a shot and the thing under it are the same picture twice.
|
||||
*
|
||||
* @returns {object} the metadata file's contents
|
||||
*/
|
||||
export function measureLibrary({ onScene = null, contexts = null } = {}) {
|
||||
const ctx = contexts || galleryContexts(6);
|
||||
// The same six songs, sampled the way a BED is. Coverage is mostly a
|
||||
// function of a scene's parameters, so "how much does this paint" has two
|
||||
// answers and the generator needs both: one for the budget, and one for
|
||||
// whether it may be a ground at all. See GROUND_BIAS.
|
||||
const bedCtx = ctx.map((c) => ({
|
||||
...c,
|
||||
bias: { ...c.bias, ...GROUND_BIAS },
|
||||
personality: {
|
||||
...groundPersonalityFrom(c.personality),
|
||||
temperament: groundTemperamentFrom(c.personality.temperament),
|
||||
},
|
||||
}));
|
||||
const engine = new Engine({ ...THUMB });
|
||||
const out = {};
|
||||
try {
|
||||
const list = scenes.filter((m) => m.kind === 'fragment');
|
||||
for (const module of list) {
|
||||
const { thumbs, variety, byBlock, descriptors, error } = renderScene(engine, module, ctx);
|
||||
const bed = renderScene(engine, module, bedCtx);
|
||||
out[module.name] = {
|
||||
coverage: round(thumbs.reduce((s, px) => s + litFraction(px), 0) / thumbs.length, 3),
|
||||
// The WORST of the six, not the mean. What a ground has to
|
||||
// promise is a filled frame in the video it lands in, and the
|
||||
// spread across identities is enormous: a track whose ink
|
||||
// treatment is `hollow` draws outlines instead of fills, so a
|
||||
// scene that paints 61% averaged over six songs paints 2% in
|
||||
// the one that asked for outlines — measured, and it is how a
|
||||
// section with a ground under it still rendered near-black.
|
||||
// A mean cannot make a promise; a minimum can.
|
||||
bedCoverage: round(Math.min(...bed.thumbs.map(litFraction)), 3),
|
||||
bedCoverageMean: round(
|
||||
bed.thumbs.reduce((s, px) => s + litFraction(px), 0) / bed.thumbs.length, 3),
|
||||
variety: round(variety, 3),
|
||||
blocks: Object.fromEntries(
|
||||
Object.entries(byBlock).map(([b, v]) => [b, round(v, 3)])),
|
||||
// Declared, not measured, and carried here anyway: it is a fact
|
||||
// about the scene that the composition rules read, and having
|
||||
// every compositional input in one file is the point.
|
||||
readsHistory: readsHistory(module),
|
||||
profile: meanProfile(descriptors),
|
||||
...(error ? { error } : {}),
|
||||
};
|
||||
if (onScene) onScene(Object.keys(out).length, list.length, module.name);
|
||||
}
|
||||
} finally {
|
||||
engine.dispose();
|
||||
}
|
||||
|
||||
return {
|
||||
fingerprint: metricsFingerprint(),
|
||||
schema: SCHEMA,
|
||||
measured: new Date().toISOString().slice(0, 10),
|
||||
contexts: ctx.map((c) => c.name),
|
||||
scenes: out,
|
||||
};
|
||||
}
|
||||
|
||||
/** What moved against the checked-in file. */
|
||||
export function metadataDrift(fresh) {
|
||||
// Tolerates a missing or half-written file on purpose: this runs on the way
|
||||
// to REPLACING it, and refusing to report because the thing being replaced
|
||||
// is malformed is the least useful moment to be strict.
|
||||
const previous = (metadata && metadata.scenes) || {};
|
||||
const moved = [];
|
||||
for (const [name, row] of Object.entries(fresh.scenes)) {
|
||||
const was = previous[name];
|
||||
if (!was) {
|
||||
moved.push({ name, note: 'new' });
|
||||
continue;
|
||||
}
|
||||
const delta = row.coverage - was.coverage;
|
||||
if (Math.abs(delta) > 0.02) {
|
||||
moved.push({
|
||||
name, note: `coverage ${(was.coverage * 100).toFixed(0)}% → ${(row.coverage * 100).toFixed(0)}%`,
|
||||
delta,
|
||||
});
|
||||
}
|
||||
}
|
||||
const gone = Object.keys(previous).filter((n) => !fresh.scenes[n]);
|
||||
return {
|
||||
moved: moved.sort((a, b) => Math.abs(b.delta || 0) - Math.abs(a.delta || 0)),
|
||||
gone,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Ask the dev server to write the file back into the source tree.
|
||||
*
|
||||
* Dev-only by construction — the endpoint is a middleware in vite.config.js. A
|
||||
* built page has no source tree to write to, and failing there is correct.
|
||||
*/
|
||||
export async function writeMetadata(fresh) {
|
||||
const response = await fetch('/__metadata', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(fresh, null, 2) + '\n',
|
||||
});
|
||||
if (!response.ok) throw new Error(`${response.status} ${await response.text()}`);
|
||||
return response.text();
|
||||
}
|
||||
@ -37,6 +37,7 @@ import { frameDistance, frameLuminance } from '../engine/hash.js';
|
||||
import { battery as timbreBattery } from './phase3.js';
|
||||
import { grainEnvelope } from '../look/grain.js';
|
||||
import { signatureUniforms } from '../look/Personality.js';
|
||||
import { subjectOf, overlaysOf } from '../look/stack.js';
|
||||
|
||||
/**
|
||||
* Several tracks that genuinely differ in what they sound like — the population
|
||||
@ -124,8 +125,8 @@ check(10, 'one scene looks different in two different videos', () => {
|
||||
// track's temperament — the question is what this scene WOULD
|
||||
// look like in that video, and falling back to defaults would
|
||||
// compare two identical parameter sets and prove nothing.
|
||||
const stack = stacksOf(look).find((s) => s[0].module === module);
|
||||
const params = stack ? stack[0].params : sampleValues(
|
||||
const stack = stacksOf(look).find((s) => subjectOf(s).module === module);
|
||||
const params = stack ? subjectOf(stack).params : sampleValues(
|
||||
module,
|
||||
new Rng(look.seed ^ 0x51ed270b),
|
||||
look.sections[0].bias,
|
||||
@ -187,7 +188,7 @@ check(10, 'overlays happen sometimes and not always', () => {
|
||||
const look = generateLook(track, { seed: 900 + s * 5231 });
|
||||
for (const stack of stacksOf(look)) {
|
||||
stacks++;
|
||||
const overlay = stack[1];
|
||||
const overlay = overlaysOf(stack)[0];
|
||||
if (overlay) {
|
||||
withOverlay++;
|
||||
blends.add(overlay.blend);
|
||||
@ -214,7 +215,7 @@ check(10, 'an overlay never hides the shot underneath it', () => {
|
||||
for (const { track } of battery()) {
|
||||
for (let s = 0; s < 4; s++) {
|
||||
for (const stack of stacksOf(generateLook(track, { seed: 1300 + s * 8641 }))) {
|
||||
for (const layer of stack.slice(1)) {
|
||||
for (const layer of overlaysOf(stack)) {
|
||||
if (layer.opacity > 0.6) {
|
||||
problems.push(`${layer.module.name} at ${layer.opacity.toFixed(2)}`);
|
||||
}
|
||||
|
||||
@ -26,6 +26,7 @@ import { Rng } from '../engine/rng.js';
|
||||
import { frameDistance, frameLuminance } from '../engine/hash.js';
|
||||
import { SHOT_SIZES, SHOT_SIZE_NAMES, describeFraming } from '../look/framing.js';
|
||||
import { reachFor, CURVE_NAMES } from '../look/Camera.js';
|
||||
import { subjectOf } from '../look/stack.js';
|
||||
|
||||
/** Tracks at several tempos: shot length is measured in bars, so tempo is the axis. */
|
||||
let cached = null;
|
||||
@ -273,7 +274,7 @@ check(11, 'two tracks do not agree on what a section kind looks like', () => {
|
||||
directors.add(look.director);
|
||||
look.sections.forEach((section) => {
|
||||
const set = seen.get(section.kind) || new Set();
|
||||
(section.variants || [section.layers]).forEach((v) => set.add(v[0].module.family));
|
||||
(section.variants || [section.layers]).forEach((v) => set.add(subjectOf(v).module.family));
|
||||
seen.set(section.kind, set);
|
||||
});
|
||||
}
|
||||
@ -299,7 +300,7 @@ check(11, 'a track shows more of the library than it used to', () => {
|
||||
for (let s = 0; s < 4; s++) {
|
||||
const look = generateLook(t, { seed: (i * 2654435761 + s * 40503) >>> 0 });
|
||||
look.sections.forEach((section) =>
|
||||
(section.variants || [section.layers]).forEach((v) => cast.add(v[0].module.name)));
|
||||
(section.variants || [section.layers]).forEach((v) => cast.add(subjectOf(v).module.name)));
|
||||
}
|
||||
});
|
||||
const pool = scenes.filter(canBackground).length;
|
||||
@ -492,8 +493,11 @@ check(11, 'the slow axis is a journey rather than a cycle', () => {
|
||||
|
||||
try {
|
||||
const cue = arc.cues[0];
|
||||
const at = (time) => arc._paramsAt(cue, 0, time, t.at(Math.round(time * 60)));
|
||||
const spec = arc._specFor(cue.sectionIndex, cue.variant, 0);
|
||||
// The shot, not the bed under it — the slow axis is a claim about the
|
||||
// scene the section is about.
|
||||
const slot = arc._subjectSlot(cue);
|
||||
const at = (time) => arc._paramsAt(cue, slot, time, t.at(Math.round(time * 60)));
|
||||
const spec = arc._specFor(cue.sectionIndex, cue.variant, slot);
|
||||
const axis = arc._slowAxisFor(spec.module);
|
||||
|
||||
const start = at(t.duration * 0.05);
|
||||
|
||||
@ -20,10 +20,14 @@ import { song } from '../audio/songbank.js';
|
||||
import { Show } from '../Show.js';
|
||||
import { generateLook } from '../look/LookGenerator.js';
|
||||
import { scenes } from '../scenes/registry.js';
|
||||
import { surfaceOf, canBackground } from '../params/schema.js';
|
||||
import { Engine } from '../engine/Engine.js';
|
||||
import { defaultValues } from '../params/schema.js';
|
||||
import { featureProviderFor } from '../audio/FeatureTrack.js';
|
||||
import { canBackground } from '../params/schema.js';
|
||||
import {
|
||||
GROUND_MIN, canGround, isMeasured, surfaceOf, METADATA,
|
||||
coverageOf as declaredCoverage, structuralDistance,
|
||||
} from '../scenes/surface.js';
|
||||
import { metadataIsFresh, metricsFingerprint } from './metadata.js';
|
||||
import { RESTFUL_FAMILIES } from '../look/directors.js';
|
||||
import { stackCoverage } from '../look/stack.js';
|
||||
import { frameDescriptor, rotate90, recolour } from './variety/descriptors.js';
|
||||
import { descriptorDistance, signatureDistance } from './variety/signature.js';
|
||||
import {
|
||||
@ -185,62 +189,249 @@ check(12, 'seed variety · different seeds render structurally different videos'
|
||||
|
||||
|
||||
/**
|
||||
* How much of the frame a scene actually paints.
|
||||
* The metadata is measured from the code that is actually here.
|
||||
*
|
||||
* A composable scene has to leave room for what it sits on. One that claims to
|
||||
* and covers the frame anyway will hide its background completely, which is the
|
||||
* failure the declaration exists to prevent — and it is not something you can
|
||||
* see from the source, only from the render.
|
||||
* scenes/metadata.json is a cache of a render — how much frame each scene
|
||||
* paints, how much it changes between songs, what it looks like structurally —
|
||||
* and the generator composes with it: which scenes may ground a section, and
|
||||
* which pairs are unalike enough to be worth stacking. A stale file therefore
|
||||
* does not produce a stale REPORT, it produces wrong videos, silently.
|
||||
*
|
||||
* So the file carries a fingerprint of everything that can move a number in it
|
||||
* — the scenes, the shader contract, the identities and palettes they are
|
||||
* handed, and the metric definitions — and this fails when it stops matching.
|
||||
* The fix is not code: open gallery.html and press "refresh metadata".
|
||||
*/
|
||||
function coverageOf(engine, module) {
|
||||
engine.setLayerSpecs([{
|
||||
module, params: defaultValues(module), seed: 4242,
|
||||
opacity: 1, blend: 'normal',
|
||||
palette: [[0.05, 0.05, 0.1], [0.9, 0.3, 0.5], [0.3, 0.8, 0.9], [0.95, 0.9, 0.4]],
|
||||
}]);
|
||||
engine.compositor.reset();
|
||||
for (let f = 594; f < 600; f++) engine.renderFrame(f);
|
||||
const px = Uint8Array.from(engine.readPixels(engine.renderFrame(600)));
|
||||
let lit = 0;
|
||||
for (let i = 0; i < px.length; i += 4) {
|
||||
// Anything a viewer would read as painted rather than as backdrop.
|
||||
if (px[i] + px[i + 1] + px[i + 2] > 90) lit++;
|
||||
check(12, 'metadata · the measurements match the code that produced them', () => {
|
||||
const fresh = metadataIsFresh();
|
||||
const rows = Object.keys(METADATA.scenes || {}).length;
|
||||
const unmeasured = scenes
|
||||
.filter((m) => m.kind === 'fragment' && !isMeasured(m))
|
||||
.map((m) => m.name);
|
||||
|
||||
return expect(fresh && unmeasured.length === 0,
|
||||
(fresh ? '' : `stale: measured against ${METADATA.fingerprint}, code is ${metricsFingerprint()} — ` +
|
||||
'rebuild it from gallery.html → refresh metadata · ') +
|
||||
(unmeasured.length ? `never measured: ${unmeasured.slice(0, 5).join(', ')} · ` : '') +
|
||||
`${rows} scenes measured ${METADATA.measured} · ` +
|
||||
`${scenes.filter(canGround).length} can ground a section`);
|
||||
});
|
||||
|
||||
/**
|
||||
* The labels the measurements produce are usable ones.
|
||||
*
|
||||
* Not a re-measurement — the check above covers staleness. This asks whether
|
||||
* the library, as labelled, can still cast a video: every section needs a
|
||||
* ground, quiet sections are held to the restful families, so a threshold that
|
||||
* left one ground in the whole library would pass every other gate here and
|
||||
* make every video identical underneath.
|
||||
*/
|
||||
check(12, 'metadata · the measured labels leave enough grounds to cast with', () => {
|
||||
const grounds = scenes.filter(canGround);
|
||||
const byFamily = {};
|
||||
for (const m of grounds) byFamily[m.family] = (byFamily[m.family] || 0) + 1;
|
||||
const restful = RESTFUL_FAMILIES.reduce((n, f) => n + (byFamily[f] || 0), 0);
|
||||
|
||||
const problems = [];
|
||||
// Low bars, and they are where the library actually is rather than where it
|
||||
// ought to be. Seven scenes can ground a section — five geometric, two
|
||||
// organic — and the two organic ones carry every quiet section of every
|
||||
// video, because no minimal or flow canvas in the library fills half the
|
||||
// frame without reading prev(). That is the library's largest hole and it
|
||||
// is scene work, not generator work. The numbers are printed on every run
|
||||
// so it stays visible instead of becoming the way things are.
|
||||
if (grounds.length < 6) problems.push(`only ${grounds.length} grounds in the library`);
|
||||
if (restful < 2) problems.push(`only ${restful} restful grounds for quiet sections`);
|
||||
|
||||
return expect(problems.length === 0,
|
||||
(problems.length ? problems.join(' · ') + ' — ' : '') +
|
||||
`${grounds.length} grounds · ` +
|
||||
Object.entries(byFamily).map(([f, n]) => `${f} ${n}`).join(' · '));
|
||||
});
|
||||
|
||||
/**
|
||||
* COMPOSITION — what every section's frame is made of.
|
||||
*
|
||||
* Two rules, and they are the two ends of the same idea: a section always has a
|
||||
* filled picture underneath it, and it never stacks up more than two frames'
|
||||
* worth of material on top of that. Between them they rule out both failures
|
||||
* the layer stack can produce — a shot floating on black, and four passes of
|
||||
* texture over each other.
|
||||
*
|
||||
* Measured across the song bank rather than one track, because both rules are
|
||||
* decided per director, per kind and per point in the story, and a single song
|
||||
* exercises one director's opinion about six sections.
|
||||
*/
|
||||
check(12, 'composition · every section stands on a canvas, under a budget', () => {
|
||||
const problems = [];
|
||||
const budgets = [];
|
||||
let stacks = 0;
|
||||
let grounded = 0;
|
||||
|
||||
for (const name of ['centre', 'drone', 'lattice']) {
|
||||
const track = song(name).track;
|
||||
for (let s = 0; s < 4; s++) {
|
||||
const look = generateLook(track, { seed: 4000 + s * 7717 });
|
||||
for (const section of look.sections) {
|
||||
for (const stack of section.variants || [section.layers]) {
|
||||
stacks++;
|
||||
const base = stack[0];
|
||||
const cover = declaredCoverage(base.module);
|
||||
if (surfaceOf(base.module) !== 'canvas' || cover < GROUND_MIN) {
|
||||
problems.push(`${section.kind} stands on ${base.module.name} at ${(cover * 100).toFixed(0)}%`);
|
||||
} else {
|
||||
grounded++;
|
||||
}
|
||||
if (base.opacity < 1 || base.blend !== 'normal') {
|
||||
problems.push(`${section.kind} ground ${base.module.name} is ${base.blend} at ${base.opacity.toFixed(2)}`);
|
||||
}
|
||||
const total = stackCoverage(stack);
|
||||
budgets.push(total);
|
||||
if (total > 2.0001) {
|
||||
problems.push(`${section.kind} paints ${(total * 100).toFixed(0)}% — over budget`);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return lit / (px.length / 4);
|
||||
}
|
||||
|
||||
check(12, 'surface · a composable scene leaves room for what it sits on', () => {
|
||||
const engine = new Engine({ width: 128, height: 72 });
|
||||
const track = varietyTrack();
|
||||
engine.timeline.setDuration(track.duration);
|
||||
engine.setFeatureProvider(featureProviderFor(track));
|
||||
const mean = budgets.reduce((a, b) => a + b, 0) / Math.max(1, budgets.length);
|
||||
const max = Math.max(...budgets);
|
||||
return expect(problems.length === 0,
|
||||
(problems.length ? problems.slice(0, 4).join(' · ') + ' — ' : '') +
|
||||
`${grounded}/${stacks} grounded · coverage mean ${(mean * 100).toFixed(0)}% ` +
|
||||
`· peak ${(max * 100).toFixed(0)}% of the 200% ceiling`);
|
||||
});
|
||||
|
||||
const wrong = [];
|
||||
const measured = [];
|
||||
/**
|
||||
* The rendered frame, at BOTH ends.
|
||||
*
|
||||
* The spec-level check above says the generator intended a filled frame. This
|
||||
* renders the middle of every section of twelve videos and looks at what came
|
||||
* out, which is the only statement that matters to someone watching.
|
||||
*
|
||||
* Two failures, and they are opposites that arrive by the same route:
|
||||
*
|
||||
* too black a section that is a few bright things on nothing. Measured at
|
||||
* 9 of 40 sampled frames under 20% painted before grounds, the
|
||||
* darkest at 0.3%.
|
||||
* too white a section clipped to paper. Measured at a median 24% of pixels
|
||||
* at full white and entire sections at 100% when the shot was
|
||||
* screened over its ground and the feedback loop was still an
|
||||
* accumulator. Clipped white is not brightness, it is missing
|
||||
* information: every difference inside it has been deleted.
|
||||
*
|
||||
* `ink` is counted at a low threshold on purpose — the question is whether the
|
||||
* frame has anything IN it, not whether it is bright, and a legitimately dark
|
||||
* scene is not a failure. `blown` is counted at near-full white on all three
|
||||
* channels, which no grade should produce over a quarter of a frame.
|
||||
*/
|
||||
check(12, 'composition · a rendered section is neither black nor blown out', () => {
|
||||
const dark = [];
|
||||
const white = [];
|
||||
const ink = [];
|
||||
const blown = [];
|
||||
const lum = [];
|
||||
|
||||
for (const name of ['centre', 'drone', 'lattice', 'glare', 'murk', 'runner']) {
|
||||
const track = song(name).track;
|
||||
for (const seed of [4001, 9931]) {
|
||||
const show = new Show({ width: 160, height: 90 });
|
||||
try {
|
||||
for (const module of scenes) {
|
||||
if (module.kind !== 'fragment') continue;
|
||||
const cover = coverageOf(engine, module);
|
||||
const surface = surfaceOf(module);
|
||||
measured.push({ name: module.name, surface, cover });
|
||||
// A composable scene painting most of the frame hides its
|
||||
// background; a canvas leaving it nearly empty is a canvas in name
|
||||
// only and will read as a black frame when nothing is under it.
|
||||
if (surface === 'composable' && cover > 0.55) wrong.push(`${module.name} claims composable but covers ${(cover * 100).toFixed(0)}%`);
|
||||
if (surface === 'canvas' && cover < 0.08) wrong.push(`${module.name} claims canvas but covers only ${(cover * 100).toFixed(0)}%`);
|
||||
show.useTrack(track, generateLook(track, { seed }));
|
||||
for (const section of show.look.sections) {
|
||||
const f = Math.round(section.startFrame
|
||||
+ (section.endFrame - section.startFrame) * 0.5);
|
||||
show.engine.compositor.reset();
|
||||
for (let k = Math.max(0, f - 12); k < f; k++) show.renderFrame(k);
|
||||
const px = Uint8Array.from(show.readPixels(show.renderFrame(f)));
|
||||
|
||||
let inked = 0, clipped = 0, light = 0;
|
||||
const n = px.length / 4;
|
||||
for (let i = 0; i < px.length; i += 4) {
|
||||
const r = px[i], g = px[i + 1], b = px[i + 2];
|
||||
const l = 0.2126 * r + 0.7152 * g + 0.0722 * b;
|
||||
if (l > 13) inked++;
|
||||
if (r > 238 && g > 238 && b > 238) clipped++;
|
||||
light += l;
|
||||
}
|
||||
const where = `${name}/${seed.toString(16)} ${section.kind}`;
|
||||
ink.push(inked / n);
|
||||
blown.push(clipped / n);
|
||||
lum.push(light / n / 255);
|
||||
if (inked / n < 0.3) dark.push(`${where} only ${((inked / n) * 100).toFixed(0)}% painted`);
|
||||
// Per section: the hard cap only. A deliberate blaze is
|
||||
// allowed to be bright; nothing is allowed to be gone.
|
||||
if (clipped / n > 0.5) white.push(`${where} ${((clipped / n) * 100).toFixed(0)}% clipped white`);
|
||||
if (light / n / 255 > 0.9) white.push(`${where} mean luminance ${((light / n / 255) * 100).toFixed(0)}%`);
|
||||
}
|
||||
} finally {
|
||||
engine.dispose();
|
||||
show.dispose();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Reported whatever the verdict, because the useful output of this check is
|
||||
// the list itself — it is how the library gets labelled in the first place.
|
||||
const sorted = measured.sort((a, b) => a.cover - b.cover);
|
||||
window.__COVERAGE__ = sorted;
|
||||
const sparse = sorted.filter((m) => m.cover < 0.3).length;
|
||||
// A BLAZE is allowed; blazing by default is not.
|
||||
//
|
||||
// The ceiling is therefore two numbers rather than one. No single section
|
||||
// may be wholly gone — past about half the frame at full white there is no
|
||||
// picture left to read — and only a minority of them may be hot at all. The
|
||||
// second is the one that matters: screening every shot over its ground blew
|
||||
// a median quarter of EVERY frame, which is not a director choosing to peak,
|
||||
// it is a pipeline with no headroom. See blazeOf in look/directors.js.
|
||||
const hot = blown.filter((b) => b > 0.25).length;
|
||||
const hotShare = hot / Math.max(1, blown.length);
|
||||
if (hotShare > 0.2) {
|
||||
white.push(`${(hotShare * 100).toFixed(0)}% of sections clipped past 25% — blazing is the default, not a choice`);
|
||||
}
|
||||
|
||||
return expect(wrong.length === 0,
|
||||
(wrong.length ? wrong.slice(0, 4).join(' · ') + ' — ' : '') +
|
||||
`${measured.length} scenes · ${sparse} paint under 30% of the frame ` +
|
||||
`· sparsest ${sorted[0].name} at ${(sorted[0].cover * 100).toFixed(0)}%`);
|
||||
const problems = [...dark, ...white];
|
||||
const mean = (a) => a.reduce((x, y) => x + y, 0) / Math.max(1, a.length);
|
||||
return expect(problems.length === 0,
|
||||
(problems.length ? problems.slice(0, 4).join(' · ') + ' — ' : '') +
|
||||
`${ink.length} sections · painted mean ${(mean(ink) * 100).toFixed(0)}% ` +
|
||||
`darkest ${(Math.min(...ink) * 100).toFixed(0)}% · ` +
|
||||
`clipped mean ${(mean(blown) * 100).toFixed(0)}% worst ${(Math.max(...blown) * 100).toFixed(0)}% · ` +
|
||||
`${hot} of ${blown.length} sections blazing`);
|
||||
}, { slow: true });
|
||||
|
||||
/**
|
||||
* A stack is two things happening, not one thing twice.
|
||||
*
|
||||
* The point of the measured profiles: family labels say a `flow` scene and an
|
||||
* `organic` scene are different, and the render can disagree — two of them can
|
||||
* sit 0.04 apart, which stacked is one texture at double density. This asks
|
||||
* whether the generator's stacks are actually made of unalike material.
|
||||
*
|
||||
* A mean rather than a per-stack floor, because a lean is what the generator
|
||||
* applies. Demanding every pair clear a bar would be demanding a particular
|
||||
* draw, and the seed is supposed to be able to make an ordinary choice.
|
||||
*/
|
||||
check(12, 'composition · stacked layers are structurally unalike', () => {
|
||||
const pairs = [];
|
||||
for (const name of ['centre', 'drone', 'lattice']) {
|
||||
const track = song(name).track;
|
||||
for (let s = 0; s < 4; s++) {
|
||||
const look = generateLook(track, { seed: 5200 + s * 3931 });
|
||||
for (const section of look.sections) {
|
||||
for (const stack of section.variants || [section.layers]) {
|
||||
for (let i = 0; i < stack.length; i++) {
|
||||
for (let j = i + 1; j < stack.length; j++) {
|
||||
const d = structuralDistance(stack[i].module, stack[j].module);
|
||||
if (d !== null) pairs.push(d);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const mean = pairs.reduce((a, b) => a + b, 0) / Math.max(1, pairs.length);
|
||||
const twins = pairs.filter((d) => d < 0.03).length;
|
||||
// Against the library's own median distance, so this measures the CHOOSING
|
||||
// rather than the library — a bar in absolute units would drift every time
|
||||
// a scene was added.
|
||||
return expect(pairs.length > 0 && mean > 0.08 && twins / pairs.length < 0.05,
|
||||
`${pairs.length} stacked pairs · mean distance ${mean.toFixed(3)} · ` +
|
||||
`${twins} near-twins (${((twins / Math.max(1, pairs.length)) * 100).toFixed(0)}%)`);
|
||||
});
|
||||
|
||||
@ -27,6 +27,7 @@ import { FeatureTrack } from '../audio/FeatureTrack.js';
|
||||
import { synthesizeSectioned } from '../audio/synth.js';
|
||||
import { song } from '../audio/songbank.js';
|
||||
import { storyStateAt, STORY_VARS, PLOT_NAMES } from '../look/Story.js';
|
||||
import { subjectOf } from '../look/stack.js';
|
||||
|
||||
/**
|
||||
* Bank songs rather than the two-section synthetics the other phases use.
|
||||
@ -107,7 +108,7 @@ check(13, 'the second time a kind happens is not the first time again', () => {
|
||||
if (list.length < 2) continue;
|
||||
const first = list[0];
|
||||
const last = list[list.length - 1];
|
||||
const sceneChanged = first.layers[0].module.name !== last.layers[0].module.name;
|
||||
const sceneChanged = subjectOf(first.layers).module.name !== subjectOf(last.layers).module.name;
|
||||
const pushed = Math.abs(last.story.tension - first.story.tension);
|
||||
seen.push({ name, kind, sceneChanged, pushed });
|
||||
}
|
||||
@ -210,7 +211,7 @@ check(13, 'a recapitulation actually recapitulates', () => {
|
||||
// residue on everything else that makes the ending a return rather
|
||||
// than a rewind.
|
||||
rows.push({
|
||||
same: intro.layers[0].module.name === outro.layers[0].module.name,
|
||||
same: subjectOf(intro.layers).module.name === subjectOf(outro.layers).module.name,
|
||||
moved: Math.max(...['journey', 'reveal', 'tension'].map((k) =>
|
||||
Math.abs(outro.story[k] - intro.story[k]))),
|
||||
});
|
||||
|
||||
@ -14,6 +14,7 @@ import { generateLook, rerollSection, rerollLook, describeLook } from '../look/L
|
||||
import { paletteContrast, relativeLuminance } from '../look/palette.js';
|
||||
import { frameDistance, frameLuminance, frameVariance } from '../engine/hash.js';
|
||||
import { testTrack } from './phase1.js';
|
||||
import { subjectOf } from '../look/stack.js';
|
||||
|
||||
/** Four deliberately different tracks: the differentiation gate needs real spread. */
|
||||
let cachedBattery = null;
|
||||
@ -34,15 +35,22 @@ export function battery() {
|
||||
|
||||
export function renderLookFrame(engine, track, look, frame) {
|
||||
const section = look.sections[track.sectionIndexAt(frame)] || look.sections[0];
|
||||
const layer = section.layers[0];
|
||||
engine.setLayerSpecs([{
|
||||
// The WHOLE stack, not the subject alone.
|
||||
//
|
||||
// This rendered `layers[0]` back when that was the section's only layer,
|
||||
// and rendering one layer of a composed section answers a question nobody
|
||||
// asked: a composable shot on its own is a few bright things on black,
|
||||
// which is exactly what it is not supposed to be shown as. Measured, it
|
||||
// reported a dead frame — luminance 0.0006 — for a section that renders
|
||||
// perfectly well with the ground it was built with underneath it.
|
||||
engine.setLayerSpecs(section.layers.map((layer) => ({
|
||||
module: layer.module,
|
||||
params: layer.params,
|
||||
seed: layer.seed,
|
||||
opacity: layer.opacity,
|
||||
blend: layer.blend,
|
||||
palette: look.palette,
|
||||
}]);
|
||||
})));
|
||||
engine.compositor.setPost(look.post).setFeedback(look.feedback);
|
||||
engine.compositor.reset();
|
||||
return Uint8Array.from(engine.readPixels(engine.renderFrame(frame)));
|
||||
@ -55,9 +63,10 @@ check(3, 'the same audio always produces the same look', () => {
|
||||
const b = generateLook(track, { samples });
|
||||
|
||||
if (a.seed !== b.seed) return expect(false, `seeds differ: ${a.seed} vs ${b.seed}`);
|
||||
const sameScenes = a.sections.every((s, i) => s.layers[0].module.name === b.sections[i].layers[0].module.name);
|
||||
const sameParams = JSON.stringify(a.sections.map((s) => s.layers[0].params))
|
||||
=== JSON.stringify(b.sections.map((s) => s.layers[0].params));
|
||||
const sameScenes = a.sections.every((s, i) =>
|
||||
subjectOf(s.layers).module.name === subjectOf(b.sections[i].layers).module.name);
|
||||
const sameParams = JSON.stringify(a.sections.map((s) => subjectOf(s.layers).params))
|
||||
=== JSON.stringify(b.sections.map((s) => subjectOf(s.layers).params));
|
||||
const samePalette = JSON.stringify(a.palette) === JSON.stringify(b.palette);
|
||||
|
||||
return expect(sameScenes && sameParams && samePalette,
|
||||
@ -144,7 +153,7 @@ check(3, 'different tracks get different looks at the same seed', () => {
|
||||
const palettes = looks.map(({ look }) => look.palette.map(relativeLuminance).join(','));
|
||||
const uniquePalettes = new Set(palettes).size;
|
||||
const sceneSets = looks.map(({ look }) =>
|
||||
[...new Set(look.sections.map((s) => s.layers[0].module.name))].sort().join('+'));
|
||||
[...new Set(look.sections.map((s) => subjectOf(s.layers).module.name))].sort().join('+'));
|
||||
|
||||
return expect(uniquePalettes === looks.length,
|
||||
`${uniquePalettes}/${looks.length} distinct palettes at a fixed seed · ` +
|
||||
@ -173,7 +182,7 @@ check(3, 'every generated look renders a live frame on every track', () => {
|
||||
const lum = frameLuminance(pixels);
|
||||
const variance = frameVariance(pixels);
|
||||
if (lum < 0.0008 || lum > 0.99 || variance < 0.0015) {
|
||||
problems.push(`${name} s${s} ${section.kind}/${section.layers[0].module.name}: ` +
|
||||
problems.push(`${name} s${s} ${section.kind}/${subjectOf(section.layers).module.name}: ` +
|
||||
`lum ${lum.toFixed(4)} var ${variance.toFixed(4)}`);
|
||||
}
|
||||
}
|
||||
@ -194,7 +203,7 @@ check(3, 'sections of the same kind share a scene', () => {
|
||||
const byKind = new Map();
|
||||
let violations = 0;
|
||||
for (const s of look.sections) {
|
||||
const name = s.layers[0].module.name;
|
||||
const name = subjectOf(s.layers).module.name;
|
||||
if (byKind.has(s.kind) && byKind.get(s.kind) !== name) violations++;
|
||||
byKind.set(s.kind, name);
|
||||
}
|
||||
@ -205,14 +214,14 @@ check(3, 'sections of the same kind share a scene', () => {
|
||||
check(3, 'reroll changes a section and respects locks', () => {
|
||||
const track = testTrack();
|
||||
const look = generateLook(track, { seed: 555 });
|
||||
const before = JSON.stringify(look.sections[0].layers[0].params);
|
||||
const before = JSON.stringify(subjectOf(look.sections[0].layers).params);
|
||||
|
||||
rerollSection(look, track, 0, 1);
|
||||
const afterUnlocked = JSON.stringify(look.sections[0].layers[0].params);
|
||||
const afterUnlocked = JSON.stringify(subjectOf(look.sections[0].layers).params);
|
||||
|
||||
look.sections[0].locked = true;
|
||||
rerollSection(look, track, 0, 2);
|
||||
const afterLocked = JSON.stringify(look.sections[0].layers[0].params);
|
||||
const afterLocked = JSON.stringify(subjectOf(look.sections[0].layers).params);
|
||||
|
||||
return expect(before !== afterUnlocked && afterUnlocked === afterLocked,
|
||||
`changed when unlocked: ${before !== afterUnlocked}, held when locked: ${afterUnlocked === afterLocked}`);
|
||||
@ -222,12 +231,12 @@ check(3, 'a whole-track reroll preserves locked sections', () => {
|
||||
const track = testTrack();
|
||||
const look = generateLook(track, { seed: 777 });
|
||||
look.sections[0].locked = true;
|
||||
const lockedScene = look.sections[0].layers[0].module.name;
|
||||
const lockedParams = JSON.stringify(look.sections[0].layers[0].params);
|
||||
const lockedScene = subjectOf(look.sections[0].layers).module.name;
|
||||
const lockedParams = JSON.stringify(subjectOf(look.sections[0].layers).params);
|
||||
|
||||
const next = rerollLook(look, track, 888);
|
||||
return expect(
|
||||
next.sections[0].layers[0].module.name === lockedScene &&
|
||||
JSON.stringify(next.sections[0].layers[0].params) === lockedParams,
|
||||
subjectOf(next.sections[0].layers).module.name === lockedScene &&
|
||||
JSON.stringify(subjectOf(next.sections[0].layers).params) === lockedParams,
|
||||
`locked section survived a full reroll (${lockedScene})`);
|
||||
});
|
||||
|
||||
@ -18,6 +18,7 @@ import { peakFlashRate } from '../engine/flash.js';
|
||||
import { particleField } from '../scenes/layers3d/particles.js';
|
||||
import { nebula } from '../scenes/shader/nebula.js';
|
||||
import { BLEND_MODES } from '../engine/Layer.js';
|
||||
import { subjectOf } from '../look/stack.js';
|
||||
|
||||
const PALETTE = [
|
||||
[0.06, 0.03, 0.16], [0.85, 0.15, 0.55], [0.15, 0.75, 0.95],
|
||||
@ -192,7 +193,7 @@ check(5, 'generated looks stay within the flash-rate ceiling', () => {
|
||||
}
|
||||
const rate = peakFlashRate(luminance, 60);
|
||||
const section = show.look.sections[cue.sectionIndex];
|
||||
const scene = (section.variants[cue.variant] || section.layers)[0].module.name;
|
||||
const scene = subjectOf(section.variants[cue.variant] || section.layers).module.name;
|
||||
const label = `seed ${s} ${section.kind}/${scene}`;
|
||||
if (rate > worst) { worst = rate; worstLabel = label; }
|
||||
if (rate > 3) problems.push(`${label}: ${rate}/s`);
|
||||
|
||||
@ -16,6 +16,7 @@ import { FeatureTrack, featureProviderFor } from '../audio/FeatureTrack.js';
|
||||
import { synthesizeSectioned } from '../audio/synth.js';
|
||||
import { generateLook } from '../look/LookGenerator.js';
|
||||
import { frameDistance, frameMaxDelta, frameLuminance, frameVariance } from '../engine/hash.js';
|
||||
import { subjectOf } from '../look/stack.js';
|
||||
|
||||
const PALETTE = [
|
||||
[0.06, 0.03, 0.16], [0.85, 0.15, 0.55], [0.15, 0.75, 0.95],
|
||||
@ -247,7 +248,7 @@ check(7, 'quiet sections now get minimal scenes', () => {
|
||||
for (const section of look.sections) {
|
||||
if (!(section.kind in kinds)) continue;
|
||||
total++;
|
||||
const family = section.layers[0].module.family;
|
||||
const family = subjectOf(section.layers).module.family;
|
||||
if (restful.has(family)) restfulCount++;
|
||||
if (family === 'minimal') minimalCount++;
|
||||
}
|
||||
|
||||
@ -12,6 +12,7 @@ import { synthesizeSectioned } from '../audio/synth.js';
|
||||
import { generateLook } from '../look/LookGenerator.js';
|
||||
import { MIN_SHOT_SECONDS, MAX_SHOT_SECONDS, HARD_CUT_ENERGY } from '../look/shots.js';
|
||||
import { frameDistance } from '../engine/hash.js';
|
||||
import { subjectOf } from '../look/stack.js';
|
||||
|
||||
let cached = null;
|
||||
function track8() {
|
||||
@ -131,7 +132,7 @@ check(8, 'the same section kind reuses the same roster', () => {
|
||||
for (const look of looks()) {
|
||||
const byKind = new Map();
|
||||
for (const section of look.sections) {
|
||||
const roster = (section.variants || [section.layers]).map((v) => v[0].module.name).join('+');
|
||||
const roster = (section.variants || [section.layers]).map((v) => subjectOf(v).module.name).join('+');
|
||||
const seen = byKind.get(section.kind);
|
||||
if (seen && seen !== roster) problems.push(`${section.kind}: ${seen} vs ${roster}`);
|
||||
byKind.set(section.kind, roster);
|
||||
|
||||
@ -20,6 +20,7 @@ import { Rng } from '../engine/rng.js';
|
||||
import { FeatureTrack, featureProviderFor } from '../audio/FeatureTrack.js';
|
||||
import { synthesizeSectioned } from '../audio/synth.js';
|
||||
import { frameMaxDelta } from '../engine/hash.js';
|
||||
import { subjectOf } from '../look/stack.js';
|
||||
|
||||
const PALETTE = [
|
||||
[0.06, 0.03, 0.16], [0.85, 0.15, 0.55], [0.15, 0.75, 0.95],
|
||||
@ -40,7 +41,7 @@ function looks(count = 8) {
|
||||
return Array.from({ length: count }, (_, i) => generateLook(track, { seed: 3000 + i * 6841 }));
|
||||
}
|
||||
|
||||
const castOf = (look) => look.sections.flatMap((s) => (s.variants || [s.layers]).map((v) => v[0].module));
|
||||
const castOf = (look) => look.sections.flatMap((s) => (s.variants || [s.layers]).map((v) => subjectOf(v).module));
|
||||
|
||||
/** Two personalities differing in exactly one trait, for the "does it show" checks. */
|
||||
function pairDifferingIn(trait) {
|
||||
@ -140,7 +141,7 @@ check(9, 'casting leans hard on the track\'s signature', () => {
|
||||
}
|
||||
for (const section of look.sections) {
|
||||
anchors++;
|
||||
if (sceneHonours(section.layers[0].module, signature)) anchorsHonoured++;
|
||||
if (sceneHonours(subjectOf(section.layers).module, signature)) anchorsHonoured++;
|
||||
}
|
||||
}
|
||||
const share = total ? honoured / total : 0;
|
||||
|
||||
@ -35,6 +35,8 @@ import { Rng } from '../../engine/rng.js';
|
||||
import { videoSignature, signatureDistance, STRUCTURAL } from './signature.js';
|
||||
import { songBank } from '../../audio/songbank.js';
|
||||
import { hashString } from '../../engine/rng.js';
|
||||
import { subjectOf, isGround } from '../../look/stack.js';
|
||||
import { canGround } from '../../scenes/surface.js';
|
||||
|
||||
const RENDER = { width: 160, height: 90 };
|
||||
|
||||
@ -70,11 +72,17 @@ export function signatureForChaos(track, seed, options = {}) {
|
||||
const look = generateLook(track, { seed: seed >>> 0 });
|
||||
const rng = new Rng((seed * 2246822519) >>> 0);
|
||||
const pool = scenes.filter(canBackground);
|
||||
// The ground keeps its JOB when its identity is thrown away. Recasting
|
||||
// it from the whole library would give the reference videos thin,
|
||||
// half-black frames no real video can have any more, and a ceiling
|
||||
// measured on those is a ceiling for a pipeline that does not exist —
|
||||
// it fell below the floor the first time this ran.
|
||||
const groundPool = scenes.filter(canGround);
|
||||
const temperament = look.personality && look.personality.temperament;
|
||||
for (const section of look.sections) {
|
||||
for (const variant of section.variants) {
|
||||
for (const layer of variant) {
|
||||
layer.module = rng.pick(pool);
|
||||
layer.module = rng.pick(isGround(layer) ? groundPool : pool);
|
||||
layer.params = sampleValues(layer.module, rng, section.bias, temperament);
|
||||
layer.seed = rng.int(0, 0x7fffffff);
|
||||
}
|
||||
@ -435,7 +443,7 @@ export function measureSpecDiversity(track, { seeds = 32, seed0 = 0x5eed } = {})
|
||||
grain: entropy(looks.map((l) => l.grain.mode)),
|
||||
framing: entropy(looks.map((l) => l.framing.mode)),
|
||||
paletteArc: entropy(looks.map((l) => l.paletteArc.mode)),
|
||||
anchorScenes: entropy(looks.map((l) => l.sections.map((s) => s.layers[0].module.name).join('>'))),
|
||||
anchorScenes: entropy(looks.map((l) => l.sections.map((s) => subjectOf(s.layers).module.name).join('>'))),
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@ -30,6 +30,23 @@ void main() {
|
||||
else if (u_mode == 3) result = mix(base.rgb, base.rgb * src.rgb, a); // multiply
|
||||
else if (u_mode == 4) result = mix(base.rgb, blendOverlay(base.rgb, src.rgb), a);
|
||||
else if (u_mode == 5) result = mix(base.rgb, blendSoftLight(base.rgb, src.rgb), a);
|
||||
// LUMAKEY — the source's own brightness is its alpha.
|
||||
//
|
||||
// For a layer that is a PICTURE with black around it, which is what a
|
||||
// composable scene is: it replaces the base where it paints and leaves it
|
||||
// where it does not. Screen was doing this job and screen is a lightening
|
||||
// operator — over a filled ground it drives everything toward white, which
|
||||
// measured as a median 24% of the frame clipped and whole sections at
|
||||
// 100%. This keeps the shot's own colour instead of adding it to the bed's.
|
||||
//
|
||||
// The key is smoothstepped rather than raw luma so a dark-but-present shot
|
||||
// does not dissolve into the ground, and gamma-ish weighted to keep thin
|
||||
// bright lines opaque.
|
||||
else if (u_mode == 6) {
|
||||
float key = dot(src.rgb, vec3(0.2126, 0.7152, 0.0722));
|
||||
key = smoothstep(0.02, 0.32, key);
|
||||
result = mix(base.rgb, src.rgb, key * a);
|
||||
}
|
||||
else result = mix(base.rgb, src.rgb, a); // normal
|
||||
|
||||
gl_FragColor = vec4(result, max(base.a, a));
|
||||
@ -65,7 +82,26 @@ void main() {
|
||||
vec3 hist = texture2D(u_history, clamp(warped, 0.0, 1.0)).rgb;
|
||||
// Decay strictly below 1 keeps the loop convergent; the 10k-frame stability
|
||||
// check in tools/ verifies it neither saturates to white nor dies to black.
|
||||
vec3 outC = cur + hist * u_decay * u_amount;
|
||||
//
|
||||
// NORMALISED, which it was not. cur + hist * decay * amount is an
|
||||
// accumulator: a static image settles at 1/(1 - decay*amount) times its own
|
||||
// brightness, which is 2.3x at the settings the generator hands out. That
|
||||
// was survivable while a frame was a few bright things on black and stopped
|
||||
// being survivable the moment every section stood on a filled ground —
|
||||
// measured, entire sections rendered as pure white, and turning feedback off
|
||||
// took the same frame from 100% blown to 34% mean luminance.
|
||||
//
|
||||
// Dividing by the gain keeps the trail — moving content still smears, which
|
||||
// is the whole point — instead of stacking exposures.
|
||||
//
|
||||
// PARTIALLY, at 0.6, rather than all the way. Full normalisation is the
|
||||
// mathematically tidy answer and it takes the lift out with the blowout:
|
||||
// measured over 74 sections, the median frame went from 95% painted to 71%
|
||||
// and fourteen fell under the black-frame floor. Feedback contributing SOME
|
||||
// brightness is part of what the looks were built around. At 0.6 a still
|
||||
// frame settles about 1.2x its drawn brightness instead of 2.3x.
|
||||
float gain = u_decay * u_amount;
|
||||
vec3 outC = (cur + hist * gain) / (1.0 + gain * 0.6);
|
||||
gl_FragColor = vec4(min(outC, vec3(4.0)), 1.0);
|
||||
}
|
||||
`;
|
||||
@ -155,6 +191,19 @@ void main() {
|
||||
float v = 1.0 - u_vignette * dot(dir, dir) * 2.0;
|
||||
col *= clamp(v, 0.0, 1.0);
|
||||
|
||||
// HIGHLIGHT SHOULDER — the top end rolls off instead of clipping.
|
||||
//
|
||||
// Everything under the knee is untouched, so the image keeps its contrast;
|
||||
// above it, values compress toward but never reach 1. Without this, bloom
|
||||
// plus a filled ground plus a bright palette clips large areas to pure
|
||||
// white — measured at 38% of a drop's frame — and clipped white is not
|
||||
// bright, it is missing: every difference inside it is gone.
|
||||
//
|
||||
// Cheap, per channel, and deliberately not a full filmic curve. The job is
|
||||
// to stop the frame flattening out at the top, not to grade it.
|
||||
vec3 over = max(col - 0.75, vec3(0.0));
|
||||
col = min(col, vec3(0.75)) + over / (1.0 + over * 4.0);
|
||||
|
||||
// Deterministic grain: keyed on frame index, never on a random source.
|
||||
//
|
||||
// Cell size and refresh rate are separate on purpose. Fine-and-boiling is
|
||||
@ -204,4 +253,5 @@ void main() { gl_FragColor = texture2D(u_tex, vUv); }
|
||||
|
||||
export const BLEND_MODE_IDS = {
|
||||
normal: 0, add: 1, screen: 2, multiply: 3, overlay: 4, softlight: 5,
|
||||
lumakey: 6,
|
||||
};
|
||||
|
||||
@ -6,6 +6,8 @@ import { shiftPalette } from './palette.js';
|
||||
import { frameShot, neutralFraming } from './framing.js';
|
||||
import { planGaze, gazeAt } from './Camera.js';
|
||||
import { storyStateAt, NEUTRAL_STATE } from './Story.js';
|
||||
import { subjectIndexOf, isGround } from './stack.js';
|
||||
import { groundPersonalityFrom } from '../scenes/surface.js';
|
||||
|
||||
/**
|
||||
* Drives the look across the song.
|
||||
@ -597,7 +599,7 @@ export class ArcDriver {
|
||||
layer.opacity = slot === 0 ? 1 : spec.opacity;
|
||||
layer.blend = slot === 0 ? 'normal' : spec.blend;
|
||||
layer.setPalette(palette);
|
||||
layer.setPersonality(personality);
|
||||
layer.setPersonality(isGround(spec) ? groundPersonalityFrom(personality) : personality);
|
||||
layer.setFraming(outgoingFraming);
|
||||
layers.push(layer);
|
||||
}
|
||||
@ -610,7 +612,7 @@ export class ArcDriver {
|
||||
layer.opacity = (slot === 0 ? 1 : spec.opacity) * (fading ? eased : 1);
|
||||
layer.blend = slot === 0 ? 'normal' : spec.blend;
|
||||
layer.setPalette(palette);
|
||||
layer.setPersonality(personality);
|
||||
layer.setPersonality(isGround(spec) ? groundPersonalityFrom(personality) : personality);
|
||||
layer.setFraming(framing);
|
||||
layers.push(layer);
|
||||
}
|
||||
@ -622,7 +624,11 @@ export class ArcDriver {
|
||||
variant: cue.variant,
|
||||
kind: section.kind,
|
||||
crossfade: fading ? eased : 0,
|
||||
sceneName: this._specFor(cue.sectionIndex, cue.variant, 0).module.name,
|
||||
// The SHOT's name, not the ground's. Every stack starts with a bed
|
||||
// now, and the HUD naming it would report the same handful of
|
||||
// canvases for every section of every video.
|
||||
sceneName: this._specFor(
|
||||
cue.sectionIndex, cue.variant, this._subjectSlot(cue)).module.name,
|
||||
buildSlope: features ? features.buildSlope || 0 : 0,
|
||||
// Where the story is, for the HUD and the checks. A video that is
|
||||
// supposed to be going somewhere should be able to say where.
|
||||
@ -636,10 +642,18 @@ export class ArcDriver {
|
||||
return layers;
|
||||
}
|
||||
|
||||
_stackSize(cue) {
|
||||
/** Which slot of a cue's stack is the shot. See look/stack.js. */
|
||||
_subjectSlot(cue) {
|
||||
return subjectIndexOf(this._stackFor(cue));
|
||||
}
|
||||
|
||||
_stackFor(cue) {
|
||||
const section = this.look.sections[cue.sectionIndex];
|
||||
const stack = (section.variants && section.variants[cue.variant]) || section.layers;
|
||||
return stack.length;
|
||||
return (section.variants && section.variants[cue.variant]) || section.layers;
|
||||
}
|
||||
|
||||
_stackSize(cue) {
|
||||
return this._stackFor(cue).length;
|
||||
}
|
||||
|
||||
/** Layers changed identity — the compositor needs the new list. */
|
||||
|
||||
@ -7,13 +7,20 @@
|
||||
import { Rng, hashSamples } from '../engine/rng.js';
|
||||
import { AudioPalette, generateUsablePalette } from './palette.js';
|
||||
import { scenes, scenesInFamily, FAMILIES } from '../scenes/registry.js';
|
||||
import { sampleValues, defaultValues, surfaceOf, canBackground } from '../params/schema.js';
|
||||
import { sampleValues, defaultValues, canBackground } from '../params/schema.js';
|
||||
import {
|
||||
canGround, surfaceOf, structuralDistance, groundBiasFrom, groundTemperamentFrom,
|
||||
coverageOf as sceneCoverage, GROUND_MIN,
|
||||
} from '../scenes/surface.js';
|
||||
import { GROUND, subjectOf } from './stack.js';
|
||||
import { planShots } from './shots.js';
|
||||
import {
|
||||
generatePersonality, sceneHonours, signatureWeight, describePersonality,
|
||||
} from './Personality.js';
|
||||
import { deriveGrain, describeGrain, applyGrainToPost } from './grain.js';
|
||||
import { pickDirector, directorByName } from './directors.js';
|
||||
import {
|
||||
pickDirector, directorByName, crowdOf, blazeOf, RESTFUL_FAMILIES, QUIET_KINDS,
|
||||
} from './directors.js';
|
||||
import { derivePaletteArc, describePaletteArc } from './paletteArc.js';
|
||||
import { deriveFramingStyle, describeFraming } from './framing.js';
|
||||
import { deriveCamera, describeCamera } from './Camera.js';
|
||||
@ -208,6 +215,120 @@ function rosterSizeFor(kind) {
|
||||
return (KIND_ENERGY[kind] ?? 0.5) > 0.5 ? 4 : 3;
|
||||
}
|
||||
|
||||
/**
|
||||
* The most painted frame a stack is allowed to add up to.
|
||||
*
|
||||
* 1.0 is one filled picture. 2.0 is two of them stacked, which is where the
|
||||
* compositor's blends stop producing depth and start producing mud — past it
|
||||
* the layers are no longer readable as separate things, so nothing is gained
|
||||
* by the third pass except cost.
|
||||
*/
|
||||
const MAX_COVERAGE = 2.0;
|
||||
|
||||
|
||||
/**
|
||||
* How full THIS section's frame is allowed to get, in painted coverage.
|
||||
*
|
||||
* Three inputs, in the order they matter:
|
||||
*
|
||||
* the director — how much this point of view lets happen at once. A
|
||||
* brutalist video is one large thing everywhere in it; a
|
||||
* corrupt one is everything over everything. See crowdOf.
|
||||
* the song — a loud, dense section carries more than a quiet one.
|
||||
* the stage — where the story is. `population` is literally how crowded
|
||||
* this point in the video wants to be, and layering is the
|
||||
* one lever on it that needs no cooperation from the scenes.
|
||||
*
|
||||
* The floor is GROUND_MIN because the ground is not optional: a section always
|
||||
* pays for its bed first, and the budget governs what may be stacked on it.
|
||||
*/
|
||||
function coverageBudgetFor(bias, story, director) {
|
||||
const stage = story ? story.population * 0.6 + story.tension * 0.4 : 0.5;
|
||||
const want = 0.6 + bias.energy * 0.5 + bias.density * 0.2 + (stage - 0.5) * 0.5;
|
||||
return Math.max(GROUND_MIN, Math.min(MAX_COVERAGE, want * crowdOf(director)));
|
||||
}
|
||||
|
||||
/** The budget a KIND is planned against, before a section's own bias exists. */
|
||||
function kindBudget(kind, director) {
|
||||
return coverageBudgetFor(
|
||||
{ energy: KIND_ENERGY[kind] ?? 0.5, density: 0.5 }, null, director);
|
||||
}
|
||||
|
||||
/**
|
||||
* The GROUND a kind's sections stand on: a canvas that paints at least half the
|
||||
* frame, cast once per kind so a section's cuts change the shot without moving
|
||||
* the video to another world.
|
||||
*
|
||||
* Most of the library cannot do this job and is not supposed to — two thirds of
|
||||
* it is composable, which means it reads as elements ON something and has
|
||||
* nothing of its own behind them. Those scenes were being cast as backgrounds
|
||||
* anyway, which is why a section could be a few bright things on black for
|
||||
* ninety seconds. The ground is what they are on.
|
||||
*
|
||||
* Chosen against the kind's budget rather than at random: a scene that paints
|
||||
* 98% of the frame is a legitimate ground for a drop and the wrong bed for an
|
||||
* intro, because everything the intro puts on it has to remain visible.
|
||||
*/
|
||||
function castGround(kind, roster, rng, signature, director, used) {
|
||||
const pool = scenes.filter(canGround);
|
||||
if (!pool.length) return null;
|
||||
|
||||
const families = director.families[kind] || Object.keys(FAMILIES);
|
||||
const quiet = QUIET_KINDS.includes(kind);
|
||||
// What the shots standing on it will paint, so the ground leaves room for
|
||||
// the section it is under.
|
||||
const reserve = roster.length
|
||||
? roster.reduce((sum, m) => sum + sceneCoverage(m), 0) / roster.length
|
||||
: 0.2;
|
||||
const headroom = kindBudget(kind, director) - reserve;
|
||||
|
||||
const weights = pool.map((m) => {
|
||||
const at = families.indexOf(m.family);
|
||||
// Off-family grounds stay reachable — the ground is a bed, not the
|
||||
// director's statement — but the director still leads.
|
||||
let w = at >= 0 ? families.length - at : 0.35;
|
||||
// The quiet-kind rule applies to the floor as well. An intro standing on
|
||||
// a strobing glitch canvas is the mistake that rule exists to prevent,
|
||||
// and it is worse underneath than on top because nothing hides it.
|
||||
if (quiet && !RESTFUL_FAMILIES.includes(m.family)) w *= 0.15;
|
||||
w *= signatureWeight(m, signature);
|
||||
// Overshooting the budget is allowed and discouraged: the ground is
|
||||
// mandatory, so an oversized one is spent frame the shot cannot use.
|
||||
w /= 1 + 4 * Math.max(0, sceneCoverage(m) - headroom);
|
||||
// The bed has to be unlike the things standing on it, or the section is
|
||||
// one texture at double density. Measured against the whole roster,
|
||||
// because every member of it will be shot against this ground.
|
||||
w *= contrastWeight(m, roster.map((r) => ({ module: r })));
|
||||
// A video returns to its world rather than visiting six of them.
|
||||
if (used.has(m.name)) w *= 3;
|
||||
// Nothing stands on itself. If the kind's own anchor is groundable it
|
||||
// will be its own ground in buildStack, and this pick is for the rest.
|
||||
if (roster.some((r) => r.name === m.name)) w *= 0.1;
|
||||
return Math.max(1e-4, w);
|
||||
});
|
||||
|
||||
const ground = rng.pickWeighted(pool, weights);
|
||||
used.add(ground.name);
|
||||
return ground;
|
||||
}
|
||||
|
||||
/** One ground per section kind. See castGround. */
|
||||
function assignGroundsByKind(rosterByKind, rng, signature, director) {
|
||||
const grounds = new Map();
|
||||
const used = new Set();
|
||||
// Loud kinds first, for the same reason rosters are assigned that way: they
|
||||
// are what the video is remembered for, so they choose their world first.
|
||||
const priority = ['drop', 'sustain', 'build', 'breakdown', 'intro', 'outro'];
|
||||
const kinds = [...rosterByKind.keys()]
|
||||
.sort((a, b) => priority.indexOf(a) - priority.indexOf(b));
|
||||
for (const kind of kinds) {
|
||||
grounds.set(kind, castGround(
|
||||
kind, rosterByKind.get(kind) || [], rng.fork(`ground:${kind}`),
|
||||
signature, director, used));
|
||||
}
|
||||
return grounds;
|
||||
}
|
||||
|
||||
/**
|
||||
* Scenes are chosen per section KIND, not per section — and a kind gets a
|
||||
* ROSTER of two or three, not one.
|
||||
@ -341,10 +462,49 @@ function derivePost(summary, rng, grain) {
|
||||
}
|
||||
|
||||
/**
|
||||
* One layer stack: a background scene, and sometimes one or two composable
|
||||
* scenes composited over it.
|
||||
* How much a candidate would add to a stack, structurally.
|
||||
*
|
||||
* background — the shot. Always present, always opaque.
|
||||
* The question a stack has to answer is not "are these two scenes different
|
||||
* things" but "will a viewer see two things". Those come apart: the measured
|
||||
* distance between two scenes' structural profiles is what a viewer reads, and
|
||||
* it does not follow the family labels. So a candidate is weighted by how far
|
||||
* it sits from everything already in the stack, taking the CLOSEST such
|
||||
* distance — one near-twin in the stack is enough to make the addition read as
|
||||
* more of the same, however unlike the other layers it is.
|
||||
*
|
||||
* A lean, not a filter, and for the same reason the signature weighting is:
|
||||
* measured distances are a description of the library as it is today, and a
|
||||
* generator that obeyed them exactly would cast the same handful of contrasts
|
||||
* in every video. Unmeasured scenes score neutral rather than zero — never
|
||||
* having been rendered is not evidence of sameness.
|
||||
*/
|
||||
const CONTRAST_NEUTRAL = 0.12;
|
||||
|
||||
function contrastWeight(candidate, stack) {
|
||||
let closest = Infinity;
|
||||
for (const layer of stack) {
|
||||
const d = structuralDistance(candidate, layer.module);
|
||||
if (d !== null) closest = Math.min(closest, d);
|
||||
}
|
||||
if (closest === Infinity) closest = CONTRAST_NEUTRAL;
|
||||
// 0.02 apart (twins) → 0.25; 0.12 (typical) → 1.0; 0.30 (unalike) → 2.1.
|
||||
return Math.max(0.15, Math.min(2.5, 0.15 + (closest / CONTRAST_NEUTRAL) * 0.85));
|
||||
}
|
||||
|
||||
/**
|
||||
* One layer stack: the ground, the shot standing on it, and sometimes a pass
|
||||
* or two composited over both.
|
||||
*
|
||||
* ground — a canvas painting at least half the frame. Always present,
|
||||
* always opaque, and usually NOT the scene the section is about:
|
||||
* two thirds of the library is composable, and a composable
|
||||
* scene on its own is a few bright things on black. It is the
|
||||
* one layer the section does not choose freely — see castGround.
|
||||
* When the shot is itself a full canvas it IS the ground, because
|
||||
* two canvases stacked is two pictures fighting.
|
||||
* shot — what the section is about. Screened over the ground rather
|
||||
* than replacing it, so what it does not paint is the ground
|
||||
* rather than black. `subjectOf` finds it; see look/stack.js.
|
||||
* overlay — a composable scene at partial opacity. Not always: this is the
|
||||
* variation valve, and a stack that always doubled up would read
|
||||
* as permanently cluttered rather than as occasionally layered.
|
||||
@ -360,14 +520,80 @@ function derivePost(summary, rng, grain) {
|
||||
*
|
||||
* Quiet material mostly goes without any — an intro is supposed to be sparse.
|
||||
*/
|
||||
function buildStack(module, overlayRoster, bias, rng, temperament, story = null) {
|
||||
const layers = [{
|
||||
function buildStack(module, overlayRoster, bias, rng, temperament, story = null,
|
||||
{ ground = null, director = null, kind = null } = {}) {
|
||||
const sectionKind = kind;
|
||||
// A shot that fills the frame by itself is its own ground; anything else
|
||||
// gets one under it.
|
||||
const standsAlone = canGround(module) || !ground;
|
||||
|
||||
// --- the blaze ------------------------------------------------------
|
||||
// Whether THIS section is one the director lets bloom out: the shot added
|
||||
// to its ground rather than keyed onto it, so the two brightnesses sum and
|
||||
// the highlights go to paper.
|
||||
//
|
||||
// A decision, and a rationed one. Screening every shot over its ground is
|
||||
// how a median quarter of every frame in every video ended up clipped —
|
||||
// the effect was not wrong, being the default was. It has to be earned:
|
||||
// the director's appetite, times a loud section, times a late point in the
|
||||
// story. Quiet kinds never blaze; a breakdown that goes white is not a
|
||||
// decision, it is a bug with a rationale.
|
||||
const blaze = !standsAlone
|
||||
&& !QUIET_KINDS.includes(sectionKind)
|
||||
&& rng.bool(blazeOf(director) * clamp01(bias.energy * 1.2)
|
||||
* (story ? 0.4 + story.tension * 0.9 : 0.7));
|
||||
|
||||
// The shot is sampled first and from the caller's rng, so a stack draws the
|
||||
// same shot it always did and the ground arrives underneath it rather than
|
||||
// in front of it in the seed stream.
|
||||
const shot = {
|
||||
module,
|
||||
params: sampleValues(module, rng, bias, temperament),
|
||||
seed: rng.int(0, 0x7fffffff),
|
||||
// Keyed over the ground on its own brightness by default: what the shot
|
||||
// leaves unpainted is then the ground rather than black, which is the
|
||||
// whole point of standing it on one, and what it DOES paint stays its
|
||||
// own colour. 'screen' is the blaze — see above, and passes.js.
|
||||
blend: standsAlone ? 'normal' : (blaze ? 'screen' : 'lumakey'),
|
||||
// Carried so the HUD, the checks and a later pass over the look can all
|
||||
// tell a deliberate bloom-out from a broken one.
|
||||
blaze,
|
||||
opacity: 1,
|
||||
};
|
||||
|
||||
// --- ground ---------------------------------------------------------
|
||||
// Sampled calmer and sparser than it would be as a shot, because a bed the
|
||||
// shot cannot be read against is not a bed.
|
||||
const layers = [];
|
||||
if (!standsAlone) {
|
||||
const groundRng = rng.fork(`ground:${ground.name}`);
|
||||
layers.push({
|
||||
module: ground,
|
||||
role: GROUND,
|
||||
// Calmed, but NOT thinned — see GROUND_BIAS in scenes/surface.js,
|
||||
// which is also the bias the ground was MEASURED at. The first
|
||||
// version subtracted 0.3 from density here, which is exactly
|
||||
// backwards for a bed: an intro is already biased sparse, so the
|
||||
// ground came out at density zero and the section was thin again
|
||||
// for a new reason. Three of forty rendered sections fell under 30%
|
||||
// painted with a ground under every one of them.
|
||||
params: sampleValues(ground, groundRng, groundBiasFrom(bias),
|
||||
groundTemperamentFrom(temperament)),
|
||||
seed: groundRng.int(0, 0x7fffffff),
|
||||
blend: 'normal',
|
||||
opacity: 1,
|
||||
}];
|
||||
});
|
||||
}
|
||||
layers.push(shot);
|
||||
|
||||
// --- the budget -----------------------------------------------------
|
||||
// What is on the frame so far, and how much more this section is allowed
|
||||
// to put on it. See coverageBudgetFor: the ground and the shot are not
|
||||
// negotiable, so the budget governs the passes over them — a quiet intro
|
||||
// spends everything on its bed and stacks nothing, a crowded drop under a
|
||||
// director with an appetite for it gets two passes.
|
||||
const budget = coverageBudgetFor(bias, story, director);
|
||||
let spent = layers.reduce((sum, l) => sum + sceneCoverage(l.module), 0);
|
||||
|
||||
// --- overlay --------------------------------------------------------
|
||||
// Roughly a third of stacks on busy material, rarely on quiet material, and
|
||||
@ -410,10 +636,26 @@ function buildStack(module, overlayRoster, bias, rng, temperament, story = null)
|
||||
Math.min(0.25, overlayChance * bias.energy * 0.5),
|
||||
];
|
||||
for (const [pass, chance] of chances.entries()) {
|
||||
if (!available.length || !rng.bool(chance)) break;
|
||||
// Prefer a different family so the images argue instead of blurring.
|
||||
// The budget is a wall, and it is also a lean: as the frame fills up
|
||||
// the odds of adding to it fall away before the wall is reached, so a
|
||||
// stack that is already nearly full rarely gets a token last pass.
|
||||
const headroom = budget - spent;
|
||||
const fits = available.filter((m) => sceneCoverage(m) <= headroom);
|
||||
if (!fits.length || !rng.bool(chance * clamp01(headroom / 0.35))) break;
|
||||
available = fits;
|
||||
// Prefer a different family so the images argue instead of blurring —
|
||||
// and then, within that, prefer the ones that MEASURE different.
|
||||
//
|
||||
// Family is a label somebody typed; structural distance is what the
|
||||
// gallery saw when it rendered the two scenes under the same six songs.
|
||||
// They disagree often enough to matter: two 'geometric' scenes can be
|
||||
// 0.31 apart and a 'flow' and an 'organic' scene 0.04, and stacking the
|
||||
// second pair is one picture at double density rather than a picture
|
||||
// with something happening in it. See scenes/surface.js.
|
||||
const offFamily = available.filter((m) => m.family !== module.family);
|
||||
const overlay = rng.pick(offFamily.length ? offFamily : available);
|
||||
const pool = offFamily.length ? offFamily : available;
|
||||
const overlay = rng.pickWeighted(pool, pool.map((m) => contrastWeight(m, layers)));
|
||||
spent += sceneCoverage(overlay);
|
||||
available = available.filter((m) => m.name !== overlay.name
|
||||
&& m.family !== overlay.family);
|
||||
// Screen and add keep the background readable underneath; softlight and
|
||||
@ -486,6 +728,10 @@ export function generateLook(track, {
|
||||
const rosterByKind = assignRostersByKind(
|
||||
track.sections, rng.fork('scenes'), personality.signature, director, pool);
|
||||
if (story.recap) applyRecap(rosterByKind);
|
||||
// What each kind's sections stand on. After the recap, so the outro is
|
||||
// grounded against the roster it actually ends up with.
|
||||
const groundByKind = assignGroundsByKind(
|
||||
rosterByKind, rng.fork('grounds'), personality.signature, director);
|
||||
// The grain treatment: usually none, and when present described rather than
|
||||
// dialled. See look/grain.js.
|
||||
const grain = deriveGrain(summary, rng.fork('grain'));
|
||||
@ -523,9 +769,12 @@ export function generateLook(track, {
|
||||
// so the last occurrence of a kind samples further out than the first.
|
||||
const temperament = temperamentFor(personality.temperament, state);
|
||||
|
||||
const ground = groundByKind.get(section.kind) || null;
|
||||
|
||||
const variants = roster.map((module, v) => buildStack(
|
||||
module, overlayRoster, bias,
|
||||
sectionRng.fork(`variant:${section.index}:${v}`), temperament, state,
|
||||
{ ground, director, kind: section.kind },
|
||||
));
|
||||
|
||||
const shots = planShots(
|
||||
@ -579,7 +828,8 @@ export function rerollSection(look, track, sectionIndex, salt = 0) {
|
||||
|
||||
const rng = new Rng((look.seed ^ (sectionIndex * 0x9e3779b9) ^ (salt * 0x85ebca6b)) >>> 0);
|
||||
const signature = (look.personality && look.personality.signature) || [];
|
||||
const families = directorByName(look.director).families[section.kind] || Object.keys(FAMILIES);
|
||||
const director = directorByName(look.director);
|
||||
const families = director.families[section.kind] || Object.keys(FAMILIES);
|
||||
// A reroll re-draws this section's cast from the same kind of pool the track
|
||||
// was built with, weighted by the signature rather than filtered by it.
|
||||
let candidates = families.flatMap((f) => scenesInFamily(f))
|
||||
@ -602,9 +852,14 @@ export function rerollSection(look, track, sectionIndex, salt = 0) {
|
||||
// A reroll changes what this section is made of. Where it sits in the story
|
||||
// is a property of the song, so it survives untouched.
|
||||
const state = section.story || NEUTRAL_STATE;
|
||||
// The section is re-cast, so its ground is re-cast with it — a reroll that
|
||||
// kept the old bed under new shots would be answering half the question.
|
||||
const ground = castGround(
|
||||
section.kind, roster, rng.fork('ground'), signature, director, new Set());
|
||||
section.variants = roster.map((module, v) => buildStack(
|
||||
module, overlayRoster, section.bias, rng.fork(`variant:${v}`),
|
||||
temperamentFor(look.personality && look.personality.temperament, state), state,
|
||||
{ ground, director, kind: section.kind },
|
||||
));
|
||||
section.shots = planShots(
|
||||
section, track, section.bias, section.variants.length, rng.fork('shots'), state,
|
||||
@ -633,7 +888,7 @@ function applyOverrides(look, overrides) {
|
||||
overrides.sections.forEach((o, i) => {
|
||||
if (!look.sections[i]) return;
|
||||
if (o.locked !== undefined) look.sections[i].locked = o.locked;
|
||||
if (o.params) Object.assign(look.sections[i].layers[0].params, o.params);
|
||||
if (o.params) Object.assign(subjectOf(look.sections[i].layers).params, o.params);
|
||||
});
|
||||
}
|
||||
return look;
|
||||
@ -641,7 +896,7 @@ function applyOverrides(look, overrides) {
|
||||
|
||||
/** Compact description, used by the HUD and by check output. */
|
||||
export function describeLook(look) {
|
||||
const kinds = look.sections.map((s) => `${s.kind}:${s.layers[0].module.name}`);
|
||||
const kinds = look.sections.map((s) => `${s.kind}:${subjectOf(s.layers).module.name}`);
|
||||
return `seed ${look.seed.toString(16)} · ${look.director} · ${look.paletteScheme} · ` +
|
||||
`${describePersonality(look.personality)} · ${describeGrain(look.grain)} · ` +
|
||||
`${describePaletteArc(look.paletteArc)} · ${describeFraming(look.framing)} · ` +
|
||||
|
||||
@ -35,7 +35,34 @@
|
||||
* when it is being quiet.
|
||||
*/
|
||||
export const RESTFUL_FAMILIES = ['minimal', 'flow', 'organic'];
|
||||
const QUIET_KINDS = ['intro', 'breakdown', 'outro'];
|
||||
export const QUIET_KINDS = ['intro', 'breakdown', 'outro'];
|
||||
|
||||
/**
|
||||
* How full a director lets the frame get.
|
||||
*
|
||||
* Scales the coverage budget in LookGenerator — the painted-area ceiling a
|
||||
* section's stack is built against, where 1.0 is one filled frame and 2.0 is
|
||||
* the hard cap. A point of view about what a song looks like includes how much
|
||||
* is allowed to be happening at once, and it is the difference between a
|
||||
* brutalist drop (one big thing) and a corrupt one (everything, over
|
||||
* everything). Absent, a director is read as 1.2.
|
||||
*/
|
||||
const DEFAULT_CROWD = 1.2;
|
||||
|
||||
/**
|
||||
* How willing a director is to let the frame BLAZE.
|
||||
*
|
||||
* A blaze is the shot composited additively over its ground instead of keyed
|
||||
* onto it, so the two brightnesses sum and the highlights bloom out. It is a
|
||||
* real effect and worth having — a drop that goes white for eight bars reads as
|
||||
* the song peaking — and it was, until it was made a decision, simply what
|
||||
* every section did: screen over a filled ground blew a median quarter of every
|
||||
* frame to paper, all the time, in every video.
|
||||
*
|
||||
* So it is rationed. This is the appetite; the section still has to be loud and
|
||||
* late in the story to earn one. Absent, a director is read as 0.15.
|
||||
*/
|
||||
const DEFAULT_BLAZE = 0.15;
|
||||
|
||||
/**
|
||||
* Each director maps every section kind to three families, most-preferred
|
||||
@ -45,6 +72,11 @@ const QUIET_KINDS = ['intro', 'breakdown', 'outro'];
|
||||
export const DIRECTORS = [
|
||||
{
|
||||
name: 'ambient',
|
||||
// Space, not glare. A blaze here is the exception that proves it.
|
||||
blaze: 0.1,
|
||||
// Space is the subject, so the frame stays owed to it: one
|
||||
// picture, and a pass over it only when the song is at its loudest.
|
||||
crowd: 1.1,
|
||||
// Patient camera to match: long moves, mostly along one line. See
|
||||
// look/Camera.js — a director's point of view now includes how it
|
||||
// shoots, not only what it points at.
|
||||
@ -63,6 +95,11 @@ export const DIRECTORS = [
|
||||
},
|
||||
{
|
||||
name: 'brutalist',
|
||||
// Mass does not glow. Almost never.
|
||||
blaze: 0.05,
|
||||
// One large thing, seen whole. Layering hides mass, which is the
|
||||
// only thing this director is interested in.
|
||||
crowd: 1.05,
|
||||
// Holds, then commits to one large move. Architecture is looked AT.
|
||||
camera: 'deliberate',
|
||||
// Everything is architecture. Quiet means empty rather than soft, so it
|
||||
@ -79,6 +116,11 @@ export const DIRECTORS = [
|
||||
},
|
||||
{
|
||||
name: 'organicist',
|
||||
// Light through leaves — bloom belongs to this world.
|
||||
blaze: 0.2,
|
||||
// Growth accumulates. Things overlap here because that is what
|
||||
// living material does — nothing in this world is a single clean plate.
|
||||
crowd: 1.45,
|
||||
// Never settles, because nothing here is ever finished settling.
|
||||
camera: 'roaming',
|
||||
// Nothing is ever built; things grow and dissolve. Deliberately never
|
||||
@ -95,6 +137,11 @@ export const DIRECTORS = [
|
||||
},
|
||||
{
|
||||
name: 'corrupt',
|
||||
// Overload is the subject. Half its drops go white.
|
||||
blaze: 0.5,
|
||||
// Everything over everything. The damage is the subject and it is
|
||||
// never confined to one plate.
|
||||
crowd: 1.7,
|
||||
// Cuts with the camera already moving.
|
||||
camera: 'kinetic',
|
||||
// The signal is damaged and the damage is the subject — everywhere the
|
||||
@ -112,6 +159,11 @@ export const DIRECTORS = [
|
||||
},
|
||||
{
|
||||
name: 'geometer',
|
||||
// Exact, and occasionally exact and incandescent.
|
||||
blaze: 0.15,
|
||||
// Pattern on pattern is a moiré, which is a pattern. Layers are
|
||||
// welcome as long as they are exact.
|
||||
crowd: 1.3,
|
||||
// Small, exact, always arrives — the pattern is the subject and the
|
||||
// camera does not editorialise about it.
|
||||
camera: 'precise',
|
||||
@ -156,6 +208,16 @@ export function pickDirector(summary, rng) {
|
||||
return rng.pickWeighted(DIRECTORS, weights);
|
||||
}
|
||||
|
||||
/** How full this director lets the frame get. See DEFAULT_CROWD. */
|
||||
export function crowdOf(director) {
|
||||
return (director && director.crowd) || DEFAULT_CROWD;
|
||||
}
|
||||
|
||||
/** How willing this director is to let a section blaze. See DEFAULT_BLAZE. */
|
||||
export function blazeOf(director) {
|
||||
return (director && director.blaze !== undefined) ? director.blaze : DEFAULT_BLAZE;
|
||||
}
|
||||
|
||||
export function directorByName(name) {
|
||||
return DIRECTORS.find((d) => d.name === name) || DIRECTORS[0];
|
||||
}
|
||||
|
||||
55
flow-state/src/look/stack.js
Normal file
55
flow-state/src/look/stack.js
Normal file
@ -0,0 +1,55 @@
|
||||
// What a layer stack is made of, and how to ask it questions.
|
||||
//
|
||||
// A stack used to be "the scene, then whatever was composited over it", so
|
||||
// `layers[0]` meant the section's scene everywhere in the program. It no longer
|
||||
// does: every stack now starts with a GROUND — a filled canvas the shot happens
|
||||
// on — and the scene the section is ABOUT sits above it.
|
||||
//
|
||||
// Everything that used to reach for `layers[0]` wants the subject, not the
|
||||
// ground: the param panel edits it, the HUD names it, and the variety report
|
||||
// counts it. Those all go through `subjectOf` now. Reading the ground as the
|
||||
// section's scene would be actively wrong for the measurements — grounds come
|
||||
// from a twenty-scene pool, so a report that counted them would show a library
|
||||
// three times smaller than the one actually on screen.
|
||||
|
||||
import { coverageOf } from '../scenes/surface.js';
|
||||
|
||||
/** Layers carrying this role are the bed, not the shot. */
|
||||
export const GROUND = 'ground';
|
||||
|
||||
export function isGround(layer) {
|
||||
return !!layer && layer.role === GROUND;
|
||||
}
|
||||
|
||||
/** Index of the layer the section is about. */
|
||||
export function subjectIndexOf(stack) {
|
||||
const at = stack.findIndex((l) => !isGround(l));
|
||||
return at < 0 ? 0 : at;
|
||||
}
|
||||
|
||||
/** The layer the section is about — the shot, as opposed to what it stands on. */
|
||||
export function subjectOf(stack) {
|
||||
return stack[subjectIndexOf(stack)];
|
||||
}
|
||||
|
||||
/** The ground under a stack, or null if the subject is its own ground. */
|
||||
export function groundOf(stack) {
|
||||
return stack.find(isGround) || null;
|
||||
}
|
||||
|
||||
/** Everything above the subject: the passes composited over the shot. */
|
||||
export function overlaysOf(stack) {
|
||||
return stack.slice(subjectIndexOf(stack) + 1);
|
||||
}
|
||||
|
||||
/**
|
||||
* How much painted frame a stack adds up to, in units of one filled frame.
|
||||
*
|
||||
* Deliberately a SUM and not a union: two layers each painting 60% do not add
|
||||
* up to 120% of a screen, but they do add up to two things happening at once,
|
||||
* and that is the quantity the budget is about. 2.0 is the ceiling — see
|
||||
* MAX_COVERAGE in LookGenerator.
|
||||
*/
|
||||
export function stackCoverage(stack) {
|
||||
return stack.reduce((sum, l) => sum + coverageOf(l.module), 0);
|
||||
}
|
||||
@ -7,6 +7,8 @@ import { toHex } from './look/palette.js';
|
||||
import { applyGrainToPost, describeGrain, GRAIN_MASKS, GRAIN_MODES } from './look/grain.js';
|
||||
import { renderClickTrack, audioBufferToWavBlob } from './audio/metronome.js';
|
||||
import { Exporter, exportSegment, downloadBlob, PRESETS, isSupported } from './export/Exporter.js';
|
||||
import { subjectOf, groundOf, overlaysOf, stackCoverage } from './look/stack.js';
|
||||
import { coverageOf as sceneCoverage } from './scenes/surface.js';
|
||||
|
||||
const QUALITY = {
|
||||
draft: 0.5, // half resolution — for scrubbing heavy stacks
|
||||
@ -56,7 +58,7 @@ const paramPanel = new ParamPanel(document.createElement('div'), onParamChange);
|
||||
|
||||
// ---------------------------------------------------------------- loading
|
||||
|
||||
async function loadFile(file) {
|
||||
async function loadFile(file, { seed = null } = {}) {
|
||||
if (state.busy) return;
|
||||
state.busy = true;
|
||||
stopPlayback();
|
||||
@ -76,7 +78,7 @@ async function loadFile(file) {
|
||||
await state.show.load(file, (stage, fraction) => {
|
||||
dom.thStep.textContent = stage;
|
||||
dom.thFill.style.width = `${Math.round((fraction || 0) * 100)}%`;
|
||||
});
|
||||
}, { seed });
|
||||
|
||||
dom.audio.src = URL.createObjectURL(file);
|
||||
dom.thLabel.textContent = 'loaded track';
|
||||
@ -113,6 +115,35 @@ if (dom.changeTrack) dom.changeTrack.addEventListener('click', () => dom.fileInp
|
||||
dom.fileInput.addEventListener('change', (e) => {
|
||||
if (e.target.files[0]) loadFile(e.target.files[0]);
|
||||
});
|
||||
/**
|
||||
* `?song=centre` — open a bank song straight from a debug page.
|
||||
*
|
||||
* The filmstrip reduces a song to nine stills; the only way to argue with that
|
||||
* reading is to watch the thing move, and until now that meant remembering
|
||||
* which of seventeen near-identically-named wavs to drag in. `&seed=` carries
|
||||
* the strip's seed across so the video you watch is the one it measured.
|
||||
*
|
||||
* Dev-only in practice: `test/songs/` is served by vite from the project root
|
||||
* and is not part of a build. The name is matched against a bare word rather
|
||||
* than the bank list so the app does not have to import the synth.
|
||||
*/
|
||||
async function loadBankSong(name, seed) {
|
||||
if (!/^[a-z0-9_-]+$/i.test(name)) return;
|
||||
const url = `/test/songs/${name}.wav`;
|
||||
try {
|
||||
const res = await fetch(url);
|
||||
if (!res.ok) throw new Error(`${res.status} ${res.statusText}`);
|
||||
const blob = await res.blob();
|
||||
await loadFile(new File([blob], `${name}.wav`, { type: 'audio/wav' }), { seed });
|
||||
} catch (err) {
|
||||
dom.thLabel.textContent = 'no such song';
|
||||
dom.thName.hidden = false;
|
||||
dom.thName.textContent = name;
|
||||
dom.thStep.textContent = `${url}: ${err.message} — run npm run build:songs`;
|
||||
console.error(err);
|
||||
}
|
||||
}
|
||||
|
||||
document.addEventListener('dragover', (e) => e.preventDefault());
|
||||
document.addEventListener('drop', (e) => {
|
||||
e.preventDefault();
|
||||
@ -265,7 +296,8 @@ function renderPanel() {
|
||||
|
||||
if (state.tab === 'scene') {
|
||||
paramPanel.container = dom.panelBody;
|
||||
paramPanel.build(section.layers[0].module, section.layers[0].params);
|
||||
const subject = subjectOf(section.layers);
|
||||
paramPanel.build(subject.module, subject.params);
|
||||
|
||||
// The section's stage visuals, with the one currently on screen marked.
|
||||
// Params above edit the anchor (variant 0) — the image the section opens
|
||||
@ -278,20 +310,29 @@ function renderPanel() {
|
||||
list.innerHTML = `<div class="pp-sub">stage visuals · ${shots.length} shots</div>` +
|
||||
section.variants.map((stack, v) =>
|
||||
`<div class="pp-react-row${v === active ? ' current' : ''}">` +
|
||||
`<span>${v === 0 ? '●' : '○'} ${stack[0].module.name}</span>` +
|
||||
`<span>${v === 0 ? '●' : '○'} ${subjectOf(stack).module.name}</span>` +
|
||||
`<span class="pp-feature">${shots.filter((s) => s.variant === v).length}×</span>` +
|
||||
`</div>`).join('');
|
||||
dom.panelBody.appendChild(list);
|
||||
}
|
||||
|
||||
if (section.layers.length > 1) {
|
||||
// The rest of the stack, named by the job each layer is doing rather
|
||||
// than by its index: what is under the shot and what is over it are
|
||||
// different questions, and the panel used to call both "layered over".
|
||||
const ground = groundOf(section.layers);
|
||||
const overlays = overlaysOf(section.layers);
|
||||
if (ground || overlays.length) {
|
||||
const row = (l, label) =>
|
||||
`<div class="pp-react-row"><span>${l.module.name}</span>` +
|
||||
`<span class="pp-feature">${label}</span>` +
|
||||
`<span class="pp-amount">${(sceneCoverage(l.module) * 100).toFixed(0)}%</span></div>`;
|
||||
const note = document.createElement('div');
|
||||
note.className = 'pp-reactive';
|
||||
note.innerHTML = `<div class="pp-sub">layered over (${section.layers.length - 1})</div>` +
|
||||
section.layers.slice(1).map((l) =>
|
||||
`<div class="pp-react-row"><span>${l.module.name}</span>` +
|
||||
`<span class="pp-feature">${l.blend}</span>` +
|
||||
`<span class="pp-amount">${l.opacity.toFixed(2)}</span></div>`).join('');
|
||||
note.innerHTML =
|
||||
`<div class="pp-sub">stack · ${(stackCoverage(section.layers) * 100).toFixed(0)}% painted</div>` +
|
||||
(ground ? row(ground, 'ground') : '') +
|
||||
row(subject, 'shot') +
|
||||
overlays.map((l) => row(l, l.blend)).join('');
|
||||
dom.panelBody.appendChild(note);
|
||||
}
|
||||
return;
|
||||
@ -326,7 +367,7 @@ function renderPanel() {
|
||||
<div class="kv ${i === index ? 'current' : ''}">
|
||||
<span>${s.kind}${s.locked ? ' 🔒' : ''}
|
||||
${s.shots ? `<i class="dim">${s.shots.length} shots</i>` : ''}</span>
|
||||
<b>${(s.variants || [s.layers]).map((v) => v[0].module.name).join(' / ')}</b>
|
||||
<b>${(s.variants || [s.layers]).map((v) => subjectOf(v).module.name).join(' / ')}</b>
|
||||
</div>`).join('')}
|
||||
<button id="btn-metronome" class="wide">download click track</button>
|
||||
<div class="hint">Mixes clicks onto the detected beat grid. If they don't sit on
|
||||
@ -664,3 +705,9 @@ resize();
|
||||
// "never going to start", and the failure it exists to catch is exactly the one
|
||||
// that produces no error at all in the page.
|
||||
window.__FLOW_STATE_READY__ = true;
|
||||
|
||||
const query = new URLSearchParams(location.search);
|
||||
if (query.get('song')) {
|
||||
const seed = query.get('seed');
|
||||
loadBankSong(query.get('song'), seed === null ? null : (Number(seed) >>> 0));
|
||||
}
|
||||
|
||||
@ -68,34 +68,26 @@ export const REACTIVE_RESPONSES = ['linear', 'spike', 'smooth', 'inverse'];
|
||||
export const ARTIFACT_NAMES = ['cast', 'ink', 'staging', 'form'];
|
||||
|
||||
/**
|
||||
* What a scene is for, compositionally.
|
||||
* Whether a scene's image depends on the FRAME BEFORE IT.
|
||||
*
|
||||
* canvas fills the frame. Belongs underneath, and two of them stacked is
|
||||
* two pictures fighting rather than one picture with depth.
|
||||
* composable mostly empty by design. Reads as elements ON something, and
|
||||
* shown alone it is a few bright things on black — which scores
|
||||
* well for variety and is thin to watch.
|
||||
* Read off the shader rather than declared, because the declaration would be a
|
||||
* second copy of something the source already says exactly: a scene reads
|
||||
* history if and only if it calls `prev()`.
|
||||
*
|
||||
* The distinction was implicit in `role: 'accent'` and that was not enough: it
|
||||
* marked the depth passes and said nothing about the sixty scenes that are all
|
||||
* treated as interchangeable backgrounds despite half of them being sparse.
|
||||
* What it decides is where a scene may sit in a stack. `prev()` returns the
|
||||
* whole composited previous frame — everything, including whatever was layered
|
||||
* ON TOP of this scene — so a datamosh or a time smear underneath a shot is not
|
||||
* grounding it, it is recycling it. Two things follow, and both were measured
|
||||
* the moment such a scene became a bed: the render stops being reproducible from
|
||||
* a seek (state carries across frames that a seek has not rendered), and small
|
||||
* numeric differences compound frame over frame instead of staying put — 91/255
|
||||
* between two WebGL contexts, against a ceiling of 4.
|
||||
*
|
||||
* It also has to be DECLARED to mean anything. For a while the field existed,
|
||||
* the generator gated layering on it, and no scene carried it — so nothing could
|
||||
* ever be layered and the one scene with `role: 'accent'` was the only thing
|
||||
* that ever appeared on top of anything. The labels below come from the measured
|
||||
* coverage the phase 12 gate prints; anything under 30% is composable.
|
||||
*
|
||||
* Verified rather than trusted — see the coverage gate in checks/phase12. A
|
||||
* scene that claims to be composable and paints the whole frame will cover
|
||||
* whatever it is layered over.
|
||||
* These scenes are exactly right as a shot or as a pass over one. They are only
|
||||
* wrong as the thing underneath. See canGround in scenes/surface.js.
|
||||
*/
|
||||
export const SURFACES = ['canvas', 'composable'];
|
||||
|
||||
/** What a scene is, defaulting to the behaviour it had before this existed. */
|
||||
export function surfaceOf(module) {
|
||||
if (module.surface) return module.surface;
|
||||
return 'canvas';
|
||||
export function readsHistory(module) {
|
||||
return typeof module.shader === 'string' && /\bprev\s*\(/.test(module.shader);
|
||||
}
|
||||
|
||||
/**
|
||||
@ -291,8 +283,11 @@ export function validateModule(module) {
|
||||
if (!TRAIT_NAMES.includes(t)) errors.push(`${id}: unknown trait '${t}'`);
|
||||
}
|
||||
}
|
||||
if (module.surface !== undefined && !SURFACES.includes(module.surface)) {
|
||||
errors.push(`${id}: unknown surface '${module.surface}' — one of ${SURFACES.join('/')}`);
|
||||
// `surface` was a declaration and is now measured — see scenes/surface.js.
|
||||
// Rejected rather than ignored, so a scene file carrying a stale one is a
|
||||
// loud error instead of a line that quietly means nothing.
|
||||
if (module.surface !== undefined) {
|
||||
errors.push(`${id}: 'surface' is derived from scenes/metadata.json — remove the declaration`);
|
||||
}
|
||||
if (module.consumes !== undefined) {
|
||||
if (!Array.isArray(module.consumes)) {
|
||||
|
||||
@ -19,7 +19,6 @@ export const particleField = {
|
||||
// top — every layered stack in every song was these particles — while the
|
||||
// general overlay path sat dead because nothing else was labelled. It is one
|
||||
// composable scene among many now.
|
||||
surface: 'composable',
|
||||
// Points in empty space with no ground of their own — the one scene in the
|
||||
// library that genuinely cannot carry a section alone. See schema.canBackground.
|
||||
background: false,
|
||||
|
||||
8190
flow-state/src/scenes/metadata.json
Normal file
8190
flow-state/src/scenes/metadata.json
Normal file
File diff suppressed because it is too large
Load Diff
@ -20,8 +20,6 @@ export const aqueductMarch = {
|
||||
name: 'Aqueduct March',
|
||||
family: 'structural',
|
||||
kind: 'fragment',
|
||||
// Paints 27% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
texture: 0.9,
|
||||
consumes: ['cast', 'ink'],
|
||||
traits: ['shape', 'camera', 'space', 'style'],
|
||||
|
||||
@ -13,8 +13,6 @@ export const auroraVeil = {
|
||||
name: 'Aurora Veil',
|
||||
family: 'flow',
|
||||
kind: 'fragment',
|
||||
// Paints 13% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
consumes: ['ink'],
|
||||
traits: ['camera', 'space', 'style'],
|
||||
|
||||
|
||||
@ -13,8 +13,6 @@ export const cargoBelt = {
|
||||
name: 'Cargo Belt',
|
||||
family: 'structural',
|
||||
kind: 'fragment',
|
||||
// Paints 12% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Takes the track's surface grain, but lightly — this is drawn, not filmed.
|
||||
texture: 0.4,
|
||||
consumes: ['cast', 'ink'],
|
||||
|
||||
@ -13,8 +13,6 @@ export const circuitBloom = {
|
||||
name: 'Circuit Bloom',
|
||||
family: 'geometric',
|
||||
kind: 'fragment',
|
||||
// Paints 1% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Takes the track's surface grain, but lightly — this is drawn, not filmed.
|
||||
texture: 0.4,
|
||||
consumes: ['cast', 'ink'],
|
||||
|
||||
@ -8,8 +8,6 @@ export const curlFlow = {
|
||||
name: 'Curl Flow',
|
||||
family: 'flow',
|
||||
kind: 'fragment',
|
||||
// Paints 21% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Personality: see look/Personality.js.
|
||||
consumes: ['ink'],
|
||||
traits: ['camera', 'space', 'style'],
|
||||
|
||||
@ -11,8 +11,6 @@ export const dataAisle = {
|
||||
name: 'Data Aisle',
|
||||
family: 'structural',
|
||||
kind: 'fragment',
|
||||
// Paints 21% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
texture: 0.6,
|
||||
consumes: ['ink'],
|
||||
traits: ['camera', 'space', 'style'],
|
||||
|
||||
@ -21,8 +21,6 @@ export const drosteFeedback = {
|
||||
name: 'Droste Feedback',
|
||||
family: 'glitch',
|
||||
kind: 'fragment',
|
||||
// Paints 2% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
texture: 0.5,
|
||||
consumes: ['ink'],
|
||||
traits: ['camera', 'style'],
|
||||
|
||||
@ -13,8 +13,6 @@ export const dustChamber = {
|
||||
name: 'Dust Chamber',
|
||||
family: 'minimal',
|
||||
kind: 'fragment',
|
||||
// Paints 2% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
consumes: ['cast', 'ink'],
|
||||
traits: ['shape', 'camera', 'space', 'style'],
|
||||
|
||||
|
||||
@ -16,8 +16,6 @@ export const eclipseField = {
|
||||
name: 'Eclipse Field',
|
||||
family: 'minimal',
|
||||
kind: 'fragment',
|
||||
// Paints 17% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Crisp line work: the track's surface grain would only fur the edges.
|
||||
texture: 0,
|
||||
consumes: ['cast', 'ink'],
|
||||
|
||||
@ -9,8 +9,6 @@ export const fireflyDrift = {
|
||||
name: 'Firefly Drift',
|
||||
family: 'flow',
|
||||
kind: 'fragment',
|
||||
// Paints 2% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Personality: see look/Personality.js.
|
||||
consumes: ['cast', 'ink', 'staging'],
|
||||
traits: ['camera', 'style'],
|
||||
|
||||
@ -11,8 +11,6 @@ export const flora = {
|
||||
name: 'Flora',
|
||||
family: 'organic',
|
||||
kind: 'fragment',
|
||||
// Paints 15% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Personality: see look/Personality.js.
|
||||
consumes: ['cast', 'ink'],
|
||||
traits: ['shape', 'camera', 'style'],
|
||||
|
||||
@ -14,8 +14,6 @@ export const gateCorridor = {
|
||||
name: 'Gate Corridor',
|
||||
family: 'structural',
|
||||
kind: 'fragment',
|
||||
// Paints 29% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Takes the track's surface grain, but lightly — this is drawn, not filmed.
|
||||
texture: 0.4,
|
||||
consumes: ['cast', 'ink'],
|
||||
|
||||
@ -6,8 +6,6 @@ export const horizonLines = {
|
||||
name: 'Horizon Lines',
|
||||
family: 'minimal',
|
||||
kind: 'fragment',
|
||||
// Paints 16% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Personality: see look/Personality.js.
|
||||
// Crisp line work: the track's surface grain would only fur the edges.
|
||||
texture: 0,
|
||||
|
||||
@ -13,8 +13,6 @@ export const inkBleed = {
|
||||
name: 'Ink Bleed',
|
||||
family: 'organic',
|
||||
kind: 'fragment',
|
||||
// Paints 1% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
consumes: ['ink'],
|
||||
traits: ['camera', 'space', 'style'],
|
||||
|
||||
|
||||
@ -12,8 +12,6 @@ export const magnetLines = {
|
||||
name: 'Magnet Lines',
|
||||
family: 'flow',
|
||||
kind: 'fragment',
|
||||
// Paints 18% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Line work. Grain furs it up, so take only a dusting.
|
||||
texture: 0.5,
|
||||
consumes: ['cast', 'ink'],
|
||||
|
||||
@ -7,8 +7,6 @@ export const metaballs = {
|
||||
name: 'Metaballs',
|
||||
family: 'organic',
|
||||
kind: 'fragment',
|
||||
// Paints 26% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Personality: see look/Personality.js.
|
||||
consumes: ['cast', 'ink', 'staging'],
|
||||
traits: ['shape', 'camera', 'style'],
|
||||
|
||||
@ -12,8 +12,6 @@ export const myceliumWeb = {
|
||||
name: 'Mycelium Web',
|
||||
family: 'organic',
|
||||
kind: 'fragment',
|
||||
// Paints 22% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
consumes: ['ink'],
|
||||
traits: ['camera', 'space', 'style'],
|
||||
|
||||
|
||||
@ -11,8 +11,6 @@ export const neonCity = {
|
||||
name: 'Neon City',
|
||||
family: 'structural',
|
||||
kind: 'fragment',
|
||||
// Paints 26% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Personality: see look/Personality.js.
|
||||
consumes: ['ink'],
|
||||
traits: ['space', 'camera', 'style'],
|
||||
|
||||
@ -13,8 +13,6 @@ export const pendulumTrace = {
|
||||
name: 'Pendulum Trace',
|
||||
family: 'minimal',
|
||||
kind: 'fragment',
|
||||
// Paints 0% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Fine line work; grain only furs it up.
|
||||
texture: 0.25,
|
||||
consumes: ['ink'],
|
||||
|
||||
@ -7,8 +7,6 @@ export const prismBloom = {
|
||||
name: 'Prism Bloom',
|
||||
family: 'geometric',
|
||||
kind: 'fragment',
|
||||
// Paints 1% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Personality: see look/Personality.js.
|
||||
// Crisp line work: the track's surface grain would only fur the edges.
|
||||
texture: 0,
|
||||
|
||||
@ -9,8 +9,6 @@ export const psychedelicDrift = {
|
||||
name: 'Psychedelic Drift',
|
||||
family: 'glitch',
|
||||
kind: 'fragment',
|
||||
// Paints 6% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Personality: see look/Personality.js.
|
||||
consumes: ['cast', 'ink'],
|
||||
traits: ['shape', 'camera', 'style'],
|
||||
|
||||
@ -48,8 +48,6 @@ export const pylonGrid = {
|
||||
name: 'Pylon Grid',
|
||||
family: 'structural',
|
||||
kind: 'fragment',
|
||||
// Paints 25% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Personality: see look/Personality.js.
|
||||
// Takes the track's surface grain, but lightly — this is drawn, not filmed.
|
||||
texture: 0.4,
|
||||
|
||||
@ -7,8 +7,6 @@ export const ridgeTerrain = {
|
||||
name: 'Ridge Terrain',
|
||||
family: 'structural',
|
||||
kind: 'fragment',
|
||||
// Paints 13% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Personality: see look/Personality.js.
|
||||
consumes: ['ink'],
|
||||
traits: ['space', 'camera', 'style'],
|
||||
|
||||
@ -12,8 +12,6 @@ export const shojiGrid = {
|
||||
name: 'Shoji Grid',
|
||||
family: 'minimal',
|
||||
kind: 'fragment',
|
||||
// Paints 23% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Paper has a tooth, and this is the one scene that wants it.
|
||||
texture: 1.0,
|
||||
consumes: ['cast', 'ink'],
|
||||
|
||||
@ -10,8 +10,6 @@ export const signalDecay = {
|
||||
name: 'Signal Decay',
|
||||
family: 'glitch',
|
||||
kind: 'fragment',
|
||||
// Paints 5% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
consumes: ['ink'],
|
||||
traits: ['camera', 'style'],
|
||||
|
||||
|
||||
@ -8,8 +8,6 @@ export const silkRibbon = {
|
||||
name: 'Silk Ribbon',
|
||||
family: 'minimal',
|
||||
kind: 'fragment',
|
||||
// Paints 3% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Personality: see look/Personality.js.
|
||||
// A soft drape of light. Grain furs the one thing it is made of.
|
||||
texture: 0,
|
||||
|
||||
@ -9,8 +9,6 @@ export const slowOrb = {
|
||||
name: 'Slow Orb',
|
||||
family: 'minimal',
|
||||
kind: 'fragment',
|
||||
// Paints 11% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Personality: see look/Personality.js.
|
||||
// One soft body in an empty frame — the emptiest scene in the library, and
|
||||
// the one where speckle is most visible for being least justified. It keeps
|
||||
|
||||
@ -12,8 +12,6 @@ export const smokeColumn = {
|
||||
name: 'Smoke Column',
|
||||
family: 'flow',
|
||||
kind: 'fragment',
|
||||
// Paints 16% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
consumes: ['ink'],
|
||||
traits: ['camera', 'space', 'style'],
|
||||
|
||||
|
||||
@ -9,8 +9,6 @@ export const spectrumSculpture = {
|
||||
name: 'Spectrum Sculpture',
|
||||
family: 'minimal',
|
||||
kind: 'fragment',
|
||||
// Paints 2% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Personality: see look/Personality.js.
|
||||
// Takes the track's surface grain, but lightly — this is drawn, not filmed.
|
||||
texture: 0.4,
|
||||
|
||||
@ -13,8 +13,6 @@ export const stormRift = {
|
||||
name: 'Storm Rift',
|
||||
family: 'glitch',
|
||||
kind: 'fragment',
|
||||
// Paints 0% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
consumes: ['ink'],
|
||||
traits: ['camera', 'space', 'style'],
|
||||
|
||||
|
||||
@ -12,8 +12,6 @@ export const suspensionSpan = {
|
||||
name: 'Suspension Span',
|
||||
family: 'structural',
|
||||
kind: 'fragment',
|
||||
// Paints 21% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
texture: 0.6,
|
||||
consumes: ['cast', 'ink'],
|
||||
traits: ['shape', 'camera', 'space', 'style'],
|
||||
|
||||
@ -10,8 +10,6 @@ export const synthwaveRun = {
|
||||
name: 'Synthwave Run',
|
||||
family: 'structural',
|
||||
kind: 'fragment',
|
||||
// Paints 5% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
// Personality: see look/Personality.js.
|
||||
consumes: ['ink'],
|
||||
traits: ['space', 'camera', 'style'],
|
||||
|
||||
@ -13,8 +13,6 @@ export const vortexDrift = {
|
||||
name: 'Vortex Drift',
|
||||
family: 'flow',
|
||||
kind: 'fragment',
|
||||
// Paints 21% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
consumes: ['cast', 'ink'],
|
||||
traits: ['shape', 'camera', 'style'],
|
||||
|
||||
|
||||
@ -8,8 +8,6 @@ export const constellation = {
|
||||
name: 'Constellation',
|
||||
family: 'minimal',
|
||||
kind: 'fragment',
|
||||
// Paints 1% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
consumes: ['cast', 'ink', 'staging'],
|
||||
texture: 0.6,
|
||||
traits: ['shape', 'camera', 'space', 'style'],
|
||||
|
||||
@ -15,7 +15,6 @@ export const effigy = {
|
||||
family: 'geometric',
|
||||
// One object against a dark wash — most of the frame is legitimately empty,
|
||||
// and it reads as something standing in a space rather than as the space.
|
||||
surface: 'composable',
|
||||
kind: 'fragment',
|
||||
consumes: ['form', 'ink', 'staging'],
|
||||
texture: 0.6,
|
||||
|
||||
@ -9,8 +9,6 @@ export const procession = {
|
||||
name: 'Procession',
|
||||
family: 'structural',
|
||||
kind: 'fragment',
|
||||
// Paints 18% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
consumes: ['cast', 'ink', 'staging'],
|
||||
texture: 0.4,
|
||||
traits: ['shape', 'camera', 'style'],
|
||||
|
||||
@ -8,8 +8,6 @@ export const soloist = {
|
||||
name: 'Soloist',
|
||||
family: 'minimal',
|
||||
kind: 'fragment',
|
||||
// Paints 20% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
consumes: ['cast', 'ink', 'staging'],
|
||||
texture: 0.5,
|
||||
traits: ['shape', 'camera', 'style'],
|
||||
|
||||
@ -16,8 +16,6 @@ export const swarm = {
|
||||
name: 'Swarm',
|
||||
family: 'organic',
|
||||
kind: 'fragment',
|
||||
// Paints 2% of the frame — see checks/phase12 coverage.
|
||||
surface: 'composable',
|
||||
consumes: ['form', 'ink', 'staging'],
|
||||
texture: 0.5,
|
||||
traits: ['shape', 'camera', 'style'],
|
||||
|
||||
241
flow-state/src/scenes/surface.js
Normal file
241
flow-state/src/scenes/surface.js
Normal file
@ -0,0 +1,241 @@
|
||||
// What the measurements MEAN — the thresholds and the rules, kept apart from
|
||||
// the numbers they read.
|
||||
//
|
||||
// scenes/metadata.json is generated and holds no opinions: it says a scene
|
||||
// paints 32% of the frame across six songs and nothing about whether that is
|
||||
// enough. This file is where that becomes a decision, and it is hand-written
|
||||
// precisely because a threshold is a judgement about the videos rather than a
|
||||
// fact about the library.
|
||||
//
|
||||
// `surface` used to be declared in each scene file, and it drifted: nine scenes
|
||||
// said `canvas` while painting under a third of the frame, which is how a
|
||||
// section ended up standing on a few bright things and black. It is derived
|
||||
// now. A scene cannot lie about what it paints, because nobody writes it down.
|
||||
|
||||
import metadata from './metadata.json';
|
||||
import { readsHistory } from '../params/schema.js';
|
||||
|
||||
/**
|
||||
* The line between a picture and elements on one.
|
||||
*
|
||||
* Half the frame. Under it a scene is not somewhere a shot can happen — it is
|
||||
* the shot, or something in it, and what would be behind it is black.
|
||||
*
|
||||
* The same number does both jobs on purpose: `canvas` and "may be a ground" are
|
||||
* one idea, and having a scene that is a canvas but not ground-capable would be
|
||||
* a distinction with nothing behind it.
|
||||
*/
|
||||
export const GROUND_MIN = 0.5;
|
||||
|
||||
/**
|
||||
* The bias a GROUND is sampled with, wherever it appears.
|
||||
*
|
||||
* A bed is not sampled like a shot. It keeps a density floor whatever the
|
||||
* section wants — an intro biased sparse would otherwise draw the ground at
|
||||
* density zero and be thin again for a new reason — and it sits back through
|
||||
* energy and motion instead.
|
||||
*
|
||||
* Defined here rather than in the generator because the measurement has to use
|
||||
* it too. Coverage depends enormously on parameters: Metaballs paints 69% of
|
||||
* the frame sampled as a drop's shot and 27% sampled as an intro's bed, and a
|
||||
* label taken at the first bias tells you nothing about the second. So the
|
||||
* number that decides whether a scene may be a ground is measured AS a ground.
|
||||
* One definition, both places, or the two silently disagree.
|
||||
*/
|
||||
export const GROUND_BIAS = { energy: 0.5, density: 0.45, motion: 0.3 };
|
||||
|
||||
/**
|
||||
* The hand a GROUND is drawn with.
|
||||
*
|
||||
* `extremity` is how far toward the ends of its own ranges a track pushes every
|
||||
* parameter, and it is the right dial for a shot: the ends are where a scene is
|
||||
* most itself. It is the wrong dial for a bed. Half of what decides how much a
|
||||
* scene paints is a parameter with no `bias` key at all — Metaballs' threshold,
|
||||
* for one — so extremity is the only thing moving it, and one end of that range
|
||||
* is an empty frame. Measured: a ground drawn at the track's own temperament
|
||||
* landed on threshold 1.85 of 2.0 and painted 6% of the frame, while the same
|
||||
* scene measures 58% drawn moderately.
|
||||
*
|
||||
* So a bed is drawn near the middle and the extremes are left to the shot,
|
||||
* which is also the right instinct visually.
|
||||
*/
|
||||
export function groundTemperamentFrom(temperament) {
|
||||
if (!temperament) return temperament;
|
||||
return { ...temperament, extremity: temperament.extremity * 0.2, intensity: 0 };
|
||||
}
|
||||
|
||||
/**
|
||||
* The song's hand, as it applies to a GROUND.
|
||||
*
|
||||
* The ink treatment — hatch, stipple, halftone, hollow — is how the song draws.
|
||||
* `hollow` draws outlines and no fill at all, which is a strong, legible
|
||||
* identity for the thing the section is ABOUT and a disaster for the thing it
|
||||
* stands on: a scene that paints 61% of the frame averaged over six identities
|
||||
* paints 2% in the song that asked for outlines, and the section renders black
|
||||
* with a ground under it. Measured, holding grounds to their worst identity
|
||||
* instead left four castable beds in the whole library.
|
||||
*
|
||||
* So the world is drawn solid and the hand is kept for the subject. Everything
|
||||
* else about the identity — the cast, the lattice, the line weight, the
|
||||
* palette — reaches the ground untouched, so it is still unmistakably this
|
||||
* song's world.
|
||||
*/
|
||||
export function groundPersonalityFrom(personality) {
|
||||
if (!personality || !personality.identity || !personality.identity.ink) return personality;
|
||||
const { identity } = personality;
|
||||
if (identity.ink.fill !== 'hollow') return personality;
|
||||
return {
|
||||
...personality,
|
||||
identity: { ...identity, ink: { ...identity.ink, fill: 'flat' } },
|
||||
};
|
||||
}
|
||||
|
||||
/** The bias `ground` would be sampled with inside `section`. */
|
||||
export function groundBiasFrom(bias) {
|
||||
return {
|
||||
...bias,
|
||||
// FLOORS, not reductions, on both of the axes that fill a frame.
|
||||
//
|
||||
// The first version of this lowered energy to make the bed sit back,
|
||||
// which turned out to be precisely how to empty it: scenes bias the
|
||||
// parameters that decide how much they paint against `energy` as often
|
||||
// as against `density` — Metaballs sizes its blobs on energy, so an
|
||||
// intro's ground drew radius 0.14 out of a 0.1–0.6 range and painted 4%
|
||||
// of the frame. A bed sits back through MOTION, which costs it nothing
|
||||
// in coverage, and through being behind everything else.
|
||||
density: Math.max(GROUND_BIAS.density, bias.density),
|
||||
energy: Math.max(GROUND_BIAS.energy, bias.energy),
|
||||
motion: Math.max(0, (bias.motion ?? 0.5) - 0.2),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* What a scene is, compositionally.
|
||||
*
|
||||
* canvas fills the frame. Belongs underneath, and two of them stacked is
|
||||
* two pictures fighting rather than one picture with depth.
|
||||
* composable mostly empty by design. Reads as elements ON something, and
|
||||
* shown alone it is a few bright things on black — which scores
|
||||
* well for variety and is thin to watch.
|
||||
*/
|
||||
export const SURFACES = ['canvas', 'composable'];
|
||||
|
||||
const ROW = (module) => (metadata.scenes && metadata.scenes[module.name]) || null;
|
||||
|
||||
/**
|
||||
* Coverage for a scene nobody has measured.
|
||||
*
|
||||
* Two ways to land here: a `layer3d`, which the measuring pass does not render,
|
||||
* and a scene added since the last refresh. Deliberately pessimistic — an
|
||||
* unmeasured scene must not be able to talk its way into the ground slot, which
|
||||
* `canGround` enforces by requiring a real row rather than by trusting this.
|
||||
*/
|
||||
const ASSUMED_COVERAGE = 0.15;
|
||||
|
||||
export function coverageOf(module) {
|
||||
const row = ROW(module);
|
||||
return row ? row.coverage : ASSUMED_COVERAGE;
|
||||
}
|
||||
|
||||
/**
|
||||
* How much this scene paints when it is used as a BED — measured at
|
||||
* GROUND_BIAS, which is the only figure that answers the question `canGround`
|
||||
* asks. See GROUND_BIAS for why the two numbers differ so much.
|
||||
*/
|
||||
export function groundCoverageOf(module) {
|
||||
const row = ROW(module);
|
||||
if (!row) return ASSUMED_COVERAGE;
|
||||
if (row.bedCoverageMean !== undefined) return row.bedCoverageMean;
|
||||
return row.bedCoverage === undefined ? row.coverage : row.bedCoverage;
|
||||
}
|
||||
|
||||
/**
|
||||
* What this scene paints as a bed in the IDENTITY THAT SUITS IT LEAST.
|
||||
*
|
||||
* A song's identity can gut a scene's fill — `hollow` ink draws outlines and no
|
||||
* fill, a dark palette drops it under the threshold — and the spread is wide:
|
||||
* Metaballs paints 61% averaged over six identities and 14% in the worst of
|
||||
* them, which is a black frame with a ground under it.
|
||||
*
|
||||
* Held as a SEPARATE, lower bar rather than as the eligibility number. Demanding
|
||||
* the worst case clear 50% leaves four castable grounds in the whole library —
|
||||
* every video in the world standing on one of four beds is a worse failure than
|
||||
* an occasional dim intro. So: fills the frame on average, and never vanishes.
|
||||
*/
|
||||
export function groundFloorCoverageOf(module) {
|
||||
const row = ROW(module);
|
||||
if (!row) return ASSUMED_COVERAGE;
|
||||
return row.bedCoverage === undefined ? groundCoverageOf(module) : row.bedCoverage;
|
||||
}
|
||||
|
||||
/** A ground may dim under a hostile identity. It may not disappear. */
|
||||
export const GROUND_FLOOR_MIN = 0.25;
|
||||
|
||||
export function isMeasured(module) {
|
||||
return ROW(module) !== null;
|
||||
}
|
||||
|
||||
/** Derived, never declared. See the header. */
|
||||
export function surfaceOf(module) {
|
||||
return coverageOf(module) >= GROUND_MIN ? 'canvas' : 'composable';
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a scene can be the ground under a section.
|
||||
*
|
||||
* Measured to fill the frame, and self-contained. The second half rules out
|
||||
* `prev()`: that returns the whole composited previous frame, INCLUDING the
|
||||
* layers above this one, so a datamosh under a shot is not grounding it, it is
|
||||
* eating it. Measured when one first became a bed — the render stopped
|
||||
* reproducing from a seek, and two WebGL contexts diverged by 91/255 against a
|
||||
* tolerance of 4.
|
||||
*/
|
||||
export function canGround(module) {
|
||||
return isMeasured(module)
|
||||
&& surfaceOf(module) === 'canvas'
|
||||
&& groundCoverageOf(module) >= GROUND_MIN
|
||||
&& groundFloorCoverageOf(module) >= GROUND_FLOOR_MIN
|
||||
&& !readsHistory(module);
|
||||
}
|
||||
|
||||
/**
|
||||
* How unalike two scenes are, structurally — 0 is the same picture.
|
||||
*
|
||||
* Both profiles are means over the same six songs, so this is a fair comparison
|
||||
* in a way that two arbitrary renders would not be. Colour is excluded: six
|
||||
* palettes would otherwise make any two scenes look unalike, and the whole
|
||||
* point is to find pairs that differ in STRUCTURE.
|
||||
*
|
||||
* This is the quantity that makes a stack interesting rather than merely full.
|
||||
* Two scenes at 0.02 layered over each other are one picture at double density;
|
||||
* the same two at 0.3 are a picture with something happening in it.
|
||||
*/
|
||||
export function structuralDistance(a, b) {
|
||||
const rowA = ROW(a), rowB = ROW(b);
|
||||
if (!rowA || !rowB || !rowA.profile || !rowB.profile) return null;
|
||||
|
||||
let total = 0, blocks = 0;
|
||||
for (const block of ['scale', 'orient', 'layout', 'region', 'texture']) {
|
||||
const x = rowA.profile[block], y = rowB.profile[block];
|
||||
if (!x || !y || x.length !== y.length) continue;
|
||||
// Chi-square, matching what variety/signature.js compares descriptors
|
||||
// with. Duplicated rather than imported because this runs inside the
|
||||
// generator and the checks must not be a dependency of it.
|
||||
let d = 0;
|
||||
for (let i = 0; i < x.length; i++) {
|
||||
const sum = x[i] + y[i];
|
||||
if (sum > 1e-9) d += ((x[i] - y[i]) ** 2) / sum;
|
||||
}
|
||||
total += Math.min(1, d / 2);
|
||||
blocks++;
|
||||
}
|
||||
return blocks ? total / blocks : null;
|
||||
}
|
||||
|
||||
/** The variety score from the gallery: how much a scene changes between songs. */
|
||||
export function varietyOf(module) {
|
||||
const row = ROW(module);
|
||||
return row ? row.variety : 0;
|
||||
}
|
||||
|
||||
export const METADATA = metadata;
|
||||
@ -1,3 +1,4 @@
|
||||
import { subjectOf } from '../look/stack.js';
|
||||
// The transport strip: sections coloured by kind, bar ticks, scene-change
|
||||
// markers, playhead.
|
||||
//
|
||||
@ -144,12 +145,12 @@ export class TimelineStrip {
|
||||
if (sw > 34) {
|
||||
const stack = look.variants ? look.variants[shot.variant] : look.layers;
|
||||
ctx.fillStyle = 'rgba(255,255,255,0.45)';
|
||||
ctx.fillText(stack[0].module.name, sx + 4, h - 6);
|
||||
ctx.fillText(subjectOf(stack).module.name, sx + 4, h - 6);
|
||||
}
|
||||
}
|
||||
} else if (look) {
|
||||
ctx.fillStyle = 'rgba(255,255,255,0.45)';
|
||||
ctx.fillText(look.layers[0].module.name, x0 + 5, h - 6);
|
||||
ctx.fillText(subjectOf(look.layers).module.name, x0 + 5, h - 6);
|
||||
}
|
||||
ctx.restore();
|
||||
|
||||
|
||||
@ -200,3 +200,9 @@ console.log('next:');
|
||||
console.log(' 1. write the scene() body — everything else is done');
|
||||
console.log(' 2. npm run lint:scenes');
|
||||
console.log(` 3. open http://localhost:5180/checks.html?scene=${encodeURIComponent(name.trim())}`);
|
||||
// A scene with no measured coverage cannot be a ground and fails the phase 12
|
||||
// table check, which is the point: how much frame it paints decides where it is
|
||||
// allowed to sit in a stack, and that is a render result rather than a claim.
|
||||
console.log(' 4. open http://localhost:5180/checks.html?phase=12&slow=1 and paste');
|
||||
console.log(' your scene\'s line from window.__COVERAGE_TABLE__ into');
|
||||
console.log(' src/scenes/coverage.js — declare `surface` to match it');
|
||||
|
||||
@ -1,7 +1,53 @@
|
||||
import { defineConfig } from 'vite';
|
||||
import { resolve } from 'path';
|
||||
import { writeFileSync } from 'fs';
|
||||
|
||||
/**
|
||||
* Let the measuring pages write their results back into the source tree.
|
||||
*
|
||||
* The measurements that describe the scene library — how much frame each one
|
||||
* paints, how much it changes between songs, what it looks like structurally —
|
||||
* can only be taken by rendering, and rendering needs WebGL, which means a
|
||||
* browser. So the numbers are produced in a page and have to get from there
|
||||
* into a git-tracked file. This is that hole in the wall.
|
||||
*
|
||||
* Dev only, by construction: `apply: 'serve'` means it does not exist in a
|
||||
* build, so a deployed gallery cannot write anything. The path it may write is
|
||||
* a fixed allow-list rather than a parameter — a request body decides the
|
||||
* CONTENTS of a known file and never which file.
|
||||
*/
|
||||
function measurementWriter() {
|
||||
const WRITABLE = {
|
||||
'/__metadata': 'src/scenes/metadata.json',
|
||||
};
|
||||
|
||||
return {
|
||||
name: 'flow-state-measurement-writer',
|
||||
apply: 'serve',
|
||||
configureServer(server) {
|
||||
server.middlewares.use((req, res, next) => {
|
||||
const target = WRITABLE[req.url];
|
||||
if (!target || req.method !== 'POST') return next();
|
||||
|
||||
let body = '';
|
||||
req.on('data', (chunk) => { body += chunk; });
|
||||
req.on('end', () => {
|
||||
try {
|
||||
writeFileSync(resolve(process.cwd(), target), body);
|
||||
res.statusCode = 200;
|
||||
res.end(`wrote ${target} (${body.length} bytes)`);
|
||||
} catch (err) {
|
||||
res.statusCode = 500;
|
||||
res.end(String(err && err.message));
|
||||
}
|
||||
});
|
||||
});
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export default defineConfig({
|
||||
plugins: [measurementWriter()],
|
||||
server: {
|
||||
port: 5180,
|
||||
host: '127.0.0.1',
|
||||
|
||||
Loading…
Reference in New Issue
Block a user