Compare commits
29 Commits
070f4d6e01
...
8dfd8392f3
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8dfd8392f3 | ||
|
|
b7c6fd1c5a | ||
|
|
d31d0fc3a1 | ||
|
|
19bfff8651 | ||
|
|
189328587d | ||
|
|
acc33cedd7 | ||
|
|
d14d473974 | ||
|
|
f050eaaad1 | ||
|
|
b69eaa799d | ||
|
|
14d1204e82 | ||
|
|
44e1826fcb | ||
|
|
2806ef1386 | ||
|
|
0ee7a6a3b3 | ||
|
|
2e4e1d7613 | ||
|
|
045ba6a322 | ||
|
|
7d59ef8e5f | ||
|
|
ca5a68eb84 | ||
|
|
9d15c3cf49 | ||
|
|
8a94a3f3a5 | ||
|
|
40188493ac | ||
|
|
d0f93e5c55 | ||
|
|
968101c532 | ||
|
|
59180948d0 | ||
|
|
5f25437b89 | ||
|
|
022c267888 | ||
|
|
c9b80308d2 | ||
|
|
cb320557e6 | ||
|
|
7e31c19d6e | ||
|
|
e8fb647f11 |
11
.claude/launch.json
Normal file
11
.claude/launch.json
Normal file
@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"version": "0.0.1",
|
||||||
|
"configurations": [
|
||||||
|
{
|
||||||
|
"name": "flow-state",
|
||||||
|
"runtimeExecutable": "npm",
|
||||||
|
"runtimeArgs": ["--prefix", "flow-state", "run", "dev"],
|
||||||
|
"port": 5180
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
11
flow-state/.claude/launch.json
Normal file
11
flow-state/.claude/launch.json
Normal file
@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"version": "0.0.1",
|
||||||
|
"configurations": [
|
||||||
|
{
|
||||||
|
"name": "flow-state",
|
||||||
|
"runtimeExecutable": "npm",
|
||||||
|
"runtimeArgs": ["run", "dev"],
|
||||||
|
"port": 5180
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
99
flow-state/.claude/skills/build-visualizer/SKILL.md
Normal file
99
flow-state/.claude/skills/build-visualizer/SKILL.md
Normal file
@ -0,0 +1,99 @@
|
|||||||
|
---
|
||||||
|
name: build-visualizer
|
||||||
|
description: Add a new visualizer (scene) to flow-state, or fix one that is failing its gates. Use when asked to build/add/write a visualizer, scene, or shader for this project, when a scene fails lint or the phase gates, or when the scene library needs another entry in some family. Covers the scaffolder, the shader contract, the personality traits, and the single-scene gate loop.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Build a visualizer
|
||||||
|
|
||||||
|
A scene is **a fragment shader plus a params block**. Uniform binding, UI sliders,
|
||||||
|
per-track sampling, arc drift and every gate are derived from the schema — there
|
||||||
|
is no per-scene wiring to write.
|
||||||
|
|
||||||
|
The full reference is [HOWTO-visualizers.md](../../../HOWTO-visualizers.md). Read
|
||||||
|
it once for the contract details. This file is the *procedure*, and following it
|
||||||
|
in order is what keeps a new scene from costing several rounds of trial and error.
|
||||||
|
|
||||||
|
## The loop
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm run new:scene -- "Salt Flat" --family=minimal --traits=shape,camera,space,style
|
||||||
|
```
|
||||||
|
|
||||||
|
That writes `src/scenes/shader/salt-flat.js`, registers it, and leaves a skeleton
|
||||||
|
that already passes every gate — live, animated, seeded, distinct from every
|
||||||
|
other scene, honouring exactly the traits it declares. Add `--feedback` if the
|
||||||
|
scene will read `prev()`.
|
||||||
|
|
||||||
|
Then, in order:
|
||||||
|
|
||||||
|
1. **Write the concept comment first.** Two lines: what it looks like, and what
|
||||||
|
makes it different from the scenes it sits beside. "No two scenes render the
|
||||||
|
same image" is a gate with a numeric floor, not a guideline. If you cannot
|
||||||
|
write the second line, the scene does not exist yet.
|
||||||
|
2. **Replace the `scene()` body.** Keep the skeleton's trait calls; they are
|
||||||
|
what the casting rule is checked against.
|
||||||
|
3. **Lint.** `npm run lint:scenes` — one second, no browser, catches schema and
|
||||||
|
shader disagreeing, undeclared uniforms, missing `rate: true`, a declared
|
||||||
|
trait with no evidence, a dead camera, `prev()` with no base image, and
|
||||||
|
unbounded loops.
|
||||||
|
4. **Gate the one scene.** Open
|
||||||
|
`http://localhost:5180/checks.html?scene=Salt%20Flat`.
|
||||||
|
Ten-ish lines: schema, renders, animates, deterministic, distinct, param
|
||||||
|
sweep, flash rate, and one line per declared trait. This is the same battery
|
||||||
|
Phases 2, 5 and 7 apply library-wide, filtered to your scene.
|
||||||
|
5. **Run the library gates** once at the end: `checks.html?slow=1`. Phases 2, 5
|
||||||
|
and 7 iterate the registry, so the new scene is covered automatically.
|
||||||
|
|
||||||
|
Do not skip 3 before 4, or 4 before 5. Each step is roughly ten times cheaper
|
||||||
|
than the next and catches a different class of mistake.
|
||||||
|
|
||||||
|
## Choosing family and traits
|
||||||
|
|
||||||
|
**Family** decides which section kinds can cast the scene — a breakdown never
|
||||||
|
lands on a strobing glitch scene. Aim for 4-6 scenes per family; check the
|
||||||
|
current spread with:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
node -e "import('./src/scenes/registry.js').then(({scenes})=>{const b={};for(const m of scenes)(b[m.family]??=[]).push(m.name);console.log(b)})"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Traits** are a contract, not a hint. Each track is built on a signature of one
|
||||||
|
or two traits and **a scene that does not honour all of them is never cast in
|
||||||
|
that track**. Declare only what the shader genuinely uses:
|
||||||
|
|
||||||
|
| trait | call | what it means for your scene |
|
||||||
|
|---|---|---|
|
||||||
|
| `shape` | `sigShape(p)` / `sigForm(p, at, size)` | every element you draw is the track's signature form, not your own circle or box |
|
||||||
|
| `camera` | `sigCamera(p)` | your coordinate is filmed by the track's operator. **Must feed the image** — assigning it to a `p` you then ignore is a dead camera, and both the lint and the render gate will say so |
|
||||||
|
| `space` | `sigHorizonY()`, `sigAir(col, p, d)` | your ground is at the track's horizon and your distance haze is the track's |
|
||||||
|
| `style` | `sigEdge(d)`, `sigGrain(uv)`, `sigFolded(p)`, `u_sigLine/Soft/Texture/Fold` | your lines are drawn in the track's weight |
|
||||||
|
|
||||||
|
Prefer thin traits. `space` and `shape` carry the most identity and have the
|
||||||
|
fewest scenes, so they are usually where another scene is worth most.
|
||||||
|
|
||||||
|
## The five mistakes that actually happen
|
||||||
|
|
||||||
|
1. **A rate param that is not flagged.** Anything multiplying `u_time` needs
|
||||||
|
`rate: true`, or reactivity jumps the phase by `elapsed × Δrate` and the scene
|
||||||
|
strobes. Add a bounded term instead: `u_time * u_speed + u_bandLow * 2.0`.
|
||||||
|
2. **Whole-frame luminance on the beat.** That is the WCAG 2.3.1 failure the
|
||||||
|
flash gate exists for. Pulse something local; quantise glitches onto
|
||||||
|
`floor(u_barPhase * n)` so they step with the music.
|
||||||
|
3. **A declared trait the image does not respond to.** Passes the eye, fails the
|
||||||
|
gate. Both cost the same to fix before you commit and much more after.
|
||||||
|
4. **Reading `prev()` with nothing underneath.** Black for the first frames,
|
||||||
|
different after a seek than after playback. Always draw a base field.
|
||||||
|
5. **A contract uniform name reused as a param** (`u_width`, `u_time`, `u_seed`).
|
||||||
|
GLSL redefinition; the only symptom is a black frame.
|
||||||
|
|
||||||
|
## When a gate fails
|
||||||
|
|
||||||
|
- **not distinct** — the closest scene is named in the output. Change the
|
||||||
|
structure, not the palette; colour comes from the track.
|
||||||
|
- **dead/blown in the param sweep** — the sweep pushes each param to its limits
|
||||||
|
alone. Usually a range that should not reach 0, or one that saturates.
|
||||||
|
- **flash rate over 3/s** — find the term that swings the whole frame and make it
|
||||||
|
local or smooth.
|
||||||
|
- **not deterministic** — something is reading wall-clock or unseeded randomness;
|
||||||
|
the lint greps for the usual suspects, but `fwidth`-style derivative tricks can
|
||||||
|
also differ. Everything must be a function of `u_time`/`u_frame` and `u_seed`.
|
||||||
4
flow-state/.gitignore
vendored
Normal file
4
flow-state/.gitignore
vendored
Normal file
@ -0,0 +1,4 @@
|
|||||||
|
node_modules
|
||||||
|
dist
|
||||||
|
.vite
|
||||||
|
out
|
||||||
235
flow-state/EPIC-2.md
Normal file
235
flow-state/EPIC-2.md
Normal file
@ -0,0 +1,235 @@
|
|||||||
|
# Epic 2 — from texture generator to something worth watching
|
||||||
|
|
||||||
|
PLAN.md took this from nothing to a deterministic engine with a 42-scene library, a
|
||||||
|
production-design layer, and an editor. Epic 2 is the answer to a different question, asked
|
||||||
|
after watching the result rather than after building it:
|
||||||
|
|
||||||
|
> Is it varied enough per track? Interesting enough to hold five minutes? Pleasant enough to
|
||||||
|
> watch for long? Is there a unifying factor? Are we reusing the same visuals between tracks?
|
||||||
|
> Are the visuals active enough?
|
||||||
|
|
||||||
|
The honest answers were: partly, no, mixed, yes, yes, and no in both directions. This document
|
||||||
|
is what those answers turn into.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. What was measured
|
||||||
|
|
||||||
|
Everything below is a number off the real system, not an impression. The measurements are
|
||||||
|
reproducible from the check harness and several have become gates (§4).
|
||||||
|
|
||||||
|
**Shot rhythm is a metronome.** A five-minute track at 90 BPM gets sixteen shots of 19.3,
|
||||||
|
18.7, 18.7, 18.7 … 18.7, 19.3 seconds. At 124 BPM it is 15.5/15.5/13.6 repeating. `planShots`
|
||||||
|
picks a bar count *once per section* and then divides the section evenly, so every cut inside
|
||||||
|
a section lands on the same pulse. Over five minutes that is the most fatiguing edit rhythm
|
||||||
|
available.
|
||||||
|
|
||||||
|
**The palette is frozen for the whole track.** One `look.palette`, pushed to every layer of
|
||||||
|
every section for the entire runtime. Colour is the strongest perceptual variable the system
|
||||||
|
has and it never moves.
|
||||||
|
|
||||||
|
**39% of the library does not develop.** Rendering every scene at generated parameters and
|
||||||
|
comparing frames 0.5 s / 5 s / 30 s / 2 min apart, normalised for brightness:
|
||||||
|
|
||||||
|
| behaviour | count | examples (0.5 s → 2 min) |
|
||||||
|
|---|---|---|
|
||||||
|
| churn — changes as much in half a second as in two minutes | 11 | Curl Flow 0.81 → 0.75, Moiré Grid 1.42 → 1.40, Circuit Bloom 2.86 → 2.83 |
|
||||||
|
| nearly static | 5 | Pylon Grid 0.15 at 2 min, Dust Chamber 0.39, Salt Flat 0.64 |
|
||||||
|
| genuinely evolving | ~25 | Flora 0.29 → 3.25, Silk Ribbon 0.26 → 2.68, Slow Orb 0.09 → 1.21 |
|
||||||
|
|
||||||
|
Churn is the more interesting failure. Those scenes are violently animated and read as
|
||||||
|
*static*, because motion without structure is texture: the eye adapts in about two seconds and
|
||||||
|
then there is nothing left to find. "Active enough" fails at both ends of the range.
|
||||||
|
|
||||||
|
**The most-cast scenes are among the least developing.** Across twelve tracks, four scenes
|
||||||
|
appeared in ten to twelve of them — Dust Chamber, Slow Orb, Salt Flat, Eclipse Field — and two
|
||||||
|
of those are in the nearly-static bucket. Twenty-nine of forty-two scenes appeared in none of
|
||||||
|
the twelve.
|
||||||
|
|
||||||
|
**Every track makes the same genre decisions.** `FAMILY_BY_KIND` is a module constant: an
|
||||||
|
intro is always minimal/flow/organic, a drop always geometric/glitch/structural, and intro and
|
||||||
|
outro are literally the same list. Cross-track sameness is baked into a table.
|
||||||
|
|
||||||
|
**A five-minute track shows five or six distinct images.**
|
||||||
|
|
||||||
|
**The production-design layer works and is not in scope for this epic.** Changing only the
|
||||||
|
personality moves the image 0.846; changing only the parameters 0.882; changing only the
|
||||||
|
palette 0.467. The unifying mechanism is as strong as the variation mechanism. Leave it alone.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. The diagnosis
|
||||||
|
|
||||||
|
Two separate problems, and it matters that they are separate:
|
||||||
|
|
||||||
|
**The system has no arc longer than fifteen seconds.** It has a per-frame arc (reactivity), a
|
||||||
|
per-shot arc (cuts), and a per-section arc (drift), and then it stops. Nothing is built to pay
|
||||||
|
off at minute four. Held attention over five minutes is a property of *structure over
|
||||||
|
minutes*, and there is currently no mechanism that operates on that timescale.
|
||||||
|
|
||||||
|
**Every scene is the same shot.** A locked-off, full-frame, abstract texture. There is no
|
||||||
|
scale, no framing, no composition — no wide against close, no negative space, no subject the
|
||||||
|
eye can track across a cut. Music-video attention is held by framing changes at least as much
|
||||||
|
as by content changes, and the vocabulary for that does not exist here at all.
|
||||||
|
|
||||||
|
The first is a set of bugs in disguise. The second is a ceiling.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. The work
|
||||||
|
|
||||||
|
Five workstreams, ordered by impact per unit of effort. Each is independently shippable and
|
||||||
|
independently valuable; nothing here is a prerequisite for anything below it except where
|
||||||
|
noted.
|
||||||
|
|
||||||
|
### 3.1 Break the cut metronome
|
||||||
|
|
||||||
|
Shot lengths must vary *within* a section. The target is a phrase shape rather than a pulse: a
|
||||||
|
long hold, two quick ones, a long hold. Cuts still land on downbeats and still respect the
|
||||||
|
floor and ceiling — this changes where the cuts are, not the rules they obey.
|
||||||
|
|
||||||
|
The mechanism is a per-section **rhythm pattern**: a small set of bar-length sequences (`[8,
|
||||||
|
8, 4, 4, 16]` and friends) chosen by seed and energy, walked in order and repeated as needed,
|
||||||
|
instead of one bar count divided evenly. A section then reads as edited rather than as
|
||||||
|
metered.
|
||||||
|
|
||||||
|
Two things to preserve, both load-bearing and both easy to break here: cuts must stay on
|
||||||
|
phrase lines, and `MIN_SHOT_SECONDS` / `MAX_SHOT_SECONDS` remain hard.
|
||||||
|
|
||||||
|
### 3.2 A per-track director
|
||||||
|
|
||||||
|
`FAMILY_BY_KIND` becomes one of several mappings, chosen per track at look-generation time.
|
||||||
|
Each mapping is a coherent point of view about what a song looks like — one that answers a
|
||||||
|
drop with geometry, one that answers it with glitch, one that stays organic throughout, one
|
||||||
|
that treats the intro as structural rather than minimal.
|
||||||
|
|
||||||
|
This is a small change that directly attacks the largest source of cross-track sameness, and
|
||||||
|
it makes the twenty-nine unused scenes reachable without touching the library or the casting
|
||||||
|
weights.
|
||||||
|
|
||||||
|
### 3.3 Let the palette move
|
||||||
|
|
||||||
|
Per-section colour movement, driven by the arc that already exists: a hue rotation, a
|
||||||
|
saturation shift, or a scheme change into a drop. The track keeps one identity — the movement
|
||||||
|
is *from* the track's palette, not a replacement for it — so this does not fight §3.5 of
|
||||||
|
PLAN.md or the personality layer.
|
||||||
|
|
||||||
|
Constraint: the contrast floor the Phase 3 palette gate enforces has to hold at every point in
|
||||||
|
the movement, not just at the endpoints.
|
||||||
|
|
||||||
|
### 3.4 Give the churn scenes a slow axis
|
||||||
|
|
||||||
|
Each of the eleven needs at least one parameter on a multi-minute ramp, so that the frame at
|
||||||
|
three minutes is not the frame at thirty seconds. This is per-scene work and it is mechanical,
|
||||||
|
but it is the only thing that fixes the specific failure of "animated and yet static".
|
||||||
|
|
||||||
|
The five nearly-static scenes get the opposite treatment, and Phase 7's movement gate already
|
||||||
|
describes what "enough" means.
|
||||||
|
|
||||||
|
**Status after the first pass: mechanism delivered, most of the scene work still open.**
|
||||||
|
|
||||||
|
The mechanism exists and is proven. `ArcDriver._slowAxisFor` walks a declared param across the
|
||||||
|
whole track, monotonically — the existing drift is a 20-70 second LFO, and an LFO returns,
|
||||||
|
which is precisely why ten cycles of it over five minutes reads as static.
|
||||||
|
|
||||||
|
Two things were learned the hard way and are worth not relearning:
|
||||||
|
|
||||||
|
*Which param you move decides everything.* The first version chose at random from everything
|
||||||
|
eligible and measured as doing **nothing whatsoever** — identical structural change with the
|
||||||
|
axis applied and with it disabled. Sweeping Moiré Grid's `width` moves its time-averaged
|
||||||
|
structure by 0.110 and its `offset` by 0.002; a random draw finds the second kind almost every
|
||||||
|
time. Hence `slowAxis: true` as a declaration rather than a heuristic.
|
||||||
|
|
||||||
|
*Single-frame distance cannot measure this.* A churning scene's consecutive frames are already
|
||||||
|
~0.6 apart, so every pair of its frames scores the same whether the structure moved or not.
|
||||||
|
The gate measures **ten-second time-averaged** frames, which cancels the churn and leaves the
|
||||||
|
structure. The window was measured: at one second, Moiré Grid's frozen-parameter control still
|
||||||
|
reads 0.024; at ten it reads 0.0095 while the axis-driven signal stays at 0.037.
|
||||||
|
|
||||||
|
Measured across ten candidate scenes (ratio of axis-driven structural change to what the scene
|
||||||
|
does on its own):
|
||||||
|
|
||||||
|
| works | Moiré Grid 3.93× · Gate Corridor 2.88× · Truchet Fold 1.40× |
|
||||||
|
|---|---|
|
||||||
|
| **no parameter helps** | Curl Flow 1.31× · Signal Decay 1.21× · Circuit Bloom 1.10× |
|
||||||
|
| **already develops; never the problem** | Firefly Drift · Vortex Drift · Kaleido Tunnel · Plasma Bloom |
|
||||||
|
|
||||||
|
The middle row is the remaining work, and it is **not** mechanical: those scenes have no
|
||||||
|
parameter that changes their structure, so they need shader changes that introduce one.
|
||||||
|
Parameter automation cannot substitute for structure a scene does not have.
|
||||||
|
|
||||||
|
### 3.5 A framing layer
|
||||||
|
|
||||||
|
The one that raises the ceiling rather than the floor. A shared zoom / crop / scale envelope
|
||||||
|
sitting above the scenes, so the same image can be played as a wide and as a close, and so a
|
||||||
|
cut can change the framing without changing the subject.
|
||||||
|
|
||||||
|
This is deliberately last: it is the largest change, it interacts with resolution
|
||||||
|
independence (§1 of PLAN.md) and with the feedback buffer, and the four items above will have
|
||||||
|
changed what it needs to do.
|
||||||
|
|
||||||
|
**Status after the first pass: shipped, deliberately simple.**
|
||||||
|
|
||||||
|
A shot now carries a size and a recentre, applied inside `sigCamera` in scene coordinates —
|
||||||
|
rendered close rather than magnified, which is what keeps it resolution-independent and free at
|
||||||
|
4K. `look/framing.js` owns the vocabulary: three shot sizes with measured headroom (a close-up
|
||||||
|
past ~2.2 is a blurry wide, a wide past ~0.55 is a speck), and a per-track style (`locked` /
|
||||||
|
`gentle` / `edited`) that decides how far sizes travel and how often a cut also changes the shot.
|
||||||
|
The arc driver plans one framing per cue, walking the cuts in order so the whole video gets a
|
||||||
|
consistent hand and a seek finds the same framing as playback. The gates hold it honest: cuts
|
||||||
|
actually change size, sizes stay in headroom, and two drivers over one look agree on every shot.
|
||||||
|
|
||||||
|
What this pass does not do, and the reason it is "simple": the framing is constant within a shot.
|
||||||
|
A zoom that moves during a shot is a separate device and would fight the drift LFO and the slow
|
||||||
|
axis, both of which already own continuous motion. Scenes built outside the shader contract (the
|
||||||
|
single 3D layer) are not framed yet. Both are the obvious next steps and neither is required for
|
||||||
|
a working project.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Validation
|
||||||
|
|
||||||
|
The rule from PLAN.md §11 stands: variety is a property of a population, not of one render, so
|
||||||
|
every gate here samples many tracks and asks about a spread.
|
||||||
|
|
||||||
|
| workstream | gate |
|
||||||
|
|---|---|
|
||||||
|
| 3.1 rhythm | Within one section, shot lengths must have real spread (no section where every shot is within a few percent of the mean). Cuts still land on downbeats; floor and ceiling still hold. |
|
||||||
|
| 3.2 director | Across a population, the same section kind must draw from more than one family set. Every scene in the library must be reachable by some director. No director may leave a kind with too few scenes to build a roster from. |
|
||||||
|
| 3.3 palette | Contrast floor holds at every sampled point of the movement, not just the endpoints. Colour actually moves across a track — measurably, not decoratively. |
|
||||||
|
| 3.4 slow axis | Every scene's two-minute frame distance must exceed its half-second distance by a real margin. This is the churn number from §1, promoted to a gate. |
|
||||||
|
| 3.5 framing | Resolution independence survives (the existing dual-resolution diff). Framing changes are visible. Determinism holds. |
|
||||||
|
|
||||||
|
Two failure modes are worth naming in advance, because both have already happened once in this
|
||||||
|
project's history:
|
||||||
|
|
||||||
|
- **A gate that measures the wrong thing.** The Phase 3 look-space check was propped up by
|
||||||
|
grain until grain was removed. Any new gate here should be sanity-checked by deliberately
|
||||||
|
breaking the thing it is supposed to catch.
|
||||||
|
- **A change that a gate rewards and a viewer does not.** Cut variety that is *random* rather
|
||||||
|
than *phrased* would pass 3.1's gate and look worse. The gates are necessary and not
|
||||||
|
sufficient; the end-to-end watch stays the real test.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Sequencing
|
||||||
|
|
||||||
|
1. **3.1 rhythm** — biggest felt change, smallest diff, no dependencies.
|
||||||
|
2. **3.2 director** — small, orthogonal, attacks cross-track sameness directly.
|
||||||
|
3. **3.3 palette** — medium, touches the arc driver.
|
||||||
|
4. **3.4 slow axis** — large but mechanical, eleven-plus scenes, one at a time.
|
||||||
|
5. **3.5 framing** — largest, and best attempted once the other four have landed.
|
||||||
|
|
||||||
|
Each ships as its own commit with its gate green.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. What this epic does not do
|
||||||
|
|
||||||
|
- **It does not touch the personality layer.** Measured, it works.
|
||||||
|
- **It does not grow the library.** Twenty-nine scenes are currently unreachable; making them
|
||||||
|
reachable (3.2) is worth more than adding a forty-third.
|
||||||
|
- **It does not add audio analysis.** Every mechanism here is driven by features that already
|
||||||
|
exist in the FeatureTrack.
|
||||||
|
- **It does not relax determinism.** Every rule in PLAN.md §1 still applies, and the seek /
|
||||||
|
export / repeat gates are non-negotiable.
|
||||||
194
flow-state/HOWTO-visualizers.md
Normal file
194
flow-state/HOWTO-visualizers.md
Normal file
@ -0,0 +1,194 @@
|
|||||||
|
# How to build a visualizer (scene)
|
||||||
|
|
||||||
|
> **Start here:** `npm run new:scene -- "My Scene" --family=glitch --traits=camera,style`
|
||||||
|
>
|
||||||
|
> That writes the module, registers it, and leaves a skeleton that already passes
|
||||||
|
> every gate. Then write the `scene()` body, `npm run lint:scenes`, and open
|
||||||
|
> `checks.html?scene=My%20Scene`. The rest of this file is the reference.
|
||||||
|
>
|
||||||
|
> There is also a repo skill — `.claude/skills/build-visualizer/` — which is the
|
||||||
|
> same procedure in the form an agent will follow.
|
||||||
|
|
||||||
|
A scene is **a fragment shader plus a params block** — nothing else. Uniform
|
||||||
|
binding, the generated UI sliders, seeded per-track sampling, arc drift and the
|
||||||
|
phase-gate checks are all derived from the schema. There is no per-scene wiring.
|
||||||
|
|
||||||
|
Reference: `src/scenes/shader/kaleido-tunnel.js` (simple), `block-mosh.js`
|
||||||
|
(feedback/glitch), `scan-tear.js` (beat-quantised), `metaballs.js` (loops).
|
||||||
|
Read these before writing anything.
|
||||||
|
|
||||||
|
## The shape of a scene
|
||||||
|
|
||||||
|
```js
|
||||||
|
export const myScene = {
|
||||||
|
name: 'My Scene',
|
||||||
|
family: 'glitch', // flow | organic | minimal | structural | geometric | glitch
|
||||||
|
kind: 'fragment',
|
||||||
|
traits: ['camera', 'style'], // which personality traits it honours (see below)
|
||||||
|
|
||||||
|
params: {
|
||||||
|
density: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_density', bias: 'density' },
|
||||||
|
speed: { type: 'float', range: [0.1, 2], default: 0.5, uniform: 'u_speed', rate: true },
|
||||||
|
palette: { type: 'palette', count: 4 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
density: { feature: 'bandLow', amount: 0.3, response: 'smooth' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
return vec4(palRamp(fbm(p * 4.0 + t, 4)), 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
The scaffolder writes all of that for you, including the registry import and
|
||||||
|
entry, and bakes name-derived constants into the skeleton field so two freshly
|
||||||
|
scaffolded scenes are not identical to each other. To do it by hand: write the
|
||||||
|
file and add the import + `MODULES` entry in `src/scenes/registry.js`.
|
||||||
|
|
||||||
|
## The shader contract
|
||||||
|
|
||||||
|
You write one function: `vec4 scene(vec2 uv, vec2 p)`.
|
||||||
|
|
||||||
|
- `uv` is 0..1 across the frame; `p` is centred, aspect-corrected, ~-1..1 on the
|
||||||
|
short axis. **Work in these, never pixels** — scale pixel-sized things by
|
||||||
|
`u_pixelScale` so a 720p preview matches a 4K export.
|
||||||
|
- The preamble is injected for you. Never declare `main()`, `u_resolution`,
|
||||||
|
`vUv`, or any contract uniform yourself.
|
||||||
|
|
||||||
|
**Uniforms available** (see `engine/shader-contract.js`):
|
||||||
|
|
||||||
|
- Frame: `u_time`, `u_frame`, `u_progress`, `u_seed`, `u_resolution`, `u_aspect`,
|
||||||
|
`u_pixelScale`, `u_opacity`.
|
||||||
|
- Audio, filled per frame from the FeatureTrack: `u_loudness`, `u_rms`,
|
||||||
|
`u_bandSub/low/mid/high/air`, `u_flux`, `u_centroid`, `u_flatness`, `u_width`,
|
||||||
|
`u_beat`, `u_beatPhase`, `u_barPhase`, `u_phrasePhase`, `u_sectionProgress`,
|
||||||
|
`u_sectionEnergy`, `u_buildSlope`.
|
||||||
|
- Personality (`u_sigSides`, `u_sigDrift`, `u_sigLine`, …) — see below.
|
||||||
|
- Feedback: `u_prev` sampler, read with `prev(vec2 uv)` (returns the previous
|
||||||
|
frame's colour; black if none). This is what makes trails/smear/datamosh work
|
||||||
|
for free.
|
||||||
|
|
||||||
|
**Helpers** (already in the preamble):
|
||||||
|
|
||||||
|
- Palette: `pal(int)`, `palRamp(float)` — always use these, or the look can't
|
||||||
|
recolour the scene.
|
||||||
|
- Noise: `hash11/12/22`, `vnoise`, `fbm`, `curl`, `rot`, `kaleido`.
|
||||||
|
- Personality: `sigShape`/`sigForm`, `sigCamera`, `sigFolded`, `sigEdge`,
|
||||||
|
`sigGrain`, `sigHorizonY`, `sigAir`.
|
||||||
|
- `sat(x)` = `clamp(x, 0, 1)`.
|
||||||
|
|
||||||
|
## Param schema fields
|
||||||
|
|
||||||
|
- `type`: `float` | `int` | `bool` | `vec2` | `palette`.
|
||||||
|
- `range` `[min, max]` required for numerics; `default` required unless you want
|
||||||
|
range[0] (prefer explicit defaults).
|
||||||
|
- `uniform`: the GLSL name. Must be `u_*` and must not collide with the contract
|
||||||
|
(see "traps").
|
||||||
|
- `bias`: which track-character axis nudges sampling — `energy`, `density`,
|
||||||
|
`motion`. This is how a loud drop gets denser scenes without the scene knowing
|
||||||
|
about audio.
|
||||||
|
- `rate: true`: **mandatory if the shader multiplies `u_time` by this param.**
|
||||||
|
- `reactive`: `{ feature, amount, response }`. `response`: `linear` (default) |
|
||||||
|
`spike` | `smooth` | `inverse`.
|
||||||
|
|
||||||
|
## Personality traits (`traits`)
|
||||||
|
|
||||||
|
The look generator builds each track on a signature of 1-2 traits, and a scene
|
||||||
|
that doesn't honour **all** of them is never cast in that track — so the library
|
||||||
|
intentionally shrinks per track. Only declare what you genuinely use:
|
||||||
|
|
||||||
|
| trait | what to call | lint evidence |
|
||||||
|
|---|---|---|
|
||||||
|
| `shape` | `sigShape` / `sigForm` | a `sigShape`/`sigForm(` call |
|
||||||
|
| `camera` | `sigCamera(p)` | a `sigCamera(` call |
|
||||||
|
| `space` | `sigHorizonY` / `sigAir` | those calls or `u_sigHorizon/Depth/Wash` |
|
||||||
|
| `style` | `sigEdge` / `sigGrain` / `sigFolded`, or `u_sigLine/Soft/Texture/Fold` | the calls / those uniforms |
|
||||||
|
|
||||||
|
The lint greps your shader and fails a declared trait with no evidence. Declaring
|
||||||
|
`[]` (none) is valid.
|
||||||
|
|
||||||
|
### `texture` — how much surface grain your scene accepts
|
||||||
|
|
||||||
|
Optional module field, `0..2`, default `1`. It scales `u_sigTexture` — and so
|
||||||
|
every `sigGrain(uv)` in your shader — for this scene only. Set it to `0` if your
|
||||||
|
scene is crisp line work that grain only furs up, or to something under 1 if it
|
||||||
|
should be dusted rather than dirty. It has nothing to do with the *grade's*
|
||||||
|
grain, which is a per-track treatment (see `look/grain.js`) and is off entirely
|
||||||
|
for most tracks.
|
||||||
|
|
||||||
|
## Traps that have actually bitten here
|
||||||
|
|
||||||
|
1. **Anything multiplying `u_time` must be `rate: true`.** Phase is
|
||||||
|
`elapsed × rate`; modulating a rate jumps the phase by `elapsed × Δrate` — a
|
||||||
|
minute in a small wobble throws the image several units between frames, which
|
||||||
|
measured as strobing at 2× the accessibility limit. So don't react a rate
|
||||||
|
param, and add a *bounded* term instead: `u_time * u_speed + u_bandLow * 2.0`
|
||||||
|
is fine; `u_time * (u_speed + u_bandLow)` is not.
|
||||||
|
2. **Never reuse a contract uniform name** (`u_width`, `u_time`, `u_seed`, …).
|
||||||
|
It's a GLSL redefinition error; the only symptom is a black frame.
|
||||||
|
3. **Don't modulate whole-frame luminance on the beat.** A per-kick min→max→min
|
||||||
|
cycle is exactly what the WCAG 2.3.1 / Harding 3-flashes-per-second ceiling
|
||||||
|
bans. Pulse a *small local* term (per-block tint) instead, and use `smooth`
|
||||||
|
responses on loud things. If it's glitchy, **quantise it** — `floor(u_barPhase
|
||||||
|
* n) + floor(t * k) * n` makes corruption step on the grid instead of
|
||||||
|
crawling, which both reads better and stays below the flash rate (see
|
||||||
|
`scan-tear.js`, `block-mosh.js`).
|
||||||
|
4. **Determinism is absolute.** No `Math.random()`, `performance.now()`,
|
||||||
|
`Date.now()`, `new Date()` — use `hash*`/`fbm` for variation, and let time flow
|
||||||
|
through `u_time`/`u_frame` only. The lint greps for these.
|
||||||
|
5. **Set a base image.** A scene that only reads `prev()` is black for the first
|
||||||
|
frames and fragile under seek. Generate your own field underneath the effect.
|
||||||
|
6. Don't hardcode saturated `vec3(r,g,b)` literals when you declared a palette —
|
||||||
|
the lint flags more than two.
|
||||||
|
|
||||||
|
## Families
|
||||||
|
|
||||||
|
Chosen by section kind in the arc driver — a breakdown never lands on a strobing
|
||||||
|
glitch scene. The library is at **seven per family** (42 scenes). Check the
|
||||||
|
current spread before adding another:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
node -e "import('./src/scenes/registry.js').then(({scenes})=>{const b={};for(const m of scenes)(b[m.family]??=[]).push(m.name);console.log(b)})"
|
||||||
|
```
|
||||||
|
|
||||||
|
Depth matters more than it looks: the casting rule in `look/Personality.js`
|
||||||
|
disqualifies scenes that do not honour the track's signature traits, so the pool
|
||||||
|
a given track draws from is smaller than the library. Thin traits (`space`,
|
||||||
|
`shape`) are worth more than thin families.
|
||||||
|
|
||||||
|
## Verify
|
||||||
|
|
||||||
|
Three rungs, each about ten times cheaper than the next. Climb them in order.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm run lint:scenes # ~1s, no browser
|
||||||
|
```
|
||||||
|
|
||||||
|
Static gates: schema and shader agreeing both ways, determinism grep, rate flags,
|
||||||
|
a declared trait with no evidence in the source, a **dead camera**
|
||||||
|
(`p = sigCamera(p)` and then nothing reads `p`), `prev()` with no base image, and
|
||||||
|
loops with a large bound and no early break. A loop whose cost is genuinely fixed
|
||||||
|
— sampling a curve at a set resolution — can say so with a `// lint: fixed-cost`
|
||||||
|
comment just above it.
|
||||||
|
|
||||||
|
```
|
||||||
|
http://localhost:5180/checks.html?scene=My%20Scene
|
||||||
|
```
|
||||||
|
|
||||||
|
The per-scene acceptance battery for one scene: schema, renders, animates,
|
||||||
|
deterministic, distinct from every other scene, param sweep, flash rate, and one
|
||||||
|
line per declared trait proving the image actually responds to it. Ten lines and
|
||||||
|
a verdict — this is the loop to stay in while writing.
|
||||||
|
|
||||||
|
```
|
||||||
|
http://localhost:5180/checks.html?slow=1
|
||||||
|
```
|
||||||
|
|
||||||
|
Everything. Phases 2, 5 and 7 iterate the registry so a new scene is covered
|
||||||
|
automatically; Phases 8-10 cover how the look generator uses it. Run this once
|
||||||
|
before committing.
|
||||||
653
flow-state/PLAN.md
Normal file
653
flow-state/PLAN.md
Normal file
@ -0,0 +1,653 @@
|
|||||||
|
# flow-state — plan
|
||||||
|
|
||||||
|
An ambient/EDM music video generator. Drop in a track, get a full-length, non-story,
|
||||||
|
music-reactive video that evolves through the song. No sourced footage: every frame is
|
||||||
|
generated, and everything the look depends on is derived from the audio itself.
|
||||||
|
|
||||||
|
Name is a placeholder, easy to rename before Phase 0 lands.
|
||||||
|
|
||||||
|
## Decisions taken
|
||||||
|
|
||||||
|
| Question | Decision |
|
||||||
|
|---|---|
|
||||||
|
| Render model | Deterministic core. Realtime preview **and** offline frame-exact export, from the same code path. |
|
||||||
|
| Preview | **Required.** Full transport, section jumping, live param editing, and segment test-renders — see §8. |
|
||||||
|
| Art direction | Automatic from the audio, every derived parameter overridable and savable per track. |
|
||||||
|
| Scene architecture | Hybrid compositor. Layers are usually fullscreen fragment shaders; three.js particle/geometry layers are also valid layers. |
|
||||||
|
| Cover art | Not available. Palette is audio-derived. A `PaletteSource` seam is left so artwork can be added later without touching scenes. |
|
||||||
|
| Relationship to `party-stage` | Fork and **detach**. Copy what's useful, then zero cross-imports — see §12. |
|
||||||
|
|
||||||
|
## The two problems this has to solve
|
||||||
|
|
||||||
|
They are separate and both are load-bearing:
|
||||||
|
|
||||||
|
1. **Sameness across tracks.** Track #12 must not look like track #11. Solved by deriving
|
||||||
|
the look from the track's own measured character plus a content-derived seed.
|
||||||
|
2. **Monotony within a track.** A single shader gets dull around minute three. Solved by
|
||||||
|
segmenting the song and driving scene changes, palette shifts and parameter motion off
|
||||||
|
that structure.
|
||||||
|
|
||||||
|
Both have explicit validation gates in §11 — they are easy to *believe* you've solved and
|
||||||
|
hard to actually solve.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. The determinism spine
|
||||||
|
|
||||||
|
This is the single architectural constraint everything else obeys. It is what makes
|
||||||
|
"realtime preview" and "4K60 offline export" the same program rather than two programs
|
||||||
|
that drift apart.
|
||||||
|
|
||||||
|
**Mechanism: the audio is fully analyzed before the first frame renders, into a
|
||||||
|
frame-indexed table.**
|
||||||
|
|
||||||
|
```
|
||||||
|
track file
|
||||||
|
↓ decode() OfflineAudioContext → AudioBuffer (a 6-min track decodes in ~1-2s)
|
||||||
|
↓ analyze() STFT at hop = 1/60 s, plus global passes
|
||||||
|
↓
|
||||||
|
FeatureTrack a typed-array table, one row per output frame
|
||||||
|
```
|
||||||
|
|
||||||
|
Nothing in the engine ever reads a live `AnalyserNode`. Realtime preview maps
|
||||||
|
`audio.currentTime → frameIndex` and reads row *n*. The offline exporter counts
|
||||||
|
`frameIndex` directly and reads row *n*. **Identical input to the visuals in both modes**,
|
||||||
|
so what you preview is exactly what you export.
|
||||||
|
|
||||||
|
This also unlocks the thing a causal analyser fundamentally cannot do:
|
||||||
|
|
||||||
|
> **Lookahead.** Because the whole song is analyzed up front, a build can *anticipate* its
|
||||||
|
> drop — ramp density and warp toward the drop's target values over the eight bars before
|
||||||
|
> it, and hit the transition already at full tension. A causal analyser can never do this;
|
||||||
|
> it only knows the drop landed after it landed. This is the biggest single visual win in
|
||||||
|
> the whole plan.
|
||||||
|
|
||||||
|
### Determinism rules (violating any of these breaks export)
|
||||||
|
|
||||||
|
- **No wall-clock anywhere in scene or layer code.** No `performance.now()`, no `Date`, no
|
||||||
|
`clock.getElapsedTime()`. A `Timeline` object injects `{ frame, time, dt, progress }`.
|
||||||
|
- **No `Math.random()` in anything that animates.** Seeded PRNG only (mulberry32, with the
|
||||||
|
seed held per-instance rather than on a global).
|
||||||
|
- **`dt` is constant** — `1/fps` — even in realtime preview. If the browser drops a frame,
|
||||||
|
preview shows a hitch; it does not change the animation. Frame 5400 is the same image
|
||||||
|
every run.
|
||||||
|
- **Deterministic buffer initialization.** Feedback and ping-pong targets must be explicitly
|
||||||
|
cleared at start; never inherit whatever was in GPU memory.
|
||||||
|
- **Resolution independence.** Scene math lives in normalized aspect-corrected coordinates.
|
||||||
|
Anything genuinely pixel-sized takes a `u_pixelScale = resolution / referenceResolution`
|
||||||
|
uniform. This is what lets a 720p preview match a 4K export.
|
||||||
|
|
||||||
|
### Stated limit on the guarantee
|
||||||
|
|
||||||
|
Determinism is **per-machine**: same browser, same GPU, same driver → bit-identical frames.
|
||||||
|
Across different GPUs, floating-point and `fwidth` derivative differences make bit-exactness
|
||||||
|
unrealistic, and chasing it would be wasted effort. The cross-machine guarantee is
|
||||||
|
*perceptual*: frames must match within a small diff threshold. The validation checks in §11
|
||||||
|
are written to that distinction, so the acceptance criteria are actually achievable.
|
||||||
|
|
||||||
|
Two refinements, both found by building it:
|
||||||
|
|
||||||
|
**Programs must be primed before the first frame.** Shader programs link asynchronously
|
||||||
|
(`KHR_parallel_shader_compile`), and a draw issued against an unlinked program produces
|
||||||
|
wrong output. Measured on the heaviest scene, the first *ten* frames rendered differently
|
||||||
|
from every later render of the same frames. Preview hides this completely — the frames go
|
||||||
|
past and the next pass is right — but an export renders each frame exactly once, so those
|
||||||
|
frames would ship broken. `Engine.prime()` compiles every program and discards a warm frame,
|
||||||
|
and the exporter calls it before encoding anything. Rendering a throwaway frame and reading
|
||||||
|
it back is *not* sufficient; `WebGLRenderer.compile()` is.
|
||||||
|
|
||||||
|
**Even same-machine, the heaviest shaders vary by one LSB.** With priming in place most
|
||||||
|
scenes reproduce byte-for-byte, but a few come back with scattered pixels differing by 1/255
|
||||||
|
— floating-point variance under differing GPU load. That is below any perceptual threshold
|
||||||
|
and is not something the hardware offers to fix, so the per-scene criterion is a max
|
||||||
|
channel delta of ≤ 1 rather than an identical hash. A real determinism bug scores in the
|
||||||
|
tens or hundreds on that metric, so the check keeps its teeth.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Audio analysis
|
||||||
|
|
||||||
|
`src/audio/`. Pure functions over an `AudioBuffer` — no DOM, no three.js — so it's unit
|
||||||
|
testable and can move to a worker without rework.
|
||||||
|
|
||||||
|
**Per frame** (hop 1/60 s, window 2048, Hann):
|
||||||
|
|
||||||
|
| Feature | Used for |
|
||||||
|
|---|---|
|
||||||
|
| `rms`, `loudness` | overall intensity, quiet-passage detection |
|
||||||
|
| band energies (sub / low / mid / high / air) | per-element reactivity |
|
||||||
|
| `flux` (spectral flux, half-wave rectified) | onset strength, the beat signal |
|
||||||
|
| `centroid` | perceived brightness → color temperature |
|
||||||
|
| `flatness` | noisy vs tonal → distinguishes pads from percussion |
|
||||||
|
| `stereoWidth` | width of the visual field |
|
||||||
|
|
||||||
|
**Global passes:**
|
||||||
|
|
||||||
|
- **Tempo** — autocorrelation over the onset envelope, octave-normalized to 90-180 BPM.
|
||||||
|
Whole-song, so it's exact rather than converging. Yields a phase-locked **beat grid** and
|
||||||
|
downbeats, hence exact `beatPhase` / `barPhase` / `phrasePhase` per frame.
|
||||||
|
- **Segmentation** — novelty curve from a self-similarity matrix over the band-energy
|
||||||
|
vectors, peak-picked into section boundaries.
|
||||||
|
- **Section classification** — each section labelled by energy percentile, flux density and
|
||||||
|
centroid: `intro | build | drop | sustain | breakdown | outro`. Deliberately simple
|
||||||
|
heuristics over features we already have; this is the fuzziest part of the pipeline and is
|
||||||
|
not worth an ML detour.
|
||||||
|
|
||||||
|
**Lookahead fields, per frame:** `timeToNextSection`, `nextSectionKind`, `nextSectionEnergy`,
|
||||||
|
`buildSlope`. These are what the arc driver leans on.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Scene modules and the parameter schema
|
||||||
|
|
||||||
|
**This is the decision that makes a large library affordable.** Adding a scene must cost a
|
||||||
|
shader and a schema block — no plumbing, no UI code, no wiring.
|
||||||
|
|
||||||
|
```js
|
||||||
|
export const NebulaDrift = {
|
||||||
|
name: 'Nebula Drift',
|
||||||
|
family: 'organic',
|
||||||
|
kind: 'fragment', // or 'layer3d'
|
||||||
|
shader: fragmentSource,
|
||||||
|
|
||||||
|
params: {
|
||||||
|
density: { type:'float', range:[0,1], default:0.5, uniform:'u_density' },
|
||||||
|
warp: { type:'float', range:[0,3], default:1.0, uniform:'u_warp' },
|
||||||
|
symmetry: { type:'int', range:[1,8], default:1, uniform:'u_symmetry' },
|
||||||
|
grain: { type:'float', range:[0,1], default:0.2, uniform:'u_grain' },
|
||||||
|
palette: { type:'palette', count:5, uniform:'u_colors' },
|
||||||
|
},
|
||||||
|
|
||||||
|
// which audio features modulate which params, and how hard
|
||||||
|
reactive: {
|
||||||
|
density: { feature:'bandLow', amount:0.3 },
|
||||||
|
warp: { feature:'beat', amount:0.5, response:'spike' },
|
||||||
|
grain: { feature:'flatness', amount:0.2 },
|
||||||
|
},
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
One schema, five payoffs:
|
||||||
|
|
||||||
|
1. Uniforms bind automatically — no per-scene wiring.
|
||||||
|
2. The config UI generates its own sliders, so override-ability is free for every new scene.
|
||||||
|
3. `LookGenerator` samples the declared ranges with the track's seed → per-track variation
|
||||||
|
with no hand-tuning.
|
||||||
|
4. `ArcDriver` can animate any declared param across a section boundary generically.
|
||||||
|
5. Presets are just serialized param sets, savable per track.
|
||||||
|
|
||||||
|
It also makes scenes **machine-checkable**, which is what keeps a thirty-scene library from
|
||||||
|
rotting — see the schema lint in §10.
|
||||||
|
|
||||||
|
### Library targets
|
||||||
|
|
||||||
|
Interest across songs comes from breadth of *family*, not raw count. Six families, growing
|
||||||
|
to roughly 4-6 scenes each:
|
||||||
|
|
||||||
|
| Family | Character | Fits |
|
||||||
|
|---|---|---|
|
||||||
|
| **flow** | curl noise, fluid, drifting particles | ambient, sustain |
|
||||||
|
| **geometric** | tunnels, kaleidoscope, grids, synthwave | drops |
|
||||||
|
| **organic** | nebula, plasma, reaction-diffusion, metaballs | sustain, breakdown |
|
||||||
|
| **structural** | raymarched terrain, cities, architecture | builds |
|
||||||
|
| **minimal** | lines, waveform, spectrum sculpture, negative space | intros, quiet passages |
|
||||||
|
| **glitch** | feedback, datamosh, scanlines, chromatic tearing | drops, transitions |
|
||||||
|
|
||||||
|
The arc driver picks a *family* from section kind, then a scene within it from the seed — so
|
||||||
|
a quiet passage never lands on a strobing glitch scene, and two tracks with similar structure
|
||||||
|
still choose differently.
|
||||||
|
|
||||||
|
### Seeded from the five that exist
|
||||||
|
|
||||||
|
`party-stage`'s five visualizer shaders are the library's starting point. Porting each means:
|
||||||
|
strip the LED-grid mask, replace hardcoded colors with palette lookups, pull magic numbers
|
||||||
|
out into declared params, swap live uniforms for `FeatureTrack` reads. Note their
|
||||||
|
`u_resolution` is not pixels — it's the LED grid cell count. Rename it `u_gridSize` on port
|
||||||
|
so it doesn't collide with the real resolution uniform.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Compositor
|
||||||
|
|
||||||
|
`src/engine/Compositor.js`. A layer stack rendered to ping-pong render targets.
|
||||||
|
|
||||||
|
- **Layers**: fragment-shader (a fullscreen quad) or 3D (a three.js sub-scene — particles,
|
||||||
|
geometry, camera path). Both present the same `render(target, timeline, features)` face,
|
||||||
|
so the stack doesn't care which it holds.
|
||||||
|
- **Blend modes** per layer: normal, add, screen, multiply, overlay.
|
||||||
|
- **Feedback buffer** — previous frame available to any layer as a texture, with configurable
|
||||||
|
decay and warp. Cheap, and disproportionately responsible for things looking "alive".
|
||||||
|
- **Post chain**: bloom, chromatic aberration, grain, vignette, final grade.
|
||||||
|
|
||||||
|
Typical stack: background scene → accent scene (screen blend, low opacity) → particles →
|
||||||
|
feedback → post.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Look generation and the arc driver
|
||||||
|
|
||||||
|
**`LookGenerator`** runs once per track, after analysis:
|
||||||
|
|
||||||
|
```
|
||||||
|
seed = hash(decoded PCM) → same track always renders identically
|
||||||
|
palette = f(centroid, flatness, energy distribution, seed)
|
||||||
|
sceneAssignments = per section: family from kind, scene from seed
|
||||||
|
paramSets = sampled from declared ranges, biased by track character
|
||||||
|
```
|
||||||
|
|
||||||
|
`PaletteSource` is an interface here, with an `AudioPalette` implementation. Adding a
|
||||||
|
`CoverArtPalette` later is a new file and one line of config — no scene changes.
|
||||||
|
|
||||||
|
**`ArcDriver`** runs per frame:
|
||||||
|
|
||||||
|
- Cross-fades scenes at section boundaries (compositor holds both briefly, opacity ramp).
|
||||||
|
- Ramps params toward the *next* section's targets during builds, using the lookahead fields.
|
||||||
|
- Applies slow seeded LFO drift to params within a section, so nothing sits still even
|
||||||
|
during a four-minute sustain.
|
||||||
|
- Applies the `reactive` mappings on top: beat spikes, band-energy modulation.
|
||||||
|
|
||||||
|
Three timescales stacked — per-frame reactivity, per-section drift, whole-song arc — is what
|
||||||
|
keeps a six-minute track from reading as a loop.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Preview
|
||||||
|
|
||||||
|
**Non-negotiable requirement: the composition is fully reviewable before anything is
|
||||||
|
exported.** Export is slow and committing to a six-minute render to discover a bad transition
|
||||||
|
at 4:10 is unacceptable. Preview is not a debug view — it is the primary working surface, and
|
||||||
|
export is the thing you do once you're happy.
|
||||||
|
|
||||||
|
### What it must do
|
||||||
|
|
||||||
|
- **Same everything as export.** Identical compositor, identical look spec, identical
|
||||||
|
`FeatureTrack`. The *only* permitted differences are output resolution and warm-up state
|
||||||
|
(below). Preview must never make a different look decision than export would.
|
||||||
|
- **Transport**: play / pause / scrub, frame step, and **jump to previous/next section
|
||||||
|
boundary** — because transitions are what you actually need to review.
|
||||||
|
- **Loop a section** while tuning it.
|
||||||
|
- **Live param editing** with instant feedback, no track restart. Edits apply to the current
|
||||||
|
section's param set and are savable.
|
||||||
|
- **Timeline strip** under the viewport: sections as colored blocks by kind, beat/bar ticks,
|
||||||
|
playhead, and markers for scene changes. Segmentation problems are visible at a glance here.
|
||||||
|
- **Re-roll controls**: new track seed, re-roll just this section's scene, **lock** a section
|
||||||
|
so further re-rolls leave it alone.
|
||||||
|
- **Quality toggle**: draft (half resolution, post chain simplified) for smooth scrubbing on
|
||||||
|
heavy stacks, versus full-quality preview at export settings but reduced resolution.
|
||||||
|
- **Segment test-render**: export ~20 seconds around the playhead at full export quality.
|
||||||
|
This is the bridge between "looks right in preview" and "commit to the full render", and
|
||||||
|
it is the thing that makes a bad 4K export a rare event rather than a routine one.
|
||||||
|
|
||||||
|
### The seek problem, honestly
|
||||||
|
|
||||||
|
Stateful layers — feedback buffers, particle systems — mean frame *N* depends on the frames
|
||||||
|
before it. Scrubbing to an arbitrary frame therefore can't be exact for free. Three-part
|
||||||
|
resolution:
|
||||||
|
|
||||||
|
1. **Warm-up.** On seek, render *K* frames off-screen before displaying. *K* is computed
|
||||||
|
from the feedback decay rather than fixed — the residual after *n* frames is `decay^n`,
|
||||||
|
so `log(0.001)/log(decay)` frames converges to a tenth of a percent (96 frames at
|
||||||
|
decay 0.93). A light-feedback look seeks almost instantly; a heavy one still lands.
|
||||||
|
2. **Export is always sequential**, so it is exact everywhere by construction.
|
||||||
|
|
||||||
|
> **Correction, made during Phase 4.** The original plan claimed section-boundary seeks
|
||||||
|
> would be frame-exact with no warm-up, because layer state is re-seeded there. That is
|
||||||
|
> wrong: layer state is only half the story, and the compositor's feedback buffer is
|
||||||
|
> *global* — it carries straight across a boundary like any other frame. Making boundary
|
||||||
|
> seeks exact would mean clearing feedback at every transition, which trades a cheap
|
||||||
|
> warm-up for a visible flash on every scene change. Warm-up is the better trade, and it
|
||||||
|
> applies everywhere rather than only mid-section. Measured: converges to a 0.00000 mean
|
||||||
|
> difference. With feedback disabled, seeks are bit-exact anywhere, which is what proves
|
||||||
|
> nothing *else* in the pipeline is carrying state.
|
||||||
|
|
||||||
|
Documented consequence: scrubbing shows a *converged*, not bit-exact, image whenever
|
||||||
|
feedback is enabled — which is visually indistinguishable, and exact once feedback is off.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Export
|
||||||
|
|
||||||
|
Same engine, `Timeline` in fixed-step mode, rendering to an offscreen target at export
|
||||||
|
resolution.
|
||||||
|
|
||||||
|
- **Primary: WebCodecs `VideoEncoder`.** Hardware-accelerated H.264/VP9 straight from the
|
||||||
|
render target, muxed with the decoded audio. No ffmpeg.wasm payload, and fast.
|
||||||
|
- **Fallback: frame dump + ffmpeg.** A small vite dev-server middleware accepts POSTed frames
|
||||||
|
and pipes them to a local `ffmpeg`. Slower, trivially debuggable.
|
||||||
|
- Title card and fade in/out at the edges.
|
||||||
|
- Progress reporting and cancel, since a 4K render is minutes not seconds.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Validation tooling
|
||||||
|
|
||||||
|
Built early (most of it in Phases 0-1) because every later phase depends on it. This is the
|
||||||
|
difference between "seems fine" and "verified".
|
||||||
|
|
||||||
|
| Tool | What it catches |
|
||||||
|
|---|---|
|
||||||
|
| **Frame hash log** | Renders *N* frames headless, hashes each (FNV over `readPixels`), writes JSON. Diffing two runs is the determinism regression test. |
|
||||||
|
| **Click track** | Mixes an audible click on the detected beat grid into the track. You can *hear* whether tempo detection is right — far more reliable than watching visuals and guessing. |
|
||||||
|
| **Section timeline overlay** | Detected sections and labels drawn over the transport. Bad segmentation is obvious immediately. |
|
||||||
|
| **Feature scope** | Oscilloscope plot of selected features with playhead. Catches dead, constant, or saturated features — e.g. a band that's always 0 because the split was wrong. |
|
||||||
|
| **Dual-resolution diff** | Renders one frame at 720p and 4K, downsamples, diffs. Catches resolution dependence, which is otherwise silent until export looks wrong. |
|
||||||
|
| **Seed contact sheet** | Same frame across 16 seeds as a grid. Makes look-space collapse visible instantly — this is the primary defense against "sameness across tracks". |
|
||||||
|
| **Schema lint** | Parses each scene's shader for `uniform` declarations and cross-checks against its `params` block, both directions. Catches typos and orphans; the thing that keeps a 30-scene library maintainable. |
|
||||||
|
| **Param range sweep** | Renders each param at several points across its declared range, asserts no NaN, no all-black, no all-white frames. |
|
||||||
|
| **Perf HUD** | Frame time, GPU time per layer, frame index, active section, live param values. |
|
||||||
|
| **Flash-rate meter** | Counts light-dark cycles per second against the WCAG 2.3.1 / Harding ceiling of three. Added during Phase 5 and not in the original plan — this generates beat-reactive video for publication, an unsupervised generator finds unsafe states on its own, and nobody watches every frame of every export. It caught a scene running at 7-8 flashes/s at *every* output resolution. |
|
||||||
|
| **First-render check** | Renders a frame in a fresh engine and compares against the same frame rendered later. Catches anything that is correct on repeat but wrong the first time — invisible to fresh-vs-fresh comparison, and wrong in every export, which renders each frame exactly once. Caught two separate bugs (a reused feature row, and unlinked shader programs). |
|
||||||
|
|
||||||
|
### The track battery
|
||||||
|
|
||||||
|
A fixed set of **6 of your own tracks** spanning ambient → mid-tempo → hard EDM, checked in
|
||||||
|
as the standing regression corpus (paths only, not the audio). Every phase gate below is
|
||||||
|
evaluated against the same battery, so improvements and regressions are comparable across
|
||||||
|
time. Pick these before Phase 1 — the analysis work is much easier to judge against real
|
||||||
|
material you know well.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Phases, with gates
|
||||||
|
|
||||||
|
Each phase has an explicit gate. Don't start the next phase until the gate passes; these are
|
||||||
|
cheap to check now and expensive to retrofit later.
|
||||||
|
|
||||||
|
### Phase 0 — skeleton and determinism spine
|
||||||
|
`Timeline`, `Renderer`, `Compositor` with a single layer, seeded RNG, fullscreen quad. One
|
||||||
|
ported shader with the LED grid stripped. Frame hash tool.
|
||||||
|
|
||||||
|
**Gate:**
|
||||||
|
- 300 frames rendered twice on the same machine → **bit-identical hashes**.
|
||||||
|
- Artificially stalling the render loop (simulated dropped frames) changes nothing in the
|
||||||
|
output hashes — proves `dt` is truly fixed.
|
||||||
|
- Dual-resolution diff on 5 sample frames passes the perceptual threshold.
|
||||||
|
- Grep gate: zero occurrences of `Math.random`, `performance.now`, `Date.now` under
|
||||||
|
`src/scenes/` and `src/engine/` outside of explicitly allowed spots.
|
||||||
|
|
||||||
|
### Phase 1 — offline audio pipeline
|
||||||
|
`decode`, `FeatureTrack`, tempo and beat grid. The ported scene reads features by frame index
|
||||||
|
instead of a live analyser. Click track, feature scope.
|
||||||
|
|
||||||
|
**Gate:**
|
||||||
|
- **Click track lines up by ear on all 6 battery tracks.** If tempo is wrong anywhere, stop
|
||||||
|
and fix it here — every downstream timing artifact traces back to this.
|
||||||
|
- Feature sanity: no NaN or Inf anywhere in the table; every feature's realized range covers
|
||||||
|
a meaningful span (flag any that are constant or pinned at an extreme).
|
||||||
|
- Realtime-driven playback and fixed-step render of frames 1000-1100 → identical hashes.
|
||||||
|
- Analysis of a 6-minute track completes in under ~3s.
|
||||||
|
- Seeking to frame *N* after warm-up matches sequential playback to *N* within threshold.
|
||||||
|
|
||||||
|
### Phase 2 — parameter schema, automatic binding, generated UI
|
||||||
|
Remaining four shaders ported with schemas. Schema lint, param range sweep.
|
||||||
|
|
||||||
|
**Gate:**
|
||||||
|
- Schema lint clean on every scene, both directions.
|
||||||
|
- Range sweep clean: no scene produces NaN, black or white frames anywhere in its declared
|
||||||
|
ranges.
|
||||||
|
- Every declared param appears in the generated UI, edits take visible effect, and values
|
||||||
|
round-trip through save/load unchanged.
|
||||||
|
- Ported scenes reviewed side-by-side against the `party-stage` originals — confirm nothing
|
||||||
|
was lost in the port.
|
||||||
|
|
||||||
|
### Phase 3 — look generation ("A" complete)
|
||||||
|
`LookGenerator`, `PaletteSource`, content-derived seed. Seed contact sheet.
|
||||||
|
|
||||||
|
**Gate:**
|
||||||
|
- Same file loaded twice → identical seed → identical look spec (JSON compare).
|
||||||
|
- **Seed contact sheet shows genuine spread**: mean pairwise perceptual distance across 16
|
||||||
|
seeds above a set threshold. This is the "sameness" gate — a collapsed look space fails here.
|
||||||
|
- All 6 battery tracks produce meaningfully different look specs (different scene selections,
|
||||||
|
distinguishable palettes).
|
||||||
|
- Palettes pass a contrast/luminance-spread check — no muddy, low-separation sets.
|
||||||
|
- Every scene survives every generated param set across the battery without crashing.
|
||||||
|
|
||||||
|
*At this gate the tool is genuinely useful: drop a track in, get an uploadable video with no
|
||||||
|
input.*
|
||||||
|
|
||||||
|
### Phase 4 — segmentation and arc driver
|
||||||
|
Sections, classification, cross-fades, lookahead ramps, intra-section drift.
|
||||||
|
|
||||||
|
**Gate:**
|
||||||
|
- Hand-label section boundaries on the battery; detected boundaries score an acceptable F1
|
||||||
|
within ±2s tolerance. Record the number — it's the baseline for future tuning.
|
||||||
|
- Transition check: frame-to-frame perceptual delta across the whole track, flagging spikes.
|
||||||
|
No black frames or pops at boundaries except where intended.
|
||||||
|
- Param traces logged over a build section show monotonic ramps into the following drop —
|
||||||
|
proves lookahead is actually wired, not just present in the data.
|
||||||
|
- **Watch all 6 battery tracks end to end.** Unavoidable and not substitutable. Budget the
|
||||||
|
time; the failure mode this catches — subtle monotony — is invisible to every automated check.
|
||||||
|
|
||||||
|
### Phase 5 — compositing depth ("C" complete)
|
||||||
|
Multi-layer, blend modes, feedback, post chain, 3D particle layers.
|
||||||
|
|
||||||
|
**Gate:**
|
||||||
|
- Full stack holds 60fps at preview resolution; per-layer GPU cost recorded against a budget.
|
||||||
|
- Feedback stability: 10,000-frame run neither saturates to white nor decays to black.
|
||||||
|
- Determinism still passes with feedback and particles active — this is where deterministic
|
||||||
|
buffer initialization gets tested for real.
|
||||||
|
- Every layer renders correctly in isolation when soloed.
|
||||||
|
|
||||||
|
### Phase 6 — export
|
||||||
|
WebCodecs path, title card, fades, progress and cancel.
|
||||||
|
|
||||||
|
**Gate:**
|
||||||
|
- **Preview/export parity**: hashes of exported frames match preview-rendered frames for the
|
||||||
|
same range.
|
||||||
|
- A/V sync measured at start, middle and end of a 6-minute export — click track against video
|
||||||
|
frames, drift within one frame.
|
||||||
|
- `ffprobe` frame count and duration exactly as expected; no dropped or duplicated frames.
|
||||||
|
- Exported audio matches the source.
|
||||||
|
- Plays correctly in VLC and in a browser, **and survives a real upload** to the video site
|
||||||
|
you actually publish on. Test this once, early, with a short file — container quirks are
|
||||||
|
much cheaper to find now than after a 4K render.
|
||||||
|
|
||||||
|
### Phase 7 — grow the library
|
||||||
|
Toward six families. Each new scene is a shader plus a schema block.
|
||||||
|
|
||||||
|
**Per-scene checklist** (the gate is per scene, not per phase):
|
||||||
|
- Schema lint clean; range sweep clean; determinism hashes stable.
|
||||||
|
- Within GPU budget at 4K.
|
||||||
|
- Sits correctly in its declared family — reviewed in both a quiet and a loud section.
|
||||||
|
- Library-wide regression contact sheet at a fixed seed, diffed against last known good, so
|
||||||
|
shared-code changes can't silently break existing scenes.
|
||||||
|
|
||||||
|
### Phase 8 — shots
|
||||||
|
Not in the original plan. It exists because the manual gate in Phase 4 caught exactly what it
|
||||||
|
was written to catch: watching whole tracks, long stretches went by without the image
|
||||||
|
changing. Sections are the song's *stages*, and a stage can run ninety seconds; one scene held
|
||||||
|
for ninety seconds reads as a still image with a wobble on it, however reactive the wobble is.
|
||||||
|
|
||||||
|
So a third level sits between section and frame. Each section KIND gets a roster of three or
|
||||||
|
four **stage visuals** instead of one scene, and each section is cut into **shots** that
|
||||||
|
rotate between them on phrase lines — four to eight bars in a drop, eight to sixteen in an
|
||||||
|
intro, never longer than twenty-two seconds. The roster stays per kind, so all of a track's
|
||||||
|
drops still cut between the same visuals and the video keeps its identity; the first entry is
|
||||||
|
the anchor, opens every section of that kind, and the rotation keeps returning to it. When a
|
||||||
|
companion is due it is the least recently shown one, so a long section reaches its whole
|
||||||
|
roster instead of ping-ponging between two images.
|
||||||
|
|
||||||
|
Mechanically this is one change: the arc driver stopped working in sections and started
|
||||||
|
working in **cues**, one per shot, so a shot cut and a section change take the same code path
|
||||||
|
and differ only in transition length. The default transition is a slow dissolve — two bars on
|
||||||
|
calm material, one on loud, capped at 40% of the shot. A straight cut is reserved for genuinely
|
||||||
|
high-energy sections: below the energy threshold nothing ever cuts, because on calm material a
|
||||||
|
cut reads as a glitch rather than as an edit.
|
||||||
|
|
||||||
|
**Gate:**
|
||||||
|
- No image held longer than the ceiling, and none shorter than the floor. Both measured across
|
||||||
|
several seeds; the first is the complaint that started the phase, expressed as a number.
|
||||||
|
- Intra-section cuts land within a quarter-bar of a downbeat.
|
||||||
|
- Identity holds: a kind's roster is stable across its sections, the anchor opens each section
|
||||||
|
and stays within one showing of the most-shown visual, and no variant repeats back to back.
|
||||||
|
- A section with four or more shots reaches at least three distinct visuals — otherwise the
|
||||||
|
rotation has collapsed back to A/B and the roster is decoration.
|
||||||
|
- Dissolves outnumber cuts, and no section below the energy threshold cuts at all.
|
||||||
|
- No black frame or discontinuity at a cut; the flash-rate sweep runs per shot rather than per
|
||||||
|
section, so the visuals that only appear mid-section are measured too.
|
||||||
|
- **Watch the battery again.** The phase exists because of a manual finding and the fix is a
|
||||||
|
pacing judgement, which no delta threshold can make.
|
||||||
|
|
||||||
|
### Phase 9 — production design
|
||||||
|
Also not in the original plan, and for the same reason as Phase 8: watching real tracks. With
|
||||||
|
cuts every fifteen seconds the next problem became obvious — the images being cut between had
|
||||||
|
nothing in common but the palette. That is a slideshow, not a music video.
|
||||||
|
|
||||||
|
What a music video actually shares across its shots is a location, a cast, a camera operator
|
||||||
|
and an art direction. So each track now generates a **personality** in four traits, once, off
|
||||||
|
the look seed:
|
||||||
|
|
||||||
|
| trait | what it fixes | how a scene expresses it |
|
||||||
|
|---|---|---|
|
||||||
|
| `shape` | the cast | a signature form — round, or an n-gon at a tilt — stamped wherever a scene draws elements |
|
||||||
|
| `camera` | the operator | one slow returning pan, sway, roll and bar-locked breath, applied to the coordinate a scene works in |
|
||||||
|
| `space` | the location | a shared horizon height, depth falloff and background wash |
|
||||||
|
| `style` | the art direction | line weight, edge softness, surface grain, fold count |
|
||||||
|
|
||||||
|
The traits reach shaders as ordinary uniforms plus four helper functions in the contract
|
||||||
|
(`sigShape`, `sigCamera`, `sigAir`, `sigEdge`/`sigGrain`), so a scene honours a trait in its own
|
||||||
|
way — the rings of Classic Wave become hexagonal, Metaballs merge as hexagons, Floating
|
||||||
|
Geometry stops choosing between a box and a circle because the production already decided.
|
||||||
|
|
||||||
|
The part that makes it a design rather than a filter: each scene DECLARES which traits it
|
||||||
|
honours, each track is built on one or two of them, and **a scene that does not honour all of
|
||||||
|
them is not cast in that track**. The library shrinks per track on purpose. A scene with no way
|
||||||
|
to draw a hexagon should not appear in the hexagon video.
|
||||||
|
|
||||||
|
**Gate:**
|
||||||
|
- Personality reproduces exactly from the seed, and differs between seeds.
|
||||||
|
- Every trait has at least six scenes, or a track built on it could not fill its rosters.
|
||||||
|
- No scene is ever cast in a track it does not honour, and no signature starves a track below
|
||||||
|
three distinct scenes (the fallback to a one-trait signature exists for this).
|
||||||
|
- **Every declared trait visibly changes the scene that declares it.** The lint proves a scene
|
||||||
|
mentions a trait; only rendering proves it matters. Measured as a frame delta against a
|
||||||
|
one-LSB floor — the tolerance the determinism checks already call noise.
|
||||||
|
- Changing the personality moves every scene in a track's cast, not just one.
|
||||||
|
- A layer with no personality renders exactly what it rendered before the phase existed, which
|
||||||
|
is what keeps every earlier sweep and regression valid.
|
||||||
|
- **Watch the battery.** The target is that a viewer could name the through-line on a second
|
||||||
|
viewing — not the first. No metric expresses that.
|
||||||
|
|
||||||
|
### Phase 10 — variety
|
||||||
|
The third finding from watching real tracks, after Phase 8 (too few cuts) and Phase 9 (no
|
||||||
|
through-line): **the same scene cast in two different videos looked like the same footage
|
||||||
|
twice.** Section bias is nearly identical between two tracks' drops, so both sampled their
|
||||||
|
parameters around the same centre, and the library's own averageness did the rest.
|
||||||
|
|
||||||
|
Three answers, none of them a new scene:
|
||||||
|
|
||||||
|
- **Temperament.** A per-track hand on every parameter dial — intensity, pace, detail, and an
|
||||||
|
*extremity* that decides how far toward the ends of a range the track is willing to sample.
|
||||||
|
Bias comes from the section and is shared; temperament comes from the track and is not.
|
||||||
|
- **Overlays.** Sometimes a second full scene is composited over the shot at partial opacity,
|
||||||
|
from a different family, in a blend that preserves what is underneath. Not always — a stack
|
||||||
|
that always doubled up would read as permanently cluttered rather than as occasionally
|
||||||
|
layered.
|
||||||
|
- **A wider palette.** Hue derives from **spectral tilt** — the log ratio of treble to body —
|
||||||
|
rather than the centroid or a plain body fraction. Both of those collapse: the centroid is a
|
||||||
|
number most masters sit in the middle of, and low frequencies carry most of the energy in
|
||||||
|
all music, so the plain fraction read 0.98-1.00 for everything and four different battery
|
||||||
|
tracks came out within 0.02 of each other. The ratio is multiplicative, so its logarithm is
|
||||||
|
what spreads. Plus: both ways round the wheel (violet, magenta and pink were previously
|
||||||
|
unreachable by construction), four new schemes, and seeded chroma profile and lightness
|
||||||
|
curve.
|
||||||
|
|
||||||
|
The library also grew to **36 scenes, six per family** — depth matters more than it looks,
|
||||||
|
because the Phase 9 casting rule means the pool a given track draws from is smaller than the
|
||||||
|
library.
|
||||||
|
|
||||||
|
**Gate:**
|
||||||
|
- Temperaments spread across a battery rather than collapsing to one value.
|
||||||
|
- One scene rendered under two tracks' parameters differs — measured RELATIVE to how much
|
||||||
|
image there is, because most scenes are mostly dark and an absolute frame distance scores
|
||||||
|
two genuinely different sparse renders as nearly identical.
|
||||||
|
- Overlays occur on 5-55% of stacks, never at `normal` blend, never above 0.6 opacity.
|
||||||
|
- Every sixth of the colour wheel is reachable across a sampled population, and two tracks
|
||||||
|
that sound different do not get the same palette.
|
||||||
|
|
||||||
|
### Tooling, added with Phase 10
|
||||||
|
Adding a scene was mostly boilerplate and round-trips, which is expensive in both senses.
|
||||||
|
|
||||||
|
- `npm run new:scene -- "Name" --family=… --traits=…` writes the module, registers it, and
|
||||||
|
leaves a skeleton that already passes every gate (with name-derived constants, so two
|
||||||
|
skeletons are not identical to each other).
|
||||||
|
- The lint grew the rules that used to need a GPU to catch: a **dead camera**
|
||||||
|
(`p = sigCamera(p)` and then nothing reads `p` — a real scene shipped like that and the
|
||||||
|
Phase 9 render gate measured its response at exactly zero), `prev()` with no base image, and
|
||||||
|
large loops with no early break (with a `// lint: fixed-cost` opt-out).
|
||||||
|
- `checks.html?scene=Name` runs the per-scene acceptance battery for ONE scene: ten lines and
|
||||||
|
a verdict, instead of rendering the whole library to find out whether one shader is alive.
|
||||||
|
- `.claude/skills/build-visualizer/` is the same procedure as a repo skill.
|
||||||
|
|
||||||
|
### Phase 10b — grain as a treatment, tempo as a governor
|
||||||
|
|
||||||
|
Watching the library again surfaced two things no gate was asking about.
|
||||||
|
|
||||||
|
**Grain was in every video.** It was added twice unconditionally — every scene called
|
||||||
|
`sigGrain`, and the grade added its own on top — so the only thing that varied between two
|
||||||
|
tracks was how much. That makes grain the renderer's fingerprint rather than a decision about
|
||||||
|
one video. It is now DESCRIBED (`look/grain.js`): a mode (`off` / `constant` / `swell` /
|
||||||
|
`sections` / `transient`), a cell size in pixels, a refresh rate in frames, a mask (uniform,
|
||||||
|
shadows, highlights, edges, bands) and a chroma amount. Roughly 45% of tracks get none at all;
|
||||||
|
the non-constant modes have a per-frame envelope computed in `Show._postAt` from frame and
|
||||||
|
features only, so preview and export still agree. Scene-side grain is gated the same way, and
|
||||||
|
a module can decline it outright with `texture: 0` — crisp line work should stay crisp.
|
||||||
|
|
||||||
|
**Slow songs got fast scenes.** `motion` bias was mostly section energy with tempo as a small
|
||||||
|
correction, so a 70bpm track's drop asked for nearly as much speed as a 150bpm one. Motion is
|
||||||
|
now tempo-dominated, and every `rate: true` param is additionally scaled by a per-track
|
||||||
|
`rateScale`, so absolute animation speed follows the song rather than only the sampled
|
||||||
|
position in a range.
|
||||||
|
|
||||||
|
Parameter sampling also commits harder: `extremity` starts at 0.5 rather than 0.25 and shapes
|
||||||
|
the draw more aggressively, because a range is the scene author's statement of what the scene
|
||||||
|
can survive and a library that samples the middle of every range shows every scene's default.
|
||||||
|
|
||||||
|
**Gate:** a quarter to two thirds of tracks have no grain and at least four modes appear;
|
||||||
|
thirty grainy tracks produce mostly-distinct treatments; every time-varying grain reaches both
|
||||||
|
zero and full; a `texture: 0` scene gets nothing even from a maximally gritty track; and a
|
||||||
|
84bpm track's rate params sample materially below a 148bpm track's.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. Detachment from `party-stage`
|
||||||
|
|
||||||
|
Fork, copy what's useful, then **detach completely**. This matches how the rest of the repo
|
||||||
|
works — each generator stands alone — and here it's also technically right: `party-stage` is
|
||||||
|
causal and realtime by design, this is two-pass and offline, and a shared module would serve
|
||||||
|
neither well.
|
||||||
|
|
||||||
|
**Copied, then owned outright** (edit freely, no upstream obligation):
|
||||||
|
- the mulberry32 seeded PRNG
|
||||||
|
- `MediaStorage`, the IndexedDB track persistence
|
||||||
|
- config-UI patterns
|
||||||
|
- the postprocessing setup
|
||||||
|
- the five visualizer shaders
|
||||||
|
|
||||||
|
**Referenced as prior art, not copied**: `music-visualizer.js`. Its three-detector onset logic
|
||||||
|
and band splits are a good specification for *what to measure*; the offline analyzer computes
|
||||||
|
all of it better, so none of the code carries over.
|
||||||
|
|
||||||
|
**Deliberately not ported**: `SceneFeature` / `SceneFeatureManager`. `Layer` + `Compositor` is
|
||||||
|
the right shape for a layer stack, and retrofitting the old pattern would fight the design.
|
||||||
|
|
||||||
|
**Enforced**: zero imports crossing the directory boundary, in either direction. A grep for
|
||||||
|
`party-stage` under `flow-state/src/` returns nothing. `party-stage` keeps working exactly as
|
||||||
|
it does today and is never touched by this project.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. Risks
|
||||||
|
|
||||||
|
- **Segmentation quality** is the fuzziest component. Mitigation: energy-novelty heuristics
|
||||||
|
only, tuned by ear against the battery; a misjudged section means a scene change in a
|
||||||
|
slightly odd place, not a broken video. The Phase 4 F1 number keeps it honest.
|
||||||
|
- **Shader compile time** grows with the library. Mitigation: compile lazily, only the scenes
|
||||||
|
the look actually selected.
|
||||||
|
- **4K export speed.** Mitigation: WebCodecs, plus the fact that it's offline — slow is fine
|
||||||
|
as long as preview stays fast.
|
||||||
|
- **Resolution-independence discipline** is easy to violate silently. Mitigation: the
|
||||||
|
dual-resolution diff, run every phase gate, not just Phase 0.
|
||||||
|
- **Monotony can survive every automated check.** Only watching full tracks catches it. This
|
||||||
|
is why Phase 4's gate includes end-to-end viewing and why the battery exists — and it's the
|
||||||
|
risk most likely to be the one that actually bites.
|
||||||
124
flow-state/README.md
Normal file
124
flow-state/README.md
Normal file
@ -0,0 +1,124 @@
|
|||||||
|
# flow-state
|
||||||
|
|
||||||
|
Ambient/EDM music video generator. Drop in a track, get a full-length, non-story,
|
||||||
|
music-reactive video. No sourced footage — every frame is generated, and the whole
|
||||||
|
look is derived from the audio.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm install
|
||||||
|
npm run dev # http://localhost:5180
|
||||||
|
```
|
||||||
|
|
||||||
|
Drop an audio file onto the page (mp3, flac, wav, ogg). Analysis takes a second or
|
||||||
|
two, then the video is ready to preview and export.
|
||||||
|
|
||||||
|
## How it works
|
||||||
|
|
||||||
|
The track is decoded and analysed **before the first frame renders**, into a table
|
||||||
|
with one row per video frame: band energies, onset flux, spectral centroid and
|
||||||
|
flatness, a phase-locked beat grid, section boundaries, and lookahead fields.
|
||||||
|
|
||||||
|
Nothing reads a live `AnalyserNode`. Realtime preview maps `audio.currentTime` to a
|
||||||
|
frame index; export counts frames. Both read the same rows, so **what you preview is
|
||||||
|
what you export** — the exporter has no render path of its own.
|
||||||
|
|
||||||
|
Analysing the whole track up front also buys the thing a causal analyser cannot do:
|
||||||
|
a build can *anticipate* its drop and arrive at the transition already at full
|
||||||
|
tension, instead of reacting once the drop has landed.
|
||||||
|
|
||||||
|
## Working with it
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| `space` | play / pause |
|
||||||
|
| `←` `→` | previous / next section boundary |
|
||||||
|
| `L` | loop the current section |
|
||||||
|
| `D` | debug HUD |
|
||||||
|
| `O` | toggle the corner title plate |
|
||||||
|
| `,` `.` | step one frame |
|
||||||
|
|
||||||
|
**test render** exports 20 seconds around the playhead at full export quality. Use
|
||||||
|
it before committing to a full render.
|
||||||
|
|
||||||
|
**reroll** re-seeds the whole track; **reroll section** changes only the section
|
||||||
|
under the playhead; **lock** protects a section from further rerolls. Every
|
||||||
|
parameter the generator chose is exposed under the *scene* tab and can be edited
|
||||||
|
live.
|
||||||
|
|
||||||
|
The **click track** button (look tab) mixes an audible click onto the detected beat
|
||||||
|
grid. If the clicks don't sit on the beat, tempo detection is wrong and everything
|
||||||
|
downstream inherits it — check this first when a track looks off.
|
||||||
|
|
||||||
|
## Checks
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm test # audio pipeline against synthetic ground truth
|
||||||
|
npm run lint:scenes # determinism grep + scene schema/shader agreement
|
||||||
|
```
|
||||||
|
|
||||||
|
`http://localhost:5180/checks.html` runs the GPU gates for every phase. Add
|
||||||
|
`?slow=1` for the full suite, `?phase=5` for one phase.
|
||||||
|
|
||||||
|
## Adding a scene
|
||||||
|
|
||||||
|
Quick walkthrough: `HOWTO-visualizers.md`.
|
||||||
|
|
||||||
|
A scene is a shader plus a params block. Everything else — uniform binding, UI
|
||||||
|
controls, seeded per-track sampling, arc automation — is derived from the schema.
|
||||||
|
|
||||||
|
```js
|
||||||
|
export const myScene = {
|
||||||
|
name: 'My Scene',
|
||||||
|
family: 'organic', // flow organic minimal structural geometric glitch
|
||||||
|
kind: 'fragment',
|
||||||
|
params: {
|
||||||
|
density: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_density', bias: 'density' },
|
||||||
|
speed: { type: 'float', range: [0.1, 2], default: 0.5, uniform: 'u_speed', rate: true },
|
||||||
|
palette: { type: 'palette', count: 4 },
|
||||||
|
},
|
||||||
|
reactive: {
|
||||||
|
density: { feature: 'bandLow', amount: 0.3 },
|
||||||
|
},
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
return vec4(palRamp(fbm(p * 4.0 + t, 4)), 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
Register it in `src/scenes/registry.js`, then run `npm run lint:scenes` and the
|
||||||
|
Phase 7 checks. Three rules the linter enforces, each of which has already caused a
|
||||||
|
real bug here:
|
||||||
|
|
||||||
|
- **Anything multiplying `u_time` must be `rate: true`.** Phase is `elapsed × rate`,
|
||||||
|
so modulating a rate jumps the phase by `elapsed × delta` — a minute in, a small
|
||||||
|
wobble throws the image several whole units between frames. It measured as
|
||||||
|
strobing at twice the accessibility limit.
|
||||||
|
- **Don't reuse a contract uniform name** (`u_width`, `u_time`, `u_seed`, …). It's a
|
||||||
|
GLSL redefinition error, and the only symptom is a black frame.
|
||||||
|
- **Use `pal()` / `palRamp()`**, not hardcoded colours, or the look generator can't
|
||||||
|
recolour the scene.
|
||||||
|
|
||||||
|
Scenes that composite over a background rather than being one declare
|
||||||
|
`role: 'accent'`.
|
||||||
|
|
||||||
|
## Layout
|
||||||
|
|
||||||
|
```
|
||||||
|
src/
|
||||||
|
audio/ decode, STFT analysis, tempo, segmentation, FeatureTrack, click track
|
||||||
|
engine/ Timeline, Renderer, Layer, Compositor, passes, seeded rng, flash safety
|
||||||
|
look/ palette (OKLCH), LookGenerator, ArcDriver
|
||||||
|
params/ declarative schema, validation, serialisation
|
||||||
|
scenes/ the library — shader/ and layers3d/
|
||||||
|
export/ WebCodecs exporter
|
||||||
|
ui/ preview surface
|
||||||
|
checks/ phase gates, run from checks.html
|
||||||
|
```
|
||||||
|
|
||||||
|
`PLAN.md` has the full design and the reasoning behind each gate.
|
||||||
|
|
||||||
|
Forked from `party-stage` by copying what was useful, then fully detached — there
|
||||||
|
are no imports across the directory boundary in either direction.
|
||||||
199
flow-state/SIDE-QUESTS.md
Normal file
199
flow-state/SIDE-QUESTS.md
Normal file
@ -0,0 +1,199 @@
|
|||||||
|
# Side quests
|
||||||
|
|
||||||
|
Three follow-ups left behind by Epic 2. None of them blocks anything; all three are debts that
|
||||||
|
were taken on knowingly, with the reason recorded at the time. They are written up here so that
|
||||||
|
picking one up does not require reconstructing why it exists.
|
||||||
|
|
||||||
|
Each is independent. Each has an existing gate that will tell you when it is done.
|
||||||
|
|
||||||
|
| # | quest | size | what it unblocks |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 1 | Four scenes express `style` only through grain | small | lets them opt out of grain entirely |
|
||||||
|
| 2 | Three scenes have no structure a parameter can move | medium, per-scene | the open half of EPIC-2 §3.4 |
|
||||||
|
| 3 | Horizon Lines determinism flake | small, investigative | a gate that currently depends on GPU load |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Four scenes whose only art direction is grain
|
||||||
|
|
||||||
|
### What is wrong
|
||||||
|
|
||||||
|
`Classic Wave`, `Silk Ribbon`, `Kaleido Tunnel` and `Slow Orb` all declare the `style`
|
||||||
|
personality trait. A scene that declares a trait is promising it visibly honours it — that is a
|
||||||
|
contract, not a hint, because the disqualification rule in `Personality.js` is the only thing
|
||||||
|
keeping off-design scenes out of a track.
|
||||||
|
|
||||||
|
These four honour it with exactly one line:
|
||||||
|
|
||||||
|
```glsl
|
||||||
|
col += sigGrain(uv);
|
||||||
|
```
|
||||||
|
|
||||||
|
That is the whole of their art direction. So when the grain work (commit `14d1204`) let a scene
|
||||||
|
decline the track's surface grain with `texture: 0`, these four could not take it: Phase 9's
|
||||||
|
"every declared trait visibly changes the scene that declares it" measured their style response
|
||||||
|
at exactly **0**. They were parked at `texture: 0.35` as a stopgap, which is the wrong shape of
|
||||||
|
fix — it keeps grain on scenes that would look better clean, purely so a gate stays green.
|
||||||
|
|
||||||
|
### Why it matters
|
||||||
|
|
||||||
|
The user's original complaint was that grain was everywhere. Three of these four are the
|
||||||
|
library's cleanest, softest scenes — a drifting orb, a silk ribbon, a ring wave. They are
|
||||||
|
exactly the scenes that should be able to say "not on me".
|
||||||
|
|
||||||
|
### The approach
|
||||||
|
|
||||||
|
Each already has a parameter that *is* an edge-weight or a softness knob. The work is routing
|
||||||
|
`u_sigLine` and `u_sigSoft` into it, so the track's hand shows in the line quality rather than
|
||||||
|
in the dirt. Concretely, per scene:
|
||||||
|
|
||||||
|
- **Classic Wave** — `src/scenes/shader/classic-wave.js:50` already has
|
||||||
|
`v = mix(v, smoothstep(0.2, 0.8, v), u_softness)`. Widen or narrow that smoothstep by
|
||||||
|
`u_sigSoft`, and let `u_sigLine` drive how hard the ring crests read.
|
||||||
|
- **Silk Ribbon** — `:61` is `exp(-best * best / (u_thickness * u_thickness))`. The strand width
|
||||||
|
is `u_thickness` (range `0.008..0.08`); scale it by `u_sigLine`, and put `u_sigSoft` on the
|
||||||
|
falloff exponent so a soft-handed track gets a haze and a sharp one gets a filament.
|
||||||
|
- **Kaleido Tunnel** — draws its grid with `smoothstep(0.42, 0.0, ringLines)`. That constant is
|
||||||
|
a line weight with no name. Drive the threshold from `u_sigLine` and the smoothstep width from
|
||||||
|
`u_sigSoft`.
|
||||||
|
- **Slow Orb** — `:47` is `smoothstep(u_softness * 0.5, -u_softness * 0.5, d)`, the body edge.
|
||||||
|
Fold `u_sigSoft` into it, and consider a `sigEdge(d)` rim so the shape reads in the track's
|
||||||
|
hand rather than only in its own.
|
||||||
|
|
||||||
|
Then set `texture: 0` on the ones that genuinely want to be clean — probably all four, but that
|
||||||
|
is a judgement call to make while looking at them, not now.
|
||||||
|
|
||||||
|
### Done when
|
||||||
|
|
||||||
|
- Phase 9's trait check reports a non-zero style delta for all four with `texture: 0`.
|
||||||
|
- `npm run lint:scenes` still finds style evidence in each shader (it greps for `sigEdge` /
|
||||||
|
`sigGrain` / `sigFolded` / `u_sigLine` / `u_sigSoft` / `u_sigTexture` / `u_sigFold`).
|
||||||
|
- The Phase 11 grain checks are unaffected — this quest must not put grain back.
|
||||||
|
|
||||||
|
### Do not
|
||||||
|
|
||||||
|
Do not solve this by dropping `style` from their `traits` arrays. That would pass every gate and
|
||||||
|
would shrink the library for every style-signature track, which is the opposite of the point.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Three scenes with no structure to move
|
||||||
|
|
||||||
|
### What is wrong
|
||||||
|
|
||||||
|
EPIC-2 §3.4 gave scenes a **slow axis**: a declared parameter walked one way across the whole
|
||||||
|
track, so the frame at four minutes is not the frame at thirty seconds. It works — Moiré Grid
|
||||||
|
3.93×, Gate Corridor 2.88×, Truchet Fold 1.40×, measured as axis-driven structural change
|
||||||
|
against what the scene does on its own.
|
||||||
|
|
||||||
|
For three scenes it cannot work, because **no parameter changes their structure**:
|
||||||
|
|
||||||
|
| scene | best parameter | ratio |
|
||||||
|
|---|---|---|
|
||||||
|
| `curl-flow.js` | `veins` | 1.31× |
|
||||||
|
| `signal-decay.js` | `traces` | 1.21× |
|
||||||
|
| `circuit-bloom.js` | `cells` | 1.10× |
|
||||||
|
|
||||||
|
Sweeping any parameter through most of its range barely moves the ten-second time-averaged
|
||||||
|
image. That is the technical statement of "it looks the same for five minutes": these scenes are
|
||||||
|
statistically identical everywhere and at all times. They churn — every pixel moving, nothing
|
||||||
|
developing — and the eye adapts in about two seconds.
|
||||||
|
|
||||||
|
### Why it matters
|
||||||
|
|
||||||
|
Churn is the failure that survives every other fix. Cut rhythm, colour movement and framing all
|
||||||
|
change *what surrounds* the image; none of them change the fact that the image itself has
|
||||||
|
nothing to find after two seconds.
|
||||||
|
|
||||||
|
### The approach
|
||||||
|
|
||||||
|
This is genuinely shader work and it is not mechanical — that assessment was wrong when EPIC-2
|
||||||
|
was first written and is corrected in §3.4. Each scene needs a persistent large-scale structure
|
||||||
|
that a parameter can then move. Sketches, one per scene, to be argued with rather than followed:
|
||||||
|
|
||||||
|
- **Curl Flow** is a curl-noise field at one scale, so it looks the same everywhere by
|
||||||
|
construction. Give it a low-frequency term that varies across the frame — a density gradient,
|
||||||
|
a region where the flow stalls, a dominant current — and put the parameter that positions or
|
||||||
|
scales *that* on the axis.
|
||||||
|
- **Signal Decay** already quantises to an era grid (`floor(u_barPhase * 8.0)`), which is good
|
||||||
|
rhythm and no structure: every trace is the same kind of trace. Consider making traces
|
||||||
|
differ from one another — a hierarchy, one dominant channel, a slow reordering — so the
|
||||||
|
arrangement can change over minutes.
|
||||||
|
- **Circuit Bloom** is a uniform grid of cells with per-cell hashes. The `reach` term already
|
||||||
|
gates growth by distance from the centre; that is the seed of a structure. Let the board grow,
|
||||||
|
reroute or densify along a slow axis instead of being fully populated from frame one.
|
||||||
|
|
||||||
|
Then declare the parameter with `slowAxis: true` in the params block.
|
||||||
|
|
||||||
|
### Done when
|
||||||
|
|
||||||
|
Phase 11's `a declared slow axis actually changes the scene` holds the scene at **1.25×**, and
|
||||||
|
its companion check still reads ~1.00× with the axis disabled. Run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
http://localhost:5180/checks.html?phase=11&slow=1
|
||||||
|
```
|
||||||
|
|
||||||
|
### Do not
|
||||||
|
|
||||||
|
Do not add `slowAxis: true` before the shader can back it up — the gate will fail, correctly.
|
||||||
|
And do not reach for more motion: these scenes already move too much. The missing thing is
|
||||||
|
structure, which is the opposite of motion.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. The Horizon Lines determinism flake
|
||||||
|
|
||||||
|
### What is wrong
|
||||||
|
|
||||||
|
Phase 7's `every scene is deterministic` renders each scene twice and requires a max channel
|
||||||
|
delta of ≤ 1/255. `Horizon Lines` reports **2**, but only sometimes:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
http://localhost:5180/checks.html?phase=7&slow=1 # FAILS — delta 2
|
||||||
|
http://localhost:5180/checks.html?slow=1 # PASSES — delta ≤ 1
|
||||||
|
```
|
||||||
|
|
||||||
|
Verified present on `f050eaa`, so it predates Epic 2. The difference is how much GPU work ran
|
||||||
|
before it: in a full suite, phases 0-6 warm the device first.
|
||||||
|
|
||||||
|
### Why it matters
|
||||||
|
|
||||||
|
Not for the image — 2/255 is invisible, and PLAN.md §1 already accepts 1/255 as the floor the
|
||||||
|
hardware offers. It matters because **a gate whose verdict depends on preceding load will
|
||||||
|
eventually stay green through a real determinism regression**, and determinism is the single
|
||||||
|
architectural constraint this whole project is built on. A flaky guard on the load-bearing
|
||||||
|
property is worse than no guard, because it is trusted.
|
||||||
|
|
||||||
|
### The approach
|
||||||
|
|
||||||
|
Two steps, in order.
|
||||||
|
|
||||||
|
**First, find out why this scene and not the other 41.** The suspects are in
|
||||||
|
`src/scenes/shader/horizon-lines.js`:
|
||||||
|
|
||||||
|
- The loop accumulates up to **40** line contributions into `col`. Summing many small terms is
|
||||||
|
where float ordering shows up, and no other scene in the library sums this many.
|
||||||
|
- `float line = smoothstep(u_thickness * (0.5 + u_sigLine), 0.0, d)` with `u_thickness` as low
|
||||||
|
as **0.002**. That is a razor-thin edge: a pixel sitting on it is decided by the last bit of
|
||||||
|
`d`, and `d = abs(p.y - y)` is a subtraction of two nearby numbers — cancellation, right where
|
||||||
|
the result is most sensitive.
|
||||||
|
- `exp(-d * 26.0)` on the same `d`.
|
||||||
|
|
||||||
|
If reformulating the line term stabilises it — accumulating in a saturated or normalised form,
|
||||||
|
or widening the minimum edge so no pixel sits exactly on a discontinuity — that is the real fix
|
||||||
|
and it costs nothing visually.
|
||||||
|
|
||||||
|
**Only if it cannot be made stable**, fix the check instead: either warm the GPU to a known
|
||||||
|
state before measuring, or raise the tolerance with the reason written into the comment that
|
||||||
|
already explains the 1-LSB rationale. Either is defensible; silently raising the number is not.
|
||||||
|
|
||||||
|
### Done when
|
||||||
|
|
||||||
|
`?phase=7&slow=1` passes in isolation, repeatedly, on a cold device.
|
||||||
|
|
||||||
|
### Do not
|
||||||
|
|
||||||
|
Do not raise the tolerance as the first move. The whole value of this check is that a real bug
|
||||||
|
scores in the tens or hundreds while the hardware floor is 1 — that separation is what makes it
|
||||||
|
worth having, and widening it without understanding the cause spends it for nothing.
|
||||||
42
flow-state/checks.html
Normal file
42
flow-state/checks.html
Normal file
@ -0,0 +1,42 @@
|
|||||||
|
<!DOCTYPE html>
|
||||||
|
<html lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8">
|
||||||
|
<title>flow-state · checks</title>
|
||||||
|
<style>
|
||||||
|
:root { color-scheme: dark; }
|
||||||
|
body {
|
||||||
|
margin: 0; padding: 24px;
|
||||||
|
background: #0b0d12; color: #d6dae3;
|
||||||
|
font: 13px/1.5 ui-monospace, SFMono-Regular, Menlo, monospace;
|
||||||
|
}
|
||||||
|
h1 { font-size: 15px; letter-spacing: .12em; text-transform: uppercase; color: #7d8698; margin: 0 0 4px; }
|
||||||
|
#summary { margin: 0 0 20px; font-size: 14px; }
|
||||||
|
#summary.ok { color: #4ade80; }
|
||||||
|
#summary.bad { color: #f87171; }
|
||||||
|
.row {
|
||||||
|
display: grid; grid-template-columns: 52px 34px 1fr 60px;
|
||||||
|
gap: 10px; align-items: baseline;
|
||||||
|
padding: 7px 10px; border-left: 3px solid transparent; margin-bottom: 2px;
|
||||||
|
background: #11141b;
|
||||||
|
}
|
||||||
|
.row.pass { border-color: #22c55e; }
|
||||||
|
.row.fail { border-color: #ef4444; background: #1c1214; }
|
||||||
|
.row.manual { border-color: #eab308; }
|
||||||
|
.badge { font-weight: 700; font-size: 11px; }
|
||||||
|
.pass .badge { color: #4ade80; }
|
||||||
|
.fail .badge { color: #f87171; }
|
||||||
|
.manual .badge { color: #facc15; }
|
||||||
|
.phase { color: #6b7280; }
|
||||||
|
.ms { color: #4b5563; text-align: right; }
|
||||||
|
.detail { grid-column: 3 / 5; color: #8b93a5; font-size: 12px; }
|
||||||
|
.detail:empty { display: none; }
|
||||||
|
</style>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<h1>flow-state — phase gates</h1>
|
||||||
|
<div id="summary">starting…</div>
|
||||||
|
<div id="results"></div>
|
||||||
|
<script type="module" src="/src/checks/main.js"></script>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
85
flow-state/index.html
Normal file
85
flow-state/index.html
Normal file
@ -0,0 +1,85 @@
|
|||||||
|
<!DOCTYPE html>
|
||||||
|
<html lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8">
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||||
|
<title>flow-state</title>
|
||||||
|
<link rel="stylesheet" href="/src/ui/style.css">
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<div id="app">
|
||||||
|
<div id="stage">
|
||||||
|
<canvas id="canvas"></canvas>
|
||||||
|
<div id="overlay">
|
||||||
|
<div id="dropzone">
|
||||||
|
<div class="dz-inner">
|
||||||
|
<div class="dz-title">flow-state</div>
|
||||||
|
<div class="dz-sub">drop an audio file, or click to choose</div>
|
||||||
|
<div class="dz-hint">mp3 · flac · wav · ogg</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div id="hud" hidden></div>
|
||||||
|
<div id="toast" hidden></div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div id="transport">
|
||||||
|
<div id="timeline">
|
||||||
|
<canvas id="timeline-canvas"></canvas>
|
||||||
|
</div>
|
||||||
|
<div id="controls">
|
||||||
|
<button id="btn-play" title="Play / pause (space)">▶</button>
|
||||||
|
<button id="btn-prev-section" title="Previous section (←)">⇤</button>
|
||||||
|
<button id="btn-next-section" title="Next section (→)">⇥</button>
|
||||||
|
<button id="btn-loop" title="Loop current section (L)">↻</button>
|
||||||
|
<span id="time-display">0:00 / 0:00</span>
|
||||||
|
<!-- The section readout is the row's only flexible item: it
|
||||||
|
absorbs all the free space and centres its own text, which
|
||||||
|
is what the two spacers used to do. They competed with it
|
||||||
|
for that space and squeezed it to nothing. -->
|
||||||
|
<span id="section-display"></span>
|
||||||
|
<label class="ctl">quality
|
||||||
|
<select id="sel-quality">
|
||||||
|
<option value="draft">draft</option>
|
||||||
|
<option value="full" selected>full</option>
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
<button id="btn-reroll" title="New seed for the whole track">reroll</button>
|
||||||
|
<button id="btn-reroll-section" title="New scene for this section only">reroll section</button>
|
||||||
|
<button id="btn-lock" title="Lock this section against rerolls">lock</button>
|
||||||
|
<button id="btn-hud" title="Toggle debug HUD (D)">hud</button>
|
||||||
|
<button id="btn-osd" title="Toggle the title plate in the corner (O)">osd</button>
|
||||||
|
<button id="btn-segment" title="Render 20s around the playhead at export quality">test render</button>
|
||||||
|
<button id="btn-export" class="primary" title="Export the full video">export</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<aside id="panel">
|
||||||
|
<div id="track-header">
|
||||||
|
<div class="th-top-row">
|
||||||
|
<div class="th-info">
|
||||||
|
<span id="th-label" class="th-label">no track loaded</span>
|
||||||
|
<span id="th-name" class="th-name" hidden>no track</span>
|
||||||
|
</div>
|
||||||
|
<button id="btn-change-track" class="th-btn" title="Choose or drop an audio file">choose track</button>
|
||||||
|
</div>
|
||||||
|
<div id="th-progress" class="th-progress" hidden>
|
||||||
|
<div id="th-step" class="th-step">decoding…</div>
|
||||||
|
<div class="an-bar"><div id="th-fill" class="an-fill"></div></div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div id="panel-tabs">
|
||||||
|
<button data-tab="look" class="active">look</button>
|
||||||
|
<button data-tab="scene">scene</button>
|
||||||
|
<button data-tab="post">post</button>
|
||||||
|
<button data-tab="export">export</button>
|
||||||
|
</div>
|
||||||
|
<div id="panel-body"></div>
|
||||||
|
</aside>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<input type="file" id="file-input" accept="audio/*" hidden>
|
||||||
|
<audio id="audio" hidden></audio>
|
||||||
|
<script type="module" src="/src/main.js"></script>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
1156
flow-state/package-lock.json
generated
Normal file
1156
flow-state/package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load Diff
22
flow-state/package.json
Normal file
22
flow-state/package.json
Normal file
@ -0,0 +1,22 @@
|
|||||||
|
{
|
||||||
|
"name": "flow-state",
|
||||||
|
"version": "1.0.0",
|
||||||
|
"description": "Ambient/EDM music video generator with a deterministic render core",
|
||||||
|
"type": "module",
|
||||||
|
"scripts": {
|
||||||
|
"dev": "vite",
|
||||||
|
"build": "vite build",
|
||||||
|
"preview": "vite preview",
|
||||||
|
"lint:scenes": "node tools/lint-scenes.js",
|
||||||
|
"test": "node --test test/*.test.js",
|
||||||
|
"new:scene": "node tools/new-scene.js"
|
||||||
|
},
|
||||||
|
"license": "ISC",
|
||||||
|
"devDependencies": {
|
||||||
|
"vite": "^7.2.2"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"mp4-muxer": "^5.2.2",
|
||||||
|
"three": "^0.181.1"
|
||||||
|
}
|
||||||
|
}
|
||||||
266
flow-state/src/Show.js
Normal file
266
flow-state/src/Show.js
Normal file
@ -0,0 +1,266 @@
|
|||||||
|
import { Engine } from './engine/Engine.js';
|
||||||
|
import { OSDLayer } from './engine/OSD.js';
|
||||||
|
import { FeatureTrack, featureProviderFor } from './audio/FeatureTrack.js';
|
||||||
|
import { decodeFile, monoSamples } from './audio/decode.js';
|
||||||
|
import { generateLook, rerollLook, rerollSection } from './look/LookGenerator.js';
|
||||||
|
import { ArcDriver } from './look/ArcDriver.js';
|
||||||
|
import { grainEnvelope } from './look/grain.js';
|
||||||
|
import { hashSamples } from './engine/rng.js';
|
||||||
|
|
||||||
|
const FADE_SECONDS = 1.5;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A loaded track plus its look, rendered.
|
||||||
|
*
|
||||||
|
* This is the object both the preview and the exporter drive, and the only way
|
||||||
|
* they can be guaranteed to agree: neither has its own render path. The preview
|
||||||
|
* differs from the export in output resolution and warm-up state, and in nothing
|
||||||
|
* else.
|
||||||
|
*/
|
||||||
|
export class Show {
|
||||||
|
constructor({ canvas = null, width = 1280, height = 720, fps = 60 } = {}) {
|
||||||
|
this.engine = new Engine({ canvas, width, height, fps });
|
||||||
|
this.fps = fps;
|
||||||
|
this.track = null;
|
||||||
|
this.look = null;
|
||||||
|
this.arc = null;
|
||||||
|
this.audioBuffer = null;
|
||||||
|
this.fileName = '';
|
||||||
|
this.osd = new OSDLayer(this.engine.renderer);
|
||||||
|
this.osdEnabled = true;
|
||||||
|
this._lastLayers = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
get ready() { return !!(this.track && this.look && this.arc); }
|
||||||
|
get duration() { return this.track ? this.track.duration : 0; }
|
||||||
|
get frameCount() { return this.track ? this.track.frameCount : 1; }
|
||||||
|
get timeline() { return this.engine.timeline; }
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Decode, analyse, and generate a look. `onProgress(stage, fraction)` is
|
||||||
|
* called throughout; analysis is CPU-bound and will block the main thread
|
||||||
|
* for a second or two on a long track.
|
||||||
|
*/
|
||||||
|
async load(file, onProgress = null) {
|
||||||
|
const report = (stage, p) => onProgress && onProgress(stage, p);
|
||||||
|
|
||||||
|
report('decoding', 0);
|
||||||
|
const audioBuffer = await decodeFile(file);
|
||||||
|
this.audioBuffer = audioBuffer;
|
||||||
|
this.fileName = file.name.replace(/\.[^/.]+$/, '');
|
||||||
|
|
||||||
|
// Yield so the progress UI can paint before the analysis pass blocks.
|
||||||
|
await new Promise((r) => setTimeout(r, 0));
|
||||||
|
|
||||||
|
this.track = FeatureTrack.fromAudioBuffer(audioBuffer, {
|
||||||
|
fps: this.fps,
|
||||||
|
onProgress: (stage, p) => report(stage, p),
|
||||||
|
});
|
||||||
|
|
||||||
|
report('look', 0.97);
|
||||||
|
const samples = monoSamples(audioBuffer);
|
||||||
|
this.setLook(generateLook(this.track, { samples }));
|
||||||
|
|
||||||
|
this.engine.timeline.setDuration(this.track.duration);
|
||||||
|
this.engine.setFeatureProvider(featureProviderFor(this.track));
|
||||||
|
|
||||||
|
report('scenes', 0.99);
|
||||||
|
this.prewarm();
|
||||||
|
|
||||||
|
report('ready', 1);
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Attach an already-analysed track. Used by the check harness and by tests. */
|
||||||
|
useTrack(track, look) {
|
||||||
|
this.track = track;
|
||||||
|
this.engine.timeline.setDuration(track.duration);
|
||||||
|
this.engine.setFeatureProvider(featureProviderFor(track));
|
||||||
|
this.setLook(look || generateLook(track, { seed: 1 }));
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
setLook(look) {
|
||||||
|
if (this.arc) this.arc.dispose();
|
||||||
|
this.look = look;
|
||||||
|
this.arc = new ArcDriver(look, this.track);
|
||||||
|
this.osd.setText(this.fileName, look.personality, look.palette);
|
||||||
|
this._lastLayers = null;
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
reroll(seed) {
|
||||||
|
this.setLook(rerollLook(this.look, this.track, seed));
|
||||||
|
this.prewarm();
|
||||||
|
}
|
||||||
|
|
||||||
|
rerollSection(index, salt) {
|
||||||
|
rerollSection(this.look, this.track, index, salt);
|
||||||
|
this.arc.invalidateSection(index);
|
||||||
|
this._lastLayers = null;
|
||||||
|
this.prewarm();
|
||||||
|
}
|
||||||
|
|
||||||
|
setPalette(palette) {
|
||||||
|
this.look.palette = palette;
|
||||||
|
this.arc.setPalette(palette);
|
||||||
|
this.osd.setPalette(palette);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Live param edit on a section's primary layer. */
|
||||||
|
setSectionParam(sectionIndex, name, value) {
|
||||||
|
const section = this.look.sections[sectionIndex];
|
||||||
|
if (!section) return;
|
||||||
|
section.layers[0].params[name] = value;
|
||||||
|
this._lastLayers = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
setSize(width, height) {
|
||||||
|
this.engine.setSize(width, height);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Fade in at the head and out at the tail; nothing starts or ends abruptly. */
|
||||||
|
_fadeAt(frame) {
|
||||||
|
const fadeFrames = FADE_SECONDS * this.fps;
|
||||||
|
const fromStart = frame;
|
||||||
|
const toEnd = this.frameCount - 1 - frame;
|
||||||
|
const a = Math.min(1, Math.max(0, fromStart / fadeFrames));
|
||||||
|
const b = Math.min(1, Math.max(0, toEnd / fadeFrames));
|
||||||
|
return Math.min(a, b);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The post settings for one frame.
|
||||||
|
*
|
||||||
|
* Everything in the grade is constant per track except grain, which has a
|
||||||
|
* time envelope: it can swell in and out, or belong to particular section
|
||||||
|
* kinds, or answer transients. See look/grain.js. The envelope is computed
|
||||||
|
* from frame and features only, so preview and export agree.
|
||||||
|
*/
|
||||||
|
_postAt(timeline, features) {
|
||||||
|
const grain = this.look.grain;
|
||||||
|
if (!grain || grain.mode === 'off') return this.look.post;
|
||||||
|
if (grain.mode === 'constant') return this.look.post;
|
||||||
|
|
||||||
|
const kind = this.track.sectionAt(timeline.frame)?.kind || '';
|
||||||
|
const env = grainEnvelope(grain, { time: timeline.time, sectionKind: kind, features });
|
||||||
|
|
||||||
|
// Reused object: renderFrame runs 60 times a second and setPost copies
|
||||||
|
// out of it anyway.
|
||||||
|
this._post = this._post || {};
|
||||||
|
Object.assign(this._post, this.look.post);
|
||||||
|
this._post.grain = grain.amount * env;
|
||||||
|
return this._post;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Toggle the title plate. Off removes it from the stack entirely. */
|
||||||
|
setOSDEnabled(enabled) {
|
||||||
|
this.osdEnabled = !!enabled;
|
||||||
|
this._lastLayers = null;
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Render one frame. Identical in preview and export — the only difference is
|
||||||
|
* the size of the target and whether the result is presented or encoded.
|
||||||
|
*/
|
||||||
|
renderFrame(frame) {
|
||||||
|
const timeline = this.engine.timeline;
|
||||||
|
timeline.seek(frame);
|
||||||
|
|
||||||
|
const features = this.track.at(timeline.frame);
|
||||||
|
const sceneLayers = this.arc.update(timeline.frame, features);
|
||||||
|
const layers = this.osdEnabled ? [...sceneLayers, this.osd] : sceneLayers;
|
||||||
|
|
||||||
|
if (this._lastLayers === null || this.arc.layersChanged(this._lastLayers)) {
|
||||||
|
this.engine.compositor.setLayers(layers);
|
||||||
|
this._lastLayers = layers.slice();
|
||||||
|
}
|
||||||
|
|
||||||
|
this.engine.compositor
|
||||||
|
.setPost(this._postAt(timeline, features))
|
||||||
|
.setFeedback(this.look.feedback);
|
||||||
|
this.engine.compositor.fade = this._fadeAt(timeline.frame);
|
||||||
|
|
||||||
|
return this.engine.compositor.render({ timeline, features });
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Build and compile every layer in the look up front.
|
||||||
|
*
|
||||||
|
* Without this the first frame of every shot pays for a shader link, which
|
||||||
|
* shows as a hitch exactly on the cut. One pass at load costs a few hundred
|
||||||
|
* milliseconds and removes all of them.
|
||||||
|
*/
|
||||||
|
prewarm() {
|
||||||
|
if (!this.arc) return this;
|
||||||
|
this.arc.prewarm();
|
||||||
|
this.engine.compositor.primeLayers([this.osd, ...this.arc.layerCache.values()]);
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** See Engine.prime — required before frame-exact rendering. */
|
||||||
|
prime(frame = 0) {
|
||||||
|
this.prewarm();
|
||||||
|
this.renderFrame(frame); // ensures the arc has built its layers
|
||||||
|
this.engine.prime(frame);
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Advance stateful layers so an arbitrary seek lands on converged state. */
|
||||||
|
warmUp(frame, warmupFrames = 120) {
|
||||||
|
const start = Math.max(0, frame - warmupFrames);
|
||||||
|
this.engine.compositor.reset();
|
||||||
|
for (let f = start; f < frame; f++) this.renderFrame(f);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Frames of warm-up needed for the feedback loop to converge.
|
||||||
|
*
|
||||||
|
* Feedback decays geometrically, so the residual after n frames is decay^n.
|
||||||
|
* Converging to 0.1% rather than 1% costs only ~50% more frames and takes the
|
||||||
|
* result from "close" to "indistinguishable" — measured, a 1% target still
|
||||||
|
* left a visible 0.015 mean difference at heavy settings.
|
||||||
|
*/
|
||||||
|
warmupFrames() {
|
||||||
|
const amount = this.look ? this.look.feedback.amount : 0;
|
||||||
|
if (amount <= 0.01) return 0;
|
||||||
|
const decay = Math.min(0.99, this.look.feedback.decay);
|
||||||
|
return Math.min(400, Math.ceil(Math.log(0.001) / Math.log(decay)));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Seek for review.
|
||||||
|
*
|
||||||
|
* NOTE, corrected from the original plan: a section boundary is NOT exact for
|
||||||
|
* free. Layer state is re-seeded there, but the compositor's feedback buffer
|
||||||
|
* is global and carries straight across the boundary, so a look with feedback
|
||||||
|
* enabled still needs warm-up wherever you land. Resetting feedback at
|
||||||
|
* boundaries would make seeks exact at the cost of a visible flash at every
|
||||||
|
* transition, which is a much worse trade. Warm-up is cheap; the flash is not.
|
||||||
|
*/
|
||||||
|
seek(frame, { warmup = true } = {}) {
|
||||||
|
const frames = warmup ? this.warmupFrames() : 0;
|
||||||
|
if (frames > 0) {
|
||||||
|
this.warmUp(frame, frames);
|
||||||
|
} else {
|
||||||
|
this.engine.compositor.reset();
|
||||||
|
this.engine.timeline.seek(frame);
|
||||||
|
}
|
||||||
|
return this.renderFrame(frame);
|
||||||
|
}
|
||||||
|
|
||||||
|
present(target) { this.engine.present(target); }
|
||||||
|
readPixels(target) { return this.engine.readPixels(target); }
|
||||||
|
hashFrame(frame) { return this.engine.hashCurrent(this.renderFrame(frame)); }
|
||||||
|
|
||||||
|
contentSeed() {
|
||||||
|
return this.audioBuffer ? hashSamples(monoSamples(this.audioBuffer)) : 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
dispose() {
|
||||||
|
if (this.arc) this.arc.dispose();
|
||||||
|
this.osd.dispose();
|
||||||
|
this.engine.dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
198
flow-state/src/audio/FeatureTrack.js
Normal file
198
flow-state/src/audio/FeatureTrack.js
Normal file
@ -0,0 +1,198 @@
|
|||||||
|
import { analyzeBuffer } from './analyze.js';
|
||||||
|
import { detectTempo, buildBeatTracks } from './tempo.js';
|
||||||
|
import { segment } from './segment.js';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The frame-indexed feature table. Everything visual reads from here and from
|
||||||
|
* nowhere else — no live AnalyserNode exists in this project.
|
||||||
|
*
|
||||||
|
* Realtime preview maps audio.currentTime to a frame index and reads row n.
|
||||||
|
* The exporter counts frames and reads row n. Same rows, same visuals, which is
|
||||||
|
* the whole basis of preview/export parity.
|
||||||
|
*/
|
||||||
|
export class FeatureTrack {
|
||||||
|
constructor(data) {
|
||||||
|
Object.assign(this, data);
|
||||||
|
// Reused row object: at() is called 60 times a second and there is no
|
||||||
|
// reason to allocate for it.
|
||||||
|
this._row = {};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The feature row for a frame, clamped to range.
|
||||||
|
*
|
||||||
|
* WARNING: the returned object is REUSED between calls — at() is called every
|
||||||
|
* frame and allocating for it is pointless. The consequence is that you must
|
||||||
|
* never call at() again while still holding a previous result, and in
|
||||||
|
* particular never inside a render pass that is using one. Index the typed
|
||||||
|
* arrays in `raw`/`tracks` directly for incidental lookups.
|
||||||
|
*/
|
||||||
|
at(frame) {
|
||||||
|
const f = Math.max(0, Math.min(this.frameCount - 1, frame | 0));
|
||||||
|
const row = this._row;
|
||||||
|
const raw = this.raw;
|
||||||
|
|
||||||
|
row.rms = raw.rms[f];
|
||||||
|
row.loudness = raw.loudness[f];
|
||||||
|
row.bandSub = raw.bandSub[f];
|
||||||
|
row.bandLow = raw.bandLow[f];
|
||||||
|
row.bandMid = raw.bandMid[f];
|
||||||
|
row.bandHigh = raw.bandHigh[f];
|
||||||
|
row.bandAir = raw.bandAir[f];
|
||||||
|
row.flux = raw.flux[f];
|
||||||
|
row.centroid = raw.centroid[f];
|
||||||
|
row.flatness = raw.flatness[f];
|
||||||
|
row.width = raw.width[f];
|
||||||
|
|
||||||
|
row.beat = this.tracks.beat[f];
|
||||||
|
row.beatPhase = this.tracks.beatPhase[f];
|
||||||
|
row.barPhase = this.tracks.barPhase[f];
|
||||||
|
row.phrasePhase = this.tracks.phrasePhase[f];
|
||||||
|
|
||||||
|
row.sectionProgress = this.tracks.sectionProgress[f];
|
||||||
|
row.sectionEnergy = this.tracks.sectionEnergy[f];
|
||||||
|
row.buildSlope = this.tracks.buildSlope[f];
|
||||||
|
|
||||||
|
return row;
|
||||||
|
}
|
||||||
|
|
||||||
|
sectionIndexAt(frame) {
|
||||||
|
return this.tracks.sectionIndex[Math.max(0, Math.min(this.frameCount - 1, frame | 0))];
|
||||||
|
}
|
||||||
|
|
||||||
|
sectionAt(frame) {
|
||||||
|
return this.sections[this.sectionIndexAt(frame)] || this.sections[0];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Nearest section boundary frame in a direction. Powers the transport buttons. */
|
||||||
|
boundaryFrame(frame, direction) {
|
||||||
|
const bounds = this.sections.map((s) => s.startFrame).concat([this.frameCount - 1]);
|
||||||
|
if (direction < 0) {
|
||||||
|
for (let i = bounds.length - 1; i >= 0; i--) if (bounds[i] < frame - 2) return bounds[i];
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
for (let i = 0; i < bounds.length; i++) if (bounds[i] > frame + 2) return bounds[i];
|
||||||
|
return this.frameCount - 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Feature series decimated for plotting in the debug scope. */
|
||||||
|
series(name, points = 600) {
|
||||||
|
const src = this.raw[name] || this.tracks[name];
|
||||||
|
if (!src) return null;
|
||||||
|
const out = new Float32Array(points);
|
||||||
|
const step = this.frameCount / points;
|
||||||
|
for (let i = 0; i < points; i++) {
|
||||||
|
const start = Math.floor(i * step);
|
||||||
|
const end = Math.min(this.frameCount, Math.floor((i + 1) * step));
|
||||||
|
let peak = 0;
|
||||||
|
for (let f = start; f < end; f++) if (src[f] > peak) peak = src[f];
|
||||||
|
out[i] = peak;
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Build the whole table from decoded audio. Synchronous and CPU-bound;
|
||||||
|
* callers should run it off the main thread or accept a short freeze.
|
||||||
|
*/
|
||||||
|
static fromAudioBuffer(audioBuffer, { fps = 60, onProgress = null } = {}) {
|
||||||
|
const report = (stage, p) => onProgress && onProgress(stage, p);
|
||||||
|
|
||||||
|
report('spectrum', 0);
|
||||||
|
const analysis = analyzeBuffer(audioBuffer, {
|
||||||
|
fps,
|
||||||
|
onProgress: (p) => report('spectrum', p * 0.7),
|
||||||
|
});
|
||||||
|
|
||||||
|
report('tempo', 0.7);
|
||||||
|
const tempo = detectTempo(analysis.onsetEnvelope, fps);
|
||||||
|
const beatTracks = buildBeatTracks(tempo, analysis.frameCount, fps, analysis.raw.loudness);
|
||||||
|
|
||||||
|
report('structure', 0.85);
|
||||||
|
const sections = segment(analysis.raw, analysis.frameCount, fps, tempo);
|
||||||
|
|
||||||
|
report('lookahead', 0.95);
|
||||||
|
const tracks = {
|
||||||
|
...beatTracks,
|
||||||
|
...buildSectionTracks(sections, analysis.frameCount, fps, tempo),
|
||||||
|
};
|
||||||
|
|
||||||
|
report('done', 1);
|
||||||
|
|
||||||
|
return new FeatureTrack({
|
||||||
|
frameCount: analysis.frameCount,
|
||||||
|
fps,
|
||||||
|
duration: analysis.duration,
|
||||||
|
sampleRate: analysis.sampleRate,
|
||||||
|
raw: analysis.raw,
|
||||||
|
summary: { ...analysis.summary, bpm: tempo.bpm, tempoConfidence: tempo.confidence },
|
||||||
|
normalization: analysis.normalization,
|
||||||
|
onsetEnvelope: analysis.onsetEnvelope,
|
||||||
|
tempo,
|
||||||
|
sections,
|
||||||
|
tracks,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Per-frame section tracks, including the lookahead fields.
|
||||||
|
*
|
||||||
|
* `buildSlope` is the anticipation signal and the main payoff of analysing
|
||||||
|
* offline: it rises through the bars leading into a HIGHER-energy section, so a
|
||||||
|
* build can ramp its visuals into the drop instead of reacting once the drop has
|
||||||
|
* already landed. A causal analyser cannot produce this at all.
|
||||||
|
*/
|
||||||
|
function buildSectionTracks(sections, frameCount, fps, tempo) {
|
||||||
|
const sectionIndex = new Int32Array(frameCount);
|
||||||
|
const sectionProgress = new Float32Array(frameCount);
|
||||||
|
const sectionEnergy = new Float32Array(frameCount);
|
||||||
|
const buildSlope = new Float32Array(frameCount);
|
||||||
|
const timeToNextSection = new Float32Array(frameCount);
|
||||||
|
|
||||||
|
const barSeconds = (tempo.period * tempo.beatsPerBar) / fps;
|
||||||
|
const maxEnergy = Math.max(1e-6, ...sections.map((s) => s.energy));
|
||||||
|
|
||||||
|
for (let si = 0; si < sections.length; si++) {
|
||||||
|
const s = sections[si];
|
||||||
|
const next = sections[si + 1] || null;
|
||||||
|
const n = Math.max(1, s.endFrame - s.startFrame);
|
||||||
|
|
||||||
|
// Anticipation window: eight bars, or a third of the section if shorter.
|
||||||
|
const windowSeconds = Math.min(barSeconds * 8, s.duration / 3);
|
||||||
|
const windowFrames = Math.max(1, Math.round(windowSeconds * fps));
|
||||||
|
|
||||||
|
const rises = next ? next.energy > s.energy * 1.08 : false;
|
||||||
|
const rise = next ? Math.min(1, (next.energy - s.energy) / Math.max(1e-6, maxEnergy * 0.5)) : 0;
|
||||||
|
|
||||||
|
for (let f = s.startFrame; f < s.endFrame && f < frameCount; f++) {
|
||||||
|
sectionIndex[f] = si;
|
||||||
|
sectionProgress[f] = (f - s.startFrame) / n;
|
||||||
|
sectionEnergy[f] = s.energy / maxEnergy;
|
||||||
|
const toNext = (s.endFrame - f) / fps;
|
||||||
|
timeToNextSection[f] = toNext;
|
||||||
|
|
||||||
|
if (rises && s.endFrame - f <= windowFrames) {
|
||||||
|
const t = 1 - (s.endFrame - f) / windowFrames;
|
||||||
|
buildSlope[f] = t * t * rise; // eased, so the ramp starts gently
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Frames past the last section boundary (rounding slack at the tail).
|
||||||
|
for (let f = 0; f < frameCount; f++) {
|
||||||
|
if (sectionIndex[f] === 0 && sections.length && f >= sections[0].endFrame) {
|
||||||
|
sectionIndex[f] = sections.length - 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return { sectionIndex, sectionProgress, sectionEnergy, buildSlope, timeToNextSection };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Feature provider interface the Engine expects. */
|
||||||
|
export function featureProviderFor(featureTrack) {
|
||||||
|
return {
|
||||||
|
at: (frame) => featureTrack.at(frame),
|
||||||
|
frameCount: featureTrack.frameCount,
|
||||||
|
};
|
||||||
|
}
|
||||||
209
flow-state/src/audio/analyze.js
Normal file
209
flow-state/src/audio/analyze.js
Normal file
@ -0,0 +1,209 @@
|
|||||||
|
// STFT feature extraction. One pass over the whole track, producing one row per
|
||||||
|
// output video frame.
|
||||||
|
//
|
||||||
|
// Two details that matter more than they look:
|
||||||
|
//
|
||||||
|
// * Windows are CENTRED on the frame's timestamp, not started at it. A window
|
||||||
|
// that starts at the timestamp reports energy that arrives up to 23ms later,
|
||||||
|
// which reads on screen as the visuals lagging the music. Centring removes
|
||||||
|
// that systematic offset.
|
||||||
|
//
|
||||||
|
// * Energy features are normalised against the track's own 5th/95th percentiles
|
||||||
|
// at the end of the pass. A quiet ambient master and a brickwalled EDM master
|
||||||
|
// then both use the full reactive range, without anyone touching a gain knob.
|
||||||
|
// Absolute (un-normalised) statistics are kept in `summary` for the look
|
||||||
|
// generator, which does need to know that one track is genuinely darker.
|
||||||
|
|
||||||
|
import { FFT, hannWindow } from './fft.js';
|
||||||
|
|
||||||
|
export const FFT_SIZE = 2048;
|
||||||
|
|
||||||
|
/** Band edges in Hz. Sub is deliberately narrow — it is the kick, not the bass. */
|
||||||
|
export const BANDS = {
|
||||||
|
bandSub: [20, 60],
|
||||||
|
bandLow: [60, 250],
|
||||||
|
bandMid: [250, 2000],
|
||||||
|
bandHigh: [2000, 6000],
|
||||||
|
bandAir: [6000, 16000],
|
||||||
|
};
|
||||||
|
|
||||||
|
export const ENERGY_FEATURES = ['rms', 'loudness', 'bandSub', 'bandLow', 'bandMid', 'bandHigh', 'bandAir', 'flux'];
|
||||||
|
export const SCALE_FREE_FEATURES = ['centroid', 'flatness', 'width'];
|
||||||
|
export const RAW_FEATURES = [...ENERGY_FEATURES, ...SCALE_FREE_FEATURES];
|
||||||
|
|
||||||
|
function percentile(sorted, p) {
|
||||||
|
if (!sorted.length) return 0;
|
||||||
|
const i = Math.min(sorted.length - 1, Math.max(0, Math.round((sorted.length - 1) * p)));
|
||||||
|
return sorted[i];
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @param {AudioBuffer|{sampleRate,length,duration,numberOfChannels,getChannelData}} audioBuffer
|
||||||
|
* @returns {{frameCount, fps, sampleRate, duration, raw, summary, onsetEnvelope}}
|
||||||
|
*/
|
||||||
|
export function analyzeBuffer(audioBuffer, { fps = 60, fftSize = FFT_SIZE, onProgress = null } = {}) {
|
||||||
|
const sampleRate = audioBuffer.sampleRate;
|
||||||
|
const duration = audioBuffer.duration;
|
||||||
|
const hop = Math.max(1, Math.round(sampleRate / fps));
|
||||||
|
const frameCount = Math.max(1, Math.ceil(duration * fps));
|
||||||
|
|
||||||
|
const channels = audioBuffer.numberOfChannels;
|
||||||
|
const left = audioBuffer.getChannelData(0);
|
||||||
|
const right = channels > 1 ? audioBuffer.getChannelData(1) : left;
|
||||||
|
const totalSamples = left.length;
|
||||||
|
|
||||||
|
const fft = new FFT(fftSize);
|
||||||
|
const window = hannWindow(fftSize);
|
||||||
|
const frameBuffer = new Float32Array(fftSize);
|
||||||
|
const halfFft = fftSize >> 1;
|
||||||
|
|
||||||
|
// Precompute band bin ranges.
|
||||||
|
const binHz = sampleRate / fftSize;
|
||||||
|
const bandRanges = {};
|
||||||
|
for (const [name, [lo, hi]] of Object.entries(BANDS)) {
|
||||||
|
bandRanges[name] = [
|
||||||
|
Math.max(1, Math.floor(lo / binHz)),
|
||||||
|
Math.min(halfFft - 1, Math.ceil(hi / binHz)),
|
||||||
|
];
|
||||||
|
}
|
||||||
|
const binFrequencies = new Float32Array(halfFft);
|
||||||
|
for (let i = 0; i < halfFft; i++) binFrequencies[i] = i * binHz;
|
||||||
|
|
||||||
|
const raw = {};
|
||||||
|
for (const name of RAW_FEATURES) raw[name] = new Float32Array(frameCount);
|
||||||
|
|
||||||
|
let prevMagnitude = new Float32Array(halfFft);
|
||||||
|
const progressEvery = Math.max(1, Math.floor(frameCount / 50));
|
||||||
|
|
||||||
|
for (let f = 0; f < frameCount; f++) {
|
||||||
|
// Centred window.
|
||||||
|
const centre = f * hop;
|
||||||
|
const start = centre - (fftSize >> 1);
|
||||||
|
|
||||||
|
let sumSq = 0;
|
||||||
|
let midSq = 0;
|
||||||
|
let sideSq = 0;
|
||||||
|
for (let i = 0; i < fftSize; i++) {
|
||||||
|
const s = start + i;
|
||||||
|
let l = 0, r = 0;
|
||||||
|
if (s >= 0 && s < totalSamples) { l = left[s]; r = right[s]; }
|
||||||
|
const mono = (l + r) * 0.5;
|
||||||
|
const side = (l - r) * 0.5;
|
||||||
|
frameBuffer[i] = mono * window[i];
|
||||||
|
sumSq += mono * mono;
|
||||||
|
midSq += mono * mono;
|
||||||
|
sideSq += side * side;
|
||||||
|
}
|
||||||
|
|
||||||
|
raw.rms[f] = Math.sqrt(sumSq / fftSize);
|
||||||
|
raw.width[f] = Math.sqrt(sideSq / fftSize) / (Math.sqrt(midSq / fftSize) + Math.sqrt(sideSq / fftSize) + 1e-9);
|
||||||
|
|
||||||
|
const mag = fft.forward(frameBuffer);
|
||||||
|
|
||||||
|
let total = 0;
|
||||||
|
let weighted = 0;
|
||||||
|
let logSum = 0;
|
||||||
|
let flux = 0;
|
||||||
|
for (let i = 1; i < halfFft; i++) {
|
||||||
|
const m = mag[i];
|
||||||
|
total += m;
|
||||||
|
weighted += m * binFrequencies[i];
|
||||||
|
logSum += Math.log(m + 1e-10);
|
||||||
|
const d = m - prevMagnitude[i];
|
||||||
|
if (d > 0) flux += d;
|
||||||
|
}
|
||||||
|
|
||||||
|
raw.loudness[f] = total / halfFft;
|
||||||
|
raw.flux[f] = flux / halfFft;
|
||||||
|
|
||||||
|
// Spectral centroid, mapped to a log-frequency 0..1 so it matches how
|
||||||
|
// brightness is actually perceived.
|
||||||
|
const centroidHz = total > 1e-9 ? weighted / total : 0;
|
||||||
|
raw.centroid[f] = centroidHz > 20
|
||||||
|
? Math.min(1, Math.max(0, Math.log2(centroidHz / 20) / Math.log2(20000 / 20)))
|
||||||
|
: 0;
|
||||||
|
|
||||||
|
const arithmeticMean = total / (halfFft - 1);
|
||||||
|
const geometricMean = Math.exp(logSum / (halfFft - 1));
|
||||||
|
raw.flatness[f] = arithmeticMean > 1e-9 ? Math.min(1, geometricMean / arithmeticMean) : 0;
|
||||||
|
|
||||||
|
for (const [name, [lo, hi]] of Object.entries(bandRanges)) {
|
||||||
|
let sum = 0;
|
||||||
|
for (let i = lo; i <= hi; i++) sum += mag[i];
|
||||||
|
raw[name][f] = sum / (hi - lo + 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
prevMagnitude.set(mag);
|
||||||
|
|
||||||
|
if (onProgress && f % progressEvery === 0) onProgress(f / frameCount);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------- summary
|
||||||
|
// Computed from RAW values, before normalisation flattens them out.
|
||||||
|
const sortedLoudness = Float32Array.from(raw.loudness).sort();
|
||||||
|
const loudFloor = percentile(sortedLoudness, 0.1);
|
||||||
|
const activeFrames = [];
|
||||||
|
for (let f = 0; f < frameCount; f++) if (raw.loudness[f] > loudFloor) activeFrames.push(f);
|
||||||
|
const activeCount = activeFrames.length || 1;
|
||||||
|
|
||||||
|
const meanOf = (name) => {
|
||||||
|
let s = 0;
|
||||||
|
for (const f of activeFrames) s += raw[name][f];
|
||||||
|
return s / activeCount;
|
||||||
|
};
|
||||||
|
|
||||||
|
const sortedRms = Float32Array.from(raw.rms).sort();
|
||||||
|
const summary = {
|
||||||
|
duration,
|
||||||
|
sampleRate,
|
||||||
|
fps,
|
||||||
|
frameCount,
|
||||||
|
meanCentroid: meanOf('centroid'),
|
||||||
|
meanFlatness: meanOf('flatness'),
|
||||||
|
meanWidth: meanOf('width'),
|
||||||
|
meanLoudness: meanOf('loudness'),
|
||||||
|
// Crest-ish: how much room there is between typical and peak level.
|
||||||
|
// High on dynamic ambient, low on limitered club masters.
|
||||||
|
dynamicRange: percentile(sortedRms, 0.95) > 1e-9
|
||||||
|
? 1 - percentile(sortedRms, 0.4) / percentile(sortedRms, 0.95)
|
||||||
|
: 0,
|
||||||
|
bandBalance: {
|
||||||
|
sub: meanOf('bandSub'), low: meanOf('bandLow'), mid: meanOf('bandMid'),
|
||||||
|
high: meanOf('bandHigh'), air: meanOf('bandAir'),
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
// ------------------------------------------------------------ normalise
|
||||||
|
const normalization = {};
|
||||||
|
for (const name of ENERGY_FEATURES) {
|
||||||
|
const sorted = Float32Array.from(raw[name]).sort();
|
||||||
|
const lo = percentile(sorted, 0.05);
|
||||||
|
const hi = percentile(sorted, 0.95);
|
||||||
|
normalization[name] = { lo, hi };
|
||||||
|
const span = hi - lo;
|
||||||
|
const arr = raw[name];
|
||||||
|
if (span > 1e-12) {
|
||||||
|
for (let f = 0; f < frameCount; f++) {
|
||||||
|
arr[f] = Math.min(1, Math.max(0, (arr[f] - lo) / span));
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
arr.fill(0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (const name of SCALE_FREE_FEATURES) {
|
||||||
|
const arr = raw[name];
|
||||||
|
for (let f = 0; f < frameCount; f++) arr[f] = Math.min(1, Math.max(0, arr[f]));
|
||||||
|
}
|
||||||
|
|
||||||
|
// The onset envelope drives tempo detection. Kept separate from the
|
||||||
|
// normalised flux because tempo wants raw contrast, not a clipped range.
|
||||||
|
const onsetEnvelope = new Float32Array(frameCount);
|
||||||
|
for (let f = 0; f < frameCount; f++) {
|
||||||
|
// Weight the low band up: in this material the kick is the clock.
|
||||||
|
onsetEnvelope[f] = raw.flux[f] * 0.6 + raw.bandSub[f] * 0.25 + raw.bandLow[f] * 0.15;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (onProgress) onProgress(1);
|
||||||
|
|
||||||
|
return { frameCount, fps, sampleRate, duration, raw, summary, normalization, onsetEnvelope };
|
||||||
|
}
|
||||||
105
flow-state/src/audio/clicktrack.js
Normal file
105
flow-state/src/audio/clicktrack.js
Normal file
@ -0,0 +1,105 @@
|
|||||||
|
// Click track — the Phase 1 gate, and the most useful validation tool in the
|
||||||
|
// project.
|
||||||
|
//
|
||||||
|
// Beat detection cannot be judged by watching visuals: a grid that is 20ms late
|
||||||
|
// or at half tempo still "looks kind of right". Mixing an audible click onto the
|
||||||
|
// detected grid makes the answer immediate and unambiguous. Downbeats get a
|
||||||
|
// higher pitch so bar alignment is audible too.
|
||||||
|
//
|
||||||
|
// If the clicks don't sit on the beat, stop and fix tempo.js before touching
|
||||||
|
// anything downstream — every timing artefact in the finished video originates here.
|
||||||
|
|
||||||
|
const CLICK_MS = 25;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Render the track with clicks mixed over it.
|
||||||
|
* @returns {Promise<AudioBuffer>}
|
||||||
|
*/
|
||||||
|
export async function renderClickTrack(audioBuffer, tempo, { clickGain = 0.5, musicGain = 0.6, downbeatsOnly = false } = {}) {
|
||||||
|
const sampleRate = audioBuffer.sampleRate;
|
||||||
|
const length = audioBuffer.length;
|
||||||
|
const offline = new OfflineAudioContext(2, length, sampleRate);
|
||||||
|
|
||||||
|
const music = offline.createBufferSource();
|
||||||
|
music.buffer = audioBuffer;
|
||||||
|
const musicNode = offline.createGain();
|
||||||
|
musicNode.gain.value = musicGain;
|
||||||
|
music.connect(musicNode).connect(offline.destination);
|
||||||
|
|
||||||
|
const clickBuffer = makeClick(offline, sampleRate, 1000);
|
||||||
|
const downbeatBuffer = makeClick(offline, sampleRate, 1800);
|
||||||
|
|
||||||
|
const downbeatSet = new Set(tempo.downbeats.map((t) => Math.round(t * 1000)));
|
||||||
|
|
||||||
|
for (const time of tempo.beats) {
|
||||||
|
if (time >= audioBuffer.duration) break;
|
||||||
|
const isDownbeat = downbeatSet.has(Math.round(time * 1000));
|
||||||
|
if (downbeatsOnly && !isDownbeat) continue;
|
||||||
|
|
||||||
|
const src = offline.createBufferSource();
|
||||||
|
src.buffer = isDownbeat ? downbeatBuffer : clickBuffer;
|
||||||
|
const gain = offline.createGain();
|
||||||
|
gain.gain.value = clickGain * (isDownbeat ? 1.0 : 0.7);
|
||||||
|
src.connect(gain).connect(offline.destination);
|
||||||
|
src.start(time);
|
||||||
|
}
|
||||||
|
|
||||||
|
music.start(0);
|
||||||
|
return await offline.startRendering();
|
||||||
|
}
|
||||||
|
|
||||||
|
function makeClick(ctx, sampleRate, frequency) {
|
||||||
|
const length = Math.round((CLICK_MS / 1000) * sampleRate);
|
||||||
|
const buffer = ctx.createBuffer(1, length, sampleRate);
|
||||||
|
const data = buffer.getChannelData(0);
|
||||||
|
for (let i = 0; i < length; i++) {
|
||||||
|
const t = i / sampleRate;
|
||||||
|
const env = Math.exp(-t * 180);
|
||||||
|
data[i] = Math.sin(2 * Math.PI * frequency * t) * env;
|
||||||
|
}
|
||||||
|
return buffer;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Wrap an AudioBuffer as a WAV blob so it can be played or downloaded. */
|
||||||
|
export function audioBufferToWavBlob(audioBuffer) {
|
||||||
|
const channels = audioBuffer.numberOfChannels;
|
||||||
|
const length = audioBuffer.length;
|
||||||
|
const sampleRate = audioBuffer.sampleRate;
|
||||||
|
const bytesPerSample = 2;
|
||||||
|
const blockAlign = channels * bytesPerSample;
|
||||||
|
const dataSize = length * blockAlign;
|
||||||
|
|
||||||
|
const buffer = new ArrayBuffer(44 + dataSize);
|
||||||
|
const view = new DataView(buffer);
|
||||||
|
|
||||||
|
const writeString = (offset, str) => {
|
||||||
|
for (let i = 0; i < str.length; i++) view.setUint8(offset + i, str.charCodeAt(i));
|
||||||
|
};
|
||||||
|
|
||||||
|
writeString(0, 'RIFF');
|
||||||
|
view.setUint32(4, 36 + dataSize, true);
|
||||||
|
writeString(8, 'WAVE');
|
||||||
|
writeString(12, 'fmt ');
|
||||||
|
view.setUint32(16, 16, true);
|
||||||
|
view.setUint16(20, 1, true);
|
||||||
|
view.setUint16(22, channels, true);
|
||||||
|
view.setUint32(24, sampleRate, true);
|
||||||
|
view.setUint32(28, sampleRate * blockAlign, true);
|
||||||
|
view.setUint16(32, blockAlign, true);
|
||||||
|
view.setUint16(34, 16, true);
|
||||||
|
writeString(36, 'data');
|
||||||
|
view.setUint32(40, dataSize, true);
|
||||||
|
|
||||||
|
const data = [];
|
||||||
|
for (let c = 0; c < channels; c++) data.push(audioBuffer.getChannelData(c));
|
||||||
|
|
||||||
|
let offset = 44;
|
||||||
|
for (let i = 0; i < length; i++) {
|
||||||
|
for (let c = 0; c < channels; c++) {
|
||||||
|
const s = Math.max(-1, Math.min(1, data[c][i]));
|
||||||
|
view.setInt16(offset, s < 0 ? s * 0x8000 : s * 0x7fff, true);
|
||||||
|
offset += 2;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return new Blob([buffer], { type: 'audio/wav' });
|
||||||
|
}
|
||||||
47
flow-state/src/audio/decode.js
Normal file
47
flow-state/src/audio/decode.js
Normal file
@ -0,0 +1,47 @@
|
|||||||
|
// File -> AudioBuffer. Decoding is done once, up front, and the PCM is kept:
|
||||||
|
// the analysis pass, the seed hash, the click track and the export mux all read
|
||||||
|
// the same decoded samples.
|
||||||
|
|
||||||
|
let sharedContext = null;
|
||||||
|
|
||||||
|
function context() {
|
||||||
|
if (!sharedContext) {
|
||||||
|
sharedContext = new (window.AudioContext || window.webkitAudioContext)();
|
||||||
|
}
|
||||||
|
return sharedContext;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function decodeFile(file) {
|
||||||
|
const arrayBuffer = await file.arrayBuffer();
|
||||||
|
return decodeArrayBuffer(arrayBuffer);
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function decodeArrayBuffer(arrayBuffer) {
|
||||||
|
const ctx = context();
|
||||||
|
// decodeAudioData detaches the buffer, so hand it a copy if the caller may
|
||||||
|
// still need the original bytes.
|
||||||
|
return await ctx.decodeAudioData(arrayBuffer.slice(0));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Interleaved mono mixdown. Used for the content hash that seeds the look. */
|
||||||
|
export function monoSamples(audioBuffer) {
|
||||||
|
const channels = audioBuffer.numberOfChannels;
|
||||||
|
const left = audioBuffer.getChannelData(0);
|
||||||
|
if (channels === 1) return left;
|
||||||
|
const right = audioBuffer.getChannelData(1);
|
||||||
|
const out = new Float32Array(left.length);
|
||||||
|
for (let i = 0; i < left.length; i++) out[i] = (left[i] + right[i]) * 0.5;
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function audioContext() {
|
||||||
|
return context();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Human-readable duration, used by the transport display. */
|
||||||
|
export function formatTime(seconds) {
|
||||||
|
if (!isFinite(seconds) || seconds < 0) return '0:00';
|
||||||
|
const m = Math.floor(seconds / 60);
|
||||||
|
const s = Math.floor(seconds % 60);
|
||||||
|
return `${m}:${String(s).padStart(2, '0')}`;
|
||||||
|
}
|
||||||
79
flow-state/src/audio/fft.js
Normal file
79
flow-state/src/audio/fft.js
Normal file
@ -0,0 +1,79 @@
|
|||||||
|
// Iterative radix-2 Cooley-Tukey FFT with precomputed twiddles and bit-reversal.
|
||||||
|
//
|
||||||
|
// Allocated once per size and reused across all ~21k frames of a track — the
|
||||||
|
// analysis pass is the one place in this project where JS throughput actually
|
||||||
|
// matters, and per-frame allocation is what would sink it.
|
||||||
|
|
||||||
|
export class FFT {
|
||||||
|
constructor(size) {
|
||||||
|
if ((size & (size - 1)) !== 0) throw new Error(`FFT size must be a power of two, got ${size}`);
|
||||||
|
this.size = size;
|
||||||
|
this.half = size >> 1;
|
||||||
|
|
||||||
|
this.re = new Float32Array(size);
|
||||||
|
this.im = new Float32Array(size);
|
||||||
|
this.magnitude = new Float32Array(this.half);
|
||||||
|
|
||||||
|
// Bit-reversal permutation table.
|
||||||
|
this.rev = new Uint32Array(size);
|
||||||
|
const bits = Math.log2(size);
|
||||||
|
for (let i = 0; i < size; i++) {
|
||||||
|
let r = 0;
|
||||||
|
for (let b = 0; b < bits; b++) if (i & (1 << b)) r |= 1 << (bits - 1 - b);
|
||||||
|
this.rev[i] = r;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Twiddle factors, flattened per stage.
|
||||||
|
this.cos = new Float32Array(this.half);
|
||||||
|
this.sin = new Float32Array(this.half);
|
||||||
|
for (let i = 0; i < this.half; i++) {
|
||||||
|
this.cos[i] = Math.cos((-2 * Math.PI * i) / size);
|
||||||
|
this.sin[i] = Math.sin((-2 * Math.PI * i) / size);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Forward transform of a real windowed signal. Writes into `this.magnitude`
|
||||||
|
* (first size/2 bins) and returns it. The buffer is reused: copy it if you
|
||||||
|
* need to keep it.
|
||||||
|
*/
|
||||||
|
forward(input) {
|
||||||
|
const { size, re, im, rev, cos, sin } = this;
|
||||||
|
|
||||||
|
for (let i = 0; i < size; i++) {
|
||||||
|
re[i] = input[rev[i]];
|
||||||
|
im[i] = 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
for (let len = 2; len <= size; len <<= 1) {
|
||||||
|
const halfLen = len >> 1;
|
||||||
|
const step = size / len;
|
||||||
|
for (let i = 0; i < size; i += len) {
|
||||||
|
for (let j = 0, k = 0; j < halfLen; j++, k += step) {
|
||||||
|
const c = cos[k], s = sin[k];
|
||||||
|
const a = i + j, b = a + halfLen;
|
||||||
|
const tre = re[b] * c - im[b] * s;
|
||||||
|
const tim = re[b] * s + im[b] * c;
|
||||||
|
re[b] = re[a] - tre;
|
||||||
|
im[b] = im[a] - tim;
|
||||||
|
re[a] += tre;
|
||||||
|
im[a] += tim;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const mag = this.magnitude;
|
||||||
|
const scale = 2 / size;
|
||||||
|
for (let i = 0; i < this.half; i++) {
|
||||||
|
mag[i] = Math.sqrt(re[i] * re[i] + im[i] * im[i]) * scale;
|
||||||
|
}
|
||||||
|
return mag;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Periodic Hann window. Periodic (not symmetric) is correct for STFT analysis. */
|
||||||
|
export function hannWindow(size) {
|
||||||
|
const w = new Float32Array(size);
|
||||||
|
for (let i = 0; i < size; i++) w[i] = 0.5 * (1 - Math.cos((2 * Math.PI * i) / size));
|
||||||
|
return w;
|
||||||
|
}
|
||||||
266
flow-state/src/audio/segment.js
Normal file
266
flow-state/src/audio/segment.js
Normal file
@ -0,0 +1,266 @@
|
|||||||
|
// Structural segmentation: where the song changes, and what each part is.
|
||||||
|
//
|
||||||
|
// Standard approach — self-similarity plus a checkerboard novelty kernel — kept
|
||||||
|
// deliberately simple. This is the fuzziest stage in the pipeline, and a wrong
|
||||||
|
// boundary costs a scene change in a slightly odd place, not a broken video. An
|
||||||
|
// ML detour here would buy very little.
|
||||||
|
//
|
||||||
|
// The one domain-specific trick that earns its keep: boundaries are SNAPPED to
|
||||||
|
// the bar grid, preferring 4- and 8-bar multiples. Electronic music changes on
|
||||||
|
// phrase lines, so snapping converts a boundary that is roughly right into one
|
||||||
|
// that is exactly right.
|
||||||
|
|
||||||
|
export const SECTION_KINDS = ['intro', 'build', 'drop', 'sustain', 'breakdown', 'outro'];
|
||||||
|
|
||||||
|
const ANALYSIS_HZ = 4; // coarse grid for the similarity matrix
|
||||||
|
const KERNEL_SECONDS = 6; // half-width of the checkerboard kernel
|
||||||
|
const MIN_SECTION_SECONDS = 12;
|
||||||
|
|
||||||
|
function median(values) {
|
||||||
|
if (!values.length) return 0;
|
||||||
|
const s = Float64Array.from(values).sort();
|
||||||
|
return s[Math.floor(s.length / 2)];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Coarse, L2-normalised feature vectors — the rows of the similarity matrix. */
|
||||||
|
function buildCoarseVectors(raw, frameCount, fps) {
|
||||||
|
const step = Math.max(1, Math.round(fps / ANALYSIS_HZ));
|
||||||
|
const count = Math.max(1, Math.floor(frameCount / step));
|
||||||
|
const dims = ['bandSub', 'bandLow', 'bandMid', 'bandHigh', 'bandAir', 'centroid', 'flatness'];
|
||||||
|
const vectors = [];
|
||||||
|
|
||||||
|
for (let c = 0; c < count; c++) {
|
||||||
|
const start = c * step;
|
||||||
|
const end = Math.min(frameCount, start + step);
|
||||||
|
const v = new Float32Array(dims.length);
|
||||||
|
for (let d = 0; d < dims.length; d++) {
|
||||||
|
let sum = 0;
|
||||||
|
for (let f = start; f < end; f++) sum += raw[dims[d]][f];
|
||||||
|
v[d] = sum / Math.max(1, end - start);
|
||||||
|
}
|
||||||
|
let norm = 0;
|
||||||
|
for (let d = 0; d < v.length; d++) norm += v[d] * v[d];
|
||||||
|
norm = Math.sqrt(norm) + 1e-9;
|
||||||
|
for (let d = 0; d < v.length; d++) v[d] /= norm;
|
||||||
|
vectors.push(v);
|
||||||
|
}
|
||||||
|
return { vectors, step };
|
||||||
|
}
|
||||||
|
|
||||||
|
function cosine(a, b) {
|
||||||
|
let dot = 0;
|
||||||
|
for (let i = 0; i < a.length; i++) dot += a[i] * b[i];
|
||||||
|
return dot;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Novelty curve. Only the band around the diagonal is needed, so this never
|
||||||
|
* materialises the full N×N matrix.
|
||||||
|
*/
|
||||||
|
function noveltyCurve(vectors, kernelHalf) {
|
||||||
|
const n = vectors.length;
|
||||||
|
const novelty = new Float32Array(n);
|
||||||
|
|
||||||
|
// Gaussian-tapered checkerboard weights.
|
||||||
|
const size = kernelHalf * 2;
|
||||||
|
const weights = new Float32Array(size * size);
|
||||||
|
const sigma = kernelHalf / 1.5;
|
||||||
|
for (let a = 0; a < size; a++) {
|
||||||
|
for (let b = 0; b < size; b++) {
|
||||||
|
const da = a - kernelHalf + 0.5;
|
||||||
|
const db = b - kernelHalf + 0.5;
|
||||||
|
const taper = Math.exp(-(da * da + db * db) / (2 * sigma * sigma));
|
||||||
|
const sign = (da * db) > 0 ? 1 : -1;
|
||||||
|
weights[a * size + b] = sign * taper;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for (let i = kernelHalf; i < n - kernelHalf; i++) {
|
||||||
|
let sum = 0;
|
||||||
|
for (let a = 0; a < size; a++) {
|
||||||
|
const ia = i - kernelHalf + a;
|
||||||
|
for (let b = 0; b < size; b++) {
|
||||||
|
const ib = i - kernelHalf + b;
|
||||||
|
sum += weights[a * size + b] * cosine(vectors[ia], vectors[ib]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
novelty[i] = Math.max(0, sum);
|
||||||
|
}
|
||||||
|
|
||||||
|
let peak = 0;
|
||||||
|
for (let i = 0; i < n; i++) if (novelty[i] > peak) peak = novelty[i];
|
||||||
|
if (peak > 1e-9) for (let i = 0; i < n; i++) novelty[i] /= peak;
|
||||||
|
return novelty;
|
||||||
|
}
|
||||||
|
|
||||||
|
function pickPeaks(novelty, minDistance, threshold) {
|
||||||
|
const peaks = [];
|
||||||
|
const n = novelty.length;
|
||||||
|
for (let i = 1; i < n - 1; i++) {
|
||||||
|
if (novelty[i] < threshold) continue;
|
||||||
|
if (novelty[i] < novelty[i - 1] || novelty[i] < novelty[i + 1]) continue;
|
||||||
|
// Local maximum within the exclusion window.
|
||||||
|
let isMax = true;
|
||||||
|
const lo = Math.max(0, i - minDistance);
|
||||||
|
const hi = Math.min(n - 1, i + minDistance);
|
||||||
|
for (let j = lo; j <= hi; j++) if (novelty[j] > novelty[i]) { isMax = false; break; }
|
||||||
|
if (!isMax) continue;
|
||||||
|
if (peaks.length && i - peaks[peaks.length - 1] < minDistance) {
|
||||||
|
if (novelty[i] > novelty[peaks[peaks.length - 1]]) peaks[peaks.length - 1] = i;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
peaks.push(i);
|
||||||
|
}
|
||||||
|
return peaks;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Snap a time to the musical grid. Prefers, in order: an 8-bar line, a 4-bar
|
||||||
|
* line, then any downbeat — but only if one is close enough to be plausibly the
|
||||||
|
* same boundary.
|
||||||
|
*/
|
||||||
|
function snapToGrid(time, downbeats, barsPerPhrase = 8) {
|
||||||
|
if (!downbeats.length) return time;
|
||||||
|
const tolerance = 2.0;
|
||||||
|
|
||||||
|
const tryLevels = [barsPerPhrase, 4, 1];
|
||||||
|
for (const level of tryLevels) {
|
||||||
|
let best = null;
|
||||||
|
let bestDist = Infinity;
|
||||||
|
for (let i = 0; i < downbeats.length; i += level) {
|
||||||
|
const d = Math.abs(downbeats[i] - time);
|
||||||
|
if (d < bestDist) { bestDist = d; best = downbeats[i]; }
|
||||||
|
}
|
||||||
|
if (best !== null && bestDist <= tolerance) return best;
|
||||||
|
}
|
||||||
|
return time;
|
||||||
|
}
|
||||||
|
|
||||||
|
function linearSlope(values) {
|
||||||
|
const n = values.length;
|
||||||
|
if (n < 2) return 0;
|
||||||
|
let sx = 0, sy = 0, sxy = 0, sxx = 0;
|
||||||
|
for (let i = 0; i < n; i++) {
|
||||||
|
sx += i; sy += values[i]; sxy += i * values[i]; sxx += i * i;
|
||||||
|
}
|
||||||
|
const denom = n * sxx - sx * sx;
|
||||||
|
if (Math.abs(denom) < 1e-12) return 0;
|
||||||
|
return ((n * sxy - sx * sy) / denom) * n; // normalised to "change across the section"
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Label a section from its own statistics plus where it sits in the track.
|
||||||
|
* Thresholds are relative to the track, never absolute, so a quiet ambient piece
|
||||||
|
* still gets a full set of labels rather than being classified as one long intro.
|
||||||
|
*/
|
||||||
|
function classify(section, context) {
|
||||||
|
const { energy, slope, flux, index, count, endEnergy } = section;
|
||||||
|
const { medianEnergy, isFirst, isLast } = context;
|
||||||
|
|
||||||
|
const low = energy < medianEnergy * 0.72;
|
||||||
|
const high = energy > medianEnergy * 1.12;
|
||||||
|
|
||||||
|
if (isFirst && energy < medianEnergy) return 'intro';
|
||||||
|
if (isLast && (low || slope < -0.15)) return 'outro';
|
||||||
|
if (slope > 0.18 && endEnergy > medianEnergy * 0.95) return 'build';
|
||||||
|
if (high && flux > context.medianFlux * 0.95) return 'drop';
|
||||||
|
if (low) return 'breakdown';
|
||||||
|
if (index === 0) return 'intro';
|
||||||
|
if (index === count - 1) return 'outro';
|
||||||
|
return 'sustain';
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @returns {Array<{index,start,end,startFrame,endFrame,kind,energy,slope,flux,centroid}>}
|
||||||
|
*/
|
||||||
|
export function segment(raw, frameCount, fps, tempo) {
|
||||||
|
const { vectors, step } = buildCoarseVectors(raw, frameCount, fps);
|
||||||
|
const kernelHalf = Math.max(4, Math.round(KERNEL_SECONDS * ANALYSIS_HZ));
|
||||||
|
|
||||||
|
let boundaryTimes = [];
|
||||||
|
if (vectors.length > kernelHalf * 2 + 4) {
|
||||||
|
const novelty = noveltyCurve(vectors, kernelHalf);
|
||||||
|
const minDistance = Math.round(MIN_SECTION_SECONDS * ANALYSIS_HZ);
|
||||||
|
|
||||||
|
// Adaptive threshold: mean + a fraction of the spread, so a track with
|
||||||
|
// gentle transitions still yields boundaries and a busy one isn't shredded.
|
||||||
|
let mean = 0;
|
||||||
|
for (let i = 0; i < novelty.length; i++) mean += novelty[i];
|
||||||
|
mean /= novelty.length;
|
||||||
|
let variance = 0;
|
||||||
|
for (let i = 0; i < novelty.length; i++) variance += (novelty[i] - mean) ** 2;
|
||||||
|
const std = Math.sqrt(variance / novelty.length);
|
||||||
|
const threshold = mean + std * 0.6;
|
||||||
|
|
||||||
|
boundaryTimes = pickPeaks(novelty, minDistance, threshold)
|
||||||
|
.map((i) => (i * step) / fps);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Snap to the bar grid, dedupe, and drop anything too close to the edges.
|
||||||
|
const duration = frameCount / fps;
|
||||||
|
const snapped = boundaryTimes
|
||||||
|
.map((t) => snapToGrid(t, tempo.downbeats))
|
||||||
|
.filter((t) => t > MIN_SECTION_SECONDS * 0.5 && t < duration - MIN_SECTION_SECONDS * 0.5)
|
||||||
|
.sort((a, b) => a - b);
|
||||||
|
|
||||||
|
const bounds = [0];
|
||||||
|
for (const t of snapped) {
|
||||||
|
if (t - bounds[bounds.length - 1] >= MIN_SECTION_SECONDS) bounds.push(t);
|
||||||
|
}
|
||||||
|
bounds.push(duration);
|
||||||
|
|
||||||
|
// Build sections and gather their statistics.
|
||||||
|
const sections = [];
|
||||||
|
for (let i = 0; i < bounds.length - 1; i++) {
|
||||||
|
const start = bounds[i];
|
||||||
|
const end = bounds[i + 1];
|
||||||
|
const startFrame = Math.round(start * fps);
|
||||||
|
const endFrame = Math.min(frameCount, Math.round(end * fps));
|
||||||
|
const n = Math.max(1, endFrame - startFrame);
|
||||||
|
|
||||||
|
let energySum = 0, fluxSum = 0, centroidSum = 0, flatnessSum = 0, widthSum = 0;
|
||||||
|
const coarseEnergy = [];
|
||||||
|
const coarseStep = Math.max(1, Math.floor(n / 24));
|
||||||
|
for (let f = startFrame; f < endFrame; f++) {
|
||||||
|
energySum += raw.loudness[f];
|
||||||
|
fluxSum += raw.flux[f];
|
||||||
|
centroidSum += raw.centroid[f];
|
||||||
|
flatnessSum += raw.flatness[f];
|
||||||
|
widthSum += raw.width[f];
|
||||||
|
if ((f - startFrame) % coarseStep === 0) coarseEnergy.push(raw.loudness[f]);
|
||||||
|
}
|
||||||
|
|
||||||
|
const tailStart = Math.max(startFrame, endFrame - Math.round(fps * 4));
|
||||||
|
let endEnergy = 0;
|
||||||
|
for (let f = tailStart; f < endFrame; f++) endEnergy += raw.loudness[f];
|
||||||
|
endEnergy /= Math.max(1, endFrame - tailStart);
|
||||||
|
|
||||||
|
sections.push({
|
||||||
|
index: i,
|
||||||
|
start, end,
|
||||||
|
startFrame, endFrame,
|
||||||
|
duration: end - start,
|
||||||
|
energy: energySum / n,
|
||||||
|
flux: fluxSum / n,
|
||||||
|
centroid: centroidSum / n,
|
||||||
|
flatness: flatnessSum / n,
|
||||||
|
width: widthSum / n,
|
||||||
|
slope: linearSlope(coarseEnergy),
|
||||||
|
endEnergy,
|
||||||
|
kind: 'sustain',
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
const context = {
|
||||||
|
medianEnergy: median(sections.map((s) => s.energy)) || 1e-6,
|
||||||
|
medianFlux: median(sections.map((s) => s.flux)) || 1e-6,
|
||||||
|
};
|
||||||
|
sections.forEach((s, i) => {
|
||||||
|
s.kind = classify(
|
||||||
|
{ ...s, index: i, count: sections.length },
|
||||||
|
{ ...context, isFirst: i === 0, isLast: i === sections.length - 1 },
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
return sections;
|
||||||
|
}
|
||||||
125
flow-state/src/audio/synth.js
Normal file
125
flow-state/src/audio/synth.js
Normal file
@ -0,0 +1,125 @@
|
|||||||
|
// Synthetic audio with known ground truth, so tempo and segmentation can be
|
||||||
|
// tested against an exact answer instead of "sounds about right". Real music
|
||||||
|
// goes through the click track and the battery; this catches regressions in CI
|
||||||
|
// speed and without a GPU.
|
||||||
|
|
||||||
|
/** Minimal stand-in for AudioBuffer — the analysis code only needs this surface. */
|
||||||
|
export class MockAudioBuffer {
|
||||||
|
constructor(channels, length, sampleRate) {
|
||||||
|
this.numberOfChannels = channels;
|
||||||
|
this.length = length;
|
||||||
|
this.sampleRate = sampleRate;
|
||||||
|
this.duration = length / sampleRate;
|
||||||
|
this._data = Array.from({ length: channels }, () => new Float32Array(length));
|
||||||
|
}
|
||||||
|
getChannelData(i) { return this._data[i]; }
|
||||||
|
}
|
||||||
|
|
||||||
|
function addKick(data, sampleRate, at, gain = 1) {
|
||||||
|
const start = Math.round(at * sampleRate);
|
||||||
|
const length = Math.round(0.12 * sampleRate);
|
||||||
|
for (let i = 0; i < length; i++) {
|
||||||
|
const s = start + i;
|
||||||
|
if (s < 0 || s >= data.length) continue;
|
||||||
|
const t = i / sampleRate;
|
||||||
|
const env = Math.exp(-t * 30);
|
||||||
|
const freq = 55 * Math.exp(-t * 20) + 40; // pitch drop, like a real kick
|
||||||
|
data[s] += Math.sin(2 * Math.PI * freq * t) * env * gain;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function addHat(data, sampleRate, at, gain = 0.3, seed = 1) {
|
||||||
|
const start = Math.round(at * sampleRate);
|
||||||
|
const length = Math.round(0.04 * sampleRate);
|
||||||
|
let s0 = seed >>> 0;
|
||||||
|
const rnd = () => {
|
||||||
|
s0 = (Math.imul(s0 ^ (s0 >>> 15), s0 | 1) + 0x6d2b79f5) >>> 0;
|
||||||
|
return ((s0 >>> 14) & 0xffff) / 0xffff - 0.5;
|
||||||
|
};
|
||||||
|
for (let i = 0; i < length; i++) {
|
||||||
|
const s = start + i;
|
||||||
|
if (s < 0 || s >= data.length) continue;
|
||||||
|
const env = Math.exp(-(i / sampleRate) * 90);
|
||||||
|
data[s] += rnd() * env * gain;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function addPad(data, sampleRate, from, to, gain = 0.15, root = 110) {
|
||||||
|
const start = Math.round(from * sampleRate);
|
||||||
|
const end = Math.min(data.length, Math.round(to * sampleRate));
|
||||||
|
for (let s = start; s < end; s++) {
|
||||||
|
const t = s / sampleRate;
|
||||||
|
data[s] += (Math.sin(2 * Math.PI * root * t) + Math.sin(2 * Math.PI * root * 1.5 * t)) * gain * 0.5;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A four-to-the-floor track at a known BPM.
|
||||||
|
* @param {object} opts
|
||||||
|
* @returns {MockAudioBuffer}
|
||||||
|
*/
|
||||||
|
export function synthesizeBeat({
|
||||||
|
bpm = 128,
|
||||||
|
duration = 40,
|
||||||
|
sampleRate = 44100,
|
||||||
|
hats = true,
|
||||||
|
pad = true,
|
||||||
|
kickGain = 1,
|
||||||
|
hatGain = 0.25,
|
||||||
|
padRoot = 110,
|
||||||
|
padGain = 0.12,
|
||||||
|
} = {}) {
|
||||||
|
const length = Math.round(duration * sampleRate);
|
||||||
|
const buffer = new MockAudioBuffer(2, length, sampleRate);
|
||||||
|
const left = buffer.getChannelData(0);
|
||||||
|
const right = buffer.getChannelData(1);
|
||||||
|
|
||||||
|
const beat = 60 / bpm;
|
||||||
|
let index = 0;
|
||||||
|
for (let t = 0; t < duration; t += beat, index++) {
|
||||||
|
// Accent the downbeat so the bar phase is detectable.
|
||||||
|
addKick(left, sampleRate, t, kickGain * (index % 4 === 0 ? 1.0 : 0.8));
|
||||||
|
if (hats) addHat(left, sampleRate, t + beat / 2, hatGain, index + 1);
|
||||||
|
}
|
||||||
|
if (pad) addPad(left, sampleRate, 0, duration, padGain, padRoot);
|
||||||
|
|
||||||
|
for (let i = 0; i < length; i++) right[i] = left[i] * 0.98;
|
||||||
|
return buffer;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A track with a deliberate structural change at `changeAt` seconds: sparse and
|
||||||
|
* dark before, dense and bright after. Segmentation must find that boundary.
|
||||||
|
*/
|
||||||
|
export function synthesizeSectioned({
|
||||||
|
bpm = 128,
|
||||||
|
duration = 120,
|
||||||
|
changeAt = 60,
|
||||||
|
sampleRate = 44100,
|
||||||
|
} = {}) {
|
||||||
|
const length = Math.round(duration * sampleRate);
|
||||||
|
const buffer = new MockAudioBuffer(2, length, sampleRate);
|
||||||
|
const left = buffer.getChannelData(0);
|
||||||
|
const right = buffer.getChannelData(1);
|
||||||
|
|
||||||
|
const beat = 60 / bpm;
|
||||||
|
let index = 0;
|
||||||
|
for (let t = 0; t < duration; t += beat, index++) {
|
||||||
|
const after = t >= changeAt;
|
||||||
|
addKick(left, sampleRate, t, after ? 1.0 : 0.35);
|
||||||
|
if (after) {
|
||||||
|
addHat(left, sampleRate, t + beat / 2, 0.45, index + 1);
|
||||||
|
addHat(left, sampleRate, t + beat / 4, 0.25, index + 7);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
addPad(left, sampleRate, 0, changeAt, 0.10, 110);
|
||||||
|
addPad(left, sampleRate, changeAt, duration, 0.22, 440); // brighter after
|
||||||
|
|
||||||
|
for (let i = 0; i < length; i++) right[i] = left[i] * 0.98;
|
||||||
|
return buffer;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Silence, for degenerate-input checks. */
|
||||||
|
export function synthesizeSilence({ duration = 10, sampleRate = 44100 } = {}) {
|
||||||
|
return new MockAudioBuffer(2, Math.round(duration * sampleRate), sampleRate);
|
||||||
|
}
|
||||||
369
flow-state/src/audio/tempo.js
Normal file
369
flow-state/src/audio/tempo.js
Normal file
@ -0,0 +1,369 @@
|
|||||||
|
// Tempo, beat grid and downbeats from the onset envelope.
|
||||||
|
//
|
||||||
|
// This runs over the WHOLE track, which is the entire reason to analyse offline:
|
||||||
|
// a causal detector has to converge, and lags for the first several bars of every
|
||||||
|
// section. Here the grid is exact from frame zero, and phase is fitted globally.
|
||||||
|
//
|
||||||
|
// The click track (audio/clicktrack.js) exists to validate this by ear. If the
|
||||||
|
// clicks don't sit on the beat, nothing downstream can be trusted — every timing
|
||||||
|
// artefact in the finished video traces back to this file.
|
||||||
|
|
||||||
|
const MIN_BPM = 60;
|
||||||
|
const MAX_BPM = 200;
|
||||||
|
const PREFERRED_BPM = 124; // log-space centre of the prior; club-ish but broad
|
||||||
|
const PRIOR_WIDTH = 0.85;
|
||||||
|
|
||||||
|
/** Remove the slow-moving floor so autocorrelation sees onsets, not loudness. */
|
||||||
|
function whiten(envelope, fps) {
|
||||||
|
const n = envelope.length;
|
||||||
|
const out = new Float32Array(n);
|
||||||
|
const halfWindow = Math.max(2, Math.round(fps * 0.35));
|
||||||
|
let sum = 0;
|
||||||
|
const queue = [];
|
||||||
|
for (let i = 0; i < n; i++) {
|
||||||
|
queue.push(envelope[i]);
|
||||||
|
sum += envelope[i];
|
||||||
|
if (queue.length > halfWindow * 2 + 1) sum -= queue.shift();
|
||||||
|
const mean = sum / queue.length;
|
||||||
|
out[i] = Math.max(0, envelope[i] - mean);
|
||||||
|
}
|
||||||
|
let peak = 0;
|
||||||
|
for (let i = 0; i < n; i++) if (out[i] > peak) peak = out[i];
|
||||||
|
if (peak > 1e-9) for (let i = 0; i < n; i++) out[i] /= peak;
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
function autocorrelate(signal, minLag, maxLag) {
|
||||||
|
const n = signal.length;
|
||||||
|
const scores = new Float32Array(maxLag + 1);
|
||||||
|
for (let lag = minLag; lag <= maxLag; lag++) {
|
||||||
|
let sum = 0;
|
||||||
|
const limit = n - lag;
|
||||||
|
for (let i = 0; i < limit; i++) sum += signal[i] * signal[i + lag];
|
||||||
|
scores[lag] = limit > 0 ? sum / limit : 0;
|
||||||
|
}
|
||||||
|
return scores;
|
||||||
|
}
|
||||||
|
|
||||||
|
function bpmPrior(bpm) {
|
||||||
|
const x = Math.log2(bpm / PREFERRED_BPM) / PRIOR_WIDTH;
|
||||||
|
return Math.exp(-0.5 * x * x);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Discrete onset peaks: local maxima of the whitened envelope.
|
||||||
|
*
|
||||||
|
* Peak positions are refined to sub-frame precision by parabolic interpolation.
|
||||||
|
* At 60fps a whole-frame quantisation is 16.7ms, and that error accumulates
|
||||||
|
* through the period fit into visible drift by the end of a long track.
|
||||||
|
*/
|
||||||
|
function pickOnsetPeaks(signal) {
|
||||||
|
const peaks = [];
|
||||||
|
let total = 0;
|
||||||
|
for (let i = 1; i < signal.length - 1; i++) {
|
||||||
|
const v = signal[i];
|
||||||
|
if (v < 0.06) continue;
|
||||||
|
if (v < signal[i - 1] || v < signal[i + 1]) continue;
|
||||||
|
|
||||||
|
const a = signal[i - 1], b = v, c = signal[i + 1];
|
||||||
|
const denom = a - 2 * b + c;
|
||||||
|
const shift = Math.abs(denom) > 1e-12 ? (0.5 * (a - c)) / denom : 0;
|
||||||
|
peaks.push({ frame: i + Math.max(-0.5, Math.min(0.5, shift)), strength: v });
|
||||||
|
total += v;
|
||||||
|
}
|
||||||
|
return { peaks, total };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Least-squares fit of the grid to the onsets it already matches.
|
||||||
|
*
|
||||||
|
* The search above only ever tests integer offsets and a discrete set of periods,
|
||||||
|
* which leaves up to half a frame of phase error and a small period error that
|
||||||
|
* compounds — 64 beats into a track a 0.02-frame period error is already 20ms of
|
||||||
|
* drift. Regressing matched onset positions against their beat indices recovers
|
||||||
|
* both to sub-frame precision in one pass.
|
||||||
|
*/
|
||||||
|
function refineByRegression(peaks, period, offset, length) {
|
||||||
|
const tolerance = Math.max(1.5, Math.min(3, period * 0.15));
|
||||||
|
const gridCount = Math.max(1, Math.floor((length - offset) / period));
|
||||||
|
|
||||||
|
let sw = 0, sk = 0, st = 0, skk = 0, skt = 0;
|
||||||
|
let matched = 0;
|
||||||
|
for (const peak of peaks) {
|
||||||
|
const k = Math.round((peak.frame - offset) / period);
|
||||||
|
if (k < 0 || k > gridCount) continue;
|
||||||
|
if (Math.abs(peak.frame - (offset + k * period)) > tolerance) continue;
|
||||||
|
const w = peak.strength;
|
||||||
|
sw += w; sk += w * k; st += w * peak.frame;
|
||||||
|
skk += w * k * k; skt += w * k * peak.frame;
|
||||||
|
matched++;
|
||||||
|
}
|
||||||
|
if (matched < 4 || sw < 1e-9) return { period, offset };
|
||||||
|
|
||||||
|
const denom = sw * skk - sk * sk;
|
||||||
|
if (Math.abs(denom) < 1e-9) return { period, offset };
|
||||||
|
|
||||||
|
const slope = (sw * skt - sk * st) / denom;
|
||||||
|
const intercept = (st - slope * sk) / sw;
|
||||||
|
|
||||||
|
// Reject a fit that has wandered — that means the matching was wrong, not
|
||||||
|
// that the tempo is unusual.
|
||||||
|
if (!isFinite(slope) || !isFinite(intercept)) return { period, offset };
|
||||||
|
if (Math.abs(slope - period) > period * 0.05) return { period, offset };
|
||||||
|
|
||||||
|
return { period: slope, offset: intercept };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* F-measure between a candidate grid and the detected onsets.
|
||||||
|
*
|
||||||
|
* This is what resolves tempo octaves, and a plain "mean energy at grid points"
|
||||||
|
* cannot. A half-tempo grid hits every other kick at FULL strength, so its mean
|
||||||
|
* energy per beat is identical to the true grid's — it only loses on RECALL,
|
||||||
|
* because half the onsets go unexplained. A double-tempo grid has perfect recall
|
||||||
|
* but half its beats land on silence, so it loses on PRECISION. Combining the two
|
||||||
|
* is the only formulation that penalises both errors.
|
||||||
|
*/
|
||||||
|
function gridFScore(peaks, totalStrength, period, offset, length) {
|
||||||
|
if (!peaks.length || period < 2) return 0;
|
||||||
|
const tolerance = Math.max(1.5, Math.min(3, period * 0.15));
|
||||||
|
const gridCount = Math.max(1, Math.floor((length - offset) / period));
|
||||||
|
|
||||||
|
const matchedGrid = new Set();
|
||||||
|
let matchedStrength = 0;
|
||||||
|
|
||||||
|
for (const peak of peaks) {
|
||||||
|
const k = Math.round((peak.frame - offset) / period);
|
||||||
|
if (k < 0 || k > gridCount) continue;
|
||||||
|
const gridFrame = offset + k * period;
|
||||||
|
if (Math.abs(peak.frame - gridFrame) <= tolerance) {
|
||||||
|
matchedStrength += peak.strength;
|
||||||
|
matchedGrid.add(k);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const recall = totalStrength > 1e-9 ? matchedStrength / totalStrength : 0;
|
||||||
|
const precision = matchedGrid.size / gridCount;
|
||||||
|
if (precision + recall < 1e-9) return 0;
|
||||||
|
return (2 * precision * recall) / (precision + recall);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Mean matched onset strength at even vs odd grid positions.
|
||||||
|
*
|
||||||
|
* Unmatched grid points count as zero — a grid point that lands on silence is
|
||||||
|
* evidence against the grid, not a missing sample.
|
||||||
|
*/
|
||||||
|
function alternation(peaks, period, offset, length) {
|
||||||
|
const tolerance = Math.max(1.5, Math.min(3, period * 0.15));
|
||||||
|
const gridCount = Math.max(2, Math.floor((length - offset) / period));
|
||||||
|
const strength = new Float64Array(gridCount);
|
||||||
|
|
||||||
|
for (const peak of peaks) {
|
||||||
|
const k = Math.round((peak.frame - offset) / period);
|
||||||
|
if (k < 0 || k >= gridCount) continue;
|
||||||
|
if (Math.abs(peak.frame - (offset + k * period)) <= tolerance) {
|
||||||
|
strength[k] = Math.max(strength[k], peak.strength);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let evenSum = 0, evenCount = 0, oddSum = 0, oddCount = 0;
|
||||||
|
for (let k = 0; k < gridCount; k++) {
|
||||||
|
if (k % 2 === 0) { evenSum += strength[k]; evenCount++; }
|
||||||
|
else { oddSum += strength[k]; oddCount++; }
|
||||||
|
}
|
||||||
|
const even = evenCount ? evenSum / evenCount : 0;
|
||||||
|
const odd = oddCount ? oddSum / oddCount : 0;
|
||||||
|
const hi = Math.max(even, odd);
|
||||||
|
return { even, odd, ratio: hi > 1e-9 ? Math.min(even, odd) / hi : 1 };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Correct a double-tempo reading.
|
||||||
|
*
|
||||||
|
* Offbeat hi-hats make a double-tempo grid score perfectly on both precision and
|
||||||
|
* recall — every grid point genuinely has an onset — so the F-measure alone
|
||||||
|
* cannot tell 90 BPM with hats from 180 BPM. What separates them is that the
|
||||||
|
* onsets ALTERNATE strong/weak: kick, hat, kick, hat. A systematic alternation
|
||||||
|
* means the real beat is every other grid point. This is the same cue a listener
|
||||||
|
* uses, and it is why the grid is fitted first and metrically interpreted second.
|
||||||
|
*/
|
||||||
|
function correctOctave(peaks, totalStrength, period, offset, length, fps) {
|
||||||
|
let currentPeriod = period;
|
||||||
|
let currentOffset = offset;
|
||||||
|
|
||||||
|
for (let iteration = 0; iteration < 2; iteration++) {
|
||||||
|
const bpm = (60 * fps) / currentPeriod;
|
||||||
|
const halvedBpm = bpm / 2;
|
||||||
|
if (halvedBpm < MIN_BPM) break;
|
||||||
|
|
||||||
|
const alt = alternation(peaks, currentPeriod, currentOffset, length);
|
||||||
|
if (alt.ratio >= 0.62) break;
|
||||||
|
|
||||||
|
const strongIsEven = alt.even >= alt.odd;
|
||||||
|
const nextPeriod = currentPeriod * 2;
|
||||||
|
const nextOffset = strongIsEven ? currentOffset : currentOffset + currentPeriod;
|
||||||
|
|
||||||
|
// Only accept if the slower grid still explains the material well: this
|
||||||
|
// guards against halving genuinely syncopated but correctly-fitted music.
|
||||||
|
const before = gridFScore(peaks, totalStrength, currentPeriod, currentOffset, length);
|
||||||
|
const after = gridFScore(peaks, totalStrength, nextPeriod, nextOffset, length);
|
||||||
|
if (after < before * 0.55) break;
|
||||||
|
|
||||||
|
currentPeriod = nextPeriod;
|
||||||
|
currentOffset = nextOffset;
|
||||||
|
}
|
||||||
|
|
||||||
|
return { period: currentPeriod, offset: currentOffset };
|
||||||
|
}
|
||||||
|
|
||||||
|
function bestOffset(peaks, totalStrength, period, length) {
|
||||||
|
let best = 0;
|
||||||
|
let bestScore = -1;
|
||||||
|
const steps = Math.ceil(period);
|
||||||
|
for (let o = 0; o < steps; o++) {
|
||||||
|
const s = gridFScore(peaks, totalStrength, period, o, length);
|
||||||
|
if (s > bestScore) { bestScore = s; best = o; }
|
||||||
|
}
|
||||||
|
return { offset: best, score: bestScore };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @param {Float32Array} onsetEnvelope per-frame onset strength
|
||||||
|
* @param {number} fps
|
||||||
|
* @returns {{bpm, period, offset, beats, downbeats, confidence, beatsPerBar}}
|
||||||
|
*/
|
||||||
|
export function detectTempo(onsetEnvelope, fps, { beatsPerBar = 4 } = {}) {
|
||||||
|
const signal = whiten(onsetEnvelope, fps);
|
||||||
|
const minLag = Math.floor((60 * fps) / MAX_BPM);
|
||||||
|
const maxLag = Math.ceil((60 * fps) / MIN_BPM);
|
||||||
|
const acf = autocorrelate(signal, minLag, maxLag);
|
||||||
|
|
||||||
|
// Weight autocorrelation by the tempo prior, then take the peak.
|
||||||
|
let bestLag = minLag;
|
||||||
|
let bestValue = -Infinity;
|
||||||
|
for (let lag = minLag; lag <= maxLag; lag++) {
|
||||||
|
const bpm = (60 * fps) / lag;
|
||||||
|
const v = acf[lag] * bpmPrior(bpm);
|
||||||
|
if (v > bestValue) { bestValue = v; bestLag = lag; }
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parabolic refinement around the peak for sub-frame period accuracy.
|
||||||
|
let period = bestLag;
|
||||||
|
if (bestLag > minLag && bestLag < maxLag) {
|
||||||
|
const a = acf[bestLag - 1], b = acf[bestLag], c = acf[bestLag + 1];
|
||||||
|
const denom = a - 2 * b + c;
|
||||||
|
if (Math.abs(denom) > 1e-12) period = bestLag - (0.5 * (c - a)) / denom;
|
||||||
|
}
|
||||||
|
|
||||||
|
const { peaks, total: totalStrength } = pickOnsetPeaks(signal);
|
||||||
|
const length = signal.length;
|
||||||
|
|
||||||
|
// Octave resolution. The prior only breaks near-ties; the F-measure does the
|
||||||
|
// actual work, which is why a 174 BPM track no longer reads as 87.
|
||||||
|
const candidates = [period / 4, period / 2, period, period * 2, period * 4].filter((p) => {
|
||||||
|
const bpm = (60 * fps) / p;
|
||||||
|
return bpm >= MIN_BPM && bpm <= MAX_BPM && p >= 2;
|
||||||
|
});
|
||||||
|
|
||||||
|
let chosen = { period, offset: 0, score: -1 };
|
||||||
|
for (const p of candidates) {
|
||||||
|
const { offset, score } = bestOffset(peaks, totalStrength, p, length);
|
||||||
|
const adjusted = score * (0.6 + 0.4 * bpmPrior((60 * fps) / p));
|
||||||
|
if (adjusted > chosen.score) chosen = { period: p, offset, score: adjusted };
|
||||||
|
}
|
||||||
|
|
||||||
|
period = chosen.period;
|
||||||
|
let offset = chosen.offset;
|
||||||
|
|
||||||
|
// Local refinement of period and offset together — catches a grid that is
|
||||||
|
// right at the start and drifts by the end of a six-minute track.
|
||||||
|
let bestRefined = gridFScore(peaks, totalStrength, period, offset, length);
|
||||||
|
for (let dp = -0.5; dp <= 0.5001; dp += 0.02) {
|
||||||
|
const p = period + dp;
|
||||||
|
if (p < 2) continue;
|
||||||
|
const { offset: o, score } = bestOffset(peaks, totalStrength, p, length);
|
||||||
|
if (score > bestRefined) { bestRefined = score; period = p; offset = o; }
|
||||||
|
}
|
||||||
|
|
||||||
|
// Metrical interpretation, after the grid itself is fitted.
|
||||||
|
({ period, offset } = correctOctave(peaks, totalStrength, period, offset, length, fps));
|
||||||
|
|
||||||
|
// Sub-frame fit last, so it refines the grid we actually committed to.
|
||||||
|
({ period, offset } = refineByRegression(peaks, period, offset, length));
|
||||||
|
|
||||||
|
const bpm = (60 * fps) / period;
|
||||||
|
|
||||||
|
const beats = [];
|
||||||
|
for (let t = offset; t < onsetEnvelope.length; t += period) beats.push(t / fps);
|
||||||
|
|
||||||
|
// Downbeat: of the `beatsPerBar` possible bar phases, the one whose beats
|
||||||
|
// carry the most energy.
|
||||||
|
//
|
||||||
|
// Scored on the RAW envelope, not the whitened one. Whitening subtracts the
|
||||||
|
// local mean, which is exactly the accent information that distinguishes beat
|
||||||
|
// one from the other three — a four-to-the-floor pattern has a kick on every
|
||||||
|
// beat and the only cue is that one of them is louder.
|
||||||
|
let bestPhase = 0;
|
||||||
|
let bestPhaseScore = -1;
|
||||||
|
for (let phase = 0; phase < beatsPerBar; phase++) {
|
||||||
|
let sum = 0;
|
||||||
|
let count = 0;
|
||||||
|
for (let b = phase; b < beats.length; b += beatsPerBar) {
|
||||||
|
const i = Math.round(beats[b] * fps);
|
||||||
|
if (i >= 1 && i < onsetEnvelope.length - 1) {
|
||||||
|
sum += Math.max(onsetEnvelope[i - 1], onsetEnvelope[i], onsetEnvelope[i + 1]);
|
||||||
|
count++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const score = count ? sum / count : 0;
|
||||||
|
if (score > bestPhaseScore) { bestPhaseScore = score; bestPhase = phase; }
|
||||||
|
}
|
||||||
|
const downbeats = [];
|
||||||
|
for (let b = bestPhase; b < beats.length; b += beatsPerBar) downbeats.push(beats[b]);
|
||||||
|
|
||||||
|
// Confidence is the grid's own F-measure: how much of the onset energy the
|
||||||
|
// chosen grid explains, and how many of its beats are actually occupied.
|
||||||
|
const confidence = gridFScore(peaks, totalStrength, period, offset, length);
|
||||||
|
|
||||||
|
return { bpm, period, offset, beats, downbeats, beatsPerBar, barPhase: bestPhase, confidence };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Per-frame phase tracks from the grid.
|
||||||
|
*
|
||||||
|
* `beat` is a decaying spike, but gated by a smoothed loudness envelope so a
|
||||||
|
* drumless breakdown doesn't strobe on a grid that is technically still running.
|
||||||
|
*/
|
||||||
|
export function buildBeatTracks(tempo, frameCount, fps, loudness) {
|
||||||
|
const beatPhase = new Float32Array(frameCount);
|
||||||
|
const barPhase = new Float32Array(frameCount);
|
||||||
|
const phrasePhase = new Float32Array(frameCount);
|
||||||
|
const beat = new Float32Array(frameCount);
|
||||||
|
|
||||||
|
const { period, offset, beatsPerBar } = tempo;
|
||||||
|
const barPeriod = period * beatsPerBar;
|
||||||
|
const phrasePeriod = barPeriod * 8;
|
||||||
|
const barOffset = offset + tempo.barPhase * period;
|
||||||
|
|
||||||
|
// Smoothed loudness gate.
|
||||||
|
const gate = new Float32Array(frameCount);
|
||||||
|
const attack = 0.25, release = 0.02;
|
||||||
|
let g = 0;
|
||||||
|
for (let f = 0; f < frameCount; f++) {
|
||||||
|
const target = loudness ? loudness[f] : 1;
|
||||||
|
g += (target - g) * (target > g ? attack : release);
|
||||||
|
gate[f] = Math.min(1, g * 1.6);
|
||||||
|
}
|
||||||
|
|
||||||
|
const decay = Math.max(1e-3, period * 0.28);
|
||||||
|
for (let f = 0; f < frameCount; f++) {
|
||||||
|
const sinceBeat = ((f - offset) % period + period) % period;
|
||||||
|
beatPhase[f] = sinceBeat / period;
|
||||||
|
barPhase[f] = (((f - barOffset) % barPeriod) + barPeriod) % barPeriod / barPeriod;
|
||||||
|
phrasePhase[f] = (((f - barOffset) % phrasePeriod) + phrasePeriod) % phrasePeriod / phrasePeriod;
|
||||||
|
beat[f] = Math.exp(-sinceBeat / decay) * gate[f];
|
||||||
|
}
|
||||||
|
|
||||||
|
return { beatPhase, barPhase, phrasePhase, beat };
|
||||||
|
}
|
||||||
81
flow-state/src/checks/framework.js
Normal file
81
flow-state/src/checks/framework.js
Normal file
@ -0,0 +1,81 @@
|
|||||||
|
// Minimal check harness. Every phase gate in PLAN.md is a check registered here
|
||||||
|
// and run from checks.html, so "the gate passes" is something you execute rather
|
||||||
|
// than something you assert in a commit message.
|
||||||
|
|
||||||
|
const registry = [];
|
||||||
|
|
||||||
|
export function check(phase, name, fn, options = {}) {
|
||||||
|
registry.push({ phase, name, fn, manual: !!options.manual, slow: !!options.slow });
|
||||||
|
}
|
||||||
|
|
||||||
|
export function allChecks() {
|
||||||
|
return registry;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function runAll({ phases = null, onResult = null, skipSlow = false } = {}) {
|
||||||
|
const results = [];
|
||||||
|
for (const entry of registry) {
|
||||||
|
if (phases && !phases.includes(entry.phase)) continue;
|
||||||
|
if (skipSlow && entry.slow) continue;
|
||||||
|
|
||||||
|
const started = Date.now();
|
||||||
|
let result;
|
||||||
|
try {
|
||||||
|
const r = await entry.fn();
|
||||||
|
result = {
|
||||||
|
phase: entry.phase,
|
||||||
|
name: entry.name,
|
||||||
|
pass: r === true || (r && r.pass !== false),
|
||||||
|
detail: (r && r.detail) || '',
|
||||||
|
manual: entry.manual,
|
||||||
|
ms: Date.now() - started,
|
||||||
|
};
|
||||||
|
} catch (err) {
|
||||||
|
result = {
|
||||||
|
phase: entry.phase,
|
||||||
|
name: entry.name,
|
||||||
|
pass: false,
|
||||||
|
detail: `threw: ${err && err.message ? err.message : String(err)}`,
|
||||||
|
manual: entry.manual,
|
||||||
|
ms: Date.now() - started,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
results.push(result);
|
||||||
|
if (onResult) onResult(result);
|
||||||
|
}
|
||||||
|
return results;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function summarize(results) {
|
||||||
|
const total = results.length;
|
||||||
|
const failed = results.filter((r) => !r.pass && !r.manual);
|
||||||
|
const manual = results.filter((r) => r.manual);
|
||||||
|
return {
|
||||||
|
total,
|
||||||
|
passed: results.filter((r) => r.pass).length,
|
||||||
|
failed: failed.length,
|
||||||
|
manual: manual.length,
|
||||||
|
ok: failed.length === 0,
|
||||||
|
failures: failed.map((r) => `[P${r.phase}] ${r.name}: ${r.detail}`),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Assertion helpers that produce useful detail strings rather than bare booleans. */
|
||||||
|
export function expect(condition, detail) {
|
||||||
|
return { pass: !!condition, detail };
|
||||||
|
}
|
||||||
|
|
||||||
|
export function expectClose(actual, expected, tolerance, label) {
|
||||||
|
const diff = Math.abs(actual - expected);
|
||||||
|
return {
|
||||||
|
pass: diff <= tolerance,
|
||||||
|
detail: `${label}: ${actual.toFixed(6)} vs ${expected.toFixed(6)} (Δ${diff.toFixed(6)}, tol ${tolerance})`,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export function expectBelow(actual, limit, label) {
|
||||||
|
return {
|
||||||
|
pass: actual <= limit,
|
||||||
|
detail: `${label}: ${typeof actual === 'number' ? actual.toFixed(6) : actual} (limit ${limit})`,
|
||||||
|
};
|
||||||
|
}
|
||||||
73
flow-state/src/checks/main.js
Normal file
73
flow-state/src/checks/main.js
Normal file
@ -0,0 +1,73 @@
|
|||||||
|
import { runAll, summarize, allChecks } from './framework.js';
|
||||||
|
import { runSceneGate } from './scene-gate.js';
|
||||||
|
|
||||||
|
// Registering a phase's checks is a side effect of importing it.
|
||||||
|
import './phase0.js';
|
||||||
|
import './phase1.js';
|
||||||
|
import './phase2.js';
|
||||||
|
import './phase3.js';
|
||||||
|
import './phase4.js';
|
||||||
|
import './phase5.js';
|
||||||
|
import './phase6.js';
|
||||||
|
import './phase7.js';
|
||||||
|
import './phase8.js';
|
||||||
|
import './phase9.js';
|
||||||
|
import './phase10.js';
|
||||||
|
import './phase11.js';
|
||||||
|
|
||||||
|
const out = document.getElementById('results');
|
||||||
|
const summaryEl = document.getElementById('summary');
|
||||||
|
|
||||||
|
function row(result) {
|
||||||
|
const el = document.createElement('div');
|
||||||
|
el.className = 'row ' + (result.pass ? 'pass' : result.manual ? 'manual' : 'fail');
|
||||||
|
el.innerHTML = `
|
||||||
|
<span class="badge">${result.pass ? 'PASS' : 'FAIL'}</span>
|
||||||
|
<span class="phase">P${result.phase}</span>
|
||||||
|
<span class="name">${result.name}</span>
|
||||||
|
<span class="ms">${result.ms}ms</span>
|
||||||
|
<div class="detail">${result.detail || ''}</div>`;
|
||||||
|
out.appendChild(el);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
const params = new URLSearchParams(location.search);
|
||||||
|
|
||||||
|
// Single-scene mode: the per-scene acceptance battery for one scene, in a
|
||||||
|
// form that is cheap to run and cheap to read. This is the loop you are in
|
||||||
|
// while writing a scene; running all ten phases to find out whether one new
|
||||||
|
// shader is alive is both slow and a page of output to wade through.
|
||||||
|
const sceneArg = params.get('scene');
|
||||||
|
if (sceneArg) {
|
||||||
|
summaryEl.textContent = `gating "${sceneArg}"…`;
|
||||||
|
const { ok, lines } = runSceneGate(sceneArg);
|
||||||
|
out.innerHTML = `<pre class="scene-gate">${lines.join('\n')}</pre>`;
|
||||||
|
summaryEl.textContent = `${sceneArg}: ${ok ? 'PASS' : 'FAIL'} — ` +
|
||||||
|
`${lines.filter((l) => l.startsWith('PASS')).length}/${lines.length} criteria`;
|
||||||
|
summaryEl.className = ok ? 'ok' : 'bad';
|
||||||
|
window.__CHECKS__ = { scene: sceneArg, ok, lines };
|
||||||
|
window.__CHECKS_DONE__ = true;
|
||||||
|
console.log('[scene-gate]', sceneArg, ok ? 'PASS' : 'FAIL', '\n' + lines.join('\n'));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const phaseArg = params.get('phase');
|
||||||
|
const phases = phaseArg ? phaseArg.split(',').map(Number) : null;
|
||||||
|
const skipSlow = params.get('slow') !== '1';
|
||||||
|
|
||||||
|
summaryEl.textContent = `running ${allChecks().length} checks…`;
|
||||||
|
|
||||||
|
const results = await runAll({ phases, skipSlow, onResult: row });
|
||||||
|
const s = summarize(results);
|
||||||
|
|
||||||
|
summaryEl.textContent =
|
||||||
|
`${s.passed}/${s.total} passed · ${s.failed} failed` + (skipSlow ? ' · slow checks skipped (?slow=1 to include)' : '');
|
||||||
|
summaryEl.className = s.ok ? 'ok' : 'bad';
|
||||||
|
|
||||||
|
// Read by the browser automation that drives these gates.
|
||||||
|
window.__CHECKS__ = { results, summary: s };
|
||||||
|
window.__CHECKS_DONE__ = true;
|
||||||
|
console.log('[checks]', JSON.stringify(s, null, 2));
|
||||||
|
}
|
||||||
|
|
||||||
|
main();
|
||||||
163
flow-state/src/checks/phase0.js
Normal file
163
flow-state/src/checks/phase0.js
Normal file
@ -0,0 +1,163 @@
|
|||||||
|
// Phase 0 gate — the determinism spine.
|
||||||
|
//
|
||||||
|
// These are cheap now and impossible to retrofit later: every phase after this
|
||||||
|
// one assumes that frame N is a pure function of (seed, params, frame index).
|
||||||
|
|
||||||
|
import { check, expect, expectBelow } from './framework.js';
|
||||||
|
import { Engine } from '../engine/Engine.js';
|
||||||
|
import { frameDistance, downsample } from '../engine/hash.js';
|
||||||
|
import { Rng } from '../engine/rng.js';
|
||||||
|
import { nebula } from '../scenes/shader/nebula.js';
|
||||||
|
import { defaultValues } from '../params/schema.js';
|
||||||
|
|
||||||
|
const TEST_PALETTE = [
|
||||||
|
[0.05, 0.02, 0.15],
|
||||||
|
[0.85, 0.15, 0.55],
|
||||||
|
[0.15, 0.75, 0.95],
|
||||||
|
[0.98, 0.85, 0.35],
|
||||||
|
];
|
||||||
|
|
||||||
|
function makeEngine(width = 320, height = 180) {
|
||||||
|
const engine = new Engine({ width, height });
|
||||||
|
engine.timeline.setDuration(60);
|
||||||
|
engine.setLayerSpecs([{
|
||||||
|
module: nebula,
|
||||||
|
params: defaultValues(nebula),
|
||||||
|
seed: 12345,
|
||||||
|
opacity: 1,
|
||||||
|
blend: 'normal',
|
||||||
|
palette: TEST_PALETTE,
|
||||||
|
}]);
|
||||||
|
return engine;
|
||||||
|
}
|
||||||
|
|
||||||
|
check(0, 'repeat run produces identical frames', () => {
|
||||||
|
const engine = makeEngine();
|
||||||
|
try {
|
||||||
|
const a = engine.hashRun(0, 300);
|
||||||
|
const b = engine.hashRun(0, 300);
|
||||||
|
const mismatches = a.filter((h, i) => h !== b[i]).length;
|
||||||
|
return expect(mismatches === 0, `${mismatches}/300 frames differed between runs`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(0, 'fresh engine reproduces the same frames', () => {
|
||||||
|
// A second Engine instance must agree with the first: proves nothing is
|
||||||
|
// carried in module-level or GPU state between constructions.
|
||||||
|
const a = makeEngine();
|
||||||
|
const b = makeEngine();
|
||||||
|
try {
|
||||||
|
const ha = a.hashRun(0, 60);
|
||||||
|
const hb = b.hashRun(0, 60);
|
||||||
|
const mismatches = ha.filter((h, i) => h !== hb[i]).length;
|
||||||
|
return expect(mismatches === 0, `${mismatches}/60 frames differed between engines`);
|
||||||
|
} finally {
|
||||||
|
a.dispose(); b.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(0, 'simulated dropped frames change nothing', () => {
|
||||||
|
// Renders the same frame indices but with irregular gaps in between, the way
|
||||||
|
// a stuttering browser would. Because dt is fixed and features are indexed by
|
||||||
|
// frame, output must be byte-identical to the smooth run.
|
||||||
|
const engine = makeEngine();
|
||||||
|
try {
|
||||||
|
const smooth = engine.hashRun(0, 120);
|
||||||
|
|
||||||
|
engine.compositor.reset();
|
||||||
|
const stuttered = [];
|
||||||
|
for (let i = 0; i < 120; i++) {
|
||||||
|
// Burn some GPU work between frames without advancing the timeline.
|
||||||
|
if (i % 7 === 0) engine.renderFrame(i);
|
||||||
|
const target = engine.renderFrame(i);
|
||||||
|
stuttered.push(engine.hashCurrent(target));
|
||||||
|
}
|
||||||
|
const mismatches = smooth.filter((h, i) => h !== stuttered[i]).length;
|
||||||
|
return expect(mismatches === 0, `${mismatches}/120 frames differed under simulated stutter`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(0, 'seek equals sequential playback', () => {
|
||||||
|
const engine = makeEngine();
|
||||||
|
try {
|
||||||
|
const sequential = engine.hashRun(0, 100);
|
||||||
|
engine.compositor.reset();
|
||||||
|
const direct = engine.hashCurrent(engine.renderFrame(99));
|
||||||
|
// With feedback off (Phase 0 default) a direct seek must match exactly.
|
||||||
|
return expect(direct === sequential[99],
|
||||||
|
`seek→99 ${direct} vs sequential ${sequential[99]}`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(0, 'resolution independence (320x180 vs 1280x720)', () => {
|
||||||
|
const small = new Engine({ width: 320, height: 180 });
|
||||||
|
const large = new Engine({ width: 1280, height: 720 });
|
||||||
|
try {
|
||||||
|
for (const e of [small, large]) {
|
||||||
|
e.timeline.setDuration(60);
|
||||||
|
e.setLayerSpecs([{
|
||||||
|
module: nebula, params: defaultValues(nebula), seed: 12345,
|
||||||
|
opacity: 1, blend: 'normal', palette: TEST_PALETTE,
|
||||||
|
}]);
|
||||||
|
}
|
||||||
|
const frames = [10, 90, 200];
|
||||||
|
let worst = 0;
|
||||||
|
for (const f of frames) {
|
||||||
|
small.compositor.reset(); large.compositor.reset();
|
||||||
|
const st = small.renderFrame(f);
|
||||||
|
const lt = large.renderFrame(f);
|
||||||
|
const sp = Uint8Array.from(small.readPixels(st));
|
||||||
|
const lp = Uint8Array.from(large.readPixels(lt));
|
||||||
|
const reduced = downsample(lp, 1280, 720, 4);
|
||||||
|
worst = Math.max(worst, frameDistance(sp, reduced.pixels));
|
||||||
|
}
|
||||||
|
// 4x downsampling of a noise-bearing shader will never be exact; this
|
||||||
|
// threshold catches genuine pixel-space dependence, not filtering error.
|
||||||
|
return expectBelow(worst, 0.06, 'worst mean channel distance');
|
||||||
|
} finally {
|
||||||
|
small.dispose(); large.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(0, 'seeded rng is reproducible and independent per stream', () => {
|
||||||
|
const a = new Rng(42);
|
||||||
|
const b = new Rng(42);
|
||||||
|
const seqA = Array.from({ length: 500 }, () => a.next());
|
||||||
|
const seqB = Array.from({ length: 500 }, () => b.next());
|
||||||
|
if (seqA.some((v, i) => v !== seqB[i])) return expect(false, 'same seed diverged');
|
||||||
|
|
||||||
|
const parent = new Rng(7);
|
||||||
|
const c1 = parent.fork('layer:0').next();
|
||||||
|
const parent2 = new Rng(7);
|
||||||
|
const c2 = parent2.fork('layer:0').next();
|
||||||
|
const c3 = parent2.fork('layer:1').next();
|
||||||
|
if (c1 !== c2) return expect(false, 'fork(label) not stable');
|
||||||
|
if (c1 === c3) return expect(false, 'different fork labels collided');
|
||||||
|
|
||||||
|
const inRange = seqA.every((v) => v >= 0 && v < 1);
|
||||||
|
return expect(inRange, `500 draws reproducible, forks independent, range ok=${inRange}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(0, 'compositor reset clears all history', () => {
|
||||||
|
const engine = makeEngine();
|
||||||
|
try {
|
||||||
|
engine.compositor.setFeedback({ amount: 0.8, decay: 0.95 });
|
||||||
|
engine.hashRun(0, 30);
|
||||||
|
const afterHistory = engine.hashRun(0, 30); // hashRun resets first
|
||||||
|
const fresh = makeEngine();
|
||||||
|
fresh.compositor.setFeedback({ amount: 0.8, decay: 0.95 });
|
||||||
|
const freshHashes = fresh.hashRun(0, 30);
|
||||||
|
fresh.dispose();
|
||||||
|
const mismatches = afterHistory.filter((h, i) => h !== freshHashes[i]).length;
|
||||||
|
return expect(mismatches === 0,
|
||||||
|
`${mismatches}/30 frames differed after reset — feedback history leaked`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
150
flow-state/src/checks/phase1.js
Normal file
150
flow-state/src/checks/phase1.js
Normal file
@ -0,0 +1,150 @@
|
|||||||
|
// Phase 1 gate — the offline audio pipeline feeding the renderer.
|
||||||
|
//
|
||||||
|
// The numeric correctness of tempo and segmentation is covered by node tests
|
||||||
|
// against synthetic ground truth (test/audio.test.js) and, for real material, by
|
||||||
|
// the click track. What can only be checked here is the join: that the frame-indexed
|
||||||
|
// table reaches the shader, and that driving the clock from audio produces exactly
|
||||||
|
// the same frames as counting them.
|
||||||
|
|
||||||
|
import { check, expect, expectBelow } from './framework.js';
|
||||||
|
import { Engine } from '../engine/Engine.js';
|
||||||
|
import { FeatureTrack, featureProviderFor } from '../audio/FeatureTrack.js';
|
||||||
|
import { synthesizeSectioned } from '../audio/synth.js';
|
||||||
|
import { nebula } from '../scenes/shader/nebula.js';
|
||||||
|
import { defaultValues } from '../params/schema.js';
|
||||||
|
|
||||||
|
const PALETTE = [[0.05, 0.02, 0.15], [0.85, 0.15, 0.55], [0.15, 0.75, 0.95], [0.98, 0.85, 0.35]];
|
||||||
|
|
||||||
|
let cachedTrack = null;
|
||||||
|
export function testTrack() {
|
||||||
|
if (!cachedTrack) {
|
||||||
|
const buffer = synthesizeSectioned({ bpm: 128, duration: 90, changeAt: 45 });
|
||||||
|
cachedTrack = FeatureTrack.fromAudioBuffer(buffer, { fps: 60 });
|
||||||
|
}
|
||||||
|
return cachedTrack;
|
||||||
|
}
|
||||||
|
|
||||||
|
function makeEngine(track, width = 256, height = 144) {
|
||||||
|
const engine = new Engine({ width, height });
|
||||||
|
engine.timeline.setDuration(track.duration);
|
||||||
|
engine.setFeatureProvider(featureProviderFor(track));
|
||||||
|
engine.setLayerSpecs([{
|
||||||
|
module: nebula, params: defaultValues(nebula), seed: 4242,
|
||||||
|
opacity: 1, blend: 'normal', palette: PALETTE,
|
||||||
|
}]);
|
||||||
|
return engine;
|
||||||
|
}
|
||||||
|
|
||||||
|
check(1, 'feature track builds and every row is finite and in range', () => {
|
||||||
|
const track = testTrack();
|
||||||
|
let bad = 0;
|
||||||
|
for (let f = 0; f < track.frameCount; f += 7) {
|
||||||
|
const row = track.at(f);
|
||||||
|
for (const v of Object.values(row)) {
|
||||||
|
if (!Number.isFinite(v) || v < 0 || v > 1) { bad++; break; }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(bad === 0,
|
||||||
|
`${bad} bad rows of ${Math.ceil(track.frameCount / 7)} sampled · ` +
|
||||||
|
`${track.frameCount} frames · ${track.sections.length} sections · ` +
|
||||||
|
`${track.summary.bpm.toFixed(1)} BPM · confidence ${track.tempo.confidence.toFixed(2)}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(1, 'audio-driven clock and fixed-step counting agree', () => {
|
||||||
|
// The core preview/export parity claim. Realtime maps currentTime to a frame
|
||||||
|
// index; export counts frames. Both must land on identical images.
|
||||||
|
const track = testTrack();
|
||||||
|
const engine = makeEngine(track);
|
||||||
|
try {
|
||||||
|
const fixed = engine.hashRun(1000, 100);
|
||||||
|
|
||||||
|
engine.compositor.reset();
|
||||||
|
const driven = [];
|
||||||
|
for (let i = 0; i < 100; i++) {
|
||||||
|
const frame = 1000 + i;
|
||||||
|
// Jittered playback position within the frame's window, as a real
|
||||||
|
// audio element would report it.
|
||||||
|
const jitter = (((i * 7919) % 1000) / 1000) * 0.9;
|
||||||
|
engine.timeline.syncToAudio((frame + jitter) / 60);
|
||||||
|
const target = engine.renderCurrent();
|
||||||
|
driven.push(engine.hashCurrent(target));
|
||||||
|
}
|
||||||
|
const mismatches = fixed.filter((h, i) => h !== driven[i]).length;
|
||||||
|
return expect(mismatches === 0, `${mismatches}/100 frames differed`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(1, 'features actually reach the shader', () => {
|
||||||
|
// A scene wired to a table it never reads would pass every other check here.
|
||||||
|
const track = testTrack();
|
||||||
|
const engine = makeEngine(track);
|
||||||
|
try {
|
||||||
|
const quiet = track.sections.reduce((a, b) => (a.energy < b.energy ? a : b));
|
||||||
|
const loud = track.sections.reduce((a, b) => (a.energy > b.energy ? a : b));
|
||||||
|
|
||||||
|
engine.compositor.reset();
|
||||||
|
const quietHash = engine.hashCurrent(engine.renderFrame(quiet.startFrame + 60));
|
||||||
|
engine.compositor.reset();
|
||||||
|
const loudHash = engine.hashCurrent(engine.renderFrame(loud.startFrame + 60));
|
||||||
|
|
||||||
|
return expect(quietHash !== loudHash,
|
||||||
|
`quiet ${quiet.kind} ${quietHash} vs loud ${loud.kind} ${loudHash}`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(1, 'section-boundary seek is exact without warm-up', () => {
|
||||||
|
const track = testTrack();
|
||||||
|
const engine = makeEngine(track);
|
||||||
|
try {
|
||||||
|
const boundary = track.sections[1] ? track.sections[1].startFrame : 600;
|
||||||
|
const sequential = engine.hashRun(boundary, 3);
|
||||||
|
engine.compositor.reset();
|
||||||
|
const direct = engine.hashCurrent(engine.renderFrame(boundary));
|
||||||
|
return expect(direct === sequential[0], `direct ${direct} vs sequential ${sequential[0]}`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(1, 'mid-section seek converges under warm-up with feedback active', () => {
|
||||||
|
// With feedback on, an arbitrary seek can only converge, not match exactly.
|
||||||
|
// The gate is that warm-up gets it visually indistinguishable — the documented
|
||||||
|
// behaviour in PLAN.md §6.
|
||||||
|
const track = testTrack();
|
||||||
|
const engine = makeEngine(track);
|
||||||
|
try {
|
||||||
|
engine.compositor.setFeedback({ amount: 0.7, decay: 0.92, zoom: 0.99 });
|
||||||
|
|
||||||
|
const targetFrame = 1500;
|
||||||
|
engine.compositor.reset();
|
||||||
|
for (let f = 1200; f < targetFrame; f++) engine.renderFrame(f);
|
||||||
|
const sequentialPixels = Uint8Array.from(
|
||||||
|
engine.readPixels(engine.renderFrame(targetFrame)),
|
||||||
|
);
|
||||||
|
|
||||||
|
engine.warmUp(targetFrame, 120);
|
||||||
|
const warmedPixels = Uint8Array.from(engine.readPixels(engine.renderCurrent()));
|
||||||
|
|
||||||
|
let sum = 0;
|
||||||
|
for (let i = 0; i < sequentialPixels.length; i += 4) {
|
||||||
|
sum += Math.abs(sequentialPixels[i] - warmedPixels[i]);
|
||||||
|
}
|
||||||
|
const distance = sum / (sequentialPixels.length / 4) / 255;
|
||||||
|
return expectBelow(distance, 0.02, 'mean red-channel distance after warm-up');
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(1, 'analysis of a five-minute track stays within budget', () => {
|
||||||
|
const buffer = synthesizeSectioned({ bpm: 128, duration: 300, changeAt: 150 });
|
||||||
|
const started = Date.now();
|
||||||
|
const track = FeatureTrack.fromAudioBuffer(buffer, { fps: 60 });
|
||||||
|
const elapsed = (Date.now() - started) / 1000;
|
||||||
|
return expect(elapsed < 6,
|
||||||
|
`${elapsed.toFixed(2)}s for a 5-minute track (${track.frameCount} frames)`);
|
||||||
|
}, { slow: true });
|
||||||
408
flow-state/src/checks/phase10.js
Normal file
408
flow-state/src/checks/phase10.js
Normal file
@ -0,0 +1,408 @@
|
|||||||
|
// Phase 10 gate — variety.
|
||||||
|
//
|
||||||
|
// Phase 8 gave a track more cuts, Phase 9 gave it a coherent identity. Watching
|
||||||
|
// several tracks side by side exposed what neither addressed: the SAME scene
|
||||||
|
// cast in two different videos looked like the same footage twice. Section bias
|
||||||
|
// is nearly identical between two tracks' drops, so both sampled their params
|
||||||
|
// around the same centre, and the library's own averageness did the rest.
|
||||||
|
//
|
||||||
|
// Three answers, and this gate is what holds them honest:
|
||||||
|
//
|
||||||
|
// temperament — a per-track hand on every parameter dial (Personality.js)
|
||||||
|
// overlays — a second full scene composited over the first, sometimes
|
||||||
|
// palette — a wider reachable colour space, so two tracks differ on
|
||||||
|
// colour before they differ on anything else
|
||||||
|
//
|
||||||
|
// Two later additions are held here for the same reason. GRAIN was in every
|
||||||
|
// video — applied unconditionally per scene and again in the grade, with only
|
||||||
|
// its amount varying — so it is now a treatment most tracks go without and no
|
||||||
|
// two grainy tracks wear the same way (look/grain.js). And TEMPO now dominates
|
||||||
|
// how fast a scene animates, because a slow song was getting scenes that
|
||||||
|
// skittered over it.
|
||||||
|
//
|
||||||
|
// The hard part of testing "variety" is that it is a property of a POPULATION,
|
||||||
|
// not of one render. Every check here therefore samples many tracks and asks
|
||||||
|
// about the spread, never about a single value.
|
||||||
|
|
||||||
|
import { check, expect } from './framework.js';
|
||||||
|
import { Engine } from '../engine/Engine.js';
|
||||||
|
import { scenes } from '../scenes/registry.js';
|
||||||
|
import { generateLook } from '../look/LookGenerator.js';
|
||||||
|
import { AudioPalette, generateUsablePalette, relativeLuminance } from '../look/palette.js';
|
||||||
|
import { Rng } from '../engine/rng.js';
|
||||||
|
import { FeatureTrack, featureProviderFor } from '../audio/FeatureTrack.js';
|
||||||
|
import { synthesizeSectioned } from '../audio/synth.js';
|
||||||
|
import { sampleValues } from '../params/schema.js';
|
||||||
|
import { frameDistance, frameLuminance } from '../engine/hash.js';
|
||||||
|
import { battery as timbreBattery } from './phase3.js';
|
||||||
|
import { grainEnvelope } from '../look/grain.js';
|
||||||
|
import { signatureUniforms } from '../look/Personality.js';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Several tracks that genuinely differ in what they sound like — the population
|
||||||
|
* every check here measures. Synthetic, so the gate does not depend on assets.
|
||||||
|
*/
|
||||||
|
let cachedBattery = null;
|
||||||
|
function battery() {
|
||||||
|
if (!cachedBattery) {
|
||||||
|
cachedBattery = [
|
||||||
|
{ name: 'slow ambient', buffer: synthesizeSectioned({ bpm: 84, duration: 110, changeAt: 55 }) },
|
||||||
|
{ name: 'mid house', buffer: synthesizeSectioned({ bpm: 122, duration: 110, changeAt: 50 }) },
|
||||||
|
{ name: 'fast techno', buffer: synthesizeSectioned({ bpm: 148, duration: 110, changeAt: 45 }) },
|
||||||
|
{ name: 'broken beat', buffer: synthesizeSectioned({ bpm: 104, duration: 110, changeAt: 70 }) },
|
||||||
|
].map((t) => ({ ...t, track: FeatureTrack.fromAudioBuffer(t.buffer, { fps: 60 }) }));
|
||||||
|
}
|
||||||
|
return cachedBattery;
|
||||||
|
}
|
||||||
|
|
||||||
|
function makeEngine(track, width = 160, height = 90) {
|
||||||
|
const engine = new Engine({ width, height });
|
||||||
|
engine.timeline.setDuration(track.duration);
|
||||||
|
engine.setFeatureProvider(featureProviderFor(track));
|
||||||
|
return engine;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Every layer stack in a look, flattened. */
|
||||||
|
const stacksOf = (look) => look.sections.flatMap((s) => s.variants || [s.layers]);
|
||||||
|
|
||||||
|
check(10, 'two tracks do not share a temperament', () => {
|
||||||
|
// The per-track hand on the dials. If these collapsed toward one value the
|
||||||
|
// whole mechanism would be decorative, and the symptom — every video
|
||||||
|
// sampling around the library average — is exactly what it was built for.
|
||||||
|
// Sampled over several seeds per track rather than one. Four samples of a
|
||||||
|
// ±0.8 draw can land close together by luck, and a gate that fails on that
|
||||||
|
// is measuring the seeds, not the mechanism. What the mechanism has to
|
||||||
|
// deliver is both: a wide population AND no two tracks landing on the same
|
||||||
|
// hand — so both are asserted.
|
||||||
|
const t = [];
|
||||||
|
battery().forEach(({ track }, i) => {
|
||||||
|
for (let s = 0; s < 3; s++) t.push(generateLook(track, { seed: 100 + i * 7919 + s * 104729 }).personality.temperament);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Measured as a fraction of each dial's own range, because they are not the
|
||||||
|
// same width — extremity spans 0.5 in total and intensity over three times
|
||||||
|
// that, so one absolute floor would be either trivial or unreachable.
|
||||||
|
const dialRange = { intensity: 1.7, pace: 1.6, extremity: 0.5 };
|
||||||
|
const keys = Object.keys(dialRange);
|
||||||
|
const spread = (key) => Math.max(...t.map((x) => x[key])) - Math.min(...t.map((x) => x[key]));
|
||||||
|
const worst = Math.min(...keys.map((k) => spread(k) / dialRange[k]));
|
||||||
|
|
||||||
|
let closest = Infinity;
|
||||||
|
for (let a = 0; a < t.length; a++) {
|
||||||
|
for (let b = a + 1; b < t.length; b++) {
|
||||||
|
closest = Math.min(closest, Math.max(...keys.map((k) => Math.abs(t[a][k] - t[b][k]))));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return expect(worst > 0.5 && closest > 0.02,
|
||||||
|
`${t.length} tracks · worst dial covers ${(worst * 100).toFixed(0)}% of its range · ` +
|
||||||
|
`spread intensity ${spread('intensity').toFixed(2)} · ` +
|
||||||
|
`pace ${spread('pace').toFixed(2)} · detail ${spread('detail').toFixed(2)} · ` +
|
||||||
|
`extremity ${spread('extremity').toFixed(2)} · closest pair ${closest.toFixed(3)}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(10, 'one scene looks different in two different videos', () => {
|
||||||
|
// The complaint, as a number. Take a scene, render it with the parameters
|
||||||
|
// and personality two different tracks gave it, and require the images to
|
||||||
|
// actually differ — far above the 1/255 the determinism checks treat as
|
||||||
|
// noise, but far below "unrecognisable". The same scene should still be
|
||||||
|
// itself; it just should not be the same footage.
|
||||||
|
const looks = battery().map(({ track }, i) => ({
|
||||||
|
track,
|
||||||
|
look: generateLook(track, { seed: 500 + i * 6841 }),
|
||||||
|
}));
|
||||||
|
|
||||||
|
const distances = [];
|
||||||
|
const problems = [];
|
||||||
|
|
||||||
|
for (const module of scenes.filter((m) => m.kind === 'fragment').slice(0, 12)) {
|
||||||
|
const engine = makeEngine(looks[0].track);
|
||||||
|
try {
|
||||||
|
const frames = looks.map(({ look }) => {
|
||||||
|
// Sample this scene the way each track's own generator would.
|
||||||
|
// If the track did not cast it, sample it anyway with that
|
||||||
|
// track's temperament — the question is what this scene WOULD
|
||||||
|
// look like in that video, and falling back to defaults would
|
||||||
|
// compare two identical parameter sets and prove nothing.
|
||||||
|
const stack = stacksOf(look).find((s) => s[0].module === module);
|
||||||
|
const params = stack ? stack[0].params : sampleValues(
|
||||||
|
module,
|
||||||
|
new Rng(look.seed ^ 0x51ed270b),
|
||||||
|
look.sections[0].bias,
|
||||||
|
look.personality.temperament,
|
||||||
|
);
|
||||||
|
engine.setLayerSpecs([{
|
||||||
|
module,
|
||||||
|
params,
|
||||||
|
seed: look.seed & 0x7fffffff,
|
||||||
|
opacity: 1,
|
||||||
|
blend: 'normal',
|
||||||
|
palette: look.palette,
|
||||||
|
personality: look.personality,
|
||||||
|
}]);
|
||||||
|
engine.prime(300);
|
||||||
|
engine.compositor.reset();
|
||||||
|
return Uint8Array.from(engine.readPixels(engine.renderFrame(300)));
|
||||||
|
});
|
||||||
|
|
||||||
|
for (let i = 0; i < frames.length; i++) {
|
||||||
|
for (let j = i + 1; j < frames.length; j++) {
|
||||||
|
// Relative to how much image there is, not absolute.
|
||||||
|
// frameDistance averages over every pixel, and most scenes
|
||||||
|
// here are mostly dark — two genuinely different renders of
|
||||||
|
// a sparse scene (25 bars versus 53) score under 0.02 in
|
||||||
|
// absolute terms simply because the black background agrees
|
||||||
|
// with itself. Dividing by the images' own brightness asks
|
||||||
|
// the question that was meant: how different is this, as a
|
||||||
|
// fraction of what is actually on screen.
|
||||||
|
const brightness = Math.max(1e-3,
|
||||||
|
(frameLuminance(frames[i]) + frameLuminance(frames[j])) * 0.5);
|
||||||
|
const d = frameDistance(frames[i], frames[j]) / brightness;
|
||||||
|
distances.push(d);
|
||||||
|
if (d < 0.12) problems.push(`${module.name}: ${d.toFixed(3)}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const mean = distances.reduce((a, b) => a + b, 0) / Math.max(1, distances.length);
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.slice(0, 4).join(' · ') ||
|
||||||
|
`${distances.length} cross-track pairs, mean relative distance ${mean.toFixed(2)}, ` +
|
||||||
|
`closest ${Math.min(...distances).toFixed(2)} (floor 0.12)`);
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(10, 'overlays happen sometimes and not always', () => {
|
||||||
|
// A second scene over the first is the variation valve. Always-on would read
|
||||||
|
// as permanently cluttered and never-on is the state this fixed, so the
|
||||||
|
// check is on the RATE across many tracks rather than on any one stack.
|
||||||
|
let stacks = 0;
|
||||||
|
let withOverlay = 0;
|
||||||
|
const blends = new Set();
|
||||||
|
|
||||||
|
for (const { track } of battery()) {
|
||||||
|
for (let s = 0; s < 6; s++) {
|
||||||
|
const look = generateLook(track, { seed: 900 + s * 5231 });
|
||||||
|
for (const stack of stacksOf(look)) {
|
||||||
|
stacks++;
|
||||||
|
const overlay = stack.slice(1).find((l) => l.module.role !== 'accent');
|
||||||
|
if (overlay) {
|
||||||
|
withOverlay++;
|
||||||
|
blends.add(overlay.blend);
|
||||||
|
if (overlay.blend === 'normal') {
|
||||||
|
// 'normal' would hide the shot entirely rather than
|
||||||
|
// compositing over it — that is what a cut is for.
|
||||||
|
return expect(false, `${overlay.module.name} overlaid with 'normal'`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const rate = withOverlay / Math.max(1, stacks);
|
||||||
|
return expect(rate > 0.05 && rate < 0.55,
|
||||||
|
`${withOverlay}/${stacks} stacks carry an overlay (${(rate * 100).toFixed(0)}%), ` +
|
||||||
|
`blends: ${[...blends].join(', ')}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(10, 'an overlay never hides the shot underneath it', () => {
|
||||||
|
// Opacity is the other half of "composited over" — a 0.9 overlay is a
|
||||||
|
// replacement with extra steps.
|
||||||
|
const problems = [];
|
||||||
|
for (const { track } of battery()) {
|
||||||
|
for (let s = 0; s < 4; s++) {
|
||||||
|
for (const stack of stacksOf(generateLook(track, { seed: 1300 + s * 8641 }))) {
|
||||||
|
for (const layer of stack.slice(1)) {
|
||||||
|
if (layer.module.role === 'accent') continue;
|
||||||
|
if (layer.opacity > 0.6) {
|
||||||
|
problems.push(`${layer.module.name} at ${layer.opacity.toFixed(2)}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0, problems.slice(0, 3).join(' · ') || 'all overlays stay under 0.6');
|
||||||
|
});
|
||||||
|
|
||||||
|
check(10, 'the palette reaches the whole colour wheel', () => {
|
||||||
|
// Hue used to be a one-way sweep from red down to blue, which made violet,
|
||||||
|
// magenta and pink unreachable for every track ever generated — a third of
|
||||||
|
// the wheel the tool simply could not produce. Sampled across many tracks,
|
||||||
|
// every sixth of the wheel should now show up.
|
||||||
|
const buckets = new Array(6).fill(0);
|
||||||
|
const samples = 40;
|
||||||
|
|
||||||
|
for (let i = 0; i < samples; i++) {
|
||||||
|
const rng = new Rng(2000 + i * 7919);
|
||||||
|
const summary = {
|
||||||
|
meanCentroid: 0.2 + (i % 7) * 0.1,
|
||||||
|
meanFlatness: 0.1 + (i % 5) * 0.08,
|
||||||
|
dynamicRange: 0.3 + (i % 4) * 0.15,
|
||||||
|
bpm: 80 + (i % 9) * 12,
|
||||||
|
bandBalance: {
|
||||||
|
sub: 0.2 + (i % 5) * 0.12, low: 0.3 + (i % 3) * 0.2, mid: 0.25 + (i % 4) * 0.1,
|
||||||
|
high: 0.2 + (i % 6) * 0.1, air: 0.15 + (i % 7) * 0.09,
|
||||||
|
},
|
||||||
|
};
|
||||||
|
for (const [r, g, b] of generateUsablePalette(new AudioPalette(summary, rng), 6)) {
|
||||||
|
const max = Math.max(r, g, b);
|
||||||
|
const min = Math.min(r, g, b);
|
||||||
|
if (max - min < 0.08) continue; // greys carry no hue
|
||||||
|
let h;
|
||||||
|
if (max === r) h = ((g - b) / (max - min) + 6) % 6;
|
||||||
|
else if (max === g) h = (b - r) / (max - min) + 2;
|
||||||
|
else h = (r - g) / (max - min) + 4;
|
||||||
|
buckets[Math.floor(h) % 6]++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const empty = buckets.filter((n) => n === 0).length;
|
||||||
|
const names = ['red', 'yellow', 'green', 'cyan', 'blue', 'magenta'];
|
||||||
|
return expect(empty === 0,
|
||||||
|
buckets.map((n, i) => `${names[i]}:${n}`).join(' ') +
|
||||||
|
(empty ? ` — ${empty} sixth(s) unreachable` : ''));
|
||||||
|
});
|
||||||
|
|
||||||
|
check(10, 'two tracks do not get the same palette', () => {
|
||||||
|
// Distinctness, not just coverage: a wheel that is fully reachable but where
|
||||||
|
// every track lands in the same place would pass the check above.
|
||||||
|
//
|
||||||
|
// Measured on the Phase 3 battery rather than this file's, because those
|
||||||
|
// four tracks differ in TIMBRE and this file's differ mostly in tempo.
|
||||||
|
// Palette follows timbre by design, so two tracks that sound alike SHOULD
|
||||||
|
// get similar colour — asking otherwise would be asking the generator to
|
||||||
|
// ignore its own input.
|
||||||
|
const palettes = timbreBattery().map(({ track }) =>
|
||||||
|
generateLook(track, { seed: 61 }).palette); // fixed seed: difference must come from the audio
|
||||||
|
|
||||||
|
let closest = Infinity;
|
||||||
|
for (let i = 0; i < palettes.length; i++) {
|
||||||
|
for (let j = i + 1; j < palettes.length; j++) {
|
||||||
|
let sum = 0;
|
||||||
|
for (let k = 0; k < palettes[i].length; k++) {
|
||||||
|
const a = palettes[i][k], b = palettes[j][k];
|
||||||
|
sum += Math.hypot(a[0] - b[0], a[1] - b[1], a[2] - b[2]);
|
||||||
|
}
|
||||||
|
closest = Math.min(closest, sum / palettes[i].length);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const lums = palettes.map((p) => p.map(relativeLuminance).join(','));
|
||||||
|
return expect(closest > 0.08 && new Set(lums).size === palettes.length,
|
||||||
|
`closest pair mean channel distance ${closest.toFixed(3)} (floor 0.08), ` +
|
||||||
|
`${new Set(lums).size}/${palettes.length} distinct`);
|
||||||
|
});
|
||||||
|
|
||||||
|
// --- grain -----------------------------------------------------------------
|
||||||
|
// The complaint these answer: grain was in every video. It was applied twice
|
||||||
|
// unconditionally — once per scene, once in the grade — so the only thing that
|
||||||
|
// varied between tracks was how much. See look/grain.js.
|
||||||
|
|
||||||
|
check(10, 'most tracks carry no grain at all', () => {
|
||||||
|
const modes = [];
|
||||||
|
for (let i = 0; i < 60; i++) {
|
||||||
|
modes.push(generateLook(battery()[i % 4].track, { seed: 900 + i * 5779 }).grain.mode);
|
||||||
|
}
|
||||||
|
const off = modes.filter((m) => m === 'off').length / modes.length;
|
||||||
|
const distinct = new Set(modes).size;
|
||||||
|
|
||||||
|
// A quarter to two thirds. Never grainy is as much a failure as always
|
||||||
|
// grainy — the treatment has to remain available.
|
||||||
|
return expect(off > 0.25 && off < 0.7 && distinct >= 4,
|
||||||
|
`${(off * 100).toFixed(0)}% of 60 tracks have no grain · ${distinct} modes used: ` +
|
||||||
|
`${[...new Set(modes)].join(', ')}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(10, 'two grainy tracks are not grainy the same way', () => {
|
||||||
|
// Amount alone was never the difference that mattered. Cell size, refresh
|
||||||
|
// rate, mask and chroma are, so the population has to spread across them.
|
||||||
|
const specs = [];
|
||||||
|
for (let i = 0; i < 80 && specs.length < 30; i++) {
|
||||||
|
const g = generateLook(battery()[i % 4].track, { seed: 4100 + i * 7717 }).grain;
|
||||||
|
if (g.mode !== 'off') specs.push(g);
|
||||||
|
}
|
||||||
|
const uniq = (key) => new Set(specs.map((s) => s[key])).size;
|
||||||
|
const signatures = new Set(specs.map((s) => `${s.mode}/${s.scale}/${s.rate}/${s.mask}`)).size;
|
||||||
|
|
||||||
|
return expect(uniq('scale') >= 3 && uniq('rate') >= 3 && uniq('mask') >= 3
|
||||||
|
&& signatures >= specs.length * 0.5,
|
||||||
|
`${specs.length} grainy tracks · ${uniq('scale')} cell sizes · ${uniq('rate')} rates · ` +
|
||||||
|
`${uniq('mask')} masks · ${signatures} distinct treatments`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(10, 'grain that is not constant actually comes and goes', () => {
|
||||||
|
// A 'swell' or 'sections' grain that never reaches zero is just constant
|
||||||
|
// grain with extra steps, and one that never reaches full is decoration.
|
||||||
|
const { track } = battery()[1];
|
||||||
|
const problems = [];
|
||||||
|
let tested = 0;
|
||||||
|
|
||||||
|
for (let i = 0; i < 120 && tested < 6; i++) {
|
||||||
|
const look = generateLook(track, { seed: 7000 + i * 3571 });
|
||||||
|
const g = look.grain;
|
||||||
|
if (g.mode === 'off' || g.mode === 'constant') continue;
|
||||||
|
tested++;
|
||||||
|
|
||||||
|
let lo = Infinity, hi = -Infinity;
|
||||||
|
for (let f = 0; f < track.frameCount; f += 7) {
|
||||||
|
const env = grainEnvelope(g, {
|
||||||
|
time: f / 60,
|
||||||
|
sectionKind: track.sectionAt(f).kind,
|
||||||
|
features: track.at(f),
|
||||||
|
});
|
||||||
|
lo = Math.min(lo, env);
|
||||||
|
hi = Math.max(hi, env);
|
||||||
|
}
|
||||||
|
if (lo > 0.05 || hi < 0.4) problems.push(`${g.mode}: ${lo.toFixed(2)}..${hi.toFixed(2)}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
return expect(tested > 0 && problems.length === 0,
|
||||||
|
problems.length ? problems.join(' · ') : `${tested} time-varying grains all reach 0 and full`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(10, 'a scene that refuses grain never gets any', () => {
|
||||||
|
// `texture: 0` on a module has to survive a track that wants maximum grit,
|
||||||
|
// because the point of it is that crisp line work stays crisp.
|
||||||
|
const gritty = { style: { lineWeight: 0.5, softness: 0.5, texture: 1, symmetry: 1 },
|
||||||
|
shape: { sides: 0, roundness: 0, elongation: 1, tilt: 0 },
|
||||||
|
camera: { driftAngle: 0, driftRate: 0, sway: 0, swayRate: 0, spin: 0, breathe: 0 },
|
||||||
|
space: { horizon: 0.5, depth: 0, washAngle: 0, wash: 0 } };
|
||||||
|
|
||||||
|
const refusing = scenes.filter((m) => m.texture === 0);
|
||||||
|
const leaked = refusing.filter((m) => signatureUniforms(gritty, m).u_sigTexture > 0);
|
||||||
|
const takesIt = signatureUniforms(gritty, scenes.find((m) => m.texture === undefined));
|
||||||
|
|
||||||
|
return expect(refusing.length > 0 && leaked.length === 0 && takesIt.u_sigTexture === 1,
|
||||||
|
`${refusing.length} scenes opt out, ${leaked.length} leaked · ` +
|
||||||
|
`an opted-in scene still gets ${takesIt.u_sigTexture}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
// --- tempo -----------------------------------------------------------------
|
||||||
|
|
||||||
|
check(10, 'a slow track animates slower than a fast one', () => {
|
||||||
|
// Motion used to be mostly energy with tempo as a small correction, so a
|
||||||
|
// 70bpm ballad's drop asked its scenes for nearly as much speed as a
|
||||||
|
// 150bpm track's — and the scenes obliged, over a song that was not moving.
|
||||||
|
const slow = generateLook(battery()[0].track, { seed: 31 }); // 84bpm
|
||||||
|
const fast = generateLook(battery()[2].track, { seed: 31 }); // 148bpm
|
||||||
|
|
||||||
|
const rateMean = (look) => {
|
||||||
|
const vals = [];
|
||||||
|
for (const stack of stacksOf(look)) {
|
||||||
|
for (const layer of stack) {
|
||||||
|
for (const [name, def] of Object.entries(layer.module.params || {})) {
|
||||||
|
if (!def.rate || typeof layer.params[name] !== 'number') continue;
|
||||||
|
const [lo, hi] = def.range;
|
||||||
|
vals.push((layer.params[name] - lo) / (hi - lo)); // normalised
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return vals.length ? vals.reduce((a, b) => a + b, 0) / vals.length : 0;
|
||||||
|
};
|
||||||
|
|
||||||
|
const s = rateMean(slow), f = rateMean(fast);
|
||||||
|
return expect(f > s * 1.25,
|
||||||
|
`mean normalised rate: 84bpm ${s.toFixed(3)} vs 148bpm ${f.toFixed(3)} ` +
|
||||||
|
`(${(f / Math.max(1e-6, s)).toFixed(2)}×)`);
|
||||||
|
});
|
||||||
784
flow-state/src/checks/phase11.js
Normal file
784
flow-state/src/checks/phase11.js
Normal file
@ -0,0 +1,784 @@
|
|||||||
|
// Phase 11 gate — Epic 2. See EPIC-2.md.
|
||||||
|
//
|
||||||
|
// Phase 10 asked whether two tracks look different from each other. This phase
|
||||||
|
// asks the question that only shows up when you actually sit and watch one:
|
||||||
|
// does a single track hold five minutes?
|
||||||
|
//
|
||||||
|
// The failures it exists to catch were all measured on a working build, not
|
||||||
|
// imagined — a cut metronome, a palette that never moves, scenes that are
|
||||||
|
// violently animated and read as static, and a kind-to-family table that made
|
||||||
|
// every video make the same genre decisions.
|
||||||
|
|
||||||
|
import { check, expect } from './framework.js';
|
||||||
|
import { generateLook } from '../look/LookGenerator.js';
|
||||||
|
import { FeatureTrack } from '../audio/FeatureTrack.js';
|
||||||
|
import { synthesizeSectioned } from '../audio/synth.js';
|
||||||
|
import { MIN_SHOT_SECONDS, MAX_SHOT_SECONDS } from '../look/shots.js';
|
||||||
|
import { DIRECTORS, RESTFUL_FAMILIES, pickDirector } from '../look/directors.js';
|
||||||
|
import { scenes, scenesInFamily } from '../scenes/registry.js';
|
||||||
|
import { paletteShiftAt, MAX_HUE_ROTATION } from '../look/paletteArc.js';
|
||||||
|
import { shiftPalette, paletteContrast } from '../look/palette.js';
|
||||||
|
import { ArcDriver } from '../look/ArcDriver.js';
|
||||||
|
import { Engine } from '../engine/Engine.js';
|
||||||
|
import { featureProviderFor } from '../audio/FeatureTrack.js';
|
||||||
|
import { sampleValues, clampValue } from '../params/schema.js';
|
||||||
|
import { Rng } from '../engine/rng.js';
|
||||||
|
import { frameDistance, frameLuminance } from '../engine/hash.js';
|
||||||
|
import { SHOT_SIZES, SHOT_SIZE_NAMES, describeFraming } from '../look/framing.js';
|
||||||
|
|
||||||
|
/** Tracks at several tempos: shot length is measured in bars, so tempo is the axis. */
|
||||||
|
let cached = null;
|
||||||
|
function tempoBattery() {
|
||||||
|
if (!cached) {
|
||||||
|
cached = [
|
||||||
|
{ name: '90bpm', track: track(90) },
|
||||||
|
{ name: '124bpm', track: track(124) },
|
||||||
|
{ name: '150bpm', track: track(150) },
|
||||||
|
];
|
||||||
|
}
|
||||||
|
return cached;
|
||||||
|
}
|
||||||
|
function track(bpm) {
|
||||||
|
return FeatureTrack.fromAudioBuffer(
|
||||||
|
synthesizeSectioned({ bpm, duration: 300, changeAt: 150 }), { fps: 60 });
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Every shot in a look, with its length in seconds, grouped by section. */
|
||||||
|
function shotLengths(look) {
|
||||||
|
return look.sections.map((s) => (s.shots || []).map((sh) => (sh.endFrame - sh.startFrame) / 60));
|
||||||
|
}
|
||||||
|
|
||||||
|
check(11, 'a section does not cut on a metronome', () => {
|
||||||
|
// The complaint, as a number. Before this, a five-minute track at 90 BPM
|
||||||
|
// was sixteen shots of 18.7 seconds — coefficient of variation about 0.01.
|
||||||
|
// A section has to show real spread between its shot lengths, or the eye
|
||||||
|
// starts predicting the cuts and the video reads as a slideshow.
|
||||||
|
const problems = [];
|
||||||
|
let measured = 0;
|
||||||
|
|
||||||
|
for (const { name, track: t } of tempoBattery()) {
|
||||||
|
for (let s = 0; s < 6; s++) {
|
||||||
|
const look = generateLook(t, { seed: 2100 + s * 7919 });
|
||||||
|
shotLengths(look).forEach((lengths, si) => {
|
||||||
|
if (lengths.length < 3) return; // too few shots to have a rhythm
|
||||||
|
measured++;
|
||||||
|
const mean = lengths.reduce((a, b) => a + b, 0) / lengths.length;
|
||||||
|
const sd = Math.sqrt(
|
||||||
|
lengths.reduce((a, b) => a + (b - mean) ** 2, 0) / lengths.length);
|
||||||
|
const cv = sd / Math.max(1e-6, mean);
|
||||||
|
if (cv < 0.12) {
|
||||||
|
problems.push(`${name} s${s}/§${si}: cv ${cv.toFixed(3)} over ` +
|
||||||
|
`${lengths.length} shots (${lengths.map((l) => l.toFixed(1)).join(', ')})`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return expect(measured > 0 && problems.length === 0,
|
||||||
|
problems.length ? problems.slice(0, 3).join(' · ')
|
||||||
|
: `${measured} sections all cut with varying shot lengths`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(11, 'the cutting rhythm repeats rather than wandering', () => {
|
||||||
|
// The counter-check, and the reason the gate above is not sufficient on its
|
||||||
|
// own: RANDOM shot lengths would pass it and would look worse than a
|
||||||
|
// metronome. The ear is following an eight-bar structure; the eye has to be
|
||||||
|
// following one too. So a section's shot lengths must come from a small set
|
||||||
|
// of values that recur, not from a continuum.
|
||||||
|
const problems = [];
|
||||||
|
let measured = 0;
|
||||||
|
|
||||||
|
for (const { name, track: t } of tempoBattery()) {
|
||||||
|
for (let s = 0; s < 6; s++) {
|
||||||
|
const look = generateLook(t, { seed: 3300 + s * 6841 });
|
||||||
|
shotLengths(look).forEach((lengths, si) => {
|
||||||
|
if (lengths.length < 5) return;
|
||||||
|
measured++;
|
||||||
|
// Quantise to a quarter second: two shots from the same pattern
|
||||||
|
// entry differ only by downbeat snapping.
|
||||||
|
const buckets = new Set(lengths.map((l) => Math.round(l * 4)));
|
||||||
|
if (buckets.size > Math.ceil(lengths.length * 0.7)) {
|
||||||
|
problems.push(`${name} s${s}/§${si}: ${buckets.size} distinct lengths ` +
|
||||||
|
`over ${lengths.length} shots — wandering, not phrased`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return expect(measured > 0 && problems.length === 0,
|
||||||
|
problems.length ? problems.slice(0, 3).join(' · ')
|
||||||
|
: `${measured} sections reuse a small set of shot lengths`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(11, 'shot length floor and ceiling still hold', () => {
|
||||||
|
// The rhythm work moves cuts around; these two limits are what keep it from
|
||||||
|
// producing a subliminal flash or a two-minute hold.
|
||||||
|
const problems = [];
|
||||||
|
let count = 0;
|
||||||
|
|
||||||
|
for (const { name, track: t } of tempoBattery()) {
|
||||||
|
for (let s = 0; s < 8; s++) {
|
||||||
|
const look = generateLook(t, { seed: 4400 + s * 15485863 });
|
||||||
|
shotLengths(look).forEach((lengths, si) => {
|
||||||
|
lengths.forEach((l, i) => {
|
||||||
|
count++;
|
||||||
|
// Half a frame of slack: shot bounds are rounded to frames.
|
||||||
|
if (l < MIN_SHOT_SECONDS - 0.02) problems.push(`${name} s${s}/§${si}#${i}: ${l.toFixed(2)}s under floor`);
|
||||||
|
if (l > MAX_SHOT_SECONDS + 0.02) problems.push(`${name} s${s}/§${si}#${i}: ${l.toFixed(2)}s over ceiling`);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.slice(0, 4).join(' · ')
|
||||||
|
: `${count} shots all within ${MIN_SHOT_SECONDS}–${MAX_SHOT_SECONDS}s`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(11, 'cuts land on the beat grid', () => {
|
||||||
|
// A cut that lands across a phrase instead of on it reads as a mistake, and
|
||||||
|
// it is the thing most easily lost when shot timing stops being uniform.
|
||||||
|
const problems = [];
|
||||||
|
let total = 0;
|
||||||
|
let onGrid = 0;
|
||||||
|
|
||||||
|
for (const { name, track: t } of tempoBattery()) {
|
||||||
|
const beatSeconds = t.tempo.period / 60;
|
||||||
|
for (let s = 0; s < 6; s++) {
|
||||||
|
const look = generateLook(t, { seed: 5500 + s * 2654435761 });
|
||||||
|
look.sections.forEach((section) => {
|
||||||
|
(section.shots || []).forEach((shot, i) => {
|
||||||
|
if (i === 0) return; // section starts are not cuts
|
||||||
|
total++;
|
||||||
|
const at = shot.startFrame / 60;
|
||||||
|
// Distance to the nearest downbeat, in beats.
|
||||||
|
let best = Infinity;
|
||||||
|
for (const d of (t.tempo.downbeats || [])) {
|
||||||
|
best = Math.min(best, Math.abs(d - at));
|
||||||
|
}
|
||||||
|
if (best <= beatSeconds * 0.75) onGrid++;
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
if (!t.tempo.downbeats || !t.tempo.downbeats.length) problems.push(`${name}: no downbeat grid`);
|
||||||
|
}
|
||||||
|
|
||||||
|
const ratio = total ? onGrid / total : 0;
|
||||||
|
return expect(problems.length === 0 && total > 0 && ratio > 0.8,
|
||||||
|
`${onGrid}/${total} cuts within three quarters of a beat of a downbeat ` +
|
||||||
|
`(${(ratio * 100).toFixed(0)}%, floor 80%)`);
|
||||||
|
});
|
||||||
|
|
||||||
|
// --- the director ----------------------------------------------------------
|
||||||
|
// EPIC-2.md §3.2. The kind-to-family mapping used to be a module constant, so
|
||||||
|
// every video made the same genre decisions before a single seeded draw
|
||||||
|
// happened. It is now a per-track choice — see look/directors.js.
|
||||||
|
|
||||||
|
check(11, 'every scene in the library is reachable', () => {
|
||||||
|
// The number that made this worth doing: across twelve tracks, twenty-nine
|
||||||
|
// of forty-two scenes were cast in none of them. A scene no director can
|
||||||
|
// reach is dead weight, and the fault is in the mapping rather than in the
|
||||||
|
// scene, so this is asked of the mapping directly.
|
||||||
|
const reachable = new Set();
|
||||||
|
for (const d of DIRECTORS) {
|
||||||
|
for (const families of Object.values(d.families)) {
|
||||||
|
for (const f of families) {
|
||||||
|
for (const m of scenesInFamily(f)) reachable.add(m.name);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Accents are cast by role rather than by family and are always eligible.
|
||||||
|
const missing = scenes.filter((m) => m.role !== 'accent' && !reachable.has(m.name));
|
||||||
|
|
||||||
|
return expect(missing.length === 0,
|
||||||
|
missing.length ? `unreachable: ${missing.map((m) => m.name).join(', ')}`
|
||||||
|
: `all ${reachable.size} non-accent scenes reachable across ${DIRECTORS.length} directors`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(11, 'no director starves a section kind', () => {
|
||||||
|
// A director is a point of view, not a corner. Every kind it defines has to
|
||||||
|
// leave enough scenes to build a roster from and still have room for the
|
||||||
|
// signature filter to remove some.
|
||||||
|
const problems = [];
|
||||||
|
for (const d of DIRECTORS) {
|
||||||
|
for (const [kind, families] of Object.entries(d.families)) {
|
||||||
|
const pool = new Set();
|
||||||
|
for (const f of families) {
|
||||||
|
for (const m of scenesInFamily(f)) if (m.role !== 'accent') pool.add(m.name);
|
||||||
|
}
|
||||||
|
if (pool.size < 12) problems.push(`${d.name}/${kind}: only ${pool.size}`);
|
||||||
|
}
|
||||||
|
const kinds = Object.keys(d.families);
|
||||||
|
for (const k of ['intro', 'build', 'drop', 'sustain', 'breakdown', 'outro']) {
|
||||||
|
if (!kinds.includes(k)) problems.push(`${d.name}: no mapping for '${k}'`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.join(' · ')
|
||||||
|
: `${DIRECTORS.length} directors, every kind backed by 12+ scenes`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(11, 'no director puts a loud family in a quiet section', () => {
|
||||||
|
// Phase 7 has enforced this since the minimal family existed, and the first
|
||||||
|
// draft of directors.js broke it — `corrupt` opened on glitch, `geometer`
|
||||||
|
// on geometric. An intro that opens strobing and a breakdown that answers a
|
||||||
|
// lull with a dense pattern are the two specific mistakes the family
|
||||||
|
// coupling exists to prevent, and a viewer meets them in the first fifteen
|
||||||
|
// seconds. Asserted against the mappings directly so a new director cannot
|
||||||
|
// reintroduce it without tripping this rather than a downstream render gate.
|
||||||
|
const problems = [];
|
||||||
|
for (const d of DIRECTORS) {
|
||||||
|
for (const kind of ['intro', 'breakdown', 'outro']) {
|
||||||
|
for (const f of d.families[kind] || []) {
|
||||||
|
if (!RESTFUL_FAMILIES.includes(f)) problems.push(`${d.name}/${kind}: ${f}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.join(' · ')
|
||||||
|
: `${DIRECTORS.length} directors keep intro/breakdown/outro on ${RESTFUL_FAMILIES.join('/')}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(11, 'every director is castable', () => {
|
||||||
|
// Weighting tilts the odds by timbre; none of it may drive a weight to zero,
|
||||||
|
// or a director exists in the file and never in a video.
|
||||||
|
const probes = [
|
||||||
|
{ meanFlatness: 0.02, meanCentroid: 0.15 }, // tonal and dark
|
||||||
|
{ meanFlatness: 0.5, meanCentroid: 0.9 }, // noisy and bright
|
||||||
|
{ meanFlatness: 0.2, meanCentroid: 0.5 },
|
||||||
|
];
|
||||||
|
const rng = { pickWeighted: (arr, w) => w }; // capture the weights
|
||||||
|
const problems = [];
|
||||||
|
for (const p of probes) {
|
||||||
|
const weights = pickDirector(p, rng);
|
||||||
|
weights.forEach((w, i) => {
|
||||||
|
if (!(w > 0)) problems.push(`${DIRECTORS[i].name} unreachable at flatness ${p.meanFlatness}`);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.join(' · ')
|
||||||
|
: `all ${DIRECTORS.length} directors carry weight on every timbre probed`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(11, 'two tracks do not agree on what a section kind looks like', () => {
|
||||||
|
// The population question. Across a battery, the same section kind must be
|
||||||
|
// answered by more than one family, or the director layer is decorative.
|
||||||
|
const seen = new Map(); // kind -> Set of family names actually cast
|
||||||
|
const directors = new Set();
|
||||||
|
|
||||||
|
for (const { track: t } of tempoBattery()) {
|
||||||
|
for (let s = 0; s < 20; s++) {
|
||||||
|
const look = generateLook(t, { seed: 6600 + s * 7919 });
|
||||||
|
directors.add(look.director);
|
||||||
|
look.sections.forEach((section) => {
|
||||||
|
const set = seen.get(section.kind) || new Set();
|
||||||
|
(section.variants || [section.layers]).forEach((v) => set.add(v[0].module.family));
|
||||||
|
seen.set(section.kind, set);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const thin = [...seen.entries()].filter(([, set]) => set.size < 3)
|
||||||
|
.map(([kind, set]) => `${kind}: ${[...set].join('/')}`);
|
||||||
|
|
||||||
|
// Three of five in a population this size is a real spread; demanding all
|
||||||
|
// five would be demanding a particular draw rather than a working mechanism,
|
||||||
|
// and reachability is asserted directly by the check above.
|
||||||
|
return expect(directors.size >= 3 && thin.length === 0,
|
||||||
|
`${directors.size}/${DIRECTORS.length} directors cast · ` +
|
||||||
|
(thin.length ? `thin kinds — ${thin.join(' · ')}`
|
||||||
|
: [...seen.entries()].map(([k, v]) => `${k}:${v.size}fam`).join(' ')));
|
||||||
|
});
|
||||||
|
|
||||||
|
check(11, 'a track shows more of the library than it used to', () => {
|
||||||
|
// Twelve tracks used to reach thirteen distinct scenes between them. This
|
||||||
|
// asks the same question of the same size population.
|
||||||
|
const cast = new Set();
|
||||||
|
tempoBattery().forEach(({ track: t }, i) => {
|
||||||
|
for (let s = 0; s < 4; s++) {
|
||||||
|
const look = generateLook(t, { seed: (i * 2654435761 + s * 40503) >>> 0 });
|
||||||
|
look.sections.forEach((section) =>
|
||||||
|
(section.variants || [section.layers]).forEach((v) => cast.add(v[0].module.name)));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
const pool = scenes.filter((m) => m.role !== 'accent').length;
|
||||||
|
return expect(cast.size >= pool * 0.4,
|
||||||
|
`${cast.size}/${pool} scenes cast across 12 tracks (floor ${Math.ceil(pool * 0.4)})`);
|
||||||
|
});
|
||||||
|
|
||||||
|
// --- colour movement -------------------------------------------------------
|
||||||
|
// EPIC-2.md §3.3. The palette was generated once and pushed to every layer of
|
||||||
|
// every section for the whole runtime — five minutes, one scheme, no movement,
|
||||||
|
// with colour being the strongest perceptual variable available.
|
||||||
|
|
||||||
|
check(11, 'the palette moves across a track', () => {
|
||||||
|
// The complaint as a number. Sample the arc from first frame to last and
|
||||||
|
// require the colours to actually end up somewhere else.
|
||||||
|
const moved = [];
|
||||||
|
const held = [];
|
||||||
|
|
||||||
|
for (const { track: t } of tempoBattery()) {
|
||||||
|
for (let s = 0; s < 8; s++) {
|
||||||
|
const look = generateLook(t, { seed: 7700 + s * 6841 });
|
||||||
|
const arc = look.paletteArc;
|
||||||
|
let worst = 0;
|
||||||
|
for (const kind of ['intro', 'drop', 'breakdown', 'outro']) {
|
||||||
|
for (const p of [0, 0.5, 1]) {
|
||||||
|
const shift = paletteShiftAt(arc, {
|
||||||
|
progress: p, sectionKind: kind,
|
||||||
|
features: { sectionEnergy: p, buildSlope: 0 },
|
||||||
|
});
|
||||||
|
const shifted = shiftPalette(look.palette, shift);
|
||||||
|
for (let i = 0; i < look.palette.length; i++) {
|
||||||
|
const a = look.palette[i], b = shifted[i];
|
||||||
|
worst = Math.max(worst, Math.hypot(a[0] - b[0], a[1] - b[1], a[2] - b[2]));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
(arc.mode === 'static' ? held : moved).push(worst);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const weak = moved.filter((d) => d < 0.03).length;
|
||||||
|
const ratio = moved.length / (moved.length + held.length);
|
||||||
|
return expect(weak === 0 && ratio > 0.6,
|
||||||
|
`${moved.length} moving / ${held.length} held · weakest move ` +
|
||||||
|
`${(moved.length ? Math.min(...moved) : 0).toFixed(3)} channel distance (floor 0.03)`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(11, 'colour movement never breaks the contrast floor', () => {
|
||||||
|
// The Phase 3 palette gate proves the palette is usable when it is
|
||||||
|
// generated. That says nothing about where the arc takes it — a saturation
|
||||||
|
// lift or a lightness shift can flatten a perfectly good set. So the floor
|
||||||
|
// is asserted at every sampled point of the movement rather than only at
|
||||||
|
// the two ends, which is where this would otherwise be silently violated.
|
||||||
|
const problems = [];
|
||||||
|
let sampled = 0;
|
||||||
|
|
||||||
|
for (const { track: t } of tempoBattery()) {
|
||||||
|
for (let s = 0; s < 6; s++) {
|
||||||
|
const look = generateLook(t, { seed: 8800 + s * 15485863 });
|
||||||
|
const base = paletteContrast(look.palette).luminanceSpread;
|
||||||
|
for (const kind of ['intro', 'build', 'drop', 'sustain', 'breakdown', 'outro']) {
|
||||||
|
for (let p = 0; p <= 1.001; p += 0.25) {
|
||||||
|
for (const energy of [0, 0.5, 1]) {
|
||||||
|
sampled++;
|
||||||
|
const shift = paletteShiftAt(look.paletteArc, {
|
||||||
|
progress: p, sectionKind: kind,
|
||||||
|
features: { sectionEnergy: energy, buildSlope: energy },
|
||||||
|
});
|
||||||
|
const spread = paletteContrast(
|
||||||
|
shiftPalette(look.palette, shift)).luminanceSpread;
|
||||||
|
// Held against the palette's OWN spread, not an absolute:
|
||||||
|
// the arc must not degrade what generation achieved.
|
||||||
|
if (spread < Math.min(0.18, base * 0.75)) {
|
||||||
|
problems.push(`seed ${s} ${kind}@${p.toFixed(2)}/e${energy}: ` +
|
||||||
|
`spread ${spread.toFixed(3)} from base ${base.toFixed(3)}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? `${problems.length} of ${sampled} points muddy — ${problems[0]}`
|
||||||
|
: `${sampled} points across the movement all clear the contrast floor`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(11, 'colour movement stays inside its identity', () => {
|
||||||
|
// The counter-check. A palette that moves far enough is a different
|
||||||
|
// palette, and then the track has no colour identity at all — which would
|
||||||
|
// pass the movement gate above with room to spare.
|
||||||
|
const problems = [];
|
||||||
|
for (const { track: t } of tempoBattery()) {
|
||||||
|
for (let s = 0; s < 8; s++) {
|
||||||
|
const look = generateLook(t, { seed: 9900 + s * 7919 });
|
||||||
|
for (const kind of ['intro', 'drop', 'breakdown']) {
|
||||||
|
for (const p of [0, 0.5, 1]) {
|
||||||
|
const shift = paletteShiftAt(look.paletteArc, {
|
||||||
|
progress: p, sectionKind: kind,
|
||||||
|
features: { sectionEnergy: p, buildSlope: p },
|
||||||
|
});
|
||||||
|
if (Math.abs(shift.hue) > MAX_HUE_ROTATION + 1e-6) {
|
||||||
|
problems.push(`seed ${s}: hue ${shift.hue.toFixed(2)} past the ceiling`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems[0]
|
||||||
|
: `hue travel stays within ±${(MAX_HUE_ROTATION * 57.3).toFixed(0)}° of the track's palette`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(11, 'a moved palette is the same on a seek as on playback', () => {
|
||||||
|
// The movement is memoised on a rounded shift. Rounding is what keeps a
|
||||||
|
// seeked frame bit-identical to a played one rather than merely close, and
|
||||||
|
// getting that wrong would be a determinism fault that only shows up in an
|
||||||
|
// export.
|
||||||
|
const t = tempoBattery()[1].track;
|
||||||
|
const look = generateLook(t, { seed: 4321 });
|
||||||
|
const a = new ArcDriver(look, t);
|
||||||
|
const b = new ArcDriver(look, t);
|
||||||
|
|
||||||
|
const probes = [0, 900, 4500, 9000, 12000];
|
||||||
|
let worst = 0;
|
||||||
|
try {
|
||||||
|
// One driver plays through; the other jumps straight to each probe.
|
||||||
|
for (let f = 0; f <= 12000; f += 30) a.update(f, t.at(f));
|
||||||
|
for (const f of probes) {
|
||||||
|
const seq = a._paletteAt(f, t.at(f));
|
||||||
|
const jump = b._paletteAt(f, t.at(f));
|
||||||
|
for (let i = 0; i < seq.length; i++) {
|
||||||
|
for (let c = 0; c < 3; c++) worst = Math.max(worst, Math.abs(seq[i][c] - jump[i][c]));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
a.dispose(); b.dispose();
|
||||||
|
}
|
||||||
|
|
||||||
|
return expect(worst === 0,
|
||||||
|
`worst channel difference between seeked and played colours: ${worst}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
// --- the slow axis ---------------------------------------------------------
|
||||||
|
// EPIC-2.md §3.4. Eleven scenes changed as much in half a second as in two
|
||||||
|
// minutes: everything moving in them was cyclic, so the eye adapted in about
|
||||||
|
// two seconds. Drift is a 20-70s LFO and an LFO returns; the slow axis is
|
||||||
|
// monotonic across the whole track. See ArcDriver._slowAxisFor.
|
||||||
|
|
||||||
|
check(11, 'every scene gets a slow axis with real travel', () => {
|
||||||
|
// Structural half of the gate, so a scene added later cannot quietly end up
|
||||||
|
// with nothing to evolve.
|
||||||
|
const t = tempoBattery()[1].track;
|
||||||
|
const look = generateLook(t, { seed: 20250 });
|
||||||
|
const arc = new ArcDriver(look, t);
|
||||||
|
const problems = [];
|
||||||
|
|
||||||
|
try {
|
||||||
|
for (const module of scenes) {
|
||||||
|
const axis = arc._slowAxisFor(module);
|
||||||
|
const eligible = Object.entries(module.params || {}).filter(([, d]) =>
|
||||||
|
d.type !== 'palette' && d.type !== 'bool' && !d.fixed && !d.rate && !d.noDrift && d.range);
|
||||||
|
if (!eligible.length) continue; // nothing it could legally move
|
||||||
|
|
||||||
|
if (!axis.length) { problems.push(`${module.name}: no axis`); continue; }
|
||||||
|
for (const item of axis) {
|
||||||
|
const [lo, hi] = item.def.range;
|
||||||
|
const fraction = Math.abs(item.travel) / (hi - lo);
|
||||||
|
if (fraction < 0.2) problems.push(`${module.name}.${item.name}: travels only ${(fraction * 100).toFixed(0)}%`);
|
||||||
|
if (item.def.rate) problems.push(`${module.name}.${item.name}: rate param on the axis`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
arc.dispose();
|
||||||
|
}
|
||||||
|
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.slice(0, 3).join(' · ')
|
||||||
|
: `${scenes.length} scenes all carry a slow axis travelling 20%+ of range`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(11, 'the slow axis is a journey rather than a cycle', () => {
|
||||||
|
// The counter-check. An axis that returned to where it started would satisfy
|
||||||
|
// "params move" and would leave the churn exactly as it was — which is what
|
||||||
|
// the existing drift LFO already did.
|
||||||
|
const t = tempoBattery()[1].track;
|
||||||
|
const look = generateLook(t, { seed: 31337 });
|
||||||
|
const arc = new ArcDriver(look, t);
|
||||||
|
const problems = [];
|
||||||
|
|
||||||
|
try {
|
||||||
|
const cue = arc.cues[0];
|
||||||
|
const at = (time) => arc._paramsAt(cue, 0, time, t.at(Math.round(time * 60)));
|
||||||
|
const spec = arc._specFor(cue.sectionIndex, cue.variant, 0);
|
||||||
|
const axis = arc._slowAxisFor(spec.module);
|
||||||
|
|
||||||
|
const start = at(t.duration * 0.05);
|
||||||
|
const mid = at(t.duration * 0.5);
|
||||||
|
const end = at(t.duration * 0.95);
|
||||||
|
|
||||||
|
for (const item of axis) {
|
||||||
|
const [lo, hi] = item.def.range;
|
||||||
|
const span = hi - lo;
|
||||||
|
const a = start[item.name], m = mid[item.name], z = end[item.name];
|
||||||
|
// Monotonic in the sense that matters: the end is further from the
|
||||||
|
// start than the middle is, in the direction of travel.
|
||||||
|
const total = Math.abs(z - a) / span;
|
||||||
|
if (total < 0.12) {
|
||||||
|
problems.push(`${spec.module.name}.${item.name}: start ${a.toFixed(3)} ` +
|
||||||
|
`mid ${m.toFixed(3)} end ${z.toFixed(3)} — only ${(total * 100).toFixed(0)}% travelled`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (!axis.length) problems.push('no axis on the opening scene');
|
||||||
|
} finally {
|
||||||
|
arc.dispose();
|
||||||
|
}
|
||||||
|
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.join(' · ') : 'the opening scene ends the track somewhere else');
|
||||||
|
});
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Structure, with the churn averaged out.
|
||||||
|
*
|
||||||
|
* Single-frame distance cannot answer "did this develop?" for exactly the
|
||||||
|
* scenes that fail it: a churning scene's consecutive frames are already ~0.6
|
||||||
|
* apart, so every pair of its frames scores the same whether the structure
|
||||||
|
* moved or not. Averaging ten seconds of frames cancels the churn and leaves
|
||||||
|
* the structure — and is much closer to what a viewer perceives over seconds
|
||||||
|
* than any single frame is.
|
||||||
|
*
|
||||||
|
* Ten seconds was measured, not guessed. At a one-second window Moiré Grid's
|
||||||
|
* frozen-parameter control still read 0.024; at ten it reads 0.0095, while the
|
||||||
|
* axis-driven change stays at 0.037. The window has to be wide enough that the
|
||||||
|
* control collapses and the signal does not.
|
||||||
|
*/
|
||||||
|
function averagedFrame(engine, look, module, params, frame0, n = 90, step = 7) {
|
||||||
|
engine.setLayerSpecs([{ module, params, seed: 9, opacity: 1, blend: 'normal',
|
||||||
|
palette: look.palette, personality: look.personality }]);
|
||||||
|
engine.prime(frame0);
|
||||||
|
engine.compositor.reset();
|
||||||
|
let acc = null;
|
||||||
|
for (let k = 0; k < n; k++) {
|
||||||
|
const px = engine.readPixels(engine.renderFrame(frame0 + k * step));
|
||||||
|
if (!acc) acc = new Float64Array(px.length);
|
||||||
|
for (let i = 0; i < px.length; i++) acc[i] += px[i];
|
||||||
|
}
|
||||||
|
for (let i = 0; i < acc.length; i++) acc[i] /= n;
|
||||||
|
return acc;
|
||||||
|
}
|
||||||
|
|
||||||
|
function meanAbs(a, b) {
|
||||||
|
let sum = 0;
|
||||||
|
for (let i = 0; i < a.length; i++) sum += Math.abs(a[i] - b[i]);
|
||||||
|
return sum / a.length / 255;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** How much the declared axis moves a scene, against how much it moves anyway. */
|
||||||
|
function axisRatio(engine, arc, look, track, module, { withAxis }) {
|
||||||
|
const paramsAt = (seconds) => {
|
||||||
|
const out = sampleValues(module, new Rng(77), look.sections[0].bias,
|
||||||
|
look.personality.temperament);
|
||||||
|
if (!withAxis) return out;
|
||||||
|
const p = Math.min(1, seconds / track.duration);
|
||||||
|
const journey = p * p * (3 - 2 * p);
|
||||||
|
for (const item of arc._slowAxisFor(module)) {
|
||||||
|
if (typeof out[item.name] !== 'number') continue;
|
||||||
|
out[item.name] = clampValue(item.def, out[item.name] + item.travel * (journey - 0.5));
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
};
|
||||||
|
// The control is the same scene with its parameters held: whatever it does
|
||||||
|
// on its own between these two points in the track.
|
||||||
|
const frozenA = averagedFrame(engine, look, module,
|
||||||
|
sampleValues(module, new Rng(77), look.sections[0].bias, look.personality.temperament), 1800);
|
||||||
|
const frozenB = averagedFrame(engine, look, module,
|
||||||
|
sampleValues(module, new Rng(77), look.sections[0].bias, look.personality.temperament), 9000);
|
||||||
|
const movedA = averagedFrame(engine, look, module, paramsAt(30), 1800);
|
||||||
|
const movedB = averagedFrame(engine, look, module, paramsAt(150), 9000);
|
||||||
|
|
||||||
|
const own = meanAbs(frozenA, frozenB);
|
||||||
|
const moved = meanAbs(movedA, movedB);
|
||||||
|
return { own, moved, ratio: moved / Math.max(1e-6, own) };
|
||||||
|
}
|
||||||
|
|
||||||
|
check(11, 'a declared slow axis actually changes the scene', () => {
|
||||||
|
// EPIC-2.md §3.4, and the honest scope of it. A scene that declares an axis
|
||||||
|
// is claiming that walking that param is what it looks like changing, so
|
||||||
|
// the claim is measured: the structural change with the axis has to beat
|
||||||
|
// what the scene does on its own by a real margin.
|
||||||
|
const t = tempoBattery()[1].track;
|
||||||
|
const look = generateLook(t, { seed: 5150 });
|
||||||
|
const engine = new Engine({ width: 128, height: 72 });
|
||||||
|
engine.timeline.setDuration(t.duration);
|
||||||
|
engine.setFeatureProvider(featureProviderFor(t));
|
||||||
|
const arc = new ArcDriver(look, t);
|
||||||
|
|
||||||
|
const problems = [];
|
||||||
|
const detail = [];
|
||||||
|
try {
|
||||||
|
const declared = scenes.filter((m) => Object.values(m.params || {}).some((d) => d.slowAxis));
|
||||||
|
for (const module of declared) {
|
||||||
|
const r = axisRatio(engine, arc, look, t, module, { withAxis: true });
|
||||||
|
detail.push(`${module.name} ${r.ratio.toFixed(2)}x`);
|
||||||
|
if (r.ratio < 1.25) {
|
||||||
|
problems.push(`${module.name}: axis moved ${r.moved.toFixed(4)} against ` +
|
||||||
|
`${r.own.toFixed(4)} on its own — only ${r.ratio.toFixed(2)}x`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (!declared.length) problems.push('no scene declares a slow axis');
|
||||||
|
} finally {
|
||||||
|
arc.dispose();
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.join(' · ') : `${detail.join(', ')}`);
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
// --- framing ---------------------------------------------------------------
|
||||||
|
// EPIC-2.md §3.5. Every scene was a locked-off, full-frame wide, so a cut changed
|
||||||
|
// the subject and never the framing. A shot now carries a scale and a recentre,
|
||||||
|
// applied in scene coordinates inside sigCamera — see look/framing.js. These
|
||||||
|
// gates hold the layer honest: it must actually change, stay within headroom, and
|
||||||
|
// be the same on a seek as on playback.
|
||||||
|
|
||||||
|
check(11, 'a shot cut actually changes the framing', () => {
|
||||||
|
// The complaint as a number: before this, every shot in every video was the
|
||||||
|
// same full-frame wide, and a cut moved the image and not the camera. The
|
||||||
|
// gate is on the population — most tracks should be framed at all, and of
|
||||||
|
// those, the cuts must really change the shot size rather than cosmetically
|
||||||
|
// vary a number no eye can see.
|
||||||
|
const tracks = tempoBattery();
|
||||||
|
const modes = [];
|
||||||
|
let nonLocked = 0;
|
||||||
|
let withSizeChange = 0;
|
||||||
|
let transitions = 0;
|
||||||
|
let changed = 0;
|
||||||
|
|
||||||
|
// Two seed families so the draw is not at the mercy of one: the whole gate
|
||||||
|
// ran once against a single family whose first few seeds happened to land
|
||||||
|
// heavy on `locked`, and a sample that small has no business holding a
|
||||||
|
// population claim. Locked is ~15% of tracks by design; 24 seeds a family
|
||||||
|
// put it far from that.
|
||||||
|
for (const { track: t } of tracks) {
|
||||||
|
for (const family of [11100, 12200]) {
|
||||||
|
for (let s = 0; s < 12; s++) {
|
||||||
|
const look = generateLook(t, { seed: family + s * 7919 });
|
||||||
|
const style = look.framing;
|
||||||
|
modes.push(style.mode);
|
||||||
|
if (style.mode === 'locked') continue;
|
||||||
|
nonLocked++;
|
||||||
|
|
||||||
|
const arc = new ArcDriver(look, t);
|
||||||
|
try {
|
||||||
|
for (let i = 1; i < arc.cues.length; i++) {
|
||||||
|
const a = arc.cues[i - 1].framing;
|
||||||
|
const b = arc.cues[i].framing;
|
||||||
|
if (!a || !b) continue;
|
||||||
|
transitions++;
|
||||||
|
if (a.size !== b.size) changed++;
|
||||||
|
}
|
||||||
|
const sizes = new Set(arc.cues.map((c) => c.framing && c.framing.size));
|
||||||
|
if (sizes.size > 1) withSizeChange++;
|
||||||
|
} finally {
|
||||||
|
arc.dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const lockedShare = modes.filter((m) => m === 'locked').length / modes.length;
|
||||||
|
const changeRate = transitions ? changed / transitions : 0;
|
||||||
|
return expect(
|
||||||
|
nonLocked > 0 && withSizeChange >= nonLocked * 0.6
|
||||||
|
&& lockedShare < 0.5 && changeRate >= 0.15,
|
||||||
|
`${modes.length} tracks · locked ${(lockedShare * 100).toFixed(0)}% · ` +
|
||||||
|
`${withSizeChange}/${nonLocked} framed tracks show 2+ sizes · ` +
|
||||||
|
`${changed}/${transitions} cuts change size (${(changeRate * 100).toFixed(0)}%)`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(11, 'framing stays inside the library\'s headroom', () => {
|
||||||
|
// A size is only worth using if the scene still has detail at that size.
|
||||||
|
// Past ~2.2 the library runs out and a close-up is a blurry wide; below
|
||||||
|
// ~0.55 the subject is a speck. Those bounds are the measured edges of what
|
||||||
|
// is watchable, so every framed shot has to respect them.
|
||||||
|
const problems = [];
|
||||||
|
let framed = 0;
|
||||||
|
|
||||||
|
for (const { track: t } of tempoBattery()) {
|
||||||
|
for (let s = 0; s < 6; s++) {
|
||||||
|
const look = generateLook(t, { seed: 12200 + s * 6841 });
|
||||||
|
if (look.framing.mode === 'locked') continue;
|
||||||
|
const arc = new ArcDriver(look, t);
|
||||||
|
try {
|
||||||
|
for (const cue of arc.cues) {
|
||||||
|
const f = cue.framing;
|
||||||
|
if (!f) continue;
|
||||||
|
framed++;
|
||||||
|
if (f.scale < 0.55 || f.scale > 2.2) {
|
||||||
|
problems.push(`§${cue.sectionIndex}#${cue.shotIndex}: scale ${f.scale.toFixed(2)}`);
|
||||||
|
}
|
||||||
|
if (Math.abs(f.shift[0]) > 0.3 || Math.abs(f.shift[1]) > 0.3) {
|
||||||
|
problems.push(`§${cue.sectionIndex}#${cue.shotIndex}: shift ${f.shift.map((v) => v.toFixed(2))}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
arc.dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return expect(problems.length === 0 && framed > 0,
|
||||||
|
problems.length ? problems.slice(0, 3).join(' · ')
|
||||||
|
: `${framed} framed shots all within scale headroom and off-centre`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(11, 'framing is identical on a seek and on playback', () => {
|
||||||
|
// Framing is planned once per cue, but a reroll re-plans it. The gate that
|
||||||
|
// matters is the same one every other layer honours: the same frame has to
|
||||||
|
// carry the same framing however you reach it. Two drivers over the same
|
||||||
|
// look must agree on every shot.
|
||||||
|
const t = tempoBattery()[1].track;
|
||||||
|
const look = generateLook(t, { seed: 31337 });
|
||||||
|
const a = new ArcDriver(look, t);
|
||||||
|
const b = new ArcDriver(look, t);
|
||||||
|
|
||||||
|
const problems = [];
|
||||||
|
try {
|
||||||
|
for (let i = 0; i < a.cues.length; i++) {
|
||||||
|
const fa = a.cues[i].framing;
|
||||||
|
const fb = b.cues[i].framing;
|
||||||
|
if (fa && fb && (fa.scale !== fb.scale
|
||||||
|
|| fa.shift[0] !== fb.shift[0] || fa.shift[1] !== fb.shift[1])) {
|
||||||
|
problems.push(`cue ${i}: ${JSON.stringify(fa)} vs ${JSON.stringify(fb)}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
a.dispose(); b.dispose();
|
||||||
|
}
|
||||||
|
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.slice(0, 3).join(' · ')
|
||||||
|
: `${a.cues.length} cues carry the same framing in both drivers`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(11, 'the axis measurement would notice if the axis stopped working', () => {
|
||||||
|
// EPIC-2.md §4 names this failure mode by name: a gate that measures the
|
||||||
|
// wrong thing. This one has already happened once here — the first version
|
||||||
|
// of the check above used single-frame distance, which is saturated on
|
||||||
|
// churning scenes, and passed while the axis was provably doing nothing.
|
||||||
|
//
|
||||||
|
// So the metric is verified by breaking what it is supposed to catch. Run
|
||||||
|
// the identical measurement with the axis disabled; it must come back at
|
||||||
|
// about 1.0 and fail the threshold the real check applies.
|
||||||
|
const t = tempoBattery()[1].track;
|
||||||
|
const look = generateLook(t, { seed: 5150 });
|
||||||
|
const engine = new Engine({ width: 128, height: 72 });
|
||||||
|
engine.timeline.setDuration(t.duration);
|
||||||
|
engine.setFeatureProvider(featureProviderFor(t));
|
||||||
|
const arc = new ArcDriver(look, t);
|
||||||
|
|
||||||
|
const problems = [];
|
||||||
|
const detail = [];
|
||||||
|
try {
|
||||||
|
const declared = scenes.filter((m) => Object.values(m.params || {}).some((d) => d.slowAxis));
|
||||||
|
for (const module of declared) {
|
||||||
|
const off = axisRatio(engine, arc, look, t, module, { withAxis: false });
|
||||||
|
detail.push(`${module.name} ${off.ratio.toFixed(2)}x`);
|
||||||
|
if (off.ratio >= 1.25) {
|
||||||
|
problems.push(`${module.name}: reads ${off.ratio.toFixed(2)}x with the axis ` +
|
||||||
|
`DISABLED — the measurement is not tracking the axis`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
arc.dispose();
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.join(' · ')
|
||||||
|
: `axis disabled reads ${detail.join(', ')} — the measurement tracks the axis`);
|
||||||
|
}, { slow: true });
|
||||||
204
flow-state/src/checks/phase2.js
Normal file
204
flow-state/src/checks/phase2.js
Normal file
@ -0,0 +1,204 @@
|
|||||||
|
// Phase 2 gate — parameter schema, automatic binding, generated UI.
|
||||||
|
//
|
||||||
|
// The range sweep is the check that earns its keep: it renders every scene at
|
||||||
|
// several points across every declared range and rejects frames that are black,
|
||||||
|
// blown out or flat. Those are exactly the states a look generator will wander
|
||||||
|
// into on some seed, and finding them here is far cheaper than finding them at
|
||||||
|
// minute four of an export.
|
||||||
|
|
||||||
|
import { check, expect } from './framework.js';
|
||||||
|
import { Engine } from '../engine/Engine.js';
|
||||||
|
import { scenes, FAMILIES } from '../scenes/registry.js';
|
||||||
|
import { defaultValues, sweepValues, validateModule } from '../params/schema.js';
|
||||||
|
import { serializeParams, deserializeParams } from '../params/serialize.js';
|
||||||
|
import { ParamPanel } from '../ui/ParamPanel.js';
|
||||||
|
import { frameLuminance, frameVariance } from '../engine/hash.js';
|
||||||
|
import { AUDIO_UNIFORMS, SIGNATURE_UNIFORMS } from '../engine/shader-contract.js';
|
||||||
|
import { featureProviderFor } from '../audio/FeatureTrack.js';
|
||||||
|
import { testTrack } from './phase1.js';
|
||||||
|
|
||||||
|
const PALETTE = [
|
||||||
|
[0.06, 0.03, 0.16], [0.85, 0.15, 0.55], [0.15, 0.75, 0.95],
|
||||||
|
[0.98, 0.85, 0.35], [0.55, 0.25, 0.85], [0.2, 0.95, 0.6],
|
||||||
|
];
|
||||||
|
|
||||||
|
function makeEngine(module, params, track) {
|
||||||
|
const engine = new Engine({ width: 192, height: 108 });
|
||||||
|
engine.timeline.setDuration(track.duration);
|
||||||
|
engine.setFeatureProvider(featureProviderFor(track));
|
||||||
|
engine.compositor.postEnabled = false; // sweep the scene, not the grade
|
||||||
|
engine.setLayerSpecs([{
|
||||||
|
module, params, seed: 777, opacity: 1, blend: 'normal', palette: PALETTE,
|
||||||
|
}]);
|
||||||
|
return engine;
|
||||||
|
}
|
||||||
|
|
||||||
|
check(2, 'every scene schema validates', () => {
|
||||||
|
const problems = [];
|
||||||
|
for (const module of scenes) {
|
||||||
|
problems.push(...validateModule(module));
|
||||||
|
if (!FAMILIES[module.family]) problems.push(`${module.name}: unknown family`);
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.join(' · ') : `${scenes.length} scenes valid`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(2, 'declared uniforms and shader sources agree both ways', () => {
|
||||||
|
// Derived from the contract rather than retyped: the signature uniforms
|
||||||
|
// arrived in Phase 9 and a hand-maintained copy of this list would have
|
||||||
|
// reported every scene that reads one as reading an undeclared uniform.
|
||||||
|
const CONTRACT = new Set([
|
||||||
|
'u_resolution', 'u_aspect', 'u_pixelScale', 'u_time', 'u_frame', 'u_progress',
|
||||||
|
'u_seed', 'u_opacity', 'u_colors', 'u_colorCount', 'u_prev', 'u_hasPrev',
|
||||||
|
...AUDIO_UNIFORMS,
|
||||||
|
...Object.keys(SIGNATURE_UNIFORMS),
|
||||||
|
]);
|
||||||
|
const problems = [];
|
||||||
|
for (const module of scenes) {
|
||||||
|
if (module.kind !== 'fragment') continue;
|
||||||
|
const declared = new Map();
|
||||||
|
for (const [name, def] of Object.entries(module.params || {})) {
|
||||||
|
if (def.uniform) declared.set(def.uniform, name);
|
||||||
|
}
|
||||||
|
for (const [uniform, param] of declared) {
|
||||||
|
if (!new RegExp(`\\b${uniform}\\b`).test(module.shader)) {
|
||||||
|
problems.push(`${module.name}: ${param} declares ${uniform}, never read`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (const uniform of new Set(module.shader.match(/\bu_[A-Za-z0-9_]+\b/g) || [])) {
|
||||||
|
if (!CONTRACT.has(uniform) && !declared.has(uniform)) {
|
||||||
|
problems.push(`${module.name}: reads undeclared ${uniform}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.join(' · ') : 'all uniforms accounted for');
|
||||||
|
});
|
||||||
|
|
||||||
|
check(2, 'every scene compiles and renders', () => {
|
||||||
|
const track = testTrack();
|
||||||
|
const problems = [];
|
||||||
|
for (const module of scenes) {
|
||||||
|
const engine = makeEngine(module, defaultValues(module), track);
|
||||||
|
try {
|
||||||
|
const pixels = engine.readPixels(engine.renderFrame(1200));
|
||||||
|
const lum = frameLuminance(pixels);
|
||||||
|
const variance = frameVariance(pixels);
|
||||||
|
// Accent scenes composite over a background; most of their frame is
|
||||||
|
// legitimately black, so only variance is meaningful for them.
|
||||||
|
if (module.role !== 'accent' && !(lum > 0.001)) problems.push(`${module.name}: black frame`);
|
||||||
|
if (variance < 0.002) problems.push(`${module.name}: flat (var ${variance.toFixed(4)})`);
|
||||||
|
} catch (err) {
|
||||||
|
problems.push(`${module.name}: ${err.message}`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.join(' · ') : `${scenes.length} scenes render`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(2, 'param range sweep produces no dead or blown frames', () => {
|
||||||
|
const track = testTrack();
|
||||||
|
const problems = [];
|
||||||
|
let rendered = 0;
|
||||||
|
|
||||||
|
for (const module of scenes) {
|
||||||
|
const base = defaultValues(module);
|
||||||
|
for (const [name, def] of Object.entries(module.params || {})) {
|
||||||
|
if (def.type === 'palette') continue;
|
||||||
|
for (const value of sweepValues(def, 4)) {
|
||||||
|
const params = { ...base, [name]: value };
|
||||||
|
const engine = makeEngine(module, params, track);
|
||||||
|
try {
|
||||||
|
const pixels = engine.readPixels(engine.renderFrame(1200));
|
||||||
|
rendered++;
|
||||||
|
const lum = frameLuminance(pixels);
|
||||||
|
const variance = frameVariance(pixels);
|
||||||
|
const label = `${module.name}.${name}=${JSON.stringify(value)}`;
|
||||||
|
const accent = module.role === 'accent';
|
||||||
|
// An accent at brightness 0 really is black, and that is a
|
||||||
|
// legitimate value — judge those on variance alone.
|
||||||
|
if (lum > 0.985) problems.push(`${label} blown (lum ${lum.toFixed(3)})`);
|
||||||
|
if (!accent && lum < 0.0008) problems.push(`${label} black (lum ${lum.toFixed(5)})`);
|
||||||
|
if (!accent && variance < 0.0015) problems.push(`${label} flat (var ${variance.toFixed(4)})`);
|
||||||
|
} catch (err) {
|
||||||
|
problems.push(`${module.name}.${name}: ${err.message}`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const shown = problems.slice(0, 6).join(' · ');
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length
|
||||||
|
? shown + (problems.length > 6 ? ` · +${problems.length - 6} more` : '')
|
||||||
|
: `${rendered} sweep frames across ${scenes.length} scenes, all live`);
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(2, 'every declared param gets a generated control', () => {
|
||||||
|
const host = document.createElement('div');
|
||||||
|
const problems = [];
|
||||||
|
for (const module of scenes) {
|
||||||
|
const panel = new ParamPanel(host, () => {});
|
||||||
|
panel.build(module, defaultValues(module));
|
||||||
|
const expected = Object.entries(module.params || {})
|
||||||
|
.filter(([, def]) => def.type !== 'palette')
|
||||||
|
.map(([name]) => name);
|
||||||
|
const actual = panel.controlNames();
|
||||||
|
for (const name of expected) {
|
||||||
|
if (!actual.includes(name)) problems.push(`${module.name}.${name} has no control`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.join(' · ') : 'all params exposed in the UI');
|
||||||
|
});
|
||||||
|
|
||||||
|
check(2, 'control edits emit clamped values', () => {
|
||||||
|
const host = document.createElement('div');
|
||||||
|
const module = scenes.find((s) => Object.values(s.params).some((d) => d.type === 'float'));
|
||||||
|
const received = [];
|
||||||
|
const panel = new ParamPanel(host, (name, value) => received.push([name, value]));
|
||||||
|
panel.build(module, defaultValues(module));
|
||||||
|
|
||||||
|
const [name, control] = [...panel.controls.entries()][0];
|
||||||
|
control.input.value = control.input.max;
|
||||||
|
control.input.dispatchEvent(new Event('input'));
|
||||||
|
|
||||||
|
const def = module.params[name];
|
||||||
|
const hi = def.range[1];
|
||||||
|
const last = received[received.length - 1];
|
||||||
|
const value = last && (Array.isArray(last[1]) ? last[1][0] : last[1]);
|
||||||
|
return expect(last && last[0] === name && Math.abs(value - hi) < 1e-6,
|
||||||
|
`emitted ${JSON.stringify(last)}, expected ${name}=${hi}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(2, 'params round-trip through serialisation unchanged', () => {
|
||||||
|
const problems = [];
|
||||||
|
for (const module of scenes) {
|
||||||
|
const values = defaultValues(module);
|
||||||
|
const stored = JSON.parse(JSON.stringify(serializeParams(module, values)));
|
||||||
|
const restored = deserializeParams(module, stored);
|
||||||
|
for (const [name, def] of Object.entries(module.params || {})) {
|
||||||
|
if (def.type === 'palette') continue;
|
||||||
|
const a = values[name], b = restored[name];
|
||||||
|
const same = def.type === 'vec2' ? a[0] === b[0] && a[1] === b[1] : a === b;
|
||||||
|
if (!same) problems.push(`${module.name}.${name}: ${JSON.stringify(a)} → ${JSON.stringify(b)}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.join(' · ') : `${scenes.length} scenes round-trip cleanly`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(2, 'a preset missing or gaining keys still loads', () => {
|
||||||
|
// Presets outlive schema edits; failing to load one is worse than losing a value.
|
||||||
|
const module = scenes[0];
|
||||||
|
const partial = deserializeParams(module, { __gone__: 5 });
|
||||||
|
const defaults = defaultValues(module);
|
||||||
|
const same = Object.keys(defaults).every((k) => {
|
||||||
|
const a = defaults[k], b = partial[k];
|
||||||
|
return Array.isArray(a) ? a[0] === b[0] : a === b;
|
||||||
|
});
|
||||||
|
return expect(same, 'unknown keys ignored, missing keys defaulted');
|
||||||
|
});
|
||||||
233
flow-state/src/checks/phase3.js
Normal file
233
flow-state/src/checks/phase3.js
Normal file
@ -0,0 +1,233 @@
|
|||||||
|
// Phase 3 gate — look generation. "A" complete: a track in, a coherent video out.
|
||||||
|
//
|
||||||
|
// The load-bearing check here is look-space spread. It is entirely possible to
|
||||||
|
// build a generator that is deterministic, valid and well-typed and that produces
|
||||||
|
// visually identical output for every seed — failing the whole premise of the
|
||||||
|
// project (PLAN.md, "sameness across tracks") while passing every other test.
|
||||||
|
// The contact sheet measures it directly.
|
||||||
|
|
||||||
|
import { check, expect } from './framework.js';
|
||||||
|
import { Engine } from '../engine/Engine.js';
|
||||||
|
import { FeatureTrack, featureProviderFor } from '../audio/FeatureTrack.js';
|
||||||
|
import { synthesizeBeat, synthesizeSectioned } from '../audio/synth.js';
|
||||||
|
import { generateLook, rerollSection, rerollLook, describeLook } from '../look/LookGenerator.js';
|
||||||
|
import { paletteContrast, relativeLuminance } from '../look/palette.js';
|
||||||
|
import { frameDistance, frameLuminance, frameVariance } from '../engine/hash.js';
|
||||||
|
import { testTrack } from './phase1.js';
|
||||||
|
|
||||||
|
/** Four deliberately different tracks: the differentiation gate needs real spread. */
|
||||||
|
let cachedBattery = null;
|
||||||
|
export function battery() {
|
||||||
|
if (cachedBattery) return cachedBattery;
|
||||||
|
const specs = [
|
||||||
|
{ name: 'dark ambient', buffer: synthesizeBeat({ bpm: 92, duration: 60, hats: false, kickGain: 0.4, padRoot: 55, padGain: 0.22 }) },
|
||||||
|
{ name: 'mid house', buffer: synthesizeBeat({ bpm: 124, duration: 60, hatGain: 0.25, padRoot: 165 }) },
|
||||||
|
{ name: 'bright techno', buffer: synthesizeBeat({ bpm: 140, duration: 60, hatGain: 0.5, padRoot: 440, padGain: 0.2 }) },
|
||||||
|
{ name: 'structured', buffer: synthesizeSectioned({ bpm: 128, duration: 120, changeAt: 60 }) },
|
||||||
|
];
|
||||||
|
cachedBattery = specs.map((s) => ({
|
||||||
|
name: s.name,
|
||||||
|
track: FeatureTrack.fromAudioBuffer(s.buffer, { fps: 60 }),
|
||||||
|
}));
|
||||||
|
return cachedBattery;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function renderLookFrame(engine, track, look, frame) {
|
||||||
|
const section = look.sections[track.sectionIndexAt(frame)] || look.sections[0];
|
||||||
|
const layer = section.layers[0];
|
||||||
|
engine.setLayerSpecs([{
|
||||||
|
module: layer.module,
|
||||||
|
params: layer.params,
|
||||||
|
seed: layer.seed,
|
||||||
|
opacity: layer.opacity,
|
||||||
|
blend: layer.blend,
|
||||||
|
palette: look.palette,
|
||||||
|
}]);
|
||||||
|
engine.compositor.setPost(look.post).setFeedback(look.feedback);
|
||||||
|
engine.compositor.reset();
|
||||||
|
return Uint8Array.from(engine.readPixels(engine.renderFrame(frame)));
|
||||||
|
}
|
||||||
|
|
||||||
|
check(3, 'the same audio always produces the same look', () => {
|
||||||
|
const track = testTrack();
|
||||||
|
const samples = new Float32Array(2048).map((_, i) => Math.sin(i * 0.01));
|
||||||
|
const a = generateLook(track, { samples });
|
||||||
|
const b = generateLook(track, { samples });
|
||||||
|
|
||||||
|
if (a.seed !== b.seed) return expect(false, `seeds differ: ${a.seed} vs ${b.seed}`);
|
||||||
|
const sameScenes = a.sections.every((s, i) => s.layers[0].module.name === b.sections[i].layers[0].module.name);
|
||||||
|
const sameParams = JSON.stringify(a.sections.map((s) => s.layers[0].params))
|
||||||
|
=== JSON.stringify(b.sections.map((s) => s.layers[0].params));
|
||||||
|
const samePalette = JSON.stringify(a.palette) === JSON.stringify(b.palette);
|
||||||
|
|
||||||
|
return expect(sameScenes && sameParams && samePalette,
|
||||||
|
`seed ${a.seed.toString(16)} · scenes ${sameScenes} · params ${sameParams} · palette ${samePalette}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(3, 'different audio content produces a different seed', () => {
|
||||||
|
const track = testTrack();
|
||||||
|
const a = generateLook(track, { samples: new Float32Array(4096).map((_, i) => Math.sin(i * 0.01)) });
|
||||||
|
const b = generateLook(track, { samples: new Float32Array(4096).map((_, i) => Math.sin(i * 0.013)) });
|
||||||
|
return expect(a.seed !== b.seed, `${a.seed.toString(16)} vs ${b.seed.toString(16)}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(3, 'palettes clear the contrast floor', () => {
|
||||||
|
const problems = [];
|
||||||
|
for (const { name, track } of battery()) {
|
||||||
|
for (let s = 0; s < 12; s++) {
|
||||||
|
const look = generateLook(track, { seed: 1000 + s * 7919 });
|
||||||
|
const { luminanceSpread, chromaSpread } = paletteContrast(look.palette);
|
||||||
|
if (luminanceSpread < 0.18) {
|
||||||
|
problems.push(`${name} seed ${s}: luminance spread ${luminanceSpread.toFixed(3)}`);
|
||||||
|
}
|
||||||
|
if (chromaSpread < 0.20) {
|
||||||
|
problems.push(`${name} seed ${s}: chroma spread ${chromaSpread.toFixed(3)}`);
|
||||||
|
}
|
||||||
|
if (look.palette.some((c) => c.some((v) => !Number.isFinite(v) || v < 0 || v > 1))) {
|
||||||
|
problems.push(`${name} seed ${s}: colour out of gamut`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.slice(0, 4).join(' · ') : '48 palettes all usable');
|
||||||
|
});
|
||||||
|
|
||||||
|
check(3, 'look space is genuinely wide across seeds', () => {
|
||||||
|
// The seed contact sheet, measured. One frame per seed, mean pairwise
|
||||||
|
// distance. A collapsed generator fails here and nowhere else.
|
||||||
|
const track = testTrack();
|
||||||
|
const engine = new Engine({ width: 160, height: 90 });
|
||||||
|
try {
|
||||||
|
engine.timeline.setDuration(track.duration);
|
||||||
|
engine.setFeatureProvider(featureProviderFor(track));
|
||||||
|
|
||||||
|
const frames = [];
|
||||||
|
for (let s = 0; s < 16; s++) {
|
||||||
|
const look = generateLook(track, { seed: 7000 + s * 104729 });
|
||||||
|
frames.push(renderLookFrame(engine, track, look, 1500));
|
||||||
|
}
|
||||||
|
|
||||||
|
let sum = 0;
|
||||||
|
let pairs = 0;
|
||||||
|
let minRelative = Infinity;
|
||||||
|
for (let i = 0; i < frames.length; i++) {
|
||||||
|
for (let j = i + 1; j < frames.length; j++) {
|
||||||
|
const d = frameDistance(frames[i], frames[j]);
|
||||||
|
sum += d; pairs++;
|
||||||
|
// The closest pair is judged RELATIVE to how much image the two
|
||||||
|
// frames contain, for the reason Phase 10 documents at length:
|
||||||
|
// an absolute distance scores two genuinely different renders of
|
||||||
|
// a sparse minimal scene as nearly identical, because the black
|
||||||
|
// they share agrees with itself. Three of these sixteen seeds
|
||||||
|
// cast the same near-black intro scene, and the absolute number
|
||||||
|
// was reporting that as a collapsed generator.
|
||||||
|
const brightness = Math.max(1e-3,
|
||||||
|
(frameLuminance(frames[i]) + frameLuminance(frames[j])) * 0.5);
|
||||||
|
minRelative = Math.min(minRelative, d / brightness);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const mean = sum / pairs;
|
||||||
|
return expect(mean > 0.08 && minRelative > 0.12,
|
||||||
|
`mean pairwise distance ${mean.toFixed(4)} (floor 0.08), ` +
|
||||||
|
`closest pair ${minRelative.toFixed(3)} of its own brightness (floor 0.12)`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(3, 'different tracks get different looks at the same seed', () => {
|
||||||
|
const looks = battery().map(({ name, track }) => ({
|
||||||
|
name,
|
||||||
|
look: generateLook(track, { seed: 42 }), // fixed seed: the difference must come from the audio
|
||||||
|
}));
|
||||||
|
|
||||||
|
const palettes = looks.map(({ look }) => look.palette.map(relativeLuminance).join(','));
|
||||||
|
const uniquePalettes = new Set(palettes).size;
|
||||||
|
const sceneSets = looks.map(({ look }) =>
|
||||||
|
[...new Set(look.sections.map((s) => s.layers[0].module.name))].sort().join('+'));
|
||||||
|
|
||||||
|
return expect(uniquePalettes === looks.length,
|
||||||
|
`${uniquePalettes}/${looks.length} distinct palettes at a fixed seed · ` +
|
||||||
|
looks.map((l, i) => `${l.name}→${sceneSets[i]}`).join(' · '));
|
||||||
|
});
|
||||||
|
|
||||||
|
check(3, 'every generated look renders a live frame on every track', () => {
|
||||||
|
// The generator wanders into corners of the parameter space that the Phase 2
|
||||||
|
// sweep only tests one axis at a time; this tests them in combination.
|
||||||
|
const problems = [];
|
||||||
|
let rendered = 0;
|
||||||
|
|
||||||
|
for (const { name, track } of battery()) {
|
||||||
|
const engine = new Engine({ width: 160, height: 90 });
|
||||||
|
try {
|
||||||
|
engine.timeline.setDuration(track.duration);
|
||||||
|
engine.setFeatureProvider(featureProviderFor(track));
|
||||||
|
|
||||||
|
for (let s = 0; s < 6; s++) {
|
||||||
|
const look = generateLook(track, { seed: 300 + s * 15485863 });
|
||||||
|
for (const section of look.sections) {
|
||||||
|
const frame = Math.min(track.frameCount - 1,
|
||||||
|
section.startFrame + Math.floor((section.endFrame - section.startFrame) / 2));
|
||||||
|
const pixels = renderLookFrame(engine, track, look, frame);
|
||||||
|
rendered++;
|
||||||
|
const lum = frameLuminance(pixels);
|
||||||
|
const variance = frameVariance(pixels);
|
||||||
|
if (lum < 0.0008 || lum > 0.99 || variance < 0.0015) {
|
||||||
|
problems.push(`${name} s${s} ${section.kind}/${section.layers[0].module.name}: ` +
|
||||||
|
`lum ${lum.toFixed(4)} var ${variance.toFixed(4)}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.slice(0, 5).join(' · ') : `${rendered} generated frames, all live`);
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(3, 'sections of the same kind share a scene', () => {
|
||||||
|
// Coherence: a track's drops should look like each other, or the video reads
|
||||||
|
// as a shuffle rather than as one piece.
|
||||||
|
const { track } = battery()[3];
|
||||||
|
const look = generateLook(track, { seed: 99 });
|
||||||
|
const byKind = new Map();
|
||||||
|
let violations = 0;
|
||||||
|
for (const s of look.sections) {
|
||||||
|
const name = s.layers[0].module.name;
|
||||||
|
if (byKind.has(s.kind) && byKind.get(s.kind) !== name) violations++;
|
||||||
|
byKind.set(s.kind, name);
|
||||||
|
}
|
||||||
|
return expect(violations === 0,
|
||||||
|
violations ? `${violations} kind(s) using multiple scenes` : describeLook(look));
|
||||||
|
});
|
||||||
|
|
||||||
|
check(3, 'reroll changes a section and respects locks', () => {
|
||||||
|
const track = testTrack();
|
||||||
|
const look = generateLook(track, { seed: 555 });
|
||||||
|
const before = JSON.stringify(look.sections[0].layers[0].params);
|
||||||
|
|
||||||
|
rerollSection(look, track, 0, 1);
|
||||||
|
const afterUnlocked = JSON.stringify(look.sections[0].layers[0].params);
|
||||||
|
|
||||||
|
look.sections[0].locked = true;
|
||||||
|
rerollSection(look, track, 0, 2);
|
||||||
|
const afterLocked = JSON.stringify(look.sections[0].layers[0].params);
|
||||||
|
|
||||||
|
return expect(before !== afterUnlocked && afterUnlocked === afterLocked,
|
||||||
|
`changed when unlocked: ${before !== afterUnlocked}, held when locked: ${afterUnlocked === afterLocked}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(3, 'a whole-track reroll preserves locked sections', () => {
|
||||||
|
const track = testTrack();
|
||||||
|
const look = generateLook(track, { seed: 777 });
|
||||||
|
look.sections[0].locked = true;
|
||||||
|
const lockedScene = look.sections[0].layers[0].module.name;
|
||||||
|
const lockedParams = JSON.stringify(look.sections[0].layers[0].params);
|
||||||
|
|
||||||
|
const next = rerollLook(look, track, 888);
|
||||||
|
return expect(
|
||||||
|
next.sections[0].layers[0].module.name === lockedScene &&
|
||||||
|
JSON.stringify(next.sections[0].layers[0].params) === lockedParams,
|
||||||
|
`locked section survived a full reroll (${lockedScene})`);
|
||||||
|
});
|
||||||
350
flow-state/src/checks/phase4.js
Normal file
350
flow-state/src/checks/phase4.js
Normal file
@ -0,0 +1,350 @@
|
|||||||
|
// Phase 4 gate — segmentation driving the arc.
|
||||||
|
//
|
||||||
|
// Segmentation accuracy itself is measured in node against synthetic ground
|
||||||
|
// truth. What is checked here is that the structure actually reaches the screen:
|
||||||
|
// scenes change where the music changes, transitions don't pop, and the lookahead
|
||||||
|
// ramp is genuinely wired rather than merely present in the table.
|
||||||
|
//
|
||||||
|
// The check this phase CANNOT automate is monotony. Watching full tracks is the
|
||||||
|
// only way to catch it, and PLAN.md §9 keeps that as an explicit manual gate.
|
||||||
|
|
||||||
|
import { check, expect } from './framework.js';
|
||||||
|
import { Show } from '../Show.js';
|
||||||
|
import { generateLook } from '../look/LookGenerator.js';
|
||||||
|
import { frameDistance, frameLuminance } from '../engine/hash.js';
|
||||||
|
import { FeatureTrack } from '../audio/FeatureTrack.js';
|
||||||
|
import { synthesizeSectioned } from '../audio/synth.js';
|
||||||
|
|
||||||
|
let cached = null;
|
||||||
|
function arcTrack() {
|
||||||
|
if (!cached) {
|
||||||
|
const buffer = synthesizeSectioned({ bpm: 128, duration: 150, changeAt: 75 });
|
||||||
|
cached = FeatureTrack.fromAudioBuffer(buffer, { fps: 60 });
|
||||||
|
}
|
||||||
|
return cached;
|
||||||
|
}
|
||||||
|
|
||||||
|
function makeShow(seed = 2024, width = 160, height = 90) {
|
||||||
|
const show = new Show({ width, height });
|
||||||
|
const track = arcTrack();
|
||||||
|
show.useTrack(track, generateLook(track, { seed }));
|
||||||
|
return show;
|
||||||
|
}
|
||||||
|
|
||||||
|
check(4, 'the scene changes only on a planned cut', () => {
|
||||||
|
// Since Phase 8 a cut is a SHOT boundary, not only a section boundary — a
|
||||||
|
// section rotates between its stage visuals. What must still hold is that no
|
||||||
|
// change happens anywhere the look did not plan one.
|
||||||
|
const show = makeShow();
|
||||||
|
try {
|
||||||
|
const track = show.track;
|
||||||
|
const boundaries = show.arc.cues.map((c) => c.startFrame);
|
||||||
|
const changes = [];
|
||||||
|
let previous = null;
|
||||||
|
|
||||||
|
for (let f = 0; f < track.frameCount; f += 5) {
|
||||||
|
show.arc.update(f, track.at(f));
|
||||||
|
const name = show.arc.state.sceneName;
|
||||||
|
if (previous !== null && name !== previous) changes.push(f);
|
||||||
|
previous = name;
|
||||||
|
}
|
||||||
|
|
||||||
|
const stray = changes.filter((f) => !boundaries.some((b) => Math.abs(f - b) <= 10));
|
||||||
|
return expect(stray.length === 0,
|
||||||
|
`${changes.length} scene change(s), ${stray.length} away from a cut · ` +
|
||||||
|
`${track.sections.length} sections, ${boundaries.length} shots: ` +
|
||||||
|
track.sections.map((s) => s.kind).join(', '));
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(4, 'transitions produce no pops or black frames', () => {
|
||||||
|
// Neither a raw delta threshold nor an outlier-vs-local-median test works
|
||||||
|
// here: these scenes flash on the beat, so large isolated deltas are the
|
||||||
|
// intended behaviour and both metrics flag them. The only meaningful question
|
||||||
|
// is whether a boundary is worse than the same scene's ordinary behaviour, so
|
||||||
|
// this A/Bs each boundary window against a control window with no boundary in
|
||||||
|
// it. Beat flashes appear in both and cancel out.
|
||||||
|
const show = makeShow();
|
||||||
|
try {
|
||||||
|
const track = show.track;
|
||||||
|
|
||||||
|
const scan = (start, end) => {
|
||||||
|
show.engine.compositor.reset();
|
||||||
|
for (let f = Math.max(0, start - 30); f < start; f++) show.renderFrame(f);
|
||||||
|
|
||||||
|
let previous = null;
|
||||||
|
let peak = 0;
|
||||||
|
let peakAt = start;
|
||||||
|
let darkest = 1;
|
||||||
|
for (let f = start; f < Math.min(end, track.frameCount); f++) {
|
||||||
|
const pixels = Uint8Array.from(show.readPixels(show.renderFrame(f)));
|
||||||
|
darkest = Math.min(darkest, frameLuminance(pixels));
|
||||||
|
if (previous) {
|
||||||
|
const d = frameDistance(previous, pixels);
|
||||||
|
if (d > peak) { peak = d; peakAt = f; }
|
||||||
|
}
|
||||||
|
previous = pixels;
|
||||||
|
}
|
||||||
|
return { peak, peakAt, darkest };
|
||||||
|
};
|
||||||
|
|
||||||
|
// The control must sit in the SAME scenes the boundary window contains.
|
||||||
|
// Scenes differ enormously in inherent frame-to-frame motion — one busy
|
||||||
|
// scene next to a calm one reads as an 8x "spike" against a control taken
|
||||||
|
// from the calm one, with no cut anywhere near it.
|
||||||
|
//
|
||||||
|
// Since Phase 8 that means the same SHOT, not merely the same section: a
|
||||||
|
// section rotates between two or three visuals, and the middle of the
|
||||||
|
// section is often not the visual that is on screen at the boundary.
|
||||||
|
const interior = (cue) => {
|
||||||
|
const mid = cue.startFrame + Math.floor((cue.endFrame - cue.startFrame) / 2);
|
||||||
|
return scan(mid, Math.min(mid + 200, cue.endFrame));
|
||||||
|
};
|
||||||
|
|
||||||
|
const cueAt = (frame) => show.arc.cueAt(frame);
|
||||||
|
|
||||||
|
let worstRatio = 0;
|
||||||
|
let worstBoundary = -1;
|
||||||
|
let darkest = 1;
|
||||||
|
let controlUsed = 0;
|
||||||
|
let worstPeakAt = -1;
|
||||||
|
|
||||||
|
for (let i = 1; i < track.sections.length; i++) {
|
||||||
|
const s = track.sections[i];
|
||||||
|
const before = interior(cueAt(s.startFrame - 1));
|
||||||
|
const after = interior(cueAt(s.startFrame));
|
||||||
|
const control = Math.max(before.peak, after.peak);
|
||||||
|
|
||||||
|
// The window starts just before the boundary rather than a second
|
||||||
|
// before it. The transition runs FORWARD from the boundary, so a
|
||||||
|
// handful of pre-frames is all it takes to catch a pop on the
|
||||||
|
// boundary frame itself — while a longer pre-roll would drag in the
|
||||||
|
// lookahead build ramp, where a scene is deliberately driven to the
|
||||||
|
// top of its range and flashes accordingly. That is the intended
|
||||||
|
// climax of a build, not a transition fault, and the control window
|
||||||
|
// (mid-shot, no ramp) has no equivalent to cancel it against.
|
||||||
|
const w = scan(s.startFrame - 8, s.startFrame + cueAt(s.startFrame).fadeFrames + 60);
|
||||||
|
darkest = Math.min(darkest, w.darkest, before.darkest, after.darkest);
|
||||||
|
|
||||||
|
const ratio = w.peak / Math.max(control, 1e-6);
|
||||||
|
if (ratio > worstRatio) {
|
||||||
|
worstRatio = ratio; worstBoundary = s.startFrame; controlUsed = control;
|
||||||
|
worstPeakAt = w.peakAt;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (worstBoundary < 0) return expect(true, 'single-section track');
|
||||||
|
|
||||||
|
return expect(worstRatio < 1.6 && darkest > 0.002,
|
||||||
|
`worst boundary peak ${worstRatio.toFixed(2)}x the adjacent scenes' own peak ` +
|
||||||
|
`(control ${controlUsed.toFixed(4)}) at boundary ${worstBoundary}, peak frame ${worstPeakAt}, darkest ${darkest.toFixed(4)}`);
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(4, 'crossfade ramps rather than cuts', () => {
|
||||||
|
const show = makeShow();
|
||||||
|
try {
|
||||||
|
const track = show.track;
|
||||||
|
const boundary = track.sections[1] && track.sections[1].startFrame;
|
||||||
|
if (!boundary) return expect(true, 'single-section track, nothing to cross-fade');
|
||||||
|
|
||||||
|
// Read the length off the cue rather than assuming the nominal one: since
|
||||||
|
// Phase 8 a dissolve is two bars on calm material and one on loud.
|
||||||
|
const fadeFrames = show.arc.cueAt(boundary).fadeFrames;
|
||||||
|
const samples = [];
|
||||||
|
for (let f = boundary; f < boundary + fadeFrames; f += 2) {
|
||||||
|
show.arc.update(f, track.at(f));
|
||||||
|
samples.push(show.arc.state.crossfade);
|
||||||
|
}
|
||||||
|
const monotonic = samples.slice(1).every((v, i) => v >= samples[i] - 1e-6);
|
||||||
|
const spans = samples[0] < 0.15 && samples[samples.length - 1] > 0.85;
|
||||||
|
|
||||||
|
return expect(monotonic && spans,
|
||||||
|
`${fadeFrames}-frame fade, monotonic ${monotonic}, ` +
|
||||||
|
`${samples[0].toFixed(2)}→${samples[samples.length - 1].toFixed(2)}`);
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(4, 'lookahead ramps params into a higher-energy section', () => {
|
||||||
|
// The payoff of offline analysis. buildSlope must rise before the boundary
|
||||||
|
// AND actually move a parameter, not merely exist in the table.
|
||||||
|
const track = arcTrack();
|
||||||
|
const rising = [];
|
||||||
|
for (let i = 0; i < track.sections.length - 1; i++) {
|
||||||
|
if (track.sections[i + 1].energy > track.sections[i].energy * 1.08) rising.push(i);
|
||||||
|
}
|
||||||
|
if (!rising.length) return expect(true, 'no rising transition in this track');
|
||||||
|
|
||||||
|
const show = makeShow();
|
||||||
|
try {
|
||||||
|
const section = track.sections[rising[0]];
|
||||||
|
const traces = [];
|
||||||
|
for (let f = Math.max(section.startFrame, section.endFrame - 300); f < section.endFrame; f += 20) {
|
||||||
|
const features = track.at(f);
|
||||||
|
show.arc.update(f, features);
|
||||||
|
const layer = show.arc.activeLayers[show.arc.activeLayers.length - 1];
|
||||||
|
traces.push({ slope: features.buildSlope, params: { ...layer.baseParams } });
|
||||||
|
}
|
||||||
|
if (traces.length < 3) return expect(true, 'section too short to sample a ramp');
|
||||||
|
|
||||||
|
const first = traces[0], last = traces[traces.length - 1];
|
||||||
|
const slopeRises = last.slope > first.slope + 1e-6;
|
||||||
|
const moved = Object.keys(first.params).filter((k) =>
|
||||||
|
typeof first.params[k] === 'number' && Math.abs(last.params[k] - first.params[k]) > 1e-6);
|
||||||
|
|
||||||
|
return expect(slopeRises && moved.length > 0,
|
||||||
|
`buildSlope ${first.slope.toFixed(3)}→${last.slope.toFixed(3)}, ` +
|
||||||
|
`${moved.length} param(s) ramped: ${moved.slice(0, 4).join(', ')}`);
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(4, 'params drift within a held shot', () => {
|
||||||
|
// Guards the failure mode automated checks are worst at: an image that is
|
||||||
|
// technically correct and completely static. Measured over the longest SHOT
|
||||||
|
// rather than the longest section — across a shot cut the scene itself
|
||||||
|
// changes, which would pass this trivially and prove nothing about drift.
|
||||||
|
const show = makeShow();
|
||||||
|
try {
|
||||||
|
const track = show.track;
|
||||||
|
const longest = show.arc.cues.reduce((a, b) =>
|
||||||
|
(b.endFrame - b.startFrame > a.endFrame - a.startFrame ? b : a));
|
||||||
|
|
||||||
|
const sample = (frame) => {
|
||||||
|
show.arc.update(frame, track.at(frame));
|
||||||
|
const layer = show.arc.activeLayers[show.arc.activeLayers.length - 1];
|
||||||
|
return { ...layer.baseParams };
|
||||||
|
};
|
||||||
|
|
||||||
|
const a = sample(longest.startFrame + 120);
|
||||||
|
const b = sample(Math.max(longest.startFrame + 121, longest.endFrame - 120));
|
||||||
|
|
||||||
|
const numeric = Object.keys(a).filter((k) => typeof a[k] === 'number');
|
||||||
|
const moved = numeric.filter((k) => Math.abs(b[k] - a[k]) > 1e-4);
|
||||||
|
|
||||||
|
return expect(moved.length >= Math.ceil(numeric.length * 0.5),
|
||||||
|
`${moved.length}/${numeric.length} params moved across a ` +
|
||||||
|
`${((longest.endFrame - longest.startFrame) / 60).toFixed(0)}s shot`);
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(4, 'the arc-driven render is still deterministic', () => {
|
||||||
|
const a = makeShow();
|
||||||
|
const b = makeShow();
|
||||||
|
try {
|
||||||
|
const hashes = (show) => {
|
||||||
|
show.engine.compositor.reset();
|
||||||
|
const out = [];
|
||||||
|
for (let f = 4400; f < 4460; f++) out.push(show.hashFrame(f));
|
||||||
|
return out;
|
||||||
|
};
|
||||||
|
const ha = hashes(a);
|
||||||
|
const hb = hashes(b);
|
||||||
|
const mismatches = ha.filter((h, i) => h !== hb[i]).length;
|
||||||
|
return expect(mismatches === 0,
|
||||||
|
`${mismatches}/60 frames differed between two independently built shows`);
|
||||||
|
} finally {
|
||||||
|
a.dispose(); b.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(4, 'a frame renders the same the first time as every later time', () => {
|
||||||
|
// Regression guard for an aliasing class no other check covered: FeatureTrack.at()
|
||||||
|
// returns a reused row, so anything that calls it mid-render corrupts the row the
|
||||||
|
// layer is about to read. The symptom is a frame that is correct on every repeat
|
||||||
|
// and wrong on its first render — invisible to fresh-vs-fresh comparison, and
|
||||||
|
// visible in an export, which renders every frame exactly once.
|
||||||
|
const show = makeShow();
|
||||||
|
try {
|
||||||
|
const track = show.track;
|
||||||
|
show.look.feedback.amount = 0;
|
||||||
|
|
||||||
|
const frames = [
|
||||||
|
...track.sections.map((s) => s.startFrame),
|
||||||
|
...track.sections.map((s) => s.startFrame + 30),
|
||||||
|
1000, 4000,
|
||||||
|
].filter((f) => f > 0 && f < track.frameCount);
|
||||||
|
|
||||||
|
const problems = [];
|
||||||
|
for (const f of frames) {
|
||||||
|
const fresh = makeShow();
|
||||||
|
fresh.look.feedback.amount = 0;
|
||||||
|
fresh.engine.compositor.reset();
|
||||||
|
const first = fresh.engine.hashCurrent(fresh.renderFrame(f));
|
||||||
|
fresh.dispose();
|
||||||
|
|
||||||
|
show.engine.compositor.reset();
|
||||||
|
show.renderFrame(f);
|
||||||
|
show.engine.compositor.reset();
|
||||||
|
const repeat = show.engine.hashCurrent(show.renderFrame(f));
|
||||||
|
|
||||||
|
if (first !== repeat) problems.push(`frame ${f}: first ${first} vs repeat ${repeat}`);
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.join(' · ') : `${frames.length} frames stable on first render`);
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(4, 'seek converges to sequential playback, and is exact without feedback', () => {
|
||||||
|
// Corrected from the original plan: a boundary is not exact for free. Layer
|
||||||
|
// state is re-seeded there, but the feedback buffer is global and carries
|
||||||
|
// across, so any look with feedback needs warm-up wherever you land.
|
||||||
|
const show = makeShow();
|
||||||
|
try {
|
||||||
|
const track = show.track;
|
||||||
|
const boundary = track.sections[1] ? track.sections[1].startFrame : 600;
|
||||||
|
|
||||||
|
// Force heavy feedback so the check is actually exercising convergence
|
||||||
|
// rather than passing because this seed happened to generate very little.
|
||||||
|
show.look.feedback.amount = 0.7;
|
||||||
|
show.look.feedback.decay = 0.93;
|
||||||
|
const warmup = show.warmupFrames();
|
||||||
|
|
||||||
|
show.engine.compositor.reset();
|
||||||
|
for (let f = boundary - 300; f < boundary; f++) show.renderFrame(f);
|
||||||
|
const sequential = Uint8Array.from(show.readPixels(show.renderFrame(boundary)));
|
||||||
|
|
||||||
|
const warmed = Uint8Array.from(show.readPixels(show.seek(boundary)));
|
||||||
|
const distance = frameDistance(sequential, warmed);
|
||||||
|
|
||||||
|
// And with feedback off it must be bit-exact, proving nothing else is stateful.
|
||||||
|
show.look.feedback.amount = 0;
|
||||||
|
show.engine.compositor.reset();
|
||||||
|
for (let f = boundary - 60; f < boundary; f++) show.renderFrame(f);
|
||||||
|
const seqExact = show.engine.hashCurrent(show.renderFrame(boundary));
|
||||||
|
const directExact = show.engine.hashCurrent(show.seek(boundary));
|
||||||
|
|
||||||
|
return expect(distance < 0.01 && seqExact === directExact,
|
||||||
|
`with feedback 0.7/0.93: converged to ${distance.toFixed(5)} after ${warmup} ` +
|
||||||
|
`warm-up frames · without feedback: exact ${seqExact === directExact}`);
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(4, 'head and tail fade rather than cut', () => {
|
||||||
|
const show = makeShow();
|
||||||
|
try {
|
||||||
|
show.engine.compositor.reset();
|
||||||
|
const first = frameLuminance(show.readPixels(show.renderFrame(0)));
|
||||||
|
show.engine.compositor.reset();
|
||||||
|
const early = frameLuminance(show.readPixels(show.renderFrame(400)));
|
||||||
|
show.engine.compositor.reset();
|
||||||
|
const last = frameLuminance(show.readPixels(show.renderFrame(show.frameCount - 1)));
|
||||||
|
return expect(first < early * 0.4 && last < early * 0.4,
|
||||||
|
`frame 0 ${first.toFixed(4)} · frame 400 ${early.toFixed(4)} · last ${last.toFixed(4)}`);
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
360
flow-state/src/checks/phase5.js
Normal file
360
flow-state/src/checks/phase5.js
Normal file
@ -0,0 +1,360 @@
|
|||||||
|
// Phase 5 gate — compositing depth. "C" complete.
|
||||||
|
//
|
||||||
|
// Multi-layer stacks, blend modes, feedback, the post chain and 3D layers. The
|
||||||
|
// feedback stability run and the flash-rate check matter most: both fail
|
||||||
|
// silently, slowly, and only in the finished file.
|
||||||
|
|
||||||
|
import { check, expect, expectBelow } from './framework.js';
|
||||||
|
import { Engine } from '../engine/Engine.js';
|
||||||
|
import { Show } from '../Show.js';
|
||||||
|
import { generateLook } from '../look/LookGenerator.js';
|
||||||
|
import { FeatureTrack, featureProviderFor } from '../audio/FeatureTrack.js';
|
||||||
|
import { synthesizeSectioned } from '../audio/synth.js';
|
||||||
|
import { defaultValues, sampleValues } from '../params/schema.js';
|
||||||
|
import { scenes } from '../scenes/registry.js';
|
||||||
|
import { Rng } from '../engine/rng.js';
|
||||||
|
import { frameLuminance, frameVariance, frameMaxDelta } from '../engine/hash.js';
|
||||||
|
import { peakFlashRate } from '../engine/flash.js';
|
||||||
|
import { particleField } from '../scenes/layers3d/particles.js';
|
||||||
|
import { nebula } from '../scenes/shader/nebula.js';
|
||||||
|
import { BLEND_MODES } from '../engine/Layer.js';
|
||||||
|
|
||||||
|
const PALETTE = [
|
||||||
|
[0.06, 0.03, 0.16], [0.85, 0.15, 0.55], [0.15, 0.75, 0.95],
|
||||||
|
[0.98, 0.85, 0.35], [0.55, 0.25, 0.85], [0.2, 0.95, 0.6],
|
||||||
|
];
|
||||||
|
|
||||||
|
let cached = null;
|
||||||
|
function track5() {
|
||||||
|
if (!cached) {
|
||||||
|
cached = FeatureTrack.fromAudioBuffer(
|
||||||
|
synthesizeSectioned({ bpm: 128, duration: 120, changeAt: 60 }), { fps: 60 });
|
||||||
|
}
|
||||||
|
return cached;
|
||||||
|
}
|
||||||
|
|
||||||
|
function makeEngine(width = 192, height = 108) {
|
||||||
|
const engine = new Engine({ width, height });
|
||||||
|
const track = track5();
|
||||||
|
engine.timeline.setDuration(track.duration);
|
||||||
|
engine.setFeatureProvider(featureProviderFor(track));
|
||||||
|
return engine;
|
||||||
|
}
|
||||||
|
|
||||||
|
check(5, 'a 3D layer renders and composites', () => {
|
||||||
|
const engine = makeEngine();
|
||||||
|
try {
|
||||||
|
engine.setLayerSpecs([{
|
||||||
|
module: particleField, params: defaultValues(particleField),
|
||||||
|
seed: 31337, opacity: 1, blend: 'normal', palette: PALETTE,
|
||||||
|
}]);
|
||||||
|
const pixels = engine.readPixels(engine.renderFrame(1200));
|
||||||
|
const lum = frameLuminance(pixels);
|
||||||
|
const variance = frameVariance(pixels);
|
||||||
|
// Judged on variance, not mean luminance: this is an additive accent over
|
||||||
|
// a dark field, so most of the frame is legitimately black and a mean of
|
||||||
|
// ~0.001 is the correct result rather than a dead render.
|
||||||
|
return expect(variance > 0.005 && lum > 0.0002,
|
||||||
|
`particle field: variance ${variance.toFixed(4)}, mean luminance ${lum.toFixed(4)}`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(5, '3D layer motion is analytic, so a seek matches sequential playback', () => {
|
||||||
|
// The rule 3D layers must obey: no integrated state. An integrated particle
|
||||||
|
// system drifts apart between a seek and playback and silently breaks export
|
||||||
|
// parity.
|
||||||
|
const engine = makeEngine();
|
||||||
|
try {
|
||||||
|
engine.setLayerSpecs([{
|
||||||
|
module: particleField, params: defaultValues(particleField),
|
||||||
|
seed: 31337, opacity: 1, blend: 'normal', palette: PALETTE,
|
||||||
|
}]);
|
||||||
|
const sequential = engine.hashRun(0, 200);
|
||||||
|
engine.compositor.reset();
|
||||||
|
const direct = engine.hashCurrent(engine.renderFrame(199));
|
||||||
|
return expect(direct === sequential[199],
|
||||||
|
`seek→199 ${direct} vs sequential ${sequential[199]}`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(5, 'every blend mode composites two layers distinctly', () => {
|
||||||
|
const engine = makeEngine();
|
||||||
|
try {
|
||||||
|
const seen = new Map();
|
||||||
|
for (const blend of BLEND_MODES) {
|
||||||
|
engine.setLayerSpecs([
|
||||||
|
{ module: nebula, params: defaultValues(nebula), seed: 11, opacity: 1, blend: 'normal', palette: PALETTE },
|
||||||
|
{ module: particleField, params: defaultValues(particleField), seed: 22, opacity: 0.6, blend, palette: PALETTE },
|
||||||
|
]);
|
||||||
|
engine.compositor.reset();
|
||||||
|
seen.set(blend, engine.hashCurrent(engine.renderFrame(1200)));
|
||||||
|
}
|
||||||
|
const unique = new Set(seen.values()).size;
|
||||||
|
return expect(unique === BLEND_MODES.length,
|
||||||
|
`${unique}/${BLEND_MODES.length} blend modes produced distinct output`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(5, 'layer solo isolates each layer', () => {
|
||||||
|
const engine = makeEngine();
|
||||||
|
try {
|
||||||
|
engine.setLayerSpecs([
|
||||||
|
{ module: nebula, params: defaultValues(nebula), seed: 11, opacity: 1, blend: 'normal', palette: PALETTE },
|
||||||
|
{ module: particleField, params: defaultValues(particleField), seed: 22, opacity: 0.8, blend: 'add', palette: PALETTE },
|
||||||
|
]);
|
||||||
|
engine.compositor.reset();
|
||||||
|
const both = engine.hashCurrent(engine.renderFrame(1200));
|
||||||
|
|
||||||
|
engine.compositor.soloIndex = 0;
|
||||||
|
engine.compositor.reset();
|
||||||
|
const first = engine.hashCurrent(engine.renderFrame(1200));
|
||||||
|
|
||||||
|
engine.compositor.soloIndex = 1;
|
||||||
|
engine.compositor.reset();
|
||||||
|
const second = engine.hashCurrent(engine.renderFrame(1200));
|
||||||
|
engine.compositor.soloIndex = -1;
|
||||||
|
|
||||||
|
return expect(new Set([both, first, second]).size === 3,
|
||||||
|
`combined ${both}, solo-0 ${first}, solo-1 ${second}`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(5, 'feedback stays stable over a long run', () => {
|
||||||
|
// A feedback loop with gain at or above 1 saturates to white; too much decay
|
||||||
|
// and it dies to black. Either takes thousands of frames to show, so it never
|
||||||
|
// appears in a short check — and always appears in a six-minute export.
|
||||||
|
const engine = makeEngine(128, 72);
|
||||||
|
try {
|
||||||
|
engine.setLayerSpecs([{
|
||||||
|
module: nebula, params: defaultValues(nebula),
|
||||||
|
seed: 4242, opacity: 1, blend: 'normal', palette: PALETTE,
|
||||||
|
}]);
|
||||||
|
engine.compositor.setFeedback({ amount: 0.75, decay: 0.94, zoom: 1.006, rotate: 0.003 });
|
||||||
|
engine.compositor.reset();
|
||||||
|
|
||||||
|
const track = track5();
|
||||||
|
const samples = [];
|
||||||
|
for (let f = 0; f < 10000; f++) {
|
||||||
|
engine.timeline.seek(f % track.frameCount);
|
||||||
|
const target = engine.compositor.render({
|
||||||
|
timeline: engine.timeline, features: track.at(f % track.frameCount),
|
||||||
|
});
|
||||||
|
if (f % 500 === 0 || f === 9999) samples.push(frameLuminance(engine.readPixels(target)));
|
||||||
|
}
|
||||||
|
|
||||||
|
const min = Math.min(...samples);
|
||||||
|
const max = Math.max(...samples);
|
||||||
|
const tail = samples.slice(-4);
|
||||||
|
return expect(max < 0.97 && min > 0.003,
|
||||||
|
`10000 frames · luminance ${min.toFixed(4)}..${max.toFixed(4)} · ` +
|
||||||
|
`tail ${tail.map((v) => v.toFixed(3)).join(', ')}`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(5, 'generated looks stay within the flash-rate ceiling', () => {
|
||||||
|
// WCAG 2.3.1 / Harding: at most three light-dark cycles per second. An
|
||||||
|
// unsupervised generator finds unsafe states on its own, and this is the only
|
||||||
|
// thing between one of them and a published video.
|
||||||
|
const track = track5();
|
||||||
|
const problems = [];
|
||||||
|
let worst = 0;
|
||||||
|
let worstLabel = '';
|
||||||
|
|
||||||
|
for (let s = 0; s < 5; s++) {
|
||||||
|
const show = new Show({ width: 96, height: 54 });
|
||||||
|
try {
|
||||||
|
show.useTrack(track, generateLook(track, { seed: 5000 + s * 7919 }));
|
||||||
|
// Per SHOT, not per section: a section rotates between two or three
|
||||||
|
// stage visuals and only the first of them sits at the section start,
|
||||||
|
// so sweeping sections would leave most of what ships unmeasured. The
|
||||||
|
// window opens before the cut so the cut itself is inside it.
|
||||||
|
for (const cue of show.arc.cues.slice(0, 8)) {
|
||||||
|
const start = Math.max(0, cue.startFrame - 20);
|
||||||
|
const end = Math.min(cue.endFrame, start + 300); // 5 seconds
|
||||||
|
if (end - start < 120) continue;
|
||||||
|
|
||||||
|
show.engine.compositor.reset();
|
||||||
|
for (let f = Math.max(0, start - 40); f < start; f++) show.renderFrame(f);
|
||||||
|
|
||||||
|
const luminance = [];
|
||||||
|
for (let f = start; f < end; f++) {
|
||||||
|
luminance.push(frameLuminance(show.readPixels(show.renderFrame(f))));
|
||||||
|
}
|
||||||
|
const rate = peakFlashRate(luminance, 60);
|
||||||
|
const section = show.look.sections[cue.sectionIndex];
|
||||||
|
const scene = (section.variants[cue.variant] || section.layers)[0].module.name;
|
||||||
|
const label = `seed ${s} ${section.kind}/${scene}`;
|
||||||
|
if (rate > worst) { worst = rate; worstLabel = label; }
|
||||||
|
if (rate > 3) problems.push(`${label}: ${rate}/s`);
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length
|
||||||
|
? problems.slice(0, 4).join(' · ')
|
||||||
|
: `peak ${worst} flashes/s (ceiling 3) at ${worstLabel}`);
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(5, 'no scene in the library strobes at aggressive settings', () => {
|
||||||
|
// Per-scene rather than per-look: the look generator only samples a slice of
|
||||||
|
// the parameter space, so a scene can hide an unsafe region for a long time.
|
||||||
|
// This drives every scene to high energy/density/motion across several seeds,
|
||||||
|
// which is where strobing lives. Phase 7 runs the same check for each new
|
||||||
|
// scene before it joins the library.
|
||||||
|
const track = track5();
|
||||||
|
const problems = [];
|
||||||
|
let worst = 0;
|
||||||
|
let worstScene = '';
|
||||||
|
|
||||||
|
for (const module of scenes) {
|
||||||
|
for (let s = 0; s < 4; s++) {
|
||||||
|
const engine = makeEngine(240, 135);
|
||||||
|
try {
|
||||||
|
const rng = new Rng(1000 + s * 7919);
|
||||||
|
engine.setLayerSpecs([{
|
||||||
|
module,
|
||||||
|
params: sampleValues(module, rng, { energy: 0.9, density: 0.9, motion: 0.9 }),
|
||||||
|
seed: s * 31 + 7, opacity: 1, blend: 'normal', palette: PALETTE,
|
||||||
|
}]);
|
||||||
|
engine.compositor.reset();
|
||||||
|
const luminance = [];
|
||||||
|
for (let f = 4000; f < 4240; f++) {
|
||||||
|
luminance.push(frameLuminance(engine.readPixels(engine.renderFrame(f))));
|
||||||
|
}
|
||||||
|
const rate = peakFlashRate(luminance, 60);
|
||||||
|
if (rate > worst) { worst = rate; worstScene = `${module.name} seed ${s}`; }
|
||||||
|
if (rate > 3) problems.push(`${module.name} seed ${s}: ${rate}/s`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.slice(0, 4).join(' · ')
|
||||||
|
: `${scenes.length} scenes, worst ${worst} flashes/s (ceiling 3) at ${worstScene}`);
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(5, 'the full stack holds interactive frame rates at preview resolution', () => {
|
||||||
|
const track = track5();
|
||||||
|
const show = new Show({ width: 1280, height: 720 });
|
||||||
|
try {
|
||||||
|
show.useTrack(track, generateLook(track, { seed: 2468 }));
|
||||||
|
show.look.feedback.amount = Math.max(0.4, show.look.feedback.amount);
|
||||||
|
|
||||||
|
for (let f = 1200; f < 1230; f++) show.renderFrame(f); // warm caches
|
||||||
|
|
||||||
|
const started = performance.now();
|
||||||
|
const frames = 60;
|
||||||
|
for (let f = 1300; f < 1300 + frames; f++) show.renderFrame(f);
|
||||||
|
const perFrame = (performance.now() - started) / frames;
|
||||||
|
|
||||||
|
const layers = show.arc.activeLayers.length;
|
||||||
|
return expectBelow(perFrame, 16.7,
|
||||||
|
`${perFrame.toFixed(2)}ms/frame at 1280x720 with ${layers} layer(s) + feedback + post`);
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(5, 'multi-layer looks render live frames across the library', () => {
|
||||||
|
const track = track5();
|
||||||
|
const problems = [];
|
||||||
|
let stacks = 0;
|
||||||
|
|
||||||
|
for (let s = 0; s < 8; s++) {
|
||||||
|
const show = new Show({ width: 160, height: 90 });
|
||||||
|
try {
|
||||||
|
show.useTrack(track, generateLook(track, { seed: 9000 + s * 104729 }));
|
||||||
|
for (const section of show.look.sections) {
|
||||||
|
if (section.layers.length > 1) stacks++;
|
||||||
|
const frame = section.startFrame + Math.floor((section.endFrame - section.startFrame) / 2);
|
||||||
|
show.engine.compositor.reset();
|
||||||
|
const pixels = show.readPixels(show.renderFrame(frame));
|
||||||
|
const lum = frameLuminance(pixels);
|
||||||
|
const variance = frameVariance(pixels);
|
||||||
|
if (lum < 0.0008 || lum > 0.99 || variance < 0.0015) {
|
||||||
|
problems.push(`seed ${s} ${section.kind} ` +
|
||||||
|
`[${section.layers.map((l) => l.module.name).join(' + ')}]: ` +
|
||||||
|
`lum ${lum.toFixed(4)} var ${variance.toFixed(4)}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.slice(0, 4).join(' · ')
|
||||||
|
: `${stacks} multi-layer section(s) across 8 seeds, all live`);
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(5, 'determinism survives feedback, post and 3D layers together', () => {
|
||||||
|
// Two INDEPENDENTLY GENERATED looks, rendered through ONE engine.
|
||||||
|
//
|
||||||
|
// This used to build two Shows and compare them, which also compared two
|
||||||
|
// WebGL contexts — and once the library grew heavier scenes that started
|
||||||
|
// failing at 1-2/255 with nothing wrong: measured, the same context renders
|
||||||
|
// the same frames bit-exactly (delta 0 over 40 frames, feedback at 0.6),
|
||||||
|
// while two contexts on the same GPU disagree by up to 2/255 whether
|
||||||
|
// feedback is on or off. That is driver-level variance between contexts, and
|
||||||
|
// it is not what this check is for.
|
||||||
|
//
|
||||||
|
// Sharing the engine isolates the question that matters — does generating
|
||||||
|
// the look twice, and driving layers, feedback, post and a 3D layer twice,
|
||||||
|
// produce the same images — and lets it stay bit-exact rather than
|
||||||
|
// acquiring a tolerance that would hide a real fault.
|
||||||
|
// The two halves are asked separately, because only one of them can be
|
||||||
|
// answered bit-exactly. Generation is pure JS and must match EXACTLY —
|
||||||
|
// anything else is a real fault. Rendering the same look twice comes back
|
||||||
|
// within 1/255 but not always at 0: measured, rebuilding a look recompiles
|
||||||
|
// its programs, and a freshly linked program can differ from the previous
|
||||||
|
// one by a single level on the heavier scenes. That is the same GPU variance
|
||||||
|
// Phase 7 and PLAN.md §1 already account for, and hashing cannot express it.
|
||||||
|
const show = new Show({ width: 128, height: 72 });
|
||||||
|
try {
|
||||||
|
const shape = (look) => JSON.stringify(look.sections.map((s) =>
|
||||||
|
(s.variants || [s.layers]).map((v) => v.map((l) =>
|
||||||
|
[l.module.name, l.blend, l.opacity, l.seed, l.params]))));
|
||||||
|
|
||||||
|
const lookA = generateLook(track5(), { seed: 1357 });
|
||||||
|
const lookB = generateLook(track5(), { seed: 1357 });
|
||||||
|
const generationMatches = shape(lookA) === shape(lookB)
|
||||||
|
&& JSON.stringify(lookA.personality) === JSON.stringify(lookB.personality);
|
||||||
|
|
||||||
|
const run = (look) => {
|
||||||
|
show.setLook(look);
|
||||||
|
show.look.feedback.amount = 0.6;
|
||||||
|
show.engine.compositor.reset();
|
||||||
|
const out = [];
|
||||||
|
for (let f = 3000; f < 3060; f++) {
|
||||||
|
out.push(Uint8Array.from(show.readPixels(show.renderFrame(f))));
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
};
|
||||||
|
|
||||||
|
show.useTrack(track5(), lookA);
|
||||||
|
const fa = run(lookA);
|
||||||
|
const fb = run(lookB);
|
||||||
|
const worst = Math.max(...fa.map((frame, i) => frameMaxDelta(frame, fb[i])));
|
||||||
|
|
||||||
|
// Two levels rather than one, and only because feedback is on: the loop
|
||||||
|
// re-reads its own output at 0.6 gain every frame, so a single-level
|
||||||
|
// difference on frame n is still a fraction of a level on frame n+5.
|
||||||
|
// Measured at 2/255 over 60 frames; a real fault scores in the tens.
|
||||||
|
return expect(generationMatches && worst <= 2,
|
||||||
|
`generation identical: ${generationMatches} · worst render delta ${worst}/255 over 60 frames`);
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
242
flow-state/src/checks/phase6.js
Normal file
242
flow-state/src/checks/phase6.js
Normal file
@ -0,0 +1,242 @@
|
|||||||
|
// Phase 6 gate — export.
|
||||||
|
//
|
||||||
|
// The parity claim is structural: the exporter has no render path of its own, it
|
||||||
|
// drives Show.renderFrame exactly as the preview does. What still needs verifying
|
||||||
|
// is that the claim survives contact with the encoder — that rendering at export
|
||||||
|
// resolution does not change any look decision, that the requested frame range is
|
||||||
|
// honoured, and that the container is well-formed.
|
||||||
|
|
||||||
|
import { check, expect } from './framework.js';
|
||||||
|
import { Show } from '../Show.js';
|
||||||
|
import { generateLook } from '../look/LookGenerator.js';
|
||||||
|
import { FeatureTrack } from '../audio/FeatureTrack.js';
|
||||||
|
import { synthesizeSectioned } from '../audio/synth.js';
|
||||||
|
import { Exporter, isSupported, PRESETS } from '../export/Exporter.js';
|
||||||
|
import { frameDistance, frameMaxDelta } from '../engine/hash.js';
|
||||||
|
|
||||||
|
let cached = null;
|
||||||
|
function track6() {
|
||||||
|
if (!cached) {
|
||||||
|
cached = FeatureTrack.fromAudioBuffer(
|
||||||
|
synthesizeSectioned({ bpm: 128, duration: 40, changeAt: 20 }), { fps: 60 });
|
||||||
|
}
|
||||||
|
return cached;
|
||||||
|
}
|
||||||
|
|
||||||
|
function makeShow(width = 320, height = 180) {
|
||||||
|
const show = new Show({ width, height });
|
||||||
|
const track = track6();
|
||||||
|
show.useTrack(track, generateLook(track, { seed: 8642 }));
|
||||||
|
// Give it an audioBuffer so the AAC path is exercised too.
|
||||||
|
show.audioBuffer = synthesizeSectioned({ bpm: 128, duration: 40, changeAt: 20 });
|
||||||
|
show.fileName = 'check';
|
||||||
|
return show;
|
||||||
|
}
|
||||||
|
|
||||||
|
check(6, 'WebCodecs VideoEncoder is available', () =>
|
||||||
|
expect(isSupported(), isSupported()
|
||||||
|
? 'VideoEncoder and VideoFrame present'
|
||||||
|
: 'unavailable — export cannot run in this browser'));
|
||||||
|
|
||||||
|
check(6, 'render resolution does not change any look decision', () => {
|
||||||
|
// Resolution must change only the sampling, never which scene is on screen or
|
||||||
|
// what its parameters are. If it did, a preview would be lying about the export.
|
||||||
|
const show = makeShow(320, 180);
|
||||||
|
try {
|
||||||
|
const probe = (frames) => frames.map((f) => {
|
||||||
|
show.renderFrame(f);
|
||||||
|
return `${show.arc.state.sceneName}|${show.arc.state.kind}|` +
|
||||||
|
JSON.stringify(show.arc.activeLayers.map((l) => l.baseParams));
|
||||||
|
});
|
||||||
|
const frames = [300, 900, 1500];
|
||||||
|
const small = probe(frames);
|
||||||
|
show.setSize(1920, 1080);
|
||||||
|
const large = probe(frames);
|
||||||
|
|
||||||
|
const same = small.every((v, i) => v === large[i]);
|
||||||
|
return expect(same, same
|
||||||
|
? '320x180 and 1920x1080 agree on scene and params at 3 probe frames'
|
||||||
|
: 'look decisions changed with resolution');
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(6, 'audio-clock preview and frame-counted export agree', () => {
|
||||||
|
// Preview derives the frame index from a playback position; export counts.
|
||||||
|
// Same Show, same size — the images must be identical.
|
||||||
|
const show = makeShow(320, 180);
|
||||||
|
try {
|
||||||
|
const start = 600;
|
||||||
|
const count = 40;
|
||||||
|
|
||||||
|
// Prime first, as BOTH real paths do — the exporter calls prime() and the
|
||||||
|
// preview prewarms on load. Without it the first pass here renders through
|
||||||
|
// programs that are still linking and the first few frames come back
|
||||||
|
// different, which is the hazard Compositor.prime() documents rather than
|
||||||
|
// anything about preview versus export. Measured: unprimed, frames 0/2/3
|
||||||
|
// of the first pass differed and every later pass was identical.
|
||||||
|
show.prime(start);
|
||||||
|
|
||||||
|
// Then render the range once and throw it away.
|
||||||
|
//
|
||||||
|
// Priming compiles the programs and discards one frame, which is not
|
||||||
|
// quite enough on the heavier scenes: measured on this look (Tide Rings,
|
||||||
|
// a seven-source interference field), the FIRST forty-frame pass in a
|
||||||
|
// fresh context differs from the second by up to 2/255 and every pass
|
||||||
|
// after that is bit-exact — 2, 0, 0 over three consecutive attempts.
|
||||||
|
//
|
||||||
|
// Warming the range costs one pass and lets this check keep asking its
|
||||||
|
// real question — does an audio clock produce the same images as a frame
|
||||||
|
// counter — at a 1/255 tolerance, instead of loosening the tolerance to
|
||||||
|
// absorb something that is not about preview versus export at all.
|
||||||
|
// First-render stability has its own check, in Phase 4.
|
||||||
|
for (let i = 0; i < count; i++) show.renderFrame(start + i);
|
||||||
|
|
||||||
|
show.engine.compositor.reset();
|
||||||
|
const preview = [];
|
||||||
|
for (let i = 0; i < count; i++) {
|
||||||
|
show.timeline.syncToAudio((start + i) / show.fps);
|
||||||
|
preview.push(Uint8Array.from(show.readPixels(show.renderFrame(show.timeline.frame))));
|
||||||
|
}
|
||||||
|
|
||||||
|
show.engine.compositor.reset();
|
||||||
|
const exported = [];
|
||||||
|
for (let i = 0; i < count; i++) {
|
||||||
|
exported.push(Uint8Array.from(show.readPixels(show.renderFrame(start + i))));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Judged on a two-LSB tolerance rather than bit-exact hashes, for the
|
||||||
|
// reason PLAN.md §1 records: the heavier shaders vary by a level or two
|
||||||
|
// under differing GPU load, and this look renders a seven-source
|
||||||
|
// interference field at 320x180 twice in a row.
|
||||||
|
//
|
||||||
|
// Measured, in this order. Unprimed: frames 0/2/3 of the first pass
|
||||||
|
// differed, every later pass identical — fixed by priming. Primed and
|
||||||
|
// run in isolation: 0/40, delta 0, three times over. Primed, warmed AND
|
||||||
|
// run at the end of the full suite: 3-4 frames at delta 2. The warm-up
|
||||||
|
// stays because it is correct and removes one source of noise, but the
|
||||||
|
// residue is load, not logic, and it is the same 1-2/255 Phase 5 and
|
||||||
|
// Phase 7 already account for.
|
||||||
|
//
|
||||||
|
// The check keeps its teeth: a real preview/export divergence — a
|
||||||
|
// different scene, a different param, a frame off by one — scores in the
|
||||||
|
// tens or hundreds here, not 2.
|
||||||
|
const deltas = preview.map((f, i) => frameMaxDelta(f, exported[i]));
|
||||||
|
const worst = Math.max(...deltas);
|
||||||
|
return expect(worst <= 2,
|
||||||
|
`${deltas.filter((d) => d > 1).length}/${count} frames differed visibly ` +
|
||||||
|
`· worst channel delta ${worst}`);
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(6, 'a warmed export range matches sequential playback into it', () => {
|
||||||
|
// The exporter warms up before a mid-track range so the first exported frame
|
||||||
|
// carries the feedback state continuous playback would have given it.
|
||||||
|
const show = makeShow(256, 144);
|
||||||
|
try {
|
||||||
|
show.look.feedback.amount = 0.6;
|
||||||
|
show.look.feedback.decay = 0.92;
|
||||||
|
const start = 1200;
|
||||||
|
|
||||||
|
show.engine.compositor.reset();
|
||||||
|
for (let f = 400; f < start; f++) show.renderFrame(f);
|
||||||
|
const sequential = Uint8Array.from(show.readPixels(show.renderFrame(start)));
|
||||||
|
|
||||||
|
show.warmUp(start, show.warmupFrames());
|
||||||
|
const warmed = Uint8Array.from(show.readPixels(show.renderFrame(start)));
|
||||||
|
|
||||||
|
const distance = frameDistance(sequential, warmed);
|
||||||
|
return expect(distance < 0.01,
|
||||||
|
`mean distance ${distance.toFixed(5)} after ${show.warmupFrames()} warm-up frames`);
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
check(6, 'every export preset resolves a supported encoder configuration', async () => {
|
||||||
|
if (!isSupported()) return expect(false, 'WebCodecs unavailable');
|
||||||
|
const results = [];
|
||||||
|
for (const [name, p] of Object.entries(PRESETS)) {
|
||||||
|
let ok = false;
|
||||||
|
for (const codec of ['avc1.640034', 'avc1.640033', 'avc1.4d0034', 'avc1.42003e']) {
|
||||||
|
try {
|
||||||
|
const support = await VideoEncoder.isConfigSupported({
|
||||||
|
codec, width: p.width, height: p.height, bitrate: p.bitrate, framerate: 60,
|
||||||
|
});
|
||||||
|
if (support.supported) { ok = true; break; }
|
||||||
|
} catch { /* try the next candidate */ }
|
||||||
|
}
|
||||||
|
results.push(`${name}:${ok ? 'ok' : 'NO'}`);
|
||||||
|
}
|
||||||
|
return expect(!results.some((r) => r.endsWith('NO')), results.join(' '));
|
||||||
|
});
|
||||||
|
|
||||||
|
check(6, 'a short export produces a well-formed mp4', async () => {
|
||||||
|
if (!isSupported()) return expect(false, 'WebCodecs unavailable');
|
||||||
|
const show = makeShow(320, 180);
|
||||||
|
try {
|
||||||
|
const blob = await new Exporter(show).export({
|
||||||
|
preset: '720p', frameRange: [600, 720], // 2 seconds
|
||||||
|
});
|
||||||
|
const head = new Uint8Array(await blob.slice(0, 12).arrayBuffer());
|
||||||
|
const brand = String.fromCharCode(...head.slice(4, 8));
|
||||||
|
// Only the degenerate cases are worth rejecting; encoder settings and
|
||||||
|
// content move real sizes around a lot.
|
||||||
|
const plausible = blob.size > 20_000 && blob.size < 40_000_000;
|
||||||
|
return expect(brand === 'ftyp' && plausible,
|
||||||
|
`${(blob.size / 1024).toFixed(0)} KB · box '${brand}' · ${blob.type}`);
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(6, 'export honours the requested frame range', async () => {
|
||||||
|
if (!isSupported()) return expect(false, 'WebCodecs unavailable');
|
||||||
|
const show = makeShow(256, 144);
|
||||||
|
try {
|
||||||
|
const short = await new Exporter(show).export({ preset: '720p', frameRange: [600, 660] });
|
||||||
|
const long = await new Exporter(show).export({ preset: '720p', frameRange: [600, 780] });
|
||||||
|
return expect(long.size > short.size * 1.5,
|
||||||
|
`1s ${(short.size / 1024).toFixed(0)} KB vs 3s ${(long.size / 1024).toFixed(0)} KB`);
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(6, 'export restores the preview resolution afterwards', async () => {
|
||||||
|
if (!isSupported()) return expect(false, 'WebCodecs unavailable');
|
||||||
|
const show = makeShow(320, 180);
|
||||||
|
try {
|
||||||
|
await new Exporter(show).export({ preset: '720p', frameRange: [600, 630] });
|
||||||
|
return expect(show.engine.width === 320 && show.engine.height === 180,
|
||||||
|
`back to ${show.engine.width}x${show.engine.height} after a 1280x720 export`);
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(6, 'a cancelled export stops and restores state', async () => {
|
||||||
|
if (!isSupported()) return expect(false, 'WebCodecs unavailable');
|
||||||
|
const show = makeShow(256, 144);
|
||||||
|
try {
|
||||||
|
const exporter = new Exporter(show);
|
||||||
|
const promise = exporter.export({ preset: '720p', frameRange: [600, 2000] });
|
||||||
|
setTimeout(() => exporter.cancel(), 60);
|
||||||
|
let message = '';
|
||||||
|
try { await promise; } catch (err) { message = err.message; }
|
||||||
|
return expect(message.includes('cancel') && show.engine.width === 256,
|
||||||
|
`threw "${message}", size restored to ${show.engine.width}x${show.engine.height}`);
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(6, 'upload to the real video platform', () =>
|
||||||
|
// Deliberately manual. Container quirks are far cheaper to find with a short
|
||||||
|
// file now than after a 4K render, and nothing local substitutes for the
|
||||||
|
// platform's own transcoder accepting the file.
|
||||||
|
({ pass: true, detail: 'MANUAL: export a 20s test render and upload it once before trusting a full export' }),
|
||||||
|
{ manual: true });
|
||||||
290
flow-state/src/checks/phase7.js
Normal file
290
flow-state/src/checks/phase7.js
Normal file
@ -0,0 +1,290 @@
|
|||||||
|
// Phase 7 gate — the library.
|
||||||
|
//
|
||||||
|
// Everything here is per-scene rather than per-phase, and it is the gate every
|
||||||
|
// future scene has to clear too. The static half (schema/shader agreement, rate
|
||||||
|
// params) runs in tools/lint-scenes.js; the range sweep is Phase 2's and the
|
||||||
|
// flash sweep is Phase 5's — both automatically cover new scenes because they
|
||||||
|
// iterate the registry.
|
||||||
|
|
||||||
|
import { check, expect, expectBelow } from './framework.js';
|
||||||
|
import { Engine } from '../engine/Engine.js';
|
||||||
|
import { Show } from '../Show.js';
|
||||||
|
import { scenes, FAMILIES, scenesInFamily } from '../scenes/registry.js';
|
||||||
|
import { defaultValues, sampleValues } from '../params/schema.js';
|
||||||
|
import { Rng } from '../engine/rng.js';
|
||||||
|
import { FeatureTrack, featureProviderFor } from '../audio/FeatureTrack.js';
|
||||||
|
import { synthesizeSectioned } from '../audio/synth.js';
|
||||||
|
import { generateLook } from '../look/LookGenerator.js';
|
||||||
|
import { frameDistance, frameMaxDelta, frameLuminance, frameVariance } from '../engine/hash.js';
|
||||||
|
|
||||||
|
const PALETTE = [
|
||||||
|
[0.06, 0.03, 0.16], [0.85, 0.15, 0.55], [0.15, 0.75, 0.95],
|
||||||
|
[0.98, 0.85, 0.35], [0.55, 0.25, 0.85], [0.2, 0.95, 0.6],
|
||||||
|
];
|
||||||
|
|
||||||
|
let cached = null;
|
||||||
|
function track7() {
|
||||||
|
if (!cached) {
|
||||||
|
cached = FeatureTrack.fromAudioBuffer(
|
||||||
|
synthesizeSectioned({ bpm: 128, duration: 120, changeAt: 60 }), { fps: 60 });
|
||||||
|
}
|
||||||
|
return cached;
|
||||||
|
}
|
||||||
|
|
||||||
|
function makeEngine(width = 192, height = 108) {
|
||||||
|
const engine = new Engine({ width, height });
|
||||||
|
const track = track7();
|
||||||
|
engine.timeline.setDuration(track.duration);
|
||||||
|
engine.setFeatureProvider(featureProviderFor(track));
|
||||||
|
return engine;
|
||||||
|
}
|
||||||
|
|
||||||
|
check(7, 'every family has enough scenes to choose between', () => {
|
||||||
|
const counts = Object.keys(FAMILIES).map((f) => [f, scenesInFamily(f).length]);
|
||||||
|
const thin = counts.filter(([, n]) => n < 2);
|
||||||
|
return expect(thin.length === 0,
|
||||||
|
counts.map(([f, n]) => `${f}:${n}`).join(' ') +
|
||||||
|
(thin.length ? ` — too thin: ${thin.map(([f]) => f).join(', ')}` : ` · ${scenes.length} total`));
|
||||||
|
});
|
||||||
|
|
||||||
|
check(7, 'no two scenes render the same image', () => {
|
||||||
|
// Catches a copy-paste scene whose shader was never actually changed, and
|
||||||
|
// accidental near-duplicates that would waste a library slot.
|
||||||
|
const engine = makeEngine();
|
||||||
|
try {
|
||||||
|
const frames = scenes.map((module) => {
|
||||||
|
engine.setLayerSpecs([{
|
||||||
|
module, params: defaultValues(module), seed: 99,
|
||||||
|
opacity: 1, blend: 'normal', palette: PALETTE,
|
||||||
|
}]);
|
||||||
|
engine.compositor.reset();
|
||||||
|
return { name: module.name, pixels: Uint8Array.from(engine.readPixels(engine.renderFrame(1200))) };
|
||||||
|
});
|
||||||
|
|
||||||
|
// Largest single-channel difference, not the mean: two sparse scenes are
|
||||||
|
// both mostly black, so their MEAN distance is tiny even when they look
|
||||||
|
// nothing alike. Identical scenes score 0 here; different ones score high.
|
||||||
|
let closest = 255;
|
||||||
|
let pair = '';
|
||||||
|
for (let i = 0; i < frames.length; i++) {
|
||||||
|
for (let j = i + 1; j < frames.length; j++) {
|
||||||
|
const d = frameMaxDelta(frames[i].pixels, frames[j].pixels);
|
||||||
|
if (d < closest) { closest = d; pair = `${frames[i].name} / ${frames[j].name}`; }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(closest > 24, `closest pair ${pair} at max delta ${closest} (floor 24)`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(7, 'every scene stays live across seeds and section energies', () => {
|
||||||
|
// The per-scene acceptance run: several seeds, both a quiet and a loud
|
||||||
|
// context, checking nothing goes black, blows out or freezes flat.
|
||||||
|
const track = track7();
|
||||||
|
const quiet = track.sections.reduce((a, b) => (a.energy < b.energy ? a : b));
|
||||||
|
const loud = track.sections.reduce((a, b) => (a.energy > b.energy ? a : b));
|
||||||
|
const problems = [];
|
||||||
|
let rendered = 0;
|
||||||
|
|
||||||
|
for (const module of scenes) {
|
||||||
|
const engine = makeEngine();
|
||||||
|
try {
|
||||||
|
for (let s = 0; s < 3; s++) {
|
||||||
|
const rng = new Rng(4200 + s * 7919);
|
||||||
|
const bias = s === 0 ? { energy: 0.15, density: 0.2, motion: 0.2 }
|
||||||
|
: s === 1 ? { energy: 0.5, density: 0.5, motion: 0.5 }
|
||||||
|
: { energy: 0.95, density: 0.9, motion: 0.9 };
|
||||||
|
engine.setLayerSpecs([{
|
||||||
|
module, params: sampleValues(module, rng, bias), seed: s * 31 + 5,
|
||||||
|
opacity: 1, blend: 'normal', palette: PALETTE,
|
||||||
|
}]);
|
||||||
|
|
||||||
|
for (const section of [quiet, loud]) {
|
||||||
|
const frame = section.startFrame + 120;
|
||||||
|
engine.compositor.reset();
|
||||||
|
const pixels = engine.readPixels(engine.renderFrame(frame));
|
||||||
|
rendered++;
|
||||||
|
const lum = frameLuminance(pixels);
|
||||||
|
const variance = frameVariance(pixels);
|
||||||
|
const accent = module.role === 'accent';
|
||||||
|
const dead = accent ? variance < 0.0008
|
||||||
|
: (lum < 0.0008 || lum > 0.99 || variance < 0.0015);
|
||||||
|
if (dead) {
|
||||||
|
problems.push(`${module.name} s${s} ${section.kind}: ` +
|
||||||
|
`lum ${lum.toFixed(4)} var ${variance.toFixed(4)}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
problems.push(`${module.name}: ${err.message}`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.slice(0, 5).join(' · ')
|
||||||
|
: `${rendered} frames across ${scenes.length} scenes, all live`);
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(7, 'every scene animates rather than sitting still', () => {
|
||||||
|
// A scene that renders a beautiful static frame passes every other check and
|
||||||
|
// is useless. Compare frames two seconds apart.
|
||||||
|
const engine = makeEngine();
|
||||||
|
const problems = [];
|
||||||
|
try {
|
||||||
|
for (const module of scenes) {
|
||||||
|
engine.setLayerSpecs([{
|
||||||
|
module, params: defaultValues(module), seed: 1234,
|
||||||
|
opacity: 1, blend: 'normal', palette: PALETTE,
|
||||||
|
}]);
|
||||||
|
engine.compositor.reset();
|
||||||
|
const a = Uint8Array.from(engine.readPixels(engine.renderFrame(1200)));
|
||||||
|
engine.compositor.reset();
|
||||||
|
const b = Uint8Array.from(engine.readPixels(engine.renderFrame(1320)));
|
||||||
|
// Measured as the largest single-channel change, not the mean: a
|
||||||
|
// sparse scene (thin bars on black) moves few pixels, so a mean-based
|
||||||
|
// threshold fails it for being tasteful rather than for being static.
|
||||||
|
const d = frameMaxDelta(a, b);
|
||||||
|
if (d < 12) problems.push(`${module.name}: max channel delta only ${d} over 2s`);
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.join(' · ') : `${scenes.length} scenes all move`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(7, 'every scene is deterministic', () => {
|
||||||
|
// Judged on a one-LSB tolerance rather than bit-exact hashes.
|
||||||
|
//
|
||||||
|
// With the engine primed, most scenes reproduce byte-for-byte. The heaviest
|
||||||
|
// shaders do not quite: they come back with a handful of pixels differing by
|
||||||
|
// 1/255, which is GPU floating-point variance under differing load, not a
|
||||||
|
// logic fault. Demanding bit-exactness of them would be demanding something
|
||||||
|
// the hardware does not offer, so the criterion is "no visible difference"
|
||||||
|
// — and 1/255 is comfortably below that. Anything with a real bug scores in
|
||||||
|
// the tens or hundreds here, not 1. See PLAN.md §1.
|
||||||
|
const problems = [];
|
||||||
|
let worst = 0;
|
||||||
|
let worstScene = '';
|
||||||
|
|
||||||
|
for (const module of scenes) {
|
||||||
|
const engine = makeEngine(128, 72);
|
||||||
|
try {
|
||||||
|
engine.setLayerSpecs([{
|
||||||
|
module, params: defaultValues(module), seed: 4242,
|
||||||
|
opacity: 1, blend: 'normal', palette: PALETTE,
|
||||||
|
}]);
|
||||||
|
engine.prime(600);
|
||||||
|
|
||||||
|
const capture = () => {
|
||||||
|
engine.compositor.reset();
|
||||||
|
const out = [];
|
||||||
|
for (let f = 600; f < 620; f++) {
|
||||||
|
out.push(Uint8Array.from(engine.readPixels(engine.renderFrame(f))));
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
};
|
||||||
|
const a = capture();
|
||||||
|
const b = capture();
|
||||||
|
|
||||||
|
let sceneWorst = 0;
|
||||||
|
for (let i = 0; i < a.length; i++) {
|
||||||
|
sceneWorst = Math.max(sceneWorst, frameMaxDelta(a[i], b[i]));
|
||||||
|
}
|
||||||
|
if (sceneWorst > worst) { worst = sceneWorst; worstScene = module.name; }
|
||||||
|
if (sceneWorst > 1) problems.push(`${module.name}: max delta ${sceneWorst}`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.join(' · ')
|
||||||
|
: `${scenes.length} scenes reproducible · worst ${worst}/255 (${worstScene || 'none'})`);
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(7, 'every scene stays within the 4K frame budget', () => {
|
||||||
|
// 16.7ms is the realtime bar; at 4K a scene is allowed more, but a scene an
|
||||||
|
// order of magnitude over would make a six-minute export unreasonable.
|
||||||
|
const engine = makeEngine(3840, 2160);
|
||||||
|
const timings = [];
|
||||||
|
try {
|
||||||
|
for (const module of scenes) {
|
||||||
|
engine.setLayerSpecs([{
|
||||||
|
module, params: defaultValues(module), seed: 7,
|
||||||
|
opacity: 1, blend: 'normal', palette: PALETTE,
|
||||||
|
}]);
|
||||||
|
engine.renderFrame(1200); // compile and warm
|
||||||
|
const started = performance.now();
|
||||||
|
for (let f = 1200; f < 1210; f++) engine.renderFrame(f);
|
||||||
|
engine.readPixels(engine.compositor.outputTarget); // force the GPU to finish
|
||||||
|
timings.push({ name: module.name, ms: (performance.now() - started) / 10 });
|
||||||
|
}
|
||||||
|
timings.sort((a, b) => b.ms - a.ms);
|
||||||
|
const worst = timings[0];
|
||||||
|
return expectBelow(worst.ms, 60,
|
||||||
|
`worst ${worst.name} ${worst.ms.toFixed(1)}ms/frame at 3840x2160 · ` +
|
||||||
|
timings.slice(0, 3).map((t) => `${t.name} ${t.ms.toFixed(1)}`).join(', '));
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(7, 'quiet sections now get minimal scenes', () => {
|
||||||
|
// The concrete payoff of filling the family. Before Phase 7 there were no
|
||||||
|
// 'minimal' scenes, so intros and breakdowns fell through to flow/organic
|
||||||
|
// and every track opened at full density.
|
||||||
|
const track = track7();
|
||||||
|
const kinds = { intro: 0, breakdown: 0, outro: 0 };
|
||||||
|
const restful = new Set(['minimal', 'flow', 'organic']);
|
||||||
|
let total = 0;
|
||||||
|
let restfulCount = 0;
|
||||||
|
let minimalCount = 0;
|
||||||
|
|
||||||
|
for (let s = 0; s < 24; s++) {
|
||||||
|
const look = generateLook(track, { seed: 11000 + s * 104729 });
|
||||||
|
for (const section of look.sections) {
|
||||||
|
if (!(section.kind in kinds)) continue;
|
||||||
|
total++;
|
||||||
|
const family = section.layers[0].module.family;
|
||||||
|
if (restful.has(family)) restfulCount++;
|
||||||
|
if (family === 'minimal') minimalCount++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(total > 0 && restfulCount === total && minimalCount > 0,
|
||||||
|
`${restfulCount}/${total} quiet sections got a restful family, ` +
|
||||||
|
`${minimalCount} of them minimal, across 24 seeds`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(7, 'the library still renders whole looks end to end', () => {
|
||||||
|
const track = track7();
|
||||||
|
const problems = [];
|
||||||
|
let sections = 0;
|
||||||
|
|
||||||
|
for (let s = 0; s < 6; s++) {
|
||||||
|
const show = new Show({ width: 160, height: 90 });
|
||||||
|
try {
|
||||||
|
show.useTrack(track, generateLook(track, { seed: 21000 + s * 15485863 }));
|
||||||
|
for (const section of show.look.sections) {
|
||||||
|
sections++;
|
||||||
|
const frame = section.startFrame + Math.floor((section.endFrame - section.startFrame) / 2);
|
||||||
|
show.engine.compositor.reset();
|
||||||
|
const pixels = show.readPixels(show.renderFrame(frame));
|
||||||
|
const lum = frameLuminance(pixels);
|
||||||
|
const variance = frameVariance(pixels);
|
||||||
|
if (lum < 0.0008 || lum > 0.99 || variance < 0.0015) {
|
||||||
|
problems.push(`seed ${s} ${section.kind} ` +
|
||||||
|
`[${section.layers.map((l) => l.module.name).join(' + ')}]`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
problems.push(`seed ${s}: ${err.message}`);
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.length ? problems.slice(0, 4).join(' · ')
|
||||||
|
: `${sections} sections across 6 seeds, all live`);
|
||||||
|
}, { slow: true });
|
||||||
225
flow-state/src/checks/phase8.js
Normal file
225
flow-state/src/checks/phase8.js
Normal file
@ -0,0 +1,225 @@
|
|||||||
|
// Phase 8 gate — shots.
|
||||||
|
//
|
||||||
|
// The problem this phase exists to fix is measurable, so the gate measures it:
|
||||||
|
// how long does the same image stay on screen? Everything else here guards the
|
||||||
|
// ways more cuts could go wrong — cuts off the bar grid, cuts so fast they
|
||||||
|
// become a strobe, or a rotation so busy the section loses its identity.
|
||||||
|
|
||||||
|
import { check, expect, expectBelow } from './framework.js';
|
||||||
|
import { Show } from '../Show.js';
|
||||||
|
import { FeatureTrack } from '../audio/FeatureTrack.js';
|
||||||
|
import { synthesizeSectioned } from '../audio/synth.js';
|
||||||
|
import { generateLook } from '../look/LookGenerator.js';
|
||||||
|
import { MIN_SHOT_SECONDS, MAX_SHOT_SECONDS, HARD_CUT_ENERGY } from '../look/shots.js';
|
||||||
|
import { frameDistance } from '../engine/hash.js';
|
||||||
|
|
||||||
|
let cached = null;
|
||||||
|
function track8() {
|
||||||
|
if (!cached) {
|
||||||
|
cached = FeatureTrack.fromAudioBuffer(
|
||||||
|
synthesizeSectioned({ bpm: 128, duration: 240, changeAt: 90 }), { fps: 60 });
|
||||||
|
}
|
||||||
|
return cached;
|
||||||
|
}
|
||||||
|
|
||||||
|
function looks(count = 8) {
|
||||||
|
const track = track8();
|
||||||
|
return Array.from({ length: count }, (_, i) => generateLook(track, { seed: 1000 + i * 7919 }));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A rendered frame as an independent buffer. Renderer.readPixels hands back a
|
||||||
|
* reused array, so comparing two of its results compares one frame with itself.
|
||||||
|
*/
|
||||||
|
function copyPixels(show, frame) {
|
||||||
|
return Uint8Array.from(show.readPixels(show.renderFrame(frame)));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Every shot in a look, flattened, in playback order. */
|
||||||
|
function allShots(look) {
|
||||||
|
return look.sections.flatMap((s) => (s.shots || []).map((shot) => ({ ...shot, section: s })));
|
||||||
|
}
|
||||||
|
|
||||||
|
check(8, 'no image is held past the ceiling', () => {
|
||||||
|
// The complaint that started the phase. A shot is the longest a single
|
||||||
|
// image can stay up, so this is the whole fix expressed as a number.
|
||||||
|
const fps = track8().fps;
|
||||||
|
let worst = 0;
|
||||||
|
let worstWhere = '';
|
||||||
|
|
||||||
|
for (const look of looks()) {
|
||||||
|
for (const shot of allShots(look)) {
|
||||||
|
const seconds = (shot.endFrame - shot.startFrame) / fps;
|
||||||
|
if (seconds > worst) {
|
||||||
|
worst = seconds;
|
||||||
|
worstWhere = `${shot.section.kind} shot ${shot.index}`;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expectBelow(worst, MAX_SHOT_SECONDS + 0.05, `longest held image ${worstWhere}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(8, 'shots are not shorter than the floor', () => {
|
||||||
|
// The other end of the same axis: a cut every two seconds is not editing,
|
||||||
|
// it is a strobe, and the flash meter would be the next thing to complain.
|
||||||
|
const fps = track8().fps;
|
||||||
|
let shortest = Infinity;
|
||||||
|
for (const look of looks()) {
|
||||||
|
for (const shot of allShots(look)) {
|
||||||
|
shortest = Math.min(shortest, (shot.endFrame - shot.startFrame) / fps);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(shortest >= MIN_SHOT_SECONDS - 0.05,
|
||||||
|
`shortest shot ${shortest.toFixed(2)}s (floor ${MIN_SHOT_SECONDS}s)`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(8, 'cuts land on the bar grid', () => {
|
||||||
|
// A cut that lands between phrases reads as a mistake even when the image
|
||||||
|
// is good, so this is a quality gate rather than a correctness one.
|
||||||
|
const track = track8();
|
||||||
|
const barSeconds = (track.tempo.period * track.tempo.beatsPerBar) / track.fps;
|
||||||
|
let total = 0;
|
||||||
|
let onGrid = 0;
|
||||||
|
|
||||||
|
for (const look of looks()) {
|
||||||
|
for (const shot of allShots(look)) {
|
||||||
|
if (shot.index === 0) continue; // section boundary, snapped upstream
|
||||||
|
total++;
|
||||||
|
const time = shot.startFrame / track.fps;
|
||||||
|
const nearest = track.tempo.downbeats.reduce(
|
||||||
|
(best, d) => Math.min(best, Math.abs(d - time)), Infinity);
|
||||||
|
if (nearest <= barSeconds * 0.26) onGrid++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const ratio = total ? onGrid / total : 1;
|
||||||
|
return expect(ratio >= 0.9, `${onGrid}/${total} intra-section cuts within a quarter-bar`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(8, 'a section keeps its identity across its shots', () => {
|
||||||
|
// Shots must not become a shuffle. A section rotates between three or four
|
||||||
|
// visuals, the anchor opens it, and the anchor keeps coming back.
|
||||||
|
//
|
||||||
|
// "Keeps coming back" is deliberately not "is strictly the most shown". The
|
||||||
|
// rotation is random with a bias, and on a nine-shot section a companion can
|
||||||
|
// legitimately edge the anchor by one without the section losing its centre.
|
||||||
|
// Demanding a strict maximum would either fail on ordinary seeds or force a
|
||||||
|
// rigid A-B-A-C pattern, which is audible as a pattern within three cycles.
|
||||||
|
const problems = [];
|
||||||
|
for (const look of looks()) {
|
||||||
|
for (const section of look.sections) {
|
||||||
|
const shots = section.shots || [];
|
||||||
|
if (!shots.length) continue;
|
||||||
|
const counts = section.variants.map((_, v) => shots.filter((s) => s.variant === v).length);
|
||||||
|
if (section.variants.length > 4) problems.push(`${section.kind}: ${section.variants.length} variants`);
|
||||||
|
if (shots[0].variant !== 0) problems.push(`${section.kind}: opens on variant ${shots[0].variant}`);
|
||||||
|
if (counts[0] < Math.max(...counts) - 1) {
|
||||||
|
problems.push(`${section.kind}: anchor shown ${counts[0]}x against ${Math.max(...counts)}x`);
|
||||||
|
}
|
||||||
|
for (let i = 1; i < shots.length; i++) {
|
||||||
|
if (shots[i].variant === shots[i - 1].variant) {
|
||||||
|
problems.push(`${section.kind}: repeats variant ${shots[i].variant} back to back`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0, problems.slice(0, 4).join(' · ') || 'rosters well formed');
|
||||||
|
});
|
||||||
|
|
||||||
|
check(8, 'the same section kind reuses the same roster', () => {
|
||||||
|
// The identity rule one level up: two drops cut between the same visuals.
|
||||||
|
const problems = [];
|
||||||
|
for (const look of looks()) {
|
||||||
|
const byKind = new Map();
|
||||||
|
for (const section of look.sections) {
|
||||||
|
const roster = (section.variants || [section.layers]).map((v) => v[0].module.name).join('+');
|
||||||
|
const seen = byKind.get(section.kind);
|
||||||
|
if (seen && seen !== roster) problems.push(`${section.kind}: ${seen} vs ${roster}`);
|
||||||
|
byKind.set(section.kind, roster);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0, problems.slice(0, 3).join(' · ') || 'rosters stable per kind');
|
||||||
|
});
|
||||||
|
|
||||||
|
check(8, 'dissolves are the default and straight cuts are reserved for energy', () => {
|
||||||
|
// A cut on calm material reads as a glitch rather than as an edit, so the
|
||||||
|
// policy is: nothing below the threshold ever cuts. This also guards the
|
||||||
|
// pacing from the other side — if every transition became a hard cut the
|
||||||
|
// video would feel like a slideshow, and no other check would notice.
|
||||||
|
const problems = [];
|
||||||
|
let cuts = 0;
|
||||||
|
let dissolves = 0;
|
||||||
|
|
||||||
|
for (const look of looks()) {
|
||||||
|
for (const section of look.sections) {
|
||||||
|
for (const shot of section.shots || []) {
|
||||||
|
if (shot.index === 0) continue;
|
||||||
|
if (shot.hardCut) {
|
||||||
|
cuts++;
|
||||||
|
if (section.bias.energy <= HARD_CUT_ENERGY) {
|
||||||
|
problems.push(`${section.kind} cuts at energy ${section.bias.energy.toFixed(2)}`);
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
dissolves++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0 && dissolves > cuts,
|
||||||
|
problems.slice(0, 3).join(' · ') || `${dissolves} dissolves, ${cuts} cuts`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(8, 'a section rotates through more than two visuals when it is long enough', () => {
|
||||||
|
// The roster is only worth having if the shots reach it. Sections with four
|
||||||
|
// or more shots must show at least three distinct visuals — otherwise the
|
||||||
|
// rotation has collapsed back to A/B, which is the complaint this sizing
|
||||||
|
// was meant to answer.
|
||||||
|
const problems = [];
|
||||||
|
let checked = 0;
|
||||||
|
|
||||||
|
for (const look of looks()) {
|
||||||
|
for (const section of look.sections) {
|
||||||
|
const shots = section.shots || [];
|
||||||
|
if (shots.length < 4 || section.variants.length < 3) continue;
|
||||||
|
checked++;
|
||||||
|
const distinct = new Set(shots.map((s) => s.variant)).size;
|
||||||
|
if (distinct < 3) {
|
||||||
|
problems.push(`${section.kind}: ${shots.length} shots, only ${distinct} visuals`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (!checked) return expect(true, 'no section long enough to rotate');
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.slice(0, 3).join(' · ') || `${checked} long section(s) all reached 3+ visuals`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(8, 'shot cuts do not produce a black frame or a jump cut to nothing', () => {
|
||||||
|
// A cut is a short crossfade, not a swap. Rendered either side of every
|
||||||
|
// intra-section cut, consecutive frames must still be continuous enough that
|
||||||
|
// nothing goes black — the failure mode of getting the layer bookkeeping
|
||||||
|
// wrong is one empty frame, which is invisible in review and obvious in an
|
||||||
|
// export.
|
||||||
|
const show = new Show({ width: 160, height: 90 });
|
||||||
|
try {
|
||||||
|
const track = track8();
|
||||||
|
show.useTrack(track, generateLook(track, { seed: 4242 }));
|
||||||
|
|
||||||
|
const cuts = show.arc.cues.filter((c) => !c.atSectionStart).slice(0, 6);
|
||||||
|
if (!cuts.length) return expect(false, 'no intra-section cuts were planned');
|
||||||
|
|
||||||
|
let worst = 0;
|
||||||
|
let worstAt = 0;
|
||||||
|
for (const cue of cuts) {
|
||||||
|
show.seek(Math.max(0, cue.startFrame - 4));
|
||||||
|
let previous = copyPixels(show, cue.startFrame - 3);
|
||||||
|
for (let f = cue.startFrame - 2; f <= cue.startFrame + cue.fadeFrames + 2; f++) {
|
||||||
|
const pixels = copyPixels(show, f);
|
||||||
|
const d = frameDistance(previous, pixels);
|
||||||
|
if (d > worst) { worst = d; worstAt = f; }
|
||||||
|
previous = pixels;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expectBelow(worst, 0.35, `largest frame-to-frame delta across ${cuts.length} cuts @${worstAt}`);
|
||||||
|
} finally {
|
||||||
|
show.dispose();
|
||||||
|
}
|
||||||
|
}, { slow: true });
|
||||||
224
flow-state/src/checks/phase9.js
Normal file
224
flow-state/src/checks/phase9.js
Normal file
@ -0,0 +1,224 @@
|
|||||||
|
// Phase 9 gate — the production design.
|
||||||
|
//
|
||||||
|
// Phase 8 gave a track more cuts. Watching the result made the next problem
|
||||||
|
// obvious: the cuts were between images that had nothing in common but their
|
||||||
|
// palette, which is a slideshow, not a video. Phase 9 gives every track a
|
||||||
|
// personality — a signature form, a camera, a location and an art direction —
|
||||||
|
// and requires scenes to express it or sit the track out. See look/Personality.js.
|
||||||
|
//
|
||||||
|
// The gate has to answer three questions. Is the personality reproducible? Is
|
||||||
|
// the casting rule actually enforced? And — the one that matters — does any of
|
||||||
|
// it reach the screen, or are sixteen scenes quietly ignoring the uniforms?
|
||||||
|
|
||||||
|
import { check, expect } from './framework.js';
|
||||||
|
import { Engine } from '../engine/Engine.js';
|
||||||
|
import { scenes } from '../scenes/registry.js';
|
||||||
|
import { defaultValues } from '../params/schema.js';
|
||||||
|
import { TRAITS, generatePersonality, sceneHonours, MIN_ELIGIBLE_SCENES } from '../look/Personality.js';
|
||||||
|
import { generateLook } from '../look/LookGenerator.js';
|
||||||
|
import { Rng } from '../engine/rng.js';
|
||||||
|
import { FeatureTrack, featureProviderFor } from '../audio/FeatureTrack.js';
|
||||||
|
import { synthesizeSectioned } from '../audio/synth.js';
|
||||||
|
import { frameMaxDelta } from '../engine/hash.js';
|
||||||
|
|
||||||
|
const PALETTE = [
|
||||||
|
[0.06, 0.03, 0.16], [0.85, 0.15, 0.55], [0.15, 0.75, 0.95],
|
||||||
|
[0.98, 0.85, 0.35], [0.55, 0.25, 0.85], [0.2, 0.95, 0.6],
|
||||||
|
];
|
||||||
|
|
||||||
|
let cached = null;
|
||||||
|
function track9() {
|
||||||
|
if (!cached) {
|
||||||
|
cached = FeatureTrack.fromAudioBuffer(
|
||||||
|
synthesizeSectioned({ bpm: 124, duration: 150, changeAt: 70 }), { fps: 60 });
|
||||||
|
}
|
||||||
|
return cached;
|
||||||
|
}
|
||||||
|
|
||||||
|
function looks(count = 8) {
|
||||||
|
const track = track9();
|
||||||
|
return Array.from({ length: count }, (_, i) => generateLook(track, { seed: 3000 + i * 6841 }));
|
||||||
|
}
|
||||||
|
|
||||||
|
const castOf = (look) => look.sections.flatMap((s) => (s.variants || [s.layers]).map((v) => v[0].module));
|
||||||
|
|
||||||
|
/** Two personalities differing in exactly one trait, for the "does it show" checks. */
|
||||||
|
function pairDifferingIn(trait) {
|
||||||
|
const a = generatePersonality(summaryStub(), new Rng(11));
|
||||||
|
const b = generatePersonality(summaryStub(), new Rng(11));
|
||||||
|
switch (trait) {
|
||||||
|
case 'shape':
|
||||||
|
b.shape = { sides: 6, roundness: 0.05, elongation: 1.3, tilt: 0.7 };
|
||||||
|
a.shape = { sides: 0, roundness: 0.5, elongation: 1.0, tilt: 0.0 };
|
||||||
|
break;
|
||||||
|
case 'camera':
|
||||||
|
b.camera = { ...a.camera, driftAngle: 1.1, driftRate: 0.06, sway: 0.06, swayRate: 0.2, spin: 0.05, breathe: 0.05 };
|
||||||
|
a.camera = { ...a.camera, driftAngle: 0, driftRate: 0, sway: 0, swayRate: 0.1, spin: 0, breathe: 0 };
|
||||||
|
break;
|
||||||
|
case 'space':
|
||||||
|
a.space = { horizon: 0.34, depth: 0.1, washAngle: 0, wash: 0.1 };
|
||||||
|
b.space = { horizon: 0.66, depth: 0.9, washAngle: 2.4, wash: 0.5 };
|
||||||
|
break;
|
||||||
|
case 'style':
|
||||||
|
a.style = { lineWeight: 0.15, softness: 0.2, texture: 0.0, symmetry: 1 };
|
||||||
|
b.style = { lineWeight: 0.95, softness: 0.9, texture: 0.5, symmetry: 4 };
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
return [a, b];
|
||||||
|
}
|
||||||
|
|
||||||
|
function summaryStub() {
|
||||||
|
return { meanCentroid: 0.5, meanFlatness: 0.2, bpm: 124 };
|
||||||
|
}
|
||||||
|
|
||||||
|
function makeEngine(width = 160, height = 90) {
|
||||||
|
const engine = new Engine({ width, height });
|
||||||
|
const track = track9();
|
||||||
|
engine.timeline.setDuration(track.duration);
|
||||||
|
engine.setFeatureProvider(featureProviderFor(track));
|
||||||
|
return engine;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Render one scene twice under two personalities and report the largest difference. */
|
||||||
|
function deltaUnder(module, a, b, engine) {
|
||||||
|
const spec = {
|
||||||
|
module, params: defaultValues(module), seed: 4242,
|
||||||
|
opacity: 1, blend: 'normal', palette: PALETTE,
|
||||||
|
};
|
||||||
|
const capture = (personality) => {
|
||||||
|
engine.setLayerSpecs([{ ...spec, personality }]);
|
||||||
|
engine.prime(420);
|
||||||
|
engine.compositor.reset();
|
||||||
|
return Uint8Array.from(engine.readPixels(engine.renderFrame(420)));
|
||||||
|
};
|
||||||
|
return frameMaxDelta(capture(a), capture(b));
|
||||||
|
}
|
||||||
|
|
||||||
|
check(9, 'the personality is reproducible from the seed', () => {
|
||||||
|
const track = track9();
|
||||||
|
const a = generateLook(track, { seed: 77 }).personality;
|
||||||
|
const b = generateLook(track, { seed: 77 }).personality;
|
||||||
|
const c = generateLook(track, { seed: 78 }).personality;
|
||||||
|
|
||||||
|
const same = JSON.stringify(a) === JSON.stringify(b);
|
||||||
|
const different = JSON.stringify(a) !== JSON.stringify(c);
|
||||||
|
return expect(same && different,
|
||||||
|
`same seed identical: ${same} · different seed differs: ${different} · ` +
|
||||||
|
`signature ${a.signature.join('+')}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
check(9, 'every trait has enough scenes to build a track from', () => {
|
||||||
|
// The casting rule only works if the library can staff it. A trait declared
|
||||||
|
// by three scenes cannot carry a track — the rosters would collapse and every
|
||||||
|
// section would show the same two images, which is Phase 8 undone.
|
||||||
|
const counts = TRAITS.map((t) => [t, scenes.filter((m) => m.role !== 'accent'
|
||||||
|
&& sceneHonours(m, [t])).length]);
|
||||||
|
const thin = counts.filter(([, n]) => n < MIN_ELIGIBLE_SCENES);
|
||||||
|
return expect(thin.length === 0,
|
||||||
|
counts.map(([t, n]) => `${t}:${n}`).join(' ') +
|
||||||
|
(thin.length ? ` — too thin: ${thin.map(([t]) => t).join(', ')}` : ''));
|
||||||
|
});
|
||||||
|
|
||||||
|
check(9, 'no scene is cast in a track it cannot express', () => {
|
||||||
|
// The whole point. A scene that ignores the trait a track is built on is the
|
||||||
|
// shot that was obviously filmed somewhere else.
|
||||||
|
const problems = [];
|
||||||
|
for (const look of looks()) {
|
||||||
|
const signature = look.personality.signature;
|
||||||
|
for (const module of castOf(look)) {
|
||||||
|
if (!sceneHonours(module, signature)) {
|
||||||
|
problems.push(`${module.name} cast in a ${signature.join('+')} track`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
[...new Set(problems)].slice(0, 4).join(' · ') || 'every scene honours its track');
|
||||||
|
});
|
||||||
|
|
||||||
|
check(9, 'the signature still leaves a track enough scenes to cut between', () => {
|
||||||
|
// The failure mode of a hard filter: a personality so specific that the
|
||||||
|
// whole video is two scenes. The fallback in pickSignature exists for this,
|
||||||
|
// and this is what proves it fires.
|
||||||
|
const problems = [];
|
||||||
|
for (const look of looks()) {
|
||||||
|
const distinct = new Set(castOf(look).map((m) => m.name));
|
||||||
|
if (distinct.size < 3) {
|
||||||
|
problems.push(`${look.personality.signature.join('+')}: only ${distinct.size} scenes`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return expect(problems.length === 0, problems.slice(0, 3).join(' · ') || 'all casts 3+ scenes');
|
||||||
|
});
|
||||||
|
|
||||||
|
check(9, 'every declared trait visibly changes the scene that declares it', () => {
|
||||||
|
// The check that would have caught the whole thing being decorative. The
|
||||||
|
// lint proves a scene MENTIONS the trait; only rendering proves it matters.
|
||||||
|
// One LSB is the tolerance the determinism checks already treat as noise, so
|
||||||
|
// anything at or under it counts as ignored.
|
||||||
|
const engine = makeEngine();
|
||||||
|
const problems = [];
|
||||||
|
const measured = [];
|
||||||
|
try {
|
||||||
|
for (const trait of TRAITS) {
|
||||||
|
const [a, b] = pairDifferingIn(trait);
|
||||||
|
for (const module of scenes) {
|
||||||
|
if (!(module.traits || []).includes(trait)) continue;
|
||||||
|
if (module.kind !== 'fragment') continue; // 3D layers move the camera, not the frame
|
||||||
|
const delta = deltaUnder(module, a, b, engine);
|
||||||
|
measured.push(delta);
|
||||||
|
if (delta <= 1) problems.push(`${module.name}/${trait}: delta ${delta}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
const worst = Math.min(...measured);
|
||||||
|
return expect(problems.length === 0,
|
||||||
|
problems.slice(0, 4).join(' · ') ||
|
||||||
|
`${measured.length} scene/trait pairs, weakest response delta ${worst}`);
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(9, 'a personality changes the whole cast, not one scene', () => {
|
||||||
|
// Coherence, measured the only way it can be: if the track's design reaches
|
||||||
|
// every scene it cast, then changing the design changes every one of them.
|
||||||
|
const engine = makeEngine();
|
||||||
|
try {
|
||||||
|
const track = track9();
|
||||||
|
const look = generateLook(track, { seed: 4242 });
|
||||||
|
const [a, b] = pairDifferingIn(look.personality.signature[0] || 'camera');
|
||||||
|
|
||||||
|
const cast = [...new Map(castOf(look).map((m) => [m.name, m])).values()]
|
||||||
|
.filter((m) => m.kind === 'fragment');
|
||||||
|
const deltas = cast.map((m) => [m.name, deltaUnder(m, a, b, engine)]);
|
||||||
|
const unmoved = deltas.filter(([, d]) => d <= 1);
|
||||||
|
|
||||||
|
return expect(unmoved.length === 0,
|
||||||
|
unmoved.length
|
||||||
|
? `unmoved: ${unmoved.map(([n]) => n).join(', ')}`
|
||||||
|
: `${deltas.length} cast scenes all respond · ` +
|
||||||
|
deltas.map(([n, d]) => `${n} ${d}`).join(', '));
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
}, { slow: true });
|
||||||
|
|
||||||
|
check(9, 'a layer with no personality renders what it always rendered', () => {
|
||||||
|
// The neutral-default promise. Every range sweep, flash sweep and library
|
||||||
|
// regression builds layers directly with no personality attached, and they
|
||||||
|
// all compare against numbers recorded before this phase existed.
|
||||||
|
const engine = makeEngine();
|
||||||
|
try {
|
||||||
|
const module = scenes.find((m) => m.kind === 'fragment' && (m.traits || []).length);
|
||||||
|
const neutral = {
|
||||||
|
shape: { sides: 0, roundness: 0.25, elongation: 1, tilt: 0 },
|
||||||
|
camera: { driftAngle: 0, driftRate: 0, sway: 0, swayRate: 0.1, spin: 0, breathe: 0 },
|
||||||
|
space: { horizon: 0.5, depth: 0, washAngle: 0, wash: 0 },
|
||||||
|
style: { lineWeight: 0.5, softness: 0.5, texture: 0, symmetry: 1 },
|
||||||
|
signature: [],
|
||||||
|
};
|
||||||
|
const delta = deltaUnder(module, null, neutral, engine);
|
||||||
|
return expect(delta <= 1,
|
||||||
|
`${module.name}: no-personality vs explicit-neutral delta ${delta}`);
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
}, { slow: true });
|
||||||
160
flow-state/src/checks/scene-gate.js
Normal file
160
flow-state/src/checks/scene-gate.js
Normal file
@ -0,0 +1,160 @@
|
|||||||
|
// The per-scene acceptance battery, runnable for ONE scene.
|
||||||
|
//
|
||||||
|
// The library-wide gates iterate the registry, so a new scene is covered the
|
||||||
|
// moment it is registered — but running them means rendering all thirty-six
|
||||||
|
// scenes and reading a page of results to find out whether the one you just
|
||||||
|
// wrote is alive. That is slow to run and expensive to read, and it is the loop
|
||||||
|
// you are in constantly while writing a scene.
|
||||||
|
//
|
||||||
|
// This runs the same acceptance criteria against a single scene and prints one
|
||||||
|
// line per criterion plus a single verdict. Open:
|
||||||
|
//
|
||||||
|
// checks.html?scene=Aurora%20Veil
|
||||||
|
//
|
||||||
|
// The criteria are deliberately the same ones Phase 2, 5 and 7 apply — this is
|
||||||
|
// a filter over the existing gates, not a second, weaker set of them.
|
||||||
|
|
||||||
|
import { Engine } from '../engine/Engine.js';
|
||||||
|
import { sceneByName, scenes } from '../scenes/registry.js';
|
||||||
|
import { defaultValues, sampleValues, sweepValues, validateModule } from '../params/schema.js';
|
||||||
|
import { Rng } from '../engine/rng.js';
|
||||||
|
import { FeatureTrack, featureProviderFor } from '../audio/FeatureTrack.js';
|
||||||
|
import { synthesizeSectioned } from '../audio/synth.js';
|
||||||
|
import { frameLuminance, frameVariance, frameMaxDelta } from '../engine/hash.js';
|
||||||
|
import { peakFlashRate } from '../engine/flash.js';
|
||||||
|
import { generatePersonality } from '../look/Personality.js';
|
||||||
|
|
||||||
|
const PALETTE = [
|
||||||
|
[0.06, 0.03, 0.16], [0.85, 0.15, 0.55], [0.15, 0.75, 0.95],
|
||||||
|
[0.98, 0.85, 0.35], [0.55, 0.25, 0.85], [0.2, 0.95, 0.6],
|
||||||
|
];
|
||||||
|
|
||||||
|
const SUMMARY = { meanCentroid: 0.5, meanFlatness: 0.25, dynamicRange: 0.5, bpm: 126, meanLoudness: 0.4 };
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @param {string} name scene name as registered
|
||||||
|
* @returns {{ok: boolean, lines: string[]}}
|
||||||
|
*/
|
||||||
|
export function runSceneGate(name) {
|
||||||
|
const module = sceneByName(name);
|
||||||
|
const lines = [];
|
||||||
|
if (!module) {
|
||||||
|
return {
|
||||||
|
ok: false,
|
||||||
|
lines: [`FAIL no scene named "${name}" — registered: ${scenes.map((m) => m.name).join(', ')}`],
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
let ok = true;
|
||||||
|
const record = (pass, label, detail) => {
|
||||||
|
ok = ok && pass;
|
||||||
|
lines.push(`${pass ? 'PASS' : 'FAIL'} ${label.padEnd(26)} ${detail}`);
|
||||||
|
};
|
||||||
|
|
||||||
|
const errors = validateModule(module);
|
||||||
|
record(errors.length === 0, 'schema', errors.length ? errors.join(' · ') : 'valid');
|
||||||
|
|
||||||
|
const track = FeatureTrack.fromAudioBuffer(
|
||||||
|
synthesizeSectioned({ bpm: 128, duration: 60, changeAt: 30 }), { fps: 60 });
|
||||||
|
|
||||||
|
const engine = new Engine({ width: 192, height: 108 });
|
||||||
|
engine.timeline.setDuration(track.duration);
|
||||||
|
engine.setFeatureProvider(featureProviderFor(track));
|
||||||
|
|
||||||
|
const personality = generatePersonality(SUMMARY, new Rng(9001));
|
||||||
|
const draw = (params, frame, seed = 4242) => {
|
||||||
|
engine.setLayerSpecs([{
|
||||||
|
module, params, seed, opacity: 1, blend: 'normal', palette: PALETTE, personality,
|
||||||
|
}]);
|
||||||
|
engine.compositor.reset();
|
||||||
|
return Uint8Array.from(engine.readPixels(engine.renderFrame(frame)));
|
||||||
|
};
|
||||||
|
|
||||||
|
try {
|
||||||
|
// --- alive -------------------------------------------------------
|
||||||
|
const base = draw(defaultValues(module), 600);
|
||||||
|
const lum = frameLuminance(base);
|
||||||
|
const variance = frameVariance(base);
|
||||||
|
record(lum > 0.004 && variance > 0.0008, 'renders something',
|
||||||
|
`luminance ${lum.toFixed(4)} · variance ${variance.toFixed(4)}`);
|
||||||
|
|
||||||
|
// --- animates ----------------------------------------------------
|
||||||
|
const later = draw(defaultValues(module), 600 + 120);
|
||||||
|
const motion = frameMaxDelta(base, later);
|
||||||
|
record(motion > 3, 'animates', `max channel delta ${motion} over 2s`);
|
||||||
|
|
||||||
|
// --- deterministic -----------------------------------------------
|
||||||
|
const again = draw(defaultValues(module), 600);
|
||||||
|
const repeat = frameMaxDelta(base, again);
|
||||||
|
record(repeat <= 1, 'deterministic', `repeat delta ${repeat}/255`);
|
||||||
|
|
||||||
|
// --- distinct from every other scene -------------------------------
|
||||||
|
let closest = 255;
|
||||||
|
let closestName = '';
|
||||||
|
for (const other of scenes) {
|
||||||
|
if (other === module || other.kind !== 'fragment') continue;
|
||||||
|
engine.setLayerSpecs([{
|
||||||
|
module: other, params: defaultValues(other), seed: 4242,
|
||||||
|
opacity: 1, blend: 'normal', palette: PALETTE, personality,
|
||||||
|
}]);
|
||||||
|
engine.compositor.reset();
|
||||||
|
const d = frameMaxDelta(base, Uint8Array.from(engine.readPixels(engine.renderFrame(600))));
|
||||||
|
if (d < closest) { closest = d; closestName = other.name; }
|
||||||
|
}
|
||||||
|
record(closest >= 24, 'distinct', `closest ${closestName} at ${closest} (floor 24)`);
|
||||||
|
|
||||||
|
// --- param sweep ---------------------------------------------------
|
||||||
|
const dead = [];
|
||||||
|
for (const [pname, def] of Object.entries(module.params || {})) {
|
||||||
|
if (def.type === 'palette') continue;
|
||||||
|
for (const value of sweepValues(def, 4)) {
|
||||||
|
const params = { ...defaultValues(module), [pname]: value };
|
||||||
|
const pixels = draw(params, 700);
|
||||||
|
const l = frameLuminance(pixels);
|
||||||
|
const v = frameVariance(pixels);
|
||||||
|
if (!(l > 0.002) || !(v > 0.0002) || l > 0.97) {
|
||||||
|
dead.push(`${pname}=${Array.isArray(value) ? value.join(',') : value}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
record(dead.length === 0, 'param sweep',
|
||||||
|
dead.length ? `dead/blown at ${dead.slice(0, 4).join(', ')}` : 'all values live');
|
||||||
|
|
||||||
|
// --- flash rate ------------------------------------------------------
|
||||||
|
const hot = sampleValues(module, new Rng(77), { energy: 0.95, density: 0.9, motion: 0.9 });
|
||||||
|
engine.setLayerSpecs([{
|
||||||
|
module, params: hot, seed: 99, opacity: 1, blend: 'normal', palette: PALETTE, personality,
|
||||||
|
}]);
|
||||||
|
engine.compositor.reset();
|
||||||
|
const luminance = [];
|
||||||
|
for (let f = 600; f < 900; f++) {
|
||||||
|
luminance.push(frameLuminance(engine.readPixels(engine.renderFrame(f))));
|
||||||
|
}
|
||||||
|
const rate = peakFlashRate(luminance, 60);
|
||||||
|
record(rate <= 3, 'flash rate', `${rate}/s at aggressive settings (ceiling 3)`);
|
||||||
|
|
||||||
|
// --- personality response --------------------------------------------
|
||||||
|
// Every declared trait must move the image; a trait declared and ignored
|
||||||
|
// gets the scene cast in tracks it cannot express.
|
||||||
|
for (const trait of module.traits || []) {
|
||||||
|
const other = generatePersonality(SUMMARY, new Rng(9001));
|
||||||
|
if (trait === 'shape') other.shape = { sides: 6, roundness: 0.05, elongation: 1.3, tilt: 0.7 };
|
||||||
|
if (trait === 'camera') other.camera = { ...other.camera, driftAngle: 1.1, driftRate: 0.06, sway: 0.06, swayRate: 0.2, spin: 0.05, breathe: 0.05 };
|
||||||
|
if (trait === 'space') other.space = { horizon: 0.68, depth: 0.9, washAngle: 2.4, wash: 0.5 };
|
||||||
|
if (trait === 'style') other.style = { lineWeight: 0.95, softness: 0.9, texture: 0.5, symmetry: 4 };
|
||||||
|
|
||||||
|
engine.setLayerSpecs([{
|
||||||
|
module, params: defaultValues(module), seed: 4242,
|
||||||
|
opacity: 1, blend: 'normal', palette: PALETTE, personality: other,
|
||||||
|
}]);
|
||||||
|
engine.compositor.reset();
|
||||||
|
const changed = frameMaxDelta(base,
|
||||||
|
Uint8Array.from(engine.readPixels(engine.renderFrame(600))));
|
||||||
|
record(changed > 1, `trait: ${trait}`, `delta ${changed}/255`);
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
engine.dispose();
|
||||||
|
}
|
||||||
|
|
||||||
|
return { ok, lines };
|
||||||
|
}
|
||||||
358
flow-state/src/engine/Compositor.js
Normal file
358
flow-state/src/engine/Compositor.js
Normal file
@ -0,0 +1,358 @@
|
|||||||
|
import * as THREE from 'three';
|
||||||
|
import { makePassMaterial } from './Renderer.js';
|
||||||
|
import {
|
||||||
|
BLEND_FRAG, FEEDBACK_FRAG, BRIGHT_FRAG, BLUR_FRAG, COMPOSITE_FRAG, COPY_FRAG,
|
||||||
|
BLEND_MODE_IDS,
|
||||||
|
} from './passes.js';
|
||||||
|
|
||||||
|
const DEFAULT_POST = {
|
||||||
|
bloom: 0.35,
|
||||||
|
bloomThreshold: 0.6,
|
||||||
|
bloomKnee: 0.3,
|
||||||
|
chroma: 0.15,
|
||||||
|
// Grain defaults to absent. What it looks like when a look does ask for it
|
||||||
|
// is described by the four fields below — see look/grain.js.
|
||||||
|
grain: 0,
|
||||||
|
grainScale: 1,
|
||||||
|
grainRate: 1,
|
||||||
|
grainMask: 0,
|
||||||
|
grainChroma: 0,
|
||||||
|
vignette: 0.35,
|
||||||
|
contrast: 1.05,
|
||||||
|
saturation: 1.1,
|
||||||
|
lift: 0.0,
|
||||||
|
exposure: 1.0,
|
||||||
|
};
|
||||||
|
|
||||||
|
const DEFAULT_FEEDBACK = {
|
||||||
|
amount: 0.0,
|
||||||
|
decay: 0.9,
|
||||||
|
zoom: 0.995,
|
||||||
|
rotate: 0.0,
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The layer stack. Layers render into their own target, then blend into an
|
||||||
|
* accumulator; the result goes through feedback and the post chain.
|
||||||
|
*
|
||||||
|
* Every target is explicitly cleared on allocation and on reset, because
|
||||||
|
* inheriting stale GPU memory is exactly the kind of thing that makes an export
|
||||||
|
* differ from a preview.
|
||||||
|
*/
|
||||||
|
export class Compositor {
|
||||||
|
constructor(renderer, { width, height } = {}) {
|
||||||
|
this.renderer = renderer;
|
||||||
|
this.width = width || renderer.width;
|
||||||
|
this.height = height || renderer.height;
|
||||||
|
|
||||||
|
this.layers = [];
|
||||||
|
this.post = { ...DEFAULT_POST };
|
||||||
|
this.feedback = { ...DEFAULT_FEEDBACK };
|
||||||
|
this.fade = 1;
|
||||||
|
this.soloIndex = -1; // debug: render one layer alone
|
||||||
|
this.postEnabled = true;
|
||||||
|
this._primed = new WeakSet();
|
||||||
|
|
||||||
|
this._buildTargets();
|
||||||
|
this._buildMaterials();
|
||||||
|
}
|
||||||
|
|
||||||
|
_buildTargets() {
|
||||||
|
const r = this.renderer;
|
||||||
|
const w = this.width, h = this.height;
|
||||||
|
const bw = Math.max(1, Math.floor(w / 2));
|
||||||
|
const bh = Math.max(1, Math.floor(h / 2));
|
||||||
|
|
||||||
|
this.layerTarget = r.createTarget(w, h, { depth: true });
|
||||||
|
this.accumA = r.createTarget(w, h);
|
||||||
|
this.accumB = r.createTarget(w, h);
|
||||||
|
this.historyA = r.createTarget(w, h, { float: true });
|
||||||
|
this.historyB = r.createTarget(w, h, { float: true });
|
||||||
|
this.bloomA = r.createTarget(bw, bh);
|
||||||
|
this.bloomB = r.createTarget(bw, bh);
|
||||||
|
this.outputTarget = r.createTarget(w, h);
|
||||||
|
}
|
||||||
|
|
||||||
|
_buildMaterials() {
|
||||||
|
this.blendMaterial = makePassMaterial(BLEND_FRAG, {
|
||||||
|
u_base: { value: null },
|
||||||
|
u_src: { value: null },
|
||||||
|
u_amount: { value: 1 },
|
||||||
|
u_mode: { value: 0 },
|
||||||
|
});
|
||||||
|
this.feedbackMaterial = makePassMaterial(FEEDBACK_FRAG, {
|
||||||
|
u_current: { value: null },
|
||||||
|
u_history: { value: null },
|
||||||
|
u_decay: { value: 0.9 },
|
||||||
|
u_amount: { value: 0 },
|
||||||
|
u_zoom: { value: 0.995 },
|
||||||
|
u_rotate: { value: 0 },
|
||||||
|
u_aspect: { value: 1 },
|
||||||
|
});
|
||||||
|
this.brightMaterial = makePassMaterial(BRIGHT_FRAG, {
|
||||||
|
u_tex: { value: null },
|
||||||
|
u_threshold: { value: 0.6 },
|
||||||
|
u_knee: { value: 0.3 },
|
||||||
|
});
|
||||||
|
this.blurMaterial = makePassMaterial(BLUR_FRAG, {
|
||||||
|
u_tex: { value: null },
|
||||||
|
u_direction: { value: new THREE.Vector2(0, 0) },
|
||||||
|
});
|
||||||
|
this.compositeMaterial = makePassMaterial(COMPOSITE_FRAG, {
|
||||||
|
u_tex: { value: null },
|
||||||
|
u_bloom: { value: null },
|
||||||
|
u_bloomAmount: { value: 0 },
|
||||||
|
u_chroma: { value: 0 },
|
||||||
|
u_grain: { value: 0 },
|
||||||
|
u_grainScale: { value: 1 },
|
||||||
|
u_grainRate: { value: 1 },
|
||||||
|
u_grainMask: { value: 0 },
|
||||||
|
u_grainChroma: { value: 0 },
|
||||||
|
u_vignette: { value: 0 },
|
||||||
|
u_contrast: { value: 1 },
|
||||||
|
u_saturation: { value: 1 },
|
||||||
|
u_lift: { value: 0 },
|
||||||
|
u_exposure: { value: 1 },
|
||||||
|
u_fade: { value: 1 },
|
||||||
|
u_frame: { value: 0 },
|
||||||
|
u_resolution: { value: new THREE.Vector2(1, 1) },
|
||||||
|
});
|
||||||
|
this.copyMaterial = makePassMaterial(COPY_FRAG, { u_tex: { value: null } });
|
||||||
|
}
|
||||||
|
|
||||||
|
setSize(width, height) {
|
||||||
|
if (width === this.width && height === this.height) return;
|
||||||
|
this.width = width;
|
||||||
|
this.height = height;
|
||||||
|
this.disposeTargets();
|
||||||
|
this._buildTargets();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The compositor does NOT own its layers and never disposes them — the arc
|
||||||
|
* driver caches Layer instances across sections and swaps them in and out
|
||||||
|
* every crossfade, and disposing on removal would destroy shaders that are
|
||||||
|
* about to be reused (and recompile them on the way back).
|
||||||
|
*/
|
||||||
|
setLayers(layers) {
|
||||||
|
for (const layer of layers) {
|
||||||
|
if (!this._primed.has(layer)) {
|
||||||
|
this._primeLayer(layer);
|
||||||
|
this._primed.add(layer);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
this.layers = layers;
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Compile layers that are not on screen yet, so the frame they first appear
|
||||||
|
* on does not pay for the link. Used by the arc driver's prewarm pass.
|
||||||
|
*/
|
||||||
|
primeLayers(layers) {
|
||||||
|
for (const layer of layers) {
|
||||||
|
if (this._primed.has(layer)) continue;
|
||||||
|
this._primeLayer(layer);
|
||||||
|
this._primed.add(layer);
|
||||||
|
}
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Force a layer's shader program to finish linking before it is used for real.
|
||||||
|
*
|
||||||
|
* three.js links programs through KHR_parallel_shader_compile, so the first
|
||||||
|
* draws after a material is created can run against a program that is not
|
||||||
|
* ready and produce wrong output. Measured on the heaviest scene in the
|
||||||
|
* library, the first TEN frames rendered differently from every later render
|
||||||
|
* of the same frames. Preview hides this — the frames go by and the next pass
|
||||||
|
* is correct — but an export renders each frame exactly once, so those frames
|
||||||
|
* would ship broken.
|
||||||
|
*
|
||||||
|
* Rendering a throwaway frame and reading it back is NOT sufficient: measured,
|
||||||
|
* it left 3-5 frames still wrong. WebGLRenderer.compile() is the API that
|
||||||
|
* actually waits for the link, and it clears the problem completely.
|
||||||
|
*/
|
||||||
|
_primeLayer(layer) {
|
||||||
|
try {
|
||||||
|
if (layer.material) this.renderer.compileMaterial(layer.material);
|
||||||
|
else if (layer.scene && layer.camera) this.renderer.compileScene(layer.scene, layer.camera);
|
||||||
|
} catch (err) {
|
||||||
|
console.warn('[compositor] priming failed for', layer.module && layer.module.name, err);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Bring the whole chain to a state where the next rendered frame is correct.
|
||||||
|
*
|
||||||
|
* Shader programs link asynchronously (KHR_parallel_shader_compile), and until
|
||||||
|
* they are ready a draw produces wrong output. compile() covers the programs;
|
||||||
|
* the discarded frame covers everything else that is lazily created on first
|
||||||
|
* use. Cheap, and it converts "the first few frames may be wrong" into "the
|
||||||
|
* first few frames were thrown away".
|
||||||
|
*
|
||||||
|
* The exporter calls this before encoding anything, because an export renders
|
||||||
|
* each frame exactly once and has no second chance to get frame 0 right.
|
||||||
|
*/
|
||||||
|
prime(ctx = null) {
|
||||||
|
const materials = [
|
||||||
|
this.blendMaterial, this.feedbackMaterial, this.brightMaterial,
|
||||||
|
this.blurMaterial, this.compositeMaterial, this.copyMaterial,
|
||||||
|
];
|
||||||
|
for (const material of materials) {
|
||||||
|
try { this.renderer.compileMaterial(material); } catch { /* non-fatal */ }
|
||||||
|
}
|
||||||
|
for (const layer of this.layers) this._primeLayer(layer);
|
||||||
|
|
||||||
|
if (ctx) {
|
||||||
|
try { this.render(ctx); } catch { /* non-fatal */ }
|
||||||
|
}
|
||||||
|
this.reset();
|
||||||
|
}
|
||||||
|
|
||||||
|
setPost(post) {
|
||||||
|
this.post = { ...this.post, ...post };
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
setFeedback(feedback) {
|
||||||
|
this.feedback = { ...this.feedback, ...feedback };
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Wipe all history. Called on seek and before an export run so a render never
|
||||||
|
* depends on what was on screen beforehand.
|
||||||
|
*/
|
||||||
|
reset() {
|
||||||
|
const r = this.renderer;
|
||||||
|
[this.accumA, this.accumB, this.historyA, this.historyB,
|
||||||
|
this.bloomA, this.bloomB, this.layerTarget, this.outputTarget]
|
||||||
|
.forEach((t) => r.clear(t));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Render one frame. Returns the target holding the finished image, so the
|
||||||
|
* caller decides whether it goes to the canvas or to the encoder.
|
||||||
|
*/
|
||||||
|
render(ctx) {
|
||||||
|
const r = this.renderer;
|
||||||
|
const { timeline, features } = ctx;
|
||||||
|
|
||||||
|
r.clear(this.accumA);
|
||||||
|
let accum = this.accumA;
|
||||||
|
let spare = this.accumB;
|
||||||
|
|
||||||
|
const active = this.soloIndex >= 0
|
||||||
|
? this.layers.slice(this.soloIndex, this.soloIndex + 1)
|
||||||
|
: this.layers;
|
||||||
|
|
||||||
|
for (const layer of active) {
|
||||||
|
if (layer.opacity <= 0.001) continue;
|
||||||
|
|
||||||
|
r.clear(this.layerTarget);
|
||||||
|
layer.render(r, this.layerTarget, {
|
||||||
|
timeline,
|
||||||
|
features,
|
||||||
|
prevTexture: this.historyA.texture,
|
||||||
|
});
|
||||||
|
|
||||||
|
const bu = this.blendMaterial.uniforms;
|
||||||
|
bu.u_base.value = accum.texture;
|
||||||
|
bu.u_src.value = this.layerTarget.texture;
|
||||||
|
bu.u_amount.value = 1.0; // layer opacity already applied in-shader
|
||||||
|
bu.u_mode.value = BLEND_MODE_IDS[layer.blend] ?? 0;
|
||||||
|
r.blit(this.blendMaterial, spare);
|
||||||
|
|
||||||
|
const t = accum; accum = spare; spare = t;
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- feedback -------------------------------------------------------
|
||||||
|
let composited = accum;
|
||||||
|
if (this.feedback.amount > 0.001) {
|
||||||
|
const fu = this.feedbackMaterial.uniforms;
|
||||||
|
fu.u_current.value = accum.texture;
|
||||||
|
fu.u_history.value = this.historyA.texture;
|
||||||
|
fu.u_decay.value = Math.min(0.99, this.feedback.decay);
|
||||||
|
fu.u_amount.value = this.feedback.amount;
|
||||||
|
fu.u_zoom.value = this.feedback.zoom;
|
||||||
|
fu.u_rotate.value = this.feedback.rotate;
|
||||||
|
fu.u_aspect.value = this.width / this.height;
|
||||||
|
r.blit(this.feedbackMaterial, this.historyB);
|
||||||
|
composited = this.historyB;
|
||||||
|
const t = this.historyA; this.historyA = this.historyB; this.historyB = t;
|
||||||
|
} else {
|
||||||
|
// Keep history tracking the image even when feedback is off, so
|
||||||
|
// enabling it mid-track doesn't pop from black.
|
||||||
|
this.copyMaterial.uniforms.u_tex.value = accum.texture;
|
||||||
|
r.blit(this.copyMaterial, this.historyA);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!this.postEnabled) {
|
||||||
|
this.copyMaterial.uniforms.u_tex.value = composited.texture;
|
||||||
|
r.blit(this.copyMaterial, this.outputTarget);
|
||||||
|
return this.outputTarget;
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- bloom ----------------------------------------------------------
|
||||||
|
const p = this.post;
|
||||||
|
if (p.bloom > 0.001) {
|
||||||
|
this.brightMaterial.uniforms.u_tex.value = composited.texture;
|
||||||
|
this.brightMaterial.uniforms.u_threshold.value = p.bloomThreshold;
|
||||||
|
this.brightMaterial.uniforms.u_knee.value = p.bloomKnee;
|
||||||
|
r.blit(this.brightMaterial, this.bloomA);
|
||||||
|
|
||||||
|
const bw = this.bloomA.width, bh = this.bloomA.height;
|
||||||
|
for (let i = 0; i < 2; i++) {
|
||||||
|
this.blurMaterial.uniforms.u_tex.value = this.bloomA.texture;
|
||||||
|
this.blurMaterial.uniforms.u_direction.value.set((1 + i) / bw, 0);
|
||||||
|
r.blit(this.blurMaterial, this.bloomB);
|
||||||
|
|
||||||
|
this.blurMaterial.uniforms.u_tex.value = this.bloomB.texture;
|
||||||
|
this.blurMaterial.uniforms.u_direction.value.set(0, (1 + i) / bh);
|
||||||
|
r.blit(this.blurMaterial, this.bloomA);
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
r.clear(this.bloomA);
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- final grade ----------------------------------------------------
|
||||||
|
const cu = this.compositeMaterial.uniforms;
|
||||||
|
cu.u_tex.value = composited.texture;
|
||||||
|
cu.u_bloom.value = this.bloomA.texture;
|
||||||
|
cu.u_bloomAmount.value = p.bloom;
|
||||||
|
cu.u_chroma.value = p.chroma;
|
||||||
|
cu.u_grain.value = p.grain;
|
||||||
|
cu.u_grainScale.value = p.grainScale;
|
||||||
|
cu.u_grainRate.value = p.grainRate;
|
||||||
|
cu.u_grainMask.value = p.grainMask;
|
||||||
|
cu.u_grainChroma.value = p.grainChroma;
|
||||||
|
cu.u_vignette.value = p.vignette;
|
||||||
|
cu.u_contrast.value = p.contrast;
|
||||||
|
cu.u_saturation.value = p.saturation;
|
||||||
|
cu.u_lift.value = p.lift;
|
||||||
|
cu.u_exposure.value = p.exposure;
|
||||||
|
cu.u_fade.value = this.fade;
|
||||||
|
cu.u_frame.value = timeline.frame;
|
||||||
|
cu.u_resolution.value.set(this.width, this.height);
|
||||||
|
r.blit(this.compositeMaterial, this.outputTarget);
|
||||||
|
|
||||||
|
return this.outputTarget;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Present a finished target to the canvas. */
|
||||||
|
present(target) {
|
||||||
|
this.copyMaterial.uniforms.u_tex.value = target.texture;
|
||||||
|
this.renderer.blit(this.copyMaterial, null);
|
||||||
|
}
|
||||||
|
|
||||||
|
disposeTargets() {
|
||||||
|
[this.layerTarget, this.accumA, this.accumB, this.historyA, this.historyB,
|
||||||
|
this.bloomA, this.bloomB, this.outputTarget].forEach((t) => t && t.dispose());
|
||||||
|
}
|
||||||
|
|
||||||
|
dispose() {
|
||||||
|
this.layers = []; // owned elsewhere; see setLayers
|
||||||
|
this.disposeTargets();
|
||||||
|
}
|
||||||
|
}
|
||||||
148
flow-state/src/engine/Engine.js
Normal file
148
flow-state/src/engine/Engine.js
Normal file
@ -0,0 +1,148 @@
|
|||||||
|
import { Renderer } from './Renderer.js';
|
||||||
|
import { Compositor } from './Compositor.js';
|
||||||
|
import { Timeline, FIXED, REALTIME } from './Timeline.js';
|
||||||
|
import { createLayer } from './Layer.js';
|
||||||
|
import { hashFrame } from './hash.js';
|
||||||
|
|
||||||
|
/** Zeroed features, so the engine runs before any audio is loaded. */
|
||||||
|
export const NULL_FEATURES = Object.freeze({
|
||||||
|
loudness: 0, rms: 0,
|
||||||
|
bandSub: 0, bandLow: 0, bandMid: 0, bandHigh: 0, bandAir: 0,
|
||||||
|
flux: 0, centroid: 0.5, flatness: 0, width: 0.5,
|
||||||
|
beat: 0, beatPhase: 0, barPhase: 0, phrasePhase: 0,
|
||||||
|
sectionProgress: 0, sectionEnergy: 0, buildSlope: 0,
|
||||||
|
});
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Ties the clock, the renderer and the layer stack together. Deliberately the
|
||||||
|
* only place that knows about all three, and deliberately unaware of the DOM
|
||||||
|
* beyond its canvas — the exporter and the check harness drive the exact same
|
||||||
|
* object the preview does, which is what keeps them from diverging.
|
||||||
|
*/
|
||||||
|
export class Engine {
|
||||||
|
constructor({ width = 1280, height = 720, canvas = null, fps = 60 } = {}) {
|
||||||
|
this.renderer = new Renderer({ width, height, canvas });
|
||||||
|
this.compositor = new Compositor(this.renderer, { width, height });
|
||||||
|
this.timeline = new Timeline({ fps, mode: REALTIME });
|
||||||
|
this.featureProvider = null;
|
||||||
|
this.ownedLayers = [];
|
||||||
|
this.lastFrameRendered = -1;
|
||||||
|
}
|
||||||
|
|
||||||
|
get width() { return this.renderer.width; }
|
||||||
|
get height() { return this.renderer.height; }
|
||||||
|
|
||||||
|
setSize(width, height) {
|
||||||
|
this.renderer.setSize(width, height);
|
||||||
|
this.compositor.setSize(width, height);
|
||||||
|
}
|
||||||
|
|
||||||
|
setFeatureProvider(provider) {
|
||||||
|
this.featureProvider = provider;
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
featuresAt(frame) {
|
||||||
|
if (!this.featureProvider) return NULL_FEATURES;
|
||||||
|
return this.featureProvider.at(frame) || NULL_FEATURES;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Replace the stack from specs. `specs` are { module, params, seed, opacity,
|
||||||
|
* blend }. Layers built this way are owned by the Engine and disposed with
|
||||||
|
* it; layers supplied directly by the arc driver are owned by the driver.
|
||||||
|
*/
|
||||||
|
setLayerSpecs(specs) {
|
||||||
|
this.ownedLayers.forEach((l) => l.dispose());
|
||||||
|
const layers = specs.map((s) => {
|
||||||
|
const layer = createLayer(s.module, s);
|
||||||
|
if (s.palette) layer.setPalette(s.palette);
|
||||||
|
if (s.personality) layer.setPersonality(s.personality);
|
||||||
|
return layer;
|
||||||
|
});
|
||||||
|
this.ownedLayers = layers;
|
||||||
|
this.compositor.setLayers(layers);
|
||||||
|
return layers;
|
||||||
|
}
|
||||||
|
|
||||||
|
setPalette(colors) {
|
||||||
|
this.compositor.layers.forEach((l) => l.setPalette(colors));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Compile every shader and discard a warm frame, so the next frame rendered
|
||||||
|
* is correct. Required before any frame-exact use (export, hashing).
|
||||||
|
*/
|
||||||
|
prime(frame = 0) {
|
||||||
|
this.timeline.seek(frame);
|
||||||
|
this.compositor.prime({ timeline: this.timeline, features: this.featuresAt(frame) });
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Render exactly one frame at the timeline's current position. */
|
||||||
|
renderCurrent() {
|
||||||
|
const features = this.featuresAt(this.timeline.frame);
|
||||||
|
const target = this.compositor.render({ timeline: this.timeline, features });
|
||||||
|
this.lastFrameRendered = this.timeline.frame;
|
||||||
|
return target;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Render a specific frame without warm-up. Used by checks and by export. */
|
||||||
|
renderFrame(frameIndex) {
|
||||||
|
this.timeline.seek(frameIndex);
|
||||||
|
return this.renderCurrent();
|
||||||
|
}
|
||||||
|
|
||||||
|
present(target) {
|
||||||
|
this.compositor.present(target);
|
||||||
|
}
|
||||||
|
|
||||||
|
readPixels(target) {
|
||||||
|
return this.renderer.readPixels(target);
|
||||||
|
}
|
||||||
|
|
||||||
|
hashCurrent(target) {
|
||||||
|
return hashFrame(this.renderer.readPixels(target));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Render frames [start, start+count) sequentially from a clean state and
|
||||||
|
* return a hash per frame. Sequential and reset-first, so the result depends
|
||||||
|
* only on the inputs — this is the primitive every determinism check uses.
|
||||||
|
*/
|
||||||
|
hashRun(start, count, { reset = true, prime = true } = {}) {
|
||||||
|
if (prime) this.prime(start);
|
||||||
|
if (reset) this.compositor.reset();
|
||||||
|
const hashes = [];
|
||||||
|
for (let i = 0; i < count; i++) {
|
||||||
|
const target = this.renderFrame(start + i);
|
||||||
|
hashes.push(hashFrame(this.renderer.readPixels(target)));
|
||||||
|
}
|
||||||
|
return hashes;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Advance stateful layers up to `frame` without presenting, so a seek lands
|
||||||
|
* on converged feedback state. Section-boundary seeks pass warmup: 0,
|
||||||
|
* because layer state is re-seeded there and is exact by construction.
|
||||||
|
*/
|
||||||
|
warmUp(frame, warmupFrames = 120) {
|
||||||
|
const start = Math.max(0, frame - warmupFrames);
|
||||||
|
this.compositor.reset();
|
||||||
|
for (let f = start; f < frame; f++) {
|
||||||
|
this.timeline.seek(f);
|
||||||
|
const features = this.featuresAt(f);
|
||||||
|
this.compositor.render({ timeline: this.timeline, features });
|
||||||
|
}
|
||||||
|
this.timeline.seek(frame);
|
||||||
|
}
|
||||||
|
|
||||||
|
dispose() {
|
||||||
|
this.ownedLayers.forEach((l) => l.dispose());
|
||||||
|
this.ownedLayers = [];
|
||||||
|
this.compositor.dispose();
|
||||||
|
this.renderer.dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export { FIXED, REALTIME };
|
||||||
296
flow-state/src/engine/Layer.js
Normal file
296
flow-state/src/engine/Layer.js
Normal file
@ -0,0 +1,296 @@
|
|||||||
|
import * as THREE from 'three';
|
||||||
|
import { VERTEX_SHADER, buildFragmentShader, AUDIO_UNIFORMS, SIGNATURE_UNIFORMS } from './shader-contract.js';
|
||||||
|
import { signatureUniforms, NEUTRAL_UNIFORMS } from '../look/Personality.js';
|
||||||
|
import { clampValue } from '../params/schema.js';
|
||||||
|
|
||||||
|
export const BLEND_MODES = ['normal', 'add', 'screen', 'multiply', 'overlay', 'softlight'];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Map a raw feature value through a response curve. `spike` squares the input so
|
||||||
|
* beat-driven params punch rather than wobble; `smooth` rounds the shoulders so
|
||||||
|
* slow features don't step.
|
||||||
|
*/
|
||||||
|
function applyResponse(value, response) {
|
||||||
|
switch (response) {
|
||||||
|
case 'spike': return value * value;
|
||||||
|
case 'smooth': return value * value * (3 - 2 * value);
|
||||||
|
case 'inverse': return 1 - value;
|
||||||
|
default: return value;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The standard uniform dictionary every fragment layer is compiled against.
|
||||||
|
* Shared by ShaderLayer and the OSD plate so a text layer honours exactly the
|
||||||
|
* same contract — palette, audio features and personality — as a scene.
|
||||||
|
*/
|
||||||
|
export function buildShaderUniforms(module, baseParams, seed) {
|
||||||
|
const uniforms = {
|
||||||
|
u_resolution: { value: new THREE.Vector2(1, 1) },
|
||||||
|
u_aspect: { value: 1 },
|
||||||
|
u_pixelScale: { value: 1 },
|
||||||
|
u_time: { value: 0 },
|
||||||
|
u_frame: { value: 0 },
|
||||||
|
u_progress: { value: 0 },
|
||||||
|
u_seed: { value: (seed >>> 0) % 100000 / 1000 },
|
||||||
|
u_opacity: { value: 1 },
|
||||||
|
u_colors: { value: Array.from({ length: 8 }, () => new THREE.Vector3(1, 1, 1)) },
|
||||||
|
u_colorCount: { value: 1 },
|
||||||
|
u_prev: { value: null },
|
||||||
|
u_hasPrev: { value: 0 },
|
||||||
|
};
|
||||||
|
for (const name of AUDIO_UNIFORMS) uniforms[name] = { value: 0 };
|
||||||
|
for (const [name, type] of Object.entries(SIGNATURE_UNIFORMS)) {
|
||||||
|
const v = NEUTRAL_UNIFORMS[name];
|
||||||
|
uniforms[name] = { value: type === 'vec2' ? new THREE.Vector2(v[0], v[1]) : v };
|
||||||
|
}
|
||||||
|
|
||||||
|
for (const [name, def] of Object.entries(module.params || {})) {
|
||||||
|
if (!def.uniform || def.type === 'palette') continue;
|
||||||
|
const v = baseParams[name];
|
||||||
|
uniforms[def.uniform] = {
|
||||||
|
value: def.type === 'vec2'
|
||||||
|
? new THREE.Vector2(v ? v[0] : 0, v ? v[1] : 0)
|
||||||
|
: def.type === 'bool' ? (v ? 1 : 0) : (v || 0),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
return uniforms;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Push the per-frame values into a fragment layer's uniforms: the frame, the
|
||||||
|
* audio features, the palette and the personality. Shared by scene layers and
|
||||||
|
* the OSD plate so they all see the same inputs — the same render path, so
|
||||||
|
* what a scene honours a plate honours too.
|
||||||
|
*/
|
||||||
|
export function setFrameUniforms(layer, renderer, target, ctx) {
|
||||||
|
const u = layer.uniforms;
|
||||||
|
const { timeline, features, prevTexture } = ctx;
|
||||||
|
const w = target ? target.width : renderer.width;
|
||||||
|
const h = target ? target.height : renderer.height;
|
||||||
|
|
||||||
|
u.u_resolution.value.set(w, h);
|
||||||
|
u.u_aspect.value = w / h;
|
||||||
|
u.u_pixelScale.value = h / 1080; // reference height; keeps 720p ≡ 4K
|
||||||
|
u.u_time.value = timeline.time;
|
||||||
|
u.u_frame.value = timeline.frame;
|
||||||
|
u.u_progress.value = timeline.progress;
|
||||||
|
u.u_opacity.value = layer.opacity;
|
||||||
|
|
||||||
|
if (features) {
|
||||||
|
for (const name of AUDIO_UNIFORMS) {
|
||||||
|
const key = name.slice(2); // u_bandLow -> bandLow
|
||||||
|
const v = features[key];
|
||||||
|
u[name].value = v === undefined ? 0 : v;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const colors = layer.palette || [];
|
||||||
|
u.u_colorCount.value = Math.max(1, Math.min(8, colors.length));
|
||||||
|
for (let i = 0; i < 8; i++) {
|
||||||
|
const c = colors[i % Math.max(1, colors.length)];
|
||||||
|
if (c) u.u_colors.value[i].set(c[0], c[1], c[2]);
|
||||||
|
}
|
||||||
|
|
||||||
|
const signature = signatureUniforms(layer.personality, layer.module);
|
||||||
|
for (const [name, type] of Object.entries(SIGNATURE_UNIFORMS)) {
|
||||||
|
const v = signature[name];
|
||||||
|
if (type === 'vec2') u[name].value.set(v[0], v[1]);
|
||||||
|
else u[name].value = v;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Framing is per shot and wins over the personality's neutral defaults —
|
||||||
|
// it is the operator's hand on a shot that is already set up, not a trait
|
||||||
|
// of the track. See look/framing.js.
|
||||||
|
if (layer.framing) {
|
||||||
|
u.u_sigFrameScale.value = layer.framing.scale;
|
||||||
|
u.u_sigFrameShift.value.set(layer.framing.shift[0], layer.framing.shift[1]);
|
||||||
|
}
|
||||||
|
|
||||||
|
u.u_prev.value = prevTexture || null;
|
||||||
|
u.u_hasPrev.value = prevTexture ? 1 : 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Common surface for shader layers and 3D layers, so Compositor holds one type. */
|
||||||
|
export class Layer {
|
||||||
|
constructor({ module, params = {}, seed = 1, opacity = 1, blend = 'normal' }) {
|
||||||
|
this.module = module;
|
||||||
|
this.baseParams = { ...params };
|
||||||
|
this.params = { ...params };
|
||||||
|
this.seed = seed >>> 0;
|
||||||
|
this.opacity = opacity;
|
||||||
|
this.blend = blend;
|
||||||
|
this.palette = [];
|
||||||
|
this.personality = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The track's production design. Constant for the whole video — see
|
||||||
|
* look/Personality.js — and pushed in the same way the palette is, so a
|
||||||
|
* layer never reaches for global state.
|
||||||
|
*/
|
||||||
|
setPersonality(personality) {
|
||||||
|
this.personality = personality;
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
setParams(params) {
|
||||||
|
this.baseParams = { ...this.baseParams, ...params };
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
setPalette(colors) {
|
||||||
|
this.palette = colors;
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* This shot's FRAMING — a per-shot scale/recentre pushed by the arc driver,
|
||||||
|
* applied inside sigCamera. A layer that is never framed renders at the
|
||||||
|
* neutral (full-frame) scale, so nothing predating framing changes.
|
||||||
|
*/
|
||||||
|
setFraming(framing) {
|
||||||
|
this.framing = framing || null;
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Resolve base params + reactive modulation into the values used this frame. */
|
||||||
|
resolveParams(features) {
|
||||||
|
const defs = this.module.params || {};
|
||||||
|
const reactive = this.module.reactive || {};
|
||||||
|
const out = this.params;
|
||||||
|
|
||||||
|
for (const name of Object.keys(defs)) out[name] = this.baseParams[name];
|
||||||
|
|
||||||
|
if (!features) return out;
|
||||||
|
|
||||||
|
for (const [name, r] of Object.entries(reactive)) {
|
||||||
|
const def = defs[name];
|
||||||
|
if (!def || def.type === 'palette') continue;
|
||||||
|
// Rate params multiply absolute time; modulating them jumps the phase
|
||||||
|
// by elapsed * delta. See params/schema.js RATE_FLAG.
|
||||||
|
if (def.rate) continue;
|
||||||
|
const raw = features[r.feature];
|
||||||
|
if (raw === undefined) continue;
|
||||||
|
|
||||||
|
const shaped = applyResponse(Math.max(0, Math.min(1, raw)), r.response);
|
||||||
|
const [lo, hi] = def.range || [0, 1];
|
||||||
|
const span = hi - lo;
|
||||||
|
const base = out[name] !== undefined ? out[name] : lo;
|
||||||
|
|
||||||
|
if (def.type === 'vec2') {
|
||||||
|
out[name] = [
|
||||||
|
clampValue(def, [base[0] + shaped * r.amount * span, 0])[0],
|
||||||
|
clampValue(def, [0, base[1] + shaped * r.amount * span])[1],
|
||||||
|
];
|
||||||
|
} else if (def.type === 'bool') {
|
||||||
|
out[name] = base;
|
||||||
|
} else {
|
||||||
|
out[name] = clampValue(def, base + shaped * r.amount * span);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
render() { throw new Error('Layer.render not implemented'); }
|
||||||
|
dispose() {}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A fullscreen fragment-shader scene. The common case. */
|
||||||
|
export class ShaderLayer extends Layer {
|
||||||
|
constructor(options) {
|
||||||
|
super(options);
|
||||||
|
|
||||||
|
this.uniforms = buildShaderUniforms(this.module, this.baseParams, this.seed);
|
||||||
|
this.material = new THREE.ShaderMaterial({
|
||||||
|
vertexShader: VERTEX_SHADER,
|
||||||
|
fragmentShader: buildFragmentShader(this.module),
|
||||||
|
uniforms: this.uniforms,
|
||||||
|
depthTest: false,
|
||||||
|
depthWrite: false,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
render(renderer, target, ctx) {
|
||||||
|
const { features } = ctx;
|
||||||
|
setFrameUniforms(this, renderer, target, ctx);
|
||||||
|
|
||||||
|
const resolved = this.resolveParams(features);
|
||||||
|
for (const [name, def] of Object.entries(this.module.params || {})) {
|
||||||
|
if (!def.uniform || def.type === 'palette') continue;
|
||||||
|
const target_u = this.uniforms[def.uniform];
|
||||||
|
const v = resolved[name];
|
||||||
|
if (v === undefined) continue;
|
||||||
|
if (def.type === 'vec2') target_u.value.set(v[0], v[1]);
|
||||||
|
else if (def.type === 'bool') target_u.value = v ? 1 : 0;
|
||||||
|
else target_u.value = v;
|
||||||
|
}
|
||||||
|
|
||||||
|
renderer.blit(this.material, target);
|
||||||
|
}
|
||||||
|
|
||||||
|
dispose() {
|
||||||
|
this.material.dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A layer backed by a real three.js scene — particles, geometry, camera motion.
|
||||||
|
* The module supplies build/update hooks; everything determinism-related (seeded
|
||||||
|
* rng, fixed dt, explicit re-seed at section boundaries) is handled here so 3D
|
||||||
|
* modules can't accidentally reintroduce wall-clock or Math.random.
|
||||||
|
*/
|
||||||
|
export class SceneLayer extends Layer {
|
||||||
|
constructor(options) {
|
||||||
|
super(options);
|
||||||
|
this.scene = new THREE.Scene();
|
||||||
|
this.camera = new THREE.PerspectiveCamera(60, 16 / 9, 0.1, 200);
|
||||||
|
this.camera.position.set(0, 0, 5);
|
||||||
|
this.instance = this.module.build({
|
||||||
|
scene: this.scene,
|
||||||
|
camera: this.camera,
|
||||||
|
seed: this.seed,
|
||||||
|
params: this.baseParams,
|
||||||
|
THREE,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
render(renderer, target, ctx) {
|
||||||
|
const { timeline, features } = ctx;
|
||||||
|
const w = target ? target.width : renderer.width;
|
||||||
|
const h = target ? target.height : renderer.height;
|
||||||
|
if (this.camera.aspect !== w / h) {
|
||||||
|
this.camera.aspect = w / h;
|
||||||
|
this.camera.updateProjectionMatrix();
|
||||||
|
}
|
||||||
|
const resolved = this.resolveParams(features);
|
||||||
|
this.module.update({
|
||||||
|
instance: this.instance,
|
||||||
|
scene: this.scene,
|
||||||
|
camera: this.camera,
|
||||||
|
timeline,
|
||||||
|
features: features || {},
|
||||||
|
params: resolved,
|
||||||
|
palette: this.palette,
|
||||||
|
personality: this.personality,
|
||||||
|
opacity: this.opacity,
|
||||||
|
THREE,
|
||||||
|
});
|
||||||
|
renderer.renderScene(this.scene, this.camera, target, true);
|
||||||
|
}
|
||||||
|
|
||||||
|
dispose() {
|
||||||
|
this.scene.traverse((obj) => {
|
||||||
|
if (obj.geometry) obj.geometry.dispose();
|
||||||
|
if (obj.material) {
|
||||||
|
const mats = Array.isArray(obj.material) ? obj.material : [obj.material];
|
||||||
|
mats.forEach((m) => m.dispose());
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function createLayer(module, options) {
|
||||||
|
if (module.kind === 'layer3d') return new SceneLayer({ module, ...options });
|
||||||
|
return new ShaderLayer({ module, ...options });
|
||||||
|
}
|
||||||
213
flow-state/src/engine/OSD.js
Normal file
213
flow-state/src/engine/OSD.js
Normal file
@ -0,0 +1,213 @@
|
|||||||
|
import * as THREE from 'three';
|
||||||
|
import { VERTEX_SHADER, PREAMBLE } from './shader-contract.js';
|
||||||
|
import { Layer, buildShaderUniforms, setFrameUniforms } from './Layer.js';
|
||||||
|
|
||||||
|
//
|
||||||
|
// OSD — the on-screen title plate.
|
||||||
|
//
|
||||||
|
// A small music-video plate that sits in the corner and names the track. It is
|
||||||
|
// NOT a DOM overlay: it is rendered into the same layer stack an export runs,
|
||||||
|
// so the title appears in the exported video exactly where it shows in preview.
|
||||||
|
//
|
||||||
|
// Following the personality is not optional fat, it is the point. The plate is
|
||||||
|
// built out of the same four traits every scene is:
|
||||||
|
//
|
||||||
|
// shape — the track's signature form is stamped beside the title as a
|
||||||
|
// monogram, so the plate carries the same subject the video does.
|
||||||
|
// camera — the same operator holds the plate: it drifts and sways and takes
|
||||||
|
// the bar-locked breath every scene takes.
|
||||||
|
// space — a soft wash of the location's air sits behind it so it stays
|
||||||
|
// readable over a busy scene.
|
||||||
|
// style — the title is edged in the track's line weight and softness, folds
|
||||||
|
// nothing, and carries the same surface grain.
|
||||||
|
//
|
||||||
|
// And it is reactive rather than inert: the plate's glow rides the loudness and
|
||||||
|
// it breathes on the bar, so even a static corner moves with the music.
|
||||||
|
//
|
||||||
|
// DETERMINISM: the plate's glyph mask is rasterised once off-frame and held as
|
||||||
|
// a texture. It uses no clock, no Math.random, no wall time — the same seed
|
||||||
|
// always produces the same title plate, and preview and export share the one
|
||||||
|
// texture, so they cannot disagree.
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The corner plate, a fullscreen pass like any other layer so it slots into the
|
||||||
|
* compositor's blend loop. It masks to a block in the bottom-left corner.
|
||||||
|
*
|
||||||
|
* "OSD" follows the personality because it shares the Layer contract: the same
|
||||||
|
* palette, audio features and signature uniforms as every scene, driven by the
|
||||||
|
* same per-frame pass.
|
||||||
|
*/
|
||||||
|
export class OSDLayer extends Layer {
|
||||||
|
constructor(renderer, { text = '', personality = null, palette = [] } = {}) {
|
||||||
|
// A layer with no rendered params; it is its own document, not a scene.
|
||||||
|
// opacity is 0 until a title exists: the compositor skips zero-opacity
|
||||||
|
// layers before they draw, so a nameless plate never writes over a frame.
|
||||||
|
super({ module: { name: 'OSD title' }, blend: 'normal', opacity: 0 });
|
||||||
|
this.renderer = renderer;
|
||||||
|
this.personality = personality;
|
||||||
|
this.palette = palette;
|
||||||
|
this.text = '';
|
||||||
|
this._titleTex = null;
|
||||||
|
|
||||||
|
this.uniforms = buildShaderUniforms({ params: {} }, {}, 0);
|
||||||
|
this.uniforms.u_title = { value: null };
|
||||||
|
this.uniforms.u_titleAspect = { value: 1 };
|
||||||
|
|
||||||
|
this.material = new THREE.ShaderMaterial({
|
||||||
|
vertexShader: VERTEX_SHADER,
|
||||||
|
fragmentShader: PREAMBLE + `\nuniform sampler2D u_title;\nuniform float u_titleAspect;\n${OSD_FRAGMENT}`,
|
||||||
|
uniforms: this.uniforms,
|
||||||
|
depthTest: false,
|
||||||
|
depthWrite: false,
|
||||||
|
});
|
||||||
|
|
||||||
|
this.setText(text, personality, palette);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Set the track name (and refresh personality/palette it renders with). */
|
||||||
|
setText(text, personality = null, palette = null) {
|
||||||
|
this.text = text || '';
|
||||||
|
if (personality) this.personality = personality;
|
||||||
|
if (palette) this.palette = palette;
|
||||||
|
this._rebuildTitleTex();
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
setPersonality(personality) { this.personality = personality; this._rebuildTitleTex(); return this; }
|
||||||
|
setPalette(palette) {
|
||||||
|
// Palette lives in uniforms (setFrameUniforms), not the texture, so a
|
||||||
|
// recolour needs no re-raster. Keep the reference for parity anyway.
|
||||||
|
this.palette = palette;
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Rasterise the song name once, off-frame. See module notes on determinism. */
|
||||||
|
_rebuildTitleTex() {
|
||||||
|
if (this._titleTex) { this._titleTex.dispose(); this._titleTex = null; }
|
||||||
|
const canvas = renderTitlePlate(this.text, this.personality);
|
||||||
|
this.uniforms.u_title.value = null;
|
||||||
|
this.opacity = canvas ? 1 : 0;
|
||||||
|
if (!canvas) return;
|
||||||
|
const tex = new THREE.CanvasTexture(canvas);
|
||||||
|
tex.colorSpace = THREE.SRGBColorSpace;
|
||||||
|
this.uniforms.u_title.value = tex;
|
||||||
|
this.uniforms.u_titleAspect.value = canvas.width / canvas.height;
|
||||||
|
this._titleTex = tex;
|
||||||
|
}
|
||||||
|
|
||||||
|
render(renderer, target, ctx) {
|
||||||
|
if (!this.uniforms.u_title.value) return;
|
||||||
|
setFrameUniforms(this, renderer, target, ctx);
|
||||||
|
renderer.blit(this.material, target);
|
||||||
|
}
|
||||||
|
|
||||||
|
dispose() {
|
||||||
|
if (this.material) this.material.dispose();
|
||||||
|
if (this._titleTex) this._titleTex.dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Rasterise the title as a white-on-transparent mask the shader recolours.
|
||||||
|
* Sized to the glyph run, so the texture's aspect is the plate's aspect.
|
||||||
|
* Returns a canvas, or null if there is nothing to draw.
|
||||||
|
*/
|
||||||
|
function renderTitlePlate(text, personality) {
|
||||||
|
const label = (text || '').trim().toUpperCase();
|
||||||
|
if (!label) return null;
|
||||||
|
|
||||||
|
// Letterspacing reflects the track's line weight; a heavier art direction
|
||||||
|
// spreads the type further apart.
|
||||||
|
const spacing = (personality && personality.style ? personality.style.lineWeight : 0.6) * 14;
|
||||||
|
const font = (px) => `700 ${px}px "Arial Black", "Avenir Next", "Helvetica Neue", "Segoe UI", sans-serif`;
|
||||||
|
const widthCap = 2048;
|
||||||
|
|
||||||
|
let canvas = document.createElement('canvas');
|
||||||
|
let ctx = canvas.getContext('2d');
|
||||||
|
const measure = (px) => {
|
||||||
|
ctx.font = font(px);
|
||||||
|
try { ctx.letterSpacing = `${spacing}px`; } catch { /* older engines ignore it */ }
|
||||||
|
return ctx.measureText(label).width;
|
||||||
|
};
|
||||||
|
|
||||||
|
let px = 1500;
|
||||||
|
let width = measure(px);
|
||||||
|
while (width > widthCap && px > 260) { px *= 0.8; width = measure(px); }
|
||||||
|
|
||||||
|
const pad = Math.round(px * 0.06);
|
||||||
|
canvas.width = Math.ceil(width + pad * 2);
|
||||||
|
canvas.height = Math.ceil(px + pad * 2);
|
||||||
|
|
||||||
|
ctx = canvas.getContext('2d');
|
||||||
|
ctx.fillStyle = '#fff';
|
||||||
|
ctx.font = font(px);
|
||||||
|
try { ctx.letterSpacing = `${spacing}px`; } catch { /* older engines */}
|
||||||
|
ctx.textBaseline = 'middle';
|
||||||
|
ctx.textAlign = 'center';
|
||||||
|
ctx.fillText(label, canvas.width / 2, canvas.height / 2);
|
||||||
|
return canvas;
|
||||||
|
}
|
||||||
|
|
||||||
|
// The plate pass. Works in centred aspect-corrected coordinates same as every
|
||||||
|
// scene, and reads the personality and audio uniforms the contract provides.
|
||||||
|
const OSD_FRAGMENT = `
|
||||||
|
void main() {
|
||||||
|
vec2 uv = vUv;
|
||||||
|
vec2 p = (uv - 0.5) * 2.0;
|
||||||
|
p.x *= u_aspect;
|
||||||
|
|
||||||
|
// --- the operator films the plate too: sway, and the bar-locked breath ---
|
||||||
|
vec2 cam = vec2(
|
||||||
|
sin(u_time * u_sigSwayRate) * u_sigSway,
|
||||||
|
cos(u_time * u_sigSwayRate * 0.83) * u_sigSway
|
||||||
|
) * 0.08;
|
||||||
|
float breath = 1.0 + u_sigBreathe * sin(u_barPhase * 6.28318530718);
|
||||||
|
|
||||||
|
vec2 corner = vec2(-u_aspect + 0.3, -1.0 + 0.1) + cam;
|
||||||
|
float blockH = 0.09 * breath;
|
||||||
|
float blockW = blockH * u_titleAspect;
|
||||||
|
vec2 blockBL = corner;
|
||||||
|
vec2 blockTR = corner + vec2(blockW, blockH);
|
||||||
|
vec2 blockC = (blockBL + blockTR) * 0.5;
|
||||||
|
|
||||||
|
vec3 acc = vec3(0.0);
|
||||||
|
float alpha = 0.0;
|
||||||
|
|
||||||
|
// --- space: the location's backing wash, soft, so the plate reads ---
|
||||||
|
vec2 rel = (p - blockC) / vec2(blockW + blockH, blockH * 2.2);
|
||||||
|
float back = exp(-dot(rel, rel) * 4.0) * 0.32;
|
||||||
|
acc += pal(0) * back * 0.4;
|
||||||
|
alpha += back;
|
||||||
|
|
||||||
|
// --- shape: the signature form stamped as a monogram left of the title ---
|
||||||
|
float emD = blockH * 0.95;
|
||||||
|
vec2 emC = vec2(blockBL.x - blockH * 1.15, blockC.y);
|
||||||
|
float d = sigShape((p - emC) / max(emD * 0.5, 1e-3)) * (emD * 0.5);
|
||||||
|
float emFill = smoothstep(u_sigSoft * emD * 0.15, -u_sigSoft * emD * 0.15, d);
|
||||||
|
float emEdge = sigEdge(d);
|
||||||
|
vec3 emCol = mix(pal(2), pal(1), 0.35);
|
||||||
|
acc += emCol * emFill * 0.2 + emCol * emEdge;
|
||||||
|
alpha += emFill * 0.2 + emEdge;
|
||||||
|
|
||||||
|
// --- the title itself, recoloured by the track ---
|
||||||
|
vec2 tuv = (p - blockBL) / vec2(blockW, blockH);
|
||||||
|
if (tuv.x >= 0.0 && tuv.x <= 1.0 && tuv.y >= 0.0 && tuv.y <= 1.0) {
|
||||||
|
float mask = texture2D(u_title, tuv).a;
|
||||||
|
float core = smoothstep(0.5, 0.62, mask);
|
||||||
|
// A soft glow instead of a hard outline: offset-sampled outlines break
|
||||||
|
// into a dotted line around the glyphs once the plate is downsampled.
|
||||||
|
float fringe = smoothstep(0.06, 0.5, mask) * (1.0 - core);
|
||||||
|
float live = 0.55 + u_loudness * 0.6;
|
||||||
|
|
||||||
|
vec3 fill = pal(1);
|
||||||
|
vec3 glow = pal(0);
|
||||||
|
acc += fill * core + glow * fringe * 0.55 * live;
|
||||||
|
alpha += core + fringe * 0.55 * live;
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- the same surface grain every scene carries ---
|
||||||
|
acc += sigGrain(uv) * 0.3;
|
||||||
|
|
||||||
|
gl_FragColor = vec4(acc, min(alpha, 1.0) * u_opacity);
|
||||||
|
}
|
||||||
|
`;
|
||||||
139
flow-state/src/engine/Renderer.js
Normal file
139
flow-state/src/engine/Renderer.js
Normal file
@ -0,0 +1,139 @@
|
|||||||
|
import * as THREE from 'three';
|
||||||
|
import { VERTEX_SHADER } from './shader-contract.js';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Thin wrapper over WebGLRenderer that provides the two primitives everything
|
||||||
|
* else is built from: allocate a render target, and run a fullscreen shader pass
|
||||||
|
* into one. Keeping this small matters — preview and export share it exactly,
|
||||||
|
* and any state that leaks between frames here would break determinism.
|
||||||
|
*/
|
||||||
|
export class Renderer {
|
||||||
|
constructor({ width = 1280, height = 720, canvas = null } = {}) {
|
||||||
|
this.gl = new THREE.WebGLRenderer({
|
||||||
|
canvas: canvas || undefined,
|
||||||
|
antialias: false, // we render through targets; MSAA here buys nothing
|
||||||
|
preserveDrawingBuffer: true, // required to read pixels back for hashing/export
|
||||||
|
powerPreference: 'high-performance',
|
||||||
|
});
|
||||||
|
this.gl.autoClear = false;
|
||||||
|
this.gl.setPixelRatio(1); // never device-dependent: output size is explicit
|
||||||
|
this.gl.setSize(width, height, false);
|
||||||
|
|
||||||
|
this.width = width;
|
||||||
|
this.height = height;
|
||||||
|
|
||||||
|
// Fullscreen quad rig, reused for every pass.
|
||||||
|
this.quadScene = new THREE.Scene();
|
||||||
|
this.quadCamera = new THREE.OrthographicCamera(-1, 1, 1, -1, 0, 1);
|
||||||
|
this.quadGeometry = new THREE.PlaneGeometry(2, 2);
|
||||||
|
this.quadMesh = new THREE.Mesh(this.quadGeometry, new THREE.MeshBasicMaterial());
|
||||||
|
this.quadMesh.frustumCulled = false;
|
||||||
|
this.quadScene.add(this.quadMesh);
|
||||||
|
|
||||||
|
this._readBuffer = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
get canvas() {
|
||||||
|
return this.gl.domElement;
|
||||||
|
}
|
||||||
|
|
||||||
|
setSize(width, height) {
|
||||||
|
if (width === this.width && height === this.height) return;
|
||||||
|
this.width = width;
|
||||||
|
this.height = height;
|
||||||
|
this.gl.setSize(width, height, false);
|
||||||
|
this._readBuffer = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
createTarget(width = this.width, height = this.height, options = {}) {
|
||||||
|
const target = new THREE.WebGLRenderTarget(width, height, {
|
||||||
|
minFilter: THREE.LinearFilter,
|
||||||
|
magFilter: THREE.LinearFilter,
|
||||||
|
format: THREE.RGBAFormat,
|
||||||
|
type: options.float ? THREE.HalfFloatType : THREE.UnsignedByteType,
|
||||||
|
depthBuffer: options.depth === true,
|
||||||
|
stencilBuffer: false,
|
||||||
|
generateMipmaps: false,
|
||||||
|
});
|
||||||
|
target.texture.wrapS = THREE.ClampToEdgeWrapping;
|
||||||
|
target.texture.wrapT = THREE.ClampToEdgeWrapping;
|
||||||
|
// Deterministic initial contents: never inherit whatever was in GPU memory.
|
||||||
|
this.clear(target);
|
||||||
|
return target;
|
||||||
|
}
|
||||||
|
|
||||||
|
clear(target = null, r = 0, g = 0, b = 0, a = 1) {
|
||||||
|
const prev = this.gl.getClearColor(new THREE.Color());
|
||||||
|
const prevAlpha = this.gl.getClearAlpha();
|
||||||
|
this.gl.setRenderTarget(target);
|
||||||
|
this.gl.setClearColor(new THREE.Color(r, g, b), a);
|
||||||
|
this.gl.clear(true, true, true);
|
||||||
|
this.gl.setClearColor(prev, prevAlpha);
|
||||||
|
this.gl.setRenderTarget(null);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Run a fullscreen shader pass. target === null renders to the canvas. */
|
||||||
|
blit(material, target = null) {
|
||||||
|
this.quadMesh.material = material;
|
||||||
|
this.gl.setRenderTarget(target);
|
||||||
|
this.gl.clear(true, false, false);
|
||||||
|
this.gl.render(this.quadScene, this.quadCamera);
|
||||||
|
this.gl.setRenderTarget(null);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Render a real three.js scene (used by 3D layers). */
|
||||||
|
renderScene(scene, camera, target = null, clear = true) {
|
||||||
|
this.gl.setRenderTarget(target);
|
||||||
|
if (clear) this.gl.clear(true, true, true);
|
||||||
|
this.gl.render(scene, camera);
|
||||||
|
this.gl.setRenderTarget(null);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Force a material's shader program to compile and link NOW.
|
||||||
|
*
|
||||||
|
* three.js links through KHR_parallel_shader_compile, so a freshly created
|
||||||
|
* material can be drawn with a program that is not ready yet, producing wrong
|
||||||
|
* frames until it is. Rendering a throwaway frame and reading it back does not
|
||||||
|
* reliably wait for the link; WebGLRenderer.compile() does.
|
||||||
|
*/
|
||||||
|
compileMaterial(material) {
|
||||||
|
const previous = this.quadMesh.material;
|
||||||
|
this.quadMesh.material = material;
|
||||||
|
this.gl.compile(this.quadScene, this.quadCamera);
|
||||||
|
this.quadMesh.material = previous;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Same, for a 3D layer's own scene. */
|
||||||
|
compileScene(scene, camera) {
|
||||||
|
this.gl.compile(scene, camera);
|
||||||
|
}
|
||||||
|
|
||||||
|
readPixels(target) {
|
||||||
|
const w = target ? target.width : this.width;
|
||||||
|
const h = target ? target.height : this.height;
|
||||||
|
const needed = w * h * 4;
|
||||||
|
if (!this._readBuffer || this._readBuffer.length !== needed) {
|
||||||
|
this._readBuffer = new Uint8Array(needed);
|
||||||
|
}
|
||||||
|
this.gl.readRenderTargetPixels(target, 0, 0, w, h, this._readBuffer);
|
||||||
|
return this._readBuffer;
|
||||||
|
}
|
||||||
|
|
||||||
|
dispose() {
|
||||||
|
this.quadGeometry.dispose();
|
||||||
|
this.gl.dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Convenience for building the shader materials used by passes. */
|
||||||
|
export function makePassMaterial(fragmentShader, uniforms) {
|
||||||
|
return new THREE.ShaderMaterial({
|
||||||
|
vertexShader: VERTEX_SHADER,
|
||||||
|
fragmentShader,
|
||||||
|
uniforms,
|
||||||
|
depthTest: false,
|
||||||
|
depthWrite: false,
|
||||||
|
transparent: true,
|
||||||
|
});
|
||||||
|
}
|
||||||
81
flow-state/src/engine/Timeline.js
Normal file
81
flow-state/src/engine/Timeline.js
Normal file
@ -0,0 +1,81 @@
|
|||||||
|
// The only clock the engine has. Scene and layer code never reads wall time —
|
||||||
|
// it reads whatever this hands it. That is what lets a realtime preview and an
|
||||||
|
// offline export produce identical frames.
|
||||||
|
//
|
||||||
|
// dt is CONSTANT (1/fps) in both modes. A dropped frame in preview shows as a
|
||||||
|
// hitch on screen; it never changes what gets rendered.
|
||||||
|
|
||||||
|
export const REALTIME = 'realtime';
|
||||||
|
export const FIXED = 'fixed';
|
||||||
|
|
||||||
|
export class Timeline {
|
||||||
|
constructor({ fps = 60, duration = 0, mode = REALTIME } = {}) {
|
||||||
|
this.fps = fps;
|
||||||
|
this.duration = duration;
|
||||||
|
this.mode = mode;
|
||||||
|
this.frame = 0;
|
||||||
|
this.playing = false;
|
||||||
|
}
|
||||||
|
|
||||||
|
get dt() {
|
||||||
|
return 1 / this.fps;
|
||||||
|
}
|
||||||
|
|
||||||
|
get time() {
|
||||||
|
return this.frame / this.fps;
|
||||||
|
}
|
||||||
|
|
||||||
|
get frameCount() {
|
||||||
|
return Math.max(1, Math.round(this.duration * this.fps));
|
||||||
|
}
|
||||||
|
|
||||||
|
get progress() {
|
||||||
|
return this.duration > 0 ? Math.min(1, this.time / this.duration) : 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
get finished() {
|
||||||
|
return this.frame >= this.frameCount - 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
setDuration(seconds) {
|
||||||
|
this.duration = seconds;
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Fixed-step advance. Used by the exporter and by preview warm-up. */
|
||||||
|
advance(frames = 1) {
|
||||||
|
this.frame = Math.min(this.frameCount - 1, this.frame + frames);
|
||||||
|
return this.frame;
|
||||||
|
}
|
||||||
|
|
||||||
|
seek(frame) {
|
||||||
|
this.frame = Math.max(0, Math.min(this.frameCount - 1, Math.round(frame)));
|
||||||
|
return this.frame;
|
||||||
|
}
|
||||||
|
|
||||||
|
seekTime(seconds) {
|
||||||
|
return this.seek(seconds * this.fps);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Realtime mode: derive the frame index from the audio element's clock.
|
||||||
|
* Audio is the master — the visuals follow it, never the other way round,
|
||||||
|
* so a slow GPU desynchronises nothing.
|
||||||
|
*/
|
||||||
|
syncToAudio(currentTime) {
|
||||||
|
this.frame = Math.max(0, Math.min(this.frameCount - 1, Math.floor(currentTime * this.fps)));
|
||||||
|
return this.frame;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A snapshot handed to layers, so nothing holds a mutable reference to the clock. */
|
||||||
|
snapshot() {
|
||||||
|
return {
|
||||||
|
frame: this.frame,
|
||||||
|
time: this.time,
|
||||||
|
dt: this.dt,
|
||||||
|
fps: this.fps,
|
||||||
|
progress: this.progress,
|
||||||
|
duration: this.duration,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
54
flow-state/src/engine/flash.js
Normal file
54
flow-state/src/engine/flash.js
Normal file
@ -0,0 +1,54 @@
|
|||||||
|
// Flash-rate safety.
|
||||||
|
//
|
||||||
|
// This project generates beat-reactive video that gets published. Rapid
|
||||||
|
// light-dark cycling between roughly 3 and 50 Hz is the photosensitive-epilepsy
|
||||||
|
// trigger, and a generator that flashes a bright scene on every kick of a 128 BPM
|
||||||
|
// track sits right in that band. WCAG 2.3.1 and the Harding test both use a
|
||||||
|
// three-flashes-per-second ceiling, which is what this measures.
|
||||||
|
//
|
||||||
|
// It is deliberately part of the gate rather than an afterthought: an
|
||||||
|
// unsupervised generator will find these states on its own, and nobody watches
|
||||||
|
// every frame of every export.
|
||||||
|
|
||||||
|
/** One flash = a min→max→min luminance cycle with amplitude at or above `threshold`. */
|
||||||
|
export function countFlashes(luminance, { threshold = 0.1 } = {}) {
|
||||||
|
if (luminance.length < 3) return 0;
|
||||||
|
|
||||||
|
const extrema = [];
|
||||||
|
for (let i = 1; i < luminance.length - 1; i++) {
|
||||||
|
const a = luminance[i - 1], b = luminance[i], c = luminance[i + 1];
|
||||||
|
if ((b > a && b >= c) || (b < a && b <= c)) {
|
||||||
|
extrema.push({ index: i, value: b, isMax: b > a });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let flashes = 0;
|
||||||
|
for (let i = 1; i < extrema.length - 1; i++) {
|
||||||
|
const prev = extrema[i - 1], here = extrema[i], next = extrema[i + 1];
|
||||||
|
if (!here.isMax) continue;
|
||||||
|
const rise = here.value - prev.value;
|
||||||
|
const fall = here.value - next.value;
|
||||||
|
if (rise >= threshold && fall >= threshold) flashes++;
|
||||||
|
}
|
||||||
|
return flashes;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Flashes per second over a luminance series sampled at `fps`. */
|
||||||
|
export function flashRate(luminance, fps) {
|
||||||
|
const seconds = luminance.length / fps;
|
||||||
|
return seconds > 0 ? countFlashes(luminance) / seconds : 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Worst flash rate in any one-second window. A track that averages 2/s but has a
|
||||||
|
* drop running at 8/s is not safe, and the average would hide it.
|
||||||
|
*/
|
||||||
|
export function peakFlashRate(luminance, fps) {
|
||||||
|
const window = Math.round(fps);
|
||||||
|
if (luminance.length <= window) return flashRate(luminance, fps);
|
||||||
|
let worst = 0;
|
||||||
|
for (let i = 0; i + window < luminance.length; i += Math.max(1, Math.round(fps / 4))) {
|
||||||
|
worst = Math.max(worst, countFlashes(luminance.slice(i, i + window)));
|
||||||
|
}
|
||||||
|
return worst;
|
||||||
|
}
|
||||||
97
flow-state/src/engine/hash.js
Normal file
97
flow-state/src/engine/hash.js
Normal file
@ -0,0 +1,97 @@
|
|||||||
|
// Frame hashing and comparison — the backbone of every determinism check.
|
||||||
|
//
|
||||||
|
// Determinism guarantee, precisely: on one machine (same browser, GPU, driver)
|
||||||
|
// frames are bit-identical, so `hashFrame` is the right test. ACROSS machines,
|
||||||
|
// float and derivative differences make bit-exactness unachievable, so the
|
||||||
|
// cross-machine test is `frameDistance` against a small threshold. Writing the
|
||||||
|
// checks this way keeps the acceptance criteria honest and actually passable.
|
||||||
|
|
||||||
|
/** FNV-1a over raw RGBA bytes. Bit-exact test, same-machine. */
|
||||||
|
export function hashFrame(pixels) {
|
||||||
|
let h = 0x811c9dc5 >>> 0;
|
||||||
|
for (let i = 0; i < pixels.length; i++) {
|
||||||
|
h ^= pixels[i];
|
||||||
|
h = Math.imul(h, 0x01000193) >>> 0;
|
||||||
|
}
|
||||||
|
return h.toString(16).padStart(8, '0');
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Mean absolute per-channel difference, 0..1. Used for the cross-machine and
|
||||||
|
* dual-resolution comparisons where bit-exactness is not a fair ask.
|
||||||
|
*/
|
||||||
|
export function frameDistance(a, b) {
|
||||||
|
if (a.length !== b.length) return 1;
|
||||||
|
let sum = 0;
|
||||||
|
for (let i = 0; i < a.length; i += 4) {
|
||||||
|
sum += Math.abs(a[i] - b[i]) + Math.abs(a[i + 1] - b[i + 1]) + Math.abs(a[i + 2] - b[i + 2]);
|
||||||
|
}
|
||||||
|
return sum / ((a.length / 4) * 3 * 255);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Largest single-channel difference. Catches localised breakage a mean would hide. */
|
||||||
|
export function frameMaxDelta(a, b) {
|
||||||
|
if (a.length !== b.length) return 255;
|
||||||
|
let max = 0;
|
||||||
|
for (let i = 0; i < a.length; i++) {
|
||||||
|
const d = Math.abs(a[i] - b[i]);
|
||||||
|
if (d > max) max = d;
|
||||||
|
}
|
||||||
|
return max;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Mean luminance, 0..1. Used by the range sweep to catch black/white-out frames. */
|
||||||
|
export function frameLuminance(pixels) {
|
||||||
|
let sum = 0;
|
||||||
|
const n = pixels.length / 4;
|
||||||
|
for (let i = 0; i < pixels.length; i += 4) {
|
||||||
|
sum += 0.2126 * pixels[i] + 0.7152 * pixels[i + 1] + 0.0722 * pixels[i + 2];
|
||||||
|
}
|
||||||
|
return sum / n / 255;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Per-channel standard deviation, averaged. Near zero means a flat, dead frame. */
|
||||||
|
export function frameVariance(pixels) {
|
||||||
|
const n = pixels.length / 4;
|
||||||
|
let mr = 0, mg = 0, mb = 0;
|
||||||
|
for (let i = 0; i < pixels.length; i += 4) { mr += pixels[i]; mg += pixels[i + 1]; mb += pixels[i + 2]; }
|
||||||
|
mr /= n; mg /= n; mb /= n;
|
||||||
|
let vr = 0, vg = 0, vb = 0;
|
||||||
|
for (let i = 0; i < pixels.length; i += 4) {
|
||||||
|
vr += (pixels[i] - mr) ** 2; vg += (pixels[i + 1] - mg) ** 2; vb += (pixels[i + 2] - mb) ** 2;
|
||||||
|
}
|
||||||
|
return (Math.sqrt(vr / n) + Math.sqrt(vg / n) + Math.sqrt(vb / n)) / 3 / 255;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** True if the frame contains any non-finite pixel artefact of a NaN in the shader. */
|
||||||
|
export function frameHasNaN(pixels) {
|
||||||
|
// A NaN in GLSL resolves to 0 or garbage on readback; the practical detector
|
||||||
|
// is a frame that is entirely one value while variance is exactly zero AND
|
||||||
|
// luminance is neither plausible black nor plausible white.
|
||||||
|
return false; // superseded by the luminance/variance checks in sweepScene
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Downsample RGBA pixels by integer box filter. Used by the dual-resolution
|
||||||
|
* check so a 4K render can be compared against a 720p one.
|
||||||
|
*/
|
||||||
|
export function downsample(pixels, width, height, factor) {
|
||||||
|
const ow = Math.floor(width / factor);
|
||||||
|
const oh = Math.floor(height / factor);
|
||||||
|
const out = new Uint8Array(ow * oh * 4);
|
||||||
|
for (let y = 0; y < oh; y++) {
|
||||||
|
for (let x = 0; x < ow; x++) {
|
||||||
|
let r = 0, g = 0, b = 0, a = 0;
|
||||||
|
for (let dy = 0; dy < factor; dy++) {
|
||||||
|
for (let dx = 0; dx < factor; dx++) {
|
||||||
|
const si = ((y * factor + dy) * width + (x * factor + dx)) * 4;
|
||||||
|
r += pixels[si]; g += pixels[si + 1]; b += pixels[si + 2]; a += pixels[si + 3];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const n = factor * factor;
|
||||||
|
const di = (y * ow + x) * 4;
|
||||||
|
out[di] = r / n; out[di + 1] = g / n; out[di + 2] = b / n; out[di + 3] = a / n;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return { pixels: out, width: ow, height: oh };
|
||||||
|
}
|
||||||
207
flow-state/src/engine/passes.js
Normal file
207
flow-state/src/engine/passes.js
Normal file
@ -0,0 +1,207 @@
|
|||||||
|
// Fullscreen shader passes used by the compositor: blending, feedback, bloom, grade.
|
||||||
|
// All noise here is keyed off u_frame rather than any random source, so grain is
|
||||||
|
// identical between a preview run and an export of the same frame.
|
||||||
|
|
||||||
|
export const BLEND_FRAG = `
|
||||||
|
precision highp float;
|
||||||
|
uniform sampler2D u_base;
|
||||||
|
uniform sampler2D u_src;
|
||||||
|
uniform float u_amount;
|
||||||
|
uniform int u_mode;
|
||||||
|
varying vec2 vUv;
|
||||||
|
|
||||||
|
vec3 blendOverlay(vec3 b, vec3 s) {
|
||||||
|
return mix(2.0 * b * s, 1.0 - 2.0 * (1.0 - b) * (1.0 - s), step(0.5, b));
|
||||||
|
}
|
||||||
|
vec3 blendSoftLight(vec3 b, vec3 s) {
|
||||||
|
return mix(2.0 * b * s + b * b * (1.0 - 2.0 * s),
|
||||||
|
sqrt(b) * (2.0 * s - 1.0) + 2.0 * b * (1.0 - s),
|
||||||
|
step(0.5, s));
|
||||||
|
}
|
||||||
|
|
||||||
|
void main() {
|
||||||
|
vec4 base = texture2D(u_base, vUv);
|
||||||
|
vec4 src = texture2D(u_src, vUv);
|
||||||
|
float a = src.a * u_amount;
|
||||||
|
vec3 result;
|
||||||
|
|
||||||
|
if (u_mode == 1) result = base.rgb + src.rgb * a; // add
|
||||||
|
else if (u_mode == 2) result = 1.0 - (1.0 - base.rgb) * (1.0 - src.rgb * a); // screen
|
||||||
|
else if (u_mode == 3) result = mix(base.rgb, base.rgb * src.rgb, a); // multiply
|
||||||
|
else if (u_mode == 4) result = mix(base.rgb, blendOverlay(base.rgb, src.rgb), a);
|
||||||
|
else if (u_mode == 5) result = mix(base.rgb, blendSoftLight(base.rgb, src.rgb), a);
|
||||||
|
else result = mix(base.rgb, src.rgb, a); // normal
|
||||||
|
|
||||||
|
gl_FragColor = vec4(result, max(base.a, a));
|
||||||
|
}
|
||||||
|
`;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Frame feedback. Disproportionately responsible for images looking alive rather
|
||||||
|
* than merely animated: the previous frame is re-sampled through a small zoom and
|
||||||
|
* rotation, decayed, and added back.
|
||||||
|
*/
|
||||||
|
export const FEEDBACK_FRAG = `
|
||||||
|
precision highp float;
|
||||||
|
uniform sampler2D u_current;
|
||||||
|
uniform sampler2D u_history;
|
||||||
|
uniform float u_decay;
|
||||||
|
uniform float u_amount;
|
||||||
|
uniform float u_zoom;
|
||||||
|
uniform float u_rotate;
|
||||||
|
uniform float u_aspect;
|
||||||
|
varying vec2 vUv;
|
||||||
|
|
||||||
|
void main() {
|
||||||
|
vec3 cur = texture2D(u_current, vUv).rgb;
|
||||||
|
|
||||||
|
vec2 p = (vUv - 0.5);
|
||||||
|
p.x *= u_aspect;
|
||||||
|
float c = cos(u_rotate), s = sin(u_rotate);
|
||||||
|
p = mat2(c, -s, s, c) * p * u_zoom;
|
||||||
|
p.x /= u_aspect;
|
||||||
|
vec2 warped = p + 0.5;
|
||||||
|
|
||||||
|
vec3 hist = texture2D(u_history, clamp(warped, 0.0, 1.0)).rgb;
|
||||||
|
// Decay strictly below 1 keeps the loop convergent; the 10k-frame stability
|
||||||
|
// check in tools/ verifies it neither saturates to white nor dies to black.
|
||||||
|
vec3 outC = cur + hist * u_decay * u_amount;
|
||||||
|
gl_FragColor = vec4(min(outC, vec3(4.0)), 1.0);
|
||||||
|
}
|
||||||
|
`;
|
||||||
|
|
||||||
|
export const BRIGHT_FRAG = `
|
||||||
|
precision highp float;
|
||||||
|
uniform sampler2D u_tex;
|
||||||
|
uniform float u_threshold;
|
||||||
|
uniform float u_knee;
|
||||||
|
varying vec2 vUv;
|
||||||
|
void main() {
|
||||||
|
vec3 c = texture2D(u_tex, vUv).rgb;
|
||||||
|
float l = dot(c, vec3(0.2126, 0.7152, 0.0722));
|
||||||
|
float k = smoothstep(u_threshold, u_threshold + u_knee, l);
|
||||||
|
gl_FragColor = vec4(c * k, 1.0);
|
||||||
|
}
|
||||||
|
`;
|
||||||
|
|
||||||
|
export const BLUR_FRAG = `
|
||||||
|
precision highp float;
|
||||||
|
uniform sampler2D u_tex;
|
||||||
|
uniform vec2 u_direction; // texel-space step, already scaled by resolution
|
||||||
|
varying vec2 vUv;
|
||||||
|
void main() {
|
||||||
|
// 9-tap gaussian, separable.
|
||||||
|
vec3 sum = texture2D(u_tex, vUv).rgb * 0.227027;
|
||||||
|
vec2 o1 = u_direction * 1.3846153846;
|
||||||
|
vec2 o2 = u_direction * 3.2307692308;
|
||||||
|
sum += (texture2D(u_tex, vUv + o1).rgb + texture2D(u_tex, vUv - o1).rgb) * 0.3162162162;
|
||||||
|
sum += (texture2D(u_tex, vUv + o2).rgb + texture2D(u_tex, vUv - o2).rgb) * 0.0702702703;
|
||||||
|
gl_FragColor = vec4(sum, 1.0);
|
||||||
|
}
|
||||||
|
`;
|
||||||
|
|
||||||
|
export const COMPOSITE_FRAG = `
|
||||||
|
precision highp float;
|
||||||
|
uniform sampler2D u_tex;
|
||||||
|
uniform sampler2D u_bloom;
|
||||||
|
uniform float u_bloomAmount;
|
||||||
|
uniform float u_chroma;
|
||||||
|
uniform float u_grain;
|
||||||
|
uniform float u_grainScale; // pixels per noise cell, 1 = per-pixel
|
||||||
|
uniform float u_grainRate; // frames a noise field survives
|
||||||
|
uniform int u_grainMask; // see look/grain.js GRAIN_MASKS
|
||||||
|
uniform float u_grainChroma; // 0 = mono speckle, 1 = full colour speckle
|
||||||
|
uniform float u_vignette;
|
||||||
|
uniform float u_contrast;
|
||||||
|
uniform float u_saturation;
|
||||||
|
uniform float u_lift;
|
||||||
|
uniform float u_exposure;
|
||||||
|
uniform float u_fade;
|
||||||
|
uniform float u_frame;
|
||||||
|
uniform vec2 u_resolution;
|
||||||
|
varying vec2 vUv;
|
||||||
|
|
||||||
|
float hash13(vec3 p) {
|
||||||
|
p = fract(p * 0.1031);
|
||||||
|
p += dot(p, p.yzx + 33.33);
|
||||||
|
return fract((p.x + p.y) * p.z);
|
||||||
|
}
|
||||||
|
|
||||||
|
void main() {
|
||||||
|
vec2 uv = vUv;
|
||||||
|
vec2 dir = (uv - 0.5);
|
||||||
|
|
||||||
|
// Chromatic aberration, radial, resolution-independent.
|
||||||
|
vec3 col;
|
||||||
|
if (u_chroma > 0.0001) {
|
||||||
|
float amt = u_chroma * 0.01;
|
||||||
|
col.r = texture2D(u_tex, uv - dir * amt).r;
|
||||||
|
col.g = texture2D(u_tex, uv).g;
|
||||||
|
col.b = texture2D(u_tex, uv + dir * amt).b;
|
||||||
|
} else {
|
||||||
|
col = texture2D(u_tex, uv).rgb;
|
||||||
|
}
|
||||||
|
|
||||||
|
col += texture2D(u_bloom, uv).rgb * u_bloomAmount;
|
||||||
|
col *= u_exposure;
|
||||||
|
|
||||||
|
// Grade: lift, contrast, saturation.
|
||||||
|
col += u_lift;
|
||||||
|
col = (col - 0.5) * u_contrast + 0.5;
|
||||||
|
float l = dot(col, vec3(0.2126, 0.7152, 0.0722));
|
||||||
|
col = mix(vec3(l), col, u_saturation);
|
||||||
|
|
||||||
|
// Vignette in normalised space so it matches at any output size.
|
||||||
|
float v = 1.0 - u_vignette * dot(dir, dir) * 2.0;
|
||||||
|
col *= clamp(v, 0.0, 1.0);
|
||||||
|
|
||||||
|
// Deterministic grain: keyed on frame index, never on a random source.
|
||||||
|
//
|
||||||
|
// Cell size and refresh rate are separate on purpose. Fine-and-boiling is
|
||||||
|
// film; coarse-and-sticky is a dirty sensor; and they are different enough
|
||||||
|
// that two tracks carrying grain do not read as the same treatment. The
|
||||||
|
// mask decides where it lands, which does more for that than amount ever
|
||||||
|
// did — grain only in the shadows is a look, grain everywhere is a filter.
|
||||||
|
if (u_grain > 0.0001) {
|
||||||
|
float cellPx = max(u_grainScale, 1.0);
|
||||||
|
vec2 cell = floor(uv * u_resolution / cellPx);
|
||||||
|
float slot = floor(u_frame / max(u_grainRate, 1.0));
|
||||||
|
|
||||||
|
float m = 1.0;
|
||||||
|
float lum = dot(clamp(col, 0.0, 1.0), vec3(0.2126, 0.7152, 0.0722));
|
||||||
|
if (u_grainMask == 1) m = 1.0 - smoothstep(0.02, 0.6, lum); // shadows
|
||||||
|
else if (u_grainMask == 2) m = smoothstep(0.2, 0.9, lum); // highlights
|
||||||
|
else if (u_grainMask == 3) m = smoothstep(0.02, 0.45, dot(dir, dir)); // toward the edges
|
||||||
|
else if (u_grainMask == 4) {
|
||||||
|
// Horizontal bands: grain in stripes, so it reads as a signal
|
||||||
|
// problem rather than as a surface.
|
||||||
|
float band = fract(uv.y * u_resolution.y / (cellPx * 8.0));
|
||||||
|
m = mix(0.15, 1.0, smoothstep(0.35, 0.5, band) * smoothstep(0.95, 0.8, band));
|
||||||
|
}
|
||||||
|
|
||||||
|
float n = hash13(vec3(cell, slot));
|
||||||
|
vec3 speckle = vec3(n);
|
||||||
|
if (u_grainChroma > 0.001) {
|
||||||
|
vec3 rgb = vec3(n,
|
||||||
|
hash13(vec3(cell + 19.0, slot)),
|
||||||
|
hash13(vec3(cell + 47.0, slot)));
|
||||||
|
speckle = mix(speckle, rgb, u_grainChroma);
|
||||||
|
}
|
||||||
|
col += (speckle - 0.5) * u_grain * m;
|
||||||
|
}
|
||||||
|
|
||||||
|
col *= u_fade;
|
||||||
|
gl_FragColor = vec4(clamp(col, 0.0, 1.0), 1.0);
|
||||||
|
}
|
||||||
|
`;
|
||||||
|
|
||||||
|
export const COPY_FRAG = `
|
||||||
|
precision highp float;
|
||||||
|
uniform sampler2D u_tex;
|
||||||
|
varying vec2 vUv;
|
||||||
|
void main() { gl_FragColor = texture2D(u_tex, vUv); }
|
||||||
|
`;
|
||||||
|
|
||||||
|
export const BLEND_MODE_IDS = {
|
||||||
|
normal: 0, add: 1, screen: 2, multiply: 3, overlay: 4, softlight: 5,
|
||||||
|
};
|
||||||
99
flow-state/src/engine/rng.js
Normal file
99
flow-state/src/engine/rng.js
Normal file
@ -0,0 +1,99 @@
|
|||||||
|
// Seeded PRNG. Every random value in the engine comes from here — never Math.random,
|
||||||
|
// which would break frame-for-frame reproducibility between preview and export.
|
||||||
|
// Mulberry32, carried over from party-stage but with the state held per instance
|
||||||
|
// instead of on a global, so layers can't perturb each other's sequences.
|
||||||
|
|
||||||
|
export class Rng {
|
||||||
|
constructor(seed = 1) {
|
||||||
|
this.seed = seed >>> 0;
|
||||||
|
this.state = this.seed;
|
||||||
|
}
|
||||||
|
|
||||||
|
reset(seed = this.seed) {
|
||||||
|
this.seed = seed >>> 0;
|
||||||
|
this.state = this.seed;
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Uniform [0, 1) */
|
||||||
|
next() {
|
||||||
|
let t = (this.state += 0x6d2b79f5) >>> 0;
|
||||||
|
t = Math.imul(t ^ (t >>> 15), t | 1);
|
||||||
|
t ^= t + Math.imul(t ^ (t >>> 7), t | 61);
|
||||||
|
return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Uniform [min, max) */
|
||||||
|
range(min, max) {
|
||||||
|
return min + this.next() * (max - min);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Integer in [min, max] inclusive */
|
||||||
|
int(min, max) {
|
||||||
|
return Math.floor(this.range(min, max + 1));
|
||||||
|
}
|
||||||
|
|
||||||
|
bool(p = 0.5) {
|
||||||
|
return this.next() < p;
|
||||||
|
}
|
||||||
|
|
||||||
|
pick(array) {
|
||||||
|
return array[this.int(0, array.length - 1)];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Weighted pick. `weights` parallels `array`; non-positive weights are skipped. */
|
||||||
|
pickWeighted(array, weights) {
|
||||||
|
let total = 0;
|
||||||
|
for (let i = 0; i < array.length; i++) total += Math.max(0, weights[i] || 0);
|
||||||
|
if (total <= 0) return this.pick(array);
|
||||||
|
let r = this.next() * total;
|
||||||
|
for (let i = 0; i < array.length; i++) {
|
||||||
|
r -= Math.max(0, weights[i] || 0);
|
||||||
|
if (r <= 0) return array[i];
|
||||||
|
}
|
||||||
|
return array[array.length - 1];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Fisher-Yates, returns a new array. */
|
||||||
|
shuffle(array) {
|
||||||
|
const out = array.slice();
|
||||||
|
for (let i = out.length - 1; i > 0; i--) {
|
||||||
|
const j = this.int(0, i);
|
||||||
|
[out[i], out[j]] = [out[j], out[i]];
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Derive an independent stream. Same parent + same label always yields the same child. */
|
||||||
|
fork(label) {
|
||||||
|
return new Rng(hashString(String(label), this.seed));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** FNV-1a over a string, optionally salted. Used for deriving seeds from names. */
|
||||||
|
export function hashString(str, salt = 0x811c9dc5) {
|
||||||
|
let h = salt >>> 0;
|
||||||
|
for (let i = 0; i < str.length; i++) {
|
||||||
|
h ^= str.charCodeAt(i);
|
||||||
|
h = Math.imul(h, 0x01000193) >>> 0;
|
||||||
|
}
|
||||||
|
return h >>> 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* FNV-1a over PCM samples, decimated so a six-minute track hashes in milliseconds.
|
||||||
|
* This is what makes a given audio file always produce the same look: the seed is a
|
||||||
|
* property of the content, not of the filename or the wall clock.
|
||||||
|
*/
|
||||||
|
export function hashSamples(float32, stride = 997) {
|
||||||
|
let h = 0x811c9dc5 >>> 0;
|
||||||
|
for (let i = 0; i < float32.length; i += stride) {
|
||||||
|
// Quantize so imperceptible float noise can't change the seed.
|
||||||
|
const q = Math.round(float32[i] * 32767) & 0xffff;
|
||||||
|
h ^= q & 0xff;
|
||||||
|
h = Math.imul(h, 0x01000193) >>> 0;
|
||||||
|
h ^= (q >>> 8) & 0xff;
|
||||||
|
h = Math.imul(h, 0x01000193) >>> 0;
|
||||||
|
}
|
||||||
|
return h >>> 0;
|
||||||
|
}
|
||||||
315
flow-state/src/engine/shader-contract.js
Normal file
315
flow-state/src/engine/shader-contract.js
Normal file
@ -0,0 +1,315 @@
|
|||||||
|
// The uniform contract every shader scene is compiled against.
|
||||||
|
//
|
||||||
|
// Scenes do not write `main()`. They define:
|
||||||
|
//
|
||||||
|
// vec4 scene(vec2 uv, vec2 p)
|
||||||
|
//
|
||||||
|
// where `uv` is 0..1 across the frame and `p` is centred, aspect-corrected,
|
||||||
|
// roughly -1..1 on the short axis. Everything else — the preamble, the varying,
|
||||||
|
// main() itself — is injected here. That boilerplate reduction is what makes a
|
||||||
|
// thirty-scene library affordable to write and to keep consistent.
|
||||||
|
//
|
||||||
|
// RESOLUTION INDEPENDENCE: work in `uv`/`p`, never in pixels. If you genuinely
|
||||||
|
// need a pixel-sized feature, scale it by u_pixelScale so a 720p preview and a
|
||||||
|
// 4K export agree. The dual-resolution diff in tools/ exists to catch violations.
|
||||||
|
|
||||||
|
export const VERTEX_SHADER = `
|
||||||
|
varying vec2 vUv;
|
||||||
|
void main() {
|
||||||
|
vUv = uv;
|
||||||
|
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
|
||||||
|
}
|
||||||
|
`;
|
||||||
|
|
||||||
|
/** Audio-reactive uniforms, filled per frame from the FeatureTrack. */
|
||||||
|
export const AUDIO_UNIFORMS = [
|
||||||
|
'u_loudness', // overall level, track-normalised 0..1
|
||||||
|
'u_rms',
|
||||||
|
'u_bandSub', // 20-60 Hz
|
||||||
|
'u_bandLow', // 60-250 Hz
|
||||||
|
'u_bandMid', // 250-2k
|
||||||
|
'u_bandHigh', // 2k-6k
|
||||||
|
'u_bandAir', // 6k-16k
|
||||||
|
'u_flux', // onset strength
|
||||||
|
'u_centroid', // spectral brightness 0..1
|
||||||
|
'u_flatness', // noisy vs tonal 0..1
|
||||||
|
'u_width', // stereo width 0..1
|
||||||
|
'u_beat', // decaying spike on each beat, 1 at the hit
|
||||||
|
'u_beatPhase', // 0..1 within the beat
|
||||||
|
'u_barPhase', // 0..1 within the bar
|
||||||
|
'u_phrasePhase',// 0..1 within an 8-bar phrase
|
||||||
|
'u_sectionProgress',
|
||||||
|
'u_sectionEnergy',
|
||||||
|
'u_buildSlope', // >0 while energy is ramping toward the next section
|
||||||
|
];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The track's personality, constant for the whole video. See look/Personality.js.
|
||||||
|
*
|
||||||
|
* These are what make sixteen unrelated shaders read as one production. A scene
|
||||||
|
* declares in `traits` which of them it honours, and the look generator will not
|
||||||
|
* cast a scene that cannot express what the track is built on.
|
||||||
|
*
|
||||||
|
* All of them are neutral by default, so a layer built without a personality
|
||||||
|
* renders exactly what it always rendered.
|
||||||
|
*/
|
||||||
|
export const SIGNATURE_UNIFORMS = {
|
||||||
|
u_sigSides: 'float', // signature form: 0 = round, else polygon sides
|
||||||
|
u_sigRound: 'float', // corner rounding of that form
|
||||||
|
u_sigElong: 'float', // how far from square the form is
|
||||||
|
u_sigTilt: 'float', // its resting angle
|
||||||
|
|
||||||
|
u_sigDrift: 'vec2', // camera translation per second
|
||||||
|
u_sigSway: 'float', // camera sway amplitude
|
||||||
|
u_sigSwayRate: 'float',
|
||||||
|
u_sigSpin: 'float', // slow camera roll, radians per second
|
||||||
|
u_sigBreathe: 'float', // bar-locked zoom
|
||||||
|
|
||||||
|
u_sigHorizon: 'float', // where the ground meets the sky, 0..1 up the frame
|
||||||
|
u_sigDepth: 'float', // distance falloff
|
||||||
|
u_sigWash: 'vec2', // background gradient direction and strength
|
||||||
|
|
||||||
|
u_sigLine: 'float', // line weight
|
||||||
|
u_sigSoft: 'float', // edge softness
|
||||||
|
u_sigTexture: 'float', // surface grain
|
||||||
|
u_sigFold: 'float', // kaleidoscopic folds, 1 = none
|
||||||
|
|
||||||
|
// FRAMING. Not a personality trait — this one is per SHOT, pushed by the
|
||||||
|
// arc driver rather than derived from the track. See look/framing.js.
|
||||||
|
//
|
||||||
|
// Every scene in the library is a locked-off, full-frame wide, and always
|
||||||
|
// has been. That is one shot type, held for the length of a song, and it is
|
||||||
|
// the reason cutting between two scenes changes the subject but never the
|
||||||
|
// FRAMING — which is at least half of how a real edit holds attention.
|
||||||
|
//
|
||||||
|
// Applied inside sigCamera, in scene coordinates, so a close-up is rendered
|
||||||
|
// close rather than being a magnified 720p image. That distinction is the
|
||||||
|
// whole reason this is a coordinate transform and not a post pass.
|
||||||
|
u_sigFrameScale: 'float', // >1 pushes in, <1 pulls back
|
||||||
|
u_sigFrameShift: 'vec2', // recentre, in scene units
|
||||||
|
};
|
||||||
|
|
||||||
|
export const FRAME_UNIFORMS = [
|
||||||
|
'u_time', 'u_frame', 'u_progress', 'u_seed',
|
||||||
|
'u_resolution', 'u_aspect', 'u_pixelScale', 'u_opacity',
|
||||||
|
];
|
||||||
|
|
||||||
|
export const PREAMBLE = `
|
||||||
|
precision highp float;
|
||||||
|
|
||||||
|
uniform vec2 u_resolution;
|
||||||
|
uniform float u_aspect;
|
||||||
|
uniform float u_pixelScale;
|
||||||
|
uniform float u_time;
|
||||||
|
uniform float u_frame;
|
||||||
|
uniform float u_progress;
|
||||||
|
uniform float u_seed;
|
||||||
|
uniform float u_opacity;
|
||||||
|
|
||||||
|
uniform vec3 u_colors[8];
|
||||||
|
uniform int u_colorCount;
|
||||||
|
|
||||||
|
${AUDIO_UNIFORMS.map((u) => `uniform float ${u};`).join('\n')}
|
||||||
|
|
||||||
|
${Object.entries(SIGNATURE_UNIFORMS).map(([u, t]) => `uniform ${t} ${u};`).join('\n')}
|
||||||
|
|
||||||
|
uniform sampler2D u_prev;
|
||||||
|
uniform int u_hasPrev;
|
||||||
|
|
||||||
|
varying vec2 vUv;
|
||||||
|
|
||||||
|
// --- palette helpers -------------------------------------------------------
|
||||||
|
// Scenes should reach for these instead of hardcoding colours, so the look
|
||||||
|
// generator can actually recolour them per track.
|
||||||
|
|
||||||
|
vec3 pal(int i) {
|
||||||
|
int n = max(u_colorCount, 1);
|
||||||
|
int k = int(mod(float(i), float(n)));
|
||||||
|
for (int j = 0; j < 8; j++) { if (j == k) return u_colors[j]; }
|
||||||
|
return u_colors[0];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Continuous ramp through the palette; t wraps. */
|
||||||
|
vec3 palRamp(float t) {
|
||||||
|
int n = max(u_colorCount, 1);
|
||||||
|
float f = fract(t) * float(n);
|
||||||
|
int i = int(floor(f));
|
||||||
|
return mix(pal(i), pal(i + 1), smoothstep(0.0, 1.0, fract(f)));
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- noise -----------------------------------------------------------------
|
||||||
|
|
||||||
|
float hash11(float n) { return fract(sin(n) * 43758.5453123); }
|
||||||
|
float hash12(vec2 p) { return fract(sin(dot(p, vec2(12.9898, 78.233))) * 43758.5453123); }
|
||||||
|
vec2 hash22(vec2 p) {
|
||||||
|
vec3 a = fract(vec3(p.xyx) * vec3(123.34, 234.34, 345.65));
|
||||||
|
a += dot(a, a + 34.45);
|
||||||
|
return fract(vec2(a.x * a.y, a.y * a.z));
|
||||||
|
}
|
||||||
|
|
||||||
|
float vnoise(vec2 p) {
|
||||||
|
vec2 i = floor(p), f = fract(p);
|
||||||
|
f = f * f * (3.0 - 2.0 * f);
|
||||||
|
float a = hash12(i), b = hash12(i + vec2(1.0, 0.0));
|
||||||
|
float c = hash12(i + vec2(0.0, 1.0)), d = hash12(i + vec2(1.0, 1.0));
|
||||||
|
return mix(mix(a, b, f.x), mix(c, d, f.x), f.y);
|
||||||
|
}
|
||||||
|
|
||||||
|
float fbm(vec2 p, int octaves) {
|
||||||
|
float v = 0.0, amp = 0.5;
|
||||||
|
for (int i = 0; i < 8; i++) {
|
||||||
|
if (i >= octaves) break;
|
||||||
|
v += amp * vnoise(p);
|
||||||
|
p *= 2.02;
|
||||||
|
amp *= 0.5;
|
||||||
|
}
|
||||||
|
return v;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Cheap divergence-free-ish flow field. The backbone of the "flow" family. */
|
||||||
|
vec2 curl(vec2 p, float t) {
|
||||||
|
float e = 0.1;
|
||||||
|
float n1 = fbm(p + vec2(0.0, e) + t, 4);
|
||||||
|
float n2 = fbm(p - vec2(0.0, e) + t, 4);
|
||||||
|
float n3 = fbm(p + vec2(e, 0.0) + t, 4);
|
||||||
|
float n4 = fbm(p - vec2(e, 0.0) + t, 4);
|
||||||
|
return vec2(n1 - n2, n4 - n3) / (2.0 * e);
|
||||||
|
}
|
||||||
|
|
||||||
|
mat2 rot(float a) { float c = cos(a), s = sin(a); return mat2(c, -s, s, c); }
|
||||||
|
|
||||||
|
/** N-fold kaleidoscopic fold of a centred coordinate. */
|
||||||
|
vec2 kaleido(vec2 p, float sides) {
|
||||||
|
if (sides < 1.5) return p;
|
||||||
|
float a = atan(p.y, p.x);
|
||||||
|
float r = length(p);
|
||||||
|
float seg = 6.28318530718 / sides;
|
||||||
|
a = abs(mod(a + seg * 0.5, seg) - seg * 0.5);
|
||||||
|
return vec2(cos(a), sin(a)) * r;
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- personality -----------------------------------------------------------
|
||||||
|
// The four traits, as functions a scene applies in its own way. A scene that
|
||||||
|
// calls none of these must not declare the matching trait, or it will be cast
|
||||||
|
// in a track it cannot express. See look/Personality.js.
|
||||||
|
|
||||||
|
/**
|
||||||
|
* TRAIT: shape. Signed distance to the track's signature form, radius ~1.
|
||||||
|
* Round tracks return a circle, so a scene can call this unconditionally.
|
||||||
|
*/
|
||||||
|
float sigShape(vec2 q) {
|
||||||
|
q = rot(u_sigTilt) * q;
|
||||||
|
q.x /= max(u_sigElong, 0.05);
|
||||||
|
|
||||||
|
if (u_sigSides < 2.5) return length(q) - 1.0;
|
||||||
|
|
||||||
|
// Regular polygon by angular folding, then rounded back toward the circle.
|
||||||
|
float seg = 6.28318530718 / u_sigSides;
|
||||||
|
float a = atan(q.y, q.x);
|
||||||
|
float r = length(q);
|
||||||
|
float folded = cos(mod(a + seg * 0.5, seg) - seg * 0.5);
|
||||||
|
float poly = r * folded - cos(seg * 0.5);
|
||||||
|
return mix(poly, r - 1.0, clamp(u_sigRound, 0.0, 1.0));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** TRAIT: shape, as a filled mask of the given radius, centred on a point. */
|
||||||
|
float sigForm(vec2 p, vec2 centre, float size) {
|
||||||
|
float d = sigShape((p - centre) / max(size, 1e-3)) * max(size, 1e-3);
|
||||||
|
return smoothstep(u_sigSoft * 0.25 + 0.004, -u_sigSoft * 0.25, d);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* TRAIT: camera. The same operator filming every scene — a slow drift, a sway,
|
||||||
|
* a roll, and a bar-locked breath. Apply to a centred coordinate before using it.
|
||||||
|
*/
|
||||||
|
vec2 sigCamera(vec2 p) {
|
||||||
|
float t = u_time;
|
||||||
|
// Framing first: everything below is the operator's hand on a shot that has
|
||||||
|
// already been set up, so it composes on top of the framing rather than
|
||||||
|
// fighting it.
|
||||||
|
p = p / max(u_sigFrameScale, 0.05) + u_sigFrameShift;
|
||||||
|
p = rot(u_sigSpin * t) * p;
|
||||||
|
p *= 1.0 - u_sigBreathe * sin(u_barPhase * 6.28318530718);
|
||||||
|
p += vec2(sin(t * u_sigSwayRate), cos(t * u_sigSwayRate * 0.83)) * u_sigSway;
|
||||||
|
// The pan RETURNS. A constant translation would be a camera on rails: two
|
||||||
|
// minutes in, every scene has left the frame entirely. This is a slow track
|
||||||
|
// across the subject and back, roughly a two-minute cycle.
|
||||||
|
p -= u_sigDrift * 20.0 * sin(t * 0.05);
|
||||||
|
return p;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** TRAIT: style. Fold the frame the track's way. 1 fold means no fold. */
|
||||||
|
vec2 sigFolded(vec2 p) {
|
||||||
|
return kaleido(p, u_sigFold);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** TRAIT: style. Turn a signed distance into an edge drawn in the track's hand. */
|
||||||
|
float sigEdge(float d) {
|
||||||
|
float w = 0.004 + u_sigLine * 0.03;
|
||||||
|
float soft = w * (0.25 + u_sigSoft * 1.5);
|
||||||
|
return smoothstep(w + soft, w - soft, abs(d));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** TRAIT: style. The track's surface grain, for a scene to add at its own weight. */
|
||||||
|
float sigGrain(vec2 uv) {
|
||||||
|
if (u_sigTexture <= 0.001) return 0.0;
|
||||||
|
return (hash12(uv * 512.0 + floor(u_frame)) - 0.5) * u_sigTexture;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* TRAIT: space. Height of the shared horizon in the same units as 'p'.
|
||||||
|
* Positive is up; a scene with any sense of ground should sit on it.
|
||||||
|
*/
|
||||||
|
float sigHorizonY() {
|
||||||
|
return (u_sigHorizon - 0.5) * 2.0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** TRAIT: space. The location's air: distance haze plus the background wash. */
|
||||||
|
vec3 sigAir(vec3 col, vec2 p, float distance01) {
|
||||||
|
vec3 far = pal(0) * (0.25 + 0.35 * u_sigDepth);
|
||||||
|
col = mix(col, far, clamp(distance01, 0.0, 1.0) * u_sigDepth);
|
||||||
|
col += pal(1) * dot(p, u_sigWash) * 0.35;
|
||||||
|
return col;
|
||||||
|
}
|
||||||
|
|
||||||
|
vec3 prev(vec2 uv) {
|
||||||
|
if (u_hasPrev == 0) return vec3(0.0);
|
||||||
|
return texture2D(u_prev, uv).rgb;
|
||||||
|
}
|
||||||
|
|
||||||
|
float sat(float x) { return clamp(x, 0.0, 1.0); }
|
||||||
|
vec3 sat3(vec3 x) { return clamp(x, 0.0, 1.0); }
|
||||||
|
`;
|
||||||
|
|
||||||
|
const EPILOGUE = `
|
||||||
|
void main() {
|
||||||
|
vec2 uv = vUv;
|
||||||
|
vec2 p = (uv - 0.5) * 2.0;
|
||||||
|
p.x *= u_aspect;
|
||||||
|
vec4 col = scene(uv, p);
|
||||||
|
gl_FragColor = vec4(col.rgb, col.a * u_opacity);
|
||||||
|
}
|
||||||
|
`;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Assemble a complete fragment shader from a scene body plus its declared params.
|
||||||
|
* Param uniforms are appended to the preamble so a scene never declares them itself —
|
||||||
|
* the schema is the single source of truth, which is what the lint checks.
|
||||||
|
*/
|
||||||
|
export function buildFragmentShader(sceneModule) {
|
||||||
|
const paramUniforms = [];
|
||||||
|
for (const [name, def] of Object.entries(sceneModule.params || {})) {
|
||||||
|
if (!def.uniform) continue;
|
||||||
|
if (def.type === 'palette') continue; // palette rides in u_colors
|
||||||
|
const glslType = def.type === 'int' ? 'int' : def.type === 'vec2' ? 'vec2' : 'float';
|
||||||
|
paramUniforms.push(`uniform ${glslType} ${def.uniform}; // param: ${name}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
return [
|
||||||
|
PREAMBLE,
|
||||||
|
paramUniforms.join('\n'),
|
||||||
|
'\n// ---- scene ----\n',
|
||||||
|
sceneModule.shader,
|
||||||
|
EPILOGUE,
|
||||||
|
].join('\n');
|
||||||
|
}
|
||||||
474
flow-state/src/export/Exporter.js
Normal file
474
flow-state/src/export/Exporter.js
Normal file
@ -0,0 +1,474 @@
|
|||||||
|
import { Muxer, ArrayBufferTarget } from 'mp4-muxer';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Offline export.
|
||||||
|
*
|
||||||
|
* Drives the same Show object the preview does, with the timeline in fixed-step
|
||||||
|
* mode — the frame index is counted, never derived from a clock. That is the
|
||||||
|
* whole basis of preview/export parity: the exporter has no render path of its
|
||||||
|
* own, so there is nothing for it to diverge from.
|
||||||
|
*
|
||||||
|
* Video goes through WebCodecs VideoEncoder (hardware accelerated where
|
||||||
|
* available) into an mp4. Audio is re-encoded from the already-decoded PCM —
|
||||||
|
* AAC where available, Opus otherwise — so the file carries the same samples
|
||||||
|
* the analysis ran on.
|
||||||
|
*/
|
||||||
|
|
||||||
|
export const PRESETS = {
|
||||||
|
'720p': { width: 1280, height: 720, bitrate: 8_000_000 },
|
||||||
|
'1080p': { width: 1920, height: 1080, bitrate: 16_000_000 },
|
||||||
|
'1440p': { width: 2560, height: 1440, bitrate: 28_000_000 },
|
||||||
|
'4K': { width: 3840, height: 2160, bitrate: 45_000_000 },
|
||||||
|
};
|
||||||
|
|
||||||
|
export function isSupported() {
|
||||||
|
return typeof VideoEncoder !== 'undefined' && typeof VideoFrame !== 'undefined';
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Frames to run through a candidate encoder when testing it for reordering.
|
||||||
|
*
|
||||||
|
* Hierarchical B-pyramids repeat every 4 frames, so two GOPs is enough to see
|
||||||
|
* the pattern while costing almost nothing even at 4K.
|
||||||
|
*/
|
||||||
|
const ORDER_PROBE_FRAMES = 12;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Does this configuration emit chunks in presentation order?
|
||||||
|
*
|
||||||
|
* It matters because of what WebCodecs does NOT expose. An encoder that emits
|
||||||
|
* B-frames delivers chunks in *decode* order, but `EncodedVideoChunk` carries
|
||||||
|
* only a presentation timestamp and no decode timestamp — so there is no way to
|
||||||
|
* recover the decode timeline after the fact. mp4-muxer, handed presentation
|
||||||
|
* timestamps, sees DTS run backwards and rejects the chunk; and because that
|
||||||
|
* throw happens inside the encoder's output callback it cannot reach the export
|
||||||
|
* loop at all. The result is a file missing three quarters of its frames with
|
||||||
|
* nothing reported. Writing the correct timeline instead would need negative
|
||||||
|
* composition offsets, which mp4-muxer emits as a version-0 `ctts` box —
|
||||||
|
* unsigned, so it cannot represent them.
|
||||||
|
*
|
||||||
|
* `isConfigSupported` says nothing about reordering, and neither does
|
||||||
|
* `latencyMode: 'realtime'` — measured, Chrome still emits B-frames under it.
|
||||||
|
* So actually encode a few frames and watch what order they come back in.
|
||||||
|
*/
|
||||||
|
async function emitsInPresentationOrder(config) {
|
||||||
|
const stamps = [];
|
||||||
|
let failed = false;
|
||||||
|
let encoder = null;
|
||||||
|
try {
|
||||||
|
encoder = new VideoEncoder({
|
||||||
|
output: (chunk) => stamps.push(chunk.timestamp),
|
||||||
|
error: () => { failed = true; },
|
||||||
|
});
|
||||||
|
encoder.configure(config);
|
||||||
|
|
||||||
|
const canvas = new OffscreenCanvas(config.width, config.height);
|
||||||
|
const ctx = canvas.getContext('2d');
|
||||||
|
const period = Math.round(1e6 / config.framerate);
|
||||||
|
for (let i = 0; i < ORDER_PROBE_FRAMES; i++) {
|
||||||
|
// Vary the content: an encoder fed identical frames may collapse
|
||||||
|
// them and never exercise its reordering path.
|
||||||
|
ctx.fillStyle = `rgb(${(i * 37) % 256} ${(i * 91) % 256} ${(i * 17) % 256})`;
|
||||||
|
ctx.fillRect(0, 0, config.width, config.height);
|
||||||
|
const frame = new VideoFrame(canvas, { timestamp: i * period, duration: period });
|
||||||
|
encoder.encode(frame, { keyFrame: i === 0 });
|
||||||
|
frame.close();
|
||||||
|
}
|
||||||
|
await encoder.flush();
|
||||||
|
} catch {
|
||||||
|
failed = true;
|
||||||
|
} finally {
|
||||||
|
try { if (encoder) encoder.close(); } catch { /* already closed */ }
|
||||||
|
}
|
||||||
|
if (failed || stamps.length !== ORDER_PROBE_FRAMES) return false;
|
||||||
|
return stamps.every((t, i) => i === 0 || t > stamps[i - 1]);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Probe for a codec configuration the browser will actually accept — and will
|
||||||
|
* encode in presentation order.
|
||||||
|
*
|
||||||
|
* The candidates run high profile first for compression efficiency, down to
|
||||||
|
* baseline last. Baseline forbids B-slices outright, so it is the profile that
|
||||||
|
* cannot reorder; the earlier entries are tried first because when a browser
|
||||||
|
* does not reorder there is no reason to give up their quality.
|
||||||
|
*/
|
||||||
|
async function pickVideoConfig(width, height, bitrate, fps) {
|
||||||
|
const candidates = [
|
||||||
|
'avc1.640034', 'avc1.640033', 'avc1.4d0034', 'avc1.42003e',
|
||||||
|
];
|
||||||
|
const supported = [];
|
||||||
|
for (const codec of candidates) {
|
||||||
|
const config = {
|
||||||
|
codec, width, height, bitrate, framerate: fps,
|
||||||
|
avc: { format: 'avc' },
|
||||||
|
};
|
||||||
|
try {
|
||||||
|
const support = await VideoEncoder.isConfigSupported(config);
|
||||||
|
if (!support.supported) continue;
|
||||||
|
} catch { continue; }
|
||||||
|
supported.push(config);
|
||||||
|
if (await emitsInPresentationOrder(config)) return config;
|
||||||
|
}
|
||||||
|
// Every supported profile reorders. Refuse rather than write a file that
|
||||||
|
// silently loses most of its frames — see emitsInPresentationOrder.
|
||||||
|
if (supported.length) {
|
||||||
|
throw new Error(
|
||||||
|
'every supported H.264 profile emits frames out of order on this browser ' +
|
||||||
|
`(tried ${supported.map((c) => c.codec).join(', ')}), which this exporter ` +
|
||||||
|
'cannot mux correctly',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Pick an audio codec the browser can actually encode.
|
||||||
|
*
|
||||||
|
* AAC is the obvious choice for mp4 but is absent from Chromium builds without
|
||||||
|
* proprietary codecs — which still ship H.264 encoding, so video succeeds and
|
||||||
|
* only audio fails. Opus in mp4 is well supported by players and by every
|
||||||
|
* platform worth uploading to, so it is the fallback rather than an error.
|
||||||
|
*/
|
||||||
|
async function pickAudioConfig(sampleRate, numberOfChannels) {
|
||||||
|
const candidates = [
|
||||||
|
{ codec: 'mp4a.40.2', muxerCodec: 'aac', bitrate: 192_000 },
|
||||||
|
{ codec: 'opus', muxerCodec: 'opus', bitrate: 160_000 },
|
||||||
|
];
|
||||||
|
for (const candidate of candidates) {
|
||||||
|
try {
|
||||||
|
const support = await AudioEncoder.isConfigSupported({
|
||||||
|
codec: candidate.codec, sampleRate, numberOfChannels, bitrate: candidate.bitrate,
|
||||||
|
});
|
||||||
|
if (support.supported) return candidate;
|
||||||
|
} catch { /* try the next one */ }
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Opus only produces a native 48 kHz stream (the input rate is resampled
|
||||||
|
* inside the encoder), so a 44.1 kHz source would end up in a file whose
|
||||||
|
* container metadata claims 44100 while the bitstream is 48000. Every stream
|
||||||
|
* rate — encoder config, chunk timestamps and the muxer timescale — must
|
||||||
|
* agree, so resample to the codec's native rate instead of trusting the
|
||||||
|
* encoder to paper over the mismatch.
|
||||||
|
*/
|
||||||
|
export const OPUS_NATIVE_RATE = 48000;
|
||||||
|
|
||||||
|
/** Linear-interpolation resampler for interleaved PCM. */
|
||||||
|
function resampleInterleaved(input, fromRate, toRate, channels) {
|
||||||
|
if (fromRate === toRate) return input;
|
||||||
|
const outFrames = Math.floor((input.length / channels) * (toRate / fromRate));
|
||||||
|
const out = new Float32Array(outFrames * channels);
|
||||||
|
const ratio = fromRate / toRate;
|
||||||
|
const last = input.length / channels - 1;
|
||||||
|
for (let s = 0; s < outFrames; s++) {
|
||||||
|
const pos = s * ratio;
|
||||||
|
const i0 = Math.floor(pos);
|
||||||
|
const i1 = Math.min(i0 + 1, last);
|
||||||
|
const frac = pos - i0;
|
||||||
|
for (let c = 0; c < channels; c++) {
|
||||||
|
out[s * channels + c] =
|
||||||
|
input[i0 * channels + c] * (1 - frac) + input[i1 * channels + c] * frac;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Describe what actually came out of the encoder.
|
||||||
|
*
|
||||||
|
* A frame that goes in and never comes out is otherwise invisible — no error
|
||||||
|
* fires, the muxer simply receives fewer samples and spreads their timestamps
|
||||||
|
* across the full duration, so the file plays at a fraction of the intended
|
||||||
|
* rate. The gap histogram says whether losses were a steady decimation (one
|
||||||
|
* dominant gap size) or bursts (a long tail), which are different bugs.
|
||||||
|
*/
|
||||||
|
function frameStats(emittedAt, encoded, emitted, fps) {
|
||||||
|
const period = 1e6 / fps;
|
||||||
|
// Chunks arrive in decode order. A timestamp that goes backwards means the
|
||||||
|
// encoder reordered — the condition that silently ate three quarters of
|
||||||
|
// every export until latencyMode pinned it down.
|
||||||
|
const reordered = emittedAt.filter((t, i) => i > 0 && t < emittedAt[i - 1]).length;
|
||||||
|
const stamps = emittedAt.slice().sort((a, b) => a - b);
|
||||||
|
const gaps = stamps.slice(1).map((t, i) => Math.round((t - stamps[i]) / period));
|
||||||
|
const gapHistogram = {};
|
||||||
|
for (const g of gaps) gapHistogram[g] = (gapHistogram[g] || 0) + 1;
|
||||||
|
const firstGap = gaps.findIndex((g) => g !== 1);
|
||||||
|
return {
|
||||||
|
encoded,
|
||||||
|
emitted,
|
||||||
|
reordered,
|
||||||
|
effectiveFps: encoded > 0 ? (emitted / encoded) * fps : 0,
|
||||||
|
maxGap: gaps.length ? Math.max(...gaps) : 0,
|
||||||
|
gapHistogram,
|
||||||
|
firstGapAt: firstGap < 0 ? null : Math.round(stamps[firstGap] / period),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export class Exporter {
|
||||||
|
constructor(show) {
|
||||||
|
this.show = show;
|
||||||
|
this.cancelled = false;
|
||||||
|
}
|
||||||
|
|
||||||
|
cancel() { this.cancelled = true; }
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @param {object} options
|
||||||
|
* @param {string} options.preset key of PRESETS
|
||||||
|
* @param {[number, number]} [options.frameRange] inclusive-exclusive frame range
|
||||||
|
* @param {(progress: {frame, total, fraction, stage}) => void} [options.onProgress]
|
||||||
|
* @returns {Promise<Blob>}
|
||||||
|
*/
|
||||||
|
async export({ preset = '1080p', frameRange = null, onProgress = null } = {}) {
|
||||||
|
if (!isSupported()) {
|
||||||
|
throw new Error('WebCodecs VideoEncoder is unavailable in this browser');
|
||||||
|
}
|
||||||
|
const show = this.show;
|
||||||
|
if (!show.ready) throw new Error('no track loaded');
|
||||||
|
|
||||||
|
const { width, height, bitrate } = PRESETS[preset] || PRESETS['1080p'];
|
||||||
|
const fps = show.fps;
|
||||||
|
|
||||||
|
const [startFrame, endFrame] = frameRange || [0, show.frameCount];
|
||||||
|
const total = Math.max(1, endFrame - startFrame);
|
||||||
|
|
||||||
|
const videoConfig = await pickVideoConfig(width, height, bitrate, fps);
|
||||||
|
if (!videoConfig) throw new Error('no supported H.264 configuration found');
|
||||||
|
|
||||||
|
const channels = show.audioBuffer ? Math.min(2, show.audioBuffer.numberOfChannels) : 0;
|
||||||
|
const audioConfig = show.audioBuffer && typeof AudioEncoder !== 'undefined'
|
||||||
|
? await pickAudioConfig(show.audioBuffer.sampleRate, channels)
|
||||||
|
: null;
|
||||||
|
const hasAudio = !!audioConfig;
|
||||||
|
this.warnings = [];
|
||||||
|
if (show.audioBuffer && !hasAudio) {
|
||||||
|
this.warnings.push('no supported audio encoder — exporting video only');
|
||||||
|
}
|
||||||
|
|
||||||
|
// Every stream rate must agree so container metadata, chunk timestamps
|
||||||
|
// and the bitstream describe the same timeline. Opus is resampled to
|
||||||
|
// its native 48 kHz; AAC keeps the decoded PCM's rate.
|
||||||
|
const sourceRate = show.audioBuffer ? show.audioBuffer.sampleRate : null;
|
||||||
|
const audioRate = hasAudio && audioConfig.muxerCodec === 'opus'
|
||||||
|
? OPUS_NATIVE_RATE
|
||||||
|
: sourceRate;
|
||||||
|
|
||||||
|
const muxer = new Muxer({
|
||||||
|
target: new ArrayBufferTarget(),
|
||||||
|
video: { codec: 'avc', width, height, frameRate: fps },
|
||||||
|
...(hasAudio ? {
|
||||||
|
audio: {
|
||||||
|
codec: audioConfig.muxerCodec,
|
||||||
|
sampleRate: audioRate,
|
||||||
|
numberOfChannels: channels,
|
||||||
|
},
|
||||||
|
} : {}),
|
||||||
|
fastStart: 'in-memory',
|
||||||
|
});
|
||||||
|
|
||||||
|
const errors = [];
|
||||||
|
// A frame that goes into the encoder and never comes out is invisible:
|
||||||
|
// no error fires, the muxer just receives fewer samples and spreads
|
||||||
|
// their timestamps over the full duration, so the file plays at a
|
||||||
|
// fraction of the intended rate. Count both ends and refuse to hand
|
||||||
|
// back a video that lost frames.
|
||||||
|
let framesEncoded = 0;
|
||||||
|
let chunksEmitted = 0;
|
||||||
|
const emittedAt = [];
|
||||||
|
const videoEncoder = new VideoEncoder({
|
||||||
|
// This callback runs from the encoder, not from the export loop, so
|
||||||
|
// a throw here escapes as an uncaught error and the loop never
|
||||||
|
// learns the chunk was lost. Catch it and route it to `errors`,
|
||||||
|
// which the loop does check. Count only chunks the muxer accepted —
|
||||||
|
// counting them on arrival would report success for frames that
|
||||||
|
// were rejected a line later.
|
||||||
|
output: (chunk, meta) => {
|
||||||
|
try {
|
||||||
|
muxer.addVideoChunk(chunk, meta);
|
||||||
|
chunksEmitted++;
|
||||||
|
emittedAt.push(chunk.timestamp);
|
||||||
|
} catch (e) {
|
||||||
|
errors.push(e);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
error: (e) => errors.push(e),
|
||||||
|
});
|
||||||
|
videoEncoder.configure(videoConfig);
|
||||||
|
|
||||||
|
// Render at export resolution. The preview's own size is restored after.
|
||||||
|
const previousWidth = show.engine.width;
|
||||||
|
const previousHeight = show.engine.height;
|
||||||
|
show.setSize(width, height);
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Compile every shader and discard a warm frame first. Programs link
|
||||||
|
// asynchronously, and an export renders each frame exactly once — there
|
||||||
|
// is no second pass to fix frame 0 with.
|
||||||
|
onProgress && onProgress({ frame: 0, total, fraction: 0, stage: 'compiling shaders' });
|
||||||
|
show.prime(startFrame);
|
||||||
|
|
||||||
|
// Warm-up so the first exported frame has the same feedback state it
|
||||||
|
// would have had in sequential playback from the range start.
|
||||||
|
if (startFrame > 0) {
|
||||||
|
onProgress && onProgress({ frame: 0, total, fraction: 0, stage: 'warming up' });
|
||||||
|
show.warmUp(startFrame, show.warmupFrames());
|
||||||
|
} else {
|
||||||
|
show.engine.compositor.reset();
|
||||||
|
}
|
||||||
|
|
||||||
|
for (let i = 0; i < total; i++) {
|
||||||
|
if (this.cancelled) throw new Error('export cancelled');
|
||||||
|
|
||||||
|
const frameIndex = startFrame + i;
|
||||||
|
const target = show.renderFrame(frameIndex);
|
||||||
|
show.present(target); // encode from the canvas, which now holds this frame
|
||||||
|
|
||||||
|
const timestamp = Math.round(((frameIndex - startFrame) * 1e6) / fps);
|
||||||
|
const videoFrame = new VideoFrame(show.engine.renderer.canvas, {
|
||||||
|
timestamp,
|
||||||
|
duration: Math.round(1e6 / fps),
|
||||||
|
});
|
||||||
|
// Keyframe every two seconds: seekable output without bloating size.
|
||||||
|
videoEncoder.encode(videoFrame, { keyFrame: i % (fps * 2) === 0 });
|
||||||
|
videoFrame.close();
|
||||||
|
framesEncoded++;
|
||||||
|
|
||||||
|
// Yield periodically so the progress UI paints, and cap how far
|
||||||
|
// the encoder may fall behind. This is a memory bound, not a
|
||||||
|
// correctness one: encode() queues without limit and does not
|
||||||
|
// drop, so the only cost of an unbounded queue is holding every
|
||||||
|
// pending frame's pixels at once — which at 4K is gigabytes.
|
||||||
|
if (i % 10 === 0) {
|
||||||
|
while (videoEncoder.encodeQueueSize > 30) {
|
||||||
|
await new Promise((r) => setTimeout(r, 4));
|
||||||
|
}
|
||||||
|
onProgress && onProgress({
|
||||||
|
frame: i, total, fraction: i / total, stage: 'rendering',
|
||||||
|
});
|
||||||
|
await new Promise((r) => setTimeout(r, 0));
|
||||||
|
}
|
||||||
|
if (errors.length) throw errors[0];
|
||||||
|
}
|
||||||
|
|
||||||
|
onProgress && onProgress({ frame: total, total, fraction: 1, stage: 'finishing video' });
|
||||||
|
await videoEncoder.flush();
|
||||||
|
|
||||||
|
this.frameStats = frameStats(emittedAt, framesEncoded, chunksEmitted, fps);
|
||||||
|
// Always report, not just on mismatch: a silent success that lost
|
||||||
|
// frames is exactly the failure this is here to catch.
|
||||||
|
console.info('[export] frameStats', this.frameStats);
|
||||||
|
if (chunksEmitted !== framesEncoded) {
|
||||||
|
const s = this.frameStats;
|
||||||
|
throw new Error(
|
||||||
|
`encoder lost ${framesEncoded - chunksEmitted} of ${framesEncoded} frames ` +
|
||||||
|
`(${chunksEmitted} encoded chunks) — the export would play at ` +
|
||||||
|
`${s.effectiveFps.toFixed(1)} fps instead of ${fps}. ` +
|
||||||
|
`First gap at frame ${s.firstGapAt}; gap sizes ${JSON.stringify(s.gapHistogram)}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (hasAudio) {
|
||||||
|
onProgress && onProgress({ frame: total, total, fraction: 1, stage: 'encoding audio' });
|
||||||
|
try {
|
||||||
|
await this._encodeAudio(muxer, audioConfig, startFrame, endFrame, fps, audioRate);
|
||||||
|
} catch (err) {
|
||||||
|
// A finished silent video beats losing a long render outright.
|
||||||
|
// The muxer tolerates an audio track that received no chunks.
|
||||||
|
this.warnings.push(`audio encoding failed (${err.message}) — video only`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
muxer.finalize();
|
||||||
|
return new Blob([muxer.target.buffer], { type: 'video/mp4' });
|
||||||
|
} finally {
|
||||||
|
try { videoEncoder.close(); } catch { /* already closed */ }
|
||||||
|
show.setSize(previousWidth, previousHeight);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Encode the exported time range of the decoded PCM and mux it. */
|
||||||
|
async _encodeAudio(muxer, audioConfig, startFrame, endFrame, fps, audioRate) {
|
||||||
|
const buffer = this.show.audioBuffer;
|
||||||
|
const sourceRate = buffer.sampleRate;
|
||||||
|
const channels = Math.min(2, buffer.numberOfChannels);
|
||||||
|
|
||||||
|
const startSample = Math.floor((startFrame / fps) * sourceRate);
|
||||||
|
const endSample = Math.min(buffer.length, Math.ceil((endFrame / fps) * sourceRate));
|
||||||
|
const length = Math.max(0, endSample - startSample);
|
||||||
|
if (!length) return;
|
||||||
|
|
||||||
|
const errors = [];
|
||||||
|
const encoder = new AudioEncoder({
|
||||||
|
output: (chunk, meta) => muxer.addAudioChunk(chunk, meta),
|
||||||
|
error: (e) => errors.push(e),
|
||||||
|
});
|
||||||
|
encoder.configure({
|
||||||
|
codec: audioConfig.codec,
|
||||||
|
sampleRate: audioRate,
|
||||||
|
numberOfChannels: channels,
|
||||||
|
bitrate: audioConfig.bitrate,
|
||||||
|
});
|
||||||
|
|
||||||
|
const chunkFrames = 1024;
|
||||||
|
const sources = [];
|
||||||
|
for (let c = 0; c < channels; c++) sources.push(buffer.getChannelData(c));
|
||||||
|
|
||||||
|
// Slice the range, resampling to the codec's native rate when needed so
|
||||||
|
// chunk timestamps and the muxer timescale describe the same timeline.
|
||||||
|
const interleaved = new Float32Array(length * channels);
|
||||||
|
for (let i = 0; i < length; i++) {
|
||||||
|
for (let c = 0; c < channels; c++) {
|
||||||
|
interleaved[i * channels + c] = sources[c][startSample + i];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const pcm = audioRate === sourceRate
|
||||||
|
? interleaved
|
||||||
|
: resampleInterleaved(interleaved, sourceRate, audioRate, channels);
|
||||||
|
|
||||||
|
const outFrames = pcm.length / channels;
|
||||||
|
const buf = new Float32Array(chunkFrames * channels);
|
||||||
|
|
||||||
|
for (let offset = 0; offset < outFrames; offset += chunkFrames) {
|
||||||
|
const count = Math.min(chunkFrames, outFrames - offset);
|
||||||
|
buf.set(pcm.subarray(offset * channels, (offset + count) * channels), 0);
|
||||||
|
const data = new AudioData({
|
||||||
|
format: 'f32',
|
||||||
|
sampleRate: audioRate,
|
||||||
|
numberOfFrames: count,
|
||||||
|
numberOfChannels: channels,
|
||||||
|
timestamp: Math.round((offset / audioRate) * 1e6),
|
||||||
|
data: buf.slice(0, count * channels),
|
||||||
|
});
|
||||||
|
encoder.encode(data);
|
||||||
|
data.close();
|
||||||
|
if (errors.length) throw errors[0];
|
||||||
|
if (offset % (chunkFrames * 64) === 0) await new Promise((r) => setTimeout(r, 0));
|
||||||
|
}
|
||||||
|
await encoder.flush();
|
||||||
|
encoder.close();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Render a short range around a frame — the "test render" bridge before a full export. */
|
||||||
|
export async function exportSegment(show, centreFrame,
|
||||||
|
{ seconds = 20, preset = '1080p', onProgress, exporter = null } = {}) {
|
||||||
|
const half = Math.round((seconds * show.fps) / 2);
|
||||||
|
const start = Math.max(0, centreFrame - half);
|
||||||
|
const end = Math.min(show.frameCount, centreFrame + half);
|
||||||
|
// Accept a caller-supplied Exporter so the caller can read `warnings` and cancel.
|
||||||
|
return (exporter || new Exporter(show)).export({ preset, frameRange: [start, end], onProgress });
|
||||||
|
}
|
||||||
|
|
||||||
|
export function downloadBlob(blob, filename) {
|
||||||
|
const url = URL.createObjectURL(blob);
|
||||||
|
const a = document.createElement('a');
|
||||||
|
a.href = url;
|
||||||
|
a.download = filename;
|
||||||
|
document.body.appendChild(a);
|
||||||
|
a.click();
|
||||||
|
a.remove();
|
||||||
|
setTimeout(() => URL.revokeObjectURL(url), 10000);
|
||||||
|
}
|
||||||
539
flow-state/src/look/ArcDriver.js
Normal file
539
flow-state/src/look/ArcDriver.js
Normal file
@ -0,0 +1,539 @@
|
|||||||
|
import { createLayer } from '../engine/Layer.js';
|
||||||
|
import { Rng } from '../engine/rng.js';
|
||||||
|
import { clampValue } from '../params/schema.js';
|
||||||
|
import { paletteShiftAt } from './paletteArc.js';
|
||||||
|
import { shiftPalette } from './palette.js';
|
||||||
|
import { frameShot, neutralFraming } from './framing.js';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Drives the look across the song.
|
||||||
|
*
|
||||||
|
* Four timescales are stacked here, and it takes all four to keep six minutes
|
||||||
|
* from reading as a loop:
|
||||||
|
*
|
||||||
|
* per frame — reactive mappings (handled in Layer, from the feature row)
|
||||||
|
* per shot — cuts between the section's stage visuals, on phrase lines
|
||||||
|
* per section — seeded LFO drift, so nothing sits still during a long sustain
|
||||||
|
* whole song — scene changes at real boundaries, plus lookahead ramps that
|
||||||
|
* build INTO a drop rather than reacting after it lands
|
||||||
|
*
|
||||||
|
* The shot level is what stops a ninety-second sustain from being one held
|
||||||
|
* image. Everything below works in CUES — a flat list of (section, shot) spans
|
||||||
|
* built from the look, so a shot cut and a section change take exactly the same
|
||||||
|
* code path and differ only in how long the transition is. See look/shots.js.
|
||||||
|
*
|
||||||
|
* Layer instances are created once per (section, variant) and reused across
|
||||||
|
* every shot that shows that variant. Rebuilding them per shot would recompile
|
||||||
|
* shaders at every cut, which is the obvious way to make this unusably slow.
|
||||||
|
*/
|
||||||
|
export class ArcDriver {
|
||||||
|
constructor(look, track, { crossfadeBars = 1, driftAmount = 0.09 } = {}) {
|
||||||
|
this.look = look;
|
||||||
|
this.track = track;
|
||||||
|
this.driftAmount = driftAmount;
|
||||||
|
|
||||||
|
const barSeconds = (track.tempo.period * track.tempo.beatsPerBar) / track.fps;
|
||||||
|
this.crossfadeFrames = Math.max(12, Math.round(barSeconds * crossfadeBars * track.fps));
|
||||||
|
this.barFrames = Math.max(1, Math.round(barSeconds * track.fps));
|
||||||
|
|
||||||
|
this.layerCache = new Map();
|
||||||
|
this.driftPlans = new Map();
|
||||||
|
this.activeLayers = [];
|
||||||
|
this.cues = this._buildCues();
|
||||||
|
this.framingStyle = (look.framing && look.framing.mode !== 'locked')
|
||||||
|
? look.framing : null;
|
||||||
|
this._planFraming();
|
||||||
|
this.state = { sectionIndex: 0, shotIndex: 0, crossfade: 0, incoming: null };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Give every cue the framing its shot will be played with.
|
||||||
|
*
|
||||||
|
* Framing is the one device that lives at the SHOT level — a wide answered
|
||||||
|
* by a close is the whole point of it, and both belong to the same scene.
|
||||||
|
* Planned here, once, walking the cues in order, so the whole video gets a
|
||||||
|
* consistent hand and a seek always finds the same framing as playback.
|
||||||
|
* A track that decided to stay locked-off (or a hand-built look with no
|
||||||
|
* framing) gets the neutral, unframed read everywhere.
|
||||||
|
*/
|
||||||
|
_planFraming() {
|
||||||
|
if (!this.framingStyle) return;
|
||||||
|
const rng = new Rng((this.look.seed ^ 0x517cc1b7) >>> 0);
|
||||||
|
let previous = null;
|
||||||
|
for (const cue of this.cues) {
|
||||||
|
const section = this.look.sections[cue.sectionIndex];
|
||||||
|
const energy = (section.bias && section.bias.energy) || 0;
|
||||||
|
const framing = frameShot(this.framingStyle, previous, energy, rng);
|
||||||
|
cue.framing = framing;
|
||||||
|
previous = framing;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
dispose() {
|
||||||
|
for (const layer of this.layerCache.values()) layer.dispose();
|
||||||
|
this.layerCache.clear();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Flatten the look into cues: one per shot, in playback order.
|
||||||
|
*
|
||||||
|
* A look generated before shots existed (or hand-built by a check) has no
|
||||||
|
* `shots` array; it degrades to exactly one cue per section, which is the
|
||||||
|
* old behaviour.
|
||||||
|
*/
|
||||||
|
_buildCues() {
|
||||||
|
const cues = [];
|
||||||
|
this.look.sections.forEach((section, sectionIndex) => {
|
||||||
|
const shots = (section.shots && section.shots.length) ? section.shots : [{
|
||||||
|
startFrame: section.startFrame, endFrame: section.endFrame,
|
||||||
|
variant: 0, hardCut: false,
|
||||||
|
}];
|
||||||
|
const energy = (section.bias && section.bias.energy) || 0;
|
||||||
|
shots.forEach((shot, shotIndex) => {
|
||||||
|
const atSectionStart = shotIndex === 0;
|
||||||
|
const span = shot.endFrame - shot.startFrame;
|
||||||
|
cues.push({
|
||||||
|
index: cues.length,
|
||||||
|
sectionIndex,
|
||||||
|
shotIndex,
|
||||||
|
variant: shot.variant || 0,
|
||||||
|
startFrame: shot.startFrame,
|
||||||
|
endFrame: shot.endFrame,
|
||||||
|
atSectionStart,
|
||||||
|
fadeFrames: shot.hardCut && !atSectionStart
|
||||||
|
? Math.max(2, Math.round(this.track.fps * 0.06))
|
||||||
|
: this._dissolveFrames(energy, span),
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
return cues;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* How long a dissolve takes: the default transition, and deliberately slow.
|
||||||
|
*
|
||||||
|
* Two bars on calm material, one on loud — a long dissolve between two
|
||||||
|
* quiet scenes reads as the image evolving, while the same length under a
|
||||||
|
* drop reads as mush, because both images are moving too fast to overlay.
|
||||||
|
* Capped at 40% of the incoming shot so a transition never occupies most of
|
||||||
|
* the shot it is transitioning into.
|
||||||
|
*/
|
||||||
|
_dissolveFrames(energy, spanFrames) {
|
||||||
|
const bars = energy > 0.6 ? 1 : 2;
|
||||||
|
const wanted = Math.max(this.crossfadeFrames, this.barFrames * bars);
|
||||||
|
return Math.max(12, Math.round(Math.min(wanted, spanFrames * 0.4)));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Cue covering a frame. Binary search — a seek can land anywhere. */
|
||||||
|
_cueIndexAt(frame) {
|
||||||
|
const cues = this.cues;
|
||||||
|
let lo = 0;
|
||||||
|
let hi = cues.length - 1;
|
||||||
|
while (lo < hi) {
|
||||||
|
const mid = (lo + hi + 1) >> 1;
|
||||||
|
if (cues[mid].startFrame <= frame) lo = mid; else hi = mid - 1;
|
||||||
|
}
|
||||||
|
return lo;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The cue on screen at a frame. For the UI and the checks. */
|
||||||
|
cueAt(frame) {
|
||||||
|
return this.cues[this._cueIndexAt(frame)];
|
||||||
|
}
|
||||||
|
|
||||||
|
_specFor(sectionIndex, variant, slot) {
|
||||||
|
const section = this.look.sections[sectionIndex];
|
||||||
|
const stack = (section.variants && section.variants[variant]) || section.layers;
|
||||||
|
return stack[slot] || null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** One Layer per (section, variant, layer slot), built lazily and kept. */
|
||||||
|
_layerFor(sectionIndex, variant, slot = 0) {
|
||||||
|
const key = `${sectionIndex}:${variant}:${slot}`;
|
||||||
|
let layer = this.layerCache.get(key);
|
||||||
|
if (!layer) {
|
||||||
|
const spec = this._specFor(sectionIndex, variant, slot);
|
||||||
|
layer = createLayer(spec.module, {
|
||||||
|
params: spec.params,
|
||||||
|
seed: spec.seed,
|
||||||
|
opacity: spec.opacity,
|
||||||
|
blend: spec.blend,
|
||||||
|
});
|
||||||
|
layer.setPalette(this.look.palette);
|
||||||
|
layer.setPersonality(this.look.personality);
|
||||||
|
this.layerCache.set(key, layer);
|
||||||
|
}
|
||||||
|
return layer;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Per-param LFO plan for a section: amplitude, period and phase, all seeded.
|
||||||
|
* Slow enough to read as evolution rather than wobble — 20 to 70 seconds.
|
||||||
|
*/
|
||||||
|
_driftPlan(sectionIndex, variant, slot = 0) {
|
||||||
|
const key = `${sectionIndex}:${variant}:${slot}`;
|
||||||
|
let plan = this.driftPlans.get(key);
|
||||||
|
if (plan) return plan;
|
||||||
|
|
||||||
|
const spec = this._specFor(sectionIndex, variant, slot);
|
||||||
|
const rng = new Rng(spec.seed ^ 0x5bf03635);
|
||||||
|
plan = [];
|
||||||
|
for (const [name, def] of Object.entries(spec.module.params || {})) {
|
||||||
|
if (def.type === 'palette' || def.type === 'bool' || def.fixed) continue;
|
||||||
|
if (def.noDrift || def.rate) continue; // see schema.js RATE_FLAG
|
||||||
|
const [lo, hi] = def.range || [0, 1];
|
||||||
|
plan.push({
|
||||||
|
name,
|
||||||
|
def,
|
||||||
|
amplitude: (hi - lo) * this.driftAmount * rng.range(0.4, 1.3),
|
||||||
|
period: rng.range(20, 70),
|
||||||
|
phase: rng.next(),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
this.driftPlans.set(key, plan);
|
||||||
|
return plan;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The scene's SLOW AXIS: one or two params that travel one way across the
|
||||||
|
* whole track.
|
||||||
|
*
|
||||||
|
* Drift above is an LFO with a 20-70 second period, and an LFO returns.
|
||||||
|
* Measured over the library, that is exactly what several scenes' problem
|
||||||
|
* was: they change as much in half a second as in two minutes, because
|
||||||
|
* everything moving in them is cyclic, so the eye adapts in about two
|
||||||
|
* seconds and then there is nothing left to find. Violently animated and
|
||||||
|
* read as static. Ten cycles of a 30-second wobble is not five minutes of
|
||||||
|
* anything.
|
||||||
|
*
|
||||||
|
* So this is deliberately monotonic. Where drift is the wobble, this is the
|
||||||
|
* journey: the frame at four minutes has a different STRUCTURE — density,
|
||||||
|
* scale, count — from the frame at thirty seconds, and no amount of
|
||||||
|
* per-frame reactivity substitutes for that.
|
||||||
|
*
|
||||||
|
* Keyed on the module rather than on the section, so a scene that comes back
|
||||||
|
* in the last section arrives further along its own axis rather than
|
||||||
|
* resetting. Rate params are excluded for the reason schema.js gives: they
|
||||||
|
* multiply absolute time, so moving one jumps the phase.
|
||||||
|
*/
|
||||||
|
_slowAxisFor(module) {
|
||||||
|
if (!this._slowAxes) this._slowAxes = new Map();
|
||||||
|
const cached = this._slowAxes.get(module.name);
|
||||||
|
if (cached) return cached;
|
||||||
|
|
||||||
|
// Stable per (track, scene): the same scene evolves the same way
|
||||||
|
// wherever it appears in this video, and differently in the next one.
|
||||||
|
let h = (this.look.seed || 1) >>> 0;
|
||||||
|
for (let i = 0; i < module.name.length; i++) {
|
||||||
|
h = (Math.imul(h ^ module.name.charCodeAt(i), 0x01000193) >>> 0);
|
||||||
|
}
|
||||||
|
const rng = new Rng(h);
|
||||||
|
|
||||||
|
const eligible = Object.entries(module.params || {}).filter(([, def]) =>
|
||||||
|
def.type !== 'palette' && def.type !== 'bool' && !def.fixed
|
||||||
|
&& !def.rate && !def.noDrift && def.range);
|
||||||
|
|
||||||
|
// A param the scene DECLARES as its axis wins outright, and travels
|
||||||
|
// much further than a guessed one.
|
||||||
|
//
|
||||||
|
// The first version of this picked at random from everything eligible
|
||||||
|
// and measured as doing nothing whatsoever: the time-averaged image at
|
||||||
|
// thirty seconds and at two and a half minutes differed by the same
|
||||||
|
// amount with the axis applied as without it. The reason is that which
|
||||||
|
// param you move decides everything. Sweeping Moiré Grid's `width`
|
||||||
|
// moves its averaged structure by 0.110 and its `offset` by 0.002, and
|
||||||
|
// a random draw finds the second kind almost every time.
|
||||||
|
const declared = eligible.filter(([, def]) => def.slowAxis);
|
||||||
|
|
||||||
|
const axis = [];
|
||||||
|
if (declared.length) {
|
||||||
|
for (const [name, def] of declared) {
|
||||||
|
const [lo, hi] = def.range;
|
||||||
|
axis.push({
|
||||||
|
name,
|
||||||
|
def,
|
||||||
|
declared: true,
|
||||||
|
// Most of the range. This param was chosen because moving it
|
||||||
|
// is what the scene looks like changing, so a timid walk
|
||||||
|
// wastes the one lever that works.
|
||||||
|
travel: (hi - lo) * rng.range(0.45, 0.7) * (rng.bool() ? 1 : -1),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// Nothing declared: fall back to a guess. Worth keeping — it costs
|
||||||
|
// nothing and occasionally lands on something structural — but it is
|
||||||
|
// not what makes this mechanism work, and no gate should rely on it.
|
||||||
|
const count = Math.min(eligible.length, rng.bool(0.45) ? 2 : 1);
|
||||||
|
const pool = rng.shuffle(eligible.slice());
|
||||||
|
for (let i = 0; i < count; i++) {
|
||||||
|
const [name, def] = pool[i];
|
||||||
|
const [lo, hi] = def.range;
|
||||||
|
axis.push({
|
||||||
|
name,
|
||||||
|
def,
|
||||||
|
declared: false,
|
||||||
|
travel: (hi - lo) * rng.range(0.25, 0.5) * (rng.bool() ? 1 : -1),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
this._slowAxes.set(module.name, axis);
|
||||||
|
return axis;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Base params for a section at a given time: the look's sampled values, plus
|
||||||
|
* the slow axis, plus drift, plus the lookahead ramp toward what comes next.
|
||||||
|
*/
|
||||||
|
_paramsAt(cue, slot, time, features) {
|
||||||
|
const spec = this._specFor(cue.sectionIndex, cue.variant, slot);
|
||||||
|
const out = { ...spec.params };
|
||||||
|
|
||||||
|
// --- slow axis ------------------------------------------------------
|
||||||
|
// Eased rather than linear, so the travel is slowest at the head and
|
||||||
|
// tail. A video should not open mid-move.
|
||||||
|
const duration = Math.max(1e-6, this.track.duration);
|
||||||
|
const p = Math.max(0, Math.min(1, time / duration));
|
||||||
|
const journey = p * p * (3 - 2 * p);
|
||||||
|
for (const item of this._slowAxisFor(spec.module)) {
|
||||||
|
const base = out[item.name];
|
||||||
|
if (typeof base !== 'number') continue;
|
||||||
|
out[item.name] = clampValue(item.def, base + item.travel * (journey - 0.5));
|
||||||
|
}
|
||||||
|
|
||||||
|
for (const item of this._driftPlan(cue.sectionIndex, cue.variant, slot)) {
|
||||||
|
const base = out[item.name];
|
||||||
|
if (typeof base !== 'number') continue;
|
||||||
|
const wave = Math.sin(2 * Math.PI * (time / item.period + item.phase));
|
||||||
|
out[item.name] = clampValue(item.def, base + wave * item.amplitude);
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- lookahead ------------------------------------------------------
|
||||||
|
// buildSlope rises through the bars before a higher-energy section. This
|
||||||
|
// is the payoff of analysing offline: the visuals arrive at the drop
|
||||||
|
// already at tension instead of catching up afterwards.
|
||||||
|
const slope = features ? features.buildSlope || 0 : 0;
|
||||||
|
if (slope > 0.001) {
|
||||||
|
const nextCue = this.cues[cue.index + 1];
|
||||||
|
const next = nextCue
|
||||||
|
? this._specFor(nextCue.sectionIndex, nextCue.variant, slot)
|
||||||
|
: null;
|
||||||
|
if (next && next.module === spec.module) {
|
||||||
|
// Same scene either side: ramp the actual target values.
|
||||||
|
const target = next.params;
|
||||||
|
for (const [name, def] of Object.entries(spec.module.params || {})) {
|
||||||
|
if (def.type === 'palette' || typeof out[name] !== 'number') continue;
|
||||||
|
if (typeof target[name] !== 'number') continue;
|
||||||
|
out[name] = clampValue(def, out[name] + (target[name] - out[name]) * slope);
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// Different scene: push the intensity-ish params toward the top
|
||||||
|
// of their range so the build still reads as a build.
|
||||||
|
for (const [name, def] of Object.entries(spec.module.params || {})) {
|
||||||
|
if (typeof out[name] !== 'number') continue;
|
||||||
|
if (def.bias !== 'energy' && def.bias !== 'density') continue;
|
||||||
|
const hi = (def.range || [0, 1])[1];
|
||||||
|
out[name] = clampValue(def, out[name] + (hi - out[name]) * slope * 0.5);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The buildSlope value on the frame before a boundary. Read from the table
|
||||||
|
* rather than remembered, so a seek and playback agree.
|
||||||
|
*
|
||||||
|
* Indexes the typed array DIRECTLY rather than calling track.at(). at()
|
||||||
|
* returns a single reused row object, so calling it here — mid-render, while
|
||||||
|
* the caller is still holding the row for the current frame — silently
|
||||||
|
* rewrites the features the layer is about to read. That produced a render
|
||||||
|
* that was correct on every repeat but wrong the first time through, which is
|
||||||
|
* exactly the kind of fault the determinism checks exist to surface.
|
||||||
|
*/
|
||||||
|
_boundarySlope(cue) {
|
||||||
|
if (!this._slopeCache) this._slopeCache = new Map();
|
||||||
|
if (this._slopeCache.has(cue.index)) return this._slopeCache.get(cue.index);
|
||||||
|
|
||||||
|
const frame = Math.max(0, cue.startFrame - 1);
|
||||||
|
const value = this.track.tracks.buildSlope[frame] || 0;
|
||||||
|
this._slopeCache.set(cue.index, value);
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Compute the active layer stack for a frame.
|
||||||
|
*
|
||||||
|
* The transition runs FORWARD from a cue: the outgoing image holds at full
|
||||||
|
* opacity while the incoming one fades in over it. That keeps the cue frame
|
||||||
|
* itself a clean state, which is what makes a boundary seek exact without
|
||||||
|
* warm-up. A hard cut is the same path with a two-frame fade — it is still
|
||||||
|
* a ramp rather than a jump, because a single-frame swap of two bright
|
||||||
|
* scenes is a flash, and the flash meter is not decorative.
|
||||||
|
*/
|
||||||
|
/**
|
||||||
|
* The track's palette, moved to where this frame sits in the arc.
|
||||||
|
*
|
||||||
|
* Recomputed once per frame rather than once per layer, and memoised on the
|
||||||
|
* rounded shift: the movement is slow by design, so consecutive frames
|
||||||
|
* almost always want the same colours and the OKLCH round trip is wasted
|
||||||
|
* work. Rounding also makes the cache key stable under a seek, which keeps
|
||||||
|
* the frame-exactness guarantee — a seeked frame gets bit-identical colours
|
||||||
|
* to a played one rather than merely similar ones.
|
||||||
|
*/
|
||||||
|
_paletteAt(frame, features) {
|
||||||
|
const arc = this.look.paletteArc;
|
||||||
|
if (!arc || arc.mode === 'static') return this.look.palette;
|
||||||
|
|
||||||
|
const shift = paletteShiftAt(arc, {
|
||||||
|
progress: frame / Math.max(1, this.track.frameCount - 1),
|
||||||
|
sectionKind: this.track.sectionAt(frame).kind,
|
||||||
|
features,
|
||||||
|
});
|
||||||
|
|
||||||
|
const key = `${shift.hue.toFixed(3)}|${shift.chroma.toFixed(3)}|${shift.lightness.toFixed(3)}`;
|
||||||
|
if (this._paletteKey === key) return this._palette;
|
||||||
|
|
||||||
|
this._paletteKey = key;
|
||||||
|
this._palette = shiftPalette(this.look.palette, shift);
|
||||||
|
return this._palette;
|
||||||
|
}
|
||||||
|
|
||||||
|
update(frame, features) {
|
||||||
|
const time = frame / this.track.fps;
|
||||||
|
const palette = this._paletteAt(frame, features);
|
||||||
|
const cueIndex = this._cueIndexAt(frame);
|
||||||
|
const cue = this.cues[cueIndex];
|
||||||
|
if (!cue) return this.activeLayers;
|
||||||
|
const section = this.look.sections[cue.sectionIndex];
|
||||||
|
|
||||||
|
const framesIntoCue = frame - cue.startFrame;
|
||||||
|
const previous = cueIndex > 0 ? this.cues[cueIndex - 1] : null;
|
||||||
|
const fading = !!previous && framesIntoCue < cue.fadeFrames;
|
||||||
|
const t = fading ? framesIntoCue / cue.fadeFrames : 1;
|
||||||
|
const eased = t * t * (3 - 2 * t);
|
||||||
|
|
||||||
|
// The shot being played INTO carries its own framing; the shot fading
|
||||||
|
// out keeps the framing it was filmed with, so a cut changes the size
|
||||||
|
// exactly when the cut changes the image rather than half a beat after.
|
||||||
|
const framing = cue.framing || neutralFraming();
|
||||||
|
const outgoingFraming = previous ? (previous.framing || neutralFraming()) : framing;
|
||||||
|
|
||||||
|
const layers = [];
|
||||||
|
|
||||||
|
if (fading) {
|
||||||
|
// buildSlope is discontinuous at a section boundary by construction:
|
||||||
|
// it ramps to ~1 through the bars before the change and is 0
|
||||||
|
// immediately after. The outgoing layer is still on screen when that
|
||||||
|
// happens, so feeding it the new section's features collapses its
|
||||||
|
// lookahead ramp in a single frame — a visible pop precisely at the
|
||||||
|
// transition. Hold the slope it had going into the boundary; it
|
||||||
|
// finished its build, and it stays there while it fades out. Within a
|
||||||
|
// section the slope is continuous, so the live value is correct there.
|
||||||
|
const outgoingFeatures = cue.atSectionStart
|
||||||
|
? { ...features, buildSlope: this._boundarySlope(cue) }
|
||||||
|
: features;
|
||||||
|
|
||||||
|
for (let slot = 0; slot < this._stackSize(previous); slot++) {
|
||||||
|
const spec = this._specFor(previous.sectionIndex, previous.variant, slot);
|
||||||
|
const layer = this._layerFor(previous.sectionIndex, previous.variant, slot);
|
||||||
|
layer.setParams(this._paramsAt(previous, slot, time, outgoingFeatures));
|
||||||
|
layer.opacity = slot === 0 ? 1 : spec.opacity;
|
||||||
|
layer.blend = slot === 0 ? 'normal' : spec.blend;
|
||||||
|
layer.setPalette(palette);
|
||||||
|
layer.setPersonality(this.look.personality);
|
||||||
|
layer.setFraming(outgoingFraming);
|
||||||
|
layers.push(layer);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for (let slot = 0; slot < this._stackSize(cue); slot++) {
|
||||||
|
const spec = this._specFor(cue.sectionIndex, cue.variant, slot);
|
||||||
|
const layer = this._layerFor(cue.sectionIndex, cue.variant, slot);
|
||||||
|
layer.setParams(this._paramsAt(cue, slot, time, features));
|
||||||
|
layer.opacity = (slot === 0 ? 1 : spec.opacity) * (fading ? eased : 1);
|
||||||
|
layer.blend = slot === 0 ? 'normal' : spec.blend;
|
||||||
|
layer.setPalette(palette);
|
||||||
|
layer.setPersonality(this.look.personality);
|
||||||
|
layer.setFraming(framing);
|
||||||
|
layers.push(layer);
|
||||||
|
}
|
||||||
|
|
||||||
|
this.state = {
|
||||||
|
sectionIndex: cue.sectionIndex,
|
||||||
|
shotIndex: cue.shotIndex,
|
||||||
|
shotCount: section.shots ? section.shots.length : 1,
|
||||||
|
variant: cue.variant,
|
||||||
|
kind: section.kind,
|
||||||
|
crossfade: fading ? eased : 0,
|
||||||
|
sceneName: this._specFor(cue.sectionIndex, cue.variant, 0).module.name,
|
||||||
|
buildSlope: features ? features.buildSlope || 0 : 0,
|
||||||
|
};
|
||||||
|
|
||||||
|
this.activeLayers = layers;
|
||||||
|
return layers;
|
||||||
|
}
|
||||||
|
|
||||||
|
_stackSize(cue) {
|
||||||
|
const section = this.look.sections[cue.sectionIndex];
|
||||||
|
const stack = (section.variants && section.variants[cue.variant]) || section.layers;
|
||||||
|
return stack.length;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Layers changed identity — the compositor needs the new list. */
|
||||||
|
layersChanged(previous) {
|
||||||
|
if (!previous || previous.length !== this.activeLayers.length) return true;
|
||||||
|
return this.activeLayers.some((l, i) => l !== previous[i]);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Invalidate caches for one section after an edit or reroll. */
|
||||||
|
invalidateSection(sectionIndex) {
|
||||||
|
for (const key of [...this.layerCache.keys()]) {
|
||||||
|
if (key.startsWith(`${sectionIndex}:`)) {
|
||||||
|
this.layerCache.get(key).dispose();
|
||||||
|
this.layerCache.delete(key);
|
||||||
|
this.driftPlans.delete(key);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// A reroll re-plans the section's shots, so the cue list is stale too.
|
||||||
|
this.cues = this._buildCues();
|
||||||
|
this._planFraming();
|
||||||
|
this._slopeCache = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
invalidateAll() {
|
||||||
|
this.dispose();
|
||||||
|
this.driftPlans.clear();
|
||||||
|
this.cues = this._buildCues();
|
||||||
|
this._planFraming();
|
||||||
|
this._slopeCache = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Create and compile every layer the look will ever show.
|
||||||
|
*
|
||||||
|
* Layers are otherwise built on first use, which means a shader compile on
|
||||||
|
* the frame of a cut — a visible hitch, and there are now many more cuts
|
||||||
|
* than there were sections. Paying for all of them once at load is cheaper
|
||||||
|
* than paying for one at every transition.
|
||||||
|
*/
|
||||||
|
prewarm(onLayer = null) {
|
||||||
|
for (const cue of this.cues) {
|
||||||
|
for (let slot = 0; slot < this._stackSize(cue); slot++) {
|
||||||
|
const layer = this._layerFor(cue.sectionIndex, cue.variant, slot);
|
||||||
|
if (onLayer) onLayer(layer);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Push a palette change through without rebuilding layers. */
|
||||||
|
setPalette(palette) {
|
||||||
|
this.look.palette = palette;
|
||||||
|
// The moved palette is memoised on the SHIFT, so a new base palette at
|
||||||
|
// an unchanged point in the arc would otherwise keep serving the old
|
||||||
|
// colours until the arc happened to move.
|
||||||
|
this._paletteKey = null;
|
||||||
|
for (const layer of this.layerCache.values()) layer.setPalette(palette);
|
||||||
|
}
|
||||||
|
}
|
||||||
438
flow-state/src/look/LookGenerator.js
Normal file
438
flow-state/src/look/LookGenerator.js
Normal file
@ -0,0 +1,438 @@
|
|||||||
|
// Turns a FeatureTrack into a complete LookSpec: palette, per-section scene
|
||||||
|
// assignments, parameter sets, and the post/feedback settings.
|
||||||
|
//
|
||||||
|
// Runs once per track. Deterministic in the seed, and the seed is derived from
|
||||||
|
// the decoded audio, so a given file always renders the same video.
|
||||||
|
|
||||||
|
import { Rng, hashSamples } from '../engine/rng.js';
|
||||||
|
import { AudioPalette, generateUsablePalette } from './palette.js';
|
||||||
|
import { scenes, scenesInFamily, FAMILIES } from '../scenes/registry.js';
|
||||||
|
import { sampleValues, defaultValues } from '../params/schema.js';
|
||||||
|
import { planShots } from './shots.js';
|
||||||
|
import { generatePersonality, sceneHonours, describePersonality } from './Personality.js';
|
||||||
|
import { deriveGrain, describeGrain, applyGrainToPost } from './grain.js';
|
||||||
|
import { pickDirector, directorByName } from './directors.js';
|
||||||
|
import { derivePaletteArc, describePaletteArc } from './paletteArc.js';
|
||||||
|
import { deriveFramingStyle, describeFraming } from './framing.js';
|
||||||
|
|
||||||
|
// Which families suit which section kind now comes from the track's DIRECTOR
|
||||||
|
// (look/directors.js) rather than from a constant here. The coupling it
|
||||||
|
// provides is the same — it is what stops a breakdown landing on a strobing
|
||||||
|
// glitch scene and an intro opening at full density — but which coupling a
|
||||||
|
// given track gets is a decision, not a fact about the program.
|
||||||
|
|
||||||
|
const KIND_ENERGY = {
|
||||||
|
intro: 0.25, build: 0.55, drop: 0.95, sustain: 0.6, breakdown: 0.25, outro: 0.2,
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parameter bias per section: the values scenes declare a `bias` key against.
|
||||||
|
*
|
||||||
|
* This is how a track's measured character reaches a scene's parameters without
|
||||||
|
* the scene knowing anything about audio. A dense, loud drop pushes `density`
|
||||||
|
* and `energy` up; a breakdown pulls them down. Seed variation still dominates,
|
||||||
|
* so two tracks with the same structure do not converge on the same look.
|
||||||
|
*/
|
||||||
|
function biasFor(section, summary) {
|
||||||
|
const kindEnergy = KIND_ENERGY[section.kind] ?? 0.5;
|
||||||
|
const measured = Math.min(1, section.energy / Math.max(1e-6, summary.meanLoudness * 1.6));
|
||||||
|
const energy = kindEnergy * 0.6 + measured * 0.4;
|
||||||
|
|
||||||
|
// 60bpm → 0, 180bpm → 1. Tempo, not energy, is what a viewer reads as
|
||||||
|
// "this is moving too fast for the song": a slow track can have a huge drop
|
||||||
|
// and still want scenes that drift. Motion used to be mostly energy with
|
||||||
|
// tempo as a small correction, which is why a 70bpm ballad got a drop
|
||||||
|
// biased to 0.9 motion and scenes that skittered over it.
|
||||||
|
const tempo = clamp01((summary.bpm - 60) / 120);
|
||||||
|
|
||||||
|
return {
|
||||||
|
energy,
|
||||||
|
density: Math.min(1, energy * 0.7 + section.flux * 1.2),
|
||||||
|
motion: clamp01(0.12 + tempo * 0.55 + energy * 0.28),
|
||||||
|
// Applied on top of every `rate: true` param, so absolute animation
|
||||||
|
// speed scales with the song rather than only its sampled position in
|
||||||
|
// a range. Bounded well short of a stop or a blur. See params/schema.js.
|
||||||
|
rateScale: 0.45 + tempo * 0.95,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
const clamp01 = (x) => Math.max(0, Math.min(1, x));
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Scenes eligible for a section kind, weighted by how well the family fits.
|
||||||
|
*
|
||||||
|
* `signature` is the track's personality signature, and it is a hard filter
|
||||||
|
* rather than a weight: a scene with no way to express what the track is built
|
||||||
|
* on is not a worse choice, it is the shot that was clearly filmed somewhere
|
||||||
|
* else. See look/Personality.js.
|
||||||
|
*/
|
||||||
|
function candidatesForKind(kind, used, signature = [], director) {
|
||||||
|
const families = director.families[kind] || Object.keys(FAMILIES);
|
||||||
|
const candidates = [];
|
||||||
|
for (const family of families) {
|
||||||
|
const inFamily = scenesInFamily(family)
|
||||||
|
.filter((m) => m.role !== 'accent' && sceneHonours(m, signature));
|
||||||
|
// Weight by family preference order, and push down anything already
|
||||||
|
// used so a five-section track doesn't show one scene five times.
|
||||||
|
const weight = families.length - families.indexOf(family);
|
||||||
|
for (const scene of inFamily) {
|
||||||
|
candidates.push({ scene, weight: weight * (used.has(scene.name) ? 0.15 : 1) });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (!candidates.length) {
|
||||||
|
// Every family for this kind was emptied by the signature filter. Widen
|
||||||
|
// to the whole library, still honouring the signature; only if that is
|
||||||
|
// empty too does the personality lose and the video keep its scenes.
|
||||||
|
const anywhere = scenes.filter((m) => m.role !== 'accent' && sceneHonours(m, signature));
|
||||||
|
const pool = anywhere.length ? anywhere : scenes.filter((m) => m.role !== 'accent');
|
||||||
|
return pool.map((scene) => ({ scene, weight: 1 }));
|
||||||
|
}
|
||||||
|
return candidates;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* How many stage visuals a kind rotates between. Busy material takes more.
|
||||||
|
*
|
||||||
|
* Sized against the library rather than picked out of the air: a kind draws
|
||||||
|
* from three families, which is seven to nine non-accent scenes, so a roster of
|
||||||
|
* four still leaves the weighting room to avoid what other kinds already took.
|
||||||
|
* Variants a section never reaches cost nothing — layers are built per cue, so
|
||||||
|
* only the ones its shots actually show are ever compiled.
|
||||||
|
*/
|
||||||
|
function rosterSizeFor(kind) {
|
||||||
|
return (KIND_ENERGY[kind] ?? 0.5) > 0.5 ? 4 : 3;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Scenes are chosen per section KIND, not per section — and a kind gets a
|
||||||
|
* ROSTER of two or three, not one.
|
||||||
|
*
|
||||||
|
* All of a track's drops therefore cut between the same small set of visuals,
|
||||||
|
* all its breakdowns between another, and the video acquires an identity
|
||||||
|
* instead of reading as a shuffle. The first entry is the anchor: it opens
|
||||||
|
* every section of that kind and comes back most often, so the rotation reads
|
||||||
|
* as one idea with variations rather than as three unrelated scenes.
|
||||||
|
*
|
||||||
|
* Variation between two sections of the same kind comes from their parameter
|
||||||
|
* sets, from where their shots fall, and from the arc driver's drift.
|
||||||
|
*/
|
||||||
|
function assignRostersByKind(sections, rng, signature = [], director) {
|
||||||
|
const byKind = new Map();
|
||||||
|
const used = new Set();
|
||||||
|
|
||||||
|
const kinds = [...new Set(sections.map((s) => s.kind))];
|
||||||
|
// Order matters for variety: assign the high-impact kinds first so they get
|
||||||
|
// first pick of the library rather than whatever is left.
|
||||||
|
const priority = ['drop', 'sustain', 'build', 'breakdown', 'intro', 'outro'];
|
||||||
|
kinds.sort((a, b) => priority.indexOf(a) - priority.indexOf(b));
|
||||||
|
|
||||||
|
for (const kind of kinds) {
|
||||||
|
const roster = [];
|
||||||
|
const size = rosterSizeFor(kind);
|
||||||
|
|
||||||
|
for (let slot = 0; slot < size; slot++) {
|
||||||
|
const pool = candidatesForKind(kind, used, signature, director)
|
||||||
|
.filter((c) => !roster.includes(c.scene))
|
||||||
|
.map((c) => ({
|
||||||
|
scene: c.scene,
|
||||||
|
// Companions stay in the anchor's family where possible: a
|
||||||
|
// cut inside a section should change the image, not the
|
||||||
|
// whole visual language.
|
||||||
|
weight: c.weight * (roster.length && c.scene.family === roster[0].family ? 3 : 1),
|
||||||
|
}));
|
||||||
|
if (!pool.length) break;
|
||||||
|
|
||||||
|
const chosen = rng.pickWeighted(pool.map((c) => c.scene), pool.map((c) => c.weight));
|
||||||
|
roster.push(chosen);
|
||||||
|
used.add(chosen.name);
|
||||||
|
}
|
||||||
|
|
||||||
|
byKind.set(kind, roster.length ? roster : [scenes[0]]);
|
||||||
|
}
|
||||||
|
return byKind;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Post-processing and feedback derived from track character.
|
||||||
|
* Ambient material gets more feedback and bloom; dense club material gets
|
||||||
|
* tighter, punchier settings.
|
||||||
|
*
|
||||||
|
* Grain is NOT decided here — see look/grain.js. `post.grain` carries only the
|
||||||
|
* amount for the current frame, which the Show multiplies by the grain
|
||||||
|
* envelope, so a track can be clean, permanently dirty, or anything between.
|
||||||
|
*/
|
||||||
|
function derivePost(summary, rng, grain) {
|
||||||
|
const bright = summary.meanCentroid;
|
||||||
|
const noisy = Math.min(1, summary.meanFlatness * 3);
|
||||||
|
const dynamic = Math.min(1, summary.dynamicRange);
|
||||||
|
|
||||||
|
return {
|
||||||
|
post: {
|
||||||
|
bloom: 0.25 + (1 - noisy) * 0.35 + rng.range(-0.05, 0.05),
|
||||||
|
bloomThreshold: 0.45 + bright * 0.25,
|
||||||
|
bloomKnee: 0.25,
|
||||||
|
chroma: 0.05 + noisy * 0.25 + rng.range(0, 0.08),
|
||||||
|
...applyGrainToPost(grain, {}),
|
||||||
|
vignette: 0.25 + (1 - bright) * 0.25,
|
||||||
|
contrast: 1.0 + dynamic * 0.15,
|
||||||
|
saturation: 1.0 + (1 - noisy) * 0.25,
|
||||||
|
lift: 0.0,
|
||||||
|
exposure: 1.0,
|
||||||
|
},
|
||||||
|
feedback: {
|
||||||
|
// Dynamic, spacious material tolerates long trails; dense material
|
||||||
|
// turns to smear, so it gets much less.
|
||||||
|
amount: Math.min(0.75, 0.15 + dynamic * 0.5),
|
||||||
|
decay: 0.86 + dynamic * 0.08,
|
||||||
|
zoom: 1.0 + rng.range(-0.006, 0.006),
|
||||||
|
rotate: rng.range(-0.004, 0.004),
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* One layer stack: a background scene, sometimes a second scene composited over
|
||||||
|
* it, sometimes an accent on top of that.
|
||||||
|
*
|
||||||
|
* Three deliberately different jobs:
|
||||||
|
*
|
||||||
|
* background — the shot. Always present, always opaque.
|
||||||
|
* overlay — a SECOND full scene at partial opacity. Not always: this is the
|
||||||
|
* variation valve, and a stack that always doubled up would read
|
||||||
|
* as permanently cluttered rather than as occasionally layered.
|
||||||
|
* Drawn from a different family so the two images argue instead
|
||||||
|
* of blurring, and kept off scenes that are already busy.
|
||||||
|
* accent — the depth pass. Mostly-empty by design (role: 'accent'),
|
||||||
|
* additive, low opacity.
|
||||||
|
*
|
||||||
|
* Quiet material mostly goes without either — an intro is supposed to be sparse.
|
||||||
|
*/
|
||||||
|
function buildStack(module, accentRoster, overlayRoster, bias, rng, temperament) {
|
||||||
|
const layers = [{
|
||||||
|
module,
|
||||||
|
params: sampleValues(module, rng, bias, temperament),
|
||||||
|
seed: rng.int(0, 0x7fffffff),
|
||||||
|
blend: 'normal',
|
||||||
|
opacity: 1,
|
||||||
|
}];
|
||||||
|
|
||||||
|
// --- overlay --------------------------------------------------------
|
||||||
|
// Roughly a third of stacks on busy material, rarely on quiet material, and
|
||||||
|
// never on a background that is itself a full-frame glitch — two competing
|
||||||
|
// corruption passes is noise, not depth.
|
||||||
|
const overlayChance = module.family === 'glitch'
|
||||||
|
? 0.05
|
||||||
|
: 0.12 + bias.energy * 0.35 + (temperament ? Math.max(0, temperament.detail) * 0.2 : 0);
|
||||||
|
|
||||||
|
const overlays = overlayRoster.filter((m) => m.family !== module.family && m.name !== module.name);
|
||||||
|
if (overlays.length && rng.bool(Math.min(0.6, overlayChance))) {
|
||||||
|
const overlay = rng.pick(overlays);
|
||||||
|
// Screen and add keep the background readable underneath; softlight and
|
||||||
|
// overlay tint it instead. All four preserve the shot; 'normal' would
|
||||||
|
// simply replace it, which is what the shot cut is for.
|
||||||
|
const blend = rng.pickWeighted(['screen', 'add', 'softlight', 'overlay'], [3, 2, 2, 1]);
|
||||||
|
layers.push({
|
||||||
|
module: overlay,
|
||||||
|
params: sampleValues(overlay, rng.fork(`overlay:${overlay.name}`), {
|
||||||
|
// An overlay reads as texture over the shot, so it is sampled
|
||||||
|
// sparser and calmer than it would be as a background.
|
||||||
|
...bias,
|
||||||
|
density: Math.max(0, bias.density - 0.25),
|
||||||
|
energy: Math.max(0, bias.energy - 0.2),
|
||||||
|
}, temperament),
|
||||||
|
seed: rng.int(0, 0x7fffffff),
|
||||||
|
blend,
|
||||||
|
opacity: blend === 'add' ? rng.range(0.18, 0.42) : rng.range(0.25, 0.55),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- accent ---------------------------------------------------------
|
||||||
|
if (accentRoster.length && rng.bool(bias.energy * 0.8)) {
|
||||||
|
const eligible = accentRoster.filter((m) => m.family !== module.family);
|
||||||
|
const accent = rng.pick(eligible.length ? eligible : accentRoster);
|
||||||
|
layers.push({
|
||||||
|
module: accent,
|
||||||
|
params: sampleValues(accent, rng.fork('accent'), bias, temperament),
|
||||||
|
seed: rng.int(0, 0x7fffffff),
|
||||||
|
blend: rng.pickWeighted(['add', 'screen'], [2, 1]),
|
||||||
|
opacity: rng.range(0.18, 0.5),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return layers;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @param {FeatureTrack} track
|
||||||
|
* @param {object} options
|
||||||
|
* @returns {object} LookSpec
|
||||||
|
*/
|
||||||
|
export function generateLook(track, { seed = null, samples = null, overrides = null } = {}) {
|
||||||
|
const resolvedSeed = seed !== null
|
||||||
|
? seed >>> 0
|
||||||
|
: samples ? hashSamples(samples) : 0x9e3779b9;
|
||||||
|
|
||||||
|
const rng = new Rng(resolvedSeed);
|
||||||
|
const summary = track.summary;
|
||||||
|
|
||||||
|
const paletteSource = new AudioPalette(summary, rng.fork('palette'));
|
||||||
|
const palette = generateUsablePalette(paletteSource, 6);
|
||||||
|
|
||||||
|
// The production design, decided before a single scene is cast — casting
|
||||||
|
// depends on it. See look/Personality.js.
|
||||||
|
const personality = generatePersonality(summary, rng.fork('personality'), (signature) =>
|
||||||
|
scenes.filter((m) => m.role !== 'accent' && sceneHonours(m, signature)).length);
|
||||||
|
|
||||||
|
// The track's point of view about what a song looks like. Cast before any
|
||||||
|
// scene is, because it decides which scenes are even candidates.
|
||||||
|
const director = pickDirector(summary, rng.fork('director'));
|
||||||
|
|
||||||
|
const rosterByKind = assignRostersByKind(
|
||||||
|
track.sections, rng.fork('scenes'), personality.signature, director);
|
||||||
|
// The grain treatment: usually none, and when present described rather than
|
||||||
|
// dialled. See look/grain.js.
|
||||||
|
const grain = deriveGrain(summary, rng.fork('grain'));
|
||||||
|
// How the palette MOVES over the track. See look/paletteArc.js.
|
||||||
|
const paletteArc = derivePaletteArc(summary, rng.fork('paletteArc'));
|
||||||
|
// Whether shots change SIZE at the cut, and how boldly. See look/framing.js.
|
||||||
|
const framing = deriveFramingStyle(summary, rng.fork('framing'));
|
||||||
|
const { post, feedback } = derivePost(summary, rng.fork('post'), grain);
|
||||||
|
|
||||||
|
// Scenes that declare role 'accent' composite over a background rather than
|
||||||
|
// being one — most of their frame is empty by design. They are never chosen
|
||||||
|
// as a section's primary scene.
|
||||||
|
// Accents honour the signature too where they can. If none can, the track
|
||||||
|
// goes without depth layers rather than putting an off-design element into
|
||||||
|
// every stack.
|
||||||
|
const accentRoster = scenes.filter((m) => m.role === 'accent'
|
||||||
|
&& sceneHonours(m, personality.signature));
|
||||||
|
|
||||||
|
// Scenes eligible to be composited OVER a background. Same casting rule as
|
||||||
|
// everything else — an overlay is on screen as much as the shot under it,
|
||||||
|
// so an off-design one would be just as visible.
|
||||||
|
const overlayRoster = scenes.filter((m) => m.role !== 'accent'
|
||||||
|
&& sceneHonours(m, personality.signature));
|
||||||
|
|
||||||
|
const sections = track.sections.map((section) => {
|
||||||
|
const roster = rosterByKind.get(section.kind) || [scenes[0]];
|
||||||
|
const sectionRng = rng.fork(`section:${section.index}:${roster[0].name}`);
|
||||||
|
const bias = biasFor(section, summary);
|
||||||
|
|
||||||
|
const variants = roster.map((module, v) => buildStack(
|
||||||
|
module, accentRoster, overlayRoster, bias,
|
||||||
|
sectionRng.fork(`variant:${section.index}:${v}`), personality.temperament,
|
||||||
|
));
|
||||||
|
|
||||||
|
const shots = planShots(
|
||||||
|
section, track, bias, variants.length, sectionRng.fork(`shots:${section.index}`),
|
||||||
|
);
|
||||||
|
|
||||||
|
return {
|
||||||
|
index: section.index,
|
||||||
|
kind: section.kind,
|
||||||
|
startFrame: section.startFrame,
|
||||||
|
endFrame: section.endFrame,
|
||||||
|
start: section.start,
|
||||||
|
end: section.end,
|
||||||
|
locked: false,
|
||||||
|
bias,
|
||||||
|
variants,
|
||||||
|
shots,
|
||||||
|
// The anchor stack, aliased. Everything that predates shots — the
|
||||||
|
// param panel, presets, the checks — edits a section through this,
|
||||||
|
// and it is the same object the first variant holds.
|
||||||
|
layers: variants[0],
|
||||||
|
};
|
||||||
|
});
|
||||||
|
|
||||||
|
const look = {
|
||||||
|
seed: resolvedSeed,
|
||||||
|
palette,
|
||||||
|
personality,
|
||||||
|
paletteScheme: paletteSource.lastScheme,
|
||||||
|
director: director.name,
|
||||||
|
paletteArc,
|
||||||
|
framing,
|
||||||
|
grain,
|
||||||
|
post,
|
||||||
|
feedback,
|
||||||
|
sections,
|
||||||
|
summary,
|
||||||
|
};
|
||||||
|
|
||||||
|
return overrides ? applyOverrides(look, overrides) : look;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Re-roll one section, leaving everything else — and locked sections — alone. */
|
||||||
|
export function rerollSection(look, track, sectionIndex, salt = 0) {
|
||||||
|
const section = look.sections[sectionIndex];
|
||||||
|
if (!section || section.locked) return look;
|
||||||
|
|
||||||
|
const rng = new Rng((look.seed ^ (sectionIndex * 0x9e3779b9) ^ (salt * 0x85ebca6b)) >>> 0);
|
||||||
|
const signature = (look.personality && look.personality.signature) || [];
|
||||||
|
const families = directorByName(look.director).families[section.kind] || Object.keys(FAMILIES);
|
||||||
|
let candidates = families.flatMap((f) => scenesInFamily(f))
|
||||||
|
.filter((m) => m.role !== 'accent' && sceneHonours(m, signature));
|
||||||
|
if (!candidates.length) {
|
||||||
|
candidates = scenes.filter((m) => m.role !== 'accent' && sceneHonours(m, signature));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Re-roll the whole roster, not just the anchor: the section's shots cut
|
||||||
|
// between all of them, so replacing one would leave the section half old.
|
||||||
|
const size = Math.min(rosterSizeFor(section.kind), Math.max(1, candidates.length));
|
||||||
|
const roster = [];
|
||||||
|
while (roster.length < size) {
|
||||||
|
const pool = candidates.filter((m) => !roster.includes(m));
|
||||||
|
if (!pool.length) break;
|
||||||
|
roster.push(rng.pick(pool));
|
||||||
|
}
|
||||||
|
if (!roster.length) roster.push(scenes[0]);
|
||||||
|
|
||||||
|
const accentRoster = scenes.filter((m) => m.role === 'accent');
|
||||||
|
const overlayRoster = scenes.filter((m) => m.role !== 'accent' && sceneHonours(m, signature));
|
||||||
|
section.variants = roster.map((module, v) => buildStack(
|
||||||
|
module, accentRoster, overlayRoster, section.bias, rng.fork(`variant:${v}`),
|
||||||
|
look.personality && look.personality.temperament,
|
||||||
|
));
|
||||||
|
section.shots = planShots(
|
||||||
|
section, track, section.bias, section.variants.length, rng.fork('shots'),
|
||||||
|
);
|
||||||
|
section.layers = section.variants[0];
|
||||||
|
return look;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Reroll the whole track with a new seed, preserving locked sections. */
|
||||||
|
export function rerollLook(look, track, newSeed) {
|
||||||
|
const locked = new Map();
|
||||||
|
look.sections.forEach((s) => { if (s.locked) locked.set(s.index, s); });
|
||||||
|
|
||||||
|
const next = generateLook(track, { seed: newSeed >>> 0 });
|
||||||
|
next.sections.forEach((s, i) => {
|
||||||
|
if (locked.has(i)) next.sections[i] = locked.get(i);
|
||||||
|
});
|
||||||
|
return next;
|
||||||
|
}
|
||||||
|
|
||||||
|
function applyOverrides(look, overrides) {
|
||||||
|
if (overrides.palette) look.palette = overrides.palette;
|
||||||
|
if (overrides.post) look.post = { ...look.post, ...overrides.post };
|
||||||
|
if (overrides.feedback) look.feedback = { ...look.feedback, ...overrides.feedback };
|
||||||
|
if (overrides.sections) {
|
||||||
|
overrides.sections.forEach((o, i) => {
|
||||||
|
if (!look.sections[i]) return;
|
||||||
|
if (o.locked !== undefined) look.sections[i].locked = o.locked;
|
||||||
|
if (o.params) Object.assign(look.sections[i].layers[0].params, o.params);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return look;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Compact description, used by the HUD and by check output. */
|
||||||
|
export function describeLook(look) {
|
||||||
|
const kinds = look.sections.map((s) => `${s.kind}:${s.layers[0].module.name}`);
|
||||||
|
return `seed ${look.seed.toString(16)} · ${look.director} · ${look.paletteScheme} · ` +
|
||||||
|
`${describePersonality(look.personality)} · ${describeGrain(look.grain)} · ` +
|
||||||
|
`${describePaletteArc(look.paletteArc)} · ${describeFraming(look.framing)} · ` +
|
||||||
|
`${[...new Set(kinds)].join(', ')}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
export { defaultValues };
|
||||||
253
flow-state/src/look/Personality.js
Normal file
253
flow-state/src/look/Personality.js
Normal file
@ -0,0 +1,253 @@
|
|||||||
|
// The track's production design.
|
||||||
|
//
|
||||||
|
// A music video is not held together by its cuts. It is held together by the
|
||||||
|
// fact that every shot was filmed in the same location, with the same actors,
|
||||||
|
// the same camera operator and the same art direction. Cut between two shots of
|
||||||
|
// that and it reads as one piece even when the framing changes completely.
|
||||||
|
//
|
||||||
|
// Nothing in this project had an equivalent. Sections shared a palette and a
|
||||||
|
// post grade, and past that every scene was a separate short film. This module
|
||||||
|
// is the missing layer: one procedurally generated PERSONALITY per track, in
|
||||||
|
// four traits that map onto the four things a production shares.
|
||||||
|
//
|
||||||
|
// shape — the actors. A signature form: how many sides, how round, how
|
||||||
|
// elongated, at what tilt. Scenes that draw discrete elements stamp
|
||||||
|
// this form instead of whatever primitive they would have used.
|
||||||
|
// camera — the operator. A drift direction, a sway, a slow spin, a breathing
|
||||||
|
// zoom. Applied to the coordinate a scene works in, so every scene
|
||||||
|
// is filmed by the same hand.
|
||||||
|
// space — the location. A horizon height, a depth falloff, a background
|
||||||
|
// wash direction. Scenes that have a sense of place share one.
|
||||||
|
// style — the art direction. Line weight, edge softness, texture, and how
|
||||||
|
// many times the frame is folded.
|
||||||
|
//
|
||||||
|
// Plus a fifth thing that is not a trait and is not declared by anyone: the
|
||||||
|
// TEMPERAMENT. Traits decide what a track looks like; temperament decides how
|
||||||
|
// hard it commits. It is the track's hand on every scene's parameter dials, and
|
||||||
|
// it exists because section bias alone is nearly identical between two tracks'
|
||||||
|
// drops — so one scene cast in two videos sampled around the same centre both
|
||||||
|
// times and the videos looked like each other. Temperament is per track and
|
||||||
|
// pushes those samples apart. See params/schema.js sampleValues.
|
||||||
|
//
|
||||||
|
// A scene declares which traits it can honour. Each track picks a SIGNATURE of
|
||||||
|
// one or two traits, and a scene that does not honour all of them is
|
||||||
|
// disqualified from that track — the library shrinks per track, on purpose. A
|
||||||
|
// scene with no way to express a hexagon should not appear in the hexagon
|
||||||
|
// video; it would be the shot that was clearly filmed somewhere else.
|
||||||
|
//
|
||||||
|
// Everything here is seeded off the look seed, so a track's personality is as
|
||||||
|
// reproducible as everything else.
|
||||||
|
|
||||||
|
export const TRAITS = ['shape', 'camera', 'space', 'style'];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Traits eligible to be a track's signature, and how often.
|
||||||
|
*
|
||||||
|
* `shape` and `space` carry the most identity — they are the ones a viewer can
|
||||||
|
* actually name on a second watch — so they are the ones a signature is built
|
||||||
|
* around. `camera` and `style` are near-universally supported and read as
|
||||||
|
* treatment rather than as subject, so they join a signature but rarely define
|
||||||
|
* one alone.
|
||||||
|
*/
|
||||||
|
const SIGNATURE_WEIGHTS = { shape: 4, space: 3, camera: 2, style: 2 };
|
||||||
|
|
||||||
|
/** Minimum scenes that must survive the signature filter for it to be usable. */
|
||||||
|
export const MIN_ELIGIBLE_SCENES = 6;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Generate the personality.
|
||||||
|
*
|
||||||
|
* Trait VALUES lean on what was measured in the audio — a bright, noisy track
|
||||||
|
* gets sharper lines and more texture; a slow one gets a lazier camera — but
|
||||||
|
* the seed dominates, so two tracks with similar statistics still look like
|
||||||
|
* different productions.
|
||||||
|
*
|
||||||
|
* @param {object} summary FeatureTrack summary
|
||||||
|
* @param {Rng} rng
|
||||||
|
* @param {(traits: string[]) => number} countEligible
|
||||||
|
* How many scenes would survive a given signature. Injected rather than
|
||||||
|
* imported so this module never has to know the registry exists.
|
||||||
|
*/
|
||||||
|
export function generatePersonality(summary, rng, countEligible = null) {
|
||||||
|
const bright = summary.meanCentroid;
|
||||||
|
const noisy = Math.min(1, summary.meanFlatness * 3);
|
||||||
|
const fast = Math.min(1, Math.max(0, (summary.bpm - 80) / 80));
|
||||||
|
const loud = Math.min(1, (summary.dynamicRange ?? 0.5) + (summary.meanLoudness ?? 0.3));
|
||||||
|
|
||||||
|
const shape = {
|
||||||
|
// 0 sides means round. Everything else is a polygon the whole track
|
||||||
|
// shares — the single most recognisable thing here.
|
||||||
|
sides: rng.pickWeighted([0, 3, 4, 5, 6, 8], [3, 2, 3, 2, 3, 1]),
|
||||||
|
roundness: rng.range(0.05, 0.5),
|
||||||
|
elongation: rng.range(0.85, 1.45),
|
||||||
|
tilt: rng.range(0, Math.PI),
|
||||||
|
};
|
||||||
|
|
||||||
|
const camera = {
|
||||||
|
driftAngle: rng.range(0, Math.PI * 2),
|
||||||
|
// A slow track should not be filmed from a moving car.
|
||||||
|
driftRate: rng.range(0.01, 0.06) * (0.6 + fast * 0.8),
|
||||||
|
sway: rng.range(0.0, 0.06),
|
||||||
|
swayRate: rng.range(0.05, 0.22),
|
||||||
|
spin: rng.range(-0.05, 0.05),
|
||||||
|
// Breathing is locked to the bar, so it is the one camera move that
|
||||||
|
// reads as musical rather than as drifting.
|
||||||
|
breathe: rng.range(0.0, 0.05),
|
||||||
|
};
|
||||||
|
|
||||||
|
const space = {
|
||||||
|
horizon: rng.range(0.32, 0.62),
|
||||||
|
depth: rng.range(0.2, 0.9),
|
||||||
|
washAngle: rng.range(0, Math.PI * 2),
|
||||||
|
wash: rng.range(0.1, 0.5),
|
||||||
|
};
|
||||||
|
|
||||||
|
const style = {
|
||||||
|
lineWeight: 0.4 + bright * 0.4 + rng.range(-0.15, 0.25),
|
||||||
|
softness: 0.25 + (1 - bright) * 0.4 + rng.range(-0.1, 0.2),
|
||||||
|
// Surface grain is a texture trait, not a default, and most tracks have
|
||||||
|
// none at all. Every scene adds sigGrain and the grade can add its own,
|
||||||
|
// so anything short of a hard gate here reads as "grainy by default" —
|
||||||
|
// which is exactly what it read as when this was a floor of ~0.12 and
|
||||||
|
// then again when it was a small unconditional amount. A scene can also
|
||||||
|
// opt out entirely with `texture: 0` in its module.
|
||||||
|
texture: rng.bool(0.25 + noisy * 0.4) ? noisy * 0.35 + rng.range(0.02, 0.12) : 0,
|
||||||
|
// Fold counts stay low and are usually off. Symmetry is the fastest way
|
||||||
|
// to make a library look like one series and also the fastest way to
|
||||||
|
// make every track look like a screensaver.
|
||||||
|
symmetry: rng.pickWeighted([1, 1, 2, 3, 4, 6], [6, 4, 2, 2, 2, 1]),
|
||||||
|
};
|
||||||
|
|
||||||
|
// How hard this track pushes every scene it casts. Deliberately wide, and
|
||||||
|
// deliberately not derived from the section: two tracks must be able to
|
||||||
|
// disagree about what "a drop" means.
|
||||||
|
const temperament = {
|
||||||
|
// Up or down on the energy/density dials.
|
||||||
|
intensity: rng.range(-0.85, 0.85) * (0.5 + loud * 0.9),
|
||||||
|
// Up or down on anything that moves.
|
||||||
|
pace: rng.range(-0.8, 0.8) * (0.55 + fast * 0.8),
|
||||||
|
// Fine and busy, or few and large. Independent of loudness on purpose —
|
||||||
|
// a quiet track can be intricate and a loud one can be blunt.
|
||||||
|
detail: rng.range(-0.6, 0.6),
|
||||||
|
// How far toward the ends of a range this track is willing to sample.
|
||||||
|
// The single most effective knob against "every video looks average",
|
||||||
|
// so the floor sits well above timid: a track at 0.25 sampled almost
|
||||||
|
// uniformly and produced the library's average look, and enough tracks
|
||||||
|
// did that to make the average look like the house style.
|
||||||
|
extremity: rng.range(0.45, 1.0),
|
||||||
|
};
|
||||||
|
|
||||||
|
const signature = pickSignature(rng, countEligible);
|
||||||
|
|
||||||
|
return { signature, shape, camera, space, style, temperament };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Choose the one or two traits this track is BUILT on.
|
||||||
|
*
|
||||||
|
* Two is the target: one trait alone is not enough to recognise, and three
|
||||||
|
* disqualifies most of the library. If the pair leaves too few scenes to build
|
||||||
|
* rosters from, fall back to the stronger of the two rather than shipping a
|
||||||
|
* track whose every section is forced onto the same two scenes.
|
||||||
|
*/
|
||||||
|
function pickSignature(rng, countEligible) {
|
||||||
|
const primary = rng.pickWeighted(TRAITS, TRAITS.map((t) => SIGNATURE_WEIGHTS[t]));
|
||||||
|
const rest = TRAITS.filter((t) => t !== primary);
|
||||||
|
const secondary = rng.pickWeighted(rest, rest.map((t) => SIGNATURE_WEIGHTS[t]));
|
||||||
|
|
||||||
|
const pair = [primary, secondary];
|
||||||
|
if (!countEligible || countEligible(pair) >= MIN_ELIGIBLE_SCENES) return pair;
|
||||||
|
|
||||||
|
const single = [primary];
|
||||||
|
if (countEligible(single) >= MIN_ELIGIBLE_SCENES) return single;
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Does a scene honour everything this track is built on? */
|
||||||
|
export function sceneHonours(module, signature) {
|
||||||
|
const traits = module.traits || [];
|
||||||
|
return signature.every((t) => traits.includes(t));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Flatten to the uniform values the shader contract expects.
|
||||||
|
*
|
||||||
|
* Neutral defaults matter: a layer with no personality attached must render
|
||||||
|
* exactly what it rendered before this existed, because the range sweeps and
|
||||||
|
* the library regression checks build layers directly and would otherwise all
|
||||||
|
* shift at once.
|
||||||
|
*/
|
||||||
|
export function signatureUniforms(personality, module = null) {
|
||||||
|
if (!personality) return NEUTRAL_UNIFORMS;
|
||||||
|
const { shape, camera, space, style } = personality;
|
||||||
|
// A scene may scale — or refuse — the track's surface grain. A clean vector
|
||||||
|
// look has no business being speckled just because the track is gritty.
|
||||||
|
const textureAffinity = module && module.texture !== undefined ? module.texture : 1;
|
||||||
|
return {
|
||||||
|
u_sigSides: shape.sides,
|
||||||
|
u_sigRound: shape.roundness,
|
||||||
|
u_sigElong: shape.elongation,
|
||||||
|
u_sigTilt: shape.tilt,
|
||||||
|
|
||||||
|
u_sigDrift: [Math.cos(camera.driftAngle) * camera.driftRate,
|
||||||
|
Math.sin(camera.driftAngle) * camera.driftRate],
|
||||||
|
u_sigSway: camera.sway,
|
||||||
|
u_sigSwayRate: camera.swayRate,
|
||||||
|
u_sigSpin: camera.spin,
|
||||||
|
u_sigBreathe: camera.breathe,
|
||||||
|
|
||||||
|
u_sigHorizon: space.horizon,
|
||||||
|
u_sigDepth: space.depth,
|
||||||
|
u_sigWash: [Math.cos(space.washAngle) * space.wash,
|
||||||
|
Math.sin(space.washAngle) * space.wash],
|
||||||
|
|
||||||
|
u_sigLine: style.lineWeight,
|
||||||
|
u_sigSoft: style.softness,
|
||||||
|
u_sigTexture: style.texture * textureAffinity,
|
||||||
|
u_sigFold: style.symmetry,
|
||||||
|
|
||||||
|
// Framing is per shot, not per track: the arc driver overwrites these
|
||||||
|
// every frame. Neutral here so a layer built without one is unframed.
|
||||||
|
u_sigFrameScale: 1,
|
||||||
|
u_sigFrameShift: [0, 0],
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export const NEUTRAL_UNIFORMS = {
|
||||||
|
u_sigSides: 0,
|
||||||
|
u_sigRound: 0.25,
|
||||||
|
u_sigElong: 1,
|
||||||
|
u_sigTilt: 0,
|
||||||
|
u_sigDrift: [0, 0],
|
||||||
|
u_sigSway: 0,
|
||||||
|
u_sigSwayRate: 0.1,
|
||||||
|
u_sigSpin: 0,
|
||||||
|
u_sigBreathe: 0,
|
||||||
|
u_sigHorizon: 0.5,
|
||||||
|
u_sigDepth: 0,
|
||||||
|
u_sigWash: [0, 0],
|
||||||
|
u_sigLine: 0.5,
|
||||||
|
u_sigSoft: 0.5,
|
||||||
|
u_sigTexture: 0,
|
||||||
|
u_sigFold: 1,
|
||||||
|
u_sigFrameScale: 1,
|
||||||
|
u_sigFrameShift: [0, 0],
|
||||||
|
};
|
||||||
|
|
||||||
|
const SHAPE_NAMES = { 0: 'round', 3: 'triangular', 4: 'square', 5: 'pentagonal', 6: 'hexagonal', 8: 'octagonal' };
|
||||||
|
|
||||||
|
/** One line for the HUD, the look panel and check output. */
|
||||||
|
export function describePersonality(personality) {
|
||||||
|
if (!personality) return 'no personality';
|
||||||
|
const { signature, shape, style } = personality;
|
||||||
|
const parts = [
|
||||||
|
`on ${signature.length ? signature.join('+') : 'nothing'}`,
|
||||||
|
SHAPE_NAMES[shape.sides] || `${shape.sides}-sided`,
|
||||||
|
];
|
||||||
|
if (style.symmetry > 1) parts.push(`${style.symmetry}-fold`);
|
||||||
|
if (personality.temperament) {
|
||||||
|
const t = personality.temperament;
|
||||||
|
parts.push(`${t.intensity >= 0 ? 'hot' : 'cool'} ${t.extremity.toFixed(2)} bold`);
|
||||||
|
}
|
||||||
|
return parts.join(' · ');
|
||||||
|
}
|
||||||
148
flow-state/src/look/directors.js
Normal file
148
flow-state/src/look/directors.js
Normal file
@ -0,0 +1,148 @@
|
|||||||
|
// Which families answer which section kind — as a per-track CHOICE rather than
|
||||||
|
// as a constant.
|
||||||
|
//
|
||||||
|
// This was a module-level table in LookGenerator, identical for every track ever
|
||||||
|
// generated: an intro was always minimal/flow/organic, a drop always
|
||||||
|
// geometric/glitch/structural, and intro and outro were literally the same list.
|
||||||
|
// Every video therefore made the same genre decisions before a single seeded
|
||||||
|
// draw happened, which is a large source of cross-track sameness hiding inside
|
||||||
|
// something that looks like configuration.
|
||||||
|
//
|
||||||
|
// Measured, the cost was concrete: across twelve tracks, twenty-nine of
|
||||||
|
// forty-two scenes were cast in none of them. The library was not too small —
|
||||||
|
// most of it was unreachable.
|
||||||
|
//
|
||||||
|
// A director is one coherent point of view about what a song looks like. Not a
|
||||||
|
// shuffle of families: each mapping is internally consistent, and the ordering
|
||||||
|
// within a kind matters because the first family is weighted three times the
|
||||||
|
// third. Two tracks with different directors disagree about what a drop IS,
|
||||||
|
// which is the level at which videos should differ.
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Families a QUIET section is allowed to draw from, whatever the director
|
||||||
|
* thinks. Phase 7 has enforced this since minimal existed, and the first draft
|
||||||
|
* of this file broke it — `corrupt` opened on glitch and `geometer` opened on
|
||||||
|
* geometric, on the theory that a point of view should apply everywhere.
|
||||||
|
*
|
||||||
|
* It should not. An intro that opens on a strobing scene and a breakdown that
|
||||||
|
* answers a lull with a dense pattern are not bold, they are the two specific
|
||||||
|
* mistakes the family coupling was introduced to prevent, and a viewer meets
|
||||||
|
* them within fifteen seconds of pressing play. So intro, breakdown and outro
|
||||||
|
* are off limits to the loud families for every director.
|
||||||
|
*
|
||||||
|
* The identity lives in build, drop and sustain — half the kinds, and the half
|
||||||
|
* anyone remembers — plus which of the restful families a director leads with
|
||||||
|
* when it is being quiet.
|
||||||
|
*/
|
||||||
|
export const RESTFUL_FAMILIES = ['minimal', 'flow', 'organic'];
|
||||||
|
const QUIET_KINDS = ['intro', 'breakdown', 'outro'];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Each director maps every section kind to three families, most-preferred
|
||||||
|
* first. Every kind must be present — a missing one silently falls back to the
|
||||||
|
* whole library and the point of view is lost exactly where it matters most.
|
||||||
|
*/
|
||||||
|
export const DIRECTORS = [
|
||||||
|
{
|
||||||
|
name: 'ambient',
|
||||||
|
// The original table. A drop resolves into geometry; everything quiet is
|
||||||
|
// minimal. Still the most broadly applicable, so it keeps the most weight.
|
||||||
|
weight: 3,
|
||||||
|
families: {
|
||||||
|
intro: ['minimal', 'flow', 'organic'],
|
||||||
|
build: ['structural', 'geometric', 'flow'],
|
||||||
|
drop: ['geometric', 'glitch', 'structural'],
|
||||||
|
sustain: ['organic', 'flow', 'geometric'],
|
||||||
|
breakdown: ['minimal', 'organic', 'flow'],
|
||||||
|
outro: ['minimal', 'flow', 'organic'],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: 'brutalist',
|
||||||
|
// Everything is architecture. Quiet means empty rather than soft, so it
|
||||||
|
// leads on minimal and reaches for organic last.
|
||||||
|
weight: 2,
|
||||||
|
families: {
|
||||||
|
intro: ['minimal', 'organic', 'flow'],
|
||||||
|
build: ['structural', 'geometric', 'glitch'],
|
||||||
|
drop: ['structural', 'glitch', 'geometric'],
|
||||||
|
sustain: ['structural', 'geometric', 'flow'],
|
||||||
|
breakdown: ['minimal', 'flow', 'organic'],
|
||||||
|
outro: ['minimal', 'organic', 'flow'],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: 'organicist',
|
||||||
|
// Nothing is ever built; things grow and dissolve. Deliberately never
|
||||||
|
// reaches for glitch — a point of view is defined by what it refuses.
|
||||||
|
weight: 2,
|
||||||
|
families: {
|
||||||
|
intro: ['organic', 'flow', 'minimal'],
|
||||||
|
build: ['organic', 'flow', 'structural'],
|
||||||
|
drop: ['organic', 'geometric', 'flow'],
|
||||||
|
sustain: ['organic', 'flow', 'minimal'],
|
||||||
|
breakdown: ['organic', 'minimal', 'flow'],
|
||||||
|
outro: ['flow', 'organic', 'minimal'],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: 'corrupt',
|
||||||
|
// The signal is damaged and the damage is the subject — everywhere the
|
||||||
|
// damage is allowed to be. Its quiet sections lead on flow, so the calm
|
||||||
|
// reads as signal drifting rather than as rest.
|
||||||
|
weight: 2,
|
||||||
|
families: {
|
||||||
|
intro: ['flow', 'minimal', 'organic'],
|
||||||
|
build: ['glitch', 'structural', 'geometric'],
|
||||||
|
drop: ['glitch', 'geometric', 'structural'],
|
||||||
|
sustain: ['glitch', 'organic', 'flow'],
|
||||||
|
breakdown: ['minimal', 'flow', 'organic'],
|
||||||
|
outro: ['flow', 'minimal', 'organic'],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: 'geometer',
|
||||||
|
// Pattern first, everywhere, at every energy. The drop is not an
|
||||||
|
// explosion, it is the pattern at its densest.
|
||||||
|
weight: 2,
|
||||||
|
families: {
|
||||||
|
intro: ['minimal', 'flow', 'organic'],
|
||||||
|
build: ['geometric', 'structural', 'flow'],
|
||||||
|
drop: ['geometric', 'glitch', 'structural'],
|
||||||
|
sustain: ['geometric', 'organic', 'flow'],
|
||||||
|
breakdown: ['minimal', 'organic', 'flow'],
|
||||||
|
outro: ['minimal', 'flow', 'organic'],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
export const DIRECTOR_NAMES = DIRECTORS.map((d) => d.name);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Cast the director for a track.
|
||||||
|
*
|
||||||
|
* The audio tilts the odds and never decides: a noisy, loud track is more
|
||||||
|
* likely to be filmed as corrupt or brutalist and a tonal one as organicist,
|
||||||
|
* but every director stays reachable for every track. Predictable mapping from
|
||||||
|
* measured features to look is the failure mode this whole layer exists to
|
||||||
|
* avoid — it is how a library ends up with one house style per genre.
|
||||||
|
*/
|
||||||
|
export function pickDirector(summary, rng) {
|
||||||
|
const noisy = Math.min(1, (summary.meanFlatness ?? 0.2) * 3);
|
||||||
|
const bright = summary.meanCentroid ?? 0.5;
|
||||||
|
|
||||||
|
const weights = DIRECTORS.map((d) => {
|
||||||
|
let w = d.weight;
|
||||||
|
if (d.name === 'corrupt') w *= 0.5 + noisy * 2.0;
|
||||||
|
if (d.name === 'brutalist') w *= 0.6 + noisy * 1.2;
|
||||||
|
if (d.name === 'organicist') w *= 0.6 + (1 - noisy) * 1.4;
|
||||||
|
if (d.name === 'geometer') w *= 0.7 + bright * 1.0;
|
||||||
|
return w;
|
||||||
|
});
|
||||||
|
|
||||||
|
return rng.pickWeighted(DIRECTORS, weights);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function directorByName(name) {
|
||||||
|
return DIRECTORS.find((d) => d.name === name) || DIRECTORS[0];
|
||||||
|
}
|
||||||
114
flow-state/src/look/framing.js
Normal file
114
flow-state/src/look/framing.js
Normal file
@ -0,0 +1,114 @@
|
|||||||
|
// How a shot is FRAMED, as opposed to what it contains.
|
||||||
|
//
|
||||||
|
// Every scene in the library is a locked-off, full-frame wide, and always has
|
||||||
|
// been. That is one shot type, held for the length of a song. Cutting between
|
||||||
|
// two scenes therefore changes the subject and never the framing — and framing
|
||||||
|
// is at least half of how a real edit holds attention. A wide answered by a
|
||||||
|
// close reads as two shots of one thing; two wides read as two things.
|
||||||
|
//
|
||||||
|
// So a shot now carries a scale and a recentre, applied in SCENE coordinates
|
||||||
|
// inside sigCamera. That distinction matters: a close-up is rendered close
|
||||||
|
// rather than being a magnified 720p frame, which is why this is a coordinate
|
||||||
|
// transform and not a post pass. It is also why it costs nothing at 4K.
|
||||||
|
//
|
||||||
|
// Framing is per shot and constant within it. A zoom that moves during a shot
|
||||||
|
// is a different device — one that would fight the drift LFO and the slow axis,
|
||||||
|
// both of which already own continuous motion.
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The shot sizes, as multipliers on the scene's coordinate scale.
|
||||||
|
*
|
||||||
|
* Bounded much more tightly than a real camera would be. Past about 2.2 most
|
||||||
|
* scenes in this library run out of detail and a close-up is just a blurry
|
||||||
|
* wide; below about 0.55 the subject is a speck in an empty frame. Both were
|
||||||
|
* measured by pushing until the image stopped being worth looking at.
|
||||||
|
*/
|
||||||
|
export const SHOT_SIZES = {
|
||||||
|
wide: { scale: 0.62, drift: 0.06 },
|
||||||
|
normal: { scale: 1.0, drift: 0.05 },
|
||||||
|
close: { scale: 1.7, drift: 0.10 },
|
||||||
|
};
|
||||||
|
|
||||||
|
export const SHOT_SIZE_NAMES = Object.keys(SHOT_SIZES);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Whether this track uses framing at all, and how boldly.
|
||||||
|
*
|
||||||
|
* A track that never changes size is a legitimate look — locked-off is a style,
|
||||||
|
* and it is the one the whole library was built in — so it stays reachable.
|
||||||
|
* What is not acceptable is it being the only option, which is what it was.
|
||||||
|
*/
|
||||||
|
export function deriveFramingStyle(summary, rng) {
|
||||||
|
const mode = rng.pickWeighted(['locked', 'gentle', 'edited'], [1, 2.5, 3]);
|
||||||
|
return {
|
||||||
|
mode,
|
||||||
|
// How far from `normal` this track is willing to go.
|
||||||
|
range: mode === 'locked' ? 0 : mode === 'gentle' ? 0.45 : 1,
|
||||||
|
// Chance a cut also changes the shot size, rather than only the image.
|
||||||
|
changeChance: mode === 'locked' ? 0 : mode === 'gentle' ? 0.35 : 0.6,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Choose the framing for one shot.
|
||||||
|
*
|
||||||
|
* `previous` is the framing of the shot before it, and it is the whole point:
|
||||||
|
* a size only means something relative to the size before it. The rule is that
|
||||||
|
* a change of size must be a real change — a wide answered by a slightly less
|
||||||
|
* wide is not a cut, it is a mistake — so sizes step rather than slide.
|
||||||
|
*
|
||||||
|
* @param {object} style from deriveFramingStyle
|
||||||
|
* @param {object|null} previous the previous shot's framing
|
||||||
|
* @param {number} energy section energy, 0..1
|
||||||
|
* @param {Rng} rng
|
||||||
|
*/
|
||||||
|
export function frameShot(style, previous, energy, rng) {
|
||||||
|
if (style.mode === 'locked') return neutralFraming();
|
||||||
|
|
||||||
|
const keep = previous && !rng.bool(style.changeChance);
|
||||||
|
if (keep) return { ...previous };
|
||||||
|
|
||||||
|
// Loud material earns the close-ups; quiet material earns the wides. This
|
||||||
|
// is a lean rather than a rule, so an intro can still land on a close and
|
||||||
|
// read as intimate instead of empty.
|
||||||
|
const weights = [
|
||||||
|
1 + (1 - energy) * 2.5, // wide
|
||||||
|
2, // normal
|
||||||
|
1 + energy * 2.5, // close
|
||||||
|
];
|
||||||
|
let size = rng.pickWeighted(SHOT_SIZE_NAMES, weights);
|
||||||
|
|
||||||
|
// Never repeat the previous size when we have decided to change: repeating
|
||||||
|
// it is what "no change" already means.
|
||||||
|
if (previous && size === previous.size) {
|
||||||
|
const others = SHOT_SIZE_NAMES.filter((n) => n !== size);
|
||||||
|
size = rng.pick(others);
|
||||||
|
}
|
||||||
|
|
||||||
|
const spec = SHOT_SIZES[size];
|
||||||
|
// Scale toward 1 for a timid track, so `gentle` is genuinely gentle rather
|
||||||
|
// than the same sizes drawn less often.
|
||||||
|
const scale = 1 + (spec.scale - 1) * style.range;
|
||||||
|
|
||||||
|
// Recentring is what stops a close-up being a centre crop of the wide. Held
|
||||||
|
// small: the scenes are centred compositions and pushing far off centre
|
||||||
|
// finds their empty corners.
|
||||||
|
const angle = rng.range(0, Math.PI * 2);
|
||||||
|
const amount = spec.drift * style.range * rng.range(0.3, 1);
|
||||||
|
|
||||||
|
return {
|
||||||
|
size,
|
||||||
|
scale,
|
||||||
|
shift: [Math.cos(angle) * amount, Math.sin(angle) * amount],
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export function neutralFraming() {
|
||||||
|
return { size: 'normal', scale: 1, shift: [0, 0] };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** One line for the HUD and check output. */
|
||||||
|
export function describeFraming(style) {
|
||||||
|
if (!style || style.mode === 'locked') return 'framing: locked off';
|
||||||
|
return `framing: ${style.mode}`;
|
||||||
|
}
|
||||||
193
flow-state/src/look/grain.js
Normal file
193
flow-state/src/look/grain.js
Normal file
@ -0,0 +1,193 @@
|
|||||||
|
// Grain as a deliberate treatment rather than a permanent surface.
|
||||||
|
//
|
||||||
|
// Grain used to be two unconditional additions — every scene added sigGrain and
|
||||||
|
// the grade added its own on top — with only the amount varying. The result was
|
||||||
|
// that every track in the library was grainy, which made grain read as the
|
||||||
|
// renderer's fingerprint instead of as a choice about one video.
|
||||||
|
//
|
||||||
|
// So grain is now DESCRIBED, not dialled:
|
||||||
|
//
|
||||||
|
// mode — when it is present at all. A third of tracks get none, and of the
|
||||||
|
// rest most only carry it some of the time.
|
||||||
|
// scale — the size of a noise cell in pixels. 1 is film-fine; 5 is a coarse
|
||||||
|
// dither that reads as a different medium entirely.
|
||||||
|
// rate — how many frames a noise field survives. 1 boils; 5 is sticky
|
||||||
|
// static that sits on the image like dirt on a lens.
|
||||||
|
// mask — where it lands. Only in the shadows, only in the highlights, only
|
||||||
|
// toward the edges, or in horizontal bands.
|
||||||
|
// chroma — mono speckle or colour speckle.
|
||||||
|
//
|
||||||
|
// Two tracks that both "have grain" should still not look like each other.
|
||||||
|
|
||||||
|
/** Mask ids, mirrored in COMPOSITE_FRAG. */
|
||||||
|
export const GRAIN_MASKS = { uniform: 0, shadows: 1, highlights: 2, edges: 3, bands: 4 };
|
||||||
|
|
||||||
|
export const GRAIN_MODES = ['off', 'constant', 'swell', 'sections', 'transient'];
|
||||||
|
|
||||||
|
/** Section kinds a 'sections' grain can be pinned to. */
|
||||||
|
const GATEABLE_KINDS = ['intro', 'build', 'drop', 'sustain', 'breakdown', 'outro'];
|
||||||
|
|
||||||
|
export const NO_GRAIN = {
|
||||||
|
mode: 'off',
|
||||||
|
amount: 0,
|
||||||
|
scale: 1,
|
||||||
|
rate: 1,
|
||||||
|
mask: 'uniform',
|
||||||
|
chroma: 0,
|
||||||
|
kinds: [],
|
||||||
|
period: 24,
|
||||||
|
duty: 0.4,
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The track's grain treatment.
|
||||||
|
*
|
||||||
|
* `noisy` (spectral flatness) tilts the odds but never forces the issue: a
|
||||||
|
* clean tonal track can still be the one that gets heavy dirt, because that is
|
||||||
|
* a legitimate art-direction choice and predictable mapping is what made the
|
||||||
|
* library uniform in the first place.
|
||||||
|
*/
|
||||||
|
export function deriveGrain(summary, rng) {
|
||||||
|
const noisy = Math.min(1, (summary.meanFlatness ?? 0.2) * 3);
|
||||||
|
|
||||||
|
// 'off' is the single most likely outcome, and deliberately so — a library
|
||||||
|
// where two videos in five have no grain at all is what makes the ones that
|
||||||
|
// do read as a decision.
|
||||||
|
const mode = rng.pickWeighted(
|
||||||
|
['off', 'constant', 'swell', 'sections', 'transient'],
|
||||||
|
[6 - noisy * 3, 1.5 + noisy * 2.5, 2.5, 2.5, 2],
|
||||||
|
);
|
||||||
|
if (mode === 'off') return { ...NO_GRAIN };
|
||||||
|
|
||||||
|
// Constant grain has to live with the image for the whole video, so it is
|
||||||
|
// held well below what an intermittent treatment can get away with.
|
||||||
|
const ceiling = mode === 'constant' ? 0.05 : 0.11;
|
||||||
|
const amount = rng.range(0.015, ceiling);
|
||||||
|
|
||||||
|
const scale = rng.pickWeighted([1, 1.5, 2, 3, 5], [4, 3, 3, 2, 1]);
|
||||||
|
// Coarse cells that also boil every frame read as a broken video signal, so
|
||||||
|
// the bigger the cell the more likely it is to hold still for a few frames.
|
||||||
|
const rate = rng.pickWeighted([1, 2, 3, 5], [5, 2 + scale, 1 + scale, scale]);
|
||||||
|
|
||||||
|
const mask = rng.pickWeighted(
|
||||||
|
['uniform', 'shadows', 'highlights', 'edges', 'bands'],
|
||||||
|
[3, 3, 1.5, 2, 1],
|
||||||
|
);
|
||||||
|
|
||||||
|
const kinds = mode === 'sections' ? pickKinds(rng) : [];
|
||||||
|
|
||||||
|
return {
|
||||||
|
mode,
|
||||||
|
amount,
|
||||||
|
scale,
|
||||||
|
rate,
|
||||||
|
mask,
|
||||||
|
// Colour speckle is the loudest of these choices and stays rare.
|
||||||
|
chroma: rng.bool(0.25) ? rng.range(0.3, 1) : 0,
|
||||||
|
kinds,
|
||||||
|
// Swell period in seconds. Long enough that it reads as the image
|
||||||
|
// breathing rather than as a flicker.
|
||||||
|
period: rng.range(12, 40),
|
||||||
|
duty: rng.range(0.25, 0.6),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** One to three section kinds this grain belongs to. */
|
||||||
|
function pickKinds(rng) {
|
||||||
|
const count = rng.pickWeighted([1, 2, 3], [3, 3, 1]);
|
||||||
|
const pool = [...GATEABLE_KINDS];
|
||||||
|
const out = [];
|
||||||
|
for (let i = 0; i < count && pool.length; i++) {
|
||||||
|
const pick = rng.pick(pool);
|
||||||
|
out.push(pick);
|
||||||
|
pool.splice(pool.indexOf(pick), 1);
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The 0..1 envelope on the grain amount for one frame.
|
||||||
|
*
|
||||||
|
* Deterministic in frame and features only — no state, no random source — so a
|
||||||
|
* preview frame and the exported frame agree, which is the same rule the noise
|
||||||
|
* itself follows.
|
||||||
|
*
|
||||||
|
* @param {object} spec from deriveGrain
|
||||||
|
* @param {object} ctx
|
||||||
|
* @param {number} ctx.time seconds into the track
|
||||||
|
* @param {string} ctx.sectionKind kind of the section this frame is in
|
||||||
|
* @param {object} ctx.features FeatureTrack row for this frame
|
||||||
|
*/
|
||||||
|
export function grainEnvelope(spec, { time = 0, sectionKind = '', features = null } = {}) {
|
||||||
|
if (!spec || spec.mode === 'off' || spec.amount <= 0) return 0;
|
||||||
|
|
||||||
|
switch (spec.mode) {
|
||||||
|
case 'constant':
|
||||||
|
return 1;
|
||||||
|
|
||||||
|
case 'swell': {
|
||||||
|
// Raised cosine over `period`, on for `duty` of it. The image drifts
|
||||||
|
// into grain and back out with nothing in the audio triggering it,
|
||||||
|
// which is what makes it feel like film rather than like a reaction.
|
||||||
|
const phase = (time % spec.period) / spec.period;
|
||||||
|
if (phase > spec.duty) return 0;
|
||||||
|
return 0.5 - 0.5 * Math.cos((phase / spec.duty) * Math.PI * 2);
|
||||||
|
}
|
||||||
|
|
||||||
|
case 'sections': {
|
||||||
|
if (!spec.kinds.includes(sectionKind)) return 0;
|
||||||
|
// Ease across the section edges so grain arrives with the section
|
||||||
|
// rather than snapping on at the cut.
|
||||||
|
const p = features ? features.sectionProgress : 0.5;
|
||||||
|
return smoothstep(0, 0.08, p) * smoothstep(0, 0.08, 1 - p);
|
||||||
|
}
|
||||||
|
|
||||||
|
case 'transient': {
|
||||||
|
if (!features) return 0;
|
||||||
|
// Rides flux, so grain answers hits and edits. Floored slightly
|
||||||
|
// above zero on loud material so it does not strobe on and off.
|
||||||
|
const hit = Math.min(1, (features.flux ?? 0) * 2.5);
|
||||||
|
const bed = Math.min(0.35, (features.sectionEnergy ?? 0) * 0.35);
|
||||||
|
return Math.max(bed, hit);
|
||||||
|
}
|
||||||
|
|
||||||
|
default:
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function smoothstep(a, b, x) {
|
||||||
|
const t = Math.max(0, Math.min(1, (x - a) / (b - a)));
|
||||||
|
return t * t * (3 - 2 * t);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Push a grain spec's static fields into a post object.
|
||||||
|
*
|
||||||
|
* The amount is the exception: for anything other than 'constant' it is owned
|
||||||
|
* by the per-frame envelope (see Show._postAt), so it is set to zero here and
|
||||||
|
* the envelope writes it every frame.
|
||||||
|
*/
|
||||||
|
export function applyGrainToPost(spec, post) {
|
||||||
|
post.grain = spec.mode === 'constant' ? spec.amount : 0;
|
||||||
|
post.grainScale = spec.scale;
|
||||||
|
post.grainRate = spec.rate;
|
||||||
|
post.grainMask = GRAIN_MASKS[spec.mask] ?? 0;
|
||||||
|
post.grainChroma = spec.chroma;
|
||||||
|
return post;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** One line for the look panel and check output. */
|
||||||
|
export function describeGrain(spec) {
|
||||||
|
if (!spec || spec.mode === 'off') return 'grain: none';
|
||||||
|
const bits = [
|
||||||
|
`grain: ${spec.mode}`,
|
||||||
|
`${spec.amount.toFixed(3)}`,
|
||||||
|
`${spec.scale}px`,
|
||||||
|
spec.rate > 1 ? `every ${spec.rate}f` : 'per frame',
|
||||||
|
spec.mask,
|
||||||
|
];
|
||||||
|
if (spec.chroma > 0) bits.push('colour');
|
||||||
|
if (spec.kinds.length) bits.push(`on ${spec.kinds.join('+')}`);
|
||||||
|
return bits.join(' · ');
|
||||||
|
}
|
||||||
301
flow-state/src/look/palette.js
Normal file
301
flow-state/src/look/palette.js
Normal file
@ -0,0 +1,301 @@
|
|||||||
|
// Palette generation.
|
||||||
|
//
|
||||||
|
// Colours are built in OKLCH rather than HSL. HSL's lightness is not perceptual —
|
||||||
|
// pure yellow and pure blue at the same "lightness" differ enormously in how
|
||||||
|
// bright they look — so an HSL palette with even lightness steps produces a set
|
||||||
|
// where some colours vanish and others dominate. OKLCH steps look even because
|
||||||
|
// they are even, which matters a lot when the generator is choosing palettes
|
||||||
|
// unsupervised and nobody is there to correct a bad one.
|
||||||
|
//
|
||||||
|
// Cover art is not available (PLAN.md §Decisions), so everything here derives
|
||||||
|
// from the audio. `PaletteSource` is the seam: adding a CoverArtPalette later is
|
||||||
|
// a new implementation of this interface and one line of config, with no change
|
||||||
|
// to any scene.
|
||||||
|
|
||||||
|
/** OKLCH -> sRGB, components 0..1. h in radians. */
|
||||||
|
export function oklchToRgb(L, C, h) {
|
||||||
|
const a = C * Math.cos(h);
|
||||||
|
const b = C * Math.sin(h);
|
||||||
|
|
||||||
|
const l_ = L + 0.3963377774 * a + 0.2158037573 * b;
|
||||||
|
const m_ = L - 0.1055613458 * a - 0.0638541728 * b;
|
||||||
|
const s_ = L - 0.0894841775 * a - 1.2914855480 * b;
|
||||||
|
|
||||||
|
const l = l_ * l_ * l_;
|
||||||
|
const m = m_ * m_ * m_;
|
||||||
|
const s = s_ * s_ * s_;
|
||||||
|
|
||||||
|
const lr = 4.0767416621 * l - 3.3077115913 * m + 0.2309699292 * s;
|
||||||
|
const lg = -1.2684380046 * l + 2.6097574011 * m - 0.3413193965 * s;
|
||||||
|
const lb = -0.0041960863 * l - 0.7034186147 * m + 1.7076147010 * s;
|
||||||
|
|
||||||
|
const gamma = (x) => {
|
||||||
|
const v = Math.max(0, Math.min(1, x));
|
||||||
|
return v <= 0.0031308 ? 12.92 * v : 1.055 * Math.pow(v, 1 / 2.4) - 0.055;
|
||||||
|
};
|
||||||
|
return [gamma(lr), gamma(lg), gamma(lb)];
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* sRGB -> OKLCH, the exact inverse of the above.
|
||||||
|
*
|
||||||
|
* Needed because a palette is stored as RGB but has to be MOVED perceptually:
|
||||||
|
* rotating hue in RGB space changes brightness as a side effect, which is
|
||||||
|
* exactly the artefact OKLCH was chosen to avoid in the first place. See
|
||||||
|
* look/paletteArc.js.
|
||||||
|
*/
|
||||||
|
export function rgbToOklch([r, g, b]) {
|
||||||
|
const linear = (x) => {
|
||||||
|
const v = Math.max(0, Math.min(1, x));
|
||||||
|
return v <= 0.04045 ? v / 12.92 : Math.pow((v + 0.055) / 1.055, 2.4);
|
||||||
|
};
|
||||||
|
const R = linear(r), G = linear(g), B = linear(b);
|
||||||
|
|
||||||
|
const l = 0.4122214708 * R + 0.5363325363 * G + 0.0514459929 * B;
|
||||||
|
const m = 0.2119034982 * R + 0.6806995451 * G + 0.1073969566 * B;
|
||||||
|
const s = 0.0883024619 * R + 0.2817188376 * G + 0.6299787005 * B;
|
||||||
|
|
||||||
|
const l_ = Math.cbrt(l), m_ = Math.cbrt(m), s_ = Math.cbrt(s);
|
||||||
|
|
||||||
|
const L = 0.2104542553 * l_ + 0.7936177850 * m_ - 0.0040720468 * s_;
|
||||||
|
const a = 1.9779984951 * l_ - 2.4285922050 * m_ + 0.4505937099 * s_;
|
||||||
|
const bb = 0.0259040371 * l_ + 0.7827717662 * m_ - 0.8086757660 * s_;
|
||||||
|
|
||||||
|
return [L, Math.hypot(a, bb), Math.atan2(bb, a)];
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Move a whole palette in OKLCH: rotate hue, scale chroma, offset lightness.
|
||||||
|
*
|
||||||
|
* The rotation is applied to every colour equally, so the relationships that
|
||||||
|
* made the palette a palette — its scheme, its spread — survive the move. A
|
||||||
|
* per-colour rotation would be a different palette rather than the same one
|
||||||
|
* somewhere else.
|
||||||
|
*/
|
||||||
|
export function shiftPalette(colors, { hue = 0, chroma = 1, lightness = 0 } = {}) {
|
||||||
|
if (!hue && chroma === 1 && !lightness) return colors;
|
||||||
|
return colors.map((c) => {
|
||||||
|
const [L, C, h] = rgbToOklch(c);
|
||||||
|
return oklchToRgb(
|
||||||
|
Math.max(0, Math.min(1, L + lightness)),
|
||||||
|
Math.max(0, C * chroma),
|
||||||
|
h + hue,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
export function relativeLuminance([r, g, b]) {
|
||||||
|
return 0.2126 * r + 0.7152 * g + 0.0722 * b;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Spread of a palette's luminance and hue. The Phase 3 gate uses this to reject
|
||||||
|
* muddy sets — palettes where everything sits at the same brightness read as a
|
||||||
|
* single colour once they are composited and bloomed.
|
||||||
|
*/
|
||||||
|
export function paletteContrast(colors) {
|
||||||
|
if (!colors || colors.length < 2) return { luminanceSpread: 0, chromaSpread: 0 };
|
||||||
|
const lums = colors.map(relativeLuminance);
|
||||||
|
const luminanceSpread = Math.max(...lums) - Math.min(...lums);
|
||||||
|
|
||||||
|
let chromaSpread = 0;
|
||||||
|
for (let i = 0; i < colors.length; i++) {
|
||||||
|
for (let j = i + 1; j < colors.length; j++) {
|
||||||
|
const d = Math.hypot(
|
||||||
|
colors[i][0] - colors[j][0],
|
||||||
|
colors[i][1] - colors[j][1],
|
||||||
|
colors[i][2] - colors[j][2],
|
||||||
|
);
|
||||||
|
chromaSpread = Math.max(chromaSpread, d);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return { luminanceSpread, chromaSpread };
|
||||||
|
}
|
||||||
|
|
||||||
|
const SCHEMES = {
|
||||||
|
analogous: (h, rng) => [h, h + 0.35, h - 0.35, h + 0.7, h - 0.6, h + 1.0],
|
||||||
|
complement: (h) => [h, h + Math.PI, h + 0.4, h + Math.PI - 0.4, h + 0.8, h + Math.PI + 0.3],
|
||||||
|
triad: (h) => [h, h + 2.094, h + 4.189, h + 0.5, h + 2.6, h + 4.7],
|
||||||
|
split: (h) => [h, h + 2.6, h + 3.7, h + 0.35, h + 2.9, h + 3.4],
|
||||||
|
duo: (h) => [h, h + 1.9, h + 0.2, h + 2.1, h - 0.25, h + 1.7],
|
||||||
|
// Four evenly spaced hues plus two repeats: the widest spread available, and
|
||||||
|
// the reason a track can now come out looking like four colours rather than
|
||||||
|
// a gradient between two.
|
||||||
|
tetrad: (h) => [h, h + 1.571, h + 3.142, h + 4.712, h + 0.8, h + 2.4],
|
||||||
|
// One hue family carrying the frame, with a single far-off pop. Reads as a
|
||||||
|
// deliberate art-directed choice rather than as a spectrum.
|
||||||
|
accented: (h) => [h, h + 0.25, h - 0.2, h + 0.45, h + 2.9, h + 3.05],
|
||||||
|
// One hue, everything else carried by lightness and chroma. Needs the
|
||||||
|
// widened L range below to stay legible, and gives the library the
|
||||||
|
// near-monochrome look it could not previously reach at all.
|
||||||
|
mono: (h) => [h, h + 0.12, h - 0.1, h + 0.18, h - 0.16, h + 0.08],
|
||||||
|
};
|
||||||
|
|
||||||
|
export const SCHEME_NAMES = Object.keys(SCHEMES);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Stretch a value around a centre so a narrow real-world range fills 0..1.
|
||||||
|
*
|
||||||
|
* A logistic rather than a linear rescale, because the tails must stay bounded:
|
||||||
|
* an unusually bass-heavy track should land at the warm end, not past it.
|
||||||
|
*/
|
||||||
|
function expand(x, centre = 0.5, slope = 3.0) {
|
||||||
|
return 1 / (1 + Math.exp(-slope * (x - centre) * 4));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The interface a palette source implements. */
|
||||||
|
export class PaletteSource {
|
||||||
|
/** @returns {number[][]} array of [r,g,b] in 0..1 */
|
||||||
|
generate() { throw new Error('PaletteSource.generate not implemented'); }
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Derives a palette from what the track actually sounds like.
|
||||||
|
*
|
||||||
|
* - spectral centroid -> hue family. A bass-heavy track lands in deep blues and
|
||||||
|
* violets; a bright one moves toward cyan, green and amber. This is the single
|
||||||
|
* strongest differentiator between two tracks, because it tracks the thing a
|
||||||
|
* listener would call the track's colour anyway.
|
||||||
|
* - flatness (noisy vs tonal) -> chroma. Noisy material gets desaturated so it
|
||||||
|
* doesn't turn to mud once bloom is applied.
|
||||||
|
* - dynamic range -> lightness spread. A dynamic track earns a wider range
|
||||||
|
* between its darkest and brightest colour.
|
||||||
|
*/
|
||||||
|
export class AudioPalette extends PaletteSource {
|
||||||
|
constructor(summary, rng) {
|
||||||
|
super();
|
||||||
|
this.summary = summary;
|
||||||
|
this.rng = rng;
|
||||||
|
}
|
||||||
|
|
||||||
|
generate(count = 6) {
|
||||||
|
const {
|
||||||
|
meanCentroid = 0.5, meanFlatness = 0.2, dynamicRange = 0.5,
|
||||||
|
bandBalance = {}, bpm = 120,
|
||||||
|
} = this.summary;
|
||||||
|
const rng = this.rng;
|
||||||
|
|
||||||
|
// --- Temperature: the track's timbre signature, not its loudness ---
|
||||||
|
// SPECTRAL TILT — the log ratio of treble energy to body energy — rather
|
||||||
|
// than either the centroid or a plain body/(body+treble) fraction.
|
||||||
|
//
|
||||||
|
// Both of those were tried and both collapse. The centroid is one number
|
||||||
|
// most mastered music sits in the middle of. The plain fraction is worse:
|
||||||
|
// low frequencies carry most of the energy in essentially all music, so
|
||||||
|
// it reads 0.98-1.00 for everything and the four check-battery tracks
|
||||||
|
// came out within 0.02 of each other. The ratio is MULTIPLICATIVE, so its
|
||||||
|
// logarithm is what actually spreads: the same four tracks measure -9.3,
|
||||||
|
// -5.0, -4.1 and -3.8, which is a real axis to hang a palette on.
|
||||||
|
const bands = {
|
||||||
|
sub: bandBalance.sub ?? 0.2, low: bandBalance.low ?? 0.2, mid: bandBalance.mid ?? 0.2,
|
||||||
|
high: bandBalance.high ?? 0.2, air: bandBalance.air ?? 0.2,
|
||||||
|
};
|
||||||
|
const body = bands.sub * 1.0 + bands.low * 0.9 + bands.mid * 0.35 + 1e-7;
|
||||||
|
const treble = bands.high * 0.9 + bands.air * 1.0 + bands.mid * 0.15 + 1e-7;
|
||||||
|
const tilt = Math.log(treble / body);
|
||||||
|
|
||||||
|
// -9 (nothing above the low mids) .. -2 (bright, airy) covers the range
|
||||||
|
// real material occupies; the centroid keeps a minority vote so two
|
||||||
|
// tracks with the same tilt but different brightness still differ.
|
||||||
|
const tiltWarmth = Math.max(0, Math.min(1, (-2 - tilt) / 7));
|
||||||
|
const warmth = tiltWarmth * 0.7 + (1 - meanCentroid) * 0.3;
|
||||||
|
|
||||||
|
// Hue sweeps cold -> warm, but which WAY round the wheel is seeded.
|
||||||
|
// Going down from red through yellow and green to blue is the obvious
|
||||||
|
// route and the only one that existed; it also means violet, magenta and
|
||||||
|
// pink were unreachable for every track ever generated, because they sit
|
||||||
|
// on the arc the sweep skipped. Half of tracks now take the other way
|
||||||
|
// round, so the same warm/cool reading can land on crimson-through-
|
||||||
|
// magenta instead of crimson-through-amber.
|
||||||
|
//
|
||||||
|
// Both routes span the same arc. A short return leg would mean tracks
|
||||||
|
// that took it barely differ in hue however different they sound.
|
||||||
|
const clockwise = rng.bool(0.5);
|
||||||
|
const span = (clockwise ? 1 : -1) * Math.PI * 4 / 3;
|
||||||
|
|
||||||
|
// Tempo and dynamics nudge the hue too. Timbre is the main axis, but two
|
||||||
|
// tracks can be timbrally alike and still feel different — a slow
|
||||||
|
// spacious one and a fast compressed one should not be handed the same
|
||||||
|
// colour just because they occupy the same part of the spectrum.
|
||||||
|
const feel = ((bpm - 120) / 200 + (dynamicRange - 0.5) * 0.5) * 0.6;
|
||||||
|
|
||||||
|
const baseHue = (1 - warmth) * span + feel + rng.range(-0.45, 0.45);
|
||||||
|
|
||||||
|
const schemeName = rng.pick(SCHEME_NAMES);
|
||||||
|
const hues = SCHEMES[schemeName](baseHue, rng);
|
||||||
|
|
||||||
|
// --- Energy: how vivid and punchy the palette is ---
|
||||||
|
// Upbeat, dynamic material earns saturated colour; chill, flat material
|
||||||
|
// stays muted. This is the "does it pop" axis, orthogonal to timbre.
|
||||||
|
const fast = Math.min(1, Math.max(0, (bpm - 80) / 150));
|
||||||
|
const energy = Math.min(1, fast * 0.4 + (1 - Math.min(1, meanFlatness)) * 0.4 + dynamicRange * 0.3);
|
||||||
|
// Vividness is the track's, but how far it commits is seeded — the old
|
||||||
|
// fixed mapping meant two tracks with similar statistics got not just
|
||||||
|
// similar hues but the same saturation, which is most of why they read
|
||||||
|
// as the same palette.
|
||||||
|
const vividness = rng.range(0.55, 1.45);
|
||||||
|
const chromaBase = (0.09 + energy * 0.19) * vividness;
|
||||||
|
|
||||||
|
// Chroma profile: does the palette saturate in the middle (the old fixed
|
||||||
|
// behaviour), at the bright end, or barely at all? A near-neutral set
|
||||||
|
// with one vivid accent is a look the generator could not previously
|
||||||
|
// produce.
|
||||||
|
const profile = rng.pickWeighted(['arch', 'rising', 'flat', 'accent'], [3, 2, 2, 2]);
|
||||||
|
const chromaAt = (t) => {
|
||||||
|
switch (profile) {
|
||||||
|
case 'rising': return 0.35 + t * 1.1;
|
||||||
|
case 'flat': return 0.9;
|
||||||
|
case 'accent': return t > 0.72 ? 1.5 : 0.28;
|
||||||
|
default: return 0.55 + Math.sin(t * Math.PI) * 0.75;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
// A dynamic mercury gets a wider light-to-dark range; warmth keeps warm
|
||||||
|
// tones from sinking into brown, since dark + orange is mud.
|
||||||
|
const spread = (0.30 + Math.min(1, dynamicRange) * 0.30) * rng.range(0.85, 1.5);
|
||||||
|
const anchor = 0.40 - warmth * 0.06 + rng.range(-0.14, 0.16);
|
||||||
|
// How the lightness steps are distributed: 1.7 keeps most entries dark
|
||||||
|
// with a couple of bright accents (the old fixed curve), below 1 spreads
|
||||||
|
// them evenly, above 2 makes the set almost entirely dark with one
|
||||||
|
// highlight. Another axis two similar tracks can differ on.
|
||||||
|
const curve = rng.range(0.75, 2.4);
|
||||||
|
|
||||||
|
const colors = [];
|
||||||
|
for (let i = 0; i < count; i++) {
|
||||||
|
const t = count > 1 ? i / (count - 1) : 0;
|
||||||
|
// Deliberately non-linear: most entries mid-dark, one or two bright.
|
||||||
|
// Scenes use pal(0) as a base and higher indices as accents.
|
||||||
|
const L = Math.max(0.05, Math.min(0.97, anchor + Math.pow(t, curve) * spread));
|
||||||
|
const C = chromaBase * chromaAt(t) + rng.range(-0.012, 0.012);
|
||||||
|
const h = hues[i % hues.length] + rng.range(-0.08, 0.08);
|
||||||
|
colors.push(oklchToRgb(L, Math.max(0, C), h));
|
||||||
|
}
|
||||||
|
|
||||||
|
this.lastScheme = schemeName;
|
||||||
|
this.lastProfile = profile;
|
||||||
|
return colors;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Retry until the palette clears the contrast floor. Unsupervised generation
|
||||||
|
* will occasionally land on a muddy set; regenerating is cheap and beats
|
||||||
|
* shipping a video where every colour is the same grey-violet.
|
||||||
|
*/
|
||||||
|
export function generateUsablePalette(source, count = 6, { minLuminanceSpread = 0.22, attempts = 12 } = {}) {
|
||||||
|
let best = null;
|
||||||
|
let bestScore = -1;
|
||||||
|
for (let i = 0; i < attempts; i++) {
|
||||||
|
const colors = source.generate(count);
|
||||||
|
const { luminanceSpread, chromaSpread } = paletteContrast(colors);
|
||||||
|
const score = luminanceSpread + chromaSpread * 0.4;
|
||||||
|
if (score > bestScore) { bestScore = score; best = colors; }
|
||||||
|
if (luminanceSpread >= minLuminanceSpread) return colors;
|
||||||
|
}
|
||||||
|
return best;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function toHex([r, g, b]) {
|
||||||
|
const c = (v) => Math.round(Math.max(0, Math.min(1, v)) * 255).toString(16).padStart(2, '0');
|
||||||
|
return `#${c(r)}${c(g)}${c(b)}`;
|
||||||
|
}
|
||||||
117
flow-state/src/look/paletteArc.js
Normal file
117
flow-state/src/look/paletteArc.js
Normal file
@ -0,0 +1,117 @@
|
|||||||
|
// Colour movement across a track.
|
||||||
|
//
|
||||||
|
// The palette was generated once and pushed to every layer of every section for
|
||||||
|
// the entire runtime. Five minutes, one colour scheme, no movement — and colour
|
||||||
|
// is the strongest perceptual variable the system has, so freezing it wastes
|
||||||
|
// the biggest lever available for making a long video feel like it is going
|
||||||
|
// somewhere.
|
||||||
|
//
|
||||||
|
// The fix is deliberately NOT "a new palette per section". A track has one
|
||||||
|
// identity and the palette is most of it; replacing it mid-video reads as a
|
||||||
|
// different video. What moves is the palette itself — rotated, warmed, opened
|
||||||
|
// up — so at four minutes the image is somewhere the first minute implied.
|
||||||
|
//
|
||||||
|
// Everything here is bounded on purpose. A full hue rotation would destroy the
|
||||||
|
// identity as surely as a new palette; the movement has to be the kind you
|
||||||
|
// notice on a rewatch rather than the kind you notice as an effect.
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Ceilings on the whole mechanism. Nothing downstream may exceed these, and the
|
||||||
|
* Phase 11 gate samples the movement against the palette contrast floor rather
|
||||||
|
* than trusting them.
|
||||||
|
*/
|
||||||
|
export const MAX_HUE_ROTATION = 0.6; // radians, ~34 degrees
|
||||||
|
export const MAX_CHROMA_SCALE = 0.35; // ±35% saturation
|
||||||
|
export const MAX_LIGHT_SHIFT = 0.07; // OKLCH lightness
|
||||||
|
|
||||||
|
export const ARC_MODES = ['static', 'drift', 'sections', 'lift'];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* How this track's colour moves.
|
||||||
|
*
|
||||||
|
* 'static' stays rare. This layer exists because nothing moved, and a library
|
||||||
|
* where most tracks still do not move would not have fixed anything — but a
|
||||||
|
* track whose colour holds is a legitimate choice and one in six or so gets it.
|
||||||
|
*/
|
||||||
|
export function derivePaletteArc(summary, rng) {
|
||||||
|
const mode = rng.pickWeighted(ARC_MODES, [1.5, 3, 3, 2.5]);
|
||||||
|
const dir = rng.bool() ? 1 : -1;
|
||||||
|
|
||||||
|
return {
|
||||||
|
mode,
|
||||||
|
// Total hue travel from the first frame to the last, for 'drift'.
|
||||||
|
hueTravel: dir * rng.range(0.25, MAX_HUE_ROTATION),
|
||||||
|
// Per-section-kind offsets, for 'sections': drops consistently warmer
|
||||||
|
// or cooler than breakdowns, so the colour tells you where you are.
|
||||||
|
kindHue: {
|
||||||
|
intro: rng.range(-0.2, 0.2),
|
||||||
|
build: rng.range(-0.3, 0.3),
|
||||||
|
drop: dir * rng.range(0.2, MAX_HUE_ROTATION),
|
||||||
|
sustain: rng.range(-0.25, 0.25),
|
||||||
|
breakdown: -dir * rng.range(0.1, 0.4),
|
||||||
|
outro: rng.range(-0.3, 0.3),
|
||||||
|
},
|
||||||
|
// For 'lift': how much energy opens the colour up. Saturation rising
|
||||||
|
// into a drop is the single most legible colour gesture available.
|
||||||
|
chromaLift: rng.range(0.12, MAX_CHROMA_SCALE),
|
||||||
|
lightLift: rng.range(0.02, MAX_LIGHT_SHIFT),
|
||||||
|
// Every mode carries a little of the slow drift underneath, so even a
|
||||||
|
// 'sections' track is not the same colour at the end as at the start.
|
||||||
|
underDrift: dir * rng.range(0.05, 0.2),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The colour shift for one frame, as arguments to palette.shiftPalette.
|
||||||
|
*
|
||||||
|
* Deterministic in progress and features only — no state, no random source —
|
||||||
|
* the same rule the rest of the render path follows.
|
||||||
|
*
|
||||||
|
* @param {object} arc from derivePaletteArc
|
||||||
|
* @param {object} ctx
|
||||||
|
* @param {number} ctx.progress 0..1 through the track
|
||||||
|
* @param {string} ctx.sectionKind kind of the section this frame is in
|
||||||
|
* @param {object} ctx.features FeatureTrack row
|
||||||
|
*/
|
||||||
|
export function paletteShiftAt(arc, { progress = 0, sectionKind = '', features = null } = {}) {
|
||||||
|
if (!arc) return { hue: 0, chroma: 1, lightness: 0 };
|
||||||
|
|
||||||
|
// The slow underlying travel, present in every mode. Eased rather than
|
||||||
|
// linear so the move is least visible at the ends, where a cut to the
|
||||||
|
// opening image would otherwise expose it.
|
||||||
|
const eased = progress * progress * (3 - 2 * progress);
|
||||||
|
let hue = arc.underDrift * eased;
|
||||||
|
let chroma = 1;
|
||||||
|
let lightness = 0;
|
||||||
|
|
||||||
|
if (arc.mode === 'drift') {
|
||||||
|
hue = arc.hueTravel * eased;
|
||||||
|
} else if (arc.mode === 'sections') {
|
||||||
|
// Held per section rather than ramped: the colour changing AT the cut
|
||||||
|
// is the point, and a ramp would smear it into nothing.
|
||||||
|
hue += arc.kindHue[sectionKind] ?? 0;
|
||||||
|
} else if (arc.mode === 'lift') {
|
||||||
|
const energy = features ? (features.sectionEnergy ?? 0) : 0;
|
||||||
|
const build = features ? (features.buildSlope ?? 0) : 0;
|
||||||
|
const drive = Math.min(1, energy * 0.7 + build * 0.6);
|
||||||
|
chroma = 1 + arc.chromaLift * (drive * 2 - 1);
|
||||||
|
lightness = arc.lightLift * (drive - 0.35);
|
||||||
|
hue += arc.kindHue[sectionKind] * 0.4 || 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
hue: clamp(hue, -MAX_HUE_ROTATION, MAX_HUE_ROTATION),
|
||||||
|
chroma: clamp(chroma, 1 - MAX_CHROMA_SCALE, 1 + MAX_CHROMA_SCALE),
|
||||||
|
lightness: clamp(lightness, -MAX_LIGHT_SHIFT, MAX_LIGHT_SHIFT),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function clamp(x, lo, hi) { return Math.max(lo, Math.min(hi, x)); }
|
||||||
|
|
||||||
|
/** One line for the HUD and check output. */
|
||||||
|
export function describePaletteArc(arc) {
|
||||||
|
if (!arc || arc.mode === 'static') return 'colour: held';
|
||||||
|
if (arc.mode === 'drift') return `colour: drift ${(arc.hueTravel * 57.3).toFixed(0)}°`;
|
||||||
|
if (arc.mode === 'lift') return `colour: lift ±${(arc.chromaLift * 100).toFixed(0)}% sat`;
|
||||||
|
return `colour: per-section ±${(Math.max(...Object.values(arc.kindHue).map(Math.abs)) * 57.3).toFixed(0)}°`;
|
||||||
|
}
|
||||||
230
flow-state/src/look/shots.js
Normal file
230
flow-state/src/look/shots.js
Normal file
@ -0,0 +1,230 @@
|
|||||||
|
// Shot planning: the level of hierarchy between a song section and a frame.
|
||||||
|
//
|
||||||
|
// A section is a STAGE of the song (intro, build, drop, …) and can easily run
|
||||||
|
// ninety seconds. One scene held for ninety seconds reads as a still image with
|
||||||
|
// a wobble on it, no matter how much per-frame reactivity is underneath. So a
|
||||||
|
// section is cut into SHOTS, each showing one of the section's few "stage
|
||||||
|
// visuals" — the roster the look generator picked for that kind of section.
|
||||||
|
//
|
||||||
|
// Two rules keep this from turning into a shuffle:
|
||||||
|
//
|
||||||
|
// * the roster is per section KIND, not per section, so all of a track's drops
|
||||||
|
// still cut between the same two or three visuals and the video keeps an
|
||||||
|
// identity;
|
||||||
|
// * cuts land on phrase lines, so a change of image lands with the music
|
||||||
|
// instead of across it.
|
||||||
|
//
|
||||||
|
// Shot length follows energy: a drop cuts every four to eight bars, an intro
|
||||||
|
// holds for eight to sixteen, and nothing holds past the ceiling below.
|
||||||
|
// Everything here is seeded, so a track always cuts in the same places.
|
||||||
|
|
||||||
|
/** Never cut faster than this, whatever the tempo or the energy says. */
|
||||||
|
export const MIN_SHOT_SECONDS = 5;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* And never hold longer than this either. Half a minute of one image is the
|
||||||
|
* complaint this whole level of hierarchy exists to answer, so it is a hard
|
||||||
|
* ceiling rather than something the bar maths is trusted to stay under: at a
|
||||||
|
* slow tempo sixteen bars is already past it.
|
||||||
|
*/
|
||||||
|
export const MAX_SHOT_SECONDS = 22;
|
||||||
|
|
||||||
|
/** Below this section energy, a shot change is always a dissolve, never a cut. */
|
||||||
|
export const HARD_CUT_ENERGY = 0.66;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The section's cutting RHYTHM, in bars per shot.
|
||||||
|
*
|
||||||
|
* This used to be a single bar count, and the section was then divided into
|
||||||
|
* that many equal pieces. Measured, a five-minute track at 90 BPM came out as
|
||||||
|
* sixteen shots of 19.3, 18.7, 18.7, 18.7 … 18.7, 19.3 seconds — a metronome.
|
||||||
|
* Every cut landing on the same pulse for five minutes is the most fatiguing
|
||||||
|
* edit rhythm available, and no amount of variety in what the shots CONTAIN
|
||||||
|
* fixes it, because the fatigue is in the timing rather than in the images.
|
||||||
|
*
|
||||||
|
* So a section carries a repeating PATTERN instead: a long hold, two quick
|
||||||
|
* ones, a long hold. The pattern is walked in order and repeats, which is what
|
||||||
|
* makes it read as phrasing rather than as randomness — random shot lengths
|
||||||
|
* would satisfy any "lengths must vary" test and would look worse, because the
|
||||||
|
* ear is following an eight-bar structure and the eye would not be.
|
||||||
|
*
|
||||||
|
* Every entry is a power-of-two bar count, so a cut is always on a phrase line
|
||||||
|
* of some depth even before it is snapped to a downbeat.
|
||||||
|
*/
|
||||||
|
function rhythmFor(energy, rng) {
|
||||||
|
if (energy > 0.72) {
|
||||||
|
// Loud material: quick cuts, but still answered by a longer hold.
|
||||||
|
return rng.pick([[4, 4, 8], [8, 4, 4], [4, 4, 4, 8], [8, 8, 4, 4], [4, 8, 4, 4]]);
|
||||||
|
}
|
||||||
|
if (energy > 0.45) {
|
||||||
|
return rng.pick([[8, 8, 16], [16, 8, 8], [8, 16, 8], [8, 8, 8, 16], [16, 8, 16, 8]]);
|
||||||
|
}
|
||||||
|
// Quiet material holds, and departs from the hold rather than the reverse.
|
||||||
|
return rng.pick([[16, 16, 8], [16, 8, 16], [16, 16, 16, 8], [8, 16, 16]]);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fit a bar pattern to real seconds.
|
||||||
|
*
|
||||||
|
* The ceiling is applied by scaling the WHOLE pattern rather than by clamping
|
||||||
|
* each entry, because clamping destroys exactly what the pattern is for: at 90
|
||||||
|
* BPM a 16-bar hold is 42 s and a 4-bar one is 10 s, and clamping both to 22
|
||||||
|
* turns 16/16/8 into a metronome again. Scaling keeps the 2:1 relationships
|
||||||
|
* that make the rhythm legible. Only after that is each entry clamped, to catch
|
||||||
|
* whatever the scale could not reconcile.
|
||||||
|
*/
|
||||||
|
function fitPattern(pattern, barSeconds) {
|
||||||
|
const unit = barSeconds > 0.2 ? barSeconds : 3;
|
||||||
|
const raw = pattern.map((bars) => bars * unit);
|
||||||
|
|
||||||
|
const hi = Math.max(...raw);
|
||||||
|
const scale = hi > MAX_SHOT_SECONDS ? MAX_SHOT_SECONDS / hi : 1;
|
||||||
|
|
||||||
|
return raw.map((s) => Math.min(MAX_SHOT_SECONDS, Math.max(MIN_SHOT_SECONDS, s * scale)));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The downbeat nearest `time` that also makes a LEGAL shot when measured from
|
||||||
|
* `from`, or null if there isn't one within tolerance.
|
||||||
|
*
|
||||||
|
* The legality bound is the point. Snapping to the merely-nearest downbeat can
|
||||||
|
* push a cut later than the ideal, and a 22-second shot snapped 0.75s late is a
|
||||||
|
* 22.75-second shot — over the ceiling the pattern was fitted to respect. So the
|
||||||
|
* search is restricted to downbeats that keep the shot inside the floor and the
|
||||||
|
* ceiling, which usually means taking the downbeat just before the ideal rather
|
||||||
|
* than the one just after. Landing on the grid matters more than landing on the
|
||||||
|
* closest line.
|
||||||
|
*/
|
||||||
|
function snapCut(from, ideal, downbeats, tolerance) {
|
||||||
|
let best = null;
|
||||||
|
let bestDist = Infinity;
|
||||||
|
for (const d of downbeats) {
|
||||||
|
if (d - from < MIN_SHOT_SECONDS) continue;
|
||||||
|
if (d - from > MAX_SHOT_SECONDS) break; // sorted: only gets worse
|
||||||
|
const dist = Math.abs(d - ideal);
|
||||||
|
if (dist < bestDist) { bestDist = dist; best = d; }
|
||||||
|
else if (d > ideal) break; // past the minimum
|
||||||
|
}
|
||||||
|
return best !== null && bestDist <= tolerance ? best : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Divide a section into shots.
|
||||||
|
*
|
||||||
|
* @param {object} section a track section (start/end/startFrame/endFrame)
|
||||||
|
* @param {object} track FeatureTrack, for fps and the bar grid
|
||||||
|
* @param {object} bias the section's bias, for energy
|
||||||
|
* @param {number} variantCount how many stage visuals the section has
|
||||||
|
* @param {Rng} rng
|
||||||
|
* @returns {Array<{index,startFrame,endFrame,variant,hardCut}>}
|
||||||
|
*/
|
||||||
|
export function planShots(section, track, bias, variantCount, rng) {
|
||||||
|
const fps = track.fps;
|
||||||
|
const duration = Math.max(0, section.end - section.start);
|
||||||
|
const barSeconds = (track.tempo.period * track.tempo.beatsPerBar) / fps;
|
||||||
|
|
||||||
|
const lengths = fitPattern(rhythmFor(bias.energy, rng), barSeconds);
|
||||||
|
const shortest = Math.min(...lengths);
|
||||||
|
|
||||||
|
// A section with only one visual to show has nothing to cut to.
|
||||||
|
const single = variantCount < 2;
|
||||||
|
|
||||||
|
// Walk the pattern, laying shots end to end from the section start. Each cut
|
||||||
|
// is then pulled onto the nearest downbeat; the tolerance stays under half
|
||||||
|
// the shortest shot so a snap can never reorder two cuts or collapse one
|
||||||
|
// onto another. Drift from snapping does not accumulate, because the next
|
||||||
|
// shot is measured from the snapped time rather than from the ideal one.
|
||||||
|
const tolerance = Math.min(barSeconds * 1.5, shortest * 0.35);
|
||||||
|
const downbeats = track.tempo.downbeats || [];
|
||||||
|
const cuts = [];
|
||||||
|
|
||||||
|
if (!single) {
|
||||||
|
let at = section.start;
|
||||||
|
for (let k = 0; k < 512; k++) {
|
||||||
|
const raw = at + lengths[k % lengths.length];
|
||||||
|
const snapped = snapCut(at, raw, downbeats, tolerance);
|
||||||
|
const time = snapped !== null ? snapped : raw;
|
||||||
|
|
||||||
|
// Stop when the remainder would be shorter than a legal shot: the
|
||||||
|
// tail belongs to the shot already running rather than becoming a
|
||||||
|
// stub. This is also what ends the loop on any section length.
|
||||||
|
if (section.end - time < MIN_SHOT_SECONDS) break;
|
||||||
|
cuts.push(time);
|
||||||
|
at = time;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Absorbing the tail can push the closing shot past the ceiling — a
|
||||||
|
// 22-second shot plus a 4-second remainder is 26. Split it. The span is
|
||||||
|
// over MAX by construction here, and MAX is more than twice MIN, so both
|
||||||
|
// halves are legal shots.
|
||||||
|
const lastCut = cuts.length ? cuts[cuts.length - 1] : section.start;
|
||||||
|
if (section.end - lastCut > MAX_SHOT_SECONDS) {
|
||||||
|
cuts.push(lastCut + (section.end - lastCut) / 2);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const bounds = [section.start, ...cuts, section.end];
|
||||||
|
const shots = [];
|
||||||
|
const lastSeen = new Array(variantCount).fill(-1);
|
||||||
|
let previousVariant = -1;
|
||||||
|
for (let i = 0; i < bounds.length - 1; i++) {
|
||||||
|
const variant = i === 0 ? 0 : pickVariant(variantCount, previousVariant, lastSeen, i, rng);
|
||||||
|
lastSeen[variant] = i;
|
||||||
|
previousVariant = variant;
|
||||||
|
shots.push({
|
||||||
|
index: i,
|
||||||
|
startFrame: i === 0 ? section.startFrame : Math.round(bounds[i] * fps),
|
||||||
|
endFrame: i === bounds.length - 2 ? section.endFrame : Math.round(bounds[i + 1] * fps),
|
||||||
|
start: bounds[i],
|
||||||
|
end: bounds[i + 1],
|
||||||
|
variant,
|
||||||
|
// A dissolve is the default. A straight cut is what makes a drop feel
|
||||||
|
// edited, but on anything calmer it reads as a glitch, so cuts are
|
||||||
|
// gated on real energy rather than sprinkled everywhere: nothing below
|
||||||
|
// the threshold ever cuts, and only the loudest material cuts often.
|
||||||
|
hardCut: bias.energy > HARD_CUT_ENERGY
|
||||||
|
&& rng.bool(Math.min(0.85, (bias.energy - HARD_CUT_ENERGY) * 2.5)),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return shots;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Next visual in the rotation.
|
||||||
|
*
|
||||||
|
* The shape is A B A C A D: the anchor comes back between companions, so the
|
||||||
|
* section reads as one idea with departures from it rather than as a playlist.
|
||||||
|
* It is a strong tendency and not a rule — strict alternation is audible as a
|
||||||
|
* pattern within about three cycles.
|
||||||
|
*
|
||||||
|
* When a companion is due, the LEAST RECENTLY SHOWN one wins. With a roster of
|
||||||
|
* four that is the difference between a section showing B, C, D and a section
|
||||||
|
* showing B twice and never reaching D.
|
||||||
|
*/
|
||||||
|
function pickVariant(variantCount, previous, lastSeen, shotIndex, rng) {
|
||||||
|
if (previous !== 0 && rng.bool(0.75)) return 0;
|
||||||
|
|
||||||
|
// A companion this section has not shown yet wins outright. Weighting it
|
||||||
|
// heavily was not enough — measured, a five-shot section still came out
|
||||||
|
// 0,2,0,2,0 about a fifth of the time, so the roster existed and the shots
|
||||||
|
// never reached it. Which unseen one is still a free choice, so the order
|
||||||
|
// varies between sections; only the coverage is guaranteed.
|
||||||
|
const unseen = [];
|
||||||
|
for (let v = 1; v < variantCount; v++) {
|
||||||
|
if (v !== previous && lastSeen[v] < 0) unseen.push(v);
|
||||||
|
}
|
||||||
|
if (unseen.length) return rng.pick(unseen);
|
||||||
|
|
||||||
|
const options = [];
|
||||||
|
const weights = [];
|
||||||
|
for (let v = 0; v < variantCount; v++) {
|
||||||
|
if (v === previous) continue;
|
||||||
|
options.push(v);
|
||||||
|
// Everything has been shown at least once: fall back to least recently
|
||||||
|
// seen, with the anchor kept in the draw so the rotation cannot become
|
||||||
|
// a rigid cycle.
|
||||||
|
weights.push(v === 0 ? 1 : 2 + (shotIndex - lastSeen[v]));
|
||||||
|
}
|
||||||
|
if (!options.length) return 0;
|
||||||
|
return rng.pickWeighted(options, weights);
|
||||||
|
}
|
||||||
654
flow-state/src/main.js
Normal file
654
flow-state/src/main.js
Normal file
@ -0,0 +1,654 @@
|
|||||||
|
import { Show } from './Show.js';
|
||||||
|
import { TimelineStrip } from './ui/TimelineStrip.js';
|
||||||
|
import { ParamPanel } from './ui/ParamPanel.js';
|
||||||
|
import { formatTime } from './audio/decode.js';
|
||||||
|
import { describeLook } from './look/LookGenerator.js';
|
||||||
|
import { toHex } from './look/palette.js';
|
||||||
|
import { applyGrainToPost, describeGrain, GRAIN_MASKS, GRAIN_MODES } from './look/grain.js';
|
||||||
|
import { renderClickTrack, audioBufferToWavBlob } from './audio/clicktrack.js';
|
||||||
|
import { Exporter, exportSegment, downloadBlob, PRESETS, isSupported } from './export/Exporter.js';
|
||||||
|
|
||||||
|
const QUALITY = {
|
||||||
|
draft: 0.5, // half resolution — for scrubbing heavy stacks
|
||||||
|
full: 1.0,
|
||||||
|
};
|
||||||
|
|
||||||
|
const dom = {
|
||||||
|
canvas: document.getElementById('canvas'),
|
||||||
|
stage: document.getElementById('stage'),
|
||||||
|
overlay: document.getElementById('overlay'),
|
||||||
|
dropzone: document.getElementById('dropzone'),
|
||||||
|
fileInput: document.getElementById('file-input'),
|
||||||
|
audio: document.getElementById('audio'),
|
||||||
|
timelineCanvas: document.getElementById('timeline-canvas'),
|
||||||
|
timeDisplay: document.getElementById('time-display'),
|
||||||
|
sectionDisplay: document.getElementById('section-display'),
|
||||||
|
panelBody: document.getElementById('panel-body'),
|
||||||
|
panelTabs: document.getElementById('panel-tabs'),
|
||||||
|
hud: document.getElementById('hud'),
|
||||||
|
toast: document.getElementById('toast'),
|
||||||
|
osd: document.getElementById('btn-osd'),
|
||||||
|
play: document.getElementById('btn-play'),
|
||||||
|
thLabel: document.getElementById('th-label'),
|
||||||
|
thName: document.getElementById('th-name'),
|
||||||
|
thProgress: document.getElementById('th-progress'),
|
||||||
|
thStep: document.getElementById('th-step'),
|
||||||
|
thFill: document.getElementById('th-fill'),
|
||||||
|
changeTrack: document.getElementById('btn-change-track'),
|
||||||
|
};
|
||||||
|
|
||||||
|
const state = {
|
||||||
|
show: new Show({ canvas: dom.canvas, width: 1280, height: 720 }),
|
||||||
|
playing: false,
|
||||||
|
quality: 'full',
|
||||||
|
loopSection: -1,
|
||||||
|
tab: 'look',
|
||||||
|
hudVisible: false,
|
||||||
|
osdVisible: true,
|
||||||
|
busy: false,
|
||||||
|
rerollSalt: 0,
|
||||||
|
lastFrameTime: 0,
|
||||||
|
fps: 0,
|
||||||
|
};
|
||||||
|
|
||||||
|
const strip = new TimelineStrip(dom.timelineCanvas, { onSeek: seekTo });
|
||||||
|
const paramPanel = new ParamPanel(document.createElement('div'), onParamChange);
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------- loading
|
||||||
|
|
||||||
|
async function loadFile(file) {
|
||||||
|
if (state.busy) return;
|
||||||
|
state.busy = true;
|
||||||
|
stopPlayback();
|
||||||
|
|
||||||
|
if (dom.overlay) {
|
||||||
|
dom.overlay.hidden = true;
|
||||||
|
dom.overlay.style.display = 'none';
|
||||||
|
}
|
||||||
|
dom.thLabel.textContent = 'loading track';
|
||||||
|
dom.thName.hidden = false;
|
||||||
|
dom.thName.textContent = file.name.replace(/\.[^/.]+$/, '');
|
||||||
|
dom.thName.title = file.name;
|
||||||
|
dom.thProgress.hidden = false;
|
||||||
|
dom.changeTrack.hidden = true;
|
||||||
|
|
||||||
|
try {
|
||||||
|
await state.show.load(file, (stage, fraction) => {
|
||||||
|
dom.thStep.textContent = stage;
|
||||||
|
dom.thFill.style.width = `${Math.round((fraction || 0) * 100)}%`;
|
||||||
|
});
|
||||||
|
|
||||||
|
dom.audio.src = URL.createObjectURL(file);
|
||||||
|
dom.thLabel.textContent = 'loaded track';
|
||||||
|
dom.thName.textContent = state.show.fileName;
|
||||||
|
dom.thName.title = state.show.fileName;
|
||||||
|
dom.thProgress.hidden = true;
|
||||||
|
dom.changeTrack.hidden = false;
|
||||||
|
dom.changeTrack.textContent = 'change track';
|
||||||
|
|
||||||
|
strip.setShow(state.show);
|
||||||
|
if (dom.osd) dom.osd.classList.toggle('on', state.osdVisible);
|
||||||
|
resize();
|
||||||
|
seekTo(0);
|
||||||
|
renderPanel();
|
||||||
|
document.title = `flow-state · ${state.show.fileName}`;
|
||||||
|
console.info('[flow-state]', describeLook(state.show.look));
|
||||||
|
} catch (err) {
|
||||||
|
dom.thLabel.textContent = 'analysis failed';
|
||||||
|
dom.thStep.textContent = `failed: ${err.message}`;
|
||||||
|
console.error(err);
|
||||||
|
dom.changeTrack.hidden = false;
|
||||||
|
dom.changeTrack.textContent = 'try again';
|
||||||
|
if (dom.overlay) {
|
||||||
|
dom.overlay.hidden = false;
|
||||||
|
dom.overlay.style.display = 'flex';
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
state.busy = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (dom.dropzone) dom.dropzone.addEventListener('click', () => dom.fileInput.click());
|
||||||
|
if (dom.changeTrack) dom.changeTrack.addEventListener('click', () => dom.fileInput.click());
|
||||||
|
dom.fileInput.addEventListener('change', (e) => {
|
||||||
|
if (e.target.files[0]) loadFile(e.target.files[0]);
|
||||||
|
});
|
||||||
|
document.addEventListener('dragover', (e) => e.preventDefault());
|
||||||
|
document.addEventListener('drop', (e) => {
|
||||||
|
e.preventDefault();
|
||||||
|
const file = [...e.dataTransfer.files].find(
|
||||||
|
(f) => f.type.startsWith('audio/') || /\.(mp3|flac|wav|ogg|m4a)$/i.test(f.name));
|
||||||
|
if (file) loadFile(file);
|
||||||
|
});
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------- transport
|
||||||
|
|
||||||
|
function seekTo(frame, options = {}) {
|
||||||
|
if (!state.show.ready) return;
|
||||||
|
const clamped = Math.max(0, Math.min(state.show.frameCount - 1, Math.round(frame)));
|
||||||
|
dom.audio.currentTime = clamped / state.show.fps;
|
||||||
|
state.show.seek(clamped, options);
|
||||||
|
strip.setFrame(clamped);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Force the transport to a stopped state.
|
||||||
|
*
|
||||||
|
* Loading a track replaces `audio.src`, which stops playback without telling
|
||||||
|
* anyone — so `state.playing` stayed true, the button stayed on ❚❚, and the
|
||||||
|
* first click after a track change only toggled the flag back rather than
|
||||||
|
* starting anything. Every path that stops playback behind the UI's back has
|
||||||
|
* to come through here.
|
||||||
|
*/
|
||||||
|
function stopPlayback() {
|
||||||
|
state.playing = false;
|
||||||
|
dom.audio.pause();
|
||||||
|
dom.play.textContent = '▶';
|
||||||
|
}
|
||||||
|
|
||||||
|
function togglePlay() {
|
||||||
|
if (!state.show.ready) return;
|
||||||
|
state.playing = !state.playing;
|
||||||
|
if (state.playing) {
|
||||||
|
dom.audio.play().catch((err) => {
|
||||||
|
console.warn('[flow-state] playback failed:', err);
|
||||||
|
state.playing = false;
|
||||||
|
dom.play.textContent = '▶';
|
||||||
|
});
|
||||||
|
} else {
|
||||||
|
dom.audio.pause();
|
||||||
|
}
|
||||||
|
dom.play.textContent = state.playing ? '❚❚' : '▶';
|
||||||
|
}
|
||||||
|
|
||||||
|
function jumpSection(direction) {
|
||||||
|
if (!state.show.ready) return;
|
||||||
|
seekTo(state.show.track.boundaryFrame(state.show.timeline.frame, direction));
|
||||||
|
}
|
||||||
|
|
||||||
|
function toggleLoop() {
|
||||||
|
if (!state.show.ready) return;
|
||||||
|
const index = state.show.track.sectionIndexAt(state.show.timeline.frame);
|
||||||
|
state.loopSection = state.loopSection === index ? -1 : index;
|
||||||
|
strip.setLoopSection(state.loopSection);
|
||||||
|
}
|
||||||
|
|
||||||
|
dom.play.addEventListener('click', togglePlay);
|
||||||
|
document.getElementById('btn-prev-section').addEventListener('click', () => jumpSection(-1));
|
||||||
|
document.getElementById('btn-next-section').addEventListener('click', () => jumpSection(1));
|
||||||
|
document.getElementById('btn-loop').addEventListener('click', toggleLoop);
|
||||||
|
document.getElementById('btn-hud').addEventListener('click', () => {
|
||||||
|
state.hudVisible = !state.hudVisible;
|
||||||
|
dom.hud.hidden = !state.hudVisible;
|
||||||
|
});
|
||||||
|
if (dom.osd) dom.osd.addEventListener('click', () => setOSDVisible(!state.osdVisible));
|
||||||
|
document.getElementById('sel-quality').addEventListener('change', (e) => {
|
||||||
|
state.quality = e.target.value;
|
||||||
|
resize();
|
||||||
|
});
|
||||||
|
|
||||||
|
function setOSDVisible(visible) {
|
||||||
|
state.osdVisible = !!visible;
|
||||||
|
state.show.setOSDEnabled(state.osdVisible);
|
||||||
|
if (dom.osd) dom.osd.classList.toggle('on', state.osdVisible);
|
||||||
|
}
|
||||||
|
|
||||||
|
document.addEventListener('keydown', (e) => {
|
||||||
|
if (e.target.tagName === 'INPUT' || e.target.tagName === 'SELECT') return;
|
||||||
|
switch (e.key) {
|
||||||
|
case ' ': e.preventDefault(); togglePlay(); break;
|
||||||
|
case 'ArrowLeft': jumpSection(-1); break;
|
||||||
|
case 'ArrowRight': jumpSection(1); break;
|
||||||
|
case 'l': case 'L': toggleLoop(); break;
|
||||||
|
case 'd': case 'D':
|
||||||
|
state.hudVisible = !state.hudVisible;
|
||||||
|
dom.hud.hidden = !state.hudVisible;
|
||||||
|
break;
|
||||||
|
case 'o': case 'O': setOSDVisible(!state.osdVisible); break;
|
||||||
|
case ',': seekTo(state.show.timeline.frame - 1); break;
|
||||||
|
case '.': seekTo(state.show.timeline.frame + 1); break;
|
||||||
|
default: break;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------- look edits
|
||||||
|
|
||||||
|
document.getElementById('btn-reroll').addEventListener('click', () => {
|
||||||
|
if (!state.show.ready) return;
|
||||||
|
state.show.reroll((state.show.look.seed ^ (++state.rerollSalt * 0x9e3779b9)) >>> 0);
|
||||||
|
refreshAfterLookChange();
|
||||||
|
});
|
||||||
|
|
||||||
|
document.getElementById('btn-reroll-section').addEventListener('click', () => {
|
||||||
|
if (!state.show.ready) return;
|
||||||
|
const index = state.show.track.sectionIndexAt(state.show.timeline.frame);
|
||||||
|
state.show.rerollSection(index, ++state.rerollSalt);
|
||||||
|
refreshAfterLookChange();
|
||||||
|
});
|
||||||
|
|
||||||
|
document.getElementById('btn-lock').addEventListener('click', () => {
|
||||||
|
if (!state.show.ready) return;
|
||||||
|
const index = state.show.track.sectionIndexAt(state.show.timeline.frame);
|
||||||
|
const section = state.show.look.sections[index];
|
||||||
|
section.locked = !section.locked;
|
||||||
|
renderPanel();
|
||||||
|
});
|
||||||
|
|
||||||
|
function refreshAfterLookChange() {
|
||||||
|
state.show.seek(state.show.timeline.frame);
|
||||||
|
renderPanel();
|
||||||
|
}
|
||||||
|
|
||||||
|
function onParamChange(name, value) {
|
||||||
|
if (!state.show.ready) return;
|
||||||
|
const index = state.show.track.sectionIndexAt(state.show.timeline.frame);
|
||||||
|
state.show.setSectionParam(index, name, value);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------- panel
|
||||||
|
|
||||||
|
dom.panelTabs.addEventListener('click', (e) => {
|
||||||
|
const tab = e.target.dataset.tab;
|
||||||
|
if (!tab) return;
|
||||||
|
state.tab = tab;
|
||||||
|
[...dom.panelTabs.children].forEach((b) => b.classList.toggle('active', b.dataset.tab === tab));
|
||||||
|
renderPanel();
|
||||||
|
});
|
||||||
|
|
||||||
|
function renderPanel() {
|
||||||
|
if (!state.show.ready) return;
|
||||||
|
const show = state.show;
|
||||||
|
const index = show.track.sectionIndexAt(show.timeline.frame);
|
||||||
|
const section = show.look.sections[index];
|
||||||
|
dom.panelBody.innerHTML = '';
|
||||||
|
dom.panelBody.oninput = null;
|
||||||
|
|
||||||
|
if (state.tab === 'scene') {
|
||||||
|
paramPanel.container = dom.panelBody;
|
||||||
|
paramPanel.build(section.layers[0].module, section.layers[0].params);
|
||||||
|
|
||||||
|
// The section's stage visuals, with the one currently on screen marked.
|
||||||
|
// Params above edit the anchor (variant 0) — the image the section opens
|
||||||
|
// and returns to.
|
||||||
|
if (section.variants && section.variants.length > 1) {
|
||||||
|
const active = show.arc.state.variant || 0;
|
||||||
|
const shots = section.shots || [];
|
||||||
|
const list = document.createElement('div');
|
||||||
|
list.className = 'pp-reactive';
|
||||||
|
list.innerHTML = `<div class="pp-sub">stage visuals · ${shots.length} shots</div>` +
|
||||||
|
section.variants.map((stack, v) =>
|
||||||
|
`<div class="pp-react-row${v === active ? ' current' : ''}">` +
|
||||||
|
`<span>${v === 0 ? '●' : '○'} ${stack[0].module.name}</span>` +
|
||||||
|
`<span class="pp-feature">${shots.filter((s) => s.variant === v).length}×</span>` +
|
||||||
|
`</div>`).join('');
|
||||||
|
dom.panelBody.appendChild(list);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (section.layers.length > 1) {
|
||||||
|
const note = document.createElement('div');
|
||||||
|
note.className = 'pp-reactive';
|
||||||
|
note.innerHTML = '<div class="pp-sub">accent layer</div>' +
|
||||||
|
section.layers.slice(1).map((l) =>
|
||||||
|
`<div class="pp-react-row"><span>${l.module.name}</span>` +
|
||||||
|
`<span class="pp-feature">${l.blend}</span>` +
|
||||||
|
`<span class="pp-amount">${l.opacity.toFixed(2)}</span></div>`).join('');
|
||||||
|
dom.panelBody.appendChild(note);
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (state.tab === 'look') {
|
||||||
|
const summary = show.track.summary;
|
||||||
|
// The track's production design. Scenes that cannot express what it is
|
||||||
|
// built on were never cast — see look/Personality.js.
|
||||||
|
const personality = show.look.personality;
|
||||||
|
dom.panelBody.innerHTML = `
|
||||||
|
<div class="pp-heading"><span class="pp-name">${show.fileName || 'track'}</span></div>
|
||||||
|
<div class="kv"><span>seed</span><b>${show.look.seed.toString(16)}</b></div>
|
||||||
|
<div class="kv"><span>bpm</span><b>${summary.bpm.toFixed(1)}</b></div>
|
||||||
|
<div class="kv"><span>tempo conf.</span><b>${show.track.tempo.confidence.toFixed(2)}</b></div>
|
||||||
|
<div class="kv"><span>duration</span><b>${formatTime(show.duration)}</b></div>
|
||||||
|
<div class="kv"><span>sections</span><b>${show.track.sections.length}</b></div>
|
||||||
|
<div class="kv"><span>scheme</span><b>${show.look.paletteScheme}</b></div>
|
||||||
|
<div class="kv"><span>built on</span><b>${personality.signature.join(' + ') || 'nothing'}</b></div>
|
||||||
|
<div class="kv"><span>form</span><b>${personality.shape.sides || 'round'}${
|
||||||
|
personality.shape.sides ? '-sided' : ''}</b></div>
|
||||||
|
<div class="kv"><span>camera</span><b>${(personality.camera.driftRate * 100).toFixed(1)} drift · ${
|
||||||
|
personality.camera.spin >= 0 ? '+' : ''}${personality.camera.spin.toFixed(3)} spin</b></div>
|
||||||
|
<div class="kv"><span>art</span><b>${personality.style.symmetry > 1
|
||||||
|
? personality.style.symmetry + '-fold · ' : ''}line ${personality.style.lineWeight.toFixed(2)}</b></div>
|
||||||
|
<div class="kv"><span>brightness</span><b>${summary.meanCentroid.toFixed(3)}</b></div>
|
||||||
|
<div class="kv"><span>dynamics</span><b>${summary.dynamicRange.toFixed(3)}</b></div>
|
||||||
|
<div class="swatches">${show.look.palette.map((c) =>
|
||||||
|
`<span class="sw" style="background:${toHex(c)}" title="${toHex(c)}"></span>`).join('')}</div>
|
||||||
|
<div class="pp-sub">sections</div>
|
||||||
|
${show.look.sections.map((s, i) => `
|
||||||
|
<div class="kv ${i === index ? 'current' : ''}">
|
||||||
|
<span>${s.kind}${s.locked ? ' 🔒' : ''}
|
||||||
|
${s.shots ? `<i class="dim">${s.shots.length} shots</i>` : ''}</span>
|
||||||
|
<b>${(s.variants || [s.layers]).map((v) => v[0].module.name).join(' / ')}</b>
|
||||||
|
</div>`).join('')}
|
||||||
|
<button id="btn-clicktrack" class="wide">download click track</button>
|
||||||
|
<div class="hint">Mixes clicks onto the detected beat grid. If they don't sit on
|
||||||
|
the beat, tempo detection is wrong and everything downstream inherits it.</div>`;
|
||||||
|
document.getElementById('btn-clicktrack').addEventListener('click', downloadClickTrack);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (state.tab === 'post') {
|
||||||
|
const post = show.look.post;
|
||||||
|
const fb = show.look.feedback;
|
||||||
|
const wide = new Set(['contrast', 'saturation', 'exposure']);
|
||||||
|
// Grain has its own block below: its fields are a mode, a mask id and a
|
||||||
|
// pixel size, none of which are a 0..1 slider.
|
||||||
|
const rows = Object.entries(post)
|
||||||
|
.filter(([key]) => !key.startsWith('grain'))
|
||||||
|
.map(([key, value]) => `
|
||||||
|
<div class="pp-row">
|
||||||
|
<label class="pp-label">${key}</label>
|
||||||
|
<input class="pp-input" type="range" min="0" max="${wide.has(key) ? 2 : 1}"
|
||||||
|
step="0.01" value="${value}" data-post="${key}">
|
||||||
|
<span class="pp-value">${(+value).toFixed(2)}</span>
|
||||||
|
</div>`).join('');
|
||||||
|
const fbRows = Object.entries(fb).map(([key, value]) => `
|
||||||
|
<div class="pp-row">
|
||||||
|
<label class="pp-label">${key}</label>
|
||||||
|
<input class="pp-input" type="range"
|
||||||
|
min="${key === 'rotate' ? -0.02 : key === 'zoom' ? 0.97 : 0}"
|
||||||
|
max="${key === 'zoom' ? 1.03 : key === 'rotate' ? 0.02 : 1}"
|
||||||
|
step="0.001" value="${value}" data-feedback="${key}">
|
||||||
|
<span class="pp-value">${(+value).toFixed(3)}</span>
|
||||||
|
</div>`).join('');
|
||||||
|
dom.panelBody.innerHTML =
|
||||||
|
`<div class="pp-sub">post</div>${rows}` +
|
||||||
|
`<div class="pp-sub">grain</div>${grainRows(show.look.grain)}` +
|
||||||
|
`<div class="pp-sub">feedback</div>${fbRows}`;
|
||||||
|
dom.panelBody.oninput = (e) => {
|
||||||
|
const t = e.target;
|
||||||
|
if (t.dataset.post) {
|
||||||
|
post[t.dataset.post] = +t.value;
|
||||||
|
t.nextElementSibling.textContent = (+t.value).toFixed(2);
|
||||||
|
}
|
||||||
|
if (t.dataset.feedback) {
|
||||||
|
fb[t.dataset.feedback] = +t.value;
|
||||||
|
t.nextElementSibling.textContent = (+t.value).toFixed(3);
|
||||||
|
}
|
||||||
|
if (t.dataset.grain) {
|
||||||
|
const key = t.dataset.grain;
|
||||||
|
if (key === 'kinds') {
|
||||||
|
const kinds = new Set(show.look.grain.kinds);
|
||||||
|
if (t.checked) kinds.add(t.value); else kinds.delete(t.value);
|
||||||
|
show.look.grain.kinds = [...kinds];
|
||||||
|
} else {
|
||||||
|
show.look.grain[key] = t.tagName === 'SELECT' ? t.value : +t.value;
|
||||||
|
}
|
||||||
|
// Turning grain on for a track that was generated without it
|
||||||
|
// would otherwise select a mode and still show nothing.
|
||||||
|
if (show.look.grain.mode !== 'off' && show.look.grain.amount <= 0) {
|
||||||
|
show.look.grain.amount = 0.04;
|
||||||
|
}
|
||||||
|
applyGrainToPost(show.look.grain, post);
|
||||||
|
// The mode decides which of the other controls exist, so it is
|
||||||
|
// the one edit that has to rebuild the panel.
|
||||||
|
if (key === 'mode') { renderPanel(); return; }
|
||||||
|
if (t.nextElementSibling) {
|
||||||
|
t.nextElementSibling.textContent = (+t.value).toFixed(3);
|
||||||
|
}
|
||||||
|
const desc = dom.panelBody.querySelector('#grain-desc');
|
||||||
|
if (desc) desc.textContent = describeGrain(show.look.grain);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
dom.panelBody.onchange = dom.panelBody.oninput;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (state.tab === 'export') {
|
||||||
|
dom.panelBody.innerHTML = `
|
||||||
|
<div class="pp-sub">export</div>
|
||||||
|
<div class="pp-row">
|
||||||
|
<label class="pp-label">preset</label>
|
||||||
|
<select id="sel-preset" class="pp-input">
|
||||||
|
${Object.keys(PRESETS).map((k) =>
|
||||||
|
`<option ${k === '1080p' ? 'selected' : ''}>${k}</option>`).join('')}
|
||||||
|
</select>
|
||||||
|
<span class="pp-value"></span>
|
||||||
|
</div>
|
||||||
|
<div id="export-status" class="hint">${isSupported()
|
||||||
|
? 'WebCodecs available. Test render first — it uses the same encoder at the same settings.'
|
||||||
|
: 'WebCodecs VideoEncoder is unavailable here; export will not work in this browser.'}</div>
|
||||||
|
<div class="an-bar"><div class="an-fill" id="export-fill"></div></div>`;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Grain controls for the post tab.
|
||||||
|
*
|
||||||
|
* Grain is the one part of the grade with a shape rather than a level — when it
|
||||||
|
* is present, how coarse it is, how often it refreshes and where it lands — so
|
||||||
|
* it gets its own block instead of five sliders that all read 0..1.
|
||||||
|
*/
|
||||||
|
function grainRows(grain) {
|
||||||
|
const slider = (key, min, max, step, value) => `
|
||||||
|
<div class="pp-row">
|
||||||
|
<label class="pp-label">${key}</label>
|
||||||
|
<input class="pp-input" type="range" min="${min}" max="${max}" step="${step}"
|
||||||
|
value="${value}" data-grain="${key}">
|
||||||
|
<span class="pp-value">${(+value).toFixed(3)}</span>
|
||||||
|
</div>`;
|
||||||
|
|
||||||
|
const modeRow = `
|
||||||
|
<div class="pp-row">
|
||||||
|
<label class="pp-label">mode</label>
|
||||||
|
<select class="pp-input" data-grain="mode">
|
||||||
|
${GRAIN_MODES.map((m) =>
|
||||||
|
`<option ${m === grain.mode ? 'selected' : ''}>${m}</option>`).join('')}
|
||||||
|
</select>
|
||||||
|
<span class="pp-value"></span>
|
||||||
|
</div>`;
|
||||||
|
|
||||||
|
if (grain.mode === 'off') {
|
||||||
|
return modeRow + `<div class="hint" id="grain-desc">${describeGrain(grain)}</div>`;
|
||||||
|
}
|
||||||
|
|
||||||
|
const maskRow = `
|
||||||
|
<div class="pp-row">
|
||||||
|
<label class="pp-label">mask</label>
|
||||||
|
<select class="pp-input" data-grain="mask">
|
||||||
|
${Object.keys(GRAIN_MASKS).map((m) =>
|
||||||
|
`<option ${m === grain.mask ? 'selected' : ''}>${m}</option>`).join('')}
|
||||||
|
</select>
|
||||||
|
<span class="pp-value"></span>
|
||||||
|
</div>`;
|
||||||
|
|
||||||
|
const kindsRow = grain.mode !== 'sections' ? '' : `
|
||||||
|
<div class="pp-row">
|
||||||
|
<label class="pp-label">sections</label>
|
||||||
|
<span class="pp-input">
|
||||||
|
${['intro', 'build', 'drop', 'sustain', 'breakdown', 'outro'].map((k) => `
|
||||||
|
<label class="dim"><input type="checkbox" data-grain="kinds" value="${k}"
|
||||||
|
${grain.kinds.includes(k) ? 'checked' : ''}>${k}</label>`).join(' ')}
|
||||||
|
</span>
|
||||||
|
</div>`;
|
||||||
|
|
||||||
|
const swellRows = grain.mode !== 'swell' ? ''
|
||||||
|
: slider('period', 4, 60, 1, grain.period) + slider('duty', 0.05, 1, 0.01, grain.duty);
|
||||||
|
|
||||||
|
return modeRow
|
||||||
|
+ slider('amount', 0, 0.25, 0.002, grain.amount)
|
||||||
|
+ slider('scale', 1, 8, 0.5, grain.scale)
|
||||||
|
+ slider('rate', 1, 8, 1, grain.rate)
|
||||||
|
+ maskRow
|
||||||
|
+ slider('chroma', 0, 1, 0.01, grain.chroma)
|
||||||
|
+ kindsRow
|
||||||
|
+ swellRows
|
||||||
|
+ `<div class="hint" id="grain-desc">${describeGrain(grain)}</div>`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------- export
|
||||||
|
|
||||||
|
function currentPreset() {
|
||||||
|
const el = document.getElementById('sel-preset');
|
||||||
|
return el ? el.value : '1080p';
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Surface a message over the stage.
|
||||||
|
*
|
||||||
|
* Errors stay until dismissed: an export that fails after several minutes of
|
||||||
|
* rendering must not scroll past unnoticed, which is exactly what the status
|
||||||
|
* line inside the export panel allowed.
|
||||||
|
*/
|
||||||
|
function showToast(message, { ok = false, timeout = 0 } = {}) {
|
||||||
|
if (!dom.toast) return;
|
||||||
|
clearTimeout(showToast._timer);
|
||||||
|
dom.toast.textContent = message;
|
||||||
|
dom.toast.classList.toggle('ok', ok);
|
||||||
|
dom.toast.hidden = false;
|
||||||
|
if (timeout) showToast._timer = setTimeout(() => { dom.toast.hidden = true; }, timeout);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (dom.toast) dom.toast.addEventListener('click', () => { dom.toast.hidden = true; });
|
||||||
|
|
||||||
|
function exportProgress(p) {
|
||||||
|
const status = document.getElementById('export-status');
|
||||||
|
const fill = document.getElementById('export-fill');
|
||||||
|
if (status) status.textContent = `${p.stage} — ${Math.round(p.fraction * 100)}% (${p.frame}/${p.total})`;
|
||||||
|
if (fill) fill.style.width = `${Math.round(p.fraction * 100)}%`;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function runExport(segment) {
|
||||||
|
if (!state.show.ready || state.busy) return;
|
||||||
|
if (state.playing) togglePlay();
|
||||||
|
state.busy = true;
|
||||||
|
state.tab = 'export';
|
||||||
|
[...dom.panelTabs.children].forEach((b) => b.classList.toggle('active', b.dataset.tab === 'export'));
|
||||||
|
renderPanel();
|
||||||
|
|
||||||
|
if (dom.toast) dom.toast.hidden = true;
|
||||||
|
try {
|
||||||
|
const preset = currentPreset();
|
||||||
|
const exporter = new Exporter(state.show);
|
||||||
|
const blob = segment
|
||||||
|
? await exportSegment(state.show, state.show.timeline.frame,
|
||||||
|
{ seconds: 20, preset, onProgress: exportProgress, exporter })
|
||||||
|
: await exporter.export({ preset, onProgress: exportProgress });
|
||||||
|
|
||||||
|
const suffix = segment ? `-segment-${state.show.timeline.frame}` : '';
|
||||||
|
downloadBlob(blob, `${state.show.fileName || 'flow-state'}${suffix}-${preset}.mp4`);
|
||||||
|
const status = document.getElementById('export-status');
|
||||||
|
// Codec fallbacks must never be silent: a video that quietly lost its
|
||||||
|
// audio track is worse than one that says so.
|
||||||
|
const warnings = (exporter.warnings || []).join(' · ');
|
||||||
|
if (status) {
|
||||||
|
status.textContent = `done — ${(blob.size / 1e6).toFixed(1)} MB` +
|
||||||
|
(warnings ? ` (${warnings})` : '');
|
||||||
|
}
|
||||||
|
showToast(`export done — ${(blob.size / 1e6).toFixed(1)} MB` +
|
||||||
|
(warnings ? `\n${warnings}` : ''), { ok: true, timeout: 6000 });
|
||||||
|
} catch (err) {
|
||||||
|
const status = document.getElementById('export-status');
|
||||||
|
if (status) status.textContent = `failed: ${err.message}`;
|
||||||
|
showToast(`export failed\n${err.message}`);
|
||||||
|
console.error(err);
|
||||||
|
} finally {
|
||||||
|
state.busy = false;
|
||||||
|
resize();
|
||||||
|
state.show.seek(state.show.timeline.frame);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
document.getElementById('btn-export').addEventListener('click', () => runExport(false));
|
||||||
|
document.getElementById('btn-segment').addEventListener('click', () => runExport(true));
|
||||||
|
|
||||||
|
async function downloadClickTrack() {
|
||||||
|
if (!state.show.ready) return;
|
||||||
|
const button = document.getElementById('btn-clicktrack');
|
||||||
|
button.textContent = 'rendering…';
|
||||||
|
try {
|
||||||
|
const buffer = await renderClickTrack(state.show.audioBuffer, state.show.track.tempo);
|
||||||
|
downloadBlob(audioBufferToWavBlob(buffer), `${state.show.fileName}-clicktrack.wav`);
|
||||||
|
button.textContent = 'download click track';
|
||||||
|
} catch (err) {
|
||||||
|
button.textContent = `failed: ${err.message}`;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------- loop
|
||||||
|
|
||||||
|
function resize() {
|
||||||
|
const rect = dom.stage.getBoundingClientRect();
|
||||||
|
const scale = QUALITY[state.quality];
|
||||||
|
const width = Math.max(64, Math.round(rect.width * scale));
|
||||||
|
const height = Math.max(36, Math.round(((rect.width * 9) / 16) * scale));
|
||||||
|
state.show.setSize(width, height);
|
||||||
|
dom.canvas.style.width = '100%';
|
||||||
|
dom.canvas.style.height = 'auto';
|
||||||
|
strip.resize();
|
||||||
|
}
|
||||||
|
window.addEventListener('resize', resize);
|
||||||
|
|
||||||
|
let lastPanelSection = -1;
|
||||||
|
let lastRenderedFrame = -1;
|
||||||
|
|
||||||
|
function frame(now) {
|
||||||
|
requestAnimationFrame(frame);
|
||||||
|
const show = state.show;
|
||||||
|
if (!show.ready || state.busy) return;
|
||||||
|
|
||||||
|
if (state.playing) {
|
||||||
|
if (state.loopSection >= 0) {
|
||||||
|
const section = show.track.sections[state.loopSection];
|
||||||
|
if (show.timeline.frame >= section.endFrame - 1) {
|
||||||
|
dom.audio.currentTime = section.startFrame / show.fps;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
show.timeline.syncToAudio(dom.audio.currentTime);
|
||||||
|
if (dom.audio.ended) stopPlayback();
|
||||||
|
}
|
||||||
|
|
||||||
|
if (state.playing || show.timeline.frame !== lastRenderedFrame) {
|
||||||
|
show.present(show.renderFrame(show.timeline.frame));
|
||||||
|
lastRenderedFrame = show.timeline.frame;
|
||||||
|
}
|
||||||
|
|
||||||
|
strip.setFrame(show.timeline.frame);
|
||||||
|
strip.draw();
|
||||||
|
|
||||||
|
const dt = now - state.lastFrameTime;
|
||||||
|
state.lastFrameTime = now;
|
||||||
|
if (dt > 0) state.fps = state.fps * 0.9 + (1000 / dt) * 0.1;
|
||||||
|
|
||||||
|
dom.timeDisplay.textContent = `${formatTime(show.timeline.time)} / ${formatTime(show.duration)}`;
|
||||||
|
|
||||||
|
const sectionIndex = show.track.sectionIndexAt(show.timeline.frame);
|
||||||
|
const arc = show.arc.state;
|
||||||
|
dom.sectionDisplay.textContent =
|
||||||
|
`${arc.kind || ''} · ${arc.sceneName || ''}` +
|
||||||
|
`${arc.crossfade > 0 ? ` · fade ${arc.crossfade.toFixed(2)}` : ''}`;
|
||||||
|
|
||||||
|
if (sectionIndex !== lastPanelSection) {
|
||||||
|
lastPanelSection = sectionIndex;
|
||||||
|
if (state.tab === 'scene' || state.tab === 'look') renderPanel();
|
||||||
|
}
|
||||||
|
|
||||||
|
if (state.hudVisible) {
|
||||||
|
const f = show.track.at(show.timeline.frame);
|
||||||
|
dom.hud.innerHTML =
|
||||||
|
`<div>${state.fps.toFixed(0)} fps · frame ${show.timeline.frame}/${show.frameCount}</div>` +
|
||||||
|
`<div>${show.engine.width}×${show.engine.height} · ${state.quality}</div>` +
|
||||||
|
`<div>section ${sectionIndex} ${arc.kind} · ${arc.sceneName}</div>` +
|
||||||
|
`<div>layers ${show.arc.activeLayers.length} · build ${(f.buildSlope || 0).toFixed(2)}</div>` +
|
||||||
|
`<div>loud ${f.loudness.toFixed(2)} low ${f.bandLow.toFixed(2)} high ${f.bandHigh.toFixed(2)}</div>` +
|
||||||
|
`<div>beat ${f.beat.toFixed(2)} bar ${f.barPhase.toFixed(2)} flux ${f.flux.toFixed(2)}</div>`;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Dev-only handle for the console and for browser automation. The render loop is
|
||||||
|
// requestAnimationFrame-driven, and rAF does not fire in some headless/automated
|
||||||
|
// contexts, so `tick()` provides a way to advance the app by hand.
|
||||||
|
if (import.meta.env && import.meta.env.DEV) {
|
||||||
|
window.__flowState = { state, dom, strip, tick: (t) => frame(t ?? 0), resize, seekTo, renderPanel };
|
||||||
|
}
|
||||||
|
|
||||||
|
requestAnimationFrame(frame);
|
||||||
|
resize();
|
||||||
300
flow-state/src/params/schema.js
Normal file
300
flow-state/src/params/schema.js
Normal file
@ -0,0 +1,300 @@
|
|||||||
|
// Declarative parameter schema.
|
||||||
|
//
|
||||||
|
// This is the load-bearing abstraction for library scale. One declaration drives:
|
||||||
|
// 1. uniform binding (Layer)
|
||||||
|
// 2. generated UI controls (ui/ParamPanel)
|
||||||
|
// 3. seeded per-track sampling (look/LookGenerator)
|
||||||
|
// 4. arc automation (look/ArcDriver)
|
||||||
|
// 5. save/load of presets
|
||||||
|
//
|
||||||
|
// Adding a scene therefore costs a shader plus a params block, and nothing else.
|
||||||
|
// tools/lint-scenes.js machine-checks every declaration against its shader source.
|
||||||
|
|
||||||
|
export const PARAM_TYPES = ['float', 'int', 'bool', 'vec2', 'palette'];
|
||||||
|
|
||||||
|
// RATE PARAMS
|
||||||
|
//
|
||||||
|
// A param flagged `rate: true` is one the shader multiplies absolute time by —
|
||||||
|
// `u_time * u_speed` and friends. Those must never be modulated per frame, by
|
||||||
|
// audio reactivity or by LFO drift, and the engine enforces it.
|
||||||
|
//
|
||||||
|
// The reason is that phase is `u_time * rate`, so changing the rate at time T
|
||||||
|
// jumps the phase by `T * Δrate`. Sixty seconds into a track, a wobble of 0.05
|
||||||
|
// throws the phase by three whole units between one frame and the next — and it
|
||||||
|
// gets worse the longer the track runs. The visible result is high-frequency
|
||||||
|
// flicker that looks like the scene is broken, and which measured 6 flashes per
|
||||||
|
// second on Classic Wave, twice the WCAG 2.3.1 ceiling.
|
||||||
|
//
|
||||||
|
// A scene that wants audio-driven motion should add a bounded term rather than
|
||||||
|
// scaling the clock: `u_time * u_speed + u_bandLow * 2.0` is continuous;
|
||||||
|
// `u_time * (u_speed + u_bandLow)` is not.
|
||||||
|
export const RATE_FLAG = 'rate';
|
||||||
|
|
||||||
|
// A param flagged `slowAxis: true` is the one the arc driver walks from one end
|
||||||
|
// of its range to the other across the WHOLE track — the scene's long journey,
|
||||||
|
// as opposed to the drift LFO's wobble. See ArcDriver._slowAxisFor.
|
||||||
|
//
|
||||||
|
// It has to be declared rather than guessed. Measured across the churning
|
||||||
|
// scenes, which param you pick decides everything: moving Moiré Grid's `width`
|
||||||
|
// changes its time-averaged structure by 0.110, and moving its `offset` by
|
||||||
|
// 0.002. A randomly chosen param is overwhelmingly likely to be the second kind,
|
||||||
|
// which is why the first version of the slow axis measured as doing nothing at
|
||||||
|
// all.
|
||||||
|
export const SLOW_AXIS_FLAG = 'slowAxis';
|
||||||
|
|
||||||
|
/** Valid feature names a `reactive` entry may reference. Lint enforces this. */
|
||||||
|
export const REACTIVE_FEATURES = [
|
||||||
|
'loudness', 'rms',
|
||||||
|
'bandSub', 'bandLow', 'bandMid', 'bandHigh', 'bandAir',
|
||||||
|
'flux', 'centroid', 'flatness', 'width',
|
||||||
|
'beat', 'beatPhase', 'barPhase', 'phrasePhase',
|
||||||
|
'sectionProgress', 'sectionEnergy', 'buildSlope',
|
||||||
|
];
|
||||||
|
|
||||||
|
export const REACTIVE_RESPONSES = ['linear', 'spike', 'smooth', 'inverse'];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Personality traits a scene can honour. See look/Personality.js.
|
||||||
|
*
|
||||||
|
* This is a CONTRACT, not a hint: a track built on `shape` will only cast scenes
|
||||||
|
* that declare `shape`, and it will cast them believing they actually stamp the
|
||||||
|
* signature form. Declaring a trait a scene ignores is worse than declaring
|
||||||
|
* none, because the disqualification rule is the only thing keeping off-design
|
||||||
|
* scenes out of a track.
|
||||||
|
*/
|
||||||
|
export const TRAIT_NAMES = ['shape', 'camera', 'space', 'style'];
|
||||||
|
|
||||||
|
export function defaultValue(def) {
|
||||||
|
if (def.default !== undefined) return def.default;
|
||||||
|
switch (def.type) {
|
||||||
|
case 'bool': return false;
|
||||||
|
case 'int': return Math.round(def.range ? def.range[0] : 0);
|
||||||
|
case 'vec2': return [0, 0];
|
||||||
|
case 'palette': return null; // supplied by the look, not sampled here
|
||||||
|
default: return def.range ? def.range[0] : 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function defaultValues(module) {
|
||||||
|
const out = {};
|
||||||
|
for (const [name, def] of Object.entries(module.params || {})) {
|
||||||
|
out[name] = defaultValue(def);
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function clampValue(def, value) {
|
||||||
|
if (def.type === 'bool') return !!value;
|
||||||
|
if (def.type === 'palette') return value;
|
||||||
|
if (def.type === 'vec2') {
|
||||||
|
const [lo, hi] = def.range || [0, 1];
|
||||||
|
return [Math.min(hi, Math.max(lo, value[0])), Math.min(hi, Math.max(lo, value[1]))];
|
||||||
|
}
|
||||||
|
const [lo, hi] = def.range || [0, 1];
|
||||||
|
let v = Math.min(hi, Math.max(lo, value));
|
||||||
|
if (def.type === 'int') v = Math.round(v);
|
||||||
|
return v;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Sample a full parameter set from the declared ranges.
|
||||||
|
*
|
||||||
|
* `bias` (0..1 per key, optional) nudges sampling toward the top of a range —
|
||||||
|
* this is how a track's measured character reaches the parameters without every
|
||||||
|
* scene needing to know about audio features. `energy: 0.8` on a hard track
|
||||||
|
* pushes density-ish params up without pinning them, so seed variation survives.
|
||||||
|
*
|
||||||
|
* `temperament` is the track's own hand on the same dials — see
|
||||||
|
* look/Personality.js. Bias comes from the SECTION and is therefore nearly the
|
||||||
|
* same for every track's drop; temperament comes from the TRACK and is not.
|
||||||
|
* Without it, one scene cast in two different videos sampled around the same
|
||||||
|
* centre both times and the two videos looked like the same video, which is
|
||||||
|
* exactly the complaint temperament exists to answer.
|
||||||
|
*/
|
||||||
|
export function sampleValues(module, rng, bias = {}, temperament = null) {
|
||||||
|
const out = {};
|
||||||
|
for (const [name, def] of Object.entries(module.params || {})) {
|
||||||
|
if (def.type === 'palette') { out[name] = null; continue; }
|
||||||
|
if (def.fixed) { out[name] = defaultValue(def); continue; }
|
||||||
|
|
||||||
|
let b = def.bias && bias[def.bias] !== undefined ? bias[def.bias] : 0.5;
|
||||||
|
if (temperament) b = clamp01(b + temperamentShift(def.bias, temperament));
|
||||||
|
|
||||||
|
if (def.type === 'bool') {
|
||||||
|
out[name] = rng.bool(0.25 + b * 0.5);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const [lo, hi] = def.range || [0, 1];
|
||||||
|
|
||||||
|
// How far this track is willing to push a param toward its limits. A
|
||||||
|
// timid track samples near the middle of everything and reads as the
|
||||||
|
// library's average; a bold one commits. This is the difference between
|
||||||
|
// "the same scene again" and "that scene, but this video's version".
|
||||||
|
const extremity = temperament ? temperament.extremity : 0.5;
|
||||||
|
|
||||||
|
// Bias still moves the centre of mass, but a bold track overrides more
|
||||||
|
// of it — otherwise every drop in every video converges on one point.
|
||||||
|
const mixAmount = (def.biasStrength !== undefined ? def.biasStrength : 0.45)
|
||||||
|
* (1 - extremity * 0.45);
|
||||||
|
|
||||||
|
const u = boldUniform(rng.next(), extremity);
|
||||||
|
const target = lo + (hi - lo) * b;
|
||||||
|
let v = (lo + (hi - lo) * u) * (1 - mixAmount) + target * mixAmount;
|
||||||
|
|
||||||
|
if (def.type === 'vec2') {
|
||||||
|
const u2 = boldUniform(rng.next(), extremity);
|
||||||
|
const v2 = (lo + (hi - lo) * u2) * (1 - mixAmount) + target * mixAmount;
|
||||||
|
out[name] = [clampValue(def, [v, v2])[0], clampValue(def, [v, v2])[1]];
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Absolute animation speed follows the song, not the scene's taste. A
|
||||||
|
// rate param sampled at 0.7 of its range means the same visual speed
|
||||||
|
// whether the track is 70bpm or 170, which is how slow songs ended up
|
||||||
|
// with scenes skittering over them. See look/LookGenerator biasFor.
|
||||||
|
if (def[RATE_FLAG] && bias.rateScale) v *= bias.rateScale;
|
||||||
|
|
||||||
|
if (def.type === 'int') v = Math.round(v);
|
||||||
|
out[name] = clampValue(def, v);
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
const clamp01 = (x) => Math.max(0, Math.min(1, x));
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Reshape a uniform draw so a bold track reaches the ends of a range.
|
||||||
|
*
|
||||||
|
* At extremity 0 this is unchanged. As it rises the distribution hollows out:
|
||||||
|
* the same draw lands further from the centre, so a track that wants density
|
||||||
|
* gets scenes at their dense end rather than at a polite 60%.
|
||||||
|
*
|
||||||
|
* The exponent floor is low on purpose. A param range is the scene author's
|
||||||
|
* statement of what the scene can survive, so the ends of it are supposed to be
|
||||||
|
* usable — a library that samples the middle of every range is a library where
|
||||||
|
* every scene shows its default.
|
||||||
|
*
|
||||||
|
* There is a ceiling on this, found by overshooting it. Pushed harder (0.82),
|
||||||
|
* enough draws piled onto the range ends that two different seeds started
|
||||||
|
* producing near-identical frames — the Phase 3 look-space check caught a
|
||||||
|
* closest pair at 0.0096 against a floor of 0.01 — and a sparse scene sampled
|
||||||
|
* at its low end rendered as effectively black. Extremes are where the variety
|
||||||
|
* is; the extremes are also where every scene collapses onto the same extreme.
|
||||||
|
*/
|
||||||
|
function boldUniform(u, extremity) {
|
||||||
|
const signed = (u - 0.5) * 2;
|
||||||
|
const shaped = Math.sign(signed) * Math.pow(Math.abs(signed), 1 - clamp01(extremity) * 0.72);
|
||||||
|
return clamp01(0.5 + shaped * 0.5);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Which way this track leans on each of the three bias axes. */
|
||||||
|
function temperamentShift(axis, temperament) {
|
||||||
|
switch (axis) {
|
||||||
|
case 'energy': return temperament.intensity * 0.3;
|
||||||
|
case 'density': return temperament.intensity * 0.2 + temperament.detail * 0.3;
|
||||||
|
case 'motion': return temperament.pace * 0.35;
|
||||||
|
default: return 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Evenly spaced probe values across a param's range, for the range-sweep check. */
|
||||||
|
export function sweepValues(def, steps = 5) {
|
||||||
|
if (def.type === 'bool') return [false, true];
|
||||||
|
if (def.type === 'palette') return [null];
|
||||||
|
const [lo, hi] = def.range || [0, 1];
|
||||||
|
const out = [];
|
||||||
|
for (let i = 0; i < steps; i++) {
|
||||||
|
let v = lo + ((hi - lo) * i) / (steps - 1);
|
||||||
|
if (def.type === 'int') v = Math.round(v);
|
||||||
|
out.push(def.type === 'vec2' ? [v, v] : v);
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Structural validation of a scene module. Returns an array of human-readable
|
||||||
|
* problems; empty means clean. Shared by the lint tool and the runtime registry,
|
||||||
|
* so a malformed scene can't reach the compositor.
|
||||||
|
*/
|
||||||
|
export function validateModule(module) {
|
||||||
|
const errors = [];
|
||||||
|
const id = module?.name || '<unnamed>';
|
||||||
|
|
||||||
|
if (!module.name) errors.push('missing `name`');
|
||||||
|
if (!module.family) errors.push(`${id}: missing \`family\``);
|
||||||
|
if (!module.kind) errors.push(`${id}: missing \`kind\``);
|
||||||
|
if (!Array.isArray(module.traits)) {
|
||||||
|
errors.push(`${id}: missing \`traits\` — declare which personality traits it honours ` +
|
||||||
|
`(any of ${TRAIT_NAMES.join(', ')}, or [] for none)`);
|
||||||
|
} else {
|
||||||
|
for (const t of module.traits) {
|
||||||
|
if (!TRAIT_NAMES.includes(t)) errors.push(`${id}: unknown trait '${t}'`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (module.texture !== undefined
|
||||||
|
&& (typeof module.texture !== 'number' || module.texture < 0 || module.texture > 2)) {
|
||||||
|
errors.push(`${id}: \`texture\` must be a number 0..2 — how much of the track's ` +
|
||||||
|
`surface grain this scene takes (1 = all, 0 = none)`);
|
||||||
|
}
|
||||||
|
if (module.kind === 'fragment' && !module.shader) errors.push(`${id}: kind 'fragment' but no \`shader\``);
|
||||||
|
if (module.kind === 'fragment' && module.shader && !/vec4\s+scene\s*\(/.test(module.shader)) {
|
||||||
|
errors.push(`${id}: shader must define \`vec4 scene(vec2 uv, vec2 p)\``);
|
||||||
|
}
|
||||||
|
|
||||||
|
const params = module.params || {};
|
||||||
|
const uniformNames = new Set();
|
||||||
|
|
||||||
|
for (const [name, def] of Object.entries(params)) {
|
||||||
|
const where = `${id}.${name}`;
|
||||||
|
if (!def.type) errors.push(`${where}: missing \`type\``);
|
||||||
|
else if (!PARAM_TYPES.includes(def.type)) errors.push(`${where}: unknown type '${def.type}'`);
|
||||||
|
|
||||||
|
if (def.type !== 'palette' && def.type !== 'bool' && !def.range) {
|
||||||
|
errors.push(`${where}: numeric param needs a \`range\``);
|
||||||
|
}
|
||||||
|
if (def.range && (def.range.length !== 2 || def.range[0] >= def.range[1])) {
|
||||||
|
errors.push(`${where}: \`range\` must be [min, max] with min < max`);
|
||||||
|
}
|
||||||
|
if (def.default !== undefined && def.range && typeof def.default === 'number') {
|
||||||
|
if (def.default < def.range[0] || def.default > def.range[1]) {
|
||||||
|
errors.push(`${where}: default ${def.default} outside range [${def.range}]`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (def.uniform) {
|
||||||
|
if (uniformNames.has(def.uniform)) errors.push(`${where}: duplicate uniform '${def.uniform}'`);
|
||||||
|
uniformNames.add(def.uniform);
|
||||||
|
if (!/^u_[A-Za-z0-9_]+$/.test(def.uniform)) {
|
||||||
|
errors.push(`${where}: uniform '${def.uniform}' should be named u_*`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (def.bias && typeof def.bias !== 'string') errors.push(`${where}: \`bias\` must be a key name`);
|
||||||
|
if (def[SLOW_AXIS_FLAG]) {
|
||||||
|
if (def[RATE_FLAG]) {
|
||||||
|
errors.push(`${where}: cannot be both \`rate\` and \`slowAxis\` — walking a rate ` +
|
||||||
|
`param jumps the animation phase (see the RATE_FLAG note above)`);
|
||||||
|
}
|
||||||
|
if (def.type === 'palette' || def.type === 'bool' || def.fixed) {
|
||||||
|
errors.push(`${where}: \`slowAxis\` needs a numeric range to walk`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for (const [name, r] of Object.entries(module.reactive || {})) {
|
||||||
|
const where = `${id}.reactive.${name}`;
|
||||||
|
if (!params[name]) errors.push(`${where}: no such param`);
|
||||||
|
if (!r.feature) errors.push(`${where}: missing \`feature\``);
|
||||||
|
else if (!REACTIVE_FEATURES.includes(r.feature)) {
|
||||||
|
errors.push(`${where}: unknown feature '${r.feature}'`);
|
||||||
|
}
|
||||||
|
if (r.response && !REACTIVE_RESPONSES.includes(r.response)) {
|
||||||
|
errors.push(`${where}: unknown response '${r.response}'`);
|
||||||
|
}
|
||||||
|
if (typeof r.amount !== 'number') errors.push(`${where}: missing numeric \`amount\``);
|
||||||
|
if (params[name] && params[name].rate) {
|
||||||
|
errors.push(`${where}: '${name}' is a rate param and cannot be reactive — ` +
|
||||||
|
`modulating it jumps phase by elapsed*delta (see RATE_FLAG)`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return errors;
|
||||||
|
}
|
||||||
72
flow-state/src/params/serialize.js
Normal file
72
flow-state/src/params/serialize.js
Normal file
@ -0,0 +1,72 @@
|
|||||||
|
// Preset serialisation. A preset is just a param set plus the scene it belongs
|
||||||
|
// to, so "save the look I tuned" and "the look generator's output" are the same
|
||||||
|
// kind of object and can be diffed, stored and round-tripped identically.
|
||||||
|
|
||||||
|
import { clampValue, defaultValues } from './schema.js';
|
||||||
|
|
||||||
|
export const PRESET_VERSION = 1;
|
||||||
|
|
||||||
|
export function serializeParams(module, values) {
|
||||||
|
const out = {};
|
||||||
|
for (const [name, def] of Object.entries(module.params || {})) {
|
||||||
|
if (def.type === 'palette') continue; // palettes live on the look, not the scene
|
||||||
|
const v = values[name];
|
||||||
|
if (v === undefined) continue;
|
||||||
|
out[name] = def.type === 'vec2' ? [v[0], v[1]] : v;
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parse a stored param set back onto a module. Unknown keys are dropped and
|
||||||
|
* missing ones fall back to defaults, so a preset saved before a scene gained or
|
||||||
|
* lost a param still loads instead of throwing.
|
||||||
|
*/
|
||||||
|
export function deserializeParams(module, stored) {
|
||||||
|
const values = defaultValues(module);
|
||||||
|
if (!stored) return values;
|
||||||
|
for (const [name, def] of Object.entries(module.params || {})) {
|
||||||
|
if (def.type === 'palette') continue;
|
||||||
|
if (stored[name] === undefined) continue;
|
||||||
|
values[name] = clampValue(def, stored[name]);
|
||||||
|
}
|
||||||
|
return values;
|
||||||
|
}
|
||||||
|
|
||||||
|
function serializeLayer(layer) {
|
||||||
|
return {
|
||||||
|
scene: layer.module.name,
|
||||||
|
blend: layer.blend,
|
||||||
|
opacity: layer.opacity,
|
||||||
|
seed: layer.seed,
|
||||||
|
params: layer.params,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export function serializeLook(look) {
|
||||||
|
return {
|
||||||
|
version: PRESET_VERSION,
|
||||||
|
seed: look.seed,
|
||||||
|
palette: look.palette,
|
||||||
|
// The production design decides which scenes were even eligible, so a
|
||||||
|
// preset without it cannot be reproduced.
|
||||||
|
personality: look.personality,
|
||||||
|
post: look.post,
|
||||||
|
feedback: look.feedback,
|
||||||
|
sections: look.sections.map((s) => ({
|
||||||
|
index: s.index,
|
||||||
|
kind: s.kind,
|
||||||
|
locked: !!s.locked,
|
||||||
|
// `layers` is the anchor stack and stays first for compatibility;
|
||||||
|
// `variants` is the full roster the section's shots cut between.
|
||||||
|
layers: s.layers.map(serializeLayer),
|
||||||
|
variants: (s.variants || [s.layers]).map((stack) => stack.map(serializeLayer)),
|
||||||
|
shots: (s.shots || []).map((shot) => ({
|
||||||
|
startFrame: shot.startFrame,
|
||||||
|
endFrame: shot.endFrame,
|
||||||
|
variant: shot.variant,
|
||||||
|
hardCut: !!shot.hardCut,
|
||||||
|
})),
|
||||||
|
})),
|
||||||
|
};
|
||||||
|
}
|
||||||
155
flow-state/src/scenes/layers3d/particles.js
Normal file
155
flow-state/src/scenes/layers3d/particles.js
Normal file
@ -0,0 +1,155 @@
|
|||||||
|
// A 3D particle field — the proof that the compositor is genuinely hybrid and
|
||||||
|
// not just a fragment-shader stack.
|
||||||
|
//
|
||||||
|
// DETERMINISM: particle positions are ANALYTIC functions of (time, index, seed),
|
||||||
|
// never integrated frame to frame. An integrated system would accumulate state,
|
||||||
|
// which would make a seek land somewhere different from sequential playback and
|
||||||
|
// break export parity. Anything added here must follow the same rule: if you find
|
||||||
|
// yourself writing `position += velocity * dt`, it belongs in a closed form instead.
|
||||||
|
|
||||||
|
export const particleField = {
|
||||||
|
name: 'Particle Field',
|
||||||
|
family: 'flow',
|
||||||
|
kind: 'layer3d',
|
||||||
|
// Composited over a background, never used as one: most of the frame is
|
||||||
|
// legitimately black, so it is judged on variance rather than luminance and
|
||||||
|
// the look generator only picks it as an accent layer.
|
||||||
|
role: 'accent',
|
||||||
|
// Personality: see look/Personality.js. A point cloud cannot draw the
|
||||||
|
// signature form and has no horizon, so it claims only the camera — which
|
||||||
|
// it can honour exactly, being the one scene with a real one.
|
||||||
|
traits: ['camera'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
count: { type: 'int', range: [200, 4000], default: 1200, bias: 'density', noDrift: true },
|
||||||
|
size: { type: 'float', range: [0.01, 0.12], default: 0.04 },
|
||||||
|
spread: { type: 'float', range: [2, 14], default: 7 },
|
||||||
|
swirl: { type: 'float', range: [0, 2], default: 0.6, bias: 'motion', rate: true },
|
||||||
|
rise: { type: 'float', range: [-1, 1], default: 0.25, rate: true },
|
||||||
|
depth: { type: 'float', range: [2, 20], default: 9 },
|
||||||
|
brightness:{ type: 'float', range: [0, 2], default: 0.8, bias: 'energy' },
|
||||||
|
palette: { type: 'palette', count: 4 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
brightness: { feature: 'beat', amount: 0.5, response: 'spike' },
|
||||||
|
size: { feature: 'bandHigh', amount: 0.2 },
|
||||||
|
},
|
||||||
|
|
||||||
|
build({ scene, seed, params, THREE }) {
|
||||||
|
const max = 4000;
|
||||||
|
const geometry = new THREE.BufferGeometry();
|
||||||
|
const positions = new Float32Array(max * 3);
|
||||||
|
const colors = new Float32Array(max * 3);
|
||||||
|
const phases = new Float32Array(max * 4); // per-particle constants
|
||||||
|
|
||||||
|
// Mulberry32 inline: build() runs once, and importing the engine's Rng
|
||||||
|
// here would couple a scene module to the engine for four lines.
|
||||||
|
let state = seed >>> 0;
|
||||||
|
const rnd = () => {
|
||||||
|
let t = (state += 0x6d2b79f5) >>> 0;
|
||||||
|
t = Math.imul(t ^ (t >>> 15), t | 1);
|
||||||
|
t ^= t + Math.imul(t ^ (t >>> 7), t | 61);
|
||||||
|
return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
|
||||||
|
};
|
||||||
|
|
||||||
|
for (let i = 0; i < max; i++) {
|
||||||
|
phases[i * 4 + 0] = rnd() * Math.PI * 2; // orbital phase
|
||||||
|
phases[i * 4 + 1] = 0.3 + rnd() * 1.4; // radius factor
|
||||||
|
phases[i * 4 + 2] = rnd(); // depth position
|
||||||
|
phases[i * 4 + 3] = 0.4 + rnd() * 1.2; // speed factor
|
||||||
|
}
|
||||||
|
|
||||||
|
geometry.setAttribute('position', new THREE.BufferAttribute(positions, 3));
|
||||||
|
geometry.setAttribute('color', new THREE.BufferAttribute(colors, 3));
|
||||||
|
geometry.setDrawRange(0, params.count || 1200);
|
||||||
|
|
||||||
|
const material = new THREE.PointsMaterial({
|
||||||
|
size: 0.04,
|
||||||
|
vertexColors: true,
|
||||||
|
transparent: true,
|
||||||
|
blending: THREE.AdditiveBlending,
|
||||||
|
depthWrite: false,
|
||||||
|
sizeAttenuation: true,
|
||||||
|
});
|
||||||
|
|
||||||
|
const points = new THREE.Points(geometry, material);
|
||||||
|
points.frustumCulled = false;
|
||||||
|
scene.add(points);
|
||||||
|
|
||||||
|
return { points, geometry, material, positions, colors, phases, max };
|
||||||
|
},
|
||||||
|
|
||||||
|
update({ instance, camera, timeline, features, params, palette, personality }) {
|
||||||
|
const { geometry, material, positions, colors, phases, max } = instance;
|
||||||
|
const count = Math.min(max, Math.round(params.count));
|
||||||
|
const t = timeline.time;
|
||||||
|
|
||||||
|
const spread = params.spread;
|
||||||
|
const depth = params.depth;
|
||||||
|
const swirl = params.swirl;
|
||||||
|
const rise = params.rise;
|
||||||
|
const brightness = Math.max(0, params.brightness);
|
||||||
|
|
||||||
|
const colorCount = palette && palette.length ? palette.length : 0;
|
||||||
|
|
||||||
|
for (let i = 0; i < count; i++) {
|
||||||
|
const phase = phases[i * 4 + 0];
|
||||||
|
const radiusFactor = phases[i * 4 + 1];
|
||||||
|
const depthSeed = phases[i * 4 + 2];
|
||||||
|
const speed = phases[i * 4 + 3];
|
||||||
|
|
||||||
|
const angle = phase + t * swirl * speed * 0.35;
|
||||||
|
const radius = radiusFactor * spread * 0.5;
|
||||||
|
|
||||||
|
// Depth wraps analytically: fract() of a linear ramp, so a seek to
|
||||||
|
// any frame reproduces the exact same layout.
|
||||||
|
const z = ((depthSeed + t * rise * 0.05 * speed) % 1 + 1) % 1;
|
||||||
|
|
||||||
|
positions[i * 3 + 0] = Math.cos(angle) * radius;
|
||||||
|
positions[i * 3 + 1] = Math.sin(angle) * radius * 0.6
|
||||||
|
+ Math.sin(t * 0.4 * speed + phase) * 0.6;
|
||||||
|
positions[i * 3 + 2] = -z * depth;
|
||||||
|
|
||||||
|
// Fade with depth so the field reads as volume rather than confetti.
|
||||||
|
const fade = (1 - z) * brightness;
|
||||||
|
if (colorCount) {
|
||||||
|
const c = palette[i % colorCount];
|
||||||
|
colors[i * 3 + 0] = c[0] * fade;
|
||||||
|
colors[i * 3 + 1] = c[1] * fade;
|
||||||
|
colors[i * 3 + 2] = c[2] * fade;
|
||||||
|
} else {
|
||||||
|
colors[i * 3 + 0] = colors[i * 3 + 1] = colors[i * 3 + 2] = fade;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
geometry.setDrawRange(0, count);
|
||||||
|
geometry.attributes.position.needsUpdate = true;
|
||||||
|
geometry.attributes.color.needsUpdate = true;
|
||||||
|
material.size = params.size;
|
||||||
|
material.opacity = 1;
|
||||||
|
|
||||||
|
// The track's camera, applied to the only literal camera in the library:
|
||||||
|
// the same slow returning pan, sway and roll every shader scene fakes in
|
||||||
|
// its coordinate space. Bounded and periodic, so a seek still lands on
|
||||||
|
// the same frame as sequential playback.
|
||||||
|
const cam = personality ? personality.camera : null;
|
||||||
|
if (cam) {
|
||||||
|
const pan = 20 * Math.sin(t * 0.05);
|
||||||
|
camera.position.set(
|
||||||
|
Math.cos(cam.driftAngle) * cam.driftRate * pan
|
||||||
|
+ Math.sin(t * cam.swayRate) * cam.sway,
|
||||||
|
Math.sin(cam.driftAngle) * cam.driftRate * pan
|
||||||
|
+ Math.cos(t * cam.swayRate * 0.83) * cam.sway,
|
||||||
|
4,
|
||||||
|
);
|
||||||
|
camera.rotation.z = cam.spin * t;
|
||||||
|
} else {
|
||||||
|
camera.position.set(0, 0, 4);
|
||||||
|
camera.rotation.z = 0;
|
||||||
|
}
|
||||||
|
camera.lookAt(camera.position.x, camera.position.y, -depth * 0.4);
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
export default particleField;
|
||||||
141
flow-state/src/scenes/registry.js
Normal file
141
flow-state/src/scenes/registry.js
Normal file
@ -0,0 +1,141 @@
|
|||||||
|
import { validateModule } from '../params/schema.js';
|
||||||
|
|
||||||
|
import { nebula } from './shader/nebula.js';
|
||||||
|
import { classicWave } from './shader/classic-wave.js';
|
||||||
|
import { floatingGeometry } from './shader/floating-geometry.js';
|
||||||
|
import { synthwaveRun } from './shader/synthwave-run.js';
|
||||||
|
import { psychedelicDrift } from './shader/psychedelic-drift.js';
|
||||||
|
import { particleField } from './layers3d/particles.js';
|
||||||
|
import { horizonLines } from './shader/horizon-lines.js';
|
||||||
|
import { spectrumSculpture } from './shader/spectrum-sculpture.js';
|
||||||
|
import { slowOrb } from './shader/slow-orb.js';
|
||||||
|
import { curlFlow } from './shader/curl-flow.js';
|
||||||
|
import { plasmaBloom } from './shader/plasma-bloom.js';
|
||||||
|
import { metaballs } from './shader/metaballs.js';
|
||||||
|
import { kaleidoTunnel } from './shader/kaleido-tunnel.js';
|
||||||
|
import { moireGrid } from './shader/moire-grid.js';
|
||||||
|
import { ridgeTerrain } from './shader/ridge-terrain.js';
|
||||||
|
import { scanTear } from './shader/scan-tear.js';
|
||||||
|
import { blockMosh } from './shader/block-mosh.js';
|
||||||
|
import { neonCity } from './shader/neon-city.js';
|
||||||
|
import { flora } from './shader/flora.js';
|
||||||
|
import { fireflyDrift } from './shader/firefly-drift.js';
|
||||||
|
import { silkRibbon } from './shader/silk-ribbon.js';
|
||||||
|
import { pylonGrid } from './shader/pylon-grid.js';
|
||||||
|
import { prismBloom } from './shader/prism-bloom.js';
|
||||||
|
import { pitchShatter } from './shader/pitch-shatter.js';
|
||||||
|
import { auroraVeil } from './shader/aurora-veil.js';
|
||||||
|
import { tideRings } from './shader/tide-rings.js';
|
||||||
|
import { dustChamber } from './shader/dust-chamber.js';
|
||||||
|
import { cargoBelt } from './shader/cargo-belt.js';
|
||||||
|
import { circuitBloom } from './shader/circuit-bloom.js';
|
||||||
|
import { truchetFold } from './shader/truchet-fold.js';
|
||||||
|
import { signalDecay } from './shader/signal-decay.js';
|
||||||
|
import { inkBleed } from './shader/ink-bleed.js';
|
||||||
|
import { saltFlat } from './shader/salt-flat.js';
|
||||||
|
import { stormRift } from './shader/storm-rift.js';
|
||||||
|
import { vortexDrift } from './shader/vortex-drift.js';
|
||||||
|
import { gateCorridor } from './shader/gate-corridor.js';
|
||||||
|
import { smokeColumn } from './shader/smoke-column.js';
|
||||||
|
import { cellDivide } from './shader/cell-divide.js';
|
||||||
|
import { eclipseField } from './shader/eclipse-field.js';
|
||||||
|
import { girderLattice } from './shader/girder-lattice.js';
|
||||||
|
import { quasicrystal } from './shader/quasicrystal.js';
|
||||||
|
import { timeSmear } from './shader/time-smear.js';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The scene library. Families exist so the arc driver can choose by section
|
||||||
|
* character rather than at random — a breakdown never lands on a strobing glitch
|
||||||
|
* scene, and an intro never opens at full density.
|
||||||
|
*/
|
||||||
|
export const FAMILIES = {
|
||||||
|
flow: { label: 'Flow', energy: [0.0, 0.7] },
|
||||||
|
organic: { label: 'Organic', energy: [0.0, 0.8] },
|
||||||
|
minimal: { label: 'Minimal', energy: [0.0, 0.45] },
|
||||||
|
structural: { label: 'Structural', energy: [0.3, 0.9] },
|
||||||
|
geometric: { label: 'Geometric', energy: [0.4, 1.0] },
|
||||||
|
glitch: { label: 'Glitch', energy: [0.6, 1.0] },
|
||||||
|
};
|
||||||
|
|
||||||
|
const MODULES = [
|
||||||
|
nebula,
|
||||||
|
classicWave,
|
||||||
|
floatingGeometry,
|
||||||
|
synthwaveRun,
|
||||||
|
psychedelicDrift,
|
||||||
|
particleField,
|
||||||
|
|
||||||
|
// Phase 7 additions. 'minimal' came first: with the family empty, intros and
|
||||||
|
// breakdowns fell through to flow/organic and every track opened at density.
|
||||||
|
horizonLines,
|
||||||
|
spectrumSculpture,
|
||||||
|
slowOrb,
|
||||||
|
curlFlow,
|
||||||
|
plasmaBloom,
|
||||||
|
metaballs,
|
||||||
|
kaleidoTunnel,
|
||||||
|
moireGrid,
|
||||||
|
ridgeTerrain,
|
||||||
|
scanTear,
|
||||||
|
blockMosh,
|
||||||
|
neonCity,
|
||||||
|
flora,
|
||||||
|
fireflyDrift,
|
||||||
|
silkRibbon,
|
||||||
|
pylonGrid,
|
||||||
|
prismBloom,
|
||||||
|
pitchShatter,
|
||||||
|
|
||||||
|
// Ten added to widen the library past the point where a track's rosters
|
||||||
|
// start repeating: the casting rule shrinks the pool per track, so depth in
|
||||||
|
// every family is what keeps two videos from drawing the same four scenes.
|
||||||
|
// Weighted toward the 'space' and 'shape' traits, which were the thinnest
|
||||||
|
// and therefore the signatures most likely to run out of cast.
|
||||||
|
auroraVeil,
|
||||||
|
tideRings,
|
||||||
|
dustChamber,
|
||||||
|
cargoBelt,
|
||||||
|
circuitBloom,
|
||||||
|
truchetFold,
|
||||||
|
signalDecay,
|
||||||
|
inkBleed,
|
||||||
|
saltFlat,
|
||||||
|
stormRift,
|
||||||
|
vortexDrift,
|
||||||
|
gateCorridor,
|
||||||
|
smokeColumn,
|
||||||
|
cellDivide,
|
||||||
|
eclipseField,
|
||||||
|
girderLattice,
|
||||||
|
quasicrystal,
|
||||||
|
timeSmear,
|
||||||
|
];
|
||||||
|
|
||||||
|
const errors = [];
|
||||||
|
for (const m of MODULES) {
|
||||||
|
const e = validateModule(m);
|
||||||
|
if (e.length) errors.push(...e);
|
||||||
|
if (m.family && !FAMILIES[m.family]) errors.push(`${m.name}: unknown family '${m.family}'`);
|
||||||
|
}
|
||||||
|
if (errors.length) {
|
||||||
|
// Fail loudly at import: a malformed scene must never reach the compositor,
|
||||||
|
// where the symptom would be a black frame with no explanation.
|
||||||
|
console.error('[registry] invalid scene modules:\n' + errors.join('\n'));
|
||||||
|
}
|
||||||
|
|
||||||
|
export const scenes = MODULES;
|
||||||
|
export const sceneErrors = errors;
|
||||||
|
|
||||||
|
export function sceneByName(name) {
|
||||||
|
return MODULES.find((m) => m.name === name) || null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function scenesInFamily(family) {
|
||||||
|
return MODULES.filter((m) => m.family === family);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function familyNames() {
|
||||||
|
return Object.keys(FAMILIES);
|
||||||
|
}
|
||||||
|
|
||||||
|
export default scenes;
|
||||||
81
flow-state/src/scenes/shader/aurora-veil.js
Normal file
81
flow-state/src/scenes/shader/aurora-veil.js
Normal file
@ -0,0 +1,81 @@
|
|||||||
|
// Flow family: curtains of light standing above the track's horizon.
|
||||||
|
//
|
||||||
|
// Distinct from Curl Flow (a full-frame advected field) and Silk Ribbon (one
|
||||||
|
// strand): this is several tall vertical sheets, each rippling on its own phase,
|
||||||
|
// dense at the base and dissolving upward. The rippling is a sum of sines rather
|
||||||
|
// than noise, which is what gives an aurora its folded-sheet look instead of a
|
||||||
|
// smoky one — noise curtains read as fog.
|
||||||
|
//
|
||||||
|
// The horizon is the track's, so this stands in the same place as every other
|
||||||
|
// scene that has ground.
|
||||||
|
|
||||||
|
export const auroraVeil = {
|
||||||
|
name: 'Aurora Veil',
|
||||||
|
family: 'flow',
|
||||||
|
kind: 'fragment',
|
||||||
|
traits: ['camera', 'space', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
curtains: { type: 'int', range: [2, 9], default: 4, uniform: 'u_curtains', bias: 'density' },
|
||||||
|
height: { type: 'float', range: [0.4, 1.8], default: 1.0, uniform: 'u_height' },
|
||||||
|
fold: { type: 'float', range: [0.1, 1.4], default: 0.55, uniform: 'u_fold', bias: 'density' },
|
||||||
|
speed: { type: 'float', range: [0.03, 0.5], default: 0.15, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
glow: { type: 'float', range: [0, 1.4], default: 0.5, uniform: 'u_glow', bias: 'energy' },
|
||||||
|
ground: { type: 'float', range: [0, 0.8], default: 0.3, uniform: 'u_ground' },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
glow: { feature: 'bandHigh', amount: 0.4, response: 'smooth' },
|
||||||
|
fold: { feature: 'bandLow', amount: 0.3, response: 'smooth' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
float horizon = sigHorizonY() * 0.8 - 0.55;
|
||||||
|
float above = p.y - horizon;
|
||||||
|
|
||||||
|
vec3 col = pal(0) * 0.06;
|
||||||
|
|
||||||
|
for (int i = 0; i < 9; i++) {
|
||||||
|
if (i >= u_curtains) break;
|
||||||
|
float fi = float(i);
|
||||||
|
float s = u_seed + fi * 37.13;
|
||||||
|
|
||||||
|
// Each sheet is a vertical line whose x wanders as a sum of three sines.
|
||||||
|
// Three is the fewest that stops reading as a single wobble.
|
||||||
|
float x = (fract(s * 0.61) - 0.5) * 2.2
|
||||||
|
+ sin(above * 1.7 + t * 1.3 + s) * u_fold * 0.35
|
||||||
|
+ sin(above * 3.1 - t * 0.8 + s * 1.7) * u_fold * 0.18
|
||||||
|
+ sin(above * 0.7 + t * 0.4 + s * 2.3) * u_fold * 0.5;
|
||||||
|
|
||||||
|
float d = abs(p.x - x);
|
||||||
|
float widthAt = 0.05 + above * 0.06 + u_sigLine * 0.05;
|
||||||
|
|
||||||
|
// Bright and tight at the base, wide and faint at the top: the vertical
|
||||||
|
// falloff is what makes it a curtain rather than a stripe.
|
||||||
|
float rise = smoothstep(-0.05, 0.0, above) * exp(-max(above, 0.0) / max(u_height, 0.05));
|
||||||
|
float sheet = exp(-d * d / max(widthAt * widthAt, 1e-5)) * rise;
|
||||||
|
|
||||||
|
vec3 tint = palRamp(fract(s) * 0.5 + above * 0.12 + 0.1);
|
||||||
|
col += tint * sheet * (0.55 + u_glow * 0.6);
|
||||||
|
col += tint * exp(-d * 5.0) * rise * u_glow * 0.12;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ground: the curtains reflected, dim and compressed.
|
||||||
|
if (u_ground > 0.01 && above < 0.0) {
|
||||||
|
float below = -above;
|
||||||
|
col += pal(2) * u_ground * exp(-below * 6.0) * 0.35;
|
||||||
|
}
|
||||||
|
|
||||||
|
col = sigAir(col, p, smoothstep(0.0, 1.6, abs(p.x)));
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default auroraVeil;
|
||||||
82
flow-state/src/scenes/shader/block-mosh.js
Normal file
82
flow-state/src/scenes/shader/block-mosh.js
Normal file
@ -0,0 +1,82 @@
|
|||||||
|
// Glitch family: a datamosh. The frame is cut into a coarse grid of blocks and
|
||||||
|
// each block is pulled along a per-block stroke drawn from the PREVIOUS frame —
|
||||||
|
// so the corruption accrues and slithers across frames instead of tearing once.
|
||||||
|
//
|
||||||
|
// Everything steps on one quantised grid: the block field re-rolls on bar lines
|
||||||
|
// rather than crawling, which is what makes this read as an edit rather than as
|
||||||
|
// a smooth warp. On an onset the spill widens, which is the "hit".
|
||||||
|
|
||||||
|
export const blockMosh = {
|
||||||
|
name: 'Block Mosh',
|
||||||
|
family: 'glitch',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
traits: ['camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
block: { type: 'float', range: [4, 48], default: 22, uniform: 'u_blocks', bias: 'density' },
|
||||||
|
smear: { type: 'float', range: [0, 0.5], default: 0.18, uniform: 'u_smear', bias: 'energy' },
|
||||||
|
quantize: { type: 'float', range: [1, 8], default: 4, uniform: 'u_quantize' },
|
||||||
|
bleed: { type: 'float', range: [0, 1], default: 0.45, uniform: 'u_bleed' },
|
||||||
|
speed: { type: 'float', range: [0.1, 1.4], default: 0.5, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
burst: { type: 'float', range: [0, 0.4], default: 0.12, uniform: 'u_burst' },
|
||||||
|
palette: { type: 'palette', count: 4 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
smear: { feature: 'bandHigh', amount: 0.3, response: 'smooth' },
|
||||||
|
bleed: { feature: 'flux', amount: 0.2, response: 'spike' },
|
||||||
|
burst: { feature: 'beat', amount: 0.25, response: 'smooth' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
float beat = u_beat;
|
||||||
|
|
||||||
|
// The camera moves the CONTENT, not the block grid. The grid has to stay
|
||||||
|
// screen-aligned or the mosh stops reading as a codec artefact and starts
|
||||||
|
// reading as a moving texture — but the field being moshed is part of the
|
||||||
|
// shot and is filmed by the same operator as every other scene. (Assigning
|
||||||
|
// sigCamera(p) to an otherwise unused p is how this scene originally
|
||||||
|
// "honoured" the trait; the Phase 9 render gate measured the resulting
|
||||||
|
// difference at exactly zero.)
|
||||||
|
vec2 cp = sigCamera(p);
|
||||||
|
|
||||||
|
// The thing being moshed: a sheared banded field, recoloured per track, so
|
||||||
|
// the corruption has real content to drag around even standing alone.
|
||||||
|
float shear = cp.y * (2.5 + u_sigLine * 3.0) + t * 0.45;
|
||||||
|
float field = fbm(vec2(cp.x * 0.9 + sin(shear) * 0.3, cp.y * 1.5 + t * 0.35), 3);
|
||||||
|
vec3 base = palRamp(field * 1.15 + cp.x * 0.18 + t * 0.03);
|
||||||
|
base *= 0.35 + 0.55 * sat(field * 1.5);
|
||||||
|
|
||||||
|
// One quantised grid for every jump below, so the mosh steps like an edit
|
||||||
|
// rather than crawling like a smooth warp.
|
||||||
|
float q = max(u_quantize, 1.0);
|
||||||
|
float st = floor(u_barPhase * q * 6.0) + floor(t * 4.0) * (q * 6.0);
|
||||||
|
vec2 cell = floor(uv * u_blocks);
|
||||||
|
|
||||||
|
// Each block gets one fixed stroke per step: a direction and a reach.
|
||||||
|
vec2 stroke = (hash22(cell + st * 31.0) - 0.5) * 2.0;
|
||||||
|
float reach = hash12(cell + st * 17.0);
|
||||||
|
vec2 dst = clamp(uv + stroke * u_smear * (0.3 + reach), 0.0, 1.0);
|
||||||
|
|
||||||
|
// The mosh pulls the PREVIOUS frame along the stroke and layers it over the
|
||||||
|
// fresh field. The previous frame is itself moshed, so smear accrues.
|
||||||
|
vec3 mosh = prev(dst);
|
||||||
|
float m = clamp(u_bleed * (0.5 + reach), 0.0, 1.0);
|
||||||
|
vec3 col = mix(base, mosh, m);
|
||||||
|
|
||||||
|
// Weak block tint, pulse-pinned to the beat but kept small — the frame as a
|
||||||
|
// whole never swings in luminance, so it stays clear of the WCAG 2.3.1
|
||||||
|
// flash ceiling even on a hard four-on-the-floor.
|
||||||
|
float blk = hash12(cell + st * 7.0);
|
||||||
|
col += pal(2) * u_burst * (0.5 + 0.5 * blk) * (0.5 + 0.5 * beat);
|
||||||
|
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default blockMosh;
|
||||||
90
flow-state/src/scenes/shader/cargo-belt.js
Normal file
90
flow-state/src/scenes/shader/cargo-belt.js
Normal file
@ -0,0 +1,90 @@
|
|||||||
|
// Structural family: horizontal belts of cargo running in alternating
|
||||||
|
// directions, stacked up the frame.
|
||||||
|
//
|
||||||
|
// Distinct from Neon City (a static skyline) and Pylon Grid (perspective depth):
|
||||||
|
// this has no depth at all. It is flat, industrial and lateral — the only scene
|
||||||
|
// in the library whose motion is purely sideways, which is exactly what makes it
|
||||||
|
// cut well against everything that recedes.
|
||||||
|
//
|
||||||
|
// Each crate is stamped in the track's signature form, and the belts step in
|
||||||
|
// bar-quantised lurches rather than sliding, so the movement is mechanical.
|
||||||
|
|
||||||
|
export const cargoBelt = {
|
||||||
|
name: 'Cargo Belt',
|
||||||
|
family: 'structural',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Takes the track's surface grain, but lightly — this is drawn, not filmed.
|
||||||
|
texture: 0.4,
|
||||||
|
traits: ['shape', 'camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
belts: { type: 'int', range: [2, 8], default: 4, uniform: 'u_belts', bias: 'density' },
|
||||||
|
crates: { type: 'float', range: [2, 14], default: 6, uniform: 'u_crates', bias: 'density' },
|
||||||
|
crateSize:{ type: 'float', range: [0.15, 0.75],default: 0.42, uniform: 'u_crateSize' },
|
||||||
|
speed: { type: 'float', range: [0.05, 1.0], default: 0.3, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
gap: { type: 'float', range: [0.02, 0.3], default: 0.1, uniform: 'u_gap' },
|
||||||
|
rails: { type: 'float', range: [0, 1], default: 0.45, uniform: 'u_rails' },
|
||||||
|
lamp: { type: 'float', range: [0, 1.3], default: 0.5, uniform: 'u_lamp', bias: 'energy' },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
lamp: { feature: 'beat', amount: 0.35, response: 'spike' },
|
||||||
|
rails: { feature: 'bandLow', amount: 0.2, response: 'smooth' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
vec3 col = pal(0) * 0.05;
|
||||||
|
|
||||||
|
float span = 2.0 / max(float(u_belts), 1.0);
|
||||||
|
|
||||||
|
for (int i = 0; i < 8; i++) {
|
||||||
|
if (i >= u_belts) break;
|
||||||
|
float fi = float(i);
|
||||||
|
float centre = -1.0 + span * (fi + 0.5);
|
||||||
|
float dy = p.y - centre;
|
||||||
|
if (abs(dy) > span * 0.5) continue;
|
||||||
|
|
||||||
|
float dir = mod(fi, 2.0) < 0.5 ? 1.0 : -1.0;
|
||||||
|
|
||||||
|
// Quantised travel: the belt advances in eighth-bar steps, so cargo
|
||||||
|
// lurches from cell to cell the way a conveyor does.
|
||||||
|
float march = floor((t + fi * 0.37) * 4.0) * 0.25 * dir;
|
||||||
|
float lane = p.x * 0.5 + march;
|
||||||
|
|
||||||
|
float cell = floor(lane * u_crates);
|
||||||
|
float withinCell = fract(lane * u_crates);
|
||||||
|
float rnd = hash12(vec2(cell, fi));
|
||||||
|
|
||||||
|
// Not every cell carries a crate; the gaps are what make it read as
|
||||||
|
// cargo rather than as a stripe pattern.
|
||||||
|
if (rnd > u_gap) {
|
||||||
|
vec2 local = vec2((withinCell - 0.5) * 2.0, dy / max(span * 0.5, 1e-3));
|
||||||
|
float size = u_crateSize * (0.7 + rnd * 0.5);
|
||||||
|
float d = sigShape(local / max(size, 1e-3)) * size;
|
||||||
|
|
||||||
|
vec3 crateColor = pal(int(mod(cell + fi, 4.0)) + 1);
|
||||||
|
col = mix(col, crateColor * (0.35 + rnd * 0.5), smoothstep(0.02, -0.02, d));
|
||||||
|
col += crateColor * sigEdge(d) * (0.4 + u_lamp * 0.5);
|
||||||
|
|
||||||
|
// Lamp on a minority of crates, pulsing on the beat. Local, not
|
||||||
|
// whole-frame: a per-crate blink is not a flash.
|
||||||
|
if (rnd > 0.82) col += pal(4) * smoothstep(0.06, 0.0, length(local)) * u_lamp;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Belt rails, top and bottom of each lane.
|
||||||
|
float rail = smoothstep(0.06, 0.0, abs(abs(dy) - span * 0.45));
|
||||||
|
col += pal(2) * rail * u_rails * 0.5;
|
||||||
|
}
|
||||||
|
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default cargoBelt;
|
||||||
109
flow-state/src/scenes/shader/cell-divide.js
Normal file
109
flow-state/src/scenes/shader/cell-divide.js
Normal file
@ -0,0 +1,109 @@
|
|||||||
|
// Organic family: a packed tissue of cells that drift, crowd and split.
|
||||||
|
//
|
||||||
|
// Distinct from Metaballs (separate bodies merging in empty space) and Tide
|
||||||
|
// Rings (interference across a continuous surface): this is a PARTITION. Every
|
||||||
|
// pixel belongs to exactly one cell, the boundaries are hard, and there is no
|
||||||
|
// background — which is what makes it read as tissue rather than as objects.
|
||||||
|
// Each nucleus is stamped in the track's signature form.
|
||||||
|
//
|
||||||
|
// Splits step on the phrase grid, so the tissue reorganises on the music
|
||||||
|
// instead of crawling.
|
||||||
|
//
|
||||||
|
// Scaffolded by tools/new-scene.js. See HOWTO-visualizers.md.
|
||||||
|
|
||||||
|
export const cellDivide = {
|
||||||
|
name: 'Cell Divide',
|
||||||
|
family: 'organic',
|
||||||
|
kind: 'fragment',
|
||||||
|
traits: ['shape', 'camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
cells: { type: 'int', range: [3, 16], default: 7, uniform: 'u_cells', bias: 'density' },
|
||||||
|
wander: { type: 'float', range: [0.02, 0.5], default: 0.18, uniform: 'u_wander' },
|
||||||
|
speed: { type: 'float', range: [0.05, 1.0], default: 0.22, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
wall: { type: 'float', range: [0.01, 0.14],default: 0.045,uniform: 'u_wall' },
|
||||||
|
nucleus: { type: 'float', range: [0.0, 0.5], default: 0.22, uniform: 'u_nucleus' },
|
||||||
|
glow: { type: 'float', range: [0, 1.5], default: 0.5, uniform: 'u_glow', bias: 'energy' },
|
||||||
|
split: { type: 'float', range: [0, 1], default: 0.4, uniform: 'u_split' },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
nucleus: { feature: 'beat', amount: 0.2, response: 'spike' },
|
||||||
|
glow: { feature: 'bandMid', amount: 0.3, response: 'smooth' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
// Splits are quantised: on a phrase line a share of the cells jump to a new
|
||||||
|
// resting place, which reads as division rather than as constant drift.
|
||||||
|
float era = floor(u_phrasePhase * 2.0);
|
||||||
|
|
||||||
|
// Nearest and second-nearest seed. The DIFFERENCE between the two distances
|
||||||
|
// is the classic membrane term: it goes to zero exactly on the boundary
|
||||||
|
// between two cells, whatever their sizes, so walls come out even.
|
||||||
|
float d1 = 1e3, d2 = 1e3;
|
||||||
|
vec2 nearestAt = vec2(0.0);
|
||||||
|
float nearestId = 0.0;
|
||||||
|
float secondId = 0.0;
|
||||||
|
|
||||||
|
for (int i = 0; i < 16; i++) {
|
||||||
|
if (i >= u_cells) break;
|
||||||
|
float fi = float(i);
|
||||||
|
float s = u_seed + fi * 47.3;
|
||||||
|
|
||||||
|
float jump = step(1.0 - u_split, hash12(vec2(fi, era)));
|
||||||
|
vec2 home = (hash22(vec2(fi, floor(era * jump))) - 0.5) * 1.9;
|
||||||
|
vec2 at = home + vec2(sin(t * 0.7 + s), cos(t * 0.55 + s * 1.6)) * u_wander;
|
||||||
|
|
||||||
|
float d = length(p - at);
|
||||||
|
if (d < d1) {
|
||||||
|
d2 = d1; secondId = nearestId;
|
||||||
|
d1 = d;
|
||||||
|
nearestAt = at; nearestId = fi;
|
||||||
|
} else if (d < d2) {
|
||||||
|
d2 = d; secondId = fi;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
float membrane = d2 - d1;
|
||||||
|
|
||||||
|
// Cytoplasm, tinted by which cell owns the pixel — but BLENDED with the
|
||||||
|
// runner-up across the membrane rather than switched at it.
|
||||||
|
//
|
||||||
|
// Switching is the obvious way and it is not deterministic enough: exactly
|
||||||
|
// on a tie, which seed is nearest comes down to the last bit of a distance,
|
||||||
|
// and the two GPUs' answers disagree. The tint then jumps a whole palette
|
||||||
|
// step and the per-scene determinism gate measured 6/255 on a repeat render
|
||||||
|
// where the ceiling is 1. Blending makes the two answers agree in the limit,
|
||||||
|
// and it also looks better: cell walls read as membranes rather than as
|
||||||
|
// cuts.
|
||||||
|
vec3 mine = palRamp(hash11(nearestId * 13.7 + u_seed) * 0.55 + 0.1);
|
||||||
|
vec3 theirs = palRamp(hash11(secondId * 13.7 + u_seed) * 0.55 + 0.1);
|
||||||
|
float own = smoothstep(0.0, u_wall * 2.5, membrane) * 0.5 + 0.5;
|
||||||
|
vec3 col = mix(theirs, mine, own) * 0.35;
|
||||||
|
col *= 0.55 + 0.45 * smoothstep(0.0, 0.35, membrane); // darker toward the wall
|
||||||
|
|
||||||
|
// The wall itself.
|
||||||
|
float wall = smoothstep(u_wall * (0.5 + u_sigLine), 0.0, membrane);
|
||||||
|
col = mix(col, pal(4), wall * 0.8);
|
||||||
|
col += pal(3) * exp(-membrane * 26.0) * u_glow * 0.3;
|
||||||
|
|
||||||
|
// Nucleus, in the track's signature form.
|
||||||
|
if (u_nucleus > 0.01) {
|
||||||
|
float size = u_nucleus * 0.35;
|
||||||
|
float d = sigShape((p - nearestAt) / max(size, 1e-3)) * size;
|
||||||
|
col = mix(col, pal(1), smoothstep(0.008, -0.008, d) * 0.85);
|
||||||
|
col += pal(2) * sigEdge(d) * (0.35 + u_glow * 0.4);
|
||||||
|
}
|
||||||
|
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default cellDivide;
|
||||||
90
flow-state/src/scenes/shader/circuit-bloom.js
Normal file
90
flow-state/src/scenes/shader/circuit-bloom.js
Normal file
@ -0,0 +1,90 @@
|
|||||||
|
// Geometric family: orthogonal traces growing outward from the centre, with a
|
||||||
|
// pad in the track's signature form at every junction.
|
||||||
|
//
|
||||||
|
// Distinct from Moiré Grid (two interfering line grids) and Prism Bloom (folded
|
||||||
|
// radial geometry): the structure here is Manhattan — everything runs at right
|
||||||
|
// angles, and the only curves are the pads. That right-angle language is the
|
||||||
|
// thing the library was missing, and it cuts hard against every radial scene.
|
||||||
|
//
|
||||||
|
// Traces light up in travelling pulses rather than all at once, so the frame is
|
||||||
|
// busy without ever changing brightness as a whole.
|
||||||
|
|
||||||
|
export const circuitBloom = {
|
||||||
|
name: 'Circuit Bloom',
|
||||||
|
family: 'geometric',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Takes the track's surface grain, but lightly — this is drawn, not filmed.
|
||||||
|
texture: 0.4,
|
||||||
|
traits: ['shape', 'camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
cells: { type: 'float', range: [2, 14], default: 6, uniform: 'u_cells', bias: 'density' },
|
||||||
|
trace: { type: 'float', range: [0.01, 0.1], default: 0.035,uniform: 'u_trace' },
|
||||||
|
pads: { type: 'float', range: [0, 1], default: 0.55, uniform: 'u_pads' },
|
||||||
|
padSize: { type: 'float', range: [0.04, 0.22],default: 0.1, uniform: 'u_padSize' },
|
||||||
|
pulse: { type: 'float', range: [0, 1.5], default: 0.6, uniform: 'u_pulse', bias: 'energy' },
|
||||||
|
speed: { type: 'float', range: [0.05, 1.2], default: 0.35, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
fill: { type: 'float', range: [0.2, 0.95], default: 0.6, uniform: 'u_fill', bias: 'density' },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
pulse: { feature: 'bandHigh', amount: 0.4, response: 'smooth' },
|
||||||
|
pads: { feature: 'beat', amount: 0.2, response: 'spike' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
vec2 g = p * u_cells;
|
||||||
|
vec2 cell = floor(g);
|
||||||
|
vec2 f = fract(g) - 0.5;
|
||||||
|
|
||||||
|
float rnd = hash12(cell + u_seed);
|
||||||
|
float rnd2 = hash12(cell * 1.7 + 11.3 + u_seed);
|
||||||
|
|
||||||
|
vec3 col = pal(0) * 0.05;
|
||||||
|
|
||||||
|
// Each cell carries a horizontal trace, a vertical one, or both — the L
|
||||||
|
// junctions are what make it read as routing rather than as a grid.
|
||||||
|
float horizontal = step(1.0 - u_fill, rnd);
|
||||||
|
float vertical = step(1.0 - u_fill, rnd2);
|
||||||
|
|
||||||
|
float width = u_trace * (0.6 + u_sigLine);
|
||||||
|
float dH = abs(f.y);
|
||||||
|
float dV = abs(f.x);
|
||||||
|
|
||||||
|
// Distance from the centre of the board, used to gate growth outward.
|
||||||
|
float reach = sat(1.4 - length(p) * 0.5);
|
||||||
|
|
||||||
|
float traceMask = 0.0;
|
||||||
|
if (horizontal > 0.5) traceMask += smoothstep(width, width * 0.35, dH);
|
||||||
|
if (vertical > 0.5) traceMask += smoothstep(width, width * 0.35, dV);
|
||||||
|
traceMask = sat(traceMask) * reach;
|
||||||
|
|
||||||
|
vec3 traceColor = palRamp(rnd * 0.4 + 0.15);
|
||||||
|
col += traceColor * traceMask * 0.5;
|
||||||
|
|
||||||
|
// Travelling pulse: a bright packet running along the trace, its position a
|
||||||
|
// function of the cell's own hash so packets are out of step with each other.
|
||||||
|
float along = horizontal > 0.5 ? f.x : f.y;
|
||||||
|
float packet = fract(rnd * 3.1 + t * (0.4 + rnd2 * 0.8));
|
||||||
|
float dPacket = abs(along - (packet - 0.5));
|
||||||
|
col += pal(4) * traceMask * exp(-dPacket * dPacket * 260.0) * u_pulse;
|
||||||
|
|
||||||
|
// Pads sit where both traces meet, stamped in the signature form.
|
||||||
|
if (horizontal > 0.5 && vertical > 0.5 && rnd2 > 1.0 - u_pads) {
|
||||||
|
float d = sigShape(f / max(u_padSize, 1e-3)) * u_padSize;
|
||||||
|
col += pal(2) * smoothstep(0.01, -0.01, d) * reach * 0.7;
|
||||||
|
col += pal(3) * sigEdge(d) * reach * (0.4 + u_pulse * 0.4);
|
||||||
|
}
|
||||||
|
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default circuitBloom;
|
||||||
66
flow-state/src/scenes/shader/classic-wave.js
Normal file
66
flow-state/src/scenes/shader/classic-wave.js
Normal file
@ -0,0 +1,66 @@
|
|||||||
|
// Ported from party-stage's "Classic Wave". The original picked a hue from a
|
||||||
|
// rolling HSV ramp; here the ramp indexes the palette instead, so the look
|
||||||
|
// generator can actually recolour it.
|
||||||
|
|
||||||
|
export const classicWave = {
|
||||||
|
name: 'Classic Wave',
|
||||||
|
family: 'flow',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
// Grain is this scene's ONLY expression of the style trait — it draws no
|
||||||
|
// hard edges to weight — so it keeps a share of it rather than opting out.
|
||||||
|
texture: 0.35,
|
||||||
|
traits: ['shape', 'camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
rings: { type: 'float', range: [4, 40], default: 18, uniform: 'u_rings', bias: 'density' },
|
||||||
|
spokes: { type: 'int', range: [0, 12], default: 5, uniform: 'u_spokes' },
|
||||||
|
speed: { type: 'float', range: [0.2, 2.5], default: 1.0, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
// colorRoll is capped low on purpose. palRamp() steps through all six
|
||||||
|
// palette entries per unit of t, and the palette generator deliberately
|
||||||
|
// spreads their luminance — so this multiplies into a whole-frame
|
||||||
|
// brightness oscillation at six times its own rate. At the original
|
||||||
|
// range of [0, 0.5] this scene measured 7 flashes per second at every
|
||||||
|
// output resolution, well past the WCAG 2.3.1 ceiling of 3. Capped here,
|
||||||
|
// the worst case is ~0.9 Hz. See engine/flash.js.
|
||||||
|
colorRoll: { type: 'float', range: [0, 0.06], default: 0.02, uniform: 'u_colorRoll', rate: true },
|
||||||
|
softness: { type: 'float', range: [0, 1], default: 0.4, uniform: 'u_softness' },
|
||||||
|
bloomCore: { type: 'float', range: [0, 1.5], default: 0.5, uniform: 'u_bloomCore', bias: 'energy' },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
bloomCore: { feature: 'beat', amount: 0.4, response: 'spike' },
|
||||||
|
rings: { feature: 'bandMid', amount: 0.12 },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
p = sigCamera(p);
|
||||||
|
// The rings take the track's signature form: round tracks get circles,
|
||||||
|
// hexagonal tracks get hexagonal rings, and it costs one call.
|
||||||
|
float d = sigShape(p) + 1.0;
|
||||||
|
float angle = atan(p.y, p.x);
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
|
||||||
|
float wave = sin(d * u_rings - t * 2.0);
|
||||||
|
float spoke = u_spokes > 0 ? sin(angle * float(u_spokes) + t) * u_beat : 0.0;
|
||||||
|
|
||||||
|
float v = 0.5 + 0.5 * sin(wave + spoke);
|
||||||
|
v = mix(v, smoothstep(0.2, 0.8, v), u_softness);
|
||||||
|
|
||||||
|
vec3 col = palRamp(t * u_colorRoll + d * 0.25) * v;
|
||||||
|
|
||||||
|
// Core glow, the part that reads as the "hit".
|
||||||
|
col += pal(0) * (1.0 - smoothstep(0.0, 0.7, d)) * u_bloomCore * 0.6;
|
||||||
|
|
||||||
|
// Keep the corners from clipping to flat colour.
|
||||||
|
col *= 0.6 + 0.4 * (1.0 - smoothstep(0.8, 1.8, d));
|
||||||
|
col += sigGrain(uv);
|
||||||
|
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default classicWave;
|
||||||
59
flow-state/src/scenes/shader/curl-flow.js
Normal file
59
flow-state/src/scenes/shader/curl-flow.js
Normal file
@ -0,0 +1,59 @@
|
|||||||
|
// Flow family: streaks advected along a curl-noise field.
|
||||||
|
//
|
||||||
|
// Uses the compositor's feedback texture rather than integrating positions, so
|
||||||
|
// the trails cost nothing in state and a seek still lands correctly once the
|
||||||
|
// feedback buffer has converged.
|
||||||
|
|
||||||
|
export const curlFlow = {
|
||||||
|
name: 'Curl Flow',
|
||||||
|
family: 'flow',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
traits: ['camera', 'space', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
scale: { type: 'float', range: [0.5, 6], default: 2.0, uniform: 'u_scale', bias: 'density' },
|
||||||
|
speed: { type: 'float', range: [0.02, 0.5], default: 0.12, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
streak: { type: 'float', range: [0, 1], default: 0.55, uniform: 'u_streak' },
|
||||||
|
contrast: { type: 'float', range: [0.5, 4], default: 1.6, uniform: 'u_contrast' },
|
||||||
|
veins: { type: 'float', range: [1, 12], default: 5.0, uniform: 'u_veins', bias: 'density' },
|
||||||
|
glow: { type: 'float', range: [0, 1.2], default: 0.35, uniform: 'u_glow', bias: 'energy' },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
glow: { feature: 'beat', amount: 0.3, response: 'spike' },
|
||||||
|
veins: { feature: 'bandMid', amount: 0.25 },
|
||||||
|
streak: { feature: 'flux', amount: 0.2, response: 'smooth' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
vec2 flow = curl(p * u_scale + vec2(t, -t * 0.7), t * 0.5);
|
||||||
|
vec2 q = p + flow * 0.35;
|
||||||
|
|
||||||
|
// Ridged noise gives filament-like veins rather than soft cloud.
|
||||||
|
float n = fbm(q * u_scale * 1.4 + t * 0.6, 5);
|
||||||
|
float veins = 1.0 - abs(sin(n * u_veins + t * 2.0));
|
||||||
|
veins = pow(sat(veins), u_contrast);
|
||||||
|
|
||||||
|
vec3 col = mix(pal(0) * 0.12, pal(1), veins);
|
||||||
|
col += pal(2) * pow(veins, 3.0) * u_glow;
|
||||||
|
col = mix(col, pal(3), sat(length(flow) * 0.4) * 0.35);
|
||||||
|
|
||||||
|
// Feedback trails: the previous frame, pulled slightly along the flow.
|
||||||
|
vec3 trail = prev(uv - flow * 0.004);
|
||||||
|
col = max(col, trail * u_streak);
|
||||||
|
|
||||||
|
col *= 0.6 + 0.4 * exp(-dot(p, p) * 0.35);
|
||||||
|
col = sigAir(col, p, smoothstep(0.0, 1.7, length(p)));
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default curlFlow;
|
||||||
91
flow-state/src/scenes/shader/dust-chamber.js
Normal file
91
flow-state/src/scenes/shader/dust-chamber.js
Normal file
@ -0,0 +1,91 @@
|
|||||||
|
// Minimal family: a nearly empty volume with a shaft of light through it and a
|
||||||
|
// few motes suspended in the beam.
|
||||||
|
//
|
||||||
|
// Distinct from Firefly Drift (a swarm advected along a flow) and Slow Orb (one
|
||||||
|
// body): almost nothing moves here. The motes hold position and only breathe;
|
||||||
|
// what changes is the light. That makes this one of the very few scenes in the
|
||||||
|
// library that can hold a thirty-second intro without asking for attention.
|
||||||
|
//
|
||||||
|
// The beam lands on the track's horizon, so the room is the same room every
|
||||||
|
// other scene with a floor is standing in.
|
||||||
|
|
||||||
|
export const dustChamber = {
|
||||||
|
name: 'Dust Chamber',
|
||||||
|
family: 'minimal',
|
||||||
|
kind: 'fragment',
|
||||||
|
traits: ['shape', 'camera', 'space', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
motes: { type: 'int', range: [6, 40], default: 18, uniform: 'u_motes', bias: 'density' },
|
||||||
|
moteSize: { type: 'float', range: [0.004, 0.05], default: 0.014, uniform: 'u_moteSize' },
|
||||||
|
beam: { type: 'float', range: [0.1, 1.0], default: 0.45, uniform: 'u_beam', bias: 'energy' },
|
||||||
|
beamWidth:{ type: 'float', range: [0.15, 1.2], default: 0.5, uniform: 'u_beamWidth' },
|
||||||
|
sway: { type: 'float', range: [0, 0.12], default: 0.04, uniform: 'u_sway' },
|
||||||
|
drift: { type: 'float', range: [0.01, 0.3], default: 0.06, uniform: 'u_drift', bias: 'motion', rate: true },
|
||||||
|
palette: { type: 'palette', count: 4 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
beam: { feature: 'loudness', amount: 0.25, response: 'smooth' },
|
||||||
|
moteSize: { feature: 'beat', amount: 0.2, response: 'spike' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_drift + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
float floorY = sigHorizonY() * 0.7 - 0.6;
|
||||||
|
|
||||||
|
// The shaft: a soft wedge widening as it falls, cut off at the floor.
|
||||||
|
float axis = sin(u_seed * 0.7) * 0.35;
|
||||||
|
float down = sat((1.0 - (p.y - floorY)) * 0.6);
|
||||||
|
float halfWidth = u_beamWidth * (0.25 + down * 0.75);
|
||||||
|
float inBeam = exp(-pow((p.x - axis) / max(halfWidth, 1e-3), 2.0) * 2.2);
|
||||||
|
inBeam *= smoothstep(floorY - 0.05, floorY + 0.5, p.y);
|
||||||
|
|
||||||
|
// The light is the thing that moves here, and it has to move on its own
|
||||||
|
// clock rather than on u_drift: at a low drift the motes barely breathe and
|
||||||
|
// the whole frame measured as static. Fixed rate, so it is not a param and
|
||||||
|
// cannot be reacted into a phase jump.
|
||||||
|
float breath = 0.82 + 0.18 * sin(u_time * 0.4 + u_seed);
|
||||||
|
// Floored, because the shaft IS the scene. An intro sampling u_beam near
|
||||||
|
// its minimum with a narrow beam produced a frame dark enough to fail the
|
||||||
|
// live-frame gate — a dim room is the point, an empty one is a bug.
|
||||||
|
float beam = (0.22 + u_beam * 0.78) * breath;
|
||||||
|
|
||||||
|
// Ambient fill. It carries the whole frame wherever the shaft is not, so it
|
||||||
|
// is what decides whether "almost black" stays on the right side of black.
|
||||||
|
vec3 col = pal(0) * 0.11;
|
||||||
|
col += pal(1) * inBeam * beam * 0.5;
|
||||||
|
|
||||||
|
// The pool where the shaft meets the floor.
|
||||||
|
float pool = exp(-abs(p.y - floorY) * 14.0)
|
||||||
|
* exp(-pow((p.x - axis) / max(halfWidth * 1.3, 1e-3), 2.0));
|
||||||
|
col += pal(2) * pool * beam * 0.7;
|
||||||
|
|
||||||
|
// Motes: fixed positions, breathing brightness, only visible in the light.
|
||||||
|
for (int i = 0; i < 40; i++) {
|
||||||
|
if (i >= u_motes) break;
|
||||||
|
float fi = float(i);
|
||||||
|
float s = u_seed + fi * 53.7;
|
||||||
|
|
||||||
|
vec2 at = vec2(
|
||||||
|
(hash11(s) - 0.5) * 2.4 + sin(t * 0.8 + s) * u_sway,
|
||||||
|
(hash11(s + 9.1) - 0.5) * 1.8 + cos(t * 0.6 + s * 1.3) * u_sway
|
||||||
|
);
|
||||||
|
|
||||||
|
float lit = exp(-pow((at.x - axis) / max(halfWidth, 1e-3), 2.0) * 2.0);
|
||||||
|
float pulse = 0.55 + 0.45 * sin(t * 2.0 + s * 3.0);
|
||||||
|
float m = sigForm(p, at, u_moteSize * (0.6 + hash11(s + 3.3)));
|
||||||
|
col += pal(3) * m * lit * pulse * (0.5 + beam);
|
||||||
|
}
|
||||||
|
|
||||||
|
col = sigAir(col, p, smoothstep(0.0, 1.5, length(p)));
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default dustChamber;
|
||||||
88
flow-state/src/scenes/shader/eclipse-field.js
Normal file
88
flow-state/src/scenes/shader/eclipse-field.js
Normal file
@ -0,0 +1,88 @@
|
|||||||
|
// Minimal family: a large occluder covering a bright source, with the light
|
||||||
|
// escaping past its edge.
|
||||||
|
//
|
||||||
|
// Distinct from Slow Orb (a small soft body drifting through empty space): the
|
||||||
|
// object here nearly fills the frame, its edge is hard, and it is not the
|
||||||
|
// subject — the subject is the corona around it. That inversion is the whole
|
||||||
|
// idea, and it gives the library its one genuinely high-contrast minimal scene:
|
||||||
|
// almost black, with a thin ring of the brightest thing in the video.
|
||||||
|
//
|
||||||
|
// The occluder is the track's signature form, so this is where a hexagonal
|
||||||
|
// track's hexagon is most legible.
|
||||||
|
//
|
||||||
|
// Scaffolded by tools/new-scene.js. See HOWTO-visualizers.md.
|
||||||
|
|
||||||
|
export const eclipseField = {
|
||||||
|
name: 'Eclipse Field',
|
||||||
|
family: 'minimal',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Crisp line work: the track's surface grain would only fur the edges.
|
||||||
|
texture: 0,
|
||||||
|
traits: ['shape', 'camera', 'space', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
size: { type: 'float', range: [0.2, 0.85], default: 0.45, uniform: 'u_size' },
|
||||||
|
corona: { type: 'float', range: [0.05, 1.2], default: 0.45, uniform: 'u_corona', bias: 'energy' },
|
||||||
|
rays: { type: 'float', range: [0, 1], default: 0.35, uniform: 'u_rays' },
|
||||||
|
rayCount: { type: 'float', range: [3, 24], default: 9, uniform: 'u_rayCount', bias: 'density' },
|
||||||
|
offset: { type: 'float', range: [0, 0.5], default: 0.12, uniform: 'u_offset' },
|
||||||
|
drift: { type: 'float', range: [0.01, 0.3], default: 0.05, uniform: 'u_drift', bias: 'motion', rate: true },
|
||||||
|
sky: { type: 'float', range: [0, 0.6], default: 0.15, uniform: 'u_sky' },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
corona: { feature: 'loudness', amount: 0.3, response: 'smooth' },
|
||||||
|
rays: { feature: 'bandAir', amount: 0.25, response: 'smooth' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_drift + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
// The light sits a little off centre from the occluder, so the corona is
|
||||||
|
// brighter on one side — a perfectly concentric eclipse looks like a decal.
|
||||||
|
vec2 lightAt = vec2(sin(t * 0.8), cos(t * 0.6)) * u_offset;
|
||||||
|
vec2 discAt = vec2(0.0, sigHorizonY() * 0.25);
|
||||||
|
|
||||||
|
// Distance to the occluder's edge, in the track's form.
|
||||||
|
float d = sigShape((p - discAt) / max(u_size, 1e-3)) * u_size;
|
||||||
|
|
||||||
|
// The source behind it.
|
||||||
|
float toLight = length(p - lightAt);
|
||||||
|
float source = exp(-toLight * toLight * 2.2);
|
||||||
|
|
||||||
|
vec3 col = pal(0) * u_sky * (1.0 - smoothstep(0.0, 1.6, length(p)));
|
||||||
|
col += pal(1) * source * 0.5;
|
||||||
|
|
||||||
|
// Corona: brightest exactly at the edge, falling off outward only. Inside
|
||||||
|
// the disc it must be zero, or the occluder stops occluding.
|
||||||
|
float outside = smoothstep(0.0, 0.02, d);
|
||||||
|
float ring = exp(-max(d, 0.0) / max(u_corona * 0.25, 0.01));
|
||||||
|
// The corona breathes on a fixed clock rather than on u_drift. At a low
|
||||||
|
// drift — which is most of this scene's range, it is a minimal scene —
|
||||||
|
// nothing else in the frame moved enough to register as animation at all.
|
||||||
|
float breath = 0.82 + 0.18 * sin(u_time * 0.5 + u_seed * 1.7);
|
||||||
|
col += palRamp(0.35 + d * 0.5) * ring * outside * (0.8 + u_corona) * breath;
|
||||||
|
|
||||||
|
// Rays: angular streaks in the corona, rotating slowly. Confined to the
|
||||||
|
// corona by the same falloff, so they never light the whole frame.
|
||||||
|
if (u_rays > 0.01) {
|
||||||
|
float ang = atan(p.y - discAt.y, p.x - discAt.x);
|
||||||
|
float spokes = 0.5 + 0.5 * sin(ang * u_rayCount + t * 2.0 + u_time * 0.3);
|
||||||
|
col += pal(4) * pow(spokes, 3.0) * ring * outside * u_rays;
|
||||||
|
}
|
||||||
|
|
||||||
|
// The occluder: not pure black, so it reads as an object rather than a hole.
|
||||||
|
col = mix(col, pal(0) * 0.12, smoothstep(0.004, -0.004, d));
|
||||||
|
col += pal(3) * sigEdge(d) * (0.4 + u_corona * 0.5);
|
||||||
|
|
||||||
|
col = sigAir(col, p, smoothstep(0.0, 1.7, length(p)));
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default eclipseField;
|
||||||
62
flow-state/src/scenes/shader/firefly-drift.js
Normal file
62
flow-state/src/scenes/shader/firefly-drift.js
Normal file
@ -0,0 +1,62 @@
|
|||||||
|
// Flow family: a drifting swarm of glowing motes advected along a curl-noise
|
||||||
|
// field. Each mote rides a divergence-free curl flow and wanders on top of it,
|
||||||
|
// so the swarm pools and diverges like embers on a breeze rather than orbiting.
|
||||||
|
//
|
||||||
|
// A base field is always present so the scene stands alone before any feedback
|
||||||
|
// has converged; the motes just make the movement legible.
|
||||||
|
|
||||||
|
export const fireflyDrift = {
|
||||||
|
name: 'Firefly Drift',
|
||||||
|
family: 'flow',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
traits: ['camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
count: { type: 'float', range: [6, 48], default: 22, uniform: 'u_count', bias: 'density' },
|
||||||
|
speed: { type: 'float', range: [0.05, 0.8], default: 0.3, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
glow: { type: 'float', range: [0, 1.2], default: 0.5, uniform: 'u_glow', bias: 'energy' },
|
||||||
|
jitter: { type: 'float', range: [0, 0.4], default: 0.12, uniform: 'u_jitter' },
|
||||||
|
spread: { type: 'float', range: [0.4, 1.3], default: 0.95, uniform: 'u_spread' },
|
||||||
|
palette: { type: 'palette', count: 4 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
glow: { feature: 'beat', amount: 0.3, response: 'spike' },
|
||||||
|
jitter: { feature: 'bandHigh', amount: 0.2, response: 'smooth' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
// Soft ambient wash so the frame is never black, even standing alone.
|
||||||
|
vec3 col = pal(0) * (0.06 + 0.05 * fbm(p * 1.5 + vec2(0.0, t * 0.3), 3));
|
||||||
|
|
||||||
|
for (int i = 0; i < 48; i++) {
|
||||||
|
if (float(i) >= u_count) break;
|
||||||
|
float fi = float(i);
|
||||||
|
|
||||||
|
// Seed the mote to a stable place in the frame (deterministic per i).
|
||||||
|
vec2 grid = vec2(hash12(vec2(fi, 1.7)), hash12(vec2(fi, 9.1)));
|
||||||
|
vec2 base = (grid - 0.5) * 2.0 * u_spread * vec2(1.0, 0.7);
|
||||||
|
|
||||||
|
// Advect along the curl field plus a bounded wander term on top.
|
||||||
|
vec2 flow = curl(grid * 3.0 + vec2(0.0, t * 0.15), t * 0.5);
|
||||||
|
vec2 pos = base + flow * 1.3
|
||||||
|
+ vec2(sin(t * (0.4 + fract(fi * 0.13))),
|
||||||
|
cos(t * (0.5 + fract(fi * 0.29)))) * u_jitter;
|
||||||
|
|
||||||
|
float d = length(p - pos);
|
||||||
|
|
||||||
|
vec3 hc = pal(int(mod(fi, 4.0)));
|
||||||
|
col += hc * (exp(-d * d / (0.015 + u_glow * 0.04)) * (1.0 + u_glow * 0.6));
|
||||||
|
col += hc * exp(-d * d * 60.0);
|
||||||
|
}
|
||||||
|
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
72
flow-state/src/scenes/shader/floating-geometry.js
Normal file
72
flow-state/src/scenes/shader/floating-geometry.js
Normal file
@ -0,0 +1,72 @@
|
|||||||
|
// Ported from party-stage's "Floating Geometry". Shape count, size and motion
|
||||||
|
// were fixed constants in the original; they are the whole point of the scene,
|
||||||
|
// so they are now params the look generator can move.
|
||||||
|
|
||||||
|
export const floatingGeometry = {
|
||||||
|
name: 'Floating Geometry',
|
||||||
|
family: 'geometric',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js. This scene is nothing but a handful
|
||||||
|
// of shapes, so `shape` is the trait it exists to express.
|
||||||
|
// Takes the track's surface grain, but lightly — this is drawn, not filmed.
|
||||||
|
texture: 0.4,
|
||||||
|
traits: ['shape', 'camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
count: { type: 'int', range: [2, 14], default: 6, uniform: 'u_count', bias: 'density' },
|
||||||
|
size: { type: 'float', range: [0.04, 0.3],default: 0.15,uniform: 'u_size' },
|
||||||
|
drift: { type: 'float', range: [0.1, 1.5], default: 0.4, uniform: 'u_drift', bias: 'motion', rate: true },
|
||||||
|
spin: { type: 'float', range: [0, 2], default: 0.6, uniform: 'u_spin', rate: true },
|
||||||
|
variety: { type: 'float', range: [0, 0.6], default: 0.25, uniform: 'u_variety' },
|
||||||
|
aura: { type: 'float', range: [0, 1], default: 0.25,uniform: 'u_aura', bias: 'energy' },
|
||||||
|
spread: { type: 'float', range: [0.3, 1.2], default: 0.8, uniform: 'u_spread' },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
size: { feature: 'beat', amount: 0.18, response: 'spike' },
|
||||||
|
aura: { feature: 'bandHigh', amount: 0.4 },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_drift + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
// Background wash from the two darkest palette entries.
|
||||||
|
vec3 col = mix(pal(0) * 0.18, pal(1) * 0.24, sin(t) * 0.5 + 0.5);
|
||||||
|
|
||||||
|
for (int i = 0; i < 14; i++) {
|
||||||
|
if (i >= u_count) break;
|
||||||
|
float fi = float(i);
|
||||||
|
float s = u_seed + fi * 123.456;
|
||||||
|
|
||||||
|
vec2 pos = vec2(
|
||||||
|
sin(t * 0.5 + s) * u_spread,
|
||||||
|
cos(t * 0.3 + s * 1.1) * u_spread * 0.55
|
||||||
|
);
|
||||||
|
|
||||||
|
vec2 sp = rot(t * (0.2 + fract(s) * u_spin)) * (p - pos);
|
||||||
|
|
||||||
|
float size = u_size * (0.6 + fract(s * 0.7) * 0.8);
|
||||||
|
|
||||||
|
// Every element is the track's signature form. This scene used to pick
|
||||||
|
// between a box and a circle per element, which is precisely the choice
|
||||||
|
// the production design should be making — one video, one cast.
|
||||||
|
// The variety param only scales them apart; it never changes what they are.
|
||||||
|
float scale = size * (1.0 + (fract(s * 0.37) - 0.5) * u_variety);
|
||||||
|
float d = sigShape(sp / max(scale, 1e-3)) * scale;
|
||||||
|
|
||||||
|
vec3 shapeColor = pal(i + int(floor(t * 0.3)));
|
||||||
|
float intensity = smoothstep(0.012, 0.0, d) + sigEdge(d) * 0.35;
|
||||||
|
col = mix(col, shapeColor, intensity * 0.85);
|
||||||
|
col += shapeColor * (1.0 - smoothstep(0.0, size * 2.2, abs(d))) * u_aura;
|
||||||
|
}
|
||||||
|
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default floatingGeometry;
|
||||||
119
flow-state/src/scenes/shader/flora.js
Normal file
119
flow-state/src/scenes/shader/flora.js
Normal file
@ -0,0 +1,119 @@
|
|||||||
|
// Organic family: an abstract plant — a plume of swaying stalks with teardrop
|
||||||
|
// leaves and a glowing bloom at each crown.
|
||||||
|
//
|
||||||
|
// Abstract on purpose: no soil, no foreground, just the silhouette of growth.
|
||||||
|
// Every petal and the crown are stamped in the track's signature form, so a
|
||||||
|
// hexagonal track grows hexagonal flora. The stalk bends on bass and the bloom
|
||||||
|
// pulses on the beat — the plant dances to the track rather than the camera
|
||||||
|
// shaking.
|
||||||
|
|
||||||
|
export const flora = {
|
||||||
|
name: 'Flora',
|
||||||
|
family: 'organic',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
traits: ['shape', 'camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
count: { type: 'float', range: [1, 5], default: 3, uniform: 'u_count', bias: 'density' },
|
||||||
|
height: { type: 'float', range: [0.6, 2.0], default: 1.1, uniform: 'u_height', bias: 'energy' },
|
||||||
|
stem: { type: 'float', range: [0.008, 0.05], default: 0.02, uniform: 'u_stem' },
|
||||||
|
sway: { type: 'float', range: [0.05, 0.5], default: 0.2, uniform: 'u_sway', bias: 'motion' },
|
||||||
|
swayRate:{ type: 'float', range: [0.1, 1.2], default: 0.6, uniform: 'u_swayRate', rate: true },
|
||||||
|
leaf: { type: 'float', range: [0.12, 0.5], default: 0.26, uniform: 'u_leaf' },
|
||||||
|
bud: { type: 'float', range: [0, 1], default: 0.55, uniform: 'u_bud', bias: 'energy' },
|
||||||
|
palette: { type: 'palette', count: 4 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
sway: { feature: 'bandLow', amount: 0.25, response: 'smooth' },
|
||||||
|
bud: { feature: 'beat', amount: 0.3, response: 'smooth' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
// Horizontal lean of a spine at parametric height h (0 base .. 1 tip), seeded
|
||||||
|
// per plant and swaying more toward the crown like a real stem. Returns a value
|
||||||
|
// scaled by u_sway later, so the reactive bend applies to every part at once.
|
||||||
|
float swayAt(float h, float phase, float t) {
|
||||||
|
return sin(t * (0.35 + fract(phase * 1.7)) + phase)
|
||||||
|
+ h * sin(t * (0.35 + fract(phase * 0.6)) + phase * 2.1);
|
||||||
|
}
|
||||||
|
|
||||||
|
// A petal: an elongated blossom drawn in the track's signature form, growing
|
||||||
|
// forward along dir from a node and clipped to a blade shape.
|
||||||
|
float petal(vec2 p, vec2 node, vec2 dir, float len, float width) {
|
||||||
|
vec2 off = p - node;
|
||||||
|
float along = dot(off, dir);
|
||||||
|
vec2 perp = vec2(-dir.y, dir.x);
|
||||||
|
float across = dot(off, perp);
|
||||||
|
float taper = 0.6 + 0.4 * sat(along / max(len, 1e-3));
|
||||||
|
vec2 q = vec2(along / max(len, 1e-3), across / max(width * taper, 1e-3));
|
||||||
|
float form = sigForm(q * 0.5, vec2(0.0), 0.5);
|
||||||
|
float clip = smoothstep(0.0, -0.1, along) * smoothstep(len + 0.12, len * 0.72, along);
|
||||||
|
return form * clip;
|
||||||
|
}
|
||||||
|
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_swayRate + u_seed;
|
||||||
|
float beat = u_beat;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
// Soft vertical wash behind the plants.
|
||||||
|
vec3 col = mix(pal(0) * 0.03, pal(1) * 0.06, sat((p.y + 1.0) * 0.5));
|
||||||
|
|
||||||
|
for (int i = 0; i < 5; i++) {
|
||||||
|
if (i >= int(u_count)) break;
|
||||||
|
float fi = float(i);
|
||||||
|
float s = u_seed + fi * 43.7;
|
||||||
|
float baseX = (hash11(s * 1.7) - 0.5) * 1.25;
|
||||||
|
float height = u_height * (0.55 + 0.45 * hash11(s * 3.1));
|
||||||
|
float baseY = -0.92;
|
||||||
|
|
||||||
|
// Stem, sampled so the bend follows the sway curve rather than a chord.
|
||||||
|
float stemMask = 0.0;
|
||||||
|
for (int j = 0; j < 12; j++) {
|
||||||
|
float hj = (float(j) + 0.5) / 12.0;
|
||||||
|
float cx = baseX + swayAt(hj, s, t) * u_sway;
|
||||||
|
float cy = baseY + hj * height;
|
||||||
|
float d = length(p - vec2(cx, cy));
|
||||||
|
float taper = max(u_stem * (1.0 - hj * 0.8) + 0.002, 0.003);
|
||||||
|
stemMask = max(stemMask, smoothstep(taper + 0.004, taper - 0.004, d));
|
||||||
|
}
|
||||||
|
col = mix(col, mix(pal(1), pal(3), 0.3), stemMask);
|
||||||
|
|
||||||
|
// Stalk tip, reused for the petal rows and the bloom.
|
||||||
|
float tipX = baseX + swayAt(1.0, s, t) * u_sway;
|
||||||
|
float tipY = baseY + height;
|
||||||
|
|
||||||
|
// Petals up the upper half, alternating sides.
|
||||||
|
for (int l = 0; l < 6; l++) {
|
||||||
|
float fll = float(l);
|
||||||
|
float node = 0.45 + floor(fll * 0.5) * 0.12;
|
||||||
|
float side = fract(fll * 0.5) < 0.5 ? -1.0 : 1.0;
|
||||||
|
float nx = baseX + swayAt(node, s, t) * u_sway;
|
||||||
|
float ny = baseY + node * height;
|
||||||
|
vec2 dir = normalize(vec2(side, 0.55));
|
||||||
|
float len = u_leaf * 2.0;
|
||||||
|
float m = petal(p, vec2(nx, ny), dir, len, u_leaf);
|
||||||
|
vec3 leafCol = mix(pal(2), pal(3), fract(s * 0.4));
|
||||||
|
col = mix(col, leafCol, m);
|
||||||
|
col += pal(3) * m * (0.2 + 0.3 * beat);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Bloom: a small corona of signature forms around the crown.
|
||||||
|
float bloom = 0.0;
|
||||||
|
for (int k = 0; k < 6; k++) {
|
||||||
|
float a = 6.2831853 * (float(k) + 0.5) / 6.0 + s;
|
||||||
|
vec2 off = vec2(cos(a), sin(a)) * u_leaf * (0.30 + 0.2 * beat);
|
||||||
|
bloom += sigForm(p - vec2(tipX, tipY), off, u_leaf * 0.5);
|
||||||
|
}
|
||||||
|
col = mix(col, pal(2), sat(bloom) * u_bud * (0.6 + 0.4 * beat));
|
||||||
|
}
|
||||||
|
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default flora;
|
||||||
91
flow-state/src/scenes/shader/gate-corridor.js
Normal file
91
flow-state/src/scenes/shader/gate-corridor.js
Normal file
@ -0,0 +1,91 @@
|
|||||||
|
// Structural family: a corridor of nested gates receding to a vanishing point,
|
||||||
|
// travelled through.
|
||||||
|
//
|
||||||
|
// Rectilinear depth, which the library did not have: Kaleido Tunnel recedes but
|
||||||
|
// is radial and folded, Pylon Grid stands still, Neon City is front-on. Here the
|
||||||
|
// camera moves forward through a series of frames that scale up and pass, and
|
||||||
|
// the ring geometry is the track's signature form, so a hexagonal video travels
|
||||||
|
// through hexagonal gates.
|
||||||
|
//
|
||||||
|
// Motion is a saw on log-depth, which is what makes gates emerge from the
|
||||||
|
// vanishing point at a constant apparent rate instead of rushing at the end.
|
||||||
|
|
||||||
|
export const gateCorridor = {
|
||||||
|
name: 'Gate Corridor',
|
||||||
|
family: 'structural',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Takes the track's surface grain, but lightly — this is drawn, not filmed.
|
||||||
|
texture: 0.4,
|
||||||
|
traits: ['shape', 'camera', 'space', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
gates: { type: 'int', range: [3, 14], default: 8, uniform: 'u_gates', bias: 'density' , slowAxis: true },
|
||||||
|
aperture: { type: 'float', range: [0.15, 0.9], default: 0.45, uniform: 'u_aperture' },
|
||||||
|
thickness:{ type: 'float', range: [0.02, 0.3], default: 0.09, uniform: 'u_thickness' },
|
||||||
|
travel: { type: 'float', range: [0.02, 0.7], default: 0.2, uniform: 'u_travel', bias: 'motion', rate: true },
|
||||||
|
rails: { type: 'float', range: [0, 1], default: 0.4, uniform: 'u_rails' },
|
||||||
|
lamps: { type: 'float', range: [0, 1.4], default: 0.5, uniform: 'u_lamps', bias: 'energy' },
|
||||||
|
vanish: { type: 'float', range: [0, 0.5], default: 0.15, uniform: 'u_vanish' },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
lamps: { feature: 'beat', amount: 0.35, response: 'spike' },
|
||||||
|
aperture: { feature: 'bandLow', amount: 0.15, response: 'smooth' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_travel + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
// The vanishing point sits on the track's horizon, slightly off centre.
|
||||||
|
vec2 vanishAt = vec2(sin(u_seed) * u_vanish, sigHorizonY() * 0.35);
|
||||||
|
vec2 q = p - vanishAt;
|
||||||
|
|
||||||
|
vec3 col = pal(0) * 0.04;
|
||||||
|
|
||||||
|
// Depth rails converging on the vanishing point.
|
||||||
|
if (u_rails > 0.01) {
|
||||||
|
float ang = atan(q.y, q.x);
|
||||||
|
float spokes = abs(fract(ang * 1.9098 + 0.5) - 0.5) * 2.0; // 12 rails
|
||||||
|
float rail = smoothstep(0.06, 0.0, spokes) * smoothstep(0.02, 0.5, length(q));
|
||||||
|
col += pal(1) * rail * u_rails * 0.25;
|
||||||
|
}
|
||||||
|
|
||||||
|
for (int i = 0; i < 14; i++) {
|
||||||
|
if (i >= u_gates) break;
|
||||||
|
float fi = float(i);
|
||||||
|
|
||||||
|
// Log-spaced depth with a saw: each gate walks forward, and when it
|
||||||
|
// passes the camera it wraps to the far end.
|
||||||
|
float phase = fract((fi / float(u_gates)) + t);
|
||||||
|
float scale = u_aperture * exp(phase * 3.2) * 0.35;
|
||||||
|
|
||||||
|
float d = abs(sigShape(q / max(scale, 1e-3)) * max(scale, 1e-3));
|
||||||
|
|
||||||
|
// Near gates are drawn thicker and brighter: the only depth cue that
|
||||||
|
// matters once the geometry is right.
|
||||||
|
float near = phase;
|
||||||
|
float w = u_thickness * (0.25 + near * 1.2) * (0.5 + u_sigLine);
|
||||||
|
float frame = smoothstep(w, w * 0.25, d);
|
||||||
|
|
||||||
|
vec3 tint = palRamp(fi * 0.13 + 0.1);
|
||||||
|
col += tint * frame * (0.25 + near * 0.75);
|
||||||
|
col += tint * exp(-d * 14.0) * near * 0.2;
|
||||||
|
|
||||||
|
// A lamp at the top of every third gate, pulsing on the beat.
|
||||||
|
if (mod(fi, 3.0) < 0.5) {
|
||||||
|
vec2 lampAt = vanishAt + vec2(0.0, scale);
|
||||||
|
col += pal(4) * exp(-length(p - lampAt) * 26.0) * u_lamps * (0.3 + near);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
col = sigAir(col, p, 1.0 - smoothstep(0.0, 1.2, length(q)));
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default gateCorridor;
|
||||||
127
flow-state/src/scenes/shader/girder-lattice.js
Normal file
127
flow-state/src/scenes/shader/girder-lattice.js
Normal file
@ -0,0 +1,127 @@
|
|||||||
|
// Structural family: an overhead truss, seen from underneath.
|
||||||
|
//
|
||||||
|
// Distinct from Pylon Grid (columns standing on the ground), Gate Corridor
|
||||||
|
// (frames receding head-on) and Neon City (a skyline at eye level): this is the
|
||||||
|
// only scene in the library whose subject is ABOVE the camera. Two parallel
|
||||||
|
// chords run away toward the horizon with diagonal bracing zigzagging between
|
||||||
|
// them, and the whole thing is looked up at, so the perspective converges
|
||||||
|
// downward rather than inward.
|
||||||
|
//
|
||||||
|
// Joint plates are stamped in the track's signature form.
|
||||||
|
//
|
||||||
|
// Scaffolded by tools/new-scene.js. See HOWTO-visualizers.md.
|
||||||
|
|
||||||
|
export const girderLattice = {
|
||||||
|
name: 'Girder Lattice',
|
||||||
|
family: 'structural',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Crisp line work: the track's surface grain would only fur the edges.
|
||||||
|
texture: 0,
|
||||||
|
traits: ['shape', 'camera', 'space', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
bays: { type: 'int', range: [3, 16], default: 8, uniform: 'u_bays', bias: 'density' },
|
||||||
|
beam: { type: 'float', range: [0.01, 0.12],default: 0.04, uniform: 'u_beam' },
|
||||||
|
depth: { type: 'float', range: [0.3, 2.2], default: 1.0, uniform: 'u_depth' },
|
||||||
|
brace: { type: 'float', range: [0, 1], default: 0.7, uniform: 'u_brace', bias: 'density' },
|
||||||
|
travel: { type: 'float', range: [0.0, 0.6], default: 0.12, uniform: 'u_travel', bias: 'motion', rate: true },
|
||||||
|
plate: { type: 'float', range: [0, 0.12], default: 0.05, uniform: 'u_plate' },
|
||||||
|
lamp: { type: 'float', range: [0, 1.4], default: 0.45, uniform: 'u_lamp', bias: 'energy' },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
lamp: { feature: 'beat', amount: 0.3, response: 'spike' },
|
||||||
|
brace: { feature: 'bandLow', amount: 0.15, response: 'smooth' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
// Distance to a line segment. The truss is nothing but segments.
|
||||||
|
float segment(vec2 p, vec2 a, vec2 b) {
|
||||||
|
vec2 pa = p - a, ba = b - a;
|
||||||
|
float h = clamp(dot(pa, ba) / max(dot(ba, ba), 1e-6), 0.0, 1.0);
|
||||||
|
return length(pa - ba * h);
|
||||||
|
}
|
||||||
|
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_travel + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
// Looking up: the vanishing point sits BELOW the frame's horizon, and the
|
||||||
|
// structure spans above it. Everything converges toward that point.
|
||||||
|
float vanishY = sigHorizonY() * 0.5 - 0.35;
|
||||||
|
float above = p.y - vanishY;
|
||||||
|
|
||||||
|
// Sky between the members.
|
||||||
|
vec3 col = mix(pal(0) * 0.1, pal(1) * 0.06, sat(above * 0.6));
|
||||||
|
|
||||||
|
if (above <= 0.001) {
|
||||||
|
col = sigAir(col, p, 1.0);
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
|
||||||
|
float best = 1e3;
|
||||||
|
float nearest = 0.0;
|
||||||
|
float plateHit = 1e3;
|
||||||
|
|
||||||
|
// Bays march toward the camera on a wrapping saw, so the truss travels
|
||||||
|
// overhead rather than sitting still.
|
||||||
|
for (int i = 0; i < 16; i++) {
|
||||||
|
if (i >= u_bays) break;
|
||||||
|
float fi = float(i);
|
||||||
|
float phase = fract(fi / float(u_bays) + t);
|
||||||
|
|
||||||
|
// Perspective: nearer bays are further up the frame and further apart.
|
||||||
|
float y0 = vanishY + exp(phase * 2.4) * 0.12 * u_depth;
|
||||||
|
float y1 = vanishY + exp((phase + 1.0 / float(u_bays)) * 2.4) * 0.12 * u_depth;
|
||||||
|
float halfW0 = (y0 - vanishY) * 1.5;
|
||||||
|
float halfW1 = (y1 - vanishY) * 1.5;
|
||||||
|
|
||||||
|
// The two chords, running away from the camera.
|
||||||
|
best = min(best, segment(p, vec2(-halfW0, y0), vec2(-halfW1, y1)));
|
||||||
|
best = min(best, segment(p, vec2(halfW0, y0), vec2(halfW1, y1)));
|
||||||
|
|
||||||
|
// Cross member at this joint.
|
||||||
|
best = min(best, segment(p, vec2(-halfW0, y0), vec2(halfW0, y0)));
|
||||||
|
|
||||||
|
// Diagonal bracing, alternating direction bay to bay: the zigzag is
|
||||||
|
// what makes it a truss rather than a ladder.
|
||||||
|
if (u_brace > 0.05) {
|
||||||
|
float flip = mod(fi, 2.0) < 0.5 ? 1.0 : -1.0;
|
||||||
|
float d = segment(p, vec2(-halfW0 * flip, y0), vec2(halfW1 * flip, y1));
|
||||||
|
best = min(best, d + (1.0 - u_brace) * 0.05);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (u_plate > 0.004) {
|
||||||
|
float pd = sigShape((p - vec2(halfW0, y0)) / u_plate) * u_plate;
|
||||||
|
plateHit = min(plateHit, pd);
|
||||||
|
pd = sigShape((p - vec2(-halfW0, y0)) / u_plate) * u_plate;
|
||||||
|
plateHit = min(plateHit, pd);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (phase > 0.75) nearest = max(nearest, phase);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Members are drawn thicker nearer the camera, which is the only depth cue
|
||||||
|
// a wireframe has.
|
||||||
|
float weight = u_beam * (0.4 + above * 0.9) * (0.5 + u_sigLine);
|
||||||
|
float steel = smoothstep(weight, weight * 0.3, best);
|
||||||
|
|
||||||
|
col = mix(col, pal(2) * (0.3 + above * 0.5), steel);
|
||||||
|
col += pal(3) * sigEdge(best - weight) * 0.4;
|
||||||
|
|
||||||
|
// Joint plates.
|
||||||
|
col = mix(col, pal(4) * 0.7, smoothstep(0.006, -0.006, plateHit));
|
||||||
|
|
||||||
|
// A lamp slung under the nearest bay, pulsing on the beat. Local.
|
||||||
|
col += pal(4) * exp(-length(p - vec2(0.0, vanishY + nearest * 1.4)) * 14.0) * u_lamp;
|
||||||
|
|
||||||
|
col = sigAir(col, p, 1.0 - smoothstep(0.0, 1.4, above));
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default girderLattice;
|
||||||
69
flow-state/src/scenes/shader/horizon-lines.js
Normal file
69
flow-state/src/scenes/shader/horizon-lines.js
Normal file
@ -0,0 +1,69 @@
|
|||||||
|
// Minimal family: a sparse field of horizontal lines that bend around the
|
||||||
|
// centre. Most of the frame is negative space, which is exactly what an intro or
|
||||||
|
// a breakdown wants — the arc driver has nowhere restful to go otherwise.
|
||||||
|
|
||||||
|
export const horizonLines = {
|
||||||
|
name: 'Horizon Lines',
|
||||||
|
family: 'minimal',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
// Crisp line work: the track's surface grain would only fur the edges.
|
||||||
|
texture: 0,
|
||||||
|
traits: ['space', 'camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
count: { type: 'float', range: [3, 40], default: 14, uniform: 'u_count', bias: 'density' },
|
||||||
|
thickness: { type: 'float', range: [0.002, 0.03], default: 0.008, uniform: 'u_thickness' },
|
||||||
|
bend: { type: 'float', range: [0, 1.2], default: 0.35, uniform: 'u_bend' },
|
||||||
|
speed: { type: 'float', range: [0.02, 0.4], default: 0.12, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
spread: { type: 'float', range: [0.2, 1.4], default: 0.9, uniform: 'u_spread' },
|
||||||
|
glow: { type: 'float', range: [0, 1], default: 0.3, uniform: 'u_glow', bias: 'energy' },
|
||||||
|
palette: { type: 'palette', count: 4 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
bend: { feature: 'bandLow', amount: 0.35, response: 'smooth' },
|
||||||
|
glow: { feature: 'beat', amount: 0.3, response: 'spike' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
// The bundle sits on the track's horizon rather than in the middle of the
|
||||||
|
// frame — this scene is nothing but a horizon, so it should be the shared one.
|
||||||
|
p.y -= sigHorizonY() * 0.6;
|
||||||
|
|
||||||
|
// Displace vertically by a slow wave, strongest at the centre of the frame.
|
||||||
|
float envelope = exp(-p.x * p.x * 1.2);
|
||||||
|
float offset = sin(p.x * 2.2 + t * 2.0) * u_bend * envelope;
|
||||||
|
|
||||||
|
vec3 col = pal(0) * 0.06;
|
||||||
|
float total = 0.0;
|
||||||
|
|
||||||
|
for (int i = 0; i < 40; i++) {
|
||||||
|
if (float(i) >= u_count) break;
|
||||||
|
float fi = float(i);
|
||||||
|
float slot = (fi / max(u_count - 1.0, 1.0) - 0.5) * 2.0 * u_spread;
|
||||||
|
|
||||||
|
float y = slot + offset * (0.4 + fract(fi * 0.37));
|
||||||
|
float d = abs(p.y - y);
|
||||||
|
|
||||||
|
float line = smoothstep(u_thickness * (0.5 + u_sigLine), 0.0, d);
|
||||||
|
float halo = exp(-d * 26.0) * u_glow;
|
||||||
|
|
||||||
|
vec3 c = pal(i);
|
||||||
|
col += c * (line + halo * 0.55);
|
||||||
|
total += line;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Keep the far edges dark so the lines read as a subject, not wallpaper.
|
||||||
|
col *= 0.55 + 0.45 * exp(-dot(p, p) * 0.5);
|
||||||
|
col = sigAir(col, p, smoothstep(0.0, 1.6, abs(p.y)));
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default horizonLines;
|
||||||
81
flow-state/src/scenes/shader/ink-bleed.js
Normal file
81
flow-state/src/scenes/shader/ink-bleed.js
Normal file
@ -0,0 +1,81 @@
|
|||||||
|
// Organic family: ink dropped into wet paper, spreading along the fibre.
|
||||||
|
//
|
||||||
|
// The bleed is done with feedback — each frame the previous one is sampled
|
||||||
|
// slightly outward along a noise-warped direction and darkened, which is a
|
||||||
|
// diffusion step in everything but name. Distinct from Curl Flow's trails
|
||||||
|
// (advected along a flow field, so they streak) because this expands in all
|
||||||
|
// directions at once, so it blooms.
|
||||||
|
//
|
||||||
|
// A base field is always drawn, so the scene stands alone before feedback has
|
||||||
|
// converged and survives a seek.
|
||||||
|
|
||||||
|
export const inkBleed = {
|
||||||
|
name: 'Ink Bleed',
|
||||||
|
family: 'organic',
|
||||||
|
kind: 'fragment',
|
||||||
|
traits: ['camera', 'space', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
drops: { type: 'int', range: [1, 6], default: 3, uniform: 'u_drops', bias: 'density' },
|
||||||
|
spread: { type: 'float', range: [0.002, 0.02], default: 0.007, uniform: 'u_spread' },
|
||||||
|
fibre: { type: 'float', range: [0.5, 8], default: 3.0, uniform: 'u_fibre', bias: 'density' },
|
||||||
|
soak: { type: 'float', range: [0.7, 0.99], default: 0.93, uniform: 'u_soak' },
|
||||||
|
density: { type: 'float', range: [0.1, 1.2], default: 0.5, uniform: 'u_density', bias: 'energy' },
|
||||||
|
pace: { type: 'float', range: [0.02, 0.4], default: 0.1, uniform: 'u_pace', bias: 'motion', rate: true },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
density: { feature: 'bandLow', amount: 0.35, response: 'smooth' },
|
||||||
|
fibre: { feature: 'bandAir', amount: 0.2 },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_pace + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
// Paper: a still fibre texture that the ink will follow.
|
||||||
|
float grainField = fbm(p * u_fibre * 2.0 + 17.0, 4);
|
||||||
|
vec3 col = mix(pal(0) * 0.09, pal(1) * 0.14, grainField);
|
||||||
|
|
||||||
|
// Fresh ink. Each drop pulses in and out on its own slow cycle, so the page
|
||||||
|
// is never uniformly saturated and there is always somewhere new bleeding.
|
||||||
|
float ink = 0.0;
|
||||||
|
for (int i = 0; i < 6; i++) {
|
||||||
|
if (i >= u_drops) break;
|
||||||
|
float fi = float(i);
|
||||||
|
float s = u_seed + fi * 61.3;
|
||||||
|
|
||||||
|
vec2 at = vec2(sin(t * 0.6 + s) * 0.6, cos(t * 0.47 + s * 1.7) * 0.45);
|
||||||
|
float life = 0.5 + 0.5 * sin(t * 1.3 + s * 2.1);
|
||||||
|
float d = length(p - at) + (grainField - 0.5) * 0.15;
|
||||||
|
ink += exp(-d * d * 90.0) * life;
|
||||||
|
}
|
||||||
|
ink = sat(ink) * u_density;
|
||||||
|
|
||||||
|
vec3 inkColor = palRamp(0.45 + grainField * 0.3);
|
||||||
|
col = mix(col, inkColor, ink);
|
||||||
|
|
||||||
|
// The bleed: sample the previous frame outward along the fibre. Reading four
|
||||||
|
// offsets rather than one is what makes it spread in every direction instead
|
||||||
|
// of sliding — one sample is a smear, four is diffusion.
|
||||||
|
vec2 warp = (vec2(fbm(p * u_fibre + 3.0, 3), fbm(p * u_fibre - 7.0, 3)) - 0.5) * 2.0;
|
||||||
|
float r = u_spread;
|
||||||
|
vec3 soaked = (
|
||||||
|
prev(uv + (vec2( 1.0, 0.0) + warp * 0.6) * r) +
|
||||||
|
prev(uv + (vec2(-1.0, 0.0) + warp * 0.6) * r) +
|
||||||
|
prev(uv + (vec2( 0.0, 1.0) + warp * 0.6) * r) +
|
||||||
|
prev(uv + (vec2( 0.0, -1.0) + warp * 0.6) * r)
|
||||||
|
) * 0.25;
|
||||||
|
|
||||||
|
col = max(col, soaked * u_soak);
|
||||||
|
|
||||||
|
col = sigAir(col, p, smoothstep(0.0, 1.7, length(p)));
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default inkBleed;
|
||||||
68
flow-state/src/scenes/shader/kaleido-tunnel.js
Normal file
68
flow-state/src/scenes/shader/kaleido-tunnel.js
Normal file
@ -0,0 +1,68 @@
|
|||||||
|
// Geometric family: a kaleidoscopic tunnel. The default drop scene — strong
|
||||||
|
// forward motion, hard symmetry, and it takes the beat well.
|
||||||
|
|
||||||
|
export const kaleidoTunnel = {
|
||||||
|
name: 'Kaleido Tunnel',
|
||||||
|
family: 'geometric',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
// Grain is this scene's ONLY expression of the style trait — it draws no
|
||||||
|
// hard edges to weight — so it keeps a share of it rather than opting out.
|
||||||
|
texture: 0.35,
|
||||||
|
traits: ['shape', 'camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
sides: { type: 'int', range: [2, 12], default: 6, uniform: 'u_sides' },
|
||||||
|
depth: { type: 'float', range: [1, 8], default: 3.0, uniform: 'u_depth', bias: 'density' },
|
||||||
|
speed: { type: 'float', range: [0.1, 1.5], default: 0.45, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
twist: { type: 'float', range: [0, 2], default: 0.5, uniform: 'u_twist' },
|
||||||
|
rings: { type: 'float', range: [2, 24], default: 8, uniform: 'u_rings', bias: 'density' },
|
||||||
|
glow: { type: 'float', range: [0, 1.5], default: 0.5, uniform: 'u_glow', bias: 'energy' },
|
||||||
|
palette: { type: 'palette', count: 6 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
glow: { feature: 'beat', amount: 0.45, response: 'spike' },
|
||||||
|
twist: { feature: 'bandLow', amount: 0.3 },
|
||||||
|
rings: { feature: 'bandHigh', amount: 0.2 },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
// A track with a signature polygon dictates the fold count — this is the
|
||||||
|
// scene where that reads most strongly, so its own sides yields to it.
|
||||||
|
float sides = u_sigSides > 2.5 ? u_sigSides : float(u_sides);
|
||||||
|
|
||||||
|
// Cross-section measured in the signature form: the tunnel mouth is the
|
||||||
|
// track's shape rather than a circle.
|
||||||
|
float radius = max(sigShape(p) + 1.0, 1e-4);
|
||||||
|
vec2 folded = kaleido(p, sides);
|
||||||
|
float angle = atan(folded.y, folded.x);
|
||||||
|
|
||||||
|
// Tunnel coordinates: 1/r is depth, angle is the wall.
|
||||||
|
float z = u_depth / radius + t * 2.0;
|
||||||
|
float wall = angle / 3.14159265 + sin(z * 0.5 + t) * u_twist * 0.25;
|
||||||
|
|
||||||
|
float ringLines = abs(fract(z * u_rings * 0.1) - 0.5) * 2.0;
|
||||||
|
float wallLines = abs(fract(wall * sides) - 0.5) * 2.0;
|
||||||
|
|
||||||
|
float grid = smoothstep(0.42, 0.0, ringLines) + smoothstep(0.42, 0.0, wallLines);
|
||||||
|
|
||||||
|
vec3 col = palRamp(z * 0.05 + wall * 0.2) * 0.35;
|
||||||
|
col += pal(int(mod(floor(z * u_rings * 0.1), 6.0))) * grid * 0.7;
|
||||||
|
|
||||||
|
// Depth cue: far end of the tunnel darkens, mouth glows.
|
||||||
|
float fade = smoothstep(0.0, 1.1, radius);
|
||||||
|
col *= 0.25 + 0.9 * fade;
|
||||||
|
col += pal(3) * (1.0 - fade) * u_glow * 0.6;
|
||||||
|
col += sigGrain(uv);
|
||||||
|
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default kaleidoTunnel;
|
||||||
72
flow-state/src/scenes/shader/metaballs.js
Normal file
72
flow-state/src/scenes/shader/metaballs.js
Normal file
@ -0,0 +1,72 @@
|
|||||||
|
// Organic family: merging metaballs on an analytic orbit.
|
||||||
|
//
|
||||||
|
// Positions come from closed-form orbits rather than any simulation, which keeps
|
||||||
|
// the scene seek-exact — the same rule the 3D particle layer follows.
|
||||||
|
|
||||||
|
export const metaballs = {
|
||||||
|
name: 'Metaballs',
|
||||||
|
family: 'organic',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
traits: ['shape', 'camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
count: { type: 'int', range: [2, 10], default: 5, uniform: 'u_count', bias: 'density' },
|
||||||
|
radius: { type: 'float', range: [0.1, 0.6], default: 0.3, uniform: 'u_radius', bias: 'energy' },
|
||||||
|
threshold: { type: 'float', range: [0.4, 2.2], default: 1.0, uniform: 'u_threshold' },
|
||||||
|
speed: { type: 'float', range: [0.03, 0.5], default: 0.15, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
spread: { type: 'float', range: [0.2, 1.1], default: 0.6, uniform: 'u_spread' },
|
||||||
|
rim: { type: 'float', range: [0, 1], default: 0.45, uniform: 'u_rim' },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
radius: { feature: 'bandLow', amount: 0.25 },
|
||||||
|
rim: { feature: 'beat', amount: 0.3, response: 'spike' },
|
||||||
|
threshold: { feature: 'flux', amount: 0.2, response: 'inverse' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
float field = 0.0;
|
||||||
|
vec3 tint = vec3(0.0);
|
||||||
|
|
||||||
|
for (int i = 0; i < 10; i++) {
|
||||||
|
if (i >= u_count) break;
|
||||||
|
float fi = float(i);
|
||||||
|
float s = u_seed + fi * 71.3;
|
||||||
|
|
||||||
|
vec2 centre = vec2(
|
||||||
|
sin(t * (0.7 + fract(s * 0.13)) + s) * u_spread,
|
||||||
|
cos(t * (0.5 + fract(s * 0.29)) + s * 1.7) * u_spread * 0.62
|
||||||
|
);
|
||||||
|
|
||||||
|
// Distance measured in the track's form rather than as a circle: for a
|
||||||
|
// round personality this is exactly length(p - centre), and for a
|
||||||
|
// hexagonal one the blobs merge as hexagons.
|
||||||
|
float d = sigShape((p - centre) / max(u_radius, 1e-3)) * u_radius + u_radius;
|
||||||
|
float contribution = (u_radius * u_radius) / max(d * d, 1e-4);
|
||||||
|
field += contribution;
|
||||||
|
tint += pal(i) * contribution;
|
||||||
|
}
|
||||||
|
|
||||||
|
tint /= max(field, 1e-4);
|
||||||
|
|
||||||
|
float surface = smoothstep(u_threshold - 0.25, u_threshold + 0.25, field);
|
||||||
|
float rim = smoothstep(u_threshold + 0.35, u_threshold, field)
|
||||||
|
* smoothstep(u_threshold - 0.3, u_threshold, field);
|
||||||
|
|
||||||
|
vec3 col = pal(0) * 0.06;
|
||||||
|
col = mix(col, tint, surface);
|
||||||
|
col += tint * rim * u_rim;
|
||||||
|
col += sigGrain(uv);
|
||||||
|
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default metaballs;
|
||||||
75
flow-state/src/scenes/shader/moire-grid.js
Normal file
75
flow-state/src/scenes/shader/moire-grid.js
Normal file
@ -0,0 +1,75 @@
|
|||||||
|
// Geometric family: two rotating line grids interfering.
|
||||||
|
//
|
||||||
|
// Moiré is a spatial-aliasing effect by nature, so this scene is the most likely
|
||||||
|
// in the library to alias badly. Line width is held above a floor and scaled by
|
||||||
|
// u_pixelScale, which is what keeps a 720p preview and a 4K export looking the
|
||||||
|
// same rather than the preview shimmering.
|
||||||
|
|
||||||
|
export const moireGrid = {
|
||||||
|
name: 'Moiré Grid',
|
||||||
|
family: 'geometric',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
// Crisp line work: the track's surface grain would only fur the edges.
|
||||||
|
texture: 0,
|
||||||
|
traits: ['camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
density: { type: 'float', range: [6, 60], default: 22, uniform: 'u_density', bias: 'density' },
|
||||||
|
offset: { type: 'float', range: [0.0, 0.5], default: 0.08, uniform: 'u_offset' },
|
||||||
|
rotate: { type: 'float', range: [0, 0.25], default: 0.04, uniform: 'u_rotate', bias: 'motion', rate: true },
|
||||||
|
// Named u_lineWidth, not u_width: the shader contract already declares
|
||||||
|
// `uniform float u_width` for stereo width, and a colliding name is a
|
||||||
|
// redefinition error that renders the scene as a black frame.
|
||||||
|
width: { type: 'float', range: [0.06, 0.5], default: 0.2, uniform: 'u_lineWidth' , slowAxis: true },
|
||||||
|
warp: { type: 'float', range: [0, 1], default: 0.25, uniform: 'u_warp' },
|
||||||
|
glow: { type: 'float', range: [0, 1.2], default: 0.35, uniform: 'u_glow', bias: 'energy' },
|
||||||
|
palette: { type: 'palette', count: 4 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
offset: { feature: 'bandLow', amount: 0.35 },
|
||||||
|
glow: { feature: 'beat', amount: 0.35, response: 'spike' },
|
||||||
|
warp: { feature: 'flux', amount: 0.2, response: 'smooth' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
// Anti-aliased line grid: the smoothstep edge is widened by the screen-space
|
||||||
|
// derivative, so lines stay a consistent visual weight at any resolution.
|
||||||
|
float grid(vec2 q, float density, float width) {
|
||||||
|
vec2 g = q * density;
|
||||||
|
vec2 f = abs(fract(g) - 0.5);
|
||||||
|
float d = min(f.x, f.y);
|
||||||
|
float aa = max(fwidth(d), 0.001);
|
||||||
|
return smoothstep(width * 0.5 + aa, width * 0.5 - aa, d);
|
||||||
|
}
|
||||||
|
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_rotate + u_seed;
|
||||||
|
p = sigFolded(sigCamera(p));
|
||||||
|
|
||||||
|
vec2 warp = vec2(fbm(p * 1.5 + t, 3), fbm(p * 1.5 - t, 3)) - 0.5;
|
||||||
|
vec2 q = p + warp * u_warp;
|
||||||
|
|
||||||
|
// Drawn in the track's hand: its line weight scales this scene's.
|
||||||
|
float weight = u_lineWidth * (0.5 + u_sigLine);
|
||||||
|
float a = grid(rot(t * 6.28318530718) * q, u_density, weight);
|
||||||
|
float b = grid(rot(-t * 6.28318530718 + u_offset * 3.14159) * (q + u_offset), u_density, weight);
|
||||||
|
|
||||||
|
// The interference term is the point: where both grids land, it peaks.
|
||||||
|
float interference = a * b;
|
||||||
|
float either = max(a, b);
|
||||||
|
|
||||||
|
vec3 col = pal(0) * 0.05;
|
||||||
|
col += pal(1) * either * 0.35;
|
||||||
|
col += pal(2) * interference * (0.8 + u_glow);
|
||||||
|
col += pal(3) * pow(interference, 3.0) * u_glow;
|
||||||
|
|
||||||
|
col *= 0.6 + 0.4 * exp(-dot(p, p) * 0.3);
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default moireGrid;
|
||||||
67
flow-state/src/scenes/shader/nebula.js
Normal file
67
flow-state/src/scenes/shader/nebula.js
Normal file
@ -0,0 +1,67 @@
|
|||||||
|
// Ported from party-stage's "Deep Nebula". Changes on port:
|
||||||
|
// - LED-grid mask removed (it existed to sell a screen inside a 3D room)
|
||||||
|
// - hardcoded vec3 colours replaced with palette lookups
|
||||||
|
// - magic numbers lifted into declared params
|
||||||
|
// - beat reactivity moved from a single u_beat multiply to the reactive block
|
||||||
|
|
||||||
|
export const nebula = {
|
||||||
|
name: 'Deep Nebula',
|
||||||
|
family: 'organic',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
traits: ['camera', 'space', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
scale: { type: 'float', range: [4, 24], default: 12, uniform: 'u_scale', bias: 'density' },
|
||||||
|
swirl: { type: 'float', range: [0, 3], default: 1.0, uniform: 'u_swirl', bias: 'motion' },
|
||||||
|
rings: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_rings' },
|
||||||
|
glow: { type: 'float', range: [0, 1.5], default: 0.5, uniform: 'u_glow', bias: 'energy' },
|
||||||
|
speed: { type: 'float', range: [0.1, 1.5],default: 0.5, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
depth: { type: 'float', range: [0, 1], default: 0.4, uniform: 'u_depth' },
|
||||||
|
palette: { type: 'palette', count: 4 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
glow: { feature: 'beat', amount: 0.35, response: 'spike' },
|
||||||
|
swirl: { feature: 'bandLow', amount: 0.20 },
|
||||||
|
rings: { feature: 'flux', amount: 0.25, response: 'smooth' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
// The track's camera films this scene too.
|
||||||
|
p = sigCamera(p);
|
||||||
|
float r = length(p);
|
||||||
|
float a = atan(p.y, p.x);
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
|
||||||
|
// Layered drift, the "nebula" body.
|
||||||
|
float n = fbm(p * u_scale * 0.25 + vec2(t * 0.2, -t * 0.15), 5);
|
||||||
|
float n2 = fbm(p * u_scale * 0.6 - vec2(t * 0.1, t * 0.25) + n, 4);
|
||||||
|
float body = sin(r * u_scale - t * 2.0 + n * 6.0 * u_swirl) * 0.5 + 0.5;
|
||||||
|
body = mix(body, n2, u_depth);
|
||||||
|
|
||||||
|
vec3 col = mix(pal(0), pal(1), sat(body));
|
||||||
|
col = mix(col, pal(2), sat(n2 * n2) * u_depth);
|
||||||
|
|
||||||
|
// Concentric pulse, tied to onset energy rather than a fixed rate.
|
||||||
|
float ring = smoothstep(0.4, 0.5, abs(fract(r * 2.0 - t * 3.0) - 0.5));
|
||||||
|
col += pal(1) * ring * u_rings * 0.6;
|
||||||
|
|
||||||
|
// Core glow.
|
||||||
|
float edge = 1.0 - smoothstep(0.1, 0.9, r);
|
||||||
|
col += pal(3) * edge * u_glow * 0.5;
|
||||||
|
|
||||||
|
// Vignette the far field so the frame has a subject.
|
||||||
|
col *= 0.5 + 0.5 * (1.0 - smoothstep(0.6, 1.6, r));
|
||||||
|
|
||||||
|
// The location's air, and its surface.
|
||||||
|
col = sigAir(col, p, smoothstep(0.0, 1.6, r));
|
||||||
|
col += sigGrain(uv);
|
||||||
|
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default nebula;
|
||||||
123
flow-state/src/scenes/shader/neon-city.js
Normal file
123
flow-state/src/scenes/shader/neon-city.js
Normal file
@ -0,0 +1,123 @@
|
|||||||
|
// Structural family: a receding skyline of towers with instanced lit windows.
|
||||||
|
//
|
||||||
|
// Distinct from Ridge Terrain (rolling fbm ridgelines) and Synthwave Run (a
|
||||||
|
// ground grid): this is architecture, read as a front-on skyline. Heights and
|
||||||
|
// footprints are per-tower noise, drawn far-to-near so a nearer tower correctly
|
||||||
|
// occludes the one behind while a taller far tower still rises above a short
|
||||||
|
// one in front. Windows are lit per facade cell, so the city has interior life
|
||||||
|
// rather than reading as a lit cardboard cut-out.
|
||||||
|
|
||||||
|
export const neonCity = {
|
||||||
|
name: 'Neon City',
|
||||||
|
family: 'structural',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
traits: ['space', 'camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
cells: { type: 'float', range: [0.5, 4], default: 2.0, uniform: 'u_cells', bias: 'density' },
|
||||||
|
layers: { type: 'int', range: [2, 8], default: 6, uniform: 'u_layers', bias: 'density' },
|
||||||
|
height: { type: 'float', range: [0.2, 2.5], default: 1.0, uniform: 'u_height', bias: 'energy' },
|
||||||
|
speed: { type: 'float', range: [0.02, 0.4], default: 0.08, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
horizon: { type: 'float', range: [-0.3, 0.3], default: 0.0, uniform: 'u_horizon' },
|
||||||
|
haze: { type: 'float', range: [0, 1], default: 0.45, uniform: 'u_haze' },
|
||||||
|
reflect: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_reflect' },
|
||||||
|
pulse: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_pulse' },
|
||||||
|
glow: { type: 'float', range: [0, 1.5], default: 0.4, uniform: 'u_glow', bias: 'energy' },
|
||||||
|
palette: { type: 'palette', count: 4 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
height: { feature: 'bandLow', amount: 0.18, response: 'smooth' },
|
||||||
|
pulse: { feature: 'beat', amount: 0.2, response: 'smooth' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
float towerEdge(vec2 grid, float mullions) {
|
||||||
|
vec2 g = fract(grid);
|
||||||
|
float steel = min(1.0 - smoothstep(0.0, 0.28 * mullions, g.x),
|
||||||
|
1.0 - smoothstep(0.0, 0.28 * mullions, g.y));
|
||||||
|
return steel;
|
||||||
|
}
|
||||||
|
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
float beat = u_beat;
|
||||||
|
p = sigCamera(p);
|
||||||
|
// Shares the track's ground line with every other scene that has one.
|
||||||
|
float horizon = clamp(u_horizon + sigHorizonY() * 0.3, -0.9, 0.9);
|
||||||
|
|
||||||
|
// Sky, darkening away from the horizon glow.
|
||||||
|
vec3 sky = mix(pal(3) * 0.06, pal(0) * 0.35,
|
||||||
|
smoothstep(-0.1, 0.7, p.y - horizon));
|
||||||
|
sky *= 0.4 + 0.6 * exp(-abs(p.y - horizon) * 3.0);
|
||||||
|
|
||||||
|
vec3 col = sky;
|
||||||
|
|
||||||
|
for (int i = 0; i < 8; i++) {
|
||||||
|
if (i >= u_layers) break;
|
||||||
|
float fi = float(i);
|
||||||
|
float depth = float(i) / max(float(u_layers - 1), 1.0); // 0 far .. 1 near
|
||||||
|
float parallax = mix(0.25, 1.0, depth);
|
||||||
|
|
||||||
|
// Horizontal column coordinate for this depth ring. Receding layers
|
||||||
|
// get more cells per screen and slide slower, giving parallax.
|
||||||
|
float xsc = mix(1.2, 3.4, depth) * u_cells;
|
||||||
|
float xc = p.x * xsc + t * 0.18 * parallax + fi * 31.7;
|
||||||
|
float cx = floor(xc);
|
||||||
|
float xf = fract(xc);
|
||||||
|
|
||||||
|
float h1 = hash11(cx * 13.1 + fi * 7.3 + u_seed);
|
||||||
|
float h2 = hash11(cx * 61.7 + fi * 5.9 + u_seed * 0.7);
|
||||||
|
|
||||||
|
// Footprint half width, and the tower's top above its base.
|
||||||
|
float footprint = (0.55 + h1 * 0.35) * 0.5; // in cell fracs
|
||||||
|
float inTower = smoothstep(footprint + 0.04, footprint - 0.04,
|
||||||
|
abs(xf - 0.5));
|
||||||
|
|
||||||
|
float baseY = horizon - (0.05 + depth * 0.9);
|
||||||
|
float topY = baseY + u_height * (0.35 + h1 * 1.6) * mix(0.45, 1.0, depth);
|
||||||
|
float body = step(p.y, topY) * step(baseY, p.y);
|
||||||
|
float tower = inTower * body;
|
||||||
|
if (tower < 0.01) continue;
|
||||||
|
|
||||||
|
// Facade, tinted by how far into the screen the ring sits.
|
||||||
|
float gy = clamp((p.y - baseY) / max(topY - baseY, 1e-4), 0.0, 1.0);
|
||||||
|
vec3 facade = mix(pal(1), pal(3), depth);
|
||||||
|
|
||||||
|
// Instanced window grid — each facade cell decides itself lit or dark,
|
||||||
|
// with the pulse raising the lit population and the beat underlining it.
|
||||||
|
vec2 g = vec2(xf * 5.0, gy * 9.0);
|
||||||
|
vec2 cellg = floor(g);
|
||||||
|
float lit = step(0.42, hash12(cellg + fi * 3.1));
|
||||||
|
float steel = 1.0 - min(smoothstep(0.0, 0.35, fract(g.x)),
|
||||||
|
smoothstep(0.0, 0.35, fract(g.y)));
|
||||||
|
vec3 windowCol = mix(pal(2), pal(3), 0.3);
|
||||||
|
vec3 fut = mix(col, windowCol,
|
||||||
|
lit * steel * (0.45 + 0.55 * u_pulse + beat * 0.25));
|
||||||
|
|
||||||
|
// Ground-couple: darker, breath where the tower meets the street.
|
||||||
|
fut *= 0.7 + 0.3 * smoothstep(0.0, 0.5, gy);
|
||||||
|
// Haze pushes the far rings into the sky, the way distance actually does.
|
||||||
|
col = mix(col, fut, tower * (1.0 - u_haze * (1.0 - depth)));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Street: a slate floor with a molten reflection of the nearest towers.
|
||||||
|
if (p.y < horizon) {
|
||||||
|
float street = smoothstep(0.0, -0.6, p.y - horizon);
|
||||||
|
col = mix(col, pal(3) * 0.08, street);
|
||||||
|
float refl = exp(-abs(p.y - horizon) * 4.0) * u_reflect;
|
||||||
|
col += pal(2) * refl * (0.08 + 0.2 * beat) * street;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Horizon haze bloom, the city's glow pooling where towers meet the sky.
|
||||||
|
col += pal(2) * exp(-abs(p.y - horizon) * 9.0) * u_glow;
|
||||||
|
col = sigAir(col, p, smoothstep(0.0, 1.4, length(p)));
|
||||||
|
col += sigGrain(uv);
|
||||||
|
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default neonCity;
|
||||||
71
flow-state/src/scenes/shader/pitch-shatter.js
Normal file
71
flow-state/src/scenes/shader/pitch-shatter.js
Normal file
@ -0,0 +1,71 @@
|
|||||||
|
// Glitch family: vertical transposition. The frame is cut into vertical bands,
|
||||||
|
// each of which shifts up or down by a small amount. That alone is a tear; what
|
||||||
|
// makes it churn is that the offset is re-rolled on the quantised grid and the
|
||||||
|
// previous frame's transposed image is smeared underneath — so slices translate
|
||||||
|
// to a new fixed pose each step instead of crawling, and the pitch reads as an
|
||||||
|
// edit rather than a broken renderer.
|
||||||
|
|
||||||
|
export const pitchShatter = {
|
||||||
|
name: 'Pitch Shatter',
|
||||||
|
family: 'glitch',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
traits: ['camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
slices: { type: 'float', range: [4, 40], default: 18, uniform: 'u_slices', bias: 'density' },
|
||||||
|
amp: { type: 'float', range: [0, 0.35], default: 0.1, uniform: 'u_amp', bias: 'energy' },
|
||||||
|
quantize: { type: 'float', range: [1, 8], default: 4, uniform: 'u_quantize' },
|
||||||
|
bleed: { type: 'float', range: [0, 0.9], default: 0.4, uniform: 'u_bleed' },
|
||||||
|
glow: { type: 'float', range: [0, 1], default: 0.3, uniform: 'u_glow', bias: 'energy' },
|
||||||
|
speed: { type: 'float', range: [0.05, 0.8], default: 0.25, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
amp: { feature: 'beat', amount: 0.4, response: 'spike' },
|
||||||
|
bleed: { feature: 'flux', amount: 0.25, response: 'smooth' },
|
||||||
|
glow: { feature: 'bandHigh', amount: 0.2 },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
// One shared grid for every discontinuity below: slices re-roll on the
|
||||||
|
// quantised step rather than moving continuously frame-to-frame.
|
||||||
|
float q = max(u_quantize, 1.0);
|
||||||
|
float step_ = floor(u_barPhase * q) + floor(t * 6.0) * q;
|
||||||
|
|
||||||
|
// A vertical slice is a column of pixels; each gets one fixed offset per
|
||||||
|
// step, decided by a hash of (slice, step) so it jumps cleanly on the grid.
|
||||||
|
float slice = floor(uv.x * u_slices);
|
||||||
|
float sliceRand = hash12(vec2(slice, step_ * 31.0));
|
||||||
|
float pitch = (sliceRand - 0.5) * 2.0 * u_amp;
|
||||||
|
|
||||||
|
// The base image is always sampled from the un-pitched field so the scene
|
||||||
|
// has real content behind the displacement, even on its first frame.
|
||||||
|
vec2 baseUv = vec2(uv.x, fract(uv.y - pitch));
|
||||||
|
float field = fbm(vec2(baseUv.x * 2.5, baseUv.y * 4.0) + t * 0.4, 4);
|
||||||
|
float ramp = fract(field * 2.0 + baseUv.y * 2.0 - t * 0.4);
|
||||||
|
vec3 col = palRamp(ramp * 0.7 + slice * 0.02);
|
||||||
|
col *= 0.4 + 0.6 * smoothstep(0.1, 0.9, field);
|
||||||
|
|
||||||
|
// The smear: pull the previous frame across the pitch so slices leave a
|
||||||
|
// transient ghost as they land, and the corruption feels sludgy.
|
||||||
|
vec3 ghost = prev(vec2(uv.x, fract(uv.y + pitch * 0.55)));
|
||||||
|
col = mix(col, ghost, u_bleed * (0.4 + sliceRand));
|
||||||
|
|
||||||
|
// A thin scan beam rooting the hard edge of each step, kept local and beat
|
||||||
|
// -pinned so the frame never swings in whole-frame luminance.
|
||||||
|
float beamY = fract(u_beatPhase) * 2.0 - 1.0;
|
||||||
|
float beam = exp(-abs(p.y - beamY * 0.6) * 6.0);
|
||||||
|
col += pal(3) * beam * u_slices * 0.001 * (0.5 + u_beat);
|
||||||
|
|
||||||
|
col += pal(int(mod(slice, 5.0))) * (1.0 - sliceRand) * u_glow * 0.3 * sliceRand;
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
56
flow-state/src/scenes/shader/plasma-bloom.js
Normal file
56
flow-state/src/scenes/shader/plasma-bloom.js
Normal file
@ -0,0 +1,56 @@
|
|||||||
|
// Organic family: domain-warped plasma. The workhorse sustain scene — it holds
|
||||||
|
// up for minutes because the warp keeps folding new structure into itself rather
|
||||||
|
// than cycling.
|
||||||
|
|
||||||
|
export const plasmaBloom = {
|
||||||
|
name: 'Plasma Bloom',
|
||||||
|
family: 'organic',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
traits: ['camera', 'space', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
scale: { type: 'float', range: [0.8, 6], default: 2.4, uniform: 'u_scale', bias: 'density' },
|
||||||
|
warp: { type: 'float', range: [0, 3], default: 1.2, uniform: 'u_warp' },
|
||||||
|
speed: { type: 'float', range: [0.02, 0.5], default: 0.1, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
bands: { type: 'float', range: [1, 10], default: 3.5, uniform: 'u_bands' },
|
||||||
|
softness:{ type: 'float', range: [0, 1], default: 0.5, uniform: 'u_softness' },
|
||||||
|
glow: { type: 'float', range: [0, 1.5], default: 0.4, uniform: 'u_glow', bias: 'energy' },
|
||||||
|
palette: { type: 'palette', count: 6 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
warp: { feature: 'bandLow', amount: 0.35 },
|
||||||
|
glow: { feature: 'beat', amount: 0.35, response: 'spike' },
|
||||||
|
bands: { feature: 'centroid', amount: 0.2, response: 'smooth' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
// Two rounds of domain warping. One looks like noise; two looks organic.
|
||||||
|
vec2 q = vec2(fbm(p * u_scale + t, 4), fbm(p * u_scale + vec2(5.2, 1.3) - t, 4));
|
||||||
|
vec2 r = vec2(fbm(p * u_scale + q * u_warp * 2.0 + vec2(1.7, 9.2) + t * 0.6, 5),
|
||||||
|
fbm(p * u_scale + q * u_warp * 2.0 + vec2(8.3, 2.8) - t * 0.4, 5));
|
||||||
|
|
||||||
|
float v = fbm(p * u_scale + r * u_warp * 2.0, 5);
|
||||||
|
float shaped = sin(v * u_bands * 3.14159 + t * 1.5) * 0.5 + 0.5;
|
||||||
|
shaped = mix(shaped, smoothstep(0.25, 0.75, shaped), u_softness);
|
||||||
|
|
||||||
|
vec3 col = palRamp(shaped * 0.6 + length(r) * 0.25);
|
||||||
|
col *= 0.35 + 0.75 * shaped;
|
||||||
|
col += pal(4) * pow(shaped, 5.0) * u_glow;
|
||||||
|
|
||||||
|
// Dark corners so the bloom has somewhere to sit.
|
||||||
|
col *= 0.55 + 0.45 * exp(-dot(p, p) * 0.4);
|
||||||
|
|
||||||
|
col = sigAir(col, p, smoothstep(0.0, 1.8, length(p)));
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default plasmaBloom;
|
||||||
77
flow-state/src/scenes/shader/prism-bloom.js
Normal file
77
flow-state/src/scenes/shader/prism-bloom.js
Normal file
@ -0,0 +1,77 @@
|
|||||||
|
// Geometric family: a burst of rotating folded prisms. A core sits at the
|
||||||
|
// centre in the track's signature form, and around it a constellation of folded
|
||||||
|
// rings counter-rotate and breathe — prism facets read as hard geometry meeting
|
||||||
|
// a soft bloom. Everything is closed-form, so the whole lattice is seek-exact.
|
||||||
|
|
||||||
|
export const prismBloom = {
|
||||||
|
name: 'Prism Bloom',
|
||||||
|
family: 'geometric',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
// Crisp line work: the track's surface grain would only fur the edges.
|
||||||
|
texture: 0,
|
||||||
|
traits: ['shape', 'camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
sites: { type: 'float', range: [2, 12], default: 6, uniform: 'u_sites', bias: 'density' },
|
||||||
|
layers: { type: 'float', range: [1, 8], default: 5, uniform: 'u_layers', bias: 'density' },
|
||||||
|
thickness: { type: 'float', range: [0.008, 0.05], default: 0.02, uniform: 'u_thickness' },
|
||||||
|
radius: { type: 'float', range: [0.15, 0.7], default: 0.32, uniform: 'u_radius', bias: 'energy' },
|
||||||
|
spin: { type: 'float', range: [0, 1.5], default: 0.4, uniform: 'u_spin', bias: 'motion' },
|
||||||
|
glow: { type: 'float', range: [0, 1.4], default: 0.5, uniform: 'u_glow', bias: 'energy' },
|
||||||
|
speed: { type: 'float', range: [0.05, 0.8], default: 0.3, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
palette: { type: 'palette', count: 6 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
glow: { feature: 'beat', amount: 0.35, response: 'smooth' },
|
||||||
|
radius: { feature: 'bandLow', amount: 0.2 },
|
||||||
|
spin: { feature: 'flux', amount: 0.15, response: 'smooth' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
// A ring whose radius is folded into N star points, keeping the geometry angular
|
||||||
|
// rather than circular.
|
||||||
|
float ring(vec2 q, float sites, float radius, float width) {
|
||||||
|
float a = atan(q.y, q.x);
|
||||||
|
float seg = 6.28318530718 / sites;
|
||||||
|
float wa = abs(mod(a + seg * 0.5, seg) - seg * 0.5);
|
||||||
|
float starRadius = radius * max(cos(wa * sites), 1e-3);
|
||||||
|
float d = abs(length(q) - starRadius);
|
||||||
|
return smoothstep(width + 0.004, width - 0.004, d);
|
||||||
|
}
|
||||||
|
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
p = sigFolded(sigCamera(p));
|
||||||
|
|
||||||
|
// Facet count yields to the track's signature form when it names one.
|
||||||
|
float sites = u_sigSides > 2.5 ? u_sigSides : float(u_sites);
|
||||||
|
|
||||||
|
vec3 col = pal(0) * 0.04;
|
||||||
|
for (int i = 0; i < 8; i++) {
|
||||||
|
if (float(i) >= u_layers) break;
|
||||||
|
float fi = float(i);
|
||||||
|
|
||||||
|
// Layers counter-rotate and breathe independently; all closed-form.
|
||||||
|
float layerRadius = u_radius * (0.5 + fi * 0.24);
|
||||||
|
float breathe = 0.78 + 0.22 * sin(u_time * 0.9 + fi * 1.7);
|
||||||
|
vec2 lp = rot(fi * 0.7 + t * (fi * 0.5 + 0.3) * u_spin) * p;
|
||||||
|
|
||||||
|
float r = ring(lp, sites, layerRadius * breathe, u_thickness);
|
||||||
|
col += pal(int(mod(fi, 6.0))) * r * (0.4 + u_glow * 0.8);
|
||||||
|
|
||||||
|
float halo = exp(-(length(lp) / max(layerRadius * breathe, 1e-3)) * 3.0) * u_glow * 0.25;
|
||||||
|
col += pal(0) * halo;
|
||||||
|
}
|
||||||
|
|
||||||
|
// The heart of the bloom is the track's own form.
|
||||||
|
float core = sigForm(p, vec2(0.0), 0.09 + u_beat * 0.06);
|
||||||
|
col = mix(col, pal(3), core);
|
||||||
|
col += pal(3) * exp(-dot(p, p) * 8.0) * (0.5 + u_glow);
|
||||||
|
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
138
flow-state/src/scenes/shader/psychedelic-drift.js
Normal file
138
flow-state/src/scenes/shader/psychedelic-drift.js
Normal file
@ -0,0 +1,138 @@
|
|||||||
|
// Ported from party-stage's "Drugged Out Flow".
|
||||||
|
//
|
||||||
|
// The original ran its own 15-second internal scene timer to keep itself
|
||||||
|
// interesting. That job now belongs to the arc driver, which knows where the
|
||||||
|
// song's actual transitions are — so the internal timer is gone and the scene
|
||||||
|
// variety is driven by `variant`, a param the arc can step.
|
||||||
|
|
||||||
|
export const psychedelicDrift = {
|
||||||
|
name: 'Psychedelic Drift',
|
||||||
|
family: 'glitch',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
traits: ['shape', 'camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
count: { type: 'int', range: [4, 24], default: 12, uniform: 'u_count', bias: 'density' },
|
||||||
|
variant: { type: 'float', range: [0, 8], default: 0, uniform: 'u_variant' },
|
||||||
|
warp: { type: 'float', range: [0, 0.6], default: 0.2, uniform: 'u_warp', bias: 'energy' },
|
||||||
|
beams: { type: 'int', range: [0, 5], default: 3, uniform: 'u_beams' },
|
||||||
|
symbolSize: { type: 'float', range: [0.03, 0.14], default: 0.07, uniform: 'u_symbolSize' },
|
||||||
|
speed: { type: 'float', range: [0.05, 0.8], default: 0.2, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
palette: { type: 'palette', count: 6 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
warp: { feature: 'beat', amount: 0.35, response: 'spike' },
|
||||||
|
symbolSize: { feature: 'bandLow', amount: 0.25 },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
float sdCircle(vec2 p, float r) { return length(p) - r; }
|
||||||
|
|
||||||
|
float sdStar(vec2 p, float r, float points) {
|
||||||
|
float a = atan(p.y, p.x);
|
||||||
|
float seg = 6.28318530718 / points;
|
||||||
|
a = mod(a + seg * 0.5, seg) - seg * 0.5;
|
||||||
|
return length(p) * cos(a) - r * 0.5;
|
||||||
|
}
|
||||||
|
|
||||||
|
float sdSmiley(vec2 p, float r, float t, float id) {
|
||||||
|
float d = sdCircle(p, r);
|
||||||
|
float blink = smoothstep(0.9, 0.95, sin(t * 0.5 + id * 1.2));
|
||||||
|
float eyeR = 0.18 * r * (1.0 - blink * 0.9);
|
||||||
|
d = max(d, -sdCircle(vec2(abs(p.x) - 0.3 * r, p.y - 0.35 * r), eyeR));
|
||||||
|
|
||||||
|
float mouthType = sin(t * 1.2 + id);
|
||||||
|
if (mouthType > 0.6) {
|
||||||
|
d = max(d, -sdCircle(p - vec2(0.0, -0.2) * r, 0.15 * r));
|
||||||
|
} else if (mouthType > -0.6) {
|
||||||
|
float mouth = sdCircle(p - vec2(0.0, -0.1) * r, 0.45 * r);
|
||||||
|
mouth = max(mouth, -(p.y - (0.05 + 0.1 * u_beat) * r));
|
||||||
|
d = max(d, -mouth);
|
||||||
|
} else {
|
||||||
|
d = max(d, -sdCircle(p - vec2(0.0, -0.25) * r, 0.08 * r));
|
||||||
|
}
|
||||||
|
return d;
|
||||||
|
}
|
||||||
|
|
||||||
|
float distToSegment(vec2 p, vec2 a, vec2 b) {
|
||||||
|
vec2 pa = p - a, ba = b - a;
|
||||||
|
float h = clamp(dot(pa, ba) / dot(ba, ba), 0.0, 1.0);
|
||||||
|
return length(pa - ba * h);
|
||||||
|
}
|
||||||
|
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
float beat = u_beat;
|
||||||
|
float vs = u_seed + floor(u_variant) * 100.0;
|
||||||
|
|
||||||
|
// Swirling background.
|
||||||
|
p = sigCamera(p);
|
||||||
|
float bg = sin(p.x * 3.0 + t) * cos(p.y * 3.0 - t * 0.7) + sin(length(p) * 4.0 - t);
|
||||||
|
vec3 col = mix(pal(0) * 0.12, pal(1) * 0.18, bg * 0.5 + 0.5);
|
||||||
|
col += pal(0) * 0.05 * beat;
|
||||||
|
|
||||||
|
// Lens distortion that drags everything toward a wandering centre.
|
||||||
|
vec2 centre = vec2(sin(t * 0.3 + vs) * 0.5, cos(t * 0.4 + vs * 1.1) * 0.5);
|
||||||
|
float radius = 0.35 + beat * 0.15;
|
||||||
|
float dc = length(p - centre);
|
||||||
|
vec2 dp = p;
|
||||||
|
if (dc < radius && dc > 1e-4) {
|
||||||
|
dp += normalize(p - centre) * smoothstep(radius, 0.0, dc) * (u_warp + beat * 0.2);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Beam endpoints, precomputed so symbols can react to them.
|
||||||
|
vec2 b1[5], b2[5];
|
||||||
|
for (int j = 0; j < 5; j++) {
|
||||||
|
float ls = vs + float(j) * 789.0;
|
||||||
|
b1[j] = vec2(sin(t * 0.4 + ls) * 0.8, cos(t * 0.3 + ls * 1.1) * 0.5);
|
||||||
|
b2[j] = vec2(sin(t * 0.5 + ls * 1.5) * 0.8, cos(t * 0.4 + ls * 1.7) * 0.5);
|
||||||
|
}
|
||||||
|
|
||||||
|
for (int i = 0; i < 24; i++) {
|
||||||
|
if (i >= u_count) break;
|
||||||
|
float fi = float(i);
|
||||||
|
float s = vs + fi * 123.456;
|
||||||
|
|
||||||
|
float speed = 0.3 + hash11(s) * 0.4;
|
||||||
|
vec2 basePos = vec2(sin(t * speed + s) * 0.8, cos(t * speed * 0.8 + s * 1.4) * 0.5);
|
||||||
|
vec2 pos = mix(basePos, centre, 0.15 + beat * 0.1);
|
||||||
|
|
||||||
|
vec2 sp = rot(t * (0.1 + fract(s * 0.7))) * (dp - pos);
|
||||||
|
float size = u_symbolSize * (0.7 + fract(s * 0.3) * 0.6);
|
||||||
|
|
||||||
|
float d;
|
||||||
|
float kind = mod(fi + floor(u_variant), 3.0);
|
||||||
|
// One of the three symbol kinds is the track's own form, so its cast of
|
||||||
|
// characters includes the one every other scene is built from.
|
||||||
|
if (kind == 0.0) d = sdStar(sp, size, 5.0 + floor(hash11(s * 1.2) * 3.0));
|
||||||
|
else if (kind == 1.0) d = sdSmiley(sp, size, u_time, fi);
|
||||||
|
else d = sigShape(sp / max(size, 1e-3)) * size;
|
||||||
|
|
||||||
|
float hit = 0.0;
|
||||||
|
for (int j = 0; j < 5; j++) {
|
||||||
|
if (j >= u_beams) break;
|
||||||
|
hit = max(hit, smoothstep(0.15, 0.0, distToSegment(pos, b1[j], b2[j])));
|
||||||
|
}
|
||||||
|
|
||||||
|
vec3 sc = mix(pal(i + int(floor(t * 0.3))), vec3(1.0), hit * 0.8);
|
||||||
|
float intensity = smoothstep(0.012, 0.0, d);
|
||||||
|
col = mix(col, sc, intensity * sat(0.7 + beat * 0.3 + hit));
|
||||||
|
col += sc * (1.0 - smoothstep(0.0, size * 2.5, abs(d))) * 0.15 * (0.5 + beat * 0.5 + hit * 2.0);
|
||||||
|
}
|
||||||
|
|
||||||
|
for (int i = 0; i < 5; i++) {
|
||||||
|
if (i >= u_beams) break;
|
||||||
|
float w = 0.008 + beat * 0.005;
|
||||||
|
float intensity = smoothstep(w, 0.0, distToSegment(dp, b1[i], b2[i]));
|
||||||
|
col += pal(i + 2) * intensity * (0.6 + beat);
|
||||||
|
}
|
||||||
|
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default psychedelicDrift;
|
||||||
86
flow-state/src/scenes/shader/pylon-grid.js
Normal file
86
flow-state/src/scenes/shader/pylon-grid.js
Normal file
@ -0,0 +1,86 @@
|
|||||||
|
// Structural family: a field of standing pylons receding to the track's
|
||||||
|
// horizon. The columns converge on a vanishing point, giving real perspective
|
||||||
|
// depth, and each column carries a crown in the signature form so a hexagonal
|
||||||
|
// track grows hexagonal pylons. The beat walks a pulse of light down the rows
|
||||||
|
// rather than flashing the whole frame.
|
||||||
|
|
||||||
|
export const pylonGrid = {
|
||||||
|
name: 'Pylon Grid',
|
||||||
|
family: 'structural',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
// Takes the track's surface grain, but lightly — this is drawn, not filmed.
|
||||||
|
texture: 0.4,
|
||||||
|
traits: ['shape', 'space'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
columns: { type: 'float', range: [3, 16], default: 8, uniform: 'u_columns', bias: 'density' },
|
||||||
|
rows: { type: 'float', range: [2, 10], default: 6, uniform: 'u_rows', bias: 'density' },
|
||||||
|
spread: { type: 'float', range: [0.6, 1.6], default: 1.1, uniform: 'u_spread' },
|
||||||
|
height: { type: 'float', range: [0.3, 1.4], default: 0.85, uniform: 'u_height', bias: 'energy' },
|
||||||
|
pulse: { type: 'float', range: [0, 1.2], default: 0.5, uniform: 'u_pulse', bias: 'energy' },
|
||||||
|
speed: { type: 'float', range: [0.05, 0.8], default: 0.3, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
palette: { type: 'palette', count: 4 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
pulse: { feature: 'beat', amount: 0.4, response: 'smooth' },
|
||||||
|
rows: { feature: 'bandLow', amount: 0.25 },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
// Distance to a thin vertical strut from (cx, y0) up to (cx, y1).
|
||||||
|
float strut(vec2 p, float cx, float y0, float y1, float w) {
|
||||||
|
float xd = abs(p.x - cx);
|
||||||
|
float yd = max(max(y0 - p.y, p.y - y1), 0.0);
|
||||||
|
float d = sqrt(xd * xd + yd * yd);
|
||||||
|
return smoothstep(w + 0.004, w - 0.004, d);
|
||||||
|
}
|
||||||
|
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
float hy = sigHorizonY();
|
||||||
|
vec3 col = pal(0) * 0.05;
|
||||||
|
|
||||||
|
for (int r = 0; r < 10; r++) {
|
||||||
|
if (float(r) >= u_rows) break;
|
||||||
|
float fr = float(r);
|
||||||
|
|
||||||
|
// Perspective: the last row sits on the horizon, the first at the bottom
|
||||||
|
// of the frame, everything between scaling by its depth fraction.
|
||||||
|
float f = fr / max(u_rows - 1.0, 1.0); // 0 near .. 1 far
|
||||||
|
float y0 = mix(-1.08, hy + 0.06, f); // row baseline
|
||||||
|
float scale = mix(1.0, 0.12, f); // perspective shrink
|
||||||
|
float depthShade = mix(1.0, 0.35, f); // far rows dim
|
||||||
|
|
||||||
|
for (int c = 0; c < 16; c++) {
|
||||||
|
if (float(c) >= u_columns) break;
|
||||||
|
float fc = float(c);
|
||||||
|
float cx = (fc / max(u_columns - 1.0, 1.0) - 0.5) * 2.0 * u_spread * scale;
|
||||||
|
|
||||||
|
float h = u_height * scale;
|
||||||
|
float yTop = y0 + h;
|
||||||
|
|
||||||
|
// The beat pulse walks a glow down the depth axis, row by row, so the
|
||||||
|
// light travels rather than strobing the whole frame.
|
||||||
|
float walk = sat(1.0 - abs(fr - mod(u_beatPhase * 6.0 + t * 0.4, u_rows)));
|
||||||
|
float inten = (0.35 + 0.65 * walk) * depthShade * (0.7 + 0.3 * u_beat);
|
||||||
|
|
||||||
|
float w = 0.02 * scale + 0.006;
|
||||||
|
col += pal(int(mod(fc, 4.0))) * strut(p, cx, y0, yTop, w) * inten;
|
||||||
|
|
||||||
|
// Crown stamped in the track's signature form.
|
||||||
|
vec2 crown = vec2(cx, yTop);
|
||||||
|
float size = (0.06 + 0.2 * scale) * (1.0 + u_beat * 0.4);
|
||||||
|
col += pal(3) * sigForm(p, crown, size) * inten * (0.6 + u_pulse);
|
||||||
|
col += pal(int(mod(fr, 4.0))) * exp(-dot(p - crown, p - crown) * 60.0) * u_pulse * depthShade;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ground haze and the shared air between here and the horizon.
|
||||||
|
col = sigAir(col, p, smoothstep(0.0, 1.6, abs(p.y - hy)));
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
79
flow-state/src/scenes/shader/quasicrystal.js
Normal file
79
flow-state/src/scenes/shader/quasicrystal.js
Normal file
@ -0,0 +1,79 @@
|
|||||||
|
// Geometric family: the interference of several plane waves at irrational
|
||||||
|
// angles — a pattern with local symmetry that never actually repeats.
|
||||||
|
//
|
||||||
|
// Distinct from Moiré Grid (two periodic grids beating against each other),
|
||||||
|
// Truchet Fold (a tiling, so a cell structure you can see) and Prism Bloom
|
||||||
|
// (radial geometry around a centre): with five or seven waves the sum is
|
||||||
|
// quasiperiodic. There is no tile, no cell and no centre, and the eye keeps
|
||||||
|
// finding order that does not survive being looked at twice. That is a texture
|
||||||
|
// the library could not previously make.
|
||||||
|
//
|
||||||
|
// Scaffolded by tools/new-scene.js. See HOWTO-visualizers.md.
|
||||||
|
|
||||||
|
export const quasicrystal = {
|
||||||
|
name: 'Quasicrystal',
|
||||||
|
family: 'geometric',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Crisp line work: the track's surface grain would only fur the edges.
|
||||||
|
texture: 0,
|
||||||
|
traits: ['camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
waves: { type: 'int', range: [3, 11], default: 5, uniform: 'u_waves', bias: 'density' },
|
||||||
|
frequency: { type: 'float', range: [2, 26], default: 9, uniform: 'u_frequency', bias: 'density' },
|
||||||
|
speed: { type: 'float', range: [0.02, 0.7], default: 0.15, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
contrast: { type: 'float', range: [0.5, 6], default: 2.0, uniform: 'u_contrast' },
|
||||||
|
threshold: { type: 'float', range: [0, 0.9], default: 0.35, uniform: 'u_threshold' },
|
||||||
|
glow: { type: 'float', range: [0, 1.5], default: 0.5, uniform: 'u_glow', bias: 'energy' },
|
||||||
|
breathe: { type: 'float', range: [0, 0.5], default: 0.15, uniform: 'u_breathe' },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
glow: { feature: 'bandHigh', amount: 0.35, response: 'smooth' },
|
||||||
|
threshold: { feature: 'bandLow', amount: 0.2, response: 'smooth' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
p = sigFolded(sigCamera(p));
|
||||||
|
|
||||||
|
// Sum of N plane waves, each rotated by pi/N. When N is odd the angles are
|
||||||
|
// incommensurate with any lattice, which is exactly why the result never
|
||||||
|
// tiles — and why an even N looks disappointingly like a grid.
|
||||||
|
float sum = 0.0;
|
||||||
|
float n = max(float(u_waves), 1.0);
|
||||||
|
float freq = u_frequency * (1.0 + sin(t * 0.5) * u_breathe);
|
||||||
|
|
||||||
|
for (int i = 0; i < 11; i++) {
|
||||||
|
if (i >= u_waves) break;
|
||||||
|
float a = 3.14159265 * float(i) / n + t * 0.08;
|
||||||
|
vec2 dir = vec2(cos(a), sin(a));
|
||||||
|
sum += cos(dot(p, dir) * freq + t * 1.7 + float(i) * 0.6);
|
||||||
|
}
|
||||||
|
|
||||||
|
float field = sum / n;
|
||||||
|
|
||||||
|
// Contrast shapes the sum into either soft blobs or hard cells; the
|
||||||
|
// threshold cuts it into the flat-topped plateaus that read as a lattice of
|
||||||
|
// rosettes rather than as a ripple.
|
||||||
|
float shaped = tanh(field * u_contrast);
|
||||||
|
float plateau = smoothstep(u_threshold - 0.12, u_threshold + 0.12, shaped);
|
||||||
|
|
||||||
|
vec3 col = mix(pal(0) * 0.08, palRamp(0.15 + field * 0.35), 0.5 + shaped * 0.5);
|
||||||
|
col = mix(col, pal(2), plateau * 0.55);
|
||||||
|
|
||||||
|
// Ridges: where the sum crosses zero, which is the aperiodic skeleton.
|
||||||
|
float ridge = 1.0 - abs(shaped);
|
||||||
|
col += pal(4) * pow(sat(ridge), 6.0) * u_glow;
|
||||||
|
col += pal(3) * sigEdge(shaped) * u_glow * 0.3;
|
||||||
|
|
||||||
|
col *= 0.65 + 0.35 * exp(-dot(p, p) * 0.22);
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default quasicrystal;
|
||||||
79
flow-state/src/scenes/shader/ridge-terrain.js
Normal file
79
flow-state/src/scenes/shader/ridge-terrain.js
Normal file
@ -0,0 +1,79 @@
|
|||||||
|
// Structural family: layered ridge silhouettes receding to a horizon.
|
||||||
|
//
|
||||||
|
// Cheap fake depth — parallax layers rather than a raymarch — which keeps it
|
||||||
|
// affordable at 4K while still reading as a place rather than a pattern.
|
||||||
|
|
||||||
|
export const ridgeTerrain = {
|
||||||
|
name: 'Ridge Terrain',
|
||||||
|
family: 'structural',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
traits: ['space', 'camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
layers: { type: 'int', range: [2, 10], default: 6, uniform: 'u_layers', bias: 'density' },
|
||||||
|
height: { type: 'float', range: [0.1, 0.8], default: 0.35, uniform: 'u_height', bias: 'energy' },
|
||||||
|
rough: { type: 'float', range: [1, 6], default: 2.5, uniform: 'u_rough' },
|
||||||
|
speed: { type: 'float', range: [0.01, 0.3], default: 0.06, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
horizon: { type: 'float', range: [-0.4, 0.4], default: 0.0, uniform: 'u_horizon' },
|
||||||
|
haze: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_haze' },
|
||||||
|
stars: { type: 'float', range: [0, 1], default: 0.3, uniform: 'u_stars' },
|
||||||
|
palette: { type: 'palette', count: 6 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
height: { feature: 'bandLow', amount: 0.25, response: 'smooth' },
|
||||||
|
haze: { feature: 'beat', amount: 0.2, response: 'spike' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
// Its own horizon param decides the framing; the track decides where the
|
||||||
|
// ground actually is, so two scenes with a horizon agree on one.
|
||||||
|
float horizon = clamp(u_horizon + sigHorizonY() * 0.35, -0.85, 0.85);
|
||||||
|
|
||||||
|
// Sky gradient above the horizon.
|
||||||
|
float sky = sat((p.y - horizon) * 0.8 + 0.5);
|
||||||
|
vec3 col = mix(pal(0) * 0.35, pal(1) * 0.18, sky);
|
||||||
|
|
||||||
|
// Sparse stars, only in the upper sky, fading as haze rises.
|
||||||
|
if (u_stars > 0.01 && p.y > horizon) {
|
||||||
|
vec2 cell = floor(uv * 220.0);
|
||||||
|
float rnd = hash12(cell);
|
||||||
|
float star = step(0.9975, rnd) * sat((p.y - horizon) * 2.0);
|
||||||
|
col += vec3(star) * u_stars * (0.6 + 0.4 * sin(t * 8.0 + rnd * 30.0));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ridges, far to near. Each is a 1D fbm silhouette.
|
||||||
|
for (int i = 0; i < 10; i++) {
|
||||||
|
if (i >= u_layers) break;
|
||||||
|
float fi = float(i);
|
||||||
|
float depth = fi / float(max(u_layers - 1, 1)); // 0 far .. 1 near
|
||||||
|
|
||||||
|
float parallax = mix(0.15, 1.0, depth);
|
||||||
|
float x = p.x * mix(0.6, 1.8, depth) + t * parallax + fi * 13.7;
|
||||||
|
|
||||||
|
float ridge = fbm(vec2(x, fi * 5.1) * u_rough, 4) - 0.5;
|
||||||
|
float base = horizon - depth * 0.28;
|
||||||
|
float top = base + ridge * u_height * mix(0.5, 1.3, depth);
|
||||||
|
|
||||||
|
float mask = smoothstep(0.004, 0.0, p.y - top);
|
||||||
|
vec3 tint = mix(pal(2), pal(4), depth);
|
||||||
|
// Distant layers wash out toward the sky colour.
|
||||||
|
tint = mix(mix(pal(1) * 0.4, tint, 0.35 + depth * 0.65), tint, 1.0 - u_haze * (1.0 - depth));
|
||||||
|
|
||||||
|
col = mix(col, tint * (0.25 + depth * 0.75), mask);
|
||||||
|
|
||||||
|
// Rim light along each crest.
|
||||||
|
col += pal(5) * smoothstep(0.02, 0.0, abs(p.y - top)) * (0.12 + depth * 0.25) * u_haze;
|
||||||
|
}
|
||||||
|
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default ridgeTerrain;
|
||||||
83
flow-state/src/scenes/shader/salt-flat.js
Normal file
83
flow-state/src/scenes/shader/salt-flat.js
Normal file
@ -0,0 +1,83 @@
|
|||||||
|
// Minimal family: an empty plain under a very large sky, with one distant form
|
||||||
|
// standing on the horizon.
|
||||||
|
//
|
||||||
|
// The emptiest scene in the library and the most deliberate about it — most of
|
||||||
|
// the frame is a gradient. Distinct from Horizon Lines (a bundle of lines) and
|
||||||
|
// Ridge Terrain (layered silhouettes): there is exactly one object, it is small,
|
||||||
|
// and it is far away. What moves is the light and the heat shimmer.
|
||||||
|
//
|
||||||
|
// The object is the track's signature form, so the thing on the horizon of a
|
||||||
|
// hexagonal video is a hexagon.
|
||||||
|
|
||||||
|
export const saltFlat = {
|
||||||
|
name: 'Salt Flat',
|
||||||
|
family: 'minimal',
|
||||||
|
kind: 'fragment',
|
||||||
|
traits: ['shape', 'camera', 'space', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
monolith: { type: 'float', range: [0.0, 0.35], default: 0.12, uniform: 'u_monolith' },
|
||||||
|
standing: { type: 'float', range: [-0.7, 0.7], default: 0.0, uniform: 'u_standing' },
|
||||||
|
shimmer: { type: 'float', range: [0, 0.09], default: 0.025,uniform: 'u_shimmer' },
|
||||||
|
glowBand: { type: 'float', range: [0, 1.2], default: 0.45, uniform: 'u_glowBand', bias: 'energy' },
|
||||||
|
ground: { type: 'float', range: [0, 1], default: 0.4, uniform: 'u_ground' },
|
||||||
|
salt: { type: 'float', range: [0, 1], default: 0.35, uniform: 'u_salt', bias: 'density' },
|
||||||
|
pace: { type: 'float', range: [0.02, 0.5], default: 0.12, uniform: 'u_pace', bias: 'motion', rate: true },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
glowBand: { feature: 'loudness', amount: 0.3, response: 'smooth' },
|
||||||
|
shimmer: { feature: 'bandAir', amount: 0.25 },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_pace + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
float horizon = sigHorizonY() * 0.6 - 0.15;
|
||||||
|
|
||||||
|
// Heat shimmer: everything near the horizon wobbles, nothing else does.
|
||||||
|
float nearHorizon = exp(-abs(p.y - horizon) * 5.0);
|
||||||
|
p.x += sin(p.y * 60.0 + t * 6.0) * u_shimmer * nearHorizon;
|
||||||
|
|
||||||
|
float above = p.y - horizon;
|
||||||
|
|
||||||
|
// Sky: a tall gradient, darkest at the top.
|
||||||
|
vec3 col = mix(pal(1) * 0.5, pal(0) * 0.25, sat(above * 0.8 + 0.15));
|
||||||
|
|
||||||
|
// The glow band sitting on the horizon — the light source of the whole scene.
|
||||||
|
col += pal(3) * exp(-abs(above) * 9.0) * u_glowBand * 0.8;
|
||||||
|
|
||||||
|
if (above < 0.0) {
|
||||||
|
float depth = sat(-above * 2.2); // 0 far .. 1 near
|
||||||
|
vec3 plain = mix(pal(2) * 0.5, pal(0) * 0.3, depth);
|
||||||
|
|
||||||
|
// Salt crust: cracked cells, only legible in the near field.
|
||||||
|
vec2 cellUv = vec2(p.x / max(-above * 0.9 + 0.06, 0.02), 1.0 / max(-above + 0.05, 0.02));
|
||||||
|
float crack = abs(fract(cellUv.x * 0.5) - 0.5) + abs(fract(cellUv.y * 0.5) - 0.5);
|
||||||
|
plain += pal(4) * smoothstep(0.42, 0.5, crack) * u_salt * depth * 0.25;
|
||||||
|
|
||||||
|
// Reflection of the glow band, compressed toward the horizon.
|
||||||
|
plain += pal(3) * exp(above * 7.0) * u_glowBand * 0.3;
|
||||||
|
|
||||||
|
col = mix(col, plain, u_ground);
|
||||||
|
}
|
||||||
|
|
||||||
|
// The one object: small, on the horizon, in the track's form.
|
||||||
|
if (u_monolith > 0.005) {
|
||||||
|
vec2 at = vec2(u_standing, horizon + u_monolith * 0.9);
|
||||||
|
float d = sigShape((p - at) / u_monolith) * u_monolith;
|
||||||
|
col = mix(col, pal(0) * 0.12, smoothstep(0.006, -0.006, d));
|
||||||
|
col += pal(4) * sigEdge(d) * (0.3 + u_glowBand * 0.5);
|
||||||
|
}
|
||||||
|
|
||||||
|
col = sigAir(col, p, smoothstep(0.0, 1.4, abs(p.x)));
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default saltFlat;
|
||||||
78
flow-state/src/scenes/shader/scan-tear.js
Normal file
78
flow-state/src/scenes/shader/scan-tear.js
Normal file
@ -0,0 +1,78 @@
|
|||||||
|
// Glitch family: horizontal block displacement, chroma tearing and scanlines,
|
||||||
|
// built on the feedback buffer so the corruption smears across frames.
|
||||||
|
//
|
||||||
|
// Everything here is quantised to a beat- or bar-locked step rather than driven
|
||||||
|
// continuously. Free-running glitch reads as a broken renderer; glitch that
|
||||||
|
// lands on the grid reads as an effect.
|
||||||
|
|
||||||
|
export const scanTear = {
|
||||||
|
name: 'Scan Tear',
|
||||||
|
family: 'glitch',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
traits: ['style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
slices: { type: 'float', range: [4, 48], default: 16, uniform: 'u_slices', bias: 'density' },
|
||||||
|
shift: { type: 'float', range: [0, 0.4], default: 0.12, uniform: 'u_shift', bias: 'energy' },
|
||||||
|
tear: { type: 'float', range: [0, 1], default: 0.4, uniform: 'u_tear' },
|
||||||
|
chroma: { type: 'float', range: [0, 0.08], default: 0.02, uniform: 'u_chromaSplit' },
|
||||||
|
scan: { type: 'float', range: [0, 1], default: 0.35, uniform: 'u_scan' },
|
||||||
|
persist: { type: 'float', range: [0, 0.9], default: 0.45, uniform: 'u_persist' },
|
||||||
|
speed: { type: 'float', range: [0.05, 0.8], default: 0.25, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
shift: { feature: 'beat', amount: 0.5, response: 'spike' },
|
||||||
|
tear: { feature: 'flux', amount: 0.35, response: 'spike' },
|
||||||
|
slices:{ feature: 'bandHigh', amount: 0.25 },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
|
||||||
|
// Quantise to the bar so displacement steps in time with the music rather
|
||||||
|
// than crawling. floor() of the bar phase gives a stable step per bar.
|
||||||
|
float step_ = floor(u_barPhase * 8.0) + floor(t * 4.0) * 8.0;
|
||||||
|
|
||||||
|
float row = floor(uv.y * u_slices);
|
||||||
|
float rowRandom = hash12(vec2(row, step_));
|
||||||
|
|
||||||
|
// Only some rows tear, and only above the tear threshold.
|
||||||
|
// Slice weight follows the track's line weight — the same art direction
|
||||||
|
// that thickens a grid line thickens a tear.
|
||||||
|
float torn = step(1.0 - u_tear * (0.6 + u_sigLine), rowRandom);
|
||||||
|
float offset = (rowRandom - 0.5) * 2.0 * u_shift * torn;
|
||||||
|
|
||||||
|
vec2 q = vec2(fract(uv.x + offset), uv.y);
|
||||||
|
|
||||||
|
// Base image: a banded field, so the scene stands alone rather than needing
|
||||||
|
// something underneath it.
|
||||||
|
float band = fbm(vec2(q.x * 3.0, q.y * 6.0 + t), 4);
|
||||||
|
float ramp = fract(band * 2.0 + q.y * 2.0 - t * 0.5);
|
||||||
|
vec3 col = palRamp(ramp * 0.7 + row * 0.02);
|
||||||
|
col *= 0.4 + 0.6 * smoothstep(0.1, 0.9, band);
|
||||||
|
|
||||||
|
// Chroma split, strongest on torn rows.
|
||||||
|
float split = u_chromaSplit * (0.35 + torn);
|
||||||
|
col.r = mix(col.r, palRamp(ramp + split).r, 0.6);
|
||||||
|
col.b = mix(col.b, palRamp(ramp - split).b, 0.6);
|
||||||
|
|
||||||
|
// Scanlines, in normalised space so they survive a resolution change.
|
||||||
|
float lines = 0.5 + 0.5 * sin(uv.y * 900.0 * u_pixelScale);
|
||||||
|
col *= 1.0 - u_scan * 0.45 * lines;
|
||||||
|
|
||||||
|
// Smear the previous frame along the displacement.
|
||||||
|
vec3 ghost = prev(vec2(fract(uv.x + offset * 0.6), uv.y));
|
||||||
|
col = max(col, ghost * u_persist);
|
||||||
|
|
||||||
|
col += pal(4) * torn * u_shift * 0.6;
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default scanTear;
|
||||||
90
flow-state/src/scenes/shader/signal-decay.js
Normal file
90
flow-state/src/scenes/shader/signal-decay.js
Normal file
@ -0,0 +1,90 @@
|
|||||||
|
// Glitch family: a stack of oscilloscope traces losing signal.
|
||||||
|
//
|
||||||
|
// Distinct from Scan Tear (rows displaced sideways), Block Mosh (block-level
|
||||||
|
// datamosh) and Pitch Shatter (vertical transposition): nothing here is
|
||||||
|
// displaced at all. The corruption is in the SIGNAL — each trace degrades from a
|
||||||
|
// clean wave into noise as its lock is lost, and regains it. Loss of lock steps
|
||||||
|
// on the bar grid, so traces drop out in time rather than flickering.
|
||||||
|
|
||||||
|
export const signalDecay = {
|
||||||
|
name: 'Signal Decay',
|
||||||
|
family: 'glitch',
|
||||||
|
kind: 'fragment',
|
||||||
|
traits: ['camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
traces: { type: 'int', range: [2, 10], default: 5, uniform: 'u_traces', bias: 'density' },
|
||||||
|
amplitude:{ type: 'float', range: [0.02, 0.3], default: 0.1, uniform: 'u_amplitude', bias: 'energy' },
|
||||||
|
frequency:{ type: 'float', range: [1, 22], default: 7, uniform: 'u_frequency', bias: 'density' },
|
||||||
|
loss: { type: 'float', range: [0, 0.9], default: 0.35, uniform: 'u_loss' },
|
||||||
|
hiss: { type: 'float', range: [0, 1], default: 0.4, uniform: 'u_hiss' },
|
||||||
|
persist: { type: 'float', range: [0, 0.85], default: 0.4, uniform: 'u_persist' },
|
||||||
|
speed: { type: 'float', range: [0.1, 2.0], default: 0.6, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
amplitude: { feature: 'bandLow', amount: 0.35, response: 'smooth' },
|
||||||
|
hiss: { feature: 'flatness', amount: 0.3 },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
// Quantised era: lock is lost and regained on the eighth-bar grid, so
|
||||||
|
// dropouts land with the music instead of crawling.
|
||||||
|
float era = floor(u_barPhase * 8.0) + floor(t * 2.0) * 8.0;
|
||||||
|
|
||||||
|
vec3 col = pal(0) * 0.05;
|
||||||
|
float span = 2.0 / max(float(u_traces), 1.0);
|
||||||
|
|
||||||
|
for (int i = 0; i < 10; i++) {
|
||||||
|
if (i >= u_traces) break;
|
||||||
|
float fi = float(i);
|
||||||
|
float centre = -1.0 + span * (fi + 0.5);
|
||||||
|
float s = u_seed + fi * 27.7;
|
||||||
|
|
||||||
|
// How much lock this trace has this era. Below zero it is pure noise.
|
||||||
|
float lock = sat(hash12(vec2(fi, era)) * 1.4 - u_loss);
|
||||||
|
|
||||||
|
// Clean signal: two sines and a slow envelope, so it looks like a
|
||||||
|
// waveform rather than a test tone.
|
||||||
|
float clean = sin(p.x * u_frequency + t * 3.0 + s) * 0.6
|
||||||
|
+ sin(p.x * u_frequency * 2.7 - t * 1.7 + s * 1.3) * 0.4;
|
||||||
|
|
||||||
|
// Noise floor: hashed per pixel column and era, held steady within a
|
||||||
|
// step so it reads as static rather than as a shimmer.
|
||||||
|
float noise = (hash12(vec2(floor(p.x * 220.0), era + fi)) - 0.5) * 2.0;
|
||||||
|
|
||||||
|
float signal = mix(noise, clean, lock);
|
||||||
|
float y = centre + signal * u_amplitude;
|
||||||
|
|
||||||
|
float d = abs(p.y - y);
|
||||||
|
float w = 0.004 + u_sigLine * 0.012;
|
||||||
|
float line = smoothstep(w * 2.5, 0.0, d);
|
||||||
|
|
||||||
|
vec3 tint = palRamp(fract(s) * 0.4 + 0.15);
|
||||||
|
col += tint * line * (0.4 + lock * 0.6);
|
||||||
|
col += tint * exp(-d * 40.0) * 0.25 * lock;
|
||||||
|
|
||||||
|
// Hiss band around an unlocked trace: the visual equivalent of the
|
||||||
|
// sound. Confined to the lane, so it never washes the whole frame.
|
||||||
|
if (lock < 0.4) {
|
||||||
|
float band = exp(-abs(p.y - centre) * 12.0);
|
||||||
|
col += pal(3) * band * abs(noise) * u_hiss * 0.3 * (1.0 - lock);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ghost of the previous frame, so a dropout leaves a trail rather than
|
||||||
|
// vanishing cleanly.
|
||||||
|
col = max(col, prev(uv) * u_persist);
|
||||||
|
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default signalDecay;
|
||||||
73
flow-state/src/scenes/shader/silk-ribbon.js
Normal file
73
flow-state/src/scenes/shader/silk-ribbon.js
Normal file
@ -0,0 +1,73 @@
|
|||||||
|
// Minimal family: a single ribbon of light draping across an almost empty
|
||||||
|
// frame. The curve is a closed-form travelling wave, so it is seek-exact; it is
|
||||||
|
// drawn by sampling the polyline and taking the minimum distance, which reads as
|
||||||
|
// one fluid strand rather than a repeated motif. Most of the frame is negative
|
||||||
|
// space — what an intro or a breakdown needs.
|
||||||
|
|
||||||
|
export const silkRibbon = {
|
||||||
|
name: 'Silk Ribbon',
|
||||||
|
family: 'minimal',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
// Grain is this scene's ONLY expression of the style trait — it draws no
|
||||||
|
// hard edges to weight — so it keeps a share of it rather than opting out.
|
||||||
|
texture: 0.35,
|
||||||
|
traits: ['camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
thickness: { type: 'float', range: [0.008, 0.08], default: 0.03, uniform: 'u_thickness' },
|
||||||
|
wave: { type: 'float', range: [0.1, 2.5], default: 1.1, uniform: 'u_wave' },
|
||||||
|
sway: { type: 'float', range: [0.05, 0.6], default: 0.25, uniform: 'u_sway', bias: 'motion' },
|
||||||
|
glow: { type: 'float', range: [0, 1.5], default: 0.5, uniform: 'u_glow', bias: 'energy' },
|
||||||
|
speed: { type: 'float', range: [0.02, 0.4], default: 0.12, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
spread: { type: 'float', range: [0.5, 1.2], default: 0.9, uniform: 'u_spread' },
|
||||||
|
palette: { type: 'palette', count: 4 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
glow: { feature: 'beat', amount: 0.3, response: 'smooth' },
|
||||||
|
wave: { feature: 'bandLow', amount: 0.25, response: 'smooth' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
// Nearest distance from p to a fixed travelling strand.
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
// Bare wash so the frame is never black.
|
||||||
|
vec3 col = pal(0) * 0.04;
|
||||||
|
|
||||||
|
// Two nested strands, offset in phase so the ribbon reads as a drape with a
|
||||||
|
// folded edge rather than a single hairline.
|
||||||
|
for (int k = 0; k < 2; k++) {
|
||||||
|
float fk = float(k);
|
||||||
|
float ph = fk * 2.2 + t * 0.3;
|
||||||
|
float best = 1e9;
|
||||||
|
|
||||||
|
// lint: fixed-cost — this samples the curve at a fixed resolution, so
|
||||||
|
// there is no param to break on. Cost is governed by the 4K budget gate.
|
||||||
|
for (int n = 0; n < 48; n++) {
|
||||||
|
float uu = (float(n) + 0.5) / 48.0;
|
||||||
|
float x = (uu - 0.5) * 2.0 * u_spread;
|
||||||
|
float y = sin(uu * 6.28318530718 * u_wave + ph) * 0.28
|
||||||
|
+ sin(uu * 13.1 - t * 0.8 + fk) * 0.06
|
||||||
|
+ u_sway * 0.4 * sin(t * 0.6 + fk);
|
||||||
|
float d = length(p - vec2(x, y));
|
||||||
|
best = min(best, d);
|
||||||
|
}
|
||||||
|
|
||||||
|
vec3 hc = pal(int(fk) % 3);
|
||||||
|
col += hc * exp(-best * best / (u_thickness * u_thickness));
|
||||||
|
col += pal(3) * exp(-best * best * 60.0) * u_glow * 0.6;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Knot the ribbon just below frame centre so it has somewhere to pull from.
|
||||||
|
float knot = length(p - vec2(0.0, 0.1));
|
||||||
|
col = mix(col, pal(2), exp(-knot * knot * 30.0) * u_glow * 0.5);
|
||||||
|
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
65
flow-state/src/scenes/shader/slow-orb.js
Normal file
65
flow-state/src/scenes/shader/slow-orb.js
Normal file
@ -0,0 +1,65 @@
|
|||||||
|
// Minimal family: one soft body drifting through a mostly empty frame.
|
||||||
|
//
|
||||||
|
// The quietest scene in the library, and the one an ambient intro or a long
|
||||||
|
// breakdown should usually land on. Nothing here reacts sharply — the beat
|
||||||
|
// mapping is deliberately weak, because a scene whose job is stillness should
|
||||||
|
// not twitch.
|
||||||
|
|
||||||
|
export const slowOrb = {
|
||||||
|
name: 'Slow Orb',
|
||||||
|
family: 'minimal',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
// Grain is this scene's ONLY expression of the style trait — it draws no
|
||||||
|
// hard edges to weight — so it keeps a share of it rather than opting out.
|
||||||
|
texture: 0.35,
|
||||||
|
traits: ['shape', 'camera', 'space', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
size: { type: 'float', range: [0.15, 0.8], default: 0.38, uniform: 'u_size', bias: 'energy' },
|
||||||
|
softness: { type: 'float', range: [0.2, 1.0], default: 0.7, uniform: 'u_softness' },
|
||||||
|
drift: { type: 'float', range: [0.01, 0.2], default: 0.05, uniform: 'u_drift', bias: 'motion', rate: true },
|
||||||
|
wobble: { type: 'float', range: [0, 0.5], default: 0.15, uniform: 'u_wobble' },
|
||||||
|
halo: { type: 'float', range: [0, 1.2], default: 0.4, uniform: 'u_halo' },
|
||||||
|
grain: { type: 'float', range: [0, 0.5], default: 0.12, uniform: 'u_grain' },
|
||||||
|
palette: { type: 'palette', count: 4 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
size: { feature: 'loudness', amount: 0.12, response: 'smooth' },
|
||||||
|
halo: { feature: 'beat', amount: 0.15, response: 'spike' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_drift + u_seed;
|
||||||
|
|
||||||
|
// The orb drifts about the track's horizon rather than about the middle of
|
||||||
|
// the frame, so it occupies the same space as the scenes that draw ground.
|
||||||
|
vec2 centre = vec2(sin(t * 1.7) * 0.28, cos(t * 1.3) * 0.18 + sigHorizonY() * 0.45);
|
||||||
|
vec2 q = sigCamera(p) - centre;
|
||||||
|
|
||||||
|
// The body is the track's signature form — this scene is one shape in an
|
||||||
|
// empty frame, so the shape had better be the track's.
|
||||||
|
float wobble = fbm(q * 2.4 + t, 4) * u_wobble;
|
||||||
|
float d = (sigShape(q / max(u_size, 1e-3)) * u_size) * (1.0 + wobble);
|
||||||
|
|
||||||
|
float body = smoothstep(u_softness * 0.5, -u_softness * 0.5, d);
|
||||||
|
float glow = exp(-max(d, 0.0) * (5.0 / max(u_halo, 0.05))) * u_halo;
|
||||||
|
|
||||||
|
vec3 col = pal(0) * 0.05;
|
||||||
|
col = mix(col, pal(1), body * 0.85);
|
||||||
|
col += pal(2) * body * body * 0.5;
|
||||||
|
col += pal(3) * glow * 0.35;
|
||||||
|
|
||||||
|
// Fine grain keeps large flat areas from banding.
|
||||||
|
col = sigAir(col, p, smoothstep(0.0, 1.6, length(q)));
|
||||||
|
col += (hash12(uv * 640.0 + floor(u_frame)) - 0.5) * u_grain * 0.08;
|
||||||
|
col += sigGrain(uv);
|
||||||
|
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default slowOrb;
|
||||||
89
flow-state/src/scenes/shader/smoke-column.js
Normal file
89
flow-state/src/scenes/shader/smoke-column.js
Normal file
@ -0,0 +1,89 @@
|
|||||||
|
// Flow family: a thermal plume rising from a source on the track's horizon.
|
||||||
|
//
|
||||||
|
// Distinct from Aurora Veil (sheets that stand still and ripple sideways), Curl
|
||||||
|
// Flow (an isotropic field with no origin) and Firefly Drift (discrete motes):
|
||||||
|
// everything here moves in one direction, and it WIDENS as it goes. The plume
|
||||||
|
// has a source, a body and a dissipating head, which is a shape none of those
|
||||||
|
// three can make — an isotropic field has no up.
|
||||||
|
//
|
||||||
|
// Scaffolded by tools/new-scene.js. See HOWTO-visualizers.md.
|
||||||
|
|
||||||
|
export const smokeColumn = {
|
||||||
|
name: 'Smoke Column',
|
||||||
|
family: 'flow',
|
||||||
|
kind: 'fragment',
|
||||||
|
traits: ['camera', 'space', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
scale: { type: 'float', range: [1, 9], default: 3.4, uniform: 'u_scale', bias: 'density' },
|
||||||
|
speed: { type: 'float', range: [0.05, 1.2], default: 0.32, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
detail: { type: 'float', range: [0.2, 2.5], default: 1.1, uniform: 'u_detail', bias: 'density' },
|
||||||
|
glow: { type: 'float', range: [0, 1.5], default: 0.5, uniform: 'u_glow', bias: 'energy' },
|
||||||
|
spread: { type: 'float', range: [0.15, 1.4], default: 0.55, uniform: 'u_spread' },
|
||||||
|
buoyancy: { type: 'float', range: [0.3, 2.5], default: 1.1, uniform: 'u_buoyancy' },
|
||||||
|
ember: { type: 'float', range: [0, 1], default: 0.35, uniform: 'u_ember', bias: 'energy' },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
glow: { feature: 'bandLow', amount: 0.3, response: 'smooth' },
|
||||||
|
detail: { feature: 'bandHigh', amount: 0.25, response: 'smooth' },
|
||||||
|
ember: { feature: 'beat', amount: 0.25, response: 'spike' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
float source = sigHorizonY() * 0.6 - 0.85;
|
||||||
|
float rise = p.y - source; // 0 at the vent, up the frame
|
||||||
|
|
||||||
|
// The cone: the plume widens with height, so the same turbulence covers
|
||||||
|
// more of the frame the further it has travelled. Dividing x by the cone
|
||||||
|
// width before sampling is what makes the noise splay rather than shear.
|
||||||
|
float cone = u_spread * (0.25 + max(rise, 0.0) * 0.9);
|
||||||
|
float across = p.x / max(cone, 0.05);
|
||||||
|
|
||||||
|
// Advection: the noise field is sampled at a y that moves DOWN with time,
|
||||||
|
// which reads as the smoke moving up. Buoyancy accelerates it with height,
|
||||||
|
// because hot gas does.
|
||||||
|
float climb = t * (0.6 + max(rise, 0.0) * u_buoyancy * 0.5);
|
||||||
|
vec2 q = vec2(across * 0.9, rise * 1.6 - climb);
|
||||||
|
|
||||||
|
float body = fbm(q * u_scale * 0.35 + vec2(0.0, 0.0), 5);
|
||||||
|
float curls = fbm(q * u_scale * 0.9 + body * u_detail, 4);
|
||||||
|
float density = body * 0.65 + curls * 0.35;
|
||||||
|
|
||||||
|
// Envelope: nothing outside the cone, nothing below the vent, and the head
|
||||||
|
// dissipates. Without the last term the column just leaves the top of frame.
|
||||||
|
float inside = exp(-across * across * 0.9);
|
||||||
|
float above = smoothstep(-0.05, 0.25, rise);
|
||||||
|
float head = 1.0 - smoothstep(0.7, 2.1, rise);
|
||||||
|
float plume = density * inside * above * head;
|
||||||
|
|
||||||
|
vec3 col = pal(0) * 0.05;
|
||||||
|
col += palRamp(0.15 + density * 0.35 + rise * 0.08) * plume * 1.15;
|
||||||
|
col += pal(4) * pow(sat(plume), 3.0) * u_glow * 0.6;
|
||||||
|
|
||||||
|
// Embers: bright points carried up inside the column, on their own hashed
|
||||||
|
// lanes. Local, so the frame never swings as a whole.
|
||||||
|
if (u_ember > 0.01) {
|
||||||
|
float lane = floor(across * 4.0);
|
||||||
|
float lanePhase = fract(hash11(lane + u_seed) + t * 0.4);
|
||||||
|
float emberY = mix(0.0, 1.8, lanePhase);
|
||||||
|
float d = length(vec2(p.x - lane * cone * 0.25, rise - emberY));
|
||||||
|
col += pal(3) * exp(-d * 40.0) * u_ember * (1.0 - lanePhase) * inside;
|
||||||
|
}
|
||||||
|
|
||||||
|
// The vent itself.
|
||||||
|
col += pal(2) * exp(-length(vec2(p.x * 2.2, rise * 5.0))) * (0.35 + u_glow * 0.5);
|
||||||
|
|
||||||
|
col = sigAir(col, p, smoothstep(0.0, 1.8, length(p)));
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default smokeColumn;
|
||||||
102
flow-state/src/scenes/shader/spectrum-sculpture.js
Normal file
102
flow-state/src/scenes/shader/spectrum-sculpture.js
Normal file
@ -0,0 +1,102 @@
|
|||||||
|
// Minimal family: a radial bar sculpture driven by the band split.
|
||||||
|
//
|
||||||
|
// This is the one scene that shows the spectrum more or less literally. It reads
|
||||||
|
// as a music visualiser rather than as an abstraction, which is why it is
|
||||||
|
// deliberately restrained — thin bars, lots of black — and why it lives in
|
||||||
|
// `minimal` rather than `geometric`.
|
||||||
|
|
||||||
|
export const spectrumSculpture = {
|
||||||
|
name: 'Spectrum Sculpture',
|
||||||
|
family: 'minimal',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
// Takes the track's surface grain, but lightly — this is drawn, not filmed.
|
||||||
|
texture: 0.4,
|
||||||
|
traits: ['shape', 'camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
bars: { type: 'float', range: [8, 96], default: 40, uniform: 'u_bars', bias: 'density' },
|
||||||
|
radius: { type: 'float', range: [0.15, 0.7], default: 0.35, uniform: 'u_radius' },
|
||||||
|
length: { type: 'float', range: [0.05, 0.6], default: 0.25, uniform: 'u_length', bias: 'energy' },
|
||||||
|
thickness: { type: 'float', range: [0.1, 0.9], default: 0.45, uniform: 'u_thickness' },
|
||||||
|
// Bar segments per second, NOT turns per second. Rotating by a full turn
|
||||||
|
// meant the bar-crossing frequency was bars x rate — at 82 bars that put a
|
||||||
|
// slow-looking 0.12 turns/s at 10 Hz of luminance flicker. In segment units
|
||||||
|
// the crossing frequency IS the rate, so it stays under the flash ceiling
|
||||||
|
// whatever the bar count.
|
||||||
|
// Capped at 0.4 by measurement, not by taste: at 0.83 the mirror fold puts
|
||||||
|
// this at 4 flashes/s, and at 0.4 it measures 0. Bar count no longer affects
|
||||||
|
// it now that rotation is in segment units.
|
||||||
|
rotate: { type: 'float', range: [0, 0.4], default: 0.2, uniform: 'u_rotate', bias: 'motion', rate: true },
|
||||||
|
mirror: { type: 'bool', default: true, uniform: 'u_mirror' },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
// Kept low deliberately: bar length scales the whole ring at once, so a
|
||||||
|
// large amount pumps global luminance and measured 4 flashes/s against a
|
||||||
|
// ceiling of 3. See engine/flash.js.
|
||||||
|
length: { feature: 'loudness', amount: 0.15, response: 'smooth' },
|
||||||
|
thickness: { feature: 'beat', amount: 0.12, response: 'spike' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
float bandByIndex(float i) {
|
||||||
|
if (i < 0.5) return u_bandSub;
|
||||||
|
if (i < 1.5) return u_bandLow;
|
||||||
|
if (i < 2.5) return u_bandMid;
|
||||||
|
if (i < 3.5) return u_bandHigh;
|
||||||
|
return u_bandAir;
|
||||||
|
}
|
||||||
|
|
||||||
|
float bandAt(float x) {
|
||||||
|
float s = clamp(x, 0.0, 1.0) * 4.0;
|
||||||
|
float i = floor(s);
|
||||||
|
float f = fract(s);
|
||||||
|
f = f * f * (3.0 - 2.0 * f);
|
||||||
|
return mix(bandByIndex(i), bandByIndex(i + 1.0), f);
|
||||||
|
}
|
||||||
|
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_rotate + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
float bars = u_bars;
|
||||||
|
float seg = 6.28318530718 / bars;
|
||||||
|
|
||||||
|
float angle = atan(p.y, p.x) + t * seg;
|
||||||
|
// Radial extent measured in the track's form, so the sculpture is built on
|
||||||
|
// the same outline as everything else in the video.
|
||||||
|
float radius = sigShape(p) + 1.0;
|
||||||
|
float index = floor((angle + 3.14159265) / seg);
|
||||||
|
float cellAngle = mod(angle + 3.14159265, seg) / seg;
|
||||||
|
|
||||||
|
// Fold the ring so the two halves mirror; reads as a designed object rather
|
||||||
|
// than a spinning readout.
|
||||||
|
float slot = u_mirror > 0.5 ? abs(index / bars - 0.5) * 2.0 : index / bars;
|
||||||
|
|
||||||
|
// Band split across the ring, sub at one end and air at the other, INTERPOLATED
|
||||||
|
// rather than switched. Hard tier boundaries made every bar jump between bands
|
||||||
|
// at the same moment as the ring rotated, which stepped whole-frame luminance
|
||||||
|
// and measured 4 flashes/s against a ceiling of 3. Blending removes the step
|
||||||
|
// and looks better besides.
|
||||||
|
float band = bandAt(slot);
|
||||||
|
|
||||||
|
float height = u_radius + u_length * (0.25 + band);
|
||||||
|
float inBar = step(u_radius, radius) * step(radius, height);
|
||||||
|
float shape = smoothstep(0.5 - u_thickness * 0.5, 0.5, cellAngle)
|
||||||
|
* smoothstep(0.5 + u_thickness * 0.5, 0.5, cellAngle);
|
||||||
|
|
||||||
|
vec3 col = pal(0) * 0.05;
|
||||||
|
col += palRamp(slot * 0.7 + 0.15) * inBar * shape * (0.6 + band);
|
||||||
|
|
||||||
|
// Inner ring outline holds the composition together.
|
||||||
|
col += pal(2) * sigEdge(radius - u_radius) * 0.35;
|
||||||
|
col += sigGrain(uv);
|
||||||
|
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default spectrumSculpture;
|
||||||
109
flow-state/src/scenes/shader/storm-rift.js
Normal file
109
flow-state/src/scenes/shader/storm-rift.js
Normal file
@ -0,0 +1,109 @@
|
|||||||
|
// Glitch family: branching discharge across the sky above the track's horizon.
|
||||||
|
//
|
||||||
|
// The bolt is a recursive-looking zigzag built from stacked hashed segments,
|
||||||
|
// re-struck on the quantised grid rather than continuously, so strikes land on
|
||||||
|
// the music. Between strikes the afterglow decays through feedback, which is
|
||||||
|
// what makes the dark frames read as "after a flash" rather than as empty.
|
||||||
|
//
|
||||||
|
// The flash itself is deliberately LOCAL — the bolt and a halo around it, not
|
||||||
|
// the frame. A full-frame white flash on every kick is exactly the WCAG 2.3.1
|
||||||
|
// failure this library is checked against.
|
||||||
|
|
||||||
|
export const stormRift = {
|
||||||
|
name: 'Storm Rift',
|
||||||
|
family: 'glitch',
|
||||||
|
kind: 'fragment',
|
||||||
|
traits: ['camera', 'space', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
bolts: { type: 'int', range: [1, 5], default: 2, uniform: 'u_bolts', bias: 'density' },
|
||||||
|
jag: { type: 'float', range: [0.05, 0.6], default: 0.25, uniform: 'u_jag' },
|
||||||
|
segments: { type: 'float', range: [4, 20], default: 10, uniform: 'u_segments', bias: 'density' },
|
||||||
|
branch: { type: 'float', range: [0, 1], default: 0.45, uniform: 'u_branch' },
|
||||||
|
afterglow:{ type: 'float', range: [0, 0.9], default: 0.55, uniform: 'u_afterglow' },
|
||||||
|
cloud: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_cloud' },
|
||||||
|
rate: { type: 'float', range: [0.2, 3.0], default: 1.0, uniform: 'u_rate', bias: 'motion', rate: true },
|
||||||
|
palette: { type: 'palette', count: 5 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
branch: { feature: 'bandHigh', amount: 0.3, response: 'smooth' },
|
||||||
|
cloud: { feature: 'bandLow', amount: 0.25, response: 'smooth' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_rate + u_seed;
|
||||||
|
p = sigCamera(p);
|
||||||
|
|
||||||
|
float horizon = sigHorizonY() * 0.7 - 0.5;
|
||||||
|
|
||||||
|
// Cloud deck above, lit from within.
|
||||||
|
float deck = fbm(vec2(p.x * 1.4 + t * 0.15, p.y * 2.2 - t * 0.05), 5);
|
||||||
|
vec3 col = mix(pal(0) * 0.06, pal(1) * 0.2, deck * u_cloud * smoothstep(horizon, horizon + 1.2, p.y));
|
||||||
|
|
||||||
|
// Strikes step on the eighth-bar grid. Each era re-rolls every bolt.
|
||||||
|
float era = floor(u_barPhase * 8.0) + floor(t) * 8.0;
|
||||||
|
|
||||||
|
for (int b = 0; b < 5; b++) {
|
||||||
|
if (b >= u_bolts) break;
|
||||||
|
float fb = float(b);
|
||||||
|
float boltSeed = hash12(vec2(era, fb * 7.3 + u_seed));
|
||||||
|
|
||||||
|
// Not every bolt fires every era; misfires are what make the ones that
|
||||||
|
// land feel like events.
|
||||||
|
float fires = step(0.35, boltSeed);
|
||||||
|
float age = fract(t * 2.0 + fb * 0.31);
|
||||||
|
float intensity = fires * exp(-age * 6.0);
|
||||||
|
if (intensity < 0.004) continue;
|
||||||
|
|
||||||
|
// The channel: a piecewise-linear zigzag from the cloud deck down to the
|
||||||
|
// horizon, each segment hashed off (era, bolt, segment).
|
||||||
|
float x0 = (hash11(boltSeed * 31.0) - 0.5) * 1.8;
|
||||||
|
float best = 1e3;
|
||||||
|
|
||||||
|
for (int s = 0; s < 20; s++) {
|
||||||
|
if (float(s) >= u_segments) break;
|
||||||
|
float f0 = float(s) / u_segments;
|
||||||
|
float f1 = float(s + 1) / u_segments;
|
||||||
|
|
||||||
|
float yA = mix(1.1, horizon, f0);
|
||||||
|
float yB = mix(1.1, horizon, f1);
|
||||||
|
float xA = x0 + (hash11(boltSeed * 17.0 + float(s) * 3.7) - 0.5) * u_jag * (0.3 + f0);
|
||||||
|
float xB = x0 + (hash11(boltSeed * 17.0 + float(s + 1) * 3.7) - 0.5) * u_jag * (0.3 + f1);
|
||||||
|
|
||||||
|
// Distance to this segment.
|
||||||
|
vec2 a = vec2(xA, yA), bb = vec2(xB, yB);
|
||||||
|
vec2 pa = p - a, ba = bb - a;
|
||||||
|
float h = clamp(dot(pa, ba) / max(dot(ba, ba), 1e-6), 0.0, 1.0);
|
||||||
|
best = min(best, length(pa - ba * h));
|
||||||
|
|
||||||
|
// Branches: a short spur off some joints, in the same hand.
|
||||||
|
if (hash11(boltSeed * 53.0 + float(s)) < u_branch * 0.4) {
|
||||||
|
vec2 tip = a + vec2((hash11(boltSeed + float(s) * 5.1) - 0.5) * 0.5, -0.12);
|
||||||
|
vec2 pb = p - a, bc = tip - a;
|
||||||
|
float h2 = clamp(dot(pb, bc) / max(dot(bc, bc), 1e-6), 0.0, 1.0);
|
||||||
|
best = min(best, length(pb - bc * h2) + 0.008);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
float core = smoothstep(0.012 + u_sigLine * 0.01, 0.0, best);
|
||||||
|
float halo = exp(-best * 12.0);
|
||||||
|
col += pal(4) * core * intensity;
|
||||||
|
col += pal(3) * halo * intensity * 0.5;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ground catches the light; below the horizon is otherwise near black.
|
||||||
|
if (p.y < horizon) {
|
||||||
|
col *= 0.25;
|
||||||
|
col += pal(2) * exp((p.y - horizon) * 5.0) * 0.15;
|
||||||
|
}
|
||||||
|
|
||||||
|
col = max(col, prev(uv) * u_afterglow);
|
||||||
|
col += sigGrain(uv);
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default stormRift;
|
||||||
111
flow-state/src/scenes/shader/synthwave-run.js
Normal file
111
flow-state/src/scenes/shader/synthwave-run.js
Normal file
@ -0,0 +1,111 @@
|
|||||||
|
// Ported from party-stage's "Synthwave Run".
|
||||||
|
//
|
||||||
|
// The original hardcoded magenta and cyan and drew a car silhouette with
|
||||||
|
// red taillights. The car is kept — it is the scene's whole identity — but the
|
||||||
|
// colours now come from the palette, and the elements (grid, mountains, sun,
|
||||||
|
// car) are individually switchable so the arc driver can strip it back to just
|
||||||
|
// the grid during a breakdown and bring the rest in for the drop.
|
||||||
|
|
||||||
|
export const synthwaveRun = {
|
||||||
|
name: 'Synthwave Run',
|
||||||
|
family: 'structural',
|
||||||
|
kind: 'fragment',
|
||||||
|
// Personality: see look/Personality.js.
|
||||||
|
traits: ['space', 'camera', 'style'],
|
||||||
|
|
||||||
|
params: {
|
||||||
|
speed: { type: 'float', range: [0.3, 4], default: 1.5, uniform: 'u_speed', bias: 'motion', rate: true },
|
||||||
|
gridDensity:{ type: 'float', range: [0.5, 3], default: 1.0, uniform: 'u_gridDensity', bias: 'density' },
|
||||||
|
horizon: { type: 'float', range: [-0.3, 0.3],default: 0.0, uniform: 'u_horizon' },
|
||||||
|
sun: { type: 'float', range: [0, 1], default: 1.0, uniform: 'u_sun' },
|
||||||
|
mountains: { type: 'float', range: [0, 1], default: 1.0, uniform: 'u_mountains' },
|
||||||
|
car: { type: 'float', range: [0, 1], default: 1.0, uniform: 'u_car' },
|
||||||
|
glow: { type: 'float', range: [0, 1.5], default: 0.5, uniform: 'u_glow', bias: 'energy' },
|
||||||
|
palette: { type: 'palette', count: 4 },
|
||||||
|
},
|
||||||
|
|
||||||
|
reactive: {
|
||||||
|
glow: { feature: 'beat', amount: 0.45, response: 'spike' },
|
||||||
|
},
|
||||||
|
|
||||||
|
shader: `
|
||||||
|
vec4 scene(vec2 uv, vec2 p) {
|
||||||
|
float t = u_time * u_speed + u_seed;
|
||||||
|
float beat = u_beat;
|
||||||
|
p = sigCamera(p);
|
||||||
|
// Shares the track's ground line with every other scene that has one.
|
||||||
|
float horizon = clamp(u_horizon + sigHorizonY() * 0.35, -0.85, 0.85);
|
||||||
|
|
||||||
|
vec3 colorMain = pal(0);
|
||||||
|
vec3 colorSec = pal(1);
|
||||||
|
vec3 col = pal(3) * 0.08;
|
||||||
|
|
||||||
|
// --- perspective ground grid ---
|
||||||
|
if (p.y < horizon) {
|
||||||
|
float perspective = 1.0 / (horizon - p.y + 0.05);
|
||||||
|
vec2 g = vec2(p.x * perspective, perspective + t * 2.0) * u_gridDensity;
|
||||||
|
float lines = smoothstep(0.05, 0.0, abs(fract(g.x + 0.5) - 0.5))
|
||||||
|
+ smoothstep(0.05, 0.0, abs(fract(g.y + 0.5) - 0.5));
|
||||||
|
col = mix(col, colorSec, lines * 0.5 * (0.5 + beat));
|
||||||
|
if (abs(p.x * perspective) < 0.8) col += colorMain * 0.15 * perspective;
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- wireframe mountains ---
|
||||||
|
if (u_mountains > 0.01 && p.y > horizon - 0.1) {
|
||||||
|
float mx = abs(p.x) - 0.5;
|
||||||
|
if (mx > 0.0) {
|
||||||
|
float h = vnoise(vec2(mx * 2.0 + t * 0.1, u_seed)) * 0.6 * smoothstep(0.0, 0.5, mx);
|
||||||
|
if (p.y - horizon < h) {
|
||||||
|
float edge = abs(p.y - horizon - h);
|
||||||
|
float wire = smoothstep(0.02, 0.0, edge)
|
||||||
|
+ smoothstep(0.01, 0.0, abs(fract((p.y - horizon) * 10.0) - 0.5))
|
||||||
|
+ smoothstep(0.01, 0.0, abs(fract(mx * 10.0) - 0.5));
|
||||||
|
col = mix(col, colorMain, wire * 0.4 * u_mountains);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- retro sun with scanline cutouts ---
|
||||||
|
if (u_sun > 0.01) {
|
||||||
|
vec2 sunPos = vec2(0.0, 0.1);
|
||||||
|
float sunRad = 0.4;
|
||||||
|
float dSun = length(p - sunPos);
|
||||||
|
if (dSun < sunRad && p.y > horizon) {
|
||||||
|
float scan = sin((p.y - t * 0.1) * 50.0);
|
||||||
|
if (scan < mix(0.8, -1.0, (p.y - sunPos.y + sunRad) / (sunRad * 2.0))) {
|
||||||
|
vec3 sunCol = mix(colorMain, pal(2), sat(p.y * 2.0));
|
||||||
|
col = mix(col, sunCol, (0.8 + beat * 0.2) * u_sun);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- car silhouette ---
|
||||||
|
if (u_car > 0.01) {
|
||||||
|
vec2 carPos = vec2(sin(t * 0.1) * 0.2, -0.78 + sin(t * 30.0) * 0.001);
|
||||||
|
vec2 cp = p - carPos;
|
||||||
|
float cpX = abs(cp.x);
|
||||||
|
|
||||||
|
float body = step(cpX, 0.35) * step(abs(cp.y), 0.12);
|
||||||
|
float glass = step(cpX, 0.22) * step(abs(cp.y - 0.18), 0.07);
|
||||||
|
float wingTop = step(cpX, 0.32) * step(abs(cp.y - 0.20), 0.025);
|
||||||
|
float wingSide = step(abs(cpX - 0.30), 0.03) * step(abs(cp.y - 0.12), 0.1);
|
||||||
|
|
||||||
|
if (body + glass + wingTop + wingSide > 0.0) {
|
||||||
|
col = mix(col, vec3(0.015), 0.95 * u_car);
|
||||||
|
float r = 0.025;
|
||||||
|
float l1 = smoothstep(r, r * 0.6, length(vec2(cpX - 0.21, cp.y - 0.03)));
|
||||||
|
float l2 = smoothstep(r, r * 0.6, length(vec2(cpX - 0.29, cp.y - 0.03)));
|
||||||
|
col = mix(col, pal(2) * (0.6 + beat * 1.4), sat(l1 + l2) * u_car);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Horizon haze, scaled by the glow param.
|
||||||
|
col += colorMain * exp(-abs(p.y - horizon) * 8.0) * u_glow * 0.35;
|
||||||
|
col += sigGrain(uv);
|
||||||
|
|
||||||
|
return vec4(col, 1.0);
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
};
|
||||||
|
|
||||||
|
export default synthwaveRun;
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user