dejvino/music-video-gen
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix a permanently-zero motion block, and let the song arrange Isometric Blocks

Browse Source 
Investigating three pieces of feedback turned up a measurement bug underneath
all of them. The gallery built four of the descriptor's five structural blocks
and never rendered a second frame, so `motion` came back 0.000 for every scene
in the library and dragged every score down by a fifth. It read as a property of
the scenes; it was a missing render. Every number in the gallery so far,
including the ones the 0.04 bar was calibrated against, was depressed by it.

    Plasma Bloom      0.0532 -> 0.0624
    Voronoi Shatter   0.0431 -> 0.0463
    Apollonian Gasket 0.0340 -> 0.0388

The Plasma Bloom versus Voronoi Shatter oddity is real and is the metric's
limit rather than a mistake. Plasma Bloom is a centred subject whose position
moves between songs, so its layout distance is 0.096; Voronoi Shatter is
edge-to-edge cells, and a full-frame texture has the same layout however its
cells fall, so its layout distance is 0.004. The descriptor cannot see "the
cells are different" as composition, and for full-frame work the layout block
contributes almost nothing. Worth knowing before the bar is used to judge that
family.

Apollonian Gasket scoring low with good-looking frames is the gallery working:
it measures how much a scene changes between songs, not how good it looks. A
scene can be beautiful six times and identical six times.

Isometric Blocks gets the displacement it was asked for. Every block sat exactly
on its lattice slot, so the plan of the field was the same plan in every song
and only the heights moved — a layout distance of 0.005. `scatter` steps each
cell off its slot by a fixed amount of its own, bounded under half a cell so
blocks do not cross and break the isometric read, and it drives the block's
height and side wall as well as its ground position. 0.034 to 0.049, clearing
the bar, with orientation variety nearly doubling as the rigid lattice softens.

Its layout distance stayed at 0.004, for the same reason Voronoi's did: the
field fills the frame either way.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Dejvino 2026-08-18 16:32:30 +02:00
parent 71ddd37711
commit 8d9b8a593a
2 changed files with 28 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
flow-state/src/checks/gallery.js
View File

@ -23,7 +23,7 @@ import { featureProviderFor } from '../audio/FeatureTrack.js';
import { songBank } from '../audio/songbank.js';
import { generateLook } from '../look/LookGenerator.js';
import { describeIdentity } from '../look/Identity.js';
import { frameDescriptor } from './variety/descriptors.js';
import { frameDescriptor, motionDescriptor } from './variety/descriptors.js';
import { descriptorDistance, STRUCTURAL } from './variety/signature.js';
const THUMB = { width: 256, height: 144 };
@ -111,9 +111,18 @@ export function renderScene(engine, module, contexts) {
// A few frames of warm-up so anything with state is past its first frame.
for (let f = ctx.frame - 6; f < ctx.frame; f++) engine.renderFrame(f);
const pixels = Uint8Array.from(engine.readPixels(engine.renderFrame(ctx.frame)));
// A second frame, so MOTION is measured rather than silently scored zero.
//
// The descriptor has five structural blocks and this only ever built
// four of them, so `motion` came back 0.000 for every scene in the
// library and dragged the mean down by a fifth across the board. It
// looked like a property of the scenes; it was a missing render.
const moved = Uint8Array.from(engine.readPixels(engine.renderFrame(ctx.frame + 5)));
thumbs.push(pixels);
descriptors.push(frameDescriptor(pixels, THUMB.width, THUMB.height));
const still = frameDescriptor(pixels, THUMB.width, THUMB.height);
const motion = motionDescriptor(pixels, moved, THUMB.width, THUMB.height);
descriptors.push({ ...still, motion: motion.scale.concat(motion.layout) });
}
// How different this scene's six outputs are from each other, on the same

20
flow-state/src/scenes/shader/isometric-blocks.js
View File

@ -19,6 +19,14 @@ export const isometricBlocks = {
params: {
grid: { type: 'float', range: [2, 10], default: 4.5, uniform: 'u_grid', bias: 'density', slowAxis: true },
height: { type: 'float', range: [0.1, 0.7], default: 0.35, uniform: 'u_height', bias: 'energy' },
// Displacement: how far each block wanders from its slot on the grid.
//
// Every block sat exactly on the lattice, so the plan of the field was
// the same plan in every song and only the heights moved — which is
// what the gallery was reporting as a layout distance of 0.005, near
// enough to nothing. Displacing them per cell puts the SONG in the
// arrangement rather than only in the elevation.
scatter: { type: 'float', range: [0, 0.85], default: 0.3, uniform: 'u_scatter', bias: 'density' },
plan: { type: 'float', range: [0.2, 0.5], default: 0.4, uniform: 'u_plan' },
tilt: { type: 'float', range: [0.35, 0.75], default: 0.55, uniform: 'u_isoTilt' },
light: { type: 'float', range: [0, 1.4], default: 0.65, uniform: 'u_light' },
@ -80,12 +88,18 @@ vec4 scene(vec2 uv, vec2 p) {
float row = rowHere - float(7 - j);
vec2 cell = vec2(colX, row);
float groundY = (row + stagger) * tilt * cw; // where this cell meets the ground
float h = columnHeight(cell, t);
// Each cell steps off its slot by a fixed amount of its own, so the
// plan of the field is the song's rather than the lattice's. Bounded
// under half a cell: past that blocks cross each other and the
// isometric read — which is the whole point of the scene — breaks down.
vec2 offset = (hash22(cell + u_seed + 3.7) - 0.5) * u_scatter * cw * 0.9;
float groundY = (row + stagger) * tilt * cw + offset.y; // where this cell meets the ground
float h = columnHeight(cell, t) * (1.0 + offset.x * 2.0);
float dy = p.y - groundY; // height up the block face
// Side wall: the plan swept up from the ground line to the top face.
float inX = smoothstep(u_plan * cw, u_plan * cw * 0.85, abs(fx));
float inX = smoothstep(u_plan * cw, u_plan * cw * 0.85, abs(fx - offset.x));
float wall = inX * step(0.0, dy) * step(dy, h);
float lit = 0.5 + 0.5 * sat(fx / (u_plan * cw) * 0.8 + 0.5);