mirror of
https://github.com/hanzoai/frames.git
synced 2026-09-24 00:08:03 +00:00
fix: resolve merge conflict in shader-transitions capture.ts
Merges main's refactored capture (CaptureSceneOptions, forceVisible, stabilizeTransformedBoxShadows, foreignObjectRendering fallback) with our HTML-in-Canvas drawElementImage capture path. The native capture tries first and falls back to html2canvas on failure.
This commit is contained in:
Miguel Ángel
@@ -80,6 +80,71 @@ npx hyperframes remove-background subject.mp4 -o transparent.mov # editi
|
||||
npx hyperframes remove-background portrait.jpg -o cutout.png # still image
|
||||
```
|
||||
|
||||
## Layer separation: emit the cutout and the background plate together
|
||||
|
||||
Pass `--background-output` (alias `-b`) to write a *second* transparent video alongside the cutout. Same source RGB, alpha is the *inverse* mask — opaque where the surroundings were, transparent where the subject is. The result is a clean two-layer separation in a single inference pass:
|
||||
|
||||
```bash Terminal
|
||||
npx hyperframes remove-background subject.mp4 \
|
||||
-o subject.webm \
|
||||
--background-output plate.webm
|
||||
```
|
||||
|
||||
| Output | Alpha | Use it as |
|
||||
| ------ | ----- | --------- |
|
||||
| `subject.webm` | Mask — subject opaque | Foreground layer (top of stack) |
|
||||
| `plate.webm` | `255 − mask` — subject region transparent | Background layer; place anything you want **under the subject's silhouette** between this and `subject.webm` |
|
||||
|
||||
Both encoders share the source W/H/fps and your `--quality` preset, so the layers are pixel-aligned. Encode cost roughly doubles; segmentation cost is unchanged.
|
||||
|
||||
<Tip>
|
||||
**This is a hole-cut plate, not an inpainted clean plate.** The subject region in `plate.webm` is fully transparent — you have to composite something opaque under it (a graphic, a blurred copy, a different scene) to fill the hole. If you need an actual filled background where the subject was, use a video inpainter (LaMa, ProPainter, RunwayML Inpaint) — `remove-background` is not the right tool for that.
|
||||
</Tip>
|
||||
|
||||
### Hole-cut vs. clean plate — when does the difference matter?
|
||||
|
||||
A **hole-cut plate** keeps the original surroundings and makes the subject region transparent. A **clean plate** fills the subject region with reconstructed background — produced by a separate inpainting model. Display each alone over black:
|
||||
|
||||
| | Hole-cut plate (this command) | Clean plate (inpainted) |
|
||||
| --- | --- | --- |
|
||||
| Subject region | Transparent silhouette | Reconstructed background pixels |
|
||||
| What you see alone | A person-shaped hole | An empty room |
|
||||
| Cost | One inference pass, one extra ffmpeg encode | A second model (LaMa, ProPainter, E2FGVI) |
|
||||
| Tool | `remove-background --background-output` | Outside this CLI |
|
||||
|
||||
The line is: **does anything ever need to be visible *through* the subject's silhouette where the subject used to be?**
|
||||
|
||||
| Use case | What you need |
|
||||
| --- | --- |
|
||||
| Text/graphics live *between* the cutout and the plate (the example above) | **Hole-cut** — the graphics fill the hole. |
|
||||
| Composite the subject onto an unrelated scene | Neither. Just use `subject.webm`; the plate is irrelevant. |
|
||||
| Show "the room without the person" as a real background | **Clean plate** — a hole-cut plate would show a transparent void. |
|
||||
| Replace the person with a different subject (re-target) | **Clean plate** — the new subject needs real pixels under it. |
|
||||
| VFX rotoscoping / "remove an extra from this take" | **Clean plate** — the canonical inpainting use case. |
|
||||
|
||||
If something opaque always covers the silhouette, hole-cut is sufficient and ~1000× cheaper than running an inpainter.
|
||||
|
||||
### The two-layer composition pattern
|
||||
|
||||
The two-layer pattern is functionally a drop-in for [text-behind-subject](#text-behind-subject-the-recommended-layout) without needing the original `presenter.mp4` in the project — the plate replaces it as the bottom layer:
|
||||
|
||||
```html
|
||||
<!-- z=1 inverse-alpha plate fills everything except the subject's silhouette -->
|
||||
<video src="plate.webm" data-start="0" data-duration="6" data-track-index="0" muted playsinline></video>
|
||||
|
||||
<!-- z=2 anything you want occluded by the subject lives here -->
|
||||
<h1 style="z-index:2; position:absolute; top:50%; left:50%; transform:translate(-50%,-50%);">
|
||||
MAKE IT IN HYPERFRAMES
|
||||
</h1>
|
||||
|
||||
<!-- z=3 the cutout puts the subject back on top -->
|
||||
<div class="cutout-wrap" style="position:absolute;inset:0;z-index:3">
|
||||
<video src="subject.webm" data-start="0" data-duration="6" data-track-index="1" muted playsinline></video>
|
||||
</div>
|
||||
```
|
||||
|
||||
Constraints: the flag requires a video input and `.webm` or `.mov` for both outputs. It's not valid for image inputs (no temporal pairing to do) and won't accept `.png` for the plate.
|
||||
|
||||
## Performance
|
||||
|
||||
Real-world numbers from the [matting eval](https://www.heygenverse.com/a/0dd5a431-1832-4858-862d-de7fb7d02654), running u²-net_human_seg on a 4-second 1080p clip:
|
||||
|
||||
@@ -356,6 +356,10 @@ This is suppressed in CI environments, non-TTY shells, and when `HYPERFRAMES_NO_
|
||||
# Single image → transparent PNG
|
||||
npx hyperframes remove-background portrait.jpg -o cutout.png
|
||||
|
||||
# Layer separation: cutout AND inverse-alpha background plate in one pass
|
||||
npx hyperframes remove-background avatar.mp4 \
|
||||
-o subject.webm --background-output plate.webm
|
||||
|
||||
# Force CPU on a machine that has CoreML or CUDA
|
||||
npx hyperframes remove-background avatar.mp4 -o transparent.webm --device cpu
|
||||
|
||||
@@ -366,8 +370,9 @@ This is suppressed in CI environments, non-TTY shells, and when `HYPERFRAMES_NO_
|
||||
| Flag | Description |
|
||||
|------|-------------|
|
||||
| `--output, -o` | Output path. Format inferred from extension: `.webm` (default), `.mov`, `.png` |
|
||||
| `--background-output, -b` | Optional second output: inverse-alpha background plate (subject region transparent, surroundings opaque). Same source RGB, complementary mask. Must be `.webm` or `.mov`. Hole-cut, not inpainted — composite something underneath to fill the hole. |
|
||||
| `--device` | Execution provider: `auto` (default), `cpu`, `coreml`, `cuda` |
|
||||
| `--quality` | WebM encoder preset: `fast` (crf 30, smallest), `balanced` (crf 18, default), `best` (crf 12, near-lossless). Higher quality keeps the cutout's RGB closer to the source mp4 — important when overlaying the cutout on its own source for text-behind-subject effects. Ignored for `.mov` / `.png`. |
|
||||
| `--quality` | WebM encoder preset: `fast` (crf 30, smallest), `balanced` (crf 18, default), `best` (crf 12, near-lossless). Higher quality keeps the cutout's RGB closer to the source mp4 — important when overlaying the cutout on its own source for text-behind-subject effects. Applies to both `--output` and `--background-output`. Ignored for `.mov` / `.png`. |
|
||||
| `--info` | Print detected execution providers and exit (no render) |
|
||||
| `--json` | Output result as JSON |
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { MEAN, STD } from "./inference.js";
|
||||
import { MEAN, STD, applyMask } from "./inference.js";
|
||||
|
||||
// Regression: the u2net_human_seg model was trained with ImageNet
|
||||
// normalization. Drifting away from these exact values changes the input
|
||||
@@ -16,3 +16,112 @@ describe("background-removal/inference — rembg u2net_human_seg parity", () =>
|
||||
expect(STD).toEqual([0.229, 0.224, 0.225]);
|
||||
});
|
||||
});
|
||||
|
||||
// These tests pin the contract that `--background-output` is built on:
|
||||
// fg.alpha + bg.alpha === 255 per pixel, and the RGB plane is byte-identical
|
||||
// between fg and bg. A future change to the postprocess loop (different mask
|
||||
// threshold, premultiplied alpha, gamma-corrected compositing) that breaks
|
||||
// either invariant should fail here loudly.
|
||||
describe("background-removal/inference — applyMask invariants", () => {
|
||||
function makeRgb(pixels: number): Buffer {
|
||||
// Deterministic but non-trivial RGB so byte equality is meaningful.
|
||||
const buf = Buffer.allocUnsafe(pixels * 3);
|
||||
for (let i = 0; i < pixels; i++) {
|
||||
buf[i * 3] = (i * 7) & 0xff;
|
||||
buf[i * 3 + 1] = (i * 13 + 31) & 0xff;
|
||||
buf[i * 3 + 2] = (i * 19 + 61) & 0xff;
|
||||
}
|
||||
return buf;
|
||||
}
|
||||
|
||||
function makeMask(pixels: number): Buffer {
|
||||
// Hit the saturation endpoints (0, 255) and a few mid-tone values so the
|
||||
// 255-m inversion is exercised across the full byte range.
|
||||
const buf = Buffer.allocUnsafe(pixels);
|
||||
for (let i = 0; i < pixels; i++) buf[i] = (i * 37) & 0xff;
|
||||
return buf;
|
||||
}
|
||||
|
||||
it("dual-output: fg.alpha + bg.alpha === 255 for every pixel", () => {
|
||||
const pixels = 64;
|
||||
const rgb = makeRgb(pixels);
|
||||
const mask = makeMask(pixels);
|
||||
const fg = Buffer.allocUnsafe(pixels * 4);
|
||||
const bg = Buffer.allocUnsafe(pixels * 4);
|
||||
|
||||
const result = applyMask(rgb, mask, fg, bg, pixels);
|
||||
|
||||
expect(result.fg).toBe(fg);
|
||||
expect(result.bg).toBe(bg);
|
||||
for (let i = 0; i < pixels; i++) {
|
||||
const sum = fg[i * 4 + 3]! + bg[i * 4 + 3]!;
|
||||
expect(sum).toBe(255);
|
||||
}
|
||||
});
|
||||
|
||||
it("dual-output: RGB triples are byte-identical between fg and bg", () => {
|
||||
const pixels = 64;
|
||||
const rgb = makeRgb(pixels);
|
||||
const mask = makeMask(pixels);
|
||||
const fg = Buffer.allocUnsafe(pixels * 4);
|
||||
const bg = Buffer.allocUnsafe(pixels * 4);
|
||||
|
||||
applyMask(rgb, mask, fg, bg, pixels);
|
||||
|
||||
for (let i = 0; i < pixels; i++) {
|
||||
expect(fg[i * 4]).toBe(bg[i * 4]);
|
||||
expect(fg[i * 4 + 1]).toBe(bg[i * 4 + 1]);
|
||||
expect(fg[i * 4 + 2]).toBe(bg[i * 4 + 2]);
|
||||
// And both match the source.
|
||||
expect(fg[i * 4]).toBe(rgb[i * 3]);
|
||||
expect(fg[i * 4 + 1]).toBe(rgb[i * 3 + 1]);
|
||||
expect(fg[i * 4 + 2]).toBe(rgb[i * 3 + 2]);
|
||||
}
|
||||
});
|
||||
|
||||
it("dual-output: fg.alpha equals the input mask", () => {
|
||||
const pixels = 32;
|
||||
const rgb = makeRgb(pixels);
|
||||
const mask = makeMask(pixels);
|
||||
const fg = Buffer.allocUnsafe(pixels * 4);
|
||||
const bg = Buffer.allocUnsafe(pixels * 4);
|
||||
|
||||
applyMask(rgb, mask, fg, bg, pixels);
|
||||
|
||||
for (let i = 0; i < pixels; i++) {
|
||||
expect(fg[i * 4 + 3]).toBe(mask[i]);
|
||||
}
|
||||
});
|
||||
|
||||
it("single-output: bg=null returns bg=null and writes only fg", () => {
|
||||
const pixels = 32;
|
||||
const rgb = makeRgb(pixels);
|
||||
const mask = makeMask(pixels);
|
||||
const fg = Buffer.allocUnsafe(pixels * 4);
|
||||
|
||||
const result = applyMask(rgb, mask, fg, null, pixels);
|
||||
|
||||
expect(result.bg).toBeNull();
|
||||
expect(result.fg).toBe(fg);
|
||||
for (let i = 0; i < pixels; i++) {
|
||||
expect(fg[i * 4]).toBe(rgb[i * 3]);
|
||||
expect(fg[i * 4 + 3]).toBe(mask[i]);
|
||||
}
|
||||
});
|
||||
|
||||
it("saturates correctly at mask=0 and mask=255", () => {
|
||||
// mask=0 → fg.alpha=0 (transparent subject), bg.alpha=255 (fully opaque plate)
|
||||
// mask=255 → fg.alpha=255 (fully opaque subject), bg.alpha=0 (transparent plate)
|
||||
const rgb = Buffer.from([10, 20, 30, 40, 50, 60]);
|
||||
const mask = Buffer.from([0, 255]);
|
||||
const fg = Buffer.allocUnsafe(8);
|
||||
const bg = Buffer.allocUnsafe(8);
|
||||
|
||||
applyMask(rgb, mask, fg, bg, 2);
|
||||
|
||||
expect(fg[3]).toBe(0);
|
||||
expect(bg[3]).toBe(255);
|
||||
expect(fg[7]).toBe(255);
|
||||
expect(bg[7]).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -24,10 +24,24 @@ interface OrtModule {
|
||||
Tensor: typeof Tensor;
|
||||
}
|
||||
|
||||
export interface SessionResult {
|
||||
/** Subject opaque, background fully transparent. */
|
||||
fg: Buffer;
|
||||
/** Inverse-alpha plate: same RGB, alpha is `255 − mask`. Null unless `withBackground` was true. */
|
||||
bg: Buffer | null;
|
||||
}
|
||||
|
||||
export interface Session {
|
||||
/** Run inference on one RGB frame, return RGBA bytes (H*W*4). */
|
||||
process(rgb: Buffer, width: number, height: number): Promise<Buffer>;
|
||||
/** ORT EP that was actually selected. */
|
||||
/**
|
||||
* Both `fg` and `bg` (when requested) are session-owned buffers reused on the
|
||||
* next call — drain the encoder's stdin before invoking `process` again.
|
||||
*/
|
||||
process(
|
||||
rgb: Buffer,
|
||||
width: number,
|
||||
height: number,
|
||||
withBackground?: boolean,
|
||||
): Promise<SessionResult>;
|
||||
provider: string;
|
||||
close(): Promise<void>;
|
||||
}
|
||||
@@ -73,16 +87,15 @@ export async function createSession(options: CreateSessionOptions = {}): Promise
|
||||
throw new Error("ONNX session is missing input or output bindings");
|
||||
}
|
||||
|
||||
// Pre-allocated per-frame buffers reused across every process() call.
|
||||
// At 1080p this saves ~9 MB of allocations per frame. rgbaBuf is sized
|
||||
// lazily on the first call (we don't know W/H until then).
|
||||
// Reused across calls; sized lazily on first frame. Saves ~9 MB/frame at 1080p.
|
||||
const inputData = new Float32Array(3 * INPUT_PLANE);
|
||||
const maskBuf = Buffer.allocUnsafe(INPUT_PLANE);
|
||||
let rgbaBuf: Buffer | null = null;
|
||||
let rgbaBgBuf: Buffer | null = null;
|
||||
|
||||
return {
|
||||
provider: providerUsed,
|
||||
async process(rgb, width, height) {
|
||||
async process(rgb, width, height, withBackground = false) {
|
||||
const tensor = await preprocess(sharp, ort, rgb, width, height, inputData);
|
||||
const outputs = await session.run({ [inputName]: tensor });
|
||||
const output = outputs[outputName];
|
||||
@@ -91,7 +104,21 @@ export async function createSession(options: CreateSessionOptions = {}): Promise
|
||||
if (!rgbaBuf || rgbaBuf.length !== expectedBytes) {
|
||||
rgbaBuf = Buffer.allocUnsafe(expectedBytes);
|
||||
}
|
||||
return await postprocess(sharp, output, rgb, width, height, maskBuf, rgbaBuf);
|
||||
if (withBackground) {
|
||||
if (!rgbaBgBuf || rgbaBgBuf.length !== expectedBytes) {
|
||||
rgbaBgBuf = Buffer.allocUnsafe(expectedBytes);
|
||||
}
|
||||
}
|
||||
return await postprocess(
|
||||
sharp,
|
||||
output,
|
||||
rgb,
|
||||
width,
|
||||
height,
|
||||
maskBuf,
|
||||
rgbaBuf,
|
||||
withBackground ? rgbaBgBuf : null,
|
||||
);
|
||||
},
|
||||
async close() {
|
||||
await session.release();
|
||||
@@ -141,7 +168,8 @@ async function postprocess(
|
||||
height: number,
|
||||
maskBuf: Buffer,
|
||||
rgbaBuf: Buffer,
|
||||
): Promise<Buffer> {
|
||||
rgbaBgBuf: Buffer | null,
|
||||
): Promise<SessionResult> {
|
||||
const raw = output.data as Float32Array;
|
||||
|
||||
let lo = Infinity;
|
||||
@@ -172,11 +200,50 @@ async function postprocess(
|
||||
.raw()
|
||||
.toBuffer();
|
||||
|
||||
for (let i = 0; i < width * height; i++) {
|
||||
rgbaBuf[i * 4] = rgb[i * 3]!;
|
||||
rgbaBuf[i * 4 + 1] = rgb[i * 3 + 1]!;
|
||||
rgbaBuf[i * 4 + 2] = rgb[i * 3 + 2]!;
|
||||
rgbaBuf[i * 4 + 3] = fullMask[i]!;
|
||||
return applyMask(rgb, fullMask, rgbaBuf, rgbaBgBuf, width * height);
|
||||
}
|
||||
|
||||
/**
|
||||
* Composite the RGB source frame with the segmentation mask into one or two
|
||||
* RGBA buffers. The contract this PR is built on:
|
||||
* - `fg`'s alpha is the mask, `bg`'s alpha (when provided) is `255 − mask`,
|
||||
* so `fg.alpha + bg.alpha === 255` for every pixel.
|
||||
* - RGB triples are byte-identical between `fg` and `bg`.
|
||||
* - When `bg` is null, only `fg` is touched.
|
||||
*
|
||||
* Exported for direct unit testing of the invariants above without spinning
|
||||
* up an ONNX session.
|
||||
*/
|
||||
export function applyMask(
|
||||
rgb: Buffer,
|
||||
mask: Buffer,
|
||||
fg: Buffer,
|
||||
bg: Buffer | null,
|
||||
pixels: number,
|
||||
): SessionResult {
|
||||
if (bg) {
|
||||
for (let i = 0; i < pixels; i++) {
|
||||
const r = rgb[i * 3]!;
|
||||
const g = rgb[i * 3 + 1]!;
|
||||
const b = rgb[i * 3 + 2]!;
|
||||
const m = mask[i]!;
|
||||
const o = i * 4;
|
||||
fg[o] = r;
|
||||
fg[o + 1] = g;
|
||||
fg[o + 2] = b;
|
||||
fg[o + 3] = m;
|
||||
bg[o] = r;
|
||||
bg[o + 1] = g;
|
||||
bg[o + 2] = b;
|
||||
bg[o + 3] = 255 - m;
|
||||
}
|
||||
return { fg, bg };
|
||||
}
|
||||
return rgbaBuf;
|
||||
for (let i = 0; i < pixels; i++) {
|
||||
fg[i * 4] = rgb[i * 3]!;
|
||||
fg[i * 4 + 1] = rgb[i * 3 + 1]!;
|
||||
fg[i * 4 + 2] = rgb[i * 3 + 2]!;
|
||||
fg[i * 4 + 3] = mask[i]!;
|
||||
}
|
||||
return { fg, bg: null };
|
||||
}
|
||||
|
||||
@@ -1,7 +1,13 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { EventEmitter } from "node:events";
|
||||
import type { spawn } from "node:child_process";
|
||||
import { inferOutputFormat, inferInputKind, buildEncoderArgs, waitForExit } from "./pipeline.js";
|
||||
import {
|
||||
inferOutputFormat,
|
||||
inferInputKind,
|
||||
buildEncoderArgs,
|
||||
resolveRenderTargets,
|
||||
waitForExit,
|
||||
} from "./pipeline.js";
|
||||
|
||||
describe("background-removal/pipeline — inferOutputFormat", () => {
|
||||
it("maps .webm → webm", () => {
|
||||
@@ -97,6 +103,54 @@ describe("background-removal/pipeline — buildEncoderArgs", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("background-removal/pipeline — resolveRenderTargets", () => {
|
||||
it("resolves a normal video → webm render", () => {
|
||||
const t = resolveRenderTargets("/tmp/clip.mp4", "/tmp/cutout.webm");
|
||||
expect(t.format).toBe("webm");
|
||||
expect(t.inputKind).toBe("video");
|
||||
expect(t.bgFormat).toBeUndefined();
|
||||
});
|
||||
|
||||
it("resolves an image → png render", () => {
|
||||
const t = resolveRenderTargets("/tmp/portrait.jpg", "/tmp/cutout.png");
|
||||
expect(t.format).toBe("png");
|
||||
expect(t.inputKind).toBe("image");
|
||||
});
|
||||
|
||||
it("rejects image input with a video output extension", () => {
|
||||
expect(() => resolveRenderTargets("/tmp/portrait.jpg", "/tmp/cutout.webm")).toThrow(
|
||||
/Image input requires a \.png output/,
|
||||
);
|
||||
});
|
||||
|
||||
it("rejects video input with a .png output", () => {
|
||||
expect(() => resolveRenderTargets("/tmp/clip.mp4", "/tmp/cutout.png")).toThrow(
|
||||
/Video input requires a \.webm or \.mov output/,
|
||||
);
|
||||
});
|
||||
|
||||
it("threads background-output format through when valid", () => {
|
||||
const t = resolveRenderTargets("/tmp/clip.mp4", "/tmp/fg.webm", "/tmp/bg.webm");
|
||||
expect(t.bgFormat).toBe("webm");
|
||||
const tMov = resolveRenderTargets("/tmp/clip.mp4", "/tmp/fg.webm", "/tmp/bg.mov");
|
||||
expect(tMov.bgFormat).toBe("mov");
|
||||
});
|
||||
|
||||
it("rejects --background-output for image inputs (no temporal pairing to do)", () => {
|
||||
expect(() =>
|
||||
resolveRenderTargets("/tmp/portrait.jpg", "/tmp/cutout.png", "/tmp/bg.png"),
|
||||
).toThrow(/--background-output is not supported for image inputs/);
|
||||
});
|
||||
|
||||
it("rejects .png as the --background-output extension", () => {
|
||||
// .png is only valid for single-image inputs, and image inputs themselves
|
||||
// can't have a background-output anyway. So .png here is always a misuse.
|
||||
expect(() => resolveRenderTargets("/tmp/clip.mp4", "/tmp/fg.webm", "/tmp/bg.png")).toThrow(
|
||||
/--background-output must be \.webm or \.mov/,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// Regression: a previous version of waitForExit treated `code === null` as
|
||||
// success. Per Node's child_process docs, that's the signal-killed case —
|
||||
// reporting it as success means a SIGTERM/SIGKILL'd ffmpeg encoder produces
|
||||
|
||||
@@ -38,6 +38,16 @@ export const isQuality = (v: unknown): v is Quality =>
|
||||
export interface RenderOptions {
|
||||
inputPath: string;
|
||||
outputPath: string;
|
||||
/**
|
||||
* Optional second output: an inverse-alpha background plate (same source
|
||||
* RGB, transparent where the subject was). Only valid for video inputs and
|
||||
* .webm/.mov outputs — not allowed alongside a .png output. The plate's
|
||||
* format is inferred from this path independently of the foreground's.
|
||||
*
|
||||
* NOTE: this is a hole-cut plate, not an inpainted clean plate. Composite
|
||||
* something opaque (graphics, blur, scene) under it to fill the hole.
|
||||
*/
|
||||
backgroundOutputPath?: string;
|
||||
device?: Device;
|
||||
model?: ModelId;
|
||||
/** Encoder CRF preset for `.webm`. See `QUALITY_CRF`. Ignored for `.mov`/`.png`. */
|
||||
@@ -52,6 +62,8 @@ export type ProgressEvent =
|
||||
|
||||
export interface RenderResult {
|
||||
outputPath: string;
|
||||
/** Present only when `backgroundOutputPath` was set. */
|
||||
backgroundOutputPath?: string;
|
||||
framesProcessed: number;
|
||||
durationSeconds: number;
|
||||
avgMsPerFrame: number;
|
||||
@@ -203,17 +215,27 @@ async function* readFrames(
|
||||
}
|
||||
}
|
||||
|
||||
export async function render(options: RenderOptions): Promise<RenderResult> {
|
||||
if (!hasFFmpeg() || !hasFFprobe()) {
|
||||
throw new Error("ffmpeg and ffprobe are required. Install: brew install ffmpeg");
|
||||
}
|
||||
export interface RenderTargets {
|
||||
format: OutputFormat;
|
||||
inputKind: "video" | "image";
|
||||
bgFormat: OutputFormat | undefined;
|
||||
}
|
||||
|
||||
const format = inferOutputFormat(options.outputPath);
|
||||
const inputKind = inferInputKind(options.inputPath);
|
||||
/**
|
||||
* Resolve and validate the input/output combination before any I/O. Pure;
|
||||
* exported so unit tests can pin the error messages without spawning ffmpeg.
|
||||
*/
|
||||
export function resolveRenderTargets(
|
||||
inputPath: string,
|
||||
outputPath: string,
|
||||
backgroundOutputPath?: string,
|
||||
): RenderTargets {
|
||||
const format = inferOutputFormat(outputPath);
|
||||
const inputKind = inferInputKind(inputPath);
|
||||
|
||||
if (inputKind === "image" && format !== "png") {
|
||||
throw new Error(
|
||||
`Image input requires a .png output (got ${extname(options.outputPath)}). Use a video input for .webm/.mov.`,
|
||||
`Image input requires a .png output (got ${extname(outputPath)}). Use a video input for .webm/.mov.`,
|
||||
);
|
||||
}
|
||||
if (inputKind === "video" && format === "png") {
|
||||
@@ -222,6 +244,35 @@ export async function render(options: RenderOptions): Promise<RenderResult> {
|
||||
);
|
||||
}
|
||||
|
||||
let bgFormat: OutputFormat | undefined;
|
||||
if (backgroundOutputPath) {
|
||||
if (inputKind === "image") {
|
||||
throw new Error(
|
||||
"--background-output is not supported for image inputs. Use a video input (mp4/mov/webm) to produce both a cutout and a background plate.",
|
||||
);
|
||||
}
|
||||
bgFormat = inferOutputFormat(backgroundOutputPath);
|
||||
if (bgFormat === "png") {
|
||||
throw new Error(
|
||||
"--background-output must be .webm or .mov; .png is only valid for single-image inputs.",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
return { format, inputKind, bgFormat };
|
||||
}
|
||||
|
||||
export async function render(options: RenderOptions): Promise<RenderResult> {
|
||||
if (!hasFFmpeg() || !hasFFprobe()) {
|
||||
throw new Error("ffmpeg and ffprobe are required. Install: brew install ffmpeg");
|
||||
}
|
||||
|
||||
const { format, bgFormat } = resolveRenderTargets(
|
||||
options.inputPath,
|
||||
options.outputPath,
|
||||
options.backgroundOutputPath,
|
||||
);
|
||||
|
||||
const media = await probeMedia(options.inputPath);
|
||||
|
||||
options.onProgress?.({
|
||||
@@ -240,12 +291,13 @@ export async function render(options: RenderOptions): Promise<RenderResult> {
|
||||
|
||||
try {
|
||||
const start = Date.now();
|
||||
const framesProcessed = await runPipeline(options, session, media, format);
|
||||
const framesProcessed = await runPipeline(options, session, media, format, bgFormat);
|
||||
const durationSeconds = (Date.now() - start) / 1000;
|
||||
const avgMsPerFrame = framesProcessed ? (durationSeconds * 1000) / framesProcessed : 0;
|
||||
|
||||
return {
|
||||
outputPath: options.outputPath,
|
||||
backgroundOutputPath: options.backgroundOutputPath,
|
||||
framesProcessed,
|
||||
durationSeconds,
|
||||
avgMsPerFrame,
|
||||
@@ -259,61 +311,77 @@ export async function render(options: RenderOptions): Promise<RenderResult> {
|
||||
|
||||
const RECENT_WINDOW = 30;
|
||||
|
||||
interface FfmpegProc {
|
||||
proc: ReturnType<typeof spawn>;
|
||||
exit: Promise<void>;
|
||||
/** Tail of stderr, captured for inclusion in error messages. */
|
||||
getStderr: () => string;
|
||||
}
|
||||
|
||||
type StdioFd = "ignore" | "pipe";
|
||||
type StdioTuple = [StdioFd, StdioFd, StdioFd];
|
||||
|
||||
function spawnFfmpeg(args: string[], label: string, stdio: StdioTuple): FfmpegProc {
|
||||
const proc = spawn("ffmpeg", args, { stdio });
|
||||
let stderrBuf = "";
|
||||
proc.stderr?.on("data", (d: Buffer) => {
|
||||
stderrBuf += d.toString();
|
||||
});
|
||||
// If the encoder dies mid-render, the next .write() to its stdin emits an
|
||||
// 'error' event on the writable. Without a listener, Node treats it as
|
||||
// unhandled and crashes the CLI before waitForExit's reject path can
|
||||
// surface the real cause (encoder stderr tail). Swallowing here is safe —
|
||||
// the process exit is the source of truth.
|
||||
proc.stdin?.on("error", () => {});
|
||||
const exit = waitForExit(proc, label, () => stderrBuf);
|
||||
return { proc, exit, getStderr: () => stderrBuf };
|
||||
}
|
||||
|
||||
async function runPipeline(
|
||||
options: RenderOptions,
|
||||
session: Session,
|
||||
media: MediaInfo,
|
||||
format: OutputFormat,
|
||||
bgFormat: OutputFormat | undefined,
|
||||
): Promise<number> {
|
||||
const { inputPath, outputPath } = options;
|
||||
const { inputPath, outputPath, backgroundOutputPath } = options;
|
||||
const { width, height, fps, frameCount } = media;
|
||||
const frameBytes = width * height * 3;
|
||||
const quality = options.quality ?? DEFAULT_QUALITY;
|
||||
|
||||
const decoder = spawn(
|
||||
"ffmpeg",
|
||||
const decoder = spawnFfmpeg(
|
||||
["-loglevel", "error", "-i", inputPath, "-f", "rawvideo", "-pix_fmt", "rgb24", "-an", "-"],
|
||||
{ stdio: ["ignore", "pipe", "pipe"] },
|
||||
"ffmpeg decoder",
|
||||
["ignore", "pipe", "pipe"],
|
||||
);
|
||||
|
||||
let decoderStderr = "";
|
||||
decoder.stderr?.on("data", (d: Buffer) => {
|
||||
decoderStderr += d.toString();
|
||||
});
|
||||
const decoderExit = waitForExit(decoder, "ffmpeg decoder", () => decoderStderr);
|
||||
|
||||
const encoder = spawn(
|
||||
"ffmpeg",
|
||||
buildEncoderArgs(
|
||||
format,
|
||||
width,
|
||||
height,
|
||||
fps || 30,
|
||||
outputPath,
|
||||
options.quality ?? DEFAULT_QUALITY,
|
||||
),
|
||||
{
|
||||
stdio: ["pipe", "ignore", "pipe"],
|
||||
},
|
||||
const fg = spawnFfmpeg(
|
||||
buildEncoderArgs(format, width, height, fps || 30, outputPath, quality),
|
||||
"ffmpeg encoder",
|
||||
["pipe", "ignore", "pipe"],
|
||||
);
|
||||
let encoderStderr = "";
|
||||
encoder.stderr?.on("data", (d: Buffer) => {
|
||||
encoderStderr += d.toString();
|
||||
});
|
||||
const encoderExit = waitForExit(encoder, "ffmpeg encoder", () => encoderStderr);
|
||||
|
||||
const bg =
|
||||
backgroundOutputPath && bgFormat
|
||||
? spawnFfmpeg(
|
||||
buildEncoderArgs(bgFormat, width, height, fps || 30, backgroundOutputPath, quality),
|
||||
"ffmpeg background encoder",
|
||||
["pipe", "ignore", "pipe"],
|
||||
)
|
||||
: null;
|
||||
|
||||
let processed = 0;
|
||||
const total = frameCount;
|
||||
|
||||
// Running average over the last RECENT_WINDOW frames.
|
||||
const recentMs = new Array<number>(RECENT_WINDOW).fill(0);
|
||||
let recentSum = 0;
|
||||
let recentSlot = 0;
|
||||
let recentCount = 0;
|
||||
|
||||
try {
|
||||
for await (const rgb of readFrames(decoder.stdout!, frameBytes)) {
|
||||
for await (const rgb of readFrames(decoder.proc.stdout!, frameBytes)) {
|
||||
const t0 = Date.now();
|
||||
const rgba = await session.process(rgb, width, height);
|
||||
const result = await session.process(rgb, width, height, bg !== null);
|
||||
const elapsed = Date.now() - t0;
|
||||
|
||||
recentSum += elapsed - recentMs[recentSlot]!;
|
||||
@@ -321,8 +389,31 @@ async function runPipeline(
|
||||
recentSlot = (recentSlot + 1) % RECENT_WINDOW;
|
||||
if (recentCount < RECENT_WINDOW) recentCount++;
|
||||
|
||||
if (!encoder.stdin!.write(rgba)) {
|
||||
await new Promise<void>((resolve) => encoder.stdin!.once("drain", () => resolve()));
|
||||
// Issue both writes before any await so a slow encoder doesn't block
|
||||
// the other. Drain anything that returned false before the next
|
||||
// session.process() — its output buffers are reused per frame.
|
||||
//
|
||||
// Subtlety: write() returning true means "highWaterMark not exceeded,"
|
||||
// NOT "libuv has flushed the chunk." The buffer reference is held by
|
||||
// libuv until the underlying syscall completes. Reusing the session's
|
||||
// output buffer is safe because the next session.process() call takes
|
||||
// ~10–50ms (ORT inference) — plenty of event-loop turns for libuv to
|
||||
// drain. If that ever stops being true, we'd need to copy here.
|
||||
const fgWroteFully = fg.proc.stdin!.write(result.fg);
|
||||
const bgWroteFully = bg && result.bg ? bg.proc.stdin!.write(result.bg) : true;
|
||||
if (!fgWroteFully || !bgWroteFully) {
|
||||
const drains: Promise<void>[] = [];
|
||||
if (!fgWroteFully) {
|
||||
drains.push(
|
||||
new Promise<void>((resolve) => fg.proc.stdin!.once("drain", () => resolve())),
|
||||
);
|
||||
}
|
||||
if (!bgWroteFully && bg) {
|
||||
drains.push(
|
||||
new Promise<void>((resolve) => bg.proc.stdin!.once("drain", () => resolve())),
|
||||
);
|
||||
}
|
||||
await Promise.all(drains);
|
||||
}
|
||||
|
||||
processed++;
|
||||
@@ -334,17 +425,21 @@ async function runPipeline(
|
||||
});
|
||||
}
|
||||
} catch (err) {
|
||||
decoder.kill("SIGKILL");
|
||||
encoder.kill("SIGKILL");
|
||||
decoder.proc.kill("SIGKILL");
|
||||
fg.proc.kill("SIGKILL");
|
||||
bg?.proc.kill("SIGKILL");
|
||||
throw err;
|
||||
}
|
||||
|
||||
encoder.stdin!.end();
|
||||
await Promise.all([decoderExit, encoderExit]);
|
||||
fg.proc.stdin!.end();
|
||||
bg?.proc.stdin!.end();
|
||||
const exits: Promise<void>[] = [decoder.exit, fg.exit];
|
||||
if (bg) exits.push(bg.exit);
|
||||
await Promise.all(exits);
|
||||
|
||||
if (processed === 0) {
|
||||
throw new Error(
|
||||
`No frames produced from ${inputPath}. Decoder stderr:\n${decoderStderr.slice(-400)}`,
|
||||
`No frames produced from ${inputPath}. Decoder stderr:\n${decoder.getStderr().slice(-400)}`,
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { defineCommand } from "citty";
|
||||
import { existsSync, readFileSync } from "node:fs";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import { dirname, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import type { Example } from "./_examples.js";
|
||||
import { c } from "../ui/colors.js";
|
||||
@@ -107,27 +107,10 @@ async function seekTo(page: import("puppeteer-core").Page, time: number): Promis
|
||||
}
|
||||
|
||||
async function bundleProjectHtml(projectDir: string): Promise<string> {
|
||||
// `bundleToSingleHtml` now inlines the runtime IIFE by default, so the
|
||||
// previous post-bundle runtime substitution is no longer needed.
|
||||
const { bundleToSingleHtml } = await import("@hyperframes/core/compiler");
|
||||
let html = await bundleToSingleHtml(projectDir);
|
||||
|
||||
const runtimePath = resolve(
|
||||
__dirname,
|
||||
"..",
|
||||
"..",
|
||||
"..",
|
||||
"core",
|
||||
"dist",
|
||||
"hyperframe.runtime.iife.js",
|
||||
);
|
||||
if (existsSync(runtimePath)) {
|
||||
const runtimeSource = readFileSync(runtimePath, "utf-8");
|
||||
html = html.replace(
|
||||
/<script[^>]*data-hyperframes-preview-runtime[^>]*src="[^"]*"[^>]*><\/script>/,
|
||||
() => `<script data-hyperframes-preview-runtime="1">${runtimeSource}</script>`,
|
||||
);
|
||||
}
|
||||
|
||||
return html;
|
||||
return bundleToSingleHtml(projectDir);
|
||||
}
|
||||
|
||||
async function alignViewportToComposition(
|
||||
|
||||
@@ -20,6 +20,10 @@ export const examples: Example[] = [
|
||||
"Remove background from a single image, output transparent PNG",
|
||||
"hyperframes remove-background portrait.jpg -o cutout.png",
|
||||
],
|
||||
[
|
||||
"Separate the layers — emit both the cutout and an inverse-alpha background plate (subject region transparent)",
|
||||
"hyperframes remove-background avatar.mp4 -o subject.webm --background-output plate.webm",
|
||||
],
|
||||
[
|
||||
"Force CPU (skip CoreML/CUDA)",
|
||||
"hyperframes remove-background avatar.mp4 -o transparent.webm --device cpu",
|
||||
@@ -52,6 +56,12 @@ export default defineCommand({
|
||||
description: "Output path. Format inferred from extension: .webm (default), .mov, .png",
|
||||
alias: "o",
|
||||
},
|
||||
"background-output": {
|
||||
type: "string",
|
||||
description:
|
||||
"Optional second output path for the inverse-alpha background plate (subject region transparent, original surroundings opaque). Hole-cut, not inpainted — composite something underneath to fill the hole. Must be .webm or .mov; not allowed for image inputs.",
|
||||
alias: "b",
|
||||
},
|
||||
device: {
|
||||
type: "string",
|
||||
description: `Execution provider: ${DEVICES.join(", ")}`,
|
||||
@@ -104,6 +114,8 @@ export default defineCommand({
|
||||
|
||||
const inputPath = resolve(args.input);
|
||||
const outputPath = resolve(args.output);
|
||||
const backgroundOutputArg = args["background-output"];
|
||||
const backgroundOutputPath = backgroundOutputArg ? resolve(backgroundOutputArg) : undefined;
|
||||
|
||||
const { render } = await import("../background-removal/pipeline.js");
|
||||
|
||||
@@ -114,6 +126,7 @@ export default defineCommand({
|
||||
const result = await render({
|
||||
inputPath,
|
||||
outputPath,
|
||||
backgroundOutputPath,
|
||||
device: args.device,
|
||||
quality: args.quality,
|
||||
onProgress: (event) => {
|
||||
@@ -137,6 +150,9 @@ export default defineCommand({
|
||||
JSON.stringify({
|
||||
ok: true,
|
||||
outputPath: result.outputPath,
|
||||
...(result.backgroundOutputPath
|
||||
? { backgroundOutputPath: result.backgroundOutputPath }
|
||||
: {}),
|
||||
framesProcessed: result.framesProcessed,
|
||||
durationSeconds: Number(result.durationSeconds.toFixed(2)),
|
||||
avgMsPerFrame: Number(result.avgMsPerFrame.toFixed(1)),
|
||||
@@ -148,9 +164,12 @@ export default defineCommand({
|
||||
const fpsThroughput = result.durationSeconds
|
||||
? (result.framesProcessed / result.durationSeconds).toFixed(1)
|
||||
: "n/a";
|
||||
const outputs = result.backgroundOutputPath
|
||||
? `${c.accent(result.outputPath)} + ${c.accent(result.backgroundOutputPath)}`
|
||||
: c.accent(result.outputPath);
|
||||
spin?.stop(
|
||||
c.success(
|
||||
`Removed background from ${c.accent(String(result.framesProcessed))} frames in ${result.durationSeconds.toFixed(1)}s (${fpsThroughput} fps, ${c.accent(result.provider)}) → ${c.accent(result.outputPath)}`,
|
||||
`Removed background from ${c.accent(String(result.framesProcessed))} frames in ${result.durationSeconds.toFixed(1)}s (${fpsThroughput} fps, ${c.accent(result.provider)}) → ${outputs}`,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
@@ -97,20 +97,9 @@ async function captureSnapshots(
|
||||
|
||||
const numFrames = opts.frames ?? 5;
|
||||
|
||||
// 1. Bundle
|
||||
let html = await bundleToSingleHtml(projectDir);
|
||||
|
||||
// Inject local runtime if available.
|
||||
// Uses the same multi-strategy resolver as the studio preview server
|
||||
// (runtimeSource.ts) so snapshot works in dev (tsx), built CLI, and npx.
|
||||
const { loadRuntimeSource } = await import("../server/runtimeSource.js");
|
||||
const runtimeSource = await loadRuntimeSource();
|
||||
if (runtimeSource) {
|
||||
html = html.replace(
|
||||
/<script[^>]*data-hyperframes-preview-runtime[^>]*src="[^"]*"[^>]*><\/script>/,
|
||||
() => `<script data-hyperframes-preview-runtime="1">${runtimeSource}</script>`,
|
||||
);
|
||||
}
|
||||
// 1. Bundle. `bundleToSingleHtml` now inlines the runtime IIFE by default,
|
||||
// so the previous post-bundle runtime substitution is no longer needed.
|
||||
const html = await bundleToSingleHtml(projectDir);
|
||||
|
||||
const server = await serveStaticProjectHtml(projectDir, html);
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { defineCommand } from "citty";
|
||||
import { existsSync, readFileSync } from "node:fs";
|
||||
import { resolve, join, dirname } from "node:path";
|
||||
import { join, dirname } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { resolveProject } from "../utils/project.js";
|
||||
import { resolveCompositionViewportFromHtml } from "../utils/compositionViewport.js";
|
||||
@@ -113,24 +113,10 @@ async function validateInBrowser(
|
||||
const { bundleToSingleHtml } = await import("@hyperframes/core/compiler");
|
||||
const { ensureBrowser } = await import("../browser/manager.js");
|
||||
|
||||
let html = await bundleToSingleHtml(projectDir);
|
||||
|
||||
const runtimePath = resolve(
|
||||
__dirname,
|
||||
"..",
|
||||
"..",
|
||||
"..",
|
||||
"core",
|
||||
"dist",
|
||||
"hyperframe.runtime.iife.js",
|
||||
);
|
||||
if (existsSync(runtimePath)) {
|
||||
const runtimeSource = readFileSync(runtimePath, "utf-8");
|
||||
html = html.replace(
|
||||
/<script[^>]*data-hyperframes-preview-runtime[^>]*src="[^"]*"[^>]*><\/script>/,
|
||||
() => `<script data-hyperframes-preview-runtime="1">${runtimeSource}</script>`,
|
||||
);
|
||||
}
|
||||
// `bundleToSingleHtml` now inlines the runtime IIFE by default, so the
|
||||
// previous post-bundle regex substitution (which matched `src="..."` on the
|
||||
// runtime tag) is no longer needed — there's no `src` attribute to match.
|
||||
const html = await bundleToSingleHtml(projectDir);
|
||||
|
||||
const { createServer } = await import("node:http");
|
||||
const { getMimeType } = await import("@hyperframes/core/studio-api");
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { shouldWatchProjectFile } from "./fileWatcher.js";
|
||||
|
||||
describe("shouldWatchProjectFile", () => {
|
||||
it("watches files that can affect the project signature", () => {
|
||||
expect(shouldWatchProjectFile("index.html")).toBe(true);
|
||||
expect(shouldWatchProjectFile("src/scene.tsx")).toBe(true);
|
||||
expect(shouldWatchProjectFile("assets/hero.png")).toBe(true);
|
||||
expect(shouldWatchProjectFile("Dockerfile")).toBe(true);
|
||||
});
|
||||
|
||||
it("skips generated and dependency directories excluded from signatures", () => {
|
||||
expect(shouldWatchProjectFile("node_modules/pkg/index.js")).toBe(false);
|
||||
expect(shouldWatchProjectFile("renders/output.mp4")).toBe(false);
|
||||
expect(shouldWatchProjectFile("dist/index.html")).toBe(false);
|
||||
expect(shouldWatchProjectFile(".hyperframes/cache.json")).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -8,9 +8,27 @@ export interface ProjectWatcher {
|
||||
close(): void;
|
||||
}
|
||||
|
||||
const WATCHED_EXTENSIONS = new Set([".html", ".css", ".js", ".json"]);
|
||||
const WATCHER_EXCLUDED_DIRS = new Set([
|
||||
".cache",
|
||||
".git",
|
||||
".hyperframes",
|
||||
".next",
|
||||
".vite",
|
||||
"build",
|
||||
"coverage",
|
||||
"dist",
|
||||
"node_modules",
|
||||
"outputs",
|
||||
"renders",
|
||||
]);
|
||||
const DEBOUNCE_MS = 300;
|
||||
|
||||
export function shouldWatchProjectFile(filename: string): boolean {
|
||||
if (!filename) return false;
|
||||
const parts = filename.split(/[\\/]+/);
|
||||
return !parts.some((part) => WATCHER_EXCLUDED_DIRS.has(part));
|
||||
}
|
||||
|
||||
export function createProjectWatcher(projectDir: string): ProjectWatcher {
|
||||
const listeners = new Set<FileChangeListener>();
|
||||
let debounceTimer: ReturnType<typeof setTimeout> | null = null;
|
||||
@@ -19,13 +37,13 @@ export function createProjectWatcher(projectDir: string): ProjectWatcher {
|
||||
try {
|
||||
watcher = watch(projectDir, { recursive: true }, (_event, filename) => {
|
||||
if (!filename) return;
|
||||
const ext = "." + filename.split(".").pop()?.toLowerCase();
|
||||
if (!WATCHED_EXTENSIONS.has(ext)) return;
|
||||
const relativePath = filename.toString();
|
||||
if (!shouldWatchProjectFile(relativePath)) return;
|
||||
|
||||
if (debounceTimer) clearTimeout(debounceTimer);
|
||||
debounceTimer = setTimeout(() => {
|
||||
for (const fn of listeners) {
|
||||
fn(filename);
|
||||
fn(relativePath);
|
||||
}
|
||||
}, DEBOUNCE_MS);
|
||||
});
|
||||
|
||||
@@ -14,6 +14,7 @@ import { loadRuntimeSource } from "./runtimeSource.js";
|
||||
import { VERSION as version } from "../version.js";
|
||||
import {
|
||||
createStudioApi,
|
||||
createProjectSignature,
|
||||
getMimeType,
|
||||
type StudioApiAdapter,
|
||||
type ResolvedProject,
|
||||
@@ -144,6 +145,10 @@ export function createStudioServer(options: StudioServerOptions): StudioServer {
|
||||
// ── CLI adapter for the shared studio API ──────────────────────────────
|
||||
|
||||
const project: ResolvedProject = { id: projectId, dir: projectDir, title: projectId };
|
||||
let cachedProjectSignature: string | null = null;
|
||||
watcher.addListener(() => {
|
||||
cachedProjectSignature = null;
|
||||
});
|
||||
|
||||
const adapter: StudioApiAdapter = {
|
||||
listProjects: () => [project],
|
||||
@@ -153,8 +158,11 @@ export function createStudioServer(options: StudioServerOptions): StudioServer {
|
||||
async bundle(dir: string): Promise<string | null> {
|
||||
try {
|
||||
const { bundleToSingleHtml } = await import("@hyperframes/core/compiler");
|
||||
let html = await bundleToSingleHtml(dir);
|
||||
// Fix empty runtime src from bundler — point to the local runtime endpoint
|
||||
// Studio dev server: ask the bundler for an empty `src=""` placeholder so
|
||||
// we can point it at our hot-reloadable local runtime endpoint. Inlining
|
||||
// ~150 KB of runtime body on every preview render would defeat browser
|
||||
// caching across composition edits.
|
||||
let html = await bundleToSingleHtml(dir, { runtime: "placeholder" });
|
||||
html = html.replace(
|
||||
'data-hyperframes-preview-runtime="1" src=""',
|
||||
'data-hyperframes-preview-runtime="1" src="/api/runtime.js"',
|
||||
@@ -166,6 +174,12 @@ export function createStudioServer(options: StudioServerOptions): StudioServer {
|
||||
}
|
||||
},
|
||||
|
||||
getProjectSignature(dir: string): string {
|
||||
if (resolve(dir) !== resolve(projectDir)) return createProjectSignature(dir);
|
||||
cachedProjectSignature ??= createProjectSignature(projectDir);
|
||||
return cachedProjectSignature;
|
||||
},
|
||||
|
||||
async lint(html: string, opts?: { filePath?: string }) {
|
||||
const { lintHyperframeHtml } = await import("@hyperframes/core/lint");
|
||||
return lintHyperframeHtml(html, opts);
|
||||
|
||||
@@ -7,6 +7,7 @@ import {
|
||||
createPublishArchive,
|
||||
getPublishApiBaseUrl,
|
||||
publishProjectArchive,
|
||||
uploadTimeoutMs,
|
||||
} from "./publishProject.js";
|
||||
|
||||
function makeProjectDir(): string {
|
||||
@@ -35,6 +36,22 @@ describe("createPublishArchive", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("uploadTimeoutMs", () => {
|
||||
it("returns the minimum timeout for small files", () => {
|
||||
expect(uploadTimeoutMs(0)).toBe(120_000);
|
||||
expect(uploadTimeoutMs(50 * 1024 * 1024)).toBe(120_000);
|
||||
});
|
||||
|
||||
it("scales above the floor for large files", () => {
|
||||
expect(uploadTimeoutMs(64 * 1024 * 1024)).toBeGreaterThan(120_000);
|
||||
expect(uploadTimeoutMs(500 * 1024 * 1024)).toBeGreaterThan(900_000);
|
||||
});
|
||||
|
||||
it("returns an integer", () => {
|
||||
expect(Number.isInteger(uploadTimeoutMs(123_456))).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("publishProjectArchive", () => {
|
||||
beforeEach(() => {
|
||||
vi.stubEnv("HYPERFRAMES_PUBLISHED_PROJECTS_API_URL", "");
|
||||
|
||||
@@ -5,7 +5,11 @@ import AdmZip from "adm-zip";
|
||||
const IGNORED_DIRS = new Set([".git", "node_modules", "dist", ".next", "coverage"]);
|
||||
const IGNORED_FILES = new Set([".DS_Store", "Thumbs.db"]);
|
||||
const PUBLISH_CONTENT_TYPE = "application/zip";
|
||||
const PUBLISH_REQUEST_TIMEOUT_MS = 30_000;
|
||||
const PUBLISH_METADATA_TIMEOUT_MS = 30_000;
|
||||
const PUBLISH_UPLOAD_MIN_TIMEOUT_MS = 120_000;
|
||||
// Conservative floor — most connections are faster, but this prevents
|
||||
// premature aborts on slow/unstable networks (hotel wifi, tethering).
|
||||
const PUBLISH_UPLOAD_BYTES_PER_SECOND = 500_000;
|
||||
|
||||
export interface PublishArchiveResult {
|
||||
buffer: Buffer;
|
||||
@@ -25,6 +29,7 @@ interface StagedUploadResponse {
|
||||
uploadKey: string;
|
||||
contentType: string;
|
||||
uploadHeaders: Record<string, string>;
|
||||
expiresInSeconds: number;
|
||||
}
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
@@ -73,11 +78,14 @@ function parseStagedUploadResponse(
|
||||
const uploadKey = stringField(data, "upload_key");
|
||||
const contentType = stringField(data, "content_type") || PUBLISH_CONTENT_TYPE;
|
||||
if (!uploadUrl || !uploadKey) return null;
|
||||
const rawExpires = data["expires_in_seconds"];
|
||||
const expiresInSeconds = typeof rawExpires === "number" && rawExpires > 0 ? rawExpires : 1800;
|
||||
return {
|
||||
uploadUrl,
|
||||
uploadKey,
|
||||
contentType,
|
||||
uploadHeaders: getUploadHeaders(data, uploadUrl, contentType, archiveByteLength),
|
||||
expiresInSeconds,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -142,6 +150,13 @@ async function readErrorMessage(response: Response, fallback: string): Promise<s
|
||||
return text.trim() ? `${fallback}: ${text.trim().slice(0, 180)}` : fallback;
|
||||
}
|
||||
|
||||
export function uploadTimeoutMs(byteLength: number): number {
|
||||
return Math.max(
|
||||
PUBLISH_UPLOAD_MIN_TIMEOUT_MS,
|
||||
Math.ceil((byteLength / PUBLISH_UPLOAD_BYTES_PER_SECOND) * 1000),
|
||||
);
|
||||
}
|
||||
|
||||
function shouldIgnoreSegment(segment: string): boolean {
|
||||
return segment.startsWith(".") || IGNORED_DIRS.has(segment) || IGNORED_FILES.has(segment);
|
||||
}
|
||||
@@ -214,7 +229,7 @@ async function publishProjectArchiveDirect(
|
||||
method: "POST",
|
||||
body,
|
||||
headers,
|
||||
signal: AbortSignal.timeout(PUBLISH_REQUEST_TIMEOUT_MS),
|
||||
signal: AbortSignal.timeout(uploadTimeoutMs(archive.buffer.byteLength)),
|
||||
});
|
||||
|
||||
const payload = await readJson(response);
|
||||
@@ -243,7 +258,7 @@ async function publishProjectArchiveStaged(
|
||||
"content-type": "application/json",
|
||||
heygen_route: "canary",
|
||||
},
|
||||
signal: AbortSignal.timeout(PUBLISH_REQUEST_TIMEOUT_MS),
|
||||
signal: AbortSignal.timeout(PUBLISH_METADATA_TIMEOUT_MS),
|
||||
});
|
||||
|
||||
if (uploadResponse.status === 404 || uploadResponse.status === 405) {
|
||||
@@ -256,11 +271,14 @@ async function publishProjectArchiveStaged(
|
||||
throw new Error(await readErrorMessage(uploadResponse, "Failed to prepare project upload"));
|
||||
}
|
||||
|
||||
const presignedUrlTtlMs = stagedUpload.expiresInSeconds * 1000 - PUBLISH_METADATA_TIMEOUT_MS;
|
||||
const s3Response = await fetch(stagedUpload.uploadUrl, {
|
||||
method: "PUT",
|
||||
body: new Blob([archiveArrayBuffer(archive)], { type: stagedUpload.contentType }),
|
||||
headers: stagedUpload.uploadHeaders,
|
||||
signal: AbortSignal.timeout(PUBLISH_REQUEST_TIMEOUT_MS),
|
||||
signal: AbortSignal.timeout(
|
||||
Math.min(uploadTimeoutMs(archive.buffer.byteLength), presignedUrlTtlMs),
|
||||
),
|
||||
});
|
||||
if (!s3Response.ok) {
|
||||
throw new Error(await readErrorMessage(s3Response, "Failed to upload project archive"));
|
||||
@@ -277,7 +295,7 @@ async function publishProjectArchiveStaged(
|
||||
"content-type": "application/json",
|
||||
heygen_route: "canary",
|
||||
},
|
||||
signal: AbortSignal.timeout(PUBLISH_REQUEST_TIMEOUT_MS),
|
||||
signal: AbortSignal.timeout(uploadTimeoutMs(archive.buffer.byteLength)),
|
||||
});
|
||||
|
||||
const completePayload = await readJson(completeResponse);
|
||||
|
||||
@@ -212,7 +212,11 @@ export function wrapScopedCompositionScript(
|
||||
value: __hfFindRoot(),
|
||||
configurable: true,
|
||||
});
|
||||
} catch (_err) {}
|
||||
} catch {
|
||||
// Best-effort: timelines coming from user code may have a frozen target
|
||||
// or a non-extensible defineProperty path. Swallow — the scoped root
|
||||
// is an enrichment, not a correctness invariant for playback.
|
||||
}
|
||||
return timeline;
|
||||
};
|
||||
var __hfBaseGsap = typeof gsap === "undefined" ? window.gsap : gsap;
|
||||
@@ -282,5 +286,5 @@ ${source}
|
||||
};
|
||||
__hfFindRoot();
|
||||
__hfRun();
|
||||
})()`;
|
||||
})();`;
|
||||
}
|
||||
|
||||
@@ -38,10 +38,116 @@ describe("bundleToSingleHtml", () => {
|
||||
)?.[0];
|
||||
|
||||
expect(runtimeBlock).toBeDefined();
|
||||
expect(runtimeBlock).not.toContain("getElementById");
|
||||
// The runtime block must contain the inlined HF runtime IIFE — bundled
|
||||
// output is self-contained, so the bundle's runtime body is loaded inline,
|
||||
// not referenced via src.
|
||||
expect(runtimeBlock).toMatch(/data-hyperframes-preview-runtime="1">/);
|
||||
expect(runtimeBlock).not.toMatch(/src=""/);
|
||||
// The author's specific composition script must NOT be merged INTO the
|
||||
// runtime tag — it stays as its own <script> elsewhere in the document.
|
||||
expect(runtimeBlock).not.toContain("window.__timelines.main = { duration:");
|
||||
expect(bundled).toContain('document.getElementById("scene")');
|
||||
});
|
||||
|
||||
it("produces a self-contained runtime script when no HYPERFRAME_RUNTIME_URL is set", async () => {
|
||||
// Regression guard: hf#XXX. The bundler used to emit
|
||||
// <script ... src=""></script> when no runtime URL was configured. An
|
||||
// empty src resolves to the page URL itself, which Chrome flags as an
|
||||
// infinite-fetch hazard. Verify that bundleToSingleHtml inlines the
|
||||
// runtime body so the bundle is genuinely self-contained.
|
||||
const dir = makeTempProject({
|
||||
"index.html": `<!doctype html>
|
||||
<html><body>
|
||||
<div data-composition-id="root" data-width="320" data-height="180"></div>
|
||||
</body></html>`,
|
||||
});
|
||||
|
||||
const previousUrl = process.env.HYPERFRAME_RUNTIME_URL;
|
||||
delete process.env.HYPERFRAME_RUNTIME_URL;
|
||||
let bundled: string;
|
||||
try {
|
||||
bundled = await bundleToSingleHtml(dir);
|
||||
} finally {
|
||||
if (previousUrl !== undefined) process.env.HYPERFRAME_RUNTIME_URL = previousUrl;
|
||||
}
|
||||
|
||||
const runtimeBlock = bundled.match(
|
||||
/<script\b[^>]*data-hyperframes-preview-runtime[^>]*>[\s\S]*?<\/script>/i,
|
||||
)?.[0];
|
||||
expect(runtimeBlock).toBeDefined();
|
||||
// Must NOT have an empty src attribute (would self-fetch).
|
||||
expect(runtimeBlock).not.toMatch(/src=""/);
|
||||
// Must have a non-trivial inlined body (the runtime IIFE is ~150KB).
|
||||
const innerLength = (runtimeBlock!.match(/>([\s\S]*?)<\/script>/)?.[1] ?? "").length;
|
||||
expect(innerLength).toBeGreaterThan(1000);
|
||||
});
|
||||
|
||||
it("preserves chunk integrity when a chunk ends with a line comment (ASI hazard guard)", async () => {
|
||||
// Regression guard for the joinJsChunks helper. If a chunk ends with `// ...`
|
||||
// and we naively appended `;` on the same line, the appended semicolon would
|
||||
// be eaten by the comment, leaving the next chunk's first statement attached
|
||||
// to the previous chunk's last expression. Verify the helper appends `\n;`
|
||||
// instead so the comment terminates and the semicolon stands alone.
|
||||
const dir = makeTempProject({
|
||||
"index.html": `<!doctype html>
|
||||
<html><body>
|
||||
<div data-composition-id="root" data-width="320" data-height="180"></div>
|
||||
<script src="local-a.js"></script>
|
||||
<script src="local-b.js"></script>
|
||||
<script>window.__timelines = window.__timelines || {}; window.__timelines.root = {}</script>
|
||||
</body></html>`,
|
||||
// Chunk A ends with a // line comment — without the \n separator before
|
||||
// the appended ;, that ; would be eaten by the comment.
|
||||
"local-a.js": "window.__a = 1 // trailing line comment",
|
||||
"local-b.js": "window.__b = 2",
|
||||
});
|
||||
|
||||
const bundled = await bundleToSingleHtml(dir);
|
||||
// Run every inline script body through esbuild; if the line comment ate
|
||||
// the separator, parse would fail with an unexpected-token error somewhere
|
||||
// around the chunk boundary. Use a real HTML parser (CodeQL flags regex-
|
||||
// based script extraction as bad-tag-filter).
|
||||
const { transformSync } = await import("esbuild");
|
||||
const { document } = parseHTML(bundled);
|
||||
for (const script of document.querySelectorAll("script")) {
|
||||
const body = script.textContent;
|
||||
if (!body || !body.trim()) continue;
|
||||
expect(() => transformSync(body, { loader: "js", minify: false })).not.toThrow();
|
||||
}
|
||||
});
|
||||
|
||||
it("does not produce stray bare-semicolon lines between concatenated JS chunks", async () => {
|
||||
// Regression guard: hf#XXX. Earlier the bundler joined script chunks with
|
||||
// `\n;\n`, which produces a lone `;` on its own line between chunks. Valid
|
||||
// JS but reads as a code smell. Each chunk should end in `;` and chunks
|
||||
// should join with `\n`.
|
||||
const dir = makeTempProject({
|
||||
"index.html": `<!doctype html>
|
||||
<html><body>
|
||||
<div data-composition-id="root" data-width="320" data-height="180">
|
||||
<div id="child-host"
|
||||
data-composition-id="child"
|
||||
data-composition-src="compositions/child.html"
|
||||
data-start="0" data-duration="2"></div>
|
||||
</div>
|
||||
<script src="local-a.js"></script>
|
||||
<script src="local-b.js"></script>
|
||||
<script>window.__timelines = window.__timelines || {}; window.__timelines.root = {}</script>
|
||||
</body></html>`,
|
||||
"local-a.js": "window.__a = 1",
|
||||
"local-b.js": "window.__b = 2",
|
||||
"compositions/child.html": `<template id="child-template">
|
||||
<div data-composition-id="child" data-width="320" data-height="180">
|
||||
<script>window.__c = 3</script>
|
||||
</div>
|
||||
</template>`,
|
||||
});
|
||||
|
||||
const bundled = await bundleToSingleHtml(dir);
|
||||
// No line is JUST a bare semicolon (with optional surrounding whitespace).
|
||||
expect(bundled).not.toMatch(/\n\s*;\s*\n/);
|
||||
});
|
||||
|
||||
it("hoists external CDN scripts from sub-compositions into the bundle", async () => {
|
||||
const dir = makeTempProject({
|
||||
"index.html": `<!doctype html>
|
||||
@@ -84,8 +190,14 @@ describe("bundleToSingleHtml", () => {
|
||||
// GSAP CDN from main doc should still be present
|
||||
expect(bundled).toContain("cdn.jsdelivr.net/npm/gsap");
|
||||
|
||||
// data-composition-src should be stripped (composition was inlined)
|
||||
expect(bundled).not.toContain("data-composition-src");
|
||||
// data-composition-src should be stripped from the host element (composition
|
||||
// was inlined). The literal string may still appear inside the inlined
|
||||
// runtime IIFE that knows how to look up that attribute — so check the DOM,
|
||||
// not the raw text.
|
||||
const { document: doc } = parseHTML(bundled);
|
||||
const hostEl = doc.getElementById("rockets-host");
|
||||
expect(hostEl).toBeTruthy();
|
||||
expect(hostEl?.hasAttribute("data-composition-src")).toBe(false);
|
||||
});
|
||||
|
||||
it("does not duplicate CDN scripts already present in the main document", async () => {
|
||||
|
||||
@@ -10,6 +10,7 @@ import {
|
||||
import { rewriteAssetPaths, rewriteCssAssetUrls } from "./rewriteSubCompPaths";
|
||||
import { scopeCssToComposition, wrapScopedCompositionScript } from "./compositionScoping";
|
||||
import { validateHyperframeHtmlContract } from "./staticGuard";
|
||||
import { getHyperframeRuntimeScript } from "../generated/runtime-inline";
|
||||
|
||||
/** Resolve a relative path within projectDir, rejecting traversal outside it. */
|
||||
function safePath(projectDir: string, relativePath: string): string | null {
|
||||
@@ -26,12 +27,28 @@ function getRuntimeScriptUrl(): string {
|
||||
return configured || DEFAULT_RUNTIME_SCRIPT_URL;
|
||||
}
|
||||
|
||||
function injectInterceptor(html: string): string {
|
||||
function injectInterceptor(html: string, runtimeMode: "inline" | "placeholder" = "inline"): string {
|
||||
const sanitized = stripEmbeddedRuntimeScripts(html);
|
||||
if (sanitized.includes(RUNTIME_BOOTSTRAP_ATTR)) return sanitized;
|
||||
|
||||
const runtimeScriptUrl = getRuntimeScriptUrl().replace(/"/g, """);
|
||||
const tag = `<script ${RUNTIME_BOOTSTRAP_ATTR}="1" src="${runtimeScriptUrl}"></script>`;
|
||||
// Three modes for the runtime <script>:
|
||||
// 1. HYPERFRAME_RUNTIME_URL env var set → emit src="<url>" (production CDN deploy).
|
||||
// 2. runtime: "placeholder" passed → emit src="" for the caller to substitute
|
||||
// (studio + vite preview hot-load a local
|
||||
// runtime endpoint via string replace).
|
||||
// 3. runtime: "inline" (default) → embed the IIFE body directly so the
|
||||
// bundle is genuinely self-contained.
|
||||
const runtimeScriptUrl = getRuntimeScriptUrl();
|
||||
let tag: string;
|
||||
if (runtimeScriptUrl) {
|
||||
const escaped = runtimeScriptUrl.replace(/"/g, """);
|
||||
tag = `<script ${RUNTIME_BOOTSTRAP_ATTR}="1" src="${escaped}"></script>`;
|
||||
} else if (runtimeMode === "placeholder") {
|
||||
tag = `<script ${RUNTIME_BOOTSTRAP_ATTR}="1" src=""></script>`;
|
||||
} else {
|
||||
const inlinedRuntime = getHyperframeRuntimeScript();
|
||||
tag = `<script ${RUNTIME_BOOTSTRAP_ATTR}="1">${inlinedRuntime}</script>`;
|
||||
}
|
||||
if (sanitized.includes("</head>")) {
|
||||
return sanitized.replace("</head>", `${tag}\n</head>`);
|
||||
}
|
||||
@@ -268,11 +285,7 @@ function coalesceHeadStylesAndBodyScripts(document: Document): void {
|
||||
return !type || type === "text/javascript" || type === "application/javascript";
|
||||
});
|
||||
if (bodyInlineScripts.length > 0) {
|
||||
const mergedJs = bodyInlineScripts
|
||||
.map((el) => (el.textContent || "").trim())
|
||||
.filter(Boolean)
|
||||
.join("\n;\n")
|
||||
.trim();
|
||||
const mergedJs = joinJsChunks(bodyInlineScripts.map((el) => el.textContent || ""));
|
||||
for (const el of bodyInlineScripts) el.remove();
|
||||
if (mergedJs) {
|
||||
const stripped = stripJsCommentsParserSafe(mergedJs);
|
||||
@@ -283,6 +296,31 @@ function coalesceHeadStylesAndBodyScripts(document: Document): void {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Concatenate JS chunks safely. Goals:
|
||||
* - Each chunk's last statement is terminated, so joining can't introduce ASI
|
||||
* surprises (e.g. `a()` followed by `(b)()` — the second chunk would parse
|
||||
* as a call on the first's return value).
|
||||
* - In the common case (chunk already ends with `;` — typical of esbuild
|
||||
* output and IIFE-wrapped composition scripts ending in `})();`), the join
|
||||
* produces clean output: chunks separated by `\n` with no stray bare
|
||||
* semicolon lines.
|
||||
* - Defensive against trailing line comments. If a chunk ends with `// ...`
|
||||
* and we appended `;` on the same line, the appended semicolon would be
|
||||
* swallowed by the comment, leaving the next chunk's first statement
|
||||
* attached to the previous chunk's last expression — exactly the ASI
|
||||
* hazard this helper exists to prevent. So when a chunk doesn't already
|
||||
* end in `;`, we append `\n;` instead — the newline closes any line
|
||||
* comment, and the standalone `;` becomes the statement separator.
|
||||
*/
|
||||
function joinJsChunks(chunks: string[]): string {
|
||||
return chunks
|
||||
.map((chunk) => chunk.trim())
|
||||
.filter((chunk) => chunk.length > 0)
|
||||
.map((chunk) => (chunk.endsWith(";") ? chunk : chunk + "\n;"))
|
||||
.join("\n");
|
||||
}
|
||||
|
||||
function stripJsCommentsParserSafe(source: string): string {
|
||||
if (!source) return source;
|
||||
try {
|
||||
@@ -296,6 +334,22 @@ function stripJsCommentsParserSafe(source: string): string {
|
||||
export interface BundleOptions {
|
||||
/** Optional media duration prober (e.g., ffprobe). If omitted, media durations are not resolved. */
|
||||
probeMediaDuration?: MediaDurationProber;
|
||||
/**
|
||||
* How to handle the HyperFrames runtime <script> tag. Default: `"inline"`.
|
||||
*
|
||||
* - `"inline"` — embed the runtime IIFE body directly into the bundle. Produces
|
||||
* genuinely self-contained HTML. Right for CLI render output, validate,
|
||||
* snapshot, and any "ship a single .html file" use case.
|
||||
* - `"placeholder"` — emit `<script ... src=""></script>` so the caller can
|
||||
* substitute it with a real URL via string replace. Used by the dev studio
|
||||
* server and vite preview to point at a local runtime endpoint, which keeps
|
||||
* the runtime cacheable across hot-reloads instead of re-inlining ~150 KB
|
||||
* on every change.
|
||||
*
|
||||
* The `HYPERFRAME_RUNTIME_URL` env var, when set, takes precedence over both
|
||||
* modes and emits `<script ... src="<URL>">` directly.
|
||||
*/
|
||||
runtime?: "inline" | "placeholder";
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -324,7 +378,7 @@ export async function bundleToSingleHtml(
|
||||
);
|
||||
}
|
||||
|
||||
const withInterceptor = injectInterceptor(compiled);
|
||||
const withInterceptor = injectInterceptor(compiled, options?.runtime ?? "inline");
|
||||
const document = parseHTMLContent(withInterceptor);
|
||||
|
||||
// Inline local CSS
|
||||
@@ -379,12 +433,13 @@ export async function bundleToSingleHtml(
|
||||
}
|
||||
if (localJsChunks.length > 0) {
|
||||
const anchor = document.querySelector('script[data-hf-bundled-local-js="1"]');
|
||||
const joinedJs = joinJsChunks(localJsChunks);
|
||||
if (anchor) {
|
||||
anchor.removeAttribute("data-hf-bundled-local-js");
|
||||
anchor.textContent = localJsChunks.join("\n;\n");
|
||||
anchor.textContent = joinedJs;
|
||||
} else {
|
||||
const script = document.createElement("script");
|
||||
script.textContent = localJsChunks.join("\n;\n");
|
||||
script.textContent = joinedJs;
|
||||
document.body.appendChild(script);
|
||||
}
|
||||
}
|
||||
@@ -623,7 +678,7 @@ export async function bundleToSingleHtml(
|
||||
}
|
||||
if (compScriptChunks.length) {
|
||||
const compScript = document.createElement("script");
|
||||
compScript.textContent = compScriptChunks.join("\n;\n");
|
||||
compScript.textContent = joinJsChunks(compScriptChunks);
|
||||
document.body.appendChild(compScript);
|
||||
}
|
||||
|
||||
|
||||
@@ -49,7 +49,7 @@ export const compositionRules: Array<(ctx: LintContext) => HyperframeLintFinding
|
||||
{
|
||||
code: "composition_file_too_large",
|
||||
severity: "warning",
|
||||
message: `This HTML composition file has ${lineCount} lines. Agents produce better results when large scenes are split into smaller sub-compositions.`,
|
||||
message: `This HTML composition file has ${lineCount} lines. Smaller sub-compositions are easier to read, iterate on, and diff.`,
|
||||
fixHint: `${splitTarget}, then mount them from the parent with data-composition-src so each file stays small enough to inspect, revise, and validate independently.`,
|
||||
},
|
||||
];
|
||||
@@ -77,7 +77,7 @@ export const compositionRules: Array<(ctx: LintContext) => HyperframeLintFinding
|
||||
findings.push({
|
||||
code: "timeline_track_too_dense",
|
||||
severity: "warning",
|
||||
message: `Track ${track} has ${count} timed elements in this HTML file. Agents produce better timelines when dense tracks are split into smaller sub-compositions.`,
|
||||
message: `Track ${track} has ${count} timed elements in this HTML file. Smaller sub-compositions keep timelines easier to read, iterate on, and diff.`,
|
||||
fixHint: `${splitTarget} and mount them from the parent with data-composition-src so the timeline stays easier to inspect, revise, and validate.`,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -460,6 +460,44 @@ export const mediaRules: Array<(ctx: LintContext) => HyperframeLintFinding[]> =
|
||||
return findings;
|
||||
},
|
||||
|
||||
// video_audio_double_source — catches audible <video> paired with a separate
|
||||
// <audio> pointing to the same file, which causes double playback at runtime
|
||||
({ tags }) => {
|
||||
const findings: HyperframeLintFinding[] = [];
|
||||
const videoSources = new Map<string, { id?: string; raw: string }>();
|
||||
const audioSources = new Map<string, { id?: string; raw: string }>();
|
||||
|
||||
for (const tag of tags) {
|
||||
if (!readAttr(tag.raw, "data-start")) continue;
|
||||
const src = readAttr(tag.raw, "src");
|
||||
if (!src) continue;
|
||||
const elementId = readAttr(tag.raw, "id") || undefined;
|
||||
if (tag.name === "video") {
|
||||
const isMuted = hasAttrName(tag.raw, "muted");
|
||||
if (!isMuted) {
|
||||
videoSources.set(src, { id: elementId, raw: tag.raw });
|
||||
}
|
||||
} else if (tag.name === "audio") {
|
||||
audioSources.set(src, { id: elementId, raw: tag.raw });
|
||||
}
|
||||
}
|
||||
|
||||
for (const [src, audioInfo] of audioSources) {
|
||||
const videoInfo = videoSources.get(src);
|
||||
if (!videoInfo) continue;
|
||||
findings.push({
|
||||
code: "video_audio_double_source",
|
||||
severity: "error",
|
||||
message: `<audio${audioInfo.id ? ` id="${audioInfo.id}"` : ""}> and <video${videoInfo.id ? ` id="${videoInfo.id}"` : ""}> both point to the same source. The unmuted video already provides audio — the duplicate <audio> will cause double playback and echo.`,
|
||||
elementId: audioInfo.id,
|
||||
fixHint:
|
||||
"Either mute the video (add `muted` attribute) and keep the separate <audio>, or remove the <audio> element and let the video provide its own audio track.",
|
||||
snippet: truncateSnippet(audioInfo.raw),
|
||||
});
|
||||
}
|
||||
return findings;
|
||||
},
|
||||
|
||||
// imperative_media_control
|
||||
findImperativeMediaControlFindings,
|
||||
];
|
||||
|
||||
@@ -204,17 +204,15 @@ describe("syncRuntimeMedia", () => {
|
||||
expect(clip.el.play).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("nudges preload to auto AND calls play() on unbuffered media in one pass", () => {
|
||||
// Assets added after the runtime already bound its metadata listeners
|
||||
// (e.g. a sub-composition that injects a late <audio>) need both the
|
||||
// preload nudge and the play() call from the same sync tick — the
|
||||
// reviewer flagged that these two concerns must not drift.
|
||||
it("forces preload=auto on every active element, not just during play", () => {
|
||||
// Streaming formats (MP3) may arrive with preload="metadata", which only
|
||||
// buffers the first few seconds. Setting preload="auto" on every active
|
||||
// tick catches elements whose preload was overridden after init.ts set it
|
||||
// — and ensures it happens even when paused (e.g. during a seek).
|
||||
const clip = createMockClip({ start: 0, end: 10 });
|
||||
Object.defineProperty(clip.el, "readyState", { value: 0, writable: true });
|
||||
Object.defineProperty(clip.el, "preload", { value: "metadata", writable: true });
|
||||
syncRuntimeMedia({ clips: [clip], timeSeconds: 5, playing: true, playbackRate: 1 });
|
||||
syncRuntimeMedia({ clips: [clip], timeSeconds: 5, playing: false, playbackRate: 1 });
|
||||
expect(clip.el.preload).toBe("auto");
|
||||
expect(clip.el.play).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("does not re-fire play() while a previous play() is in flight", () => {
|
||||
@@ -230,6 +228,77 @@ describe("syncRuntimeMedia", () => {
|
||||
expect(clip.el.play).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("re-issues play() after a hard seek clears the in-flight guard", () => {
|
||||
// A scrub during playback triggers a hard seek (offset jump > 0.5s).
|
||||
// The fix clears the playRequested guard so the very next sync tick can
|
||||
// re-issue play() instead of waiting 50-150ms for the guard to clear
|
||||
// naturally — closing the audible desync gap on timeline scrub.
|
||||
const clip = createMockClip({ start: 0, end: 20, mediaStart: 0 });
|
||||
Object.defineProperty(clip.el, "currentTime", { value: 2, writable: true });
|
||||
// Steady-state playback at t=2
|
||||
syncRuntimeMedia({ clips: [clip], timeSeconds: 2, playing: true, playbackRate: 1 });
|
||||
expect(clip.el.play).toHaveBeenCalledTimes(1);
|
||||
// Scrub to t=15 — hard seek fires, guard should be cleared
|
||||
syncRuntimeMedia({ clips: [clip], timeSeconds: 15, playing: true, playbackRate: 1 });
|
||||
// Next tick: play() should fire again (guard was cleared by the seek)
|
||||
syncRuntimeMedia({ clips: [clip], timeSeconds: 15.02, playing: true, playbackRate: 1 });
|
||||
expect(clip.el.play).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it("calls load() once when a seek fails past the buffered range (MP3 partial buffer)", () => {
|
||||
// Streaming MP3 with preload="metadata" only buffers the first ~15s.
|
||||
// When the user seeks to 20s, el.currentTime = 20 silently fails —
|
||||
// currentTime stays at 0. The fix detects this and calls load() once
|
||||
// to trigger a full network fetch.
|
||||
const clip = createMockClip({ start: 0, end: 30, mediaStart: 0 });
|
||||
// Simulate: currentTime is writable but the setter is intercepted
|
||||
// to stay at 0 (simulating failed seek past buffer).
|
||||
let internalTime = 0;
|
||||
Object.defineProperty(clip.el, "currentTime", {
|
||||
get: () => internalTime,
|
||||
set: () => {
|
||||
// Seek silently fails — stays at 0 (MP3 past buffer)
|
||||
},
|
||||
configurable: true,
|
||||
});
|
||||
clip.el.load = vi.fn();
|
||||
// First tick at t=20 — hard seek fires, fails, should call load()
|
||||
syncRuntimeMedia({ clips: [clip], timeSeconds: 20, playing: true, playbackRate: 1 });
|
||||
expect(clip.el.load).toHaveBeenCalledTimes(1);
|
||||
// Second tick — load() should NOT be called again (one-shot guard)
|
||||
syncRuntimeMedia({ clips: [clip], timeSeconds: 20.05, playing: true, playbackRate: 1 });
|
||||
expect(clip.el.load).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("does not call load() when the seek succeeds", () => {
|
||||
const clip = createMockClip({ start: 0, end: 30, mediaStart: 0 });
|
||||
Object.defineProperty(clip.el, "currentTime", { value: 0, writable: true });
|
||||
clip.el.load = vi.fn();
|
||||
// Seek to 20 — succeeds (currentTime updates)
|
||||
syncRuntimeMedia({ clips: [clip], timeSeconds: 20, playing: true, playbackRate: 1 });
|
||||
expect(clip.el.currentTime).toBe(20);
|
||||
expect(clip.el.load).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("clears the load-retry guard when clip deactivates and reactivates", () => {
|
||||
const clip = createMockClip({ start: 0, end: 10, mediaStart: 0 });
|
||||
let internalTime = 0;
|
||||
Object.defineProperty(clip.el, "currentTime", {
|
||||
get: () => internalTime,
|
||||
set: () => {},
|
||||
configurable: true,
|
||||
});
|
||||
clip.el.load = vi.fn();
|
||||
// First activation — seek fails, load() called
|
||||
syncRuntimeMedia({ clips: [clip], timeSeconds: 5, playing: true, playbackRate: 1 });
|
||||
expect(clip.el.load).toHaveBeenCalledTimes(1);
|
||||
// Deactivate
|
||||
syncRuntimeMedia({ clips: [clip], timeSeconds: 11, playing: true, playbackRate: 1 });
|
||||
// Reactivate — guard was cleared, so load() can fire again
|
||||
syncRuntimeMedia({ clips: [clip], timeSeconds: 5, playing: true, playbackRate: 1 });
|
||||
expect(clip.el.load).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it("pauses active clip when not playing", () => {
|
||||
const clip = createMockClip({ start: 0, end: 10 });
|
||||
Object.defineProperty(clip.el, "paused", { value: false, writable: true });
|
||||
|
||||
@@ -78,6 +78,13 @@ export function refreshRuntimeMediaCache(params?: {
|
||||
// inactive so the next activation gets a hard resync on its first tick.
|
||||
const lastOffset = new WeakMap<HTMLMediaElement, number>();
|
||||
|
||||
// Elements that had a seek past their buffered range (common with streaming
|
||||
// MP3 where preload="metadata" only fetches the first few seconds). After
|
||||
// setting preload="auto" and calling load(), we mark the element so subsequent
|
||||
// ticks don't restart the fetch in a loop while the browser downloads data.
|
||||
// Cleared when the clip leaves its active window.
|
||||
const seekLoadRetried = new WeakSet<HTMLMediaElement>();
|
||||
|
||||
// Elements whose play() is in flight. The sync runs on a 50 ms poll and with
|
||||
// a 1–2 s buffer that would fire 20–40 spurious play() calls per element —
|
||||
// noise in devtools and, worse, each `.catch(() => {})` would swallow a real
|
||||
@@ -137,6 +144,13 @@ export function syncRuntimeMedia(params: {
|
||||
}
|
||||
if (clip.volume != null) el.volume = clip.volume;
|
||||
if (shouldMute) el.muted = true;
|
||||
// Ensure full preload for every active media element. Streaming
|
||||
// formats (MP3) may arrive with preload="metadata", which only
|
||||
// buffers the first few seconds and causes seeks to silently fail
|
||||
// past the buffered range. Setting this on every tick is cheap
|
||||
// (no-op when already "auto") and catches elements whose preload
|
||||
// was overridden after init.ts set it.
|
||||
if (el.preload !== "auto") el.preload = "auto";
|
||||
try {
|
||||
// Per-element rate × global transport rate
|
||||
el.playbackRate = clip.playbackRate * params.playbackRate;
|
||||
@@ -177,6 +191,26 @@ export function syncRuntimeMedia(params: {
|
||||
} catch {
|
||||
// ignore browser seek restrictions
|
||||
}
|
||||
// Detect failed seek: if currentTime didn't reach the target,
|
||||
// the browser can't seek past its buffered range. Common with
|
||||
// streaming MP3 where only the first ~15s is cached. Force a
|
||||
// full network fetch via load() so the browser builds a complete
|
||||
// media index. One-shot per element — subsequent sync ticks will
|
||||
// re-attempt the seek once data arrives.
|
||||
if (Math.abs(el.currentTime - relTime) > 0.5 && !seekLoadRetried.has(el)) {
|
||||
seekLoadRetried.add(el);
|
||||
el.load();
|
||||
try {
|
||||
el.currentTime = relTime;
|
||||
} catch {
|
||||
// ignore — the seek will be retried on the next tick
|
||||
}
|
||||
}
|
||||
// After a hard seek, clear the in-flight play guard so the next tick
|
||||
// can re-issue play(). Without this, a seek during playback leaves
|
||||
// the element paused at the new position for 50-150ms (one poll
|
||||
// interval) while the timeline continues — audible desync on scrub.
|
||||
playRequested.delete(el);
|
||||
}
|
||||
if (params.playing && el.paused && !playRequested.has(el)) {
|
||||
// `HTMLMediaElement.play()` is spec'd to queue playback and resolve
|
||||
@@ -190,11 +224,6 @@ export function syncRuntimeMedia(params: {
|
||||
// seconds. The canplay listener was also racey — the event could
|
||||
// fire between `load()` and `addEventListener` attachment, wedging
|
||||
// the element waiting for a callback that never came.
|
||||
//
|
||||
// preload="auto" is already set at bind time in init.ts; the
|
||||
// re-assignment here is defensive for media elements that were
|
||||
// inserted after the runtime bound its listeners.
|
||||
if (el.preload !== "auto") el.preload = "auto";
|
||||
markPlayRequested(el);
|
||||
void el.play().catch((err: unknown) => {
|
||||
// If play() rejects — e.g. autoplay blocked, element removed
|
||||
@@ -219,6 +248,7 @@ export function syncRuntimeMedia(params: {
|
||||
// Clip left its active window — drop the offset baseline so the next
|
||||
// activation (e.g. re-entering a sub-composition) gets a hard resync.
|
||||
lastOffset.delete(el);
|
||||
seekLoadRetried.delete(el);
|
||||
if (!el.paused) el.pause();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -95,6 +95,37 @@ describe("createPickerModule", () => {
|
||||
expect(api.getCandidatesAtPoint(Infinity, 0)).toEqual([]);
|
||||
});
|
||||
|
||||
it("does not pick through blocking loading overlays", () => {
|
||||
const picker = createPickerModule({ postMessage: createMockPostMessage() });
|
||||
picker.installPickerApi();
|
||||
const scene = document.createElement("div");
|
||||
scene.id = "scene-title";
|
||||
scene.textContent = "Scene title";
|
||||
const overlay = document.createElement("div");
|
||||
overlay.setAttribute("data-hyper-shader-loading", "");
|
||||
const overlayLabel = document.createElement("span");
|
||||
overlayLabel.textContent = "Preparing scene transitions";
|
||||
overlay.appendChild(overlayLabel);
|
||||
document.body.appendChild(scene);
|
||||
document.body.appendChild(overlay);
|
||||
|
||||
const originalElementsFromPoint = document.elementsFromPoint;
|
||||
Object.defineProperty(document, "elementsFromPoint", {
|
||||
configurable: true,
|
||||
value: vi.fn(() => [overlayLabel, overlay, scene]),
|
||||
});
|
||||
|
||||
const api = (window as any).__HF_PICKER_API;
|
||||
try {
|
||||
expect(api.getCandidatesAtPoint(10, 10)).toEqual([]);
|
||||
} finally {
|
||||
Object.defineProperty(document, "elementsFromPoint", {
|
||||
configurable: true,
|
||||
value: originalElementsFromPoint,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it("pickAtPoint returns null for invalid coords", () => {
|
||||
const picker = createPickerModule({ postMessage: createMockPostMessage() });
|
||||
picker.installPickerApi();
|
||||
|
||||
@@ -4,6 +4,19 @@ type PickerModuleDeps = {
|
||||
postMessage: (payload: RuntimeOutboundMessage) => void;
|
||||
};
|
||||
|
||||
const PICKER_IGNORE_SELECTOR = [
|
||||
"[data-hyperframes-ignore]",
|
||||
"[data-hyperframes-picker-ignore]",
|
||||
"[data-hf-ignore]",
|
||||
"[data-no-inspect]",
|
||||
"[data-no-pick]",
|
||||
"[data-hyper-shader-loading]",
|
||||
].join(",");
|
||||
const PICKER_BLOCK_SELECTOR = [
|
||||
"[data-hyperframes-picker-block]",
|
||||
"[data-hyper-shader-loading]",
|
||||
].join(",");
|
||||
|
||||
export type PickerModule = {
|
||||
enablePickMode: () => void;
|
||||
disablePickMode: () => void;
|
||||
@@ -48,9 +61,14 @@ export function createPickerModule(deps: PickerModuleDeps): PickerModule {
|
||||
const tag = el.tagName.toLowerCase();
|
||||
if (tag === "script" || tag === "style" || tag === "link" || tag === "meta") return false;
|
||||
if (el.classList.contains("__hf-pick-highlight")) return false;
|
||||
if (el.closest(PICKER_IGNORE_SELECTOR)) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
function blocksPickerAtPoint(el: Element | null): boolean {
|
||||
return Boolean(el?.closest(PICKER_BLOCK_SELECTOR));
|
||||
}
|
||||
|
||||
function buildElementSelector(el: Element): string {
|
||||
const htmlEl = el as HTMLElement;
|
||||
if (htmlEl.id) return `#${htmlEl.id}`;
|
||||
@@ -97,6 +115,7 @@ export function createPickerModule(deps: PickerModuleDeps): PickerModule {
|
||||
const single = document.elementFromPoint(clientX, clientY);
|
||||
raw = single ? [single] : [];
|
||||
}
|
||||
if (blocksPickerAtPoint(raw[0] ?? null)) return [];
|
||||
const dedupe: Record<string, true> = {};
|
||||
const candidates: Element[] = [];
|
||||
for (let i = 0; i < raw.length; i += 1) {
|
||||
|
||||
@@ -0,0 +1,142 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { lstatSync, readFileSync, readdirSync } from "node:fs";
|
||||
import { extname, isAbsolute, relative, resolve } from "node:path";
|
||||
|
||||
const SIGNATURE_TEXT_EXTENSIONS = new Set([
|
||||
".cjs",
|
||||
".css",
|
||||
".html",
|
||||
".js",
|
||||
".json",
|
||||
".jsx",
|
||||
".mjs",
|
||||
".svg",
|
||||
".ts",
|
||||
".tsx",
|
||||
]);
|
||||
const SIGNATURE_EXCLUDED_DIRS = new Set([
|
||||
".cache",
|
||||
".git",
|
||||
".hyperframes",
|
||||
".next",
|
||||
".vite",
|
||||
"build",
|
||||
"coverage",
|
||||
"dist",
|
||||
"node_modules",
|
||||
"outputs",
|
||||
"renders",
|
||||
]);
|
||||
const MAX_SIGNATURE_TEXT_BYTES = 2_000_000;
|
||||
|
||||
interface ProjectSignatureFile {
|
||||
file: string;
|
||||
mtimeMs: number;
|
||||
size: number;
|
||||
textContentEligible: boolean;
|
||||
}
|
||||
|
||||
interface ProjectSignatureCacheEntry {
|
||||
fingerprint: string;
|
||||
signature: string;
|
||||
}
|
||||
|
||||
const projectSignatureCache = new Map<string, ProjectSignatureCacheEntry>();
|
||||
|
||||
function isPathWithin(parentDir: string, childPath: string): boolean {
|
||||
const childRelativePath = relative(parentDir, childPath);
|
||||
return (
|
||||
childRelativePath === "" ||
|
||||
(!childRelativePath.startsWith("..") && !isAbsolute(childRelativePath))
|
||||
);
|
||||
}
|
||||
|
||||
function isTextContentEligible(file: string, size: number): boolean {
|
||||
return (
|
||||
SIGNATURE_TEXT_EXTENSIONS.has(extname(file).toLowerCase()) && size <= MAX_SIGNATURE_TEXT_BYTES
|
||||
);
|
||||
}
|
||||
|
||||
function collectProjectSignatureFiles(
|
||||
projectDir: string,
|
||||
dir: string,
|
||||
files: ProjectSignatureFile[],
|
||||
): void {
|
||||
let entries: string[];
|
||||
try {
|
||||
entries = readdirSync(dir).sort();
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
|
||||
for (const entry of entries) {
|
||||
if (SIGNATURE_EXCLUDED_DIRS.has(entry)) continue;
|
||||
const file = resolve(dir, entry);
|
||||
if (!isPathWithin(projectDir, file)) continue;
|
||||
let stat: ReturnType<typeof lstatSync>;
|
||||
try {
|
||||
stat = lstatSync(file);
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
if (stat.isSymbolicLink()) continue;
|
||||
if (stat.isDirectory()) {
|
||||
collectProjectSignatureFiles(projectDir, file, files);
|
||||
} else if (stat.isFile()) {
|
||||
files.push({
|
||||
file,
|
||||
mtimeMs: stat.mtimeMs,
|
||||
size: stat.size,
|
||||
textContentEligible: isTextContentEligible(file, stat.size),
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function createProjectFingerprint(projectDir: string, files: ProjectSignatureFile[]): string {
|
||||
const hash = createHash("sha256");
|
||||
for (const entry of files) {
|
||||
hash.update(relative(projectDir, entry.file));
|
||||
hash.update("\0");
|
||||
hash.update(String(entry.size));
|
||||
hash.update("\0");
|
||||
hash.update(String(entry.mtimeMs));
|
||||
hash.update("\0");
|
||||
hash.update(entry.textContentEligible ? "text" : "binary");
|
||||
hash.update("\0");
|
||||
}
|
||||
return hash.digest("hex").slice(0, 24);
|
||||
}
|
||||
|
||||
export function createProjectSignature(projectDir: string): string {
|
||||
const normalizedProjectDir = resolve(projectDir);
|
||||
const files: ProjectSignatureFile[] = [];
|
||||
collectProjectSignatureFiles(normalizedProjectDir, normalizedProjectDir, files);
|
||||
files.sort((a, b) => a.file.localeCompare(b.file));
|
||||
|
||||
const fingerprint = createProjectFingerprint(normalizedProjectDir, files);
|
||||
const cached = projectSignatureCache.get(normalizedProjectDir);
|
||||
if (cached?.fingerprint === fingerprint) return cached.signature;
|
||||
|
||||
const hash = createHash("sha256");
|
||||
for (const entry of files) {
|
||||
const relativePath = relative(normalizedProjectDir, entry.file);
|
||||
hash.update(relativePath);
|
||||
hash.update("\0");
|
||||
hash.update(String(entry.size));
|
||||
hash.update("\0");
|
||||
if (entry.textContentEligible) {
|
||||
try {
|
||||
hash.update(readFileSync(entry.file));
|
||||
} catch {
|
||||
hash.update(String(entry.mtimeMs));
|
||||
}
|
||||
} else {
|
||||
hash.update(String(entry.mtimeMs));
|
||||
}
|
||||
hash.update("\0");
|
||||
}
|
||||
const signature = hash.digest("hex").slice(0, 24);
|
||||
projectSignatureCache.set(normalizedProjectDir, { fingerprint, signature });
|
||||
return signature;
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
export { createStudioApi } from "./createStudioApi.js";
|
||||
export { createProjectSignature } from "./helpers/projectSignature.js";
|
||||
export type { StudioApiAdapter, ResolvedProject, RenderJobState, LintResult } from "./types.js";
|
||||
export { isSafePath, walkDir } from "./helpers/safePath.js";
|
||||
export { getMimeType, MIME_TYPES } from "./helpers/mime.js";
|
||||
|
||||
@@ -0,0 +1,118 @@
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { Hono } from "hono";
|
||||
import { mkdtempSync, rmSync, symlinkSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { registerPreviewRoutes } from "./preview";
|
||||
import type { StudioApiAdapter } from "../types";
|
||||
|
||||
const tempDirs: string[] = [];
|
||||
|
||||
afterEach(() => {
|
||||
for (const dir of tempDirs.splice(0)) {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
function createProjectDir(): string {
|
||||
const projectDir = mkdtempSync(join(tmpdir(), "hf-preview-test-"));
|
||||
tempDirs.push(projectDir);
|
||||
writeFileSync(join(projectDir, "index.html"), "<html><head></head><body>Preview</body></html>");
|
||||
return projectDir;
|
||||
}
|
||||
|
||||
function createAdapter(
|
||||
projectDir: string,
|
||||
overrides: Partial<StudioApiAdapter> = {},
|
||||
): StudioApiAdapter {
|
||||
return {
|
||||
listProjects: () => [],
|
||||
resolveProject: async (id: string) => ({ id, dir: projectDir }),
|
||||
bundle: async () => null,
|
||||
lint: async () => ({ findings: [] }),
|
||||
runtimeUrl: "/api/runtime.js",
|
||||
rendersDir: () => "/tmp/renders",
|
||||
startRender: () => ({
|
||||
id: "job-1",
|
||||
status: "rendering",
|
||||
progress: 0,
|
||||
outputPath: "/tmp/out.mp4",
|
||||
}),
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
function tryCreateSymlink(target: string, path: string, type: "dir" | "file"): boolean {
|
||||
try {
|
||||
symlinkSync(target, path, type);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
async function getPreviewSignature(projectDir: string): Promise<string> {
|
||||
const app = new Hono();
|
||||
registerPreviewRoutes(app, createAdapter(projectDir));
|
||||
|
||||
const response = await app.request("http://localhost/projects/demo/preview");
|
||||
expect(response.status).toBe(200);
|
||||
const html = await response.text();
|
||||
const match = /<meta name="hyperframes-project-signature" content="([^"]+)">/.exec(html);
|
||||
expect(match?.[1]).toBeTruthy();
|
||||
return match![1]!;
|
||||
}
|
||||
|
||||
describe("registerPreviewRoutes", () => {
|
||||
it("uses the adapter project signature when available", async () => {
|
||||
const projectDir = createProjectDir();
|
||||
const getProjectSignature = vi.fn(() => "cached-signature");
|
||||
const app = new Hono();
|
||||
registerPreviewRoutes(app, createAdapter(projectDir, { getProjectSignature }));
|
||||
|
||||
const response = await app.request("http://localhost/projects/demo/preview");
|
||||
const html = await response.text();
|
||||
|
||||
expect(response.status).toBe(200);
|
||||
expect(getProjectSignature).toHaveBeenCalledWith(projectDir);
|
||||
expect(html).toContain(
|
||||
'<meta name="hyperframes-project-signature" content="cached-signature">',
|
||||
);
|
||||
});
|
||||
|
||||
it("updates the preview signature after project text edits", async () => {
|
||||
const projectDir = createProjectDir();
|
||||
const file = join(projectDir, "scene.js");
|
||||
writeFileSync(file, "export const label = 'first';");
|
||||
|
||||
const firstSignature = await getPreviewSignature(projectDir);
|
||||
expect(await getPreviewSignature(projectDir)).toBe(firstSignature);
|
||||
|
||||
writeFileSync(file, "export const label = 'second with changed size';");
|
||||
|
||||
await expect(getPreviewSignature(projectDir)).resolves.not.toBe(firstSignature);
|
||||
});
|
||||
|
||||
it("skips symlinked files when creating the preview signature", async () => {
|
||||
const projectDir = createProjectDir();
|
||||
const firstSignature = await getPreviewSignature(projectDir);
|
||||
|
||||
const externalDir = mkdtempSync(join(tmpdir(), "hf-preview-external-"));
|
||||
tempDirs.push(externalDir);
|
||||
const externalFile = join(externalDir, "external.js");
|
||||
writeFileSync(externalFile, "export const external = true;");
|
||||
|
||||
if (!tryCreateSymlink(externalFile, join(projectDir, "external.js"), "file")) return;
|
||||
|
||||
await expect(getPreviewSignature(projectDir)).resolves.toBe(firstSignature);
|
||||
});
|
||||
|
||||
it("skips symlinked directories when creating the preview signature", async () => {
|
||||
const projectDir = createProjectDir();
|
||||
if (!tryCreateSymlink(projectDir, join(projectDir, "loop"), "dir")) return;
|
||||
|
||||
const signature = await getPreviewSignature(projectDir);
|
||||
|
||||
expect(signature).toMatch(/^[a-f0-9]{24}$/);
|
||||
});
|
||||
});
|
||||
@@ -5,6 +5,25 @@ import type { StudioApiAdapter } from "../types.js";
|
||||
import { isSafePath } from "../helpers/safePath.js";
|
||||
import { getMimeType } from "../helpers/mime.js";
|
||||
import { buildSubCompositionHtml } from "../helpers/subComposition.js";
|
||||
import { createProjectSignature } from "../helpers/projectSignature.js";
|
||||
|
||||
const PROJECT_SIGNATURE_META = "hyperframes-project-signature";
|
||||
|
||||
function resolveProjectSignature(adapter: StudioApiAdapter, projectDir: string): string {
|
||||
return adapter.getProjectSignature?.(projectDir) ?? createProjectSignature(projectDir);
|
||||
}
|
||||
|
||||
function injectProjectSignature(html: string, signature: string): string {
|
||||
const tag = `<meta name="${PROJECT_SIGNATURE_META}" content="${signature}">`;
|
||||
if (html.includes(`name="${PROJECT_SIGNATURE_META}"`)) {
|
||||
return html.replace(
|
||||
new RegExp(`<meta\\s+name=["']${PROJECT_SIGNATURE_META}["'][^>]*>`, "i"),
|
||||
tag,
|
||||
);
|
||||
}
|
||||
if (html.includes("</head>")) return html.replace("</head>", `${tag}\n</head>`);
|
||||
return `${tag}\n${html}`;
|
||||
}
|
||||
|
||||
export function registerPreviewRoutes(api: Hono, adapter: StudioApiAdapter): void {
|
||||
// Bundled composition preview
|
||||
@@ -37,10 +56,18 @@ export function registerPreviewRoutes(api: Hono, adapter: StudioApiAdapter): voi
|
||||
bundled = bundled.replace(/<head>/i, `<head><base href="${baseHref}">`);
|
||||
}
|
||||
|
||||
bundled = injectProjectSignature(bundled, resolveProjectSignature(adapter, project.dir));
|
||||
return c.html(bundled);
|
||||
} catch {
|
||||
const file = resolve(project.dir, "index.html");
|
||||
if (existsSync(file)) return c.html(readFileSync(file, "utf-8"));
|
||||
if (existsSync(file)) {
|
||||
return c.html(
|
||||
injectProjectSignature(
|
||||
readFileSync(file, "utf-8"),
|
||||
resolveProjectSignature(adapter, project.dir),
|
||||
),
|
||||
);
|
||||
}
|
||||
return c.text("not found", 404);
|
||||
}
|
||||
});
|
||||
@@ -63,7 +90,7 @@ export function registerPreviewRoutes(api: Hono, adapter: StudioApiAdapter): voi
|
||||
const baseHref = `/api/projects/${project.id}/preview/`;
|
||||
const html = buildSubCompositionHtml(project.dir, compPath, adapter.runtimeUrl, baseHref);
|
||||
if (!html) return c.text("not found", 404);
|
||||
return c.html(html);
|
||||
return c.html(injectProjectSignature(html, resolveProjectSignature(adapter, project.dir)));
|
||||
});
|
||||
|
||||
// Static asset serving (with range request support for audio/video seeking)
|
||||
|
||||
@@ -41,6 +41,9 @@ export interface StudioApiAdapter {
|
||||
/** Bundle a project directory into a single HTML string. Returns null if unavailable. */
|
||||
bundle(projectDir: string): Promise<string | null>;
|
||||
|
||||
/** Optional: cached signature for project files that should invalidate preview frame caches. */
|
||||
getProjectSignature?: (projectDir: string) => string;
|
||||
|
||||
/** Lint a single HTML string. */
|
||||
lint(html: string, opts?: { filePath?: string }): Promise<LintResult> | LintResult;
|
||||
|
||||
|
||||
@@ -511,6 +511,16 @@ describe("blitRgba8OverRgb48le", () => {
|
||||
expect(canvas.readUInt16LE(4)).toBe(0);
|
||||
});
|
||||
|
||||
it("fully opaque DOM with srgb transfer expands 8-bit channels to 16-bit SDR", () => {
|
||||
const canvas = makeHdrFrame(1, 1, 10000, 20000, 30000);
|
||||
const dom = makeDomRgba(1, 1, 255, 128, 1, 255);
|
||||
blitRgba8OverRgb48le(dom, canvas, 1, 1, "srgb");
|
||||
|
||||
expect(canvas.readUInt16LE(0)).toBe(65535);
|
||||
expect(canvas.readUInt16LE(2)).toBe(128 * 257);
|
||||
expect(canvas.readUInt16LE(4)).toBe(257);
|
||||
});
|
||||
|
||||
it("sRGB→HLG: black stays black, white stays white", () => {
|
||||
const canvasBlack = makeHdrFrame(1, 1, 0, 0, 0);
|
||||
const domBlack = makeDomRgba(1, 1, 0, 0, 0, 255);
|
||||
|
||||
@@ -249,7 +249,7 @@ export function decodePngToRgb48le(buf: Buffer): { width: number; height: number
|
||||
* bt2020). For neutral/near-neutral content (text, UI) the gamut difference
|
||||
* is negligible.
|
||||
*/
|
||||
function buildSrgbToHdrLut(transfer: "hlg" | "pq"): Uint16Array {
|
||||
function buildSrgbToSignalLut(transfer: "hlg" | "pq" | "srgb"): Uint16Array {
|
||||
const lut = new Uint16Array(256);
|
||||
|
||||
// HLG OETF constants (Rec. 2100)
|
||||
@@ -267,6 +267,11 @@ function buildSrgbToHdrLut(transfer: "hlg" | "pq"): Uint16Array {
|
||||
const sdrNits = 203.0;
|
||||
|
||||
for (let i = 0; i < 256; i++) {
|
||||
if (transfer === "srgb") {
|
||||
lut[i] = i * 257;
|
||||
continue;
|
||||
}
|
||||
|
||||
// sRGB EOTF: signal → linear (range 0–1, relative to SDR white)
|
||||
const v = i / 255;
|
||||
const linear = v <= 0.04045 ? v / 12.92 : Math.pow((v + 0.055) / 1.055, 2.4);
|
||||
@@ -288,12 +293,15 @@ function buildSrgbToHdrLut(transfer: "hlg" | "pq"): Uint16Array {
|
||||
return lut;
|
||||
}
|
||||
|
||||
const SRGB_TO_HLG = buildSrgbToHdrLut("hlg");
|
||||
const SRGB_TO_PQ = buildSrgbToHdrLut("pq");
|
||||
const SRGB_TO_SRGB_16 = buildSrgbToSignalLut("srgb");
|
||||
const SRGB_TO_HLG = buildSrgbToSignalLut("hlg");
|
||||
const SRGB_TO_PQ = buildSrgbToSignalLut("pq");
|
||||
|
||||
/** Select the correct sRGB→HDR LUT for the given transfer function. */
|
||||
function getSrgbToHdrLut(transfer: "hlg" | "pq"): Uint16Array {
|
||||
return transfer === "pq" ? SRGB_TO_PQ : SRGB_TO_HLG;
|
||||
function getSrgbToSignalLut(transfer: "hlg" | "pq" | "srgb"): Uint16Array {
|
||||
if (transfer === "pq") return SRGB_TO_PQ;
|
||||
if (transfer === "hlg") return SRGB_TO_HLG;
|
||||
return SRGB_TO_SRGB_16;
|
||||
}
|
||||
|
||||
// ── Alpha compositing ─────────────────────────────────────────────────────────
|
||||
@@ -317,10 +325,10 @@ export function blitRgba8OverRgb48le(
|
||||
canvas: Buffer,
|
||||
width: number,
|
||||
height: number,
|
||||
transfer: "hlg" | "pq" = "hlg",
|
||||
transfer: "hlg" | "pq" | "srgb" = "hlg",
|
||||
): void {
|
||||
const pixelCount = width * height;
|
||||
const lut = getSrgbToHdrLut(transfer);
|
||||
const lut = getSrgbToSignalLut(transfer);
|
||||
|
||||
for (let i = 0; i < pixelCount; i++) {
|
||||
const da = domRgba[i * 4 + 3] ?? 0;
|
||||
|
||||
+40
-20
@@ -52,18 +52,35 @@ Show a static image before playback starts:
|
||||
|
||||
## Attributes
|
||||
|
||||
| Attribute | Type | Default | Description |
|
||||
| --------------- | ------- | ------- | -------------------------------------------- |
|
||||
| `src` | string | — | URL to the composition HTML file |
|
||||
| `audio-src` | string | — | Audio URL for parent-frame playback (mobile) |
|
||||
| `width` | number | 1920 | Composition width in pixels (aspect ratio) |
|
||||
| `height` | number | 1080 | Composition height in pixels (aspect ratio) |
|
||||
| `controls` | boolean | false | Show play/pause, scrubber, and time display |
|
||||
| `muted` | boolean | false | Mute audio playback |
|
||||
| `poster` | string | — | Image URL shown before playback starts |
|
||||
| `playback-rate` | number | 1 | Speed multiplier (0.5 = half, 2 = double) |
|
||||
| `autoplay` | boolean | false | Start playing when ready |
|
||||
| `loop` | boolean | false | Restart when the composition ends |
|
||||
| Attribute | Type | Default | Description |
|
||||
| ---------------------- | ------------------------------- | ------------- | --------------------------------------------------------------------------- |
|
||||
| `src` | string | — | URL to the composition HTML file |
|
||||
| `audio-src` | string | — | Audio URL for parent-frame playback (mobile) |
|
||||
| `width` | number | 1920 | Composition width in pixels (aspect ratio) |
|
||||
| `height` | number | 1080 | Composition height in pixels (aspect ratio) |
|
||||
| `controls` | boolean | false | Show play/pause, scrubber, and time display |
|
||||
| `muted` | boolean | false | Mute audio playback |
|
||||
| `poster` | string | — | Image URL shown before playback starts |
|
||||
| `playback-rate` | number | 1 | Speed multiplier (0.5 = half, 2 = double) |
|
||||
| `autoplay` | boolean | false | Start playing when ready |
|
||||
| `loop` | boolean | false | Restart when the composition ends |
|
||||
| `shader-capture-scale` | number | — | Shader transition snapshot scale forwarded to browser previews (`0.25`-`1`) |
|
||||
| `shader-loading` | `composition \| player \| none` | `composition` | Controls shader transition prep loading UI ownership |
|
||||
|
||||
### Shader transition previews
|
||||
|
||||
When a composition uses `@hyperframes/shader-transitions`, the player can own preview-only shader capture settings:
|
||||
|
||||
```html
|
||||
<hyperframes-player
|
||||
src="./composition/index.html"
|
||||
shader-capture-scale="1"
|
||||
shader-loading="player"
|
||||
controls
|
||||
></hyperframes-player>
|
||||
```
|
||||
|
||||
`shader-loading="player"` shows the player-owned transition-prep overlay from shader progress messages. `composition` leaves direct composition fallback behavior alone, and `none` suppresses the loader.
|
||||
|
||||
### Mobile audio
|
||||
|
||||
@@ -98,6 +115,8 @@ player.ready; // boolean (read-only)
|
||||
player.playbackRate; // number (read/write)
|
||||
player.muted; // boolean (read/write)
|
||||
player.loop; // boolean (read/write)
|
||||
player.shaderCaptureScale; // number (read/write)
|
||||
player.shaderLoading; // "composition" | "player" | "none" (read/write)
|
||||
|
||||
// Inner iframe access (for advanced consumers — see "Advanced: iframe access" below)
|
||||
player.iframeElement; // HTMLIFrameElement (read-only)
|
||||
@@ -157,14 +176,15 @@ function StudioPreview({ src }: { src: string }) {
|
||||
|
||||
## Events
|
||||
|
||||
| Event | Detail | Fired when |
|
||||
| ------------ | ----------------- | ------------------------------------------ |
|
||||
| `ready` | `{ duration }` | Composition loaded and duration determined |
|
||||
| `play` | — | Playback started |
|
||||
| `pause` | — | Playback paused |
|
||||
| `timeupdate` | `{ currentTime }` | Playback position changed (~10 fps) |
|
||||
| `ended` | — | Reached the end (when not looping) |
|
||||
| `error` | `{ message }` | Composition failed to load |
|
||||
| Event | Detail | Fired when |
|
||||
| ----------------------- | -------------------------- | ------------------------------------------ |
|
||||
| `ready` | `{ duration }` | Composition loaded and duration determined |
|
||||
| `play` | — | Playback started |
|
||||
| `pause` | — | Playback paused |
|
||||
| `timeupdate` | `{ currentTime }` | Playback position changed (~10 fps) |
|
||||
| `ended` | — | Reached the end (when not looping) |
|
||||
| `error` | `{ message }` | Composition failed to load |
|
||||
| `shadertransitionstate` | `{ compositionId, state }` | Shader transition cache/capture progress |
|
||||
|
||||
```js
|
||||
player.addEventListener("ready", (e) => {
|
||||
|
||||
@@ -323,6 +323,137 @@ describe("HyperframesPlayer parent-frame media", () => {
|
||||
});
|
||||
});
|
||||
|
||||
// ── Shader transition preview controls ──
|
||||
//
|
||||
// Shader transition capture scale and loading UI ownership are player-level
|
||||
// preview concerns. The player forwards those options into the iframe before
|
||||
// the composition runs, then renders transition-prep progress from runtime
|
||||
// messages when `shader-loading="player"` is enabled.
|
||||
|
||||
describe("HyperframesPlayer shader transition options", () => {
|
||||
type PlayerWithIframe = HTMLElement & {
|
||||
iframeElement: HTMLIFrameElement;
|
||||
};
|
||||
|
||||
beforeEach(async () => {
|
||||
await import("./hyperframes-player.js");
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
document.body.innerHTML = "";
|
||||
});
|
||||
|
||||
it("observes shader capture scale and loading attributes", () => {
|
||||
const player = document.createElement("hyperframes-player");
|
||||
const Ctor = player.constructor as typeof HTMLElement & {
|
||||
observedAttributes: string[];
|
||||
};
|
||||
|
||||
expect(Ctor.observedAttributes).toContain("shader-capture-scale");
|
||||
expect(Ctor.observedAttributes).toContain("shader-loading");
|
||||
});
|
||||
|
||||
it("passes shader options through src query parameters", () => {
|
||||
const player = document.createElement("hyperframes-player") as PlayerWithIframe;
|
||||
player.setAttribute("shader-capture-scale", "0.5");
|
||||
player.setAttribute("shader-loading", "player");
|
||||
player.setAttribute("src", "/api/projects/demo/preview?x=1#stage");
|
||||
|
||||
const url = new URL(player.iframeElement.src);
|
||||
expect(url.pathname).toBe("/api/projects/demo/preview");
|
||||
expect(url.searchParams.get("x")).toBe("1");
|
||||
expect(url.searchParams.get("__hf_shader_capture_scale")).toBe("0.5");
|
||||
expect(url.searchParams.get("__hf_shader_loading")).toBe("player");
|
||||
expect(url.hash).toBe("#stage");
|
||||
});
|
||||
|
||||
it("injects shader options into srcdoc before composition scripts run", () => {
|
||||
const player = document.createElement("hyperframes-player") as PlayerWithIframe;
|
||||
player.setAttribute("shader-capture-scale", "0.5");
|
||||
player.setAttribute("shader-loading", "player");
|
||||
player.setAttribute(
|
||||
"srcdoc",
|
||||
'<!doctype html><html><head><script src="composition.js"></script></head><body></body></html>',
|
||||
);
|
||||
|
||||
const srcdoc = player.iframeElement.srcdoc;
|
||||
expect(srcdoc).toContain('window.__HF_SHADER_CAPTURE_SCALE="0.5";');
|
||||
expect(srcdoc).toContain('window.__HF_SHADER_LOADING="player";');
|
||||
expect(srcdoc.indexOf("data-hyperframes-player-shader-options")).toBeLessThan(
|
||||
srcdoc.indexOf("composition.js"),
|
||||
);
|
||||
});
|
||||
|
||||
it("shows and hides the player-owned shader loader from transition state messages", () => {
|
||||
vi.useFakeTimers();
|
||||
const player = document.createElement("hyperframes-player") as PlayerWithIframe;
|
||||
player.setAttribute("shader-loading", "player");
|
||||
document.body.appendChild(player);
|
||||
|
||||
const iframeWindow = player.iframeElement.contentWindow;
|
||||
expect(iframeWindow).toBeTruthy();
|
||||
window.dispatchEvent(
|
||||
new MessageEvent("message", {
|
||||
source: iframeWindow,
|
||||
data: {
|
||||
source: "hf-preview",
|
||||
type: "shader-transition-state",
|
||||
compositionId: "main",
|
||||
state: {
|
||||
loading: true,
|
||||
progress: 3,
|
||||
total: 10,
|
||||
currentTransition: 1,
|
||||
transitionTotal: 2,
|
||||
transitionFrame: 3,
|
||||
transitionFrames: 5,
|
||||
phase: "capturing",
|
||||
},
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
const loader = player.shadowRoot?.querySelector(".hfp-shader-loader");
|
||||
expect(loader?.classList.contains("hfp-visible")).toBe(true);
|
||||
expect(loader?.textContent).toContain("1/2");
|
||||
expect(loader?.textContent).toContain("3/5");
|
||||
|
||||
const playEvents: Event[] = [];
|
||||
player.addEventListener("play", (event) => playEvents.push(event));
|
||||
loader?.dispatchEvent(new MouseEvent("click", { bubbles: true, composed: true }));
|
||||
expect(playEvents).toHaveLength(0);
|
||||
|
||||
window.dispatchEvent(
|
||||
new MessageEvent("message", {
|
||||
source: iframeWindow,
|
||||
data: {
|
||||
source: "hf-preview",
|
||||
type: "shader-transition-state",
|
||||
compositionId: "main",
|
||||
state: { loading: false, ready: true },
|
||||
},
|
||||
}),
|
||||
);
|
||||
window.dispatchEvent(
|
||||
new MessageEvent("message", {
|
||||
source: iframeWindow,
|
||||
data: {
|
||||
source: "hf-preview",
|
||||
type: "shader-transition-state",
|
||||
compositionId: "main",
|
||||
state: { loading: false, ready: true },
|
||||
},
|
||||
}),
|
||||
);
|
||||
expect(loader?.classList.contains("hfp-visible")).toBe(false);
|
||||
expect(loader?.classList.contains("hfp-hiding")).toBe(true);
|
||||
vi.advanceTimersByTime(420);
|
||||
expect(loader?.classList.contains("hfp-hiding")).toBe(false);
|
||||
vi.useRealTimers();
|
||||
});
|
||||
});
|
||||
|
||||
// ── Shared stylesheet (adoptedStyleSheets) ──
|
||||
//
|
||||
// Every player constructed in the same document should adopt the *same*
|
||||
|
||||
@@ -20,6 +20,114 @@ function getSharedSheet(): CSSStyleSheet | null {
|
||||
const DEFAULT_FPS = 30;
|
||||
const RUNTIME_CDN_URL =
|
||||
"https://cdn.jsdelivr.net/npm/@hyperframes/core/dist/hyperframe.runtime.iife.js";
|
||||
const SHADER_CAPTURE_SCALE_ATTR = "shader-capture-scale";
|
||||
const SHADER_LOADING_ATTR = "shader-loading";
|
||||
const SHADER_CAPTURE_SCALE_PARAM = "__hf_shader_capture_scale";
|
||||
const SHADER_LOADING_PARAM = "__hf_shader_loading";
|
||||
|
||||
export type ShaderLoadingMode = "composition" | "player" | "none";
|
||||
|
||||
interface ShaderTransitionState {
|
||||
ready?: boolean;
|
||||
progress?: number;
|
||||
total?: number;
|
||||
currentTransition?: number;
|
||||
transitionTotal?: number;
|
||||
transitionFrame?: number;
|
||||
transitionFrames?: number;
|
||||
phase?: "cached" | "capturing" | "finalizing";
|
||||
loading?: boolean;
|
||||
}
|
||||
|
||||
interface ShaderLoaderElements {
|
||||
root: HTMLDivElement;
|
||||
fill: HTMLDivElement;
|
||||
title: HTMLSpanElement;
|
||||
detail: HTMLDivElement;
|
||||
transitionValue: HTMLSpanElement;
|
||||
frameLabel: HTMLSpanElement;
|
||||
frameValue: HTMLSpanElement;
|
||||
frameRow: HTMLDivElement;
|
||||
}
|
||||
|
||||
const SHADER_LOADING_PHRASES = [
|
||||
"Preparing scene transitions",
|
||||
"Sampling outgoing scene motion",
|
||||
"Sampling incoming scene motion",
|
||||
"Caching transition frames",
|
||||
"Finalizing transition preview",
|
||||
];
|
||||
|
||||
function normalizeShaderCaptureScale(value: string | null): string | null {
|
||||
if (value === null) return null;
|
||||
const parsed = Number(value);
|
||||
if (!Number.isFinite(parsed) || parsed <= 0) return null;
|
||||
return String(Math.min(1, Math.max(0.25, parsed)));
|
||||
}
|
||||
|
||||
function normalizeShaderLoadingMode(value: string | null): ShaderLoadingMode {
|
||||
if (value === null || value.trim() === "") return "composition";
|
||||
const normalized = value.trim().toLowerCase();
|
||||
if (
|
||||
normalized === "none" ||
|
||||
normalized === "false" ||
|
||||
normalized === "0" ||
|
||||
normalized === "off"
|
||||
) {
|
||||
return "none";
|
||||
}
|
||||
if (
|
||||
normalized === "player" ||
|
||||
normalized === "true" ||
|
||||
normalized === "1" ||
|
||||
normalized === "on"
|
||||
) {
|
||||
return "player";
|
||||
}
|
||||
return "composition";
|
||||
}
|
||||
|
||||
function setQueryParam(params: URLSearchParams, key: string, value: string | null): void {
|
||||
if (value === null) params.delete(key);
|
||||
else params.set(key, value);
|
||||
}
|
||||
|
||||
function withShaderQueryParams(
|
||||
src: string,
|
||||
scale: string | null,
|
||||
loadingMode: ShaderLoadingMode,
|
||||
): string {
|
||||
const hashIndex = src.indexOf("#");
|
||||
const beforeHash = hashIndex >= 0 ? src.slice(0, hashIndex) : src;
|
||||
const hash = hashIndex >= 0 ? src.slice(hashIndex) : "";
|
||||
const queryIndex = beforeHash.indexOf("?");
|
||||
const path = queryIndex >= 0 ? beforeHash.slice(0, queryIndex) : beforeHash;
|
||||
const query = queryIndex >= 0 ? beforeHash.slice(queryIndex + 1) : "";
|
||||
const params = new URLSearchParams(query);
|
||||
setQueryParam(params, SHADER_CAPTURE_SCALE_PARAM, scale);
|
||||
setQueryParam(params, SHADER_LOADING_PARAM, loadingMode === "composition" ? null : loadingMode);
|
||||
const nextQuery = params.toString();
|
||||
return `${path}${nextQuery ? `?${nextQuery}` : ""}${hash}`;
|
||||
}
|
||||
|
||||
function injectShaderOptionsIntoSrcdoc(
|
||||
html: string,
|
||||
scale: string | null,
|
||||
loadingMode: ShaderLoadingMode,
|
||||
): string {
|
||||
if (scale === null && loadingMode === "composition") return html;
|
||||
const lines: string[] = [];
|
||||
if (scale !== null) lines.push(`window.__HF_SHADER_CAPTURE_SCALE=${JSON.stringify(scale)};`);
|
||||
if (loadingMode !== "composition") {
|
||||
lines.push(`window.__HF_SHADER_LOADING=${JSON.stringify(loadingMode)};`);
|
||||
}
|
||||
const script = `<script data-hyperframes-player-shader-options>${lines.join("")}</script>`;
|
||||
if (/<head\b[^>]*>/i.test(html))
|
||||
return html.replace(/<head\b[^>]*>/i, (match) => `${match}${script}`);
|
||||
if (/<html\b[^>]*>/i.test(html))
|
||||
return html.replace(/<html\b[^>]*>/i, (match) => `${match}${script}`);
|
||||
return `${script}${html}`;
|
||||
}
|
||||
|
||||
class HyperframesPlayer extends HTMLElement {
|
||||
static get observedAttributes() {
|
||||
@@ -33,6 +141,8 @@ class HyperframesPlayer extends HTMLElement {
|
||||
"poster",
|
||||
"playback-rate",
|
||||
"audio-src",
|
||||
SHADER_CAPTURE_SCALE_ATTR,
|
||||
SHADER_LOADING_ATTR,
|
||||
];
|
||||
}
|
||||
|
||||
@@ -42,6 +152,15 @@ class HyperframesPlayer extends HTMLElement {
|
||||
private posterEl: HTMLImageElement | null = null;
|
||||
private controlsApi: ReturnType<typeof createControls> | null = null;
|
||||
private resizeObserver: ResizeObserver;
|
||||
private shaderLoaderEl: HTMLDivElement;
|
||||
private shaderLoaderFillEl: HTMLDivElement;
|
||||
private shaderLoaderTitleEl: HTMLSpanElement;
|
||||
private shaderLoaderDetailEl: HTMLDivElement;
|
||||
private shaderLoaderTransitionValueEl: HTMLSpanElement;
|
||||
private shaderLoaderFrameLabelEl: HTMLSpanElement;
|
||||
private shaderLoaderFrameValueEl: HTMLSpanElement;
|
||||
private shaderLoaderFrameRowEl: HTMLDivElement;
|
||||
private shaderLoaderHideTimeout: ReturnType<typeof setTimeout> | null = null;
|
||||
|
||||
private _ready = false;
|
||||
private _duration = 0;
|
||||
@@ -141,6 +260,16 @@ class HyperframesPlayer extends HTMLElement {
|
||||
|
||||
this.container.appendChild(this.iframe);
|
||||
this.shadow.appendChild(this.container);
|
||||
const shaderLoader = this._createShaderLoader();
|
||||
this.shaderLoaderEl = shaderLoader.root;
|
||||
this.shaderLoaderFillEl = shaderLoader.fill;
|
||||
this.shaderLoaderTitleEl = shaderLoader.title;
|
||||
this.shaderLoaderDetailEl = shaderLoader.detail;
|
||||
this.shaderLoaderTransitionValueEl = shaderLoader.transitionValue;
|
||||
this.shaderLoaderFrameLabelEl = shaderLoader.frameLabel;
|
||||
this.shaderLoaderFrameValueEl = shaderLoader.frameValue;
|
||||
this.shaderLoaderFrameRowEl = shaderLoader.frameRow;
|
||||
this.shadow.appendChild(this.shaderLoaderEl);
|
||||
|
||||
// Clicking the bare player surface toggles play/pause.
|
||||
// Ignore shadow-DOM control interactions so overlay clicks don't double-handle.
|
||||
@@ -167,8 +296,9 @@ class HyperframesPlayer extends HTMLElement {
|
||||
this._setupParentAudioFromUrl(this.getAttribute("audio-src")!);
|
||||
// srcdoc wins over src per HTML spec when both are set; mirror both attributes
|
||||
// so the browser applies the standard precedence rules.
|
||||
if (this.hasAttribute("srcdoc")) this.iframe.srcdoc = this.getAttribute("srcdoc")!;
|
||||
if (this.hasAttribute("src")) this.iframe.src = this.getAttribute("src")!;
|
||||
if (this.hasAttribute("srcdoc"))
|
||||
this.iframe.srcdoc = this._prepareSrcdoc(this.getAttribute("srcdoc")!);
|
||||
if (this.hasAttribute("src")) this.iframe.src = this._prepareSrc(this.getAttribute("src")!);
|
||||
}
|
||||
|
||||
disconnectedCallback() {
|
||||
@@ -176,6 +306,8 @@ class HyperframesPlayer extends HTMLElement {
|
||||
window.removeEventListener("message", this._onMessage);
|
||||
this.iframe.removeEventListener("load", this._onIframeLoad);
|
||||
if (this._probeInterval) clearInterval(this._probeInterval);
|
||||
if (this.shaderLoaderHideTimeout) clearTimeout(this.shaderLoaderHideTimeout);
|
||||
this.shaderLoaderHideTimeout = null;
|
||||
this._teardownMediaObserver();
|
||||
this.controlsApi?.destroy();
|
||||
for (const m of this._parentMedia) {
|
||||
@@ -190,7 +322,7 @@ class HyperframesPlayer extends HTMLElement {
|
||||
case "src":
|
||||
if (val) {
|
||||
this._ready = false;
|
||||
this.iframe.src = val;
|
||||
this.iframe.src = this._prepareSrc(val);
|
||||
}
|
||||
break;
|
||||
case "srcdoc":
|
||||
@@ -198,7 +330,7 @@ class HyperframesPlayer extends HTMLElement {
|
||||
// srcdoc and let src take over. Always reset readiness; the iframe will
|
||||
// load a new document either way.
|
||||
this._ready = false;
|
||||
if (val !== null) this.iframe.srcdoc = val;
|
||||
if (val !== null) this.iframe.srcdoc = this._prepareSrcdoc(val);
|
||||
else this.iframe.removeAttribute("srcdoc");
|
||||
break;
|
||||
case "width":
|
||||
@@ -234,6 +366,10 @@ class HyperframesPlayer extends HTMLElement {
|
||||
case "audio-src":
|
||||
if (val) this._setupParentAudioFromUrl(val);
|
||||
break;
|
||||
case SHADER_CAPTURE_SCALE_ATTR:
|
||||
case SHADER_LOADING_ATTR:
|
||||
this._reloadShaderOptions();
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -355,6 +491,21 @@ class HyperframesPlayer extends HTMLElement {
|
||||
this.setAttribute("playback-rate", String(r));
|
||||
}
|
||||
|
||||
get shaderCaptureScale() {
|
||||
return Number(normalizeShaderCaptureScale(this.getAttribute(SHADER_CAPTURE_SCALE_ATTR)) ?? "1");
|
||||
}
|
||||
set shaderCaptureScale(scale: number) {
|
||||
this.setAttribute(SHADER_CAPTURE_SCALE_ATTR, String(scale));
|
||||
}
|
||||
|
||||
get shaderLoading() {
|
||||
return normalizeShaderLoadingMode(this.getAttribute(SHADER_LOADING_ATTR));
|
||||
}
|
||||
set shaderLoading(mode: ShaderLoadingMode) {
|
||||
if (mode === "composition") this.removeAttribute(SHADER_LOADING_ATTR);
|
||||
else this.setAttribute(SHADER_LOADING_ATTR, mode);
|
||||
}
|
||||
|
||||
get muted() {
|
||||
return this.hasAttribute("muted");
|
||||
}
|
||||
@@ -384,6 +535,236 @@ class HyperframesPlayer extends HTMLElement {
|
||||
}
|
||||
}
|
||||
|
||||
private _shaderCaptureScaleParam(): string | null {
|
||||
return normalizeShaderCaptureScale(this.getAttribute(SHADER_CAPTURE_SCALE_ATTR));
|
||||
}
|
||||
|
||||
private _shaderLoadingMode(): ShaderLoadingMode {
|
||||
return normalizeShaderLoadingMode(this.getAttribute(SHADER_LOADING_ATTR));
|
||||
}
|
||||
|
||||
private _prepareSrc(src: string): string {
|
||||
return withShaderQueryParams(src, this._shaderCaptureScaleParam(), this._shaderLoadingMode());
|
||||
}
|
||||
|
||||
private _prepareSrcdoc(srcdoc: string): string {
|
||||
return injectShaderOptionsIntoSrcdoc(
|
||||
srcdoc,
|
||||
this._shaderCaptureScaleParam(),
|
||||
this._shaderLoadingMode(),
|
||||
);
|
||||
}
|
||||
|
||||
private _reloadShaderOptions(): void {
|
||||
if (this._shaderLoadingMode() !== "player") {
|
||||
this._resetShaderLoader();
|
||||
}
|
||||
if (this.hasAttribute("srcdoc")) {
|
||||
this.iframe.srcdoc = this._prepareSrcdoc(this.getAttribute("srcdoc") || "");
|
||||
return;
|
||||
}
|
||||
if (this.hasAttribute("src")) {
|
||||
this.iframe.src = this._prepareSrc(this.getAttribute("src") || "");
|
||||
}
|
||||
}
|
||||
|
||||
private _createShaderLoader(): ShaderLoaderElements {
|
||||
const root = document.createElement("div");
|
||||
root.className = "hfp-shader-loader";
|
||||
root.setAttribute("role", "status");
|
||||
root.setAttribute("aria-live", "polite");
|
||||
root.setAttribute("aria-label", "Preparing scene transitions");
|
||||
root.setAttribute("data-hyperframes-ignore", "");
|
||||
root.draggable = false;
|
||||
|
||||
const blockOverlayInteraction = (event: Event) => {
|
||||
event.preventDefault();
|
||||
event.stopPropagation();
|
||||
};
|
||||
for (const eventName of [
|
||||
"selectstart",
|
||||
"dragstart",
|
||||
"pointerdown",
|
||||
"mousedown",
|
||||
"click",
|
||||
"dblclick",
|
||||
"contextmenu",
|
||||
"touchstart",
|
||||
]) {
|
||||
root.addEventListener(eventName, blockOverlayInteraction, { capture: true });
|
||||
}
|
||||
|
||||
const panel = document.createElement("div");
|
||||
panel.className = "hfp-shader-loader-panel";
|
||||
panel.draggable = false;
|
||||
|
||||
const markFrame = document.createElement("div");
|
||||
markFrame.className = "hfp-shader-loader-mark";
|
||||
markFrame.draggable = false;
|
||||
markFrame.innerHTML = [
|
||||
'<svg width="78" height="78" viewBox="0 0 100 100" fill="none" aria-hidden="true" draggable="false">',
|
||||
'<path d="M10.1851 57.8021L33.1145 73.8313C36.2202 75.9978 41.5173 73.5433 42.4816 69.4984L51.7611 30.4271C52.7253 26.3822 48.5802 23.9277 44.4602 26.0942L13.917 42.1235C6.96677 45.7676 4.97564 54.1579 10.1851 57.8021Z" fill="url(#hfp-shader-loader-grad-left)"/>',
|
||||
'<path d="M87.5129 57.5141L56.9696 73.5433C52.8371 75.7098 48.7046 73.2553 49.6688 69.2104L58.9483 30.1391C59.9125 26.0942 65.2097 23.6397 68.3154 25.8062L91.2447 41.8354C96.4668 45.4796 94.4631 53.8699 87.5129 57.5141Z" fill="url(#hfp-shader-loader-grad-right)"/>',
|
||||
"<defs>",
|
||||
'<linearGradient id="hfp-shader-loader-grad-left" x1="48.5676" y1="25" x2="44.7804" y2="71.9384" gradientUnits="userSpaceOnUse">',
|
||||
'<stop stop-color="#06E3FA"/>',
|
||||
'<stop offset="1" stop-color="#4FDB5E"/>',
|
||||
"</linearGradient>",
|
||||
'<linearGradient id="hfp-shader-loader-grad-right" x1="54.8282" y1="73.8392" x2="72.0989" y2="32.8932" gradientUnits="userSpaceOnUse">',
|
||||
'<stop stop-color="#06E3FA"/>',
|
||||
'<stop offset="1" stop-color="#4FDB5E"/>',
|
||||
"</linearGradient>",
|
||||
"</defs>",
|
||||
"</svg>",
|
||||
].join("");
|
||||
|
||||
const title = document.createElement("div");
|
||||
title.className = "hfp-shader-loader-title";
|
||||
const titleText = document.createElement("span");
|
||||
titleText.className = "hfp-shader-loader-title-text";
|
||||
titleText.textContent = SHADER_LOADING_PHRASES[0] || "Preparing scene transitions";
|
||||
title.appendChild(titleText);
|
||||
|
||||
const detail = document.createElement("div");
|
||||
detail.className = "hfp-shader-loader-detail";
|
||||
detail.textContent = "Rendering animated scene samples for shader transitions.";
|
||||
|
||||
const track = document.createElement("div");
|
||||
track.className = "hfp-shader-loader-track";
|
||||
track.setAttribute("aria-hidden", "true");
|
||||
const fill = document.createElement("div");
|
||||
fill.className = "hfp-shader-loader-fill";
|
||||
track.appendChild(fill);
|
||||
|
||||
const progress = document.createElement("div");
|
||||
progress.className = "hfp-shader-loader-progress";
|
||||
const createProgressRow = (labelText: string) => {
|
||||
const row = document.createElement("div");
|
||||
row.className = "hfp-shader-loader-row";
|
||||
const label = document.createElement("span");
|
||||
label.className = "hfp-shader-loader-label";
|
||||
label.textContent = labelText;
|
||||
const value = document.createElement("span");
|
||||
value.className = "hfp-shader-loader-value";
|
||||
row.appendChild(label);
|
||||
row.appendChild(value);
|
||||
progress.appendChild(row);
|
||||
return { row, label, value };
|
||||
};
|
||||
const transitionStatus = createProgressRow("transition");
|
||||
const frameStatus = createProgressRow("transition frame");
|
||||
|
||||
panel.appendChild(markFrame);
|
||||
panel.appendChild(title);
|
||||
panel.appendChild(detail);
|
||||
panel.appendChild(track);
|
||||
panel.appendChild(progress);
|
||||
root.appendChild(panel);
|
||||
|
||||
return {
|
||||
root,
|
||||
fill,
|
||||
title: titleText,
|
||||
detail,
|
||||
transitionValue: transitionStatus.value,
|
||||
frameLabel: frameStatus.label,
|
||||
frameValue: frameStatus.value,
|
||||
frameRow: frameStatus.row,
|
||||
};
|
||||
}
|
||||
|
||||
private _showShaderLoader(): void {
|
||||
if (this.shaderLoaderHideTimeout) {
|
||||
clearTimeout(this.shaderLoaderHideTimeout);
|
||||
this.shaderLoaderHideTimeout = null;
|
||||
}
|
||||
this.shaderLoaderEl.classList.remove("hfp-hiding");
|
||||
this.shaderLoaderEl.classList.add("hfp-visible");
|
||||
}
|
||||
|
||||
private _hideShaderLoader(): void {
|
||||
if (this.shaderLoaderEl.classList.contains("hfp-hiding")) {
|
||||
if (!this.shaderLoaderHideTimeout) this._scheduleShaderLoaderHideCleanup();
|
||||
return;
|
||||
}
|
||||
if (!this.shaderLoaderEl.classList.contains("hfp-visible")) return;
|
||||
this.shaderLoaderEl.classList.add("hfp-hiding");
|
||||
this.shaderLoaderEl.classList.remove("hfp-visible");
|
||||
this._scheduleShaderLoaderHideCleanup();
|
||||
}
|
||||
|
||||
private _scheduleShaderLoaderHideCleanup(): void {
|
||||
if (this.shaderLoaderHideTimeout) clearTimeout(this.shaderLoaderHideTimeout);
|
||||
this.shaderLoaderHideTimeout = setTimeout(() => {
|
||||
this.shaderLoaderEl.classList.remove("hfp-hiding");
|
||||
this.shaderLoaderHideTimeout = null;
|
||||
}, 420);
|
||||
}
|
||||
|
||||
private _resetShaderLoader(): void {
|
||||
if (this.shaderLoaderHideTimeout) {
|
||||
clearTimeout(this.shaderLoaderHideTimeout);
|
||||
this.shaderLoaderHideTimeout = null;
|
||||
}
|
||||
this.shaderLoaderEl.classList.remove("hfp-visible", "hfp-hiding");
|
||||
this.shaderLoaderFillEl.style.transform = "scaleX(0)";
|
||||
this.shaderLoaderTransitionValueEl.textContent = "";
|
||||
this.shaderLoaderFrameValueEl.textContent = "";
|
||||
this.shaderLoaderFrameRowEl.style.visibility = "hidden";
|
||||
}
|
||||
|
||||
private _updateShaderLoader(status: ShaderTransitionState): void {
|
||||
if (this._shaderLoadingMode() !== "player") {
|
||||
this._resetShaderLoader();
|
||||
return;
|
||||
}
|
||||
if (status.ready || !status.loading) {
|
||||
this._hideShaderLoader();
|
||||
return;
|
||||
}
|
||||
|
||||
const progress =
|
||||
typeof status.progress === "number" && Number.isFinite(status.progress) ? status.progress : 0;
|
||||
const total =
|
||||
typeof status.total === "number" && Number.isFinite(status.total) ? status.total : 0;
|
||||
const ratio = total > 0 ? Math.min(1, Math.max(0, progress / total)) : 0;
|
||||
const phraseIndex = Math.min(
|
||||
SHADER_LOADING_PHRASES.length - 1,
|
||||
Math.floor(ratio * SHADER_LOADING_PHRASES.length),
|
||||
);
|
||||
this.shaderLoaderTitleEl.textContent =
|
||||
SHADER_LOADING_PHRASES[phraseIndex] || "Preparing scene transitions";
|
||||
this.shaderLoaderDetailEl.textContent =
|
||||
status.phase === "cached"
|
||||
? "Loading cached transition frames before playback."
|
||||
: status.phase === "finalizing"
|
||||
? "Uploading transition textures for smooth playback."
|
||||
: "Rendering animated scene samples for shader transitions.";
|
||||
this.shaderLoaderFillEl.style.transform = `scaleX(${ratio})`;
|
||||
|
||||
this.shaderLoaderTransitionValueEl.textContent =
|
||||
status.currentTransition !== undefined && status.transitionTotal !== undefined
|
||||
? `${status.currentTransition}/${status.transitionTotal}`
|
||||
: total > 0
|
||||
? `${progress}/${total}`
|
||||
: "";
|
||||
|
||||
const frameValue =
|
||||
status.transitionFrame !== undefined && status.transitionFrames !== undefined
|
||||
? `${status.transitionFrame}/${status.transitionFrames}`
|
||||
: "";
|
||||
this.shaderLoaderFrameLabelEl.textContent =
|
||||
status.phase === "cached"
|
||||
? "cached transition frames"
|
||||
: status.phase === "finalizing"
|
||||
? "finalizing transition frames"
|
||||
: "rendering transition frames";
|
||||
this.shaderLoaderFrameValueEl.textContent = frameValue;
|
||||
this.shaderLoaderFrameRowEl.style.visibility = frameValue ? "visible" : "hidden";
|
||||
this.shaderLoaderEl.setAttribute("aria-valuenow", String(Math.round(ratio * 100)));
|
||||
this._showShaderLoader();
|
||||
}
|
||||
|
||||
/**
|
||||
* Reach into the runtime's `window.__player.seek` directly, skipping the
|
||||
* postMessage hop. Same-origin only — cross-origin embeds throw a
|
||||
@@ -426,6 +807,18 @@ class HyperframesPlayer extends HTMLElement {
|
||||
const data = e.data;
|
||||
if (!data || data.source !== "hf-preview") return;
|
||||
|
||||
if (data.type === "shader-transition-state") {
|
||||
const state: ShaderTransitionState =
|
||||
data.state && typeof data.state === "object" ? data.state : {};
|
||||
this._updateShaderLoader(state);
|
||||
this.dispatchEvent(
|
||||
new CustomEvent("shadertransitionstate", {
|
||||
detail: { compositionId: data.compositionId, state },
|
||||
}),
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
if (data.type === "state") {
|
||||
this._currentTime = (data.frame ?? 0) / DEFAULT_FPS;
|
||||
const wasPlaying = !this._paused;
|
||||
@@ -501,6 +894,7 @@ class HyperframesPlayer extends HTMLElement {
|
||||
private _onIframeLoad() {
|
||||
let attempts = 0;
|
||||
this._runtimeInjected = false;
|
||||
this._resetShaderLoader();
|
||||
// A fresh iframe means a fresh runtime — `mediaOutputMuted` and the
|
||||
// autoplay-blocked latch are both reset inside it. The web component's
|
||||
// `_audioOwner` must reset to match, otherwise a composition switch on
|
||||
|
||||
@@ -31,6 +31,161 @@ export const PLAYER_STYLES = /* css */ `
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
.hfp-shader-loader {
|
||||
position: absolute;
|
||||
inset: 0;
|
||||
z-index: 20;
|
||||
display: grid;
|
||||
place-items: center;
|
||||
visibility: hidden;
|
||||
opacity: 0;
|
||||
pointer-events: none;
|
||||
background: #030504;
|
||||
color: #f4f7fb;
|
||||
cursor: default;
|
||||
user-select: none;
|
||||
-webkit-user-select: none;
|
||||
transition: opacity 420ms ease-out, visibility 420ms ease-out;
|
||||
}
|
||||
|
||||
.hfp-shader-loader.hfp-visible,
|
||||
.hfp-shader-loader.hfp-hiding {
|
||||
visibility: visible;
|
||||
}
|
||||
|
||||
.hfp-shader-loader.hfp-visible {
|
||||
opacity: 1;
|
||||
pointer-events: auto;
|
||||
}
|
||||
|
||||
.hfp-shader-loader.hfp-hiding {
|
||||
opacity: 0;
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
.hfp-shader-loader-panel {
|
||||
display: grid;
|
||||
grid-template-rows: 86px 40px 26px 12px 44px;
|
||||
justify-items: center;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
width: min(620px, 82%);
|
||||
text-align: center;
|
||||
font-family: Inter, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
|
||||
}
|
||||
|
||||
.hfp-shader-loader-mark {
|
||||
width: 86px;
|
||||
height: 86px;
|
||||
display: grid;
|
||||
place-items: center;
|
||||
overflow: visible;
|
||||
}
|
||||
|
||||
.hfp-shader-loader-mark svg {
|
||||
display: block;
|
||||
overflow: visible;
|
||||
filter: drop-shadow(0 0 5px rgba(79, 219, 94, 0.16));
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
.hfp-shader-loader-title {
|
||||
width: 100%;
|
||||
height: 40px;
|
||||
overflow: hidden;
|
||||
white-space: nowrap;
|
||||
text-overflow: ellipsis;
|
||||
font-size: 26px;
|
||||
line-height: 40px;
|
||||
font-weight: 700;
|
||||
letter-spacing: 0;
|
||||
}
|
||||
|
||||
.hfp-shader-loader-title-text {
|
||||
color: transparent;
|
||||
background: linear-gradient(
|
||||
90deg,
|
||||
rgba(244, 247, 251, 0.84) 0%,
|
||||
#ffffff 42%,
|
||||
#80efe4 52%,
|
||||
#ffffff 62%,
|
||||
rgba(244, 247, 251, 0.84) 100%
|
||||
);
|
||||
background-size: 220% 100%;
|
||||
-webkit-background-clip: text;
|
||||
background-clip: text;
|
||||
animation: hfp-shader-loader-sheen 1.9s linear infinite;
|
||||
}
|
||||
|
||||
.hfp-shader-loader-detail {
|
||||
width: 100%;
|
||||
height: 26px;
|
||||
overflow: hidden;
|
||||
white-space: nowrap;
|
||||
text-overflow: ellipsis;
|
||||
color: rgba(244, 247, 251, 0.62);
|
||||
font-size: 15px;
|
||||
line-height: 26px;
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.hfp-shader-loader-track {
|
||||
width: min(360px, 100%);
|
||||
height: 8px;
|
||||
overflow: hidden;
|
||||
border-radius: 999px;
|
||||
background: rgba(255, 255, 255, 0.1);
|
||||
}
|
||||
|
||||
.hfp-shader-loader-fill {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
border-radius: inherit;
|
||||
background: linear-gradient(90deg, #06e3fa, #4fdb5e);
|
||||
transform: scaleX(0);
|
||||
transform-origin: left center;
|
||||
transition: transform 160ms ease;
|
||||
}
|
||||
|
||||
.hfp-shader-loader-progress {
|
||||
width: min(420px, 100%);
|
||||
height: 44px;
|
||||
display: grid;
|
||||
grid-template-rows: repeat(2, 22px);
|
||||
color: rgba(244, 247, 251, 0.48);
|
||||
font: 600 13px/22px "IBM Plex Mono", "SF Mono", "Fira Code", "Courier New", monospace;
|
||||
font-variant-numeric: tabular-nums;
|
||||
}
|
||||
|
||||
.hfp-shader-loader-row {
|
||||
display: grid;
|
||||
grid-template-columns: minmax(0, 1fr) 74px;
|
||||
align-items: center;
|
||||
column-gap: 20px;
|
||||
width: 100%;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.hfp-shader-loader-label {
|
||||
min-width: 0;
|
||||
overflow: hidden;
|
||||
text-align: left;
|
||||
text-overflow: ellipsis;
|
||||
}
|
||||
|
||||
.hfp-shader-loader-value {
|
||||
text-align: right;
|
||||
}
|
||||
|
||||
@keyframes hfp-shader-loader-sheen {
|
||||
from {
|
||||
background-position: 140% 0;
|
||||
}
|
||||
to {
|
||||
background-position: -140% 0;
|
||||
}
|
||||
}
|
||||
|
||||
/* ── Theming via CSS custom properties ──
|
||||
*
|
||||
* Override from outside the shadow DOM:
|
||||
|
||||
@@ -21,8 +21,10 @@ import {
|
||||
materializeExtractedFramesForCompiledDir,
|
||||
projectBrowserEndToCompositionTimeline,
|
||||
resolveRenderWorkerCount,
|
||||
resolveCompositeTransfer,
|
||||
selectCaptureCalibrationFrames,
|
||||
shouldFallbackToScreenshotAfterCalibrationError,
|
||||
shouldUseLayeredComposite,
|
||||
shouldUseStreamingEncode,
|
||||
writeCompiledArtifacts,
|
||||
} from "./renderOrchestrator.js";
|
||||
@@ -543,6 +545,48 @@ describe("estimateCaptureCostMultiplier", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("shouldUseLayeredComposite", () => {
|
||||
it("uses the layered compositor for SDR shader transition renders", () => {
|
||||
expect(
|
||||
shouldUseLayeredComposite({
|
||||
hasHdrContent: false,
|
||||
hasShaderTransitions: true,
|
||||
isPngSequence: false,
|
||||
}),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("does not route PNG sequence shader renders through the streaming layered compositor", () => {
|
||||
expect(
|
||||
shouldUseLayeredComposite({
|
||||
hasHdrContent: false,
|
||||
hasShaderTransitions: true,
|
||||
isPngSequence: true,
|
||||
}),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it("keeps HDR content on the layered compositor even without shader transitions", () => {
|
||||
expect(
|
||||
shouldUseLayeredComposite({
|
||||
hasHdrContent: true,
|
||||
hasShaderTransitions: false,
|
||||
isPngSequence: false,
|
||||
}),
|
||||
).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("resolveCompositeTransfer", () => {
|
||||
it("uses 16-bit-expanded sRGB for SDR layered shader transition renders", () => {
|
||||
expect(resolveCompositeTransfer(false, undefined)).toBe("srgb");
|
||||
});
|
||||
|
||||
it("uses the active HDR transfer when HDR content is being preserved", () => {
|
||||
expect(resolveCompositeTransfer(true, { transfer: "hlg" })).toBe("hlg");
|
||||
});
|
||||
});
|
||||
|
||||
describe("estimateMeasuredCaptureCostMultiplier", () => {
|
||||
it("turns slow calibration samples into a capture cost multiplier", () => {
|
||||
const estimate = estimateMeasuredCaptureCostMultiplier([
|
||||
|
||||
@@ -1534,6 +1534,23 @@ function blitHdrImageLayer(
|
||||
* extracting them into an explicit struct lets the helper live at module
|
||||
* scope (no closure-over-renderJob) and keeps the per-call signature small.
|
||||
*/
|
||||
type CompositeTransfer = HdrTransfer | "srgb";
|
||||
|
||||
export function shouldUseLayeredComposite(options: {
|
||||
hasHdrContent: boolean;
|
||||
hasShaderTransitions: boolean;
|
||||
isPngSequence: boolean;
|
||||
}): boolean {
|
||||
return options.hasHdrContent || (options.hasShaderTransitions && !options.isPngSequence);
|
||||
}
|
||||
|
||||
export function resolveCompositeTransfer(
|
||||
hasHdrContent: boolean,
|
||||
effectiveHdr: { transfer: HdrTransfer } | undefined,
|
||||
): CompositeTransfer {
|
||||
return hasHdrContent && effectiveHdr ? effectiveHdr.transfer : "srgb";
|
||||
}
|
||||
|
||||
interface HdrCompositeContext {
|
||||
log: ProducerLogger;
|
||||
domSession: CaptureSession;
|
||||
@@ -1541,7 +1558,7 @@ interface HdrCompositeContext {
|
||||
width: number;
|
||||
height: number;
|
||||
fps: number;
|
||||
effectiveHdr: { transfer: HdrTransfer };
|
||||
compositeTransfer: CompositeTransfer;
|
||||
nativeHdrImageIds: Set<string>;
|
||||
hdrImageBuffers: Map<string, HdrImageBuffer>;
|
||||
hdrImageTransferCache: HdrImageTransferCache;
|
||||
@@ -1596,7 +1613,7 @@ async function compositeHdrFrame(
|
||||
width,
|
||||
height,
|
||||
fps,
|
||||
effectiveHdr,
|
||||
compositeTransfer,
|
||||
nativeHdrImageIds,
|
||||
hdrImageBuffers,
|
||||
hdrImageTransferCache,
|
||||
@@ -1648,6 +1665,7 @@ async function compositeHdrFrame(
|
||||
if (layer.element.opacity <= 0) continue;
|
||||
const before = shouldLog ? countNonZeroRgb48(canvas) : 0;
|
||||
const isHdrImage = nativeHdrImageIds.has(layer.element.id);
|
||||
const hdrTargetTransfer = compositeTransfer === "srgb" ? undefined : compositeTransfer;
|
||||
if (isHdrImage) {
|
||||
blitHdrImageLayer(
|
||||
canvas,
|
||||
@@ -1658,7 +1676,7 @@ async function compositeHdrFrame(
|
||||
height,
|
||||
log,
|
||||
imageTransfers.get(layer.element.id),
|
||||
effectiveHdr.transfer,
|
||||
hdrTargetTransfer,
|
||||
hdrPerf,
|
||||
);
|
||||
} else {
|
||||
@@ -1673,7 +1691,7 @@ async function compositeHdrFrame(
|
||||
height,
|
||||
log,
|
||||
videoTransfers.get(layer.element.id),
|
||||
effectiveHdr.transfer,
|
||||
hdrTargetTransfer,
|
||||
hdrPerf,
|
||||
);
|
||||
}
|
||||
@@ -1774,7 +1792,7 @@ async function compositeHdrFrame(
|
||||
const before = shouldLog ? countNonZeroRgb48(canvas) : 0;
|
||||
const alphaPixels = shouldLog ? countNonZeroAlpha(domRgba) : 0;
|
||||
timingStart = Date.now();
|
||||
blitRgba8OverRgb48le(domRgba, canvas, width, height, effectiveHdr.transfer);
|
||||
blitRgba8OverRgb48le(domRgba, canvas, width, height, compositeTransfer);
|
||||
addHdrTiming(hdrPerf, "domBlitMs", timingStart);
|
||||
if (shouldLog && debugDumpDir) {
|
||||
const after = countNonZeroRgb48(canvas);
|
||||
@@ -2699,7 +2717,12 @@ export async function executeRenderJob(
|
||||
// auto mode stays SDR since H.265 10-bit causes browser color management
|
||||
// issues (orange shift) with no quality benefit.
|
||||
const nativeHdrIds = new Set([...nativeHdrVideoIds, ...nativeHdrImageIds]);
|
||||
const hasHdrContent = effectiveHdr && nativeHdrIds.size > 0;
|
||||
const hasHdrContent = Boolean(effectiveHdr && nativeHdrIds.size > 0);
|
||||
const useLayeredComposite = shouldUseLayeredComposite({
|
||||
hasHdrContent,
|
||||
hasShaderTransitions: compiled.hasShaderTransitions,
|
||||
isPngSequence,
|
||||
});
|
||||
const encoderHdr = hasHdrContent ? effectiveHdr : undefined;
|
||||
// png-sequence has no encoder, but the rest of the orchestrator still
|
||||
// reads `preset.quality` for `effectiveQuality` and `preset.codec` for
|
||||
@@ -2730,21 +2753,27 @@ export async function executeRenderJob(
|
||||
|
||||
job.framesRendered = 0;
|
||||
|
||||
// ── HDR z-ordered multi-layer compositing ──────────────────────────────
|
||||
// ── Z-ordered multi-layer compositing ─────────────────────────────────
|
||||
// Per frame: query all elements' z-order, group into layers (DOM or HDR),
|
||||
// composite bottom-to-top in Node.js memory. HDR layers use native
|
||||
// pre-extracted HLG pixels; DOM layers use Chrome alpha screenshots
|
||||
// with sRGB→HLG conversion. Video position/opacity applied via queried bounds.
|
||||
if (hasHdrContent) {
|
||||
log.info("[Render] HDR layered composite: z-ordered DOM + native HLG video layers");
|
||||
// pre-extracted pixels; DOM layers use Chrome alpha screenshots converted
|
||||
// into the active rgb48le signal space. Shader transitions use this same
|
||||
// path for SDR compositions so the engine can apply transition math to
|
||||
// isolated scene buffers instead of recording plain DOM screenshots.
|
||||
if (useLayeredComposite) {
|
||||
log.info(
|
||||
hasHdrContent
|
||||
? "[Render] HDR layered composite: z-ordered DOM + native HDR video/image layers"
|
||||
: "[Render] Shader transition composite: z-ordered SDR DOM layers",
|
||||
);
|
||||
hdrPerf = createHdrPerfCollector();
|
||||
|
||||
// HDR layered compositing relies on captureAlphaPng (Page.captureScreenshot
|
||||
// with a transparent background) for the SDR DOM overlay layer. That CDP
|
||||
// call hangs indefinitely when Chrome is launched with --enable-begin-frame-control
|
||||
// Layered compositing relies on captureAlphaPng (Page.captureScreenshot
|
||||
// with a transparent background) for DOM layers. That CDP call hangs
|
||||
// indefinitely when Chrome is launched with --enable-begin-frame-control
|
||||
// (the default on Linux/headless-shell), because the compositor is paused
|
||||
// and never produces a frame to capture. Force screenshot mode for the
|
||||
// entire HDR path — same constraint as alpha output formats above.
|
||||
// entire layered path — same constraint as alpha output formats above.
|
||||
cfg.forceScreenshot = true;
|
||||
|
||||
// Use NATIVE HDR IDs (probed before SDR→HDR conversion) so only originally-HDR
|
||||
@@ -2823,8 +2852,13 @@ export async function executeRenderJob(
|
||||
const scenes = document.querySelectorAll(".scene");
|
||||
const map: Record<string, string[]> = {};
|
||||
for (const scene of scenes) {
|
||||
const els = scene.querySelectorAll("[data-start]");
|
||||
map[scene.id] = Array.from(els).map((e) => e.id);
|
||||
if (!scene.id) continue;
|
||||
const ids = new Set<string>([scene.id]);
|
||||
const els = scene.querySelectorAll("[id]");
|
||||
for (const el of els) {
|
||||
if (el.id) ids.add(el.id);
|
||||
}
|
||||
map[scene.id] = Array.from(ids);
|
||||
}
|
||||
return map;
|
||||
});
|
||||
@@ -2836,7 +2870,7 @@ export async function executeRenderJob(
|
||||
}));
|
||||
|
||||
if (transitionRanges.length > 0) {
|
||||
log.info("[Render] Detected shader transitions for HDR compositing", {
|
||||
log.info("[Render] Detected shader transitions for layered compositing", {
|
||||
count: transitionRanges.length,
|
||||
transitions: transitionRanges.map((t) => ({
|
||||
shader: t.shader,
|
||||
@@ -3109,15 +3143,8 @@ export async function executeRenderJob(
|
||||
if (debugDumpDir && !existsSync(debugDumpDir)) {
|
||||
mkdirSync(debugDumpDir, { recursive: true });
|
||||
}
|
||||
// INVARIANT: this entire `try` block is reachable only when HDR
|
||||
// output is enabled (`if (effectiveHdr) { ... try { ... } }`), so
|
||||
// narrowing here is safe even though `effectiveHdr` is typed as
|
||||
// `... | undefined` at the outer scope.
|
||||
if (!effectiveHdr) {
|
||||
throw new Error(
|
||||
"Internal: HDR render path entered without effectiveHdr — this is a bug.",
|
||||
);
|
||||
}
|
||||
const compositeTransfer = resolveCompositeTransfer(hasHdrContent, effectiveHdr);
|
||||
const hdrTargetTransfer = compositeTransfer === "srgb" ? undefined : compositeTransfer;
|
||||
// Per-job LRU cache for transfer-converted HDR image buffers. Static HDR
|
||||
// images that need PQ↔HLG conversion are converted exactly once per
|
||||
// (imageId, targetTransfer) and then reused for every subsequent frame
|
||||
@@ -3137,7 +3164,7 @@ export async function executeRenderJob(
|
||||
width,
|
||||
height,
|
||||
fps: job.config.fps,
|
||||
effectiveHdr,
|
||||
compositeTransfer,
|
||||
nativeHdrImageIds,
|
||||
hdrImageBuffers,
|
||||
hdrImageTransferCache,
|
||||
@@ -3281,7 +3308,7 @@ export async function executeRenderJob(
|
||||
height,
|
||||
log,
|
||||
imageTransfers.get(el.id),
|
||||
effectiveHdr?.transfer,
|
||||
hdrTargetTransfer,
|
||||
hdrPerf,
|
||||
);
|
||||
} else {
|
||||
@@ -3296,7 +3323,7 @@ export async function executeRenderJob(
|
||||
height,
|
||||
log,
|
||||
videoTransfers.get(el.id),
|
||||
effectiveHdr?.transfer,
|
||||
hdrTargetTransfer,
|
||||
hdrPerf,
|
||||
);
|
||||
}
|
||||
@@ -3328,19 +3355,13 @@ export async function executeRenderJob(
|
||||
timingStart = Date.now();
|
||||
const { data: domRgba } = decodePng(domPng);
|
||||
addHdrTiming(hdrPerf, "domPngDecodeMs", timingStart);
|
||||
// Invariant: `hasHdrVideo` requires `effectiveHdr` to be set (see line ~919).
|
||||
if (!effectiveHdr) {
|
||||
throw new Error(
|
||||
"Invariant violation: effectiveHdr is undefined inside hasHdrVideo branch",
|
||||
);
|
||||
}
|
||||
timingStart = Date.now();
|
||||
blitRgba8OverRgb48le(
|
||||
domRgba,
|
||||
sceneBuf as Buffer,
|
||||
width,
|
||||
height,
|
||||
effectiveHdr.transfer,
|
||||
compositeTransfer,
|
||||
);
|
||||
addHdrTiming(hdrPerf, "domBlitMs", timingStart);
|
||||
} catch (err) {
|
||||
@@ -3352,11 +3373,12 @@ export async function executeRenderJob(
|
||||
}
|
||||
}
|
||||
|
||||
// Apply shader transition blend directly in PQ/HLG signal space.
|
||||
// Linearization was attempted but destroys dark PQ content — values below
|
||||
// PQ ~5000 quantize to zero in 16-bit linear, wiping out the bottom portion
|
||||
// of dark video content. PQ space is perceptual and works well enough
|
||||
// for shader math since the shaders were designed for perceptual (sRGB) space.
|
||||
// Apply shader transition blend directly in the active rgb48le
|
||||
// signal space. Linearizing HDR was attempted but destroys dark
|
||||
// PQ content — values below PQ ~5000 quantize to zero in 16-bit
|
||||
// linear, wiping out the bottom portion of dark video content.
|
||||
// SDR compositions use 16-bit-expanded sRGB, which matches the
|
||||
// shader design space.
|
||||
const transitionFn: TransitionFn = TRANSITIONS[activeTransition.shader] ?? crossfade;
|
||||
transitionFn(transBufferA, transBufferB, transOutput, width, height, progress);
|
||||
addHdrTiming(hdrPerf, "transitionCompositeMs", transitionTimingStart);
|
||||
@@ -3431,7 +3453,7 @@ export async function executeRenderJob(
|
||||
updateJobStatus(
|
||||
job,
|
||||
"rendering",
|
||||
`HDR composite frame ${i + 1}/${job.totalFrames}`,
|
||||
`Layered composite frame ${i + 1}/${job.totalFrames}`,
|
||||
Math.round(25 + frameProgress * 55),
|
||||
onProgress,
|
||||
);
|
||||
|
||||
@@ -30,7 +30,7 @@ const tl = init({
|
||||
});
|
||||
```
|
||||
|
||||
The `init()` function captures each scene to a WebGL texture at transition time, crossfades between them using the selected shader, and returns a GSAP timeline. If WebGL is unavailable, it falls back to hard cuts.
|
||||
The `init()` function pre-captures animated scene samples for every transition, composites cached samples with the selected shader during playback, and returns a GSAP timeline. Scene animations keep advancing through shader transitions without running DOM captures in the playback loop. If WebGL is unavailable, it falls back to normal timeline playback without shader compositing.
|
||||
|
||||
When the browser exposes Chrome's experimental CanvasDrawElement API, scene
|
||||
capture uses native HTML-in-canvas via `drawElementImage()`. Other browsers keep
|
||||
@@ -79,14 +79,19 @@ init({
|
||||
|
||||
### `init(config): GsapTimeline`
|
||||
|
||||
| Option | Type | Required | Description |
|
||||
| --------------- | -------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `bgColor` | `string` | yes | Fallback background color (hex) for scene capture. Use the composition's body/canvas background — individual scenes set their own `background-color` via CSS. |
|
||||
| `accentColor` | `string` | no | Accent color (hex) for shader glow effects |
|
||||
| `scenes` | `string[]` | yes | Element IDs of each scene, in order |
|
||||
| `transitions` | `TransitionConfig[]` | yes | Transition definitions (see below) |
|
||||
| `timeline` | `GsapTimeline` | no | Existing timeline to attach transitions to |
|
||||
| `compositionId` | `string` | no | Override the `data-composition-id` for timeline registration |
|
||||
| Option | Type | Required | Description |
|
||||
| ------------------- | -------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `bgColor` | `string` | yes | Fallback background color (hex) for scene capture. Use the composition's body/canvas background — individual scenes set their own `background-color` via CSS. |
|
||||
| `accentColor` | `string` | no | Accent color (hex) for shader glow effects |
|
||||
| `scenes` | `string[]` | yes | Element IDs of each scene, in order |
|
||||
| `transitions` | `TransitionConfig[]` | yes | Transition definitions (see below) |
|
||||
| `timeline` | `GsapTimeline` | no | Existing timeline to attach transitions to |
|
||||
| `compositionId` | `string` | no | Override the `data-composition-id` for timeline registration |
|
||||
| `previewCaptureFps` | `number` | no | Browser preview pre-capture samples per transition second. Defaults to `30`; rendering uses deterministic per-frame compositing instead. |
|
||||
|
||||
Browser preview capture scale and transition-prep loading UI ownership are controlled by `<hyperframes-player>` (`shader-capture-scale`, `shader-loading`) instead of composition code. Direct non-player previews keep the built-in full-fidelity loading fallback.
|
||||
|
||||
Browser previews store captured transition snapshots in IndexedDB using a key derived from composition ID, scene DOM/style signatures, transition timing, capture FPS, scale, and dimensions. On refresh, matching snapshots are reloaded into WebGL textures instead of being captured again. Runtime scene or stylesheet edits mark only adjacent transition caches dirty; recapture is deferred until playback so editing stays responsive.
|
||||
|
||||
### `TransitionConfig`
|
||||
|
||||
|
||||
@@ -1,22 +1,23 @@
|
||||
import html2canvas from "html2canvas";
|
||||
import { DEFAULT_WIDTH, DEFAULT_HEIGHT } from "./webgl.js";
|
||||
|
||||
type CanvasWithLayoutSubtree = HTMLCanvasElement & {
|
||||
layoutSubtree: boolean;
|
||||
requestPaint: () => void;
|
||||
};
|
||||
|
||||
type DrawElementImageContext = CanvasRenderingContext2D & {
|
||||
drawElementImage: (
|
||||
element: Element,
|
||||
dx: number,
|
||||
dy: number,
|
||||
dwidth: number,
|
||||
dheight: number,
|
||||
) => DOMMatrix;
|
||||
};
|
||||
|
||||
let patched = false;
|
||||
const VOID_ELEMENT_TAGS = new Set([
|
||||
"AREA",
|
||||
"BASE",
|
||||
"BR",
|
||||
"COL",
|
||||
"EMBED",
|
||||
"HR",
|
||||
"IMG",
|
||||
"INPUT",
|
||||
"LINK",
|
||||
"META",
|
||||
"PARAM",
|
||||
"SOURCE",
|
||||
"TRACK",
|
||||
"WBR",
|
||||
]);
|
||||
|
||||
function patchCreatePattern(): void {
|
||||
if (patched) return;
|
||||
@@ -42,6 +43,64 @@ export function initCapture(): void {
|
||||
patchCreatePattern();
|
||||
}
|
||||
|
||||
export interface CaptureSceneOptions {
|
||||
forceVisible?: boolean;
|
||||
preferBrowserPaint?: boolean;
|
||||
scale?: number;
|
||||
}
|
||||
|
||||
function forceSceneVisibleInClone(source: HTMLElement, cloneDoc: Document): void {
|
||||
if (!source.id) return;
|
||||
const clone = cloneDoc.getElementById(source.id);
|
||||
if (!(clone instanceof HTMLElement)) return;
|
||||
|
||||
clone.style.opacity = "1";
|
||||
clone.style.visibility = "visible";
|
||||
clone.querySelectorAll<HTMLElement>("[data-start]").forEach((el) => {
|
||||
el.style.visibility = "visible";
|
||||
});
|
||||
}
|
||||
|
||||
function stabilizeTransformedBoxShadows(root: HTMLElement): void {
|
||||
const view = root.ownerDocument.defaultView;
|
||||
if (!view) return;
|
||||
|
||||
[root, ...Array.from(root.querySelectorAll<HTMLElement>("*"))].forEach((el) => {
|
||||
if (VOID_ELEMENT_TAGS.has(el.tagName)) return;
|
||||
const styles = view.getComputedStyle(el);
|
||||
if (styles.boxShadow === "none" || styles.transform === "none") return;
|
||||
|
||||
const shadow = root.ownerDocument.createElement("div");
|
||||
shadow.setAttribute("data-hyper-shader-shadow-shim", "");
|
||||
shadow.style.cssText = [
|
||||
"position:absolute",
|
||||
"inset:0",
|
||||
"border-radius:inherit",
|
||||
`box-shadow:${styles.boxShadow}`,
|
||||
"background:transparent",
|
||||
"pointer-events:none",
|
||||
"z-index:0",
|
||||
].join(";");
|
||||
|
||||
if (styles.position === "static") {
|
||||
el.style.position = "relative";
|
||||
}
|
||||
el.style.boxShadow = "none";
|
||||
el.insertBefore(shadow, el.firstChild);
|
||||
});
|
||||
}
|
||||
|
||||
// ── HTML-in-Canvas (drawElementImage) native capture ──────────────────────
|
||||
|
||||
interface CanvasWithLayoutSubtree extends HTMLCanvasElement {
|
||||
layoutSubtree: boolean;
|
||||
requestPaint: () => void;
|
||||
}
|
||||
|
||||
interface CanvasRenderingContext2DWithDrawElement extends CanvasRenderingContext2D {
|
||||
drawElementImage: (element: Element, x: number, y: number, w: number, h: number) => void;
|
||||
}
|
||||
|
||||
function hasLayoutSubtreeCanvas(canvas: HTMLCanvasElement): canvas is CanvasWithLayoutSubtree {
|
||||
const candidate = canvas as HTMLCanvasElement & {
|
||||
layoutSubtree?: unknown;
|
||||
@@ -50,47 +109,15 @@ function hasLayoutSubtreeCanvas(canvas: HTMLCanvasElement): canvas is CanvasWith
|
||||
return "layoutSubtree" in candidate && typeof candidate.requestPaint === "function";
|
||||
}
|
||||
|
||||
function getDrawElementImageContext(canvas: HTMLCanvasElement): DrawElementImageContext | null {
|
||||
const ctx = canvas.getContext("2d");
|
||||
const candidate = ctx as (CanvasRenderingContext2D & { drawElementImage?: unknown }) | null;
|
||||
if (!candidate || typeof candidate.drawElementImage !== "function") {
|
||||
return null;
|
||||
}
|
||||
return candidate as DrawElementImageContext;
|
||||
}
|
||||
|
||||
export function isHtmlInCanvasCaptureSupported(): boolean {
|
||||
if (typeof document === "undefined") {
|
||||
return false;
|
||||
}
|
||||
|
||||
const canvas = document.createElement("canvas");
|
||||
return hasLayoutSubtreeCanvas(canvas) && getDrawElementImageContext(canvas) !== null;
|
||||
}
|
||||
|
||||
function waitForNextFrame(): Promise<void> {
|
||||
return new Promise((resolve) => {
|
||||
requestAnimationFrame(() => {
|
||||
requestAnimationFrame(() => resolve());
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function waitForPaint(canvas: CanvasWithLayoutSubtree): Promise<void> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const timeout = setTimeout(() => {
|
||||
canvas.removeEventListener("paint", onPaint);
|
||||
reject(new Error("Timed out waiting for canvas paint event"));
|
||||
}, 1000);
|
||||
|
||||
const onPaint = () => {
|
||||
clearTimeout(timeout);
|
||||
resolve();
|
||||
};
|
||||
|
||||
canvas.addEventListener("paint", onPaint, { once: true });
|
||||
canvas.requestPaint();
|
||||
});
|
||||
if (typeof document === "undefined") return false;
|
||||
const probe = document.createElement("canvas") as HTMLCanvasElement & {
|
||||
layoutSubtree?: boolean;
|
||||
};
|
||||
probe.setAttribute("layoutsubtree", "");
|
||||
if (!("layoutSubtree" in probe)) return false;
|
||||
const ctx = probe.getContext("2d") as CanvasRenderingContext2DWithDrawElement | null;
|
||||
return ctx != null && typeof ctx.drawElementImage === "function";
|
||||
}
|
||||
|
||||
async function captureSceneWithHtmlInCanvas(
|
||||
@@ -99,165 +126,108 @@ async function captureSceneWithHtmlInCanvas(
|
||||
width: number,
|
||||
height: number,
|
||||
): Promise<HTMLCanvasElement> {
|
||||
const canvas = document.createElement("canvas");
|
||||
if (!hasLayoutSubtreeCanvas(canvas)) {
|
||||
throw new Error("HTML-in-canvas layoutsubtree support is unavailable");
|
||||
}
|
||||
|
||||
const ctx = getDrawElementImageContext(canvas);
|
||||
if (!ctx) {
|
||||
throw new Error("HTML-in-canvas drawElementImage support is unavailable");
|
||||
}
|
||||
|
||||
const clone = sceneEl.cloneNode(true);
|
||||
if (!(clone instanceof HTMLElement)) {
|
||||
throw new Error("Scene clone is not an HTMLElement");
|
||||
}
|
||||
|
||||
const canvas = document.createElement("canvas") as CanvasWithLayoutSubtree;
|
||||
canvas.width = width;
|
||||
canvas.height = height;
|
||||
canvas.layoutSubtree = true;
|
||||
canvas.setAttribute("layoutsubtree", "");
|
||||
canvas.style.cssText = [
|
||||
"position:fixed",
|
||||
"left:0",
|
||||
"top:0",
|
||||
`width:${width}px`,
|
||||
`height:${height}px`,
|
||||
"pointer-events:none",
|
||||
"opacity:0.001",
|
||||
"z-index:-2147483648",
|
||||
].join(";");
|
||||
|
||||
clone.style.position = "absolute";
|
||||
clone.style.left = "0";
|
||||
clone.style.top = "0";
|
||||
clone.style.width = `${width}px`;
|
||||
clone.style.height = `${height}px`;
|
||||
|
||||
canvas.appendChild(clone);
|
||||
canvas.style.cssText = `position:fixed;top:0;left:0;width:${width}px;height:${height}px;z-index:-9999;pointer-events:none;opacity:0`;
|
||||
canvas.appendChild(sceneEl.cloneNode(true));
|
||||
document.body.appendChild(canvas);
|
||||
|
||||
try {
|
||||
await waitForNextFrame();
|
||||
await waitForPaint(canvas);
|
||||
|
||||
ctx.setTransform(1, 0, 0, 1, 0, 0);
|
||||
ctx.clearRect(0, 0, width, height);
|
||||
await new Promise<void>((r) => requestAnimationFrame(() => requestAnimationFrame(() => r())));
|
||||
const ctx = canvas.getContext("2d") as CanvasRenderingContext2DWithDrawElement;
|
||||
ctx.fillStyle = bgColor;
|
||||
ctx.fillRect(0, 0, width, height);
|
||||
ctx.drawElementImage(clone, 0, 0, width, height);
|
||||
const child = canvas.firstElementChild;
|
||||
if (child) ctx.drawElementImage(child, 0, 0, width, height);
|
||||
const result = document.createElement("canvas");
|
||||
result.width = width;
|
||||
result.height = height;
|
||||
result.getContext("2d")!.drawImage(canvas, 0, 0);
|
||||
canvas.remove();
|
||||
return canvas;
|
||||
return result;
|
||||
} catch (err) {
|
||||
canvas.remove();
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
function captureSceneWithHtml2Canvas(
|
||||
sceneEl: HTMLElement,
|
||||
bgColor: string,
|
||||
width: number = DEFAULT_WIDTH,
|
||||
height: number = DEFAULT_HEIGHT,
|
||||
): Promise<HTMLCanvasElement> {
|
||||
return html2canvas(sceneEl, {
|
||||
width,
|
||||
height,
|
||||
scale: 1,
|
||||
backgroundColor: bgColor,
|
||||
logging: false,
|
||||
// Safari applies stricter canvas-taint rules than Chrome. SVG data URLs
|
||||
// with <filter> elements (e.g. feTurbulence grain backgrounds), certain
|
||||
// cross-origin images, and mask/clip-path url() refs can taint the
|
||||
// output canvas on WebKit. Without these flags, html2canvas throws
|
||||
// `SecurityError: The operation is insecure` during its own read-back
|
||||
// path and every shader transition falls through to the catch handler
|
||||
// — observed in Safari + Claude Design's cross-origin iframe sandbox.
|
||||
//
|
||||
// useCORS: send CORS headers on image fetches so cross-origin images
|
||||
// with proper `Access-Control-Allow-Origin` don't taint the
|
||||
// canvas in the first place. Strict improvement.
|
||||
// allowTaint: let html2canvas complete and return a canvas even when it
|
||||
// becomes tainted (instead of throwing). Important caveat:
|
||||
// a tainted canvas CANNOT be uploaded to WebGL via
|
||||
// `gl.texImage2D` — WebGL spec requires SecurityError on
|
||||
// non-origin-clean sources, with no opt-out. So this flag
|
||||
// only moves the failure point from html2canvas to the
|
||||
// texImage2D call in webgl.ts. In both cases `hyper-shader.ts`
|
||||
// catches the rejected promise and runs the CSS crossfade
|
||||
// fallback. Net effect: the end-user UX is the same (smooth
|
||||
// CSS fade in either case), but we get a cleaner, more
|
||||
// predictable error site and the flag is defensively
|
||||
// correct for the non-taint branches where it genuinely
|
||||
// helps (e.g., `crossOrigin="anonymous"` image fetches
|
||||
// that already had CORS headers).
|
||||
useCORS: true,
|
||||
allowTaint: true,
|
||||
ignoreElements: (el: Element) => el.tagName === "CANVAS" || el.hasAttribute("data-no-capture"),
|
||||
});
|
||||
}
|
||||
|
||||
export function captureScene(
|
||||
sceneEl: HTMLElement,
|
||||
bgColor: string,
|
||||
width: number = DEFAULT_WIDTH,
|
||||
height: number = DEFAULT_HEIGHT,
|
||||
options: CaptureSceneOptions = {},
|
||||
): Promise<HTMLCanvasElement> {
|
||||
if (!isHtmlInCanvasCaptureSupported()) {
|
||||
return captureSceneWithHtml2Canvas(sceneEl, bgColor, width, height);
|
||||
if (isHtmlInCanvasCaptureSupported() && !options.preferBrowserPaint) {
|
||||
return captureSceneWithHtmlInCanvas(sceneEl, bgColor, width, height).catch(() =>
|
||||
captureSceneWithHtml2Canvas(sceneEl, bgColor, width, height, options),
|
||||
);
|
||||
}
|
||||
return captureSceneWithHtml2Canvas(sceneEl, bgColor, width, height, options);
|
||||
}
|
||||
|
||||
function captureSceneWithHtml2Canvas(
|
||||
sceneEl: HTMLElement,
|
||||
bgColor: string,
|
||||
width: number,
|
||||
height: number,
|
||||
options: CaptureSceneOptions = {},
|
||||
): Promise<HTMLCanvasElement> {
|
||||
const captureWithRenderer = (foreignObjectRendering: boolean): Promise<HTMLCanvasElement> => {
|
||||
return html2canvas(sceneEl, {
|
||||
width,
|
||||
height,
|
||||
scale: options.scale ?? 1,
|
||||
backgroundColor: bgColor,
|
||||
logging: false,
|
||||
foreignObjectRendering,
|
||||
// Safari applies stricter canvas-taint rules than Chrome. SVG data URLs
|
||||
// with <filter> elements (e.g. feTurbulence grain backgrounds), certain
|
||||
// cross-origin images, and mask/clip-path url() refs can taint the
|
||||
// output canvas on WebKit. Without these flags, html2canvas throws
|
||||
// `SecurityError: The operation is insecure` during its own read-back
|
||||
// path and every shader transition falls through to the catch handler
|
||||
// — observed in Safari + Claude Design's cross-origin iframe sandbox.
|
||||
//
|
||||
// useCORS: send CORS headers on image fetches so cross-origin images
|
||||
// with proper `Access-Control-Allow-Origin` don't taint the
|
||||
// canvas in the first place. Strict improvement.
|
||||
// allowTaint: let html2canvas complete and return a canvas even when it
|
||||
// becomes tainted (instead of throwing). Important caveat:
|
||||
// a tainted canvas CANNOT be uploaded to WebGL via
|
||||
// `gl.texImage2D` — WebGL spec requires SecurityError on
|
||||
// non-origin-clean sources, with no opt-out. So this flag
|
||||
// only moves the failure point from html2canvas to the
|
||||
// texImage2D call in webgl.ts. The caller catches the
|
||||
// rejected promise and keeps the DOM fallback visible. Net
|
||||
// effect: the end-user UX avoids blank frames either way,
|
||||
// but we get a cleaner, more predictable error site and the
|
||||
// flag is defensively correct for the non-taint branches
|
||||
// where it genuinely helps (e.g.,
|
||||
// `crossOrigin="anonymous"` image fetches that already had
|
||||
// CORS headers).
|
||||
useCORS: true,
|
||||
allowTaint: true,
|
||||
onclone: (cloneDoc) => {
|
||||
if (!sceneEl.id) return;
|
||||
const clone = cloneDoc.getElementById(sceneEl.id);
|
||||
if (clone instanceof HTMLElement) {
|
||||
stabilizeTransformedBoxShadows(clone);
|
||||
}
|
||||
if (options.forceVisible) {
|
||||
forceSceneVisibleInClone(sceneEl, cloneDoc);
|
||||
}
|
||||
},
|
||||
ignoreElements: (el: Element) =>
|
||||
el.tagName === "CANVAS" || el.hasAttribute("data-no-capture"),
|
||||
});
|
||||
};
|
||||
|
||||
if (options.preferBrowserPaint === true) {
|
||||
return captureWithRenderer(true).catch(() => captureWithRenderer(false));
|
||||
}
|
||||
|
||||
return captureSceneWithHtmlInCanvas(sceneEl, bgColor, width, height).catch(() =>
|
||||
captureSceneWithHtml2Canvas(sceneEl, bgColor, width, height),
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Capture the incoming scene with .scene-content hidden (background + decoratives only).
|
||||
* Shows the scene behind the outgoing scene via z-index, waits 2 rAFs for font rendering,
|
||||
* captures, then restores.
|
||||
*
|
||||
* IMPORTANT: We force `visibility: visible` during capture because the HyperFrames runtime's
|
||||
* time-based visibility gate (in `packages/core/src/runtime/init.ts`) sets `style.visibility
|
||||
* = "hidden"` on every `[data-start]` element that's outside its current playback window —
|
||||
* every frame. When a shader transition fires *before* the incoming scene's `data-start`
|
||||
* boundary (the recommended "transition.time = boundary - duration/2" centered placement),
|
||||
* the runtime has `visibility: hidden` on the incoming scene. Without the visibility override
|
||||
* here, `html2canvas` captures the element as blank → shader transitions from the real
|
||||
* outgoing scene to a blank incoming texture → users see content fade/morph into the
|
||||
* background color mid-transition (a visible "blink"). Forcing `visibility: visible` only
|
||||
* for the duration of the capture fixes this without affecting what the user sees during
|
||||
* normal playback.
|
||||
*/
|
||||
export function captureIncomingScene(
|
||||
toScene: HTMLElement,
|
||||
bgColor: string,
|
||||
width: number = DEFAULT_WIDTH,
|
||||
height: number = DEFAULT_HEIGHT,
|
||||
): Promise<HTMLCanvasElement> {
|
||||
return new Promise<HTMLCanvasElement>((resolve, reject) => {
|
||||
const origZ = toScene.style.zIndex;
|
||||
const origOpacity = toScene.style.opacity;
|
||||
const origVisibility = toScene.style.visibility;
|
||||
toScene.style.zIndex = "-1";
|
||||
toScene.style.opacity = "1";
|
||||
toScene.style.visibility = "visible";
|
||||
|
||||
const contentEl = toScene.querySelector<HTMLElement>(".scene-content");
|
||||
if (contentEl) contentEl.style.visibility = "hidden";
|
||||
|
||||
const restore = () => {
|
||||
if (contentEl) contentEl.style.visibility = "";
|
||||
toScene.style.visibility = origVisibility;
|
||||
toScene.style.opacity = origOpacity;
|
||||
toScene.style.zIndex = origZ;
|
||||
};
|
||||
|
||||
requestAnimationFrame(() => {
|
||||
requestAnimationFrame(() => {
|
||||
captureScene(toScene, bgColor, width, height).then(resolve, reject).finally(restore);
|
||||
});
|
||||
});
|
||||
});
|
||||
return captureWithRenderer(false);
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -36,13 +36,14 @@ function compileShader(gl: WebGLRenderingContext, src: string, type: number): We
|
||||
return s;
|
||||
}
|
||||
|
||||
export function createProgram(gl: WebGLRenderingContext, fragSrc: string): WebGLProgram {
|
||||
if (!cachedVertexShader) {
|
||||
cachedVertexShader = compileShader(gl, vertSrc, gl.VERTEX_SHADER);
|
||||
}
|
||||
function linkProgram(
|
||||
gl: WebGLRenderingContext,
|
||||
vertexShader: WebGLShader,
|
||||
fragSrc: string,
|
||||
): WebGLProgram {
|
||||
const p = gl.createProgram();
|
||||
if (!p) throw new Error("[HyperShader] Failed to create program");
|
||||
gl.attachShader(p, cachedVertexShader);
|
||||
gl.attachShader(p, vertexShader);
|
||||
gl.attachShader(p, compileShader(gl, fragSrc, gl.FRAGMENT_SHADER));
|
||||
gl.linkProgram(p);
|
||||
if (!gl.getProgramParameter(p, gl.LINK_STATUS)) {
|
||||
@@ -51,6 +52,21 @@ export function createProgram(gl: WebGLRenderingContext, fragSrc: string): WebGL
|
||||
return p;
|
||||
}
|
||||
|
||||
export function createProgram(gl: WebGLRenderingContext, fragSrc: string): WebGLProgram {
|
||||
if (!cachedVertexShader) {
|
||||
cachedVertexShader = compileShader(gl, vertSrc, gl.VERTEX_SHADER);
|
||||
}
|
||||
return linkProgram(gl, cachedVertexShader, fragSrc);
|
||||
}
|
||||
|
||||
export function createProgramWithVertex(
|
||||
gl: WebGLRenderingContext,
|
||||
vertexSrc: string,
|
||||
fragSrc: string,
|
||||
): WebGLProgram {
|
||||
return linkProgram(gl, compileShader(gl, vertexSrc, gl.VERTEX_SHADER), fragSrc);
|
||||
}
|
||||
|
||||
export interface AccentColors {
|
||||
accent: [number, number, number];
|
||||
dark: [number, number, number];
|
||||
@@ -136,8 +152,16 @@ export function uploadTexture(
|
||||
tex: WebGLTexture,
|
||||
canvas: HTMLCanvasElement,
|
||||
): void {
|
||||
gl.bindTexture(gl.TEXTURE_2D, tex);
|
||||
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, canvas);
|
||||
uploadTextureSource(gl, tex, canvas);
|
||||
canvas.width = 0;
|
||||
canvas.height = 0;
|
||||
}
|
||||
|
||||
export function uploadTextureSource(
|
||||
gl: WebGLRenderingContext,
|
||||
tex: WebGLTexture,
|
||||
source: TexImageSource,
|
||||
): void {
|
||||
gl.bindTexture(gl.TEXTURE_2D, tex);
|
||||
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, source);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,104 @@
|
||||
export interface HyperframesLoaderProps {
|
||||
/** Status text shown below the mark. */
|
||||
title: string;
|
||||
/** Optional secondary detail line. */
|
||||
detail?: string;
|
||||
/** Optional monospace third line for IDs, counts, or percentages. */
|
||||
mono?: string;
|
||||
/** Pixel size of the mark itself; status text scales independently. */
|
||||
size?: number;
|
||||
/** Optional normalized progress value from 0 to 1. */
|
||||
progress?: number;
|
||||
}
|
||||
|
||||
export function HyperframesLoader({
|
||||
title,
|
||||
detail,
|
||||
mono,
|
||||
size = 64,
|
||||
progress,
|
||||
}: HyperframesLoaderProps) {
|
||||
const boundedProgress =
|
||||
typeof progress === "number" && Number.isFinite(progress)
|
||||
? Math.min(1, Math.max(0, progress))
|
||||
: undefined;
|
||||
const markFrameSize = Math.round(size * 1.16);
|
||||
|
||||
return (
|
||||
<div className="hf-loader" draggable={false}>
|
||||
<div
|
||||
className="hf-loader-mark-frame"
|
||||
style={{ width: markFrameSize, height: markFrameSize }}
|
||||
draggable={false}
|
||||
>
|
||||
<svg
|
||||
className="hf-loader-mark"
|
||||
width={size}
|
||||
height={size}
|
||||
viewBox="0 0 100 100"
|
||||
fill="none"
|
||||
xmlns="http://www.w3.org/2000/svg"
|
||||
aria-hidden="true"
|
||||
>
|
||||
<g className="hf-loader-mark__mark" transform="translate(50 50)">
|
||||
<g className="hf-loader-mark__core" transform="scale(1)" opacity=".92">
|
||||
<g transform="translate(-50 -50)">
|
||||
<path
|
||||
d="M10.1851 57.8021L33.1145 73.8313C36.2202 75.9978 41.5173 73.5433 42.4816 69.4984L51.7611 30.4271C52.7253 26.3822 48.5802 23.9277 44.4602 26.0942L13.917 42.1235C6.96677 45.7676 4.97564 54.1579 10.1851 57.8021Z"
|
||||
fill="url(#hf-loader-grad-left)"
|
||||
/>
|
||||
<path
|
||||
d="M87.5129 57.5141L56.9696 73.5433C52.8371 75.7098 48.7046 73.2553 49.6688 69.2104L58.9483 30.1391C59.9125 26.0942 65.2097 23.6397 68.3154 25.8062L91.2447 41.8354C96.4668 45.4796 94.4631 53.8699 87.5129 57.5141Z"
|
||||
fill="url(#hf-loader-grad-right)"
|
||||
/>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
<defs>
|
||||
<linearGradient
|
||||
id="hf-loader-grad-left"
|
||||
x1="48.5676"
|
||||
y1="25"
|
||||
x2="44.7804"
|
||||
y2="71.9384"
|
||||
gradientUnits="userSpaceOnUse"
|
||||
>
|
||||
<stop stopColor="#06E3FA" />
|
||||
<stop offset="1" stopColor="#4FDB5E" />
|
||||
</linearGradient>
|
||||
<linearGradient
|
||||
id="hf-loader-grad-right"
|
||||
x1="54.8282"
|
||||
y1="73.8392"
|
||||
x2="72.0989"
|
||||
y2="32.8932"
|
||||
gradientUnits="userSpaceOnUse"
|
||||
>
|
||||
<stop stopColor="#06E3FA" />
|
||||
<stop offset="1" stopColor="#4FDB5E" />
|
||||
</linearGradient>
|
||||
</defs>
|
||||
</svg>
|
||||
</div>
|
||||
<div className="hf-loader-title">{title}</div>
|
||||
{detail && <div className="hf-loader-detail">{detail}</div>}
|
||||
{boundedProgress !== undefined && (
|
||||
<div className="hf-loader-progress" aria-hidden="true">
|
||||
<div
|
||||
className="hf-loader-progress__fill"
|
||||
style={{ transform: `scaleX(${boundedProgress})` }}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
{mono && <div className="hf-loader-mono">{mono}</div>}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
export function StatusFrame(props: HyperframesLoaderProps) {
|
||||
return (
|
||||
<div className="hf-frame">
|
||||
<HyperframesLoader {...props} />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -1,2 +1,4 @@
|
||||
// Minimal UI primitives for studio canvas components
|
||||
export { Button, IconButton } from "./Button";
|
||||
export { HyperframesLoader, StatusFrame } from "./HyperframesLoader";
|
||||
export type { HyperframesLoaderProps } from "./HyperframesLoader";
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { forwardRef, useRef, useState } from "react";
|
||||
import { forwardRef, useEffect, useRef, useState } from "react";
|
||||
import { isLottieAnimationLoaded } from "@hyperframes/core/runtime/lottie-readiness";
|
||||
import { useMountEffect } from "../../hooks/useMountEffect";
|
||||
import { HyperframesLoader } from "../../components/ui";
|
||||
// NOTE: importing "@hyperframes/player" registers a class extending HTMLElement
|
||||
// at module load, which throws under SSR. Defer the import to the mount effect
|
||||
// so it only runs in the browser.
|
||||
@@ -16,6 +17,19 @@ interface HyperframesPlayerElement extends HTMLElement {
|
||||
iframeElement: HTMLIFrameElement;
|
||||
}
|
||||
|
||||
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||
return typeof value === "object" && value !== null;
|
||||
}
|
||||
|
||||
function getShaderTransitionLoading(event: Event): boolean | null {
|
||||
if (!(event instanceof CustomEvent)) return null;
|
||||
const detail: unknown = event.detail;
|
||||
if (!isRecord(detail)) return null;
|
||||
const state = detail.state;
|
||||
if (!isRecord(state)) return null;
|
||||
return state.loading === true && state.ready !== true;
|
||||
}
|
||||
|
||||
// Assets are considered ready when every `<video>`/`<audio>` has enough data
|
||||
// to play through without buffering, and every registered Lottie animation has
|
||||
// finished loading.
|
||||
@@ -62,7 +76,11 @@ export const Player = forwardRef<HTMLIFrameElement, PlayerProps>(
|
||||
const containerRef = useRef<HTMLDivElement>(null);
|
||||
const loadCountRef = useRef(0);
|
||||
const assetPollRef = useRef<ReturnType<typeof setInterval> | null>(null);
|
||||
const assetFadeRef = useRef<ReturnType<typeof setTimeout> | null>(null);
|
||||
const [assetsLoading, setAssetsLoading] = useState(false);
|
||||
const [assetOverlayVisible, setAssetOverlayVisible] = useState(false);
|
||||
const [assetOverlayFading, setAssetOverlayFading] = useState(false);
|
||||
const [shaderTransitionLoading, setShaderTransitionLoading] = useState(false);
|
||||
|
||||
useMountEffect(() => {
|
||||
const container = containerRef.current;
|
||||
@@ -78,6 +96,8 @@ export const Player = forwardRef<HTMLIFrameElement, PlayerProps>(
|
||||
// Create the web component imperatively to avoid JSX custom-element typing.
|
||||
const player = document.createElement("hyperframes-player") as HyperframesPlayerElement;
|
||||
const src = directUrl || `/api/projects/${projectId}/preview`;
|
||||
player.setAttribute("shader-capture-scale", "1");
|
||||
player.setAttribute("shader-loading", "player");
|
||||
player.setAttribute("src", src);
|
||||
player.setAttribute("width", String(portrait ? 1080 : 1920));
|
||||
player.setAttribute("height", String(portrait ? 1920 : 1080));
|
||||
@@ -99,9 +119,16 @@ export const Player = forwardRef<HTMLIFrameElement, PlayerProps>(
|
||||
const preventToggle = (e: Event) => e.stopImmediatePropagation();
|
||||
player.addEventListener("click", preventToggle, { capture: true });
|
||||
|
||||
const handleShaderTransitionState = (event: Event) => {
|
||||
const loading = getShaderTransitionLoading(event);
|
||||
if (loading !== null) setShaderTransitionLoading(loading);
|
||||
};
|
||||
player.addEventListener("shadertransitionstate", handleShaderTransitionState);
|
||||
|
||||
// Forward the iframe's native load event to the studio's onIframeLoad.
|
||||
const handleLoad = () => {
|
||||
loadCountRef.current++;
|
||||
setShaderTransitionLoading(false);
|
||||
// Reveal animation on reload (hot-reload, composition switch)
|
||||
if (loadCountRef.current > 1) {
|
||||
container.classList.remove("preview-revealing");
|
||||
@@ -151,6 +178,7 @@ export const Player = forwardRef<HTMLIFrameElement, PlayerProps>(
|
||||
cleanup = () => {
|
||||
iframe.removeEventListener("load", handleLoad);
|
||||
player.removeEventListener("click", preventToggle, { capture: true });
|
||||
player.removeEventListener("shadertransitionstate", handleShaderTransitionState);
|
||||
if (assetPollRef.current) clearInterval(assetPollRef.current);
|
||||
assetPollRef.current = null;
|
||||
container.removeChild(player);
|
||||
@@ -169,13 +197,57 @@ export const Player = forwardRef<HTMLIFrameElement, PlayerProps>(
|
||||
};
|
||||
});
|
||||
|
||||
useEffect(() => {
|
||||
if (assetFadeRef.current) {
|
||||
clearTimeout(assetFadeRef.current);
|
||||
assetFadeRef.current = null;
|
||||
}
|
||||
|
||||
if (assetsLoading) {
|
||||
setAssetOverlayVisible(true);
|
||||
setAssetOverlayFading(false);
|
||||
return;
|
||||
}
|
||||
|
||||
setAssetOverlayFading(true);
|
||||
assetFadeRef.current = setTimeout(() => {
|
||||
setAssetOverlayVisible(false);
|
||||
setAssetOverlayFading(false);
|
||||
assetFadeRef.current = null;
|
||||
}, 240);
|
||||
|
||||
return () => {
|
||||
if (assetFadeRef.current) {
|
||||
clearTimeout(assetFadeRef.current);
|
||||
assetFadeRef.current = null;
|
||||
}
|
||||
};
|
||||
}, [assetsLoading]);
|
||||
|
||||
const showAssetOverlay = assetOverlayVisible && !shaderTransitionLoading;
|
||||
|
||||
return (
|
||||
<div className="relative w-full h-full max-w-full max-h-full overflow-hidden bg-black flex items-center justify-center">
|
||||
<div ref={containerRef} className="w-full h-full" />
|
||||
{assetsLoading && (
|
||||
<div className="absolute inset-0 bg-black/80 flex flex-col items-center justify-center z-20 pointer-events-none">
|
||||
<div className="w-8 h-8 border-2 border-white/20 border-t-white rounded-full animate-spin" />
|
||||
<span className="text-white/60 text-xs mt-3">Loading assets…</span>
|
||||
{showAssetOverlay && (
|
||||
<div
|
||||
className="absolute inset-0 bg-black flex items-center justify-center z-20 select-none"
|
||||
data-hyperframes-ignore=""
|
||||
draggable={false}
|
||||
style={{
|
||||
opacity: assetOverlayFading ? 0 : 1,
|
||||
pointerEvents: assetOverlayFading ? "none" : "auto",
|
||||
transition: "opacity 240ms ease-out",
|
||||
}}
|
||||
onDragStart={(event) => event.preventDefault()}
|
||||
onMouseDown={(event) => event.preventDefault()}
|
||||
onPointerDown={(event) => event.preventDefault()}
|
||||
>
|
||||
<HyperframesLoader
|
||||
title="Preparing preview assets"
|
||||
detail="Waiting for media and motion assets before playback starts."
|
||||
size={56}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
@@ -49,3 +49,115 @@ body {
|
||||
.cm-editor.cm-focused {
|
||||
outline: none;
|
||||
}
|
||||
|
||||
/*
|
||||
* HyperFrames brand loader. Shared by preview overlays that need a calm,
|
||||
* branded loading state instead of a generic spinner.
|
||||
*/
|
||||
.hf-loader {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
gap: 0.75rem;
|
||||
width: min(34rem, 100%);
|
||||
padding: 1.5rem;
|
||||
box-sizing: border-box;
|
||||
text-align: center;
|
||||
cursor: default;
|
||||
user-select: none;
|
||||
-webkit-user-select: none;
|
||||
-webkit-user-drag: none;
|
||||
}
|
||||
|
||||
.hf-frame {
|
||||
display: grid;
|
||||
place-items: center;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
min-height: 12rem;
|
||||
border: 1px solid rgba(255, 255, 255, 0.08);
|
||||
background: rgba(0, 0, 0, 0.52);
|
||||
}
|
||||
|
||||
.hf-loader-mark-frame {
|
||||
display: grid;
|
||||
place-items: center;
|
||||
overflow: visible;
|
||||
transform-origin: 50% 50%;
|
||||
user-select: none;
|
||||
-webkit-user-select: none;
|
||||
-webkit-user-drag: none;
|
||||
}
|
||||
|
||||
.hf-loader-mark {
|
||||
display: block;
|
||||
overflow: visible;
|
||||
filter: drop-shadow(0 0 7px rgba(79, 219, 94, 0.2));
|
||||
user-select: none;
|
||||
-webkit-user-select: none;
|
||||
-webkit-user-drag: none;
|
||||
}
|
||||
|
||||
.hf-loader-title {
|
||||
font-family:
|
||||
Inter,
|
||||
-apple-system,
|
||||
BlinkMacSystemFont,
|
||||
"Segoe UI",
|
||||
sans-serif;
|
||||
font-size: 1rem;
|
||||
font-weight: 600;
|
||||
letter-spacing: 0;
|
||||
color: var(--hf-heading, #f4f4f5);
|
||||
max-width: 100%;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.hf-loader-detail {
|
||||
max-width: 32rem;
|
||||
min-height: 2.5rem;
|
||||
overflow: hidden;
|
||||
color: var(--hf-text-secondary, rgba(244, 244, 245, 0.68));
|
||||
font-family:
|
||||
Inter,
|
||||
-apple-system,
|
||||
BlinkMacSystemFont,
|
||||
"Segoe UI",
|
||||
sans-serif;
|
||||
font-size: 0.82rem;
|
||||
line-height: 1.6;
|
||||
}
|
||||
|
||||
.hf-loader-mono {
|
||||
width: min(36rem, 100%);
|
||||
min-height: 1.5rem;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
color: var(--hf-text-tertiary, rgba(244, 244, 245, 0.46));
|
||||
font-family: "IBM Plex Mono", "SF Mono", "Fira Code", monospace;
|
||||
font-size: 0.75rem;
|
||||
letter-spacing: 0;
|
||||
font-variant-numeric: tabular-nums;
|
||||
}
|
||||
|
||||
.hf-loader-progress {
|
||||
width: min(18rem, 72vw);
|
||||
height: 0.375rem;
|
||||
overflow: hidden;
|
||||
border-radius: 999px;
|
||||
background: rgba(255, 255, 255, 0.1);
|
||||
}
|
||||
|
||||
.hf-loader-progress__fill {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
transform: scaleX(0);
|
||||
transform-origin: left center;
|
||||
border-radius: inherit;
|
||||
background: linear-gradient(90deg, #06e3fa, #4fdb5e);
|
||||
transition: transform 160ms ease;
|
||||
}
|
||||
|
||||
@@ -8,12 +8,13 @@ import {
|
||||
lstatSync,
|
||||
realpathSync,
|
||||
} from "node:fs";
|
||||
import { join, resolve } from "node:path";
|
||||
import { isAbsolute, join, relative, resolve } from "node:path";
|
||||
import type {
|
||||
StudioApiAdapter,
|
||||
ResolvedProject,
|
||||
RenderJobState,
|
||||
} from "@hyperframes/core/studio-api";
|
||||
import { createProjectSignature } from "../core/src/studio-api/helpers/projectSignature";
|
||||
import { createRetryingModuleLoader, ensureProducerDist } from "./vite.producer";
|
||||
import { readNodeRequestBody } from "./vite.request-body.js";
|
||||
import { seekThumbnailPreview } from "./vite.thumbnail";
|
||||
@@ -56,11 +57,21 @@ interface ScreenshotClip {
|
||||
height: number;
|
||||
}
|
||||
|
||||
function isPathWithin(parentDir: string, childPath: string): boolean {
|
||||
const childRelativePath = relative(resolve(parentDir), resolve(childPath));
|
||||
return (
|
||||
childRelativePath === "" ||
|
||||
(!childRelativePath.startsWith("..") && !isAbsolute(childRelativePath))
|
||||
);
|
||||
}
|
||||
|
||||
// ── Vite adapter for the shared studio API ───────────────────────────────────
|
||||
|
||||
function createViteAdapter(dataDir: string, server: ViteDevServer): StudioApiAdapter {
|
||||
// Lazy-load the bundler via Vite's SSR module loader
|
||||
let _bundler: ((dir: string) => Promise<string>) | null = null;
|
||||
let _bundler:
|
||||
| ((dir: string, options?: { runtime?: "inline" | "placeholder" }) => Promise<string>)
|
||||
| null = null;
|
||||
let _producerModulePromise: Promise<{
|
||||
createRenderJob: (config: {
|
||||
fps: 24 | 30 | 60;
|
||||
@@ -74,11 +85,17 @@ function createViteAdapter(dataDir: string, server: ViteDevServer): StudioApiAda
|
||||
onProgress?: (job: { progress: number; currentStage?: string }) => void,
|
||||
) => Promise<void>;
|
||||
}> | null = null;
|
||||
const projectSignatureCache = new Map<string, string>();
|
||||
server.watcher.on("all", (_event, file) => {
|
||||
for (const projectDir of projectSignatureCache.keys()) {
|
||||
if (isPathWithin(projectDir, file)) projectSignatureCache.delete(projectDir);
|
||||
}
|
||||
});
|
||||
const getBundler = async () => {
|
||||
if (!_bundler) {
|
||||
try {
|
||||
const mod = await server.ssrLoadModule("@hyperframes/core/compiler");
|
||||
_bundler = (dir: string) => mod.bundleToSingleHtml(dir);
|
||||
_bundler = (dir, options) => mod.bundleToSingleHtml(dir, options);
|
||||
} catch (err) {
|
||||
console.warn("[Studio] Failed to load compiler, previews will use raw HTML:", err);
|
||||
_bundler = null as never;
|
||||
@@ -171,8 +188,10 @@ function createViteAdapter(dataDir: string, server: ViteDevServer): StudioApiAda
|
||||
async bundle(dir: string) {
|
||||
const bundler = await getBundler();
|
||||
if (!bundler) return null;
|
||||
let html = await bundler(dir);
|
||||
// Fix empty runtime src from bundler — point to the CDN runtime
|
||||
// Studio vite preview: bundler emits an empty `src=""` placeholder so we
|
||||
// can point it at the local /api/runtime.js endpoint. Cached by the browser
|
||||
// across composition hot-reloads instead of being inlined fresh each time.
|
||||
let html = await bundler(dir, { runtime: "placeholder" });
|
||||
html = html.replace(
|
||||
'data-hyperframes-preview-runtime="1" src=""',
|
||||
`data-hyperframes-preview-runtime="1" src="${this.runtimeUrl}"`,
|
||||
@@ -180,6 +199,16 @@ function createViteAdapter(dataDir: string, server: ViteDevServer): StudioApiAda
|
||||
return html;
|
||||
},
|
||||
|
||||
getProjectSignature(projectDir: string): string {
|
||||
const cacheKey = resolve(projectDir);
|
||||
const cached = projectSignatureCache.get(cacheKey);
|
||||
if (cached) return cached;
|
||||
|
||||
const signature = createProjectSignature(cacheKey);
|
||||
projectSignatureCache.set(cacheKey, signature);
|
||||
return signature;
|
||||
},
|
||||
|
||||
async lint(html: string, opts?: { filePath?: string }) {
|
||||
const mod = await server.ssrLoadModule("@hyperframes/core/lint");
|
||||
return mod.lintHyperframeHtml(html, opts);
|
||||
|
||||
@@ -108,12 +108,67 @@ Remove the background from a video or image so the subject (typically a person
|
||||
npx hyperframes remove-background subject.mp4 -o transparent.webm # default: VP9 alpha WebM
|
||||
npx hyperframes remove-background subject.mp4 -o transparent.mov # ProRes 4444 (editing)
|
||||
npx hyperframes remove-background portrait.jpg -o cutout.png # single-image cutout
|
||||
npx hyperframes remove-background subject.mp4 -o subject.webm \
|
||||
--background-output plate.webm # both layers in one pass
|
||||
npx hyperframes remove-background subject.mp4 -o transparent.webm --device cpu
|
||||
npx hyperframes remove-background --info # detected providers
|
||||
```
|
||||
|
||||
Uses `u2net_human_seg` (MIT). First run downloads ~168 MB of weights to `~/.cache/hyperframes/background-removal/models/`.
|
||||
|
||||
### Layer separation (`--background-output`)
|
||||
|
||||
Pass `--background-output` (or `-b`) to emit a **second** transparent video alongside the cutout: same source RGB, alpha is `255 − mask` instead of `mask`. The cutout is the subject with a transparent background; the plate is the original surroundings with a transparent hole where the subject was.
|
||||
|
||||
| File | Alpha is… | Use it for |
|
||||
| -------------------------------- | --------------------------------------------------------- | --------------------------------------------------------------- |
|
||||
| `-o subject.webm` | The mask — subject opaque, background transparent | Foreground layer, place on top |
|
||||
| `--background-output plate.webm` | Inverse — surroundings opaque, subject region transparent | Bottom layer; put text or graphics between this and the subject |
|
||||
|
||||
Both outputs share the same `--quality` preset and run from a single inference pass — encode cost roughly doubles, segmentation cost stays the same. Only valid for video inputs and `.webm`/`.mov` outputs.
|
||||
|
||||
**Hole-cut plate, not an inpainted clean plate.** The subject region in `plate.webm` is fully transparent — composite something opaque under it to fill the hole. The single test for whether `--background-output` is the right tool: _will anything ever be visible through the subject's silhouette where the subject used to be?_
|
||||
|
||||
| Use case | Right tool |
|
||||
| ----------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- |
|
||||
| Text/graphics between the cutout and the plate (this command's reason for existing) | **Hole-cut** (`--background-output`) |
|
||||
| Subject onto an unrelated scene | Just `subject.webm`; ignore the plate |
|
||||
| Show the room _without_ the person, alone over no other content | **Clean plate** — needs an inpainter (LaMa, ProPainter, E2FGVI). Not this command. |
|
||||
| Replace the subject with a different subject | **Clean plate** — same as above |
|
||||
|
||||
If a user asks for "the room with the person removed" and intends to display it standalone, do **not** reach for `--background-output`. Tell them they need an inpainter.
|
||||
|
||||
Typical layered composition (the canonical hole-cut use case):
|
||||
|
||||
```html
|
||||
<!-- z=1 the inverse-alpha plate fills everything except the subject region -->
|
||||
<video
|
||||
src="plate.webm"
|
||||
data-start="0"
|
||||
data-duration="6"
|
||||
data-track-index="0"
|
||||
muted
|
||||
playsinline
|
||||
></video>
|
||||
|
||||
<!-- z=2 graphics / text live between the two layers -->
|
||||
<h1 id="headline" style="z-index:2; ...">MAKE IT IN HYPERFRAMES</h1>
|
||||
|
||||
<!-- z=3 the cutout floats the subject back over the headline -->
|
||||
<div class="cutout-wrap" style="position:absolute;inset:0;z-index:3">
|
||||
<video
|
||||
src="subject.webm"
|
||||
data-start="0"
|
||||
data-duration="6"
|
||||
data-track-index="1"
|
||||
muted
|
||||
playsinline
|
||||
></video>
|
||||
</div>
|
||||
```
|
||||
|
||||
This is functionally equivalent to the text-behind-subject pattern below, but you don't need the original `presenter.mp4` in the project — the plate replaces it. Useful when you want to ship just the two transparent layers and let the user drop arbitrary content between them.
|
||||
|
||||
### Output Format
|
||||
|
||||
| Format | When |
|
||||
|
||||
Reference in New Issue
Block a user