feat(engine,producer): drawElement fast-capture default-on with runtime self-verification safety net (#1998)

* feat(engine,producer): drawElement fast-capture default-on with runtime self-verification safety net

Flip useDrawElement + worker-encode defaults on (HF_DE_BATCH default 4),
clamped in resolveConfig to hosts where drawElement can engage (macOS +
hardware-GPU browser) so page-side shader compositing is untouched
everywhere else; explicit env opt-in keeps attempt-and-gate semantics.

Safety net makes default-on safe: the compile/init gates catch predictable
incompatibility; this catches the intermittent residue no static analysis
can see (stale paints, dropped background images, transient blank frames).

- engine: captureDeVerificationFrames — K=4 (HF_DE_VERIFY) ground-truth
  screenshots at init, after gates + armStaticDedup, BEFORE canvas
  injection (post-injection screenshots show the canvas bitmap, not the
  DOM). Runs the video-injection hook per sample; double-captures so
  rAF-driven text counters settle (a single immediate screenshot captures
  stale text and false-positives). Skips png, <10 frames, implausible
  __hf.duration (infinite-repeat GSAP sentinel).
- producer: guardFrame on both worker-encode drains — rolling-median blank
  guard with retry-once at drain (byte-identical retry ⇒ deterministic dark
  frame, accepted; retry save/restores the static-dedup anchor) + ffmpeg
  PSNR self-verify vs ground truth (HF_DE_VERIFY_MIN_DB, default 32dB;
  natural agreement ≥45dB, damage ≤25dB). Breach dumps the frame pair to
  tmpdir and throws DrawElementVerificationError.
- orchestrator: one-shot retry — on verification error the whole render
  re-runs with forceScreenshot (slower, never wrong); telemetry flag
  deSelfVerifyFallback.
- tooling: de-canary-suite.sh (7-comp release gate with expected verdicts),
  de-gatecheck.sh (init-only corpus routing classifier), we-render.mjs.

Validated: canary suite 7/7; 611-comp routing sample 54% drawelement /
37.5% gated / 8.3% comp-defect; 12/12 risk-band renders clean on bare
defaults (48/48 verify samples); engine suite 888 passed; caught two real
intermittent damage classes in the wild (background-image drop, root-props
offset) that previously shipped silently.

Kill switches: PRODUCER_EXPERIMENTAL_FAST_CAPTURE=false,
HF_DE_WORKER_ENCODE=false, HF_DE_BATCH=0, HF_DE_VERIFY=0.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(engine,producer): harden the drawElement self-verification net (max code-review findings)

15 confirmed findings from the adversarial review of the default-on flip;
the load-bearing five:

- Ground-truth capture no longer scrubs GSAP state: seek(0) + forced frame
  FIRST (lazy .from()/overlap tweens record start values on first seek —
  mid-timeline scrubs corrupted them for the whole render, and since DE
  frames and truth shared the corruption, PSNR passed on damaged output),
  then ascending even-spread fractions, page left at frame 0.
- Default-on drawElement is confined to the verified path: resolveConfig
  requires worker-encode (the drain that runs the net), the orchestrator
  disengages the default when the render takes the disk path or parallel
  capture (no drain verification there), and closes a drawElement-initialized
  probe session rather than letting the unverified path reuse it. Explicit
  PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true keeps old attempt-and-gate behavior.
- Blank-frame retry can no longer splice wrong-frame pixels: recapture goes
  through recaptureDrawElementFrameForVerify — no static-dedup shortcut
  (lastEncodeResult runs ahead of the drain) and no "No cached paint record"
  screenshot fallback (post-injection that captures the canvas = the LAST
  drawn frame); any recapture failure falls back the whole render.
- Verify indices derive from the producer-resolved duration
  (CaptureOptions.compositionDurationSeconds) instead of raw __hf.duration,
  so samples always land inside the drained range.
- The platform clamp accepts "auto" GPU mode — the stock CLI resolves auto,
  and the literal-"hardware" clamp made default-on a no-op for the primary
  audience (masked in validation by explicitly-set env).

Also: NaN-safe env parses (HF_DE_VERIFY / HF_DE_VERIFY_MIN_DB / HF_DE_BATCH);
video comps skip verification when the session has no frame injector (probe
sessions — black-video truth false-positived); psnr infrastructure failures
skip the sample instead of failing the render; boundary-saturated sample
indices are skipped; shader-transition comps prefer page-side compositing
over default drawElement and compile-gated comps get page-side compositing
restored; observability.clearFailure un-brands the recovered first streaming
attempt; canary suite exempts known-marginal "any" comps from the cross-path
PSNR gate; dead we-render options removed; clamp tests pin their env.

Validated: canary suite 7/7; auto-GPU bare render engages the full stack;
disk-path and worker-encode-off renders disengage default drawElement;
malformed HF_DE_VERIFY_MIN_DB still verifies at the default threshold;
engine suite 890 passed.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(engine,producer): review fixes — PSC intent, verify-threshold clamp, fail-closed canaries

Addresses miguel-heygen's review on #1998:

- Page-side compositing restore preserves explicit caller intent (blocker):
  resolveConfig now records pageSideCompositingAutoDisabled only when IT
  turned page-side compositing off because drawElement was on; the
  compile-time drawElement gates restore page-side compositing only when
  that flag is set. An explicit enablePageSideCompositing:false from the
  programmatic API or HF_PAGE_SIDE_COMPOSITING=false stays off. Pinned by
  two config tests.
- HF_DE_VERIFY_MIN_DB clamped to [10, 60] with a warning on out-of-range
  values: below ~10dB the check passes severe damage; above ~60dB natural
  encoder differences force a screenshot fallback on every verified render.
- de-canary-suite.sh + de-gatecheck.sh run under set -euo pipefail with
  explicit `|| true` on expected-nonzero commands (render exits handled by
  the suite's own checks, grep no-match, kill/pkill/wait races) and a hard
  FAIL when the PSNR compare produces no value — release canaries fail
  closed. Full suite re-run green (7/7) under the new flags.

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5 <[email protected]>
This commit is contained in:
Vance Ingalls
2026-07-06 16:29:30 -07:00
committed by GitHub
co-authored by Claude Fable 5
parent ad7dce3fc2
commit febdbfc7c4
17 changed files with 764 additions and 88 deletions
+1
View File
@@ -14,6 +14,7 @@
"packages/producer/src/runtime-conformance.ts",
"packages/producer/src/benchmark.ts",
"packages/producer/scripts/generate-font-data.ts",
"packages/producer/we-render.mjs",
"packages/producer/scripts/validate-fast-video.ts",
"packages/cli/scripts/generate-font-data.ts",
"packages/engine/scripts/test-fitTextFontSize-browser.ts",
+6 -5
View File
@@ -361,11 +361,12 @@ export default defineCommand({
"experimental-fast-capture": {
type: "boolean",
description:
"EXPERIMENTAL. Capture frames via Chrome's drawElementImage API " +
"instead of Page.captureScreenshot — reads DOM paint records directly, " +
"~46% faster on GPU. Transparent (PNG) renders on SwiftShader (Docker) " +
"auto-fall back to screenshot capture. Incompatible with page-side " +
"shader compositing. Default: false. Env: PRODUCER_EXPERIMENTAL_FAST_CAPTURE.",
"Capture frames via Chrome's drawElementImage API instead of " +
"Page.captureScreenshot — reads DOM paint records directly, ~2x faster. " +
"Default: on where it can engage (macOS + hardware-GPU browser); " +
"incompatible compositions and self-verification failures fall back to " +
"screenshot capture automatically. Pass =false to disable. " +
"Env: PRODUCER_EXPERIMENTAL_FAST_CAPTURE.",
// No `default` — an omitted flag must stay `undefined` so the `!= null`
// guard below leaves PRODUCER_EXPERIMENTAL_FAST_CAPTURE untouched and the
// env fallback survives (matches the --low-memory-mode idiom).
+47 -2
View File
@@ -222,17 +222,53 @@ describe("resolveConfig", () => {
});
describe("useDrawElement (PRODUCER_EXPERIMENTAL_FAST_CAPTURE)", () => {
it("defaults to false", () => {
it("default is clamped off on software-GPU hosts (page-side compositing preserved)", () => {
setEnv("PRODUCER_BROWSER_GPU_MODE", "software");
unsetEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE");
unsetEnv("HF_DE_WORKER_ENCODE");
const config = resolveConfig();
expect(config.useDrawElement).toBe(false);
expect(config.enablePageSideCompositing).toBe(true);
});
it("default engages on macOS with a hardware-GPU browser", () => {
setEnv("PRODUCER_BROWSER_GPU_MODE", "hardware");
unsetEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE");
unsetEnv("HF_DE_WORKER_ENCODE");
const config = resolveConfig();
expect(config.useDrawElement).toBe(process.platform === "darwin");
});
it("default engages on macOS with auto GPU mode (the stock CLI path)", () => {
setEnv("PRODUCER_BROWSER_GPU_MODE", "auto");
unsetEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE");
unsetEnv("HF_DE_WORKER_ENCODE");
const config = resolveConfig();
expect(config.useDrawElement).toBe(process.platform === "darwin");
});
it("default requires worker-encode (the verified drain)", () => {
setEnv("PRODUCER_BROWSER_GPU_MODE", "hardware");
setEnv("HF_DE_WORKER_ENCODE", "false");
unsetEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE");
const config = resolveConfig();
expect(config.useDrawElement).toBe(false);
});
it("enabled when PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true", () => {
it("explicit env opt-in skips the platform clamp", () => {
setEnv("PRODUCER_BROWSER_GPU_MODE", "software");
setEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE", "true");
const config = resolveConfig();
expect(config.useDrawElement).toBe(true);
});
it("env kill switch wins over the default", () => {
setEnv("PRODUCER_BROWSER_GPU_MODE", "hardware");
setEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE", "false");
const config = resolveConfig();
expect(config.useDrawElement).toBe(false);
});
it("explicit override wins over the env var", () => {
setEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE", "true");
const config = resolveConfig({ useDrawElement: false });
@@ -243,6 +279,15 @@ describe("resolveConfig", () => {
const config = resolveConfig({ useDrawElement: true, enablePageSideCompositing: true });
expect(config.useDrawElement).toBe(true);
expect(config.enablePageSideCompositing).toBe(false);
// The auto-disable is recorded so compile-time gates can restore it.
expect(config.pageSideCompositingAutoDisabled).toBe(true);
});
it("does NOT mark auto-disabled when the caller explicitly opted out of page-side compositing", () => {
const config = resolveConfig({ useDrawElement: true, enablePageSideCompositing: false });
expect(config.enablePageSideCompositing).toBe(false);
// Explicit caller intent — a compile-time drawElement gate must not restore it.
expect(config.pageSideCompositingAutoDisabled).not.toBe(true);
});
it("leaves page-side compositing on when fast capture is off", () => {
+56 -13
View File
@@ -64,20 +64,30 @@ export interface EngineConfig {
*/
staticFrameDedup: boolean;
/**
* EXPERIMENTAL. Use drawElementImage for frame capture (requires the
* CanvasDrawElement Chrome flag, added globally in buildChromeArgs).
* Surfaced via the CLI `--experimental-fast-capture` flag.
* Env fallback: `PRODUCER_EXPERIMENTAL_FAST_CAPTURE`.
* Use drawElementImage for frame capture (requires the CanvasDrawElement
* Chrome flag, added globally in buildChromeArgs). Default ON, clamped in
* `resolveConfig` to hosts where it can actually engage (macOS + hardware-GPU
* browser); compile/init gates and the runtime self-verification net route
* incompatible or damaged renders back to screenshot capture.
* Kill switch: `PRODUCER_EXPERIMENTAL_FAST_CAPTURE=false` (or the CLI
* `--experimental-fast-capture=false`).
*/
useDrawElement: boolean;
/**
* EXPERIMENTAL. Pipeline JPEG encode into an in-page OffscreenCanvas Worker
* for the drawElement fast-capture path (macOS hardware GPU only). The worker
* encodes frame N while the main thread seeks+paints frame N+1, targeting
* ~1.65–1.96× wall-time speedup. No-op unless `useDrawElement` is also true.
* Default: off. Env: `HF_DE_WORKER_ENCODE=true`.
* Pipeline JPEG encode into an in-page OffscreenCanvas Worker for the
* drawElement fast-capture path (macOS hardware GPU only). The worker
* encodes frame N while the main thread seeks+paints frame N+1
* (~1.65–1.96× wall-time speedup). No-op unless `useDrawElement` is also
* true. Kill switch: `HF_DE_WORKER_ENCODE=false`.
*/
enableDrawElementWorkerEncode: boolean;
/**
* INTERNAL. Set by resolveConfig when it disabled enablePageSideCompositing
* solely because drawElement was on. Lets the producer's compile-time gates
* restore page-side compositing without overriding an explicit caller/env
* opt-out. Not intended to be set by callers.
*/
pageSideCompositingAutoDisabled?: boolean;
/**
* Low-memory render profile. When `true`, the orchestrator collapses the
* pipeline to its cheapest shape on memory-constrained hosts: it skips the
@@ -250,8 +260,8 @@ export const DEFAULT_CONFIG: EngineConfig = {
protocolTimeout: 300_000,
forceScreenshot: false,
staticFrameDedup: true,
useDrawElement: false,
enableDrawElementWorkerEncode: false,
useDrawElement: true,
enableDrawElementWorkerEncode: true,
// Auto-detected per host in `resolveConfig`; defaults off for the raw
// DEFAULT_CONFIG (used directly by tests and worker-sizing fallbacks).
lowMemoryMode: false,
@@ -516,15 +526,48 @@ export function resolveConfig(overrides?: Partial<EngineConfig>): EngineConfig {
...overrides,
};
// Default-on drawElement is clamped to hosts where it can actually engage
// (macOS with a non-software-GPU browser; SwiftShader drops transparent
// sub-layers — crbug 521434899). "auto" passes the clamp: the stock CLI
// resolves GPU mode to auto, which probes to hardware on real Macs — and if
// it resolves to software after all, the SwiftShader init-time gate still
// routes the session to the screenshot baseline. Without the clamp, the
// default would needlessly disable page-side shader compositing (below) on
// Linux/Docker hosts where DE never runs. An EXPLICIT opt-in (env or caller override)
// skips the clamp and keeps the old semantics — attempt DE, let the
// init-time gates route away — which debugging relies on.
const explicitDrawElementOptIn =
env("PRODUCER_EXPERIMENTAL_FAST_CAPTURE") === "true" || overrides?.useDrawElement === true;
if (
merged.useDrawElement &&
!explicitDrawElementOptIn &&
!(process.platform === "darwin" && merged.browserGpuMode !== "software")
) {
merged.useDrawElement = false;
}
// The runtime self-verification net lives in the worker-encode drain — the
// serial drawElement path has only the blank guard. Default-on drawElement
// therefore requires worker-encode; disabling HF_DE_WORKER_ENCODE without an
// explicit drawElement opt-in falls back to the screenshot baseline rather
// than shipping unverified drawElement frames.
if (merged.useDrawElement && !explicitDrawElementOptIn && !merged.enableDrawElementWorkerEncode) {
merged.useDrawElement = false;
}
// drawElement capture and page-side shader compositing are mutually
// incompatible capture strategies (drawElement reads paint records directly
// and bypasses the page-side prepare→composite→resolve protocol). When
// experimental fast capture is on, force page-side compositing off so shader
// fast capture is on, force page-side compositing off so shader
// transitions fall back to the Node-side layered blend rather than silently
// dropping. This keeps the flag self-consistent and avoids a per-session
// incompatibility warning on every fast-capture render.
if (merged.useDrawElement) {
if (merged.useDrawElement && merged.enablePageSideCompositing) {
merged.enablePageSideCompositing = false;
// Record that THIS resolution (not the caller) turned page-side
// compositing off, so a later compile-time drawElement gate can restore
// it without clobbering an explicit enablePageSideCompositing:false from
// the programmatic API or HF_PAGE_SIDE_COMPOSITING=false.
merged.pageSideCompositingAutoDisabled = true;
}
return {
+3
View File
@@ -84,6 +84,9 @@ export {
captureFrameToBuffer,
captureFrameToBufferPipelined,
captureFramesBatchPipelined,
DrawElementVerificationError,
isDrawElementVerificationError,
recaptureDrawElementFrameForVerify,
writeCapturedFrame,
discardWarmupCapture,
getCompositionDuration,
+181 -1
View File
@@ -146,6 +146,42 @@ export interface CaptureSession {
* worker-encode path (mirrors `lastFrameBuffer` on the screenshot path). Only set
* when static-frame dedup is armed on the drawElement path. */
lastEncodeResult?: Promise<Buffer>;
/** Per-render self-verification ground truth (ungated-release safety net):
* K screenshot frames captured at init BEFORE the drawElement canvas is
* injected (the only window where a page screenshot shows the live DOM, not
* the capture canvas's stale bitmap). The producer drain compares the DE
* frame at each index against these; a breach aborts the render with
* DrawElementVerificationError and the orchestrator re-renders via the
* screenshot path. */
deVerifyFrames?: Map<number, Buffer>;
}
/**
* drawElement self-verification failure — a captured DE frame diverged from its
* pre-injection screenshot ground truth (or a blank frame survived a retry).
* The orchestrator catches this and re-renders the whole job with
* forceScreenshot. Discriminant-based guard (not instanceof) so it survives
* duplicated module instances across package boundaries.
*/
export class DrawElementVerificationError extends Error {
constructor(message: string) {
super(message);
this.name = "DrawElementVerificationError";
// Discriminant property, assigned dynamically: isDrawElementVerificationError
// reads it structurally so detection survives duplicated module instances
// across package boundaries (where instanceof fails).
(this as unknown as { deVerificationFailure: boolean }).deVerificationFailure = true;
}
}
export function isDrawElementVerificationError(err: unknown): boolean {
// Walk the cause chain — the producer wraps capture errors (CaptureStageError).
let e: unknown = err;
for (let depth = 0; depth < 5 && typeof e === "object" && e !== null; depth++) {
if ((e as { deVerificationFailure?: boolean }).deVerificationFailure === true) return true;
e = (e as { cause?: unknown }).cause;
}
return false;
}
// Circular buffer for browser console messages dumped on render failure diagnostics.
@@ -581,6 +617,10 @@ async function initDrawElementOrTransparentBackground(
// passed) and the DOM is still normal (canvas not yet injected, so the
// verification screenshots are valid). drawElement-path only; see armStaticDedup.
await armStaticDedup(session, page, logInitPhase);
// Self-verification ground truth: must ALSO run pre-injection — after the
// canvas wraps the root, a page screenshot shows the canvas's last-drawn
// bitmap, not the live DOM (see the Lim 6 boundary-screenshot note).
await captureDeVerificationFrames(session, page, logInitPhase);
await injectDrawElementCanvas(page, session.options.width, session.options.height);
if (transparent) {
await initTransparentBackground(session.page);
@@ -2176,7 +2216,7 @@ async function computeTimelineAtRiskFrames(
* toggled / detached / freshly-shown at a clip-cut boundary). Per-frame, not
* whole-comp — callers fall back to screenshot for the single frame.
*/
export function isNoCachedPaintRecordError(err: unknown): boolean {
function isNoCachedPaintRecordError(err: unknown): boolean {
const msg = err instanceof Error ? err.message : String(err);
return msg.includes("No cached paint record");
}
@@ -2512,6 +2552,38 @@ export async function captureFrameToBufferPipelined(
}
}
/**
* Verification-grade single-frame recapture for the producer's blank-frame
* guard. Unlike {@link captureFrameToBufferPipelined} it takes NO shortcuts
* and has NO fallbacks, both of which can return the WRONG FRAME's pixels at
* drain time:
* - the static-dedup fast path returns session.lastEncodeResult, which by
* drain time can hold a frame several indices AHEAD of the suspect frame;
* - the per-frame "No cached paint record" screenshot fallback captures the
* injected canvas — i.e. the LAST drawn drawElement frame, not this one.
* Any failure here throws; the caller treats that as verification failure and
* falls back the whole render (correct, never wrong-frame).
*/
export async function recaptureDrawElementFrameForVerify(
session: CaptureSession,
frameIndex: number,
time: number,
): Promise<Buffer> {
const { page, options } = session;
if (!session.isInitialized) {
throw new Error("[FrameCapture] Session not initialized");
}
await prepareFrameForCapture(session, frameIndex, time);
const { encodeResult } = await produceDrawElementFrame(
page,
options.width,
options.height,
options.quality ?? 80,
true,
);
return encodeResult;
}
/**
* P6 prototype (HF_DE_BATCH): capture N consecutive frames in one CDP
* round-trip via {@link produceDrawElementFrameBatch}. The caller pre-plans the
@@ -2740,6 +2812,114 @@ export async function getCompositionDuration(session: CaptureSession): Promise<n
});
}
/**
* Ungated-release safety net, part 1: capture K screenshot ground-truth frames
* BEFORE the drawElement canvas is injected (the only window where a page
* screenshot shows the live DOM). The producer's drain compares each DE frame
* at these indices against its screenshot; a breach (or a blank frame that
* survives one retry) throws DrawElementVerificationError, and the orchestrator
* re-renders the whole job via the screenshot path.
*
* Env: HF_DE_VERIFY = sample count (default 4, clamp 0..8; 0 disables).
* Deterministic index selection: fixed fractions of the timeline, nudged off
* clip-cut boundaries (screenshot-vs-DE is legitimately ±1-frame desynced
* there — Lim 6). Skipped for tiny comps (<10 frames) and under
* HF_FORCE_DRAWELEMENT (debug escape hatch).
*
* False-positive bias is intentional: a nondeterministic comp may mismatch its
* init-time screenshot → the render falls back to the screenshot path (slower,
* never wrong). Cost when passing: ~K×(seek+screenshot) ≈ 150–300ms at init.
*/
async function captureDeVerificationFrames(
session: CaptureSession,
page: Page,
logInitPhase: (phase: string) => void,
): Promise<void> {
const kRaw = Number(process.env.HF_DE_VERIFY ?? "4");
const k = Number.isFinite(kRaw) ? Math.max(0, Math.min(8, Math.floor(kRaw))) : 4;
if (k === 0 || process.env.HF_FORCE_DRAWELEMENT === "1") return;
if (session.options.format === "png") return; // worker-encode drain (the consumer) is jpeg-only
const fps = fpsToNumber(session.options.fps);
// Prefer the producer-resolved duration (the range that will actually be
// drained). The page's raw __hf.duration can exceed it — timelines outrun
// their data-duration, and infinite-repeat GSAP reports a huge sentinel —
// and indices derived from it would never be drained, silently disarming
// verification for exactly the comps that need it.
const duration =
session.options.compositionDurationSeconds ??
(await page.evaluate(
() => (window as unknown as { __hf?: { duration?: number } }).__hf?.duration ?? 0,
));
const totalFrames = Math.floor(duration * fps);
if (totalFrames < 10) return;
if (duration > 3600) {
// No producer duration and the page reports an implausible one.
logInitPhase(`drawElement self-verify skipped: implausible duration ${duration}s`);
return;
}
// Ground truth must show what the real capture paths would show: <video>
// pixels come from the onBeforeCapture injector. When this session has no
// injector (e.g. a probe session initialized before the render wires one
// up) a video comp's truth would screenshot black boxes and every sample
// would false-positive into the screenshot fallback — skip instead.
if (!session.onBeforeCapture) {
const hasVideos = await page.evaluate(() => document.querySelector("video") !== null);
if (hasVideos) {
logInitPhase("drawElement self-verify skipped: video comp without frame injector");
return;
}
}
const boundary = await computeClipBoundaryFrames(page, fps);
// Ascending order, and seek frame 0 first: GSAP .from()/overlapping tweens
// lazily record their start values on FIRST seek — scrubbing mid-timeline
// before the render's frame-0 seek corrupts those caches for the whole
// render (the detectCssEffectRisk lesson), and because DE frames and truth
// would share the corruption, PSNR would pass on the damaged output.
// Seeking 0 → ascending reproduces the render's own seek order.
const fractions = Array.from({ length: k }, (_, i) => (i + 1) / (k + 1));
const seekTo = async (t: number): Promise<void> => {
await page.evaluate((tt: number) => {
const hf = (window as unknown as { __hf?: { seek?: (x: number) => void } }).__hf;
if (hf && typeof hf.seek === "function") hf.seek(tt);
}, t);
};
await seekTo(quantizeTimeToFrame(0, fps));
// Force one frame so lazy tween initialization paints at t=0 state.
await pageScreenshotCapture(page, session.options);
const frames = new Map<number, Buffer>();
for (const f of fractions) {
let idx = Math.min(totalFrames - 1, Math.max(1, Math.round(totalFrames * f)));
// Nudge off clip-cut boundaries (±1-frame desync is legitimate there);
// if the nudge saturates on a boundary index, skip the sample entirely.
let guard = 0;
while (boundary.has(idx) && guard++ < 6) idx = Math.min(totalFrames - 1, idx + 2);
if (boundary.has(idx)) continue;
if (frames.has(idx)) continue;
const t = quantizeTimeToFrame(idx / fps, fps);
await seekTo(t);
// Video frame injection (same hook the real capture paths run) — without
// it, <video> elements screenshot black and every video comp would
// false-positive into the screenshot fallback.
if (session.onBeforeCapture) await session.onBeforeCapture(page, t);
// Double-capture: the first screenshot forces a frame, which is what runs
// rAF-driven callbacks (count-up text counters land a tick after seek()
// returns — a single immediate screenshot captures stale text and
// false-positives the verify: 3bea8c73 28.7dB vs a truth missing its stat
// values while the DE frame was correct). NOTE: waiting on rAF via
// evaluate instead deadlocks — headless only fires rAF when a frame is
// produced, and nothing produces one until a screenshot asks.
await pageScreenshotCapture(page, session.options);
frames.set(idx, await pageScreenshotCapture(page, session.options));
}
// Leave the page at frame 0 so the render's first seek starts from the
// same state as an unverified render.
await seekTo(quantizeTimeToFrame(0, fps));
session.deVerifyFrames = frames;
logInitPhase(
`drawElement self-verify armed: ${frames.size} ground-truth frame(s) @ [${[...frames.keys()].join(", ")}] of ${totalFrames}`,
);
}
export function getCapturePerfSummary(session: CaptureSession): CapturePerfSummary {
const frames = Math.max(1, session.capturePerf.frames);
return {
+9
View File
@@ -82,6 +82,15 @@ export interface HfProtocol {
export interface CaptureOptions {
width: number;
height: number;
/**
* Producer-resolved composition duration (seconds) — the data-duration
* clamp actually rendered, which can differ from the page's raw
* `__hf.duration` (infinite-repeat GSAP timelines report a huge sentinel;
* timelines can outrun their declared duration). Consumers that derive
* frame indices meant to be drained by the producer (drawElement
* self-verification) MUST prefer this over `__hf.duration`.
*/
compositionDurationSeconds?: number;
/**
* Frame rate as an exact rational. Integer fps is `{ num: 30, den: 1 }`;
* NTSC is `{ num: 30000, den: 1001 }`. Captures are scheduled by the
+64
View File
@@ -0,0 +1,64 @@
#!/usr/bin/env bash
# drawElement release canary suite — run before/after any DE-path change and on a
# schedule once DE is default-on. macOS + hardware GPU only (DE never engages
# elsewhere). Each canary renders with the full DE stack + safety net; the suite
# asserts the render COMPLETES and the safety net's verdict matches expectation:
# clean — DE end-to-end, no fallback; cross-checked vs a screenshot render (PSNR)
# fallback — self-verify/blank-guard fires and the auto-fallback completes (these
# comps have known intermittent DE damage or are defect comps; the net
# diverting them IS the pass condition)
# any — either outcome passes (known-flaky DE damage: fallback ~2/3 of runs)
# Exit nonzero on any FAIL. Usage: COMP_ROOT=/tmp/cc-all bash de-canary-suite.sh
set -euo pipefail
cd "$(dirname "$0")"
ROOT="${COMP_ROOT:-/tmp/cc-all}"
OUT="${1:-/tmp/de-canary-suite}"
MIN_DB="${MIN_DB:-32}"
mkdir -p "$OUT"
# id expectation — see fast-capture-architecture.md § Runtime safety net for provenance
CANARIES="
d95f20b6 any # paint-race canary; intermittent background-image drop (16.4dB when it fires)
0531c45f clean # video comp + clip-boundary frames (Lim 6 regression guard)
3bea8c73 clean # dense motion graphics, deterministic small dark frames (blank-guard guard)
11c1c878 clean # media-heavy; false de-fail history (harness artifacts)
398c0655 any # marginal DE-vs-truth divergence hovering at the 32dB line (32-34dB post seek-fix, 23dB before); flips run to run
3e051c00 fallback # broken comp (runtime error banner) — must divert, not ship
42e7c33e fallback # error comp — must divert, not ship
"
DE_ENV="PRODUCER_BROWSER_GPU_MODE=hardware PRODUCER_ENABLE_BROWSER_POOL=false PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true HF_DE_WORKER_ENCODE=true HF_DE_BATCH=4"
SS_ENV="PRODUCER_BROWSER_GPU_MODE=hardware PRODUCER_ENABLE_BROWSER_POOL=false PRODUCER_EXPERIMENTAL_FAST_CAPTURE=false"
fail=0
echo "$CANARIES" | grep -v '^\s*$' | while read -r pre expect _; do
dir=$(find "$ROOT/" -maxdepth 1 -type d -name "${pre}*" | head -1)
if [ -z "$dir" ]; then echo "FAIL $pre: comp not found under $ROOT"; exit 1; fi
log="$OUT/$pre.log"
env $DE_ENV BUN_RUNTIME_TRANSPILER_CACHE_PATH=0 bun we-render.mjs "$dir" "$OUT/$pre-de.mp4" </dev/null >"$log" 2>&1 || true
if ! grep -q "RENDER_OK" "$log"; then echo "FAIL $pre: render did not complete (see $log)"; exit 1; fi
if grep -q "re-rendering via screenshot" "$log"; then verdict=fallback; else verdict=clean; fi
case "$expect" in
any) : ;;
"$verdict") : ;;
*) echo "FAIL $pre: expected $expect, got $verdict (see $log)"; exit 1 ;;
esac
# Clean DE renders get a cross-path quality check against a screenshot render.
# "any"-expectation comps are exempt: they are in the suite BECAUSE their
# DE-vs-screenshot agreement is marginal/intermittent — the check would fail
# on their intrinsic divergence, not on a regression.
if [ "$verdict" = clean ] && [ "$expect" != any ]; then
env $SS_ENV BUN_RUNTIME_TRANSPILER_CACHE_PATH=0 bun we-render.mjs "$dir" "$OUT/$pre-ss.mp4" </dev/null >"$OUT/$pre-ss.log" 2>&1 || true
if ! grep -q "RENDER_OK" "$OUT/$pre-ss.log"; then echo "FAIL $pre: screenshot arm failed"; exit 1; fi
db=$(ffmpeg -nostdin -hide_banner -i "$OUT/$pre-de.mp4" -i "$OUT/$pre-ss.mp4" -lavfi psnr -f null - 2>&1 \
| grep -oE "average:(inf|[0-9.]+)" | head -1 | cut -d: -f2 || true)
if [ -z "$db" ]; then echo "FAIL $pre: PSNR compare produced no value"; exit 1; fi
if [ "$db" != "inf" ] && awk -v d="$db" -v m="$MIN_DB" 'BEGIN{exit !(d<m)}'; then
echo "FAIL $pre: DE-vs-screenshot ${db}dB < ${MIN_DB}dB"; exit 1
fi
echo "PASS $pre ($verdict, ${db:-?}dB)"
else
echo "PASS $pre ($verdict)"
fi
done || fail=1
[ "$fail" = 0 ] && echo "CANARY_SUITE_OK" || { echo "CANARY_SUITE_FAILED"; exit 1; }
+53
View File
@@ -0,0 +1,53 @@
#!/usr/bin/env bash
# Init-only gate classification over the full 500. The gate decision is logged at init
# ("drawElement canvas injected" OR "falling back to ... — <reason>"), so launch the fast
# render, wait for that line, kill before frames render. Writes /tmp/gatecheck/<pre>.txt.
# Resumable. PAR workers. Usage: PAR=4 bash de-gatecheck.sh
set -euo pipefail
cd "$(dirname "$0")"
OUT=/tmp/gatecheck; mkdir -p "$OUT"
PAR=${PAR:-4}
SAMPLE=${SAMPLE:-/tmp/sweep-sample.txt}
GPU="PRODUCER_BROWSER_GPU_MODE=hardware PRODUCER_ENABLE_BROWSER_POOL=false HF_DE_WORKER_ENCODE=false HF_STATIC_DEDUP=false PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true"
export GPU OUT CAP
worker(){
local pre=$1
[ -f "$OUT/$pre.txt" ] && return
local dir; dir=$(find /tmp/cc-all/ -maxdepth 1 -type d -name "${pre}*" | head -1 || true)
[ -z "$dir" ] && { echo "$pre NODIR" > "$OUT/$pre.txt"; return; }
local log=/tmp/gc-$pre.log
( eval "env $GPU bun we-render.mjs \"$dir\" /tmp/gc-$pre.mp4" >"$log" 2>&1 </dev/null ) & local p=$!
local t=0
while kill -0 "$p" 2>/dev/null; do
sleep 1; t=$((t+1))
if grep -qE "drawElement canvas injected|falling back to (screenshot|beginframe) capture —|Fast capture: composition uses|render-mode compatibility hint" "$log" 2>/dev/null; then break; fi
[ "$t" -ge "${CAP:-120}" ] && break
done
# kill the render tree (don't need full render)
for c in $(pgrep -P "$p" 2>/dev/null || true); do kill -9 "$c" 2>/dev/null || true; done
kill -9 "$p" 2>/dev/null || true
pkill -f "we-render.mjs $dir" 2>/dev/null || true
wait "$p" 2>/dev/null || true
if grep -q "drawElement canvas injected" "$log" 2>/dev/null; then
echo "$pre drawelement" > "$OUT/$pre.txt"
else
local reason; reason=$(grep -oE "falling back to (screenshot|beginframe) capture — [^(]*" "$log" | head -1 | sed -E 's/.*— //' || true)
# Compile-time gates (3D / mix-blend-mode) log a different shape than the
# engine's init-time gates — "Fast capture: composition uses X — disabling".
if [ -z "$reason" ]; then
reason=$(grep -oE "Fast capture: composition uses [a-z0-9 -]*" "$log" | head -1 | sed -E 's/.*uses /compile:/' || true)
fi
# Compat-hint routing (raw rAF etc.): "fast capture: falling back to screenshot — render-mode ..."
if [ -z "$reason" ] && grep -q "render-mode compatibility hint" "$log" 2>/dev/null; then
reason="render-mode compat hint (raw rAF/alpha)"
fi
if [ -n "$reason" ]; then echo "$pre gated $reason" > "$OUT/$pre.txt"
elif grep -q "zero duration" "$log" 2>/dev/null; then echo "$pre comp-defect zero-duration" > "$OUT/$pre.txt"
elif grep -qiE "Failure summary|error:" "$log" 2>/dev/null; then echo "$pre comp-defect error" > "$OUT/$pre.txt"
else echo "$pre unknown" > "$OUT/$pre.txt"; fi
fi
rm -f /tmp/gc-$pre.mp4 /tmp/gc-$pre.log
}
export -f worker
awk '{print $1}' "$SAMPLE" | xargs -P "$PAR" -I{} bash -c 'set -uo pipefail; worker "$@"' _ {} || true
echo "GATECHECK_DONE ($(ls "$OUT"/*.txt 2>/dev/null | wc -l | tr -d ' ') comps)"
@@ -40,6 +40,8 @@ export interface RenderCaptureObservability {
usePageSideCompositing?: boolean;
hasHdrContent?: boolean;
browserGpuMode?: string;
/** drawElement per-render self-verification tripped → whole render re-ran via screenshot. */
deSelfVerifyFallback?: boolean;
protocolTimeoutMs?: number;
pageNavigationTimeoutMs?: number;
playerReadyTimeoutMs?: number;
@@ -306,6 +308,13 @@ export class RenderObservabilityRecorder {
return this.failedPhase !== undefined;
}
/** A phase failure that was subsequently recovered (e.g. the drawElement
* self-verify fallback re-rendering via screenshot) should not brand the
* whole render as failed in the summary. */
clearFailure(phase: string): void {
if (this.failedPhase === phase) this.failedPhase = undefined;
}
private record(event: RenderObservationEvent): RenderObservationEvent {
this.eventCount++;
const eventWithJob = { ...event, renderJobId: this.input.renderJobId };
@@ -41,6 +41,11 @@
* into a shared module so the stages can import without reaching back.
*/
import { execFile } from "node:child_process";
import { mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { promisify } from "node:util";
import {
type BeforeCaptureHook,
type CaptureOptions,
@@ -48,6 +53,7 @@ import {
type CaptureSession,
type EngineConfig,
type StreamingEncoder,
DrawElementVerificationError,
captureFrameToBuffer,
captureFrameToBufferPipelined,
captureFramesBatchPipelined,
@@ -57,6 +63,8 @@ import {
distributeFrames,
executeParallelCapture,
getCapturePerfSummary,
getFfmpegBinary,
recaptureDrawElementFrameForVerify,
initializeSession,
prepareCaptureSessionForReuse,
spawnStreamingEncoder,
@@ -137,6 +145,28 @@ export type CaptureStreamingStageResult =
success: false;
};
const execFileP = promisify(execFile);
/** PSNR (average, dB) between two same-dimension encoded images via ffmpeg. */
async function psnrDb(a: Buffer, b: Buffer): Promise<number> {
const dir = await mkdtemp(join(tmpdir(), "hf-de-verify-"));
try {
const pa = join(dir, "a.jpg");
const pb = join(dir, "b.jpg");
await Promise.all([writeFile(pa, a), writeFile(pb, b)]);
const { stderr } = await execFileP(
getFfmpegBinary(),
["-hide_banner", "-i", pa, "-i", pb, "-lavfi", "psnr", "-f", "null", "-"],
{ maxBuffer: 4 * 1024 * 1024 },
);
const m = /average:(inf|[\d.]+)/.exec(stderr);
if (!m) throw new Error(`psnr parse failed: ${stderr.slice(-300)}`);
return m[1] === "inf" ? Infinity : Number(m[1]);
} finally {
await rm(dir, { recursive: true, force: true }).catch(() => {});
}
}
async function runWorkerEncodePipelineLoop(
session: CaptureSession,
totalFrames: number,
@@ -145,8 +175,137 @@ async function runWorkerEncodePipelineLoop(
reorderBuffer: ReturnType<typeof createFrameReorderBuffer>,
assertNotAborted: () => void,
onProgress: CaptureStreamingStageInput["onProgress"],
log: CaptureStreamingStageInput["log"],
): Promise<void> {
let prev: { idx: number; encodeResult: Promise<Buffer> } | null = null;
const frameTime = (i: number) => (i * job.config.fps.den) / job.config.fps.num;
// ── drawElement drain-time safety checks (ungated-release safety net) ──
// Runs on every drained frame. Drains execute strictly between capture
// evaluates (see the batch NOTE below), so a re-seek + recapture here is
// safe — nothing else is mutating page time.
//
// 1. Blank guard (worker-path analogue of the serial-path guard in
// frameCapture): drawElement intermittently returns an anomalously small
// blank frame with no throw. A frame far below the rolling-median byte
// size is re-captured once (verification-grade recapture — no dedup
// shortcut, no screenshot fallback); still blank → verification error.
// The floor only arms after 12 drained frames — early frames are covered
// by the PSNR verify samples rather than the size heuristic, which has
// no stable median yet.
// 2. Self-verification: at K sampled indices, compare the DE frame against
// its pre-injection screenshot ground truth (session.deVerifyFrames).
// PSNR below HF_DE_VERIFY_MIN_DB (default 32; natural DE-vs-screenshot
// agreement measures ≥45, damage <30) → verification error.
// A DrawElementVerificationError propagates to the orchestrator, which
// re-renders the whole job via the screenshot path (never-wrong fallback).
// Clamp to a defensible band: below ~10dB even severe damage passes (the
// check stops meaning anything); above ~60dB natural DE-vs-screenshot
// encoder differences (~45dB+) would force a screenshot fallback on every
// verified render. Out-of-range or malformed values fall back to 32.
const verifyMinDbRaw = Number(process.env.HF_DE_VERIFY_MIN_DB ?? "32");
const verifyMinDb =
Number.isFinite(verifyMinDbRaw) && verifyMinDbRaw >= 10 && verifyMinDbRaw <= 60
? verifyMinDbRaw
: 32;
if (process.env.HF_DE_VERIFY_MIN_DB !== undefined && verifyMinDb !== verifyMinDbRaw) {
log.warn("[Render] HF_DE_VERIFY_MIN_DB out of range [10,60]; using 32", {
raw: process.env.HF_DE_VERIFY_MIN_DB,
});
}
const sizes: number[] = [];
// Absolute floor lowers once a small frame proves deterministic (dark /
// low-detail content), so a dark stretch doesn't re-capture every frame.
let absFloor = 20_000;
const blankFloor = (): number => {
if (sizes.length < 12) return 0;
const sorted = [...sizes].sort((x, y) => x - y);
const median = sorted[Math.floor(sorted.length / 2)] ?? 0;
return Math.max(absFloor, median * 0.12);
};
let acceptedSmall: Buffer | null = null;
const guardFrame = async (idx: number, buf: Buffer): Promise<Buffer> => {
if (process.env.HF_FORCE_DRAWELEMENT !== "1") {
const floor = blankFloor();
if (floor > 0 && buf.length < floor && acceptedSmall?.equals(buf)) {
// Identical to a small frame already proven deterministic (dark
// clip-gap runs repeat the same bytes) — skip the recapture.
} else if (floor > 0 && buf.length < floor) {
log.warn("[Render] drawElement blank-frame suspect; re-capturing", {
frame: idx,
bytes: buf.length,
floor: Math.round(floor),
});
// Verification-grade recapture: NOT captureFrameToBufferPipelined,
// whose static-dedup fast path and "No cached paint record" screenshot
// fallback can both return a DIFFERENT frame's pixels at drain time
// (dedup anchor runs ahead of the drain; the fallback screenshots the
// injected canvas = last drawn DE frame). Any recapture failure is a
// verification failure — fall back the whole render.
let retryBuf: Buffer;
try {
retryBuf = await recaptureDrawElementFrameForVerify(session, idx, frameTime(idx));
} catch (err) {
throw new DrawElementVerificationError(
`blank drawElement frame ${idx}: ${buf.length}B < floor ${Math.round(floor)}B and recapture failed (${err instanceof Error ? err.message : String(err)})`,
);
}
if (retryBuf.equals(buf)) {
// Byte-identical on re-capture ⇒ deterministic content, not a
// transient blank drop (those are intermittent by nature) — the
// frame is legitimately small (dark / low-detail). Accept it;
// deterministic damage classes are the PSNR self-verify's job.
log.info("[Render] drawElement small frame is deterministic; accepted", {
frame: idx,
bytes: buf.length,
});
absFloor = Math.min(absFloor, Math.floor(buf.length / 2));
acceptedSmall = buf;
} else if (retryBuf.length < floor) {
throw new DrawElementVerificationError(
`blank drawElement frame ${idx}: ${buf.length}B (retry ${retryBuf.length}B) < floor ${Math.round(floor)}B`,
);
} else {
buf = retryBuf;
}
}
sizes.push(buf.length);
if (sizes.length > 60) sizes.shift();
}
const truth = session.deVerifyFrames?.get(idx);
if (truth) {
let db: number;
try {
db = await psnrDb(buf, truth);
} catch (err) {
// Infrastructure failure (ffmpeg spawn/parse/tmpdir), not evidence of
// damage — skip this sample rather than failing or falling back.
log.warn("[Render] drawElement self-verify sample skipped (psnr infrastructure)", {
frame: idx,
error: err instanceof Error ? err.message : String(err),
});
return buf;
}
if (db < verifyMinDb) {
// Keep the mismatched pair for diagnosis (tmpdir; OS-reaped).
const dumpDir = await mkdtemp(join(tmpdir(), "hf-de-verify-fail-")).catch(() => null);
if (dumpDir) {
await Promise.all([
writeFile(join(dumpDir, `frame-${idx}-de.jpg`), buf),
writeFile(join(dumpDir, `frame-${idx}-truth.jpg`), truth),
]).catch(() => {});
}
throw new DrawElementVerificationError(
`drawElement self-verify failed at frame ${idx}: ${db.toFixed(1)}dB < ${verifyMinDb}dB vs pre-injection screenshot${dumpDir ? ` (pair: ${dumpDir})` : ""}`,
);
}
log.info("[Render] drawElement self-verify passed", {
frame: idx,
psnrDb: db === Infinity ? "inf" : Number(db.toFixed(1)),
});
}
return buf;
};
const drainPrev = async (): Promise<void> => {
if (!prev) return;
@@ -154,7 +313,7 @@ async function runWorkerEncodePipelineLoop(
// longest stretch of the loop); without this an abort isn't seen until the
// next produce iteration.
assertNotAborted();
const buf = await prev.encodeResult;
const buf = await guardFrame(prev.idx, await prev.encodeResult);
await reorderBuffer.waitForFrame(prev.idx);
ensureFrameWritten(await currentEncoder.writeFrame(buf), prev.idx, currentEncoder);
reorderBuffer.advanceTo(prev.idx + 1);
@@ -168,32 +327,33 @@ async function runWorkerEncodePipelineLoop(
);
};
// P6 (HF_DE_BATCH=N, N>1): capture runs of consecutive frames in ONE CDP
// round-trip each (in-page seek+paint+draw loop), amortizing protocol latency
// N-fold. Batches break at static-dedup frames (skipped via lastEncodeResult
// reuse — order-dependent) and at opt-in boundary-screenshot frames; those go
// through the per-frame path unchanged. onBeforeCapture (video frame
// injection) needs a node-side hook per frame → no batching.
const batchN = Math.floor(Number(process.env.HF_DE_BATCH ?? "0"));
// Batch capture (HF_DE_BATCH=N, N>1, default 4 — validated +1.20x lossless on
// the 19-comp gate): capture runs of consecutive frames in ONE CDP round-trip
// each (in-page seek+paint+draw loop), amortizing protocol latency N-fold.
// Batches break at static-dedup frames (skipped via lastEncodeResult reuse —
// order-dependent) and at opt-in boundary-screenshot frames; those go through
// the per-frame path unchanged. onBeforeCapture (video frame injection) needs
// a node-side hook per frame → no batching. Kill switch: HF_DE_BATCH=0.
const batchNRaw = Number(process.env.HF_DE_BATCH ?? "4");
const batchN = Number.isFinite(batchNRaw) ? Math.floor(batchNRaw) : 4;
const drainBatch = async (batch: Array<{ idx: number; encodeResult: Promise<Buffer> }>) => {
for (const item of batch) {
assertNotAborted();
const buf = await guardFrame(item.idx, await item.encodeResult);
await reorderBuffer.waitForFrame(item.idx);
ensureFrameWritten(await currentEncoder.writeFrame(buf), item.idx, currentEncoder);
reorderBuffer.advanceTo(item.idx + 1);
job.framesRendered = item.idx + 1;
updateJobStatus(
job,
"rendering",
`Streaming frame ${item.idx + 1}/${totalFrames}`,
Math.round(25 + ((item.idx + 1) / totalFrames) * 55),
onProgress,
);
}
};
if (batchN > 1 && !session.onBeforeCapture) {
const frameTime = (f: number) => (f * job.config.fps.den) / job.config.fps.num;
const drainBatch = async (batch: Array<{ idx: number; encodeResult: Promise<Buffer> }>) => {
for (const item of batch) {
assertNotAborted();
const buf = await item.encodeResult;
await reorderBuffer.waitForFrame(item.idx);
ensureFrameWritten(await currentEncoder.writeFrame(buf), item.idx, currentEncoder);
reorderBuffer.advanceTo(item.idx + 1);
job.framesRendered = item.idx + 1;
updateJobStatus(
job,
"rendering",
`Streaming frame ${item.idx + 1}/${totalFrames}`,
Math.round(25 + ((item.idx + 1) / totalFrames) * 55),
onProgress,
);
}
};
const boundarySS = process.env.HF_FAST_CAPTURE_BOUNDARY_SS === "true";
const batchable = (f: number) =>
!session.staticFrames?.has(f) && !(boundarySS && session.clipBoundaryFrames?.has(f));
@@ -207,6 +367,12 @@ async function runWorkerEncodePipelineLoop(
idxs.push(i);
i++;
}
// NOTE: draining the previous batch CONCURRENTLY with this capture
// (kick the evaluate un-awaited, drain, then await) was prototyped and
// REJECTED 2026-07-03: ~1.0–1.03× on medium/long comps but it perturbed
// pixels on a deterministic comp (87dB vs ∞ noise floor — main-thread
// contention shifting a paint-wait to the timeout path). Keep the
// drain strictly between batch evaluates.
const results = await captureFramesBatchPipelined(session, idxs, idxs.map(frameTime));
await drainBatch(prevBatch);
prevBatch = results.map((r) => ({ idx: r.frameIndex, encodeResult: r.encodeResult }));
@@ -228,7 +394,7 @@ async function runWorkerEncodePipelineLoop(
// the loop needs no special guard here.
for (let i = 0; i < totalFrames; i++) {
assertNotAborted();
const time = (i * job.config.fps.den) / job.config.fps.num;
const time = frameTime(i);
const { encodeResult } = await captureFrameToBufferPipelined(session, i, time);
await drainPrev();
prev = { idx: i, encodeResult };
@@ -388,6 +554,7 @@ export async function runCaptureStreamingStage(
reorderBuffer,
assertNotAborted,
onProgress,
log,
);
} else {
for (let i = 0; i < totalFrames; i++) {
@@ -191,6 +191,31 @@ export async function runCompileStage(input: CompileStageInput): Promise<Compile
"for this render. Capture uses the platform's baseline route.",
);
}
// Shader-transition comps: page-side compositing is the faster, purpose-built
// path for them, and resolveConfig force-disables it whenever drawElement is
// on. With drawElement default-on that trade is backwards — prefer page-side
// compositing and route these comps to the baseline capture. An explicit
// drawElement opt-in keeps the old preference.
if (
cfg.useDrawElement &&
process.env.PRODUCER_EXPERIMENTAL_FAST_CAPTURE !== "true" &&
compiled.hasShaderTransitions
) {
cfg.useDrawElement = false;
log.info(
"[Render] Fast capture: composition uses shader transitions — disabling drawElementImage " +
"so page-side compositing stays available.",
);
}
// Whenever a compile-time gate cleared useDrawElement, restore page-side
// compositing — but ONLY when resolveConfig auto-disabled it because
// drawElement was going to run. An explicit enablePageSideCompositing:false
// from the programmatic API or HF_PAGE_SIDE_COMPOSITING=false never had the
// auto-disabled flag set and stays off.
if (!cfg.useDrawElement && cfg.pageSideCompositingAutoDisabled) {
cfg.enablePageSideCompositing = true;
cfg.pageSideCompositingAutoDisabled = false;
}
const callerForced = cfg.forceScreenshot || (needsAlpha && !cfg.useDrawElement);
const { forceScreenshot: hintForced } = applyRenderModeHints(callerForced, compiled, log);
let forceScreenshot = hintForced;
@@ -72,6 +72,7 @@ import {
scaleProtocolTimeoutForComposition,
isMemoryExhaustionError,
isTransientBrowserError,
isDrawElementVerificationError,
} from "@hyperframes/engine";
import { join, dirname, resolve } from "path";
import { randomUUID } from "crypto";
@@ -1443,6 +1444,9 @@ export async function executeRenderJob(
...captureOptions,
videoMetadataHints,
skipReadinessVideoIds: videoReadinessSkipIds,
// Probe-resolved duration: drawElement self-verification derives its
// sample frame indices from this so they land inside the drained range.
compositionDurationSeconds: job.duration,
});
// The URL-served frame path (PR #596) hands each injected `<img>` a
// fileServer URL instead of a base64 data URI, on the theory that
@@ -1569,6 +1573,29 @@ export async function executeRenderJob(
durationSeconds: job.duration,
maxDurationSeconds: cfg.streamingEncodeMaxDurationSeconds,
});
// Default-on drawElement is only safe where the runtime self-verification
// net actually runs: the single-worker streaming worker-encode drain. The
// disk path (png-sequence / over the streaming duration cap) and parallel
// capture ship frames no drain verifies — route those renders to the
// screenshot baseline unless drawElement was explicitly opted into.
if (
cfg.useDrawElement &&
process.env.PRODUCER_EXPERIMENTAL_FAST_CAPTURE !== "true" &&
(!useStreamingEncode || workerCount > 1)
) {
cfg.useDrawElement = false;
log.info(
"[Render] Fast capture: default-on drawElement disabled for this render — " +
(workerCount > 1 ? "parallel capture" : "the disk capture path") +
" has no runtime self-verification. Set PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true to override.",
);
// The probe session already initialized in drawElement mode (canvas
// injected); it must not be reused by the unverified path.
if (probeSession && probeSession.captureMode === "drawelement") {
await closeCaptureSession(probeSession);
probeSession = null;
}
}
// `captureAttempts` is declared at function scope above (shared with the
// catch block). Static-dedup perf, appended per sequential session / per
@@ -1759,46 +1786,75 @@ export async function executeRenderJob(
let streamingHandled = false;
if (useStreamingEncode) {
const captureFrameStart = Date.now();
const streamingRes = await observeRenderStage(
observability,
"capture_streaming",
{ workerCount, forceScreenshot: captureForceScreenshot },
() =>
runCaptureStreamingStage({
fileServer: activeFileServer,
workDir,
framesDir,
videoOnlyPath,
job,
totalFrames,
cfg,
forceScreenshot: captureForceScreenshot,
log,
workerCount,
probeSession,
outputFormat,
streamingEncoderOptions: {
fps: job.config.fps,
width,
height,
codec: preset.codec,
preset: preset.preset,
quality: effectiveQuality,
bitrate: effectiveBitrate,
pixelFormat: preset.pixelFormat,
vp9CpuUsed: cfg.vp9CpuUsed,
useGpu: job.config.useGpu,
imageFormat: captureOptions.format || "jpeg",
hdr: preset.hdr,
},
buildCaptureOptions,
createRenderVideoFrameInjector,
abortSignal,
assertNotAborted,
onProgress,
dedupPerfs,
}),
);
const invokeStreaming = () =>
observeRenderStage(
observability,
"capture_streaming",
{ workerCount, forceScreenshot: captureForceScreenshot },
() =>
runCaptureStreamingStage({
fileServer: activeFileServer,
workDir,
framesDir,
videoOnlyPath,
job,
totalFrames,
cfg,
forceScreenshot: captureForceScreenshot,
log,
workerCount,
probeSession,
outputFormat,
streamingEncoderOptions: {
fps: job.config.fps,
width,
height,
codec: preset.codec,
preset: preset.preset,
quality: effectiveQuality,
bitrate: effectiveBitrate,
pixelFormat: preset.pixelFormat,
vp9CpuUsed: cfg.vp9CpuUsed,
useGpu: job.config.useGpu,
imageFormat: captureOptions.format || "jpeg",
hdr: preset.hdr,
},
buildCaptureOptions,
createRenderVideoFrameInjector,
abortSignal,
assertNotAborted,
onProgress,
dedupPerfs,
}),
);
let streamingRes;
try {
streamingRes = await invokeStreaming();
} catch (err) {
// drawElement self-verification tripped (blank frame or PSNR breach
// vs the pre-injection ground truth). The whole render restarts on
// the screenshot path — slower, never wrong. The failed attempt's
// session was closed by the stage's finally; probeSession (if any)
// was consumed by it, so a fresh session spawns on retry.
if (!isDrawElementVerificationError(err)) throw err;
log.warn("[Render] drawElement self-verification failed; re-rendering via screenshot", {
error: err instanceof Error ? err.message : String(err),
});
observability.checkpoint(
"capture_streaming",
"drawElement self-verify failed; retrying with forceScreenshot",
);
captureForceScreenshot = true;
updateCaptureObservability({
forceScreenshot: true,
deSelfVerifyFallback: true,
});
probeSession = null;
streamingRes = await invokeStreaming();
// The first attempt's error marked the phase failed; the retry
// recovered it — don't brand the render as failed in telemetry.
observability.clearFailure("capture_streaming");
}
const captureFrameMs = Date.now() - captureFrameStart;
if (streamingRes.success) {
streamingHandled = true;
+20
View File
@@ -0,0 +1,20 @@
// Single render for the worker-encode eval. Mode from env
// (PRODUCER_EXPERIMENTAL_FAST_CAPTURE + HF_DE_WORKER_ENCODE set by caller).
// Engine logs (capture_streaming durationMs, "worker encode initialized") go to
// stdout; the suite shell greps them.
import { createRenderJob, executeRenderJob } from "./src/index.ts";
const [dir, out] = process.argv.slice(2);
process.env.PRODUCER_ENABLE_BROWSER_POOL = "false";
process.env.PRODUCER_BROWSER_GPU_MODE = "hardware";
const job = createRenderJob({
fps: 30,
quality: "high",
format: "mp4",
workers: Math.max(1, Number(process.env.WORKERS || "1")),
useGpu: false,
hdrMode: "force-sdr",
});
const __t = Date.now();
await executeRenderJob(job, dir, out);
console.log("TOTAL_MS " + (Date.now() - __t));
console.log("RENDER_OK");