mirror of
https://github.com/hanzoai/frames.git
synced 2026-09-24 00:08:03 +00:00
feat(engine,producer): drawElement fast-capture default-on with runtime self-verification safety net (#1998)
* feat(engine,producer): drawElement fast-capture default-on with runtime self-verification safety net Flip useDrawElement + worker-encode defaults on (HF_DE_BATCH default 4), clamped in resolveConfig to hosts where drawElement can engage (macOS + hardware-GPU browser) so page-side shader compositing is untouched everywhere else; explicit env opt-in keeps attempt-and-gate semantics. Safety net makes default-on safe: the compile/init gates catch predictable incompatibility; this catches the intermittent residue no static analysis can see (stale paints, dropped background images, transient blank frames). - engine: captureDeVerificationFrames — K=4 (HF_DE_VERIFY) ground-truth screenshots at init, after gates + armStaticDedup, BEFORE canvas injection (post-injection screenshots show the canvas bitmap, not the DOM). Runs the video-injection hook per sample; double-captures so rAF-driven text counters settle (a single immediate screenshot captures stale text and false-positives). Skips png, <10 frames, implausible __hf.duration (infinite-repeat GSAP sentinel). - producer: guardFrame on both worker-encode drains — rolling-median blank guard with retry-once at drain (byte-identical retry ⇒ deterministic dark frame, accepted; retry save/restores the static-dedup anchor) + ffmpeg PSNR self-verify vs ground truth (HF_DE_VERIFY_MIN_DB, default 32dB; natural agreement ≥45dB, damage ≤25dB). Breach dumps the frame pair to tmpdir and throws DrawElementVerificationError. - orchestrator: one-shot retry — on verification error the whole render re-runs with forceScreenshot (slower, never wrong); telemetry flag deSelfVerifyFallback. - tooling: de-canary-suite.sh (7-comp release gate with expected verdicts), de-gatecheck.sh (init-only corpus routing classifier), we-render.mjs. Validated: canary suite 7/7; 611-comp routing sample 54% drawelement / 37.5% gated / 8.3% comp-defect; 12/12 risk-band renders clean on bare defaults (48/48 verify samples); engine suite 888 passed; caught two real intermittent damage classes in the wild (background-image drop, root-props offset) that previously shipped silently. Kill switches: PRODUCER_EXPERIMENTAL_FAST_CAPTURE=false, HF_DE_WORKER_ENCODE=false, HF_DE_BATCH=0, HF_DE_VERIFY=0. Co-Authored-By: Claude Fable 5 <[email protected]> * fix(engine,producer): harden the drawElement self-verification net (max code-review findings) 15 confirmed findings from the adversarial review of the default-on flip; the load-bearing five: - Ground-truth capture no longer scrubs GSAP state: seek(0) + forced frame FIRST (lazy .from()/overlap tweens record start values on first seek — mid-timeline scrubs corrupted them for the whole render, and since DE frames and truth shared the corruption, PSNR passed on damaged output), then ascending even-spread fractions, page left at frame 0. - Default-on drawElement is confined to the verified path: resolveConfig requires worker-encode (the drain that runs the net), the orchestrator disengages the default when the render takes the disk path or parallel capture (no drain verification there), and closes a drawElement-initialized probe session rather than letting the unverified path reuse it. Explicit PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true keeps old attempt-and-gate behavior. - Blank-frame retry can no longer splice wrong-frame pixels: recapture goes through recaptureDrawElementFrameForVerify — no static-dedup shortcut (lastEncodeResult runs ahead of the drain) and no "No cached paint record" screenshot fallback (post-injection that captures the canvas = the LAST drawn frame); any recapture failure falls back the whole render. - Verify indices derive from the producer-resolved duration (CaptureOptions.compositionDurationSeconds) instead of raw __hf.duration, so samples always land inside the drained range. - The platform clamp accepts "auto" GPU mode — the stock CLI resolves auto, and the literal-"hardware" clamp made default-on a no-op for the primary audience (masked in validation by explicitly-set env). Also: NaN-safe env parses (HF_DE_VERIFY / HF_DE_VERIFY_MIN_DB / HF_DE_BATCH); video comps skip verification when the session has no frame injector (probe sessions — black-video truth false-positived); psnr infrastructure failures skip the sample instead of failing the render; boundary-saturated sample indices are skipped; shader-transition comps prefer page-side compositing over default drawElement and compile-gated comps get page-side compositing restored; observability.clearFailure un-brands the recovered first streaming attempt; canary suite exempts known-marginal "any" comps from the cross-path PSNR gate; dead we-render options removed; clamp tests pin their env. Validated: canary suite 7/7; auto-GPU bare render engages the full stack; disk-path and worker-encode-off renders disengage default drawElement; malformed HF_DE_VERIFY_MIN_DB still verifies at the default threshold; engine suite 890 passed. Co-Authored-By: Claude Fable 5 <[email protected]> * fix(engine,producer): review fixes — PSC intent, verify-threshold clamp, fail-closed canaries Addresses miguel-heygen's review on #1998: - Page-side compositing restore preserves explicit caller intent (blocker): resolveConfig now records pageSideCompositingAutoDisabled only when IT turned page-side compositing off because drawElement was on; the compile-time drawElement gates restore page-side compositing only when that flag is set. An explicit enablePageSideCompositing:false from the programmatic API or HF_PAGE_SIDE_COMPOSITING=false stays off. Pinned by two config tests. - HF_DE_VERIFY_MIN_DB clamped to [10, 60] with a warning on out-of-range values: below ~10dB the check passes severe damage; above ~60dB natural encoder differences force a screenshot fallback on every verified render. - de-canary-suite.sh + de-gatecheck.sh run under set -euo pipefail with explicit `|| true` on expected-nonzero commands (render exits handled by the suite's own checks, grep no-match, kill/pkill/wait races) and a hard FAIL when the PSNR compare produces no value — release canaries fail closed. Full suite re-run green (7/7) under the new flags. Co-Authored-By: Claude Fable 5 <[email protected]> --------- Co-authored-by: Claude Fable 5 <[email protected]>
This commit is contained in:
committed by
GitHub
co-authored by
Claude Fable 5
parent
ad7dce3fc2
commit
febdbfc7c4
@@ -14,6 +14,7 @@
|
||||
"packages/producer/src/runtime-conformance.ts",
|
||||
"packages/producer/src/benchmark.ts",
|
||||
"packages/producer/scripts/generate-font-data.ts",
|
||||
"packages/producer/we-render.mjs",
|
||||
"packages/producer/scripts/validate-fast-video.ts",
|
||||
"packages/cli/scripts/generate-font-data.ts",
|
||||
"packages/engine/scripts/test-fitTextFontSize-browser.ts",
|
||||
|
||||
@@ -361,11 +361,12 @@ export default defineCommand({
|
||||
"experimental-fast-capture": {
|
||||
type: "boolean",
|
||||
description:
|
||||
"EXPERIMENTAL. Capture frames via Chrome's drawElementImage API " +
|
||||
"instead of Page.captureScreenshot — reads DOM paint records directly, " +
|
||||
"~46% faster on GPU. Transparent (PNG) renders on SwiftShader (Docker) " +
|
||||
"auto-fall back to screenshot capture. Incompatible with page-side " +
|
||||
"shader compositing. Default: false. Env: PRODUCER_EXPERIMENTAL_FAST_CAPTURE.",
|
||||
"Capture frames via Chrome's drawElementImage API instead of " +
|
||||
"Page.captureScreenshot — reads DOM paint records directly, ~2x faster. " +
|
||||
"Default: on where it can engage (macOS + hardware-GPU browser); " +
|
||||
"incompatible compositions and self-verification failures fall back to " +
|
||||
"screenshot capture automatically. Pass =false to disable. " +
|
||||
"Env: PRODUCER_EXPERIMENTAL_FAST_CAPTURE.",
|
||||
// No `default` — an omitted flag must stay `undefined` so the `!= null`
|
||||
// guard below leaves PRODUCER_EXPERIMENTAL_FAST_CAPTURE untouched and the
|
||||
// env fallback survives (matches the --low-memory-mode idiom).
|
||||
|
||||
@@ -222,17 +222,53 @@ describe("resolveConfig", () => {
|
||||
});
|
||||
|
||||
describe("useDrawElement (PRODUCER_EXPERIMENTAL_FAST_CAPTURE)", () => {
|
||||
it("defaults to false", () => {
|
||||
it("default is clamped off on software-GPU hosts (page-side compositing preserved)", () => {
|
||||
setEnv("PRODUCER_BROWSER_GPU_MODE", "software");
|
||||
unsetEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE");
|
||||
unsetEnv("HF_DE_WORKER_ENCODE");
|
||||
const config = resolveConfig();
|
||||
expect(config.useDrawElement).toBe(false);
|
||||
expect(config.enablePageSideCompositing).toBe(true);
|
||||
});
|
||||
|
||||
it("default engages on macOS with a hardware-GPU browser", () => {
|
||||
setEnv("PRODUCER_BROWSER_GPU_MODE", "hardware");
|
||||
unsetEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE");
|
||||
unsetEnv("HF_DE_WORKER_ENCODE");
|
||||
const config = resolveConfig();
|
||||
expect(config.useDrawElement).toBe(process.platform === "darwin");
|
||||
});
|
||||
|
||||
it("default engages on macOS with auto GPU mode (the stock CLI path)", () => {
|
||||
setEnv("PRODUCER_BROWSER_GPU_MODE", "auto");
|
||||
unsetEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE");
|
||||
unsetEnv("HF_DE_WORKER_ENCODE");
|
||||
const config = resolveConfig();
|
||||
expect(config.useDrawElement).toBe(process.platform === "darwin");
|
||||
});
|
||||
|
||||
it("default requires worker-encode (the verified drain)", () => {
|
||||
setEnv("PRODUCER_BROWSER_GPU_MODE", "hardware");
|
||||
setEnv("HF_DE_WORKER_ENCODE", "false");
|
||||
unsetEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE");
|
||||
const config = resolveConfig();
|
||||
expect(config.useDrawElement).toBe(false);
|
||||
});
|
||||
|
||||
it("enabled when PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true", () => {
|
||||
it("explicit env opt-in skips the platform clamp", () => {
|
||||
setEnv("PRODUCER_BROWSER_GPU_MODE", "software");
|
||||
setEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE", "true");
|
||||
const config = resolveConfig();
|
||||
expect(config.useDrawElement).toBe(true);
|
||||
});
|
||||
|
||||
it("env kill switch wins over the default", () => {
|
||||
setEnv("PRODUCER_BROWSER_GPU_MODE", "hardware");
|
||||
setEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE", "false");
|
||||
const config = resolveConfig();
|
||||
expect(config.useDrawElement).toBe(false);
|
||||
});
|
||||
|
||||
it("explicit override wins over the env var", () => {
|
||||
setEnv("PRODUCER_EXPERIMENTAL_FAST_CAPTURE", "true");
|
||||
const config = resolveConfig({ useDrawElement: false });
|
||||
@@ -243,6 +279,15 @@ describe("resolveConfig", () => {
|
||||
const config = resolveConfig({ useDrawElement: true, enablePageSideCompositing: true });
|
||||
expect(config.useDrawElement).toBe(true);
|
||||
expect(config.enablePageSideCompositing).toBe(false);
|
||||
// The auto-disable is recorded so compile-time gates can restore it.
|
||||
expect(config.pageSideCompositingAutoDisabled).toBe(true);
|
||||
});
|
||||
|
||||
it("does NOT mark auto-disabled when the caller explicitly opted out of page-side compositing", () => {
|
||||
const config = resolveConfig({ useDrawElement: true, enablePageSideCompositing: false });
|
||||
expect(config.enablePageSideCompositing).toBe(false);
|
||||
// Explicit caller intent — a compile-time drawElement gate must not restore it.
|
||||
expect(config.pageSideCompositingAutoDisabled).not.toBe(true);
|
||||
});
|
||||
|
||||
it("leaves page-side compositing on when fast capture is off", () => {
|
||||
|
||||
@@ -64,20 +64,30 @@ export interface EngineConfig {
|
||||
*/
|
||||
staticFrameDedup: boolean;
|
||||
/**
|
||||
* EXPERIMENTAL. Use drawElementImage for frame capture (requires the
|
||||
* CanvasDrawElement Chrome flag, added globally in buildChromeArgs).
|
||||
* Surfaced via the CLI `--experimental-fast-capture` flag.
|
||||
* Env fallback: `PRODUCER_EXPERIMENTAL_FAST_CAPTURE`.
|
||||
* Use drawElementImage for frame capture (requires the CanvasDrawElement
|
||||
* Chrome flag, added globally in buildChromeArgs). Default ON, clamped in
|
||||
* `resolveConfig` to hosts where it can actually engage (macOS + hardware-GPU
|
||||
* browser); compile/init gates and the runtime self-verification net route
|
||||
* incompatible or damaged renders back to screenshot capture.
|
||||
* Kill switch: `PRODUCER_EXPERIMENTAL_FAST_CAPTURE=false` (or the CLI
|
||||
* `--experimental-fast-capture=false`).
|
||||
*/
|
||||
useDrawElement: boolean;
|
||||
/**
|
||||
* EXPERIMENTAL. Pipeline JPEG encode into an in-page OffscreenCanvas Worker
|
||||
* for the drawElement fast-capture path (macOS hardware GPU only). The worker
|
||||
* encodes frame N while the main thread seeks+paints frame N+1, targeting
|
||||
* ~1.65–1.96× wall-time speedup. No-op unless `useDrawElement` is also true.
|
||||
* Default: off. Env: `HF_DE_WORKER_ENCODE=true`.
|
||||
* Pipeline JPEG encode into an in-page OffscreenCanvas Worker for the
|
||||
* drawElement fast-capture path (macOS hardware GPU only). The worker
|
||||
* encodes frame N while the main thread seeks+paints frame N+1
|
||||
* (~1.65–1.96× wall-time speedup). No-op unless `useDrawElement` is also
|
||||
* true. Kill switch: `HF_DE_WORKER_ENCODE=false`.
|
||||
*/
|
||||
enableDrawElementWorkerEncode: boolean;
|
||||
/**
|
||||
* INTERNAL. Set by resolveConfig when it disabled enablePageSideCompositing
|
||||
* solely because drawElement was on. Lets the producer's compile-time gates
|
||||
* restore page-side compositing without overriding an explicit caller/env
|
||||
* opt-out. Not intended to be set by callers.
|
||||
*/
|
||||
pageSideCompositingAutoDisabled?: boolean;
|
||||
/**
|
||||
* Low-memory render profile. When `true`, the orchestrator collapses the
|
||||
* pipeline to its cheapest shape on memory-constrained hosts: it skips the
|
||||
@@ -250,8 +260,8 @@ export const DEFAULT_CONFIG: EngineConfig = {
|
||||
protocolTimeout: 300_000,
|
||||
forceScreenshot: false,
|
||||
staticFrameDedup: true,
|
||||
useDrawElement: false,
|
||||
enableDrawElementWorkerEncode: false,
|
||||
useDrawElement: true,
|
||||
enableDrawElementWorkerEncode: true,
|
||||
// Auto-detected per host in `resolveConfig`; defaults off for the raw
|
||||
// DEFAULT_CONFIG (used directly by tests and worker-sizing fallbacks).
|
||||
lowMemoryMode: false,
|
||||
@@ -516,15 +526,48 @@ export function resolveConfig(overrides?: Partial<EngineConfig>): EngineConfig {
|
||||
...overrides,
|
||||
};
|
||||
|
||||
// Default-on drawElement is clamped to hosts where it can actually engage
|
||||
// (macOS with a non-software-GPU browser; SwiftShader drops transparent
|
||||
// sub-layers — crbug 521434899). "auto" passes the clamp: the stock CLI
|
||||
// resolves GPU mode to auto, which probes to hardware on real Macs — and if
|
||||
// it resolves to software after all, the SwiftShader init-time gate still
|
||||
// routes the session to the screenshot baseline. Without the clamp, the
|
||||
// default would needlessly disable page-side shader compositing (below) on
|
||||
// Linux/Docker hosts where DE never runs. An EXPLICIT opt-in (env or caller override)
|
||||
// skips the clamp and keeps the old semantics — attempt DE, let the
|
||||
// init-time gates route away — which debugging relies on.
|
||||
const explicitDrawElementOptIn =
|
||||
env("PRODUCER_EXPERIMENTAL_FAST_CAPTURE") === "true" || overrides?.useDrawElement === true;
|
||||
if (
|
||||
merged.useDrawElement &&
|
||||
!explicitDrawElementOptIn &&
|
||||
!(process.platform === "darwin" && merged.browserGpuMode !== "software")
|
||||
) {
|
||||
merged.useDrawElement = false;
|
||||
}
|
||||
// The runtime self-verification net lives in the worker-encode drain — the
|
||||
// serial drawElement path has only the blank guard. Default-on drawElement
|
||||
// therefore requires worker-encode; disabling HF_DE_WORKER_ENCODE without an
|
||||
// explicit drawElement opt-in falls back to the screenshot baseline rather
|
||||
// than shipping unverified drawElement frames.
|
||||
if (merged.useDrawElement && !explicitDrawElementOptIn && !merged.enableDrawElementWorkerEncode) {
|
||||
merged.useDrawElement = false;
|
||||
}
|
||||
|
||||
// drawElement capture and page-side shader compositing are mutually
|
||||
// incompatible capture strategies (drawElement reads paint records directly
|
||||
// and bypasses the page-side prepare→composite→resolve protocol). When
|
||||
// experimental fast capture is on, force page-side compositing off so shader
|
||||
// fast capture is on, force page-side compositing off so shader
|
||||
// transitions fall back to the Node-side layered blend rather than silently
|
||||
// dropping. This keeps the flag self-consistent and avoids a per-session
|
||||
// incompatibility warning on every fast-capture render.
|
||||
if (merged.useDrawElement) {
|
||||
if (merged.useDrawElement && merged.enablePageSideCompositing) {
|
||||
merged.enablePageSideCompositing = false;
|
||||
// Record that THIS resolution (not the caller) turned page-side
|
||||
// compositing off, so a later compile-time drawElement gate can restore
|
||||
// it without clobbering an explicit enablePageSideCompositing:false from
|
||||
// the programmatic API or HF_PAGE_SIDE_COMPOSITING=false.
|
||||
merged.pageSideCompositingAutoDisabled = true;
|
||||
}
|
||||
|
||||
return {
|
||||
|
||||
@@ -84,6 +84,9 @@ export {
|
||||
captureFrameToBuffer,
|
||||
captureFrameToBufferPipelined,
|
||||
captureFramesBatchPipelined,
|
||||
DrawElementVerificationError,
|
||||
isDrawElementVerificationError,
|
||||
recaptureDrawElementFrameForVerify,
|
||||
writeCapturedFrame,
|
||||
discardWarmupCapture,
|
||||
getCompositionDuration,
|
||||
|
||||
@@ -146,6 +146,42 @@ export interface CaptureSession {
|
||||
* worker-encode path (mirrors `lastFrameBuffer` on the screenshot path). Only set
|
||||
* when static-frame dedup is armed on the drawElement path. */
|
||||
lastEncodeResult?: Promise<Buffer>;
|
||||
/** Per-render self-verification ground truth (ungated-release safety net):
|
||||
* K screenshot frames captured at init BEFORE the drawElement canvas is
|
||||
* injected (the only window where a page screenshot shows the live DOM, not
|
||||
* the capture canvas's stale bitmap). The producer drain compares the DE
|
||||
* frame at each index against these; a breach aborts the render with
|
||||
* DrawElementVerificationError and the orchestrator re-renders via the
|
||||
* screenshot path. */
|
||||
deVerifyFrames?: Map<number, Buffer>;
|
||||
}
|
||||
|
||||
/**
|
||||
* drawElement self-verification failure — a captured DE frame diverged from its
|
||||
* pre-injection screenshot ground truth (or a blank frame survived a retry).
|
||||
* The orchestrator catches this and re-renders the whole job with
|
||||
* forceScreenshot. Discriminant-based guard (not instanceof) so it survives
|
||||
* duplicated module instances across package boundaries.
|
||||
*/
|
||||
export class DrawElementVerificationError extends Error {
|
||||
constructor(message: string) {
|
||||
super(message);
|
||||
this.name = "DrawElementVerificationError";
|
||||
// Discriminant property, assigned dynamically: isDrawElementVerificationError
|
||||
// reads it structurally so detection survives duplicated module instances
|
||||
// across package boundaries (where instanceof fails).
|
||||
(this as unknown as { deVerificationFailure: boolean }).deVerificationFailure = true;
|
||||
}
|
||||
}
|
||||
|
||||
export function isDrawElementVerificationError(err: unknown): boolean {
|
||||
// Walk the cause chain — the producer wraps capture errors (CaptureStageError).
|
||||
let e: unknown = err;
|
||||
for (let depth = 0; depth < 5 && typeof e === "object" && e !== null; depth++) {
|
||||
if ((e as { deVerificationFailure?: boolean }).deVerificationFailure === true) return true;
|
||||
e = (e as { cause?: unknown }).cause;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
// Circular buffer for browser console messages dumped on render failure diagnostics.
|
||||
@@ -581,6 +617,10 @@ async function initDrawElementOrTransparentBackground(
|
||||
// passed) and the DOM is still normal (canvas not yet injected, so the
|
||||
// verification screenshots are valid). drawElement-path only; see armStaticDedup.
|
||||
await armStaticDedup(session, page, logInitPhase);
|
||||
// Self-verification ground truth: must ALSO run pre-injection — after the
|
||||
// canvas wraps the root, a page screenshot shows the canvas's last-drawn
|
||||
// bitmap, not the live DOM (see the Lim 6 boundary-screenshot note).
|
||||
await captureDeVerificationFrames(session, page, logInitPhase);
|
||||
await injectDrawElementCanvas(page, session.options.width, session.options.height);
|
||||
if (transparent) {
|
||||
await initTransparentBackground(session.page);
|
||||
@@ -2176,7 +2216,7 @@ async function computeTimelineAtRiskFrames(
|
||||
* toggled / detached / freshly-shown at a clip-cut boundary). Per-frame, not
|
||||
* whole-comp — callers fall back to screenshot for the single frame.
|
||||
*/
|
||||
export function isNoCachedPaintRecordError(err: unknown): boolean {
|
||||
function isNoCachedPaintRecordError(err: unknown): boolean {
|
||||
const msg = err instanceof Error ? err.message : String(err);
|
||||
return msg.includes("No cached paint record");
|
||||
}
|
||||
@@ -2512,6 +2552,38 @@ export async function captureFrameToBufferPipelined(
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Verification-grade single-frame recapture for the producer's blank-frame
|
||||
* guard. Unlike {@link captureFrameToBufferPipelined} it takes NO shortcuts
|
||||
* and has NO fallbacks, both of which can return the WRONG FRAME's pixels at
|
||||
* drain time:
|
||||
* - the static-dedup fast path returns session.lastEncodeResult, which by
|
||||
* drain time can hold a frame several indices AHEAD of the suspect frame;
|
||||
* - the per-frame "No cached paint record" screenshot fallback captures the
|
||||
* injected canvas — i.e. the LAST drawn drawElement frame, not this one.
|
||||
* Any failure here throws; the caller treats that as verification failure and
|
||||
* falls back the whole render (correct, never wrong-frame).
|
||||
*/
|
||||
export async function recaptureDrawElementFrameForVerify(
|
||||
session: CaptureSession,
|
||||
frameIndex: number,
|
||||
time: number,
|
||||
): Promise<Buffer> {
|
||||
const { page, options } = session;
|
||||
if (!session.isInitialized) {
|
||||
throw new Error("[FrameCapture] Session not initialized");
|
||||
}
|
||||
await prepareFrameForCapture(session, frameIndex, time);
|
||||
const { encodeResult } = await produceDrawElementFrame(
|
||||
page,
|
||||
options.width,
|
||||
options.height,
|
||||
options.quality ?? 80,
|
||||
true,
|
||||
);
|
||||
return encodeResult;
|
||||
}
|
||||
|
||||
/**
|
||||
* P6 prototype (HF_DE_BATCH): capture N consecutive frames in one CDP
|
||||
* round-trip via {@link produceDrawElementFrameBatch}. The caller pre-plans the
|
||||
@@ -2740,6 +2812,114 @@ export async function getCompositionDuration(session: CaptureSession): Promise<n
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Ungated-release safety net, part 1: capture K screenshot ground-truth frames
|
||||
* BEFORE the drawElement canvas is injected (the only window where a page
|
||||
* screenshot shows the live DOM). The producer's drain compares each DE frame
|
||||
* at these indices against its screenshot; a breach (or a blank frame that
|
||||
* survives one retry) throws DrawElementVerificationError, and the orchestrator
|
||||
* re-renders the whole job via the screenshot path.
|
||||
*
|
||||
* Env: HF_DE_VERIFY = sample count (default 4, clamp 0..8; 0 disables).
|
||||
* Deterministic index selection: fixed fractions of the timeline, nudged off
|
||||
* clip-cut boundaries (screenshot-vs-DE is legitimately ±1-frame desynced
|
||||
* there — Lim 6). Skipped for tiny comps (<10 frames) and under
|
||||
* HF_FORCE_DRAWELEMENT (debug escape hatch).
|
||||
*
|
||||
* False-positive bias is intentional: a nondeterministic comp may mismatch its
|
||||
* init-time screenshot → the render falls back to the screenshot path (slower,
|
||||
* never wrong). Cost when passing: ~K×(seek+screenshot) ≈ 150–300ms at init.
|
||||
*/
|
||||
async function captureDeVerificationFrames(
|
||||
session: CaptureSession,
|
||||
page: Page,
|
||||
logInitPhase: (phase: string) => void,
|
||||
): Promise<void> {
|
||||
const kRaw = Number(process.env.HF_DE_VERIFY ?? "4");
|
||||
const k = Number.isFinite(kRaw) ? Math.max(0, Math.min(8, Math.floor(kRaw))) : 4;
|
||||
if (k === 0 || process.env.HF_FORCE_DRAWELEMENT === "1") return;
|
||||
if (session.options.format === "png") return; // worker-encode drain (the consumer) is jpeg-only
|
||||
const fps = fpsToNumber(session.options.fps);
|
||||
// Prefer the producer-resolved duration (the range that will actually be
|
||||
// drained). The page's raw __hf.duration can exceed it — timelines outrun
|
||||
// their data-duration, and infinite-repeat GSAP reports a huge sentinel —
|
||||
// and indices derived from it would never be drained, silently disarming
|
||||
// verification for exactly the comps that need it.
|
||||
const duration =
|
||||
session.options.compositionDurationSeconds ??
|
||||
(await page.evaluate(
|
||||
() => (window as unknown as { __hf?: { duration?: number } }).__hf?.duration ?? 0,
|
||||
));
|
||||
const totalFrames = Math.floor(duration * fps);
|
||||
if (totalFrames < 10) return;
|
||||
if (duration > 3600) {
|
||||
// No producer duration and the page reports an implausible one.
|
||||
logInitPhase(`drawElement self-verify skipped: implausible duration ${duration}s`);
|
||||
return;
|
||||
}
|
||||
// Ground truth must show what the real capture paths would show: <video>
|
||||
// pixels come from the onBeforeCapture injector. When this session has no
|
||||
// injector (e.g. a probe session initialized before the render wires one
|
||||
// up) a video comp's truth would screenshot black boxes and every sample
|
||||
// would false-positive into the screenshot fallback — skip instead.
|
||||
if (!session.onBeforeCapture) {
|
||||
const hasVideos = await page.evaluate(() => document.querySelector("video") !== null);
|
||||
if (hasVideos) {
|
||||
logInitPhase("drawElement self-verify skipped: video comp without frame injector");
|
||||
return;
|
||||
}
|
||||
}
|
||||
const boundary = await computeClipBoundaryFrames(page, fps);
|
||||
// Ascending order, and seek frame 0 first: GSAP .from()/overlapping tweens
|
||||
// lazily record their start values on FIRST seek — scrubbing mid-timeline
|
||||
// before the render's frame-0 seek corrupts those caches for the whole
|
||||
// render (the detectCssEffectRisk lesson), and because DE frames and truth
|
||||
// would share the corruption, PSNR would pass on the damaged output.
|
||||
// Seeking 0 → ascending reproduces the render's own seek order.
|
||||
const fractions = Array.from({ length: k }, (_, i) => (i + 1) / (k + 1));
|
||||
const seekTo = async (t: number): Promise<void> => {
|
||||
await page.evaluate((tt: number) => {
|
||||
const hf = (window as unknown as { __hf?: { seek?: (x: number) => void } }).__hf;
|
||||
if (hf && typeof hf.seek === "function") hf.seek(tt);
|
||||
}, t);
|
||||
};
|
||||
await seekTo(quantizeTimeToFrame(0, fps));
|
||||
// Force one frame so lazy tween initialization paints at t=0 state.
|
||||
await pageScreenshotCapture(page, session.options);
|
||||
const frames = new Map<number, Buffer>();
|
||||
for (const f of fractions) {
|
||||
let idx = Math.min(totalFrames - 1, Math.max(1, Math.round(totalFrames * f)));
|
||||
// Nudge off clip-cut boundaries (±1-frame desync is legitimate there);
|
||||
// if the nudge saturates on a boundary index, skip the sample entirely.
|
||||
let guard = 0;
|
||||
while (boundary.has(idx) && guard++ < 6) idx = Math.min(totalFrames - 1, idx + 2);
|
||||
if (boundary.has(idx)) continue;
|
||||
if (frames.has(idx)) continue;
|
||||
const t = quantizeTimeToFrame(idx / fps, fps);
|
||||
await seekTo(t);
|
||||
// Video frame injection (same hook the real capture paths run) — without
|
||||
// it, <video> elements screenshot black and every video comp would
|
||||
// false-positive into the screenshot fallback.
|
||||
if (session.onBeforeCapture) await session.onBeforeCapture(page, t);
|
||||
// Double-capture: the first screenshot forces a frame, which is what runs
|
||||
// rAF-driven callbacks (count-up text counters land a tick after seek()
|
||||
// returns — a single immediate screenshot captures stale text and
|
||||
// false-positives the verify: 3bea8c73 28.7dB vs a truth missing its stat
|
||||
// values while the DE frame was correct). NOTE: waiting on rAF via
|
||||
// evaluate instead deadlocks — headless only fires rAF when a frame is
|
||||
// produced, and nothing produces one until a screenshot asks.
|
||||
await pageScreenshotCapture(page, session.options);
|
||||
frames.set(idx, await pageScreenshotCapture(page, session.options));
|
||||
}
|
||||
// Leave the page at frame 0 so the render's first seek starts from the
|
||||
// same state as an unverified render.
|
||||
await seekTo(quantizeTimeToFrame(0, fps));
|
||||
session.deVerifyFrames = frames;
|
||||
logInitPhase(
|
||||
`drawElement self-verify armed: ${frames.size} ground-truth frame(s) @ [${[...frames.keys()].join(", ")}] of ${totalFrames}`,
|
||||
);
|
||||
}
|
||||
|
||||
export function getCapturePerfSummary(session: CaptureSession): CapturePerfSummary {
|
||||
const frames = Math.max(1, session.capturePerf.frames);
|
||||
return {
|
||||
|
||||
@@ -82,6 +82,15 @@ export interface HfProtocol {
|
||||
export interface CaptureOptions {
|
||||
width: number;
|
||||
height: number;
|
||||
/**
|
||||
* Producer-resolved composition duration (seconds) — the data-duration
|
||||
* clamp actually rendered, which can differ from the page's raw
|
||||
* `__hf.duration` (infinite-repeat GSAP timelines report a huge sentinel;
|
||||
* timelines can outrun their declared duration). Consumers that derive
|
||||
* frame indices meant to be drained by the producer (drawElement
|
||||
* self-verification) MUST prefer this over `__hf.duration`.
|
||||
*/
|
||||
compositionDurationSeconds?: number;
|
||||
/**
|
||||
* Frame rate as an exact rational. Integer fps is `{ num: 30, den: 1 }`;
|
||||
* NTSC is `{ num: 30000, den: 1001 }`. Captures are scheduled by the
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
#!/usr/bin/env bash
|
||||
# drawElement release canary suite — run before/after any DE-path change and on a
|
||||
# schedule once DE is default-on. macOS + hardware GPU only (DE never engages
|
||||
# elsewhere). Each canary renders with the full DE stack + safety net; the suite
|
||||
# asserts the render COMPLETES and the safety net's verdict matches expectation:
|
||||
# clean — DE end-to-end, no fallback; cross-checked vs a screenshot render (PSNR)
|
||||
# fallback — self-verify/blank-guard fires and the auto-fallback completes (these
|
||||
# comps have known intermittent DE damage or are defect comps; the net
|
||||
# diverting them IS the pass condition)
|
||||
# any — either outcome passes (known-flaky DE damage: fallback ~2/3 of runs)
|
||||
# Exit nonzero on any FAIL. Usage: COMP_ROOT=/tmp/cc-all bash de-canary-suite.sh
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")"
|
||||
ROOT="${COMP_ROOT:-/tmp/cc-all}"
|
||||
OUT="${1:-/tmp/de-canary-suite}"
|
||||
MIN_DB="${MIN_DB:-32}"
|
||||
mkdir -p "$OUT"
|
||||
|
||||
# id expectation — see fast-capture-architecture.md § Runtime safety net for provenance
|
||||
CANARIES="
|
||||
d95f20b6 any # paint-race canary; intermittent background-image drop (16.4dB when it fires)
|
||||
0531c45f clean # video comp + clip-boundary frames (Lim 6 regression guard)
|
||||
3bea8c73 clean # dense motion graphics, deterministic small dark frames (blank-guard guard)
|
||||
11c1c878 clean # media-heavy; false de-fail history (harness artifacts)
|
||||
398c0655 any # marginal DE-vs-truth divergence hovering at the 32dB line (32-34dB post seek-fix, 23dB before); flips run to run
|
||||
3e051c00 fallback # broken comp (runtime error banner) — must divert, not ship
|
||||
42e7c33e fallback # error comp — must divert, not ship
|
||||
"
|
||||
|
||||
DE_ENV="PRODUCER_BROWSER_GPU_MODE=hardware PRODUCER_ENABLE_BROWSER_POOL=false PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true HF_DE_WORKER_ENCODE=true HF_DE_BATCH=4"
|
||||
SS_ENV="PRODUCER_BROWSER_GPU_MODE=hardware PRODUCER_ENABLE_BROWSER_POOL=false PRODUCER_EXPERIMENTAL_FAST_CAPTURE=false"
|
||||
|
||||
fail=0
|
||||
echo "$CANARIES" | grep -v '^\s*$' | while read -r pre expect _; do
|
||||
dir=$(find "$ROOT/" -maxdepth 1 -type d -name "${pre}*" | head -1)
|
||||
if [ -z "$dir" ]; then echo "FAIL $pre: comp not found under $ROOT"; exit 1; fi
|
||||
log="$OUT/$pre.log"
|
||||
env $DE_ENV BUN_RUNTIME_TRANSPILER_CACHE_PATH=0 bun we-render.mjs "$dir" "$OUT/$pre-de.mp4" </dev/null >"$log" 2>&1 || true
|
||||
if ! grep -q "RENDER_OK" "$log"; then echo "FAIL $pre: render did not complete (see $log)"; exit 1; fi
|
||||
if grep -q "re-rendering via screenshot" "$log"; then verdict=fallback; else verdict=clean; fi
|
||||
case "$expect" in
|
||||
any) : ;;
|
||||
"$verdict") : ;;
|
||||
*) echo "FAIL $pre: expected $expect, got $verdict (see $log)"; exit 1 ;;
|
||||
esac
|
||||
# Clean DE renders get a cross-path quality check against a screenshot render.
|
||||
# "any"-expectation comps are exempt: they are in the suite BECAUSE their
|
||||
# DE-vs-screenshot agreement is marginal/intermittent — the check would fail
|
||||
# on their intrinsic divergence, not on a regression.
|
||||
if [ "$verdict" = clean ] && [ "$expect" != any ]; then
|
||||
env $SS_ENV BUN_RUNTIME_TRANSPILER_CACHE_PATH=0 bun we-render.mjs "$dir" "$OUT/$pre-ss.mp4" </dev/null >"$OUT/$pre-ss.log" 2>&1 || true
|
||||
if ! grep -q "RENDER_OK" "$OUT/$pre-ss.log"; then echo "FAIL $pre: screenshot arm failed"; exit 1; fi
|
||||
db=$(ffmpeg -nostdin -hide_banner -i "$OUT/$pre-de.mp4" -i "$OUT/$pre-ss.mp4" -lavfi psnr -f null - 2>&1 \
|
||||
| grep -oE "average:(inf|[0-9.]+)" | head -1 | cut -d: -f2 || true)
|
||||
if [ -z "$db" ]; then echo "FAIL $pre: PSNR compare produced no value"; exit 1; fi
|
||||
if [ "$db" != "inf" ] && awk -v d="$db" -v m="$MIN_DB" 'BEGIN{exit !(d<m)}'; then
|
||||
echo "FAIL $pre: DE-vs-screenshot ${db}dB < ${MIN_DB}dB"; exit 1
|
||||
fi
|
||||
echo "PASS $pre ($verdict, ${db:-?}dB)"
|
||||
else
|
||||
echo "PASS $pre ($verdict)"
|
||||
fi
|
||||
done || fail=1
|
||||
[ "$fail" = 0 ] && echo "CANARY_SUITE_OK" || { echo "CANARY_SUITE_FAILED"; exit 1; }
|
||||
@@ -0,0 +1,53 @@
|
||||
#!/usr/bin/env bash
|
||||
# Init-only gate classification over the full 500. The gate decision is logged at init
|
||||
# ("drawElement canvas injected" OR "falling back to ... — <reason>"), so launch the fast
|
||||
# render, wait for that line, kill before frames render. Writes /tmp/gatecheck/<pre>.txt.
|
||||
# Resumable. PAR workers. Usage: PAR=4 bash de-gatecheck.sh
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")"
|
||||
OUT=/tmp/gatecheck; mkdir -p "$OUT"
|
||||
PAR=${PAR:-4}
|
||||
SAMPLE=${SAMPLE:-/tmp/sweep-sample.txt}
|
||||
GPU="PRODUCER_BROWSER_GPU_MODE=hardware PRODUCER_ENABLE_BROWSER_POOL=false HF_DE_WORKER_ENCODE=false HF_STATIC_DEDUP=false PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true"
|
||||
export GPU OUT CAP
|
||||
worker(){
|
||||
local pre=$1
|
||||
[ -f "$OUT/$pre.txt" ] && return
|
||||
local dir; dir=$(find /tmp/cc-all/ -maxdepth 1 -type d -name "${pre}*" | head -1 || true)
|
||||
[ -z "$dir" ] && { echo "$pre NODIR" > "$OUT/$pre.txt"; return; }
|
||||
local log=/tmp/gc-$pre.log
|
||||
( eval "env $GPU bun we-render.mjs \"$dir\" /tmp/gc-$pre.mp4" >"$log" 2>&1 </dev/null ) & local p=$!
|
||||
local t=0
|
||||
while kill -0 "$p" 2>/dev/null; do
|
||||
sleep 1; t=$((t+1))
|
||||
if grep -qE "drawElement canvas injected|falling back to (screenshot|beginframe) capture —|Fast capture: composition uses|render-mode compatibility hint" "$log" 2>/dev/null; then break; fi
|
||||
[ "$t" -ge "${CAP:-120}" ] && break
|
||||
done
|
||||
# kill the render tree (don't need full render)
|
||||
for c in $(pgrep -P "$p" 2>/dev/null || true); do kill -9 "$c" 2>/dev/null || true; done
|
||||
kill -9 "$p" 2>/dev/null || true
|
||||
pkill -f "we-render.mjs $dir" 2>/dev/null || true
|
||||
wait "$p" 2>/dev/null || true
|
||||
if grep -q "drawElement canvas injected" "$log" 2>/dev/null; then
|
||||
echo "$pre drawelement" > "$OUT/$pre.txt"
|
||||
else
|
||||
local reason; reason=$(grep -oE "falling back to (screenshot|beginframe) capture — [^(]*" "$log" | head -1 | sed -E 's/.*— //' || true)
|
||||
# Compile-time gates (3D / mix-blend-mode) log a different shape than the
|
||||
# engine's init-time gates — "Fast capture: composition uses X — disabling".
|
||||
if [ -z "$reason" ]; then
|
||||
reason=$(grep -oE "Fast capture: composition uses [a-z0-9 -]*" "$log" | head -1 | sed -E 's/.*uses /compile:/' || true)
|
||||
fi
|
||||
# Compat-hint routing (raw rAF etc.): "fast capture: falling back to screenshot — render-mode ..."
|
||||
if [ -z "$reason" ] && grep -q "render-mode compatibility hint" "$log" 2>/dev/null; then
|
||||
reason="render-mode compat hint (raw rAF/alpha)"
|
||||
fi
|
||||
if [ -n "$reason" ]; then echo "$pre gated $reason" > "$OUT/$pre.txt"
|
||||
elif grep -q "zero duration" "$log" 2>/dev/null; then echo "$pre comp-defect zero-duration" > "$OUT/$pre.txt"
|
||||
elif grep -qiE "Failure summary|error:" "$log" 2>/dev/null; then echo "$pre comp-defect error" > "$OUT/$pre.txt"
|
||||
else echo "$pre unknown" > "$OUT/$pre.txt"; fi
|
||||
fi
|
||||
rm -f /tmp/gc-$pre.mp4 /tmp/gc-$pre.log
|
||||
}
|
||||
export -f worker
|
||||
awk '{print $1}' "$SAMPLE" | xargs -P "$PAR" -I{} bash -c 'set -uo pipefail; worker "$@"' _ {} || true
|
||||
echo "GATECHECK_DONE ($(ls "$OUT"/*.txt 2>/dev/null | wc -l | tr -d ' ') comps)"
|
||||
@@ -40,6 +40,8 @@ export interface RenderCaptureObservability {
|
||||
usePageSideCompositing?: boolean;
|
||||
hasHdrContent?: boolean;
|
||||
browserGpuMode?: string;
|
||||
/** drawElement per-render self-verification tripped → whole render re-ran via screenshot. */
|
||||
deSelfVerifyFallback?: boolean;
|
||||
protocolTimeoutMs?: number;
|
||||
pageNavigationTimeoutMs?: number;
|
||||
playerReadyTimeoutMs?: number;
|
||||
@@ -306,6 +308,13 @@ export class RenderObservabilityRecorder {
|
||||
return this.failedPhase !== undefined;
|
||||
}
|
||||
|
||||
/** A phase failure that was subsequently recovered (e.g. the drawElement
|
||||
* self-verify fallback re-rendering via screenshot) should not brand the
|
||||
* whole render as failed in the summary. */
|
||||
clearFailure(phase: string): void {
|
||||
if (this.failedPhase === phase) this.failedPhase = undefined;
|
||||
}
|
||||
|
||||
private record(event: RenderObservationEvent): RenderObservationEvent {
|
||||
this.eventCount++;
|
||||
const eventWithJob = { ...event, renderJobId: this.input.renderJobId };
|
||||
|
||||
@@ -41,6 +41,11 @@
|
||||
* into a shared module so the stages can import without reaching back.
|
||||
*/
|
||||
|
||||
import { execFile } from "node:child_process";
|
||||
import { mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
import {
|
||||
type BeforeCaptureHook,
|
||||
type CaptureOptions,
|
||||
@@ -48,6 +53,7 @@ import {
|
||||
type CaptureSession,
|
||||
type EngineConfig,
|
||||
type StreamingEncoder,
|
||||
DrawElementVerificationError,
|
||||
captureFrameToBuffer,
|
||||
captureFrameToBufferPipelined,
|
||||
captureFramesBatchPipelined,
|
||||
@@ -57,6 +63,8 @@ import {
|
||||
distributeFrames,
|
||||
executeParallelCapture,
|
||||
getCapturePerfSummary,
|
||||
getFfmpegBinary,
|
||||
recaptureDrawElementFrameForVerify,
|
||||
initializeSession,
|
||||
prepareCaptureSessionForReuse,
|
||||
spawnStreamingEncoder,
|
||||
@@ -137,6 +145,28 @@ export type CaptureStreamingStageResult =
|
||||
success: false;
|
||||
};
|
||||
|
||||
const execFileP = promisify(execFile);
|
||||
|
||||
/** PSNR (average, dB) between two same-dimension encoded images via ffmpeg. */
|
||||
async function psnrDb(a: Buffer, b: Buffer): Promise<number> {
|
||||
const dir = await mkdtemp(join(tmpdir(), "hf-de-verify-"));
|
||||
try {
|
||||
const pa = join(dir, "a.jpg");
|
||||
const pb = join(dir, "b.jpg");
|
||||
await Promise.all([writeFile(pa, a), writeFile(pb, b)]);
|
||||
const { stderr } = await execFileP(
|
||||
getFfmpegBinary(),
|
||||
["-hide_banner", "-i", pa, "-i", pb, "-lavfi", "psnr", "-f", "null", "-"],
|
||||
{ maxBuffer: 4 * 1024 * 1024 },
|
||||
);
|
||||
const m = /average:(inf|[\d.]+)/.exec(stderr);
|
||||
if (!m) throw new Error(`psnr parse failed: ${stderr.slice(-300)}`);
|
||||
return m[1] === "inf" ? Infinity : Number(m[1]);
|
||||
} finally {
|
||||
await rm(dir, { recursive: true, force: true }).catch(() => {});
|
||||
}
|
||||
}
|
||||
|
||||
async function runWorkerEncodePipelineLoop(
|
||||
session: CaptureSession,
|
||||
totalFrames: number,
|
||||
@@ -145,8 +175,137 @@ async function runWorkerEncodePipelineLoop(
|
||||
reorderBuffer: ReturnType<typeof createFrameReorderBuffer>,
|
||||
assertNotAborted: () => void,
|
||||
onProgress: CaptureStreamingStageInput["onProgress"],
|
||||
log: CaptureStreamingStageInput["log"],
|
||||
): Promise<void> {
|
||||
let prev: { idx: number; encodeResult: Promise<Buffer> } | null = null;
|
||||
const frameTime = (i: number) => (i * job.config.fps.den) / job.config.fps.num;
|
||||
|
||||
// ── drawElement drain-time safety checks (ungated-release safety net) ──
|
||||
// Runs on every drained frame. Drains execute strictly between capture
|
||||
// evaluates (see the batch NOTE below), so a re-seek + recapture here is
|
||||
// safe — nothing else is mutating page time.
|
||||
//
|
||||
// 1. Blank guard (worker-path analogue of the serial-path guard in
|
||||
// frameCapture): drawElement intermittently returns an anomalously small
|
||||
// blank frame with no throw. A frame far below the rolling-median byte
|
||||
// size is re-captured once (verification-grade recapture — no dedup
|
||||
// shortcut, no screenshot fallback); still blank → verification error.
|
||||
// The floor only arms after 12 drained frames — early frames are covered
|
||||
// by the PSNR verify samples rather than the size heuristic, which has
|
||||
// no stable median yet.
|
||||
// 2. Self-verification: at K sampled indices, compare the DE frame against
|
||||
// its pre-injection screenshot ground truth (session.deVerifyFrames).
|
||||
// PSNR below HF_DE_VERIFY_MIN_DB (default 32; natural DE-vs-screenshot
|
||||
// agreement measures ≥45, damage <30) → verification error.
|
||||
// A DrawElementVerificationError propagates to the orchestrator, which
|
||||
// re-renders the whole job via the screenshot path (never-wrong fallback).
|
||||
// Clamp to a defensible band: below ~10dB even severe damage passes (the
|
||||
// check stops meaning anything); above ~60dB natural DE-vs-screenshot
|
||||
// encoder differences (~45dB+) would force a screenshot fallback on every
|
||||
// verified render. Out-of-range or malformed values fall back to 32.
|
||||
const verifyMinDbRaw = Number(process.env.HF_DE_VERIFY_MIN_DB ?? "32");
|
||||
const verifyMinDb =
|
||||
Number.isFinite(verifyMinDbRaw) && verifyMinDbRaw >= 10 && verifyMinDbRaw <= 60
|
||||
? verifyMinDbRaw
|
||||
: 32;
|
||||
if (process.env.HF_DE_VERIFY_MIN_DB !== undefined && verifyMinDb !== verifyMinDbRaw) {
|
||||
log.warn("[Render] HF_DE_VERIFY_MIN_DB out of range [10,60]; using 32", {
|
||||
raw: process.env.HF_DE_VERIFY_MIN_DB,
|
||||
});
|
||||
}
|
||||
const sizes: number[] = [];
|
||||
// Absolute floor lowers once a small frame proves deterministic (dark /
|
||||
// low-detail content), so a dark stretch doesn't re-capture every frame.
|
||||
let absFloor = 20_000;
|
||||
const blankFloor = (): number => {
|
||||
if (sizes.length < 12) return 0;
|
||||
const sorted = [...sizes].sort((x, y) => x - y);
|
||||
const median = sorted[Math.floor(sorted.length / 2)] ?? 0;
|
||||
return Math.max(absFloor, median * 0.12);
|
||||
};
|
||||
let acceptedSmall: Buffer | null = null;
|
||||
const guardFrame = async (idx: number, buf: Buffer): Promise<Buffer> => {
|
||||
if (process.env.HF_FORCE_DRAWELEMENT !== "1") {
|
||||
const floor = blankFloor();
|
||||
if (floor > 0 && buf.length < floor && acceptedSmall?.equals(buf)) {
|
||||
// Identical to a small frame already proven deterministic (dark
|
||||
// clip-gap runs repeat the same bytes) — skip the recapture.
|
||||
} else if (floor > 0 && buf.length < floor) {
|
||||
log.warn("[Render] drawElement blank-frame suspect; re-capturing", {
|
||||
frame: idx,
|
||||
bytes: buf.length,
|
||||
floor: Math.round(floor),
|
||||
});
|
||||
// Verification-grade recapture: NOT captureFrameToBufferPipelined,
|
||||
// whose static-dedup fast path and "No cached paint record" screenshot
|
||||
// fallback can both return a DIFFERENT frame's pixels at drain time
|
||||
// (dedup anchor runs ahead of the drain; the fallback screenshots the
|
||||
// injected canvas = last drawn DE frame). Any recapture failure is a
|
||||
// verification failure — fall back the whole render.
|
||||
let retryBuf: Buffer;
|
||||
try {
|
||||
retryBuf = await recaptureDrawElementFrameForVerify(session, idx, frameTime(idx));
|
||||
} catch (err) {
|
||||
throw new DrawElementVerificationError(
|
||||
`blank drawElement frame ${idx}: ${buf.length}B < floor ${Math.round(floor)}B and recapture failed (${err instanceof Error ? err.message : String(err)})`,
|
||||
);
|
||||
}
|
||||
if (retryBuf.equals(buf)) {
|
||||
// Byte-identical on re-capture ⇒ deterministic content, not a
|
||||
// transient blank drop (those are intermittent by nature) — the
|
||||
// frame is legitimately small (dark / low-detail). Accept it;
|
||||
// deterministic damage classes are the PSNR self-verify's job.
|
||||
log.info("[Render] drawElement small frame is deterministic; accepted", {
|
||||
frame: idx,
|
||||
bytes: buf.length,
|
||||
});
|
||||
absFloor = Math.min(absFloor, Math.floor(buf.length / 2));
|
||||
acceptedSmall = buf;
|
||||
} else if (retryBuf.length < floor) {
|
||||
throw new DrawElementVerificationError(
|
||||
`blank drawElement frame ${idx}: ${buf.length}B (retry ${retryBuf.length}B) < floor ${Math.round(floor)}B`,
|
||||
);
|
||||
} else {
|
||||
buf = retryBuf;
|
||||
}
|
||||
}
|
||||
sizes.push(buf.length);
|
||||
if (sizes.length > 60) sizes.shift();
|
||||
}
|
||||
const truth = session.deVerifyFrames?.get(idx);
|
||||
if (truth) {
|
||||
let db: number;
|
||||
try {
|
||||
db = await psnrDb(buf, truth);
|
||||
} catch (err) {
|
||||
// Infrastructure failure (ffmpeg spawn/parse/tmpdir), not evidence of
|
||||
// damage — skip this sample rather than failing or falling back.
|
||||
log.warn("[Render] drawElement self-verify sample skipped (psnr infrastructure)", {
|
||||
frame: idx,
|
||||
error: err instanceof Error ? err.message : String(err),
|
||||
});
|
||||
return buf;
|
||||
}
|
||||
if (db < verifyMinDb) {
|
||||
// Keep the mismatched pair for diagnosis (tmpdir; OS-reaped).
|
||||
const dumpDir = await mkdtemp(join(tmpdir(), "hf-de-verify-fail-")).catch(() => null);
|
||||
if (dumpDir) {
|
||||
await Promise.all([
|
||||
writeFile(join(dumpDir, `frame-${idx}-de.jpg`), buf),
|
||||
writeFile(join(dumpDir, `frame-${idx}-truth.jpg`), truth),
|
||||
]).catch(() => {});
|
||||
}
|
||||
throw new DrawElementVerificationError(
|
||||
`drawElement self-verify failed at frame ${idx}: ${db.toFixed(1)}dB < ${verifyMinDb}dB vs pre-injection screenshot${dumpDir ? ` (pair: ${dumpDir})` : ""}`,
|
||||
);
|
||||
}
|
||||
log.info("[Render] drawElement self-verify passed", {
|
||||
frame: idx,
|
||||
psnrDb: db === Infinity ? "inf" : Number(db.toFixed(1)),
|
||||
});
|
||||
}
|
||||
return buf;
|
||||
};
|
||||
|
||||
const drainPrev = async (): Promise<void> => {
|
||||
if (!prev) return;
|
||||
@@ -154,7 +313,7 @@ async function runWorkerEncodePipelineLoop(
|
||||
// longest stretch of the loop); without this an abort isn't seen until the
|
||||
// next produce iteration.
|
||||
assertNotAborted();
|
||||
const buf = await prev.encodeResult;
|
||||
const buf = await guardFrame(prev.idx, await prev.encodeResult);
|
||||
await reorderBuffer.waitForFrame(prev.idx);
|
||||
ensureFrameWritten(await currentEncoder.writeFrame(buf), prev.idx, currentEncoder);
|
||||
reorderBuffer.advanceTo(prev.idx + 1);
|
||||
@@ -168,32 +327,33 @@ async function runWorkerEncodePipelineLoop(
|
||||
);
|
||||
};
|
||||
|
||||
// P6 (HF_DE_BATCH=N, N>1): capture runs of consecutive frames in ONE CDP
|
||||
// round-trip each (in-page seek+paint+draw loop), amortizing protocol latency
|
||||
// N-fold. Batches break at static-dedup frames (skipped via lastEncodeResult
|
||||
// reuse — order-dependent) and at opt-in boundary-screenshot frames; those go
|
||||
// through the per-frame path unchanged. onBeforeCapture (video frame
|
||||
// injection) needs a node-side hook per frame → no batching.
|
||||
const batchN = Math.floor(Number(process.env.HF_DE_BATCH ?? "0"));
|
||||
// Batch capture (HF_DE_BATCH=N, N>1, default 4 — validated +1.20x lossless on
|
||||
// the 19-comp gate): capture runs of consecutive frames in ONE CDP round-trip
|
||||
// each (in-page seek+paint+draw loop), amortizing protocol latency N-fold.
|
||||
// Batches break at static-dedup frames (skipped via lastEncodeResult reuse —
|
||||
// order-dependent) and at opt-in boundary-screenshot frames; those go through
|
||||
// the per-frame path unchanged. onBeforeCapture (video frame injection) needs
|
||||
// a node-side hook per frame → no batching. Kill switch: HF_DE_BATCH=0.
|
||||
const batchNRaw = Number(process.env.HF_DE_BATCH ?? "4");
|
||||
const batchN = Number.isFinite(batchNRaw) ? Math.floor(batchNRaw) : 4;
|
||||
const drainBatch = async (batch: Array<{ idx: number; encodeResult: Promise<Buffer> }>) => {
|
||||
for (const item of batch) {
|
||||
assertNotAborted();
|
||||
const buf = await guardFrame(item.idx, await item.encodeResult);
|
||||
await reorderBuffer.waitForFrame(item.idx);
|
||||
ensureFrameWritten(await currentEncoder.writeFrame(buf), item.idx, currentEncoder);
|
||||
reorderBuffer.advanceTo(item.idx + 1);
|
||||
job.framesRendered = item.idx + 1;
|
||||
updateJobStatus(
|
||||
job,
|
||||
"rendering",
|
||||
`Streaming frame ${item.idx + 1}/${totalFrames}`,
|
||||
Math.round(25 + ((item.idx + 1) / totalFrames) * 55),
|
||||
onProgress,
|
||||
);
|
||||
}
|
||||
};
|
||||
if (batchN > 1 && !session.onBeforeCapture) {
|
||||
const frameTime = (f: number) => (f * job.config.fps.den) / job.config.fps.num;
|
||||
const drainBatch = async (batch: Array<{ idx: number; encodeResult: Promise<Buffer> }>) => {
|
||||
for (const item of batch) {
|
||||
assertNotAborted();
|
||||
const buf = await item.encodeResult;
|
||||
await reorderBuffer.waitForFrame(item.idx);
|
||||
ensureFrameWritten(await currentEncoder.writeFrame(buf), item.idx, currentEncoder);
|
||||
reorderBuffer.advanceTo(item.idx + 1);
|
||||
job.framesRendered = item.idx + 1;
|
||||
updateJobStatus(
|
||||
job,
|
||||
"rendering",
|
||||
`Streaming frame ${item.idx + 1}/${totalFrames}`,
|
||||
Math.round(25 + ((item.idx + 1) / totalFrames) * 55),
|
||||
onProgress,
|
||||
);
|
||||
}
|
||||
};
|
||||
const boundarySS = process.env.HF_FAST_CAPTURE_BOUNDARY_SS === "true";
|
||||
const batchable = (f: number) =>
|
||||
!session.staticFrames?.has(f) && !(boundarySS && session.clipBoundaryFrames?.has(f));
|
||||
@@ -207,6 +367,12 @@ async function runWorkerEncodePipelineLoop(
|
||||
idxs.push(i);
|
||||
i++;
|
||||
}
|
||||
// NOTE: draining the previous batch CONCURRENTLY with this capture
|
||||
// (kick the evaluate un-awaited, drain, then await) was prototyped and
|
||||
// REJECTED 2026-07-03: ~1.0–1.03× on medium/long comps but it perturbed
|
||||
// pixels on a deterministic comp (87dB vs ∞ noise floor — main-thread
|
||||
// contention shifting a paint-wait to the timeout path). Keep the
|
||||
// drain strictly between batch evaluates.
|
||||
const results = await captureFramesBatchPipelined(session, idxs, idxs.map(frameTime));
|
||||
await drainBatch(prevBatch);
|
||||
prevBatch = results.map((r) => ({ idx: r.frameIndex, encodeResult: r.encodeResult }));
|
||||
@@ -228,7 +394,7 @@ async function runWorkerEncodePipelineLoop(
|
||||
// the loop needs no special guard here.
|
||||
for (let i = 0; i < totalFrames; i++) {
|
||||
assertNotAborted();
|
||||
const time = (i * job.config.fps.den) / job.config.fps.num;
|
||||
const time = frameTime(i);
|
||||
const { encodeResult } = await captureFrameToBufferPipelined(session, i, time);
|
||||
await drainPrev();
|
||||
prev = { idx: i, encodeResult };
|
||||
@@ -388,6 +554,7 @@ export async function runCaptureStreamingStage(
|
||||
reorderBuffer,
|
||||
assertNotAborted,
|
||||
onProgress,
|
||||
log,
|
||||
);
|
||||
} else {
|
||||
for (let i = 0; i < totalFrames; i++) {
|
||||
|
||||
@@ -191,6 +191,31 @@ export async function runCompileStage(input: CompileStageInput): Promise<Compile
|
||||
"for this render. Capture uses the platform's baseline route.",
|
||||
);
|
||||
}
|
||||
// Shader-transition comps: page-side compositing is the faster, purpose-built
|
||||
// path for them, and resolveConfig force-disables it whenever drawElement is
|
||||
// on. With drawElement default-on that trade is backwards — prefer page-side
|
||||
// compositing and route these comps to the baseline capture. An explicit
|
||||
// drawElement opt-in keeps the old preference.
|
||||
if (
|
||||
cfg.useDrawElement &&
|
||||
process.env.PRODUCER_EXPERIMENTAL_FAST_CAPTURE !== "true" &&
|
||||
compiled.hasShaderTransitions
|
||||
) {
|
||||
cfg.useDrawElement = false;
|
||||
log.info(
|
||||
"[Render] Fast capture: composition uses shader transitions — disabling drawElementImage " +
|
||||
"so page-side compositing stays available.",
|
||||
);
|
||||
}
|
||||
// Whenever a compile-time gate cleared useDrawElement, restore page-side
|
||||
// compositing — but ONLY when resolveConfig auto-disabled it because
|
||||
// drawElement was going to run. An explicit enablePageSideCompositing:false
|
||||
// from the programmatic API or HF_PAGE_SIDE_COMPOSITING=false never had the
|
||||
// auto-disabled flag set and stays off.
|
||||
if (!cfg.useDrawElement && cfg.pageSideCompositingAutoDisabled) {
|
||||
cfg.enablePageSideCompositing = true;
|
||||
cfg.pageSideCompositingAutoDisabled = false;
|
||||
}
|
||||
const callerForced = cfg.forceScreenshot || (needsAlpha && !cfg.useDrawElement);
|
||||
const { forceScreenshot: hintForced } = applyRenderModeHints(callerForced, compiled, log);
|
||||
let forceScreenshot = hintForced;
|
||||
|
||||
@@ -72,6 +72,7 @@ import {
|
||||
scaleProtocolTimeoutForComposition,
|
||||
isMemoryExhaustionError,
|
||||
isTransientBrowserError,
|
||||
isDrawElementVerificationError,
|
||||
} from "@hyperframes/engine";
|
||||
import { join, dirname, resolve } from "path";
|
||||
import { randomUUID } from "crypto";
|
||||
@@ -1443,6 +1444,9 @@ export async function executeRenderJob(
|
||||
...captureOptions,
|
||||
videoMetadataHints,
|
||||
skipReadinessVideoIds: videoReadinessSkipIds,
|
||||
// Probe-resolved duration: drawElement self-verification derives its
|
||||
// sample frame indices from this so they land inside the drained range.
|
||||
compositionDurationSeconds: job.duration,
|
||||
});
|
||||
// The URL-served frame path (PR #596) hands each injected `<img>` a
|
||||
// fileServer URL instead of a base64 data URI, on the theory that
|
||||
@@ -1569,6 +1573,29 @@ export async function executeRenderJob(
|
||||
durationSeconds: job.duration,
|
||||
maxDurationSeconds: cfg.streamingEncodeMaxDurationSeconds,
|
||||
});
|
||||
// Default-on drawElement is only safe where the runtime self-verification
|
||||
// net actually runs: the single-worker streaming worker-encode drain. The
|
||||
// disk path (png-sequence / over the streaming duration cap) and parallel
|
||||
// capture ship frames no drain verifies — route those renders to the
|
||||
// screenshot baseline unless drawElement was explicitly opted into.
|
||||
if (
|
||||
cfg.useDrawElement &&
|
||||
process.env.PRODUCER_EXPERIMENTAL_FAST_CAPTURE !== "true" &&
|
||||
(!useStreamingEncode || workerCount > 1)
|
||||
) {
|
||||
cfg.useDrawElement = false;
|
||||
log.info(
|
||||
"[Render] Fast capture: default-on drawElement disabled for this render — " +
|
||||
(workerCount > 1 ? "parallel capture" : "the disk capture path") +
|
||||
" has no runtime self-verification. Set PRODUCER_EXPERIMENTAL_FAST_CAPTURE=true to override.",
|
||||
);
|
||||
// The probe session already initialized in drawElement mode (canvas
|
||||
// injected); it must not be reused by the unverified path.
|
||||
if (probeSession && probeSession.captureMode === "drawelement") {
|
||||
await closeCaptureSession(probeSession);
|
||||
probeSession = null;
|
||||
}
|
||||
}
|
||||
|
||||
// `captureAttempts` is declared at function scope above (shared with the
|
||||
// catch block). Static-dedup perf, appended per sequential session / per
|
||||
@@ -1759,46 +1786,75 @@ export async function executeRenderJob(
|
||||
let streamingHandled = false;
|
||||
if (useStreamingEncode) {
|
||||
const captureFrameStart = Date.now();
|
||||
const streamingRes = await observeRenderStage(
|
||||
observability,
|
||||
"capture_streaming",
|
||||
{ workerCount, forceScreenshot: captureForceScreenshot },
|
||||
() =>
|
||||
runCaptureStreamingStage({
|
||||
fileServer: activeFileServer,
|
||||
workDir,
|
||||
framesDir,
|
||||
videoOnlyPath,
|
||||
job,
|
||||
totalFrames,
|
||||
cfg,
|
||||
forceScreenshot: captureForceScreenshot,
|
||||
log,
|
||||
workerCount,
|
||||
probeSession,
|
||||
outputFormat,
|
||||
streamingEncoderOptions: {
|
||||
fps: job.config.fps,
|
||||
width,
|
||||
height,
|
||||
codec: preset.codec,
|
||||
preset: preset.preset,
|
||||
quality: effectiveQuality,
|
||||
bitrate: effectiveBitrate,
|
||||
pixelFormat: preset.pixelFormat,
|
||||
vp9CpuUsed: cfg.vp9CpuUsed,
|
||||
useGpu: job.config.useGpu,
|
||||
imageFormat: captureOptions.format || "jpeg",
|
||||
hdr: preset.hdr,
|
||||
},
|
||||
buildCaptureOptions,
|
||||
createRenderVideoFrameInjector,
|
||||
abortSignal,
|
||||
assertNotAborted,
|
||||
onProgress,
|
||||
dedupPerfs,
|
||||
}),
|
||||
);
|
||||
const invokeStreaming = () =>
|
||||
observeRenderStage(
|
||||
observability,
|
||||
"capture_streaming",
|
||||
{ workerCount, forceScreenshot: captureForceScreenshot },
|
||||
() =>
|
||||
runCaptureStreamingStage({
|
||||
fileServer: activeFileServer,
|
||||
workDir,
|
||||
framesDir,
|
||||
videoOnlyPath,
|
||||
job,
|
||||
totalFrames,
|
||||
cfg,
|
||||
forceScreenshot: captureForceScreenshot,
|
||||
log,
|
||||
workerCount,
|
||||
probeSession,
|
||||
outputFormat,
|
||||
streamingEncoderOptions: {
|
||||
fps: job.config.fps,
|
||||
width,
|
||||
height,
|
||||
codec: preset.codec,
|
||||
preset: preset.preset,
|
||||
quality: effectiveQuality,
|
||||
bitrate: effectiveBitrate,
|
||||
pixelFormat: preset.pixelFormat,
|
||||
vp9CpuUsed: cfg.vp9CpuUsed,
|
||||
useGpu: job.config.useGpu,
|
||||
imageFormat: captureOptions.format || "jpeg",
|
||||
hdr: preset.hdr,
|
||||
},
|
||||
buildCaptureOptions,
|
||||
createRenderVideoFrameInjector,
|
||||
abortSignal,
|
||||
assertNotAborted,
|
||||
onProgress,
|
||||
dedupPerfs,
|
||||
}),
|
||||
);
|
||||
let streamingRes;
|
||||
try {
|
||||
streamingRes = await invokeStreaming();
|
||||
} catch (err) {
|
||||
// drawElement self-verification tripped (blank frame or PSNR breach
|
||||
// vs the pre-injection ground truth). The whole render restarts on
|
||||
// the screenshot path — slower, never wrong. The failed attempt's
|
||||
// session was closed by the stage's finally; probeSession (if any)
|
||||
// was consumed by it, so a fresh session spawns on retry.
|
||||
if (!isDrawElementVerificationError(err)) throw err;
|
||||
log.warn("[Render] drawElement self-verification failed; re-rendering via screenshot", {
|
||||
error: err instanceof Error ? err.message : String(err),
|
||||
});
|
||||
observability.checkpoint(
|
||||
"capture_streaming",
|
||||
"drawElement self-verify failed; retrying with forceScreenshot",
|
||||
);
|
||||
captureForceScreenshot = true;
|
||||
updateCaptureObservability({
|
||||
forceScreenshot: true,
|
||||
deSelfVerifyFallback: true,
|
||||
});
|
||||
probeSession = null;
|
||||
streamingRes = await invokeStreaming();
|
||||
// The first attempt's error marked the phase failed; the retry
|
||||
// recovered it — don't brand the render as failed in telemetry.
|
||||
observability.clearFailure("capture_streaming");
|
||||
}
|
||||
const captureFrameMs = Date.now() - captureFrameStart;
|
||||
if (streamingRes.success) {
|
||||
streamingHandled = true;
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,20 @@
|
||||
// Single render for the worker-encode eval. Mode from env
|
||||
// (PRODUCER_EXPERIMENTAL_FAST_CAPTURE + HF_DE_WORKER_ENCODE set by caller).
|
||||
// Engine logs (capture_streaming durationMs, "worker encode initialized") go to
|
||||
// stdout; the suite shell greps them.
|
||||
import { createRenderJob, executeRenderJob } from "./src/index.ts";
|
||||
const [dir, out] = process.argv.slice(2);
|
||||
process.env.PRODUCER_ENABLE_BROWSER_POOL = "false";
|
||||
process.env.PRODUCER_BROWSER_GPU_MODE = "hardware";
|
||||
const job = createRenderJob({
|
||||
fps: 30,
|
||||
quality: "high",
|
||||
format: "mp4",
|
||||
workers: Math.max(1, Number(process.env.WORKERS || "1")),
|
||||
useGpu: false,
|
||||
hdrMode: "force-sdr",
|
||||
});
|
||||
const __t = Date.now();
|
||||
await executeRenderJob(job, dir, out);
|
||||
console.log("TOTAL_MS " + (Date.now() - __t));
|
||||
console.log("RENDER_OK");
|
||||
Reference in New Issue
Block a user