6 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Marcos HernanzandGitHub	a80c0c0588	Cache immutable current task IDs outside the task-state lock (#96180) ... ## What changed Introduce a cloneable `CurrentTaskStateHandle` that keeps the execution's immutable `Option<TaskId>` beside the existing shared `Arc<RwLock<CurrentTaskState>>`. Global tasks, local tasks, detached futures, and top-level runs continue to share the same mutable state and task-local scope. Only current-task ID reads bypass the lock; all other state remains behind the existing `RwLock`. This was split from #96179 following review feedback. It is source-independent and changes only `turbopack/crates/turbo-tasks/src/manager.rs`. ## Why `current_task_if_available` and `current_task` acquired a shared standard-library `RwLock` on every native function call and tracked read, even though the task ID is fixed for the entire execution. The lock and its contention path remained visible in both HMR and production profiles. ## Performance evidence Measured at `0950e8ae05db20c33302f02257114f5b857edabc` as an incremental change on top of the independently proposed immutable-read bypass in #96179. These measurements are directional evidence, not a precise standalone estimate. Changes around 1–2% are difficult to measure reliably. The #96179 and #96180 campaigns used separate run sets, and #96179's absolute endpoint does not match this campaign's starting point; that mismatch demonstrates cross-campaign noise. The percentages must not be added, and the absolute values must not be compared across the two PRs. The standalone GitHub aggregate benchmark also moved in the opposite direction, reinforcing that build-level percentages are inconclusive at this scale. ### Browser-driven HMR, 30-second B/C/C/B - Evaluation: 19.772807 → 19.356650 ms (**-2.105%**) - Commit: 20.632962 → 20.167726 ms (**-2.255%**) - Both candidate runs beat both controls for both metrics. ### Production builds - Matrix 1: **-1.804%** - Complementary matrix 2: **-1.474%** - Combined 24 campaigns: 1,641.839610 → 1,614.918617 ms (**-1.640%**) - All six balanced blocks favored the candidate. - Six-block mean: -1.636%, with 95% interval **[-2.718%, -0.555%]**. ### Causal profiles - The `current_task_if_available` helper and contended read-lock symbols disappeared from candidate profiles. - HMR voluntary context switches fell 2.700%. - Production voluntary context switches fell 2.576%. - All eight accepted perf captures reported zero lost samples. ## Validation On the exact standalone head (`bce272743ff44021fda204d98924bf8519cd8c1d`): - Complete `turbo-tasks` package tests and doctests - Complete `turbo-tasks-backend` package tests, integrations, and doctests - `cargo fmt --check -p turbo-tasks -p turbo-tasks-backend` - `cargo clippy -p turbo-tasks -p turbo-tasks-backend --all-targets -- -D warnings` - `git diff --check` - Conflict-free merge-tree check against current `canary` (`91c6309c52`)	2026-07-29 15:22:34 -04:00
3 people	779fbad904	Optimize implicit cache tag derivation (#96120) ... ## What Derive implicit cache tags by scanning pathname slash boundaries incrementally instead of repeatedly rebuilding prefixes with `split`, `slice`, and `join`. The new loop emits exactly the same derived-tag array as the old helper, including the full-path prefix. The caller's existing `Set` continues to deduplicate that tag against `pathname` when they are equal. This PR's optimization is specifically the removal of repeated intermediate path arrays and prefix joins; it does not rely on changing deduplication semantics or tag order. Coverage includes root paths, `page` and `route` suffixes, missing leading slashes, repeated/trailing slashes, Unicode, malformed UTF-16, and already-encoded pathnames. ## Correctness The candidate was compared element-by-element with the original `split('/').slice(0, i).join('/')` implementation over **1,713,653 paths** on each of Node 20.9.0, Node 22.23.1, and Node 24.14.1: - 349,525 exhaustive strings of length 0–9 over a slash/ASCII alphabet; - every UTF-16 code unit: 65,536 cases; - every valid surrogate pair: 1,048,576 cases; - 250,000 deterministic randomized paths with Unicode, malformed surrogates, controls, reserved characters, suffix-like content, and arbitrary slash placement; - 16 targeted cases, including a one-million-character segment and a 1,024-segment path. All three runtime runs completed with **0 mismatches**. ## Performance Pinned CPU 4 on a 16-vCPU / 32-GiB Vercel DevBox. Each of 35 scenarios used 31 interleaved rounds, adaptive iterations, and two identical baseline lanes (A/A control). Scenarios cover depth 1–256, slash-dense paths, segment widths 1–512, suffix cases, no-leading-slash inputs, Unicode/malformed code units, and seeded shallow/medium/deep corpora. | Runtime | Scenarios improved | Seeded shallow | Seeded medium | Seeded deep | | --- | ---: | ---: | ---: | ---: | | Node 20.9.0 / V8 11.3 | 35/35 | **-77.11%** | **-82.59%** | **-92.69%** | | Node 22.23.1 / V8 12.4 | 35/35 | **-81.07%** | **-88.60%** | **-95.47%** | | Node 24.14.1 / V8 13.6 | 35/35 | **-80.98%** | **-89.00%** | **-95.88%** | No candidate p05–p95 interval crosses zero on any runtime. Selected Node 24 results: | Scenario | Median delta | p05–p95 | | --- | ---: | ---: | | Realistic root/suffix corpus | **-77.70%** | -78.02% to -77.22% | | Repeated/trailing slashes | **-77.88%** | -79.24% to -77.07% | | Unicode and malformed UTF-16 | **-63.49%** | -64.59% to -62.31% | | Depth 8, 4-char segments | **-83.14%** | -83.98% to -82.74% | | Depth 32, 4-char segments | **-92.48%** | -92.60% to -92.22% | | Depth 256, 4-char segments | **-98.77%** | -98.80% to -98.75% | | Depth 32, 512-char segments | **-97.02%** | -97.09% to -96.98% | Inspector heap-allocation sampling, five repeats (median sampled bytes): | Corpus | Baseline | Candidate | Change | | --- | ---: | ---: | ---: | | Realistic, 500k calls | 489,583,800 | 206,253,144 | **-57.87%** | | Shallow, 500k calls | 493,465,152 | 203,994,592 | **-58.66%** | | Deep, 20k calls | 843,885,312 | 112,216,736 | **-86.70%** | ### Alternative tested A split-once implementation that accumulated prefix strings also passed all 1.71M cases, but a direct 35-scenario interleaved comparison found the shipped `indexOf` implementation faster in **35/35** cases. Accumulation was 26.49% to 1,810.86% slower and sampled 47.5%/70.3%/280.8% more bytes on realistic/shallow/deep corpora. It was rejected. ### Production profile The original matched render-pipeline profile used 1,000 serial plus 10,000 concurrent-load requests: | Attribution | Baseline | Candidate | | --- | ---: | ---: | | Total sampled CPU | 20,534.6 ms | 20,461.6 ms | | Next runtime sampled CPU | 7,786.1 ms | 7,856.8 ms | | `implicit-tags.ts` | 54.1 ms (0.69%) | 41.4 ms (0.53%) | That is a **23.48% reduction** in source-attributed production CPU. A single load-throughput run improved 3.64%, but that is inside the workload's established 6.76% full range, so this PR does not claim a separately resolvable total-throughput improvement. ## Validation - `pnpm exec jest packages/next/src/server/lib/implicit-tags.test.ts --runInBand` — 12/12 passed - `pnpm prettier --check packages/next/src/server/lib/implicit-tags.ts packages/next/src/server/lib/implicit-tags.test.ts` - `pnpm eslint packages/next/src/server/lib/implicit-tags.ts packages/next/src/server/lib/implicit-tags.test.ts` - `pnpm --filter next types` - `pnpm --filter next build` - `pnpm build-all` — 18/18 tasks successful, including `@vercel/turbopack-next` checks - Correctly scoped Codex GPT-5.6 Sol xhigh review: clean, 0.98 correctness confidence - Correctly scoped Claude Opus 4.8 xhigh review: clean, 0.90 correctness confidence GitHub CI has 111 passing checks. The Windows integration job failed after `curl` received a connection reset while downloading Datadog tooling (exit 35); the aggregate `thank you, next` check reflects that infrastructure failure. No changed-file or implicit-tag test failed. --------- Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>	2026-07-25 00:45:44 +00:00
3 people	d5cc1e12a7	Avoid quadratic HMR queue shifts (#96137) ... ## What - Replace repeated `Array.shift()` calls in the HMR affected-module breadth-first traversal with a monotonically increasing queue index. - Clear each consumed slot immediately so copied dependency chains can be collected during deep traversals. ## Why `getAffectedModuleEffects` can enqueue a wide set of parents when a shared dependency changes. Repeatedly shifting the queue moves the remaining array contents on every iteration, making broad invalidations increasingly expensive. Indexed reads keep traversal FIFO and make queue consumption linear. The slot clearing is important for deep graphs: it avoids retaining every processed item's copied `dependencyChain` until traversal returns. ## Benchmark Three independent 8-vCPU/16-GiB Vercel Sandbox workers ran Node.js 24.14.1 from exact commit `0950e8ae05db20c33302f02257114f5b857edabc`. The balanced `A1/B1/B2/A2` benchmark mirrors this loop for one changed shared dependency imported by N root modules, including dependency-chain allocation. Each phase used three warm-ups and nine samples. | Affected parents | `shift()` median | Indexed median | Improvement | Speedup | Worker agreement | Max baseline A/A drift | |---:|---:|---:|---:|---:|---:|---:| | 1,000 | 0.160 ms | 0.089 ms | 45.81% | 1.85x | 3/3 | 9.91% | | 20,000 | 27.895 ms | 2.929 ms | 89.43% | 9.46x | 3/3 | 7.85% | | 50,000 | 164.960 ms | 7.342 ms | 95.52% | 22.33x | 3/3 | 1.21% | | 100,000 | 665.564 ms | 15.840 ms | 97.62% | 42.02x | 3/3 | 2.23% | The small-graph effect is intentionally modest; this targets very broad HMR invalidations in large applications. ## Validation - targeted Prettier: pass - targeted ESLint: pass - `pnpm --filter @vercel/turbopack-ecmascript-runtime check:nodejs`: pass - `CARGO_TARGET_DIR=/home/vercel-sandbox/next.js/target cargo check -p turbopack-ecmascript-runtime`: pass - GPT-5.6 Sol xhigh + Claude Opus 4.8 xhigh autoreview: clean after addressing a first-round dependency-chain retention finding (0 findings, correctness 0.99) The aggregate runtime TypeScript check remains blocked by an existing `runtime-utils.ts:239` `EsmExport` type mismatch. The identical `check:browser-dev-client` failure reproduces on the untouched baseline and is unrelated to this file. --------- Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>	2026-07-24 17:34:59 -07:00
Marcos HernanzandGitHub	6c5e0bbd74	[turbopack] Skip redundant top-level root updates (#95903) ... ## Summary - skip the redundant aggregation-root update when `ConnectChildOperation` reconnects an already constructed top-level task that is already an aggregation root - avoid entering the aggregation operation loop when no update was queued The fast path is intentionally narrow. It applies only when the connection has no parent, the task has materialized output, and the task's aggregation number is already the root sentinel. The task may still be dirty and need to run; this only skips an idempotent aggregation update. A task that is not yet a root still goes through the normal update on its first top-level connection. Tasks without output are still scheduled, parented connections retain their activeness and aggregation updates, and transient-state boundaries are unchanged. ## Findings The earlier scheduler experiment has been removed completely. The apparent benefit from keeping a worker alive with `std::thread::yield_now()` was specific to the benchmark topology: Criterion's `Runtime::block_on` caller runs outside the single-worker Tokio runtime, so that external caller can enqueue the next root task while the runtime worker is yielding at the OS level. The sole Tokio worker itself cannot produce work during that loop. In normal scheduling, tasks typically enqueue their children before the current job exits, and a multi-worker runtime can start another worker when it is below its target. That does not justify burning CPU in an idle worker, nor replacing the loop with an async-yield retry loop. The retained optimization is independent of that artifact. An already constructed top-level task that is already rooted has no new root update to perform; re-enqueuing the idempotent update only forces the operation executor through unnecessary suspend points. This PR now contains no changes to `priority_runner`, worker lifetime, busy waiting, OS-thread yielding, async yielding, HMR, filesystem watching, or documentation. ## Benchmark results The benchmark source, runtime construction, profiles, allocator, toolchain, and worker count were unchanged. Frozen baseline and candidate executables were run adjacently in alternating order. Three focused pairs met the control-drift threshold; one pair was excluded before analysis because its Tokio control moved by more than 2%. Median candidate changes across the valid pairs: | Case | Change | | --- | ---: | | direct control | +0.071% | | Tokio control | -0.177% | | uncached task | +1.266% | | cached, same key | -37.868% | | cached, different keys | -26.783% | The complete all-duration local sweeps did not meet the control thresholds, so their task measurements are deliberately not cited. Fresh remote CodeSpeed and Stats results are the authoritative guardrail for the full benchmark. ## Verification - `cargo fmt --all -- --check` - `cargo clippy -p turbo-tasks-backend --all-targets -- -D warnings -A deprecated` - `cargo test -p turbo-tasks-backend` - `pnpm build-all` - two-engine autoreview after the final changes, with no findings - Not run: `pnpm --filter=next dev` (the final diff is Rust-only and was covered by the full bootstrap build and Cargo suites)	2026-07-20 16:28:18 -07:00
Marcos HernanzandGitHub	cc57d6521d	Run Rust doctests in CI (#95911) ... ## Summary Run all Rust workspace doctests as part of the existing Turbo-cached cargo unit test task so future documentation regressions fail CI without adding an uncached workspace build. ## Verification - `cargo test --workspace --doc --profile=release-with-assertions --no-fail-fast` - Not run: `pnpm --filter=@next/swc test-cargo-unit` (`cargo-nextest` is installed in CI but unavailable locally; the new doctest suffix passed independently) <!-- NEXT_JS_LLM -->	2026-07-20 13:22:20 -07:00
Marcos HernanzandGitHub	7e817e863b	Fix Rust doctest failures (#95909) ... ## Summary - Fix Rust documentation examples so workspace doctests compile successfully. - Mark incomplete illustrative snippets as ignored while keeping runnable examples checked. - Update stale imports and API usage, and classify directory trees as text. ## Verification - cargo fmt --all -- --check - cargo test --workspace --doc --no-fail-fast - autoreview --mode local --stream-engine-output (Codex + Claude: 0 findings)	2026-07-17 15:39:16 -07:00