This is a pretty minor win, but it can save a few seconds per job.
https://github.blog/changelog/2026-06-25-actions-steps-can-now-be-run-in-parallel/
Also remove the installation of `lld` for rustc with `apt-get`. It's
very slow, and rustc should already ship with `lld` these days? The
composite action to install the rust toolchain isn't run often (test
shards download a prebuilt native binary), but `build-native` it can
block other jobs from running and become a bottleneck.
Testing the full `build-and-deploy` job, which uses the rust install
action: https://github.com/vercel/next.js/actions/runs/30138483769
If we get metadata events, or if we (on Linux with inotify) get file
access events, these aren't useful, and we filter them, but we were
unintentionally extending the deadline for the batch because of them.
Previously we we had 5 total `FxHashSet`s in `BatchedInvalidations`. Now
we've got one `FxHashMap` and one `FxHashSet`. There was potential
overlap in paths represented in all of these sets.
The old implementation was suboptimal for two reasons:
- We ended up cloning and storing `PathBuf` more times. (Note: there's a
good future opportunity for an arena here too)
- If a path was in both `path` and `path_and_children`, we'd later try
to invalidate paths associated with that twice. This will be a no-op as
we have a lock on the invalidator map and the duplicated path has
already removed from the invalidator map, but it makes us do extra map
lookups that aren't needed.
This still keeps around a separate set for `new_paths` so that we can
more efficiently iterate over them. It's possible it could benefit from
being a flag in the same set, but it's less obvious.
This change should be a no-op, I just moved some of the logic for
handling event processing that didn't have any control flow implications
out of the nested loop. The level of nesting here was making the code a
little hard to read.
This file was way too huge and was wasting a ton of tokens when trying
to use it with agents.
`turbo-tasks-fs/src/disk.rs` is still a little bigger than I'd like, but
I think this is an overall improvement.
Addresses
https://github.com/vercel/next.js/pull/95951/changes#r3617631291
- We often already have an `Arc<PathBuf>` here.
- The data stored in the queue is temporary, so the slight extra size of
`Arc<PathBuf>` is fine, and saving the clone is nice.
- The other references to the `Arc<PathBuf>` are not long-lived, so
convert it into a `Box<Path>` once it goes into the locked map.
We also mitigate the security impact of e2e tests by running everything
in VMs, and by not reading caches in release jobs, but we should still
be using `minimumReleaseAge` in e2e tests.
Ideally we'd also use lockfiles here too, and I'll work on that next,
but that is likely a lot more complicated because we probably need some
way to merge lockfiles, since we can't pin the next.js and react
versions in every test, and we need a reasonable/efficient way to
bulk-update all the lockfiles for tests that need it.
Depends on https://github.com/vercel/next.js/pull/95668 for a bunch of
windows path representation fixes that this ends up exposing in CI,
because adding workspace files causes pnpm to change the absolute path
representation used in the Windows junction points that it creates.
I was seeing failures on https://github.com/vercel/next.js/pull/95628
and went down this rabbit hole.
- We were (incorrectly) not canonicalizing the `DiskFileSystem` root
dir. This happened to work out for us because pnpm also wasn't
canonicalizing junction point targets, and so the root dir prefix
matched. However after `#95628`, pnpm started canonicalizing junction
point targets because we added `pnpm-workspace.yaml` files. This was
just coincidence and indicative of much deeper problems.
- `dunce` isn't a good fit for us. It picks a win32 or verbatim
representation based on path length, which would mean that long paths
inside of a short root path would never work because we'd never be able
to strip the prefix, since we'd be looking for a win32 path prefix.
Omnipath is better for what we want to do:
https://docs.rs/omnipath/latest/omnipath/windows/trait.WinPathExt.html
- The windows verbatim path format works with long paths (>260
characters) and behaves a lot closer to cross-platform unix paths, which
is what we want. E.g. if somebody tries to import a JS module from a
Windows 8.3 short path, it should fail. This will fail with verbatim
paths (good!). This PR *always* uses the verbatim path format internally
within `DiskFileSystem`.
- `read_link` was re-inventing `try_from_sys_path` but badly. Just call
`try_from_sys_path`.
- `turbopack/crates/turbo-tasks-fs/src/embed/file.rs` was dead code.
- `validate_path_length_inner` was incorrectly referring to verbatim
paths as UNC paths. These are two different things.
- Canonicalize any absolute symlink targets outside of the
`DiskFileSystem` root path, resolving any symlinks they may have,
normalizing case-insensitive paths, and handling Windows 8.3 short name
format.
## Some Context About Windows Path Formats:
- win32 paths: These are your normal user-friendly `C:\foo\blah` paths.
When calling most filesystem APIs, these are limited to 260 characters
unless the user has enabled long paths on the system, and avoiding the
limit with these paths also requires a [`longPathAware` application
manifest](https://learn.microsoft.com/en-us/windows/win32/sbscs/application-manifests#longPathAware).
You can mix `/` and `\` in win32 paths, and Windows will normalize the
path separator for you.
- verbatim paths: This is a rust-specific name (used by the stdlib) for
the "extended length" prefixed paths. These are paths starting with
`\\?\`. These paths can be up to ~32,767 characters long. They do not
support Windows 8.3 shortnames. They do not normalize between `/` and
`\`, and do not support `.` or `..` components, however
[`Path::join`](https://doc.rust-lang.org/std/path/struct.Path.html#method.join)
will do this normalization if given a verbatim path.
- Windows 8.3 shortnames: A legacy format where paths can be represented
by 8 characters, followed by a tilde and a digit (or hash in some
extreme cases where there are many collisions), followed by a 3
character file extension. NTFS supports this,
[ReFS](https://learn.microsoft.com/en-us/windows-server/storage/refs/refs-overview)
(used by [Windows Dev
Drive](https://learn.microsoft.com/en-us/windows/dev-drive/)) does not.
- UNC ("Universal Naming Convention"): This is a specific representation
of paths used for remote files on network servers.
- `file://` URIs: These must be formed from [win32 style
paths](https://docs.rs/url/latest/url/struct.Url.html#method.from_file_path),
though the normal 260 character limit typically does not apply here.
- canonicalize: Corresponds to
[`std::fs::canonicalize`](https://doc.rust-lang.org/stable/std/fs/fn.canonicalize.html).
This involves actual fs operations (that we can't easily track), and
recursively resolves symlinks. It will fail if the file path does not
exist. On windows, this always returns a verbatim path.
- wide format: Windows paths are utf-16. Generally we don't support
paths that can't be encoded to utf-8, but we should try to fail as
obviously as we can in these cases. We don't have good filesystem issue
handling yet though.
- case sensitivity: Generally paths are not case-sensitive on Windows or
macOS, but are case-sensitive on most Linux filesystems. The exact
definition of how case sensitivity works with unicode paths is highly
platform and filesystem dependent.
- junction points: The windows equivalent of symlinks (symlinks are also
supported, but require developer mode to be turned on). These can
contain an arbitrary path reference in any format (win32, verbatim, etc)
with any case and with windows 8.3 short names. Junction points are
always target absolute paths to directories.
The idea here is that we waste a bunch of CPU setting up each test suite
by spawning an entirely new Chrome process, but playwright can share the
same browser across many tests. Tests are still isolated at the browser
level, just not at the OS process level.
Hoping to see some performance improvement in e2e tests. Compare to:
https://github.com/vercel/next.js/pull/95617
Somewhat unscientific (single sample) results:
```
Comparing the head-commit build-and-test runs — PR #95589 run [28981702219 ](https://github.com/vercel/next.js/actions/runs/28981702219)vs PR #95617 run [28981686692](https://github.com/vercel/next.js/actions/runs/28981686692). Both ran 101 jobs and both succeeded.
┌─────────────────────────┬────────────────┬───────────────────────┬───────────────────┐
│ Metric │ #95617 control │ #95589 shared browser │ Delta │
├─────────────────────────┼────────────────┼───────────────────────┼───────────────────┤
│ Raw wall-time sum │ 616.2 min │ 571.2 min │ −45.0 min (−7.3%) │
├─────────────────────────┼────────────────┼───────────────────────┼───────────────────┤
│ Billable (ceil per job) │ 658 min │ 617 min │ −41 min (−6.2%) │
└─────────────────────────┴────────────────┴───────────────────────┴───────────────────┘
The shared-browser experiment is cheaper, and the savings land almost exactly where you'd expect — the browser-driven test suites — while non-browser jobs (rust, lint, unit, windows) are flat within noise:
┌─────────────────────────────────┬─────────┬────────┬───────┐
│ Job category │ control │ shared │ delta │
├─────────────────────────────────┼─────────┼────────┼───────┤
│ test prod │ 130.7 │ 110.5 │ −20.2 │
├─────────────────────────────────┼─────────┼────────┼───────┤
│ cache components dev │ 51.5 │ 42.9 │ −8.7 │
├─────────────────────────────────┼─────────┼────────┼───────┤
│ turbopack dev │ 75.1 │ 69.5 │ −5.5 │
├─────────────────────────────────┼─────────┼────────┼───────┤
│ turbopack production │ 86.3 │ 81.1 │ −5.2 │
├─────────────────────────────────┼─────────┼────────┼───────┤
│ test dev │ 110.4 │ 105.5 │ −4.9 │
├─────────────────────────────────┼─────────┼────────┼───────┤
│ cache components prod │ 48.7 │ 49.0 │ +0.3 │
├─────────────────────────────────┼─────────┼────────┼───────┤
│ firefox and safari │ 5.3 │ 7.3 │ +2.0 │
├─────────────────────────────────┼─────────┼────────┼───────┤
│ flake-detection jobs (combined) │ ~15 │ ~21 │ +~6 │
└─────────────────────────────────┴─────────┴────────┴───────┘
Takeaways:
- Net savings of ~41 billable minutes per run (~6%), concentrated in the Chromium-driven test prod/dev, turbopack, and cache components suites — consistent with reusing one browser process instead of spawning per-suite.
- The small regressions are in firefox and safari and the "new/changed tests for flakes" jobs (+~8 min combined). Worth a glance, though they may just be run-to-run variance.
Caveat: this is a single run per PR, so there's real variance run-to-run. That said, the fact that the deltas track the browser-heavy jobs specifically — and not the Rust/lint/unit jobs — is a good signal the effect is genuine rather than noise. If you want a firmer number, re-running each PR 2–3× and averaging would tighten it up.
```
Noticed while reading through the symlink code.
The symlink logic had some clever `ErrorKind::AlreadyExists` logic that
folded the retry into the main retry loop, but we still had a separate
retry loop for after handling `ErrorKind::NotFound`. This folds the
loops together.
I hand-implemented the symlink version of this and had opus port it to
the non-symlink version.
## Summary
Documents the policy around AI-assisted contributions, for both humans
and agents:
- `.github/pull_request_template.md`: notes that AI use is encouraged
for researching, creating, and reviewing changes, but contributors must
deeply understand their contributions, and PR descriptions from external
contributors must be written by a human.
- `.github/ISSUE_TEMPLATE/1.bug_report.yml`: adds the equivalent note
for bug reports — issue descriptions from external contributors must be
written by a human, though AI may help create reproductions.
- `AGENTS.md`: replaces the old "PR Descriptions" section with two new
sections. "GitHub Pull Requests" distinguishes branch PRs (agents may
write descriptions) from fork PRs targeting `vercel/next.js` (agents may
not). "GitHub Issues, Comments, and Discussions" restricts agent-written
issues, discussions, and comments to members of the
`vercel`/`vercel-labs` GitHub orgs, tells agents how to check membership
via the GitHub API, lists what agents can still help non-members with
(drafting details, reviewing, reproductions, translation, searching for
duplicates), and carves out exceptions (commenting on the user's own PR,
Vercel-operated bots, GitHub/Graphite review bots, and forks of the
repo).
- Renames the LLM watermark marker from `NEXT_JS_LLM_PR` to
`NEXT_JS_LLM` everywhere it appears (`AGENTS.md`,
`contributing/repository/pull-request-descriptions.md`, and the
`create-pr`/`backport-pr` skills), since it now also applies to issues,
discussions, and comments. Nothing in the repo consumes the old marker
name.
- `.agents/skills/create-pr/SKILL.md`: adds a "Fork PRs vs Branch PRs"
section referencing the `AGENTS.md` policy. Assorted typo fixes along
the way.
## Verification
- `gh api /user/memberships/orgs` and `gh api
orgs/vercel/members/<login>` (confirmed the documented membership checks
work and return the documented statuses)
- Not run: no test/build commands (docs-only change; prettier applied
via the pre-commit hook)
<!-- NEXT_JS_LLM -->
See https://github.com/vercel/next.js/pull/95553 for an explanation of
why we want to split these up. TL;DR it makes CI faster and saves us
some money.
In CI we have to run tests at the suite/file granularity, so if a test
suite is too long it limits parallelism and if it has to be retried, we
have to retry the full suite.
See https://github.com/vercel/next.js/pull/95553 for an explanation of
why we want to split these up. TL;DR it makes CI faster and saves us
some money.
In CI we have to run tests at the suite/file granularity, so if a test
suite is too long it limits parallelism and if it has to be retried, we
have to retry the full suite.
See https://github.com/vercel/next.js/pull/95553 for an explanation of
why we want to split these up. TL;DR it makes CI faster and saves us
some money.
In CI we have to run tests at the suite/file granularity, so if a test
suite is too long it limits parallelism and if it has to be retried, we
have to retry the full suite.
Same motivation as #95553, but another test suite.
This is a bit messier than the other PR because there's not as clear of
a split point, so it uses sharding.
This splits two of our slowest test suites into smaller pieces that can
be run in parallel and retried in isolation.
## Context
Jest treats test suites as the smallest test execution unit.
Slow test suites can lead to situations where a test shard is using very
little CPU waiting for one last test suite to finish, and when tests
fail and have to be retried, we have to retry the entire suite.
Here's a particularly bad example:
<img width="3116" height="2634" alt="Screenshot 2026-06-08 at 20-00-05
CI Telemetry — turbopack production tests (1)"
src="https://github.com/user-attachments/assets/b5534200-4b77-40d5-a219-4b6fc552ed77"
/>
I instructed Claude to fetch test timing information from GitHub Actions
Artifact, and to report the slowest test suites.
> How timings work today
>
> - The `fetch-test-timings` job runs `node run-tests.js --timings
--write-timings`, which pulls per-file durations from a Vercel KV store
and uploads them as the test-timings artifact (test-timings.json, 5-day
retention).
> - Each sharded job downloads that artifact and run-tests.js -g N/M
greedily bin-packs whole test files into groups by predicted duration
(run-tests.js:528-549).
> - I pulled the artifact from the July 7 canary run (28843797348): 1684
test files, ~61,500s total, median 20s, but max 1023s.
>
> Since the file is the unit of scheduling, a shard can never finish
faster than its largest file. The job durations from that run confirm
the imbalance this causes: test dev 4/10 took 28 min while siblings took
10–17; test cache components dev 6/6 took 13 min vs 5–6 for siblings;
test turbopack production 5/7 took 20 min vs 10–11.
> The slowest suites
```
┌────────────────────┬────────────────────────────────────────────────────────────────────────────┐
│ Time │ Suite │
├────────────────────┼────────────────────────────────────────────────────────────────────────────┤
│ 1023s + 933s │ test/development/app-dir/cache-components-dev-warmup/ (both variants) │
├────────────────────┼────────────────────────────────────────────────────────────────────────────┤
│ 942s │ test/production/create-next-app/templates/matrix.test.ts │
├────────────────────┼────────────────────────────────────────────────────────────────────────────┤
│ 788s + 432s + 232s │ test/e2e/app-dir/cache-components-errors/cache-components-errors.*.test.ts │
├────────────────────┼────────────────────────────────────────────────────────────────────────────┤
│ 779s │ test/development/acceptance-app/rsc-build-errors.test.ts │
├────────────────────┼────────────────────────────────────────────────────────────────────────────┤
│ 470s │ test/e2e/opentelemetry/instrumentation/opentelemetry.test.ts │
├────────────────────┼────────────────────────────────────────────────────────────────────────────┤
│ 457s │ test/e2e/app-dir/actions/app-action-node-middleware.test.ts │
├────────────────────┼────────────────────────────────────────────────────────────────────────────┤
│ 451s │ test/e2e/app-dir/next-after-app/index.test.ts │
├────────────────────┼────────────────────────────────────────────────────────────────────────────┤
│ 447s │ test/e2e/app-dir/css-order/css-order.test.ts │
├────────────────────┼────────────────────────────────────────────────────────────────────────────┤
│ 426s │ test/e2e/telemetry/config.test.ts │
├────────────────────┼────────────────────────────────────────────────────────────────────────────┤
│ 402s │ test/production/debug-build-path/debug-build-paths.test.ts │
└────────────────────┴────────────────────────────────────────────────────────────────────────────┘
```
> Reading these files, almost every one is slow for the same structural
reason: a `describe.each(...)` over fixtures/modes/runtimes where each
iteration boots its own `nextTestSetup` (dev server or full production
build), run serially within one Jest worker:
> - **cache-components-dev-warmup:** `describe.each` over 2 fixture
dirs, and it restarts the dev server (stop/clean/start) before every one
of ~17 tests. It's slow by design (cache-warmup isolation), so per-test
cost can't shrink — but the file can be split.
> - **create-next-app matrix:** `describe.each(app|pages)` × `it.each`
over a ~16–48-combination flag matrix, each doing a full CNA scaffold (+
dev-server cycle for pages), fully serial.
> - cache-components-errors.test.ts: 7490 lines, 129 tests, with
per-describe `next build --experimental-build-mode compile` and per-path
generate builds in prod mode.
> - **css-order:** three sibling `describe.each` blocks over 4–6
chunking modes, each mode booting its own server, ~3 blocks × modes ×
page-pair orderings.
> - **next-after-app, opentelemetry:** `describe.each` over
runtimes/configs, again one server per iteration.
The hypothesis is that this is just installing some font stuff that
isn't actually needed for Chromium to work:
```
The following additional packages will be installed:
libnss3-tools xfonts-encodings xfonts-utils
Recommended packages:
fonts-ipafont-mincho fonts-liberation-sans-narrow fonts-tlwg-loma
The following NEW packages will be installed:
fonts-freefont-ttf fonts-ipafont-gothic fonts-liberation fonts-tlwg-loma-otf
fonts-unifont fonts-wqy-zenhei xfonts-cyrillic xfonts-encodings
xfonts-scalable xfonts-utils
The following packages will be upgraded:
libnss3 libnss3-tools
2 upgraded, 10 newly installed, 0 to remove and 35 not upgraded.
```
`apt` is very slow, so skipping this step should significantly reduce
setup time per test shard.
It seems like this isn't always true for other browsers, so only skip
this for Chromium (that's fine, that's what most of our jobs use
anyways).
On 16 core arm runners, "Install Playwright Browsers" typically took
18-24 seconds. Now it takes 5-6s.
Let's estimate that saves about 15s per Linux job that installs
Chromium. Claude estimates that there are 73 chromium-only Linux jobs on
this PR:
<img width="891" height="148" alt="Screenshot 2026-07-06 at 10 30 16 PM"
src="https://github.com/user-attachments/assets/725893fa-9f6f-4806-a3b4-9858ab0ebe66"
/>
So that's 15*73/60=18.25 minutes saved per PR push (the average PR gets
many pushes). At the public (before any volume discounts) price of
$0.026/minute for the 16 core arm runners, that comes out to $0.4745 per
PR push saved.
This workflow has had 3,894 runs in the last 30 days, so that could be
$1,847/mo saved, but many of those workflow runs were canceled, and some
likely failed before getting to these jobs, so the actual savings are
probably somewhere around half of that -- scroll through
https://github.com/vercel/next.js/actions/workflows/.github/workflows/build_and_test.yml
and see that a little less than half of the workflow runs end up
canceled.
This is embarrassing, I wasn't paying enough attention when I put up
#95374. We have to both change the build dependency *and* switch the
machine type...
A 16 core x86-64 runner is $0.042/minute. A 16 core arm64 runner is
$0.026/minute.
https://docs.github.com/en/billing/reference/actions-runner-pricing
It also looks like they're faster! After factoring the performance
improvement, this appears to be a ~50% cost reduction for migrated jobs.
Datadog SQL:
```
SELECT
REGEXP_REPLACE(ci_job_name, '\s*\([0-9]+/[0-9]+\)', '') AS ci_job_name,
labels,
branch,
commit_date,
SUM(duration) / 1000 / 1000 / 1000 / 60 as minutes,
ci_pipeline_id
FROM dd.ci_pipelines(
columns => ARRAY ['@git.commit.committer.date', '@ci.pipeline.id', '@duration', '@git.branch', '@ci.node.labels', '@ci.job.name'],
event_type => 'job',
filter => '@git.repository.id_v2:github.com/vercel/next.js @ci.pipeline.name:build-and-test @ci.provider.instance:github-actions @ci.pipeline.id:(28266192824-1 OR 28266190122-2) [email protected]:*changed*',
from_timestamp => now() - INTERVAL '30 days',
to_timestamp => now()
) AS (
commit_date VARCHAR,
ci_pipeline_id VARCHAR,
duration BIGINT,
branch VARCHAR,
labels VARCHAR,
ci_job_name VARCHAR
)
GROUP BY
REGEXP_REPLACE(ci_job_name, '\s*\([0-9]+/[0-9]+\)', ''),
labels,
commit_date,
branch,
ci_pipeline_id
ORDER BY
REGEXP_REPLACE(ci_job_name, '\s*\([0-9]+/[0-9]+\)', ''),
branch,
commit_date,
minutes;
```
https://app.datadoghq.com/ddsql/editor?ddsql_dataset_id=34de66b9-30f4-48dd-ac98-354dc7e67fdf
Full Data:
https://docs.google.com/spreadsheets/d/1SszR3RsPeJ13nVW5ggMakwy87fkQMqWMHbZDehHH8IA/edit?usp=sharing
Here's the abbreviated data for jobs longer than 5 minutes:
<img width="1187" height="656" alt="Screenshot 2026-06-26 at 4 38 00 PM"
src="https://github.com/user-attachments/assets/9341039e-c4ea-405c-8d24-cdbc5ed1ba24"
/>
It seems like for the jobs switched over to arm64, they run in about
~73-82% of the time as arm.
Minor nit: In the worst-case (module id can be an `RcStr` or a u64),
this eliminates two `RcStr` clones+drops per iteration of the sort
algorithm.
The code is cleaner with `sort_by_key`, but `sort_by_key` introduces an
artificial lifetime issue that `sort_by` avoids.
This is a long-standing and widely documented language issue:
https://github.com/rust-lang/rust/issues/34162
Because it's not sure which one you'll want, playwright installs TWO
copies of Chromium by default. A full copy and a stripped down headless
version.
Most of the install time is spent in `apt-get`/`dpkg`, but this at least
buys us a few seconds back.
Before:
After:
<img width="1700" height="552" alt="Screenshot 2026-06-16 at 6 24 50 PM"
src="https://github.com/user-attachments/assets/d382cb8e-a875-4d67-92d6-32ff51c06488"
/>
Playwright also installs ffmpeg in case you want to perform a screen
recording (we don't), but it's much smaller (2.3 MiB), and there doesn't
appear to be a way to disable it.
I noticed the `node-version-file` option for `setup-node` while
reviewing https://github.com/vercel/nextjs-react-compiler/pull/2
We should use it in places where we're already checking out the
repository anyways. It should make it easier to keep node versions in
sync.
I determined that [`nvm` wasn't sufficient for our use-case since we
need to support
windows](https://github.com/vercel/next.js/pull/94175#issuecomment-4654622674),
so this doubles down on `fnm`.
This reduces risks of supply chain attacks with `fnm`.
This also cleans up some old stuff that was left over from self-hosted
runners and/or that was there for debugging, if it isn't useful anymore.
We're consolidating on `actions/cache` now that we got rid of
self-hosted runners. This doesn't remove all of the uses of turborepo's
cache (notably sccache), but this does eliminate the remote turborepo
cache for the native build cache and the docker image.
These build steps use docker so that we're building against an old
pinned glibc (2.31), and so that we can have access to both `aarch64`
and `x86_64` musl sysroots.
## Testing
**Local Build:** `node scripts/docker-native-build.js` still works (this
script just exists for being able to debug these builds locally)
**CI Build:**
- Run an automated-preview CI job, but cancel it before the native build
succeeds to test writing to the docker cache but not the native cache:
https://github.com/vercel/next.js/actions/runs/26986955366/job/79638633136?pr=94319
- Re-run the job with only the docker image cache, see that we get a
cache hit for the docker image:
https://github.com/vercel/next.js/actions/runs/26986955366/job/79639365126?pr=94319
- Re-run the job again and see that we hit the native build cache hit
and don't touch docker at all:
https://github.com/vercel/next.js/actions/runs/26986955366/job/79647324952?pr=94319
- Manually trigger the job to do a full preview build:
https://github.com/vercel/next.js/actions/runs/26989861549
We reverted the use of this docker image in
https://github.com/vercel/next.js/pull/94366
We're going to use GitHub actions custom runner images instead (which
don't use docker)
The actual security implications of this are likely *very* minor, but in
the interest of reducing how many credentials we have, there's little
reason for us not to set this to `false`.
In cases where we create a new commit via GH actions, we already do that
via the GitHub API, so those don't need the on-disk git credentials to
push.
---
From: https://github.com/actions/checkout :
> The auth token is persisted in the local git config. This enables your
scripts to run authenticated git commands. The token is removed during
post-job cleanup. Set persist-credentials: false to opt-out.
```
# Whether to configure the token or SSH key with the local git config
# Default: true
persist-credentials: ''
```
Matches the environment name I asked IT to use for NPM trusted
publishing:
https://vercel.slack.com/archives/C01LN7C5QR5/p1780600987032909
Nothing else uses the `npm` environment, but the name is confusing. We
want to use separate environments for these publishing workflows as they
allow us to define things like reviewers, if we want to block the
release on a manual review.
We're going to go with custom images instead, though we don't have an
implementation for this yet.
This does save a meaningful amount of minutes, but it also significantly
increases complexity.
This is a clean revert, minus the fix that the PR included for listening
on localhost in `packages/next/src/build/analyze/index.ts`, which is
still good.
The filtering logic is generally very cheap, many issues are
short-lived, and filtering is unlikely to trigger more than 1-2 turbo
tasks per issue.
Wrapping it in another task per issue doesn't make a ton of sense.
Previously we pinned all of these to their current versions at the time.
That meant that some of these got pinned to really old versions. This
updates all of the "first-party" `actions/*` dependencies.
Prompted claude with:
```
For all of these uses of actions/*, make sure we're pulling in the latest stable version (check the gh cli). When upgrading across major versions, fetch the release notes from the GH releases using the gh CLI and make sure that there are no problems that the upgrade will cause.
```
Claude read all the release notes and seems to think these upgrades are
safe.
Turbopack is the default bundler, so we should prioritize testing that
in these variants of the CI runs (cache components, node streams,
windows).
There's also some limited evidence that indicates that it might be
faster for e2e test runs, though that's not conclusive (because of
different shard sizes):
https://vercel.slack.com/archives/C0B0LKK5QD7/p1779320615501259
If this action detects that you're likely using `npm` it (by default)
enables a cache for `npm`. We use `pnpm`, so it doesn't do anything (and
we manually set up a pnpm cache in some places), but the implicit
behavior here is a little dangerous: e.g.
https://adnanthekhan.com/posts/angular-compromise-through-dev-infra/#finding-the-pivot
So let's explicitly disable this everywhere. It wasn't doing anything
for us anyways.
Claude identified these as clear candidates for shrinking machine size.
Here are the runner types we have that are attributed to the OSS cost
center:
Ideally we'd have 1 and 2 core machines as well, but I don't really want
to bother IT right now.
By default, turbo-tasks will try to use every available CPU core.
While we do want it to use as many resources as possible if the CPU is
otherwise idle, this may hurt us when the CPU is busy, as it will lead
to more locking, more thread contention, more memory usage, etc.
**This environment variable does not strictly limit Turbopack to using 4
cores, it just restricts the number of main tokio worker threads and
changes how we shard dashmaps.**
We currently are using 16 core CI machines, but I want to experiment
with 32 core build machines with a higher `TEST_CONCURRENCY`, and the
per-core scaling of Turbopack was a concern that came up, as wall time
improvements are sub-linear starting somewhere around 16 cores.
## Test Plan
Watch CI, compare build times in this commit and the parent one.
Attempt with a value of 2: https://github.com/vercel/next.js/pull/94001
Attempt with a value of 8: https://github.com/vercel/next.js/pull/94002
I think this data is too noisy to draw any conclusions.
We want to use this to potentially improve efficiency in e2e tests where
we're running many instances of Turbopack at once. By default,
turbo-tasks will try to use every available CPU core.
**This environment variable does not strictly limit Turbopack to using 4
cores, it just restricts the number of main tokio worker threads and
changes how we shard dashmaps.**
This option will not make sense for the vast majority of users, it only
makes sense when you are running *many* instances of Turbopack at once,
and the exact semantics are potentially confusing, as it is not a hard
limit.
## Test Plan
Build vercel-site with this set to 1 and see a little over 100% CPU
usage. Set it to 8 and see around 800% CPU usage.
```
TURBO_TASKS_AVAILABLE_PARALLELISM=1 pnpm next build --experimental-build-mode=compile
```
This is a mostly-by-hand refactor of some of the caching logic in
https://github.com/vercel/next.js/pull/92300
- **Cache errors.** We often end up applying effects multiple times in
frequent succession, and many filesystem effects already have their own
backoff/retry loops, which can make them very slow when they fail, so it
seems generally better to cache the failures.
- **Add `EffectLastApplied` enum:** It's a matter of opinion, but I
think the `EffectLastApplied` enum makes it easier to reason about the
different states in the system. `Option` was difficult to reason about,
and it would've been even more confiusing if it was an
`Option<Result<(), Arc<dyn EffectError>>`.
- `tokio::sync::Mutex<()>` is overkill here, `tokio::sync::Notify` is a
simpler (and smaller) primitive that does the core of what we needed the
`Mutex` for. We could also use `EventListener` here, but I didn't.
(`EventListener` is really just smol's version of `Notify`)
- We can lazily allocate the `Notify` instance inside of `last_applied`.
We have to grab this lock anyways to check for the cached fast-path. If
we're not in the fast-path, we can re-use the lock guard to create and
store the `Notify` instance.
- **Perf:** The old code's slow path had to always lock `last_applied`
three times. Now, if there's no concurrent writer (common case), it only
locks twice: Once before application, and once after.
- **Perf:** Use a smallvec when computing unique indices to avoid a
bunch of heap allocations. Unless there's a bug, we don't expect any of
the keys to have more than one value (returns an error).
- **Perf:** Store keys as `Box<[u8]>` instead of `Vec<u8>`, saving 8
bytes per key.
Currently, if `effect.dyn_apply()` panics, we just do our best to mark
the effect as unapplied to avoid a deadlock. This could alternatively
poison the object.
## Memory Usage
Memory usage can increase if we're storing errors (heap allocated,
stored in `Arc`) we previously wouldn't have, but if we assume that
errors are uncommon, this refactor decreases the size of the data
structures involved.
Before this change, eagerly allocating a `tokio::sync::Mutex` was
costing 40 bytes per effect, making the entire `EffectEntryState` struct
64 bytes:
<img
src="https://app.graphite.com/user-attachments/assets/989ac229-9e2f-4370-8e32-b14e10e09248.png"
width=500>
<img
src="https://app.graphite.com/user-attachments/assets/f2e14416-17c3-4961-8c00-fa7a81846fa0.png"
width=500>
After the changes, the entire `EffectLastApplied` enum is 40 bytes
<img
src="https://app.graphite.com/user-attachments/assets/18e4fc0a-e7f8-4fad-b9bc-c22ba3eefb6c.png"
width=500>
After wrapping the `EffectLastApplied` enum in a `parking_lot::Mutex`,
it comes in at 48 bytes (after alignment), 16 bytes smaller than the old
struct.
We know that we have a lot of lock contention when creating watchers, because many watchers are often set up at roughly the same time near the start of building a route. Internal discussion thread: https://vercel.slack.com/archives/C09R44U5HQW/p1776271283799709
@lukesandberg's suggestion was that the lock is held over syscalls and file IO, so it's possible that this could starve the Tokio worker threads, so we should use an async lock so that we can yield back the worker thread.
Because this would simply unblock *other* work, and because this codepath is only used in `dev`, there's no sane way to benchmark or measure this, so we'll just have to hope and pray it's an improvement.
This PR is entirely AI-generated.
`lld` is much faster than the default gnu `ld` linker on most Linux
systems.
Timeline:
- https://github.com/vercel/next.js/pull/91477 removed these flags as
part of cleanup because they're now default on x86-64 glibc linux.
However, they are not default on arm64.
- https://github.com/vercel/next.js/pull/91799 removed these flags from
musl, and lost the comment explaining reason for the `gnu-lld-cc`
option.
Without this change, doing a dev build of a trivial change to
`crates/next-napi-bindings/src/lib.rs` can take over a minute.
I tested this by looking at htop near the end of a build and observing
`lld` instead of `ld`.
In reality, I expect this to compile to the same code, but using `from_raw_parts` is technically more correct here as there are no guarantees about the memory layout of `Vec<T>` or `[T]` for any two different values of `T`, even if both values of `T` have the same size and alignment.
`pnpm dlx` isn't using a lockfile. Using the self-contained binary from
GitHub and validate the checksum lets us effectively pin `datadog-ci`
and all of its possible transitive dependencies.
This is a follow-up for
https://vercel.slack.com/archives/C0APPN2LC83/p1775073127894859
This job has been failing on canary pushes, but not on PRs. I'm not sure why it's only happening in canary pushes, but hopefully this should fix the specific error we're seeing when it does fail.
Discussion: https://vercel.slack.com/archives/C04KC8A53T7/p1775571376147179
Previously, we were just copying the `package.json` into the container and `pnpm install`ing it.
This wouldn't respect the lockfile or `minimumReleaseAge`. This uses `pnpm deploy` based on the documentation here: https://pnpm.io/docker
This should fix the stats action: We were checking out Next.js 16.2.2, and that doesn't support the `--out` flag for `pnpm pack`. Let's just force pnpm 10.33.0 here for now.
While we're here, I made some other updates and improvements:
- Update dockerfile to use Ubuntu 24.04
- Update dockerfile to use corepack 0.34.6
- Fix an issue where `pnpm install`ing outside of the docker container could break the docker container build (use `--exclude=node_modules`)
- Avoid using a shell with string interpolation (potential escaping issues) for `cd`ing, just set `cwd`.
Having both `apply_effects` and `Effects::apply` is confusing.
`apply_effect`'s API seems a bit bad/dangerous because it combines work that should be done inside of a strongly-consistent operation (collecting the effects and reading their `Vc`s) with work that should be done at the top-level outside of any operation or turbo-task function (applying the effects).
The first commit in this PR removes the `apply_effect` API and tries to improve documentation, including adding a doctest that compiles.
The second commit in this PR is an LLM-driven refactor to remove the `apply_effect` callsites.
PR is AI-generated
- `extension` and `extension_ref` do the same thing.
- `extension_ref`'s API that returns an `Option` was better than
`extension`'s API that returns a potentially-empty string.
- In some places we were splitting into a stem and extension, and
joining them back to a filename, but those places could just get the
original path's filename.
---------
Co-authored-by: Tobias Koppers <[email protected]>
Co-authored-by: Claude <[email protected]>
Co-authored-by: Benjamin Woodruff <[email protected]>
`FileSystemPath::value_to_string` was just calling `ValueToString::to_string`.
But:
- It's better to call `ValueToStringRef::to_string_ref`
- We don't really get much value from this inherent method except for avoiding a few imports?
I suspect this is left over from when we used to always wrap `FileSystemPath` in `Vc`.
This is inspired by
https://glama.ai/blog/2026-03-19-open-source-has-a-bot-problem#user-content-how-are-these-agents-setup
If agents follow this pattern, we could potentially add labels (this PR
does not touch the labeler) to these PRs which may help with review and
triage. This does not change how we'd deal with these PRs (AI generated
PRs are okay!), but I would like to know if a PR is AI generated when
reviewing it.
The field holds a Module, not a generic asset. This aligns the naming
with the type it actually stores.
---------
Co-authored-by: Tobias Koppers <[email protected]>
Co-authored-by: Claude <[email protected]>
Co-authored-by: Benjamin Woodruff <[email protected]>
In subsequent PRs, I want to store a `ResolvedVc` (specifically, a `FileSystemPath`, which contains a `Vc<Box<dyn FileSystem>>`) inside of an `Effect`.
To do this in a way that wouldn't break a hypothetical future tracing garbage collector, we must implement `TraceRawVcs` on `Effect`.
The previous PR changed the `Effect` type from a closure to a trait in order to enable this.
This switches effects (special `Collectible`s used to defer file writes) from using a closure representation to using a trait representation.
This makes constructing effects a little more annoying, but it ensures that we'll be able to correctly implement `TraceRawVcs` on these in a subsequent PR.
Changes to `turbo-tasks/src/effect.rs` were done by hand, an LLM was used for updating the two callsites.
This defaults to `true`: https://doc.rust-lang.org/cargo/reference/profiles.html#debug
Nobody on the team uses a debugger often enough for this to be worth the time tradeoff: https://kobzol.github.io/rust/rustc/2025/05/20/disable-debuginfo-to-improve-rust-compile-times.html
I'm gathering some rough numbers, and I'll add them to the PR.
After `cargo clean` and one `cargo build -p next-napi-bindings` with each profile (Linux arm64):
```
du -sh target/*
4.0K target/CACHEDIR.TAG
7.9G target/debug
14G target/dev-with-debug
7.5M target/generated
```
This is nearly half the size.
Due to cargo quirks, the `dev` profile defaults to the `target/debug` directory. Of course, any disk size wins you get are more than offset if you need to use a debugger and end up with an additional profile.
```
hyperfine -w 1 -r 5 -p 'cargo clean' 'cargo build --profile dev-with-debug -p next-napi-bindings' 'cargo build -p next-napi-bindings'
```
...pending...
(this is on https://github.com/bgw/benchmark-scripts/, on my M2 Macbook inside of a Linux VM this takes closer to 2m20s)
- More profiles means more bloat to the `target` directory and more cold builds.
- There's no practical use case where we'd be okay with the performance hit of `debug-assertions` and not okay with the performance hit of `lto = false`. Neither profile was suitable for benchmarking.
- `release-with-assertions-no-lto` is too long to type and was giving me carpal tunnel.
Copied from https://turbopack-rust-docs.vercel.sh/turbopack/chunking.html
I had claude review it for correctness against the code. It found a few things where the names weren't correct or a trait was actually a struct, and it grouped things into sections. It also added some more details to a few sections, like the dev vs production chunking. I think this is a reasonable improvement.
I did a second pass with claude where I had it remove things it 1:1 duplicated with the existing rustdocs, and I had it add intradoc links.
Rendered:
This gets `cargo doc` clean, and enables it in CI so that we won't regress it (I'm open to removing this if it proves to be annoying).
I also cleaned up some documentation and even found some dead code (deleted `turbopack/crates/turbopack-dev-server/src/source/conditional.rs`).
@wbinnssmith added these scripts a long time ago at my suggestion/urging, but I think they ended up bad for a couple reasons that were hard to foresee at the time:
- afsctool operates in-place and doesn't atomically write the compressed file, so if the process gets interrupted, your `target` directory is corrupted. https://github.com/Dr-Emann/applesauce is better for this reason.
- This doesn't acquire the cargo lock, and modifying files while cargo is running is a good way to get corruption of the `target` directory.
https://github.com/bgw/cargo-apfs-compress is my latest attempt at solving this, though I don't have a `LaunchAgents` config for it.
Regarding `node_modules`: pnpm creates reflinks from a shared global store on apfs. Trying to compress these reflinked files is just going to hurt you because it'll break the data deduplication that would've otherwise happened.
This is basically one-shot from Opus 4.6, with a minor tweak to the doc comment by hand.
Discussed this with @lukesandberg, and we realized that there's only one implementation of this trait, so it doesn't make a ton of sense as a trait.
