As of #77894, there's a new way to indicate Turbopack should run, which breaks flaky tests detection with Turbopack. This swaps out the flag to unblock my downstack PR.
This section will be for developers who want to understand how things
work. It's not essential knowledge, but good to know.
A place for conceptual information and colorful diagrams that explain
the magic behind things โจ๐
## Convert unpack-next.cjs to TypeScript
### What?
Converted `scripts/unpack-next.cjs` to TypeScript, renaming it to
`scripts/unpack-next.ts` and updating the imports and type definitions.
### Why?
This change continues the ongoing TypeScript migration efforts in the
codebase, providing better type safety and developer experience.
### How?
- Renamed file from `.cjs` to `.ts`
- Converted CommonJS `require()` statements to ES module `import`
statements
- Added type annotation for the `path` parameter in the `realPathIfAny`
function
Fixes #
Yargs automatically creates `--no-` variations of options. Using
`--no-js-build` as an option directly confuses it.
`--no-js-build` remains a valid cli argument.
Test Plan: Verify `pnpm pack-next` builds JS by default and `pnpm
pack-next --no-js-build` doesnโt.
Help now looks like this:
```
pack-next.ts
Options:
--help Show help [boolean]
--version Show version number [boolean]
--js-build Build JavaScript code (default). Use `--no-js-build` to skip
building JavaScript [boolean] [default: true]
-p, --project [string]
--tar Create tarballs instead of direct reflinks [boolean]
--compress How compress the binary, useful on platforms where tarballs
can exceed 2 GiB, which causes ERR_FS_FILE_TOO_LARGE with
pnpm. Defaults to "strip" on Linux, otherwise defaults to
"none". Requires `--tar` to be set.
[string] [choices: "none", "strip"]
```
Written with @bgw.
- Transitions to using `tsx` for script execution
- Replaces `patch-package` with TypeScript implementation
- Implements argument parsing and help message with `yargs`
- Uses `execa` for some command execution
Test Plan: Run `pnpm pack-next`, `pnpm swc-build-wasm`, `pnpm unpack-next`, and `pnpm sweep` without errors.
Previously, we marked the server startup whenever a local url was shown
in stdio. Next.js shows this url as soon as it can to make it actionable
sooner, before the server is truly ready.
This changes makes the event record the time until the โReady in Nmsโ
message is shown instead.
Test Plan: `./node_modules/.bin/devlow-bench ./scripts/devlow-bench.mjs`
and validate server startup is recorded.
On Linux we must either compress or strip debuginfo, otherwise the debug tarball will be >2GiB, and pnpm (it's actually a node/libuv bug) has issues with files larger than 2GiB.
As far as I can tell Rust can generate line numbers for backtraces with zlib-compressed debuginfo, but not with newer zstd-compressed debuginfo.
## Summary
Follow up #73548 and #73633.
#73548 changed the `with-next-sass` to `with-sass` directory.
However, some tests still use old `with-next-sass` name.
This PR updates them.
Co-authored-by: Sam Ko <[email protected]>
We run flake detection on PRs to help catch flaky tests before they hit
canary, but we currently only run flaky test detection in Webpack and
not Turbopack. This is problematic because there's sufficient forking
logic in tests for the different bundlers & we've seen a high likelihood
of a test to flake more frequently in one bundler vs another due to
things like differing compilation speeds and differing implementation of
core features.
This will add some additional time to this CI job to account for needing
to run the tests 6 times, but will hopefully catch things before they
become a problem later. I think we could also consider reducing the
number of times per bundler to be 2 if it turns out to be adding too
much time.
This updates the flake detection job to run the tests multiple times in
Turbopack as well.
CI run:
https://github.com/vercel/next.js/actions/runs/11822662639/job/32940264160?pr=72773
### What?
Added a step in both `build-native.cjs` and `build-wasm.cjs` scripts to
run Prettier on the generated types file after writing it.
Should have been added with #69680, but I didn't think about it.
Currently, every time you built the native module you have changes in
the git repo (which would be fixed on commit, but graphite might not be
happy with uncommitted changes).
### What?
A couple of small fixes for the scripts ported from nextpack.
- use ignore globs correctly
- get path for patch after removing other arguments
- display cargo colors
## What is `pack-next`?
This is a script we use for developing turbopack. It builds tarballs for
all the next.js packages, including `next-swc`, and then provides
instructions about how to modify a project's `package.json` to install
from that tarball.
## What does this PR do?
If you provide a `--project ~/path/to/directory` argument to `pnpm
pack-next`, we'll auto-update the `package.json` there for you, instead
of telling you how to do it manually!
This also understands the React 19 RC peer dependencies, which is
something that was missing from the generated instructions.
## Supported Package Managers
I've tested this with `npm` and `pnpm`.
- `bun` fails with `ENOTDIR` for unknown reasons (maybe it's trying to
handle the local tarball as a directory instead of a tarball?).
- `yarn` fails due to an upstream issue:
https://github.com/yarnpkg/yarn/issues/6339
This does generate the entries for yarn's `resolutions` feature,
assuming the upstream issue is eventually fixed.
## Why Rust and `cargo xtask`?
https://github.com/matklad/cargo-xtask
I'd love for these Turbopack-centric scripts to be written in Rust with
xtask, since they're mostly touched by the turbopack team.
However, it's also hard to justify putting too much effort into these
development scripts, so I don't want to rewrite the whole thing right
now.
I figured I'd write the new functionality in Rust and call it from the
old JS script.
## Test Plan
Perform a fresh checkout of shadcn/ui, and then:
```
pnpm pack-next --project ~/shadcn-ui/apps/www/
```
```
Found workspace! Patching "/home/bgw.linux/shadcn-ui/package.json" with overrides.
Successfully patched "/home/bgw.linux/shadcn-ui/package.json" to use local tarball files!
Run `pnpm i` or `npm i` to install the overrides
```
Check the resulting changes to `~/shadcn-ui`:
```diff
diff --git a/package.json b/package.json
index be47ad7..262c13b 100644
--- a/package.json
+++ b/package.json
@@ -63,12 +63,31 @@
"tsx": "^4.1.4",
"turbo": "^1.9.9",
"vite-tsconfig-paths": "^4.2.0",
- "vitest": "^0.31.1"
+ "vitest": "^0.31.1",
+ "@next/swc": "file:/home/bgw.linux/next.js/tarballs/next-swc.tar"
},
"devDependencies": {
"@types/node": "^20.11.27",
"@types/react": "^18.2.65",
"@types/react-dom": "^18.2.22",
"typescript": "^5.5.3"
+ },
+ "overrides": {
+ "next": "file:/home/bgw.linux/next.js/tarballs/next.tar",
+ "@next/mdx": "file:/home/bgw.linux/next.js/tarballs/next-mdx.tar",
+ "@next/env": "file:/home/bgw.linux/next.js/tarballs/next-env.tar",
+ "@next/bundle-analyzer": "file:/home/bgw.linux/next.js/tarballs/next-bundle-analyzer.tar",
+ "@next/swc": "file:/home/bgw.linux/next.js/tarballs/next-swc.tar",
+ "react": "19.0.0-rc-7771d3a7-20240827",
+ "react-dom": "19.0.0-rc-7771d3a7-20240827"
+ },
+ "resolutions": {
+ "next": "file:/home/bgw.linux/next.js/tarballs/next.tar",
+ "@next/mdx": "file:/home/bgw.linux/next.js/tarballs/next-mdx.tar",
+ "@next/env": "file:/home/bgw.linux/next.js/tarballs/next-env.tar",
+ "@next/bundle-analyzer": "file:/home/bgw.linux/next.js/tarballs/next-bundle-analyzer.tar",
+ "@next/swc": "file:/home/bgw.linux/next.js/tarballs/next-swc.tar",
+ "react": "19.0.0-rc-7771d3a7-20240827",
+ "react-dom": "19.0.0-rc-7771d3a7-20240827"
}
}
```
Fixes#68533
A link to docs in `next-dev.d.ts` linked to the pages router docs in app
router templates. This was confusing, because some sections on this page
only work with pages router. This PR fixes that
---------
Co-authored-by: Sam Ko <[email protected]>
### What?
Ports a few more useful scripts from nextpack, namely:
- `patch-next`
- `sweep`
- `build-native` (to clean up incremental artifacts on compiler panics)
- the macOS compression agent
Replace uses of `globby` (which is not in the `package.json`) with
`glob` (which is in the `package.json`).
These scripts were part of the nextpack repository (RIP), and
@wbinnssmith copied them over in #68471
Tested by running `pnpm pack-next`, inspecting the `tarballs` directory,
installing the tarballs into shadcn-ui, and building with turbopack.
`globby` and `glob` share the same API, but Next.js pulls in a pretty
old version of `glob`, so some of the newer APIs don't exist, and it
doesn't natively use promises:
https://github.com/isaacs/node-glob/tree/v7.1.6
We could upgrade `glob` without too much pain... It's only used for
`devDependencies`.
