### What?
There is a race condition in the `pull-build-cache` script. When the
remote cache entry was removed between the dry run and the real run, it
will run the command and caches whatever is in the native folder.
Seems like there are some very old leftover files there which lead to an
broken publish.
This changes the command to fail when there are no files and empties the
folder before running the script. This should lead to pull-build-cache
to failing instead.
Closes PACK-2957
Fixes#64468
### What?
This adds a script that first gets a list of changed files compared to
our base branch (`canary`) in PRs and identifies related test files by
looking them up from a `related-tests-manifest.json`, then runs them.
### Why?
We would like to be able to run related E2E tests when certain files
change, so we can test deployment stability.
NOTE: We still want to run all E2E tests on PRs, but this script can be
used to also test deployments for hot paths. Since these tests can take
up to [3-4
hours](https://github.com/vercel/next.js/actions/workflows/test_e2e_deploy.yml),
we aim to run a subset of these tests first.
Closes NEXT-2947
To help detect when newly added/changed assertions are flakey this adds
a job to our build and test workflow to re-run them 3 times. If the
changed test is an E2E test it will re-run in both development and
production mode to ensure it's not flakey specifically in one of those
modes.
Test run with changed test can be seen here
https://github.com/vercel/next.js/actions/runs/8511797725/job/23312158523?pr=63943
Closes NEXT-2973
#62980 migrated `examples/basic-css` to use app router. A handful of our
create-next-app tests use this example when asserting the correct
project files are created, so we were asserting on a pages file existing
rather than an app dir page. This PR updates those tests.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
Closes NEXT-2754
This avoids the need to manually check when these get out of sync by
having an automated check to keep them in sync when one or the other is
updated.
x-ref: https://github.com/vercel/next.js/pull/62966
Closes NEXT-2734
As discussed this updates our canary semver bumping to use `preminor`
instead of `prerelease` to avoid overlap with patches/backports and make
it more clear for users which versions go along together.
x-ref: [slack
thread](https://vercel.slack.com/archives/C04DUD7EB1B/p1709583075769789)
Closes NEXT-2701
This removes the hacky handling for executing lerna with using the
official commands directly, this also removes the legacy package.json
scripts we had for publishing.
Closes NEXT-2700
---------
Co-authored-by: Zack Tanner <[email protected]>
### What
This PR changes setup for the turbopack documentation.
Long story in short, we want to 2 types of documentation: one for high
level description (archtecture, etcs) and another for the api doc.
rustdoc / cargo doc unfortunately doesn't support to integrate both
easily.
PR adds mdbook setup to workaround those - instead of using rustdoc's
index as entrypoint, mdbook becomes an entrypoint to rustdoc. This
allows to host plain text document, and also able to access full rustdoc
pages.
preview: https://turbopack-rust-docs-b6qta4xdq.vercel.sh/
rustdoc:
https://turbopack-rust-docs-b6qta4xdq.vercel.sh/rustdoc/index.html
Closes PACK-2475
Closes PACK-2468
### What
We'd like to deploy generated rustdocs from turbopack & related packages
(next-swc), setting up a CI workflow to automate those.
Since we need next.js side packages to build docs, this CI sets up
things in this repo.
Closes PACK-2375.
## History
Previously, we added support for `squoosh` because it was a wasm
implementation that "just worked" on all platforms when running `next
dev` for the first time. However, it was slow so we always recommended
manually installing `sharp` for production use cases running `next
build` and `next start`.
Now that [`sharp` supports
webassembly](https://sharp.pixelplumbing.com/install#webassembly), we no
longer need to maintain `squoosh`, so it can be removed. We also don't
need to make the user install sharp manually because it can be installed
under `optionalDependencies`. I left it optional in case there was some
platform that still needed to manually install the wasm variant with
`npm install --cpu=wasm32 sharp` such as codesandbox/stackblitz (I don't
believe sharp has any fallback built in yet).
Since we can guarantee `sharp`, we can also remove `get-orientation` dep
and upgrade `image-size` dep.
I also moved an [existing `sharp`
test](https://github.com/vercel/next.js/pull/56674) into its own fixture
since it was unrelated to image optimization.
## Related Issues
- Fixes https://github.com/vercel/next.js/issues/41417
- Closes https://github.com/vercel/next.js/pull/54670
- Related https://github.com/vercel/next.js/issues/54708
- Related https://github.com/vercel/next.js/issues/44804
- Related https://github.com/vercel/next.js/issues/48820
Since we reset the test project on every e2e CI run, deployment protection is automatically enabled by default.
This adds an option to the reset project workflow to disable deployment protection. Our test runners need to be able to hit these pages from an unauthenticated browser in order for the tests to work.
Verified tests are running properly in [this run](https://github.com/vercel/next.js/actions/runs/6971348806/job/18971225559) (fixing any failing tests themselves are out of scope for this PR; will evaluate once the run finishes)
Closes NEXT-1732
We have identical `resetProject` code used in `bench/vercel` and our e2e workflow action -- this updates the `resetProject` script to side-effects free (hence removing the env var) and shared between bench & e2e
Closes NEXT-1731
This updates the rust toolchain to 2023-11-16 and:
- Removes now-unnecessary `#![feature(async_fn_in_trait)]`
- Applies auto-fixable lint fixes
- Uses `Cargo.toml` new `lint` section instead of command line rustc flags
Test Plan: Tested with updated turbo in a create-next-app
Closes PACK-1979
Co-authored-by: OJ Kwon <[email protected]>
The PR is the continuation of #56917 and #57030.
The PR replaces `fs-extra#copy` with Node.js built-in `fs.cp` API (which is almost identical to `fs-extra#copy`, and has been available since Node.js 16). The PR also provides a workaround for flaky Windows `fs.rename` operation (#57030).
cc @styfle @wbinnssmith
Doesn't make any functional changes. Going through the current setup for isolated tests to figure out a better way to cache the repository setup, so that we don't have to wait ~30s+ when running tests locally.
### What?
This PR adds an hourly workflow that will update the test manifest used when testing with Turbopack.
### Why?
To ensure we don't regress any test suites.
### How?
I use the existing `scripts/update-fonts-data-workflow.js` workflow script which will execute a script, then create a PR with the current working tree. If any pending automated PRs exist, they will be closed when a new one is opened.
The PR follows #56536 and #56491, replacing `fs-extra` usages inside the `scripts/` folder.
Note that the `copy` and `move` haven't been replaced yet. Currently, there is no better recursive copy (lightweight, promise-based, Node.js built-in `copyFile` API-based, support the `filter` option) library alternative available on npm, and Node.js built-in `fs.rename` doesn't support `overwrite`.
The PR also replaces many async fs API usage with their sync versions.
cc @wbinnssmith
### What?
Merged a bunch of dependabot alerts in my own canary branch, mainly
postcss patch updates, and one graphql minor update, to fix moderate
security vulnerabilities in examples. Spot checked most and look good
still. EDIT: also one in scripts/send-trace-to-jaeger
### Why?
Because safety
### How?
Dependabot
---------
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Steven <[email protected]>
This consolidates how we're evaluating when to opt into `react@experimental` since it's sprinkled in a lot of spots. Also adds a new flag to opt into the experimental channel
Closes NEXT-1632
## What?
In Next, rendering a route involves 3 layers:
- the routing layer, which will direct the request to the correct route to render
- the rendering layer, which will take a route and render it appropriately
- the user layer, which contains the user code
In #51831, in order to optimise the boot time of Next.js, I introduced a change that allowed the routing layer to be bundled. In this PR, I'm doing the same for the rendering layer. This is building up on @wyattjoh's work that initially split the routing and the rendering layer into separate entry-points.
The benefits of having this approach is that this allows us to compartmentalise the different part of Next, optimise them individually and making sure that serving a request is as efficient as possible, e.g. rendering a `pages` route should not need code from the `app router` to be used.
There are now 4 different rendering runtimes, depending on the route type:
- app pages: for App Router pages
- app routes: for App Router route handlers
- pages: for legacy pages
- pages api: for legacy API routes
This change should be transparent to the end user, beside faster cold boots.
## Notable changes
Doing this change required a lot of changes for Next.js under the hood in order to make the different layers play well together.
### New conventions for externals/shared modules
The big issue of bundling the rendering runtimes is that the user code needs to be able to reference an instance of a module/value created in Next during the render. This is the case when the user wants to access the router context during SSR via `next/link` for example; when you call `useContext(value)` the value needs to be the exact same reference to one as the one created by `createContext` earlier.
Previously, we were handling this case by making all files from Next that were affected by this `externals`, meaning that we were marking them not to be bundled.
**Why not keep it this way?**
The goal of this PR as stated previously was to make the rendering process as efficient as possible, so I really wanted to avoid extraneous fs reads to unoptimised code.
In order to "fix" it, I introduced two new conventions to the codebase:
- all files that explicitly need to be shared between a rendering runtime and the user code must be suffixed by `.shared-runtime` and exposed via adding a reference in the relevant `externals` file. At compilation time, a reference to a file ending with this will get re-written to the appropriate runtime.
- all files that need to be truly externals need to be suffixed by `.external`. At compilation time, a reference to it will stay as-is. This special case is needed mostly only for the async local storages that need to be shared with all three layers of Next.
As a side effect, we should be bundling more of the Next code in the user bundles, so it should be slightly more efficient.
### App route handlers are compiled on their own layer
App route handlers should be compiled in their own layer, this allows us to separate more cleanly the compilation logic here (we don't need to run the RSC logic for example).
### New rendering bundles
We now generate a prod and a dev bundle for:
- the routing server
- the app/pages SSR rendering process
- the API routes process
The development bundle is needed because:
- there is code in Next that relies on NODE_ENV
- because we opt out of the logic referencing the correct rendering runtime in dev for a `shared-runtime` file. This is because we don't need to and that Turbopack does not support rewriting an external to something that looks like this `require('foo').bar.baz` yet. We will need to fix that when Turbopack build ships.
### New development pipeline
Bundling Next is now required when developing on the repo so I extended the taskfile setup to account for that. The webpack config for Next itself lives in `webpack.config.js` and contains the logic for all the new bundles generated.
### Misc changes
There are some misc reshuffling in the code to better use the tree shaking abilities that we can now use.
fixes NEXT-1573
Co-authored-by: Alex Kirszenberg <[email protected]>
### What?
Switch the default for `--turbo` to the new `--experimental-turbo`, remove the old code in next.js
### Why?
The new approach will be used in future
Closes WEB-1506
