Will Binns-SmithandGitHub
466cdfaa27
Currently, importing a new module causes a complete eviction (`clear()`) and re-evaluation of server chunks, both in the Turbopack runtime's module cache and Node's `require.cache`. Here's what happens today: - A new module is imported into a chunk's graph - This changes its `availability_info`, which in dev for non-entry chunks is encoded into the chunk's filepath - `VersionedContentMap` works entirely on chunk paths. When we construct instructions to transition a chunk into its new state, we fail to find the prior state and fall back to `clear()` as described above. An ideal version of this is a refactor that implements `VersionedContentMap` on a per-module basis, not a per-chunk one. This PR doesn't do that, but achieves consistent module-level updates when a chunk's availability info changes. It does this by implementing chunk lists for server entry chunks the same way the client hmr implementation does: an entry chunk's version aggregates the versions of its dependent chunks (including dynamically imported ones), keyed by merger rather than path. Entry chunks do not encode availability info in their paths, so they are insulated from missing versions. Updates are applied in Node.js through the same shared merged-update machinery in the unified hmr runtime that the client already uses, plus a new `ChunkListUpdate` branch in the server hmr client to unwrap the merged updates. Details: - The merged-update wire format (`EcmascriptMergedUpdate` etc.) moves from turbopack-browser into shared `turbopack_ecmascript::chunk_list`, along with runtime-agnostic `ChunkListVersion` and `update_chunk_list` implementations. - `turbopack-nodejs` gains a chunk content merger mirroring the browser one. `EcmascriptBuildNodeEntryChunk`'s versioned content is now a chunk-list content over its sync and async chunks. It is version-only and never emitted; the entry chunk still inlines its loader calls. - The aggregate server hmr subscription now tracks only entry chunks. Shared chunks under `server/chunks/` ride the entry's `ChunkListUpdate` as module deltas, so their content-hash paths no longer matter. - The Node hmr client applies `ChunkListUpdate` by feeding each nested merged update through the shared apply path. Modules that appear in an "added" chunk but already exist in the module cache were moved by a chunk rename and are treated as modified. - On a successful partial apply, the hot reloader clears the manifest cache for updated chunks and notifies browsers to refetch RSC, without clearing `require.cache`. ### Test Plan Adds a series of e2e tests to catch the described manual test plan in both entry and dynamic chunks.
turbopack-ecmascript-runtime
This crate contains Turbopack's ECMAScript runtimes. These include:
- Development runtimes for the browser, Node.js, and Edge-like runtimes;
- Production runtimes for the browser, Node.js, and Edge-like runtimes (only Node.js is implemented for now).
<reference path="...">
The TypeScript files corresponding to the runtime itself all use <reference path="..."> instead of import/export
to import dependencies. This is because the runtime doesn't use a module system. Instead, the files are concatenated
together in a specific order.
As such, the <reference path="..."> statements more closely map to the way the runtime is actually built. They also
allow us to refer to top-level declarations in another file without having to import/export them, which makes no
sense in the context of a runtime.
Why is everything in one crate?
The runtime-agnostic code (js/src/shared) was originally placed in turbopack-ecmascript, and the runtime-specific
code (js/src/{build,dev}) in turbopack-{build,dev}.
However, <reference path="..."> statements only support relative paths. You can't refer to a file in a dependency. For
the typings to work properly, and for them to be usable from outside of this repo (e.g. in the Next.js repo), it's much
easier to have everything in one package.
Why so many tsconfig.json?
Since different runtimes are meant to run in different environments, they use different tsconfig.json files to
customize what APIs are available to them. For example, the browser runtime can use window and document, but the
Node.js runtime can't.
All of these tsconfig.json files extend tsconfig.base.json, which contains the common configuration for all
runtimes.