Turbopack: Add importModule() support to webpack loaders (#89630)

## What?

Adds support for `this.importModule()` in Turbopack's webpack loader
compatibility layer. This API allows webpack loaders to dynamically
import and execute modules (CJS, ESM, TypeScript, JSON, WebAssembly,
etc.) during the build process, matching webpack's native
`this.importModule()` behavior.

## Why?

Webpack loaders like `@vanilla-extract/webpack-plugin`, `val-loader`,
and custom loaders use `this.importModule()` to:
- Load configuration files that are themselves modules with dependencies
- Execute code at build time to generate derived source
- Process dependency chains (e.g., a TS config that imports a CJS module
that requires JSON)

Without this, loaders relying on `importModule()` cannot work with
Turbopack.

## How?

### Architecture

The implementation follows a request/response pattern between the
Node.js loader runner and the Rust-side Turbopack compiler:

1. **Loader calls `this.importModule(request)`** in the Node.js loader
runner
2. **IPC message sent to Rust** with the module request and lookup path
3. **Rust resolves the module** using Turbopack's full resolver (with
aliases, loader rules, etc.)
4. **Rust builds a Node.js bundle** containing the module and all its
dependencies using Turbopack's code generation pipeline
5. **Bundle chunks sent back to Node.js** via IPC
6. **Node.js evaluates the bundle in-memory** using `vm.compileFunction`
with a minimal CJS module system

This approach leverages Turbopack's full module graph, so imported
modules benefit from the same resolution (aliases, custom loaders,
TypeScript support, etc.) as regular imports.

### Key Changes

**Rust side (`turbopack-node/src/transforms/webpack.rs`):**
- New `ImportModule` IPC response message type
- Resolves modules through Turbopack's `esm_resolve` with full
`AssetContext`
- Generates a complete Node.js bundle (runtime + chunks + entry) for the
imported module
- Tracks file dependencies for proper cache invalidation

**`SourceTransform` trait (`turbopack-core/src/source_transform.rs`):**
- Added `asset_context` parameter to `SourceTransform::transform()` so
transforms can access the full asset context for module resolution and
bundling
- Updated all implementors: `JsonSourceTransform`,
`TextSourceTransform`, `BytesSourceTransform`, `MdxTransform`,
`PostCssTransform`, `WebpackLoaderItems`

**Node.js runtime (`turbopack-node/js/src/transforms/`):**
- `webpack-loaders.ts`: Implements `this.importModule()` on the loader
context, sends IPC request and evaluates the returned bundle
- `webpack-loaders-runtime.ts`: In-memory CJS module evaluator that
loads bundle chunks using `vm.compileFunction`, with support for:
  - Relative and absolute path resolution within the bundle
  - External package delegation to real Node.js `require()`
- Patched `fs.createReadStream` for in-memory WebAssembly binary assets

**New `ImportModule` reference subtype
(`turbopack-core/src/reference_type.rs`):**
- Added `EcmaScriptModulesReferenceSubType::ImportModule` to mark
references created by importModule for proper handling in the module
graph

### Test Coverage

Comprehensive e2e test
(`test/e2e/app-dir/webpack-loader-import-module/`) covering:
- **TypeScript modules** with CJS and ESM dependencies
- **CJS dependency chains** (TS → CJS → JSON)
- **ESM `.mjs` modules** with shared dependencies
- **Resolve aliases** (`alias-data` → `alias-data.mjs` via
`resolveAlias` config)
- **Custom loader rules** (`.custom-data` files processed by
`text-to-export-loader`)
- **Transitive dependencies** through aliases and custom loaders
- **Turbopack-only features**: `new URL()` asset references, WebAssembly
imports (`add.wasm`), and dynamic `import()` within importModule targets

All tests pass for both webpack and Turbopack modes.

---------

Co-authored-by: Claude <[email protected]>
Co-authored-by: Tobias Koppers <[email protected]>
This commit is contained in:
Tobias Koppers
2026-03-19 10:17:36 +01:00
committed by GitHub
co-authored by Claude Tobias Koppers
parent f8f77da914
commit bdb2f2ce4d
34 changed files with 761 additions and 26 deletions
@@ -0,0 +1,7 @@
import { label } from 'alias-dep'
const config = {
depLabel: label,
}
export default config
@@ -0,0 +1 @@
export const aliasValue = 'resolved via alias'
@@ -0,0 +1,17 @@
export const title = 'wrong'
export const items = 'wrong'
export const cjsGreeting = 'wrong'
export const version = 'wrong'
export const esmLabel = 'wrong'
export const imageUrl = 'wrong'
export const wasmAddResult = 'wrong'
export const dynamicValue = 'wrong'
export const mjsTitle = 'wrong'
export const mjsEsmLabel = 'wrong'
export const mjsImageUrl = 'wrong'
export const mjsWasmAddResult = 'wrong'
export const mjsDynamicValue = 'wrong'
export const aliasValue = 'wrong'
export const aliasDepLabel = 'wrong'
export const customDataValue = 'wrong'
export const consumedValue = 'wrong'
@@ -0,0 +1,7 @@
export default function Root({ children }: { children: React.ReactNode }) {
return (
<html>
<body>{children}</body>
</html>
)
}
@@ -0,0 +1,43 @@
import {
title,
items,
cjsGreeting,
version,
esmLabel,
imageUrl,
wasmAddResult,
dynamicValue,
mjsTitle,
mjsEsmLabel,
mjsImageUrl,
mjsWasmAddResult,
mjsDynamicValue,
aliasValue,
aliasDepLabel,
customDataValue,
consumedValue,
} from './file.test-file'
export default function Page() {
return (
<div>
<p id="title">{title}</p>
<p id="items">{items}</p>
<p id="cjs-greeting">{cjsGreeting}</p>
<p id="version">{version}</p>
<p id="esm-label">{esmLabel}</p>
<p id="image-url">{imageUrl}</p>
<p id="wasm-add-result">{wasmAddResult}</p>
<p id="dynamic-value">{dynamicValue}</p>
<p id="mjs-title">{mjsTitle}</p>
<p id="mjs-esm-label">{mjsEsmLabel}</p>
<p id="mjs-image-url">{mjsImageUrl}</p>
<p id="mjs-wasm-add-result">{mjsWasmAddResult}</p>
<p id="mjs-dynamic-value">{mjsDynamicValue}</p>
<p id="alias-value">{aliasValue}</p>
<p id="alias-dep-label">{aliasDepLabel}</p>
<p id="custom-data-value">{customDataValue}</p>
<p id="consumed-value">{consumedValue}</p>
</div>
)
}
@@ -0,0 +1,13 @@
import metadata from './metadata.json'
interface CjsDep {
greeting: string
version: string
}
const dep: CjsDep = {
greeting: 'hello from cjs',
version: metadata.version,
}
export default dep
@@ -0,0 +1,8 @@
import { label } from './esm-dep.mjs'
const config = {
mjsTitle: 'ESM Config Works',
esmLabel: label,
}
export default config
@@ -0,0 +1,20 @@
import cjsDep from './cjs-dep'
import { label as esmLabel } from './esm-dep.mjs'
interface Config {
title: string
items: string[]
cjsGreeting: string
version: string
esmLabel: string
}
const config: Config = {
title: 'Import Module Works',
items: ['apple', 'banana', 'cherry'],
cjsGreeting: cjsDep.greeting,
version: cjsDep.version,
esmLabel,
}
export default config
@@ -0,0 +1,3 @@
import value from './values.custom-data'
export const consumedValue = value
@@ -0,0 +1 @@
export const label = 'hello from esm'
Binary file not shown.

After

Width:  |  Height:  |  Size: 70 B

@@ -0,0 +1,4 @@
{
"version": "1.0.0",
"author": "test"
}
@@ -0,0 +1 @@
export const dynamicValue = 'loaded dynamically'
@@ -0,0 +1,61 @@
const path = require('path')
/**
* @type {import('next').NextConfig}
*/
const nextConfig = {
turbopack: {
resolveAlias: {
'alias-data': './alias-data.mjs',
'alias-dep': './esm-dep.mjs',
},
rules: {
'*.test-file.ts': {
loaders: [require.resolve('./test-file-loader.js')],
as: '*.js',
},
'*.custom-data': {
loaders: [require.resolve('./text-to-export-loader.js')],
as: '*.js',
},
},
},
webpack(config) {
config.resolve.alias = {
...config.resolve.alias,
'alias-data': path.resolve(__dirname, 'alias-data.mjs'),
'alias-dep': path.resolve(__dirname, 'esm-dep.mjs'),
}
// Wrap externals handlers to prevent aliased bare specifiers from
// being treated as external ESM packages by Next.js.
const aliasedModules = new Set(['alias-data', 'alias-dep'])
config.externals = config.externals.map((fn) => {
if (typeof fn !== 'function') return fn
return async function (ctx) {
if (aliasedModules.has(ctx.request)) return
return fn.apply(this, arguments)
}
})
config.module.rules.push(
{
test: /\.test-file\.ts/,
use: require.resolve('./test-file-loader.js'),
},
{
test: /\.custom-data$/,
use: require.resolve('./text-to-export-loader.js'),
}
)
// Required so webpack can compile .wasm files referenced by
// url-wasm-data.ts (used via importModule in the loader).
// The actual wasm execution in importModule doesn't work in
// webpack, but compilation needs to succeed.
config.experiments = {
...config.experiments,
asyncWebAssembly: true,
}
return config
},
}
module.exports = nextConfig
@@ -0,0 +1,106 @@
const path = require('path')
module.exports = async function () {
// Import a TypeScript module that has its own dependencies
// (cjs-dep.ts -> metadata.json, esm-dep.mjs)
const configPath = path.resolve(__dirname, 'config-data.ts')
const configModule = await this.importModule(configPath)
const config = configModule.default || configModule
// Import an ESM .mjs module that also imports esm-dep.mjs
// Uses a relative path (resolved relative to the transformed module, app/file.test-file.ts)
const mjsModule = await this.importModule('../config-data.mjs')
const mjsConfig = mjsModule.default || mjsModule
const title = config.title
const items = config.items.join(', ')
const cjsGreeting = config.cjsGreeting
const version = config.version
const esmLabel = config.esmLabel
const mjsTitle = mjsConfig.mjsTitle
const mjsEsmLabel = mjsConfig.esmLabel
// --- resolveAlias tests ---
let aliasValue = 'unsupported'
let aliasDepLabel = 'unsupported'
try {
// importModule with alias as the request
const aliasModule = await this.importModule('alias-data')
const aliasData = aliasModule.default || aliasModule
aliasValue = aliasData.aliasValue
// importModule on a module whose dependency uses an alias
// Uses a relative path (resolved relative to the transformed module)
const aliasConfigModule = await this.importModule('../alias-config.mjs')
const aliasConfig = aliasConfigModule.default || aliasConfigModule
aliasDepLabel = aliasConfig.depLabel
} catch (e) {
// resolveAlias may not be supported in all contexts
console.error('resolveAlias importModule failed:', e)
}
// --- loader rules tests ---
let customDataValue = 'unsupported'
let consumedValue = 'unsupported'
try {
// importModule on a file that requires a custom loader rule
const customDataPath = path.resolve(__dirname, 'values.custom-data')
const customDataModule = await this.importModule(customDataPath)
customDataValue = customDataModule.default || customDataModule
// importModule on a module whose dependency needs a custom loader rule
// Uses a relative path (resolved relative to the transformed module)
const dataConsumerModule = await this.importModule('../data-consumer.mjs')
const dataConsumer = dataConsumerModule.default || dataConsumerModule
consumedValue = dataConsumer.consumedValue
} catch (e) {
// loader rules may not be supported in importModule in all contexts
console.error('loader rules importModule failed:', e)
}
// Try importing the wasm+URL+dynamic-import module (supported in
// Turbopack, not in webpack's importModule)
let imageUrl = 'unsupported'
let wasmAddResult = 'unsupported'
let dynamicValue = 'unsupported'
let mjsImageUrl = 'unsupported'
let mjsWasmAddResult = 'unsupported'
let mjsDynamicValue = 'unsupported'
try {
const urlWasmPath = path.resolve(__dirname, 'url-wasm-data.ts')
const urlWasmModule = await this.importModule(urlWasmPath)
const urlWasm = urlWasmModule.default || urlWasmModule
imageUrl = urlWasm.imageUrl
wasmAddResult = String(urlWasm.wasmAddResult)
dynamicValue = urlWasm.dynamicValue
const mjsUrlWasmModule = await this.importModule('../url-wasm-data.mjs')
const mjsUrlWasm = mjsUrlWasmModule.default || mjsUrlWasmModule
mjsImageUrl = mjsUrlWasm.mjsImageUrl
mjsWasmAddResult = String(mjsUrlWasm.mjsWasmAddResult)
mjsDynamicValue = mjsUrlWasm.mjsDynamicValue
} catch {
// webpack's importModule doesn't support wasm/URL asset patterns
}
return `
export const title = ${JSON.stringify(title)};
export const items = ${JSON.stringify(items)};
export const cjsGreeting = ${JSON.stringify(cjsGreeting)};
export const version = ${JSON.stringify(version)};
export const esmLabel = ${JSON.stringify(esmLabel)};
export const imageUrl = ${JSON.stringify(imageUrl)};
export const wasmAddResult = ${JSON.stringify(wasmAddResult)};
export const dynamicValue = ${JSON.stringify(dynamicValue)};
export const mjsTitle = ${JSON.stringify(mjsTitle)};
export const mjsEsmLabel = ${JSON.stringify(mjsEsmLabel)};
export const mjsImageUrl = ${JSON.stringify(mjsImageUrl)};
export const mjsWasmAddResult = ${JSON.stringify(mjsWasmAddResult)};
export const mjsDynamicValue = ${JSON.stringify(mjsDynamicValue)};
export const aliasValue = ${JSON.stringify(aliasValue)};
export const aliasDepLabel = ${JSON.stringify(aliasDepLabel)};
export const customDataValue = ${JSON.stringify(customDataValue)};
export const consumedValue = ${JSON.stringify(consumedValue)};
`
}
@@ -0,0 +1,3 @@
module.exports = function (source) {
return `export default ${JSON.stringify(source.trim())};`
}
@@ -0,0 +1,13 @@
import { add } from './add.wasm'
const imageUrl = new URL('./image.png', import.meta.url)
const dynamicModule = await import('./module.js')
const data = {
mjsImageUrl: imageUrl.pathname,
mjsWasmAddResult: add(10, 20),
mjsDynamicValue: dynamicModule.dynamicValue,
}
export default data
@@ -0,0 +1,13 @@
import { add } from './add.wasm'
const imageUrl = new URL('./image.png', import.meta.url)
const dynamicModule = await import('./module.js')
const data = {
imageUrl: imageUrl.pathname,
wasmAddResult: add(1, 2),
dynamicValue: dynamicModule.dynamicValue,
}
export default data
@@ -0,0 +1 @@
hello from custom loader
@@ -0,0 +1 @@
declare module '*.wasm'
@@ -0,0 +1,52 @@
import { nextTestSetup } from 'e2e-utils'
describe('webpack-loader-import-module', () => {
const { next, skipped, isTurbopack } = nextTestSetup({
files: __dirname,
skipDeployment: true,
})
if (skipped) {
return
}
it('should support this.importModule() in a webpack loader', async () => {
const $ = await next.render$('/')
expect($('#title').text()).toBe('Import Module Works')
expect($('#items').text()).toBe('apple, banana, cherry')
// CJS dependency that itself requires a JSON file
expect($('#cjs-greeting').text()).toBe('hello from cjs')
expect($('#version').text()).toBe('1.0.0')
// ESM dependency imported from config-data.ts
expect($('#esm-label').text()).toBe('hello from esm')
// ESM .mjs module (config-data.mjs)
expect($('#mjs-title').text()).toBe('ESM Config Works')
expect($('#mjs-esm-label').text()).toBe('hello from esm')
// resolveAlias: importModule with alias as request
expect($('#alias-value').text()).toBe('resolved via alias')
// resolveAlias: dependency of importModule target uses alias
expect($('#alias-dep-label').text()).toBe('hello from esm')
// loader rules: importModule on file requiring custom loader
expect($('#custom-data-value').text()).toBe('hello from custom loader')
// loader rules: dependency of importModule target needs custom loader
expect($('#consumed-value').text()).toBe('hello from custom loader')
if (isTurbopack) {
// new URL('./image.png', import.meta.url) in url-wasm-data.ts
expect($('#image-url').text()).toContain('image')
expect($('#image-url').text()).toMatch(/\.png/)
// WebAssembly add(1, 2) from add.wasm in url-wasm-data.ts
expect($('#wasm-add-result').text()).toBe('3')
// Dynamic import('./module.js') in url-wasm-data.ts
expect($('#dynamic-value').text()).toBe('loaded dynamically')
// new URL('./image.png', import.meta.url) in url-wasm-data.mjs
expect($('#mjs-image-url').text()).toContain('image')
expect($('#mjs-image-url').text()).toMatch(/\.png/)
// WebAssembly add(10, 20) from add.wasm in url-wasm-data.mjs
expect($('#mjs-wasm-add-result').text()).toBe('30')
// Dynamic import('./module.js') in url-wasm-data.mjs
expect($('#mjs-dynamic-value').text()).toBe('loaded dynamically')
}
})
})
@@ -74,6 +74,9 @@ pub enum ImportWithType {
pub enum EcmaScriptModulesReferenceSubType {
ImportPart(ModulePart),
Import,
/// Used for `importModule()` in webpack loaders, where a module and its
/// transitive dependencies are compiled and executed in a loader context.
ImportModule,
ImportWithType(RcStr),
/// Import with `turbopackLoader` attribute specifying an inline loader.
ImportWithTurbopackUse {
@@ -1,11 +1,15 @@
use turbo_tasks::{ResolvedVc, Vc};
use crate::source::Source;
use crate::{context::AssetContext, source::Source};
#[turbo_tasks::value_trait]
pub trait SourceTransform {
#[turbo_tasks::function]
fn transform(self: Vc<Self>, source: Vc<Box<dyn Source>>) -> Vc<Box<dyn Source>>;
fn transform(
self: Vc<Self>,
source: Vc<Box<dyn Source>>,
asset_context: Vc<Box<dyn AssetContext>>,
) -> Vc<Box<dyn Source>>;
}
#[turbo_tasks::value(transparent)]
@@ -14,9 +18,13 @@ pub struct SourceTransforms(Vec<ResolvedVc<Box<dyn SourceTransform>>>);
#[turbo_tasks::value_impl]
impl SourceTransforms {
#[turbo_tasks::function]
pub fn transform(&self, source: Vc<Box<dyn Source>>) -> Vc<Box<dyn Source>> {
self.0
.iter()
.fold(source, |source, transform| transform.transform(source))
pub fn transform(
&self,
source: Vc<Box<dyn Source>>,
asset_context: Vc<Box<dyn AssetContext>>,
) -> Vc<Box<dyn Source>> {
self.0.iter().fold(source, |source, transform| {
transform.transform(source, asset_context)
})
}
}
@@ -5,6 +5,7 @@ use turbo_tasks::Vc;
use turbo_tasks_fs::{File, FileContent};
use turbopack_core::{
asset::{Asset, AssetContent},
context::AssetContext,
source::Source,
source_transform::SourceTransform,
virtual_source::VirtualSource,
@@ -30,7 +31,11 @@ impl BytesSourceTransform {
#[turbo_tasks::value_impl]
impl SourceTransform for BytesSourceTransform {
#[turbo_tasks::function]
async fn transform(self: Vc<Self>, source: Vc<Box<dyn Source>>) -> Result<Vc<Box<dyn Source>>> {
async fn transform(
self: Vc<Self>,
source: Vc<Box<dyn Source>>,
_asset_context: Vc<Box<dyn AssetContext>>,
) -> Result<Vc<Box<dyn Source>>> {
let ident = source.ident();
let path = ident.path().await?;
let content = source.content().file_content().await?;
@@ -4,6 +4,7 @@ use turbo_tasks::Vc;
use turbo_tasks_fs::{File, FileContent, FileJsonContent};
use turbopack_core::{
asset::{Asset, AssetContent},
context::AssetContext,
issue::{IssueExt, IssueSeverity, IssueSource, StyledString, code_gen::CodeGenerationIssue},
source::Source,
source_transform::SourceTransform,
@@ -51,7 +52,11 @@ impl JsonSourceTransform {
#[turbo_tasks::value_impl]
impl SourceTransform for JsonSourceTransform {
#[turbo_tasks::function]
async fn transform(self: Vc<Self>, source: Vc<Box<dyn Source>>) -> Result<Vc<Box<dyn Source>>> {
async fn transform(
self: Vc<Self>,
source: Vc<Box<dyn Source>>,
_asset_context: Vc<Box<dyn AssetContext>>,
) -> Result<Vc<Box<dyn Source>>> {
let this = self.await?;
let ident = source.ident();
let path = ident.path().await?;
@@ -3,6 +3,7 @@ use turbo_tasks::Vc;
use turbo_tasks_fs::{File, FileContent};
use turbopack_core::{
asset::{Asset, AssetContent},
context::AssetContext,
source::Source,
source_transform::SourceTransform,
virtual_source::VirtualSource,
@@ -28,7 +29,11 @@ impl TextSourceTransform {
#[turbo_tasks::value_impl]
impl SourceTransform for TextSourceTransform {
#[turbo_tasks::function]
async fn transform(self: Vc<Self>, source: Vc<Box<dyn Source>>) -> Result<Vc<Box<dyn Source>>> {
async fn transform(
self: Vc<Self>,
source: Vc<Box<dyn Source>>,
_asset_context: Vc<Box<dyn AssetContext>>,
) -> Result<Vc<Box<dyn Source>>> {
let ident = source.ident();
let path = ident.path().await?;
let content = source.content().file_content().await?;
+6 -1
View File
@@ -10,6 +10,7 @@ use turbo_tasks::{ResolvedVc, ValueDefault, Vc};
use turbo_tasks_fs::{File, FileContent, FileSystemPath, rope::Rope};
use turbopack_core::{
asset::{Asset, AssetContent},
context::AssetContext,
ident::AssetIdent,
issue::{
Issue, IssueExt, IssueSource, IssueStage, OptionIssueSource, OptionStyledString,
@@ -90,7 +91,11 @@ impl MdxTransform {
#[turbo_tasks::value_impl]
impl SourceTransform for MdxTransform {
#[turbo_tasks::function]
fn transform(&self, source: ResolvedVc<Box<dyn Source>>) -> Vc<Box<dyn Source>> {
fn transform(
&self,
source: ResolvedVc<Box<dyn Source>>,
_asset_context: ResolvedVc<Box<dyn AssetContext>>,
) -> Vc<Box<dyn Source>> {
Vc::upcast(
MdxTransformedAsset {
options: self.options,
@@ -41,6 +41,11 @@ export type IpcRequestMessage =
type: 'trackFileRead'
file: string
}
| {
type: 'importModule'
lookupPath: string
request: string
}
export type TransformIpc = Ipc<IpcInfoMessage, IpcRequestMessage>
@@ -0,0 +1,102 @@
/**
* In-memory CJS module evaluator for importModule bundles.
*
* The Rust side generates a full Node.js bundle (runtime + module chunks + entry
* chunk) and sends the chunk code via IPC. This evaluator loads them in memory
* using vm.compileFunction, wiring up a minimal CJS require that resolves
* relative paths against an in-memory chunk map and delegates external packages
* to the real Node.js require.
*/
import { createRequire } from 'module'
import realFs from 'fs'
import path from 'path'
import { Readable } from 'stream'
import vm from 'vm'
// Create a real Node.js require function rooted at the project directory.
// This bypasses Turbopack's module analysis (which can't handle dynamic
// require calls) while still resolving npm packages correctly.
const nodeRequire = createRequire(process.cwd() + '/')
export interface ImportModuleResult {
entryPath: string
chunks: Array<{
path: string
code?: string
binary?: string
sourceMap?: string
}>
}
export function evaluateBundle(result: ImportModuleResult): any {
const chunkMap = new Map<string, string>()
const binaryMap = new Map<string, Buffer>()
for (const chunk of result.chunks) {
if (chunk.code != null) {
chunkMap.set('/' + chunk.path, chunk.code)
}
if (chunk.binary != null) {
binaryMap.set('/' + chunk.path, Buffer.from(chunk.binary, 'base64'))
}
}
// Patched fs module that intercepts reads of in-memory binary assets.
// The Node.js runtime's WASM loader uses createReadStream() to load .wasm
// files from disk; we need to redirect those reads to our in-memory buffers.
let patchedFs: typeof realFs | undefined
if (binaryMap.size > 0) {
patchedFs = Object.create(realFs)
patchedFs!.createReadStream = ((filePath: string, ...args: any[]): any => {
const buf = binaryMap.get(filePath)
if (buf) {
return Readable.from(buf)
}
return realFs.createReadStream(filePath, ...args)
}) as any
}
const moduleCache = new Map<string, { exports: any }>()
function inMemoryRequire(fromPath: string, request: string): any {
let resolved: string
if (request.startsWith('/')) {
resolved = request
} else if (request.startsWith('./') || request.startsWith('../')) {
resolved = path.resolve(path.dirname(fromPath), request)
} else {
// Intercept 'fs' to return the patched version when binary
// assets are present (needed for WASM loading)
if (request === 'fs' && patchedFs) {
return patchedFs
}
// External package — use the real Node.js require
return nodeRequire(request)
}
const cached = moduleCache.get(resolved)
if (cached) return cached.exports
const code = chunkMap.get(resolved)
if (!code) {
throw new Error(
`importModule: chunk not found: ${resolved} (required from ${fromPath})`
)
}
const mod = { exports: {} as any }
moduleCache.set(resolved, mod)
const localRequire: any = (req: string) => inMemoryRequire(resolved, req)
localRequire.resolve = nodeRequire.resolve
const fn = vm.compileFunction(
code,
['module', 'exports', 'require', '__filename', '__dirname'],
{ filename: resolved }
)
fn(mod, mod.exports, localRequire, resolved, path.dirname(resolved))
return mod.exports
}
return inMemoryRequire('/', '/' + result.entryPath)
}
@@ -15,6 +15,10 @@ import {
toPath,
type TransformIpc,
} from './transforms'
import {
evaluateBundle,
type ImportModuleResult,
} from './webpack-loaders-runtime'
import fs from 'fs'
import path from 'path'
@@ -41,12 +45,18 @@ export type IpcInfoMessage =
}>
}
export type IpcRequestMessage = {
type: 'resolve'
options: any
lookupPath: string
request: string
}
export type IpcRequestMessage =
| {
type: 'resolve'
options: any
lookupPath: string
request: string
}
| {
type: 'importModule'
lookupPath: string
request: string
}
type LoaderConfig =
| string
@@ -351,6 +361,53 @@ const transform = (
},
emitWarning: makeErrorEmitter('warning', ipc),
emitError: makeErrorEmitter('error', ipc),
importModule(
request: string,
optionsOrCallback?: any,
maybeCallback?: (err: Error | null, result?: any) => void
) {
// Support both (request, options, callback) and (request, options) -> Promise
let callback:
| ((err: Error | null, result?: any) => void)
| undefined
if (typeof optionsOrCallback === 'function') {
callback = optionsOrCallback
} else {
callback = maybeCallback
}
const doImport = async () => {
let actualRequest = request
// The webpack API allows absolute paths in importModule requests,
// but Turbopack's resolver expects relative paths. Convert absolute
// paths to relative ones for backward compatibility with webpack loaders.
if (path.isAbsolute(request)) {
actualRequest = path.relative(resourceDir, request)
if (
!path.isAbsolute(actualRequest) &&
actualRequest.split(path.sep)[0] !== '..'
) {
actualRequest = './' + actualRequest
}
}
const result = (await ipc.sendRequest({
type: 'importModule',
lookupPath: toPath(resourceDir),
request: actualRequest,
})) as ImportModuleResult
return evaluateBundle(result)
}
if (!callback) {
return doImport()
}
doImport().then(
(result) => callback!(null, result),
(err) => callback!(err as Error)
)
},
getLogger(name: unknown) {
const logFn = (logType: string, ...args: unknown[]) => {
let trace: StackFrame[] | undefined
@@ -144,7 +144,11 @@ impl PostCssTransform {
#[turbo_tasks::value_impl]
impl SourceTransform for PostCssTransform {
#[turbo_tasks::function]
fn transform(&self, source: ResolvedVc<Box<dyn Source>>) -> Vc<Box<dyn Source>> {
fn transform(
&self,
source: ResolvedVc<Box<dyn Source>>,
asset_context: ResolvedVc<Box<dyn AssetContext>>,
) -> Vc<Box<dyn Source>> {
Vc::upcast(
PostCssTransformedAsset {
evaluate_context: self.evaluate_context,
@@ -152,6 +156,7 @@ impl SourceTransform for PostCssTransform {
execution_context: self.execution_context,
config_location: self.config_location,
source,
asset_context,
source_map: self.source_maps,
}
.cell(),
@@ -166,6 +171,7 @@ struct PostCssTransformedAsset {
execution_context: ResolvedVc<ExecutionContext>,
config_location: PostCssConfigLocation,
source: ResolvedVc<Box<dyn Source>>,
asset_context: ResolvedVc<Box<dyn AssetContext>>,
source_map: bool,
}
@@ -553,8 +559,10 @@ impl PostCssTransformedAsset {
node_backend: *node_backend,
context_source_for_issue: self.source,
chunking_context: *chunking_context,
evaluate_context: self.evaluate_context,
module_graph,
resolve_options_context: None,
asset_context: self.asset_context,
args: vec![
ResolvedVc::cell(content.into()),
ResolvedVc::cell(css_path.into()),
@@ -23,7 +23,7 @@ use turbo_tasks_fs::{
};
use turbopack_core::{
asset::{Asset, AssetContent},
chunk::ChunkingContext,
chunk::{ChunkingContext, ChunkingContextExt, EvaluatableAsset},
context::{AssetContext, ProcessResult},
file_source::FileSource,
ident::AssetIdent,
@@ -31,10 +31,16 @@ use turbopack_core::{
Issue, IssueExt, IssueSeverity, IssueSource, IssueStage, OptionIssueSource,
OptionStyledString, StyledString,
},
module_graph::{ModuleGraph, SingleModuleGraph},
reference_type::{InnerAssets, ReferenceType},
module_graph::{
ModuleGraph, SingleModuleGraph,
chunk_group_info::{ChunkGroup, ChunkGroupEntry},
},
output::{ExpandOutputAssetsInput, OutputAsset, OutputAssets, expand_output_assets},
reference_type::{EcmaScriptModulesReferenceSubType, InnerAssets, ReferenceType},
resolve::{
ResolveErrorMode,
options::{ConditionValue, ResolveInPackage, ResolveIntoPackage, ResolveOptions},
origin::PlainResolveOrigin,
parse::Request,
pattern::Pattern,
resolve,
@@ -45,7 +51,8 @@ use turbopack_core::{
virtual_source::VirtualSource,
};
use turbopack_resolve::{
ecmascript::get_condition_maps, resolve::resolve_options,
ecmascript::{esm_resolve, get_condition_maps},
resolve::resolve_options,
resolve_options_context::ResolveOptionsContext,
};
@@ -125,11 +132,13 @@ impl SourceTransform for WebpackLoaders {
fn transform(
self: ResolvedVc<Self>,
source: ResolvedVc<Box<dyn Source>>,
asset_context: ResolvedVc<Box<dyn AssetContext>>,
) -> Vc<Box<dyn Source>> {
Vc::upcast(
WebpackLoadersProcessedAsset {
transform: self,
source,
asset_context,
}
.cell(),
)
@@ -140,6 +149,7 @@ impl SourceTransform for WebpackLoaders {
struct WebpackLoadersProcessedAsset {
transform: ResolvedVc<WebpackLoaders>,
source: ResolvedVc<Box<dyn Source>>,
asset_context: ResolvedVc<Box<dyn AssetContext>>,
}
#[turbo_tasks::value_impl]
@@ -290,8 +300,10 @@ impl WebpackLoadersProcessedAsset {
node_backend: *node_backend,
context_source_for_issue: self.source,
chunking_context: *chunking_context,
evaluate_context: transform.evaluate_context,
module_graph,
resolve_options_context: Some(transform.resolve_options_context),
asset_context: self.asset_context,
args: vec![
ResolvedVc::cell(content),
// We need to pass the query string to the loader
@@ -443,14 +455,35 @@ pub enum RequestMessage {
},
#[serde(rename_all = "camelCase")]
TrackFileRead { file: RcStr },
#[serde(rename_all = "camelCase")]
ImportModule { lookup_path: RcStr, request: RcStr },
}
#[derive(Serialize, Debug)]
#[serde(rename_all = "camelCase")]
pub struct ImportModuleChunk {
path: RcStr,
#[serde(skip_serializing_if = "Option::is_none")]
code: Option<RcStr>,
#[serde(skip_serializing_if = "Option::is_none")]
binary: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
source_map: Option<RcStr>,
}
#[derive(Serialize, Debug)]
#[serde(untagged)]
pub enum ResponseMessage {
Resolve { path: RcStr },
Resolve {
path: RcStr,
},
// Only used for tracking invalidations, no content is returned.
TrackFileRead {},
#[serde(rename_all = "camelCase")]
ImportModule {
entry_path: RcStr,
chunks: Vec<ImportModuleChunk>,
},
}
#[derive(Clone, PartialEq, Eq, Hash, TaskInput, Debug, TraceRawVcs, Encode, Decode)]
@@ -462,7 +495,9 @@ pub struct WebpackLoaderContext {
pub context_source_for_issue: ResolvedVc<Box<dyn Source>>,
pub module_graph: ResolvedVc<ModuleGraph>,
pub chunking_context: ResolvedVc<Box<dyn ChunkingContext>>,
pub evaluate_context: ResolvedVc<Box<dyn AssetContext>>,
pub resolve_options_context: Option<ResolvedVc<ResolveOptionsContext>>,
pub asset_context: ResolvedVc<Box<dyn AssetContext>>,
pub args: Vec<ResolvedVc<JsonValue>>,
pub additional_invalidation: ResolvedVc<Completion>,
}
@@ -646,6 +681,120 @@ impl EvaluateContext for WebpackLoaderContext {
let _ = &*self.cwd.join(&file)?.read().await?;
Ok(ResponseMessage::TrackFileRead {})
}
RequestMessage::ImportModule {
lookup_path,
request,
} => {
let lookup_path = self.cwd.join(&lookup_path)?;
let request_vc = Request::parse(Pattern::Constant(request.clone()));
let origin = PlainResolveOrigin::new(*self.asset_context, lookup_path.join("_")?);
let resolved = esm_resolve(
Vc::upcast(origin),
request_vc,
EcmaScriptModulesReferenceSubType::ImportModule,
ResolveErrorMode::Error,
Some(IssueSource::from_source_only(self.context_source_for_issue)),
)
.await?;
let Some(module) = *resolved.first_module().await? else {
bail!(
"importModule: unable to resolve {} in {}",
request,
lookup_path.value_to_string().await?
);
};
// Cast to evaluatable asset for bundle generation
let evaluatable = ResolvedVc::try_sidecast::<Box<dyn EvaluatableAsset>>(module)
.context("importModule: module is not evaluatable")?;
// Build a module graph from the resolved module and its
// transitive dependencies
let single_graph = SingleModuleGraph::new_with_entry(
ChunkGroupEntry::Entry(vec![module]),
false,
false,
);
let import_module_graph = ModuleGraph::from_single_graph(single_graph)
.connect()
.to_resolved()
.await?;
// Generate a full Node.js bundle using the real runtime
let output_root = self.chunking_context.output_root().owned().await?;
let entry_path = output_root.join("importModule.js")?;
let bootstrap = self.chunking_context.root_entry_chunk_group_asset(
entry_path.clone(),
ChunkGroup::Entry(vec![ResolvedVc::upcast(evaluatable)]),
*import_module_graph,
OutputAssets::empty(),
OutputAssets::empty(),
);
// Collect all internal assets as {path, code} pairs
let bootstrap_resolved = bootstrap.to_resolved().await?;
let all_assets = expand_output_assets(
std::iter::once(ExpandOutputAssetsInput::Asset(bootstrap_resolved)),
true,
)
.await?;
let mut chunks = Vec::new();
for asset in all_assets {
let asset_path = asset.path().owned().await?;
if !asset_path.is_inside_ref(&output_root) {
continue;
}
let Some(rel_path) = output_root.get_path_to(&asset_path) else {
continue;
};
// Skip source map files
if rel_path.ends_with(".map") {
continue;
}
let content = asset.content().await?;
let AssetContent::File(file_vc) = *content else {
continue;
};
let file_content = file_vc.await?;
let FileContent::Content(file) = &*file_content else {
continue;
};
if rel_path.ends_with(".js") {
// JavaScript chunk — send as text
let code: RcStr = file.content().to_str()?.into_owned().into();
chunks.push(ImportModuleChunk {
path: rel_path.into(),
code: Some(code),
binary: None,
source_map: None,
});
} else {
// Binary asset (wasm, images, etc.) — send base64-encoded
let bytes = file.content().to_bytes();
let encoded = base64::engine::general_purpose::STANDARD.encode(&*bytes);
chunks.push(ImportModuleChunk {
path: rel_path.into(),
code: None,
binary: Some(encoded),
source_map: None,
});
}
}
let entry_rel = output_root
.get_path_to(&entry_path)
.context("entry path should be inside output root")?;
Ok(ResponseMessage::ImportModule {
entry_path: entry_rel.into(),
chunks,
})
}
}
}
+12 -4
View File
@@ -743,7 +743,10 @@ async fn process_default_internal(
.await?;
let transforms = Vc::<SourceTransforms>::cell(vec![ResolvedVc::upcast(webpack_loaders)]);
current_source = transforms.transform(*current_source).to_resolved().await?;
current_source = transforms
.transform(*current_source, Vc::upcast(module_asset_context))
.to_resolved()
.await?;
// If turbopackModuleType is specified, skip rule matching and directly
// apply the requested module type with empty transforms (loader output
@@ -773,7 +776,10 @@ async fn process_default_internal(
.await;
}
ModuleRuleEffect::SourceTransforms(transforms) => {
current_source = transforms.transform(*current_source).to_resolved().await?;
current_source = transforms
.transform(*current_source, Vc::upcast(module_asset_context))
.to_resolved()
.await?;
// Fall through to re-process with new ident
}
_ => bail!("Unexpected module rule effect for turbopackModuleType"),
@@ -829,8 +835,10 @@ async fn process_default_internal(
return Ok(ProcessResult::Ignore.cell());
}
ModuleRuleEffect::SourceTransforms(transforms) => {
current_source =
transforms.transform(*current_source).to_resolved().await?;
current_source = transforms
.transform(*current_source, Vc::upcast(module_asset_context))
.to_resolved()
.await?;
if current_source.ident().to_resolved().await? != ident {
// The ident has been changed, so we need to apply new rules.
if let Some(transition) = module_asset_context