mirror of
https://github.com/hanzoai/next.js.git
synced 2026-10-30 14:46:38 +00:00
43f9017719
### What? Adds a \`next internal post-build\` CLI command that performs Turbopack persistent cache database compaction as a separate step after \`next build\`. ### Why? Database compaction during shutdown can be slow and blocks the build process. By deferring compaction to a separate post-build step, we can: - Keep the main \`next build\` fast by skipping compaction at shutdown (controlled via \`NEXT_USE_POST_BUILD=1\`) - Run compaction independently, e.g. as a separate CI step or in the background - Perform a full compaction (\`max_merge_segment_count: usize::MAX\`) that is more thorough than the runtime shutdown compaction **Intended usage in a build pipeline:** \`\`\`sh NEXT_USE_POST_BUILD=1 next build # fast build, skips compaction at shutdown next internal post-build # separate step: full compaction of the cache \`\`\` ### How? **Rust side:** - Added \`compact_database()\` to \`turbo-tasks-backend\` that opens the database and runs a full compaction using \`TurboTasksParallelScheduler\` backed by a dedicated multi-threaded Tokio runtime (same scheduler as the normal runtime path, enabling parallel work-stealing) - Added \`turbopack_database_compact\` NAPI binding that resolves the versioned DB path and calls \`compact_database()\` - Added \`skip_compaction\` flag to \`NapiTurboEngineOptions\` / \`TurboKeyValueDatabase\` to skip compaction during normal shutdown when post-build will handle it; asserted that \`skip_compaction=true\` requires \`is_short_session=true\` (enforcing the logical dependency) - Extracted shared \`DB_CONFIG\`, \`FAMILIES\`, and \`COMPACT_CONFIG\` constants and \`git_version_info()\` helper to avoid duplication between the runtime DB and standalone compaction paths **TypeScript side:** - Added \`next internal post-build\` command that loads the Next.js config, finds the Turbopack cache directory, and calls the native compaction binding - Added \`databaseCompact\` to the turbo bindings interface (native + WASM stub) - \`turbopack-build/impl.ts\` sets \`skipCompaction: true\` when \`NEXT_USE_POST_BUILD=1\` **Tests:** - Added \`test/production/app-dir/post-build/\` integration test that verifies compaction runs successfully on a persistent-cache build and reports nothing to do when persistent caching is disabled --------- Co-authored-by: Tobias Koppers <[email protected]> Co-authored-by: Claude <[email protected]>
29 lines
793 B
Plaintext
29 lines
793 B
Plaintext
|name, initial| {
|
|
use std::hash::BuildHasher;
|
|
let path = std::path::PathBuf::from(
|
|
format!("{}/.cache/{}", env!("CARGO_TARGET_TMPDIR"), rustc_hash::FxBuildHasher.hash_one(name)));
|
|
if initial {
|
|
let _ = std::fs::remove_dir_all(&path);
|
|
}
|
|
std::fs::create_dir_all(&path).unwrap();
|
|
turbo_tasks::TurboTasks::new(
|
|
turbo_tasks_backend::TurboTasksBackend::new(
|
|
turbo_tasks_backend::BackendOptions {
|
|
num_workers: Some(2),
|
|
small_preallocation: true,
|
|
..Default::default()
|
|
},
|
|
turbo_tasks_backend::turbo_backing_storage(
|
|
path.as_path(),
|
|
&turbo_tasks_backend::GitVersionInfo {
|
|
describe: "test-unversioned",
|
|
dirty: false,
|
|
},
|
|
false,
|
|
true,
|
|
false,
|
|
).unwrap().0
|
|
)
|
|
)
|
|
}
|