Tobias KoppersGitHubv-work-app[bot] <262237222+v-work-app[bot]@users.noreply.github.com>ClaudeTobias Koppers### What? Make the aggregation `optimize_queue` in the Turbopack persistent backend bounded and lazy. Cap the in-memory queue size, persist an `optimization_pending` flag per task, drain the queue per `process()` call with a per-queue lifetime budget, and recover dropped optimizations opportunistically via the flag instead of via an unbounded scheduler-side queue. ### Why? The previous implementation pushed an `OptimizeJob` for every `push_optimize_task` call into a single in-memory queue with no upper bound. On large workloads (or pathological aggregation churn), this queue could grow very large and cause the thread that scheduled the optimization to do unbounded work, regressing latency for the operations that triggered the schedule. The goals of this change: - Bound the worst-case work any single `process()` call does for optimizations (per-queue budget). - Bound the in-memory queue size so memory use is predictable. - Avoid losing optimizations: anything we drop must be eventually recovered. - Keep the common fast path cheap — no extra `Meta`-category guard acquisitions when the optimization flows through normally. ### How? Persist a new `optimization_pending` flag on `TaskStorage` (`storage_schema.rs`) and use it to drive lazy recovery in `aggregation_update.rs`: - `push_optimize_task` only enqueues an in-memory `OptimizeJob` if the queue is under `MAX_OPTIMIZE_QUEUE_SIZE` (10000) and the per-queue lifetime budget `MAX_OPTIMIZATIONS_PER_QUEUE` (1000) hasn't been exhausted. If we can't enqueue, we set `optimization_pending = true` on the task so a future operation that visits this task will re-discover and re-enqueue the optimization. - Every `AggregationUpdateJob` handler calls `check_optimization_pending` on the primary task(s) it touches, which re-enqueues the optimize job if the flag is set (and the queue/budget allow). - `process()` drains the `optimize_queue` one job at a time (preserving the original "root first" ordering), counting against the per-queue budget. Once the budget is exhausted, further `OptimizeJob`s in the queue are dropped and the flag is left set on those tasks (so they recover later). - `optimize_task` clears `optimization_pending` at entry so the recovery loop eventually settles. - The flag is **only** written on the drop path — the common case (enqueue → process normally) does not touch `optimization_pending`, so no `Meta`-category guard contention is added on the hot path. - `OptimizeJob` carries a best-effort `flag_already_set` snapshot so that when a job is dropped at process time and the snapshot says the flag was already set, we skip the redundant write entirely. Most jobs originating from `check_optimization_pending` (the recovery path) and `optimize_task`'s self-re-enqueue carry this hint. - `try_enqueue_optimize_job` is `#[must_use]` so the contract \"if this returns false, set the flag\" is enforced at the type level. `lock_and_mark_optimization_pending` is shared between `push_optimize_task_by_id` and the budget-exhausted drop branch. - `optimizations_executed` is intentionally persisted with the queue so that suspending and resuming the queue cannot reset the per-queue budget. <!-- NEXT_JS_LLM_PR --> --------- Co-authored-by: v-work-app[bot] <262237222+v-work-app[bot]@users.noreply.github.com> Co-authored-by: Claude <[email protected]> Co-authored-by: Tobias Koppers <[email protected]>
Next.js
Getting Started
Used by some of the world's largest companies, Next.js enables you to create full-stack web applications by extending the latest React features, and integrating powerful Rust-based JavaScript tooling for the fastest builds.
- Visit our Learn Next.js course to get started with Next.js.
- Visit the Next.js Showcase to see more sites built with Next.js.
Documentation
Visit https://nextjs.org/docs to view the full documentation.
Community
The Next.js community can be found on GitHub Discussions where you can ask questions, voice ideas, and share your projects with other people.
To chat with other community members you can join the Next.js Discord server.
Do note that our Code of Conduct applies to all Next.js community channels. Users are highly encouraged to read and adhere to it to avoid repercussions.
Contributing
Contributions to Next.js are welcome and highly appreciated. However, before you jump right into it, we would like you to review our Contribution Guidelines to make sure you have a smooth experience contributing to Next.js.
Good First Issues:
We have a list of good first issues that contain bugs that have a relatively limited scope. This is a great place for newcomers and beginners alike to get started, gain experience, and get familiar with our contribution process.
Security
If you believe you have found a security vulnerability in Next.js, we encourage you to responsibly disclose this and NOT open a public issue.
To participate in our Open Source Software Bug Bounty program, please email [email protected]. We will add you to the program and provide further instructions for submitting your report.