mirror of
https://github.com/hanzoai/next.js.git
synced 2026-10-30 14:46:38 +00:00
Luke SandbergandGitHub
d069c2e163
## What
Tightens the value-type persistence API and sets the table for a future eviction policy. Two user-visible changes on the `#[turbo_tasks::value(...)]` macro:
- **`serialization = "none"` → `serialization = "skip"`** — imperative ("skip persisting") instead of descriptive. Making it clear that it isn't that we are missing a feature but rather that we are choosing to not persist (of course persisting might be impossible but that is generally rare)
- **New `evict = "always" | "last" | "never"` attribute** — replaces the old overloaded `"none"` semantic. Only valid with `serialization = "skip"`. Defaults to `"always"`.
Internally this collapses `persistent_cell_data` + `transient_cell_data` into one `cell_data` map and replaces the old `bincode: Option<(enc, dec)>` field with a four-variant `ValueTypePersistence` enum. Eviction machinery itself is a follow-up PR; this PR just gives each value type a precise, queryable persistence/eviction descriptor.
## Why break out a new parameter?
`serialization = "none"` on canary conflated three different intents:
1. **Cheap recomputable outputs** (SWC ASTs, codegen `Rope`s) — fine to evict, recompute is cheap.
2. **Expensive recomputable outputs** (WASM modules, Node process pools) — re-derivable but costly.
3. **Session-scoped state** (`State<>` cells, `Arc<Mutex<_>>` dedup histories) — can't be recomputed without losing accumulated mutations.
They all produced identical runtime behavior (stored in transient_cell_data), so eviction can't tell them apart. The fix is two orthogonal attributes:
```rust
// A cheap skip — default evict = "always"
#[turbo_tasks::value(serialization = "skip")]
// Expensive recompute — evict last under pressure
#[turbo_tasks::value(serialization = "skip", evict = "last")]
// Session-scoped state — never evict
#[turbo_tasks::value(serialization = "skip", evict = "never")]
```
The macro rejects `evict` on any other `serialization` mode.
## `ValueTypePersistence` enum
Replaces `ValueType.bincode: Option<(enc, dec)>`:
```rust
pub enum ValueTypePersistence {
Persistable(AnyEncodeFn, AnyDecodeFn<SharedReference>), // auto, custom
SkipPersist { expensive: bool }, // skip (+ evict = last)
HashOnly, // hash
SessionStateful, // skip + evict = never
}
```
The existing `"hash"` mode gets its own `HashOnly` variant rather than being folded into `SkipPersist`, which lets the backend gate its hash-writing and hash-comparison paths precisely.
## Unified `cell_data` storage
`persistent_cell_data: AutoMap<CellId, TypedSharedReference>` + `transient_cell_data: AutoMap<CellId, SharedReference>` collapse into `cell_data: CellData`. `CellData` is a newtype over `AutoMap<CellId, SharedReference>` with a custom bincode impl that filters non-`Persistable` entries at encode time. This removes the `is_serializable_cell_content: bool` parameter that was threading through ~14 read/write call sites.
Uses `SharedReference` instead of `TypedSharedReference` — `CellId` already carries the `ValueTypeId`.
## Annotation sweep
All prior `serialization = "none"` sites move to either `serialization = "skip", evict = "never"` or `serialization = "skip", evict = "last"` based on a per-site audit. Summary:
**`evict = "last"` (6 sites)** — re-derivable but expensive:
- `SwcPluginModule`, `EvaluatePool`, `ChildProcessPool`, `WorkerThreadPool`, `EffectInstance`, `Effects`
**`evict = "never"` (2 sites)** — interior-mutable state accumulated across the session:
- `ConsoleUi` (`Arc<Mutex<SeenIssues>>`), `VersionState` (`State<VersionRef>` with HMR invalidators)
The distinguishing rule: `evict = "never"` only when the value holds interior mutability accumulated across the session. Everything else can be re-derived (possibly expensively) by re-running the producing task.
## Follow-ups (separate PRs)
- Wire an eviction policy that consumes `ValueTypePersistence` — respects `SessionStateful` (never evict), prefers cheap `SkipPersist` over `expensive: true` ones. Either as part of #91790 or afterwards depending on when things land
<!-- NEXT_JS_LLM_PR -->
95 lines
2.9 KiB
Rust
95 lines
2.9 KiB
Rust
#![feature(arbitrary_self_types)]
|
|
#![feature(arbitrary_self_types_pointers)]
|
|
#![allow(clippy::needless_return)] // clippy bug causes false positive
|
|
|
|
use std::{collections::HashSet, mem::take, sync::Mutex};
|
|
|
|
use anyhow::Result;
|
|
use turbo_tasks::{
|
|
Invalidator, ReadRef, Vc, get_invalidator,
|
|
unmark_top_level_task_may_leak_eventually_consistent_state, with_turbo_tasks,
|
|
};
|
|
use turbo_tasks_testing::{Registration, register, run_once};
|
|
|
|
static REGISTRATION: Registration = register!();
|
|
|
|
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
|
async fn test_read_ref() {
|
|
run_once(®ISTRATION, || async {
|
|
unmark_top_level_task_may_leak_eventually_consistent_state();
|
|
let counter = Counter::cell(Counter {
|
|
value: Mutex::new((0, Default::default())),
|
|
});
|
|
|
|
let counter_value = counter.get_value();
|
|
|
|
assert_eq!(*counter.get_value().strongly_consistent().await?, 0);
|
|
assert_eq!(*counter_value.strongly_consistent().await?, 0);
|
|
|
|
counter.await?.incr();
|
|
|
|
assert_eq!(*counter.get_value().strongly_consistent().await?, 1);
|
|
assert_eq!(*counter_value.strongly_consistent().await?, 1);
|
|
|
|
// `ref_counter` will still point to the same `counter` instance as `counter`.
|
|
let ref_counter = ReadRef::cell(counter.await?);
|
|
let ref_counter_value = ref_counter.get_value();
|
|
|
|
// However, `local_counter_value` will point to the value of `counter_value`
|
|
// at the time it was turned into a trait reference (just like a `ReadRef`
|
|
// would).
|
|
let local_counter_value = ReadRef::cell(counter_value.await?).get_value();
|
|
|
|
counter.await?.incr();
|
|
|
|
assert_eq!(*counter.get_value().strongly_consistent().await?, 2);
|
|
assert_eq!(*counter_value.strongly_consistent().await?, 2);
|
|
assert_eq!(*ref_counter_value.strongly_consistent().await?, 2);
|
|
assert_eq!(*local_counter_value.strongly_consistent().await?, 1);
|
|
|
|
anyhow::Ok(())
|
|
})
|
|
.await
|
|
.unwrap()
|
|
}
|
|
|
|
#[turbo_tasks::value(transparent)]
|
|
struct CounterValue(usize);
|
|
|
|
#[turbo_tasks::value(serialization = "skip", evict = "never", cell = "new", eq = "manual")]
|
|
struct Counter {
|
|
#[turbo_tasks(debug_ignore, trace_ignore)]
|
|
value: Mutex<(usize, HashSet<Invalidator>)>,
|
|
}
|
|
|
|
impl Counter {
|
|
fn incr(&self) {
|
|
with_turbo_tasks(|tt| {
|
|
let mut lock = self.value.lock().unwrap();
|
|
lock.0 += 1;
|
|
let invalidators = take(&mut lock.1);
|
|
for i in invalidators {
|
|
i.invalidate(&**tt);
|
|
}
|
|
});
|
|
}
|
|
}
|
|
|
|
#[turbo_tasks::value_impl]
|
|
impl Counter {
|
|
#[turbo_tasks::function]
|
|
fn get_value(&self) -> Result<Vc<CounterValue>> {
|
|
let mut lock = self.value.lock().unwrap();
|
|
lock.1.insert(get_invalidator().unwrap());
|
|
Ok(Vc::cell(lock.0))
|
|
}
|
|
}
|
|
|
|
#[turbo_tasks::value_impl]
|
|
impl CounterValue {
|
|
#[turbo_tasks::function]
|
|
fn get_value(self: Vc<Self>) -> Vc<Self> {
|
|
self
|
|
}
|
|
}
|