mirror of
https://github.com/hanzoai/wasm2go.git
synced 2026-10-30 15:06:48 +00:00
32841813d9
* spin guard: yield on every cold call when agents outnumber processors A wasm thread is a goroutine, and the spin guard's cold call yielded its processor once per 64 calls: enough to keep stop-the-world pauses bounded and cheap when every agent has a processor of its own. With more agents than GOMAXPROCS it is the wrong rate. A barrier the guest spins on releases only when every agent has arrived, the missing agents are exactly the ones waiting for a processor, and the spinners hold those processors for milliseconds between yields. On a 4-core runner an engine with 8 threads decoded at 1/100 of its single-thread rate; native ggml under an OpenMP barrier loses little in the same setting. threadLaunch now maintains a live spawned-agent gauge, and spinRelax yields on every cold call while that gauge reaches GOMAXPROCS (the spawner itself is the extra goroutine). Below that the 1/64 rate is unchanged, so the uncontended path keeps its measured parity. Qwen2.5-0.5B Q8_0 on a 10-core M-series host, decode tok/s (old -> new): 12 threads 8.3 -> 87.7, 16 threads 3.5 -> 57.2, 32 threads 0.5 -> 25.7; prompt at 32 threads 220 -> 1312; 1 thread unchanged. Co-Authored-By: Claude Fable 5.1 <[email protected]> * spin guard: yield rarely when every agent has a processor One yield per 64 cold calls kept donation proportional to spin time, but on an uncontended box every yield wakes an idle scheduler thread (wakep, pthread_cond_signal) that spins in findRunnable and parks again. Profiled at n_threads = 4 on a 10-core host that churn was 60% of a decode step's CPU samples, and on a 4-core host it takes cores from the workers themselves. With every agent on its own processor there is nobody to donate the core to: the uncontended rate is now one yield per 1024 cold calls (tens of milliseconds of spinning, the granularity sysmon preempts at anyway); oversubscription still yields on every cold call. Co-Authored-By: Claude Fable 5.1 <[email protected]> --------- Co-authored-by: Claude Fable 5.1 <[email protected]>
16 lines
648 B
Go
16 lines
648 B
Go
package helpers
|
|
|
|
// spinRelaxColdCalls mirrors the declaration in the wasip1_native.go
|
|
// runtime template so this package compiles (and its tests run) on its
|
|
// own. Only helpers.go is embedded and extracted into generated
|
|
// output; there the generated base package supplies the counter from
|
|
// the template, so the two declarations never meet.
|
|
var spinRelaxColdCalls uint32
|
|
|
|
// spinAgents mirrors the runtime template's live spawned-agent gauge
|
|
// (threadLaunch maintains it, spinRelax reads it), for the same reason.
|
|
var spinAgents int32
|
|
|
|
// spinOversubscribed mirrors the template's flag derived from spinAgents.
|
|
var spinOversubscribed uint32
|