mirror of
https://github.com/hanzoai/wasm2go.git
synced 2026-10-30 15:06:48 +00:00
32841813d9
* spin guard: yield on every cold call when agents outnumber processors A wasm thread is a goroutine, and the spin guard's cold call yielded its processor once per 64 calls: enough to keep stop-the-world pauses bounded and cheap when every agent has a processor of its own. With more agents than GOMAXPROCS it is the wrong rate. A barrier the guest spins on releases only when every agent has arrived, the missing agents are exactly the ones waiting for a processor, and the spinners hold those processors for milliseconds between yields. On a 4-core runner an engine with 8 threads decoded at 1/100 of its single-thread rate; native ggml under an OpenMP barrier loses little in the same setting. threadLaunch now maintains a live spawned-agent gauge, and spinRelax yields on every cold call while that gauge reaches GOMAXPROCS (the spawner itself is the extra goroutine). Below that the 1/64 rate is unchanged, so the uncontended path keeps its measured parity. Qwen2.5-0.5B Q8_0 on a 10-core M-series host, decode tok/s (old -> new): 12 threads 8.3 -> 87.7, 16 threads 3.5 -> 57.2, 32 threads 0.5 -> 25.7; prompt at 32 threads 220 -> 1312; 1 thread unchanged. Co-Authored-By: Claude Fable 5.1 <[email protected]> * spin guard: yield rarely when every agent has a processor One yield per 64 cold calls kept donation proportional to spin time, but on an uncontended box every yield wakes an idle scheduler thread (wakep, pthread_cond_signal) that spins in findRunnable and parks again. Profiled at n_threads = 4 on a 10-core host that churn was 60% of a decode step's CPU samples, and on a 4-core host it takes cores from the workers themselves. With every agent on its own processor there is nobody to donate the core to: the uncontended rate is now one yield per 1024 cold calls (tens of milliseconds of spinning, the granularity sysmon preempts at anyway); oversubscription still yields on every cold call. Co-Authored-By: Claude Fable 5.1 <[email protected]> --------- Co-authored-by: Claude Fable 5.1 <[email protected]>
42 lines
1.1 KiB
Go
42 lines
1.1 KiB
Go
package helpers
|
|
|
|
import (
|
|
"runtime"
|
|
"sync/atomic"
|
|
"testing"
|
|
)
|
|
|
|
// TestSpinRelax exercises the cold half of the spin-loop preemption
|
|
// guard: reaching runtime.Gosched must never block the caller, only
|
|
// donate the core.
|
|
func TestSpinRelax(t *testing.T) {
|
|
for i := 0; i < 64; i++ {
|
|
spinRelax()
|
|
}
|
|
}
|
|
|
|
// TestSpinRelaxOversubscribed: with more live agents than processors
|
|
// every cold call yields (the barrier-latency fix); the call still
|
|
// returns promptly and leaves the gauge untouched.
|
|
func TestSpinRelaxOversubscribed(t *testing.T) {
|
|
procs := int32(runtime.GOMAXPROCS(0))
|
|
spinAgentsAdd(procs - 1)
|
|
if atomic.LoadUint32(&spinOversubscribed) != 0 {
|
|
t.Fatalf("%d agents on %d procs flagged as oversubscribed", procs-1, procs)
|
|
}
|
|
spinAgentsAdd(1)
|
|
if atomic.LoadUint32(&spinOversubscribed) == 0 {
|
|
t.Fatalf("%d agents on %d procs not flagged", procs, procs)
|
|
}
|
|
for i := 0; i < 64; i++ {
|
|
spinRelax()
|
|
}
|
|
spinAgentsAdd(-procs)
|
|
if got := atomic.LoadInt32(&spinAgents); got != 0 {
|
|
t.Errorf("spinAgents = %d after every agent left", got)
|
|
}
|
|
if atomic.LoadUint32(&spinOversubscribed) != 0 {
|
|
t.Error("still flagged with no agents")
|
|
}
|
|
}
|