mirror of
https://github.com/hanzoai/wasm2go.git
synced 2026-10-30 15:06:48 +00:00
b7d0e54805
* codegen: add base.AccessMemory — synchronised out-of-band memory window
Hosts sometimes need to write a word into guest linear memory from a
goroutine OTHER than the one executing the module — go-python's
Interrupter is the motivating case: it raises CPython's eval-breaker
bit from a watchdog while an Eval is busy in an infinite loop. Today
there is no sound way to do that: memory.grow mutates the memory
slice header without synchronisation and, when the spare capacity
runs out, relocates the whole backing array — so an out-of-band
writer either races the header read (go test -race flags it) or
lands its write in an abandoned array mid-relocation and the signal
is silently lost. go-python's first fix attempt worked around this
with a snapshot + //go:norace + bounded retry loop; the retry/polling
shape was rejected in review, and rightly so — the runtime owns the
relocation, so the runtime is where the synchronisation belongs.
This adds the primitive at that layer:
- Module gains a `memMu sync.Mutex`, declared LAST in the struct so
the memory/maxMem/M offsets the generated asm hardcodes
(moduleMOffset = 32) are untouched; the existing offset pin tests
guard that.
- memoryGrow takes memMu for its whole body — both the reslice
header rewrite and the relocating copy. Ordinary guest loads and
stores never touch the lock; the cost is one uncontended
lock/unlock per memory.grow call, on a path that already does
bounds math and possibly a full copy.
- New helper `accessMemory(m, f func(mem []byte))` runs f under the
same lock with the CURRENT memory slice: for the duration of f
the memory can neither be resliced nor relocated, so f's writes
land in the array the guest observes, and a grow that raced in
just before will copy them forward with the rest of the contents.
Deterministic — no snapshots, no retries, no polling; the only
wait is bounded by an in-flight grow. Emitted for every module
that has a linear memory (nothing generated calls it; it is
host-facing API, exported as base.AccessMemory in multi-package
mode).
rewriteHelperNode learns *ast.DeferStmt — the new helpers are the
first ones using defer, and the multi-package field-capitalisation
pass previously passed defer statements through unrewritten.
Verification (all exit 0):
- go test ./... (host amd64)
- GOARCH=arm64 go test ./... (native arm64)
- make lint (0 issues)
- go test -race ./internal/codegen/helpers/ — includes the new
TestAccessMemoryConcurrentGrow, which hammers memoryGrow
(alternating reslice and relocating grows) against a concurrent
accessMemory writer and asserts the final write survived every
relocation.
- Emission pin tests: modules with memory emit
accessMemory/memMu/locked grow with memMu AFTER M; memory-less
modules emit none of it.
- Consumer verification per docs/consumer-verification.md:
regenerated googlesql + python bundles; googlesqlite full
`go test -race ./...` on amd64 and GOARCH=arm64; go-python full
matrix (amd64/arm64 × race on/off) against a Fire reworked on
top of base.AccessMemory — the -race interrupt test that
motivated all of this passes with the deterministic
implementation (count=5, both arches).
Co-Authored-By: Claude Fable 5 <[email protected]>
* codegen: register the sync import before the multi-package base snapshot
CI caught non-WASI multi-package modules failing to build: base/base.go
used sync.Mutex (the new memMu field) with no "sync" import. The
multi-package path snapshots base's import set AFTER emitHelpers but
BEFORE emitModuleStruct runs, so the use("sync") inside the struct
emitter came too late — it only worked for WASI modules by accident
(the wasip1-native decls already pull sync into the snapshot). The
WASI-less matrix fixtures (vtable_dispatch, recursive_init, the data
sidecar test) had no such cover.
Register the import at the same early point that registers the
accessMemory helper. The struct-emitter use() stays for the
single-file path.
Verification (all exit 0, checked by exit code):
- make test-cover (the exact CI target; 85.8% >= 85% gate)
- go test ./... (host amd64) and GOARCH=arm64 go test ./...
- make lint (0 issues)
Co-Authored-By: Claude Fable 5 <[email protected]>
---------
Co-authored-by: Claude Fable 5 <[email protected]>
64 lines
2.2 KiB
Go
64 lines
2.2 KiB
Go
package codegen_test
|
|
|
|
import (
|
|
"bytes"
|
|
"strings"
|
|
"testing"
|
|
|
|
"github.com/goccy/wasm2go/internal/codegen"
|
|
"github.com/goccy/wasm2go/internal/testfixture"
|
|
"github.com/goccy/wasm2go/internal/wasm"
|
|
)
|
|
|
|
// TestAccessMemoryEmitted pins the host-facing synchronised memory
|
|
// window: every module WITH a linear memory must emit the accessMemory
|
|
// helper, the memMu field it locks, and a memoryGrow that takes the
|
|
// same lock — that trio is what makes out-of-band writers (e.g. an
|
|
// interrupt watchdog) sound against concurrent grows. The memMu field
|
|
// must sit AFTER the M field so the memory/maxMem/M offsets the asm
|
|
// hardcodes (moduleMOffset) stay put.
|
|
func TestAccessMemoryEmitted(t *testing.T) {
|
|
bin := testfixture.Wasm(t, "cg_memops")
|
|
mod, err := wasm.Parse(bytes.NewReader(bin))
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
var buf bytes.Buffer
|
|
if _, err := codegen.Translate(&buf, mod, codegen.Options{Package: "memmod", OutputImportPath: "gentest/memmod"}); err != nil {
|
|
t.Fatalf("translate: %v", err)
|
|
}
|
|
src := buf.String()
|
|
for _, expect := range []string{
|
|
"func accessMemory(m *Module, f func(mem []byte))",
|
|
// gofmt aligns struct fields, so match name and type separately.
|
|
"memMu",
|
|
"sync.Mutex",
|
|
"m.memMu.Lock()",
|
|
} {
|
|
if !strings.Contains(src, expect) {
|
|
t.Errorf("output missing %q", expect)
|
|
}
|
|
}
|
|
if mIdx, muIdx := strings.Index(src, "M unsafe.Pointer"), strings.Index(src, "memMu"); mIdx >= 0 && muIdx >= 0 && muIdx < mIdx {
|
|
t.Errorf("memMu declared before M — the moduleMOffset layout contract requires new fields AFTER M")
|
|
}
|
|
}
|
|
|
|
// TestAccessMemoryNotEmittedWithoutMemory: a memory-less module must
|
|
// not pull the helper (its body references memory fields the struct
|
|
// would not have).
|
|
func TestAccessMemoryNotEmittedWithoutMemory(t *testing.T) {
|
|
bin := testfixture.Wasm(t, "arith")
|
|
mod, err := wasm.Parse(bytes.NewReader(bin))
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
var buf bytes.Buffer
|
|
if _, err := codegen.Translate(&buf, mod, codegen.Options{Package: "arithmod", OutputImportPath: "gentest/arithmod"}); err != nil {
|
|
t.Fatalf("translate: %v", err)
|
|
}
|
|
if src := buf.String(); strings.Contains(src, "accessMemory") {
|
|
t.Errorf("memory-less module emitted accessMemory")
|
|
}
|
|
}
|