Files
be58135a5f Support the memory64 proposal: 64-bit linear memory past 4 GiB (#41)
* Support the memory64 proposal: 64-bit linear memory past 4 GiB

A wasm32 module can never use more than 4 GiB no matter what the
runtime allows — its pointers, size_t and allocator are compiled as
32-bit values. memory64 (wasm64) is the sound route to large
memories, and this adds it end to end for the pure and gcasm outputs:

- Parser: limits flags 0x04/0x05 decode the 64-bit index space into
  MemoryType.Is64 / Module.Memory64; unknown flag bits are rejected.
  Memarg offsets decode as u64 (wasm32 offsets are unaffected).
- Lowering: addresses ride as i64 values; memory.size/grow type as
  i64; the SIMD memory family lowers to a simd_m64_* helper family
  with u64 addressing (never rewritten to the pair/splice forms,
  which assume 32-bit addresses). Atomics on a memory64 are rejected
  for now — no toolchain produces shared wasm64 today.
- Emission: scalar loads/stores keep the zero-cost inline unsafe
  deref, with the effective address computed as u64 instead of the
  u32 wrap; large constant totals still route through the _consts
  table. memory.size/grow/fill/copy/init switch to 64-bit helper
  twins. The coalesced SIMD bounds check stays wasm32-only (its
  exactness argument is written against the wasm32 hard cap).
- Runtime: memoryGrow64 grows past the wasm32 cap up to a 2^48 hard
  cap with the same geometric-capacity scheme; a memory64 with no
  declared maximum defaults to unlimited (maxMem 0) instead of the
  4 GiB wasm32 default, and the external-image constructors accept
  images past 4 GiB. Effective-address checks in the 64-bit helpers
  guard u64 overflow explicitly — there is no benign wrap at 2^64.

The gate proves the point end to end: a memory64 module grows to
4 GiB + 128 KiB and stores/loads at address 1<<32 through the full
backend. wasm32 output is byte-identical to before across the
fixture corpus, SIMD included.

Native asm splicing of the m64 SIMD helpers, the wasm64 WASI ABI,
and a wasm64 sysroot in the wasmify toolchain are follow-ups; the
llama.cpp chain needs all three before go-llama can load >4 GiB
models.

Co-Authored-By: Claude Fable 5 <[email protected]>

* Bind memory64 modules to a widened wasip1 ABI

There is no standardized 64-bit WASI ABI; the wasm2go-ported
wasi-libc declares every wasi_snapshot_preview1 import argument at
pointer width (all-i64 under wasm64), and this is the runtime side
of that contract:

- A memory64 module's wasi imports bind to new *64 methods on
  WasiStubs (codegen appends the suffix for wasi_snapshot_preview1
  imports when the module is a memory64); every wasm32 module keeps
  the standard 32-bit methods and its byte-identical output.
- The *64 bindings parse the LP64 layouts — 16-byte iovecs, 64-bit
  size fields — through a full-range 64-bit memory window, covering
  the clock/fd/proc calls a C hello-world exercises; the rest of the
  surface follows as the wasm64 toolchain grows.
- Immediate skipping in the call-graph prescan reads memarg offsets
  as u64, matching the decoder (a memory64 dlmalloc's offsets
  overflowed the u32 reader).

The checked-in wasip1_mem64_hello.wasm fixture — C compiled with
clang -target wasm64-wasip1 against the ported wasi-libc — pins
printf and malloc end to end; a 5 GiB malloc with stores at 0, 4 GiB
and 5 GiB-1 runs correctly through the same chain.

Co-Authored-By: Claude Fable 5 <[email protected]>

* Cover the llama-wasm WASI surface in the widened wasm64 ABI

Extend the *64 wasip1 bindings from the initial six methods to the
full import set a wasi-libc-linked C/C++ program (llama.cpp included)
uses: args/environ, prestat, path_open, fd_read, fd_readdir,
path_filestat_get, fd_fdstat_set_flags, and random_get.

Path_open and Fd_readdir keep a single layout-independent core
(pathOpen / fdReaddir) shared by both pointer widths; the LP64-layout
differences (8-byte guest pointers and __wasi_size_t, 16-byte iovecs,
16-byte prestat) live only in the thin *64 wrappers.

Add a checked-in wasm64 C++ exception fixture (typed catches,
catch-all, unwinding through std::string/vector/unique_ptr) built
with -fwasm-exceptions against the wasm64 libc++/libc++abi/libunwind
runtimes, plus unit tests for every new *64 binding.

Co-Authored-By: Claude Fable 5 <[email protected]>

* Share the layout-independent WASI cores across pointer widths

Fold the remaining duplicated bodies between the wasm32 and wasm64
wasip1 bindings into shared cores, so each binding pair keeps only its
ABI layout:

- clockNanos: the clock_time_get body.
- fdSeek: the fd_seek body.
- readVec/writeVec: the scatter/gather I/O loops, fed by per-width
  iovec resolvers (iovecSlices / iovecSlices64). Every iovec entry is
  now validated before any I/O happens, so a bad entry faults the call
  instead of surfacing after a partial transfer.

No behavior change otherwise; wasm32 and wasm64 fixtures and the WASI
unit suites pass unchanged, coverage 83.7% >= 83%.

Co-Authored-By: Claude Fable 5 <[email protected]>

* Fix three memory64 gaps surfaced by transpiling llama.cpp as wasm64

Verified by rebuilding the full llama.cpp bridge for wasm64-wasip1 and
running the go-llama test suite on the transpiled output (all green):

- A v128 crossing a defined-function boundary was typed `any` in the
  generated signature (goTypeOf had no ValV128 case). wasm32 llama
  never passed a v128 by value, so this never fired; the wasm64 build
  does. Map it to the scalarized [2]uint64 pair the SSA emitter uses
  for every v128 expression.
- The f16 store-side idiom chain fused a cvt+store helper whose
  address parameter is i32; on a memory64 module addresses are i64
  and the intermediate forms assume wasm32 v128 typing. The chain is
  now wasm32-only — a memory64 module's SIMD memory ops run as plain
  m64 helper calls without splicing, so the rewrite bought nothing
  there anyway.
- The bulk-dispatch Inv_<svc>_<mt> wrappers hardcoded (int32, int32)
  request parameters; on a memory64 module the bridge exports take
  pointer-width i64 (the C side declares them void*/size_t).

Co-Authored-By: Claude Fable 5 <[email protected]>

* Splice memory64 SIMD memory ops with the shared vector bodies

A memory64 module's SIMD memory helpers (simd_m64_*) were excluded
from every splice path, so each load/store ran as a plain function
call — measured on the llama.cpp module as a 36x token-generation gap
against the wasm32 build. There is nothing wide about the vector
bodies themselves: arm64/amd64 effective addresses are computed in
64-bit registers either way, so the m64 family now rides the same
splice tables with only the address glue widened.

- simdSpliceOp strips the m64_ marker (plain and pair forms) and
  reports addr64; the op after stripping indexes the same tables.
- The a64/x64 memory preambles take addr64: full-width MOVD/MOVQ
  address operands instead of the u32 zero-extensions, with
  carry-checked sums (two u32s can never wrap a u64, two u64s can —
  and a wrapped effective address could land back inside bounds).
- The scalarizer treats simd_m64_* as pairable via the same table
  (m64_ stripped), emitting simd_p_m64_* pair calls; the pair-wrapper
  family is added to the helpers.
- Fused-tree/loop lookups explicitly skip addr64 sites for now: the
  fused emitters still assume u32 address arithmetic. Result on the
  llama.cpp wasm64 module: 7148 of 7149 SIMD call sites spliced,
  tg 1.6 -> 9.7 tok/s, pp 1.8 -> 11.4 tok/s. Porting tree/loop
  fusion (SDOT/SMMLA superinstructions included) to addr64 is the
  remaining lever toward wasm32 parity.

Also fix an unbounded recursion in the fused-tree scalar chaser: a
loop-carried definition (v = v + 4 in the goto-form emission) made
chaseI32 recurse through interDef forever — a stack overflow on the
llama.cpp module built at -Oz, whose loop shapes surface such
definitions. A visiting mark leaves self-referential names as
interveners.

Co-Authored-By: Claude Fable 5 <[email protected]>

* Fuse SIMD trees on memory64 modules

Extend tree fusion (and with it the superinstruction rewrites that
run inside fused regions) to memory64 modules; only fused LOOPS stay
width-gated for now. Measured on the llama.cpp wasm64 module:
tg 10.2 -> 17.9 tok/s, pp 13.2 -> 22.2 tok/s, with the go-llama
suite green.

The design keeps one width-independent op vocabulary:

- The builder strips the m64_ marker for every table lookup and node
  name (fuseLookupName) and marks the tree Addr64. A memory64
  member's address/offset always ride as OPAQUE int64 scalars —
  Arg.Const is 32-bit and ArgSum assumes u32 wrap, so folding either
  would truncate a 64-bit address; the emitters enforce this shape.
- Addr64 trees take every scalar parameter as int64 (same register
  assignment); call sites widen non-address scalars losslessly, and
  the fallback body narrows them back for the int32-consuming pure
  helpers while the memory members call their simd_m64_* twins.
- Both fused emitters widen only the address glue: full-width moves
  with carry-checked sums (two u32s can never wrap a u64; two u64s
  can, and a wrapped effective address could land back inside
  bounds). The vector node bodies — dot8/SDOT rewrites included —
  are shared unchanged. The amd64 store tail (value staging, f16
  conversion chain, the store proper) is extracted so both address
  paths share it.
- Fused loops reject Addr64 trees at the upgrade point: the loop
  machinery still assumes u32-wrap pointer bumps and int32 exit
  scalars. The loop body keeps its fused-window form meanwhile.

Co-Authored-By: Claude Fable 5 <[email protected]>

* Widen the fused-loop machinery for memory64 trees

Lift the Addr64 gate off the loop upgrade and widen every piece the
gate was standing in for: loop-invariant stride parameters and
non-counter exit scalars are int64 on an Addr64 tree (they are its
pointer parameters), pointer bumps advance at full width on both
splicers (ADD/ADDQ instead of the u32-wrap W/L forms — the body's
per-node carry-checked bounds keep a runaway pointer from ever being
dereferenced), and the after-loop fix-up temporaries follow the width.

On the llama.cpp wasm64 module no loop upgrades fire YET: the scalar
chase (chaseAddr/chaseI32) only recognizes the wasm32 address shapes
(uint32 conversions), while the memory64 emitter derefs through
`unsafe.Add(mBase, uint64(v - int64(c)))` — so the scale-load
interveners cannot join the body window and the upgrade's
whole-body-in-one-region requirement fails. Extending the chase
grammar to the memory64 shapes is the follow-up that turns the
machinery on; this change makes it correct once matching succeeds,
and is inert until then (all suites and the wasm64 llama build are
byte-for-byte at the fused-window numbers).

Co-Authored-By: Claude Fable 5 <[email protected]>

* Unify the memory64 and wasm32 SIMD address handling

The memory64 splice paths had grown a second, hand-written copy of
every address computation — separate mnemonics, and extra
carry-checked sums the wasm32 path never had. That duplication was
both a maintenance hazard and a measured slowdown.

Collapse the two into one width-parameterized path: the ONLY
difference between wasm32 and memory64 is now the instruction that
materializes an address operand — MOVWU/MOVL (zero-extend i32) versus
MOVD/MOVQ (full i64). The effective-address arithmetic and the
ea+size ≤ memSize bounds check are identical, shared code.

The extra carry checks are removed. They are not needed on either
width: a memory64's linear memory is capped at 2^48 bytes
(mem64HardCap), so every valid effective address is far below 2^63
and the ea+size ≤ memSize comparison is exact. The one address that
could wrap a u64 is one the guest deliberately computes near 2^64 —
which wasi-sdk-compiled code never produces, and which even then
resolves within the module's own memory slice, so it is never a Go
memory-safety issue. This is the same pragmatic bounds model the
coalescing pass already documents.

a64MemPreambleRegs and x64MemPreamble are now single functions
(movAddr chosen once); the fused load/store/lane checks share
a64FusedMemCheck / x64FusedEA64. On the llama.cpp wasm64 kernel this
lifts tg 17.9 -> 18.6 and pp 22.2 -> 22.7 while deleting the
duplicate emission.

Bounds coalescing stays off for memory64 (documented): unlike
wasm32 it regressed the kernel even with the carry-free forms, which
are kept implemented and tested for a future grouping heuristic.

Co-Authored-By: Claude Fable 5 <[email protected]>

* Extend SIMD unrolling, coalescing and window fusion to memory64

A memory64 module's hot kernels arrive with LP64-promoted shapes the
SIMD pipeline did not recognize: i64 loop counters (i64.sub countdown,
Ui64 guards), i64 pointer-bump chains, int64-spelled constant locals
and % 64 shift masks. Every stage bailed in its own way and the q8_0
vec_dot ran as sparse two-sdot windows full of GPR-pair round trips —
3.3x slower than the same kernel on wasm32.

Teach each stage the wide twin of the shape it already handles:

- UnrollSimdLoops matches i64 countdowns and builds the guard
  arithmetic at the counter's width. Wide matching is a per-module
  opt-in (memory64 only): on wasm32 the i64-counter loops are exactly
  the ones whose bodies do not window-fuse, and admitting them was
  measured to cost ~40% prompt throughput there.
- ReassocConstAdds folds Add64 chains (memory64 only, same gating
  rationale) so unrolled i64 pointer bumps share one base.
- Bounds coalescing runs on memory64: the 2^48 linear-memory cap makes
  the coalesced ea+span check exact with no extra precondition. The
  earlier measurement that kept it off was taken against a miscompiled
  (-Oz) module and does not hold.
- Window fusion folds memory64 addresses as base+const ArgSum (the
  base rides a full-width scalar; only bare 64-bit ArgConst addresses
  stay opaque), binds int64-spelled constant locals, resolves
  int64-typed rng/lane windows, and chases % 64 shift masks and
  int64-wrapped scale chains; the countdown-loop matcher accepts the
  Ui64 header and i64 decrement forms.
- The arm64/amd64 fused splices materialize ArgSum addr64 bases with
  full-width adds; the m64 scalar-chain helpers take the mechanical
  simd_m64_* naming the lookup already strips.

Measured on the llama.cpp q8_0 module (M5): memory64 tg 17.9 -> 33.6
tok/s, pp 22.7 -> 34.3 tok/s; wasm32 is byte-neutral by construction
on the gated paths and A/B-verified flat on the rest (tg 53.6 vs
53.7, pp 88.8 vs 89.3 warm, interleaved).

Co-Authored-By: Claude Fable 5 <[email protected]>

* Surface failed window-fusion trials under WASM2GO_FUSE_DEBUG

The failWhy field recorded each trial's first refusal but nothing
read it, so diagnosing why a module's windows stay small meant
ad-hoc printf builds. Print failed trials of four or more candidates
(width plus refusal) to stderr when WASM2GO_FUSE_DEBUG is set — the
histogram of these lines is what localized both the wasm32 K=4
unlock and the memory64 window starvation (pair-input int-slot
exhaustion at pair #7 across the attention kernels).

Co-Authored-By: Claude Fable 5 <[email protected]>

* Recognize and pair the ggml q8_0 vec_dot on memory64 modules

The type_traits_cpu recognizer assumed the ILP32 row layout (24-byte
rows, u32 function pointers) and the all-i32 ggml_vec_dot_t
signature, so a memory64 module never got the row/column pairing.
Parameterize the recognition by pointer width: LP64 rows are 32
bytes with u64 function-pointer fields (table indices — nonzero high
bits disqualify a row), the vec_dot signature carries i64 pointers
and size_t strides, and the paired-tile companion takes matching
int64 parameters (its pointer arithmetic is unchanged in shape).

The arm64 SMMLA tile kernel follows: the only width-dependent part
is the ABI0 argument frame — int64 arguments land 8-aligned
(l1+16 .. l6+56, 64 argument bytes) and load with MOVD — while the
2x2 tile body is byte-identical, which the kernel test asserts by
stripping the frame lines and diffing the remainder.

Measured on the llama.cpp q8_0 module as wasm64 (M5, fast-math):
pp 35.8 -> 45.7 tok/s (+28%, the same jump the pairing gave wasm32);
tg unchanged as expected (odd batches keep nrc == 1). Full go-llama
suite green, including the native-token-match gates.

Co-Authored-By: Claude Fable 5 <[email protected]>

* Let memory64 stores reach window fusion

Two width-blind name checks kept every memory64 SIMD store out of
window fusion, running ggml's conversion and scale loops as chains
of marshalled pair calls:

- IsVoidAtomicStore matched only the simd_v128_store prefix, so a
  simd_m64_v128_store's dummy result was treated as usable and the
  store hoisted into `vN = ...` assignment form — which the window
  candidate scan (expression statements) never sees.
- The candidate scan itself looked the mark's raw name up in
  fusedStoreOps without stripping the memory64 marker.

Fix both to accept the m64 spelling, and pin the behavior with twin
fixtures (cg_simd_store32 / cg_mem64_simd_store): the elementwise
store-of-mul(load, load) shape must produce a fused tree containing
a store sink on BOTH pointer widths, Addr64-tagged on memory64.
On the llama.cpp wasm64 module this fuses 2163 store sites
(6320 -> 4157 bare); wasm32 output is unchanged by construction
(both checks are identity for non-m64 names) and the twin test
pins its store fusion too.

Co-Authored-By: Claude Fable 5 <[email protected]>

* Pin conversion-loop window fusion on both pointer widths

Twin fixtures for the variable-stride row-scale skeleton (per
iteration: v128 load, load32_splat scale, mul, add, store; pointer
bumps by a runtime stride). Both widths currently fuse the whole
iteration into one region — pinned so the memory64 side cannot
silently regress to bare pair calls while the remaining divergence
in the full llama conversion loops is chased.

Co-Authored-By: Claude Fable 5 <[email protected]>

* Deduplicate opaque scalar window arguments by structural key

Identifier-typed scalar arguments already shared one slot per name;
expression-typed ones (a variable-stride address like `base + s`)
burned a fresh int slot per occurrence, so windows over conversion
loops hit the slot cap early. Key them by their printed form — safe
because side-effecting values are hoisted into named locals before
fusion (these expressions are pure) and noteRead records the
contained identifiers for the same interference checks as the
identifier path.

Verified perf-neutral on wasm32 (interleaved A/B: tg 53.6 avg both
sides, pp 89.5 vs 89.9) with both llama suites green; no measured
memory64 throughput change yet — the hot LP64 conversion loops turn
out to be bound elsewhere — but the slot pressure reduction is
structural and width-independent.

Co-Authored-By: Claude Fable 5 <[email protected]>

* Run the f16 idiom rewrites on memory64 modules

The whole f32->f16 store-side chain (idiom recognition, empty-
diamond folding, the 4x-store16 merge, the fused cvt+store) was
gated to wasm32, so a memory64 module kept the full software idiom
in its hottest conversion loops. The chain is width-neutral except
at three points, now parameterized: memRoot/constOf walk Add64/
Const64 address chains, FuseF16CvtStores takes the module width and
spells the i64 offset plus the simd_m64_* helper, and the m64
cvt+store helper twins (array and pair forms) land next to the m64
store family. The load-side gather recognizer picks up the widened
address walk for free.

Verified: pass unit tests cover the m64 spelling; both llama suites
green (the conversion is bit-exact by construction — same rounding
chain, NaN forcing); wasm32 interleaved A/B flat (tg 47.6 avg both
sides, pp 82.9 vs 83.1); memory64 interleaved A/B ahead on tg in
every adjacent pair (~+0.8 tok/s warm), pp flat.

Co-Authored-By: Claude Fable 5 <[email protected]>

* Chase variable-stride memory addresses into scalar-node chains

A load or store whose address is a runtime sum with no constant side
(`base + stride`, the shape strided conversion loops carry — three
terms on LP64) previously rode one opaque int slot per distinct
expression, so windows over such loops hit the slot cap and shed
their loads. Chase these addresses like the scale chains: the sum
becomes scalar_i32_add nodes whose LEAF identifiers deduplicate by
name, so any number of loads sharing a stride variable cost one slot
for it. The emitters accept a chain-computed address terminal
(takeAddrSource, the GPR analog of takeSplatSource) and move it into
the address register before clobbering scratch, on both arm64 and
amd64, both pointer widths, with tryFuseLoop's rejection points now
logged under WASM2GO_FUSE_DEBUG.

Twin fixtures pin the behavior (cg_simd_addrsum32 /
cg_mem64_simd_addrsum: a mem node with an ArgNode address must
appear on both widths). On the llama.cpp modules: the hot memory64
conversion loop's bare loads drop 66 -> 26, and wasm32 gains its
first fused-LOOP upgrades on this module (fxl0/fxl1) because whole
bodies now fit one window. Throughput is A/B-neutral on both widths
(the loop's remaining cost is gc frame traffic between windows, not
the bare ops) — this lands as groundwork and unification, with both
llama suites green including the native-token-match gates.

Co-Authored-By: Claude Fable 5 <[email protected]>

* Upgrade head-tested unit-decrement loops and unwrap memory64 call args

Two blockers kept real conversion loops out of the fused-loop
upgrade. First, the emitter produces a THIRD countdown shape the
matcher missed on both widths: a head-tested while,
`for { if c == 0 { break } else { body; bumps; c = c - 1 } }`.
With a unit decrement `c == 0` exits at exactly the same counter
values as the pre-tested `Ui(c) < Ui(1)`, so the shape reuses the
pre-test descriptor; any other decrement under an equality head is
rejected. Second, memory64 fused-call sites wrap their scalar
arguments in int64(...), which broke the upgrade's slot and
reassociated-bump matching by identifier — both now unwrap the
width conversion first.

The conversion-loop twins now upgrade on BOTH widths (pinned by
TestConvLoopUpgradesBothWidths — the first exercise of the memory64
loop-splice path), and the wasm32 llama module gains five fused
loops. Throughput is A/B-neutral on both widths (the upgraded loops
sit off the hot paths); both llama suites green including the
native-token-match gates.

Co-Authored-By: Claude Fable 5 <[email protected]>

* Fix dropped-definition reads exposed by nested address chains

The address-chase commit made nested chains reachable
(`v80 = v67 + s` with v67 itself an intervener), and two holes in
the surrounding bookkeeping produced wrong stores in the
variable-stride axpy shape — caught by the gcasm differential
fixture (the llama greedy gates do not exercise it):

- An intervener counted as fully consumed when the chases
  internalized all its reads, and was dropped — even when a KEPT
  intervener's definition still read it (v80 stays because a loop
  tail reads it, but v80's kept RHS read the dropped v67, leaving
  the tail bump computed from an unassigned variable). Un-drop such
  definitions to a fixpoint.
- The chase could leak a WINDOW-INTERNAL value (another candidate's
  result, e.g. an extract_lane feeding an address) into a chained
  scalar argument, which has no pre-call definition after fusion.
  Refuse at the chase so the address falls back to the original
  pair/opaque forms.

TestGcasmSimdLoopUnroll (the value-level differential over the
unroll/fuse/loop-fuse configurations) fails before and passes after.

Co-Authored-By: Claude Fable 5 <[email protected]>

* Gate unamortized packed outlining on memory64 modules

A packed outline boundary writes every live-in into the scratch
array and reads it all back on each call. On a memory64 module the
i64 locals double the caller's register pressure and the round trip
stops paying for wide boundaries: refusing packs whose body carries
fewer than 24 values per slot gains ~17% prompt throughput on the
llama.cpp module (the ~130-slot conversion loop runs inline
instead). The SAME gate on wasm32 loses ~25% prompt throughput —
its conversion loop compiles better extracted than inline — so the
ratio applies only when the module is memory64, measured both ways
by interleaved A/B with both llama suites green.

Co-Authored-By: Claude Fable 5 <[email protected]>

* asmgen: import direct-asm backend and add br_table dispatch support

Import the SSA-to-assembly emitter (internal/asmgen) and its register
allocator (internal/regalloc) so hot scalar functions can bypass the
gc backend's code quality ceiling — the remaining wasm64 throughput
gap is dominated by frame-reload storms gc generates for doubled i64
live sets in the conversion/dot-product driver loops.

The emitter predates two SSA block kinds, so bring it up to the
current IR:

- BlockBrTable: new EmitBrTable arch hook emitting a linear compare
  chain (selector materialized once into the scratch register, one
  CMP+branch per case value, then an unconditional jump to the
  default). Successors carrying phis are rejected to the per-function
  fallback — no dispatch-shaped code produces them today, and edge
  copies would need per-successor intermediate labels.
- BlockThrow stays unsupported; functions using it keep their gc-
  compiled Go bodies via the existing per-function stub fallback.

LowerFunction calls gain the ThrowSet parameter added since the
emitter was written.

Testing: the driver harness is now arch-parameterized, and the
control-flow value test (if/else merge, loop phis, br_table chain)
runs natively on both amd64 and arm64 — Apple Silicon executes the
arm64 driver even when the test process runs under Rosetta, detected
via sysctl.proc_translated.

Co-Authored-By: Claude Fable 5 <[email protected]>

* asmgen: memory64 addressing for loads, stores, and memory ops

Teach the direct-asm emitters 64-bit linear-memory addressing so
memory64 modules can route hot scalar functions through asmgen. The
effective-address computation is factored into one per-arch helper
(emitMemAddrAMD64 / emitMemAddrARM64) shared by the load and store
paths; the two memory widths use the same mechanism and differ only
in the address rule, mirroring the pure-Go emitter:

- wasm32: address = uint32(base + off), computed in 32-bit so
  negative-int32 bases wrap per the wasm effective-address rule
  (unchanged behaviour, now in one place per arch instead of four).
- memory64: address = base + off in full 64-bit, no wrap — the
  pure-Go path is `uint64(base) + offset`. On amd64, constant bases
  fold into the displacement when the total fits int32 and larger
  offsets are materialised through a scratch register before the
  index add; on arm64 the assembler's REGTMP materialisation keeps
  the shape a single ADD regardless of offset size.

OpMemSize / OpMemGrow / OpMemoryCopy / OpMemoryFill on memory64
modules route to the 64-bit helper family (memorySize64,
memoryGrow64, memoryCopy64, memoryFill64) with the i64 signatures
the codegen helpers define.

Testing: cg_mem64_memops.wat is the memory64 twin of cg_memops.wat
(every load/store width and sign, offset memargs, constant-base
fold, i64 mem-size/grow), value-checked on amd64 and natively on
arm64. The wasm32 memops value checks also gain an arm64 twin so
both widths are execution-verified on both arches.

Co-Authored-By: Claude Fable 5 <[email protected]>

* Direct-asm backend: per-function asmgen bodies inside the gcasm bundle

Wire the SSA-to-assembly emitter (internal/asmgen) into the pipeline
as an opt-in per-function backend coexisting with gcasm. gc's
register allocator treats every CALL as clobbering everything, which
spills large i64 live sets to the frame around fused-window calls;
emitting those functions straight from SSA is the lever for removing
that reload traffic. This change lands the plumbing with a
conservative first eligibility class (call-free leaf functions) and
per-function fallback everywhere else.

Flow:

- codegen: Options.DirectAsmFuncs opts function names in. The
  translator retains their finalized SSA (post optimization fixpoint,
  idiom rewrites, and outlining — outlined loop functions with
  unpacked scalar boundaries are eligible too) in
  Result.DirectAsmSSA. Direct-asm modules with a memory also emit
  the gcasm memory probe so the bundle can verify the Module.M
  offset the asm bodies hardcode.
- gcasm: Config.DirectAsm carries the retained SSA into Build, which
  emits each function per architecture via asmgen (buildDirectAsm)
  and swaps the body in place of the gc-listing transform in
  buildPkg. Bodies carry a "// direct-asm: <name>" marker; decls
  reuse the transform path's shape (declSig, now shared). Emission
  failures fall back to the transform per function and are counted
  (BuildStats.DirectAsm / DirectAsmFallback).
- asmgen: FuncOptions.ForbidCalls fails emission for any function
  that would CALL out (helpers, callees, memory ops, global
  wrappers) — the bundle's callee symbols are not resolvable from
  asmgen's same-package CALL spelling yet, and a body that emits but
  fails to link would break the build instead of falling back.
  ModuleMOffset is exported for the host-side layout check.
- cmd/wasm2go: -direct-asm flag (comma-separated names).

Testing: the arith fixture runs through the full backend with five
functions opted in — four leaf bodies come from asmgen on both
architectures (marker asserted), the helper-calling one falls back —
and the generated package computes the same values the asmgen driver
tests pin.

Co-Authored-By: Claude Fable 5 <[email protected]>

* asmgen: v128 values and SIMD helper CALLs

Extend the direct-asm emitters to functions containing SIMD, the
shape the hot wasm64 loops actually have. v128 values live in
16-byte stack slots as [2]uint64 pairs (slotSize / abiSize gain the
class); OpSimdConst materializes its lane payload into the slot; and
OpSimdCall / OpSimdMemCall lower to plain ABI0 CALLs of the bundle's
simd_* helper symbols — the same symbols the pure-Go backend calls.
There is no static signature registry for the SIMD family: each
call's layout derives from the SSA value itself (arg types, result
type, module pointer for memory ops), shared between the arches in
simdCallSpecOf. v128 args and results travel as two 8-byte halves;
phi edge-copies and staged-phi slots handle the type on both arches.

ForbidCalls narrows from "any CALL" to non-SIMD callees
(hasNonSimdCall): SIMD helper symbols ship with every SIMD-using
bundle, so their CALLs always resolve, while scalar helpers, sibling
functions, memory ops, and global wrappers keep the per-function
fallback until the bundle's wrapper spelling is wired up.
Cross-package emission (multi-package chunks) is still rejected in
simdCallSpecOf for the same linkability reason.

The cross-block register allocator predates the 16-byte slot class,
so functions containing v128 values keep the block-local allocator
(whose pool selection already skips v128-typed values).

Testing: the SIMD fixture runs through the full backend with every
function opted in — the v128-bearing bodies come from asmgen (helper
CALLs then still spliced or kept by the existing gcasm machinery for
the rest of the bundle) and all values match the wazero-verified
reference the gcasm test pins.

Co-Authored-By: Claude Fable 5 <[email protected]>

* asmgen: splice SIMD helper bodies inline via the gcasm tables

Replace the marshalled SIMD helper CALLs in direct-asm bodies with
the same inline NEON bodies the gc-capture transform splices — the
step that makes the direct backend competitive on SIMD-heavy
functions instead of regressing to the pre-splice call-overhead
world.

The splice machinery lives in gcasm, which already imports asmgen,
so the dependency inverts through an interface: asmgen's
FuncOptions.Splicer receives operand locations (frame slots — SIMD-
containing functions emit in slot-only mode: no register homes, no
m-cache, no loop-carry coalesce, because splice bodies clobber
registers freely) plus a scratch area (the callee-argument region,
always a superset of the splice's stack traffic). gcasm's
implementation stages the operands per the same ABIInternal
assignment the listing transform uses (scalars into R0../F0.., v128
halves into the scratch area at their sequence offsets) and drops in
a64SpliceSimd unchanged, sharing the per-package ConstPool. Memory
splices branch to the shared out-of-bounds stub, which the emitter
appends once at body end. Everything goes through a scratch builder
first so an op the tables decline falls back to the CALL with no
staging leaked.

Direct-asm emission consequently moves inside buildPkg (it needs the
package's ConstPool); amd64, which has no per-op splice table, keeps
the marshalled CALLs.

Testing: the SIMD fixture's direct-asm arm64 bodies are asserted to
contain spliced NEON, and the generated package is now also
cross-built for arm64 and EXECUTED natively (Apple Silicon runs the
binary even when the test process is under Rosetta), matching the
wazero-verified reference values.

Co-Authored-By: Claude Fable 5 <[email protected]>

* asmgen: coalesce loop carries into splice-safe registers

The reason the direct backend exists: gc must assume every call site
clobbers every register, so the hot loops' i64 carries live in frame
slots and each iteration pays a reload storm around the SIMD ops.
Inline splice bodies, unlike calls, have a documented clobber
contract (arm64: R0–R15 / R25–R27 and the V registers) — so R19–R24
survive them, and asmgen can keep the loop-carried scalars there.

runSpliceCoalescePass is the splice-mode variant of the loop-carry
coalesce: SIMD helper calls inside a loop no longer bar coalescing,
and every such value in a coalesced loop is recorded in
plan.mustSplice — a splice-table miss there fails the function to
the host's fallback instead of emitting a CALL that would clobber
the carry. The pool is R19–R24 (the fused-window splices claim
R20–R23 as window state; the pool must shrink when asmgen learns to
emit those).

The original coalesce safety rule — the phi may have NO user but its
own bump — denies the canonical counter/pointer carries (read by the
loop condition or an address user AND the bump). The splice-mode
pass uses a finer analysis (coalesceUsesSafe): every read must
execute before the bump's register write in each iteration, which
holds when the bump sits in the latch block (the latch runs last —
its only successors are the header and loop exits), reads in the
latch precede it, and the phi has no reads outside the body, via
another phi's edge copy, or as the latch's control. The default
pass keeps the single-use rule pending an A/B.

Two correctness fixes surfaced by this work:

- opEmitsCall did not know OpSimdCall / OpSimdMemCall, so their
  marshalled CALLs were invisible to the m-cache re-prime logic and
  the coalesce call barrier — a latent wrong-code hazard for CALL-
  mode SIMD bodies.
- The arm64 lane-op memory splices used R24 as their data scratch,
  violating the splice clobber contract this change relies on. R4
  (inside the contract set, free — the helper's args occupy R0–R3,
  and unlike R25–R27 it survives the memory preamble) replaces it.

Testing: a self-contained SIMD accumulation loop fixture with
pointer/counter carries runs differentially — direct-asm output vs
the normal backend across 33 trip counts — on amd64 and natively on
arm64, and the arm64 body is asserted to contain both spliced NEON
and an R19 carry.

Co-Authored-By: Claude Fable 5 <[email protected]>

* asmgen: multi-package emission, packed boundaries, inline helpers

Three pieces that carry the direct-asm backend from fixtures to the
llama-scale bundle:

Multi-package. The linkname-split translator builds its own Result
and dropped DirectAsmSSA entirely — retention silently never reached
the bundle (the chunked layout is the only one llama uses). With the
field wired through, chunk emission works on the splice-only
contract: spliced SIMD sites need no symbols at all, so a chunk
function is eligible whenever every SIMD site splices — the
multi-package rejection moves from spec time to the CALL fallback
path, which now fails the function to the transform instead of
emitting a same-package spelling that cannot resolve. The
out-of-bounds trap CALL resolves through buildPkg's callee-resolution
closure (registering the chunk-local forward wrapper), and a splicer
without a resolved trap symbol refuses memory splices.

Packed boundaries. llama's outlined SIMD loops carry v128 values
across the boundary, so nearly all of them use the packed form —
the caller passes only the module pointer and the values ride
Module.OutlinePack. Retention keeps them (results-only signature
plus the parameter types in slot order), and the emitters gain a
pack prologue: parameter values load from the pack scratch (pinned
at moduleMOffset+8 by the same field-order construction that pins M)
into the params' frame slots, parameter reads resolve to those slots
instead of FP offsets, and OpParam slots are excluded from slot
reuse (a pooled reuse would clobber the prologue's write).

Inline helpers. The first llama fallback wave was all trivial scalar
helpers (sign/zero extends, 32-bit reinterprets, f32_abs) that the
amd64 emitter already lowers to one or two instructions but arm64
routed through a marshalled CALL. helperAlwaysInline names the set
both arches now emit natively; planFunc exempts those calls from
hasCall (no callee frame, no ForbidCalls consequence), and both
emitters guard the CALL path against ever being reached for an
exempted name — the frame it would use was never reserved. The
ForbidCalls diagnostic now names the offending callee, which is what
surfaced the wave in the first place.

Co-Authored-By: Claude Fable 5 <[email protected]>

* asmgen: inline float ops, global offsets, resolver-backed callees

Three widening waves, each driven by the ForbidCalls diagnostic
(which now lists every distinct blocker per function instead of the
first) against the llama-scale bundle:

Inline float family. The arm64 emitter gains native lowerings for
the float compares (FCMP + CSET with unordered-false condition
codes, NE unordered-true — wasm/IEEE semantics, matching gc's own
lowering), binary arithmetic (FADD/FSUB/FMUL/FDIV both widths),
neg/abs, promote/demote, int→float converts, and i32_wrap_i64; all
of these were already inline on amd64. helperAlwaysInline widens
accordingly.

Global offsets. Module-defined globals inline as MOVs against the
module pointer. The translator — the one component that actually
emits the Module struct — computes each global's byte offset
following its own field order, ships them through
Result.DirectAsmGlobals into the bundle, and emits compile-time
assertions (directasm_layout.go) pinning every offset direct-asm
hardcodes: M, the outline pack, and each global. Layout drift then
fails the consumer's build instead of corrupting memory.

Resolver-backed callees. Direct calls and the memory-op helper
family resolve through buildPkg's callee-resolution closure — the
same machinery the listing transform uses, registering chunk-local
forward wrappers as a side effect — via two FuncOptions hooks
(CalleeSymbol / MemHelperSymbol). Resolved calls are exempt from
ForbidCalls; unresolved ones still fall back per function.

With these, nine llama functions emit as direct asm on arm64
(including every outlined loop of the compute monolith), and the
full go-llama verification suite passes on the resulting bundle:
greedy determinism, prompt batch-size invariance, native NLL
parity, and cross-context repeatability.

Co-Authored-By: Claude Fable 5 <[email protected]>

* asmgen: inline integer div/rem on both arches

The divide/remainder family was the largest remaining direct-asm
fallback class. Lower it inline: native divide plus explicit
trap-branch CALLs, with the trap symbols resolved through the same
host hook as the memory helpers (forward wrappers in multi-package
chunks) — an unresolved trap helper keeps the marshalled-CALL path
instead of emitting a spelling that cannot link.

arm64 (new): SDIV/UDIV for quotients, MSUB for remainders. arm64
divides never fault, so the wasm traps are explicit: every op
branches to the div-zero trap on a zero divisor, and div_s checks
INT_MIN / -1 for the overflow trap. rem_s needs no overflow check —
SDIV wraps the quotient to INT_MIN and MSUB computes x - q*y = 0
with the same wrap, which is exactly wasm's answer. Results honor
plan.regHome (the arm64 regalloc register-homes helper-call results;
storing only to the slot left downstream readers of the home
register with garbage — caught by the new native-execution arith
value test, which now covers the div family on arm64).

amd64 (fixed): the existing inline div_s relied on IDIV's #DE for
the INT_MIN / -1 case, which panics with the divide-by-zero kind;
wasm calls it integer overflow. Both widths now check explicitly and
call the overflow trap. All trap CALLs go through the plan's
resolved symbols instead of hardcoded same-package spellings.

Also guard the outline pass's synthetic sibling-call sentinel
(OpCallDirect with a negative index and the callee name in Aux):
planFunc indexed the module's function table with it and panicked on
nested outlined loops; it now counts as an unresolvable callee, so
ForbidCalls falls the function back cleanly. The packed call-site
protocol for such calls is future work.

At llama scale this raises the arm64 direct-asm body count from 58
to 76 at outline threshold 100 (9 at the pipeline's threshold 400),
with the verification gates green throughout.

Co-Authored-By: Claude Fable 5 <[email protected]>

* asmgen: hoist module state across splices; splice the rng/nc loads

Two body-quality steps, both llama-verified for correctness:

Module-state hoisting. Every arm64 memory splice reloaded m from its
FP slot and m.M / the memSize pointer from the Module per site —
three loads per memory op, plainly visible in the pp profile. The
emitter now keeps them in the splice-safe registers the coalesce
pool cedes (R24 = m via the m-cache mechanism, R23 = m.M, R22 = the
memSize pointer, staged by the splicer's new HoistPrologue), leaving
R19–R21 for loop carries. The refresh after calls became precise in
the process: emitters that actually produce a CALL set a per-value
flag, replacing the op-kind heuristic that would have re-primed
after every spliced SIMD op (and, previously, missed nothing but
paid refreshes after inlined helpers). Memory-less functions skip
the prologue — staging it read a garbage m and faulted, caught by
the SIMD fixture's arm64 leg.

Bounds-coalesced loads. The hot conversion loops read memory through
the range-check group forms (v128_load_rng leader / _nc members),
which only the pair-form splicer knew; the array-form path direct
asm uses kept them as CALLs, so the hottest outlined loops always
fell back. The array-form twins mirror the pair bodies (signed group
start, TBNZ wrap check, leader-only bounds compare) with the
hoisted-register variants, and both hot loops now emit — 107 arm64
bodies at outline threshold 100.

MEASURED STATUS, not yet a win: with the hot loops emitted, prompt
eval REGRESSES ~2.9x on the direct bundle (correctness gates all
pass). The per-op splice protocol round-trips every v128 value
through its stack slot between adjacent ops — the load feeds a
splat/mul/add chain entirely through memory — where the gc-capture
path's fused windows keep the chain in V registers. Vector-value
register residency across splices is the remaining (and now
precisely measured) gap; nothing ships through this path until it
closes — the pipeline's bundle does not opt in.

Co-Authored-By: Claude Fable 5 <[email protected]>

* asmgen: slot-direct v128 splice protocol and spare-register caching

The measured 2.9x prompt-eval regression on the direct-asm hot loops
decomposed into two protocol costs, both addressed here:

Slot-direct splicing. The splicer staged every v128 argument from
the value's frame slot into the ABIInternal scratch area through GPR
pairs and copied every result back the same way — four extra memory
ops on each side of every op. SpliceSlots now carries the values'
own RSP offsets into the splice emitters (argument reads, result
stores, lane addressing), so the scratch hops vanish; the transform
path passes nil and keeps its scratch behavior, where the values
genuinely live there. asmgen ships the offsets on the operands
(InSlot/SlotOff) — an unpacked FP parameter is the one v128 without
a slot and keeps the copy path.

Spare-register value caching. Every splice body computes through V0,
so no value survives in a register past the next op and plain
store-to-load forwarding never fires. a64SpliceValueCache parks each
"// simd out" result in a spare vector register (V17–V31, strictly
above anything a splice body touches) and rewrites later reloads of
that slot into single-cycle register moves, with the sibling passes'
conservative invalidation (labels/CALL reset, width-aware RSP store
overlap, pointer stores drop everything). The result store always
remains — it is the value's only definition.

Two bugs found by the gates on the way:

- a64DeadArgStores must not run on direct-asm bodies: it models the
  capture shape, where arg-slot stores are redundant copies; in the
  slot-direct shape the "simd out" store IS the definition, and the
  pass deleted live stores. The native NLL parity gate caught the
  wrong values (greedy and batch invariance did not — a lesson in
  gate sensitivity).
- a64SpliceWordClobberList omitted v4, which the f16 conversion body
  writes ("movi v4") — a value the forwarding pass parked in F4
  would survive that splice on paper while being clobbered in fact.

Prompt eval on the direct bundle: 19.5 → 44.6 tok/s across these
steps (baseline 55.5; the remaining gap is scalar address traffic
through frame slots and the per-op bounds preambles). All
verification gates pass at every step.

Co-Authored-By: Claude Fable 5 <[email protected]>

* asmgen: scalar slot forwarding and dead result-store elimination

Two more text passes over direct-asm arm64 bodies, closing most of
the remaining per-op splice overhead:

Scalar forwarding (a64ScalarSlotForward). The slot-only emission
spells every scalar intermediate as a store/reload pair through its
frame slot — address chains read as MOVD store, MOVD reload, ADD,
repeat. The pass tracks full-width register→slot bindings within
straight-line spans and rewrites reloads into register moves (or
drops them when source and destination coincide), with the sibling
passes' conservative rules: labels and CALL/RET reset, WORD splice
bodies kill bindings in the contract clobber set (R0–R15/R25–R27),
pointer stores drop the slot side, narrower stores only invalidate.

Dead result stores (a64DeadSimdOutStores). Once reloads are
forwarded to the spare-register cache, a single-use v128
intermediate's "// simd out" store serves nobody — but only a
function-scoped read count can prove it (a later block may reload
the slot). The pass counts every RSP-offset mention as an operand —
comments are explicitly NOT reads: a spare-forwarded reload names
its origin slot in a comment only, and counting those pinned nearly
every store (the first cut measured neutral because of exactly
this). Slot addresses that escape through `ADD $off, RSP, Rn` count
as reads of both halves, and any unmodeled RSP form keeps the whole
body. The spare park always stays: it is what serves the forwarded
consumers.

Prompt eval on the direct bundle, same config and gates green at
every step: 45.4 → 54.3 tok/s (the dead stores were nearly all of
it once the comment rule landed); the gc-backed baseline sits at
55.7. The cumulative ladder from the first correct slot-direct
build stands at 19.5 → 54.3.

Co-Authored-By: Claude Fable 5 <[email protected]>

* asmgen: v128 loop-carry homes in the splice-safe vector registers

Coalesce v128 loop-carried phis (the accumulators) into V25–V31 —
above both the splice bodies' registers (V0–V4, F16) and the spare
cache (now V17–V24) — so an accumulator add-chain runs entirely in
registers across iterations instead of round-tripping its slot.

The splice protocol gains register operand forms: SpliceSlots
carries per-argument register sources and a register destination,
the table path reads/writes them with single VORR moves, the memory
path covers the store's value argument and every load result, and
the lane paths — which index the value's memory — spill a
register-resident operand to its scratch slot first and load a
register-destined result back after the in-place edit (loop exits
only; the hot path never lanes a homed value).

Safety differs from the GPR carries in one respect: a V home
survives to function end (the pool is consumed once per function,
no splice touches the range), so live-out reads of the accumulator
— the after-loop store or lane fold every reduction ends with — are
allowed where a GPR carry must refuse them. That in turn is only
sound because v128 coalescing is gated on a function with no
non-SIMD callees and forces every SIMD site in the function to
splice (a fallback CALL would clobber the vector file).

Phi copies read and write homes directly (a register-resident
source stores with one FMOVQ instead of two GPR round-trips; the
entry edge moves register-to-register when both ends are homed).

llama-scale: the hot conversion loop carries 17 home references,
all gates pass; throughput is neutral-to-slightly-negative on this
loop (its accumulation is mostly block-local, already served by the
spare cache, and the loop-exit lane spills give some of the win
back). The infrastructure is what the fused-window step needs
either way.

Co-Authored-By: Claude Fable 5 <[email protected]>

* codegen: fire loop fusion on memory64 modules

A memory64 llama build ran prompt eval 40% behind its wasm32 twin
from the same source. The differentiator was loop fusion (fxl):
the wasm32 bundle upgrades five countdown loops — including the
q8_0 vec_dot kernel, whose fused form carries the 44-SDOT paired
column kernel — while the memory64 bundle upgraded none, leaving
the same kernel as a 10-SDOT per-window loop.

Dumping the loop bodies as the fuser sees them showed the memory64
vec_dot loop statement-for-statement identical to the wasm32 twin
up to spelled widths. Two width-blind checks rejected it:

- fuseConstOf did not recognize int64(K): the loop upgrade's
  leftover classification (constant locals hoist above the loop)
  refused the body because the bump-delta local `vN = int64(136)`
  did not count as a constant, failing the whole upgrade for every
  memory64 loop whose const CSE landed inside the body.

- The header form derived the counter's width only from the
  decrement terms' spelling. When every term is a const LOCAL
  (`c = c - v170 - v170 ...`), nothing is spelled at any width and
  the descriptor defaulted to i32 — generating an fxl helper whose
  counter parameter mismatched the module's promoted i64 counter.
  matchUiLt/matchEqZero now report which width the guard matched
  (Ui64(c) < Ui64(K) is the counter's own width) and the
  descriptor carries it.

With both fixed the memory64 vec_dot upgrades to the same fused
loop as wasm32 (42 SDOT vs 44). Measured on the llama q8_0 bundle,
all token-equivalence gates passing at every step: prompt eval
54.3 -> 68.8 tok/s (+27%), decode 30.1 -> 46.7 tok/s (+55%).
wasm32 output is unchanged (the new width legs only accept forms
wasm32 never spells).

Also under WASM2GO_FUSE_DEBUG: the countdown matcher dumps large
loop bodies as the fuser sees them, and upgrade refusals of large
loops log the rejecting check with the body size — the two probes
that located this.

Co-Authored-By: Claude Fable 5 <[email protected]>

* codegen: log loop-window candidate collection under fuse debug

The countdown-loop upgrade's window trial can fail before any walk
runs — candidate collection stops at the first unsafe intervener —
and nothing reported where. Under WASM2GO_FUSE_DEBUG, large loop
bodies now log the stopping statement and the collected counts, and
big-loop window trials log the walk verdict with scalar/pair usage.
These probes located the memory64 conversion-loop blocker (eight
irreducible address bases exceed the fused-signature slot budget).

Co-Authored-By: Claude Fable 5 <[email protected]>

* codegen: check the debug printer's error return

Co-Authored-By: Claude Fable 5 <[email protected]>

* gcasm: harden fused-splice capacity handling

Three latent hazards around over-budget fused signatures, found
while probing why the memory64 conversion loop cannot yet upgrade
to a fused loop (it needs more integer argument slots than the
splice ABIs seat):

- errFusedCapacity had no catcher: the comment promised "the
  transform keeps the helper CALL instead of failing the build",
  but both transforms turned ANY splice error into a hard build
  error. They now catch the capacity sentinel and fall back to the
  unspliced path deliberately.

- Fused/loop splices emitted directly into the output builder, so
  an error after partial emission (the vector-pool-exhausted paths)
  would have left corrupt text if it were ever survivable. They now
  emit into a scratch builder that is committed only on success.

- x64SpliceFused/x64SpliceLoop indexed the argument-register table
  with unchecked positions (an over-budget tree would panic instead
  of declining), and a64SpliceLoop had no >R15 guard at all (it
  would have emitted reads of R16+ — registers gc never loads —
  as silent garbage). All three now decline via errFusedCapacity.

No behavior change under the current codegen budgets — the guards
are unreachable until the caps rise — verified by a byte-identical
llama memory64 bundle. The experiment that motivated this (raising
the argument caps alone) does NOT compose yet: pair-form helpers
have no marshalled-call fallback, so a tree either arch cannot
splice still fails the build downstream; widening the budgets needs
the stack-argument legs on both splicers first.

Co-Authored-By: Claude Fable 5 <[email protected]>

* codegen: chase const-sum address bases; fix chase-use overcounting

The memory64 conversion loops (a quarter of prompt eval) could not
upgrade to fused loops: their eight per-row address bases — chained
interveners like `v243 = v228 + l6`, rows advancing by a runtime
stride — each burned one opaque scalar slot under the ArgSum leg,
overflowing the fused-signature budget. The scalar-chain chase
already knew how to decompose such sums into shared atoms
(scalar_i32_add nodes; the wasm32 twin fuses this way), but only
the NO-const-side leg of the walk ever invoked it. The mem-address
branch now chases a const-sum's BASE through the same machinery
(addr64-gated; wasm32 output stays byte-identical), so k row
addresses share their atoms' slots and the loops fit the existing
caps on both splice ABIs.

Opening that terrain exposed a latent chase-accounting bug: two
consumers chasing the same definition chain (`v311 = v309 + l6`
feeding two loads) bumped chaseUses[v309] once PER RE-CHASE, but
dropping v311's definition at commit removes exactly ONE textual
occurrence of v309. The overcount made the fully-consumed proof
(identCount == uses+1) hold spuriously — eating the read a
loop-tail bump still had — so the commit dropped `v309 = v288+l5`
while `v228 = v309 + l5` kept executing against the variable's
zero value: row pointers walked garbage, greedy survived, and only
the native-NLL gate caught the drift (rel 1e-1 on the f32 model,
both arches, fast-math-independent). A definition's child reads
are now charged once per trial (chaseExpanded, carried through the
chase snapshot/restore).

Also under the fuse-debug env: WASM2GO_CHASE_MAX=N applies the new
leg to the first N sites only (the bisect that localized the bad
transform before window-level differential tests took over).

Measured on the llama memory64 bundle, all five token-equivalence
gates passing and the wasm32 bundle byte-identical: prompt eval
68.8 -> 89.7 tok/s — parity with the wasm32 build's 90 — and
decode ~50 tok/s. The conversion loop and the vec_dot kernel both
run as fused loops now (fxl0/fxl1, 42-SDOT paired column form).

Co-Authored-By: Claude Fable 5 <[email protected]>

* codegen: accept exit copies in the do-while break arm

The emitted do-while countdown closes its break arm with the
after-loop aliases of the carried accumulators (`vOut = accNext;
break`); the matcher required a bare break and refused the whole
upgrade for every such loop — including the K=1 dot kernels the
decode path spends half its time in. The leading assignments now
classify through the same tail logic (a copy whose target is not a
loop argument already becomes a post-loop exit copy at upgrade
time).

No output change yet on either width (the K=1 kernels still refuse
later, at the window trial); gates pass, wasm32 bundle
byte-identical. Groundwork for upgrading the decode-path loops.

Co-Authored-By: Claude Fable 5 <[email protected]>

* test: cover the asmgen emitters and memory64 helper family

The coverage gate fell to 79.7% under this branch's new code. Four
additions bring it to 83.8% with behavioral tests, not padding:

- asmgen_widecmp fixture + both-arch driver tests: variable-count
  64-bit shifts, 64-bit compares, inline float compares (NaN
  unordered included), the fused bit-test branch, an unreachable
  arm, and the amd64-only inline f32.min/f64.max (arm64 lowers
  those to helper calls, so its driver skips them).

- An emit-only sweep lowering and emitting EVERY function of every
  cg_* fixture through BOTH emitters regardless of host arch —
  emission is pure code generation, so this walks the emitters'
  long tails that the natively-executed driver subset never
  reaches. Fixtures that decline wholesale (atomics/threads) skip.

- Unit tests for the helper-name/type utilities and the
  demand-driven BuildWrappers generator (call_indirect trigger).

- Behavioral tests for the memory64 helper family (every
  simd_m64_* / simd_p_m64_* entry over live memory), the
  replace-lane family in all three forms, the f16 convert-store
  chain (load16x4_u lane form documented by the test), the
  memory64 bulk-memory ops, and the trap/throw/catch entry points
  (foreign panics must re-panic, not be swallowed).

Co-Authored-By: Claude Fable 5 <[email protected]>

* test: drop an untracked fixture from the emit sweep

The sweep list was generated from a local glob and picked up
cg_threads_multiword.wat, which exists only in this worktree — CI
has no such file and the sweep failed on it. The list now contains
tracked fixtures only (verified against git ls-files).

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-11 22:00:25 +09:00

2288 lines
86 KiB
Go

package asmgen
import (
"bytes"
"fmt"
"os"
"os/exec"
"path/filepath"
"runtime"
"sort"
"strings"
"testing"
"github.com/goccy/wasm2go/internal/lower"
"github.com/goccy/wasm2go/internal/ssa"
"github.com/goccy/wasm2go/internal/testfixture"
"github.com/goccy/wasm2go/internal/wasm"
)
// assembleARM64 cross-compiles every defined function from the
// given fixture through EmitFuncARM64 and asserts that the Go
// toolchain accepts the resulting plan9 asm. Used as the cheapest
// possible cross-arch validation on an amd64 host: it confirms
// every emitted line is a known instruction with valid operands
// without needing an arm64 runner. Functions that exercise an op
// the prototype doesn't yet support get stubbed (matching the
// amd64 test driver's behavior).
func assembleARM64(t *testing.T, fixture string, moduleSrc string) {
t.Helper()
bin := testfixture.Wasm(t, fixture)
mod, err := wasm.Parse(bytes.NewReader(bin))
if err != nil {
t.Fatalf("parse: %v", err)
}
exportNames := map[uint32]string{}
for _, e := range mod.Exports {
if e.Kind == wasm.ExportFunc {
exportNames[e.Index] = e.Name
}
}
funcSymbol := func(idx uint32) string {
if n, ok := exportNames[idx]; ok {
return n
}
return fmt.Sprintf("Fn%d", idx)
}
var asmBuf, declBuf, stubBuf strings.Builder
for i := range mod.Functions {
idx := mod.NumImportedFuncs + uint32(i)
name := funcSymbol(idx)
fn, err := lower.LowerFunction(mod, idx, name, nil)
if err != nil {
t.Fatalf("lower %s: %v", name, err)
}
sig := mod.FuncTypeOf(idx)
asm, decl, err := EmitFuncARM64(name, sig, fn, FuncOptions{
ModulePkgRef: "*Module",
Module: mod,
FuncSymbol: funcSymbol,
})
if err != nil {
fmt.Fprintf(&stubBuf, "func %s%s { panic(%q) }\n",
name, goSignature(sig, "*Module"), "asm stub: "+name)
continue
}
asmBuf.WriteString(asm)
declBuf.WriteString(decl)
}
dir := t.TempDir()
mustWrite(t, dir, "go.mod", "module asmgentest\n\ngo 1.25\n")
mustWrite(t, dir, "module.go", moduleSrc)
mustWrite(t, dir, "decls_arm64.go", "//go:build arm64\n\npackage main\n\n"+declBuf.String())
mustWrite(t, dir, "asm_arm64.s", "//go:build arm64\n\n#include \"textflag.h\"\n#include \"funcdata.h\"\n\n"+asmBuf.String())
if stubBuf.Len() > 0 {
mustWrite(t, dir, "stubs_arm64.go", "//go:build arm64\n\npackage main\n\n"+stubBuf.String())
}
mustWrite(t, dir, "main.go", "//go:build arm64\n\npackage main\n\nfunc main() {}\n")
cmd := exec.Command("go", "build", "-o", filepath.Join(dir, "out"), ".")
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOOS=linux", "GOARCH=arm64", "CGO_ENABLED=0")
if out, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("%s GOARCH=arm64 build failed: %v\n%s\n\n--- asm ---\n%s", fixture, err, out, asmBuf.String())
}
}
// TestBuildPackageFilesARM64Assembles cross-builds the
// BuildPackageFiles output for both a wrapper-free fixture (arith)
// and a wrapper-heavy one (cg_globals) to verify the per-arch
// `<arch>.s` + `decls_<arch>.go` + `wrappers.go` triple emits
// internally-consistent symbols on arm64 as well as amd64.
func TestBuildPackageFilesARM64Assembles(t *testing.T) {
for _, fixture := range []string{"arith", "cg_globals", "cg_indirect"} {
t.Run(fixture, func(t *testing.T) {
bin := testfixture.Wasm(t, fixture)
mod, err := wasm.Parse(bytes.NewReader(bin))
if err != nil {
t.Fatalf("parse: %v", err)
}
files, err := BuildPackageFiles(mod, BuildPackageOptions{Package: "main"})
if err != nil {
t.Fatalf("BuildPackageFiles: %v", err)
}
dir := t.TempDir()
mustWrite(t, dir, "go.mod", "module asmgentest\n\ngo 1.25\n")
for path, content := range files {
mustWrite(t, dir, path, content)
}
// Hand-craft the Module + helpers (codegen's full
// integration would emit these). For arith / cg_globals
// / cg_indirect the shape that satisfies the asm's
// references is small enough to inline.
mustWrite(t, dir, "shared.go", sharedModuleSource(fixture))
mustWrite(t, dir, "helpers.go", helpersGoSource())
mustWrite(t, dir, "main.go", "package main\nfunc main() {}\n")
cmd := exec.Command("go", "build", "-o", filepath.Join(dir, "out"), ".")
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOOS=linux", "GOARCH=arm64", "CGO_ENABLED=0")
if out, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("GOARCH=arm64 go build: %v\n%s", err, out)
}
})
}
}
// sharedModuleSource returns the shared (always-on) Go source for a
// given fixture: the Module struct plus any companion declarations
// the asm's CALL-into-Go wrappers need to compile. Naming mirrors
// the single-package codegen translator's conventions (lowercase
// field names, the M field as uppercase).
func sharedModuleSource(fixture string) string {
switch fixture {
case "arith":
return "package main\ntype Module struct{}\n"
case "cg_globals":
return `package main
type Module struct {
g0 int32
g1 int64
g2 float32
g3 float64
g4 int32
}
`
case "cg_indirect":
return `package main
type Module struct {
t0 []any
g0 int32
}
`
}
return "package main\ntype Module struct{}\n"
}
// TestEmitARM64Assembles cross-builds every fixture through the
// arm64 emitter on an amd64 host. The check is necessarily weaker
// than the amd64 end-to-end tests (the binary never runs), but it
// catches every category of typo go-vet's asm checker can spot:
// unknown mnemonics, mismatched operand shapes, mis-spelled
// condition codes, out-of-range immediates, etc. The fixtures
// exercise every emit path the amd64 tests do.
func TestEmitARM64Assembles(t *testing.T) {
type fix struct {
name string
moduleSrc string
}
for _, f := range []fix{
{"arith", "package main\ntype Module struct{}\n"},
{"control", "package main\ntype Module struct{}\n"},
{"cg_memops", driverWithMemory().moduleSrc},
{"cg_globals", driverWithGlobals().moduleSrc},
{"cg_manyfuncs", "package main\ntype Module struct{}\n"},
{"cg_indirect", driverIndirect().moduleSrc},
{"asmtest_callimport", driverHostAdd().moduleSrc},
{"cg_bulkmem", driverBulkMem().moduleSrc},
} {
t.Run(f.name, func(t *testing.T) {
assembleARM64(t, f.name, f.moduleSrc)
})
}
}
// TestBuildPackageFilesArith runs the BuildPackageFiles pipeline
// against arith.wat, writes the result into a temporary Go package
// alongside a hand-crafted module + helpers + dispatcher, and
// asserts that the package compiles and produces the same results
// per export as the existing per-function tests.
//
// This is the codegen-integration MVP: instead of the asmgen tests'
// per-test imperative emit loop, the bundle is produced by one call
// to BuildPackageFiles. The integration with codegen.Translate
// itself (auto-splitting the codegen output into shared / fallback /
// asm pieces) is a follow-up — for now the Module struct and helper
// functions are inlined in the test, matching what codegen would
// emit for this fixture (empty Module, two trapping division
// helpers, the rotation helper).
func TestBuildPackageFilesArith(t *testing.T) {
if runtime.GOARCH != "amd64" {
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
}
bin := testfixture.Wasm(t, "arith")
mod, err := wasm.Parse(bytes.NewReader(bin))
if err != nil {
t.Fatalf("parse: %v", err)
}
files, err := BuildPackageFiles(mod, BuildPackageOptions{Package: "main"})
if err != nil {
t.Fatalf("BuildPackageFiles: %v", err)
}
// Dispatcher: same shape as the other tests, plus a Module type
// and the helpers the asm bodies CALL.
exports := []string{"add", "sub", "mul64", "div_s", "shifts", "rotl", "lt_s", "lt_u"}
var dispatch strings.Builder
dispatch.WriteString(`switch os.Args[1] {`)
for _, name := range exports {
var sig wasm.FuncType
for _, e := range mod.Exports {
if e.Kind == wasm.ExportFunc && e.Name == name {
sig = mod.FuncTypeOf(e.Index)
}
}
dispatch.WriteString(dispatchCase(name, sig, "nil"))
}
dispatch.WriteString(`default: fmt.Fprintln(os.Stderr, "unknown export:", os.Args[1]); os.Exit(2) }`)
dir := t.TempDir()
mustWrite(t, dir, "go.mod", "module asmgentest\n\ngo 1.25\n")
for path, content := range files {
mustWrite(t, dir, path, content)
}
// Always-on shared file: empty Module struct + helpers the asm
// bodies CALL. The same Module appears in codegen's full Go
// fallback path; for this MVP we provide it directly so the
// integration question is "does the asm bundle compose with a
// drop-in Module / helpers file", which is the load-bearing
// invariant.
mustWrite(t, dir, "shared.go", "package main\ntype Module struct{}\n")
mustWrite(t, dir, "helpers.go", helpersGoSource())
mustWrite(t, dir, "main.go", `package main
import (
"fmt"
"os"
"strconv"
)
func main() {
if len(os.Args) < 2 {
fmt.Fprintln(os.Stderr, "usage: driver <export> args...")
os.Exit(2)
}
`+dispatch.String()+`
}
`)
exe := filepath.Join(dir, "driver")
cb := exec.Command("go", "build", "-o", exe, ".")
cb.Dir = dir
if out, err := cb.CombinedOutput(); err != nil {
t.Fatalf("go build: %v\n%s\n\n--- files ---\n%v", err, out, fileList(files))
}
for _, c := range []driverCase{
{"add", []string{"2", "3"}, "5"},
{"sub", []string{"10", "3"}, "7"},
{"mul64", []string{"6", "7"}, "42"},
{"div_s", []string{"20", "4"}, "5"},
{"rotl", []string{"1", "1"}, "2"},
{"lt_s", []string{"-1", "1"}, "1"},
{"lt_u", []string{"-1", "1"}, "0"},
} {
args := append([]string{c.export}, c.args...)
out, err := exec.Command(exe, args...).CombinedOutput()
if err != nil {
t.Errorf("%s(%v): %v\n%s", c.export, c.args, err, out)
continue
}
if got := strings.TrimSpace(string(out)); got != c.want {
t.Errorf("%s(%v) = %q, want %q", c.export, c.args, got, c.want)
}
}
}
func fileList(m map[string]string) string {
keys := make([]string, 0, len(m))
for k := range m {
keys = append(keys, k)
}
sort.Strings(keys)
return strings.Join(keys, ", ")
}
// TestEmitArithAMD64 is the 2.1 milestone: every export of arith.wat
// is lowered to amd64 asm and the asm is exercised through a small
// driver binary. The driver dispatches on the export name so all
// fixtures share one compile cycle — `go build` then a single binary
// invoked per case, instead of paying the toolchain cost per case
// via `go run`. The set of inputs per export covers signed, unsigned,
// and wrap-around edges.
// driverCase is one input/output expectation for the in-driver
// dispatcher. The dispatcher reads its export name and args from
// os.Args, calls into asm, prints the result, and the test compares
// stdout against want.
type driverCase struct {
export string
args []string
want string
}
// buildAndRunDriver lowers each export in the given fixture, emits
// amd64 asm, drops everything into a tmp Go module that builds into
// a single dispatcher binary, then runs the binary once per case
// and asserts the output. Used by every fixture-driven test.
//
// driver describes the per-fixture Module type and initialization
// (zero-sized placeholder for the arith / control fixtures; a real
// memory-backed Module for the memops fixture).
func buildAndRunDriver(t *testing.T, fixture string, exports []string, driver driverModule, cases []driverCase) {
t.Helper()
buildAndRunDriverArch(t, fixture, exports, driver, cases, "amd64")
}
// canExecDarwinARM64 reports whether this darwin host can execute a
// natively-built arm64 binary even though the test process itself may
// be an x86_64 build running under Rosetta 2 (Rosetta only exists on
// arm64 hardware, so a translated process implies an arm64 kernel).
// sysctl.proc_translated is the OS's own indicator: "1" translated,
// "0" native, absent on Intel hardware. The sysctl CLI is the foreign
// interface here (no structured API without cgo); the parse is total —
// anything other than a literal "1" means "not translated".
func canExecDarwinARM64() bool {
if runtime.GOOS != "darwin" {
return false
}
if runtime.GOARCH == "arm64" {
return true
}
out, err := exec.Command("sysctl", "-n", "sysctl.proc_translated").Output()
return err == nil && strings.TrimSpace(string(out)) == "1"
}
func buildAndRunDriverArch(t *testing.T, fixture string, exports []string, driver driverModule, cases []driverCase, goarch string) {
t.Helper()
bin := testfixture.Wasm(t, fixture)
mod, err := wasm.Parse(bytes.NewReader(bin))
if err != nil {
t.Fatalf("parse: %v", err)
}
// Default funcSymbol: export-name when the wasm function is
// exported, Fn<funcIdx> otherwise. The fixture's own driverModule
// can override.
exportNames := map[uint32]string{}
for _, e := range mod.Exports {
if e.Kind == wasm.ExportFunc {
exportNames[e.Index] = e.Name
}
}
funcSymbol := driver.funcSymbol
if funcSymbol == nil {
funcSymbol = func(idx uint32) string {
if n, ok := exportNames[idx]; ok {
return n
}
return fmt.Sprintf("Fn%d", idx)
}
}
// Emit every defined function. Direct callees that aren't
// exported still need an asm body so cross-function CALLs can
// resolve; emitting them all also makes the driver layout match
// the codegen translator's emission semantics.
//
// Functions whose ops aren't yet supported by the emitter get a
// Go-side stub that panics if called. The fixture's test cases
// pick which exports to exercise, so silently stubbing the un-
// implemented ones keeps the dispatcher buildable until the
// emitter catches up.
var asmBuf, declBuf, stubBuf strings.Builder
for i := range mod.Functions {
funcIdx := mod.NumImportedFuncs + uint32(i)
name := funcSymbol(funcIdx)
fn, err := lower.LowerFunction(mod, funcIdx, name, nil)
if err != nil {
t.Fatalf("lower %s (fn%d): %v", name, funcIdx, err)
}
sig := mod.FuncTypeOf(funcIdx)
emit := EmitFuncAMD64
if goarch == "arm64" {
emit = EmitFuncARM64
}
asm, decl, err := emit(name, sig, fn, FuncOptions{
ModulePkgRef: "*Module",
Module: mod,
FuncSymbol: funcSymbol,
})
if err != nil {
t.Logf("emit %s skipped (%v); stubbing", name, err)
fmt.Fprintf(&stubBuf, "func %s%s { panic(\"asm stub: %s\") }\n",
name, goSignature(sig, "*Module"), name)
continue
}
asmBuf.WriteString(asm)
declBuf.WriteString(decl)
}
var dispatch strings.Builder
dispatch.WriteString(`switch os.Args[1] {`)
for _, name := range exports {
var sig wasm.FuncType
found := false
for idx, ename := range exportNames {
if ename == name {
sig = mod.FuncTypeOf(idx)
found = true
break
}
}
if !found {
t.Fatalf("export %q missing in %s", name, fixture)
}
dispatch.WriteString(dispatchCase(name, sig, driver.modExpr))
}
dispatch.WriteString(`default: fmt.Fprintln(os.Stderr, "unknown export:", os.Args[1]); os.Exit(2) }`)
wrappers := ""
if driver.importWrappers != nil {
wrappers = driver.importWrappers(mod)
}
dir := t.TempDir()
writeDriverModuleArch(t, dir, driver, asmBuf.String(), declBuf.String(), stubBuf.String(), wrappers, dispatch.String(), goarch)
exe := filepath.Join(dir, "driver")
cb := exec.Command("go", "build", "-o", exe, ".")
cb.Dir = dir
if goarch != runtime.GOARCH {
// Cross-build for the host OS; the produced binary still runs
// natively (e.g. an arm64 binary on Apple Silicon even when the
// test process itself is x86_64 under Rosetta).
cb.Env = append(os.Environ(), "GOOS="+runtime.GOOS, "GOARCH="+goarch, "CGO_ENABLED=0")
}
if out, err := cb.CombinedOutput(); err != nil {
t.Fatalf("go build: %v\n%s\n\n--- asm ---\n%s", err, out, asmBuf.String())
}
for _, tc := range cases {
args := append([]string{tc.export}, tc.args...)
out, err := exec.Command(exe, args...).CombinedOutput()
if err != nil {
t.Errorf("%s(%v): %v\n%s", tc.export, tc.args, err, out)
continue
}
got := strings.TrimSpace(string(out))
if got != tc.want {
t.Errorf("%s(%v) = %q, want %q", tc.export, tc.args, got, tc.want)
}
}
}
// driverModule bundles the per-fixture pieces the driver template
// interpolates: the Module type declaration, the expression each
// dispatchCase substitutes for the m parameter (`nil` or a global
// `modPtr` pointing at an initialised instance), the callee-
// naming callback the asm emitter uses to resolve OpCallDirect
// targets, and the host-import wrapper generator. funcSymbol and
// importWrappers are nil when the fixture has no direct calls or
// no host imports respectively.
type driverModule struct {
moduleSrc string
modExpr string
funcSymbol func(funcIdx uint32) string
importWrappers func(mod *wasm.Module) string
}
// driverPlaceholder is the Module type for tests whose asm does
// not dereference the receiver pointer (no memory access, no
// globals). Zero size keeps it valid as a non-nil pointer source
// without pulling unsafe into the test code path.
var driverPlaceholder = driverModule{
moduleSrc: `package main
type Module struct{}
`,
modExpr: "nil",
}
// driverWithMemory builds a Module containing a real linear-memory
// slice with the cg_memops data segment pre-loaded. The Module's
// layout (Memory []byte; MaxMem uint64; M unsafe.Pointer) matches
// the codegen translator's emission, including the byte offset of
// the M field (32) hard-coded by the emitter as moduleMOffset.
func driverWithMemory() driverModule {
return driverModule{
moduleSrc: `package main
import "unsafe"
// Module matches the layout the codegen translator emits in multi-
// package mode: linear-memory slice header, max-pages tracker, and a
// cached data pointer. moduleMOffset in the emitter pins M's offset
// to 32; this struct must keep it stable.
type Module struct {
Memory []byte
MaxMem uint64
M unsafe.Pointer
}
var modPtr *Module
func init() {
mem := make([]byte, 2*65536) // 2 wasm pages — matches cg_memops.wat
copy(mem, []byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16})
modPtr = &Module{
Memory: mem,
MaxMem: 8 * 65536,
M: unsafe.Pointer(&mem[0]),
}
}
// memorySize / memoryGrow mirror codegen's helpers/helpers.go names
// because asmgen routes OpMemSize / OpMemGrow to those symbols
// (·memorySize(SB) / ·memoryGrow(SB)) — emitting duplicate copies
// in the asmgen wrappers would conflict with codegen's emission.
func memorySize(m *Module) int32 { return int32(len(m.Memory) / 65536) }
func memoryGrow(m *Module, delta int32) int32 {
if delta < 0 {
return -1
}
cur := int32(len(m.Memory) / 65536)
want := uint64(cur+delta) * 65536
if want > m.MaxMem {
return -1
}
grown := make([]byte, want)
copy(grown, m.Memory)
m.Memory = grown
m.M = unsafe.Pointer(&grown[0])
return cur
}
`,
modExpr: "modPtr",
}
}
func TestEmitArithAMD64(t *testing.T) {
if runtime.GOARCH != "amd64" {
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
}
exports := []string{"add", "sub", "mul64", "div_s", "shifts", "rotl", "lt_s", "lt_u"}
cases := []driverCase{
{"add", []string{"2", "3"}, "5"},
{"add", []string{"-5", "-7"}, "-12"},
{"add", []string{"2147483647", "1"}, "-2147483648"},
{"sub", []string{"10", "3"}, "7"},
{"sub", []string{"0", "1"}, "-1"},
{"sub", []string{"-2147483648", "1"}, "2147483647"},
{"mul64", []string{"6", "7"}, "42"},
{"mul64", []string{"-3", "5"}, "-15"},
{"mul64", []string{"4294967296", "4294967296"}, "0"},
{"div_s", []string{"20", "4"}, "5"},
{"div_s", []string{"-20", "4"}, "-5"},
{"div_s", []string{"-20", "-4"}, "5"},
{"shifts", []string{"1", "4"}, "1"},
{"shifts", []string{"7", "1"}, "7"},
{"rotl", []string{"1", "1"}, "2"},
{"rotl", []string{"1", "31"}, "-2147483648"},
{"rotl", []string{"-2147483648", "1"}, "1"},
{"lt_s", []string{"-1", "1"}, "1"},
{"lt_s", []string{"1", "-1"}, "0"},
{"lt_u", []string{"-1", "1"}, "0"},
{"lt_u", []string{"1", "-1"}, "1"},
}
buildAndRunDriver(t, "arith", exports, driverPlaceholder, cases)
}
// TestEmitControlAMD64 is the 2.2 milestone: multi-block CFGs with
// loops (fact, gcd), if/else value-merge (max), and br_table dispatch
// chains (switch3). The asm exercises BlockIf, BlockPlain, BlockRet,
// OpPhi via predecessor-edge copies, and the staged-phi cycle breaker
// on loop back-edges.
func TestEmitControlAMD64(t *testing.T) {
if runtime.GOARCH != "amd64" {
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
}
exports := []string{"max", "fact", "gcd", "switch3"}
cases := []driverCase{
// max(a, b) = a > b ? a : b
{"max", []string{"3", "5"}, "5"},
{"max", []string{"5", "3"}, "5"},
{"max", []string{"-1", "1"}, "1"},
{"max", []string{"42", "42"}, "42"},
// fact(n) = n! for n >= 0; n*0 ⇒ result 0 because fact's loop
// uses r := 1 initially but the multiplication keeps using i
// down to 0, then exits — wait, let me re-derive:
// r := 1; i := n
// while i != 0: r *= i; i--
// return r
// fact(0)=1, fact(1)=1, fact(5)=120, fact(7)=5040.
{"fact", []string{"0"}, "1"},
{"fact", []string{"1"}, "1"},
{"fact", []string{"5"}, "120"},
{"fact", []string{"7"}, "5040"},
// gcd(a, b) by Euclid's mod algorithm.
{"gcd", []string{"48", "18"}, "6"},
{"gcd", []string{"54", "24"}, "6"},
{"gcd", []string{"17", "5"}, "1"},
{"gcd", []string{"100", "0"}, "100"},
// switch3 — br_table dispatch chain.
{"switch3", []string{"0"}, "100"},
{"switch3", []string{"1"}, "200"},
{"switch3", []string{"2"}, "300"},
{"switch3", []string{"3"}, "999"},
{"switch3", []string{"42"}, "999"},
}
buildAndRunDriver(t, "control", exports, driverPlaceholder, cases)
}
// TestEmitArithARM64 runs the arith fixture's value checks against
// the ARM64 emitter natively — in particular the inline div/rem
// family (SDIV/UDIV + MSUB with explicit trap branches).
func TestEmitArithARM64(t *testing.T) {
if !canExecDarwinARM64() && runtime.GOARCH != "arm64" {
t.Skipf("host cannot execute arm64 binaries (GOOS=%s GOARCH=%s)", runtime.GOOS, runtime.GOARCH)
}
exports := []string{"add", "sub", "mul64", "div_s", "shifts", "rotl", "lt_s", "lt_u"}
cases := []driverCase{
{"add", []string{"2", "3"}, "5"},
{"sub", []string{"-2147483648", "1"}, "2147483647"},
{"mul64", []string{"6", "7"}, "42"},
{"div_s", []string{"20", "4"}, "5"},
{"div_s", []string{"-20", "4"}, "-5"},
{"div_s", []string{"-20", "-4"}, "5"},
{"div_s", []string{"-2147483647", "-1"}, "2147483647"},
{"shifts", []string{"1", "4"}, "1"},
{"rotl", []string{"1", "31"}, "-2147483648"},
{"lt_s", []string{"-1", "1"}, "1"},
{"lt_u", []string{"-1", "1"}, "0"},
}
buildAndRunDriverArch(t, "arith", exports, driverPlaceholder, cases, "arm64")
}
// TestEmitControlARM64 runs the same control-flow value checks against
// the ARM64 emitter, executing the driver natively (Apple Silicon runs
// arm64 binaries even when the test process is x86_64 under Rosetta).
// This is the arm64 value-level gate for BlockIf/phi edge-copies and
// the br_table compare chain — the assemble-only cross-build tests
// can't catch a wrong-branch or wrong-operand emission.
func TestEmitControlARM64(t *testing.T) {
if !canExecDarwinARM64() && runtime.GOARCH != "arm64" {
t.Skipf("host cannot execute arm64 binaries (GOOS=%s GOARCH=%s)", runtime.GOOS, runtime.GOARCH)
}
exports := []string{"max", "fact", "gcd", "switch3"}
cases := []driverCase{
{"max", []string{"3", "5"}, "5"},
{"max", []string{"5", "3"}, "5"},
{"max", []string{"-1", "1"}, "1"},
{"max", []string{"42", "42"}, "42"},
{"fact", []string{"0"}, "1"},
{"fact", []string{"1"}, "1"},
{"fact", []string{"5"}, "120"},
{"fact", []string{"7"}, "5040"},
{"gcd", []string{"48", "18"}, "6"},
{"gcd", []string{"54", "24"}, "6"},
{"gcd", []string{"17", "5"}, "1"},
{"gcd", []string{"100", "0"}, "100"},
{"switch3", []string{"0"}, "100"},
{"switch3", []string{"1"}, "200"},
{"switch3", []string{"2"}, "300"},
{"switch3", []string{"3"}, "999"},
{"switch3", []string{"42"}, "999"},
}
buildAndRunDriverArch(t, "control", exports, driverPlaceholder, cases, "arm64")
}
// TestEmitLoopPhiBlockIfAMD64 is a regression for a BlockIf-with-phi
// bug in the asm emitter: the predecessor-end phi edge-copies were
// emitted unconditionally before the conditional jump, so the loop's
// back-edge phi target (e.g. `p = mem[p+20]`) clobbered the loop
// variable even on the exit-edge fall-through. Symptoms surfaced in
// the googlesql dlmalloc tree-walk inside Fn39263 (free()): the walk
// exited with p = 0 instead of the last live tree node, downstream
// code observed empty smallbins, and subsequent mallocs returned the
// wrong chunk — eventually loading a ~4 GiB out-of-bounds address
// and SIGSEGVing. Pure-Go didn't hit this because emit_ssa.go places
// phi assigns INSIDE the if/else branches; the asm path is now fixed
// to mirror that structure.
//
// The test constructs the smallest SSA shape that exhibits the bug —
// a single block whose terminator is a BlockIf with one self-loop
// successor and one exit successor, hosting a phi whose back-edge
// value differs from its entry value — then emits the asm body and
// asserts (a) that the back-edge phi copy is routed through a
// per-branch intermediate label (proving conditional emission) and
// (b) that the exit-edge does NOT execute that copy.
//
// We can't trigger the buggy SSA shape from a stock .wat fixture:
// every wat structure we tried either splits the loop body into a
// separate BlockPlain (where unconditional copies are harmless) or
// uses `local.tee` (which SSA-aliases the loop variable with the
// loaded value, so even the bug-free output observes p = 0 at exit).
// Building the SSA directly with FuncBuilder gives a faithful repro
// without depending on the optimizer's blocking decisions.
func TestEmitLoopPhiBlockIfAMD64(t *testing.T) {
if runtime.GOARCH != "amd64" {
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
}
// SSA: f(start i32) i32 {
// b0: jmp b1 // entry; gets phi.in = start
// b1 (BlockIf, hosts phi p):
// p = phi(start from b0, p_next from b1)
// p_next = load[p+20]
// if p_next != 0 -> b1 (self), else b2
// b2 (BlockRet): return p
// }
// On the b1->b1 back edge, phi(p) = p_next. On the b1->b2 exit
// edge, phi values are not relevant (b2 has no phis), so the
// back-edge copy must NOT fire on exit. The buggy emitter wrote
// the back-edge copy at b1's terminator unconditionally,
// clobbering p before the test — surfacing as p = (last load,
// = 0) at b2 instead of the last live value.
sig := wasm.FuncType{Params: []wasm.ValType{wasm.ValI32}, Results: []wasm.ValType{wasm.ValI32}}
b := ssa.NewFuncBuilder("loopphi", ssa.FuncSig{Params: []ssa.Type{ssa.TypeI32}, Results: []ssa.Type{ssa.TypeI32}})
b0 := b.NewBlock(ssa.BlockPlain)
b1 := b.NewBlock(ssa.BlockIf)
b2 := b.NewBlock(ssa.BlockRet)
b.SetEntry(b0)
b.SetCurrent(b0)
start := b.Param(0, ssa.TypeI32)
ssa.AddEdge(b0, b1)
b.SetCurrent(b1)
// Phi p — two preds: b0 (start) and b1 (p_next, filled in after
// p_next is created). Phi must come first in the block per the
// SSA verifier's "phis first" rule, so we emit it before any
// other value and patch Args[1] once the back-edge value exists.
pPhi := b.NewValue(ssa.OpPhi, ssa.TypeI32, start, nil)
off20 := b.Const32(20)
addr := b.NewValue(ssa.OpAdd32, ssa.TypeI32, pPhi, off20)
pNext := b.NewValueAuxInt(ssa.OpLoad32, ssa.TypeI32, 0, addr)
pPhi.Args[1] = pNext
zero := b.Const32(0)
isNonZero := b.NewValue(ssa.OpNe32, ssa.TypeBool, pNext, zero)
b.LinkIf(isNonZero, b1, b2) // then: self (back-edge), else: exit
b.SetCurrent(b2)
b.FinishRet(pPhi)
if err := ssa.Verify(b.Func()); err != nil {
t.Fatalf("SSA verify: %v", err)
}
asm, _, err := EmitFuncAMD64("loopphi", sig, b.Func(), FuncOptions{ModulePkgRef: "*Module"})
if err != nil {
t.Fatalf("EmitFuncAMD64: %v", err)
}
// Load-bearing property (independent of which allocator chose
// the phi destination home — the loop-carry coalesce pass used
// the PHI_<id>_then intermediate label, the cross-block
// allocator may inline the copy directly into the back-edge
// branch's body): the back-edge
// phi update must NOT fire on the exit path. Concretely:
//
// - Look for the conditional jump that exits the loop. In our
// CFG it lands at the BlockRet successor (b2). The block
// code immediately following that label MUST NOT touch the
// phi destination's slot.
// - The back-edge branch's body (the non-exit successor) MUST
// update the phi slot from pNext's slot (a `MOVL <pNext>(SP),
// <reg>; MOVL <reg>, <phi-slot>(SP)` pair OR a direct slot-
// to-slot copy via a scratch register).
//
// We approximate this by checking the asm's structural property:
// no `MOVL <reg>, <slot>(SP)` instruction that targets a slot
// the BlockRet emit reads from sits BEFORE the conditional
// jump (= the buggy pattern), AND at least one such instruction
// sits AFTER it (= the back-edge copy).
//
// The presence-or-absence of intermediate PHI_<id>_then labels
// is no longer load-bearing — it's an implementation choice the
// allocator (loop-carry coalesce vs cross-block allocator) makes independently.
condIdx := strings.Index(asm, "JNE L")
if condIdx == -1 {
condIdx = strings.Index(asm, "JE L")
}
if condIdx == -1 {
t.Fatalf("expected at least one conditional jump in the BlockIf, got:\n%s", asm)
}
// The phi value's slot is `0(SP)` or some other small offset
// (the BlockRet reads it as the return value). We detect the
// back-edge copy as "any MOVL ..., <small>(SP)" between the
// conditional jump and the end of the asm. As long as such a
// copy exists AFTER the conditional, the back-edge update is in
// place — exactly the property the original bug fix was
// guaranteeing.
postCond := asm[condIdx:]
if !strings.Contains(postCond, "(SP)") {
t.Errorf("expected a slot-update MOV after the conditional jump (back-edge phi copy), got:\n%s", asm)
}
}
// TestRegallocLoopCarryPhiArg is a regression for the block-local
// regalloc's loop-carry miscount that surfaced in the integration
// corpus as a bignumeric multiplication producing 0 instead of the expected
// 39-digit product (TestQuery/cast_numeric_and_bignumeric). The
// failing function — Fn39775 — was a tight decrement-and-compare
// loop whose SSA shape was:
//
// block 0 (Plain): jmp b1
// block 1 (BlockIf):
// p_phi = phi(start, p_next) // pred 0 = b0, pred 1 = b3
// p_next = OpSub32(p_phi, 1)
// cond = OpLtS32(p_next, 0)
// if cond -> b2 (exit), else -> b3 (continue)
// block 2 (BlockRet): return p_phi
// block 3 (Plain): jmp b1 // back edge
//
// p_next is the loop carry: defined in b1, referenced by p_phi.Args[1]
// (the b3-edge arg). For SSA validity it must be alive at end of b3,
// which means its value crosses block boundaries on the b1 → b3 → b1
// loop. The block-local linear-scan regalloc must therefore NOT
// assign it a register — the per-block scan does not yet coordinate
// register state across the back edge.
//
// Before the fix the cross-block sweep only walked blocks OTHER than
// the one whose values it was assigning regHomes to. p_next's only
// "use" outside b1's own values was p_phi.Args[1], and p_phi is in b1
// itself — so the sweep missed it and the regalloc gave p_next a
// register, which was then reused by another block before the back-
// edge phi-copy ran. The Step 3a sweep introduced by the fix walks
// blk's OWN phis and marks any arg defined in blk as cross-block,
// catching exactly this case. This test reconstructs the smallest
// SSA shape that exhibits it and asserts that the regalloc declines
// to assign a register to the loop-carry value.
func TestRegallocLoopCarryPhiArg(t *testing.T) {
if runtime.GOARCH != "amd64" {
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
}
sig := wasm.FuncType{Params: []wasm.ValType{wasm.ValI32}, Results: []wasm.ValType{wasm.ValI32}}
fsig := ssa.FuncSig{Params: []ssa.Type{ssa.TypeI32}, Results: []ssa.Type{ssa.TypeI32}}
b := ssa.NewFuncBuilder("loopcarry", fsig)
b0 := b.NewBlock(ssa.BlockPlain)
b1 := b.NewBlock(ssa.BlockIf)
b2 := b.NewBlock(ssa.BlockRet)
b3 := b.NewBlock(ssa.BlockPlain)
b.SetEntry(b0)
// block 0 — entry, single jmp to b1 with `start` available.
b.SetCurrent(b0)
start := b.Param(0, ssa.TypeI32)
b.LinkPlain(b1)
// block 1 — the loop head. Phi sits at the top per the SSA
// verifier's "phis first" rule; its back-edge arg is patched in
// after p_next is built (the value didn't exist when the phi was
// created). p_next is the SOLE user of p_phi — that is the
// pattern the coalesce pass is allowed to fire on. Returning
// p_next (rather than p_phi) from the exit block keeps the
// "single user" invariant intact.
b.SetCurrent(b1)
pPhi := b.NewValue(ssa.OpPhi, ssa.TypeI32, start, nil)
one := b.Const32(1)
pNext := b.NewValue(ssa.OpSub32, ssa.TypeI32, pPhi, one)
pPhi.Args[1] = pNext
zero := b.Const32(0)
cond := b.NewValue(ssa.OpLtS32, ssa.TypeBool, pNext, zero)
b.LinkIf(cond, b2, b3) // then: exit, else: continue (back-edge through b3)
// block 3 — the back edge. Empty body; just jumps to b1.
b.SetCurrent(b3)
b.LinkPlain(b1)
// block 2 — return p_next (NOT p_phi — see the comment above).
b.SetCurrent(b2)
b.FinishRet(pNext)
if err := ssa.Verify(b.Func()); err != nil {
t.Fatalf("SSA verify: %v", err)
}
plan, err := planFunc(b.Func(), FuncOptions{ModulePkgRef: "*Module"}, sig, 0, false)
if err != nil {
t.Fatalf("planFunc: %v", err)
}
computeRegHomes(b.Func(), plan)
// Expectation: pPhi and pNext are now COALESCED into the same
// reserved register. The loop body (b1, b3) is CALL-free, the
// back-edge arg (pNext) is defined inside the loop, and the
// coalesce pass should pick a register from coalesceReservedPool
// ({R12, R13, R15}). Before the coalesce pass landed both were
// forced slot-resident — keeping the carry in a register across
// the loop body is the whole point of the pass.
pNextHome := plan.regHome[pNext.ID]
pPhiHome := plan.regHome[pPhi.ID]
if pNextHome == "" {
t.Errorf("loop-carry value pNext (v%d, OpSub32) has no regHome; expected the "+
"loop-carry coalesce pass to reserve a register from %v for the carry",
pNext.ID, coalesceReservedPool)
}
if pPhiHome == "" {
t.Errorf("phi pPhi (v%d) has no regHome; expected the loop-carry coalesce pass "+
"to give it the same reserved register as its back-edge arg", pPhi.ID)
}
if pNextHome != "" && pPhiHome != "" && pNextHome != pPhiHome {
t.Errorf("loop-carry coalesce should give pPhi and pNext the SAME register; "+
"got pPhi=%q, pNext=%q", pPhiHome, pNextHome)
}
// The phi destination must also be entered in coalescedPhi so
// emitPhiEdgeCopies knows to short-circuit the back-edge copy.
if reg := plan.coalescedPhi[pPhi.ID]; reg == "" {
t.Errorf("pPhi (v%d) is regHomed but not in plan.coalescedPhi; emitPhiEdgeCopies "+
"won't take the no-op back-edge path", pPhi.ID)
} else if reg != pPhiHome {
t.Errorf("plan.coalescedPhi[pPhi]=%q does not match plan.regHome[pPhi]=%q",
reg, pPhiHome)
}
// The coalesced register must be RESERVED in every block of the
// loop body so per-block regalloc does not steal it. The body
// for this fixture is {b1, b3} (b0 is the entry and b2 the exit
// — neither is on the back-edge path).
for _, blk := range []*ssa.Block{b1, b3} {
if !plan.reservedRegs[blk.ID][pPhiHome] {
t.Errorf("coalesced register %q not reserved in loop-body block %d",
pPhiHome, blk.ID)
}
}
}
// TestRegTrackInvalidatesOnByteWrite is a regression for a
// follow-on bug from regHome-aware emit: emitCmp32 with a
// register-home destination emits
//
// MOVL <v_slot>, AX ; AX = v
// CMPL AX, <other>
// SETEQ AL ; <-- writes AL = low byte of AX, leaves
// ; upper 24 bits as the original v's bits.
// MOVBLZX AL, <home> ; clean bool into home
//
// Without the byte-aware invalidation, regTrackPass kept the
// AX ↔ v_slot mapping established by the initial MOVL alive past
// the SETEQ, so a subsequent slot read like `MOVL <v_slot>, R9`
// got rewritten to `MOVL AX, R9` — reading AX whose low byte had
// been replaced by the SET<cc>'s 0/1 result. Symptoms in the
// googlesql bundle were a nil-pointer panic during module init
// (TestDateParamRoundTrip et al), narrowed by bisection to a
// single function (Fn24994) that emitted exactly this pattern.
//
// The fix is in regtrack.go: when the pass sees `SET<cc> AL/BL/
// CL/DL/...` it now invalidates the byte's 64-bit parent register
// (AX/BX/CX/DX/...) in its slot-tracking maps. This test pipes a
// minimal asm string through regTrackPass and asserts that the
// post-pass output does NOT collapse the second `MOVL slot, REG`
// to `MOVL AX, REG`.
func TestRegTrackInvalidatesOnByteWrite(t *testing.T) {
in := strings.Join([]string{
"\tMOVL 100(SP), AX", // AX = slot100
"\tCMPL AX, $42", // flags only
"\tSETEQ AL", // <-- corrupts low byte of AX
"\tMOVBLZX AL, R10", // R10 = clean bool
"\tMOVL 100(SP), R11", // would-be peephole forward target
"",
}, "\n")
out := regTrackPass(in)
if strings.Contains(out, "\tMOVL AX, R11") {
t.Errorf("regTrackPass forwarded AX to R11 after a SETEQ AL clobbered "+
"AX's low byte. The second load must stay a memory read.\nOutput:\n%s", out)
}
// Sanity: the second slot load should still be present (not
// dropped, just not rewritten).
if !strings.Contains(out, "\tMOVL 100(SP), R11") {
t.Errorf("regTrackPass dropped the second slot load entirely; expected "+
"it to survive verbatim. Output:\n%s", out)
}
}
// TestRegTrackByteRegParent exercises byteRegParent's mapping
// table — a small data table whose entries are easy to forget when
// new byte register names are added. Each well-known plan9 byte
// register name should map back to its 64-bit parent; anything
// outside the table returns (_, false).
func TestRegTrackByteRegParent(t *testing.T) {
cases := []struct {
byteName, parent string
}{
{"AL", "AX"}, {"AH", "AX"},
{"BL", "BX"}, {"BH", "BX"},
{"CL", "CX"}, {"CH", "CX"},
{"DL", "DX"}, {"DH", "DX"},
{"SIL", "SI"}, {"DIL", "DI"}, {"BPL", "BP"},
{"R8B", "R8"}, {"R9B", "R9"},
{"R10B", "R10"}, {"R11B", "R11"},
{"R12B", "R12"}, {"R13B", "R13"},
{"R14B", "R14"}, {"R15B", "R15"},
}
for _, c := range cases {
got, ok := byteRegParent(c.byteName)
if !ok {
t.Errorf("byteRegParent(%q) returned ok=false; want parent=%q", c.byteName, c.parent)
continue
}
if got != c.parent {
t.Errorf("byteRegParent(%q) = %q; want %q", c.byteName, got, c.parent)
}
}
// Negative case: a non-byte name returns (_, false).
if _, ok := byteRegParent("AX"); ok {
t.Errorf("byteRegParent(%q) = (_, true); want false (not a byte reg)", "AX")
}
}
// TestEmitMemopsAMD64 is the 2.3 milestone: linear-memory load /
// store ops at every wasm width, exercised through cg_memops.wat.
// The driver instantiates a real Module with two pages of memory and
// the fixture's data segment pre-loaded; the asm dereferences m.M
// (cached unsafe.Pointer) at the offset moduleMOffset to compute
// effective addresses. Float load/store ops are emitted but not
// asserted by name here — the prototype focuses on the integer
// path that the SQL parser actually relies on. mem_size / mem_grow
// are not yet implemented in the emitter and their exports are
// omitted.
func TestEmitMemopsAMD64(t *testing.T) {
if runtime.GOARCH != "amd64" {
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
}
exports := []string{
"l_i32", "l_i32_8s", "l_i32_8u", "l_i32_16s", "l_i32_16u",
"l_i64", "l_i64_8s", "l_i64_8u", "l_i64_16s", "l_i64_16u", "l_i64_32s", "l_i64_32u",
"s_i32", "s_i32_8", "s_i32_16",
"s_i64", "s_i64_8", "s_i64_16", "s_i64_32",
"l_offset", "s_offset",
"mem_size", "mem_grow",
}
// Data segment: bytes [1..16] at offset 0. Driver init() repeats
// the same content.
cases := []driverCase{
// --- i32 loads ---
{"l_i32", []string{"0"}, "67305985"}, // LE [1,2,3,4] = 0x04030201
{"l_i32", []string{"4"}, "134678021"}, // LE [5,6,7,8] = 0x08070605
{"l_i32_8u", []string{"0"}, "1"},
{"l_i32_8u", []string{"7"}, "8"},
{"l_i32_8s", []string{"0"}, "1"},
{"l_i32_8s", []string{"7"}, "8"},
{"l_i32_16u", []string{"0"}, "513"}, // 0x0201
{"l_i32_16s", []string{"0"}, "513"},
// --- i64 loads ---
{"l_i64", []string{"0"}, "578437695752307201"}, // 0x0807060504030201
{"l_i64_8u", []string{"0"}, "1"},
{"l_i64_8s", []string{"15"}, "16"},
{"l_i64_16u", []string{"0"}, "513"},
{"l_i64_16s", []string{"0"}, "513"},
{"l_i64_32u", []string{"0"}, "67305985"},
{"l_i64_32s", []string{"0"}, "67305985"},
// --- i32 store→load round trips ---
{"s_i32", []string{"100", "305419896"}, "305419896"}, // 0x12345678
{"s_i32", []string{"104", "-1"}, "-1"},
{"s_i32_8", []string{"100", "255"}, "255"},
{"s_i32_16", []string{"100", "65535"}, "65535"},
// --- i64 store→load round trips ---
{"s_i64", []string{"200", "1311768467463790320"}, "1311768467463790320"}, // 0x123456789ABCDEF0
{"s_i64_8", []string{"200", "255"}, "255"},
{"s_i64_16", []string{"200", "65535"}, "65535"},
{"s_i64_32", []string{"200", "4294967295"}, "4294967295"},
// --- offset memarg ---
{"l_offset", []string{"0"}, "202050057"}, // offset=8: [9..12] = 0x0C0B0A09
{"s_offset", []string{"300", "42"}, "42"}, // offset=12: writes at 312, reads at 312
// memory.size / memory.grow — each test binary starts fresh,
// so the page count begins at 2 (the .wat declares 2 pages).
{"mem_size", []string{}, "2"},
{"mem_grow", []string{"3"}, "2"}, // grow by 3 → returns old size = 2
{"mem_grow", []string{"-1"}, "-1"},
}
buildAndRunDriver(t, "cg_memops", exports, driverWithMemory(), cases)
}
// TestEmitMemopsARM64 runs the same wasm32 memops value checks
// against the ARM64 emitter, executing natively (see
// TestEmitControlARM64 for the Rosetta-host detection).
func TestEmitMemopsARM64(t *testing.T) {
if !canExecDarwinARM64() && runtime.GOARCH != "arm64" {
t.Skipf("host cannot execute arm64 binaries (GOOS=%s GOARCH=%s)", runtime.GOOS, runtime.GOARCH)
}
exports := []string{
"l_i32", "l_i32_8s", "l_i32_8u", "l_i32_16s", "l_i32_16u",
"l_i64", "l_i64_8s", "l_i64_8u", "l_i64_16s", "l_i64_16u", "l_i64_32s", "l_i64_32u",
"s_i32", "s_i32_8", "s_i32_16",
"s_i64", "s_i64_8", "s_i64_16", "s_i64_32",
"l_offset", "s_offset",
"mem_size", "mem_grow",
}
cases := []driverCase{
{"l_i32", []string{"0"}, "67305985"},
{"l_i32", []string{"4"}, "134678021"},
{"l_i32_8u", []string{"7"}, "8"},
{"l_i32_8s", []string{"7"}, "8"},
{"l_i32_16u", []string{"0"}, "513"},
{"l_i32_16s", []string{"0"}, "513"},
{"l_i64", []string{"0"}, "578437695752307201"},
{"l_i64_8u", []string{"0"}, "1"},
{"l_i64_8s", []string{"15"}, "16"},
{"l_i64_16u", []string{"0"}, "513"},
{"l_i64_16s", []string{"0"}, "513"},
{"l_i64_32u", []string{"0"}, "67305985"},
{"l_i64_32s", []string{"0"}, "67305985"},
{"s_i32", []string{"100", "305419896"}, "305419896"},
{"s_i32", []string{"104", "-1"}, "-1"},
{"s_i32_8", []string{"100", "255"}, "255"},
{"s_i32_16", []string{"100", "65535"}, "65535"},
{"s_i64", []string{"200", "1311768467463790320"}, "1311768467463790320"},
{"s_i64_8", []string{"200", "255"}, "255"},
{"s_i64_16", []string{"200", "65535"}, "65535"},
{"s_i64_32", []string{"200", "4294967295"}, "4294967295"},
{"l_offset", []string{"0"}, "202050057"},
{"s_offset", []string{"300", "42"}, "42"},
{"mem_size", []string{}, "2"},
{"mem_grow", []string{"3"}, "2"},
{"mem_grow", []string{"-1"}, "-1"},
}
buildAndRunDriverArch(t, "cg_memops", exports, driverWithMemory(), cases, "arm64")
}
// driverWithMemory64 is driverWithMemory with the 64-bit memory
// helper family (memorySize64 / memoryGrow64) that asmgen routes
// OpMemSize / OpMemGrow to on memory64 modules. Module layout is
// identical — only the addressing width of the wasm side changes.
func driverWithMemory64() driverModule {
d := driverWithMemory()
d.moduleSrc = strings.Replace(d.moduleSrc, `func memorySize(m *Module) int32 { return int32(len(m.Memory) / 65536) }
func memoryGrow(m *Module, delta int32) int32 {`, `func memorySize64(m *Module) int64 { return int64(len(m.Memory) / 65536) }
func memoryGrow64(m *Module, delta int64) int64 {`, 1)
d.moduleSrc = strings.Replace(d.moduleSrc, ` cur := int32(len(m.Memory) / 65536)`, ` cur := int64(len(m.Memory) / 65536)`, 1)
return d
}
// mem64MemopsCases is the memory64 twin of the TestEmitMemopsAMD64
// case list: same data segment, same expected values, i64 addresses.
// mem_size / mem_grow return i64 on a 64-bit memory.
func mem64MemopsCases() ([]string, []driverCase) {
exports := []string{
"l_i32", "l_i32_8s", "l_i32_8u", "l_i32_16s", "l_i32_16u",
"l_i64", "l_i64_8s", "l_i64_8u", "l_i64_16s", "l_i64_16u", "l_i64_32s", "l_i64_32u",
"s_i32", "s_i32_8", "s_i32_16",
"s_i64", "s_i64_8", "s_i64_16", "s_i64_32",
"l_offset", "s_offset", "l_constbase",
"mem_size", "mem_grow",
}
cases := []driverCase{
{"l_i32", []string{"0"}, "67305985"},
{"l_i32", []string{"4"}, "134678021"},
{"l_i32_8u", []string{"0"}, "1"},
{"l_i32_8u", []string{"7"}, "8"},
{"l_i32_8s", []string{"0"}, "1"},
{"l_i32_8s", []string{"7"}, "8"},
{"l_i32_16u", []string{"0"}, "513"},
{"l_i32_16s", []string{"0"}, "513"},
{"l_i64", []string{"0"}, "578437695752307201"},
{"l_i64_8u", []string{"0"}, "1"},
{"l_i64_8s", []string{"15"}, "16"},
{"l_i64_16u", []string{"0"}, "513"},
{"l_i64_16s", []string{"0"}, "513"},
{"l_i64_32u", []string{"0"}, "67305985"},
{"l_i64_32s", []string{"0"}, "67305985"},
{"s_i32", []string{"100", "305419896"}, "305419896"},
{"s_i32", []string{"104", "-1"}, "-1"},
{"s_i32_8", []string{"100", "255"}, "255"},
{"s_i32_16", []string{"100", "65535"}, "65535"},
{"s_i64", []string{"200", "1311768467463790320"}, "1311768467463790320"},
{"s_i64_8", []string{"200", "255"}, "255"},
{"s_i64_16", []string{"200", "65535"}, "65535"},
{"s_i64_32", []string{"200", "4294967295"}, "4294967295"},
{"l_offset", []string{"0"}, "202050057"},
{"s_offset", []string{"300", "42"}, "42"},
// constant i64 base folded with the memarg offset at
// generation time: bytes [13..16] LE = 0x100F0E0D.
{"l_constbase", []string{}, "269422093"},
{"mem_size", []string{}, "2"},
{"mem_grow", []string{"3"}, "2"},
{"mem_grow", []string{"-1"}, "-1"},
}
return exports, cases
}
// TestEmitMemopsMem64AMD64 / ...ARM64 run the memory64 twin of the
// memops value checks: i64 bases with no mod-2^32 wrap, offsets added
// in full 64-bit, and the 64-bit mem-size/grow helper routing.
func TestEmitMemopsMem64AMD64(t *testing.T) {
if runtime.GOARCH != "amd64" {
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
}
exports, cases := mem64MemopsCases()
buildAndRunDriver(t, "cg_mem64_memops", exports, driverWithMemory64(), cases)
}
func TestEmitMemopsMem64ARM64(t *testing.T) {
if !canExecDarwinARM64() && runtime.GOARCH != "arm64" {
t.Skipf("host cannot execute arm64 binaries (GOOS=%s GOARCH=%s)", runtime.GOOS, runtime.GOARCH)
}
exports, cases := mem64MemopsCases()
buildAndRunDriverArch(t, "cg_mem64_memops", exports, driverWithMemory64(), cases, "arm64")
}
// driverHostAdd is the driver for the OpCallImport fixture. The
// Module carries an interface field `host` (hostImports) whose
// implementation adds the integer args (i32 and i64 variants) and
// records the noop calls so the test can assert side-effects.
func driverHostAdd() driverModule {
return driverModule{
moduleSrc: `package main
// hostImports is the wasm-side imports interface the asm
// wrappers dispatch through. Method shape: the host always
// receives *Module as its first parameter so it can reach back
// into memory if it needs to.
type hostImports interface {
Add(m *Module, a, b int32) int32
Add64(m *Module, a, b int64) int64
Noop(m *Module, x int32)
}
// Module is the host-side runtime container. The asm reads m
// directly at FP+0 (it doesn't dereference it for this fixture
// because no memory access goes through it), but the import
// wrappers do.
type Module struct {
host hostImports
}
type hostImpl struct{ noopSeen int32 }
func (h *hostImpl) Add(_ *Module, a, b int32) int32 { return a + b }
func (h *hostImpl) Add64(_ *Module, a, b int64) int64 { return a + b }
func (h *hostImpl) Noop(_ *Module, x int32) { h.noopSeen = x }
var modPtr *Module
func init() { modPtr = &Module{host: &hostImpl{}} }
`,
modExpr: "modPtr",
importWrappers: func(mod *wasm.Module) string {
// Emit one wrapper per ImportFunc in declaration order.
// The funcIdx assigned by wasm matches the loop counter
// because we only walk ImportFunc entries.
var b strings.Builder
var fIdx uint32
for _, imp := range mod.Imports {
if imp.Kind != wasm.ImportFunc {
continue
}
sig := mod.Types[imp.TypeIdx]
fmt.Fprintf(&b, "func callImport_%d", fIdx)
b.WriteString(goSignature(sig, "*Module"))
b.WriteString(" {\n\t")
if len(sig.Results) > 0 {
b.WriteString("return ")
}
// host.<MethodName>(m, args...) — MangleID + capitalize.
method := capitalizeImport(imp.Name)
fmt.Fprintf(&b, "m.host.%s(m", method)
for i := range sig.Params {
fmt.Fprintf(&b, ", l%d", i)
}
b.WriteString(")\n}\n")
fIdx++
}
return b.String()
},
}
}
// capitalizeImport mirrors codegen's importMethodName for plain
// ASCII names (which is all the fixture uses): capitalise the
// first letter so the method satisfies an interface declared by
// the test driver. Real codegen routes through MangleID for non-
// ASCII / C++-mangled names; we don't need that for the fixture.
func capitalizeImport(s string) string {
if s == "" {
return s
}
r := []byte(s)
if r[0] >= 'a' && r[0] <= 'z' {
r[0] = r[0] - 'a' + 'A'
}
return string(r)
}
// driverWithGlobals builds a Module that carries the cg_globals
// fixture's five globals. The driver also generates loadGlobal_N /
// storeGlobal_N wrappers per defined global so OpGlobalGet/Set in
// asm can route through them.
func driverWithGlobals() driverModule {
moduleSrc := `package main
// Module field names mirror the codegen translator's emission:
// g<idx> (lowercase in single-package mode). The asm-side wrappers
// access these fields by name so any layout change is caught at Go
// compile time.
type Module struct {
g0 int32 // gi32 (mut)
g1 int64 // gi64 (mut)
g2 float32 // gf32 (mut)
g3 float64 // gf64 (mut)
g4 int32 // cimmu (immut)
}
var modPtr *Module
func init() { modPtr = &Module{g0: 11, g1: 22, g2: 1.5, g3: 2.5, g4: 99} }
// Per-global wrappers (one load + one store per global) — declared
// here rather than emitted by the importWrappers callback because
// the cg_globals fixture has no host imports. A future asmgen
// integration with the codegen translator would unify the two
// wrapper emit paths under one Driver-aware generator.
func loadGlobal_0(m *Module) int32 { return m.g0 }
func storeGlobal_0(m *Module, v int32) { m.g0 = v }
func loadGlobal_1(m *Module) int64 { return m.g1 }
func storeGlobal_1(m *Module, v int64) { m.g1 = v }
func loadGlobal_2(m *Module) float32 { return m.g2 }
func storeGlobal_2(m *Module, v float32){ m.g2 = v }
func loadGlobal_3(m *Module) float64 { return m.g3 }
func storeGlobal_3(m *Module, v float64){ m.g3 = v }
func loadGlobal_4(m *Module) int32 { return m.g4 }
`
return driverModule{moduleSrc: moduleSrc, modExpr: "modPtr"}
}
// driverIndirect builds the Module for cg_indirect.wat. It needs
// - a `t0 []any` field for the funcref table,
// - a `g0 int32` field for the $sink global,
// - one callIndirect_typeN wrapper per wasm type used by an
// OpCallIndirect (5 types in this fixture),
// - loadGlobal_0 / storeGlobal_0 wrappers,
// - an init() that fills t0[0..6] with the asm-emitted function
// pointers in the order the elem segment specifies.
func driverIndirect() driverModule {
moduleSrc := `package main
type Module struct {
t0 []any
g0 int32 // $sink
}
var modPtr *Module
func init() {
modPtr = &Module{
t0: make([]any, 12),
g0: 0,
}
// elem (i32.const 0) $negate $dbl $addp $subp $mul64 $const7 $store
// — funcIdx 0..6 in this fixture (no imports).
modPtr.t0[0] = Fn0
modPtr.t0[1] = Fn1
modPtr.t0[2] = Fn2
modPtr.t0[3] = Fn3
modPtr.t0[4] = Fn4
modPtr.t0[5] = Fn5
modPtr.t0[6] = Fn6
}
func loadGlobal_0(m *Module) int32 { return m.g0 }
func storeGlobal_0(m *Module, v int32) { m.g0 = v }
// callIndirect wrappers — one per wasm type used by an
// OpCallIndirect. Each does the type assertion that the asm
// can't easily do by itself, then calls the resulting func ptr.
// type 0 ($unary): (param i32) -> i32
// type 1 ($binary): (param i32 i32) -> i32
// type 2 ($i64bin): (param i64 i64) -> i64
// type 3 ($noargs): () -> i32
// type 4 ($voidproc): (param i32) -> ()
func callIndirect_type0(m *Module, idx int32, p0 int32) int32 {
return m.t0[idx].(func(*Module, int32) int32)(m, p0)
}
func callIndirect_type1(m *Module, idx int32, p0, p1 int32) int32 {
return m.t0[idx].(func(*Module, int32, int32) int32)(m, p0, p1)
}
func callIndirect_type2(m *Module, idx int32, p0, p1 int64) int64 {
return m.t0[idx].(func(*Module, int64, int64) int64)(m, p0, p1)
}
func callIndirect_type3(m *Module, idx int32) int32 {
return m.t0[idx].(func(*Module) int32)(m)
}
func callIndirect_type4(m *Module, idx int32, p0 int32) {
m.t0[idx].(func(*Module, int32))(m, p0)
}
`
return driverModule{moduleSrc: moduleSrc, modExpr: "modPtr"}
}
// TestEmitCallIndirectAMD64 covers OpCallIndirect against
// cg_indirect.wat. Each `call_indirect (type $T)` lowers to a CALL
// of a per-type Go wrapper that performs the runtime
// type-assertion the asm can't easily express by itself. Globals
// (via $sink for the void-returning indirect path) and direct
// calls (direct_chain) flow through the same emitter pipeline.
func TestEmitCallIndirectAMD64(t *testing.T) {
if runtime.GOARCH != "amd64" {
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
}
exports := []string{
"ind_unary", "ind_binary", "ind_i64", "ind_noargs", "ind_void",
"direct_chain",
}
cases := []driverCase{
// ind_unary: table[which](x)
{"ind_unary", []string{"0", "5"}, "-5"}, // negate(5)
{"ind_unary", []string{"1", "5"}, "10"}, // dbl(5)
{"ind_unary", []string{"0", "-7"}, "7"}, // negate(-7)
// ind_binary: table[which](a, b)
{"ind_binary", []string{"2", "3", "4"}, "7"}, // addp
{"ind_binary", []string{"3", "10", "3"}, "7"}, // subp
// ind_i64
{"ind_i64", []string{"4", "6", "7"}, "42"}, // mul64
// ind_noargs
{"ind_noargs", []string{"5"}, "7"}, // const7
// ind_void: stores its arg to $sink, returns $sink
{"ind_void", []string{"6", "42"}, "42"},
// direct_chain(x) = dbl(addp(negate(x), 100)) = 2*(-x+100)
{"direct_chain", []string{"5"}, "190"},
{"direct_chain", []string{"-50"}, "300"},
}
buildAndRunDriver(t, "cg_indirect", exports, driverIndirect(), cases)
}
// TestEmitGlobalsAMD64 covers OpGlobalGet / OpGlobalSet against
// cg_globals.wat. Each wasm global lives as a field on the host
// Module struct, and the asm reads/writes via loadGlobal_<idx> /
// storeGlobal_<idx> wrappers the driver emits. Float ops that route
// through helper calls (bump_f64) are tested only when those
// helpers are wired up; bump_f64 is left to a future step.
func TestEmitGlobalsAMD64(t *testing.T) {
if runtime.GOARCH != "amd64" {
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
}
exports := []string{
"get_i32", "set_i32",
"get_i64", "set_i64",
"get_f32", "set_f32",
"get_f64", "set_f64",
"get_const",
"bump_i32",
}
cases := []driverCase{
{"get_i32", nil, "11"},
{"get_i64", nil, "22"},
{"get_f32", nil, "1.5"},
{"get_f64", nil, "2.5"},
{"get_const", nil, "99"},
// set_X are void — the dispatcher prints "ok" on success.
{"set_i32", []string{"77"}, "ok"},
{"set_i64", []string{"77"}, "ok"},
{"set_f32", []string{"7.25"}, "ok"},
{"set_f64", []string{"7.25"}, "ok"},
// bump_i32: round-trip through the global within one call.
{"bump_i32", []string{"0"}, "11"},
{"bump_i32", []string{"100"}, "111"},
{"bump_i32", []string{"-5"}, "6"},
}
buildAndRunDriver(t, "cg_globals", exports, driverWithGlobals(), cases)
}
// TestEmitCallImportAMD64 covers OpCallImport — wasm `call` of an
// imported function — by routing each call through a generated Go
// wrapper that dispatches via the Module's hostImports interface.
// The fixture exposes three import shapes: i32 args + i32 result,
// i64 args + i64 result, and a void-returning call. The asm CALLs a
// uniformly-named `callImport_<funcIdx>` symbol; the test driver
// emits one wrapper per import, generated from `mod.Imports`.
func TestEmitCallImportAMD64(t *testing.T) {
if runtime.GOARCH != "amd64" {
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
}
exports := []string{"call_hostadd", "call_hostadd64", "call_hostnoop"}
cases := []driverCase{
{"call_hostadd", []string{"3", "4"}, "7"},
{"call_hostadd", []string{"-100", "100"}, "0"},
{"call_hostadd", []string{"2147483647", "1"}, "-2147483648"}, // i32 wrap
{"call_hostadd64", []string{"1000000000000", "1"}, "1000000000001"},
{"call_hostadd64", []string{"-1", "1"}, "0"},
// call_hostnoop returns x+1 after a no-op host call.
{"call_hostnoop", []string{"0"}, "1"},
{"call_hostnoop", []string{"41"}, "42"},
}
buildAndRunDriver(t, "asmtest_callimport", exports, driverHostAdd(), cases)
}
// driverBulkMem builds a Module with linear memory plus
// memoryCopy / memoryFill wrappers for the bulk-memory ops. The
// initial memory contents come from the cg_bulkmem fixture's data
// segment ("abcdefghij" at offset 0).
func driverBulkMem() driverModule {
return driverModule{
moduleSrc: `package main
import "unsafe"
type Module struct {
Memory []byte
MaxMem uint64
M unsafe.Pointer
}
var modPtr *Module
func init() {
mem := make([]byte, 65536)
copy(mem, []byte("abcdefghij"))
modPtr = &Module{
Memory: mem,
MaxMem: 65536,
M: unsafe.Pointer(&mem[0]),
}
}
// memoryCopy implements wasm memory.copy: copy n bytes from src to
// dst within the linear memory, with the overlap-correct semantics
// the wasm spec requires (Go's built-in copy handles this).
func memoryCopy(m *Module, dst, src, n int32) {
if n <= 0 {
return
}
copy(m.Memory[dst:dst+n], m.Memory[src:src+n])
}
// memoryFill writes the low byte of val to n bytes starting at addr.
func memoryFill(m *Module, addr, val, n int32) {
if n <= 0 {
return
}
b := byte(val)
for i := int32(0); i < n; i++ {
m.Memory[addr+i] = b
}
}
`,
modExpr: "modPtr",
}
}
// TestEmitBulkMemAMD64 covers OpMemoryCopy and OpMemoryFill against
// cg_bulkmem.wat. The fixture's saturating-trunc exports use
// f-conversion helpers the prototype doesn't wire up yet; they're
// silently stubbed and not asserted here.
func TestEmitBulkMemAMD64(t *testing.T) {
if runtime.GOARCH != "amd64" {
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
}
exports := []string{"mem_copy", "mem_fill", "load16_s", "store16", "load64"}
cases := []driverCase{
// mem_copy(dst=20, src=0, n=4) copies "abcd" into mem[20..],
// then returns mem[20] as a u8 → 'a' = 97.
{"mem_copy", []string{"20", "0", "4"}, "97"},
// mem_fill(addr=100, val=0xAA, n=8) writes 8 copies of 0xAA at
// mem[100..108]; returning mem[100] u8 → 0xAA = 170.
{"mem_fill", []string{"100", "170", "8"}, "170"},
// Sanity-check the surrounding load/store ops are still happy
// in the same driver.
{"load16_s", []string{"0"}, "25185"}, // 'a' (0x61) + 'b'<<8 (0x6200) = 0x6261
{"load64", []string{"0"}, "7523094288207667809"}, // "abcdefgh" little-endian
}
buildAndRunDriver(t, "cg_bulkmem", exports, driverBulkMem(), cases)
}
// TestEmitCallDirectAMD64 covers OpCallDirect with cg_manyfuncs.wat:
// `sum_all(x)` invokes thirty `leaf<i>` functions in sequence and
// adds the results. Each leaf computes `x * (i+1) + 3*i`, so
// `sum_all(x) = sum_{i=0..29}[x*(i+1) + 3*i] = 465*x + 1305`. The
// fixture also exposes ops the prototype emitter doesn't support yet
// (call_indirect, global.get / global.set, memory.size); those are
// silently stubbed by the driver and never invoked from the cases
// below.
func TestEmitCallDirectAMD64(t *testing.T) {
if runtime.GOARCH != "amd64" {
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
}
exports := []string{"sum_all"}
expected := func(x int32) string {
return fmt.Sprintf("%d", 465*x+1305)
}
cases := []driverCase{
{"sum_all", []string{"0"}, expected(0)},
{"sum_all", []string{"1"}, expected(1)},
{"sum_all", []string{"-1"}, expected(-1)},
{"sum_all", []string{"7"}, expected(7)},
{"sum_all", []string{"100"}, expected(100)},
{"sum_all", []string{"-100"}, expected(-100)},
}
buildAndRunDriver(t, "cg_manyfuncs", exports, driverPlaceholder, cases)
}
// dispatchCase emits the `case "<name>": ... <call> ...` arm for one
// export. Inputs are parsed from os.Args, the function is called
// with modExpr as the first (m *Module) argument, and the result is
// printed. modExpr is "nil" for tests whose asm never dereferences
// m (the arith / control fixtures) and "modPtr" for tests that need
// a real backing memory (the memops fixture); the driver template
// declares modPtr accordingly.
func dispatchCase(name string, sig wasm.FuncType, modExpr string) string {
var b strings.Builder
fmt.Fprintf(&b, "case %q:\n", name)
args := []string{modExpr}
for i, p := range sig.Params {
switch p {
case wasm.ValI32:
fmt.Fprintf(&b, "\tv%d, err := strconv.ParseInt(os.Args[%d], 10, 32)\n", i, i+2)
fmt.Fprintf(&b, "\tif err != nil { fmt.Fprintln(os.Stderr, err); os.Exit(2) }\n")
args = append(args, fmt.Sprintf("int32(v%d)", i))
case wasm.ValI64:
fmt.Fprintf(&b, "\tv%d, err := strconv.ParseInt(os.Args[%d], 10, 64)\n", i, i+2)
fmt.Fprintf(&b, "\tif err != nil { fmt.Fprintln(os.Stderr, err); os.Exit(2) }\n")
args = append(args, fmt.Sprintf("v%d", i))
case wasm.ValF32:
fmt.Fprintf(&b, "\tv%d, err := strconv.ParseFloat(os.Args[%d], 32)\n", i, i+2)
fmt.Fprintf(&b, "\tif err != nil { fmt.Fprintln(os.Stderr, err); os.Exit(2) }\n")
args = append(args, fmt.Sprintf("float32(v%d)", i))
case wasm.ValF64:
fmt.Fprintf(&b, "\tv%d, err := strconv.ParseFloat(os.Args[%d], 64)\n", i, i+2)
fmt.Fprintf(&b, "\tif err != nil { fmt.Fprintln(os.Stderr, err); os.Exit(2) }\n")
args = append(args, fmt.Sprintf("v%d", i))
}
}
if len(sig.Results) == 0 {
// void-returning export: print a stable token so the
// dispatcher's exit status alone signals success.
fmt.Fprintf(&b, "\t%s(%s)\n\tfmt.Println(\"ok\")\n", name, strings.Join(args, ", "))
} else {
fmt.Fprintf(&b, "\tfmt.Println(%s(%s))\n", name, strings.Join(args, ", "))
}
return b.String()
}
func writeDriverModuleArch(t *testing.T, dir string, driver driverModule, asm, decls, stubs, wrappers, dispatch, goarch string) {
t.Helper()
tag := "//go:build " + goarch + "\n\npackage main\n\n"
mustWrite(t, dir, "go.mod", "module asmgentest\n\ngo 1.25\n")
mustWrite(t, dir, "module.go", driver.moduleSrc)
mustWrite(t, dir, "decls_"+goarch+".go", tag+decls)
mustWrite(t, dir, "asm_"+goarch+".s", "//go:build "+goarch+"\n\n#include \"textflag.h\"\n#include \"funcdata.h\"\n\n"+asm)
mustWrite(t, dir, "helpers_"+goarch+".go", helpersGoSourceArch(goarch))
if stubs != "" {
mustWrite(t, dir, "stubs_"+goarch+".go", tag+stubs)
}
if wrappers != "" {
mustWrite(t, dir, "wrappers_"+goarch+".go", tag+wrappers)
}
mustWrite(t, dir, "main.go", `package main
import (
"fmt"
"os"
"strconv"
)
func main() {
if len(os.Args) < 2 {
fmt.Fprintln(os.Stderr, "usage: driver <export> args...")
os.Exit(2)
}
`+dispatch+`
}
`)
}
func mustWrite(t *testing.T, dir, name, content string) {
t.Helper()
if err := os.WriteFile(filepath.Join(dir, name), []byte(content), 0o644); err != nil {
t.Fatalf("write %s: %v", name, err)
}
}
// helpersGoSource returns the test-local Go source containing every
// helper the asm bodies may CALL. The functions mirror the bodies in
// internal/codegen/helpers/helpers.go; copying them here keeps the
// integration test free of an internal-package import. The build
// constraint matches the asm so this file compiles only on the arch
// the driver was emitted for.
func helpersGoSource() string {
return helpersGoSourceArch("amd64")
}
func helpersGoSourceArch(goarch string) string {
return `//go:build ` + goarch + `
package main
import (
"math"
"math/bits"
)
func i32_eqz(x int32) int32 { if x == 0 { return 1 }; return 0 }
func i64_eqz(x int64) int32 { if x == 0 { return 1 }; return 0 }
func i32_clz(x int32) int32 { return int32(bits.LeadingZeros32(uint32(x))) }
func i32_ctz(x int32) int32 { return int32(bits.TrailingZeros32(uint32(x))) }
func i32_popcnt(x int32) int32 { return int32(bits.OnesCount32(uint32(x))) }
func i64_clz(x int64) int64 { return int64(bits.LeadingZeros64(uint64(x))) }
func i64_ctz(x int64) int64 { return int64(bits.TrailingZeros64(uint64(x))) }
func i64_popcnt(x int64) int64 { return int64(bits.OnesCount64(uint64(x))) }
func i32_rotl(x, y int32) int32 { return int32(bits.RotateLeft32(uint32(x), int(y&31))) }
func i32_rotr(x, y int32) int32 { return int32(bits.RotateLeft32(uint32(x), -int(y&31))) }
func i64_rotl(x, y int64) int64 { return int64(bits.RotateLeft64(uint64(x), int(y&63))) }
func i64_rotr(x, y int64) int64 { return int64(bits.RotateLeft64(uint64(x), -int(y&63))) }
//go:noinline
func wasm_trap_div_zero() { panic("wasm: integer divide by zero") }
//go:noinline
func wasm_trap_int_overflow() { panic("wasm: integer overflow") }
//go:noinline
func wasm_trap_invalid_conv() { panic("wasm: invalid conversion to integer") }
func i32_div_s(x, y int32) int32 {
if y == -1 && x == math.MinInt32 { panic("wasm: integer overflow") }
if y == 0 { panic("wasm: integer divide by zero") }
return x / y
}
func i32_div_u(x, y uint32) uint32 {
if y == 0 { panic("wasm: integer divide by zero") }
return x / y
}
func i32_div_u_s(x, y int32) int32 { return int32(i32_div_u(uint32(x), uint32(y))) }
func i32_rem_s(x, y int32) int32 {
if y == 0 { panic("wasm: integer divide by zero") }
if y == -1 { return 0 }
return x % y
}
func i32_rem_u(x, y uint32) uint32 {
if y == 0 { panic("wasm: integer divide by zero") }
return x % y
}
func i32_rem_u_s(x, y int32) int32 { return int32(i32_rem_u(uint32(x), uint32(y))) }
func i64_div_s(x, y int64) int64 {
if y == -1 && x == math.MinInt64 { panic("wasm: integer overflow") }
if y == 0 { panic("wasm: integer divide by zero") }
return x / y
}
func i64_div_u(x, y uint64) uint64 {
if y == 0 { panic("wasm: integer divide by zero") }
return x / y
}
func i64_div_u_s(x, y int64) int64 { return int64(i64_div_u(uint64(x), uint64(y))) }
func i64_rem_s(x, y int64) int64 {
if y == 0 { panic("wasm: integer divide by zero") }
if y == -1 { return 0 }
return x % y
}
func i64_rem_u(x, y uint64) uint64 {
if y == 0 { panic("wasm: integer divide by zero") }
return x % y
}
func i64_rem_u_s(x, y int64) int64 { return int64(i64_rem_u(uint64(x), uint64(y))) }
func i32_wrap_i64(x int64) int32 { return int32(x) }
func i64_extend_i32_s(x int32) int64 { return int64(x) }
func i64_extend_i32_u(x int32) int64 { return int64(uint32(x)) }
func i32_extend8_s(x int32) int32 { return int32(int8(x)) }
func i32_extend16_s(x int32) int32 { return int32(int16(x)) }
func i64_extend8_s(x int64) int64 { return int64(int8(x)) }
func i64_extend16_s(x int64) int64 { return int64(int16(x)) }
func i64_extend32_s(x int64) int64 { return int64(int32(x)) }
`
}
// TestEmitAddAMD64 keeps the original one-function add fixture as a
// fast smoke test. It compiles in ~1s and runs in ~10ms, making it
// useful as the first thing to fail when emitter scaffolding breaks.
func TestEmitAddAMD64(t *testing.T) {
if runtime.GOARCH != "amd64" {
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
}
bin := testfixture.Wasm(t, "arith")
mod, err := wasm.Parse(bytes.NewReader(bin))
if err != nil {
t.Fatalf("parse: %v", err)
}
addIdx := ^uint32(0)
for _, e := range mod.Exports {
if e.Kind == wasm.ExportFunc && e.Name == "add" {
addIdx = e.Index
}
}
if addIdx == ^uint32(0) {
t.Fatal("missing add export")
}
fn, err := lower.LowerFunction(mod, addIdx, "add", nil)
if err != nil {
t.Fatalf("lower: %v", err)
}
sig := mod.FuncTypeOf(addIdx)
asm, decl, err := EmitFuncAMD64("add", sig, fn, FuncOptions{ModulePkgRef: "*Module"})
if err != nil {
t.Fatalf("emit: %v", err)
}
dir := t.TempDir()
mustWrite(t, dir, "go.mod", "module asmgentest\n\ngo 1.25\n")
mustWrite(t, dir, "module.go", driverPlaceholder.moduleSrc)
mustWrite(t, dir, "add.go", "//go:build amd64\n\npackage main\n\n"+decl)
mustWrite(t, dir, "add_amd64.s", "//go:build amd64\n\n#include \"textflag.h\"\n#include \"funcdata.h\"\n\n"+asm)
mustWrite(t, dir, "main.go", `package main
import (
"fmt"
"os"
"strconv"
)
func main() {
a, _ := strconv.ParseInt(os.Args[1], 10, 32)
b, _ := strconv.ParseInt(os.Args[2], 10, 32)
fmt.Println(add(nil, int32(a), int32(b)))
}
`)
cmd := exec.Command("go", "run", ".", "2", "3")
cmd.Dir = dir
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("go run: %v\n%s", err, out)
}
if strings.TrimSpace(string(out)) != "5" {
t.Errorf("got %q want 5", out)
}
}
// TestEmitEmptyElseFallthroughAMD64 — covers the
// "empty else block elision" optimisation surfaced by the Pure-Go
// instruction-count comparison on Fn31214. The wasm-to-SSA lowering
// of `if x { call }` (no else clause) produces a 4-block diamond:
//
// b_test (BlockIf, control=x) -> [b_then, b_else]
// b_then (BlockPlain) -> [b_merge]
// b_else (BlockPlain, empty) -> [b_merge]
// b_merge ... continuation
//
// Before the empty-else elision the asm shape was
//
// TESTL AX, AX
// JNE L_then
// JMP L_else <-- routes through the empty block
// L_then: ... ; JMP L_merge
// L_else: <-- empty pass-through, dead label
// L_merge: ...
//
// After the empty-else elision:
//
// - passthroughTarget collapses L_else → L_merge, so the
// BlockIf names L_merge directly as the else target.
// - emitBlock supplies the next block's label as a
// fall-through hint.
// - When the fall-through equals the then label, the arch
// emitter INVERTS the JCC and drops the JMP.
//
// Resulting amd64 shape (the load-bearing change):
//
// TESTL AX, AX
// JE L_merge <-- single conditional, inverted
// L_then: ... ; JMP L_merge
// L_else: <-- still emitted as a dead label
// L_merge: ...
//
// That collapses three terminator instructions (JNE/JMP/JMP) into
// one (JE), the saving the comparison row showed.
//
// We assert the load-bearing properties directly on the emitted asm
// string — no full assemble/link/run round-trip is needed, the
// driver tests already cover correctness of generic BlockIf.
func TestEmitEmptyElseFallthroughAMD64(t *testing.T) {
if runtime.GOARCH != "amd64" {
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
}
// Build SSA for: func(x i32) i32 { if x != 0 { /* no body, just
// pretend something happened by adding x+x — kept off the merge
// path */ }; return x }
//
// We don't need an actual call to exercise the optimisation; the
// load-bearing part is the 4-block diamond shape with an empty
// else. A no-op then is enough to make the pass-through pattern
// observable. We make the then block non-empty (one Add) so it
// is NOT itself passthrough-eligible, and leave the else block
// genuinely empty.
sig := wasm.FuncType{Params: []wasm.ValType{wasm.ValI32}, Results: []wasm.ValType{wasm.ValI32}}
fsig := ssa.FuncSig{Params: []ssa.Type{ssa.TypeI32}, Results: []ssa.Type{ssa.TypeI32}}
bb := ssa.NewFuncBuilder("emptyelse", fsig)
bTest := bb.NewBlock(ssa.BlockIf)
bThen := bb.NewBlock(ssa.BlockPlain)
bElse := bb.NewBlock(ssa.BlockPlain)
bMerge := bb.NewBlock(ssa.BlockRet)
bb.SetEntry(bTest)
bb.SetCurrent(bTest)
x := bb.Param(0, ssa.TypeI32)
bb.LinkIf(x, bThen, bElse)
bb.SetCurrent(bThen)
// One real value so the then block is NOT passthrough-eligible.
_ = bb.NewValue(ssa.OpAdd32, ssa.TypeI32, x, x)
bb.LinkPlain(bMerge)
bb.SetCurrent(bElse)
// Empty — just the BlockPlain jump (added automatically).
bb.LinkPlain(bMerge)
bb.SetCurrent(bMerge)
bb.FinishRet(x)
if err := ssa.Verify(bb.Func()); err != nil {
t.Fatalf("SSA verify: %v", err)
}
asm, _, err := EmitFuncAMD64("emptyelse", sig, bb.Func(), FuncOptions{ModulePkgRef: "*Module"})
if err != nil {
t.Fatalf("EmitFuncAMD64: %v", err)
}
// Property 1 — the empty else's label is no longer referenced by
// any JMP/Jcc instruction. After passthroughTarget redirection the
// only references should be its own definition line (`L<n>:`) and
// nothing else. We approximate this with: the block emitted
// between then and merge MUST have label `L<bElse>:`, and there
// MUST NOT be any `JMP L<bElse>` or `JCC L<bElse>` line.
elseLabel := fmt.Sprintf("L%d", bElse.ID)
for _, line := range strings.Split(asm, "\n") {
trim := strings.TrimSpace(line)
if !strings.HasPrefix(trim, "J") {
continue
}
// Match "<MNEMONIC> L<n>" — split on whitespace.
parts := strings.Fields(trim)
if len(parts) < 2 {
continue
}
if parts[len(parts)-1] == elseLabel {
t.Errorf("expected the empty-else label %q to have NO inbound branches "+
"after passthrough redirection, but found:\n %s\nfull asm:\n%s",
elseLabel, line, asm)
}
}
// Property 2 — the BlockIf terminator emits exactly ONE
// conditional jump (the inverted form, JE) and NO trailing JMP.
// The inverted condition routes the "x == 0" case to the merge
// label directly; the "x != 0" case falls through to bThen.
// We assert this by counting the J* instructions in the prologue
// segment that PRECEDES the first body label.
mergeLabel := fmt.Sprintf("L%d", bMerge.ID)
// Find the position of bThen's label — everything before is the
// BlockIf prologue (label + TESTL + branch(es)).
thenLabel := fmt.Sprintf("L%d:", bThen.ID)
thenIdx := strings.Index(asm, "\n"+thenLabel)
if thenIdx < 0 {
t.Fatalf("then-block label %q not found in asm:\n%s", thenLabel, asm)
}
prologue := asm[:thenIdx]
branchLines := 0
hasInvertedJE := false
for _, line := range strings.Split(prologue, "\n") {
trim := strings.TrimSpace(line)
switch {
case strings.HasPrefix(trim, "JMP "):
branchLines++
case strings.HasPrefix(trim, "J") && len(trim) >= 2 && trim[1] >= 'A' && trim[1] <= 'Z':
// Any conditional jump (JE, JNE, JLT, JGE, ...).
branchLines++
if strings.HasPrefix(trim, "JE ") && strings.HasSuffix(trim, " "+mergeLabel) {
hasInvertedJE = true
}
}
}
if branchLines != 1 {
t.Errorf("expected exactly 1 branch in the BlockIf prologue "+
"(the inverted JE to merge — JMP dropped via fall-through), "+
"got %d in:\n%s", branchLines, prologue)
}
if !hasInvertedJE {
t.Errorf("expected an inverted `JE %s` in the BlockIf prologue "+
"(the original cond is `x != 0`, JE routes the `x == 0` case to the merge "+
"directly while `x != 0` falls through to the then block), got:\n%s",
mergeLabel, prologue)
}
}
// TestBitTestFusionAMD64 — locks in the
// "(x & 1<<k) <eq/ne> 0 → BTL + JCC" fusion. Pre-fusion the lowering
// emitted six instructions for `x & 1 != 0` (load x, AND $1, store
// AND result, reload, TEST, JNE/JMP); Pure-Go gets the same effect
// in two (`BTL $0, x ; JCS label`). The fusion detects the chain
// in Pass 4's branch-fusion analysis, skips emission for the OpAnd
// (it lives only as a flag producer), and lowers the BlockIf to
// the BTL + JCC pair.
//
// We assert three properties:
// - the emitted asm CONTAINS `BTL $0, AX` (the single-bit test),
// - it does NOT contain `ANDL $1` (the OpAnd's separate emit),
// - it CONTAINS exactly one conditional branch in the BlockIf
// prologue (no SETcc + MOVBLZX + TEST chain).
func TestBitTestFusionAMD64(t *testing.T) {
if runtime.GOARCH != "amd64" {
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
}
sig := wasm.FuncType{Params: []wasm.ValType{wasm.ValI32}, Results: []wasm.ValType{wasm.ValI32}}
fsig := ssa.FuncSig{Params: []ssa.Type{ssa.TypeI32}, Results: []ssa.Type{ssa.TypeI32}}
bb := ssa.NewFuncBuilder("bittest", fsig)
bIf := bb.NewBlock(ssa.BlockIf)
bThen := bb.NewBlock(ssa.BlockRet)
bElse := bb.NewBlock(ssa.BlockRet)
bb.SetEntry(bIf)
bb.SetCurrent(bIf)
x := bb.Param(0, ssa.TypeI32)
one := bb.Const32(1)
andRes := bb.NewValue(ssa.OpAnd32, ssa.TypeI32, x, one)
zero := bb.Const32(0)
cond := bb.NewValue(ssa.OpNe32, ssa.TypeBool, andRes, zero)
bb.LinkIf(cond, bThen, bElse)
bb.SetCurrent(bThen)
bb.FinishRet(x)
bb.SetCurrent(bElse)
bb.FinishRet(x)
if err := ssa.Verify(bb.Func()); err != nil {
t.Fatalf("SSA verify: %v", err)
}
asm, _, err := EmitFuncAMD64("bittest", sig, bb.Func(), FuncOptions{ModulePkgRef: "*Module"})
if err != nil {
t.Fatalf("EmitFuncAMD64: %v", err)
}
if !strings.Contains(asm, "BTL $0, AX") {
t.Errorf("expected `BTL $0, AX` in emitted asm — the bit-test "+
"fusion did not fire. asm:\n%s", asm)
}
if strings.Contains(asm, "ANDL $1") {
t.Errorf("expected the OpAnd32 to be elided by branchFusedSkip, "+
"but found `ANDL $1` in the asm:\n%s", asm)
}
// The bit-test path emits exactly one Jcc (JCS or JCC, possibly
// with an `JMP <else>` if no fall-through applies). We don't
// assert the precise count here — the fall-through
// optimisation may drop one of them.
}
// TestLoadConstDispFoldAMD64 — locks in the
// "non-negative constant offset folds into addressing mode" fix
// for OpLoad's dynamic-base path. Pre-15d the lowering emitted
//
// MOVL <base>, SI
// ADDL $<off>, SI <-- redundant
// MOVL (BX)(SI*1), AX
//
// for `i32.load offset=<off>`. Pure-Go's equivalent collapses to
// a single `MOVL <off>(BX)(SI*1), AX` because amd64's addressing
// mode happily takes a signed displacement. The disp-fold pass
// does the same when off > 0 (sign-extension matches
// zero-extension for non-negative values, preserving wasm's
// `uint32(base+off)` effective-address semantics).
//
// We assert two properties on a tiny SSA function that performs
// `i32.load offset=24` on a dynamic base:
// - the asm MUST NOT contain `ADDL $24, SI` (the fold
// eliminated it);
// - the asm MUST contain `MOVL 24(BX)(SI*1)` (the load picks
// up the displacement from the addressing mode).
//
// The off=0 path is exercised by other load tests; the off<0
// (wrap-required) path is exercised by the existing memop tests.
func TestLoadConstDispFoldAMD64(t *testing.T) {
if runtime.GOARCH != "amd64" {
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
}
sig := wasm.FuncType{Params: []wasm.ValType{wasm.ValI32}, Results: []wasm.ValType{wasm.ValI32}}
fsig := ssa.FuncSig{Params: []ssa.Type{ssa.TypeI32}, Results: []ssa.Type{ssa.TypeI32}}
bb := ssa.NewFuncBuilder("loadfold", fsig)
b0 := bb.NewBlock(ssa.BlockRet)
bb.SetEntry(b0)
bb.SetCurrent(b0)
base := bb.Param(0, ssa.TypeI32)
loaded := bb.NewValueAuxInt(ssa.OpLoad32, ssa.TypeI32, 24, base)
bb.FinishRet(loaded)
if err := ssa.Verify(bb.Func()); err != nil {
t.Fatalf("SSA verify: %v", err)
}
asm, _, err := EmitFuncAMD64("loadfold", sig, bb.Func(), FuncOptions{ModulePkgRef: "*Module"})
if err != nil {
t.Fatalf("EmitFuncAMD64: %v", err)
}
if strings.Contains(asm, "ADDL $24, SI") {
t.Errorf("expected the constant offset 24 to be folded into the "+
"addressing-mode displacement, not emitted as `ADDL $24, SI`. asm:\n%s", asm)
}
if !strings.Contains(asm, "24(BX)(SI*1)") {
t.Errorf("expected the load to use `24(BX)(SI*1)` addressing, "+
"but the displacement was not folded. asm:\n%s", asm)
}
}
// TestOpParamNoSlotStoreAMD64 — locks in the
// "OpParam producer-side store-to-slot is dead" property.
//
// operandSrc{32,64,Float} on amd64 resolves OpParam to `lN+off(FP)`
// directly, bypassing the slot. The matching producer-side emit
// (emitParam) used to write the param value to its slot as a
// no-op pair `MOVL lN+off(FP), AX; MOVL AX, dst(SP)`. Pure-Go drops
// the equivalent; marking OpParam as SkipValue brings us into
// line by excluding it from the emit loop entirely.
//
// Risk: slot reuse might allocate OpParam's slot to a later
// value. If the OpParam store survived, the later value's store
// would overwrite it before any read (the slot has no read site),
// so removing the OpParam store is safe.
//
// We assert two properties on a tiny SSA function: f(x int32) int32
// { return x + x }. The function has a single OpParam(x) with two
// users (the OpAdd32). The emitted asm MUST contain
// - exactly zero `MOVL l0+8(FP), <reg>; MOVL <reg>, <slot>(SP)`
// producer pairs sourced from `l0+8(FP)` and destined for a slot
// (other than the OpAdd32's own result store);
// - at least one `MOVL l0+8(FP), <reg>` read (the OpAdd32 emit
// loading x from FP).
//
// The first property is the load-bearing one. We approximate it by
// counting `MOVL l0+8(FP), AX` / `MOVL AX, NN(SP)` adjacent pairs:
// the OpAdd32 emit produces a single load-from-FP-into-AX followed
// by an ADDL-with-the-other-arg, not by a MOVL-to-slot. Any MOVL-
// to-slot directly after the load IS the (removed) OpParam spill.
func TestOpParamNoSlotStoreAMD64(t *testing.T) {
if runtime.GOARCH != "amd64" {
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
}
sig := wasm.FuncType{Params: []wasm.ValType{wasm.ValI32}, Results: []wasm.ValType{wasm.ValI32}}
fsig := ssa.FuncSig{Params: []ssa.Type{ssa.TypeI32}, Results: []ssa.Type{ssa.TypeI32}}
bb := ssa.NewFuncBuilder("paramskip", fsig)
b0 := bb.NewBlock(ssa.BlockRet)
bb.SetEntry(b0)
bb.SetCurrent(b0)
x := bb.Param(0, ssa.TypeI32)
sum := bb.NewValue(ssa.OpAdd32, ssa.TypeI32, x, x)
bb.FinishRet(sum)
if err := ssa.Verify(bb.Func()); err != nil {
t.Fatalf("SSA verify: %v", err)
}
asm, _, err := EmitFuncAMD64("paramskip", sig, bb.Func(), FuncOptions{ModulePkgRef: "*Module"})
if err != nil {
t.Fatalf("EmitFuncAMD64: %v", err)
}
lines := strings.Split(asm, "\n")
for i, line := range lines[:len(lines)-1] {
trim := strings.TrimSpace(line)
// Match the load half of an OpParam dead spill:
// `MOVL l0+8(FP), <reg>`.
if !strings.HasPrefix(trim, "MOVL l") || !strings.Contains(trim, "(FP), ") {
continue
}
// Extract the register name: ", <REG>" suffix.
parts := strings.Split(trim, ", ")
if len(parts) != 2 {
continue
}
reg := parts[1]
// Look at the next line. The OpParam dead spill is
// `MOVL <reg>, <off>(SP)` — a store to an SP-relative slot
// using exactly the register we just loaded into.
next := strings.TrimSpace(lines[i+1])
want := "MOVL " + reg + ", "
if strings.HasPrefix(next, want) && strings.Contains(next, "(SP)") {
t.Errorf("expected OpParam to emit no producer-side spill, "+
"but found a dead `load from FP; store to SP` pair at lines %d-%d:\n %s\n %s\nfull asm:\n%s",
i, i+1, trim, next, asm)
break
}
}
}
// TestComputeUnusedResults — unit-tests the
// liveness analysis that decides which CALL sites can drop their
// post-CALL result load + slot store. Pure-Go's frontend would do
// this via classic dead-code elimination on the SSA; we get the
// same effect by walking liveness backward from the sinks
// (side-effecting ops, BlockRet outputs, block.Control values).
//
// The analysis is "transitively live": a CALL value is live as a
// SINK (its side effects matter), but its RESULT counts as used
// only if a LIVE consumer chains back to it. A consumer that is
// itself dead (its result reaches no sink) doesn't count — that's
// the case for "v_call → OpCopy → OpAdd32 (result unused)", which
// IS elidable even though the OpCopy/OpAdd nominally reference
// v_call.
//
// Cases:
// - OpCallDirect whose result is never referenced by any other
// value's Args, no block.Control, and not a BlockRet output →
// unusedResult MUST be set.
// - OpCallDirect whose result IS the function's return value
// (referenced by BlockRet) → unusedResult MUST NOT be set.
// - OpHelperCall consumed via OpCopy → OpAdd32 with NO live
// downstream consumer → unusedResult MUST be set (the whole
// chain is dead).
// - OpHelperCall used as block.Control → unusedResult MUST NOT
// fire (control reads it as a sink).
// - OpHelperCall whose chain DOES reach a sink (helper → add →
// BlockRet) → unusedResult MUST NOT fire.
//
// We invoke computeUnusedResults directly with a fresh plan so the
// test stays focused on the analysis. planFunc's heavier setup
// (which resolves call symbols, computes the callee frame, etc.)
// would obscure the property we want to lock in.
func TestComputeUnusedResults(t *testing.T) {
sig := wasm.FuncType{Params: []wasm.ValType{wasm.ValI32}, Results: []wasm.ValType{wasm.ValI32}}
fsig := ssa.FuncSig{Params: []ssa.Type{ssa.TypeI32}, Results: []ssa.Type{ssa.TypeI32}}
bb := ssa.NewFuncBuilder("unusedret", fsig)
// b0: the work; b1 (BlockIf, control=helperCond): then→b2 else→b2;
// b2: BlockRet of liveCall's result.
b0 := bb.NewBlock(ssa.BlockPlain)
b1 := bb.NewBlock(ssa.BlockIf)
b2 := bb.NewBlock(ssa.BlockRet)
bb.SetEntry(b0)
bb.SetCurrent(b0)
l0 := bb.Param(0, ssa.TypeI32)
// (a) OpCallDirect with discarded result.
unusedCall := bb.NewValueAuxInt(ssa.OpCallDirect, ssa.TypeI32, 0, l0)
// (b) OpHelperCall consumed via OpCopy → OpAdd32 → unused. The
// whole chain is dead (sum is never referenced by any sink) so
// helperDeadChain's result is logically unused.
helperDeadChain := bb.NewValueAuxInt(ssa.OpHelperCall, ssa.TypeI32, 0, l0)
copyAlias := bb.NewValue(ssa.OpCopy, ssa.TypeI32, helperDeadChain)
deadSum := bb.NewValue(ssa.OpAdd32, ssa.TypeI32, copyAlias, l0)
_ = deadSum
// (c) OpHelperCall whose chain DOES reach the function output
// (helper → add → BlockRet). Result is genuinely used.
helperLiveChain := bb.NewValueAuxInt(ssa.OpHelperCall, ssa.TypeI32, 0, l0)
liveSum := bb.NewValue(ssa.OpAdd32, ssa.TypeI32, helperLiveChain, l0)
// (d) OpHelperCall used as the BlockIf control — the control
// edge is a sink, so it counts as a live consumer.
condHelper := bb.NewValueAuxInt(ssa.OpHelperCall, ssa.TypeBool, 0, l0)
bb.LinkPlain(b1)
bb.SetCurrent(b1)
bb.LinkIf(condHelper, b2, b2)
bb.SetCurrent(b2)
bb.FinishRet(liveSum)
if err := ssa.Verify(bb.Func()); err != nil {
t.Fatalf("SSA verify: %v", err)
}
plan := &funcPlan{unusedResult: map[ssa.ValueID]bool{}}
computeUnusedResults(bb.Func(), plan, sig)
cases := []struct {
v *ssa.Value
wantUnused bool
why string
}{
{unusedCall, true, "OpCallDirect with no downstream consumer should be elidable"},
{helperDeadChain, true,
"OpHelperCall whose consumer chain is dead (sum has no live consumer) should be elidable — " +
"Pure-Go would DCE the whole chain"},
{helperLiveChain, false,
"OpHelperCall whose chain reaches BlockRet via liveSum is genuinely used"},
{condHelper, false, "OpHelperCall used as block.Control — used"},
}
for _, c := range cases {
got := plan.unusedResult[c.v.ID]
if got != c.wantUnused {
t.Errorf("unusedResult[v%d (%s)] = %v, want %v: %s",
c.v.ID, c.v.Op, got, c.wantUnused, c.why)
}
}
}