mirror of
https://github.com/hanzoai/wasm2go.git
synced 2026-10-30 15:06:48 +00:00
* Support the memory64 proposal: 64-bit linear memory past 4 GiB A wasm32 module can never use more than 4 GiB no matter what the runtime allows — its pointers, size_t and allocator are compiled as 32-bit values. memory64 (wasm64) is the sound route to large memories, and this adds it end to end for the pure and gcasm outputs: - Parser: limits flags 0x04/0x05 decode the 64-bit index space into MemoryType.Is64 / Module.Memory64; unknown flag bits are rejected. Memarg offsets decode as u64 (wasm32 offsets are unaffected). - Lowering: addresses ride as i64 values; memory.size/grow type as i64; the SIMD memory family lowers to a simd_m64_* helper family with u64 addressing (never rewritten to the pair/splice forms, which assume 32-bit addresses). Atomics on a memory64 are rejected for now — no toolchain produces shared wasm64 today. - Emission: scalar loads/stores keep the zero-cost inline unsafe deref, with the effective address computed as u64 instead of the u32 wrap; large constant totals still route through the _consts table. memory.size/grow/fill/copy/init switch to 64-bit helper twins. The coalesced SIMD bounds check stays wasm32-only (its exactness argument is written against the wasm32 hard cap). - Runtime: memoryGrow64 grows past the wasm32 cap up to a 2^48 hard cap with the same geometric-capacity scheme; a memory64 with no declared maximum defaults to unlimited (maxMem 0) instead of the 4 GiB wasm32 default, and the external-image constructors accept images past 4 GiB. Effective-address checks in the 64-bit helpers guard u64 overflow explicitly — there is no benign wrap at 2^64. The gate proves the point end to end: a memory64 module grows to 4 GiB + 128 KiB and stores/loads at address 1<<32 through the full backend. wasm32 output is byte-identical to before across the fixture corpus, SIMD included. Native asm splicing of the m64 SIMD helpers, the wasm64 WASI ABI, and a wasm64 sysroot in the wasmify toolchain are follow-ups; the llama.cpp chain needs all three before go-llama can load >4 GiB models. Co-Authored-By: Claude Fable 5 <[email protected]> * Bind memory64 modules to a widened wasip1 ABI There is no standardized 64-bit WASI ABI; the wasm2go-ported wasi-libc declares every wasi_snapshot_preview1 import argument at pointer width (all-i64 under wasm64), and this is the runtime side of that contract: - A memory64 module's wasi imports bind to new *64 methods on WasiStubs (codegen appends the suffix for wasi_snapshot_preview1 imports when the module is a memory64); every wasm32 module keeps the standard 32-bit methods and its byte-identical output. - The *64 bindings parse the LP64 layouts — 16-byte iovecs, 64-bit size fields — through a full-range 64-bit memory window, covering the clock/fd/proc calls a C hello-world exercises; the rest of the surface follows as the wasm64 toolchain grows. - Immediate skipping in the call-graph prescan reads memarg offsets as u64, matching the decoder (a memory64 dlmalloc's offsets overflowed the u32 reader). The checked-in wasip1_mem64_hello.wasm fixture — C compiled with clang -target wasm64-wasip1 against the ported wasi-libc — pins printf and malloc end to end; a 5 GiB malloc with stores at 0, 4 GiB and 5 GiB-1 runs correctly through the same chain. Co-Authored-By: Claude Fable 5 <[email protected]> * Cover the llama-wasm WASI surface in the widened wasm64 ABI Extend the *64 wasip1 bindings from the initial six methods to the full import set a wasi-libc-linked C/C++ program (llama.cpp included) uses: args/environ, prestat, path_open, fd_read, fd_readdir, path_filestat_get, fd_fdstat_set_flags, and random_get. Path_open and Fd_readdir keep a single layout-independent core (pathOpen / fdReaddir) shared by both pointer widths; the LP64-layout differences (8-byte guest pointers and __wasi_size_t, 16-byte iovecs, 16-byte prestat) live only in the thin *64 wrappers. Add a checked-in wasm64 C++ exception fixture (typed catches, catch-all, unwinding through std::string/vector/unique_ptr) built with -fwasm-exceptions against the wasm64 libc++/libc++abi/libunwind runtimes, plus unit tests for every new *64 binding. Co-Authored-By: Claude Fable 5 <[email protected]> * Share the layout-independent WASI cores across pointer widths Fold the remaining duplicated bodies between the wasm32 and wasm64 wasip1 bindings into shared cores, so each binding pair keeps only its ABI layout: - clockNanos: the clock_time_get body. - fdSeek: the fd_seek body. - readVec/writeVec: the scatter/gather I/O loops, fed by per-width iovec resolvers (iovecSlices / iovecSlices64). Every iovec entry is now validated before any I/O happens, so a bad entry faults the call instead of surfacing after a partial transfer. No behavior change otherwise; wasm32 and wasm64 fixtures and the WASI unit suites pass unchanged, coverage 83.7% >= 83%. Co-Authored-By: Claude Fable 5 <[email protected]> * Fix three memory64 gaps surfaced by transpiling llama.cpp as wasm64 Verified by rebuilding the full llama.cpp bridge for wasm64-wasip1 and running the go-llama test suite on the transpiled output (all green): - A v128 crossing a defined-function boundary was typed `any` in the generated signature (goTypeOf had no ValV128 case). wasm32 llama never passed a v128 by value, so this never fired; the wasm64 build does. Map it to the scalarized [2]uint64 pair the SSA emitter uses for every v128 expression. - The f16 store-side idiom chain fused a cvt+store helper whose address parameter is i32; on a memory64 module addresses are i64 and the intermediate forms assume wasm32 v128 typing. The chain is now wasm32-only — a memory64 module's SIMD memory ops run as plain m64 helper calls without splicing, so the rewrite bought nothing there anyway. - The bulk-dispatch Inv_<svc>_<mt> wrappers hardcoded (int32, int32) request parameters; on a memory64 module the bridge exports take pointer-width i64 (the C side declares them void*/size_t). Co-Authored-By: Claude Fable 5 <[email protected]> * Splice memory64 SIMD memory ops with the shared vector bodies A memory64 module's SIMD memory helpers (simd_m64_*) were excluded from every splice path, so each load/store ran as a plain function call — measured on the llama.cpp module as a 36x token-generation gap against the wasm32 build. There is nothing wide about the vector bodies themselves: arm64/amd64 effective addresses are computed in 64-bit registers either way, so the m64 family now rides the same splice tables with only the address glue widened. - simdSpliceOp strips the m64_ marker (plain and pair forms) and reports addr64; the op after stripping indexes the same tables. - The a64/x64 memory preambles take addr64: full-width MOVD/MOVQ address operands instead of the u32 zero-extensions, with carry-checked sums (two u32s can never wrap a u64, two u64s can — and a wrapped effective address could land back inside bounds). - The scalarizer treats simd_m64_* as pairable via the same table (m64_ stripped), emitting simd_p_m64_* pair calls; the pair-wrapper family is added to the helpers. - Fused-tree/loop lookups explicitly skip addr64 sites for now: the fused emitters still assume u32 address arithmetic. Result on the llama.cpp wasm64 module: 7148 of 7149 SIMD call sites spliced, tg 1.6 -> 9.7 tok/s, pp 1.8 -> 11.4 tok/s. Porting tree/loop fusion (SDOT/SMMLA superinstructions included) to addr64 is the remaining lever toward wasm32 parity. Also fix an unbounded recursion in the fused-tree scalar chaser: a loop-carried definition (v = v + 4 in the goto-form emission) made chaseI32 recurse through interDef forever — a stack overflow on the llama.cpp module built at -Oz, whose loop shapes surface such definitions. A visiting mark leaves self-referential names as interveners. Co-Authored-By: Claude Fable 5 <[email protected]> * Fuse SIMD trees on memory64 modules Extend tree fusion (and with it the superinstruction rewrites that run inside fused regions) to memory64 modules; only fused LOOPS stay width-gated for now. Measured on the llama.cpp wasm64 module: tg 10.2 -> 17.9 tok/s, pp 13.2 -> 22.2 tok/s, with the go-llama suite green. The design keeps one width-independent op vocabulary: - The builder strips the m64_ marker for every table lookup and node name (fuseLookupName) and marks the tree Addr64. A memory64 member's address/offset always ride as OPAQUE int64 scalars — Arg.Const is 32-bit and ArgSum assumes u32 wrap, so folding either would truncate a 64-bit address; the emitters enforce this shape. - Addr64 trees take every scalar parameter as int64 (same register assignment); call sites widen non-address scalars losslessly, and the fallback body narrows them back for the int32-consuming pure helpers while the memory members call their simd_m64_* twins. - Both fused emitters widen only the address glue: full-width moves with carry-checked sums (two u32s can never wrap a u64; two u64s can, and a wrapped effective address could land back inside bounds). The vector node bodies — dot8/SDOT rewrites included — are shared unchanged. The amd64 store tail (value staging, f16 conversion chain, the store proper) is extracted so both address paths share it. - Fused loops reject Addr64 trees at the upgrade point: the loop machinery still assumes u32-wrap pointer bumps and int32 exit scalars. The loop body keeps its fused-window form meanwhile. Co-Authored-By: Claude Fable 5 <[email protected]> * Widen the fused-loop machinery for memory64 trees Lift the Addr64 gate off the loop upgrade and widen every piece the gate was standing in for: loop-invariant stride parameters and non-counter exit scalars are int64 on an Addr64 tree (they are its pointer parameters), pointer bumps advance at full width on both splicers (ADD/ADDQ instead of the u32-wrap W/L forms — the body's per-node carry-checked bounds keep a runaway pointer from ever being dereferenced), and the after-loop fix-up temporaries follow the width. On the llama.cpp wasm64 module no loop upgrades fire YET: the scalar chase (chaseAddr/chaseI32) only recognizes the wasm32 address shapes (uint32 conversions), while the memory64 emitter derefs through `unsafe.Add(mBase, uint64(v - int64(c)))` — so the scale-load interveners cannot join the body window and the upgrade's whole-body-in-one-region requirement fails. Extending the chase grammar to the memory64 shapes is the follow-up that turns the machinery on; this change makes it correct once matching succeeds, and is inert until then (all suites and the wasm64 llama build are byte-for-byte at the fused-window numbers). Co-Authored-By: Claude Fable 5 <[email protected]> * Unify the memory64 and wasm32 SIMD address handling The memory64 splice paths had grown a second, hand-written copy of every address computation — separate mnemonics, and extra carry-checked sums the wasm32 path never had. That duplication was both a maintenance hazard and a measured slowdown. Collapse the two into one width-parameterized path: the ONLY difference between wasm32 and memory64 is now the instruction that materializes an address operand — MOVWU/MOVL (zero-extend i32) versus MOVD/MOVQ (full i64). The effective-address arithmetic and the ea+size ≤ memSize bounds check are identical, shared code. The extra carry checks are removed. They are not needed on either width: a memory64's linear memory is capped at 2^48 bytes (mem64HardCap), so every valid effective address is far below 2^63 and the ea+size ≤ memSize comparison is exact. The one address that could wrap a u64 is one the guest deliberately computes near 2^64 — which wasi-sdk-compiled code never produces, and which even then resolves within the module's own memory slice, so it is never a Go memory-safety issue. This is the same pragmatic bounds model the coalescing pass already documents. a64MemPreambleRegs and x64MemPreamble are now single functions (movAddr chosen once); the fused load/store/lane checks share a64FusedMemCheck / x64FusedEA64. On the llama.cpp wasm64 kernel this lifts tg 17.9 -> 18.6 and pp 22.2 -> 22.7 while deleting the duplicate emission. Bounds coalescing stays off for memory64 (documented): unlike wasm32 it regressed the kernel even with the carry-free forms, which are kept implemented and tested for a future grouping heuristic. Co-Authored-By: Claude Fable 5 <[email protected]> * Extend SIMD unrolling, coalescing and window fusion to memory64 A memory64 module's hot kernels arrive with LP64-promoted shapes the SIMD pipeline did not recognize: i64 loop counters (i64.sub countdown, Ui64 guards), i64 pointer-bump chains, int64-spelled constant locals and % 64 shift masks. Every stage bailed in its own way and the q8_0 vec_dot ran as sparse two-sdot windows full of GPR-pair round trips — 3.3x slower than the same kernel on wasm32. Teach each stage the wide twin of the shape it already handles: - UnrollSimdLoops matches i64 countdowns and builds the guard arithmetic at the counter's width. Wide matching is a per-module opt-in (memory64 only): on wasm32 the i64-counter loops are exactly the ones whose bodies do not window-fuse, and admitting them was measured to cost ~40% prompt throughput there. - ReassocConstAdds folds Add64 chains (memory64 only, same gating rationale) so unrolled i64 pointer bumps share one base. - Bounds coalescing runs on memory64: the 2^48 linear-memory cap makes the coalesced ea+span check exact with no extra precondition. The earlier measurement that kept it off was taken against a miscompiled (-Oz) module and does not hold. - Window fusion folds memory64 addresses as base+const ArgSum (the base rides a full-width scalar; only bare 64-bit ArgConst addresses stay opaque), binds int64-spelled constant locals, resolves int64-typed rng/lane windows, and chases % 64 shift masks and int64-wrapped scale chains; the countdown-loop matcher accepts the Ui64 header and i64 decrement forms. - The arm64/amd64 fused splices materialize ArgSum addr64 bases with full-width adds; the m64 scalar-chain helpers take the mechanical simd_m64_* naming the lookup already strips. Measured on the llama.cpp q8_0 module (M5): memory64 tg 17.9 -> 33.6 tok/s, pp 22.7 -> 34.3 tok/s; wasm32 is byte-neutral by construction on the gated paths and A/B-verified flat on the rest (tg 53.6 vs 53.7, pp 88.8 vs 89.3 warm, interleaved). Co-Authored-By: Claude Fable 5 <[email protected]> * Surface failed window-fusion trials under WASM2GO_FUSE_DEBUG The failWhy field recorded each trial's first refusal but nothing read it, so diagnosing why a module's windows stay small meant ad-hoc printf builds. Print failed trials of four or more candidates (width plus refusal) to stderr when WASM2GO_FUSE_DEBUG is set — the histogram of these lines is what localized both the wasm32 K=4 unlock and the memory64 window starvation (pair-input int-slot exhaustion at pair #7 across the attention kernels). Co-Authored-By: Claude Fable 5 <[email protected]> * Recognize and pair the ggml q8_0 vec_dot on memory64 modules The type_traits_cpu recognizer assumed the ILP32 row layout (24-byte rows, u32 function pointers) and the all-i32 ggml_vec_dot_t signature, so a memory64 module never got the row/column pairing. Parameterize the recognition by pointer width: LP64 rows are 32 bytes with u64 function-pointer fields (table indices — nonzero high bits disqualify a row), the vec_dot signature carries i64 pointers and size_t strides, and the paired-tile companion takes matching int64 parameters (its pointer arithmetic is unchanged in shape). The arm64 SMMLA tile kernel follows: the only width-dependent part is the ABI0 argument frame — int64 arguments land 8-aligned (l1+16 .. l6+56, 64 argument bytes) and load with MOVD — while the 2x2 tile body is byte-identical, which the kernel test asserts by stripping the frame lines and diffing the remainder. Measured on the llama.cpp q8_0 module as wasm64 (M5, fast-math): pp 35.8 -> 45.7 tok/s (+28%, the same jump the pairing gave wasm32); tg unchanged as expected (odd batches keep nrc == 1). Full go-llama suite green, including the native-token-match gates. Co-Authored-By: Claude Fable 5 <[email protected]> * Let memory64 stores reach window fusion Two width-blind name checks kept every memory64 SIMD store out of window fusion, running ggml's conversion and scale loops as chains of marshalled pair calls: - IsVoidAtomicStore matched only the simd_v128_store prefix, so a simd_m64_v128_store's dummy result was treated as usable and the store hoisted into `vN = ...` assignment form — which the window candidate scan (expression statements) never sees. - The candidate scan itself looked the mark's raw name up in fusedStoreOps without stripping the memory64 marker. Fix both to accept the m64 spelling, and pin the behavior with twin fixtures (cg_simd_store32 / cg_mem64_simd_store): the elementwise store-of-mul(load, load) shape must produce a fused tree containing a store sink on BOTH pointer widths, Addr64-tagged on memory64. On the llama.cpp wasm64 module this fuses 2163 store sites (6320 -> 4157 bare); wasm32 output is unchanged by construction (both checks are identity for non-m64 names) and the twin test pins its store fusion too. Co-Authored-By: Claude Fable 5 <[email protected]> * Pin conversion-loop window fusion on both pointer widths Twin fixtures for the variable-stride row-scale skeleton (per iteration: v128 load, load32_splat scale, mul, add, store; pointer bumps by a runtime stride). Both widths currently fuse the whole iteration into one region — pinned so the memory64 side cannot silently regress to bare pair calls while the remaining divergence in the full llama conversion loops is chased. Co-Authored-By: Claude Fable 5 <[email protected]> * Deduplicate opaque scalar window arguments by structural key Identifier-typed scalar arguments already shared one slot per name; expression-typed ones (a variable-stride address like `base + s`) burned a fresh int slot per occurrence, so windows over conversion loops hit the slot cap early. Key them by their printed form — safe because side-effecting values are hoisted into named locals before fusion (these expressions are pure) and noteRead records the contained identifiers for the same interference checks as the identifier path. Verified perf-neutral on wasm32 (interleaved A/B: tg 53.6 avg both sides, pp 89.5 vs 89.9) with both llama suites green; no measured memory64 throughput change yet — the hot LP64 conversion loops turn out to be bound elsewhere — but the slot pressure reduction is structural and width-independent. Co-Authored-By: Claude Fable 5 <[email protected]> * Run the f16 idiom rewrites on memory64 modules The whole f32->f16 store-side chain (idiom recognition, empty- diamond folding, the 4x-store16 merge, the fused cvt+store) was gated to wasm32, so a memory64 module kept the full software idiom in its hottest conversion loops. The chain is width-neutral except at three points, now parameterized: memRoot/constOf walk Add64/ Const64 address chains, FuseF16CvtStores takes the module width and spells the i64 offset plus the simd_m64_* helper, and the m64 cvt+store helper twins (array and pair forms) land next to the m64 store family. The load-side gather recognizer picks up the widened address walk for free. Verified: pass unit tests cover the m64 spelling; both llama suites green (the conversion is bit-exact by construction — same rounding chain, NaN forcing); wasm32 interleaved A/B flat (tg 47.6 avg both sides, pp 82.9 vs 83.1); memory64 interleaved A/B ahead on tg in every adjacent pair (~+0.8 tok/s warm), pp flat. Co-Authored-By: Claude Fable 5 <[email protected]> * Chase variable-stride memory addresses into scalar-node chains A load or store whose address is a runtime sum with no constant side (`base + stride`, the shape strided conversion loops carry — three terms on LP64) previously rode one opaque int slot per distinct expression, so windows over such loops hit the slot cap and shed their loads. Chase these addresses like the scale chains: the sum becomes scalar_i32_add nodes whose LEAF identifiers deduplicate by name, so any number of loads sharing a stride variable cost one slot for it. The emitters accept a chain-computed address terminal (takeAddrSource, the GPR analog of takeSplatSource) and move it into the address register before clobbering scratch, on both arm64 and amd64, both pointer widths, with tryFuseLoop's rejection points now logged under WASM2GO_FUSE_DEBUG. Twin fixtures pin the behavior (cg_simd_addrsum32 / cg_mem64_simd_addrsum: a mem node with an ArgNode address must appear on both widths). On the llama.cpp modules: the hot memory64 conversion loop's bare loads drop 66 -> 26, and wasm32 gains its first fused-LOOP upgrades on this module (fxl0/fxl1) because whole bodies now fit one window. Throughput is A/B-neutral on both widths (the loop's remaining cost is gc frame traffic between windows, not the bare ops) — this lands as groundwork and unification, with both llama suites green including the native-token-match gates. Co-Authored-By: Claude Fable 5 <[email protected]> * Upgrade head-tested unit-decrement loops and unwrap memory64 call args Two blockers kept real conversion loops out of the fused-loop upgrade. First, the emitter produces a THIRD countdown shape the matcher missed on both widths: a head-tested while, `for { if c == 0 { break } else { body; bumps; c = c - 1 } }`. With a unit decrement `c == 0` exits at exactly the same counter values as the pre-tested `Ui(c) < Ui(1)`, so the shape reuses the pre-test descriptor; any other decrement under an equality head is rejected. Second, memory64 fused-call sites wrap their scalar arguments in int64(...), which broke the upgrade's slot and reassociated-bump matching by identifier — both now unwrap the width conversion first. The conversion-loop twins now upgrade on BOTH widths (pinned by TestConvLoopUpgradesBothWidths — the first exercise of the memory64 loop-splice path), and the wasm32 llama module gains five fused loops. Throughput is A/B-neutral on both widths (the upgraded loops sit off the hot paths); both llama suites green including the native-token-match gates. Co-Authored-By: Claude Fable 5 <[email protected]> * Fix dropped-definition reads exposed by nested address chains The address-chase commit made nested chains reachable (`v80 = v67 + s` with v67 itself an intervener), and two holes in the surrounding bookkeeping produced wrong stores in the variable-stride axpy shape — caught by the gcasm differential fixture (the llama greedy gates do not exercise it): - An intervener counted as fully consumed when the chases internalized all its reads, and was dropped — even when a KEPT intervener's definition still read it (v80 stays because a loop tail reads it, but v80's kept RHS read the dropped v67, leaving the tail bump computed from an unassigned variable). Un-drop such definitions to a fixpoint. - The chase could leak a WINDOW-INTERNAL value (another candidate's result, e.g. an extract_lane feeding an address) into a chained scalar argument, which has no pre-call definition after fusion. Refuse at the chase so the address falls back to the original pair/opaque forms. TestGcasmSimdLoopUnroll (the value-level differential over the unroll/fuse/loop-fuse configurations) fails before and passes after. Co-Authored-By: Claude Fable 5 <[email protected]> * Gate unamortized packed outlining on memory64 modules A packed outline boundary writes every live-in into the scratch array and reads it all back on each call. On a memory64 module the i64 locals double the caller's register pressure and the round trip stops paying for wide boundaries: refusing packs whose body carries fewer than 24 values per slot gains ~17% prompt throughput on the llama.cpp module (the ~130-slot conversion loop runs inline instead). The SAME gate on wasm32 loses ~25% prompt throughput — its conversion loop compiles better extracted than inline — so the ratio applies only when the module is memory64, measured both ways by interleaved A/B with both llama suites green. Co-Authored-By: Claude Fable 5 <[email protected]> * asmgen: import direct-asm backend and add br_table dispatch support Import the SSA-to-assembly emitter (internal/asmgen) and its register allocator (internal/regalloc) so hot scalar functions can bypass the gc backend's code quality ceiling — the remaining wasm64 throughput gap is dominated by frame-reload storms gc generates for doubled i64 live sets in the conversion/dot-product driver loops. The emitter predates two SSA block kinds, so bring it up to the current IR: - BlockBrTable: new EmitBrTable arch hook emitting a linear compare chain (selector materialized once into the scratch register, one CMP+branch per case value, then an unconditional jump to the default). Successors carrying phis are rejected to the per-function fallback — no dispatch-shaped code produces them today, and edge copies would need per-successor intermediate labels. - BlockThrow stays unsupported; functions using it keep their gc- compiled Go bodies via the existing per-function stub fallback. LowerFunction calls gain the ThrowSet parameter added since the emitter was written. Testing: the driver harness is now arch-parameterized, and the control-flow value test (if/else merge, loop phis, br_table chain) runs natively on both amd64 and arm64 — Apple Silicon executes the arm64 driver even when the test process runs under Rosetta, detected via sysctl.proc_translated. Co-Authored-By: Claude Fable 5 <[email protected]> * asmgen: memory64 addressing for loads, stores, and memory ops Teach the direct-asm emitters 64-bit linear-memory addressing so memory64 modules can route hot scalar functions through asmgen. The effective-address computation is factored into one per-arch helper (emitMemAddrAMD64 / emitMemAddrARM64) shared by the load and store paths; the two memory widths use the same mechanism and differ only in the address rule, mirroring the pure-Go emitter: - wasm32: address = uint32(base + off), computed in 32-bit so negative-int32 bases wrap per the wasm effective-address rule (unchanged behaviour, now in one place per arch instead of four). - memory64: address = base + off in full 64-bit, no wrap — the pure-Go path is `uint64(base) + offset`. On amd64, constant bases fold into the displacement when the total fits int32 and larger offsets are materialised through a scratch register before the index add; on arm64 the assembler's REGTMP materialisation keeps the shape a single ADD regardless of offset size. OpMemSize / OpMemGrow / OpMemoryCopy / OpMemoryFill on memory64 modules route to the 64-bit helper family (memorySize64, memoryGrow64, memoryCopy64, memoryFill64) with the i64 signatures the codegen helpers define. Testing: cg_mem64_memops.wat is the memory64 twin of cg_memops.wat (every load/store width and sign, offset memargs, constant-base fold, i64 mem-size/grow), value-checked on amd64 and natively on arm64. The wasm32 memops value checks also gain an arm64 twin so both widths are execution-verified on both arches. Co-Authored-By: Claude Fable 5 <[email protected]> * Direct-asm backend: per-function asmgen bodies inside the gcasm bundle Wire the SSA-to-assembly emitter (internal/asmgen) into the pipeline as an opt-in per-function backend coexisting with gcasm. gc's register allocator treats every CALL as clobbering everything, which spills large i64 live sets to the frame around fused-window calls; emitting those functions straight from SSA is the lever for removing that reload traffic. This change lands the plumbing with a conservative first eligibility class (call-free leaf functions) and per-function fallback everywhere else. Flow: - codegen: Options.DirectAsmFuncs opts function names in. The translator retains their finalized SSA (post optimization fixpoint, idiom rewrites, and outlining — outlined loop functions with unpacked scalar boundaries are eligible too) in Result.DirectAsmSSA. Direct-asm modules with a memory also emit the gcasm memory probe so the bundle can verify the Module.M offset the asm bodies hardcode. - gcasm: Config.DirectAsm carries the retained SSA into Build, which emits each function per architecture via asmgen (buildDirectAsm) and swaps the body in place of the gc-listing transform in buildPkg. Bodies carry a "// direct-asm: <name>" marker; decls reuse the transform path's shape (declSig, now shared). Emission failures fall back to the transform per function and are counted (BuildStats.DirectAsm / DirectAsmFallback). - asmgen: FuncOptions.ForbidCalls fails emission for any function that would CALL out (helpers, callees, memory ops, global wrappers) — the bundle's callee symbols are not resolvable from asmgen's same-package CALL spelling yet, and a body that emits but fails to link would break the build instead of falling back. ModuleMOffset is exported for the host-side layout check. - cmd/wasm2go: -direct-asm flag (comma-separated names). Testing: the arith fixture runs through the full backend with five functions opted in — four leaf bodies come from asmgen on both architectures (marker asserted), the helper-calling one falls back — and the generated package computes the same values the asmgen driver tests pin. Co-Authored-By: Claude Fable 5 <[email protected]> * asmgen: v128 values and SIMD helper CALLs Extend the direct-asm emitters to functions containing SIMD, the shape the hot wasm64 loops actually have. v128 values live in 16-byte stack slots as [2]uint64 pairs (slotSize / abiSize gain the class); OpSimdConst materializes its lane payload into the slot; and OpSimdCall / OpSimdMemCall lower to plain ABI0 CALLs of the bundle's simd_* helper symbols — the same symbols the pure-Go backend calls. There is no static signature registry for the SIMD family: each call's layout derives from the SSA value itself (arg types, result type, module pointer for memory ops), shared between the arches in simdCallSpecOf. v128 args and results travel as two 8-byte halves; phi edge-copies and staged-phi slots handle the type on both arches. ForbidCalls narrows from "any CALL" to non-SIMD callees (hasNonSimdCall): SIMD helper symbols ship with every SIMD-using bundle, so their CALLs always resolve, while scalar helpers, sibling functions, memory ops, and global wrappers keep the per-function fallback until the bundle's wrapper spelling is wired up. Cross-package emission (multi-package chunks) is still rejected in simdCallSpecOf for the same linkability reason. The cross-block register allocator predates the 16-byte slot class, so functions containing v128 values keep the block-local allocator (whose pool selection already skips v128-typed values). Testing: the SIMD fixture runs through the full backend with every function opted in — the v128-bearing bodies come from asmgen (helper CALLs then still spliced or kept by the existing gcasm machinery for the rest of the bundle) and all values match the wazero-verified reference the gcasm test pins. Co-Authored-By: Claude Fable 5 <[email protected]> * asmgen: splice SIMD helper bodies inline via the gcasm tables Replace the marshalled SIMD helper CALLs in direct-asm bodies with the same inline NEON bodies the gc-capture transform splices — the step that makes the direct backend competitive on SIMD-heavy functions instead of regressing to the pre-splice call-overhead world. The splice machinery lives in gcasm, which already imports asmgen, so the dependency inverts through an interface: asmgen's FuncOptions.Splicer receives operand locations (frame slots — SIMD- containing functions emit in slot-only mode: no register homes, no m-cache, no loop-carry coalesce, because splice bodies clobber registers freely) plus a scratch area (the callee-argument region, always a superset of the splice's stack traffic). gcasm's implementation stages the operands per the same ABIInternal assignment the listing transform uses (scalars into R0../F0.., v128 halves into the scratch area at their sequence offsets) and drops in a64SpliceSimd unchanged, sharing the per-package ConstPool. Memory splices branch to the shared out-of-bounds stub, which the emitter appends once at body end. Everything goes through a scratch builder first so an op the tables decline falls back to the CALL with no staging leaked. Direct-asm emission consequently moves inside buildPkg (it needs the package's ConstPool); amd64, which has no per-op splice table, keeps the marshalled CALLs. Testing: the SIMD fixture's direct-asm arm64 bodies are asserted to contain spliced NEON, and the generated package is now also cross-built for arm64 and EXECUTED natively (Apple Silicon runs the binary even when the test process is under Rosetta), matching the wazero-verified reference values. Co-Authored-By: Claude Fable 5 <[email protected]> * asmgen: coalesce loop carries into splice-safe registers The reason the direct backend exists: gc must assume every call site clobbers every register, so the hot loops' i64 carries live in frame slots and each iteration pays a reload storm around the SIMD ops. Inline splice bodies, unlike calls, have a documented clobber contract (arm64: R0–R15 / R25–R27 and the V registers) — so R19–R24 survive them, and asmgen can keep the loop-carried scalars there. runSpliceCoalescePass is the splice-mode variant of the loop-carry coalesce: SIMD helper calls inside a loop no longer bar coalescing, and every such value in a coalesced loop is recorded in plan.mustSplice — a splice-table miss there fails the function to the host's fallback instead of emitting a CALL that would clobber the carry. The pool is R19–R24 (the fused-window splices claim R20–R23 as window state; the pool must shrink when asmgen learns to emit those). The original coalesce safety rule — the phi may have NO user but its own bump — denies the canonical counter/pointer carries (read by the loop condition or an address user AND the bump). The splice-mode pass uses a finer analysis (coalesceUsesSafe): every read must execute before the bump's register write in each iteration, which holds when the bump sits in the latch block (the latch runs last — its only successors are the header and loop exits), reads in the latch precede it, and the phi has no reads outside the body, via another phi's edge copy, or as the latch's control. The default pass keeps the single-use rule pending an A/B. Two correctness fixes surfaced by this work: - opEmitsCall did not know OpSimdCall / OpSimdMemCall, so their marshalled CALLs were invisible to the m-cache re-prime logic and the coalesce call barrier — a latent wrong-code hazard for CALL- mode SIMD bodies. - The arm64 lane-op memory splices used R24 as their data scratch, violating the splice clobber contract this change relies on. R4 (inside the contract set, free — the helper's args occupy R0–R3, and unlike R25–R27 it survives the memory preamble) replaces it. Testing: a self-contained SIMD accumulation loop fixture with pointer/counter carries runs differentially — direct-asm output vs the normal backend across 33 trip counts — on amd64 and natively on arm64, and the arm64 body is asserted to contain both spliced NEON and an R19 carry. Co-Authored-By: Claude Fable 5 <[email protected]> * asmgen: multi-package emission, packed boundaries, inline helpers Three pieces that carry the direct-asm backend from fixtures to the llama-scale bundle: Multi-package. The linkname-split translator builds its own Result and dropped DirectAsmSSA entirely — retention silently never reached the bundle (the chunked layout is the only one llama uses). With the field wired through, chunk emission works on the splice-only contract: spliced SIMD sites need no symbols at all, so a chunk function is eligible whenever every SIMD site splices — the multi-package rejection moves from spec time to the CALL fallback path, which now fails the function to the transform instead of emitting a same-package spelling that cannot resolve. The out-of-bounds trap CALL resolves through buildPkg's callee-resolution closure (registering the chunk-local forward wrapper), and a splicer without a resolved trap symbol refuses memory splices. Packed boundaries. llama's outlined SIMD loops carry v128 values across the boundary, so nearly all of them use the packed form — the caller passes only the module pointer and the values ride Module.OutlinePack. Retention keeps them (results-only signature plus the parameter types in slot order), and the emitters gain a pack prologue: parameter values load from the pack scratch (pinned at moduleMOffset+8 by the same field-order construction that pins M) into the params' frame slots, parameter reads resolve to those slots instead of FP offsets, and OpParam slots are excluded from slot reuse (a pooled reuse would clobber the prologue's write). Inline helpers. The first llama fallback wave was all trivial scalar helpers (sign/zero extends, 32-bit reinterprets, f32_abs) that the amd64 emitter already lowers to one or two instructions but arm64 routed through a marshalled CALL. helperAlwaysInline names the set both arches now emit natively; planFunc exempts those calls from hasCall (no callee frame, no ForbidCalls consequence), and both emitters guard the CALL path against ever being reached for an exempted name — the frame it would use was never reserved. The ForbidCalls diagnostic now names the offending callee, which is what surfaced the wave in the first place. Co-Authored-By: Claude Fable 5 <[email protected]> * asmgen: inline float ops, global offsets, resolver-backed callees Three widening waves, each driven by the ForbidCalls diagnostic (which now lists every distinct blocker per function instead of the first) against the llama-scale bundle: Inline float family. The arm64 emitter gains native lowerings for the float compares (FCMP + CSET with unordered-false condition codes, NE unordered-true — wasm/IEEE semantics, matching gc's own lowering), binary arithmetic (FADD/FSUB/FMUL/FDIV both widths), neg/abs, promote/demote, int→float converts, and i32_wrap_i64; all of these were already inline on amd64. helperAlwaysInline widens accordingly. Global offsets. Module-defined globals inline as MOVs against the module pointer. The translator — the one component that actually emits the Module struct — computes each global's byte offset following its own field order, ships them through Result.DirectAsmGlobals into the bundle, and emits compile-time assertions (directasm_layout.go) pinning every offset direct-asm hardcodes: M, the outline pack, and each global. Layout drift then fails the consumer's build instead of corrupting memory. Resolver-backed callees. Direct calls and the memory-op helper family resolve through buildPkg's callee-resolution closure — the same machinery the listing transform uses, registering chunk-local forward wrappers as a side effect — via two FuncOptions hooks (CalleeSymbol / MemHelperSymbol). Resolved calls are exempt from ForbidCalls; unresolved ones still fall back per function. With these, nine llama functions emit as direct asm on arm64 (including every outlined loop of the compute monolith), and the full go-llama verification suite passes on the resulting bundle: greedy determinism, prompt batch-size invariance, native NLL parity, and cross-context repeatability. Co-Authored-By: Claude Fable 5 <[email protected]> * asmgen: inline integer div/rem on both arches The divide/remainder family was the largest remaining direct-asm fallback class. Lower it inline: native divide plus explicit trap-branch CALLs, with the trap symbols resolved through the same host hook as the memory helpers (forward wrappers in multi-package chunks) — an unresolved trap helper keeps the marshalled-CALL path instead of emitting a spelling that cannot link. arm64 (new): SDIV/UDIV for quotients, MSUB for remainders. arm64 divides never fault, so the wasm traps are explicit: every op branches to the div-zero trap on a zero divisor, and div_s checks INT_MIN / -1 for the overflow trap. rem_s needs no overflow check — SDIV wraps the quotient to INT_MIN and MSUB computes x - q*y = 0 with the same wrap, which is exactly wasm's answer. Results honor plan.regHome (the arm64 regalloc register-homes helper-call results; storing only to the slot left downstream readers of the home register with garbage — caught by the new native-execution arith value test, which now covers the div family on arm64). amd64 (fixed): the existing inline div_s relied on IDIV's #DE for the INT_MIN / -1 case, which panics with the divide-by-zero kind; wasm calls it integer overflow. Both widths now check explicitly and call the overflow trap. All trap CALLs go through the plan's resolved symbols instead of hardcoded same-package spellings. Also guard the outline pass's synthetic sibling-call sentinel (OpCallDirect with a negative index and the callee name in Aux): planFunc indexed the module's function table with it and panicked on nested outlined loops; it now counts as an unresolvable callee, so ForbidCalls falls the function back cleanly. The packed call-site protocol for such calls is future work. At llama scale this raises the arm64 direct-asm body count from 58 to 76 at outline threshold 100 (9 at the pipeline's threshold 400), with the verification gates green throughout. Co-Authored-By: Claude Fable 5 <[email protected]> * asmgen: hoist module state across splices; splice the rng/nc loads Two body-quality steps, both llama-verified for correctness: Module-state hoisting. Every arm64 memory splice reloaded m from its FP slot and m.M / the memSize pointer from the Module per site — three loads per memory op, plainly visible in the pp profile. The emitter now keeps them in the splice-safe registers the coalesce pool cedes (R24 = m via the m-cache mechanism, R23 = m.M, R22 = the memSize pointer, staged by the splicer's new HoistPrologue), leaving R19–R21 for loop carries. The refresh after calls became precise in the process: emitters that actually produce a CALL set a per-value flag, replacing the op-kind heuristic that would have re-primed after every spliced SIMD op (and, previously, missed nothing but paid refreshes after inlined helpers). Memory-less functions skip the prologue — staging it read a garbage m and faulted, caught by the SIMD fixture's arm64 leg. Bounds-coalesced loads. The hot conversion loops read memory through the range-check group forms (v128_load_rng leader / _nc members), which only the pair-form splicer knew; the array-form path direct asm uses kept them as CALLs, so the hottest outlined loops always fell back. The array-form twins mirror the pair bodies (signed group start, TBNZ wrap check, leader-only bounds compare) with the hoisted-register variants, and both hot loops now emit — 107 arm64 bodies at outline threshold 100. MEASURED STATUS, not yet a win: with the hot loops emitted, prompt eval REGRESSES ~2.9x on the direct bundle (correctness gates all pass). The per-op splice protocol round-trips every v128 value through its stack slot between adjacent ops — the load feeds a splat/mul/add chain entirely through memory — where the gc-capture path's fused windows keep the chain in V registers. Vector-value register residency across splices is the remaining (and now precisely measured) gap; nothing ships through this path until it closes — the pipeline's bundle does not opt in. Co-Authored-By: Claude Fable 5 <[email protected]> * asmgen: slot-direct v128 splice protocol and spare-register caching The measured 2.9x prompt-eval regression on the direct-asm hot loops decomposed into two protocol costs, both addressed here: Slot-direct splicing. The splicer staged every v128 argument from the value's frame slot into the ABIInternal scratch area through GPR pairs and copied every result back the same way — four extra memory ops on each side of every op. SpliceSlots now carries the values' own RSP offsets into the splice emitters (argument reads, result stores, lane addressing), so the scratch hops vanish; the transform path passes nil and keeps its scratch behavior, where the values genuinely live there. asmgen ships the offsets on the operands (InSlot/SlotOff) — an unpacked FP parameter is the one v128 without a slot and keeps the copy path. Spare-register value caching. Every splice body computes through V0, so no value survives in a register past the next op and plain store-to-load forwarding never fires. a64SpliceValueCache parks each "// simd out" result in a spare vector register (V17–V31, strictly above anything a splice body touches) and rewrites later reloads of that slot into single-cycle register moves, with the sibling passes' conservative invalidation (labels/CALL reset, width-aware RSP store overlap, pointer stores drop everything). The result store always remains — it is the value's only definition. Two bugs found by the gates on the way: - a64DeadArgStores must not run on direct-asm bodies: it models the capture shape, where arg-slot stores are redundant copies; in the slot-direct shape the "simd out" store IS the definition, and the pass deleted live stores. The native NLL parity gate caught the wrong values (greedy and batch invariance did not — a lesson in gate sensitivity). - a64SpliceWordClobberList omitted v4, which the f16 conversion body writes ("movi v4") — a value the forwarding pass parked in F4 would survive that splice on paper while being clobbered in fact. Prompt eval on the direct bundle: 19.5 → 44.6 tok/s across these steps (baseline 55.5; the remaining gap is scalar address traffic through frame slots and the per-op bounds preambles). All verification gates pass at every step. Co-Authored-By: Claude Fable 5 <[email protected]> * asmgen: scalar slot forwarding and dead result-store elimination Two more text passes over direct-asm arm64 bodies, closing most of the remaining per-op splice overhead: Scalar forwarding (a64ScalarSlotForward). The slot-only emission spells every scalar intermediate as a store/reload pair through its frame slot — address chains read as MOVD store, MOVD reload, ADD, repeat. The pass tracks full-width register→slot bindings within straight-line spans and rewrites reloads into register moves (or drops them when source and destination coincide), with the sibling passes' conservative rules: labels and CALL/RET reset, WORD splice bodies kill bindings in the contract clobber set (R0–R15/R25–R27), pointer stores drop the slot side, narrower stores only invalidate. Dead result stores (a64DeadSimdOutStores). Once reloads are forwarded to the spare-register cache, a single-use v128 intermediate's "// simd out" store serves nobody — but only a function-scoped read count can prove it (a later block may reload the slot). The pass counts every RSP-offset mention as an operand — comments are explicitly NOT reads: a spare-forwarded reload names its origin slot in a comment only, and counting those pinned nearly every store (the first cut measured neutral because of exactly this). Slot addresses that escape through `ADD $off, RSP, Rn` count as reads of both halves, and any unmodeled RSP form keeps the whole body. The spare park always stays: it is what serves the forwarded consumers. Prompt eval on the direct bundle, same config and gates green at every step: 45.4 → 54.3 tok/s (the dead stores were nearly all of it once the comment rule landed); the gc-backed baseline sits at 55.7. The cumulative ladder from the first correct slot-direct build stands at 19.5 → 54.3. Co-Authored-By: Claude Fable 5 <[email protected]> * asmgen: v128 loop-carry homes in the splice-safe vector registers Coalesce v128 loop-carried phis (the accumulators) into V25–V31 — above both the splice bodies' registers (V0–V4, F16) and the spare cache (now V17–V24) — so an accumulator add-chain runs entirely in registers across iterations instead of round-tripping its slot. The splice protocol gains register operand forms: SpliceSlots carries per-argument register sources and a register destination, the table path reads/writes them with single VORR moves, the memory path covers the store's value argument and every load result, and the lane paths — which index the value's memory — spill a register-resident operand to its scratch slot first and load a register-destined result back after the in-place edit (loop exits only; the hot path never lanes a homed value). Safety differs from the GPR carries in one respect: a V home survives to function end (the pool is consumed once per function, no splice touches the range), so live-out reads of the accumulator — the after-loop store or lane fold every reduction ends with — are allowed where a GPR carry must refuse them. That in turn is only sound because v128 coalescing is gated on a function with no non-SIMD callees and forces every SIMD site in the function to splice (a fallback CALL would clobber the vector file). Phi copies read and write homes directly (a register-resident source stores with one FMOVQ instead of two GPR round-trips; the entry edge moves register-to-register when both ends are homed). llama-scale: the hot conversion loop carries 17 home references, all gates pass; throughput is neutral-to-slightly-negative on this loop (its accumulation is mostly block-local, already served by the spare cache, and the loop-exit lane spills give some of the win back). The infrastructure is what the fused-window step needs either way. Co-Authored-By: Claude Fable 5 <[email protected]> * codegen: fire loop fusion on memory64 modules A memory64 llama build ran prompt eval 40% behind its wasm32 twin from the same source. The differentiator was loop fusion (fxl): the wasm32 bundle upgrades five countdown loops — including the q8_0 vec_dot kernel, whose fused form carries the 44-SDOT paired column kernel — while the memory64 bundle upgraded none, leaving the same kernel as a 10-SDOT per-window loop. Dumping the loop bodies as the fuser sees them showed the memory64 vec_dot loop statement-for-statement identical to the wasm32 twin up to spelled widths. Two width-blind checks rejected it: - fuseConstOf did not recognize int64(K): the loop upgrade's leftover classification (constant locals hoist above the loop) refused the body because the bump-delta local `vN = int64(136)` did not count as a constant, failing the whole upgrade for every memory64 loop whose const CSE landed inside the body. - The header form derived the counter's width only from the decrement terms' spelling. When every term is a const LOCAL (`c = c - v170 - v170 ...`), nothing is spelled at any width and the descriptor defaulted to i32 — generating an fxl helper whose counter parameter mismatched the module's promoted i64 counter. matchUiLt/matchEqZero now report which width the guard matched (Ui64(c) < Ui64(K) is the counter's own width) and the descriptor carries it. With both fixed the memory64 vec_dot upgrades to the same fused loop as wasm32 (42 SDOT vs 44). Measured on the llama q8_0 bundle, all token-equivalence gates passing at every step: prompt eval 54.3 -> 68.8 tok/s (+27%), decode 30.1 -> 46.7 tok/s (+55%). wasm32 output is unchanged (the new width legs only accept forms wasm32 never spells). Also under WASM2GO_FUSE_DEBUG: the countdown matcher dumps large loop bodies as the fuser sees them, and upgrade refusals of large loops log the rejecting check with the body size — the two probes that located this. Co-Authored-By: Claude Fable 5 <[email protected]> * codegen: log loop-window candidate collection under fuse debug The countdown-loop upgrade's window trial can fail before any walk runs — candidate collection stops at the first unsafe intervener — and nothing reported where. Under WASM2GO_FUSE_DEBUG, large loop bodies now log the stopping statement and the collected counts, and big-loop window trials log the walk verdict with scalar/pair usage. These probes located the memory64 conversion-loop blocker (eight irreducible address bases exceed the fused-signature slot budget). Co-Authored-By: Claude Fable 5 <[email protected]> * codegen: check the debug printer's error return Co-Authored-By: Claude Fable 5 <[email protected]> * gcasm: harden fused-splice capacity handling Three latent hazards around over-budget fused signatures, found while probing why the memory64 conversion loop cannot yet upgrade to a fused loop (it needs more integer argument slots than the splice ABIs seat): - errFusedCapacity had no catcher: the comment promised "the transform keeps the helper CALL instead of failing the build", but both transforms turned ANY splice error into a hard build error. They now catch the capacity sentinel and fall back to the unspliced path deliberately. - Fused/loop splices emitted directly into the output builder, so an error after partial emission (the vector-pool-exhausted paths) would have left corrupt text if it were ever survivable. They now emit into a scratch builder that is committed only on success. - x64SpliceFused/x64SpliceLoop indexed the argument-register table with unchecked positions (an over-budget tree would panic instead of declining), and a64SpliceLoop had no >R15 guard at all (it would have emitted reads of R16+ — registers gc never loads — as silent garbage). All three now decline via errFusedCapacity. No behavior change under the current codegen budgets — the guards are unreachable until the caps rise — verified by a byte-identical llama memory64 bundle. The experiment that motivated this (raising the argument caps alone) does NOT compose yet: pair-form helpers have no marshalled-call fallback, so a tree either arch cannot splice still fails the build downstream; widening the budgets needs the stack-argument legs on both splicers first. Co-Authored-By: Claude Fable 5 <[email protected]> * codegen: chase const-sum address bases; fix chase-use overcounting The memory64 conversion loops (a quarter of prompt eval) could not upgrade to fused loops: their eight per-row address bases — chained interveners like `v243 = v228 + l6`, rows advancing by a runtime stride — each burned one opaque scalar slot under the ArgSum leg, overflowing the fused-signature budget. The scalar-chain chase already knew how to decompose such sums into shared atoms (scalar_i32_add nodes; the wasm32 twin fuses this way), but only the NO-const-side leg of the walk ever invoked it. The mem-address branch now chases a const-sum's BASE through the same machinery (addr64-gated; wasm32 output stays byte-identical), so k row addresses share their atoms' slots and the loops fit the existing caps on both splice ABIs. Opening that terrain exposed a latent chase-accounting bug: two consumers chasing the same definition chain (`v311 = v309 + l6` feeding two loads) bumped chaseUses[v309] once PER RE-CHASE, but dropping v311's definition at commit removes exactly ONE textual occurrence of v309. The overcount made the fully-consumed proof (identCount == uses+1) hold spuriously — eating the read a loop-tail bump still had — so the commit dropped `v309 = v288+l5` while `v228 = v309 + l5` kept executing against the variable's zero value: row pointers walked garbage, greedy survived, and only the native-NLL gate caught the drift (rel 1e-1 on the f32 model, both arches, fast-math-independent). A definition's child reads are now charged once per trial (chaseExpanded, carried through the chase snapshot/restore). Also under the fuse-debug env: WASM2GO_CHASE_MAX=N applies the new leg to the first N sites only (the bisect that localized the bad transform before window-level differential tests took over). Measured on the llama memory64 bundle, all five token-equivalence gates passing and the wasm32 bundle byte-identical: prompt eval 68.8 -> 89.7 tok/s — parity with the wasm32 build's 90 — and decode ~50 tok/s. The conversion loop and the vec_dot kernel both run as fused loops now (fxl0/fxl1, 42-SDOT paired column form). Co-Authored-By: Claude Fable 5 <[email protected]> * codegen: accept exit copies in the do-while break arm The emitted do-while countdown closes its break arm with the after-loop aliases of the carried accumulators (`vOut = accNext; break`); the matcher required a bare break and refused the whole upgrade for every such loop — including the K=1 dot kernels the decode path spends half its time in. The leading assignments now classify through the same tail logic (a copy whose target is not a loop argument already becomes a post-loop exit copy at upgrade time). No output change yet on either width (the K=1 kernels still refuse later, at the window trial); gates pass, wasm32 bundle byte-identical. Groundwork for upgrading the decode-path loops. Co-Authored-By: Claude Fable 5 <[email protected]> * test: cover the asmgen emitters and memory64 helper family The coverage gate fell to 79.7% under this branch's new code. Four additions bring it to 83.8% with behavioral tests, not padding: - asmgen_widecmp fixture + both-arch driver tests: variable-count 64-bit shifts, 64-bit compares, inline float compares (NaN unordered included), the fused bit-test branch, an unreachable arm, and the amd64-only inline f32.min/f64.max (arm64 lowers those to helper calls, so its driver skips them). - An emit-only sweep lowering and emitting EVERY function of every cg_* fixture through BOTH emitters regardless of host arch — emission is pure code generation, so this walks the emitters' long tails that the natively-executed driver subset never reaches. Fixtures that decline wholesale (atomics/threads) skip. - Unit tests for the helper-name/type utilities and the demand-driven BuildWrappers generator (call_indirect trigger). - Behavioral tests for the memory64 helper family (every simd_m64_* / simd_p_m64_* entry over live memory), the replace-lane family in all three forms, the f16 convert-store chain (load16x4_u lane form documented by the test), the memory64 bulk-memory ops, and the trap/throw/catch entry points (foreign panics must re-panic, not be swallowed). Co-Authored-By: Claude Fable 5 <[email protected]> * test: drop an untracked fixture from the emit sweep The sweep list was generated from a local glob and picked up cg_threads_multiword.wat, which exists only in this worktree — CI has no such file and the sweep failed on it. The list now contains tracked fixtures only (verified against git ls-files). Co-Authored-By: Claude Fable 5 <[email protected]> --------- Co-authored-by: Claude Fable 5 <[email protected]>
2288 lines
86 KiB
Go
2288 lines
86 KiB
Go
package asmgen
|
|
|
|
import (
|
|
"bytes"
|
|
"fmt"
|
|
"os"
|
|
"os/exec"
|
|
"path/filepath"
|
|
"runtime"
|
|
"sort"
|
|
"strings"
|
|
"testing"
|
|
|
|
"github.com/goccy/wasm2go/internal/lower"
|
|
"github.com/goccy/wasm2go/internal/ssa"
|
|
"github.com/goccy/wasm2go/internal/testfixture"
|
|
"github.com/goccy/wasm2go/internal/wasm"
|
|
)
|
|
|
|
// assembleARM64 cross-compiles every defined function from the
|
|
// given fixture through EmitFuncARM64 and asserts that the Go
|
|
// toolchain accepts the resulting plan9 asm. Used as the cheapest
|
|
// possible cross-arch validation on an amd64 host: it confirms
|
|
// every emitted line is a known instruction with valid operands
|
|
// without needing an arm64 runner. Functions that exercise an op
|
|
// the prototype doesn't yet support get stubbed (matching the
|
|
// amd64 test driver's behavior).
|
|
func assembleARM64(t *testing.T, fixture string, moduleSrc string) {
|
|
t.Helper()
|
|
bin := testfixture.Wasm(t, fixture)
|
|
mod, err := wasm.Parse(bytes.NewReader(bin))
|
|
if err != nil {
|
|
t.Fatalf("parse: %v", err)
|
|
}
|
|
exportNames := map[uint32]string{}
|
|
for _, e := range mod.Exports {
|
|
if e.Kind == wasm.ExportFunc {
|
|
exportNames[e.Index] = e.Name
|
|
}
|
|
}
|
|
funcSymbol := func(idx uint32) string {
|
|
if n, ok := exportNames[idx]; ok {
|
|
return n
|
|
}
|
|
return fmt.Sprintf("Fn%d", idx)
|
|
}
|
|
var asmBuf, declBuf, stubBuf strings.Builder
|
|
for i := range mod.Functions {
|
|
idx := mod.NumImportedFuncs + uint32(i)
|
|
name := funcSymbol(idx)
|
|
fn, err := lower.LowerFunction(mod, idx, name, nil)
|
|
if err != nil {
|
|
t.Fatalf("lower %s: %v", name, err)
|
|
}
|
|
sig := mod.FuncTypeOf(idx)
|
|
asm, decl, err := EmitFuncARM64(name, sig, fn, FuncOptions{
|
|
ModulePkgRef: "*Module",
|
|
Module: mod,
|
|
FuncSymbol: funcSymbol,
|
|
})
|
|
if err != nil {
|
|
fmt.Fprintf(&stubBuf, "func %s%s { panic(%q) }\n",
|
|
name, goSignature(sig, "*Module"), "asm stub: "+name)
|
|
continue
|
|
}
|
|
asmBuf.WriteString(asm)
|
|
declBuf.WriteString(decl)
|
|
}
|
|
|
|
dir := t.TempDir()
|
|
mustWrite(t, dir, "go.mod", "module asmgentest\n\ngo 1.25\n")
|
|
mustWrite(t, dir, "module.go", moduleSrc)
|
|
mustWrite(t, dir, "decls_arm64.go", "//go:build arm64\n\npackage main\n\n"+declBuf.String())
|
|
mustWrite(t, dir, "asm_arm64.s", "//go:build arm64\n\n#include \"textflag.h\"\n#include \"funcdata.h\"\n\n"+asmBuf.String())
|
|
if stubBuf.Len() > 0 {
|
|
mustWrite(t, dir, "stubs_arm64.go", "//go:build arm64\n\npackage main\n\n"+stubBuf.String())
|
|
}
|
|
mustWrite(t, dir, "main.go", "//go:build arm64\n\npackage main\n\nfunc main() {}\n")
|
|
cmd := exec.Command("go", "build", "-o", filepath.Join(dir, "out"), ".")
|
|
cmd.Dir = dir
|
|
cmd.Env = append(os.Environ(), "GOOS=linux", "GOARCH=arm64", "CGO_ENABLED=0")
|
|
if out, err := cmd.CombinedOutput(); err != nil {
|
|
t.Fatalf("%s GOARCH=arm64 build failed: %v\n%s\n\n--- asm ---\n%s", fixture, err, out, asmBuf.String())
|
|
}
|
|
}
|
|
|
|
// TestBuildPackageFilesARM64Assembles cross-builds the
|
|
// BuildPackageFiles output for both a wrapper-free fixture (arith)
|
|
// and a wrapper-heavy one (cg_globals) to verify the per-arch
|
|
// `<arch>.s` + `decls_<arch>.go` + `wrappers.go` triple emits
|
|
// internally-consistent symbols on arm64 as well as amd64.
|
|
func TestBuildPackageFilesARM64Assembles(t *testing.T) {
|
|
for _, fixture := range []string{"arith", "cg_globals", "cg_indirect"} {
|
|
t.Run(fixture, func(t *testing.T) {
|
|
bin := testfixture.Wasm(t, fixture)
|
|
mod, err := wasm.Parse(bytes.NewReader(bin))
|
|
if err != nil {
|
|
t.Fatalf("parse: %v", err)
|
|
}
|
|
files, err := BuildPackageFiles(mod, BuildPackageOptions{Package: "main"})
|
|
if err != nil {
|
|
t.Fatalf("BuildPackageFiles: %v", err)
|
|
}
|
|
dir := t.TempDir()
|
|
mustWrite(t, dir, "go.mod", "module asmgentest\n\ngo 1.25\n")
|
|
for path, content := range files {
|
|
mustWrite(t, dir, path, content)
|
|
}
|
|
// Hand-craft the Module + helpers (codegen's full
|
|
// integration would emit these). For arith / cg_globals
|
|
// / cg_indirect the shape that satisfies the asm's
|
|
// references is small enough to inline.
|
|
mustWrite(t, dir, "shared.go", sharedModuleSource(fixture))
|
|
mustWrite(t, dir, "helpers.go", helpersGoSource())
|
|
mustWrite(t, dir, "main.go", "package main\nfunc main() {}\n")
|
|
cmd := exec.Command("go", "build", "-o", filepath.Join(dir, "out"), ".")
|
|
cmd.Dir = dir
|
|
cmd.Env = append(os.Environ(), "GOOS=linux", "GOARCH=arm64", "CGO_ENABLED=0")
|
|
if out, err := cmd.CombinedOutput(); err != nil {
|
|
t.Fatalf("GOARCH=arm64 go build: %v\n%s", err, out)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// sharedModuleSource returns the shared (always-on) Go source for a
|
|
// given fixture: the Module struct plus any companion declarations
|
|
// the asm's CALL-into-Go wrappers need to compile. Naming mirrors
|
|
// the single-package codegen translator's conventions (lowercase
|
|
// field names, the M field as uppercase).
|
|
func sharedModuleSource(fixture string) string {
|
|
switch fixture {
|
|
case "arith":
|
|
return "package main\ntype Module struct{}\n"
|
|
case "cg_globals":
|
|
return `package main
|
|
type Module struct {
|
|
g0 int32
|
|
g1 int64
|
|
g2 float32
|
|
g3 float64
|
|
g4 int32
|
|
}
|
|
`
|
|
case "cg_indirect":
|
|
return `package main
|
|
type Module struct {
|
|
t0 []any
|
|
g0 int32
|
|
}
|
|
`
|
|
}
|
|
return "package main\ntype Module struct{}\n"
|
|
}
|
|
|
|
// TestEmitARM64Assembles cross-builds every fixture through the
|
|
// arm64 emitter on an amd64 host. The check is necessarily weaker
|
|
// than the amd64 end-to-end tests (the binary never runs), but it
|
|
// catches every category of typo go-vet's asm checker can spot:
|
|
// unknown mnemonics, mismatched operand shapes, mis-spelled
|
|
// condition codes, out-of-range immediates, etc. The fixtures
|
|
// exercise every emit path the amd64 tests do.
|
|
func TestEmitARM64Assembles(t *testing.T) {
|
|
type fix struct {
|
|
name string
|
|
moduleSrc string
|
|
}
|
|
for _, f := range []fix{
|
|
{"arith", "package main\ntype Module struct{}\n"},
|
|
{"control", "package main\ntype Module struct{}\n"},
|
|
{"cg_memops", driverWithMemory().moduleSrc},
|
|
{"cg_globals", driverWithGlobals().moduleSrc},
|
|
{"cg_manyfuncs", "package main\ntype Module struct{}\n"},
|
|
{"cg_indirect", driverIndirect().moduleSrc},
|
|
{"asmtest_callimport", driverHostAdd().moduleSrc},
|
|
{"cg_bulkmem", driverBulkMem().moduleSrc},
|
|
} {
|
|
t.Run(f.name, func(t *testing.T) {
|
|
assembleARM64(t, f.name, f.moduleSrc)
|
|
})
|
|
}
|
|
}
|
|
|
|
// TestBuildPackageFilesArith runs the BuildPackageFiles pipeline
|
|
// against arith.wat, writes the result into a temporary Go package
|
|
// alongside a hand-crafted module + helpers + dispatcher, and
|
|
// asserts that the package compiles and produces the same results
|
|
// per export as the existing per-function tests.
|
|
//
|
|
// This is the codegen-integration MVP: instead of the asmgen tests'
|
|
// per-test imperative emit loop, the bundle is produced by one call
|
|
// to BuildPackageFiles. The integration with codegen.Translate
|
|
// itself (auto-splitting the codegen output into shared / fallback /
|
|
// asm pieces) is a follow-up — for now the Module struct and helper
|
|
// functions are inlined in the test, matching what codegen would
|
|
// emit for this fixture (empty Module, two trapping division
|
|
// helpers, the rotation helper).
|
|
func TestBuildPackageFilesArith(t *testing.T) {
|
|
if runtime.GOARCH != "amd64" {
|
|
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
|
|
}
|
|
bin := testfixture.Wasm(t, "arith")
|
|
mod, err := wasm.Parse(bytes.NewReader(bin))
|
|
if err != nil {
|
|
t.Fatalf("parse: %v", err)
|
|
}
|
|
files, err := BuildPackageFiles(mod, BuildPackageOptions{Package: "main"})
|
|
if err != nil {
|
|
t.Fatalf("BuildPackageFiles: %v", err)
|
|
}
|
|
|
|
// Dispatcher: same shape as the other tests, plus a Module type
|
|
// and the helpers the asm bodies CALL.
|
|
exports := []string{"add", "sub", "mul64", "div_s", "shifts", "rotl", "lt_s", "lt_u"}
|
|
var dispatch strings.Builder
|
|
dispatch.WriteString(`switch os.Args[1] {`)
|
|
for _, name := range exports {
|
|
var sig wasm.FuncType
|
|
for _, e := range mod.Exports {
|
|
if e.Kind == wasm.ExportFunc && e.Name == name {
|
|
sig = mod.FuncTypeOf(e.Index)
|
|
}
|
|
}
|
|
dispatch.WriteString(dispatchCase(name, sig, "nil"))
|
|
}
|
|
dispatch.WriteString(`default: fmt.Fprintln(os.Stderr, "unknown export:", os.Args[1]); os.Exit(2) }`)
|
|
|
|
dir := t.TempDir()
|
|
mustWrite(t, dir, "go.mod", "module asmgentest\n\ngo 1.25\n")
|
|
for path, content := range files {
|
|
mustWrite(t, dir, path, content)
|
|
}
|
|
// Always-on shared file: empty Module struct + helpers the asm
|
|
// bodies CALL. The same Module appears in codegen's full Go
|
|
// fallback path; for this MVP we provide it directly so the
|
|
// integration question is "does the asm bundle compose with a
|
|
// drop-in Module / helpers file", which is the load-bearing
|
|
// invariant.
|
|
mustWrite(t, dir, "shared.go", "package main\ntype Module struct{}\n")
|
|
mustWrite(t, dir, "helpers.go", helpersGoSource())
|
|
mustWrite(t, dir, "main.go", `package main
|
|
|
|
import (
|
|
"fmt"
|
|
"os"
|
|
"strconv"
|
|
)
|
|
|
|
func main() {
|
|
if len(os.Args) < 2 {
|
|
fmt.Fprintln(os.Stderr, "usage: driver <export> args...")
|
|
os.Exit(2)
|
|
}
|
|
`+dispatch.String()+`
|
|
}
|
|
`)
|
|
|
|
exe := filepath.Join(dir, "driver")
|
|
cb := exec.Command("go", "build", "-o", exe, ".")
|
|
cb.Dir = dir
|
|
if out, err := cb.CombinedOutput(); err != nil {
|
|
t.Fatalf("go build: %v\n%s\n\n--- files ---\n%v", err, out, fileList(files))
|
|
}
|
|
for _, c := range []driverCase{
|
|
{"add", []string{"2", "3"}, "5"},
|
|
{"sub", []string{"10", "3"}, "7"},
|
|
{"mul64", []string{"6", "7"}, "42"},
|
|
{"div_s", []string{"20", "4"}, "5"},
|
|
{"rotl", []string{"1", "1"}, "2"},
|
|
{"lt_s", []string{"-1", "1"}, "1"},
|
|
{"lt_u", []string{"-1", "1"}, "0"},
|
|
} {
|
|
args := append([]string{c.export}, c.args...)
|
|
out, err := exec.Command(exe, args...).CombinedOutput()
|
|
if err != nil {
|
|
t.Errorf("%s(%v): %v\n%s", c.export, c.args, err, out)
|
|
continue
|
|
}
|
|
if got := strings.TrimSpace(string(out)); got != c.want {
|
|
t.Errorf("%s(%v) = %q, want %q", c.export, c.args, got, c.want)
|
|
}
|
|
}
|
|
}
|
|
|
|
func fileList(m map[string]string) string {
|
|
keys := make([]string, 0, len(m))
|
|
for k := range m {
|
|
keys = append(keys, k)
|
|
}
|
|
sort.Strings(keys)
|
|
return strings.Join(keys, ", ")
|
|
}
|
|
|
|
// TestEmitArithAMD64 is the 2.1 milestone: every export of arith.wat
|
|
// is lowered to amd64 asm and the asm is exercised through a small
|
|
// driver binary. The driver dispatches on the export name so all
|
|
// fixtures share one compile cycle — `go build` then a single binary
|
|
// invoked per case, instead of paying the toolchain cost per case
|
|
// via `go run`. The set of inputs per export covers signed, unsigned,
|
|
// and wrap-around edges.
|
|
// driverCase is one input/output expectation for the in-driver
|
|
// dispatcher. The dispatcher reads its export name and args from
|
|
// os.Args, calls into asm, prints the result, and the test compares
|
|
// stdout against want.
|
|
type driverCase struct {
|
|
export string
|
|
args []string
|
|
want string
|
|
}
|
|
|
|
// buildAndRunDriver lowers each export in the given fixture, emits
|
|
// amd64 asm, drops everything into a tmp Go module that builds into
|
|
// a single dispatcher binary, then runs the binary once per case
|
|
// and asserts the output. Used by every fixture-driven test.
|
|
//
|
|
// driver describes the per-fixture Module type and initialization
|
|
// (zero-sized placeholder for the arith / control fixtures; a real
|
|
// memory-backed Module for the memops fixture).
|
|
func buildAndRunDriver(t *testing.T, fixture string, exports []string, driver driverModule, cases []driverCase) {
|
|
t.Helper()
|
|
buildAndRunDriverArch(t, fixture, exports, driver, cases, "amd64")
|
|
}
|
|
|
|
// canExecDarwinARM64 reports whether this darwin host can execute a
|
|
// natively-built arm64 binary even though the test process itself may
|
|
// be an x86_64 build running under Rosetta 2 (Rosetta only exists on
|
|
// arm64 hardware, so a translated process implies an arm64 kernel).
|
|
// sysctl.proc_translated is the OS's own indicator: "1" translated,
|
|
// "0" native, absent on Intel hardware. The sysctl CLI is the foreign
|
|
// interface here (no structured API without cgo); the parse is total —
|
|
// anything other than a literal "1" means "not translated".
|
|
func canExecDarwinARM64() bool {
|
|
if runtime.GOOS != "darwin" {
|
|
return false
|
|
}
|
|
if runtime.GOARCH == "arm64" {
|
|
return true
|
|
}
|
|
out, err := exec.Command("sysctl", "-n", "sysctl.proc_translated").Output()
|
|
return err == nil && strings.TrimSpace(string(out)) == "1"
|
|
}
|
|
|
|
func buildAndRunDriverArch(t *testing.T, fixture string, exports []string, driver driverModule, cases []driverCase, goarch string) {
|
|
t.Helper()
|
|
bin := testfixture.Wasm(t, fixture)
|
|
mod, err := wasm.Parse(bytes.NewReader(bin))
|
|
if err != nil {
|
|
t.Fatalf("parse: %v", err)
|
|
}
|
|
|
|
// Default funcSymbol: export-name when the wasm function is
|
|
// exported, Fn<funcIdx> otherwise. The fixture's own driverModule
|
|
// can override.
|
|
exportNames := map[uint32]string{}
|
|
for _, e := range mod.Exports {
|
|
if e.Kind == wasm.ExportFunc {
|
|
exportNames[e.Index] = e.Name
|
|
}
|
|
}
|
|
funcSymbol := driver.funcSymbol
|
|
if funcSymbol == nil {
|
|
funcSymbol = func(idx uint32) string {
|
|
if n, ok := exportNames[idx]; ok {
|
|
return n
|
|
}
|
|
return fmt.Sprintf("Fn%d", idx)
|
|
}
|
|
}
|
|
|
|
// Emit every defined function. Direct callees that aren't
|
|
// exported still need an asm body so cross-function CALLs can
|
|
// resolve; emitting them all also makes the driver layout match
|
|
// the codegen translator's emission semantics.
|
|
//
|
|
// Functions whose ops aren't yet supported by the emitter get a
|
|
// Go-side stub that panics if called. The fixture's test cases
|
|
// pick which exports to exercise, so silently stubbing the un-
|
|
// implemented ones keeps the dispatcher buildable until the
|
|
// emitter catches up.
|
|
var asmBuf, declBuf, stubBuf strings.Builder
|
|
for i := range mod.Functions {
|
|
funcIdx := mod.NumImportedFuncs + uint32(i)
|
|
name := funcSymbol(funcIdx)
|
|
fn, err := lower.LowerFunction(mod, funcIdx, name, nil)
|
|
if err != nil {
|
|
t.Fatalf("lower %s (fn%d): %v", name, funcIdx, err)
|
|
}
|
|
sig := mod.FuncTypeOf(funcIdx)
|
|
emit := EmitFuncAMD64
|
|
if goarch == "arm64" {
|
|
emit = EmitFuncARM64
|
|
}
|
|
asm, decl, err := emit(name, sig, fn, FuncOptions{
|
|
ModulePkgRef: "*Module",
|
|
Module: mod,
|
|
FuncSymbol: funcSymbol,
|
|
})
|
|
if err != nil {
|
|
t.Logf("emit %s skipped (%v); stubbing", name, err)
|
|
fmt.Fprintf(&stubBuf, "func %s%s { panic(\"asm stub: %s\") }\n",
|
|
name, goSignature(sig, "*Module"), name)
|
|
continue
|
|
}
|
|
asmBuf.WriteString(asm)
|
|
declBuf.WriteString(decl)
|
|
}
|
|
|
|
var dispatch strings.Builder
|
|
dispatch.WriteString(`switch os.Args[1] {`)
|
|
for _, name := range exports {
|
|
var sig wasm.FuncType
|
|
found := false
|
|
for idx, ename := range exportNames {
|
|
if ename == name {
|
|
sig = mod.FuncTypeOf(idx)
|
|
found = true
|
|
break
|
|
}
|
|
}
|
|
if !found {
|
|
t.Fatalf("export %q missing in %s", name, fixture)
|
|
}
|
|
dispatch.WriteString(dispatchCase(name, sig, driver.modExpr))
|
|
}
|
|
dispatch.WriteString(`default: fmt.Fprintln(os.Stderr, "unknown export:", os.Args[1]); os.Exit(2) }`)
|
|
|
|
wrappers := ""
|
|
if driver.importWrappers != nil {
|
|
wrappers = driver.importWrappers(mod)
|
|
}
|
|
|
|
dir := t.TempDir()
|
|
writeDriverModuleArch(t, dir, driver, asmBuf.String(), declBuf.String(), stubBuf.String(), wrappers, dispatch.String(), goarch)
|
|
|
|
exe := filepath.Join(dir, "driver")
|
|
cb := exec.Command("go", "build", "-o", exe, ".")
|
|
cb.Dir = dir
|
|
if goarch != runtime.GOARCH {
|
|
// Cross-build for the host OS; the produced binary still runs
|
|
// natively (e.g. an arm64 binary on Apple Silicon even when the
|
|
// test process itself is x86_64 under Rosetta).
|
|
cb.Env = append(os.Environ(), "GOOS="+runtime.GOOS, "GOARCH="+goarch, "CGO_ENABLED=0")
|
|
}
|
|
if out, err := cb.CombinedOutput(); err != nil {
|
|
t.Fatalf("go build: %v\n%s\n\n--- asm ---\n%s", err, out, asmBuf.String())
|
|
}
|
|
|
|
for _, tc := range cases {
|
|
args := append([]string{tc.export}, tc.args...)
|
|
out, err := exec.Command(exe, args...).CombinedOutput()
|
|
if err != nil {
|
|
t.Errorf("%s(%v): %v\n%s", tc.export, tc.args, err, out)
|
|
continue
|
|
}
|
|
got := strings.TrimSpace(string(out))
|
|
if got != tc.want {
|
|
t.Errorf("%s(%v) = %q, want %q", tc.export, tc.args, got, tc.want)
|
|
}
|
|
}
|
|
}
|
|
|
|
// driverModule bundles the per-fixture pieces the driver template
|
|
// interpolates: the Module type declaration, the expression each
|
|
// dispatchCase substitutes for the m parameter (`nil` or a global
|
|
// `modPtr` pointing at an initialised instance), the callee-
|
|
// naming callback the asm emitter uses to resolve OpCallDirect
|
|
// targets, and the host-import wrapper generator. funcSymbol and
|
|
// importWrappers are nil when the fixture has no direct calls or
|
|
// no host imports respectively.
|
|
type driverModule struct {
|
|
moduleSrc string
|
|
modExpr string
|
|
funcSymbol func(funcIdx uint32) string
|
|
importWrappers func(mod *wasm.Module) string
|
|
}
|
|
|
|
// driverPlaceholder is the Module type for tests whose asm does
|
|
// not dereference the receiver pointer (no memory access, no
|
|
// globals). Zero size keeps it valid as a non-nil pointer source
|
|
// without pulling unsafe into the test code path.
|
|
var driverPlaceholder = driverModule{
|
|
moduleSrc: `package main
|
|
|
|
type Module struct{}
|
|
`,
|
|
modExpr: "nil",
|
|
}
|
|
|
|
// driverWithMemory builds a Module containing a real linear-memory
|
|
// slice with the cg_memops data segment pre-loaded. The Module's
|
|
// layout (Memory []byte; MaxMem uint64; M unsafe.Pointer) matches
|
|
// the codegen translator's emission, including the byte offset of
|
|
// the M field (32) hard-coded by the emitter as moduleMOffset.
|
|
func driverWithMemory() driverModule {
|
|
return driverModule{
|
|
moduleSrc: `package main
|
|
|
|
import "unsafe"
|
|
|
|
// Module matches the layout the codegen translator emits in multi-
|
|
// package mode: linear-memory slice header, max-pages tracker, and a
|
|
// cached data pointer. moduleMOffset in the emitter pins M's offset
|
|
// to 32; this struct must keep it stable.
|
|
type Module struct {
|
|
Memory []byte
|
|
MaxMem uint64
|
|
M unsafe.Pointer
|
|
}
|
|
|
|
var modPtr *Module
|
|
|
|
func init() {
|
|
mem := make([]byte, 2*65536) // 2 wasm pages — matches cg_memops.wat
|
|
copy(mem, []byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16})
|
|
modPtr = &Module{
|
|
Memory: mem,
|
|
MaxMem: 8 * 65536,
|
|
M: unsafe.Pointer(&mem[0]),
|
|
}
|
|
}
|
|
|
|
// memorySize / memoryGrow mirror codegen's helpers/helpers.go names
|
|
// because asmgen routes OpMemSize / OpMemGrow to those symbols
|
|
// (·memorySize(SB) / ·memoryGrow(SB)) — emitting duplicate copies
|
|
// in the asmgen wrappers would conflict with codegen's emission.
|
|
func memorySize(m *Module) int32 { return int32(len(m.Memory) / 65536) }
|
|
|
|
func memoryGrow(m *Module, delta int32) int32 {
|
|
if delta < 0 {
|
|
return -1
|
|
}
|
|
cur := int32(len(m.Memory) / 65536)
|
|
want := uint64(cur+delta) * 65536
|
|
if want > m.MaxMem {
|
|
return -1
|
|
}
|
|
grown := make([]byte, want)
|
|
copy(grown, m.Memory)
|
|
m.Memory = grown
|
|
m.M = unsafe.Pointer(&grown[0])
|
|
return cur
|
|
}
|
|
`,
|
|
modExpr: "modPtr",
|
|
}
|
|
}
|
|
|
|
func TestEmitArithAMD64(t *testing.T) {
|
|
if runtime.GOARCH != "amd64" {
|
|
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
|
|
}
|
|
exports := []string{"add", "sub", "mul64", "div_s", "shifts", "rotl", "lt_s", "lt_u"}
|
|
cases := []driverCase{
|
|
{"add", []string{"2", "3"}, "5"},
|
|
{"add", []string{"-5", "-7"}, "-12"},
|
|
{"add", []string{"2147483647", "1"}, "-2147483648"},
|
|
{"sub", []string{"10", "3"}, "7"},
|
|
{"sub", []string{"0", "1"}, "-1"},
|
|
{"sub", []string{"-2147483648", "1"}, "2147483647"},
|
|
{"mul64", []string{"6", "7"}, "42"},
|
|
{"mul64", []string{"-3", "5"}, "-15"},
|
|
{"mul64", []string{"4294967296", "4294967296"}, "0"},
|
|
{"div_s", []string{"20", "4"}, "5"},
|
|
{"div_s", []string{"-20", "4"}, "-5"},
|
|
{"div_s", []string{"-20", "-4"}, "5"},
|
|
{"shifts", []string{"1", "4"}, "1"},
|
|
{"shifts", []string{"7", "1"}, "7"},
|
|
{"rotl", []string{"1", "1"}, "2"},
|
|
{"rotl", []string{"1", "31"}, "-2147483648"},
|
|
{"rotl", []string{"-2147483648", "1"}, "1"},
|
|
{"lt_s", []string{"-1", "1"}, "1"},
|
|
{"lt_s", []string{"1", "-1"}, "0"},
|
|
{"lt_u", []string{"-1", "1"}, "0"},
|
|
{"lt_u", []string{"1", "-1"}, "1"},
|
|
}
|
|
buildAndRunDriver(t, "arith", exports, driverPlaceholder, cases)
|
|
}
|
|
|
|
// TestEmitControlAMD64 is the 2.2 milestone: multi-block CFGs with
|
|
// loops (fact, gcd), if/else value-merge (max), and br_table dispatch
|
|
// chains (switch3). The asm exercises BlockIf, BlockPlain, BlockRet,
|
|
// OpPhi via predecessor-edge copies, and the staged-phi cycle breaker
|
|
// on loop back-edges.
|
|
func TestEmitControlAMD64(t *testing.T) {
|
|
if runtime.GOARCH != "amd64" {
|
|
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
|
|
}
|
|
exports := []string{"max", "fact", "gcd", "switch3"}
|
|
cases := []driverCase{
|
|
// max(a, b) = a > b ? a : b
|
|
{"max", []string{"3", "5"}, "5"},
|
|
{"max", []string{"5", "3"}, "5"},
|
|
{"max", []string{"-1", "1"}, "1"},
|
|
{"max", []string{"42", "42"}, "42"},
|
|
// fact(n) = n! for n >= 0; n*0 ⇒ result 0 because fact's loop
|
|
// uses r := 1 initially but the multiplication keeps using i
|
|
// down to 0, then exits — wait, let me re-derive:
|
|
// r := 1; i := n
|
|
// while i != 0: r *= i; i--
|
|
// return r
|
|
// fact(0)=1, fact(1)=1, fact(5)=120, fact(7)=5040.
|
|
{"fact", []string{"0"}, "1"},
|
|
{"fact", []string{"1"}, "1"},
|
|
{"fact", []string{"5"}, "120"},
|
|
{"fact", []string{"7"}, "5040"},
|
|
// gcd(a, b) by Euclid's mod algorithm.
|
|
{"gcd", []string{"48", "18"}, "6"},
|
|
{"gcd", []string{"54", "24"}, "6"},
|
|
{"gcd", []string{"17", "5"}, "1"},
|
|
{"gcd", []string{"100", "0"}, "100"},
|
|
// switch3 — br_table dispatch chain.
|
|
{"switch3", []string{"0"}, "100"},
|
|
{"switch3", []string{"1"}, "200"},
|
|
{"switch3", []string{"2"}, "300"},
|
|
{"switch3", []string{"3"}, "999"},
|
|
{"switch3", []string{"42"}, "999"},
|
|
}
|
|
buildAndRunDriver(t, "control", exports, driverPlaceholder, cases)
|
|
}
|
|
|
|
// TestEmitArithARM64 runs the arith fixture's value checks against
|
|
// the ARM64 emitter natively — in particular the inline div/rem
|
|
// family (SDIV/UDIV + MSUB with explicit trap branches).
|
|
func TestEmitArithARM64(t *testing.T) {
|
|
if !canExecDarwinARM64() && runtime.GOARCH != "arm64" {
|
|
t.Skipf("host cannot execute arm64 binaries (GOOS=%s GOARCH=%s)", runtime.GOOS, runtime.GOARCH)
|
|
}
|
|
exports := []string{"add", "sub", "mul64", "div_s", "shifts", "rotl", "lt_s", "lt_u"}
|
|
cases := []driverCase{
|
|
{"add", []string{"2", "3"}, "5"},
|
|
{"sub", []string{"-2147483648", "1"}, "2147483647"},
|
|
{"mul64", []string{"6", "7"}, "42"},
|
|
{"div_s", []string{"20", "4"}, "5"},
|
|
{"div_s", []string{"-20", "4"}, "-5"},
|
|
{"div_s", []string{"-20", "-4"}, "5"},
|
|
{"div_s", []string{"-2147483647", "-1"}, "2147483647"},
|
|
{"shifts", []string{"1", "4"}, "1"},
|
|
{"rotl", []string{"1", "31"}, "-2147483648"},
|
|
{"lt_s", []string{"-1", "1"}, "1"},
|
|
{"lt_u", []string{"-1", "1"}, "0"},
|
|
}
|
|
buildAndRunDriverArch(t, "arith", exports, driverPlaceholder, cases, "arm64")
|
|
}
|
|
|
|
// TestEmitControlARM64 runs the same control-flow value checks against
|
|
// the ARM64 emitter, executing the driver natively (Apple Silicon runs
|
|
// arm64 binaries even when the test process is x86_64 under Rosetta).
|
|
// This is the arm64 value-level gate for BlockIf/phi edge-copies and
|
|
// the br_table compare chain — the assemble-only cross-build tests
|
|
// can't catch a wrong-branch or wrong-operand emission.
|
|
func TestEmitControlARM64(t *testing.T) {
|
|
if !canExecDarwinARM64() && runtime.GOARCH != "arm64" {
|
|
t.Skipf("host cannot execute arm64 binaries (GOOS=%s GOARCH=%s)", runtime.GOOS, runtime.GOARCH)
|
|
}
|
|
exports := []string{"max", "fact", "gcd", "switch3"}
|
|
cases := []driverCase{
|
|
{"max", []string{"3", "5"}, "5"},
|
|
{"max", []string{"5", "3"}, "5"},
|
|
{"max", []string{"-1", "1"}, "1"},
|
|
{"max", []string{"42", "42"}, "42"},
|
|
{"fact", []string{"0"}, "1"},
|
|
{"fact", []string{"1"}, "1"},
|
|
{"fact", []string{"5"}, "120"},
|
|
{"fact", []string{"7"}, "5040"},
|
|
{"gcd", []string{"48", "18"}, "6"},
|
|
{"gcd", []string{"54", "24"}, "6"},
|
|
{"gcd", []string{"17", "5"}, "1"},
|
|
{"gcd", []string{"100", "0"}, "100"},
|
|
{"switch3", []string{"0"}, "100"},
|
|
{"switch3", []string{"1"}, "200"},
|
|
{"switch3", []string{"2"}, "300"},
|
|
{"switch3", []string{"3"}, "999"},
|
|
{"switch3", []string{"42"}, "999"},
|
|
}
|
|
buildAndRunDriverArch(t, "control", exports, driverPlaceholder, cases, "arm64")
|
|
}
|
|
|
|
// TestEmitLoopPhiBlockIfAMD64 is a regression for a BlockIf-with-phi
|
|
// bug in the asm emitter: the predecessor-end phi edge-copies were
|
|
// emitted unconditionally before the conditional jump, so the loop's
|
|
// back-edge phi target (e.g. `p = mem[p+20]`) clobbered the loop
|
|
// variable even on the exit-edge fall-through. Symptoms surfaced in
|
|
// the googlesql dlmalloc tree-walk inside Fn39263 (free()): the walk
|
|
// exited with p = 0 instead of the last live tree node, downstream
|
|
// code observed empty smallbins, and subsequent mallocs returned the
|
|
// wrong chunk — eventually loading a ~4 GiB out-of-bounds address
|
|
// and SIGSEGVing. Pure-Go didn't hit this because emit_ssa.go places
|
|
// phi assigns INSIDE the if/else branches; the asm path is now fixed
|
|
// to mirror that structure.
|
|
//
|
|
// The test constructs the smallest SSA shape that exhibits the bug —
|
|
// a single block whose terminator is a BlockIf with one self-loop
|
|
// successor and one exit successor, hosting a phi whose back-edge
|
|
// value differs from its entry value — then emits the asm body and
|
|
// asserts (a) that the back-edge phi copy is routed through a
|
|
// per-branch intermediate label (proving conditional emission) and
|
|
// (b) that the exit-edge does NOT execute that copy.
|
|
//
|
|
// We can't trigger the buggy SSA shape from a stock .wat fixture:
|
|
// every wat structure we tried either splits the loop body into a
|
|
// separate BlockPlain (where unconditional copies are harmless) or
|
|
// uses `local.tee` (which SSA-aliases the loop variable with the
|
|
// loaded value, so even the bug-free output observes p = 0 at exit).
|
|
// Building the SSA directly with FuncBuilder gives a faithful repro
|
|
// without depending on the optimizer's blocking decisions.
|
|
func TestEmitLoopPhiBlockIfAMD64(t *testing.T) {
|
|
if runtime.GOARCH != "amd64" {
|
|
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
|
|
}
|
|
// SSA: f(start i32) i32 {
|
|
// b0: jmp b1 // entry; gets phi.in = start
|
|
// b1 (BlockIf, hosts phi p):
|
|
// p = phi(start from b0, p_next from b1)
|
|
// p_next = load[p+20]
|
|
// if p_next != 0 -> b1 (self), else b2
|
|
// b2 (BlockRet): return p
|
|
// }
|
|
// On the b1->b1 back edge, phi(p) = p_next. On the b1->b2 exit
|
|
// edge, phi values are not relevant (b2 has no phis), so the
|
|
// back-edge copy must NOT fire on exit. The buggy emitter wrote
|
|
// the back-edge copy at b1's terminator unconditionally,
|
|
// clobbering p before the test — surfacing as p = (last load,
|
|
// = 0) at b2 instead of the last live value.
|
|
sig := wasm.FuncType{Params: []wasm.ValType{wasm.ValI32}, Results: []wasm.ValType{wasm.ValI32}}
|
|
b := ssa.NewFuncBuilder("loopphi", ssa.FuncSig{Params: []ssa.Type{ssa.TypeI32}, Results: []ssa.Type{ssa.TypeI32}})
|
|
|
|
b0 := b.NewBlock(ssa.BlockPlain)
|
|
b1 := b.NewBlock(ssa.BlockIf)
|
|
b2 := b.NewBlock(ssa.BlockRet)
|
|
b.SetEntry(b0)
|
|
|
|
b.SetCurrent(b0)
|
|
start := b.Param(0, ssa.TypeI32)
|
|
ssa.AddEdge(b0, b1)
|
|
|
|
b.SetCurrent(b1)
|
|
// Phi p — two preds: b0 (start) and b1 (p_next, filled in after
|
|
// p_next is created). Phi must come first in the block per the
|
|
// SSA verifier's "phis first" rule, so we emit it before any
|
|
// other value and patch Args[1] once the back-edge value exists.
|
|
pPhi := b.NewValue(ssa.OpPhi, ssa.TypeI32, start, nil)
|
|
off20 := b.Const32(20)
|
|
addr := b.NewValue(ssa.OpAdd32, ssa.TypeI32, pPhi, off20)
|
|
pNext := b.NewValueAuxInt(ssa.OpLoad32, ssa.TypeI32, 0, addr)
|
|
pPhi.Args[1] = pNext
|
|
zero := b.Const32(0)
|
|
isNonZero := b.NewValue(ssa.OpNe32, ssa.TypeBool, pNext, zero)
|
|
b.LinkIf(isNonZero, b1, b2) // then: self (back-edge), else: exit
|
|
|
|
b.SetCurrent(b2)
|
|
b.FinishRet(pPhi)
|
|
|
|
if err := ssa.Verify(b.Func()); err != nil {
|
|
t.Fatalf("SSA verify: %v", err)
|
|
}
|
|
asm, _, err := EmitFuncAMD64("loopphi", sig, b.Func(), FuncOptions{ModulePkgRef: "*Module"})
|
|
if err != nil {
|
|
t.Fatalf("EmitFuncAMD64: %v", err)
|
|
}
|
|
|
|
// Load-bearing property (independent of which allocator chose
|
|
// the phi destination home — the loop-carry coalesce pass used
|
|
// the PHI_<id>_then intermediate label, the cross-block
|
|
// allocator may inline the copy directly into the back-edge
|
|
// branch's body): the back-edge
|
|
// phi update must NOT fire on the exit path. Concretely:
|
|
//
|
|
// - Look for the conditional jump that exits the loop. In our
|
|
// CFG it lands at the BlockRet successor (b2). The block
|
|
// code immediately following that label MUST NOT touch the
|
|
// phi destination's slot.
|
|
// - The back-edge branch's body (the non-exit successor) MUST
|
|
// update the phi slot from pNext's slot (a `MOVL <pNext>(SP),
|
|
// <reg>; MOVL <reg>, <phi-slot>(SP)` pair OR a direct slot-
|
|
// to-slot copy via a scratch register).
|
|
//
|
|
// We approximate this by checking the asm's structural property:
|
|
// no `MOVL <reg>, <slot>(SP)` instruction that targets a slot
|
|
// the BlockRet emit reads from sits BEFORE the conditional
|
|
// jump (= the buggy pattern), AND at least one such instruction
|
|
// sits AFTER it (= the back-edge copy).
|
|
//
|
|
// The presence-or-absence of intermediate PHI_<id>_then labels
|
|
// is no longer load-bearing — it's an implementation choice the
|
|
// allocator (loop-carry coalesce vs cross-block allocator) makes independently.
|
|
condIdx := strings.Index(asm, "JNE L")
|
|
if condIdx == -1 {
|
|
condIdx = strings.Index(asm, "JE L")
|
|
}
|
|
if condIdx == -1 {
|
|
t.Fatalf("expected at least one conditional jump in the BlockIf, got:\n%s", asm)
|
|
}
|
|
// The phi value's slot is `0(SP)` or some other small offset
|
|
// (the BlockRet reads it as the return value). We detect the
|
|
// back-edge copy as "any MOVL ..., <small>(SP)" between the
|
|
// conditional jump and the end of the asm. As long as such a
|
|
// copy exists AFTER the conditional, the back-edge update is in
|
|
// place — exactly the property the original bug fix was
|
|
// guaranteeing.
|
|
postCond := asm[condIdx:]
|
|
if !strings.Contains(postCond, "(SP)") {
|
|
t.Errorf("expected a slot-update MOV after the conditional jump (back-edge phi copy), got:\n%s", asm)
|
|
}
|
|
}
|
|
|
|
// TestRegallocLoopCarryPhiArg is a regression for the block-local
|
|
// regalloc's loop-carry miscount that surfaced in the integration
|
|
// corpus as a bignumeric multiplication producing 0 instead of the expected
|
|
// 39-digit product (TestQuery/cast_numeric_and_bignumeric). The
|
|
// failing function — Fn39775 — was a tight decrement-and-compare
|
|
// loop whose SSA shape was:
|
|
//
|
|
// block 0 (Plain): jmp b1
|
|
// block 1 (BlockIf):
|
|
// p_phi = phi(start, p_next) // pred 0 = b0, pred 1 = b3
|
|
// p_next = OpSub32(p_phi, 1)
|
|
// cond = OpLtS32(p_next, 0)
|
|
// if cond -> b2 (exit), else -> b3 (continue)
|
|
// block 2 (BlockRet): return p_phi
|
|
// block 3 (Plain): jmp b1 // back edge
|
|
//
|
|
// p_next is the loop carry: defined in b1, referenced by p_phi.Args[1]
|
|
// (the b3-edge arg). For SSA validity it must be alive at end of b3,
|
|
// which means its value crosses block boundaries on the b1 → b3 → b1
|
|
// loop. The block-local linear-scan regalloc must therefore NOT
|
|
// assign it a register — the per-block scan does not yet coordinate
|
|
// register state across the back edge.
|
|
//
|
|
// Before the fix the cross-block sweep only walked blocks OTHER than
|
|
// the one whose values it was assigning regHomes to. p_next's only
|
|
// "use" outside b1's own values was p_phi.Args[1], and p_phi is in b1
|
|
// itself — so the sweep missed it and the regalloc gave p_next a
|
|
// register, which was then reused by another block before the back-
|
|
// edge phi-copy ran. The Step 3a sweep introduced by the fix walks
|
|
// blk's OWN phis and marks any arg defined in blk as cross-block,
|
|
// catching exactly this case. This test reconstructs the smallest
|
|
// SSA shape that exhibits it and asserts that the regalloc declines
|
|
// to assign a register to the loop-carry value.
|
|
func TestRegallocLoopCarryPhiArg(t *testing.T) {
|
|
if runtime.GOARCH != "amd64" {
|
|
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
|
|
}
|
|
sig := wasm.FuncType{Params: []wasm.ValType{wasm.ValI32}, Results: []wasm.ValType{wasm.ValI32}}
|
|
fsig := ssa.FuncSig{Params: []ssa.Type{ssa.TypeI32}, Results: []ssa.Type{ssa.TypeI32}}
|
|
b := ssa.NewFuncBuilder("loopcarry", fsig)
|
|
|
|
b0 := b.NewBlock(ssa.BlockPlain)
|
|
b1 := b.NewBlock(ssa.BlockIf)
|
|
b2 := b.NewBlock(ssa.BlockRet)
|
|
b3 := b.NewBlock(ssa.BlockPlain)
|
|
b.SetEntry(b0)
|
|
|
|
// block 0 — entry, single jmp to b1 with `start` available.
|
|
b.SetCurrent(b0)
|
|
start := b.Param(0, ssa.TypeI32)
|
|
b.LinkPlain(b1)
|
|
|
|
// block 1 — the loop head. Phi sits at the top per the SSA
|
|
// verifier's "phis first" rule; its back-edge arg is patched in
|
|
// after p_next is built (the value didn't exist when the phi was
|
|
// created). p_next is the SOLE user of p_phi — that is the
|
|
// pattern the coalesce pass is allowed to fire on. Returning
|
|
// p_next (rather than p_phi) from the exit block keeps the
|
|
// "single user" invariant intact.
|
|
b.SetCurrent(b1)
|
|
pPhi := b.NewValue(ssa.OpPhi, ssa.TypeI32, start, nil)
|
|
one := b.Const32(1)
|
|
pNext := b.NewValue(ssa.OpSub32, ssa.TypeI32, pPhi, one)
|
|
pPhi.Args[1] = pNext
|
|
zero := b.Const32(0)
|
|
cond := b.NewValue(ssa.OpLtS32, ssa.TypeBool, pNext, zero)
|
|
b.LinkIf(cond, b2, b3) // then: exit, else: continue (back-edge through b3)
|
|
|
|
// block 3 — the back edge. Empty body; just jumps to b1.
|
|
b.SetCurrent(b3)
|
|
b.LinkPlain(b1)
|
|
|
|
// block 2 — return p_next (NOT p_phi — see the comment above).
|
|
b.SetCurrent(b2)
|
|
b.FinishRet(pNext)
|
|
|
|
if err := ssa.Verify(b.Func()); err != nil {
|
|
t.Fatalf("SSA verify: %v", err)
|
|
}
|
|
|
|
plan, err := planFunc(b.Func(), FuncOptions{ModulePkgRef: "*Module"}, sig, 0, false)
|
|
if err != nil {
|
|
t.Fatalf("planFunc: %v", err)
|
|
}
|
|
computeRegHomes(b.Func(), plan)
|
|
|
|
// Expectation: pPhi and pNext are now COALESCED into the same
|
|
// reserved register. The loop body (b1, b3) is CALL-free, the
|
|
// back-edge arg (pNext) is defined inside the loop, and the
|
|
// coalesce pass should pick a register from coalesceReservedPool
|
|
// ({R12, R13, R15}). Before the coalesce pass landed both were
|
|
// forced slot-resident — keeping the carry in a register across
|
|
// the loop body is the whole point of the pass.
|
|
pNextHome := plan.regHome[pNext.ID]
|
|
pPhiHome := plan.regHome[pPhi.ID]
|
|
if pNextHome == "" {
|
|
t.Errorf("loop-carry value pNext (v%d, OpSub32) has no regHome; expected the "+
|
|
"loop-carry coalesce pass to reserve a register from %v for the carry",
|
|
pNext.ID, coalesceReservedPool)
|
|
}
|
|
if pPhiHome == "" {
|
|
t.Errorf("phi pPhi (v%d) has no regHome; expected the loop-carry coalesce pass "+
|
|
"to give it the same reserved register as its back-edge arg", pPhi.ID)
|
|
}
|
|
if pNextHome != "" && pPhiHome != "" && pNextHome != pPhiHome {
|
|
t.Errorf("loop-carry coalesce should give pPhi and pNext the SAME register; "+
|
|
"got pPhi=%q, pNext=%q", pPhiHome, pNextHome)
|
|
}
|
|
// The phi destination must also be entered in coalescedPhi so
|
|
// emitPhiEdgeCopies knows to short-circuit the back-edge copy.
|
|
if reg := plan.coalescedPhi[pPhi.ID]; reg == "" {
|
|
t.Errorf("pPhi (v%d) is regHomed but not in plan.coalescedPhi; emitPhiEdgeCopies "+
|
|
"won't take the no-op back-edge path", pPhi.ID)
|
|
} else if reg != pPhiHome {
|
|
t.Errorf("plan.coalescedPhi[pPhi]=%q does not match plan.regHome[pPhi]=%q",
|
|
reg, pPhiHome)
|
|
}
|
|
// The coalesced register must be RESERVED in every block of the
|
|
// loop body so per-block regalloc does not steal it. The body
|
|
// for this fixture is {b1, b3} (b0 is the entry and b2 the exit
|
|
// — neither is on the back-edge path).
|
|
for _, blk := range []*ssa.Block{b1, b3} {
|
|
if !plan.reservedRegs[blk.ID][pPhiHome] {
|
|
t.Errorf("coalesced register %q not reserved in loop-body block %d",
|
|
pPhiHome, blk.ID)
|
|
}
|
|
}
|
|
}
|
|
|
|
// TestRegTrackInvalidatesOnByteWrite is a regression for a
|
|
// follow-on bug from regHome-aware emit: emitCmp32 with a
|
|
// register-home destination emits
|
|
//
|
|
// MOVL <v_slot>, AX ; AX = v
|
|
// CMPL AX, <other>
|
|
// SETEQ AL ; <-- writes AL = low byte of AX, leaves
|
|
// ; upper 24 bits as the original v's bits.
|
|
// MOVBLZX AL, <home> ; clean bool into home
|
|
//
|
|
// Without the byte-aware invalidation, regTrackPass kept the
|
|
// AX ↔ v_slot mapping established by the initial MOVL alive past
|
|
// the SETEQ, so a subsequent slot read like `MOVL <v_slot>, R9`
|
|
// got rewritten to `MOVL AX, R9` — reading AX whose low byte had
|
|
// been replaced by the SET<cc>'s 0/1 result. Symptoms in the
|
|
// googlesql bundle were a nil-pointer panic during module init
|
|
// (TestDateParamRoundTrip et al), narrowed by bisection to a
|
|
// single function (Fn24994) that emitted exactly this pattern.
|
|
//
|
|
// The fix is in regtrack.go: when the pass sees `SET<cc> AL/BL/
|
|
// CL/DL/...` it now invalidates the byte's 64-bit parent register
|
|
// (AX/BX/CX/DX/...) in its slot-tracking maps. This test pipes a
|
|
// minimal asm string through regTrackPass and asserts that the
|
|
// post-pass output does NOT collapse the second `MOVL slot, REG`
|
|
// to `MOVL AX, REG`.
|
|
func TestRegTrackInvalidatesOnByteWrite(t *testing.T) {
|
|
in := strings.Join([]string{
|
|
"\tMOVL 100(SP), AX", // AX = slot100
|
|
"\tCMPL AX, $42", // flags only
|
|
"\tSETEQ AL", // <-- corrupts low byte of AX
|
|
"\tMOVBLZX AL, R10", // R10 = clean bool
|
|
"\tMOVL 100(SP), R11", // would-be peephole forward target
|
|
"",
|
|
}, "\n")
|
|
out := regTrackPass(in)
|
|
if strings.Contains(out, "\tMOVL AX, R11") {
|
|
t.Errorf("regTrackPass forwarded AX to R11 after a SETEQ AL clobbered "+
|
|
"AX's low byte. The second load must stay a memory read.\nOutput:\n%s", out)
|
|
}
|
|
// Sanity: the second slot load should still be present (not
|
|
// dropped, just not rewritten).
|
|
if !strings.Contains(out, "\tMOVL 100(SP), R11") {
|
|
t.Errorf("regTrackPass dropped the second slot load entirely; expected "+
|
|
"it to survive verbatim. Output:\n%s", out)
|
|
}
|
|
}
|
|
|
|
// TestRegTrackByteRegParent exercises byteRegParent's mapping
|
|
// table — a small data table whose entries are easy to forget when
|
|
// new byte register names are added. Each well-known plan9 byte
|
|
// register name should map back to its 64-bit parent; anything
|
|
// outside the table returns (_, false).
|
|
func TestRegTrackByteRegParent(t *testing.T) {
|
|
cases := []struct {
|
|
byteName, parent string
|
|
}{
|
|
{"AL", "AX"}, {"AH", "AX"},
|
|
{"BL", "BX"}, {"BH", "BX"},
|
|
{"CL", "CX"}, {"CH", "CX"},
|
|
{"DL", "DX"}, {"DH", "DX"},
|
|
{"SIL", "SI"}, {"DIL", "DI"}, {"BPL", "BP"},
|
|
{"R8B", "R8"}, {"R9B", "R9"},
|
|
{"R10B", "R10"}, {"R11B", "R11"},
|
|
{"R12B", "R12"}, {"R13B", "R13"},
|
|
{"R14B", "R14"}, {"R15B", "R15"},
|
|
}
|
|
for _, c := range cases {
|
|
got, ok := byteRegParent(c.byteName)
|
|
if !ok {
|
|
t.Errorf("byteRegParent(%q) returned ok=false; want parent=%q", c.byteName, c.parent)
|
|
continue
|
|
}
|
|
if got != c.parent {
|
|
t.Errorf("byteRegParent(%q) = %q; want %q", c.byteName, got, c.parent)
|
|
}
|
|
}
|
|
// Negative case: a non-byte name returns (_, false).
|
|
if _, ok := byteRegParent("AX"); ok {
|
|
t.Errorf("byteRegParent(%q) = (_, true); want false (not a byte reg)", "AX")
|
|
}
|
|
}
|
|
|
|
// TestEmitMemopsAMD64 is the 2.3 milestone: linear-memory load /
|
|
// store ops at every wasm width, exercised through cg_memops.wat.
|
|
// The driver instantiates a real Module with two pages of memory and
|
|
// the fixture's data segment pre-loaded; the asm dereferences m.M
|
|
// (cached unsafe.Pointer) at the offset moduleMOffset to compute
|
|
// effective addresses. Float load/store ops are emitted but not
|
|
// asserted by name here — the prototype focuses on the integer
|
|
// path that the SQL parser actually relies on. mem_size / mem_grow
|
|
// are not yet implemented in the emitter and their exports are
|
|
// omitted.
|
|
func TestEmitMemopsAMD64(t *testing.T) {
|
|
if runtime.GOARCH != "amd64" {
|
|
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
|
|
}
|
|
exports := []string{
|
|
"l_i32", "l_i32_8s", "l_i32_8u", "l_i32_16s", "l_i32_16u",
|
|
"l_i64", "l_i64_8s", "l_i64_8u", "l_i64_16s", "l_i64_16u", "l_i64_32s", "l_i64_32u",
|
|
"s_i32", "s_i32_8", "s_i32_16",
|
|
"s_i64", "s_i64_8", "s_i64_16", "s_i64_32",
|
|
"l_offset", "s_offset",
|
|
"mem_size", "mem_grow",
|
|
}
|
|
// Data segment: bytes [1..16] at offset 0. Driver init() repeats
|
|
// the same content.
|
|
cases := []driverCase{
|
|
// --- i32 loads ---
|
|
{"l_i32", []string{"0"}, "67305985"}, // LE [1,2,3,4] = 0x04030201
|
|
{"l_i32", []string{"4"}, "134678021"}, // LE [5,6,7,8] = 0x08070605
|
|
{"l_i32_8u", []string{"0"}, "1"},
|
|
{"l_i32_8u", []string{"7"}, "8"},
|
|
{"l_i32_8s", []string{"0"}, "1"},
|
|
{"l_i32_8s", []string{"7"}, "8"},
|
|
{"l_i32_16u", []string{"0"}, "513"}, // 0x0201
|
|
{"l_i32_16s", []string{"0"}, "513"},
|
|
// --- i64 loads ---
|
|
{"l_i64", []string{"0"}, "578437695752307201"}, // 0x0807060504030201
|
|
{"l_i64_8u", []string{"0"}, "1"},
|
|
{"l_i64_8s", []string{"15"}, "16"},
|
|
{"l_i64_16u", []string{"0"}, "513"},
|
|
{"l_i64_16s", []string{"0"}, "513"},
|
|
{"l_i64_32u", []string{"0"}, "67305985"},
|
|
{"l_i64_32s", []string{"0"}, "67305985"},
|
|
// --- i32 store→load round trips ---
|
|
{"s_i32", []string{"100", "305419896"}, "305419896"}, // 0x12345678
|
|
{"s_i32", []string{"104", "-1"}, "-1"},
|
|
{"s_i32_8", []string{"100", "255"}, "255"},
|
|
{"s_i32_16", []string{"100", "65535"}, "65535"},
|
|
// --- i64 store→load round trips ---
|
|
{"s_i64", []string{"200", "1311768467463790320"}, "1311768467463790320"}, // 0x123456789ABCDEF0
|
|
{"s_i64_8", []string{"200", "255"}, "255"},
|
|
{"s_i64_16", []string{"200", "65535"}, "65535"},
|
|
{"s_i64_32", []string{"200", "4294967295"}, "4294967295"},
|
|
// --- offset memarg ---
|
|
{"l_offset", []string{"0"}, "202050057"}, // offset=8: [9..12] = 0x0C0B0A09
|
|
{"s_offset", []string{"300", "42"}, "42"}, // offset=12: writes at 312, reads at 312
|
|
// memory.size / memory.grow — each test binary starts fresh,
|
|
// so the page count begins at 2 (the .wat declares 2 pages).
|
|
{"mem_size", []string{}, "2"},
|
|
{"mem_grow", []string{"3"}, "2"}, // grow by 3 → returns old size = 2
|
|
{"mem_grow", []string{"-1"}, "-1"},
|
|
}
|
|
buildAndRunDriver(t, "cg_memops", exports, driverWithMemory(), cases)
|
|
}
|
|
|
|
// TestEmitMemopsARM64 runs the same wasm32 memops value checks
|
|
// against the ARM64 emitter, executing natively (see
|
|
// TestEmitControlARM64 for the Rosetta-host detection).
|
|
func TestEmitMemopsARM64(t *testing.T) {
|
|
if !canExecDarwinARM64() && runtime.GOARCH != "arm64" {
|
|
t.Skipf("host cannot execute arm64 binaries (GOOS=%s GOARCH=%s)", runtime.GOOS, runtime.GOARCH)
|
|
}
|
|
exports := []string{
|
|
"l_i32", "l_i32_8s", "l_i32_8u", "l_i32_16s", "l_i32_16u",
|
|
"l_i64", "l_i64_8s", "l_i64_8u", "l_i64_16s", "l_i64_16u", "l_i64_32s", "l_i64_32u",
|
|
"s_i32", "s_i32_8", "s_i32_16",
|
|
"s_i64", "s_i64_8", "s_i64_16", "s_i64_32",
|
|
"l_offset", "s_offset",
|
|
"mem_size", "mem_grow",
|
|
}
|
|
cases := []driverCase{
|
|
{"l_i32", []string{"0"}, "67305985"},
|
|
{"l_i32", []string{"4"}, "134678021"},
|
|
{"l_i32_8u", []string{"7"}, "8"},
|
|
{"l_i32_8s", []string{"7"}, "8"},
|
|
{"l_i32_16u", []string{"0"}, "513"},
|
|
{"l_i32_16s", []string{"0"}, "513"},
|
|
{"l_i64", []string{"0"}, "578437695752307201"},
|
|
{"l_i64_8u", []string{"0"}, "1"},
|
|
{"l_i64_8s", []string{"15"}, "16"},
|
|
{"l_i64_16u", []string{"0"}, "513"},
|
|
{"l_i64_16s", []string{"0"}, "513"},
|
|
{"l_i64_32u", []string{"0"}, "67305985"},
|
|
{"l_i64_32s", []string{"0"}, "67305985"},
|
|
{"s_i32", []string{"100", "305419896"}, "305419896"},
|
|
{"s_i32", []string{"104", "-1"}, "-1"},
|
|
{"s_i32_8", []string{"100", "255"}, "255"},
|
|
{"s_i32_16", []string{"100", "65535"}, "65535"},
|
|
{"s_i64", []string{"200", "1311768467463790320"}, "1311768467463790320"},
|
|
{"s_i64_8", []string{"200", "255"}, "255"},
|
|
{"s_i64_16", []string{"200", "65535"}, "65535"},
|
|
{"s_i64_32", []string{"200", "4294967295"}, "4294967295"},
|
|
{"l_offset", []string{"0"}, "202050057"},
|
|
{"s_offset", []string{"300", "42"}, "42"},
|
|
{"mem_size", []string{}, "2"},
|
|
{"mem_grow", []string{"3"}, "2"},
|
|
{"mem_grow", []string{"-1"}, "-1"},
|
|
}
|
|
buildAndRunDriverArch(t, "cg_memops", exports, driverWithMemory(), cases, "arm64")
|
|
}
|
|
|
|
// driverWithMemory64 is driverWithMemory with the 64-bit memory
|
|
// helper family (memorySize64 / memoryGrow64) that asmgen routes
|
|
// OpMemSize / OpMemGrow to on memory64 modules. Module layout is
|
|
// identical — only the addressing width of the wasm side changes.
|
|
func driverWithMemory64() driverModule {
|
|
d := driverWithMemory()
|
|
d.moduleSrc = strings.Replace(d.moduleSrc, `func memorySize(m *Module) int32 { return int32(len(m.Memory) / 65536) }
|
|
|
|
func memoryGrow(m *Module, delta int32) int32 {`, `func memorySize64(m *Module) int64 { return int64(len(m.Memory) / 65536) }
|
|
|
|
func memoryGrow64(m *Module, delta int64) int64 {`, 1)
|
|
d.moduleSrc = strings.Replace(d.moduleSrc, ` cur := int32(len(m.Memory) / 65536)`, ` cur := int64(len(m.Memory) / 65536)`, 1)
|
|
return d
|
|
}
|
|
|
|
// mem64MemopsCases is the memory64 twin of the TestEmitMemopsAMD64
|
|
// case list: same data segment, same expected values, i64 addresses.
|
|
// mem_size / mem_grow return i64 on a 64-bit memory.
|
|
func mem64MemopsCases() ([]string, []driverCase) {
|
|
exports := []string{
|
|
"l_i32", "l_i32_8s", "l_i32_8u", "l_i32_16s", "l_i32_16u",
|
|
"l_i64", "l_i64_8s", "l_i64_8u", "l_i64_16s", "l_i64_16u", "l_i64_32s", "l_i64_32u",
|
|
"s_i32", "s_i32_8", "s_i32_16",
|
|
"s_i64", "s_i64_8", "s_i64_16", "s_i64_32",
|
|
"l_offset", "s_offset", "l_constbase",
|
|
"mem_size", "mem_grow",
|
|
}
|
|
cases := []driverCase{
|
|
{"l_i32", []string{"0"}, "67305985"},
|
|
{"l_i32", []string{"4"}, "134678021"},
|
|
{"l_i32_8u", []string{"0"}, "1"},
|
|
{"l_i32_8u", []string{"7"}, "8"},
|
|
{"l_i32_8s", []string{"0"}, "1"},
|
|
{"l_i32_8s", []string{"7"}, "8"},
|
|
{"l_i32_16u", []string{"0"}, "513"},
|
|
{"l_i32_16s", []string{"0"}, "513"},
|
|
{"l_i64", []string{"0"}, "578437695752307201"},
|
|
{"l_i64_8u", []string{"0"}, "1"},
|
|
{"l_i64_8s", []string{"15"}, "16"},
|
|
{"l_i64_16u", []string{"0"}, "513"},
|
|
{"l_i64_16s", []string{"0"}, "513"},
|
|
{"l_i64_32u", []string{"0"}, "67305985"},
|
|
{"l_i64_32s", []string{"0"}, "67305985"},
|
|
{"s_i32", []string{"100", "305419896"}, "305419896"},
|
|
{"s_i32", []string{"104", "-1"}, "-1"},
|
|
{"s_i32_8", []string{"100", "255"}, "255"},
|
|
{"s_i32_16", []string{"100", "65535"}, "65535"},
|
|
{"s_i64", []string{"200", "1311768467463790320"}, "1311768467463790320"},
|
|
{"s_i64_8", []string{"200", "255"}, "255"},
|
|
{"s_i64_16", []string{"200", "65535"}, "65535"},
|
|
{"s_i64_32", []string{"200", "4294967295"}, "4294967295"},
|
|
{"l_offset", []string{"0"}, "202050057"},
|
|
{"s_offset", []string{"300", "42"}, "42"},
|
|
// constant i64 base folded with the memarg offset at
|
|
// generation time: bytes [13..16] LE = 0x100F0E0D.
|
|
{"l_constbase", []string{}, "269422093"},
|
|
{"mem_size", []string{}, "2"},
|
|
{"mem_grow", []string{"3"}, "2"},
|
|
{"mem_grow", []string{"-1"}, "-1"},
|
|
}
|
|
return exports, cases
|
|
}
|
|
|
|
// TestEmitMemopsMem64AMD64 / ...ARM64 run the memory64 twin of the
|
|
// memops value checks: i64 bases with no mod-2^32 wrap, offsets added
|
|
// in full 64-bit, and the 64-bit mem-size/grow helper routing.
|
|
func TestEmitMemopsMem64AMD64(t *testing.T) {
|
|
if runtime.GOARCH != "amd64" {
|
|
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
|
|
}
|
|
exports, cases := mem64MemopsCases()
|
|
buildAndRunDriver(t, "cg_mem64_memops", exports, driverWithMemory64(), cases)
|
|
}
|
|
|
|
func TestEmitMemopsMem64ARM64(t *testing.T) {
|
|
if !canExecDarwinARM64() && runtime.GOARCH != "arm64" {
|
|
t.Skipf("host cannot execute arm64 binaries (GOOS=%s GOARCH=%s)", runtime.GOOS, runtime.GOARCH)
|
|
}
|
|
exports, cases := mem64MemopsCases()
|
|
buildAndRunDriverArch(t, "cg_mem64_memops", exports, driverWithMemory64(), cases, "arm64")
|
|
}
|
|
|
|
// driverHostAdd is the driver for the OpCallImport fixture. The
|
|
// Module carries an interface field `host` (hostImports) whose
|
|
// implementation adds the integer args (i32 and i64 variants) and
|
|
// records the noop calls so the test can assert side-effects.
|
|
func driverHostAdd() driverModule {
|
|
return driverModule{
|
|
moduleSrc: `package main
|
|
|
|
// hostImports is the wasm-side imports interface the asm
|
|
// wrappers dispatch through. Method shape: the host always
|
|
// receives *Module as its first parameter so it can reach back
|
|
// into memory if it needs to.
|
|
type hostImports interface {
|
|
Add(m *Module, a, b int32) int32
|
|
Add64(m *Module, a, b int64) int64
|
|
Noop(m *Module, x int32)
|
|
}
|
|
|
|
// Module is the host-side runtime container. The asm reads m
|
|
// directly at FP+0 (it doesn't dereference it for this fixture
|
|
// because no memory access goes through it), but the import
|
|
// wrappers do.
|
|
type Module struct {
|
|
host hostImports
|
|
}
|
|
|
|
type hostImpl struct{ noopSeen int32 }
|
|
|
|
func (h *hostImpl) Add(_ *Module, a, b int32) int32 { return a + b }
|
|
func (h *hostImpl) Add64(_ *Module, a, b int64) int64 { return a + b }
|
|
func (h *hostImpl) Noop(_ *Module, x int32) { h.noopSeen = x }
|
|
|
|
var modPtr *Module
|
|
|
|
func init() { modPtr = &Module{host: &hostImpl{}} }
|
|
`,
|
|
modExpr: "modPtr",
|
|
importWrappers: func(mod *wasm.Module) string {
|
|
// Emit one wrapper per ImportFunc in declaration order.
|
|
// The funcIdx assigned by wasm matches the loop counter
|
|
// because we only walk ImportFunc entries.
|
|
var b strings.Builder
|
|
var fIdx uint32
|
|
for _, imp := range mod.Imports {
|
|
if imp.Kind != wasm.ImportFunc {
|
|
continue
|
|
}
|
|
sig := mod.Types[imp.TypeIdx]
|
|
fmt.Fprintf(&b, "func callImport_%d", fIdx)
|
|
b.WriteString(goSignature(sig, "*Module"))
|
|
b.WriteString(" {\n\t")
|
|
if len(sig.Results) > 0 {
|
|
b.WriteString("return ")
|
|
}
|
|
// host.<MethodName>(m, args...) — MangleID + capitalize.
|
|
method := capitalizeImport(imp.Name)
|
|
fmt.Fprintf(&b, "m.host.%s(m", method)
|
|
for i := range sig.Params {
|
|
fmt.Fprintf(&b, ", l%d", i)
|
|
}
|
|
b.WriteString(")\n}\n")
|
|
fIdx++
|
|
}
|
|
return b.String()
|
|
},
|
|
}
|
|
}
|
|
|
|
// capitalizeImport mirrors codegen's importMethodName for plain
|
|
// ASCII names (which is all the fixture uses): capitalise the
|
|
// first letter so the method satisfies an interface declared by
|
|
// the test driver. Real codegen routes through MangleID for non-
|
|
// ASCII / C++-mangled names; we don't need that for the fixture.
|
|
func capitalizeImport(s string) string {
|
|
if s == "" {
|
|
return s
|
|
}
|
|
r := []byte(s)
|
|
if r[0] >= 'a' && r[0] <= 'z' {
|
|
r[0] = r[0] - 'a' + 'A'
|
|
}
|
|
return string(r)
|
|
}
|
|
|
|
// driverWithGlobals builds a Module that carries the cg_globals
|
|
// fixture's five globals. The driver also generates loadGlobal_N /
|
|
// storeGlobal_N wrappers per defined global so OpGlobalGet/Set in
|
|
// asm can route through them.
|
|
func driverWithGlobals() driverModule {
|
|
moduleSrc := `package main
|
|
|
|
// Module field names mirror the codegen translator's emission:
|
|
// g<idx> (lowercase in single-package mode). The asm-side wrappers
|
|
// access these fields by name so any layout change is caught at Go
|
|
// compile time.
|
|
type Module struct {
|
|
g0 int32 // gi32 (mut)
|
|
g1 int64 // gi64 (mut)
|
|
g2 float32 // gf32 (mut)
|
|
g3 float64 // gf64 (mut)
|
|
g4 int32 // cimmu (immut)
|
|
}
|
|
|
|
var modPtr *Module
|
|
|
|
func init() { modPtr = &Module{g0: 11, g1: 22, g2: 1.5, g3: 2.5, g4: 99} }
|
|
|
|
// Per-global wrappers (one load + one store per global) — declared
|
|
// here rather than emitted by the importWrappers callback because
|
|
// the cg_globals fixture has no host imports. A future asmgen
|
|
// integration with the codegen translator would unify the two
|
|
// wrapper emit paths under one Driver-aware generator.
|
|
func loadGlobal_0(m *Module) int32 { return m.g0 }
|
|
func storeGlobal_0(m *Module, v int32) { m.g0 = v }
|
|
func loadGlobal_1(m *Module) int64 { return m.g1 }
|
|
func storeGlobal_1(m *Module, v int64) { m.g1 = v }
|
|
func loadGlobal_2(m *Module) float32 { return m.g2 }
|
|
func storeGlobal_2(m *Module, v float32){ m.g2 = v }
|
|
func loadGlobal_3(m *Module) float64 { return m.g3 }
|
|
func storeGlobal_3(m *Module, v float64){ m.g3 = v }
|
|
func loadGlobal_4(m *Module) int32 { return m.g4 }
|
|
`
|
|
return driverModule{moduleSrc: moduleSrc, modExpr: "modPtr"}
|
|
}
|
|
|
|
// driverIndirect builds the Module for cg_indirect.wat. It needs
|
|
// - a `t0 []any` field for the funcref table,
|
|
// - a `g0 int32` field for the $sink global,
|
|
// - one callIndirect_typeN wrapper per wasm type used by an
|
|
// OpCallIndirect (5 types in this fixture),
|
|
// - loadGlobal_0 / storeGlobal_0 wrappers,
|
|
// - an init() that fills t0[0..6] with the asm-emitted function
|
|
// pointers in the order the elem segment specifies.
|
|
func driverIndirect() driverModule {
|
|
moduleSrc := `package main
|
|
|
|
type Module struct {
|
|
t0 []any
|
|
g0 int32 // $sink
|
|
}
|
|
|
|
var modPtr *Module
|
|
|
|
func init() {
|
|
modPtr = &Module{
|
|
t0: make([]any, 12),
|
|
g0: 0,
|
|
}
|
|
// elem (i32.const 0) $negate $dbl $addp $subp $mul64 $const7 $store
|
|
// — funcIdx 0..6 in this fixture (no imports).
|
|
modPtr.t0[0] = Fn0
|
|
modPtr.t0[1] = Fn1
|
|
modPtr.t0[2] = Fn2
|
|
modPtr.t0[3] = Fn3
|
|
modPtr.t0[4] = Fn4
|
|
modPtr.t0[5] = Fn5
|
|
modPtr.t0[6] = Fn6
|
|
}
|
|
|
|
func loadGlobal_0(m *Module) int32 { return m.g0 }
|
|
func storeGlobal_0(m *Module, v int32) { m.g0 = v }
|
|
|
|
// callIndirect wrappers — one per wasm type used by an
|
|
// OpCallIndirect. Each does the type assertion that the asm
|
|
// can't easily do by itself, then calls the resulting func ptr.
|
|
// type 0 ($unary): (param i32) -> i32
|
|
// type 1 ($binary): (param i32 i32) -> i32
|
|
// type 2 ($i64bin): (param i64 i64) -> i64
|
|
// type 3 ($noargs): () -> i32
|
|
// type 4 ($voidproc): (param i32) -> ()
|
|
func callIndirect_type0(m *Module, idx int32, p0 int32) int32 {
|
|
return m.t0[idx].(func(*Module, int32) int32)(m, p0)
|
|
}
|
|
func callIndirect_type1(m *Module, idx int32, p0, p1 int32) int32 {
|
|
return m.t0[idx].(func(*Module, int32, int32) int32)(m, p0, p1)
|
|
}
|
|
func callIndirect_type2(m *Module, idx int32, p0, p1 int64) int64 {
|
|
return m.t0[idx].(func(*Module, int64, int64) int64)(m, p0, p1)
|
|
}
|
|
func callIndirect_type3(m *Module, idx int32) int32 {
|
|
return m.t0[idx].(func(*Module) int32)(m)
|
|
}
|
|
func callIndirect_type4(m *Module, idx int32, p0 int32) {
|
|
m.t0[idx].(func(*Module, int32))(m, p0)
|
|
}
|
|
`
|
|
return driverModule{moduleSrc: moduleSrc, modExpr: "modPtr"}
|
|
}
|
|
|
|
// TestEmitCallIndirectAMD64 covers OpCallIndirect against
|
|
// cg_indirect.wat. Each `call_indirect (type $T)` lowers to a CALL
|
|
// of a per-type Go wrapper that performs the runtime
|
|
// type-assertion the asm can't easily express by itself. Globals
|
|
// (via $sink for the void-returning indirect path) and direct
|
|
// calls (direct_chain) flow through the same emitter pipeline.
|
|
func TestEmitCallIndirectAMD64(t *testing.T) {
|
|
if runtime.GOARCH != "amd64" {
|
|
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
|
|
}
|
|
exports := []string{
|
|
"ind_unary", "ind_binary", "ind_i64", "ind_noargs", "ind_void",
|
|
"direct_chain",
|
|
}
|
|
cases := []driverCase{
|
|
// ind_unary: table[which](x)
|
|
{"ind_unary", []string{"0", "5"}, "-5"}, // negate(5)
|
|
{"ind_unary", []string{"1", "5"}, "10"}, // dbl(5)
|
|
{"ind_unary", []string{"0", "-7"}, "7"}, // negate(-7)
|
|
// ind_binary: table[which](a, b)
|
|
{"ind_binary", []string{"2", "3", "4"}, "7"}, // addp
|
|
{"ind_binary", []string{"3", "10", "3"}, "7"}, // subp
|
|
// ind_i64
|
|
{"ind_i64", []string{"4", "6", "7"}, "42"}, // mul64
|
|
// ind_noargs
|
|
{"ind_noargs", []string{"5"}, "7"}, // const7
|
|
// ind_void: stores its arg to $sink, returns $sink
|
|
{"ind_void", []string{"6", "42"}, "42"},
|
|
// direct_chain(x) = dbl(addp(negate(x), 100)) = 2*(-x+100)
|
|
{"direct_chain", []string{"5"}, "190"},
|
|
{"direct_chain", []string{"-50"}, "300"},
|
|
}
|
|
buildAndRunDriver(t, "cg_indirect", exports, driverIndirect(), cases)
|
|
}
|
|
|
|
// TestEmitGlobalsAMD64 covers OpGlobalGet / OpGlobalSet against
|
|
// cg_globals.wat. Each wasm global lives as a field on the host
|
|
// Module struct, and the asm reads/writes via loadGlobal_<idx> /
|
|
// storeGlobal_<idx> wrappers the driver emits. Float ops that route
|
|
// through helper calls (bump_f64) are tested only when those
|
|
// helpers are wired up; bump_f64 is left to a future step.
|
|
func TestEmitGlobalsAMD64(t *testing.T) {
|
|
if runtime.GOARCH != "amd64" {
|
|
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
|
|
}
|
|
exports := []string{
|
|
"get_i32", "set_i32",
|
|
"get_i64", "set_i64",
|
|
"get_f32", "set_f32",
|
|
"get_f64", "set_f64",
|
|
"get_const",
|
|
"bump_i32",
|
|
}
|
|
cases := []driverCase{
|
|
{"get_i32", nil, "11"},
|
|
{"get_i64", nil, "22"},
|
|
{"get_f32", nil, "1.5"},
|
|
{"get_f64", nil, "2.5"},
|
|
{"get_const", nil, "99"},
|
|
// set_X are void — the dispatcher prints "ok" on success.
|
|
{"set_i32", []string{"77"}, "ok"},
|
|
{"set_i64", []string{"77"}, "ok"},
|
|
{"set_f32", []string{"7.25"}, "ok"},
|
|
{"set_f64", []string{"7.25"}, "ok"},
|
|
// bump_i32: round-trip through the global within one call.
|
|
{"bump_i32", []string{"0"}, "11"},
|
|
{"bump_i32", []string{"100"}, "111"},
|
|
{"bump_i32", []string{"-5"}, "6"},
|
|
}
|
|
buildAndRunDriver(t, "cg_globals", exports, driverWithGlobals(), cases)
|
|
}
|
|
|
|
// TestEmitCallImportAMD64 covers OpCallImport — wasm `call` of an
|
|
// imported function — by routing each call through a generated Go
|
|
// wrapper that dispatches via the Module's hostImports interface.
|
|
// The fixture exposes three import shapes: i32 args + i32 result,
|
|
// i64 args + i64 result, and a void-returning call. The asm CALLs a
|
|
// uniformly-named `callImport_<funcIdx>` symbol; the test driver
|
|
// emits one wrapper per import, generated from `mod.Imports`.
|
|
func TestEmitCallImportAMD64(t *testing.T) {
|
|
if runtime.GOARCH != "amd64" {
|
|
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
|
|
}
|
|
exports := []string{"call_hostadd", "call_hostadd64", "call_hostnoop"}
|
|
cases := []driverCase{
|
|
{"call_hostadd", []string{"3", "4"}, "7"},
|
|
{"call_hostadd", []string{"-100", "100"}, "0"},
|
|
{"call_hostadd", []string{"2147483647", "1"}, "-2147483648"}, // i32 wrap
|
|
{"call_hostadd64", []string{"1000000000000", "1"}, "1000000000001"},
|
|
{"call_hostadd64", []string{"-1", "1"}, "0"},
|
|
// call_hostnoop returns x+1 after a no-op host call.
|
|
{"call_hostnoop", []string{"0"}, "1"},
|
|
{"call_hostnoop", []string{"41"}, "42"},
|
|
}
|
|
buildAndRunDriver(t, "asmtest_callimport", exports, driverHostAdd(), cases)
|
|
}
|
|
|
|
// driverBulkMem builds a Module with linear memory plus
|
|
// memoryCopy / memoryFill wrappers for the bulk-memory ops. The
|
|
// initial memory contents come from the cg_bulkmem fixture's data
|
|
// segment ("abcdefghij" at offset 0).
|
|
func driverBulkMem() driverModule {
|
|
return driverModule{
|
|
moduleSrc: `package main
|
|
|
|
import "unsafe"
|
|
|
|
type Module struct {
|
|
Memory []byte
|
|
MaxMem uint64
|
|
M unsafe.Pointer
|
|
}
|
|
|
|
var modPtr *Module
|
|
|
|
func init() {
|
|
mem := make([]byte, 65536)
|
|
copy(mem, []byte("abcdefghij"))
|
|
modPtr = &Module{
|
|
Memory: mem,
|
|
MaxMem: 65536,
|
|
M: unsafe.Pointer(&mem[0]),
|
|
}
|
|
}
|
|
|
|
// memoryCopy implements wasm memory.copy: copy n bytes from src to
|
|
// dst within the linear memory, with the overlap-correct semantics
|
|
// the wasm spec requires (Go's built-in copy handles this).
|
|
func memoryCopy(m *Module, dst, src, n int32) {
|
|
if n <= 0 {
|
|
return
|
|
}
|
|
copy(m.Memory[dst:dst+n], m.Memory[src:src+n])
|
|
}
|
|
|
|
// memoryFill writes the low byte of val to n bytes starting at addr.
|
|
func memoryFill(m *Module, addr, val, n int32) {
|
|
if n <= 0 {
|
|
return
|
|
}
|
|
b := byte(val)
|
|
for i := int32(0); i < n; i++ {
|
|
m.Memory[addr+i] = b
|
|
}
|
|
}
|
|
`,
|
|
modExpr: "modPtr",
|
|
}
|
|
}
|
|
|
|
// TestEmitBulkMemAMD64 covers OpMemoryCopy and OpMemoryFill against
|
|
// cg_bulkmem.wat. The fixture's saturating-trunc exports use
|
|
// f-conversion helpers the prototype doesn't wire up yet; they're
|
|
// silently stubbed and not asserted here.
|
|
func TestEmitBulkMemAMD64(t *testing.T) {
|
|
if runtime.GOARCH != "amd64" {
|
|
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
|
|
}
|
|
exports := []string{"mem_copy", "mem_fill", "load16_s", "store16", "load64"}
|
|
cases := []driverCase{
|
|
// mem_copy(dst=20, src=0, n=4) copies "abcd" into mem[20..],
|
|
// then returns mem[20] as a u8 → 'a' = 97.
|
|
{"mem_copy", []string{"20", "0", "4"}, "97"},
|
|
// mem_fill(addr=100, val=0xAA, n=8) writes 8 copies of 0xAA at
|
|
// mem[100..108]; returning mem[100] u8 → 0xAA = 170.
|
|
{"mem_fill", []string{"100", "170", "8"}, "170"},
|
|
// Sanity-check the surrounding load/store ops are still happy
|
|
// in the same driver.
|
|
{"load16_s", []string{"0"}, "25185"}, // 'a' (0x61) + 'b'<<8 (0x6200) = 0x6261
|
|
{"load64", []string{"0"}, "7523094288207667809"}, // "abcdefgh" little-endian
|
|
}
|
|
buildAndRunDriver(t, "cg_bulkmem", exports, driverBulkMem(), cases)
|
|
}
|
|
|
|
// TestEmitCallDirectAMD64 covers OpCallDirect with cg_manyfuncs.wat:
|
|
// `sum_all(x)` invokes thirty `leaf<i>` functions in sequence and
|
|
// adds the results. Each leaf computes `x * (i+1) + 3*i`, so
|
|
// `sum_all(x) = sum_{i=0..29}[x*(i+1) + 3*i] = 465*x + 1305`. The
|
|
// fixture also exposes ops the prototype emitter doesn't support yet
|
|
// (call_indirect, global.get / global.set, memory.size); those are
|
|
// silently stubbed by the driver and never invoked from the cases
|
|
// below.
|
|
func TestEmitCallDirectAMD64(t *testing.T) {
|
|
if runtime.GOARCH != "amd64" {
|
|
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
|
|
}
|
|
exports := []string{"sum_all"}
|
|
expected := func(x int32) string {
|
|
return fmt.Sprintf("%d", 465*x+1305)
|
|
}
|
|
cases := []driverCase{
|
|
{"sum_all", []string{"0"}, expected(0)},
|
|
{"sum_all", []string{"1"}, expected(1)},
|
|
{"sum_all", []string{"-1"}, expected(-1)},
|
|
{"sum_all", []string{"7"}, expected(7)},
|
|
{"sum_all", []string{"100"}, expected(100)},
|
|
{"sum_all", []string{"-100"}, expected(-100)},
|
|
}
|
|
buildAndRunDriver(t, "cg_manyfuncs", exports, driverPlaceholder, cases)
|
|
}
|
|
|
|
// dispatchCase emits the `case "<name>": ... <call> ...` arm for one
|
|
// export. Inputs are parsed from os.Args, the function is called
|
|
// with modExpr as the first (m *Module) argument, and the result is
|
|
// printed. modExpr is "nil" for tests whose asm never dereferences
|
|
// m (the arith / control fixtures) and "modPtr" for tests that need
|
|
// a real backing memory (the memops fixture); the driver template
|
|
// declares modPtr accordingly.
|
|
func dispatchCase(name string, sig wasm.FuncType, modExpr string) string {
|
|
var b strings.Builder
|
|
fmt.Fprintf(&b, "case %q:\n", name)
|
|
args := []string{modExpr}
|
|
for i, p := range sig.Params {
|
|
switch p {
|
|
case wasm.ValI32:
|
|
fmt.Fprintf(&b, "\tv%d, err := strconv.ParseInt(os.Args[%d], 10, 32)\n", i, i+2)
|
|
fmt.Fprintf(&b, "\tif err != nil { fmt.Fprintln(os.Stderr, err); os.Exit(2) }\n")
|
|
args = append(args, fmt.Sprintf("int32(v%d)", i))
|
|
case wasm.ValI64:
|
|
fmt.Fprintf(&b, "\tv%d, err := strconv.ParseInt(os.Args[%d], 10, 64)\n", i, i+2)
|
|
fmt.Fprintf(&b, "\tif err != nil { fmt.Fprintln(os.Stderr, err); os.Exit(2) }\n")
|
|
args = append(args, fmt.Sprintf("v%d", i))
|
|
case wasm.ValF32:
|
|
fmt.Fprintf(&b, "\tv%d, err := strconv.ParseFloat(os.Args[%d], 32)\n", i, i+2)
|
|
fmt.Fprintf(&b, "\tif err != nil { fmt.Fprintln(os.Stderr, err); os.Exit(2) }\n")
|
|
args = append(args, fmt.Sprintf("float32(v%d)", i))
|
|
case wasm.ValF64:
|
|
fmt.Fprintf(&b, "\tv%d, err := strconv.ParseFloat(os.Args[%d], 64)\n", i, i+2)
|
|
fmt.Fprintf(&b, "\tif err != nil { fmt.Fprintln(os.Stderr, err); os.Exit(2) }\n")
|
|
args = append(args, fmt.Sprintf("v%d", i))
|
|
}
|
|
}
|
|
if len(sig.Results) == 0 {
|
|
// void-returning export: print a stable token so the
|
|
// dispatcher's exit status alone signals success.
|
|
fmt.Fprintf(&b, "\t%s(%s)\n\tfmt.Println(\"ok\")\n", name, strings.Join(args, ", "))
|
|
} else {
|
|
fmt.Fprintf(&b, "\tfmt.Println(%s(%s))\n", name, strings.Join(args, ", "))
|
|
}
|
|
return b.String()
|
|
}
|
|
|
|
func writeDriverModuleArch(t *testing.T, dir string, driver driverModule, asm, decls, stubs, wrappers, dispatch, goarch string) {
|
|
t.Helper()
|
|
tag := "//go:build " + goarch + "\n\npackage main\n\n"
|
|
mustWrite(t, dir, "go.mod", "module asmgentest\n\ngo 1.25\n")
|
|
mustWrite(t, dir, "module.go", driver.moduleSrc)
|
|
mustWrite(t, dir, "decls_"+goarch+".go", tag+decls)
|
|
mustWrite(t, dir, "asm_"+goarch+".s", "//go:build "+goarch+"\n\n#include \"textflag.h\"\n#include \"funcdata.h\"\n\n"+asm)
|
|
mustWrite(t, dir, "helpers_"+goarch+".go", helpersGoSourceArch(goarch))
|
|
if stubs != "" {
|
|
mustWrite(t, dir, "stubs_"+goarch+".go", tag+stubs)
|
|
}
|
|
if wrappers != "" {
|
|
mustWrite(t, dir, "wrappers_"+goarch+".go", tag+wrappers)
|
|
}
|
|
mustWrite(t, dir, "main.go", `package main
|
|
|
|
import (
|
|
"fmt"
|
|
"os"
|
|
"strconv"
|
|
)
|
|
|
|
func main() {
|
|
if len(os.Args) < 2 {
|
|
fmt.Fprintln(os.Stderr, "usage: driver <export> args...")
|
|
os.Exit(2)
|
|
}
|
|
`+dispatch+`
|
|
}
|
|
`)
|
|
}
|
|
|
|
func mustWrite(t *testing.T, dir, name, content string) {
|
|
t.Helper()
|
|
if err := os.WriteFile(filepath.Join(dir, name), []byte(content), 0o644); err != nil {
|
|
t.Fatalf("write %s: %v", name, err)
|
|
}
|
|
}
|
|
|
|
// helpersGoSource returns the test-local Go source containing every
|
|
// helper the asm bodies may CALL. The functions mirror the bodies in
|
|
// internal/codegen/helpers/helpers.go; copying them here keeps the
|
|
// integration test free of an internal-package import. The build
|
|
// constraint matches the asm so this file compiles only on the arch
|
|
// the driver was emitted for.
|
|
func helpersGoSource() string {
|
|
return helpersGoSourceArch("amd64")
|
|
}
|
|
|
|
func helpersGoSourceArch(goarch string) string {
|
|
return `//go:build ` + goarch + `
|
|
|
|
package main
|
|
|
|
import (
|
|
"math"
|
|
"math/bits"
|
|
)
|
|
|
|
func i32_eqz(x int32) int32 { if x == 0 { return 1 }; return 0 }
|
|
func i64_eqz(x int64) int32 { if x == 0 { return 1 }; return 0 }
|
|
|
|
func i32_clz(x int32) int32 { return int32(bits.LeadingZeros32(uint32(x))) }
|
|
func i32_ctz(x int32) int32 { return int32(bits.TrailingZeros32(uint32(x))) }
|
|
func i32_popcnt(x int32) int32 { return int32(bits.OnesCount32(uint32(x))) }
|
|
func i64_clz(x int64) int64 { return int64(bits.LeadingZeros64(uint64(x))) }
|
|
func i64_ctz(x int64) int64 { return int64(bits.TrailingZeros64(uint64(x))) }
|
|
func i64_popcnt(x int64) int64 { return int64(bits.OnesCount64(uint64(x))) }
|
|
|
|
func i32_rotl(x, y int32) int32 { return int32(bits.RotateLeft32(uint32(x), int(y&31))) }
|
|
func i32_rotr(x, y int32) int32 { return int32(bits.RotateLeft32(uint32(x), -int(y&31))) }
|
|
func i64_rotl(x, y int64) int64 { return int64(bits.RotateLeft64(uint64(x), int(y&63))) }
|
|
func i64_rotr(x, y int64) int64 { return int64(bits.RotateLeft64(uint64(x), -int(y&63))) }
|
|
|
|
//go:noinline
|
|
func wasm_trap_div_zero() { panic("wasm: integer divide by zero") }
|
|
|
|
//go:noinline
|
|
func wasm_trap_int_overflow() { panic("wasm: integer overflow") }
|
|
|
|
//go:noinline
|
|
func wasm_trap_invalid_conv() { panic("wasm: invalid conversion to integer") }
|
|
|
|
func i32_div_s(x, y int32) int32 {
|
|
if y == -1 && x == math.MinInt32 { panic("wasm: integer overflow") }
|
|
if y == 0 { panic("wasm: integer divide by zero") }
|
|
return x / y
|
|
}
|
|
func i32_div_u(x, y uint32) uint32 {
|
|
if y == 0 { panic("wasm: integer divide by zero") }
|
|
return x / y
|
|
}
|
|
func i32_div_u_s(x, y int32) int32 { return int32(i32_div_u(uint32(x), uint32(y))) }
|
|
func i32_rem_s(x, y int32) int32 {
|
|
if y == 0 { panic("wasm: integer divide by zero") }
|
|
if y == -1 { return 0 }
|
|
return x % y
|
|
}
|
|
func i32_rem_u(x, y uint32) uint32 {
|
|
if y == 0 { panic("wasm: integer divide by zero") }
|
|
return x % y
|
|
}
|
|
func i32_rem_u_s(x, y int32) int32 { return int32(i32_rem_u(uint32(x), uint32(y))) }
|
|
|
|
func i64_div_s(x, y int64) int64 {
|
|
if y == -1 && x == math.MinInt64 { panic("wasm: integer overflow") }
|
|
if y == 0 { panic("wasm: integer divide by zero") }
|
|
return x / y
|
|
}
|
|
func i64_div_u(x, y uint64) uint64 {
|
|
if y == 0 { panic("wasm: integer divide by zero") }
|
|
return x / y
|
|
}
|
|
func i64_div_u_s(x, y int64) int64 { return int64(i64_div_u(uint64(x), uint64(y))) }
|
|
func i64_rem_s(x, y int64) int64 {
|
|
if y == 0 { panic("wasm: integer divide by zero") }
|
|
if y == -1 { return 0 }
|
|
return x % y
|
|
}
|
|
func i64_rem_u(x, y uint64) uint64 {
|
|
if y == 0 { panic("wasm: integer divide by zero") }
|
|
return x % y
|
|
}
|
|
func i64_rem_u_s(x, y int64) int64 { return int64(i64_rem_u(uint64(x), uint64(y))) }
|
|
|
|
func i32_wrap_i64(x int64) int32 { return int32(x) }
|
|
func i64_extend_i32_s(x int32) int64 { return int64(x) }
|
|
func i64_extend_i32_u(x int32) int64 { return int64(uint32(x)) }
|
|
func i32_extend8_s(x int32) int32 { return int32(int8(x)) }
|
|
func i32_extend16_s(x int32) int32 { return int32(int16(x)) }
|
|
func i64_extend8_s(x int64) int64 { return int64(int8(x)) }
|
|
func i64_extend16_s(x int64) int64 { return int64(int16(x)) }
|
|
func i64_extend32_s(x int64) int64 { return int64(int32(x)) }
|
|
`
|
|
}
|
|
|
|
// TestEmitAddAMD64 keeps the original one-function add fixture as a
|
|
// fast smoke test. It compiles in ~1s and runs in ~10ms, making it
|
|
// useful as the first thing to fail when emitter scaffolding breaks.
|
|
func TestEmitAddAMD64(t *testing.T) {
|
|
if runtime.GOARCH != "amd64" {
|
|
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
|
|
}
|
|
bin := testfixture.Wasm(t, "arith")
|
|
mod, err := wasm.Parse(bytes.NewReader(bin))
|
|
if err != nil {
|
|
t.Fatalf("parse: %v", err)
|
|
}
|
|
addIdx := ^uint32(0)
|
|
for _, e := range mod.Exports {
|
|
if e.Kind == wasm.ExportFunc && e.Name == "add" {
|
|
addIdx = e.Index
|
|
}
|
|
}
|
|
if addIdx == ^uint32(0) {
|
|
t.Fatal("missing add export")
|
|
}
|
|
fn, err := lower.LowerFunction(mod, addIdx, "add", nil)
|
|
if err != nil {
|
|
t.Fatalf("lower: %v", err)
|
|
}
|
|
sig := mod.FuncTypeOf(addIdx)
|
|
asm, decl, err := EmitFuncAMD64("add", sig, fn, FuncOptions{ModulePkgRef: "*Module"})
|
|
if err != nil {
|
|
t.Fatalf("emit: %v", err)
|
|
}
|
|
|
|
dir := t.TempDir()
|
|
mustWrite(t, dir, "go.mod", "module asmgentest\n\ngo 1.25\n")
|
|
mustWrite(t, dir, "module.go", driverPlaceholder.moduleSrc)
|
|
mustWrite(t, dir, "add.go", "//go:build amd64\n\npackage main\n\n"+decl)
|
|
mustWrite(t, dir, "add_amd64.s", "//go:build amd64\n\n#include \"textflag.h\"\n#include \"funcdata.h\"\n\n"+asm)
|
|
mustWrite(t, dir, "main.go", `package main
|
|
|
|
import (
|
|
"fmt"
|
|
"os"
|
|
"strconv"
|
|
)
|
|
|
|
func main() {
|
|
a, _ := strconv.ParseInt(os.Args[1], 10, 32)
|
|
b, _ := strconv.ParseInt(os.Args[2], 10, 32)
|
|
fmt.Println(add(nil, int32(a), int32(b)))
|
|
}
|
|
`)
|
|
cmd := exec.Command("go", "run", ".", "2", "3")
|
|
cmd.Dir = dir
|
|
out, err := cmd.CombinedOutput()
|
|
if err != nil {
|
|
t.Fatalf("go run: %v\n%s", err, out)
|
|
}
|
|
if strings.TrimSpace(string(out)) != "5" {
|
|
t.Errorf("got %q want 5", out)
|
|
}
|
|
}
|
|
|
|
// TestEmitEmptyElseFallthroughAMD64 — covers the
|
|
// "empty else block elision" optimisation surfaced by the Pure-Go
|
|
// instruction-count comparison on Fn31214. The wasm-to-SSA lowering
|
|
// of `if x { call }` (no else clause) produces a 4-block diamond:
|
|
//
|
|
// b_test (BlockIf, control=x) -> [b_then, b_else]
|
|
// b_then (BlockPlain) -> [b_merge]
|
|
// b_else (BlockPlain, empty) -> [b_merge]
|
|
// b_merge ... continuation
|
|
//
|
|
// Before the empty-else elision the asm shape was
|
|
//
|
|
// TESTL AX, AX
|
|
// JNE L_then
|
|
// JMP L_else <-- routes through the empty block
|
|
// L_then: ... ; JMP L_merge
|
|
// L_else: <-- empty pass-through, dead label
|
|
// L_merge: ...
|
|
//
|
|
// After the empty-else elision:
|
|
//
|
|
// - passthroughTarget collapses L_else → L_merge, so the
|
|
// BlockIf names L_merge directly as the else target.
|
|
// - emitBlock supplies the next block's label as a
|
|
// fall-through hint.
|
|
// - When the fall-through equals the then label, the arch
|
|
// emitter INVERTS the JCC and drops the JMP.
|
|
//
|
|
// Resulting amd64 shape (the load-bearing change):
|
|
//
|
|
// TESTL AX, AX
|
|
// JE L_merge <-- single conditional, inverted
|
|
// L_then: ... ; JMP L_merge
|
|
// L_else: <-- still emitted as a dead label
|
|
// L_merge: ...
|
|
//
|
|
// That collapses three terminator instructions (JNE/JMP/JMP) into
|
|
// one (JE), the saving the comparison row showed.
|
|
//
|
|
// We assert the load-bearing properties directly on the emitted asm
|
|
// string — no full assemble/link/run round-trip is needed, the
|
|
// driver tests already cover correctness of generic BlockIf.
|
|
func TestEmitEmptyElseFallthroughAMD64(t *testing.T) {
|
|
if runtime.GOARCH != "amd64" {
|
|
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
|
|
}
|
|
// Build SSA for: func(x i32) i32 { if x != 0 { /* no body, just
|
|
// pretend something happened by adding x+x — kept off the merge
|
|
// path */ }; return x }
|
|
//
|
|
// We don't need an actual call to exercise the optimisation; the
|
|
// load-bearing part is the 4-block diamond shape with an empty
|
|
// else. A no-op then is enough to make the pass-through pattern
|
|
// observable. We make the then block non-empty (one Add) so it
|
|
// is NOT itself passthrough-eligible, and leave the else block
|
|
// genuinely empty.
|
|
sig := wasm.FuncType{Params: []wasm.ValType{wasm.ValI32}, Results: []wasm.ValType{wasm.ValI32}}
|
|
fsig := ssa.FuncSig{Params: []ssa.Type{ssa.TypeI32}, Results: []ssa.Type{ssa.TypeI32}}
|
|
bb := ssa.NewFuncBuilder("emptyelse", fsig)
|
|
|
|
bTest := bb.NewBlock(ssa.BlockIf)
|
|
bThen := bb.NewBlock(ssa.BlockPlain)
|
|
bElse := bb.NewBlock(ssa.BlockPlain)
|
|
bMerge := bb.NewBlock(ssa.BlockRet)
|
|
bb.SetEntry(bTest)
|
|
|
|
bb.SetCurrent(bTest)
|
|
x := bb.Param(0, ssa.TypeI32)
|
|
bb.LinkIf(x, bThen, bElse)
|
|
|
|
bb.SetCurrent(bThen)
|
|
// One real value so the then block is NOT passthrough-eligible.
|
|
_ = bb.NewValue(ssa.OpAdd32, ssa.TypeI32, x, x)
|
|
bb.LinkPlain(bMerge)
|
|
|
|
bb.SetCurrent(bElse)
|
|
// Empty — just the BlockPlain jump (added automatically).
|
|
bb.LinkPlain(bMerge)
|
|
|
|
bb.SetCurrent(bMerge)
|
|
bb.FinishRet(x)
|
|
|
|
if err := ssa.Verify(bb.Func()); err != nil {
|
|
t.Fatalf("SSA verify: %v", err)
|
|
}
|
|
|
|
asm, _, err := EmitFuncAMD64("emptyelse", sig, bb.Func(), FuncOptions{ModulePkgRef: "*Module"})
|
|
if err != nil {
|
|
t.Fatalf("EmitFuncAMD64: %v", err)
|
|
}
|
|
|
|
// Property 1 — the empty else's label is no longer referenced by
|
|
// any JMP/Jcc instruction. After passthroughTarget redirection the
|
|
// only references should be its own definition line (`L<n>:`) and
|
|
// nothing else. We approximate this with: the block emitted
|
|
// between then and merge MUST have label `L<bElse>:`, and there
|
|
// MUST NOT be any `JMP L<bElse>` or `JCC L<bElse>` line.
|
|
elseLabel := fmt.Sprintf("L%d", bElse.ID)
|
|
for _, line := range strings.Split(asm, "\n") {
|
|
trim := strings.TrimSpace(line)
|
|
if !strings.HasPrefix(trim, "J") {
|
|
continue
|
|
}
|
|
// Match "<MNEMONIC> L<n>" — split on whitespace.
|
|
parts := strings.Fields(trim)
|
|
if len(parts) < 2 {
|
|
continue
|
|
}
|
|
if parts[len(parts)-1] == elseLabel {
|
|
t.Errorf("expected the empty-else label %q to have NO inbound branches "+
|
|
"after passthrough redirection, but found:\n %s\nfull asm:\n%s",
|
|
elseLabel, line, asm)
|
|
}
|
|
}
|
|
|
|
// Property 2 — the BlockIf terminator emits exactly ONE
|
|
// conditional jump (the inverted form, JE) and NO trailing JMP.
|
|
// The inverted condition routes the "x == 0" case to the merge
|
|
// label directly; the "x != 0" case falls through to bThen.
|
|
// We assert this by counting the J* instructions in the prologue
|
|
// segment that PRECEDES the first body label.
|
|
mergeLabel := fmt.Sprintf("L%d", bMerge.ID)
|
|
// Find the position of bThen's label — everything before is the
|
|
// BlockIf prologue (label + TESTL + branch(es)).
|
|
thenLabel := fmt.Sprintf("L%d:", bThen.ID)
|
|
thenIdx := strings.Index(asm, "\n"+thenLabel)
|
|
if thenIdx < 0 {
|
|
t.Fatalf("then-block label %q not found in asm:\n%s", thenLabel, asm)
|
|
}
|
|
prologue := asm[:thenIdx]
|
|
branchLines := 0
|
|
hasInvertedJE := false
|
|
for _, line := range strings.Split(prologue, "\n") {
|
|
trim := strings.TrimSpace(line)
|
|
switch {
|
|
case strings.HasPrefix(trim, "JMP "):
|
|
branchLines++
|
|
case strings.HasPrefix(trim, "J") && len(trim) >= 2 && trim[1] >= 'A' && trim[1] <= 'Z':
|
|
// Any conditional jump (JE, JNE, JLT, JGE, ...).
|
|
branchLines++
|
|
if strings.HasPrefix(trim, "JE ") && strings.HasSuffix(trim, " "+mergeLabel) {
|
|
hasInvertedJE = true
|
|
}
|
|
}
|
|
}
|
|
if branchLines != 1 {
|
|
t.Errorf("expected exactly 1 branch in the BlockIf prologue "+
|
|
"(the inverted JE to merge — JMP dropped via fall-through), "+
|
|
"got %d in:\n%s", branchLines, prologue)
|
|
}
|
|
if !hasInvertedJE {
|
|
t.Errorf("expected an inverted `JE %s` in the BlockIf prologue "+
|
|
"(the original cond is `x != 0`, JE routes the `x == 0` case to the merge "+
|
|
"directly while `x != 0` falls through to the then block), got:\n%s",
|
|
mergeLabel, prologue)
|
|
}
|
|
}
|
|
|
|
// TestBitTestFusionAMD64 — locks in the
|
|
// "(x & 1<<k) <eq/ne> 0 → BTL + JCC" fusion. Pre-fusion the lowering
|
|
// emitted six instructions for `x & 1 != 0` (load x, AND $1, store
|
|
// AND result, reload, TEST, JNE/JMP); Pure-Go gets the same effect
|
|
// in two (`BTL $0, x ; JCS label`). The fusion detects the chain
|
|
// in Pass 4's branch-fusion analysis, skips emission for the OpAnd
|
|
// (it lives only as a flag producer), and lowers the BlockIf to
|
|
// the BTL + JCC pair.
|
|
//
|
|
// We assert three properties:
|
|
// - the emitted asm CONTAINS `BTL $0, AX` (the single-bit test),
|
|
// - it does NOT contain `ANDL $1` (the OpAnd's separate emit),
|
|
// - it CONTAINS exactly one conditional branch in the BlockIf
|
|
// prologue (no SETcc + MOVBLZX + TEST chain).
|
|
func TestBitTestFusionAMD64(t *testing.T) {
|
|
if runtime.GOARCH != "amd64" {
|
|
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
|
|
}
|
|
sig := wasm.FuncType{Params: []wasm.ValType{wasm.ValI32}, Results: []wasm.ValType{wasm.ValI32}}
|
|
fsig := ssa.FuncSig{Params: []ssa.Type{ssa.TypeI32}, Results: []ssa.Type{ssa.TypeI32}}
|
|
bb := ssa.NewFuncBuilder("bittest", fsig)
|
|
bIf := bb.NewBlock(ssa.BlockIf)
|
|
bThen := bb.NewBlock(ssa.BlockRet)
|
|
bElse := bb.NewBlock(ssa.BlockRet)
|
|
bb.SetEntry(bIf)
|
|
|
|
bb.SetCurrent(bIf)
|
|
x := bb.Param(0, ssa.TypeI32)
|
|
one := bb.Const32(1)
|
|
andRes := bb.NewValue(ssa.OpAnd32, ssa.TypeI32, x, one)
|
|
zero := bb.Const32(0)
|
|
cond := bb.NewValue(ssa.OpNe32, ssa.TypeBool, andRes, zero)
|
|
bb.LinkIf(cond, bThen, bElse)
|
|
|
|
bb.SetCurrent(bThen)
|
|
bb.FinishRet(x)
|
|
bb.SetCurrent(bElse)
|
|
bb.FinishRet(x)
|
|
|
|
if err := ssa.Verify(bb.Func()); err != nil {
|
|
t.Fatalf("SSA verify: %v", err)
|
|
}
|
|
asm, _, err := EmitFuncAMD64("bittest", sig, bb.Func(), FuncOptions{ModulePkgRef: "*Module"})
|
|
if err != nil {
|
|
t.Fatalf("EmitFuncAMD64: %v", err)
|
|
}
|
|
|
|
if !strings.Contains(asm, "BTL $0, AX") {
|
|
t.Errorf("expected `BTL $0, AX` in emitted asm — the bit-test "+
|
|
"fusion did not fire. asm:\n%s", asm)
|
|
}
|
|
if strings.Contains(asm, "ANDL $1") {
|
|
t.Errorf("expected the OpAnd32 to be elided by branchFusedSkip, "+
|
|
"but found `ANDL $1` in the asm:\n%s", asm)
|
|
}
|
|
// The bit-test path emits exactly one Jcc (JCS or JCC, possibly
|
|
// with an `JMP <else>` if no fall-through applies). We don't
|
|
// assert the precise count here — the fall-through
|
|
// optimisation may drop one of them.
|
|
}
|
|
|
|
// TestLoadConstDispFoldAMD64 — locks in the
|
|
// "non-negative constant offset folds into addressing mode" fix
|
|
// for OpLoad's dynamic-base path. Pre-15d the lowering emitted
|
|
//
|
|
// MOVL <base>, SI
|
|
// ADDL $<off>, SI <-- redundant
|
|
// MOVL (BX)(SI*1), AX
|
|
//
|
|
// for `i32.load offset=<off>`. Pure-Go's equivalent collapses to
|
|
// a single `MOVL <off>(BX)(SI*1), AX` because amd64's addressing
|
|
// mode happily takes a signed displacement. The disp-fold pass
|
|
// does the same when off > 0 (sign-extension matches
|
|
// zero-extension for non-negative values, preserving wasm's
|
|
// `uint32(base+off)` effective-address semantics).
|
|
//
|
|
// We assert two properties on a tiny SSA function that performs
|
|
// `i32.load offset=24` on a dynamic base:
|
|
// - the asm MUST NOT contain `ADDL $24, SI` (the fold
|
|
// eliminated it);
|
|
// - the asm MUST contain `MOVL 24(BX)(SI*1)` (the load picks
|
|
// up the displacement from the addressing mode).
|
|
//
|
|
// The off=0 path is exercised by other load tests; the off<0
|
|
// (wrap-required) path is exercised by the existing memop tests.
|
|
func TestLoadConstDispFoldAMD64(t *testing.T) {
|
|
if runtime.GOARCH != "amd64" {
|
|
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
|
|
}
|
|
sig := wasm.FuncType{Params: []wasm.ValType{wasm.ValI32}, Results: []wasm.ValType{wasm.ValI32}}
|
|
fsig := ssa.FuncSig{Params: []ssa.Type{ssa.TypeI32}, Results: []ssa.Type{ssa.TypeI32}}
|
|
bb := ssa.NewFuncBuilder("loadfold", fsig)
|
|
b0 := bb.NewBlock(ssa.BlockRet)
|
|
bb.SetEntry(b0)
|
|
bb.SetCurrent(b0)
|
|
base := bb.Param(0, ssa.TypeI32)
|
|
loaded := bb.NewValueAuxInt(ssa.OpLoad32, ssa.TypeI32, 24, base)
|
|
bb.FinishRet(loaded)
|
|
if err := ssa.Verify(bb.Func()); err != nil {
|
|
t.Fatalf("SSA verify: %v", err)
|
|
}
|
|
asm, _, err := EmitFuncAMD64("loadfold", sig, bb.Func(), FuncOptions{ModulePkgRef: "*Module"})
|
|
if err != nil {
|
|
t.Fatalf("EmitFuncAMD64: %v", err)
|
|
}
|
|
if strings.Contains(asm, "ADDL $24, SI") {
|
|
t.Errorf("expected the constant offset 24 to be folded into the "+
|
|
"addressing-mode displacement, not emitted as `ADDL $24, SI`. asm:\n%s", asm)
|
|
}
|
|
if !strings.Contains(asm, "24(BX)(SI*1)") {
|
|
t.Errorf("expected the load to use `24(BX)(SI*1)` addressing, "+
|
|
"but the displacement was not folded. asm:\n%s", asm)
|
|
}
|
|
}
|
|
|
|
// TestOpParamNoSlotStoreAMD64 — locks in the
|
|
// "OpParam producer-side store-to-slot is dead" property.
|
|
//
|
|
// operandSrc{32,64,Float} on amd64 resolves OpParam to `lN+off(FP)`
|
|
// directly, bypassing the slot. The matching producer-side emit
|
|
// (emitParam) used to write the param value to its slot as a
|
|
// no-op pair `MOVL lN+off(FP), AX; MOVL AX, dst(SP)`. Pure-Go drops
|
|
// the equivalent; marking OpParam as SkipValue brings us into
|
|
// line by excluding it from the emit loop entirely.
|
|
//
|
|
// Risk: slot reuse might allocate OpParam's slot to a later
|
|
// value. If the OpParam store survived, the later value's store
|
|
// would overwrite it before any read (the slot has no read site),
|
|
// so removing the OpParam store is safe.
|
|
//
|
|
// We assert two properties on a tiny SSA function: f(x int32) int32
|
|
// { return x + x }. The function has a single OpParam(x) with two
|
|
// users (the OpAdd32). The emitted asm MUST contain
|
|
// - exactly zero `MOVL l0+8(FP), <reg>; MOVL <reg>, <slot>(SP)`
|
|
// producer pairs sourced from `l0+8(FP)` and destined for a slot
|
|
// (other than the OpAdd32's own result store);
|
|
// - at least one `MOVL l0+8(FP), <reg>` read (the OpAdd32 emit
|
|
// loading x from FP).
|
|
//
|
|
// The first property is the load-bearing one. We approximate it by
|
|
// counting `MOVL l0+8(FP), AX` / `MOVL AX, NN(SP)` adjacent pairs:
|
|
// the OpAdd32 emit produces a single load-from-FP-into-AX followed
|
|
// by an ADDL-with-the-other-arg, not by a MOVL-to-slot. Any MOVL-
|
|
// to-slot directly after the load IS the (removed) OpParam spill.
|
|
func TestOpParamNoSlotStoreAMD64(t *testing.T) {
|
|
if runtime.GOARCH != "amd64" {
|
|
t.Skipf("amd64-only test (GOARCH=%s)", runtime.GOARCH)
|
|
}
|
|
sig := wasm.FuncType{Params: []wasm.ValType{wasm.ValI32}, Results: []wasm.ValType{wasm.ValI32}}
|
|
fsig := ssa.FuncSig{Params: []ssa.Type{ssa.TypeI32}, Results: []ssa.Type{ssa.TypeI32}}
|
|
bb := ssa.NewFuncBuilder("paramskip", fsig)
|
|
b0 := bb.NewBlock(ssa.BlockRet)
|
|
bb.SetEntry(b0)
|
|
bb.SetCurrent(b0)
|
|
x := bb.Param(0, ssa.TypeI32)
|
|
sum := bb.NewValue(ssa.OpAdd32, ssa.TypeI32, x, x)
|
|
bb.FinishRet(sum)
|
|
|
|
if err := ssa.Verify(bb.Func()); err != nil {
|
|
t.Fatalf("SSA verify: %v", err)
|
|
}
|
|
asm, _, err := EmitFuncAMD64("paramskip", sig, bb.Func(), FuncOptions{ModulePkgRef: "*Module"})
|
|
if err != nil {
|
|
t.Fatalf("EmitFuncAMD64: %v", err)
|
|
}
|
|
|
|
lines := strings.Split(asm, "\n")
|
|
for i, line := range lines[:len(lines)-1] {
|
|
trim := strings.TrimSpace(line)
|
|
// Match the load half of an OpParam dead spill:
|
|
// `MOVL l0+8(FP), <reg>`.
|
|
if !strings.HasPrefix(trim, "MOVL l") || !strings.Contains(trim, "(FP), ") {
|
|
continue
|
|
}
|
|
// Extract the register name: ", <REG>" suffix.
|
|
parts := strings.Split(trim, ", ")
|
|
if len(parts) != 2 {
|
|
continue
|
|
}
|
|
reg := parts[1]
|
|
// Look at the next line. The OpParam dead spill is
|
|
// `MOVL <reg>, <off>(SP)` — a store to an SP-relative slot
|
|
// using exactly the register we just loaded into.
|
|
next := strings.TrimSpace(lines[i+1])
|
|
want := "MOVL " + reg + ", "
|
|
if strings.HasPrefix(next, want) && strings.Contains(next, "(SP)") {
|
|
t.Errorf("expected OpParam to emit no producer-side spill, "+
|
|
"but found a dead `load from FP; store to SP` pair at lines %d-%d:\n %s\n %s\nfull asm:\n%s",
|
|
i, i+1, trim, next, asm)
|
|
break
|
|
}
|
|
}
|
|
}
|
|
|
|
// TestComputeUnusedResults — unit-tests the
|
|
// liveness analysis that decides which CALL sites can drop their
|
|
// post-CALL result load + slot store. Pure-Go's frontend would do
|
|
// this via classic dead-code elimination on the SSA; we get the
|
|
// same effect by walking liveness backward from the sinks
|
|
// (side-effecting ops, BlockRet outputs, block.Control values).
|
|
//
|
|
// The analysis is "transitively live": a CALL value is live as a
|
|
// SINK (its side effects matter), but its RESULT counts as used
|
|
// only if a LIVE consumer chains back to it. A consumer that is
|
|
// itself dead (its result reaches no sink) doesn't count — that's
|
|
// the case for "v_call → OpCopy → OpAdd32 (result unused)", which
|
|
// IS elidable even though the OpCopy/OpAdd nominally reference
|
|
// v_call.
|
|
//
|
|
// Cases:
|
|
// - OpCallDirect whose result is never referenced by any other
|
|
// value's Args, no block.Control, and not a BlockRet output →
|
|
// unusedResult MUST be set.
|
|
// - OpCallDirect whose result IS the function's return value
|
|
// (referenced by BlockRet) → unusedResult MUST NOT be set.
|
|
// - OpHelperCall consumed via OpCopy → OpAdd32 with NO live
|
|
// downstream consumer → unusedResult MUST be set (the whole
|
|
// chain is dead).
|
|
// - OpHelperCall used as block.Control → unusedResult MUST NOT
|
|
// fire (control reads it as a sink).
|
|
// - OpHelperCall whose chain DOES reach a sink (helper → add →
|
|
// BlockRet) → unusedResult MUST NOT fire.
|
|
//
|
|
// We invoke computeUnusedResults directly with a fresh plan so the
|
|
// test stays focused on the analysis. planFunc's heavier setup
|
|
// (which resolves call symbols, computes the callee frame, etc.)
|
|
// would obscure the property we want to lock in.
|
|
func TestComputeUnusedResults(t *testing.T) {
|
|
sig := wasm.FuncType{Params: []wasm.ValType{wasm.ValI32}, Results: []wasm.ValType{wasm.ValI32}}
|
|
fsig := ssa.FuncSig{Params: []ssa.Type{ssa.TypeI32}, Results: []ssa.Type{ssa.TypeI32}}
|
|
bb := ssa.NewFuncBuilder("unusedret", fsig)
|
|
|
|
// b0: the work; b1 (BlockIf, control=helperCond): then→b2 else→b2;
|
|
// b2: BlockRet of liveCall's result.
|
|
b0 := bb.NewBlock(ssa.BlockPlain)
|
|
b1 := bb.NewBlock(ssa.BlockIf)
|
|
b2 := bb.NewBlock(ssa.BlockRet)
|
|
bb.SetEntry(b0)
|
|
|
|
bb.SetCurrent(b0)
|
|
l0 := bb.Param(0, ssa.TypeI32)
|
|
// (a) OpCallDirect with discarded result.
|
|
unusedCall := bb.NewValueAuxInt(ssa.OpCallDirect, ssa.TypeI32, 0, l0)
|
|
// (b) OpHelperCall consumed via OpCopy → OpAdd32 → unused. The
|
|
// whole chain is dead (sum is never referenced by any sink) so
|
|
// helperDeadChain's result is logically unused.
|
|
helperDeadChain := bb.NewValueAuxInt(ssa.OpHelperCall, ssa.TypeI32, 0, l0)
|
|
copyAlias := bb.NewValue(ssa.OpCopy, ssa.TypeI32, helperDeadChain)
|
|
deadSum := bb.NewValue(ssa.OpAdd32, ssa.TypeI32, copyAlias, l0)
|
|
_ = deadSum
|
|
// (c) OpHelperCall whose chain DOES reach the function output
|
|
// (helper → add → BlockRet). Result is genuinely used.
|
|
helperLiveChain := bb.NewValueAuxInt(ssa.OpHelperCall, ssa.TypeI32, 0, l0)
|
|
liveSum := bb.NewValue(ssa.OpAdd32, ssa.TypeI32, helperLiveChain, l0)
|
|
// (d) OpHelperCall used as the BlockIf control — the control
|
|
// edge is a sink, so it counts as a live consumer.
|
|
condHelper := bb.NewValueAuxInt(ssa.OpHelperCall, ssa.TypeBool, 0, l0)
|
|
bb.LinkPlain(b1)
|
|
|
|
bb.SetCurrent(b1)
|
|
bb.LinkIf(condHelper, b2, b2)
|
|
|
|
bb.SetCurrent(b2)
|
|
bb.FinishRet(liveSum)
|
|
|
|
if err := ssa.Verify(bb.Func()); err != nil {
|
|
t.Fatalf("SSA verify: %v", err)
|
|
}
|
|
|
|
plan := &funcPlan{unusedResult: map[ssa.ValueID]bool{}}
|
|
computeUnusedResults(bb.Func(), plan, sig)
|
|
|
|
cases := []struct {
|
|
v *ssa.Value
|
|
wantUnused bool
|
|
why string
|
|
}{
|
|
{unusedCall, true, "OpCallDirect with no downstream consumer should be elidable"},
|
|
{helperDeadChain, true,
|
|
"OpHelperCall whose consumer chain is dead (sum has no live consumer) should be elidable — " +
|
|
"Pure-Go would DCE the whole chain"},
|
|
{helperLiveChain, false,
|
|
"OpHelperCall whose chain reaches BlockRet via liveSum is genuinely used"},
|
|
{condHelper, false, "OpHelperCall used as block.Control — used"},
|
|
}
|
|
for _, c := range cases {
|
|
got := plan.unusedResult[c.v.ID]
|
|
if got != c.wantUnused {
|
|
t.Errorf("unusedResult[v%d (%s)] = %v, want %v: %s",
|
|
c.v.ID, c.v.Op, got, c.wantUnused, c.why)
|
|
}
|
|
}
|
|
}
|