Files
wasm2go/docs/asm-overrides.md
f57f53361e Replace the engine-specific body retargets with project-supplied assembly overrides (#67)
* codegen: keep fused-tree operands ahead of their consumers after the f16 gather rewrite

rewriteF16GatherGroup appends the collapsed v128_load16x4_u at the end
of the node list and points the chain's tail (now f16x4_cvt) at it, a
forward reference. The asm splicers walk the tree recursively and never
noticed, but the pure twin body emits nodes in list order, so any tree
that reached it through this rewrite declared the load after its use:
"undefined: n6 / declared and not used: n6" from the gcasm capture
build. It surfaced once ggml's wasm f16 vector loads started producing
the gather idiom everywhere (vec_dot_f16, vec_mad_f16, flash attention)
instead of only in the repack kernels.

compactDeadNodes, which every rewrite already runs, now places a node's
operands before the node while otherwise keeping list order — the
collapsed load lands where the chain's last lane load used to be.

* helpers: detect FEAT_I8MM on linux/arm64 (HWCAP2 bit 13)

The kernel-override dispatch and the SMMLA splices gate on it; darwin
already reported it through hw.optional.arm.FEAT_I8MM.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* Replace the engine-specific body retargets with project-supplied kernel overrides

wasm2go's job is to read a wasm module and emit Go and, where it can do
better than gc, assembly derived from that wasm. Two features had grown
past that line: the repack GEMM retarget swapped the body of an export
named dbg_gemm_q8_0_4x4 for a hand-written kernel that knows one
engine's quantized block layout, and the vec_dot row/column pairing
recognized one engine's trait-table layout and handed the paired
companion to another hand-written kernel of the same kind. Neither body
came from the wasm, and both tied the transpiler to that engine. On the
current engine build the pairing also no longer measures (prompt and
decode throughput within noise with it off), because the repack path
carries the hot matmuls.

Both are removed: the retarget and its -no-repack-gemm switch, the
pairing and its -vec-dot-pair-entry / -vec-dot-rows switches, the
kernels and the trait-table recognizer. The passive-segment placement
scan the f16-table detection shares moves to its own file.

In their place, a generic mechanism: -kernel-overrides <manifest>. The
project that produced the wasm may supply the assembly body of an
exported leaf function, per architecture and CPU feature level.
wasm2go validates the manifest against the module (export present,
signature and pointer width as declared), wraps each body in a fixed
ABI — the linear-memory base and size in named registers, arguments by
the ABI0 layout rule, a kov_oob label that reaches the module's
out-of-bounds trap — emits a feature-dispatch stub (i8mm > dotprod >
neon; avx512vnni > avx2 > sse4) with the lowered body as the portable
fallback, and pins the exports out of line. It never inspects what a
body computes; the body must stay inside the wasm execution model
(linear memory through the given base and size, bounds-checked, no
calls), and the manifest loader rejects TEXT, calls, system
instructions and foreign symbol references. docs/kernel-overrides.md
states the contract and the verification the project owes.

Tests: manifest validation, ABI text and dispatch shape, and an
end-to-end transpile of a module with and without an override body on
the host architecture — same sums, and the override traps past the end
of memory.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* helpers: detect FEAT_I8MM on darwin/arm64 too

The linux and fallback twins gained CPUI8MM; a darwin build of a bundle
whose kernel overrides declare an i8mm body needs the same symbol in
the base package.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* Rename kernel overrides to assembly overrides

"Kernel" is numeric-computing vocabulary; the mechanism itself knows
nothing about what the function computes — it replaces the body of an
exported function with project-supplied assembly. The name now says
that: -asm-overrides / Options.AsmOverrides, docs/asm-overrides.md,
the manifest's "functions" array, the ovr_oob label and ovr_ data
prefix, and Fn<idx>ovr<feature> body symbols.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-04 00:10:00 +09:00

6.0 KiB

Assembly overrides

A assembly override lets the project that produced a wasm module supply the assembly body of one of its exported functions. wasm2go wraps the body in a fixed ABI, keeps the lowered body as the portable fallback, and dispatches at runtime on the CPU features the body declares. The transpiler never learns what the function computes; it only checks that the manifest describes the module truthfully.

The mechanism exists for one situation: a leaf compute function whose wasm shape the transpiler cannot yet turn into competitive machine code. It is not a way to bypass the wasm execution model, and every override should be paired with the wasm2go issue that would make it unnecessary. When wasm2go can lower the function well from the wasm, delete the override.

What a body may do

The contract keeps a body inside the wasm execution model:

  • Linear memory only, through the prologue's registers. The body receives the memory base and the current memory size (bytes). Every access range must be checked against the size before the first load or store of that range, and a miss must branch to the ovr_oob label, which wasm2go appends: it calls the module's out-of-bounds trap (the same trap the lowered code raises). Nothing outside [base, base+size) may be read or written. Stack scratch inside the declared frame is fine.
  • No calls. No host imports, no other module functions, no Go runtime symbols. The manifest loader rejects CALL/BL/SVC/ SYSCALL and any ·symbol reference that is not the body's own ·ovr_-prefixed data.
  • Same meaning as the wasm function. Under the module's own semantics, or under the rounding the project has explicitly accepted (fused multiply-add, accumulation order). The project's tests must check the override against the lowered body.
  • A leaf with one entry and its own RET. wasm2go writes the TEXT header and the prologue; the body ends with RET (falling off the end reaches ovr_oob).

Manifest

{
  "version": 1,
  "memory64": true,
  "functions": [
    {
      "export": "my_dot",
      "params": ["i32", "i64", "i64"],
      "result": "f32",
      "bodies": [
        {"arch": "arm64", "feature": "dotprod", "frame": 16, "file": "my_dot_arm64_dotprod.s"},
        {"arch": "amd64", "feature": "avx2",    "frame": 16, "file": "my_dot_amd64_avx2.s"}
      ]
    }
  ]
}
  • memory64 must match the module (it decides the pointer width of the arguments).
  • export must name an exported, module-defined function whose signature is exactly params / result (result omitted or null when there is none). Value types: i32, i64, f32, f64.
  • arch is arm64 or amd64. feature is one of, most specific first: arm64 i8mm, dotprod, neon; amd64 avx512vnni, avx2, sse4. neon and sse4 are the architecture baselines (the same baselines the lowered asm assumes: NEON, x86-64-v2). A body listed for a feature runs only when the CPU reports it; the baseline body, when given, replaces the portable twin entirely; otherwise the lowered body remains the fallback.
  • frame is the body's stack frame in bytes (a multiple of 8).
  • file is the body, resolved relative to the manifest.

Pass the manifest with wasm2go -asm-overrides <path> (or Options.AsmOverrides). The exports it names are kept out of line by the inliner so every call reaches the replaced body.

ABI

wasm2go emits, per body:

TEXT ·<sym>(SB), $<frame>-<argBytes>
	NO_LOCAL_POINTERS
	<prologue>
	<body>
ovr_oob:
	[VZEROUPPER]           // amd64, non-sse4 bodies
	CALL ·<trap>(SB)
	RET

Prologue registers:

arch memory base memory size (bytes) module pointer (clobberable)
arm64 R20 R21 R0
amd64 R14 R15 AX

Every other register follows Go's ABI0 assembly rules (all caller saved; on arm64 do not touch R18, R28, R29, R30).

Arguments are read from the frame as l<i>+<offset>(FP) and the result written to r0+<offset>(FP). Offsets follow the ABI0 layout: the module pointer occupies bytes 0..8, then each parameter at its natural size and alignment (4 for i32/f32, 8 for i64/f64), then the result at the next 8-aligned offset. argBytes is the end of the last value with no trailing padding. For example (i32, i64, i64) -> f32 on memory64 is l0+8, l1+16, l2+24, r0+32, argBytes 36; the same export on a 32-bit memory is l0+8, l1+12, l2+16, r0+24, argBytes 28. Pointer-typed parameters are wasm addresses: add the memory base after checking the range.

wasm2go cross-checks the rule against the frame size the compiler derived for the lowered body and fails the transpile on any mismatch, so a body cannot be wired to a frame it did not expect.

Labels inside the body are function-local; the kov_ prefix is reserved for wasm2go. Constant data belongs to the body: DATA/GLOBL entries named ·ovr_<something> in the same file.

Dispatch

Under the function's own name wasm2go emits a frameless stub that branches on package-local mirrors of the CPU feature variables — one predicted branch per declared level, most specific first — and finally jumps to the baseline: the baseline override body if one is declared, else the lowered body transformed for the portable instruction set. Architectures without a body, and the pure-Go build, are untouched.

Verification the project owes

wasm2go validates structure, not meaning. The project that ships overrides is responsible for:

  • a numeric test of every body against a reference (and against the lowered body, so the override never drifts from the wasm), on every input length class the function handles (vector body, tails, zero);
  • running that test on hardware that has the declared features, or skipping the level explicitly when it cannot;
  • recording, per override, the wasm2go issue that tracks lowering the function from the wasm instead.