mirror of
https://github.com/hanzoai/wasm2go.git
synced 2026-10-30 15:06:48 +00:00
* codegen: compact memops via m.M cache + _consts table
Two related changes that together reduce per-access source size and
fix Linux ARM64 cross-compilation of very large transpiled modules,
without changing the runtime model or the public API.
1) m.M field cache.
Each memory load/store previously expanded to
*(*T)(unsafe.Add(unsafe.Pointer(unsafe.SliceData(m.memory)),
uintptr(uint32(<base>)+<offset>)))
re-fetching the slice header on every access. Pull the pointer
into a `M unsafe.Pointer` field on Module that New() initialises
and memoryGrow refreshes on the reallocate path. Reslice grows
leave the underlying data pointer alone, so the cache stays
valid across the common-case grow without any extra refresh.
Accesses now read
*(*T)(unsafe.Add(m.M, <off>))
The bare `<off>` form leans on unsafe.Add's IntegerType acceptance
(the spec performs the implicit uintptr conversion), elides the
uintptr() wrap, and unwraps the redundant int32() cast around
integer-constant bases. Negative-int32 constants keep the uint32
wrap so the bit pattern survives Go's sign-extension to uintptr.
2) Package-level _consts table.
Large constant offsets (>= 4 KiB) are routed through a per-file
var _consts = [N]uintptr{ ... }
slice and emitted as `_consts[<idx>]` at access sites. The
indirection prevents the Go ARM64 backend from folding the
constant into LDR/LDPSW immediates and falling back to a
PC-relative literal pool that the assembler then cannot reach
inside very large transpiled functions ("constant is not in
pool"). `_consts[N]` is a runtime memory load — the compiler
must use register-offset addressing, which has no immediate-
magnitude limit. Functions whose offsets stay below the
threshold continue to emit bare literals so the table stays
compact.
The table lives at file scope, one per chunk in the linkname-
split layout and one per output file in the single-package
layout (currentChunk == 0 for the single-package case).
Combined effect, measured against the googlesql @ 13.8 MB
wasm-opt'd wasm:
- Linux ARM64 `go build ./...` now succeeds where it previously
aborted with "constant is not in pool".
- Per-access generated source drops ~55 % (about 93 chars -> 40
chars in the constant-base / small-offset case).
- Peak compile memory matches the wazero baseline at ~5.4 GB
inside `golang:1.26 --memory=6g`.
- googlesqlite bench numbers stay within noise of the earlier
inline form on amd64; pprof shows the bridge layer is no
longer visible in either CPU or allocation top samples.
Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
* codegen: tighten error-handling lint and audit existing sites
Strengthens the golangci-lint config so the codebase can no longer
silently swallow errors, and walks the existing production code to
fix every site the stricter rules flag.
Lint changes:
- errcheck.check-blank: true catches `_, _ = f()` style explicit
suppressions in addition to fully-unchecked returns.
- errorlint enables errorf / asserts / comparison so `fmt.Errorf`
must wrap with %w and `err == io.EOF` becomes `errors.Is(err,
io.EOF)`.
- nilerr + nilnesserr cover the `return nil` when err is non-nil
family.
- No exclude-functions list — every Close/Flush/Fprintf is held
to the same standard as anything else.
Code changes follow the user's stated principle: if an error is a
precondition violation, panic; if it can legitimately occur, return
it to the caller (using errors.Join + named/local error on the
defer-close paths so the close failure cannot be lost).
Notable production-side fixes:
- `WasiStubs.Fd_close` and `Fd_renumber` now collect Close errors
from the f / conn / listener handles via a new closeWasiOpen
helper and surface them via mapOSError instead of dropping them.
- `Path_open`'s close-on-error paths join the Close error into
the primary statErr / not-dir error so neither failure is lost.
- `Sock_shutdown` joins CloseRead / CloseWrite errors into a
single mapped errno; the guest now sees a real failure if the
half-close fails.
- `resolveFiletimes` previously fell silently to epoch when
f.Stat failed; it now returns the Stat error so
Fd_filestat_set_times can map it to a wasi errno. The Stat
call is also skipped entirely when no preserve bits are set,
which is the only case where the current on-disk time matters.
- `translate_multi.go` chunk-name parsing replaced the Atoi
`err == nil` guard with a panic on inconsistency, matching the
`isChunkPkgName` precondition (the chunker's invariant
guarantees a numeric suffix; an Atoi failure here is an
emitter bug, not a runtime condition).
- `WASM2GO_MULTIPACKAGE_THRESHOLD` env-var parsing logs the
malformed value to stderr instead of silently using the default.
- `cmd/wasm2go/main.go` deferred Close/Flush calls now route a
failure through `fail()`; printSummary returns a single
accumulated write error via a small errWriter helper.
- `internal/wasm/leb128.go` and `parser.go` swap `err == io.EOF`
for `errors.Is(err, io.EOF)` so wrapped EOFs are detected.
Test-side changes are the same shape: `defer something.Close()` is
swapped for `t.Cleanup` with `t.Errorf` on failure, Read/Stat/
ReadFile in tests check err, and the captureStdout / captureStderr
helpers distinguish io.EOF from real read failures.
constInt32 itself is unchanged from the previous commit: it returns
(int32, bool) for the pattern-match classification, the caller
(`memOffsetExpr`) does an explicit early-return on `!ok`, and the
unreachable parse-failure inside a matched `int32(N)` panics as an
emitter-invariant assertion.
Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
170 lines
4.6 KiB
Go
170 lines
4.6 KiB
Go
package wasm
|
|
|
|
import (
|
|
"errors"
|
|
"fmt"
|
|
"io"
|
|
)
|
|
|
|
var (
|
|
errLEB128Overflow = errors.New("wasm: LEB128 integer overflow")
|
|
errLEB128Truncated = errors.New("wasm: LEB128 integer truncated")
|
|
)
|
|
|
|
type byteReader interface {
|
|
io.Reader
|
|
io.ByteReader
|
|
}
|
|
|
|
func readU32(r io.ByteReader) (uint32, error) {
|
|
v, err := readU64(r)
|
|
if err != nil {
|
|
return 0, err
|
|
}
|
|
if v > 0xffffffff {
|
|
return 0, errLEB128Overflow
|
|
}
|
|
return uint32(v), nil
|
|
}
|
|
|
|
func readU64(r io.ByteReader) (uint64, error) {
|
|
var result uint64
|
|
var shift uint
|
|
for i := 0; i < 10; i++ {
|
|
b, err := r.ReadByte()
|
|
if err != nil {
|
|
if errors.Is(err, io.EOF) {
|
|
return 0, errLEB128Truncated
|
|
}
|
|
return 0, err
|
|
}
|
|
if shift >= 64 {
|
|
return 0, errLEB128Overflow
|
|
}
|
|
result |= uint64(b&0x7f) << shift
|
|
if b&0x80 == 0 {
|
|
if i == 9 && b > 1 {
|
|
return 0, errLEB128Overflow
|
|
}
|
|
return result, nil
|
|
}
|
|
shift += 7
|
|
}
|
|
return 0, errLEB128Overflow
|
|
}
|
|
|
|
func readS32(r io.ByteReader) (int32, error) {
|
|
v, err := readS64Bits(r, 32)
|
|
if err != nil {
|
|
return 0, err
|
|
}
|
|
return int32(v), nil
|
|
}
|
|
|
|
func readS64(r io.ByteReader) (int64, error) {
|
|
return readS64Bits(r, 64)
|
|
}
|
|
|
|
// readS64Bits decodes a signed LEB128 of at most `bits` bits. The
|
|
// `bits` argument is the declared spec width (32 for an i32, 64 for
|
|
// an i64). Payload bits beyond bit (bits-1) must be a clean sign-
|
|
// extension of bit (bits-1) per "Integers: numerical formats" in the
|
|
// wasm binary spec; any other pattern is reported as overflow rather
|
|
// than silently truncated. The 10-byte iteration cap matches the
|
|
// maximum legal LEB128 length for a 64-bit value, so a malformed
|
|
// input can never loop us indefinitely.
|
|
//
|
|
// The bit-width validation must inspect the assembled `result` AFTER
|
|
// the current byte's 7-bit payload has been OR'd in, because the bit
|
|
// that determines the sign-extension (bit `bits-1`) may live inside
|
|
// the very byte we are about to validate. Checking before the OR
|
|
// rejects legitimate encodings such as `0xff 0xff 0xff 0xff 0x7f`
|
|
// (SLEB128 for -1 as i32) — bit 31 of the value is in this last byte.
|
|
func readS64Bits(r io.ByteReader, bits uint) (int64, error) {
|
|
var result int64
|
|
var shift uint
|
|
var b byte
|
|
for i := 0; i < 10; i++ {
|
|
var err error
|
|
b, err = r.ReadByte()
|
|
if err != nil {
|
|
if errors.Is(err, io.EOF) {
|
|
return 0, errLEB128Truncated
|
|
}
|
|
return 0, err
|
|
}
|
|
// OR this byte's payload into the assembled result first so
|
|
// the sign-extension check below sees the final value of bit
|
|
// (bits-1).
|
|
result |= int64(b&0x7f) << shift
|
|
// Once we have at least one bit's worth of payload beyond bit
|
|
// (bits-1) in this byte, every payload bit at-or-above bit
|
|
// (bits-1) must equal the sign-extension of the assembled
|
|
// value's high bit.
|
|
if shift+7 > bits {
|
|
// signByte = 0x00 for a non-negative value-so-far,
|
|
// 0x7f for a negative one.
|
|
signBit := (result >> (bits - 1)) & 1
|
|
var signByte byte
|
|
if signBit != 0 {
|
|
signByte = 0x7f
|
|
}
|
|
// mask = bits in this byte's payload at-or-above bit
|
|
// (bits-1) of the assembled value — those are the bits
|
|
// that must equal the sign-extension of the value's
|
|
// high bit. (Strictly: bit `bits-1` itself lies in this
|
|
// byte at position `bits-1-shift` and lower-position
|
|
// bits of the payload contribute to the value while
|
|
// higher-position bits must be a clean sign extension.
|
|
// The bit at position `bits-1-shift` belongs to the
|
|
// value, so legalInThisByte is `bits-shift`.)
|
|
legalInThisByte := uint(0)
|
|
if shift < bits {
|
|
legalInThisByte = bits - shift
|
|
}
|
|
mask := byte(0x7f) &^ byte((1<<legalInThisByte)-1)
|
|
if b&mask != signByte&mask {
|
|
return 0, errLEB128Overflow
|
|
}
|
|
}
|
|
shift += 7
|
|
if b&0x80 == 0 {
|
|
if shift < 64 && (b&0x40) != 0 {
|
|
result |= -1 << shift
|
|
}
|
|
return result, nil
|
|
}
|
|
}
|
|
return 0, errLEB128Overflow
|
|
}
|
|
|
|
// readByteVecCap is the safety cap on raw byte-vector lengths read
|
|
// from a wasm input. Matches the parser-side maxSectionPayload —
|
|
// nothing in any single section payload can plausibly exceed it,
|
|
// but a malformed LEB128 could otherwise demand a multi-gigabyte
|
|
// allocation before io.ReadFull truncates.
|
|
const readByteVecCap = 1 << 28
|
|
|
|
func readByteVec(r byteReader) ([]byte, error) {
|
|
n, err := readU32(r)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
if n > readByteVecCap {
|
|
return nil, fmt.Errorf("wasm: byte vector length %d exceeds %d-byte safety cap", n, readByteVecCap)
|
|
}
|
|
buf := make([]byte, n)
|
|
if _, err := io.ReadFull(r, buf); err != nil {
|
|
return nil, err
|
|
}
|
|
return buf, nil
|
|
}
|
|
|
|
func readName(r byteReader) (string, error) {
|
|
b, err := readByteVec(r)
|
|
if err != nil {
|
|
return "", err
|
|
}
|
|
return string(b), nil
|
|
}
|