Files
wasm2go/internal/gcasm/crosschunk_pull_test.go
cfdd475d60 gcasm: reach remote fns from fallback bodies via trampolines, never linkname pulls; consumer-link gates (#65)
* gcasm: reach remote fns from fallback bodies via trampolines, never linkname pulls; link gates

Root cause of the unlinkable llamawasm2go v0.3.1 bundle ("nosplit stack
over 792 byte limit ... infinite cycle" on linux/arm64 and linux/amd64
GOAMD64=v2), latent since the direct cross-chunk calls (#61):

A chunk package whose pure-Go fallback bodies call a remote FnN
declared that FnN through a bodyless //go:linkname pull, while the
same package's transformed asm CALLed the remote through its spelled
dotted symbol. symabis reports the asm reference, the pull supplies a
declaration, and the compiler emits a DUPOK ABI0 wrapper under the
REMOTE symbol's name. The linker keeps whichever of that wrapper and
the remote's real TEXT it loads first; when the wrapper wins, the
remote's ABIInternal wrapper and the DUPOK ABI0 wrapper call each other
forever and the link fails. Small fixtures never had a fallback body
calling a remote fn, so the shape went unexercised; the llama bundle
has hundreds of such overlaps, and which of them fail is decided by
package load order.

Fix (the direct asm-to-asm CALLs are untouched — they are the hop-count
win and are harmless on their own):

- Remote fns referenced from fallback bodies on a plan-9-spellable
  owner path get a LOCAL bodyless decl implemented by a NOSPLIT $0
  tail-JMP trampoline to the spelled remote symbol, so the compiler
  binds the decl to this package's own asm and never declares the
  remote symbol. Unspellable owner paths keep the linkname pull (no
  asm reference can be spelled there). The trampoline design the
  bundle.go comments described was never implemented; it is now.
- abi0ArgBytes derives the trampolines' argument frame from the wasm
  signature (validated against real bundle TEXT frames).
- Build refuses to emit a package whose //go:linkname-pulled symbols
  intersect its spelled asm references (crossChunkPullConflicts), so
  the hazard is a generator error, never a downstream link failure.

Gates:

- cg_crosscall_fallbackcaller fixture (fallback caller + transformed
  sibling packed into one chunk, calling a recursive remote); shape
  tests for the trampoline / pull split; a consumer link test for
  linux/arm64, linux/amd64 v2 and v1 with the chunk packages imported
  in ascending AND descending order (the pre-fix generator fails the
  descending order exactly like the shipped bundle); gate4 executes
  the fixture's exports.
- make llama-bundle-link (+ CI job): transpile the pinned llama-wasm
  release wasm with llama-wasm's own options and link a consumer that
  reaches every export for the three targets. With this change the
  v0.3.1 llama.wasm links on all of them.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* ci: link the llama bundle consumer in both chunk import orders; serialize the capture builds

The consumer now imports the chunk packages explicitly in ascending
and descending order (see scripts/bundle-link-consumer.sh): the linker
resolves DUPOK symbols first-come, so a wrapper emitted under another
package's symbol breaks the link only when the emitting package loads
first, and a single import order passed on a defective bundle by
layout luck in a downstream CI run while the shipped v0.3.1 failed.

The llama-bundle-link job was terminated (exit 143) four minutes in:
the gcasm capture builds the chunk packages in parallel and exhausts
the runner. GOFLAGS=-p=1 and GOMEMLIMIT bound it.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* ci: trace memory/disk around the llama bundle link job

The job is terminated with exit 143 about four minutes after the SSA
phase, during the gcasm capture, with no message from the runner and
regardless of GOFLAGS=-p=1 / GOMEMLIMIT. Sample free/df/top-RSS every
ten seconds and print the tail (and any OOM kernel lines) in an
always-run step so the next termination carries its cause.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* ci: stream the llama bundle job's resource samples into the step log

The always() trace step never ran: after the exit-143 termination the
runner skipped every remaining step, post steps included, i.e. the
runner itself went away. Emit the free/df/top-RSS samples from a
background loop inside the transpile step so the streamed step log
keeps everything sampled before the runner dies, and print the
runner's cpu/RAM/disk at the start.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* ci: name the package the top-RSS compile is building in the resource samples

The samples show one gc compile process at 15 GiB on the 16 GiB
runner before it dies; record its -p import path so the offending
chunk package is identified.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* ci: build the llama bundle gate with the latest Go 1.25 patch, not go.mod's floor

The job was terminated (exit 143) on every run: go1.25.0's compiler
needs 12-15 GiB for the largest chunk package (p0) and takes the 16 GiB
runner down, with every later step skipped. The same package compiles
in 1.8 GiB with go1.26.2 (measured locally, cold cache, -p=1), and the
production bundle is built from golang:1.25 (latest patch) inside the
wasmify image, so pin the gate to the latest 1.25 patch. GOMEMLIMIT is
dropped (it did not help; the live heap is the problem); -p=1 stays.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

* gcasm: drop the llama-specific bundle gate from this repo

wasm2go is a general transpiler and carries no configuration tied to
one downstream project: the pinned goccy/llama-wasm release, its
transpile options, the Makefile target and the CI job that reproduced
its bundle do not belong here. The consumer-link check lives in the
repos that own that bundle (goccy/llama-wasm, goccy/llamawasm2go);
this repo keeps the project-neutral gates — the Build-time
pull/spelled-reference conflict error and the fixture link test in
both chunk import orders.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

---------

Co-authored-by: Claude Fable 5.1 <[email protected]>
2026-09-03 09:27:53 +09:00

180 lines
6.9 KiB
Go
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
package gcasm
import (
"os"
"os/exec"
"path/filepath"
"regexp"
"sort"
"strings"
"testing"
"github.com/goccy/wasm2go/internal/codegen"
)
// fallbackCallerChunkBytes packs the fixture's transformed sibling and
// its pure-Go fallback caller into ONE chunk package (the planner
// first-fit packs SCCs by size), so that package both references the
// remote callee from Go (the fallback body) and CALLs it from asm
// (the sibling's spelled direct call).
const fallbackCallerChunkBytes = 100
var (
plainPullRe = regexp.MustCompile(`(?m)^//go:linkname (Fn\d+) (\S+)$`)
trampRe = regexp.MustCompile(`(?m)^TEXT ·(Fn\d+)\(SB\), NOSPLIT, \$0-(\d+)\n\tJMP (\S+)·(Fn\d+)\(SB\)`)
)
// chunkDirsOf lists the pN chunk packages of a materialised tree.
func chunkDirsOf(tree map[string][]byte) []string {
seen := map[string]bool{}
for name := range tree {
if i := strings.IndexByte(name, '/'); i > 0 && strings.HasPrefix(name, "p") {
seen[name[:i]] = true
}
}
var dirs []string
for d := range seen {
dirs = append(dirs, d)
}
sort.Strings(dirs)
return dirs
}
// TestCrossChunkFallbackBodyCallsUseTrampolines: on a plan-9-spellable
// import path, a fallback body's Go-level reference to a remote FnN
// reaches it through a local NOSPLIT tail-JMP trampoline — never a
// bodyless //go:linkname pull of the remote symbol. A pull next to the
// same package's spelled asm CALLs makes the compiler emit a DUPOK
// ABI0 wrapper under the REMOTE name; whichever of that wrapper and
// the remote's real TEXT the linker loads first wins, and when the
// wrapper wins the two ABI wrappers call each other forever (the
// "nosplit stack over 792 byte limit ... infinite cycle" link error).
// The direct asm CALLs themselves stay: they are the hop-count win of
// the direct-call path and are harmless on their own.
func TestCrossChunkFallbackBodyCallsUseTrampolines(t *testing.T) {
const importPath = "github.com/gentest/pkg"
tree := finalBundleTreeAt(t, "cg_crosscall_fallbackcaller", importPath, fallbackCallerChunkBytes)
spelledRoot := asmSpellPath(importPath)
var directCalls, tramps int
for _, dir := range chunkDirsOf(tree) {
for _, arch := range []string{"amd64", "arm64"} {
decls := string(tree[dir+"/decls_"+arch+".go"])
asm := string(tree[dir+"/"+arch+".s"])
for _, m := range plainPullRe.FindAllStringSubmatch(decls, -1) {
t.Errorf("%s/decls_%s.go: bodyless //go:linkname pull of %s (%s) — asm-referenced remote fns must use a trampoline", dir, arch, m[2], m[1])
}
directCalls += strings.Count(asm, "CALL "+spelledRoot)
for _, m := range trampRe.FindAllStringSubmatch(asm, -1) {
tramps++
if m[1] != m[4] {
t.Errorf("%s/%s.s: trampoline %s jumps to %s", dir, arch, m[1], m[4])
}
if !strings.HasPrefix(m[3], spelledRoot+"∕") {
t.Errorf("%s/%s.s: trampoline %s target %q is not a spelled remote symbol", dir, arch, m[1], m[3])
}
// The local decl the trampoline implements must be a bare
// asm decl (no linkname) so the compiler binds it locally.
if !strings.Contains(decls, "\nfunc "+m[1]+"(") {
t.Errorf("%s/decls_%s.go: no local decl for trampoline %s", dir, arch, m[1])
}
}
}
}
if tramps == 0 {
t.Error("no cross-chunk trampoline emitted for the fallback body's remote call")
}
if directCalls == 0 {
t.Error("no spelled direct cross-chunk CALL emitted; the direct-call path must survive the trampoline change")
}
}
// An unspellable import path never spells asm references, so the
// bodyless linkname pull is safe there and stays (no trampoline can
// name the remote either).
func TestCrossChunkFallbackBodyCallsKeepPullOnUnspellablePath(t *testing.T) {
const importPath = "github.com/gen-test/pkg"
tree := finalBundleTreeAt(t, "cg_crosscall_fallbackcaller", importPath, fallbackCallerChunkBytes)
var pulls, tramps, spelled int
for _, dir := range chunkDirsOf(tree) {
for _, arch := range []string{"amd64", "arm64"} {
pulls += len(plainPullRe.FindAllString(string(tree[dir+"/decls_"+arch+".go"]), -1))
asm := string(tree[dir+"/"+arch+".s"])
tramps += len(trampRe.FindAllString(asm, -1))
spelled += strings.Count(asm, "∕")
}
}
if pulls == 0 {
t.Error("unspellable path emitted no //go:linkname pull for the fallback body's remote call")
}
if tramps != 0 || spelled != 0 {
t.Errorf("unspellable path emitted %d trampolines / %d spelled references; must be 0", tramps, spelled)
}
}
// TestCrossChunkBundleLinksInEveryPackageOrder links a consumer of the
// fixture bundle for both asm targets with the chunk packages imported
// in ascending AND descending order. The linker resolves DUPOK symbols
// first-come, so a bundle that is only correct for one package load
// order is not correct: the production consumer's import graph decides
// the order, not the bundle.
func TestCrossChunkBundleLinksInEveryPackageOrder(t *testing.T) {
const modPath = "github.com/gentest"
tree := finalBundleTreeAt(t, "cg_crosscall_fallbackcaller", modPath+"/pkg", fallbackCallerChunkBytes)
dirs := chunkDirsOf(tree)
if len(dirs) < 2 {
t.Fatalf("fixture did not split into multiple chunks: %v", dirs)
}
root := t.TempDir()
write := func(name string, data []byte) {
t.Helper()
p := filepath.Join(root, name)
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(p, data, 0o644); err != nil {
t.Fatal(err)
}
}
write("go.mod", []byte("module "+modPath+"\n\ngo 1.25.0\n"))
for name, data := range tree {
write("pkg/"+name, data)
}
orders := map[string][]string{"ascending": dirs}
rev := make([]string, len(dirs))
for i, d := range dirs {
rev[len(dirs)-1-i] = d
}
orders["descending"] = rev
targets := []struct {
name string
env []string
}{
{"linux/arm64", []string{"GOOS=linux", "GOARCH=arm64"}},
{"linux/amd64-v2", []string{"GOOS=linux", "GOARCH=amd64", "GOAMD64=v2"}},
{"linux/amd64-v1", []string{"GOOS=linux", "GOARCH=amd64", "GOAMD64=v1"}},
}
for orderName, order := range orders {
// The consumer must REACH the exports: the linker's dead-code
// pass drops unreferenced text before the nosplit walk, so a
// blank-import consumer would link even a poisoned bundle.
var main strings.Builder
main.WriteString("package main\n\nimport (\n\t\"fmt\"\n\n")
for _, d := range order {
main.WriteString("\t_ \"" + modPath + "/pkg/" + d + "\"\n")
}
main.WriteString("\t\"" + modPath + "/pkg\"\n)\n\nfunc main() {\n\tm := pkg.New()\n\tfmt.Println(pkg." +
codegen.ExportMethodName("fallbackcaller") + "(m, 1, 2))\n}\n")
write("cmd/"+orderName+"/main.go", []byte(main.String()))
for _, tg := range targets {
t.Run(orderName+"/"+tg.name, func(t *testing.T) {
cmd := exec.Command("go", "build", "-o", os.DevNull, "./cmd/"+orderName)
cmd.Dir = root
cmd.Env = append(append(os.Environ(), "CGO_ENABLED=0"), tg.env...)
if out, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("consumer does not link with chunks imported in %s order on %s: %v\n%s", orderName, tg.name, err, out)
}
})
}
}
}