mirror of
https://github.com/hanzoai/base.git
synced 2026-10-21 03:41:50 +00:00
zeekayandGitHub
99f11c3897
* feat(base): extension runtime interface + native impl Foundation for pluggable extension runtimes. One Runtime interface that wazero, v8go, goja, and native-Go all satisfy. Extensions declare their preferred runtime in extension.json; the loader dispatches. This sits alongside plugins/jsvm — goja-via-the-existing-jsvm-path remains the default for .base.js hooks. The new path is for users who want hard-sandboxed WASM extensions or full V8 perf. Files: - plugins/extruntime/extruntime.go — Runtime + Module interfaces, Manifest - plugins/extruntime/manifest.go — JSON manifest loader (stdlib only) - plugins/extruntime/native.go — reference impl, registers Go funcs Concrete runtimes land in sibling commits: - plugins/wasmvm — wazero (pure Go, hard sandbox) - plugins/v8vm — v8go (cgo, -tags v8vm) - plugins/gojavm — goja adapter (no new deps) - plugins/extbench — fixtures + benchmarks across all 5 * feat(base): goja runtime adapter + extension loader * feat(base): v8go extension runtime (cgo, -tags v8vm) V8-via-cgo runtime satisfying plugins/extruntime.Runtime. Build-tag gated so the default base binary never pulls v8go's ~50MB of cgo deps — a stub at !v8vm reports the same Name/Capabilities and errors at Load with ErrUnsupported. JS calling convention: payload bytes are JSON.parse'd into a real JS value and passed as the single argument to globalThis[fn]; the return value is JSON.stringify'd back to bytes. Much simpler than the wasm ptr/len dance — V8 has actual objects. Single shared Isolate across all modules (creation is expensive, isolates are not thread-safe so a global mutex serializes anyway). Per-module Context pool (default 8, BASE_V8VM_POOL_SIZE) for state isolation across back-to-back invocations. Heap cap intent via BASE_V8VM_HEAP_MB_MAX; v8go v0.9.0 has no public per-isolate heap API, so the value is advisory and enforcement is the V8 default plus ctx-cancel TerminateExecution() watchdog. Tests cover the cancel path (200ms timeout on `while(true)`), heap bomb (2^30-char string), pool reuse cap, wrong-runtime manifest, and closed-module rejection. * feat(base): wazero extension runtime Implements plugins/wasmvm — the wazero-backed extruntime.Runtime. - Shared wazero.Runtime with compilation cache + WASI snapshot preview1 instantiated once so guest modules can do stdio/clock/random. - Per-Module instance pool (default 8, BASE_WASMVM_POOL_SIZE override) to amortize InstantiateModule across invocations. - Host-guest ABI: guest exports fn(i32 ptr, i32 len) -> i64 (packed ptr<<32 | len), plus __base_alloc / __base_free for the buffer handshake. Missing alloc/free returns ErrUnsupported. - Hard abort on ctx cancel via api.Module.Close(); cancelled instances are replaced from a background ctx so pool size stays stable. - Tests cover identity, load+invoke, ctx cancel, concurrent pool reuse (16 invokes over pool size 4), invoke-after-close, and wrong-runtime manifest rejection. Fixture-dependent tests skip with TODO until extbench fixtures land. * feat(base): extension benchmark fixtures across 5 runtimes * feat(base): extension runtime benchmarks + writeup plugins/extbench/ benches all five extension runtimes (native, goja, wazero AS, wazero Javy, v8go) against an identical validate(email,age) workload. Per-runtime Serial + Parallel + Coldstart + Memory tests. run.sh runs the suite at -benchtime=2s -count=3 (~3 min total). Headline numbers (Apple M1 Max, Go 1.26.3, median of 3): native 1,197 ns/op serial / 694 ns/op parallel goja 4,513 ns/op serial / 1,652 ns/op parallel wazero AS 9,956 ns/op serial / 3,271 ns/op parallel v8go 11,895 ns/op serial / 12,667 ns/op parallel (no parallelism) Cold start ranges from native 17us to wazero AS 3.9ms. Binary size delta from +89 KB (native) to +47.8 MB (all four with -tags v8vm). Javy is wired but skipped pending a stdio-shim or custom Javy plugin (its WASI command ABI doesn't match wasmvm's pointer-based contract). Full writeup in docs/EXTENSIONS_BENCHMARK.md, raw runs in plugins/extbench/bench-results.txt. * feat(base): scale study — concurrent module / tenant / goroutine fanout Five-section fan-out study answering: 1) per-loaded-module marginal memory (N up to 10000) 2) per-tenant concurrent invocation latency (T up to 1000) + pool-size elbow 3) sustained concurrency on ONE module (M up to 10000) 4) goroutine cost (1000 parked goroutines, HeapInuse delta per runtime) 5) cold pool startup vs steady-state (N=100 sequential Load) Headlines: goja is the multi-tenant winner (~9KB/tenant, 188k ops/s at M=1000); wazero's per-module cost is the 8-instance pool — elbow at pool=4; v8go fatal-aborts above ~10 concurrent invokes (darwin/arm64 SIGSEGV inside libv8); native pays nothing per goroutine. Wazero pool=256 explodes RSS to 10.8 GB at 1000 modules without latency win. Implementation: cloneTenants() materializes a temp dir per tenant with a unique extension name, symlinked module artifact, and native-handler re-registration. v8go capped at concurrency=10 and skipped in goroutine-cost — the cap itself is the data, recorded in the writeup. Exports Validate from fixtures/native-go so scale_test can register the same handler under N unique extension names without re-implementing. Wall clock: <1 minute for the full study on Apple M1 Max. * feat(base): pyvm runtime — cgo CPython 3.13 (-tags pyvm) Fifth extension runtime alongside native/goja/wazero/v8go. Default build links the stub; real impl behind `-tags pyvm` requires pkg-config python-3.13-embed. Architecture: one process-wide CPython init, per-module pool of PEP 684 OWN_GIL sub-interpreters (default 4 / BASE_PYVM_POOL_SIZE). Each Invoke pins to OS thread, attaches a sub-interpreter, runs __main__.fn(json.loads(payload)), returns json.dumps(result). Free-threading (PEP 703) detected at runtime via Py_GIL_DISABLED. Honest limitations documented in README + capability flags: - HardSandbox=false (C extension segfault kills the host) - SupportsAbort=false (no primitive to abort a running sub-interp from another OS thread without holding its GIL) Direct cgo bridge (~150 lines), no DataDog/go-python3 or cpy3 dependency — both fail to compile against CPython 3.12+ because they reference removed PyFloat_ClearFreeList / PyDict_ClearFreeList. JSON marshaling stays inside Python's stdlib json module so the cgo surface stays narrow. Tests: 7 table tests pass on darwin/arm64 + Python 3.13.13. * feat(base): pyvm fixture + bench integration Adds plugins/extbench/fixtures/pyvm-py/ (validate.py + manifest) implementing the same validate(email, age) spec as the other four fixtures — pure stdlib, no third-party Python deps. bench_test.go: BenchmarkPyvm_Serial + BenchmarkPyvm_Parallel skip gracefully when built without -tags pyvm (same shape as v8go). scale_test.go: pyvm registered alongside the four existing runtimes with a 100-module cap (each module holds up to 4 sub-interpreters at ~4.6 MB each — 1000 modules would be ~18 GB and dominate the study). BASE_PYVM_POOL_SIZE plumbed through setPoolEnv / restorePoolEnv for the pool-elbow subtest. * docs: pyvm benchmark numbers vs existing 4 runtimes Five-way throughput table on darwin/arm64 + Apple M1 Max + Python 3.13.13 (GIL-enabled). Headline: - pyvm Serial 4,089 ns/op — between native (1,229) and goja (4,646) - pyvm Parallel 2,765 ns/op — faster than wazero (4,530) and v8go (12,311), close to goja (1,982) - pyvm B/op 496, allocs/op 10 — smallest footprint of any runtime - pyvm cold start 14.3 ms/module — worst in the field; warm-load at boot, never lazy-load - pyvm per-module RSS asymptote 4.6 MB — highest in the field HIP-0105 verdict: experimental-by-default, production-recommended for single-tenant runtime:pyvm deployments only. Multi-tenant must add crash-isolation:none + hard-abort:none labels so operators can policy-reject. Free-threading (PEP 703) NOT measured — homebrew does not ship [email protected] on darwin/arm64 as of 2026-05-18. Runtime detection path is wired; verification needs a Linux host. Bench crash mode found + fixed: defer-order LIFO race in pyvm_leave/release on Parallel — restructured to straight-line ordering. No further crash modes observed across 90+ s of load. * feat(base): starlark extension runtime (plugins/starkvm) Adds Google's go.starlark.net interpreter as the sixth extruntime Runtime. Pure Go, no cgo, no GIL, per-thread Cancel() for cooperative abort. Same Module shape as gojavm/wasmvm: pre-warmed pool, JSON payload in / JSON result out, ctx-cancel honored. Manifest: "runtime": "starlark", "module": "validate.star". Fixture plugins/extbench/fixtures/starkvm-star/ implements the shared validate(email, age) workload. Uses the permissive Starlark dialect (while + recursion enabled via FileOptions); strict Bazel Starlark would forbid while, doc note in the .star file. Wired into bench_test.go (Serial/Parallel), coldstart_test.go, memory_test.go (TestMemory_Starkvm), and scale_test.go (spec entry + BASE_STARKVM_POOL_SIZE pool-elbow sweep). Tests cover capabilities, roundtrip, unknown fn, post-close, ctx cancel mid-loop, pool reuse, and the full validate fixture against 5 input cases (8 tests total). First measurement on Apple M1 Max: 3701 B/invoke, 1 GC per 1000 calls -- comparable to goja (3433 B/invoke). * feat(extbench): wasm-Rust fixture + bench Pure-Rust validate(email, age) compiled to wasm32-unknown-unknown, loaded by the existing wasmvm runtime. Implements HIP-0105 pointer/ length ABI directly (no zip-rs dep, no Hanzo-internal deps). Compares apples-to-apples with the AssemblyScript fixture: same ABI, same payload, different source language. - Cargo.toml: standalone crate (workspace=[] to escape hanzo workspace) - src/lib.rs: no_std + bump allocator + tolerant JSON scrape - build.sh: cargo build --release --target wasm32-unknown-unknown - validate.wasm: 9.1 KB committed artifact (no toolchain dep at bench run) - bench_test.go: BenchmarkWazero_Rust_{Serial,Parallel} Run: ~4.5us serial / 3.9us parallel on M1 Max. * docs: deferred CPython-WASI + RustPython ABI shim Both runtimes emit WASI command modules (stdin/stdout via _start) which don't match Base's wasmvm pointer ABI (validate(ptr,len) -> i64). CPython-WASI deferred: requires either a 30 MB python.wasm artifact + a stdio-piping Rust/AS shim, or building CPython from source against the WASI SDK (~15-20 min) and forking _start. Out of 60-min budget. RustPython deferred: tried embedding rustpython-vm 0.5.0 in a Rust shim that exposes the pointer ABI directly. The {freeze-stdlib + rustpython-compiler} feature combo fails to compile on wasm32-wasip1 because of cfg-gated paths in define_exception_fn!. Resolving requires either patching the crate or waiting on upstream 0.6.x. Documented the exact attempt + failure in fixtures/wazero-rustpython/README.md so the next agent doesn't repeat the work. Both fixtures stay in tree with extension.json + validate.py + README; the wasmvm runtime reports "module not built" and the bench harness Skip()s them with a clear message. * test(extbench): add pyvm + wasm-Rust memory + coldstart tests memory_test.go gains TestMemory_Pyvm + TestMemory_Wazero_Rust + deferred-fixture skips (TestMemory_Wazero_CPythonWASI/RustPython). runMemory helper now Skip()s when manifest.module artifact is missing — same loadFixture path the bench harness uses, so deferred fixtures (extension.json present, .wasm not built) skip cleanly instead of failing. coldstart_test.go gains BenchmarkColdstart_Pyvm + ..._Wazero_Rust + the two deferred-fixture skips. benchColdstart already probes Load and skips on failure, so the new ones inherit clean Skip behavior. All tests pass under default + v8vm + pyvm tag combinations. * bench: full cross-runtime comparison results Captured 2026-05-19 on Apple M1 Max, default + -tags 'v8vm pyvm'. 8 runtimes measured (5 built, 3 deferred-by-ABI): - native-go, goja, starkvm, wazero-as, wazero-rust, v8go, pyvm: full throughput + memory + cold-start numbers - wazero-cpython-wasi, wazero-rustpython: Skip with "module not built" message (deferred per fixture README) Headline: - pyvm 6.9us serial (only native-go is faster) but 34ms cold-start - starkvm 21us serial — comparable to goja, pure-Go, no JIT - wazero-rust 14.6us serial — slightly better than wazero-as - v8go is the slowest non-deferred runtime (cgo marshalling cost) * docs: consolidated runtime comparison in EXTENSIONS_BENCHMARK.md New section "Full Cross-Runtime Comparison (2026-05-19)" appended. Covers 8 runtimes side-by-side: - 5 originally measured (native, goja, wazero-as, v8go, pyvm) - 3 added in this sprint (starkvm, wazero-rust, plus two documented deferrals: wazero-cpython-wasi + wazero-rustpython) Sections: - Throughput table (serial / parallel / B/op / allocs/op / coldstart) - Per-invocation memory (TestMemory) - Cold-start (BenchmarkColdstart) - Scale + crash characteristics (qualitative) - Recommendation grid (use case -> best/second/avoid) - Headline findings (5 bullets, honest, including the deferral wall) Numbers come from plugins/extbench/bench-results-all.txt; reproduce section at the bottom. * docs(pyvm): add measured 3.13t free-threading numbers + cpy3 fork note - EXTENSIONS_BENCHMARK.md "Free-threading vs sub-interpreter parallelism" section now has the measured numbers (3.13 vs 3.13t, Apple M1 Max, default-bridge pyvm): free-threading is a non-event for the embedding workload (4,073 / 5,733 ns/op default-GIL vs 4,478 / 5,805 ns/op free-threaded). PEP 684 OWN_GIL pool already delivers per-OS-thread parallelism; PEP 703 doesn't add headroom in this contract. - Library-choice paragraph now references hanzoai/cpy3 (the canonical Hanzo Go binding for embedded CPython — fork of go-python/cpy3 with 3.12+ compat + sub-interp wrapper + free-threading detection + JSON hot-path helper + 3.13t build script) and explains why pyvm itself keeps the consolidated direct cgo bridge (one cgo call per Invoke, ~30% faster than routing through cpy3 wrappers for this workload). - go.mod tidied — cpy3 no longer in pyvm's import graph; cpy3 is documented but not directly used by base.