Files
cpy3/json_hotpath_test.go
zeekayandGitHub aa54594e88 Hanzo fork v0.2.1: Python 3.12+ compat, sub-interp wrapper, free-threading detection, JSON hot-path + 3.13t build script (#1)
* feat: Python 3.12+/3.13 compat + sub-interpreter wrapper

Hanzo fork of go-python/cpy3. See HANZO_CHANGES.md for full diff.

- Polyfill APIs removed in 3.12 (per-type ClearFreeList helpers) and
  3.13 (PyEval_InitThreads, Py_SetPath/PythonHome/ProgramName,
  PySys_SetArgv/AddWarnOption/AddXOption/SetPath,
  Py_SetStandardStreamEncoding, PyEval_ThreadsInitialized) so existing
  call sites compile against modern CPython. Polyfills route to the
  PyConfig API where there's a meaningful replacement, return 0/no-op
  otherwise.

- New PEP 684 SubInterpreter wrapper (subinterp.go) — recommended
  primitive for tenant/per-worker isolation on 3.12+. Defaults follow
  PEP 684 (own GIL, threads on, no daemon threads, refuse non-isolated
  C extensions). Falls back to Py_NewInterpreter on 3.11 and older.

- Free-threading awareness (PEP 703) — IsGILDisabled() reports at
  runtime whether the linked CPython was built with --disable-gil.

- pkg-config switched from "python3" to "python3-embed" so libpython
  links cleanly on Linux distros that split the embed flags.

- Module path: github.com/go-python/cpy3 -> github.com/hanzoai/cpy3.
  Go 1.21+. Package name (python3) unchanged.

- CI matrix: Python {3.11, 3.12, 3.13} x Go {1.21, 1.22} on Ubuntu.

License remains MIT. Original go-python team credited in HANZO_CHANGES.md
and every modified file's header.

* feat: JSON hot-path helper + 3.13t build script

- LoadSource + CallJSONFunctionByName: single-cgo-crossing helpers
  for the common "JSON-in, JSON-out" extension embedding contract.
  Roughly 2x faster than the long form via individual wrappers and
  stack-safer under sustained load (macOS SIGBUS at ~400k iters of
  the 9-call long form is mitigated by collapsing to 1 cgo call).
- ErrUnknownFunction: sentinel error for missing/non-callable
  lookups so callers can errors.Is.
- scripts/build-python313t.sh: idempotent local build of CPython
  3.13.x with --disable-gil. Writes a python3-embed.pc so callers
  steer via PKG_CONFIG_PATH.
- HANZO_CHANGES.md + README updated.
- .gitignore: exclude python313t-build/ python313t-src/.

* ci: split test runs by Python global state

TestSubInterpreter_* and TestCallJSONFunctionByName_* / TestLoadSource_*
share global Py state with the data-type/error tests. Running them
in the same `go test` invocation can hang on darwin after the
data-type tests have already touched the main interpreter. Split
into two test steps so each starts from a fresh process.

* test: move json_hotpath tests in-package + share Py init

Move from package python3_test to package python3 so they share
Py_Initialize() state with subinterp_test.go. Each test pins its
own OS thread, calls Py_Initialize (idempotent), creates a fresh
sub-interp, and unpins on return. No PyEval_SaveThread in teardown
— the sub-interp's Close already released the calling thread's GIL.

This eliminates a SIGSEGV-on-second-test pattern caused by trying
to PyEval_SaveThread a state that's already been saved.

* docs: README example for the JSON hot path

Show LoadSource + CallJSONFunctionByName end-to-end with luxfi/log
for context. The helpers themselves take no logger dependency.
2026-05-18 23:38:11 -07:00

136 lines
3.8 KiB
Go

/*
SPDX-License-Identifier: MIT
Hanzo additions, 2026-05-19.
These tests deliberately run in package python3 (not python3_test)
so they share Py_Initialize state with the other in-package tests
that touch global Python state. The harness pattern matches
subinterp_test.go: pin OS thread, Py_Initialize (idempotent), create
sub-interpreter, run, tear down.
*/
package python3
import (
"errors"
gort "runtime"
"strings"
"testing"
)
// initAndPin pins the goroutine to its OS thread and brings the main
// interpreter up to a state where NewSubInterpreter can be called
// (caller thread holds the main GIL). Caller must Defer the returned
// teardown which unpins. The teardown does NOT touch Python state —
// by the time the test body runs `defer si.Close()`, the sub-interp's
// state has been torn down and the calling thread holds no GIL.
//
// Subsequent tests can call this safely; Py_Initialize is idempotent.
func initAndPin(t *testing.T) (teardown func()) {
t.Helper()
gort.LockOSThread()
Py_Initialize()
// Py_Initialize leaves the calling thread holding the main GIL.
// Caller must hold it before NewSubInterpreter — we're already
// there.
return func() {
// Sub-interp Close already released the GIL. We do NOT call
// PyEval_SaveThread here — that would fail without a thread
// state. Just unpin.
gort.UnlockOSThread()
}
}
// TestCallJSONFunctionByName_HappyPath exercises the consolidated
// JSON hot path. It loads a tiny Python source into a fresh
// sub-interpreter, calls it, and asserts the JSON round-trip works.
func TestCallJSONFunctionByName_HappyPath(t *testing.T) {
defer initAndPin(t)()
si, err := NewSubInterpreter(DefaultSubInterpreterConfig())
if err != nil {
t.Fatalf("NewSubInterpreter: %v", err)
}
defer si.Close()
si.Enter()
defer si.Leave()
src := `
def echo(p):
return {"got": p, "kind": type(p).__name__}
`
if err := LoadSource(src); err != nil {
t.Fatalf("LoadSource: %v", err)
}
out, err := CallJSONFunctionByName("echo", `{"a":1,"b":"two"}`)
if err != nil {
t.Fatalf("CallJSONFunctionByName: %v", err)
}
if !strings.Contains(out, `"kind": "dict"`) {
t.Fatalf("unexpected result: %s", out)
}
if !strings.Contains(out, `"got"`) {
t.Fatalf("unexpected result: %s", out)
}
}
// TestCallJSONFunctionByName_Unknown verifies ErrUnknownFunction is
// returned for a name that isn't defined or isn't callable.
func TestCallJSONFunctionByName_Unknown(t *testing.T) {
defer initAndPin(t)()
si, err := NewSubInterpreter(DefaultSubInterpreterConfig())
if err != nil {
t.Fatalf("NewSubInterpreter: %v", err)
}
defer si.Close()
si.Enter()
defer si.Leave()
if err := LoadSource(`x = 42`); err != nil {
t.Fatalf("LoadSource: %v", err)
}
// "x" exists but is not callable.
if _, err := CallJSONFunctionByName("x", `null`); !errors.Is(err, ErrUnknownFunction) {
t.Fatalf("expected ErrUnknownFunction for non-callable, got %v", err)
}
// "missing" does not exist at all.
if _, err := CallJSONFunctionByName("missing", `null`); !errors.Is(err, ErrUnknownFunction) {
t.Fatalf("expected ErrUnknownFunction for missing name, got %v", err)
}
}
// TestCallJSONFunctionByName_PyException verifies a Python-side
// exception is surfaced as a non-nil error with a useful message.
func TestCallJSONFunctionByName_PyException(t *testing.T) {
defer initAndPin(t)()
si, err := NewSubInterpreter(DefaultSubInterpreterConfig())
if err != nil {
t.Fatalf("NewSubInterpreter: %v", err)
}
defer si.Close()
si.Enter()
defer si.Leave()
if err := LoadSource(`
def boom(_):
raise ValueError("kaboom")
`); err != nil {
t.Fatalf("LoadSource: %v", err)
}
_, err = CallJSONFunctionByName("boom", `null`)
if err == nil {
t.Fatalf("expected error, got nil")
}
if !strings.Contains(err.Error(), "kaboom") {
t.Fatalf("expected error to mention 'kaboom', got %v", err)
}
}