mirror of
https://github.com/hanzoai/dev.git
synced 2026-10-17 15:34:54 +00:00
feat(go): a Go host runs the real core, one wasm instance a session
go/ is the module github.com/hanzoai/dev/go. It embeds the module `make wasm` builds and runs it under wazero through hanzoai/wasm, so cloud drives the core itself, with no cgo and no Rust toolchain. Open compiles once and refuses a module whose dev_abi is not 1. New and Restore each start an instance with 64 MiB of its own; Step and Snapshot cross into it; Close drops the session and the instance. The out-parameter is one twelve-byte loan per session, every input loan is released and every dev_buf freed on error paths too, and ten thousand steps leave the memory exactly the size it was. dev_free takes its dev_buf by value, which the wasm32 C ABI passes as one pointer: a test pins that signature and the other eight, and that the imports are the four WASI calls and nothing else. dev.h's statuses are sentinel errors. A trap answers ErrPanic, then ErrPoison, for its session alone: a 24 MiB prompt makes the allocator refuse inside serde, and the session beside it finishes its turn. A call after Close, or after the Core closed, answers ErrHandle; a call while another runs answers ErrBusy. The types are crates/protocol: an enum is a struct of its variants with exactly one set, and the encoder refuses any other count. Every type goes through the module itself, and Go writes the core's answers back byte for byte. Breaking the host thirteen ways, one at a time, failed a test each time. `make wasm` copies the module to go/dev.wasm without DWARF and with the cargo home remapped, so the bytes do not depend on where they were built: 320,088 bytes, the same from another worktree and another CARGO_HOME. Co-authored-by: Hanzo Dev <[email protected]>
This commit is contained in:
Hanzo Dev
@@ -31,7 +31,8 @@ test: prepare
|
||||
@$(CARGO) nextest run --no-fail-fast --manifest-path crates/hanzo-upstream/Cargo.toml $(ARGS)
|
||||
|
||||
## wasm: build the reasoning half for wasm32-wasip1
|
||||
# The loop as a wasm module: target/wasm32-wasip1/release/dev.wasm.
|
||||
# The loop as a wasm module: target/wasm32-wasip1/release/dev.wasm, copied to
|
||||
# go/dev.wasm, which the Go host embeds.
|
||||
#
|
||||
# protocol and core carry no effects, so they run wherever a wasm host runs — and
|
||||
# so does ffi, which is what a host actually calls. Its C ABI is the module's
|
||||
@@ -40,8 +41,18 @@ test: prepare
|
||||
# nothing: environ_get, environ_sizes_get, fd_write and proc_exit, and no file,
|
||||
# socket, clock or random source. A change that grows that list has given the
|
||||
# loop an effect of its own.
|
||||
#
|
||||
# The copy is committed, so it is built to be the same bytes wherever it is
|
||||
# built: no DWARF, which spells out the build directory, and the cargo home
|
||||
# written as /cargo in the panic locations the dependencies carry. The name
|
||||
# section stays, so a trap still says which function it stopped in.
|
||||
CARGO_HOME ?= $(HOME)/.cargo
|
||||
wasm:
|
||||
@$(CARGO) build --locked --release --target wasm32-wasip1 -p dev-ffi
|
||||
@$(CARGO) build --locked --release --target wasm32-wasip1 -p dev-ffi \
|
||||
--config 'profile.release.debug=false' \
|
||||
--config 'profile.release.strip="debuginfo"' \
|
||||
--config 'target.wasm32-wasip1.rustflags=["--remap-path-prefix=$(CARGO_HOME)=/cargo"]'
|
||||
@install -m 644 target/wasm32-wasip1/release/dev.wasm go/dev.wasm
|
||||
|
||||
## test-upstream: run the upstream suite against the pinned submodule
|
||||
# voice-host wants GStreamer >= 1.28, which no current distribution ships, and
|
||||
|
||||
+119
@@ -0,0 +1,119 @@
|
||||
# github.com/hanzoai/dev/go
|
||||
|
||||
The dev loop (HIP-1330), for a Go host.
|
||||
|
||||
The loop is the Rust core in this repository, built as a wasm module and
|
||||
embedded here. This package runs it in the calling process under wazero: no
|
||||
cgo, and no Rust toolchain in the build that imports it. The core reasons and
|
||||
the host acts. The core asks for work as `Action`s, and the host performs them
|
||||
and reports what happened as `Event`s.
|
||||
|
||||
```go
|
||||
import dev "github.com/hanzoai/dev/go"
|
||||
```
|
||||
|
||||
## The calls
|
||||
|
||||
Open(ctx) (*Core, error) compile the module, once per process
|
||||
(*Core).New(ctx, Config) (*Session, error) start a session in an instance of its own
|
||||
(*Core).Restore(ctx, state) (*Session, error) rebuild one from a snapshot
|
||||
(*Session).Step(ctx, Event) ([]Action, error) one event in, the actions it asks for out
|
||||
(*Session).Snapshot(ctx) ([]byte, error) the core's state, for Restore
|
||||
(*Session).Close(ctx) error drop the session and its instance
|
||||
(*Core).Close(ctx) error release the runtime
|
||||
|
||||
A `Core` is safe for concurrent use. A `Session` is not: it runs one call at a
|
||||
time, and a call made while another is running answers `ErrBusy`.
|
||||
|
||||
The statuses of `crates/ffi/include/dev.h` are errors to match with
|
||||
`errors.Is`: `ErrHandle`, `ErrMalformed`, `ErrPoison`, `ErrPanic`, `ErrNull`,
|
||||
`ErrBusy` and `ErrFull`. A trap in the core, such as a panic or a refused
|
||||
allocation, answers `ErrPanic` and then `ErrPoison` for that session only.
|
||||
Other sessions keep running. A closed session answers `ErrHandle`.
|
||||
|
||||
## The types
|
||||
|
||||
The types are `crates/protocol`, field for field. A Rust enum is a struct whose
|
||||
fields are its variants: a pointer for a variant that carries a value, a bool
|
||||
for one that does not. Exactly one is set. A union that sets none, or two, is
|
||||
refused as `ErrMalformed` before the core sees it.
|
||||
|
||||
dev.Event{Turn: &dev.Turn{ID: 1, Prompt: "fix the build"}}
|
||||
dev.Event{Exec: &dev.Output{ID: a.ID, Text: out, Failed: code != 0}}
|
||||
dev.Event{Timer: true}
|
||||
|
||||
Text crosses as UTF-8. A host that has read bytes that are not UTF-8 must turn
|
||||
them into text before it steps with them.
|
||||
|
||||
## A loop
|
||||
|
||||
The model here is a stand-in: it asks for one file, and then answers.
|
||||
|
||||
```go
|
||||
ctx := context.Background()
|
||||
core, err := dev.Open(ctx)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
defer core.Close(ctx)
|
||||
|
||||
s, err := core.New(ctx, dev.Config{Model: new("zen5")})
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
defer s.Close(ctx)
|
||||
|
||||
events := []dev.Event{{Turn: &dev.Turn{ID: 1, Prompt: "What does main.go do?"}}}
|
||||
for len(events) > 0 {
|
||||
actions, err := s.Step(ctx, events[0])
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
events = events[1:]
|
||||
for _, a := range actions {
|
||||
switch op := a.Op; {
|
||||
case op.Model != nil:
|
||||
events = append(events, dev.Event{Model: &dev.Answer{ID: a.ID, Reply: model(op.Model)}})
|
||||
case op.Read != nil:
|
||||
events = append(events, dev.Event{File: &dev.Output{ID: a.ID, Text: "package main\n"}})
|
||||
case op.Emit != nil:
|
||||
fmt.Println(*op.Emit)
|
||||
case op.Save:
|
||||
// Write s.Snapshot(ctx) beside the journal, at a.ID.
|
||||
case op.Done != nil:
|
||||
fmt.Println("done", op.Done.Turn)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func model(ask *dev.Ask) dev.Reply {
|
||||
if last := ask.Messages[len(ask.Messages)-1]; last.Tool == nil {
|
||||
return dev.Reply{Calls: []dev.Call{{Read: &dev.Read{Path: "main.go"}}}}
|
||||
}
|
||||
return dev.Reply{Text: new("It declares package main and does nothing.")}
|
||||
}
|
||||
```
|
||||
|
||||
`example_test.go` runs the same loop as a test.
|
||||
|
||||
## Persisting a session
|
||||
|
||||
A session is three facts, kept apart: the core's snapshot, the workspace, and
|
||||
the host's journal of actions. Every action carries an id. When the host
|
||||
records dispatch and completion under that id, a result can be replayed
|
||||
instead of an effect run twice. The core refuses a turn id it has already
|
||||
accepted and a result for an id it is not waiting on, so replaying an event
|
||||
is safe.
|
||||
|
||||
A restored snapshot continues the id sequence where it stopped. Intact bytes do
|
||||
not prove the snapshot agrees with the journal. One older than the journal
|
||||
mints ids the host has already dispatched, so reconcile the two before
|
||||
dispatching again.
|
||||
|
||||
## The module
|
||||
|
||||
`dev.wasm` is `make wasm` at the repository root, copied here. Its imports are
|
||||
`environ_get`, `environ_sizes_get`, `fd_write` and `proc_exit`, and nothing
|
||||
else. The core cannot reach a file, a socket, a clock or a source of
|
||||
randomness, and a test checks that list. Each session gets at most 64 MiB of
|
||||
memory.
|
||||
@@ -0,0 +1,86 @@
|
||||
package dev
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Open is the compile, paid once per process.
|
||||
func BenchmarkOpen(b *testing.B) {
|
||||
ctx := context.Background()
|
||||
for b.Loop() {
|
||||
c, err := Open(ctx)
|
||||
if err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
_ = c.Close(ctx)
|
||||
}
|
||||
}
|
||||
|
||||
// New is an instance and a session in it, and Close to give both back.
|
||||
func BenchmarkNew(b *testing.B) {
|
||||
ctx := context.Background()
|
||||
c := core(b)
|
||||
for b.Loop() {
|
||||
s, err := c.New(ctx, Config{})
|
||||
if err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
_ = s.Close(ctx)
|
||||
}
|
||||
}
|
||||
|
||||
// Step is one step of a session with a turn in flight: a turn that queues
|
||||
// behind it, then the cancel that takes it out again. Both answer with
|
||||
// actions, and neither leaves the session larger than it found it.
|
||||
func BenchmarkStep(b *testing.B) {
|
||||
s := session(b, core(b), Config{})
|
||||
step(b, s, turn(1, "fix the build"))
|
||||
id := uint64(1)
|
||||
for b.Loop() {
|
||||
if id++; id%2 == 0 {
|
||||
step(b, s, turn(id, "and the tests"))
|
||||
} else {
|
||||
step(b, s, Event{Cancel: &Cancel{Turn: id - 1}})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Snapshot is the state of a session one turn old.
|
||||
func BenchmarkSnapshot(b *testing.B) {
|
||||
ctx := context.Background()
|
||||
s := finished(b, core(b))
|
||||
for b.Loop() {
|
||||
if _, err := s.Snapshot(ctx); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Restore is an instance and that snapshot unpacked into it, and Close to
|
||||
// give it back.
|
||||
func BenchmarkRestore(b *testing.B) {
|
||||
ctx := context.Background()
|
||||
c := core(b)
|
||||
state, err := finished(b, c).Snapshot(ctx)
|
||||
if err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
for b.Loop() {
|
||||
s, err := c.Restore(ctx, state)
|
||||
if err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
_ = s.Close(ctx)
|
||||
}
|
||||
}
|
||||
|
||||
// finished is a session that has run one turn: a read, and an answer.
|
||||
func finished(b *testing.B, c *Core) *Session {
|
||||
s := session(b, c, Config{Model: new("zen5")})
|
||||
step(b, s, turn(1, "fix the build"))
|
||||
step(b, s, Event{Model: &Answer{ID: 1, Reply: Reply{Calls: []Call{{Read: &Read{Path: "src/lib.rs"}}}}}})
|
||||
step(b, s, Event{File: &Output{ID: 2, Text: "pub fn f() -> u8 { 1 }"}})
|
||||
step(b, s, Event{Model: &Answer{ID: 3, Reply: Reply{Text: new("It returns one.")}}})
|
||||
return s
|
||||
}
|
||||
@@ -0,0 +1,363 @@
|
||||
// Package dev runs the dev reasoning core (HIP-1330) inside the calling
|
||||
// process.
|
||||
//
|
||||
// The core is this repository's loop — crates/core behind the C ABI of
|
||||
// crates/ffi — built as a wasm module, which this package embeds and runs
|
||||
// under wazero: no cgo, and no Rust toolchain in a build that imports it. A
|
||||
// Core compiles the module once. A Session is one instance of it, with memory
|
||||
// of its own, holding one conversation: step it with an Event, perform the
|
||||
// Actions it answers with, and step it again with what they observed.
|
||||
//
|
||||
// The module imports environ_get, environ_sizes_get, fd_write and proc_exit,
|
||||
// and nothing else. It reaches no file, socket, clock or source of randomness,
|
||||
// so every effect an Action names is the host's to perform or refuse.
|
||||
package dev
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
_ "embed"
|
||||
json "encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"sync/atomic"
|
||||
|
||||
"github.com/hanzoai/wasm"
|
||||
"github.com/tetratelabs/wazero/sys"
|
||||
)
|
||||
|
||||
// ABI is the version of the C ABI, and of the payload encoding behind it,
|
||||
// that this package speaks. Open refuses a module that answers dev_abi with
|
||||
// any other.
|
||||
const ABI = 1
|
||||
|
||||
// The status codes of crates/ffi/include/dev.h. Match them with errors.Is.
|
||||
var (
|
||||
ErrHandle = errors.New("dev: the handle names no live session")
|
||||
ErrMalformed = errors.New("dev: the payload is not a message of this protocol")
|
||||
ErrPoison = errors.New("dev: a previous call on this session panicked")
|
||||
ErrPanic = errors.New("dev: this call panicked; the session is now poisoned")
|
||||
ErrNull = errors.New("dev: a pointer the caller must supply was null")
|
||||
ErrBusy = errors.New("dev: the session is inside another call")
|
||||
ErrFull = errors.New("dev: no slot is left for another session")
|
||||
)
|
||||
|
||||
// statuses is dev.h's table, indexed by the negated code.
|
||||
var statuses = [...]error{nil, ErrHandle, ErrMalformed, ErrPoison, ErrPanic, ErrNull, ErrBusy, ErrFull}
|
||||
|
||||
// module is target/wasm32-wasip1/release/dev.wasm, as `make wasm` copies it.
|
||||
//
|
||||
//go:embed dev.wasm
|
||||
var module []byte
|
||||
|
||||
const (
|
||||
// pages bounds one session's memory: 1024 pages of 64 KiB, 64 MiB.
|
||||
pages = 1024
|
||||
// buf is the size of a dev_buf on wasm32: ptr, len and cap, three
|
||||
// little-endian u32s.
|
||||
buf = 12
|
||||
)
|
||||
|
||||
// Core is the compiled module. Open one per process, and New or Restore a
|
||||
// Session from it per conversation. A Core is safe for concurrent use.
|
||||
type Core struct {
|
||||
engine *wasm.Engine
|
||||
module *wasm.Module
|
||||
}
|
||||
|
||||
// Open compiles the embedded module, the one slow call, and checks that it
|
||||
// speaks ABI.
|
||||
func Open(ctx context.Context) (*Core, error) {
|
||||
return open(ctx, module)
|
||||
}
|
||||
|
||||
func open(ctx context.Context, src []byte) (*Core, error) {
|
||||
engine, err := wasm.New(ctx, wasm.Limits{Pages: pages})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c := &Core{engine: engine}
|
||||
if c.module, err = engine.Compile(ctx, src); err == nil {
|
||||
err = c.speaks(ctx)
|
||||
}
|
||||
if err != nil {
|
||||
_ = engine.Close(ctx)
|
||||
return nil, err
|
||||
}
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// speaks refuses a module that answers dev_abi with another ABI than ours.
|
||||
func (c *Core) speaks(ctx context.Context) error {
|
||||
inst, err := c.module.Start(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer inst.Close(ctx)
|
||||
out, err := inst.Call(ctx, "dev_abi")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if abi := uint32(out[0]); abi != ABI {
|
||||
return fmt.Errorf("dev: the module speaks ABI %d, and this package ABI %d", abi, ABI)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Close releases the runtime. A Session started from it has nothing left to
|
||||
// run in, and answers ErrHandle from then on.
|
||||
func (c *Core) Close(ctx context.Context) error {
|
||||
return c.engine.Close(ctx)
|
||||
}
|
||||
|
||||
// New starts a session.
|
||||
func (c *Core) New(ctx context.Context, cfg Config) (*Session, error) {
|
||||
in, err := json.Marshal(cfg)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w: %v", ErrMalformed, err)
|
||||
}
|
||||
return c.start(ctx, "dev_new", in)
|
||||
}
|
||||
|
||||
// Restore rebuilds a session from its Snapshot, in an instance of its own.
|
||||
//
|
||||
// A snapshot that was truncated, edited, or written under another ABI answers
|
||||
// ErrMalformed. Intact bytes are not proof that the sequence agrees with the
|
||||
// host's journal: a snapshot older than the journal mints again ids the host
|
||||
// has already dispatched under, so reconcile the two before dispatching.
|
||||
func (c *Core) Restore(ctx context.Context, state []byte) (*Session, error) {
|
||||
return c.start(ctx, "dev_restore", state)
|
||||
}
|
||||
|
||||
func (c *Core) start(ctx context.Context, name string, in []byte) (*Session, error) {
|
||||
inst, err := c.module.Start(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s := &Session{inst: inst}
|
||||
if err := s.open(ctx, name, in); err != nil {
|
||||
_ = s.Close(ctx)
|
||||
return nil, err
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
|
||||
// Session is one conversation, in a wasm instance of its own: the unit of
|
||||
// isolation. What a session does to its memory, a panic included, reaches no
|
||||
// other.
|
||||
//
|
||||
// A Session is not safe for concurrent use. It runs one call at a time and
|
||||
// does not queue: a call that arrives while another is running answers
|
||||
// ErrBusy, as DEV_BUSY does in C, and never reaches the instance.
|
||||
type Session struct {
|
||||
inst *wasm.Instance
|
||||
handle uint64
|
||||
// out is the session's one out-parameter, lent at birth and released at
|
||||
// Close: the handle is written there, then every dev_buf after it.
|
||||
out uint32
|
||||
busy atomic.Bool
|
||||
// gone is what every call answers once the instance can answer no more:
|
||||
// ErrHandle once it is closed, ErrPoison once it has trapped.
|
||||
gone error
|
||||
}
|
||||
|
||||
// Step feeds the core one event and returns the actions it answers with, in
|
||||
// the order it asks for them. An event it cannot read answers ErrMalformed
|
||||
// and changes nothing.
|
||||
func (s *Session) Step(ctx context.Context, event Event) ([]Action, error) {
|
||||
in, err := json.Marshal(event)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w: %v", ErrMalformed, err)
|
||||
}
|
||||
var actions []Action
|
||||
err = s.step(ctx, in, func(out []byte) error {
|
||||
if err := json.Unmarshal(out, &actions, json.RejectUnknownMembers(true)); err != nil {
|
||||
return fmt.Errorf("%w: the core's answer: %v", ErrMalformed, err)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
return actions, err
|
||||
}
|
||||
|
||||
// step is Step on encoded bytes: use reads the encoded answer before it goes
|
||||
// back to the core.
|
||||
func (s *Session) step(ctx context.Context, in []byte, use func([]byte) error) error {
|
||||
if err := s.enter(); err != nil {
|
||||
return err
|
||||
}
|
||||
defer s.leave()
|
||||
return s.lend(ctx, in, func(ptr uint32) error {
|
||||
if err := status(s.call(ctx, "dev_step", s.handle, uint64(ptr), uint64(len(in)), uint64(s.out))); err != nil {
|
||||
return err
|
||||
}
|
||||
return s.take(ctx, use)
|
||||
})
|
||||
}
|
||||
|
||||
// Snapshot is the core's own state, one of the three facts a session is
|
||||
// (HIP-1330): packed under the ABI with a checksum, for Restore.
|
||||
func (s *Session) Snapshot(ctx context.Context) ([]byte, error) {
|
||||
if err := s.enter(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer s.leave()
|
||||
if err := status(s.call(ctx, "dev_snapshot", s.handle, uint64(s.out))); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var state []byte
|
||||
err := s.take(ctx, func(out []byte) error {
|
||||
state = bytes.Clone(out)
|
||||
return nil
|
||||
})
|
||||
return state, err
|
||||
}
|
||||
|
||||
// Close drops the session and closes its instance, which takes every byte the
|
||||
// session held with it. A call after Close answers ErrHandle, and a second
|
||||
// Close answers nil.
|
||||
func (s *Session) Close(ctx context.Context) error {
|
||||
if !s.busy.CompareAndSwap(false, true) {
|
||||
return ErrBusy
|
||||
}
|
||||
defer s.leave()
|
||||
if s.inst == nil {
|
||||
return nil
|
||||
}
|
||||
// A handle is never zero, and an instance that has trapped or closed is
|
||||
// not called again: either way, what these would free goes with it.
|
||||
if s.handle != 0 {
|
||||
_, _ = s.call(ctx, "dev_drop", s.handle)
|
||||
}
|
||||
if s.out != 0 {
|
||||
_, _ = s.call(ctx, "dev_release", uint64(s.out), buf)
|
||||
}
|
||||
err := s.inst.Close(ctx)
|
||||
s.inst, s.gone = nil, ErrHandle
|
||||
return err
|
||||
}
|
||||
|
||||
// open lends the out-parameter and runs dev_new or dev_restore into it.
|
||||
func (s *Session) open(ctx context.Context, name string, in []byte) error {
|
||||
out, err := s.alloc(ctx, buf)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
s.out = out
|
||||
err = s.lend(ctx, in, func(ptr uint32) error {
|
||||
return status(s.call(ctx, name, uint64(ptr), uint64(len(in)), uint64(s.out)))
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
s.handle, _ = s.inst.Memory().ReadUint64Le(s.out)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Session) enter() error {
|
||||
if !s.busy.CompareAndSwap(false, true) {
|
||||
return ErrBusy
|
||||
}
|
||||
if s.gone != nil {
|
||||
s.leave()
|
||||
return s.gone
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Session) leave() {
|
||||
s.busy.Store(false)
|
||||
}
|
||||
|
||||
// call runs one export and answers with its first result.
|
||||
func (s *Session) call(ctx context.Context, name string, args ...uint64) (uint64, error) {
|
||||
if s.gone != nil {
|
||||
return 0, s.gone
|
||||
}
|
||||
out, err := s.inst.Call(ctx, name, args...)
|
||||
if err != nil {
|
||||
return 0, s.lose(err)
|
||||
}
|
||||
if len(out) == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
return out[0], nil
|
||||
}
|
||||
|
||||
// lose answers for a call the instance could not finish, and marks what every
|
||||
// later call answers. An instance that has closed holds no session: ErrHandle.
|
||||
// Anything else is a trap — a panic, or memory the core needed and was refused
|
||||
// — after which the instance's memory is not trusted again: ErrPanic now,
|
||||
// ErrPoison after.
|
||||
func (s *Session) lose(err error) error {
|
||||
if _, ok := errors.AsType[*sys.ExitError](err); ok {
|
||||
s.gone = ErrHandle
|
||||
return fmt.Errorf("%w: %v", ErrHandle, err)
|
||||
}
|
||||
s.gone = ErrPoison
|
||||
return fmt.Errorf("%w: %v", ErrPanic, err)
|
||||
}
|
||||
|
||||
// status turns what an export answered into dev.h's error for it.
|
||||
func status(code uint64, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
n := -int64(int32(code))
|
||||
if n < 0 || n >= int64(len(statuses)) {
|
||||
return fmt.Errorf("dev: status %d is not one ABI %d has", int32(code), ABI)
|
||||
}
|
||||
return statuses[n]
|
||||
}
|
||||
|
||||
// alloc borrows n bytes inside the instance.
|
||||
func (s *Session) alloc(ctx context.Context, n int) (uint32, error) {
|
||||
ptr, err := s.call(ctx, "dev_alloc", uint64(n))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if ptr == 0 {
|
||||
return 0, fmt.Errorf("%w: the core lent nothing for %d bytes", ErrNull, n)
|
||||
}
|
||||
return uint32(ptr), nil
|
||||
}
|
||||
|
||||
// lend copies in into memory borrowed from the instance for as long as use
|
||||
// runs, and gives it back afterwards whatever use answered. Nothing is lent
|
||||
// for nothing: an empty input crosses as a null pointer and a zero length.
|
||||
func (s *Session) lend(ctx context.Context, in []byte, use func(ptr uint32) error) error {
|
||||
if len(in) == 0 {
|
||||
return use(0)
|
||||
}
|
||||
ptr, err := s.alloc(ctx, len(in))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if s.inst.Memory().Write(ptr, in) {
|
||||
err = use(ptr)
|
||||
} else {
|
||||
err = s.lose(fmt.Errorf("the core lent %d bytes at %#x, outside its memory", len(in), ptr))
|
||||
}
|
||||
if _, released := s.call(ctx, "dev_release", uint64(ptr), uint64(len(in))); err == nil {
|
||||
err = released
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// take hands use the bytes of the dev_buf the last call wrote to out, then
|
||||
// frees the buffer whatever use answered. dev_free takes a dev_buf by value,
|
||||
// which the wasm32 C ABI passes as a pointer to a copy: out is one.
|
||||
func (s *Session) take(ctx context.Context, use func([]byte) error) error {
|
||||
mem := s.inst.Memory()
|
||||
ptr, _ := mem.ReadUint32Le(s.out)
|
||||
n, _ := mem.ReadUint32Le(s.out + 4)
|
||||
var err error
|
||||
if answer, ok := mem.Read(ptr, n); ok {
|
||||
err = use(answer)
|
||||
} else {
|
||||
err = s.lose(fmt.Errorf("the core answered %d bytes at %#x, outside its memory", n, ptr))
|
||||
}
|
||||
if _, freed := s.call(ctx, "dev_free", uint64(s.out)); err == nil {
|
||||
err = freed
|
||||
}
|
||||
return err
|
||||
}
|
||||
BIN
Binary file not shown.
+522
@@ -0,0 +1,522 @@
|
||||
package dev
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
json "encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// shared is one Core for every test that does not close its own. Open is the
|
||||
// once-per-process call, and under -race a compile takes seconds.
|
||||
var shared = sync.OnceValues(func() (*Core, error) {
|
||||
return Open(context.Background())
|
||||
})
|
||||
|
||||
func core(t testing.TB) *Core {
|
||||
t.Helper()
|
||||
c, err := shared()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
func session(t testing.TB, c *Core, cfg Config) *Session {
|
||||
t.Helper()
|
||||
s, err := c.New(t.Context(), cfg)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { _ = s.Close(context.Background()) })
|
||||
return s
|
||||
}
|
||||
|
||||
func step(t testing.TB, s *Session, event Event) []Action {
|
||||
t.Helper()
|
||||
actions, err := s.Step(t.Context(), event)
|
||||
if err != nil {
|
||||
t.Fatalf("step %s: %v", show(event), err)
|
||||
}
|
||||
return actions
|
||||
}
|
||||
|
||||
func turn(id uint64, prompt string) Event {
|
||||
return Event{Turn: &Turn{ID: id, Prompt: prompt}}
|
||||
}
|
||||
|
||||
// show is v as the protocol encodes it.
|
||||
func show(v any) string {
|
||||
out, err := json.Marshal(v)
|
||||
if err != nil {
|
||||
return fmt.Sprintf("%#v (%v)", v, err)
|
||||
}
|
||||
return string(out)
|
||||
}
|
||||
|
||||
// same fails the test unless got and want are the same message.
|
||||
func same(t testing.TB, what string, got, want any) {
|
||||
t.Helper()
|
||||
if g, w := show(got), show(want); g != w {
|
||||
t.Errorf("%s\n got %s\nwant %s", what, g, w)
|
||||
}
|
||||
}
|
||||
|
||||
// The whole loop once: a prompt, a model that asks for two tools, their
|
||||
// results, and a model that answers in words.
|
||||
func TestATurnRunsToDone(t *testing.T) {
|
||||
s := session(t, core(t), Config{})
|
||||
|
||||
same(t, "a turn", step(t, s, turn(1, "fix the build")), []Action{
|
||||
{ID: 1, Op: Op{Model: &Ask{Messages: []Message{{User: new("fix the build")}}}}},
|
||||
})
|
||||
|
||||
build := Exec{Argv: []string{"cargo", "build"}}
|
||||
lib := Read{Path: "src/lib.rs"}
|
||||
two := step(t, s, Event{Model: &Answer{ID: 1, Reply: Reply{
|
||||
Text: new("Let me look."),
|
||||
Calls: []Call{{Exec: &build}, {Read: &lib}},
|
||||
}}})
|
||||
same(t, "two calls", two, []Action{
|
||||
{ID: 2, Op: Op{Exec: &build}},
|
||||
{ID: 3, Op: Op{Read: &lib}},
|
||||
})
|
||||
|
||||
failed := "error[E0425]: cannot find value `x` in this scope"
|
||||
if actions := step(t, s, Event{Exec: &Output{ID: 2, Text: failed, Failed: true}}); len(actions) != 0 {
|
||||
t.Fatalf("one result of two asked for %s", show(actions))
|
||||
}
|
||||
same(t, "both results", step(t, s, Event{File: &Output{ID: 3, Text: "pub fn f() -> u8 { x }"}}), []Action{
|
||||
{ID: 4, Op: Op{Model: &Ask{Messages: []Message{
|
||||
{User: new("fix the build")},
|
||||
{Agent: &Agent{Text: new("Let me look."), Calls: []Dispatch{
|
||||
{ID: 2, Call: Call{Exec: &build}},
|
||||
{ID: 3, Call: Call{Read: &lib}},
|
||||
}}},
|
||||
{Tool: &Tool{ID: 2, Text: failed, Failed: true}},
|
||||
{Tool: &Tool{ID: 3, Text: "pub fn f() -> u8 { x }"}},
|
||||
}}}},
|
||||
})
|
||||
|
||||
same(t, "a text answer", step(t, s, Event{Model: &Answer{ID: 4, Reply: Reply{Text: new("`x` was never declared.")}}}), []Action{
|
||||
{ID: 5, Op: Op{Emit: new("`x` was never declared.")}},
|
||||
{ID: 6, Op: Op{Save: true}},
|
||||
{ID: 7, Op: Op{Done: &Done{Turn: 1, Outcome: Outcome{Complete: true}}}},
|
||||
})
|
||||
}
|
||||
|
||||
// A turn the core has accepted is refused at its id and below, so a prompt
|
||||
// delivered twice runs once.
|
||||
func TestARedeliveredTurnAsksForNothing(t *testing.T) {
|
||||
s := session(t, core(t), Config{})
|
||||
step(t, s, turn(7, "fix the build"))
|
||||
for _, id := range []uint64{7, 6, 1} {
|
||||
if actions := step(t, s, turn(id, "fix the build")); len(actions) != 0 {
|
||||
t.Errorf("turn %d again asked for %s", id, show(actions))
|
||||
}
|
||||
}
|
||||
step(t, s, Event{Model: &Answer{ID: 1, Reply: Reply{Calls: []Call{{Read: &Read{Path: "Makefile"}}}}}})
|
||||
same(t, "the transcript", step(t, s, Event{File: &Output{ID: 2, Text: "all: build"}}), []Action{
|
||||
{ID: 3, Op: Op{Model: &Ask{Messages: []Message{
|
||||
{User: new("fix the build")},
|
||||
{Agent: &Agent{Calls: []Dispatch{{ID: 2, Call: Call{Read: &Read{Path: "Makefile"}}}}}},
|
||||
{Tool: &Tool{ID: 2, Text: "all: build"}},
|
||||
}}}},
|
||||
})
|
||||
}
|
||||
|
||||
// A result is taken only while its id is outstanding and was dispatched to
|
||||
// that family. Anything else changes nothing, and in particular does not use
|
||||
// up the entry the genuine result still needs.
|
||||
func TestAResultNoOneAwaitsChangesNothing(t *testing.T) {
|
||||
s := session(t, core(t), Config{})
|
||||
step(t, s, turn(1, "fix the build"))
|
||||
build := Exec{Argv: []string{"make"}}
|
||||
file := Read{Path: "Makefile"}
|
||||
step(t, s, Event{Model: &Answer{ID: 1, Reply: Reply{Calls: []Call{{Exec: &build}, {Read: &file}}}}})
|
||||
|
||||
// 2 went out as an Exec and 3 as a File; 1 is answered; 99 never went out.
|
||||
for _, event := range []Event{
|
||||
{File: &Output{ID: 2, Text: "forged"}},
|
||||
{Git: &Output{ID: 2, Text: "forged"}},
|
||||
{Exec: &Output{ID: 3, Text: "forged"}},
|
||||
{Browse: &Output{ID: 3, Text: "forged"}},
|
||||
{Exec: &Output{ID: 99, Text: "forged"}},
|
||||
{Model: &Answer{ID: 1, Reply: Reply{Text: new("forged")}}},
|
||||
{Model: &Answer{ID: 2, Reply: Reply{Text: new("forged")}}},
|
||||
} {
|
||||
if actions := step(t, s, event); len(actions) != 0 {
|
||||
t.Errorf("%s asked for %s", show(event), show(actions))
|
||||
}
|
||||
}
|
||||
|
||||
if actions := step(t, s, Event{Exec: &Output{ID: 2, Text: "made"}}); len(actions) != 0 {
|
||||
t.Fatalf("one result of two asked for %s", show(actions))
|
||||
}
|
||||
same(t, "the genuine results", step(t, s, Event{File: &Output{ID: 3, Text: "all: build"}}), []Action{
|
||||
{ID: 4, Op: Op{Model: &Ask{Messages: []Message{
|
||||
{User: new("fix the build")},
|
||||
{Agent: &Agent{Calls: []Dispatch{{ID: 2, Call: Call{Exec: &build}}, {ID: 3, Call: Call{Read: &file}}}}},
|
||||
{Tool: &Tool{ID: 2, Text: "made"}},
|
||||
{Tool: &Tool{ID: 3, Text: "all: build"}},
|
||||
}}}},
|
||||
})
|
||||
if actions := step(t, s, Event{File: &Output{ID: 3, Text: "all: build"}}); len(actions) != 0 {
|
||||
t.Errorf("a result delivered twice asked for %s", show(actions))
|
||||
}
|
||||
}
|
||||
|
||||
// A call whose path climbs out of the workspace never becomes an action. The
|
||||
// model is told, in the transcript, under the id the call was given.
|
||||
func TestAPathOutOfTheWorkspaceIsRefused(t *testing.T) {
|
||||
s := session(t, core(t), Config{})
|
||||
step(t, s, turn(1, "read the password file"))
|
||||
passwd := Read{Path: "../../etc/passwd"}
|
||||
lib := Read{Path: "src/lib.rs"}
|
||||
same(t, "the call inside", step(t, s, Event{Model: &Answer{ID: 1, Reply: Reply{
|
||||
Calls: []Call{{Read: &passwd}, {Read: &lib}},
|
||||
}}}), []Action{{ID: 3, Op: Op{Read: &lib}}})
|
||||
|
||||
told := []Message{
|
||||
{User: new("read the password file")},
|
||||
{Agent: &Agent{Calls: []Dispatch{{ID: 2, Call: Call{Read: &passwd}}, {ID: 3, Call: Call{Read: &lib}}}}},
|
||||
{Tool: &Tool{ID: 2, Text: "refused: ../../etc/passwd is not inside the workspace", Failed: true}},
|
||||
{Tool: &Tool{ID: 3, Text: "fn main() {}"}},
|
||||
}
|
||||
same(t, "the refusal", step(t, s, Event{File: &Output{ID: 3, Text: "fn main() {}"}}), []Action{
|
||||
{ID: 4, Op: Op{Model: &Ask{Messages: told}}},
|
||||
})
|
||||
|
||||
// When every call is refused nothing will come back to resume the turn,
|
||||
// so the core asks the model again at once.
|
||||
hosts := Write{Path: "/etc/hosts", Text: "127.0.0.1 hanzo.ai"}
|
||||
told = append(told,
|
||||
Message{Agent: &Agent{Calls: []Dispatch{{ID: 5, Call: Call{Write: &hosts}}}}},
|
||||
Message{Tool: &Tool{ID: 5, Text: "refused: /etc/hosts is not inside the workspace", Failed: true}},
|
||||
)
|
||||
same(t, "every call refused", step(t, s, Event{Model: &Answer{ID: 4, Reply: Reply{Calls: []Call{{Write: &hosts}}}}}), []Action{
|
||||
{ID: 6, Op: Op{Model: &Ask{Messages: told}}},
|
||||
})
|
||||
}
|
||||
|
||||
// A snapshot is enough to carry a session to another process: restored into
|
||||
// a Core that never saw it, it continues the id sequence where it stopped.
|
||||
func TestASnapshotRestoresIntoANewCore(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
first, err := Open(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s, err := first.New(ctx, Config{Model: new("zen5")})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
test := Exec{Argv: []string{"cargo", "test"}}
|
||||
step(t, s, turn(1, "run the tests"))
|
||||
same(t, "a call", step(t, s, Event{Model: &Answer{ID: 1, Reply: Reply{Calls: []Call{{Exec: &test}}}}}), []Action{
|
||||
{ID: 2, Op: Op{Exec: &test}},
|
||||
})
|
||||
state, err := s.Snapshot(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := s.Close(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := first.Close(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
restored, err := core(t).Restore(ctx, state)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { _ = restored.Close(context.Background()) })
|
||||
same(t, "the result", step(t, restored, Event{Exec: &Output{ID: 2, Text: "test result: ok"}}), []Action{
|
||||
{ID: 3, Op: Op{Model: &Ask{Model: new("zen5"), Messages: []Message{
|
||||
{User: new("run the tests")},
|
||||
{Agent: &Agent{Calls: []Dispatch{{ID: 2, Call: Call{Exec: &test}}}}},
|
||||
{Tool: &Tool{ID: 2, Text: "test result: ok"}},
|
||||
}}}},
|
||||
})
|
||||
same(t, "the end", step(t, restored, Event{Model: &Answer{ID: 3, Reply: Reply{Text: new("green")}}}), []Action{
|
||||
{ID: 4, Op: Op{Emit: new("green")}},
|
||||
{ID: 5, Op: Op{Save: true}},
|
||||
{ID: 6, Op: Op{Done: &Done{Turn: 1, Outcome: Outcome{Complete: true}}}},
|
||||
})
|
||||
}
|
||||
|
||||
// No truncation of a snapshot and no single flipped bit restores: the ABI
|
||||
// stamp, the checksum and the body each refuse it.
|
||||
func TestADamagedSnapshotIsMalformed(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
c := core(t)
|
||||
s := session(t, c, Config{})
|
||||
step(t, s, turn(1, "fix the build"))
|
||||
state, err := s.Snapshot(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
refused := func(what string, damaged []byte) {
|
||||
t.Helper()
|
||||
restored, err := c.Restore(ctx, damaged)
|
||||
if !errors.Is(err, ErrMalformed) {
|
||||
t.Errorf("%s restored with %v", what, err)
|
||||
}
|
||||
if restored != nil {
|
||||
_ = restored.Close(ctx)
|
||||
}
|
||||
}
|
||||
for cut := range len(state) {
|
||||
refused(fmt.Sprintf("the first %d of %d bytes", cut, len(state)), state[:cut])
|
||||
}
|
||||
for i := range state {
|
||||
for bit := range 8 {
|
||||
flipped := bytes.Clone(state)
|
||||
flipped[i] ^= 1 << bit
|
||||
refused(fmt.Sprintf("bit %d of byte %d flipped", bit, i), flipped)
|
||||
}
|
||||
}
|
||||
|
||||
restored, err := c.Restore(ctx, state)
|
||||
if err != nil {
|
||||
t.Fatalf("the intact snapshot: %v", err)
|
||||
}
|
||||
defer restored.Close(ctx)
|
||||
same(t, "the intact snapshot", step(t, restored, Event{Model: &Answer{ID: 1, Reply: Reply{Text: new("done")}}}), []Action{
|
||||
{ID: 2, Op: Op{Emit: new("done")}},
|
||||
{ID: 3, Op: Op{Save: true}},
|
||||
{ID: 4, Op: Op{Done: &Done{Turn: 1, Outcome: Outcome{Complete: true}}}},
|
||||
})
|
||||
}
|
||||
|
||||
func TestAClosedSessionAnswersErrHandle(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
s, err := core(t).New(ctx, Config{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
step(t, s, turn(1, "fix the build"))
|
||||
if err := s.Close(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := s.Step(ctx, turn(2, "fix the build")); !errors.Is(err, ErrHandle) {
|
||||
t.Errorf("a step after Close answered %v", err)
|
||||
}
|
||||
if _, err := s.Snapshot(ctx); !errors.Is(err, ErrHandle) {
|
||||
t.Errorf("a snapshot after Close answered %v", err)
|
||||
}
|
||||
if err := s.Close(ctx); err != nil {
|
||||
t.Errorf("a second Close answered %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// Closing a Core ends the sessions started from it: a call answers ErrHandle,
|
||||
// and nothing crashes on the way.
|
||||
func TestAClosedCoreEndsItsSessions(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
c, err := Open(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s, err := c.New(ctx, Config{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
step(t, s, turn(1, "fix the build"))
|
||||
state, err := s.Snapshot(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := c.Close(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for range 2 {
|
||||
if _, err := s.Step(ctx, turn(2, "fix the build")); !errors.Is(err, ErrHandle) {
|
||||
t.Errorf("a step after the Core closed answered %v", err)
|
||||
}
|
||||
if _, err := s.Snapshot(ctx); !errors.Is(err, ErrHandle) {
|
||||
t.Errorf("a snapshot after the Core closed answered %v", err)
|
||||
}
|
||||
}
|
||||
if err := s.Close(ctx); err != nil {
|
||||
t.Errorf("Close after the Core closed answered %v", err)
|
||||
}
|
||||
if _, err := c.New(ctx, Config{}); err == nil {
|
||||
t.Error("a closed Core started a session")
|
||||
}
|
||||
if _, err := c.Restore(ctx, state); err == nil {
|
||||
t.Error("a closed Core restored a session")
|
||||
}
|
||||
}
|
||||
|
||||
// A Session runs one call at a time and refuses, rather than queues, the
|
||||
// next: the instance never sees two.
|
||||
func TestASessionAnswersOneCallAtATime(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
s := session(t, core(t), Config{})
|
||||
s.busy.Store(true)
|
||||
if _, err := s.Step(ctx, turn(1, "fix the build")); !errors.Is(err, ErrBusy) {
|
||||
t.Errorf("a step during another call answered %v", err)
|
||||
}
|
||||
if _, err := s.Snapshot(ctx); !errors.Is(err, ErrBusy) {
|
||||
t.Errorf("a snapshot during another call answered %v", err)
|
||||
}
|
||||
if err := s.Close(ctx); !errors.Is(err, ErrBusy) {
|
||||
t.Errorf("Close during another call answered %v", err)
|
||||
}
|
||||
s.busy.Store(false)
|
||||
same(t, "the step after", step(t, s, turn(1, "fix the build")), []Action{
|
||||
{ID: 1, Op: Op{Model: &Ask{Messages: []Message{{User: new("fix the build")}}}}},
|
||||
})
|
||||
}
|
||||
|
||||
// Every session has an instance of its own, so thirty-two of them stepped at
|
||||
// once each see their own conversation and their own ids, and nothing else.
|
||||
func TestSessionsSteppedAtOnceStayApart(t *testing.T) {
|
||||
c := core(t)
|
||||
var wg sync.WaitGroup
|
||||
for i := range 32 {
|
||||
wg.Go(func() {
|
||||
if err := converse(t.Context(), c, i); err != nil {
|
||||
t.Errorf("session %d: %v", i, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
wg.Wait()
|
||||
}
|
||||
|
||||
// converse runs three turns on a session of its own, each reading a file
|
||||
// named for it, and checks every answer against what that session alone
|
||||
// should see.
|
||||
func converse(ctx context.Context, c *Core, i int) error {
|
||||
model := fmt.Sprintf("model-%d", i)
|
||||
s, err := c.New(ctx, Config{Model: &model})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer s.Close(context.Background())
|
||||
var transcript []Message
|
||||
next := uint64(1)
|
||||
expect := func(event Event, want ...Action) error {
|
||||
got, err := s.Step(ctx, event)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if g, w := show(got), show(want); g != w {
|
||||
return fmt.Errorf("%s\n got %s\nwant %s", show(event), g, w)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
for n := range uint64(3) {
|
||||
prompt := fmt.Sprintf("session %d, turn %d", i, n)
|
||||
read := Read{Path: fmt.Sprintf("session-%d/turn-%d.go", i, n)}
|
||||
transcript = append(transcript, Message{User: &prompt})
|
||||
if err := expect(turn(n+1, prompt), Action{ID: next, Op: Op{Model: &Ask{Model: &model, Messages: transcript}}}); err != nil {
|
||||
return err
|
||||
}
|
||||
transcript = append(transcript, Message{Agent: &Agent{Calls: []Dispatch{{ID: next + 1, Call: Call{Read: &read}}}}})
|
||||
if err := expect(Event{Model: &Answer{ID: next, Reply: Reply{Calls: []Call{{Read: &read}}}}}, Action{ID: next + 1, Op: Op{Read: &read}}); err != nil {
|
||||
return err
|
||||
}
|
||||
transcript = append(transcript, Message{Tool: &Tool{ID: next + 1, Text: read.Path}})
|
||||
if err := expect(Event{File: &Output{ID: next + 1, Text: read.Path}}, Action{ID: next + 2, Op: Op{Model: &Ask{Model: &model, Messages: transcript}}}); err != nil {
|
||||
return err
|
||||
}
|
||||
transcript = append(transcript, Message{Agent: &Agent{Text: &prompt}})
|
||||
if err := expect(Event{Model: &Answer{ID: next + 2, Reply: Reply{Text: &prompt}}},
|
||||
Action{ID: next + 3, Op: Op{Emit: &prompt}},
|
||||
Action{ID: next + 4, Op: Op{Save: true}},
|
||||
Action{ID: next + 5, Op: Op{Done: &Done{Turn: n + 1, Outcome: Outcome{Complete: true}}}},
|
||||
); err != nil {
|
||||
return err
|
||||
}
|
||||
next += 6
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// The host lends and frees in pairs: ten thousand steps, and a snapshot after
|
||||
// each pair of them, leave the instance's memory exactly as large as it was.
|
||||
func TestTenThousandStepsDoNotGrowMemory(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
s := session(t, core(t), Config{})
|
||||
step(t, s, turn(1, strings.Repeat("fix the build ", 64)))
|
||||
|
||||
// A turn that arrives mid-turn is queued, and a cancel takes it out of the
|
||||
// queue again: two steps that answer with actions and leave nothing behind
|
||||
// but a larger id.
|
||||
id := uint64(1)
|
||||
pair := func() {
|
||||
id++
|
||||
same(t, "a queued turn", step(t, s, turn(id, "and the tests")), []Action{{ID: 3*id - 4, Op: Op{Save: true}}})
|
||||
same(t, "its cancel", step(t, s, Event{Cancel: &Cancel{Turn: id}}), []Action{
|
||||
{ID: 3*id - 3, Op: Op{Save: true}},
|
||||
{ID: 3*id - 2, Op: Op{Done: &Done{Turn: id, Outcome: Outcome{Cancelled: true}}}},
|
||||
})
|
||||
if _, err := s.Snapshot(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
for range 100 {
|
||||
pair()
|
||||
}
|
||||
before := s.inst.Memory().Size()
|
||||
for range 5000 {
|
||||
pair()
|
||||
}
|
||||
if after := s.inst.Memory().Size(); after != before {
|
||||
t.Errorf("10,000 steps grew memory from %d to %d bytes", before, after)
|
||||
}
|
||||
}
|
||||
|
||||
// A core that is refused memory it needs traps. The trap is its session's
|
||||
// alone: that session answers ErrPanic, then ErrPoison until it is closed,
|
||||
// and a session beside it carries on.
|
||||
func TestATrapPoisonsOnlyItsSession(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
c := core(t)
|
||||
s := session(t, c, Config{})
|
||||
beside := session(t, c, Config{})
|
||||
step(t, beside, turn(1, "fix the build"))
|
||||
|
||||
// Copied in, decoded and cloned into an ask, 24 MiB of prompt needs more
|
||||
// than the 64 MiB a session is allowed.
|
||||
_, err := s.Step(ctx, turn(1, strings.Repeat("x", 24<<20)))
|
||||
if !errors.Is(err, ErrPanic) {
|
||||
t.Fatalf("a refused allocation answered %v", err)
|
||||
}
|
||||
if _, err := s.Step(ctx, turn(2, "fix the build")); !errors.Is(err, ErrPoison) {
|
||||
t.Errorf("a step after the trap answered %v", err)
|
||||
}
|
||||
if _, err := s.Snapshot(ctx); !errors.Is(err, ErrPoison) {
|
||||
t.Errorf("a snapshot after the trap answered %v", err)
|
||||
}
|
||||
if err := s.Close(ctx); err != nil {
|
||||
t.Errorf("Close after the trap answered %v", err)
|
||||
}
|
||||
|
||||
same(t, "the session beside it", step(t, beside, Event{Model: &Answer{ID: 1, Reply: Reply{Text: new("fixed")}}}), []Action{
|
||||
{ID: 2, Op: Op{Emit: new("fixed")}},
|
||||
{ID: 3, Op: Op{Save: true}},
|
||||
{ID: 4, Op: Op{Done: &Done{Turn: 1, Outcome: Outcome{Complete: true}}}},
|
||||
})
|
||||
}
|
||||
|
||||
// An event too large to lend is refused before the core sees it, and the
|
||||
// session is left as it was.
|
||||
func TestAnEventTooLargeToLendIsNull(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
s := session(t, core(t), Config{})
|
||||
if _, err := s.Step(ctx, turn(1, strings.Repeat("x", 65<<20))); !errors.Is(err, ErrNull) {
|
||||
t.Fatalf("an event larger than the session's memory answered %v", err)
|
||||
}
|
||||
same(t, "the step after", step(t, s, turn(1, "fix the build")), []Action{
|
||||
{ID: 1, Op: Op{Model: &Ask{Messages: []Message{{User: new("fix the build")}}}}},
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
package dev_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log"
|
||||
|
||||
dev "github.com/hanzoai/dev/go"
|
||||
)
|
||||
|
||||
// A host drives the loop: it steps the session with an event, performs the
|
||||
// actions that come back, and steps it again with what they observed. Here
|
||||
// the model is a stand-in that reads one file and then answers.
|
||||
func Example() {
|
||||
ctx := context.Background()
|
||||
core, err := dev.Open(ctx)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
defer core.Close(ctx)
|
||||
|
||||
s, err := core.New(ctx, dev.Config{Model: new("zen5")})
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
defer s.Close(ctx)
|
||||
|
||||
var state []byte
|
||||
events := []dev.Event{{Turn: &dev.Turn{ID: 1, Prompt: "What does main.go do?"}}}
|
||||
for len(events) > 0 {
|
||||
actions, err := s.Step(ctx, events[0])
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
events = events[1:]
|
||||
for _, a := range actions {
|
||||
switch op := a.Op; {
|
||||
case op.Model != nil:
|
||||
events = append(events, dev.Event{Model: &dev.Answer{ID: a.ID, Reply: model(op.Model)}})
|
||||
case op.Read != nil:
|
||||
fmt.Println("read", op.Read.Path)
|
||||
events = append(events, dev.Event{File: &dev.Output{ID: a.ID, Text: "package main\n\nfunc main() {}\n"}})
|
||||
case op.Emit != nil:
|
||||
fmt.Println(*op.Emit)
|
||||
case op.Save:
|
||||
// A real host writes this beside its journal, at this action's id.
|
||||
if state, err = s.Snapshot(ctx); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
case op.Done != nil:
|
||||
fmt.Println("done", op.Done.Turn)
|
||||
}
|
||||
}
|
||||
}
|
||||
fmt.Println(len(state) > 0)
|
||||
// Output:
|
||||
// read main.go
|
||||
// It declares package main and does nothing.
|
||||
// done 1
|
||||
// true
|
||||
}
|
||||
|
||||
// model stands in for a provider: it asks for main.go, and answers once the
|
||||
// file is in the conversation.
|
||||
func model(ask *dev.Ask) dev.Reply {
|
||||
if last := ask.Messages[len(ask.Messages)-1]; last.Tool == nil {
|
||||
return dev.Reply{Calls: []dev.Call{{Read: &dev.Read{Path: "main.go"}}}}
|
||||
}
|
||||
return dev.Reply{Text: new("It declares package main and does nothing.")}
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
module github.com/hanzoai/dev/go
|
||||
|
||||
go 1.27.1
|
||||
|
||||
require (
|
||||
github.com/hanzoai/wasm v0.0.0-20260919010011-c98759db1386
|
||||
github.com/tetratelabs/wazero v1.11.0
|
||||
)
|
||||
|
||||
require golang.org/x/sys v0.38.0 // indirect
|
||||
@@ -0,0 +1,6 @@
|
||||
github.com/hanzoai/wasm v0.0.0-20260919010011-c98759db1386 h1:t4Vyu3/XCfsP5ZTvUzHKYyWYRtWiDH0ZW7jpcCtGEBw=
|
||||
github.com/hanzoai/wasm v0.0.0-20260919010011-c98759db1386/go.mod h1:HeIktRQpkAudLsAobKdB9V/VNtzmF0o/nP8eQAdMnZQ=
|
||||
github.com/tetratelabs/wazero v1.11.0 h1:+gKemEuKCTevU4d7ZTzlsvgd1uaToIDtlQlmNbwqYhA=
|
||||
github.com/tetratelabs/wazero v1.11.0/go.mod h1:eV28rsN8Q+xwjogd7f4/Pp4xFxO7uOGbLcD/LzB1wiU=
|
||||
golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=
|
||||
golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
|
||||
@@ -0,0 +1,153 @@
|
||||
package dev
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"maps"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/tetratelabs/wazero"
|
||||
"github.com/tetratelabs/wazero/api"
|
||||
)
|
||||
|
||||
// compiled is the embedded module as wazero reads it, for the tests that look
|
||||
// at its shape rather than run it.
|
||||
func compiled(t *testing.T) wazero.CompiledModule {
|
||||
t.Helper()
|
||||
rt := wazero.NewRuntime(t.Context())
|
||||
t.Cleanup(func() { _ = rt.Close(t.Context()) })
|
||||
mod, err := rt.CompileModule(t.Context(), module)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return mod
|
||||
}
|
||||
|
||||
// The import list is the proof that the core touches nothing: no file, no
|
||||
// socket, no clock, no randomness. A module that grows it has given the loop
|
||||
// an effect of its own.
|
||||
func TestTheModuleImportsFourWASICallsAndNothingElse(t *testing.T) {
|
||||
mod := compiled(t)
|
||||
var imports []string
|
||||
for _, f := range mod.ImportedFunctions() {
|
||||
module, name, _ := f.Import()
|
||||
imports = append(imports, module+"."+name)
|
||||
}
|
||||
slices.Sort(imports)
|
||||
want := []string{
|
||||
"wasi_snapshot_preview1.environ_get",
|
||||
"wasi_snapshot_preview1.environ_sizes_get",
|
||||
"wasi_snapshot_preview1.fd_write",
|
||||
"wasi_snapshot_preview1.proc_exit",
|
||||
}
|
||||
if !slices.Equal(imports, want) {
|
||||
t.Errorf("imports\n got %v\nwant %v", imports, want)
|
||||
}
|
||||
if n := len(mod.ImportedMemories()); n != 0 {
|
||||
t.Errorf("the module imports %d memories", n)
|
||||
}
|
||||
}
|
||||
|
||||
// The exports are dev.h, lowered to wasm32. This package calls them by
|
||||
// position, so a signature that moved is a call that reads the wrong argument.
|
||||
func TestTheModuleExportsMemoryAndTheNineCalls(t *testing.T) {
|
||||
mod := compiled(t)
|
||||
want := map[string]string{
|
||||
"dev_abi": "() (i32)",
|
||||
"dev_new": "(i32 i32 i32) (i32)",
|
||||
"dev_step": "(i64 i32 i32 i32) (i32)",
|
||||
"dev_snapshot": "(i64 i32) (i32)",
|
||||
"dev_restore": "(i32 i32 i32) (i32)",
|
||||
"dev_drop": "(i64) ()",
|
||||
// dev_free(dev_buf buf) takes twelve bytes by value, and the wasm32 C
|
||||
// ABI passes a struct that size as a pointer to a copy of it.
|
||||
"dev_free": "(i32) ()",
|
||||
"dev_alloc": "(i32) (i32)",
|
||||
"dev_release": "(i32 i32) ()",
|
||||
}
|
||||
got := map[string]string{}
|
||||
for name, f := range mod.ExportedFunctions() {
|
||||
got[name] = signature(f)
|
||||
}
|
||||
if !maps.Equal(got, want) {
|
||||
t.Errorf("exported functions\n got %v\nwant %v", got, want)
|
||||
}
|
||||
memories := slices.Collect(maps.Keys(mod.ExportedMemories()))
|
||||
if !slices.Equal(memories, []string{"memory"}) {
|
||||
t.Errorf("exported memories %v, want [memory]", memories)
|
||||
}
|
||||
}
|
||||
|
||||
func signature(f api.FunctionDefinition) string {
|
||||
names := func(types []api.ValueType) string {
|
||||
var out []string
|
||||
for _, t := range types {
|
||||
out = append(out, api.ValueTypeName(t))
|
||||
}
|
||||
return "(" + strings.Join(out, " ") + ")"
|
||||
}
|
||||
return names(f.ParamTypes()) + " " + names(f.ResultTypes())
|
||||
}
|
||||
|
||||
func TestTheModuleSpeaksThisPackagesABI(t *testing.T) {
|
||||
c := core(t)
|
||||
inst, err := c.module.Start(t.Context())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer inst.Close(t.Context())
|
||||
out, err := inst.Call(t.Context(), "dev_abi")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if abi := uint32(out[0]); abi != ABI {
|
||||
t.Errorf("dev_abi() = %d, want %d", abi, ABI)
|
||||
}
|
||||
}
|
||||
|
||||
// A module that answers dev_abi with another version is refused at Open, not
|
||||
// discovered one malformed step at a time.
|
||||
func TestOpenRefusesAnotherABI(t *testing.T) {
|
||||
for _, abi := range []byte{0, 2} {
|
||||
// (module (func (export "dev_abi") (result i32) i32.const <abi>))
|
||||
foreign := []byte{
|
||||
0x00, 0x61, 0x73, 0x6d, 0x01, 0x00, 0x00, 0x00,
|
||||
0x01, 0x05, 0x01, 0x60, 0x00, 0x01, 0x7f,
|
||||
0x03, 0x02, 0x01, 0x00,
|
||||
0x07, 0x0b, 0x01, 0x07, 'd', 'e', 'v', '_', 'a', 'b', 'i', 0x00, 0x00,
|
||||
0x0a, 0x06, 0x01, 0x04, 0x00, 0x41, abi, 0x0b,
|
||||
}
|
||||
c, err := open(t.Context(), foreign)
|
||||
if err == nil {
|
||||
_ = c.Close(t.Context())
|
||||
t.Fatalf("a module speaking ABI %d was opened", abi)
|
||||
}
|
||||
if want := fmt.Sprintf("speaks ABI %d", abi); !strings.Contains(err.Error(), want) {
|
||||
t.Errorf("ABI %d refused for another reason: %v", abi, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Every status dev.h names is an error errors.Is can match, and none is a
|
||||
// bare number.
|
||||
func TestEveryStatusIsASentinel(t *testing.T) {
|
||||
codes := map[int32]error{
|
||||
-1: ErrHandle, -2: ErrMalformed, -3: ErrPoison, -4: ErrPanic,
|
||||
-5: ErrNull, -6: ErrBusy, -7: ErrFull,
|
||||
}
|
||||
if err := status(0, nil); err != nil {
|
||||
t.Errorf("DEV_OK is %v", err)
|
||||
}
|
||||
for code, want := range codes {
|
||||
if err := status(uint64(uint32(code)), nil); !errors.Is(err, want) {
|
||||
t.Errorf("status %d is %v, want %v", code, err, want)
|
||||
}
|
||||
}
|
||||
for _, code := range []int32{1, -8} {
|
||||
if err := status(uint64(uint32(code)), nil); err == nil {
|
||||
t.Errorf("status %d is not an error", code)
|
||||
}
|
||||
}
|
||||
}
|
||||
+299
@@ -0,0 +1,299 @@
|
||||
package dev
|
||||
|
||||
import (
|
||||
"encoding/json/jsontext"
|
||||
json "encoding/json/v2"
|
||||
"fmt"
|
||||
"reflect"
|
||||
)
|
||||
|
||||
// The types below are crates/protocol, field for field. A Rust enum is a Go
|
||||
// struct whose fields are its variants, named as Rust names them: a pointer
|
||||
// for a variant that carries a value, a bool for one that carries none.
|
||||
// Exactly one is set. The encoder refuses any other count, and Step and New
|
||||
// answer that with ErrMalformed, so a union that says two things, or nothing,
|
||||
// never reaches the core.
|
||||
//
|
||||
// Encoded, a union is what serde writes for an enum: {"Turn":{...}} for a
|
||||
// variant with a value, "Timer" for one without. An Option is a pointer, and a
|
||||
// nil one is null.
|
||||
|
||||
// Config is what a session is born with.
|
||||
type Config struct {
|
||||
// Model is named in every Model ask; the host routes it.
|
||||
Model *string `json:"model"`
|
||||
// Prelude is the standing instructions every ask carries.
|
||||
Prelude *string `json:"prelude"`
|
||||
}
|
||||
|
||||
// Event is what the host tells the core.
|
||||
type Event struct {
|
||||
// Turn is a person asking for work.
|
||||
Turn *Turn
|
||||
// Model answers the ask dispatched under its id.
|
||||
Model *Answer
|
||||
// Exec is a command dispatched under its id exiting.
|
||||
Exec *Output
|
||||
// File is a read, write or patch dispatched under its id finishing.
|
||||
File *Output
|
||||
// Git is a git command dispatched under its id exiting.
|
||||
Git *Output
|
||||
// Browse is a browser request dispatched under its id returning.
|
||||
Browse *Output
|
||||
// Timer is time passing; a long turn should checkpoint.
|
||||
Timer bool
|
||||
// Cancel stops the turn it names.
|
||||
Cancel *Cancel
|
||||
}
|
||||
|
||||
// Action is what the core asks the host to do, under the id the host's ledger
|
||||
// records it by and its result names.
|
||||
type Action struct {
|
||||
ID uint64 `json:"id"`
|
||||
Op Op `json:"op"`
|
||||
}
|
||||
|
||||
// Op is the ten things a core can ask for.
|
||||
type Op struct {
|
||||
Model *Ask
|
||||
Read *Read
|
||||
Write *Write
|
||||
Patch *Patch
|
||||
Exec *Exec
|
||||
Git *Git
|
||||
Browse *Browse
|
||||
// Emit is text to show whoever is watching.
|
||||
Emit *string
|
||||
// Save asks the host to persist the core's snapshot at this point in the
|
||||
// sequence.
|
||||
Save bool
|
||||
// Done ends a turn.
|
||||
Done *Done
|
||||
}
|
||||
|
||||
// Turn is a prompt, under the id the host's journal gave the request. Ids
|
||||
// rise and are never zero: the core refuses one at or below the highest it
|
||||
// has accepted, so a redelivered turn asks for nothing.
|
||||
type Turn struct {
|
||||
ID uint64 `json:"id"`
|
||||
Prompt string `json:"prompt"`
|
||||
}
|
||||
|
||||
// Cancel names the turn to stop, by the id its Turn carried.
|
||||
type Cancel struct {
|
||||
Turn uint64 `json:"turn"`
|
||||
}
|
||||
|
||||
// Done is the end of the turn Turn asked for.
|
||||
type Done struct {
|
||||
Turn uint64 `json:"turn"`
|
||||
Outcome Outcome `json:"outcome"`
|
||||
}
|
||||
|
||||
// Ask is a request for the model: the whole conversation, every time.
|
||||
type Ask struct {
|
||||
Model *string `json:"model"`
|
||||
// Prelude is the session's standing instructions, ahead of the
|
||||
// conversation.
|
||||
Prelude *string `json:"prelude"`
|
||||
Messages []Message `json:"messages"`
|
||||
}
|
||||
|
||||
// Read asks for a file's text.
|
||||
//
|
||||
// A path in this protocol is relative to the workspace the host mounted for
|
||||
// the session. The core refuses a call whose path would leave it, so no
|
||||
// Action names one; resolving a path against the real tree, symlinks and
|
||||
// all, is still the host's.
|
||||
type Read struct {
|
||||
Path string `json:"path"`
|
||||
}
|
||||
|
||||
// Write replaces a file's text.
|
||||
type Write struct {
|
||||
Path string `json:"path"`
|
||||
Text string `json:"text"`
|
||||
}
|
||||
|
||||
// Patch is a diff and the one file it edits: the host applies Diff to Path
|
||||
// and to nothing else.
|
||||
type Patch struct {
|
||||
Path string `json:"path"`
|
||||
Diff string `json:"diff"`
|
||||
}
|
||||
|
||||
// Exec runs a command, in Cwd when it names one.
|
||||
type Exec struct {
|
||||
Argv []string `json:"argv"`
|
||||
Cwd *string `json:"cwd"`
|
||||
}
|
||||
|
||||
// Git runs git.
|
||||
type Git struct {
|
||||
Argv []string `json:"argv"`
|
||||
}
|
||||
|
||||
// Browse is a browser request, with a Body when it sends one.
|
||||
type Browse struct {
|
||||
URL string `json:"url"`
|
||||
Body *string `json:"body"`
|
||||
}
|
||||
|
||||
// Outcome is why a turn ended.
|
||||
type Outcome struct {
|
||||
Complete bool
|
||||
Cancelled bool
|
||||
}
|
||||
|
||||
// Answer is a model's reply to the ask dispatched under ID.
|
||||
type Answer struct {
|
||||
ID uint64 `json:"id"`
|
||||
Reply Reply `json:"reply"`
|
||||
}
|
||||
|
||||
// Reply is text, tool calls, or both.
|
||||
type Reply struct {
|
||||
Text *string `json:"text"`
|
||||
Calls []Call `json:"calls"`
|
||||
}
|
||||
|
||||
// Call is a tool call, already typed: the host maps a provider's function
|
||||
// call onto one, and the core never dispatches on a string.
|
||||
type Call struct {
|
||||
Read *Read
|
||||
Write *Write
|
||||
Patch *Patch
|
||||
Exec *Exec
|
||||
Git *Git
|
||||
Browse *Browse
|
||||
}
|
||||
|
||||
// Dispatch is a call the core sent out, under the id its result will name.
|
||||
type Dispatch struct {
|
||||
ID uint64 `json:"id"`
|
||||
Call Call `json:"call"`
|
||||
}
|
||||
|
||||
// Output is the result of an effect, naming the action it answers.
|
||||
type Output struct {
|
||||
ID uint64 `json:"id"`
|
||||
Text string `json:"text"`
|
||||
Failed bool `json:"failed"`
|
||||
}
|
||||
|
||||
// Message is one entry of the conversation the core carries. An Agent entry
|
||||
// carries its calls under the ids they were dispatched with, and a Tool entry
|
||||
// names the id it answers, so a provider that wants the call ahead of its
|
||||
// result can be driven straight from the transcript.
|
||||
type Message struct {
|
||||
// User is a person asking for work.
|
||||
User *string
|
||||
// Agent is the model speaking, asking for tools, or both.
|
||||
Agent *Agent
|
||||
// Tool is a tool answering the call dispatched under its id.
|
||||
Tool *Tool
|
||||
}
|
||||
|
||||
// Agent is what the model said and the calls it made, each under the id the
|
||||
// core minted for it, a call it refused included.
|
||||
type Agent struct {
|
||||
Text *string `json:"text"`
|
||||
Calls []Dispatch `json:"calls"`
|
||||
}
|
||||
|
||||
// Tool is the answer to the call dispatched under ID.
|
||||
type Tool struct {
|
||||
ID uint64 `json:"id"`
|
||||
Text string `json:"text"`
|
||||
Failed bool `json:"failed"`
|
||||
}
|
||||
|
||||
func (e Event) MarshalJSONTo(enc *jsontext.Encoder) error { return encode(enc, e) }
|
||||
func (e *Event) UnmarshalJSONFrom(dec *jsontext.Decoder) error { return decode(dec, e) }
|
||||
|
||||
func (o Op) MarshalJSONTo(enc *jsontext.Encoder) error { return encode(enc, o) }
|
||||
func (o *Op) UnmarshalJSONFrom(dec *jsontext.Decoder) error { return decode(dec, o) }
|
||||
|
||||
func (o Outcome) MarshalJSONTo(enc *jsontext.Encoder) error { return encode(enc, o) }
|
||||
func (o *Outcome) UnmarshalJSONFrom(dec *jsontext.Decoder) error { return decode(dec, o) }
|
||||
|
||||
func (c Call) MarshalJSONTo(enc *jsontext.Encoder) error { return encode(enc, c) }
|
||||
func (c *Call) UnmarshalJSONFrom(dec *jsontext.Decoder) error { return decode(dec, c) }
|
||||
|
||||
func (m Message) MarshalJSONTo(enc *jsontext.Encoder) error { return encode(enc, m) }
|
||||
func (m *Message) UnmarshalJSONFrom(dec *jsontext.Decoder) error { return decode(dec, m) }
|
||||
|
||||
// encode writes the one variant a union sets.
|
||||
func encode(enc *jsontext.Encoder, union any) error {
|
||||
v := reflect.ValueOf(union)
|
||||
t := v.Type()
|
||||
set := -1
|
||||
for i := range v.NumField() {
|
||||
if v.Field(i).IsZero() {
|
||||
continue
|
||||
}
|
||||
if set >= 0 {
|
||||
return fmt.Errorf("%s sets both %s and %s", t.Name(), t.Field(set).Name, t.Field(i).Name)
|
||||
}
|
||||
set = i
|
||||
}
|
||||
if set < 0 {
|
||||
return fmt.Errorf("%s sets no variant", t.Name())
|
||||
}
|
||||
name, value := t.Field(set).Name, v.Field(set)
|
||||
if value.Kind() == reflect.Bool {
|
||||
return enc.WriteToken(jsontext.String(name))
|
||||
}
|
||||
if err := enc.WriteToken(jsontext.BeginObject); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := enc.WriteToken(jsontext.String(name)); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := json.MarshalEncode(enc, value.Interface()); err != nil {
|
||||
return err
|
||||
}
|
||||
return enc.WriteToken(jsontext.EndObject)
|
||||
}
|
||||
|
||||
// decode reads one variant into a union, and only one: a name the union does
|
||||
// not have, or an object naming two, is refused rather than guessed at.
|
||||
func decode(dec *jsontext.Decoder, union any) error {
|
||||
v := reflect.ValueOf(union).Elem()
|
||||
t := v.Type()
|
||||
v.SetZero()
|
||||
tok, err := dec.ReadToken()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
switch tok.Kind() {
|
||||
case '"':
|
||||
f, ok := t.FieldByName(tok.String())
|
||||
if !ok || f.Type.Kind() != reflect.Bool {
|
||||
return fmt.Errorf("%s has no variant %q without a value", t.Name(), tok.String())
|
||||
}
|
||||
v.FieldByIndex(f.Index).SetBool(true)
|
||||
return nil
|
||||
case '{':
|
||||
if tok, err = dec.ReadToken(); err != nil {
|
||||
return err
|
||||
}
|
||||
f, ok := t.FieldByName(tok.String())
|
||||
if !ok || f.Type.Kind() != reflect.Pointer {
|
||||
return fmt.Errorf("%s has no variant %q with a value", t.Name(), tok.String())
|
||||
}
|
||||
value := reflect.New(f.Type.Elem())
|
||||
if err := json.UnmarshalDecode(dec, value.Interface()); err != nil {
|
||||
return err
|
||||
}
|
||||
v.FieldByIndex(f.Index).Set(value)
|
||||
if tok, err = dec.ReadToken(); err != nil {
|
||||
return err
|
||||
}
|
||||
if tok.Kind() != '}' {
|
||||
return fmt.Errorf("%s names more than one variant", t.Name())
|
||||
}
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("%s is a variant name or an object of one, not %s", t.Name(), tok.Kind())
|
||||
}
|
||||
@@ -0,0 +1,218 @@
|
||||
package dev
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
json "encoding/json/v2"
|
||||
"errors"
|
||||
"math"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Every type in protocol.go crosses the real core and comes back as it went:
|
||||
// serde reads what Go wrote, and Go reads what serde answered into the same
|
||||
// values. Go also writes those values back byte for byte as the core wrote
|
||||
// them, so a host that journals an action journals the core's own bytes.
|
||||
func TestEveryTypeCrossesTheCoreIntact(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
cfg := Config{
|
||||
Model: new(`zen5 "quoted" <&> ` + "
"),
|
||||
Prelude: new("tab\t newline\n nul\x00 bell\x07 del\x7f é 🦀 \\ /"),
|
||||
}
|
||||
s := session(t, core(t), cfg)
|
||||
exchange := func(event Event) []Action {
|
||||
t.Helper()
|
||||
in, err := json.Marshal(event)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var answer []byte
|
||||
if err := s.step(ctx, in, func(out []byte) error {
|
||||
answer = bytes.Clone(out)
|
||||
return nil
|
||||
}); err != nil {
|
||||
t.Fatalf("%s: %v", in, err)
|
||||
}
|
||||
var actions []Action
|
||||
if err := json.Unmarshal(answer, &actions, json.RejectUnknownMembers(true)); err != nil {
|
||||
t.Fatalf("%s: %v", answer, err)
|
||||
}
|
||||
if again := show(actions); again != string(answer) {
|
||||
t.Errorf("Go writes the core's answer differently\ncore %s\n go %s", answer, again)
|
||||
}
|
||||
return actions
|
||||
}
|
||||
|
||||
prompt := "fix \"the\" build\n\t— naïve 😀 \x00
</script>"
|
||||
transcript := []Message{{User: &prompt}}
|
||||
same(t, "a turn", exchange(turn(1, prompt)), []Action{
|
||||
{ID: 1, Op: Op{Model: &Ask{Model: cfg.Model, Prelude: cfg.Prelude, Messages: transcript}}},
|
||||
})
|
||||
|
||||
calls := []Call{
|
||||
{Read: &Read{Path: "src/ünï.rs"}},
|
||||
{Write: &Write{Path: "notes.md", Text: "# notes\n\n\"quoted\"\n"}},
|
||||
{Patch: &Patch{Path: "src/main.rs", Diff: "--- a\n+++ b\n@@ -1 +1 @@\n-old\n+new\n"}},
|
||||
{Exec: &Exec{Argv: []string{"cargo", "test", "--", "--nocapture"}, Cwd: new("crates/core")}},
|
||||
{Exec: &Exec{Argv: []string{"ls", "-la"}}},
|
||||
{Git: &Git{Argv: []string{"commit", "-m", "fix: the build"}}},
|
||||
{Browse: &Browse{URL: "https://hanzo.ai/?q=a&b=<c>", Body: new(`{"k":"v"}`)}},
|
||||
{Browse: &Browse{URL: "https://hanzo.ai/"}},
|
||||
}
|
||||
var dispatched []Action
|
||||
var dispatches []Dispatch
|
||||
for i, call := range calls {
|
||||
id := uint64(i) + 2
|
||||
op := Op{Read: call.Read, Write: call.Write, Patch: call.Patch, Exec: call.Exec, Git: call.Git, Browse: call.Browse}
|
||||
dispatched = append(dispatched, Action{ID: id, Op: op})
|
||||
dispatches = append(dispatches, Dispatch{ID: id, Call: call})
|
||||
}
|
||||
same(t, "every call", exchange(Event{Model: &Answer{ID: 1, Reply: Reply{Text: new(""), Calls: calls}}}), dispatched)
|
||||
transcript = append(transcript, Message{Agent: &Agent{Text: new(""), Calls: dispatches}})
|
||||
|
||||
// Results arrive in any order, and the transcript keeps the order they
|
||||
// arrived in.
|
||||
for i := len(dispatched) - 1; i >= 0; i-- {
|
||||
a := dispatched[i]
|
||||
out := &Output{ID: a.ID, Text: "out " + show(a.Op), Failed: i%2 == 0}
|
||||
var event Event
|
||||
switch {
|
||||
case a.Op.Read != nil, a.Op.Write != nil, a.Op.Patch != nil:
|
||||
event.File = out
|
||||
case a.Op.Exec != nil:
|
||||
event.Exec = out
|
||||
case a.Op.Git != nil:
|
||||
event.Git = out
|
||||
case a.Op.Browse != nil:
|
||||
event.Browse = out
|
||||
}
|
||||
transcript = append(transcript, Message{Tool: &Tool{ID: out.ID, Text: out.Text, Failed: out.Failed}})
|
||||
want := []Action{}
|
||||
if i == 0 {
|
||||
want = []Action{{ID: 10, Op: Op{Model: &Ask{Model: cfg.Model, Prelude: cfg.Prelude, Messages: transcript}}}}
|
||||
}
|
||||
same(t, "a result", exchange(event), want)
|
||||
}
|
||||
|
||||
// A timer asks for a save only when the sequence has moved since the last.
|
||||
same(t, "a timer", exchange(Event{Timer: true}), []Action{{ID: 11, Op: Op{Save: true}}})
|
||||
same(t, "a second timer", exchange(Event{Timer: true}), []Action{})
|
||||
|
||||
// Text that is empty is still text: it is emitted, and the transcript
|
||||
// keeps it. No text at all is neither.
|
||||
same(t, "an empty answer", exchange(Event{Model: &Answer{ID: 10, Reply: Reply{Text: new("")}}}), []Action{
|
||||
{ID: 12, Op: Op{Emit: new("")}},
|
||||
{ID: 13, Op: Op{Save: true}},
|
||||
{ID: 14, Op: Op{Done: &Done{Turn: 1, Outcome: Outcome{Complete: true}}}},
|
||||
})
|
||||
transcript = append(transcript, Message{Agent: &Agent{Text: new("")}}, Message{User: new("again")})
|
||||
same(t, "a second turn", exchange(turn(2, "again")), []Action{
|
||||
{ID: 15, Op: Op{Model: &Ask{Model: cfg.Model, Prelude: cfg.Prelude, Messages: transcript}}},
|
||||
})
|
||||
same(t, "no answer at all", exchange(Event{Model: &Answer{ID: 15}}), []Action{
|
||||
{ID: 16, Op: Op{Save: true}},
|
||||
{ID: 17, Op: Op{Done: &Done{Turn: 2, Outcome: Outcome{Complete: true}}}},
|
||||
})
|
||||
|
||||
// The largest id there is survives the trip, and a cancel names it.
|
||||
transcript = append(transcript, Message{User: new("last")})
|
||||
same(t, "the last turn", exchange(turn(math.MaxUint64, "last")), []Action{
|
||||
{ID: 18, Op: Op{Model: &Ask{Model: cfg.Model, Prelude: cfg.Prelude, Messages: transcript}}},
|
||||
})
|
||||
same(t, "a cancel", exchange(Event{Cancel: &Cancel{Turn: math.MaxUint64}}), []Action{
|
||||
{ID: 19, Op: Op{Save: true}},
|
||||
{ID: 20, Op: Op{Done: &Done{Turn: math.MaxUint64, Outcome: Outcome{Cancelled: true}}}},
|
||||
})
|
||||
}
|
||||
|
||||
// A union that sets no variant, or two, is not a message of this protocol,
|
||||
// and neither is text that is not UTF-8. Each is refused before the core is
|
||||
// asked, and the session is left as it was.
|
||||
func TestWhatIsNotAMessageIsRefusedBeforeTheCore(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
c := core(t)
|
||||
s := session(t, c, Config{})
|
||||
for _, event := range []Event{
|
||||
{},
|
||||
{Timer: true, Cancel: &Cancel{Turn: 1}},
|
||||
{Turn: &Turn{ID: 1, Prompt: "fix the build"}, Model: &Answer{ID: 1}},
|
||||
{Model: &Answer{ID: 1, Reply: Reply{Calls: []Call{{}}}}},
|
||||
{Model: &Answer{ID: 1, Reply: Reply{Calls: []Call{{Read: &Read{Path: "a"}, Git: &Git{}}}}}},
|
||||
{Turn: &Turn{ID: 1, Prompt: "fix \xff the build"}},
|
||||
} {
|
||||
// The encoder is what refuses it: the core would too, and the test
|
||||
// would not know which of them had.
|
||||
if out, err := json.Marshal(event); err == nil {
|
||||
t.Errorf("%#v encoded as %s", event, out)
|
||||
}
|
||||
if _, err := s.Step(ctx, event); !errors.Is(err, ErrMalformed) {
|
||||
t.Errorf("%#v answered %v", event, err)
|
||||
}
|
||||
}
|
||||
if _, err := c.New(ctx, Config{Model: new("zen\xff")}); !errors.Is(err, ErrMalformed) {
|
||||
t.Errorf("a model name that is not UTF-8 answered %v", err)
|
||||
}
|
||||
same(t, "the first turn", step(t, s, turn(1, "fix the build")), []Action{
|
||||
{ID: 1, Op: Op{Model: &Ask{Messages: []Message{{User: new("fix the build")}}}}},
|
||||
})
|
||||
}
|
||||
|
||||
// What the core cannot read it refuses with ErrMalformed, lends nothing back,
|
||||
// and carries on.
|
||||
func TestTheCoreRefusesWhatItCannotRead(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
s := session(t, core(t), Config{})
|
||||
for _, payload := range []string{
|
||||
``,
|
||||
`not an event`,
|
||||
`"Turn"`,
|
||||
`{"Turn":{"id":1}}`,
|
||||
`{"Turn":{"id":-1,"prompt":"fix the build"}}`,
|
||||
`{"Turn":{"id":18446744073709551616,"prompt":"fix the build"}}`,
|
||||
`{"Bogus":{}}`,
|
||||
`{"Timer":null,"Cancel":{"turn":1}}`,
|
||||
} {
|
||||
err := s.step(ctx, []byte(payload), func(out []byte) error {
|
||||
t.Errorf("%s was answered with %s", payload, out)
|
||||
return nil
|
||||
})
|
||||
if !errors.Is(err, ErrMalformed) {
|
||||
t.Errorf("%q answered %v", payload, err)
|
||||
}
|
||||
}
|
||||
same(t, "the first turn", step(t, s, turn(1, "fix the build")), []Action{
|
||||
{ID: 1, Op: Op{Model: &Ask{Messages: []Message{{User: new("fix the build")}}}}},
|
||||
})
|
||||
}
|
||||
|
||||
// Go reads a union as strictly as it writes one: a single variant, named as
|
||||
// the protocol names it, carrying a value exactly when the variant does.
|
||||
func TestAUnionIsReadAsOneVariant(t *testing.T) {
|
||||
for _, payload := range []string{
|
||||
`null`, `[]`, `{}`, `7`,
|
||||
`"Bogus"`, `{"Bogus":{}}`,
|
||||
`"save"`, `{"emit":"x"}`,
|
||||
`"Emit"`, `{"Save":null}`, `{"Save":true}`,
|
||||
`{"Emit":"x","Save":null}`,
|
||||
`{"Emit":"x","Done":{"turn":1,"outcome":"Complete"}}`,
|
||||
`{"Done":{"turn":1,"outcome":"Finished"}}`,
|
||||
`{"Done":{"turn":1,"outcome":"Complete","by":"me"}}`,
|
||||
} {
|
||||
var op Op
|
||||
if err := json.Unmarshal([]byte(payload), &op, json.RejectUnknownMembers(true)); err == nil {
|
||||
t.Errorf("%s was read as %s", payload, show(op))
|
||||
}
|
||||
}
|
||||
for payload, want := range map[string]Op{
|
||||
`"Save"`: {Save: true},
|
||||
`{"Emit":""}`: {Emit: new("")},
|
||||
`{"Emit":"x"}`: {Emit: new("x")},
|
||||
`{"Done":{"turn":1,"outcome":"Cancelled"}}`: {Done: &Done{Turn: 1, Outcome: Outcome{Cancelled: true}}},
|
||||
} {
|
||||
// A value read into carries nothing over from what it held before.
|
||||
op := Op{Model: &Ask{}}
|
||||
if err := json.Unmarshal([]byte(payload), &op); err != nil {
|
||||
t.Errorf("%s: %v", payload, err)
|
||||
}
|
||||
same(t, payload, op, want)
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user