Compare commits

...
Author SHA1 Message Date
zeekay 8884afeba4 sandbox: the pod stops overriding what the image already said
`command: ["sleep","infinity"]` was set for every class. It reads like a sentence
about what a sandbox is — a place to run commands, not a program — written in the
one field that cannot say only that.

A Kubernetes `command` with no `args` replaces the image's ENTRYPOINT *and* its
CMD, together. So it did not ADD `sleep infinity`; it DELETED whatever each class
had decided, and two things went with it:

  tini    the image ends ENTRYPOINT ["/usr/bin/tini","--"] so that something
          reaps. Overridden, PID 1 was `sleep`, which reaps nothing — and a
          sandbox's children are compilers, language servers and browsers started
          through the exec subresource, whose zombies then accumulate for the
          whole lease.
  the screen
          the desktop class's CMD is `sandbox-desktop`, the script that starts
          Xvfb, openbox, x0vncserver and websockify. It never ran. A class whose
          entire reason to exist is a screen had no X server and nothing on the
          VNC port its image EXPOSEs.

Measured in a live desktop pod on hanzo-k8s before this change: /proc/1/cmdline
was `sleep infinity`, `ss -ltn` was empty, `DISPLAY=:1 xdpyinfo` failed. Running
/usr/local/bin/sandbox-desktop by hand in that same pod brought X up on :1 and
left 127.0.0.1:5900 and 127.0.0.1:6080 LISTENing — nothing was missing from the
image.

The fix is not a second field here that decides per class; that is one fact in
two places, and they drift. The image already says it once per class — exec and
dev end CMD ["sleep","infinity"], desktop ends CMD ["sandbox-desktop"], which
itself ends `exec sleep infinity`. Naming neither here lets that stand, and
podspec_test.go refuses `command` or `args` on any class so it cannot come back.
2026-08-06 14:28:28 -07:00
zeekay 6dc3874e43 coding: a sandbox run is a computer, so the wire says what kind
The runtime already grew `tool`, an optional repo and `desktop`; this is the
cloud half of that contract.

`tool` (dev|claude|codex|python|node) and `desktop` are carried, not
interpreted. Cloud ships a NAME; the runtime owns the name→argv table, so
adding a tool is one edit over there and none here. `desktop` selects an image
variant — a tag — and nothing on this side branches on it.

THE REPO IS OPTIONAL, AND THE CREDENTIAL LIVES INSIDE IT. Every git field is
`omitempty` and `credential` is a POINTER: a value type always marshals, so a
repo-less run would still ship a blank `credential` object and the runtime could
not tell an absent grant from an empty one. A caller that supplies a credential
with no repo is REFUSED rather than quietly trimmed — silently dropping a secret
hides the bug that minted it, and the runtime says the same thing at its own
boundary, so both ends agree.

`Secret` is now true only when the body actually carries a credential. It was
unconditional, which meant the cleartext guard — a rule written to protect a git
token — refused every research and bare-exec run that has no token to protect.

SANDBOX_URL names where a run goes. A sandbox is not the bot: coding, deep
research and bare exec all want a computer to run something in, and none of them
wants the service that runs Slack channels. BOT_GATEWAY_URL stays correct for
bot traffic and is accepted here for one release, then deleted.

The transport learns none of this. `Call.Base` is a DESTINATION, which is a
transport concern; the caller resolves the address because resolving it inside
transport.go would mean that file learning what a run is — the exact line its
header draws. requireSecure now checks the base the call will ACTUALLY use;
checking the default while the bytes went elsewhere was a guard on the wrong hop.
2026-08-06 14:12:08 -07:00
7 changed files with 225 additions and 28 deletions
+1 -1
View File
@@ -85,7 +85,7 @@ func mountRelay(app cloud.Router, deps cloud.Deps) error {
return fmt.Errorf("bots.mountRelay: nil deps.Logger")
}
s := &relay{
target: executorURL(),
target: executorURL(""),
log: deps.Logger.New("subsystem", "bots"),
cc: &http.Client{Timeout: 60 * time.Second},
}
+25 -8
View File
@@ -96,6 +96,17 @@ type Call struct {
User string
Body any
Secret bool
// Base overrides the destination for THIS call. Empty means the bot runtime.
//
// This stays a transport concern and not a meaning one: a caller names WHERE
// its bytes go, and this file still does not know a coding run from a channel
// relay. It exists because a sandbox is not the bot — deep research, bare
// exec and coding all want a SANDBOX, and whatever runs sandboxes may not be
// the service that runs channels. Resolving that address inside here would
// mean this file learning what a run is, which is exactly the line the header
// draws.
Base string
}
// Do invokes c and discards any response payload — the command form. It returns
@@ -173,7 +184,7 @@ func Stream(ctx context.Context, c Call, fn func(msg []byte)) error {
// off: the runtime then fails the request closed at its own auth gate.
func send(ctx context.Context, c Call, method, accept string) (*http.Response, error) {
if c.Secret {
if err := requireSecure(); err != nil {
if err := requireSecure(c.Base); err != nil {
return nil, err
}
}
@@ -185,7 +196,7 @@ func send(ctx context.Context, c Call, method, accept string) (*http.Response, e
}
body = bytes.NewReader(b)
}
req, err := http.NewRequestWithContext(ctx, method, executorURL()+c.Op, body)
req, err := http.NewRequestWithContext(ctx, method, executorURL(c.Base)+c.Op, body)
if err != nil {
return nil, fmt.Errorf("runtime: build call: %w", err)
}
@@ -262,8 +273,12 @@ func ErrBody(resp *http.Response) string {
return strings.TrimSpace(string(b))
}
// executorURL resolves the executor base, no trailing slash.
func executorURL() string {
// executorURL resolves the base for one call, no trailing slash. A Call's own
// Base wins; otherwise the bot runtime.
func executorURL(base string) string {
if base != "" {
return strings.TrimRight(base, "/")
}
if v := getenv(urlEnv); v != "" {
return strings.TrimRight(v, "/")
}
@@ -279,13 +294,15 @@ func executorURL() string {
// This guard exists because the transport does not authenticate its peer. A ZAP
// entry point pins X25519MLKEM768 and refuses a classical-only peer structurally,
// which is what makes this check unnecessary rather than merely satisfied.
func requireSecure() error {
u := executorURL()
// It checks the base the call will ACTUALLY use: guarding the default while the
// bytes go somewhere else would be a guard on the wrong hop.
func requireSecure(base string) error {
u := executorURL(base)
if strings.HasPrefix(u, "https://") || getenv(plaintextEnv) == "1" {
return nil
}
return fmt.Errorf("runtime: refusing to send a credential over cleartext %q (set %s to https, or %s=1 if the hop is mesh-mTLS secured)",
u, urlEnv, plaintextEnv)
return fmt.Errorf("runtime: refusing to send a credential over cleartext %q (set the destination to https, or %s=1 if the hop is mesh-mTLS secured)",
u, plaintextEnv)
}
func getenv(key string) string { return strings.TrimSpace(os.Getenv(key)) }
+13
View File
@@ -82,6 +82,19 @@ type PRRef struct {
// RunRequest / Step / RunResult mirror the bot coding contract.
type RunRequest struct {
// Tool names what runs inside the sandbox: dev | claude | codex | python |
// node. Empty means dev. The runtime owns the name→argv table; cloud only
// carries the name, so adding a tool is one edit over there and none here.
Tool string
// Desktop selects the xvfb IMAGE VARIANT — a tag, not a mode. A desktop run
// gets an X server, so a real browser window exists to drive; every class can
// already drive a headless one.
Desktop bool
// The repo is OPTIONAL. CloneURL empty means the run has no checkout — and
// then CredUser/CredToken MUST be empty too, because a credential a run can
// never use only exists to leak. The runtime refuses the combination.
CloneURL string
BaseBranch string
Branch string
+62 -15
View File
@@ -3,7 +3,10 @@ package coding
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"strings"
"github.com/hanzoai/cloud/apps/bots"
)
@@ -24,9 +27,29 @@ import (
// BODY (never a URL, never argv, never a log). That is why the call declares
// Secret — the transport then refuses to carry it over a cleartext hop.
// taskOp addresses the runtime's coding-task operation.
// taskOp addresses the runtime's sandbox-run operation.
const taskOp = "/v1/coding-tasks"
// sandboxURL is WHERE A RUN GOES, and it is deliberately not the bot address.
//
// A sandbox is not the bot. Coding, deep research and bare exec all want the
// same thing — a computer to run something in — and none of them wants the
// service that runs Slack channels. BOT_GATEWAY_URL is the right name for bot
// traffic and stays that; this is the name for a sandbox.
//
// The old name is accepted for ONE release so a deploy cannot half-land, then it
// is deleted. Not a permanent alias: two live names for one address is how the
// two ends stop agreeing about where a run went, with nothing in a log to say so.
// Empty here means the transport's own default, which today is the same pod.
func sandboxURL() string {
for _, k := range []string{"SANDBOX_URL", "BOT_GATEWAY_URL"} {
if v := strings.TrimSpace(os.Getenv(k)); v != "" {
return v
}
}
return ""
}
// credential is the per-org agent git credential the sandbox presents to native
// git. Token is the secret (an sk- key); Username is the basic-auth user label.
// Encoded into the request body only — never logged.
@@ -35,15 +58,23 @@ type credential struct {
Token string `json:"token"`
}
// taskRequest is the cloud→runtime body for one coding run.
// taskRequest is the cloud→runtime body for one sandbox run.
//
// EVERY GIT FIELD IS omitempty, AND THAT IS THE CONTRACT, NOT A TIDINESS
// PREFERENCE. A run with no repo must put NO credential on the wire at all —
// not an empty one. `credential` is a pointer for the same reason: a value type
// would always marshal, so "no repo" would still ship a `credential` object and
// the runtime could not tell an absent grant from a blank one.
type taskRequest struct {
CloneURL string `json:"cloneUrl"` // https://<domain>/v1/git/<org>/<repo>.git
BaseBranch string `json:"baseBranch"` // branch to start from (default repo default)
Branch string `json:"branch"` // branch to create + push (e.g. agent/<sessionid>)
Prompt string `json:"prompt"` // the engineering task
SessionID string `json:"sessionId"` // cloud session id (correlation)
RunTimeoutSeconds int `json:"runTimeoutSeconds"` // sandbox run budget
Credential credential `json:"credential"` // agent git credential (write-only)
Prompt string `json:"prompt"` // the task
Tool string `json:"tool,omitempty"` // dev|claude|codex|python|node (default dev)
Desktop bool `json:"desktop,omitempty"` // select the xvfb image variant
SessionID string `json:"sessionId"` // cloud session id (correlation)
RunTimeoutSeconds int `json:"runTimeoutSeconds"` // sandbox run budget
CloneURL string `json:"cloneUrl,omitempty"` // https://<domain>/v1/git/<org>/<repo>.git
BaseBranch string `json:"baseBranch,omitempty"` // branch to start from (default repo default)
Branch string `json:"branch,omitempty"` // branch to create + push (e.g. agent/<sessionid>)
Credential *credential `json:"credential,omitempty"` // agent git credential (write-only); nil when there is no repo
}
// message is the discriminated shape of one streamed line: step/log while the job
@@ -73,16 +104,32 @@ type runner struct{}
func (runner) Run(ctx context.Context, org, userID string, req RunRequest, onStep func(Step)) (RunResult, error) {
var out RunResult
var terminal bool
body := taskRequest{
Prompt: req.Prompt, Tool: req.Tool, Desktop: req.Desktop,
SessionID: req.SessionID, RunTimeoutSeconds: req.RunTimeoutSeconds,
}
// The git half travels together or not at all — there is no path here that
// puts a credential on the wire without the repo it belongs to. A caller
// that supplies one anyway is REFUSED rather than quietly trimmed: silently
// dropping a secret hides the bug that minted it, and the runtime says the
// same thing at its own boundary, so the two ends agree.
if req.CloneURL == "" && (req.CredToken != "" || req.CredUser != "") {
return out, errors.New("coding: a credential without a repo cannot be used, and must not be sent")
}
if req.CloneURL != "" {
body.CloneURL, body.BaseBranch, body.Branch = req.CloneURL, req.BaseBranch, req.Branch
body.Credential = &credential{Username: req.CredUser, Token: req.CredToken}
}
err := bots.Stream(ctx, bots.Call{
Op: taskOp,
Org: org,
User: userID,
Body: taskRequest{
CloneURL: req.CloneURL, BaseBranch: req.BaseBranch, Branch: req.Branch,
Prompt: req.Prompt, SessionID: req.SessionID, RunTimeoutSeconds: req.RunTimeoutSeconds,
Credential: credential{Username: req.CredUser, Token: req.CredToken},
},
Secret: true, // the body carries the org's git credential
Base: sandboxURL(),
Body: body,
// Secret ONLY when the body actually carries the org's git credential.
// A run with no repo has no secret to protect, so it must not be refused
// by the cleartext guard that exists to protect one.
Secret: body.Credential != nil,
}, func(msg []byte) {
var m message
if json.Unmarshal(msg, &m) != nil {
+54
View File
@@ -140,7 +140,11 @@ func TestTask_RefusesCleartextByDefault(t *testing.T) {
defer srv.Close()
t.Setenv("BOT_GATEWAY_URL", srv.URL) // http://
t.Setenv("BOT_GATEWAY_ALLOW_PLAINTEXT", "")
// The repo is what makes this POST credential-bearing: with no CloneURL the
// credential never reaches the wire, so there would be no secret for the
// cleartext guard to protect and nothing for this test to prove.
_, err := runner{}.Run(context.Background(), "acme", "u", RunRequest{
CloneURL: "https://git.hanzo.ai/v1/git/acme/api.git", Branch: "agent/abc123",
CredUser: "x", CredToken: "sk-SECRET",
}, nil)
if err == nil {
@@ -150,3 +154,53 @@ func TestTask_RefusesCleartextByDefault(t *testing.T) {
t.Fatalf("error must not leak the credential: %v", err)
}
}
// A run with NO repo carries no credential, so it is not a "secret" call and the
// cleartext guard must not refuse it — otherwise every research and bare-exec run
// is blocked by a rule written to protect a git token that is not there.
func TestTask_NoRepoRunIsNotSecret(t *testing.T) {
var raw []byte
srv := ndjsonServer(t, []string{`{"type":"result","ok":true}`}, nil, &raw)
defer srv.Close()
t.Setenv("BOT_GATEWAY_URL", srv.URL) // http://
t.Setenv("BOT_GATEWAY_ALLOW_PLAINTEXT", "")
res, err := runner{}.Run(context.Background(), "acme", "u", RunRequest{
Prompt: "read the docs and summarise", Tool: "python", Desktop: true,
}, nil)
if err != nil {
t.Fatalf("a repo-less run must not be refused as cleartext-secret: %v", err)
}
if !res.OK {
t.Fatal("expected the terminal result to carry through")
}
// The wire says what the run is, and says nothing about git.
var got map[string]any
if err := json.Unmarshal(raw, &got); err != nil {
t.Fatalf("body was not JSON: %v", err)
}
if got["tool"] != "python" || got["desktop"] != true {
t.Fatalf("tool/desktop did not reach the runtime: %#v", got)
}
for _, k := range []string{"credential", "cloneUrl", "branch", "baseBranch"} {
if _, ok := got[k]; ok {
t.Fatalf("a repo-less run must not put %q on the wire: %#v", k, got)
}
}
}
// A credential with no repo is a caller bug, and it is refused at the door rather
// than trimmed in silence.
func TestTask_RefusesCredentialWithoutRepo(t *testing.T) {
srv := ndjsonServer(t, []string{`{"type":"result","ok":true}`}, nil, nil)
defer srv.Close()
t.Setenv("BOT_GATEWAY_URL", strings.Replace(srv.URL, "http://", "https://", 1))
_, err := runner{}.Run(context.Background(), "acme", "u", RunRequest{
Prompt: "x", CredUser: "x", CredToken: "sk-SECRET",
}, nil)
if err == nil {
t.Fatal("a credential with no repo must be refused")
}
if strings.Contains(err.Error(), "sk-SECRET") {
t.Fatalf("error must not leak the credential: %v", err)
}
}
+42
View File
@@ -0,0 +1,42 @@
package sandbox
// The pod does not restate what the image already says.
//
// A Kubernetes `command` with no `args` replaces the image's ENTRYPOINT AND its
// CMD together. Setting it for every class therefore did two invisible things:
// it took tini out of PID 1 (so nothing reaped a sandbox's compilers, language
// servers and browsers), and it deleted the `desktop` class's own CMD, the
// script that starts X and the VNC bridge — so the class whose entire reason to
// exist is a screen had no X server and nothing on the port its image EXPOSEs.
//
// This is the assertion that keeps it out. Neither key, for any class.
import (
"testing"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
)
func TestPodSpecLeavesTheEntrypointToTheImage(t *testing.T) {
r := &runtime{ns: "hanzo-sandboxes", image: "oci.hanzo.ai/hanzoai/sandbox"}
for _, class := range []string{"exec", "dev", "desktop"} {
t.Run(class, func(t *testing.T) {
pod := r.podSpec(Sandbox{
ID: "sbx_1", Org: "acme", Class: class, Pod: "sbx-1",
Image: r.imageFor(class),
})
containers, _, _ := unstructured.NestedSlice(pod.Object, "spec", "containers")
if len(containers) != 1 {
t.Fatalf("want exactly one container, got %d", len(containers))
}
one, _ := containers[0].(map[string]any)
for _, key := range []string{"command", "args"} {
if v, present := one[key]; present {
t.Fatalf("%s class sets %q=%v; the image decides that, and overriding "+
"it here loses tini for every class and the screen for this one",
class, key, v)
}
}
})
}
}
+28 -4
View File
@@ -276,10 +276,34 @@ func (r *runtime) podSpec(m Sandbox) *unstructured.Unstructured {
c := map[string]any{
"name": container,
"image": m.Image,
// `sleep infinity` and nothing else. The pod is a place to run commands,
// not a program — every lifetime, from a one-shot invoke to a week-long
// session, is the same pod entered through the same channel.
"command": []any{"sleep", "infinity"},
// NO `command`, AND THAT ABSENCE IS THE DESIGN.
//
// This used to say `command: ["sleep","infinity"]` for every class, which
// is a sentence about what a pod is — a place to run commands, not a
// program — written in the one field that cannot say only that. A
// Kubernetes `command` with no `args` replaces the image's ENTRYPOINT
// *and* its CMD, both, so it did not add `sleep infinity`; it deleted
// whatever the image had decided, for all three classes at once. Two
// things died there:
//
// tini the image ends `ENTRYPOINT ["/usr/bin/tini","--"]` so that
// something reaps. Overridden, PID 1 was `sleep`, which reaps
// nothing — and a sandbox's children are compilers, language
// servers and browsers started by `kubectl exec`, whose
// zombies then accumulate for the pod's whole lease.
// the screen the `desktop` class's CMD is `sandbox-desktop`, the script
// that starts Xvfb, openbox, x0vncserver and websockify. It
// never ran. So a class whose entire reason to exist is a
// screen had no X server and nothing listening on the VNC port
// its image EXPOSEs. Measured in a live desktop pod:
// `/proc/1/cmdline` was `sleep infinity`, `ss -ltn` was empty,
// and running the script by hand brought all four up.
//
// The fix is not a second field here that decides per class — that would
// be the same fact stated in two places, and they drift. The image already
// says it, once, per class: `exec` and `dev` end `CMD ["sleep","infinity"]`
// and `desktop` ends `CMD ["sandbox-desktop"]`, which itself ends with
// `exec sleep infinity`. Naming neither here lets that stand.
"workingDir": workdirFor(m.Class),
// EPHEMERAL STORAGE IS REQUESTED AND LIMITED, both, and it is not
// optional. A pod that requests less than it uses is permanently first in