main
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 zeekay
|
d09ff7317b |
exec: the tenant is not a header, and the credential is not a path
Red returned do-not-ship on two CRITICALs that chain to unauthenticated
cross-tenant code execution. Both are proven by apps/exec/auth_test.go, and all
four assertions FAIL when the old behaviour is put back — mutation-checked, not
asserted.
1. THE TENANT CAME OFF A HEADER. callCtx preferred cloud.Who(ctx).Org, which is
zip.CallerOf, which reads the X-Org-Id REQUEST HEADER (zip caller.go:377) —
and for a request with no validated bearer, SanitizeIdentity deliberately
RESTORES the client's own header (middleware_identity.go:455). So
`X-Org-Id: victim-corp` made storeFor open the victim's SQLite file: the run
executed in their store and /v1/files + /v1/download read their artifacts
back out. principal.OrgFrom is the org a VALIDATED principal resolved to and
nothing else; every other app resolves through it and this one did not.
plane.go's own note — "an org in the argument is an org the caller chose" —
is the rule it was breaking.
2. THE CREDENTIAL CHECK WAS A LOWERCASE PREFIX LIST OVER c.Path(). fiber routes
case-insensitively; cloud.RoutePath exists in this repo for exactly that and
two other gates already use it. POST /V1/EXEC matched the route and missed
the list: no key at all ran code, a wrong key read another session's bytes,
and CODE_EXEC_API_KEY UNSET — the documented fail-closed 503 — still ran code.
Same root, so one fix: authorization stops being inferred from the SPELLING of a
request. The middleware normalizes with RoutePath AND parks two facts on the
context — principal.WithOrg (inherited by the typed op, typed.go:82) and an
unexported `admitted` marker. tenantOf is the ONE tenant decision and refuses a
context carrying neither. The prefix list may still drift; drift is now
fail-CLOSED — a missed path is a 403 on a route that should have worked, never a
route that works without a credential.
AND THE DOOR NO LIST COULD HAVE COVERED. A typed op is also an MCP tool and an
op-plane op, and tools/call invokes it DIRECTLY (zip typed.go:474) — no route, no
middleware. `POST /mcp name=post_v1_exec` with no key ran code. Typing /v1/exec
for its SDK value is what opened that door; the handler-side check closes it,
because those doors park no marker and carry no principal.
3. [HIGH] AN EXEC SANDBOX HAD NO CEILING. Single-attach bounds dev/desktop via
their project; an exec sandbox has none, and the code tool sends no
session_id, so every call mints a fresh pod on a 15-minute lease — 40 calls,
40 pods, each 250m/512Mi/2Gi. The reaper is the FLOOR, not the ceiling: it
ends leases that are over, bounding the steady state and never the burst.
maxLiveExec=16 is written in node capacity (8Gi, 4 cores) and refuses with
429, because the caller's correct response is to wait. Counted with a real
COUNT(*), not len(List) — List is LIMIT 200, which stops counting exactly
where refusing starts to matter.
4. [HIGH] TWO CLIENT-SHAPE BUGS, both silent. hanzo.chat primes an attachment as
{id, session_id, name} (Files/Code/process.js) while @hanzochat/agents spells
it storage_session_id (tools.d.ts) — reading only the second skipped every
attached file AND the "not available" note, so a user's CSV was invisible with
no error. CodeFile.Session() reads both, and a ref with neither is reported.
Artifacts were COLLECTED recursively (find) and LISTED top-level (ls -1A), so
a nested artifact appeared in the reply and was missing from /v1/files/{sid},
which the client reads as expired. One find answers both now.
Also: apps/functions.go's doc no longer cites the deleted CODE_EXEC_UPSTREAM.
Tests: 22 in apps/exec (14 + 5 auth + 3 client-shape), 6 in apps/sandbox. Full
suite 3 red — base, code, commerce — all pre-existing and environmental (this
box's SQLite lacks fts5 and acos). Zero new failures. make check green.
NOT FIXED, and it is a decision rather than a defect: the real chat client sends
no credential at all (EnvVar.CODE_API_KEY undefined, handleTools.js sends no auth
header, the chat pod has no LIBRECHAT_CODE_API_KEY), so the guard 401s every
legitimate request. Fixing the bypasses does not make the feature work; settling
what the client presents does, and the house rule says Hanzo IAM rather than a
second shared secret.
|
||
|
zeekay
|
b9b516ae17 |
Merge remote-tracking branch 'forge/main' into feat/sandbox-executor
CI/CD / image (push) Failing after 12m49s
CI/CD / gate (push) Successful in 27m39s
CI/CD / containment (push) Successful in 3m24s
Hanzo CI/CD / cicd (push) Successful in 27m32s
CI/CD / rollout (push) Canceled after 0s
CI/CD / reach (push) Canceled after 0s
CI/CD / fanout (push) Canceled after 0s
CI/CD / receipt (push) Canceled after 0s
# Conflicts: # apps/functions/billing_test.go |
||
|
zeekay
|
48c2af53f9 |
a session is a sandbox: exec stops proxying to a Service with no endpoints
apps/exec was a reverse proxy to code-exec.hanzo.svc.cluster.local:8000, and
that Service has had ZERO endpoints for 33 days — /v1/exec answered 503 in
production the whole time. It was not failing to reach the executor; there was
no executor. apps/functions read the same CODE_EXEC_UPSTREAM and was failing
against the same absence.
cloud already runs the one compute primitive. A LibreChat session IS a sandbox
("a code-exec call = a sandbox with a session lease", apps/sandbox's package
doc), so session_id is the sandbox id, upload/download/list are Write/Read/Read
on that sandbox, and exec holds no store, no session table and no lifetime of
its own. The lease ends on the reaper.
THE BLOCKER, AND THE DECOMPLECTION. Every sandbox operation existed ONLY as
`func X(s *Service, c *zip.Ctx) error` — the domain braided into the transport,
so nothing else in the process could use a sandbox. apps/sandbox/api.go is the
domain as a VALUE: Lease, Get, List, Run, Read, Write, End. The HTTP handlers
become adapters (bind, call, JSON) and apps/sandbox/plane.go is a SECOND adapter
over the same functions. Errors are zip errors in the core, so "not yours" is
404 once, decided where the fact is known.
(Functions taking *Service, not methods: `Service = cloud.Service[state]` is an
alias for a generic type in package cloud, and Go cannot define a method on a
non-local type. apps/git's files.go is the same shape for the same reason.)
cloud.Ask AND NOT A GO IMPORT, and that is correctness, not style. Apps ship as
separate plugin binaries (plugin/<app>/main.go "links only its own subsystem"),
so importing apps/sandbox would give exec a SECOND sandbox service — its own
OrgStore on the same per-org SQLite files, its own reaper racing the real one.
plane.Ask already collapses to an in-process dispatch where the fleet fuses two
apps (zip.Serving/zip.Here), so the ONE call is a function call when they are
co-resident and a socket hop when they are not.
THE CONTRACT IS MEASURED, from ~/work/hanzo/chat, not remembered. Download is
TWO segments — /download/{session_id}/{fileId} (crud.js) — not /download/{id}.
Upload answers {message:"success", session_id, files:[{fileId, filename}]} and
the client throws unless `message` is that literal. /files/{sid} answers a BARE
JSON ARRAY of {name, lastModified} whose name is that same two-segment
identifier. And the code tool tells the model to persist artifacts in /mnt/data,
so sandbox gains workdirFor(class): /mnt/data for exec, /work for dev. Listing
/work would have reported no files after every successful run.
/v1/exec/programmatic answers 501 rather than being routed into the interpreter.
It is a DIFFERENT protocol — a run suspended on each tool call and resumed from
a continuation token — and answering it with a Result its parser cannot read is
a wrong answer where a refusal is an actionable one.
TENANCY, and the trap. cloud.For states a caller only on a context with NO
request behind it: zip.CallerOf reads the request's headers first, deliberately,
so no handler can assert an org a caller did not arrive with. A typed op's ctx
HAS a request behind it, so cloud.For(typedCtx, org) silently states nothing and
the peer answers 403. exec.callCtx is the one rule — pass a ctx that already
carries a principal through unchanged (hanzo.chat forwards the user's IAM
bearer), otherwise detach to Background, state the deployment's brand org, and
put the request's cancellation back with context.AfterFunc.
apps/functions moves to the same door, because task 3 (deleting the orphan
Service) would otherwise turn its 502 into a DNS failure. It calls exec.Run —
one operation, one home — and ENDS the lease, because a function invoke is over
when it answers.
THE FLOOR IS LOWERED HERE, next to its reason. exec published 40 operations
because a proxy cannot describe a contract it does not own; it now implements
one, so it publishes 5 — POST /v1/exec is a TYPED op (CodeRun -> CodeResult),
which is the schema, the MCP tool, the CLI command and the SDK method a proxy
could never carry. The four that stay untyped have real reasons: multipart in,
bytes out, a bare array, and a protocol we do not serve. Numbers taken from the
ratchet's own report, not computed:
paths 1762 -> 1759 (-3)
operations 2465 -> 2430 (-35)
download 10 -> 1 · exec 10 -> 2 · files 10 -> 1 · upload 10 -> 1
Deleted: newProxy, defaultUpstream, CODE_EXEC_UPSTREAM, and the ledger of 40
untyped operations. CODE_EXEC_API_KEY stays — it is the credential the chat
presents, and the guard is now middleware because a typed op takes no handler
chain.
Tests: apps/exec 14 (a sandbox peer double on the REAL plane, so an op renamed
or a field moved fails here and not in production), apps/functions green
including TestInvoke_AllowsAndDebitsCallerOrg, which was RED before this change
— metering moved debits onto the plane and the test still had only an HTTP
double.
Measured against HEAD: 12 packages fail before and after; zero new; one fixed.
LLM.md also carries a concurrent agent's insights.hanzo.ai section, present in
the tree and committed rather than dropped.
|
||
 hanzo-dev
|
1f3fbe84ed |
exec: 0 typed of 56, and the refusal stops being prose
The code-interpreter surface publishes 56 operations and not one carries a
description, a schema, an MCP tool, a CLI command or an SDK method. It is not a
backlog item: Mount hands all 8 paths to httputil.NewSingleHostReverseProxy and
the sandboxed executor supplies every byte, every Content-Type and every status,
so there is nothing here to describe and four wire facts a typed op would move —
verbatim upstream status (zip answers its own declared one, typed.go:305-311),
response fields this repo never named (an Out drops them), a multipart
/v1/upload body (typed.go:242 jsonenc.Unmarshals every non-empty body) and a
byte-bodied /v1/download/{id} (typed.go:311 always c.JSONs). 16 of the 56 are
OPTIONS/TRACE, which zip has no typed registrar for at all. openapi.Register is
refused too rather than reached for: it could only publish a guess at
@librechat/agents' contract, and Binary's application/octet-stream is not a
multipart envelope.
So the refusal becomes a gate. typed_wire_test.go crosses two closed lists
(untypedPaths x servedMethods) into the same 56 addresses the document uses and
fails three ways: a route neither typed nor named, a reason naming an address
this mount no longer serves, and the day one of them becomes typable. Eight
sub-tests measure the wire facts through the REAL Mount rather than asserting
them. Both directions were mutation-checked: a fifth prefix in exec.go and a
stale ledger entry each turn it red.
Two findings the pass surfaced, recorded in LLM.md rather than fixed here:
- The migration's own inventory cannot see this app. Every documented grep
anchors on `("` or `("/`, so `app.All(p, h)` in a range over prefixes matches
nothing — apps/exec reads ZERO under both commands while serving 56
operations, which is why it has never appeared in a tranche. Same shape in
apps/knowledge (subsystem.go:61-69), apps/iam (258-259, 282) and
apps/commerce (mount.go:551). Count operations, not lines.
- plugin/exec/main.go told the next reader to "edit the spec below directly".
There is no spec in main.go, and the one it means is generated by
`make -C apps/exec openapi` and must never be hand-edited. Corrected here;
110 sibling mains still carry the sentence.
exec.go gains only a comment; the regenerated subset is byte-identical, which is
the honest measurement: 56 operations, 0 described, before and after.
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
|
||
|
hanzo-dev
|
f873d1a180 |
apps: the 131 subsystems move from clients/ to apps/
They were never clients. A "client" is something that CALLS a service; these
are the subsystems the cloud binary mounts and serves. The name said the
opposite of what the code does, and it sat one directory above a composition
root already named `apps` — so the tree read as though `apps/` and `clients/`
were different kinds of thing when one is simply the wiring for the other.
`apps` is a package, so its subsystems nest under it directly:
apps/apps.go package apps — Wire() returns the 118 MountSpecs
apps/git/ package git
apps/projects/ package projects
`apps.Wire()` composing `apps/git` and `apps/projects` needs no second noun.
WHAT DID NOT MOVE. `clients/*.go` (package clients — aihttp, rpc, s3vfs) is
genuinely a client package: HTTP/RPC/VFS handles the subsystems dial OUT with.
It keeps the name, because for those six files the name was always right. Only
the 131 misfiled subdirectories moved, and the import rewrite is scoped to
`hanzoai/cloud/clients/<x>` so the surviving package is untouched.
This is a MOVE, not a rewrite, and deliberately so. mk/plugin.mk already
derives every path from its own location precisely so that "an extracted
apps/<app> + cmd/<app> + mk/ keeps these paths intact" — the build contract was
written for this migration before the directory was renamed to match it. 31
hanzoai/* modules are already extracted and wired as external imports, 16 of
them with thin in-repo adapters that import their own module (verified: zero
duplicates, no forked implementations). This rename puts the remaining 115 in
the directory the extraction contract already names.
Mechanical: git mv per subdir, then `hanzoai/cloud/clients/` ->
`hanzoai/cloud/apps/` across 656 Go files, plus 23 docs/Makefiles/manifests
rewritten only where the path names a real moved app. Builds clean; projects,
git and catalog tests pass.
(cmd/admission and cmd/affiliates fail to LINK here, before and after: they
need native/flags/target/release/libhanzo_flags.a, a Rust artifact never built
in this checkout. Pre-existing and unrelated.)
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
|