the language servers move out of the fleet, and lsp moves under code
CI/CD / gate (push) Canceled after 0s
CI/CD / containment (push) Canceled after 0s
CI/CD / image (push) Canceled after 0s
CI/CD / rollout (push) Canceled after 0s
CI/CD / reach (push) Canceled after 0s
CI/CD / fanout (push) Canceled after 0s
CI/CD / receipt (push) Canceled after 0s

apps/lsp ran gopls, tsserver and the rest INSIDE a fleet pod: server.go,
workspace.go and langs.go checked repositories out, fetched their dependencies
and executed a third-party toolchain over untrusted bytes in the same process
that holds a principal, a ledger and a KMS-injected environment. That is the
wrong shape however carefully it is written, and hanzoai/lsp exists to be the
right one — a jailed daemon with gVisor, no egress but a module proxy, and no
git credential of any kind.

So this side becomes a PROXY over that daemon, and keeps the three things the
daemon must never hold:

  TENANT      the org is the validated principal's, never a body field, and it
              is the daemon's isolation key.
  REPOSITORY  the revision and the tree come from git's own object plane, for
              the caller's own org. The daemon cannot fetch a repository — a
              credential that could reach every repository is exactly what must
              not sit next to an unjailed compiler — so a cold revision is
              /ask → 409 {"need":"tree"} → /root → ask again, ONCE.
  LEDGER      the gate runs at the prepare price before any work, the debit
              after the answer. "prepare" and "query" are the two Models, and
              spend.go's meteredApps entry still holds.

The surface moves from /v1/lsp to /v1/code/lsp, and from one door with a
`method` field to five typed ops — hover, locate, symbols, diagnostics,
complete. code and lsp are two reads of ONE repository, not two products: code
is the static index, lsp the live server that follows a symbol out of the
repository and into a dependency. One home means one place to look for it, in
the document and in the MCP tool list alike, and an agent picks a tool by its
name rather than by a union behind one. Nested static prefixes resolve by
specificity, so /v1/code/lsp beats /v1/code and both beat ai's bare /v1 — the
same relation storage's /v1/s3/buckets already has to provisioning's /v1/s3.

git gains ONE op, git_rev: the commit a ref names. It is separate from git_files
because the two questions cost differently — resolving is a ref lookup and runs
on every position query; reading is a walk of the whole tree and runs only when
the daemon says it holds no root. Folded together, a hover would drag a monorepo
across a socket. One resolve (coreRev) now backs both.

plane/agents was generated before agents_run_on_behalf existed; running the
generator carries it in.

The published surface GREW: 1762 → 1766 paths, 2480 → 2484 operations. The
floor's "lsp" product is gone because its five operations are tagged "code" now,
which is the move, and code rises 7 → 12.
This commit is contained in:
antje
2026-08-06 05:19:23 -07:00
parent 113bed315a
commit b3ded8a57f
21 changed files with 1554 additions and 2478 deletions
+58 -17
View File
@@ -11,18 +11,43 @@ import (
"github.com/zap-proto/zip"
)
// files.go — the delivery inventory read, and its two adapters.
// files.go — the delivery inventory read, the revision it is pinned to, and
// their adapters.
//
// ONE core (coreFiles) answers "which files does this glob select at this
// commit, and what do they say". The REST route serves it to browsers and the
// CLI; the internal plane serves it to delivery. Two thin adapters, one core —
// so the answer cannot differ by who asked.
// ONE resolve (coreRev) answers "which commit does this ref name". ONE core over
// it (coreFiles) answers "which files does this glob select at that commit, and
// what do they say". The REST route serves the second to browsers and the CLI;
// the internal plane serves both to its peers. Thin adapters over one core — so
// the answer cannot differ by who asked.
//
// The two are separate ops on the plane because they cost differently. Resolving
// is a ref lookup; reading is a walk. A caller that pins a revision on every
// request would otherwise read a whole tree to learn a sha.
//
// This is what replaces cloning for delivery. A generator never needs a
// packfile; it needs the bytes of some files at one revision. Serving that as a
// tree read is what lets a repository sit on object storage — no pack
// negotiation, no working copy, nothing on this path that needs POSIX.
// coreRev opens one of the tenant's repos and resolves ref against it, returning
// the open repository beside the commit so a reader can walk the same handle it
// resolved through. An empty ref means the repo's own default branch.
func coreRev(s *cloud.Service[state], ctx context.Context, t tenant, name, ref string) (Repository, Revision, string, error) {
r, found := findRepo(s, ctx, t.org, normalizeName(name))
if !found {
return nil, "", "", errNotFound
}
repo, err := openRepository(s, r)
if err != nil {
return nil, "", "", errNotFound
}
rev, label, err := repo.Resolve(ctx, strings.TrimSpace(ref))
if err != nil {
return nil, "", "", errNotFound
}
return repo, rev, label, nil
}
// coreFiles resolves ref once and reads every path the glob selects.
//
// ONE resolve backs the whole reply. A caller that listed at `main` and then
@@ -33,17 +58,9 @@ func coreFiles(s *cloud.Service[state], ctx context.Context, t tenant, name, ref
if strings.TrimSpace(glob) == "" {
return "", nil, errBadInput
}
r, found := findRepo(s, ctx, t.org, normalizeName(name))
if !found {
return "", nil, errNotFound
}
repo, err := openRepository(s, r)
repo, res, _, err := coreRev(s, ctx, t, name, ref)
if err != nil {
return "", nil, errNotFound
}
res, _, err := repo.Resolve(ctx, strings.TrimSpace(ref))
if err != nil {
return "", nil, errNotFound
return "", nil, err
}
paths, err := MatchPaths(ctx, repo, res, glob)
@@ -69,8 +86,8 @@ func coreFiles(s *cloud.Service[state], ctx context.Context, t tenant, name, ref
return res.String(), out, nil
}
// exposeFiles publishes the inventory read on the internal plane. Called from
// Mount, beside the other cross-app seams.
// exposeFiles publishes the inventory read and the revision resolve on the
// internal plane. Called from Mount, beside the other cross-app seams.
//
// The tenant comes from the CALLER, never the argument: the identity is what the
// edge minted, so an argument cannot widen the org it is answered for. Anonymous
@@ -80,6 +97,30 @@ func exposeFiles() {
zip.Post[plane.FilesIn, plane.Files](cloud.Plane(), "/git/files", planeFiles,
zip.WithOperationID(plane.GitFiles),
zip.WithSummary("A repo's files at one revision"))
zip.Post[plane.RevIn, plane.Rev](cloud.Plane(), "/git/rev", planeRev,
zip.WithOperationID(plane.GitRev),
zip.WithSummary("The commit a ref resolves to"))
}
// planeRev resolves one of the caller's repos at one ref to the commit it names,
// returning that commit and the branch or tag label it was reached by. The org is
// the CALLER's plane identity, never the argument — an anonymous caller is
// refused. A named handler, not a closure, so zipdoc can lift this prose into the
// registry.
func planeRev(ctx context.Context, in *plane.RevIn) (*plane.Rev, error) {
who := cloud.Who(ctx)
if who.Org == "" {
return nil, zip.ErrForbidden("git rev: org required")
}
s := mounted.Load()
if s == nil {
return nil, zip.Errorf(503, "git not mounted")
}
_, rev, label, err := coreRev(s, ctx, tenant{org: who.Org, project: who.Project}, in.Repo, in.Ref)
if err != nil {
return nil, zip.ErrNotFound("repo, ref or revision not found")
}
return &plane.Rev{Rev: rev.String(), Ref: label}, nil
}
// planeFiles reads the glob-selected files of one of the caller's repos at one
+3
View File
@@ -295,6 +295,9 @@ func init() {
zip.Describe("POST /git/publish", zip.Doc{
Description: "Reconciles a project's canonical repo to the project's published\nvisibility: it provisions the repo on first publish and thereafter flips only\nthe public bit, then keeps the GitHub replica's visibility in step.\nIdempotent, so projects can fire it on every create, visibility change and\nmoderation event. The org is the CALLER's plane identity, never the argument —\na caller that could name the org would be publishing into another tenant's\nrepos — and an anonymous caller is refused. A named handler, not a closure, so\nzipdoc can lift this prose into the registry.",
})
zip.Describe("POST /git/rev", zip.Doc{
Description: "Resolves one of the caller's repos at one ref to the commit it names,\nreturning that commit and the branch or tag label it was reached by. The org is\nthe CALLER's plane identity, never the argument — an anonymous caller is\nrefused. A named handler, not a closure, so zipdoc can lift this prose into the\nregistry.",
})
zip.Describe("POST /git/status", zip.Doc{
Description: "Reports which of the named repos the CALLER's org has imported and\nwhich a prior inbound sync left in conflict.\n\nThe app that DRAWS the repo list is integrations (it has the provider's\ncatalogue of what could be imported); the app that knows what WAS is this one.\nIn a split fleet the in-process importer is nil over there, so the list\nrendered every repo as never-imported — a wrong answer delivered confidently,\nwhich is worse than the import failure the same split caused, because nothing\nerrored.\n\nThe reply is a SLICE, not a map: a map cannot cross this wire, so each row\ncarries the name it answers for. A name git holds nothing under is ABSENT\nrather than a false row — the caller reads absence as not-imported, which is\nthe same value the in-process leg's zero entry yields, so neither leg can be\ntold from the other by its result.\n\nIt calls the in-process implementation directly rather than\ncloud.GitRepoStatuses: the package func dispatches to whatever is registered,\nand in THIS process that resolution would come back around through the plane\nto this same handler.\n\nA named handler, not a closure, so zipdoc can lift this prose into the registry.",
Fields: map[string]string{
+212
View File
@@ -0,0 +1,212 @@
package lsp
// daemon.go is the client for hanzoai/lsp — the jailed language-server daemon.
//
// # Why the language servers are not here
//
// Answering "where is this defined" for a symbol that lands in a DEPENDENCY means
// fetching that dependency and type-checking both, which means running a
// third-party toolchain over untrusted bytes. That belongs in a pod with gVisor,
// no egress but a module proxy, and no credential — not in a fleet app that holds
// a principal and a ledger. So this package resolves the tenant, prices the work
// and forwards the position; the daemon runs the compiler.
//
// # Two calls, and the daemon can never make the first one
//
// A root is an immutable (org, repo, commit) tree the daemon HOLDS. It has no git
// credential and no way to get one — a daemon that could fetch a repository would
// need a credential that could reach every repository — so it cannot go and get a
// tree it lacks. It says so instead: /ask answers 409 {"need":"tree"} and the
// caller, which already has the repository, POSTs /root and asks again. Two calls
// on a cold root, one on a warm one, and no path by which the sandbox reaches
// anything.
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"strings"
"time"
"github.com/zap-proto/zip"
)
// upstream is the daemon's in-cluster address. WHERE it runs is infra wiring, so
// it is env with a constant default — never a request field.
const (
upstreamEnv = "LSP_UPSTREAM"
upstreamDefault = "http://lsp.hanzo.svc.cluster.local:8000"
)
// keyEnv names the shared service key this proxy presents. The value is a KMS
// secret the Deployment injects; nothing here ever writes it to a log or a reply.
const keyEnv = "LSP_KEY"
// The two budgets, which are two different kinds of work.
//
// A PREPARE is a tree write, a dependency fetch and a language server's first
// index: minutes, legitimately. A QUERY is a JSON-RPC round trip to a process
// that already holds the index: milliseconds. One deadline for both would either
// cut an honest cold start in half or let a wedged server hold a hover request
// for five minutes.
const (
prepareWait = 5 * time.Minute
askWait = 10 * time.Second
)
// bodyMost bounds a reply. The daemon caps what it answers; this caps what a
// compromised or confused one could make this process allocate.
const bodyMost = 32 << 20
// errNeedTree is the daemon's 409: it holds no root for that revision, and only
// the caller can supply one.
var errNeedTree = errors.New("lsp: the daemon holds no root for this revision")
// daemon is the upstream: one address, one key, one client. Package-level and
// shared, because a connection pool that is rebuilt per request is not a pool.
type daemon struct {
url string
key string
http *http.Client
}
// newDaemon reads the deployment's wiring once, at Mount.
//
// The client's own Timeout is the PREPARE ceiling — the longest legitimate call —
// and each call narrows it further with a context deadline. One ceiling, one
// per-call bound, and no request that can outlive both.
func newDaemon() *daemon {
return &daemon{
url: env(upstreamEnv, upstreamDefault),
key: strings.TrimSpace(os.Getenv(keyEnv)),
http: &http.Client{Timeout: prepareWait},
}
}
func env(key, def string) string {
if v := strings.TrimSpace(os.Getenv(key)); v != "" {
return v
}
return def
}
// tree is the body of /root: a whole working tree, which only the caller can
// produce.
type tree struct {
Org string `json:"org"`
Repo string `json:"repo"`
Rev string `json:"rev"`
Files []file `json:"files"`
}
// file is one file of that tree, tree-relative path and whole content.
type file struct {
Path string `json:"path"`
Content string `json:"content"`
}
// ready is what /root answers. Cold reports that THIS call paid for the build —
// the fetch and the first index — which is the event that carries a fee.
type ready struct {
Ready bool `json:"ready"`
Cold bool `json:"cold"`
Langs []string `json:"langs"`
}
// question is the body of /ask: one op at one position in one root.
type question struct {
Org string `json:"org"`
Repo string `json:"repo"`
Rev string `json:"rev"`
Op string `json:"op"`
Relation string `json:"relation,omitempty"`
Path string `json:"path"`
Line int `json:"line"`
Character int `json:"character"`
}
// ask puts one question to a root the daemon already holds, decoding the reply
// into out. A daemon that holds no such root answers errNeedTree.
func (d *daemon) ask(ctx context.Context, in *question, out *Answer) error {
ctx, cancel := context.WithTimeout(ctx, askWait)
defer cancel()
return d.post(ctx, "/ask", in, out)
}
// root hands the daemon the tree it said it needed.
func (d *daemon) root(ctx context.Context, in *tree) (*ready, error) {
ctx, cancel := context.WithTimeout(ctx, prepareWait)
defer cancel()
out := &ready{}
if err := d.post(ctx, "/root", in, out); err != nil {
return nil, err
}
return out, nil
}
// post is the ONE way this package speaks to the daemon: encode, present the
// key, decode, and translate a status into an error a caller can act on.
//
// The daemon's own error text names its paths and can echo tenant source, so it
// is never returned to a client — the status decides what this says, and the
// detail goes to the log at the call site.
func (d *daemon) post(ctx context.Context, path string, in, out any) error {
if d.key == "" {
// Fail CLOSED and say so as an outage, not as a refusal of the caller: a
// proxy with no credential is a misconfigured deployment, and the daemon
// would 503 this same request anyway.
return zip.Errorf(http.StatusServiceUnavailable, "language server unavailable")
}
body, err := json.Marshal(in)
if err != nil {
return fmt.Errorf("encode %s: %w", path, err)
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, d.url+path, bytes.NewReader(body))
if err != nil {
return fmt.Errorf("build %s: %w", path, err)
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("X-API-Key", d.key)
res, err := d.http.Do(req)
if err != nil {
return fmt.Errorf("call %s: %w", path, err)
}
defer func() { _, _ = io.Copy(io.Discard, res.Body); _ = res.Body.Close() }()
reply, err := io.ReadAll(io.LimitReader(res.Body, bodyMost))
if err != nil {
return fmt.Errorf("read %s: %w", path, err)
}
switch {
case res.StatusCode == http.StatusOK:
if err := json.Unmarshal(reply, out); err != nil {
return fmt.Errorf("decode %s: %w", path, err)
}
return nil
case res.StatusCode == http.StatusConflict && needsTree(reply):
return errNeedTree
case res.StatusCode == http.StatusBadRequest:
// The daemon narrows the same inputs this package does — op, relation,
// path, position, key shape. Reaching here means the two disagree, which
// is the caller's request being wrong in a way worth telling them.
return zip.ErrBadRequest("the language server refused this request")
default:
return fmt.Errorf("%s: upstream status %d", path, res.StatusCode)
}
}
// needsTree reads the one discriminator on a 409. The daemon has two: "tree" (no
// root for this revision) and "held" (a warm request for a repository with no
// live root). Only the first is one this proxy can answer by sending a tree.
func needsTree(reply []byte) bool {
var body struct {
Need string `json:"need"`
}
return json.Unmarshal(reply, &body) == nil && body.Need == "tree"
}
-220
View File
@@ -1,220 +0,0 @@
package lsp
import (
"os"
"path/filepath"
"slices"
"strings"
)
// langs.go is the language table — the ONE place that answers three questions
// about a language: how to RECOGNIZE it in a checkout, how to START its server,
// and how to FETCH its dependencies.
//
// The servers and their argv are PORTED, not invented, from the Python tool at
// hanzo/python-sdk/pkg/hanzo-tools-lsp/hanzo_tools/lsp/lsp_tool.py (LSP_SERVERS):
// same binaries, same flags, same root markers, same extensions. A second
// opinion about how to spawn gopls is a second bug surface, so there is not one.
//
// What is NOT ported is install_cmd. The Python tool installs a language server
// on demand onto the machine it runs on; this service runs the server in a cloud
// image that already ships the toolchain (phase 2: the cloud-lsp Dockerfile). A
// cloud worker that can `npm install -g` at request time is a worker an attacker
// can make write to its own filesystem, so the capability is removed rather than
// guarded.
//
// # Scripts-off is the default, and it is stated here
//
// Fetching dependencies is the dangerous half of this service. `npm install`
// runs postinstall; `cargo build` runs build.rs; `pip install` of an sdist runs
// setup.py. Each is arbitrary code from a third party executing inside the
// worker — remote code execution by design, triggered by whatever the caller
// asked us to check out.
//
// It is also UNNECESSARY. A language server resolves definitions, references and
// types from SOURCE — the dependency's .go/.d.ts/.pyi/.rs files — not from the
// artifacts a build script produces. Turning scripts off costs some generated
// code and some proc-macro expansions; it does not cost go-to-definition.
//
// So Executes marks the fetches that run dependency-authored code, and fetchable
// (workspace.go) is the ONE predicate that reads it. Today it refuses them all.
type Lang struct {
// Name is BOTH the table key and the LSP languageId sent on didOpen. One
// string, so a language cannot be called one thing here and another on the
// wire. (The per-file refinement TypeScript needs — tsx vs jsx vs plain js —
// is a property of the FILE, not the language, and lives in ID.)
Name string
// Start is the argv that runs the server speaking JSON-RPC on its stdio.
Start []string
// Roots are the marker files that make a directory this language's root.
Roots []string
// Exts are the file extensions this server answers for.
Exts []string
// Fetch is the argv that populates the dependency tree, run once per
// checkout. Nil means the language has nothing to fetch.
Fetch []string
// Executes reports that Fetch runs code authored by the DEPENDENCIES rather
// than only downloading them. It is the whole of the scripts-off policy's
// input; see fetchable in workspace.go for the policy itself.
Executes bool
// Env is added to the server's and the fetch's environment.
Env []string
// Init is initializationOptions on the initialize request — server-specific
// settings. It is where a server that would otherwise run project code at
// load time is told not to.
Init map[string]any
}
// table is every language this service speaks, keyed by Lang.Name.
var table = map[string]Lang{
"go": {
Name: "go",
Start: []string{"gopls", "serve", "-mode=stdio"},
Roots: []string{"go.work", "go.mod", "go.sum"},
Exts: []string{".go"},
// `go mod download` resolves and verifies modules against the checksum
// database. It does NOT build them, so no module's code runs: the Go
// toolchain has no install-time hook to abuse. Executes stays false.
Fetch: []string{"go", "mod", "download"},
Env: []string{"GOWORK=auto", "GOFLAGS=-mod=mod"},
},
"python": {
Name: "python",
Start: []string{"pyright-langserver", "--stdio"},
Roots: []string{"pyproject.toml", "setup.py", "requirements.txt", "pyrightconfig.json"},
Exts: []string{".py", ".pyi"},
// uv sync BUILDS any dependency published only as an sdist, which runs
// that dependency's setup.py / PEP-517 backend as us. --no-install-project
// spares us the CHECKOUT's own build, not its dependencies'. So this one
// executes, and today it does not run: pyright reads .py/.pyi source and
// resolves the stdlib and any vendored packages without it.
Fetch: []string{"uv", "sync", "--frozen", "--no-install-project"},
Executes: true,
},
"typescript": {
Name: "typescript",
Start: []string{"typescript-language-server", "--stdio"},
Roots: []string{"tsconfig.json", "package.json"},
Exts: []string{".ts", ".tsx", ".js", ".jsx"},
// --ignore-scripts is the whole reason this fetch is allowed: it is npm's
// own switch for "place the tree, run none of its lifecycle hooks". The
// .d.ts files under node_modules are what the server actually reads.
Fetch: []string{"npm", "ci", "--ignore-scripts", "--no-audit", "--no-fund"},
},
"rust": {
Name: "rust",
Start: []string{"rust-analyzer"},
Roots: []string{"Cargo.toml"},
Exts: []string{".rs"},
// `cargo fetch` downloads and unpacks; it does not compile, so no build.rs
// runs. `cargo build` would, which is why the fetch is not that.
Fetch: []string{"cargo", "fetch", "--locked"},
// The fetch being safe is not enough: rust-analyzer COMPILES AND RUNS
// build.rs and expands proc macros at workspace load, by default. That is
// the same remote code execution the fetch was careful to avoid, arriving
// through the server instead. Both are turned off here — scripts-off has
// to hold for the server or it does not hold at all.
Init: map[string]any{
"cargo": map[string]any{"buildScripts": map[string]any{"enable": false}},
"procMacro": map[string]any{"enable": false},
},
},
"cpp": {
Name: "cpp",
Start: []string{"clangd"},
Roots: []string{"compile_commands.json", "CMakeLists.txt"},
Exts: []string{".cpp", ".cc", ".cxx", ".c", ".h", ".hpp"},
// No fetch: C++ has no dependency resolver to run. clangd answers from
// compile_commands.json when the checkout carries one and degrades to
// single-file mode when it does not.
},
}
// langFor picks the language for a repo-relative path by extension.
//
// Extension, not root markers, decides — the question being asked is "which
// server answers about THIS file", and a polyglot repo (a Go service with a
// TypeScript console) has several right answers at once, one per file. Roots
// then narrow WHERE that server is rooted, which is rootFor's job.
func langFor(path string) (Lang, bool) {
ext := strings.ToLower(filepath.Ext(path))
if ext == "" {
return Lang{}, false
}
// Deterministic: map iteration is randomized, and a file that resolved to a
// different server between two identical requests would answer differently
// for no reason the caller can see. Names are walked in sorted order.
for _, name := range langNames {
l := table[name]
if slices.Contains(l.Exts, ext) {
return l, true
}
}
return Lang{}, false
}
// langNames is table's keys in sorted order — the tie-break that makes langFor a
// function of its argument alone.
var langNames = func() []string {
names := make([]string, 0, len(table))
for name := range table {
names = append(names, name)
}
slices.Sort(names)
return names
}()
// rootFor finds the directory a server should be rooted at: the DEEPEST marker
// at or above the file, bounded by the checkout.
//
// Deepest wins because the marker nearest the file describes it best — a file in
// a repo whose root go.mod is the umbrella and whose subdirectory go.mod is the
// real module belongs to the subdirectory. With no marker anywhere the checkout
// root is the answer, which is what a single-file language wants.
//
// dir is the checkout root and is the hard ceiling: the walk starts there and
// only descends, so no marker outside the tenant's own tree can ever root a
// server.
func rootFor(dir, path string, l Lang) string {
best, cur := dir, dir
for _, seg := range strings.Split(filepath.Dir(path), string(filepath.Separator)) {
if seg == "" || seg == "." {
continue
}
cur = filepath.Join(cur, seg)
for _, marker := range l.Roots {
if _, err := os.Stat(filepath.Join(cur, marker)); err == nil {
best = cur
break
}
}
}
return best
}
// ID is the LSP languageId for one FILE. It is Name for every language but
// TypeScript, whose server distinguishes four dialects that share one toolchain
// — and gets the wrong answer for a React file told it is plain TypeScript.
// Ported from the Python tool's _open_document.
func (l Lang) ID(path string) string {
if l.Name != "typescript" {
return l.Name
}
switch strings.ToLower(filepath.Ext(path)) {
case ".tsx":
return "typescriptreact"
case ".jsx":
return "javascriptreact"
case ".js":
return "javascript"
default:
return "typescript"
}
}
+275 -299
View File
@@ -1,69 +1,75 @@
// Package lsp is live semantic code intelligence — definitions, references,
// types, hover and diagnostics — over a repository AND its resolved
// dependencies, served from the cloud with no toolchain on the caller's machine.
// types, hover, outline and diagnostics — over a repository AND its resolved
// dependencies, with no toolchain on the caller's machine.
//
// # One value
// # Where it sits
//
// An lsp query is a language server rooted at a workspace, asked about a
// position. Everything on the wire is that value spelled out: WHICH workspace
// (repo, rev), WHICH position (path, line, character), and WHICH question
// (method). There is one door, POST /v1/lsp, because there is one value — eight
// endpoints differing only in a verb would be eight spellings of it.
// Under /v1/code, beside the static index. code and lsp are two reads of ONE
// repository, not two products: code is lexical, symbolic and semantic search —
// fast, approximate, always available — and lsp is a real language server —
// exact, typed, and able to follow a symbol out of the repository and into a
// dependency. An agent searches with code and is certain with lsp. One home, so
// there is one place to look for "what does this code mean".
//
// lsp and apps/code are two reads of the SAME checkout, not two systems: code is
// the static index (lexical, symbolic, semantic — fast, approximate, always
// available), lsp is the live server (exact, typed, resolves through
// dependencies, costs a cold start). An agent uses code to find candidates and
// lsp to be certain.
// # What this package is
//
// A PROXY. The language servers run in hanzoai/lsp, a jailed daemon on its own
// deployment, because answering a cross-dependency question means running a
// third-party toolchain over untrusted bytes (see daemon.go). This side owns the
// three things the daemon must never hold: the tenant, the repository and the
// ledger.
//
// - TENANT. Every request resolves its org from the validated principal, and
// that org is the daemon's isolation key. A caller supplies a repo SLUG,
// never an owner and never a URL, so there is no input from which one tenant
// could name another tenant's repository.
// - REPOSITORY. The revision and the tree come from the git plane, over the
// socket, for the caller's own org. The daemon holds no git credential —
// one that could fetch any repository is exactly what must not exist next to
// an unjailed compiler — so the tree is pushed to it, never pulled by it.
// - LEDGER. The gate runs before the work and the debit after it (meter.go).
//
// # Positions are the LSP's, not a translation of them
//
// line and character are 0-BASED, and character counts UTF-16 code units, per the
// LSP specification. That is deliberately not the 1-based line an editor shows a
// human: this door's callers are agents and editors that already speak LSP, and a
// human: these callers are agents and editors that already speak LSP, and a
// service that silently re-based positions would corrupt every multi-byte line —
// an emoji before the cursor is one UTF-16 unit in the protocol's arithmetic and
// two in Go's. Positions pass through untouched, so the protocol's answer is the
// answer.
//
// # Isolation
//
// Every request resolves its org from the validated principal, and that org is
// BOTH the pool key and the owner segment of the git URL. A caller supplies a
// repo slug, never a URL and never an owner. There is no input from which one
// tenant could name another tenant's repository.
package lsp
import (
"bytes"
"context"
"encoding/json"
"fmt"
"path/filepath"
"errors"
"regexp"
"slices"
"strings"
"time"
"unicode/utf8"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/apps/principal"
"github.com/hanzoai/cloud/plane"
gitplane "github.com/hanzoai/cloud/plane/git"
"github.com/zap-proto/zip"
)
// diagSettle is how long diagnostics must stay unchanged before the snapshot is
// taken. See Conn.Diagnostics — LSP has no completion signal for them.
const diagSettle = 400 * time.Millisecond
// Query is one position question against one repository.
// Query is one position in one file of one repository — the value every op here
// takes, because every op here is one question about one position.
type Query struct {
// Repo is the repository NAME within the caller's own org, e.g. "cloud".
// Not a URL and not an owner/name pair: the owner is the validated
// principal's org, so this names a repository the caller already owns.
Repo string `json:"repo"`
// Rev is a branch, tag or commit sha. Empty means the default branch. A
// workspace is keyed by revision, so pinning a sha is what makes an answer
// reproducible.
// Rev is a branch, tag or commit sha. Empty means the default branch. It is
// resolved to a commit before anything else happens, so an answer is always
// about one immutable tree.
Rev string `json:"rev,omitempty"`
// Path is the repo-relative file, e.g. "apps/lsp/server.go".
// Path is the repo-relative file, e.g. "apps/lsp/lsp.go".
Path string `json:"path"`
// Line is 0-based, per the LSP specification.
@@ -73,24 +79,24 @@ type Query struct {
// specification — not a byte offset and not a rune index.
Character int `json:"character"`
// Method is the question: hover, definition, references, typeDefinition,
// implementation, documentSymbol, completion or diagnostics.
Method string `json:"method"`
// Relation refines locate: definition, reference, type or implementation.
// Empty means definition. Every other op ignores it.
Relation string `json:"relation,omitempty"`
}
// Answer carries whichever result the method produces. Exactly one of the result
// fields is populated; the rest are omitted, so a client reads the field its
// method names and never has to discriminate a union.
// Answer carries whichever result the op produces. Exactly one result field is
// populated, so a client reads the field its op names and never discriminates a
// union.
type Answer struct {
Method string `json:"method"`
Repo string `json:"repo"`
Rev string `json:"rev,omitempty"`
Path string `json:"path"`
Lang string `json:"lang"`
Op string `json:"op"`
Lang string `json:"lang"`
Repo string `json:"repo"`
Rev string `json:"rev"`
Path string `json:"path"`
// Cold reports that this request paid for a workspace cold start — the
// checkout, the dependency fetch and the server's first index. It is the
// billed event, surfaced so a caller can see what it was charged for.
// Cold reports that this request paid to PREPARE the revision — the tree
// write, the dependency fetch and the language server's first index. It is
// the billed event, surfaced so a caller can see what it was charged for.
Cold bool `json:"cold"`
Locations []Location `json:"locations,omitempty"`
@@ -100,13 +106,14 @@ type Answer struct {
Diagnostics []Diagnostic `json:"diagnostics,omitempty"`
}
// Location is one place in the workspace. Path is repo-relative when the target
// is inside the checkout; for a target in the dependency cache it is the absolute
// path the server reported, which is what makes "definition in a dependency"
// answerable at all.
// Location is one place an answer resolved to. External false means Path is
// repo-relative; true means the answer left the repository and Path is the module
// coordinate it landed in ("golang.org/x/mod@v0.14.0/semver/semver.go"), which is
// the whole reason this service exists.
type Location struct {
Path string `json:"path"`
Range Range `json:"range"`
Path string `json:"path"`
External bool `json:"external,omitempty"`
Range Range `json:"range"`
}
// Position is the LSP's: 0-based line, 0-based UTF-16 character.
@@ -115,11 +122,13 @@ type Position struct {
Character int `json:"character"`
}
// Range is a half-open span between two positions.
type Range struct {
Start Position `json:"start"`
End Position `json:"end"`
}
// Symbol is one entry in a file's outline.
type Symbol struct {
Name string `json:"name"`
Kind int `json:"kind"`
@@ -127,6 +136,7 @@ type Symbol struct {
Range Range `json:"range"`
}
// Completion is one candidate at a position.
type Completion struct {
Label string `json:"label"`
Kind int `json:"kind,omitempty"`
@@ -143,52 +153,118 @@ type Diagnostic struct {
Message string `json:"message"`
}
// zipdoc lifts the doc comment off the typed op and its In/Out fields into
// The two shapes a caller may name, narrowed HERE so a malformed one costs no
// socket. slug is a repository name under an owner; ref is a branch, tag or sha
// whose leading character is alphanumeric, which is what stops it being read as a
// flag by anything downstream that shells out.
var (
slug = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9._-]{0,99}$`)
ref = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9._/-]{0,199}$`)
)
// relations is the CLOSED set locate refines by. The door names what it serves,
// so an unknown string is a 400 here rather than an arbitrary method handed to a
// language server.
var relations = []string{"definition", "reference", "type", "implementation"}
// zipdoc lifts the doc comment off each typed op and its In/Out fields into
// zipdoc_gen.go, which is the ONLY way that prose reaches the published document
// and the MCP tool list — Go drops comments at compile time. Run by
// `make -C apps/lsp describe`.
//
//go:generate go run github.com/zap-proto/zip/cmd/zipdoc
// ask resolves one position in one repository through a live language server:
// definition, references, type, implementation, hover, document symbols,
// completion or diagnostics — over the repo AND its resolved dependencies, with
// no toolchain on the caller's machine.
// hover renders the type and documentation of the symbol at a position, as the
// language server itself renders it.
//
// Positions are the LSP's: line and character are 0-BASED and character counts
// UTF-16 code units, so an editor's 1-based line must have 1 subtracted before it
// is sent. The repository is named by slug and is always one in the caller's own
// org. rev pins a branch, tag or commit sha; empty means the default branch.
// org; rev pins a branch, tag or commit sha, and empty means the default branch.
//
// The first query against a (repo, rev) pays a cold start — checkout, dependency
// fetch and the server's first index — and is the billed event; later queries
// against the same revision are served from the warm workspace and are free. The
// answer says which it was.
// Example: {"repo":"cloud","path":"apps/lsp/lsp.go","line":120,"character":18}
func (s *state) hover(ctx context.Context, in *Query) (*Answer, error) {
return s.query(ctx, in, "hover")
}
// locate finds where a symbol lives: its definition, its references, its type or
// its implementations, chosen by relation (definition, reference, type,
// implementation — empty means definition).
//
// Example: {"repo":"cloud","path":"apps/lsp/server.go","line":120,"character":18,"method":"definition"}
func (s *state) ask(ctx context.Context, in *Query) (*Answer, error) {
org, ok := principal.OrgFrom(ctx)
if !ok {
return nil, zip.ErrForbidden("valid principal required")
}
method := strings.TrimSpace(in.Method)
if !known(method) {
return nil, zip.ErrBadRequest("method must be one of " + strings.Join(methods, ", "))
// It resolves THROUGH dependencies. An answer whose external flag is set left the
// repository, and its path is then the module coordinate it landed in — which is
// the question a static index cannot answer and this service exists for.
//
// Example: {"repo":"cloud","path":"apps/lsp/lsp.go","line":120,"character":18,"relation":"definition"}
func (s *state) locate(ctx context.Context, in *Query) (*Answer, error) {
return s.query(ctx, in, "locate")
}
// symbols outlines one file: every declaration in it, with its kind and its span.
// The position is ignored — the answer is the whole file.
//
// Example: {"repo":"cloud","path":"apps/lsp/lsp.go"}
func (s *state) symbols(ctx context.Context, in *Query) (*Answer, error) {
return s.query(ctx, in, "symbols")
}
// diagnostics reports every problem the language server finds in one file —
// compile errors, type errors and lints, each with its span and its severity (1
// error, 2 warning, 3 information, 4 hint). The position is ignored.
//
// Example: {"repo":"cloud","path":"apps/lsp/lsp.go"}
func (s *state) diagnostics(ctx context.Context, in *Query) (*Answer, error) {
return s.query(ctx, in, "diagnostics")
}
// complete offers the candidates a language server has at a position, typed and
// resolved through the repository's dependencies rather than guessed from text.
//
// Example: {"repo":"cloud","path":"apps/lsp/lsp.go","line":120,"character":18}
func (s *state) complete(ctx context.Context, in *Query) (*Answer, error) {
return s.query(ctx, in, "complete")
}
// query is the ONE path every op takes, in the order that order matters:
//
// principal → repository → price → resolve → ask → (prepare → ask) → debit
//
// The gate is before the resolve so an out-of-funds caller is refused rather than
// served work nobody can be billed for; the debit is after the answer so nothing
// is charged for work that failed.
func (s *state) query(ctx context.Context, in *Query, op string) (*Answer, error) {
org, err := tenant(ctx)
if err != nil {
return nil, err
}
repo := strings.TrimSpace(in.Repo)
if !slug.MatchString(repo) {
return nil, zip.ErrBadRequest("repo must be a repository name in your org")
}
rev := strings.TrimSpace(in.Rev)
if rev != "" && !revision.MatchString(rev) {
want := strings.TrimSpace(in.Rev)
if want != "" && !ref.MatchString(want) {
return nil, zip.ErrBadRequest("rev must be a branch, tag or commit sha")
}
path := strings.TrimSpace(in.Path)
if path == "" {
return nil, zip.ErrBadRequest("path is required")
}
if in.Line < 0 || in.Character < 0 {
return nil, zip.ErrBadRequest("line and character are 0-based and cannot be negative")
}
relation := ""
if op == "locate" {
if relation = strings.TrimSpace(in.Relation); relation == "" {
relation = "definition"
}
if !slices.Contains(relations, relation) {
return nil, zip.ErrBadRequest("relation must be one of " + strings.Join(relations, ", "))
}
}
// MONEY GATE — before the checkout, so an out-of-funds caller is refused
// rather than served work nobody can be billed for.
// MONEY GATE — before the first byte of work, at the PREPARE price, because
// whether this revision is already prepared is not known until the daemon is
// asked and is not a fact about the caller.
c, onHTTP := cloud.Request(ctx)
if onHTTP {
if err := s.gate(ctx, c, org); err != nil {
@@ -196,261 +272,161 @@ func (s *state) ask(ctx context.Context, in *Query) (*Answer, error) {
}
}
tree, cold, err := s.pool.get(ctx, key{org: org, repo: repo, rev: rev}, in.Path)
// The commit, from the git plane, for the caller's own org. The daemon keys a
// root by a RESOLVED sha and refuses anything else — which is what makes a
// root immutable, and therefore what removes cache invalidation from the
// whole service: a branch moves, a commit never does.
sha, err := s.rev(ctx, c, org, repo, want)
if err != nil {
s.Log.Warn("lsp workspace failed", "org", org, "repo", repo, "err", err)
return nil, zip.ErrInternal("workspace unavailable")
return nil, err
}
abs, err := clean(tree.dir, in.Path)
if err != nil {
return nil, zip.ErrBadRequest(err.Error())
}
uri, err := tree.conn.Open(tree.lang, abs)
if err != nil {
return nil, zip.ErrInternal("open document")
q := &question{
Org: org, Repo: repo, Rev: sha,
Op: op, Relation: relation,
Path: path, Line: in.Line, Character: in.Character,
}
out := &Answer{Repo: repo, Rev: sha, Path: path}
out := &Answer{
Method: method, Repo: repo, Rev: rev,
Path: in.Path, Lang: tree.lang.Name, Cold: cold,
err = s.daemon.ask(ctx, q, out)
if errors.Is(err, errNeedTree) {
// The daemon holds no root for this commit and cannot go and get one. Send
// the tree, then ask again — ONCE. A second 409 is the daemon evicting a
// root as fast as this fills it, and retrying that is a loop, not a fix.
if err = s.prepare(ctx, c, org, repo, sha, out); err != nil {
return nil, err
}
err = s.daemon.ask(ctx, q, out)
}
if err := s.resolve(ctx, tree, out, uri, in); err != nil {
s.Log.Warn("lsp query failed", "org", org, "repo", repo, "method", method, "err", err)
if err != nil {
if decided(err) {
return nil, err
}
s.Log.Warn("lsp query failed", "org", org, "repo", repo, "op", op, "err", err)
return nil, zip.ErrInternal("language server did not answer")
}
if onHTTP {
s.charge(c, org, method, cold)
s.charge(c, org, out.Cold)
}
return out, nil
}
// resolve asks the server the one question and folds its reply into out.
//
// diagnostics is the outlier and is handled first: it is not a request at all in
// LSP but an unsolicited notification the server publishes after didOpen, so it
// is collected rather than called.
func (s *state) resolve(ctx context.Context, t *Tree, out *Answer, uri string, in *Query) error {
ctx, cancel := context.WithTimeout(ctx, callWait)
defer cancel()
if in.Method == "diagnostics" {
out.Diagnostics = t.conn.Diagnostics(ctx, uri, diagSettle)
if out.Diagnostics == nil {
out.Diagnostics = []Diagnostic{}
}
return nil
}
doc := map[string]any{"uri": uri}
pos := map[string]any{"line": in.Line, "character": in.Character}
params := map[string]any{"textDocument": doc, "position": pos}
var call string
switch in.Method {
case "hover":
call = "textDocument/hover"
case "definition":
call = "textDocument/definition"
case "typeDefinition":
call = "textDocument/typeDefinition"
case "implementation":
call = "textDocument/implementation"
case "completion":
call = "textDocument/completion"
case "references":
call = "textDocument/references"
params["context"] = map[string]any{"includeDeclaration": true}
case "documentSymbol":
call = "textDocument/documentSymbol"
params = map[string]any{"textDocument": doc} // no position: the whole file
default:
return fmt.Errorf("unroutable method %q", in.Method) // known() already refused this
}
raw, err := t.conn.Call(ctx, call, params)
// prepare hands the daemon the tree for one commit and records on out whether
// that call actually built it.
func (s *state) prepare(ctx context.Context, c *zip.Ctx, org, repo, sha string, out *Answer) error {
files, err := s.files(ctx, c, org, repo, sha)
if err != nil {
return err
}
fold(out, in.Method, raw, t.dir)
got, err := s.daemon.root(ctx, &tree{Org: org, Repo: repo, Rev: sha, Files: files})
if err != nil {
if decided(err) {
return err
}
s.Log.Warn("lsp prepare failed", "org", org, "repo", repo, "rev", sha, "err", err)
return zip.ErrInternal("language server did not accept the tree")
}
out.Cold = got.Cold
return nil
}
// fold decodes the server's result into the field the method names.
//
// A null result is not an error: "no definition here" is a real, useful answer,
// and it arrives as JSON null. Every branch therefore leaves out's slice empty
// rather than failing, so a caller distinguishes "nothing found" from "the server
// broke" by status code and not by guesswork.
func fold(out *Answer, method string, raw json.RawMessage, dir string) {
if len(raw) == 0 || string(raw) == "null" {
return
}
switch method {
case "hover":
out.Hover = hover(raw)
case "documentSymbol":
out.Symbols = symbols(raw)
case "completion":
out.Completions = completions(raw)
default: // every location-shaped method
out.Locations = locations(raw, dir)
}
// decided reports whether err already carries the status and message a client
// should see. Anything else is this deployment's problem and not the caller's, so
// it is logged where it happened and answered generically.
func decided(err error) bool {
var he *zip.HTTPError
return errors.As(err, &he)
}
// locations decodes the three shapes a location-returning request may answer with
// — a single Location, an array of them, or an array of LocationLink (the
// linkSupport form, whose target range lives under a different key). All three
// are in the specification and gopls, rust-analyzer and tsserver do not agree on
// which to send, so all three are read.
func locations(raw json.RawMessage, dir string) []Location {
var many []struct {
URI string `json:"uri"`
Range Range `json:"range"`
TargetURI string `json:"targetUri"`
Target Range `json:"targetSelectionRange"`
}
if json.Unmarshal(raw, &many) != nil {
var one struct {
URI string `json:"uri"`
Range Range `json:"range"`
}
if json.Unmarshal(raw, &one) != nil || one.URI == "" {
return []Location{}
}
return []Location{{Path: rel(uriPath(one.URI), dir), Range: one.Range}}
}
// ── the git plane ────────────────────────────────────────────────────────────
out := make([]Location, 0, len(many))
for _, m := range many {
uri, rng := m.URI, m.Range
if uri == "" { // a LocationLink
uri, rng = m.TargetURI, m.Target
}
p := uriPath(uri)
if p == "" {
continue // a jar:/zipfile: target names no path of ours
}
out = append(out, Location{Path: rel(p, dir), Range: rng})
// The two calls this package makes to git, held in variables for the one thing a
// variable buys here: a test can drive the real handler without standing up a
// second process to answer it. Neither is ever reassigned in production — the
// only writer is a test, and the compiler holds each signature to the generated
// client's.
//
// They are two ops rather than one because they cost differently. resolveRev is a
// ref lookup and runs on EVERY request; readTree is a walk of the whole
// repository and runs only when the daemon says it holds no root. Folding them
// into one call would drag a monorepo across a socket to answer a hover.
var (
resolveRev = gitplane.GitRev
readTree = gitplane.GitFiles
)
// rev resolves what the caller named to the commit it names, in the caller's own
// org. An empty ref is the repository's default branch.
func (s *state) rev(ctx context.Context, c *zip.Ctx, org, repo, want string) (string, error) {
got, err := resolveRev(as(ctx, c, org), &plane.RevIn{Repo: repo, Ref: want})
if err != nil || got == nil {
s.Log.Warn("lsp resolve failed", "org", org, "repo", repo, "ref", want, "err", err)
return "", zip.ErrNotFound("no such repository or revision in your org")
}
return out
return got.Rev, nil
}
// rel renders a path repo-relative when it is inside the checkout. A path OUTSIDE
// it — a definition in the module cache — is returned as the server gave it,
// because that is a real location and pretending otherwise would lose it.
// files reads the repository's TEXT at one commit, through git's own object
// plane — the read that replaced cloning for delivery, and the ONE read of a
// repository this fleet has. Nothing here checks anything out: a language server
// needs the bytes of some files at one revision, which is a tree read, not a
// packfile.
//
// The checkout is matched in BOTH spellings, resolved and raw. A language server
// reports paths as the OS handed them to it, and a data directory reached through
// a symlink — /var → /private/var on a Mac, a mounted volume in the cluster —
// gives one file two spellings. Comparing one resolved path against one raw one
// puts every location "outside" the checkout, and the fallback then hands the
// caller the worker's ABSOLUTE path for files that were in their own repo all
// along.
//
// This is presentation, not the security boundary: [clean] is what proves a
// requested path is inside the tree, and it resolves symlinks precisely because
// it has to.
func rel(abs, dir string) string {
if abs == "" {
return ""
// Binary and truncated blobs are dropped rather than sent. A language server
// parses source; a binary spends the daemon's tree budget on bytes no server will
// read, and a truncated file is a HALF file, which type-checks to errors that are
// not in the repository.
func (s *state) files(ctx context.Context, c *zip.Ctx, org, repo, sha string) ([]file, error) {
got, err := readTree(as(ctx, c, org), &plane.FilesIn{Repo: repo, Ref: sha, Glob: whole})
if err != nil || got == nil {
s.Log.Warn("lsp tree read failed", "org", org, "repo", repo, "rev", sha, "err", err)
return nil, zip.ErrInternal("repository unavailable")
}
roots := []string{dir}
if resolved, err := filepath.EvalSymlinks(dir); err == nil && resolved != dir {
roots = append(roots, resolved)
}
for _, root := range roots {
if r, err := filepath.Rel(root, abs); err == nil && !strings.HasPrefix(r, "..") {
return filepath.ToSlash(r)
}
}
return abs
}
// hover decodes MarkupContent, a MarkedString, or an array of either.
func hover(raw json.RawMessage) string {
var h struct {
Contents json.RawMessage `json:"contents"`
}
if json.Unmarshal(raw, &h) != nil || len(h.Contents) == 0 {
return ""
}
var markup struct {
Value string `json:"value"`
}
if json.Unmarshal(h.Contents, &markup) == nil && markup.Value != "" {
return markup.Value
}
var plain string
if json.Unmarshal(h.Contents, &plain) == nil {
return plain
}
var list []json.RawMessage
if json.Unmarshal(h.Contents, &list) != nil {
return ""
}
parts := make([]string, 0, len(list))
for _, item := range list {
if json.Unmarshal(item, &markup) == nil && markup.Value != "" {
parts = append(parts, markup.Value)
out := make([]file, 0, len(got.Files))
for _, f := range got.Files {
if f.Truncated || !text(f.Data) {
continue
}
if json.Unmarshal(item, &plain) == nil && plain != "" {
parts = append(parts, plain)
}
out = append(out, file{Path: f.Path, Content: string(f.Data)})
}
return strings.Join(parts, "\n\n")
if len(out) == 0 {
return nil, zip.ErrBadRequest("this revision holds no source the language servers read")
}
return out, nil
}
func symbols(raw json.RawMessage) []Symbol {
var list []struct {
Name string `json:"name"`
Kind int `json:"kind"`
Detail string `json:"detail"`
Range Range `json:"range"`
Location struct {
Range Range `json:"range"`
} `json:"location"`
}
if json.Unmarshal(raw, &list) != nil {
return []Symbol{}
}
out := make([]Symbol, 0, len(list))
for _, s := range list {
rng := s.Range
if rng == (Range{}) { // SymbolInformation carries it under location
rng = s.Location.Range
}
out = append(out, Symbol{Name: s.Name, Kind: s.Kind, Detail: s.Detail, Range: rng})
}
return out
// whole is the glob for a whole tree: `**` matches zero or more whole segments,
// so as the only segment it selects every file beneath the root.
const whole = "**"
// text reports whether a blob is source. NUL and invalid UTF-8 are what separate
// a compiled object or an image from a file a parser can open.
func text(b []byte) bool {
return len(b) > 0 && !bytes.ContainsRune(b, 0) && utf8.Valid(b)
}
// completions decodes CompletionList or a bare CompletionItem array, and bounds
// the reply: a server offering every identifier in a large dependency tree can
// answer with tens of thousands of items, which is not an answer anybody reads.
func completions(raw json.RawMessage) []Completion {
const maxItems = 200
type item struct {
Label string `json:"label"`
Kind int `json:"kind"`
Detail string `json:"detail"`
// ── the identity seam ────────────────────────────────────────────────────────
// tenant is the VALIDATED org for a typed op — the one the gateway asserted and
// cloud.Bridge parked on the context, never a field of Query. A Query field is
// caller-supplied, so a tenant key read from one is a cross-tenant read the
// caller asserted for itself. Fails closed off the HTTP path.
func tenant(ctx context.Context) (string, error) {
org, ok := principal.OrgFrom(ctx)
if !ok {
return "", zip.ErrForbidden("valid principal required")
}
var list struct {
Items []item `json:"items"`
}
var items []item
if json.Unmarshal(raw, &list) == nil && list.Items != nil {
items = list.Items // CompletionList
} else if json.Unmarshal(raw, &items) != nil {
return []Completion{} // neither shape
}
if len(items) > maxItems {
items = items[:maxItems]
}
out := make([]Completion, 0, len(items))
for _, i := range items {
out = append(out, Completion{Label: i.Label, Kind: i.Kind, Detail: i.Detail})
}
return out
return org, nil
}
// as is the context a git-plane call rides: THIS request's principal, delegated
// unchanged, so git answers for the caller's own authority and this package can
// never name another org. Off the HTTP path there is no request to delegate, so
// the already-validated org is stated explicitly instead.
func as(ctx context.Context, c *zip.Ctx, org string) context.Context {
if c == nil {
return cloud.For(ctx, org)
}
return cloud.As(c, "")
}
+473
View File
@@ -0,0 +1,473 @@
package lsp
// lsp_test.go drives the REAL routes against a real daemon on loopback.
//
// The daemon here is an httptest server speaking the actual wire contract — POST
// /root, POST /ask, 409 {"need":"tree"}, X-API-Key — so these tests need no
// gVisor, no gopls and no network, and still exercise the same client the binary
// ships. The git peer is substituted at its two variables (resolveRev, readTree),
// which is what lets one process stand in for two.
//
// Four properties, and they are the four this rewrite has to hold:
//
// THE OPS REACH THE RIGHT DOOR. Five ops, one daemon URL each, with locate's
// relation carried and defaulted.
// A COLD REVISION IS ONE ROUND TRIP MORE, NOT A LOOP. 409 → /root → ask again,
// exactly once.
// THE ORG IS THE PRINCIPAL'S. A body that names an org is answered for the
// principal's org anyway, because the body cannot carry one at all.
// THE KEY IS PRESENTED. Every call to the daemon carries LSP_KEY, and a proxy
// without one refuses instead of calling out.
import (
"bytes"
"context"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"slices"
"testing"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/plane"
luxlog "github.com/luxfi/log"
"github.com/zap-proto/zip"
)
const (
testKey = "test-service-key"
testSHA = "0123456789abcdef0123456789abcdef01234567"
)
// call is one request the daemon received: which door, what it said, and which
// key it presented.
type call struct {
path string
key string
ask question
tree tree
}
// fleet is one test's world: a daemon on loopback, a git peer answering from
// memory, and the real routes over both.
type fleet struct {
app *zip.App
// held, once true, is a daemon that holds a root for the revision it is
// asked about. It starts false — the cold state — and /root sets it, unless
// stuck says this daemon never keeps one.
held bool
stuck bool
// files is what the git peer answers a whole-tree read with.
files []plane.File
calls []call // every daemon request, in order
orgs []string // the org each git-plane call was made FOR
refs []string // the ref each whole-tree read was pinned to
globs []string
}
// newFleet stands the world up. key is what the proxy is configured with, so a
// test can also describe a deployment that never got one.
func newFleet(t *testing.T, key string, files []plane.File) *fleet {
t.Helper()
f := &fleet{files: files}
mux := http.NewServeMux()
mux.HandleFunc("POST /ask", func(w http.ResponseWriter, r *http.Request) {
var in question
f.take(t, r, "/ask", &in, func(c *call) { c.ask = in })
if !f.held {
w.WriteHeader(http.StatusConflict)
_ = json.NewEncoder(w).Encode(map[string]any{
"need": "tree", "org": in.Org, "repo": in.Repo, "rev": in.Rev,
})
return
}
_ = json.NewEncoder(w).Encode(map[string]any{
"op": in.Op, "lang": "go", "hover": "func Hello()",
"locations": []map[string]any{{"path": "a.go"}},
})
})
mux.HandleFunc("POST /root", func(w http.ResponseWriter, r *http.Request) {
var in tree
f.take(t, r, "/root", &in, func(c *call) { c.tree = in })
f.held = !f.stuck
_ = json.NewEncoder(w).Encode(ready{Ready: true, Cold: true, Langs: []string{"go"}})
})
up := httptest.NewServer(mux)
t.Cleanup(up.Close)
prevRev, prevTree := resolveRev, readTree
resolveRev = func(ctx context.Context, in *plane.RevIn) (*plane.Rev, error) {
f.orgs = append(f.orgs, cloud.Who(ctx).Org)
return &plane.Rev{Rev: testSHA, Ref: "main"}, nil
}
readTree = func(ctx context.Context, in *plane.FilesIn) (*plane.Files, error) {
f.orgs = append(f.orgs, cloud.Who(ctx).Org)
f.refs = append(f.refs, in.Ref)
f.globs = append(f.globs, in.Glob)
return &plane.Files{Rev: testSHA, Files: f.files}, nil
}
t.Cleanup(func() { resolveRev, readTree = prevRev, prevTree })
s := &state{
Base: cloud.Base{Log: luxlog.New("test")},
daemon: &daemon{url: up.URL, key: key, http: &http.Client{Timeout: prepareWait}},
}
f.app = zip.New(zip.Config{Logger: luxlog.New("test")})
// A subsystem never installs cloud.Bridge — the composer does, once, after
// the identity check that mints the validated org. In a test the test IS the
// composer, so it owes the same thing.
f.app.Use(cloud.Bridge())
// The REAL registration, not a reconstruction of it: routes() is what Mount
// calls, so every typed op is exercised here exactly as the binary serves it.
if err := routes(f.app, s); err != nil {
t.Fatalf("routes: %v", err)
}
return f
}
// warm is a fleet whose daemon already holds the root, which is the steady state.
func warm(t *testing.T) *fleet {
t.Helper()
f := newFleet(t, testKey, source())
f.held = true
return f
}
func (f *fleet) take(t *testing.T, r *http.Request, path string, in any, set func(*call)) {
t.Helper()
body, err := io.ReadAll(r.Body)
if err != nil {
t.Fatalf("read %s body: %v", path, err)
}
if err := json.Unmarshal(body, in); err != nil {
t.Fatalf("decode %s body %s: %v", path, body, err)
}
c := call{path: path, key: r.Header.Get("X-API-Key")}
set(&c)
f.calls = append(f.calls, c)
}
// paths is the door sequence the daemon saw.
func (f *fleet) paths() []string {
out := make([]string, 0, len(f.calls))
for _, c := range f.calls {
out = append(out, c.path)
}
return out
}
// sent is the tree the daemon was given, if it was given one.
func (f *fleet) sent() tree {
for _, c := range f.calls {
if c.path == "/root" {
return c.tree
}
}
return tree{}
}
func (f *fleet) post(t *testing.T, op, org string, body any) (int, []byte) {
t.Helper()
b, err := json.Marshal(body)
if err != nil {
t.Fatalf("marshal: %v", err)
}
req := httptest.NewRequest(http.MethodPost, "/v1/code/lsp/"+op, bytes.NewReader(b))
req.Header.Set("Content-Type", "application/json")
if org != "" {
// A VALIDATED principal, as SanitizeIdentity mints one from a verified
// token — which is the only thing that satisfies the org gate.
req.Header.Set("X-Org-Id", org)
req.Header.Set("X-User-Id", "u_"+org)
}
res, err := f.app.Test(req)
if err != nil {
t.Fatalf("Test %s: %v", op, err)
}
defer func() { _ = res.Body.Close() }()
out, _ := io.ReadAll(res.Body)
return res.StatusCode, out
}
func source() []plane.File {
return []plane.File{{Path: "a.go", Data: []byte("package p\n")}}
}
// TestEachOpReachesAskUnderItsOwnName proves the door→op mapping: every route
// posts /ask, and the op it names is its own. A mapping that drifts here answers
// a hover as a completion, which no status code would reveal.
func TestEachOpReachesAskUnderItsOwnName(t *testing.T) {
for _, tc := range []struct {
op string
relation string
in Query
}{
{"hover", "", Query{Repo: "cloud", Path: "a.go", Line: 3, Character: 7}},
{"symbols", "", Query{Repo: "cloud", Path: "a.go"}},
{"diagnostics", "", Query{Repo: "cloud", Path: "a.go"}},
{"complete", "", Query{Repo: "cloud", Path: "a.go", Line: 1, Character: 4}},
// locate defaults its relation to definition — the question an editor's
// go-to means — and carries an explicit one through untouched.
{"locate", "definition", Query{Repo: "cloud", Path: "a.go"}},
{"locate", "reference", Query{Repo: "cloud", Path: "a.go", Relation: "reference"}},
{"locate", "type", Query{Repo: "cloud", Path: "a.go", Relation: "type"}},
{"locate", "implementation", Query{Repo: "cloud", Path: "a.go", Relation: "implementation"}},
} {
t.Run(tc.op+"/"+tc.relation, func(t *testing.T) {
f := warm(t)
code, body := f.post(t, tc.op, "acme", tc.in)
if code != http.StatusOK {
t.Fatalf("status=%d body=%s", code, body)
}
if got := f.paths(); !slices.Equal(got, []string{"/ask"}) {
t.Fatalf("daemon saw %v, want one /ask", got)
}
got := f.calls[0].ask
if got.Op != tc.op {
t.Errorf("op = %q, want %q", got.Op, tc.op)
}
if got.Relation != tc.relation {
t.Errorf("relation = %q, want %q", got.Relation, tc.relation)
}
// The position is the LSP's and passes through untouched.
if got.Line != tc.in.Line || got.Character != tc.in.Character {
t.Errorf("position = %d:%d, want %d:%d",
got.Line, got.Character, tc.in.Line, tc.in.Character)
}
// The rev on the wire is the RESOLVED commit, never what was named.
if got.Rev != testSHA {
t.Errorf("rev = %q, want the resolved sha %q", got.Rev, testSHA)
}
})
}
}
// TestAnUnknownRelationIsRefused proves locate's relation is a CLOSED set, so an
// arbitrary string never reaches a language server.
func TestAnUnknownRelationIsRefused(t *testing.T) {
f := warm(t)
code, _ := f.post(t, "locate", "acme", Query{Repo: "cloud", Path: "a.go", Relation: "everything"})
if code != http.StatusBadRequest {
t.Fatalf("status=%d, want 400", code)
}
if len(f.calls) != 0 {
t.Fatalf("daemon saw %v; a refused relation must cost no round trip", f.paths())
}
}
// TestAColdRevisionSendsTheTreeAndAsksAgainOnce proves the 409 contract: the
// daemon says it holds no root, this side supplies one, and asks EXACTLY once
// more.
func TestAColdRevisionSendsTheTreeAndAsksAgainOnce(t *testing.T) {
f := newFleet(t, testKey, source())
code, body := f.post(t, "hover", "acme", Query{Repo: "cloud", Path: "a.go"})
if code != http.StatusOK {
t.Fatalf("status=%d body=%s", code, body)
}
if got := f.paths(); !slices.Equal(got, []string{"/ask", "/root", "/ask"}) {
t.Fatalf("daemon saw %v, want [/ask /root /ask]", got)
}
// The tree is keyed by the caller's org and the RESOLVED commit, and carries
// the repository's text.
sent := f.sent()
if sent.Org != "acme" || sent.Repo != "cloud" || sent.Rev != testSHA {
t.Errorf("tree keyed (%q,%q,%q), want (acme,cloud,%s)", sent.Org, sent.Repo, sent.Rev, testSHA)
}
if len(sent.Files) != 1 || sent.Files[0].Path != "a.go" || sent.Files[0].Content != "package p\n" {
t.Errorf("tree files = %+v, want a.go with its content", sent.Files)
}
// Preparing a revision is the BILLED event, and the answer says so.
var out Answer
if err := json.Unmarshal(body, &out); err != nil {
t.Fatalf("decode answer %s: %v", body, err)
}
if !out.Cold {
t.Error("cold = false; the request that prepared the revision must report it")
}
if out.Rev != testSHA || out.Repo != "cloud" || out.Path != "a.go" {
t.Errorf("answer echoed (%q,%q,%q)", out.Repo, out.Rev, out.Path)
}
}
// TestTheRetryIsOnceAndNotALoop bounds the retry. A daemon that refuses again
// after being given a tree — one evicting roots as fast as they arrive — gets one
// tree and one more ask, never a third, and the caller gets an outage.
func TestTheRetryIsOnceAndNotALoop(t *testing.T) {
f := newFleet(t, testKey, source())
f.stuck = true
code, _ := f.post(t, "hover", "acme", Query{Repo: "cloud", Path: "a.go"})
if code != http.StatusInternalServerError {
t.Fatalf("status=%d, want 500 — a daemon that never holds a root is an outage", code)
}
if got := f.paths(); !slices.Equal(got, []string{"/ask", "/root", "/ask"}) {
t.Fatalf("daemon saw %v, want exactly [/ask /root /ask] — the retry is once", got)
}
}
// TestTheOrgIsThePrincipalsAndNeverTheBodys is the tenant boundary. A body that
// names another org is answered for the principal's org, because Query has no org
// field for one to land in — and the git plane is asked for that same org, so a
// repository slug can only ever resolve under the caller.
func TestTheOrgIsThePrincipalsAndNeverTheBodys(t *testing.T) {
f := newFleet(t, testKey, source())
body := map[string]any{
"repo": "cloud", "path": "a.go",
"org": "victim", "Org": "victim", "owner": "victim", "tenant": "victim",
}
code, out := f.post(t, "hover", "acme", body)
if code != http.StatusOK {
t.Fatalf("status=%d body=%s", code, out)
}
for _, c := range f.calls {
if c.path == "/ask" && c.ask.Org != "acme" {
t.Errorf("/ask carried org %q, want acme", c.ask.Org)
}
if c.path == "/root" && c.tree.Org != "acme" {
t.Errorf("/root carried org %q, want acme", c.tree.Org)
}
}
if len(f.orgs) == 0 {
t.Fatal("the git plane was never asked")
}
for _, org := range f.orgs {
if org != "acme" {
t.Errorf("git plane called for org %q, want acme", org)
}
}
}
// TestNoPrincipalIsRefusedBeforeAnythingIsReached is the fail-closed spine: an
// anonymous request reaches neither the git plane nor the daemon.
func TestNoPrincipalIsRefusedBeforeAnythingIsReached(t *testing.T) {
f := warm(t)
code, _ := f.post(t, "hover", "", Query{Repo: "cloud", Path: "a.go"})
if code != http.StatusForbidden {
t.Fatalf("status=%d, want 403", code)
}
if len(f.calls) != 0 || len(f.orgs) != 0 {
t.Fatalf("an anonymous request reached daemon=%v git=%v", f.paths(), f.orgs)
}
}
// TestEveryDaemonCallPresentsTheKey proves the shared service key is sent on both
// doors. The daemon compares it in constant time and 401s without it, so a
// forgotten header is a fleet that cannot answer at all.
func TestEveryDaemonCallPresentsTheKey(t *testing.T) {
f := newFleet(t, testKey, source())
if code, body := f.post(t, "locate", "acme", Query{Repo: "cloud", Path: "a.go"}); code != http.StatusOK {
t.Fatalf("status=%d body=%s", code, body)
}
if len(f.calls) == 0 {
t.Fatal("daemon saw nothing")
}
for _, c := range f.calls {
if c.key != testKey {
t.Errorf("%s presented key %q, want the configured one", c.path, c.key)
}
}
}
// TestAnUnkeyedProxyRefusesRatherThanCallsOut is the other half of the same fact:
// a deployment that never got LSP_KEY fails CLOSED here rather than making a call
// the daemon would refuse anyway.
func TestAnUnkeyedProxyRefusesRatherThanCallsOut(t *testing.T) {
f := newFleet(t, "", source())
f.held = true
code, _ := f.post(t, "hover", "acme", Query{Repo: "cloud", Path: "a.go"})
if code != http.StatusServiceUnavailable {
t.Fatalf("status=%d, want 503", code)
}
if len(f.calls) != 0 {
t.Fatalf("daemon saw %v; an unkeyed proxy must not call out", f.paths())
}
}
// TestNewDaemonReadsItsWiringFromTheEnvironment proves the two facts a Deployment
// supplies — where the daemon is and which key reaches it — are read from the
// environment and nowhere else, and that the address has a working default.
func TestNewDaemonReadsItsWiringFromTheEnvironment(t *testing.T) {
t.Setenv(upstreamEnv, "")
t.Setenv(keyEnv, "")
if d := newDaemon(); d.url != upstreamDefault || d.key != "" {
t.Fatalf("unconfigured daemon = (%q,%q), want (%q,\"\")", d.url, d.key, upstreamDefault)
}
t.Setenv(upstreamEnv, "http://elsewhere:9000")
t.Setenv(keyEnv, " k ")
if d := newDaemon(); d.url != "http://elsewhere:9000" || d.key != "k" {
t.Fatalf("configured daemon = (%q,%q)", d.url, d.key)
}
}
// TestOnlySourceIsSentToTheDaemon proves the tree read is filtered: a binary blob
// and a truncated one are dropped. A binary spends the daemon's tree budget on
// bytes no parser reads; a truncated file is a HALF file, and type-checking one
// invents errors that are not in the repository.
func TestOnlySourceIsSentToTheDaemon(t *testing.T) {
f := newFleet(t, testKey, []plane.File{
{Path: "a.go", Data: []byte("package p\n")},
{Path: "logo.png", Data: []byte{0x89, 'P', 'N', 'G', 0x00, 0x1a}},
{Path: "huge.go", Truncated: true},
})
if code, body := f.post(t, "hover", "acme", Query{Repo: "cloud", Path: "a.go"}); code != http.StatusOK {
t.Fatalf("status=%d body=%s", code, body)
}
sent := f.sent()
if len(sent.Files) != 1 || sent.Files[0].Path != "a.go" {
t.Fatalf("tree carried %+v, want only a.go", sent.Files)
}
}
// TestTheTreeIsReadAtTheResolvedCommit proves the git read is pinned to the
// commit the daemon was told about, not to the ref the caller named — so the tree
// and the root key can never come from two sides of a push. The glob is the whole
// tree, which is the SAME read the code index takes.
func TestTheTreeIsReadAtTheResolvedCommit(t *testing.T) {
f := newFleet(t, testKey, source())
if code, body := f.post(t, "hover", "acme",
Query{Repo: "cloud", Rev: "main", Path: "a.go"}); code != http.StatusOK {
t.Fatalf("status=%d body=%s", code, body)
}
if !slices.Equal(f.refs, []string{testSHA}) {
t.Errorf("tree read at %v, want the resolved commit [%s]", f.refs, testSHA)
}
if !slices.Equal(f.globs, []string{whole}) {
t.Errorf("tree read with globs %v, want [%s]", f.globs, whole)
}
}
// TestAMalformedRequestCostsNoCall proves the narrowing happens HERE: a slug or a
// ref that cannot name anything is refused before the git plane or the daemon is
// reached.
func TestAMalformedRequestCostsNoCall(t *testing.T) {
f := warm(t)
for _, in := range []Query{
{Repo: "", Path: "a.go"},
{Repo: "../other", Path: "a.go"},
{Repo: "a/b", Path: "a.go"},
{Repo: "-flag", Path: "a.go"},
{Repo: "cloud", Path: ""},
{Repo: "cloud", Rev: "--upload-pack=x", Path: "a.go"},
{Repo: "cloud", Path: "a.go", Line: -1},
{Repo: "cloud", Path: "a.go", Character: -1},
} {
if code, _ := f.post(t, "hover", "acme", in); code != http.StatusBadRequest {
t.Errorf("%+v: status=%d, want 400", in, code)
}
}
if len(f.calls) != 0 || len(f.orgs) != 0 {
t.Fatalf("a malformed request reached daemon=%v git=%v", f.paths(), f.orgs)
}
}
+38 -27
View File
@@ -5,21 +5,22 @@ package lsp
// the composition root builds, and the prepaid Commerce ledger behind it is the
// one ledger. Nothing here is a second accounting of anything.
//
// # What is billed, and why it is the cold start
// # What is billed, and why it is the prepare
//
// A COLD start is a git checkout, a dependency fetch and a language server
// PREPARING a revision is a tree write, a dependency fetch and a language server
// indexing a repository: seconds to minutes of CPU, hundreds of megabytes, and a
// process that then sits resident. That is the cost this service actually incurs,
// so that is the event that carries a fee.
// so that is the event that carries a fee. There are two Models on the ledger and
// they name exactly that difference: "prepare" and "query".
//
// A WARM point query is a JSON-RPC round trip to a process that is already
// running and already holds the index. It costs microseconds. Charging per query
// would price the cheap thing and hide the expensive one, which teaches callers
// to re-key their workspace instead of reusing it — the opposite of what the pool
// is for. Warm queries are recorded for attribution and cost nothing.
// A QUERY against a prepared revision is a JSON-RPC round trip to a process that
// is already running and already holds the index. It costs microseconds. Charging
// per query would price the cheap thing and hide the expensive one, which teaches
// callers to re-key their revision instead of reusing it — the opposite of what
// the daemon's pool is for. Queries are recorded for attribution and cost nothing.
//
// The GATE runs before the work, not after: an out-of-funds caller gets a clean
// 402 instead of a checkout we performed and cannot bill.
// 402 instead of a dependency fetch we paid for and cannot bill.
import (
"context"
@@ -35,11 +36,19 @@ import (
// the one string commerce groups these charges by.
const kind = "lsp"
// coldCents is the flat fee for one cold start. Flat rather than measured because
// the caller chooses the repository, not the cost of indexing it, and a bill that
// varies with how large somebody else's dependency tree turned out to be is not
// one anybody can predict.
const coldCents = 2
// prepareCents is the flat fee for preparing one revision. Flat rather than
// measured because the caller chooses the repository, not the cost of indexing
// it, and a bill that varies with how large somebody else's dependency tree
// turned out to be is not one anybody can predict.
const prepareCents = 2
// The two Models this surface books under — what the ledger row says the money
// bought. They are the only two things that happen here, and which one it was is
// the daemon's answer, never a guess made on this side.
const (
modelPrepare = "prepare"
modelQuery = "query"
)
// payer is the org whose ledger this request debits: principal.Ledger, the
// SELECTED billing org, which a SuperAdmin masquerade deliberately moves off the
@@ -57,12 +66,13 @@ func payer(c *zip.Ctx, org string) string {
return org
}
// gate refuses the request unless the payer can cover a cold start.
// gate refuses the request unless the payer can cover a prepare.
//
// It gates the COLD price on every request, including ones that will turn out to
// be warm, because whether a workspace is warm is not known until the pool is
// asked — and it is not a fact about the caller. Gating the worst case and
// charging the real one is the order that never bills for work it refused.
// It gates the PREPARE price on every request, including ones the daemon will
// answer from a revision it already holds, because whether it holds one is not
// known until it is asked — and it is not a fact about the caller. Gating the
// worst case and charging the real one is the order that never bills for work it
// refused.
func (s *state) gate(ctx context.Context, c *zip.Ctx, org string) error {
subject := payer(c, org)
if subject == "" {
@@ -72,25 +82,26 @@ func (s *state) gate(ctx context.Context, c *zip.Ctx, org string) error {
// the gate enforces — which is a different question from the attribution
// scope the ledger records below, and answered by a different call.
project, validated := principal.ValidatedProject(c)
if err := s.Bill.Gate(ctx, subject, project, validated, kind, coldCents); err != nil {
if err := s.Bill.Gate(ctx, subject, project, validated, kind, prepareCents); err != nil {
return cloud.DenyResource(c, err)
}
return nil
}
// charge records the debit once the work is done. A warm query debits zero — it
// is still recorded, so per-project attribution sees the traffic.
func (s *state) charge(c *zip.Ctx, org, method string, cold bool) {
// charge records the debit once the work is done. A query against an already
// prepared revision debits zero — it is still recorded, so per-project
// attribution sees the traffic.
func (s *state) charge(c *zip.Ctx, org string, prepared bool) {
subject := payer(c, org)
if subject == "" {
return
}
var cents int64
if cold {
cents = coldCents
model, cents := modelQuery, int64(0)
if prepared {
model, cents = modelPrepare, prepareCents
}
s.Bill.MeterUsage(subject, kind, metering.Usage{
Model: method,
Model: model,
AmountCents: cents,
Project: principal.ProjectScope(c),
RequestID: strings.Clone(strings.TrimSpace(c.Header("X-Request-Id"))),
+37 -80
View File
@@ -1,52 +1,40 @@
package lsp
// mount.go is this subsystem's registration: build the state, bind the door.
// mount.go is this subsystem's registration: build the state, bind the doors.
//
// It follows apps/code's Mount exactly — same signature, same fail-closed
// argument checks, same package-global for Shutdown to reach, and routes() as a
// FUNCTION rather than inline so this package's tests drive the REAL registration
// instead of a reconstruction of it that can drift from what the binary serves.
// argument checks, and routes() as a FUNCTION rather than inline so this
// package's tests drive the REAL registration instead of a reconstruction of it
// that can drift from what the binary serves.
//
// A NOTE ON MOUNT ORDER. The brief asked for "order ~135, before ai's /v1/*
// catch-all at 150". Those integers no longer exist: apps.Wire() is gone, and
// manifest/apps.go is the hand-authored fleet list whose SLICE POSITION is the
// order (build.go: "There is NO Order field"). The "Order 134" in code.go's
// header is a comment describing a position, not a field. So lsp's row sits
// immediately after code's in manifest.Apps — the same intent expressed in the
// mechanism that actually exists — and manifest/order_test.go's frozen sequence
// is updated in the same commit, which is how a reorder stays a decision.
// Routing does not in fact depend on it: nested static prefixes resolve by
// SPECIFICITY, so /v1/lsp beats ai's /v1 wherever it registers.
// A NOTE ON MOUNT ORDER. manifest/apps.go is the hand-authored fleet list whose
// SLICE POSITION is the order (build.go: "There is NO Order field"). lsp's row
// sits immediately after code's, and manifest/order_test.go freezes that
// sequence, so a reorder stays a decision. Routing does not depend on it:
// /v1/code/lsp is a deeper static prefix than both /v1/code and ai's bare /v1,
// and nested static prefixes resolve by SPECIFICITY.
//
// There is no Shutdown. The subprocesses and the checkouts this app used to own
// are the daemon's now; what is left here is an http.Client, which the process
// exiting reclaims. A hook that closed nothing would be a hook nobody could
// delete later without proving it closed nothing.
import (
"context"
"fmt"
"os"
"strings"
"github.com/hanzoai/cloud"
"github.com/zap-proto/zip"
)
// forgeDefault is the git host repositories are checked out from. WHICH host
// this deployment's repositories live on is infra wiring, so it is env with a
// constant default — not a policy row, and never a request field: see checkout,
// where the owner segment is the validated principal's org and the caller
// supplies only a slug.
const forgeDefault = "https://git.hanzo.ai"
// state is the subsystem: the shared Base plus the warm-workspace pool. It holds
// state is the subsystem: the shared Base plus the one daemon client. It holds
// no org in a field — the org is a parameter on every call, so one process serves
// all orgs and an org can never be captured from stale state.
type state struct {
cloud.Base
forge string
pool *pool
daemon *daemon
}
var mounted *state
// Mount wires /v1/lsp onto app per HIP-0106.
// Mount wires /v1/code/lsp onto app per HIP-0106.
func Mount(app cloud.Router, deps cloud.Deps) error {
if app == nil {
return fmt.Errorf("lsp.Mount: nil app")
@@ -57,68 +45,37 @@ func Mount(app cloud.Router, deps cloud.Deps) error {
if deps.DataDir == "" {
return fmt.Errorf("lsp.Mount: empty DataDir")
}
s := &state{Base: cloud.NewBase(deps, "lsp"), forge: forge()}
s.pool = newPool(s.build)
mounted = s
s := &state{Base: cloud.NewBase(deps, "lsp"), daemon: newDaemon()}
if err := routes(app, s); err != nil {
return err
}
s.Log.Info("lsp surface mounted (native)",
"brand", deps.Brand, "forge", s.forge, "languages", len(table))
// The key is a KMS secret, so what is logged is whether one is PRESENT. A
// deployment that forgot it otherwise looks healthy until the first query
// answers 503.
s.Log.Info("lsp surface mounted (proxy)",
"brand", deps.Brand, "upstream", s.daemon.url, "keyed", s.daemon.key != "")
return nil
}
// routes registers the /v1/lsp surface: ONE typed op, because there is one value.
// routes registers the /v1/code/lsp surface: one typed op per question.
//
// Typed rather than raw so the OpenAPI operation, the MCP tool, the CLI command
// and every generated SDK method are all projected from this one entry — a
// surface built for coding agents, where the MCP tool is the point.
// and every generated SDK method are all projected from these five entries — a
// surface built for coding agents, where the MCP tool list is not a side benefit
// of typing it but the point. Five doors rather than one door with an `op` field
// for the same reason: an agent picks a tool by its name and its description, and
// a union behind one name is a tool it has to be told how to use.
func routes(app cloud.Router, s *state) error {
// cloud.Bridge is not installed here: the composer installs it once at the
// root, after the identity check that mints the validated org and before any
// subsystem registers a route — an order only the whole program can assert.
//
// Grouped at "/v1" with "/lsp" as the member, the shape apps/bots and
// apps/account already use for a route that IS its prefix. The obvious
// spelling — Group("/v1/lsp") with an empty member — addresses "/v1/lsp/",
// a different path from the "/v1/lsp" the manifest row publishes, and that
// one-character mismatch between what the binary serves and what the host
// routes is invisible until a client 404s.
g := app.Group("/v1")
zip.Post(g, "/lsp", s.ask)
g := app.Group("/v1/code/lsp")
zip.Post(g, "/hover", s.hover)
zip.Post(g, "/locate", s.locate)
zip.Post(g, "/symbols", s.symbols)
zip.Post(g, "/diagnostics", s.diagnostics)
zip.Post(g, "/complete", s.complete)
return nil
}
// Shutdown closes every warm workspace: each is a live subprocess and a directory
// on disk, and neither is reclaimed by the process exiting cleanly. Idempotent.
func Shutdown(_ context.Context) error {
if mounted == nil {
return nil
}
mounted.pool.closeAll()
mounted = nil
return nil
}
// Invalidate drops every warm revision of one repository, for when a push or a
// re-index makes a checkout stale.
//
// It is exported and unused IN THIS BINARY, which is the honest state of it:
// apps/code and apps/lsp are separate processes, so code cannot call this
// in-process when it re-indexes, and the push signal has to arrive over the bus.
// That is phase 2. It is mostly self-correcting meanwhile — workspaces are keyed
// by revision, so new content is a new key — and the gap is a caller that tracks
// a branch while the branch moves.
func Invalidate(org, repo string) {
if mounted != nil {
mounted.pool.drop(org, repo)
}
}
func forge() string {
if v := strings.TrimSpace(os.Getenv("LSP_FORGE_URL")); v != "" {
return v
}
return forgeDefault
}
-472
View File
@@ -1,472 +0,0 @@
package lsp
// server.go is a JSON-RPC 2.0 client speaking the Language Server Protocol over
// a language server's stdio: Content-Length framing, one multiplexed connection,
// initialize → didOpen → ask.
//
// It is the testable core of this app, so it knows nothing about orgs, HTTP,
// billing or git. It is given a reader, a writer and a Lang; everything else is
// the caller's. [newConn] is the seam that makes that true — [Start] spawns a
// real server and hands it here, and a test hands it an in-process pipe, and
// both drive the identical code.
//
// The one structural decision: a SINGLE reader goroutine owns the stdout side
// and demultiplexes it. LSP is not request/response — a server interleaves
// responses, its own requests, and unsolicited notifications on the same stream,
// and diagnostics are only ever the third kind. Reading inline from Call (which
// is what the Python tool does) drops every message that is not the response
// being waited for, which is why that tool cannot report diagnostics without a
// second read path. One reader, three destinations, no second path.
import (
"bufio"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/url"
"os"
"os/exec"
"path/filepath"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
)
// maxFrame bounds one inbound message. A language server is a subprocess we
// spawned, but it is parsing a tenant's checkout, and a hostile input that makes
// it emit an enormous frame must not become an allocation the pod dies on.
const maxFrame = 32 << 20 // 32 MiB
// handshake, request and shutdown budgets. A cold rust-analyzer or gopls indexes
// before it answers, so initialize is generous where a point query is not.
const (
initWait = 90 * time.Second
callWait = 30 * time.Second
closeWait = 3 * time.Second
)
// Conn is one live language server: a process (or, in a test, a pipe) plus the
// bookkeeping to route its stream. Safe for concurrent use — Call may be entered
// from several requests against the same warm workspace.
type Conn struct {
w io.WriteCloser
stop func() // releases the transport (kills the process)
wmu sync.Mutex // serializes frame writes; a torn frame desynchronizes the stream
seq atomic.Int64
mu sync.Mutex
wait map[int64]chan msg
dmu sync.Mutex
diag map[string][]Diagnostic
done chan struct{} // closed when the reader stops
err error // why it stopped; read only after done
once sync.Once
}
// msg is any JSON-RPC frame in either direction. Which of the four kinds it is
// follows from which fields are present, which is what [Conn.read] switches on.
type msg struct {
ID json.RawMessage `json:"id,omitempty"`
Method string `json:"method,omitempty"`
Params json.RawMessage `json:"params,omitempty"`
Result json.RawMessage `json:"result,omitempty"`
Error *rpcError `json:"error,omitempty"`
}
type rpcError struct {
Code int `json:"code"`
Message string `json:"message"`
}
func (e *rpcError) Error() string { return fmt.Sprintf("lsp: rpc %d: %s", e.Code, e.Message) }
// Start spawns l's language server rooted at root and completes the handshake.
//
// The process is deliberately NOT tied to ctx: a Conn outlives the request that
// warmed it (that is the entire point of the pool), so binding it to the
// request's context would kill the server the moment the caller got its answer.
// [Conn.Close] is what ends it.
func Start(ctx context.Context, l Lang, root string) (*Conn, error) {
if len(l.Start) == 0 {
return nil, fmt.Errorf("lsp: language %q has no server", l.Name)
}
cmd := exec.Command(l.Start[0], l.Start[1:]...)
cmd.Dir = root
cmd.Env = env(l)
// The server's stderr is its own log, not ours to relay: it can be chatty and
// it can echo tenant source. Dropping it keeps both out of our logs.
cmd.Stderr = io.Discard
in, err := cmd.StdinPipe()
if err != nil {
return nil, fmt.Errorf("lsp: stdin: %w", err)
}
out, err := cmd.StdoutPipe()
if err != nil {
return nil, fmt.Errorf("lsp: stdout: %w", err)
}
if err := cmd.Start(); err != nil {
return nil, fmt.Errorf("lsp: start %s: %w", l.Start[0], err)
}
c := newConn(in, out, func() {
if cmd.Process != nil {
_ = cmd.Process.Kill()
}
_ = cmd.Wait()
})
if err := c.handshake(ctx, l, root); err != nil {
c.Close()
return nil, err
}
return c, nil
}
// newConn wires a Conn onto an already-open transport and starts its reader.
// Start uses it for a real process; a test uses it for a pipe.
func newConn(w io.WriteCloser, r io.Reader, stop func()) *Conn {
c := &Conn{
w: w,
stop: stop,
wait: make(map[int64]chan msg),
diag: make(map[string][]Diagnostic),
done: make(chan struct{}),
}
go c.read(bufio.NewReaderSize(r, 64<<10))
return c
}
// handshake performs initialize → initialized. Ported from the Python tool's
// _initialize_lsp, with initializationOptions added (langs.go: Init) because
// that is where rust-analyzer's build scripts get turned off.
func (c *Conn) handshake(ctx context.Context, l Lang, root string) error {
ctx, cancel := context.WithTimeout(ctx, initWait)
defer cancel()
uri := pathURI(root)
params := map[string]any{
"processId": os.Getpid(),
"rootUri": uri,
"rootPath": root,
"capabilities": map[string]any{
"workspace": map[string]any{"workspaceFolders": true, "applyEdit": false},
"textDocument": map[string]any{
"synchronization": map[string]any{"dynamicRegistration": true, "didSave": true},
"completion": map[string]any{"completionItem": map[string]any{"snippetSupport": true}},
"hover": map[string]any{"contentFormat": []string{"markdown", "plaintext"}},
"definition": map[string]any{"dynamicRegistration": true, "linkSupport": true},
"references": map[string]any{"dynamicRegistration": true},
"publishDiagnostics": map[string]any{"relatedInformation": false},
},
},
"workspaceFolders": []map[string]any{{"uri": uri, "name": filepath.Base(root)}},
}
if l.Init != nil {
params["initializationOptions"] = l.Init
}
if _, err := c.Call(ctx, "initialize", params); err != nil {
return fmt.Errorf("lsp: initialize: %w", err)
}
return c.Notify("initialized", map[string]any{})
}
// Call issues a request and waits for the response with that id.
func (c *Conn) Call(ctx context.Context, method string, params any) (json.RawMessage, error) {
id := c.seq.Add(1)
ch := make(chan msg, 1)
c.mu.Lock()
c.wait[id] = ch
c.mu.Unlock()
defer func() {
c.mu.Lock()
delete(c.wait, id)
c.mu.Unlock()
}()
if err := c.send(map[string]any{
"jsonrpc": "2.0", "id": id, "method": method, "params": params,
}); err != nil {
return nil, err
}
select {
case m := <-ch:
if m.Error != nil {
return nil, m.Error
}
return m.Result, nil
case <-c.done:
return nil, c.stopped()
case <-ctx.Done():
return nil, ctx.Err()
}
}
// Notify sends a notification — no id, so no response is expected or waited for.
func (c *Conn) Notify(method string, params any) error {
return c.send(map[string]any{"jsonrpc": "2.0", "method": method, "params": params})
}
// Open sends textDocument/didOpen for a file already inside the workspace, which
// is what makes the server willing to answer about it. Content is read from
// disk: the checkout is the source of truth, and accepting caller-supplied text
// would let one request answer about a file the tenant's repo does not have.
func (c *Conn) Open(l Lang, abs string) (string, error) {
text, err := os.ReadFile(abs)
if err != nil {
return "", fmt.Errorf("lsp: read %s: %w", filepath.Base(abs), err)
}
uri := pathURI(abs)
return uri, c.Notify("textDocument/didOpen", map[string]any{
"textDocument": map[string]any{
"uri": uri, "languageId": l.ID(abs), "version": 1, "text": string(text),
},
})
}
// Diagnostics collects what the server published for uri.
//
// LSP has no "diagnostics complete" signal — publishDiagnostics is unsolicited
// and a server may publish several times as analysis deepens. So this waits for a
// first publication, then for a settle window in which nothing new arrives, and
// reports what it has. A server that publishes nothing (a clean file) is reported
// as clean at the deadline, which is the honest reading.
func (c *Conn) Diagnostics(ctx context.Context, uri string, settle time.Duration) []Diagnostic {
const tick = 50 * time.Millisecond
var last int
var quiet time.Duration
for {
select {
case <-ctx.Done():
return c.published(uri)
case <-c.done:
return c.published(uri)
case <-time.After(tick):
}
got := c.published(uri)
if len(got) != last {
last, quiet = len(got), 0
continue
}
if last > 0 {
if quiet += tick; quiet >= settle {
return got
}
}
}
}
func (c *Conn) published(uri string) []Diagnostic {
c.dmu.Lock()
defer c.dmu.Unlock()
if d, ok := c.diag[uri]; ok {
return append([]Diagnostic(nil), d...)
}
return nil
}
// Close shuts the server down politely, then unconditionally. Idempotent.
//
// Polite first because a server asked to exit flushes and releases its own
// children. But the polite phase can BLOCK: a server that has stopped reading its
// stdin — crashed, or wedged mid-index — leaves our write with nowhere to go, and
// Close would then never return. It would hold a pool slot, a process handle
// and, at Shutdown, the whole binary.
//
// So the courtesy runs off to the side and this waits on a clock, never on the
// server. Closing the writer afterwards is what releases that goroutine: a
// blocked write to a closed pipe returns rather than waits.
func (c *Conn) Close() {
c.once.Do(func() {
select {
case <-c.done:
// Already dead. There is nobody to say goodbye to, and saying it
// anyway is exactly how this used to hang.
default:
polite := make(chan struct{})
go func() {
defer close(polite)
ctx, cancel := context.WithTimeout(context.Background(), closeWait)
defer cancel()
_, _ = c.Call(ctx, "shutdown", nil)
_ = c.Notify("exit", nil)
}()
select {
case <-polite:
case <-time.After(closeWait):
}
}
_ = c.w.Close()
select {
case <-c.done:
case <-time.After(closeWait):
}
if c.stop != nil {
c.stop()
}
})
}
// send frames one message. The write lock spans header and body: two goroutines
// interleaving there would produce a frame whose length does not match its
// payload, and the stream never recovers from that.
func (c *Conn) send(v any) error {
b, err := json.Marshal(v)
if err != nil {
return fmt.Errorf("lsp: encode: %w", err)
}
c.wmu.Lock()
defer c.wmu.Unlock()
if _, err := fmt.Fprintf(c.w, "Content-Length: %d\r\n\r\n", len(b)); err != nil {
return fmt.Errorf("lsp: write header: %w", err)
}
if _, err := c.w.Write(b); err != nil {
return fmt.Errorf("lsp: write body: %w", err)
}
return nil
}
// read is the sole owner of the inbound stream. Every frame is exactly one of
// four kinds, and each has exactly one destination.
func (c *Conn) read(r *bufio.Reader) {
defer close(c.done)
for {
body, err := readFrame(r)
if err != nil {
c.err = err
return
}
var m msg
if json.Unmarshal(body, &m) != nil {
continue // a frame we cannot parse is not a reason to drop the session
}
switch {
case m.Method != "" && len(m.ID) > 0:
// A server→client REQUEST (workspace/configuration,
// client/registerCapability). It BLOCKS the server until answered,
// so silence here is a hang, not a no-op. A null result is a valid
// answer to every one of them and commits us to nothing.
_ = c.send(map[string]any{"jsonrpc": "2.0", "id": m.ID, "result": nil})
case m.Method != "" && len(m.ID) == 0:
if m.Method == "textDocument/publishDiagnostics" {
c.publish(m.Params)
}
case len(m.ID) > 0:
var id int64
if json.Unmarshal(m.ID, &id) != nil {
continue // a response to an id we never issued
}
c.mu.Lock()
ch := c.wait[id]
c.mu.Unlock()
if ch != nil {
ch <- m // buffered, and the waiter is the only receiver
}
}
}
}
func (c *Conn) publish(params json.RawMessage) {
var p struct {
URI string `json:"uri"`
Diagnostics []Diagnostic `json:"diagnostics"`
}
if json.Unmarshal(params, &p) != nil || p.URI == "" {
return
}
c.dmu.Lock()
c.diag[p.URI] = p.Diagnostics
c.dmu.Unlock()
}
func (c *Conn) stopped() error {
if c.err != nil && !errors.Is(c.err, io.EOF) {
return fmt.Errorf("lsp: server stopped: %w", c.err)
}
return errors.New("lsp: server stopped")
}
// readFrame reads one Content-Length-framed message.
//
// Content-Length is REQUIRED and is the only header that matters; Content-Type is
// accepted and ignored, as the spec allows. A frame is refused rather than
// truncated when it exceeds maxFrame, because a truncated read leaves the stream
// pointing at the middle of a message.
func readFrame(r *bufio.Reader) ([]byte, error) {
n := -1
for {
line, err := r.ReadString('\n')
if err != nil {
return nil, err
}
line = strings.TrimRight(line, "\r\n")
if line == "" {
break // end of headers
}
k, v, ok := strings.Cut(line, ":")
if !ok || !strings.EqualFold(strings.TrimSpace(k), "Content-Length") {
continue
}
if n, err = strconv.Atoi(strings.TrimSpace(v)); err != nil {
return nil, fmt.Errorf("lsp: bad Content-Length %q", v)
}
}
if n < 0 {
return nil, errors.New("lsp: frame without Content-Length")
}
if n > maxFrame {
return nil, fmt.Errorf("lsp: frame of %d bytes exceeds %d", n, maxFrame)
}
body := make([]byte, n)
if _, err := io.ReadFull(r, body); err != nil {
return nil, err
}
return body, nil
}
// env is the environment a language server and its dependency fetch run under.
//
// It is BUILT, never inherited. The cloud process holds gateway credentials, KMS
// addresses and cluster tokens in its own environment, and a language server is a
// third-party binary parsing tenant source — the two must not meet. PATH and HOME
// are what a toolchain needs to find itself and its caches; nothing else is
// passed, and the deployment adds proxy settings through Lang.Env.
func env(l Lang) []string {
base := []string{
"PATH=" + os.Getenv("PATH"),
"HOME=" + os.Getenv("HOME"),
"GIT_TERMINAL_PROMPT=0",
"GIT_CONFIG_GLOBAL=/dev/null",
"GIT_CONFIG_SYSTEM=/dev/null",
}
return append(base, l.Env...)
}
// pathURI renders an absolute path as a file: URI. url.URL does the escaping, so
// a path with a space or a percent survives the round trip.
func pathURI(p string) string {
return (&url.URL{Scheme: "file", Path: p}).String()
}
// uriPath is pathURI's inverse for a file: URI, and the empty string for anything
// else — a server may cite a definition inside a jar: or zipfile: URI, which
// names no path on our disk.
func uriPath(raw string) string {
u, err := url.Parse(raw)
if err != nil || u.Scheme != "file" {
return ""
}
return u.Path
}
-351
View File
@@ -1,351 +0,0 @@
package lsp
// server_test.go drives the real client against a FAKE language server.
//
// The fake speaks the actual wire protocol — Content-Length framing, JSON-RPC 2.0
// — over an in-process pipe, so these tests need no gopls, no npm and no network,
// and still exercise the same code Start hands a real subprocess. newConn is the
// seam that makes that possible; nothing here reimplements the client under test.
import (
"bufio"
"context"
"encoding/json"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
)
// fake is a language server: it records the methods it is asked, in order, and
// answers from a table of canned replies.
type fake struct {
mu sync.Mutex
seen []string
reply map[string]any // method → result
// push, when set, is published as a diagnostics notification after didOpen,
// which is how a real server delivers them: unsolicited, not as a response.
push map[string]any
}
// serve reads frames from r and writes answers to w until r closes.
func (f *fake) serve(r io.Reader, w io.WriteCloser) {
defer w.Close()
br := bufio.NewReader(r)
for {
body, err := readFrame(br)
if err != nil {
return
}
var m struct {
ID json.RawMessage `json:"id"`
Method string `json:"method"`
}
if json.Unmarshal(body, &m) != nil {
continue
}
f.mu.Lock()
f.seen = append(f.seen, m.Method)
f.mu.Unlock()
if m.Method == "exit" {
return
}
if len(m.ID) > 0 { // a request: answer it
result, ok := f.reply[m.Method]
if !ok {
result = map[string]any{}
}
f.write(w, map[string]any{"jsonrpc": "2.0", "id": m.ID, "result": result})
}
if m.Method == "textDocument/didOpen" && f.push != nil {
f.write(w, map[string]any{
"jsonrpc": "2.0",
"method": "textDocument/publishDiagnostics",
"params": f.push,
})
}
}
}
func (f *fake) write(w io.Writer, v any) {
b, _ := json.Marshal(v)
fmt.Fprintf(w, "Content-Length: %d\r\n\r\n", len(b))
w.Write(b)
}
func (f *fake) methods() []string {
f.mu.Lock()
defer f.mu.Unlock()
return append([]string(nil), f.seen...)
}
// dial wires a Conn to a fake over two pipes.
func dial(t *testing.T, f *fake) *Conn {
t.Helper()
toServer, fromClient := io.Pipe()
toClient, fromServer := io.Pipe()
go f.serve(toServer, fromServer)
c := newConn(fromClient, toClient, func() {})
t.Cleanup(c.Close)
return c
}
// sample writes a file into a temp dir and returns the dir and the abs path.
func sample(t *testing.T, name, body string) (string, string) {
t.Helper()
dir := t.TempDir()
abs := filepath.Join(dir, name)
if err := os.WriteFile(abs, []byte(body), 0o600); err != nil {
t.Fatalf("write %s: %v", name, err)
}
return dir, abs
}
// TestDefinitionDrivesTheProtocol is the core claim: given a position, the client
// performs initialize → initialized → didOpen → definition, in that order, and
// parses the Location the server answered with.
func TestDefinitionDrivesTheProtocol(t *testing.T) {
dir, abs := sample(t, "main.go", "package main\n\nfunc main() {}\n")
target := filepath.Join(dir, "other.go")
f := &fake{reply: map[string]any{
"initialize": map[string]any{"capabilities": map[string]any{}},
"textDocument/definition": []any{map[string]any{
"uri": pathURI(target),
"range": map[string]any{
"start": map[string]any{"line": 41, "character": 8},
"end": map[string]any{"line": 41, "character": 16},
},
}},
}}
c := dial(t, f)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
l := table["go"]
if err := c.handshake(ctx, l, dir); err != nil {
t.Fatalf("handshake: %v", err)
}
uri, err := c.Open(l, abs)
if err != nil {
t.Fatalf("didOpen: %v", err)
}
raw, err := c.Call(ctx, "textDocument/definition", map[string]any{
"textDocument": map[string]any{"uri": uri},
"position": map[string]any{"line": 2, "character": 5},
})
if err != nil {
t.Fatalf("definition: %v", err)
}
// The ORDER is the contract: a server asked before initialize completes is
// entitled to refuse, and one asked about a document it was never told about
// answers null.
want := []string{"initialize", "initialized", "textDocument/didOpen", "textDocument/definition"}
got := f.methods()
if len(got) != len(want) {
t.Fatalf("methods = %v, want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Errorf("method %d = %q, want %q", i, got[i], want[i])
}
}
locs := locations(raw, dir)
if len(locs) != 1 {
t.Fatalf("locations = %d, want 1", len(locs))
}
if locs[0].Path != "other.go" {
t.Errorf("path = %q, want %q (a target inside the checkout is repo-relative)", locs[0].Path, "other.go")
}
// Positions pass through untouched — 0-based line, 0-based UTF-16 character.
if locs[0].Range.Start.Line != 41 || locs[0].Range.Start.Character != 8 {
t.Errorf("start = %+v, want {41 8} (LSP positions must not be re-based)", locs[0].Range.Start)
}
}
// TestServerRequestIsAnswered pins the deadlock this client is built to avoid: a
// server→client request (workspace/configuration, client/registerCapability)
// BLOCKS the server until it is answered. A client that only reads responses
// hangs here.
func TestServerRequestIsAnswered(t *testing.T) {
toServer, fromClient := io.Pipe()
toClient, fromServer := io.Pipe()
answered := make(chan struct{})
go func() {
defer fromServer.Close()
br := bufio.NewReader(toServer)
f := &fake{}
// Read the client's initialize, then turn around and ASK it something
// before answering — exactly what rust-analyzer does.
body, err := readFrame(br)
if err != nil {
return
}
var m struct {
ID json.RawMessage `json:"id"`
}
json.Unmarshal(body, &m)
f.write(fromServer, map[string]any{
"jsonrpc": "2.0", "id": 9001, "method": "client/registerCapability",
"params": map[string]any{"registrations": []any{}},
})
if _, err := readFrame(br); err != nil { // the client's reply
return
}
close(answered)
f.write(fromServer, map[string]any{"jsonrpc": "2.0", "id": m.ID, "result": map[string]any{}})
for {
if _, err := readFrame(br); err != nil {
return
}
}
}()
c := newConn(fromClient, toClient, func() {})
t.Cleanup(c.Close)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if err := c.handshake(ctx, table["go"], t.TempDir()); err != nil {
t.Fatalf("handshake: %v", err)
}
select {
case <-answered:
default:
t.Fatal("client never answered the server's request — a real server would still be blocked")
}
}
// TestDiagnosticsAreCollectedFromNotifications: diagnostics are published, not
// returned, so they only arrive if the reader routes notifications.
func TestDiagnosticsAreCollectedFromNotifications(t *testing.T) {
dir, abs := sample(t, "main.go", "package main\n")
uri := pathURI(abs)
f := &fake{
reply: map[string]any{"initialize": map[string]any{}},
push: map[string]any{
"uri": uri,
"diagnostics": []any{map[string]any{
"range": map[string]any{
"start": map[string]any{"line": 0, "character": 0},
"end": map[string]any{"line": 0, "character": 7},
},
"severity": 1,
"source": "compiler",
"message": "undefined: x",
}},
},
}
c := dial(t, f)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
l := table["go"]
if err := c.handshake(ctx, l, dir); err != nil {
t.Fatalf("handshake: %v", err)
}
if _, err := c.Open(l, abs); err != nil {
t.Fatalf("didOpen: %v", err)
}
got := c.Diagnostics(ctx, uri, 100*time.Millisecond)
if len(got) != 1 {
t.Fatalf("diagnostics = %d, want 1", len(got))
}
if got[0].Message != "undefined: x" || got[0].Severity != 1 {
t.Errorf("diagnostic = %+v, want severity 1 'undefined: x'", got[0])
}
}
// TestCallFailsWhenTheServerDies proves the client fails CLOSED: a dead server is
// an error, never a hang until the request's own deadline and never a silently
// empty answer that reads as "no definition found".
func TestCallFailsWhenTheServerDies(t *testing.T) {
toServer, fromClient := io.Pipe()
toClient, fromServer := io.Pipe()
go func() {
br := bufio.NewReader(toServer)
readFrame(br) // take the request
fromServer.Close() // then die without answering
}()
c := newConn(fromClient, toClient, func() {})
t.Cleanup(c.Close)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if _, err := c.Call(ctx, "textDocument/hover", map[string]any{}); err == nil {
t.Fatal("Call succeeded against a dead server")
} else if ctx.Err() != nil {
t.Fatalf("Call waited for its own deadline instead of noticing the server: %v", err)
}
}
// TestCloseDoesNotHangOnAWedgedServer is the regression this suite found: Close
// used to write a polite `shutdown` unconditionally, and a server that had
// stopped reading its stdin left that write with nowhere to go — Close never
// returned, holding a pool slot and, at Shutdown, the whole binary.
func TestCloseDoesNotHangOnAWedgedServer(t *testing.T) {
toServer, fromClient := io.Pipe()
toClient, fromServer := io.Pipe()
// A server that reads NOTHING and answers nothing. Writes to it block.
_ = toServer
c := newConn(fromClient, toClient, func() {})
done := make(chan struct{})
go func() { defer close(done); c.Close() }()
select {
case <-done:
case <-time.After(30 * time.Second):
t.Fatal("Close hung on a server that stopped reading its stdin")
}
fromServer.Close()
}
// TestFrameRejectsAnOversizeLength: Content-Length is attacker-influenced (a
// server parsing tenant source emits it), so an absurd one must be refused rather
// than allocated.
func TestFrameRejectsAnOversizeLength(t *testing.T) {
r := bufio.NewReader(strings.NewReader("Content-Length: 999999999999\r\n\r\n"))
if _, err := readFrame(r); err == nil {
t.Fatal("readFrame accepted a length past maxFrame")
}
}
// TestFrameRejectsAMissingLength: a frame with no Content-Length has no boundary,
// so continuing to read desynchronizes the stream.
func TestFrameRejectsAMissingLength(t *testing.T) {
r := bufio.NewReader(strings.NewReader("Content-Type: application/vscode-jsonrpc\r\n\r\n"))
if _, err := readFrame(r); err == nil {
t.Fatal("readFrame accepted a frame with no Content-Length")
}
}
// TestURIRoundTrip: paths with characters that must be escaped survive both
// directions, so a definition in a directory with a space is still findable.
func TestURIRoundTrip(t *testing.T) {
for _, p := range []string{"/tmp/x/main.go", "/tmp/a b/c#d/main.go", "/tmp/100%/x.go"} {
if got := uriPath(pathURI(p)); got != p {
t.Errorf("round trip %q → %q", p, got)
}
}
if got := uriPath("jar:file:///x.jar!/A.class"); got != "" {
t.Errorf("non-file URI resolved to a path %q — it names nothing on our disk", got)
}
}
-457
View File
@@ -1,457 +0,0 @@
package lsp
// workspace.go owns the working tree: check a repo out at a revision, fetch its
// dependencies under the scripts-off policy, and keep the resulting live server
// warm in a bounded pool.
//
// # On not reusing apps/code's checkout
//
// There is nothing to reuse. apps/code holds a static INDEX, not a tree — its
// door is POST /v1/code/index {repo,files:[{path,content}]}, files PUSHED to it
// by a client, and its own doc comment says so ("get_repo_structure over the
// org's own index, with no git checkout involved"). It never clones anything.
//
// The repo does have ONE git working-tree checkout already: apps/deploy's
// unexported gitSource.render — shallow clone, rev-parse, hardened env. It is not
// importable: apps/deploy pulls in the Kubernetes machinery (unstructured, the
// sync engine) that would then be linked into the lsp binary, and each app here
// is its own binary precisely so that does not happen.
//
// So this is the second working-tree checkout in the tree, and that is a real
// duplicate, not a resolved one. It follows deploy's invocation and its hardened
// environment exactly so the two cannot drift in BEHAVIOUR while they wait for
// the fix, which is to hoist the primitive into the root cloud package and have
// both call it. That is a change to a live deployment path and is not made here.
import (
"context"
"fmt"
"os"
"os/exec"
"path/filepath"
"regexp"
"slices"
"strings"
"sync"
"time"
"github.com/hanzoai/cloud"
)
// fetchable is the ENTIRE scripts-off policy: one predicate, one place, read by
// the one caller that runs a dependency fetch.
//
// A fetch that runs dependency-authored code (langs.go: Executes) is remote code
// execution triggered by whatever the caller asked us to check out, and it buys
// nothing a language server needs — definitions and references come from source.
// So it does not run, and no configuration turns it on: the phase-2 sandbox is
// what would earn that, and until it exists an env var here would only be a way
// to lose the argument at 3am.
//
// THE DEPLOYED WORKER MUST STILL BE SANDBOXED. Even scripts-off, this process
// runs third-party language servers over untrusted source. The cloud-lsp worker's
// pod (phase 2) must additionally run non-root with a read-only root filesystem
// and all capabilities dropped, restrict egress to the module mirrors alone, cap
// CPU/memory/PIDs by cgroup, and mount no KMS material, no cloud metadata
// endpoint and no other tenant's volume. Scripts-off narrows the blast radius; it
// is not the boundary.
func fetchable(l Lang) bool { return len(l.Fetch) > 0 && !l.Executes }
// Bounds. A workspace is a checkout plus a language server process, so the cap is
// on memory and file descriptors, not on rows.
const (
warmMax = 8 // live workspaces held at once
warmTTL = 20 * time.Minute // idle before eviction
fetchMax = 10 * time.Minute // dependency fetch budget
cloneMax = 5 * time.Minute // checkout budget
)
// key names one workspace. org is FIRST and is always the validated principal's
// org: two tenants naming the same repo at the same revision get two keys, two
// directories and two servers, so a warm workspace can never be handed across the
// tenant boundary.
type key struct{ org, repo, rev string }
// Tree is one checked-out revision with its language server attached.
//
// ready is closed when it is usable. It is inserted into the pool BEFORE the slow
// work starts, so a second request for the same key waits for the first checkout
// instead of starting a competing one into the same directory.
type Tree struct {
key key
dir string
lang Lang
conn *Conn
used time.Time
ready chan struct{}
err error
}
func (t *Tree) close() {
if t == nil {
return
}
if t.conn != nil {
t.conn.Close()
}
if t.dir != "" {
_ = os.RemoveAll(t.dir)
}
}
// pool is the bounded set of warm workspaces, LRU by last use with an idle TTL.
//
// open and now are FIELDS rather than calls so the eviction policy can be tested
// as the arithmetic it is, with no git, no toolchain and no wall clock.
//
// The path is an argument to open rather than part of key on purpose: a workspace
// is a CHECKOUT, and which file you are asking about is not part of which
// checkout you want. Building one still needs it — the path picks the language
// and roots the server — so it is passed, not keyed.
type pool struct {
mu sync.Mutex
warm map[key]*Tree
max int
ttl time.Duration
open func(ctx context.Context, k key, path string) (*Tree, error)
now func() time.Time
}
func newPool(open func(ctx context.Context, k key, path string) (*Tree, error)) *pool {
return &pool{
warm: make(map[key]*Tree),
max: warmMax,
ttl: warmTTL,
open: open,
now: time.Now,
}
}
// get returns the workspace for k, building it if it is not warm. The bool
// reports a COLD start — the checkout, the fetch and the server handshake — which
// is the event meter.go charges for.
func (p *pool) get(ctx context.Context, k key, path string) (*Tree, bool, error) {
p.mu.Lock()
p.sweep()
if t, ok := p.warm[k]; ok {
t.used = p.now()
p.mu.Unlock()
select {
case <-t.ready:
case <-ctx.Done():
return nil, false, ctx.Err()
}
if t.err != nil {
return nil, false, t.err
}
return t, false, nil
}
t := &Tree{key: k, used: p.now(), ready: make(chan struct{})}
p.warm[k] = t
p.evict()
p.mu.Unlock()
built, err := p.open(ctx, k, path)
if err != nil {
t.err = err
close(t.ready)
p.mu.Lock()
if p.warm[k] == t {
delete(p.warm, k)
}
p.mu.Unlock()
return nil, true, err
}
t.dir, t.lang, t.conn = built.dir, built.lang, built.conn
close(t.ready)
return t, true, nil
}
// sweep drops workspaces idle past the TTL. Caller holds mu.
func (p *pool) sweep() {
cut := p.now().Add(-p.ttl)
for k, t := range p.warm {
if t.settled() && t.used.Before(cut) {
delete(p.warm, k)
go t.close()
}
}
}
// evict enforces max by dropping least-recently-used entries. Caller holds mu.
//
// An entry still being built is never chosen: it has no server to close and a
// request is already waiting on it. That does mean a burst of cold starts can
// briefly exceed max, which is the right trade — the alternative is evicting the
// checkout somebody is blocked on.
func (p *pool) evict() {
for len(p.warm) > p.max {
var oldest key
var found bool
for k, t := range p.warm {
if !t.settled() {
continue
}
if !found || t.used.Before(p.warm[oldest].used) {
oldest, found = k, true
}
}
if !found {
return
}
t := p.warm[oldest]
delete(p.warm, oldest)
go t.close()
}
}
func (t *Tree) settled() bool {
select {
case <-t.ready:
return true
default:
return false
}
}
// drop discards every warm revision of one repo for one org.
//
// A push or a re-index makes a checkout stale. Mostly that resolves itself — keys
// are revision-pinned, so new content is a new key — but a caller that tracks a
// BRANCH keeps asking for the same key while the branch moves, and this is what
// releases it.
//
// Cross-process invalidation is phase 2 and is stated here rather than implied:
// code and lsp are separate binaries, so apps/code cannot call this in-process.
// The push signal has to arrive over the bus.
func (p *pool) drop(org, repo string) {
p.mu.Lock()
defer p.mu.Unlock()
for k, t := range p.warm {
if k.org == org && k.repo == repo && t.settled() {
delete(p.warm, k)
go t.close()
}
}
}
func (p *pool) closeAll() {
p.mu.Lock()
warm := p.warm
p.warm = make(map[key]*Tree)
p.mu.Unlock()
for _, t := range warm {
if t.settled() {
t.close()
}
}
}
// ── building one workspace ───────────────────────────────────────────────────
// build is the cold path: check out, fetch dependencies, start the server.
func (s *state) build(ctx context.Context, k key, path string) (*Tree, error) {
l, ok := langFor(path)
if !ok {
return nil, fmt.Errorf("no language server for %q", filepath.Ext(path))
}
dir, err := s.dirFor(k)
if err != nil {
return nil, err
}
if err := checkout(ctx, dir, k, s.forge); err != nil {
_ = os.RemoveAll(dir)
return nil, err
}
root := rootFor(dir, path, l)
fetch(ctx, l, root)
conn, err := Start(ctx, l, root)
if err != nil {
_ = os.RemoveAll(dir)
return nil, err
}
return &Tree{key: k, dir: dir, lang: l, conn: conn}, nil
}
// dirFor is where a workspace lives: inside the ORG's own partition, under the
// same {DataDir}/orgs/{slug} convention every per-org store in this binary uses
// (orgdb.go). The org segment is rendered by cloud.OrgNamespace from the
// validated principal — the one door that turns a principal into a name — so the
// isolation is a property of the path, not of a check somebody has to remember.
func (s *state) dirFor(k key) (string, error) {
ns, err := cloud.OrgNamespace(k.org, "")
if err != nil {
return "", err
}
rev := k.rev
if rev == "" {
rev = "_default"
}
dir := filepath.Join(s.DataDir, "orgs", ns.ID(), "lsp", k.repo, rev)
if err := os.MkdirAll(dir, 0o700); err != nil {
return "", fmt.Errorf("workspace dir: %w", err)
}
return dir, nil
}
// checkout materializes repo@rev into dir.
//
// init + fetch, not clone, because it is ONE path for both a branch name and a
// bare sha — `clone --branch` cannot take a sha, and having two checkout paths
// would mean the answer depends on which kind of revision you named.
//
// The URL is BUILT, never accepted: base is the deployment's forge and the owner
// segment is k.org, the validated principal's org. A caller supplies only the
// repo slug, already narrowed to [a-zA-Z0-9._-] by [slug]. There is therefore no
// input from which a request could name another tenant's repo, an internal
// address, or a host of its own choosing — the class of bug is absent rather than
// defended against.
func checkout(ctx context.Context, dir string, k key, base string) error {
ctx, cancel := context.WithTimeout(ctx, cloneMax)
defer cancel()
url := strings.TrimSuffix(base, "/") + "/" + k.org + "/" + k.repo + ".git"
rev := k.rev
if rev == "" {
rev = "HEAD"
}
steps := [][]string{
{"init", "-q"},
{"remote", "add", "origin", url},
{"fetch", "--depth", "1", "--no-tags", "origin", rev},
{"checkout", "-q", "--detach", "FETCH_HEAD"},
}
for _, args := range steps {
if err := git(ctx, dir, args...); err != nil {
return err
}
}
return nil
}
// git runs one git subprocess in dir under the hardened environment apps/deploy
// and apps/git both use: no terminal prompt, no system or global config (so no
// inherited credential helper, insteadOf rewrite or proxy), protocols restricted
// to http/https, and redirects refused so a moved ref cannot bounce the fetch to
// another host. Arguments are a SLICE, never a shell string.
func git(ctx context.Context, dir string, args ...string) error {
cmd := exec.CommandContext(ctx, "git", append([]string{"-C", dir}, args...)...)
cmd.Env = append([]string{
"GIT_TERMINAL_PROMPT=0",
"GIT_CONFIG_GLOBAL=/dev/null",
"GIT_CONFIG_SYSTEM=/dev/null",
"GIT_ALLOW_PROTOCOL=http:https",
"HOME=" + os.TempDir(),
"PATH=" + os.Getenv("PATH"),
}, gitConfig("http.followRedirects=false")...)
out, err := cmd.CombinedOutput()
if err != nil {
// git's stderr can echo a URL; it never carries our credential, which
// rides env-injected config and not argv. Bounded so a chatty failure
// cannot balloon an error string.
return fmt.Errorf("git %s: %w: %s", args[0], err, trim(string(out), 512))
}
return nil
}
// gitConfig renders git config as env (GIT_CONFIG_COUNT/KEY/VALUE), the form
// apps/git/mirror.go uses — config that never appears on argv or in a log.
func gitConfig(kv ...string) []string {
env := []string{fmt.Sprintf("GIT_CONFIG_COUNT=%d", len(kv))}
for i, pair := range kv {
k, v, _ := strings.Cut(pair, "=")
env = append(env,
fmt.Sprintf("GIT_CONFIG_KEY_%d=%s", i, k),
fmt.Sprintf("GIT_CONFIG_VALUE_%d=%s", i, v))
}
return env
}
// fetch populates the dependency tree when the policy allows it.
//
// Failure is NOT fatal and is not reported to the caller: a language server still
// answers about the checkout's own source with no dependencies resolved, and a
// private module the worker cannot reach is a degraded answer, not a 500.
func fetch(ctx context.Context, l Lang, root string) {
if !fetchable(l) {
return
}
ctx, cancel := context.WithTimeout(ctx, fetchMax)
defer cancel()
cmd := exec.CommandContext(ctx, l.Fetch[0], l.Fetch[1:]...)
cmd.Dir = root
cmd.Env = env(l)
_ = cmd.Run()
}
// ── input narrowing ──────────────────────────────────────────────────────────
// slug is a repository name: the shape apps/git gives a repo under an owner. No
// slash, so it cannot name another owner's repository; no leading dot, so it
// cannot climb out of the org's data directory.
var slug = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9._-]{0,99}$`)
// revision is a branch, tag or sha. The leading character is constrained to
// alphanumeric, which is what stops a revision from being read by git as a FLAG —
// `--upload-pack=…` in the rev position is command execution on the fetch.
var revision = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9._/-]{0,199}$`)
// clean narrows a repo-relative path to a location proven to be inside dir.
//
// The check is done on the RESOLVED path, and that is the part that matters.
// Rejecting ".." is not sufficient: a checkout is TENANT-CONTROLLED content, and a
// repo may contain a symlink named `src` pointing at /etc, at the KMS mount, or at
// another org's directory one level up. Only resolving symlinks and then proving
// containment refuses that, so that is what happens — lexical checks first (they
// refuse the cheap attacks without a syscall), then EvalSymlinks, then a
// containment proof against the resolved root.
func clean(dir, path string) (string, error) {
if path == "" {
return "", fmt.Errorf("path is required")
}
if filepath.IsAbs(path) {
return "", fmt.Errorf("path must be repo-relative")
}
rel := filepath.Clean(filepath.FromSlash(path))
if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return "", fmt.Errorf("path escapes the repository")
}
root, err := filepath.EvalSymlinks(dir)
if err != nil {
return "", fmt.Errorf("resolve workspace: %w", err)
}
abs, err := filepath.EvalSymlinks(filepath.Join(root, rel))
if err != nil {
return "", fmt.Errorf("no such file in the repository")
}
if abs != root && !strings.HasPrefix(abs, root+string(filepath.Separator)) {
return "", fmt.Errorf("path escapes the repository")
}
if info, err := os.Stat(abs); err != nil || info.IsDir() {
return "", fmt.Errorf("no such file in the repository")
}
return abs, nil
}
func trim(s string, n int) string {
s = strings.TrimSpace(s)
if len(s) > n {
return s[:n] + "…"
}
return s
}
// methods is every LSP request this door forwards. A CLOSED set: the door names
// what it serves, so a method the table does not carry is a 400 here rather than
// an arbitrary string handed to a language server.
var methods = []string{
"hover", "definition", "references", "typeDefinition",
"implementation", "documentSymbol", "completion", "diagnostics",
}
func known(m string) bool { return slices.Contains(methods, m) }
-452
View File
@@ -1,452 +0,0 @@
package lsp
// workspace_test.go tests the pool as the arithmetic it is — an injected opener
// and an injected clock, no git, no toolchain, no wall clock — and the input
// narrowing that keeps one tenant's query inside that tenant's checkout.
import (
"context"
"errors"
"os"
"path/filepath"
"sync"
"testing"
"time"
)
// clock is a hand-advanced time source, so a TTL test states the elapsed time it
// means instead of sleeping for it.
type clock struct {
mu sync.Mutex
t time.Time
}
func (c *clock) now() time.Time {
c.mu.Lock()
defer c.mu.Unlock()
return c.t
}
func (c *clock) advance(d time.Duration) {
c.mu.Lock()
c.t = c.t.Add(d)
c.mu.Unlock()
}
// testPool builds a pool whose opener makes a Tree with a real directory (so
// close() has something to remove) and no server process.
func testPool(t *testing.T, max int, ttl time.Duration) (*pool, *clock, func() int) {
t.Helper()
clk := &clock{t: time.Unix(1<<30, 0)}
root := t.TempDir()
var mu sync.Mutex
var opened int
p := newPool(func(_ context.Context, k key, _ string) (*Tree, error) {
mu.Lock()
opened++
n := opened
mu.Unlock()
dir := filepath.Join(root, k.org, k.repo, k.rev, string(rune('a'+n%26)))
if err := os.MkdirAll(dir, 0o700); err != nil {
return nil, err
}
return &Tree{key: k, dir: dir, lang: table["go"]}, nil
})
p.max, p.ttl, p.now = max, ttl, clk.now
t.Cleanup(p.closeAll)
return p, clk, func() int {
mu.Lock()
defer mu.Unlock()
return opened
}
}
func get(t *testing.T, p *pool, k key) (*Tree, bool) {
t.Helper()
tree, cold, err := p.get(context.Background(), k, "main.go")
if err != nil {
t.Fatalf("get %v: %v", k, err)
}
return tree, cold
}
// TestWarmWorkspaceIsReused: the second query for the same (org, repo, rev) must
// not check out again. This is the whole reason the pool exists, and it is also
// what the billing model rests on — only a cold start is charged.
func TestWarmWorkspaceIsReused(t *testing.T) {
p, _, opened := testPool(t, 4, time.Hour)
k := key{org: "acme", repo: "cloud", rev: "main"}
if _, cold := get(t, p, k); !cold {
t.Fatal("first get was not reported as a cold start")
}
if _, cold := get(t, p, k); cold {
t.Fatal("second get reported a cold start — the workspace was not reused")
}
if opened() != 1 {
t.Fatalf("opened %d workspaces for one key, want 1", opened())
}
}
// TestIdleWorkspaceIsEvictedAtTTL: a workspace holds a subprocess and a checkout,
// so an idle one must not hold them forever.
func TestIdleWorkspaceIsEvictedAtTTL(t *testing.T) {
p, clk, opened := testPool(t, 4, 10*time.Minute)
k := key{org: "acme", repo: "cloud", rev: "main"}
get(t, p, k)
clk.advance(11 * time.Minute)
if _, cold := get(t, p, k); !cold {
t.Fatal("workspace survived its idle TTL")
}
if opened() != 2 {
t.Fatalf("opened %d, want 2 (evicted then rebuilt)", opened())
}
}
// TestUseKeepsAWorkspaceWarm: the TTL is on IDLE time, so a workspace queried
// steadily must never be evicted out from under its caller.
func TestUseKeepsAWorkspaceWarm(t *testing.T) {
p, clk, opened := testPool(t, 4, 10*time.Minute)
k := key{org: "acme", repo: "cloud", rev: "main"}
get(t, p, k)
for range 5 {
clk.advance(6 * time.Minute) // past half the TTL, never past all of it
if _, cold := get(t, p, k); cold {
t.Fatal("an actively used workspace was evicted")
}
}
if opened() != 1 {
t.Fatalf("opened %d, want 1", opened())
}
}
// TestPoolEvictsLeastRecentlyUsed: over the cap, the workspace nobody has touched
// goes first — not an arbitrary one.
func TestPoolEvictsLeastRecentlyUsed(t *testing.T) {
p, clk, _ := testPool(t, 2, time.Hour)
a := key{org: "acme", repo: "a", rev: "main"}
b := key{org: "acme", repo: "b", rev: "main"}
c := key{org: "acme", repo: "c", rev: "main"}
get(t, p, a)
clk.advance(time.Minute)
get(t, p, b)
clk.advance(time.Minute)
get(t, p, a) // a is now the most recent, b the least
clk.advance(time.Minute)
get(t, p, c) // over the cap: b must go
p.mu.Lock()
defer p.mu.Unlock()
if len(p.warm) != 2 {
t.Fatalf("pool holds %d workspaces, want max 2", len(p.warm))
}
if _, ok := p.warm[b]; ok {
t.Error("evicted something other than the least-recently-used workspace")
}
if _, ok := p.warm[a]; !ok {
t.Error("evicted the most-recently-used workspace")
}
}
// TestOrgsDoNotShareAWorkspace is the isolation claim at the pool layer: the same
// repo name and revision in two orgs are two keys, two directories and two
// servers. Nothing about a warm workspace can be reached across the tenant
// boundary because there is no shared entry to reach.
func TestOrgsDoNotShareAWorkspace(t *testing.T) {
p, _, opened := testPool(t, 4, time.Hour)
one, _ := get(t, p, key{org: "acme", repo: "cloud", rev: "main"})
two, cold := get(t, p, key{org: "other", repo: "cloud", rev: "main"})
if !cold {
t.Fatal("a second org was served the first org's warm workspace")
}
if one.dir == two.dir {
t.Fatalf("two orgs share a checkout directory %q", one.dir)
}
if opened() != 2 {
t.Fatalf("opened %d, want 2", opened())
}
}
// TestConcurrentGetsOpenOnce: several requests racing for the same cold workspace
// must produce ONE checkout, not N competing clones into one directory.
func TestConcurrentGetsOpenOnce(t *testing.T) {
p, _, opened := testPool(t, 4, time.Hour)
k := key{org: "acme", repo: "cloud", rev: "main"}
var wg sync.WaitGroup
for range 16 {
wg.Add(1)
go func() {
defer wg.Done()
p.get(context.Background(), k, "main.go")
}()
}
wg.Wait()
if opened() != 1 {
t.Fatalf("opened %d workspaces concurrently for one key, want 1", opened())
}
}
// TestDropInvalidatesOneRepo: a push makes a branch checkout stale, and drop is
// what releases it — without touching another repo or another org.
func TestDropInvalidatesOneRepo(t *testing.T) {
p, _, _ := testPool(t, 4, time.Hour)
stale := key{org: "acme", repo: "cloud", rev: "main"}
other := key{org: "acme", repo: "other", rev: "main"}
foreign := key{org: "rival", repo: "cloud", rev: "main"}
get(t, p, stale)
get(t, p, other)
get(t, p, foreign)
p.drop("acme", "cloud")
p.mu.Lock()
defer p.mu.Unlock()
if _, ok := p.warm[stale]; ok {
t.Error("drop left the invalidated workspace warm")
}
if _, ok := p.warm[other]; !ok {
t.Error("drop evicted a different repo")
}
if _, ok := p.warm[foreign]; !ok {
t.Error("drop reached across the org boundary")
}
}
// TestFailedOpenIsNotCached: a checkout that failed must not be remembered as a
// workspace, or one transient forge outage poisons the key until the TTL.
func TestFailedOpenIsNotCached(t *testing.T) {
p := newPool(func(context.Context, key, string) (*Tree, error) {
return nil, errors.New("forge unreachable")
})
t.Cleanup(p.closeAll)
k := key{org: "acme", repo: "cloud", rev: "main"}
if _, _, err := p.get(context.Background(), k, "main.go"); err == nil {
t.Fatal("get succeeded despite a failing opener")
}
p.mu.Lock()
defer p.mu.Unlock()
if len(p.warm) != 0 {
t.Fatalf("a failed open left %d entries in the pool", len(p.warm))
}
}
// ── input narrowing ──────────────────────────────────────────────────────────
// TestCleanRefusesEscapes is the containment proof, and the symlink cases are the
// ones that matter: a checkout is TENANT-CONTROLLED content, so a repo can contain
// a symlink pointing anywhere. A lexical ".." check alone would pass every one of
// these.
func TestCleanRefusesEscapes(t *testing.T) {
dir := t.TempDir()
elsewhere := t.TempDir()
outside := filepath.Join(elsewhere, "secret.txt")
if err := os.WriteFile(outside, []byte("another tenant's data"), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "main.go"), []byte("package main\n"), 0o600); err != nil {
t.Fatal(err)
}
// A file symlink out of the tree, and a directory symlink out of the tree.
if err := os.Symlink(outside, filepath.Join(dir, "escape.go")); err != nil {
t.Skipf("symlinks unavailable: %v", err)
}
if err := os.Symlink(elsewhere, filepath.Join(dir, "away")); err != nil {
t.Fatal(err)
}
for _, path := range []string{
"../../etc/passwd", // lexical traversal
"/etc/passwd", // absolute
"escape.go", // symlink to a file outside
"away/secret.txt", // through a symlinked directory
"", // empty
"nope.go", // absent
".", // the directory itself
"../" + "elsewhere", // traversal by name
} {
if got, err := clean(dir, path); err == nil {
t.Errorf("clean(%q) allowed %q — it must stay inside the checkout", path, got)
}
}
// The honest case still works.
got, err := clean(dir, "main.go")
if err != nil {
t.Fatalf("clean rejected a legitimate path: %v", err)
}
if filepath.Base(got) != "main.go" {
t.Errorf("clean returned %q", got)
}
}
// TestRevisionRefusesAFlag: a revision is passed to `git fetch` in the ref
// position, so one beginning with "-" would be read as an OPTION.
// --upload-pack=… there is command execution.
func TestRevisionRefusesAFlag(t *testing.T) {
for _, bad := range []string{
"--upload-pack=/bin/sh",
"-x",
"main;rm -rf /",
"main branch",
"$(whoami)",
"--",
} {
if revision.MatchString(bad) {
t.Errorf("revision accepted %q", bad)
}
}
for _, ok := range []string{"main", "v1.2.3", "release/2026-01", "a1b2c3d4e5f6"} {
if !revision.MatchString(ok) {
t.Errorf("revision rejected the legitimate %q", ok)
}
}
}
// TestSlugRefusesAPath: the repo slug becomes a URL segment and a directory name.
// A slug carrying a slash could name another org's repository on the forge; one
// carrying ".." could climb out of the org's data directory.
func TestSlugRefusesAPath(t *testing.T) {
for _, bad := range []string{
"rival/private", "../../etc", "..", ".", "a b",
"https://evil.example/x", "-x", "", "repo.git/../../other",
} {
if slug.MatchString(bad) {
t.Errorf("slug accepted %q", bad)
}
}
for _, ok := range []string{"cloud", "hanzo-node", "go.mod-tools", "a"} {
if !slug.MatchString(ok) {
t.Errorf("slug rejected the legitimate %q", ok)
}
}
}
// TestScriptsAreOff is the policy, asserted rather than described: no language
// whose dependency fetch executes dependency-authored code may run that fetch.
func TestScriptsAreOff(t *testing.T) {
for name, l := range table {
if l.Executes && fetchable(l) {
t.Errorf("%s: a fetch that runs dependency code is enabled (%v)", name, l.Fetch)
}
}
if !fetchable(table["typescript"]) {
t.Error("the npm fetch is disabled, but --ignore-scripts makes it safe and it is needed for .d.ts")
}
if fetchable(table["python"]) {
t.Error("the python fetch builds sdists, which runs setup.py as us")
}
// npm must never be invoked without --ignore-scripts.
npm := table["typescript"].Fetch
var guarded bool
for _, a := range npm {
if a == "--ignore-scripts" {
guarded = true
}
}
if !guarded {
t.Errorf("npm fetch %v is missing --ignore-scripts", npm)
}
// rust-analyzer must be told not to run build scripts or expand proc macros:
// the server does at load time exactly what `cargo fetch` was chosen to avoid.
rust := table["rust"].Init
cargo, _ := rust["cargo"].(map[string]any)
scripts, _ := cargo["buildScripts"].(map[string]any)
if scripts["enable"] != false {
t.Error("rust-analyzer may run build.rs — cargo.buildScripts.enable is not false")
}
proc, _ := rust["procMacro"].(map[string]any)
if proc["enable"] != false {
t.Error("rust-analyzer may expand proc macros — procMacro.enable is not false")
}
}
// TestLangForIsDeterministic: map iteration is randomized, so a table walked
// directly would resolve a file to different servers on different requests.
func TestLangForIsDeterministic(t *testing.T) {
for range 50 {
l, ok := langFor("apps/lsp/server.go")
if !ok || l.Name != "go" {
t.Fatalf("langFor(.go) = %q, %v", l.Name, ok)
}
}
if _, ok := langFor("README"); ok {
t.Error("langFor matched a file with no extension")
}
if _, ok := langFor("notes.txt"); ok {
t.Error("langFor matched an unknown extension")
}
}
// TestRootForStopsAtTheCheckout: the deepest marker wins, and the walk may never
// climb above the checkout — a marker outside the tenant's tree must never root a
// server.
func TestRootForStopsAtTheCheckout(t *testing.T) {
dir := t.TempDir()
nested := filepath.Join(dir, "svc", "api")
if err := os.MkdirAll(nested, 0o700); err != nil {
t.Fatal(err)
}
for _, at := range []string{dir, filepath.Join(dir, "svc")} {
if err := os.WriteFile(filepath.Join(at, "go.mod"), []byte("module x\n"), 0o600); err != nil {
t.Fatal(err)
}
}
got := rootFor(dir, "svc/api/main.go", table["go"])
if want := filepath.Join(dir, "svc"); got != want {
t.Errorf("rootFor = %q, want the deepest marker %q", got, want)
}
if got := rootFor(dir, "main.go", table["go"]); got != dir {
t.Errorf("rootFor = %q, want the checkout root %q", got, dir)
}
// No marker anywhere: the checkout root, never a parent.
bare := t.TempDir()
if got := rootFor(bare, "a/b/c.go", table["go"]); got != bare {
t.Errorf("rootFor = %q, want %q — the walk must not climb out", got, bare)
}
}
// TestMethodsAreAClosedSet: the door forwards a fixed list, so an arbitrary
// string never reaches a language server.
func TestMethodsAreAClosedSet(t *testing.T) {
for _, m := range methods {
if !known(m) {
t.Errorf("known(%q) = false for a listed method", m)
}
}
for _, m := range []string{"", "workspace/executeCommand", "shutdown", "exit", "HOVER"} {
if known(m) {
t.Errorf("known(%q) = true for an unlisted method", m)
}
}
}
// TestLanguageIDNamesTheDialect: TypeScript's server gets the wrong answer for a
// React file told it is plain TypeScript.
func TestLanguageIDNamesTheDialect(t *testing.T) {
ts := table["typescript"]
for path, want := range map[string]string{
"a.ts": "typescript", "a.tsx": "typescriptreact",
"a.js": "javascript", "a.jsx": "javascriptreact",
} {
if got := ts.ID(path); got != want {
t.Errorf("ID(%q) = %q, want %q", path, got, want)
}
}
if got := table["go"].ID("main.go"); got != "go" {
t.Errorf("ID(main.go) = %q, want go", got)
}
}
+59 -7
View File
@@ -9,17 +9,69 @@ import (
)
func init() {
zip.Describe("POST /v1/lsp", zip.Doc{
Description: "Resolves one position in one repository through a live language server:\ndefinition, references, type, implementation, hover, document symbols,\ncompletion or diagnostics — over the repo AND its resolved dependencies, with\nno toolchain on the caller's machine.\n\nPositions are the LSP's: line and character are 0-BASED and character counts\nUTF-16 code units, so an editor's 1-based line must have 1 subtracted before it\nis sent. The repository is named by slug and is always one in the caller's own\norg. rev pins a branch, tag or commit sha; empty means the default branch.\n\nThe first query against a (repo, rev) pays a cold start — checkout, dependency\nfetch and the server's first index — and is the billed event; later queries\nagainst the same revision are served from the warm workspace and are free. The\nanswer says which it was.",
zip.Describe("POST /v1/code/lsp/complete", zip.Doc{
Description: "Offers the candidates a language server has at a position, typed and\nresolved through the repository's dependencies rather than guessed from text.",
Fields: map[string]string{
"Answer.cold": "Cold reports that this request paid for a workspace cold start — the\ncheckout, the dependency fetch and the server's first index. It is the\nbilled event, surfaced so a caller can see what it was charged for.",
"Answer.cold": "Cold reports that this request paid to PREPARE the revision — the tree\nwrite, the dependency fetch and the language server's first index. It is\nthe billed event, surfaced so a caller can see what it was charged for.",
"Query.character": "Character is a 0-based UTF-16 code-unit offset within Line, per the LSP\nspecification — not a byte offset and not a rune index.",
"Query.line": "Line is 0-based, per the LSP specification.",
"Query.method": "Method is the question: hover, definition, references, typeDefinition,\nimplementation, documentSymbol, completion or diagnostics.",
"Query.path": "Path is the repo-relative file, e.g. \"apps/lsp/server.go\".",
"Query.path": "Path is the repo-relative file, e.g. \"apps/lsp/lsp.go\".",
"Query.relation": "Relation refines locate: definition, reference, type or implementation.\nEmpty means definition. Every other op ignores it.",
"Query.repo": "Repo is the repository NAME within the caller's own org, e.g. \"cloud\".\nNot a URL and not an owner/name pair: the owner is the validated\nprincipal's org, so this names a repository the caller already owns.",
"Query.rev": "Rev is a branch, tag or commit sha. Empty means the default branch. A\nworkspace is keyed by revision, so pinning a sha is what makes an answer\nreproducible.",
"Query.rev": "Rev is a branch, tag or commit sha. Empty means the default branch. It is\nresolved to a commit before anything else happens, so an answer is always\nabout one immutable tree.",
},
Example: json.RawMessage(`{"repo":"cloud","path":"apps/lsp/server.go","line":120,"character":18,"method":"definition"}`),
Example: json.RawMessage(`{"repo":"cloud","path":"apps/lsp/lsp.go","line":120,"character":18}`),
})
zip.Describe("POST /v1/code/lsp/diagnostics", zip.Doc{
Description: "Reports every problem the language server finds in one file —\ncompile errors, type errors and lints, each with its span and its severity (1\nerror, 2 warning, 3 information, 4 hint). The position is ignored.",
Fields: map[string]string{
"Answer.cold": "Cold reports that this request paid to PREPARE the revision — the tree\nwrite, the dependency fetch and the language server's first index. It is\nthe billed event, surfaced so a caller can see what it was charged for.",
"Query.character": "Character is a 0-based UTF-16 code-unit offset within Line, per the LSP\nspecification — not a byte offset and not a rune index.",
"Query.line": "Line is 0-based, per the LSP specification.",
"Query.path": "Path is the repo-relative file, e.g. \"apps/lsp/lsp.go\".",
"Query.relation": "Relation refines locate: definition, reference, type or implementation.\nEmpty means definition. Every other op ignores it.",
"Query.repo": "Repo is the repository NAME within the caller's own org, e.g. \"cloud\".\nNot a URL and not an owner/name pair: the owner is the validated\nprincipal's org, so this names a repository the caller already owns.",
"Query.rev": "Rev is a branch, tag or commit sha. Empty means the default branch. It is\nresolved to a commit before anything else happens, so an answer is always\nabout one immutable tree.",
},
Example: json.RawMessage(`{"repo":"cloud","path":"apps/lsp/lsp.go"}`),
})
zip.Describe("POST /v1/code/lsp/hover", zip.Doc{
Description: "Renders the type and documentation of the symbol at a position, as the\nlanguage server itself renders it.\n\nPositions are the LSP's: line and character are 0-BASED and character counts\nUTF-16 code units, so an editor's 1-based line must have 1 subtracted before it\nis sent. The repository is named by slug and is always one in the caller's own\norg; rev pins a branch, tag or commit sha, and empty means the default branch.",
Fields: map[string]string{
"Answer.cold": "Cold reports that this request paid to PREPARE the revision — the tree\nwrite, the dependency fetch and the language server's first index. It is\nthe billed event, surfaced so a caller can see what it was charged for.",
"Query.character": "Character is a 0-based UTF-16 code-unit offset within Line, per the LSP\nspecification — not a byte offset and not a rune index.",
"Query.line": "Line is 0-based, per the LSP specification.",
"Query.path": "Path is the repo-relative file, e.g. \"apps/lsp/lsp.go\".",
"Query.relation": "Relation refines locate: definition, reference, type or implementation.\nEmpty means definition. Every other op ignores it.",
"Query.repo": "Repo is the repository NAME within the caller's own org, e.g. \"cloud\".\nNot a URL and not an owner/name pair: the owner is the validated\nprincipal's org, so this names a repository the caller already owns.",
"Query.rev": "Rev is a branch, tag or commit sha. Empty means the default branch. It is\nresolved to a commit before anything else happens, so an answer is always\nabout one immutable tree.",
},
Example: json.RawMessage(`{"repo":"cloud","path":"apps/lsp/lsp.go","line":120,"character":18}`),
})
zip.Describe("POST /v1/code/lsp/locate", zip.Doc{
Description: "Finds where a symbol lives: its definition, its references, its type or\nits implementations, chosen by relation (definition, reference, type,\nimplementation — empty means definition).\n\nIt resolves THROUGH dependencies. An answer whose external flag is set left the\nrepository, and its path is then the module coordinate it landed in — which is\nthe question a static index cannot answer and this service exists for.",
Fields: map[string]string{
"Answer.cold": "Cold reports that this request paid to PREPARE the revision — the tree\nwrite, the dependency fetch and the language server's first index. It is\nthe billed event, surfaced so a caller can see what it was charged for.",
"Query.character": "Character is a 0-based UTF-16 code-unit offset within Line, per the LSP\nspecification — not a byte offset and not a rune index.",
"Query.line": "Line is 0-based, per the LSP specification.",
"Query.path": "Path is the repo-relative file, e.g. \"apps/lsp/lsp.go\".",
"Query.relation": "Relation refines locate: definition, reference, type or implementation.\nEmpty means definition. Every other op ignores it.",
"Query.repo": "Repo is the repository NAME within the caller's own org, e.g. \"cloud\".\nNot a URL and not an owner/name pair: the owner is the validated\nprincipal's org, so this names a repository the caller already owns.",
"Query.rev": "Rev is a branch, tag or commit sha. Empty means the default branch. It is\nresolved to a commit before anything else happens, so an answer is always\nabout one immutable tree.",
},
Example: json.RawMessage(`{"repo":"cloud","path":"apps/lsp/lsp.go","line":120,"character":18,"relation":"definition"}`),
})
zip.Describe("POST /v1/code/lsp/symbols", zip.Doc{
Description: "Outlines one file: every declaration in it, with its kind and its span.\nThe position is ignored — the answer is the whole file.",
Fields: map[string]string{
"Answer.cold": "Cold reports that this request paid to PREPARE the revision — the tree\nwrite, the dependency fetch and the language server's first index. It is\nthe billed event, surfaced so a caller can see what it was charged for.",
"Query.character": "Character is a 0-based UTF-16 code-unit offset within Line, per the LSP\nspecification — not a byte offset and not a rune index.",
"Query.line": "Line is 0-based, per the LSP specification.",
"Query.path": "Path is the repo-relative file, e.g. \"apps/lsp/lsp.go\".",
"Query.relation": "Relation refines locate: definition, reference, type or implementation.\nEmpty means definition. Every other op ignores it.",
"Query.repo": "Repo is the repository NAME within the caller's own org, e.g. \"cloud\".\nNot a URL and not an owner/name pair: the owner is the validated\nprincipal's org, so this names a repository the caller already owns.",
"Query.rev": "Rev is a branch, tag or commit sha. Empty means the default branch. It is\nresolved to a commit before anything else happens, so an answer is always\nabout one immutable tree.",
},
Example: json.RawMessage(`{"repo":"cloud","path":"apps/lsp/lsp.go"}`),
})
}
+10 -5
View File
@@ -281,11 +281,16 @@ var Apps = []App{
{Name: "venue", Prefixes: []string{"/v1/cloud"}},
{Name: "captable", Prefixes: []string{"/v1/captable"}},
{Name: "code", Prefixes: []string{"/v1/code"}},
// lsp sits next to code because they are two reads of one checkout: code is
// the static index, lsp the live language server. Adjacency is documentation,
// not routing — /v1/lsp is a deeper static prefix than ai's "/v1", so it wins
// on specificity wherever it registers.
{Name: "lsp", Prefixes: []string{"/v1/lsp"}},
// lsp lives UNDER code, at /v1/code/lsp, because they are two reads of one
// repository: code is the static index, lsp the live language server that
// resolves through dependencies. One home for code intelligence means one
// place to look for it, in the document and in the MCP tool list alike.
//
// The nesting is not a routing hazard, it is how routing works: nested static
// prefixes resolve by SPECIFICITY, so /v1/code/lsp beats code's /v1/code and
// both beat ai's bare "/v1" — the same relation storage's /v1/s3/buckets has
// to provisioning's /v1/s3. Adjacency in this list is documentation.
{Name: "lsp", Prefixes: []string{"/v1/code/lsp"}},
// zt held "/v1/edge/nodes" — a top-level name for something that was never a
// product. Four unrelated things wore "edge": the on-device inference runtime
// (hanzoai/edge, a binary a customer runs on their own machine, so it has no cloud
+170 -61
View File
@@ -229,9 +229,9 @@ components:
properties:
cold:
description: |-
Cold reports that this request paid for a workspace cold start — the
checkout, the dependency fetch and the server's first index. It is the
billed event, surfaced so a caller can see what it was charged for.
Cold reports that this request paid to PREPARE the revision — the tree
write, the dependency fetch and the language server's first index. It is
the billed event, surfaced so a caller can see what it was charged for.
type: boolean
completions:
items:
@@ -249,7 +249,7 @@ components:
items:
$ref: '#/components/schemas/Location'
type: array
method:
op:
type: string
path:
type: string
@@ -3821,6 +3821,8 @@ components:
type: object
Location:
properties:
external:
type: boolean
path:
type: string
range:
@@ -5174,13 +5176,13 @@ components:
line:
description: Line is 0-based, per the LSP specification.
type: integer
method:
description: |-
Method is the question: hover, definition, references, typeDefinition,
implementation, documentSymbol, completion or diagnostics.
type: string
path:
description: Path is the repo-relative file, e.g. "apps/lsp/server.go".
description: Path is the repo-relative file, e.g. "apps/lsp/lsp.go".
type: string
relation:
description: |-
Relation refines locate: definition, reference, type or implementation.
Empty means definition. Every other op ignores it.
type: string
repo:
description: |-
@@ -5190,9 +5192,9 @@ components:
type: string
rev:
description: |-
Rev is a branch, tag or commit sha. Empty means the default branch. A
workspace is keyed by revision, so pinning a sha is what makes an answer
reproducible.
Rev is a branch, tag or commit sha. Empty means the default branch. It is
resolved to a commit before anything else happens, so an answer is always
about one immutable tree.
type: string
type: object
Question:
@@ -56714,6 +56716,161 @@ paths:
tags:
- code
x-app: code
/v1/code/lsp/complete:
post:
description: |-
Offers the candidates a language server has at a position, typed and
resolved through the repository's dependencies rather than guessed from text.
operationId: post_v1_code_lsp_complete
requestBody:
content:
application/json:
example:
character: 18
line: 120
path: apps/lsp/lsp.go
repo: cloud
schema:
$ref: '#/components/schemas/Query'
required: true
responses:
"200":
content:
application/json:
schema:
$ref: '#/components/schemas/Answer'
description: ok
summary: Offers the candidates a language server has at a position, typed and
resolved through the repository's dependencies rather than guessed from text.
tags:
- code
x-app: lsp
/v1/code/lsp/diagnostics:
post:
description: |-
Reports every problem the language server finds in one file —
compile errors, type errors and lints, each with its span and its severity (1
error, 2 warning, 3 information, 4 hint). The position is ignored.
operationId: post_v1_code_lsp_diagnostics
requestBody:
content:
application/json:
example:
path: apps/lsp/lsp.go
repo: cloud
schema:
$ref: '#/components/schemas/Query'
required: true
responses:
"200":
content:
application/json:
schema:
$ref: '#/components/schemas/Answer'
description: ok
summary: Reports every problem the language server finds in one file — compile
errors, type errors and lints, each with its span and its severity (1 error,
2 warning, 3 information, 4 hint).
tags:
- code
x-app: lsp
/v1/code/lsp/hover:
post:
description: |-
Renders the type and documentation of the symbol at a position, as the
language server itself renders it.
Positions are the LSP's: line and character are 0-BASED and character counts
UTF-16 code units, so an editor's 1-based line must have 1 subtracted before it
is sent. The repository is named by slug and is always one in the caller's own
org; rev pins a branch, tag or commit sha, and empty means the default branch.
operationId: post_v1_code_lsp_hover
requestBody:
content:
application/json:
example:
character: 18
line: 120
path: apps/lsp/lsp.go
repo: cloud
schema:
$ref: '#/components/schemas/Query'
required: true
responses:
"200":
content:
application/json:
schema:
$ref: '#/components/schemas/Answer'
description: ok
summary: Renders the type and documentation of the symbol at a position, as
the language server itself renders it.
tags:
- code
x-app: lsp
/v1/code/lsp/locate:
post:
description: |-
Finds where a symbol lives: its definition, its references, its type or
its implementations, chosen by relation (definition, reference, type,
implementation — empty means definition).
It resolves THROUGH dependencies. An answer whose external flag is set left the
repository, and its path is then the module coordinate it landed in — which is
the question a static index cannot answer and this service exists for.
operationId: post_v1_code_lsp_locate
requestBody:
content:
application/json:
example:
character: 18
line: 120
path: apps/lsp/lsp.go
relation: definition
repo: cloud
schema:
$ref: '#/components/schemas/Query'
required: true
responses:
"200":
content:
application/json:
schema:
$ref: '#/components/schemas/Answer'
description: ok
summary: 'Finds where a symbol lives: its definition, its references, its type
or its implementations, chosen by relation (definition, reference, type, implementation
— empty means definition).'
tags:
- code
x-app: lsp
/v1/code/lsp/symbols:
post:
description: |-
Outlines one file: every declaration in it, with its kind and its span.
The position is ignored — the answer is the whole file.
operationId: post_v1_code_lsp_symbols
requestBody:
content:
application/json:
example:
path: apps/lsp/lsp.go
repo: cloud
schema:
$ref: '#/components/schemas/Query'
required: true
responses:
"200":
content:
application/json:
schema:
$ref: '#/components/schemas/Answer'
description: ok
summary: 'Outlines one file: every declaration in it, with its kind and its
span.'
tags:
- code
x-app: lsp
/v1/code/search:
get:
description: |-
@@ -77336,50 +77493,6 @@ paths:
tags:
- logs
x-app: metrics
/v1/lsp:
post:
description: |-
Resolves one position in one repository through a live language server:
definition, references, type, implementation, hover, document symbols,
completion or diagnostics — over the repo AND its resolved dependencies, with
no toolchain on the caller's machine.
Positions are the LSP's: line and character are 0-BASED and character counts
UTF-16 code units, so an editor's 1-based line must have 1 subtracted before it
is sent. The repository is named by slug and is always one in the caller's own
org. rev pins a branch, tag or commit sha; empty means the default branch.
The first query against a (repo, rev) pays a cold start — checkout, dependency
fetch and the server's first index — and is the billed event; later queries
against the same revision are served from the warm workspace and are free. The
answer says which it was.
operationId: post_v1_lsp
requestBody:
content:
application/json:
example:
character: 18
line: 120
method: definition
path: apps/lsp/server.go
repo: cloud
schema:
$ref: '#/components/schemas/Query'
required: true
responses:
"200":
content:
application/json:
schema:
$ref: '#/components/schemas/Answer'
description: ok
summary: 'Resolves one position in one repository through a live language server:
definition, references, type, implementation, hover, document symbols, completion
or diagnostics — over the repo AND its resolved dependencies, with no toolchain
on the caller''s machine.'
tags:
- lsp
x-app: lsp
/v1/machines:
get:
description: |-
@@ -105567,10 +105680,6 @@ tags:
account''s latest usage snapshot.'
name: links
- name: logs
- description: Package lsp is live semantic code intelligence — definitions, references,
types, hover and diagnostics — over a repository AND its resolved dependencies,
served from the cloud with no toolchain on the caller's machine.
name: lsp
- description: 'Package visor is the compute you rent from Hanzo: machines, GPUs and
clusters — launch one, resize it, tear it down.'
name: machines
+3 -4
View File
@@ -1,6 +1,6 @@
{
"paths": 1762,
"operations": 2480,
"paths": 1766,
"operations": 2484,
"products": {
"admin": 86,
"ads": 7,
@@ -35,7 +35,7 @@
"cloud": 5,
"cloudflare": 33,
"clusters": 6,
"code": 7,
"code": 12,
"collections": 10,
"commands": 1,
"commerce": 123,
@@ -106,7 +106,6 @@
"licensing": 11,
"links": 11,
"logs": 3,
"lsp": 1,
"machines": 8,
"marketing": 35,
"marketplace": 6,
+8
View File
@@ -30,10 +30,18 @@ const App = "agents"
// from source without the document drifting away from the program. See
// plane_registry_test.go.
var Ops = []string{
plane.AgentsRunOnBehalf,
plane.AgentsSessionsCount,
plane.AgentsSessionsStop,
}
// AgentsRunOnBehalf run one agent turn as a linked user, for a chat bridge in another pr....
//
// Calls plane.AgentsRunOnBehalf on agents over the peer plane.
func AgentsRunOnBehalf(ctx context.Context, in *plane.RunOnBehalfIn) (*plane.RunOnBehalfOut, error) {
return plane.Ask[plane.RunOnBehalfIn, plane.RunOnBehalfOut](ctx, App, plane.AgentsRunOnBehalf, in)
}
// AgentsSessionsCount count the live sessions a match selects.
//
// Calls plane.AgentsSessionsCount on agents over the peer plane.
+8
View File
@@ -35,6 +35,7 @@ var Ops = []string{
plane.GitInbound,
plane.GitMirror,
plane.GitPublish,
plane.GitRev,
plane.GitStatus,
}
@@ -73,6 +74,13 @@ func GitPublish(ctx context.Context, in *plane.Visibility) (*struct{}, error) {
return plane.Ask[plane.Visibility, struct{}](ctx, App, plane.GitPublish, in)
}
// GitRev the commit a ref resolves to.
//
// Calls plane.GitRev on git over the peer plane.
func GitRev(ctx context.Context, in *plane.RevIn) (*plane.Rev, error) {
return plane.Ask[plane.RevIn, plane.Rev](ctx, App, plane.GitRev, in)
}
// GitStatus which of these repos are imported, and which are in conflict.
//
// Calls plane.GitStatus on git over the peer plane.
+24
View File
@@ -165,6 +165,16 @@ const (
GitInbound = "git_inbound"
GitPublish = "git_publish"
// GitRev answers which commit a ref names, and nothing else.
//
// It is separate from GitFiles because the two questions have different
// COSTS, not merely different shapes. A caller that pins a revision on every
// request — the language-server proxy asks one per position query — would
// otherwise have to read a whole tree to learn a sha, which is a monorepo
// crossing a socket to answer forty bytes. Resolving is a ref lookup; reading
// is a walk. One op each.
GitRev = "git_rev"
// GitStatus reads the per-repo import/sync status the console repo list
// renders. Same boundary as GitImport: the app that lists the repos is
// integrations, the app that knows whether one is imported is git.
@@ -824,6 +834,20 @@ type Imported struct {
Repo string `json:"repo"`
}
// RevIn asks which commit a ref names. An empty Ref means the repo's default
// branch.
type RevIn struct {
Repo string `json:"repo" validate:"required"`
Ref string `json:"ref,omitempty"`
}
// Rev is a resolved commit and the label it was reached by, so a caller can echo
// which branch it is looking at without re-deriving it.
type Rev struct {
Rev string `json:"rev"`
Ref string `json:"ref,omitempty"`
}
// FilesIn asks for a repo's files at one ref.
type FilesIn struct {
Repo string `json:"repo" validate:"required"`
+9 -10
View File
@@ -18,19 +18,18 @@ import (
// Scaffolded by plugin/gen-app-cmds from the manifest.Apps row; now hand-owned.
//
// Metered, not Free: a query's cost is not a property of the door but of whether
// it had to check the repository out and index it, which only the app knows. The
// edge therefore charges nothing and apps/lsp/meter.go owns the debit — a cold
// start is billed, a warm point query is not.
// the revision had to be prepared — fetched and indexed — which only the app
// knows. The edge therefore charges nothing and apps/lsp/meter.go owns the debit:
// a prepare is billed, a query against a prepared revision is not.
//
// Shutdown is not optional here. A warm workspace is a LIVE language-server
// subprocess and a checkout on disk; neither is reclaimed by this process exiting,
// so without the hook a rolling deploy leaves orphaned gopls processes behind.
// There is no Shutdown. The language servers and the checkouts run in the
// hanzoai/lsp daemon on its own deployment, not here; this app holds an
// http.Client, which the process exiting reclaims.
func main() {
if err := cloud.Listen([]cloud.Plugin{{
Name: "lsp",
Price: cloud.Metered,
Mount: lsp.Mount,
Shutdown: lsp.Shutdown,
Name: "lsp",
Price: cloud.Metered,
Mount: lsp.Mount,
}}, []string{"lsp"}); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
+167 -16
View File
@@ -2,7 +2,7 @@
"openapi": "3.1.0",
"info": {
"title": "Hanzo Cloud API",
"description": "Package lsp is live semantic code intelligence — definitions, references, types, hover and diagnostics — over a repository AND its resolved dependencies, served from the cloud with no toolchain on the caller's machine.",
"description": "Package lsp is live semantic code intelligence — definitions, references, types, hover, outline and diagnostics — over a repository AND its resolved dependencies, with no toolchain on the caller's machine.",
"version": "v1"
},
"servers": [
@@ -12,17 +12,17 @@
],
"tags": [
{
"name": "lsp"
"name": "code"
}
],
"paths": {
"/v1/lsp": {
"/v1/code/lsp/complete": {
"post": {
"operationId": "post_v1_lsp",
"summary": "Resolves one position in one repository through a live language server: definition, references, type, implementation, hover, document symbols, completion or diagnostics — over the repo AND its resolved dependencies, with no toolchain on the caller's machine.",
"description": "Resolves one position in one repository through a live language server:\ndefinition, references, type, implementation, hover, document symbols,\ncompletion or diagnostics — over the repo AND its resolved dependencies, with\nno toolchain on the caller's machine.\n\nPositions are the LSP's: line and character are 0-BASED and character counts\nUTF-16 code units, so an editor's 1-based line must have 1 subtracted before it\nis sent. The repository is named by slug and is always one in the caller's own\norg. rev pins a branch, tag or commit sha; empty means the default branch.\n\nThe first query against a (repo, rev) pays a cold start — checkout, dependency\nfetch and the server's first index — and is the billed event; later queries\nagainst the same revision are served from the warm workspace and are free. The\nanswer says which it was.",
"operationId": "post_v1_code_lsp_complete",
"summary": "Offers the candidates a language server has at a position, typed and resolved through the repository's dependencies rather than guessed from text.",
"description": "Offers the candidates a language server has at a position, typed and\nresolved through the repository's dependencies rather than guessed from text.",
"tags": [
"lsp"
"code"
],
"requestBody": {
"content": {
@@ -30,8 +30,156 @@
"example": {
"character": 18,
"line": 120,
"method": "definition",
"path": "apps/lsp/server.go",
"path": "apps/lsp/lsp.go",
"repo": "cloud"
},
"schema": {
"$ref": "#/components/schemas/Query"
}
}
},
"required": true
},
"responses": {
"200": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Answer"
}
}
},
"description": "ok"
}
}
}
},
"/v1/code/lsp/diagnostics": {
"post": {
"operationId": "post_v1_code_lsp_diagnostics",
"summary": "Reports every problem the language server finds in one file — compile errors, type errors and lints, each with its span and its severity (1 error, 2 warning, 3 information, 4 hint).",
"description": "Reports every problem the language server finds in one file —\ncompile errors, type errors and lints, each with its span and its severity (1\nerror, 2 warning, 3 information, 4 hint). The position is ignored.",
"tags": [
"code"
],
"requestBody": {
"content": {
"application/json": {
"example": {
"path": "apps/lsp/lsp.go",
"repo": "cloud"
},
"schema": {
"$ref": "#/components/schemas/Query"
}
}
},
"required": true
},
"responses": {
"200": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Answer"
}
}
},
"description": "ok"
}
}
}
},
"/v1/code/lsp/hover": {
"post": {
"operationId": "post_v1_code_lsp_hover",
"summary": "Renders the type and documentation of the symbol at a position, as the language server itself renders it.",
"description": "Renders the type and documentation of the symbol at a position, as the\nlanguage server itself renders it.\n\nPositions are the LSP's: line and character are 0-BASED and character counts\nUTF-16 code units, so an editor's 1-based line must have 1 subtracted before it\nis sent. The repository is named by slug and is always one in the caller's own\norg; rev pins a branch, tag or commit sha, and empty means the default branch.",
"tags": [
"code"
],
"requestBody": {
"content": {
"application/json": {
"example": {
"character": 18,
"line": 120,
"path": "apps/lsp/lsp.go",
"repo": "cloud"
},
"schema": {
"$ref": "#/components/schemas/Query"
}
}
},
"required": true
},
"responses": {
"200": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Answer"
}
}
},
"description": "ok"
}
}
}
},
"/v1/code/lsp/locate": {
"post": {
"operationId": "post_v1_code_lsp_locate",
"summary": "Finds where a symbol lives: its definition, its references, its type or its implementations, chosen by relation (definition, reference, type, implementation — empty means definition).",
"description": "Finds where a symbol lives: its definition, its references, its type or\nits implementations, chosen by relation (definition, reference, type,\nimplementation — empty means definition).\n\nIt resolves THROUGH dependencies. An answer whose external flag is set left the\nrepository, and its path is then the module coordinate it landed in — which is\nthe question a static index cannot answer and this service exists for.",
"tags": [
"code"
],
"requestBody": {
"content": {
"application/json": {
"example": {
"character": 18,
"line": 120,
"path": "apps/lsp/lsp.go",
"relation": "definition",
"repo": "cloud"
},
"schema": {
"$ref": "#/components/schemas/Query"
}
}
},
"required": true
},
"responses": {
"200": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Answer"
}
}
},
"description": "ok"
}
}
}
},
"/v1/code/lsp/symbols": {
"post": {
"operationId": "post_v1_code_lsp_symbols",
"summary": "Outlines one file: every declaration in it, with its kind and its span.",
"description": "Outlines one file: every declaration in it, with its kind and its span.\nThe position is ignored — the answer is the whole file.",
"tags": [
"code"
],
"requestBody": {
"content": {
"application/json": {
"example": {
"path": "apps/lsp/lsp.go",
"repo": "cloud"
},
"schema": {
@@ -61,7 +209,7 @@
"Answer": {
"properties": {
"cold": {
"description": "Cold reports that this request paid for a workspace cold start — the\ncheckout, the dependency fetch and the server's first index. It is the\nbilled event, surfaced so a caller can see what it was charged for.",
"description": "Cold reports that this request paid to PREPARE the revision — the tree\nwrite, the dependency fetch and the language server's first index. It is\nthe billed event, surfaced so a caller can see what it was charged for.",
"type": "boolean"
},
"completions": {
@@ -88,7 +236,7 @@
},
"type": "array"
},
"method": {
"op": {
"type": "string"
},
"path": {
@@ -145,6 +293,9 @@
},
"Location": {
"properties": {
"external": {
"type": "boolean"
},
"path": {
"type": "string"
},
@@ -175,12 +326,12 @@
"description": "Line is 0-based, per the LSP specification.",
"type": "integer"
},
"method": {
"description": "Method is the question: hover, definition, references, typeDefinition,\nimplementation, documentSymbol, completion or diagnostics.",
"path": {
"description": "Path is the repo-relative file, e.g. \"apps/lsp/lsp.go\".",
"type": "string"
},
"path": {
"description": "Path is the repo-relative file, e.g. \"apps/lsp/server.go\".",
"relation": {
"description": "Relation refines locate: definition, reference, type or implementation.\nEmpty means definition. Every other op ignores it.",
"type": "string"
},
"repo": {
@@ -188,7 +339,7 @@
"type": "string"
},
"rev": {
"description": "Rev is a branch, tag or commit sha. Empty means the default branch. A\nworkspace is keyed by revision, so pinning a sha is what makes an answer\nreproducible.",
"description": "Rev is a branch, tag or commit sha. Empty means the default branch. It is\nresolved to a commit before anything else happens, so an answer is always\nabout one immutable tree.",
"type": "string"
}
},