Files
ai/bootstrap.go
zeekay 39d9ec81ae ai serves its own routes: the second router is gone
The whole module builds on zip. This is the rest of it — routers/, the relay, and
the 2,823-line in-house router underneath.

ONE ROUTE TABLE. Every address was registered on a hand-written router held in a
package variable built at import time, and the zip app in front of it was a single
app.All("/v1/*", relay) forwarding through a net/http adaptor into that router. A
short "promoted" list re-registered three addresses specifically so the fleet
document could NAME them, because a glob tells a host a prefix and nothing under
it: three operations authenticated, reachable and answering in production with no
generated SDK, no MCP tool, no CLI command, and nothing saying who served them.
Every address is its own route now, so every one is described, and the boundary
the relay crossed — with the OTel context it had to stow and restore over it — is
gone with the relay.

Registration is one entry point, called with the app instead of writing onto a
global at import. The filters go on FIRST, and that is not cosmetic: the router
walks its stack in registration order, so a filter added after a route never runs
before it.

ONE IDENTITY. The filters read a per-pod in-memory session store; the handlers read
the request's verified IAM token. The store's own comment recorded the cost — it is
filled by a hook that does not run when the embedded binary serves these routes, so
it answered nil on every request and the filters billed and rate-limited by the raw
key instead. Filters now ask the same function the handlers ask.

Two filters went entirely, both doing somebody else's job:

  - Auto-signin accepted an MD5 of clientId:clientSecret, or HTTP Basic against the
    configured pair, and minted an IsAdmin identity into that store. IAM mints
    credentials. It was reachable only for /storage, which is why it survived.
  - Static served a landing folder, the admin bundle, and /storage from a path
    taken off the URL — with no cleaning, beside a sibling branch that checks for
    "/.." — then rewrote main.<hash>.js on the way out. Static content is ingress
    and the static plugin. Every route here is /v1.

A REFUSAL IS THE END OF THE REQUEST, and now says so. A filter used to write a
response and return, and the router inferred a refusal from something having been
written. Two places got that wrong: the demo-mode gate denied and then ran the
permission filter anyway, and the error envelope defaulted to 200 with a second
function beside it for callers who wanted a real status. One envelope, status
required.

Recording was two filters passing a username through a request parameter, because
a before-filter and an after-filter had no other way to share; it is one
middleware holding a local across the handler. The cookie rewriter went with the
store it patched. The prometheus filter counted /api throughput — a prefix this
service has never served — so its counter has always been zero.

The standalone binary serves the app on zip's own listener rather than a net/http
server wrapping it, and the ZAP transports take the same app as an http.Handler,
resolved per request: App.Fiber() hands out the current generation's router, and a
generation is a projection.

web/ keeps its React source and loses its Go.
2026-08-17 08:35:46 -07:00

280 lines
13 KiB
Go

// Copyright 2023-2025 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package ai
import (
"context"
"encoding/json"
"fmt"
"sync"
"github.com/hanzoai/ai/log"
webtools "github.com/hanzoai/ai/agent/builtin_tool/web"
"github.com/hanzoai/ai/conf"
"github.com/hanzoai/ai/controllers"
"github.com/hanzoai/ai/model"
"github.com/hanzoai/ai/object"
"github.com/hanzoai/ai/proxy"
"github.com/hanzoai/ai/routers"
"github.com/hanzoai/ai/util"
)
// Bootstrap performs the AI runtime initialization shared by BOTH
// entrypoints — the standalone server (cmd/aid) and the embedded unified
// cloud binary (Mount, per HIP-0106). It is the SINGLE source of the
// runtime boot sequence: DB + adapter + tables, model/pricing config, the
// HTTP client, GeoIP/parser, the background maintenance tasks, the
// Commerce-backed balance gate + tier cache + per-key rate limiter, the
// full BeforeRouter/AfterExec filter chain, the session config, and
// the billing usage queue. After wiring those it publishes the
// fully-configured native router via SetHandler so the unified
// binary's /v1/ai/* adapter stops returning 503 "ai runtime not
// initialized".
//
// What Bootstrap deliberately does NOT do (those are standalone-only
// concerns that the embedded binary owns differently, and several would
// break or collide when co-resident in the unified process):
//
// - It binds NO listeners. The native ZAP inference node (port 9999),
// the inter-service ZAP transport (CLOUD_ZAP_PORT, default 9320) and
// the standalone HTTP listener (:httpport) stay in cmd/aid. The unified
// binary serves routers.App through zip on its own :8080 and runs its own
// ZAP at :9653; starting the legacy nodes here would collide.
// - It calls NO util.StopOldInstance — that races on the legacy standalone
// port and is meaningless when the embedded binary never listens there.
// - It installs NO signal handler and never calls os.Exit. cloud.Serve
// owns graceful shutdown for the unified binary; cmd/aid keeps its own
// drain goroutine, wired to the handles returned here.
//
// Bootstrap is guarded by sync.Once: it is safe to call more than once, so
// Mount can call it at mount time and the standalone can call it explicitly
// without double-initializing global runtime state. The cached result
// (including the error) is returned on every call.
//
// It returns an error rather than panicking: several init steps (notably
// the DB open) panic deep inside on a missing/unreachable backend, which
// would take down the whole multi-subsystem cloud binary. Bootstrap
// recovers those into a precise error so Mount can surface
// "mount ai: bootstrap: ..." and the operator sees exactly what failed.
//
// The rate limiter and billing queue created here are exposed via
// RateLimiter() and BillingQueue() for the standalone's graceful-drain
// wiring. Either may be nil (e.g. the billing queue is nil when no Commerce
// endpoint is configured).
func Bootstrap() error {
bootstrapOnce.Do(func() {
bootErr = doBootstrap()
})
return bootErr
}
// RateLimiter returns the per-key rate limiter created by Bootstrap, or nil
// if Bootstrap has not run (or failed before creating it).
func RateLimiter() *routers.RateLimiter { return bootRateLimiter }
// BillingQueue returns the Commerce billing usage queue created by
// Bootstrap, or nil if none was configured / Bootstrap did not reach it.
func BillingQueue() *util.BillingQueue { return bootBillingQueue }
var (
bootstrapOnce sync.Once
bootErr error
bootRateLimiter *routers.RateLimiter
bootBillingQueue *util.BillingQueue
)
func doBootstrap() (err error) {
// Convert the deep DB-open panic (and any other init panic) into a
// precise error so a missing backend fails the AI mount cleanly instead
// of crashing the entire cloud process.
defer func() {
if r := recover(); r != nil {
err = fmt.Errorf("ai: bootstrap: %v", r)
}
}()
// Persistence: flags, ORM adapter, schema, DB connection.
object.InitFlag()
object.InitAdapter()
object.CreateTables()
object.InitDb()
// Analytics datastore (datastore): the OLAP ledger behind the usage/spend
// Overview (hanzo.cloud_usage) and the per-tenant trace ledger
// (hanzo.observations). Opt-in via DATASTORE_ADDR; connects in the background
// and is a no-op when unset, so boot never blocks on the warehouse. This runs
// in BOTH cmd/aid and the unified cloud binary (the ai ZAP node does not, so
// the datastore is reached directly, not via a ZAP peer). See object.InitDatastore.
object.InitDatastore()
// OTel GenAI telemetry: one gen_ai span per LLM call → o11y OTLP
// collector. Opt-in via OTEL_EXPORTER_OTLP_ENDPOINT; connects in the
// background and is a no-op when unset, so boot never blocks on the
// collector. Runs in BOTH cmd/aid and the unified cloud binary, mirroring
// InitDatastore. See object.InitTelemetry.
object.InitTelemetry()
// Cross-pod cache-invalidation bus: broadcast org-settings (router policy /
// savings↔quality dial) writes over Valkey pub/sub so a console change converges
// sub-second fleet-wide. Opt-in via KV_URL; connects in the background and is a
// no-op when unset, so boot never blocks on Valkey and the per-pod TTL still
// backstops. Mirrors InitDatastore/InitTelemetry. See object.InitCacheBus.
object.InitCacheBus()
// Model routing/pricing config from YAML. Non-fatal: static fallback.
configPath := conf.GetConfigString("modelConfigPath")
if configPath == "" {
configPath = "conf/models.yaml"
}
if err := controllers.InitModelConfig(configPath); err != nil {
log.Warn("Model config: %v (using static fallback)", err)
}
// Wire the YAML-backed context-window resolver: models.yaml context_window
// becomes the per-model source of truth for max context, overriding the
// static getContextLength table. Without this, glm-5.2 (1M context) fell
// through the table to a 16K fallback and dead-ended long prompts + /compact.
// Nil-safe: if ModelConfig failed to load, the table-only path stays.
if mc := controllers.GetModelConfig(); mc != nil {
model.SetContextWindowResolver(mc.ContextWindow)
}
// Provider HTTP client, GeoIP, request parsing, maintenance tasks.
proxy.InitHttpClient()
util.InitMaxmindFiles()
util.InitIpDb()
util.InitParser()
object.InitCleanupChats()
object.InitStoreCount()
object.InitCommitRecordsTask()
object.InitScanJobProcessor()
object.InitMessageTransactionRetry()
// The in-pod balance ledger (object.GlobalBalanceLedger) reserves/settles in
// process memory — correct ONLY at a SINGLE replica (deploy strategy=Recreate
// + HPA min=max=1). If the operator stamped CLOUD_API_REPLICAS > 1, refuse to
// start rather than silently double-spend across pods; scaling out requires a
// Commerce-atomic conditional reserve first.
if n, ok := object.SinglePodReplicaHint(); ok && n > 1 {
log.Error("FATAL: CLOUD_API_REPLICAS=%d — the in-pod balance ledger double-spends across pods. Refusing to start; implement a Commerce-atomic reserve before scaling out.", n)
panic("cloud-api: in-pod balance ledger requires a single replica (CLOUD_API_REPLICAS>1)")
}
log.Info("balance ledger: single-pod invariant active (reservation ledger requires replicas=1, strategy=Recreate, HPA min=max=1)")
// The page fetcher behind the fetch_url builtin tool. Installed HERE rather than
// called directly because agent/builtin_tool/web must stay a leaf: `object`
// imports `agent`, and `agent` imports the tool registry, so a tool reaching into
// object for Crawl closes the loop and the module stops compiling. The root
// package imports both and can hand one to the other.
//
// Search and deep research are the HOST's (hanzoai/cloud serves /v1/websearch and
// /v1/ask); it installs those at mount time. Until it does, those two tools answer
// that they are unavailable in this deployment — never an empty result, which an
// agent would read as "the web holds nothing on this" and answer from memory.
webtools.SetFetch(func(_ context.Context, urls []string) ([]webtools.FetchResult, error) {
results, err := object.Crawl(urls)
if err != nil {
return nil, err
}
out := make([]webtools.FetchResult, 0, len(results))
for _, r := range results {
out = append(out, webtools.FetchResult{
URL: r.URL, Title: r.Title, Markdown: r.Markdown, Success: r.Success,
})
}
return out, nil
})
// Commerce balance gate (pre-request balance enforcement).
routers.InitBalanceGate()
// Commerce-backed tier cache — must precede InitRateLimiter so
// DefaultTierFunc can resolve tiers through it.
routers.InitTierCache()
// Per-key rate limiting (env override → tier cache → zen-free).
bootRateLimiter = routers.InitRateLimiter(routers.DefaultTierFunc)
log.Info("Per-key rate limiter initialized (tiers: free=10/min, starter=60/min, pro=300/min, enterprise=1000/min)")
// The filter chain is installed with the ROUTES, by routers.Register, because
// the router walks its stack in registration order and a filter added after a
// route never runs before it. Boot is runtime init — the database, the
// providers, the queues — and no longer wires the surface.
//
// There is no session store. It was a per-pod in-memory store backing a cookie
// flow, written only by the auto-signin filter that is gone; identity is the
// verified IAM token, read where it is needed.
// Optional log adapter reconfig. Guarded: in the embedded binary there
// is no conf/app.conf, so logConfig is empty — json.Unmarshal("") would
// panic. Skip when unset and keep the default logger.
if raw := conf.GetConfigString("logConfig"); raw != "" {
logConfigMap := make(map[string]interface{})
if err := json.Unmarshal([]byte(raw), &logConfigMap); err != nil {
log.Warn("logConfig parse failed: %v (keeping default logger)", err)
} else {
logAdapter := "file"
if v, ok := logConfigMap["adapter"].(string); ok {
logAdapter = v
}
if logAdapter == "console" {
log.Reset()
}
if err := log.SetLogger(logAdapter, raw); err != nil {
log.Warn("logConfig SetLogger failed: %v (keeping default logger)", err)
}
}
}
// Commerce-backed billing usage queue (retried with backoff). nil when
// no Commerce endpoint is configured.
bootBillingQueue = controllers.InitBillingQueue()
if bootBillingQueue != nil {
log.Info("Billing queue started (Commerce endpoint configured)")
}
// Nothing to publish. The app IS the handler: ai.App builds it, registers every
// address on it, and stores it — so the socket and the ZAP transports reach one
// route table with one filter chain, rather than a handler being set here and
// found by whoever asks.
// Enso router flywheel (HIP-510): the learned-routing loop that trains on real
// traffic. Started HERE — in the SINGLE sync.Once boot sequence BOTH cmd/aid and
// ai.Mount run — so it is the ONE launch site (DRY): the deployed unified binary
// mounts via cloud.Wire → ai.Mount → Bootstrap, so wiring it here (not in a caller)
// is what makes the flywheel actually run in production, where it previously never
// did (retrains: [], rewarded_events: 0). Both Start* self-gate on env and default
// OFF (ROUTER_TRAIN_ENABLED for the trainer; ROUTER_PROBE_RPH+ROUTER_PROBE_TOKEN for
// the probe), so this is dark-by-default and turns real only via deployment config.
// sync.Once guarantees the goroutines are launched exactly once per process.
// StartRouterJudge arms the LLM-as-a-judge dense quality reward (openai_api.go's
// post-response hook): it turns served (prompt, response) pairs into 0..1 quality
// rewards for the flywheel — anonymous, consent-gated, content-transient. Unlike
// the probe/trainer it is DB-backed and ON by default: it reads its config live
// from the "*" GlobalDefaultOwner OrgSettings row (judge.enabled/models/url/sample),
// tunable at admin.hanzo.ai with no restart, and runs the Mean-Field Judge Panel on
// ~10% of eligible (opted-in, non-EU-protected) traffic out of the box. Point
// judge.url at a TEE endpoint for confidential inference (controllers/router_judge.go).
// SeedInternalTrainingConsent first flips our OWN reserved orgs to explicit opt-in
// so the EU per-request guard never suppresses our internal traffic.
controllers.SeedInternalTrainingConsent()
controllers.StartRouterProbe()
controllers.StartRouterTrainer()
controllers.StartRouterJudge()
return nil
}