Files
ai/bootstrap.go
zeekayandHanzo Dev bcb27958ec agent: every Responses-API agent gets the web — search, fetch, deep research
The builtin registry held five TIME tools. An agent on /v1/responses asking to
search the web had nothing to call, while /v1/websearch/search, /v1/crawl and
/v1/ask had been live the whole time. The capability existed and no agent could
reach it.

Three tools, registered unconditionally so an agent's tool list does not change
shape with deployment:

  web_search     ranked hits with titles, URLs, snippets
  fetch_url      pages as clean Markdown (this module's own crawl)
  deep_research  a cited report synthesised across sources

EVERY BACKEND IS A SEAM, and that is forced rather than chosen.
agent/builtin_tool/web must stay a LEAF: `object` imports `agent`, `agent`
imports the registry, so a tool reaching into object for Crawl closes the loop —
agent → builtin_tool → web → object → agent — and the module stops compiling.
Search and research are the HOST's anyway (hanzoai/cloud serves them and imports
this module, so it cannot be imported back). Crawl is installed by this module's
bootstrap; the host installs the other two at mount time, beside the balance and
tier readers it already installs.

That shape is also the lesson this repo already paid for: the ai module once
reached its own platform over api.hanzo.ai, which carries a CUSTOMER credential
and answers 401 to a service, and every completion fail-closed at 503 on a
healthy pod. The edge is for customers; in-process is for us.

AN UNAVAILABLE CAPABILITY IS AN ERROR, NEVER AN EMPTY RESULT — the property
worth the most here. A model reads ordinary tool text as a FINDING: hand it "no
results" when the backend was never installed and it concludes the web holds
nothing on the subject, then answers from memory in a confident voice. That is a
fabricated answer caused by a deployment gap, and nothing downstream can tell it
from a real one. A genuinely empty search still returns a normal result, because
that IS a finding.

Bounded on purpose: limit clamped to 10 (a confused call must not exhaust the
agent's context window), 5 URLs per fetch, 20k markdown per page, and a failed
page is NAMED rather than omitted — an agent cannot tell "unreachable" from
"said nothing" if it silently disappears.

fetch_url accepts only absolute http(s). The URLs it is handed come from PAGES
the model read, so following file:// or a custom scheme would read the host's own
disk on behalf of whoever wrote that page. It also accepts a bare string, which
is the shape a model sends most often and is unambiguous.

13 tests, including the refuse-vs-empty property for all three tools, the scheme
filter, the clamps, and that the tools are both REGISTERED and exposed in
GetToolsAsProtocolTools — a tool absent from that list is invisible to agents
however well it is registered.

Co-authored-by: Hanzo Dev <[email protected]>
2026-08-05 01:38:04 -07:00

290 lines
13 KiB
Go

// Copyright 2023-2025 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package ai
import (
"context"
"encoding/json"
"fmt"
"sync"
"time"
"github.com/hanzoai/ai/log"
"github.com/hanzoai/ai/web"
webtools "github.com/hanzoai/ai/agent/builtin_tool/web"
"github.com/hanzoai/ai/conf"
"github.com/hanzoai/ai/controllers"
"github.com/hanzoai/ai/model"
"github.com/hanzoai/ai/object"
"github.com/hanzoai/ai/proxy"
"github.com/hanzoai/ai/routers"
"github.com/hanzoai/ai/util"
)
// Bootstrap performs the AI runtime initialization shared by BOTH
// entrypoints — the standalone server (cmd/aid) and the embedded unified
// cloud binary (Mount, per HIP-0106). It is the SINGLE source of the
// runtime boot sequence: DB + adapter + tables, model/pricing config, the
// HTTP client, GeoIP/parser, the background maintenance tasks, the
// Commerce-backed balance gate + tier cache + per-key rate limiter, the
// full BeforeRouter/AfterExec filter chain, the session config, and
// the billing usage queue. After wiring those it publishes the
// fully-configured native router via SetHandler so the unified
// binary's /v1/ai/* adapter stops returning 503 "ai runtime not
// initialized".
//
// What Bootstrap deliberately does NOT do (those are standalone-only
// concerns that the embedded binary owns differently, and several would
// break or collide when co-resident in the unified process):
//
// - It binds NO listeners. The native ZAP inference node (port 9999),
// the inter-service ZAP transport (CLOUD_ZAP_PORT, default 9320) and
// the standalone HTTP listener (:httpport) stay in cmd/aid. The unified
// binary serves routers.App through zip on its own :8080 and runs its own
// ZAP at :9653; starting the legacy nodes here would collide.
// - It calls NO util.StopOldInstance — that races on the legacy standalone
// port and is meaningless when the embedded binary never listens there.
// - It installs NO signal handler and never calls os.Exit. cloud.Serve
// owns graceful shutdown for the unified binary; cmd/aid keeps its own
// drain goroutine, wired to the handles returned here.
//
// Bootstrap is guarded by sync.Once: it is safe to call more than once, so
// Mount can call it at mount time and the standalone can call it explicitly
// without double-initializing global runtime state. The cached result
// (including the error) is returned on every call.
//
// It returns an error rather than panicking: several init steps (notably
// the DB open) panic deep inside on a missing/unreachable backend, which
// would take down the whole multi-subsystem cloud binary. Bootstrap
// recovers those into a precise error so Mount can surface
// "mount ai: bootstrap: ..." and the operator sees exactly what failed.
//
// The rate limiter and billing queue created here are exposed via
// RateLimiter() and BillingQueue() for the standalone's graceful-drain
// wiring. Either may be nil (e.g. the billing queue is nil when no Commerce
// endpoint is configured).
func Bootstrap() error {
bootstrapOnce.Do(func() {
bootErr = doBootstrap()
})
return bootErr
}
// RateLimiter returns the per-key rate limiter created by Bootstrap, or nil
// if Bootstrap has not run (or failed before creating it).
func RateLimiter() *routers.RateLimiter { return bootRateLimiter }
// BillingQueue returns the Commerce billing usage queue created by
// Bootstrap, or nil if none was configured / Bootstrap did not reach it.
func BillingQueue() *util.BillingQueue { return bootBillingQueue }
var (
bootstrapOnce sync.Once
bootErr error
bootRateLimiter *routers.RateLimiter
bootBillingQueue *util.BillingQueue
)
func doBootstrap() (err error) {
// Convert the deep DB-open panic (and any other init panic) into a
// precise error so a missing backend fails the AI mount cleanly instead
// of crashing the entire cloud process.
defer func() {
if r := recover(); r != nil {
err = fmt.Errorf("ai: bootstrap: %v", r)
}
}()
// Persistence: flags, ORM adapter, schema, DB connection.
object.InitFlag()
object.InitAdapter()
object.CreateTables()
object.InitDb()
// Analytics datastore (datastore): the OLAP ledger behind the usage/spend
// Overview (hanzo.cloud_usage) and the per-tenant trace ledger
// (hanzo.observations). Opt-in via DATASTORE_ADDR; connects in the background
// and is a no-op when unset, so boot never blocks on the warehouse. This runs
// in BOTH cmd/aid and the unified cloud binary (the ai ZAP node does not, so
// the datastore is reached directly, not via a ZAP peer). See object.InitDatastore.
object.InitDatastore()
// OTel GenAI telemetry: one gen_ai span per LLM call → o11y OTLP
// collector. Opt-in via OTEL_EXPORTER_OTLP_ENDPOINT; connects in the
// background and is a no-op when unset, so boot never blocks on the
// collector. Runs in BOTH cmd/aid and the unified cloud binary, mirroring
// InitDatastore. See object.InitTelemetry.
object.InitTelemetry()
// Cross-pod cache-invalidation bus: broadcast org-settings (router policy /
// savings↔quality dial) writes over Valkey pub/sub so a console change converges
// sub-second fleet-wide. Opt-in via KV_URL; connects in the background and is a
// no-op when unset, so boot never blocks on Valkey and the per-pod TTL still
// backstops. Mirrors InitDatastore/InitTelemetry. See object.InitCacheBus.
object.InitCacheBus()
// Model routing/pricing config from YAML. Non-fatal: static fallback.
configPath := conf.GetConfigString("modelConfigPath")
if configPath == "" {
configPath = "conf/models.yaml"
}
if err := controllers.InitModelConfig(configPath); err != nil {
log.Warn("Model config: %v (using static fallback)", err)
}
// Wire the YAML-backed context-window resolver: models.yaml context_window
// becomes the per-model source of truth for max context, overriding the
// static getContextLength table. Without this, glm-5.2 (1M context) fell
// through the table to a 16K fallback and dead-ended long prompts + /compact.
// Nil-safe: if ModelConfig failed to load, the table-only path stays.
if mc := controllers.GetModelConfig(); mc != nil {
model.SetContextWindowResolver(mc.ContextWindow)
}
// Provider HTTP client, GeoIP, request parsing, maintenance tasks.
proxy.InitHttpClient()
util.InitMaxmindFiles()
util.InitIpDb()
util.InitParser()
object.InitCleanupChats()
object.InitStoreCount()
object.InitCommitRecordsTask()
object.InitScanJobProcessor()
object.InitMessageTransactionRetry()
// The in-pod balance ledger (object.GlobalBalanceLedger) reserves/settles in
// process memory — correct ONLY at a SINGLE replica (deploy strategy=Recreate
// + HPA min=max=1). If the operator stamped CLOUD_API_REPLICAS > 1, refuse to
// start rather than silently double-spend across pods; scaling out requires a
// Commerce-atomic conditional reserve first.
if n, ok := object.SinglePodReplicaHint(); ok && n > 1 {
log.Error("FATAL: CLOUD_API_REPLICAS=%d — the in-pod balance ledger double-spends across pods. Refusing to start; implement a Commerce-atomic reserve before scaling out.", n)
panic("cloud-api: in-pod balance ledger requires a single replica (CLOUD_API_REPLICAS>1)")
}
log.Info("balance ledger: single-pod invariant active (reservation ledger requires replicas=1, strategy=Recreate, HPA min=max=1)")
// The page fetcher behind the fetch_url builtin tool. Installed HERE rather than
// called directly because agent/builtin_tool/web must stay a leaf: `object`
// imports `agent`, and `agent` imports the tool registry, so a tool reaching into
// object for Crawl closes the loop and the module stops compiling. The root
// package imports both and can hand one to the other.
//
// Search and deep research are the HOST's (hanzoai/cloud serves /v1/websearch and
// /v1/ask); it installs those at mount time. Until it does, those two tools answer
// that they are unavailable in this deployment — never an empty result, which an
// agent would read as "the web holds nothing on this" and answer from memory.
webtools.SetFetch(func(_ context.Context, urls []string) ([]webtools.FetchResult, error) {
results, err := object.Crawl(urls)
if err != nil {
return nil, err
}
out := make([]webtools.FetchResult, 0, len(results))
for _, r := range results {
out = append(out, webtools.FetchResult{
URL: r.URL, Title: r.Title, Markdown: r.Markdown, Success: r.Success,
})
}
return out, nil
})
// Commerce balance gate (pre-request balance enforcement).
routers.InitBalanceGate()
// Commerce-backed tier cache — must precede InitRateLimiter so
// DefaultTierFunc can resolve tiers through it.
routers.InitTierCache()
// Per-key rate limiting (env override → tier cache → zen-free).
bootRateLimiter = routers.InitRateLimiter(routers.DefaultTierFunc)
log.Info("Per-key rate limiter initialized (tiers: free=10/min, starter=60/min, pro=300/min, enterprise=1000/min)")
// Filter chain on the ai router, in the same order as the standalone
// surface so the handler enforces the same rewrite/auth/tenant/balance
// pipeline. The router caches the request body itself (no CopyRequestBody
// knob), and the /swagger static is served at the edge, not here.
routers.InstallFilters()
// Session store. API requests authenticate per-request via bearer key/JWT;
// sessions back only the cookie web-admin flow, so their contents are
// incidental. An in-process memory store needs no filesystem (the runtime
// ships read-only) and suits the single-replica runtime. Built once and
// bound to the router so every request has a session bound before the
// filters run (setSessionUser needs a present store). SameSite=Lax.
if web.Sessions == nil {
web.Sessions = web.NewMemorySessions("cloud_session_id", 365*24*time.Hour)
}
routers.App.UseSessions(web.Sessions)
// Optional log adapter reconfig. Guarded: in the embedded binary there
// is no conf/app.conf, so logConfig is empty — json.Unmarshal("") would
// panic. Skip when unset and keep the default logger.
if raw := conf.GetConfigString("logConfig"); raw != "" {
logConfigMap := make(map[string]interface{})
if err := json.Unmarshal([]byte(raw), &logConfigMap); err != nil {
log.Warn("logConfig parse failed: %v (keeping default logger)", err)
} else {
logAdapter := "file"
if v, ok := logConfigMap["adapter"].(string); ok {
logAdapter = v
}
if logAdapter == "console" {
log.Reset()
}
if err := log.SetLogger(logAdapter, raw); err != nil {
log.Warn("logConfig SetLogger failed: %v (keeping default logger)", err)
}
}
}
// Commerce-backed billing usage queue (retried with backoff). nil when
// no Commerce endpoint is configured.
bootBillingQueue = controllers.InitBillingQueue()
if bootBillingQueue != nil {
log.Info("Billing queue started (Commerce endpoint configured)")
}
// Publish the router as the request handler. Every BeforeRouter filter
// inserted above runs on each request the unified binary forwards into
// /v1/ai/*; the session store was bound to the router above.
SetHandler(routers.App)
// Enso router flywheel (HIP-510): the learned-routing loop that trains on real
// traffic. Started HERE — in the SINGLE sync.Once boot sequence BOTH cmd/aid and
// ai.Mount run — so it is the ONE launch site (DRY): the deployed unified binary
// mounts via cloud.Wire → ai.Mount → Bootstrap, so wiring it here (not in a caller)
// is what makes the flywheel actually run in production, where it previously never
// did (retrains: [], rewarded_events: 0). Both Start* self-gate on env and default
// OFF (ROUTER_TRAIN_ENABLED for the trainer; ROUTER_PROBE_RPH+ROUTER_PROBE_TOKEN for
// the probe), so this is dark-by-default and turns real only via deployment config.
// sync.Once guarantees the goroutines are launched exactly once per process.
// StartRouterJudge arms the LLM-as-a-judge dense quality reward (openai_api.go's
// post-response hook): it turns served (prompt, response) pairs into 0..1 quality
// rewards for the flywheel — anonymous, consent-gated, content-transient. Unlike
// the probe/trainer it is DB-backed and ON by default: it reads its config live
// from the "*" GlobalDefaultOwner OrgSettings row (judge.enabled/models/url/sample),
// tunable at admin.hanzo.ai with no restart, and runs the Mean-Field Judge Panel on
// ~10% of eligible (opted-in, non-EU-protected) traffic out of the box. Point
// judge.url at a TEE endpoint for confidential inference (controllers/router_judge.go).
// SeedInternalTrainingConsent first flips our OWN reserved orgs to explicit opt-in
// so the EU per-request guard never suppresses our internal traffic.
controllers.SeedInternalTrainingConsent()
controllers.StartRouterProbe()
controllers.StartRouterTrainer()
controllers.StartRouterJudge()
return nil
}