Files
ai/object/balance_ledger.go
zeekay 531e9f423a lane: no platform ceiling on the paid lane; a payer spends what they bought
The paid lane capped every paid-lane call together at PAID_LANE_DAILY ($10 a UTC
day by default) and each org at PAID_LANE_ORG_SHARE (25% of it), and counted
wallet-paid calls against both: a customer with $1,000 prepaid was held to $2.50 a
day, and every payer together to $10. The rule is no ceiling but what a payer
prepays, or their usage. Free users get free models; payers spend their own money.

Deleted: the platform day and the org share (paidLaneDaily, orgShare, the day,
spent and per-org accounting in laneBook), the paid_day store (object.PaidDay, its
table registration, the writer, flush, drain, load and turn), Settled's drain of
it, laneSpend, PaidLaneOpen (the gate reads !FreeOnly()), and the refusals only the
day produced ("closed", "after 00:00 UTC"). paid_lane_ceiling named a platform
ceiling that no longer exists: a payer the lane has no room for is told
paid_lane_full, the code its refusal already carried, in X-Hanzo-Lane-Reason and
X-Hanzo-Usage-Reason. Stores keep an orphan paid_day table nothing reads.

Kept: the wallet hold (reserveFor), the plan class's LimitGrant.Spend less the
payer's calls in flight, quote sizing against it, 8 seats a payer, lapse and
stream-idle give-back, race losers holding the seat, BYO, and the free-model
hand-off for a payer with no room.

The replica guard. The lane was closed unless CLOUD_API_REPLICAS=1 because the day
lived in this process. No deployment sets CLOUD_API_REPLICAS: the cloud chart
injects CLOUD_REPLICAS (1 in production). So the boot assertion meant to stop a
second replica double-spending the in-pod wallet ledger never fired, and the lane
guard was the one live check; it also kept the paid lane closed in production. The
plan holds that remain are in-pod exactly as the wallet ledger is, so they take its
guard: SinglePodReplicaHint reads CLOUD_REPLICAS, and bootstrap refuses to start
when the deployment states more than one replica. A declared second replica stops
the process instead of closing the lane, and no switch is left whose only purpose
was the day.
2026-10-09 14:08:38 -07:00

221 lines
8.1 KiB
Go

// Copyright 2023-2025 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package object
import (
"os"
"strconv"
"strings"
"sync"
"time"
)
// BalanceLedgerTTL is how long a cached Commerce balance is considered fresh.
// It matches the router gate's cache window so both read one consistent clock.
const BalanceLedgerTTL = 30 * time.Second
// ledgerEntry is the per-subject in-pod billing state.
type ledgerEntry struct {
balanceCents int64 // last balance fetched from Commerce
reservedCents int64 // sum of holds for in-flight requests
fetchedAt time.Time // when balanceCents was last set
// spentNano is settled spend below a whole cent, not yet taken off
// balanceCents. A call priced at a fraction of a cent is still spend: it counts
// against what is available as the whole cent it has begun, and becomes one
// once enough of them add up.
spentNano int64
}
// nanoPerCent is how many nano-USD make one cent.
const nanoPerCent = 10_000_000
// available is balance less holds less the cent any sub-cent spend has begun.
func (e *ledgerEntry) available() int64 {
return e.balanceCents - e.reservedCents - (e.spentNano+nanoPerCent-1)/nanoPerCent
}
// BalanceLedger is the in-pod source of truth for a billing subject's SPENDABLE
// balance: the last-known Commerce balance MINUS outstanding reservations for
// in-flight requests. The router balance gate reserves an estimated (upper-bound)
// cost before a paid request runs; the debit path settles the real cost after.
//
// Reserve and Settle are atomic under one mutex, so concurrent requests for the
// same subject can neither double-spend the same balance nor drive it negative —
// each in-flight request's worst-case cost is held until it settles. Applying the
// actual spend locally on Settle also closes the stale-cache window: a balance
// read that is up to TTL old no longer over-grants, because every settled request
// has already decremented the cached balance.
type BalanceLedger struct {
mu sync.Mutex
entries map[string]*ledgerEntry
ttl time.Duration
}
// GlobalBalanceLedger is the process-wide singleton shared by the router gate
// (reserve / balance cache) and the controller debit path (settle).
//
// SINGLE-POD INVARIANT: this ledger is in-pod (per-process) memory. Reserve/Settle
// are correct ONLY when cloud-api runs as a SINGLE replica — two pods each reserve
// against their own cached Commerce balance and would double-spend. The deployment
// enforces this with strategy=Recreate and one replica (universe
// charts/app/values/hanzo/cloud.yaml); SinglePodReplicaHint lets the process ALSO
// assert it at boot. The paid lane's plan holds (controllers' lane.go) are in-pod
// too and rest on the same invariant. Scaling out REQUIRES a Commerce-atomic
// conditional reserve (move Reserve/Settle behind a Commerce endpoint) — until
// then, do not raise replicas. See LLM.md "Balance ledger — single-pod invariant".
var GlobalBalanceLedger = NewBalanceLedger(BalanceLedgerTTL)
// SinglePodReplicaHint parses the CLOUD_REPLICAS deployment hint, the replica count
// the cloud chart injects into every pod it deploys (writers.replicasEnv). ok=false
// when the hint is unset or unparseable (the invariant is then logged, not enforced). A
// returned count > 1 means the in-pod ledger is unsafe and the process must refuse
// to start — the boot path (bootstrap.go) treats >1 as fatal.
func SinglePodReplicaHint() (count int, ok bool) {
raw := strings.TrimSpace(os.Getenv("CLOUD_REPLICAS"))
if raw == "" {
return 0, false
}
n, err := strconv.Atoi(raw)
if err != nil {
return 0, false
}
return n, true
}
// NewBalanceLedger creates an empty ledger with the given freshness TTL.
func NewBalanceLedger(ttl time.Duration) *BalanceLedger {
return &BalanceLedger{entries: make(map[string]*ledgerEntry), ttl: ttl}
}
// SetBalance records a freshly-fetched Commerce balance for a subject and resets
// the freshness clock. Outstanding reservations are PRESERVED — a fresh read of
// the upstream balance does not cancel in-flight holds.
func (l *BalanceLedger) SetBalance(subject string, cents int64) {
if subject == "" {
return
}
l.mu.Lock()
defer l.mu.Unlock()
e := l.entries[subject]
if e == nil {
e = &ledgerEntry{}
l.entries[subject] = e
}
e.balanceCents = cents
// A fresh balance has had every posted debit taken off it, the sub-cent ones
// included, so the local remainder starts again.
e.spentNano = 0
e.fetchedAt = time.Now()
}
// Snapshot returns the cached balance, outstanding reservations, whether the
// cached balance is fresh (within TTL), and whether the subject is known.
func (l *BalanceLedger) Snapshot(subject string) (balance, reserved int64, fresh, known bool) {
l.mu.Lock()
defer l.mu.Unlock()
e := l.entries[subject]
if e == nil {
return 0, 0, false, false
}
return e.balanceCents, e.reservedCents, time.Since(e.fetchedAt) <= l.ttl, true
}
// Available returns balance - reserved for a subject, and whether the subject is
// known. A cold subject returns (0, false).
func (l *BalanceLedger) Available(subject string) (cents int64, known bool) {
l.mu.Lock()
defer l.mu.Unlock()
e := l.entries[subject]
if e == nil {
return 0, false
}
return e.available(), true
}
// Reserve atomically holds estCents against the subject's available balance.
// It records the hold and returns true ONLY if the subject is known AND
// available (balance - reserved) >= estCents. A cold subject (no cached balance)
// cannot be reserved against — the caller must SetBalance first (fetch from
// Commerce), so the gate never reserves blind. estCents <= 0 holds nothing but
// still requires a known, non-overdrawn balance.
func (l *BalanceLedger) Reserve(subject string, estCents int64) bool {
if estCents < 0 {
estCents = 0
}
l.mu.Lock()
defer l.mu.Unlock()
e := l.entries[subject]
if e == nil {
return false
}
if e.available() < estCents {
return false
}
e.reservedCents += estCents
return true
}
// EvictIdle removes entries whose balance is older than maxIdle AND have no
// outstanding reservations, to bound memory. An entry with live holds is never
// evicted (that would lose an in-flight reservation).
func (l *BalanceLedger) EvictIdle(maxIdle time.Duration) {
l.mu.Lock()
defer l.mu.Unlock()
now := time.Now()
for k, e := range l.entries {
if e.reservedCents == 0 && now.Sub(e.fetchedAt) > maxIdle {
delete(l.entries, k)
}
}
}
// Settle releases a prior reservation of estCents and applies the actual spend
// locally (balance -= actualCents) so subsequent reads reflect it immediately,
// without waiting for the async Commerce refresh. Pass the SAME estCents used at
// Reserve time. actualCents may be 0 (a failed/empty request that still reserved).
// Reserved never goes below zero (defensive against a double-settle).
func (l *BalanceLedger) Settle(subject string, estCents, actualCents int64) {
if actualCents < 0 {
actualCents = 0
}
l.SettleNano(subject, estCents, actualCents*nanoPerCent)
}
// SettleNano is Settle with the actual spend in nano-USD, for a call priced below
// a cent. Whole cents come off the balance; the remainder is carried until it
// makes one, and counts against what is available meanwhile.
func (l *BalanceLedger) SettleNano(subject string, estCents, actualNano int64) {
if estCents < 0 {
estCents = 0
}
if actualNano < 0 {
actualNano = 0
}
l.mu.Lock()
defer l.mu.Unlock()
e := l.entries[subject]
if e == nil {
return
}
e.reservedCents -= estCents
if e.reservedCents < 0 {
e.reservedCents = 0
}
e.spentNano += actualNano
whole := e.spentNano / nanoPerCent
e.balanceCents -= whole
e.spentNano -= whole * nanoPerCent
}