Files
zeekay 531e9f423a lane: no platform ceiling on the paid lane; a payer spends what they bought
The paid lane capped every paid-lane call together at PAID_LANE_DAILY ($10 a UTC
day by default) and each org at PAID_LANE_ORG_SHARE (25% of it), and counted
wallet-paid calls against both: a customer with $1,000 prepaid was held to $2.50 a
day, and every payer together to $10. The rule is no ceiling but what a payer
prepays, or their usage. Free users get free models; payers spend their own money.

Deleted: the platform day and the org share (paidLaneDaily, orgShare, the day,
spent and per-org accounting in laneBook), the paid_day store (object.PaidDay, its
table registration, the writer, flush, drain, load and turn), Settled's drain of
it, laneSpend, PaidLaneOpen (the gate reads !FreeOnly()), and the refusals only the
day produced ("closed", "after 00:00 UTC"). paid_lane_ceiling named a platform
ceiling that no longer exists: a payer the lane has no room for is told
paid_lane_full, the code its refusal already carried, in X-Hanzo-Lane-Reason and
X-Hanzo-Usage-Reason. Stores keep an orphan paid_day table nothing reads.

Kept: the wallet hold (reserveFor), the plan class's LimitGrant.Spend less the
payer's calls in flight, quote sizing against it, 8 seats a payer, lapse and
stream-idle give-back, race losers holding the seat, BYO, and the free-model
hand-off for a payer with no room.

The replica guard. The lane was closed unless CLOUD_API_REPLICAS=1 because the day
lived in this process. No deployment sets CLOUD_API_REPLICAS: the cloud chart
injects CLOUD_REPLICAS (1 in production). So the boot assertion meant to stop a
second replica double-spending the in-pod wallet ledger never fired, and the lane
guard was the one live check; it also kept the paid lane closed in production. The
plan holds that remain are in-pod exactly as the wallet ledger is, so they take its
guard: SinglePodReplicaHint reads CLOUD_REPLICAS, and bootstrap refuses to start
when the deployment states more than one replica. A declared second replica stops
the process instead of closing the lane, and no switch is left whose only purpose
was the day.
2026-10-09 14:08:38 -07:00

137 lines
5.2 KiB
Go

// Copyright 2023-2025 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package controllers
import (
"context"
"encoding/json"
"strings"
"time"
iam "github.com/hanzoai/ai/internal/iam"
)
// fastWidth is how many providers a fast request is offered to at once.
//
// TWO, not three. The tail this cuts is one provider drawing a slow response
// while another would have answered normally, and a second opinion removes most
// of it; a third removes a little more for another whole completion. The cost is
// linear in this number and the benefit is not, so it starts where the trade is
// clearly worth it.
const fastWidth = 2
// fastFlag is the ask as the BODY carries it, for the same reason retrieval is:
// a browser preflights custom headers and the edge's CORS allow-list names only
// the standard ones, so a public page cannot send X-Fast. The header form stays
// for server-side callers; either spelling works.
type fastFlag struct {
Fast bool `json:"fast"`
}
// wantsFast reports whether this request asked for fast mode.
//
// Asking is not getting. What it costs is settled where the money is —
// widthFor — because the answer depends on whether the caller can be billed for
// what the extra attempts spend.
func (c *ApiController) wantsFast() bool {
if v := c.Header("X-Fast"); v != "" {
return v == "1" || strings.EqualFold(v, "true")
}
var f fastFlag
_ = json.Unmarshal(c.Body(), &f)
return f.Fast
}
// widthFor answers how many providers this request may be offered to at once,
// and reserves for all of them.
//
// FAST MODE IS AN OPTIMISATION, NOT AN ENTITLEMENT. A caller who asks for it and
// cannot cover N completions gets the ordinary one rather than a refusal: they
// asked to go faster, not to be turned away, and refusing here would invent a
// second way for a funded account to be told no.
//
// A caller nobody can be billed for never gets it at all. The public lane has no
// reservation to make, so hedging there would spend N completions on a stranger
// and settle none.
func (c *ApiController) widthFor(user *iam.User, subject string, est int64) (*budgetHold, int, bool) {
if user != nil && c.wantsFast() {
if hold, ok := reserveFor(c.Context(), subject, est*fastWidth); ok {
return hold, fastWidth, true
}
}
hold, ok := reserveFor(c.Context(), subject, est)
return hold, 1, ok
}
// billRaced is what fan.bill calls for each provider beaten in a race: one real
// usage row for real tokens, because a cancelled attempt was charged to us by
// the vendor whether or not its answer was ever read.
//
// EVERY REQUEST FIELD IS READ NOW, not inside the closure. By the time this
// fires the handler has returned and fasthttp has recycled the request, so
// reading the IP there reads whatever the next caller put in its place. The
// context is Background for the same reason: the request's own is cancelled the
// moment the response ends, which is before most losers have finished.
//
// A request on the paid lane is the exception: its loser is billed to whoever the
// request was — the request's values without its cancellation, its grant among them
// — and what it spent is added to the day on the request's seat (seat.lost), which
// holds until every loser is counted. A loser never gives the seat back: only the
// answer's own usage does.
func (c *ApiController) billRaced(model string, user *iam.User, premium, stream bool,
requestId string, start time.Time,
) func(attempt) {
if user == nil {
return nil
}
owner, ip := c.billingOrg(user), c.Fiber().IP()
place, ctx := seatOf(c.Context()), billing(c.Context())
if ctx == nil {
ctx = context.Background()
}
return func(a attempt) {
rec := &usageRecord{
Owner: owner,
Organization: user.Owner,
Model: model,
Provider: a.provider,
Origin: a.origin,
PromptTokens: a.prompt,
CompletionTokens: a.completion,
TotalTokens: a.prompt + a.completion,
Currency: "USD",
Premium: premium,
Stream: stream,
Status: "hedged",
ErrorMsg: a.err.Error(),
ClientIP: ip,
RequestID: requestId,
}
rec.bind(ctx, user)
// A loser is counted on the seat by place.lost, never settled by recordUsage.
rec.seat = nil
// Whose key paid decides what this costs, and for a beaten provider that
// is ITS row — not the one auth resolved before the race started.
rec.BYO, rec.Account = providerBYO(a.row, user)
// A vendor that refused before producing anything spent nothing, and the
// row still goes to the trace so the refusal is visible.
if a.prompt > 0 || a.completion > 0 {
recordUsage(rec)
}
place.lost()
recordTrace(ctx, rec, start)
}
}