Files
zeekay 74bd659492 billing: a program in this process is told what its call cost
A call cloud makes for a customer over the plane is covered with a
PaysCaller grant and debits no wallet here, because cloud's meter
charges the customer. It charged its own flat rate, $2 a million
tokens for any model, so an in-process enso-ultra call was billed
below what it cost to serve and an enso-free call was billed at all.

The stated caller now carries a Bill. The caller-paid grant hands it
on, and recordUsage adds each answer's billed amount to it: what the
same answer debits any other caller. The program charges that and
has no price of its own. PriceOf exposes the billed row as a Rate
for a program that must reserve before it calls; a model nobody
priced answers false rather than the conservative default.
2026-10-18 21:46:32 -07:00

92 lines
3.1 KiB
Go

// Copyright 2026 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package object
import (
"context"
"strings"
"sync/atomic"
"github.com/valyala/fasthttp"
)
// Caller is the customer a program in this process makes a model call for: a
// sibling app's embeddings, rerank or completion bought over the host's plane. Org
// is whom the call is for; Person ("<org>/<name>") and Project where it names them.
//
// Only code in this process states one (ai.For): it rides the request as a value the
// network cannot write, so nothing a request carries can claim it. The program that
// states it meters the call itself and charges the customer for it, so the call is
// attributed to Org and debits no wallet here (PaysCaller): the customer is charged
// once, by the program, and the deployment's wallet not at all.
//
// Bill is where the program learns what to charge: every answer the call buys is
// priced here exactly as it would be for any other caller, and the price is told to
// Bill instead of being debited. A program that states no Bill is told nothing.
type Caller struct {
Org string
Person string
Project string
Bill *Bill
}
// Bill is what a program's call cost, in nano-USD: the sum of what each answer it
// bought would have debited any other caller — a hedge's losing provider included,
// since that is billed to every caller too. It is told, never estimated: a call
// that filed no usage leaves it untold, and the program can tell the difference.
type Bill struct {
nanos atomic.Int64
told atomic.Bool
}
// Add tells the bill one answer's price.
func (b *Bill) Add(nanos int64) {
b.nanos.Add(nanos)
b.told.Store(true)
}
// Nanos is the call's price so far, and whether any answer told one.
func (b *Bill) Nanos() (int64, bool) {
return b.nanos.Load(), b.told.Load()
}
type callerKey struct{}
// SetCaller states c on a request this process is about to serve.
func SetCaller(fctx *fasthttp.RequestCtx, c Caller) {
fctx.SetUserValue(callerKey{}, c)
}
// RequestCaller is the caller stated on fctx, if a program in this process stated one
// with an org.
func RequestCaller(fctx *fasthttp.RequestCtx) (Caller, bool) {
c, ok := fctx.UserValue(callerKey{}).(Caller)
return c, ok && strings.TrimSpace(c.Org) != ""
}
// WithCaller is ctx carrying c.
func WithCaller(ctx context.Context, c Caller) context.Context {
return context.WithValue(ctx, callerKey{}, c)
}
// CallerOf is the caller ctx carries, if any.
func CallerOf(ctx context.Context) (Caller, bool) {
if ctx == nil {
return Caller{}, false
}
c, ok := ctx.Value(callerKey{}).(Caller)
return c, ok
}