Files
zeekay 3327c6d3b5 decisions: hold an LLM answer at what it can cost, and bill no more
The hold read a body at four bytes a token, but a chat model's tokenizer
reads a digit, a hex character or an emoji's byte as one, and the decision
service writes the body out again before the router counts it (1e15 becomes
1000000000000000.0): a 65 KB state the hold priced at 7 cents settled at 56,
and a wallet went negative. A model whose output is priced is now held at a
token a byte plus 1,024 for the prompt around it, and an answer is settled
and debited at most its hold whoever pays, a plan included; the excess is
ours, and logged. Until the service passes the state's bytes through, a
number-heavy body can still cost past its hold, about $1.76 a call at worst.

A token count is the service's report of what a router relayed, so each is
bounded where it is priced: none below zero, output at the 4,096 a completion
is asked for, input short of where count times price overflows. A reported
1e9 output tokens billed $7,260 on a 3-cent hold, and 1.3e15 wrapped to a
negative debit.

An answer whose usage does not read as two whole token counts (and a cost,
where stated, a number of 0 or more) is refused 502 and billed nothing,
whatever model answered, rather than billed at zero for the tokens it did not
state. The service writes usage from integer fields, so only a broken service
reaches it, and the loss is the one any refusal already accepts.

Kai's and Jev's bills and holds are unchanged for every count: their output
is priced at zero and the byte-rate hold and the cap apply to priced output
alone.
2026-10-11 10:27:00 -07:00
..