Files
aml/pkg/api/identity_test.go
zeekayandHanzo Dev 7de786224f planes: six ways a monitoring plane could be turned against the institution running it
Each of these is a control that worked and could be made not to, by a caller
holding an ordinary token for its own tenant. Every one carries a test, and every
one carries a mutation in tools/mutate.py, because a bound with no test is a bound
that comes back.

THE EXPENSIVE PLANE COULD TAKE THE MACHINE. A model search replays a tenant's
history through up to MaxTrials detectors and a fit replays it through one; both
were registered plain. Four bounds now, each doing one thing and nothing else:
one study per tenant (api.gate), a deadline derived from the request's own
context so a client that goes away cancels the work (api.within, 2 min), a
process budget whose tokens ARE the trial workers and are taken BEFORE the
history is read (topology.Budget, half the cores), and a clamp on the width one
request may name (MaxWorkers). The gate is a map keyed by the tenant with nothing
ever evicted from it, so one institution's study is never another's refusal; the
budget is the one bound that is not per tenant, deliberately, because the CPU is
shared however it is modelled and waiting for the machine is not the same as
being refused because of somebody else. A rule replay is admitted the same way.
Fit now records what it cost, as a run already did.

THE DECIDER WAS FREE TEXT. Every plane asks a governed act for a reason and a
decider so a supervisor can be told who turned a control off, and the answer was
whatever the caller typed. types.Decider is a distinct type carrying `json:"-"`,
so no wire format has a name for one, and pkg/api/typed.go writes the
AUTHENTICATED SUBJECT onto every field of that type after the body and after the
path. Identity now yields a Caller — two projections of one verified credential,
the tenant that indexes every store and the subject that names a decision, both
from the same signature. A credential naming nobody still authenticates, because
reads and ingest are the tenant's work and not a person's, and refuses every
operation that would record a decision.

A TENANT COULD STOP ITS OWN PAYMENTS, AND THE RETRY DOUBLE-COUNTED. suppress.Cover
runs once per activation on the ingest path and failed closed past
MaxCandidates, which h.monitor turned into a 503 on every transaction. It
degrades now: the answer is over what was read and says so (Cover.Partial,
Activation.Unchecked), and the crowding is refused at DECLARATION instead
(MaxInForce), where a refusal costs one operator request rather than every
payment. The retry that followed such a 503 wrote a second activation for one
firing, which counted in the streak, so a declared repetition policy fired on a
repeat that never happened — a payment blocked because the client retried. An
activation naming a transaction is now identified by (tenant, transaction, rule,
subject), hashed so no customer value is in a key. The retained record is
idempotent by the same reasoning: token.Vault.Fingerprint names the plaintext
deterministically, since a seal draws a fresh nonce and sealed bytes never match;
and the transaction now carries ONE clock, its own. The reception clock ingest
stamped made every retry a different fact under one id, which no retry could ever
clear. A genuine conflict answers 409 rather than 503, because 503 reads as
"retry" and a client retrying a permanent refusal retries forever.

A RUNG'S COUNT WAS THE LIMIT OF AN UNBOUNDED READ. It becomes a store LIMIT on
the ingest path, so it is bounded at declaration (MaxCount, MaxWithin) and
clamped again where it is used, because a bound introduced after a row was
written does not reach that row.

THE CATALOG STORED PAYLOAD VALUES AFTER ALL. A minimum IS a value, exactly, at
any volume, and at a count of one so are the sum and the mean. Declared numbers
are the transaction model's own and keep their moments; a custom number is
whatever the institution put under its own key and gets the sketch and nothing
else, absent rather than zero. And the vocabulary itself is now bounded PER
TENANT, in the accumulator and in the rows alike: MaxKeys bounds one payload used
as a bag, it does not bound a tenant sending a different key on every
transaction, and that growth was in the memory every other institution's ingest
runs in. Reaching the bound degrades only the tenant that reached it, refuses no
payload, and publishes what it turned away.

AN ADOPTED CONTROL COULD GO QUIET WITH NOBODY DECIDING IT SHOULD. The fit is
durable and the adoption is durable, but what adoption DOES is install learned
state into a model in memory — and this is one replica with a Recreate strategy,
so a rollout empties it, and the store holds a bounded number of tenants' models,
so an idle institution's can be dropped for a busy one's. Either way the tenant
returns to warming, which reports no alerts, which is what a quiet institution
also reports. The model now asks the model plane what this tenant last adopted
when it plants a model for it, and State publishes when the model started and
whether it started from an adoption, so a reset is visible instead of inferred
from a silence.

Co-authored-by: Hanzo Dev <[email protected]>
2026-08-01 16:37:20 -07:00

234 lines
8.5 KiB
Go

package api
import (
"errors"
"net/http"
"net/http/httptest"
"testing"
"github.com/hanzoai/base/core"
"github.com/luxfi/aml/pkg/types"
)
// event builds a request event a handler can be called with directly.
func event(method, target string, headers map[string]string) (*core.RequestEvent, *httptest.ResponseRecorder) {
req := httptest.NewRequest(method, target, nil)
for k, v := range headers {
req.Header.Set(k, v)
}
rec := httptest.NewRecorder()
e := &core.RequestEvent{}
e.Request = req
e.Response = rec
return e, rec
}
// handlers is every route that reads tenant-scoped data, so a new route cannot be
// added without deciding what it does about the tenant.
func (h *Handler) readers() map[string]func(*core.RequestEvent) error {
return map[string]func(*core.RequestEvent) error{
"getAlerts": h.getAlerts(),
"listCases": h.listCases(),
"testRule": h.testRule(),
"searchRelationships": h.searchRelationships(),
"addCaseEvent": h.addCaseEvent(),
"resolveCase": h.resolveCase(),
"openRelationship": h.openRelationship(),
"closeRelationship": h.closeRelationship(),
"ingestTransaction": h.ingestTransaction(),
}
}
// With no Identity wired, every tenant-scoped route must refuse. The failure this
// prevents is a deployment that serves data because nobody configured who is
// allowed to ask for it.
func TestWithoutIdentityEveryRouteRefuses(t *testing.T) {
h := &Handler{Alerts: NewAlertStore()}
for name, handle := range h.readers() {
e, rec := event(http.MethodGet, "/v1/aml/x", map[string]string{"X-Org-Id": "victim"})
if err := handle(e); err != nil {
t.Fatalf("%s returned a transport error: %v", name, err)
}
if rec.Code != http.StatusUnauthorized {
t.Errorf("%s served with no Identity configured: status %d, want 401", name, rec.Code)
}
}
}
// A caller-supplied tenant header must not be honoured on its own. This is the
// exact request that read another tenant's alerts: no credential, a header naming
// somebody else's org.
func TestForgedTenantHeaderIsRefused(t *testing.T) {
alerts := NewAlertStore()
alerts.Add("tx-1", []types.Alert{{ID: "a1", OrgID: "hanzo/victim"}})
// An Identity that authenticates nobody — the shape of an unauthenticated
// request reaching a service that requires one.
h := &Handler{
Alerts: alerts,
Identity: func(*http.Request) (Caller, error) {
return Caller{}, errors.New("no credential presented")
},
}
e, rec := event(http.MethodGet, "/v1/aml/transactions/tx-1/alerts", map[string]string{"X-Org-Id": "victim"})
if err := h.getAlerts()(e); err != nil {
t.Fatalf("transport error: %v", err)
}
if rec.Code != http.StatusUnauthorized {
t.Fatalf("status %d, want 401", rec.Code)
}
if body := rec.Body.String(); body != "" && rec.Code == http.StatusOK {
t.Fatalf("unauthenticated request received a body: %s", body)
}
}
// An authenticated tenant must see only its own alerts, even knowing the exact
// transaction id of another tenant's. A transaction id is not a secret.
func TestAuthenticatedTenantCannotReadAnotherTenant(t *testing.T) {
alerts := NewAlertStore()
alerts.Add("tx-1", []types.Alert{{ID: "victim-alert", OrgID: "hanzo/victim"}})
alerts.Add("tx-1", []types.Alert{{ID: "attacker-alert", OrgID: "hanzo/attacker"}})
// Authenticated as "attacker", asking for a transaction it shares an id with.
h := &Handler{
Alerts: alerts,
Identity: func(*http.Request) (Caller, error) { return Caller{Tenant: "hanzo/attacker"}, nil },
}
e, rec := event(http.MethodGet, "/v1/aml/transactions/tx-1/alerts",
map[string]string{"X-Org-Id": "victim"}) // header says victim; identity says attacker
e.Request.SetPathValue("id", "tx-1")
if err := h.getAlerts()(e); err != nil {
t.Fatalf("transport error: %v", err)
}
if rec.Code != http.StatusOK {
t.Fatalf("status %d, want 200", rec.Code)
}
body := rec.Body.String()
if contains(body, "victim-alert") {
t.Fatalf("cross-tenant read: attacker received victim's alert. body=%s", body)
}
if !contains(body, "attacker-alert") {
t.Fatalf("tenant did not receive its own alert. body=%s", body)
}
}
// The identity, not the header, decides the tenant. If the header could override
// it, authenticating would not bound anything.
func TestHeaderDoesNotOverrideIdentity(t *testing.T) {
h := &Handler{Identity: func(*http.Request) (Caller, error) { return Caller{Tenant: "hanzo/real"}, nil }}
e, _ := event(http.MethodGet, "/v1/aml/x", map[string]string{"X-Org-Id": "spoofed"})
got, err := h.tenant(e)
if err != nil {
t.Fatalf("tenant: %v", err)
}
if got != "hanzo/real" {
t.Fatalf("tenant = %q, want %q — the header must not decide the tenant", got, "hanzo/real")
}
}
// An Identity that resolves to anything but a brand-qualified tenant is a failure,
// not a tenant whose name happens to be odd. An empty org would scope every read to
// nothing, or to everything, depending on the store; a bare org would put two
// brands' institutions of the same name in one tenant and one vault.
//
// This is the boundary the value crosses into the store index, the history column
// and the vault salt, and it is checked here rather than trusted from the Identity,
// because the Identity is supplied by the deployment.
func TestUnqualifiedTenantIsRefused(t *testing.T) {
for _, resolved := range []string{
"", // nothing at all
" ", // whitespace
"acme", // a bare org: the RED-1 shape
"/acme", // no brand
"hanzo/", // no org
"nobody/acme", // a brand no registry row claims
"hanzo/a/b", // an org carrying the separator
"hanzo / acme", // padding around the separator
" hanzo/acme", // padding at the front
"HANZO/acme", // a brand id that is not canonical
"hanzo/acme/", // a trailing separator
"hanzo\\acme", // the wrong separator
"hanzo:acme", // another wrong separator
"hanzo/ acme ", // padding around the org
} {
h := &Handler{Identity: func(*http.Request) (Caller, error) { return Caller{Tenant: resolved}, nil }}
e, _ := event(http.MethodGet, "/v1/aml/x", nil)
if got, err := h.tenant(e); err == nil {
t.Errorf("an identity resolving %q was accepted as tenant %q", resolved, got)
}
}
// And the canonical form is accepted, so the rows above fail for their own
// reason and not because everything fails.
h := &Handler{Identity: func(*http.Request) (Caller, error) { return Caller{Tenant: "hanzo/acme"}, nil }}
e, _ := event(http.MethodGet, "/v1/aml/x", nil)
if got, err := h.tenant(e); err != nil || got != "hanzo/acme" {
t.Fatalf("tenant = %q, %v; want hanzo/acme", got, err)
}
}
// The proxy header names the ORG. The brand comes from the request's own Host, so
// the two Identity implementations produce the same key and a deployment cannot
// take the brand from a header a client can write.
func TestTrustedProxyHeaderQualifiesWithTheHost(t *testing.T) {
id := TrustedProxyHeader("X-Org-Id", "X-User-Id")
proxied := func(host, org string) *http.Request {
r := httptest.NewRequest(http.MethodGet, "/", nil)
r.Host = host
if org != "" {
r.Header.Set("X-Org-Id", org)
}
return r
}
for _, tc := range []struct{ host, org, want string }{
{"api.hanzo.ai", " acme ", "hanzo/acme"},
{"api.lux.network", "acme", "lux/acme"},
{"console.zoo.cloud", "acme", "zoo/acme"},
} {
who, err := id(proxied(tc.host, tc.org))
got := who.Tenant
if err != nil {
t.Fatalf("%s: %v", tc.host, err)
}
if got != tc.want {
t.Errorf("%s + %q = %q, want %q", tc.host, tc.org, got, tc.want)
}
}
// No header is no tenant; and no brand on the Host is no tenant space to put an
// org in, however trusted the header is. An in-cluster caller that reaches this
// service directly arrives on exactly those hosts.
for _, tc := range []struct{ name, host, org string }{
{"no header", "api.hanzo.ai", ""},
{"blank header", "api.hanzo.ai", " "},
{"an org carrying the separator", "api.hanzo.ai", "lux/acme"},
{"a pod IP", "10.42.0.7:8090", "acme"},
{"a service name", "aml.aml.svc.cluster.local", "acme"},
{"localhost", "localhost:8090", "acme"},
{"no host at all", "", "acme"},
{"a lookalike domain", "a.zoo.ngo.attacker.example", "acme"},
} {
if who, err := id(proxied(tc.host, tc.org)); err == nil {
t.Errorf("%s resolved to tenant %q", tc.name, who.Tenant)
}
}
}
func contains(haystack, needle string) bool {
return len(haystack) >= len(needle) && indexOf(haystack, needle) >= 0
}
func indexOf(haystack, needle string) int {
for i := 0; i+len(needle) <= len(haystack); i++ {
if haystack[i:i+len(needle)] == needle {
return i
}
}
return -1
}