mirror of
https://github.com/luxfi/aml.git
synced 2026-09-24 00:25:20 +00:00
main
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  zeekayandHanzo Dev
|
1b7a28d31f |
planes: one assembly, one identity for an offer, and a bound nobody can spend twice
The defining failure of the last review was not in a plane. It was that there were TWO wirings: cmd/amld assembled a Handler by hand and the tests assembled a different one, so the record fingerprint could be a struct field the suite exercised and no column stored — every retry a permanent 409 in production while every test passed. api.Wire is now the only assembly. It creates every collection, opens every shelf, makes every join and registers what has to run on a cadence; cmd/amld supplies only what an installation answers differently, the tests call the same function, and internal/source.NoLiteral reads cmd/amld to refuse a second copy. Handler completeness is checked by reflection, because a hand-written field list would be the third copy. A retry is not a second transaction. pkg/receipt resolves the identity of an OFFER once, at the top of ingest, before anything is read or written, and returns the prior answer byte for byte — so the ledger, the aggregates, the history, the alerts and the case plane all agree on one transaction instead of the ledger holding one record while the aggregates hold three. Two different facts under one reference is 409 and never 503. Because the answer is kept AFTER the work, a process that dies in between leaves no receipt, so every plane past it also recognises the transaction for itself: history does not append a second row, alerts keep the first judgement, and a judged transaction opens no second case. A bound over a COUNT of caller-sized values is not a bound. types.MaxIdent bounds every identifier at the door, which is what turns the counts downstream into figures in bytes; dictionary.MaxName does the same for a payload key, and dictionary.Ceiling is the product a test WEIGHS a full accumulator against. The field catalog's accumulator moves into roster, which admits and never removes, so per-tenant bounds multiply by ONE number an operator can hold against a pod. engine.Evaluator now keeps no compiled rules at all. A candidate's text arrives on the wire at /v1/aml/rules/test, so a compiled-rule table there was keyed by a string a caller wrote, sized by how many distinct ones a caller sends, with no cap and no removal, in the memory every institution's ingest runs in — an OOM from one authenticated caller, which on this deployment is every control off at once. A rule's compiled form is a value with an owner: the installed library, or one replay. Nothing shared, nothing to grow. Found here, not reported. Handler.history is the one function that materialises a whole retained history, so it now REQUIRES a *topology.Grant, and a Grant comes from Budget.Admit and nowhere else. An ungated whole-history read is something a caller has to write on purpose rather than something a new door can forget. The rule replay and the activation rate fold take the same one budget the model studies take. Reaching a bound is allowed; reaching it quietly is not. GET /v1/aml/load is the one door for "is anything of mine quietly degraded": the tenant's aggregates against its own ceiling in bytes with a grade, whether it has a behavioural model at all, the field catalog's pressure, and the process's counts of institutions held and refused. It names no other institution. Closing R1-R8 from the review plus R11, the two recurring classes, and the silent degradation the load door answers. Twenty-four mutations, twenty-four killed. Co-authored-by: Hanzo Dev <[email protected]> |
||
|
zeekayandHanzo Dev
|
7de786224f |
planes: six ways a monitoring plane could be turned against the institution running it
Each of these is a control that worked and could be made not to, by a caller holding an ordinary token for its own tenant. Every one carries a test, and every one carries a mutation in tools/mutate.py, because a bound with no test is a bound that comes back. THE EXPENSIVE PLANE COULD TAKE THE MACHINE. A model search replays a tenant's history through up to MaxTrials detectors and a fit replays it through one; both were registered plain. Four bounds now, each doing one thing and nothing else: one study per tenant (api.gate), a deadline derived from the request's own context so a client that goes away cancels the work (api.within, 2 min), a process budget whose tokens ARE the trial workers and are taken BEFORE the history is read (topology.Budget, half the cores), and a clamp on the width one request may name (MaxWorkers). The gate is a map keyed by the tenant with nothing ever evicted from it, so one institution's study is never another's refusal; the budget is the one bound that is not per tenant, deliberately, because the CPU is shared however it is modelled and waiting for the machine is not the same as being refused because of somebody else. A rule replay is admitted the same way. Fit now records what it cost, as a run already did. THE DECIDER WAS FREE TEXT. Every plane asks a governed act for a reason and a decider so a supervisor can be told who turned a control off, and the answer was whatever the caller typed. types.Decider is a distinct type carrying `json:"-"`, so no wire format has a name for one, and pkg/api/typed.go writes the AUTHENTICATED SUBJECT onto every field of that type after the body and after the path. Identity now yields a Caller — two projections of one verified credential, the tenant that indexes every store and the subject that names a decision, both from the same signature. A credential naming nobody still authenticates, because reads and ingest are the tenant's work and not a person's, and refuses every operation that would record a decision. A TENANT COULD STOP ITS OWN PAYMENTS, AND THE RETRY DOUBLE-COUNTED. suppress.Cover runs once per activation on the ingest path and failed closed past MaxCandidates, which h.monitor turned into a 503 on every transaction. It degrades now: the answer is over what was read and says so (Cover.Partial, Activation.Unchecked), and the crowding is refused at DECLARATION instead (MaxInForce), where a refusal costs one operator request rather than every payment. The retry that followed such a 503 wrote a second activation for one firing, which counted in the streak, so a declared repetition policy fired on a repeat that never happened — a payment blocked because the client retried. An activation naming a transaction is now identified by (tenant, transaction, rule, subject), hashed so no customer value is in a key. The retained record is idempotent by the same reasoning: token.Vault.Fingerprint names the plaintext deterministically, since a seal draws a fresh nonce and sealed bytes never match; and the transaction now carries ONE clock, its own. The reception clock ingest stamped made every retry a different fact under one id, which no retry could ever clear. A genuine conflict answers 409 rather than 503, because 503 reads as "retry" and a client retrying a permanent refusal retries forever. A RUNG'S COUNT WAS THE LIMIT OF AN UNBOUNDED READ. It becomes a store LIMIT on the ingest path, so it is bounded at declaration (MaxCount, MaxWithin) and clamped again where it is used, because a bound introduced after a row was written does not reach that row. THE CATALOG STORED PAYLOAD VALUES AFTER ALL. A minimum IS a value, exactly, at any volume, and at a count of one so are the sum and the mean. Declared numbers are the transaction model's own and keep their moments; a custom number is whatever the institution put under its own key and gets the sketch and nothing else, absent rather than zero. And the vocabulary itself is now bounded PER TENANT, in the accumulator and in the rows alike: MaxKeys bounds one payload used as a bag, it does not bound a tenant sending a different key on every transaction, and that growth was in the memory every other institution's ingest runs in. Reaching the bound degrades only the tenant that reached it, refuses no payload, and publishes what it turned away. AN ADOPTED CONTROL COULD GO QUIET WITH NOBODY DECIDING IT SHOULD. The fit is durable and the adoption is durable, but what adoption DOES is install learned state into a model in memory — and this is one replica with a Recreate strategy, so a rollout empties it, and the store holds a bounded number of tenants' models, so an idle institution's can be dropped for a busy one's. Either way the tenant returns to warming, which reports no alerts, which is what a quiet institution also reports. The model now asks the model plane what this tenant last adopted when it plants a model for it, and State publishes when the model started and whether it started from an adoption, so a reset is visible instead of inferred from a silence. Co-authored-by: Hanzo Dev <[email protected]> |
||
|
zeekayandHanzo Dev
|
e144a68a5b |
cases, jwks, docs: close Red's three low findings before the tag
Red's re-review passed (0 critical/high/medium) but left three low findings, and two are "one and only one way" violations worth closing before a version tag. LOW-2 — one way to mutate a case, org-checked. There were two ways to append a case event: an org-checked AddEventFor with no callers, and an unchecked AddEvent the handler actually used, safe only because the handler happened to resolve the tenant first. That is a cross-tenant write one refactor away. AddEvent now takes the tenant and checks it (absorbing AddEventFor, which is deleted), and Resolve and UpdateStatus do the same — there is no unchecked public path left to reach for. A test proves each refuses a caller from another tenant holding the exact case id. LOW-1 — the tenant-key invariant was false for Rule.OrgID. The types doc said every OrgID holds the brand-qualified tenant key, but the rule library is the deployment's catalog, seeded at startup from AML_DEFAULT_ORG with no Host in scope, so its OrgID is a bare org and nothing scopes on it. The invariant now names that one exception, on the field where an auditor reads it, so it is true rather than aspirational. LOW-3 — a duplicate kid in a JWKS was overwritten last-wins and silently, against this package's own "left out loudly" principle: a token the shadowed key signed would be refused with nothing logged. It is now first-wins and announced. This is unreachable from the real IAM (which dedupes by cert name) and is a guard against a malformed set. Also documented, not changed: the aud pin does not prove a token transited this app's login gate — IAM's RFC 8693 token-exchange path can stamp a chosen aud, and is gated instead by the IAM_KEY_MINT_ALLOWED_APPS allowlist. Named in iam.go as a deploy-time trust dependency rather than left as an implied guarantee. 18 packages green, race clean, gofmt clean. The case org-check is mutation-killed. Co-authored-by: Hanzo Dev <[email protected]> |
||
|
zeekayandHanzo Dev
|
11a44f6010 |
tenant: a tenant is a brand and an org, and the vault is keyed by both
The tenant key was the bare org, and an org name is unique within an issuer
rather than across issuers. `acme` on lux.id and `acme` on zoolabs.id are two
unrelated financial institutions, and every store keyed them as one — including
the tokenisation vault, where the org WAS the HKDF salt (pkg/token New). Two
brands therefore derived the same keys: one institution's customer names
tokenised to the other's pseudonyms, and its sealed records opened under the
other's tenant. That is a plaintext cross-brand disclosure of retained personal
data, against the purpose limitation retention exists under (Directive (EU)
2015/849 Art. 41(2)) and against HIP-0302. It is fixed now, before any record
exists, because doing it later re-keys every vault.
The tenant is now `<brand>/<org>`, minted in one place (api/tenant.go qualify)
and checked where it crosses into the store, the history column and the salt.
Both Identity implementations produce the same key: the brand comes from the
request's own Host in both, never from a header a client can write.
The auth path is now the caller that refuses rather than defaulting:
- brand.ForHost is gone, with brand.Default and brand.IssuerFor. While a
defaulting resolver existed the auth path used it, so hanzo.id
authenticated callers on every Host nobody claimed — a pod IP, a
*.svc.cluster.local name, localhost through a port-forward, an absent Host,
a misrouted vhost. brand.For reports ok, and there is no fallback to reach.
- `aud` must contain this deployment's IAM clientId (AML_CLIENT_ID, no
default, refuses to start without it). IAM stamps aud = the app's clientId,
so with no pin any first-party token from the same issuer was a credential
here, including one minted for a marketing site. RFC 9068 §4.
- `tokenType` must be "access-token". An id_token carries the same iss and
aud and is issued to a browser, not to an API caller.
- `owner` must be an org the caller is a member of, when the token carries a
membership set. IAM stamps the APP's org as `owner` (internal/oidc/jwt.go
Sign), so a shared or org-choice application would make every one of its
users one tenant. IAM refuses a cross-org login for a dedicated app
(internal/oidc/login.go), which is what makes the claim correct for the
shape this supports: one application per institution.
- a bearer that is not a compact JWS is refused before any keyset is
resolved, so an unauthenticated caller cannot spend a network round trip.
Keys, in api/jwks.go with the algorithms that accept them, because the two
lists have to agree:
- one lock per issuer, taken outside the map, so an issuer that does not
answer holds up only itself. One process-wide mutex held across the HTTP
fetch meant a single unreachable issuer — selected by a caller's Host —
serialised every request in the process behind it.
- a failed fetch is remembered briefly, so an unreachable issuer costs one
attempt per window instead of one per request.
- EC (ES256/384/512) and ML-DSA-65 verify, which is what IAM publishes
besides RSA (internal/oidc/jwks.go signingAlgs). A dropped key type is a
silent lockout of a whole brand, so a key that cannot be loaded now names
its kty and alg in the log.
Also here: caseOf returned refuse()'s value as an error, and refuse() returns
nil once it has answered the request — so an unauthenticated POST to a case
route dereferenced a nil case. It reports errNoCase and writes nothing.
Transaction.TenantID is deleted: a second tenant field nothing reads is an
invitation to scope data by a value the caller supplies.
Co-authored-by: Hanzo Dev <[email protected]>
|
||
|
zeekayandHanzo Dev
|
3c041ed331 |
merge the detection library and the screening rewrite into one engine
Two branches had independently rebuilt the same two packages, which is one
implementation too many. This resolves them into one.
Their side is the base for detection and screening, because it is better:
- Rules name evidence, and SetRules refuses a set whose evidence this
deployment cannot supply — so the eleven stub helpers that silently disabled
thirteen rules cannot exist. A rule that cannot fire is worse than an absent
one, because it still appears in the catalog a regulator reads.
- One screening store, filled by the refresh and read by the endpoints. Two of
them is why sanctions search answered "no match" for every name while
designations were being loaded somewhere else.
- Parsers for all four publishers against their real files, recovering what the
single-parser path could not read at all: OFAC's digital-currency addresses,
weak aliases held apart from screening candidates, and OFSI's row-per-name
layout collapsed to subjects.
- Name matching over runes, not bytes. Two distinct Han names scored 0.867
byte-wise, above the 0.85 threshold, and 0.700 correctly.
- Deviation from the median and MAD rather than the mean, because a monitoring
baseline contains the outliers it is used to hunt.
- A rule vocabulary with providers behind every term, in place of expressions
reaching into a struct.
Kept from this side, because their branch did not have it:
- The tenant comes from an authenticated identity, not from a request header.
- A rule that could not run carries no weight. Their action() already capped a
failing rule at review, which is right, but the hit still entered the score
at full weight — so a broken rule saturated every transaction's score and the
queue stopped ranking. Faults are now excluded from the aggregate while still
raising their own alert and still forcing review.
- Per-source readiness: which list, how many designations, how long ago, and
the reason if the last attempt failed. Their store answers whether screening
can be relied on at all; this answers what is behind it, which is what an
operator can act on.
- Behavioural scoring, retention, tokenisation, replay and reference data.
Conversion now refuses an unknown currency instead of passing the amount
through. Passing it through understates every currency worth more than a dollar,
so a payment three times over a reporting limit tested as sitting under it.
One ordering table for actions (types.ActionRank), one EvalAll signature, one
screening store, one readiness surface. 15 packages, tests and race clean.
Co-authored-by: Hanzo Dev <[email protected]>
|
||
|
zeekayandHanzo Dev
|
644d8be410 |
wire the engine to one screening store, and serve the catalog
The refresh filled a Base-backed sanctions store and the search endpoint queried a separate empty in-memory one. Neither was reachable from the other, so /v1/aml/sanctions/search returned no matches for every name, permanently, and nothing in the system reported it. There is now one store, shared by the endpoints and by the engine's screening term. Screening refuses rather than reports clean. An unloaded or stale list set, a list that parsed to zero entries, or a class the deployment cannot answer all produce an error, which the engine turns into a review. Political exposure is refused outright: there is no authoritative public list of politically exposed persons and none of the four sanctions publishers is one, so answering from sanctions data would report every customer as not politically exposed. Each publisher is fetched and parsed by its own parser, and a failure for one leaves the other three loaded rather than clearing them. Transactions carry a direction and the customer's name and jurisdiction. The previous ingestion path synthesised a customer from the identifier alone, so every screening and jurisdiction rule evaluated against an empty name and an empty country and could not fire whatever the engine did. Case events check that the case belongs to the requesting organisation. A case identifier is returned to whoever submitted the transaction, so possession of it was authority to write on another tenant's investigation. Health reports function rather than process: whether the lists are ready, which loaded when, and how many terms this deployment can answer. Currency rates and the higher-risk jurisdiction listing are loaded, never compiled in. /v1/aml/catalog publishes the rules with their citations, the obligations claimed, and the obligations outstanding. Screening 31,338 real designations across the four published lists produced two false positives, both a one-word list entry matching an unrelated three-part name on a shared prefix. Raising the threshold could not separate them from genuine transliterations, which score lower; the asymmetry could. A single-token name now has to match a multi-token name near-exactly. Zero false positives after. Co-authored-by: Hanzo Dev <[email protected]> |
||
|
zeekayandHanzo Dev
|
eaab7eeda3 |
token: compare the mac, not the label, when checking domain separation
The test asserted that two domains mint different tokens, which the domain prefix guarantees on its own — so it passed with the per-domain key derivation removed and one key minting every domain. It now compares the mac behind the prefix, which is the thing the key schedule is responsible for. Found by mutating the HKDF info to a constant: the mutant survived, which is the only way to learn that a test passes for the wrong reason. Co-authored-by: Hanzo Dev <[email protected]> |
||
|
zeekayandHanzo Dev
|
c1372cb0cc |
aml: anomaly scoring, retention, tokenisation, replay, reference data
Five gaps that a monitoring product cannot be sold without. anomaly scores whether a transaction is unusual for the entity that made it, which is the statutory question — consistency with the customer's own profile — and one no fixed threshold can express. It complements the rule library rather than replacing it: rules encode typologies someone has already named, and are blind to anything nobody has written down. The design is deliberately attributable: a score can name the features that drove it, which is what a supervisor asks for and what a neural scorer cannot answer. retention keeps records for their statutory period and destroys them when it runs out, including the retained assessment whether or not it produced a report. token protects direct identifiers at rest while keeping the engine able to correlate: one key schedule, a deterministic org-scoped pseudonym to index on and a sealed record for reads that are entitled to it, so a record can be found by party without holding the party's name in the clear. replay runs a rule over real history and reports what it would have done before anyone activates it — the same question as "how many alerts would this raise", which is what makes a rule change reviewable rather than a live experiment. reference holds the published data a rule needs but cannot derive: currency conversion and jurisdiction risk listings, each carrying the date it was current at, each refusing to answer rather than guess. Passing an unknown currency through unchanged moves every threshold by the exchange rate — the structuring rule compared raw notional, so 9,500 units of a minor currency read as a near-threshold cash amount. A jurisdiction list that answers "not listed" because it was never loaded is indistinguishable from a clean world. Co-authored-by: Hanzo Dev <[email protected]> |
||
|
zeekayandHanzo Dev
|
e6f58397ab |
sanctions: parse all four published list families
Each parser was written against the published file, not against a description of it. Verified counts: OFAC 19,175 (matching the Record_Count the file declares), UN 1,011, EU 6,017, OFSI 5,135. What the previous single parser lost, and why each matters: OFAC publishes 963 digital currency addresses across 19 networks in idList. Dropping idList meant a payment to a designated blockchain address could not be detected by any means. OFAC marks 4,383 of its 24,546 aliases weak and the UN and OFSI mark theirs Low; flattening them to plain strings matched fragments at full confidence. OFAC flags which of several dates of birth is the main one, and for 37 subjects it is not the first — taking the first dropped the tie-break for exactly the subjects that need one. UN, EU and OFSI were parsed with the OFAC parser on the stated grounds that the formats are similar. They are not: the roots are CONSOLIDATED_LIST, export and a CSV. All three failed every night, logged a line, and continued. OFSI publishes one row per name and joins them by group; 19,761 rows are 5,135 subjects. It also puts the header on line two behind a preamble, orders the name columns Name 6 first, and writes partial dates as 00/00/1994. A parse that yields fewer records than the publisher declares is refused rather than loaded, because a partial sanctions list clears designated subjects. Co-authored-by: Hanzo Dev <[email protected]> |
||
|
zeekayandHanzo Dev
|
07e69e3ee2 |
luxfi/aml — real-time AML/CFT transaction monitoring
A Go AML engine: a configurable rules DSL over expr-lang with weight-of-evidence scoring, sanctions screening against OFAC/UN/EU/HMT, case management, and scored alerting. Single binary, embedded SQLite per org, embedded UI. Surface (all /v1, org-scoped): /v1/aml/transactions ingest and score /v1/aml/rules /rules/test the rules DSL and its dry-run /v1/aml/sanctions/search list screening /v1/aml/cases investigation workflow /v1/aml/health Build: make build produces one static binary with the UI embedded (embedui tag); bare Go tooling needs no Node. Third-party license and notice texts are generated from the module graph the linker actually walks and ship beside every artifact. Co-authored-by: Hanzo Dev <[email protected]> |