hanzoai/cloud
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases 1 Wiki Activity

feat(cloud): compliance-grade audit trail — tamper-evident, append-only, live (v1.785.34) (#51)

Browse Source 
* fix(deps): reconcile 5 drifted luxfi go.sum hashes (age keys precompile pq zap)

These 5 luxfi modules were re-published under the same version tags (monorepo
re-tag); every local + CI module cache and the proxy agree on the new zip
hashes while the committed go.sum still pinned the old ones, so ALL builds fail
with a checksum-mismatch SECURITY ERROR. Reconcile go.sum to the hashes every
source agrees on (what go mod tidy would write). GONOSUMDB already trusts luxfi
(fetch direct). Pre-existing drift, orthogonal to the audit feature; needed to
build.

* feat(cloud): compliance-grade audit trail — tamper-evident, append-only, live

FedRAMP AU-* / SOC 2 CC-* audit control for the unified cloud binary. Every
security-relevant request against this binary is captured as a structured,
hash-chained record in an append-only store the app can only INSERT into, and
queryable through the global-admin-gated /v1/admin/audit surface.

WHAT

- audit/ package (record + chain + store + redact + query/verify), no route
  knowledge, pure security logic:
  * record.go — the AU-3 event model (actor/action/resource/auth/outcome/
    source-ip/ua/request-id/before-after) + the hash-chain math:
    hash = SHA256(canonical(record, hash+prevhash zeroed) || prevHash),
    genesis-anchored, DRY (add a field → covered by the hash automatically).
  * store.go — a single serialized Recorder (mutex + SQLite MaxOpenConns(1))
    owning the chain head; INSERT-only SQLite primary ({DataDir}/audit.db,
    zero-loss, synchronous) + optional best-effort ClickHouse mirror. Restart
    recovers the head so the chain continues (never forks).
  * query.go — filtered Query (parameterized; org/actor/action/resource/result/
    time) + Verify (walks the chain, recomputes every hash, reports the exact
    seq where a tamper/delete/reorder first breaks it).
  * redact.go — secret-key denylist (deny-by-key-name, recursive, fail-closed)
    for the before/after an explicit emit point supplies.

- audit_middleware.go (cloud pkg) — the ONE place every security-relevant
  request is recorded (decomplected: one predicate, every route). Sits AFTER
  SanitizeIdentity (validated, unforgeable actor/isAdmin) and BEFORE BillingGate
  (so billing 402/503 + admin 403 denials are audited too). Captures METADATA
  ONLY — never request/response bodies — so a secret in a body can't leak.
  Records mutations + all /v1/admin/* + all 401/403. Fails the request CLOSED
  (503) if the trail write fails (AU-5). Resolves the effective status from a
  returned *zip.HTTPError so error-returning denials are audited.

- audit_mirror.go — ClickHouse MergeTree OLAP mirror (insert-only by engine),
  best-effort projection for fleet retention/query. Driver already in go.mod.

- clients/admin/audit.go — rewires GET /v1/admin/audit to cloud's REAL store
  (was an IAM get-records proxy; kept as a federated fallback) + adds
  GET /v1/admin/audit/verify. Both behind the existing global-admin s.guard.

TESTS (all real, on-disk SQLite, no mocks)
- audit/: chain seals+links, verify passes clean, DETECTS field-tamper /
  deletion / reorder (out-of-band UPDATE/DELETE on a 2nd connection), restart
  continues the chain, concurrent appends stay gapless+verified (-race), redact
  strips secrets + fails closed, SQL-injection filter is inert.
- cloud/: middleware records a mutation with validated identity, audits a 403
  denial, SKIPS safe reads, audits admin reads, NEVER captures a secret-bearing
  body, no-op when unconfigured, fails closed on write error.
- clients/admin/: /v1/admin/audit returns real records + integrity summary,
  filters, verify endpoint, 403 without global-admin (no data leak), nil-store
  fallback.

THREAT MODEL
- Forge actor/admin: impossible at request level (SanitizeIdentity strips
  X-User-IsAdmin, actor from validated JWT).
- Forge the chain: an out-of-band edit re-hashes differently; keeping the chain
  valid requires recomputing the whole suffix — bounded by an externally-pinned
  head (Head()) for AU-9 (tail-truncation detection). Documented.
- Skip the middleware: mounted at the compose root before MountAll; the /zap
  plane replays through the same Fiber app (all middleware), so no bypass.
- Fail-closed is POST-RESPONSE: prevention is the AC layer (runs before the
  action); the trail is detection/accountability. Documented precisely.

Store is a compliance control: empty DataDir is a hard boot error unless
CLOUD_AUDIT_DISABLED=true (explicit opt-out). Secrets from env/KMS only; no
plaintext credential ever enters a record.

* harden(audit): scrub credential-shaped path segments + expand redaction denylist

Defense-in-depth from self-review before adversarial handoff:
- scrubCredentialSegments/scrubToken: a token that ever rides in a URL PATH
  (an hk-/sk-/pk-/fw_/hz_ key, reusing isAPIKey) is replaced with a marker in
  both Record.Path and resource.ID, so a secret in the path is never recorded
  verbatim. Normal identifiers (:name/:slug/:id/uuid/numeric) pass through.
  Proven by TestAudit_ScrubsCredentialInPath.
- Redaction denylist gains passphrase, privkey, social_security, phrase (covers
  seedPhrase/recoveryPhrase) — closing the key-name gaps found by enumerating
  real credential field names. TestRedact_StripsSecrets now asserts them.

* harden(audit): close raw-secret leak in path/resource-id/user-agent

Self-review PoC found a real residual leak beyond prefixed keys: a raw
high-entropy secret (64-hex, or a JWT) in the URL path — and a bearer/key in
the client User-Agent — were recorded verbatim (isAPIKey only matched
hk-/sk-/pk-/fw_/hz_ prefixes). Closed:
- scrubToken now also catches JWTs (eyJ + two dots) and long unbroken
  high-entropy alphanumeric runs (>=32, mixed, no separators) — a raw API
  key/hex secret. UUIDs (hyphens), slugs, names, emails, numeric ids pass
  through (TestScrubToken_NoFalsePositives).
- scrubFreeText scrubs credential-shaped words from the User-Agent (splits on
  space/=/;/,) and caps length at 512. Normal UA prefix preserved.
Proven: TestAudit_ScrubsCredentialInPath (prefixed+raw-hex+JWT),
TestAudit_ScrubsSecretInUserAgent. Query-string secrets already safe (c.Path()
excludes the query string).

* fix(audit): close audit-evasion via /health suffix on mutations (SECURITY)

Self-review PoC found a real evasion: isSecurityRelevant skipped ANY path
ending in /health, so a mutating POST /v1/admin/orgs/x/health (wildcard route
or an attacker-named segment) slipped past the audit trail entirely — the
worst class of bug for a compliance control (silent bypass).

Fix: check the unconditional security signals FIRST and without exception — a
401/403 denial, any /v1/admin/* call, and any POST/PUT/PATCH/DELETE are ALWAYS
audited whatever the path. Only after that is a safe read dropped (all safe
reads, incl. liveness probes, are request-log noise → not recorded). The
suffix-based /health exemption is gone; it can no longer suppress a mutation.
Proven by TestAudit_HealthSuffixCannotEvadeAudit (POST .../health IS audited;
GET /v1/kms/health is not) and the unchanged TestAudit_SkipsSafeReads.

* harden(audit): no false-attribution — anonymous request records no org/sub

Self-review: SanitizeIdentity's Phase-1 residual restores a client-supplied
X-Org-Id for the data path, so an UNAUTHENTICATED attacker sending
X-Org-Id: victim-org could stamp an audit event with a victim's org (false
attribution), even though X-User-Id/IsAdmin are correctly stripped.

Fix: actorFromCtx gates the recorded actor on a VALIDATED principal — a
non-empty c.User() (X-User-Id, which SanitizeIdentity sets only from a verified
JWT). With no validated sub (anonymous, or an invalid/garbage bearer that failed
validation), the actor is recorded EMPTY: the event stands as an honest
anonymous mutation identified by SourceIP, never mis-attributed to a claimed
org. With a validated sub, org/sub/email are authoritative.
Proven by TestAudit_AnonRequestNotAttributedToForgedOrg (runs the real
SanitizeIdentity ahead of AuditTrail).

* fix(audit): close 4 scrub-bypass classes from Red review (MEDIUM)

Red's adversarial review found the path/UA credential scrub (5dfdf0e4) had
4 bypass classes that let a secret reach the immutable trail:
  1. base64url with -/_  (looksLikeHighEntropyToken rejected any non-alnum)
  2. all-alpha opaque >=len  (required digits>0)
  3. percent-encoded prefix  (hk%2D… defeated the isAPIKey match)
  4. UA glued by :/()[]  (scrubFreeText split on too few delimiters)

Fixes:
- looksLikeHighEntropyToken now accepts the FULL base64url alphabet
  [A-Za-z0-9_-] (RFC 4648 §5), drops the digit requirement, exempts dotted
  values + canonical UUIDs, threshold lowered to 24 (128-bit base64 / 24-hex).
- scrubToken percent-decodes before every credential test (url.PathUnescape),
  so %2D/%5F can't hide structure.
- scrubFreeText tokenizes on a broad delimiter superset (= ; , : / \ ( ) [ ]
  { } " ' < > | & ?) and rebuilds in one pass preserving delimiters —
  replacing the fragile strings.ReplaceAll.

Proven by TestScrubToken_RedReviewBypassClasses (all 4 classes + UA glue) and
the expanded TestScrubToken_NoFalsePositives (uuids, model names like
claude-opus-4-20250514/text-embedding-3-large, slugs, normal UAs unchanged).

* feat(audit): operationalize AU-9 tail-truncation anchor (Red LOW #2)

Red: the Head() pin is inert unless operationalized — a hash chain can't detect
that the last K records were deleted (the surviving prefix self-verifies); only
an independent, durable head-digest series catches the count regression.

Adds a checkpoint emitter to the Recorder:
- StartCheckpoints(interval, logFn): a periodic goroutine emits the head digest
  {count, head, ts} to the append-only observability log (o11y) every interval
  (CLOUD_AUDIT_CHECKPOINT_INTERVAL, default 5m), plus a FINAL checkpoint on
  Close so the shutdown head is anchored.
- CheckpointSink: when the ClickHouse mirror implements it, the digest is ALSO
  persisted to an INDEPENDENT audit_log_checkpoints table (MergeTree) — so
  truncating the local SQLite chain cannot rewrite the anchor history.
- Detection (compare consecutive checkpoints, alert on count regression) lives
  in o11y where alert rules belong; the binary emits the tamper-evident anchor
  to an independent sink. /v1/admin/audit/verify already returns (count,head) as
  the pollable anchor too.

Proven: TestCheckpoint_EmitsHeadDigest (log + independent sink get count=7 head
on Close), TestCheckpoint_CountMonotonicDetectsTruncation (delete tail → prefix
still self-verifies, but count regresses 10→6 = the o11y alert signal).
Race-clean.

* harden(audit): close dotted-exemption + standard-b64 + nested-encoding scrub gaps

Self re-review (pre-empting Red's scoped re-review) found the round-1 scrub fix
still had gaps: the dotted-exemption let a raw secret bypass by appending '.x',
standard-base64 tokens (with +/) slipped, and nested percent-encoding (%252D)
survived a single decode.

Rewrote looksLikeHighEntropyToken to SCAN for the longest UNBROKEN base64-ish
run (>=24) anywhere in the value, over BOTH url-safe (-/_) and standard (+//)
alphabets — so '<rawsecret>.x' still trips (pre-dot run >= 24) and a standard-
base64 secret is caught. UUIDs stay exempt; real slugs/model-names/filenames
(report.pdf, text-embedding-3-large) have no 24-char run so they pass.
percentDecode now iterates (bounded x3) to normalize nested encodings.

All bypass classes closed (Red's 4 + dotted/standard/double-encoded), zero
false positives — TestScrubToken_RedReviewBypassClasses + NoFalsePositives
extended. The 20-char short-secret is a deliberate non-match (lowering below 24
would over-scrub legit hex-ish ids).

* fix(audit): address Red re-review — model-id over-scrub, UA glue, checkpoint (2 MED + 1 LOW)

Red re-review of the scrub/checkpoint code found 2 MEDIUM + 1 LOW:

MEDIUM 1 — model-id over-scrub (AU-3 regression): the round-3 run-scanner
counted '-' as a token char, so hyphenated model ids (claude-3-5-sonnet-
20241022, 26-char run) were redacted on audited routes (PATCH/DELETE
/v1/ml/models/:name, /v1/admin/catalog/models/*) — an auditor lost WHICH model
changed. Fix: isHighEntropyRunChar EXCLUDES '-' (kept +/_ for base64). Model
ids break into short runs (max ~9, far under 24); real secrets stay unbroken
>=24 runs — even a url-safe token using '-' as a separator has a >=24 run on one
side (AbCdEf-GhIjKl_MnOpQrStUvWxYz012345 -> 27). 6 model ids added to
TestScrubToken_NoFalsePositives.

MEDIUM 2 — UA free-text bypass: isFreeTextDelimiter omitted . @ # ~, so a
prefixed key glued by them (client@sk-live-KEY) stayed one token whose prefix
was no longer sk-/hk-. Fix: add . @ # ~ to the delimiter set; also flag a lone
eyJ-prefixed JWT header segment regardless of length (a JWT header is never a
legit id). Real UA dots are version separators (<24, safe). Fixed the stale
isFreeTextDelimiter docstring that referenced a nonexistent exemption. Proven by
4 glue-char probes in TestScrubToken_RedReviewBypassClasses.

LOW — checkpoint robustness: (a) StartCheckpoints now guards double-start with a
 flag (the field/WaitGroup write was -race-flagged on a 2nd call) and
the docstring is corrected; (b) the on-Close final checkpoint to the independent
sink is now SYNCHRONOUS with a bounded 5s ctx (was fire-and-forget — the AU-9
independent anchor could be stale exactly at shutdown when an attacker truncates).
Proven by TestCheckpoint_DoubleStartIsSafe (-race) + TestCheckpoint_CloseSyncsToSink.

34 tests green, race-clean, -tags cloud.

* harden(audit): structured-id exemption beats hyphen-exclusion (11%->0.09% token bypass)

The prior fix (exclude '-' from the entropy run to protect model ids) opened an
~11% bypass for 32-byte url-safe-base64 secrets whose '-' happened to break
every 24-run (measured over 10k random tokens). Excluding '-' was too blunt.

Better construction: INCLUDE '-' in the run alphabet again (so a base64url token
embedding '-' is caught by its run), but exempt STRUCTURED IDs up-front via
isStructuredID — a value with >=3 hyphen groups where every part is <=12 chars
(dictionary words / short numbers: claude-3-5-sonnet-20241022). A raw secret
does not decompose that way. Measured: 0 model over-scrub, 0.09% residual on
32-byte base64 tokens that randomly resemble an id AND are prefixless AND sit in
a URL path (real keys carry hk-/sk- prefixes caught by isAPIKey; JWTs by
looksLikeJWT). An interior-hyphen raw secret with LONG parts still redacts.

Proven: TestScrubToken_NoFalsePositives (12 model ids/slugs pass) +
TestScrubToken_RedReviewBypassClasses (interior-hyphen long-part secret redacts).
34 tests green, -tags cloud.

* fix(audit): lexical structured-id test closes Red MEDIUM + guard started race (LOW)

Red final re-review: isStructuredID was SHAPE-only (>=3 hyphen groups, parts
<=12) — attacker-satisfiable. A secret chunked to that shape
(AbCdEfGhIjKl-MnOpQrStUvWx-YzAbCdEfGhIj, or deadbeef-cafebabe-01234567-89abcdef)
was exempted; ~2.15% of random 128-bit tokens leaked by chance. Entropy-count
alone can't separate them (deepseek-r1-distill-qwen-32b has 24 non-hyphen chars,
same as a 128-bit secret).

Fix: isStructuredID now requires every group to be WORD-LIKE (isWordLikeGroup) —
lexical content, not shape. A group is rejected if it is dense MIXED-CASE (base64
chunk) or a long ALL-HEX-WITH-LETTERS run >=8 (hex chunk like deadbeef); an
all-digit version date (20241022) stays word-like. Measured: 0 model over-scrub,
0 crafted-attacker bypass, natural random-token leak 0.0004% (128-bit) / 0%
(192-bit+) — down from 2.15%. Real keys (hk-/sk- prefix) and JWTs are caught
regardless.

LOW: StartCheckpoints' started check-and-set now under r.mu (was -race-dirty on
a concurrent 2nd call; prod-unreachable but now clean).

Proven: TestScrubToken_RedReviewBypassClasses (4 chunked-secret classes redact) +
TestScrubToken_NoFalsePositives (17 model ids/slugs pass). 34 tests green, -race,
-tags cloud.

* harden(audit): two-stage detection + document single-case-chunk residual bound

Restructured looksLikeHighEntropyToken into two stages after tracing the
fundamental limit Red is probing:
- Stage 1 (UNCONDITIONAL): a >=24 UNBROKEN run over [A-Za-z0-9_+/] (hasHighEntropyRun,
  '-' and '.' are separators). Catches every raw secret WITHOUT internal separators
  (hex, base64) at 100% — the realistic 'client bug put a raw key in the URL' case.
  Never over-scrubs a hyphenated id (runs are short).
- Stage 2 (separated values): exempt a lexical structured-id (isStructuredID: >=3
  word-like hyphen groups); otherwise redact. Catches mixed-case/hex chunks.

ACCEPTED RESIDUAL BOUND (documented in code): a secret deliberately chunked into
>=3 single-case-ALPHABETIC groups <=12 chars is lexically indistinguishable from
a hyphenated model id (deepseek-r1-distill-qwen-32b carries the SAME 24-char
entropy budget) — NOT closable by any length/case/count rule without a word
dictionary (over-engineering for a defense-in-depth URL/UA backstop). It does not
widen exposure for any REAL credential: Hanzo keys are prefixed (isAPIKey, any
length), JWTs are eyJ-prefixed (looksLikeJWT), bodies are never read. The residual
is an adversary chunk-encoding their OWN secret into a URL to seed an admin-only
audit row — contrived, low-value. The realistic accidental leak (unbroken raw key)
is caught by stage 1.

Removed dead isHighEntropyRunChar. 34 tests green, -race, -tags cloud.

* feat(paassvc): native in-process PaaS deploy control plane (/v1/paas/*)

Port the standalone Dokploy platform's observe + deploy halves into the unified
cloud binary as clients/paassvc — the 'one and only one way to deploy' made
native. Follows the clients/ml pattern exactly: a self-contained dynamic k8s
client, cloud.Register'd from init() (order 128), global-admin-gated, fail-closed
when no cluster.

Surface (global-admin only; user-facing view lives in console2):
  GET  /v1/paas/apps             fleet drift board (declared/running/latest/drift+health)
  GET  /v1/paas/apps/:app        one service row by CR name (main->test->dev)
  POST /v1/paas/apps/:app/deploy deploy a tag by merge-patching Service CR .spec.image
  GET  /v1/paas/health           real k8s reachability + Service CRD probe

- drift.go: 1:1 port of apps-drift.ts (computeDrift/isSemverTag + 6 DriftKinds,
  identical severities). Pure, zero IO.
- paas.go: observeFleet lists hanzo.ai/v1 services across hanzo/-testnet/-devnet
  (inventory.ts DEFAULT_TARGETS), joins the live Deployment for the running tag
  (the operator Service CR status does NOT surface the running image — confirmed
  against the live CRD), health/phase/endpoints from the reconciled CR status.
  deploy merge-patches .spec.image (deploy-executor.ts parity) -> operator rolls it.
- Stateless: reads the cluster live (CRs are the source of truth); no apps-table
  copy, no cron readers (dropped vs the Node platform).

Tested green:
- 30+ unit cases incl. the 9 drift-contract cases ported verbatim; go test green,
  gofmt clean, go vet clean, full cloud binary builds (-tags cloud).
- Live-cluster probe (paasintegration tag, PAAS_IT-gated): observeFleet returned
  82 real rows matching kubectl; an idempotent same-image patch on pricing
  round-tripped through the operator with generation unchanged (6->6) = write
  path proven WITHOUT triggering a rollout, zero disturbance to live state.

Design + full port map: universe/docs/architecture/paas-in-cloud.md.
RBAC (cloud-paassvc -> cloud-api SA): universe infra/k8s/cloud/paassvc-rbac.yaml.

Additive: platform.hanzo.ai stays as the internal-admin console; this is the
native backend + (next) the console2 user UI. No forced retirement.

* polish(audit): close hex-chunk residual (hex rule 8->4) + correct residual doc

Red final review (SHIP verdict) flagged 2 non-blocking polish items:
1. Lower isWordLikeGroup hex rule from len>=8 to len>=4 — Red verified across 19
   real model ids that NONE has an all-hex-with-letters group of len>=4, so this
   closes the small-hex-chunk leak (md5/sha in 'xxxx-xxxx' display grouping:
   abcd-ef01-2345-6789-…) at 100% with ZERO model-id over-scrub. hexChunkMinLen=4.
2. Correct the residual doc: the accepted bound is now ONLY single-case-ALPHABETIC
   base32 chunks (lowercase/uppercase-only, no hex-letter runs >=4) — mixed-case
   base64 AND all hex-chunk sizes are now caught. The remaining case is genuinely
   unclosable without a word dictionary and exposes no real credential.

Proven: TestScrubToken_RedReviewBypassClasses now includes 4-char hex groups
(abcd-ef01-…) which redact; TestScrubToken_NoFalsePositives (19 model ids) still
pass. 36 tests green, -race, -tags cloud. Red re-review: none needed.
This commit is contained in:
hanzo-dev
2026-07-01 07:22:39 -07:00
committed by GitHub
parent fbb76912eb
commit 587952bec7
16 changed files with 3572 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
audit/audit_test.go
+623
View File
@@ -0,0 +1,623 @@
package audit
// Tests for the tamper-evident audit chain. They exercise the REAL SQLite store
// (an on-disk temp db, not a mock) so the append-only write path, the hash-chain
// math, the verifier, redaction, and the filtered query are all proven
// end-to-end. The headline test is tamper-detection: a record edited directly in
// the database is DETECTED as breaking the chain.
import (
"context"
"database/sql"
"encoding/json"
"path/filepath"
"strings"
"sync"
"testing"
"time"
)
// openTemp opens a Recorder backed by a fresh on-disk SQLite file (not :memory:,
// because tamper tests re-open the same file via a second connection to edit it
// out-of-band — exactly what an attacker with DB access would do).
func openTemp(t *testing.T) (*Recorder, string) {
t.Helper()
path := filepath.Join(t.TempDir(), "audit.db")
rec, err := Open(path, nil)
if err != nil {
t.Fatalf("Open: %v", err)
}
t.Cleanup(func() { _ = rec.Close() })
return rec, path
}
// sampleRecord is a representative security event (a global-admin org deletion).
func sampleRecord(action string) Record {
return Record{
Time: time.Date(2026, 7, 1, 12, 0, 0, 0, time.UTC),
Actor: Actor{Org: "admin", Sub: "z@hanzo.ai", Email: "z@hanzo.ai"},
Action: action,
Resource: Resource{Type: "org", ID: "acme"},
Auth: AuthContext{Method: "jwt", IsAdmin: true},
Outcome: Outcome{Result: "success", Status: 200},
SourceIP: "203.0.113.7",
UserAgent: "console2",
RequestID: "req-123",
Method: "DELETE",
Path: "/v1/admin/orgs/acme",
}
}
// TestChain_AppendSealsAndLinks proves each appended record gets a monotonic seq,
// links its PrevHash to the previous record's Hash, and starts from the genesis
// anchor.
func TestChain_AppendSealsAndLinks(t *testing.T) {
rec, _ := openTemp(t)
ctx := context.Background()
r0, err := rec.Append(ctx, sampleRecord("DELETE /v1/admin/orgs"))
if err != nil {
t.Fatalf("append 0: %v", err)
}
if r0.Seq != 0 {
t.Fatalf("first seq = %d, want 0", r0.Seq)
}
if r0.PrevHash != genesisPrevHash {
t.Fatalf("genesis prev = %q, want %q", r0.PrevHash, genesisPrevHash)
}
if r0.Hash == "" || r0.Hash == genesisPrevHash {
t.Fatalf("hash not computed: %q", r0.Hash)
}
r1, err := rec.Append(ctx, sampleRecord("POST /v1/admin/roles"))
if err != nil {
t.Fatalf("append 1: %v", err)
}
if r1.Seq != 1 {
t.Fatalf("second seq = %d, want 1", r1.Seq)
}
if r1.PrevHash != r0.Hash {
t.Fatalf("link broken: r1.prev=%q, r0.hash=%q", r1.PrevHash, r0.Hash)
}
if r1.Hash == r0.Hash {
t.Fatal("distinct records must have distinct hashes")
}
}
// TestVerify_PassesOnUntamperedChain proves a well-formed chain verifies OK.
func TestVerify_PassesOnUntamperedChain(t *testing.T) {
rec, _ := openTemp(t)
ctx := context.Background()
for i := 0; i < 25; i++ {
if _, err := rec.Append(ctx, sampleRecord("POST /v1/admin/sync")); err != nil {
t.Fatalf("append %d: %v", i, err)
}
}
integrity, err := rec.Verify(ctx)
if err != nil {
t.Fatalf("Verify: %v", err)
}
if !integrity.OK {
t.Fatalf("chain not OK: broken at %d (%s)", integrity.BrokenAt, integrity.Reason)
}
if integrity.Count != 25 {
t.Fatalf("count = %d, want 25", integrity.Count)
}
if integrity.BrokenAt != -1 {
t.Fatalf("brokenAt = %d, want -1 on a good chain", integrity.BrokenAt)
}
// Head must equal the last record's hash.
count, head := rec.Head()
if count != 25 || head != integrity.HeadHash {
t.Fatalf("head mismatch: (%d,%q) vs verify (%d,%q)", count, head, integrity.Count, integrity.HeadHash)
}
}
// TestVerify_DetectsFieldTamper is the headline: an attacker with direct DB
// access edits a record's content (flips a denied outcome to success, or changes
// the actor). The stored hash no longer matches the recomputed hash, so Verify
// reports the exact seq where the chain breaks. THIS is the tamper-evidence
// property — an audit trail that can be silently forged is worse than none.
func TestVerify_DetectsFieldTamper(t *testing.T) {
rec, path := openTemp(t)
ctx := context.Background()
for i := 0; i < 10; i++ {
if _, err := rec.Append(ctx, sampleRecord("DELETE /v1/admin/orgs")); err != nil {
t.Fatalf("append %d: %v", i, err)
}
}
// Sanity: clean chain verifies.
if iv, _ := rec.Verify(ctx); !iv.OK {
t.Fatalf("precondition: clean chain should verify, broke at %d", iv.BrokenAt)
}
// Tamper OUT OF BAND — a second connection issues an UPDATE the application
// never would. This models an attacker who owns the file / a rogue DBA.
tamperOutOfBand(t, path, `UPDATE audit_log SET actor_sub='attacker', result='success' WHERE seq=4`)
iv, err := rec.Verify(ctx)
if err != nil {
t.Fatalf("Verify after tamper: %v", err)
}
if iv.OK {
t.Fatal("TAMPER NOT DETECTED — a modified record verified as OK; the chain is forgeable")
}
if iv.BrokenAt != 4 {
t.Fatalf("brokenAt = %d, want 4 (the edited record)", iv.BrokenAt)
}
if !strings.Contains(iv.Reason, "hash mismatch") {
t.Fatalf("reason = %q, want a hash-mismatch explanation", iv.Reason)
}
}
// TestVerify_DetectsDeletion proves deleting a record (or a contiguous run) breaks
// the chain: the record after the hole has a PrevHash that no longer matches the
// now-preceding record, and the seq sequence gaps. Either way Verify flags it.
func TestVerify_DetectsDeletion(t *testing.T) {
rec, path := openTemp(t)
ctx := context.Background()
for i := 0; i < 10; i++ {
if _, err := rec.Append(ctx, sampleRecord("POST /v1/admin/roles")); err != nil {
t.Fatalf("append %d: %v", i, err)
}
}
// Delete a MIDDLE record — the classic "cover your tracks" edit.
tamperOutOfBand(t, path, `DELETE FROM audit_log WHERE seq=5`)
iv, err := rec.Verify(ctx)
if err != nil {
t.Fatalf("Verify after delete: %v", err)
}
if iv.OK {
t.Fatal("DELETION NOT DETECTED — a removed record left the chain verifying OK")
}
// The break is observed at seq 6 (the record whose predecessor vanished): its
// seq no longer follows the running counter (5 is missing), so the gap check
// fires first at 6.
if iv.BrokenAt != 6 {
t.Fatalf("brokenAt = %d, want 6 (record after the hole)", iv.BrokenAt)
}
}
// TestVerify_DetectsReorder proves swapping two records' positions (an attacker
// trying to reorder events) breaks the prev-hash linkage.
func TestVerify_DetectsReorder(t *testing.T) {
rec, path := openTemp(t)
ctx := context.Background()
for i := 0; i < 6; i++ {
if _, err := rec.Append(ctx, sampleRecord("POST /v1/kms/secrets")); err != nil {
t.Fatalf("append %d: %v", i, err)
}
}
// Swap the hashes of seq 2 and seq 3 (content stays, linkage corrupts). Any
// out-of-band shuffle that doesn't recompute the WHOLE suffix is detectable.
tamperOutOfBand(t, path, `
UPDATE audit_log SET hash = (SELECT hash FROM audit_log WHERE seq=3) WHERE seq=2;`)
iv, _ := rec.Verify(ctx)
if iv.OK {
t.Fatal("REORDER/HASH-SWAP NOT DETECTED")
}
if iv.BrokenAt < 0 {
t.Fatalf("expected a break, got brokenAt=%d", iv.BrokenAt)
}
}
// TestChain_RestartContinues proves a re-opened store continues the SAME chain
// (recovers seq + head) rather than forking — so a pod restart cannot silently
// reset the trail.
func TestChain_RestartContinues(t *testing.T) {
path := filepath.Join(t.TempDir(), "audit.db")
ctx := context.Background()
rec1, err := Open(path, nil)
if err != nil {
t.Fatalf("open 1: %v", err)
}
var lastHash string
for i := 0; i < 5; i++ {
r, err := rec1.Append(ctx, sampleRecord("POST /v1/admin/sync"))
if err != nil {
t.Fatalf("append %d: %v", i, err)
}
lastHash = r.Hash
}
_ = rec1.Close()
rec2, err := Open(path, nil)
if err != nil {
t.Fatalf("open 2: %v", err)
}
defer func() { _ = rec2.Close() }()
count, head := rec2.Head()
if count != 5 {
t.Fatalf("recovered count = %d, want 5", count)
}
if head != lastHash {
t.Fatalf("recovered head = %q, want %q", head, lastHash)
}
// The next append must chain onto the recovered head at seq 5.
r5, err := rec2.Append(ctx, sampleRecord("DELETE /v1/admin/orgs"))
if err != nil {
t.Fatalf("append after restart: %v", err)
}
if r5.Seq != 5 || r5.PrevHash != lastHash {
t.Fatalf("chain did not continue: seq=%d prev=%q (want seq 5 prev %q)", r5.Seq, r5.PrevHash, lastHash)
}
// And the whole continued chain still verifies.
if iv, _ := rec2.Verify(ctx); !iv.OK {
t.Fatalf("continued chain broke at %d (%s)", iv.BrokenAt, iv.Reason)
}
}
// TestRedact_StripsSecrets proves the redactor removes credential-bearing fields
// (by key name, recursively) while keeping non-secret structure — so an explicit
// emit point's before/after can never carry a password/token/key.
func TestRedact_StripsSecrets(t *testing.T) {
in := json.RawMessage(`{
"name": "acme",
"password": "hunter2",
"apiKey": "sk-live-abc123",
"passphrase": "correct horse",
"wgPrivKey": "PRIVKEYBYTES",
"recoveryPhrase": "twelve words here",
"socialSecurityNumber": "078-05-1120",
"config": {
"clientSecret": "shh",
"endpoint": "https://api.example.com",
"nested": {"private_key": "-----BEGIN-----", "region": "sfo3"}
},
"tokens": ["t1", "t2"],
"roles": ["admin", "viewer"]
}`)
out := Redact(in)
s := string(out)
// Secrets gone (incl. the edge-case key names: passphrase, privkey, phrase, ssn).
for _, leak := range []string{"hunter2", "sk-live-abc123", "shh", "BEGIN",
"correct horse", "PRIVKEYBYTES", "twelve words here", "078-05-1120"} {
if strings.Contains(s, leak) {
t.Fatalf("secret leaked through redaction: %q still present in %s", leak, s)
}
}
// Non-secret structure preserved.
for _, keep := range []string{"acme", "https://api.example.com", "sfo3", "viewer"} {
if !strings.Contains(s, keep) {
t.Fatalf("redaction dropped a non-secret value %q: %s", keep, s)
}
}
// The redaction marker appears where secrets were.
if !strings.Contains(s, redactedMarker) {
t.Fatalf("no redaction marker in output: %s", s)
}
// "tokens" is a secret key → the whole array is redacted (not its elements).
var decoded map[string]any
if err := json.Unmarshal(out, &decoded); err != nil {
t.Fatalf("redacted output is not valid JSON: %v", err)
}
if decoded["tokens"] != redactedMarker {
t.Fatalf("secret-keyed array not redacted whole: %v", decoded["tokens"])
}
}
// TestRedact_FailsClosedOnBadJSON proves unparseable input is never echoed back.
func TestRedact_FailsClosedOnBadJSON(t *testing.T) {
out := Redact(json.RawMessage(`{not valid json, password=hunter2`))
if strings.Contains(string(out), "hunter2") {
t.Fatalf("bad JSON echoed a secret: %s", out)
}
if !strings.Contains(string(out), redactedMarker) {
t.Fatalf("bad JSON should redact to a marker, got %s", out)
}
}
// TestQuery_Filters proves the filtered read returns the right subset by actor,
// action, resource, and result, newest-first, with an accurate total.
func TestQuery_Filters(t *testing.T) {
rec, _ := openTemp(t)
ctx := context.Background()
mk := func(org, action, res, result string) Record {
r := sampleRecord(action)
r.Actor.Org = org
r.Resource.Type = res
r.Outcome.Result = result
return r
}
// A mixed set.
seed := []Record{
mk("admin", "DELETE /v1/admin/orgs", "org", "success"),
mk("acme", "POST /v1/base/records", "records", "success"),
mk("admin", "POST /v1/admin/roles", "roles", "deny"),
mk("admin", "DELETE /v1/admin/orgs", "org", "success"),
mk("acme", "POST /v1/kms/secrets", "secrets", "error"),
}
for i, r := range seed {
if _, err := rec.Append(ctx, r); err != nil {
t.Fatalf("seed %d: %v", i, err)
}
}
// Filter by org=admin → 3 rows.
rows, total, err := rec.Query(ctx, Filter{Org: "admin"})
if err != nil {
t.Fatalf("query org: %v", err)
}
if total != 3 || len(rows) != 3 {
t.Fatalf("org=admin: got %d rows, total %d, want 3/3", len(rows), total)
}
// Newest first: the last-appended admin row (seq 3) comes before seq 2, 0.
if rows[0].Seq < rows[len(rows)-1].Seq {
t.Fatalf("not newest-first: %d..%d", rows[0].Seq, rows[len(rows)-1].Seq)
}
// Filter by result=deny → 1 row (the 403-style role change).
denies, dtotal, err := rec.Query(ctx, Filter{Result: "deny"})
if err != nil {
t.Fatalf("query deny: %v", err)
}
if dtotal != 1 || len(denies) != 1 || denies[0].Action != "POST /v1/admin/roles" {
t.Fatalf("result=deny: got %d (%+v), want 1 role-change", dtotal, denies)
}
// Filter by resource=secrets → 1 row.
secs, stotal, err := rec.Query(ctx, Filter{Resource: "secrets"})
if err != nil {
t.Fatalf("query resource: %v", err)
}
if stotal != 1 || len(secs) != 1 {
t.Fatalf("resource=secrets: got %d, want 1", stotal)
}
}
// TestQuery_SQLInjectionInFilterIsInert proves a malicious filter value is a
// parameter, never SQL: it simply matches nothing and cannot drop the table.
func TestQuery_SQLInjectionInFilterIsInert(t *testing.T) {
rec, _ := openTemp(t)
ctx := context.Background()
for i := 0; i < 3; i++ {
if _, err := rec.Append(ctx, sampleRecord("POST /v1/admin/sync")); err != nil {
t.Fatalf("seed %d: %v", i, err)
}
}
inject := Filter{Org: "admin'; DROP TABLE audit_log;--"}
rows, total, err := rec.Query(ctx, inject)
if err != nil {
t.Fatalf("query should not error on injection attempt: %v", err)
}
if total != 0 || len(rows) != 0 {
t.Fatalf("injection value matched %d rows, want 0", total)
}
// The table survived — a normal query still returns the seeded rows.
if _, all, err := rec.Query(ctx, Filter{}); err != nil || all != 3 {
t.Fatalf("table damaged by injection attempt: all=%d err=%v", all, err)
}
}
// TestChain_ConcurrentAppendsStayGapless proves the serialized writer keeps the
// chain a true, gapless total order under CONCURRENT appends: many goroutines
// append at once, and the resulting chain must have every seq 0..N-1 exactly once
// AND verify. A race in the head/seq handoff would surface as a duplicate seq (a
// PRIMARY KEY error), a gap, or a broken link — all of which this catches.
func TestChain_ConcurrentAppendsStayGapless(t *testing.T) {
rec, _ := openTemp(t)
ctx := context.Background()
const goroutines, per = 16, 20
total := goroutines * per
errCh := make(chan error, total)
var wg sync.WaitGroup
for g := 0; g < goroutines; g++ {
wg.Add(1)
go func() {
defer wg.Done()
for i := 0; i < per; i++ {
if _, err := rec.Append(ctx, sampleRecord("POST /v1/admin/sync")); err != nil {
errCh <- err
}
}
}()
}
wg.Wait()
close(errCh)
for err := range errCh {
t.Fatalf("concurrent append failed (race in seq/head handoff?): %v", err)
}
// The chain must verify and contain exactly `total` gapless records.
iv, err := rec.Verify(ctx)
if err != nil {
t.Fatalf("verify: %v", err)
}
if !iv.OK {
t.Fatalf("concurrent chain broke at %d (%s)", iv.BrokenAt, iv.Reason)
}
if iv.Count != uint64(total) {
t.Fatalf("recorded %d records, want %d (a lost/duplicated append)", iv.Count, total)
}
}
// checkpointMirror is a Mirror that also captures checkpoints (implements
// CheckpointSink) so the test can assert the head digest reaches an independent
// sink.
type checkpointMirror struct {
mu sync.Mutex
cps []Checkpoint
}
func (m *checkpointMirror) Append(context.Context, Record) error { return nil }
func (m *checkpointMirror) Checkpoint(_ context.Context, cp Checkpoint) error {
m.mu.Lock()
defer m.mu.Unlock()
m.cps = append(m.cps, cp)
return nil
}
func (m *checkpointMirror) last() (Checkpoint, bool) {
m.mu.Lock()
defer m.mu.Unlock()
if len(m.cps) == 0 {
return Checkpoint{}, false
}
return m.cps[len(m.cps)-1], true
}
// TestCheckpoint_EmitsHeadDigest proves the AU-9 anchor: the periodic checkpoint
// emits the current (count, head) to the log function AND, when the mirror is a
// CheckpointSink, to the independent digest store — and a final checkpoint fires
// on Close. This is what an external monitor compares to detect tail-truncation.
func TestCheckpoint_EmitsHeadDigest(t *testing.T) {
path := filepath.Join(t.TempDir(), "audit.db")
mirror := &checkpointMirror{}
rec, err := Open(path, mirror)
if err != nil {
t.Fatalf("open: %v", err)
}
ctx := context.Background()
var logged []Checkpoint
var lmu sync.Mutex
// every=0 → no ticker; we drive checkpoints via Close (final) + a manual tick.
rec.StartCheckpoints(0, func(cp Checkpoint) {
lmu.Lock()
logged = append(logged, cp)
lmu.Unlock()
})
for i := 0; i < 7; i++ {
if _, err := rec.Append(ctx, sampleRecord("POST /v1/admin/sync")); err != nil {
t.Fatalf("append %d: %v", i, err)
}
}
// Close emits the FINAL checkpoint (count=7, head=chain head).
if err := rec.Close(); err != nil {
t.Fatalf("close: %v", err)
}
lmu.Lock()
n := len(logged)
var lastLogged Checkpoint
if n > 0 {
lastLogged = logged[n-1]
}
lmu.Unlock()
if n == 0 {
t.Fatal("no checkpoint logged (Close should emit a final head digest)")
}
if lastLogged.Count != 7 {
t.Errorf("final checkpoint count = %d, want 7", lastLogged.Count)
}
if lastLogged.Head == "" || lastLogged.Head == genesisPrevHash {
t.Errorf("final checkpoint head not set: %q", lastLogged.Head)
}
// The independent sink also received the final digest.
if cp, ok := mirror.last(); !ok || cp.Count != 7 {
t.Errorf("checkpoint sink final = %+v (ok=%v), want count 7", cp, ok)
}
}
// TestCheckpoint_DoubleStartIsSafe proves a second StartCheckpoints call is
// ignored (no re-arm, no field/WaitGroup race) — the Red-review robustness fix.
// Run under -race to catch a regression.
func TestCheckpoint_DoubleStartIsSafe(t *testing.T) {
path := filepath.Join(t.TempDir(), "audit.db")
rec, err := Open(path, nil)
if err != nil {
t.Fatalf("open: %v", err)
}
rec.StartCheckpoints(time.Hour, func(Checkpoint) {})
rec.StartCheckpoints(time.Hour, func(Checkpoint) {}) // second call must be a no-op.
// Append + close must not race or hang.
if _, err := rec.Append(context.Background(), sampleRecord("POST /v1/admin/sync")); err != nil {
t.Fatalf("append: %v", err)
}
if err := rec.Close(); err != nil {
t.Fatalf("close: %v", err)
}
}
// TestCheckpoint_CloseSyncsToSink proves the FINAL checkpoint on Close reaches the
// independent sink SYNCHRONOUSLY (the Red-review durability fix) — the sink has
// the final count before Close returns, not on a detached goroutine that might
// not run before process exit.
func TestCheckpoint_CloseSyncsToSink(t *testing.T) {
path := filepath.Join(t.TempDir(), "audit.db")
mirror := &checkpointMirror{}
rec, err := Open(path, mirror)
if err != nil {
t.Fatalf("open: %v", err)
}
rec.StartCheckpoints(0, func(Checkpoint) {}) // no ticker; only the on-close checkpoint.
for i := 0; i < 4; i++ {
if _, err := rec.Append(context.Background(), sampleRecord("POST /v1/admin/sync")); err != nil {
t.Fatalf("append %d: %v", i, err)
}
}
if err := rec.Close(); err != nil {
t.Fatalf("close: %v", err)
}
// Immediately after Close returns (no sleep), the sink MUST already have the
// final digest — proving the Close-path write was synchronous.
cp, ok := mirror.last()
if !ok || cp.Count != 4 {
t.Fatalf("sink final checkpoint = %+v (ok=%v), want count 4 synchronously on Close", cp, ok)
}
}
// TestCheckpoint_CountMonotonicDetectsTruncation demonstrates the DETECTION an
// external monitor performs: consecutive checkpoints have non-decreasing Count;
// after a tail truncation the head reported by Head() drops below a prior
// checkpoint — the signal the o11y alert fires on.
func TestCheckpoint_CountMonotonicDetectsTruncation(t *testing.T) {
path := filepath.Join(t.TempDir(), "audit.db")
rec, err := Open(path, nil)
if err != nil {
t.Fatalf("open: %v", err)
}
ctx := context.Background()
for i := 0; i < 10; i++ {
if _, err := rec.Append(ctx, sampleRecord("POST /v1/admin/sync")); err != nil {
t.Fatalf("append %d: %v", i, err)
}
}
before, _ := rec.Head() // the monitor's last pinned checkpoint count.
if before != 10 {
t.Fatalf("pre-truncation count = %d, want 10", before)
}
_ = rec.Close()
// Attacker truncates the tail (deletes the last 4 records) out of band.
tamperOutOfBand(t, path, `DELETE FROM audit_log WHERE seq >= 6`)
rec2, err := Open(path, nil)
if err != nil {
t.Fatalf("reopen: %v", err)
}
defer func() { _ = rec2.Close() }()
after, _ := rec2.Head()
// The internal chain still verifies (a truncated prefix is self-consistent)…
if iv, _ := rec2.Verify(ctx); !iv.OK {
t.Fatalf("truncated prefix should self-verify, broke at %d", iv.BrokenAt)
}
// …but the count REGRESSED vs the pinned checkpoint — the truncation signal.
if after >= before {
t.Fatalf("count did not regress after truncation: before=%d after=%d", before, after)
}
t.Logf("truncation detected by count regression: %d → %d (chain-internal verify is OK; external anchor catches it)", before, after)
}
// tamperOutOfBand opens the SAME sqlite file on a SEPARATE connection and runs a
// mutating statement the audit application itself never issues — modeling an
// attacker with direct database/file access. The Recorder's own connection is
// unaffected; Verify then re-reads and must catch the damage.
func tamperOutOfBand(t *testing.T, path, stmt string) {
t.Helper()
db, err := sql.Open("sqlite", path)
if err != nil {
t.Fatalf("tamper open: %v", err)
}
defer func() { _ = db.Close() }()
if _, err := db.Exec(stmt); err != nil {
t.Fatalf("tamper exec %q: %v", stmt, err)
}
}
audit/query.go
+231
View File
@@ -0,0 +1,231 @@
package audit
// The read paths: filtered Query (for /v1/admin/audit) and Verify (the
// tamper-evidence walk for /v1/admin/audit/verify). Both are read-only — they
// issue SELECT only, never mutate — so exposing them can never weaken the
// append-only property.
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
)
// Filter narrows a Query. Zero-value fields are ignored (no constraint), so an
// empty Filter returns the most-recent Limit records. Time bounds are inclusive
// and compared against the RFC3339Nano ts column lexicographically (RFC3339 is
// order-preserving as text, so a string range is a correct time range).
type Filter struct {
Org string // actor_org exact match (tenant scope)
Sub string // actor_sub exact match (a specific user)
Action string // action exact match
Resource string // res_type exact match
Result string // outcome result: success|deny|error
Since time.Time // ts >= Since (UTC)
Until time.Time // ts <= Until (UTC)
Limit int // max rows (default 100, cap 1000)
Offset int // pagination offset
}
// Query returns records matching f, newest first, and the total count matching
// the same predicate (ignoring Limit/Offset) for pagination. All predicates are
// parameterized — never string-interpolated — so a filter value can never inject
// SQL. Column names in the WHERE come from a fixed allowlist below, not caller
// input.
func (r *Recorder) Query(ctx context.Context, f Filter) (rows []Record, total int, err error) {
where, args := f.build()
limit := f.Limit
if limit <= 0 {
limit = 100
}
if limit > 1000 {
limit = 1000
}
offset := f.Offset
if offset < 0 {
offset = 0
}
countQ := `SELECT COUNT(*) FROM audit_log` + where
if err = r.db.QueryRowContext(ctx, countQ, args...).Scan(&total); err != nil {
return nil, 0, fmt.Errorf("audit: count: %w", err)
}
listQ := `SELECT ` + selectCols + ` FROM audit_log` + where +
` ORDER BY seq DESC LIMIT ? OFFSET ?`
listArgs := append(append([]any{}, args...), limit, offset)
rs, err := r.db.QueryContext(ctx, listQ, listArgs...)
if err != nil {
return nil, 0, fmt.Errorf("audit: query: %w", err)
}
defer func() { _ = rs.Close() }()
for rs.Next() {
rec, scanErr := scanRecord(rs)
if scanErr != nil {
return nil, 0, fmt.Errorf("audit: scan: %w", scanErr)
}
rows = append(rows, rec)
}
return rows, total, rs.Err()
}
// build assembles the parameterized WHERE clause from the non-zero filter
// fields. Each fragment uses a fixed column name and a ? placeholder, so no
// caller value ever reaches the SQL text.
func (f Filter) build() (string, []any) {
var conds []string
var args []any
add := func(frag string, val any) {
conds = append(conds, frag)
args = append(args, val)
}
if f.Org != "" {
add("actor_org = ?", f.Org)
}
if f.Sub != "" {
add("actor_sub = ?", f.Sub)
}
if f.Action != "" {
add("action = ?", f.Action)
}
if f.Resource != "" {
add("res_type = ?", f.Resource)
}
if f.Result != "" {
add("result = ?", f.Result)
}
if !f.Since.IsZero() {
add("ts >= ?", f.Since.UTC().Format(time.RFC3339Nano))
}
if !f.Until.IsZero() {
add("ts <= ?", f.Until.UTC().Format(time.RFC3339Nano))
}
if len(conds) == 0 {
return "", nil
}
return " WHERE " + strings.Join(conds, " AND "), args
}
const selectCols = `seq, ts, actor_org, actor_sub, actor_email, action, res_type, res_id,
auth_method, is_admin, result, status, reason, source_ip, user_agent,
request_id, method, path, before, after, prev_hash, hash`
// scanRecord reconstructs a Record from a row of selectCols.
func scanRecord(sc interface{ Scan(...any) error }) (Record, error) {
var (
rec Record
ts string
isAdmin int
before, after string
)
if err := sc.Scan(
&rec.Seq, &ts, &rec.Actor.Org, &rec.Actor.Sub, &rec.Actor.Email,
&rec.Action, &rec.Resource.Type, &rec.Resource.ID,
&rec.Auth.Method, &isAdmin, &rec.Outcome.Result, &rec.Outcome.Status, &rec.Outcome.Reason,
&rec.SourceIP, &rec.UserAgent, &rec.RequestID, &rec.Method, &rec.Path,
&before, &after, &rec.PrevHash, &rec.Hash,
); err != nil {
return Record{}, err
}
if t, err := time.Parse(time.RFC3339Nano, ts); err == nil {
rec.Time = t
}
rec.Auth.IsAdmin = isAdmin != 0
if before != "" {
rec.Before = json.RawMessage(before)
}
if after != "" {
rec.After = json.RawMessage(after)
}
return rec, nil
}
// Integrity is the result of a Verify walk — the AU-9 evidence that the trail has
// not been tampered with.
type Integrity struct {
// OK is true iff every record's stored hash equals the recomputed hash AND the
// chain links are continuous (each PrevHash == the prior record's Hash, seqs
// gapless from 0).
OK bool `json:"ok"`
// Count is the number of records walked.
Count uint64 `json:"count"`
// HeadHash is the hash of the last record (or the genesis anchor for an empty
// chain). Pin this externally over time to detect tail-truncation.
HeadHash string `json:"headHash"`
// BrokenAt is the seq of the FIRST record that failed verification, or -1 when
// OK. Reason describes the break (recomputed-hash mismatch, prev-hash
// discontinuity, or a seq gap).
BrokenAt int64 `json:"brokenAt"`
Reason string `json:"reason,omitempty"`
}
// Verify walks the entire chain in seq order, recomputing each record's hash from
// its content + the running prev-hash and checking continuity. It is the
// tamper-detector: any modification (a changed field re-hashes differently), any
// deletion or reordering (a seq gap or a broken prev-hash link), or a forged row
// (its recomputed hash won't match unless the attacker also recomputed the entire
// suffix — which they cannot do without re-inserting every subsequent record) is
// reported with the exact seq where the chain first breaks.
//
// Complexity is O(n) over the records; for very large trails this streams row by
// row (no full materialization). At cloud's audit volume this is fine; if a trail
// grows past what an on-demand full walk should touch, verify a seq WINDOW
// (Verify is easily extended with a bound) or rely on the externally-pinned head.
func (r *Recorder) Verify(ctx context.Context) (Integrity, error) {
rs, err := r.db.QueryContext(ctx,
`SELECT `+selectCols+` FROM audit_log ORDER BY seq ASC`)
if err != nil {
return Integrity{}, fmt.Errorf("audit: verify query: %w", err)
}
defer func() { _ = rs.Close() }()
prevHash := genesisPrevHash
var expectSeq uint64
var count uint64
headHash := genesisPrevHash
for rs.Next() {
rec, scanErr := scanRecord(rs)
if scanErr != nil {
return Integrity{}, fmt.Errorf("audit: verify scan: %w", scanErr)
}
// Gapless, 0-based ordering.
if rec.Seq != expectSeq {
return Integrity{
OK: false, Count: count, HeadHash: headHash,
BrokenAt: int64(rec.Seq),
Reason: fmt.Sprintf("seq gap: expected %d, got %d", expectSeq, rec.Seq),
}, nil
}
// Link continuity: this record must chain to the previous record's hash.
if rec.PrevHash != prevHash {
return Integrity{
OK: false, Count: count, HeadHash: headHash,
BrokenAt: int64(rec.Seq),
Reason: "prev_hash discontinuity (a record was deleted, reordered, or altered)",
}, nil
}
// Content integrity: recompute the hash from the record's own fields.
want, hErr := computeHash(rec, rec.PrevHash)
if hErr != nil {
return Integrity{}, fmt.Errorf("audit: verify hash: %w", hErr)
}
if want != rec.Hash {
return Integrity{
OK: false, Count: count, HeadHash: headHash,
BrokenAt: int64(rec.Seq),
Reason: "hash mismatch (record content was modified after it was written)",
}, nil
}
prevHash = rec.Hash
headHash = rec.Hash
expectSeq = rec.Seq + 1
count++
}
if err := rs.Err(); err != nil {
return Integrity{}, fmt.Errorf("audit: verify rows: %w", err)
}
return Integrity{OK: true, Count: count, HeadHash: headHash, BrokenAt: -1}, nil
}
audit/record.go
+176
View File
@@ -0,0 +1,176 @@
// Package audit is the unified cloud binary's compliance-grade audit trail —
// tamper-evident, append-only, and complete over the security-relevant request
// surface (FedRAMP AU-* / SOC 2 CC-* controls).
//
// THE CONTROL, IN ONE SENTENCE. Every security-relevant action against this
// binary is captured as a structured Record, hash-chained to its predecessor so
// any later deletion or modification is detectable, and written INLINE (never
// dropped) to an append-only store the application can only INSERT into.
//
// THREE PIECES, EACH IN ITS LANE (orthogonal, per the Zen of Hanzo):
// - record.go — the event model + the hash-chain math (what a record IS and
// how it links to the one before it). Pure, no I/O.
// - store.go — the append-only sink (SQLite primary, INSERT-only; a
// best-effort OLAP mirror) and the serialized Recorder that
// owns the chain head. All persistence.
// - redact.go — the secret-stripping allowlist/denylist for any structured
// before/after an explicit emit point supplies. No secret ever
// reaches a record.
//
// The HTTP middleware (Middleware, in the cloud package) and the query/verify
// endpoints (in clients/admin) are thin callers of this package. This package
// holds the security logic; it has zero knowledge of routes.
package audit
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"time"
)
// Actor identifies WHO performed the action. It is populated ONLY from a
// validated principal (the sanitized X-User-* headers SanitizeIdentity mints
// from a verified IAM JWT), never from a raw client header — so an actor can
// never be forged by the request that is being audited. A service principal
// (M2M / no user sub) records Org with an empty Sub.
type Actor struct {
// Org is the tenant (IAM `owner`). Empty for an unauthenticated request.
Org string `json:"org"`
// Sub is the user id (IAM `sub`/`preferred_username`). Empty for a service
// principal or an anonymous request.
Sub string `json:"sub"`
// Email is the validated user email, when present.
Email string `json:"email,omitempty"`
}
// Resource identifies WHAT was acted upon: a type (e.g. "org", "role",
// "secret", "provider-config", "credit") and its id. For a plain HTTP mutation
// with no finer resource semantics, Type is the route family and ID is empty —
// the Action verb + path already pin the object.
type Resource struct {
Type string `json:"type"`
ID string `json:"id,omitempty"`
}
// AuthContext records HOW the actor authenticated and what authority they held
// at decision time — the AC-* evidence (was this a global admin? by what
// credential?). Method is "jwt" | "api-key" | "none". IsAdmin is the VALIDATED
// global-admin bit (owner == AdminOrg), never a raw X-User-IsAdmin.
type AuthContext struct {
Method string `json:"method"`
IsAdmin bool `json:"isAdmin"`
}
// Outcome is the result of the action: whether it was allowed and what
// happened. Result is "success" | "deny" | "error". Status is the HTTP status.
// Reason is a short, non-sensitive explanation for a deny/error (e.g.
// "global admin required", "insufficient_balance") — never a secret, never a
// raw upstream error body.
type Outcome struct {
Result string `json:"result"`
Status int `json:"status"`
Reason string `json:"reason,omitempty"`
}
// Record is one audit event. The JSON tags ARE the on-disk and on-wire contract.
//
// Field order in the struct is deliberate but IRRELEVANT to the hash: the chain
// hashes the CANONICAL (sorted-key) JSON of the record with Hash/PrevHash zeroed
// (see canonicalBytes), so re-ordering fields or adding an omitempty field can
// never change an existing record's hash.
type Record struct {
// Seq is the strictly-increasing chain position (0-based). It is assigned by
// the Recorder under its lock, so it is a true total order with no gaps.
Seq uint64 `json:"seq"`
// Time is the UTC event timestamp (RFC3339Nano).
Time time.Time `json:"time"`
// Actor / Action / Resource / Auth / Outcome — the AU-3 "content of audit
// records" core: who, what, on what, how-authenticated, with what result.
Actor Actor `json:"actor"`
Action string `json:"action"`
Resource Resource `json:"resource"`
Auth AuthContext `json:"auth"`
Outcome Outcome `json:"outcome"`
// SourceIP + UserAgent — the AU-3 "source of the event" fields.
SourceIP string `json:"sourceIp,omitempty"`
UserAgent string `json:"userAgent,omitempty"`
// RequestID correlates the record to the request-line log and any downstream
// trace (the X-Request-Id the pipeline mints).
RequestID string `json:"requestId,omitempty"`
// Method + Path are the HTTP verb and route for a request-sourced event.
Method string `json:"method,omitempty"`
Path string `json:"path,omitempty"`
// Before / After capture a mutation's prior and resulting state for the
// AU-required "before/after" on config-affecting changes. They are populated
// ONLY by explicit emit points and ONLY after Redact has stripped secrets —
// the HTTP middleware never sets them (it never reads bodies), so a secret in
// a request body can never leak here. Raw JSON so any shape round-trips.
Before json.RawMessage `json:"before,omitempty"`
After json.RawMessage `json:"after,omitempty"`
// PrevHash is the hash of record Seq-1 (hex). For the genesis record (Seq 0)
// it is genesisPrevHash. Hash is this record's hash. Neither participates in
// its own hash computation (both are zeroed in canonicalBytes).
PrevHash string `json:"prevHash"`
Hash string `json:"hash"`
}
// genesisPrevHash is the PrevHash of the first record in a fresh chain: 32 zero
// bytes, hex-encoded. A non-empty, fixed anchor so the genesis record's hash is
// still a function of a known constant (not the empty string, which would be
// indistinguishable from "field omitted").
const genesisPrevHash = "0000000000000000000000000000000000000000000000000000000000000000"
// canonicalBytes returns the deterministic byte string a record hashes over: the
// record with Hash AND PrevHash zeroed, marshaled by encoding/json (which sorts
// struct fields in declaration order and, critically, is stable for a given
// struct — the SAME bytes on every machine and every run). Zeroing PrevHash here
// means the hash covers only the record's OWN content; the link to the previous
// record is added explicitly in computeHash by appending prevHash. This keeps
// the two concerns separable and the math obvious.
//
// We marshal a copy with the two hash fields cleared rather than a parallel
// struct so there is exactly ONE definition of a record's fields (DRY): add a
// field to Record and it is covered by the hash automatically.
func canonicalBytes(r Record) ([]byte, error) {
r.Hash = ""
r.PrevHash = ""
return json.Marshal(r)
}
// computeHash returns the hex SHA-256 of (canonical(record) || prevHash-bytes).
// The prevHash is folded in as its RAW hex string bytes — the exact value stored
// in the record's PrevHash field — so the verifier reproduces it byte-for-byte
// from stored data alone. Any change to the record's content OR to which record
// precedes it changes this output, which is the whole tamper-evidence property.
func computeHash(r Record, prevHash string) (string, error) {
body, err := canonicalBytes(r)
if err != nil {
return "", err
}
h := sha256.New()
h.Write(body)
h.Write([]byte(prevHash))
return hex.EncodeToString(h.Sum(nil)), nil
}
// seal finalizes a record into position seq linked to prevHash: it stamps Seq
// and PrevHash, computes Hash, and returns the sealed record ready to append.
// The Recorder calls this under its lock so seq/prevHash reflect the true head.
func seal(r Record, seq uint64, prevHash string) (Record, error) {
r.Seq = seq
r.PrevHash = prevHash
hash, err := computeHash(r, prevHash)
if err != nil {
return Record{}, err
}
r.Hash = hash
return r, nil
}
audit/redact.go
+140
View File
@@ -0,0 +1,140 @@
package audit
// Secret redaction for the before/after captured on a mutation.
//
// THE RULE. An audit record must NEVER contain a credential — no password,
// token, API key, private key, card number, or session secret. Two layers
// enforce this:
//
// 1. The HTTP middleware captures METADATA ONLY (actor/action/resource/outcome).
// It NEVER reads a request or response body, so a secret in a POST body can
// never reach a record through the automatic path. This is the primary
// guarantee: the code that can't see a secret can't leak one.
//
// 2. An EXPLICIT emit point that supplies structured before/after (e.g. a config
// change diff) runs it through Redact first. Redact walks the JSON and
// replaces the VALUE of any key whose name matches the secret denylist with a
// fixed marker, recursively. It is deny-by-key-name — the same allowlist
// PATTERN cloud already uses for user-secret redaction — chosen because a
// mutation diff has arbitrary shape and key-name matching is the robust,
// well-understood control (vs. trying to detect "secret-looking" values).
//
// Redact is conservative: on any structural surprise it returns the redaction
// marker rather than the input, so a parser edge case fails CLOSED (no raw
// passthrough).
import (
"encoding/json"
"strings"
)
// redactedMarker replaces every redacted value. A constant so tests and the
// query UI recognize it unambiguously.
const redactedMarker = "[REDACTED]"
// secretKeyParts are substrings that, when contained (case-insensitively) in a
// JSON object key, mark that key's value as secret. Kept as a small, auditable
// denylist of the credential-bearing field names that actually occur across the
// Hanzo surface (IAM, KMS, commerce, provider config). Matching is substring so
// "clientSecret", "api_key", "PRIVATE_KEY", "accessToken" all match.
var secretKeyParts = []string{
"password",
"passwd",
"secret",
"token",
"apikey",
"api_key",
"api-key",
"authorization",
"auth_token",
"private_key",
"privatekey",
"privkey", // privkey, wgPrivKey
"passphrase",
"client_secret",
"credential",
"session",
"cookie",
"card", // card_number, cardNumber
"cvv",
"cvc",
"pin",
"ssn",
"social_security", // socialSecurityNumber (lower-cased match covers camelCase)
"socialsecurity",
"otp",
"mnemonic",
"phrase", // seed_phrase, seedPhrase, recoveryPhrase
"access_key",
"secret_key",
"refresh_token",
"id_token",
"bearer",
"signing_key",
"encryption_key",
}
// isSecretKey reports whether a JSON key names a credential-bearing field.
func isSecretKey(key string) bool {
k := strings.ToLower(key)
for _, part := range secretKeyParts {
if strings.Contains(k, part) {
return true
}
}
return false
}
// Redact returns a copy of the JSON value with every secret-keyed value replaced
// by the redaction marker, recursively through objects and arrays. Non-JSON or
// empty input yields nil (nothing to record). On a JSON parse error the input is
// dropped (returns the marker as a JSON string) rather than passed through —
// fail closed.
//
// Use it at any explicit emit point that supplies before/after:
//
// audit.Emit(ctx, rec.WithChange(audit.Redact(before), audit.Redact(after)))
func Redact(raw json.RawMessage) json.RawMessage {
if len(raw) == 0 {
return nil
}
var v any
if err := json.Unmarshal(raw, &v); err != nil {
// Unparseable — never echo it back verbatim; record a marker instead.
b, _ := json.Marshal(redactedMarker)
return b
}
cleaned := redactValue("", v)
out, err := json.Marshal(cleaned)
if err != nil {
b, _ := json.Marshal(redactedMarker)
return b
}
return out
}
// redactValue walks a decoded JSON value. key is the object key under which v
// sits (empty at the root and for array elements); when key is a secret key, the
// ENTIRE value v is replaced (whether it is a scalar, object, or array — a secret
// nested object is redacted whole). Otherwise objects/arrays are recursed.
func redactValue(key string, v any) any {
if key != "" && isSecretKey(key) {
return redactedMarker
}
switch t := v.(type) {
case map[string]any:
out := make(map[string]any, len(t))
for k, val := range t {
out[k] = redactValue(k, val)
}
return out
case []any:
out := make([]any, len(t))
for i, val := range t {
out[i] = redactValue("", val) // array elements inherit no key
}
return out
default:
return v
}
}
audit/store.go
+378
View File
@@ -0,0 +1,378 @@
package audit
// The append-only sink + the serialized Recorder that owns the hash-chain head.
//
// WHY SQLITE IS THE PRIMARY, DURABLE STORE (not ClickHouse). The chain is only
// tamper-EVIDENT if records are appended in a strict, gapless total order and
// each record's PrevHash is the immediately-preceding record's Hash. That demands
// a single serializing writer with a synchronous, read-your-write head. cloud's
// canonical store is embedded SQLite (one store per the storagelock lockdown;
// pricing/provisioning already persist to {DataDir}/*.db). A local SQLite table
// the application can only INSERT into gives us: (a) a real total order under one
// connection, (b) synchronous durability so NO record is ever lost on the request
// path (unlike a fire-and-forget mirror), and (c) an append-only surface — the
// app issues no UPDATE/DELETE, and the hash-chain detects any out-of-band edit to
// the file. That is the compliance-grade primary control.
//
// THE CLICKHOUSE MIRROR IS A PROJECTION, NOT THE SOURCE OF TRUTH. The datastore
// (ClickHouse MergeTree — insert-only, mutation-rejected at parse time) is the
// fleet-wide OLAP mirror for long-retention, cross-deployment query. It is
// best-effort and asynchronous: a mirror outage must never block or fail an
// audited request, and the local chain remains the authority the verifier walks.
// Losing a mirror row is a query-completeness issue, not an integrity one.
//
// FAIL MODE. Append is INLINE and its error is RETURNED to the middleware, which
// fails the request CLOSED (a security-relevant action that cannot be recorded is
// not permitted to silently succeed). This is the AU-5 "response to audit logging
// process failure": deny rather than act-unlogged.
import (
"context"
"database/sql"
"errors"
"fmt"
"sync"
"time"
// modernc.org/sqlite is the pure-Go SQLite driver already in the cloud dep
// graph (see clients/pricing, clients/provisioning). Blank import registers
// the "sqlite" driver name.
_ "modernc.org/sqlite"
)
// Mirror is the optional OLAP projection sink (the datastore/ClickHouse). It is
// deliberately a tiny interface, not a concrete client, so the Recorder has no
// compile-time dependency on ClickHouse and tests can supply a fake. Append is
// called best-effort, asynchronously, off the request path.
type Mirror interface {
// Append writes one sealed record to the projection. A returned error is
// logged and dropped by the Recorder — the mirror never gates a request.
Append(ctx context.Context, r Record) error
}
// Checkpoint is a periodic, tamper-EVIDENCE digest of the chain head: the record
// count and the head hash at a moment in time. It is the AU-9 anchor for
// TAIL-TRUNCATION detection — an internal chain walk cannot notice that the last
// K records were deleted (the surviving prefix still verifies), but a durable,
// INDEPENDENT series of head checkpoints can: Count is monotonic, so any decrease
// between two consecutive checkpoints is deletion, and an attacker cannot forge a
// higher count without appending records whose hashes the chain walk would reject.
type Checkpoint struct {
Time time.Time `json:"time"`
Count uint64 `json:"count"`
Head string `json:"head"`
}
// CheckpointSink is an optional capability a Mirror may implement to persist the
// head digest series to an INDEPENDENT store (so truncating the local SQLite
// cannot also rewrite the anchor history). A Mirror that does not implement it
// still gets its records; checkpoints then flow only to the structured log.
type CheckpointSink interface {
Checkpoint(ctx context.Context, cp Checkpoint) error
}
// Recorder is the single serialized writer that owns the audit chain head and
// the append-only store. Every Record flows through Append, which under one lock
// assigns the next Seq, links PrevHash to the current head, seals (hashes), and
// synchronously persists to SQLite before returning. Concurrency is serialized
// by mu AND by the single-connection SQLite pool, so the on-disk order equals
// the chain order with no gaps.
type Recorder struct {
db *sql.DB
mirror Mirror // nil when no OLAP mirror is configured.
mu sync.Mutex // guards nextSeq/headHash and serializes appends.
nextSeq uint64 // Seq to assign to the next record.
headHash string // Hash of the last-appended record (PrevHash for the next).
// Checkpoint emission (AU-9 tail-truncation anchor). logCheckpoint, when set,
// receives each head digest so it lands in the append-only observability log;
// stopCh/wg manage the periodic emitter goroutine's lifecycle; started guards
// against a second StartCheckpoints call (the field write + WaitGroup use are
// not safe to race). All three are set once, before any concurrent Append.
logCheckpoint func(cp Checkpoint)
stopCh chan struct{}
wg sync.WaitGroup
started bool
}
// CheckpointFunc receives a head digest for the structured (o11y) log. It is a
// plain func so the pure audit package stays free of any concrete logger type;
// the cloud wiring adapts luxlog to it.
type CheckpointFunc func(cp Checkpoint)
// Open opens (creating if needed) the append-only audit DB at path and recovers
// the chain head from it, so a restart continues the SAME chain rather than
// forking a new one. path may be ":memory:" for tests. mirror may be nil.
//
// modernc's "sqlite" driver; MaxOpenConns(1) serializes every statement against
// the file lock — the same single-writer discipline pricing/provisioning use,
// here doubling as the chain's serialization guarantee.
func Open(path string, mirror Mirror) (*Recorder, error) {
db, err := sql.Open("sqlite", path)
if err != nil {
return nil, fmt.Errorf("audit: open sqlite %q: %w", path, err)
}
db.SetMaxOpenConns(1)
for _, pragma := range []string{
"PRAGMA busy_timeout=5000",
"PRAGMA journal_mode=WAL",
"PRAGMA synchronous=NORMAL",
} {
if _, err := db.Exec(pragma); err != nil {
_ = db.Close()
return nil, fmt.Errorf("audit: pragma %q: %w", pragma, err)
}
}
r := &Recorder{db: db, mirror: mirror}
if err := r.migrate(); err != nil {
_ = db.Close()
return nil, err
}
if err := r.recoverHead(); err != nil {
_ = db.Close()
return nil, err
}
return r, nil
}
// migrate creates the append-only audit table. It is INSERT-only by application
// discipline: this package issues no UPDATE or DELETE against it, and seq is the
// PRIMARY KEY so a replayed/duplicated seq is rejected by the engine. The hash
// columns make any out-of-band row edit detectable by Verify regardless of the
// storage layer's own guarantees.
func (r *Recorder) migrate() error {
const ddl = `
CREATE TABLE IF NOT EXISTS audit_log (
seq INTEGER PRIMARY KEY, -- chain position; gapless, assigned under lock
ts TEXT NOT NULL, -- RFC3339Nano UTC event time
actor_org TEXT NOT NULL DEFAULT '',
actor_sub TEXT NOT NULL DEFAULT '',
actor_email TEXT NOT NULL DEFAULT '',
action TEXT NOT NULL,
res_type TEXT NOT NULL DEFAULT '',
res_id TEXT NOT NULL DEFAULT '',
auth_method TEXT NOT NULL DEFAULT '',
is_admin INTEGER NOT NULL DEFAULT 0,
result TEXT NOT NULL, -- success|deny|error
status INTEGER NOT NULL DEFAULT 0,
reason TEXT NOT NULL DEFAULT '',
source_ip TEXT NOT NULL DEFAULT '',
user_agent TEXT NOT NULL DEFAULT '',
request_id TEXT NOT NULL DEFAULT '',
method TEXT NOT NULL DEFAULT '',
path TEXT NOT NULL DEFAULT '',
before TEXT NOT NULL DEFAULT '', -- redacted JSON (explicit emit only)
after TEXT NOT NULL DEFAULT '', -- redacted JSON (explicit emit only)
prev_hash TEXT NOT NULL,
hash TEXT NOT NULL
);
-- Query indexes for the /v1/admin/audit filters (actor/action/resource/time).
CREATE INDEX IF NOT EXISTS ix_audit_org_seq ON audit_log(actor_org, seq);
CREATE INDEX IF NOT EXISTS ix_audit_action_seq ON audit_log(action, seq);
CREATE INDEX IF NOT EXISTS ix_audit_result_seq ON audit_log(result, seq);
CREATE INDEX IF NOT EXISTS ix_audit_ts ON audit_log(ts);
`
if _, err := r.db.Exec(ddl); err != nil {
return fmt.Errorf("audit: migrate: %w", err)
}
return nil
}
// recoverHead loads the highest-seq record so a restarted process continues the
// existing chain (nextSeq = maxSeq+1, headHash = its hash). An empty table starts
// the genesis chain (nextSeq 0, headHash = genesisPrevHash).
func (r *Recorder) recoverHead() error {
var (
maxSeq sql.NullInt64
hash sql.NullString
)
row := r.db.QueryRow(`SELECT seq, hash FROM audit_log WHERE seq = (SELECT MAX(seq) FROM audit_log)`)
if err := row.Scan(&maxSeq, &hash); err != nil {
if errors.Is(err, sql.ErrNoRows) {
r.nextSeq = 0
r.headHash = genesisPrevHash
return nil
}
return fmt.Errorf("audit: recover head: %w", err)
}
if !maxSeq.Valid { // empty table (MAX over zero rows is NULL)
r.nextSeq = 0
r.headHash = genesisPrevHash
return nil
}
r.nextSeq = uint64(maxSeq.Int64) + 1
r.headHash = hash.String
return nil
}
// Append seals r into the next chain position and persists it. It fills Seq,
// PrevHash, and Hash (the caller sets everything else), advances the in-memory
// head only AFTER the durable INSERT succeeds, and mirrors best-effort. A
// persistence error is returned so the caller can fail the request CLOSED — the
// head is NOT advanced on failure, so the chain never gaps.
//
// The whole critical section (assign seq → seal → INSERT → advance head) holds
// mu, so two concurrent requests can never claim the same seq or race the head.
func (r *Recorder) Append(ctx context.Context, rec Record) (Record, error) {
if rec.Time.IsZero() {
rec.Time = time.Now().UTC()
} else {
rec.Time = rec.Time.UTC()
}
r.mu.Lock()
defer r.mu.Unlock()
sealed, err := seal(rec, r.nextSeq, r.headHash)
if err != nil {
return Record{}, fmt.Errorf("audit: seal: %w", err)
}
if err := r.insert(ctx, sealed); err != nil {
// Head not advanced; the next append reuses this seq. Fail closed upstream.
return Record{}, fmt.Errorf("audit: persist: %w", err)
}
// Durable — advance the chain head.
r.nextSeq = sealed.Seq + 1
r.headHash = sealed.Hash
// Best-effort OLAP mirror, detached so a slow/failed mirror never blocks the
// request or corrupts the reply. The local chain is already durable and is the
// authority; a lost mirror row is a query-completeness gap, not an integrity
// one. Copy by value: the record is immutable and safe to hand to a goroutine.
if r.mirror != nil {
m, out := r.mirror, sealed
go func() { _ = m.Append(context.Background(), out) }()
}
return sealed, nil
}
// insert writes one sealed record. INSERT-only — the sole write statement in this
// package. A duplicate seq (PRIMARY KEY) fails here, which is the desired
// invariant: the chain never overwrites a position.
func (r *Recorder) insert(ctx context.Context, rec Record) error {
_, err := r.db.ExecContext(ctx, `
INSERT INTO audit_log (
seq, ts, actor_org, actor_sub, actor_email, action, res_type, res_id,
auth_method, is_admin, result, status, reason, source_ip, user_agent,
request_id, method, path, before, after, prev_hash, hash
) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)`,
rec.Seq, rec.Time.Format(time.RFC3339Nano),
rec.Actor.Org, rec.Actor.Sub, rec.Actor.Email,
rec.Action, rec.Resource.Type, rec.Resource.ID,
rec.Auth.Method, boolToInt(rec.Auth.IsAdmin),
rec.Outcome.Result, rec.Outcome.Status, rec.Outcome.Reason,
rec.SourceIP, rec.UserAgent, rec.RequestID, rec.Method, rec.Path,
string(rec.Before), string(rec.After), rec.PrevHash, rec.Hash,
)
return err
}
// checkpointCloseTimeout bounds the final (synchronous) checkpoint write to the
// independent sink at shutdown, so Close cannot hang on an unreachable datastore.
const checkpointCloseTimeout = 5 * time.Second
// Close stops the periodic checkpoint emitter, emits a FINAL checkpoint
// SYNCHRONOUSLY (so the head at shutdown reaches both the o11y log and the
// independent digest store before the process exits — the AU-9 anchor must be
// current exactly when an attacker might trigger shutdown then truncate), and
// closes the underlying database.
func (r *Recorder) Close() error {
if r == nil || r.db == nil {
return nil
}
if r.stopCh != nil {
close(r.stopCh)
r.wg.Wait()
r.stopCh = nil
}
// Anchor the final head before the DB closes — independent of whether the
// periodic ticker was running (every<=0 still gets a shutdown checkpoint).
// Synchronous to the sink (bounded), so the independent store's last count is
// as fresh as the local chain at the moment of shutdown.
r.emitCheckpoint(true)
return r.db.Close()
}
// StartCheckpoints begins periodic head-digest emission every `every` (no ticker
// if every<=0; the on-Close checkpoint still fires). logFn, when non-nil,
// receives each digest for the append-only observability log (o11y), and a mirror
// implementing CheckpointSink also gets it persisted to an INDEPENDENT store —
// together the AU-9 anchor an external monitor compares to detect tail-truncation
// (count regression). MUST be called at most once, before any concurrent Append
// (a second call is ignored); the emitter stops on Close.
func (r *Recorder) StartCheckpoints(every time.Duration, logFn CheckpointFunc) {
// Guard the check-and-set under mu so a (mis)use that calls this concurrently
// is race-free, not just the single-call production path.
r.mu.Lock()
if r.started {
r.mu.Unlock()
return // already started — do not re-arm (avoids a field/WaitGroup race).
}
r.started = true
r.logCheckpoint = logFn
r.mu.Unlock()
if every <= 0 {
return
}
r.stopCh = make(chan struct{})
stop := r.stopCh
r.wg.Add(1)
go func() {
defer r.wg.Done()
t := time.NewTicker(every)
defer t.Stop()
for {
select {
case <-stop:
return
case <-t.C:
r.emitCheckpoint(false)
}
}
}()
}
// emitCheckpoint snapshots the head and emits it to the log and (if the mirror is
// a CheckpointSink) the independent digest store. sync controls the sink write:
// on the periodic path (sync=false) it is detached so a slow sink never delays
// the ticker; on the Close path (sync=true) it BLOCKS on a bounded context so the
// final anchor is durable before shutdown. The log emission is always synchronous
// (it is the primary anchor and o11y ingests it append-only).
func (r *Recorder) emitCheckpoint(sync bool) {
count, head := r.Head()
cp := Checkpoint{Time: time.Now().UTC(), Count: count, Head: head}
if r.logCheckpoint != nil {
r.logCheckpoint(cp)
}
cs, ok := r.mirror.(CheckpointSink)
if !ok || cs == nil {
return
}
if sync {
ctx, cancel := context.WithTimeout(context.Background(), checkpointCloseTimeout)
defer cancel()
_ = cs.Checkpoint(ctx, cp)
return
}
go func() { _ = cs.Checkpoint(context.Background(), cp) }()
}
// Head returns the current chain head (count of records, and the head hash). A
// count of 0 means the genesis (empty) chain, headHash == genesisPrevHash. An
// external monitor can pin (count, headHash) over time to detect tail-truncation
// — which an internal chain walk alone cannot catch (a truncated prefix still
// verifies). This is the anchor point for AU-9 protection against deletion of the
// most-recent records.
func (r *Recorder) Head() (count uint64, headHash string) {
r.mu.Lock()
defer r.mu.Unlock()
return r.nextSeq, r.headHash
}
func boolToInt(b bool) int {
if b {
return 1
}
return 0
}
audit_middleware.go
+677
View File
@@ -0,0 +1,677 @@
package cloud
// The audit middleware — the ONE place every security-relevant request is
// recorded to the tamper-evident trail (decomplected: one function, every route).
//
// PLACEMENT (why it sits exactly where serve.go puts it). The pipeline is
// Recover → RequestID → Logger → SanitizeIdentity → AuditTrail → BillingGate →
// subsystems. AuditTrail runs:
// - AFTER SanitizeIdentity, so the actor/isAdmin it records come from a
// VALIDATED IAM principal (the sanitized X-User-* headers), never a raw
// client header. The request being audited cannot forge its own actor.
// - BEFORE BillingGate and every subsystem, so it WRAPS the whole handler
// chain and observes the FINAL outcome — including a 402/503 billing denial
// and a 403 admin-guard denial (both security-relevant) — via the response
// status after Continue(), exactly like the Logger middleware reads it.
//
// WHAT IT CAPTURES: metadata only — actor, action (method+route family),
// resource, source ip, user agent, request id, auth context, and the outcome
// (result/status/reason). It NEVER reads the request or response BODY, so a
// secret in a POST body can never reach a record through this path. before/after
// diffs are the job of explicit emit points (audit.Recorder.Append with a
// redacted diff), not this middleware.
//
// WHAT IT RECORDS (the coverage predicate, auditable in one place — see
// isSecurityRelevant): every mutating request (POST/PUT/PATCH/DELETE), every
// /v1/admin/* request (read or write — admin reads are AC-relevant), and every
// auth-failure outcome (401/403) on ANY method (a denied GET is an access-control
// event). Safe, unauthenticated reads (a 200 GET on a public route) are NOT
// audited — that is request-log noise, not a security event, and auditing it
// would bury the signal and balloon the trail.
//
// FAIL MODE (AU-5): if the trail write fails on a request we decided to audit,
// the CLIENT gets a fail-closed 503 rather than a success it can rely on — the
// AU-5 "response to an audit logging process failure" is to interrupt, not to
// operate silently unlogged. A write to local SQLite is sub-millisecond, so this
// is a real integrity stance, not a latency tax. When no Recorder is configured
// the middleware is a no-op passthrough (an unconfigured deployment is never
// blocked), exactly like BillingGate.
//
// PRECISE SEMANTIC (do not over-read the 503). This is POST-RESPONSE audit: the
// handler has already run when the record is written, so a 503 here means "this
// event could not be RECORDED", NOT "the action did not execute". PREVENTION is
// the access-control layer's job and runs BEFORE the action — SanitizeIdentity
// (identity can't be forged) + the per-route admin guard both execute inside
// c.Next() ahead of any side effect. The audit trail's job is DETECTION and
// ACCOUNTABILITY (tamper-evident record of what happened), which it does. On a
// persistent audit-store outage every mutation returns 503 (loud, logged), so
// the system degrades to read-only rather than mutating unaudited — the intended
// compliance posture.
//
// PANIC BOUND. If a handler PANICS, the outermost middleware.Recover catches it
// and renders 500 with a full stack trace (loud, never silent); the panic unwinds
// PAST this middleware's post-c.Next() code, so a panicking request is not written
// to the trail. This is an accepted bound, not an evasion: an attacker cannot turn
// a panic into a SUCCESSFUL-but-unaudited mutation (a panic yields 500, not a
// completed action), and every panic is already captured by Recover's logging.
// Normal outcomes — including billing 402/503 and admin 403 denials, which return
// through c.Next() rather than panicking — are always audited.
import (
"errors"
"net/url"
"strings"
"github.com/hanzoai/cloud/audit"
"github.com/hanzoai/zip"
)
// AuditTrail returns the audit middleware bound to rec. A nil rec makes it a
// no-op passthrough so callers always Use() it unconditionally.
func AuditTrail(rec *audit.Recorder) zip.Handler {
if rec == nil {
return func(c *zip.Ctx) error { return c.Next() }
}
return func(c *zip.Ctx) error {
// Capture the pre-decision inputs BEFORE running the chain: the request
// context is recycled by Fiber after the handler returns, so identity and
// request fields must be read now (mirrors BillingGate capturing usage by
// value pre-Record).
method := c.Method()
path := c.Path()
err := c.Next()
// Resolve the EFFECTIVE status. A handler may set it on the response
// directly (c.Status(...).JSON(...)) OR return a *zip.HTTPError that the
// framework's error handler renders AFTER this middleware unwinds — in the
// latter case the response still reads 200 here, so the returned error is
// the authoritative source of a 401/403. Prefer the error's status when it
// carries one; this is what makes admin-guard denials (which return
// ErrForbidden) get audited as 403.
status := effectiveStatus(c.Fiber().Response().StatusCode(), err)
if !isSecurityRelevant(method, path, status) {
return err // not an audited event; pass the handler result through.
}
record := audit.Record{
Actor: actorFromCtx(c),
Action: method + " " + routeFamily(path),
Resource: resourceFromPath(path),
Auth: authFromCtx(c),
Outcome: outcomeOf(status, err),
SourceIP: ClientIP(c),
// User-Agent is client-controlled free text; a misconfigured/malicious
// client could embed a bearer token in it. Scrub credential-shaped runs
// (and cap length) so the UA can never carry a secret into the record.
UserAgent: scrubFreeText(c.Header("User-Agent")),
RequestID: c.RequestID(),
Method: method,
// Path is scrubbed of any credential-shaped segment: Hanzo routes use
// identifiers (:name/:slug/:id), not secrets, but a token that ever
// rides in the path (an hk-/sk-/pk-/fw_/hz_ key) must never be recorded
// verbatim. resourceFromPath applies the same scrub to the resource id.
Path: scrubCredentialSegments(path),
}
if _, aerr := rec.Append(c.Context(), record); aerr != nil {
// AU-5: could not record a security-relevant event — fail the request
// closed. Do NOT leak the audit error to the client; log it loud.
c.Log().Error("audit append failed — failing request closed",
"path", path, "method", method, "err", aerr)
return c.JSON(503, map[string]any{
"error": map[string]string{
"code": "audit_unavailable",
"message": "Request could not be securely recorded",
},
})
}
return err
}
}
// isSecurityRelevant is the coverage predicate — the ONE place that decides which
// requests enter the audit trail (AC/AU scope). Kept tiny and total so the
// coverage matrix is reviewable at a glance. Order matters and is deliberate:
// the security-relevant conditions are checked FIRST and are unconditional, so
// the health-probe exemption can NEVER be used to evade audit of a mutation or a
// denial (a POST/DELETE, or any 401/403, is always audited whatever the path).
// - any auth-failure outcome (401/403) on any method (a denied access attempt),
// - any /v1/admin/* request (admin reads are access-control-relevant),
// - any mutating request (POST/PUT/PATCH/DELETE).
//
// Only then, a genuine liveness probe (a GET to an exact health route) is
// exempted — it is neither a mutation, a denial, nor an admin call, so it is pure
// request-log noise. The exemption matches EXACT probe paths, never an arbitrary
// path that merely ends in "/health" (which a wildcard/attacker-named segment
// like POST /v1/admin/orgs/x/health could otherwise abuse to slip past audit).
func isSecurityRelevant(method, path string, status int) bool {
// Unconditional security signals — never suppressed by any path shape. A
// denial, an admin call, or a mutation is ALWAYS audited, whatever the path
// (so a wildcard/attacker-named "/health" tail cannot evade it).
if status == 401 || status == 403 {
return true
}
if strings.HasPrefix(path, "/v1/admin/") {
return true
}
if isMutation(method) {
return true
}
// Everything left is a safe read (non-mutating, non-admin, non-denied). None
// are audited — they are request-log noise, not security events. (Liveness
// probes fall here too; there is no separate case because the answer is the
// same: not recorded.)
return false
}
// isMutation reports whether a method changes state.
func isMutation(method string) bool {
switch method {
case "POST", "PUT", "PATCH", "DELETE":
return true
default:
return false
}
}
// actorFromCtx builds the Actor from the sanitized identity headers, gating
// ANTI-FORGERY of the recorded actor on a VALIDATED principal.
//
// The authoritative "this request carried a validated principal" signal is a
// non-empty X-User-Id (c.User()): SanitizeIdentity sets X-User-Id ONLY from a
// JWT it verified, and strips any client-supplied copy on ingress. A request
// with no principal — anonymous, OR one bearing an INVALID/garbage bearer that
// failed validation — has an empty c.User().
//
// In that unvalidated case the org header is NOT trustworthy: SanitizeIdentity's
// Phase-1 residual restores a client-supplied X-Org-Id for the data path, so an
// anonymous attacker could send X-Org-Id: victim-org and, if we recorded it,
// forge a FALSE ATTRIBUTION (an event stamped with a victim's org). So when there
// is no validated sub, the actor is left EMPTY — the record stands as an honest
// anonymous event identified by SourceIP, never mis-attributed to a claimed org.
//
// With a validated sub, org/sub/email all reflect the verified principal and are
// recorded authoritatively.
func actorFromCtx(c *zip.Ctx) audit.Actor {
sub := strings.TrimSpace(c.User())
if sub == "" {
// No validated principal — do not trust the client-asserted org.
return audit.Actor{}
}
return audit.Actor{
Org: strings.TrimSpace(c.Org()),
Sub: sub,
Email: strings.TrimSpace(c.UserEmail()),
}
}
// authFromCtx records HOW the caller authenticated and the VALIDATED admin bit.
// IsAdmin comes from c.IsAdmin() (the sanitized X-User-IsAdmin, true only for a
// verified global admin), never a raw header. Method is inferred from the
// presence/shape of a credential: an Authorization/X-Authorization bearer or a
// session cookie ⇒ "jwt" (or "api-key" for an opaque hk-/sk- token); none ⇒
// "none".
func authFromCtx(c *zip.Ctx) audit.AuthContext {
return audit.AuthContext{
Method: authMethodOf(c),
IsAdmin: c.IsAdmin(),
}
}
// authMethodOf classifies the credential kind WITHOUT capturing it — it inspects
// only the token PREFIX (never stores the value). Order mirrors the sanitizer's
// extraction (bearer, then cookie).
func authMethodOf(c *zip.Ctx) string {
auth := c.Header("Authorization")
if auth == "" {
auth = c.Header("X-Authorization")
}
if tok := bearerFromAuth(auth); tok != "" {
if isAPIKey(tok) {
return "api-key"
}
return "jwt"
}
if basicFromAuth(auth) != "" {
return "basic"
}
for _, name := range cookieTokenNames {
if c.Fiber().Cookies(name) != "" {
return "jwt"
}
}
return "none"
}
// effectiveStatus reconciles the response status with a returned error. If the
// handler returned a *zip.HTTPError (e.g. ErrForbidden), its Status is
// authoritative — the framework renders it after this middleware unwinds, so the
// live response status does not yet reflect it. A non-HTTPError returned error
// with a still-2xx response means the framework will render a 500. Otherwise the
// response status stands.
func effectiveStatus(respStatus int, err error) int {
if err != nil {
var he *zip.HTTPError
if errors.As(err, &he) && he.Status != 0 {
return he.Status
}
// A non-HTTPError propagating up renders as 500, unless the handler already
// set an explicit error status on the response.
if respStatus < 400 {
return 500
}
}
return respStatus
}
// outcomeOf maps the final HTTP status + handler error to an audit Outcome.
// 2xx/3xx ⇒ success; 401/403 ⇒ deny; everything else (4xx/5xx) ⇒ error. The
// reason is a short, non-sensitive label derived from the status class — never a
// raw upstream error body (which could echo sensitive detail).
func outcomeOf(status int, err error) audit.Outcome {
switch {
case status == 401:
return audit.Outcome{Result: "deny", Status: status, Reason: "unauthenticated"}
case status == 403:
return audit.Outcome{Result: "deny", Status: status, Reason: "forbidden"}
case status >= 500:
return audit.Outcome{Result: "error", Status: status, Reason: "server_error"}
case status >= 400:
return audit.Outcome{Result: "error", Status: status, Reason: "client_error"}
default:
return audit.Outcome{Result: "success", Status: status}
}
}
// routeFamily reduces a concrete path to its stable route family for the action
// verb, dropping trailing high-cardinality id segments so "DELETE /v1/admin/
// orgs/acme" and "DELETE /v1/admin/orgs/globex" share the action "DELETE
// /v1/admin/orgs". The full concrete path is preserved separately in Record.Path.
func routeFamily(path string) string {
segs := strings.Split(strings.Trim(path, "/"), "/")
// Keep the leading "v1/<subsystem>/<noun>" and stop before an id-looking tail.
out := make([]string, 0, len(segs))
for _, s := range segs {
if looksLikeID(s) {
break
}
out = append(out, s)
}
if len(out) == 0 {
return "/" + strings.Join(segs, "/")
}
return "/" + strings.Join(out, "/")
}
// resourceFromPath derives the {type,id} resource from a /v1/<subsystem>/<type>/
// [<id>] path. Type is the noun after the subsystem; ID is the following segment
// when it looks like an identifier. Best-effort — the Action verb + Path are the
// authoritative locator; this is a convenience for filtering by resource type.
func resourceFromPath(path string) audit.Resource {
segs := strings.Split(strings.Trim(path, "/"), "/")
// segs: [v1, <subsystem>, <type>, <id?>, ...]
if len(segs) < 3 {
return audit.Resource{}
}
res := audit.Resource{Type: segs[2]}
if len(segs) >= 4 && looksLikeID(segs[3]) {
res.ID = scrubToken(segs[3])
}
return res
}
// scrubToken replaces a path segment that is a credential-shaped token with a
// fixed marker, so a secret that ever appears in a URL is never recorded
// verbatim. It catches, in increasing generality:
// - a known API-key prefix (hk-/sk-/pk-/fw_/hz_ — what isAPIKey recognizes),
// ALSO after percent-decoding, so hk%2DKEY can't slip the prefix check,
// - a JWT (three base64url parts split by '.', starting eyJ),
// - a long, high-entropy base64url/hex run (>=24) — a raw API key / access
// token / hex secret that carries no telltale prefix.
//
// A normal identifier passes through unchanged: a UUID (5 hyphen-split groups,
// each short), a slug, a numeric id, a dotted model name — none is a long
// unbroken high-entropy blob. See TestScrubToken_NoFalsePositives.
//
// RED-review hardening (finding: scrub bypass): the entropy test now (a) accepts
// the FULL base64url alphabet incl. '-' and '_' (RFC 4648 §5), (b) does NOT
// require a digit (an all-alpha opaque key is still a secret), (c) percent-
// decodes first so %2D/%5F can't hide structure, and (d) drops the threshold to
// 24 (short enough for a 128-bit base64 or a 24-hex key, long enough that no
// human-readable slug reaches it).
func scrubToken(seg string) string {
dec := percentDecode(seg)
if isAPIKey(seg) || isAPIKey(dec) ||
looksLikeJWT(seg) || looksLikeJWT(dec) ||
looksLikeHighEntropyToken(seg) || looksLikeHighEntropyToken(dec) {
return "[REDACTED-TOKEN]"
}
return seg
}
// percentDecode best-effort URL-decodes s so a percent-encoded credential
// (hk%2DKEY, sk%5Flive%5F…) is normalized before the credential tests run. On a
// malformed escape it returns s unchanged (the raw form is then tested as-is).
func percentDecode(s string) string {
// Decode repeatedly (bounded) so a NESTED encoding (%252D -> %2D -> -) is
// fully normalized before the credential tests run. Stop when a pass makes no
// change, on a malformed escape, or after a small cap (defeats a decode bomb).
for i := 0; i < 3 && strings.Contains(s, "%"); i++ {
dec, err := url.PathUnescape(s)
if err != nil || dec == s {
break
}
s = dec
}
return s
}
// looksLikeJWT reports whether s is a JSON Web Token OR a JWT header segment. A
// full JWT is three base64url parts split by '.', header starting "eyJ". But when
// free text (a UA) is tokenized on '.', a dotted JWT splits into parts; a real
// header part is >=24 chars (caught by the high-entropy run), yet to be safe we
// ALSO flag any lone segment starting with the canonical base64url header prefix
// "eyJ" (which decodes to '{"') regardless of length — a JWT header can never be
// a legitimate resource id, so redacting it has no false-positive cost.
func looksLikeJWT(s string) bool {
if !strings.HasPrefix(s, "eyJ") {
return false
}
// A full token (two dots) or a bare header segment — either way, redact.
return true
}
// highEntropyMinLen is the length at/above which an UNBROKEN run of base64/hex
// chars is treated as an opaque secret. 24 covers a 128-bit base64 token, a
// 24-nibble hex key, and short API keys, while every human-readable path
// segment/slug/model-name stays under it once split on its separators — no
// single run of a name like "text-embedding-3-large" reaches 24.
const highEntropyMinLen = 24
// looksLikeHighEntropyToken reports whether s CONTAINS an unbroken run of
// >= highEntropyMinLen base64/hex chars — the shape of a raw API key / access
// token / hex secret — UNLESS s is a structured human identifier.
//
// The run alphabet is [A-Za-z0-9_+/-] — the base64 alphabets (url-safe §5 '-”_'
// and standard §4 '+”/') and hex. '-' is INCLUDED so a url-safe-base64 token
// that embeds '-' is still caught by its run (excluding '-' left an ~11% bypass
// for 32-byte url-safe tokens whose '-' happened to break every 24-run — measured).
//
// TWO-STAGE DETECTION:
//
// 1. UNCONDITIONAL run scan — a >= highEntropyMinLen UNBROKEN run over
// [A-Za-z0-9_+/-] flags the value REGARDLESS of the structured-id exemption.
// This catches every raw secret WITHOUT internal separators (hex, base64) —
// the realistic "a client bug put a raw key in the URL" case — at 100%. A
// structured identifier never has a 24-char unbroken run, so this stage never
// over-scrubs one.
//
// 2. STRUCTURED-ID EXEMPTION for the rest (values with separators that DON'T have
// a 24-run): exempt a clearly hyphen-joined human id, judged by LEXICAL
// content — every group WORD-LIKE (single-case word or decimal number), which
// a mixed-case base64 chunk or a long hex-with-letters chunk is NOT (those are
// redacted). Shape alone is attacker-satisfiable (RED found a 3x12-chunked
// secret slipped a shape-only check); the lexical test rejects the common
// secret encodings.
//
// ACCEPTED RESIDUAL BOUND (documented, per RED review): the ONLY residual is a
// secret deliberately chunked into >=3 SINGLE-CASE-ALPHABETIC groups of <=12 chars
// with no hex-letter runs >= hexChunkMinLen (a lowercase- or uppercase-only base32
// alphabet, e.g. abcdefghijkl-mnopqrstuvwx-…). Such a value is lexically
// indistinguishable from a hyphenated model id (deepseek-r1-distill-qwen-32b
// carries the SAME 24-char entropy budget), so it passes stage 2. This is NOT
// closable by any length/case/count rule without a word dictionary
// (over-engineering for a defense-in-depth URL/UA backstop). Mixed-case base64
// chunks AND small-hex chunks (md5/sha display grouping) ARE now caught
// (isWordLikeGroup). The residual does not widen exposure for any REAL credential:
// Hanzo keys are hk-/sk-/pk-/fw_/hz_-prefixed (isAPIKey, caught at any
// length/shape), JWTs are eyJ-prefixed (looksLikeJWT), and request/response BODIES
// are never read. It is an adversary DELIBERATELY base32-chunking their OWN secret
// into a URL path to seed an admin-only audit row — contrived, low-value. The
// realistic accidental leak (an unbroken raw key) is caught by stage 1.
//
// A canonical UUID is exempt (its longest run is 12; the check documents intent).
func looksLikeHighEntropyToken(s string) bool {
if len(s) < highEntropyMinLen {
return false
}
// Stage 1 — unconditional: a long unbroken run is always a secret.
if hasHighEntropyRun(s) {
return true
}
// Stage 2 — separated values: redact unless it's a lexical structured id.
if isUUID(s) || isStructuredID(s) {
return false
}
// A >=24-length value with separators, not a UUID, not a structured id — e.g.
// "sk.live.LONGSECRET…" dotted, or a chunk pattern that is not word-like.
return true
}
// hasHighEntropyRun reports whether s contains an unbroken run of
// >= highEntropyMinLen high-entropy chars where the run alphabet EXCLUDES '-'
// (and '.'): a real separator breaks the run. This is stage 1 — it fires only on
// a genuinely UNBROKEN opaque blob (a raw hex/base64 key with no separators), so
// it never catches a hyphenated identifier (which stage 2 then classifies). A
// url-safe-base64 secret that embeds '-' still trips because the run on ONE side
// of the hyphen is >= 24 (verified: "AbCdEf-GhIjKl_MnOpQrStUvWxYz012345" -> 27).
func hasHighEntropyRun(s string) bool {
run := 0
for _, r := range s {
if isUnbrokenTokenChar(r) {
run++
if run >= highEntropyMinLen {
return true
}
} else {
run = 0
}
}
return false
}
// isUnbrokenTokenChar is the stage-1 run alphabet: base64/hex MINUS '-' (and the
// implicit exclusion of '.', space, etc.). '_' '+' '/' are kept — they appear
// inside opaque tokens and are not identifier separators.
func isUnbrokenTokenChar(r rune) bool {
return (r >= 'A' && r <= 'Z') || (r >= 'a' && r <= 'z') ||
(r >= '0' && r <= '9') || r == '_' || r == '+' || r == '/'
}
// idPartMaxLen bounds a hyphen-group length in the structured-id exemption. 12
// covers the longest word in real model ids ("embedding", "20241022", "preview")
// while a raw secret's random hyphen groups routinely exceed it.
const idPartMaxLen = 12
// isStructuredID reports whether s is a hyphen-joined human identifier (a model
// name, slug): >= 3 hyphen groups where EVERY group is non-empty, <= idPartMaxLen
// chars, and WORD-LIKE. The word-like test is the anti-bypass core — a raw secret
// chunk cannot satisfy it — so the exemption is not attacker-satisfiable by shape.
func isStructuredID(s string) bool {
groups := strings.Split(s, "-")
if len(groups) < 3 {
return false
}
for _, g := range groups {
if g == "" || len(g) > idPartMaxLen || !isWordLikeGroup(g) {
return false
}
}
return true
}
// isWordLikeGroup reports whether a hyphen group looks like a model-id token (a
// dictionary word or a decimal number) rather than a random secret chunk. Two
// lexical signals reject a secret chunk:
// - MIXED CASE (both upper and lower letters) — base64 tokens are dense
// mixed-case; real model tokens are single-case ("sonnet", "Instruct", "3").
// - an all-hex-with-letters run (>= hexChunkMinLen chars, all [0-9a-fA-F], and
// not all-digits) — a hex secret chunk ("dead", "beef", "cafebabe"); a version
// date ("20241022", all digits) is NOT hex-with-letters, so it stays
// word-like. The threshold is 4: RED re-review verified that NO real model-id
// group is all-hex-with-letters of length >= 4, so 4 (vs the prior 8) closes
// the small-hex-chunk leak (md5/sha shown in "xxxx-xxxx" display grouping)
// with zero model-id over-scrub.
func isWordLikeGroup(g string) bool {
var hasUpper, hasLower, allHex, allDigit bool = false, false, true, true
for _, r := range g {
switch {
case r >= 'A' && r <= 'Z':
hasUpper = true
case r >= 'a' && r <= 'z':
hasLower = true
}
if r < '0' || r > '9' {
allDigit = false
}
isHex := (r >= '0' && r <= '9') || (r >= 'a' && r <= 'f') || (r >= 'A' && r <= 'F')
if !isHex {
allHex = false
}
}
if hasUpper && hasLower {
return false // dense mixed-case → base64 secret chunk, not a word.
}
if allHex && !allDigit && len(g) >= hexChunkMinLen {
return false // hex-with-letters → hex secret chunk (not a version date).
}
return true
}
// hexChunkMinLen is the length at/above which an all-hex-with-letters group is
// treated as a secret chunk rather than a word. 4 is the tightest bound that does
// not over-scrub any real model-id group (RED-verified across 19 model ids) while
// catching hex secrets displayed in short groups (md5 "xxxx-xxxx", uuid-ish).
const hexChunkMinLen = 4
// isUUID reports whether s is a canonical 8-4-4-4-12 hex UUID (case-insensitive).
// Used to exempt uuids from the high-entropy secret test — a uuid is a legitimate
// resource id, not a credential.
func isUUID(s string) bool {
if len(s) != 36 {
return false
}
for i, r := range s {
switch i {
case 8, 13, 18, 23:
if r != '-' {
return false
}
default:
if !((r >= '0' && r <= '9') || (r >= 'a' && r <= 'f') || (r >= 'A' && r <= 'F')) {
return false
}
}
}
return true
}
// maxUserAgentLen caps the recorded User-Agent so an oversized UA can neither
// bloat the trail nor smuggle a long payload. 512 chars covers every real UA.
const maxUserAgentLen = 512
// scrubFreeText scrubs credential-shaped words out of client-controlled free
// text (the User-Agent) and caps its length. It tokenizes on a broad delimiter
// superset — whitespace and the punctuation that commonly glues a token into a
// UA/header value (= ; , : / ( ) [ ] { } " ' < > | and backslash) — scrubs each
// token, and rebuilds the string preserving the exact delimiters between tokens.
//
// RED-review hardening (finding: UA tokenizer split on too few delimiters, and
// strings.ReplaceAll was substring-fragile): this walks the string in one pass
// (token-run, delimiter-run, …) and replaces each credential token IN PLACE, so a
// secret delimited by ':' '/' '(' etc. is caught and a token that is a substring
// of another is never mis-replaced. A normal UA ("Mozilla/5.0 (Macintosh …)") is
// unchanged because none of its words is credential-shaped.
func scrubFreeText(s string) string {
if s == "" {
return ""
}
if len(s) > maxUserAgentLen {
s = s[:maxUserAgentLen]
}
var b strings.Builder
b.Grow(len(s))
start := -1 // start index of the current token run, or -1 in a delimiter run.
flush := func(end int) {
if start >= 0 {
b.WriteString(scrubToken(s[start:end]))
start = -1
}
}
for i, r := range s {
if isFreeTextDelimiter(r) {
flush(i)
b.WriteRune(r)
} else if start < 0 {
start = i
}
}
flush(len(s))
return b.String()
}
// isFreeTextDelimiter reports whether r separates tokens in free text (a UA /
// header value). Deliberately broad so a credential can't hide behind an unusual
// separator.
//
// RED re-review (UA bypass persists): '.' '@' '#' '~' are INCLUDED — a prefixed
// key glued by one of them (client@sk-live-KEY, app.sk-live-KEY, build#hk-KEY)
// otherwise stayed one token whose PREFIX was no longer sk-/hk-, so isAPIKey
// missed it. Splitting on them exposes the bare key to scrubToken. Real UA dots
// are numeric version separators (<24, safe) and are split harmlessly. A JWT
// (eyJ.h.p.s) is handled up-front by scrubToken via looksLikeJWT before any
// tokenizer runs on a URL path segment; in free text a dotted JWT will split,
// but each ~40-char base64 part is itself a >=24 high-entropy run, so every part
// is still redacted.
func isFreeTextDelimiter(r rune) bool {
switch r {
case ' ', '\t', '\n', '\r', '=', ';', ',', ':', '/', '\\',
'(', ')', '[', ']', '{', '}', '"', '\'', '<', '>', '|', '&', '?',
'.', '@', '#', '~':
return true
}
return false
}
// scrubCredentialSegments applies scrubToken to every segment of a path, so the
// recorded Path can never carry a credential even if a future route embeds one.
// Route nouns/ids pass through unchanged (they are not isAPIKey-shaped).
func scrubCredentialSegments(path string) string {
if !strings.ContainsAny(path, "/") {
return scrubToken(path)
}
segs := strings.Split(path, "/")
changed := false
for i, s := range segs {
if scrubbed := scrubToken(s); scrubbed != s {
segs[i] = scrubbed
changed = true
}
}
if !changed {
return path
}
return strings.Join(segs, "/")
}
// looksLikeID heuristically flags a path segment as a high-cardinality id (a
// uuid, a long hex/opaque token, or a numeric id) vs a fixed route noun. Used to
// collapse route families; false positives only make the action slightly more
// specific, never leak anything.
func looksLikeID(s string) bool {
if s == "" {
return false
}
if len(s) >= 16 { // long opaque/uuid-ish segment
return true
}
allDigits := true
for _, r := range s {
if r < '0' || r > '9' {
allDigits = false
break
}
}
return allDigits && len(s) > 0
}
audit_middleware_test.go
+579
View File
@@ -0,0 +1,579 @@
package cloud
// Integration tests for the audit middleware. They drive REAL requests through
// the zip/fiber stack (app.Fiber().Test) with the audit middleware in front of a
// handler, backed by a REAL on-disk audit store, then read the store back to
// assert what was (and was not) recorded. No mocks — the whole capture path runs.
//
// The middleware trusts SanitizeIdentity to have already validated identity, so
// these tests set the sanitized X-User-* headers directly (as SanitizeIdentity
// would after verifying a JWT) — that is the contract boundary under test here.
// The forgery/bypass properties of SanitizeIdentity itself are proven in
// middleware_identity_test.go; here we prove the middleware records the VALIDATED
// identity and the correct outcome for every security-relevant request.
import (
"encoding/json"
"net/http"
"net/http/httptest"
"path/filepath"
"strings"
"testing"
"github.com/hanzoai/cloud/audit"
"github.com/hanzoai/zip"
)
// newAuditApp wires a zip app with the audit middleware in front of a small set
// of routes covering the coverage matrix: a mutating POST, a safe GET, an
// admin-gated route that 403s, and a route that echoes a (secret-bearing) body
// so we can prove the body never reaches a record.
func newAuditApp(t *testing.T) (*zip.App, *audit.Recorder) {
t.Helper()
path := filepath.Join(t.TempDir(), "audit.db")
rec, err := audit.Open(path, nil)
if err != nil {
t.Fatalf("audit.Open: %v", err)
}
t.Cleanup(func() { _ = rec.Close() })
app := zip.New(zip.Config{})
app.Use(AuditTrail(rec))
// Mutating route — must be audited.
app.Post("/v1/kms/secrets", func(c *zip.Ctx) error {
return c.JSON(http.StatusCreated, map[string]string{"id": "sec_1"})
})
// Safe read — must NOT be audited (not a mutation, not admin, not a denial).
app.Get("/v1/pricing/models", func(c *zip.Ctx) error {
return c.JSON(http.StatusOK, map[string]string{"ok": "true"})
})
// Admin route that denies (as the real admin guard would) — the 403 is a
// security event and MUST be audited even though it is a GET.
app.Get("/v1/admin/orgs", func(c *zip.Ctx) error {
if !c.IsAdmin() {
return zip.ErrForbidden("global admin required")
}
return c.JSON(http.StatusOK, map[string]string{"ok": "true"})
})
// A mutation whose request body carries secrets — used to prove the body is
// never captured. The handler ignores the body; the point is what the
// middleware records (metadata only).
app.Post("/v1/iam/users", func(c *zip.Ctx) error {
return c.JSON(http.StatusOK, map[string]string{"ok": "true"})
})
return app, rec
}
// asAdmin sets the sanitized identity headers a VALIDATED global admin would
// carry after SanitizeIdentity (X-User-IsAdmin=true, org=admin).
func asAdmin(req *http.Request) {
req.Header.Set("X-User-Id", "z@hanzo.ai")
req.Header.Set("X-User-Email", "z@hanzo.ai")
req.Header.Set("X-Org-Id", "admin")
req.Header.Set("X-User-IsAdmin", "true")
req.Header.Set("Authorization", "Bearer eyJ.validated.jwt") // shape only; classifies as jwt
}
// asUser sets sanitized headers for a normal (non-admin) validated principal.
func asUser(req *http.Request) {
req.Header.Set("X-User-Id", "alice")
req.Header.Set("X-Org-Id", "acme")
req.Header.Set("Authorization", "Bearer eyJ.validated.jwt")
}
func mustTest(t *testing.T, app *zip.App, req *http.Request) *http.Response {
t.Helper()
resp, err := app.Fiber().Test(req)
if err != nil {
t.Fatalf("Test %s %s: %v", req.Method, req.URL.Path, err)
}
return resp
}
// TestAudit_RecordsMutation proves a mutating request is captured with the
// correct validated actor, action, resource, outcome, and auth context — and the
// record is hash-chained (has a hash) and verifies.
func TestAudit_RecordsMutation(t *testing.T) {
app, rec := newAuditApp(t)
req := httptest.NewRequest(http.MethodPost, "/v1/kms/secrets", nil)
asUser(req)
req.Header.Set("X-Forwarded-For", "203.0.113.9, 10.0.0.1")
req.Header.Set("User-Agent", "test-agent/1.0")
resp := mustTest(t, app, req)
if resp.StatusCode != http.StatusCreated {
t.Fatalf("status = %d, want 201", resp.StatusCode)
}
rows, total, err := rec.Query(t.Context(), audit.Filter{})
if err != nil {
t.Fatalf("Query: %v", err)
}
if total != 1 || len(rows) != 1 {
t.Fatalf("recorded %d events, want exactly 1", total)
}
r := rows[0]
if r.Actor.Org != "acme" || r.Actor.Sub != "alice" {
t.Errorf("actor = %+v, want org=acme sub=alice (the VALIDATED identity)", r.Actor)
}
if r.Method != "POST" || r.Path != "/v1/kms/secrets" {
t.Errorf("method/path = %s %s, want POST /v1/kms/secrets", r.Method, r.Path)
}
if r.Resource.Type != "secrets" {
t.Errorf("resource type = %q, want secrets", r.Resource.Type)
}
if r.Outcome.Result != "success" || r.Outcome.Status != 201 {
t.Errorf("outcome = %+v, want success/201", r.Outcome)
}
if r.Auth.Method != "jwt" {
t.Errorf("auth method = %q, want jwt", r.Auth.Method)
}
if r.SourceIP != "203.0.113.9" {
t.Errorf("source ip = %q, want the left-most XFF entry", r.SourceIP)
}
if r.UserAgent != "test-agent/1.0" {
t.Errorf("user agent = %q, want test-agent/1.0", r.UserAgent)
}
if r.Hash == "" {
t.Error("record has no hash — not chained")
}
if iv, _ := rec.Verify(t.Context()); !iv.OK {
t.Errorf("chain broke after one record at %d (%s)", iv.BrokenAt, iv.Reason)
}
}
// TestAudit_RecordsDenial proves a 403 (access-control denial) is audited even on
// a GET, with outcome result="deny", and records the actor who was denied.
func TestAudit_RecordsDenial(t *testing.T) {
app, rec := newAuditApp(t)
// A NON-admin hits the admin route → 403.
req := httptest.NewRequest(http.MethodGet, "/v1/admin/orgs", nil)
asUser(req) // not admin
resp := mustTest(t, app, req)
if resp.StatusCode != http.StatusForbidden {
t.Fatalf("status = %d, want 403", resp.StatusCode)
}
rows, total, err := rec.Query(t.Context(), audit.Filter{Result: "deny"})
if err != nil {
t.Fatalf("Query: %v", err)
}
if total != 1 {
t.Fatalf("recorded %d denials, want 1", total)
}
r := rows[0]
if r.Outcome.Result != "deny" || r.Outcome.Status != 403 {
t.Errorf("outcome = %+v, want deny/403", r.Outcome)
}
if r.Actor.Sub != "alice" {
t.Errorf("denied actor sub = %q, want alice", r.Actor.Sub)
}
if r.Auth.IsAdmin {
t.Error("denied non-admin recorded as admin")
}
}
// TestAudit_SkipsSafeReads proves an ordinary successful GET on a non-admin route
// is NOT audited — the trail captures security events, not read-log noise.
func TestAudit_SkipsSafeReads(t *testing.T) {
app, rec := newAuditApp(t)
req := httptest.NewRequest(http.MethodGet, "/v1/pricing/models", nil)
asUser(req)
resp := mustTest(t, app, req)
if resp.StatusCode != http.StatusOK {
t.Fatalf("status = %d, want 200", resp.StatusCode)
}
_, total, err := rec.Query(t.Context(), audit.Filter{})
if err != nil {
t.Fatalf("Query: %v", err)
}
if total != 0 {
t.Fatalf("a safe GET was audited (%d rows) — trail should skip non-security reads", total)
}
}
// TestAudit_AdminReadIsAudited proves a SUCCESSFUL admin read is audited (admin
// access itself is an AC-relevant event), distinguishing it from a normal read.
func TestAudit_AdminReadIsAudited(t *testing.T) {
app, rec := newAuditApp(t)
req := httptest.NewRequest(http.MethodGet, "/v1/admin/orgs", nil)
asAdmin(req)
resp := mustTest(t, app, req)
if resp.StatusCode != http.StatusOK {
t.Fatalf("status = %d, want 200", resp.StatusCode)
}
rows, total, err := rec.Query(t.Context(), audit.Filter{})
if err != nil {
t.Fatalf("Query: %v", err)
}
if total != 1 {
t.Fatalf("admin read recorded %d, want 1", total)
}
if !rows[0].Auth.IsAdmin || rows[0].Outcome.Result != "success" {
t.Errorf("admin read record = auth.isAdmin=%v outcome=%+v, want admin+success", rows[0].Auth.IsAdmin, rows[0].Outcome)
}
}
// TestAudit_NeverCapturesRequestBody is the secret-safety proof: a mutation whose
// body is FULL of credentials is audited, but the stored record contains NONE of
// the body — the middleware captures metadata only, so a secret in a body can
// never leak into the trail. This is the "reuse RedactUserSecrets" guarantee at
// its strongest: the code that could leak a secret never reads it.
func TestAudit_NeverCapturesRequestBody(t *testing.T) {
app, rec := newAuditApp(t)
secretBody := `{"username":"bob","password":"hunter2","apiKey":"sk-live-DEADBEEF","token":"ghp_SECRET"}`
req := httptest.NewRequest(http.MethodPost, "/v1/iam/users", strings.NewReader(secretBody))
req.Header.Set("Content-Type", "application/json")
asAdmin(req)
// Also stuff a secret into a header value that is NOT an identity header — it
// must not be captured either (we only record User-Agent + XFF, never arbitrary
// headers, and never Authorization's value).
req.Header.Set("Authorization", "Bearer eyJsuper.secret.token.value")
resp := mustTest(t, app, req)
if resp.StatusCode != http.StatusOK {
t.Fatalf("status = %d, want 200", resp.StatusCode)
}
rows, total, err := rec.Query(t.Context(), audit.Filter{})
if err != nil {
t.Fatalf("Query: %v", err)
}
if total != 1 {
t.Fatalf("recorded %d, want 1", total)
}
// Serialize the WHOLE record and scan for any secret substring.
blob, _ := json.Marshal(rows[0])
for _, secret := range []string{"hunter2", "sk-live-DEADBEEF", "ghp_SECRET", "super.secret.token.value"} {
if strings.Contains(string(blob), secret) {
t.Fatalf("SECRET LEAKED into audit record: %q found in %s", secret, blob)
}
}
// But the metadata IS there: the auth method is classified without the token.
if rows[0].Auth.Method != "jwt" {
t.Errorf("auth method = %q, want jwt (classified from prefix, token not stored)", rows[0].Auth.Method)
}
if rows[0].Before != nil || rows[0].After != nil {
t.Errorf("middleware set before/after (%s / %s) — it must never read bodies", rows[0].Before, rows[0].After)
}
}
// TestAudit_ScrubsCredentialInPath proves a credential-shaped token that rides in
// the URL PATH (e.g. a KMS route where a caller wrongly puts an sk-/hk- key in the
// path) is never recorded verbatim in either Path or resource.ID — defense in
// depth beyond "bodies are never read". A normal identifier is untouched.
func TestAudit_ScrubsCredentialInPath(t *testing.T) {
path := filepath.Join(t.TempDir(), "audit.db")
rec, err := audit.Open(path, nil)
if err != nil {
t.Fatalf("open: %v", err)
}
t.Cleanup(func() { _ = rec.Close() })
app := zip.New(zip.Config{})
app.Use(AuditTrail(rec))
app.Delete("/v1/kms/secrets/*", func(c *zip.Ctx) error {
return c.JSON(http.StatusOK, map[string]string{"ok": "true"})
})
// Three credential shapes smuggled into the path across three requests: a
// prefixed key, a raw high-entropy hex secret (NO telltale prefix), and a JWT.
paths := []struct{ path, secret string }{
{"/v1/kms/secrets/sk-live-SUPERSECRETKEY1234567890", "SUPERSECRETKEY"},
{"/v1/kms/secrets/deadbeefcafe0123456789abcdef0123456789abcdef0123", "deadbeefcafe0123"},
{"/v1/kms/secrets/eyJhbGciOiJIUzI1NiJ9.cGF5bG9hZA.c2ln", "eyJhbGciOiJIUzI1NiJ9"},
}
for _, tc := range paths {
req := httptest.NewRequest(http.MethodDelete, tc.path, nil)
asAdmin(req)
if resp := mustTest(t, app, req); resp.StatusCode != http.StatusOK {
t.Fatalf("%s: status = %d, want 200", tc.path, resp.StatusCode)
}
}
rows, total, err := rec.Query(t.Context(), audit.Filter{})
if err != nil {
t.Fatalf("Query: %v", err)
}
if total != len(paths) {
t.Fatalf("recorded %d, want %d", total, len(paths))
}
blob, _ := json.Marshal(rows)
for _, tc := range paths {
if strings.Contains(string(blob), tc.secret) {
t.Fatalf("credential in path leaked into record: %q present in %s", tc.secret, blob)
}
}
for _, r := range rows {
if !strings.Contains(r.Path, "[REDACTED-TOKEN]") {
t.Errorf("path token not scrubbed: %q", r.Path)
}
}
}
// TestAudit_ScrubsSecretInUserAgent proves a bearer/API-key embedded in the
// client-controlled User-Agent is scrubbed, and a normal UA is untouched.
func TestAudit_ScrubsSecretInUserAgent(t *testing.T) {
path := filepath.Join(t.TempDir(), "audit.db")
rec, err := audit.Open(path, nil)
if err != nil {
t.Fatalf("open: %v", err)
}
t.Cleanup(func() { _ = rec.Close() })
app := zip.New(zip.Config{})
app.Use(AuditTrail(rec))
app.Post("/v1/kms/secrets", func(c *zip.Ctx) error { return c.JSON(http.StatusOK, map[string]string{"ok": "1"}) })
req := httptest.NewRequest(http.MethodPost, "/v1/kms/secrets", nil)
asAdmin(req)
req.Header.Set("User-Agent", "myclient/1.0 Bearer eyJhbGciOiJI.pay.sig key=sk-live-LEAKME99999")
mustTest(t, app, req)
rows, _, _ := rec.Query(t.Context(), audit.Filter{})
if len(rows) != 1 {
t.Fatalf("got %d rows, want 1", len(rows))
}
for _, secret := range []string{"eyJhbGciOiJI", "sk-live-LEAKME99999"} {
if strings.Contains(rows[0].UserAgent, secret) {
t.Errorf("UA secret leaked: %q in %q", secret, rows[0].UserAgent)
}
}
// The non-secret UA prefix survives (audit usefulness preserved).
if !strings.Contains(rows[0].UserAgent, "myclient/1.0") {
t.Errorf("UA over-scrubbed, lost the client name: %q", rows[0].UserAgent)
}
}
// TestScrubToken_NoFalsePositives proves legitimate identifiers are NEVER
// scrubbed — the guard fires only on genuinely secret-shaped segments, so the
// audit trail keeps its query precision for normal resource ids.
func TestScrubToken_NoFalsePositives(t *testing.T) {
for _, id := range []string{
"acme-corp", "gpt-4o-mini", "my_project_123", "user@example.com",
"550e8400-e29b-41d4-a716-446655440000", // uuid (hyphens)
"claude-opus-4-20250514", "text-embedding-3-large",
"my-cool-project-name", "feature-branch-xyz",
"deployment-2024-01-15", "report.pdf", "data.json",
// Red re-review round 2 — hyphenated model ids MUST pass (AU-3: an auditor
// must still see WHICH model a config change touched).
"claude-3-5-sonnet-20241022", "claude-3-5-haiku-20241022",
"claude-sonnet-4-20250514", "claude-3-7-sonnet-20250219",
"claude-3-5-sonnet-latest", "deepseek-r1-distill-qwen-32b",
"claude-3-opus-20240229", "stable-diffusion-xl-base",
"mixtral-8x7b-instruct", "llama-3-1-8b-instruct", "whisper-large-v3-turbo",
"v1", "models", "sync", "12345", "a", "",
} {
if got := scrubToken(id); got != id {
t.Errorf("false scrub: legit id %q → %q", id, got)
}
}
// And genuine secrets ARE scrubbed.
for _, sec := range []string{
"sk-live-abcdef", "hk-1234567890abcdef", "eyJhbG.payload.signature",
"deadbeefcafe0123456789abcdef0123456789abcdef0123", // 48-char raw hex
} {
if scrubToken(sec) == sec {
t.Errorf("missed secret: %q not scrubbed", sec)
}
}
}
// TestScrubToken_RedReviewBypassClasses is the regression for the 4 scrub-bypass
// classes Red found: base64url with -/_, all-alpha opaque >=len, percent-encoded
// prefixes, and delimiter-glued UA tokens. Each MUST now be redacted.
func TestScrubToken_RedReviewBypassClasses(t *testing.T) {
for _, sec := range []string{
"AbCdEf-GhIjKl_MnOpQrStUvWxYz012345", // base64url with - and _
"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMN", // all-alpha opaque, no digit
"hk%2DROTATEKEY0001SECRETKEY", // percent-encoded hk-
"sk%5Flive%5FBYPASS0001SECRETKEY", // percent-encoded sk_
// Red re-review round 2 — dotted-exemption + encoding + standard-base64:
"deadbeefcafe0123456789abcdef0123456789abcdef.x", // ".x" tail forces dotted exemption
"AbCdEfGhIjKlMnOpQrStUvWxYz012345.json", // secret with a filename-ish suffix
"hk%252DROTATEKEY0001SECRETKEY", // double percent-encoded hk-
"AbCdEfGhIjKlMnOpQrStUvWx0123456789", // 34-char opaque run (standard/url b64)
// Red re-review round 3 — interior-hyphen raw secret (NOT a structured id:
// only 1 hyphen, long parts) must still redact despite '-' in the run.
"aaaaaaaaaaaaaaaaaaaaaaaa-bbbbbbbbbbbbbbbbbbbbbbbb",
// Red final re-review — a secret CHUNKED to satisfy the structured-id shape
// (>=3 groups <=12 chars) must STILL redact: the lexical (word-like) test
// rejects mixed-case base64 chunks and hex chunks (>= hexChunkMinLen=4).
"AbCdEfGhIjKl-MnOpQrStUvWx-YzAbCdEfGhIj", // mixed-case base64 chunks
"A1b2C3d4-E5f6G7h8-I9j0K1l2", // 128-bit mixed-case chunks
"deadbeef-cafebabe-01234567-89abcdef", // all-hex chunks (8-char groups)
"abcdef01-23456789-abcdef01", // hex chunks
// Red final polish — SMALL hex chunks (md5/sha "xxxx-xxxx" display) now
// caught by the len>=4 hex rule.
"abcd-ef01-2345-6789-abcd-ef01", // 4-char hex groups
"dead-beef-cafe-babe-0123-4567", // 4-char hex groups
} {
if got := scrubToken(sec); got != "[REDACTED-TOKEN]" {
t.Errorf("Red bypass STILL OPEN: %q → %q (want redacted)", sec, got)
}
}
// UA with a secret glued by :/()[]= — must be scrubbed, client name kept.
ua := scrubFreeText("myapp/1.0 (token:sk-live-BYPASS0001) [key=hk-1234567890abcdef]")
for _, leak := range []string{"sk-live-BYPASS0001", "hk-1234567890abcdef"} {
if strings.Contains(ua, leak) {
t.Errorf("UA bypass: %q leaked in %q", leak, ua)
}
}
if !strings.Contains(ua, "myapp/1.0") {
t.Errorf("UA over-scrubbed, lost client name: %q", ua)
}
// Red re-review round 2 — a key glued by . @ # ~ must NOT survive.
for _, glued := range []string{
"client@sk-live-SECRETKEY00001", "app.sk-live-SECRETKEY00001",
"build#hk-SECRETKEY000000001", "v1~sk-live-SECRETKEY00001",
} {
if got := scrubFreeText(glued); strings.Contains(got, "SECRETKEY") {
t.Errorf("UA glue-char bypass: %q → %q (secret survives)", glued, got)
}
}
// Normal UAs must be byte-identical (no false scrub).
for _, ua := range []string{
"console2", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)",
"curl/8.1.2", "Go-http-client/2.0",
} {
if got := scrubFreeText(ua); got != ua {
t.Errorf("UA false positive: %q → %q", ua, got)
}
}
}
// TestAudit_HealthSuffixCannotEvadeAudit proves a mutating request whose path
// ENDS in /health (an attacker-named wildcard segment) is STILL audited — the
// liveness exemption is exact and never suppresses a mutation or a denial. This
// closes an audit-evasion hole where POST /v1/admin/orgs/x/health would slip past.
func TestAudit_HealthSuffixCannotEvadeAudit(t *testing.T) {
path := filepath.Join(t.TempDir(), "audit.db")
rec, err := audit.Open(path, nil)
if err != nil {
t.Fatalf("open: %v", err)
}
t.Cleanup(func() { _ = rec.Close() })
app := zip.New(zip.Config{})
app.Use(AuditTrail(rec))
app.Post("/v1/admin/orgs/*", func(c *zip.Ctx) error { return c.JSON(http.StatusOK, map[string]string{"ok": "1"}) })
app.Get("/v1/kms/health", func(c *zip.Ctx) error { return c.JSON(http.StatusOK, map[string]string{"status": "ok"}) })
// (a) A mutating POST ending in /health MUST be audited.
req := httptest.NewRequest(http.MethodPost, "/v1/admin/orgs/evil/health", nil)
asAdmin(req)
mustTest(t, app, req)
_, total, _ := rec.Query(t.Context(), audit.Filter{})
if total != 1 {
t.Fatalf("EVASION: mutating POST ending /health audited %d times, want 1", total)
}
// (b) A genuine liveness GET /v1/kms/health MUST still be skipped.
req2 := httptest.NewRequest(http.MethodGet, "/v1/kms/health", nil)
mustTest(t, app, req2)
_, total2, _ := rec.Query(t.Context(), audit.Filter{})
if total2 != 1 {
t.Fatalf("liveness probe was audited (total went %d→%d) — should be exempt", total, total2)
}
}
// TestAudit_AnonRequestNotAttributedToForgedOrg proves an UNAUTHENTICATED
// attacker cannot forge a false attribution: sending X-Org-Id/X-User-Id/
// X-User-IsAdmin with no validated principal records an ANONYMOUS actor (empty
// org+sub, not admin), never the claimed victim org. Runs the REAL SanitizeIdentity
// (nil validator ⇒ strips authority, restores client X-Org-Id for the data path)
// ahead of AuditTrail, exactly as serve.go wires them.
func TestAudit_AnonRequestNotAttributedToForgedOrg(t *testing.T) {
path := filepath.Join(t.TempDir(), "audit.db")
rec, err := audit.Open(path, nil)
if err != nil {
t.Fatalf("open: %v", err)
}
t.Cleanup(func() { _ = rec.Close() })
app := zip.New(zip.Config{})
app.Use(SanitizeIdentity(nil, "admin")) // trust boundary
app.Use(AuditTrail(rec))
app.Post("/v1/kms/secrets", func(c *zip.Ctx) error { return c.JSON(http.StatusOK, map[string]string{"ok": "1"}) })
req := httptest.NewRequest(http.MethodPost, "/v1/kms/secrets", nil)
// Anonymous attacker forging every identity header.
req.Header.Set("X-Org-Id", "victim-org")
req.Header.Set("X-User-Id", "victim-user")
req.Header.Set("X-User-IsAdmin", "true")
mustTest(t, app, req)
rows, _, _ := rec.Query(t.Context(), audit.Filter{})
if len(rows) != 1 {
t.Fatalf("got %d rows, want 1", len(rows))
}
r := rows[0]
if r.Auth.IsAdmin {
t.Error("forged X-User-IsAdmin survived into the record")
}
if r.Actor.Sub != "" {
t.Errorf("forged X-User-Id recorded as actor.Sub = %q", r.Actor.Sub)
}
if r.Actor.Org == "victim-org" {
t.Errorf("FALSE ATTRIBUTION: anonymous request stamped with claimed org %q", r.Actor.Org)
}
if r.Auth.Method != "none" {
t.Errorf("auth method = %q, want none (no valid credential)", r.Auth.Method)
}
// The event is still recorded (a mutation), honestly anonymous.
if r.Outcome.Result != "success" {
t.Errorf("outcome = %+v, want the anonymous mutation recorded", r.Outcome)
}
}
// TestAudit_NoopWhenUnconfigured proves a nil Recorder makes the middleware a
// pass-through (an unconfigured deployment is never blocked), exactly like
// BillingGate's nil-client behavior.
func TestAudit_NoopWhenUnconfigured(t *testing.T) {
app := zip.New(zip.Config{})
app.Use(AuditTrail(nil))
var ran bool
app.Post("/v1/kms/secrets", func(c *zip.Ctx) error {
ran = true
return c.JSON(http.StatusOK, map[string]string{"ok": "true"})
})
req := httptest.NewRequest(http.MethodPost, "/v1/kms/secrets", nil)
resp := mustTest(t, app, req)
if resp.StatusCode != http.StatusOK || !ran {
t.Fatalf("nil-recorder gate must pass through: status=%d ran=%v", resp.StatusCode, ran)
}
}
// TestAudit_FailsClosedOnWriteError proves that when the audit store cannot
// record a security-relevant event, the request is failed CLOSED (503) rather
// than allowed to succeed unlogged (AU-5). We force the failure by closing the
// store's DB before the request, so Append errors.
func TestAudit_FailsClosedOnWriteError(t *testing.T) {
path := filepath.Join(t.TempDir(), "audit.db")
rec, err := audit.Open(path, nil)
if err != nil {
t.Fatalf("open: %v", err)
}
// Close the underlying store so every subsequent Append fails.
_ = rec.Close()
app := zip.New(zip.Config{})
app.Use(AuditTrail(rec))
var ran bool
app.Post("/v1/kms/secrets", func(c *zip.Ctx) error {
ran = true
return c.JSON(http.StatusCreated, map[string]string{"id": "x"})
})
req := httptest.NewRequest(http.MethodPost, "/v1/kms/secrets", nil)
asAdmin(req)
resp := mustTest(t, app, req)
// The handler may have run (audit wraps AFTER the chain), but the response the
// CLIENT sees must be the fail-closed 503, not the handler's 201 — a
// security-relevant action that could not be recorded is not acknowledged as
// success.
if resp.StatusCode != http.StatusServiceUnavailable {
t.Fatalf("status = %d, want 503 (fail-closed when audit write fails)", resp.StatusCode)
}
_ = ran
}
audit_mirror.go
+199
View File
@@ -0,0 +1,199 @@
package cloud
// The datastore (ClickHouse) OLAP mirror — a best-effort projection of the audit
// trail for fleet-wide, long-retention, cross-deployment query. It implements
// audit.Mirror.
//
// The datastore is the natural OLAP audit sink: the table is a MergeTree, which
// is INSERT-ONLY by engine — ClickHouse rejects UPDATE/DELETE against it at parse
// time ("MergeTree does not support mutations"), so the mirror is append-only at
// the storage layer, matching the local chain's discipline. We create the table
// idempotently on first connect (CREATE TABLE IF NOT EXISTS) and insert via the
// canonical clickhouse-go PrepareBatch → Append → Send idiom (the same the
// provisioning subsystem uses; the driver is already in cloud's module graph, so
// this adds no dependency).
//
// This mirror is NEVER the integrity authority — the local SQLite hash-chain is.
// Its rows carry the same seq + hash so an operator CAN cross-check the OLAP copy
// against the chain, but a mirror gap is a query-completeness issue, not a
// tamper-evidence one. Every mirror error is logged and dropped by the Recorder;
// the request path never sees it.
import (
"context"
"fmt"
"os"
"strings"
"time"
clickhouse "github.com/ClickHouse/clickhouse-go/v2"
"github.com/hanzoai/cloud/audit"
luxlog "github.com/luxfi/log"
)
// clickhouseMirror writes audit records to a ClickHouse MergeTree table.
type clickhouseMirror struct {
conn clickhouse.Conn
table string
log luxlog.Logger
}
// newAuditMirror builds the OLAP mirror from operator config, or returns nil when
// no datastore is configured (mirroring is optional — the local chain is the
// authority). It connects lazily-validated (a Ping) and ensures the table exists.
//
// Config (all from env / KMS-injected secrets, never hard-coded):
//
// CLOUD_AUDIT_CLICKHOUSE_ADDR host:9000 of the datastore native port
// CLOUD_AUDIT_CLICKHOUSE_DB database (default "hanzo")
// CLOUD_AUDIT_CLICKHOUSE_TABLE table (default "audit_log")
// CLOUD_AUDIT_CLICKHOUSE_USER user
// CLOUD_AUDIT_CLICKHOUSE_PASSWORD password (KMS-backed secret)
func newAuditMirror(log luxlog.Logger) (audit.Mirror, error) {
addr := strings.TrimSpace(os.Getenv("CLOUD_AUDIT_CLICKHOUSE_ADDR"))
if addr == "" {
return nil, nil // no datastore configured — local chain only.
}
db := getenv("CLOUD_AUDIT_CLICKHOUSE_DB", "hanzo")
table := getenv("CLOUD_AUDIT_CLICKHOUSE_TABLE", "audit_log")
conn, err := clickhouse.Open(&clickhouse.Options{
Addr: []string{addr},
Auth: clickhouse.Auth{
Database: db,
Username: os.Getenv("CLOUD_AUDIT_CLICKHOUSE_USER"),
Password: os.Getenv("CLOUD_AUDIT_CLICKHOUSE_PASSWORD"),
},
DialTimeout: 5 * time.Second,
})
if err != nil {
return nil, fmt.Errorf("audit mirror: open: %w", err)
}
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := conn.Ping(ctx); err != nil {
_ = conn.Close()
return nil, fmt.Errorf("audit mirror: ping %s: %w", addr, err)
}
qualified := db + "." + table
m := &clickhouseMirror{conn: conn, table: qualified, log: log}
if err := m.ensureTable(ctx); err != nil {
_ = conn.Close()
return nil, err
}
if log != nil {
log.Info("audit OLAP mirror connected", "addr", addr, "table", qualified)
}
return m, nil
}
// ensureTable creates the append-only audit table if it does not exist. MergeTree
// = insert-only (mutations rejected at parse time). Partitioned by month and
// ordered for the (org, time) query pattern; seq + hash are carried so the OLAP
// copy is cross-checkable against the local chain.
func (m *clickhouseMirror) ensureTable(ctx context.Context) error {
ddl := fmt.Sprintf(`
CREATE TABLE IF NOT EXISTS %s (
seq UInt64,
ts DateTime64(3, 'UTC'),
actor_org LowCardinality(String),
actor_sub String,
actor_email String,
action LowCardinality(String),
res_type LowCardinality(String),
res_id String,
auth_method LowCardinality(String),
is_admin UInt8,
result LowCardinality(String),
status UInt16,
reason String,
source_ip String,
user_agent String,
request_id String,
method LowCardinality(String),
path String,
prev_hash String,
hash String
) ENGINE = MergeTree
PARTITION BY toYYYYMM(ts)
ORDER BY (actor_org, ts, seq)`, m.table)
if err := m.conn.Exec(ctx, ddl); err != nil {
return fmt.Errorf("audit mirror: ensure table: %w", err)
}
return nil
}
// Append writes one record to the OLAP mirror via the canonical batch idiom. The
// before/after diffs are DELIBERATELY not mirrored — the OLAP copy is for
// query/analytics over the event stream, and keeping the (already-redacted but
// still payload-bearing) diffs out of the fleet warehouse minimizes the blast
// radius of a warehouse compromise. The full record (with diffs) lives only in
// the local, access-controlled chain.
func (m *clickhouseMirror) Append(ctx context.Context, r audit.Record) error {
batch, err := m.conn.PrepareBatch(ctx, "INSERT INTO "+m.table+` (
seq, ts, actor_org, actor_sub, actor_email, action, res_type, res_id,
auth_method, is_admin, result, status, reason, source_ip, user_agent,
request_id, method, path, prev_hash, hash)`)
if err != nil {
return fmt.Errorf("audit mirror: prepare: %w", err)
}
if err := batch.Append(
r.Seq, r.Time.UTC(), r.Actor.Org, r.Actor.Sub, r.Actor.Email,
r.Action, r.Resource.Type, r.Resource.ID,
r.Auth.Method, boolToUint8(r.Auth.IsAdmin),
r.Outcome.Result, uint16(r.Outcome.Status), r.Outcome.Reason,
r.SourceIP, r.UserAgent, r.RequestID, r.Method, r.Path,
r.PrevHash, r.Hash,
); err != nil {
_ = batch.Abort()
return fmt.Errorf("audit mirror: append: %w", err)
}
return batch.Send()
}
// Checkpoint persists a head-digest checkpoint to an INDEPENDENT digest table in
// the datastore — the AU-9 tail-truncation anchor. Because this lives in a store
// SEPARATE from the local SQLite chain, truncating the chain cannot also rewrite
// the checkpoint history: an external monitor querying this table sees the count
// series and alerts on any regression. Best-effort; a failure is dropped by the
// Recorder (the structured log carries the same digest). Implements
// audit.CheckpointSink.
func (m *clickhouseMirror) Checkpoint(ctx context.Context, cp audit.Checkpoint) error {
if err := m.ensureCheckpointTable(ctx); err != nil {
return err
}
batch, err := m.conn.PrepareBatch(ctx, "INSERT INTO "+m.table+"_checkpoints (ts, count, head)")
if err != nil {
return fmt.Errorf("audit mirror: checkpoint prepare: %w", err)
}
if err := batch.Append(cp.Time.UTC(), cp.Count, cp.Head); err != nil {
_ = batch.Abort()
return fmt.Errorf("audit mirror: checkpoint append: %w", err)
}
return batch.Send()
}
// ensureCheckpointTable creates the append-only checkpoint digest table. A plain
// MergeTree ordered by time — the monitor reads the latest rows and checks that
// count never decreases.
func (m *clickhouseMirror) ensureCheckpointTable(ctx context.Context) error {
ddl := fmt.Sprintf(`
CREATE TABLE IF NOT EXISTS %s_checkpoints (
ts DateTime64(3, 'UTC'),
count UInt64,
head String
) ENGINE = MergeTree
ORDER BY ts`, m.table)
if err := m.conn.Exec(ctx, ddl); err != nil {
return fmt.Errorf("audit mirror: ensure checkpoint table: %w", err)
}
return nil
}
func boolToUint8(b bool) uint8 {
if b {
return 1
}
return 0
}
audit_serve.go
+95
View File
@@ -0,0 +1,95 @@
package cloud
// Audit trail construction — the wiring Serve calls to stand up the Recorder.
//
// The audit store is a COMPLIANCE CONTROL, so its persistence is treated like the
// pricing catalog overlay's: a non-persistent (in-memory) audit trail would
// silently lose the record of every prior action on each restart — a fail-OPEN
// degradation of an integrity control. So an empty DataDir is a hard boot error
// in a normal run (prod always sets CLOUD_DATA_DIR; provisioning + pricing already
// require it, so the unified binary always has one). The trail can be turned OFF
// deliberately (CLOUD_AUDIT_DISABLED=true) for a minimal single-service dev run —
// an explicit opt-out, never a silent one.
import (
"fmt"
"os"
"path/filepath"
"time"
"github.com/hanzoai/cloud/audit"
luxlog "github.com/luxfi/log"
)
// buildAuditRecorder constructs the audit Recorder from cfg: the append-only
// SQLite chain at {DataDir}/audit.db plus a best-effort ClickHouse OLAP mirror
// when a datastore is configured. Returns (nil, nil) only when the trail is
// explicitly disabled — the caller then wires a no-op middleware.
func buildAuditRecorder(cfg *Config, logger luxlog.Logger) (*audit.Recorder, error) {
if getenvBool("CLOUD_AUDIT_DISABLED") {
if logger != nil {
logger.Warn("audit trail DISABLED by CLOUD_AUDIT_DISABLED — no tamper-evident record will be kept")
}
return nil, nil
}
if cfg.DataDir == "" {
return nil, fmt.Errorf("empty DataDir — the audit trail is a compliance control and requires a persistent data dir (set CLOUD_DATA_DIR); refusing to boot with a non-persistent trail that would lose all prior records on restart (or set CLOUD_AUDIT_DISABLED=true to opt out explicitly)")
}
if err := os.MkdirAll(cfg.DataDir, 0o755); err != nil {
return nil, fmt.Errorf("data dir: %w", err)
}
// OLAP mirror is optional and best-effort. A mirror that cannot be reached at
// boot must NOT stop the binary — the local chain is the authority — so a
// mirror construction error is logged and the trail runs local-only.
var mirror audit.Mirror
if m, err := newAuditMirror(logger); err != nil {
if logger != nil {
logger.Warn("audit OLAP mirror unavailable — running local-only (chain integrity unaffected)", "err", err)
}
} else {
mirror = m
}
dbPath := filepath.Join(cfg.DataDir, "audit.db")
rec, err := audit.Open(dbPath, mirror)
if err != nil {
return nil, fmt.Errorf("open audit store: %w", err)
}
// AU-9 tail-truncation anchor: emit a periodic head-digest checkpoint to the
// append-only observability log (and, when a mirror supports it, an
// independent digest store). An external o11y monitor compares consecutive
// checkpoints and alerts on a count regression — the only way to detect that
// the most-recent records were deleted (an internal chain walk cannot). The
// interval is CLOUD_AUDIT_CHECKPOINT_INTERVAL (default 5m; 0 disables).
interval := auditCheckpointInterval()
if logger != nil {
rec.StartCheckpoints(interval, func(cp audit.Checkpoint) {
logger.Info("audit_head_checkpoint",
"count", cp.Count, "head", cp.Head, "ts", cp.Time.Format(time.RFC3339Nano))
})
} else {
rec.StartCheckpoints(interval, nil)
}
if logger != nil {
count, head := rec.Head()
logger.Info("audit trail ready (tamper-evident, append-only)",
"store", dbPath, "records", count, "head", head,
"mirror", mirror != nil, "checkpoint_interval", interval.String())
}
return rec, nil
}
// auditCheckpointInterval resolves the head-digest checkpoint cadence.
// CLOUD_AUDIT_CHECKPOINT_INTERVAL is a Go duration (e.g. "5m", "1h"); default 5m;
// "0" disables periodic checkpoints (the on-close checkpoint still fires).
func auditCheckpointInterval() time.Duration {
if v := getenv("CLOUD_AUDIT_CHECKPOINT_INTERVAL", ""); v != "" {
if d, err := time.ParseDuration(v); err == nil {
return d
}
}
return 5 * time.Minute
}
clients/admin/admin.go
+17 -10
View File
@@ -36,6 +36,7 @@ import (
"time"
"github.com/hanzoai/cloud"
"github.com/hanzoai/cloud/audit"
"github.com/hanzoai/zip"
)
@@ -45,6 +46,10 @@ type svc struct {
commerce *commerceClient
health *healthClient
adminOrg string
// auditStore is cloud's OWN tamper-evident audit store (nil when unconfigured,
// in which case /v1/admin/audit falls back to the IAM get-records proxy). Serve
// builds it and hands it over via deps.Audit. See audit.go.
auditStore *audit.Recorder
}
// Mount registers the /v1/admin/* surface on app. Every handler gates on
@@ -60,10 +65,11 @@ func Mount(app *zip.App, deps cloud.Deps) error {
logger = logger.New("subsystem", "admin")
s := &svc{
iam: newIAMClient(iamBase(deps)),
commerce: newCommerceClient(os.Getenv("CLOUD_COMMERCE_HTTP_URL"), os.Getenv("COMMERCE_SERVICE_TOKEN")),
health: newHealthClient(o11yHealthURL()),
adminOrg: adminOrgOf(deps),
iam: newIAMClient(iamBase(deps)),
commerce: newCommerceClient(os.Getenv("CLOUD_COMMERCE_HTTP_URL"), os.Getenv("COMMERCE_SERVICE_TOKEN")),
health: newHealthClient(o11yHealthURL()),
adminOrg: adminOrgOf(deps),
auditStore: deps.Audit,
}
app.Get("/v1/admin/me", s.guard(s.me))
@@ -73,6 +79,7 @@ func Mount(app *zip.App, deps cloud.Deps) error {
app.Get("/v1/admin/roles", s.guard(s.roles))
app.Get("/v1/admin/applications", s.guard(s.applications))
app.Get("/v1/admin/audit", s.guard(s.audit))
app.Get("/v1/admin/audit/verify", s.guard(s.auditVerify))
app.Get("/v1/admin/usage", s.guard(s.usage))
app.Get("/v1/admin/products", s.guard(s.products))
app.Post("/v1/admin/sync", s.guard(s.sync))
@@ -270,8 +277,12 @@ func (s *svc) iamPassthrough(c *zip.Ctx, path string) error {
}
// ── /v1/admin/audit — records directory (AuditRow[]) ─────────────────────────
//
// The handler lives in audit.go (it reads cloud's OWN tamper-evident store).
// iamAuditQuery builds the IAM get-records query for the federated fallback
// auditFromIAM uses when no local store is configured.
func (s *svc) audit(c *zip.Ctx) error {
func iamAuditQuery(c *zip.Ctx) url.Values {
q := url.Values{}
if org := strings.TrimSpace(c.Query("org")); org != "" {
q.Set("organizationName", org)
@@ -284,11 +295,7 @@ func (s *svc) audit(c *zip.Ctx) error {
q.Set("pageSize", ps)
q.Set("sortField", "createdTime")
q.Set("sortOrder", "descend")
res, err := s.iam.getList(c.Context(), callerCreds(c), "/v1/iam/get-records", q)
if err != nil {
return fail(c, err.Error())
}
return okRaw(c, res.rows, res.total)
return q
}
// ── /v1/admin/usage — fleet usage roll-up (UsageData) ────────────────────────
clients/admin/admin_test.go
+2
View File
@@ -33,6 +33,7 @@ func mount(t *testing.T, iamURL, commerceURL, healthURL string) func(method, pat
app.Get("/v1/admin/roles", s.guard(s.roles))
app.Get("/v1/admin/applications", s.guard(s.applications))
app.Get("/v1/admin/audit", s.guard(s.audit))
app.Get("/v1/admin/audit/verify", s.guard(s.auditVerify))
app.Get("/v1/admin/usage", s.guard(s.usage))
app.Get("/v1/admin/products", s.guard(s.products))
app.Post("/v1/admin/sync", s.guard(s.sync))
@@ -63,6 +64,7 @@ var adminRoutes = []struct{ method, path string }{
{"GET", "/v1/admin/roles"},
{"GET", "/v1/admin/applications"},
{"GET", "/v1/admin/audit"},
{"GET", "/v1/admin/audit/verify"},
{"GET", "/v1/admin/usage"},
{"GET", "/v1/admin/products"},
{"POST", "/v1/admin/sync"},
clients/admin/audit.go
+182
View File
@@ -0,0 +1,182 @@
package admin
// The /v1/admin/audit query surface, wired to cloud's REAL tamper-evident audit
// store (the audit.Recorder Serve builds and hands over via deps.Audit).
//
// This REPLACES the previous behavior — proxying IAM get-records — as the primary
// source: cloud now keeps its OWN append-only, hash-chained trail of every
// security-relevant request against this binary, and that is what a compliance
// auditor queries here. IAM's own login/session records remain available in IAM;
// they are a DIFFERENT trail (IAM's request surface), and admin still federates
// them as a fallback when cloud's local store is not configured, so no capability
// is lost.
//
// SECURITY. Both handlers are registered behind the SAME s.guard as every other
// /v1/admin/* route (global-admin only, fail-closed). They are READ-ONLY (Query
// and Verify issue SELECT only), so exposing them cannot weaken the append-only
// property. The verify endpoint returns integrity STATUS, never a way to mutate.
import (
"strconv"
"strings"
"time"
"github.com/hanzoai/cloud/audit"
"github.com/hanzoai/zip"
)
// auditRow is one record in the operator's audit table (AuditRow). The JSON tags
// are the operator contract. It is cloud's OWN record shape — richer than the IAM
// Record it supersedes: it carries the outcome, the validated auth context, and
// the hash-chain linkage so the console can show integrity per row.
type auditRow struct {
Seq uint64 `json:"seq"`
Time string `json:"time"`
Org string `json:"org"`
Sub string `json:"sub"`
Email string `json:"email,omitempty"`
Action string `json:"action"`
Resource string `json:"resource"`
ResourceID string `json:"resourceId,omitempty"`
Method string `json:"method,omitempty"`
Path string `json:"path,omitempty"`
Result string `json:"result"`
Status int `json:"status"`
Reason string `json:"reason,omitempty"`
SourceIP string `json:"sourceIp,omitempty"`
UserAgent string `json:"userAgent,omitempty"`
RequestID string `json:"requestId,omitempty"`
IsAdmin bool `json:"isAdmin"`
Auth string `json:"authMethod,omitempty"`
Hash string `json:"hash"`
PrevHash string `json:"prevHash"`
}
// audit answers GET /v1/admin/audit from cloud's local tamper-evident store when
// configured, else falls back to the IAM get-records proxy (federated view).
// Filters: org, sub, action, resource, result, since, until, pageSize, p (page).
// The response is the casibase list envelope { data:[rows], data2:total } the
// operator decodes, with the current chain integrity summary attached.
func (s *svc) audit(c *zip.Ctx) error {
// No local store configured → preserve the legacy federated IAM view so the
// endpoint never regresses to empty.
if s.auditStore == nil {
return s.auditFromIAM(c)
}
f := auditFilterFromQuery(c)
rows, total, err := s.auditStore.Query(c.Context(), f)
if err != nil {
return fail(c, err.Error())
}
out := make([]auditRow, 0, len(rows))
for _, r := range rows {
out = append(out, toAuditRow(r))
}
// Attach the live integrity summary so the console can badge the trail as
// verified. Best-effort: a verify error must not fail the listing.
integrity, ivErr := s.auditStore.Verify(c.Context())
var integrityPayload any
if ivErr == nil {
integrityPayload = integrity
}
return c.JSON(200, map[string]any{
"status": "ok",
"msg": "",
"data": out,
"data2": total,
"integrity": integrityPayload,
})
}
// auditVerify answers GET /v1/admin/audit/verify — the tamper-evidence check. It
// walks the whole hash chain and returns the integrity result (ok, count, head,
// and the seq where the chain first breaks if tampered). Global-admin gated like
// every admin route.
func (s *svc) auditVerify(c *zip.Ctx) error {
if s.auditStore == nil {
return fail(c, "audit store not configured")
}
integrity, err := s.auditStore.Verify(c.Context())
if err != nil {
return fail(c, err.Error())
}
return ok(c, integrity)
}
// auditFilterFromQuery builds an audit.Filter from the request query params. Time
// bounds accept RFC3339. pageSize (default 100, cap 1000) and p (1-based page)
// drive Limit/Offset. Unknown/blank params are simply not applied.
func auditFilterFromQuery(c *zip.Ctx) audit.Filter {
f := audit.Filter{
Org: strings.TrimSpace(c.Query("org")),
Sub: strings.TrimSpace(c.Query("sub")),
Action: strings.TrimSpace(c.Query("action")),
Resource: strings.TrimSpace(c.Query("resource")),
Result: strings.TrimSpace(c.Query("result")),
}
if v := strings.TrimSpace(c.Query("since")); v != "" {
if t, err := time.Parse(time.RFC3339, v); err == nil {
f.Since = t
}
}
if v := strings.TrimSpace(c.Query("until")); v != "" {
if t, err := time.Parse(time.RFC3339, v); err == nil {
f.Until = t
}
}
pageSize := 100
if v := strings.TrimSpace(c.Query("pageSize")); v != "" {
if n, err := strconv.Atoi(v); err == nil && n > 0 {
pageSize = n
}
}
f.Limit = pageSize
if v := strings.TrimSpace(c.Query("p")); v != "" {
if page, err := strconv.Atoi(v); err == nil && page > 1 {
f.Offset = (page - 1) * pageSize
}
}
return f
}
// toAuditRow maps a stored audit.Record to the operator wire row.
func toAuditRow(r audit.Record) auditRow {
return auditRow{
Seq: r.Seq,
Time: r.Time.UTC().Format(time.RFC3339Nano),
Org: r.Actor.Org,
Sub: r.Actor.Sub,
Email: r.Actor.Email,
Action: r.Action,
Resource: r.Resource.Type,
ResourceID: r.Resource.ID,
Method: r.Method,
Path: r.Path,
Result: r.Outcome.Result,
Status: r.Outcome.Status,
Reason: r.Outcome.Reason,
SourceIP: r.SourceIP,
UserAgent: r.UserAgent,
RequestID: r.RequestID,
IsAdmin: r.Auth.IsAdmin,
Auth: r.Auth.Method,
Hash: r.Hash,
PrevHash: r.PrevHash,
}
}
// auditFromIAM is the legacy federated view: when cloud has no local audit store,
// forward the IAM get-records read verbatim (the prior behavior), so the endpoint
// still surfaces IAM's own audit trail rather than an empty list.
func (s *svc) auditFromIAM(c *zip.Ctx) error {
q := iamAuditQuery(c)
res, err := s.iam.getList(c.Context(), callerCreds(c), "/v1/iam/get-records", q)
if err != nil {
return fail(c, err.Error())
}
return okRaw(c, res.rows, res.total)
}
clients/admin/audit_test.go
+238
View File
@@ -0,0 +1,238 @@
package admin
// Tests for the store-backed /v1/admin/audit + /v1/admin/audit/verify surface.
// They wire admin against a REAL audit.Recorder (on-disk SQLite) seeded with
// records, drive requests through the whole zip app, and assert the query
// results, the integrity summary, and the global-admin gate.
import (
"context"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"path/filepath"
"testing"
"time"
fiber "github.com/gofiber/fiber/v3"
"github.com/hanzoai/cloud/audit"
"github.com/hanzoai/zip"
luxlog "github.com/luxfi/log"
)
// mountWithStore builds a zip app with admin's audit routes wired to a real audit
// store, and returns the store + a request helper. Only the audit routes are
// mounted here (the rest are covered by mount()); this keeps the store-backed
// tests focused.
func mountWithStore(t *testing.T) (*audit.Recorder, func(method, path string, hdr map[string]string) (*http.Response, []byte)) {
t.Helper()
path := filepath.Join(t.TempDir(), "audit.db")
rec, err := audit.Open(path, nil)
if err != nil {
t.Fatalf("audit.Open: %v", err)
}
t.Cleanup(func() { _ = rec.Close() })
app := zip.New(zip.Config{Logger: luxlog.New("test")})
s := &svc{adminOrg: "admin", auditStore: rec}
app.Get("/v1/admin/audit", s.guard(s.audit))
app.Get("/v1/admin/audit/verify", s.guard(s.auditVerify))
fa := app.Fiber()
do := func(method, p string, hdr map[string]string) (*http.Response, []byte) {
t.Helper()
req := httptest.NewRequest(method, p, nil)
for k, v := range hdr {
req.Header.Set(k, v)
}
resp, err := fa.Test(req, fiber.TestConfig{Timeout: 30 * time.Second})
if err != nil {
t.Fatalf("%s %s: %v", method, p, err)
}
b, _ := io.ReadAll(resp.Body)
return resp, b
}
return rec, do
}
func seedAudit(t *testing.T, rec *audit.Recorder, n int) {
t.Helper()
ctx := context.Background()
for i := 0; i < n; i++ {
_, err := rec.Append(ctx, audit.Record{
Time: time.Now().UTC(),
Actor: audit.Actor{Org: "admin", Sub: "z@hanzo.ai"},
Action: "DELETE /v1/admin/orgs",
Resource: audit.Resource{Type: "org", ID: "acme"},
Auth: audit.AuthContext{Method: "jwt", IsAdmin: true},
Outcome: audit.Outcome{Result: "success", Status: 200},
Method: "DELETE",
Path: "/v1/admin/orgs/acme",
})
if err != nil {
t.Fatalf("seed %d: %v", i, err)
}
}
}
var globalAdmin = map[string]string{"X-User-IsAdmin": "true", "X-Org-Id": "admin", "X-User-Id": "z@hanzo.ai"}
// TestAdminAudit_ReturnsRealRecords proves GET /v1/admin/audit returns the
// store's records (newest-first) with an accurate total and an integrity summary.
func TestAdminAudit_ReturnsRealRecords(t *testing.T) {
rec, do := mountWithStore(t)
seedAudit(t, rec, 5)
resp, body := do("GET", "/v1/admin/audit", globalAdmin)
if resp.StatusCode != http.StatusOK {
t.Fatalf("audit: got %d (body=%s)", resp.StatusCode, body)
}
var env struct {
Data []struct {
Seq uint64 `json:"seq"`
Action string `json:"action"`
Hash string `json:"hash"`
Result string `json:"result"`
} `json:"data"`
Data2 int `json:"data2"`
Integrity struct {
OK bool `json:"ok"`
Count uint64 `json:"count"`
} `json:"integrity"`
}
if err := json.Unmarshal(body, &env); err != nil {
t.Fatalf("decode: %v (body=%s)", err, body)
}
if env.Data2 != 5 || len(env.Data) != 5 {
t.Fatalf("got %d rows / total %d, want 5/5", len(env.Data), env.Data2)
}
if env.Data[0].Seq < env.Data[len(env.Data)-1].Seq {
t.Errorf("not newest-first: %d..%d", env.Data[0].Seq, env.Data[len(env.Data)-1].Seq)
}
if env.Data[0].Hash == "" {
t.Error("row has no hash — chain linkage not surfaced")
}
if !env.Integrity.OK || env.Integrity.Count != 5 {
t.Errorf("integrity summary = %+v, want ok/count=5", env.Integrity)
}
}
// TestAdminAudit_Filters proves the query filters (result) reach the store.
func TestAdminAudit_Filters(t *testing.T) {
rec, do := mountWithStore(t)
ctx := context.Background()
// One deny among successes.
_, _ = rec.Append(ctx, audit.Record{Action: "POST /v1/admin/roles", Actor: audit.Actor{Org: "admin"}, Outcome: audit.Outcome{Result: "deny", Status: 403}})
seedAudit(t, rec, 3)
resp, body := do("GET", "/v1/admin/audit?result=deny", globalAdmin)
if resp.StatusCode != http.StatusOK {
t.Fatalf("got %d (body=%s)", resp.StatusCode, body)
}
var env struct {
Data []map[string]any `json:"data"`
Data2 int `json:"data2"`
}
_ = json.Unmarshal(body, &env)
if env.Data2 != 1 || len(env.Data) != 1 {
t.Fatalf("result=deny returned %d/%d, want 1/1", len(env.Data), env.Data2)
}
if env.Data[0]["result"] != "deny" {
t.Errorf("filtered row result = %v, want deny", env.Data[0]["result"])
}
}
// TestAdminAudit_VerifyEndpoint proves GET /v1/admin/audit/verify returns the
// integrity result for the chain.
func TestAdminAudit_VerifyEndpoint(t *testing.T) {
rec, do := mountWithStore(t)
seedAudit(t, rec, 8)
resp, body := do("GET", "/v1/admin/audit/verify", globalAdmin)
if resp.StatusCode != http.StatusOK {
t.Fatalf("verify: got %d (body=%s)", resp.StatusCode, body)
}
var env struct {
Data struct {
OK bool `json:"ok"`
Count uint64 `json:"count"`
BrokenAt int64 `json:"brokenAt"`
HeadHash string `json:"headHash"`
} `json:"data"`
}
if err := json.Unmarshal(body, &env); err != nil {
t.Fatalf("decode: %v (body=%s)", err, body)
}
if !env.Data.OK || env.Data.Count != 8 || env.Data.BrokenAt != -1 {
t.Errorf("verify result = %+v, want ok/count=8/brokenAt=-1", env.Data)
}
if env.Data.HeadHash == "" {
t.Error("verify returned no head hash")
}
}
// TestAdminAudit_DeniedWithoutGlobalAdmin proves BOTH audit endpoints fail-closed
// 403 for a non-global-admin, and — critically — the store is NEVER read on a
// denied request (the gate runs before the handler, so no records leak to an
// unauthorized caller). We assert non-leakage by seeding records and confirming
// the denied response body contains none of them.
func TestAdminAudit_DeniedWithoutGlobalAdmin(t *testing.T) {
rec, do := mountWithStore(t)
seedAudit(t, rec, 3)
cases := []struct {
name string
hdr map[string]string
}{
{"no identity", map[string]string{}},
{"tenant admin (org != adminOrg, no minted IsAdmin)", map[string]string{"X-Org-Id": "acme", "X-User-Id": "mallory"}},
{"forged-looking but non-admin", map[string]string{"X-Org-Id": "acme"}},
}
for _, ep := range []string{"/v1/admin/audit", "/v1/admin/audit/verify"} {
for _, tc := range cases {
resp, body := do("GET", ep, tc.hdr)
if resp.StatusCode != http.StatusForbidden {
t.Errorf("%s [%s]: got %d, want 403 (body=%s)", ep, tc.name, resp.StatusCode, body)
}
// No record content must appear in a denied response.
if len(body) > 0 && (contains(body, "DELETE /v1/admin/orgs") || contains(body, `"hash"`)) {
t.Errorf("%s [%s]: denied response leaked audit data: %s", ep, tc.name, body)
}
}
}
}
// TestAdminAudit_FallsBackToIAMWhenNoStore proves that when no local store is
// configured (auditStore == nil), /v1/admin/audit still serves the federated IAM
// view rather than erroring — preserving the prior capability. Covered by the
// existing TestAudit_MapsRecords (IAM proxy path); here we assert the nil-store
// verify endpoint reports "not configured" rather than panicking.
func TestAdminAudit_VerifyWithoutStore(t *testing.T) {
app := zip.New(zip.Config{Logger: luxlog.New("test")})
s := &svc{adminOrg: "admin"} // no auditStore
app.Get("/v1/admin/audit/verify", s.guard(s.auditVerify))
req := httptest.NewRequest("GET", "/v1/admin/audit/verify", nil)
for k, v := range globalAdmin {
req.Header.Set(k, v)
}
resp, err := app.Fiber().Test(req, fiber.TestConfig{Timeout: 30 * time.Second})
if err != nil {
t.Fatalf("verify: %v", err)
}
body, _ := io.ReadAll(resp.Body)
// A well-formed error envelope, not a 500/panic.
if resp.StatusCode != http.StatusOK || !contains(body, "not configured") {
t.Errorf("nil-store verify = %d %s, want an ok-envelope 'not configured' error", resp.StatusCode, body)
}
}
func contains(b []byte, sub string) bool {
s := string(b)
for i := 0; i+len(sub) <= len(s); i++ {
if s[i:i+len(sub)] == sub {
return true
}
}
return false
}
deps.go
+9
View File
@@ -13,6 +13,7 @@ import (
"github.com/hanzoai/commerce/metering"
luxlog "github.com/luxfi/log"
"github.com/hanzoai/cloud/audit"
"github.com/hanzoai/cloud/types"
)
@@ -73,6 +74,14 @@ type Deps struct {
// (separate from the ZAP Commerce client above, which is for typed
// inter-subsystem calls). Nil or not-Enabled() makes the gate a no-op.
Metering *metering.Client
// Audit is the tamper-evident, append-only audit trail Recorder (FedRAMP AU-*
// / SOC 2 CC-*). Serve constructs it once, wires the AuditTrail middleware to
// it, and hands it here so the /v1/admin/audit query + /v1/admin/audit/verify
// endpoints read the SAME store the middleware writes. Nil makes the audit
// middleware a no-op and the query endpoint fall back to the IAM proxy (an
// unconfigured deployment is never blocked). See audit/ and audit_middleware.go.
Audit *audit.Recorder
}
// Per-subsystem client interfaces live in cloud/types so the
go.sum
+5 -5
View File
@@ -1471,7 +1471,7 @@ github.com/luxfi/accel v1.2.4 h1:5VbIHyEvvfobn2zBiTFODxDw1CeqxCepZOLlvkuf9yQ=
github.com/luxfi/accel v1.2.4/go.mod h1:ISIwAX+ZfsL/S5nsP2JvfldXN6Nc+QzoWf6Jtaq+xsQ=
github.com/luxfi/address v1.0.1 h1:Sc4keyuVzBIvHr7uVeYZf2/WY9YDGUgDi/iiWenj49g=
github.com/luxfi/address v1.0.1/go.mod h1:5j3Eh66v9zvv1GbNdZwt+23krV8JlSDaRzmWZU8ZRM0=
github.com/luxfi/age v1.5.0 h1:G69HbSV4R3vKEH9B0CulnRaMdSdf4RalMgP8xKmxHeI=
github.com/luxfi/age v1.5.0 h1:zC/Fw/ptZwAXr9nqrxmrcf8752EIl1Lq9RECp9OmCO0=
github.com/luxfi/age v1.5.0/go.mod h1:iAYAxgvrXxcy746+Ovh/eWWDuF9teJLNcCSSOX9RYW0=
github.com/luxfi/atomic v1.0.0 h1:xUV60MuzRvXngaQ1sM0yVC2v4TRoLlUGkkH7M9PS4yw=
github.com/luxfi/atomic v1.0.0/go.mod h1:0G2mTlQ6TXWHICUHrUUPu1/qAiIyR4gSZ2tva9ci/bI=
@@ -1507,7 +1507,7 @@ github.com/luxfi/go-bip39 v1.1.2 h1:p+wLMPGs6MLQh7q0YIsmy2EhHL7LHiELEGTJko6t/Jg=
github.com/luxfi/go-bip39 v1.1.2/go.mod h1:96de9VkR2kY/ASAnhMtvt3TSh+PZkAFAngNj0GjRGDo=
github.com/luxfi/ids v1.2.15 h1:omE+E4+0Poj9DzM11ejSFgteaSQ3KDHi5g54iH6jcxI=
github.com/luxfi/ids v1.2.15/go.mod h1:Fj73K5xcblvdE0SxU/ip+jE8VqNdu+80548su5KJ7xI=
github.com/luxfi/keys v1.2.0 h1:+AriQNM7FOylAEls1XvFdlSOXDfoyc6X3ZfJRWQ2I9g=
github.com/luxfi/keys v1.2.0 h1:3TAcr4twyMpwQp7J29ZRtIa5vzAoDrnXnLcPKVHJWmw=
github.com/luxfi/keys v1.2.0/go.mod h1:SjsAaxo6sGmSp9OaHXUiVCqsknO8iPspN6jMOoEAMb8=
github.com/luxfi/kms v1.11.6 h1:qLcPjurqr/GA2rC9PEDPfTNn6hEPsccuJy2gKLkqA/k=
github.com/luxfi/kms v1.11.6/go.mod h1:XhLUVqN4RBv6j4Bj3MNgTZmHCnm74jH7RqqK0b9xbzw=
@@ -1531,9 +1531,9 @@ github.com/luxfi/mock v0.1.1 h1:0HEtIjg1J6CWz+IUyP6rsGqNWTcmxjFnSQIhaDuARwY=
github.com/luxfi/mock v0.1.1/go.mod h1:jo35akl3Vtd8LbzDts8VJ0jmSVycrd1/eBi6g6t5hKU=
github.com/luxfi/p2p v1.21.1 h1:gmz1JMDhzHIL3dQlhwIDvR4OlFuhNVfnWUl/ipYhAIo=
github.com/luxfi/p2p v1.21.1/go.mod h1:SsNPR5fPGWWNem9plGWhSmRqyDoysJ3kPAN0zG0g3iw=
github.com/luxfi/pq v1.0.3 h1:pFlQm1+5FuKTDUh2y/23bXWkN4I2Rc5iuxJypwDFFMs=
github.com/luxfi/pq v1.0.3 h1:ksw1dmfTR0dqqNMRS7BjGcprCO2Fhc+3Iiq2/NMuONw=
github.com/luxfi/pq v1.0.3/go.mod h1:8bppZcRElfrVt0n3nYCZW3iX1TvhvzNbdjNdK1irgIE=
github.com/luxfi/precompile v0.5.37 h1:2v0zTZtU3cP/hlCA1602Bz//mK+9jjUkYCBsc3KU67w=
github.com/luxfi/precompile v0.5.37 h1:Yh3dJ+dYuFuzsJgIBRmAJXNsmGEb0BUJN5fzZe3yYeE=
github.com/luxfi/precompile v0.5.37/go.mod h1:z1ZLWPKPdZXqIQZOSdVObM8nTIxrblP6a+fqf3O7OHs=
github.com/luxfi/proto v1.0.0 h1:nlxv4lt/i75XDB2Q3nTqC1o0RDPmJ9K9CJ8UyYvceak=
github.com/luxfi/proto v1.0.0/go.mod h1:pZLKsCmhiPtmm3z7ezBbnsYT2m79q+cFSmzxvuBeANE=
@@ -1557,7 +1557,7 @@ github.com/luxfi/vm v1.2.0 h1:jTwQRHdC9VmyRZTPSn+IqjMju7f6xlxLc6P9CEg+Y2M=
github.com/luxfi/vm v1.2.0/go.mod h1:qasVIBRerVQuvy9vFGrX3H8X8pPMPG5un/KbZSyq5YY=
github.com/luxfi/warp v1.19.3 h1:tU6aAniYrPiRGrht7oFSwsUmk5KBb9RDFSyZq3lJ4HY=
github.com/luxfi/warp v1.19.3/go.mod h1:kqyk7Fa5mgw1zzMLqjrZWVLuTef35CCW0gOEv10UlZk=
github.com/luxfi/zap v0.8.11 h1:NIiZp/YyS1TQHfU/PHnMXynzPOKCfZOQssn5ytCdYXg=
github.com/luxfi/zap v0.8.11 h1:jT+ol9rj557MRdmnzxrVUCR3CDFaE+8OpzUsLIn92og=
github.com/luxfi/zap v0.8.11/go.mod h1:JfqII8VtVQYLLTX6obU1DP9sjGqf9L24vfug5ifh0b8=
github.com/luxfi/zapdb v1.10.0 h1:1lLHEmkyC0BucnA/zjQYsMkUVxuEo2vQkEaQGjYfuuc=
github.com/luxfi/zapdb v1.10.0/go.mod h1:Qukh3hDRD0MnxA6z+a28JTnXhN85AiLLgp6TYr4QAMc=
serve.go
+21
View File
@@ -68,6 +68,22 @@ func Serve(enable []string) error {
identity := newIdentityValidator(cfg.IAMIssuer, cfg.JWKSURL, cfg.JWTAudiences, 0)
app.Use(SanitizeIdentity(identity, cfg.AdminOrg))
// Audit trail (FedRAMP AU-* / SOC 2 CC-*). Runs AFTER SanitizeIdentity so the
// actor/isAdmin it records come from a VALIDATED principal (never a raw
// header), and BEFORE BillingGate + every subsystem so it wraps the whole
// chain and observes the final outcome — including a billing 402/503 and an
// admin 403 denial. It is the ONE place every security-relevant request is
// recorded to the tamper-evident, append-only store (see audit_middleware.go /
// audit/). A write failure fails the request CLOSED (AU-5). Constructed here
// so the Recorder lives for the process and the /v1/admin/audit query + verify
// endpoints (clients/admin) read the SAME store via deps.Audit.
auditRec, err := buildAuditRecorder(cfg, deps.Logger)
if err != nil {
return fmt.Errorf("audit: %w", err)
}
deps.Audit = auditRec
app.Use(AuditTrail(auditRec))
// Billing gate. Sits at the (future) Auth position — after identity is
// established by Recover/RequestID/Logger and before any subsystem mounts —
// so every priced route is balance-gated once, at the edge, fail-closed.
@@ -160,6 +176,11 @@ func Serve(enable []string) error {
shutdownCtx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
_ = healthSrv.Shutdown(shutdownCtx)
// Close the audit store last so any in-flight append has drained through the
// serialized writer and the SQLite file is flushed cleanly.
if auditRec != nil {
_ = auditRec.Close()
}
return app.ShutdownWithContext(shutdownCtx)
}