Files
zeekay 3da6271eaf zapclient: native ZAP RPC with procedure-name dispatch — replaces clienthttp
clienthttp blended ZAP with HTTP by sitting on top of zap-proto/http
(http.RoundTripper). That mixed two protocols that should stay
separate: ZAP is its own binary wire, HTTP is for the public Ingress
edge and browser clients. zapclient is the canonical native ZAP
service-to-service path.

Decomplected into orthogonal pieces:

  - Discovery — peer enumeration. Default: luxfi/mdns adapter.
    Swap via WithDiscovery for Consul/etcd/static.
  - Picker — per-call peer selection. Default: round-robin. Swap
    via WithPicker for weighted/locality-aware/sticky.
  - PeerVerifier — server-side authorisation. Default:
    LocalTrustVerifier (accepts any mTLS-authenticated cluster
    peer). AllowListVerifier offered for per-procedure NodeID
    allow lists; custom impls plug in via WithVerifier.
  - Procedure codec — FNV-1a name → uint16 opcode with reserved-
    range avoidance (0x00 + 0xff). Registration refuses opcode
    collisions; callers rename until distinct.

Client API:

  c, _ := zapclient.Connect(ctx, "liquidity-ta", zapclient.WithTLS(...))
  defer c.Close()
  resp, err := c.Call(ctx, "ListSecurity", req)
  c.Send(ctx, "AuditEvent", evt)
  c.Broadcast(ctx, "Shutdown", nil)

Server API:

  srv, _ := zapclient.NewServer("liquidity-ta", zapclient.WithServerTLS(...))
  srv.Register("ListSecurity", handler)
  srv.Start(); defer srv.Stop()

ARCHITECTURE.md and SECURITY.md document the trust model: cluster
CA mTLS is the load-bearing root; LocalTrust is the default verifier
for in-cluster peers; AllowList provides explicit per-procedure
RBAC. HTTP/SSE/WS stay for browser + external clients at the
Ingress edge.
2026-05-13 09:49:37 -07:00

5.1 KiB

zapclient — Security & Trust Model

Mental model

ZAP is its own binary protocol with its own TLS handshake. There is no HTTP Authorization header. Identity, encryption, and authorisation come from three orthogonal sources:

  1. Identity — the mTLS peer cert, verified by the cluster CA during the ZAP transport handshake.
  2. Encryption — ML-KEM key exchange + AES-256-GCM bulk, baked into the ZAP TLS suite. Confidentiality survives a future quantum adversary (harvest-now-decrypt-later).
  3. Authorisation — PeerVerifier, run server-side per call. Decides whether the authenticated peer is allowed to invoke this procedure.

These three are decomplected. Swapping out the TLS suite does not touch the verifier. Replacing LocalTrustVerifier with an AllowListVerifier does not require code changes in the transport.

Two trust postures

LocalTrustVerifier (default)

srv, _ := zapclient.NewServer("liquidity-ta",
    zapclient.WithServerTLS(clusterMTLS()),
    // verifier omitted → LocalTrustVerifier
)

Accepts any peer that completed the cluster-CA mTLS handshake. Use when:

  • The peer is reachable only on the cluster-private network.
  • The cluster CA is private to the cluster (not public-web PKI).
  • All in-cluster services should be allowed to call all of this service's procedures. Per-procedure RBAC, if required, lives in the handler — not in the transport.

AllowListVerifier

srv, _ := zapclient.NewServer("liquidity-ta",
    zapclient.WithVerifier(zapclient.AllowListVerifier{
        Allowed: map[string]map[string]struct{}{
            "ListSecurity":   {"bd-prod-0": {}, "bd-prod-1": {}},
            "DelistSecurity": {"bd-prod-0": {}, "bd-prod-1": {}},
            "AddIdentity":    {"bd-prod-0": {}, "bd-prod-1": {}, "kyc-1": {}},
        },
    }),
)

Restricts each procedure to a named NodeID set. Use when:

  • Stricter posture than "any cluster peer can call any procedure" is required (e.g., a treasury procedure that only the BD pods should invoke).
  • An audit trail of "who can call what" needs to live in the source code, not just runtime config.

A procedure not present in the Allowed map is disabled — the verifier returns ErrPeerUnauthorized for every caller. This makes new procedures fail-closed by default; opt-in is explicit.

Custom Verifier

Implement PeerVerifier for richer policy (e.g. attestation-bound NodeIDs, JWT-extracted org IDs, organisation membership lookups). The verifier runs in the hot path; keep it sub-millisecond and free of network I/O.

Defence-in-depth checklist

Even under LocalTrustVerifier, server-side handlers MUST still:

  • Authorise the action. Trust mode says "you're inside the cluster"; it does not say "you can do this thing." Apply business RBAC per call regardless of mode.
  • Validate input. ZAP framing is zero-copy — trust the bytes came from a real peer, not that the bytes are well-formed.
  • Rate-limit by NodeID. A compromised in-cluster pod should not be able to DoS its neighbours.
  • Audit-log the actor. PeerInfo.NodeID (and TLSCertSubject if available) belongs in the audit envelope. Trust mode does not change what gets logged.

Threat model

Threat LocalTrustVerifier AllowListVerifier
Compromised in-cluster pod tries to call privileged procedure Pod gets in (it has a valid cluster cert) — handler-level RBAC must catch it Pod gets rejected at the verifier unless its NodeID is in the allow list
L2 attacker on same network with rogue mDNS announcement mTLS handshake fails — attacker has no cluster-CA-signed cert Same — mTLS catches it before the verifier runs
Cluster CA private key compromise Adversary forges valid certs and impersonates real services; catastrophic Same — the CA is the load-bearing root in both modes
Stale peer answers with cached data Stale peer has a valid cert; verifier accepts; data is wrong but authenticated Same — staleness is a data problem, not a transport problem
Cross-cluster call from a public-internet peer Should be impossible — mDNS doesn't cross L3 boundaries and the public peer has no cluster CA cert Same

The cluster CA is the load-bearing root. Treat its key like the most sensitive secret in the cluster: KMS-resident, ML-KEM wrapped where applicable, rotation drills tested twice a year.

ZAP vs HTTP — why this matters for security

  • HTTP bearer keys are a finger trap: easy to leak, hard to rotate, audit-burden grows linearly with the number of S2S edges.
  • mTLS over the cluster CA scales sub-linearly: rotate the CA once and every service follows; revoke a single pod by re-issuing its leaf cert with a shorter TTL.
  • Procedure-name dispatch (Call("ListSecurity", req)) makes authorisation decisions per call site, not per endpoint URL. The verifier sees the procedure name, the peer identity, and the context — that is all the input an authoriser needs.

For web/external clients (browsers, partners) the bearer pattern still applies, served by the Ingress edge over HTTP. The internal S2S fabric uses ZAP exclusively.