Files
gateway/zap_wire.go
zeekayandGitHub 69b5e25d47 feat(gateway): HIP-0110 standalone edge-process binary (#17)
* feat(middleware): own JWT validation + identity-strip + trust helper

Moves Auth + StripIdentityHeaders from zip/middleware to
gateway/middleware. Adds gateway/trust.go exporting AssertGatewayMinted
for downstream subsystems to verify the request flowed through gateway.

The unified cloud binary's pipeline now reads:
  zip generic middleware (Recover, Logger, RequestID, Telemetry, ...)
  gateway.middleware.StripIdentityHeaders
  gateway.middleware.Auth(verifier)
  ... downstream subsystem handlers ...

* feat(gateway): HIP-0110 ZAP wire envelopes + BuildApp entry

* feat(gateway): cmd/gateway HIP-0110 edge-process main

* refactor(gateway): tighten HIP-0110 surface — 588 LOC across 4 files

* build(gateway): fix module pins, MsgType uint16 overflow, drop legacy jaeger

- go.mod: hanzoai/cloud v0.0.0 → v0.1.0, hanzoai/zip v0.0.0 → v0.1.1
  (zip v0.1.1 carries the json/v2-at-edge work merged in zip#3;
   cloud v0.1.0 carries the types-leaf + clients refactor).
- zap_wire.go: MsgTypeForward/Push/Subscribe were 0x1010/0x1020/0x1030
  (uint16) but zap.FinishWithFlags expects the type in the upper byte
  (type << 8). Anything > 0xFF overflows uint16. Repacked into 0x80/0x81/0x82
  — still upper-half of the byte (no collision with base's 100-103 IDs).
- executor.go: dropped krakend-opencensus/v2/exporter/jaeger blank import.
  The exporter's Thrift API is incompatible with current jaeger-client-go;
  inter-service tracing is migrating to OTel (kotel) + ZAP anyway.
- cmd/gateway/main.go: c.SendString → c.Fiber().SendString (zip.Ctx
  exposes Send via the underlying fiber.Ctx; matches the pattern used
  for binary responses).

Verified: go build -mod=mod ./... clean. Binary at /tmp/gateway = 108 MiB.
TestStrip+TestAuth+TestSecure all pass.

* test(gateway): per-conn memory profile — 8 KiB/conn, 1:1 goroutine

Raw-TCP-dial loadgen (not http.Client, which spawns 3 client-side
goroutines that contaminate the gateway-side measurement). Spins up
zip.App on a local listener, dials N raw TCP conns each holding
/hold, samples runtime.MemStats.HeapAlloc delta.

Apple M1 Max, Go 1.26, zip on Fiber v3 v3.2.0:

  1,000 conns: 8.05 KiB/conn, 1.00 goroutine/conn,  14 MiB heap
  8,454 conns: 8.02 KiB/conn, 1.00 goroutine/conn,  72 MiB heap (linear)

Discipline: fasthttp gives us 1 goroutine per accepted conn already.
The HIP-0110 architecture preserves that — ZAP-forward to cloud is
sync from the gateway goroutine, no fan-out. Reverse-push lookups
are O(1) on a per-replica sync.Map, no extra goroutines.

Extrapolation:
  10k  conns → ~80 MiB / replica
  100k conns → ~800 MiB (1 GiB pod ceiling)
  10 replicas behind ingress → 1M concurrent client conns

Cloud side has zero per-conn cost — worker pool dispatches N ZAP
requests per pod. This is why a stateless gateway LB in front of
cloud maximizes compute efficiency: 10-100× fewer cloud replicas
than gateway replicas required for the same client population.

* deps: bump zip v0.1.1 → v0.2.0; pin cloud at resolvable pseudo-version

cloud cannot be tagged yet due to upstream GOPROXY blockers (kms repo >500MB,
vfs missing root package at v0.3.1). Pin cloud at the pseudo-version derived
from main HEAD so consumers can resolve via proxy.golang.org without local
replace dirs.
2026-05-29 19:59:38 -07:00

154 lines
4.8 KiB
Go

// Copyright © 2026 Hanzo AI. MIT License.
// zap_wire.go is the gateway-side ZAP wire contract per HIP-0110.
//
// Two envelopes cover the entire gateway↔backend boundary:
// Forward — gateway → backend, one envelope per inbound HTTP request.
// Push — backend → gateway, one envelope per server-initiated push.
// On-wire bytes match github.com/luxfi/zap. Field offsets follow the
// same packing as base/plugins/zap.
package gateway
import (
"context"
"sync"
zaplib "github.com/luxfi/zap"
)
// HIP-0110 message-type IDs. zap.FinishWithFlags expects the type in the
// upper byte of a uint16 (`type << 8`), so types must fit in uint8.
// Picked in the 0x80+ range so they don't collide with base/plugins/zap's
// lower-byte IDs (Collections=100, Records=101, Auth=102, Realtime=103).
const (
MsgTypeForward uint16 = 0x80
MsgTypePush uint16 = 0x81
MsgTypeSubscribe uint16 = 0x82
)
const (
fwdIsAdmin, fwdPermissions = 0, 4
fwdTenantID, fwdUserID = 12, 24
fwdMethod, fwdPath = 36, 48
fwdConnID = 60
fwdHeaders, fwdBody = 72, 84
fwdSlotSize = 96
pushConnID, pushFrame, pushEncoding = 0, 12, 24
pushSlotSize = 36
)
// Canonical Encoding values on a Push envelope.
const (
EncSSE = "sse"
EncWSText = "ws-text"
EncWSBinary = "ws-binary"
)
// Forward is the typed view of the gateway→backend envelope.
type Forward struct {
TenantID string // X-Org-Id (JWT 'owner')
UserID string // X-User-Id (JWT 'sub')
IsAdmin bool // X-User-IsAdmin (JWT 'isAdmin')
Permissions int64 // X-User-Permissions (bit.Field)
Method string // GET, POST, ...
Path string // /v1/iam/users/123
Headers []byte // canonicalized header map
Body []byte // raw client body, verbatim
ConnID string // gateway-assigned conn id for reverse push
}
// Push is the typed view of the backend→gateway reverse-push envelope.
type Push struct {
ConnID string
Frame []byte // pre-encoded SSE / WebSocket payload
Encoding string // "sse" | "ws-text" | "ws-binary"
}
// BuildForward serializes f into a ZAP message.
func BuildForward(f Forward) (*zaplib.Message, error) {
b := zaplib.NewBuilder(256 + len(f.Headers) + len(f.Body))
ob := b.StartObject(fwdSlotSize)
ob.SetBool(fwdIsAdmin, f.IsAdmin)
ob.SetInt64(fwdPermissions, f.Permissions)
ob.SetText(fwdTenantID, f.TenantID)
ob.SetText(fwdUserID, f.UserID)
ob.SetText(fwdMethod, f.Method)
ob.SetText(fwdPath, f.Path)
ob.SetText(fwdConnID, f.ConnID)
ob.SetBytes(fwdHeaders, f.Headers)
ob.SetBytes(fwdBody, f.Body)
ob.FinishAsRoot()
return zaplib.Parse(b.FinishWithFlags(MsgTypeForward << 8))
}
// ReadForward decodes a Forward envelope from a parsed message.
func ReadForward(msg *zaplib.Message) Forward {
r := msg.Root()
return Forward{
TenantID: r.Text(fwdTenantID),
UserID: r.Text(fwdUserID),
IsAdmin: r.Bool(fwdIsAdmin),
Permissions: r.Int64(fwdPermissions),
Method: r.Text(fwdMethod),
Path: r.Text(fwdPath),
Headers: r.Bytes(fwdHeaders),
Body: r.Bytes(fwdBody),
ConnID: r.Text(fwdConnID),
}
}
// BuildPush serializes p into a ZAP message.
func BuildPush(p Push) (*zaplib.Message, error) {
b := zaplib.NewBuilder(64 + len(p.Frame))
ob := b.StartObject(pushSlotSize)
ob.SetText(pushConnID, p.ConnID)
ob.SetBytes(pushFrame, p.Frame)
ob.SetText(pushEncoding, p.Encoding)
ob.FinishAsRoot()
return zaplib.Parse(b.FinishWithFlags(MsgTypePush << 8))
}
// ReadPush decodes a Push envelope.
func ReadPush(msg *zaplib.Message) Push {
r := msg.Root()
return Push{
ConnID: r.Text(pushConnID),
Frame: r.Bytes(pushFrame),
Encoding: r.Text(pushEncoding),
}
}
// PushSink delivers a Push frame to a client connection. gateway.BuildApp
// registers the canonical impl backed by an in-memory map keyed by ConnID.
type PushSink interface {
Deliver(ctx context.Context, p Push) error
}
var pushRegistry sync.Map // ConnID → PushSink
// RegisterReversePush records the sink for an active SSE / WS subscription.
func RegisterReversePush(connID string, sink PushSink) { pushRegistry.Store(connID, sink) }
// UnregisterReversePush is called when the client conn closes.
func UnregisterReversePush(connID string) { pushRegistry.Delete(connID) }
// HandleReversePush is the zaplib.Handler installed by cmd/gateway/main.
// Decodes the Push envelope, looks up the matching sink, delivers. Unknown
// ConnID (client gone) is a silent no-op.
func HandleReversePush(ctx context.Context, from string, msg *zaplib.Message) (*zaplib.Message, error) {
p := ReadPush(msg)
v, ok := pushRegistry.Load(p.ConnID)
if !ok {
return nil, nil
}
sink, ok := v.(PushSink)
if !ok {
return nil, nil
}
if err := sink.Deliver(ctx, p); err != nil {
pushRegistry.Delete(p.ConnID)
return nil, err
}
return nil, nil
}