v0.3.3 renamed this to package zap, and that was wrong twice. Go resolves an
unaliased import to the package's own name, so github.com/zap-proto/http should
answer to http; and this package IMPORTS zap, the wire format, so naming itself
zap put two different things under one word in the same file. An importer that
also needs net/http aliases one of them, which is the ordinary convention rather
than a reason to invent a name.
zap.Version and zap.HeaderSize are that other package and are untouched.
Co-authored-by: Hanzo Dev <[email protected]>
zaphttp was a compound word invented to disambiguate a module whose path
already reads github.com/zap-proto/http. The package is the ZAP wire; the
module path says what it speaks. An importer that also needs net/http aliases
one of them, which is the ordinary Go convention rather than a reason to build
a compound name into every file.
The file that carried the name in its own filename is zap_test.go now, for the
same reason: a file is named for what it holds.
Co-authored-by: Hanzo Dev <[email protected]>
The encoder excluded Host from the headers slot as "reconstructed from the
frame", and nothing in the frame carries a host. The decoder reconstructs it
from a Host HEADER — the very one the encoder removed. So every request over
this wire arrived with an empty Host.
It stayed invisible because a callee serving one brand never reads it. It
surfaced on a callee serving several: hanzoai/iam resolves each brand's OIDC
issuer from the request Host, so mounted behind this transport it matched no
brand and fell back to a single issuer, and lux.id minted tokens claiming
iss=hanzo.id that its own relying parties reject.
Content-Length and Trailer really are frame-owned — the body's length prefix and
the trailer slot carry them. Host had no such carrier and only looked like it
belonged in that list.
Co-authored-by: Hanzo Dev <[email protected]>
v0.3.0 merged appendRequestHeaders and appendResponseHeaders into one generic
over an interface constraint, in the name of not repeating twelve lines. The
comment it deleted had explained exactly why they were separate: escape analysis
must be able to prove the VisitAll closure does not leave the stack.
A generic there compiles to a shared GC-shape instantiation —
appendHeaders[go.shape.*uint8] in the -gcflags=-m output — so the call goes
through a dictionary, the closure escapes, and every encode picked up a heap
allocation on a path this package documents and benchmarks as allocation-free.
before 82.39 ns/op 32 B/op 1 allocs/op
after 25.83 ns/op 0 B/op 0 allocs/op
Restoring the two concrete functions is 3.2x faster and allocation-free again.
Twelve duplicated lines were buying that; deleting them cost it.
Also fixes the benchmarks, which were measuring the wrong thing. benchRequest
sets only Host, which isFrameOwnedHeaderBytes strips, so its headers slot is
NULL and forEachHeader returns on its first line — the header codec these
numbers advertise was not being exercised at all. Added Real fixtures carrying
a realistic header set (auth, trace, ratelimit, and a Location with an "&" in
it, which under the old JSON codec forced the whole message onto the allocating
fallback):
AppendRequestReal 113.5 ns/op 0 B/op 0 allocs/op
AppendResponseReal 89.28 ns/op 0 B/op 0 allocs/op
UnmarshalRequestReal 406.7 ns/op 0 B/op 0 allocs/op
UnmarshalResponseReal 301.8 ns/op 0 B/op 0 allocs/op
Co-authored-by: Hanzo Dev <[email protected]>
There was a JSON encoder sitting inside a zero-copy binary protocol, and two
implementations of it: a hand-rolled zero-alloc writer in codec.go and
json.Marshal in wire.go, kept byte-identical to each other. That cost escaping,
a sort for determinism, a hand-written JSON grammar scanner with a stdlib
fallback for anything it could not parse, and a standing obligation to keep two
encoders agreeing.
Headers now travel as:
[u32 count] then count times: [u32 nameLen][name][u32 valueLen][value]
Every name and value is a subslice of the frame, so a decode allocates nothing
and there is nothing to escape or sort. A repeated name is a repeated pair,
which covers multi-value headers without modelling a list. One encoder, one
decoder, no fallback path.
Deleted: emitHeadersJSON, appendJSONString, jsonCleanASCII, forEachHeaderKV,
scanJSONStr, decodeHeaderMap, encodeHeaderMap, collectTrailers,
decodeRequestHeadersSlow, decodeResponseHeadersSlow, and the two per-message
JSON appenders — 189 lines from codec.go, 83 from wire.go. encoding/json is
gone from the wire entirely.
Also gone: the reference JSON codec in the tests and the cross-checks against
it. MarshalRequest/MarshalResponse are thin wrappers over Append*, so those
tests were comparing a function to itself once the second encoder disappeared.
Replaced with round-trip coverage of the values JSON needed escaping for
(quotes, backslashes, HTML bytes, non-ASCII), repeated names, truncation, and
an AllocsPerRun assertion pinning decode at zero.
BREAKING: the headers and trailer slots change shape, so peers must agree on
the version. Golden vectors regenerated deliberately; goldenEmptyRequest is
unchanged, since no headers is a null slot either way.
BenchmarkUnmarshalRequest 48.00 ns/op 0 B/op 0 allocs/op
BenchmarkUnmarshalResponse 93.77 ns/op 0 B/op 0 allocs/op
Co-authored-by: Hanzo Dev <[email protected]>
The wire was already ZAP binary on both sides; only the plumbing was pinned
to TCP (net.Listen("tcp"), net.DialTimeout("tcp")). Lifting that to a value
mirrors net.Dial/net.Listen — the network is an argument, not a family of
functions — and unix sockets fall out with no change to the frames:
zaphttp.Dial("unix", "/run/hanzo/billing.sock")
&zaphttp.Server{Network: "unix", Addr: "/run/hanzo/billing.sock"}
NewTransport is gone in favour of Dial: it was a compound name for what net
already calls dialing, and keeping both would be two ways to do one thing.
All callers migrated.
Serving on unix clears a stale socket first — a crashed predecessor leaves a
file that bind refuses, so restart-in-place needs it. A socket with something
live behind it still fails, loudly, rather than being unlinked out from under
a running process.
Co-authored-by: Hanzo Dev <[email protected]>
readFrameInto passed a stack [4]byte through io.ReadFull's io.Reader
interface, forcing the length-prefix array to the heap every call (the last
2 allocs/req: server + client). Peek the prefix out of the concrete
*bufio.Reader's buffer instead — zero-copy, no escape. Full round trip is now
0 allocs/req at default GOGC; ZAP beats fasthttp on tiny/small/chat loopback.
Hand-rolled frame encoder/decoder (codec.go) replaces the zap.Builder +
map[string][]string + json.Marshal path. Wire stays byte-identical (golden
tests + TestCodec_CrossCheck vs a zap.Builder oracle across a wide corpus,
incl. special-char headers). Headers are streamed straight into the frame tail
as compact sorted-key JSON, byte-identical to json.Marshal, with a stdlib
fallback for escape/UTF-8 values; decode scans the JSON in place into the
fasthttp store with a json.Unmarshal fallback.
Server: one RequestCtx per connection (Init2 + per-request Reset) instead of a
per-request heap alloc; reused frame read/write buffers; single-syscall
length-prefixed response write.
Client: per-conn reused request/response buffers, single frame-type peek + one
decode (was double zap.Parse), single-syscall request write.
Codec allocs/op (default GOGC):
MarshalRequest 11 -> 1 (Append: 0) MarshalResponse 19 -> 1 (Append: 0)
UnmarshalResponse 11 -> 0 hot Append/Unmarshal all 0 allocs/op
Adds multi-frame streamed responses so SSE / MCP notifications / progress /
chunked bodies ride ZAP as real server→client push, not a buffered blob. A
streamed response is a head frame (status+headers) + N data frames (one per
handler flush) + an end frame — the analogue of HTTP/2 HEADERS+DATA+END_STREAM.
- wire.go: FrameResponseHead/FrameData/FrameEnd + Marshal/Unmarshal + FrameTypeOf.
- server.go: when ctx.Response.IsBodyStream(), stream via BodyWriteTo→chunkWriter
(each flush = one FrameData frame on the wire immediately). Non-stream path
unchanged (single FrameResponse) — fully backward-compatible.
- client.go: Transport.Do dispatches on frame type; a head attaches a streaming
body (streamReader) that reads data frames until end, releasing the pooled conn
only when the stream drains (or closes it if abandoned mid-stream).
Verified: TestStream_SSEOverZAP flushes 3 events one-at-a-time and the client
receives each BEFORE the next is released (a buffering regression would deadlock/
time out — the guard proves real incremental push). All existing round-trip +
keep-alive tests pass; go test -race clean; gofmt clean.
Part 1 of the fasthttp unification (Fiber + ZAP both on fasthttp). Does NOT
yet build/verify; zap-proto/zip not yet created. Do not merge.
Co-authored-by: Hanzo Dev <[email protected]>
Remove capnproto.org/go/capnp/v3; reimplement the wire codec on the pure-stdlib
github.com/zap-proto/go runtime. Public API (ListenAndServe/Server/Transport/
Marshal*) unchanged — consumers rebuild only. One wire across the ZAP stack.
Adds a free-list pool of idle TCP connections with TCP keepalive +
NoDelay. RoundTrip pulls a hot conn or dials new; release-on-Close
returns it. Pool caps at MaxIdleConns (default 32). Enables
HTTP/1.1-keepalive-style request reuse for ZAP-HTTP, closing the
sequential-throughput gap with net/http measured in
zap-proto/bench.
The wire format is unchanged (still length-prefixed framing of the
same encoded bytes), but the authoring surface is now pure ZAP:
- schema/zap_http.zap (source of truth, ZAP schema language)
- internal/wire/zap_http.go (generated bindings, package zaphttp_wire)
Drops capnp from user-facing names. The runtime still uses
capnproto.org/go/capnp/v3 for serialization plumbing — that becomes
swappable once zapc grows a Go backend; until then it's a build-time
compatibility detail, not a wire-level promise.
HTTP request/response semantics over the ZAP transport. Drop-in
net/http.Handler server and net/http.RoundTripper client; existing
handlers and http.Client code work unchanged.
Wire format: one Cap'n Proto frame per HTTP message
(schema/zap_http.capnp). v0.1 ships length-prefixed framing over TCP;
v0.2 will swap that for the full ZAP transport (X-Wing PQ KEM
handshake + AEAD framing) once zap-proto/go provides it. The
marshal/unmarshal API is stable across that change.
Includes:
- schema/zap_http.capnp (Cap'n Proto schema, source of truth)
- internal/capnp/ (generated Go bindings)
- wire.go (Marshal/Unmarshal Request/Response)
- transport.go (length-prefixed framing)
- server.go (Server + ListenAndServe)
- client.go (Transport + Get convenience)
- examples/hello (server + client in one binary)
- 7 tests covering GET, POST, large body (4 MiB), multi-value headers,
status code propagation, keep-alive, marshal idempotence
Tests pass; smoke test confirms end-to-end exchange works.
zeekayandHanzo Dev