Do took no context, so the only bound on an exchange was the transport-wide read
timeout. A caller whose own deadline was 30s could wait fifteen minutes, and a
cancelled caller could not stop a read already on the wire at all.
DoContext sets the conn deadline from ctx and closes the conn when ctx is
cancelled — the one thing that unblocks a read in flight. A conn closed that way
is never pooled, and the error is ctx's cause. A streamed body keeps ctx's hold
until the stream ends. The dial honours ctx too.
Do is DoContext with a background context, so the Client shape that
fasthttp.HostClient satisfies is unchanged.
Co-authored-by: Hanzo Dev <[email protected]>
Bumped the go directive to 1.27.1. go build, go vet and the full test
suite all pass clean on the 1.27.1 toolchain.
Co-authored-by: Hanzo Dev <[email protected]>
Half of net.go was me explaining the design at length. A reader needs two facts:
these are net/http's own types, and the net/http shapes allocate where fasthttp
does not. Everything else was restating the code above it.
63 lines to 39, same behaviour, same tests.
Co-authored-by: Hanzo Dev <[email protected]>
The package is called http and now looks like it. Request, Response, Header,
Handler, HandlerFunc, ResponseWriter and ServeMux are ALIASES of net/http's own
types, so an existing handler satisfies them exactly and middleware, httptest
and mux compose for free. A declared struct with matching fields would be a
different type — every caller converting, every middleware failing — and drop-in
would be a claim rather than a fact.
ListenAndServe takes the standard Handler, and there is one of it. A second one
taking fasthttp.RequestHandler used to sit in server.go, so there were two ways
to start a server differing only in which handler shape you happened to hold. A
caller with a fasthttp handler composes &Server{Addr, Handler} — one line that
says what it is.
The cost is named rather than hidden: net/http's Request and ResponseWriter
allocate where fasthttp's do not, and this package guards a zero-alloc encode.
So the fast path is untouched — Server.Handler is still a fasthttp handler and
nothing in this file is in its way. Adapt is exported so the bridge is visible at
the call site, which is where a net/http Request gets materialised.
The example is an ordinary net/http handler now. Nothing in it knows about ZAP,
which is the whole demonstration.
Co-authored-by: Hanzo Dev <[email protected]>
It documented 'make schema' regenerating internal/wire/zap_http.go — there is no
schema target and no internal/ directory, and the Makefile says so in a comment
three lines up: no external schema toolchain. schema/http.zap documents the wire;
it does not generate it.
A README that names machinery nobody has is worse than a short one, because the
reader spends their time looking for it.
Co-authored-by: Hanzo Dev <[email protected]>
Last one. A target that names a missing file fails at the moment someone tries
to regenerate, which is the moment they least want to debug the build.
Co-authored-by: Hanzo Dev <[email protected]>
I renamed the file and left three references behind — wire.go's format note and
two in the README, one of them the ASCII frame diagram. A doc that names a file
which is not there is worse than no doc: the reader concludes the format is
undocumented.
Co-authored-by: Hanzo Dev <[email protected]>
main carried a six-week-old conversion marked INCOMPLETE that in fact builds and
passes — and still dropped Host on the wire and still called itself zaphttp. This
branch carried the fix and the rename. They were one commit apart in the other
direction, a schema doc, so this merges rather than replaces.
schema/zap_http.zap is schema/http.zap: the module is already zap-proto/http, so
the file said it twice.
Co-authored-by: Hanzo Dev <[email protected]>
Sixty error strings still opened with "zaphttp: " and the package comment still
declared Package zaphttp. An error names the package it came from, so a reader
grepping for the source of "zaphttp: truncated header block" would have found a
package that no longer exists under that name.
Co-authored-by: Hanzo Dev <[email protected]>
v0.3.3 renamed this to package zap, and that was wrong twice. Go resolves an
unaliased import to the package's own name, so github.com/zap-proto/http should
answer to http; and this package IMPORTS zap, the wire format, so naming itself
zap put two different things under one word in the same file. An importer that
also needs net/http aliases one of them, which is the ordinary convention rather
than a reason to invent a name.
zap.Version and zap.HeaderSize are that other package and are untouched.
Co-authored-by: Hanzo Dev <[email protected]>
zaphttp was a compound word invented to disambiguate a module whose path
already reads github.com/zap-proto/http. The package is the ZAP wire; the
module path says what it speaks. An importer that also needs net/http aliases
one of them, which is the ordinary Go convention rather than a reason to build
a compound name into every file.
The file that carried the name in its own filename is zap_test.go now, for the
same reason: a file is named for what it holds.
Co-authored-by: Hanzo Dev <[email protected]>
The encoder excluded Host from the headers slot as "reconstructed from the
frame", and nothing in the frame carries a host. The decoder reconstructs it
from a Host HEADER — the very one the encoder removed. So every request over
this wire arrived with an empty Host.
It stayed invisible because a callee serving one brand never reads it. It
surfaced on a callee serving several: hanzoai/iam resolves each brand's OIDC
issuer from the request Host, so mounted behind this transport it matched no
brand and fell back to a single issuer, and lux.id minted tokens claiming
iss=hanzo.id that its own relying parties reject.
Content-Length and Trailer really are frame-owned — the body's length prefix and
the trailer slot carry them. Host had no such carrier and only looked like it
belonged in that list.
Co-authored-by: Hanzo Dev <[email protected]>
d05974f moved headers and trailers from a JSON map to length-prefixed
pairs and regenerated the golden vectors. schema/zap_http.zap — which
calls itself the source of truth — was never updated, so the one file a
reimplementer reads first described a JSON map<Text, List(Text)> and a
`List(Header)` struct list that no encoder has emitted since.
Three things were wrong, all of them the kind that only surface as a
byte mismatch in someone else's language:
- headers/trailer are Data slots holding `count` then
{keyLen,key,valLen,value}*, all little-endian. A repeated name is
repeated pairs; there is no values list. An empty block is a null
{0,0} slot, NOT a count=0 block.
- streaming was "reserved for a follow-up slice"; it shipped.
FrameResponseHead/FrameData/FrameEnd are 0x03/0x04/0x05, so the
closing note's "today there are exactly two" type IDs was off by
three.
- the preamble credited a zap-proto/go runtime dependency that
codec.go replaced by being its own encoder and decoder.
Also written down because a port gets them wrong once each: relOffset
is measured from the slot's own position, relOffset==0 is NULL whatever
the length word says, the frame is little-endian but the transport's
length prefix is big-endian, and Host/Content-Length are frame-owned so
they must never appear as pairs.
No code changes — the schema is documentation, not a codegen input
(Makefile: "no code-generation step"). wire.go's offsets remain the
binding; this makes the document agree with them.
Co-authored-by: Hanzo Dev <[email protected]>
v0.3.0 merged appendRequestHeaders and appendResponseHeaders into one generic
over an interface constraint, in the name of not repeating twelve lines. The
comment it deleted had explained exactly why they were separate: escape analysis
must be able to prove the VisitAll closure does not leave the stack.
A generic there compiles to a shared GC-shape instantiation —
appendHeaders[go.shape.*uint8] in the -gcflags=-m output — so the call goes
through a dictionary, the closure escapes, and every encode picked up a heap
allocation on a path this package documents and benchmarks as allocation-free.
before 82.39 ns/op 32 B/op 1 allocs/op
after 25.83 ns/op 0 B/op 0 allocs/op
Restoring the two concrete functions is 3.2x faster and allocation-free again.
Twelve duplicated lines were buying that; deleting them cost it.
Also fixes the benchmarks, which were measuring the wrong thing. benchRequest
sets only Host, which isFrameOwnedHeaderBytes strips, so its headers slot is
NULL and forEachHeader returns on its first line — the header codec these
numbers advertise was not being exercised at all. Added Real fixtures carrying
a realistic header set (auth, trace, ratelimit, and a Location with an "&" in
it, which under the old JSON codec forced the whole message onto the allocating
fallback):
AppendRequestReal 113.5 ns/op 0 B/op 0 allocs/op
AppendResponseReal 89.28 ns/op 0 B/op 0 allocs/op
UnmarshalRequestReal 406.7 ns/op 0 B/op 0 allocs/op
UnmarshalResponseReal 301.8 ns/op 0 B/op 0 allocs/op
Co-authored-by: Hanzo Dev <[email protected]>
There was a JSON encoder sitting inside a zero-copy binary protocol, and two
implementations of it: a hand-rolled zero-alloc writer in codec.go and
json.Marshal in wire.go, kept byte-identical to each other. That cost escaping,
a sort for determinism, a hand-written JSON grammar scanner with a stdlib
fallback for anything it could not parse, and a standing obligation to keep two
encoders agreeing.
Headers now travel as:
[u32 count] then count times: [u32 nameLen][name][u32 valueLen][value]
Every name and value is a subslice of the frame, so a decode allocates nothing
and there is nothing to escape or sort. A repeated name is a repeated pair,
which covers multi-value headers without modelling a list. One encoder, one
decoder, no fallback path.
Deleted: emitHeadersJSON, appendJSONString, jsonCleanASCII, forEachHeaderKV,
scanJSONStr, decodeHeaderMap, encodeHeaderMap, collectTrailers,
decodeRequestHeadersSlow, decodeResponseHeadersSlow, and the two per-message
JSON appenders — 189 lines from codec.go, 83 from wire.go. encoding/json is
gone from the wire entirely.
Also gone: the reference JSON codec in the tests and the cross-checks against
it. MarshalRequest/MarshalResponse are thin wrappers over Append*, so those
tests were comparing a function to itself once the second encoder disappeared.
Replaced with round-trip coverage of the values JSON needed escaping for
(quotes, backslashes, HTML bytes, non-ASCII), repeated names, truncation, and
an AllocsPerRun assertion pinning decode at zero.
BREAKING: the headers and trailer slots change shape, so peers must agree on
the version. Golden vectors regenerated deliberately; goldenEmptyRequest is
unchanged, since no headers is a null slot either way.
BenchmarkUnmarshalRequest 48.00 ns/op 0 B/op 0 allocs/op
BenchmarkUnmarshalResponse 93.77 ns/op 0 B/op 0 allocs/op
Co-authored-by: Hanzo Dev <[email protected]>
The wire was already ZAP binary on both sides; only the plumbing was pinned
to TCP (net.Listen("tcp"), net.DialTimeout("tcp")). Lifting that to a value
mirrors net.Dial/net.Listen — the network is an argument, not a family of
functions — and unix sockets fall out with no change to the frames:
zaphttp.Dial("unix", "/run/hanzo/billing.sock")
&zaphttp.Server{Network: "unix", Addr: "/run/hanzo/billing.sock"}
NewTransport is gone in favour of Dial: it was a compound name for what net
already calls dialing, and keeping both would be two ways to do one thing.
All callers migrated.
Serving on unix clears a stale socket first — a crashed predecessor leaves a
file that bind refuses, so restart-in-place needs it. A socket with something
live behind it still fails, loudly, rather than being unlinked out from under
a running process.
Co-authored-by: Hanzo Dev <[email protected]>
readFrameInto passed a stack [4]byte through io.ReadFull's io.Reader
interface, forcing the length-prefix array to the heap every call (the last
2 allocs/req: server + client). Peek the prefix out of the concrete
*bufio.Reader's buffer instead — zero-copy, no escape. Full round trip is now
0 allocs/req at default GOGC; ZAP beats fasthttp on tiny/small/chat loopback.
Hand-rolled frame encoder/decoder (codec.go) replaces the zap.Builder +
map[string][]string + json.Marshal path. Wire stays byte-identical (golden
tests + TestCodec_CrossCheck vs a zap.Builder oracle across a wide corpus,
incl. special-char headers). Headers are streamed straight into the frame tail
as compact sorted-key JSON, byte-identical to json.Marshal, with a stdlib
fallback for escape/UTF-8 values; decode scans the JSON in place into the
fasthttp store with a json.Unmarshal fallback.
Server: one RequestCtx per connection (Init2 + per-request Reset) instead of a
per-request heap alloc; reused frame read/write buffers; single-syscall
length-prefixed response write.
Client: per-conn reused request/response buffers, single frame-type peek + one
decode (was double zap.Parse), single-syscall request write.
Codec allocs/op (default GOGC):
MarshalRequest 11 -> 1 (Append: 0) MarshalResponse 19 -> 1 (Append: 0)
UnmarshalResponse 11 -> 0 hot Append/Unmarshal all 0 allocs/op
Adds multi-frame streamed responses so SSE / MCP notifications / progress /
chunked bodies ride ZAP as real server→client push, not a buffered blob. A
streamed response is a head frame (status+headers) + N data frames (one per
handler flush) + an end frame — the analogue of HTTP/2 HEADERS+DATA+END_STREAM.
- wire.go: FrameResponseHead/FrameData/FrameEnd + Marshal/Unmarshal + FrameTypeOf.
- server.go: when ctx.Response.IsBodyStream(), stream via BodyWriteTo→chunkWriter
(each flush = one FrameData frame on the wire immediately). Non-stream path
unchanged (single FrameResponse) — fully backward-compatible.
- client.go: Transport.Do dispatches on frame type; a head attaches a streaming
body (streamReader) that reads data frames until end, releasing the pooled conn
only when the stream drains (or closes it if abandoned mid-stream).
Verified: TestStream_SSEOverZAP flushes 3 events one-at-a-time and the client
receives each BEFORE the next is released (a buffering regression would deadlock/
time out — the guard proves real incremental push). All existing round-trip +
keep-alive tests pass; go test -race clean; gofmt clean.
Part 1 of the fasthttp unification (Fiber + ZAP both on fasthttp). Does NOT
yet build/verify; zap-proto/zip not yet created. Do not merge.
Co-authored-by: Hanzo Dev <[email protected]>
Remove capnproto.org/go/capnp/v3; reimplement the wire codec on the pure-stdlib
github.com/zap-proto/go runtime. Public API (ListenAndServe/Server/Transport/
Marshal*) unchanged — consumers rebuild only. One wire across the ZAP stack.
Adds a free-list pool of idle TCP connections with TCP keepalive +
NoDelay. RoundTrip pulls a hot conn or dials new; release-on-Close
returns it. Pool caps at MaxIdleConns (default 32). Enables
HTTP/1.1-keepalive-style request reuse for ZAP-HTTP, closing the
sequential-throughput gap with net/http measured in
zap-proto/bench.
The wire format is unchanged (still length-prefixed framing of the
same encoded bytes), but the authoring surface is now pure ZAP:
- schema/zap_http.zap (source of truth, ZAP schema language)
- internal/wire/zap_http.go (generated bindings, package zaphttp_wire)
Drops capnp from user-facing names. The runtime still uses
capnproto.org/go/capnp/v3 for serialization plumbing — that becomes
swappable once zapc grows a Go backend; until then it's a build-time
compatibility detail, not a wire-level promise.
HTTP request/response semantics over the ZAP transport. Drop-in
net/http.Handler server and net/http.RoundTripper client; existing
handlers and http.Client code work unchanged.
Wire format: one Cap'n Proto frame per HTTP message
(schema/zap_http.capnp). v0.1 ships length-prefixed framing over TCP;
v0.2 will swap that for the full ZAP transport (X-Wing PQ KEM
handshake + AEAD framing) once zap-proto/go provides it. The
marshal/unmarshal API is stable across that change.
Includes:
- schema/zap_http.capnp (Cap'n Proto schema, source of truth)
- internal/capnp/ (generated Go bindings)
- wire.go (Marshal/Unmarshal Request/Response)
- transport.go (length-prefixed framing)
- server.go (Server + ListenAndServe)
- client.go (Transport + Get convenience)
- examples/hello (server + client in one binary)
- 7 tests covering GET, POST, large body (4 MiB), multi-value headers,
status code propagation, keep-alive, marshal idempotence
Tests pass; smoke test confirms end-to-end exchange works.
zeekay
