zap-proto/zip
1
0
Fork 0
mirror of https://github.com/zap-proto/zip.git synced 2026-10-19 15:38:57 +00:00
Code Issues Packages Projects Releases Wiki Activity

zip in Rust serves on hyper, and an op can stream

Browse Source 
An op is an async fn. It may take &zip::Cx (request headers; the
answer's status, headers and trailers) and a zip::Body read as it
arrives, and may answer zip::Sse or zip::Body, each chunk written as
the op yields it. A client that leaves drops the op's stream.

The blocking std::net door is gone: HTTP/1.1 keep-alive and h2c on
one port (hyper_util auto), graceful shutdown bounded by a grace.
The ZAP door runs on tokio and streams with zap-proto/http's head,
data and end frames.

Wire names are serde's: the derive reads rename, rename_all, skip
and with, and the hand-rolled JSON is deleted. The info manifest
comes out byte for byte as before.

No zapgen emits Rust, so the crate never built; the frame codec is
written in zap.rs and pinned to frames Go wrote. The length prefix
is big-endian, as zap-proto/http's transport writes it.

examples/stream is a JSON op, an SSE op with a trailer and a byte
relay, tested over raw HTTP/1.1, h2c and ZAP sockets.

Claude-Session: https://claude.ai/code/session_01SCPMGSQsLNF2YtHFL4bwyp
This commit is contained in:
zeekay
2026-10-08 12:56:07 -07:00
parent dc323c13d3
commit c7694a7285
42 changed files with 4444 additions and 1320 deletions
Download Patch File Download Diff File Expand all files Collapse all files
LLM.md
+54 -1
View File
@@ -14,8 +14,11 @@ Checkout: `~/work/zap/zip` (`~/work/zap-proto` links to `~/work/zap`). `~/work/h
| race | `go test -race ./...` |
| collector module | `cd contract && go test -count=1 ./...` |
| example module | `cd examples/local-service && go tool zipdoc -check && go test -count=1 ./...` |
| Rust, every crate | `cd rust && cargo test --workspace` (about 2 s once built) |
| Rust lint | `cd rust && cargo clippy --workspace --all-targets && cargo fmt --all -- --check` |
| Rust documents | `cd rust/examples/<info\|stream> && make gen` (needs Go: `zipc` is this checkout's `cmd/zipc`) |
`./...` does not reach `contract/` or `examples/local-service/`. Each has its own go.mod with a `replace` pointing at this checkout, so it tests the working tree. CI is `hanzo.yml`, run by `.hanzo/workflows/cicd.yml` through `hanzoai/ci`: vet, the root tests and both module checks.
`./...` does not reach `contract/`, `examples/local-service/` or `rust/`. Each has its own go.mod with a `replace` pointing at this checkout, so it tests the working tree. CI is `hanzo.yml`, run by `.hanzo/workflows/cicd.yml` through `hanzoai/ci`: vet, the root tests and both module checks.
`TestSDK_CallsTheLiveServiceOverZAP` and `TestProseReachesAServiceInPackageMain` compile programs offline against this checkout (`GOPROXY=off`, zip's go.sum) and skip under `-short`.
@@ -46,6 +49,8 @@ Patch tags, one above the highest `v*` on GitHub. Push the commit to main, read
| `middleware/`, `wsx/`, `js/` | middleware; WebSocket upgrades; goja handlers |
| `contract/` | collector contract tests, a module of its own |
| `examples/local-service/` | the README's service, a module of its own |
| `rust/zip`, `rust/zip-macros` | zip in Rust: `#[zip::ops]`, `#[derive(zip::Wire)]`, the hyper and ZAP doors. See Rust below. |
| `rust/examples/info`, `rust/examples/stream` | the conformance corpus in Rust; a streaming service (JSON, SSE, byte relay) and its socket tests |
## Invariants
@@ -65,6 +70,54 @@ Patch tags, one above the highest `v*` on GitHub. Push the commit to main, read
- `app.Test` runs `prepare`, so `/mcp`, the OpenAPI document and the call plane answer under test as they do when serving.
- The `/docs` page loads Swagger UI from cdn.jsdelivr.net. Embedding swagger-ui-dist 5 would add about 1.74 MB (bundle 1,552,209 bytes, CSS 185,784 bytes, Apache-2.0) to every binary that links zip.
- A manifest op may say `"stream": "sse"` or `"bytes"` (the answer is written as produced, and `ProjectOpenAPI` publishes `text/event-stream` or `application/octet-stream` for it) and `"raw": true` (the request body is bytes, bound to no type). `Check` refuses a stream beside an `out` and a raw body beside an `in`. Go's own typed ops set neither; the Rust front end sets both.
## Rust
`rust/` is a Cargo workspace: `zip` (runtime), `zip-macros` (the two macros), `examples/info` (the corpus, compared byte for byte with `corpus/info` by `conformance_test.go`) and `examples/stream`. It is not published to crates.io; a service takes it as a git dependency pinned by commit, under the key `zip`, since the macros expand to `::zip::` paths (a service that also needs crates.io's `zip` archive crate renames that one):
```toml
zip = { git = "https://github.com/zap-proto/zip", rev = "<commit>" }
```
An op is an `async fn` in an `#[zip::ops]` impl block, with a route attribute and a doc comment. It takes `&self`, optionally `cx: &zip::Cx`, and at most one input: a `Wire + Deserialize` type bound from body, query, path and declared headers, or `body: zip::Body`, the request body as it arrives. It answers `Result<T, zip::Error>`: a `Wire + Serialize` type (200 JSON), `()` (204), `zip::Sse` or `zip::Body`.
```rust
#[zip::ops(app = "chat", title = "Chat", version = "1.0.0")]
impl Chat {
/// Complete streams a completion as server-sent events.
#[post("/v1/chat/completions")]
async fn complete(&self, cx: &zip::Cx, arg: &Request) -> Result<zip::Sse, zip::Error> {
let key = cx.header("authorization").ok_or(zip::Error::new(401, "no key"))?;
let usage = cx.trailer("x-usage")?; // declared now, set at the end
let events = self.upstream(key, arg).await? // impl Stream<Item = Result<Event, E>>
.map(|chunk| chunk.map(|c| zip::Event::data(c)));
Ok(zip::Sse::new(events).keep(Duration::from_secs(15)))
}
/// Relay passes bytes through, both ways, as they arrive.
#[post("/v1/relay")]
async fn relay(&self, cx: &zip::Cx, body: zip::Body) -> Result<zip::Body, zip::Error> {
cx.set_header("content-type", "text/event-stream")?;
Ok(body)
}
}
```
Serve with `zip::listen(app, Some(":8000"), Some(":9653"), zip::signal(), grace).await`, or one door with `zip::http::serve(Arc<App>, TcpListener, shutdown, grace)` / `zip::zaphttp::serve(...)`. `rust/examples/stream/src/lib.rs` is the worked example and `tests/serve.rs` its socket tests.
- Names are serde's. `#[derive(zip::Wire)]` reads `#[serde(rename)]`, `rename_all`, `skip` and `with`; it refuses `flatten`, a rename that differs by direction, `#[zip(json = "name")]` on a field and a `Vec<u8>` without `#[serde(with = "zip::base64")]` (serde writes bytes as a number array; the document says base64). `#[zip(...)]` holds what serde cannot: `url`, `header`, `required`, `text`, and a type's own schema (`json = r#"{...}"#`). A `text` newtype gets its serde impls from the derive (Display and FromStr). `serde_json::Value` is described as `prim: any`. A field the body may omit is an `Option` or carries `#[serde(default)]`; serde refuses a missing one otherwise.
- `bind` reads the body straight into the type when no URL value, declared header or required field applies; otherwise through a `serde_json::Value` it merges them into. A body past `MAX_BODY` (32 MiB) is 413, by its Content-Length before any byte is read.
- A stream item is a `Result`. `Ok` is written and flushed as yielded; `Err` ends the answer without its terminator (HTTP/1.1 truncated chunked body, HTTP/2 reset, ZAP no end frame), since the status already went out. A producer that wants to report in words sends an event and ends.
- Client gone: hyper drops the response body and with it the op's stream. HTTP/1.1 sees it on the read side (hyper's mid-message EOF check, `half_close` off), so an idle producer is dropped too: 1 to 3 ms in `tests/serve.rs`. `Sse::keep` writes `:` comment lines in quiet intervals for proxies.
- Trailers: `cx.trailer(name)` declares (`Trailer:` in the head) and returns a handle the stream keeps; set values go after the last chunk. HTTP/2 always carries them. hyper sends HTTP/1.1 trailers only to a request carrying `TE: trailers`, and only declared names. A JSON answer with a declared trailer goes chunked so it can carry one. Over ZAP they ride the end frame's slot (zap-proto/http v0.3.12's Go client reads past them without surfacing them).
- `serve` stops accepting on `shutdown`, lets each connection finish what it is answering, and after `grace` drops the rest (their streams' producers with them). hyper's HTTP/1.1 header read timeout is on (a timer is set).
- HTTP/1.1 keep-alive and HTTP/2 over cleartext (prior knowledge) on one port (`hyper_util` auto). No TLS: the ingress terminates it.
- The ZAP door speaks zap-proto/http's frames: request 0x01, response 0x02, streamed head 0x03, data 0x04, end 0x05, each preceded by its length as a BIG-endian u32 (header blocks are little-endian). The codec is `rust/zip/src/zap.rs`, written by hand because no zapgen emits Rust (zap-proto/go's tests assert the `rust` backend fails); `zap::tests::frames_are_the_ones_go_writes` pins it to frames Go wrote. Before this, the Rust door wrote a little-endian prefix and its build ran `zapgen -lang rust`, so it never built. A Go `zap-proto/http` client reads this door's JSON, streams (chunk by chunk) and refusals.
- A handler that panics answers 500; the server lives.
Bench (dgx, shared: load 12, GPU at 96%; `examples/stream` release, server pinned to 4 Cortex-X925 cores, `bombardier` on 4 others, `POST /v1/echo` with `{"text":"hi"}`, 10 s): 64 connections 1,201,848 requests (120k/s, p50 288 µs, p99 1.78 ms); 256 connections 1,672,052 (167k/s, p50 0.86 ms, p99 32 ms).
## Known issue
`zipdoc -check ./...` from a consumer's module root has reported files as stale that per-package runs call clean, and which files it reported varied between runs (seen in hanzoai/cloud). Run `-check` in each package directory, the way `go generate` runs it.
rust/Cargo.lock
Generated
+448 -6
View File
@@ -3,12 +3,244 @@
version = 4
[[package]]
name = "info"
version = "0.1.0"
name = "atomic-waker"
version = "1.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0"
[[package]]
name = "bytes"
version = "1.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fc652a48c352aef3ea3aed32080501cf3ef6ed5da78602a020c991775b0aff04"
[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "errno"
version = "0.3.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
dependencies = [
"libc",
"windows-sys",
]
[[package]]
name = "fnv"
version = "1.0.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1"
[[package]]
name = "futures-channel"
version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b1f9e3d69d39e4862ffed03ed071a76f9a13ba1d9109d355b0f0aa6b15e393c4"
dependencies = [
"futures-core",
]
[[package]]
name = "futures-core"
version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92d699e522242e69e3003b94ecc1f960f3a5e015aa7c5d7486e65ad01dd94f5e"
[[package]]
name = "futures-sink"
version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1944426bf7d03f1d14f708785e4b33efd750b36d48a157b836b3efc15ede8e1d"
[[package]]
name = "futures-task"
version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cd417de3d1d015fc3bfd2b1ea46dfc7bab72ef86f1cc7cc9c78e728b34a6d1fd"
[[package]]
name = "futures-util"
version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0d50a92467f8ba5dd6e3ee5d4bd04d73ab2e4e1c44474a0674821dfce14b79bc"
dependencies = [
"futures-core",
"futures-task",
"pin-project-lite",
"slab",
]
[[package]]
name = "h2"
version = "0.4.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7d29020232d6aa3fb1daca64c1127cf662cf97f254ae16c18c05b8ab635fc118"
dependencies = [
"atomic-waker",
"bytes",
"fnv",
"futures-core",
"futures-sink",
"http",
"indexmap",
"slab",
"tokio",
"tokio-util",
"tracing",
]
[[package]]
name = "hashbrown"
version = "0.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
[[package]]
name = "http"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "918d3568bebf352712bc2ef3d46a8bcf1a75b373be6539de198e9105cbbf9ce0"
dependencies = [
"bytes",
"itoa",
]
[[package]]
name = "http-body"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ca2a8f2913ee65f60facd6a5905613afaa448497a0230cc41ce022d93290bc2c"
dependencies = [
"bytes",
"http",
]
[[package]]
name = "http-body-util"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "23169fe34a5fbcdd3f3862e78fb9b6fccd5f02a6dc6f732547005d45631ce71c"
dependencies = [
"bytes",
"futures-core",
"http",
"http-body",
"pin-project-lite",
]
[[package]]
name = "httparse"
version = "1.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6dbf3de79e51f3d586ab4cb9d5c3e2c14aa28ed23d180cf89b4df0454a69cc87"
[[package]]
name = "httpdate"
version = "1.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df3b46402a9d5adb4c86a0cf463f42e19994e3ee891101b1841f30a545cb49a9"
[[package]]
name = "hyper"
version = "1.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2c3e324da4c95177d6291d4c8730197c0d1822f8a9766814a4a44fa5ab797c9c"
dependencies = [
"atomic-waker",
"bytes",
"futures-channel",
"futures-core",
"h2",
"http",
"http-body",
"httparse",
"httpdate",
"itoa",
"pin-project-lite",
"smallvec",
"tokio",
"want",
]
[[package]]
name = "hyper-util"
version = "0.1.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ddc03d96684f9226b8a787cdb71488417b53ab5ea8fdb1dac946cb9431cc8bff"
dependencies = [
"bytes",
"http",
"http-body",
"hyper",
"pin-project-lite",
"tokio",
]
[[package]]
name = "indexmap"
version = "2.14.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cc4e190f5d26ca7051642629da2c52fc03bde85a03197c99408dcd291734c855"
dependencies = [
"equivalent",
"hashbrown",
]
[[package]]
name = "info"
version = "0.2.0"
dependencies = [
"serde",
"tokio",
"zip",
]
[[package]]
name = "itoa"
version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "libc"
version = "0.2.190"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ce5d3ddc6d3fa000eb1536d85e147bfe31aacaba692ed6a876f95cb7c855be78"
[[package]]
name = "memchr"
version = "2.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98"
[[package]]
name = "mio"
version = "1.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1788edb87fdc09c7e26304471e2f5be8cdefb1b6930d6e3985fc02ff53bf86ee"
dependencies = [
"libc",
"wasi",
"windows-sys",
]
[[package]]
name = "once_cell"
version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]]
name = "pin-project-lite"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
[[package]]
name = "proc-macro2"
version = "1.0.107"
@@ -27,6 +259,95 @@ dependencies = [
"proc-macro2",
]
[[package]]
name = "serde"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba"
dependencies = [
"serde_core",
"serde_derive",
]
[[package]]
name = "serde_core"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.6",
]
[[package]]
name = "serde_json"
version = "1.0.151"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14"
dependencies = [
"itoa",
"memchr",
"serde",
"serde_core",
"zmij",
]
[[package]]
name = "signal-hook-registry"
version = "1.4.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c4db69cba1110affc0e9f7bcd48bbf87b3f4fc7c61fc9155afd4c469eb3d6c1b"
dependencies = [
"errno",
"libc",
]
[[package]]
name = "slab"
version = "0.4.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5"
[[package]]
name = "smallvec"
version = "1.16.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f9395f0f0eee849a9b707b2f06bb92a6a422090e2123bb2ef8e87a0e61892a8e"
[[package]]
name = "socket2"
version = "0.6.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3d1e2c7f27f8d4cb10542a02c49005dbd6e93095799d6f3be745fae9f8fedd4"
dependencies = [
"libc",
"windows-sys",
]
[[package]]
name = "stream"
version = "0.2.0"
dependencies = [
"bytes",
"futures-util",
"http-body-util",
"hyper",
"hyper-util",
"serde",
"tokio",
"zip",
]
[[package]]
name = "syn"
version = "2.0.119"
@@ -38,6 +359,83 @@ dependencies = [
"unicode-ident",
]
[[package]]
name = "syn"
version = "3.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8593e8e72159ed2257d083c7a454a85cbf854f37a0966d8d483aff8c8a3ebcee"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "tokio"
version = "1.53.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e95f91fcc7a621e8b030f6aa23c71fe9838ae2fb4d8118b75602a328f5144044"
dependencies = [
"bytes",
"libc",
"mio",
"pin-project-lite",
"signal-hook-registry",
"socket2",
"tokio-macros",
"windows-sys",
]
[[package]]
name = "tokio-macros"
version = "2.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78773a2a397f451582ce068015985c33193cf6dea8b74d2a639fe457b2f07b0e"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.6",
]
[[package]]
name = "tokio-util"
version = "0.7.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "494815d09bf52b5548659851081238f0ca39ff638363907596da739561c62c52"
dependencies = [
"bytes",
"futures-core",
"futures-sink",
"libc",
"pin-project-lite",
"tokio",
]
[[package]]
name = "tracing"
version = "0.1.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100"
dependencies = [
"pin-project-lite",
"tracing-core",
]
[[package]]
name = "tracing-core"
version = "0.1.36"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a"
dependencies = [
"once_cell",
]
[[package]]
name = "try-lock"
version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b"
[[package]]
name = "unicode-ident"
version = "1.0.24"
@@ -45,17 +443,61 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "zip"
version = "0.1.0"
name = "want"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec4cdd0dd910afe868b7ef477227d8d538b46b3075031afee8a9f2acb0a2ed0b"
dependencies = [
"try-lock",
]
[[package]]
name = "wasi"
version = "0.11.1+wasi-snapshot-preview1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b"
[[package]]
name = "windows-link"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-sys"
version = "0.61.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
dependencies = [
"windows-link",
]
[[package]]
name = "zip"
version = "0.2.0"
dependencies = [
"bytes",
"futures-util",
"http-body-util",
"hyper",
"hyper-util",
"serde",
"serde_json",
"tokio",
"zip-macros",
]
[[package]]
name = "zip-macros"
version = "0.1.0"
version = "0.2.0"
dependencies = [
"proc-macro2",
"quote",
"syn",
"syn 2.0.119",
]
[[package]]
name = "zmij"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b"
rust/Cargo.toml
+2 -2
View File
@@ -1,8 +1,8 @@
[workspace]
resolver = "2"
members = ["zip", "zip-macros", "examples/info"]
members = ["zip", "zip-macros", "examples/info", "examples/stream"]
[workspace.package]
version = "0.1.0"
version = "0.2.0"
edition = "2021"
license = "BSD-3-Clause-Eco"
rust/examples/info/Cargo.toml
+2
View File
@@ -7,3 +7,5 @@ publish = false
[dependencies]
zip = { path = "../../zip" }
serde = { version = "1", features = ["derive"] }
tokio = { version = "1.53", features = ["rt-multi-thread", "macros"] }
rust/examples/info/Makefile
+4 -8
View File
@@ -11,15 +11,14 @@
# make gen compile, project, and compile again against the projections
# make run serve on the default addresses
# make check the doors answer, and the wire is ZAP
# make wire the ZAP accessors for these ops, for a client that wants them
# gen names a directory as well as a step, and the step is the one meant.
.PHONY: gen run check wire
.PHONY: gen run check
ZIPC ?= zipc
ZAPGEN ?= zapgen
# zipc is this checkout's, so the documents are projected by the projector
# the manifest was written for.
ZIPC ?= go run ../../../cmd/zipc
ROOT := ../..
OUT ?= wire
gen:
cd $(ROOT) && cargo run -q -p info -- --manifest > examples/info/gen/manifest.json
@@ -31,6 +30,3 @@ run:
check:
cd $(ROOT) && cargo test -p info
wire:
$(ZAPGEN) -lang rust -out $(OUT) gen/info.zap
rust/examples/info/src/main.rs
+6 -2
View File
@@ -8,7 +8,8 @@
//! info --http 127.0.0.1:8000 --zap 127.0.0.1:9653
//! ```
fn main() {
#[tokio::main]
async fn main() {
let mut http = Some("127.0.0.1:8000".to_string());
let mut zap = Some("127.0.0.1:9653".to_string());
let args: Vec<String> = std::env::args().skip(1).collect();
@@ -47,7 +48,10 @@ fn main() {
if let Some(a) = &zap {
println!("info: zap on {a}");
}
if let Err(e) = zip::listen(app, http.as_deref(), zap.as_deref()) {
// A pod is sent SIGTERM and given 30 s before SIGKILL; the drain takes 25
// of them, so what is still streaming at the end is cut by us, not the kernel.
let grace = std::time::Duration::from_secs(25);
if let Err(e) = zip::listen(app, http.as_deref(), zap.as_deref(), zip::signal(), grace).await {
eprintln!("info: {e}");
std::process::exit(1);
}
rust/examples/info/src/ops.rs
+14 -14
View File
@@ -33,7 +33,7 @@ impl Info {
///
/// Response: {"version": "luxd/1.36.178", "databaseVersion": "v1.4.5", "rpcProtocolVersion": 39, "gitCommit": ""}
#[get("/node/version")]
fn node_version(&self) -> Result<NodeVersion, zip::Error> {
async fn node_version(&self) -> Result<NodeVersion, zip::Error> {
Ok(NodeVersion {
version: self.release.clone(),
database_version: "v1.4.5".into(),
@@ -47,7 +47,7 @@ impl Info {
///
/// Response: {"nodeID": "NodeID-7Xhw2mDxuDS44j42TCB6U5579esbSt3Lg", "nodePOP": {"publicKey": "0x8f95", "proofOfPossession": "0x86a3"}}
#[get("/node/id")]
fn identity(&self) -> Result<NodeID, zip::Error> {
async fn identity(&self) -> Result<NodeID, zip::Error> {
Ok(NodeID {
node_id: "NodeID-7Xhw2mDxuDS44j42TCB6U5579esbSt3Lg".into(),
node_pop: ProofOfPossession {
@@ -61,7 +61,7 @@ impl Info {
///
/// Response: {"ip": "203.0.113.9:9651"}
#[get("/node/ip")]
fn address(&self) -> Result<NodeIP, zip::Error> {
async fn address(&self) -> Result<NodeIP, zip::Error> {
Ok(NodeIP {
ip: "203.0.113.9:9651".into(),
})
@@ -71,7 +71,7 @@ impl Info {
///
/// Response: {"networkID": 96369}
#[get("/network/id")]
fn network_id(&self) -> Result<NetworkID, zip::Error> {
async fn network_id(&self) -> Result<NetworkID, zip::Error> {
Ok(NetworkID {
network_id: self.network,
})
@@ -81,7 +81,7 @@ impl Info {
///
/// Response: {"networkName": "mainnet"}
#[get("/network/name")]
fn network_name(&self) -> Result<NetworkName, zip::Error> {
async fn network_name(&self) -> Result<NetworkName, zip::Error> {
Ok(NetworkName {
network_name: "mainnet".into(),
})
@@ -92,7 +92,7 @@ impl Info {
/// Example: {"alias": "X"}
/// Response: {"blockchainID": "2oYMBNV4eNHyqk2fjjV5nVQLDbtmNJzq5s3qs3Lo6ftnC6FByM"}
#[get("/chain/id")]
fn chain_id(&self, arg: &ChainAlias) -> Result<ChainID, zip::Error> {
async fn chain_id(&self, arg: &ChainAlias) -> Result<ChainID, zip::Error> {
if arg.alias.is_empty() {
return Err(zip::Error::bad("argument 'alias' not given"));
}
@@ -106,7 +106,7 @@ impl Info {
/// Example: {"chain": "X"}
/// Response: {"isBootstrapped": true}
#[get("/chain/bootstrapped")]
fn bootstrapped(&self, arg: &BootstrappedArgs) -> Result<Bootstrapped, zip::Error> {
async fn bootstrapped(&self, arg: &BootstrappedArgs) -> Result<Bootstrapped, zip::Error> {
if arg.chain.is_empty() {
return Err(zip::Error::bad("argument 'chain' not given"));
}
@@ -120,7 +120,7 @@ impl Info {
///
/// Response: {"chains": [{"id": "11111111111111111111111111111111LpoYY", "name": "P-Chain", "vmID": "11111111111111111111111111111111LpoYY", "bootstrapped": true}]}
#[get("/chains")]
fn chains(&self) -> Result<Chains, zip::Error> {
async fn chains(&self) -> Result<Chains, zip::Error> {
Ok(Chains {
chains: vec![Chain {
id: "11111111111111111111111111111111LpoYY".into(),
@@ -137,7 +137,7 @@ impl Info {
/// Example: {"nodeIDs": []}
/// Response: {"numPeers": "1", "peers": [{"ip": "203.0.113.9:9651", "publicIP": "203.0.113.9:9651", "nodeID": "NodeID-7Xhw2mDxuDS44j42TCB6U5579esbSt3Lg", "version": "luxd/1.36.178", "observedUptime": 99, "trackedChains": []}]}
#[get("/peers")]
fn peers(&self, _arg: &PeersArgs) -> Result<Peers, zip::Error> {
async fn peers(&self, _arg: &PeersArgs) -> Result<Peers, zip::Error> {
let peers = vec![Peer {
ip: "203.0.113.9:9651".into(),
public_ip: "203.0.113.9:9651".into(),
@@ -156,7 +156,7 @@ impl Info {
///
/// Response: {"lps": {"23": {"supportWeight": "0", "objectWeight": "0", "abstainWeight": "1000000000000"}}}
#[get("/lps")]
fn lps(&self) -> Result<LPs, zip::Error> {
async fn lps(&self) -> Result<LPs, zip::Error> {
let mut lps = HashMap::new();
lps.insert(
"23".to_string(),
@@ -173,7 +173,7 @@ impl Info {
///
/// Response: {"vms": {"mgj786NP7uDwBCcq6YwThhaN8FLyybkCa4zBWTQbNgmK6k9A6": ["platformvm"]}, "fxs": {"spqBHsy2UBGpjaQJezEywUy1AB98eVjJ3WQ38x1vpjsx4xPkY": "secp256k1fx"}}
#[get("/vms")]
fn vms(&self) -> Result<VMs, zip::Error> {
async fn vms(&self) -> Result<VMs, zip::Error> {
let mut vms = HashMap::new();
vms.insert(
"mgj786NP7uDwBCcq6YwThhaN8FLyybkCa4zBWTQbNgmK6k9A6".to_string(),
@@ -191,7 +191,7 @@ impl Info {
///
/// Response: {"stopVertexID": "jrGWDh5Po9FMj54depyunNixpia5PN4aAYxfmNzU8n752Rjga", "epochDuration": "300000000000"}
#[get("/upgrades")]
fn upgrades(&self) -> Result<Upgrades, zip::Error> {
async fn upgrades(&self) -> Result<Upgrades, zip::Error> {
Ok(Upgrades {
stop_vertex_id: "jrGWDh5Po9FMj54depyunNixpia5PN4aAYxfmNzU8n752Rjga".into(),
epoch_duration: Uint64(300_000_000_000),
@@ -202,7 +202,7 @@ impl Info {
///
/// Response: {"rewardingStakePercentage": "100.0000", "weightedAveragePercentage": "99.9999"}
#[get("/uptime")]
fn uptime(&self) -> Result<Uptime, zip::Error> {
async fn uptime(&self) -> Result<Uptime, zip::Error> {
Ok(Uptime {
rewarding_stake_percentage: "100.0000".into(),
weighted_average_percentage: "99.9999".into(),
@@ -213,7 +213,7 @@ impl Info {
///
/// Response: {"txFee": "1000000", "createAssetTxFee": "10000000", "createChainTxFee": "1000000000"}
#[get("/fees")]
fn fees(&self) -> Result<Fees, zip::Error> {
async fn fees(&self) -> Result<Fees, zip::Error> {
Ok(Fees {
tx_fee: Uint64(1_000_000),
create_asset_tx_fee: Uint64(10_000_000),
rust/examples/info/src/types.rs
+87 -63
View File
@@ -15,6 +15,7 @@
use std::collections::HashMap;
use serde::{Deserialize, Serialize};
use zip::Wire;
/// Uint64 is a 64-bit count carried as a quoted decimal, because JSON numbers
@@ -34,232 +35,255 @@ impl From<u64> for Uint64 {
}
/// NodeVersion is what this node is running.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct NodeVersion {
/// Version is the node's own release.
#[zip(json = "version")]
#[serde(rename = "version")]
pub version: String,
/// DatabaseVersion is the database format this node reads.
#[zip(json = "databaseVersion")]
#[serde(rename = "databaseVersion")]
pub database_version: String,
/// RPCProtocolVersion is the VM protocol this node speaks.
#[zip(json = "rpcProtocolVersion")]
#[serde(rename = "rpcProtocolVersion")]
pub rpc_protocol_version: u32,
/// GitCommit is the commit this node was built from.
#[zip(json = "gitCommit")]
#[serde(rename = "gitCommit")]
pub git_commit: String,
}
/// NodeID is this node's id, with the proof it holds the staking key.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct NodeID {
/// NodeID is the id peers know this node by.
#[zip(json = "nodeID")]
#[serde(rename = "nodeID")]
pub node_id: String,
/// NodePOP is the proof this node holds the key its id is derived from.
#[zip(json = "nodePOP")]
#[serde(rename = "nodePOP")]
pub node_pop: ProofOfPossession,
}
/// ProofOfPossession is a staking key and a signature over it.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct ProofOfPossession {
/// PublicKey is the BLS public key, hex encoded.
#[zip(json = "publicKey")]
#[serde(rename = "publicKey")]
pub public_key: String,
/// ProofOfPossession is the signature over that key, hex encoded.
#[zip(json = "proofOfPossession")]
#[serde(rename = "proofOfPossession")]
pub proof_of_possession: String,
}
/// NodeIP is where this node tells its peers to reach it.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct NodeIP {
/// IP is the address and port peers dial.
#[zip(json = "ip")]
#[serde(rename = "ip")]
pub ip: String,
}
/// NetworkID is the number of the network a node is on.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct NetworkID {
/// NetworkID is the network's number.
#[zip(json = "networkID")]
#[serde(rename = "networkID")]
pub network_id: u32,
}
/// NetworkName is the name of the network a node is on.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct NetworkName {
/// NetworkName is the network's name.
#[zip(json = "networkName")]
#[serde(rename = "networkName")]
pub network_name: String,
}
/// ChainAlias names the chain a caller is asking about.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct ChainAlias {
/// Alias is the name the chain answers to, such as "X".
#[zip(json = "alias", required)]
#[serde(rename = "alias")]
#[zip(required)]
pub alias: String,
}
/// ChainID is the id an alias names.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct ChainID {
/// BlockchainID is the chain's id.
#[zip(json = "blockchainID")]
#[serde(rename = "blockchainID")]
pub blockchain_id: String,
}
/// BootstrappedArgs names the chain whose bootstrap state is being asked about.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct BootstrappedArgs {
/// Chain is the alias or id of the chain to ask about.
#[zip(json = "chain", required)]
#[serde(rename = "chain")]
#[zip(required)]
pub chain: String,
}
/// Bootstrapped is whether a chain has finished bootstrapping.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct Bootstrapped {
/// IsBootstrapped is true once the chain has caught up.
#[zip(json = "isBootstrapped")]
#[serde(rename = "isBootstrapped")]
pub is_bootstrapped: bool,
}
/// Chain is one chain this node runs.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct Chain {
/// ID is the chain's id.
#[zip(json = "id")]
#[serde(rename = "id")]
pub id: String,
/// Name is the chain's name.
#[zip(json = "name")]
#[serde(rename = "name")]
pub name: String,
/// VMID is the id of the virtual machine the chain runs.
#[zip(json = "vmID")]
#[serde(rename = "vmID")]
pub vm_id: String,
/// Bootstrapped is whether this node has caught the chain up.
#[zip(json = "bootstrapped")]
#[serde(rename = "bootstrapped")]
pub bootstrapped: bool,
}
/// Chains are the chains this node runs.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct Chains {
/// Chains are the chains this node is running.
#[zip(json = "chains")]
#[serde(rename = "chains")]
pub chains: Vec<Chain>,
}
/// PeersArgs narrows the peer list to the nodes named.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct PeersArgs {
/// NodeIDs are the peers to answer for; none asks for all of them.
#[zip(json = "nodeIDs")]
#[serde(rename = "nodeIDs")]
pub node_ids: Vec<String>,
}
/// Peer is one connection this node holds.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct Peer {
/// IP is the address this node reaches the peer at.
#[zip(json = "ip")]
#[serde(rename = "ip")]
pub ip: String,
/// PublicIP is the address the peer tells others to use.
#[zip(json = "publicIP")]
#[serde(rename = "publicIP")]
pub public_ip: String,
/// NodeID is the peer's id.
#[zip(json = "nodeID")]
#[serde(rename = "nodeID")]
pub node_id: String,
/// Version is the release the peer is running.
#[zip(json = "version")]
#[serde(rename = "version")]
pub version: String,
/// ObservedUptime is the share of time this node has seen the peer up.
#[zip(json = "observedUptime")]
#[serde(rename = "observedUptime")]
pub observed_uptime: u32,
/// TrackedChains are the chains the peer is following.
#[zip(json = "trackedChains")]
#[serde(rename = "trackedChains")]
pub tracked_chains: Vec<String>,
}
/// Peers are the nodes this one is connected to.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct Peers {
/// NumPeers is how many connections this node holds.
#[zip(json = "numPeers")]
#[serde(rename = "numPeers")]
pub num_peers: Uint64,
/// Peers are the connections themselves.
#[zip(json = "peers")]
#[serde(rename = "peers")]
pub peers: Vec<Peer>,
}
/// LP is where the stake stands on one proposal.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct LP {
/// SupportWeight is the stake in favour.
#[zip(json = "supportWeight")]
#[serde(rename = "supportWeight")]
pub support_weight: Uint64,
/// ObjectWeight is the stake against.
#[zip(json = "objectWeight")]
#[serde(rename = "objectWeight")]
pub object_weight: Uint64,
/// AbstainWeight is the stake that has not said.
#[zip(json = "abstainWeight")]
#[serde(rename = "abstainWeight")]
pub abstain_weight: Uint64,
}
/// LPs is where the network's stake stands on every current proposal.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct LPs {
/// LPs are the proposals, by number.
#[zip(json = "lps")]
#[serde(rename = "lps")]
pub lps: HashMap<String, LP>,
}
/// VMs are the virtual machines installed on this node.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct VMs {
/// VMs are the installed machines, by id, each with the names it answers to.
#[zip(json = "vms")]
#[serde(rename = "vms")]
pub vms: HashMap<String, Vec<String>>,
/// Fxs are the feature extensions the UTXO chains understand, by id.
#[zip(json = "fxs")]
#[serde(rename = "fxs")]
pub fxs: HashMap<String, String>,
}
/// Upgrades is the upgrade schedule this node runs.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct Upgrades {
/// StopVertexID is the vertex the linearisation stopped at.
#[zip(json = "stopVertexID")]
#[serde(rename = "stopVertexID")]
pub stop_vertex_id: String,
/// EpochDuration is how long one epoch lasts, in nanoseconds.
#[zip(json = "epochDuration")]
#[serde(rename = "epochDuration")]
pub epoch_duration: Uint64,
}
/// Uptime is how much of the network reports having seen this node up.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct Uptime {
/// RewardingStakePercentage is the share of stake that would reward this node.
#[zip(json = "rewardingStakePercentage")]
#[serde(rename = "rewardingStakePercentage")]
pub rewarding_stake_percentage: String,
/// WeightedAveragePercentage is the stake-weighted average uptime.
#[zip(json = "weightedAveragePercentage")]
#[serde(rename = "weightedAveragePercentage")]
pub weighted_average_percentage: String,
}
/// Fees are the transaction fees this node charges, in nLUX.
#[derive(Wire, Default)]
#[derive(Wire, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct Fees {
/// TxFee is the fee for an ordinary transaction.
#[zip(json = "txFee")]
#[serde(rename = "txFee")]
pub tx_fee: Uint64,
/// CreateAssetTxFee is the fee to create an asset.
#[zip(json = "createAssetTxFee")]
#[serde(rename = "createAssetTxFee")]
pub create_asset_tx_fee: Uint64,
/// CreateChainTxFee is the fee to create a chain.
#[zip(json = "createChainTxFee")]
#[serde(rename = "createChainTxFee")]
pub create_chain_tx_fee: Uint64,
}
rust/examples/info/tests/doors.rs
+37 -27
View File
@@ -11,34 +11,32 @@
use std::io::{Read, Write};
use std::net::TcpStream;
use std::sync::Arc;
use std::time::Duration;
fn doors() -> (String, String) {
let app = Arc::new(info::service("luxd/1.36.178", 96369));
let http = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let zap = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let (ha, za) = (
let addrs = (
http.local_addr().unwrap().to_string(),
zap.local_addr().unwrap().to_string(),
);
drop(http);
drop(zap);
for (addr, over) in [(ha.clone(), true), (za.clone(), false)] {
let app = Arc::clone(&app);
std::thread::spawn(move || {
if over {
let _ = zip::http::listen(app, &addr);
} else {
let _ = zip::zaphttp::listen(app, &addr);
}
http.set_nonblocking(true).unwrap();
zap.set_nonblocking(true).unwrap();
std::thread::spawn(move || {
let rt = tokio::runtime::Runtime::new().unwrap();
rt.block_on(async move {
let app = Arc::new(info::service("luxd/1.36.178", 96369));
let http = tokio::net::TcpListener::from_std(http).unwrap();
let zap = tokio::net::TcpListener::from_std(zap).unwrap();
let grace = Duration::from_secs(1);
let never = std::future::pending::<()>;
let _ = tokio::join!(
zip::http::serve(Arc::clone(&app), http, never(), grace),
zip::zaphttp::serve(app, zap, never(), grace),
);
});
}
for _ in 0..200 {
if TcpStream::connect(&ha).is_ok() && TcpStream::connect(&za).is_ok() {
break;
}
std::thread::sleep(std::time::Duration::from_millis(10));
}
(ha, za)
});
addrs
}
/// over_http asks one question in text and answers with the body.
@@ -56,25 +54,24 @@ fn over_http(addr: &str, target: &str) -> (u16, String) {
/// over_zap asks the same question in ZAP frames and answers with the body.
fn over_zap(addr: &str, target: &str) -> (u16, String) {
let mut c = TcpStream::connect(addr).unwrap();
let frame = zip::zaphttp::ask("GET", target, &[]);
c.write_all(&(frame.len() as u32).to_le_bytes()).unwrap();
let frame = zip::zap::request("GET", target, &[], &[]);
c.write_all(&(frame.len() as u32).to_be_bytes()).unwrap();
c.write_all(&frame).unwrap();
let mut head = [0u8; 4];
c.read_exact(&mut head).unwrap();
let mut body = vec![0u8; u32::from_le_bytes(head) as usize];
let mut body = vec![0u8; u32::from_be_bytes(head) as usize];
c.read_exact(&mut body).unwrap();
// The answer is a ZAP message: the magic is there, the flags say what kind
// of frame it is, and the fields are read where they lie.
assert_eq!(&body[0..4], b"ZAP\0", "the answer is not a ZAP message");
let flags = u16::from_le_bytes([body[6], body[7]]);
assert_eq!(
flags >> 8,
zip::zaphttp::FRAME_RESPONSE,
zip::zap::kind(&body),
Some(zip::zap::FRAME_RESPONSE),
"not a response frame"
);
let r = zip::wire::Response::wrap(&body).unwrap();
let r = zip::zap::Response::wrap(&body).unwrap();
(r.status(), String::from_utf8(r.body().to_vec()).unwrap())
}
@@ -157,3 +154,16 @@ fn an_unknown_address_is_refused() {
let (http, _) = doors();
assert_eq!(over_http(&http, "/nope").0, 404);
}
/// The documents beside the source were projected from THIS declaration: the
/// manifest the crate compiles in is the one zipc was given.
#[test]
fn the_documents_are_current() {
let written =
std::fs::read_to_string(concat!(env!("CARGO_MANIFEST_DIR"), "/gen/manifest.json")).unwrap();
assert_eq!(
written.trim_end(),
info::ops::Info::manifest(),
"the declaration changed since `make gen`"
);
}
rust/examples/stream/Cargo.toml
+18
View File
@@ -0,0 +1,18 @@
[package]
name = "stream"
version.workspace = true
edition.workspace = true
license.workspace = true
publish = false
[dependencies]
zip = { path = "../../zip" }
futures-util = { version = "0.3", default-features = false, features = ["std"] }
serde = { version = "1", features = ["derive"] }
tokio = { version = "1.53", features = ["rt-multi-thread", "macros", "time"] }
[dev-dependencies]
bytes = "1"
http-body-util = "0.1"
hyper = { version = "1.12", features = ["client", "http1", "http2"] }
hyper-util = { version = "0.1.21", features = ["tokio"] }
rust/examples/stream/Makefile
+32
View File
@@ -0,0 +1,32 @@
# The build, in the order the facts become available.
#
# The macro compiles what it read into the crate, and the binary prints it. So
# the manifest describes THIS build — there is no file beside the source to fall
# out of step, and no way to project a service that no longer exists.
#
# It is two steps and not a build script because the description exists only
# after a compile. What zipc writes is checked in beside the source, the way a
# generated file is anywhere else, so a fresh clone builds without either tool.
#
# make gen compile, project, and compile again against the projections
# make run serve on the default addresses
# make check the doors answer, and the wire is ZAP
# gen names a directory as well as a step, and the step is the one meant.
.PHONY: gen run check
# zipc is this checkout's, so the documents are projected by the projector
# the manifest was written for.
ZIPC ?= go run ../../../cmd/zipc
ROOT := ../..
gen:
cd $(ROOT) && cargo run -q -p stream -- --manifest > examples/stream/gen/manifest.json
$(ZIPC) -out gen -package stream gen/manifest.json
cd $(ROOT) && cargo build -p stream
run:
cd $(ROOT) && cargo run -p stream
check:
cd $(ROOT) && cargo test -p stream
rust/examples/stream/build.rs
+26
View File
@@ -0,0 +1,26 @@
// SPDX-License-Identifier: BSD-3-Clause-Eco
// Copyright (C) 2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//! The documents this service publishes, if they have been projected yet.
//!
//! They are checked in, so a clone builds with them in hand. But they are made
//! FROM this crate — zipc reads the manifest the compiled binary prints — so
//! the first build of a new service necessarily happens before they exist. This
//! says so instead of failing: no documents yet, and `make gen` makes them.
use std::path::PathBuf;
fn main() {
println!("cargo:rerun-if-changed=gen");
let out = PathBuf::from(std::env::var("OUT_DIR").expect("OUT_DIR"));
let mut body = String::new();
for (name, file) in [("OPENAPI", "gen/openapi.json"), ("MCP", "gen/mcp.json")] {
let text = std::fs::read_to_string(file).unwrap_or_default();
body.push_str(&format!(
"pub const {name}: &str = {:?};\n",
if text.is_empty() { "" } else { text.as_str() }
));
}
std::fs::write(out.join("documents.rs"), body).expect("write documents.rs");
}
rust/examples/stream/gen/cli.json
+66
View File
@@ -0,0 +1,66 @@
[
{
"Service": "count",
"Name": "create",
"OperationID": "post_count",
"Summary": "Count streams n ticks as server-sent events, then [DONE], and reports how many it sent in the x-sent trailer.",
"Description": "Count streams n ticks as server-sent events, then [DONE], and reports how\nmany it sent in the x-sent trailer.",
"Method": "POST",
"Path": "/v1/count",
"Args": [],
"Flags": [
{
"Name": "n",
"Field": "n",
"Type": "integer",
"Help": "N is how many ticks to send.",
"Required": true
},
{
"Name": "every-ms",
"Field": "everyMs",
"Type": "integer",
"Help": "EveryMs is the pause before each tick, in milliseconds.",
"Required": false
}
],
"Example": {
"n": 3,
"everyMs": 100
}
},
{
"Service": "echo",
"Name": "create",
"OperationID": "post_echo",
"Summary": "Echo answers the text it was sent.",
"Description": "Echo answers the text it was sent.",
"Method": "POST",
"Path": "/v1/echo",
"Args": [],
"Flags": [
{
"Name": "text",
"Field": "text",
"Type": "string",
"Help": "Text is the line.",
"Required": true
}
],
"Example": {
"text": "hi"
}
},
{
"Service": "relay",
"Name": "create",
"OperationID": "post_relay",
"Summary": "Relay answers the body it was sent, each chunk as it arrives, under the content type it was sent with.",
"Description": "Relay answers the body it was sent, each chunk as it arrives, under the\ncontent type it was sent with.",
"Method": "POST",
"Path": "/v1/relay",
"Args": [],
"Flags": null,
"Example": null
}
]
rust/examples/stream/gen/manifest.json
+1
View File
@@ -0,0 +1 @@
{"manifest":1,"app":"stream","title":"Streaming","version":"1.0.0","description":"One JSON op, one event stream and one byte relay.","ops":[{"method":"POST","path":"/v1/echo","description":"Echo answers the text it was sent.","pkg":"stream","in":"Text","out":"Text","example":{"text": "hi"},"response":{"text": "hi"}},{"method":"POST","path":"/v1/count","description":"Count streams n ticks as server-sent events, then [DONE], and reports how\nmany it sent in the x-sent trailer.","pkg":"stream","in":"Count","example":{"n": 3, "everyMs": 100},"stream":"sse"},{"method":"POST","path":"/v1/relay","description":"Relay answers the body it was sent, each chunk as it arrives, under the\ncontent type it was sent with.","pkg":"stream","stream":"bytes","raw":true}],"types":[{"id":"Count","name":"Count","kind":"struct","spell":"Count","doc":"Count asks for a stream of ticks.","fields":[{"name":"n","json":"n","required":true,"doc":"N is how many ticks to send.","type":{"prim":"u32"},"spell":"u32"},{"name":"every_ms","json":"everyMs","doc":"EveryMs is the pause before each tick, in milliseconds.","type":{"prim":"u64"},"spell":"u64"}]},{"id":"Text","name":"Text","kind":"struct","spell":"Text","doc":"Text is a line of text.","fields":[{"name":"text","json":"text","required":true,"doc":"Text is the line.","type":{"prim":"string"},"spell":"String"}]}]}
rust/examples/stream/gen/mcp.json
+51
View File
@@ -0,0 +1,51 @@
[
{
"annotations": {},
"description": "Count streams n ticks as server-sent events, then [DONE], and reports how\nmany it sent in the x-sent trailer.",
"inputSchema": {
"properties": {
"everyMs": {
"description": "EveryMs is the pause before each tick, in milliseconds.",
"format": "uint64",
"type": "integer"
},
"n": {
"description": "N is how many ticks to send.",
"format": "uint32",
"type": "integer"
}
},
"required": [
"n"
],
"type": "object"
},
"name": "post_count"
},
{
"annotations": {},
"description": "Echo answers the text it was sent.",
"inputSchema": {
"properties": {
"text": {
"description": "Text is the line.",
"type": "string"
}
},
"required": [
"text"
],
"type": "object"
},
"name": "post_echo"
},
{
"annotations": {},
"description": "Relay answers the body it was sent, each chunk as it arrives, under the\ncontent type it was sent with.",
"inputSchema": {
"properties": {},
"type": "object"
},
"name": "post_relay"
}
]
rust/examples/stream/gen/openapi.json
+205
View File
@@ -0,0 +1,205 @@
{
"components": {
"schemas": {
"Count": {
"properties": {
"everyMs": {
"description": "EveryMs is the pause before each tick, in milliseconds.",
"format": "uint64",
"type": "integer"
},
"n": {
"description": "N is how many ticks to send.",
"format": "uint32",
"type": "integer"
}
},
"required": [
"n"
],
"type": "object"
},
"Text": {
"properties": {
"text": {
"description": "Text is the line.",
"type": "string"
}
},
"required": [
"text"
],
"type": "object"
},
"problem-details": {
"description": "An RFC 9457 problem document.",
"properties": {
"code": {
"description": "A stable token a client can branch on, when the refusal names one.",
"type": "string"
},
"detail": {
"description": "What was refused, for a person to read.",
"type": "string"
},
"status": {
"description": "The HTTP status.",
"format": "int64",
"type": "integer"
},
"title": {
"description": "The status's reason phrase.",
"type": "string"
},
"type": {
"description": "about:blank",
"type": "string"
}
},
"required": [
"type",
"status",
"detail"
],
"type": "object"
}
}
},
"info": {
"description": "One JSON op, one event stream and one byte relay.",
"title": "Streaming",
"version": "1.0.0"
},
"openapi": "3.1.0",
"paths": {
"/v1/count": {
"post": {
"description": "Count streams n ticks as server-sent events, then [DONE], and reports how\nmany it sent in the x-sent trailer.",
"operationId": "post_count",
"requestBody": {
"content": {
"application/json": {
"example": {
"n": 3,
"everyMs": 100
},
"schema": {
"$ref": "#/components/schemas/Count"
}
}
},
"required": true
},
"responses": {
"200": {
"content": {
"text/event-stream": {
"schema": {
"type": "string"
}
}
},
"description": "ok"
},
"default": {
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/problem-details"
}
}
},
"description": "refused"
}
},
"summary": "Count streams n ticks as server-sent events, then [DONE], and reports how many it sent in the x-sent trailer."
}
},
"/v1/echo": {
"post": {
"description": "Echo answers the text it was sent.",
"operationId": "post_echo",
"requestBody": {
"content": {
"application/json": {
"example": {
"text": "hi"
},
"schema": {
"$ref": "#/components/schemas/Text"
}
}
},
"required": true
},
"responses": {
"200": {
"content": {
"application/json": {
"example": {
"text": "hi"
},
"schema": {
"$ref": "#/components/schemas/Text"
}
}
},
"description": "ok"
},
"default": {
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/problem-details"
}
}
},
"description": "refused"
}
},
"summary": "Echo answers the text it was sent."
}
},
"/v1/relay": {
"post": {
"description": "Relay answers the body it was sent, each chunk as it arrives, under the\ncontent type it was sent with.",
"operationId": "post_relay",
"requestBody": {
"content": {
"application/octet-stream": {
"schema": {
"format": "binary",
"type": "string"
}
}
},
"required": true
},
"responses": {
"200": {
"content": {
"application/octet-stream": {
"schema": {
"format": "binary",
"type": "string"
}
}
},
"description": "ok"
},
"default": {
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/problem-details"
}
}
},
"description": "refused"
}
},
"summary": "Relay answers the body it was sent, each chunk as it arrives, under the content type it was sent with."
}
}
}
}
rust/examples/stream/gen/stream.zap
+25
View File
@@ -0,0 +1,25 @@
# Generated from typed ops. Do not edit.
# Every struct and method below is derived from one op's In and Out, and
# every offset from the layout the op-call plane encodes against.
package stream
struct Count {
n u32 @0
every_ms u64 @8
}
struct Text {
text_ text @0
}
interface stream {
# Count streams n ticks as server-sent events, then [DONE], and reports how
# many it sent in the x-sent trailer.
post_count(req: Count)
# Echo answers the text it was sent.
post_echo(req: Text) returns (rep: Text)
# Relay answers the body it was sent, each chunk as it arrives, under the
# content type it was sent with.
post_relay()
}
rust/examples/stream/src/lib.rs
+136
View File
@@ -0,0 +1,136 @@
// SPDX-License-Identifier: BSD-3-Clause-Eco
// Copyright (C) 2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//! A streaming service: one JSON op, one event stream, one byte relay — the
//! three shapes a chat-completions router is made of.
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::Arc;
use std::time::Duration;
use futures_util::stream;
use serde::{Deserialize, Serialize};
use zip::{Body, Cx, Error, Event, Sse, Wire};
// The projections, as zipc wrote them from what this crate declared.
include!(concat!(env!("OUT_DIR"), "/documents.rs"));
/// Text is a line of text.
#[derive(Wire, Serialize, Deserialize, Default)]
#[serde(default)]
pub struct Text {
/// Text is the line.
#[zip(required)]
pub text: String,
}
/// Count asks for a stream of ticks.
#[derive(Wire, Serialize, Deserialize, Default)]
#[serde(default, rename_all = "camelCase")]
pub struct Count {
/// N is how many ticks to send.
#[zip(required)]
pub n: u32,
/// EveryMs is the pause before each tick, in milliseconds.
pub every_ms: u64,
}
/// Tick is one event of a count.
#[derive(Serialize, Deserialize)]
pub struct Tick {
/// I is the tick's number, from 1.
pub i: u32,
}
/// Chat is the service. `open` is how many streams are being produced right
/// now: a gauge a deployment exports, and what shows a stream was dropped.
#[derive(Default)]
pub struct Chat {
pub open: Arc<AtomicUsize>,
}
#[zip::ops(
app = "stream",
title = "Streaming",
version = "1.0.0",
description = "One JSON op, one event stream and one byte relay."
)]
impl Chat {
/// Echo answers the text it was sent.
///
/// Example: {"text": "hi"}
/// Response: {"text": "hi"}
#[post("/v1/echo")]
async fn echo(&self, arg: &Text) -> Result<Text, Error> {
Ok(Text {
text: arg.text.clone(),
})
}
/// Count streams n ticks as server-sent events, then [DONE], and reports how
/// many it sent in the x-sent trailer.
///
/// Example: {"n": 3, "everyMs": 100}
#[post("/v1/count")]
async fn count(&self, cx: &Cx, arg: &Count) -> Result<Sse, Error> {
let sent = cx.trailer("x-sent")?;
let open = Open::new(&self.open);
let (n, every) = (arg.n, Duration::from_millis(arg.every_ms));
// The producer's state is the stream's: dropping the stream — which is
// what the door does when the client leaves — drops `open` with it.
let ticks = stream::unfold((1, open), move |(i, open)| {
let sent = sent.clone();
async move {
if i > n + 1 {
return None;
}
if i == n + 1 {
sent.set(&n.to_string()).ok()?;
return Some((Ok::<_, Error>(Event::data("[DONE]")), (i + 1, open)));
}
tokio::time::sleep(every).await;
Some((Event::json(&Tick { i }), (i + 1, open)))
}
});
Ok(Sse::new(ticks).keep(Duration::from_secs(15)))
}
/// Relay answers the body it was sent, each chunk as it arrives, under the
/// content type it was sent with.
#[post("/v1/relay")]
async fn relay(&self, cx: &Cx, body: Body) -> Result<Body, Error> {
if let Some(kind) = cx.header("content-type") {
cx.set_header("content-type", kind)?;
}
Ok(body)
}
}
/// Open counts one stream for as long as it lives.
struct Open(Arc<AtomicUsize>);
impl Open {
fn new(gauge: &Arc<AtomicUsize>) -> Open {
gauge.fetch_add(1, Ordering::SeqCst);
Open(Arc::clone(gauge))
}
}
impl Drop for Open {
fn drop(&mut self) {
self.0.fetch_sub(1, Ordering::SeqCst);
}
}
/// service is the app, with its documents on it.
pub fn service(chat: Chat) -> zip::App {
let mut app = chat.ops();
if !OPENAPI.is_empty() {
app.serve("/openapi.json", "application/json", OPENAPI);
}
if !MCP.is_empty() {
app.serve("/mcp.json", "application/json", MCP);
}
app
}
rust/examples/stream/src/main.rs
+46
View File
@@ -0,0 +1,46 @@
// SPDX-License-Identifier: BSD-3-Clause-Eco
// Copyright (C) 2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//! The streaming service, serving REST and ZAP from one registry.
//!
//! ```text
//! stream --http 127.0.0.1:8000 --zap 127.0.0.1:9653
//! ```
#[tokio::main]
async fn main() {
let mut http = Some("127.0.0.1:8000".to_string());
let mut zap = None;
let args: Vec<String> = std::env::args().skip(1).collect();
let mut i = 0;
while i < args.len() {
match args[i].as_str() {
"--http" => {
http = args.get(i + 1).cloned();
i += 2;
}
"--zap" => {
zap = args.get(i + 1).cloned();
i += 2;
}
"--manifest" => {
println!("{}", stream::Chat::manifest());
return;
}
other => {
eprintln!("stream: unknown argument {other}");
std::process::exit(2);
}
}
}
let app = stream::service(stream::Chat::default());
for (method, path) in app.routes() {
println!(" {method} {path}");
}
let grace = std::time::Duration::from_secs(25);
if let Err(e) = zip::listen(app, http.as_deref(), zap.as_deref(), zip::signal(), grace).await {
eprintln!("stream: {e}");
std::process::exit(1);
}
}
rust/examples/stream/tests/serve.rs
+576
View File
@@ -0,0 +1,576 @@
// SPDX-License-Identifier: BSD-3-Clause-Eco
// Copyright (C) 2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//! The service on a real port, asked over real sockets.
//!
//! HTTP/1.1 is spoken by hand here, on a bare TCP stream, so the test sees
//! what the wire carries — the chunks, when each arrived, the trailers after
//! the last — and can hang up mid-answer exactly when it means to.
use std::sync::atomic::Ordering;
use std::sync::Arc;
use std::time::{Duration, Instant};
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::{TcpListener, TcpStream};
use tokio::sync::oneshot;
use tokio::task::JoinHandle;
struct Server {
addr: String,
open: Arc<std::sync::atomic::AtomicUsize>,
stop: Option<oneshot::Sender<()>>,
done: JoinHandle<std::io::Result<()>>,
}
#[derive(Clone, Copy)]
enum Door {
Http,
Zap,
}
async fn start(door: Door) -> Server {
start_with(door, Duration::from_secs(5)).await
}
async fn start_with(door: Door, grace: Duration) -> Server {
let chat = stream::Chat::default();
let open = Arc::clone(&chat.open);
let app = Arc::new(stream::service(chat));
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap().to_string();
let (stop, stopped) = oneshot::channel::<()>();
let shutdown = async move {
let _ = stopped.await;
};
let done = match door {
Door::Http => tokio::spawn(zip::http::serve(app, listener, shutdown, grace)),
Door::Zap => tokio::spawn(zip::zaphttp::serve(app, listener, shutdown, grace)),
};
Server {
addr,
open,
stop: Some(stop),
done,
}
}
/// Wire is one HTTP/1.1 connection, read by hand.
struct Wire {
conn: TcpStream,
buf: Vec<u8>,
}
impl Wire {
async fn dial(addr: &str) -> Wire {
Wire {
conn: TcpStream::connect(addr).await.unwrap(),
buf: Vec::new(),
}
}
async fn send(&mut self, text: &str) {
self.conn.write_all(text.as_bytes()).await.unwrap();
}
async fn post(&mut self, path: &str, body: &str, extra: &str) {
let req = format!(
"POST {path} HTTP/1.1\r\nHost: x\r\nContent-Type: application/json\r\nContent-Length: {}\r\n{extra}\r\n{body}",
body.len()
);
self.send(&req).await;
}
async fn fill(&mut self) {
let mut chunk = [0u8; 4096];
let n = tokio::time::timeout(Duration::from_secs(10), self.conn.read(&mut chunk))
.await
.expect("the server said nothing for 10 s")
.unwrap();
assert!(n > 0, "the server closed the connection");
self.buf.extend_from_slice(&chunk[..n]);
}
async fn line(&mut self) -> String {
loop {
if let Some(i) = self.buf.windows(2).position(|w| w == b"\r\n") {
let line = String::from_utf8(self.buf[..i].to_vec()).unwrap();
self.buf.drain(..i + 2);
return line;
}
self.fill().await;
}
}
async fn exact(&mut self, n: usize) -> Vec<u8> {
while self.buf.len() < n {
self.fill().await;
}
self.buf.drain(..n).collect()
}
/// head reads a status line and headers, names lowercased.
async fn head(&mut self) -> (u16, Vec<(String, String)>) {
let status = self.line().await;
let code = status.split_whitespace().nth(1).unwrap().parse().unwrap();
let mut headers = Vec::new();
loop {
let l = self.line().await;
if l.is_empty() {
return (code, headers);
}
let (k, v) = l.split_once(':').unwrap();
headers.push((k.trim().to_ascii_lowercase(), v.trim().to_string()));
}
}
/// chunk reads one chunk of a chunked body: Some(bytes), or None at the
/// last-chunk, after which the trailer lines are read.
async fn chunk(&mut self) -> Option<Vec<u8>> {
let size = usize::from_str_radix(self.line().await.trim(), 16).unwrap();
if size == 0 {
return None;
}
let data = self.exact(size).await;
assert_eq!(self.line().await, "", "a chunk ends in CRLF");
Some(data)
}
async fn trailers(&mut self) -> Vec<String> {
let mut out = Vec::new();
loop {
let l = self.line().await;
if l.is_empty() {
return out;
}
out.push(l.to_ascii_lowercase());
}
}
}
fn header<'a>(h: &'a [(String, String)], name: &str) -> Option<&'a str> {
h.iter().find(|(k, _)| k == name).map(|(_, v)| v.as_str())
}
/// (a) A JSON op answers, and the connection stays open for the next request.
#[tokio::test(flavor = "multi_thread")]
async fn a_json_op_round_trips_on_one_connection() {
let s = start(Door::Http).await;
let mut w = Wire::dial(&s.addr).await;
for _ in 0..3 {
w.post("/v1/echo", r#"{"text":"hi"}"#, "").await;
let (status, h) = w.head().await;
assert_eq!(status, 200);
assert_eq!(header(&h, "content-type"), Some("application/json"));
let n: usize = header(&h, "content-length").unwrap().parse().unwrap();
assert_eq!(w.exact(n).await, br#"{"text":"hi"}"#);
}
// A field the op cannot run without is refused before the handler runs.
w.post("/v1/echo", "{}", "").await;
let (status, h) = w.head().await;
assert_eq!(status, 400);
assert_eq!(header(&h, "content-type"), Some("application/problem+json"));
let n: usize = header(&h, "content-length").unwrap().parse().unwrap();
let body = String::from_utf8(w.exact(n).await).unwrap();
assert!(body.contains(r#"field \"text\" is required"#), "{body}");
}
/// events reads an event stream until its last chunk, stamping each event
/// with when its blank line arrived.
async fn events(w: &mut Wire, since: Instant) -> Vec<(String, Duration)> {
let mut out = Vec::new();
let mut pending = Vec::new();
while let Some(chunk) = w.chunk().await {
pending.extend_from_slice(&chunk);
while let Some(i) = pending.windows(2).position(|x| x == b"\n\n") {
let event = String::from_utf8(pending[..i].to_vec()).unwrap();
pending.drain(..i + 2);
out.push((event, since.elapsed()));
}
}
assert!(pending.is_empty(), "the stream ended mid-event");
out
}
/// (b) Each event is written when it is produced. Five ticks 200 ms apart
/// arrive 200 ms apart; a server that buffered them would deliver five at once.
#[tokio::test(flavor = "multi_thread")]
async fn an_event_arrives_when_it_is_produced() {
let s = start(Door::Http).await;
let mut w = Wire::dial(&s.addr).await;
let every = Duration::from_millis(200);
let since = Instant::now();
w.post("/v1/count", r#"{"n":5,"everyMs":200}"#, "").await;
let (status, h) = w.head().await;
assert_eq!(status, 200);
assert_eq!(header(&h, "content-type"), Some("text/event-stream"));
assert_eq!(header(&h, "cache-control"), Some("no-cache"));
assert_eq!(header(&h, "transfer-encoding"), Some("chunked"));
let got = events(&mut w, since).await;
// No TE: trailers, so no trailers: hyper sends them on no other terms.
assert!(w.trailers().await.is_empty());
let texts: Vec<&str> = got.iter().map(|(e, _)| e.as_str()).collect();
assert_eq!(
texts,
vec![
r#"data: {"i":1}"#,
r#"data: {"i":2}"#,
r#"data: {"i":3}"#,
r#"data: {"i":4}"#,
r#"data: {"i":5}"#,
"data: [DONE]"
]
);
let at: Vec<u128> = got.iter().map(|(_, t)| t.as_millis()).collect();
eprintln!("arrivals (ms): {at:?}");
for pair in got[..5].windows(2) {
let gap = pair[1].1 - pair[0].1;
assert!(
gap >= every / 2,
"two ticks arrived {gap:?} apart; they were produced {every:?} apart: {at:?}"
);
}
assert!(
got[4].1 - got[0].1 >= every * 4 / 2,
"the first tick waited for the last: {at:?}"
);
// The stream ended cleanly — its last-chunk arrived — so the connection
// is still good for another request.
w.post("/v1/echo", r#"{"text":"after"}"#, "").await;
assert_eq!(w.head().await.0, 200);
}
/// (c) A client that hangs up mid-stream drops the producer: the stream's
/// state, and whatever it holds, goes with it.
#[tokio::test(flavor = "multi_thread")]
async fn a_client_leaving_drops_the_producer() {
let s = start(Door::Http).await;
let mut w = Wire::dial(&s.addr).await;
w.post("/v1/count", r#"{"n":100000,"everyMs":10}"#, "")
.await;
assert_eq!(w.head().await.0, 200);
let first = w.chunk().await.unwrap();
assert_eq!(first, b"data: {\"i\":1}\n\n");
assert_eq!(
s.open.load(Ordering::SeqCst),
1,
"one stream is being produced"
);
let left = Instant::now();
drop(w);
while s.open.load(Ordering::SeqCst) != 0 {
assert!(
left.elapsed() < Duration::from_secs(5),
"the producer outlived its client by 5 s"
);
tokio::time::sleep(Duration::from_millis(1)).await;
}
eprintln!(
"producer dropped {:?} after the client left",
left.elapsed()
);
}
/// The same, while the producer is idle: no event is due for a minute, so
/// nothing is being written, and the hang-up alone has to be noticed.
#[tokio::test(flavor = "multi_thread")]
async fn a_client_leaving_an_idle_stream_drops_the_producer() {
let s = start(Door::Http).await;
let mut w = Wire::dial(&s.addr).await;
w.post("/v1/count", r#"{"n":3,"everyMs":60000}"#, "").await;
assert_eq!(w.head().await.0, 200);
while s.open.load(Ordering::SeqCst) != 1 {
tokio::time::sleep(Duration::from_millis(1)).await;
}
let left = Instant::now();
drop(w);
while s.open.load(Ordering::SeqCst) != 0 {
assert!(
left.elapsed() < Duration::from_secs(5),
"an idle producer outlived its client by 5 s"
);
tokio::time::sleep(Duration::from_millis(1)).await;
}
eprintln!(
"idle producer dropped {:?} after the client left",
left.elapsed()
);
}
/// (d) The service publishes the document its declaration projects, and the
/// document says what each op answers with.
#[tokio::test(flavor = "multi_thread")]
async fn the_document_lists_the_ops() {
let s = start(Door::Http).await;
let mut w = Wire::dial(&s.addr).await;
w.send("GET /openapi.json HTTP/1.1\r\nHost: x\r\n\r\n")
.await;
let (status, h) = w.head().await;
assert_eq!(status, 200);
let n: usize = header(&h, "content-length").unwrap().parse().unwrap();
let doc = String::from_utf8(w.exact(n).await).unwrap();
for want in [
r#""operationId": "post_echo""#,
r#""operationId": "post_count""#,
r#""operationId": "post_relay""#,
r#""text/event-stream""#,
r#""application/octet-stream""#,
] {
assert!(doc.contains(want), "the document lacks {want}");
}
}
/// The documents beside the source were projected from THIS declaration.
#[test]
fn the_documents_are_current() {
let written =
std::fs::read_to_string(concat!(env!("CARGO_MANIFEST_DIR"), "/gen/manifest.json")).unwrap();
assert_eq!(
written.trim_end(),
stream::Chat::manifest(),
"the declaration changed since `make gen`"
);
}
/// A trailer reaches an HTTP/1.1 client that asked for trailers, after the
/// last chunk, and its name went out in the head.
#[tokio::test(flavor = "multi_thread")]
async fn a_trailer_follows_the_stream() {
let s = start(Door::Http).await;
let mut w = Wire::dial(&s.addr).await;
w.post("/v1/count", r#"{"n":2,"everyMs":1}"#, "TE: trailers\r\n")
.await;
let (status, h) = w.head().await;
assert_eq!(status, 200);
assert_eq!(header(&h, "trailer"), Some("x-sent"));
let got = events(&mut w, Instant::now()).await;
assert_eq!(got.len(), 3);
assert_eq!(w.trailers().await, vec!["x-sent: 2"]);
}
/// A raw op relays bytes in both directions at once: each chunk of the
/// request comes back before the next one is sent.
#[tokio::test(flavor = "multi_thread")]
async fn a_relay_answers_each_chunk_as_it_arrives() {
let s = start(Door::Http).await;
let mut w = Wire::dial(&s.addr).await;
w.send("POST /v1/relay HTTP/1.1\r\nHost: x\r\nContent-Type: text/plain\r\nTransfer-Encoding: chunked\r\n\r\n")
.await;
w.send("5\r\nhello\r\n").await;
let (status, h) = w.head().await;
assert_eq!(status, 200);
assert_eq!(header(&h, "content-type"), Some("text/plain"));
assert_eq!(w.chunk().await.unwrap(), b"hello");
w.send("6\r\n world\r\n").await;
assert_eq!(w.chunk().await.unwrap(), b" world");
w.send("0\r\n\r\n").await;
assert_eq!(w.chunk().await, None);
assert!(w.trailers().await.is_empty());
}
/// HTTP/2 over cleartext on the same port: the stream arrives frame by frame
/// and its trailer after the last.
#[tokio::test(flavor = "multi_thread")]
async fn http2_streams_and_carries_the_trailer() {
use http_body_util::BodyExt;
let s = start(Door::Http).await;
let tcp = TcpStream::connect(&s.addr).await.unwrap();
let (mut send, conn) = hyper::client::conn::http2::handshake(
hyper_util::rt::TokioExecutor::new(),
hyper_util::rt::TokioIo::new(tcp),
)
.await
.unwrap();
tokio::spawn(conn);
let req = hyper::Request::post(format!("http://{}/v1/count", s.addr))
.header("content-type", "application/json")
.body(http_body_util::Full::new(bytes::Bytes::from_static(
br#"{"n":3,"everyMs":150}"#,
)))
.unwrap();
let res = send.send_request(req).await.unwrap();
assert_eq!(res.status(), 200);
assert_eq!(res.version(), hyper::Version::HTTP_2);
let mut body = res.into_body();
let since = Instant::now();
let mut data = Vec::new();
let mut stamps = Vec::new();
let mut trailers = None;
while let Some(frame) = body.frame().await {
let frame = frame.unwrap();
if let Some(d) = frame.data_ref() {
data.extend_from_slice(d);
stamps.push(since.elapsed());
} else if let Ok(t) = frame.into_trailers() {
trailers = Some(t);
}
}
assert_eq!(
String::from_utf8(data).unwrap(),
"data: {\"i\":1}\n\ndata: {\"i\":2}\n\ndata: {\"i\":3}\n\ndata: [DONE]\n\n"
);
assert!(
stamps[1] - stamps[0] >= Duration::from_millis(75),
"frames {stamps:?}"
);
let trailers = trailers.expect("no trailers");
assert_eq!(trailers.get("x-sent").unwrap(), "3");
}
/// Shutdown stops new connections and lets the stream in flight finish.
#[tokio::test(flavor = "multi_thread")]
async fn shutdown_drains_what_is_in_flight() {
let mut s = start(Door::Http).await;
let mut w = Wire::dial(&s.addr).await;
w.post("/v1/count", r#"{"n":4,"everyMs":150}"#, "").await;
assert_eq!(w.head().await.0, 200);
assert!(w.chunk().await.is_some());
s.stop.take().unwrap().send(()).unwrap();
let mut refused = false;
for _ in 0..100 {
if TcpStream::connect(&s.addr).await.is_err() {
refused = true;
break;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
assert!(refused, "a new connection was accepted after shutdown");
let rest = events(&mut w, Instant::now()).await;
assert_eq!(rest.last().unwrap().0, "data: [DONE]", "the stream was cut");
tokio::time::timeout(Duration::from_secs(5), s.done)
.await
.expect("serve did not return once drained")
.unwrap()
.unwrap();
}
/// The ZAP door streams the same op: a head frame, a data frame as each event
/// is produced, an end frame holding the trailer.
#[tokio::test(flavor = "multi_thread")]
async fn the_zap_door_streams_frame_by_frame() {
let s = start(Door::Zap).await;
let mut conn = TcpStream::connect(&s.addr).await.unwrap();
let headers = zip::zap::block([(&b"Content-Type"[..], &b"application/json"[..])]);
let frame = zip::zap::request("POST", "/v1/count", &headers, br#"{"n":3,"everyMs":150}"#);
conn.write_all(&(frame.len() as u32).to_be_bytes())
.await
.unwrap();
conn.write_all(&frame).await.unwrap();
let since = Instant::now();
let head = zip::zaphttp::read_frame(&mut conn).await.unwrap().unwrap();
assert_eq!(zip::zap::kind(&head), Some(zip::zap::FRAME_HEAD));
let r = zip::zap::Response::wrap(&head).unwrap();
assert_eq!(r.status(), 200);
let names: Vec<_> = zip::zap::pairs(r.headers())
.into_iter()
.map(|(k, v)| {
(
String::from_utf8_lossy(k).into_owned(),
String::from_utf8_lossy(v).into_owned(),
)
})
.collect();
assert!(
names.contains(&("content-type".into(), "text/event-stream".into())),
"{names:?}"
);
assert!(
!names.iter().any(|(k, _)| k == "trailer"),
"Trailer is the frame's slot, not a header"
);
let mut chunks = Vec::new();
loop {
let f = zip::zaphttp::read_frame(&mut conn).await.unwrap().unwrap();
match zip::zap::kind(&f) {
Some(zip::zap::FRAME_DATA) => {
let c = zip::zap::chunk(&f).unwrap().to_vec();
chunks.push((String::from_utf8(c).unwrap(), since.elapsed()));
}
Some(zip::zap::FRAME_END) => {
let t = zip::zap::pairs(zip::zap::chunk(&f).unwrap());
assert_eq!(t, vec![(&b"x-sent"[..], &b"3"[..])]);
break;
}
other => panic!("frame type {other:?} inside a stream"),
}
}
assert_eq!(chunks.len(), 4, "{chunks:?}");
assert_eq!(chunks[0].0, "data: {\"i\":1}\n\n");
assert_eq!(chunks[3].0, "data: [DONE]\n\n");
assert!(
chunks[1].1 - chunks[0].1 >= Duration::from_millis(75),
"{chunks:?}"
);
// One exchange done, the connection answers the next.
let frame = zip::zap::request("POST", "/v1/echo", &headers, br#"{"text":"zap"}"#);
conn.write_all(&(frame.len() as u32).to_be_bytes())
.await
.unwrap();
conn.write_all(&frame).await.unwrap();
let f = zip::zaphttp::read_frame(&mut conn).await.unwrap().unwrap();
let r = zip::zap::Response::wrap(&f).unwrap();
assert_eq!((r.status(), r.body()), (200, &br#"{"text":"zap"}"#[..]));
}
/// A stream still going when the grace runs out is cut, and its producer
/// dropped: shutdown is bounded by the grace, not by the longest answer.
#[tokio::test(flavor = "multi_thread")]
async fn shutdown_ends_what_outlives_the_grace() {
for door in [Door::Http, Door::Zap] {
let mut s = start_with(door, Duration::from_millis(300)).await;
let mut conn = TcpStream::connect(&s.addr).await.unwrap();
let body = br#"{"n":100000,"everyMs":10}"#;
match door {
Door::Http => {
let head = format!(
"POST /v1/count HTTP/1.1\r\nHost: x\r\nContent-Length: {}\r\n\r\n",
body.len()
);
conn.write_all(head.as_bytes()).await.unwrap();
conn.write_all(body).await.unwrap();
}
Door::Zap => {
let frame = zip::zap::request("POST", "/v1/count", &[], body);
conn.write_all(&(frame.len() as u32).to_be_bytes())
.await
.unwrap();
conn.write_all(&frame).await.unwrap();
}
}
while s.open.load(Ordering::SeqCst) != 1 {
tokio::time::sleep(Duration::from_millis(1)).await;
}
let stopped = Instant::now();
s.stop.take().unwrap().send(()).unwrap();
tokio::time::timeout(Duration::from_secs(5), s.done)
.await
.expect("serve outlived its grace")
.unwrap()
.unwrap();
let took = stopped.elapsed();
assert!(
took >= Duration::from_millis(300),
"serve returned in {took:?}, inside its grace"
);
while s.open.load(Ordering::SeqCst) != 0 {
assert!(
stopped.elapsed() < Duration::from_secs(5),
"the producer outlived the grace"
);
tokio::time::sleep(Duration::from_millis(1)).await;
}
// The client reads to the end of what it was sent and finds the
// connection closed, the answer cut.
let mut rest = Vec::new();
let _ = tokio::time::timeout(Duration::from_secs(5), conn.read_to_end(&mut rest)).await;
}
}
rust/zip-macros/src/attr.rs
+16 -10
View File
@@ -4,16 +4,19 @@
//! `#[zip(...)]` — what a declaration says about itself beyond its type.
//!
//! One attribute, one vocabulary, on a type and on a field alike. It is the
//! counterpart of a Go struct tag and holds the same four facts: the name the
//! body carries a field under, the name a URL does, the header it reads, and
//! whether the handler refuses to run without it.
//! One attribute, one vocabulary, on a type and on a field alike. With serde's
//! own attributes it is the counterpart of a Go struct tag: serde says the name
//! the body carries a field under (`#[serde(rename)]`) and whether it carries
//! it at all (`#[serde(skip)]`); this says the rest — the name a URL carries it
//! under, the header it reads, and whether the handler refuses to run without
//! it.
use syn::{Attribute, Expr, Lit, Meta, Token};
#[derive(Default)]
pub struct Attrs {
/// json is the name the body carries this under.
/// json is a field's wire name given here, which is refused: the name is
/// serde's, and two places to write it are two names.
pub json: Option<String>,
/// url is the name a URL carries it under; "-" opts out.
pub url: Option<String>,
@@ -21,8 +24,6 @@ pub struct Attrs {
pub header: Option<String>,
/// required says the handler refuses to run without it.
pub required: bool,
/// skip keeps it off the wire entirely.
pub skip: bool,
/// text says this type is carried as one word, so a URL can hold it.
pub text: bool,
/// json_schema is the shape a type states for itself, raw.
@@ -42,7 +43,12 @@ impl Attrs {
for item in items {
match &item {
Meta::Path(p) if p.is_ident("required") => out.required = true,
Meta::Path(p) if p.is_ident("skip") => out.skip = true,
Meta::Path(p) if p.is_ident("skip") => {
return Err(syn::Error::new_spanned(
p,
"zip: whether a field rides the wire is serde's — #[serde(skip)]",
))
}
Meta::Path(p) if p.is_ident("text") => out.text = true,
Meta::NameValue(nv) => {
let v = literal(&nv.value)?;
@@ -63,14 +69,14 @@ impl Attrs {
} else {
return Err(syn::Error::new_spanned(
&nv.path,
"zip: known keys are json, url, header, required, skip, text",
"zip: known keys are json, url, header, required, text",
));
}
}
other => {
return Err(syn::Error::new_spanned(
other,
"zip: known keys are json, url, header, required, skip, text",
"zip: known keys are json, url, header, required, text",
))
}
}
rust/zip-macros/src/lib.rs
+1
View File
@@ -14,6 +14,7 @@ mod attr;
mod doc;
mod frag;
mod ops;
mod serde;
mod ty;
mod wire;
rust/zip-macros/src/ops.rs
+156 -46
View File
@@ -53,7 +53,7 @@ pub fn expand(args: TokenStream, mut block: ItemImpl) -> Result<TokenStream, syn
let stated = fragment(&about, &ops);
let reaches: Vec<TokenStream> = ops
.iter()
.flat_map(|o| [o.input.clone(), o.output.clone()])
.flat_map(|o| [o.input.described(), o.output.described()])
.flatten()
.map(|t| quote!(<#t as ::zip::Wire>::reach(&mut types);))
.collect();
@@ -161,80 +161,165 @@ struct Op {
verb: String,
path: String,
call: syn::Ident,
input: Option<Type>,
output: Option<Type>,
/// args is the handler's parameters after `&self`, in the order declared.
args: Vec<Arg>,
input: In,
output: Answer,
doc: doc::Doc,
}
/// Arg is one parameter a handler takes.
enum Arg {
/// Cx is `&zip::Cx`.
Cx,
/// Input is the op's input, by reference or by value.
Input { by_ref: bool },
}
/// In is what an op reads its input from.
enum In {
None,
/// Value is a described type, bound from the body, the URL and headers.
Value(Box<Type>),
/// Body is the request body as it arrives, bound to nothing.
Body,
}
/// Answer is what an op answers with.
enum Answer {
None,
/// Value is one described type, as JSON.
Value(Box<Type>),
Sse,
Body,
}
impl In {
fn described(&self) -> Option<Type> {
match self {
In::Value(t) => Some((**t).clone()),
_ => None,
}
}
}
impl Answer {
fn described(&self) -> Option<Type> {
match self {
Answer::Value(t) => Some((**t).clone()),
_ => None,
}
}
}
impl Op {
fn read(verb: String, path: String, f: &syn::ImplItemFn) -> Result<Self, syn::Error> {
let mut input = None;
if f.sig.asyncness.is_none() {
return Err(syn::Error::new_spanned(
f.sig.fn_token,
"zip::ops: an op is an async fn; it runs on the server's tasks",
));
}
let mut args = Vec::new();
let mut input = In::None;
for arg in f.sig.inputs.iter() {
match arg {
FnArg::Receiver(_) => {}
FnArg::Typed(t) => {
if input.is_some() {
let FnArg::Typed(t) = arg else { continue };
let by_ref = matches!(&*t.ty, Type::Reference(_));
let ty = strip(&t.ty);
match last(&ty).as_deref() {
Some("Cx") => {
if !by_ref {
return Err(syn::Error::new_spanned(
arg,
"zip::ops: an op takes one input; a request is one value",
"zip::ops: the exchange is borrowed: `cx: &zip::Cx`",
));
}
input = Some(strip(&t.ty));
args.push(Arg::Cx);
continue;
}
Some("Body") => {
if by_ref {
return Err(syn::Error::new_spanned(
arg,
"zip::ops: a body is read as it arrives, so it is taken by value: `body: zip::Body`",
));
}
if !matches!(input, In::None) {
return Err(one_input(arg));
}
input = In::Body;
}
_ => {
if !matches!(input, In::None) {
return Err(one_input(arg));
}
input = In::Value(Box::new(ty));
}
}
args.push(Arg::Input { by_ref });
}
let output = match &f.sig.output {
ReturnType::Default => None,
ReturnType::Type(_, t) => answer(t),
ReturnType::Type(_, t) if last(t).as_deref() == Some("Result") => match answer(t) {
None => Answer::None,
Some(t) => match last(&t).as_deref() {
Some("Sse") => Answer::Sse,
Some("Body") => Answer::Body,
_ => Answer::Value(Box::new(t)),
},
},
other => {
return Err(syn::Error::new_spanned(
other,
"zip::ops: an op answers Result<T, zip::Error>; a refusal is part of every op's contract",
))
}
};
Ok(Op {
verb,
path,
call: f.sig.ident.clone(),
args,
input,
output,
doc: doc::read(&f.attrs),
})
}
/// register is the line that puts this op on the app: decode the input the
/// way the document says it arrives, run the handler, write the answer.
/// register is the line that puts this op on the app: read the input the
/// way the document says it arrives, run the handler, answer.
fn register(&self) -> TokenStream {
let verb = self.verb.to_uppercase();
let path = &self.path;
let call = &self.call;
let run = match (&self.input, &self.output) {
(Some(i), Some(_)) => quote! {
let arg = ::zip::bind::<#i>(input)?;
let answer = holder.#call(&arg)?;
let mut out = ::std::string::String::new();
::zip::Wire::write_json(&answer, &mut out);
Ok(::std::option::Option::Some(out))
},
(Some(i), None) => quote! {
let arg = ::zip::bind::<#i>(input)?;
holder.#call(&arg)?;
Ok(::std::option::Option::None)
},
(None, Some(_)) => quote! {
let answer = holder.#call()?;
let mut out = ::std::string::String::new();
::zip::Wire::write_json(&answer, &mut out);
Ok(::std::option::Option::Some(out))
},
(None, None) => quote! {
holder.#call()?;
Ok(::std::option::Option::None)
},
let take = match &self.input {
In::None => quote! { ::std::mem::drop(body); },
In::Body => quote! { let arg = body; },
In::Value(i) => quote! { let arg = ::zip::bind::<#i>(&cx, body).await?; },
};
let describe = match &self.input {
Some(i) => quote!(::std::option::Option::Some(<#i as ::zip::Wire>::describe())),
None => quote!(::std::option::Option::None),
let pass = self.args.iter().map(|a| match a {
Arg::Cx => quote!(&cx),
Arg::Input { by_ref: true } => quote!(&arg),
Arg::Input { by_ref: false } => quote!(arg),
});
let run = quote!(holder.#call(#(#pass),*).await?);
let reply = match &self.output {
Answer::None => quote! { #run; Ok(::zip::Reply::Empty) },
Answer::Value(_) => quote! { let answer = #run; ::zip::Reply::json(&answer) },
Answer::Sse | Answer::Body => {
quote! { let answer = #run; Ok(::zip::Reply::from(answer)) }
}
};
quote! {
app.op(#verb, #path, #describe, {
app.op(#verb, #path, {
let holder = ::std::sync::Arc::clone(&holder);
::std::boxed::Box::new(move |input: &::zip::Input| { #run })
move |cx: ::zip::Cx, body: ::zip::Body| {
let holder = ::std::sync::Arc::clone(&holder);
async move {
let _ = &cx;
#take
#reply
}
}
});
}
}
@@ -245,11 +330,11 @@ impl Op {
.text("path", &self.path)
.text("description", &self.doc.text)
.text("pkg", &std::env::var("CARGO_PKG_NAME").unwrap_or_default());
if let Some(t) = &self.input {
o.text("in", &name_of(t));
if let Some(t) = self.input.described() {
o.text("in", &name_of(&t));
}
if let Some(t) = &self.output {
o.text("out", &name_of(t));
if let Some(t) = self.output.described() {
o.text("out", &name_of(&t));
}
if !self.doc.example.is_empty() {
o.raw("example", &self.doc.example);
@@ -257,10 +342,35 @@ impl Op {
if !self.doc.response.is_empty() {
o.raw("response", &self.doc.response);
}
match self.output {
Answer::Sse => {
o.text("stream", "sse");
}
Answer::Body => {
o.text("stream", "bytes");
}
_ => {}
}
o.flag("raw", matches!(self.input, In::Body));
o.finish()
}
}
fn one_input(arg: &FnArg) -> syn::Error {
syn::Error::new_spanned(
arg,
"zip::ops: an op takes one input; a request is one value",
)
}
/// last is the last segment of a type's path: what the macro can see of it.
fn last(t: &Type) -> Option<String> {
match t {
Type::Path(p) => p.path.segments.last().map(|s| s.ident.to_string()),
_ => None,
}
}
/// fragment is the ops half of the manifest — the object's MEMBERS, without its
/// braces, because [zip::manifest] closes it around the types half.
fn fragment(about: &About, ops: &[Op]) -> String {
rust/zip-macros/src/serde.rs
+196
View File
@@ -0,0 +1,196 @@
// SPDX-License-Identifier: BSD-3-Clause-Eco
// Copyright (C) 2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//! What serde was told about a type's names.
//!
//! The derive reads `#[serde(...)]` for the four things that decide what a
//! field is called on the wire — `rename`, `rename_all`, `skip` and `flatten` —
//! plus `with`, which is how a byte field says it is base64. Everything else
//! serde accepts is passed over: it changes how a value is read, not what it is
//! called.
use proc_macro2::TokenStream;
use syn::meta::ParseNestedMeta;
use syn::{Attribute, LitStr, Token};
/// Container is what a struct told serde.
#[derive(Default)]
pub struct Container {
pub rename_all: Option<String>,
pub transparent: bool,
}
impl Container {
pub fn read(attrs: &[Attribute]) -> Result<Self, syn::Error> {
let mut out = Container::default();
for a in serde(attrs) {
a.parse_nested_meta(|meta| {
if meta.path.is_ident("rename_all") {
let rule = both_ways(&meta)?;
if !RULES.contains(&rule.as_str()) {
return Err(meta.error(format!("zip::Wire: no rename_all rule {rule:?}")));
}
out.rename_all = Some(rule);
} else if meta.path.is_ident("transparent") {
out.transparent = true;
} else {
pass(&meta)?;
}
Ok(())
})?;
}
Ok(out)
}
}
/// Field is what a field told serde.
#[derive(Default)]
pub struct Field {
pub rename: Option<String>,
pub skip: bool,
pub flatten: bool,
pub with: Option<String>,
}
impl Field {
pub fn read(attrs: &[Attribute]) -> Result<Self, syn::Error> {
let mut out = Field::default();
let (mut ser, mut de) = (false, false);
for a in serde(attrs) {
a.parse_nested_meta(|meta| {
if meta.path.is_ident("rename") {
out.rename = Some(both_ways(&meta)?);
} else if meta.path.is_ident("skip") {
out.skip = true;
} else if meta.path.is_ident("skip_serializing") {
ser = true;
} else if meta.path.is_ident("skip_deserializing") {
de = true;
} else if meta.path.is_ident("flatten") {
out.flatten = true;
} else if meta.path.is_ident("with") {
out.with = Some(meta.value()?.parse::<LitStr>()?.value());
} else {
pass(&meta)?;
}
Ok(())
})?;
}
out.skip |= ser && de;
Ok(out)
}
}
fn serde(attrs: &[Attribute]) -> impl Iterator<Item = &Attribute> {
attrs.iter().filter(|a| a.path().is_ident("serde"))
}
/// both_ways reads `name = "x"`, or `name(serialize = "x", deserialize = "x")`
/// when the two agree. A name that differs by direction is two names, and a
/// document can publish one.
fn both_ways(meta: &ParseNestedMeta) -> Result<String, syn::Error> {
if meta.input.peek(Token![=]) {
return Ok(meta.value()?.parse::<LitStr>()?.value());
}
let (mut ser, mut de) = (None, None);
meta.parse_nested_meta(|m| {
let v = m.value()?.parse::<LitStr>()?.value();
if m.path.is_ident("serialize") {
ser = Some(v);
} else if m.path.is_ident("deserialize") {
de = Some(v);
} else {
return Err(m.error("expected serialize or deserialize"));
}
Ok(())
})?;
match (ser, de) {
(Some(a), Some(b)) if a == b => Ok(a),
_ => Err(meta.error(
"zip::Wire: a name is the same both ways; one that differs by direction is two names, and a document publishes one",
)),
}
}
/// pass consumes an attribute this derive does not read, whatever its shape.
fn pass(meta: &ParseNestedMeta) -> Result<(), syn::Error> {
if meta.input.peek(Token![=]) {
meta.value()?.parse::<syn::Expr>()?;
} else if meta.input.peek(syn::token::Paren) {
let inner;
syn::parenthesized!(inner in meta.input);
inner.parse::<TokenStream>()?;
}
Ok(())
}
/// RULES are serde's rename_all rules.
const RULES: [&str; 8] = [
"lowercase",
"UPPERCASE",
"PascalCase",
"camelCase",
"snake_case",
"SCREAMING_SNAKE_CASE",
"kebab-case",
"SCREAMING-KEBAB-CASE",
];
/// case is a field's name under a rename_all rule, as serde spells it
/// (serde_derive's `RenameRule::apply_to_field`).
pub fn case(rule: &Option<String>, field: &str) -> String {
let Some(rule) = rule else {
return field.to_string();
};
match rule.as_str() {
"UPPERCASE" | "SCREAMING_SNAKE_CASE" => field.to_ascii_uppercase(),
"PascalCase" => pascal(field),
"camelCase" => {
let p = pascal(field);
let mut c = p.chars();
match c.next() {
Some(first) => first.to_ascii_lowercase().to_string() + c.as_str(),
None => p,
}
}
"kebab-case" => field.replace('_', "-"),
"SCREAMING-KEBAB-CASE" => field.to_ascii_uppercase().replace('_', "-"),
_ => field.to_string(), // lowercase and snake_case leave a field as it is.
}
}
fn pascal(field: &str) -> String {
let mut out = String::with_capacity(field.len());
let mut up = true;
for ch in field.chars() {
if ch == '_' {
up = true;
} else if up {
out.push(ch.to_ascii_uppercase());
up = false;
} else {
out.push(ch);
}
}
out
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn rules_spell_as_serde_does() {
let r = |s: &str| Some(s.to_string());
assert_eq!(case(&None, "node_id"), "node_id");
assert_eq!(case(&r("camelCase"), "node_id"), "nodeId");
assert_eq!(case(&r("camelCase"), "max_tokens"), "maxTokens");
assert_eq!(case(&r("PascalCase"), "node_id"), "NodeId");
assert_eq!(case(&r("SCREAMING_SNAKE_CASE"), "node_id"), "NODE_ID");
assert_eq!(case(&r("kebab-case"), "node_id"), "node-id");
assert_eq!(case(&r("SCREAMING-KEBAB-CASE"), "node_id"), "NODE-ID");
assert_eq!(case(&r("lowercase"), "node_id"), "node_id");
assert_eq!(case(&r("snake_case"), "node_id"), "node_id");
}
}
rust/zip-macros/src/ty.rs
+17 -1
View File
@@ -94,7 +94,12 @@ fn wrap(fields: &str, opt: bool) -> String {
pub fn prim(t: &Type) -> Option<&'static str> {
let name = match t {
Type::Reference(r) => return prim(&r.elem),
Type::Path(p) => p.path.segments.last()?.ident.to_string(),
Type::Path(p) => {
if any(&p.path) {
return Some("any");
}
p.path.segments.last()?.ident.to_string()
}
_ => return None,
};
Some(match name.as_str() {
@@ -114,6 +119,17 @@ pub fn prim(t: &Type) -> Option<&'static str> {
})
}
/// any says a path is serde_json's Value: any JSON at all, which is what a
/// relay holds when it passes a field through without reading it.
fn any(p: &syn::Path) -> bool {
let names: Vec<String> = p.segments.iter().map(|s| s.ident.to_string()).collect();
match names.as_slice() {
[v] => v == "Value",
[.., m, v] => m == "serde_json" && v == "Value",
[] => false,
}
}
fn generics(a: &PathArguments) -> Vec<&Type> {
match a {
PathArguments::AngleBracketed(b) => b
rust/zip-macros/src/wire.rs
+107 -221
View File
@@ -10,16 +10,21 @@
//! names and prose, once, where the type is declared, and everything that
//! refers to the type refers to that.
//!
//! It emits two readers of one description. The `describe` a running service
//! binds a request with, and the fragment `zipc` projects — both from this one
//! pass, so what the document says a field is called is what the binder reads
//! it under.
//! The bytes are serde's. A field's wire name is the one serde reads it under —
//! its `#[serde(rename)]`, else the struct's `rename_all` applied to it, else
//! its own name — so the derive reads those attributes rather than asking for
//! the name a second time, and what the document says a field is called is what
//! the decoder reads it under. What serde cannot be told, and so this cannot
//! describe — a flattened field, a name that differs by direction — is refused
//! at compile time rather than published wrong.
use proc_macro2::TokenStream;
use quote::{format_ident, quote};
use quote::quote;
use syn::ext::IdentExt;
use syn::{Data, DeriveInput, Fields, Type};
use crate::attr::Attrs;
use crate::serde::{self as sd, Container};
use crate::{doc, frag, ty};
pub fn derive(input: DeriveInput) -> Result<TokenStream, syn::Error> {
@@ -27,6 +32,7 @@ pub fn derive(input: DeriveInput) -> Result<TokenStream, syn::Error> {
let id = name.to_string();
let own = Attrs::read(&input.attrs)?;
let prose = doc::read(&input.attrs);
let container = Container::read(&input.attrs)?;
let Data::Struct(s) = &input.data else {
return Err(syn::Error::new_spanned(
@@ -37,10 +43,16 @@ pub fn derive(input: DeriveInput) -> Result<TokenStream, syn::Error> {
match &s.fields {
Fields::Named(named) => {
if container.transparent {
return Err(syn::Error::new_spanned(
&input.ident,
"zip::Wire: a transparent struct is one value; write it as a newtype",
));
}
let fields: Vec<Field> = named
.named
.iter()
.map(Field::read)
.map(|f| Field::read(f, &container))
.collect::<Result<_, _>>()?;
Ok(shape(
&name,
@@ -51,7 +63,7 @@ pub fn derive(input: DeriveInput) -> Result<TokenStream, syn::Error> {
}
Fields::Unnamed(un) if un.unnamed.len() == 1 => {
let inner = &un.unnamed[0].ty;
Ok(newtype(&name, &id, &value(&id, &own, inner), inner, &own))
Ok(newtype(&name, &id, &value(&id, &own, inner), &own))
}
_ => Err(syn::Error::new_spanned(
&input.ident,
@@ -73,13 +85,29 @@ struct Field {
}
impl Field {
fn read(f: &syn::Field) -> Result<Self, syn::Error> {
fn read(f: &syn::Field, container: &Container) -> Result<Self, syn::Error> {
let a = Attrs::read(&f.attrs)?;
let name = f.ident.clone().expect("named field");
let json = if a.skip {
if a.json.is_some() {
return Err(syn::Error::new_spanned(
f,
"zip::Wire: a field's wire name is serde's — #[serde(rename = \"...\")]",
));
}
let serde = sd::Field::read(&f.attrs)?;
if serde.flatten {
return Err(syn::Error::new_spanned(
f,
"zip::Wire: a flattened field has no name of its own to describe; name the struct it holds as a field",
));
}
bytes(f, &f.ty, serde.with.as_deref())?;
let json = if serde.skip {
"-".to_string()
} else if let Some(n) = serde.rename {
n
} else {
a.json.clone().unwrap_or_else(|| name.to_string())
sd::case(&container.rename_all, &name.unraw().to_string())
};
Ok(Field {
json,
@@ -94,6 +122,37 @@ impl Field {
}
}
/// bytes refuses a byte field serde would write as an array of numbers. The
/// document says a byte field is base64 in a string, which is what Go writes,
/// so a `Vec<u8>` carries `#[serde(with = "zip::base64")]`, and one inside a
/// container — which that attribute cannot reach — is refused.
fn bytes(f: &syn::Field, t: &Type, with: Option<&str>) -> Result<(), syn::Error> {
if let Shape::Prim("bytes") = shape_of(t) {
if with == Some("zip::base64") {
return Ok(());
}
return Err(syn::Error::new_spanned(
f,
"zip::Wire: a byte field rides JSON as base64 — #[serde(with = \"zip::base64\")]",
));
}
if holds_bytes(t) {
return Err(syn::Error::new_spanned(
f,
"zip::Wire: bytes inside a container have no base64 spelling here; make them a field of their own type",
));
}
Ok(())
}
fn holds_bytes(t: &Type) -> bool {
match shape_of(t) {
Shape::Prim("bytes") => true,
Shape::Opt(inner) | Shape::List(inner) | Shape::Map(inner) => holds_bytes(&inner),
_ => false,
}
}
/// record is the fragment for a struct.
fn record(id: &str, prose: &str, fields: &[Field]) -> String {
let items: Vec<String> = fields
@@ -140,8 +199,8 @@ fn value(id: &str, a: &Attrs, inner: &Type) -> String {
o.finish()
}
/// shape is the impl for a struct: the description a binder reads, the entry a
/// manifest carries, and the JSON the wire binds.
/// shape is the impl for a struct: the description a binder reads and the
/// entry a manifest carries.
fn shape(name: &syn::Ident, id: &str, stated: &str, fields: &[Field]) -> TokenStream {
let descs = fields.iter().map(|f| {
let n = f.name.to_string();
@@ -155,30 +214,6 @@ fn shape(name: &syn::Ident, id: &str, stated: &str, fields: &[Field]) -> TokenSt
}
});
let writes = fields
.iter()
.filter(|f| f.json != "-")
.enumerate()
.map(|(i, f)| {
let j = &f.json;
let at = &f.name;
let lead = if i == 0 { "\"" } else { ",\"" };
let key = format!("{lead}{j}\":");
let w = write_of(&f.ty, quote!(&self.#at));
quote! { out.push_str(#key); #w; }
});
let reads = fields.iter().map(|f| {
let at = &f.name;
if f.json == "-" {
let d = default_of(&f.ty);
return quote! { #at: #d };
}
let j = &f.json;
let r = read_of(&f.ty, quote!(v.get(#j)));
quote! { #at: #r }
});
let reaches = fields
.iter()
.filter_map(|f| named_in(&f.ty))
@@ -202,62 +237,50 @@ fn shape(name: &syn::Ident, id: &str, stated: &str, fields: &[Field]) -> TokenSt
into.push((#id, #stated));
#(#reaches)*
}
fn write_json(&self, out: &mut ::std::string::String) {
out.push('{');
#(#writes)*
out.push('}');
}
fn read_json(v: &::zip::Json) -> ::std::result::Result<Self, ::zip::Error> {
Ok(#name { #(#reads),* })
}
}
}
}
/// newtype is the impl for a value type. `text` carries it as one word, which
/// is what a quoted decimal is and what a URL can hold.
fn newtype(name: &syn::Ident, id: &str, stated: &str, inner: &Type, a: &Attrs) -> TokenStream {
let says = quote! {
fn stated() -> &'static str { #stated }
fn reach(into: &mut ::std::vec::Vec<(&'static str, &'static str)>) {
if into.iter().all(|(k, _)| *k != #id) {
into.push((#id, #stated));
/// newtype is the impl for a value type. A `text` one is carried as one word —
/// what a quoted decimal is, and what a URL can hold — so its serde impls are
/// written here, from its Display and FromStr: that is what `text` means, and
/// serde has no derive that says it. Any other newtype is serde's, which
/// carries a newtype as the value it holds.
fn newtype(name: &syn::Ident, id: &str, stated: &str, a: &Attrs) -> TokenStream {
let a_text = a.text;
let wire = quote! {
impl ::zip::Wire for #name {
fn describe() -> &'static ::zip::TypeDesc {
static DESC: ::zip::TypeDesc = ::zip::TypeDesc { id: #id, text: #a_text, fields: &[] };
&DESC
}
fn stated() -> &'static str { #stated }
fn reach(into: &mut ::std::vec::Vec<(&'static str, &'static str)>) {
if into.iter().all(|(k, _)| *k != #id) {
into.push((#id, #stated));
}
}
}
};
if a.text {
return quote! {
impl ::zip::Wire for #name {
fn describe() -> &'static ::zip::TypeDesc {
static DESC: ::zip::TypeDesc = ::zip::TypeDesc { id: #id, text: true, fields: &[] };
&DESC
}
#says
fn write_json(&self, out: &mut ::std::string::String) {
::zip::json::write_str(out, &::std::string::ToString::to_string(&self.0));
}
fn read_json(v: &::zip::Json) -> ::std::result::Result<Self, ::zip::Error> {
let s = v.text()?;
match ::std::str::FromStr::from_str(s) {
Ok(n) => Ok(#name(n)),
Err(_) => Err(::zip::Error::bad(concat!("not a ", #id, ": "))),
}
}
}
};
if !a.text {
return wire;
}
let w = write_of(inner, quote!(&self.0));
let r = read_of(inner, quote!(::std::option::Option::Some(v)));
quote! {
impl ::zip::Wire for #name {
fn describe() -> &'static ::zip::TypeDesc {
static DESC: ::zip::TypeDesc = ::zip::TypeDesc { id: #id, text: false, fields: &[] };
&DESC
#wire
impl ::zip::serde::Serialize for #name {
fn serialize<S: ::zip::serde::Serializer>(&self, s: S) -> ::std::result::Result<S::Ok, S::Error> {
s.collect_str(&self.0)
}
#says
fn write_json(&self, out: &mut ::std::string::String) { #w; }
fn read_json(v: &::zip::Json) -> ::std::result::Result<Self, ::zip::Error> {
Ok(#name(#r))
}
impl<'de> ::zip::serde::Deserialize<'de> for #name {
fn deserialize<D: ::zip::serde::Deserializer<'de>>(d: D) -> ::std::result::Result<Self, D::Error> {
let text = <::std::string::String as ::zip::serde::Deserialize>::deserialize(d)?;
match ::std::str::FromStr::from_str(&text) {
Ok(v) => Ok(#name(v)),
Err(_) => Err(<D::Error as ::zip::serde::de::Error>::custom(
::std::format!("not a {}: {:?}", #id, text),
)),
}
}
}
}
@@ -272,6 +295,7 @@ fn scalar_of(t: &Type) -> TokenStream {
"bool" => quote!(::zip::Scalar::Bool),
"f32" | "f64" => quote!(::zip::Scalar::Number),
"bytes" => quote!(::zip::Scalar::Text),
"any" => quote!(::zip::Scalar::Body),
_ => quote!(::zip::Scalar::Number),
},
Shape::Opt(inner) => scalar_of(&inner),
@@ -327,6 +351,7 @@ fn shape_of(t: &Type) -> Shape {
};
match (name.as_str(), args.len()) {
("Option", 1) => Shape::Opt(args[0].clone()),
("Box", 1) | ("Arc", 1) | ("Rc", 1) => shape_of(&args[0]),
("Vec", 1) | ("VecDeque", 1) => {
if ty::prim(&args[0]) == Some("u8") {
Shape::Prim("bytes")
@@ -344,142 +369,3 @@ fn shape_of(t: &Type) -> Shape {
_ => Shape::Other,
}
}
/// write_of is the JSON one value writes.
///
/// Every case binds the value to a name first. A generated expression is
/// TOKENS, and `&self.chains` followed by `.iter()` is `&(self.chains.iter())`
/// — a reference to an iterator, which is not one. A binding has no precedence
/// to get wrong.
fn write_of(t: &Type, at: TokenStream) -> TokenStream {
match shape_of(t) {
Shape::Prim("string") => quote!({ let v = #at; ::zip::json::write_str(out, v); }),
Shape::Prim("bool") => {
quote!({ let v = #at; out.push_str(if *v { "true" } else { "false" }); })
}
Shape::Prim("bytes") => quote!({ let v = #at; ::zip::json::write_bytes(out, v); }),
Shape::Prim("f32") | Shape::Prim("f64") => {
quote!({ let v = #at; ::zip::json::write_float(out, *v as f64); })
}
Shape::Prim(_) => {
quote!({ let v = #at; out.push_str(&::std::string::ToString::to_string(v)); })
}
Shape::Opt(inner) => {
let w = write_of(&inner, quote!(it));
quote! {
{
let v = #at;
match v {
::std::option::Option::Some(it) => { #w; }
::std::option::Option::None => out.push_str("null"),
}
}
}
}
Shape::List(inner) => {
let w = write_of(&inner, quote!(it));
quote! {
{
let v = #at;
out.push('[');
for (i, it) in v.iter().enumerate() {
if i > 0 { out.push(','); }
#w;
}
out.push(']');
}
}
}
Shape::Map(inner) => {
let w = write_of(&inner, quote!(it));
quote! {
{
let v = #at;
// Sorted, because a map has no order and a document that
// changes shape between two identical answers is one nothing
// can diff.
let mut keys: ::std::vec::Vec<_> = v.keys().collect();
keys.sort();
out.push('{');
for (i, k) in keys.into_iter().enumerate() {
if i > 0 { out.push(','); }
::zip::json::write_str(out, k);
out.push(':');
let it = &v[k];
#w;
}
out.push('}');
}
}
}
_ => quote!({ let v = #at; ::zip::Wire::write_json(v, out); }),
}
}
/// read_of is one value, out of the JSON that arrived. `from` is an
/// Option<&Json>: a field the document did not carry reads as its zero, which
/// is what a decoder does everywhere else.
fn read_of(t: &Type, from: TokenStream) -> TokenStream {
match shape_of(t) {
Shape::Prim("string") => quote!(::zip::json::as_text(#from)?),
Shape::Prim("bool") => quote!(::zip::json::as_bool(#from)?),
Shape::Prim("bytes") => quote!(::zip::json::as_bytes(#from)?),
Shape::Prim(p) => {
let cast = format_ident!("{}", num_of(p));
quote!(::zip::json::as_number(#from)? as #cast)
}
Shape::Opt(inner) => {
let r = read_of(&inner, quote!(::std::option::Option::Some(one)));
quote! {
match #from {
::std::option::Option::Some(one) if !one.is_null() => ::std::option::Option::Some(#r),
_ => ::std::option::Option::None,
}
}
}
Shape::List(inner) => {
let r = read_of(&inner, quote!(::std::option::Option::Some(one)));
quote! {
{
let mut got = ::std::vec::Vec::new();
for one in ::zip::json::as_list(#from)? { got.push(#r); }
got
}
}
}
Shape::Map(inner) => {
let r = read_of(&inner, quote!(::std::option::Option::Some(one)));
quote! {
{
let mut got = ::std::collections::HashMap::new();
for (k, one) in ::zip::json::as_map(#from)? { got.insert(k.clone(), #r); }
got
}
}
}
_ => quote!(::zip::json::as_wire(#from)?),
}
}
/// default_of is the zero a field the wire does not carry starts at.
fn default_of(t: &Type) -> TokenStream {
match shape_of(t) {
Shape::Opt(_) => quote!(::std::option::Option::None),
_ => quote!(::std::default::Default::default()),
}
}
fn num_of(p: &str) -> &'static str {
match p {
"i8" => "i8",
"i16" => "i16",
"i32" => "i32",
"i64" => "i64",
"u8" => "u8",
"u16" => "u16",
"u32" => "u32",
"f32" => "f32",
"f64" => "f64",
_ => "u64",
}
}
rust/zip/Cargo.toml
+12
View File
@@ -7,3 +7,15 @@ description = "A ZAP-native web framework: one registration, every projection."
[dependencies]
zip-macros = { path = "../zip-macros" }
bytes = "1"
futures-util = { version = "0.3", default-features = false, features = ["std"] }
http-body-util = "0.1"
hyper = { version = "1.12", features = ["server", "http1", "http2"] }
hyper-util = { version = "0.1.21", features = ["server", "server-auto", "server-graceful", "http1", "http2", "tokio"] }
serde = "1"
serde_json = "1"
tokio = { version = "1.53", features = ["rt-multi-thread", "net", "time", "sync", "signal", "macros", "io-util"] }
[dev-dependencies]
serde = { version = "1", features = ["derive"] }
tokio = { version = "1.53", features = ["test-util"] }
rust/zip/build.rs
-48
View File
@@ -1,48 +0,0 @@
// SPDX-License-Identifier: BSD-3-Clause-Eco
// Copyright (C) 2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//! The wire is generated, here, before the crate compiles.
//!
//! `zapgen` reads wire.zap and writes the ZAP runtime and the accessors for the
//! two frames this door speaks. It is a build-time tool and nothing it produces
//! carries a dependency: what ships is Rust, and what runs is Rust.
use std::path::PathBuf;
use std::process::Command;
fn main() {
println!("cargo:rerun-if-changed=wire.zap");
println!("cargo:rerun-if-env-changed=ZAPGEN");
let out = PathBuf::from(std::env::var("OUT_DIR").expect("OUT_DIR"));
let zapgen = std::env::var("ZAPGEN").unwrap_or_else(|_| "zapgen".to_string());
let run = Command::new(&zapgen)
.args(["-lang", "rust", "-out"])
.arg(&out)
.arg("wire.zap")
.output();
match run {
Ok(r) if r.status.success() => {
// One file holding both modules, because `include!` pastes tokens
// and a pasted file cannot carry the inner attributes a generated
// module begins with. Braced, they are ordinary items again.
let mut all =
String::from("// Written by zip's build script from what zapgen emitted.\n");
for (name, file) in [("zap", "zap.rs"), ("wire", "wire_zap.rs")] {
let body = std::fs::read_to_string(out.join(file))
.unwrap_or_else(|e| panic!("zapgen wrote no {file}: {e}"));
all.push_str(&format!("pub mod {name} {{\n{body}\n}}\n"));
}
std::fs::write(out.join("generated.rs"), all).expect("write generated.rs");
}
Ok(r) => panic!(
"zapgen refused wire.zap:\n{}",
String::from_utf8_lossy(&r.stderr)
),
Err(e) => panic!(
"zip needs zapgen to generate its wire ({zapgen}: {e}).\n\
Build it from zap-proto/go (cmd/zapgen) and put it on PATH, or set ZAPGEN to it."
),
}
}
rust/zip/src/app.rs
+180 -174
View File
@@ -2,50 +2,35 @@
// Copyright (C) 2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//! The app: a typed-op registry, and the two doors onto it.
//! The app: a typed-op registry, and the one path from a request to an answer.
//!
//! An op is registered once. What answers a REST request and what answers a ZAP
//! frame are the same closure, reached the same way — decode, run, write — so a
//! browser and a sibling service are talking to one handler and not to two
//! frame are the same future, reached the same way — route, bind, run, answer —
//! so a browser and a sibling service are talking to one handler and not to two
//! spellings of one.
use std::collections::HashMap;
use std::future::Future;
use std::net::SocketAddr;
use std::panic::AssertUnwindSafe;
use std::sync::{Arc, Mutex};
use crate::{Error, FieldDesc, Json, Scalar, TypeDesc, Wire};
use bytes::Bytes;
use futures_util::future::BoxFuture;
use futures_util::stream::BoxStream;
use futures_util::FutureExt;
use hyper::header::{self, HeaderMap, HeaderValue};
/// Input is one request, before it is a value: what the URL carried, what the
/// headers said, and the bytes of the body.
pub struct Input {
pub method: String,
pub path: String,
pub query: Vec<(String, String)>,
pub params: Vec<(String, String)>,
pub headers: Vec<(String, String)>,
pub body: Vec<u8>,
}
use crate::cx::{lock, Out};
use crate::{Body, Cx, Error, Reply};
impl Input {
/// header reads one request header, case-insensitively as HTTP names are.
pub fn header(&self, name: &str) -> Option<&str> {
self.headers
.iter()
.find(|(k, _)| k.eq_ignore_ascii_case(name))
.map(|(_, v)| v.as_str())
}
}
/// Call is what an op does: a request in, the answer's JSON out. None is an op
/// that answers nothing, which is a 204.
pub type Call = Box<dyn Fn(&Input) -> Result<Option<String>, Error> + Send + Sync>;
type Call = Box<dyn Fn(Cx, Body) -> BoxFuture<'static, Result<Reply, Error>> + Send + Sync>;
/// Op is one registered operation.
pub struct Op {
pub method: &'static str,
pub path: &'static str,
/// input is the description of what this op takes, or None for an op that
/// takes nothing.
pub input: Option<&'static TypeDesc>,
pub call: Call,
call: Call,
segments: Vec<Segment>,
}
@@ -65,6 +50,54 @@ pub struct App {
files: HashMap<String, (&'static str, &'static str)>,
}
/// Asked is one request as a door read it.
pub(crate) struct Asked {
pub method: String,
pub target: String,
pub headers: HeaderMap,
pub peer: Option<SocketAddr>,
pub body: Body,
}
/// Answer is what a door writes back: a head, then a body that is either in
/// hand or still being produced, then — when the op declared any — trailers.
pub(crate) struct Answer {
pub status: u16,
pub headers: HeaderMap,
pub payload: Payload,
/// trailers is the handler's state when it declared a trailer, read once
/// the payload has ended.
pub trailers: Option<Arc<Mutex<Out>>>,
}
pub(crate) enum Payload {
Full(Bytes),
Stream(BoxStream<'static, Result<Bytes, Error>>),
}
impl Answer {
pub(crate) fn refusal(e: &Error, headers: HeaderMap) -> Answer {
let mut headers = headers;
headers.insert(
header::CONTENT_TYPE,
HeaderValue::from_static("application/problem+json"),
);
Answer {
status: e.status,
headers,
payload: Payload::Full(Bytes::from(e.problem())),
trailers: None,
}
}
}
/// trailers is what the handler set by the time the body ended, or None when
/// it set nothing.
pub(crate) fn trailers(out: &Mutex<Out>) -> Option<HeaderMap> {
let t = std::mem::take(&mut lock(out).trailers);
(!t.is_empty()).then_some(t)
}
impl App {
pub fn new(
name: &'static str,
@@ -83,18 +116,15 @@ impl App {
}
/// op registers one operation. The macro calls this; a service does not.
pub fn op(
&mut self,
method: &'static str,
path: &'static str,
input: Option<&'static TypeDesc>,
call: Call,
) -> &mut Self {
pub fn op<F, R>(&mut self, method: &'static str, path: &'static str, call: F) -> &mut Self
where
F: Fn(Cx, Body) -> R + Send + Sync + 'static,
R: Future<Output = Result<Reply, Error>> + Send + 'static,
{
self.ops.push(Op {
method,
path,
input,
call,
call: Box::new(move |cx, body| Box::pin(call(cx, body))),
segments: split(path),
});
self
@@ -116,22 +146,19 @@ impl App {
self.ops.iter().map(|o| (o.method, o.path)).collect()
}
/// answer runs a request: find the op, decode, run, write. It is the ONE
/// path in, so a REST request and a ZAP frame cannot diverge.
pub fn answer(
&self,
method: &str,
target: &str,
headers: Vec<(String, String)>,
body: Vec<u8>,
) -> Answer {
let (path, query) = split_target(target);
if method == "GET" {
/// answer runs a request: route, bind, run, answer. It is the ONE path in,
/// so a REST request and a ZAP frame cannot diverge.
pub(crate) async fn answer(&self, asked: Asked) -> Answer {
let (path, query) = split_target(&asked.target);
if asked.method == "GET" {
if let Some((kind, text)) = self.files.get(path) {
let mut headers = HeaderMap::new();
headers.insert(header::CONTENT_TYPE, HeaderValue::from_static(kind));
return Answer {
status: 200,
kind,
body: text.as_bytes().to_vec(),
headers,
payload: Payload::Full(Bytes::from_static(text.as_bytes())),
trailers: None,
};
}
}
@@ -140,54 +167,85 @@ impl App {
let Some(params) = op.bind_path(path) else {
continue;
};
if op.method != method {
if op.method != asked.method {
allowed = true;
continue;
}
let input = Input {
method: method.to_string(),
path: path.to_string(),
let cx = Cx::new(
asked.method,
path.to_string(),
query,
params,
headers,
body,
};
return match (op.call)(&input) {
Ok(Some(json)) => Answer {
status: 200,
kind: "application/json",
body: json.into_bytes(),
},
Ok(None) => Answer {
status: 204,
kind: "application/json",
body: Vec::new(),
},
Err(e) => Answer {
status: e.status,
kind: "application/problem+json",
body: e.problem().into_bytes(),
},
asked.headers,
asked.peer,
);
let out = Arc::clone(&cx.out);
let run = AssertUnwindSafe((op.call)(cx, asked.body)).catch_unwind();
let reply = match run.await {
Ok(r) => r,
Err(_) => Err(Error::broke("the handler panicked")),
};
return reply_to(reply, out);
}
let e = if allowed {
Error::new(405, "that address does not answer this method")
} else {
Error::missing("no such address")
};
Answer {
status: e.status,
kind: "application/problem+json",
body: e.problem().into_bytes(),
}
Answer::refusal(&e, HeaderMap::new())
}
}
/// Answer is what a door writes back.
pub struct Answer {
pub status: u16,
pub kind: &'static str,
pub body: Vec<u8>,
/// reply_to is a reply as a head and a body: the kind's own status and content
/// type, then whatever the handler set on its Cx over them.
fn reply_to(reply: Result<Reply, Error>, out: Arc<Mutex<Out>>) -> Answer {
let (status, mut headers, declared) = {
let mut o = lock(&out);
(
o.status.take(),
std::mem::take(&mut o.headers),
std::mem::take(&mut o.declared),
)
};
let reply = match reply {
Ok(r) => r,
Err(e) => return Answer::refusal(&e, headers),
};
let (dflt, kind, payload) = match reply {
Reply::Json(b) => (200, Some("application/json"), Payload::Full(b)),
Reply::Empty => (204, None, Payload::Full(Bytes::new())),
Reply::Sse(s) => {
headers
.entry(header::CACHE_CONTROL)
.or_insert(HeaderValue::from_static("no-cache"));
(200, Some("text/event-stream"), Payload::Stream(s.bytes()))
}
Reply::Body(b) => (
200,
Some("application/octet-stream"),
Payload::Stream(Box::pin(b)),
),
};
if let Some(kind) = kind {
headers
.entry(header::CONTENT_TYPE)
.or_insert(HeaderValue::from_static(kind));
}
let trailers = if declared.is_empty() {
None
} else {
let names: Vec<&str> = declared.iter().map(|n| n.as_str()).collect();
let value = HeaderValue::from_str(&names.join(", "))
.expect("header names joined by commas are a header value");
headers.insert(header::TRAILER, value);
Some(out)
};
Answer {
status: status.unwrap_or(dflt),
headers,
payload,
trailers,
}
}
impl Op {
@@ -285,98 +343,46 @@ fn unescape(s: &str) -> String {
String::from_utf8_lossy(&out).into_owned()
}
/// bind is one request as the value the handler takes.
///
/// The body carries what a body can, and the URL carries the rest — and where
/// both spoke, the URL wins, because a URL is what addresses the resource. That
/// is one rule for every op: a GET's whole input arrives in the URL and a POST's
/// path parameters still bind, without either being a special case.
pub fn bind<T: Wire>(input: &Input) -> Result<T, Error> {
let desc = T::describe();
let mut object = match body_of(input)? {
Json::Map(m) => m,
Json::Null => HashMap::new(),
other => return T::read_json(&other), // the body IS the whole value
};
for f in desc.fields {
if !f.header.is_empty() {
if let Some(v) = input.header(f.header) {
object.insert(f.json.to_string(), scalar(v, &f.scalar));
}
#[cfg(test)]
mod tests {
use super::*;
fn op(path: &'static str) -> Op {
Op {
method: "GET",
path,
call: Box::new(|_, _| Box::pin(async { Ok(Reply::Empty) })),
segments: split(path),
}
}
for (name, value) in input.query.iter().chain(input.params.iter()) {
if let Some(f) = field_for(desc, name) {
object.insert(f.json.to_string(), scalar(value, &f.scalar));
}
}
require(desc, &object)?;
T::read_json(&Json::Map(object))
}
/// require refuses a request missing a value the op declared it cannot run
/// without.
///
/// It runs HERE, on the bound input, and not in the handler: the document says
/// the field is required, so a service that only checked it in some handlers
/// would publish a contract it keeps by habit. A field is missing when it is
/// absent or when it is the zero its type reads as — the same rule the Go side
/// applies, so one client sees one answer from either.
fn require(desc: &'static TypeDesc, object: &HashMap<String, Json>) -> Result<(), Error> {
for f in desc.fields {
if !f.required {
continue;
}
let given = match object.get(f.json) {
Some(Json::Text(s)) => !s.is_empty(),
Some(Json::Number(n)) => *n != 0.0,
Some(Json::Bool(b)) => *b,
Some(Json::List(l)) => !l.is_empty(),
Some(Json::Map(m)) => !m.is_empty(),
Some(Json::Null) | None => false,
};
if !given {
return Err(Error::bad(format!("field {:?} is required", f.json)));
}
#[test]
fn a_pattern_binds_what_it_names() {
let o = op("/v1/models/:id");
assert_eq!(
o.bind_path("/v1/models/kai"),
Some(vec![("id".to_string(), "kai".to_string())])
);
assert_eq!(o.bind_path("/v1/models"), None);
assert_eq!(o.bind_path("/v1/models/kai/x"), None);
let rest = op("/files/*");
assert_eq!(
rest.bind_path("/files/a/b"),
Some(vec![("*1".to_string(), "a/b".to_string())])
);
}
Ok(())
}
fn body_of(input: &Input) -> Result<Json, Error> {
if input.body.is_empty() {
return Ok(Json::Null);
}
let text = std::str::from_utf8(&input.body).map_err(|_| Error::bad("the body is not utf-8"))?;
if text.trim().is_empty() {
return Ok(Json::Null);
}
Json::parse(text)
}
/// field_for is the field a URL name binds to. "-" opts out, exactly as it does
/// for the body: a field can be body-only and a field can be URL-only, and the
/// two halves are asked separately because a route may mean two things by one
/// word.
fn field_for(desc: &'static TypeDesc, name: &str) -> Option<&'static FieldDesc> {
desc.fields.iter().find(|f| {
let url = f.url_name();
url != "-" && url.eq_ignore_ascii_case(name)
})
}
/// scalar is one URL value as the JSON its field expects. A list is spelled
/// comma-separated in one value, which is what `style: form, explode: false`
/// already publishes for a repeated parameter.
fn scalar(text: &str, want: &Scalar) -> Json {
match want {
Scalar::Bool => Json::Bool(text == "true" || text == "1"),
Scalar::Number => text.parse().map(Json::Number).unwrap_or(Json::Null),
Scalar::List(elem) => Json::List(
text.split(',')
.filter(|s| !s.is_empty())
.map(|s| scalar(s, elem))
.collect(),
),
_ => Json::Text(text.to_string()),
#[test]
fn a_query_is_unescaped() {
let (path, q) = split_target("/x?a=1%202&b=c+d&e");
assert_eq!(path, "/x");
assert_eq!(
q,
vec![
("a".to_string(), "1 2".to_string()),
("b".to_string(), "c d".to_string()),
("e".to_string(), String::new()),
]
);
}
}
rust/zip/src/base64.rs
+93
View File
@@ -0,0 +1,93 @@
// SPDX-License-Identifier: BSD-3-Clause-Eco
// Copyright (C) 2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//! A byte field, as JSON carries one: standard base64 in a string.
//!
//! serde writes a `Vec<u8>` as an array of numbers, which is not what the
//! document says a byte field is (`string`, `contentEncoding: base64`) nor what
//! Go's encoder writes. A byte field therefore names this module, and the
//! derive refuses one that does not:
//!
//! ```ignore
//! #[serde(with = "zip::base64")]
//! pub payload: Vec<u8>,
//! ```
use serde::{Deserialize, Deserializer, Serializer};
const ALPHABET: &[u8; 64] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
pub fn serialize<S: Serializer>(bytes: &[u8], s: S) -> Result<S::Ok, S::Error> {
s.serialize_str(&encode(bytes))
}
pub fn deserialize<'de, D: Deserializer<'de>>(d: D) -> Result<Vec<u8>, D::Error> {
let text = String::deserialize(d)?;
decode(&text).map_err(serde::de::Error::custom)
}
/// encode is bytes as base64, padded.
pub fn encode(b: &[u8]) -> String {
let mut s = String::with_capacity(b.len().div_ceil(3) * 4);
for c in b.chunks(3) {
let n = (c[0] as u32) << 16
| (*c.get(1).unwrap_or(&0) as u32) << 8
| (*c.get(2).unwrap_or(&0) as u32);
s.push(ALPHABET[(n >> 18 & 63) as usize] as char);
s.push(ALPHABET[(n >> 12 & 63) as usize] as char);
s.push(if c.len() > 1 {
ALPHABET[(n >> 6 & 63) as usize] as char
} else {
'='
});
s.push(if c.len() > 2 {
ALPHABET[(n & 63) as usize] as char
} else {
'='
});
}
s
}
/// decode is base64, back. Padding and line breaks are skipped; any other
/// byte outside the alphabet is a refusal.
pub fn decode(s: &str) -> Result<Vec<u8>, &'static str> {
let mut out = Vec::with_capacity(s.len() / 4 * 3);
let mut buf = 0u32;
let mut have = 0;
for c in s.bytes() {
let v = match c {
b'A'..=b'Z' => c - b'A',
b'a'..=b'z' => c - b'a' + 26,
b'0'..=b'9' => c - b'0' + 52,
b'+' => 62,
b'/' => 63,
b'=' | b'\n' | b'\r' => continue,
_ => return Err("not base64"),
} as u32;
buf = buf << 6 | v;
have += 6;
if have >= 8 {
have -= 8;
out.push((buf >> have) as u8);
}
}
Ok(out)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn round_trips() {
for n in 0..10 {
let b: Vec<u8> = (0..n).map(|i| (i * 37 + 11) as u8).collect();
assert_eq!(decode(&encode(&b)).unwrap(), b);
}
assert_eq!(encode(b"zip"), "emlw");
assert_eq!(encode(b"zi"), "emk=");
assert!(decode("a*b").is_err());
}
}
rust/zip/src/bind.rs
+149
View File
@@ -0,0 +1,149 @@
// SPDX-License-Identifier: BSD-3-Clause-Eco
// Copyright (C) 2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//! One request, as the value the handler takes.
//!
//! The body carries what a body can, and the URL carries the rest — and where
//! both spoke, the URL wins, because a URL is what addresses the resource. That
//! is one rule for every op: a GET's whole input arrives in the URL and a POST's
//! path parameters still bind, without either being a special case.
use serde::de::DeserializeOwned;
use serde_json::{Map, Value};
use crate::{Body, Cx, Error, FieldDesc, Scalar, TypeDesc, Wire};
/// MAX_BODY is the largest body an op's input is read from, the ZAP door's
/// frame bound. An op that must take more takes a `Body` and reads it as it
/// arrives.
pub const MAX_BODY: usize = 32 << 20;
/// bind reads the body and binds the request to the op's input.
pub async fn bind<T: Wire + DeserializeOwned>(cx: &Cx, body: Body) -> Result<T, Error> {
// A length past the bound is refused on its say-so, before a byte of it is
// waited for; a body that does not declare one is counted as it arrives.
if let Some(n) = cx
.header("content-length")
.and_then(|v| v.parse::<usize>().ok())
{
if n > MAX_BODY {
return Err(Error::new(
413,
format!("the body is over {MAX_BODY} bytes"),
));
}
}
let bytes = body.bytes(MAX_BODY).await?;
value::<T>(cx, &bytes)
}
/// value is [bind] over a body already read.
pub fn value<T: Wire + DeserializeOwned>(cx: &Cx, body: &[u8]) -> Result<T, Error> {
let desc = T::describe();
let first = body.iter().find(|b| !b.is_ascii_whitespace());
// What the URL and the headers add to the body. When they add nothing and
// nothing is required, the body is the value and is read once, straight
// into it: a chat request with an image in it is not copied through a tree
// first.
let mut from = Vec::new();
for f in desc.fields {
if !f.header.is_empty() {
if let Some(v) = cx.header(f.header) {
from.push((f, v));
}
}
}
for (name, v) in cx.query.iter().chain(cx.params.iter()) {
if let Some(f) = field_for(desc, name) {
from.push((f, v.as_str()));
}
}
let required = desc.fields.iter().any(|f| f.required);
if from.is_empty() && !required && first == Some(&b'{') {
return serde_json::from_slice(body).map_err(refuse);
}
let mut object = if first.is_none() {
Map::new()
} else {
match serde_json::from_slice::<Value>(body).map_err(refuse)? {
Value::Object(m) => m,
Value::Null => Map::new(),
other => return serde_json::from_value(other).map_err(refuse), // the body IS the whole value
}
};
for (f, v) in from {
object.insert(f.json.to_string(), scalar(v, &f.scalar));
}
require(desc, &object)?;
serde_json::from_value(Value::Object(object)).map_err(refuse)
}
fn refuse(e: serde_json::Error) -> Error {
Error::bad(format!("the input does not read: {e}"))
}
/// require refuses a request missing a value the op declared it cannot run
/// without.
///
/// It runs HERE, on the bound input, and not in the handler: the document says
/// the field is required, so a service that only checked it in some handlers
/// would publish a contract it keeps by habit. A field is missing when it is
/// absent or when it is the zero its type reads as — the same rule the Go side
/// applies, so one client sees one answer from either.
fn require(desc: &'static TypeDesc, object: &Map<String, Value>) -> Result<(), Error> {
for f in desc.fields {
if !f.required {
continue;
}
let given = match object.get(f.json) {
Some(Value::String(s)) => !s.is_empty(),
Some(Value::Number(n)) => n.as_f64() != Some(0.0),
Some(Value::Bool(b)) => *b,
Some(Value::Array(l)) => !l.is_empty(),
Some(Value::Object(m)) => !m.is_empty(),
Some(Value::Null) | None => false,
};
if !given {
return Err(Error::bad(format!("field {:?} is required", f.json)));
}
}
Ok(())
}
/// field_for is the field a URL name binds to. "-" opts out, exactly as it does
/// for the body: a field can be body-only and a field can be URL-only, and the
/// two halves are asked separately because a route may mean two things by one
/// word.
fn field_for(desc: &'static TypeDesc, name: &str) -> Option<&'static FieldDesc> {
desc.fields.iter().find(|f| {
let url = f.url_name();
url != "-" && url.eq_ignore_ascii_case(name)
})
}
/// scalar is one URL value as the JSON its field expects. A list is spelled
/// comma-separated in one value, which is what `style: form, explode: false`
/// already publishes for a repeated parameter. A number is read as an integer
/// when it is one, since serde will not read 1.0 into a u32.
fn scalar(text: &str, want: &Scalar) -> Value {
match want {
Scalar::Bool => Value::Bool(text == "true" || text == "1"),
Scalar::Number => {
if let Ok(i) = text.parse::<i64>() {
Value::from(i)
} else if let Ok(u) = text.parse::<u64>() {
Value::from(u)
} else {
text.parse::<f64>().map(Value::from).unwrap_or(Value::Null)
}
}
Scalar::List(elem) => Value::Array(
text.split(',')
.filter(|s| !s.is_empty())
.map(|s| scalar(s, elem))
.collect(),
),
_ => Value::String(text.to_string()),
}
}
rust/zip/src/body.rs
+426
View File
@@ -0,0 +1,426 @@
// SPDX-License-Identifier: BSD-3-Clause-Eco
// Copyright (C) 2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//! What an op answers, and the bytes it reads as they arrive.
//!
//! An op answers one JSON value, nothing, a stream of server-sent events, or a
//! stream of bytes. The two streams are values the handler returns, not writers
//! it is handed: a door polls them, so a chunk goes out when the producer yields
//! it, and when the peer goes away the door drops the stream — which drops
//! whatever the producer was holding, an upstream request included.
//!
//! A stream item is a `Result`. `Ok` is written; `Err` ends the answer without
//! its terminator — a truncated chunked body over HTTP/1.1, a reset stream over
//! HTTP/2 — because the status went out with the head and is no longer a thing
//! that can say the answer failed. A producer that wants to say so in words
//! sends an event first and then ends.
use std::future::Future;
use std::pin::Pin;
use std::task::{Context, Poll};
use std::time::Duration;
use bytes::Bytes;
use futures_util::stream::{BoxStream, Stream, StreamExt};
use serde::Serialize;
use crate::Error;
type BoxError = Box<dyn std::error::Error + Send + Sync>;
/// failed is a stream's error as zip's: a zip::Error keeps its status and
/// detail, anything else is the service's own fault.
fn failed(e: BoxError) -> Error {
match e.downcast::<Error>() {
Ok(e) => *e,
Err(e) => Error::broke(e.to_string()),
}
}
/// Body is bytes as they arrive: an op's request body, read as sent, or its
/// answer, written as produced.
pub struct Body {
chunks: BoxStream<'static, Result<Bytes, Error>>,
}
impl Body {
/// new is a body from any stream of chunks. An error item ends the body
/// unfinished.
pub fn new<S, B, E>(chunks: S) -> Body
where
S: Stream<Item = Result<B, E>> + Send + 'static,
B: Into<Bytes>,
E: Into<BoxError>,
{
Body {
chunks: chunks
.map(|c| c.map(Into::into).map_err(|e| failed(e.into())))
.boxed(),
}
}
/// empty is a body with nothing in it.
pub fn empty() -> Body {
Body {
chunks: futures_util::stream::empty().boxed(),
}
}
/// full is a body that is one chunk, already in hand.
pub fn full(bytes: impl Into<Bytes>) -> Body {
let bytes = bytes.into();
if bytes.is_empty() {
return Body::empty();
}
Body {
chunks: futures_util::stream::once(async move { Ok(bytes) }).boxed(),
}
}
/// bytes reads the whole body, refusing one past `limit` with a 413 as soon
/// as it is past it — before the rest is read, not after.
pub async fn bytes(mut self, limit: usize) -> Result<Bytes, Error> {
let Some(first) = self.chunks.next().await else {
return Ok(Bytes::new());
};
let first = first?;
let Some(second) = self.chunks.next().await else {
return within(first, limit);
};
let mut all = Vec::with_capacity(first.len() * 2);
all.extend_from_slice(&first);
let mut next = Some(second);
while let Some(chunk) = next {
let chunk = chunk?;
if all.len() + chunk.len() > limit {
return Err(too_large(limit));
}
all.extend_from_slice(&chunk);
next = self.chunks.next().await;
}
Ok(Bytes::from(all))
}
}
fn within(b: Bytes, limit: usize) -> Result<Bytes, Error> {
if b.len() > limit {
return Err(too_large(limit));
}
Ok(b)
}
fn too_large(limit: usize) -> Error {
Error::new(413, format!("the body is over {limit} bytes"))
}
impl Stream for Body {
type Item = Result<Bytes, Error>;
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
self.chunks.poll_next_unpin(cx)
}
}
/// Event is one server-sent event.
///
/// Every field is optional on the wire and each is written as its own line, so
/// a value holding a line break is split into as many lines as it holds and
/// cannot start a field of its own: `data` and comments become several lines
/// of the same field, and `event` and `id`, which are one line by definition,
/// lose their line breaks.
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct Event {
data: Option<String>,
event: Option<String>,
id: Option<String>,
retry: Option<u64>,
comment: Option<String>,
}
impl Event {
/// data is an event carrying text.
pub fn data(text: impl Into<String>) -> Event {
Event {
data: Some(text.into()),
..Event::default()
}
}
/// json is an event carrying one JSON value, on one line.
pub fn json<T: Serialize>(value: &T) -> Result<Event, Error> {
serde_json::to_string(value)
.map(Event::data)
.map_err(|e| Error::broke(format!("an event did not encode: {e}")))
}
/// comment is a line a client ignores. It is what keeps an idle stream
/// open through a proxy that closes a quiet one.
pub fn comment(text: impl Into<String>) -> Event {
Event {
comment: Some(text.into()),
..Event::default()
}
}
/// event names the kind of event, which a browser dispatches on.
pub fn event(mut self, name: impl Into<String>) -> Event {
self.event = Some(one_line(name.into()));
self
}
/// id is the id a reconnecting client sends back as Last-Event-ID.
pub fn id(mut self, id: impl Into<String>) -> Event {
self.id = Some(one_line(id.into()));
self
}
/// retry is how long, in milliseconds, a client waits before reconnecting.
pub fn retry(mut self, ms: u64) -> Event {
self.retry = Some(ms);
self
}
/// encode is this event as the bytes the stream carries.
pub fn encode(&self) -> Bytes {
let mut out = String::new();
if let Some(c) = &self.comment {
for line in lines(c) {
out.push(':');
if !line.is_empty() {
out.push(' ');
out.push_str(line);
}
out.push('\n');
}
}
if let Some(e) = &self.event {
out.push_str("event: ");
out.push_str(e);
out.push('\n');
}
if let Some(id) = &self.id {
out.push_str("id: ");
out.push_str(id);
out.push('\n');
}
if let Some(ms) = self.retry {
out.push_str("retry: ");
out.push_str(&ms.to_string());
out.push('\n');
}
if let Some(d) = &self.data {
for line in lines(d) {
out.push_str("data: ");
out.push_str(line);
out.push('\n');
}
}
out.push('\n');
Bytes::from(out)
}
}
/// lines splits on every line break the event-stream format knows: CRLF, LF
/// and a lone CR.
fn lines(s: &str) -> Vec<&str> {
let mut out = Vec::new();
let b = s.as_bytes();
let mut start = 0;
let mut i = 0;
while i < b.len() {
match b[i] {
b'\n' => {
out.push(&s[start..i]);
start = i + 1;
}
b'\r' => {
out.push(&s[start..i]);
if b.get(i + 1) == Some(&b'\n') {
i += 1;
}
start = i + 1;
}
_ => {}
}
i += 1;
}
out.push(&s[start..]);
out
}
fn one_line(s: String) -> String {
if s.contains(['\r', '\n']) {
s.replace(['\r', '\n'], "")
} else {
s
}
}
/// Sse is an answer of server-sent events (`text/event-stream`), each written
/// and flushed as the producer yields it.
pub struct Sse {
events: BoxStream<'static, Result<Event, Error>>,
keep: Option<Duration>,
}
impl Sse {
/// new is an event stream from any stream of events. An error item ends the
/// stream unfinished; the stream ending ends the answer cleanly.
pub fn new<S, E>(events: S) -> Sse
where
S: Stream<Item = Result<Event, E>> + Send + 'static,
E: Into<BoxError>,
{
Sse {
events: events.map(|e| e.map_err(|e| failed(e.into()))).boxed(),
keep: None,
}
}
/// keep writes a comment line whenever `every` passes with no event, so a
/// proxy does not close a stream that is only waiting, and a client that
/// left is found out by the write.
pub fn keep(mut self, every: Duration) -> Sse {
self.keep = Some(every);
self
}
/// bytes is the stream as the door writes it.
pub(crate) fn bytes(self) -> BoxStream<'static, Result<Bytes, Error>> {
match self.keep {
None => self.events.map(|e| e.map(|e| e.encode())).boxed(),
Some(every) => Keep {
events: self.events,
every,
timer: Box::pin(tokio::time::sleep(every)),
}
.boxed(),
}
}
}
/// Keep is an event stream with a comment line in every quiet interval.
struct Keep {
events: BoxStream<'static, Result<Event, Error>>,
every: Duration,
timer: Pin<Box<tokio::time::Sleep>>,
}
/// KEEP is the line a quiet interval writes: an empty comment.
const KEEP: &[u8] = b":\n\n";
impl Stream for Keep {
type Item = Result<Bytes, Error>;
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
let this = &mut *self;
if let Poll::Ready(next) = this.events.poll_next_unpin(cx) {
let at = tokio::time::Instant::now() + this.every;
this.timer.as_mut().reset(at);
return Poll::Ready(next.map(|e| e.map(|e| e.encode())));
}
if this.timer.as_mut().poll(cx).is_ready() {
let at = tokio::time::Instant::now() + this.every;
this.timer.as_mut().reset(at);
return Poll::Ready(Some(Ok(Bytes::from_static(KEEP))));
}
Poll::Pending
}
}
/// Reply is what an op answered. The macro builds it from the handler's return
/// value; a handler names `Sse`, `Body` or a type of its own and never this.
pub enum Reply {
/// Json is one value, encoded: 200, `application/json`.
Json(Bytes),
/// Empty is an op that answers nothing: 204.
Empty,
/// Sse is an event stream: 200, `text/event-stream`.
Sse(Sse),
/// Body is a byte stream: 200, `application/octet-stream` unless the
/// handler set a type.
Body(Body),
}
impl Reply {
/// json encodes one answer.
pub fn json<T: Serialize>(value: &T) -> Result<Reply, Error> {
serde_json::to_vec(value)
.map(|b| Reply::Json(Bytes::from(b)))
.map_err(|e| Error::broke(format!("the answer did not encode: {e}")))
}
}
impl From<Sse> for Reply {
fn from(s: Sse) -> Reply {
Reply::Sse(s)
}
}
impl From<Body> for Reply {
fn from(b: Body) -> Reply {
Reply::Body(b)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn an_event_is_its_fields_then_a_blank_line() {
assert_eq!(&Event::data("hi").encode()[..], b"data: hi\n\n");
assert_eq!(
&Event::data("a\nb\r\nc\rd")
.event("delta")
.id("7")
.retry(500)
.encode()[..],
b"event: delta\nid: 7\nretry: 500\ndata: a\ndata: b\ndata: c\ndata: d\n\n"
);
assert_eq!(&Event::comment("").encode()[..], b":\n\n");
assert_eq!(&Event::comment("ping").encode()[..], b": ping\n\n");
// A line break cannot start a field of its own.
assert_eq!(
&Event::data("x").event("a\ndata: forged").encode()[..],
b"event: adata: forged\ndata: x\n\n"
);
let e = Event::json(&serde_json::json!({"a": "b\nc"})).unwrap();
assert_eq!(&e.encode()[..], b"data: {\"a\":\"b\\nc\"}\n\n");
}
#[tokio::test]
async fn a_body_past_its_limit_is_refused() {
let chunks = futures_util::stream::iter(vec![
Ok::<_, Error>(Bytes::from_static(b"12345")),
Ok(Bytes::from_static(b"678")),
]);
let got = Body::new(chunks).bytes(6).await.unwrap_err();
assert_eq!(got.status, 413);
let chunks = futures_util::stream::iter(vec![
Ok::<_, Error>(Bytes::from_static(b"12345")),
Ok(Bytes::from_static(b"678")),
]);
assert_eq!(&Body::new(chunks).bytes(8).await.unwrap()[..], b"12345678");
assert_eq!(
Body::full("1234567").bytes(6).await.unwrap_err().status,
413
);
assert!(Body::empty().bytes(0).await.unwrap().is_empty());
}
#[tokio::test(start_paused = true)]
async fn a_quiet_stream_keeps_itself_open() {
let (tx, rx) = tokio::sync::mpsc::channel::<Result<Event, Error>>(1);
let mut s = Sse::new(futures_util::stream::unfold(rx, |mut rx| async move {
rx.recv().await.map(|e| (e, rx))
}))
.keep(Duration::from_secs(15))
.bytes();
// Nothing for 15 s: a comment.
assert_eq!(&s.next().await.unwrap().unwrap()[..], KEEP);
tx.send(Ok(Event::data("1"))).await.unwrap();
assert_eq!(&s.next().await.unwrap().unwrap()[..], b"data: 1\n\n");
drop(tx);
assert!(s.next().await.is_none());
}
}
rust/zip/src/cx.rs
+170
View File
@@ -0,0 +1,170 @@
// SPDX-License-Identifier: BSD-3-Clause-Eco
// Copyright (C) 2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//! One request, as a handler sees it beside its input.
//!
//! An op's input is the value the request carried; `Cx` is the rest of the
//! exchange: the headers as they arrived, and what the answer carries besides
//! its body — its status, its headers and its trailers. A handler that takes
//! `&zip::Cx` gets one; most do not need it, since a header the op depends on
//! is better declared on the input (`#[zip(header = "X-Org-Id")]`), where the
//! document can see it.
use std::net::SocketAddr;
use std::sync::{Arc, Mutex, MutexGuard};
use hyper::header::{HeaderMap, HeaderName, HeaderValue};
use crate::Error;
/// Cx is the exchange around an op's input.
pub struct Cx {
method: String,
path: String,
pub(crate) query: Vec<(String, String)>,
pub(crate) params: Vec<(String, String)>,
headers: HeaderMap,
peer: Option<SocketAddr>,
pub(crate) out: Arc<Mutex<Out>>,
}
/// Out is what the handler said about its answer besides the body.
#[derive(Default)]
pub(crate) struct Out {
pub status: Option<u16>,
pub headers: HeaderMap,
/// declared is every trailer name, in the order declared. Over HTTP/1.1 a
/// trailer reaches the client only if its name went out in the head.
pub declared: Vec<HeaderName>,
pub trailers: HeaderMap,
}
impl Cx {
pub(crate) fn new(
method: String,
path: String,
query: Vec<(String, String)>,
params: Vec<(String, String)>,
headers: HeaderMap,
peer: Option<SocketAddr>,
) -> Cx {
Cx {
method,
path,
query,
params,
headers,
peer,
out: Arc::default(),
}
}
pub fn method(&self) -> &str {
&self.method
}
/// path is the address asked for, without its query.
pub fn path(&self) -> &str {
&self.path
}
/// headers is every request header as it arrived.
pub fn headers(&self) -> &HeaderMap {
&self.headers
}
/// header is one request header, if it is there and is text.
pub fn header(&self, name: &str) -> Option<&str> {
self.headers.get(name).and_then(|v| v.to_str().ok())
}
/// peer is the socket's other end. Behind a gateway it is the gateway.
pub fn peer(&self) -> Option<SocketAddr> {
self.peer
}
/// set_status answers with this status instead of 200 (or 204 for an op
/// that answers nothing). A refusal keeps the status it carries.
pub fn set_status(&self, status: u16) -> Result<(), Error> {
if !(200..600).contains(&status) {
return Err(Error::broke(format!("{status} is not a final status")));
}
self.out().status = Some(status);
Ok(())
}
/// set_header sets one answer header, replacing any value it had —
/// including a content type the answer would otherwise carry.
pub fn set_header(&self, name: &str, value: &str) -> Result<(), Error> {
let (name, value) = pair(name, value)?;
self.out().headers.insert(name, value);
Ok(())
}
/// add_header adds one answer header beside any it already has.
pub fn add_header(&self, name: &str, value: &str) -> Result<(), Error> {
let (name, value) = pair(name, value)?;
self.out().headers.append(name, value);
Ok(())
}
/// trailer declares a trailer, and is the handle that sets it.
///
/// The name goes out in the head (`Trailer: name`) and the value after the
/// last chunk of the body, so a stream can report what it only knows once
/// it is done — a token count, a digest. Over HTTP/2 a trailer always
/// arrives. Over HTTP/1.1 it arrives only when the client asked for
/// trailers (`TE: trailers`), because hyper sends them on no other terms;
/// a client that did not ask gets the body alone.
pub fn trailer(&self, name: &str) -> Result<Trailer, Error> {
let name = HeaderName::from_bytes(name.as_bytes())
.map_err(|_| Error::broke(format!("{name:?} is not a header name")))?;
let mut out = self.out();
if !out.declared.contains(&name) {
out.declared.push(name.clone());
}
Ok(Trailer {
name,
out: Arc::clone(&self.out),
})
}
fn out(&self) -> MutexGuard<'_, Out> {
lock(&self.out)
}
}
/// Trailer is one declared trailer. It is `Send` and `'static`, so the stream
/// that produces the body can carry it to the end.
#[derive(Clone)]
pub struct Trailer {
name: HeaderName,
out: Arc<Mutex<Out>>,
}
impl Trailer {
/// set gives the trailer its value. A value set after the body ended is
/// never sent.
pub fn set(&self, value: &str) -> Result<(), Error> {
let value = HeaderValue::from_str(value)
.map_err(|_| Error::broke(format!("{value:?} is not a header value")))?;
lock(&self.out).trailers.insert(self.name.clone(), value);
Ok(())
}
}
/// lock takes the answer's state. Nothing holds it across an await or a call
/// out, so a poisoned lock is one a panic left mid-insert, and the map is still
/// a map.
pub(crate) fn lock(out: &Mutex<Out>) -> MutexGuard<'_, Out> {
out.lock().unwrap_or_else(|p| p.into_inner())
}
fn pair(name: &str, value: &str) -> Result<(HeaderName, HeaderValue), Error> {
let n = HeaderName::from_bytes(name.as_bytes())
.map_err(|_| Error::broke(format!("{name:?} is not a header name")))?;
let v = HeaderValue::from_str(value)
.map_err(|_| Error::broke(format!("{value:?} is not a header value")))?;
Ok((n, v))
}
rust/zip/src/desc.rs
+7 -9
View File
@@ -65,13 +65,14 @@ pub enum Scalar {
Body,
}
/// Wire is a type that says what it is and carries itself over JSON.
/// Wire is a type that says what it is.
///
/// Two halves of one fact. `describe` is what every projection reads — the
/// field names, the URL names, the prose — and `write_json`/`read_json` are the
/// bytes those names describe. They are derived together from one declaration,
/// which is what stops a document describing a field the decoder reads under
/// another name.
/// `describe` is what the binder reads — the field names, the URL names, the
/// headers — and `stated` is the same description as one manifest entry, for
/// the projector. The bytes themselves are serde's: the derive reads the
/// `#[serde(rename)]`, `rename_all` and `skip` a type already carries, so the
/// name a document publishes for a field is the name serde reads it under, and
/// there is one place either is written down.
pub trait Wire: Sized {
fn describe() -> &'static TypeDesc;
@@ -87,7 +88,4 @@ pub trait Wire: Sized {
/// A macro sees only the item it is on, so the graph is walked by the types
/// themselves: each one claims its entry, then asks the types of its fields.
fn reach(into: &mut Vec<(&'static str, &'static str)>);
fn write_json(&self, out: &mut String);
fn read_json(v: &crate::Json) -> Result<Self, crate::Error>;
}
rust/zip/src/error.rs
+20 -9
View File
@@ -42,19 +42,22 @@ impl Error {
}
/// problem is the body a refusal travels in — RFC 9457, the same shape the
/// Go side answers with, so one client reads both.
/// Go side answers with, so one client reads both. Its members are written
/// in name order, which is the order Go's encoder writes a map in.
pub fn problem(&self) -> String {
let mut out = String::from("{\"detail\":");
crate::json::write_str(&mut out, &self.detail);
out.push_str(",\"status\":");
out.push_str(&self.status.to_string());
out.push_str(",\"title\":");
crate::json::write_str(&mut out, crate::http::reason(self.status));
out.push_str(",\"type\":\"about:blank\"}");
out
format!(
"{{\"detail\":{},\"status\":{},\"title\":{},\"type\":\"about:blank\"}}",
quote(&self.detail),
self.status,
quote(reason(self.status)),
)
}
}
fn quote(s: &str) -> String {
serde_json::to_string(s).expect("a string always encodes")
}
impl fmt::Display for Error {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}: {}", self.status, self.detail)
@@ -62,3 +65,11 @@ impl fmt::Display for Error {
}
impl std::error::Error for Error {}
/// reason is the phrase beside a status.
pub fn reason(status: u16) -> &'static str {
hyper::StatusCode::from_u16(status)
.ok()
.and_then(|s| s.canonical_reason())
.unwrap_or("Unknown")
}
rust/zip/src/http.rs
+182 -117
View File
@@ -2,137 +2,202 @@
// Copyright (C) 2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//! The HTTP door.
//! The HTTP door: hyper on tokio.
//!
//! HTTP/1.1 over a socket, one thread per connection, and nothing else — no
//! executor to choose, no runtime to agree on, and therefore no dependency a
//! service inherits from its framework. A handler is a plain function; how a
//! deployment schedules them is the deployment's business.
//! HTTP/1.1 with keep-alive, and HTTP/2 over cleartext (prior knowledge) on the
//! same port: the connection's first bytes say which, so a client never has to
//! be told. A request is answered on its connection's task, and its body goes
//! out as the op produces it — each event of a stream is written and flushed
//! when it is yielded, not when a buffer fills.
//!
//! When a client goes away mid-answer, hyper drops the answer's body, and the
//! op's stream with it: over HTTP/1.1 on the read that sees the connection
//! close, over HTTP/2 on the stream's reset.
use std::io::{BufRead, BufReader, Read, Write};
use std::net::{TcpListener, TcpStream};
use std::sync::Arc;
use std::convert::Infallible;
use std::future::Future;
use std::net::SocketAddr;
use std::pin::Pin;
use std::sync::{Arc, Mutex};
use std::task::{Context, Poll};
use std::time::Duration;
use crate::{App, Error};
use bytes::Bytes;
use futures_util::StreamExt;
use http_body_util::BodyExt;
use hyper::body::{Frame, Incoming, SizeHint};
use hyper::service::service_fn;
use hyper_util::rt::{TokioExecutor, TokioIo, TokioTimer};
use hyper_util::server::conn::auto;
use hyper_util::server::graceful::GracefulShutdown;
use tokio::net::TcpListener;
use tokio::sync::watch;
/// listen serves app over HTTP at addr, and does not return.
pub fn listen(app: Arc<App>, addr: &str) -> std::io::Result<()> {
let door = TcpListener::bind(addr)?;
for conn in door.incoming() {
let Ok(conn) = conn else { continue };
use crate::app::{trailers, Answer, Asked, Payload};
use crate::cx::Out;
use crate::{App, Body, Error};
/// serve answers HTTP on listener until `shutdown` resolves, then stops
/// accepting, lets every connection finish what it is answering, and returns
/// once they have — or once `grace` has passed, dropping what is left.
pub async fn serve<F>(
app: Arc<App>,
listener: TcpListener,
shutdown: F,
grace: Duration,
) -> std::io::Result<()>
where
F: Future<Output = ()> + Send,
{
let mut builder = auto::Builder::new(TokioExecutor::new());
// A timer turns on hyper's header read timeout, which is what keeps a
// client that opens a connection and never finishes a request from
// holding it forever.
builder.http1().timer(TokioTimer::new());
builder.http2().timer(TokioTimer::new());
let graceful = GracefulShutdown::new();
// kill ends every connection still open when the grace runs out: its task
// drops it, and with it whatever answer it was still producing.
let (kill, killed) = watch::channel(false);
tokio::pin!(shutdown);
loop {
let (stream, peer) = tokio::select! {
conn = listener.accept() => match conn {
Ok(c) => c,
// Out of descriptors, or a connection reset before it was
// taken: the listener is fine, and the next accept may not be.
Err(_) => {
tokio::time::sleep(Duration::from_millis(10)).await;
continue;
}
},
() = &mut shutdown => break,
};
let _ = stream.set_nodelay(true);
let app = Arc::clone(&app);
std::thread::spawn(move || {
let _ = serve(&app, conn);
let svc = service_fn(move |req| respond(Arc::clone(&app), peer, req));
let conn = builder
.serve_connection(TokioIo::new(stream), svc)
.into_owned();
let conn = graceful.watch(conn);
let mut killed = killed.clone();
tokio::spawn(async move {
tokio::select! {
_ = conn => {}
_ = killed.wait_for(|k| *k) => {}
}
});
}
drop(listener);
tokio::select! {
() = graceful.shutdown() => {}
() = tokio::time::sleep(grace) => {
let _ = kill.send(true);
}
}
Ok(())
}
/// serve answers every request on one connection, until the peer stops asking.
fn serve(app: &App, conn: TcpStream) -> std::io::Result<()> {
let mut out = conn.try_clone()?;
let mut r = BufReader::new(conn);
loop {
let Some((method, target, headers)) = head(&mut r)? else {
return Ok(());
};
let mut body = Vec::new();
if let Some(n) = length(&headers) {
body.resize(n, 0);
r.read_exact(&mut body)?;
}
let close = headers
.iter()
.any(|(k, v)| k.eq_ignore_ascii_case("connection") && v.eq_ignore_ascii_case("close"));
let answer = app.answer(&method, &target, headers, body);
write(&mut out, answer.status, answer.kind, &answer.body, close)?;
if close {
return Ok(());
}
}
}
/// head reads the request line and the headers, or None at end of stream.
type Head = Option<(String, String, Vec<(String, String)>)>;
fn head(r: &mut BufReader<TcpStream>) -> std::io::Result<Head> {
let mut line = String::new();
if r.read_line(&mut line)? == 0 {
return Ok(None);
}
let mut words = line.split_whitespace();
let (Some(method), Some(target)) = (words.next(), words.next()) else {
return Ok(None);
};
let (method, target) = (method.to_string(), target.to_string());
let mut headers = Vec::new();
loop {
let mut line = String::new();
if r.read_line(&mut line)? == 0 {
break;
}
let line = line.trim_end();
if line.is_empty() {
break;
}
if let Some((k, v)) = line.split_once(':') {
headers.push((k.trim().to_string(), v.trim().to_string()));
}
}
Ok(Some((method, target, headers)))
}
fn length(headers: &[(String, String)]) -> Option<usize> {
headers
.iter()
.find(|(k, _)| k.eq_ignore_ascii_case("content-length"))
.and_then(|(_, v)| v.parse().ok())
}
fn write(
out: &mut TcpStream,
status: u16,
kind: &str,
body: &[u8],
close: bool,
) -> std::io::Result<()> {
let mut head = format!(
"HTTP/1.1 {status} {}\r\nContent-Type: {kind}\r\nContent-Length: {}\r\n",
reason(status),
body.len()
async fn respond(
app: Arc<App>,
peer: SocketAddr,
req: hyper::Request<Incoming>,
) -> Result<hyper::Response<Flow>, Infallible> {
let (parts, incoming) = req.into_parts();
let target = parts
.uri
.path_and_query()
.map(|p| p.as_str())
.unwrap_or("/")
.to_string();
let body = Body::new(
incoming
.into_data_stream()
.map(|c| c.map_err(|e| Error::bad(format!("the request body broke: {e}")))),
);
head.push_str(if close {
"Connection: close\r\n\r\n"
} else {
"\r\n"
});
out.write_all(head.as_bytes())?;
out.write_all(body)?;
out.flush()
let answer = app
.answer(Asked {
method: parts.method.as_str().to_string(),
target,
headers: parts.headers,
peer: Some(peer),
body,
})
.await;
Ok(response(answer))
}
/// reason is the phrase beside a status. Only the ones this framework answers
/// with: a phrase nothing sends is a phrase nobody reads.
pub fn reason(status: u16) -> &'static str {
match status {
200 => "OK",
201 => "Created",
202 => "Accepted",
204 => "No Content",
400 => "Bad Request",
401 => "Unauthorized",
403 => "Forbidden",
404 => "Not Found",
405 => "Method Not Allowed",
409 => "Conflict",
422 => "Unprocessable Content",
500 => "Internal Server Error",
503 => "Service Unavailable",
_ => "OK",
fn response(answer: Answer) -> hyper::Response<Flow> {
let status = hyper::StatusCode::from_u16(answer.status)
.unwrap_or(hyper::StatusCode::INTERNAL_SERVER_ERROR);
let mut res = hyper::Response::new(Flow {
payload: Some(answer.payload),
trailers: answer.trailers,
});
*res.status_mut() = status;
*res.headers_mut() = answer.headers;
res
}
/// Flow is an answer's body as hyper polls it: the bytes, then the trailers.
pub(crate) struct Flow {
payload: Option<Payload>,
trailers: Option<Arc<Mutex<Out>>>,
}
impl Flow {
/// end is the frame after the last chunk: the trailers, if the op set any.
fn end(&mut self) -> Option<Result<Frame<Bytes>, Error>> {
self.payload = None;
let set = trailers(self.trailers.take()?.as_ref())?;
Some(Ok(Frame::trailers(set)))
}
}
/// refuse is what a door writes when it cannot even read the request.
pub fn refuse(e: &Error) -> Vec<u8> {
e.problem().into_bytes()
impl hyper::body::Body for Flow {
type Data = Bytes;
type Error = Error;
fn poll_frame(
mut self: Pin<&mut Self>,
cx: &mut Context<'_>,
) -> Poll<Option<Result<Frame<Bytes>, Error>>> {
let this = &mut *self;
match &mut this.payload {
None => Poll::Ready(None),
Some(Payload::Full(b)) => {
if !b.is_empty() {
return Poll::Ready(Some(Ok(Frame::data(std::mem::take(b)))));
}
Poll::Ready(this.end())
}
Some(Payload::Stream(s)) => match s.poll_next_unpin(cx) {
Poll::Pending => Poll::Pending,
Poll::Ready(Some(Ok(b))) => Poll::Ready(Some(Ok(Frame::data(b)))),
Poll::Ready(Some(Err(e))) => {
this.payload = None;
this.trailers = None;
Poll::Ready(Some(Err(e)))
}
Poll::Ready(None) => Poll::Ready(this.end()),
},
}
}
fn is_end_stream(&self) -> bool {
self.payload.is_none() && self.trailers.is_none()
}
/// size_hint is exact for a body in hand, so it goes out with a length —
/// unless the op declared a trailer, which HTTP/1.1 carries only after a
/// chunked body.
fn size_hint(&self) -> SizeHint {
match &self.payload {
None => SizeHint::with_exact(0),
Some(Payload::Full(b)) if self.trailers.is_none() => {
SizeHint::with_exact(b.len() as u64)
}
_ => SizeHint::default(),
}
}
}
rust/zip/src/json.rs
-371
View File
@@ -1,371 +0,0 @@
// SPDX-License-Identifier: BSD-3-Clause-Eco
// Copyright (C) 2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//! JSON, as far as an API needs it.
//!
//! A value that arrived and a value being written, and nothing between them: no
//! derive of its own to keep in step with [`crate::Wire`], and no second
//! annotation on a type that has already said what its fields are called. The
//! wire name a document publishes is the wire name this reads, because there is
//! one place either is written down.
use crate::Error;
use std::collections::HashMap;
/// Json is a value that arrived.
#[derive(Clone, Debug, PartialEq)]
pub enum Json {
Null,
Bool(bool),
Number(f64),
Text(String),
List(Vec<Json>),
Map(HashMap<String, Json>),
}
impl Json {
/// get is the named member of an object, or None.
pub fn get(&self, name: &str) -> Option<&Json> {
match self {
Json::Map(m) => m.get(name),
_ => None,
}
}
pub fn is_null(&self) -> bool {
matches!(self, Json::Null)
}
/// text is this value as a string, refusing anything else.
pub fn text(&self) -> Result<&str, Error> {
match self {
Json::Text(s) => Ok(s),
_ => Err(Error::bad("expected a string")),
}
}
/// parse reads one value. Trailing bytes are a refusal: a body with a
/// second document in it is a body someone else wrote.
pub fn parse(src: &str) -> Result<Json, Error> {
let b = src.as_bytes();
let mut at = 0;
let v = value(b, &mut at)?;
skip(b, &mut at);
if at != b.len() {
return Err(Error::bad("trailing bytes after the value"));
}
Ok(v)
}
}
fn skip(b: &[u8], at: &mut usize) {
while *at < b.len() && matches!(b[*at], b' ' | b'\t' | b'\n' | b'\r') {
*at += 1;
}
}
fn value(b: &[u8], at: &mut usize) -> Result<Json, Error> {
skip(b, at);
match b.get(*at) {
None => Err(Error::bad("the value ended early")),
Some(b'{') => object(b, at),
Some(b'[') => array(b, at),
Some(b'"') => Ok(Json::Text(string(b, at)?)),
Some(b't') => word(b, at, "true", Json::Bool(true)),
Some(b'f') => word(b, at, "false", Json::Bool(false)),
Some(b'n') => word(b, at, "null", Json::Null),
_ => number(b, at),
}
}
fn word(b: &[u8], at: &mut usize, want: &str, v: Json) -> Result<Json, Error> {
if b.len() >= *at + want.len() && &b[*at..*at + want.len()] == want.as_bytes() {
*at += want.len();
return Ok(v);
}
Err(Error::bad("not a value"))
}
fn object(b: &[u8], at: &mut usize) -> Result<Json, Error> {
*at += 1;
let mut m = HashMap::new();
skip(b, at);
if b.get(*at) == Some(&b'}') {
*at += 1;
return Ok(Json::Map(m));
}
loop {
skip(b, at);
let k = string(b, at)?;
skip(b, at);
if b.get(*at) != Some(&b':') {
return Err(Error::bad("a member needs a colon"));
}
*at += 1;
m.insert(k, value(b, at)?);
skip(b, at);
match b.get(*at) {
Some(b',') => *at += 1,
Some(b'}') => {
*at += 1;
return Ok(Json::Map(m));
}
_ => return Err(Error::bad("the object ended early")),
}
}
}
fn array(b: &[u8], at: &mut usize) -> Result<Json, Error> {
*at += 1;
let mut out = Vec::new();
skip(b, at);
if b.get(*at) == Some(&b']') {
*at += 1;
return Ok(Json::List(out));
}
loop {
out.push(value(b, at)?);
skip(b, at);
match b.get(*at) {
Some(b',') => *at += 1,
Some(b']') => {
*at += 1;
return Ok(Json::List(out));
}
_ => return Err(Error::bad("the list ended early")),
}
}
}
fn string(b: &[u8], at: &mut usize) -> Result<String, Error> {
if b.get(*at) != Some(&b'"') {
return Err(Error::bad("expected a string"));
}
*at += 1;
let mut out = String::new();
while let Some(&c) = b.get(*at) {
*at += 1;
match c {
b'"' => return Ok(out),
b'\\' => {
let e = *b
.get(*at)
.ok_or_else(|| Error::bad("the string ended early"))?;
*at += 1;
match e {
b'"' => out.push('"'),
b'\\' => out.push('\\'),
b'/' => out.push('/'),
b'b' => out.push('\u{8}'),
b'f' => out.push('\u{c}'),
b'n' => out.push('\n'),
b'r' => out.push('\r'),
b't' => out.push('\t'),
b'u' => {
let n = hex(b, at)?;
// A surrogate pair is two escapes for one character, so
// the low half is read here rather than left to become a
// replacement character downstream.
let c = if (0xD800..0xDC00).contains(&n) {
if b.get(*at) == Some(&b'\\') && b.get(*at + 1) == Some(&b'u') {
*at += 2;
let lo = hex(b, at)?;
0x10000 + ((n - 0xD800) << 10) + (lo - 0xDC00)
} else {
n
}
} else {
n
};
out.push(char::from_u32(c).unwrap_or('\u{fffd}'));
}
_ => return Err(Error::bad("unknown escape")),
}
}
c if c < 0x80 => out.push(c as char),
_ => {
// A multi-byte character: take its bytes whole rather than
// one at a time, since a byte of one is not a character.
let start = *at - 1;
let mut end = *at;
while end < b.len() && b[end] & 0xC0 == 0x80 {
end += 1;
}
out.push_str(
std::str::from_utf8(&b[start..end]).map_err(|_| Error::bad("not utf-8"))?,
);
*at = end;
}
}
}
Err(Error::bad("the string ended early"))
}
fn hex(b: &[u8], at: &mut usize) -> Result<u32, Error> {
if *at + 4 > b.len() {
return Err(Error::bad("a \\u escape is four digits"));
}
let s = std::str::from_utf8(&b[*at..*at + 4]).map_err(|_| Error::bad("not utf-8"))?;
*at += 4;
u32::from_str_radix(s, 16).map_err(|_| Error::bad("a \\u escape is four hex digits"))
}
fn number(b: &[u8], at: &mut usize) -> Result<Json, Error> {
let start = *at;
while let Some(&c) = b.get(*at) {
if c.is_ascii_digit() || matches!(c, b'-' | b'+' | b'.' | b'e' | b'E') {
*at += 1;
} else {
break;
}
}
let s = std::str::from_utf8(&b[start..*at]).map_err(|_| Error::bad("not utf-8"))?;
s.parse::<f64>()
.map(Json::Number)
.map_err(|_| Error::bad("not a number"))
}
// ---- writing ---------------------------------------------------------------
/// write_str appends one JSON string.
pub fn write_str(out: &mut String, s: &str) {
out.push('"');
for c in s.chars() {
match c {
'"' => out.push_str("\\\""),
'\\' => out.push_str("\\\\"),
'\n' => out.push_str("\\n"),
'\r' => out.push_str("\\r"),
'\t' => out.push_str("\\t"),
c if (c as u32) < 0x20 => out.push_str(&format!("\\u{:04x}", c as u32)),
c => out.push(c),
}
}
out.push('"');
}
/// write_float appends a number. A value JSON cannot hold is written as null,
/// which is the only true thing to say about it.
pub fn write_float(out: &mut String, v: f64) {
if !v.is_finite() {
out.push_str("null");
return;
}
if v == v.trunc() && v.abs() < 1e15 {
out.push_str(&format!("{}", v as i64));
return;
}
out.push_str(&format!("{v}"));
}
/// write_bytes appends a byte string, base64 as JSON carries one.
pub fn write_bytes(out: &mut String, b: &[u8]) {
const A: &[u8; 64] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
let mut s = String::with_capacity(b.len().div_ceil(3) * 4);
for c in b.chunks(3) {
let n = (c[0] as u32) << 16
| (*c.get(1).unwrap_or(&0) as u32) << 8
| (*c.get(2).unwrap_or(&0) as u32);
s.push(A[(n >> 18 & 63) as usize] as char);
s.push(A[(n >> 12 & 63) as usize] as char);
s.push(if c.len() > 1 {
A[(n >> 6 & 63) as usize] as char
} else {
'='
});
s.push(if c.len() > 2 {
A[(n & 63) as usize] as char
} else {
'='
});
}
write_str(out, &s);
}
/// read_bytes is base64, back.
pub fn read_bytes(s: &str) -> Result<Vec<u8>, Error> {
let mut out = Vec::new();
let mut buf = 0u32;
let mut have = 0;
for c in s.bytes() {
let v = match c {
b'A'..=b'Z' => c - b'A',
b'a'..=b'z' => c - b'a' + 26,
b'0'..=b'9' => c - b'0' + 52,
b'+' => 62,
b'/' => 63,
b'=' | b'\n' | b'\r' => continue,
_ => return Err(Error::bad("not base64")),
} as u32;
buf = buf << 6 | v;
have += 6;
if have >= 8 {
have -= 8;
out.push((buf >> have) as u8);
}
}
Ok(out)
}
// ---- what a derived reader asks -------------------------------------------
//
// A field the body did not carry reads as its zero rather than as a refusal:
// that is what every other decoder on this wire does, and a required field is
// refused by validation, which is one rule in one place.
pub fn as_text(v: Option<&Json>) -> Result<String, Error> {
match v {
Some(Json::Text(s)) => Ok(s.clone()),
Some(Json::Null) | None => Ok(String::new()),
Some(_) => Err(Error::bad("expected a string")),
}
}
pub fn as_bool(v: Option<&Json>) -> Result<bool, Error> {
match v {
Some(Json::Bool(b)) => Ok(*b),
Some(Json::Null) | None => Ok(false),
Some(_) => Err(Error::bad("expected a boolean")),
}
}
pub fn as_number(v: Option<&Json>) -> Result<f64, Error> {
match v {
Some(Json::Number(n)) => Ok(*n),
Some(Json::Null) | None => Ok(0.0),
Some(_) => Err(Error::bad("expected a number")),
}
}
pub fn as_bytes(v: Option<&Json>) -> Result<Vec<u8>, Error> {
match v {
Some(Json::Text(s)) => read_bytes(s),
Some(Json::Null) | None => Ok(Vec::new()),
Some(_) => Err(Error::bad("expected base64")),
}
}
pub fn as_list(v: Option<&Json>) -> Result<&[Json], Error> {
match v {
Some(Json::List(l)) => Ok(l),
Some(Json::Null) | None => Ok(&[]),
Some(_) => Err(Error::bad("expected a list")),
}
}
/// EMPTY is what a map field reads as when the body did not carry one.
static EMPTY: std::sync::OnceLock<HashMap<String, Json>> = std::sync::OnceLock::new();
pub fn as_map(v: Option<&Json>) -> Result<&HashMap<String, Json>, Error> {
match v {
Some(Json::Map(m)) => Ok(m),
Some(Json::Null) | None => Ok(EMPTY.get_or_init(HashMap::new)),
Some(_) => Err(Error::bad("expected an object")),
}
}
/// as_wire reads a value that describes itself.
pub fn as_wire<T: crate::Wire>(v: Option<&Json>) -> Result<T, Error> {
T::read_json(v.unwrap_or(&Json::Null))
}
rust/zip/src/lib.rs
+92 -31
View File
@@ -10,25 +10,41 @@
//! above it. Nothing is declared twice.
//!
//! ```ignore
//! #[zip::ops(app = "info", title = "Lux node info", version = "1.0.0")]
//! impl Info {
//! /// Bootstrapped reports whether a chain has finished bootstrapping on this node.
//! #[zip::ops(app = "chat", title = "Chat", version = "1.0.0")]
//! impl Chat {
//! /// Echo answers the text it was sent.
//! ///
//! /// Example: {"chain": "X"}
//! /// Response: {"isBootstrapped": true}
//! #[get("/chain/bootstrapped")]
//! fn bootstrapped(&self, arg: &BootstrappedArgs) -> Result<Bootstrapped, zip::Error> {
//! Ok(Bootstrapped { is_bootstrapped: true })
//! /// Example: {"text": "hi"}
//! /// Response: {"text": "hi"}
//! #[post("/v1/echo")]
//! async fn echo(&self, arg: &Ping) -> Result<Pong, zip::Error> {
//! Ok(Pong { text: arg.text.clone() })
//! }
//!
//! /// Complete streams a completion, one event a token.
//! #[post("/v1/complete")]
//! async fn complete(&self, cx: &zip::Cx, arg: &Prompt) -> Result<zip::Sse, zip::Error> {
//! let tokens = cx.trailer("x-tokens")?;
//! Ok(zip::Sse::new(self.model.stream(arg, tokens)).keep(Duration::from_secs(15)))
//! }
//! }
//! ```
//!
//! An op is an `async fn` taking `&self`, optionally `&zip::Cx` (the request's
//! headers, and the answer's status, headers and trailers), and at most one
//! input: a type deriving [Wire] and serde's `Deserialize`, bound from the body,
//! the query, the path and the headers it names, or a [Body], the request body
//! as it arrives. It answers `Result<T, zip::Error>` where T is a type deriving
//! [Wire] and `Serialize` (one JSON value), `()` (204), [Sse] (an event stream)
//! or [Body] (a byte stream).
//!
//! # Where the documents come from
//!
//! The macro writes what it read — the ops, the types, the prose — as the crate
//! compiles. `zipc` projects that description into the OpenAPI document, the MCP
//! tool list, the CLI and the .zap schema, and `zapgen` compiles the schema into
//! the zero-copy accessors this crate's own wire is built from.
//! tool list, the CLI and the .zap schema. A streamed answer is part of what it
//! read: the manifest says `"stream": "sse"` or `"bytes"`, and the document
//! publishes `text/event-stream` or `application/octet-stream` for it.
//!
//! The projector is one program for all three languages, which is the whole
//! point: an operation carries one id, one summary and one schema whether it was
@@ -37,26 +53,31 @@
//! are Rust here, all the way down. No Go runs in a service built with this.
mod app;
mod bind;
mod body;
mod cx;
mod desc;
mod error;
pub mod json;
pub mod base64;
pub mod http;
pub mod zap;
pub mod zaphttp;
// zap is the ZAP wire and wire is the ZAP-HTTP frames, both generated from
// wire.zap by zapgen. They are the only wire code here, and none of it is
// written by hand.
include!(concat!(env!("OUT_DIR"), "/generated.rs"));
pub use app::{bind, Answer, App, Call, Input, Op};
pub use app::{App, Op};
pub use bind::{bind, value, MAX_BODY};
pub use body::{Body, Event, Reply, Sse};
pub use cx::{Cx, Trailer};
pub use desc::{FieldDesc, Scalar, TypeDesc, Wire};
pub use error::Error;
pub use json::Json;
/// The two macros. `ops` declares a service; `Wire` makes a type say what it is.
pub use zip_macros::{ops, Wire};
/// serde is the serde zip reads and writes with, so generated code names one
/// serde whatever a service depends on.
pub use serde;
/// manifest is this service's whole description, as one document.
///
/// The ops half comes from the macro that read the impl block; the types half
@@ -83,33 +104,73 @@ pub fn manifest(ops: &str, types: &[(&str, &str)]) -> String {
out
}
/// listen serves an app on every address given, and does not return.
/// listen serves an app on every address given until `shutdown` resolves,
/// then drains both doors for at most `grace` and returns.
///
/// One verb, whatever the transport: an address decides which door it is, so a
/// binary states all of its listeners the same way and a deployment changes one
/// without touching the service.
///
/// ```text
/// info serve --zap :9653 --http :8000
/// info --zap :9653 --http :8000
/// ```
pub fn listen(app: App, http: Option<&str>, zap: Option<&str>) -> std::io::Result<()> {
pub async fn listen<F>(
app: App,
http: Option<&str>,
zap: Option<&str>,
shutdown: F,
grace: std::time::Duration,
) -> std::io::Result<()>
where
F: std::future::Future<Output = ()> + Send,
{
let app = std::sync::Arc::new(app);
let mut doors = Vec::new();
let (stop, stopped) = tokio::sync::watch::channel(false);
let mut doors = tokio::task::JoinSet::new();
let door = |mut stopped: tokio::sync::watch::Receiver<bool>| async move {
let _ = stopped.wait_for(|s| *s).await;
};
if let Some(addr) = zap {
let listener = tokio::net::TcpListener::bind(addr).await?;
let app = std::sync::Arc::clone(&app);
let addr = addr.to_string();
doors.push(std::thread::spawn(move || zaphttp::listen(app, &addr)));
let stopped = door(stopped.clone());
doors.spawn(async move { zaphttp::serve(app, listener, stopped, grace).await });
}
if let Some(addr) = http {
let listener = tokio::net::TcpListener::bind(addr).await?;
let app = std::sync::Arc::clone(&app);
let addr = addr.to_string();
doors.push(std::thread::spawn(move || http::listen(app, &addr)));
let stopped = door(stopped.clone());
doors.spawn(async move { http::serve(app, listener, stopped, grace).await });
}
for door in doors {
match door.join() {
Ok(r) => r?,
Err(_) => return Err(std::io::Error::other("a door stopped")),
}
shutdown.await;
let _ = stop.send(true);
while let Some(done) = doors.join_next().await {
done.map_err(std::io::Error::other)??;
}
Ok(())
}
/// signal resolves when the process is asked to stop: SIGINT or SIGTERM, the
/// second being what a pod is sent.
pub async fn signal() {
let interrupt = tokio::signal::ctrl_c();
#[cfg(unix)]
{
let mut term =
match tokio::signal::unix::signal(tokio::signal::unix::SignalKind::terminate()) {
Ok(t) => t,
Err(_) => {
let _ = interrupt.await;
return;
}
};
tokio::select! {
_ = interrupt => {}
_ = term.recv() => {}
}
}
#[cfg(not(unix))]
{
let _ = interrupt.await;
}
}
rust/zip/src/zap.rs
+385
View File
@@ -0,0 +1,385 @@
// SPDX-License-Identifier: BSD-3-Clause-Eco
// Copyright (C) 2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//! The ZAP-HTTP frames: HTTP semantics on the ZAP wire, as zap-proto/http
//! (Go) writes them.
//!
//! A frame is a ZAP message: a 16-byte header (magic `ZAP\0`, version 1, flags
//! carrying the frame type in their upper byte, the root object's offset, the
//! message's size), then the root object's fixed slots, then the bytes the
//! slots point at. A text or bytes slot is eight bytes, a little-endian offset
//! relative to the slot and a length; an empty one is all zero.
//!
//! | frame | type | slots |
//! |---|---|---|
//! | request | 0x01 | method @0, target @8, proto @16, headers @24, body @32, trailer @40 |
//! | response | 0x02 | status u16 @0, reason @8, proto @16, headers @24, body @32, trailer @40 |
//! | streamed head | 0x03 | a response with no body and no trailer |
//! | data | 0x04 | one chunk of the body @0 |
//! | end | 0x05 | the trailer pairs @0 |
//!
//! Headers and trailers ride as a count, then that many length-prefixed name
//! and value runs, every integer little-endian. On a connection each frame is
//! preceded by its length as a BIG-endian u32, as zap-proto/http's transport
//! writes it.
//!
//! The offsets are zap-proto/http's (wire.go). No zapgen emits Rust, so they
//! are written here once and held to frames Go wrote, byte for byte, by the
//! tests at the bottom.
use bytes::Bytes;
pub const FRAME_REQUEST: u16 = 0x01;
pub const FRAME_RESPONSE: u16 = 0x02;
pub const FRAME_HEAD: u16 = 0x03;
pub const FRAME_DATA: u16 = 0x04;
pub const FRAME_END: u16 = 0x05;
const HEADER: usize = 16;
const MAGIC: &[u8; 4] = b"ZAP\0";
const VERSION: u16 = 1;
/// The protocol an answer names, which is what fasthttp reports for one.
const PROTO: &str = "HTTP/1.1";
/// Request is a request frame, read where it lies.
pub struct Request<'a> {
frame: &'a [u8],
root: usize,
size: usize,
}
impl<'a> Request<'a> {
pub fn wrap(frame: &'a [u8]) -> Option<Request<'a>> {
let (root, size) = root(frame, FRAME_REQUEST)?;
Some(Request { frame, root, size })
}
pub fn method(&self) -> &'a str {
text(var(self.frame, self.size, self.root, 0))
}
pub fn target(&self) -> &'a str {
text(var(self.frame, self.size, self.root, 8))
}
pub fn headers(&self) -> &'a [u8] {
var(self.frame, self.size, self.root, 24)
}
pub fn body(&self) -> &'a [u8] {
var(self.frame, self.size, self.root, 32)
}
}
/// Response is a response frame or a streamed head, read where it lies.
pub struct Response<'a> {
frame: &'a [u8],
root: usize,
size: usize,
}
impl<'a> Response<'a> {
pub fn wrap(frame: &'a [u8]) -> Option<Response<'a>> {
let (root, size) = root(frame, FRAME_RESPONSE).or_else(|| root(frame, FRAME_HEAD))?;
Some(Response { frame, root, size })
}
pub fn status(&self) -> u16 {
let at = self.root;
if at + 2 > self.size {
return 0;
}
u16::from_le_bytes([self.frame[at], self.frame[at + 1]])
}
pub fn headers(&self) -> &'a [u8] {
var(self.frame, self.size, self.root, 24)
}
pub fn body(&self) -> &'a [u8] {
var(self.frame, self.size, self.root, 32)
}
pub fn trailer(&self) -> &'a [u8] {
var(self.frame, self.size, self.root, 40)
}
}
/// kind is a frame's type, from its header alone.
pub fn kind(frame: &[u8]) -> Option<u16> {
if frame.len() < HEADER || &frame[0..4] != MAGIC {
return None;
}
Some(u16::from_le_bytes([frame[6], frame[7]]) >> 8)
}
/// chunk is a data or end frame's one slot.
pub fn chunk(frame: &[u8]) -> Option<&[u8]> {
let (root, size) = root(frame, FRAME_DATA).or_else(|| root(frame, FRAME_END))?;
Some(var(frame, size, root, 0))
}
/// root checks a frame's header — magic, version, size, type — and answers its
/// root object's offset and the message's length. Bytes past the size are not
/// the message's.
fn root(frame: &[u8], want: u16) -> Option<(usize, usize)> {
if kind(frame)? != want {
return None;
}
let version = u16::from_le_bytes([frame[4], frame[5]]);
if version != 1 && version != 2 {
return None;
}
let size = u32_at(frame, 12) as usize;
if size < HEADER || size > frame.len() {
return None;
}
let root = u32_at(frame, 8) as usize;
if root < HEADER || root >= size {
return None;
}
Some((root, size))
}
/// var is the bytes one slot points at, or none when the slot is empty or
/// points anywhere it may not: before the data, or past the message.
fn var(frame: &[u8], size: usize, root: usize, field: usize) -> &[u8] {
let slot = root + field;
if slot + 8 > size {
return &[];
}
let rel = u32_at(frame, slot) as usize;
if rel == 0 {
return &[];
}
let len = u32_at(frame, slot + 4) as usize;
let Some(at) = slot.checked_add(rel) else {
return &[];
};
match at.checked_add(len) {
Some(end) if at >= HEADER && end <= size => &frame[at..end],
_ => &[],
}
}
fn text(b: &[u8]) -> &str {
std::str::from_utf8(b).unwrap_or("")
}
fn u32_at(b: &[u8], at: usize) -> u32 {
u32::from_le_bytes([b[at], b[at + 1], b[at + 2], b[at + 3]])
}
/// pairs walks a header block. Every name and value is a slice of the frame; a
/// block that runs short ends where it ran short.
pub fn pairs(raw: &[u8]) -> Vec<(&[u8], &[u8])> {
let mut out = Vec::new();
if raw.len() < 4 {
return out;
}
let count = u32_at(raw, 0) as usize;
let mut at = 4;
for _ in 0..count {
let Some(name) = run(raw, &mut at) else { break };
let Some(value) = run(raw, &mut at) else {
break;
};
out.push((name, value));
}
out
}
fn run<'a>(raw: &'a [u8], at: &mut usize) -> Option<&'a [u8]> {
if *at + 4 > raw.len() {
return None;
}
let n = u32_at(raw, *at) as usize;
*at += 4;
let end = at.checked_add(n)?;
if end > raw.len() {
return None;
}
let s = &raw[*at..end];
*at = end;
Some(s)
}
/// block writes header pairs as a header block; no pairs is no block.
pub fn block<'a>(pairs: impl IntoIterator<Item = (&'a [u8], &'a [u8])>) -> Vec<u8> {
let mut out = vec![0, 0, 0, 0];
let mut n = 0u32;
for (name, value) in pairs {
out.extend_from_slice(&(name.len() as u32).to_le_bytes());
out.extend_from_slice(name);
out.extend_from_slice(&(value.len() as u32).to_le_bytes());
out.extend_from_slice(value);
n += 1;
}
if n == 0 {
return Vec::new();
}
out[0..4].copy_from_slice(&n.to_le_bytes());
out
}
/// Builder lays one frame out: the header, the fixed slots, then each field's
/// bytes in the order they are put.
struct Builder {
buf: Vec<u8>,
}
impl Builder {
fn new(kind: u16, slots: usize, room: usize) -> Builder {
let mut buf = Vec::with_capacity(HEADER + slots + room);
buf.extend_from_slice(MAGIC);
buf.extend_from_slice(&VERSION.to_le_bytes());
buf.extend_from_slice(&(kind << 8).to_le_bytes());
buf.extend_from_slice(&(HEADER as u32).to_le_bytes());
buf.extend_from_slice(&0u32.to_le_bytes());
buf.resize(HEADER + slots, 0);
Builder { buf }
}
fn u16(&mut self, field: usize, v: u16) {
let at = HEADER + field;
self.buf[at..at + 2].copy_from_slice(&v.to_le_bytes());
}
fn put(&mut self, field: usize, data: &[u8]) {
if data.is_empty() {
return;
}
let slot = HEADER + field;
let rel = (self.buf.len() - slot) as u32;
self.buf.extend_from_slice(data);
self.buf[slot..slot + 4].copy_from_slice(&rel.to_le_bytes());
self.buf[slot + 4..slot + 8].copy_from_slice(&(data.len() as u32).to_le_bytes());
}
fn finish(mut self) -> Vec<u8> {
let size = self.buf.len() as u32;
self.buf[12..16].copy_from_slice(&size.to_le_bytes());
self.buf
}
}
/// request builds a request frame: what a client sends.
pub fn request(method: &str, target: &str, headers: &[u8], body: &[u8]) -> Vec<u8> {
let mut b = Builder::new(
FRAME_REQUEST,
48,
method.len() + target.len() + headers.len() + body.len() + 8,
);
b.put(0, method.as_bytes());
b.put(8, target.as_bytes());
b.put(16, PROTO.as_bytes());
b.put(24, headers);
b.put(32, body);
b.finish()
}
/// response builds an answer frame, whole.
pub fn response(status: u16, headers: &[u8], body: &[u8], trailer: &[u8]) -> Vec<u8> {
let mut b = Builder::new(
FRAME_RESPONSE,
48,
headers.len() + body.len() + trailer.len() + 32,
);
b.u16(0, status);
b.put(8, crate::error::reason(status).as_bytes());
b.put(16, PROTO.as_bytes());
b.put(24, headers);
b.put(32, body);
b.put(40, trailer);
b.finish()
}
/// head builds the opening frame of a streamed answer: its status and headers.
pub fn head(status: u16, headers: &[u8]) -> Vec<u8> {
let mut b = Builder::new(FRAME_HEAD, 48, headers.len() + 32);
b.u16(0, status);
b.put(8, crate::error::reason(status).as_bytes());
b.put(16, PROTO.as_bytes());
b.put(24, headers);
b.finish()
}
/// data builds one chunk of a streamed body.
pub fn data(chunk: &Bytes) -> Vec<u8> {
let mut b = Builder::new(FRAME_DATA, 8, chunk.len());
b.put(0, chunk);
b.finish()
}
/// end builds the frame that ends a streamed body, carrying its trailers.
pub fn end(trailer: &[u8]) -> Vec<u8> {
let mut b = Builder::new(FRAME_END, 8, trailer.len());
b.put(0, trailer);
b.finish()
}
#[cfg(test)]
mod tests {
use super::*;
// What zap-proto/http v0.3.12 writes (MarshalRequest, MarshalResponse,
// MarshalResponseHead, MarshalData, MarshalEnd) for the same messages.
const GO_REQUEST: &str = "5a41500001000001100000007d00000030000000040000002c0000000c00000030000000080000003000000018000000400000000d0000000000000000000000504f53542f76312f6563686f3f783d31485454502f312e310100000008000000582d4f72672d49640400000061636d657b2274657874223a226869227d";
const GO_RESPONSE: &str = "5a41500001000002100000007f000000c800000000000000280000000200000022000000080000002200000028000000420000000d00000000000000000000004f4b485454502f312e31010000000c000000436f6e74656e742d54797065100000006170706c69636174696f6e2f6a736f6e7b2274657874223a226869227d";
const GO_HEAD: &str = "5a415000010000031000000073000000c800000000000000280000000200000022000000080000002200000029000000000000000000000000000000000000004f4b485454502f312e31010000000c000000436f6e74656e742d5479706511000000746578742f6576656e742d73747265616d";
const GO_DATA: &str = "5a4150000100000410000000210000000800000009000000646174613a20310a0a";
const GO_END: &str = "5a4150000100000510000000180000000000000000000000";
fn hex(b: &[u8]) -> String {
b.iter().map(|x| format!("{x:02x}")).collect()
}
fn unhex(s: &str) -> Vec<u8> {
(0..s.len())
.step_by(2)
.map(|i| u8::from_str_radix(&s[i..i + 2], 16).unwrap())
.collect()
}
#[test]
fn frames_are_the_ones_go_writes() {
let h = block([(&b"X-Org-Id"[..], &b"acme"[..])]);
assert_eq!(
hex(&request("POST", "/v1/echo?x=1", &h, br#"{"text":"hi"}"#)),
GO_REQUEST
);
let h = block([(&b"Content-Type"[..], &b"application/json"[..])]);
assert_eq!(
hex(&response(200, &h, br#"{"text":"hi"}"#, &[])),
GO_RESPONSE
);
let h = block([(&b"Content-Type"[..], &b"text/event-stream"[..])]);
assert_eq!(hex(&head(200, &h)), GO_HEAD);
assert_eq!(hex(&data(&Bytes::from_static(b"data: 1\n\n"))), GO_DATA);
assert_eq!(hex(&end(&[])), GO_END);
}
#[test]
fn a_go_request_reads() {
let f = unhex(GO_REQUEST);
let r = Request::wrap(&f).unwrap();
assert_eq!(r.method(), "POST");
assert_eq!(r.target(), "/v1/echo?x=1");
assert_eq!(pairs(r.headers()), vec![(&b"X-Org-Id"[..], &b"acme"[..])]);
assert_eq!(r.body(), br#"{"text":"hi"}"#);
assert!(Response::wrap(&f).is_none(), "a request is not an answer");
let f = unhex(GO_HEAD);
assert_eq!(Response::wrap(&f).unwrap().status(), 200);
assert_eq!(chunk(&unhex(GO_DATA)).unwrap(), b"data: 1\n\n");
assert_eq!(kind(&unhex(GO_END)), Some(FRAME_END));
}
#[test]
fn a_crooked_frame_reads_as_nothing() {
let mut f = unhex(GO_REQUEST);
// A slot pointing past the message.
f[HEADER + 8] = 0xff;
let r = Request::wrap(&f).unwrap();
assert_eq!(r.target(), "");
assert!(Request::wrap(&f[..10]).is_none());
let mut g = unhex(GO_REQUEST);
g[0] = b'X';
assert!(Request::wrap(&g).is_none());
// A header block whose count overstates what it holds.
assert!(pairs(&[9, 0, 0, 0, 1, 0, 0, 0]).is_empty());
}
}
rust/zip/src/zaphttp.rs
+157 -131
View File
@@ -4,164 +4,190 @@
//! The ZAP door.
//!
//! The same requests, carried as ZAP frames instead of as text. A frame is one
//! message: a four-byte length, then a ZAP buffer whose header flags say
//! whether it is a request or an answer, and whose root object holds the method,
//! the target, the headers and the body at fixed offsets.
//!
//! Nothing here decodes. A field is a bounds-checked look at bytes already in
//! hand — `target()` is a `&str` INTO the frame — so a request arrives without
//! being copied into a shape first. What is built is built once, into one
//! buffer, by the builder the generator ships.
//!
//! The offsets are the ones the ZAP-HTTP schema states (wire.zap), and the
//! accessors that read them are generated from it. There is no hand-written
//! wire in this file.
//! The same requests, carried as ZAP frames instead of as text (see [crate::zap]
//! for the frames). A connection carries one exchange at a time: a request
//! frame in, then either one response frame or — for a streamed answer — a
//! head, a data frame per chunk as the op produces it, and an end frame with
//! the trailers. That is zap-proto/http's own shape, so a Go client reads a
//! Rust stream chunk by chunk, and a Go server's stream reads here the same way.
use std::io::{Read, Write};
use std::net::{TcpListener, TcpStream};
use std::future::Future;
use std::net::SocketAddr;
use std::sync::Arc;
use std::time::Duration;
use crate::wire::{self, Request};
use crate::{zap, App};
use bytes::Bytes;
use futures_util::StreamExt;
use hyper::header::{self, HeaderMap, HeaderName, HeaderValue};
use tokio::io::{AsyncReadExt, AsyncWriteExt, BufReader};
use tokio::net::{TcpListener, TcpStream};
use tokio::sync::{mpsc, watch};
/// The frame type, carried in the ZAP header's flags. A request and an answer
/// are the two shapes this door speaks.
pub const FRAME_REQUEST: u16 = 0x01;
pub const FRAME_RESPONSE: u16 = 0x02;
use crate::app::{trailers, Answer, Asked, Payload};
use crate::{zap, App, Body, Error};
/// The largest frame this door will read. A length prefix is a promise from
/// whoever sent it, and a door that believes one has handed the peer a way to
/// ask for all the memory there is.
/// MAX_FRAME is the largest frame this door reads. A length prefix is a
/// promise from whoever sent it, and a door that believes one has handed the
/// peer a way to ask for all the memory there is.
pub const MAX_FRAME: usize = 32 << 20;
/// listen serves app over ZAP at addr, and does not return.
pub fn listen(app: Arc<App>, addr: &str) -> std::io::Result<()> {
let door = TcpListener::bind(addr)?;
for conn in door.incoming() {
let Ok(conn) = conn else { continue };
/// serve answers ZAP on listener until `shutdown` resolves, then stops
/// accepting; a connection finishes the exchange it is in and closes. It
/// returns when every connection has, or once `grace` has passed, ending
/// those still open.
pub async fn serve<F>(
app: Arc<App>,
listener: TcpListener,
shutdown: F,
grace: Duration,
) -> std::io::Result<()>
where
F: Future<Output = ()> + Send,
{
let (stop, stopped) = watch::channel(false);
// kill ends every connection still open when the grace runs out.
let (kill, killed) = watch::channel(false);
// Every connection holds a sender; the receiver ends when the last one has
// gone, which is how the drain knows it is done without counting.
let (live, mut gone) = mpsc::channel::<()>(1);
tokio::pin!(shutdown);
loop {
let (stream, peer) = tokio::select! {
conn = listener.accept() => match conn {
Ok(c) => c,
Err(_) => {
tokio::time::sleep(Duration::from_millis(10)).await;
continue;
}
},
() = &mut shutdown => break,
};
let _ = stream.set_nodelay(true);
let app = Arc::clone(&app);
std::thread::spawn(move || {
let _ = serve(&app, conn);
let stopped = stopped.clone();
let mut killed = killed.clone();
let live = live.clone();
tokio::spawn(async move {
tokio::select! {
_ = connection(&app, stream, peer, stopped) => {}
_ = killed.wait_for(|k| *k) => {}
}
drop(live);
});
}
drop(listener);
let _ = stop.send(true);
drop(live);
if tokio::time::timeout(grace, gone.recv()).await.is_err() {
let _ = kill.send(true);
}
Ok(())
}
fn serve(app: &App, mut conn: TcpStream) -> std::io::Result<()> {
let mut out = conn.try_clone()?;
async fn connection(
app: &App,
stream: TcpStream,
peer: SocketAddr,
mut stopped: watch::Receiver<bool>,
) -> std::io::Result<()> {
let (read, mut write) = stream.into_split();
let mut read = BufReader::new(read);
loop {
let Some(frame) = read_frame(&mut conn)? else {
return Ok(());
let frame = tokio::select! {
f = read_frame(&mut read) => f?,
_ = stopped.wait_for(|s| *s) => return Ok(()),
};
let answer = match Request::wrap(&frame) {
Ok(r) => {
let headers = pairs(r.headers());
app.answer(r.method(), r.target(), headers, r.body().to_vec())
let Some(frame) = frame else { return Ok(()) };
let answer = match zap::Request::wrap(&frame) {
Some(r) => {
let mut headers = HeaderMap::new();
for (name, value) in zap::pairs(r.headers()) {
if let (Ok(n), Ok(v)) =
(HeaderName::from_bytes(name), HeaderValue::from_bytes(value))
{
headers.append(n, v);
}
}
let body = Body::full(Bytes::copy_from_slice(r.body()));
app.answer(Asked {
method: r.method().to_string(),
target: r.target().to_string(),
headers,
peer: Some(peer),
body,
})
.await
}
Err(_) => crate::Answer {
status: 400,
kind: "application/problem+json",
body: crate::http::refuse(&crate::Error::bad("malformed request frame")),
},
None => Answer::refusal(&Error::bad("malformed request frame"), HeaderMap::new()),
};
let body = build(answer.status, answer.kind, &answer.body);
out.write_all(&(body.len() as u32).to_le_bytes())?;
out.write_all(&body)?;
out.flush()?;
write_answer(&mut write, answer).await?;
}
}
/// read_frame takes one length-prefixed frame off the connection.
fn read_frame(conn: &mut TcpStream) -> std::io::Result<Option<Vec<u8>>> {
/// write_answer writes one answer: one frame for a body in hand, a head, data
/// frames and an end for a stream. A stream that fails ends the connection
/// without an end frame, which is what tells the peer the answer is cut.
async fn write_answer<W: AsyncWriteExt + Unpin>(w: &mut W, answer: Answer) -> std::io::Result<()> {
// Trailer is the frame's own slot on this wire, never a header.
let headers = block(&answer.headers, true);
match answer.payload {
Payload::Full(ref body) => {
let t = answer
.trailers
.as_deref()
.and_then(trailers)
.map(|t| block(&t, false))
.unwrap_or_default();
send(w, &zap::response(answer.status, &headers, body, &t)).await
}
Payload::Stream(mut chunks) => {
send(w, &zap::head(answer.status, &headers)).await?;
while let Some(chunk) = chunks.next().await {
match chunk {
Ok(c) if c.is_empty() => {}
Ok(c) => send(w, &zap::data(&c)).await?,
Err(e) => return Err(std::io::Error::other(e)),
}
}
let t = answer
.trailers
.as_deref()
.and_then(trailers)
.map(|t| block(&t, false))
.unwrap_or_default();
send(w, &zap::end(&t)).await
}
}
}
fn block(h: &HeaderMap, head: bool) -> Vec<u8> {
zap::block(
h.iter()
.filter(|(n, _)| !head || **n != header::TRAILER)
.map(|(n, v)| (n.as_str().as_bytes(), v.as_bytes())),
)
}
async fn send<W: AsyncWriteExt + Unpin>(w: &mut W, frame: &[u8]) -> std::io::Result<()> {
w.write_all(&(frame.len() as u32).to_be_bytes()).await?;
w.write_all(frame).await?;
w.flush().await
}
/// read_frame takes one length-prefixed frame off the connection, or None at
/// its end. A frame of no bytes or past MAX_FRAME ends the connection.
pub async fn read_frame<R: AsyncReadExt + Unpin>(r: &mut R) -> std::io::Result<Option<Vec<u8>>> {
let mut head = [0u8; 4];
match conn.read_exact(&mut head) {
Ok(()) => {}
match r.read_exact(&mut head).await {
Ok(_) => {}
Err(e) if e.kind() == std::io::ErrorKind::UnexpectedEof => return Ok(None),
Err(e) => return Err(e),
}
let n = u32::from_le_bytes(head) as usize;
let n = u32::from_be_bytes(head) as usize;
if n == 0 || n > MAX_FRAME {
return Ok(None);
}
let mut frame = vec![0u8; n];
conn.read_exact(&mut frame)?;
r.read_exact(&mut frame).await?;
Ok(Some(frame))
}
/// pairs walks the header block: a count, then that many length-prefixed name
/// and value runs. Every name and value is a slice of the frame.
fn pairs(raw: &[u8]) -> Vec<(String, String)> {
let mut out = Vec::new();
if raw.len() < 4 {
return out;
}
let count = u32::from_le_bytes([raw[0], raw[1], raw[2], raw[3]]) as usize;
let mut at = 4;
for _ in 0..count {
let Some(name) = run(raw, &mut at) else { break };
let Some(value) = run(raw, &mut at) else {
break;
};
out.push((
String::from_utf8_lossy(name).into_owned(),
String::from_utf8_lossy(value).into_owned(),
));
}
out
}
fn run<'a>(raw: &'a [u8], at: &mut usize) -> Option<&'a [u8]> {
if *at + 4 > raw.len() {
return None;
}
let n = u32::from_le_bytes([raw[*at], raw[*at + 1], raw[*at + 2], raw[*at + 3]]) as usize;
*at += 4;
if *at + n > raw.len() {
return None;
}
let s = &raw[*at..*at + n];
*at += n;
Some(s)
}
/// build writes one answer frame: the status, the content type as the one
/// header, and the body.
fn build(status: u16, kind: &str, body: &[u8]) -> Vec<u8> {
let mut headers = Vec::new();
headers.extend_from_slice(&1u32.to_le_bytes());
push(&mut headers, b"Content-Type");
push(&mut headers, kind.as_bytes());
let reason = crate::http::reason(status);
let mut b = zap::Builder::new(body.len() + headers.len() + 256);
let mut ob = b.start_object(wire::RESPONSE_SIZE);
ob.set_u16(&mut b, wire::RESPONSE_STATUS, status);
ob.set_text(&mut b, wire::RESPONSE_REASON, reason);
ob.set_text(&mut b, wire::RESPONSE_PROTO, "ZAP-HTTP/1.0");
ob.set_bytes(&mut b, wire::RESPONSE_HEADERS, &headers);
ob.set_bytes(&mut b, wire::RESPONSE_BODY, body);
ob.set_bytes(&mut b, wire::RESPONSE_TRAILER, &[]);
ob.finish_as_root(&mut b);
b.finish_with_flags(FRAME_RESPONSE << 8)
}
fn push(out: &mut Vec<u8>, s: &[u8]) {
out.extend_from_slice(&(s.len() as u32).to_le_bytes());
out.extend_from_slice(s);
}
/// ask builds one request frame — what a client sends, and what the tests read
/// back to prove the door answers ZAP and not only text.
pub fn ask(method: &str, target: &str, body: &[u8]) -> Vec<u8> {
let mut b = zap::Builder::new(body.len() + 256);
let mut ob = b.start_object(wire::REQUEST_SIZE);
ob.set_text(&mut b, wire::REQUEST_METHOD, method);
ob.set_text(&mut b, wire::REQUEST_TARGET, target);
ob.set_text(&mut b, wire::REQUEST_PROTO, "ZAP-HTTP/1.0");
ob.set_bytes(&mut b, wire::REQUEST_HEADERS, &[]);
ob.set_bytes(&mut b, wire::REQUEST_BODY, body);
ob.set_bytes(&mut b, wire::REQUEST_TRAILER, &[]);
ob.finish_as_root(&mut b);
b.finish_with_flags(FRAME_REQUEST << 8)
}
rust/zip/tests/ops.rs
+242
View File
@@ -0,0 +1,242 @@
// SPDX-License-Identifier: BSD-3-Clause-Eco
// Copyright (C) 2026, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
//! What the derive describes, what the binder reads, and what the door does
//! with a handler that breaks.
use std::sync::Arc;
use std::time::Duration;
use serde::{Deserialize, Serialize};
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use zip::{Cx, Error, Wire};
/// Ask is a request whose fields arrive from everywhere a request has.
#[derive(Wire, Serialize, Deserialize, Default, Debug, PartialEq)]
#[serde(default, rename_all = "camelCase")]
struct Ask {
/// Org is read from the gateway's header.
#[zip(header = "X-Org-Id")]
org_id: String,
/// Model is required.
#[zip(required)]
model: String,
/// MaxTokens rides the query on a GET.
max_tokens: u32,
/// Stop is a list.
stop: Vec<String>,
/// Extra is passed through unread.
extra: Option<serde_json::Value>,
/// Blob is bytes, as base64.
#[serde(with = "zip::base64")]
blob: Vec<u8>,
/// Kept is never on the wire.
#[serde(skip)]
kept: u32,
/// Named is renamed outright.
#[serde(rename = "n")]
named: Count,
}
/// Count is a 64-bit count, carried as a quoted decimal.
#[derive(Wire, Default, Debug, PartialEq, Clone, Copy)]
#[zip(text)]
struct Count(u64);
struct Svc;
#[zip::ops(app = "t", title = "T", version = "1.0.0")]
impl Svc {
/// Ask answers the input it bound.
#[get("/v1/ask/:model")]
async fn ask(&self, arg: &Ask) -> Result<Ask, Error> {
Ok(Ask {
org_id: arg.org_id.clone(),
model: arg.model.clone(),
max_tokens: arg.max_tokens,
stop: arg.stop.clone(),
extra: arg.extra.clone(),
blob: arg.blob.clone(),
kept: 0,
named: arg.named,
})
}
/// Post answers its input, by value.
#[post("/v1/post")]
async fn post(&self, cx: &Cx, arg: Ask) -> Result<Ask, Error> {
cx.set_status(201)?;
cx.set_header("x-model", &arg.model)?;
Ok(arg)
}
/// Fail refuses, with a header on the refusal.
#[post("/v1/fail")]
async fn fail(&self, cx: &Cx) -> Result<(), Error> {
cx.set_header("retry-after", "7")?;
Err(Error::new(429, "slow down"))
}
/// Break panics.
#[post("/v1/break")]
async fn broken(&self) -> Result<(), Error> {
panic!("a handler bug");
}
}
fn field(name: &str) -> &'static zip::FieldDesc {
Ask::describe()
.fields
.iter()
.find(|f| f.name == name)
.unwrap()
}
#[test]
fn wire_names_are_serdes() {
assert_eq!(field("org_id").json, "orgId");
assert_eq!(field("org_id").header, "X-Org-Id");
assert_eq!(field("max_tokens").json, "maxTokens");
assert_eq!(field("kept").json, "-");
assert_eq!(field("named").json, "n");
assert!(field("model").required);
let m = Svc::manifest();
assert!(m.contains(r#""json":"maxTokens""#), "{m}");
assert!(m.contains(r#""name":"extra","json":"extra","doc":"Extra is passed through unread.","type":{"prim":"any","opt":true}"#), "{m}");
assert!(m.contains(r#""type":{"prim":"bytes"}"#), "{m}");
assert!(m.contains(r#"{"id":"Count","name":"Count","kind":"scalar","spell":"Count","repr":"u64","text":true,"json":{"type":"string"}}"#), "{m}");
}
#[test]
fn a_text_type_is_one_word() {
assert_eq!(
serde_json::to_string(&Count(18446744073709551615)).unwrap(),
"\"18446744073709551615\""
);
assert_eq!(serde_json::from_str::<Count>("\"42\"").unwrap(), Count(42));
let e = serde_json::from_str::<Count>("\"x\"")
.unwrap_err()
.to_string();
assert!(e.contains("not a Count"), "{e}");
}
async fn serve() -> String {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap().to_string();
let app = Arc::new(Svc.ops());
tokio::spawn(zip::http::serve(
app,
listener,
std::future::pending(),
Duration::from_secs(1),
));
addr
}
/// ask sends one request on its own connection and answers the status, the
/// head and the body.
async fn ask(addr: &str, req: &str) -> (u16, String, String) {
let mut c = tokio::net::TcpStream::connect(addr).await.unwrap();
c.write_all(req.as_bytes()).await.unwrap();
let mut all = Vec::new();
c.read_to_end(&mut all).await.unwrap();
let all = String::from_utf8(all).unwrap();
let (head, body) = all.split_once("\r\n\r\n").unwrap();
let status = head.split_whitespace().nth(1).unwrap().parse().unwrap();
(status, head.to_ascii_lowercase(), body.to_string())
}
#[tokio::test(flavor = "multi_thread")]
async fn a_request_binds_from_the_url_and_headers() {
let addr = serve().await;
let (status, _, body) = ask(
&addr,
"GET /v1/ask/kai?maxTokens=64&stop=a,b HTTP/1.1\r\nHost: x\r\nX-Org-Id: acme\r\nConnection: close\r\n\r\n",
)
.await;
assert_eq!(status, 200, "{body}");
let got: serde_json::Value = serde_json::from_str(&body).unwrap();
assert_eq!(
got,
serde_json::json!({"orgId": "acme", "model": "kai", "maxTokens": 64, "stop": ["a", "b"], "extra": null, "blob": "", "n": "0"})
);
}
#[tokio::test(flavor = "multi_thread")]
async fn a_body_binds_whole_and_the_answer_carries_what_cx_set() {
let addr = serve().await;
let body = r#"{"model":"kai","extra":{"tools":[1,2]},"blob":"emlw","n":"7","kept":9}"#;
let req = format!(
"POST /v1/post HTTP/1.1\r\nHost: x\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",
body.len()
);
let (status, head, out) = ask(&addr, &req).await;
assert_eq!(status, 201, "{out}");
assert!(head.contains("x-model: kai"), "{head}");
let got: serde_json::Value = serde_json::from_str(&out).unwrap();
assert_eq!(got["extra"], serde_json::json!({"tools": [1, 2]}));
assert_eq!(got["blob"], "emlw");
assert_eq!(got["n"], "7");
assert!(got.get("kept").is_none(), "a skipped field rode the wire");
// Not JSON is a 400, said as a problem document.
let req =
"POST /v1/post HTTP/1.1\r\nHost: x\r\nContent-Length: 3\r\nConnection: close\r\n\r\n{x}";
let (status, head, _) = ask(&addr, req).await;
assert_eq!(status, 400);
assert!(
head.contains("content-type: application/problem+json"),
"{head}"
);
}
#[tokio::test(flavor = "multi_thread")]
async fn a_refusal_keeps_its_status_and_the_headers_set() {
let addr = serve().await;
let (status, head, body) = ask(
&addr,
"POST /v1/fail HTTP/1.1\r\nHost: x\r\nContent-Length: 0\r\nConnection: close\r\n\r\n",
)
.await;
assert_eq!(status, 429);
assert!(head.contains("retry-after: 7"), "{head}");
assert_eq!(
body,
r#"{"detail":"slow down","status":429,"title":"Too Many Requests","type":"about:blank"}"#
);
}
#[tokio::test(flavor = "multi_thread")]
async fn a_handler_that_panics_is_a_500_and_the_server_lives() {
let addr = serve().await;
let (status, _, body) = ask(
&addr,
"POST /v1/break HTTP/1.1\r\nHost: x\r\nContent-Length: 0\r\nConnection: close\r\n\r\n",
)
.await;
assert_eq!(status, 500);
assert!(body.contains("the handler panicked"), "{body}");
let (status, _, _) = ask(
&addr,
"GET /v1/ask/kai HTTP/1.1\r\nHost: x\r\nConnection: close\r\n\r\n",
)
.await;
assert_eq!(status, 200);
}
#[tokio::test(flavor = "multi_thread")]
async fn a_body_past_the_bound_is_refused_before_it_is_read() {
let addr = serve().await;
let n = zip::MAX_BODY + 1;
let (status, _, _) = ask(
&addr,
&format!(
"POST /v1/post HTTP/1.1\r\nHost: x\r\nContent-Length: {n}\r\nConnection: close\r\n\r\n"
),
)
.await;
// Only the head was sent. A server that read before it counted would wait
// here for 32 MiB that are never coming.
assert_eq!(status, 413);
}
rust/zip/wire.zap
-29
View File
@@ -1,29 +0,0 @@
# The ZAP-HTTP frame: HTTP request and response semantics on the ZAP wire.
#
# These are the offsets zap-proto/http writes, so a Rust service and a Go one
# answer each other's frames. The accessors that read them are generated from
# this file; nothing in the runtime spells a byte offset by hand.
#
# Headers ride the frame as a count-prefixed run of length-prefixed name and
# value pairs — a repeated name is a repeated pair, which is what a multi-value
# header already is, so nothing has to model a list.
package wire
struct Request {
Method text @0
Target text @8
Proto text @16
Headers bytes @24
Body bytes @32
Trailer bytes @40
}
struct Response {
Status u16 @0
Reason text @8
Proto text @16
Headers bytes @24
Body bytes @32
Trailer bytes @40
}