mirror of
https://github.com/zap-proto/papers.git
synced 2026-10-30 15:04:17 +00:00
zeekayandHanzo Dev
80194f47f2
The paper's headline -- 622x less memory, 115x fewer allocations, 1.85-3.68x the throughput -- came from a third arm that bench-results.txt never contained. That file was written by the initial commit at 20:33:54; the native arm arrived eight minutes later and the file was never regenerated. The stored output has two arms and the string "Native" appears in it zero times. Worse than missing: the arm was not ZAP. It was a TCP server in the test file that read a 4-byte length prefix and wrote the same bytes back without looking at them -- no ZAP import, no message format, no schema, no headers -- while the paper described it as "length-prefixed frames, schema-typed bodies" and compared it against a full net/http stack. 622 is 9954/16, an HTTP allocation floor divided by a payload size. Re-running it would have produced numbers about copying bytes. So we finished the migration rather than deleting the claim. The native arm is a real zap-proto/go v1.8.3 typed message carrying body, headers and status; a net/http arm is back because that is what the published claim was made against; a fasthttp arm sits between them so the wire is not confounded with the object model; and the byte echo stays as a floor, labelled as a lower bound and not as ZAP. What came out is worse for the native arm than what was published. A native round trip allocates six objects and two heap copies of the payload -- 2B + 192 bytes at every size, and 2(896) + 192 on an 824-byte body because the copy is a slice allocation and rounds to a size class. The cause is one line: ObjectBuilder.SetBytes stores append([]byte(nil), v...) and Finish copies it again. Present in v1.1.0 and in the current v1.8.3, measured identically on both. The 622x/115x/1.85-3.68x figures and the 1233 MB/s bandwidth table are withdrawn. Better for the adapter than what was published. ZAP-HTTP is allocation-free in steady state and allocates fewer bytes than fasthttp at every payload size, on the identical handler; its frame is 0.1-13.5% larger than the equivalent HTTP/1.1 request, not the 86% the old run measured and printed as "0.54x (-86.0% smaller)". Against net/http both ZAP arms win by 3.7-10.6x in every paired repetition under concurrency. Against fasthttp the ratio brackets 1.0 in both directions and we claim nothing. Throughput was taken on a machine at load 100-200 across ten cores. Allocation counts and wire sizes do not move with load and repeat to the digit; rates do, and the paper says so instead of quoting them as hardware. The structural part: main.tex now holds no measured literal. emit.py writes results.tex -- every table row and every figure quoted in prose -- from bench-results.txt, the Makefile regenerates before building, and emit refuses a run still in flight rather than publishing half of one. A column cannot outlive the output behind it, which is the whole defect. Co-authored-by: Hanzo Dev <[email protected]>
29 lines
1.1 KiB
Makefile
29 lines
1.1 KiB
Makefile
# results.tex is generated from the harness output; main.tex holds no measured
|
|
# literal, so the paper cannot print a figure the output does not contain. If
|
|
# the output is beside this checkout, `all' regenerates before building and the
|
|
# fragment cannot be stale. If it is not, the build uses the committed
|
|
# fragment, whose header names the run it came from.
|
|
BENCH := ../../bench/bench-results.txt
|
|
|
|
.PHONY: all clean results verify
|
|
all: results main.pdf
|
|
|
|
results:
|
|
@if [ -f $(BENCH) ]; then ./emit.py $(BENCH) || \
|
|
echo "keeping the committed results.tex"; \
|
|
else echo "no $(BENCH); building from the committed results.tex"; fi
|
|
|
|
main.pdf: main.tex results.tex
|
|
latexmk -pdf -halt-on-error -Werror main.tex
|
|
|
|
# Does the committed fragment still describe the harness output on disk?
|
|
verify:
|
|
@test -f $(BENCH) || { echo "no $(BENCH) to verify against"; exit 0; }
|
|
@./emit.py $(BENCH) >/dev/null && git diff --quiet -- results.tex \
|
|
&& echo "results.tex matches $(BENCH)" \
|
|
|| { echo "results.tex is stale: run make and commit the fragment"; \
|
|
git --no-pager diff --stat -- results.tex; exit 1; }
|
|
|
|
clean:
|
|
latexmk -C main.tex
|