Files
zeekay 7618e7f014 Check in the bench harness and bytes-per-token tool
LLM.md section 3b quoted numbers from a script that lived in a home
directory and counted chunks, not tokens. Under MTP or DFlash one chunk
carries several tokens, and vLLM v0.29 streams thinking as `reasoning`
where hanzo sends `reasoning_content`, so chunk counting measured steps
and missed vLLM's thinking entirely.

bench_http.py counts token chunks by content, reasoning,
reasoning_content or tool_calls, takes decode tokens from usage (a
stream without usage fails), and reports TTFT, decode rate, per-step
gaps and a concurrent aggregate as JSON lines. It also carries the
/metrics check and the profile window used by later tests.

bytes_per_token.py gives two labeled numbers: the computed weight, KV
and state bytes per token from safetensors headers (19.44 GB of 27B
weights, matching SGLang's figure), and the measured SM-to-L2 bytes
from an nsys gb20y-top export. GB10 has no DRAM counter, so the
measured figure is a cross-check, never a substitute.

Claude-Session: https://claude.ai/code/session_019ABm6hF8RNyUFpbjQWqfxY
2026-09-24 14:36:19 -07:00

24 KiB