mirror of
https://github.com/hanzoai/site.git
synced 2026-11-21 01:07:08 +00:00
platform.hanzo.ai is the platform console (with cloud.hanzo.ai redirecting to it at the edge). Point the remaining cloud.hanzo.ai links on /benchmarks and /philosophy directly at https://platform.hanzo.ai. Claude-Session: https://claude.ai/code/session_015XYKLix9PSyn6Jw2HWKprk
1 line
3.0 KiB
JSON
1 line
3.0 KiB
JSON
{"text":["Engine prefill on CUDA vs llama.cpp","Benchmarks","measured","hanzo-engine against llama.cpp on CUDA, built against the same weights and run on the same box: prefill throughput at the prompt lengths where the whole 95% interval of the ratio lies above parity.","harness","hanzoai/engine · bench/dossier","our paper","Refutation-Driven Performance Engineering","scripts/dossier.sh --backend cuda --model <same.gguf> -p 512,500,2048,4096 -n 128 -r 7","prints the table on this page — transcribed from the committed run files on branch bench/dossier, the pass of 2026-07-21","What is measured","hanzo-engine and llama.cpp, built against the same weights, run on the same box, alternating. Prefill is tokens per second at a given prompt length, and the ratio is ours over theirs, so above 1.000 is a win. The models are Zen-1.7B at Q4_K_M and Zen-4B at Q4_K_M; 7 repetitions per cell, as its run records, with the first discarded as warmup, a Student-t 95% interval on each engine, and the ratio's interval formed by adding the two relative intervals in quadrature.","The verdict is the harness's, not a reading of ours:","where the whole interval lies above 1.000 and neither engine's coefficient of variation exceeded 5%, which is how the harness says the box was quiet. Every cell below is one.","WIN","Prefill on CUDA, cell by cell","cells from","runs on branch","· engine","1.7.87","· each run pins its engine release, its llama.cpp build and its model digest, and is linked below","bench/dossier","backend","model","quant","prompt tokens","llama.cpp t/s","hanzo t/s","ratio","95% interval","verdict","CUDA","Zen-1.7B","Q4_K_M","500","7250.2","7961.3","1.098","1.053 – 1.143","4096","6871.3","7474.8","1.088","1.051 – 1.125","Zen-4B","2048","3592.5","3774.4","1.051","1.017 – 1.085","The work that produced it","is the campaign that produced these kernels, written as a log of numbered hypotheses, each settled by a cheap decisive experiment. A long prefill is compute-bound, which is where that work landed.","Hanzo AI Cloud","Build on Hanzo Cloud","Start on Hanzo Cloud","View Models & Pricing","Previous","Sandbox cold start","All Benchmarks Board","Next","GPQA-Diamond","Other benchmarks","LoCoMo","MemoryAgentBench","LongMemEval","CronQuestions","RepoBench-R","LoCoMo · subject scope","LoCoMo-Conv","Fleet residency","Live agent footprint","the committed runs, in","bench-runs"],"links":["/benchmarks","https://github.com/hanzoai/engine/tree/bench/dossier/bench-runs","https://papers.hanzo.ai/refutation-driven-perf","https://platform.hanzo.ai","/pricing","https://hanzo.ai/benchmarks/sandbox","https://hanzo.ai/benchmarks/gpqa","https://hanzo.ai/benchmarks/locomo","https://hanzo.ai/benchmarks/memoryagentbench","https://hanzo.ai/benchmarks/longmemeval","https://hanzo.ai/benchmarks/cronquestions","https://hanzo.ai/benchmarks/repobench-r","https://hanzo.ai/benchmarks/locomo-subject-scope","https://hanzo.ai/benchmarks/locomo-conv","https://hanzo.ai/benchmarks/fleet","https://hanzo.ai/benchmarks/goroutine"]} |