zeekay 7fdca28595 feat(distributed-inference): cross-vendor TP result + open-problems companion ...

Four papers on heterogeneous distributed LLM inference:

- hanzo-heterogeneous-distributed-inference: cross-vendor (AMD ROCm gfx1151 +
  NVIDIA GB10 CUDA) tensor-parallel decode over a 2.5GbE cable, coherent at
  50.4 tok/s (TTFT 0.09s, 213 tok/s prefill). Root cause of prior corruption
  proven: per-backend auto-dtype (ROCm->f16, CUDA->bf16) reinterpreted raw on
  the wire. Fixed by a canonical f32 ring wire + world_size-1 accumulating
  all-reduce (correct for any N>=2; spurious power-of-two gate removed).
  Same-vendor f16+f16 control matches token-for-token.
- hanzo-distributed-open-problems: NEW companion enumerating 8 open research
  problems (sampling-determinism boundaries, quant-aware TP sharding, PP
  layer-assignment ILP, adaptive speculation under RTT/acceptance drift, the
  prefill launch-overhead wall, KV wire compression, the geo decode latency
  floor, and cheap verifiable inference), each cross-referenced to the paper
  that owns the surrounding result.
- hanzo-distributed-decode-latency, hanzo-consensus-zap-compute: consistency
  pass (canonical f32 wire terminology; link goodput unified to 280 MB/s =
  2.24 Gbit/s; precise f16-vs-bf16 root-cause wording).

INDEX.md regenerated via `make index`; besides the 4 above it also indexes 6
pre-existing root papers the previous index had missed. All four compile clean
with pdflatex (10/10/13/9 pages).

2026-07-09 16:31:20 -07:00

316 KiB

Raw Permalink History

View Raw