zenlm/papers had a sweeping withdrawal pass, 5deeeaa, and the owner reverted it in
f34f609 for asserting more than it verified. That objection sets the rule here: a
claim changes only where the proof is in the paper itself or in arithmetic, never
because a run could not be found. Twenty-seven of the eighty-one zen papers meet
that bar. Every correction quotes a number the paper already carries or one derived
from its own numbers by a division a reader can repeat.
Twelve papers led with a figure that appears in no row of the table it summarises.
zen-agent averaged GAIA 83.1/68.4/42.3 to 71.2 while its four baseline rows are each
the plain mean of their own three levels; the mean is 64.6. zen-embeddings-retrieval
and zen3-embedding claim MTEB averages of 72.4 and 74.3 over tables that supply their
task counts, where the 56-task weighted means are 67.1 and 67.4.
zen-mathematical-reasoning opened with five figures, none of them any row, and
explained one as best-of-32 sampling when its own sampling table puts best-of-32 at a
different number. zen-dso-protocol claimed 94% of centralized quality and a 78%
bandwidth cut against tables reading 99.4% and 99.8%-and-better, the second of them
the table that claim captions. zen-synthetic-data claimed an 8.4-point MT-Bench gain
on a scale that runs to 10, from a 7.84 baseline, where its table says 0.88.
zen-3d quoted the Chair column as the ShapeNet result. zen-multilingual understated
its own XNLI average. Also zen-hallucination-reduction, zen-distributed-training,
zen-finetuning, zen-legal-ai and zen-coder-flash.
Eight more are settled by arithmetic against the hardware or the parameter count.
zen-nano, zen-eco and zen-scribe each report INT4 needing as much memory as FP16 or
more, while their own FP16 and INT8 rows are exactly two and one bytes per parameter;
zen_family_overview already carried the right values. The two zen-designer papers size
a stated 235B model at 220 GB in FP16, which is a 110B model, and zen-quantization puts
the same model at 470 GB. zen-omni and zen-next recommend cards smaller than the memory
the same row requires. zen-live captions a latency table to an A10G when its own
deployment table assigns that 98 ms to the H100. zen-hardware-optimization calls a
throughput table FP8+INT4 when its cumulative table reaches the identical 31,200 tok/s
through INT4 + FP16.
Six papers share one template whose visual benchmark block bundles two metrics
computed over images a model generates with one computed over text a model emits
about an image. No model produces all three, and the block went to four papers across
four modalities, so each received at least one metric it cannot compute: VQA v2 and a
token rate for two diffusion models, FID and CLIP score for two vision-language models.
Each now keeps the rows its own stated output type can produce.
Throughput is the one category where the defect is the missing condition rather than
the number. Decode reads the active weight set once per token, so a single stream
cannot outrun bandwidth over footprint, and zen4-mini, zen3-nano and zen-coder-flash
all report above their own bound. Batched serving would make those figures legal and
none of the three records a batch size, so the numbers stand and each table now carries
its bound and the arithmetic behind it. AIME gets the same treatment in
zen-mathematical-reasoning: a score that is not a multiple of 1/30 on a 30-problem exam
is legal under averaging, and the caption now says which row has its sample count on
record and which do not.
What was examined and left is in LLM.md with the reason in each case. Training fleets
absent from this estate prove nothing about models trained on rented capacity.
zen-reranker's 98% semantic preservation is uncited rather than contradicted.
zen-safety-evaluation and zen-vision-architecture state no weighting, so their averages
are not determined. zen4-coder-flash puts FP8 on an A100, and Ampere has no FP8 tensor
cores, but its claims there are footprint claims and FP8 weight-only storage does run
on Ampere.
All twenty-seven rebuild under latexmk -Werror. Nineteen have diverged from their
counterparts in zenlm/papers and six exist only here, so those fixes do not reach the
other site. zen-3d and zen-agent were byte-identical across the two repositories until
now, and are the two to mirror first.
Co-authored-by: Hanzo Dev <[email protected]>
zeekayandHanzo Dev
bc1452e9f0