All ninety-six papers from 86d8915 were audited for numbers presented as
results. Three hundred and fifty-three turn out to be presented as measurements
with nothing behind them, and not one paper in the batch cites a harness, a run,
a checkpoint, a seed or a date. The thirteen product papers at the root are the
part of that this commit fixes.
Each carried a production-operations figure in its abstract — 200+ models across
48 GPUs at 99.9% availability, 2M authentications daily over 1,200 tenants,
$240M processed at 99.97%, 15TB across 400 tenants, three years of ledger
operation without a discrepancy, a 65% cut in mean-time-to-diagnosis, per-request
overhead against nginx and Caddy, an iPhone 15 Pro latency table. None of the
repositories behind them holds a benchmark that produces those numbers; no
deployment in this estate records those volumes; the hardware two of them name
does not exist here; and several papers are dated years before their own code was
first committed.
A design target can be relabelled. These could not, because a target is a claim
about the future and these were claims about a past that did not happen. So they
are removed, and every paper now carries a short section naming what it said,
why it is gone, and what run would put a number back. Nothing was replaced with a
number that was not measured either.
Two papers are different. pubsub and ledger ship harnesses that measure exactly
the quantities their tables report — the NATS benchmark suite in one, a
parallelism-flagged TPS harness in the other. Those tables stay, with their
provenance stated: not traceable to a recorded run, and here is what would make
them so. The gap is a run, not a mechanism.
The eighty-one zen/ papers hold the other 321 and are left alone. zenlm/papers
already had a sweeping withdrawal pass reverted for asserting more than it
verified, the models are largely real, and several of these are template-
generated with one pretraining sentence filled in for different models — that
needs a decision about the family, not an edit per paper. LLM.md records the
finding, the shapes it takes, and the two zen papers that already do it right.
Co-authored-by: Hanzo Dev <[email protected]>
zeekayandHanzo Dev