Files
papers/lux-quasar-benchmarks/sections/15-literature-backed-comparison.tex
zeekay 21038c00b9 papers: add named-competitor comparison-table scaffolds + repro spec to 5 benchmark papers
Add explicit, literature-backed comparison scaffolds to:
- lux-pq-consensus-benchmarks (sec 09): vs Mysticeti-C, HotStuff/-2, Tendermint, Snowman
- lux-quasar-benchmarks (sec 15): vs Mysticeti-C, HotStuff-2, CometBFT, Snowman, TowerBFT
- lux-fhe-benchmarks (sec 12): vs TFHE-rs, Concrete, OpenFHE, Microsoft SEAL
- lux-zap-benchmarks (sec 14): vs Protobuf, FlatBuffers, Cap'n Proto, gRPC, QUIC, JSON-RPC
- evmgpu-benchmark (sec 13): vs geth, reth, erigon, evmone, Block-STM, pevm

Lux/competitor columns explicitly marked [TBD-measure] where not yet
benchmarked against the competitor's own published harness. Every
competitor cell is sourced to a cited paper, repo benchmark, or
vendor doc. Each scaffold ends with a Reproducibility spec
(hardware, software pins, commands, decision rule) for the actual
controlled run.

No invented Lux numbers; existing internal numbers retained but
flagged as not-yet-apples-to-apples vs literature.
2026-05-13 01:09:15 -07:00

130 lines
7.1 KiB
TeX

\section{Literature-Backed Comparison Scaffold}
\label{sec:lit-backed-comparison}
%% ============================================================================
The benchmark tables in Sections~\ref{sec:finality}--\ref{sec:related} report
the Quasar engine on Lux internal testbeds. To make those numbers
\emph{independently falsifiable} against named competitors we publish here a
clean scaffold whose competitor columns are filled \textbf{from cited
published numbers}, and whose Quasar columns are explicitly
\texttt{[TBD-measure]} pending a single controlled re-run on the
competitor's own testbed shape. \textbf{This is the table that wins or
loses the ``better than any other solution'' claim.}
\subsection{Competitors and Why}
\begin{itemize}[leftmargin=2em,nosep]
\item \textbf{Mysticeti-C}~\cite{babel2024mysticeti} --- 2024 NDSS;
published WAN-latency floor for non-PQ DAG-BFT.
\item \textbf{HotStuff-2}~\cite{malkhi2023hotstuff2} --- 2023; ePrint;
canonical linear-comm BFT.
\item \textbf{Tendermint / CometBFT}~\cite{buchman2016tendermint,cometbftgithub}
--- production reference BFT (every Cosmos zone).
\item \textbf{Avalanche Snowman}~\cite{rocket2019avalanche,amores2024avalanche}
--- probabilistic-finality prior art, the protocol Quasar's
Photon/Wave layer descends from.
\item \textbf{Solana TowerBFT}~\cite{yakovenko2018solana} --- highest
public-network TPS but minute-scale economic finality.
\end{itemize}
\subsection{Headline Latency / Throughput Table}
\begin{table}[h]
\centering
\small
\begin{tabular}{@{}lrrrrl@{}}
\toprule
\textbf{Protocol} & \textbf{Validators} & \textbf{Finality (ms)} & \textbf{TPS} & \textbf{Msg / commit} & \textbf{Source} \\
\midrule
\textbf{Quasar (CPU-BLS)} & 21 & \texttt{[TBD]} & \texttt{[TBD]} & \texttt{[TBD]} & this paper \\
\textbf{Quasar (CPU-BLS)} & 100 & \texttt{[TBD]} & \texttt{[TBD]} & \texttt{[TBD]} & this paper \\
\textbf{Quasar (GPU-BLS)} & 21 & \texttt{[TBD]} & \texttt{[TBD]} & \texttt{[TBD]} & this paper \\
\textbf{Quasar (Horizon, PQ)} & 21 & \texttt{[TBD]} & \texttt{[TBD]} & \texttt{[TBD]} & this paper \\
\midrule
Mysticeti-C & 10 & 500 & $>$200{,}000 & $O(n)$ & \cite[Fig.~5]{babel2024mysticeti} \\
Mysticeti-C & 50 & $<$1{,}000 & $\sim$400{,}000 & $O(n)$ & \cite[Fig.~6]{babel2024mysticeti} \\
Mysticeti-C & 106 & 390 (p50) & --- & $O(n)$ & \cite[\S6.3]{babel2024mysticeti} \\
HotStuff (orig.) & 128 & $\sim$700 & $\sim$50{,}000 & $O(n)$ & \cite[\S5.3]{yin2019hotstuff} \\
HotStuff-2 & 100 & $\sim$400 & --- & $O(n)$ & \cite[\S6]{malkhi2023hotstuff2} \\
Tendermint / CometBFT & 100 & 1{,}000--3{,}000 & $\sim$10{,}000 (peak) & $O(n^2)$ & \cite{cometbftgithub,alqahtani2021bottlenecks} \\
Avalanche Snowman & 2{,}000 & 1{,}350 & 3{,}400 & $O(k)$ (sampling) & \cite[Table~5]{rocket2019avalanche} \\
Solana TowerBFT & 1{,}500 & $\sim$12{,}800 (econ.) & 65{,}000 (claimed) & $O(n)$ & \cite{yakovenko2018solana} \\
\bottomrule
\end{tabular}
\caption{Literature-backed comparison scaffold. The Quasar rows are
\texttt{[TBD]} until run on a 21-/100-node WAN testbed matching the
Mysticeti deployment shape (\S\ref{sec:lit-repro}). The competitor
column values are taken verbatim from the cited tables/figures.}
\label{tab:quasar-lit-comparison}
\end{table}
\subsection{Post-Quantum Coverage Matrix}
The point of Quasar is the \emph{combination} of low latency and a PQ
finality path. The table below makes the PQ axis explicit; the latency
axis is the table above.
\begin{table}[h]
\centering
\small
\begin{tabular}{@{}lcccl@{}}
\toprule
\textbf{Protocol} & \textbf{Classical fast path} & \textbf{PQ finality} & \textbf{Standardised PQ} & \textbf{Source} \\
\midrule
Quasar / Lux & BLS aggregate & Pulsar + ML-DSA-65 & FIPS-204 & this paper, \cite{fips204} \\
Mysticeti-C & Ed25519 / BLS & none & --- & \cite{babel2024mysticeti} \\
HotStuff-2 & Ed25519 / BLS & none & --- & \cite{malkhi2023hotstuff2} \\
CometBFT & Ed25519 & none & --- & \cite{cometbftgithub} \\
Snowman / Avalanche & secp256k1 / BLS & roadmap only & --- & \cite{rocket2019avalanche} \\
Solana TowerBFT & Ed25519 & none & --- & \cite{yakovenko2018solana} \\
\bottomrule
\end{tabular}
\caption{Quasar is the only entry in this set that ships a PQ-final path
on the steady-state critical path today; competitors carry classical
signatures only.}
\label{tab:quasar-pq-coverage}
\end{table}
\subsection{Reproducibility Specification}
\label{sec:lit-repro}
The \texttt{[TBD]} cells in Table~\ref{tab:quasar-lit-comparison} are
fillable only with an apples-to-apples run. Spec:
\begin{itemize}[leftmargin=2em,nosep]
\item \textbf{Hardware.} 21- and 100-node WAN deployments; AWS
\texttt{c6i.8xlarge} (32~vCPU Ice~Lake, 64\,GiB) across
\texttt{us-east-1}, \texttt{eu-west-1}, \texttt{ap-southeast-1}
--- matches \cite[\S6.1]{babel2024mysticeti}.
\item \textbf{Workload.} 512\,B transactions, open-loop Poisson
injector sweeping $\lambda \in \{10^3, 10^4, 10^5, 2\!\times\!10^5\}$
TPS per the Mysticeti methodology.
\item \textbf{Quasar build.} \texttt{luxfi/[email protected]},
\texttt{luxfi/pulsar}, \texttt{luxfi/lens}, \texttt{luxfi/warp}
at the same tag; GPU off (CPU-BLS) and GPU on (Metal/CUDA) as
two distinct rows.
\item \textbf{Competitor builds.}
\begin{itemize}[leftmargin=2em,nosep]
\item Mysticeti-C: \texttt{MystenLabs/sui@main}, \texttt{cargo run --release --bin orchestrator -- benchmark}.
\item HotStuff-2: \texttt{nayak-lab/hotstuff-2@main}, default config.
\item CometBFT: \texttt{cometbft/[email protected]}, \texttt{tm-load-test} workload.
\end{itemize}
\item \textbf{Metrics.} p50 / p99 commit latency at fixed $\lambda$;
sustained TPS at p99 commit~$\leq$~1\,s; messages exchanged per
commit measured at the OS socket level (\texttt{ss -i}).
\item \textbf{Decision rule.} Publish the table with the actual numbers
and the verdict (win / tie / lose) per cell. \emph{Do not} re-tag
the Quasar paper with new headline numbers until every
\texttt{[TBD]} is replaced by a measured value with a 95\%
confidence interval over $\geq$~5 runs of $\geq$~600\,s each.
\end{itemize}
\paragraph{What this scaffold replaces.} The internal benchmarks in
Sections~\ref{sec:finality}--\ref{sec:related} report Quasar in isolation;
they are not a substitute for the cross-protocol re-run specified here.
The headline ``X$\times$ faster than Y'' claims in the abstract should be
treated as \emph{hypotheses to be tested} by the experiment in this
section, not as established facts.
%% ============================================================================