papers/hanzo-webhooks/sections/10-evaluation.tex

\section{Evaluation}


\subsection{Scale}

Production statistics:

\begin{itemize}
    \item Daily event volume: 2.3 million
    \item Active subscriptions: 3,400
    \item Unique endpoints: 2,100
    \item Merchants: 1,200
\end{itemize}

\subsection{Delivery Performance}

\begin{table}[h]
\centering
\caption{Delivery Performance}
\begin{tabular}{lc}
\toprule
\textbf{Metric} & \textbf{Value} \\
\midrule
First-attempt success rate & 99.7\% \\
Eventual delivery rate & 99.99\% \\
Mean delivery latency & 1.2 seconds \\
P95 delivery latency & 4.8 seconds \\
P99 delivery latency & 12.3 seconds \\
\bottomrule
\end{tabular}
\end{table}

\subsection{Adaptive Retry Improvement}

Comparing adaptive vs. fixed exponential backoff:

\begin{table}[h]
\centering
\caption{Retry Strategy Comparison}
\begin{tabular}{lcc}
\toprule
\textbf{Metric} & \textbf{Fixed} & \textbf{Adaptive} \\
\midrule
Mean time to delivery (failed first attempt) & 8.2 min & 5.4 min \\
Unnecessary retries (healthy endpoints) & 12\% & 4\% \\
Resource usage (CPU) & Baseline & -18\% \\
\bottomrule
\end{tabular}
\end{table}

The adaptive algorithm reduces delivery latency by 34\% for transiently failing endpoints.

\subsection{Failure Analysis}

Root causes of delivery failures:

\begin{table}[h]
\centering
\caption{Failure Categories}
\begin{tabular}{lc}
\toprule
\textbf{Cause} & \textbf{Percentage} \\
\midrule
Endpoint timeout & 42\% \\
Connection refused & 23\% \\
HTTP 5xx error & 18\% \\
DNS resolution failure & 9\% \\
TLS handshake failure & 5\% \\
HTTP 4xx error & 3\% \\
\bottomrule
\end{tabular}
\end{table}