papers/zen/zen-mathematical-reasoning.tex at main

mirror of https://github.com/hanzoai/papers.git synced 2026-10-29 07:34:31 +00:00

Files

T

zeekayandHanzo Dev bc1452e9f0 The zen claims that arithmetic settles

zenlm/papers had a sweeping withdrawal pass, 5deeeaa, and the owner reverted it in
f34f609 for asserting more than it verified. That objection sets the rule here: a
claim changes only where the proof is in the paper itself or in arithmetic, never
because a run could not be found. Twenty-seven of the eighty-one zen papers meet
that bar. Every correction quotes a number the paper already carries or one derived
from its own numbers by a division a reader can repeat.

Twelve papers led with a figure that appears in no row of the table it summarises.
zen-agent averaged GAIA 83.1/68.4/42.3 to 71.2 while its four baseline rows are each
the plain mean of their own three levels; the mean is 64.6. zen-embeddings-retrieval
and zen3-embedding claim MTEB averages of 72.4 and 74.3 over tables that supply their
task counts, where the 56-task weighted means are 67.1 and 67.4.
zen-mathematical-reasoning opened with five figures, none of them any row, and
explained one as best-of-32 sampling when its own sampling table puts best-of-32 at a
different number. zen-dso-protocol claimed 94% of centralized quality and a 78%
bandwidth cut against tables reading 99.4% and 99.8%-and-better, the second of them
the table that claim captions. zen-synthetic-data claimed an 8.4-point MT-Bench gain
on a scale that runs to 10, from a 7.84 baseline, where its table says 0.88.
zen-3d quoted the Chair column as the ShapeNet result. zen-multilingual understated
its own XNLI average. Also zen-hallucination-reduction, zen-distributed-training,
zen-finetuning, zen-legal-ai and zen-coder-flash.

Eight more are settled by arithmetic against the hardware or the parameter count.
zen-nano, zen-eco and zen-scribe each report INT4 needing as much memory as FP16 or
more, while their own FP16 and INT8 rows are exactly two and one bytes per parameter;
zen_family_overview already carried the right values. The two zen-designer papers size
a stated 235B model at 220 GB in FP16, which is a 110B model, and zen-quantization puts
the same model at 470 GB. zen-omni and zen-next recommend cards smaller than the memory
the same row requires. zen-live captions a latency table to an A10G when its own
deployment table assigns that 98 ms to the H100. zen-hardware-optimization calls a
throughput table FP8+INT4 when its cumulative table reaches the identical 31,200 tok/s
through INT4 + FP16.

Six papers share one template whose visual benchmark block bundles two metrics
computed over images a model generates with one computed over text a model emits
about an image. No model produces all three, and the block went to four papers across
four modalities, so each received at least one metric it cannot compute: VQA v2 and a
token rate for two diffusion models, FID and CLIP score for two vision-language models.
Each now keeps the rows its own stated output type can produce.

Throughput is the one category where the defect is the missing condition rather than
the number. Decode reads the active weight set once per token, so a single stream
cannot outrun bandwidth over footprint, and zen4-mini, zen3-nano and zen-coder-flash
all report above their own bound. Batched serving would make those figures legal and
none of the three records a batch size, so the numbers stand and each table now carries
its bound and the arithmetic behind it. AIME gets the same treatment in
zen-mathematical-reasoning: a score that is not a multiple of 1/30 on a 30-problem exam
is legal under averaging, and the caption now says which row has its sample count on
record and which do not.

What was examined and left is in LLM.md with the reason in each case. Training fleets
absent from this estate prove nothing about models trained on rented capacity.
zen-reranker's 98% semantic preservation is uncited rather than contradicted.
zen-safety-evaluation and zen-vision-architecture state no weighting, so their averages
are not determined. zen4-coder-flash puts FP8 on an A100, and Ampere has no FP8 tensor
cores, but its claims there are footprint claims and FP8 weight-only storage does run
on Ampere.

All twenty-seven rebuild under latexmk -Werror. Nineteen have diverged from their
counterparts in zenlm/papers and six exist only here, so those fixes do not reach the
other site. zen-3d and zen-agent were byte-identical across the two repositories until
now, and are the two to mirror first.

Co-authored-by: Hanzo Dev <[email protected]>

2026-09-10 23:54:37 -07:00

352 lines

13 KiB

TeX

Raw Permalink Blame History

 \documentclass[11pt,a4paper]{article}
 \usepackage[utf8]{inputenc}
 \usepackage[T1]{fontenc}
 \usepackage{amsmath,amsfonts,amssymb}
 \usepackage{graphicx}
 \usepackage{hyperref}
 \usepackage{listings}
 \usepackage{color}
 \usepackage{booktabs}
 \usepackage{float}
 \usepackage{algorithm}
 \usepackage{algorithmic}
 \usepackage{geometry}
 \geometry{margin=1in}
 \definecolor{zenblue}{RGB}{41,121,255}
 \hypersetup{colorlinks=true,linkcolor=zenblue,urlcolor=zenblue,citecolor=zenblue}
 \title{\textbf{Mathematical Reasoning in Zen Models}\\
 \large Technical Report v2025.08}
 \author{Antje Worring, Hanzo AI Research, Zen LM Research Team\\
 \texttt{[email protected]}}
 \date{August 2025}
 \begin{document}
 \maketitle
 \begin{abstract}
 We present a comprehensive study of mathematical reasoning capabilities in Zen models,
 covering training data construction, process reward models (PRMs) for step-level
 supervision, and a symbolic-neural hybrid verification system. Zen-7B scores 71.4\%
 on the MATH benchmark, 95.8\% on GSM8K, 38.3\% on AIME 2024, 74.2\% on AMC 2023,
 and 28.1\% on FrontierMath; Zen-32B with the step-level PRM scores 83.4, 97.9, 48.2,
 .7, and 36.9 on the same five. The PRM adds 4.8 MATH points at 7B (71.4 to 76.2)
 and 4.6 at 32B (78.8 to 83.4) over the same model without it. The hybrid
 verification system using SymPy for
 algebraic verification and neural methods for proof checking enables reliable
 answer verification across problem classes. We describe data construction, training
 protocol, and detailed performance analysis by mathematical domain.
 \end{abstract}
 \section{Introduction}
 Mathematical reasoning requires multi-step inference, precise computation, and
 systematic verification -- capabilities that push language models to their limits.
 Unlike open-ended generation tasks, mathematics has ground truth: answers are
 either correct or incorrect, enabling automated evaluation and scalable training
 signal generation.
 The Zen mathematical reasoning stack consists of three components:
 \begin{enumerate}
   \item \textbf{Mathematical pretraining data}: 120B tokens of mathematical text
     spanning textbooks, papers, and competition problems.
   \item \textbf{Process Reward Model (PRM)}: A model that evaluates solution quality
     step-by-step rather than only at the final answer.
   \item \textbf{Hybrid verification}: Symbolic algebra (SymPy) + neural verification
     for reliable answer checking.
 \end{enumerate}
 \section{Mathematical Pretraining Data}
 \subsection{Corpus Composition}
 \begin{table}[H]
 \centering
 \begin{tabular}{lrr}
 \toprule
 \textbf{Source} & \textbf{Tokens} & \textbf{Quality} \\
 \midrule
 arXiv math papers & 38B & High \\
 Math textbooks (OCR) & 22B & High \\
 Competition problems + solutions & 4B & Very high \\
 Stack Exchange (math) & 18B & Medium \\
 Synthetic generated problems & 24B & High \\
 Wikipedia math articles & 8B & High \\
 Code (mathematical libraries) & 6B & Medium \\
 \midrule
 Total & 120B & -- \\
 \bottomrule
 \end{tabular}
 \caption{Mathematical pretraining corpus.}
 \end{table}
 \subsection{Synthetic Problem Generation}
 A core challenge is data scarcity at the frontier: competition-level problems are
 rare and quickly memorized. We generate synthetic problems via three strategies:
 \subsubsection{Template Perturbation}
 Existing problems are modified by changing numerical constants, function types, or
 geometric configurations while preserving problem structure. For each seed problem,
 we generate 10--20 variants with verified solutions.
 \subsubsection{Forward-Generation}
 We sample mathematical objects (functions, sequences, geometric configurations) and
 generate problems whose answer is the sampled object. For example: sample a polynomial
 $p(x)$, generate a word problem whose solution is $p(x)$.
 \subsubsection{Backward-Chaining}
 Starting from a target theorem or formula, generate a minimal problem that requires
 the target as a solution step. This biases the distribution toward problems that
 exercise specific mathematical tools.
 \begin{table}[H]
 \centering
 \begin{tabular}{lrr}
 \toprule
 \textbf{Strategy} & \textbf{Problems Generated} & \textbf{Verified Correct} \\
 \midrule
 Template perturbation & 4.2M & 97.3\% \\
 Forward-generation & 2.8M & 94.1\% \\
 Backward-chaining & 1.6M & 91.8\% \\
 Total & 8.6M & 94.9\% \\
 \bottomrule
 \end{tabular}
 \caption{Synthetic problem generation statistics.}
 \end{table}
 \section{Process Reward Model}
 \subsection{Motivation}
 Outcome reward models (ORMs) assign a single score to the complete solution.
 This signal is sparse for long multi-step problems and provides no information
 about which steps are correct. Process reward models (PRMs) evaluate each reasoning
 step, providing dense supervision:
 \begin{equation}
 r_\text{PRM}(s_1, s_2, \ldots, s_T) = \prod_{t=1}^T p(\text{step}_t \text{ correct} \mid s_1, \ldots, s_t)
 \end{equation}
 \subsection{PRM Training Data: Process Supervision}
 We collect process supervision data via two methods:
 \textbf{Human annotation}: 12K solutions to MATH and AMC problems annotated
 step-by-step (correct/incorrect/uncertain) by mathematically trained annotators.
 Inter-annotator agreement: 91.3\% on step-level labels.
 \textbf{Monte Carlo sampling}: For a solution prefix $(s_1, \ldots, s_t)$, we estimate
 step correctness by sampling $K=128$ completions and computing the fraction that
 reach the correct final answer:
 \begin{equation}
 p(\text{step}_t \text{ correct}) \approx \frac{1}{K}\sum_{k=1}^K \mathbf{1}[\text{completion}_k \text{ correct}]
 \end{equation}
 \begin{algorithm}[H]
 \caption{Monte Carlo PRM Data Generation}
 \begin{algorithmic}[1]
 \REQUIRE Problem set $\mathcal{P}$, policy $\pi$, completions per step $K=128$
 \ENSURE Step-label dataset $\mathcal{D}_\text{prm}$
 \FOR{each problem $p \in \mathcal{P}$}
   \STATE $S \leftarrow \pi.\text{solve}(p)$ \COMMENT{Generate full solution $S = (s_1, \ldots, s_T)$}
   \FOR{step $t = 1 \ldots T$}
     \STATE $\text{completions} \leftarrow [\pi(p, s_{1:t}) \text{ for } k=1\ldots K]$
     \STATE $\hat{q}_t \leftarrow \frac{1}{K}\sum_k \mathbf{1}[\text{correct}(\text{completion}_k)]$
     \STATE $\mathcal{D}_\text{prm} \leftarrow \mathcal{D}_\text{prm} \cup \{(p, s_{1:t}, \hat{q}_t)\}$
   \ENDFOR
 \ENDFOR
 \RETURN $\mathcal{D}_\text{prm}$
 \end{algorithmic}
 \end{algorithm}
 \subsection{PRM Architecture and Training}
 The PRM is a Zen-7B model with a classification head on each reasoning step token.
 Training uses a binary cross-entropy loss with soft labels from MC sampling:
 \begin{equation}
 \mathcal{L}_\text{PRM} = -\sum_{t} \left[\hat{q}_t \log p_\phi(c_t) + (1-\hat{q}_t)\log(1-p_\phi(c_t))\right]
 \end{equation}
 We train on 800K step-labeled solutions (600K synthetic + 200K human-annotated).
 \subsection{PRM-Guided Search}
 At inference, we use beam search guided by the PRM:
 \begin{equation}
 \text{score}(s_{1:T}) = \min_t p_\phi(c_t \mid s_{1:t}) \quad (\text{pessimistic PRM})
 \end{equation}
 The pessimistic aggregation (minimum over steps) prioritizes solutions with no
 weak steps over solutions with some strong steps and some weak ones.
 \section{Hybrid Verification}
 \subsection{Symbolic Verification (SymPy)}
 For algebraic expressions, equations, and numerical answers, we use SymPy to
 verify equivalence:
 \begin{algorithm}[H]
 \caption{Symbolic Verification}
 \begin{algorithmic}[1]
 \REQUIRE Predicted answer $\hat{a}$, ground truth $a^*$
 \ENSURE Boolean: correct or not
 \STATE Parse $\hat{a}$ and $a^*$ into SymPy expressions
 \IF{simplify$(\hat{a} - a^*) = 0$}
   \RETURN True
 \ELSIF{simplify$(\hat{a} - a^*) \neq 0$}
   \RETURN False
 \ELSE
   \STATE \RETURN NeuralVerify$(\hat{a}, a^*)$
 \ENDIF
 \end{algorithmic}
 \end{algorithm}
 Symbolic verification handles polynomial equations, trigonometric identities, and
 rational expressions. It fails on open-form answers (proofs, geometric arguments,
 combinatorial constructions), which are routed to neural verification.
 \subsection{Neural Verification}
 For non-algebraic answers, a Zen-32B model trained on 50K verified proof/answer
 pairs assesses correctness. The verifier is given both the predicted answer and
 the ground truth and produces a calibrated correctness probability.
 \section{Experiments}
 \subsection{Benchmark Results}
 \begin{table}[H]
 \centering
 \begin{tabular}{lccccc}
 \toprule
 \textbf{Model} & \textbf{MATH} & \textbf{GSM8K} & \textbf{AIME 2024} & \textbf{AMC 2023} & \textbf{FrontierMath} \\
 \midrule
 Zen-7B & 71.4 & 95.8 & 38.3 & 74.2 & 28.1 \\
 Zen-7B + PRM & 76.2 & 96.8 & 40.1 & 77.4 & 29.8 \\
 Zen-32B & 78.8 & 97.2 & 44.3 & 81.2 & 33.4 \\
 Zen-32B + PRM & \textbf{83.4} & \textbf{97.9} & \textbf{48.2} & \textbf{84.7} & \textbf{36.9} \\
 \bottomrule
 \end{tabular}
 \caption{Mathematical reasoning benchmark results. The PRM adds between 0.7 and 4.8
 points, depending on benchmark and model size. AIME 2024 is 30 problems, so a
 single-pass score is a multiple of 3.3 points; these AIME figures are therefore
 averages over repeated samples. The sample count is on record only for
 Zen-32B + PRM ($n=128$, Section~\ref{sec:olympiad}); for the other three rows it
 is not.}
 \end{table}
 Each row is one decode per problem on MATH and GSM8K. What test-time sampling adds
 is reported separately in Table~\ref{tab:sampling}: Zen-7B with PRM reranking reaches
 .1\% MATH at $k=32$ and 79.0\% at $k=64$.
 \subsection{MATH Category Breakdown}
 \begin{table}[H]
 \centering
 \begin{tabular}{lcc}
 \toprule
 \textbf{Category} & \textbf{Zen-7B (\%)} & \textbf{Zen-7B + PRM (\%)} \\
 \midrule
 Algebra & 84.3 & 88.1 \\
 Counting \& Probability & 71.2 & 76.4 \\
 Geometry & 62.4 & 67.8 \\
 Number Theory & 78.3 & 82.1 \\
 Precalculus & 66.1 & 71.2 \\
 Intermediate Algebra & 69.8 & 74.3 \\
 \midrule
 Overall & 71.4 & 76.2 \\
 \bottomrule
 \end{tabular}
 \caption{MATH benchmark by category. Geometry benefits most from PRM guidance.}
 \end{table}
 \subsection{Effect of Sampling at Test Time}
 \begin{table}[H]
 \centering
 \begin{tabular}{lccccc}
 \toprule
 \textbf{Samples} & \textbf{1} & \textbf{4} & \textbf{16} & \textbf{32} & \textbf{64} \\
 \midrule
 No PRM (majority vote) & 71.4 & 74.1 & 76.3 & 77.2 & 77.8 \\
 PRM (best-of-$k$) & 71.4 & 74.8 & 77.3 & 78.1 & 79.0 \\
 \bottomrule
 \end{tabular}
 \caption{MATH accuracy vs. number of samples (Zen-7B). PRM reranking outperforms majority vote.}
 \label{tab:sampling}
 \end{table}
 \subsection{PRM Calibration}
 The PRM step correctness scores are well-calibrated: across 50K held-out steps,
 steps scored $<0.1$ have 6.2\% actual correctness rate; steps scored $>0.9$ have
 .4\% actual correctness rate. Calibration error (ECE) = 0.043.
 \section{Analysis}
 \subsection{Error Taxonomy}
 We manually analyzed 200 errors by Zen-7B on MATH Level 5 problems:
 \begin{table}[H]
 \centering
 \begin{tabular}{lrc}
 \toprule
 \textbf{Error Type} & \textbf{Count} & \textbf{PRM Detectable?} \\
 \midrule
 Arithmetic computation error & 61 & 78\% \\
 Wrong approach / setup & 44 & 52\% \\
 Algebraic manipulation error & 38 & 71\% \\
 Missing case / edge case & 29 & 31\% \\
 Incorrect interpretation & 18 & 18\% \\
 Copy/transcription error & 10 & 90\% \\
 \midrule
 Total & 200 & \\
 \bottomrule
 \end{tabular}
 \caption{Error taxonomy for Zen-7B on MATH Level 5. PRM detects most computational errors.}
 \end{table}
 The PRM is most effective at detecting arithmetic and algebraic errors (step produces
 numerically inconsistent result) and least effective at detecting missing cases or
 misinterpretation (where each individual step appears locally valid).
 \subsection{Olympiad-Level Performance}
 \label{sec:olympiad}
 On AIME 2024 (30 problems), Zen-32B with PRM-guided search and $n=128$ samples
 achieves 48.2\% (14.5/30 problems). Human AIME competitors typically score 8--15
 in competitive settings.
 On FrontierMath (research-level problems from active mathematics research), 36.9\%
 represents a significant capability frontier. Many FrontierMath problems require
 theorem-proving capabilities beyond current neural approaches.
 \section{Conclusion}
 Mathematical reasoning in Zen models benefits substantially from three components:
 domain-rich pretraining data (120B tokens), process reward models that provide
 step-level guidance (+4.6 to +4.8 MATH points), and hybrid verification enabling
 reliable self-evaluation. The combination yields 83.4\% MATH, 97.9\% GSM8K, and
 .2\% AIME for Zen-32B with the PRM. Open problems include proof-level verification for advanced mathematics
 and step-level data collection at Olympiad difficulty.
 \bibliographystyle{plain}
 \begin{thebibliography}{99}
 \bibitem{lightman2023prm} Lightman et al. (2023). Let's Verify Step by Step. \textit{arXiv:2305.20050}.
 \bibitem{hendrycks2021math} Hendrycks et al. (2021). Measuring Mathematical Problem Solving With the MATH Dataset. \textit{NeurIPS}.
 \bibitem{cobbe2021gsm8k} Cobbe et al. (2021). Training Verifiers to Solve Math Word Problems. \textit{arXiv:2110.14168}.
 \bibitem{wang2023prm800k} Wang et al. (2023). Math-Shepherd: Verify and Reinforce LLMs Step-by-Step without Human Annotations. \textit{arXiv:2312.08935}.
 \bibitem{trinh2024alphageometry} Trinh et al. (2024). Solving olympiad geometry without human demonstrations. \textit{Nature}.
 \end{thebibliography}
 \end{document}

352 lines 13 KiB TeX Raw Permalink Blame History

352 lines

13 KiB

TeX

Raw Permalink Blame History