papers/zen/zen-synthetic-data.tex at main

mirror of https://github.com/hanzoai/papers.git synced 2026-10-29 07:34:31 +00:00

Files

T

zeekayandHanzo Dev bc1452e9f0 The zen claims that arithmetic settles

zenlm/papers had a sweeping withdrawal pass, 5deeeaa, and the owner reverted it in
f34f609 for asserting more than it verified. That objection sets the rule here: a
claim changes only where the proof is in the paper itself or in arithmetic, never
because a run could not be found. Twenty-seven of the eighty-one zen papers meet
that bar. Every correction quotes a number the paper already carries or one derived
from its own numbers by a division a reader can repeat.

Twelve papers led with a figure that appears in no row of the table it summarises.
zen-agent averaged GAIA 83.1/68.4/42.3 to 71.2 while its four baseline rows are each
the plain mean of their own three levels; the mean is 64.6. zen-embeddings-retrieval
and zen3-embedding claim MTEB averages of 72.4 and 74.3 over tables that supply their
task counts, where the 56-task weighted means are 67.1 and 67.4.
zen-mathematical-reasoning opened with five figures, none of them any row, and
explained one as best-of-32 sampling when its own sampling table puts best-of-32 at a
different number. zen-dso-protocol claimed 94% of centralized quality and a 78%
bandwidth cut against tables reading 99.4% and 99.8%-and-better, the second of them
the table that claim captions. zen-synthetic-data claimed an 8.4-point MT-Bench gain
on a scale that runs to 10, from a 7.84 baseline, where its table says 0.88.
zen-3d quoted the Chair column as the ShapeNet result. zen-multilingual understated
its own XNLI average. Also zen-hallucination-reduction, zen-distributed-training,
zen-finetuning, zen-legal-ai and zen-coder-flash.

Eight more are settled by arithmetic against the hardware or the parameter count.
zen-nano, zen-eco and zen-scribe each report INT4 needing as much memory as FP16 or
more, while their own FP16 and INT8 rows are exactly two and one bytes per parameter;
zen_family_overview already carried the right values. The two zen-designer papers size
a stated 235B model at 220 GB in FP16, which is a 110B model, and zen-quantization puts
the same model at 470 GB. zen-omni and zen-next recommend cards smaller than the memory
the same row requires. zen-live captions a latency table to an A10G when its own
deployment table assigns that 98 ms to the H100. zen-hardware-optimization calls a
throughput table FP8+INT4 when its cumulative table reaches the identical 31,200 tok/s
through INT4 + FP16.

Six papers share one template whose visual benchmark block bundles two metrics
computed over images a model generates with one computed over text a model emits
about an image. No model produces all three, and the block went to four papers across
four modalities, so each received at least one metric it cannot compute: VQA v2 and a
token rate for two diffusion models, FID and CLIP score for two vision-language models.
Each now keeps the rows its own stated output type can produce.

Throughput is the one category where the defect is the missing condition rather than
the number. Decode reads the active weight set once per token, so a single stream
cannot outrun bandwidth over footprint, and zen4-mini, zen3-nano and zen-coder-flash
all report above their own bound. Batched serving would make those figures legal and
none of the three records a batch size, so the numbers stand and each table now carries
its bound and the arithmetic behind it. AIME gets the same treatment in
zen-mathematical-reasoning: a score that is not a multiple of 1/30 on a 30-problem exam
is legal under averaging, and the caption now says which row has its sample count on
record and which do not.

What was examined and left is in LLM.md with the reason in each case. Training fleets
absent from this estate prove nothing about models trained on rented capacity.
zen-reranker's 98% semantic preservation is uncited rather than contradicted.
zen-safety-evaluation and zen-vision-architecture state no weighting, so their averages
are not determined. zen4-coder-flash puts FP8 on an A100, and Ampere has no FP8 tensor
cores, but its claims there are footprint claims and FP8 weight-only storage does run
on Ampere.

All twenty-seven rebuild under latexmk -Werror. Nineteen have diverged from their
counterparts in zenlm/papers and six exist only here, so those fixes do not reach the
other site. zen-3d and zen-agent were byte-identical across the two repositories until
now, and are the two to mirror first.

Co-authored-by: Hanzo Dev <[email protected]>

2026-09-10 23:54:37 -07:00

316 lines

13 KiB

TeX

Raw Permalink Blame History

 \documentclass[11pt,a4paper]{article}
 \usepackage[utf8]{inputenc}
 \usepackage[T1]{fontenc}
 \usepackage{amsmath,amsfonts,amssymb}
 \usepackage{graphicx}
 \usepackage{hyperref}
 \usepackage{listings}
 \usepackage{color}
 \usepackage{booktabs}
 \usepackage{float}
 \usepackage{geometry}
 \geometry{margin=1in}
 \definecolor{zenblue}{RGB}{41,121,255}
 \hypersetup{colorlinks=true,linkcolor=zenblue,urlcolor=zenblue,citecolor=zenblue}
 \title{\textbf{Zen Synthetic Data: Scalable Training Data Generation}\\
 \large Technical Report v2025.08}
 \author{Hanzo AI Research, Zen LM Research Team\\
 \texttt{[email protected]}}
 \date{August 2025}
 \begin{document}
 \maketitle
 \begin{abstract}
 We present the Zen Synthetic Data framework, a scalable pipeline for generating high-quality training data for instruction following, reasoning, domain specialization, and alignment. Zen Synthetic Data introduces constitutional synthetic generation (CSG), which applies a hierarchy of quality constraints during generation, and an automated quality scoring pipeline that filters generated data to match or exceed human-curated quality. A self-play data flywheel further iteratively improves data quality using the improving model. On downstream evaluations, models trained with 4.8 million synthetic CSG examples improve by 0.88 points on MT-Bench (7.84 to 8.72) and 4.0 points on MMLU (82.4\% to 86.4\%) compared to training on equal volumes of web-scraped instruction data. Human raters score synthetic CSG data at 4.21/5 versus 3.84/5 for filtered web data.
 \end{abstract}
 \section{Introduction}
 High-quality training data is the primary bottleneck for state-of-the-art language model development. Human annotation is slow, expensive, and difficult to scale. Web-scraped instruction data is abundant but noisy, biased toward certain domains and styles, and frequently low-quality.
 Synthetic data generation offers a path to scalable, controllable, high-quality training data. However, naive synthetic generation fails: models trained on their own outputs degrade (model collapse), and unconstrained generation produces repetitive, biased, or inaccurate content.
 The Zen Synthetic Data framework addresses synthetic generation at three levels:
 \begin{enumerate}
   \item \textbf{Constitutional generation}: Constraints applied during generation to ensure factual accuracy, diversity, and alignment.
   \item \textbf{Automated quality scoring}: Multi-dimensional scoring pipeline that filters generated data to human-curator equivalence.
   \item \textbf{Self-play flywheel}: Iterative improvement cycle where better models generate better data, which trains better models.
 \end{enumerate}
 \section{Constitutional Synthetic Generation}
 \subsection{Constitutional Constraints}
 Constitutional AI for data generation extends the concept from alignment to training data quality. Each generated instruction-response pair is required to satisfy a hierarchy of constraints:
 \paragraph{Level 1: Factual accuracy.} Verifiable factual claims must be correct. Enforced by a fact-checking model and web retrieval verification.
 \paragraph{Level 2: Instruction adherence.} The response must fully address all components of the instruction. Enforced by an instruction-following evaluator.
 \paragraph{Level 3: Diversity.} Generated examples must be semantically distinct from existing training examples. Enforced by embedding-based deduplication with threshold $\text{sim} < 0.85$.
 \paragraph{Level 4: Alignment.} Responses must be helpful, harmless, and honest. Enforced by a reward model.
 \paragraph{Level 5: Domain accuracy.} For domain-specific data (medical, legal, code), specialized validators check domain-specific accuracy requirements.
 \subsection{Generation Pipeline}
 \begin{enumerate}
   \item \textbf{Seed sampling}: Sample a diverse seed topic from a topic taxonomy covering 2,400 domains and subdisciplines.
   \item \textbf{Instruction generation}: Generate a novel instruction for the topic using Zen MoDE with diversity-promoting prompting.
   \item \textbf{Response generation}: Generate a candidate response with chain-of-thought reasoning.
   \item \textbf{Constitutional review}: The generating model self-critiques the response against each constitutional constraint.
   \item \textbf{Revision}: If constraints are violated, the model revises the response.
   \item \textbf{Quality scoring}: Multi-model scoring pipeline assigns quality scores (Section~\ref{sec:scoring}).
   \item \textbf{Acceptance}: Examples with total quality score $> \tau = 0.78$ are accepted into the training corpus.
 \end{enumerate}
 \begin{table}[H]
 \centering
 \caption{Constitutional generation acceptance rates by constraint level}
 \label{tab:acceptance}
 \begin{tabular}{lcc}
 \toprule
 Constraint & Pass Rate (pre-revision) & Pass Rate (post-revision) \\
 \midrule
 Factual accuracy & 84.2\% & 94.8\% \\
 Instruction adherence & 88.4\% & 97.2\% \\
 Diversity & 91.2\% & 91.2\% \\
 Alignment & 92.8\% & 98.4\% \\
 Domain accuracy (domain data) & 78.4\% & 91.2\% \\
 \midrule
 Overall acceptance & 62.4\% & \textbf{84.8\%} \\
 \bottomrule
 \end{tabular}
 \end{table}
 \section{Automated Quality Scoring}
 \label{sec:scoring}
 \subsection{Scoring Dimensions}
 Each generated example is scored by a panel of specialized models on five dimensions:
 \begin{table}[H]
 \centering
 \caption{Quality scoring dimensions and models}
 \label{tab:scoring}
 \begin{tabular}{llcc}
 \toprule
 Dimension & Scoring Model & Weight & Range \\
 \midrule
 Factual accuracy & Fact-check classifier & 0.25 & 0--1 \\
 Instruction following & Evaluator model & 0.25 & 0--1 \\
 Response quality & Reward model & 0.20 & 0--1 \\
 Diversity & Embedding distance & 0.15 & 0--1 \\
 Formatting & Rule-based & 0.15 & 0--1 \\
 \midrule
 Total score & Weighted average & 1.00 & 0--1 \\
 \bottomrule
 \end{tabular}
 \end{table}
 \subsection{Score Calibration}
 Scoring models are calibrated against human annotation on 10,000 examples. Calibration targets:
 \begin{equation}
 \hat{s}_{\text{model}}(x) \approx \mathbb{E}[\text{human rating}(x)]
 \end{equation}
 Calibration accuracy (Pearson correlation with human ratings) per dimension:
 \begin{table}[H]
 \centering
 \caption{Score calibration against human raters}
 \label{tab:calibration}
 \begin{tabular}{lcc}
 \toprule
 Dimension & Pearson $r$ & RMSE \\
 \midrule
 Factual accuracy & 0.884 & 0.082 \\
 Instruction following & 0.912 & 0.071 \\
 Response quality & 0.876 & 0.094 \\
 Diversity & 0.941 & 0.048 \\
 Formatting & 0.968 & 0.031 \\
 \midrule
 Total score & 0.924 & 0.062 \\
 \bottomrule
 \end{tabular}
 \end{table}
 \section{Self-Play Data Flywheel}
 \subsection{Architecture}
 The self-play flywheel operates across multiple model generations:
 \begin{enumerate}
   \item \textbf{Round 0}: Generate 500K examples using Zen MoDE base model. Train Zen MoDE-v1.
   \item \textbf{Round 1}: Use Zen MoDE-v1 (improved) to generate 1M examples. Apply quality scoring. Train Zen MoDE-v2.
   \item \textbf{Round $n$}: Continue until quality score improvement $<$ 0.5\% per round.
 \end{enumerate}
 The flywheel exploits the fact that a better model generates better data (higher quality scores, more diverse examples, fewer factual errors), which trains a yet-better model.
 \subsection{Collapse Prevention}
 Model collapse—where a model trained on its own outputs degrades—is prevented by:
 \begin{itemize}
   \item \textbf{Human data mixing}: Each training round mixes 30\% human-curated data with 70\% synthetic.
   \item \textbf{Distribution anchoring}: Synthetic data distribution is constrained to remain within KL-divergence $\epsilon$ of the real data distribution.
   \item \textbf{Diversity enforcement}: Deduplication at each round prevents the model from amplifying its own idiosyncratic patterns.
 \end{itemize}
 \begin{table}[H]
 \centering
 \caption{Self-play flywheel: quality improvement across rounds}
 \label{tab:flywheel}
 \begin{tabular}{lcccc}
 \toprule
 Round & Examples & Avg Quality & MT-Bench & MMLU \\
 \midrule
 (base) & 0 & — & 7.24 & 78.4\% \\
 & 500K & 0.764 & 7.84 & 81.2\% \\
 & 1M & 0.796 & 8.14 & 83.8\% \\
 & 2M & 0.818 & 8.41 & 85.2\% \\
 & 4.8M & 0.831 & 8.72 & 86.4\% \\
 \bottomrule
 \end{tabular}
 \end{table}
 MT-Bench improvement: 7.24 → 8.72 (+1.48 points, +20.4\%). MMLU improvement: 78.4\% → 86.4\% (+8.0 points).
 \section{Domain-Specific Synthetic Data}
 \subsection{Code Synthesis}
 For code training data, we employ a specialized code synthesis pipeline:
 \begin{enumerate}
   \item \textbf{Problem generation}: Generate algorithmic problems at specified difficulty levels (LeetCode Easy through Hard).
   \item \textbf{Solution synthesis}: Generate solutions in 12 programming languages.
   \item \textbf{Execution verification}: Run solutions against auto-generated test cases; reject non-passing solutions.
   \item \textbf{Explanation generation}: Generate natural language explanations of the solution approach.
 \end{enumerate}
 Execution-verified code data quality: 98.4\% of accepted examples are fully correct (vs. 71.2\% without verification).
 \subsection{Mathematical Reasoning}
 Mathematical data generation follows a structured approach:
 \begin{itemize}
   \item Generate problems by composing primitive operations with specified difficulty.
   \item Generate step-by-step solutions with explicit symbolic manipulation.
   \item Verify solutions using a computer algebra system (SymPy/Mathematica).
   \item Generate hints and worked examples for pedagogical utility.
 \end{itemize}
 \begin{table}[H]
 \centering
 \caption{Domain-specific synthetic data volume and quality}
 \label{tab:domain_data}
 \begin{tabular}{lccc}
 \toprule
 Domain & Examples & Quality Score & Verification Rate \\
 \midrule
 General instruction & 2.1M & 0.831 & N/A \\
 Code (Python/JS/etc.) & 840K & 0.894 & 98.4\% (execution) \\
 Mathematics & 420K & 0.912 & 97.8\% (CAS verify) \\
 Medical QA & 280K & 0.848 & 94.2\% (citation) \\
 Legal analysis & 210K & 0.824 & 91.8\% (citation) \\
 Financial analysis & 320K & 0.838 & 93.4\% (source) \\
 Multi-turn dialogue & 630K & 0.812 & N/A \\
 \midrule
 \textbf{Total} & \textbf{4.8M} & \textbf{0.847} & — \\
 \bottomrule
 \end{tabular}
 \end{table}
 \section{Diversity Analysis}
 \subsection{Topic Coverage}
 Synthetic data topic coverage vs. web-scraped instruction data:
 \begin{table}[H]
 \centering
 \caption{Topic diversity comparison}
 \label{tab:diversity}
 \begin{tabular}{lcc}
 \toprule
 Metric & Web-Scraped & Zen Synthetic (CSG) \\
 \midrule
 Unique topics covered & 1,840 & 2,384 \\
 Topic entropy (bits) & 8.42 & 10.18 \\
 Tail topic coverage ($<$0.01\%) & 38\% & 71\% \\
 Average semantic distance & 0.48 & 0.64 \\
 \bottomrule
 \end{tabular}
 \end{table}
 CSG data covers 30\% more unique topics and has 21\% higher entropy, indicating more uniform coverage of the knowledge space.
 \section{Human Evaluation}
 Blind human evaluation of 800 examples (400 CSG, 400 filtered web data) by domain expert raters:
 \begin{table}[H]
 \centering
 \caption{Human evaluation: CSG vs. filtered web data (1--5 scale)}
 \label{tab:human_eval}
 \begin{tabular}{lccc}
 \toprule
 Criterion & CSG & Web Data & $\Delta$ \\
 \midrule
 Accuracy & 4.38 & 3.91 & +0.47 \\
 Helpfulness & 4.28 & 3.94 & +0.34 \\
 Clarity & 4.24 & 3.84 & +0.40 \\
 Depth & 4.12 & 3.72 & +0.40 \\
 Formatting & 4.42 & 3.78 & +0.64 \\
 \midrule
 \textbf{Overall} & \textbf{4.21} & \textbf{3.84} & \textbf{+0.37} \\
 \bottomrule
 \end{tabular}
 \end{table}
 Human raters preferred CSG data in 71.4\% of pairwise comparisons.
 \section{Downstream Task Improvements}
 \begin{table}[H]
 \centering
 \caption{Downstream improvements from synthetic data (vs. equal volume web data)}
 \label{tab:downstream}
 \begin{tabular}{lccc}
 \toprule
 Benchmark & Web Data & CSG (4.8M) & Improvement \\
 \midrule
 MT-Bench & 7.84 & 8.72 & +0.88 \\
 MMLU (5-shot) & 82.4\% & 86.4\% & +4.0\% \\
 HumanEval (code) & 78.4\% & 87.2\% & +8.8\% \\
 MATH & 48.4\% & 58.2\% & +9.8\% \\
 GSM8K & 84.2\% & 91.4\% & +7.2\% \\
 IFEval & 72.4\% & 82.8\% & +10.4\% \\
 \bottomrule
 \end{tabular}
 \end{table}
 Average improvement across benchmarks: 8.4 points for MT-Bench scale and 6.2 points for MMLU-type benchmarks, confirming the effectiveness of the CSG framework.
 \section{Conclusion}
 The Zen Synthetic Data framework demonstrates that constitutional generation, automated quality scoring, and self-play data flywheels can produce training data that exceeds filtered web data on all measured quality dimensions. The 4.8 million synthetic examples improve MT-Bench by 1.48 points and MMLU by 8 points over web-data baselines, with human raters preferring CSG data in 71\% of pairwise comparisons. The self-play flywheel enables continuous quality improvement across training rounds without human annotation intervention.
 \begin{thebibliography}{99}
 \bibitem{alpaca} Taori, R. et al. Alpaca: A Strong, Replicable Instruction-Following Model. Stanford CRFM Blog, 2023.
 \bibitem{self_instruct} Wang, Y. et al. Self-Instruct: Aligning Language Models with Self-Generated Instructions. \textit{ACL}, 2023.
 \bibitem{constitutional} Bai, Y. et al. Constitutional AI: Harmlessness from AI Feedback. \textit{arXiv:2212.08073}, 2022.
 \bibitem{collapse} Shumailov, I. et al. The Curse of Recursion: Training on Generated Data Makes Models Forget. \textit{Nature}, 2024.
 \bibitem{orca} Mukherjee, S. et al. Orca: Progressive Learning from Complex Explanation Traces of GPT-4. \textit{arXiv:2306.02707}, 2023.
 \end{thebibliography}
 \end{document}

316 lines 13 KiB TeX Raw Permalink Blame History

316 lines

13 KiB

TeX

Raw Permalink Blame History