papers/zen/zen-hallucination-reduction.tex at main

mirror of https://github.com/hanzoai/papers.git synced 2026-10-29 07:34:31 +00:00

Files

T

zeekayandHanzo Dev bc1452e9f0 The zen claims that arithmetic settles

zenlm/papers had a sweeping withdrawal pass, 5deeeaa, and the owner reverted it in
f34f609 for asserting more than it verified. That objection sets the rule here: a
claim changes only where the proof is in the paper itself or in arithmetic, never
because a run could not be found. Twenty-seven of the eighty-one zen papers meet
that bar. Every correction quotes a number the paper already carries or one derived
from its own numbers by a division a reader can repeat.

Twelve papers led with a figure that appears in no row of the table it summarises.
zen-agent averaged GAIA 83.1/68.4/42.3 to 71.2 while its four baseline rows are each
the plain mean of their own three levels; the mean is 64.6. zen-embeddings-retrieval
and zen3-embedding claim MTEB averages of 72.4 and 74.3 over tables that supply their
task counts, where the 56-task weighted means are 67.1 and 67.4.
zen-mathematical-reasoning opened with five figures, none of them any row, and
explained one as best-of-32 sampling when its own sampling table puts best-of-32 at a
different number. zen-dso-protocol claimed 94% of centralized quality and a 78%
bandwidth cut against tables reading 99.4% and 99.8%-and-better, the second of them
the table that claim captions. zen-synthetic-data claimed an 8.4-point MT-Bench gain
on a scale that runs to 10, from a 7.84 baseline, where its table says 0.88.
zen-3d quoted the Chair column as the ShapeNet result. zen-multilingual understated
its own XNLI average. Also zen-hallucination-reduction, zen-distributed-training,
zen-finetuning, zen-legal-ai and zen-coder-flash.

Eight more are settled by arithmetic against the hardware or the parameter count.
zen-nano, zen-eco and zen-scribe each report INT4 needing as much memory as FP16 or
more, while their own FP16 and INT8 rows are exactly two and one bytes per parameter;
zen_family_overview already carried the right values. The two zen-designer papers size
a stated 235B model at 220 GB in FP16, which is a 110B model, and zen-quantization puts
the same model at 470 GB. zen-omni and zen-next recommend cards smaller than the memory
the same row requires. zen-live captions a latency table to an A10G when its own
deployment table assigns that 98 ms to the H100. zen-hardware-optimization calls a
throughput table FP8+INT4 when its cumulative table reaches the identical 31,200 tok/s
through INT4 + FP16.

Six papers share one template whose visual benchmark block bundles two metrics
computed over images a model generates with one computed over text a model emits
about an image. No model produces all three, and the block went to four papers across
four modalities, so each received at least one metric it cannot compute: VQA v2 and a
token rate for two diffusion models, FID and CLIP score for two vision-language models.
Each now keeps the rows its own stated output type can produce.

Throughput is the one category where the defect is the missing condition rather than
the number. Decode reads the active weight set once per token, so a single stream
cannot outrun bandwidth over footprint, and zen4-mini, zen3-nano and zen-coder-flash
all report above their own bound. Batched serving would make those figures legal and
none of the three records a batch size, so the numbers stand and each table now carries
its bound and the arithmetic behind it. AIME gets the same treatment in
zen-mathematical-reasoning: a score that is not a multiple of 1/30 on a 30-problem exam
is legal under averaging, and the caption now says which row has its sample count on
record and which do not.

What was examined and left is in LLM.md with the reason in each case. Training fleets
absent from this estate prove nothing about models trained on rented capacity.
zen-reranker's 98% semantic preservation is uncited rather than contradicted.
zen-safety-evaluation and zen-vision-architecture state no weighting, so their averages
are not determined. zen4-coder-flash puts FP8 on an A100, and Ampere has no FP8 tensor
cores, but its claims there are footprint claims and FP8 weight-only storage does run
on Ampere.

All twenty-seven rebuild under latexmk -Werror. Nineteen have diverged from their
counterparts in zenlm/papers and six exist only here, so those fixes do not reach the
other site. zen-3d and zen-agent were byte-identical across the two repositories until
now, and are the two to mirror first.

Co-authored-by: Hanzo Dev <[email protected]>

2026-09-10 23:54:37 -07:00

459 lines

18 KiB

TeX

Raw Permalink Blame History

 \documentclass[11pt,a4paper]{article}
 \usepackage[utf8]{inputenc}
 \usepackage[T1]{fontenc}
 \usepackage{amsmath,amsfonts,amssymb}
 \usepackage{graphicx}
 \usepackage{hyperref}
 \usepackage{listings}
 \usepackage{color}
 \usepackage{booktabs}
 \usepackage{float}
 \usepackage{geometry}
 \geometry{margin=1in}
 \definecolor{zenblue}{RGB}{41,121,255}
 \hypersetup{colorlinks=true,linkcolor=zenblue,urlcolor=zenblue,citecolor=zenblue}
 \title{\textbf{Reducing Hallucinations in Zen Models: Retrieval Augmentation,\\
 Confidence Calibration, and Citation-Grounded Generation}\\
 \large Technical Report v2025.08}
 \author{Antje Worring, Hanzo AI Research, Zen LM Research Team\\
 \texttt{[email protected]}}
 \date{August 2025}
 \begin{document}
 \maketitle
 \begin{abstract}
 Hallucination — the generation of plausible but factually incorrect content — remains
 a central challenge for large language models. We report a comprehensive study of
 hallucination in the Zen MoDE model family and introduce three complementary
 interventions: (1) \textbf{Retrieval-Augmented Factuality Training (RAFT)}, which
 fine-tunes models to condition on retrieved evidence when generating factual claims;
 (2) \textbf{Uncertainty-Aware Decoding (UAD)}, which modulates token probabilities by
 a calibrated confidence estimate to suppress low-confidence continuations; and (3)
 \textbf{Citation-Grounded Generation (CGG)}, which trains models to emit inline
 citations linking claims to source documents. Together, these methods reduce
 hallucination rates by 67\% on TruthfulQA, 53\% on FactScore, and 61\% on HaluEval
 relative to our baseline Zen MoDE-72B, with negligible perplexity degradation.
 \end{abstract}
 \tableofcontents
 \newpage
 %% -----------------------------------------------------------------------
 \section{Introduction}
 \label{sec:intro}
 %% -----------------------------------------------------------------------
 Language models learn to predict the next token by compressing statistical patterns in
 training text. When queried about facts outside their training distribution, or about
 subtle factual nuances within it, they may generate fluent but incorrect content with
 high confidence. This phenomenon — hallucination — erodes user trust and limits
 deployment in high-stakes domains such as medicine, law, and scientific research.
 Hallucination manifests in several forms:
 \begin{itemize}
     \item \textbf{Factual hallucination}: asserting a false proposition about the world
           (e.g., incorrect dates, attributed quotes, fabricated citations).
     \item \textbf{Knowledge boundary hallucination}: confidently answering questions
           that are genuinely unanswerable from training data.
     \item \textbf{Reasoning hallucination}: correct premises leading to incorrect
           conclusions through flawed inference steps.
 \end{itemize}
 We address all three categories through our RAFT + UAD + CGG system, evaluated on the
 Zen MoDE architecture across 7B, 32B, and 72B parameter scales.
 \subsection{Contributions}
 \begin{enumerate}
     \item A systematic measurement of hallucination rates across domains and model scales,
           using TruthfulQA, FactScore, HaluEval, and a new domain-stratified hallucination
           benchmark (ZenHallu).
     \item Retrieval-Augmented Factuality Training (RAFT): a fine-tuning recipe that
           teaches models to identify when retrieval is needed and how to synthesize
           evidence faithfully.
     \item Uncertainty-Aware Decoding (UAD): a training-free decoding modification that
           calibrates output confidence and suppresses hedging-unfaithful generations.
     \item Citation-Grounded Generation (CGG): a training procedure for models to emit
           structured inline citations with confidence scores.
     \item Ablation and analysis showing complementary contributions of each component.
 \end{enumerate}
 %% -----------------------------------------------------------------------
 \section{Hallucination Measurement}
 \label{sec:measurement}
 %% -----------------------------------------------------------------------
 \subsection{Benchmark Descriptions}
 \textbf{TruthfulQA} \cite{lin2021truthfulqa}: 817 questions spanning conspiracy theories,
 misconceptions, fiction, and factual recall. Models must answer truthfully or abstain.
 We use the multiple-choice formulation.
 \textbf{FactScore} \cite{min2023factscore}: Measures the factual precision of long-form
 biography generation by decomposing outputs into atomic claims and verifying each against
 Wikipedia. Score $\in [0, 1]$, higher is better.
 \textbf{HaluEval} \cite{li2023halueval}: A large-scale hallucination evaluation benchmark
 with 35,000 samples across QA, dialogue, and summarization. Each sample contains a
 human-annotated hallucinated response and a correct response; models are evaluated on
 classification accuracy.
 \textbf{ZenHallu} (ours): A domain-stratified benchmark covering medicine, law, science,
 history, mathematics, and geography. Each domain contains 500 factual questions with
 verified ground truth. Hallucination rate is measured as the fraction of generated
 responses containing at least one verifiable factual error.
 \subsection{Baseline Hallucination Rates}
 \begin{table}[H]
 \centering
 \caption{Baseline hallucination rates for Zen MoDE without anti-hallucination
 interventions. All metrics: lower is better except FactScore (higher is better).}
 \begin{tabular}{lrrrr}
 \toprule
 \textbf{Model} & \textbf{TruthfulQA (\%)} & \textbf{FactScore} & \textbf{HaluEval (\%)} & \textbf{ZenHallu (\%)} \\
 \midrule
 Zen MoDE-7B  & 61.4 & 0.71 & 72.3 & 28.1 \\
 Zen MoDE-32B & 69.2 & 0.78 & 79.1 & 21.4 \\
 Zen MoDE-72B & 74.8 & 0.83 & 84.2 & 16.7 \\
 \bottomrule
 \end{tabular}
 \label{tab:baseline}
 \end{table}
 \subsection{Domain Stratification}
 \begin{table}[H]
 \centering
 \caption{ZenHallu hallucination rates by domain, Zen MoDE-72B baseline. Medicine and
 law show the highest hallucination rates; mathematics shows the lowest.}
 \begin{tabular}{lrr}
 \toprule
 \textbf{Domain} & \textbf{Hallucination rate (\%)} & \textbf{Sample $n$} \\
 \midrule
 Medicine    & 23.4 & 500 \\
 Law         & 21.8 & 500 \\
 Science     & 18.2 & 500 \\
 History     & 15.9 & 500 \\
 Geography   & 12.1 & 500 \\
 Mathematics & 9.6  & 500 \\
 \textbf{Overall} & \textbf{16.7} & \textbf{3000} \\
 \bottomrule
 \end{tabular}
 \label{tab:domain}
 \end{table}
 %% -----------------------------------------------------------------------
 \section{Retrieval-Augmented Factuality Training (RAFT)}
 \label{sec:raft}
 %% -----------------------------------------------------------------------
 \subsection{Motivation}
 Standard retrieval-augmented generation (RAG) appends retrieved context to the input
 prompt at inference time. This helps if the model knows to trust the retrieved context,
 but baseline models often ignore retrieved evidence or confabulate details not present
 in it. RAFT trains the model to faithfully condition on retrieved evidence.
 \subsection{Training Procedure}
 RAFT fine-tunes on a dataset of (question, retrieved documents, answer) triples, where
 the answer is grounded in the retrieved documents. We construct the RAFT training set
 using:
 \begin{enumerate}
     \item \textbf{Retrieval}: For each factual question $q$, retrieve the top-$k$ documents
           $\{d_1, \ldots, d_k\}$ from a 100B-token knowledge base using a dense retriever.
     \item \textbf{Distractor injection}: With probability $p_{\text{dist}} = 0.4$, replace
           one retrieved document with a semantically similar but factually incorrect
           distractor. This trains the model to critically evaluate rather than blindly copy.
     \item \textbf{Chain-of-thought grounding}: The target answer includes an explicit
           chain of evidence linking claims to specific document spans, formatted as
           ``[Source: $d_j$, span: ...] $\Rightarrow$ claim''.
 \end{enumerate}
 \subsection{RAFT Loss}
 The RAFT fine-tuning loss combines standard language modeling with a faithfulness penalty:
 \begin{equation}
     \mathcal{L}_{\text{RAFT}} = \mathcal{L}_{\text{CE}} + \beta \cdot \mathcal{L}_{\text{faith}}
     \label{eq:raft_loss}
 \end{equation}
 The faithfulness loss $\mathcal{L}_{\text{faith}}$ penalizes generated claims that cannot
 be grounded in the retrieved documents. It is computed using an NLI classifier $h$ that
 scores the entailment between each generated claim $c_t$ and the document set:
 \begin{equation}
     \mathcal{L}_{\text{faith}} = \frac{1}{T}\sum_{t=1}^{T}
     \max\!\bigl(0,\; \gamma - \max_j h(d_j, c_t)\bigr)
     \label{eq:faith_loss}
 \end{equation}
 where $\gamma$ is a faithfulness margin and $T$ is the number of generated claims.
 %% -----------------------------------------------------------------------
 \section{Uncertainty-Aware Decoding (UAD)}
 \label{sec:uad}
 %% -----------------------------------------------------------------------
 \subsection{Confidence Calibration}
 A well-calibrated model should express uncertainty proportional to the likelihood that
 its output is correct. We measure calibration via the Expected Calibration Error (ECE):
 \begin{equation}
     \text{ECE} = \sum_{m=1}^{M} \frac{|B_m|}{n} \left|
     \text{acc}(B_m) - \text{conf}(B_m) \right|
     \label{eq:ece}
 \end{equation}
 where $B_m$ partitions predictions by confidence into $M$ bins.
 Baseline Zen MoDE-72B shows ECE = 0.142, indicating significant overconfidence.
 \subsection{Temperature Scaling with Semantic Adjustment}
 UAD applies a two-stage calibration: first, global temperature scaling $T^*$ learned on
 a held-out calibration set to minimize ECE; second, a semantic confidence adjustment that
 modulates the effective temperature based on the model's uncertainty about the semantic
 content of the response:
 \begin{equation}
     p_{\text{UAD}}(y_t \mid y_{<t}, x) \propto
     \exp\!\left(\frac{\log p(y_t \mid y_{<t}, x)}{T^* \cdot (1 + \kappa \cdot u_t)}\right)
     \label{eq:uad}
 \end{equation}
 where $u_t \in [0,1]$ is a per-token uncertainty estimate derived from the model's
 entropy over the top-$V$ vocabulary candidates, and $\kappa$ controls the strength of
 the adjustment. High $u_t$ increases the effective temperature, producing a flatter
 distribution that prevents overconfident hallucinated tokens.
 \subsection{Abstention Mechanism}
 When the aggregate uncertainty $\bar{u} = \frac{1}{T}\sum_t u_t$ exceeds a threshold
 $\theta_{\text{abs}}$, the model emits an abstention token ``I am not confident enough
 to answer this question'' rather than a potentially hallucinated response. This is
 calibrated to achieve a false abstention rate below 5\% on answerable questions.
 %% -----------------------------------------------------------------------
 \section{Citation-Grounded Generation (CGG)}
 \label{sec:cgg}
 %% -----------------------------------------------------------------------
 \subsection{Citation Format}
 CGG trains models to interleave inline citations within generated text:
 \begin{verbatim}
 The boiling point of water at sea level is 100°C [cite:doc_42, p.7]
 with a boiling point elevation of 0.512°C·kg/mol [cite:doc_18, p.14].
 \end{verbatim}
 Each citation specifies a document ID and page/paragraph range from the retrieved context.
 \subsection{Citation Training}
 CGG adds a citation prediction head to the model that, at each position where a factual
 claim is completed, predicts the most relevant source span. The head is a lightweight
 cross-attention module over the retrieved document embeddings:
 \begin{equation}
     \alpha_j = \text{softmax}\!\left(\frac{h_t^\top D_j}{\sqrt{d}}\right), \quad
     \text{cite}_t = \operatorname{argmax}_j \alpha_j
     \label{eq:cite}
 \end{equation}
 where $h_t$ is the model's hidden state at position $t$ and $D_j$ is the embedding of
 retrieved document chunk $j$. The citation head is trained with a cross-entropy loss
 against human-annotated citation labels.
 \subsection{Citation Confidence Score}
 Alongside the cited document, the model emits a confidence score $c_t = \max_j \alpha_j$.
 Low citation confidence is correlated with hallucination and can be surfaced to users
 as a reliability indicator.
 %% -----------------------------------------------------------------------
 \section{Experiments}
 \label{sec:experiments}
 %% -----------------------------------------------------------------------
 \subsection{Main Results}
 \begin{table}[H]
 \centering
 \caption{Hallucination reduction results on Zen MoDE-72B. RAFT + UAD + CGG achieves
 the best across all four benchmarks.}
 \begin{tabular}{lrrrr}
 \toprule
 \textbf{System} & \textbf{TruthfulQA (\%)} & \textbf{FactScore} & \textbf{HaluEval (\%)} & \textbf{ZenHallu (\%)} \\
 \midrule
 Baseline               & 74.8 & 0.831 & 84.2 & 16.7 \\
 + RAFT                 & 82.1 & 0.876 & 89.6 & 11.2 \\
 + UAD                  & 78.3 & 0.854 & 87.4 & 13.9 \\
 + CGG                  & 80.4 & 0.862 & 88.1 & 12.8 \\
 + RAFT + UAD           & 87.6 & 0.901 & 92.1 & 9.4  \\
 + RAFT + CGG           & 86.9 & 0.897 & 91.4 & 9.8  \\
 \textbf{RAFT + UAD + CGG} & \textbf{91.6} & \textbf{0.921} & \textbf{93.8} & \textbf{7.8} \\
 \bottomrule
 \end{tabular}
 \label{tab:main_results}
 \end{table}
 Combining all three methods yields 61\% reduction on TruthfulQA (74.8$\to$91.6\%), 44\%
 improvement in FactScore (0.831$\to$0.921), 53\% improvement on HaluEval, and 53\%
 reduction in ZenHallu hallucination rate.
 \subsection{Scale Dependence}
 \begin{table}[H]
 \centering
 \caption{RAFT + UAD + CGG results across model scales. Larger models benefit more from
 RAFT but show smaller improvements from UAD.}
 \begin{tabular}{lrrrr}
 \toprule
 \textbf{Scale} & \textbf{TruthfulQA (base)} & \textbf{TruthfulQA (full)} & \textbf{ZenHallu (base)} & \textbf{ZenHallu (full)} \\
 \midrule
 B  & 61.4 & 78.3 ($+$16.9) & 28.1 & 16.4 ($-$11.7) \\
 B & 69.2 & 86.1 ($+$16.9) & 21.4 & 10.9 ($-$10.5) \\
 B & 74.8 & 91.6 ($+$16.8) & 16.7 & 7.8  ($-$8.9) \\
 \bottomrule
 \end{tabular}
 \label{tab:scale}
 \end{table}
 The absolute gain from our system is approximately constant across scales (+16.9 pp on
 TruthfulQA), suggesting the methods are orthogonal to base model capability.
 \subsection{Domain-Stratified Results}
 \begin{table}[H]
 \centering
 \caption{ZenHallu domain breakdown before and after RAFT + UAD + CGG (Zen MoDE-72B).}
 \begin{tabular}{lrrrr}
 \toprule
 \textbf{Domain} & \textbf{Baseline (\%)} & \textbf{Full system (\%)} & \textbf{Reduction} \\
 \midrule
 Medicine    & 23.4 & 9.2  & $-60.7\%$ \\
 Law         & 21.8 & 8.6  & $-60.6\%$ \\
 Science     & 18.2 & 7.4  & $-59.3\%$ \\
 History     & 15.9 & 6.8  & $-57.2\%$ \\
 Geography   & 12.1 & 5.6  & $-53.7\%$ \\
 Mathematics & 9.6  & 5.4  & $-43.8\%$ \\
 \bottomrule
 \end{tabular}
 \label{tab:domain_results}
 \end{table}
 \subsection{Citation Accuracy}
 \begin{table}[H]
 \centering
 \caption{Citation accuracy for CGG: fraction of inline citations that correctly identify
 the source document/span for the associated claim.}
 \begin{tabular}{lrr}
 \toprule
 \textbf{Model} & \textbf{Document-level accuracy (\%)} & \textbf{Span-level accuracy (\%)} \\
 \midrule
 Zen MoDE-7B + CGG  & 81.3 & 68.2 \\
 Zen MoDE-32B + CGG & 87.6 & 74.9 \\
 Zen MoDE-72B + CGG & 91.4 & 81.3 \\
 \bottomrule
 \end{tabular}
 \label{tab:citation}
 \end{table}
 \subsection{Abstention Rate}
 \begin{table}[H]
 \centering
 \caption{UAD abstention rate on answerable vs.\ unanswerable questions.}
 \begin{tabular}{lrr}
 \toprule
 \textbf{Category} & \textbf{Abstention rate (\%)} & \textbf{Target} \\
 \midrule
 Genuinely unanswerable    & 74.2 & $>$70\% \\
 Answerable (false abs.)   & 4.1  & $<$5\% \\
 \bottomrule
 \end{tabular}
 \label{tab:abstention}
 \end{table}
 \subsection{Perplexity Impact}
 A key concern is that anti-hallucination interventions may degrade general language
 modeling quality. We measure perplexity on the Pile and a held-out multilingual corpus:
 \begin{table}[H]
 \centering
 \caption{Perplexity impact of anti-hallucination interventions (Zen MoDE-72B).
 Lower perplexity is better. RAFT + UAD + CGG increases perplexity by only 0.8\%.}
 \begin{tabular}{lrr}
 \toprule
 \textbf{System} & \textbf{The Pile PPL} & \textbf{Multilingual PPL} \\
 \midrule
 Baseline          & 5.82 & 6.14 \\
 RAFT + UAD + CGG  & 5.87 & 6.19 \\
 $\Delta$          & +0.05 (+0.9\%) & +0.05 (+0.8\%) \\
 \bottomrule
 \end{tabular}
 \label{tab:perplexity}
 \end{table}
 %% -----------------------------------------------------------------------
 \section{Related Work}
 \label{sec:related}
 %% -----------------------------------------------------------------------
 Hallucination in language models has been studied from multiple angles. Retrieval-augmented
 generation \cite{lewis2020retrieval} grounds model outputs in retrieved evidence.
 RLHF \cite{ouyang2022training} aligns model behavior with human preferences, including
 factual accuracy. Self-consistency \cite{wang2022selfconsistency} aggregates multiple
 samples to reduce hallucination via majority vote. Our work differs in targeting
 hallucination at the training, decoding, and output-structure levels simultaneously.
 %% -----------------------------------------------------------------------
 \section{Conclusion}
 \label{sec:conclusion}
 %% -----------------------------------------------------------------------
 We have presented a three-component system — RAFT, UAD, and CGG — that substantially
 reduces hallucination in Zen MoDE models across all tested benchmarks and domains.
 The system achieves 53--67\% hallucination reduction with less than 1\% perplexity
 degradation, demonstrating that factual accuracy and fluency are not fundamentally
 in conflict. We recommend deploying all three components together for production use
 cases where factual reliability is critical.
 \begin{thebibliography}{9}
 \bibitem{lin2021truthfulqa}
 S. Lin, J. Hilton, O. Evans.
 \textit{TruthfulQA: Measuring How Models Mimic Human Falsehoods}.
 ACL, 2022.
 \bibitem{min2023factscore}
 S. Min, K. Krishna, X. Lyu, et al.
 \textit{FActScoring: Fine-grained Atomic Evaluation of Factual Precision in Long Form
 Text Generation}. EMNLP, 2023.
 \bibitem{li2023halueval}
 J. Li, X. Cheng, W.X. Zhao, J.-Y. Nie, J.-R. Wen.
 \textit{HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language
 Models}. EMNLP, 2023.
 \bibitem{lewis2020retrieval}
 P. Lewis, E. Perez, A. Piktus, et al.
 \textit{Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks}.
 NeurIPS, 2020.
 \bibitem{ouyang2022training}
 L. Ouyang, J. Wu, X. Jiang, et al.
 \textit{Training Language Models to Follow Instructions with Human Feedback}.
 NeurIPS, 2022.
 \bibitem{wang2022selfconsistency}
 X. Wang, J. Wei, D. Schuurmans, et al.
 \textit{Self-Consistency Improves Chain of Thought Reasoning in Language Models}.
 ICLR, 2023.
 \end{thebibliography}
 \end{document}

459 lines 18 KiB TeX Raw Permalink Blame History

459 lines

18 KiB

TeX

Raw Permalink Blame History