papers/hanzo-search/sections/10-discussion.tex

\section{Discussion}

\label{sec:discussion}

\subsection{Freshness vs. Accuracy Trade-off}

For current events queries, there is a tension between using the most recent (potentially unreliable) sources and older (well-verified) sources. We address this by:
\begin{enumerate}[leftmargin=1.4em]
    \item Time-decayed relevance scoring that boosts recent documents for current-events queries.
    \item Multi-source verification: claims from a single recent source require corroboration.
    \item Explicit uncertainty markers in the UI (``Based on reports as of [date]'').
\end{enumerate}

\subsection{Limitations}

\begin{enumerate}[leftmargin=1.4em]
    \item \textbf{Index coverage}: Our 50M page index covers a fraction of the web; long-tail queries may lack relevant documents.
    \item \textbf{Multimodal}: Current system handles text queries and text answers; image and video understanding are not yet supported.
    \item \textbf{Personalization}: Limited to language preference and history-based re-ranking; no deep user modeling.
    \item \textbf{Real-time}: Index updates lag by 4--6 hours for new content; breaking news queries may return stale results.
\end{enumerate}

\subsection{Ethical Considerations}

Generative search introduces unique ethical challenges that deserve careful treatment:

\begin{enumerate}[leftmargin=1.4em]
    \item \textbf{Source fairness}: Generated answers may inadvertently favor large, well-optimized websites over smaller sources with equally valid information. We mitigate this by diversifying source selection and applying source-size-independent relevance scoring.
    \item \textbf{Bias amplification}: LLMs may amplify biases present in training data when synthesizing answers. We employ a bias detection classifier that flags potentially biased responses for human review.
    \item \textbf{Publisher impact}: By synthesizing answers directly, generative search may reduce traffic to original content publishers. We address this by prominently displaying source attributions and encouraging click-through.
    \item \textbf{Misinformation risk}: Incorrect generated answers carry more authority than a list of links, as users may trust a synthesized answer without checking sources. Our hallucination detection system and source verification are designed to minimize this risk.
\end{enumerate}

\subsection{Scalability Analysis}

We analyze the scaling properties of each pipeline stage:

\begin{table}[H]
\centering
\small
\begin{tabular}{lccc}
\toprule
\textbf{Stage} & \textbf{Complexity} & \textbf{Scalable?} & \textbf{Bottleneck} \\
\midrule
Query parsing & $O(|q|)$ & Yes & CPU \\
BM25 retrieval & $O(|q| \log N)$ & Yes (sharded) & I/O \\
Dense retrieval & $O(d \log N)$ & Yes (HNSW) & Memory \\
Re-ranking & $O(k \cdot |q|)$ & Yes (GPU) & GPU \\
Generation & $O(|C| + |a|)$ & Limited & LLM \\
\bottomrule
\end{tabular}
\caption{Scalability analysis by pipeline stage. $N$ = index size, $k$ = re-rank candidates, $d$ = embedding dimension.}
\label{tab:scalability}
\end{table}

The LLM generation stage is the primary scaling bottleneck. We address this through:
\begin{itemize}[leftmargin=1.1em]
    \item \textbf{Caching}: Frequently asked queries are served from a semantic cache (cache hit rate: 12\%).
    \item \textbf{Streaming}: Responses begin streaming before generation is complete, reducing perceived latency.
    \item \textbf{Model tiering}: Simple factual queries use faster, smaller models; complex synthesis queries use larger models.
    \item \textbf{Horizontal scaling}: LLM serving is distributed across multiple inference endpoints via the Hanzo LLM Gateway.
\end{itemize}

\subsection{Future Work}

\begin{itemize}[leftmargin=1.1em]
    \item \textbf{Multimodal search}: Extend to image, video, and audio retrieval with cross-modal generation.
    \item \textbf{Conversational refinement}: Support multi-turn search sessions with context carryover.
    \item \textbf{Personal knowledge}: Integrate user documents and notes into the retrieval pipeline.
    \item \textbf{Collaborative search}: Enable shared search sessions with annotation and discussion.
    \item \textbf{Structured data integration}: Query structured databases (Wikidata, product catalogs) alongside unstructured text for richer answers.
    \item \textbf{Federated retrieval}: Support retrieval from private organizational indices without exposing document content to the central system.
\end{itemize}