\documentclass[11pt]{article} \usepackage{shared/hanzocover} \usepackage[utf8]{inputenc} \usepackage[T1]{fontenc} \usepackage{amsmath,amssymb} \usepackage{graphicx} \usepackage{booktabs} \usepackage{xcolor} \usepackage[hidelinks]{hyperref} \emergencystretch=1.5em \title{Context Engine:\\ Typed Retrieval Across Memory and Code} \author{Zach Kelling\thanks{Corresponding author: research@hanzo.ai} \\ \textit{Hanzo Industries Inc (Techstars '17)} \\ San Francisco, California \\ \texttt{https://hanzo.ai}} \date{Draft, revision~0} \begin{document} \hanzocoverpage \twocolumn[ \begin{@twocolumnfalse} \maketitle \begin{abstract} \noindent The companion paper showed that long-term conversational memory is a linkage problem: a question reaches its evidence through a fact, an entity or a date more often than through wording, and a second index of typed links lifts complete-evidence recall on multi-hop questions from 22.7\% to 36.2\% with nothing learned. This paper asks whether the same engine, unchanged in shape, is the right retriever for a software repository. A repository is connected information with an unusually honest ground truth: the compiler and the language server know which symbol a reference resolves to, which function calls which, which test covers which file, which commit last touched a line. We describe the Context Engine as one abstraction---raw artefact, typed facts, canonical entities, time, provenance, and a read path that plans, seeds, expands along typed edges within a budget, scores once and rewrites the query after each resolved hop---and its two instantiations: conversation, where the edges are people, events, sessions and dates; and code, where they are symbols, definitions, references, implementations, callers, callees, types, files, commits, issues, tests and runtime logs. We set out a deterministic benchmark whose gold links are emitted by the language server rather than annotated by hand, and an evaluation plan on RepoBench, CrossCodeEval and RepoProbe with retrieval measured separately from generation. Results are pending and the sections that will carry them are marked. The claim this draft is written to test is narrow: that semantic similarity is one candidate generator inside a typed retrieval system, and that the system beats similarity alone on both kinds of corpus. \end{abstract} \vspace{1em} \end{@twocolumnfalse} ] \section{One Abstraction} Retrieval systems for memory and retrieval systems for code are usually built by different teams from different papers. Both start from the same mistake: embed the unit of text, embed the request, return the nearest neighbours, and hope the answer resembles the question. In a conversation the turn that answers ``where did she work after leaving Acme'' contains neither ``work'' nor ``Acme''. In a repository the file that explains why a request handler retries contains neither ``retry'' nor the handler's name; it is the interface the handler implements, two directories away, reached by a typed edge the compiler already knows. The Context Engine is the abstraction the companion paper \cite{contextmemory} arrived at for memory, restated so that nothing in it names a conversation: \begin{itemize} \item \textbf{raw} --- the artefact as received, immutable, with source and observation time. \item \textbf{facts[]} --- atomic typed statements about the artefact, each with a canonical subject, a relation, a canonical object or a literal, a time interval where one applies, a confidence, an embedding and a pointer to the exact span it came from. \item \textbf{entities[]} --- canonical identifiers resolved by type. \item \textbf{events[]} --- groups of facts that belong together. \item \textbf{supersedes[]} --- what this artefact replaces. \item \textbf{abstraction} and \textbf{anchors[]} --- one short phrase for what the artefact is about and several alternative entry points. \end{itemize} And the read path: understand the request into operators; seed from dense, lexical, fact, entity and time generators with small pools; expand along typed edges within a bounded budget; score every candidate once with recorded contributions; return the least raw context that proves the answer; and if a hop resolved something, rebuild the query from the resolved state before the next hop. \section{Two Instantiations} \begin{table*}[t] \centering \footnotesize \begin{tabular}{p{24mm}p{58mm}p{74mm}} \toprule Record field & Conversation & Code \\ \midrule raw & a turn & a file, a hunk, a log line \\ entity & person, place, org, work, date & symbol, type, module, file, commit, issue, test \\ relation & said, did, plans, moved, met & defines, references, implements, calls, imports, extends, covers, touched \\ event & the trip, the interview & the change set, the incident, the release \\ time & session date, event time & commit time, blame, run time \\ supersedes & a later fact about the same relation & a later commit on the same lines \\ abstraction & ``Caroline moved from Sweden'' & ``retry policy for outbound HTTP'' \\ anchors & names, places, dates & identifiers, error strings, config keys \\ provenance & conversation, session, turn & repo, path, line range, commit \\ \bottomrule \end{tabular} \caption{The same record, two corpora. Every edge in the code column is emitted by tools that already exist: the language server, the compiler, the version control system, the test runner.} \label{tab:two} \end{table*} \subsection{Conversation} The memory instantiation is described and measured in the companion paper. Its edges are extracted by a model and are probabilistic; its gold evidence is human-annotated and, as LoCoMo-Conv showed \cite{locomoconv}, incomplete. This is the harder of the two settings to evaluate honestly, which is why the companion paper spends a section on what is being measured. \subsection{Code} The code instantiation is the easier of the two to evaluate and the harder to build, because the typed edges are many and exact. Candidate generators: \begin{itemize} \item dense search over facts about symbols (a symbol's abstraction: its signature, docstring and the one-line purpose an extractor assigns); \item lexical search over identifiers and literal strings, which in code carries more of the signal than in prose; \item the symbol index: exact matches on canonical symbol identifiers, with aliases and re-exports resolved; \item typed edges from the language server: definition, references, implementations, type hierarchy, callers and callees; \item file and module structure: siblings, parents, the AST path; \item history: the commits that touched a span, their messages, the issues they close, blame; \item tests: the test cases whose coverage includes a span, and the runtime logs a failing test emitted. \end{itemize} Expansion is bounded exactly as in memory---at most $n$ hops along at most $m$ edges of each type---and every candidate carries the edge it entered by. A stateful hop is natural here: resolving ``the retry logic'' to a symbol on hop one turns hop two into \emph{implementations of that symbol's interface, in files touched by the commit that introduced it}, a query that no embedding of the original sentence would find. \section{A Deterministic Gold Benchmark} \label{sec:lsp} Hand-annotated retrieval gold is expensive and incomplete. For code it is also unnecessary. We build the Hanzo LSP benchmark as follows. For each of a set of repositories, run the language server and the test runner and record every definition, reference, implementation, call, import and coverage edge. Generate questions mechanically from the graph---``what implements X'', ``who calls Y'', ``which test covers Z'', ``what changed in this function in the last release''---and their answers as the exact set of spans the graph names. Phrase each question three ways, one of them without the symbol's name, so that lexical matching alone cannot answer it. Score retrieval as ALL and ANY over the gold spans at $k$, as in the companion paper, and report MRR, tokens delivered and latency. The gold is complete by construction, so the SUPPORTED grade of the memory paper is not needed here; EXACT is the whole story. This is the benchmark on which a difference between similarity and linkage will be least arguable. \section{External Lanes} \paragraph{RepoBench.} \cite{repobench} separates retrieval (RepoBench-R), completion (RepoBench-C) and the pipeline (RepoBench-P) for Python and Java. We report RepoBench-R as a retrieval number and RepoBench-P with a fixed generator across rows, so a gain is attributable to retrieval. \paragraph{CrossCodeEval.} \cite{crosscodeeval} requires cross-file context to complete code in Python, Java, TypeScript and C\#. We report exact match and edit similarity with the same generator across rows and the retrieval budget held constant. \paragraph{RepoProbe.} \cite{repoprobe} asks open-ended architectural questions drawn from GitHub Discussions across real repositories and scores answers against checklists of atomic verifiable facts. We report the checklist score and, separately, the fraction of checklist facts whose supporting span was retrieved, which is the retrieval question this paper is about. \paragraph{ContextBench.} \cite{contextbench} measures context retrieval in coding agents directly and shares a name with the Hanzo benchmark described in the companion paper; the two are distinct, and we run this one as an external lane where its harness permits. \section{Evaluation Plan} Every table carries the rows of the companion paper's protocol: semantic only; plus lexical; plus facts; plus entities; plus timeline; plus adjacency (here: siblings in the file and the AST); plus typed graph; plus iterative hops; full; and the failed variants---pseudo-relevance feedback, generic multi-query, chain search, surface-identifier expansion, global fusion. Retrieval is measured separately from generation. A declared dev set of repositories is used for every parameter choice; the commit is frozen; the test repositories are run once. \section{Results} \label{sec:results} The first external lane is measured; the deterministic benchmark of Section~\ref{sec:lsp} is still pending and its table is left as such. \paragraph{RepoBench-R.} Python, both settings, the dev split (\texttt{train\_easy[0:100] + train\_hard[0:100]}) for every weight and the test split (\texttt{test\_easy[0:250] + test\_hard[0:250]}) run once at the frozen commit. Each item is a completion point with about twelve candidate snippets from other files of the repository, one of which the next line needs. The dense generator embeds the last lines above the point and each candidate with all-MiniLM-L6-v2; BM25 is queried with the last three lines; the typed links are regular expressions over \texttt{import}, \texttt{from \ldots\ import}, \texttt{def}, \texttt{class} and identifiers — a candidate that defines an imported name the code above has not used yet, or has. No model reads anything and nothing reads the answer. Table~\ref{tab:repobench} is the memory result again in another domain. Dense similarity alone finds the right snippet first 18.8\% of the time in the cross-file-first setting; the typed links alone, with no model, 31.4\%; the two together 33.2\%, with MRR 40.8 $\rightarrow$ 52.3 and R@5 72.8 $\rightarrow$ 79.4. On the hard items, where the file's own context gives the least away, dense retrieval finds the snippet first 12.8\% of the time and the links 25.6\%. In the cross-file-random setting the module has already been used above the point, the ``unused import'' prior no longer applies, and the links' margin narrows: 24.6 $\rightarrow$ 31.2 R@1 with dense plus links. Retrieval takes under a millisecond per item. The sign of the ``used'' term flips between the two settings, and was chosen on dev for each; that is the one setting-specific decision in the lane, and the sweep table records it. \begin{table*}[t] \centering \footnotesize \input{tables/code-repobench} \caption{RepoBench-R Python, test split, retrieval only, one row per configuration. R@$k$ is whether the gold snippet is in the top $k$ of about twelve candidates; \emph{hard} is the benchmark's own level. Weights frozen on dev; test run once.} \label{tab:repobench} \end{table*} \begin{table*}[t] \centering \small \begin{tabular}{lrrrr} \toprule Hanzo LSP benchmark & ALL@10 & ANY@10 & MRR & tokens/q \\ \midrule semantic only & \ldots & \ldots & \ldots & \ldots \\ % TODO run code-lsp-test-semantic + lexical & \ldots & \ldots & \ldots & \ldots \\ + symbol index & \ldots & \ldots & \ldots & \ldots \\ + typed edges & \ldots & \ldots & \ldots & \ldots \\ + history and tests & \ldots & \ldots & \ldots & \ldots \\ + iterative hops & \ldots & \ldots & \ldots & \ldots \\ full & \ldots & \ldots & \ldots & \ldots \\ \bottomrule \end{tabular} \caption{Pending: retrieval on the deterministic benchmark of Section~\ref{sec:lsp}.} \end{table*} \begin{table*}[t] \centering \small \begin{tabular}{lrrr} \toprule External lane & retrieval & end-to-end & tokens/q \\ \midrule RepoBench-R (retrieval, Table~\ref{tab:repobench}) & 52.3 MRR & --- & --- \\ CrossCodeEval & \ldots & \ldots & \ldots \\ % TODO run code-cceval-test-full RepoProbe & \ldots & \ldots & \ldots \\ % TODO run code-repoprobe-test-full \bottomrule \end{tabular} \caption{Pending: external lanes, full configuration, one generator across rows.} \end{table*} \section{Related Work} Repository-level retrieval for completion is the subject of RepoBench \cite{repobench} and CrossCodeEval \cite{crosscodeeval}; architectural comprehension of RepoProbe \cite{repoprobe}; context retrieval for coding agents of ContextBench \cite{contextbench}. On the memory side the closest designs are AnchorMem \cite{anchormem}, which anchors retrieval on atomic facts and returns raw context, Memora \cite{memora}, which indexes an abstraction and cue anchors and keeps the value unindexed, and APEX-MEM \cite{apexmem}, an entity-centric temporal property graph with append-only history. HippoRAG \cite{hipporag} is the nearest graph-retrieval ancestor. None of these is applied to code, and none of the code benchmarks is approached with a memory system; the point of this draft is that the gap is an accident of who built what. \section{What Would Falsify It} If the full configuration does not beat semantic-only retrieval on the deterministic benchmark by a margin outside the bootstrap interval, the abstraction is wrong for code and the paper says so. If it wins there but not on RepoBench-P with a fixed generator, the gain is not reaching the task. If it wins on code but the memory numbers of the companion paper do not hold on their test split, the engine is two engines with one name. Each of these is a reportable outcome. \begin{thebibliography}{99} \bibitem{contextmemory} Z. Kelling. Linkage, Not Similarity: Context-Sensitive Retrieval for Long-Term Agent Memory. Hanzo Research, 2026. \bibitem{locomoconv} W.-Y. Chang, Y.-N. Chen. When Users Don't Ask: Benchmarking Context-Driven Memory Retrieval in Conversational Agents. arXiv:2609.03467, September 2026. \bibitem{repobench} T. Liu, C. Xu, J. McAuley. RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems. arXiv:2306.03091, 2023. \bibitem{crosscodeeval} Y. Ding, Z. Wang, W. U. Ahmad, H. Ding, M. Tan, N. Jain, M. K. Ramanathan, R. Nallapati, et al. CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion. arXiv:2310.11248, 2023. \bibitem{repoprobe} Y. Yang, A. Wu, J. Luo, R. Xuan, Z. Hu, Y. Liu, Z. Qin. RepoProbe: Benchmarking Architecture-Aware Repository Comprehension with Checklists. arXiv:2608.04783, August 2026; ASE 2026. \bibitem{contextbench} H. Li, L. Zhu, B. Zhang, R. Feng, J. Wang, Y. Pan, E. T. Barr, F. Sarro, et al. ContextBench: A Benchmark for Context Retrieval in Coding Agents. arXiv:2602.05892, 2026. \bibitem{anchormem} Z. Shen, S. Cheng, Z. Guo, W. Wang, Y. Wang, H. Huang. AnchorMem: Anchored Facts with Associative Contexts for Building Memory in Large Language Models. arXiv:2604.17377, April 2026. \bibitem{memora} M. Xia, X. Zhang, S. Dixit, P. Harimurugan, R. Wang, V. R\"uhle, R. Sim, C. Bansal, et al. Memora: A Harmonic Memory Representation Balancing Abstraction and Specificity. arXiv:2602.03315, 2026; ICML 2026. \bibitem{apexmem} P. Banerjee, M. Moshtaghi, S. Subramanian, A. Misra, A. Chadha. APEX-MEM: Agentic Semi-Structured Memory with Temporal Reasoning for Long-Term Conversational AI. arXiv:2604.14362, April 2026; ACL 2026. \bibitem{hipporag} B. Jim\'enez Guti\'errez, Y. Shu, Y. Gu, M. Yasunaga, Y. Su. HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models. arXiv:2405.14831, 2024. \end{thebibliography} \end{document}