mirror of
https://github.com/hanzoai/papers.git
synced 2026-10-29 07:34:31 +00:00
zeekayandHanzo Dev
03c6289dfc
Design, prior art and a measurement protocol for a weighted inverted index over a mixed sparse basis and part-level routing before per-part HNSW in a ClickHouse-derived store. Each component is marked prior art or proposed. The protocol fits C(N) = aN^alpha to parts, granules and bytes touched per correct answer at a held recall. No numbers: every results cell is a placeholder read from results-data.tex until the baseline lane writes it. Co-authored-by: Hanzo Dev <[email protected]>
35 lines
1.9 KiB
TeX
35 lines
1.9 KiB
TeX
\section{Conclusion}
|
|
\label{sec:conclusion}
|
|
|
|
Retrieval inside a merge-based column store is served today by structures that
|
|
are built per part and consulted per part, so the cost of a similarity query
|
|
follows the number of parts rather than the number of rows that could answer
|
|
it. The design here adds two things to that layout: a weighted inverted file
|
|
over a basis whose elements may be words, entities, code symbols, learned
|
|
concepts, relations or motifs, with block maxima on the store's own granules;
|
|
and a summary per region and per part --- centroid, covering radius, per-element
|
|
maxima, counting statistics --- aggregated into a structure the planner reads
|
|
to exclude parts before it opens them.
|
|
|
|
The components are mostly not new, and Table~\ref{tab:status} says which are
|
|
which. The arrangement is the contribution: bounds from metric indexing and
|
|
dynamic pruning, statistics from distributed resource selection, and a
|
|
maintenance path through the merges a column store already performs.
|
|
|
|
What the design owes is a measurement, and the shape of the claim decides the
|
|
measurement. Expected cost scaling with the relevant neighbourhood is a
|
|
statement about a query distribution at a fixed recall target, so the protocol
|
|
holds recall, fits an exponent with an interval to the parts, granules and
|
|
bytes touched per correct answer, compares against the granules that hold the
|
|
answer, and runs the queries the design is expected to fail on as their own
|
|
stratum. Until that lane reports, this paper is a design and a protocol, and
|
|
its results section holds placeholders.
|
|
|
|
\section*{Reproducing}
|
|
|
|
The lane that fills this paper's tables is in \texttt{hanzoai/benchmarks} and
|
|
has not been published. Its first pass runs upstream ClickHouse~26.4.2.10 at
|
|
$N = 10^5$ to $10^7$. The harness writes \texttt{results-data.tex} directly; the
|
|
tables in \S\ref{sec:results} read it, and a key it does not define prints
|
|
\ph{}.
|