mirror of
https://github.com/hanzoai/papers.git
synced 2026-10-29 07:34:31 +00:00
zeekayandHanzo Dev
03c6289dfc
Design, prior art and a measurement protocol for a weighted inverted index over a mixed sparse basis and part-level routing before per-part HNSW in a ClickHouse-derived store. Each component is marked prior art or proposed. The protocol fits C(N) = aN^alpha to parts, granules and bytes touched per correct answer at a held recall. No numbers: every results cell is a placeholder read from results-data.tex until the baseline lane writes it. Co-authored-by: Hanzo Dev <[email protected]>
40 lines
1.8 KiB
TeX
40 lines
1.8 KiB
TeX
% Read from the runs named below -- do not hand-edit values.
|
|
%
|
|
% PLACEHOLDER. No run has written this file yet, so every \res{...} in the
|
|
% paper prints XXX. The lane that will write it is in hanzoai/benchmarks and is
|
|
% not yet published; its first pass measures upstream ClickHouse 26.4.2.10 at
|
|
% N = 1e5 .. 1e7; the routed store and the larger corpora follow.
|
|
%
|
|
% Contract. One line per measured value:
|
|
%
|
|
% \setres{<system>/<N>/<metric>}{<value>}
|
|
%
|
|
% system scan exhaustive scoring, no index
|
|
% base upstream ClickHouse: primary index, skipping indexes,
|
|
% text index, one vector similarity index per part
|
|
% route the design of this paper
|
|
% oracle the granules and parts holding the exact top-k
|
|
% N 1e5 1e6 1e7 1e8 1e9
|
|
% metric parts parts read (ProfileEvents SelectedParts)
|
|
% granules granules read (SelectedMarks)
|
|
% mib bytes fetched, MiB (ReadCompressedBytes; OSReadBytes cold)
|
|
% bpc KiB fetched per correct answer
|
|
% recall recall@k against the exact top-k
|
|
% p50, p99 query latency, ms, warm
|
|
%
|
|
% and one line per fitted exponent of C(N) = a N^alpha, with its interval,
|
|
% where C is parts, granules and bytes (mib) touched per correct answer, and
|
|
% routing summaries examined per query:
|
|
%
|
|
% \setres{fit/<system>/<metric>}{alpha [lo, hi]}
|
|
%
|
|
% Every row also needs the conditions it was taken under, which the harness
|
|
% writes as its own keys and the paper prints in the caption:
|
|
%
|
|
% \setres{run/host}{...} one machine, named
|
|
% \setres{run/version}{...} store build
|
|
% \setres{run/k}{...} top-k
|
|
% \setres{run/recall}{...} the recall target held across N
|
|
% \setres{run/queries}{...} query set and its size
|
|
% \setres{run/parts/<N>}{...} part count and size distribution at each N
|