mirror of
https://github.com/hanzoai/papers.git
synced 2026-10-22 17:20:17 +00:00
A skeleton for sparse-basis indexes and part-level routing in a column store
Design, prior art and a measurement protocol for a weighted inverted index over a mixed sparse basis and part-level routing before per-part HNSW in a ClickHouse-derived store. Each component is marked prior art or proposed. The protocol fits C(N) = aN^alpha to parts, granules and bytes touched per correct answer at a held recall. No numbers: every results cell is a placeholder read from results-data.tex until the baseline lane writes it. Co-authored-by: Hanzo Dev <[email protected]>
This commit is contained in:
Hanzo Dev
@@ -0,0 +1,113 @@
|
||||
% Sparse-Basis Inverted Indexes and Part-Level Routing for Retrieval over Column Stores
|
||||
\documentclass[11pt]{article}
|
||||
\usepackage{../shared/hanzocover}
|
||||
\usepackage{amsmath, amssymb, amsthm}
|
||||
\usepackage{booktabs}
|
||||
\usepackage{graphicx}
|
||||
\usepackage{xcolor}
|
||||
\usepackage{enumitem}
|
||||
\usepackage{array}
|
||||
\usepackage{listings}
|
||||
\input{../shared/lstlang}
|
||||
\usepackage{hyperref}
|
||||
\hypersetup{colorlinks=true,linkcolor=black,citecolor=blue,urlcolor=blue}
|
||||
|
||||
% Status of every component, so a reader can tell what is established from what
|
||||
% this paper proposes, without inferring it from the tone of a sentence.
|
||||
\newcommand{\prior}{\textcolor{purple!70!black}{\small[prior art]}}
|
||||
\newcommand{\proposed}{\textcolor{orange!70!black}{\small[proposed]}}
|
||||
\newcommand{\meas}{\textcolor{blue!60!black}{\small[measured]}}
|
||||
|
||||
% Measured constants are read from results-data.tex, which the harness writes.
|
||||
% A key that file does not define prints XXX, so a table cannot show a number
|
||||
% that no run produced, and a number cannot drift from its measurement without
|
||||
% the drift showing on the page.
|
||||
\newcommand{\ph}{\textbf{XXX}}
|
||||
\newcommand{\setres}[2]{\expandafter\def\csname res:#1\endcsname{#2}}
|
||||
\newcommand{\res}[1]{\ifcsname res:#1\endcsname\csname res:#1\endcsname\else\ph\fi}
|
||||
\input{results-data}
|
||||
|
||||
% One row of the cost table: the key prefix, then the label printed for it.
|
||||
\newcommand{\costrow}[2]{#2 & \res{#1/parts} & \res{#1/granules} & \res{#1/mib}
|
||||
& \res{#1/bpc} & \res{#1/recall} & \res{#1/p50} & \res{#1/p99} \\}
|
||||
% One row of the exponent table.
|
||||
\newcommand{\fitrow}[2]{#2 & \res{fit/#1/parts} & \res{fit/#1/granules}
|
||||
& \res{fit/#1/mib} & \res{fit/#1/routing} \\}
|
||||
|
||||
\theoremstyle{definition}
|
||||
\newtheorem{definition}{Definition}
|
||||
\theoremstyle{plain}
|
||||
\newtheorem{proposition}{Proposition}
|
||||
\theoremstyle{remark}
|
||||
\newtheorem{remark}{Remark}
|
||||
|
||||
\title{Sparse-Basis Inverted Indexes and Part-Level Routing\\
|
||||
for Retrieval over Column Stores\\[0.35em]
|
||||
{\large A design, its prior art, and a protocol for measuring\\
|
||||
how retrieval cost grows with corpus size}}
|
||||
\author{
|
||||
Hanzo AI Research\thanks{Corresponding author: [email protected]} \\
|
||||
\textit{Hanzo Industries Inc (Techstars '17)} \\
|
||||
Los Angeles, California \\
|
||||
\texttt{https://hanzo.ai}
|
||||
}
|
||||
\date{Draft, 2026-09-22 \quad No measurements are reported in this version}
|
||||
|
||||
\begin{document}
|
||||
|
||||
\hanzocoverpage
|
||||
\maketitle
|
||||
|
||||
\begin{abstract}
|
||||
\noindent
|
||||
We describe a retrieval design for a merge-based column store whose rows carry
|
||||
sparse representations over a mixed basis --- words, entities, code symbols,
|
||||
learned concepts, relations and structural motifs --- alongside dense
|
||||
embeddings, and we specify how its cost is to be measured. The index is a
|
||||
weighted inverted file: one posting list per basis element, holding a row
|
||||
identifier and a weight, so a query written over the same basis is scored only
|
||||
over the posting lists of the basis elements it activates, with top-$k$
|
||||
evaluation under block-max pruning. Above it sits a routing structure built per
|
||||
MergeTree part --- regions described by a centroid, a covering radius, the
|
||||
maximum weight each basis element reaches inside the region, and counting
|
||||
statistics --- aggregated into a table-level structure the planner consults to
|
||||
exclude parts before any per-part index is searched, including the per-part
|
||||
approximate-nearest-neighbour graph a column store already builds.
|
||||
|
||||
Most of the parts are established, and we say which. Weighted inverted files
|
||||
with dynamic pruning, cluster- and shard-level routing by collection
|
||||
statistics, covering-radius bounds from metric indexes, and per-segment vector
|
||||
indexes are prior art. What is proposed here is their arrangement inside the
|
||||
part lifecycle of a merge-based store: one weighted index over a heterogeneous
|
||||
basis, block maxima aligned to granules, region summaries that merge when parts
|
||||
merge, and selection of parts by an upper bound on the score they can still
|
||||
contribute before any approximate search runs.
|
||||
|
||||
The claim we intend to test is narrow. At a fixed query distribution and a
|
||||
fixed recall target, expected retrieval cost should grow with the size of the
|
||||
relevant neighbourhood rather than with the size of the corpus. It is not a
|
||||
worst-case guarantee: a query whose relevant set is the whole corpus costs time
|
||||
linear in the corpus, under this design and under any other. The protocol fits
|
||||
$C(N) = aN^{\alpha}$ to parts, granules and bytes touched at a held recall
|
||||
target, and reports bytes fetched per correct answer, recall@$k$ and latency
|
||||
percentiles across corpus sizes, against upstream ClickHouse as the baseline.
|
||||
This version reports no measurements. The baseline lane is only starting, and
|
||||
every cell in the results section is a placeholder until it writes one.
|
||||
\end{abstract}
|
||||
|
||||
\input{sections/01-introduction}
|
||||
\input{sections/02-setting}
|
||||
\input{sections/03-index}
|
||||
\input{sections/04-routing}
|
||||
\input{sections/05-method}
|
||||
\input{sections/06-results}
|
||||
\input{sections/07-related}
|
||||
\input{sections/08-limits}
|
||||
\input{sections/99-conclusion}
|
||||
|
||||
{\small
|
||||
\bibliographystyle{plain}
|
||||
\bibliography{references}
|
||||
}
|
||||
|
||||
\end{document}
|
||||
@@ -0,0 +1,762 @@
|
||||
% References for hanzo-sparse-basis-routing.
|
||||
%
|
||||
% A line beginning "% TODO" marks an entry whose details we did not verify
|
||||
% against the publisher's record. Everything not so marked is cited from fields we are
|
||||
% confident of; page ranges we were unsure of are omitted rather than guessed.
|
||||
|
||||
@article{zobel2006inverted,
|
||||
author = {Justin Zobel and Alistair Moffat},
|
||||
title = {Inverted Files for Text Search Engines},
|
||||
journal = {ACM Computing Surveys},
|
||||
volume = {38},
|
||||
number = {2},
|
||||
year = {2006}
|
||||
}
|
||||
|
||||
@article{robertson2009bm25,
|
||||
author = {Stephen Robertson and Hugo Zaragoza},
|
||||
title = {The Probabilistic Relevance Framework: {BM25} and Beyond},
|
||||
journal = {Foundations and Trends in Information Retrieval},
|
||||
volume = {3},
|
||||
number = {4},
|
||||
pages = {333--389},
|
||||
year = {2009}
|
||||
}
|
||||
|
||||
@inproceedings{broder2003wand,
|
||||
author = {Andrei Z. Broder and David Carmel and Michael Herscovici and
|
||||
Aya Soffer and Jason Zien},
|
||||
title = {Efficient Query Evaluation using a Two-Level Retrieval Process},
|
||||
booktitle = {Proceedings of the 12th ACM Conference on Information and
|
||||
Knowledge Management (CIKM)},
|
||||
pages = {426--434},
|
||||
year = {2003}
|
||||
}
|
||||
|
||||
@inproceedings{ding2011bmw,
|
||||
author = {Shuai Ding and Torsten Suel},
|
||||
title = {Faster Top-k Document Retrieval Using Block-Max Indexes},
|
||||
booktitle = {Proceedings of the 34th International ACM SIGIR Conference on
|
||||
Research and Development in Information Retrieval},
|
||||
pages = {993--1002},
|
||||
year = {2011}
|
||||
}
|
||||
|
||||
@article{turtle1995maxscore,
|
||||
author = {Howard Turtle and James Flood},
|
||||
title = {Query Evaluation: Strategies and Optimizations},
|
||||
journal = {Information Processing and Management},
|
||||
volume = {31},
|
||||
number = {6},
|
||||
pages = {831--850},
|
||||
year = {1995}
|
||||
}
|
||||
|
||||
@inproceedings{mallia2017vbmw,
|
||||
author = {Antonio Mallia and Giuseppe Ottaviano and Elia Porciani and
|
||||
Nicola Tonellotto and Rossano Venturini},
|
||||
title = {Faster {BlockMax} {WAND} with Variable-sized Blocks},
|
||||
booktitle = {Proceedings of the 40th International ACM SIGIR Conference on
|
||||
Research and Development in Information Retrieval},
|
||||
year = {2017}
|
||||
}
|
||||
|
||||
@article{fagin2003ta,
|
||||
author = {Ronald Fagin and Amnon Lotem and Moni Naor},
|
||||
title = {Optimal Aggregation Algorithms for Middleware},
|
||||
journal = {Journal of Computer and System Sciences},
|
||||
volume = {66},
|
||||
number = {4},
|
||||
pages = {614--656},
|
||||
year = {2003}
|
||||
}
|
||||
|
||||
% TODO: verify the page range of Anh and Moffat, SIGIR 2002.
|
||||
@inproceedings{anh2002impact,
|
||||
author = {Vo Ngoc Anh and Alistair Moffat},
|
||||
title = {Impact Transformation: Effective and Efficient Web Retrieval},
|
||||
booktitle = {Proceedings of the 25th International ACM SIGIR Conference on
|
||||
Research and Development in Information Retrieval},
|
||||
year = {2002}
|
||||
}
|
||||
|
||||
@inproceedings{lin2015anytime,
|
||||
author = {Jimmy Lin and Andrew Trotman},
|
||||
title = {Anytime Ranking for Impact-Ordered Indexes},
|
||||
booktitle = {Proceedings of the 2015 International Conference on the Theory
|
||||
of Information Retrieval (ICTIR)},
|
||||
year = {2015}
|
||||
}
|
||||
|
||||
% TODO: verify the venue details of PISA (OSIRRC 2019 workshop proceedings,
|
||||
% CEUR-WS volume and pages).
|
||||
@inproceedings{mallia2019pisa,
|
||||
author = {Antonio Mallia and Micha{\l} Siedlaczek and Joel Mackenzie and
|
||||
Torsten Suel},
|
||||
title = {{PISA}: Performant Indexes and Search for Academia},
|
||||
booktitle = {Proceedings of the Open-Source IR Replicability Challenge
|
||||
(OSIRRC) at SIGIR},
|
||||
year = {2019}
|
||||
}
|
||||
|
||||
@inproceedings{formal2021splade,
|
||||
author = {Thibault Formal and Benjamin Piwowarski and
|
||||
St{\'e}phane Clinchant},
|
||||
title = {{SPLADE}: Sparse Lexical and Expansion Model for First Stage
|
||||
Ranking},
|
||||
booktitle = {Proceedings of the 44th International ACM SIGIR Conference on
|
||||
Research and Development in Information Retrieval},
|
||||
pages = {2288--2292},
|
||||
year = {2021}
|
||||
}
|
||||
|
||||
@misc{formal2021spladev2,
|
||||
author = {Thibault Formal and Carlos Lassance and
|
||||
Benjamin Piwowarski and St{\'e}phane Clinchant},
|
||||
title = {{SPLADE} v2: Sparse Lexical and Expansion Model for
|
||||
Information Retrieval},
|
||||
howpublished = {arXiv:2109.10086},
|
||||
year = {2021}
|
||||
}
|
||||
|
||||
@inproceedings{formal2022spladepp,
|
||||
author = {Thibault Formal and Carlos Lassance and Benjamin Piwowarski and
|
||||
St{\'e}phane Clinchant},
|
||||
title = {From Distillation to Hard Negative Sampling: Making Sparse
|
||||
Neural {IR} Models More Effective},
|
||||
booktitle = {Proceedings of the 45th International ACM SIGIR Conference on
|
||||
Research and Development in Information Retrieval},
|
||||
year = {2022}
|
||||
}
|
||||
|
||||
@inproceedings{gao2021coil,
|
||||
author = {Luyu Gao and Zhuyun Dai and Jamie Callan},
|
||||
title = {{COIL}: Revisit Exact Lexical Match in Information Retrieval
|
||||
with Contextualized Inverted List},
|
||||
booktitle = {Proceedings of the 2021 Conference of the North American
|
||||
Chapter of the Association for Computational Linguistics
|
||||
(NAACL)},
|
||||
pages = {3030--3042},
|
||||
year = {2021}
|
||||
}
|
||||
|
||||
@misc{lin2021unicoil,
|
||||
author = {Jimmy Lin and Xueguang Ma},
|
||||
title = {A Few Brief Notes on {DeepImpact}, {COIL}, and a Conceptual
|
||||
Framework for Information Retrieval Techniques},
|
||||
howpublished = {arXiv:2106.14807},
|
||||
year = {2021}
|
||||
}
|
||||
|
||||
@inproceedings{mallia2021deepimpact,
|
||||
author = {Antonio Mallia and Omar Khattab and Torsten Suel and
|
||||
Nicola Tonellotto},
|
||||
title = {Learning Passage Impacts for Inverted Indexes},
|
||||
booktitle = {Proceedings of the 44th International ACM SIGIR Conference on
|
||||
Research and Development in Information Retrieval},
|
||||
pages = {1723--1727},
|
||||
year = {2021}
|
||||
}
|
||||
|
||||
@inproceedings{dai2020deepct,
|
||||
author = {Zhuyun Dai and Jamie Callan},
|
||||
title = {Context-Aware Term Weighting For First Stage Passage Retrieval},
|
||||
booktitle = {Proceedings of the 43rd International ACM SIGIR Conference on
|
||||
Research and Development in Information Retrieval},
|
||||
pages = {1533--1536},
|
||||
year = {2020}
|
||||
}
|
||||
|
||||
% TODO: verify the arXiv identifier 2110.11540 for the Mackenzie, Trotman and
|
||||
% Lin note on learned sparse weights.
|
||||
@misc{mackenzie2021wacky,
|
||||
author = {Joel Mackenzie and Andrew Trotman and Jimmy Lin},
|
||||
title = {Wacky Weights in Learned Sparse Representations and the
|
||||
Revenge of Score-at-a-Time Query Evaluation},
|
||||
howpublished = {arXiv:2110.11540},
|
||||
year = {2021}
|
||||
}
|
||||
|
||||
@inproceedings{lassance2022efficiency,
|
||||
author = {Carlos Lassance and St{\'e}phane Clinchant},
|
||||
title = {An Efficiency Study for {SPLADE} Models},
|
||||
booktitle = {Proceedings of the 45th International ACM SIGIR Conference on
|
||||
Research and Development in Information Retrieval},
|
||||
year = {2022}
|
||||
}
|
||||
|
||||
% TODO: verify authors, pages and venue of the guided-traversal paper
|
||||
% (SIGIR 2022).
|
||||
@inproceedings{mallia2022guided,
|
||||
author = {Antonio Mallia and Joel Mackenzie and Torsten Suel and
|
||||
Nicola Tonellotto},
|
||||
title = {Faster Learned Sparse Retrieval with Guided Traversal},
|
||||
booktitle = {Proceedings of the 45th International ACM SIGIR Conference on
|
||||
Research and Development in Information Retrieval},
|
||||
year = {2022}
|
||||
}
|
||||
|
||||
% TODO: verify authors and venue of the block-max pruning paper (SIGIR 2024).
|
||||
@inproceedings{mallia2024bmp,
|
||||
author = {Antonio Mallia and Torsten Suel and Nicola Tonellotto},
|
||||
title = {Faster Learned Sparse Retrieval with Block-Max Pruning},
|
||||
booktitle = {Proceedings of the 47th International ACM SIGIR Conference on
|
||||
Research and Development in Information Retrieval},
|
||||
year = {2024}
|
||||
}
|
||||
|
||||
% TODO: verify the page range and exact title of the Seismic paper (SIGIR 2024).
|
||||
@inproceedings{bruch2024seismic,
|
||||
author = {Sebastian Bruch and Franco Maria Nardini and Cosimo Rulli and
|
||||
Rossano Venturini},
|
||||
title = {Efficient Inverted Indexes for Approximate Retrieval over
|
||||
Learned Sparse Representations},
|
||||
booktitle = {Proceedings of the 47th International ACM SIGIR Conference on
|
||||
Research and Development in Information Retrieval},
|
||||
year = {2024}
|
||||
}
|
||||
|
||||
@article{salton1975vsm,
|
||||
author = {Gerard Salton and Anita Wong and Chung-Shu Yang},
|
||||
title = {A Vector Space Model for Automatic Indexing},
|
||||
journal = {Communications of the ACM},
|
||||
volume = {18},
|
||||
number = {11},
|
||||
pages = {613--620},
|
||||
year = {1975}
|
||||
}
|
||||
|
||||
@inproceedings{gabrilovich2007esa,
|
||||
author = {Evgeniy Gabrilovich and Shaul Markovitch},
|
||||
title = {Computing Semantic Relatedness using {Wikipedia}-based Explicit
|
||||
Semantic Analysis},
|
||||
booktitle = {Proceedings of the 20th International Joint Conference on
|
||||
Artificial Intelligence (IJCAI)},
|
||||
pages = {1606--1611},
|
||||
year = {2007}
|
||||
}
|
||||
|
||||
@inproceedings{xiong2017duet,
|
||||
author = {Chenyan Xiong and Jamie Callan and Tie-Yan Liu},
|
||||
title = {Word-Entity Duet Representations for Document Ranking},
|
||||
booktitle = {Proceedings of the 40th International ACM SIGIR Conference on
|
||||
Research and Development in Information Retrieval},
|
||||
year = {2017}
|
||||
}
|
||||
|
||||
@inproceedings{dalton2014entity,
|
||||
author = {Jeffrey Dalton and Laura Dietz and James Allan},
|
||||
title = {Entity Query Feature Expansion using Knowledge Base Links},
|
||||
booktitle = {Proceedings of the 37th International ACM SIGIR Conference on
|
||||
Research and Development in Information Retrieval},
|
||||
pages = {365--374},
|
||||
year = {2014}
|
||||
}
|
||||
|
||||
@misc{cox2012trigram,
|
||||
author = {Russ Cox},
|
||||
title = {Regular Expression Matching with a Trigram Index, or How
|
||||
{Google} Code Search Worked},
|
||||
howpublished = {\url{https://swtch.com/~rsc/regexp/regexp4.html}},
|
||||
year = {2012}
|
||||
}
|
||||
|
||||
@article{milo2002motifs,
|
||||
author = {Ron Milo and Shai Shen-Orr and Shalev Itzkovitz and
|
||||
Nadav Kashtan and Dmitri Chklovskii and Uri Alon},
|
||||
title = {Network Motifs: Simple Building Blocks of Complex Networks},
|
||||
journal = {Science},
|
||||
volume = {298},
|
||||
number = {5594},
|
||||
pages = {824--827},
|
||||
year = {2002}
|
||||
}
|
||||
|
||||
@article{olshausen1996sparse,
|
||||
author = {Bruno A. Olshausen and David J. Field},
|
||||
title = {Emergence of Simple-Cell Receptive Field Properties by Learning a
|
||||
Sparse Code for Natural Images},
|
||||
journal = {Nature},
|
||||
volume = {381},
|
||||
number = {6583},
|
||||
pages = {607--609},
|
||||
year = {1996}
|
||||
}
|
||||
|
||||
@misc{cunningham2023sae,
|
||||
author = {Hoagy Cunningham and Aidan Ewart and Logan Riggs and
|
||||
Robert Huben and Lee Sharkey},
|
||||
title = {Sparse Autoencoders Find Highly Interpretable Features in
|
||||
Language Models},
|
||||
howpublished = {arXiv:2309.08600},
|
||||
year = {2023}
|
||||
}
|
||||
|
||||
@article{jegou2011pq,
|
||||
author = {Herv{\'e} J{\'e}gou and Matthijs Douze and Cordelia Schmid},
|
||||
title = {Product Quantization for Nearest Neighbor Search},
|
||||
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
|
||||
volume = {33},
|
||||
number = {1},
|
||||
pages = {117--128},
|
||||
year = {2011}
|
||||
}
|
||||
|
||||
@inproceedings{sivic2003videogoogle,
|
||||
author = {Josef Sivic and Andrew Zisserman},
|
||||
title = {Video {Google}: A Text Retrieval Approach to Object Matching in
|
||||
Videos},
|
||||
booktitle = {Proceedings of the 9th IEEE International Conference on
|
||||
Computer Vision (ICCV)},
|
||||
pages = {1470--1477},
|
||||
year = {2003}
|
||||
}
|
||||
|
||||
@inproceedings{babenko2012imi,
|
||||
author = {Artem Babenko and Victor Lempitsky},
|
||||
title = {The Inverted Multi-Index},
|
||||
booktitle = {Proceedings of the IEEE Conference on Computer Vision and
|
||||
Pattern Recognition (CVPR)},
|
||||
pages = {3069--3076},
|
||||
year = {2012}
|
||||
}
|
||||
|
||||
@article{johnson2021faiss,
|
||||
author = {Jeff Johnson and Matthijs Douze and Herv{\'e} J{\'e}gou},
|
||||
title = {Billion-Scale Similarity Search with {GPUs}},
|
||||
journal = {IEEE Transactions on Big Data},
|
||||
volume = {7},
|
||||
number = {3},
|
||||
pages = {535--547},
|
||||
year = {2021}
|
||||
}
|
||||
|
||||
@article{malkov2020hnsw,
|
||||
author = {Yury A. Malkov and Dmitry A. Yashunin},
|
||||
title = {Efficient and Robust Approximate Nearest Neighbor Search Using
|
||||
Hierarchical Navigable Small World Graphs},
|
||||
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
|
||||
volume = {42},
|
||||
number = {4},
|
||||
pages = {824--836},
|
||||
year = {2020}
|
||||
}
|
||||
|
||||
@inproceedings{subramanya2019diskann,
|
||||
author = {Suhas Jayaram Subramanya and Fnu Devvrit and Rohan Kadekodi and
|
||||
Ravishankar Krishnaswamy and Harsha Vardhan Simhadri},
|
||||
title = {{DiskANN}: Fast Accurate Billion-point Nearest Neighbor Search
|
||||
on a Single Node},
|
||||
booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
|
||||
year = {2019}
|
||||
}
|
||||
|
||||
@misc{singh2021freshdiskann,
|
||||
author = {Aditi Singh and Suhas Jayaram Subramanya and
|
||||
Ravishankar Krishnaswamy and Harsha Vardhan Simhadri},
|
||||
title = {{FreshDiskANN}: A Fast and Accurate Graph-Based {ANN} Index
|
||||
for Streaming Similarity Search},
|
||||
howpublished = {arXiv:2105.09613},
|
||||
year = {2021}
|
||||
}
|
||||
|
||||
% TODO: verify the author list of the SPANN paper (NeurIPS 2021).
|
||||
@inproceedings{chen2021spann,
|
||||
author = {Qi Chen and Bing Zhao and Haidong Wang and Mingqin Li and
|
||||
Chuanjie Liu and Zengzhong Li and Mao Yang and Jingdong Wang},
|
||||
title = {{SPANN}: Highly-efficient Billion-scale Approximate Nearest
|
||||
Neighbor Search},
|
||||
booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
|
||||
year = {2021}
|
||||
}
|
||||
|
||||
@inproceedings{guo2020scann,
|
||||
author = {Ruiqi Guo and Philip Sun and Erik Lindgren and Quan Geng and
|
||||
David Simcha and Felix Chern and Sanjiv Kumar},
|
||||
title = {Accelerating Large-Scale Inference with Anisotropic Vector
|
||||
Quantization},
|
||||
booktitle = {Proceedings of the 37th International Conference on Machine
|
||||
Learning (ICML)},
|
||||
year = {2020}
|
||||
}
|
||||
|
||||
% TODO: verify the author list of the SOAR paper (NeurIPS 2023).
|
||||
@inproceedings{sun2023soar,
|
||||
author = {Philip Sun and David Simcha and Dave Dopson and Ruiqi Guo and
|
||||
Sanjiv Kumar},
|
||||
title = {{SOAR}: Improved Indexing for Approximate Nearest Neighbor
|
||||
Search},
|
||||
booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
|
||||
year = {2023}
|
||||
}
|
||||
|
||||
@article{fukunaga1975branch,
|
||||
author = {Keinosuke Fukunaga and Patrenahalli M. Narendra},
|
||||
title = {A Branch and Bound Algorithm for Computing k-Nearest Neighbors},
|
||||
journal = {IEEE Transactions on Computers},
|
||||
volume = {C-24},
|
||||
number = {7},
|
||||
pages = {750--753},
|
||||
year = {1975}
|
||||
}
|
||||
|
||||
@inproceedings{nister2006vocabtree,
|
||||
author = {David Nist{\'e}r and Henrik Stew{\'e}nius},
|
||||
title = {Scalable Recognition with a Vocabulary Tree},
|
||||
booktitle = {Proceedings of the IEEE Conference on Computer Vision and
|
||||
Pattern Recognition (CVPR)},
|
||||
pages = {2161--2168},
|
||||
year = {2006}
|
||||
}
|
||||
|
||||
@article{muja2014flann,
|
||||
author = {Marius Muja and David G. Lowe},
|
||||
title = {Scalable Nearest Neighbor Algorithms for High Dimensional Data},
|
||||
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
|
||||
volume = {36},
|
||||
number = {11},
|
||||
pages = {2227--2240},
|
||||
year = {2014}
|
||||
}
|
||||
|
||||
@inproceedings{ciaccia1997mtree,
|
||||
author = {Paolo Ciaccia and Marco Patella and Pavel Zezula},
|
||||
title = {{M-tree}: An Efficient Access Method for Similarity Search in
|
||||
Metric Spaces},
|
||||
booktitle = {Proceedings of the 23rd International Conference on Very Large
|
||||
Data Bases (VLDB)},
|
||||
pages = {426--435},
|
||||
year = {1997}
|
||||
}
|
||||
|
||||
@article{uhlmann1991metric,
|
||||
author = {Jeffrey K. Uhlmann},
|
||||
title = {Satisfying General Proximity/Similarity Queries with Metric Trees},
|
||||
journal = {Information Processing Letters},
|
||||
volume = {40},
|
||||
number = {4},
|
||||
pages = {175--179},
|
||||
year = {1991}
|
||||
}
|
||||
|
||||
@inproceedings{guttman1984rtree,
|
||||
author = {Antonin Guttman},
|
||||
title = {{R-Trees}: A Dynamic Index Structure for Spatial Searching},
|
||||
booktitle = {Proceedings of the 1984 ACM SIGMOD International Conference on
|
||||
Management of Data},
|
||||
pages = {47--57},
|
||||
year = {1984}
|
||||
}
|
||||
|
||||
@inproceedings{weber1998quantitative,
|
||||
author = {Roger Weber and Hans-J{\"o}rg Schek and Stephen Blott},
|
||||
title = {A Quantitative Analysis and Performance Study for
|
||||
Similarity-Search Methods in High-Dimensional Spaces},
|
||||
booktitle = {Proceedings of the 24th International Conference on Very Large
|
||||
Data Bases (VLDB)},
|
||||
pages = {194--205},
|
||||
year = {1998}
|
||||
}
|
||||
|
||||
@inproceedings{beyer1999nn,
|
||||
author = {Kevin Beyer and Jonathan Goldstein and Raghu Ramakrishnan and
|
||||
Uri Shaft},
|
||||
title = {When Is ``Nearest Neighbor'' Meaningful?},
|
||||
booktitle = {Proceedings of the 7th International Conference on Database
|
||||
Theory (ICDT)},
|
||||
pages = {217--235},
|
||||
year = {1999}
|
||||
}
|
||||
|
||||
@inproceedings{indyk1998lsh,
|
||||
author = {Piotr Indyk and Rajeev Motwani},
|
||||
title = {Approximate Nearest Neighbors: Towards Removing the Curse of
|
||||
Dimensionality},
|
||||
booktitle = {Proceedings of the 30th Annual ACM Symposium on Theory of
|
||||
Computing (STOC)},
|
||||
pages = {604--613},
|
||||
year = {1998}
|
||||
}
|
||||
|
||||
% TODO: verify the exact title of the Indyk and Xu paper (NeurIPS 2023).
|
||||
@inproceedings{indyk2023worst,
|
||||
author = {Piotr Indyk and Haike Xu},
|
||||
title = {Worst-case Performance of Popular Approximate Nearest Neighbor
|
||||
Search Implementations: Guarantees and Limitations},
|
||||
booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
|
||||
year = {2023}
|
||||
}
|
||||
|
||||
@inproceedings{prokhorenkova2020graph,
|
||||
author = {Liudmila Prokhorenkova and Aleksandr Shekhovtsov},
|
||||
title = {Graph-based Nearest Neighbor Search: From Practice to Theory},
|
||||
booktitle = {Proceedings of the 37th International Conference on Machine
|
||||
Learning (ICML)},
|
||||
year = {2020}
|
||||
}
|
||||
|
||||
@article{aumuller2020annb,
|
||||
author = {Martin Aum{\"u}ller and Erik Bernhardsson and
|
||||
Alexander Faithfull},
|
||||
title = {{ANN-Benchmarks}: A Benchmarking Tool for Approximate Nearest
|
||||
Neighbor Algorithms},
|
||||
journal = {Information Systems},
|
||||
volume = {87},
|
||||
year = {2020}
|
||||
}
|
||||
|
||||
% TODO: complete the author list of the billion-scale challenge report; it is
|
||||
% cited here with the first author only.
|
||||
@inproceedings{simhadri2022bigann,
|
||||
author = {Harsha Vardhan Simhadri and others},
|
||||
title = {Results of the {NeurIPS'21} Challenge on Billion-Scale
|
||||
Approximate Nearest Neighbor Search},
|
||||
booktitle = {Proceedings of the NeurIPS 2021 Competitions and Demonstrations
|
||||
Track, PMLR 176},
|
||||
year = {2022}
|
||||
}
|
||||
|
||||
% TODO: complete the author list of the Milvus paper; it is cited here with the
|
||||
% first author only.
|
||||
@inproceedings{wang2021milvus,
|
||||
author = {Jianguo Wang and others},
|
||||
title = {{Milvus}: A Purpose-Built Vector Data Management System},
|
||||
booktitle = {Proceedings of the 2021 ACM SIGMOD International Conference on
|
||||
Management of Data},
|
||||
year = {2021}
|
||||
}
|
||||
|
||||
% TODO: verify the venue and volume of the ACORN paper (SIGMOD 2024 /
|
||||
% Proceedings of the ACM on Management of Data).
|
||||
@inproceedings{patel2024acorn,
|
||||
author = {Liana Patel and Peter Kraft and Carlos Guestrin and
|
||||
Matei Zaharia},
|
||||
title = {{ACORN}: Performant and Predicate-Agnostic Search Over Vector
|
||||
Embeddings and Structured Data},
|
||||
booktitle = {Proceedings of the 2024 ACM SIGMOD International Conference on
|
||||
Management of Data},
|
||||
year = {2024}
|
||||
}
|
||||
|
||||
% TODO: complete the author list of Filtered-DiskANN; it is cited here with the
|
||||
% first author only.
|
||||
@inproceedings{gollapudi2023filtered,
|
||||
author = {Siddharth Gollapudi and others},
|
||||
title = {{Filtered-DiskANN}: Graph Algorithms for Approximate Nearest
|
||||
Neighbor Search with Filters},
|
||||
booktitle = {Proceedings of the ACM Web Conference (WWW)},
|
||||
year = {2023}
|
||||
}
|
||||
|
||||
% TODO: verify the page range of the ClickHouse VLDB paper.
|
||||
@article{schulze2024clickhouse,
|
||||
author = {Robert Schulze and Tom Schreiber and Ilya Yatsishin and
|
||||
Ryadh Dahimene and Alexey Milovidov},
|
||||
title = {{ClickHouse} --- Lightning Fast Analytics for Everyone},
|
||||
journal = {Proceedings of the VLDB Endowment},
|
||||
volume = {17},
|
||||
number = {12},
|
||||
year = {2024}
|
||||
}
|
||||
|
||||
@misc{clickhousedocs,
|
||||
author = {{ClickHouse, Inc.}},
|
||||
title = {{ClickHouse} documentation and source, version 26.4:
|
||||
{MergeTree} parts and granules, data skipping indexes, text
|
||||
index, vector similarity index},
|
||||
howpublished = {\url{https://clickhouse.com/docs}},
|
||||
year = {2026},
|
||||
note = {Source read at version 26.4.2.1}
|
||||
}
|
||||
|
||||
@misc{usearch,
|
||||
author = {Ash Vardanian},
|
||||
title = {{USearch}: Smaller and Faster Single-File Vector Search
|
||||
Engine},
|
||||
howpublished = {\url{https://github.com/unum-cloud/usearch}},
|
||||
year = {2023}
|
||||
}
|
||||
|
||||
@misc{lucene,
|
||||
author = {{The Apache Software Foundation}},
|
||||
title = {Apache {Lucene}},
|
||||
howpublished = {\url{https://lucene.apache.org}},
|
||||
year = {2026}
|
||||
}
|
||||
|
||||
@inproceedings{moerkotte1998sma,
|
||||
author = {Guido Moerkotte},
|
||||
title = {Small Materialized Aggregates: A Light Weight Index Structure
|
||||
for Data Warehousing},
|
||||
booktitle = {Proceedings of the 24th International Conference on Very Large
|
||||
Data Bases (VLDB)},
|
||||
pages = {476--487},
|
||||
year = {1998}
|
||||
}
|
||||
|
||||
@inproceedings{sun2014skipping,
|
||||
author = {Liwen Sun and Michael J. Franklin and Sanjay Krishnan and
|
||||
Reynold S. Xin},
|
||||
title = {Fine-grained Partitioning for Aggressive Data Skipping},
|
||||
booktitle = {Proceedings of the 2014 ACM SIGMOD International Conference on
|
||||
Management of Data},
|
||||
pages = {1115--1126},
|
||||
year = {2014}
|
||||
}
|
||||
|
||||
@article{oneil1996lsm,
|
||||
author = {Patrick O'Neil and Edward Cheng and Dieter Gawlick and
|
||||
Elizabeth O'Neil},
|
||||
title = {The Log-Structured Merge-Tree ({LSM-Tree})},
|
||||
journal = {Acta Informatica},
|
||||
volume = {33},
|
||||
number = {4},
|
||||
pages = {351--385},
|
||||
year = {1996}
|
||||
}
|
||||
|
||||
@article{abadi2013columnstores,
|
||||
author = {Daniel Abadi and Peter Boncz and Stavros Harizopoulos and
|
||||
Stratos Idreos and Samuel Madden},
|
||||
title = {The Design and Implementation of Modern Column-Oriented Database
|
||||
Systems},
|
||||
journal = {Foundations and Trends in Databases},
|
||||
volume = {5},
|
||||
number = {3},
|
||||
pages = {197--280},
|
||||
year = {2013}
|
||||
}
|
||||
|
||||
@book{vanrijsbergen1979ir,
|
||||
author = {C. J. van Rijsbergen},
|
||||
title = {Information Retrieval},
|
||||
edition = {2nd},
|
||||
publisher = {Butterworths},
|
||||
address = {London},
|
||||
year = {1979}
|
||||
}
|
||||
|
||||
@inproceedings{callan1995cori,
|
||||
author = {James P. Callan and Zhihong Lu and W. Bruce Croft},
|
||||
title = {Searching Distributed Collections with Inference Networks},
|
||||
booktitle = {Proceedings of the 18th International ACM SIGIR Conference on
|
||||
Research and Development in Information Retrieval},
|
||||
pages = {21--28},
|
||||
year = {1995}
|
||||
}
|
||||
|
||||
@inproceedings{si2003redde,
|
||||
author = {Luo Si and Jamie Callan},
|
||||
title = {Relevant Document Distribution Estimation Method for Resource
|
||||
Selection},
|
||||
booktitle = {Proceedings of the 26th International ACM SIGIR Conference on
|
||||
Research and Development in Information Retrieval},
|
||||
year = {2003}
|
||||
}
|
||||
|
||||
@inproceedings{aly2013taily,
|
||||
author = {Robin Aly and Djoerd Hiemstra and Thomas Demeester},
|
||||
title = {{Taily}: Shard Selection Using the Tail of Score Distributions},
|
||||
booktitle = {Proceedings of the 36th International ACM SIGIR Conference on
|
||||
Research and Development in Information Retrieval},
|
||||
year = {2013}
|
||||
}
|
||||
|
||||
@article{kulkarni2015selective,
|
||||
author = {Anagha Kulkarni and Jamie Callan},
|
||||
title = {Selective Search: Efficient and Effective Search of Large Textual
|
||||
Collections},
|
||||
journal = {ACM Transactions on Information Systems},
|
||||
volume = {33},
|
||||
number = {4},
|
||||
year = {2015}
|
||||
}
|
||||
|
||||
@misc{edge2024graphrag,
|
||||
author = {Darren Edge and Ha Trinh and Newman Cheng and
|
||||
Joshua Bradley and Alex Chao and Apurva Mody and
|
||||
Steven Truitt and Jonathan Larson},
|
||||
title = {From Local to Global: A Graph {RAG} Approach to
|
||||
Query-Focused Summarization},
|
||||
howpublished = {arXiv:2404.16130},
|
||||
year = {2024}
|
||||
}
|
||||
|
||||
@inproceedings{sarthi2024raptor,
|
||||
author = {Parth Sarthi and Salman Abdullah and Aditi Tuli and
|
||||
Shubh Khanna and Anna Goldie and Christopher D. Manning},
|
||||
title = {{RAPTOR}: Recursive Abstractive Processing for Tree-Organized
|
||||
Retrieval},
|
||||
booktitle = {International Conference on Learning Representations (ICLR)},
|
||||
year = {2024}
|
||||
}
|
||||
|
||||
@article{dean2013tail,
|
||||
author = {Jeffrey Dean and Luiz Andr{\'e} Barroso},
|
||||
title = {The Tail at Scale},
|
||||
journal = {Communications of the ACM},
|
||||
volume = {56},
|
||||
number = {2},
|
||||
pages = {74--80},
|
||||
year = {2013}
|
||||
}
|
||||
|
||||
@book{efron1993bootstrap,
|
||||
author = {Bradley Efron and Robert J. Tibshirani},
|
||||
title = {An Introduction to the Bootstrap},
|
||||
publisher = {Chapman and Hall},
|
||||
year = {1993}
|
||||
}
|
||||
|
||||
@article{clauset2009powerlaw,
|
||||
author = {Aaron Clauset and Cosma Rohilla Shalizi and M. E. J. Newman},
|
||||
title = {Power-Law Distributions in Empirical Data},
|
||||
journal = {SIAM Review},
|
||||
volume = {51},
|
||||
number = {4},
|
||||
pages = {661--703},
|
||||
year = {2009}
|
||||
}
|
||||
|
||||
@inproceedings{reimers2019sbert,
|
||||
author = {Nils Reimers and Iryna Gurevych},
|
||||
title = {{Sentence-BERT}: Sentence Embeddings using {Siamese}
|
||||
{BERT}-Networks},
|
||||
booktitle = {Proceedings of the 2019 Conference on Empirical Methods in
|
||||
Natural Language Processing and the 9th International Joint
|
||||
Conference on Natural Language Processing (EMNLP-IJCNLP)},
|
||||
pages = {3982--3992},
|
||||
year = {2019}
|
||||
}
|
||||
|
||||
@article{kwiatkowski2019nq,
|
||||
author = {Tom Kwiatkowski and Jennimaria Palomaki and Olivia Redfield and
|
||||
Michael Collins and Ankur Parikh and Chris Alberti and
|
||||
Danielle Epstein and Illia Polosukhin and Jacob Devlin and
|
||||
Kenton Lee and Kristina Toutanova and Llion Jones and
|
||||
Matthew Kelcey and Ming-Wei Chang and Andrew M. Dai and
|
||||
Jakob Uszkoreit and Quoc Le and Slav Petrov},
|
||||
title = {Natural Questions: A Benchmark for Question Answering Research},
|
||||
journal = {Transactions of the Association for Computational Linguistics},
|
||||
volume = {7},
|
||||
pages = {452--466},
|
||||
year = {2019}
|
||||
}
|
||||
|
||||
@inproceedings{lee2019orqa,
|
||||
author = {Kenton Lee and Ming-Wei Chang and Kristina Toutanova},
|
||||
title = {Latent Retrieval for Weakly Supervised Open Domain Question
|
||||
Answering},
|
||||
booktitle = {Proceedings of the 57th Annual Meeting of the Association for
|
||||
Computational Linguistics (ACL)},
|
||||
pages = {6086--6096},
|
||||
year = {2019}
|
||||
}
|
||||
|
||||
@misc{hanzodatastore,
|
||||
author = {{Hanzo Industries}},
|
||||
title = {Hanzo Datastore: a fork of {ClickHouse}},
|
||||
howpublished = {\url{https://github.com/hanzoai/datastore}},
|
||||
year = {2026},
|
||||
note = {Forked from {ClickHouse} 26.4}
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
% Read from the runs named below -- do not hand-edit values.
|
||||
%
|
||||
% PLACEHOLDER. No run has written this file yet, so every \res{...} in the
|
||||
% paper prints XXX. The lane that will write it is in hanzoai/benchmarks and is
|
||||
% not yet published; its first pass measures upstream ClickHouse 26.4.2.10 at
|
||||
% N = 1e5 .. 1e7; the routed store and the larger corpora follow.
|
||||
%
|
||||
% Contract. One line per measured value:
|
||||
%
|
||||
% \setres{<system>/<N>/<metric>}{<value>}
|
||||
%
|
||||
% system scan exhaustive scoring, no index
|
||||
% base upstream ClickHouse: primary index, skipping indexes,
|
||||
% text index, one vector similarity index per part
|
||||
% route the design of this paper
|
||||
% oracle the granules and parts holding the exact top-k
|
||||
% N 1e5 1e6 1e7 1e8 1e9
|
||||
% metric parts parts read (ProfileEvents SelectedParts)
|
||||
% granules granules read (SelectedMarks)
|
||||
% mib bytes fetched, MiB (ReadCompressedBytes; OSReadBytes cold)
|
||||
% bpc KiB fetched per correct answer
|
||||
% recall recall@k against the exact top-k
|
||||
% p50, p99 query latency, ms, warm
|
||||
%
|
||||
% and one line per fitted exponent of C(N) = a N^alpha, with its interval,
|
||||
% where C is parts, granules and bytes (mib) touched per correct answer, and
|
||||
% routing summaries examined per query:
|
||||
%
|
||||
% \setres{fit/<system>/<metric>}{alpha [lo, hi]}
|
||||
%
|
||||
% Every row also needs the conditions it was taken under, which the harness
|
||||
% writes as its own keys and the paper prints in the caption:
|
||||
%
|
||||
% \setres{run/host}{...} one machine, named
|
||||
% \setres{run/version}{...} store build
|
||||
% \setres{run/k}{...} top-k
|
||||
% \setres{run/recall}{...} the recall target held across N
|
||||
% \setres{run/queries}{...} query set and its size
|
||||
% \setres{run/parts/<N>}{...} part count and size distribution at each N
|
||||
@@ -0,0 +1,77 @@
|
||||
\section{Introduction}
|
||||
\label{sec:intro}
|
||||
|
||||
A column store of the MergeTree family holds a table as a set of immutable
|
||||
parts, each sorted by the table's ordering key and cut into granules of a few
|
||||
thousand rows, with a sparse primary index and a set of skipping indexes
|
||||
written inside the part~\cite{schulze2024clickhouse,clickhousedocs}. Retrieval
|
||||
that ranks rows by similarity to a query fits that layout awkwardly. The text
|
||||
index answers which rows contain a token, not which rows rank highest for a
|
||||
weighted query. The vector index is built per part and searched per part, so a
|
||||
similarity query visits every part that survives the predicate; the same
|
||||
arrangement holds in Lucene and in Milvus, where a segment is the unit an
|
||||
approximate index is built over~\cite{lucene,wang2021milvus}. As a table grows,
|
||||
the number of parts grows, and the cost of a similarity query grows with it
|
||||
unless something excludes parts before they are searched.
|
||||
|
||||
This paper describes a design for that gap and states how we intend to measure
|
||||
it. Two structures carry it. The first is a weighted inverted file over a mixed
|
||||
sparse basis: each basis element --- a word, a linked entity, a code symbol, a
|
||||
coordinate of a learned sparse dictionary, a relation, a structural motif ---
|
||||
owns a posting list of (row, weight) pairs, and a query written over the same
|
||||
basis is scored only over the posting lists of the elements it activates, with
|
||||
top-$k$ evaluation under block-max pruning. The second is a routing structure
|
||||
built per part: regions with a centroid, a covering radius, the maximum weight
|
||||
each basis element attains in the region, and counting statistics, aggregated
|
||||
into a table-level structure that the planner consults to drop parts whose best
|
||||
possible score cannot reach the running $k$-th best, before any per-part index
|
||||
is opened.
|
||||
|
||||
\paragraph{What is claimed.}
|
||||
Expected retrieval cost should scale with the relevant neighbourhood rather
|
||||
than with corpus size: at a fixed query distribution and a fixed recall target,
|
||||
the parts, granules and bytes a query touches should grow more slowly than the
|
||||
corpus does, and should stay within a bounded factor of the granules that hold
|
||||
the answer. That is an empirical claim about a distribution of queries, and
|
||||
\S\ref{sec:method} is the protocol that would support or refute it.
|
||||
|
||||
\paragraph{What is not claimed.}
|
||||
No worst-case guarantee is available here and none is asserted. A query whose
|
||||
relevant set is the corpus --- $k$ comparable to $N$, or a request that must
|
||||
enumerate every match of a basis element every row carries --- costs time
|
||||
linear in the corpus, under this design and under any other. A query whose
|
||||
region bounds all exceed the $k$-th best score degrades the routing to a visit
|
||||
of every part, which is the cost of the baseline plus the cost of having
|
||||
consulted the summaries. High intrinsic dimension makes that degradation the
|
||||
common case rather than the rare one~\cite{weber1998quantitative,beyer1999nn},
|
||||
and graph indexes have their own hard instances~\cite{indyk2023worst}. The
|
||||
design is an expected-cost design; the protocol reports the tail as well as the
|
||||
mean so that a claim about the mean cannot hide behind it.
|
||||
|
||||
\paragraph{Status of the claims.}
|
||||
Sections~\ref{sec:setting}--\ref{sec:routing} describe a design and label each
|
||||
component \prior{} or \proposed{}; nothing in them is a measurement.
|
||||
Section~\ref{sec:method} specifies a measurement that has not been made.
|
||||
Section~\ref{sec:results} holds the table it will fill, and every cell in it
|
||||
prints \ph{} until the harness writes \texttt{results-data.tex}.
|
||||
Section~\ref{sec:related} places the design against published work, and
|
||||
Section~\ref{sec:limits} lists what would make the whole approach the wrong
|
||||
one.
|
||||
|
||||
\paragraph{Contributions.}
|
||||
\begin{itemize}[leftmargin=*]
|
||||
\item A retrieval index over a heterogeneous sparse basis, laid out inside the
|
||||
parts and granules of a merge-based column store, with block maxima aligned
|
||||
to granules so that the store's existing skipping unit is also the pruning
|
||||
unit (\S\ref{sec:index}).
|
||||
\item A part-level routing structure and the planner rule that reads it, built
|
||||
from bounds that metric indexes and dynamic pruning have used separately, and
|
||||
maintained through the merge lifecycle that produces the parts
|
||||
(\S\ref{sec:routing}).
|
||||
\item A measurement protocol for expected cost, which holds recall fixed while
|
||||
corpus size varies, fits an exponent to each cost counter, reports bytes
|
||||
fetched per correct answer, and separates the queries for which the design is
|
||||
expected to fail (\S\ref{sec:method}).
|
||||
\item A statement of which parts of the design are already published work, with
|
||||
the references, and which are not (\S\ref{sec:related}, Table~\ref{tab:status}).
|
||||
\end{itemize}
|
||||
@@ -0,0 +1,106 @@
|
||||
\section{Setting and notation}
|
||||
\label{sec:setting}
|
||||
|
||||
\subsection{The store}
|
||||
|
||||
We take the storage model of ClickHouse's MergeTree family as
|
||||
given~\cite{schulze2024clickhouse,clickhousedocs}. The store the design is
|
||||
intended for is Hanzo Datastore, a fork of ClickHouse~26.4 that keeps that
|
||||
model~\cite{hanzodatastore}. \prior{} A table is a set of
|
||||
parts $P_1,\dots,P_m$. A part is immutable: rows arrive in new parts and
|
||||
background merges replace a set of parts with one part holding their union,
|
||||
which is the log-structured merge discipline~\cite{oneil1996lsm}. Within a
|
||||
part, rows are sorted by the ordering key and divided into granules, by default
|
||||
of $8{,}192$ rows or of a byte budget, whichever binds first. The primary index
|
||||
is sparse: it stores the ordering key of the first row of each granule, so key
|
||||
predicates select ranges of granules rather than rows. Skipping indexes ---
|
||||
minimum/maximum, set, Bloom filters over tokens or $n$-grams --- are written
|
||||
inside the part, each summarising a run of granules, and exclude granules that
|
||||
cannot satisfy a predicate; they are the modern form of small materialized
|
||||
aggregates~\cite{moerkotte1998sma} and of the partitioning that data-skipping
|
||||
work tunes deliberately~\cite{sun2014skipping}.
|
||||
|
||||
Two index types in that family concern retrieval directly. The text index
|
||||
stores, per part, the rows in which each token occurs, with no weight, and is
|
||||
merged rather than rebuilt when parts merge; it answers membership predicates.
|
||||
The vector similarity index builds a USearch HNSW graph~\cite{malkov2020hnsw,
|
||||
usearch} whose default granularity is large enough that in practice one graph
|
||||
covers a whole part. Both are per part, so the work of a query is a sum over
|
||||
the parts it visits.
|
||||
|
||||
\subsection{Rows, queries, scores}
|
||||
|
||||
\begin{definition}[Basis and representation]
|
||||
A basis $\mathcal{B} = \bigcup_t \mathcal{B}_t$ is a set of elements, each
|
||||
belonging to one type $t$: a word, a linked entity, a code symbol, a coordinate
|
||||
of a learned sparse dictionary, a relation, or a structural motif. A row $x$
|
||||
carries a sparse non-negative vector $c(x) \in \mathbb{R}_{\ge 0}^{\mathcal{B}}$
|
||||
with few non-zero coordinates, and optionally a dense embedding
|
||||
$v(x) \in \mathbb{R}^d$.
|
||||
\end{definition}
|
||||
|
||||
\begin{definition}[Query and score]
|
||||
A query is $Q = \sum_{i \in A(Q)} q_i B_i$ with non-negative $q_i$ and active
|
||||
set $A(Q) \subseteq \mathcal{B}$, optionally with a dense vector $u(Q)$. The
|
||||
score is
|
||||
\[
|
||||
s(Q,x) \;=\; \sum_{i \in A(Q)} q_i\, c_i(x) \;+\; \lambda \langle u(Q), v(x)\rangle ,
|
||||
\qquad \lambda \ge 0 ,
|
||||
\]
|
||||
and the answer is the $k$ rows of highest score, written $S_k(Q)$, with
|
||||
$\theta_k(Q)$ the $k$-th highest score.
|
||||
\end{definition}
|
||||
|
||||
The form is the vector-space model~\cite{salton1975vsm} over a vocabulary that
|
||||
is not only words, and the hybrid term is the usual linear combination of a
|
||||
lexical and a dense leg. Restricting weights to be non-negative is what makes
|
||||
the bounds in \S\ref{sec:routing} hold as stated; learned sparse encoders of
|
||||
the SPLADE family produce non-negative weights by
|
||||
construction~\cite{formal2021splade}.
|
||||
|
||||
\subsection{What cost means here}
|
||||
|
||||
\begin{definition}[Cost counters]
|
||||
For a query $Q$ over a corpus of $N$ rows, $C_{\mathrm{parts}}$ is the number
|
||||
of parts read, $C_{\mathrm{gran}}$ the number of granules read,
|
||||
$C_{\mathrm{bytes}}$ the compressed bytes fetched, and $C_{\mathrm{route}}$ the
|
||||
number of routing summaries examined. The store publishes the first three as
|
||||
\texttt{SelectedParts}, \texttt{SelectedMarks} and
|
||||
\texttt{ReadCompressedBytes}; the fourth has to be instrumented.
|
||||
\end{definition}
|
||||
|
||||
\begin{definition}[Relevant neighbourhood]
|
||||
$G^{*}(Q)$ is the set of granules containing at least one row of $S_k(Q)$, and
|
||||
$P^{*}(Q)$ the set of parts containing at least one such granule.
|
||||
\end{definition}
|
||||
|
||||
Any method that reads at granule granularity and returns $S_k(Q)$ exactly must
|
||||
read every granule in $G^{*}(Q)$, so $|G^{*}(Q)|$ is a lower bound on
|
||||
$C_{\mathrm{gran}}$ for exact retrieval, and the harness reports it as the
|
||||
oracle row. It is a lower bound on reading the answer, not on finding it: a
|
||||
method also pays for whatever it reads to establish that nothing else
|
||||
qualifies, and that second term is what pruning and routing exist to reduce.
|
||||
|
||||
\paragraph{The hypothesis, stated so it can fail.}
|
||||
Let $D$ be a query distribution and $\rho$ a recall target. Corpora of size $N$
|
||||
are nested samples from one row distribution, so a row present at $N$ is
|
||||
present at every larger $N$. The hypothesis is that, with the knobs at each $N$
|
||||
set to the cheapest configuration reaching recall $\rho$,
|
||||
\[
|
||||
\mathbb{E}_{Q \sim D}\!\left[C(Q;N)\right] \;=\; a N^{\alpha},
|
||||
\qquad \alpha < 1
|
||||
\]
|
||||
where $C$ is parts, granules or bytes touched per correct answer, and that
|
||||
$\mathbb{E}[C_{\mathrm{gran}}] / \mathbb{E}[|G^{*}|]$ does not grow with $N$.
|
||||
The second part matters more than the first: an exponent below one can also be
|
||||
reached by a method whose answers get worse, which is why recall is held rather
|
||||
than reported.
|
||||
|
||||
\begin{remark}[Where the hypothesis must fail]
|
||||
Queries exist for which every method pays $\Theta(N)$: $k$ proportional to $N$;
|
||||
a request that enumerates all matches of a basis element carried by every row;
|
||||
a score distribution so flat that $\theta_k$ is met by most regions, leaving no
|
||||
bound able to exclude one. The protocol runs such queries as a separate stratum
|
||||
and reports them, rather than letting the average over a benign query
|
||||
distribution stand for the design's behaviour.
|
||||
\end{remark}
|
||||
@@ -0,0 +1,107 @@
|
||||
\section{The sparse-basis inverted index}
|
||||
\label{sec:index}
|
||||
|
||||
\subsection{Posting lists and top-\texorpdfstring{$k$}{k} evaluation}
|
||||
|
||||
\prior{} The index is an inverted file~\cite{zobel2006inverted}: basis element
|
||||
$B_i$ owns a posting list $L_i$ of pairs $(x, c_i(x))$ for every row with
|
||||
$c_i(x) > 0$, ordered by row identifier, with weights quantised to integer
|
||||
impacts~\cite{anh2002impact}. A query touches only the lists of $A(Q)$, so the
|
||||
work before any pruning is $\sum_{i \in A(Q)} |L_i|$ rather than anything
|
||||
proportional to the corpus. Top-$k$ evaluation uses the established
|
||||
document-at-a-time pruning rules: MaxScore~\cite{turtle1995maxscore},
|
||||
WAND~\cite{broder2003wand}, and block-max WAND, which stores the maximum impact
|
||||
of each block of a posting list and skips blocks whose combined bound cannot
|
||||
reach the current threshold~\cite{ding2011bmw}, with variable-sized blocks as a
|
||||
later refinement~\cite{mallia2017vbmw}. Their common shape is older than the
|
||||
inverted-file literature that carries them: maintain a threshold, order access
|
||||
by an upper bound, stop when the bound falls below the threshold, which is the
|
||||
instance-optimal threshold algorithm of Fagin et al.~\cite{fagin2003ta}.
|
||||
|
||||
None of that is proposed here. What the design does is decide where those
|
||||
structures live in a column store, and what a basis element is allowed to be.
|
||||
|
||||
\subsection{A basis that is not only words}
|
||||
|
||||
\prior{} Indexing something other than a word is long established. Explicit
|
||||
semantic analysis represents text as a weighted vector over Wikipedia concepts
|
||||
and retrieves through an inverted index over
|
||||
them~\cite{gabrilovich2007esa}. Entity-linked representations index documents
|
||||
by the entities they mention and rank with a joint word-and-entity
|
||||
model~\cite{xiong2017duet,dalton2014entity}. Code search has been served by
|
||||
inverted indexes over trigrams and symbols~\cite{cox2012trigram}. Learned
|
||||
sparse retrieval produces weights over a vocabulary from a transformer ---
|
||||
DeepCT~\cite{dai2020deepct}, DeepImpact~\cite{mallia2021deepimpact},
|
||||
COIL and uniCOIL~\cite{gao2021coil,lin2021unicoil}, SPLADE and its
|
||||
successors~\cite{formal2021splade,formal2021spladev2,formal2022spladepp} ---
|
||||
and a learned dictionary over model activations produces a sparse code whose
|
||||
coordinates are interpretable features~\cite{olshausen1996sparse,cunningham2023sae}.
|
||||
Structural vocabularies have their own literature: motifs are the small
|
||||
recurring subgraphs of a network~\cite{milo2002motifs}.
|
||||
|
||||
\proposed{} The proposal is a single weighted index over the union of those
|
||||
vocabularies, scored by one inner product. Each type is produced by its own
|
||||
extractor and carries its own weight scale: a lexical impact derived from a
|
||||
term-frequency model~\cite{robertson2009bm25}, an entity linker's confidence, a
|
||||
parser's certainty that a symbol is a definition rather than a mention, a
|
||||
dictionary coefficient, a relation's confidence, a motif's count. Those scales
|
||||
have nothing in common, so the score carries a per-type factor:
|
||||
\[
|
||||
s(Q,x) \;=\; \sum_t w_t \sum_{i \in A_t(Q)} q_i\, c_i(x),
|
||||
\qquad w_t \ge 0 .
|
||||
\]
|
||||
Fitting $w_t$ is a supervised problem on relevance data and we do not solve it
|
||||
here; we state it as the open parameter it is (\S\ref{sec:limits}).
|
||||
|
||||
\begin{proposition}[Calibration does not invalidate pruning]
|
||||
Let $\mathrm{UB}_t$ bound $\sum_{i \in A_t(Q)} q_i c_i(x)$ over a set of rows
|
||||
$R$. Then $\sum_t w_t \mathrm{UB}_t$ bounds $s(Q,x)$ over $R$ for any
|
||||
$w_t \ge 0$.
|
||||
\end{proposition}
|
||||
\begin{proof}
|
||||
Immediate from non-negativity of $w_t$ and of each per-type bound.
|
||||
\end{proof}
|
||||
|
||||
The consequence is practical rather than deep: block maxima and region
|
||||
summaries can be built per type, once, and re-weighted at query time without
|
||||
being rebuilt, so a change of calibration is a change to the planner and not to
|
||||
the index. Weights are required to be non-negative for this; a signed
|
||||
dictionary would need both a maximum and a minimum per block and is outside the
|
||||
design as written.
|
||||
|
||||
\subsection{Layout inside a part}
|
||||
|
||||
\proposed{} A posting list is built per part, over row offsets within that
|
||||
part, and its blocks are the part's granules. Two consequences follow from that
|
||||
choice.
|
||||
|
||||
The block maximum of element $B_i$ over granule $g$ is
|
||||
$M_{g,i} = \max_{x \in g} c_i(x)$, so the pruning unit and the store's skipping
|
||||
unit are the same object. A granule excluded by a bound is a granule the
|
||||
store's reader was already able to skip, and a granule excluded by a predicate
|
||||
is one the pruning need not bound. The part's own summary is
|
||||
$M_{P,i} = \max_{g \in P} M_{g,i}$, which is the first level of
|
||||
\S\ref{sec:routing}.
|
||||
|
||||
Merges maintain the structure rather than rebuilding it from rows: posting
|
||||
lists of the source parts are concatenated under an offset remapping, block
|
||||
maxima are carried, and part maxima are recomputed as a maximum over the
|
||||
carried block maxima. ClickHouse's text index already merges instead of
|
||||
rebuilding, which is the precedent for the operation; what it does not carry is
|
||||
weights, because it stores occurrence and not impact.
|
||||
|
||||
\subsection{Where this sits relative to learned sparse retrieval}
|
||||
|
||||
\prior{} The efficiency literature on learned sparse retrieval is directly
|
||||
relevant, because it reports that learned weights weaken the assumptions
|
||||
dynamic pruning was built on: impact distributions are flatter, so bounds
|
||||
separate worse and skipping is less
|
||||
effective~\cite{mackenzie2021wacky,lassance2022efficiency}. The responses are
|
||||
guided traversal~\cite{mallia2022guided}, block-max
|
||||
pruning~\cite{mallia2024bmp}, and inverted indexes whose blocks carry a summary
|
||||
vector rather than a scalar maximum~\cite{bruch2024seismic}. A mixed basis
|
||||
inherits that problem with an additional term: the flatness of the impact
|
||||
distribution now differs per type, so a single bound over a heterogeneous
|
||||
active set is looser than either leg's bound alone. The protocol therefore
|
||||
reports the pruning ratio per type as well as in aggregate
|
||||
(\S\ref{sec:method}).
|
||||
@@ -0,0 +1,145 @@
|
||||
\section{Part-level routing}
|
||||
\label{sec:routing}
|
||||
|
||||
\subsection{What the store already prunes, and what it does not}
|
||||
|
||||
\prior{} Predicate pruning in a MergeTree table happens at three levels: parts
|
||||
excluded by the minimum and maximum of the partition key, granule ranges
|
||||
selected by the sparse primary index, and granules excluded by skipping
|
||||
indexes. All three read a predicate. Similarity has no predicate, so none of
|
||||
them applies to it, and the vector index is consulted once per surviving part.
|
||||
The same arrangement in Lucene and Milvus has the same
|
||||
consequence~\cite{lucene,wang2021milvus}: the cost of a similarity query is a
|
||||
sum over segments, and it grows as segments accumulate.
|
||||
|
||||
\subsection{Region summaries}
|
||||
|
||||
The bounds in this subsection are established \prior{}; their arrangement per
|
||||
part is what is proposed \proposed{}. Rows inside a part are grouped into
|
||||
regions --- by $k$-means over the dense vectors, or by granule ranges when the
|
||||
ordering key already clusters similar rows. Region $R$ carries
|
||||
|
||||
\begin{itemize}[leftmargin=*,itemsep=1pt]
|
||||
\item a centroid $\mu_R$ and a covering radius
|
||||
$r_R = \max_{x \in R}\lVert v(x) - \mu_R \rVert$;
|
||||
\item a basis summary: $M_{R,i} = \max_{x \in R} c_i(x)$ for the elements
|
||||
present, truncated to the largest entries with one residual bound
|
||||
$M_{R,\ast}$ covering the rest;
|
||||
\item statistics: the row count, the number of rows carrying each retained
|
||||
element, and quantiles of the score distribution of the region under a
|
||||
reference query model.
|
||||
\end{itemize}
|
||||
|
||||
The centroid-and-radius pair is the routing object of metric trees --- the
|
||||
branch-and-bound $k$-nearest-neighbour search of Fukunaga and
|
||||
Narendra~\cite{fukunaga1975branch}, metric trees~\cite{uhlmann1991metric}, and
|
||||
the M-tree's covering radius~\cite{ciaccia1997mtree} --- and bounding regions
|
||||
for pruning go back to the R-tree~\cite{guttman1984rtree}. The statistics are
|
||||
what distributed retrieval calls resource-selection evidence: CORI ranks
|
||||
collections by term statistics~\cite{callan1995cori}, ReDDE estimates how many
|
||||
relevant documents a collection holds~\cite{si2003redde}, Taily models each
|
||||
shard's score distribution and selects on its tail~\cite{aly2013taily}, and
|
||||
selective search over topically clustered shards is the arrangement in
|
||||
which such selection pays~\cite{kulkarni2015selective}, resting on the older
|
||||
cluster hypothesis~\cite{vanrijsbergen1979ir}. The proposal contributes the
|
||||
container, not the bound or the statistic.
|
||||
|
||||
\begin{proposition}[Region bounds]
|
||||
For non-negative sparse weights, every $x \in R$ satisfies
|
||||
\[
|
||||
\sum_{i \in A(Q)} q_i\, c_i(x) \;\le\; \mathrm{UB}_s(Q,R)
|
||||
\;=\; \sum_{i \in A(Q)} q_i\, \widehat{M}_{R,i},
|
||||
\]
|
||||
where $\widehat{M}_{R,i}$ is $M_{R,i}$ when retained and $M_{R,\ast}$
|
||||
otherwise. For the dense leg,
|
||||
\[
|
||||
\langle u, v(x)\rangle \le \langle u, \mu_R\rangle + r_R \lVert u \rVert,
|
||||
\qquad
|
||||
\lVert u - v(x)\rVert \ge \lVert u - \mu_R \rVert - r_R .
|
||||
\]
|
||||
\end{proposition}
|
||||
\begin{proof}
|
||||
The sparse bound is termwise. The dense bounds are Cauchy--Schwarz and the
|
||||
triangle inequality applied to $v(x) - \mu_R$, whose norm is at most $r_R$ by
|
||||
the definition of the covering radius.
|
||||
\end{proof}
|
||||
|
||||
\subsection{From regions to a table-level structure}
|
||||
|
||||
\proposed{} A part summary is the union of its region summaries: an enclosing
|
||||
ball, the maximum of the region maxima per basis element, and summed
|
||||
statistics. Part summaries are themselves grouped into a tree with a small
|
||||
fan-out, each interior node carrying the same fields computed over its
|
||||
children, which is the M-tree's construction applied to parts rather than to
|
||||
rows. The structure is metadata: its size is proportional to the number of
|
||||
regions, not to the number of rows, and it is maintained by the merge that
|
||||
produces a part, since a merge replaces the summaries of its inputs with one
|
||||
computed from the output it already has in memory.
|
||||
|
||||
\subsection{The planner rule}
|
||||
|
||||
\proposed{} A retrieval query is answered by best-first traversal over that
|
||||
tree. The planner keeps a priority queue of nodes ordered by
|
||||
$\mathrm{UB}(Q,\cdot)$ and a running threshold $\theta$, the $k$-th best score
|
||||
found. It pops the node of highest bound; if the bound is at most $\theta$ it
|
||||
stops, because no descendant can beat the threshold. Interior nodes expand to
|
||||
children, parts expand to their regions, and a region is evaluated by the
|
||||
posting lists and, for the dense leg, by the part's approximate graph
|
||||
restricted to the region. This is branch and bound over the same bounds the
|
||||
metric-tree literature uses, with the threshold rule of Fagin et
|
||||
al.~\cite{fagin2003ta}.
|
||||
|
||||
Two departures from exactness are deliberate and are reported separately. The
|
||||
per-part graph search is approximate, so the answer is exact only relative to
|
||||
what that search returns; recall is governed by its parameters, and the
|
||||
protocol fixes recall rather than the parameters. And an approximate routing
|
||||
variant stops when the best remaining bound falls below $\theta/(1+\varepsilon)$,
|
||||
or after a budget of parts, which is the same trade SPANN makes when it prunes
|
||||
posting lists by their centroid distance relative to the
|
||||
closest~\cite{chen2021spann}.
|
||||
|
||||
Predicates compose with routing rather than replacing it: partition pruning and
|
||||
primary-index ranges narrow the parts before summaries are read, and a
|
||||
selective predicate can make routing unnecessary. The hard case for every
|
||||
system in this family is a predicate that is selective and uncorrelated with
|
||||
the index structure, which is what filtered graph search
|
||||
addresses~\cite{gollapudi2023filtered,patel2024acorn}; the design as written
|
||||
carries the predicate into the region scan and does not solve the general case.
|
||||
|
||||
\subsection{Where the bounds stop working}
|
||||
|
||||
\proposed{} \emph{Interpretation, not measurement.} Three failure modes are
|
||||
predictable and the protocol measures each.
|
||||
|
||||
\textbf{High intrinsic dimension.} Covering radii of $k$-means regions approach
|
||||
the distances between centroids, every region's ball bound exceeds $\theta$,
|
||||
and the traversal visits everything. That is the classical result that
|
||||
partition-based structures degrade to a scan in high
|
||||
dimension~\cite{weber1998quantitative,beyer1999nn}; the routed store then costs
|
||||
the baseline plus the summaries it read, and the protocol reports the summary
|
||||
cost separately so the overhead is visible rather than folded into a total.
|
||||
|
||||
\textbf{Heavy basis elements.} An element carried with high weight by rows
|
||||
everywhere raises $\mathrm{UB}_s$ for every region at once. Dynamic pruning has
|
||||
the same weakness for common terms, which is why impact-ordered and
|
||||
score-at-a-time evaluation exist~\cite{anh2002impact,lin2015anytime}.
|
||||
|
||||
\textbf{Small parts.} A freshly inserted part holds few rows and a summary
|
||||
computed over them, so its bound is tight but its share of the corpus is
|
||||
small; many such parts make the summary scan the dominant term until merges
|
||||
consolidate them. Part count is therefore a controlled variable of the
|
||||
protocol, not a detail of the deployment.
|
||||
|
||||
\subsection{Routing by summaries elsewhere}
|
||||
|
||||
\prior{} Summary-first retrieval appears in the retrieval-augmented generation
|
||||
literature in a different form. GraphRAG partitions an entity graph into
|
||||
communities, summarises each with a language model, and answers a global query
|
||||
by mapping over community summaries and reducing their
|
||||
partials~\cite{edge2024graphrag}; RAPTOR builds a tree of recursive abstractive
|
||||
summaries and retrieves from its levels~\cite{sarthi2024raptor}. The structural
|
||||
analogy is exact --- a summary per region, consulted before the rows --- and
|
||||
the mechanism is not: those summaries are natural language read by a model, and
|
||||
the map step visits every community at a level rather than excluding any by a
|
||||
bound. The routing here excludes regions arithmetically and reads no summary
|
||||
text.
|
||||
@@ -0,0 +1,117 @@
|
||||
\section{How the claim is to be measured}
|
||||
\label{sec:method}
|
||||
|
||||
The protocol below is what the lane that fills Section~\ref{sec:results} is to
|
||||
implement, in \texttt{hanzoai/benchmarks}, where it has not yet been published.
|
||||
It is written here before any run exists so that the analysis cannot be chosen
|
||||
after seeing the results.
|
||||
|
||||
\subsection{Corpora}
|
||||
|
||||
Corpus sizes are $N \in \{10^5, 10^6, 10^7, 10^8, 10^9\}$, drawn as nested
|
||||
samples of one source collection, so every row present at one size is present
|
||||
at every larger one and the growth of a cost counter is not confounded by a
|
||||
change of content. The first pass is configured on English Wikipedia passages
|
||||
(the 2022-12 dump, 35{,}167{,}920 passages) with published all-MiniLM-L6-v2
|
||||
vectors~\cite{reimers2019sbert}, sampled by a hash threshold on the article
|
||||
identifier, so that each size is a set of whole articles and a subset of the
|
||||
next. That collection reaches $3.5 \times 10^7$ rows; the two larger sizes need
|
||||
another source, which is not chosen. The first pass covers $10^5$ to $10^7$ on
|
||||
upstream ClickHouse~26.4.2.10, the release \texttt{hanzoai/datastore} is forked
|
||||
from~\cite{hanzodatastore}; the routed store follows. The collection, its licence
|
||||
and its digest are recorded by the harness and printed in the caption of
|
||||
Table~\ref{tab:cost}.
|
||||
|
||||
Part geometry is controlled rather than observed. Merge settings are fixed
|
||||
across sizes, no run collapses the table to a single part, and the part count
|
||||
and size distribution at each $N$ are reported with the results: routing is a
|
||||
structure over parts, and a table with one part has nothing to route.
|
||||
|
||||
\subsection{Queries, ground truth, recall}
|
||||
|
||||
A query set is sampled once from the intended distribution and held fixed
|
||||
across sizes. The first pass samples Natural Questions
|
||||
(open)~\cite{kwiatkowski2019nq,lee2019orqa}, questions posed to a search engine
|
||||
that Wikipedia answers: 200 from the training split to set the knobs and 1{,}000
|
||||
from the validation split to report. A question is embedded with the corpus
|
||||
model for the dense leg, and its content tokens, tokenised as the store's text
|
||||
index tokenises the column, form the lexical leg. Exact answers are computed per size by exhaustive scoring, which
|
||||
is affordable offline and is what makes recall meaningful; the ground-truth
|
||||
procedure and its cost are recorded, as billion-scale benchmarks
|
||||
do~\cite{simhadri2022bigann}.
|
||||
|
||||
Cost is read at a held recall target. For each system and each $N$, the knobs
|
||||
--- graph search effort, part budget, the routing slack $\varepsilon$ --- are
|
||||
set to the cheapest configuration whose mean recall@$k$ reaches $\rho$, and the
|
||||
chosen values are published with the row. Reporting cost at a fixed knob
|
||||
instead would let one system's cost fall because its answers did, which is the
|
||||
failure mode that recall-against-throughput curves exist to
|
||||
prevent~\cite{aumuller2020annb}.
|
||||
|
||||
\subsection{Counters}
|
||||
|
||||
Each query records, from the store's own profile events and from
|
||||
instrumentation:
|
||||
|
||||
\begin{itemize}[leftmargin=*,itemsep=1pt]
|
||||
\item \texttt{SelectedParts}, \texttt{SelectedPartsTotal},
|
||||
\texttt{SelectedMarks} and \texttt{SelectedMarksTotal} --- what index
|
||||
analysis kept and what it started from;
|
||||
\item \texttt{ReadCompressedBytes} and \texttt{OSReadBytes} --- bytes from the
|
||||
store and bytes from the device;
|
||||
\item routing summaries examined, posting blocks decoded, posting blocks
|
||||
skipped by a bound;
|
||||
\item recall@$k$ against the exact answer, and latency.
|
||||
\end{itemize} Cold and warm runs are reported separately, cold meaning the page
|
||||
cache was dropped before the query. Latency is reported at the median and the
|
||||
99th percentile: a design justified by an expectation has to show its tail, and
|
||||
the reasons are the familiar ones~\cite{dean2013tail}.
|
||||
|
||||
Two derived quantities carry the argument. \emph{Cost per correct answer}
|
||||
divides each counter by the number of returned rows that are in the exact
|
||||
top-$k$: parts, granules and bytes per correct answer. It is the quantity to
|
||||
compare across systems, because it prices recall and work in one number, and it
|
||||
is the quantity the exponent is fitted to. \emph{Oracle ratio} is
|
||||
$C_{\mathrm{gran}}/|G^{*}(Q)|$, which says how much more than the answer a query
|
||||
had to read.
|
||||
|
||||
\subsection{Fitting an exponent}
|
||||
|
||||
For each system and for parts, granules and bytes touched per correct answer,
|
||||
$C(N) = aN^{\alpha}$ is fitted by regressing $\log \mathbb{E}[C]$ on $\log N$ by
|
||||
least squares over the size grid, at the held recall target. The interval on
|
||||
$\alpha$ comes from a bootstrap over queries: resample the query set with
|
||||
replacement, recompute the per-size means, refit~\cite{efron1993bootstrap}. The
|
||||
same is done for the raw counters, and for the 99th percentile of each, since a
|
||||
mean exponent below one is consistent with a tail exponent at one.
|
||||
|
||||
Local slopes between consecutive sizes are reported beside the global fit,
|
||||
because curvature is what distinguishes a genuine exponent from an average of
|
||||
two regimes. A straight line through four or five points spanning three decades
|
||||
is weak evidence for a power law, and we do not claim the functional form: the
|
||||
claim is about the interval on $\alpha$ under this protocol, which is a weaker
|
||||
and checkable statement~\cite{clauset2009powerlaw}.
|
||||
|
||||
\subsection{Systems compared}
|
||||
|
||||
\begin{itemize}[leftmargin=*,itemsep=1pt]
|
||||
\item \textbf{scan} --- exhaustive scoring with no index, the upper bound on
|
||||
cost and the reference for correctness.
|
||||
\item \textbf{base} --- upstream ClickHouse 26.4: primary index, skipping
|
||||
indexes, text index for membership, one vector similarity index per part.
|
||||
\item \textbf{route} --- the design of this paper.
|
||||
\item \textbf{oracle} --- the granules and parts holding the exact top-$k$,
|
||||
which no system can beat and every system is measured against.
|
||||
\end{itemize}
|
||||
|
||||
\subsection{Strata, including the ones expected to fail}
|
||||
|
||||
Results are reported over the whole query set and over strata: by the size of
|
||||
the candidate set $\sum_{i \in A(Q)}|L_i|$, by the margin
|
||||
$\theta_k - \theta_{k+1}$, by the number of basis types the query activates,
|
||||
and by whether the query carries a selective predicate. Two adversarial strata
|
||||
are run deliberately --- a query over an element every row carries, and a query
|
||||
whose score distribution is flat enough that no region bound separates --- and
|
||||
their rows are expected to show cost linear in $N$. A protocol that reported
|
||||
only the mean over a benign distribution could not tell a design that routes
|
||||
from one that got a convenient query set.
|
||||
@@ -0,0 +1,70 @@
|
||||
\section{Results}
|
||||
\label{sec:results}
|
||||
|
||||
\emph{This section is a placeholder.} The lane that fills it is starting and
|
||||
has written no results file, so every cell below prints \ph{} and no sentence
|
||||
in this paper quotes a number. When the harness writes that file, these tables
|
||||
fill from it and the conditions of the run appear in the captions.
|
||||
|
||||
\begin{table}[h]
|
||||
\centering
|
||||
\small
|
||||
\setlength{\tabcolsep}{5pt}
|
||||
\begin{tabular}{@{}lrrrrrrr@{}}
|
||||
\toprule
|
||||
& parts & granules & MiB & KiB/ans & recall@$k$ & p50 ms & p99 ms \\
|
||||
\midrule
|
||||
\multicolumn{8}{@{}l}{\textit{scan}} \\
|
||||
\costrow{scan/1e5}{\quad $10^5$}
|
||||
\costrow{scan/1e6}{\quad $10^6$}
|
||||
\costrow{scan/1e7}{\quad $10^7$}
|
||||
\midrule
|
||||
\multicolumn{8}{@{}l}{\textit{base} --- upstream ClickHouse 26.4} \\
|
||||
\costrow{base/1e5}{\quad $10^5$}
|
||||
\costrow{base/1e6}{\quad $10^6$}
|
||||
\costrow{base/1e7}{\quad $10^7$}
|
||||
\costrow{base/1e8}{\quad $10^8$}
|
||||
\costrow{base/1e9}{\quad $10^9$}
|
||||
\midrule
|
||||
\multicolumn{8}{@{}l}{\textit{route} --- sparse basis and part-level routing} \\
|
||||
\costrow{route/1e5}{\quad $10^5$}
|
||||
\costrow{route/1e6}{\quad $10^6$}
|
||||
\costrow{route/1e7}{\quad $10^7$}
|
||||
\costrow{route/1e8}{\quad $10^8$}
|
||||
\costrow{route/1e9}{\quad $10^9$}
|
||||
\midrule
|
||||
\multicolumn{8}{@{}l}{\textit{oracle} --- the parts and granules holding the exact top-$k$} \\
|
||||
\costrow{oracle/1e6}{\quad $10^6$}
|
||||
\costrow{oracle/1e9}{\quad $10^9$}
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
\caption{Cost at a held recall target, per query, averaged over the query set.
|
||||
Host \res{run/host}; store \res{run/version}; $k = \res{run/k}$; recall target
|
||||
\res{run/recall}; queries \res{run/queries}. Part counts at each size:
|
||||
\res{run/parts/1e6} at $10^6$ and \res{run/parts/1e9} at $10^9$. Every cell is
|
||||
empty until the harness writes it.}
|
||||
\label{tab:cost}
|
||||
\end{table}
|
||||
|
||||
\begin{table}[h]
|
||||
\centering
|
||||
\small
|
||||
\begin{tabular}{@{}lllll@{}}
|
||||
\toprule
|
||||
& $\alpha$ parts & $\alpha$ granules & $\alpha$ bytes & $\alpha$ summaries \\
|
||||
& per answer & per answer & per answer & per query \\
|
||||
\midrule
|
||||
\fitrow{base}{base}
|
||||
\fitrow{route}{route}
|
||||
\fitrow{oracle}{oracle}
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
\caption{Fitted exponents of $C(N) = aN^{\alpha}$ for parts, granules and bytes
|
||||
touched per correct answer, and for routing summaries examined per query, with
|
||||
95\% bootstrap intervals over queries, at the recall target of
|
||||
Table~\ref{tab:cost}. The
|
||||
hypothesis of \S\ref{sec:setting} is a route row whose intervals lie below one
|
||||
and whose oracle ratio does not grow; the local slopes behind each fit are
|
||||
published with the run.}
|
||||
\label{tab:fit}
|
||||
\end{table}
|
||||
@@ -0,0 +1,152 @@
|
||||
\section{Related work}
|
||||
\label{sec:related}
|
||||
|
||||
\paragraph{Inverted files and dynamic pruning.}
|
||||
The inverted file and its evaluation strategies are the substrate of the sparse
|
||||
half of this design~\cite{zobel2006inverted}. Scoring follows a
|
||||
term-weighting model of the BM25 family~\cite{robertson2009bm25} or a learned
|
||||
one, and top-$k$ evaluation avoids scoring every posting by keeping an upper
|
||||
bound per term (MaxScore~\cite{turtle1995maxscore},
|
||||
WAND~\cite{broder2003wand}) or per block of a posting list (block-max
|
||||
WAND~\cite{ding2011bmw}, with variable blocks~\cite{mallia2017vbmw}). Their
|
||||
guarantee is that the top-$k$ is exact, not that the work is sublinear; the
|
||||
work depends on how well the bounds separate, which is a property of the data.
|
||||
Impact-ordered indexes and score-at-a-time evaluation trade exactness for a
|
||||
budget that can be set in advance~\cite{anh2002impact,lin2015anytime}, and
|
||||
PISA~\cite{mallia2019pisa} and Lucene~\cite{lucene} are the reference
|
||||
implementations. The threshold algorithm of Fagin et al.\ is the general form
|
||||
of all of it, with an instance-optimality result that is the nearest thing in
|
||||
this literature to the claim of \S\ref{sec:setting}~\cite{fagin2003ta}.
|
||||
|
||||
\paragraph{Learned sparse retrieval.}
|
||||
DeepCT~\cite{dai2020deepct} and DeepImpact~\cite{mallia2021deepimpact} learn
|
||||
term weights for an inverted index; COIL~\cite{gao2021coil} and
|
||||
uniCOIL~\cite{lin2021unicoil} reduce contextualised matching to one weight per
|
||||
term; SPLADE learns a sparse expansion over the vocabulary with non-negative
|
||||
weights~\cite{formal2021splade,formal2021spladev2,formal2022spladepp}. The
|
||||
efficiency consequence is documented and is the reason our protocol reports
|
||||
pruning ratios: learned weights flatten impact distributions and weaken
|
||||
skipping~\cite{mackenzie2021wacky,lassance2022efficiency}, which guided
|
||||
traversal~\cite{mallia2022guided}, block-max pruning~\cite{mallia2024bmp} and
|
||||
block summaries richer than a scalar maximum~\cite{bruch2024seismic} answer in
|
||||
different ways. Our design is on the same side of that line as Seismic: a
|
||||
region summary is a vector of maxima, not a single number.
|
||||
|
||||
\paragraph{Vocabularies that are not words.}
|
||||
Explicit semantic analysis indexes Wikipedia concepts~\cite{gabrilovich2007esa};
|
||||
entity-linked ranking indexes entities beside
|
||||
words~\cite{dalton2014entity,xiong2017duet}; code search indexes trigrams and
|
||||
symbols~\cite{cox2012trigram}; dictionary learning and sparse autoencoders
|
||||
produce interpretable sparse codes over model
|
||||
activations~\cite{olshausen1996sparse,cunningham2023sae}; network motifs are
|
||||
the structural vocabulary of graphs~\cite{milo2002motifs}. Each is prior art
|
||||
for one basis type. We are not aware of a published system that scores one
|
||||
weighted inner product over all of them inside a column store's part layout,
|
||||
and that arrangement is what we propose.
|
||||
|
||||
\paragraph{Partition-based approximate nearest neighbours.}
|
||||
Clustering the collection and probing a few clusters is the oldest routing idea
|
||||
in this space: the inverted file over visual words~\cite{sivic2003videogoogle},
|
||||
IVF with product quantisation~\cite{jegou2011pq}, the inverted
|
||||
multi-index~\cite{babenko2012imi}, Faiss as the reference
|
||||
implementation~\cite{johnson2021faiss}, ScaNN's anisotropic
|
||||
quantisation~\cite{guo2020scann} and SOAR's spilled
|
||||
assignments~\cite{sun2023soar}. SPANN keeps centroids in memory and posting
|
||||
lists on disk, and prunes the lists to search by their centroid distance
|
||||
relative to the nearest one~\cite{chen2021spann}, which is the approximate
|
||||
variant of our planner rule. The difference in our setting is not the rule but
|
||||
what it ranges over: the partitions are parts of a table that also holds
|
||||
scalars, that merge on a schedule the store controls, and that a predicate may
|
||||
have already narrowed.
|
||||
|
||||
\paragraph{Graph indexes.}
|
||||
HNSW~\cite{malkov2020hnsw} and Vamana/DiskANN~\cite{subramanya2019diskann},
|
||||
with streaming updates~\cite{singh2021freshdiskann} and filter-aware
|
||||
variants~\cite{gollapudi2023filtered,patel2024acorn}, are what a column store
|
||||
builds per part today. Their empirical scaling is excellent and their
|
||||
worst-case behaviour is not
|
||||
guaranteed~\cite{prokhorenkova2020graph,indyk2023worst}; a design that claims
|
||||
expected cost, as this one does, cannot borrow a guarantee from them.
|
||||
|
||||
\paragraph{Metric and tree indexes.}
|
||||
Centroid-and-radius pruning is old and well understood: branch-and-bound
|
||||
nearest neighbours over a hierarchical $k$-means
|
||||
tree~\cite{fukunaga1975branch}, metric trees~\cite{uhlmann1991metric}, the
|
||||
M-tree's covering radius~\cite{ciaccia1997mtree}, vocabulary
|
||||
trees~\cite{nister2006vocabtree}, FLANN's randomised and hierarchical
|
||||
structures~\cite{muja2014flann}, and, in the spatial setting, the
|
||||
R-tree~\cite{guttman1984rtree}. The reason these are not the whole answer is
|
||||
equally well documented: above modest dimension the bounds stop separating and
|
||||
a scan wins~\cite{weber1998quantitative,beyer1999nn}, which is why locality
|
||||
sensitive hashing offers a sublinear query-time guarantee for the approximate
|
||||
problem instead~\cite{indyk1998lsh}.
|
||||
|
||||
\paragraph{Routing across collections.}
|
||||
Selective search partitions a corpus into topical shards and consults a few per
|
||||
query~\cite{kulkarni2015selective}, choosing them with collection
|
||||
statistics~\cite{callan1995cori}, estimates of relevant-document
|
||||
counts~\cite{si2003redde}, or the tail of a shard's score
|
||||
distribution~\cite{aly2013taily}; the cluster hypothesis is the assumption that
|
||||
makes it work~\cite{vanrijsbergen1979ir}. A part summary is a shard statistic
|
||||
under another name, computed by the engine rather than by an offline
|
||||
partitioner, and maintained by merges rather than by a repartitioning job.
|
||||
|
||||
\paragraph{Column stores and skipping.}
|
||||
Modern column stores are surveyed in~\cite{abadi2013columnstores}; ClickHouse's
|
||||
architecture, including parts, granules and the sparse primary index, is
|
||||
described by its authors~\cite{schulze2024clickhouse,clickhousedocs}. Skipping
|
||||
by per-block aggregates is small materialized
|
||||
aggregates~\cite{moerkotte1998sma}, and choosing a layout that makes skipping
|
||||
effective is fine-grained partitioning~\cite{sun2014skipping}. The
|
||||
log-structured merge discipline that produces the parts is
|
||||
standard~\cite{oneil1996lsm}. Vector indexes inside such engines are per
|
||||
segment or per part~\cite{lucene,wang2021milvus,usearch}, which is the
|
||||
condition this paper starts from.
|
||||
|
||||
\paragraph{Summary-first retrieval for generation.}
|
||||
GraphRAG routes a global question through community
|
||||
summaries~\cite{edge2024graphrag} and RAPTOR through a tree of recursive
|
||||
summaries~\cite{sarthi2024raptor}. Both read summaries before rows, as we do,
|
||||
and neither excludes a summary by an arithmetic bound, which is the whole of
|
||||
the difference.
|
||||
|
||||
\paragraph{Evaluation practice.}
|
||||
Recall-against-cost curves rather than single operating
|
||||
points~\cite{aumuller2020annb}, published ground truth and fixed budgets at
|
||||
billion scale~\cite{simhadri2022bigann}, percentile latency rather than
|
||||
means~\cite{dean2013tail}, bootstrap intervals~\cite{efron1993bootstrap} and
|
||||
caution about power-law fits~\cite{clauset2009powerlaw} are the practices
|
||||
\S\ref{sec:method} assembles.
|
||||
|
||||
\begin{table}[h]
|
||||
\centering
|
||||
\footnotesize
|
||||
\begin{tabular}{@{}p{0.46\textwidth}ll@{}}
|
||||
\toprule
|
||||
component & status & references \\
|
||||
\midrule
|
||||
MergeTree parts, granules, sparse primary index & prior art & \cite{schulze2024clickhouse,clickhousedocs} \\
|
||||
Skipping indexes; per-part text and vector indexes & prior art & \cite{clickhousedocs,moerkotte1998sma} \\
|
||||
Weighted inverted file, impacts & prior art & \cite{zobel2006inverted,anh2002impact} \\
|
||||
MaxScore, WAND, Block-Max WAND & prior art & \cite{turtle1995maxscore,broder2003wand,ding2011bmw} \\
|
||||
Threshold rule, instance optimality & prior art & \cite{fagin2003ta} \\
|
||||
Learned sparse weights (SPLADE, uniCOIL, DeepImpact) & prior art & \cite{formal2021splade,mallia2021deepimpact,lin2021unicoil} \\
|
||||
Concept, entity, symbol, motif vocabularies & prior art & \cite{gabrilovich2007esa,xiong2017duet,cox2012trigram,milo2002motifs} \\
|
||||
One weighted index over a mixed basis & proposed & --- \\
|
||||
Per-type calibration, bounds preserved & proposed & --- \\
|
||||
Posting blocks aligned to granules & proposed & --- \\
|
||||
Summaries maintained by part merges & proposed & --- \\
|
||||
Centroid-and-radius region bounds & prior art & \cite{fukunaga1975branch,ciaccia1997mtree} \\
|
||||
Shard statistics for selection & prior art & \cite{callan1995cori,si2003redde,aly2013taily} \\
|
||||
Cluster probing for vectors (IVF, PQ, SPANN, ScaNN) & prior art & \cite{jegou2011pq,chen2021spann,guo2020scann} \\
|
||||
Graph indexes per part (HNSW, DiskANN) & prior art & \cite{malkov2020hnsw,subramanya2019diskann} \\
|
||||
Summaries read before rows (GraphRAG, RAPTOR) & prior art & \cite{edge2024graphrag,sarthi2024raptor} \\
|
||||
Part-level routing before per-part graph search & proposed & --- \\
|
||||
Held-recall scaling protocol, oracle ratio & proposed & --- \\
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
\caption{What this paper takes and what it adds. A row marked proposed is a
|
||||
design decision of ours; it is not a claim that no related construction
|
||||
exists, only that we are citing none for it.}
|
||||
\label{tab:status}
|
||||
\end{table}
|
||||
@@ -0,0 +1,49 @@
|
||||
\section{Limitations}
|
||||
\label{sec:limits}
|
||||
|
||||
\textbf{Nothing here is measured.} The design has not been built at the scale
|
||||
it is argued for, and the protocol in \S\ref{sec:method} has produced no
|
||||
numbers. Every statement about cost in this paper is a statement about what a
|
||||
bound permits, not about what a machine did.
|
||||
|
||||
\textbf{The per-type calibration is unsolved.} A single score over a mixed
|
||||
basis needs weights $w_t$ that put an entity linker's confidence and a lexical
|
||||
impact on one scale. Learning them requires relevance judgements over the
|
||||
mixture, which the lane does not yet have; until it does, any reported
|
||||
effectiveness is the effectiveness of a hand-set combination, and should be
|
||||
read as such.
|
||||
|
||||
\textbf{Bounds loosen with dimension, and that is not fixable by tuning.}
|
||||
Covering radii stop separating regions in high intrinsic dimension, so the
|
||||
exact routing rule can degenerate to a full visit; the approximate rule then
|
||||
carries the load and converts a cost problem into a recall problem. The
|
||||
protocol measures where that transition happens for a given corpus, but no
|
||||
choice of region size avoids it.
|
||||
|
||||
\textbf{The routing metadata is itself a cost that grows.} The number of region
|
||||
summaries is proportional to the number of regions, so a design that scanned
|
||||
them all would be linear in the corpus with a small constant. The tree over
|
||||
part summaries is what keeps that term sublinear, and its depth, fan-out and
|
||||
maintenance cost during merges are unmeasured.
|
||||
|
||||
\textbf{Merge-time maintenance is a write-side cost we do not price.} Carrying
|
||||
posting lists, block maxima and region summaries through a merge is work that
|
||||
competes with ingestion. A design that makes reads cheap and merges expensive
|
||||
can be a net loss for a table that is written more often than it is queried.
|
||||
|
||||
\textbf{The claim is about a query distribution.} An exponent fitted under one
|
||||
distribution says nothing about another. The strata in \S\ref{sec:method} are
|
||||
there so a reader can see which queries the average is made of, but the
|
||||
protocol cannot certify that a deployment's queries resemble the set measured.
|
||||
|
||||
\textbf{Exactness is relative to the per-part search.} The routing rule is
|
||||
exact given the region bounds, but the dense leg inside a part is an
|
||||
approximate graph search, so the system's answer is exact only with respect to
|
||||
what that search returns. Conflating the two would make a recall failure look
|
||||
like a routing success.
|
||||
|
||||
\textbf{The largest sizes are not yet reachable.} A corpus of $10^9$ rows with
|
||||
dense vectors and a mixed sparse basis needs hardware the lane has not yet been
|
||||
given, the first pass stops two decades below it, and the collection it samples
|
||||
holds $3.5 \times 10^7$ rows. A fit over the sizes actually run is a fit over
|
||||
those sizes.
|
||||
@@ -0,0 +1,34 @@
|
||||
\section{Conclusion}
|
||||
\label{sec:conclusion}
|
||||
|
||||
Retrieval inside a merge-based column store is served today by structures that
|
||||
are built per part and consulted per part, so the cost of a similarity query
|
||||
follows the number of parts rather than the number of rows that could answer
|
||||
it. The design here adds two things to that layout: a weighted inverted file
|
||||
over a basis whose elements may be words, entities, code symbols, learned
|
||||
concepts, relations or motifs, with block maxima on the store's own granules;
|
||||
and a summary per region and per part --- centroid, covering radius, per-element
|
||||
maxima, counting statistics --- aggregated into a structure the planner reads
|
||||
to exclude parts before it opens them.
|
||||
|
||||
The components are mostly not new, and Table~\ref{tab:status} says which are
|
||||
which. The arrangement is the contribution: bounds from metric indexing and
|
||||
dynamic pruning, statistics from distributed resource selection, and a
|
||||
maintenance path through the merges a column store already performs.
|
||||
|
||||
What the design owes is a measurement, and the shape of the claim decides the
|
||||
measurement. Expected cost scaling with the relevant neighbourhood is a
|
||||
statement about a query distribution at a fixed recall target, so the protocol
|
||||
holds recall, fits an exponent with an interval to the parts, granules and
|
||||
bytes touched per correct answer, compares against the granules that hold the
|
||||
answer, and runs the queries the design is expected to fail on as their own
|
||||
stratum. Until that lane reports, this paper is a design and a protocol, and
|
||||
its results section holds placeholders.
|
||||
|
||||
\section*{Reproducing}
|
||||
|
||||
The lane that fills this paper's tables is in \texttt{hanzoai/benchmarks} and
|
||||
has not been published. Its first pass runs upstream ClickHouse~26.4.2.10 at
|
||||
$N = 10^5$ to $10^7$. The harness writes \texttt{results-data.tex} directly; the
|
||||
tables in \S\ref{sec:results} read it, and a key it does not define prints
|
||||
\ph{}.
|
||||
Reference in New Issue
Block a user