mirror of
https://github.com/hanzoai/papers.git
synced 2026-10-29 07:34:31 +00:00
The prior-work section describes the 2022 backend by category, and the comparison table and production section cover the systems Base is measured against. hanzo-stack-comparison and hanzo-datastore drop one reference each. hanzo-base.pdf had no source in the repo and is superseded by hanzo-base-runtime. Committed PDFs rebuilt with make. Co-authored-by: Hanzo Dev <[email protected]>
77 lines
3.5 KiB
TeX
77 lines
3.5 KiB
TeX
\section{Deployment and Multi-Tenancy}
|
|
|
|
Datastore is built to be hosted as a provisionable, multi-tenant offering on a
|
|
native PaaS. Its compatibility surface makes the two analytics products Hanzo
|
|
operates --- a privacy-preserving web-analytics service and a product-analytics
|
|
platform --- migrate by configuration alone: both already isolate tenants by a
|
|
column key (\texttt{organization\_id} and \texttt{team\_id} respectively) over a
|
|
shared database and schema, so pointing them at a Datastore instance is a change
|
|
of host, not of data model. The leaner resident footprint compounds directly in
|
|
this setting: at $\sim\!50$\,MB per instance instead of $\sim\!116$\,MB, an
|
|
operator more than doubles the number of tenant instances per node at fixed
|
|
memory, which is the dimension that governs the unit economics of a managed
|
|
service.
|
|
|
|
Because the engine derives from ClickHouse under the permissive Apache~2.0
|
|
license, Datastore is freely re-brandable, hostable, and resellable --- as are
|
|
the adjacent Hanzo data products on equally permissive upstreams (PostgreSQL,
|
|
pgvector, SQLite, and the Valkey-derived key-value store). This
|
|
licensing posture is what makes a commercial managed-database offering built on
|
|
these components viable without copyleft entanglement.
|
|
|
|
\section{Conclusion}
|
|
|
|
We have shown that the operationally dominant costs of a columnar analytics
|
|
server --- resident memory and binary size --- yield substantially to a
|
|
disciplined build pipeline. PGO, ThinLTO, and BOLT together produce a Datastore
|
|
server at less than half the resident footprint of the upstream official binary
|
|
with equal or better read throughput, and memory that is flat from $10$ to
|
|
$10{,}000$ connections. Along the way we identified that default-enabled XRay
|
|
instrumentation was both bloating the binary by $\sim\!45$\,MB and, through
|
|
unlowerable composed \texttt{R\_AARCH64\_PREL64} relocations, silently blocking
|
|
BOLT on AArch64 --- a fix that is one disabled default. Finally, replacing gRPC
|
|
with a Cap'n-Proto-derived transport that carries native columnar blocks as
|
|
opaque payloads exposes promise pipelining to clients, flattening dependent-chain
|
|
latency by up to $63\times$ without disturbing the engine's bounded memory
|
|
behavior.
|
|
|
|
The work points to two clear next steps: promoting ZAP from a transport to the
|
|
primary, ZAP-native query path, and extending the same posture to the
|
|
coordination layer. Both are constrained by the same principle that guided this
|
|
work --- preserve compatibility, pay only for what the workload uses, and let the
|
|
measurements, not the defaults, decide what ships.
|
|
|
|
\begin{thebibliography}{9}
|
|
|
|
\bibitem{clickhouse}
|
|
ClickHouse, Inc.,
|
|
\newblock \textit{ClickHouse: Open-Source Column-Oriented DBMS for Real-Time
|
|
Analytics}.
|
|
\newblock \texttt{https://clickhouse.com}, Apache License 2.0.
|
|
|
|
\bibitem{bolt}
|
|
M.~Panchenko, R.~Auler, B.~Nell, and G.~Ottoni,
|
|
\newblock ``BOLT: A Practical Binary Optimizer for Data Centers and Beyond,''
|
|
\newblock \textit{CGO}, 2019.
|
|
|
|
\bibitem{xray}
|
|
D.~M.~Berris, A.~Veitch, N.~Lewycky, et~al.,
|
|
\newblock ``XRay: A Function Call Tracing System,''
|
|
\newblock Google Technical Report, 2016.
|
|
|
|
\bibitem{capnproto}
|
|
K.~Varda,
|
|
\newblock \textit{Cap'n Proto: Cerealization Protocol}.
|
|
\newblock \texttt{https://capnproto.org}.
|
|
|
|
\bibitem{pgo}
|
|
\newblock ``Profile Guided Optimization,'' LLVM/Clang Documentation.
|
|
\newblock \texttt{https://clang.llvm.org/docs/UsersManual.html\#profile-guided-optimization}.
|
|
|
|
\bibitem{thinlto}
|
|
T.~Johnson, M.~Amini, and X.~D.~Li,
|
|
\newblock ``ThinLTO: Scalable and Incremental LTO,''
|
|
\newblock \textit{CGO}, 2017.
|
|
|
|
\end{thebibliography}
|