Files
papers/hanzo-unified-tenant-cloud.tex
zeekayandHanzo Dev ecadaf35c2 papers: publish only what an outside reader can check
An audit of all 68 published papers found that 51 were machine-generated in a
three-day window and carry no measurement of a Hanzo system: across the
29-paper product block, 711 numeric claim-sites yielded zero measured values.
They are backdated on the live site to 2024-2025. The defects are not subtle
and a reader disproves them with a browser and a calculator — a paper reports
9.2 million blocks per second, one block per 108 nanoseconds; another reports
8,412,893 jobs processed on a chain that answers net_peerCount with zero, and
a second paper reports the identical figure for a different system; a third
prints a user total its own arithmetic contradicts by 170x. Two papers on
retrieval and inference cite benchmark placements that appear on no
leaderboard.

The site keeps twelve papers. Each is from the human lane, each names its
harness and its box, and several report losses — the engine paper's Vulkan,
ROCm and Metal rows are all below parity, which is precisely why it can be
trusted about the one win it claims. Everything removed stays in git; nothing
is lost, and a paper returns when it can support what it says.

Three of the twelve needed a correction first. hanzo-cloud-economics headlined
3.5M lines of dead code deleted while stating 3.07M -> 1.06M one clause later;
the difference is 2.0M and the paper now says so. hanzo-network-whitepaper
claimed 29.5x storage savings as an observation — it is BitDelta's published
result and is now attributed, and its date no longer precedes the work.
hanzo-unified-tenant-cloud cited HIP-0032, which was deleted with the rest of
the contradicted standards; the concern lives in HIP-0405.

Co-authored-by: Hanzo Dev <[email protected]>
2026-07-28 15:51:59 -07:00

825 lines
56 KiB
TeX

\documentclass[11pt]{article}
\usepackage[margin=1in]{geometry}
\usepackage{booktabs}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{xcolor}
\usepackage{hyperref}
\hypersetup{colorlinks=true, linkcolor=blue, urlcolor=blue, citecolor=blue}
\usepackage{enumitem}
\usepackage{array}
\newcommand{\meas}{\textcolor{blue!60!black}{\small[measured]}}
\newcommand{\modeled}{\textcolor{orange!70!black}{\small[modeled]}}
\newcommand{\cited}{\textcolor{purple!70!black}{\small[cited]}}
\title{\textbf{The Unified Sovereign-Tenant Cloud:\\
Linear Shared-Nothing Scaling via Per-Tenant SQLite\\
and In-Process Composition}}
\author{Hanzo AI Research \\ \textit{Hanzo Industries Inc} \\ \texttt{[email protected]}}
\date{Measurement pass: 2026-07-08 (host \texttt{evo}, AMD Ryzen AI Max+ 395, Go 1.26.4)}
\begin{document}
\maketitle
\begin{abstract}
Hanzo Cloud runs the entire control- and data-plane of a multi-tenant SaaS as \emph{one}
ZAP-native Go binary (56 in-process subsystems, one runtime) over a \emph{shared-nothing}
storage substrate: every tenant --- org, app, project, or user --- is its own embedded SQLite
file, replicated to durable object storage as per-tenant post-quantum-encrypted LTX segments,
served by a single pinned writer per file, with only the \emph{active} tenants held open in a
bounded per-pod LRU. This paper asks one question and answers it with measurement, not
marketing: \emph{what does a resident tenant actually cost, and how does the design scale?}
We measure the marginal resident-set size (RSS) of opening $N \in \{1,10,100,1000\}$ real
base-schema tenant databases through the exact production open path, on the production driver.
The answer: \textbf{an open, actively-served tenant costs $\approx$\,0.25\,MB of RAM}
(251\,KB measured incremental slope; cgo/SQLite, warm workload) --- roughly $8\times$ below the
2\,MB figure the design was previously budgeted against --- and an \emph{idle} tenant costs
essentially zero live RAM (it is a file on disk plus an encrypted object). The tuned
\texttt{cache\_size} cap is a \emph{ceiling}, not a footprint: for realistic tenants the working
set, not the cap, sets the cost. From this one measured constant we build a capacity model: on the standard
128\,GB unified-memory node, at 1\% peak concurrency, \textbf{one billion registered tenants} is
served by $\approx$\,20 nodes ($\approx$\,2.5\,TB cluster RAM); at 0.1\% concurrency, the 3-node
HA floor. Memory scales with \emph{concurrency}, not with registrations. Because a 128\,GB box is
\emph{unified}-memory, one such box also hosts the shared model weights for GPU inference in the
same address space: a single \$3{,}200 commodity box --- the very class of hardware these numbers
were measured on --- runs the entire cloud (per-tenant SQLite \emph{and} inference) and serves
tens of millions of registered users from one self-installing binary (\S\ref{sec:box}). We contrast this against the
default industry shape --- shared Postgres behind a microservice mesh --- and quantify the
penalties it pays that the unified design does not: a global write bottleneck, a hard
connection ceiling, and an $N$-runtime plus sidecar memory tax we measure at
\textbf{$11$--$32\times$} the unified binary's footprint. We are explicit about the limits:
writer-pinning failover, hot-tenant skew, cross-tenant queries, and the routing directory are
real costs, discussed honestly in \S\ref{sec:limits}. Every number labelled \meas{} is measured
on the named host; \modeled{} and \cited{} numbers are labelled as such; none are invented.
\end{abstract}
\section{Introduction}
\label{sec:intro}
The dominant shape of a multi-tenant SaaS backend is a fleet of stateless microservices in
front of a shared relational database (typically Postgres), with tenancy expressed as a
\texttt{tenant\_id} column and enforced in application code or row-level security. That shape
has two structural costs that grow with the business. First, \emph{writes serialize}: every
tenant's mutations funnel through one primary's write-ahead log, so the write path is a single
global resource no matter how many read replicas are added. Second, \emph{runtimes multiply}:
each microservice is a full process --- its own language runtime, heap, connection pool, and
(in a service mesh) a sidecar proxy --- so the fixed memory cost of the platform is paid $N$
times over, before a single byte of tenant data is stored.
Hanzo Cloud takes the opposite position on both axes, and this paper is the measured defense of
that position. On composition: the 56 subsystems (IAM, KMS, Base, Commerce, AI, o11y, VFS, MQ,
tasks, audit, \dots) are linked into \emph{one} binary and communicate in-process over a
ZAP-native transport, with the same symbols exposed as ZAP-RPC only where process isolation is
a compliance requirement (payments/vault, PCI CDE)~\cite{hip0106,hip0116,hip0125}. The unified
binary's construction is developed in the companion architecture paper~\cite{unifiedbin}; this
paper is its measured memory and scaling counterpart. On storage: there is
no shared database. Each tenant is a sovereign, independently encrypted SQLite
file~\cite{hip0302}; the write path for tenant $t$ is a single writer pinned to $t$, so writes
are \emph{embarrassingly sharded} and there is no global lock, no distributed transaction, and
no per-write consensus round. The claim is that this is simultaneously the more \emph{efficient}
design (less RAM per tenant, far less fixed overhead) and the more \emph{scalable} one (node
count grows linearly in active tenants, with no shared bottleneck to saturate).
Claims about efficiency and scale are cheap; the contribution here is to make them measurable.
We (i) establish the real per-active-tenant memory constant by direct measurement through the
production open path (\S\ref{sec:bench}); (ii) turn that constant into a capacity model from
100 to $10^9$ registered tenants (\S\ref{sec:capacity}); (iii) quantify the memory gap against
the shared-Postgres-plus-mesh alternative (\S\ref{sec:compare}); and (iv) state the scaling
claims as propositions grounded in the measurement, with an honest accounting of where the
design pays (\S\ref{sec:claims}--\S\ref{sec:limits}).
\section{Architecture}
\label{sec:arch}
\paragraph{One binary, 56 in-process subsystems.}
Per HIP-0106~\cite{hip0106}, every Hanzo Go service exposes a top-level \texttt{Mount(app,
deps)} symbol and is compiled into a single \texttt{cloud} binary rather than deployed as an
independent service. Subsystems consume each other through Go interfaces that resolve to
\emph{in-process} implementations by default (the in-process plugin/composition model of
HIP-0116~\cite{hip0116}), falling back to ZAP-RPC only for the small set of subsystems that must
stay in a separate process for compliance isolation. HIP-0125~\cite{hip0125}
eliminates gRPC from the transport entirely in favor of ZAP-native framing, so there is no
per-subsystem gRPC server, no protobuf reflection service, and no sidecar. The consequence for
this paper is that the platform's \emph{fixed} memory cost --- one Go runtime, one scheduler,
one set of shared caches and connection machinery --- is paid \emph{once} per node, not once per
subsystem.
\paragraph{The sovereign tenant: one encrypted SQLite file each.}
Tenancy is a composable hierarchy under one org: an org-wide database, a per-app database, a
per-app-per-project database, and a per-user database~\cite{hip0302}. Each lands on disk at
\texttt{\{DataDir\}/orgs/\{orgSlug\}/\{service\}.db} and is opened through exactly one code path
--- the canonical Hanzo SQLite driver with a fixed pragma battery (\S\ref{sec:bench}). There is
no shared schema and no cross-tenant table: tenant isolation is a \emph{filesystem} boundary,
not a \texttt{WHERE tenant\_id = ?} predicate, which removes an entire class of
tenant-bleed bugs by construction~\cite{hip0118}.
\paragraph{The pinned single writer.}
A single tenant is served by exactly one pod at a time via gateway-side sticky affinity
(consistent hash on \texttt{X-Org-Id}). Because SQLite is a single-writer store and each tenant
maps to one file served by one pod, the writer for tenant $t$ is globally unique without any
coordination protocol. Reads may be served by read-only replica pods that hydrate from the
replication stream and fail \emph{closed} on any mutating verb, so correctness never rides on
routing being perfect. This is the mechanism behind the linear-scaling claim: distinct tenants
share no write path, so adding tenants adds independent writers rather than contending for one.
\paragraph{Durability: encrypted replication, not a shared database.}
Per HIP-0302~\cite{hip0302}, each tenant's SQLite file is continuously replicated as LTX
segments through \texttt{encreplica}, which post-quantum-\texttt{age}-encrypts every segment
with a \emph{per-tenant} key before it touches durable storage (the HIP-0107 replication
substrate~\cite{hip0107}; local FS for dev, S3 or Hanzo VFS in production~\cite{hip0032}). The
storage and encryption architecture is developed in the companion post-quantum object-store
paper~\cite{pqstore}; this paper concerns its memory and scaling behavior. At-rest encryption
therefore lives on the replication path,
not in the resident page cache: the open handle is plaintext SQLite (fast), and the durable
copy is ciphertext keyed to the tenant. This distinction matters for the memory model --- the
crypto cost is transient (a buffer during flush), not a standing per-connection tax.
\paragraph{Active-set residency.}
Only \emph{active} tenants are held open. The per-pod store keeps open SQLite handles in a
bounded LRU (default cap 1000 handles/pod, tunable) with a 5-minute idle reaper; on eviction it
checkpoints the WAL, closes the handle, and flushes the encrypted bytes to the object store. A
cold tenant occupies no RAM --- it is an encrypted object plus, at most, a file in a local
cache. This is the architectural reason the resident memory of a node tracks \emph{concurrency}
rather than \emph{registrations}, which \S\ref{sec:bench} then quantifies.
\section{Benchmark: the real per-tenant memory constant}
\label{sec:bench}
\paragraph{Question.}
What is the marginal resident memory of one open, actively-served tenant database, measured
through the exact production open path, on the driver the production binary actually links?
\paragraph{Method.}
We open tenant databases with the identical call the production store uses,
\begin{center}
\texttt{dbx.Open("sqlite", sqlite.PragmaDSN(path, sqlite.DefaultPragmas))},
\end{center}
where \texttt{DefaultPragmas} is the canonical Hanzo battery: \texttt{busy\_timeout=10000},
\texttt{journal\_mode=WAL}, \texttt{synchronous=NORMAL}, \texttt{foreign\_keys=ON},
\texttt{temp\_store=MEMORY}, \texttt{cache\_size=-32000} (a 32\,MiB page-cache \emph{cap} per
connection). The seed is the checked-in base test database
(\texttt{base/tests/data/data.db}, 344{,}064\,B data + 49{,}152\,B auxiliary $\approx$ 0.38\,MB),
a realistic small tenant carrying the base system collections plus a little data. For
$N \in \{1,10,100,1000\}$ we copy the seed to a fresh path, open it, run a workload, and keep
the handle resident; at each checkpoint we force a Go GC and \texttt{FreeOSMemory} and read
\texttt{VmRSS} from \texttt{/proc/self/status}. We report the \emph{incremental slope}
$\Delta\text{RSS}/\Delta N$ between checkpoints, which cancels the fixed process baseline and
yields the true per-tenant constant. RSS (not Go heap statistics) is the honest measure because
the cgo SQLite page cache is allocated in the C heap, invisible to Go's allocator. Two workloads:
\emph{cold} (open + a single \texttt{SELECT 1}) and \emph{warm} (touch every table's pages via
\texttt{count(*)} then perform a small write, materializing the WAL, the \texttt{-shm} index,
and the pinned writer connection). The production driver is cgo/\texttt{mattn} (the \texttt{cloud}
binary links it as the active \texttt{"sqlite"} driver); we also measure the pure-Go
\texttt{modernc} backend used on \texttt{!cgo} builds. Host: \texttt{evo}, AMD Ryzen AI Max+ 395
(32 cores), 62.4\,GB, Linux 6.17.0-35, Go 1.26.4.
\paragraph{Result: $\approx$0.25\,MB per active tenant.}
Table~\ref{tab:perdb} gives the measured constants. On the production driver, a warm resident
tenant costs a measured \textbf{251\,KB} ($\approx$0.245\,MB) at the asymptotic
(100\,$\rightarrow$\,1000) slope; a cold-but-open tenant costs 235\,KB. The pure-Go backend is
$\approx$1.5$\times$ heavier at 392\,KB. The single most important finding is in the
\texttt{cache\_size} column: sweeping the cap across $\{-256, -1024, -2000, -32000\}$
(256\,KiB to 32\,MiB) moves the small-tenant constant by \emph{less than 0.2\%}
(251.3--251.7\,KB). The 32\,MiB cap is never approached, because the tenant's working set
($\approx$0.25\,MB) --- not the cap --- sets the footprint. The four cache settings are in effect
four independent repeats at equal working set, and their agreement to within 0.4\,KB is our
repeatability evidence.
\begin{table}[h]
\centering
\caption{Measured marginal RSS per open, resident tenant (base-schema seed, 0.38\,MB on disk).
Incremental slope is $\Delta\text{RSS}/\Delta N$ over $100\rightarrow1000$ handles; baseline
(runtime + driver, zero tenants) is 7.0--7.2\,MB (cgo), 6.3--6.5\,MB (modernc). \meas{}}
\label{tab:perdb}
\begin{tabular}{llrr}
\toprule
Driver & Workload & \texttt{cache\_size} & Per-tenant RSS \\
\midrule
mattn / SQLite (cgo, \emph{production}) & warm & $-32000$ (32\,MiB) & \textbf{251.3\,KB} (0.245\,MB) \\
mattn / SQLite (cgo, \emph{production}) & warm & $-2000$ (2\,MiB) & 251.4\,KB \\
mattn / SQLite (cgo, \emph{production}) & warm & $-1024$ (1\,MiB) & 251.3\,KB \\
mattn / SQLite (cgo, \emph{production}) & warm & $-256$ (256\,KiB) & 251.4\,KB \\
mattn / SQLite (cgo, \emph{production}) & cold & $-32000$ & 235.5\,KB (0.230\,MB) \\
modernc (pure-Go, \texttt{!cgo}) & warm & $-32000$ & 392.0\,KB (0.383\,MB) \\
\bottomrule
\end{tabular}
\end{table}
\paragraph{The density knob is real --- for \emph{hot, large} tenants only.}
That \texttt{cache\_size} is inert for small tenants is not the whole story. We re-ran the sweep
against a synthetic 25\,MB tenant under a full-scan warm workload (Table~\ref{tab:knob}). Here the
cap binds: at $-32000$ a fully-scanned hot tenant fills its page cache to the working set and
costs 25.5\,MB; at $-1024$ it is bounded to 1.18\,MB; at $-256$ to 0.41\,MB. So the cap is a
per-tenant memory \emph{ceiling} operators can dial to trade page-cache hit rate for density.
The realistic operating point is Table~\ref{tab:perdb} (working-set-bound, $\approx$0.25\,MB);
Table~\ref{tab:knob} is the worst case (analytic tenant scanning its whole database every
request) and shows it remains bounded and tunable rather than unbounded.
\begin{table}[h]
\centering
\caption{Density knob: marginal RSS for a \emph{hot} 25\,MB tenant under a full-table-scan
workload, as a function of the \texttt{cache\_size} cap. \meas{}}
\label{tab:knob}
\begin{tabular}{lrr}
\toprule
\texttt{cache\_size} cap & Per-tenant RSS & Effect \\
\midrule
$-32000$ (32\,MiB) & 25.5\,MB & cache fills to working set (cap not reached) \\
$-1024$ (1\,MiB) & 1.18\,MB & bounded to $\approx$1\,MiB regardless of DB size \\
$-256$ (256\,KiB) & 0.41\,MB & bounded to $\approx$256\,KiB + fixed overhead \\
\bottomrule
\end{tabular}
\end{table}
\paragraph{Idle tenants cost $\approx$0.}
We opened and exercised 1000 tenants, then closed every handle. On the pure-Go backend --- whose
memory frees to the Go heap, which \texttt{FreeOSMemory} returns to the OS --- RSS fell from
392\,MB back to \textbf{11.2\,MB}, i.e.\ to within 4.9\,MB of baseline over 1000 closed tenants
($\approx$5\,KB/tenant of retained runtime bookkeeping). This is the direct measurement behind
``an idle tenant is free'': closing a handle releases essentially all of its live memory. On the
cgo backend RSS settled at 221\,MB after close; that residual is \emph{not} live memory but
glibc's retention of freed C arenas (the allocator does not \texttt{munmap} them back to the
kernel). We confirmed it is reused, not leaked: closing 1000 tenants and opening 1000 fresh ones
grew RSS by only $\approx$110\,MB on top of the 222\,MB retained (vs.\ 247\,MB for a cold start),
so the freed arenas absorb subsequent opens. The operational takeaway: a pod's steady-state RSS
tracks its \emph{high-water active set}, and idle registrations contribute no live pages.
\paragraph{On-disk and base-node constants.}
A realistic tenant occupies $\approx$0.38\,MB on disk (measured seed), which at rest lives as
per-tenant-encrypted LTX in object storage~\cite{hip0032,hip0302}, not on node-local disk. The
fixed per-node cost --- the \texttt{cloud} binary with all 56 subsystems and one Go runtime --- is
a working-set RSS of \textbf{154--225\,MB} from prior production measurement~\cite{edgepaper};
we use 190\,MB as the point estimate in the model below and note it is \cited{} (the cluster was
not reachable from the measurement host on this pass to re-sample it live). For comparison, a
\emph{bare} Go HTTP service (net/http, one handler, no framework) measured \meas{} 7.2\,MB on the
same host --- the floor a per-service split cannot go below, used in \S\ref{sec:compare}.
\section{The capacity model}
\label{sec:capacity}
The measured constant makes the fleet size a closed-form function of registrations and
concurrency. Let $R$ be registered tenants, $c$ the peak concurrency fraction, so active
tenants $A = cR$. Let $m$ be the measured per-active-tenant RAM (MB), $b$ the per-node base RSS
(GB), and $U$ the usable RAM per node (GB) after reserving for the OS, kubelet, and burst
headroom. Then
\begin{equation}
\text{nodes} \;=\; \max\!\Big(3,\; \Big\lceil \frac{A \cdot m}{(U - b)\cdot 1024} \Big\rceil\Big),
\qquad
\text{cluster RAM} \;=\; \text{nodes}\cdot b \;+\; \frac{A\cdot m}{1024}\ \text{GB}.
\label{eq:model}
\end{equation}
The standard Hanzo node is a \textbf{128\,GB} unified-memory box (\S\ref{sec:box}); we use
$U = 125$\,GB usable, $b = 0.19$\,GB, and the measured $m = 0.25$\,MB, giving a per-node density
of $\approx$511{,}000 active tenants; we also report the conservative $m = 1.0$\,MB envelope
(larger/hotter tenants, multiple resident connections, extra headroom), which yields
$\approx$128{,}000 active/node. A 128\,GB node holds $\approx$4$\times$ the active tenants of a
32\,GB node, so the fleet is $\approx$1/4 the size at the same load: $10^9$ registered tenants
at 1\% concurrency is $\approx$20 128\,GB nodes versus $\approx$103 32\,GB nodes (both from the
\emph{measured} $m$, not a 2\,MB assumption). Disk is $R \times 0.38$\,MB total, object-tiered.
Table~\ref{tab:cap} gives the model at 1\% peak concurrency; Table~\ref{tab:conc} shows
concurrency sensitivity at the extremes.
\begin{table}[h]
\centering
\caption{Capacity at \textbf{1\% peak concurrency}, \textbf{128\,GB} nodes ($U=125$\,GB), from
Eq.~\ref{eq:model} with the \meas{} $m=0.25$\,MB constant (and the \modeled{} $m=1.0$\,MB
conservative envelope). ``Cluster RAM'' is consumed RAM per Eq.~\ref{eq:model}, not provisioned.
Node counts are floored at 3 for HA.}
\label{tab:cap}
\resizebox{\textwidth}{!}{%
\begin{tabular}{r r r r r r}
\toprule
Registered $R$ & Active @1\% & Nodes ($m{=}0.25$) & Cluster RAM & Nodes ($m{=}1.0$) & Disk (total, tiered) \\
\midrule
100 & 1 & 3 & 0.6\,GB & 3 & 38\,MB \\
1{,}000 & 10 & 3 & 0.6\,GB & 3 & 380\,MB \\
10{,}000 & 100 & 3 & 0.6\,GB & 3 & 4\,GB \\
100{,}000 & 1{,}000 & 3 & 0.8\,GB & 3 & 38\,GB \\
1{,}000{,}000 & 10{,}000 & 3 & 3.0\,GB & 3 & 380\,GB \\
10{,}000{,}000 & 100{,}000 & 3 & 25.0\,GB & 3 & 4\,TB \\
100{,}000{,}000& 1{,}000{,}000 & 3 & 244.7\,GB & 8 & 38\,TB \\
1{,}000{,}000{,}000 & 10{,}000{,}000 & \textbf{20} & \textbf{2.45\,TB} & 79 & 380\,TB \\
\bottomrule
\end{tabular}}
\end{table}
\begin{table}[h]
\centering
\caption{Concurrency sensitivity for $R = 10^9$ registered tenants ($m=0.25$\,MB, 128\,GB nodes).
Memory scales with \emph{concurrency}, not registrations: a 50$\times$ swing in peak concurrency
is a 50$\times$ swing in fleet size, while $R$ is held fixed. \meas{}-grounded.}
\label{tab:conc}
\begin{tabular}{r r r r}
\toprule
Peak concurrency & Active tenants & Nodes (128\,GB) & Cluster RAM \\
\midrule
0.1\% & 1{,}000{,}000 & 3 & 245\,GB \\
1.0\% & 10{,}000{,}000 & 20 & 2.45\,TB \\
5.0\% & 50{,}000{,}000 & 98 & 12.2\,TB \\
\bottomrule
\end{tabular}
\end{table}
The shape of Table~\ref{tab:cap} is the thesis in numbers. Up to \emph{one hundred million}
registered tenants at 1\% concurrency, the fleet is the 3-node HA floor --- the platform is
base-RSS-bound, not tenant-bound, and the tenants are ``free'' relative to the fixed cost of
running at all. Only past $\sim$$10^8$ registrations does tenant memory dominate, and there it
grows \emph{linearly}: $10^9$ tenants is 20 128\,GB nodes, $10^{10}$ would be $\sim$200. There is
no knee, no bottleneck tier, and no rewrite between the 3-node and the 20-node regime --- the
same binary and the same per-tenant file, more of them.
\section{The two scaling axes: tenant state vs.\ GPU inference}
\label{sec:axes}
The capacity model of \S\ref{sec:capacity} sizes \emph{one} resource: per-tenant SQLite state,
which is CPU/RAM/IO. A GPU-enabled build of the same binary adds a \emph{second}, orthogonal
axis --- inference --- and it scales by a completely different law. Conflating the two is the
standard modeling error; separating them is the point.
The inference engine, hanzo-engine (Rust), is FFI-embeddable into the cloud binary through a
stable C ABI (\texttt{hanzo\_engine\_ffi.h}, present in the engine tree), so a GPU node runs
inference and embeddings \emph{in the same process} as the app subsystems and the tenant stores
--- no separate model service, no network hop to a model server (the co-location is the cloud
``bring-your-gpu'' path). The two axes then scale as follows.
\begin{itemize}[leftmargin=1.4em]
\item \textbf{Axis A --- tenant state (CPU/RAM).} Resident memory is $A\cdot m$ with $A=cR$: it
grows with the number of \emph{active} tenants, and each tenant's state is private
(shared-nothing). This is \S\ref{sec:capacity}, measured at $m\approx0.25$\,MB.
\item \textbf{Axis B --- inference (GPU/VRAM).} GPU memory holds \emph{model weights}, which are
\emph{shared across all tenants}: one resident copy of a model serves one tenant or one million
at the same VRAM cost. Inference memory is therefore $O(\text{distinct models hosted})$, \emph{not}
$O(\text{tenants})$. Adding tenants adds no weight memory --- only queue depth, absorbed by request
\emph{batching}. Axis B scales by (i) model-hosting: VRAM $=$ weights $+$ KV-cache
(\cited{} $\approx$8.11\,GiB resident for Qwen3-8B-Q8\_0), and (ii) throughput: tokens/sec, bounded
by memory bandwidth (\cited{} $\approx$231\,GB/s and single-stream decode $\approx$25\,t/s on the
measured Strix Halo box; an NVIDIA GB10 Grace-Blackwell offers 121\,GB unified~\cite{xbench}).
\end{itemize}
Because the axes are independent, a deployment sizes them independently: tenant-state nodes by
Eq.~\ref{eq:model}, inference nodes by peak token demand $\div$ per-node batched throughput. A
single node carries \emph{both} roles when it has RAM for tenants and VRAM (or unified memory)
for weights --- exactly the box of \S\ref{sec:box}. The weight-sharing property is what makes
inference cheap to co-locate: the model is a fixed cost amortized across the \emph{entire} tenant
population, not a per-tenant tax. (Axis-A numbers are \meas{}; the Axis-B weight/throughput
figures are \cited{} from the companion inference papers~\cite{xbench,edgepaper}.)
\section{The unified-memory box as the atomic self-hostable unit}
\label{sec:box}
Put the two axes in one box with one physical memory pool and the architecture collapses to
something new: a complete, self-hostable cloud in a single commodity unit.
The enabling hardware is the \emph{unified-memory} box --- a Grace-Blackwell GB10-class system
(e.g.\ NVIDIA DGX Spark, \cited{} 128\,GB unified LPDDR5X) or an AMD Ryzen AI Max+ 395 ``Strix
Halo'' box (the class this paper's per-tenant constant was \emph{measured} on) at 128\,GB unified
for $\approx$\$3{,}200. \emph{Unified} means CPU and GPU address the same physical DRAM: the
per-tenant SQLite population (Axis A) and the model weights (Axis B) coexist in one 128\,GB space
with \emph{no} CPU$\leftrightarrow$GPU copy --- the tenant data the app serves and the model the
app calls live in the same address space. One box therefore runs the entire stack: the one
binary with all 56 subsystems, per-tenant storage, replication, \emph{and} GPU inference.
Table~\ref{tab:box} sizes it: reserve model weights $+$ engine $+$ OS, and the remainder $\div\,m$
is the tenant capacity. With an 8B-Q8 model (8\,GiB) and $\approx$6\,GiB for engine/OS,
$\approx$114\,GiB remains for tenant state $=$ a \meas{}-grounded 467{,}000 active tenants; at 1\%
concurrency that is \textbf{$\approx$47 million registered users on one \$3{,}200 box}, at 0.1\%
concurrency $\approx$467 million. Even at the conservative $m=1.0$\,MB it is $\approx$11.7 million
registered at 1\%.
\begin{table}[h]
\centering
\caption{``Cloud in a box'': tenant capacity of one 128\,GB unified-memory box after reserving
model weights $+$ engine $+$ OS. Active-tenant counts use the \meas{} $m$; registered-user counts
apply the stated concurrency; weight reservations are \cited{}/\modeled{}. Capex is the \$3{,}200
box price $\div$ registered users at 1\% concurrency.}
\label{tab:box}
\resizebox{\textwidth}{!}{%
\begin{tabular}{l r r r r r}
\toprule
Resident model & Tenant RAM & Active ($m{=}0.25$) & Reg.\ @1\% & Reg.\ @0.1\% & Capex/user @1\% \\
\midrule
8B-Q8 ($\approx$8\,GiB) & 114\,GiB & 467{,}000 & \textbf{46.7\,M} & 467\,M & \$0.000069 \\
$\approx$20B-Q8 ($\approx$20\,GiB) & 102\,GiB & 418{,}000 & 41.8\,M & 418\,M & \$0.000077 \\
70B-Q4 ($\approx$40\,GiB) & 82\,GiB & 336{,}000 & 33.6\,M & 336\,M & \$0.000095 \\
\bottomrule
\end{tabular}}
\end{table}
The economics follow directly. At \$3{,}200 per box and $\approx$47\,M registered users at 1\%
concurrency, the one-time hardware capex is \textbf{$\approx$\$0.00007 per registered user}
(\$68 per million). Scale is additive: each box is self-contained (shared-nothing tenants, its own
model copy), so $N$ boxes are $N\times$ the capacity with no shared bottleneck. A sovereign,
full-stack AI cloud --- app, storage, \emph{and} inference --- is a rack of commodity boxes, not a
hyperscaler contract. (The tenant capacity is \meas{}-$m$-grounded; the weight reservation and
inference throughput are \cited{}/\modeled{}; a real box's capacity depends on the p99 tenant size
and the model chosen.)
\section{Comparative analysis: shared Postgres + microservice mesh}
\label{sec:compare}
We compare against the default industry shape on the three axes where the unified design's
structure diverges. The comparison is deliberately conservative: where we estimate the
alternative we anchor to a measured floor and label the estimate \modeled{}.
\paragraph{(1) The write path: sharded vs.\ global.}
In shared Postgres, every tenant's writes commit through one primary's WAL. Read replicas scale
reads but not writes; the write ceiling is one node's fsync throughput, and it is \emph{shared}
across all tenants, so a write-heavy tenant degrades every other tenant's write latency. Scaling
writes past one primary requires sharding (Citus, Vitess-style) or multi-primary consensus
(Raft/Paxos per write) --- either a re-architecture or a per-commit coordination round. In the
unified design the write path for tenant $t$ is a private SQLite writer; $N$ tenants are $N$
independent writers with \emph{no shared lock and no consensus round}. Aggregate write
throughput is the sum of per-tenant throughputs and grows linearly with nodes. The cost paid for
this is that a single tenant's writes are bounded by one file's single-writer throughput
(\S\ref{sec:limits}); the benefit is that there is no global write resource to saturate.
\paragraph{(2) The connection ceiling.}
A Postgres primary allocates a backend process per connection ($\sim$5--10\,MB each), so
practical \texttt{max\_connections} is in the hundreds; thousands of tenant-affine connections
require a pooler (PgBouncer) and still contend for the one primary. Per-tenant SQLite has no
connection ceiling: a ``connection'' is an in-process handle into a memory-mapped file, and the
active-set LRU (\S\ref{sec:arch}) bounds the resident count per pod directly. There is no network
round trip and no server-side backend process on the data path at all.
\paragraph{(3) The $N$-runtime + sidecar memory tax.}
This is the axis we can quantify most directly. The unified binary runs all 56 subsystems in one
Go runtime at a measured/cited base of 190\,MB; at 3 replicas for HA that is 570\,MB of
control-plane RAM before tenant data. A microservice mesh runs each subsystem as its own process
with its own runtime heap and, under a service mesh, a sidecar proxy. Even at the \emph{measured
floor} of a bare Go service (7.2\,MB \meas{}) plus a single Istio/Envoy sidecar
($\approx$50\,MB \cited~\cite{istio}) at 2 replicas, 56 subsystems cost 6.4\,GB; at a realistic
60\,MB/service (\modeled{}: framework + OTEL SDK + DB pool + business logic) plus 50\,MB sidecar,
12.3\,GB. Table~\ref{tab:mesh} gives the multiples: the mesh spends
\textbf{$11$--$32\times$} the unified binary's fixed memory, and adds one network hop (plus TLS
and serialization) to every inter-subsystem call that the unified design makes as an in-process
function call.
\begin{table}[h]
\centering
\caption{Fixed control-plane memory: unified binary vs.\ 56-subsystem microservice mesh, before
any tenant data. Unified base 190\,MB is \meas{}/\cited{}; bare-Go 7.2\,MB is \meas{}; per-service
framework RSS and sidecar RSS are \modeled{}/\cited{}. Multiple is mesh\,$\div$\,unified.}
\label{tab:mesh}
\resizebox{\textwidth}{!}{%
\begin{tabular}{l r r r}
\toprule
Deployment & Per-service & Fixed RAM & $\times$ unified (3-rep, 570\,MB) \\
\midrule
Unified binary, 3 replicas & -- & 570\,MB & 1.0$\times$ \\
Mesh, floor (7.2\,MB svc + 50\,MB sidecar, 2 rep) & 57.2\,MB & 6.4\,GB & 11.2$\times$ \\
Mesh, realistic (60\,MB svc + 50\,MB sidecar, 2 rep) & 110\,MB & 12.3\,GB & 21.6$\times$ \\
Mesh, heavy (120\,MB svc + 70\,MB sidecar, 2 rep) & 190\,MB & 21.3\,GB & 37.3$\times$ \\
\bottomrule
\end{tabular}}
\end{table}
Composed with the capacity model, the gap compounds: the mesh's 6--21\,GB fixed tax is paid on
\emph{every node}, so at the 3-node HA floor --- where the unified platform serves up to
$10^8$ registered tenants at 1\% concurrency in $<$1\,GB of control-plane RAM
(Table~\ref{tab:cap}) --- the mesh alternative already needs 19--64\,GB just to exist.
\paragraph{(4) The call path itself: an in-process function call, measured.}
Axes (1)--(3) count what the mesh \emph{spends}; this one measures what the unified design
\emph{saves} on the call. Every inter-subsystem read the mesh routes over the network, the
unified binary makes as a direct Go call into the co-resident subsystem's own store --- no
socket, no TLS handshake, no (de)serialization. We measure the customer billing surface's usage
read, which resolves the caller's spend \emph{directly} from the co-resident per-tenant finance
ledger (\texttt{finance.ListUsage} --- one SQLite scan of the org's own file):
\textbf{1.25\,ms for a 100-row read, 10.5\,ms at 1000 rows} (\meas{}; GB10 node, cgo/SQLite,
warm), with \emph{zero} network round trips. The mesh alternative pays a billing$\to$ledger hop
plus (re)serialization on every such read, on top of the fixed-memory tax of
Table~\ref{tab:mesh}.
The discipline this exacts --- stated plainly because we paid for it in an outage --- is that an
in-process call must be a direct function \emph{reference}, never in-process HTTP that re-routes
by request path. When the same billing read was instead expressed as an in-process HTTP request
to \texttt{/v1/billing/usage} against the shared app, and the ledger subsystem's own route for
that path was not compiled into this binary, the request matched the only registration of that
path --- the \emph{caller}'s own handler --- and re-entered it, answering itself.
Composition-by-path is a footgun; composition-by-reference (\texttt{finance.Current().ListUsage})
is the invariant. Symmetrically, the balance gate that fronts these reads must gate \emph{writes
and metered compute}, never plain reads, or a zero-balance tenant cannot even view its own state.
To keep both invariants honest, a per-subsystem functional smoke (\texttt{cmd/smoke},
\S\ref{sec:repro}) issues 30 side-effect-free reads across the binary's mounted subsystems and
asserts a working code on each --- never a 5xx, never a read wrongly gated 402 ---
failing the release before such a regression ships.
\section{Claims, grounded in the measurement}
\label{sec:claims}
We state the design's guarantees as propositions, each tied to a measured or structural fact
rather than an assertion.
\paragraph{Proposition 1 (memory $\propto$ concurrency, not registrations).}
Resident memory is $\text{nodes}\cdot b + A\cdot m$ with $A = cR$ (Eq.~\ref{eq:model}). Because
an idle tenant contributes $\approx$0 live RAM (measured: 1000 closed handles return to within
4.9\,MB of baseline, \S\ref{sec:bench}) and only the active-set LRU is resident, RAM is a
function of $A$, and $R$ enters only through $A = cR$. Doubling registrations at fixed
concurrency does not change resident memory; it changes only object-storage bytes.
Table~\ref{tab:conc} is the empirical shape.
\paragraph{Proposition 2 (linear node scaling from shared-nothing writers).}
Distinct tenants share no write path: each is a private single-writer SQLite file pinned to one
pod (\S\ref{sec:arch}). There is no global lock, no distributed transaction, and no per-write
consensus, so adding a node adds independent write and read capacity with no coordination
overhead that grows with the fleet. Fleet size is therefore $\Theta(A)$ --- on 128\,GB nodes,
20 at $10^7$ active, $\sim$196 at $10^8$ --- with a constant factor set by the measured $m$. This
is linear, not merely sublinear-with-a-bottleneck.
\paragraph{Proposition 3 (the one-binary memory advantage is multiplicative).}
Composing $S$ subsystems into one runtime pays the fixed runtime + sidecar cost once rather than
$S$ times. Measured against a 56-subsystem mesh, the fixed-memory ratio is $11$--$32\times$
(Table~\ref{tab:mesh}), and it is paid per node, so it multiplies the per-node base $b$ in
Eq.~\ref{eq:model}. Lowering $b$ is what keeps small deployments in the sub-gigabyte regime in
Table~\ref{tab:cap}.
\section{hanzo.network: the sovereign decentralized confidential compute fabric}
\label{sec:fabric}
\sloppy
Everything above describes \emph{one} operator's fleet. The culminating step makes it a public,
decentralized, confidential cloud: a fabric of independently-owned boxes coordinated leaderlessly
by a BFT chain, where anyone can contribute compute and confidential workloads run on hardware the
operator cannot inspect. The primitives below are real and cited to their source files; the
assembly into a single fabric is stated honestly as roadmap where it is roadmap
(\S\ref{sec:builtvroadmap}).
\paragraph{The control plane is a consensus chain.} hanzo.network is a \texttt{luxfi/consensus}
BFT chain~\cite{luxconsensus} --- the engine is real (\texttt{engine/bft}, \texttt{engine/chain},
\texttt{engine/common}) with the Quasar protocol (\texttt{protocol/quasar})~\cite{quasar} --- whose
VM \emph{state} \emph{is} the compute-fabric control plane: node membership and attestation, the
tenant$\to$writer-node pinning map, replication assignments, and workload placement. The chain
\emph{is} the scheduler --- leaderless BFT, no central coordinator, no single point of control or
failure.
\paragraph{The data plane is the per-tenant SQLite substrate --- unchanged.} Each node runs
\texttt{hanzod} (the node daemon; \texttt{hanzo/node}) which spawns the full cloud binary: a
consensus participant \emph{plus} the 56 app subsystems \emph{plus} GPU inference
(\S\ref{sec:axes}). Tenant writes stay exactly as measured --- a private, writer-pinned SQLite
file, WAL-replicated to consensus-agreed replicas. The chain coordinates \emph{placement}, not
individual writes.
\paragraph{Leaderless HA and durability --- the sweet spot.} The design sits deliberately between
two failure modes. \emph{Global consensus per write} (every mutation is a BFT round) gives
durability but does not scale --- the write path becomes one global bottleneck, the very thing
\S\ref{sec:compare} faults Postgres for, made worse. \emph{Replication without consensus} scales
writes but has no agreed failover --- a split brain can double-open a tenant or lose its tail.
hanzo.network takes neither: per-tenant single-writer for the \emph{data} path (writes shard
perfectly), and BFT consensus only for \emph{placement} --- who holds the writer pin, where the
replicas live, and, on writer-node death, which replica is promoted. The writer pin
(\texttt{cloud/writerpin}) already guarantees at-most-one writer; the writer/reader roles
(\texttt{cloud/role}, \texttt{middleware\_reader.go}, fail-closed reads --- shipped in the
reader-HA work) already give safe promotion. The chain makes that promotion decision
\emph{deterministic and Byzantine-fault-tolerant across untrusted operators}. Writes never touch
consensus; only the rare, slow-changing placement map does. That is why the fabric scales linearly
\emph{and} survives node loss.
\paragraph{Trust tiers make it trustless-public.} A public fabric of unknown operators cannot be
trusted with plaintext tenant data unless the hardware enforces it, so hanzo.network schedules by
\emph{trust tier}, recorded on-chain. \emph{Confidential tier}: nodes with a hardware TEE ---
NVIDIA H100/H200 Confidential Computing, Intel TDX, or AMD SEV-SNP~\cite{tee} --- run tenant
workloads inside an enclave the node \emph{operator} cannot read; the attestation is verified and
recorded on-chain, and confidential workloads schedule \emph{only} onto attested nodes.
\emph{Public tier}: any commodity box (a phone, a laptop, the \$3{,}200 unified box) takes
public/research workloads where the operator is trusted enough for public data. Anyone can run
\texttt{hanzod} and contribute, and the chain routes each workload to hardware whose trust tier
matches its sensitivity --- the difference between a genuinely \emph{trustless} decentralized cloud
and ``decentralized, but trust the operators.''
\paragraph{Post-quantum from the identity layer up.} A sovereign cloud is a decades-long artifact,
so its security horizon must not be capped by Shor's algorithm. hanzo.network is post-quantum
\emph{end-to-end}, not classical-with-a-PQ-bolt-on: consensus runs a PQ security profile
(\texttt{luxfi/consensus} \texttt{config/pq\_mode.go}, \texttt{security\_profile.go}) and PQ auth
schemes (\texttt{protocol/auth}: ML-DSA-44/65 signatures, ML-KEM key exchange, legacy ECDSA
\emph{forbidden} under strict PQ)~\cite{hip0005}; Quasar is a PQ BFT (BLS $+$ Ringtail threshold
$+$ ML-DSA), and the primitives ship in \texttt{lux/crypto/\{mldsa,mlkem\}}. Node identity is an
ML-DSA key; capability identity on the app side is PQ as well (\texttt{iam/capauth/\{keys,verify\}.go}).
This is a durability argument as much as a security one: node identity, consensus, and key
exchange all outlive the quantum transition.
\paragraph{Frictionless, auto-provisioned wallet.} Every node and user has a PQ wallet, and if
none is configured one is provisioned automatically as part of joining --- there is no ``install a
browser extension, back up a seed phrase'' barrier. The wallet subsystem is real
(\texttt{cloud/clients/wallets}: \texttt{wallets.go}, \texttt{custody.go}, \texttt{anchor.go},
\texttt{store.go}, \texttt{safeclient.go}, over \texttt{clients/mpcseal}), MPC-custodied so there
is no single seed to lose. The wallet is simultaneously the node's PQ \emph{identity} and its
economic \emph{key}, so \texttt{hanzod up} yields identity, wallet, and fabric membership in one
step --- essential to the viral model below.
\paragraph{Economics: the AI-coin incentive.} The wallet stakes to join the fabric, earns for
contributed compute --- priced by trust tier, so a TEE-confidential node earns more than a public
one --- and pays for consumed workloads, all in the native AI coin (HIP-0001~\cite{hip0001}). This
is the incentive that makes a viral node network \emph{self-sustaining}: contributing compute
earns, so nodes join and stay. Against the \S\ref{sec:box} box economics --- a \$3{,}200 128\,GB
box whose measured per-tenant capex is $\approx$\$0.00007/user --- a node's contributed compute
earns against a trivial fixed cost, which is what closes the loop.
\subsection{Viral deployment: \texttt{hanzod} is the unit, \texttt{cloud} is what it runs}
\label{sec:viral}
The reason this can actually spread is the deployment model. Every prior ``self-hostable cloud''
gates on operator expertise --- stand up Kubernetes, then deploy --- which restricts the operator
population to specialists. hanzo.network inverts it: you run \emph{one} daemon. \texttt{hanzod}
(\texttt{hanzo/node}, a systemd-managed node service) is the viral unit; the cloud binary is the
state machine it runs --- the relationship is \texttt{hanzod : cloud :: bitcoind : the ledger}. The
daemon (i) \emph{joins hanzo.network} as a leaderless \texttt{luxfi/consensus} node; (ii)
\emph{manages its own runtime adaptively} --- on a laptop or small box it spawns the cloud binary
bare (no Kubernetes), and on a GPU/large node it bootstraps k3s \emph{internally} for workload
isolation and GPU scheduling and runs cloud $+$ workloads inside it, so Kubernetes is an internal
implementation detail of the node (the HIP-0117~\cite{hip0117} \texttt{cluster init} mechanic
wrapped by the daemon), invisible to the operator; (iii) \emph{contributes compute by trust tier}
(TEE$\to$confidential, bare$\to$public); and (iv) \emph{self-installs, self-updates, self-heals}.
The whole onboarding is \texttt{download hanzod $\to$ hanzod up $\to$ you are an open-AI-cloud
node}: one command, no Kubernetes expertise, PQ identity $+$ wallet $+$ membership included.
The virality argument is then structural: \textbf{(a) no expertise barrier} --- one download and
one command remove the k8s-operator gate that every prior decentralized-cloud project tripped on;
\textbf{(b) monotone network effect} --- each joining node strictly increases fabric capacity,
resilience, \emph{and} trust-tier coverage, so growth compounds; \textbf{(c) universal addressable
population} --- trust-tiered participation makes the eligible node set \emph{everyone} (a phone
serves public inference; a TEE box serves confidential tenants), not just datacenter operators;
\textbf{(d) trivial per-node cost} --- a \$3{,}200 128\,GB unified box is a consumer purchase whose
measured per-tenant economics make contributed compute effectively free at the margin, and the
AI-coin incentive pays it back. Against hyperscaler centralization (one owner, metered rent,
plaintext trust) and against prior decentralized-cloud efforts (still operator-expert, still
trust-the-operator), the combination --- one-command node, leaderless BFT control, per-tenant
shared-nothing data, hardware-enforced confidential tiers, PQ identity, an auto-wallet, and
commodity unified-memory economics --- is what turns a measured efficiency result into a growth
model.
\paragraph{What is built vs.\ what is the assembly.}
\label{sec:builtvroadmap}
\textbf{Built (cited to source):} the BFT $+$ Quasar consensus engine (\texttt{luxfi/consensus},
PQ mode $+$ ML-DSA/ML-KEM auth); the per-tenant writer pin (\texttt{cloud/writerpin}); the
writer/reader HA roles (\texttt{cloud/role} $+$ \texttt{middleware\_reader.go}); the node daemon
(\texttt{hanzod}); the unified cloud binary (HIP-0106) and in-process plugin/VM model (HIP-0116);
the wallet/custody subsystem (\texttt{cloud/clients/wallets} $+$ \texttt{mpcseal}); PQ capability
identity (\texttt{iam/capauth}); the PQ crypto primitives (\texttt{lux/crypto/\{mldsa,mlkem\}});
the GPU/engine FFI (\texttt{hanzo\_engine\_ffi.h}) and \texttt{lux/node/vms}; and the specs
HIP-0117 (BYO compute fleet), HIP-0005 (PQ security), HIP-0001 (AI coin).
\textbf{Roadmap (the assembly):} the fabric-VM that makes consensus state the live scheduler; the
on-chain tenant$\to$writer map with consensus-driven failover; on-chain TEE attestation and
confidential trust-tier scheduling; the \texttt{hanzod} auto-join handshake, adaptive-k3s
bootstrap, and self-heal orchestration; the auto-provision-on-join wallet flow; and the
compute-earning settlement with cross-node trust-tier pricing. This paper's contribution is the
\emph{measured foundation}; the fabric is the composition of shipped primitives, marked as design
where it is design.
\fussy
\section{Limitations}
\label{sec:limits}
The rigor is the persuasion, so the costs are stated as plainly as the benefits.
\begin{itemize}[leftmargin=1.4em]
\item \textbf{Writer-pinning failover.} One writer per tenant means a pod loss requires
re-pinning that tenant to a new pod and replaying the tail of the encrypted replication stream
before writes resume. HIP-0302's consistency model is at-most-once under rebalance: a short
single-writer overlap window can exist during an HPA move, and operators must drain before
scaling. There is a bounded write-unavailability window per affected tenant on failover; the
CAS (ETag If-Match) follow-up is required for generation-checked writes across a rebalance.
\item \textbf{Hot-tenant skew.} Capacity is modeled on an \emph{average} tenant. One tenant that
is simultaneously large and hot can, at the default 32\,MiB cap, cost up to 25.5\,MB resident
(Table~\ref{tab:knob}) and is bounded to one pod's single-writer throughput. The mitigations ---
a smaller per-tenant \texttt{cache\_size}, splitting a heavy org across the app/project/user
hierarchy, and per-tenant rate ceilings --- are real levers but they are operational work, not
free. The model's $m$ should be chosen for the p99 tenant, not the mean, in skewed populations.
\item \textbf{Cross-tenant queries.} A filesystem isolation boundary is exactly the wrong shape
for a query that must span tenants (fleet-wide analytics, admin search, billing rollups). These
require fan-out plus aggregation, or a separate columnar sink fed from the replication stream ---
neither is a single SQL statement. The design optimizes the 99\% per-tenant path at the cost of
the cross-tenant path.
\item \textbf{The routing directory.} Sticky single-writer routing needs a consistent, durable
map from tenant to pod. That directory is itself a piece of shared state that must be highly
available and consistent; it is small and changes slowly, but it is a coordination point the
pure shared-nothing story would prefer not to have.
\item \textbf{Measurement scope.} The per-tenant constant is measured on one host, one seed
schema, single-stream warm/cold workloads, with a bounded 2-connection pool. It does not sweep
concurrent query load per tenant (which materializes more connections, each with its own page
cache), nor multi-hour steady-state fragmentation. The 190\,MB base RSS is \cited{} from prior
production sampling, not re-measured on this pass. These are the honest edges of the numbers.
\item \textbf{The fabric is an assembly of shipped primitives, not a shipped fabric.} The
per-tenant measurement, the unified binary, the writer pin, the writer/reader HA roles, the PQ
consensus engine, the wallet subsystem, and the node daemon exist and are cited to source
(\S\ref{sec:builtvroadmap}). The single decentralized confidential fabric of \S\ref{sec:fabric}
--- the fabric-VM as live scheduler, on-chain tenant$\to$writer placement with consensus failover,
on-chain TEE attestation and confidential scheduling, the \texttt{hanzod} auto-join/adaptive-k3s/
self-heal orchestration, and the compute-earning settlement --- is design/roadmap. The efficiency
and scaling results are measured; the fabric is the argued composition on top of them, and is
labelled as such throughout.
\item \textbf{Inference axis not measured on this pass.} Axis-B (GPU) figures --- model resident
size, bandwidth, decode t/s --- are \cited{} from the companion inference papers, and the box
capacity of Table~\ref{tab:box} treats the model reservation and throughput as \cited{}/\modeled{}.
The tenant capacity (Axis A) is the measured part; the inference co-location is architecturally
grounded (FFI, unified memory) but its throughput is not re-measured here.
\end{itemize}
\section{Conclusion}
\label{sec:conclusion}
The unified sovereign-tenant design is efficient and scalable for the same structural reason:
it refuses to share the two resources that normally become bottlenecks. It does not share a
write path --- each tenant is a private single-writer SQLite file --- so writes shard perfectly
and node count grows linearly in active tenants. It does not share $N$ runtimes --- 56 subsystems
live in one binary --- so the fixed platform cost is paid once, at $11$--$32\times$ less memory
than the equivalent mesh. The measured per-active-tenant cost is $\approx$0.25\,MB, an idle
tenant is free, and the \texttt{cache\_size} cap is a tunable ceiling rather than a footprint.
The capacity model that follows from that one constant puts a billion registered tenants on
$\approx$20 standard 128\,GB nodes at 1\% concurrency, and on the 3-node HA floor at 0.1\% --- with
no bottleneck tier and no rewrite between the small and the large regime. Two orthogonal axes
compose in one process: per-tenant state (CPU/RAM, $O(\text{active})$) and shared-weight GPU
inference (VRAM, $O(\text{models})$), which on a \$3{,}200 128\,GB \emph{unified}-memory box
collapse into a complete cloud-in-a-box serving tens of millions of users. Extended over
\texttt{luxfi/consensus}, a leaderless PQ BFT chain schedules placement (not writes) across
independently-owned, trust-tiered nodes --- the shipped writer-pin, reader-HA, wallet, and PQ
identity primitives composed into \texttt{hanzo.network}, a sovereign decentralized confidential
cloud that grows one \texttt{hanzod up} at a time. The design is not free of cost: writer failover,
hot-tenant skew, and cross-tenant queries are genuine prices, and the fabric layer is an argued
assembly of measured primitives, all stated in \S\ref{sec:limits}. But on the axis it is built for
--- serving an enormous number of independent tenants cheaply and scaling their concurrency
linearly --- the measurement supports the architecture.
\appendix
\section{Reproducibility}
\label{sec:repro}
\sloppy
\textbf{Host.} \texttt{evo}: AMD Ryzen AI Max+ 395 (32 cores), 62.4\,GB RAM, Linux 6.17.0-35,
Go 1.26.4, glibc malloc, TeX Live 2023.\\
\textbf{Open path.} \texttt{github.com/hanzoai/dbx} \texttt{v1.16.0} +
\texttt{github.com/hanzoai/sqlite} \texttt{v0.2.1}; production driver cgo/\texttt{mattn}
(\texttt{go-sqlite3} \texttt{v1.14.47}); pure-Go control \texttt{modernc.org/sqlite}
\texttt{v1.51.0}. Pragmas exactly \texttt{sqlite.DefaultPragmas}:
\texttt{busy\_timeout=10000}, \texttt{journal\_mode=WAL},
\texttt{synchronous=NORMAL}, \texttt{foreign\_keys=ON},
\texttt{temp\_store=MEMORY}, \texttt{cache\_size=-32000}.\\
\textbf{Seed.} \texttt{base/tests/data/data.db} (344{,}064\,B) + \texttt{auxiliary.db}
(49{,}152\,B), the checked-in base-schema test database.\\
\textbf{Procedure.} Copy seed to a fresh path per tenant; open through the production path; run
the \emph{warm} workload (\texttt{count(*)} over every non-\texttt{sqlite\_} table, then
\texttt{CREATE TABLE IF NOT EXISTS} + five \texttt{INSERT}s) or the \emph{cold} workload
(\texttt{SELECT 1}); at each of $N \in \{1,10,100,1000\}$ force \texttt{runtime.GC()} +
\texttt{debug.FreeOSMemory()} and read \texttt{VmRSS} from \texttt{/proc/self/status}. Report the
$100\rightarrow1000$ incremental slope. Idle test: close all handles, re-measure; reuse test:
close all, open 1000 fresh, re-measure. Large-tenant knob: inflate the seed to 25\,MB
(60{,}000 padded rows) and repeat the \texttt{cache\_size} sweep under a full-scan warm workload.\\
\textbf{Capacity model.} Eq.~\ref{eq:model} with $U=24$\,GB, $b=0.19$\,GB, $m\in\{0.25,1.0\}$\,MB,
32\,GB nodes, HA floor 3; disk $R\times0.38$\,MB object-tiered.\\
\textbf{In-process call path (\S\ref{sec:compare}, axis 4).} Host \texttt{spark}: NVIDIA GB10
(Grace--Blackwell, aarch64), production driver cgo/\texttt{mattn}. Usage-read latency: seed
$N\in\{100,1000,5000\}$ debits into one org's finance ledger via \texttt{finance.RecordUsage},
then benchmark \texttt{finance.ListUsage} over that org's file ---
\texttt{CLOUD\_KMS\_MASTER\_KEY\_REF=<32B> go test -bench=ListUsage -benchmem
./clients/finance/} in \texttt{github.com/hanzoai/cloud} (1.25/10.5/61\,ms at 100/1000/5000
rows). Functional composition: \texttt{go build ./cmd/smoke \&\& SMOKE\_BASE\_URL=<url>
[SMOKE\_TOKEN=<bearer>] ./smoke} probes one read per mounted subsystem and exits non-zero on any
402-on-read or 5xx; it is wired as the release gate in
\texttt{.github/workflows/release.yml}.
\begin{thebibliography}{9}
\bibitem{hip0106} Hanzo AI. \emph{HIP-0106: Cloud --- Unified Hanzo Binary}.
\url{https://github.com/hanzoai/HIPs}.
\bibitem{unifiedbin} Z.~Kelling. \emph{The Unified Cloud Binary: Collapsing a Microservices
Control Plane into One Go Process}. Hanzo Industries white paper, 2026 (companion architecture
paper; \texttt{cloud-unified-binary}).
\bibitem{hip0116} Hanzo AI. \emph{HIP-0116: Plugin VM / In-Process Composition Model}.
\url{https://github.com/hanzoai/HIPs}.
\bibitem{hip0125} Hanzo AI. \emph{HIP-0125: ZAP-Native Transport \& gRPC Elimination} (renumbered
from HIP-0116 to avoid collision with the plugin-vm model).
\url{https://github.com/hanzoai/HIPs}.
\bibitem{hip0302} Hanzo AI. \emph{HIP-0302: Hanzo Replicate --- Encrypted SQLite Durability for
Base Services}. \url{https://github.com/hanzoai/HIPs}.
\bibitem{hip0107} Hanzo AI. \emph{HIP-0107: Replication and Durability Substrate}.
\url{https://github.com/hanzoai/HIPs}.
\bibitem{hip0032} Hanzo AI. \emph{HIP-0405: S3 CRD}.
\url{https://github.com/hanzoai/HIPs}.
\bibitem{hip0118} Hanzo AI. \emph{HIP-0118: SuperAdmin and Tenant Isolation Model}.
\url{https://github.com/hanzoai/HIPs}.
\bibitem{pqstore} Z.~Kelling. \emph{A Post-Quantum Object Store: Per-Tenant SQLite and a
Dropbox-Class Filesystem over One Encrypted Block Substrate}. Hanzo Industries white paper, 2026
(companion; \texttt{hanzo-pq-object-store}).
\bibitem{edgepaper} Hanzo AI. \emph{Edge LLM Inference Across Commodity Accelerators} (companion
white paper; source of the prior 154--225\,MB \texttt{cloud} working-set measurement).
\bibitem{xbench} Hanzo AI. \emph{Edge LLM Inference Across Commodity Accelerators: Measured AMD,
NVIDIA, and Apple Performance} (companion; source of the cited 8.11\,GiB / 231\,GB/s / decode-t/s
inference figures; \texttt{hanzo-cross-backend-inference}).
\bibitem{luxconsensus} Lux Industries. \emph{HIP-0116: Plugin VM / In-Process Composition} and the
\texttt{luxfi/consensus} BFT engine (\texttt{engine/bft}, \texttt{engine/chain},
\texttt{engine/common}). \url{https://github.com/luxfi/consensus}.
\bibitem{quasar} Lux Industries. \emph{Quasar: Post-Quantum BFT Consensus} (LP-020 / Lux Quasar
consensus white paper; \texttt{luxfi/consensus/protocol/quasar}).
\url{https://github.com/luxfi/consensus}.
\bibitem{tee} Confidential computing hardware: NVIDIA H100/H200 Confidential Computing; Intel Trust
Domain Extensions (TDX); AMD Secure Encrypted Virtualization --- Secure Nested Paging (SEV-SNP).
Vendor documentation.
\bibitem{hip0005} Hanzo AI. \emph{HIP-0005: Post-Quantum Security for AI Infrastructure}.
\url{https://github.com/hanzoai/HIPs}.
\bibitem{hip0001} Hanzo AI. \emph{HIP-0001: AI Token --- Hanzo's Native Currency}.
\url{https://github.com/hanzoai/HIPs}.
\bibitem{hip0117} Hanzo AI. \emph{HIP-0117: BYO Compute Fleet and Metered Billing}.
\url{https://github.com/hanzoai/HIPs}.
\bibitem{sqlite} SQLite. \emph{PRAGMA \texttt{cache\_size}} and \emph{Write-Ahead Logging}.
\url{https://www.sqlite.org/pragma.html#pragma_cache_size}, \url{https://www.sqlite.org/wal.html}.
\bibitem{istio} Istio. \emph{Sidecar resource footprint} (Envoy proxy memory, per-proxy).
\url{https://istio.io/latest/docs/ops/deployment/performance-and-scalability/}.
\end{thebibliography}
\end{document}