hanzoai/papers
2
0
Fork 0
mirror of https://github.com/hanzoai/papers.git synced 2026-10-22 17:20:17 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

papers: assured networking, test evaluation, and insights analytics revisions

Browse Source 
(cherry picked from commit b94030fbbafcfb5f460f3e0f97797b8e0a98a9da)

Co-authored-by: Hanzo Dev <[email protected]>
This commit is contained in:
zeekay
2026-07-26 20:10:12 -07:00
co-authored by Hanzo Dev
parent 163aa37538
commit 7ed17b441d
4 changed files with 19 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files
defense/hanzo-assured-networking.tex
+1 -1
View File
@@ -398,7 +398,7 @@ The production deployment runs on Kubernetes with 65+ deployments across multipl
\begin{itemize}
\item \textbf{High availability}: Multi-replica deployments (2+ for stateless services) with zero-downtime rolling updates (\texttt{maxSurge: 1, maxUnavailable: 0}).
\item \textbf{Secrets management}: KMSSecret CRDs for declarative secret synchronization from Hanzo KMS to Kubernetes secrets. Zero plaintext in YAML manifests.
\item \textbf{Storage}: Persistent volumes on block storage for stateful services (PostgreSQL, NATS JetStream, ClickHouse).
\item \textbf{Storage}: Persistent volumes on block storage for stateful services (PostgreSQL, NATS JetStream, Datastore).
\item \textbf{Multi-cluster}: \texttt{hanzo-k8s} (DigitalOcean SFO3) for services, \texttt{apps-gke} (GCP) for ARM64 CI runners.
\end{itemize}
defense/hanzo-test-evaluation.tex
+7 -7
View File
@@ -12,7 +12,7 @@
\makecover
\begin{abstract}
We present an integrated verification and validation (V\&V) framework combining continuous observability, behavioral analytics, ML pipeline orchestration, digital twin environments, and formal cryptographic proofs. The observability platform ingests logs, metrics, and distributed traces into a ClickHouse OLAP backend, with LLM-specific telemetry tracking token usage, cost, latency, and quality metrics across 100+ AI providers. Temporal workflows orchestrate LLM evaluation, trace clustering, and sentiment analysis, while Dagster DAGs manage ETL pipelines for training data preparation. A network simulation environment supports configurable topologies with 38 registered virtual machines, enabling mission rehearsal and system qualification. The framework includes 17 formal cryptographic proofs, 28+ consensus tests, and automated CI/CD with multi-platform builds. Anomaly detection, feature flag-controlled rollout, and A/B testing with statistical significance provide continuous data-driven decision support.
We present an integrated verification and validation (V\&V) framework combining continuous observability, behavioral analytics, ML pipeline orchestration, digital twin environments, and formal cryptographic proofs. The observability platform ingests logs, metrics, and distributed traces into a Datastore OLAP backend, with LLM-specific telemetry tracking token usage, cost, latency, and quality metrics across 100+ AI providers. Temporal workflows orchestrate LLM evaluation, trace clustering, and sentiment analysis, while Dagster DAGs manage ETL pipelines for training data preparation. A network simulation environment supports configurable topologies with 38 registered virtual machines, enabling mission rehearsal and system qualification. The framework includes 17 formal cryptographic proofs, 28+ consensus tests, and automated CI/CD with multi-platform builds. Anomaly detection, feature flag-controlled rollout, and A/B testing with statistical significance provide continuous data-driven decision support.
\end{abstract}
\paragraph{Executive Summary.}
@@ -52,9 +52,9 @@ Hanzo O11y provides unified observability across three telemetry signals:
\toprule
\textbf{Signal} & \textbf{Capabilities} & \textbf{Storage} \\
\midrule
Logs & Full-text search, structured filters, correlation & ClickHouse \\
Metrics & Time-series, histograms, percentiles, Apdex & ClickHouse \\
Traces & Flamegraphs, Gantt charts, span detail & ClickHouse \\
Logs & Full-text search, structured filters, correlation & Datastore \\
Metrics & Time-series, histograms, percentiles, Apdex & Datastore \\
Traces & Flamegraphs, Gantt charts, span detail & Datastore \\
\bottomrule
\end{tabular}
\caption{O11y three-signal observability architecture.}
@@ -108,7 +108,7 @@ Insights provides behavioral analytics with IAM-integrated multi-tenancy:
\begin{itemize}
\item \textbf{Event ingestion}: Rust capture service for high-throughput event collection, Kafka-compatible streaming via Hanzo Stream.
\item \textbf{OLAP engine}: ClickHouse columnar storage for sub-second queries across billions of events.
\item \textbf{OLAP engine}: Datastore columnar storage (Hanzo's Apache-2.0 ClickHouse fork~\cite{clickhouse}) for sub-second queries across billions of events.
\item \textbf{Cohort segmentation}: Group users by behavior patterns, demographics, or temporal characteristics.
\item \textbf{Funnel analysis}: Multi-step conversion tracking with statistical significance testing.
\item \textbf{Retention modeling}: Cohort-based retention curves with churn prediction.
@@ -172,7 +172,7 @@ ClassifySentimentWorkflow & Sentiment analysis on LLM outputs \\
Dagster manages ETL and data preparation workflows:
\begin{itemize}
\item \textbf{PostgreSQL $\to$ ClickHouse ETL}: Real-time event ingestion with sensor triggers.
\item \textbf{PostgreSQL $\to$ Datastore ETL}: Real-time event ingestion with sensor triggers.
\item \textbf{Historical backfill}: Time-partitioned asset materialization for training data.
\item \textbf{Materialized columns}: Pre-compute expensive aggregations for query performance.
\item \textbf{Person resolution}: User identity unification across events.
@@ -352,7 +352,7 @@ Consensus round & 4.6 ns & --- & --- & 0 \\
]
\node[box] (app) {Application Layer\\(LLM calls, services)};
\node[box, below=of app] (otel) {OpenTelemetry SDK\\(traces, metrics, logs)};
\node[box, below=of otel] (collect) {OTel Collector\\$\to$ Kafka $\to$ ClickHouse};
\node[box, below=of otel] (collect) {OTel Collector\\$\to$ Kafka $\to$ Datastore};
\node[box, below left=1cm and -0.5cm of collect] (o11y) {O11y\\Query Service};
\node[box, below=of collect] (temporal) {Temporal\\ML Workflows};
\node[box, below right=1cm and -0.5cm of collect] (insights) {Insights\\Analytics};
hanzo-insights-analytics.tex
+10 -10
View File
@@ -45,7 +45,7 @@
\hanzocoverpage
\begin{abstract}
We present Hanzo Insights, a privacy-first product analytics platform built on the Umami framework with ClickHouse as the analytical backend. Insights provides real-time event tracking, funnel analysis, cohort retention, and feature flag management without collecting personally identifiable information (PII). The system processes over 50 million events daily with sub-second query latency on dashboards spanning 90-day windows. Key innovations include: (1) a cookie-less visitor identification scheme using salted hashing of request attributes, (2) a feature flag engine with percentage rollouts, user targeting, and A/B experiment support, and (3) a streaming ingestion pipeline with at-most-once deduplication. Insights replaces Google Analytics and LaunchDarkly in the Hanzo stack, eliminating third-party data sharing while providing equivalent functionality.
We present Hanzo Insights, a privacy-first product analytics platform built on the Umami framework with Hanzo Datastore---our Apache-2.0 ClickHouse fork---as the analytical backend. Insights provides real-time event tracking, funnel analysis, cohort retention, and feature flag management without collecting personally identifiable information (PII). The system processes over 50 million events daily with sub-second query latency on dashboards spanning 90-day windows. Key innovations include: (1) a cookie-less visitor identification scheme using salted hashing of request attributes, (2) a feature flag engine with percentage rollouts, user targeting, and A/B experiment support, and (3) a streaming ingestion pipeline with at-most-once deduplication. Insights replaces Google Analytics and LaunchDarkly in the Hanzo stack, eliminating third-party data sharing while providing equivalent functionality.
\end{abstract}
\section{Introduction}
@@ -57,7 +57,7 @@ Hanzo Insights resolves this tension by combining Umami's~\cite{umami2020} cooki
\paragraph{Contributions.}
\begin{itemize}
\item A cookie-less visitor identification scheme using daily-rotated salted hashes of IP address, User-Agent, and site identifier.
\item A ClickHouse-backed analytical engine supporting sub-second queries over billions of events.
\item A Datastore-backed analytical engine supporting sub-second queries over billions of events.
\item An integrated feature flag system with percentage rollouts, targeting rules, and automatic experiment analysis.
\item A real-time dashboard with live visitor counts, event streams, and configurable metric widgets.
\end{itemize}
@@ -76,7 +76,7 @@ Hanzo Insights resolves this tension by combining Umami's~\cite{umami2020} cooki
\node[box] (sdk) {Client SDK\\(JS/React)};
\node[box, below=of sdk] (collect) {Collection\\API};
\node[box, below=of collect] (stream) {Event\\Stream};
\node[box, below left=0.6cm and 0.3cm of stream] (ch) {ClickHouse};
\node[box, below left=0.6cm and 0.3cm of stream] (ch) {Datastore};
\node[box, below right=0.6cm and 0.3cm of stream] (rt) {Real-Time\\Aggregator};
\node[box, right=1.5cm of collect] (flags) {Feature\\Flag Engine};
\node[box, left=1.5cm of collect] (dash) {Dashboard\\API};
@@ -93,7 +93,7 @@ Hanzo Insights resolves this tension by combining Umami's~\cite{umami2020} cooki
\label{fig:insights}
\end{figure}
Events flow through a three-stage pipeline (Figure~\ref{fig:insights}): collection, streaming, and storage. The client SDK sends events to the Collection API, which validates, deduplicates, and forwards them to an event stream. ClickHouse consumes the stream for persistent storage, while the Real-Time Aggregator maintains in-memory counters for live dashboards.
Events flow through a three-stage pipeline (Figure~\ref{fig:insights}): collection, streaming, and storage. The client SDK sends events to the Collection API, which validates, deduplicates, and forwards them to an event stream. Datastore consumes the stream for persistent storage, while the Real-Time Aggregator maintains in-memory counters for live dashboards.
\subsection{Visitor Identification}
@@ -107,7 +107,7 @@ The daily salt $\text{salt}_d$ rotates at midnight UTC, ensuring that visitor id
\subsection{Event Schema}
Events are stored in a ClickHouse MergeTree table:
Events are stored in a Datastore MergeTree table:
\begin{lstlisting}[language=TypeScript]
interface Event {
@@ -170,11 +170,11 @@ ClickHouse's \texttt{windowFunnel} function evaluates multi-step funnels in a si
\subsection{Real-Time Dashboard}
The dashboard displays live metrics using Server-Sent Events (SSE). The Real-Time Aggregator maintains sliding window counters in memory, flushed to ClickHouse every 10 seconds. Dashboard widgets include: active visitors, pageviews per minute, top pages, referrer breakdown, geographic distribution, and device/browser splits.
The dashboard displays live metrics using Server-Sent Events (SSE). The Real-Time Aggregator maintains sliding window counters in memory, flushed to Datastore every 10 seconds. Dashboard widgets include: active visitors, pageviews per minute, top pages, referrer breakdown, geographic distribution, and device/browser splits.
\section{Implementation}
Insights is implemented as a Go service (18,000 lines) with a React dashboard (12,000 lines). The collection endpoint processes events in batches of 1,000, inserting into ClickHouse via the native protocol for maximum throughput.
Insights is implemented as a Go service (18,000 lines) with a React dashboard (12,000 lines). The collection endpoint processes events in batches of 1,000, inserting into Datastore via the native protocol for maximum throughput.
\begin{table}[h]
\centering
@@ -195,9 +195,9 @@ Feature flag impact & 150 & 520 \\
\section{Security}
\paragraph{No PII Storage.} IP addresses are used only for visitor hashing and geolocation, then discarded. User-Agent strings are parsed into device/browser/OS categories and discarded. The raw identifiers never reach ClickHouse.
\paragraph{No PII Storage.} IP addresses are used only for visitor hashing and geolocation, then discarded. User-Agent strings are parsed into device/browser/OS categories and discarded. The raw identifiers never reach Datastore.
\paragraph{Data Isolation.} Each tenant's data is scoped by \texttt{siteId}. ClickHouse queries include a mandatory \texttt{siteId} predicate enforced at the API layer. Cross-tenant queries are impossible by construction.
\paragraph{Data Isolation.} Each tenant's data is scoped by \texttt{siteId}. Datastore queries include a mandatory \texttt{siteId} predicate enforced at the API layer. Cross-tenant queries are impossible by construction.
\paragraph{Collection Security.} The collection endpoint validates the origin header against registered site domains, preventing unauthorized event submission. Events exceeding 8KB are rejected.
@@ -205,7 +205,7 @@ Feature flag impact & 150 & 520 \\
\section{Conclusion}
Hanzo Insights demonstrates that privacy-first analytics need not sacrifice functionality. By combining cookie-less visitor identification with ClickHouse's columnar storage, Insights provides funnel analysis, cohort retention, and feature flag management with sub-second query latency---all without storing personally identifiable information. The integrated feature flag engine eliminates the need for a separate service, reducing infrastructure complexity while enabling data-driven product decisions.
Hanzo Insights demonstrates that privacy-first analytics need not sacrifice functionality. By combining cookie-less visitor identification with Datastore's columnar storage, Insights provides funnel analysis, cohort retention, and feature flag management with sub-second query latency---all without storing personally identifiable information. The integrated feature flag engine eliminates the need for a separate service, reducing infrastructure complexity while enabling data-driven product decisions.
\bibliographystyle{plain}
\begin{thebibliography}{10}
hanzo-stack-comparison/sections/08-analytics-hanzo-insights.tex
+1 -1
View File
@@ -5,7 +5,7 @@
\subsection{Overview}
Hanzo Insights is a fork of Umami adapted for the Hanzo platform. It provides privacy-first web and product analytics with no cookies, no personal data collection, and full GDPR/CCPA compliance by design. Data is stored in ClickHouse for high-throughput analytical queries.
Hanzo Insights is a fork of Umami adapted for the Hanzo platform. It provides privacy-first web and product analytics with no cookies, no personal data collection, and full GDPR/CCPA compliance by design. Data is stored in Hanzo Datastore, our Apache-2.0 ClickHouse fork, for high-throughput analytical queries.
\subsection{Feature Matrix}