Files
skills/public/hip-0068.html
zeekayandHanzo Dev 9e10c1c10e Each llms index points at the other two
hanzoskills.com/llms.txt now opens with the install line, the API
reference index and the company index, so an agent that lands on any
one of the three can find the rest.

robots.txt claimed 537 documents while llms.txt counts 920. The number
is gone from robots.txt; llms.txt derives it.

public/ is regenerated from scripts/site.py, which it had fallen
behind: the copy-prompt menu and agent bar were in the generator and
not in the committed pages. tests/site.mjs passes against the result.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-11 00:27:59 -07:00

649 lines
40 KiB
HTML

<!doctype html>
<html lang="en"><head>
<meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
<title>hip-0068</title>
<meta name="color-scheme" content="dark">
<style>
/* Committed to one visual world: a dark instrument panel. No light variant —
the dot field and the hairline rules only read on black, and a naive
inversion would wash both out. Every colour is painted explicitly so the page
never borrows a host background. */
@font-face{font-family:'Zen';src:url('/fonts/Zen-Variable.woff2') format('woff2');font-weight:100 900;font-style:normal;font-display:swap}
@font-face{font-family:'Zen';src:url('/fonts/Zen-Italic-Variable.woff2') format('woff2');font-weight:100 900;font-style:italic;font-display:swap}
@font-face{font-family:'Zen Mono';src:url('/fonts/ZenMono-Variable.woff2') format('woff2');font-weight:100 900;font-style:normal;font-display:swap}
@font-face{font-family:'Zen Mono';src:url('/fonts/ZenMono-Italic-Variable.woff2') format('woff2');font-weight:100 900;font-style:italic;font-display:swap}
@font-face{font-family:'Zen Pixel Square';src:url('/fonts/ZenPixel-Square.woff2') format('woff2');font-style:normal;font-display:swap}
:root{
--ink:#f4f4f2; --soft:#9c9c98; --faint:#6a6a66; --ground:#000; --panel:#0a0a0a;
--line:#232323; --line-hi:#3a3a3a; --accent:#f4f4f2;
--font-zen-sans:'Zen',-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif;
--font-zen-mono:'Zen Mono',ui-monospace,"SF Mono",SFMono-Regular,Menlo,Consolas,monospace;
--font-zen-pixel-square:'Zen Pixel Square',monospace;
--mono:var(--font-zen-mono);
--sans:var(--font-zen-sans);
/* Controls are pills and surfaces are soft, as on hanzo.ai. */
--r:999px; --r-soft:14px;
}
*{box-sizing:border-box}
html{-webkit-text-size-adjust:100%}
body{margin:0;background:var(--ground);color:var(--ink);font-family:var(--sans);
font-size:16px;line-height:1.65;-webkit-font-smoothing:antialiased}
.wrap{max-width:60rem;margin:0 auto;padding:0 1.75rem}
/* The dot field. A halftone made of one repeating radial-gradient, faded out
along the diagonal by a mask so it reads as a wave rather than wallpaper. */
/* No overflow clip here: the dot field is inset to the box already, and a clip
would cut off the copy-prompt menu. */
.hero{position:relative;border-bottom:1px solid var(--line)}
.hero::before{content:"";position:absolute;inset:0;
background-image:radial-gradient(var(--line-hi) 1px, transparent 1px);
background-size:7px 7px;
-webkit-mask-image:linear-gradient(115deg,#000 0%,rgba(0,0,0,.55) 38%,transparent 72%);
mask-image:linear-gradient(115deg,#000 0%,rgba(0,0,0,.55) 38%,transparent 72%);
pointer-events:none}
.hero .wrap{position:relative;padding-top:5.5rem;padding-bottom:4rem}
nav{border-bottom:1px solid var(--line)}
nav .in{max-width:60rem;margin:0 auto;padding:1.1rem 1.75rem;display:flex;
justify-content:space-between;align-items:center;gap:1rem}
.brand{font-family:var(--mono);font-size:.85rem;letter-spacing:.14em;
text-transform:uppercase;color:var(--ink);text-decoration:none;
display:inline-flex;align-items:center;gap:.6rem}
.brand i{font-style:normal;color:var(--faint)}
.mark{width:1.15em;height:1.15em;display:block;flex:0 0 auto}
nav a.up{font-family:var(--mono);font-size:.72rem;letter-spacing:.14em;
text-transform:uppercase;color:var(--soft);text-decoration:none}
nav a.up:hover{color:var(--ink)}
h1{font-family:var(--mono);font-size:clamp(2rem,5.4vw,3.4rem);line-height:1.12;
letter-spacing:-.02em;margin:0 0 1.1rem;font-weight:500;text-wrap:balance}
h1 em{font-style:normal;color:var(--faint)}
.lede{color:var(--soft);font-size:1.05rem;max-width:52ch;margin:0 0 2.25rem}
.btns{display:flex;flex-wrap:wrap;gap:.7rem;margin:0 0 .5rem;align-items:center}
.btn{display:inline-flex;align-items:center;gap:.5rem;font-family:var(--mono);
font-size:.72rem;letter-spacing:.14em;text-transform:uppercase;
padding:.8rem 1.4rem;border:1px solid var(--line-hi);border-radius:var(--r);
color:var(--ink);text-decoration:none;background:transparent;cursor:pointer;
line-height:1.2;font-weight:400}
.btn:hover{background:#141414;text-decoration:none;border-color:var(--soft)}
.btn.solid{background:var(--ink);color:#000;border-color:var(--ink)}
.btn.solid:hover{background:#d8d8d4}
.btn .chev{width:.7em;height:.7em;flex:0 0 auto;transition:transform .15s ease}
.btn[aria-expanded="true"] .chev{transform:rotate(180deg)}
/* Copy-prompt selector. Each row copies the prompt for its own target. */
.copy{position:relative;display:inline-flex}
.menu{position:absolute;top:calc(100% + .5rem);left:0;z-index:20;min-width:21rem;
background:var(--panel);border:1px solid var(--line-hi);border-radius:var(--r-soft);
padding:.4rem;display:flex;flex-direction:column;
box-shadow:0 12px 32px rgba(0,0,0,.75)}
.menu[hidden]{display:none}
.mi{display:flex;align-items:center;gap:.7rem;width:100%;text-align:left;
padding:.62rem .8rem;border:0;border-radius:calc(var(--r-soft) - .35rem);
background:transparent;color:var(--ink);font-family:var(--mono);font-size:.76rem;
letter-spacing:.04em;text-decoration:none;cursor:pointer;line-height:1.3;
white-space:nowrap}
.mi:hover,.mi:focus-visible{background:#181818;outline:none}
.mi .mark{width:1.05rem;height:1.05rem;color:var(--soft)}
.mi:hover .mark{color:var(--ink)}
.mi u{margin-left:auto;color:var(--faint);text-decoration:none;font-size:.68rem;
letter-spacing:.1em;text-transform:uppercase}
/* Works-with bar: one scrolling row, name and mark per agent. */
.strip{border-bottom:1px solid var(--line);background:var(--panel)}
.strip .in{max-width:60rem;margin:0 auto;padding:.9rem 1.75rem;display:flex;
align-items:center;gap:1.25rem;font-family:var(--mono);font-size:.72rem;
letter-spacing:.1em;text-transform:uppercase;color:var(--faint)}
.strip b{color:var(--soft);font-weight:400;flex:0 0 auto;
border-right:1px solid var(--line);padding-right:1.25rem;align-self:stretch;
display:flex;align-items:center}
.bar{display:flex;align-items:center;gap:.5rem;flex:1 1 auto;min-width:0;
overflow-x:auto;scrollbar-width:none;scroll-snap-type:x proximity;
-webkit-overflow-scrolling:touch;
-webkit-mask-image:linear-gradient(90deg,#000 calc(100% - 3rem),transparent);
mask-image:linear-gradient(90deg,#000 calc(100% - 3rem),transparent)}
.bar::-webkit-scrollbar{display:none}
.agent{display:inline-flex;align-items:center;gap:.45rem;flex:0 0 auto;
padding:.4rem .85rem;border:1px solid var(--line);border-radius:var(--r);
color:var(--soft);white-space:nowrap;scroll-snap-align:start;
text-decoration:none;background:var(--ground)}
.agent:hover{color:var(--ink);border-color:var(--line-hi);text-decoration:none}
.agent .mark{width:.95rem;height:.95rem}
h2{font-family:var(--mono);font-size:1.1rem;letter-spacing:-.01em;margin:2.75rem 0 .75rem;font-weight:500}
h3{font-size:1rem;margin:2rem 0 .5rem;font-weight:600}
h4,h5,h6{font-size:.94rem;margin:1.5rem 0 .5rem;font-weight:600}
p{margin:0 0 1rem;max-width:70ch}
ul,ol{margin:0 0 1rem;padding-left:1.35rem;max-width:70ch}
li{margin:.22rem 0}
a{color:var(--ink);text-decoration:underline;text-underline-offset:3px;
text-decoration-color:var(--line-hi)}
a:hover{text-decoration-color:var(--ink)}
a:focus-visible{outline:1px solid var(--ink);outline-offset:3px}
code{font-family:var(--mono);font-size:.85em;background:var(--panel);
border:1px solid var(--line);border-radius:5px;padding:.1em .36em}
pre{background:var(--panel);border:1px solid var(--line);padding:1rem 1.1rem;
border-radius:var(--r-soft);overflow-x:auto;margin:0 0 1.25rem}
pre code{background:none;border:0;padding:0;font-size:.82rem;line-height:1.7}
blockquote{margin:0 0 1rem;padding-left:1rem;border-left:1px solid var(--line-hi);color:var(--soft)}
hr{border:0;border-top:1px solid var(--line);margin:2.5rem 0}
table{border-collapse:collapse;width:100%;margin:0 0 1.25rem;font-size:.92rem;
display:block;overflow-x:auto}
th,td{border:1px solid var(--line);padding:.5rem .7rem;text-align:left}
.curl{display:block;background:var(--panel);border:1px solid var(--line);
border-radius:var(--r-soft);padding:.95rem 1.1rem;font-family:var(--mono);
font-size:.8rem;color:var(--soft);overflow-x:auto;margin:0 0 2.5rem;
white-space:pre;line-height:1.9}
.curl b{color:var(--ink);font-weight:400}
.curl i{font-style:normal;color:var(--faint)}
/* index */
#q{width:100%;padding:.85rem 1.2rem;font-family:var(--mono);font-size:.85rem;color:var(--ink);
background:var(--panel);border:1px solid var(--line);border-radius:var(--r);margin:0 0 .6rem}
#q::placeholder{color:var(--faint);text-transform:uppercase;letter-spacing:.1em;font-size:.72rem}
#q:focus{outline:none;border-color:var(--soft)}
#count{color:var(--faint);font-family:var(--mono);font-size:.72rem;letter-spacing:.1em;
text-transform:uppercase;margin:0 0 2rem}
.cat{margin:2.5rem 0 .5rem;font-family:var(--mono);font-size:.7rem;letter-spacing:.16em;
text-transform:uppercase;color:var(--faint);border-bottom:1px solid var(--line);
padding-bottom:.45rem}
.row{display:block;padding:.6rem .9rem;margin:0 -.9rem;border-bottom:1px solid var(--line);
border-radius:var(--r-soft);text-decoration:none}
.row:hover{background:#0a0a0a}
.nm{font-family:var(--mono);font-size:.84rem;color:var(--ink);display:block}
.ds{color:var(--soft);font-size:.9rem;display:block;margin-top:.12rem}
footer{margin-top:4rem;padding:2rem 0 4rem;border-top:1px solid var(--line);
color:var(--faint);font-size:.85rem}
footer a{color:var(--soft)}
.fmap{display:grid;gap:1.75rem;grid-template-columns:repeat(auto-fit,minmax(11rem,1fr));
margin:0 0 2rem;font-size:.85rem}
.fmap b{display:block;font-family:var(--mono);font-size:.68rem;letter-spacing:.16em;
text-transform:uppercase;color:var(--faint);font-weight:400;margin:0 0 .6rem;
border-bottom:1px solid var(--line);padding-bottom:.4rem}
.fmap a{display:inline-block;color:var(--ink);text-decoration:none;margin:.3rem .35rem 0 0}
.fmap a:hover{text-decoration:underline}
.fmap div{color:var(--faint);line-height:1.5}
.fend{border-top:1px solid var(--line);padding-top:1.25rem;font-size:.82rem}
@media (max-width:640px){
.hero .wrap{padding-top:3.25rem;padding-bottom:2.75rem}
.btns{gap:.5rem}
.btn{padding:.75rem 1.15rem;font-size:.68rem}
.copy,.copy .btn{width:100%;justify-content:center}
.menu{left:0;right:0;min-width:0}
.strip .in{flex-direction:column;align-items:flex-start;gap:.6rem;padding:.9rem 1.75rem}
.strip b{border-right:0;padding-right:0}
.bar{width:100%}
}
</style>
<link rel="alternate" type="text/markdown" href="/hip-0068.md">
<link rel="icon" href="/favicon.svg">
<link rel="canonical" href="https://hanzoskills.com/hip-0068">
</head>
<!-- Reading this with a machine? Every page here is also plain markdown.
This one: https://hanzoskills.com/hip-0068.md
Index: https://hanzoskills.com/llms.txt
Corpus: https://hanzoskills.com/all.md
Append .md to any path for the markdown twin. -->
<body>
<nav><div class="in">
<a class="brand" href="/"><svg class="mark" viewBox="0 0 67 67" fill="currentColor" aria-hidden="true"><path d="M22.21 67V44.6369H0V67H22.21Z"/><path d="M66.7038 22.3184H22.2534L0.0878906 44.6367H44.4634L66.7038 22.3184Z"/><path d="M22.21 0H0V22.3184H22.21V0Z"/><path d="M66.7198 0H44.5098V22.3184H66.7198V0Z"/><path d="M66.7198 67V44.6369H44.5098V67H66.7198Z"/></svg> HANZO<i>SKILLS</i></a>
<a class="up" href="/">All skills</a>
</div></nav>
<div class="wrap">
<h1>hip-0068</h1><p class="lede">HIP-68: Ingress Standard. Status Final. Hanzo architectural specification.</p><h1>HIP-0068: Ingress Standard</h1>
<h2>Abstract</h2>
<p>This proposal defines the Ingress standard for the Hanzo ecosystem. Hanzo Ingress is the Layer 7 reverse proxy and load balancer that sits at the edge of every Hanzo Kubernetes cluster. It handles host-based routing, TLS termination (via Cloudflare proxy mode), load balancing, health checks, and middleware chains for all inbound traffic.</p>
<p>Hanzo Ingress is a maintained fork of <strong>Traefik v3.6</strong>. It watches Kubernetes Ingress resources and IngressRoute CRDs natively -- no config generation, no external sync loops, no sidecar injection. A new Service or Ingress resource appears in the cluster; Ingress picks it up within seconds and starts routing traffic.</p>
<p>This HIP is explicitly distinct from <strong>HIP-44 (API Gateway)</strong>, which is the application-level gateway handling authentication, rate limiting, and request transformation for <code>api.hanzo.ai</code>. Ingress operates one layer below: it routes <code>api.hanzo.ai</code> traffic to the API Gateway, <code>hanzo.ai</code> traffic to the marketing site, <code>platform.hanzo.ai</code> traffic to Dokploy, and so on. The API Gateway is one of many backends behind Ingress.</p>
<p><strong>Repository</strong>: <a href="https://github.com/hanzoai/ingress">github.com/hanzoai/ingress</a> <strong>Entrypoints</strong>: web (8080), websecure (8443) <strong>Image</strong>: <code>ghcr.io/hanzoai/ingress:latest</code> <strong>Providers</strong>: KubernetesIngress, KubernetesCRD</p>
<h2>Motivation</h2>
<h3>The Edge Routing Problem</h3>
<p>Hanzo operates two Kubernetes clusters (<code>hanzo-k8s</code> and <code>lux-k8s</code>) serving 30+ domains across multiple services. Each domain needs:</p>
<ol><li><strong>Host-based routing</strong>: <code>api.hanzo.ai</code> goes to the API Gateway, <code>platform.hanzo.ai</code> goes to Dokploy, <code>kms.hanzo.ai</code> goes to KMS. Each domain is a separate routing decision at the edge.</li><li><strong>TLS termination</strong>: Every domain needs HTTPS. Managing 30+ TLS certificates manually is operationally unsustainable.</li><li><strong>Health-aware load balancing</strong>: When a backend pod crashes, the edge proxy must stop sending traffic to it immediately, not after a 30-second DNS TTL.</li><li><strong>Middleware chains</strong>: Some services need rate limiting at the edge, some need IP whitelisting, some need request buffering for large uploads. These cross-cutting concerns must be configurable per-route, not hardcoded.</li></ol>
<h3>Why Not nginx-ingress?</h3>
<p>The NGINX Ingress Controller is the Kubernetes default. It has three problems that make it unsuitable for Hanzo:</p>
<ol><li><strong>Config generation lag</strong>: nginx-ingress watches Kubernetes resources, generates an <code>nginx.conf</code> file, validates it, and performs a graceful reload. This takes 5-30 seconds. During that window, new services are unreachable and deleted services still receive traffic.</li></ol>
<ol><li><strong>No native CRDs</strong>: nginx-ingress relies on annotations (<code>nginx.ingress.kubernetes.io/rewrite-target</code>, <code>nginx.ingress.kubernetes.io/ssl-redirect</code>) for advanced configuration. Annotations are untyped strings with no schema validation. A typo silently does nothing. Traefik&#x27;s IngressRoute CRDs are typed, validated at apply time, and self-documenting.</li></ol>
<ol><li><strong>No native integration with Hanzo PaaS</strong>: Dokploy (the Hanzo PaaS platform) generates Traefik configuration natively for app deployments. Using nginx-ingress would require a translation layer between Dokploy&#x27;s Traefik config and nginx annotations. Hanzo Ingress eliminates this translation entirely.</li></ol>
<p>| Factor | nginx-ingress | Hanzo Ingress (Traefik) | |--------|--------------|------------------------| | Config update | 5-30s (generate + reload) | &lt; 1s (watch + apply) | | Advanced routing | Annotations (untyped) | IngressRoute CRDs (typed) | | PaaS integration | Translation layer needed | Native (Dokploy speaks Traefik) | | Middleware | Annotation hacks | First-class Middleware CRD | | TCP/UDP routing | Limited | Full TCP/UDP routing support | | Dashboard | None (third-party) | Built-in (disabled by default) |</p>
<p>The tradeoff: NGINX has marginally higher raw throughput for static content. For a dynamic Kubernetes environment where routing changes frequently and services come and go, Traefik&#x27;s native Kubernetes integration is decisive.</p>
<h2>Specification</h2>
<h3>Architecture</h3>
<pre><code> Internet
|
+----------+----------+
| DigitalOcean LB |
| (TCP passthrough) |
+----------+----------+
|
+----------+----------+
| Hanzo Ingress |
| (Traefik v3.6) |
| :8080 / :8443 |
+----------+----------+
|
+--------------------+--------------------+
| | | | |
+----+----+ +--+---+ +---+--+ +----+----+ +---+---+
| API | | PaaS | | IAM | | KMS | | Chat |
| Gateway | |(Dokp)| | | | | | |
| :8080 | |:3000 | |:8000 | | :8080 | |:3081 |
+---------+ +------+ +------+ +---------+ +-------+</code></pre>
<p>Ingress is the first process to handle an inbound connection after the cloud load balancer. It terminates TLS (or relies on Cloudflare to do so), inspects the <code>Host</code> header, matches the request against its routing table, applies any configured middleware, and forwards the request to the correct backend service.</p>
<h3>Entrypoints</h3>
<p>| Name | Port | Protocol | Purpose | |------|------|----------|---------| | <code>web</code> | 8080 | HTTP | Plaintext HTTP; redirects to <code>websecure</code> in production | | <code>websecure</code> | 8443 | HTTPS | TLS-terminated HTTPS traffic | | <code>traefik</code> | 9090 | HTTP | Dashboard and API (disabled by default) | | <code>metrics</code> | 9100 | HTTP | Prometheus metrics endpoint |</p>
<p>In production with Cloudflare proxy mode, the DigitalOcean LoadBalancer forwards TCP traffic on ports 80/443 to Ingress ports 8080/8443. Cloudflare terminates the external TLS connection and re-encrypts to the origin using Cloudflare&#x27;s edge certificates.</p>
<h3>Providers</h3>
<p>Hanzo Ingress watches two Kubernetes providers simultaneously:</p>
<p><strong>1. KubernetesIngress Provider</strong></p>
<p>Watches standard <code>networking.k8s.io/v1</code> Ingress resources. Any service that creates an Ingress resource with <code>ingressClassName: hanzo-ingress</code> is automatically routed.</p>
<pre><code class="lang-yaml">apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: api-gateway
namespace: hanzo
annotations:
traefik.ingress.kubernetes.io/router.entrypoints: websecure
spec:
ingressClassName: hanzo-ingress
rules:
- host: api.hanzo.ai
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: api-gateway
port:
number: 8080</code></pre>
<p><strong>2. KubernetesCRD Provider</strong></p>
<p>Watches Traefik-specific CRDs for advanced routing. IngressRoute resources support weighted routing, header-based matching, middleware chains, and TCP/UDP routing that standard Ingress resources cannot express.</p>
<pre><code class="lang-yaml">apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: platform-route
namespace: hanzo
spec:
entryPoints:
- websecure
routes:
- match: Host(`platform.hanzo.ai`)
kind: Rule
services:
- name: dokploy
port: 3000
middlewares:
- name: rate-limit
- name: security-headers</code></pre>
<h3>Host Routing Table (Production)</h3>
<p>| Host | Backend Service | Port | Cluster | |------|----------------|------|---------| | <code>api.hanzo.ai</code> | API Gateway (HIP-44) | 8080 | hanzo-k8s | | <code>llm.hanzo.ai</code> | LLM Gateway (HIP-4) | 4000 | hanzo-k8s | | <code>hanzo.id</code> | Hanzo IAM | 8000 | hanzo-k8s | | <code>lux.id</code> | Hanzo IAM | 8000 | hanzo-k8s | | <code>zoo.id</code> | Hanzo IAM | 8000 | hanzo-k8s | | <code>pars.id</code> | Hanzo IAM | 8000 | hanzo-k8s | | <code>kms.hanzo.ai</code> | KMS (Hanzo KMS) | 8080 | hanzo-k8s | | <code>platform.hanzo.ai</code> | Platform (Dokploy) | 3000 | hanzo-k8s | | <code>console.hanzo.ai</code> | Console | 3001 | hanzo-k8s | | <code>cloud.hanzo.ai</code> | Cloud | 3002 | hanzo-k8s | | <code>hanzo.app</code> | Main App | 3000 | hanzo-k8s | | <code>api.lux.network</code> | Lux Gateway (KrakenD) | 8080 | lux-k8s | | <code>cloud.lux.network</code> | Lux Cloud | 3000 | lux-k8s | | <code>markets.lux.network</code> | Markets | 3000 | lux-k8s |</p>
<h3>TLS Configuration</h3>
<p><strong>Primary mode: Cloudflare Proxy (Flexible SSL)</strong></p>
<p>All Hanzo domains use Cloudflare as DNS proxy. Cloudflare terminates TLS at the edge and forwards traffic to the origin. In this mode, Ingress receives plaintext HTTP from Cloudflare on port 8080. No certificate management is required on the origin.</p>
<pre><code>Client --[TLS]--&gt; Cloudflare --[HTTP]--&gt; DO LB --[HTTP]--&gt; Ingress :8080</code></pre>
<p><strong>Secondary mode: Full (Strict) SSL with CertManager</strong></p>
<p>For domains where end-to-end encryption is required (e.g., <code>kms.hanzo.ai</code> handling secrets), CertManager provisions Let&#x27;s Encrypt certificates and stores them as Kubernetes Secrets. Ingress reads these secrets and terminates TLS at port 8443.</p>
<pre><code class="lang-yaml">apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: kms-tls
namespace: hanzo
spec:
secretName: kms-tls-secret
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
dnsNames:
- kms.hanzo.ai</code></pre>
<pre><code class="lang-yaml">apiVersion: traefik.io/v1alpha1
kind: TLSOption
metadata:
name: default
namespace: hanzo
spec:
minVersion: VersionTLS12
preferServerCipherSuites: true
cipherSuites:
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_AES_256_GCM_SHA384
- TLS_AES_128_GCM_SHA256
- TLS_CHACHA20_POLY1305_SHA256</code></pre>
<h3>Middleware</h3>
<p>Middleware CRDs are applied per-route via IngressRoute references. Each middleware is a standalone Kubernetes resource that can be shared across routes.</p>
<p><strong>Rate Limiting</strong></p>
<pre><code class="lang-yaml">apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: rate-limit
namespace: hanzo
spec:
rateLimit:
average: 100
burst: 200
period: 1m
sourceCriterion:
ipStrategy:
depth: 1</code></pre>
<p><strong>Security Headers</strong></p>
<pre><code class="lang-yaml">apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: security-headers
namespace: hanzo
spec:
headers:
frameDeny: true
sslRedirect: true
browserXssFilter: true
contentTypeNosniff: true
stsIncludeSubdomains: true
stsPreload: true
stsSeconds: 31536000
customResponseHeaders:
X-Powered-By: &quot;&quot;
Server: &quot;&quot;</code></pre>
<p><strong>Retry</strong></p>
<pre><code class="lang-yaml">apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: retry
namespace: hanzo
spec:
retry:
attempts: 3
initialInterval: 100ms</code></pre>
<p><strong>Circuit Breaker</strong></p>
<pre><code class="lang-yaml">apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: circuit-breaker
namespace: hanzo
spec:
circuitBreaker:
expression: &quot;LatencyAtQuantileMS(50.0) &gt; 1000 || NetworkErrorRatio() &gt; 0.30&quot;</code></pre>
<p><strong>IP Whitelist (Admin Services)</strong></p>
<pre><code class="lang-yaml">apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: admin-whitelist
namespace: hanzo
spec:
ipAllowList:
sourceRange:
- &quot;127.0.0.1/8&quot;
- &quot;127.0.0.1/12&quot;</code></pre>
<h3>Health Checks</h3>
<p>Ingress performs active health checks on backend services:</p>
<pre><code class="lang-yaml">apiVersion: traefik.io/v1alpha1
kind: ServersTransport
metadata:
name: default
namespace: hanzo
spec:
serverName: &quot;&quot;
insecureSkipVerify: false
maxIdleConnsPerHost: 200
forwardingTimeouts:
dialTimeout: 5s
responseHeaderTimeout: 30s
idleConnTimeout: 90s</code></pre>
<p>Kubernetes readiness probes are the primary health signal. When a pod&#x27;s readiness probe fails, Kubernetes removes it from the Endpoints object. Ingress watches Endpoints and stops routing to that pod within one watch cycle (typically &lt; 1 second). No additional health check configuration is needed for most services.</p>
<h3>Load Balancing</h3>
<p>Default algorithm is weighted round-robin. Sticky sessions are available via cookie-based affinity for stateful services:</p>
<pre><code class="lang-yaml">apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: stateful-service
namespace: hanzo
spec:
routes:
- match: Host(`app.hanzo.ai`)
services:
- name: stateful-backend
port: 8080
sticky:
cookie:
name: hanzo_affinity
secure: true
httpOnly: true</code></pre>
<h3>Observability</h3>
<p><strong>Access Logs</strong></p>
<p>Structured JSON on stdout, one line per request:</p>
<pre><code class="lang-json">{
&quot;time&quot;: &quot;2026-02-24T12:00:00Z&quot;,
&quot;level&quot;: &quot;info&quot;,
&quot;msg&quot;: &quot;&quot;,
&quot;ClientAddr&quot;: &quot;127.0.0.1:54321&quot;,
&quot;ClientHost&quot;: &quot;127.0.0.1&quot;,
&quot;Duration&quot;: 12345678,
&quot;DownstreamStatus&quot;: 200,
&quot;RequestHost&quot;: &quot;api.hanzo.ai&quot;,
&quot;RequestMethod&quot;: &quot;POST&quot;,
&quot;RequestPath&quot;: &quot;/v1/chat/completions&quot;,
&quot;RequestProtocol&quot;: &quot;HTTP/1.1&quot;,
&quot;RouterName&quot;: &quot;api-gateway@kubernetes&quot;,
&quot;ServiceName&quot;: &quot;api-gateway-hanzo@kubernetes&quot;,
&quot;entryPointName&quot;: &quot;websecure&quot;
}</code></pre>
<p><strong>Prometheus Metrics</strong></p>
<p>Exported on port 9100 with namespace <code>hanzo_ingress</code>:</p>
<p>| Metric | Type | Description | |--------|------|-------------| | <code>hanzo_ingress_entrypoint_requests_total</code> | Counter | Total requests by entrypoint, method, protocol, code | | <code>hanzo_ingress_entrypoint_request_duration_seconds</code> | Histogram | Request duration by entrypoint | | <code>hanzo_ingress_entrypoint_open_connections</code> | Gauge | Open connections by entrypoint, method, protocol | | <code>hanzo_ingress_service_requests_total</code> | Counter | Total requests by service, method, protocol, code | | <code>hanzo_ingress_service_request_duration_seconds</code> | Histogram | Request duration by service | | <code>hanzo_ingress_service_open_connections</code> | Gauge | Open connections by service | | <code>hanzo_ingress_service_retries_total</code> | Counter | Retry count by service | | <code>hanzo_ingress_service_server_up</code> | Gauge | Backend server health (1=up, 0=down) | | <code>hanzo_ingress_tls_certs_not_after</code> | Gauge | TLS certificate expiry timestamp | | <code>hanzo_ingress_config_reloads_total</code> | Counter | Configuration reload count | | <code>hanzo_ingress_config_last_reload_success</code> | Gauge | Last reload success (1/0) |</p>
<p><strong>Tracing</strong></p>
<p>OpenTelemetry traces are emitted for every request, propagating <code>traceparent</code> and <code>tracestate</code> headers to backend services. Traces are exported to the cluster&#x27;s OTLP collector for correlation with backend spans.</p>
<h3>RBAC</h3>
<p>Ingress requires the following Kubernetes RBAC permissions:</p>
<pre><code class="lang-yaml">apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: hanzo-ingress
rules:
- apiGroups: [&quot;&quot;]
resources: [&quot;services&quot;, &quot;endpoints&quot;, &quot;secrets&quot;]
verbs: [&quot;get&quot;, &quot;list&quot;, &quot;watch&quot;]
- apiGroups: [&quot;networking.k8s.io&quot;]
resources: [&quot;ingresses&quot;, &quot;ingressclasses&quot;]
verbs: [&quot;get&quot;, &quot;list&quot;, &quot;watch&quot;]
- apiGroups: [&quot;networking.k8s.io&quot;]
resources: [&quot;ingresses/status&quot;]
verbs: [&quot;update&quot;]
- apiGroups: [&quot;traefik.io&quot;]
resources: [&quot;ingressroutes&quot;, &quot;ingressroutetcps&quot;, &quot;ingressrouteudps&quot;,
&quot;middlewares&quot;, &quot;middlewaretcps&quot;, &quot;tlsoptions&quot;, &quot;tlsstores&quot;,
&quot;traefikservices&quot;, &quot;serverstransports&quot;, &quot;serverstransporttcps&quot;]
verbs: [&quot;get&quot;, &quot;list&quot;, &quot;watch&quot;]
- apiGroups: [&quot;extensions&quot;, &quot;networking.k8s.io&quot;]
resources: [&quot;ingresses&quot;, &quot;ingressclasses&quot;]
verbs: [&quot;get&quot;, &quot;list&quot;, &quot;watch&quot;]</code></pre>
<p>The ClusterRole is bound to a dedicated ServiceAccount (<code>hanzo-ingress</code>) via ClusterRoleBinding.</p>
<h2>Deployment</h2>
<h3>Kubernetes (Production)</h3>
<p>Two replicas with rolling updates (<code>maxSurge: 1</code>, <code>maxUnavailable: 0</code>). Resource limits: 200m-1000m CPU, 128Mi-512Mi memory. Pod anti-affinity ensures replicas run on different nodes for availability.</p>
<pre><code class="lang-yaml">apiVersion: apps/v1
kind: Deployment
metadata:
name: hanzo-ingress
namespace: hanzo
spec:
replicas: 2
selector:
matchLabels:
app: hanzo-ingress
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
template:
metadata:
labels:
app: hanzo-ingress
annotations:
prometheus.io/scrape: &quot;true&quot;
prometheus.io/port: &quot;9100&quot;
spec:
serviceAccountName: hanzo-ingress
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values: [&quot;hanzo-ingress&quot;]
topologyKey: kubernetes.io/hostname
containers:
- name: ingress
image: ghcr.io/hanzoai/ingress:latest
args:
- --entrypoints.web.address=:8080
- --entrypoints.websecure.address=:8443
- --entrypoints.metrics.address=:9100
- --providers.kubernetesingress=true
- --providers.kubernetesingress.ingressclass=hanzo-ingress
- --providers.kubernetescrd=true
- --metrics.prometheus=true
- --metrics.prometheus.entrypoint=metrics
- --accesslog=true
- --accesslog.format=json
- --ping=true
- --api.dashboard=false
- --api.insecure=false
ports:
- name: web
containerPort: 8080
- name: websecure
containerPort: 8443
- name: metrics
containerPort: 9100
resources:
requests: { cpu: &quot;200m&quot;, memory: &quot;128Mi&quot; }
limits: { cpu: &quot;1000m&quot;, memory: &quot;512Mi&quot; }
readinessProbe:
httpGet:
path: /ping
port: 9100
initialDelaySeconds: 5
periodSeconds: 10
livenessProbe:
httpGet:
path: /ping
port: 9100
initialDelaySeconds: 10
periodSeconds: 15
---
apiVersion: v1
kind: Service
metadata:
name: hanzo-ingress
namespace: hanzo
annotations:
service.beta.kubernetes.io/do-loadbalancer-protocol: &quot;tcp&quot;
service.beta.kubernetes.io/do-loadbalancer-size-slug: &quot;lb-small&quot;
service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: &quot;true&quot;
spec:
type: LoadBalancer
selector:
app: hanzo-ingress
ports:
- name: web
port: 80
targetPort: 8080
- name: websecure
port: 443
targetPort: 8443
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
name: hanzo-ingress
annotations:
ingressclass.kubernetes.io/is-default-class: &quot;true&quot;
spec:
controller: traefik.io/ingress-controller</code></pre>
<h3>Docker (Development)</h3>
<pre><code class="lang-yaml"># compose.yml
services:
ingress:
image: ghcr.io/hanzoai/ingress:latest
command:
- --entrypoints.web.address=:8080
- --entrypoints.websecure.address=:8443
- --providers.docker=true
- --providers.docker.exposedbydefault=false
- --api.dashboard=true
- --api.insecure=true
- --accesslog=true
- --accesslog.format=json
ports:
- &quot;80:8080&quot;
- &quot;443:8443&quot;
- &quot;9090:8080&quot;
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro</code></pre>
<p>In development mode, the Docker provider watches container labels for routing configuration. The dashboard is enabled for debugging.</p>
<h3>IngressClass</h3>
<p>Hanzo Ingress registers itself as the default IngressClass (<code>hanzo-ingress</code>). Any Ingress resource without an explicit <code>ingressClassName</code> is handled by Hanzo Ingress. Services that need a different ingress controller (e.g., for testing) can specify an alternate IngressClass.</p>
<h2>Per-brand replication</h2>
<p>Each brand runs the identical topology with its own values:</p>
<p>| Concern | Per-brand value | |---------|-----------------| | Identity origin | <code>iam.hanzo.ai</code> / <code>lux.id</code> / <code>zoo.id</code> / <code>id.bootno.de</code> / <code>pars.id</code> | | API host | <code>api.&lt;brand-domain&gt;</code> | | Apps | brand-scoped <code>&lt;org&gt;-&lt;app&gt;</code> client IDs (HIP-0111) | | Secrets | KMS project per brand | | Container registry | <code>oci.hanzo.ai/&lt;org&gt;/&lt;app&gt;</code>, org-namespaced and never mixed (HIP-0033) |</p>
<p>Nothing in the topology is brand-special-cased. A brand is a set of values plugged into the same shape.</p>
<p>Ingress is configured with <code>ingress.kubernetes.io/<em></code> annotations ONLY. A <code>traefik.</em></code> annotation is silently ignored, which leaves a route believing it has a rate limit it does not have — the failure is invisible until load arrives.</p>
<h2>Relationship to Other HIPs</h2>
<p>| HIP | Relationship | |-----|-------------| | <strong>HIP-4</strong> (LLM Gateway) | Ingress routes <code>llm.hanzo.ai</code> traffic to LLM Gateway. Separate processes at different layers. | | <strong>HIP-14</strong> (Application Deployment) | Dokploy generates IngressRoute resources that Ingress watches and routes. | | <strong>HIP-26</strong> (IAM) | Ingress routes <code>hanzo.id</code>, <code>lux.id</code>, <code>zoo.id</code>, <code>pars.id</code> to IAM. No auth at the ingress layer; auth is handled by the API Gateway or services themselves. | | <strong>HIP-27</strong> (KMS) | Ingress routes <code>kms.hanzo.ai</code> to KMS. TLS certificates for strict SSL mode are stored as K8s Secrets provisioned by CertManager. | | <strong>HIP-132</strong> (Telemetry) | Ingress exports Prometheus metrics consumed by Grafana dashboards. Access logs feed into the log aggregation pipeline. | | <strong>HIP-106</strong> (Cloud) | Ingress routes <code>cloud.hanzo.ai</code> and <code>cloud.lux.network</code> to their respective cloud services. | | <strong>HIP-44</strong> (API Gateway) | The API Gateway (KrakenD) is a backend behind Ingress. Ingress handles L7 host routing; the API Gateway handles application-level concerns (auth, rate limiting, circuit breaking). | | <strong>HIP-49</strong> (DNS) | DNS records point domains to the DigitalOcean LoadBalancer IP. Cloudflare proxies these records for DDoS protection and edge TLS. |</p>
<h3>Layer Separation</h3>
<pre><code>Internet
--&gt; Cloudflare (DDoS protection, edge TLS, CDN caching)
--&gt; DigitalOcean LoadBalancer (TCP passthrough, health checks)
--&gt; Hanzo Ingress [HIP-68] (host routing, middleware, backend selection)
--&gt; API Gateway [HIP-44] (auth, rate limiting, request transformation)
--&gt; Backend Services (LLM Gateway, IAM, KMS, ...)</code></pre>
<p>Each layer has a single responsibility. Ingress does not authenticate requests. The API Gateway does not route by hostname. Cloudflare does not know about backend services. This separation means each layer can be replaced, scaled, or debugged independently.</p>
<h2>Security Considerations</h2>
<h3>Network Security</h3>
<ul><li>Ingress listens on cluster-internal interfaces. External access is via DigitalOcean LoadBalancer only.</li><li>Backend connections use cluster DNS (e.g., <code>localhost</code>). No traffic leaves the cluster for internal routing.</li><li>The Traefik dashboard and API are disabled by default in production. When enabled for debugging, they are accessible only via <code>kubectl port-forward</code>.</li></ul>
<h3>TLS Security</h3>
<ul><li>Minimum TLS 1.2 for direct TLS termination. TLS 1.3 preferred.</li><li>HSTS headers with <code>includeSubdomains</code> and <code>preload</code> on all HTTPS responses.</li><li>Cloudflare provides edge TLS with automatic certificate rotation. Origin certificates are managed by CertManager where strict SSL is required.</li></ul>
<h3>Request Validation</h3>
<ul><li>Maximum request header size: 16 KB</li><li>Maximum request body: configurable per-route (default 10 MB, 100 MB for upload routes)</li><li>Proxy protocol enabled to preserve client IP through the load balancer chain</li></ul>
<h3>RBAC Security</h3>
<ul><li>Ingress runs as a non-root user with a read-only root filesystem.</li><li>The ServiceAccount has minimal permissions: read-only access to Ingress, Service, Endpoint, and Secret resources. Write access only to Ingress status (for IP address annotation).</li><li>No access to ConfigMaps, Deployments, Pods, or other sensitive resources.</li></ul>
<h3>DDoS Mitigation</h3>
<ul><li>Cloudflare absorbs volumetric DDoS at the edge before traffic reaches the origin.</li><li>DigitalOcean LoadBalancer provides basic SYN flood protection.</li><li>Ingress rate limiting middleware provides per-IP throttling as a last line of defense.</li></ul>
<h2>Implementation Roadmap</h2>
<h3>Phase 1: Core Deployment (Complete)</h3>
<ul><li>Traefik v3.6 fork with Hanzo defaults</li><li>Deployment on hanzo-k8s with DigitalOcean LoadBalancer</li><li>KubernetesIngress and KubernetesCRD providers</li><li>Host routing for all production domains</li><li>Cloudflare Flexible SSL for TLS termination</li><li>Prometheus metrics on port 9100</li><li>JSON access logs on stdout</li></ul>
<h3>Phase 3: Multi-Cluster (Q2 2026)</h3>
<ul><li>Deploy Ingress on lux-k8s cluster</li><li>Cross-cluster service discovery via ExternalName services</li><li>Global server load balancing (GSLB) via Cloudflare DNS</li><li>Canary deployments with weighted IngressRoute services</li></ul>
<h2>References</h2>
<ol><li><a href="https://doc.traefik.io/traefik/">Traefik Documentation</a></li><li><a href="https://doc.traefik.io/traefik/routing/providers/kubernetes-crd/">Traefik Kubernetes IngressRoute</a></li><li><a href="https://doc.traefik.io/traefik/middlewares/overview/">Traefik Middleware Reference</a></li><li><a href="https://kubernetes.io/docs/concepts/services-networking/ingress/">Kubernetes Ingress Specification</a></li><li><a href="https://developers.cloudflare.com/ssl/origin-configuration/ssl-modes/">Cloudflare SSL/TLS Modes</a></li><li><a href="https://docs.digitalocean.com/products/networking/load-balancers/">DigitalOcean Load Balancer</a></li><li>HIP-44: API Gateway Standard</li><li><a href="./hip-0026-identity-access-management-standard.md">HIP-26: Identity &amp; Access Management</a></li><li><a href="./hip-0132-one-telemetry-plane.md">HIP-132: One Telemetry Plane</a></li><li><a href="https://github.com/hanzoai/ingress">Ingress Repository</a></li></ol>
<h2>Copyright</h2>
<p>Copyright and related rights waived via <a href="https://creativecommons.org/publicdomain/zero/1.0/">CC0</a>.</p>
<footer>
<div class="fmap">
<div><b>Build</b>
<a href="https://hanzo.app">hanzo.app</a> app builder
<a href="https://hanzo.codes">hanzo.codes</a> code with an agent
<a href="https://hanzoskills.com">hanzoskills.com</a> what an agent reads</div>
<div><b>Run</b>
<a href="https://console.hanzo.ai">console.hanzo.ai</a> cloud console
<a href="https://api.hanzo.ai">api.hanzo.ai</a> the API</div>
<div><b>Account</b>
<a href="https://hanzo.id">hanzo.id</a> sign in
<a href="https://pay.hanzo.ai">pay.hanzo.ai</a> top up
<a href="https://billing.hanzo.ai">billing.hanzo.ai</a> invoices and usage</div>
<div><b>Company</b>
<a href="https://hanzo.ai">hanzo.ai</a> canonical
<a href="https://hanzo.ventures">hanzo.ventures</a> ventures</div>
</div>
<div class="fend"><a href="/hip-0068.md">This page as plain text</a> · <a href="/llms.txt">Index for agents</a></div>
</footer>
</div></body></html>