Files
engine/engine-guard/README.md
zeekayandHanzo Dev 42e34c2d41 refactor: name only Hanzo Engine in code, comments and docs
Every mention of another inference engine is rewritten to say what our code
does, or the math it computes. Reference values are "the reference value
from the published config" or "the recorded reference output", line
citations into another engine's source become the rule itself, comparisons
with other engines are dropped, and asides about other engines' bugs go.

Identifiers follow:
- CUDA: hanzo-paged-attn's `namespace vllm` is `pagedattn`, hanzo-engine's
  MoE helpers `moe_dtype` and `moe_detail`, the rotary kernels'
  `hanzo_rotary`; the shuffle/ldg macros are HANZO_*.
- GPTQ: the shuffled-layout path (`reconstruct_exllama`, `use_exllama`) is
  `reconstruct_shuffled` / `is_shuffled`, on the kernel and the FFI side.
- Tests: `*_matches_vllm` and friends name what they compare against (the
  reference fixture, the tile rule).
- MiniCPM-o's vision-LLM embedding is `vlm_embedding`; the rotary custom op
  is `rotary-embedding`.
- hanzo-bench's board scores ours against a "rival" whose name comes from
  the manifest: `rival_<tag>.json`, `--rival*` manifest flags, `rival t/s`
  columns; dossier.sh and spec_decode.sh take `--rival-bench`.
- install.ps1 installs this repository's hanzo-cli, as install.sh does,
  with HANZO_INSTALL_NCCL / HANZO_INSTALL_NO_NCCL.
- bench_http.py reads only the hanzo: metric families.

Kept, because they are what they are: LICENSE, LICENSE-APACHE, NOTICE and
tools/upstream-map.sh; each ported file's copyright/licence/provenance
header; Cargo.toml dependency names; data identifiers (Hugging Face repo
IDs, fixture file names and fixture metadata keys the tests read, on-disk
paths, executable names the GPU and engine guards match on a host); model
and format names.

LLM.md says the rule once, under "One name: Hanzo Engine".


Co-authored-by: Hanzo Dev <[email protected]>
2026-10-16 20:59:50 -07:00

2.8 KiB

hanzo-engine-guard

Keeps the local Hanzo engine (hanzo serve) alive safely — it self-heals a crashed engine but can never DoS the machine. One portable daemon; native service install on Linux, macOS, and Windows (all user-level, no admin).

hanzo-engine-guard install     # register as a native service for this OS
hanzo-engine-guard status      # show engine ports, health, desktop state
hanzo-engine-guard uninstall   # remove the service
hanzo-engine-guard run         # run the daemon in the foreground (what the service runs)
OS Mechanism (user-level)
Linux systemd --user service (~/.config/systemd/user/) + enable-linger
macOS launchd LaunchAgent (~/Library/LaunchAgents/ai.hanzo.engine-guard.plist)
Windows Task Scheduler HanzoEngineGuard (/SC ONLOGON, runs pythonw, no console)

Only dependency: Python 3.8+ and (recommended, cross-platform) psutil. Without psutil it falls back to pgrep/pkill on Unix.

What it does

  1. Circuit breaker (always): kills any process that is the legacy server binary named by BOMB_PAT (matched precisely by exe/argv0/comm — never a substring of some innocent command), and if hanzo serve processes ever exceed RUNAWAY_CEILING (default 256) it kills them all. This is the backstop against the historical recursion fork-bomb.

  2. Keepalive (only while a desktop/node is running): if an engine port that was seen healthy goes down with no live process, it respawns it by replaying the exact command + working dir + environment captured while it was healthy (the command alone isn't enough — the engine needs its CUDA/HF env).

Why it cannot DoS the machine

  • Backoff between respawns: 15s → 30s → 60s → 120s → 300s. Never a tight loop.
  • Give-up cap: after MAX_FAILS (8) consecutive failed respawns of a port it stops and logs — a broken engine is never hammered forever.
  • Gated on a desktop/node actually running — no zombie engines after you quit.
  • Snapshot-replay only: it only respawns a port it has seen healthy; it never guesses a command/model.
  • Runaway ceiling: the hard backstop kills any exponential spawn.
  • The systemd unit uses KillMode=process (a guard restart never kills the engines it spawned) and sets no MemoryMax (a cap would count a respawned engine's multi-GB RAM and cgroup-OOM it).

Tuning (env)

  • HANZO_ENGINE_RUNAWAY_CEILING — kill-all threshold (default 256).

Known limitation

The keepalive only recovers ports it has seen healthy (it holds the command/cwd/env snapshot in memory, never on disk — no secrets persisted). If the guard restarts while a port is already down, it can't recover that port until the desktop (or a sibling engine) brings it up once. This is the safe trade-off.