mirror of
https://github.com/hanzoai/dev.git
synced 2026-10-29 18:44:44 +00:00
zeekay
3e5cc1e271
Make hanzo-node natively serve the canonical Hanzo inference engine (OpenAI / Anthropic / Responses HTTP API) in-process, alongside the existing gRPC server, behind a new default-off `engine` Cargo feature. Purely additive: with the feature off the node builds and behaves exactly as before. - Cargo.toml: optional path-deps on hanzo-server-core (default-features=false) and hanzo-engine; `engine` feature gating them; cuda/cudnn/metal/rocm/vulkan/ accelerate/mkl/nccl/flash-attn[-v3] features each forwarding to the matching engine + server-core features (mirrors hanzo-server/Cargo.toml). default = []. - .cargo/config.toml: rustflags target-cpu=native (engine's candle fork needs it). - lib.rs: add EngineConfig (enabled/cpu/engine_api_addr default 0.0.0.0:36900, + cfg-gated ModelSelected) to NodeConfig; cfg-gated hanzo_state on HanzoNode built in new() via HanzoForServerBuilder; start() runs gRPC + axum engine API concurrently (spawn + tokio::join!) when state is present, else unchanged. All engine code is #[cfg(feature = "engine")]; errors map to crate::Error, no unwrap/expect. - bin: --engine-port/HANZO_ENGINE_ADDR, --model/HANZO_ENGINE_MODEL, --engine-cpu; logs the engine URL on startup. Validation (CPU box, GPU-blocked, no weights): - cargo check -p hanzo-node (no features): pass, 0 warnings. - cargo check -p hanzo-node --features engine: pass; warnings only from upstream hanzo-engine/hanzo-server-core, none from hanzo-node. GPU feature graphs (cuda/rocm/metal/...) resolve via cargo metadata but are not compiled here (no GPU toolchains).
6 lines
284 B
TOML
6 lines
284 B
TOML
# The optional `engine` feature pulls in hanzo-engine's candle fork (hanzo-ml),
|
|
# whose SIMD kernels require `-C target-cpu=native` to compile. This is a no-op
|
|
# for the default (engine-off) build, so it is safe to set unconditionally.
|
|
[build]
|
|
rustflags = ["-C", "target-cpu=native"]
|