mirror of
https://github.com/hanzoai/engine.git
synced 2026-11-21 02:36:58 +00:00
Admission compared the prompt with the model's window (262144 for Qwen3.8) rather than the context being served, so an oversized prompt was admitted, could evict a running request, and was then dropped by the scheduler without an answer. The limit is the smaller of the model's window and the paged pool. Co-authored-by: Hanzo Dev <[email protected]>
hanzo-engine
Core crate of hanzo including the models and associated executors.
Documentation: https://docs.rs/hanzo-engine/