* Messages bugfixes * Avoid materializing flash attn masks
mistralrs
Mistral.rs provides a convenient Rust multithreaded/async API. To install, add mistralrs = { git = "https://github.com/EricLBuehler/mistral.rs.git" } to the Cargo.toml file.
mistralrs = { git = "https://github.com/EricLBuehler/mistral.rs.git" }
Examples can be found here.
