mirror of
https://github.com/zenlm/engine.git
synced 2026-10-30 15:00:05 +00:00
Eric BuehlerandGitHub
af1b5c290a
* Implement imatrix for ISQ * Implement imatrix quantization for q3k, q2k * Fix build on cuda * Implement imatrix quantization for q5k, q6k * Utilize merged rev * Fix typo * Update docs * Update docs * Track stats for imatrix * Implement internal imatrix quantization * Implement internal imatrix quantization for all models but the moe ones * Add for the moe models * Simpler * Handle clearing kv cache * Add calibration data * Add better calibration data * Add example and docs * Add example and docs