mirror of
https://github.com/zenlm/engine.git
synced 2026-10-30 15:00:05 +00:00
Eric BuehlerandGitHub
af1b5c290a
* Implement imatrix for ISQ * Implement imatrix quantization for q3k, q2k * Fix build on cuda * Implement imatrix quantization for q5k, q6k * Utilize merged rev * Fix typo * Update docs * Update docs * Track stats for imatrix * Implement internal imatrix quantization * Implement internal imatrix quantization for all models but the moe ones * Add for the moe models * Simpler * Handle clearing kv cache * Add calibration data * Add better calibration data * Add example and docs * Add example and docs
351 B
351 B
Calibration data
Sourced from: https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8
calibration_datav3.txtis the original data (64181 tokens)calibration_datav3_small.txtis up to line 1241 of the original data (30173 tokens)
Using the full dataset will take the longest but also provide the most accurate calibration.