zenlm/llama.cpp
1
0
Fork 0
mirror of https://github.com/zenlm/llama.cpp.git synced 2026-10-30 15:00:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
master
llama.cpp/models/ggml-vocab-qwen2.gguf
T
Ren XuanchengGitHubRen XuanchengGeorgi Gerganov
 9d4240051f llama : add BPE pre-tokenization for Qwen2 (#7114) 
* Add BPE pre-tokenization for Qwen2.

* minor : fixes

---------

Co-authored-by: Ren Xuancheng <[email protected]>
Co-authored-by: Georgi Gerganov <[email protected]>
2024-05-08 15:06:43 +03:00

5.7 MiB
Raw Permalink History

View Raw
Reference in New Issue View Git Blame Copy Permalink