mirror of
https://github.com/zenlm/engine.git
synced 2026-10-30 15:00:05 +00:00
Eric BuehlerandEricLBuehler
d9c6fb7fa0
* Intial work on phi3v * Add the image embedding layer * Lints * Implement the loader * Add infrastructure for phi3 image processor * Merge * Merge * Merge * Merge * Partially implement padding * Implement the hd transform step * Work on the image processor * Clippy * Complete the phi3v inputs processor * Rename * Merge * Merge * Rename to phi3v and fix deser * Fix varbuilder * Fix varbuilder * Default for do convert rgb * Some defaults * Allow no processor config * Setup debug flag * Add phi3v * Implement messages flattening * Update * Rewrite the pad, hd transform * Clippy * Detect num channels * Fix reshape * Fix global image channel dim * Fix assert * Fix dtype * Fix gt * Fix image id neg * Fix dim0 of pixel values * Fix dtype * Check if model supports gemm * Fix some shape errors * Fix some shape errors * Fix rank of slice_assign * Fix image toks * Properly downcase * Fix response * Fix response * Allow no images in prompt * Output correct hidden state * Fix nonzero and add test * Fix n image toks * Add mistralrs_vision * Typo * Fix and add tests * Fix indexing * Fix test condition * Fix unsqueeze * Fix dtype for norm * Update clip * Clippy * Run clip in f32 * Run in bf16 * Run in bf16 again * Fix dtype * Set toks to have correct context lens * Set toks to have correct context lens * Support multiple GGUF files (#379) * Move to gguf module * Add content abstraction for multiple gguf files * Fix test * Allow specifying and loading multiple gguf files * Update docs and examples * Print some info * Merge * Organize normal loading metadata (#381) * Organize normal loading metadata * Fix * Bump version 0.1.13 -> 0.1.14 (#382) * Patch incorrect unwrap and bump version (#383) * Patch incorrect unwrap * Bump version to 0.1.15 * More verbose logging during loading (#385) * More verbose logging when loading * More logging * Refactor enabling debug logging (#387) * Refactor enabling debug logging * Fix reversed order * Merge * Merge * Merge * Use precise gelu * Use correct kernel * Debugging commit * Add fused bias linear * Finish merge * Use fused layer in clip * Save progress * Remove debugs * Update example * Resize exact * Update interpolate * Fix batch dim * Update test and transform * It works * Add some examples * Allow more than one image * Add support in python api * Add to toml selector * Update python api * Overhaul readme and docs * Update * Export vision arch * Export vision arch * Export vision arch * Fix max img dim * Fix unwrap
19 lines
535 B
Python
19 lines
535 B
Python
import transformers
|
|
|
|
tokenizer = transformers.AutoTokenizer.from_pretrained(
|
|
"mistralai/Mistral-7B-Instruct-v0.3"
|
|
)
|
|
|
|
res = tokenizer.apply_chat_template(
|
|
[
|
|
{"role": "user", "content": "Hello"},
|
|
{"role": "assistant", "content": "Hi there"},
|
|
{"role": "user", "content": "Who are you"},
|
|
{"role": "assistant", "content": " I am an assistant "},
|
|
{"role": "user", "content": "Another question"},
|
|
],
|
|
add_generation_prompt=True,
|
|
tokenize=False,
|
|
)
|
|
print(res.replace("\n", "\\n"))
|