mirror of
https://github.com/zenlm/engine.git
synced 2026-10-30 15:00:05 +00:00
Eric BuehlerandGitHub
70c7f86691
* Add initial * Loading for text model * Add ple embeddings * Add altup, laurel block * Update rmsnorm * Add mlp * Update attn norm application * Currently no kv shared * Wire it up * It runs * Fix bf16 * Fix scaled embd * Fixes for mean * tmp * Attn confirmed * Fix target_magnitude * Add shared kv * Ok it works * Remove npy * Fix streaming * Remove warnings * Remove paged attn * Refactor rope * Add immediate isq * Add vision & mproj * Update image processor * Vision merge runs, not correct * Remove * Add mobilenet v5 * Add multimodal vision embedding * Fix load * runs * Fix gamma * Works but just not vision tower * It works!! * Tweak * Fix warnings * Move vision tower * Fix warn * Update cache manager things * Refactor * Add audio model, it loads * Add audio processing * It runs at least * tmp * A bit better * Audio works!!!! * Fused attn in vision * Clippy * Update audio runner * Optimized audio model * Remove unused things * Fix inputs processor bug * Remove comments * Clippy * Small optimizations * Format * Correctly register modalities * Add docs * Update readme * Runs there * Fixed padding from https://github.com/Blaizzy/mlx-vlm/pull/410 * Add better checks * Fix sdpa n_kv_groups * Vision encoder works! * Rotate image * Clippy * Fix cuda loading * Updated device mapper * Fix overflow * Fix dtype errors * Refactor image/audio embeddings * Fix metal * Fix dtype mismatch * Audio processing fixes * Audio processing fixes * Works * Audio is good * Fix boi/eoi too * Embed the chat templates * Better embedding accuracy in non f32 * More f32 * Support bf16 on metal * Add more ISQ * Fixed device map * Clippy * Gemma3n no paged attn * Fix saturating sub * Faster rmsnorm * Use sdpa for vision model * Fix ple bug * Fix name * Fix multiaudio * Add matformer config loading * Add docs * Add support for matformer in auto device mapper * Update docs * Typos * Tweak * Tweak * Fix multidevice * Fix gemma3n text model auto device map * Fix dims3 * Fix auto devic emap vision * Non-metal keeps PLE on cpu * Complete merge * Vision dtype f16 -> f32 * Fix metal nm device * Fix uqff * Typos * Reference uqff * Fix tests
50 lines
1.6 KiB
Django/Jinja
50 lines
1.6 KiB
Django/Jinja
{{ bos_token }}
|
|
{%- if messages[0]['role'] == 'system' -%}
|
|
{%- if messages[0]['content'] is string -%}
|
|
{%- set first_user_prefix = messages[0]['content'] + '
|
|
|
|
' -%}
|
|
{%- else -%}
|
|
{%- set first_user_prefix = messages[0]['content'][0]['text'] + '
|
|
|
|
' -%}
|
|
{%- endif -%}
|
|
{%- set loop_messages = messages[1:] -%}
|
|
{%- else -%}
|
|
{%- set first_user_prefix = "" -%}
|
|
{%- set loop_messages = messages -%}
|
|
{%- endif -%}
|
|
{%- for message in loop_messages -%}
|
|
{%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
|
|
{{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
|
|
{%- endif -%}
|
|
{%- if (message['role'] == 'assistant') -%}
|
|
{%- set role = "model" -%}
|
|
{%- else -%}
|
|
{%- set role = message['role'] -%}
|
|
{%- endif -%}
|
|
{{ '<start_of_turn>' + role + '
|
|
' + (first_user_prefix if loop.first else "") }}
|
|
{%- if message['content'] is string -%}
|
|
{{ message['content'] | trim }}
|
|
{%- elif message['content'] is iterable -%}
|
|
{%- for item in message['content'] -%}
|
|
{%- if item['type'] == 'audio' -%}
|
|
{{ '<audio_soft_token>' }}
|
|
{%- elif item['type'] == 'image' -%}
|
|
{{ '<image_soft_token>' }}
|
|
{%- elif item['type'] == 'text' -%}
|
|
{{ item['text'] | trim }}
|
|
{%- endif -%}
|
|
{%- endfor -%}
|
|
{%- else -%}
|
|
{{ raise_exception("Invalid content type") }}
|
|
{%- endif -%}
|
|
{{ '<end_of_turn>
|
|
' }}
|
|
{%- endfor -%}
|
|
{%- if add_generation_prompt -%}
|
|
{{'<start_of_turn>model
|
|
'}}
|
|
{%- endif -%}
|