Eric BuehlerandGitHub c82f9e033d Reworked and improved attention chunking (#1591) ...

* Add chunking to naive attn

* Add it to cublaslt as well

* Even better!

* Clippy

* Remove prompt chunksize apis

* Prompt chunksize logging

* Format

* Determine based on mask type

* Max seq len in device map

* Max seq len in device map

* Format

2025-08-03 07:52:33 -04:00

..

adapter_chat.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

chat.py

Handle base_model.model case in lora (#1514)

2025-06-25 11:51:13 -04:00

completion.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

dia.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

flux.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

gemma3.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

gemma3n.py

Add full Gemma 3n support! (#1519)

2025-07-09 06:59:51 -04:00

idefics2.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

idefics3.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

json_schema.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

lark.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

llama4.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

llama_vision.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

llava_next.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

llava.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

llguidance.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

mcp_chat.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

minicpmo_2_6.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

mistral3.py

Updated examples for #1509

2025-06-23 21:36:54 +02:00

multi_model_chat.py

Add multi-model support! (#1512)

2025-06-24 19:49:26 -04:00

openai_response_format.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

phi3_duckduckgo_mistral.rs.ipynb

Calculate perplexity of ISQ models (#931)

2024-11-25 21:54:52 -05:00

phi3v_base64.py

Fix base64

2025-04-13 19:21:32 -04:00

phi3v_local_img.py

Python file cleanup (#668)

2024-08-05 20:01:14 -04:00

phi3v.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

phi4mm_audio.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

phi4mm.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

qwen2vl.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

qwen3.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

regex.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

responses_audio.py

Reworked and improved attention chunking (#1591)

2025-08-03 07:52:33 -04:00

responses_vision.py

Initial support for OpenAI Responses API (#1580)

2025-07-26 08:49:27 -04:00

responses.py

Initial support for OpenAI Responses API (#1580)

2025-07-26 08:49:27 -04:00

smollm3.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

stream_completion_bench.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

streaming_completion.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

streaming.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

tool_calling.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00

web_search.py

Add the SmolLM3 model! (#1501)

2025-07-08 14:48:58 -04:00