mirror of
https://github.com/hanzoai/engine.git
synced 2026-11-20 14:17:52 +00:00
google/embeddinggemma-2 (EmbeddingGemma2Model, Apache-2.0) loads as `embeddinggemma2`, from its config's architecture or by name: the text backbone of transformers 5.19, with Gemma 4 norms, a value norm with no scale, attention scale 1, full layers at head dim 512 and one KV head, an inclusive sliding window, projection-only per-layer embeddings, a per-layer scalar and the 512 -> 768 projection on every token. The vision and audio towers are never read. Against sentence-transformers 6.1.0 in F32, over 64 texts in 14 languages up to 1,256 tokens, every vector's cosine is at least 0.9999993 at 768, 512, 256 and 128 dimensions, and every token count matches. To serve it: - Pooling reads sentence-transformers 6's config (embedding_dimension, pooling_mode) beside the older flags. Mean pooling is a mean, where it was a sum; every sentence-transformers mode pools, where all but mean and last-token panicked; and what it cannot do fails at load. - /v1/embeddings takes `dimensions`: the leading values, normalized again, as OpenAI and truncate_dim do. - A multi-model entry takes `revision`, so a deployment pins the weights its indexes were built from. Claude-Session: https://claude.ai/code/session_01SCPMGSQsLNF2YtHFL4bwyp
3.1 KiB
3.1 KiB
Summary
Getting Started
Using hanzo
Models
Text Model Guides
- DeepSeek V2
- DeepSeek V3
- Gemma 2
- Gemma 3
- Gemma 3n
- GLM4
- GLM-4.7-Flash
- GLM-4.7 MoE
- GPT-OSS
- Qwen 3
- Qwen 3 Next
- SmolLM3
Multimodal Model Guides
- Gemma 4
- Idefics 2
- Idefics 3
- LLaVA
- Llama 3.2 Vision
- Llama 4
- MiniCPM-O 2.6
- Mistral 3
- Phi 3.5 MoE
- Phi 3.5 Vision
- Phi 4 Multimodal
- Qwen 2-VL
- Qwen 3-VL
- Qwen 3.5
- Video Input
Other Model Guides
- FLUX (Diffusion)
- Voxtral (Speech-to-Text)
- Dia (Speech)
- EmbeddingGemma
- EmbeddingGemma 2
- Qwen3 Embedding
Performance & Optimization
- Performance Guide
- Quantization Overview
- ISQ (In-Situ Quantization)
- UQFF Format
- Topology
- Importance Matrix
- PagedAttention
- Flash Attention
- MLA (Multi-head Latent Attention)
- Speculative Decoding
- Device Mapping
- Distributed Inference
Adapters & Model Customization
- Training (LoRA fine-tuning)
- Adapter Models
- LoRA/X-LoRA Examples
- Non-Granular Scalings
- AnyMoE
- MatFormer
Features
- Agentic Features
- Tool Calling
- Web Search
- Chat Templates
- Sampling Options
- TOML Selector
- Multi-Model Support