mirror of
https://github.com/hanzoai/engine.git
synced 2026-11-20 14:17:52 +00:00
google/embeddinggemma-2 (EmbeddingGemma2Model, Apache-2.0) loads as `embeddinggemma2`, from its config's architecture or by name: the text backbone of transformers 5.19, with Gemma 4 norms, a value norm with no scale, attention scale 1, full layers at head dim 512 and one KV head, an inclusive sliding window, projection-only per-layer embeddings, a per-layer scalar and the 512 -> 768 projection on every token. The vision and audio towers are never read. Against sentence-transformers 6.1.0 in F32, over 64 texts in 14 languages up to 1,256 tokens, every vector's cosine is at least 0.9999993 at 768, 512, 256 and 128 dimensions, and every token count matches. To serve it: - Pooling reads sentence-transformers 6's config (embedding_dimension, pooling_mode) beside the older flags. Mean pooling is a mean, where it was a sum; every sentence-transformers mode pools, where all but mean and last-token panicked; and what it cannot do fails at load. - /v1/embeddings takes `dimensions`: the leading values, normalized again, as OpenAI and truncate_dim do. - A multi-model entry takes `revision`, so a deployment pins the weights its indexes were built from. Claude-Session: https://claude.ai/code/session_01SCPMGSQsLNF2YtHFL4bwyp
1.0 KiB
1.0 KiB
Embeddings Overview
Hanzo Engine can load embedding models alongside chat, multimodal, diffusion, and speech workloads. Embedding models produce dense vector representations that you can use for similarity search, clustering, reranking, and other semantic tasks.
Supported models
| Model | Notes | Documentation |
|---|---|---|
| EmbeddingGemma | Google’s multilingual embedding model. | EMBEDDINGGEMMA.md |
| EmbeddingGemma 2 | Google’s multilingual, Matryoshka (768/512/256/128) successor; text served. | EMBEDDINGGEMMA2.md |
| Qwen3 Embedding | Qwen’s general-purpose embedding encoder. | QWEN3_EMBEDDING.md |
Have another embedding model you would like supported? Open an issue with the model ID and configuration.
Usage overview
- Choose a model from the table above.
- Load it through one of our APIs:
- CLI/HTTP
- Python
- Rust
Detailed examples for each model live in their dedicated documentation pages.