Files
zeekay 183a461e09 feat(embeddings): serve EmbeddingGemma 2
google/embeddinggemma-2 (EmbeddingGemma2Model, Apache-2.0) loads as
`embeddinggemma2`, from its config's architecture or by name: the text
backbone of transformers 5.19, with Gemma 4 norms, a value norm with no
scale, attention scale 1, full layers at head dim 512 and one KV head,
an inclusive sliding window, projection-only per-layer embeddings, a
per-layer scalar and the 512 -> 768 projection on every token.  The
vision and audio towers are never read.

Against sentence-transformers 6.1.0 in F32, over 64 texts in 14
languages up to 1,256 tokens, every vector's cosine is at least
0.9999993 at 768, 512, 256 and 128 dimensions, and every token count
matches.

To serve it:

- Pooling reads sentence-transformers 6's config (embedding_dimension,
  pooling_mode) beside the older flags.  Mean pooling is a mean, where
  it was a sum; every sentence-transformers mode pools, where all but
  mean and last-token panicked; and what it cannot do fails at load.
- /v1/embeddings takes `dimensions`: the leading values, normalized
  again, as OpenAI and truncate_dim do.
- A multi-model entry takes `revision`, so a deployment pins the
  weights its indexes were built from.

Claude-Session: https://claude.ai/code/session_01SCPMGSQsLNF2YtHFL4bwyp
2026-10-08 20:17:40 -07:00

1.0 KiB
Raw Permalink Blame History

Embeddings Overview

Hanzo Engine can load embedding models alongside chat, multimodal, diffusion, and speech workloads. Embedding models produce dense vector representations that you can use for similarity search, clustering, reranking, and other semantic tasks.

Supported models

Model Notes Documentation
EmbeddingGemma Google’s multilingual embedding model. EMBEDDINGGEMMA.md
EmbeddingGemma 2 Google’s multilingual, Matryoshka (768/512/256/128) successor; text served. EMBEDDINGGEMMA2.md
Qwen3 Embedding Qwen’s general-purpose embedding encoder. QWEN3_EMBEDDING.md

Have another embedding model you would like supported? Open an issue with the model ID and configuration.

Usage overview

  1. Choose a model from the table above.
  2. Load it through one of our APIs:
    • CLI/HTTP
    • Python
    • Rust

Detailed examples for each model live in their dedicated documentation pages.