mirror of
https://github.com/hanzoai/ml.git
synced 2026-11-20 23:08:29 +00:00
Three-way per file through the rename: base 39355c6c91, theirs the upstream
merge of main with 35d7ae7c (via 7ef6a694d), ours origin/main.
Lands the barrier thread pool, runtime CPU feature dispatch, aarch64 and x86
repacked quantized kernels (indexed_gemv, gemv_fused_shared_lhs), quantized
embedding on CPU/Metal/CUDA, paged flash-attn kernels, cutile fused MoE, the
in-place op aliasing API, parking_lot storage locks, Metal buffer builders and
debug labels, safetensors 0.8, hf-hub 1.0, tokenizers 0.23.
Ours keeps ROCm, Vulkan, wgpu, the IQ/MXFP4/NVFP4/TQ dtypes, and the i-quant
weight/activation block split in matmul. Our backends ride InplaceOpN and the
quantized embedding; Metal get_rows gains the IQ kernels. ug leaves, as upstream.
Co-authored-by: Hanzo Dev <[email protected]>
hanzo-ml-xlm-roberta
This example demonstrates how to use the XLM-RoBERTa model in ML especially known for their use in reranking. It uses the fill-mask task to generate a word for a masked token. And a reranker task to rerank a list of documents for a given query.
Usage
Fill Mask:
cargo run --example xlm-roberta --release -- --task fill-mask --model xlm-roberta-base
Sentence: 0 : Hello I'm a fashion model.
Sentence: 1 : I'm a little boy.
Sentence: 2 : I'm living in berlin.
Reranker:
cargo run --example xlm-roberta --release -- --task reranker --model bge-reranker-base
Ranking Results:
--------------------------------------------------------------------------------
> Rank #4 | Score: 0.0001 | South Korea is a country in East Asia.
> Rank #5 | Score: 0.0000 | There are forests in the mountains.
> Rank #2 | Score: 0.7314 | Pandas look like bears.
> Rank #3 | Score: 0.6948 | There are some animals with black and white fur.
> Rank #1 | Score: 0.9990 | The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear or simply panda, is a bear species endemic to China.
--------------------------------------------------------------------------------
Text-Classification:
cargo run --example xlm-roberta -- --task text-classification --model xlmr-formality-classifier
Formality Scores:
Text 1: "I like you. I love you"
formal: 0.9933
informal: 0.0067
Text 2: "Hey, what's up?"
formal: 0.8812
informal: 0.1188
Text 3: "Siema, co porabiasz?"
formal: 0.9358
informal: 0.0642
Text 4: "I feel deep regret and sadness about the situation in international politics."
formal: 0.9987
informal: 0.0013