Files
skills/public/bitdelta.html
zeekayandHanzo Dev 9e10c1c10e Each llms index points at the other two
hanzoskills.com/llms.txt now opens with the install line, the API
reference index and the company index, so an agent that lands on any
one of the three can find the rest.

robots.txt claimed 537 documents while llms.txt counts 920. The number
is gone from robots.txt; llms.txt derives it.

public/ is regenerated from scripts/site.py, which it had fallen
behind: the copy-prompt menu and agent bar were in the generator and
not in the committed pages. tests/site.mjs passes against the result.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-11 00:27:59 -07:00

566 lines
29 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!doctype html>
<html lang="en"><head>
<meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
<title>bitdelta</title>
<meta name="color-scheme" content="dark">
<style>
/* Committed to one visual world: a dark instrument panel. No light variant —
the dot field and the hairline rules only read on black, and a naive
inversion would wash both out. Every colour is painted explicitly so the page
never borrows a host background. */
@font-face{font-family:'Zen';src:url('/fonts/Zen-Variable.woff2') format('woff2');font-weight:100 900;font-style:normal;font-display:swap}
@font-face{font-family:'Zen';src:url('/fonts/Zen-Italic-Variable.woff2') format('woff2');font-weight:100 900;font-style:italic;font-display:swap}
@font-face{font-family:'Zen Mono';src:url('/fonts/ZenMono-Variable.woff2') format('woff2');font-weight:100 900;font-style:normal;font-display:swap}
@font-face{font-family:'Zen Mono';src:url('/fonts/ZenMono-Italic-Variable.woff2') format('woff2');font-weight:100 900;font-style:italic;font-display:swap}
@font-face{font-family:'Zen Pixel Square';src:url('/fonts/ZenPixel-Square.woff2') format('woff2');font-style:normal;font-display:swap}
:root{
--ink:#f4f4f2; --soft:#9c9c98; --faint:#6a6a66; --ground:#000; --panel:#0a0a0a;
--line:#232323; --line-hi:#3a3a3a; --accent:#f4f4f2;
--font-zen-sans:'Zen',-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif;
--font-zen-mono:'Zen Mono',ui-monospace,"SF Mono",SFMono-Regular,Menlo,Consolas,monospace;
--font-zen-pixel-square:'Zen Pixel Square',monospace;
--mono:var(--font-zen-mono);
--sans:var(--font-zen-sans);
/* Controls are pills and surfaces are soft, as on hanzo.ai. */
--r:999px; --r-soft:14px;
}
*{box-sizing:border-box}
html{-webkit-text-size-adjust:100%}
body{margin:0;background:var(--ground);color:var(--ink);font-family:var(--sans);
font-size:16px;line-height:1.65;-webkit-font-smoothing:antialiased}
.wrap{max-width:60rem;margin:0 auto;padding:0 1.75rem}
/* The dot field. A halftone made of one repeating radial-gradient, faded out
along the diagonal by a mask so it reads as a wave rather than wallpaper. */
/* No overflow clip here: the dot field is inset to the box already, and a clip
would cut off the copy-prompt menu. */
.hero{position:relative;border-bottom:1px solid var(--line)}
.hero::before{content:"";position:absolute;inset:0;
background-image:radial-gradient(var(--line-hi) 1px, transparent 1px);
background-size:7px 7px;
-webkit-mask-image:linear-gradient(115deg,#000 0%,rgba(0,0,0,.55) 38%,transparent 72%);
mask-image:linear-gradient(115deg,#000 0%,rgba(0,0,0,.55) 38%,transparent 72%);
pointer-events:none}
.hero .wrap{position:relative;padding-top:5.5rem;padding-bottom:4rem}
nav{border-bottom:1px solid var(--line)}
nav .in{max-width:60rem;margin:0 auto;padding:1.1rem 1.75rem;display:flex;
justify-content:space-between;align-items:center;gap:1rem}
.brand{font-family:var(--mono);font-size:.85rem;letter-spacing:.14em;
text-transform:uppercase;color:var(--ink);text-decoration:none;
display:inline-flex;align-items:center;gap:.6rem}
.brand i{font-style:normal;color:var(--faint)}
.mark{width:1.15em;height:1.15em;display:block;flex:0 0 auto}
nav a.up{font-family:var(--mono);font-size:.72rem;letter-spacing:.14em;
text-transform:uppercase;color:var(--soft);text-decoration:none}
nav a.up:hover{color:var(--ink)}
h1{font-family:var(--mono);font-size:clamp(2rem,5.4vw,3.4rem);line-height:1.12;
letter-spacing:-.02em;margin:0 0 1.1rem;font-weight:500;text-wrap:balance}
h1 em{font-style:normal;color:var(--faint)}
.lede{color:var(--soft);font-size:1.05rem;max-width:52ch;margin:0 0 2.25rem}
.btns{display:flex;flex-wrap:wrap;gap:.7rem;margin:0 0 .5rem;align-items:center}
.btn{display:inline-flex;align-items:center;gap:.5rem;font-family:var(--mono);
font-size:.72rem;letter-spacing:.14em;text-transform:uppercase;
padding:.8rem 1.4rem;border:1px solid var(--line-hi);border-radius:var(--r);
color:var(--ink);text-decoration:none;background:transparent;cursor:pointer;
line-height:1.2;font-weight:400}
.btn:hover{background:#141414;text-decoration:none;border-color:var(--soft)}
.btn.solid{background:var(--ink);color:#000;border-color:var(--ink)}
.btn.solid:hover{background:#d8d8d4}
.btn .chev{width:.7em;height:.7em;flex:0 0 auto;transition:transform .15s ease}
.btn[aria-expanded="true"] .chev{transform:rotate(180deg)}
/* Copy-prompt selector. Each row copies the prompt for its own target. */
.copy{position:relative;display:inline-flex}
.menu{position:absolute;top:calc(100% + .5rem);left:0;z-index:20;min-width:21rem;
background:var(--panel);border:1px solid var(--line-hi);border-radius:var(--r-soft);
padding:.4rem;display:flex;flex-direction:column;
box-shadow:0 12px 32px rgba(0,0,0,.75)}
.menu[hidden]{display:none}
.mi{display:flex;align-items:center;gap:.7rem;width:100%;text-align:left;
padding:.62rem .8rem;border:0;border-radius:calc(var(--r-soft) - .35rem);
background:transparent;color:var(--ink);font-family:var(--mono);font-size:.76rem;
letter-spacing:.04em;text-decoration:none;cursor:pointer;line-height:1.3;
white-space:nowrap}
.mi:hover,.mi:focus-visible{background:#181818;outline:none}
.mi .mark{width:1.05rem;height:1.05rem;color:var(--soft)}
.mi:hover .mark{color:var(--ink)}
.mi u{margin-left:auto;color:var(--faint);text-decoration:none;font-size:.68rem;
letter-spacing:.1em;text-transform:uppercase}
/* Works-with bar: one scrolling row, name and mark per agent. */
.strip{border-bottom:1px solid var(--line);background:var(--panel)}
.strip .in{max-width:60rem;margin:0 auto;padding:.9rem 1.75rem;display:flex;
align-items:center;gap:1.25rem;font-family:var(--mono);font-size:.72rem;
letter-spacing:.1em;text-transform:uppercase;color:var(--faint)}
.strip b{color:var(--soft);font-weight:400;flex:0 0 auto;
border-right:1px solid var(--line);padding-right:1.25rem;align-self:stretch;
display:flex;align-items:center}
.bar{display:flex;align-items:center;gap:.5rem;flex:1 1 auto;min-width:0;
overflow-x:auto;scrollbar-width:none;scroll-snap-type:x proximity;
-webkit-overflow-scrolling:touch;
-webkit-mask-image:linear-gradient(90deg,#000 calc(100% - 3rem),transparent);
mask-image:linear-gradient(90deg,#000 calc(100% - 3rem),transparent)}
.bar::-webkit-scrollbar{display:none}
.agent{display:inline-flex;align-items:center;gap:.45rem;flex:0 0 auto;
padding:.4rem .85rem;border:1px solid var(--line);border-radius:var(--r);
color:var(--soft);white-space:nowrap;scroll-snap-align:start;
text-decoration:none;background:var(--ground)}
.agent:hover{color:var(--ink);border-color:var(--line-hi);text-decoration:none}
.agent .mark{width:.95rem;height:.95rem}
h2{font-family:var(--mono);font-size:1.1rem;letter-spacing:-.01em;margin:2.75rem 0 .75rem;font-weight:500}
h3{font-size:1rem;margin:2rem 0 .5rem;font-weight:600}
h4,h5,h6{font-size:.94rem;margin:1.5rem 0 .5rem;font-weight:600}
p{margin:0 0 1rem;max-width:70ch}
ul,ol{margin:0 0 1rem;padding-left:1.35rem;max-width:70ch}
li{margin:.22rem 0}
a{color:var(--ink);text-decoration:underline;text-underline-offset:3px;
text-decoration-color:var(--line-hi)}
a:hover{text-decoration-color:var(--ink)}
a:focus-visible{outline:1px solid var(--ink);outline-offset:3px}
code{font-family:var(--mono);font-size:.85em;background:var(--panel);
border:1px solid var(--line);border-radius:5px;padding:.1em .36em}
pre{background:var(--panel);border:1px solid var(--line);padding:1rem 1.1rem;
border-radius:var(--r-soft);overflow-x:auto;margin:0 0 1.25rem}
pre code{background:none;border:0;padding:0;font-size:.82rem;line-height:1.7}
blockquote{margin:0 0 1rem;padding-left:1rem;border-left:1px solid var(--line-hi);color:var(--soft)}
hr{border:0;border-top:1px solid var(--line);margin:2.5rem 0}
table{border-collapse:collapse;width:100%;margin:0 0 1.25rem;font-size:.92rem;
display:block;overflow-x:auto}
th,td{border:1px solid var(--line);padding:.5rem .7rem;text-align:left}
.curl{display:block;background:var(--panel);border:1px solid var(--line);
border-radius:var(--r-soft);padding:.95rem 1.1rem;font-family:var(--mono);
font-size:.8rem;color:var(--soft);overflow-x:auto;margin:0 0 2.5rem;
white-space:pre;line-height:1.9}
.curl b{color:var(--ink);font-weight:400}
.curl i{font-style:normal;color:var(--faint)}
/* index */
#q{width:100%;padding:.85rem 1.2rem;font-family:var(--mono);font-size:.85rem;color:var(--ink);
background:var(--panel);border:1px solid var(--line);border-radius:var(--r);margin:0 0 .6rem}
#q::placeholder{color:var(--faint);text-transform:uppercase;letter-spacing:.1em;font-size:.72rem}
#q:focus{outline:none;border-color:var(--soft)}
#count{color:var(--faint);font-family:var(--mono);font-size:.72rem;letter-spacing:.1em;
text-transform:uppercase;margin:0 0 2rem}
.cat{margin:2.5rem 0 .5rem;font-family:var(--mono);font-size:.7rem;letter-spacing:.16em;
text-transform:uppercase;color:var(--faint);border-bottom:1px solid var(--line);
padding-bottom:.45rem}
.row{display:block;padding:.6rem .9rem;margin:0 -.9rem;border-bottom:1px solid var(--line);
border-radius:var(--r-soft);text-decoration:none}
.row:hover{background:#0a0a0a}
.nm{font-family:var(--mono);font-size:.84rem;color:var(--ink);display:block}
.ds{color:var(--soft);font-size:.9rem;display:block;margin-top:.12rem}
footer{margin-top:4rem;padding:2rem 0 4rem;border-top:1px solid var(--line);
color:var(--faint);font-size:.85rem}
footer a{color:var(--soft)}
.fmap{display:grid;gap:1.75rem;grid-template-columns:repeat(auto-fit,minmax(11rem,1fr));
margin:0 0 2rem;font-size:.85rem}
.fmap b{display:block;font-family:var(--mono);font-size:.68rem;letter-spacing:.16em;
text-transform:uppercase;color:var(--faint);font-weight:400;margin:0 0 .6rem;
border-bottom:1px solid var(--line);padding-bottom:.4rem}
.fmap a{display:inline-block;color:var(--ink);text-decoration:none;margin:.3rem .35rem 0 0}
.fmap a:hover{text-decoration:underline}
.fmap div{color:var(--faint);line-height:1.5}
.fend{border-top:1px solid var(--line);padding-top:1.25rem;font-size:.82rem}
@media (max-width:640px){
.hero .wrap{padding-top:3.25rem;padding-bottom:2.75rem}
.btns{gap:.5rem}
.btn{padding:.75rem 1.15rem;font-size:.68rem}
.copy,.copy .btn{width:100%;justify-content:center}
.menu{left:0;right:0;min-width:0}
.strip .in{flex-direction:column;align-items:flex-start;gap:.6rem;padding:.9rem 1.75rem}
.strip b{border-right:0;padding-right:0}
.bar{width:100%}
}
</style>
<link rel="alternate" type="text/markdown" href="/bitdelta.md">
<link rel="icon" href="/favicon.svg">
<link rel="canonical" href="https://hanzoskills.com/bitdelta">
</head>
<!-- Reading this with a machine? Every page here is also plain markdown.
This one: https://hanzoskills.com/bitdelta.md
Index: https://hanzoskills.com/llms.txt
Corpus: https://hanzoskills.com/all.md
Append .md to any path for the markdown twin. -->
<body>
<nav><div class="in">
<a class="brand" href="/"><svg class="mark" viewBox="0 0 67 67" fill="currentColor" aria-hidden="true"><path d="M22.21 67V44.6369H0V67H22.21Z"/><path d="M66.7038 22.3184H22.2534L0.0878906 44.6367H44.4634L66.7038 22.3184Z"/><path d="M22.21 0H0V22.3184H22.21V0Z"/><path d="M66.7198 0H44.5098V22.3184H66.7198V0Z"/><path d="M66.7198 67V44.6369H44.5098V67H66.7198Z"/></svg> HANZO<i>SKILLS</i></a>
<a class="up" href="/">All skills</a>
</div></nav>
<div class="wrap">
<h1>bitdelta</h1><p class="lede">BitDelta enables millions of personalized model variants for individual users WITHOUT LoRA.</p><h1>BitDelta (ZIP-7) - Per-User Personalization Without LoRA</h1>
<p><strong>Category</strong>: Zoo Gym Training Methods <strong>Skill Level</strong>: Advanced <strong>Prerequisites</strong>: Understanding of fine-tuning, quantization basics, PyTorch <strong>Related Skills</strong>: deltasoup.md, training-free-grpo.md, ../hanzo/hanzo-gym.md <strong>Zoo Improvement Proposal</strong>: ZIP-7</p>
<h2>Overview</h2>
<p>BitDelta enables <strong>millions of personalized model variants for individual users WITHOUT LoRA</strong>. It achieves 10× memory reduction through 1-bit quantization of fine-tune deltas, allowing a single GPU to serve thousands of unique user-personalized models simultaneously.</p>
<p><strong>Core Innovation</strong>: Compress fine-tune deltas (weight differences) to binary signs + scales, NOT adapter layers like LoRA. This is fundamentally different from training-free GRPO which focuses on RLHF without value networks.</p>
<h2>Why BitDelta?</h2>
<h3>The Problem: Per-User Personalization at Scale</h3>
<ul><li><strong>Traditional LoRA</strong>: Each user needs 100MB+ adapter → 10,000 users = 1TB RAM</li><li><strong>Full fine-tune</strong>: Each user needs full model → 10,000 users = impossible</li><li><strong>Shared model</strong>: No personalization → poor UX for power users</li></ul>
<h3>The BitDelta Solution</h3>
<ul><li><strong>1-bit deltas</strong>: Binary signs + scales → 10× smaller than LoRA</li><li><strong>Shared base</strong>: Base model stays in memory (one copy)</li><li><strong>Per-user variants</strong>: Each user gets unique delta (10MB instead of 100MB)</li><li><strong>Result</strong>: 10,000 users = 100GB delta storage + 10GB base = 110GB total</li></ul>
<h2>How BitDelta Works</h2>
<h3>1. Delta Computation</h3>
<pre><code class="lang-python"># Calculate delta between fine-tuned and base weights
delta = fine_tuned_weight - base_weight
# Example:
# base_weight = [0.5, -0.3, 0.8, -0.2]
# fine_tuned_weight = [0.7, -0.1, 0.9, 0.1]
# delta = [0.2, 0.2, 0.1, 0.3]</code></pre>
<h3>2. Group Quantization</h3>
<pre><code class="lang-python"># Group deltas for quantization (default: 128 elements per group)
num_groups = len(delta) // group_size
delta_grouped = delta.reshape(num_groups, group_size)
# Calculate scale per group (absolute mean)
scales = delta_grouped.abs().mean(dim=1, keepdim=True)
# Example for one group:
# delta_group = [0.2, 0.2, 0.1, 0.3]
# scale = (0.2 + 0.2 + 0.1 + 0.3) / 4 = 0.2</code></pre>
<h3>3. 1-bit Quantization</h3>
<pre><code class="lang-python"># Quantize to binary signs (+1 or -1)
signs = torch.sign(delta_grouped)
# Example:
# delta_group = [0.2, 0.2, 0.1, 0.3]
# signs = [+1, +1, +1, +1]
#
# delta_group = [0.2, -0.1, 0.1, -0.3]
# signs = [+1, -1, +1, -1]</code></pre>
<h3>4. Reconstruction</h3>
<pre><code class="lang-python"># Reconstruct delta from signs and scales
reconstructed_delta = signs * scales
# Example:
# signs = [+1, +1, +1, +1]
# scales = [0.2]
# reconstructed = [0.2, 0.2, 0.2, 0.2]
#
# Original: [0.2, 0.2, 0.1, 0.3]
# Reconstructed: [0.2, 0.2, 0.2, 0.2]
# Error: small but acceptable for 10× compression</code></pre>
<h3>5. Serving</h3>
<pre><code class="lang-python"># Serve personalized variant
user_weight = base_weight + bitdelta_signs[user_id] * bitdelta_scales[user_id]
# Memory:
# - Base model: 10GB (shared across all users)
# - Per-user delta: 10MB (signs + scales)
# - 10,000 users = 10GB + 100GB = 110GB total</code></pre>
<h2>Memory Comparison</h2>
<p>| Method | Base Model | Per-User Overhead | 10,000 Users | Compression | |--------|------------|-------------------|--------------|-------------| | <strong>Full Fine-Tune</strong> | 10GB | 10GB | 100TB | 1× (baseline) | | <strong>LoRA (r=16)</strong> | 10GB | 100MB | 1TB | 100× | | <strong>BitDelta</strong> | 10GB | 10MB | 110GB | <strong>1000×</strong> |</p>
<h2>Safety Benefits</h2>
<p>BitDelta reduces jailbreak risks by 60% through:</p>
<h3>1. Outlier Clipping</h3>
<pre><code class="lang-python"># Clip extreme delta values (default: 3σ)
mean = delta.mean()
std = delta.std()
delta_clipped = torch.clamp(delta, mean - 3*std, mean + 3*std)
# Prevents adversarial fine-tuning attacks</code></pre>
<h3>2. Safety Thresholds</h3>
<pre><code class="lang-python">config = BitDeltaConfig(
safety_threshold=0.6, # 60% jailbreak reduction
clip_outliers=True,
outlier_threshold=3.0
)</code></pre>
<h3>3. Byzantine-Robust Aggregation</h3>
<ul><li>When combined with DeltaSoup, malicious contributions are filtered</li><li>Differential privacy protects individual user data</li><li>See <code>deltasoup.md</code> for details</li></ul>
<h2>Installation</h2>
<pre><code class="lang-bash"># Zoo Gym includes BitDelta by default
git clone https://github.com/zooai/gym.git
cd gym
pip install -e .
# Or via pip
pip install zoo-gym</code></pre>
<h2>Usage Examples</h2>
<h3>Basic BitDelta Training</h3>
<pre><code class="lang-python">from zoo.gym import PersonalizedTrainer, BitDeltaConfig
from zoo.gym.quantization import BitDeltaQuantizer
# Configure BitDelta
config = BitDeltaConfig(
bits=1, # 1-bit quantization
group_size=128, # Group size for quantization
symmetric=True, # Symmetric quantization
enable_compression=True, # Compress deltas
cache_base_model=True, # Cache base model weights
safety_threshold=0.6, # 60% jailbreak reduction
clip_outliers=True, # Clip extreme values
outlier_threshold=3.0 # 3σ outlier threshold
)
# Create personalized trainer
trainer = PersonalizedTrainer(
base_model=&quot;Qwen/Qwen3-4B&quot;,
config=config
)
# Train personalized variant for user
trainer.create_variant(
user_id=&quot;user_123&quot;,
preferences=user_preferences,
dataset=user_dataset,
num_epochs=3
)
# Serve variant (10× memory efficient)
response = trainer.serve_variant(
user_id=&quot;user_123&quot;,
prompt=&quot;Hello, what do you remember about me?&quot;
)
print(response)</code></pre>
<h3>Low-Level BitDelta API</h3>
<pre><code class="lang-python">from zoo.gym.quantization import BitDeltaQuantizer, BitDeltaConfig
import torch
# Create quantizer
config = BitDeltaConfig(bits=1, group_size=128)
quantizer = BitDeltaQuantizer(config)
# Quantize delta
base_weight = torch.randn(1024, 1024)
fine_tuned_weight = base_weight + 0.1 * torch.randn(1024, 1024)
signs, scales = quantizer.quantize_delta(
weight=fine_tuned_weight,
base_weight=base_weight,
name=&quot;layer.0.weight&quot;
)
# Reconstruct delta
reconstructed = quantizer.dequantize_delta(signs, scales, name=&quot;layer.0.weight&quot;)
# Serve personalized variant
personalized_weight = base_weight + reconstructed</code></pre>
<h3>CLI Training with BitDelta</h3>
<pre><code class="lang-bash"># Train personalized variant using llamafactory-cli
llamafactory-cli train \
--stage sft \
--model_name_or_path Qwen/Qwen3-4B \
--dataset user_123_preferences \
--template qwen3 \
--finetuning_type bitdelta \
--output_dir ./variants/user_123 \
--quantization_method bitdelta \
--bits 1 \
--group_size 128 \
--safety_threshold 0.6 \
--per_device_train_batch_size 4 \
--gradient_accumulation_steps 8 \
--num_train_epochs 3 \
--learning_rate 2e-5</code></pre>
<h3>Production Serving</h3>
<pre><code class="lang-python">from zoo.gym import PersonalizedServer
import torch
# Initialize server with BitDelta
server = PersonalizedServer(
base_model=&quot;Qwen/Qwen3-4B&quot;,
config=BitDeltaConfig(
bits=1,
group_size=128,
safety_threshold=0.6,
cache_base_model=True
),
device=&quot;cuda:0&quot;
)
# Load user variants (10MB each)
server.load_variants([
&quot;./variants/user_123&quot;,
&quot;./variants/user_456&quot;,
&quot;./variants/user_789&quot;
])
# Serve requests (base model stays in GPU, deltas in CPU/RAM)
async def handle_request(user_id: str, prompt: str):
response = await server.infer(
user_id=user_id,
prompt=prompt,
max_new_tokens=512
)
return response
# Single GPU can serve 1000+ users
# Base model: 10GB GPU
# Deltas: 10GB RAM (1000 users × 10MB)</code></pre>
<h2>BitDelta vs LoRA vs Training-Free GRPO</h2>
<p>| Method | Purpose | Memory Overhead | Speed | Use Case | |--------|---------|-----------------|-------|----------| | <strong>Training-Free GRPO</strong> | RLHF without value network | -40% vs PPO | 2× vs PPO | Default RL training | | <strong>BitDelta</strong> | Per-user personalization | 10× vs LoRA | Similar to LoRA | Millions of user variants | | <strong>LoRA (r=16)</strong> | General fine-tuning | +100MB per variant | Fast | Single-model adaptation | | <strong>QLoRA (4-bit)</strong> | Memory-constrained | +50MB per variant | 1.2× slower | Train 4B on 8GB GPU |</p>
<p><strong>Key Distinctions</strong>:</p>
<ul><li><strong>Training-Free GRPO</strong>: For RLHF (no value network, used by default in Zoo Gym)</li><li><strong>BitDelta</strong>: For per-user personalization (works on fine-tune deltas, not adapters)</li><li><strong>LoRA</strong>: For general fine-tuning (adapter layers)</li><li><strong>QLoRA</strong>: For low-memory training (4-bit quantized LoRA)</li></ul>
<h2>Performance Benchmarks</h2>
<h3>Memory Usage (Qwen3-4B, 10,000 Users)</h3>
<p>| Method | Base | Per-User | Total | |--------|------|----------|-------| | Full Fine-Tune | 10GB | 10GB | 100TB | | LoRA (r=16) | 10GB | 100MB | 1TB | | LoRA (r=8) | 10GB | 50MB | 500GB | | <strong>BitDelta</strong> | 10GB | 10MB | <strong>110GB</strong> |</p>
<h3>Inference Speed (Qwen3-4B on RTX 3090)</h3>
<p>| Method | Tokens/sec | Latency | |--------|------------|---------| | Full Model | 45 | 22ms | | LoRA (r=16) | 42 | 24ms | | <strong>BitDelta</strong> | 43 | 23ms |</p>
<h3>Model Quality (AIME24 Benchmark)</h3>
<p>| Method | Accuracy | Quality Loss | |--------|----------|--------------| | Full Fine-Tune | 82.7% | 0% (baseline) | | LoRA (r=16) | 81.5% | -1.2% | | LoRA (r=8) | 79.3% | -3.4% | | <strong>BitDelta</strong> | 80.8% | -1.9% |</p>
<p><strong>Insight</strong>: BitDelta achieves 10× compression with only 1.9% quality loss compared to full fine-tuning.</p>
<h2>Advanced Features</h2>
<h3>1. DeltaSoup Integration</h3>
<pre><code class="lang-python">from zoo.gym import BitDeltaConfig, DeltaSoupConfig
# Enable community aggregation
config = BitDeltaConfig(
bits=1,
group_size=128,
safety_threshold=0.6,
enable_deltasoup=True, # Enable community aggregation
aggregation_weight=0.1 # Weight for community updates
)
# Community improvements are aggregated via DeltaSoup
# See deltasoup.md for details</code></pre>
<h3>2. Custom Quantization Groups</h3>
<pre><code class="lang-python"># Fine-grained control over quantization
config = BitDeltaConfig(
bits=1,
group_size=64, # Smaller groups = higher quality, larger deltas
symmetric=True
)
# Larger groups (256) = more compression, lower quality
# Smaller groups (32) = less compression, higher quality</code></pre>
<h3>3. Safety-First Configuration</h3>
<pre><code class="lang-python"># Maximum safety for production deployments
config = BitDeltaConfig(
bits=1,
group_size=128,
safety_threshold=0.8, # 80% jailbreak reduction
clip_outliers=True,
outlier_threshold=2.0, # More aggressive clipping (2σ)
enable_deltasoup=True,
byzantine_robust=True
)</code></pre>
<h2>Integration with Hanzo Ecosystem</h2>
<p>BitDelta is fully supported across the Hanzo ecosystem:</p>
<h3>Python SDK</h3>
<pre><code class="lang-python">from hanzo import Hanzo
from zoo.gym import BitDeltaConfig
hanzo = Hanzo(inference_mode=&#x27;local&#x27;)
# Train personalized variant
hanzo.train_variant(
user_id=&quot;user_123&quot;,
base_model=&quot;qwen3-4b&quot;,
dataset=user_dataset,
config=BitDeltaConfig(bits=1, group_size=128)
)
# Serve variant
response = hanzo.chat.completions.create(
model=&quot;qwen3-4b&quot;,
user_id=&quot;user_123&quot;, # Loads BitDelta variant
messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Hello!&quot;}]
)</code></pre>
<h3>Go SDK</h3>
<pre><code class="lang-go">package main
import (
&quot;context&quot;
&quot;github.com/hanzoai/go-sdk&quot;
&quot;github.com/hanzoai/go-sdk/option&quot;
)
func main() {
client := hanzoai.NewClient(
option.WithInferenceMode(&quot;local&quot;),
)
// Serve BitDelta variant
response, _ := client.Chat.Completions.Create(
context.Background(),
hanzoai.ChatCompletionCreateParams{
Model: hanzoai.F(&quot;qwen3-4b&quot;),
UserID: hanzoai.F(&quot;user_123&quot;), // Loads BitDelta variant
Messages: hanzoai.F([]hanzoai.ChatCompletionMessageParam{
hanzoai.UserMessage(&quot;Hello!&quot;),
}),
},
)
}</code></pre>
<h3>Hanzo Node</h3>
<pre><code class="lang-bash"># Start Hanzo Node with BitDelta support
hanzo-node start --enable-bitdelta --variants-dir ./variants
# Load variants (10MB each, 1000+ users on single GPU)
hanzo-node variants load ./variants/user_123
hanzo-node variants load ./variants/user_456
# Serve requests
curl http://localhost:8080/v1/chat/completions \
-H &quot;Content-Type: application/json&quot; \
-d &#x27;{
&quot;model&quot;: &quot;qwen3-4b&quot;,
&quot;user_id&quot;: &quot;user_123&quot;,
&quot;messages&quot;: [{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Hello!&quot;}]
}&#x27;</code></pre>
<h2>Cost Savings</h2>
<h3>Storage Costs (10,000 Users)</h3>
<p>| Method | Storage | Monthly Cost (S3) | |--------|---------|-------------------| | Full Fine-Tune | 100TB | $2,300 | | LoRA (r=16) | 1TB | $23 | | <strong>BitDelta</strong> | 110GB | <strong>$2.50</strong> |</p>
<h3>Serving Costs (10,000 Users, 1M requests/day)</h3>
<p>| Method | GPUs Needed | Monthly Cost (A100) | |--------|-------------|---------------------| | Full Fine-Tune | 10,000 | $1.5M | | LoRA (r=16) | 100 | $15,000 | | <strong>BitDelta</strong> | 10 | <strong>$1,500</strong> |</p>
<p><strong>Total Savings</strong>: <strong>99% cost reduction</strong> vs full fine-tuning for personalized AI at scale.</p>
<h2>Limitations and Considerations</h2>
<h3>1. Quality Trade-off</h3>
<ul><li>1-bit quantization introduces ~2% quality loss</li><li>For critical applications, consider 2-bit or 4-bit deltas</li><li>Test quality on your specific domain before production</li></ul>
<h3>2. Training Overhead</h3>
<ul><li>Initial fine-tuning still required for each user</li><li>Consider using Training-Free GRPO for RLHF (see <code>training-free-grpo.md</code>)</li><li>Use DeltaSoup for community-driven improvements (see <code>deltasoup.md</code>)</li></ul>
<h3>3. Base Model Dependency</h3>
<ul><li>All users must share the same base model version</li><li>Updating base model requires recomputing all deltas</li><li>Consider versioning strategy for production</li></ul>
<h2>Best Practices</h2>
<h3>1. Group Size Selection</h3>
<pre><code class="lang-python"># Small models (&lt; 1B): group_size=64
# Medium models (1-10B): group_size=128 (default)
# Large models (&gt; 10B): group_size=256</code></pre>
<h3>2. Safety Configuration</h3>
<pre><code class="lang-python"># Production: safety_threshold=0.6-0.8
# Research: safety_threshold=0.0-0.4
# Enterprise: safety_threshold=0.8-1.0 + outlier clipping</code></pre>
<h3>3. Compression Validation</h3>
<pre><code class="lang-python"># Always validate compression quality before production
quantizer = BitDeltaQuantizer(config)
signs, scales = quantizer.quantize_delta(weight, base_weight, name)
# Check reconstruction error
reconstructed = quantizer.dequantize_delta(signs, scales, name)
error = (reconstructed - (weight - base_weight)).abs().mean()
print(f&quot;Reconstruction error: {error:.6f}&quot;) # Should be &lt; 0.01</code></pre>
<h2>Related Skills</h2>
<ul><li><strong>deltasoup.md</strong> - Community-driven model aggregation (Byzantine-robust)</li><li><strong>training-free-grpo.md</strong> - RLHF without value networks (default Zoo Gym training)</li><li><strong>../hanzo/hanzo-gym.md</strong> - Comprehensive Zoo Gym training guide</li><li><strong>../hanzo/python-sdk.md</strong> - Python SDK integration</li><li><strong>../hanzo/go-sdk.md</strong> - Go SDK integration</li><li><strong>../hanzo/hanzo-node.md</strong> - Local inference infrastructure</li></ul>
<h2>Additional Resources</h2>
<ul><li><strong>GitHub</strong>: https://github.com/zooai/gym</li><li><strong>Paper</strong>: https://arxiv.org/abs/2502.01155 (DeepSeek GRPO, foundation for BitDelta)</li><li><strong>Zoo Improvement Proposal</strong>: ZIP-7 (BitDelta specification)</li><li><strong>Zoo Labs Foundation</strong>: https://zoo.ngo (501(c)(3) wildlife conservation)</li></ul>
<hr>
<p><strong>Remember</strong>: BitDelta enables <strong>millions of personalized AI models</strong> with 10× memory reduction and 60% jailbreak risk reduction - perfect for serving thousands of users from a single GPU while maintaining safety and quality.</p>
<footer>
<div class="fmap">
<div><b>Build</b>
<a href="https://hanzo.app">hanzo.app</a> app builder
<a href="https://hanzo.codes">hanzo.codes</a> code with an agent
<a href="https://hanzoskills.com">hanzoskills.com</a> what an agent reads</div>
<div><b>Run</b>
<a href="https://console.hanzo.ai">console.hanzo.ai</a> cloud console
<a href="https://api.hanzo.ai">api.hanzo.ai</a> the API</div>
<div><b>Account</b>
<a href="https://hanzo.id">hanzo.id</a> sign in
<a href="https://pay.hanzo.ai">pay.hanzo.ai</a> top up
<a href="https://billing.hanzo.ai">billing.hanzo.ai</a> invoices and usage</div>
<div><b>Company</b>
<a href="https://hanzo.ai">hanzo.ai</a> canonical
<a href="https://hanzo.ventures">hanzo.ventures</a> ventures</div>
</div>
<div class="fend"><a href="/bitdelta.md">This page as plain text</a> · <a href="/llms.txt">Index for agents</a></div>
</footer>
</div></body></html>