mirror of
https://github.com/hanzoai/skills.git
synced 2026-11-19 00:47:36 +00:00
hanzoskills.com/llms.txt now opens with the install line, the API reference index and the company index, so an agent that lands on any one of the three can find the rest. robots.txt claimed 537 documents while llms.txt counts 920. The number is gone from robots.txt; llms.txt derives it. public/ is regenerated from scripts/site.py, which it had fallen behind: the copy-prompt menu and agent bar were in the generator and not in the committed pages. tests/site.mjs passes against the result. Co-authored-by: Hanzo Dev <[email protected]>
550 lines
28 KiB
HTML
550 lines
28 KiB
HTML
<!doctype html>
|
|
<html lang="en"><head>
|
|
<meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
|
|
<title>zenlm</title>
|
|
<meta name="color-scheme" content="dark">
|
|
<style>
|
|
/* Committed to one visual world: a dark instrument panel. No light variant —
|
|
the dot field and the hairline rules only read on black, and a naive
|
|
inversion would wash both out. Every colour is painted explicitly so the page
|
|
never borrows a host background. */
|
|
@font-face{font-family:'Zen';src:url('/fonts/Zen-Variable.woff2') format('woff2');font-weight:100 900;font-style:normal;font-display:swap}
|
|
@font-face{font-family:'Zen';src:url('/fonts/Zen-Italic-Variable.woff2') format('woff2');font-weight:100 900;font-style:italic;font-display:swap}
|
|
@font-face{font-family:'Zen Mono';src:url('/fonts/ZenMono-Variable.woff2') format('woff2');font-weight:100 900;font-style:normal;font-display:swap}
|
|
@font-face{font-family:'Zen Mono';src:url('/fonts/ZenMono-Italic-Variable.woff2') format('woff2');font-weight:100 900;font-style:italic;font-display:swap}
|
|
@font-face{font-family:'Zen Pixel Square';src:url('/fonts/ZenPixel-Square.woff2') format('woff2');font-style:normal;font-display:swap}
|
|
|
|
:root{
|
|
--ink:#f4f4f2; --soft:#9c9c98; --faint:#6a6a66; --ground:#000; --panel:#0a0a0a;
|
|
--line:#232323; --line-hi:#3a3a3a; --accent:#f4f4f2;
|
|
--font-zen-sans:'Zen',-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif;
|
|
--font-zen-mono:'Zen Mono',ui-monospace,"SF Mono",SFMono-Regular,Menlo,Consolas,monospace;
|
|
--font-zen-pixel-square:'Zen Pixel Square',monospace;
|
|
--mono:var(--font-zen-mono);
|
|
--sans:var(--font-zen-sans);
|
|
/* Controls are pills and surfaces are soft, as on hanzo.ai. */
|
|
--r:999px; --r-soft:14px;
|
|
}
|
|
*{box-sizing:border-box}
|
|
html{-webkit-text-size-adjust:100%}
|
|
body{margin:0;background:var(--ground);color:var(--ink);font-family:var(--sans);
|
|
font-size:16px;line-height:1.65;-webkit-font-smoothing:antialiased}
|
|
.wrap{max-width:60rem;margin:0 auto;padding:0 1.75rem}
|
|
|
|
/* The dot field. A halftone made of one repeating radial-gradient, faded out
|
|
along the diagonal by a mask so it reads as a wave rather than wallpaper. */
|
|
/* No overflow clip here: the dot field is inset to the box already, and a clip
|
|
would cut off the copy-prompt menu. */
|
|
.hero{position:relative;border-bottom:1px solid var(--line)}
|
|
.hero::before{content:"";position:absolute;inset:0;
|
|
background-image:radial-gradient(var(--line-hi) 1px, transparent 1px);
|
|
background-size:7px 7px;
|
|
-webkit-mask-image:linear-gradient(115deg,#000 0%,rgba(0,0,0,.55) 38%,transparent 72%);
|
|
mask-image:linear-gradient(115deg,#000 0%,rgba(0,0,0,.55) 38%,transparent 72%);
|
|
pointer-events:none}
|
|
.hero .wrap{position:relative;padding-top:5.5rem;padding-bottom:4rem}
|
|
|
|
nav{border-bottom:1px solid var(--line)}
|
|
nav .in{max-width:60rem;margin:0 auto;padding:1.1rem 1.75rem;display:flex;
|
|
justify-content:space-between;align-items:center;gap:1rem}
|
|
.brand{font-family:var(--mono);font-size:.85rem;letter-spacing:.14em;
|
|
text-transform:uppercase;color:var(--ink);text-decoration:none;
|
|
display:inline-flex;align-items:center;gap:.6rem}
|
|
.brand i{font-style:normal;color:var(--faint)}
|
|
.mark{width:1.15em;height:1.15em;display:block;flex:0 0 auto}
|
|
nav a.up{font-family:var(--mono);font-size:.72rem;letter-spacing:.14em;
|
|
text-transform:uppercase;color:var(--soft);text-decoration:none}
|
|
nav a.up:hover{color:var(--ink)}
|
|
|
|
h1{font-family:var(--mono);font-size:clamp(2rem,5.4vw,3.4rem);line-height:1.12;
|
|
letter-spacing:-.02em;margin:0 0 1.1rem;font-weight:500;text-wrap:balance}
|
|
h1 em{font-style:normal;color:var(--faint)}
|
|
.lede{color:var(--soft);font-size:1.05rem;max-width:52ch;margin:0 0 2.25rem}
|
|
|
|
.btns{display:flex;flex-wrap:wrap;gap:.7rem;margin:0 0 .5rem;align-items:center}
|
|
.btn{display:inline-flex;align-items:center;gap:.5rem;font-family:var(--mono);
|
|
font-size:.72rem;letter-spacing:.14em;text-transform:uppercase;
|
|
padding:.8rem 1.4rem;border:1px solid var(--line-hi);border-radius:var(--r);
|
|
color:var(--ink);text-decoration:none;background:transparent;cursor:pointer;
|
|
line-height:1.2;font-weight:400}
|
|
.btn:hover{background:#141414;text-decoration:none;border-color:var(--soft)}
|
|
.btn.solid{background:var(--ink);color:#000;border-color:var(--ink)}
|
|
.btn.solid:hover{background:#d8d8d4}
|
|
.btn .chev{width:.7em;height:.7em;flex:0 0 auto;transition:transform .15s ease}
|
|
.btn[aria-expanded="true"] .chev{transform:rotate(180deg)}
|
|
|
|
/* Copy-prompt selector. Each row copies the prompt for its own target. */
|
|
.copy{position:relative;display:inline-flex}
|
|
.menu{position:absolute;top:calc(100% + .5rem);left:0;z-index:20;min-width:21rem;
|
|
background:var(--panel);border:1px solid var(--line-hi);border-radius:var(--r-soft);
|
|
padding:.4rem;display:flex;flex-direction:column;
|
|
box-shadow:0 12px 32px rgba(0,0,0,.75)}
|
|
.menu[hidden]{display:none}
|
|
.mi{display:flex;align-items:center;gap:.7rem;width:100%;text-align:left;
|
|
padding:.62rem .8rem;border:0;border-radius:calc(var(--r-soft) - .35rem);
|
|
background:transparent;color:var(--ink);font-family:var(--mono);font-size:.76rem;
|
|
letter-spacing:.04em;text-decoration:none;cursor:pointer;line-height:1.3;
|
|
white-space:nowrap}
|
|
.mi:hover,.mi:focus-visible{background:#181818;outline:none}
|
|
.mi .mark{width:1.05rem;height:1.05rem;color:var(--soft)}
|
|
.mi:hover .mark{color:var(--ink)}
|
|
.mi u{margin-left:auto;color:var(--faint);text-decoration:none;font-size:.68rem;
|
|
letter-spacing:.1em;text-transform:uppercase}
|
|
|
|
/* Works-with bar: one scrolling row, name and mark per agent. */
|
|
.strip{border-bottom:1px solid var(--line);background:var(--panel)}
|
|
.strip .in{max-width:60rem;margin:0 auto;padding:.9rem 1.75rem;display:flex;
|
|
align-items:center;gap:1.25rem;font-family:var(--mono);font-size:.72rem;
|
|
letter-spacing:.1em;text-transform:uppercase;color:var(--faint)}
|
|
.strip b{color:var(--soft);font-weight:400;flex:0 0 auto;
|
|
border-right:1px solid var(--line);padding-right:1.25rem;align-self:stretch;
|
|
display:flex;align-items:center}
|
|
.bar{display:flex;align-items:center;gap:.5rem;flex:1 1 auto;min-width:0;
|
|
overflow-x:auto;scrollbar-width:none;scroll-snap-type:x proximity;
|
|
-webkit-overflow-scrolling:touch;
|
|
-webkit-mask-image:linear-gradient(90deg,#000 calc(100% - 3rem),transparent);
|
|
mask-image:linear-gradient(90deg,#000 calc(100% - 3rem),transparent)}
|
|
.bar::-webkit-scrollbar{display:none}
|
|
.agent{display:inline-flex;align-items:center;gap:.45rem;flex:0 0 auto;
|
|
padding:.4rem .85rem;border:1px solid var(--line);border-radius:var(--r);
|
|
color:var(--soft);white-space:nowrap;scroll-snap-align:start;
|
|
text-decoration:none;background:var(--ground)}
|
|
.agent:hover{color:var(--ink);border-color:var(--line-hi);text-decoration:none}
|
|
.agent .mark{width:.95rem;height:.95rem}
|
|
|
|
h2{font-family:var(--mono);font-size:1.1rem;letter-spacing:-.01em;margin:2.75rem 0 .75rem;font-weight:500}
|
|
h3{font-size:1rem;margin:2rem 0 .5rem;font-weight:600}
|
|
h4,h5,h6{font-size:.94rem;margin:1.5rem 0 .5rem;font-weight:600}
|
|
p{margin:0 0 1rem;max-width:70ch}
|
|
ul,ol{margin:0 0 1rem;padding-left:1.35rem;max-width:70ch}
|
|
li{margin:.22rem 0}
|
|
a{color:var(--ink);text-decoration:underline;text-underline-offset:3px;
|
|
text-decoration-color:var(--line-hi)}
|
|
a:hover{text-decoration-color:var(--ink)}
|
|
a:focus-visible{outline:1px solid var(--ink);outline-offset:3px}
|
|
code{font-family:var(--mono);font-size:.85em;background:var(--panel);
|
|
border:1px solid var(--line);border-radius:5px;padding:.1em .36em}
|
|
pre{background:var(--panel);border:1px solid var(--line);padding:1rem 1.1rem;
|
|
border-radius:var(--r-soft);overflow-x:auto;margin:0 0 1.25rem}
|
|
pre code{background:none;border:0;padding:0;font-size:.82rem;line-height:1.7}
|
|
blockquote{margin:0 0 1rem;padding-left:1rem;border-left:1px solid var(--line-hi);color:var(--soft)}
|
|
hr{border:0;border-top:1px solid var(--line);margin:2.5rem 0}
|
|
table{border-collapse:collapse;width:100%;margin:0 0 1.25rem;font-size:.92rem;
|
|
display:block;overflow-x:auto}
|
|
th,td{border:1px solid var(--line);padding:.5rem .7rem;text-align:left}
|
|
|
|
.curl{display:block;background:var(--panel);border:1px solid var(--line);
|
|
border-radius:var(--r-soft);padding:.95rem 1.1rem;font-family:var(--mono);
|
|
font-size:.8rem;color:var(--soft);overflow-x:auto;margin:0 0 2.5rem;
|
|
white-space:pre;line-height:1.9}
|
|
.curl b{color:var(--ink);font-weight:400}
|
|
.curl i{font-style:normal;color:var(--faint)}
|
|
|
|
/* index */
|
|
#q{width:100%;padding:.85rem 1.2rem;font-family:var(--mono);font-size:.85rem;color:var(--ink);
|
|
background:var(--panel);border:1px solid var(--line);border-radius:var(--r);margin:0 0 .6rem}
|
|
#q::placeholder{color:var(--faint);text-transform:uppercase;letter-spacing:.1em;font-size:.72rem}
|
|
#q:focus{outline:none;border-color:var(--soft)}
|
|
#count{color:var(--faint);font-family:var(--mono);font-size:.72rem;letter-spacing:.1em;
|
|
text-transform:uppercase;margin:0 0 2rem}
|
|
.cat{margin:2.5rem 0 .5rem;font-family:var(--mono);font-size:.7rem;letter-spacing:.16em;
|
|
text-transform:uppercase;color:var(--faint);border-bottom:1px solid var(--line);
|
|
padding-bottom:.45rem}
|
|
.row{display:block;padding:.6rem .9rem;margin:0 -.9rem;border-bottom:1px solid var(--line);
|
|
border-radius:var(--r-soft);text-decoration:none}
|
|
.row:hover{background:#0a0a0a}
|
|
.nm{font-family:var(--mono);font-size:.84rem;color:var(--ink);display:block}
|
|
.ds{color:var(--soft);font-size:.9rem;display:block;margin-top:.12rem}
|
|
footer{margin-top:4rem;padding:2rem 0 4rem;border-top:1px solid var(--line);
|
|
color:var(--faint);font-size:.85rem}
|
|
footer a{color:var(--soft)}
|
|
.fmap{display:grid;gap:1.75rem;grid-template-columns:repeat(auto-fit,minmax(11rem,1fr));
|
|
margin:0 0 2rem;font-size:.85rem}
|
|
.fmap b{display:block;font-family:var(--mono);font-size:.68rem;letter-spacing:.16em;
|
|
text-transform:uppercase;color:var(--faint);font-weight:400;margin:0 0 .6rem;
|
|
border-bottom:1px solid var(--line);padding-bottom:.4rem}
|
|
.fmap a{display:inline-block;color:var(--ink);text-decoration:none;margin:.3rem .35rem 0 0}
|
|
.fmap a:hover{text-decoration:underline}
|
|
.fmap div{color:var(--faint);line-height:1.5}
|
|
.fend{border-top:1px solid var(--line);padding-top:1.25rem;font-size:.82rem}
|
|
|
|
@media (max-width:640px){
|
|
.hero .wrap{padding-top:3.25rem;padding-bottom:2.75rem}
|
|
.btns{gap:.5rem}
|
|
.btn{padding:.75rem 1.15rem;font-size:.68rem}
|
|
.copy,.copy .btn{width:100%;justify-content:center}
|
|
.menu{left:0;right:0;min-width:0}
|
|
.strip .in{flex-direction:column;align-items:flex-start;gap:.6rem;padding:.9rem 1.75rem}
|
|
.strip b{border-right:0;padding-right:0}
|
|
.bar{width:100%}
|
|
}
|
|
</style>
|
|
<link rel="alternate" type="text/markdown" href="/zenlm.md">
|
|
|
|
<link rel="icon" href="/favicon.svg">
|
|
<link rel="canonical" href="https://hanzoskills.com/zenlm">
|
|
</head>
|
|
<!-- Reading this with a machine? Every page here is also plain markdown.
|
|
This one: https://hanzoskills.com/zenlm.md
|
|
Index: https://hanzoskills.com/llms.txt
|
|
Corpus: https://hanzoskills.com/all.md
|
|
Append .md to any path for the markdown twin. -->
|
|
<body>
|
|
<nav><div class="in">
|
|
<a class="brand" href="/"><svg class="mark" viewBox="0 0 67 67" fill="currentColor" aria-hidden="true"><path d="M22.21 67V44.6369H0V67H22.21Z"/><path d="M66.7038 22.3184H22.2534L0.0878906 44.6367H44.4634L66.7038 22.3184Z"/><path d="M22.21 0H0V22.3184H22.21V0Z"/><path d="M66.7198 0H44.5098V22.3184H66.7198V0Z"/><path d="M66.7198 67V44.6369H44.5098V67H66.7198Z"/></svg> HANZO<i>SKILLS</i></a>
|
|
<a class="up" href="/">All skills</a>
|
|
</div></nav>
|
|
|
|
<div class="wrap">
|
|
<h1>zenlm</h1><p class="lede">ZenLM is a groundbreaking collaboration between Hanzo AI(https://hanzo.ai) (Techstars-backed GenAI lab) and Zoo Labs Foundation(https://zoo.ngo) (501(c)(3) environmental non-profit), building AI models that run entirely on your device—no cl</p><h1>ZenLM - Next-Generation Local AI Models</h1>
|
|
<p><strong>Category</strong>: Hanzo Ecosystem <strong>Difficulty</strong>: Beginner <strong>Prerequisites</strong>: Python 3.10+, basic terminal knowledge <strong>Related Skills</strong>: hanzo-gym.md, python-sdk.md, hanzo-desktop.md</p>
|
|
<h2>Overview</h2>
|
|
<p><strong>ZenLM</strong> is a groundbreaking collaboration between <a href="https://hanzo.ai">Hanzo AI</a> (Techstars-backed GenAI lab) and <a href="https://zoo.ngo">Zoo Labs Foundation</a> (501(c)(3) environmental non-profit), building AI models that run entirely on your device—no cloud, no subscriptions, no surveillance.</p>
|
|
<p><strong>Mission</strong>: "Democratize AI while protecting our planet"</p>
|
|
<p><strong>Why ZenLM?</strong></p>
|
|
<ul><li><strong>Ultra-Efficient</strong>: 4B parameters achieving 70B-class performance</li><li><strong>Truly Private</strong>: 100% local processing, your data never leaves your device</li><li><strong>Environmentally Responsible</strong>: 95% less energy than cloud AI</li><li><strong>Free Forever</strong>: Apache 2.0 licensed, no premium tiers or API fees</li></ul>
|
|
<h2>Model Family</h2>
|
|
<p>| Model | Size | Description | Use Case | |-------|------|-------------|----------| | <strong>zen-nano-instruct</strong> | 4B | Ultra-fast instruction following | Chat, coding, Q&A | | <strong>zen-nano-thinking</strong> | 4B | Chain-of-thought reasoning | Math, logic, analysis | | <strong>zen-nano-instruct-4bit</strong> | 1.5GB | Quantized for mobile/edge | iOS, Android, Pi | | <strong>zen-nano-thinking-4bit</strong> | 1.5GB | Quantized reasoning | Mobile reasoning | | <strong>zen-eco</strong> | 4B | Efficient instruct/thinking | Production use | | <strong>zen-agent</strong> | 4B | Tool-calling & function execution | Sub-agents, APIs | | <strong>zen-director</strong> | 5B | Text-to-video generation | Video content | | <strong>zen-musician</strong> | 7B | Music generation with lyrics | Music creation |</p>
|
|
<p><strong>GitHub Organization</strong>: <a href="https://github.com/zenlm">github.com/zenlm</a> <strong>HuggingFace Organization</strong>: <a href="https://huggingface.co/zenlm">huggingface.co/zenlm</a></p>
|
|
<h2>Performance</h2>
|
|
<p>Zen models are optimized for local inference on consumer hardware:</p>
|
|
<ul><li><strong>Memory</strong>: ~3.8 GB for full 4B model, ~1.5 GB quantized (4-bit)</li><li><strong>Speed</strong>: 50+ tokens/sec on Apple Silicon (M1+), 30+ on modern CPUs</li><li><strong>Offline</strong>: No internet required after initial download</li></ul>
|
|
<p>See model cards on HuggingFace for specific benchmark results.</p>
|
|
<h2>Quick Start</h2>
|
|
<h3>Installation: Transformers (Python)</h3>
|
|
<pre><code class="lang-bash">pip install transformers torch</code></pre>
|
|
<pre><code class="lang-python">from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
# Load zen-nano-instruct
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|
"zenlm/zen-nano-instruct",
|
|
torch_dtype="auto",
|
|
device_map="auto"
|
|
)
|
|
tokenizer = AutoTokenizer.from_pretrained("zenlm/zen-nano-instruct")
|
|
|
|
# Generate response
|
|
input_text = "Explain quantum computing in simple terms"
|
|
inputs = tokenizer(input_text, return_tensors="pt").to(model.device)
|
|
outputs = model.generate(**inputs, max_length=200, temperature=0.7)
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True))</code></pre>
|
|
<h3>Installation: MLX (Apple Silicon)</h3>
|
|
<pre><code class="lang-bash">pip install mlx-lm</code></pre>
|
|
<pre><code class="lang-python">from mlx_lm import load, generate
|
|
|
|
# Load zen-nano-instruct (optimized for M-series)
|
|
model, tokenizer = load("zenlm/zen-nano-instruct")
|
|
|
|
# Generate at 50+ tokens/sec on M1 Pro
|
|
response = generate(
|
|
model,
|
|
tokenizer,
|
|
prompt="Write a haiku about AI",
|
|
max_tokens=100,
|
|
temp=0.7
|
|
)
|
|
print(response)</code></pre>
|
|
<h3>Installation: llama.cpp (Universal)</h3>
|
|
<pre><code class="lang-bash"># Install llama.cpp
|
|
git clone https://github.com/ggerganov/llama.cpp
|
|
cd llama.cpp && make
|
|
|
|
# Download GGUF version (1.5GB quantized)
|
|
huggingface-cli download zenlm/zen-nano-instruct-4bit \
|
|
--include "*.gguf" \
|
|
--local-dir ./models
|
|
|
|
# Run inference (50+ tokens/sec on CPU!)
|
|
./llama-cli \
|
|
-m ./models/zen-nano-instruct-Q4_K_M.gguf \
|
|
-p "Your prompt here" \
|
|
-n 512 \
|
|
--temp 0.7</code></pre>
|
|
<h2>Model Details</h2>
|
|
<h3>zen-nano-instruct (4B)</h3>
|
|
<p><strong>Best for</strong>: General chat, coding, question answering</p>
|
|
<pre><code class="lang-python">from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
model = AutoModelForCausalLM.from_pretrained("zenlm/zen-nano-instruct")
|
|
tokenizer = AutoTokenizer.from_pretrained("zenlm/zen-nano-instruct")
|
|
|
|
messages = [
|
|
{"role": "system", "content": "You are a helpful AI assistant."},
|
|
{"role": "user", "content": "Write a Python function to reverse a string"}
|
|
]
|
|
|
|
text = tokenizer.apply_chat_template(messages, tokenize=False)
|
|
inputs = tokenizer(text, return_tensors="pt")
|
|
outputs = model.generate(**inputs, max_length=500)
|
|
print(tokenizer.decode(outputs[0]))</code></pre>
|
|
<h3>zen-nano-thinking (4B)</h3>
|
|
<p><strong>Best for</strong>: Math, logic, transparent reasoning</p>
|
|
<pre><code class="lang-python">model = AutoModelForCausalLM.from_pretrained("zenlm/zen-nano-thinking")
|
|
tokenizer = AutoTokenizer.from_pretrained("zenlm/zen-nano-thinking")
|
|
|
|
# Enable chain-of-thought
|
|
messages = [
|
|
{"role": "user", "content": "Solve: If 3 apples cost $5, how much do 7 apples cost?"}
|
|
]
|
|
|
|
# Model will show reasoning steps
|
|
text = tokenizer.apply_chat_template(messages, tokenize=False)
|
|
inputs = tokenizer(text, return_tensors="pt")
|
|
outputs = model.generate(**inputs, max_length=500)
|
|
response = tokenizer.decode(outputs[0])
|
|
|
|
# Output shows thinking process:
|
|
# <think>
|
|
# Cost per apple = $5 / 3 = $1.67
|
|
# For 7 apples: 7 * $1.67 = $11.67
|
|
# </think>
|
|
# Answer: 7 apples cost $11.67</code></pre>
|
|
<h3>zen-agent (4B)</h3>
|
|
<p><strong>Best for</strong>: Tool calling, function execution, sub-agents</p>
|
|
<pre><code class="lang-python">from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
import json
|
|
|
|
model = AutoModelForCausalLM.from_pretrained("zenlm/zen-agent")
|
|
tokenizer = AutoTokenizer.from_pretrained("zenlm/zen-agent")
|
|
|
|
# Define available tools
|
|
tools = [
|
|
{
|
|
"type": "function",
|
|
"function": {
|
|
"name": "get_weather",
|
|
"description": "Get current weather in a location",
|
|
"parameters": {
|
|
"type": "object",
|
|
"properties": {
|
|
"location": {"type": "string", "description": "City name"}
|
|
},
|
|
"required": ["location"]
|
|
}
|
|
}
|
|
}
|
|
]
|
|
|
|
messages = [
|
|
{"role": "user", "content": "What's the weather in San Francisco?"}
|
|
]
|
|
|
|
# Model generates function call
|
|
text = tokenizer.apply_chat_template(
|
|
messages,
|
|
tools=tools,
|
|
tokenize=False
|
|
)
|
|
inputs = tokenizer(text, return_tensors="pt")
|
|
outputs = model.generate(**inputs, max_length=200)
|
|
response = tokenizer.decode(outputs[0])
|
|
|
|
# Parse function call and execute
|
|
# function_call = {"name": "get_weather", "arguments": {"location": "San Francisco"}}</code></pre>
|
|
<h2>Running on Different Hardware</h2>
|
|
<h3>Desktop/Laptop (16GB RAM)</h3>
|
|
<pre><code class="lang-bash"># Full 4B model (3.8GB memory)
|
|
python -c "
|
|
from mlx_lm import load, generate
|
|
model, tokenizer = load('zenlm/zen-nano-instruct')
|
|
print(generate(model, tokenizer, prompt='Hello!'))
|
|
"</code></pre>
|
|
<h3>Mobile/Edge (4GB RAM)</h3>
|
|
<pre><code class="lang-bash"># 4-bit quantized (1.5GB memory)
|
|
python -c "
|
|
from mlx_lm import load, generate
|
|
model, tokenizer = load('zenlm/zen-nano-instruct-4bit')
|
|
print(generate(model, tokenizer, prompt='Hello!'))
|
|
"</code></pre>
|
|
<h3>Raspberry Pi 4 (4GB)</h3>
|
|
<pre><code class="lang-bash"># llama.cpp for ARM
|
|
./llama-cli \
|
|
-m zen-nano-instruct-Q4_K_M.gguf \
|
|
-p "Hello from Raspberry Pi!" \
|
|
-n 100 \
|
|
--threads 4</code></pre>
|
|
<h3>Browser (WebGPU)</h3>
|
|
<pre><code class="lang-javascript">import { pipeline } from '@xenova/transformers';
|
|
|
|
// Load zen-nano in browser
|
|
const generator = await pipeline(
|
|
'text-generation',
|
|
'zenlm/zen-nano-instruct',
|
|
{ device: 'webgpu' }
|
|
);
|
|
|
|
const output = await generator('Explain neural networks', {
|
|
max_length: 200
|
|
});
|
|
console.log(output[0].generated_text);</code></pre>
|
|
<h2>Quantization Options</h2>
|
|
<h3>GGUF Formats (llama.cpp)</h3>
|
|
<p>| Quantization | Size | Quality | Speed | Use Case | |--------------|------|---------|-------|----------| | Q2_K | 1.2GB | 85% | Fastest | Mobile/Edge | | Q4_K_M | 1.5GB | 95% | Fast | <strong>Recommended</strong> | | Q5_K_M | 1.8GB | 97% | Medium | Balanced | | Q8_0 | 2.4GB | 99% | Slower | Quality-focused |</p>
|
|
<pre><code class="lang-bash"># Download specific quantization
|
|
huggingface-cli download zenlm/zen-nano-instruct-4bit \
|
|
--include "*Q4_K_M.gguf" \
|
|
--local-dir ./models</code></pre>
|
|
<h3>MLX Formats (Apple Silicon)</h3>
|
|
<pre><code class="lang-bash"># 4-bit MLX (optimized for M-series)
|
|
pip install mlx-lm
|
|
python -m mlx_lm.convert \
|
|
--hf-path zenlm/zen-nano-instruct \
|
|
--mlx-path ./zen-nano-mlx \
|
|
--quantize</code></pre>
|
|
<h3>AWQ/GPTQ (NVIDIA GPUs)</h3>
|
|
<pre><code class="lang-python">from transformers import AutoModelForCausalLM
|
|
from auto_gptq import AutoGPTQForCausalLM
|
|
|
|
# 4-bit GPTQ for NVIDIA
|
|
model = AutoGPTQForCausalLM.from_quantized(
|
|
"zenlm/zen-nano-instruct-gptq",
|
|
device="cuda:0"
|
|
)</code></pre>
|
|
<h2>Integration with Hanzo Ecosystem</h2>
|
|
<h3>Use via Hanzo LLM Gateway</h3>
|
|
<pre><code class="lang-python">from hanzoai import Hanzo
|
|
|
|
client = Hanzo()
|
|
response = client.chat.completions.create(
|
|
model="zen-nano-instruct",
|
|
messages=[{"role": "user", "content": "Hello!"}],
|
|
)
|
|
print(response.choices[0].message.content)</code></pre>
|
|
<h3>Use with OpenAI SDK (drop-in)</h3>
|
|
<pre><code class="lang-python">from openai import OpenAI
|
|
|
|
client = OpenAI(
|
|
base_url="https://api.hanzo.ai/v1",
|
|
api_key=os.environ["HANZO_API_KEY"],
|
|
)
|
|
response = client.chat.completions.create(
|
|
model="zen-nano-instruct",
|
|
messages=[{"role": "user", "content": "Hello!"}],
|
|
)</code></pre>
|
|
<h2>Fine-tuning with Hanzo Gym</h2>
|
|
<p>Train custom ZenLM models using LLaMA Factory:</p>
|
|
<pre><code class="lang-bash"># Fine-tune zen-nano for your domain
|
|
llamafactory-cli train \
|
|
--stage sft \
|
|
--model_name_or_path zenlm/zen-nano-instruct \
|
|
--dataset your_dataset \
|
|
--template qwen3 \
|
|
--finetuning_type lora \
|
|
--output_dir ./zen-nano-custom \
|
|
--flash_attn fa2 \
|
|
--use_unsloth true
|
|
|
|
# Export to GGUF for deployment
|
|
llamafactory-cli export \
|
|
--model_name_or_path ./zen-nano-custom \
|
|
--export_dir ./zen-nano-gguf \
|
|
--export_quantization_bit 4</code></pre>
|
|
<p>See <strong>hanzo-gym.md</strong> for comprehensive training guide.</p>
|
|
<h2>Local vs Cloud</h2>
|
|
<p>| | Local (ZenLM) | Cloud API | |--|---|---| | Cost | Free (Apache 2.0) | Per-token pricing | | Privacy | 100% local, data stays on device | Data sent to provider | | Latency | 20-50ms (no network) | 200-500ms (network + queue) | | Offline | Works fully offline | Requires internet | | Quality | Competitive at 4B scale | Larger models available |</p>
|
|
<h2>Zoo Labs Foundation</h2>
|
|
<p>ZenLM is co-developed with Zoo Labs Foundation (501(c)(3)), focused on sustainable and accessible AI research.</p>
|
|
<h2>Use Cases</h2>
|
|
<h3>1. Privacy-Critical Applications</h3>
|
|
<pre><code class="lang-python"># Medical records, financial data, legal documents
|
|
# Data never leaves your device
|
|
|
|
model = AutoModelForCausalLM.from_pretrained("zenlm/zen-nano-instruct")
|
|
tokenizer = AutoTokenizer.from_pretrained("zenlm/zen-nano-instruct")
|
|
|
|
# Process sensitive data locally
|
|
sensitive_text = "Patient diagnosis: ..."
|
|
inputs = tokenizer(f"Summarize: {sensitive_text}", return_tensors="pt")
|
|
outputs = model.generate(**inputs, max_length=200)
|
|
summary = tokenizer.decode(outputs[0])</code></pre>
|
|
<h3>2. Offline AI Applications</h3>
|
|
<pre><code class="lang-python"># Airplanes, remote locations, intermittent connectivity
|
|
|
|
from mlx_lm import load, generate
|
|
|
|
# Works 100% offline
|
|
model, tokenizer = load("zenlm/zen-nano-instruct-4bit")
|
|
response = generate(model, tokenizer, prompt="Help with code offline")</code></pre>
|
|
<h3>3. Cost-Optimized Production</h3>
|
|
<pre><code class="lang-python"># Use zen-nano via gateway for cost-optimized production
|
|
from hanzoai import Hanzo
|
|
|
|
client = Hanzo()
|
|
response = client.chat.completions.create(
|
|
model="zen-nano-instruct", # Cheapest Zen model
|
|
messages=[{"role": "user", "content": query}],
|
|
)</code></pre>
|
|
<h3>4. Edge AI / IoT</h3>
|
|
<pre><code class="lang-bash"># Raspberry Pi, edge servers, mobile devices
|
|
|
|
# Install llama.cpp on ARM
|
|
./llama-cli \
|
|
-m zen-nano-instruct-Q4_K_M.gguf \
|
|
-p "Process sensor data: ..." \
|
|
--threads 4</code></pre>
|
|
<h3>5. AI Sub-Agents</h3>
|
|
<pre><code class="lang-python"># Lightweight agents for tool calling
|
|
|
|
model = AutoModelForCausalLM.from_pretrained("zenlm/zen-agent")
|
|
|
|
# Each sub-agent runs zen-agent locally (4GB memory)
|
|
# Coordinate 10+ agents on single machine</code></pre>
|
|
<h2>Deployment Options</h2>
|
|
<h3>Docker Container</h3>
|
|
<pre><code class="lang-dockerfile">FROM python:3.10-slim
|
|
|
|
# Install ZenLM
|
|
RUN pip install transformers torch mlx-lm
|
|
|
|
# Download model
|
|
RUN python -c "
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
AutoModelForCausalLM.from_pretrained('zenlm/zen-nano-instruct')
|
|
"
|
|
|
|
EXPOSE 8080
|
|
CMD ["python", "server.py"]</code></pre>
|
|
<h3>Kubernetes</h3>
|
|
<pre><code class="lang-yaml">apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: zenlm-inference
|
|
spec:
|
|
replicas: 3
|
|
template:
|
|
spec:
|
|
containers:
|
|
- name: zenlm
|
|
image: zenlm/zen-nano-instruct:latest
|
|
resources:
|
|
limits:
|
|
memory: "4Gi"
|
|
cpu: "2"</code></pre>
|
|
<h3>Serverless</h3>
|
|
<p>WASM deployment is planned but not yet available. For now, use container-based serverless (Cloud Run, Lambda with container support).</p>
|
|
<h2>Community</h2>
|
|
<ul><li><strong>GitHub</strong>: <a href="https://github.com/zenlm">github.com/zenlm</a></li><li><strong>HuggingFace</strong>: <a href="https://huggingface.co/zenlm">huggingface.co/zenlm</a></li><li><strong>Discord</strong>: <a href="https://discord.gg/zenlm">discord.gg/zenlm</a></li><li><strong>Docs</strong>: <a href="https://zenlm.org">zenlm.org</a></li></ul>
|
|
<h2>Related Skills</h2>
|
|
<ul><li><strong>hanzo-gym.md</strong>: Train and fine-tune ZenLM models</li><li><strong>python-sdk.md</strong>: Deploy ZenLM with Hanzo SDK</li><li><strong>hanzo-desktop.md</strong>: Hanzo Desktop app</li><li><strong>hanzo-dev.md</strong>: Terminal AI coding agent</li></ul>
|
|
<hr>
|
|
<p><strong>Last Updated</strong>: 2026-03-13 <strong>Category</strong>: Hanzo Ecosystem <strong>Related</strong>: zenlm, models, local-ai, inference, edge <strong>Prerequisites</strong>: Python 3.10+, basic terminal knowledge</p>
|
|
<footer>
|
|
<div class="fmap">
|
|
<div><b>Build</b>
|
|
<a href="https://hanzo.app">hanzo.app</a> app builder
|
|
<a href="https://hanzo.codes">hanzo.codes</a> code with an agent
|
|
<a href="https://hanzoskills.com">hanzoskills.com</a> what an agent reads</div>
|
|
<div><b>Run</b>
|
|
<a href="https://console.hanzo.ai">console.hanzo.ai</a> cloud console
|
|
<a href="https://api.hanzo.ai">api.hanzo.ai</a> the API</div>
|
|
<div><b>Account</b>
|
|
<a href="https://hanzo.id">hanzo.id</a> sign in
|
|
<a href="https://pay.hanzo.ai">pay.hanzo.ai</a> top up
|
|
<a href="https://billing.hanzo.ai">billing.hanzo.ai</a> invoices and usage</div>
|
|
<div><b>Company</b>
|
|
<a href="https://hanzo.ai">hanzo.ai</a> canonical
|
|
<a href="https://hanzo.ventures">hanzo.ventures</a> ventures</div>
|
|
</div>
|
|
<div class="fend"><a href="/zenlm.md">This page as plain text</a> · <a href="/llms.txt">Index for agents</a></div>
|
|
</footer>
|
|
</div></body></html> |