Files
skills/public/ml-multi-model-orchestration.html
zeekayandHanzo Dev 9e10c1c10e Each llms index points at the other two
hanzoskills.com/llms.txt now opens with the install line, the API
reference index and the company index, so an agent that lands on any
one of the three can find the rest.

robots.txt claimed 537 documents while llms.txt counts 920. The number
is gone from robots.txt; llms.txt derives it.

public/ is regenerated from scripts/site.py, which it had fallen
behind: the copy-prompt menu and agent bar were in the generator and
not in the committed pages. tests/site.mjs passes against the result.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-11 00:27:59 -07:00

997 lines
48 KiB
HTML

<!doctype html>
<html lang="en"><head>
<meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
<title>ml-multi-model-orchestration</title>
<meta name="color-scheme" content="dark">
<style>
/* Committed to one visual world: a dark instrument panel. No light variant —
the dot field and the hairline rules only read on black, and a naive
inversion would wash both out. Every colour is painted explicitly so the page
never borrows a host background. */
@font-face{font-family:'Zen';src:url('/fonts/Zen-Variable.woff2') format('woff2');font-weight:100 900;font-style:normal;font-display:swap}
@font-face{font-family:'Zen';src:url('/fonts/Zen-Italic-Variable.woff2') format('woff2');font-weight:100 900;font-style:italic;font-display:swap}
@font-face{font-family:'Zen Mono';src:url('/fonts/ZenMono-Variable.woff2') format('woff2');font-weight:100 900;font-style:normal;font-display:swap}
@font-face{font-family:'Zen Mono';src:url('/fonts/ZenMono-Italic-Variable.woff2') format('woff2');font-weight:100 900;font-style:italic;font-display:swap}
@font-face{font-family:'Zen Pixel Square';src:url('/fonts/ZenPixel-Square.woff2') format('woff2');font-style:normal;font-display:swap}
:root{
--ink:#f4f4f2; --soft:#9c9c98; --faint:#6a6a66; --ground:#000; --panel:#0a0a0a;
--line:#232323; --line-hi:#3a3a3a; --accent:#f4f4f2;
--font-zen-sans:'Zen',-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif;
--font-zen-mono:'Zen Mono',ui-monospace,"SF Mono",SFMono-Regular,Menlo,Consolas,monospace;
--font-zen-pixel-square:'Zen Pixel Square',monospace;
--mono:var(--font-zen-mono);
--sans:var(--font-zen-sans);
/* Controls are pills and surfaces are soft, as on hanzo.ai. */
--r:999px; --r-soft:14px;
}
*{box-sizing:border-box}
html{-webkit-text-size-adjust:100%}
body{margin:0;background:var(--ground);color:var(--ink);font-family:var(--sans);
font-size:16px;line-height:1.65;-webkit-font-smoothing:antialiased}
.wrap{max-width:60rem;margin:0 auto;padding:0 1.75rem}
/* The dot field. A halftone made of one repeating radial-gradient, faded out
along the diagonal by a mask so it reads as a wave rather than wallpaper. */
/* No overflow clip here: the dot field is inset to the box already, and a clip
would cut off the copy-prompt menu. */
.hero{position:relative;border-bottom:1px solid var(--line)}
.hero::before{content:"";position:absolute;inset:0;
background-image:radial-gradient(var(--line-hi) 1px, transparent 1px);
background-size:7px 7px;
-webkit-mask-image:linear-gradient(115deg,#000 0%,rgba(0,0,0,.55) 38%,transparent 72%);
mask-image:linear-gradient(115deg,#000 0%,rgba(0,0,0,.55) 38%,transparent 72%);
pointer-events:none}
.hero .wrap{position:relative;padding-top:5.5rem;padding-bottom:4rem}
nav{border-bottom:1px solid var(--line)}
nav .in{max-width:60rem;margin:0 auto;padding:1.1rem 1.75rem;display:flex;
justify-content:space-between;align-items:center;gap:1rem}
.brand{font-family:var(--mono);font-size:.85rem;letter-spacing:.14em;
text-transform:uppercase;color:var(--ink);text-decoration:none;
display:inline-flex;align-items:center;gap:.6rem}
.brand i{font-style:normal;color:var(--faint)}
.mark{width:1.15em;height:1.15em;display:block;flex:0 0 auto}
nav a.up{font-family:var(--mono);font-size:.72rem;letter-spacing:.14em;
text-transform:uppercase;color:var(--soft);text-decoration:none}
nav a.up:hover{color:var(--ink)}
h1{font-family:var(--mono);font-size:clamp(2rem,5.4vw,3.4rem);line-height:1.12;
letter-spacing:-.02em;margin:0 0 1.1rem;font-weight:500;text-wrap:balance}
h1 em{font-style:normal;color:var(--faint)}
.lede{color:var(--soft);font-size:1.05rem;max-width:52ch;margin:0 0 2.25rem}
.btns{display:flex;flex-wrap:wrap;gap:.7rem;margin:0 0 .5rem;align-items:center}
.btn{display:inline-flex;align-items:center;gap:.5rem;font-family:var(--mono);
font-size:.72rem;letter-spacing:.14em;text-transform:uppercase;
padding:.8rem 1.4rem;border:1px solid var(--line-hi);border-radius:var(--r);
color:var(--ink);text-decoration:none;background:transparent;cursor:pointer;
line-height:1.2;font-weight:400}
.btn:hover{background:#141414;text-decoration:none;border-color:var(--soft)}
.btn.solid{background:var(--ink);color:#000;border-color:var(--ink)}
.btn.solid:hover{background:#d8d8d4}
.btn .chev{width:.7em;height:.7em;flex:0 0 auto;transition:transform .15s ease}
.btn[aria-expanded="true"] .chev{transform:rotate(180deg)}
/* Copy-prompt selector. Each row copies the prompt for its own target. */
.copy{position:relative;display:inline-flex}
.menu{position:absolute;top:calc(100% + .5rem);left:0;z-index:20;min-width:21rem;
background:var(--panel);border:1px solid var(--line-hi);border-radius:var(--r-soft);
padding:.4rem;display:flex;flex-direction:column;
box-shadow:0 12px 32px rgba(0,0,0,.75)}
.menu[hidden]{display:none}
.mi{display:flex;align-items:center;gap:.7rem;width:100%;text-align:left;
padding:.62rem .8rem;border:0;border-radius:calc(var(--r-soft) - .35rem);
background:transparent;color:var(--ink);font-family:var(--mono);font-size:.76rem;
letter-spacing:.04em;text-decoration:none;cursor:pointer;line-height:1.3;
white-space:nowrap}
.mi:hover,.mi:focus-visible{background:#181818;outline:none}
.mi .mark{width:1.05rem;height:1.05rem;color:var(--soft)}
.mi:hover .mark{color:var(--ink)}
.mi u{margin-left:auto;color:var(--faint);text-decoration:none;font-size:.68rem;
letter-spacing:.1em;text-transform:uppercase}
/* Works-with bar: one scrolling row, name and mark per agent. */
.strip{border-bottom:1px solid var(--line);background:var(--panel)}
.strip .in{max-width:60rem;margin:0 auto;padding:.9rem 1.75rem;display:flex;
align-items:center;gap:1.25rem;font-family:var(--mono);font-size:.72rem;
letter-spacing:.1em;text-transform:uppercase;color:var(--faint)}
.strip b{color:var(--soft);font-weight:400;flex:0 0 auto;
border-right:1px solid var(--line);padding-right:1.25rem;align-self:stretch;
display:flex;align-items:center}
.bar{display:flex;align-items:center;gap:.5rem;flex:1 1 auto;min-width:0;
overflow-x:auto;scrollbar-width:none;scroll-snap-type:x proximity;
-webkit-overflow-scrolling:touch;
-webkit-mask-image:linear-gradient(90deg,#000 calc(100% - 3rem),transparent);
mask-image:linear-gradient(90deg,#000 calc(100% - 3rem),transparent)}
.bar::-webkit-scrollbar{display:none}
.agent{display:inline-flex;align-items:center;gap:.45rem;flex:0 0 auto;
padding:.4rem .85rem;border:1px solid var(--line);border-radius:var(--r);
color:var(--soft);white-space:nowrap;scroll-snap-align:start;
text-decoration:none;background:var(--ground)}
.agent:hover{color:var(--ink);border-color:var(--line-hi);text-decoration:none}
.agent .mark{width:.95rem;height:.95rem}
h2{font-family:var(--mono);font-size:1.1rem;letter-spacing:-.01em;margin:2.75rem 0 .75rem;font-weight:500}
h3{font-size:1rem;margin:2rem 0 .5rem;font-weight:600}
h4,h5,h6{font-size:.94rem;margin:1.5rem 0 .5rem;font-weight:600}
p{margin:0 0 1rem;max-width:70ch}
ul,ol{margin:0 0 1rem;padding-left:1.35rem;max-width:70ch}
li{margin:.22rem 0}
a{color:var(--ink);text-decoration:underline;text-underline-offset:3px;
text-decoration-color:var(--line-hi)}
a:hover{text-decoration-color:var(--ink)}
a:focus-visible{outline:1px solid var(--ink);outline-offset:3px}
code{font-family:var(--mono);font-size:.85em;background:var(--panel);
border:1px solid var(--line);border-radius:5px;padding:.1em .36em}
pre{background:var(--panel);border:1px solid var(--line);padding:1rem 1.1rem;
border-radius:var(--r-soft);overflow-x:auto;margin:0 0 1.25rem}
pre code{background:none;border:0;padding:0;font-size:.82rem;line-height:1.7}
blockquote{margin:0 0 1rem;padding-left:1rem;border-left:1px solid var(--line-hi);color:var(--soft)}
hr{border:0;border-top:1px solid var(--line);margin:2.5rem 0}
table{border-collapse:collapse;width:100%;margin:0 0 1.25rem;font-size:.92rem;
display:block;overflow-x:auto}
th,td{border:1px solid var(--line);padding:.5rem .7rem;text-align:left}
.curl{display:block;background:var(--panel);border:1px solid var(--line);
border-radius:var(--r-soft);padding:.95rem 1.1rem;font-family:var(--mono);
font-size:.8rem;color:var(--soft);overflow-x:auto;margin:0 0 2.5rem;
white-space:pre;line-height:1.9}
.curl b{color:var(--ink);font-weight:400}
.curl i{font-style:normal;color:var(--faint)}
/* index */
#q{width:100%;padding:.85rem 1.2rem;font-family:var(--mono);font-size:.85rem;color:var(--ink);
background:var(--panel);border:1px solid var(--line);border-radius:var(--r);margin:0 0 .6rem}
#q::placeholder{color:var(--faint);text-transform:uppercase;letter-spacing:.1em;font-size:.72rem}
#q:focus{outline:none;border-color:var(--soft)}
#count{color:var(--faint);font-family:var(--mono);font-size:.72rem;letter-spacing:.1em;
text-transform:uppercase;margin:0 0 2rem}
.cat{margin:2.5rem 0 .5rem;font-family:var(--mono);font-size:.7rem;letter-spacing:.16em;
text-transform:uppercase;color:var(--faint);border-bottom:1px solid var(--line);
padding-bottom:.45rem}
.row{display:block;padding:.6rem .9rem;margin:0 -.9rem;border-bottom:1px solid var(--line);
border-radius:var(--r-soft);text-decoration:none}
.row:hover{background:#0a0a0a}
.nm{font-family:var(--mono);font-size:.84rem;color:var(--ink);display:block}
.ds{color:var(--soft);font-size:.9rem;display:block;margin-top:.12rem}
footer{margin-top:4rem;padding:2rem 0 4rem;border-top:1px solid var(--line);
color:var(--faint);font-size:.85rem}
footer a{color:var(--soft)}
.fmap{display:grid;gap:1.75rem;grid-template-columns:repeat(auto-fit,minmax(11rem,1fr));
margin:0 0 2rem;font-size:.85rem}
.fmap b{display:block;font-family:var(--mono);font-size:.68rem;letter-spacing:.16em;
text-transform:uppercase;color:var(--faint);font-weight:400;margin:0 0 .6rem;
border-bottom:1px solid var(--line);padding-bottom:.4rem}
.fmap a{display:inline-block;color:var(--ink);text-decoration:none;margin:.3rem .35rem 0 0}
.fmap a:hover{text-decoration:underline}
.fmap div{color:var(--faint);line-height:1.5}
.fend{border-top:1px solid var(--line);padding-top:1.25rem;font-size:.82rem}
@media (max-width:640px){
.hero .wrap{padding-top:3.25rem;padding-bottom:2.75rem}
.btns{gap:.5rem}
.btn{padding:.75rem 1.15rem;font-size:.68rem}
.copy,.copy .btn{width:100%;justify-content:center}
.menu{left:0;right:0;min-width:0}
.strip .in{flex-direction:column;align-items:flex-start;gap:.6rem;padding:.9rem 1.75rem}
.strip b{border-right:0;padding-right:0}
.bar{width:100%}
}
</style>
<link rel="alternate" type="text/markdown" href="/ml-multi-model-orchestration.md">
<link rel="icon" href="/favicon.svg">
<link rel="canonical" href="https://hanzoskills.com/ml-multi-model-orchestration">
</head>
<!-- Reading this with a machine? Every page here is also plain markdown.
This one: https://hanzoskills.com/ml-multi-model-orchestration.md
Index: https://hanzoskills.com/llms.txt
Corpus: https://hanzoskills.com/all.md
Append .md to any path for the markdown twin. -->
<body>
<nav><div class="in">
<a class="brand" href="/"><svg class="mark" viewBox="0 0 67 67" fill="currentColor" aria-hidden="true"><path d="M22.21 67V44.6369H0V67H22.21Z"/><path d="M66.7038 22.3184H22.2534L0.0878906 44.6367H44.4634L66.7038 22.3184Z"/><path d="M22.21 0H0V22.3184H22.21V0Z"/><path d="M66.7198 0H44.5098V22.3184H66.7198V0Z"/><path d="M66.7198 67V44.6369H44.5098V67H66.7198Z"/></svg> HANZO<i>SKILLS</i></a>
<a class="up" href="/">All skills</a>
</div></nav>
<div class="wrap">
<h1>ml-multi-model-orchestration</h1><p class="lede">Coordinating multiple LLMs in pipelines, ensembles, and cascading workflows</p><h1>Multi-Model Orchestration</h1>
<p><strong>Scope</strong>: Pipeline patterns, ensemble methods, specialist routing, cascading, context management, error handling <strong>Lines</strong>: ~400 <strong>Last Updated</strong>: 2025-10-26</p>
<h2>When to Use This Skill</h2>
<p>Activate this skill when:</p>
<ul><li>Building workflows that combine strengths of multiple models</li><li>Implementing consensus mechanisms for critical decisions</li><li>Cascading from fast models to accurate models based on confidence</li><li>Designing specialist systems where different models handle different subtasks</li><li>Chaining models in sequential pipelines (planning -&gt; execution -&gt; review)</li><li>Managing context and state across multi-model calls</li><li>Handling failures with fallback models</li><li>Optimizing for both quality and cost through model mixing</li></ul>
<h2>Core Concepts</h2>
<h3>Orchestration Patterns</h3>
<p><strong>1. Pipeline (Sequential)</strong>:</p>
<ul><li>Models execute in sequence, each using previous output</li><li>Example: Planner -&gt; Coder -&gt; Reviewer -&gt; Refiner</li><li><strong>Benefit</strong>: Specialization per stage</li><li><strong>Cost</strong>: Cumulative latency</li></ul>
<p><strong>2. Ensemble (Parallel)</strong>:</p>
<ul><li>Multiple models process same input, outputs combined</li><li>Example: GPT-4o + Claude + Gemini -&gt; Vote or merge</li><li><strong>Benefit</strong>: Higher accuracy, reduced bias</li><li><strong>Cost</strong>: 3x+ API costs</li></ul>
<p><strong>3. Specialist Routing</strong>:</p>
<ul><li>Route to model specialized for subtask</li><li>Example: Code-&gt;Claude, Math-&gt;DeepSeek R1, General-&gt;GPT-4o</li><li><strong>Benefit</strong>: Optimal model per task</li><li><strong>Cost</strong>: Routing overhead</li></ul>
<p><strong>4. Cascade (Conditional)</strong>:</p>
<ul><li>Start with cheap/fast model, escalate if needed</li><li>Example: GPT-4o mini -&gt; (if unsure) -&gt; GPT-4o</li><li><strong>Benefit</strong>: Cost savings with quality backstop</li><li><strong>Cost</strong>: Complexity in confidence detection</li></ul>
<p><strong>5. Hybrid (Combined)</strong>:</p>
<ul><li>Mix patterns: Pipeline + Ensemble, Cascade + Specialist</li><li>Example: Route by topic -&gt; Ensemble -&gt; Pipeline</li><li><strong>Benefit</strong>: Maximum flexibility</li><li><strong>Cost</strong>: Complex orchestration logic</li></ul>
<h3>Context Management</h3>
<p><strong>Challenges</strong>:</p>
<ul><li>Passing large context between models (token costs)</li><li>Maintaining conversation state across calls</li><li>Avoiding context window overflow</li><li>Preserving semantic coherence</li></ul>
<p><strong>Solutions</strong>:</p>
<ul><li>Prompt chaining (minimal context passing)</li><li>Shared memory/cache layer (Redis, database)</li><li>Summarization between stages (lossy but efficient)</li><li>Context compression (extract key information)</li></ul>
<h3>Error Handling Strategies</h3>
<p><strong>Fallback Chains</strong>:</p>
<pre><code>Primary model -&gt; (on error) -&gt; Fallback 1 -&gt; Fallback 2 -&gt; Default response</code></pre>
<p><strong>Retry with Backoff</strong>:</p>
<pre><code>Attempt 1 (immediate) -&gt; Attempt 2 (+1s) -&gt; Attempt 3 (+3s) -&gt; Fail gracefully</code></pre>
<p><strong>Graceful Degradation</strong>:</p>
<pre><code>Try ensemble -&gt; (on partial failure) -&gt; Use successful responses -&gt; (on total failure) -&gt; Single model</code></pre>
<h3>Arize Phoenix Multi-Model Tracing</h3>
<p><strong>Capabilities</strong>:</p>
<ul><li>Span-level tracing across model calls</li><li>Routing decision visibility</li><li>Cost/latency breakdown per model</li><li>Error rate tracking per model</li><li>A/B test comparison</li></ul>
<p><strong>Span Structure</strong>:</p>
<pre><code>Root Span (orchestration)
├─ Span: routing_decision
├─ Span: model_call_1 (claude-3.5-sonnet)
│ ├─ Attribute: tokens_used
│ ├─ Attribute: latency_ms
│ └─ Attribute: cost_usd
├─ Span: model_call_2 (gpt-4o)
└─ Span: result_aggregation</code></pre>
<hr>
<h2>Implementation Patterns</h2>
<h3>Pipeline Pattern: Plan-Code-Review</h3>
<pre><code class="lang-python">import openai
from typing import Dict, Any
class CodeGenerationPipeline:
&quot;&quot;&quot;Multi-stage pipeline: planning -&gt; coding -&gt; review&quot;&quot;&quot;
def __init__(self):
self.planner_model = &quot;gpt-4o&quot;
self.coder_model = &quot;claude-3-5-sonnet-20241022&quot;
self.reviewer_model = &quot;gpt-4o&quot;
def plan(self, requirement: str) -&gt; str:
&quot;&quot;&quot;Stage 1: Planning with GPT-4o&quot;&quot;&quot;
response = openai.ChatCompletion.create(
model=self.planner_model,
messages=[{
&quot;role&quot;: &quot;system&quot;,
&quot;content&quot;: &quot;You are a software architect. Create a detailed implementation plan.&quot;
}, {
&quot;role&quot;: &quot;user&quot;,
&quot;content&quot;: f&quot;Plan implementation for: {requirement}&quot;
}],
temperature=0.7
)
return response.choices[0].message.content
def code(self, plan: str) -&gt; str:
&quot;&quot;&quot;Stage 2: Coding with Claude (best at code)&quot;&quot;&quot;
response = openai.ChatCompletion.create(
model=self.coder_model,
messages=[{
&quot;role&quot;: &quot;system&quot;,
&quot;content&quot;: &quot;You are an expert programmer. Implement the plan precisely.&quot;
}, {
&quot;role&quot;: &quot;user&quot;,
&quot;content&quot;: f&quot;Implement this plan:\n\n{plan}&quot;
}],
temperature=0.2
)
return response.choices[0].message.content
def review(self, code: str, plan: str) -&gt; Dict[str, Any]:
&quot;&quot;&quot;Stage 3: Review with GPT-4o&quot;&quot;&quot;
response = openai.ChatCompletion.create(
model=self.reviewer_model,
messages=[{
&quot;role&quot;: &quot;system&quot;,
&quot;content&quot;: &quot;You are a code reviewer. Assess quality, correctness, and adherence to plan.&quot;
}, {
&quot;role&quot;: &quot;user&quot;,
&quot;content&quot;: f&quot;Review this code against the plan.\n\nPlan:\n{plan}\n\nCode:\n{code}&quot;
}],
temperature=0.3
)
review_text = response.choices[0].message.content
# Extract approval (simple heuristic)
approved = &quot;approved&quot; in review_text.lower() or &quot;looks good&quot; in review_text.lower()
return {
&quot;approved&quot;: approved,
&quot;review&quot;: review_text,
&quot;code&quot;: code
}
def execute(self, requirement: str) -&gt; Dict[str, Any]:
&quot;&quot;&quot;Execute full pipeline&quot;&quot;&quot;
print(&quot;Stage 1: Planning...&quot;)
plan = self.plan(requirement)
print(&quot;Stage 2: Coding...&quot;)
code = self.code(plan)
print(&quot;Stage 3: Review...&quot;)
result = self.review(code, plan)
return {
&quot;requirement&quot;: requirement,
&quot;plan&quot;: plan,
&quot;code&quot;: result[&quot;code&quot;],
&quot;review&quot;: result[&quot;review&quot;],
&quot;approved&quot;: result[&quot;approved&quot;]
}
# Usage
pipeline = CodeGenerationPipeline()
result = pipeline.execute(&quot;Build a REST API for user authentication with JWT&quot;)
print(f&quot;Approved: {result[&#x27;approved&#x27;]}&quot;)
print(f&quot;Code:\n{result[&#x27;code&#x27;][:500]}...&quot;)</code></pre>
<h3>Ensemble Pattern: Consensus Voting</h3>
<pre><code class="lang-python">import asyncio
from typing import List, Dict
from collections import Counter
class EnsembleOrchestrator:
&quot;&quot;&quot;Run multiple models in parallel and vote on results&quot;&quot;&quot;
def __init__(self, models: List[str] = None):
self.models = models or [
&quot;gpt-4o&quot;,
&quot;claude-3-5-sonnet-20241022&quot;,
&quot;gemini-2.5-pro&quot;
]
async def call_model(self, model: str, prompt: str) -&gt; str:
&quot;&quot;&quot;Call single model asynchronously&quot;&quot;&quot;
response = await asyncio.to_thread(
openai.ChatCompletion.create,
model=model,
messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: prompt}],
temperature=0.3
)
return response.choices[0].message.content
async def ensemble_vote(
self,
prompt: str,
extract_answer_fn = None
) -&gt; Dict[str, Any]:
&quot;&quot;&quot;Run ensemble and vote on answers&quot;&quot;&quot;
# Call all models in parallel
tasks = [
self.call_model(model, prompt)
for model in self.models
]
responses = await asyncio.gather(*tasks)
# Extract answers (default: use full response)
if extract_answer_fn is None:
extract_answer_fn = lambda x: x.strip().lower()
answers = [extract_answer_fn(r) for r in responses]
# Vote
vote_counts = Counter(answers)
majority_answer = vote_counts.most_common(1)[0][0]
majority_count = vote_counts[majority_answer]
# Check consensus (&gt;50%)
consensus = majority_count &gt; len(self.models) / 2
return {
&quot;models&quot;: self.models,
&quot;responses&quot;: responses,
&quot;answers&quot;: answers,
&quot;majority_answer&quot;: majority_answer,
&quot;vote_counts&quot;: dict(vote_counts),
&quot;consensus&quot;: consensus,
&quot;confidence&quot;: majority_count / len(self.models)
}
# Usage
async def main():
ensemble = EnsembleOrchestrator()
# Multiple choice question
prompt = &quot;&quot;&quot;
What is the capital of Australia?
A) Sydney
B) Melbourne
C) Canberra
D) Brisbane
Answer with just the letter (A, B, C, or D).
&quot;&quot;&quot;
def extract_letter(response: str) -&gt; str:
# Extract first letter A-D
import re
match = re.search(r&#x27;[ABCD]&#x27;, response.upper())
return match.group(0) if match else &quot;&quot;
result = await ensemble.ensemble_vote(prompt, extract_answer_fn=extract_letter)
print(f&quot;Majority answer: {result[&#x27;majority_answer&#x27;]}&quot;)
print(f&quot;Consensus: {result[&#x27;consensus&#x27;]} ({result[&#x27;confidence&#x27;]:.0%})&quot;)
print(f&quot;Vote counts: {result[&#x27;vote_counts&#x27;]}&quot;)
asyncio.run(main())</code></pre>
<h3>Cascade Pattern: Confidence-Based Escalation</h3>
<pre><code class="lang-python">import re
from typing import Optional, Dict, Any
class CascadeOrchestrator:
&quot;&quot;&quot;Start with cheap model, escalate to expensive if uncertain&quot;&quot;&quot;
def __init__(
self,
fast_model: str = &quot;gpt-4o-mini&quot;,
strong_model: str = &quot;gpt-4o&quot;,
confidence_threshold: float = 0.8
):
self.fast = fast_model
self.strong = strong_model
self.threshold = confidence_threshold
def extract_confidence(self, response: str) -&gt; Optional[float]:
&quot;&quot;&quot;Extract confidence from response (if model provides it)&quot;&quot;&quot;
# Look for patterns like &quot;Confidence: 85%&quot; or &quot;(confidence: 0.85)&quot;
patterns = [
r&quot;confidence[:\s]+(\d+(?:\.\d+)?)\%?&quot;,
r&quot;\(confidence[:\s]+(\d+(?:\.\d+)?)\)&quot;,
]
for pattern in patterns:
match = re.search(pattern, response.lower())
if match:
conf = float(match.group(1))
return conf / 100 if conf &gt; 1 else conf
return None
def call_with_confidence(
self,
model: str,
prompt: str,
request_confidence: bool = True
) -&gt; Dict[str, Any]:
&quot;&quot;&quot;Call model and extract confidence&quot;&quot;&quot;
if request_confidence:
prompt += &quot;\n\nProvide your confidence level (0-100%) at the end.&quot;
response = openai.ChatCompletion.create(
model=model,
messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: prompt}],
temperature=0.3
)
text = response.choices[0].message.content
confidence = self.extract_confidence(text)
return {
&quot;model&quot;: model,
&quot;response&quot;: text,
&quot;confidence&quot;: confidence,
&quot;tokens&quot;: response.usage.total_tokens
}
def cascade(self, prompt: str) -&gt; Dict[str, Any]:
&quot;&quot;&quot;Execute cascade: fast -&gt; strong if needed&quot;&quot;&quot;
print(f&quot;Trying fast model: {self.fast}&quot;)
fast_result = self.call_with_confidence(self.fast, prompt)
# Check if confident enough
if fast_result[&quot;confidence&quot;] and fast_result[&quot;confidence&quot;] &gt;= self.threshold:
print(f&quot;Fast model confident ({fast_result[&#x27;confidence&#x27;]:.0%}), using result&quot;)
return {
**fast_result,
&quot;escalated&quot;: False,
&quot;total_tokens&quot;: fast_result[&quot;tokens&quot;]
}
# Escalate to strong model
print(f&quot;Low confidence ({fast_result[&#x27;confidence&#x27;]:.0%}), escalating to {self.strong}&quot;)
strong_result = self.call_with_confidence(self.strong, prompt)
return {
**strong_result,
&quot;escalated&quot;: True,
&quot;fast_result&quot;: fast_result,
&quot;total_tokens&quot;: fast_result[&quot;tokens&quot;] + strong_result[&quot;tokens&quot;]
}
# Usage
cascade = CascadeOrchestrator(confidence_threshold=0.8)
result = cascade.cascade(&quot;What are the ethical implications of AGI development?&quot;)
print(f&quot;Final model: {result[&#x27;model&#x27;]}&quot;)
print(f&quot;Escalated: {result[&#x27;escalated&#x27;]}&quot;)
print(f&quot;Total tokens: {result[&#x27;total_tokens&#x27;]}&quot;)</code></pre>
<h3>Specialist Routing with Context Passing</h3>
<pre><code class="lang-python">from typing import List, Dict, Any
class SpecialistOrchestrator:
&quot;&quot;&quot;Route subtasks to specialist models&quot;&quot;&quot;
def __init__(self):
self.specialists = {
&quot;code&quot;: &quot;claude-3-5-sonnet-20241022&quot;,
&quot;math&quot;: &quot;deepseek-r1-distilled-70b&quot;,
&quot;general&quot;: &quot;gpt-4o&quot;,
&quot;fast&quot;: &quot;gemini-2.5-flash&quot;
}
self.conversation_history: List[Dict] = []
def detect_task_type(self, query: str) -&gt; str:
&quot;&quot;&quot;Simple task detection (use semantic routing for production)&quot;&quot;&quot;
query_lower = query.lower()
if any(kw in query_lower for kw in [&quot;code&quot;, &quot;function&quot;, &quot;implement&quot;, &quot;debug&quot;]):
return &quot;code&quot;
if any(kw in query_lower for kw in [&quot;calculate&quot;, &quot;solve&quot;, &quot;math&quot;, &quot;equation&quot;]):
return &quot;math&quot;
if len(query) &lt; 50:
return &quot;fast&quot;
return &quot;general&quot;
def call_specialist(
self,
task_type: str,
query: str,
include_history: bool = False
) -&gt; Dict[str, Any]:
&quot;&quot;&quot;Route to specialist and maintain context&quot;&quot;&quot;
model = self.specialists[task_type]
# Build messages with optional history
messages = []
if include_history:
messages.extend(self.conversation_history)
messages.append({&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: query})
response = openai.ChatCompletion.create(
model=model,
messages=messages,
temperature=0.3
)
answer = response.choices[0].message.content
# Update conversation history
self.conversation_history.append({&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: query})
self.conversation_history.append({&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: answer})
return {
&quot;task_type&quot;: task_type,
&quot;model&quot;: model,
&quot;query&quot;: query,
&quot;answer&quot;: answer,
&quot;tokens&quot;: response.usage.total_tokens
}
def multi_turn_conversation(self, queries: List[str]) -&gt; List[Dict[str, Any]]:
&quot;&quot;&quot;Handle multi-turn conversation with specialist routing&quot;&quot;&quot;
results = []
for query in queries:
task_type = self.detect_task_type(query)
print(f&quot;Query: {query[:60]}... -&gt; {task_type} ({self.specialists[task_type]})&quot;)
result = self.call_specialist(
task_type,
query,
include_history=True # Maintain context
)
results.append(result)
return results
# Usage
orchestrator = SpecialistOrchestrator()
conversation = [
&quot;Write a function to calculate Fibonacci numbers&quot;,
&quot;Now optimize it for large inputs&quot;,
&quot;What&#x27;s the time complexity?&quot;,
&quot;Calculate the 50th Fibonacci number using your implementation&quot;
]
results = orchestrator.multi_turn_conversation(conversation)
for r in results:
print(f&quot;\n[{r[&#x27;task_type&#x27;]}] {r[&#x27;model&#x27;]}&quot;)
print(f&quot;Answer: {r[&#x27;answer&#x27;][:200]}...&quot;)</code></pre>
<h3>Phoenix Observability Integration</h3>
<pre><code class="lang-python">import phoenix as px
from phoenix.trace import using_project
from opentelemetry import trace
# Start Phoenix
px.launch_app()
tracer = trace.get_tracer(__name__)
@using_project(&quot;multi-model-orchestration&quot;)
def orchestrated_pipeline(requirement: str):
&quot;&quot;&quot;Pipeline with full observability&quot;&quot;&quot;
with tracer.start_as_current_span(&quot;pipeline&quot;) as root_span:
root_span.set_attribute(&quot;requirement&quot;, requirement)
# Stage 1: Planning
with tracer.start_as_current_span(&quot;stage_planning&quot;) as plan_span:
plan_span.set_attribute(&quot;model&quot;, &quot;gpt-4o&quot;)
plan_response = openai.ChatCompletion.create(
model=&quot;gpt-4o&quot;,
messages=[{
&quot;role&quot;: &quot;user&quot;,
&quot;content&quot;: f&quot;Plan: {requirement}&quot;
}]
)
plan = plan_response.choices[0].message.content
plan_span.set_attribute(&quot;tokens&quot;, plan_response.usage.total_tokens)
plan_span.set_attribute(&quot;cost_usd&quot;, calculate_cost(plan_response, &quot;gpt-4o&quot;))
# Stage 2: Coding
with tracer.start_as_current_span(&quot;stage_coding&quot;) as code_span:
code_span.set_attribute(&quot;model&quot;, &quot;claude-3-5-sonnet&quot;)
code_response = openai.ChatCompletion.create(
model=&quot;claude-3-5-sonnet-20241022&quot;,
messages=[{
&quot;role&quot;: &quot;user&quot;,
&quot;content&quot;: f&quot;Implement:\n{plan}&quot;
}]
)
code = code_response.choices[0].message.content
code_span.set_attribute(&quot;tokens&quot;, code_response.usage.total_tokens)
code_span.set_attribute(&quot;cost_usd&quot;, calculate_cost(code_response, &quot;claude-3-5-sonnet&quot;))
# Stage 3: Review
with tracer.start_as_current_span(&quot;stage_review&quot;) as review_span:
review_span.set_attribute(&quot;model&quot;, &quot;gpt-4o&quot;)
review_response = openai.ChatCompletion.create(
model=&quot;gpt-4o&quot;,
messages=[{
&quot;role&quot;: &quot;user&quot;,
&quot;content&quot;: f&quot;Review:\nPlan: {plan}\nCode: {code}&quot;
}]
)
review = review_response.choices[0].message.content
review_span.set_attribute(&quot;tokens&quot;, review_response.usage.total_tokens)
review_span.set_attribute(&quot;cost_usd&quot;, calculate_cost(review_response, &quot;gpt-4o&quot;))
# Calculate total metrics
total_cost = (
calculate_cost(plan_response, &quot;gpt-4o&quot;) +
calculate_cost(code_response, &quot;claude-3-5-sonnet&quot;) +
calculate_cost(review_response, &quot;gpt-4o&quot;)
)
total_tokens = (
plan_response.usage.total_tokens +
code_response.usage.total_tokens +
review_response.usage.total_tokens
)
root_span.set_attribute(&quot;total_cost_usd&quot;, total_cost)
root_span.set_attribute(&quot;total_tokens&quot;, total_tokens)
return {
&quot;plan&quot;: plan,
&quot;code&quot;: code,
&quot;review&quot;: review,
&quot;cost&quot;: total_cost,
&quot;tokens&quot;: total_tokens
}
def calculate_cost(response, model: str) -&gt; float:
&quot;&quot;&quot;Calculate cost based on token usage&quot;&quot;&quot;
pricing = {
&quot;gpt-4o&quot;: {&quot;input&quot;: 5.0, &quot;output&quot;: 20.0},
&quot;claude-3-5-sonnet&quot;: {&quot;input&quot;: 3.0, &quot;output&quot;: 15.0},
&quot;gpt-4o-mini&quot;: {&quot;input&quot;: 0.15, &quot;output&quot;: 0.60},
&quot;gemini-2.5-flash&quot;: {&quot;input&quot;: 0.10, &quot;output&quot;: 0.40}
}
model_pricing = pricing.get(model, pricing[&quot;gpt-4o&quot;])
usage = response.usage
cost = (
(usage.prompt_tokens / 1_000_000) * model_pricing[&quot;input&quot;] +
(usage.completion_tokens / 1_000_000) * model_pricing[&quot;output&quot;]
)
return cost</code></pre>
<hr>
<h2>Quick Reference</h2>
<h3>Orchestration Pattern Selection</h3>
<pre><code>Use Case | Recommended Pattern
----------------------------------|------------------------------------
Sequential specialization | Pipeline
Critical decision (safety) | Ensemble with voting
Cost optimization | Cascade (cheap -&gt; expensive)
Domain-specific routing | Specialist routing
Complex multi-step workflows | Hybrid (Pipeline + Specialist)
High-reliability requirements | Ensemble + Fallback chains</code></pre>
<h3>Context Passing Strategies</h3>
<pre><code>Strategy | Cost | Latency | Use Case
-------------------|--------|---------|---------------------------
Full history | High | High | Short conversations (&lt;10 turns)
Summarization | Medium | Medium | Long conversations
Key extraction | Low | Low | Factual pipelines
Shared state (DB) | Low | Medium | Multi-user systems
No context | Lowest | Lowest | Independent tasks</code></pre>
<h3>Error Handling Patterns</h3>
<pre><code class="lang-python"># Pattern 1: Fallback chain
try:
response = call_model(&quot;claude-3-5-sonnet&quot;, prompt)
except Exception:
try:
response = call_model(&quot;gpt-4o&quot;, prompt)
except Exception:
response = call_model(&quot;gpt-4o-mini&quot;, prompt)
# Pattern 2: Retry with exponential backoff
for attempt in range(3):
try:
response = call_model(&quot;gpt-4o&quot;, prompt)
break
except Exception as e:
if attempt == 2:
raise
time.sleep(2 ** attempt)
# Pattern 3: Graceful degradation
results = []
for model in [&quot;gpt-4o&quot;, &quot;claude-3-5-sonnet&quot;, &quot;gemini-2.5-pro&quot;]:
try:
results.append(call_model(model, prompt))
except Exception:
pass # Continue with other models
if results:
return aggregate(results)
else:
return default_response()</code></pre>
<hr>
<h2>Anti-Patterns</h2>
<p>❌ <strong>Passing full context between all stages</strong>: Wastes tokens and money ✅ Summarize or extract key information between pipeline stages</p>
<p>❌ <strong>Ensemble without voting mechanism</strong>: Can&#x27;t resolve disagreements ✅ Implement majority voting or weighted consensus</p>
<p>❌ <strong>Cascade without confidence scores</strong>: Can&#x27;t decide when to escalate ✅ Prompt models for confidence or use logprobs for uncertainty estimation</p>
<p>❌ <strong>No fallback on model failures</strong>: Single point of failure ✅ Implement fallback chains with multiple model options</p>
<p>❌ <strong>Ignoring cost in orchestration</strong>: Ensemble costs 3x per query ✅ Track costs per pattern; use ensembles only for critical decisions</p>
<p>❌ <strong>Synchronous calls in ensemble</strong>: 3x latency penalty ✅ Use async/parallel calls for ensemble and specialist routing</p>
<p>❌ <strong>No observability in multi-model flows</strong>: Can&#x27;t debug or optimize ✅ Use Arize Phoenix or similar for span-level tracing</p>
<p>❌ <strong>Hard-coded model names in orchestration</strong>: Brittle to model updates ✅ Use configuration or model registry for flexibility</p>
<hr>
<h2>Hanzo SDK: Simplified Multi-Model Orchestration</h2>
<p>Instead of manually implementing orchestration patterns, use <strong>Hanzo SDK</strong> for automatic multi-model coordination with built-in routing, caching, and failover.</p>
<h3>Pipeline Pattern with Hanzo</h3>
<pre><code class="lang-python">from hanzo import Hanzo
# Initialize with different strategies for different stages
hanzo_fast = Hanzo(routing_strategy=&#x27;latency_optimized&#x27;)
hanzo_smart = Hanzo(routing_strategy=&#x27;quality_focused&#x27;)
async def research_pipeline(topic: str) -&gt; str:
# Stage 1: Fast outline generation
outline = await hanzo_fast.chat.completions.create_async(
messages=[{
&#x27;role&#x27;: &#x27;user&#x27;,
&#x27;content&#x27;: f&#x27;Create a research outline for: {topic}&#x27;
}]
)
# Stage 2: Quality content generation
content = await hanzo_smart.chat.completions.create_async(
messages=[{
&#x27;role&#x27;: &#x27;user&#x27;,
&#x27;content&#x27;: f&#x27;Write detailed content for: {outline.content}&#x27;
}]
)
return content.choices[0].message.content
# Hanzo automatically:
# - Routed stage 1 to fastest model (Gemini Flash)
# - Routed stage 2 to best model (GPT-4)
# - Handled retries and fallbacks
# - Tracked costs per stage</code></pre>
<h3>Ensemble Pattern with Hanzo</h3>
<pre><code class="lang-python">from hanzo import Hanzo
async def ensemble_decision(query: str) -&gt; str:
# Hanzo can query multiple models in parallel
hanzo = Hanzo(inference_mode=&#x27;hybrid&#x27;)
# Parallel calls to different models
responses = await hanzo.chat.completions.batch_create_async([
{&#x27;messages&#x27;: [{&#x27;role&#x27;: &#x27;user&#x27;, &#x27;content&#x27;: query}], &#x27;model&#x27;: &#x27;gpt-4&#x27;},
{&#x27;messages&#x27;: [{&#x27;role&#x27;: &#x27;user&#x27;, &#x27;content&#x27;: query}], &#x27;model&#x27;: &#x27;claude-3-5-sonnet&#x27;},
{&#x27;messages&#x27;: [{&#x27;role&#x27;: &#x27;user&#x27;, &#x27;content&#x27;: query}], &#x27;model&#x27;: &#x27;gemini-2.5-pro&#x27;}
])
# Extract answers
answers = [r.choices[0].message.content for r in responses]
# Aggregate with another model
final = await hanzo.chat.completions.create_async(
messages=[{
&#x27;role&#x27;: &#x27;user&#x27;,
&#x27;content&#x27;: f&#x27;Synthesize these expert opinions:\n\n&#x27; + &#x27;\n\n&#x27;.join(answers)
}],
model=&#x27;gpt-4&#x27; # Use best model for synthesis
)
return final.choices[0].message.content</code></pre>
<h3>Specialist Routing with Hanzo MCP</h3>
<pre><code class="lang-python">from hanzo import Hanzo
from hanzo.mcp import MCPClient
# Hanzo can use MCP tools for specialized tasks
hanzo = Hanzo(
inference_mode=&#x27;hybrid&#x27;,
mcp_servers=[&#x27;http://localhost:8081&#x27;]
)
mcp = MCPClient(&#x27;http://localhost:8081&#x27;)
async def route_by_intent(query: str) -&gt; str:
# First, classify intent
intent = await hanzo.chat.completions.create_async(
messages=[{
&#x27;role&#x27;: &#x27;user&#x27;,
&#x27;content&#x27;: f&#x27;Classify this query intent (code/search/analyze): {query}&#x27;
}],
model=&#x27;gpt-4o-mini&#x27; # Fast classifier
)
if &#x27;code&#x27; in intent.content.lower():
# Use code specialist model
return await hanzo.chat.completions.create_async(
messages=[{&#x27;role&#x27;: &#x27;user&#x27;, &#x27;content&#x27;: query}],
model=&#x27;claude-3-5-sonnet&#x27; # Best for code
)
elif &#x27;search&#x27; in intent.content.lower():
# Use MCP search tool
tools = mcp.get_tools()
return await hanzo.chat.completions.create_async(
messages=[{&#x27;role&#x27;: &#x27;user&#x27;, &#x27;content&#x27;: query}],
tools=tools,
tool_choice=&#x27;auto&#x27;
)
else:
# General analysis
return await hanzo.chat.completions.create_async(
messages=[{&#x27;role&#x27;: &#x27;user&#x27;, &#x27;content&#x27;: query}],
model=&#x27;gpt-4&#x27; # Best for analysis
)</code></pre>
<h3>Cascade Pattern with Hanzo</h3>
<pre><code class="lang-python">from hanzo import Hanzo
hanzo_local = Hanzo(inference_mode=&#x27;local&#x27;)
hanzo_cloud = Hanzo(inference_mode=&#x27;cloud&#x27;)
async def cascade_query(query: str, complexity_threshold: float = 0.7) -&gt; str:
# Try fast local model first
try:
response = await hanzo_local.chat.completions.create_async(
messages=[{&#x27;role&#x27;: &#x27;user&#x27;, &#x27;content&#x27;: query}],
model=&#x27;llama-3-8b&#x27;,
timeout=5 # Fast timeout
)
# Check confidence (if model provides it)
if hasattr(response, &#x27;confidence&#x27;) and response.confidence &gt; complexity_threshold:
return response.choices[0].message.content
except TimeoutError:
pass # Fall through to cloud
# Escalate to cloud model
response = await hanzo_cloud.chat.completions.create_async(
messages=[{&#x27;role&#x27;: &#x27;user&#x27;, &#x27;content&#x27;: query}],
model=&#x27;gpt-4&#x27; # Best model for complex queries
)
return response.choices[0].message.content</code></pre>
<h3>Comparison: Manual Orchestration vs Hanzo SDK</h3>
<p>| Pattern | Manual Implementation | Hanzo SDK | |---------|----------------------|-----------| | <strong>Pipeline</strong> | 50+ lines, manual routing | 10-15 lines, auto-routing | | <strong>Ensemble</strong> | Custom parallel execution | Built-in batch API | | <strong>Specialist</strong> | Manual intent classification | Automatic or MCP-based | | <strong>Cascade</strong> | Custom timeout/retry logic | Built-in with fallbacks | | <strong>Cost Tracking</strong> | Custom per-pattern | Automatic all patterns | | <strong>Caching</strong> | Manual between stages | Automatic semantic cache | | <strong>Observability</strong> | Manual OpenTelemetry | Built-in metrics | | <strong>Error Handling</strong> | Custom per model | Automatic retry/circuit breaker |</p>
<h3>Advanced: Multi-Model Workflow</h3>
<pre><code class="lang-python">from hanzo import Hanzo
from hanzo.workflows import Pipeline
# Define reusable workflow
class ResearchWorkflow(Pipeline):
def __init__(self):
super().__init__()
# Different Hanzo instances for different stages
self.fast_hanzo = Hanzo(routing_strategy=&#x27;latency_optimized&#x27;)
self.smart_hanzo = Hanzo(routing_strategy=&#x27;quality_focused&#x27;)
self.local_hanzo = Hanzo(inference_mode=&#x27;local&#x27;)
async def research(self, topic: str) -&gt; dict:
# Stage 1: Generate outline (fast)
outline = await self.fast_hanzo.chat.completions.create_async(
messages=[{&#x27;role&#x27;: &#x27;user&#x27;, &#x27;content&#x27;: f&#x27;Outline: {topic}&#x27;}]
)
# Stage 2: Parallel research on each section
sections = outline.content.split(&#x27;\n&#x27;)
research_tasks = [
self.local_hanzo.chat.completions.create_async(
messages=[{&#x27;role&#x27;: &#x27;user&#x27;, &#x27;content&#x27;: f&#x27;Research: {section}&#x27;}]
)
for section in sections
]
research_results = await asyncio.gather(*research_tasks)
# Stage 3: Synthesize with best model
synthesis = await self.smart_hanzo.chat.completions.create_async(
messages=[{
&#x27;role&#x27;: &#x27;user&#x27;,
&#x27;content&#x27;: f&#x27;Synthesize:\n\n&#x27; + &#x27;\n\n&#x27;.join([r.content for r in research_results])
}]
)
return {
&#x27;outline&#x27;: outline.content,
&#x27;research&#x27;: [r.content for r in research_results],
&#x27;synthesis&#x27;: synthesis.content,
&#x27;total_cost&#x27;: sum([
outline.cost,
sum(r.cost for r in research_results),
synthesis.cost
])
}
# Use workflow
workflow = ResearchWorkflow()
result = await workflow.research(&#x27;Quantum Computing Applications&#x27;)
print(f&quot;Total cost: ${result[&#x27;total_cost&#x27;]:.4f}&quot;)</code></pre>
<p><strong>Key Benefits</strong>:</p>
<ul><li><strong>10x less code</strong>: Hanzo SDK handles routing, retries, caching automatically</li><li><strong>Built-in observability</strong>: Automatic cost tracking and performance metrics</li><li><strong>Privacy-first</strong>: Local models via Hanzo Node before cloud escalation</li><li><strong>Production-ready</strong>: Circuit breakers, retries, fallbacks included</li><li><strong>Type-safe</strong>: Full TypeScript support with Python compatibility</li></ul>
<p><strong>See Also</strong>:</p>
<ul><li><strong>python-sdk.md</strong> - Complete Hanzo SDK documentation</li><li><strong>hanzo-mcp.md</strong> - MCP tool integration for workflows</li><li><strong>hanzo-node.md</strong> - Local model inference</li><li><strong>llm-model-routing.md</strong> - Automatic routing strategies</li></ul>
<hr>
<h2>Related Skills</h2>
<ul><li><code>llm-model-routing.md</code> - Routing strategies and RouteLLM framework</li><li><code>llm-model-selection.md</code> - Choosing models for capabilities and cost</li><li><code>api-error-handling.md</code> - Robust error handling across providers</li><li><code>observability-distributed-tracing.md</code> - OpenTelemetry and Phoenix</li><li><code>modal-web-endpoints.md</code> - Deploying orchestration as FastAPI services</li><li><code>redis-caching.md</code> - Shared state and caching for multi-model systems</li><li><strong><code>python-sdk.md</code></strong> - Hanzo SDK for multi-model orchestration</li><li><strong><code>hanzo-mcp.md</code></strong> - MCP tools for specialized tasks</li><li><strong><code>hanzo-node.md</code></strong> - Local inference infrastructure</li></ul>
<hr>
<p><strong>Last Updated</strong>: 2025-10-28 <strong>Format Version</strong>: 1.0 (Atomic)</p>
<footer>
<div class="fmap">
<div><b>Build</b>
<a href="https://hanzo.app">hanzo.app</a> app builder
<a href="https://hanzo.codes">hanzo.codes</a> code with an agent
<a href="https://hanzoskills.com">hanzoskills.com</a> what an agent reads</div>
<div><b>Run</b>
<a href="https://console.hanzo.ai">console.hanzo.ai</a> cloud console
<a href="https://api.hanzo.ai">api.hanzo.ai</a> the API</div>
<div><b>Account</b>
<a href="https://hanzo.id">hanzo.id</a> sign in
<a href="https://pay.hanzo.ai">pay.hanzo.ai</a> top up
<a href="https://billing.hanzo.ai">billing.hanzo.ai</a> invoices and usage</div>
<div><b>Company</b>
<a href="https://hanzo.ai">hanzo.ai</a> canonical
<a href="https://hanzo.ventures">hanzo.ventures</a> ventures</div>
</div>
<div class="fend"><a href="/ml-multi-model-orchestration.md">This page as plain text</a> · <a href="/llms.txt">Index for agents</a></div>
</footer>
</div></body></html>