mirror of
https://github.com/hanzoai/skills.git
synced 2026-11-19 00:47:36 +00:00
hanzoskills.com/llms.txt now opens with the install line, the API reference index and the company index, so an agent that lands on any one of the three can find the rest. robots.txt claimed 537 documents while llms.txt counts 920. The number is gone from robots.txt; llms.txt derives it. public/ is regenerated from scripts/site.py, which it had fallen behind: the copy-prompt menu and agent bar were in the generator and not in the committed pages. tests/site.mjs passes against the result. Co-authored-by: Hanzo Dev <[email protected]>
854 lines
37 KiB
HTML
854 lines
37 KiB
HTML
<!doctype html>
|
|
<html lang="en"><head>
|
|
<meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
|
|
<title>caching-caching-fundamentals</title>
|
|
<meta name="color-scheme" content="dark">
|
|
<style>
|
|
/* Committed to one visual world: a dark instrument panel. No light variant —
|
|
the dot field and the hairline rules only read on black, and a naive
|
|
inversion would wash both out. Every colour is painted explicitly so the page
|
|
never borrows a host background. */
|
|
@font-face{font-family:'Zen';src:url('/fonts/Zen-Variable.woff2') format('woff2');font-weight:100 900;font-style:normal;font-display:swap}
|
|
@font-face{font-family:'Zen';src:url('/fonts/Zen-Italic-Variable.woff2') format('woff2');font-weight:100 900;font-style:italic;font-display:swap}
|
|
@font-face{font-family:'Zen Mono';src:url('/fonts/ZenMono-Variable.woff2') format('woff2');font-weight:100 900;font-style:normal;font-display:swap}
|
|
@font-face{font-family:'Zen Mono';src:url('/fonts/ZenMono-Italic-Variable.woff2') format('woff2');font-weight:100 900;font-style:italic;font-display:swap}
|
|
@font-face{font-family:'Zen Pixel Square';src:url('/fonts/ZenPixel-Square.woff2') format('woff2');font-style:normal;font-display:swap}
|
|
|
|
:root{
|
|
--ink:#f4f4f2; --soft:#9c9c98; --faint:#6a6a66; --ground:#000; --panel:#0a0a0a;
|
|
--line:#232323; --line-hi:#3a3a3a; --accent:#f4f4f2;
|
|
--font-zen-sans:'Zen',-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif;
|
|
--font-zen-mono:'Zen Mono',ui-monospace,"SF Mono",SFMono-Regular,Menlo,Consolas,monospace;
|
|
--font-zen-pixel-square:'Zen Pixel Square',monospace;
|
|
--mono:var(--font-zen-mono);
|
|
--sans:var(--font-zen-sans);
|
|
/* Controls are pills and surfaces are soft, as on hanzo.ai. */
|
|
--r:999px; --r-soft:14px;
|
|
}
|
|
*{box-sizing:border-box}
|
|
html{-webkit-text-size-adjust:100%}
|
|
body{margin:0;background:var(--ground);color:var(--ink);font-family:var(--sans);
|
|
font-size:16px;line-height:1.65;-webkit-font-smoothing:antialiased}
|
|
.wrap{max-width:60rem;margin:0 auto;padding:0 1.75rem}
|
|
|
|
/* The dot field. A halftone made of one repeating radial-gradient, faded out
|
|
along the diagonal by a mask so it reads as a wave rather than wallpaper. */
|
|
/* No overflow clip here: the dot field is inset to the box already, and a clip
|
|
would cut off the copy-prompt menu. */
|
|
.hero{position:relative;border-bottom:1px solid var(--line)}
|
|
.hero::before{content:"";position:absolute;inset:0;
|
|
background-image:radial-gradient(var(--line-hi) 1px, transparent 1px);
|
|
background-size:7px 7px;
|
|
-webkit-mask-image:linear-gradient(115deg,#000 0%,rgba(0,0,0,.55) 38%,transparent 72%);
|
|
mask-image:linear-gradient(115deg,#000 0%,rgba(0,0,0,.55) 38%,transparent 72%);
|
|
pointer-events:none}
|
|
.hero .wrap{position:relative;padding-top:5.5rem;padding-bottom:4rem}
|
|
|
|
nav{border-bottom:1px solid var(--line)}
|
|
nav .in{max-width:60rem;margin:0 auto;padding:1.1rem 1.75rem;display:flex;
|
|
justify-content:space-between;align-items:center;gap:1rem}
|
|
.brand{font-family:var(--mono);font-size:.85rem;letter-spacing:.14em;
|
|
text-transform:uppercase;color:var(--ink);text-decoration:none;
|
|
display:inline-flex;align-items:center;gap:.6rem}
|
|
.brand i{font-style:normal;color:var(--faint)}
|
|
.mark{width:1.15em;height:1.15em;display:block;flex:0 0 auto}
|
|
nav a.up{font-family:var(--mono);font-size:.72rem;letter-spacing:.14em;
|
|
text-transform:uppercase;color:var(--soft);text-decoration:none}
|
|
nav a.up:hover{color:var(--ink)}
|
|
|
|
h1{font-family:var(--mono);font-size:clamp(2rem,5.4vw,3.4rem);line-height:1.12;
|
|
letter-spacing:-.02em;margin:0 0 1.1rem;font-weight:500;text-wrap:balance}
|
|
h1 em{font-style:normal;color:var(--faint)}
|
|
.lede{color:var(--soft);font-size:1.05rem;max-width:52ch;margin:0 0 2.25rem}
|
|
|
|
.btns{display:flex;flex-wrap:wrap;gap:.7rem;margin:0 0 .5rem;align-items:center}
|
|
.btn{display:inline-flex;align-items:center;gap:.5rem;font-family:var(--mono);
|
|
font-size:.72rem;letter-spacing:.14em;text-transform:uppercase;
|
|
padding:.8rem 1.4rem;border:1px solid var(--line-hi);border-radius:var(--r);
|
|
color:var(--ink);text-decoration:none;background:transparent;cursor:pointer;
|
|
line-height:1.2;font-weight:400}
|
|
.btn:hover{background:#141414;text-decoration:none;border-color:var(--soft)}
|
|
.btn.solid{background:var(--ink);color:#000;border-color:var(--ink)}
|
|
.btn.solid:hover{background:#d8d8d4}
|
|
.btn .chev{width:.7em;height:.7em;flex:0 0 auto;transition:transform .15s ease}
|
|
.btn[aria-expanded="true"] .chev{transform:rotate(180deg)}
|
|
|
|
/* Copy-prompt selector. Each row copies the prompt for its own target. */
|
|
.copy{position:relative;display:inline-flex}
|
|
.menu{position:absolute;top:calc(100% + .5rem);left:0;z-index:20;min-width:21rem;
|
|
background:var(--panel);border:1px solid var(--line-hi);border-radius:var(--r-soft);
|
|
padding:.4rem;display:flex;flex-direction:column;
|
|
box-shadow:0 12px 32px rgba(0,0,0,.75)}
|
|
.menu[hidden]{display:none}
|
|
.mi{display:flex;align-items:center;gap:.7rem;width:100%;text-align:left;
|
|
padding:.62rem .8rem;border:0;border-radius:calc(var(--r-soft) - .35rem);
|
|
background:transparent;color:var(--ink);font-family:var(--mono);font-size:.76rem;
|
|
letter-spacing:.04em;text-decoration:none;cursor:pointer;line-height:1.3;
|
|
white-space:nowrap}
|
|
.mi:hover,.mi:focus-visible{background:#181818;outline:none}
|
|
.mi .mark{width:1.05rem;height:1.05rem;color:var(--soft)}
|
|
.mi:hover .mark{color:var(--ink)}
|
|
.mi u{margin-left:auto;color:var(--faint);text-decoration:none;font-size:.68rem;
|
|
letter-spacing:.1em;text-transform:uppercase}
|
|
|
|
/* Works-with bar: one scrolling row, name and mark per agent. */
|
|
.strip{border-bottom:1px solid var(--line);background:var(--panel)}
|
|
.strip .in{max-width:60rem;margin:0 auto;padding:.9rem 1.75rem;display:flex;
|
|
align-items:center;gap:1.25rem;font-family:var(--mono);font-size:.72rem;
|
|
letter-spacing:.1em;text-transform:uppercase;color:var(--faint)}
|
|
.strip b{color:var(--soft);font-weight:400;flex:0 0 auto;
|
|
border-right:1px solid var(--line);padding-right:1.25rem;align-self:stretch;
|
|
display:flex;align-items:center}
|
|
.bar{display:flex;align-items:center;gap:.5rem;flex:1 1 auto;min-width:0;
|
|
overflow-x:auto;scrollbar-width:none;scroll-snap-type:x proximity;
|
|
-webkit-overflow-scrolling:touch;
|
|
-webkit-mask-image:linear-gradient(90deg,#000 calc(100% - 3rem),transparent);
|
|
mask-image:linear-gradient(90deg,#000 calc(100% - 3rem),transparent)}
|
|
.bar::-webkit-scrollbar{display:none}
|
|
.agent{display:inline-flex;align-items:center;gap:.45rem;flex:0 0 auto;
|
|
padding:.4rem .85rem;border:1px solid var(--line);border-radius:var(--r);
|
|
color:var(--soft);white-space:nowrap;scroll-snap-align:start;
|
|
text-decoration:none;background:var(--ground)}
|
|
.agent:hover{color:var(--ink);border-color:var(--line-hi);text-decoration:none}
|
|
.agent .mark{width:.95rem;height:.95rem}
|
|
|
|
h2{font-family:var(--mono);font-size:1.1rem;letter-spacing:-.01em;margin:2.75rem 0 .75rem;font-weight:500}
|
|
h3{font-size:1rem;margin:2rem 0 .5rem;font-weight:600}
|
|
h4,h5,h6{font-size:.94rem;margin:1.5rem 0 .5rem;font-weight:600}
|
|
p{margin:0 0 1rem;max-width:70ch}
|
|
ul,ol{margin:0 0 1rem;padding-left:1.35rem;max-width:70ch}
|
|
li{margin:.22rem 0}
|
|
a{color:var(--ink);text-decoration:underline;text-underline-offset:3px;
|
|
text-decoration-color:var(--line-hi)}
|
|
a:hover{text-decoration-color:var(--ink)}
|
|
a:focus-visible{outline:1px solid var(--ink);outline-offset:3px}
|
|
code{font-family:var(--mono);font-size:.85em;background:var(--panel);
|
|
border:1px solid var(--line);border-radius:5px;padding:.1em .36em}
|
|
pre{background:var(--panel);border:1px solid var(--line);padding:1rem 1.1rem;
|
|
border-radius:var(--r-soft);overflow-x:auto;margin:0 0 1.25rem}
|
|
pre code{background:none;border:0;padding:0;font-size:.82rem;line-height:1.7}
|
|
blockquote{margin:0 0 1rem;padding-left:1rem;border-left:1px solid var(--line-hi);color:var(--soft)}
|
|
hr{border:0;border-top:1px solid var(--line);margin:2.5rem 0}
|
|
table{border-collapse:collapse;width:100%;margin:0 0 1.25rem;font-size:.92rem;
|
|
display:block;overflow-x:auto}
|
|
th,td{border:1px solid var(--line);padding:.5rem .7rem;text-align:left}
|
|
|
|
.curl{display:block;background:var(--panel);border:1px solid var(--line);
|
|
border-radius:var(--r-soft);padding:.95rem 1.1rem;font-family:var(--mono);
|
|
font-size:.8rem;color:var(--soft);overflow-x:auto;margin:0 0 2.5rem;
|
|
white-space:pre;line-height:1.9}
|
|
.curl b{color:var(--ink);font-weight:400}
|
|
.curl i{font-style:normal;color:var(--faint)}
|
|
|
|
/* index */
|
|
#q{width:100%;padding:.85rem 1.2rem;font-family:var(--mono);font-size:.85rem;color:var(--ink);
|
|
background:var(--panel);border:1px solid var(--line);border-radius:var(--r);margin:0 0 .6rem}
|
|
#q::placeholder{color:var(--faint);text-transform:uppercase;letter-spacing:.1em;font-size:.72rem}
|
|
#q:focus{outline:none;border-color:var(--soft)}
|
|
#count{color:var(--faint);font-family:var(--mono);font-size:.72rem;letter-spacing:.1em;
|
|
text-transform:uppercase;margin:0 0 2rem}
|
|
.cat{margin:2.5rem 0 .5rem;font-family:var(--mono);font-size:.7rem;letter-spacing:.16em;
|
|
text-transform:uppercase;color:var(--faint);border-bottom:1px solid var(--line);
|
|
padding-bottom:.45rem}
|
|
.row{display:block;padding:.6rem .9rem;margin:0 -.9rem;border-bottom:1px solid var(--line);
|
|
border-radius:var(--r-soft);text-decoration:none}
|
|
.row:hover{background:#0a0a0a}
|
|
.nm{font-family:var(--mono);font-size:.84rem;color:var(--ink);display:block}
|
|
.ds{color:var(--soft);font-size:.9rem;display:block;margin-top:.12rem}
|
|
footer{margin-top:4rem;padding:2rem 0 4rem;border-top:1px solid var(--line);
|
|
color:var(--faint);font-size:.85rem}
|
|
footer a{color:var(--soft)}
|
|
.fmap{display:grid;gap:1.75rem;grid-template-columns:repeat(auto-fit,minmax(11rem,1fr));
|
|
margin:0 0 2rem;font-size:.85rem}
|
|
.fmap b{display:block;font-family:var(--mono);font-size:.68rem;letter-spacing:.16em;
|
|
text-transform:uppercase;color:var(--faint);font-weight:400;margin:0 0 .6rem;
|
|
border-bottom:1px solid var(--line);padding-bottom:.4rem}
|
|
.fmap a{display:inline-block;color:var(--ink);text-decoration:none;margin:.3rem .35rem 0 0}
|
|
.fmap a:hover{text-decoration:underline}
|
|
.fmap div{color:var(--faint);line-height:1.5}
|
|
.fend{border-top:1px solid var(--line);padding-top:1.25rem;font-size:.82rem}
|
|
|
|
@media (max-width:640px){
|
|
.hero .wrap{padding-top:3.25rem;padding-bottom:2.75rem}
|
|
.btns{gap:.5rem}
|
|
.btn{padding:.75rem 1.15rem;font-size:.68rem}
|
|
.copy,.copy .btn{width:100%;justify-content:center}
|
|
.menu{left:0;right:0;min-width:0}
|
|
.strip .in{flex-direction:column;align-items:flex-start;gap:.6rem;padding:.9rem 1.75rem}
|
|
.strip b{border-right:0;padding-right:0}
|
|
.bar{width:100%}
|
|
}
|
|
</style>
|
|
<link rel="alternate" type="text/markdown" href="/caching-caching-fundamentals.md">
|
|
|
|
<link rel="icon" href="/favicon.svg">
|
|
<link rel="canonical" href="https://hanzoskills.com/caching-caching-fundamentals">
|
|
</head>
|
|
<!-- Reading this with a machine? Every page here is also plain markdown.
|
|
This one: https://hanzoskills.com/caching-caching-fundamentals.md
|
|
Index: https://hanzoskills.com/llms.txt
|
|
Corpus: https://hanzoskills.com/all.md
|
|
Append .md to any path for the markdown twin. -->
|
|
<body>
|
|
<nav><div class="in">
|
|
<a class="brand" href="/"><svg class="mark" viewBox="0 0 67 67" fill="currentColor" aria-hidden="true"><path d="M22.21 67V44.6369H0V67H22.21Z"/><path d="M66.7038 22.3184H22.2534L0.0878906 44.6367H44.4634L66.7038 22.3184Z"/><path d="M22.21 0H0V22.3184H22.21V0Z"/><path d="M66.7198 0H44.5098V22.3184H66.7198V0Z"/><path d="M66.7198 67V44.6369H44.5098V67H66.7198Z"/></svg> HANZO<i>SKILLS</i></a>
|
|
<a class="up" href="/">All skills</a>
|
|
</div></nav>
|
|
|
|
<div class="wrap">
|
|
<h1>caching-caching-fundamentals</h1><p class="lede">Core caching concepts, patterns, eviction policies, and cache design principles for optimizing application performance</p><h1>Caching Fundamentals</h1>
|
|
<p><strong>Last Updated</strong>: 2025-10-25</p>
|
|
<h2>When to Use This Skill</h2>
|
|
<p><strong>Activate when</strong>:</p>
|
|
<ul><li>Designing caching strategies for applications</li><li>Optimizing application performance and reducing latency</li><li>Reducing database load and API calls</li><li>Understanding cache behavior and tradeoffs</li><li>Choosing appropriate caching patterns</li><li>Implementing cache eviction policies</li><li>Debugging cache-related issues</li></ul>
|
|
<p><strong>Prerequisites</strong>: Basic understanding of databases, APIs, and application architecture</p>
|
|
<p><strong>Related Skills</strong>: <code>http-caching.md</code>, <code>cdn-edge-caching.md</code>, <code>redis-caching-patterns.md</code>, <code>cache-invalidation-strategies.md</code></p>
|
|
<hr>
|
|
<h2>Core Concepts</h2>
|
|
<h3>What is Caching?</h3>
|
|
<p><strong>Cache</strong>: Temporary storage layer that stores frequently accessed data for fast retrieval</p>
|
|
<p><strong>Purpose</strong>:</p>
|
|
<ul><li>Reduce latency (faster data access)</li><li>Decrease load on backend systems (databases, APIs)</li><li>Improve scalability (handle more requests)</li><li>Reduce costs (fewer database queries, API calls)</li></ul>
|
|
<p><strong>Cache Hierarchy</strong>:</p>
|
|
<pre><code>Browser Cache → CDN/Edge Cache → Application Cache → Database Cache
|
|
(ms) (10-50ms) (1-10ms) (μs)</code></pre>
|
|
<h3>Cache Types</h3>
|
|
<pre><code class="lang-python">from enum import Enum
|
|
from typing import Optional, Dict, Any
|
|
|
|
class CacheType(Enum):
|
|
"""Different cache deployment types"""
|
|
LOCAL = "local" # In-process memory
|
|
DISTRIBUTED = "distributed" # Shared across instances
|
|
EDGE = "edge" # CDN, geographically distributed
|
|
|
|
class CacheLocation:
|
|
"""Cache location characteristics"""
|
|
|
|
@staticmethod
|
|
def local_cache():
|
|
"""
|
|
Local/In-Process Cache
|
|
|
|
Pros: Fastest access, no network overhead
|
|
Cons: Not shared, limited by process memory
|
|
Use: Computation results, configuration
|
|
"""
|
|
return {
|
|
"latency": "1-10 μs",
|
|
"shared": False,
|
|
"persistence": False,
|
|
"examples": ["@lru_cache", "dict", "Redis in same host"]
|
|
}
|
|
|
|
@staticmethod
|
|
def distributed_cache():
|
|
"""
|
|
Distributed Cache
|
|
|
|
Pros: Shared across instances, scalable
|
|
Cons: Network latency, complexity
|
|
Use: Session data, API responses
|
|
"""
|
|
return {
|
|
"latency": "1-10 ms",
|
|
"shared": True,
|
|
"persistence": "optional",
|
|
"examples": ["Redis Cluster", "Memcached"]
|
|
}
|
|
|
|
@staticmethod
|
|
def edge_cache():
|
|
"""
|
|
Edge/CDN Cache
|
|
|
|
Pros: Global distribution, reduce origin load
|
|
Cons: Invalidation complexity
|
|
Use: Static assets, public content
|
|
"""
|
|
return {
|
|
"latency": "10-50 ms",
|
|
"shared": True,
|
|
"persistence": True,
|
|
"examples": ["Cloudflare", "Fastly", "CloudFront"]
|
|
}</code></pre>
|
|
<hr>
|
|
<h2>Caching Patterns</h2>
|
|
<h3>1. Cache-Aside (Lazy Loading)</h3>
|
|
<p><strong>Concept</strong>: Application checks cache first, loads from source on miss, then populates cache</p>
|
|
<p><strong>Flow</strong>:</p>
|
|
<pre><code>1. Check cache for data
|
|
2. If HIT → return cached data
|
|
3. If MISS → query source (DB/API)
|
|
4. Store result in cache
|
|
5. Return data</code></pre>
|
|
<p><strong>Implementation</strong>:</p>
|
|
<pre><code class="lang-python">from functools import wraps
|
|
from typing import Callable, Any
|
|
import time
|
|
|
|
class CacheAside:
|
|
"""Cache-Aside (Lazy Loading) pattern"""
|
|
|
|
def __init__(self):
|
|
self.cache: Dict[str, tuple[Any, float]] = {}
|
|
self.ttl = 300 # 5 minutes
|
|
|
|
def get(self, key: str) -> Optional[Any]:
|
|
"""Get from cache if not expired"""
|
|
if key in self.cache:
|
|
value, timestamp = self.cache[key]
|
|
if time.time() - timestamp < self.ttl:
|
|
return value
|
|
else:
|
|
# Expired
|
|
del self.cache[key]
|
|
return None
|
|
|
|
def set(self, key: str, value: Any):
|
|
"""Set value in cache with timestamp"""
|
|
self.cache[key] = (value, time.time())
|
|
|
|
def decorator(self, key_func: Callable = None):
|
|
"""Decorator for cache-aside pattern"""
|
|
def decorator_wrapper(func):
|
|
@wraps(func)
|
|
def wrapper(*args, **kwargs):
|
|
# Generate cache key
|
|
if key_func:
|
|
cache_key = key_func(*args, **kwargs)
|
|
else:
|
|
cache_key = f"{func.__name__}:{args}:{kwargs}"
|
|
|
|
# Check cache
|
|
cached_value = self.get(cache_key)
|
|
if cached_value is not None:
|
|
return cached_value
|
|
|
|
# Cache miss - call function
|
|
result = func(*args, **kwargs)
|
|
|
|
# Store in cache
|
|
self.set(cache_key, result)
|
|
|
|
return result
|
|
return wrapper
|
|
return decorator_wrapper
|
|
|
|
# Usage
|
|
cache = CacheAside()
|
|
|
|
@cache.decorator(key_func=lambda user_id: f"user:{user_id}")
|
|
def get_user(user_id: int):
|
|
"""Fetch user from database (expensive operation)"""
|
|
print(f"Database query for user {user_id}")
|
|
# Simulate DB query
|
|
time.sleep(0.1)
|
|
return {"id": user_id, "name": f"User {user_id}"}
|
|
|
|
# First call - cache miss
|
|
user1 = get_user(1) # Prints "Database query..."
|
|
|
|
# Second call - cache hit
|
|
user1_again = get_user(1) # No print, returns from cache</code></pre>
|
|
<p><strong>Pros</strong>:</p>
|
|
<ul><li>Only caches requested data (efficient memory use)</li><li>Application controls caching logic</li><li>Resilient (cache failures don't block requests)</li></ul>
|
|
<p><strong>Cons</strong>:</p>
|
|
<ul><li>Initial request latency (cache miss penalty)</li><li>Potential cache stampede on popular keys</li><li>Stale data during TTL window</li></ul>
|
|
<h3>2. Write-Through</h3>
|
|
<p><strong>Concept</strong>: Write to cache and source simultaneously</p>
|
|
<p><strong>Flow</strong>:</p>
|
|
<pre><code>1. Write data to cache
|
|
2. Write data to source (DB/API)
|
|
3. Return success only when both complete</code></pre>
|
|
<p><strong>Implementation</strong>:</p>
|
|
<pre><code class="lang-python">class WriteThroughCache:
|
|
"""Write-Through caching pattern"""
|
|
|
|
def __init__(self, database):
|
|
self.cache: Dict[str, Any] = {}
|
|
self.db = database
|
|
|
|
def get(self, key: str) -> Optional[Any]:
|
|
"""Read from cache, fall back to DB"""
|
|
if key in self.cache:
|
|
return self.cache[key]
|
|
|
|
# Cache miss - load from DB
|
|
value = self.db.get(key)
|
|
if value is not None:
|
|
self.cache[key] = value
|
|
return value
|
|
|
|
def set(self, key: str, value: Any):
|
|
"""Write to both cache and DB"""
|
|
# Write to cache first
|
|
self.cache[key] = value
|
|
|
|
# Then write to database
|
|
self.db.set(key, value)
|
|
|
|
# Both must succeed for consistency
|
|
|
|
# Usage
|
|
class MockDatabase:
|
|
def __init__(self):
|
|
self.data = {}
|
|
|
|
def get(self, key):
|
|
return self.data.get(key)
|
|
|
|
def set(self, key, value):
|
|
self.data[key] = value
|
|
|
|
db = MockDatabase()
|
|
cache = WriteThroughCache(db)
|
|
|
|
# Write
|
|
cache.set("user:1", {"name": "Alice"})
|
|
# Cache and DB both updated
|
|
|
|
# Read
|
|
user = cache.get("user:1") # From cache</code></pre>
|
|
<p><strong>Pros</strong>:</p>
|
|
<ul><li>Data consistency (cache always matches source)</li><li>No cache miss penalty on reads</li><li>Simplifies cache warming</li></ul>
|
|
<p><strong>Cons</strong>:</p>
|
|
<ul><li>Write latency (two operations)</li><li>Wasted cache space (all writes cached, even if never read)</li><li>Write failures affect both layers</li></ul>
|
|
<h3>3. Write-Behind (Write-Back)</h3>
|
|
<p><strong>Concept</strong>: Write to cache immediately, asynchronously write to source</p>
|
|
<p><strong>Flow</strong>:</p>
|
|
<pre><code>1. Write data to cache
|
|
2. Return success immediately
|
|
3. Asynchronously batch writes to source</code></pre>
|
|
<p><strong>Implementation</strong>:</p>
|
|
<pre><code class="lang-python">import asyncio
|
|
from collections import deque
|
|
from dataclasses import dataclass
|
|
from typing import Deque
|
|
|
|
@dataclass
|
|
class WriteOperation:
|
|
"""Pending write operation"""
|
|
key: str
|
|
value: Any
|
|
timestamp: float
|
|
|
|
class WriteBehindCache:
|
|
"""Write-Behind (Write-Back) caching pattern"""
|
|
|
|
def __init__(self, database, batch_size=10, flush_interval=5.0):
|
|
self.cache: Dict[str, Any] = {}
|
|
self.db = database
|
|
self.write_queue: Deque[WriteOperation] = deque()
|
|
self.batch_size = batch_size
|
|
self.flush_interval = flush_interval
|
|
self.running = False
|
|
|
|
async def start(self):
|
|
"""Start background flushing"""
|
|
self.running = True
|
|
while self.running:
|
|
await asyncio.sleep(self.flush_interval)
|
|
await self.flush()
|
|
|
|
async def flush(self):
|
|
"""Flush pending writes to database"""
|
|
if not self.write_queue:
|
|
return
|
|
|
|
# Batch writes
|
|
batch = []
|
|
while self.write_queue and len(batch) < self.batch_size:
|
|
batch.append(self.write_queue.popleft())
|
|
|
|
# Write batch to database
|
|
for op in batch:
|
|
self.db.set(op.key, op.value)
|
|
|
|
print(f"Flushed {len(batch)} writes to database")
|
|
|
|
def get(self, key: str) -> Optional[Any]:
|
|
"""Read from cache"""
|
|
return self.cache.get(key)
|
|
|
|
def set(self, key: str, value: Any):
|
|
"""Write to cache, queue for DB write"""
|
|
# Immediate write to cache
|
|
self.cache[key] = value
|
|
|
|
# Queue for async DB write
|
|
self.write_queue.append(
|
|
WriteOperation(key, value, time.time())
|
|
)</code></pre>
|
|
<p><strong>Pros</strong>:</p>
|
|
<ul><li>Fast writes (no wait for DB)</li><li>Batching reduces DB load</li><li>Better write throughput</li></ul>
|
|
<p><strong>Cons</strong>:</p>
|
|
<ul><li>Risk of data loss (if cache crashes before flush)</li><li>Complexity (background jobs, retry logic)</li><li>Eventual consistency</li></ul>
|
|
<h3>4. Read-Through</h3>
|
|
<p><strong>Concept</strong>: Cache automatically loads data from source on miss</p>
|
|
<p><strong>Implementation</strong>:</p>
|
|
<pre><code class="lang-python">class ReadThroughCache:
|
|
"""Read-Through caching pattern"""
|
|
|
|
def __init__(self, loader_func: Callable[[str], Any]):
|
|
self.cache: Dict[str, Any] = {}
|
|
self.loader = loader_func # Function to load from source
|
|
|
|
def get(self, key: str) -> Any:
|
|
"""
|
|
Get value, automatically loading on miss
|
|
|
|
Cache handles loading - application doesn't know about source
|
|
"""
|
|
if key in self.cache:
|
|
return self.cache[key]
|
|
|
|
# Cache miss - load from source
|
|
value = self.loader(key)
|
|
|
|
# Store in cache
|
|
self.cache[key] = value
|
|
|
|
return value
|
|
|
|
# Usage
|
|
def load_user_from_db(user_id: str):
|
|
"""Loader function"""
|
|
print(f"Loading user {user_id} from database")
|
|
return {"id": user_id, "name": f"User {user_id}"}
|
|
|
|
cache = ReadThroughCache(loader_func=load_user_from_db)
|
|
|
|
# Application doesn't handle cache misses
|
|
user = cache.get("user:1") # Automatically loads if not cached</code></pre>
|
|
<hr>
|
|
<h2>Eviction Policies</h2>
|
|
<h3>LRU (Least Recently Used)</h3>
|
|
<p><strong>Concept</strong>: Evict least recently accessed item when cache is full</p>
|
|
<p><strong>Implementation</strong>:</p>
|
|
<pre><code class="lang-python">from collections import OrderedDict
|
|
|
|
class LRUCache:
|
|
"""LRU Cache with O(1) get and set"""
|
|
|
|
def __init__(self, capacity: int):
|
|
self.capacity = capacity
|
|
self.cache = OrderedDict()
|
|
|
|
def get(self, key: str) -> Optional[Any]:
|
|
"""Get value and mark as recently used"""
|
|
if key not in self.cache:
|
|
return None
|
|
|
|
# Move to end (most recent)
|
|
self.cache.move_to_end(key)
|
|
return self.cache[key]
|
|
|
|
def set(self, key: str, value: Any):
|
|
"""Set value, evict LRU if at capacity"""
|
|
if key in self.cache:
|
|
# Update existing
|
|
self.cache.move_to_end(key)
|
|
else:
|
|
# New key
|
|
if len(self.cache) >= self.capacity:
|
|
# Evict least recently used (first item)
|
|
self.cache.popitem(last=False)
|
|
|
|
self.cache[key] = value
|
|
|
|
def __len__(self):
|
|
return len(self.cache)
|
|
|
|
# Usage
|
|
lru = LRUCache(capacity=3)
|
|
|
|
lru.set("a", 1)
|
|
lru.set("b", 2)
|
|
lru.set("c", 3)
|
|
|
|
lru.get("a") # Access 'a', now most recent
|
|
|
|
lru.set("d", 4) # Evicts 'b' (least recently used)
|
|
|
|
print("b" in lru.cache) # False
|
|
print("a" in lru.cache) # True</code></pre>
|
|
<p><strong>Use when</strong>: Access patterns favor recent items (temporal locality)</p>
|
|
<h3>LFU (Least Frequently Used)</h3>
|
|
<p><strong>Concept</strong>: Evict least frequently accessed item</p>
|
|
<p><strong>Implementation</strong>:</p>
|
|
<pre><code class="lang-python">from collections import defaultdict
|
|
import heapq
|
|
|
|
class LFUCache:
|
|
"""LFU Cache with frequency tracking"""
|
|
|
|
def __init__(self, capacity: int):
|
|
self.capacity = capacity
|
|
self.cache: Dict[str, Any] = {}
|
|
self.frequency: Dict[str, int] = defaultdict(int)
|
|
self.access_time: Dict[str, int] = {}
|
|
self.time = 0
|
|
|
|
def get(self, key: str) -> Optional[Any]:
|
|
"""Get value and increment frequency"""
|
|
if key not in self.cache:
|
|
return None
|
|
|
|
self.frequency[key] += 1
|
|
self.time += 1
|
|
self.access_time[key] = self.time
|
|
|
|
return self.cache[key]
|
|
|
|
def set(self, key: str, value: Any):
|
|
"""Set value, evict LFU if at capacity"""
|
|
if self.capacity == 0:
|
|
return
|
|
|
|
if key in self.cache:
|
|
self.cache[key] = value
|
|
self.frequency[key] += 1
|
|
self.time += 1
|
|
self.access_time[key] = self.time
|
|
return
|
|
|
|
if len(self.cache) >= self.capacity:
|
|
# Find least frequently used
|
|
# Break ties by least recently used
|
|
lfu_key = min(
|
|
self.cache.keys(),
|
|
key=lambda k: (self.frequency[k], self.access_time[k])
|
|
)
|
|
|
|
del self.cache[lfu_key]
|
|
del self.frequency[lfu_key]
|
|
del self.access_time[lfu_key]
|
|
|
|
self.cache[key] = value
|
|
self.frequency[key] = 1
|
|
self.time += 1
|
|
self.access_time[key] = self.time</code></pre>
|
|
<p><strong>Use when</strong>: Some items accessed much more frequently than others</p>
|
|
<h3>TTL (Time To Live)</h3>
|
|
<p><strong>Concept</strong>: Evict items after fixed time period</p>
|
|
<pre><code class="lang-python">import time
|
|
|
|
class TTLCache:
|
|
"""TTL-based cache with automatic expiration"""
|
|
|
|
def __init__(self, default_ttl: float = 300):
|
|
self.cache: Dict[str, tuple[Any, float]] = {}
|
|
self.default_ttl = default_ttl
|
|
|
|
def get(self, key: str) -> Optional[Any]:
|
|
"""Get value if not expired"""
|
|
if key not in self.cache:
|
|
return None
|
|
|
|
value, expiry = self.cache[key]
|
|
|
|
if time.time() > expiry:
|
|
# Expired
|
|
del self.cache[key]
|
|
return None
|
|
|
|
return value
|
|
|
|
def set(self, key: str, value: Any, ttl: Optional[float] = None):
|
|
"""Set value with TTL"""
|
|
if ttl is None:
|
|
ttl = self.default_ttl
|
|
|
|
expiry = time.time() + ttl
|
|
self.cache[key] = (value, expiry)
|
|
|
|
def cleanup(self):
|
|
"""Remove all expired entries"""
|
|
now = time.time()
|
|
expired = [k for k, (_, exp) in self.cache.items() if now > exp]
|
|
for k in expired:
|
|
del self.cache[k]</code></pre>
|
|
<p><strong>Use when</strong>: Data has natural expiration (sessions, temporary tokens)</p>
|
|
<hr>
|
|
<h2>Cache Key Design</h2>
|
|
<h3>Best Practices</h3>
|
|
<pre><code class="lang-python">class CacheKeyDesign:
|
|
"""Cache key naming best practices"""
|
|
|
|
@staticmethod
|
|
def hierarchical_key(namespace: str, entity: str, id: str) -> str:
|
|
"""
|
|
Hierarchical naming for organization
|
|
|
|
Pattern: namespace:entity:id
|
|
Example: app:user:123, api:product:456
|
|
"""
|
|
return f"{namespace}:{entity}:{id}"
|
|
|
|
@staticmethod
|
|
def composite_key(*parts) -> str:
|
|
"""
|
|
Composite key from multiple values
|
|
|
|
Example: user_posts(user_id, page) → "posts:user:123:page:1"
|
|
"""
|
|
return ":".join(str(p) for p in parts)
|
|
|
|
@staticmethod
|
|
def hash_key(data: str) -> str:
|
|
"""
|
|
Hash long or complex keys
|
|
|
|
Use for: Query strings, JSON, URLs
|
|
"""
|
|
import hashlib
|
|
return hashlib.sha256(data.encode()).hexdigest()[:16]
|
|
|
|
@staticmethod
|
|
def version_key(key: str, version: int) -> str:
|
|
"""
|
|
Versioned keys for invalidation
|
|
|
|
Increment version to invalidate all old keys
|
|
"""
|
|
return f"{key}:v{version}"
|
|
|
|
# Examples
|
|
keys = CacheKeyDesign()
|
|
|
|
# User data
|
|
user_key = keys.hierarchical_key("app", "user", "123")
|
|
# "app:user:123"
|
|
|
|
# Paginated results
|
|
posts_key = keys.composite_key("posts", "user", 123, "page", 1)
|
|
# "posts:user:123:page:1"
|
|
|
|
# Complex query
|
|
query = "SELECT * FROM users WHERE age > 25 AND city = 'NYC'"
|
|
query_key = f"query:{keys.hash_key(query)}"
|
|
|
|
# Versioned cache
|
|
config_key = keys.version_key("app:config", version=2)
|
|
# "app:config:v2"</code></pre>
|
|
<hr>
|
|
<h2>When to Cache vs Not Cache</h2>
|
|
<h3>Cache These</h3>
|
|
<pre><code class="lang-python">class GoodCacheCandidates:
|
|
"""Data that benefits from caching"""
|
|
|
|
EXAMPLES = {
|
|
"Expensive computations": {
|
|
"example": "ML model inference, complex calculations",
|
|
"ttl": "hours to days",
|
|
"pattern": "Cache-Aside"
|
|
},
|
|
"Frequently accessed data": {
|
|
"example": "User profiles, product catalogs",
|
|
"ttl": "minutes to hours",
|
|
"pattern": "Read-Through"
|
|
},
|
|
"Slow external API calls": {
|
|
"example": "Third-party APIs, microservices",
|
|
"ttl": "minutes",
|
|
"pattern": "Cache-Aside"
|
|
},
|
|
"Static or rarely changing": {
|
|
"example": "Configuration, reference data",
|
|
"ttl": "hours to days",
|
|
"pattern": "Write-Through"
|
|
},
|
|
"High read-to-write ratio": {
|
|
"example": "News articles, blog posts",
|
|
"ttl": "minutes to hours",
|
|
"pattern": "Cache-Aside"
|
|
}
|
|
}</code></pre>
|
|
<h3>Don't Cache These</h3>
|
|
<pre><code class="lang-python">class PoorCacheCandidates:
|
|
"""Data that should NOT be cached"""
|
|
|
|
EXAMPLES = [
|
|
"Highly personalized data (unless user-keyed)",
|
|
"Rapidly changing data (stock prices, live scores)",
|
|
"Large objects (>1MB, unless CDN)",
|
|
"Data accessed once (no reuse benefit)",
|
|
"Security-sensitive data (PII, passwords)",
|
|
"Already fast queries (<10ms)",
|
|
]</code></pre>
|
|
<hr>
|
|
<h2>Common Anti-Patterns</h2>
|
|
<h3>❌ No Cache Expiration</h3>
|
|
<pre><code class="lang-python"># WRONG: Cache lives forever
|
|
cache = {}
|
|
cache[key] = value # Never expires
|
|
|
|
# CORRECT: Set TTL
|
|
cache.set(key, value, ttl=300)</code></pre>
|
|
<h3>❌ Caching Failures</h3>
|
|
<pre><code class="lang-python"># WRONG: Cache error responses
|
|
result = api_call()
|
|
cache.set(key, result) # What if result is error?
|
|
|
|
# CORRECT: Only cache successful responses
|
|
result = api_call()
|
|
if result.success:
|
|
cache.set(key, result)</code></pre>
|
|
<h3>❌ Cache Stampede</h3>
|
|
<pre><code class="lang-python"># WRONG: All requests miss simultaneously
|
|
# (e.g., cache expires at exact time)
|
|
|
|
# CORRECT: Probabilistic early expiration
|
|
import random
|
|
|
|
def get_with_early_expiration(key, ttl):
|
|
value, expiry = cache.get_with_expiry(key)
|
|
|
|
# Probabilistically refresh before expiry
|
|
time_left = expiry - time.time()
|
|
if time_left < ttl * random.random():
|
|
# Refresh cache
|
|
value = fetch_fresh_data(key)
|
|
cache.set(key, value, ttl)
|
|
|
|
return value</code></pre>
|
|
<hr>
|
|
<h2>Quick Reference</h2>
|
|
<h3>Cache Pattern Selection</h3>
|
|
<p>| Pattern | Read Speed | Write Speed | Consistency | Use Case | |---------|-----------|-------------|-------------|----------| | Cache-Aside | Medium (miss penalty) | Fast | Eventual | General purpose, read-heavy | | Write-Through | Fast | Slow | Strong | Consistent reads required | | Write-Behind | Fast | Very Fast | Eventual | High write throughput | | Read-Through | Fast | N/A | Eventual | Simplified read logic |</p>
|
|
<h3>Eviction Policy Selection</h3>
|
|
<p>| Policy | Best For | Worst For | |--------|----------|-----------| | LRU | Temporal locality | Scanning workloads | | LFU | Skewed access patterns | Changing patterns | | TTL | Time-sensitive data | Static data | | FIFO | Fair eviction | Performance optimization |</p>
|
|
<hr>
|
|
<h2>Related Skills</h2>
|
|
<p><strong>Next Steps</strong>:</p>
|
|
<ul><li><code>http-caching.md</code> → Browser and HTTP cache layer</li><li><code>cdn-edge-caching.md</code> → CDN and edge caching</li><li><code>redis-caching-patterns.md</code> → Distributed caching with Redis</li><li><code>cache-invalidation-strategies.md</code> → Invalidation patterns</li><li><code>cache-performance-monitoring.md</code> → Metrics and monitoring</li></ul>
|
|
<p><strong>Foundations</strong>:</p>
|
|
<ul><li><code>database-connection-pooling.md</code> → Database optimization</li><li><code>redis-data-structures.md</code> → Redis basics</li></ul>
|
|
<hr>
|
|
<h2>Summary</h2>
|
|
<p>Caching fundamentals provide the foundation for performance optimization:</p>
|
|
<ul><li><strong>Cache Types</strong>: Local (fastest), distributed (shared), edge (global)</li><li><strong>Patterns</strong>: Cache-Aside, Write-Through, Write-Behind, Read-Through</li><li><strong>Eviction</strong>: LRU (recency), LFU (frequency), TTL (time-based)</li><li><strong>Key Design</strong>: Hierarchical, composite, hashed, versioned</li></ul>
|
|
<p><strong>Key takeaways</strong>:</p>
|
|
<ol><li>Choose caching pattern based on consistency and performance needs</li><li>Set appropriate TTL values to balance freshness and hit rate</li><li>Design cache keys for organization and invalidation</li><li>Cache expensive, frequently accessed, slow-changing data</li><li>Avoid caching errors, personalized data, or one-time requests</li></ol>
|
|
<p><strong>Next</strong>: Move to <code>http-caching.md</code> for browser and HTTP caching.</p>
|
|
<footer>
|
|
<div class="fmap">
|
|
<div><b>Build</b>
|
|
<a href="https://hanzo.app">hanzo.app</a> app builder
|
|
<a href="https://hanzo.codes">hanzo.codes</a> code with an agent
|
|
<a href="https://hanzoskills.com">hanzoskills.com</a> what an agent reads</div>
|
|
<div><b>Run</b>
|
|
<a href="https://console.hanzo.ai">console.hanzo.ai</a> cloud console
|
|
<a href="https://api.hanzo.ai">api.hanzo.ai</a> the API</div>
|
|
<div><b>Account</b>
|
|
<a href="https://hanzo.id">hanzo.id</a> sign in
|
|
<a href="https://pay.hanzo.ai">pay.hanzo.ai</a> top up
|
|
<a href="https://billing.hanzo.ai">billing.hanzo.ai</a> invoices and usage</div>
|
|
<div><b>Company</b>
|
|
<a href="https://hanzo.ai">hanzo.ai</a> canonical
|
|
<a href="https://hanzo.ventures">hanzo.ventures</a> ventures</div>
|
|
</div>
|
|
<div class="fend"><a href="/caching-caching-fundamentals.md">This page as plain text</a> · <a href="/llms.txt">Index for agents</a></div>
|
|
</footer>
|
|
</div></body></html> |