mirror of
https://github.com/hanzoai/skills.git
synced 2026-11-19 00:47:36 +00:00
Every page now loads /webmcp.js, scripts/webmcp.ts with its types stripped (npm run webmcp). Where the browser has document.modelContext it registers two read-only tools: search_skills over the published skills index, and get_skill, which returns a skill's markdown checked against the index digest. The markdown alternate link moves from build.py to site.py, so CI holds every page to it. The API catalog gains the site itself (llms.txt and the skills index) and hanzo.ai/agents; the live discovery check now fetches every catalog link. tests/site.mjs drives the tools through a recording modelContext, and checks a browser without one loads clean.
661 lines
28 KiB
HTML
661 lines
28 KiB
HTML
<!doctype html>
|
|
<html lang="en"><head>
|
|
<meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
|
|
<title>testing-test-coverage-strategy</title>
|
|
<meta name="color-scheme" content="dark">
|
|
<style>
|
|
/* Committed to one visual world: a dark instrument panel. No light variant —
|
|
the dot field and the hairline rules only read on black, and a naive
|
|
inversion would wash both out. Every colour is painted explicitly so the page
|
|
never borrows a host background. */
|
|
@font-face{font-family:'Zen';src:url('/fonts/Zen-Variable.woff2') format('woff2');font-weight:100 900;font-style:normal;font-display:swap}
|
|
@font-face{font-family:'Zen';src:url('/fonts/Zen-Italic-Variable.woff2') format('woff2');font-weight:100 900;font-style:italic;font-display:swap}
|
|
@font-face{font-family:'Zen Mono';src:url('/fonts/ZenMono-Variable.woff2') format('woff2');font-weight:100 900;font-style:normal;font-display:swap}
|
|
@font-face{font-family:'Zen Mono';src:url('/fonts/ZenMono-Italic-Variable.woff2') format('woff2');font-weight:100 900;font-style:italic;font-display:swap}
|
|
@font-face{font-family:'Zen Pixel Square';src:url('/fonts/ZenPixel-Square.woff2') format('woff2');font-style:normal;font-display:swap}
|
|
|
|
:root{
|
|
--ink:#f4f4f2; --soft:#9c9c98; --faint:#6a6a66; --ground:#000; --panel:#0a0a0a;
|
|
--line:#232323; --line-hi:#3a3a3a; --accent:#f4f4f2;
|
|
--font-zen-sans:'Zen',-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif;
|
|
--font-zen-mono:'Zen Mono',ui-monospace,"SF Mono",SFMono-Regular,Menlo,Consolas,monospace;
|
|
--font-zen-pixel-square:'Zen Pixel Square',monospace;
|
|
--mono:var(--font-zen-mono);
|
|
--sans:var(--font-zen-sans);
|
|
/* Controls are pills and surfaces are soft, as on hanzo.ai. */
|
|
--r:999px; --r-soft:14px;
|
|
}
|
|
*{box-sizing:border-box}
|
|
html{-webkit-text-size-adjust:100%}
|
|
body{margin:0;background:var(--ground);color:var(--ink);font-family:var(--sans);
|
|
font-size:16px;line-height:1.65;-webkit-font-smoothing:antialiased}
|
|
.wrap{max-width:60rem;margin:0 auto;padding:0 1.75rem}
|
|
|
|
/* The dot field. A halftone made of one repeating radial-gradient, faded out
|
|
along the diagonal by a mask so it reads as a wave rather than wallpaper. */
|
|
/* No overflow clip here: the dot field is inset to the box already, and a clip
|
|
would cut off the copy-prompt menu. */
|
|
.hero{position:relative;border-bottom:1px solid var(--line)}
|
|
.hero::before{content:"";position:absolute;inset:0;
|
|
background-image:radial-gradient(var(--line-hi) 1px, transparent 1px);
|
|
background-size:7px 7px;
|
|
-webkit-mask-image:linear-gradient(115deg,#000 0%,rgba(0,0,0,.55) 38%,transparent 72%);
|
|
mask-image:linear-gradient(115deg,#000 0%,rgba(0,0,0,.55) 38%,transparent 72%);
|
|
pointer-events:none}
|
|
.hero .wrap{position:relative;padding-top:5.5rem;padding-bottom:4rem}
|
|
|
|
nav{border-bottom:1px solid var(--line)}
|
|
nav .in{max-width:60rem;margin:0 auto;padding:1.1rem 1.75rem;display:flex;
|
|
justify-content:space-between;align-items:center;gap:1rem}
|
|
.brand{font-family:var(--mono);font-size:.85rem;letter-spacing:.14em;
|
|
text-transform:uppercase;color:var(--ink);text-decoration:none;
|
|
display:inline-flex;align-items:center;gap:.6rem}
|
|
.brand i{font-style:normal;color:var(--faint)}
|
|
.mark{width:1.15em;height:1.15em;display:block;flex:0 0 auto}
|
|
nav a.up{font-family:var(--mono);font-size:.72rem;letter-spacing:.14em;
|
|
text-transform:uppercase;color:var(--soft);text-decoration:none}
|
|
nav a.up:hover{color:var(--ink)}
|
|
|
|
h1{font-family:var(--mono);font-size:clamp(2rem,5.4vw,3.4rem);line-height:1.12;
|
|
letter-spacing:-.02em;margin:0 0 1.1rem;font-weight:500;text-wrap:balance}
|
|
h1 em{font-style:normal;color:var(--faint)}
|
|
.lede{color:var(--soft);font-size:1.05rem;max-width:52ch;margin:0 0 2.25rem}
|
|
|
|
.btns{display:flex;flex-wrap:wrap;gap:.7rem;margin:0 0 .5rem;align-items:center}
|
|
.btn{display:inline-flex;align-items:center;gap:.5rem;font-family:var(--mono);
|
|
font-size:.72rem;letter-spacing:.14em;text-transform:uppercase;
|
|
padding:.8rem 1.4rem;border:1px solid var(--line-hi);border-radius:var(--r);
|
|
color:var(--ink);text-decoration:none;background:transparent;cursor:pointer;
|
|
line-height:1.2;font-weight:400}
|
|
.btn:hover{background:#141414;text-decoration:none;border-color:var(--soft)}
|
|
.btn.solid{background:var(--ink);color:#000;border-color:var(--ink)}
|
|
.btn.solid:hover{background:#d8d8d4}
|
|
.btn .chev{width:.7em;height:.7em;flex:0 0 auto;transition:transform .15s ease}
|
|
.btn[aria-expanded="true"] .chev{transform:rotate(180deg)}
|
|
|
|
/* Copy-prompt selector. Each row copies the prompt for its own target. */
|
|
.copy{position:relative;display:inline-flex}
|
|
.menu{position:absolute;top:calc(100% + .5rem);left:0;z-index:20;min-width:21rem;
|
|
background:var(--panel);border:1px solid var(--line-hi);border-radius:var(--r-soft);
|
|
padding:.4rem;display:flex;flex-direction:column;
|
|
box-shadow:0 12px 32px rgba(0,0,0,.75)}
|
|
.menu[hidden]{display:none}
|
|
.mi{display:flex;align-items:center;gap:.7rem;width:100%;text-align:left;
|
|
padding:.62rem .8rem;border:0;border-radius:calc(var(--r-soft) - .35rem);
|
|
background:transparent;color:var(--ink);font-family:var(--mono);font-size:.76rem;
|
|
letter-spacing:.04em;text-decoration:none;cursor:pointer;line-height:1.3;
|
|
white-space:nowrap}
|
|
.mi:hover,.mi:focus-visible{background:#181818;outline:none}
|
|
.mi .mark{width:1.05rem;height:1.05rem;color:var(--soft)}
|
|
.mi:hover .mark{color:var(--ink)}
|
|
.mi u{margin-left:auto;color:var(--faint);text-decoration:none;font-size:.68rem;
|
|
letter-spacing:.1em;text-transform:uppercase}
|
|
|
|
/* Works-with bar: one scrolling row, name and mark per agent. */
|
|
.strip{border-bottom:1px solid var(--line);background:var(--panel)}
|
|
.strip .in{max-width:60rem;margin:0 auto;padding:.9rem 1.75rem;display:flex;
|
|
align-items:center;gap:1.25rem;font-family:var(--mono);font-size:.72rem;
|
|
letter-spacing:.1em;text-transform:uppercase;color:var(--faint)}
|
|
.strip b{color:var(--soft);font-weight:400;flex:0 0 auto;
|
|
border-right:1px solid var(--line);padding-right:1.25rem;align-self:stretch;
|
|
display:flex;align-items:center}
|
|
.bar{display:flex;align-items:center;gap:.5rem;flex:1 1 auto;min-width:0;
|
|
overflow-x:auto;scrollbar-width:none;scroll-snap-type:x proximity;
|
|
-webkit-overflow-scrolling:touch;
|
|
-webkit-mask-image:linear-gradient(90deg,#000 calc(100% - 3rem),transparent);
|
|
mask-image:linear-gradient(90deg,#000 calc(100% - 3rem),transparent)}
|
|
.bar::-webkit-scrollbar{display:none}
|
|
.agent{display:inline-flex;align-items:center;gap:.45rem;flex:0 0 auto;
|
|
padding:.4rem .85rem;border:1px solid var(--line);border-radius:var(--r);
|
|
color:var(--soft);white-space:nowrap;scroll-snap-align:start;
|
|
text-decoration:none;background:var(--ground)}
|
|
.agent:hover{color:var(--ink);border-color:var(--line-hi);text-decoration:none}
|
|
.agent .mark{width:.95rem;height:.95rem}
|
|
|
|
h2{font-family:var(--mono);font-size:1.1rem;letter-spacing:-.01em;margin:2.75rem 0 .75rem;font-weight:500}
|
|
h3{font-size:1rem;margin:2rem 0 .5rem;font-weight:600}
|
|
h4,h5,h6{font-size:.94rem;margin:1.5rem 0 .5rem;font-weight:600}
|
|
p{margin:0 0 1rem;max-width:70ch}
|
|
ul,ol{margin:0 0 1rem;padding-left:1.35rem;max-width:70ch}
|
|
li{margin:.22rem 0}
|
|
a{color:var(--ink);text-decoration:underline;text-underline-offset:3px;
|
|
text-decoration-color:var(--line-hi)}
|
|
a:hover{text-decoration-color:var(--ink)}
|
|
a:focus-visible{outline:1px solid var(--ink);outline-offset:3px}
|
|
code{font-family:var(--mono);font-size:.85em;background:var(--panel);
|
|
border:1px solid var(--line);border-radius:5px;padding:.1em .36em}
|
|
pre{background:var(--panel);border:1px solid var(--line);padding:1rem 1.1rem;
|
|
border-radius:var(--r-soft);overflow-x:auto;margin:0 0 1.25rem}
|
|
pre code{background:none;border:0;padding:0;font-size:.82rem;line-height:1.7}
|
|
blockquote{margin:0 0 1rem;padding-left:1rem;border-left:1px solid var(--line-hi);color:var(--soft)}
|
|
hr{border:0;border-top:1px solid var(--line);margin:2.5rem 0}
|
|
table{border-collapse:collapse;width:100%;margin:0 0 1.25rem;font-size:.92rem;
|
|
display:block;overflow-x:auto}
|
|
th,td{border:1px solid var(--line);padding:.5rem .7rem;text-align:left}
|
|
|
|
.curl{display:block;background:var(--panel);border:1px solid var(--line);
|
|
border-radius:var(--r-soft);padding:.95rem 1.1rem;font-family:var(--mono);
|
|
font-size:.8rem;color:var(--soft);overflow-x:auto;margin:0 0 2.5rem;
|
|
white-space:pre;line-height:1.9}
|
|
.curl b{color:var(--ink);font-weight:400}
|
|
.curl i{font-style:normal;color:var(--faint)}
|
|
|
|
/* index */
|
|
#q{width:100%;padding:.85rem 1.2rem;font-family:var(--mono);font-size:.85rem;color:var(--ink);
|
|
background:var(--panel);border:1px solid var(--line);border-radius:var(--r);margin:0 0 .6rem}
|
|
#q::placeholder{color:var(--faint);text-transform:uppercase;letter-spacing:.1em;font-size:.72rem}
|
|
#q:focus{outline:none;border-color:var(--soft)}
|
|
#count{color:var(--faint);font-family:var(--mono);font-size:.72rem;letter-spacing:.1em;
|
|
text-transform:uppercase;margin:0 0 2rem}
|
|
.cat{margin:2.5rem 0 .5rem;font-family:var(--mono);font-size:.7rem;letter-spacing:.16em;
|
|
text-transform:uppercase;color:var(--faint);border-bottom:1px solid var(--line);
|
|
padding-bottom:.45rem}
|
|
.row{display:block;padding:.6rem .9rem;margin:0 -.9rem;border-bottom:1px solid var(--line);
|
|
border-radius:var(--r-soft);text-decoration:none}
|
|
.row:hover{background:#0a0a0a}
|
|
.nm{font-family:var(--mono);font-size:.84rem;color:var(--ink);display:block}
|
|
.ds{color:var(--soft);font-size:.9rem;display:block;margin-top:.12rem}
|
|
footer{margin-top:4rem;padding:2rem 0 4rem;border-top:1px solid var(--line);
|
|
color:var(--faint);font-size:.85rem}
|
|
footer a{color:var(--soft)}
|
|
.fmap{display:grid;gap:1.75rem;grid-template-columns:repeat(auto-fit,minmax(11rem,1fr));
|
|
margin:0 0 2rem;font-size:.85rem}
|
|
.fmap b{display:block;font-family:var(--mono);font-size:.68rem;letter-spacing:.16em;
|
|
text-transform:uppercase;color:var(--faint);font-weight:400;margin:0 0 .6rem;
|
|
border-bottom:1px solid var(--line);padding-bottom:.4rem}
|
|
.fmap a{display:inline-block;color:var(--ink);text-decoration:none;margin:.3rem .35rem 0 0}
|
|
.fmap a:hover{text-decoration:underline}
|
|
.fmap div{color:var(--faint);line-height:1.5}
|
|
.fend{border-top:1px solid var(--line);padding-top:1.25rem;font-size:.82rem}
|
|
|
|
@media (max-width:640px){
|
|
.hero .wrap{padding-top:3.25rem;padding-bottom:2.75rem}
|
|
.btns{gap:.5rem}
|
|
.btn{padding:.75rem 1.15rem;font-size:.68rem}
|
|
.copy,.copy .btn{width:100%;justify-content:center}
|
|
.menu{left:0;right:0;min-width:0}
|
|
.strip .in{flex-direction:column;align-items:flex-start;gap:.6rem;padding:.9rem 1.75rem}
|
|
.strip b{border-right:0;padding-right:0}
|
|
.bar{width:100%}
|
|
}
|
|
</style>
|
|
<link rel="icon" href="/favicon.svg">
|
|
<link rel="canonical" href="https://hanzoskills.com/testing-test-coverage-strategy">
|
|
<link rel="alternate" type="text/markdown" href="/testing-test-coverage-strategy.md">
|
|
<script type="module" src="/webmcp.js"></script>
|
|
</head>
|
|
<!-- Reading this with a machine? Every page here is also plain markdown.
|
|
This one: https://hanzoskills.com/testing-test-coverage-strategy.md
|
|
Index: https://hanzoskills.com/llms.txt
|
|
Corpus: https://hanzoskills.com/all.md
|
|
Append .md to any path for the markdown twin. -->
|
|
<body>
|
|
<nav><div class="in">
|
|
<a class="brand" href="/"><svg class="mark" viewBox="0 0 67 67" fill="currentColor" aria-hidden="true"><path d="M22.21 67V44.6369H0V67H22.21Z"/><path d="M66.7038 22.3184H22.2534L0.0878906 44.6367H44.4634L66.7038 22.3184Z"/><path d="M22.21 0H0V22.3184H22.21V0Z"/><path d="M66.7198 0H44.5098V22.3184H66.7198V0Z"/><path d="M66.7198 67V44.6369H44.5098V67H66.7198Z"/></svg> HANZO<i>SKILLS</i></a>
|
|
<a class="up" href="/">All skills</a>
|
|
</div></nav>
|
|
|
|
<div class="wrap">
|
|
<h1>testing-test-coverage-strategy</h1><p class="lede">Determine what code needs testing vs what to skip</p><h1>Test Coverage Strategy</h1>
|
|
<h2>When to Use This Skill</h2>
|
|
<p>Use this skill when you need to:</p>
|
|
<ul><li>Determine what code needs testing vs what to skip</li><li>Interpret coverage metrics (line, branch, mutation)</li><li>Set realistic coverage goals for your project</li><li>Identify untested critical paths</li><li>Implement mutation testing to verify test quality</li><li>Balance coverage goals with development speed</li><li>Audit existing test suites for gaps</li></ul>
|
|
<p><strong>ACTIVATE THIS SKILL</strong>: When planning test strategy, auditing coverage, or setting quality gates</p>
|
|
<h2>Core Concepts</h2>
|
|
<h3>Coverage Metrics</h3>
|
|
<p><strong>Line Coverage</strong>: Percentage of lines executed by tests</p>
|
|
<pre><code class="lang-python">def calculate_discount(price, discount_percent):
|
|
if discount_percent > 0: # Line executed
|
|
return price * (1 - discount_percent / 100) # Line executed
|
|
return price # Line NOT executed (100% branch coverage would require this)
|
|
|
|
# Test only with discount_percent > 0 → 67% line coverage</code></pre>
|
|
<p><strong>Branch Coverage</strong>: Percentage of decision branches taken</p>
|
|
<pre><code class="lang-python">def validate_age(age):
|
|
if age < 0: # Branch 1: True path
|
|
return False # Covered
|
|
elif age < 18: # Branch 2: True path
|
|
return False # Covered
|
|
else: # Branch 3: Else path
|
|
return True # Covered
|
|
|
|
# Need 3 tests for 100% branch coverage:
|
|
# - age < 0
|
|
# - 0 <= age < 18
|
|
# - age >= 18</code></pre>
|
|
<p><strong>Function Coverage</strong>: Percentage of functions called</p>
|
|
<pre><code class="lang-python">def function_a():
|
|
pass
|
|
|
|
def function_b():
|
|
pass
|
|
|
|
# Test calls function_a only → 50% function coverage</code></pre>
|
|
<p><strong>Path Coverage</strong>: Percentage of unique execution paths</p>
|
|
<pre><code class="lang-python">def process(x, y):
|
|
if x > 0:
|
|
if y > 0:
|
|
return "both positive"
|
|
else:
|
|
return "x positive, y not"
|
|
else:
|
|
if y > 0:
|
|
return "y positive, x not"
|
|
else:
|
|
return "both not positive"
|
|
|
|
# 4 unique paths, need 4 tests for 100% path coverage</code></pre>
|
|
<h3>Coverage Goals by Project Type</h3>
|
|
<p><strong>Libraries</strong> (Public APIs):</p>
|
|
<ul><li>Target: 90-100% coverage</li><li>Critical: All public APIs tested</li><li>Reason: Used by many consumers, high stability need</li></ul>
|
|
<p><strong>Microservices</strong> (Business Logic):</p>
|
|
<ul><li>Target: 80-90% coverage</li><li>Critical: Core business logic, error paths</li><li>Reason: Balance between confidence and speed</li></ul>
|
|
<p><strong>Web Applications</strong> (Full Stack):</p>
|
|
<ul><li>Target: 70-80% coverage</li><li>Critical: Business logic, API endpoints</li><li>Reason: UI changes frequently, focus on backend</li></ul>
|
|
<p><strong>Prototypes/Experiments</strong>:</p>
|
|
<ul><li>Target: 0-50% coverage</li><li>Critical: None</li><li>Reason: Throwaway code, optimize for learning</li></ul>
|
|
<h3>What to Test vs Skip</h3>
|
|
<p><strong>ALWAYS TEST</strong>:</p>
|
|
<ul><li>Business logic (pricing, validation, calculations)</li><li>Security-critical code (auth, permissions, encryption)</li><li>Data transformations (parsing, serialization)</li><li>Error handling (edge cases, failure modes)</li><li>Public APIs (library interfaces)</li><li>Bug fixes (regression prevention)</li></ul>
|
|
<p><strong>SKIP TESTING</strong>:</p>
|
|
<ul><li>Simple getters/setters (no logic)</li><li>Framework/library code (already tested)</li><li>Configuration files (static data)</li><li>Generated code (build artifacts)</li><li>Trivial delegation (one-line wrappers)</li><li>UI layout code (use visual testing instead)</li></ul>
|
|
<h2>Patterns</h2>
|
|
<h3>Coverage-Guided Test Writing</h3>
|
|
<p><strong>Step 1: Run coverage report</strong>:</p>
|
|
<pre><code class="lang-bash"># Python
|
|
pytest --cov=myapp --cov-report=html tests/
|
|
|
|
# JavaScript
|
|
npm test -- --coverage
|
|
|
|
# Go
|
|
go test -cover ./...
|
|
|
|
# Rust
|
|
cargo tarpaulin --out Html</code></pre>
|
|
<p><strong>Step 2: Identify untested code</strong>:</p>
|
|
<pre><code class="lang-python"># coverage.py report shows:
|
|
# myapp/payment.py 45% Lines 23-45, 67-89 not covered
|
|
|
|
# Look at payment.py:
|
|
def process_payment(amount, method):
|
|
if method == "credit_card":
|
|
return process_credit_card(amount) # Tested
|
|
elif method == "paypal":
|
|
return process_paypal(amount) # NOT TESTED (line 23-45)
|
|
elif method == "bitcoin":
|
|
return process_bitcoin(amount) # NOT TESTED (line 67-89)
|
|
else:
|
|
raise ValueError("Invalid method") # NOT TESTED</code></pre>
|
|
<p><strong>Step 3: Write tests for critical gaps</strong>:</p>
|
|
<pre><code class="lang-python">def test_process_payment_paypal():
|
|
result = process_payment(100, "paypal")
|
|
assert result.status == "success"
|
|
|
|
def test_process_payment_bitcoin():
|
|
result = process_payment(100, "bitcoin")
|
|
assert result.status == "success"
|
|
|
|
def test_process_payment_invalid_method():
|
|
with pytest.raises(ValueError):
|
|
process_payment(100, "invalid")</code></pre>
|
|
<p><strong>Step 4: Re-run coverage → Improved to 95%</strong></p>
|
|
<h3>Mutation Testing</h3>
|
|
<p><strong>Concept</strong>: Introduce bugs (mutations) to verify tests catch them</p>
|
|
<pre><code class="lang-python"># Original code
|
|
def is_adult(age):
|
|
return age >= 18
|
|
|
|
# Test
|
|
def test_is_adult():
|
|
assert is_adult(18) is True
|
|
assert is_adult(17) is False
|
|
|
|
# Mutation 1: Change >= to >
|
|
def is_adult(age):
|
|
return age > 18 # Mutant
|
|
|
|
# Run tests: PASSES (but shouldn't!)
|
|
# → Test is weak, doesn't catch boundary error
|
|
|
|
# Improved test
|
|
def test_is_adult_boundary():
|
|
assert is_adult(18) is True # Catches >= vs > mutation
|
|
assert is_adult(17) is False
|
|
assert is_adult(19) is True</code></pre>
|
|
<p><strong>Mutation Testing Tools</strong>:</p>
|
|
<pre><code class="lang-bash"># Python (mutmut)
|
|
pip install mutmut
|
|
mutmut run
|
|
mutmut results
|
|
|
|
# JavaScript (Stryker)
|
|
npm install --save-dev @stryker-mutator/core
|
|
npx stryker run
|
|
|
|
# Example output:
|
|
# Mutant 1: Changed >= to > → SURVIVED (test didn't catch it)
|
|
# Mutant 2: Changed >= to == → KILLED (test caught it)
|
|
# Mutation Score: 85% (good)</code></pre>
|
|
<h3>Critical Path Testing</h3>
|
|
<p><strong>Identify high-value code paths</strong>:</p>
|
|
<pre><code class="lang-python"># E-commerce checkout flow
|
|
def checkout(cart, user, payment_method):
|
|
# CRITICAL: Calculate total (money involved)
|
|
total = calculate_total(cart)
|
|
|
|
# CRITICAL: Apply discounts (business logic)
|
|
total = apply_discounts(total, user.discount_codes)
|
|
|
|
# CRITICAL: Process payment (money exchange)
|
|
payment = process_payment(total, payment_method)
|
|
if not payment.success:
|
|
raise PaymentFailedError()
|
|
|
|
# CRITICAL: Create order (data integrity)
|
|
order = create_order(cart, user, total, payment.id)
|
|
|
|
# LESS CRITICAL: Send email (nice-to-have)
|
|
send_order_confirmation(user.email, order)
|
|
|
|
# LESS CRITICAL: Analytics (optional)
|
|
track_purchase(user.id, total)
|
|
|
|
return order
|
|
|
|
# Test priority:
|
|
# 1. calculate_total (business logic)
|
|
# 2. apply_discounts (business logic)
|
|
# 3. process_payment (integration test)
|
|
# 4. create_order (integration test)
|
|
# 5. send_order_confirmation (optional)
|
|
# 6. track_purchase (optional)</code></pre>
|
|
<h3>Incremental Coverage Improvement</h3>
|
|
<p><strong>Strategy</strong>: Improve coverage over time, focus on new code</p>
|
|
<pre><code class="lang-python"># .coveragerc (Python)
|
|
[run]
|
|
branch = True
|
|
source = myapp
|
|
|
|
[report]
|
|
precision = 2
|
|
fail_under = 80 # CI fails if coverage drops below 80%
|
|
|
|
# Track coverage over time
|
|
# Week 1: 65%
|
|
# Week 2: 70% (added tests for user authentication)
|
|
# Week 3: 75% (added tests for payment processing)
|
|
# Week 4: 80% (reached goal)</code></pre>
|
|
<p><strong>Git hook to prevent coverage regression</strong>:</p>
|
|
<pre><code class="lang-bash">#!/bin/bash
|
|
# .git/hooks/pre-push
|
|
|
|
# Run tests with coverage
|
|
coverage run -m pytest
|
|
coverage report --fail-under=80
|
|
|
|
if [ $? -ne 0 ]; then
|
|
echo "❌ Coverage below 80%, push rejected"
|
|
exit 1
|
|
fi
|
|
|
|
echo "✅ Coverage check passed"</code></pre>
|
|
<h3>Coverage Exclusions</h3>
|
|
<p><strong>Explicitly exclude code from coverage</strong>:</p>
|
|
<pre><code class="lang-python"># Exclude debug code
|
|
if DEBUG: # pragma: no cover
|
|
print(f"Debug: {variable}")
|
|
|
|
# Exclude abstract methods
|
|
class BaseRepository:
|
|
def find_by_id(self, id: int):
|
|
raise NotImplementedError() # pragma: no cover
|
|
|
|
# Exclude platform-specific code
|
|
if sys.platform == "win32": # pragma: no cover
|
|
import msvcrt</code></pre>
|
|
<pre><code class="lang-typescript">// TypeScript/JavaScript (Istanbul)
|
|
/* istanbul ignore next */
|
|
function debugLog(message: string) {
|
|
console.log(message);
|
|
}
|
|
|
|
/* istanbul ignore else */
|
|
if (process.env.NODE_ENV === 'development') {
|
|
console.log('Dev mode');
|
|
}</code></pre>
|
|
<h2>Examples by Language</h2>
|
|
<h3>Python (coverage.py)</h3>
|
|
<pre><code class="lang-python"># Install
|
|
pip install pytest-cov
|
|
|
|
# Run with coverage
|
|
pytest --cov=myapp --cov-report=term-missing tests/
|
|
|
|
# Generate HTML report
|
|
pytest --cov=myapp --cov-report=html tests/
|
|
# Open htmlcov/index.html
|
|
|
|
# Configuration (.coveragerc)
|
|
[run]
|
|
source = myapp
|
|
omit =
|
|
*/tests/*
|
|
*/migrations/*
|
|
*/__pycache__/*
|
|
|
|
[report]
|
|
precision = 2
|
|
exclude_lines =
|
|
pragma: no cover
|
|
def __repr__
|
|
raise AssertionError
|
|
raise NotImplementedError
|
|
if __name__ == .__main__.:
|
|
|
|
# CI integration (GitHub Actions)
|
|
- name: Test with coverage
|
|
run: |
|
|
pytest --cov=myapp --cov-report=xml tests/
|
|
|
|
- name: Upload coverage to Codecov
|
|
uses: codecov/codecov-action@v3
|
|
with:
|
|
files: ./coverage.xml</code></pre>
|
|
<h3>TypeScript/JavaScript (Istanbul/c8)</h3>
|
|
<pre><code class="lang-json">// package.json
|
|
{
|
|
"scripts": {
|
|
"test": "vitest",
|
|
"test:coverage": "vitest --coverage"
|
|
},
|
|
"devDependencies": {
|
|
"@vitest/coverage-v8": "^1.0.0"
|
|
}
|
|
}
|
|
|
|
// vitest.config.ts
|
|
export default defineConfig({
|
|
test: {
|
|
coverage: {
|
|
provider: 'v8',
|
|
reporter: ['text', 'html', 'lcov'],
|
|
exclude: [
|
|
'node_modules/',
|
|
'dist/',
|
|
'**/*.test.ts',
|
|
'**/*.config.ts',
|
|
],
|
|
lines: 80,
|
|
branches: 80,
|
|
functions: 80,
|
|
statements: 80,
|
|
},
|
|
},
|
|
});</code></pre>
|
|
<pre><code class="lang-bash"># Run coverage
|
|
npm run test:coverage
|
|
|
|
# Output:
|
|
# File | % Stmts | % Branch | % Funcs | % Lines |
|
|
# -------------|---------|----------|---------|---------|
|
|
# user.ts | 85.71 | 75 | 100 | 85.71 |
|
|
# payment.ts | 66.67 | 50 | 75 | 66.67 |
|
|
# -------------|---------|----------|---------|---------|
|
|
# All files | 76.19 | 62.50 | 87.50 | 76.19 |</code></pre>
|
|
<h3>Go</h3>
|
|
<pre><code class="lang-go">// Run with coverage
|
|
go test -cover ./...
|
|
|
|
// Generate coverage profile
|
|
go test -coverprofile=coverage.out ./...
|
|
|
|
// View coverage in browser
|
|
go tool cover -html=coverage.out
|
|
|
|
// Coverage for specific package
|
|
go test -cover -coverprofile=coverage.out ./pkg/user
|
|
|
|
// Coverage threshold (CI)
|
|
#!/bin/bash
|
|
go test -coverprofile=coverage.out ./...
|
|
coverage=$(go tool cover -func=coverage.out | grep total | awk '{print $3}' | sed 's/%//')
|
|
threshold=80
|
|
|
|
if (( $(echo "$coverage < $threshold" | bc -l) )); then
|
|
echo "Coverage $coverage% is below threshold $threshold%"
|
|
exit 1
|
|
fi</code></pre>
|
|
<h3>Rust (tarpaulin)</h3>
|
|
<pre><code class="lang-bash"># Install
|
|
cargo install cargo-tarpaulin
|
|
|
|
# Run coverage
|
|
cargo tarpaulin --out Html
|
|
|
|
# Output to terminal
|
|
cargo tarpaulin --out Stdout
|
|
|
|
# Configuration (Cargo.toml or tarpaulin.toml)
|
|
[tarpaulin]
|
|
ignore-tests = true
|
|
exclude-files = [
|
|
"src/generated/*",
|
|
"tests/*"
|
|
]</code></pre>
|
|
<h2>Mutation Testing Examples</h2>
|
|
<h3>Python (mutmut)</h3>
|
|
<pre><code class="lang-bash"># Install
|
|
pip install mutmut
|
|
|
|
# Run mutation testing
|
|
mutmut run
|
|
|
|
# View results
|
|
mutmut results
|
|
|
|
# Show surviving mutants
|
|
mutmut show
|
|
|
|
# Example output:
|
|
# Mutant 1 (SURVIVED):
|
|
# --- src/calculator.py
|
|
# +++ mutant
|
|
# @@ -5,7 +5,7 @@
|
|
# def add(a, b):
|
|
# - return a + b
|
|
# + return a - b
|
|
|
|
# This survived because no test verifies add(2, 3) == 5
|
|
# Add test:
|
|
def test_add():
|
|
assert add(2, 3) == 5
|
|
assert add(0, 0) == 0
|
|
assert add(-1, 1) == 0</code></pre>
|
|
<h3>JavaScript (Stryker)</h3>
|
|
<pre><code class="lang-bash"># Install
|
|
npm install --save-dev @stryker-mutator/core
|
|
|
|
# Initialize
|
|
npx stryker init
|
|
|
|
# Run
|
|
npx stryker run
|
|
|
|
# Example stryker.conf.json
|
|
{
|
|
"mutate": ["src/**/*.ts"],
|
|
"testRunner": "vitest",
|
|
"coverageAnalysis": "perTest",
|
|
"thresholds": {
|
|
"high": 80,
|
|
"low": 60,
|
|
"break": 60
|
|
}
|
|
}
|
|
|
|
# Output:
|
|
# Mutant killed | 45
|
|
# Mutant survived | 5
|
|
# Mutation score | 90%</code></pre>
|
|
<h2>Checklist</h2>
|
|
<p><strong>Setting Coverage Strategy</strong>:</p>
|
|
<ul><li>[ ] Define coverage goals by project type (library: 90%+, app: 70-80%)</li><li>[ ] Identify critical paths to prioritize</li><li>[ ] Decide what to exclude (generated code, debug code)</li><li>[ ] Choose coverage metrics (line + branch minimum)</li><li>[ ] Set up coverage reporting in CI/CD</li></ul>
|
|
<p><strong>Measuring Coverage</strong>:</p>
|
|
<ul><li>[ ] Run coverage report locally</li><li>[ ] Identify untested lines/branches</li><li>[ ] Prioritize gaps by business criticality</li><li>[ ] Write tests for critical gaps</li><li>[ ] Verify coverage improvement</li></ul>
|
|
<p><strong>Coverage Quality</strong>:</p>
|
|
<ul><li>[ ] Use mutation testing to verify test effectiveness</li><li>[ ] Check branch coverage (not just line coverage)</li><li>[ ] Review surviving mutants (weak tests)</li><li>[ ] Test error paths and edge cases</li><li>[ ] Avoid testing for coverage sake (test behavior)</li></ul>
|
|
<p><strong>CI/CD Integration</strong>:</p>
|
|
<ul><li>[ ] Fail builds if coverage drops below threshold</li><li>[ ] Upload coverage reports (Codecov, Coveralls)</li><li>[ ] Track coverage trends over time</li><li>[ ] Require coverage for new code (diff coverage)</li><li>[ ] Set realistic thresholds (don't aim for 100%)</li></ul>
|
|
<h2>Anti-Patterns</h2>
|
|
<pre><code>❌ NEVER: Aim for 100% coverage on everything
|
|
→ Waste time testing trivial code
|
|
|
|
❌ NEVER: Write tests just to increase coverage
|
|
→ Tests without assertions, useless tests
|
|
|
|
❌ NEVER: Only measure line coverage
|
|
→ Miss untested branches/paths
|
|
|
|
❌ NEVER: Ignore coverage in code review
|
|
→ Coverage regression over time
|
|
|
|
❌ NEVER: Test private implementation details
|
|
→ Brittle tests, false coverage confidence
|
|
|
|
❌ NEVER: Exclude all error handling from coverage
|
|
→ Untested failure modes
|
|
|
|
❌ NEVER: Use coverage as only quality metric
|
|
→ High coverage ≠ good tests</code></pre>
|
|
<h2>Related Skills</h2>
|
|
<p><strong>Foundation</strong>:</p>
|
|
<ul><li><code>unit-testing-patterns.md</code> - How to write testable code</li><li><code>test-driven-development.md</code> - TDD workflow for coverage</li></ul>
|
|
<p><strong>Advanced</strong>:</p>
|
|
<ul><li><code>mutation-testing.md</code> - Advanced mutation testing patterns</li><li><code>property-based-testing.md</code> - Generate test cases automatically</li></ul>
|
|
<p><strong>Integration</strong>:</p>
|
|
<ul><li><code>integration-testing.md</code> - Coverage for integration tests</li><li><code>e2e-testing.md</code> - Coverage for E2E tests</li></ul>
|
|
<p><strong>Tools</strong>:</p>
|
|
<ul><li>Python: coverage.py, mutmut</li><li>TypeScript: c8, Stryker</li><li>Go: Built-in coverage tool</li><li>Rust: tarpaulin, cargo-mutants</li></ul>
|
|
<p><strong>Coverage Services</strong>:</p>
|
|
<ul><li>Codecov: Cross-language coverage tracking</li><li>Coveralls: GitHub integration</li><li>SonarQube: Comprehensive code quality</li></ul>
|
|
<footer>
|
|
<div class="fmap">
|
|
<div><b>Build</b>
|
|
<a href="https://hanzo.app">hanzo.app</a> app builder
|
|
<a href="https://hanzo.codes">hanzo.codes</a> code with an agent
|
|
<a href="https://hanzoskills.com">hanzoskills.com</a> what an agent reads</div>
|
|
<div><b>Run</b>
|
|
<a href="https://console.hanzo.ai">console.hanzo.ai</a> cloud console
|
|
<a href="https://api.hanzo.ai">api.hanzo.ai</a> the API</div>
|
|
<div><b>Account</b>
|
|
<a href="https://hanzo.id">hanzo.id</a> sign in
|
|
<a href="https://pay.hanzo.ai">pay.hanzo.ai</a> top up
|
|
<a href="https://billing.hanzo.ai">billing.hanzo.ai</a> invoices and usage</div>
|
|
<div><b>Company</b>
|
|
<a href="https://hanzo.ai">hanzo.ai</a> canonical
|
|
<a href="https://hanzo.ventures">hanzo.ventures</a> ventures</div>
|
|
</div>
|
|
<div class="fend"><a href="/testing-test-coverage-strategy.md">This page as plain text</a> · <a href="/llms.txt">Index for agents</a></div>
|
|
</footer>
|
|
</div></body></html> |