Files
skills/public/math-linear-algebra-computation.html
zeekay 747dafba10 Register WebMCP tools on every page, and name each page's twin from site.py
Every page now loads /webmcp.js, scripts/webmcp.ts with its types
stripped (npm run webmcp). Where the browser has document.modelContext
it registers two read-only tools: search_skills over the published
skills index, and get_skill, which returns a skill's markdown checked
against the index digest. The markdown alternate link moves from
build.py to site.py, so CI holds every page to it.

The API catalog gains the site itself (llms.txt and the skills index)
and hanzo.ai/agents; the live discovery check now fetches every
catalog link. tests/site.mjs drives the tools through a recording
modelContext, and checks a browser without one loads clean.
2026-10-26 20:57:05 -07:00

719 lines
30 KiB
HTML

<!doctype html>
<html lang="en"><head>
<meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
<title>math-linear-algebra-computation</title>
<meta name="color-scheme" content="dark">
<style>
/* Committed to one visual world: a dark instrument panel. No light variant —
the dot field and the hairline rules only read on black, and a naive
inversion would wash both out. Every colour is painted explicitly so the page
never borrows a host background. */
@font-face{font-family:'Zen';src:url('/fonts/Zen-Variable.woff2') format('woff2');font-weight:100 900;font-style:normal;font-display:swap}
@font-face{font-family:'Zen';src:url('/fonts/Zen-Italic-Variable.woff2') format('woff2');font-weight:100 900;font-style:italic;font-display:swap}
@font-face{font-family:'Zen Mono';src:url('/fonts/ZenMono-Variable.woff2') format('woff2');font-weight:100 900;font-style:normal;font-display:swap}
@font-face{font-family:'Zen Mono';src:url('/fonts/ZenMono-Italic-Variable.woff2') format('woff2');font-weight:100 900;font-style:italic;font-display:swap}
@font-face{font-family:'Zen Pixel Square';src:url('/fonts/ZenPixel-Square.woff2') format('woff2');font-style:normal;font-display:swap}
:root{
--ink:#f4f4f2; --soft:#9c9c98; --faint:#6a6a66; --ground:#000; --panel:#0a0a0a;
--line:#232323; --line-hi:#3a3a3a; --accent:#f4f4f2;
--font-zen-sans:'Zen',-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif;
--font-zen-mono:'Zen Mono',ui-monospace,"SF Mono",SFMono-Regular,Menlo,Consolas,monospace;
--font-zen-pixel-square:'Zen Pixel Square',monospace;
--mono:var(--font-zen-mono);
--sans:var(--font-zen-sans);
/* Controls are pills and surfaces are soft, as on hanzo.ai. */
--r:999px; --r-soft:14px;
}
*{box-sizing:border-box}
html{-webkit-text-size-adjust:100%}
body{margin:0;background:var(--ground);color:var(--ink);font-family:var(--sans);
font-size:16px;line-height:1.65;-webkit-font-smoothing:antialiased}
.wrap{max-width:60rem;margin:0 auto;padding:0 1.75rem}
/* The dot field. A halftone made of one repeating radial-gradient, faded out
along the diagonal by a mask so it reads as a wave rather than wallpaper. */
/* No overflow clip here: the dot field is inset to the box already, and a clip
would cut off the copy-prompt menu. */
.hero{position:relative;border-bottom:1px solid var(--line)}
.hero::before{content:"";position:absolute;inset:0;
background-image:radial-gradient(var(--line-hi) 1px, transparent 1px);
background-size:7px 7px;
-webkit-mask-image:linear-gradient(115deg,#000 0%,rgba(0,0,0,.55) 38%,transparent 72%);
mask-image:linear-gradient(115deg,#000 0%,rgba(0,0,0,.55) 38%,transparent 72%);
pointer-events:none}
.hero .wrap{position:relative;padding-top:5.5rem;padding-bottom:4rem}
nav{border-bottom:1px solid var(--line)}
nav .in{max-width:60rem;margin:0 auto;padding:1.1rem 1.75rem;display:flex;
justify-content:space-between;align-items:center;gap:1rem}
.brand{font-family:var(--mono);font-size:.85rem;letter-spacing:.14em;
text-transform:uppercase;color:var(--ink);text-decoration:none;
display:inline-flex;align-items:center;gap:.6rem}
.brand i{font-style:normal;color:var(--faint)}
.mark{width:1.15em;height:1.15em;display:block;flex:0 0 auto}
nav a.up{font-family:var(--mono);font-size:.72rem;letter-spacing:.14em;
text-transform:uppercase;color:var(--soft);text-decoration:none}
nav a.up:hover{color:var(--ink)}
h1{font-family:var(--mono);font-size:clamp(2rem,5.4vw,3.4rem);line-height:1.12;
letter-spacing:-.02em;margin:0 0 1.1rem;font-weight:500;text-wrap:balance}
h1 em{font-style:normal;color:var(--faint)}
.lede{color:var(--soft);font-size:1.05rem;max-width:52ch;margin:0 0 2.25rem}
.btns{display:flex;flex-wrap:wrap;gap:.7rem;margin:0 0 .5rem;align-items:center}
.btn{display:inline-flex;align-items:center;gap:.5rem;font-family:var(--mono);
font-size:.72rem;letter-spacing:.14em;text-transform:uppercase;
padding:.8rem 1.4rem;border:1px solid var(--line-hi);border-radius:var(--r);
color:var(--ink);text-decoration:none;background:transparent;cursor:pointer;
line-height:1.2;font-weight:400}
.btn:hover{background:#141414;text-decoration:none;border-color:var(--soft)}
.btn.solid{background:var(--ink);color:#000;border-color:var(--ink)}
.btn.solid:hover{background:#d8d8d4}
.btn .chev{width:.7em;height:.7em;flex:0 0 auto;transition:transform .15s ease}
.btn[aria-expanded="true"] .chev{transform:rotate(180deg)}
/* Copy-prompt selector. Each row copies the prompt for its own target. */
.copy{position:relative;display:inline-flex}
.menu{position:absolute;top:calc(100% + .5rem);left:0;z-index:20;min-width:21rem;
background:var(--panel);border:1px solid var(--line-hi);border-radius:var(--r-soft);
padding:.4rem;display:flex;flex-direction:column;
box-shadow:0 12px 32px rgba(0,0,0,.75)}
.menu[hidden]{display:none}
.mi{display:flex;align-items:center;gap:.7rem;width:100%;text-align:left;
padding:.62rem .8rem;border:0;border-radius:calc(var(--r-soft) - .35rem);
background:transparent;color:var(--ink);font-family:var(--mono);font-size:.76rem;
letter-spacing:.04em;text-decoration:none;cursor:pointer;line-height:1.3;
white-space:nowrap}
.mi:hover,.mi:focus-visible{background:#181818;outline:none}
.mi .mark{width:1.05rem;height:1.05rem;color:var(--soft)}
.mi:hover .mark{color:var(--ink)}
.mi u{margin-left:auto;color:var(--faint);text-decoration:none;font-size:.68rem;
letter-spacing:.1em;text-transform:uppercase}
/* Works-with bar: one scrolling row, name and mark per agent. */
.strip{border-bottom:1px solid var(--line);background:var(--panel)}
.strip .in{max-width:60rem;margin:0 auto;padding:.9rem 1.75rem;display:flex;
align-items:center;gap:1.25rem;font-family:var(--mono);font-size:.72rem;
letter-spacing:.1em;text-transform:uppercase;color:var(--faint)}
.strip b{color:var(--soft);font-weight:400;flex:0 0 auto;
border-right:1px solid var(--line);padding-right:1.25rem;align-self:stretch;
display:flex;align-items:center}
.bar{display:flex;align-items:center;gap:.5rem;flex:1 1 auto;min-width:0;
overflow-x:auto;scrollbar-width:none;scroll-snap-type:x proximity;
-webkit-overflow-scrolling:touch;
-webkit-mask-image:linear-gradient(90deg,#000 calc(100% - 3rem),transparent);
mask-image:linear-gradient(90deg,#000 calc(100% - 3rem),transparent)}
.bar::-webkit-scrollbar{display:none}
.agent{display:inline-flex;align-items:center;gap:.45rem;flex:0 0 auto;
padding:.4rem .85rem;border:1px solid var(--line);border-radius:var(--r);
color:var(--soft);white-space:nowrap;scroll-snap-align:start;
text-decoration:none;background:var(--ground)}
.agent:hover{color:var(--ink);border-color:var(--line-hi);text-decoration:none}
.agent .mark{width:.95rem;height:.95rem}
h2{font-family:var(--mono);font-size:1.1rem;letter-spacing:-.01em;margin:2.75rem 0 .75rem;font-weight:500}
h3{font-size:1rem;margin:2rem 0 .5rem;font-weight:600}
h4,h5,h6{font-size:.94rem;margin:1.5rem 0 .5rem;font-weight:600}
p{margin:0 0 1rem;max-width:70ch}
ul,ol{margin:0 0 1rem;padding-left:1.35rem;max-width:70ch}
li{margin:.22rem 0}
a{color:var(--ink);text-decoration:underline;text-underline-offset:3px;
text-decoration-color:var(--line-hi)}
a:hover{text-decoration-color:var(--ink)}
a:focus-visible{outline:1px solid var(--ink);outline-offset:3px}
code{font-family:var(--mono);font-size:.85em;background:var(--panel);
border:1px solid var(--line);border-radius:5px;padding:.1em .36em}
pre{background:var(--panel);border:1px solid var(--line);padding:1rem 1.1rem;
border-radius:var(--r-soft);overflow-x:auto;margin:0 0 1.25rem}
pre code{background:none;border:0;padding:0;font-size:.82rem;line-height:1.7}
blockquote{margin:0 0 1rem;padding-left:1rem;border-left:1px solid var(--line-hi);color:var(--soft)}
hr{border:0;border-top:1px solid var(--line);margin:2.5rem 0}
table{border-collapse:collapse;width:100%;margin:0 0 1.25rem;font-size:.92rem;
display:block;overflow-x:auto}
th,td{border:1px solid var(--line);padding:.5rem .7rem;text-align:left}
.curl{display:block;background:var(--panel);border:1px solid var(--line);
border-radius:var(--r-soft);padding:.95rem 1.1rem;font-family:var(--mono);
font-size:.8rem;color:var(--soft);overflow-x:auto;margin:0 0 2.5rem;
white-space:pre;line-height:1.9}
.curl b{color:var(--ink);font-weight:400}
.curl i{font-style:normal;color:var(--faint)}
/* index */
#q{width:100%;padding:.85rem 1.2rem;font-family:var(--mono);font-size:.85rem;color:var(--ink);
background:var(--panel);border:1px solid var(--line);border-radius:var(--r);margin:0 0 .6rem}
#q::placeholder{color:var(--faint);text-transform:uppercase;letter-spacing:.1em;font-size:.72rem}
#q:focus{outline:none;border-color:var(--soft)}
#count{color:var(--faint);font-family:var(--mono);font-size:.72rem;letter-spacing:.1em;
text-transform:uppercase;margin:0 0 2rem}
.cat{margin:2.5rem 0 .5rem;font-family:var(--mono);font-size:.7rem;letter-spacing:.16em;
text-transform:uppercase;color:var(--faint);border-bottom:1px solid var(--line);
padding-bottom:.45rem}
.row{display:block;padding:.6rem .9rem;margin:0 -.9rem;border-bottom:1px solid var(--line);
border-radius:var(--r-soft);text-decoration:none}
.row:hover{background:#0a0a0a}
.nm{font-family:var(--mono);font-size:.84rem;color:var(--ink);display:block}
.ds{color:var(--soft);font-size:.9rem;display:block;margin-top:.12rem}
footer{margin-top:4rem;padding:2rem 0 4rem;border-top:1px solid var(--line);
color:var(--faint);font-size:.85rem}
footer a{color:var(--soft)}
.fmap{display:grid;gap:1.75rem;grid-template-columns:repeat(auto-fit,minmax(11rem,1fr));
margin:0 0 2rem;font-size:.85rem}
.fmap b{display:block;font-family:var(--mono);font-size:.68rem;letter-spacing:.16em;
text-transform:uppercase;color:var(--faint);font-weight:400;margin:0 0 .6rem;
border-bottom:1px solid var(--line);padding-bottom:.4rem}
.fmap a{display:inline-block;color:var(--ink);text-decoration:none;margin:.3rem .35rem 0 0}
.fmap a:hover{text-decoration:underline}
.fmap div{color:var(--faint);line-height:1.5}
.fend{border-top:1px solid var(--line);padding-top:1.25rem;font-size:.82rem}
@media (max-width:640px){
.hero .wrap{padding-top:3.25rem;padding-bottom:2.75rem}
.btns{gap:.5rem}
.btn{padding:.75rem 1.15rem;font-size:.68rem}
.copy,.copy .btn{width:100%;justify-content:center}
.menu{left:0;right:0;min-width:0}
.strip .in{flex-direction:column;align-items:flex-start;gap:.6rem;padding:.9rem 1.75rem}
.strip b{border-right:0;padding-right:0}
.bar{width:100%}
}
</style>
<link rel="icon" href="/favicon.svg">
<link rel="canonical" href="https://hanzoskills.com/math-linear-algebra-computation">
<link rel="alternate" type="text/markdown" href="/math-linear-algebra-computation.md">
<script type="module" src="/webmcp.js"></script>
</head>
<!-- Reading this with a machine? Every page here is also plain markdown.
This one: https://hanzoskills.com/math-linear-algebra-computation.md
Index: https://hanzoskills.com/llms.txt
Corpus: https://hanzoskills.com/all.md
Append .md to any path for the markdown twin. -->
<body>
<nav><div class="in">
<a class="brand" href="/"><svg class="mark" viewBox="0 0 67 67" fill="currentColor" aria-hidden="true"><path d="M22.21 67V44.6369H0V67H22.21Z"/><path d="M66.7038 22.3184H22.2534L0.0878906 44.6367H44.4634L66.7038 22.3184Z"/><path d="M22.21 0H0V22.3184H22.21V0Z"/><path d="M66.7198 0H44.5098V22.3184H66.7198V0Z"/><path d="M66.7198 67V44.6369H44.5098V67H66.7198Z"/></svg> HANZO<i>SKILLS</i></a>
<a class="up" href="/">All skills</a>
</div></nav>
<div class="wrap">
<h1>math-linear-algebra-computation</h1><p class="lede">Solving systems of linear equations (Ax = b)</p><h1>Linear Algebra Computation</h1>
<h2>Metadata</h2>
<ul><li><strong>Skill ID</strong>: linear-algebra-computation</li><li><strong>Category</strong>: Mathematics/Computation</li><li><strong>Complexity</strong>: Advanced</li><li><strong>Last Updated</strong>: 2025-10-18</li></ul>
<h2>Overview</h2>
<p>Computational linear algebra using NumPy and SciPy for matrix operations, decompositions, solving linear systems, and dimensionality reduction. Essential for scientific computing, machine learning, and numerical analysis.</p>
<h2>When to Use This Skill</h2>
<ul><li>Solving systems of linear equations (Ax = b)</li><li>Matrix decompositions (SVD, QR, Cholesky, eigendecomposition)</li><li>Dimensionality reduction (PCA, matrix approximations)</li><li>Least squares problems and linear regression</li><li>Computing matrix norms, condition numbers, ranks</li><li>Large-scale sparse matrix operations</li><li>Numerical linear algebra in scientific computing</li></ul>
<h2>When NOT to Use This Skill</h2>
<ul><li>Symbolic mathematics (use SymPy instead)</li><li>Small matrices where performance doesn&#x27;t matter</li><li>When automatic differentiation is needed (use JAX/PyTorch)</li><li>Graph algorithms (use NetworkX)</li><li>Pure theoretical proofs (not computational)</li></ul>
<h2>Prerequisites</h2>
<ul><li>Python basics</li><li>Understanding of linear algebra concepts</li><li>NumPy array operations</li><li>Basic numerical analysis awareness</li></ul>
<h2>Core Concepts</h2>
<h3>1. Matrix Operations Fundamentals</h3>
<p><strong>Efficient Matrix Multiplication</strong></p>
<pre><code class="lang-python">import numpy as np
# Basic matrix multiplication
A = np.random.rand(1000, 500)
B = np.random.rand(500, 300)
# Standard: A @ B or np.dot(A, B)
C = A @ B # Preferred Python 3.5+
# Matrix-vector: use @ for clarity
x = np.random.rand(500)
y = A @ x
# Batch matrix multiplication
batch_A = np.random.rand(10, 100, 50)
batch_B = np.random.rand(10, 50, 30)
batch_C = np.einsum(&#x27;ijk,ikl-&gt;ijl&#x27;, batch_A, batch_B)
# Or: batch_C = batch_A @ batch_B (broadcasting)
# Outer product
u = np.random.rand(100)
v = np.random.rand(200)
outer = np.outer(u, v) # 100x200 matrix
# Inner product (dot product)
inner = np.inner(u, u) # Scalar</code></pre>
<p><strong>Matrix Norms and Properties</strong></p>
<pre><code class="lang-python">from numpy.linalg import norm
A = np.random.rand(100, 100)
# Frobenius norm (default)
fro_norm = norm(A) # sqrt(sum of squared elements)
# 2-norm (spectral norm, largest singular value)
spectral_norm = norm(A, ord=2)
# Nuclear norm (sum of singular values)
nuclear_norm = norm(A, ord=&#x27;nuc&#x27;)
# Infinity norm (max row sum)
inf_norm = norm(A, ord=np.inf)
# 1-norm (max column sum)
one_norm = norm(A, ord=1)
# Condition number (sensitivity to perturbations)
cond = np.linalg.cond(A) # ||A|| * ||A^-1||
# High condition number = ill-conditioned matrix</code></pre>
<h3>2. Solving Linear Systems</h3>
<p><strong>Direct Methods (Ax = b)</strong></p>
<pre><code class="lang-python">from scipy.linalg import solve, lu_factor, lu_solve, cho_factor, cho_solve
# General system: prefer solve() over inv()
A = np.random.rand(100, 100)
b = np.random.rand(100)
# NEVER do this (slow and numerically unstable):
# x = np.linalg.inv(A) @ b # BAD!
# ALWAYS do this instead:
x = np.linalg.solve(A, b) # GOOD!
# Multiple right-hand sides
B = np.random.rand(100, 5)
X = np.linalg.solve(A, B)
# LU decomposition for repeated solves
lu, piv = lu_factor(A)
x1 = lu_solve((lu, piv), b)
x2 = lu_solve((lu, piv), b + 0.1) # Reuse factorization
# Symmetric positive definite: use Cholesky
A_spd = A @ A.T + np.eye(100) # Force SPD
c, low = cho_factor(A_spd)
x_cho = cho_solve((c, low), b)
# Check solution quality
residual = norm(A @ x - b) / norm(b)
print(f&quot;Relative residual: {residual:.2e}&quot;)</code></pre>
<p><strong>Least Squares Problems</strong></p>
<pre><code class="lang-python">from numpy.linalg import lstsq
from scipy.linalg import qr
# Overdetermined system (more equations than unknowns)
A = np.random.rand(200, 100) # 200 equations, 100 unknowns
b = np.random.rand(200)
# Solve min ||Ax - b||^2
x, residuals, rank, s = lstsq(A, b, rcond=None)
# Underdetermined system (infinite solutions, find minimum norm)
A_under = np.random.rand(50, 100) # 50 equations, 100 unknowns
b_under = np.random.rand(50)
x_min_norm = lstsq(A_under, b_under, rcond=None)[0]
# Weighted least squares
W = np.diag(np.random.rand(200)) # Weight matrix
A_weighted = W @ A
b_weighted = W @ b
x_weighted = lstsq(A_weighted, b_weighted, rcond=None)[0]
# QR-based least squares (more stable for ill-conditioned A)
Q, R = qr(A, mode=&#x27;economic&#x27;)
x_qr = solve(R, Q.T @ b)</code></pre>
<h3>3. Matrix Decompositions</h3>
<p><strong>Singular Value Decomposition (SVD)</strong></p>
<pre><code class="lang-python">from numpy.linalg import svd
from scipy.sparse.linalg import svds
# Full SVD: A = U @ S @ V^T
A = np.random.rand(100, 50)
U, s, Vt = svd(A, full_matrices=False)
# U: 100x50 (left singular vectors)
# s: 50 (singular values, sorted descending)
# Vt: 50x50 (right singular vectors transposed)
# Reconstruct original matrix
S = np.diag(s)
A_reconstructed = U @ S @ Vt
assert np.allclose(A, A_reconstructed)
# Low-rank approximation (truncated SVD)
k = 10 # Keep top 10 components
A_approx = U[:, :k] @ np.diag(s[:k]) @ Vt[:k, :]
# Approximation error
error = norm(A - A_approx, &#x27;fro&#x27;) / norm(A, &#x27;fro&#x27;)
print(f&quot;Relative error with k={k}: {error:.4f}&quot;)
# Sparse SVD (for large matrices, top k singular values only)
A_large = np.random.rand(1000, 500)
k = 20
U_k, s_k, Vt_k = svds(A_large, k=k)
# Note: svds returns singular values in ASCENDING order
# Reverse them for consistency
idx = np.argsort(s_k)[::-1]
s_k = s_k[idx]
U_k = U_k[:, idx]
Vt_k = Vt_k[idx, :]
# Matrix pseudoinverse via SVD
tolerance = 1e-10
s_inv = np.array([1/si if si &gt; tolerance else 0 for si in s])
A_pinv = Vt.T @ np.diag(s_inv) @ U.T</code></pre>
<p><strong>Eigenvalue Decomposition</strong></p>
<pre><code class="lang-python">from numpy.linalg import eig, eigh
from scipy.linalg import eig as scipy_eig
# General matrix (possibly complex eigenvalues)
A = np.random.rand(100, 100)
eigenvalues, eigenvectors = eig(A)
# A @ v = λ * v
# eigenvectors[:, i] corresponds to eigenvalues[i]
# Symmetric/Hermitian matrices (real eigenvalues, orthogonal eigenvectors)
A_sym = (A + A.T) / 2
eigenvalues_sym, eigenvectors_sym = eigh(A_sym)
# eigenvalues are sorted in ascending order
# Verify decomposition
Lambda = np.diag(eigenvalues_sym)
assert np.allclose(A_sym, eigenvectors_sym @ Lambda @ eigenvectors_sym.T)
# Generalized eigenvalue problem: A @ v = λ * B @ v
B = np.random.rand(100, 100)
B_spd = B @ B.T + np.eye(100)
eigenvalues_gen, eigenvectors_gen = scipy_eig(A_sym, B_spd)
# Power iteration for dominant eigenvalue
def power_iteration(A, num_iterations=100):
v = np.random.rand(A.shape[1])
for _ in range(num_iterations):
v = A @ v
v = v / norm(v)
eigenvalue = (v @ A @ v) / (v @ v)
return eigenvalue, v
lambda_max, v_max = power_iteration(A_sym)</code></pre>
<p><strong>QR Decomposition</strong></p>
<pre><code class="lang-python">from numpy.linalg import qr
# A = Q @ R (Q orthogonal, R upper triangular)
A = np.random.rand(100, 50)
# Full QR (Q is 100x100)
Q_full, R_full = qr(A, mode=&#x27;complete&#x27;)
# Economic QR (Q is 100x50, preferred)
Q, R = qr(A, mode=&#x27;reduced&#x27;)
# Verify orthogonality
assert np.allclose(Q.T @ Q, np.eye(50))
assert np.allclose(A, Q @ R)
# Applications:
# 1. Least squares (already shown)
# 2. Gram-Schmidt orthogonalization
def gram_schmidt(A):
Q, R = qr(A)
return Q
# 3. QR algorithm for eigenvalues
def qr_eigenvalue_algorithm(A, iterations=100):
A_k = A.copy()
for _ in range(iterations):
Q, R = qr(A_k)
A_k = R @ Q
# Diagonal of A_k approximates eigenvalues
return np.diag(A_k)</code></pre>
<p><strong>Cholesky Decomposition</strong></p>
<pre><code class="lang-python">from numpy.linalg import cholesky
from scipy.linalg import cholesky as scipy_cholesky
# A = L @ L^T (A must be symmetric positive definite)
A_spd = np.random.rand(100, 50)
A_spd = A_spd @ A_spd.T + np.eye(100) # Force SPD
# Lower triangular
L = cholesky(A_spd)
assert np.allclose(A_spd, L @ L.T)
# Upper triangular (SciPy)
U = scipy_cholesky(A_spd, lower=False)
assert np.allclose(A_spd, U.T @ U)
# Solving Ax = b with Cholesky
# A = L @ L^T =&gt; L @ (L^T @ x) = b
# Solve L @ y = b, then L^T @ x = y
from scipy.linalg import solve_triangular
b = np.random.rand(100)
y = solve_triangular(L, b, lower=True)
x = solve_triangular(L.T, y, lower=False)
assert np.allclose(A_spd @ x, b)
# Updating Cholesky factorization (rank-1 update)
def cholesky_rank1_update(L, u):
&quot;&quot;&quot;Update L when A&#x27; = A + u @ u^T&quot;&quot;&quot;
from scipy.linalg.lapack import dchud
L_updated, info = dchud(L.T, u)
return L_updated.T</code></pre>
<h3>4. Dimensionality Reduction</h3>
<p><strong>Principal Component Analysis (PCA)</strong></p>
<pre><code class="lang-python">from sklearn.decomposition import PCA
# Data matrix: n samples x p features
X = np.random.rand(1000, 100)
# Center the data
X_centered = X - X.mean(axis=0)
# Method 1: SVD-based PCA (preferred for stability)
U, s, Vt = svd(X_centered, full_matrices=False)
# Principal components (eigenvectors of covariance)
components = Vt # p x p
# Explained variance
explained_variance = (s ** 2) / (X.shape[0] - 1)
explained_variance_ratio = explained_variance / explained_variance.sum()
# Project data onto top k components
k = 10
X_reduced = U[:, :k] @ np.diag(s[:k])
# Reconstruct approximation
X_reconstructed = X_reduced @ Vt[:k, :] + X.mean(axis=0)
# Method 2: Eigendecomposition of covariance matrix
cov_matrix = np.cov(X_centered.T) # p x p
eigenvalues, eigenvectors = eigh(cov_matrix)
# Sort in descending order
idx = np.argsort(eigenvalues)[::-1]
eigenvalues = eigenvalues[idx]
eigenvectors = eigenvectors[:, idx]
# Same principal components (up to sign)
assert np.allclose(np.abs(components[0]), np.abs(eigenvectors[:, 0]))
# Scikit-learn PCA (recommended for production)
pca = PCA(n_components=10)
X_pca = pca.fit_transform(X)
print(f&quot;Explained variance ratio: {pca.explained_variance_ratio_.sum():.4f}&quot;)</code></pre>
<p><strong>Randomized SVD (Fast Approximation)</strong></p>
<pre><code class="lang-python">from sklearn.utils.extmath import randomized_svd
# For very large matrices, randomized SVD is much faster
A_large = np.random.rand(10000, 5000)
k = 50
# Randomized SVD
U_rand, s_rand, Vt_rand = randomized_svd(A_large, n_components=k, random_state=42)
# Compare with truncated SVD
from scipy.sparse.linalg import svds
U_exact, s_exact, Vt_exact = svds(A_large, k=k)
# Randomized is approximate but much faster
# Error typically &lt; 1% for k &lt;&lt; min(m, n)</code></pre>
<h3>5. Sparse Matrices</h3>
<p><strong>Sparse Matrix Operations</strong></p>
<pre><code class="lang-python">from scipy.sparse import csr_matrix, csc_matrix, lil_matrix
from scipy.sparse.linalg import spsolve, gmres, cg
# Create sparse matrix (row-based for fast row operations)
row = np.array([0, 0, 1, 2, 2, 2])
col = np.array([0, 2, 2, 0, 1, 2])
data = np.array([1, 2, 3, 4, 5, 6])
A_sparse = csr_matrix((data, (row, col)), shape=(3, 3))
# Dense equivalent
A_dense = A_sparse.toarray()
# Matrix-vector multiplication
x = np.array([1, 2, 3])
y = A_sparse @ x # Fast for sparse matrices
# Solving sparse linear systems
n = 1000
# Create sparse positive definite matrix
from scipy.sparse import diags
A_sparse_spd = diags([1, -2, 1], [-1, 0, 1], shape=(n, n)).tocsr()
A_sparse_spd = A_sparse_spd @ A_sparse_spd.T + diags([10], [0], shape=(n, n))
b = np.random.rand(n)
# Direct solver (LU-based)
x_direct = spsolve(A_sparse_spd, b)
# Iterative solvers (better for very large systems)
# Conjugate gradient (for SPD matrices)
x_cg, info = cg(A_sparse_spd, b, tol=1e-8)
# GMRES (for general matrices)
x_gmres, info = gmres(A_sparse_spd, b, tol=1e-8)
# Check convergence
if info == 0:
print(&quot;Converged successfully&quot;)
elif info &gt; 0:
print(f&quot;Did not converge after {info} iterations&quot;)</code></pre>
<h2>Patterns and Best Practices</h2>
<h3>Pattern 1: Never Invert Matrices</h3>
<pre><code class="lang-python"># WRONG: Explicitly computing inverse
A = np.random.rand(100, 100)
b = np.random.rand(100)
x = np.linalg.inv(A) @ b # Slow and unstable!
# RIGHT: Use solve
x = np.linalg.solve(A, b) # Fast and stable!
# WRONG: Computing (A^T A)^-1 A^T b (normal equations)
A = np.random.rand(200, 100)
b = np.random.rand(200)
x = np.linalg.inv(A.T @ A) @ A.T @ b # Numerically unstable!
# RIGHT: Use lstsq
x = np.linalg.lstsq(A, b, rcond=None)[0] # Stable QR-based!</code></pre>
<h3>Pattern 2: Check Conditioning</h3>
<pre><code class="lang-python">def solve_with_conditioning_check(A, b, threshold=1e12):
cond = np.linalg.cond(A)
if cond &gt; threshold:
print(f&quot;Warning: Matrix is ill-conditioned (cond={cond:.2e})&quot;)
print(&quot;Consider regularization or preconditioning&quot;)
x = np.linalg.solve(A, b)
# Verify solution
residual = norm(A @ x - b) / norm(b)
if residual &gt; 1e-6:
print(f&quot;Warning: Large residual {residual:.2e}&quot;)
return x</code></pre>
<h3>Pattern 3: Memory-Efficient Matrix Operations</h3>
<pre><code class="lang-python"># Avoid creating intermediate large matrices
n = 10000
# WRONG: Creates intermediate n x n matrix
A = np.random.rand(n, n)
B = np.random.rand(n, n)
result = (A @ B) @ A.T # Two n x n intermediates!
# BETTER: Use einsum or explicit ordering
result = A @ (B @ A.T) # Only one intermediate
# BEST: For complex expressions, use einsum
# result = np.einsum(&#x27;ij,jk,ki-&gt;i&#x27;, A, B, A) # No intermediates!</code></pre>
<h3>Pattern 4: Numerical Stability</h3>
<pre><code class="lang-python"># Computing variance: two-pass algorithm
X = np.random.rand(1000, 100)
# WRONG: Naive variance (numerically unstable)
mean = X.mean(axis=0)
variance_naive = ((X - mean) ** 2).mean(axis=0)
# RIGHT: Use built-in (Welford&#x27;s algorithm)
variance_stable = np.var(X, axis=0)
# For covariance: use np.cov, not manual computation
cov_stable = np.cov(X.T)</code></pre>
<h2>Quick Reference</h2>
<h3>Function Selection Guide</h3>
<pre><code class="lang-python"># Solving Ax = b:
# - General: np.linalg.solve(A, b)
# - SPD: scipy.linalg.cho_solve(cho_factor(A), b)
# - Sparse: scipy.sparse.linalg.spsolve(A, b)
# - Overdetermined: np.linalg.lstsq(A, b)
# Matrix decomposition:
# - SVD: np.linalg.svd(A) or scipy.sparse.linalg.svds(A, k)
# - Eigenvalues: np.linalg.eigh(A) for symmetric
# - QR: np.linalg.qr(A)
# - Cholesky: np.linalg.cholesky(A) for SPD
# Matrix properties:
# - Norm: np.linalg.norm(A, ord=...)
# - Condition: np.linalg.cond(A)
# - Rank: np.linalg.matrix_rank(A)
# - Determinant: np.linalg.det(A)</code></pre>
<h3>Performance Tips</h3>
<ol><li>Use <code>@</code> instead of <code>np.dot()</code> for clarity and speed</li><li>Prefer <code>solve()</code> over <code>inv()</code> (10x faster, more accurate)</li><li>Use <code>eigh()</code> for symmetric matrices (2x faster than <code>eig()</code>)</li><li>Set <code>full_matrices=False</code> in SVD for efficiency</li><li>Use sparse matrices when &gt;90% zeros</li><li>Vectorize operations, avoid Python loops</li><li>Use <code>einsum()</code> for complex tensor operations</li></ol>
<h2>Anti-Patterns</h2>
<h3>Anti-Pattern 1: Matrix Inversion</h3>
<pre><code class="lang-python"># NEVER do this
x = np.linalg.inv(A) @ b
# ALWAYS do this
x = np.linalg.solve(A, b)</code></pre>
<h3>Anti-Pattern 2: Ignoring Conditioning</h3>
<pre><code class="lang-python"># WRONG: Blindly solving without checking
x = np.linalg.solve(A, b)
# RIGHT: Check condition number first
cond = np.linalg.cond(A)
if cond &gt; 1e12:
# Add regularization
A_reg = A + 1e-6 * np.eye(A.shape[0])
x = np.linalg.solve(A_reg, b)</code></pre>
<h3>Anti-Pattern 3: Inefficient Operations</h3>
<pre><code class="lang-python"># WRONG: Loop over rows/columns
result = np.zeros(n)
for i in range(n):
result[i] = A[i, :] @ x
# RIGHT: Vectorize
result = A @ x</code></pre>
<h3>Anti-Pattern 4: Premature Densification</h3>
<pre><code class="lang-python"># WRONG: Converting sparse to dense unnecessarily
A_sparse = csr_matrix(...)
A_dense = A_sparse.toarray() # Memory explosion!
x = np.linalg.solve(A_dense, b)
# RIGHT: Use sparse solvers
x = spsolve(A_sparse, b)</code></pre>
<h2>Troubleshooting</h2>
<h3>Issue: Singular Matrix Error</h3>
<pre><code class="lang-python"># Cause: Matrix is not invertible (det = 0 or near 0)
# Solution 1: Check rank
rank = np.linalg.matrix_rank(A)
if rank &lt; A.shape[0]:
print(&quot;Matrix is rank-deficient&quot;)
# Solution 2: Add regularization
A_reg = A + lambda_reg * np.eye(A.shape[0])
# Solution 3: Use pseudoinverse
x = np.linalg.pinv(A) @ b</code></pre>
<h3>Issue: Slow Performance</h3>
<pre><code class="lang-python"># Check if matrix is sparse
sparsity = 1 - np.count_nonzero(A) / A.size
if sparsity &gt; 0.9:
print(&quot;Use sparse matrix format!&quot;)
A_sparse = csr_matrix(A)</code></pre>
<h3>Issue: Numerical Instability</h3>
<pre><code class="lang-python"># Check condition number
cond = np.linalg.cond(A)
if cond &gt; 1 / np.finfo(float).eps:
print(&quot;Matrix is numerically singular!&quot;)
# Use more stable algorithm or regularization</code></pre>
<h2>Related Skills</h2>
<ul><li><code>optimization-algorithms.md</code> - Uses linear algebra for gradients, Hessians</li><li><code>numerical-methods.md</code> - Linear algebra for ODEs, PDEs</li><li><code>probability-statistics.md</code> - Covariance matrices, multivariate distributions</li><li><code>data-validation.md</code> - Matrix data quality checks</li></ul>
<h2>Learning Resources</h2>
<ul><li>NumPy Linear Algebra: https://numpy.org/doc/stable/reference/routines.linalg.html</li><li>SciPy Linear Algebra: https://docs.scipy.org/doc/scipy/reference/linalg.html</li><li>Matrix Computations (Golub &amp; Van Loan)</li><li>Numerical Linear Algebra (Trefethen &amp; Bau)</li></ul>
<footer>
<div class="fmap">
<div><b>Build</b>
<a href="https://hanzo.app">hanzo.app</a> app builder
<a href="https://hanzo.codes">hanzo.codes</a> code with an agent
<a href="https://hanzoskills.com">hanzoskills.com</a> what an agent reads</div>
<div><b>Run</b>
<a href="https://console.hanzo.ai">console.hanzo.ai</a> cloud console
<a href="https://api.hanzo.ai">api.hanzo.ai</a> the API</div>
<div><b>Account</b>
<a href="https://hanzo.id">hanzo.id</a> sign in
<a href="https://pay.hanzo.ai">pay.hanzo.ai</a> top up
<a href="https://billing.hanzo.ai">billing.hanzo.ai</a> invoices and usage</div>
<div><b>Company</b>
<a href="https://hanzo.ai">hanzo.ai</a> canonical
<a href="https://hanzo.ventures">hanzo.ventures</a> ventures</div>
</div>
<div class="fend"><a href="/math-linear-algebra-computation.md">This page as plain text</a> · <a href="/llms.txt">Index for agents</a></div>
</footer>
</div></body></html>