Files
skills/public/research-data-analysis.html
zeekay 747dafba10 Register WebMCP tools on every page, and name each page's twin from site.py
Every page now loads /webmcp.js, scripts/webmcp.ts with its types
stripped (npm run webmcp). Where the browser has document.modelContext
it registers two read-only tools: search_skills over the published
skills index, and get_skill, which returns a skill's markdown checked
against the index digest. The markdown alternate link moves from
build.py to site.py, so CI holds every page to it.

The API catalog gains the site itself (llms.txt and the skills index)
and hanzo.ai/agents; the live discovery check now fetches every
catalog link. tests/site.mjs drives the tools through a recording
modelContext, and checks a browser without one loads clean.
2026-10-26 20:57:05 -07:00

949 lines
42 KiB
HTML

<!doctype html>
<html lang="en"><head>
<meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
<title>research-data-analysis</title>
<meta name="color-scheme" content="dark">
<style>
/* Committed to one visual world: a dark instrument panel. No light variant —
the dot field and the hairline rules only read on black, and a naive
inversion would wash both out. Every colour is painted explicitly so the page
never borrows a host background. */
@font-face{font-family:'Zen';src:url('/fonts/Zen-Variable.woff2') format('woff2');font-weight:100 900;font-style:normal;font-display:swap}
@font-face{font-family:'Zen';src:url('/fonts/Zen-Italic-Variable.woff2') format('woff2');font-weight:100 900;font-style:italic;font-display:swap}
@font-face{font-family:'Zen Mono';src:url('/fonts/ZenMono-Variable.woff2') format('woff2');font-weight:100 900;font-style:normal;font-display:swap}
@font-face{font-family:'Zen Mono';src:url('/fonts/ZenMono-Italic-Variable.woff2') format('woff2');font-weight:100 900;font-style:italic;font-display:swap}
@font-face{font-family:'Zen Pixel Square';src:url('/fonts/ZenPixel-Square.woff2') format('woff2');font-style:normal;font-display:swap}
:root{
--ink:#f4f4f2; --soft:#9c9c98; --faint:#6a6a66; --ground:#000; --panel:#0a0a0a;
--line:#232323; --line-hi:#3a3a3a; --accent:#f4f4f2;
--font-zen-sans:'Zen',-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif;
--font-zen-mono:'Zen Mono',ui-monospace,"SF Mono",SFMono-Regular,Menlo,Consolas,monospace;
--font-zen-pixel-square:'Zen Pixel Square',monospace;
--mono:var(--font-zen-mono);
--sans:var(--font-zen-sans);
/* Controls are pills and surfaces are soft, as on hanzo.ai. */
--r:999px; --r-soft:14px;
}
*{box-sizing:border-box}
html{-webkit-text-size-adjust:100%}
body{margin:0;background:var(--ground);color:var(--ink);font-family:var(--sans);
font-size:16px;line-height:1.65;-webkit-font-smoothing:antialiased}
.wrap{max-width:60rem;margin:0 auto;padding:0 1.75rem}
/* The dot field. A halftone made of one repeating radial-gradient, faded out
along the diagonal by a mask so it reads as a wave rather than wallpaper. */
/* No overflow clip here: the dot field is inset to the box already, and a clip
would cut off the copy-prompt menu. */
.hero{position:relative;border-bottom:1px solid var(--line)}
.hero::before{content:"";position:absolute;inset:0;
background-image:radial-gradient(var(--line-hi) 1px, transparent 1px);
background-size:7px 7px;
-webkit-mask-image:linear-gradient(115deg,#000 0%,rgba(0,0,0,.55) 38%,transparent 72%);
mask-image:linear-gradient(115deg,#000 0%,rgba(0,0,0,.55) 38%,transparent 72%);
pointer-events:none}
.hero .wrap{position:relative;padding-top:5.5rem;padding-bottom:4rem}
nav{border-bottom:1px solid var(--line)}
nav .in{max-width:60rem;margin:0 auto;padding:1.1rem 1.75rem;display:flex;
justify-content:space-between;align-items:center;gap:1rem}
.brand{font-family:var(--mono);font-size:.85rem;letter-spacing:.14em;
text-transform:uppercase;color:var(--ink);text-decoration:none;
display:inline-flex;align-items:center;gap:.6rem}
.brand i{font-style:normal;color:var(--faint)}
.mark{width:1.15em;height:1.15em;display:block;flex:0 0 auto}
nav a.up{font-family:var(--mono);font-size:.72rem;letter-spacing:.14em;
text-transform:uppercase;color:var(--soft);text-decoration:none}
nav a.up:hover{color:var(--ink)}
h1{font-family:var(--mono);font-size:clamp(2rem,5.4vw,3.4rem);line-height:1.12;
letter-spacing:-.02em;margin:0 0 1.1rem;font-weight:500;text-wrap:balance}
h1 em{font-style:normal;color:var(--faint)}
.lede{color:var(--soft);font-size:1.05rem;max-width:52ch;margin:0 0 2.25rem}
.btns{display:flex;flex-wrap:wrap;gap:.7rem;margin:0 0 .5rem;align-items:center}
.btn{display:inline-flex;align-items:center;gap:.5rem;font-family:var(--mono);
font-size:.72rem;letter-spacing:.14em;text-transform:uppercase;
padding:.8rem 1.4rem;border:1px solid var(--line-hi);border-radius:var(--r);
color:var(--ink);text-decoration:none;background:transparent;cursor:pointer;
line-height:1.2;font-weight:400}
.btn:hover{background:#141414;text-decoration:none;border-color:var(--soft)}
.btn.solid{background:var(--ink);color:#000;border-color:var(--ink)}
.btn.solid:hover{background:#d8d8d4}
.btn .chev{width:.7em;height:.7em;flex:0 0 auto;transition:transform .15s ease}
.btn[aria-expanded="true"] .chev{transform:rotate(180deg)}
/* Copy-prompt selector. Each row copies the prompt for its own target. */
.copy{position:relative;display:inline-flex}
.menu{position:absolute;top:calc(100% + .5rem);left:0;z-index:20;min-width:21rem;
background:var(--panel);border:1px solid var(--line-hi);border-radius:var(--r-soft);
padding:.4rem;display:flex;flex-direction:column;
box-shadow:0 12px 32px rgba(0,0,0,.75)}
.menu[hidden]{display:none}
.mi{display:flex;align-items:center;gap:.7rem;width:100%;text-align:left;
padding:.62rem .8rem;border:0;border-radius:calc(var(--r-soft) - .35rem);
background:transparent;color:var(--ink);font-family:var(--mono);font-size:.76rem;
letter-spacing:.04em;text-decoration:none;cursor:pointer;line-height:1.3;
white-space:nowrap}
.mi:hover,.mi:focus-visible{background:#181818;outline:none}
.mi .mark{width:1.05rem;height:1.05rem;color:var(--soft)}
.mi:hover .mark{color:var(--ink)}
.mi u{margin-left:auto;color:var(--faint);text-decoration:none;font-size:.68rem;
letter-spacing:.1em;text-transform:uppercase}
/* Works-with bar: one scrolling row, name and mark per agent. */
.strip{border-bottom:1px solid var(--line);background:var(--panel)}
.strip .in{max-width:60rem;margin:0 auto;padding:.9rem 1.75rem;display:flex;
align-items:center;gap:1.25rem;font-family:var(--mono);font-size:.72rem;
letter-spacing:.1em;text-transform:uppercase;color:var(--faint)}
.strip b{color:var(--soft);font-weight:400;flex:0 0 auto;
border-right:1px solid var(--line);padding-right:1.25rem;align-self:stretch;
display:flex;align-items:center}
.bar{display:flex;align-items:center;gap:.5rem;flex:1 1 auto;min-width:0;
overflow-x:auto;scrollbar-width:none;scroll-snap-type:x proximity;
-webkit-overflow-scrolling:touch;
-webkit-mask-image:linear-gradient(90deg,#000 calc(100% - 3rem),transparent);
mask-image:linear-gradient(90deg,#000 calc(100% - 3rem),transparent)}
.bar::-webkit-scrollbar{display:none}
.agent{display:inline-flex;align-items:center;gap:.45rem;flex:0 0 auto;
padding:.4rem .85rem;border:1px solid var(--line);border-radius:var(--r);
color:var(--soft);white-space:nowrap;scroll-snap-align:start;
text-decoration:none;background:var(--ground)}
.agent:hover{color:var(--ink);border-color:var(--line-hi);text-decoration:none}
.agent .mark{width:.95rem;height:.95rem}
h2{font-family:var(--mono);font-size:1.1rem;letter-spacing:-.01em;margin:2.75rem 0 .75rem;font-weight:500}
h3{font-size:1rem;margin:2rem 0 .5rem;font-weight:600}
h4,h5,h6{font-size:.94rem;margin:1.5rem 0 .5rem;font-weight:600}
p{margin:0 0 1rem;max-width:70ch}
ul,ol{margin:0 0 1rem;padding-left:1.35rem;max-width:70ch}
li{margin:.22rem 0}
a{color:var(--ink);text-decoration:underline;text-underline-offset:3px;
text-decoration-color:var(--line-hi)}
a:hover{text-decoration-color:var(--ink)}
a:focus-visible{outline:1px solid var(--ink);outline-offset:3px}
code{font-family:var(--mono);font-size:.85em;background:var(--panel);
border:1px solid var(--line);border-radius:5px;padding:.1em .36em}
pre{background:var(--panel);border:1px solid var(--line);padding:1rem 1.1rem;
border-radius:var(--r-soft);overflow-x:auto;margin:0 0 1.25rem}
pre code{background:none;border:0;padding:0;font-size:.82rem;line-height:1.7}
blockquote{margin:0 0 1rem;padding-left:1rem;border-left:1px solid var(--line-hi);color:var(--soft)}
hr{border:0;border-top:1px solid var(--line);margin:2.5rem 0}
table{border-collapse:collapse;width:100%;margin:0 0 1.25rem;font-size:.92rem;
display:block;overflow-x:auto}
th,td{border:1px solid var(--line);padding:.5rem .7rem;text-align:left}
.curl{display:block;background:var(--panel);border:1px solid var(--line);
border-radius:var(--r-soft);padding:.95rem 1.1rem;font-family:var(--mono);
font-size:.8rem;color:var(--soft);overflow-x:auto;margin:0 0 2.5rem;
white-space:pre;line-height:1.9}
.curl b{color:var(--ink);font-weight:400}
.curl i{font-style:normal;color:var(--faint)}
/* index */
#q{width:100%;padding:.85rem 1.2rem;font-family:var(--mono);font-size:.85rem;color:var(--ink);
background:var(--panel);border:1px solid var(--line);border-radius:var(--r);margin:0 0 .6rem}
#q::placeholder{color:var(--faint);text-transform:uppercase;letter-spacing:.1em;font-size:.72rem}
#q:focus{outline:none;border-color:var(--soft)}
#count{color:var(--faint);font-family:var(--mono);font-size:.72rem;letter-spacing:.1em;
text-transform:uppercase;margin:0 0 2rem}
.cat{margin:2.5rem 0 .5rem;font-family:var(--mono);font-size:.7rem;letter-spacing:.16em;
text-transform:uppercase;color:var(--faint);border-bottom:1px solid var(--line);
padding-bottom:.45rem}
.row{display:block;padding:.6rem .9rem;margin:0 -.9rem;border-bottom:1px solid var(--line);
border-radius:var(--r-soft);text-decoration:none}
.row:hover{background:#0a0a0a}
.nm{font-family:var(--mono);font-size:.84rem;color:var(--ink);display:block}
.ds{color:var(--soft);font-size:.9rem;display:block;margin-top:.12rem}
footer{margin-top:4rem;padding:2rem 0 4rem;border-top:1px solid var(--line);
color:var(--faint);font-size:.85rem}
footer a{color:var(--soft)}
.fmap{display:grid;gap:1.75rem;grid-template-columns:repeat(auto-fit,minmax(11rem,1fr));
margin:0 0 2rem;font-size:.85rem}
.fmap b{display:block;font-family:var(--mono);font-size:.68rem;letter-spacing:.16em;
text-transform:uppercase;color:var(--faint);font-weight:400;margin:0 0 .6rem;
border-bottom:1px solid var(--line);padding-bottom:.4rem}
.fmap a{display:inline-block;color:var(--ink);text-decoration:none;margin:.3rem .35rem 0 0}
.fmap a:hover{text-decoration:underline}
.fmap div{color:var(--faint);line-height:1.5}
.fend{border-top:1px solid var(--line);padding-top:1.25rem;font-size:.82rem}
@media (max-width:640px){
.hero .wrap{padding-top:3.25rem;padding-bottom:2.75rem}
.btns{gap:.5rem}
.btn{padding:.75rem 1.15rem;font-size:.68rem}
.copy,.copy .btn{width:100%;justify-content:center}
.menu{left:0;right:0;min-width:0}
.strip .in{flex-direction:column;align-items:flex-start;gap:.6rem;padding:.9rem 1.75rem}
.strip b{border-right:0;padding-right:0}
.bar{width:100%}
}
</style>
<link rel="icon" href="/favicon.svg">
<link rel="canonical" href="https://hanzoskills.com/research-data-analysis">
<link rel="alternate" type="text/markdown" href="/research-data-analysis.md">
<script type="module" src="/webmcp.js"></script>
</head>
<!-- Reading this with a machine? Every page here is also plain markdown.
This one: https://hanzoskills.com/research-data-analysis.md
Index: https://hanzoskills.com/llms.txt
Corpus: https://hanzoskills.com/all.md
Append .md to any path for the markdown twin. -->
<body>
<nav><div class="in">
<a class="brand" href="/"><svg class="mark" viewBox="0 0 67 67" fill="currentColor" aria-hidden="true"><path d="M22.21 67V44.6369H0V67H22.21Z"/><path d="M66.7038 22.3184H22.2534L0.0878906 44.6367H44.4634L66.7038 22.3184Z"/><path d="M22.21 0H0V22.3184H22.21V0Z"/><path d="M66.7198 0H44.5098V22.3184H66.7198V0Z"/><path d="M66.7198 67V44.6369H44.5098V67H66.7198Z"/></svg> HANZO<i>SKILLS</i></a>
<a class="up" href="/">All skills</a>
</div></nav>
<div class="wrap">
<h1>research-data-analysis</h1><p class="lede">Master data analysis techniques including coding, statistical tests, visualization, interpretation, and reporting findings</p><h1>Data Analysis Skill</h1>
<h2>When to Use This Skill</h2>
<p>Use this skill when you need to:</p>
<ul><li>Analyze quantitative or qualitative data</li><li>Code textual data systematically</li><li>Choose appropriate statistical tests</li><li>Visualize data effectively</li><li>Interpret analysis results</li><li>Check analysis assumptions</li><li>Report findings clearly</li><li>Handle complex or messy data</li></ul>
<h2>Quantitative Data Analysis</h2>
<h3>1. Descriptive Statistics</h3>
<p><strong>Comprehensive Description Framework</strong>:</p>
<pre><code class="lang-python">import pandas as pd
import numpy as np
from scipy import stats
import matplotlib.pyplot as plt
import seaborn as sns
class DescriptiveAnalysis:
&quot;&quot;&quot;Comprehensive descriptive statistics&quot;&quot;&quot;
def __init__(self, data: pd.DataFrame):
self.data = data
def continuous_summary(self, var: str):
&quot;&quot;&quot;Summarize continuous variable&quot;&quot;&quot;
series = self.data[var].dropna()
summary = {
&#x27;n&#x27;: len(series),
&#x27;missing&#x27;: self.data[var].isna().sum(),
&#x27;mean&#x27;: series.mean(),
&#x27;median&#x27;: series.median(),
&#x27;std&#x27;: series.std(),
&#x27;min&#x27;: series.min(),
&#x27;max&#x27;: series.max(),
&#x27;q25&#x27;: series.quantile(0.25),
&#x27;q75&#x27;: series.quantile(0.75),
&#x27;skewness&#x27;: stats.skew(series),
&#x27;kurtosis&#x27;: stats.kurtosis(series)
}
# Add confidence interval for mean
ci = stats.t.interval(0.95, len(series)-1,
loc=summary[&#x27;mean&#x27;],
scale=stats.sem(series))
summary[&#x27;ci_95_lower&#x27;] = ci[0]
summary[&#x27;ci_95_upper&#x27;] = ci[1]
return summary
def categorical_summary(self, var: str):
&quot;&quot;&quot;Summarize categorical variable&quot;&quot;&quot;
counts = self.data[var].value_counts()
proportions = self.data[var].value_counts(normalize=True)
summary = pd.DataFrame({
&#x27;Count&#x27;: counts,
&#x27;Proportion&#x27;: proportions,
&#x27;Percentage&#x27;: proportions * 100
})
summary[&#x27;95% CI Lower&#x27;] = summary.apply(
lambda row: self._proportion_ci(row[&#x27;Count&#x27;], len(self.data))[0],
axis=1
)
summary[&#x27;95% CI Upper&#x27;] = summary.apply(
lambda row: self._proportion_ci(row[&#x27;Count&#x27;], len(self.data))[1],
axis=1
)
return summary
@staticmethod
def _proportion_ci(count, n, confidence=0.95):
&quot;&quot;&quot;Wilson score confidence interval for proportion&quot;&quot;&quot;
from statsmodels.stats.proportion import proportion_confint
return proportion_confint(count, n, alpha=1-confidence, method=&#x27;wilson&#x27;)
def create_descriptive_table(self, continuous_vars: list,
categorical_vars: list):
&quot;&quot;&quot;Create Table 1 style descriptive statistics&quot;&quot;&quot;
table = []
for var in continuous_vars:
summary = self.continuous_summary(var)
table.append({
&#x27;Variable&#x27;: var,
&#x27;Type&#x27;: &#x27;Continuous&#x27;,
&#x27;Summary&#x27;: f&quot;{summary[&#x27;mean&#x27;]:.2f} ± {summary[&#x27;std&#x27;]:.2f}&quot;,
&#x27;Range&#x27;: f&quot;[{summary[&#x27;min&#x27;]:.2f}, {summary[&#x27;max&#x27;]:.2f}]&quot;,
&#x27;N&#x27;: summary[&#x27;n&#x27;]
})
for var in categorical_vars:
summary = self.categorical_summary(var)
for category, row in summary.iterrows():
table.append({
&#x27;Variable&#x27;: f&quot;{var}: {category}&quot;,
&#x27;Type&#x27;: &#x27;Categorical&#x27;,
&#x27;Summary&#x27;: f&quot;{row[&#x27;Count&#x27;]} ({row[&#x27;Percentage&#x27;]:.1f}%)&quot;,
&#x27;Range&#x27;: f&quot;[{row[&#x27;95% CI Lower&#x27;]:.3f}, &quot;
f&quot;{row[&#x27;95% CI Upper&#x27;]:.3f}]&quot;,
&#x27;N&#x27;: row[&#x27;Count&#x27;]
})
return pd.DataFrame(table)
def visualize_distribution(self, var: str, var_type: str = &#x27;continuous&#x27;):
&quot;&quot;&quot;Create distribution visualizations&quot;&quot;&quot;
fig, axes = plt.subplots(1, 2, figsize=(12, 4))
if var_type == &#x27;continuous&#x27;:
# Histogram with KDE
axes[0].hist(self.data[var].dropna(), bins=30,
density=True, alpha=0.7, edgecolor=&#x27;black&#x27;)
self.data[var].plot.kde(ax=axes[0], color=&#x27;red&#x27;, linewidth=2)
axes[0].set_xlabel(var)
axes[0].set_ylabel(&#x27;Density&#x27;)
axes[0].set_title(f&#x27;Distribution of {var}&#x27;)
# Q-Q plot for normality
stats.probplot(self.data[var].dropna(), dist=&quot;norm&quot;, plot=axes[1])
axes[1].set_title(&#x27;Q-Q Plot (Normality Check)&#x27;)
else: # categorical
# Bar chart
counts = self.data[var].value_counts()
axes[0].bar(range(len(counts)), counts.values)
axes[0].set_xticks(range(len(counts)))
axes[0].set_xticklabels(counts.index, rotation=45, ha=&#x27;right&#x27;)
axes[0].set_ylabel(&#x27;Count&#x27;)
axes[0].set_title(f&#x27;Frequency of {var}&#x27;)
# Proportion with CI
summary = self.categorical_summary(var)
x = range(len(summary))
axes[1].bar(x, summary[&#x27;Percentage&#x27;])
axes[1].errorbar(x,
summary[&#x27;Percentage&#x27;],
yerr=[summary[&#x27;Percentage&#x27;] - summary[&#x27;95% CI Lower&#x27;]*100,
summary[&#x27;95% CI Upper&#x27;]*100 - summary[&#x27;Percentage&#x27;]],
fmt=&#x27;none&#x27;, color=&#x27;black&#x27;, capsize=5)
axes[1].set_xticks(x)
axes[1].set_xticklabels(summary.index, rotation=45, ha=&#x27;right&#x27;)
axes[1].set_ylabel(&#x27;Percentage&#x27;)
axes[1].set_title(f&#x27;Proportion with 95% CI&#x27;)
plt.tight_layout()
return fig
# Example usage
data = pd.DataFrame({
&#x27;age&#x27;: np.random.normal(35, 10, 100),
&#x27;satisfaction&#x27;: np.random.choice([1, 2, 3, 4, 5], 100),
&#x27;group&#x27;: np.random.choice([&#x27;Control&#x27;, &#x27;Treatment&#x27;], 100)
})
analysis = DescriptiveAnalysis(data)
print(analysis.continuous_summary(&#x27;age&#x27;))
print(analysis.categorical_summary(&#x27;group&#x27;))
table = analysis.create_descriptive_table(
continuous_vars=[&#x27;age&#x27;],
categorical_vars=[&#x27;group&#x27;, &#x27;satisfaction&#x27;]
)
print(table)</code></pre>
<h3>2. Inferential Statistics</h3>
<p><strong>Comprehensive Testing Framework</strong>:</p>
<pre><code class="lang-python">from scipy import stats
import pandas as pd
import numpy as np
from typing import Tuple, Dict
class InferentialAnalysis:
&quot;&quot;&quot;Statistical inference with proper reporting&quot;&quot;&quot;
@staticmethod
def t_test(group1, group2, paired: bool = False,
alpha: float = 0.05) -&gt; Dict:
&quot;&quot;&quot;Conduct t-test with full reporting&quot;&quot;&quot;
g1 = np.array(group1)
g2 = np.array(group2)
# Remove NaN
if paired:
mask = ~(np.isnan(g1) | np.isnan(g2))
g1, g2 = g1[mask], g2[mask]
t_stat, p_value = stats.ttest_rel(g1, g2)
df = len(g1) - 1
# Cohen&#x27;s d for paired
diff = g1 - g2
cohens_d = np.mean(diff) / np.std(diff, ddof=1)
else:
g1 = g1[~np.isnan(g1)]
g2 = g2[~np.isnan(g2)]
t_stat, p_value = stats.ttest_ind(g1, g2)
df = len(g1) + len(g2) - 2
# Cohen&#x27;s d for independent
pooled_std = np.sqrt(((len(g1)-1)*np.var(g1, ddof=1) +
(len(g2)-1)*np.var(g2, ddof=1)) / df)
cohens_d = (np.mean(g1) - np.mean(g2)) / pooled_std
# Effect size interpretation
if abs(cohens_d) &lt; 0.2:
effect_size = &#x27;negligible&#x27;
elif abs(cohens_d) &lt; 0.5:
effect_size = &#x27;small&#x27;
elif abs(cohens_d) &lt; 0.8:
effect_size = &#x27;medium&#x27;
else:
effect_size = &#x27;large&#x27;
return {
&#x27;test&#x27;: &#x27;paired t-test&#x27; if paired else &#x27;independent t-test&#x27;,
&#x27;t_statistic&#x27;: t_stat,
&#x27;df&#x27;: df,
&#x27;p_value&#x27;: p_value,
&#x27;significant&#x27;: p_value &lt; alpha,
&#x27;mean_1&#x27;: np.mean(g1),
&#x27;mean_2&#x27;: np.mean(g2),
&#x27;mean_diff&#x27;: np.mean(g1) - np.mean(g2),
&#x27;cohens_d&#x27;: cohens_d,
&#x27;effect_size&#x27;: effect_size,
&#x27;n_1&#x27;: len(g1),
&#x27;n_2&#x27;: len(g2)
}
@staticmethod
def anova(groups: list, alpha: float = 0.05) -&gt; Dict:
&quot;&quot;&quot;One-way ANOVA with post-hoc tests&quot;&quot;&quot;
# ANOVA
f_stat, p_value = stats.f_oneway(*groups)
df_between = len(groups) - 1
df_within = sum(len(g) for g in groups) - len(groups)
# Effect size (eta-squared)
grand_mean = np.mean([x for g in groups for x in g])
ss_between = sum(len(g) * (np.mean(g) - grand_mean)**2
for g in groups)
ss_total = sum((x - grand_mean)**2 for g in groups for x in g)
eta_squared = ss_between / ss_total
result = {
&#x27;test&#x27;: &#x27;One-way ANOVA&#x27;,
&#x27;f_statistic&#x27;: f_stat,
&#x27;df_between&#x27;: df_between,
&#x27;df_within&#x27;: df_within,
&#x27;p_value&#x27;: p_value,
&#x27;significant&#x27;: p_value &lt; alpha,
&#x27;eta_squared&#x27;: eta_squared,
&#x27;n_groups&#x27;: len(groups)
}
# Post-hoc pairwise comparisons if significant
if p_value &lt; alpha:
from itertools import combinations
pairwise = []
for i, j in combinations(range(len(groups)), 2):
t_result = InferentialAnalysis.t_test(groups[i], groups[j])
# Bonferroni correction
adjusted_alpha = alpha / (len(groups) * (len(groups) - 1) / 2)
pairwise.append({
&#x27;comparison&#x27;: f&#x27;Group {i+1} vs Group {j+1}&#x27;,
&#x27;p_value&#x27;: t_result[&#x27;p_value&#x27;],
&#x27;significant_bonferroni&#x27;: t_result[&#x27;p_value&#x27;] &lt; adjusted_alpha,
&#x27;mean_diff&#x27;: t_result[&#x27;mean_diff&#x27;]
})
result[&#x27;post_hoc&#x27;] = pairwise
return result
@staticmethod
def correlation(x, y, method: str = &#x27;pearson&#x27;,
alpha: float = 0.05) -&gt; Dict:
&quot;&quot;&quot;Correlation with confidence interval&quot;&quot;&quot;
# Remove NaN
mask = ~(np.isnan(x) | np.isnan(y))
x_clean = np.array(x)[mask]
y_clean = np.array(y)[mask]
if method == &#x27;pearson&#x27;:
r, p_value = stats.pearsonr(x_clean, y_clean)
elif method == &#x27;spearman&#x27;:
r, p_value = stats.spearmanr(x_clean, y_clean)
else:
raise ValueError(&quot;method must be &#x27;pearson&#x27; or &#x27;spearman&#x27;&quot;)
n = len(x_clean)
# Fisher&#x27;s z transformation for CI
if method == &#x27;pearson&#x27;:
z = 0.5 * np.log((1 + r) / (1 - r))
se = 1 / np.sqrt(n - 3)
z_crit = stats.norm.ppf(1 - alpha/2)
ci_lower = np.tanh(z - z_crit * se)
ci_upper = np.tanh(z + z_crit * se)
else:
ci_lower, ci_upper = None, None
# Coefficient of determination
r_squared = r ** 2
return {
&#x27;test&#x27;: f&#x27;{method.capitalize()} correlation&#x27;,
&#x27;r&#x27;: r,
&#x27;r_squared&#x27;: r_squared,
&#x27;p_value&#x27;: p_value,
&#x27;significant&#x27;: p_value &lt; alpha,
&#x27;ci_95_lower&#x27;: ci_lower,
&#x27;ci_95_upper&#x27;: ci_upper,
&#x27;n&#x27;: n
}
@staticmethod
def chi_square(contingency_table, alpha: float = 0.05) -&gt; Dict:
&quot;&quot;&quot;Chi-square test of independence&quot;&quot;&quot;
chi2, p_value, df, expected = stats.chi2_contingency(contingency_table)
# Effect size (Cramer&#x27;s V)
n = np.sum(contingency_table)
min_dim = min(contingency_table.shape[0] - 1,
contingency_table.shape[1] - 1)
cramers_v = np.sqrt(chi2 / (n * min_dim))
return {
&#x27;test&#x27;: &#x27;Chi-square test of independence&#x27;,
&#x27;chi2_statistic&#x27;: chi2,
&#x27;df&#x27;: df,
&#x27;p_value&#x27;: p_value,
&#x27;significant&#x27;: p_value &lt; alpha,
&#x27;cramers_v&#x27;: cramers_v,
&#x27;expected_frequencies&#x27;: expected
}
@staticmethod
def format_apa(result: Dict) -&gt; str:
&quot;&quot;&quot;Format result in APA style&quot;&quot;&quot;
test = result[&#x27;test&#x27;]
if &#x27;t-test&#x27; in test:
return (f&quot;t({result[&#x27;df&#x27;]}) = {result[&#x27;t_statistic&#x27;]:.2f}, &quot;
f&quot;p = {result[&#x27;p_value&#x27;]:.3f}, d = {result[&#x27;cohens_d&#x27;]:.2f}&quot;)
elif &#x27;ANOVA&#x27; in test:
return (f&quot;F({result[&#x27;df_between&#x27;]}, {result[&#x27;df_within&#x27;]}) = &quot;
f&quot;{result[&#x27;f_statistic&#x27;]:.2f}, p = {result[&#x27;p_value&#x27;]:.3f}, &quot;
f&quot;η² = {result[&#x27;eta_squared&#x27;]:.2f}&quot;)
elif &#x27;correlation&#x27; in test:
return (f&quot;r({result[&#x27;n&#x27;]-2}) = {result[&#x27;r&#x27;]:.2f}, &quot;
f&quot;p = {result[&#x27;p_value&#x27;]:.3f}&quot;)
elif &#x27;Chi-square&#x27; in test:
return (f&quot;χ²({result[&#x27;df&#x27;]}) = {result[&#x27;chi2_statistic&#x27;]:.2f}, &quot;
f&quot;p = {result[&#x27;p_value&#x27;]:.3f}, V = {result[&#x27;cramers_v&#x27;]:.2f}&quot;)
return str(result)
# Example usage
control = np.random.normal(50, 10, 30)
treatment = np.random.normal(55, 10, 30)
result = InferentialAnalysis.t_test(control, treatment)
print(InferentialAnalysis.format_apa(result))
print(f&quot;Interpretation: {result[&#x27;effect_size&#x27;]} effect size&quot;)</code></pre>
<h2>Qualitative Data Analysis</h2>
<h3>1. Coding Process</h3>
<p><strong>Systematic Coding Framework</strong>:</p>
<pre><code class="lang-python">from dataclasses import dataclass
from typing import List, Dict, Optional
from collections import defaultdict
import pandas as pd
@dataclass
class Code:
&quot;&quot;&quot;Represent a qualitative code&quot;&quot;&quot;
name: str
definition: str
when_to_use: str
when_not_to_use: str
examples: List[str]
@dataclass
class CodedSegment:
&quot;&quot;&quot;Represent coded text segment&quot;&quot;&quot;
source: str
text: str
codes: List[str]
line_numbers: tuple
memo: str = &quot;&quot;
class QualitativeCoding:
&quot;&quot;&quot;Systematic qualitative coding&quot;&quot;&quot;
def __init__(self):
self.codebook = {}
self.coded_segments = []
self.memos = []
def add_code(self, code: Code):
&quot;&quot;&quot;Add code to codebook&quot;&quot;&quot;
self.codebook[code.name] = code
def apply_code(self, segment: CodedSegment):
&quot;&quot;&quot;Apply codes to text segment&quot;&quot;&quot;
# Validate codes exist
for code in segment.codes:
if code not in self.codebook:
print(f&quot;Warning: Code &#x27;{code}&#x27; not in codebook&quot;)
self.coded_segments.append(segment)
def add_memo(self, memo: str, codes: Optional[List[str]] = None):
&quot;&quot;&quot;Write analytical memo&quot;&quot;&quot;
self.memos.append({
&#x27;memo&#x27;: memo,
&#x27;codes&#x27;: codes,
&#x27;date&#x27;: pd.Timestamp.now()
})
def get_code_frequency(self):
&quot;&quot;&quot;Count code applications&quot;&quot;&quot;
freq = defaultdict(int)
for segment in self.coded_segments:
for code in segment.codes:
freq[code] += 1
return dict(freq)
def get_coded_segments(self, code: str) -&gt; List[CodedSegment]:
&quot;&quot;&quot;Retrieve all segments with a specific code&quot;&quot;&quot;
return [seg for seg in self.coded_segments if code in seg.codes]
def co_occurrence_matrix(self):
&quot;&quot;&quot;Create code co-occurrence matrix&quot;&quot;&quot;
codes = list(self.codebook.keys())
matrix = pd.DataFrame(0, index=codes, columns=codes)
for segment in self.coded_segments:
seg_codes = segment.codes
for code1 in seg_codes:
for code2 in seg_codes:
if code1 != code2:
matrix.loc[code1, code2] += 1
return matrix
def inter_rater_reliability(self, coder2_segments: List[CodedSegment]):
&quot;&quot;&quot;Calculate Cohen&#x27;s Kappa for inter-rater reliability&quot;&quot;&quot;
# Match segments by source and line numbers
agreements = 0
total = 0
for seg1 in self.coded_segments:
# Find matching segment from coder 2
matching = [s for s in coder2_segments
if s.source == seg1.source and
s.line_numbers == seg1.line_numbers]
if matching:
seg2 = matching[0]
# Check if codes match
if set(seg1.codes) == set(seg2.codes):
agreements += 1
total += 1
if total == 0:
return None
observed_agreement = agreements / total
# For simplicity, calculate expected agreement assuming random
# (Full Cohen&#x27;s Kappa requires more complex calculation)
code_freq = self.get_code_frequency()
total_codes = sum(code_freq.values())
expected_agreement = sum((freq/total_codes)**2
for freq in code_freq.values())
kappa = (observed_agreement - expected_agreement) / (1 - expected_agreement)
return {
&#x27;observed_agreement&#x27;: observed_agreement,
&#x27;expected_agreement&#x27;: expected_agreement,
&#x27;cohens_kappa&#x27;: kappa,
&#x27;interpretation&#x27;: &#x27;substantial&#x27; if kappa &gt; 0.6 else
&#x27;moderate&#x27; if kappa &gt; 0.4 else &#x27;fair&#x27;
}
def generate_codebook_document(self):
&quot;&quot;&quot;Export codebook&quot;&quot;&quot;
doc = &quot;# Qualitative Codebook\n\n&quot;
for code_name, code in self.codebook.items():
doc += f&quot;## {code_name}\n\n&quot;
doc += f&quot;**Definition**: {code.definition}\n\n&quot;
doc += f&quot;**When to use**: {code.when_to_use}\n\n&quot;
doc += f&quot;**When NOT to use**: {code.when_not_to_use}\n\n&quot;
doc += &quot;**Examples**:\n&quot;
for ex in code.examples:
doc += f&quot;- {ex}\n&quot;
doc += &quot;\n&quot;
# Add frequency
freq = self.get_code_frequency().get(code_name, 0)
doc += f&quot;*Applied {freq} times*\n\n&quot;
doc += &quot;---\n\n&quot;
return doc
# Example usage
coding = QualitativeCoding()
# Build codebook
coding.add_code(Code(
name=&#x27;boundary_work&#x27;,
definition=&#x27;Actions taken to create or maintain work-life boundaries&#x27;,
when_to_use=&#x27;When participant describes specific strategies or behaviors &#x27;
&#x27;for separating work and personal life&#x27;,
when_not_to_use=&#x27;When simply mentioning boundaries without action&#x27;,
examples=[
&#x27;&quot;I close my laptop at 5pm every day&quot;&#x27;,
&#x27;&quot;I have a separate room for work&quot;&#x27;
]
))
coding.add_code(Code(
name=&#x27;role_conflict&#x27;,
definition=&#x27;Tension between work and personal role demands&#x27;,
when_to_use=&#x27;When participant describes competing demands or identity conflicts&#x27;,
when_not_to_use=&#x27;When mentioning roles without conflict&#x27;,
examples=[
&#x27;&quot;I felt torn between meeting and picking up my kids&quot;&#x27;,
&#x27;&quot;I can\&#x27;t be fully present at home because I\&#x27;m thinking about work&quot;&#x27;
]
))
# Code data
coding.apply_code(CodedSegment(
source=&#x27;Interview_P001&#x27;,
text=&#x27;I always close my laptop at 5pm, even if work isn\&#x27;t done&#x27;,
codes=[&#x27;boundary_work&#x27;],
line_numbers=(45, 47),
memo=&#x27;Strong temporal boundary - clock-based&#x27;
))
coding.apply_code(CodedSegment(
source=&#x27;Interview_P001&#x27;,
text=&#x27;But then I feel guilty about unfinished tasks&#x27;,
codes=[&#x27;role_conflict&#x27;],
line_numbers=(48, 49),
memo=&#x27;Guilt suggests incomplete boundary - work identity bleeds through&#x27;
))
# Analyze
print(coding.get_code_frequency())
print(coding.generate_codebook_document())</code></pre>
<h3>2. Thematic Development</h3>
<p><strong>Theme Building Process</strong>:</p>
<pre><code class="lang-python">class ThematicDevelopment:
&quot;&quot;&quot;Develop and refine themes from codes&quot;&quot;&quot;
def __init__(self, coding: QualitativeCoding):
self.coding = coding
self.themes = {}
def create_theme(self, theme_name: str, codes: List[str],
description: str):
&quot;&quot;&quot;Group codes into theme&quot;&quot;&quot;
self.themes[theme_name] = {
&#x27;codes&#x27;: codes,
&#x27;description&#x27;: description,
&#x27;definition&#x27;: &#x27;&#x27;,
&#x27;subthemes&#x27;: []
}
def define_theme(self, theme_name: str, definition: str):
&quot;&quot;&quot;Provide clear theme definition&quot;&quot;&quot;
self.themes[theme_name][&#x27;definition&#x27;] = definition
def add_subtheme(self, theme_name: str, subtheme_name: str,
codes: List[str]):
&quot;&quot;&quot;Add subtheme&quot;&quot;&quot;
self.themes[theme_name][&#x27;subthemes&#x27;].append({
&#x27;name&#x27;: subtheme_name,
&#x27;codes&#x27;: codes
})
def theme_prevalence(self, theme_name: str):
&quot;&quot;&quot;Calculate theme prevalence&quot;&quot;&quot;
theme = self.themes[theme_name]
segments = []
for code in theme[&#x27;codes&#x27;]:
segments.extend(self.coding.get_coded_segments(code))
# Get unique sources
sources = set(seg.source for seg in segments)
return {
&#x27;theme&#x27;: theme_name,
&#x27;n_segments&#x27;: len(segments),
&#x27;n_sources&#x27;: len(sources),
&#x27;percentage_sources&#x27;: (len(sources) /
len(set(s.source for s in self.coding.coded_segments))) * 100
}
def generate_theme_map(self):
&quot;&quot;&quot;Create visual theme hierarchy&quot;&quot;&quot;
map_str = &quot;# Thematic Map\n\n&quot;
for theme_name, theme in self.themes.items():
map_str += f&quot;## {theme_name}\n&quot;
map_str += f&quot;{theme[&#x27;description&#x27;]}\n\n&quot;
map_str += f&quot;**Codes**: {&#x27;, &#x27;.join(theme[&#x27;codes&#x27;])}\n\n&quot;
if theme[&#x27;subthemes&#x27;]:
map_str += &quot;**Subthemes**:\n&quot;
for subtheme in theme[&#x27;subthemes&#x27;]:
map_str += f&quot;- {subtheme[&#x27;name&#x27;]}: &quot;
map_str += f&quot;{&#x27;, &#x27;.join(subtheme[&#x27;codes&#x27;])}\n&quot;
prev = self.theme_prevalence(theme_name)
map_str += f&quot;\n*{prev[&#x27;n_sources&#x27;]} sources &quot;
map_str += f&quot;({prev[&#x27;percentage_sources&#x27;]:.0f}%), &quot;
map_str += f&quot;{prev[&#x27;n_segments&#x27;]} segments*\n\n&quot;
return map_str
# Example
thematic = ThematicDevelopment(coding)
thematic.create_theme(
&#x27;Negotiating Boundaries&#x27;,
codes=[&#x27;boundary_work&#x27;, &#x27;role_conflict&#x27;],
description=&#x27;The ongoing process of creating and maintaining work-life separation&#x27;
)
thematic.define_theme(
&#x27;Negotiating Boundaries&#x27;,
&#x27;Remote workers actively and continuously construct boundaries between work &#x27;
&#x27;and personal life through spatial, temporal, and communicative strategies, &#x27;
&#x27;while managing guilt and role conflicts that arise from boundary violations.&#x27;
)
print(thematic.generate_theme_map())</code></pre>
<h2>Data Visualization Best Practices</h2>
<pre><code class="lang-python">import matplotlib.pyplot as plt
import seaborn as sns
class VisualizationGuide:
&quot;&quot;&quot;Create publication-ready visualizations&quot;&quot;&quot;
@staticmethod
def set_publication_style():
&quot;&quot;&quot;Set consistent style for publication&quot;&quot;&quot;
sns.set_style(&#x27;whitegrid&#x27;)
sns.set_context(&#x27;paper&#x27;, font_scale=1.2)
plt.rcParams[&#x27;font.family&#x27;] = &#x27;sans-serif&#x27;
plt.rcParams[&#x27;font.sans-serif&#x27;] = [&#x27;Arial&#x27;]
plt.rcParams[&#x27;figure.dpi&#x27;] = 300
@staticmethod
def comparison_plot(groups: Dict[str, list], ylabel: str, title: str):
&quot;&quot;&quot;Create comparison plot with error bars&quot;&quot;&quot;
fig, ax = plt.subplots(figsize=(8, 6))
x_pos = range(len(groups))
means = [np.mean(data) for data in groups.values()]
sems = [stats.sem(data) for data in groups.values()]
ax.bar(x_pos, means, yerr=sems, capsize=5,
color=&#x27;steelblue&#x27;, alpha=0.8, edgecolor=&#x27;black&#x27;)
ax.set_xticks(x_pos)
ax.set_xticklabels(groups.keys())
ax.set_ylabel(ylabel)
ax.set_title(title)
ax.spines[&#x27;top&#x27;].set_visible(False)
ax.spines[&#x27;right&#x27;].set_visible(False)
return fig
@staticmethod
def scatter_with_regression(x, y, xlabel: str, ylabel: str):
&quot;&quot;&quot;Scatter plot with regression line&quot;&quot;&quot;
fig, ax = plt.subplots(figsize=(8, 6))
ax.scatter(x, y, alpha=0.6, s=50)
# Add regression line
z = np.polyfit(x, y, 1)
p = np.poly1d(z)
ax.plot(x, p(x), &#x27;r--&#x27;, linewidth=2)
# Add correlation
r, p_val = stats.pearsonr(x, y)
ax.text(0.05, 0.95, f&#x27;r = {r:.3f}, p = {p_val:.3f}&#x27;,
transform=ax.transAxes, verticalalignment=&#x27;top&#x27;,
bbox=dict(boxstyle=&#x27;round&#x27;, facecolor=&#x27;wheat&#x27;, alpha=0.5))
ax.set_xlabel(xlabel)
ax.set_ylabel(ylabel)
ax.spines[&#x27;top&#x27;].set_visible(False)
ax.spines[&#x27;right&#x27;].set_visible(False)
return fig</code></pre>
<h2>Best Practices</h2>
<h3>Quantitative Analysis</h3>
<ul><li>Check assumptions before tests</li><li>Report effect sizes, not just p-values</li><li>Provide confidence intervals</li><li>Correct for multiple comparisons</li><li>Document all analysis decisions</li><li>Report negative results</li><li>Make data and code available</li></ul>
<h3>Qualitative Analysis</h3>
<ul><li>Code systematically with clear definitions</li><li>Calculate inter-rater reliability</li><li>Write analytic memos throughout</li><li>Seek negative cases</li><li>Use software for transparency</li><li>Provide thick description</li><li>Maintain audit trail</li></ul>
<h3>Visualization</h3>
<ul><li>Choose appropriate chart type</li><li>Label axes clearly</li><li>Include error bars or confidence intervals</li><li>Use colorblind-friendly palettes</li><li>Remove chart junk</li><li>Export at high resolution</li><li>Caption thoroughly</li></ul>
<h2>Related Skills</h2>
<ul><li><strong>quantitative-methods</strong>: Quantitative research approaches</li><li><strong>qualitative-methods</strong>: Qualitative research approaches</li><li><strong>research-synthesis</strong>: Analyzing across studies</li><li><strong>data-collection</strong>: Obtaining analyzable data</li><li><strong>research-writing</strong>: Reporting analyses</li></ul>
<h2>Quick Reference</h2>
<h3>Test Selection Guide</h3>
<pre><code>Compare 2 groups: Independent t-test
Compare 2 timepoints: Paired t-test
Compare 3+ groups: ANOVA
Relationship: Correlation
Prediction: Regression
Categorical association: Chi-square
Non-normal data: Non-parametric alternatives</code></pre>
<h3>Effect Size Interpretation</h3>
<pre><code>Cohen&#x27;s d: 0.2 small, 0.5 medium, 0.8 large
Correlation: 0.1 small, 0.3 medium, 0.5 large
Eta-squared: 0.01 small, 0.06 medium, 0.14 large</code></pre>
<h3>Coding Reliability Standards</h3>
<pre><code>Cohen&#x27;s Kappa:
&lt; 0.20: Poor
0.21-0.40: Fair
0.41-0.60: Moderate
0.61-0.80: Substantial
0.81-1.00: Almost perfect</code></pre>
<footer>
<div class="fmap">
<div><b>Build</b>
<a href="https://hanzo.app">hanzo.app</a> app builder
<a href="https://hanzo.codes">hanzo.codes</a> code with an agent
<a href="https://hanzoskills.com">hanzoskills.com</a> what an agent reads</div>
<div><b>Run</b>
<a href="https://console.hanzo.ai">console.hanzo.ai</a> cloud console
<a href="https://api.hanzo.ai">api.hanzo.ai</a> the API</div>
<div><b>Account</b>
<a href="https://hanzo.id">hanzo.id</a> sign in
<a href="https://pay.hanzo.ai">pay.hanzo.ai</a> top up
<a href="https://billing.hanzo.ai">billing.hanzo.ai</a> invoices and usage</div>
<div><b>Company</b>
<a href="https://hanzo.ai">hanzo.ai</a> canonical
<a href="https://hanzo.ventures">hanzo.ventures</a> ventures</div>
</div>
<div class="fend"><a href="/research-data-analysis.md">This page as plain text</a> · <a href="/llms.txt">Index for agents</a></div>
</footer>
</div></body></html>