Files
skills/public/research-research-design.html
zeekayandHanzo Dev 9e10c1c10e Each llms index points at the other two
hanzoskills.com/llms.txt now opens with the install line, the API
reference index and the company index, so an agent that lands on any
one of the three can find the rest.

robots.txt claimed 537 documents while llms.txt counts 920. The number
is gone from robots.txt; llms.txt derives it.

public/ is regenerated from scripts/site.py, which it had fallen
behind: the copy-prompt menu and agent bar were in the generator and
not in the committed pages. tests/site.mjs passes against the result.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-11 00:27:59 -07:00

734 lines
34 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!doctype html>
<html lang="en"><head>
<meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
<title>research-research-design</title>
<meta name="color-scheme" content="dark">
<style>
/* Committed to one visual world: a dark instrument panel. No light variant —
the dot field and the hairline rules only read on black, and a naive
inversion would wash both out. Every colour is painted explicitly so the page
never borrows a host background. */
@font-face{font-family:'Zen';src:url('/fonts/Zen-Variable.woff2') format('woff2');font-weight:100 900;font-style:normal;font-display:swap}
@font-face{font-family:'Zen';src:url('/fonts/Zen-Italic-Variable.woff2') format('woff2');font-weight:100 900;font-style:italic;font-display:swap}
@font-face{font-family:'Zen Mono';src:url('/fonts/ZenMono-Variable.woff2') format('woff2');font-weight:100 900;font-style:normal;font-display:swap}
@font-face{font-family:'Zen Mono';src:url('/fonts/ZenMono-Italic-Variable.woff2') format('woff2');font-weight:100 900;font-style:italic;font-display:swap}
@font-face{font-family:'Zen Pixel Square';src:url('/fonts/ZenPixel-Square.woff2') format('woff2');font-style:normal;font-display:swap}
:root{
--ink:#f4f4f2; --soft:#9c9c98; --faint:#6a6a66; --ground:#000; --panel:#0a0a0a;
--line:#232323; --line-hi:#3a3a3a; --accent:#f4f4f2;
--font-zen-sans:'Zen',-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif;
--font-zen-mono:'Zen Mono',ui-monospace,"SF Mono",SFMono-Regular,Menlo,Consolas,monospace;
--font-zen-pixel-square:'Zen Pixel Square',monospace;
--mono:var(--font-zen-mono);
--sans:var(--font-zen-sans);
/* Controls are pills and surfaces are soft, as on hanzo.ai. */
--r:999px; --r-soft:14px;
}
*{box-sizing:border-box}
html{-webkit-text-size-adjust:100%}
body{margin:0;background:var(--ground);color:var(--ink);font-family:var(--sans);
font-size:16px;line-height:1.65;-webkit-font-smoothing:antialiased}
.wrap{max-width:60rem;margin:0 auto;padding:0 1.75rem}
/* The dot field. A halftone made of one repeating radial-gradient, faded out
along the diagonal by a mask so it reads as a wave rather than wallpaper. */
/* No overflow clip here: the dot field is inset to the box already, and a clip
would cut off the copy-prompt menu. */
.hero{position:relative;border-bottom:1px solid var(--line)}
.hero::before{content:"";position:absolute;inset:0;
background-image:radial-gradient(var(--line-hi) 1px, transparent 1px);
background-size:7px 7px;
-webkit-mask-image:linear-gradient(115deg,#000 0%,rgba(0,0,0,.55) 38%,transparent 72%);
mask-image:linear-gradient(115deg,#000 0%,rgba(0,0,0,.55) 38%,transparent 72%);
pointer-events:none}
.hero .wrap{position:relative;padding-top:5.5rem;padding-bottom:4rem}
nav{border-bottom:1px solid var(--line)}
nav .in{max-width:60rem;margin:0 auto;padding:1.1rem 1.75rem;display:flex;
justify-content:space-between;align-items:center;gap:1rem}
.brand{font-family:var(--mono);font-size:.85rem;letter-spacing:.14em;
text-transform:uppercase;color:var(--ink);text-decoration:none;
display:inline-flex;align-items:center;gap:.6rem}
.brand i{font-style:normal;color:var(--faint)}
.mark{width:1.15em;height:1.15em;display:block;flex:0 0 auto}
nav a.up{font-family:var(--mono);font-size:.72rem;letter-spacing:.14em;
text-transform:uppercase;color:var(--soft);text-decoration:none}
nav a.up:hover{color:var(--ink)}
h1{font-family:var(--mono);font-size:clamp(2rem,5.4vw,3.4rem);line-height:1.12;
letter-spacing:-.02em;margin:0 0 1.1rem;font-weight:500;text-wrap:balance}
h1 em{font-style:normal;color:var(--faint)}
.lede{color:var(--soft);font-size:1.05rem;max-width:52ch;margin:0 0 2.25rem}
.btns{display:flex;flex-wrap:wrap;gap:.7rem;margin:0 0 .5rem;align-items:center}
.btn{display:inline-flex;align-items:center;gap:.5rem;font-family:var(--mono);
font-size:.72rem;letter-spacing:.14em;text-transform:uppercase;
padding:.8rem 1.4rem;border:1px solid var(--line-hi);border-radius:var(--r);
color:var(--ink);text-decoration:none;background:transparent;cursor:pointer;
line-height:1.2;font-weight:400}
.btn:hover{background:#141414;text-decoration:none;border-color:var(--soft)}
.btn.solid{background:var(--ink);color:#000;border-color:var(--ink)}
.btn.solid:hover{background:#d8d8d4}
.btn .chev{width:.7em;height:.7em;flex:0 0 auto;transition:transform .15s ease}
.btn[aria-expanded="true"] .chev{transform:rotate(180deg)}
/* Copy-prompt selector. Each row copies the prompt for its own target. */
.copy{position:relative;display:inline-flex}
.menu{position:absolute;top:calc(100% + .5rem);left:0;z-index:20;min-width:21rem;
background:var(--panel);border:1px solid var(--line-hi);border-radius:var(--r-soft);
padding:.4rem;display:flex;flex-direction:column;
box-shadow:0 12px 32px rgba(0,0,0,.75)}
.menu[hidden]{display:none}
.mi{display:flex;align-items:center;gap:.7rem;width:100%;text-align:left;
padding:.62rem .8rem;border:0;border-radius:calc(var(--r-soft) - .35rem);
background:transparent;color:var(--ink);font-family:var(--mono);font-size:.76rem;
letter-spacing:.04em;text-decoration:none;cursor:pointer;line-height:1.3;
white-space:nowrap}
.mi:hover,.mi:focus-visible{background:#181818;outline:none}
.mi .mark{width:1.05rem;height:1.05rem;color:var(--soft)}
.mi:hover .mark{color:var(--ink)}
.mi u{margin-left:auto;color:var(--faint);text-decoration:none;font-size:.68rem;
letter-spacing:.1em;text-transform:uppercase}
/* Works-with bar: one scrolling row, name and mark per agent. */
.strip{border-bottom:1px solid var(--line);background:var(--panel)}
.strip .in{max-width:60rem;margin:0 auto;padding:.9rem 1.75rem;display:flex;
align-items:center;gap:1.25rem;font-family:var(--mono);font-size:.72rem;
letter-spacing:.1em;text-transform:uppercase;color:var(--faint)}
.strip b{color:var(--soft);font-weight:400;flex:0 0 auto;
border-right:1px solid var(--line);padding-right:1.25rem;align-self:stretch;
display:flex;align-items:center}
.bar{display:flex;align-items:center;gap:.5rem;flex:1 1 auto;min-width:0;
overflow-x:auto;scrollbar-width:none;scroll-snap-type:x proximity;
-webkit-overflow-scrolling:touch;
-webkit-mask-image:linear-gradient(90deg,#000 calc(100% - 3rem),transparent);
mask-image:linear-gradient(90deg,#000 calc(100% - 3rem),transparent)}
.bar::-webkit-scrollbar{display:none}
.agent{display:inline-flex;align-items:center;gap:.45rem;flex:0 0 auto;
padding:.4rem .85rem;border:1px solid var(--line);border-radius:var(--r);
color:var(--soft);white-space:nowrap;scroll-snap-align:start;
text-decoration:none;background:var(--ground)}
.agent:hover{color:var(--ink);border-color:var(--line-hi);text-decoration:none}
.agent .mark{width:.95rem;height:.95rem}
h2{font-family:var(--mono);font-size:1.1rem;letter-spacing:-.01em;margin:2.75rem 0 .75rem;font-weight:500}
h3{font-size:1rem;margin:2rem 0 .5rem;font-weight:600}
h4,h5,h6{font-size:.94rem;margin:1.5rem 0 .5rem;font-weight:600}
p{margin:0 0 1rem;max-width:70ch}
ul,ol{margin:0 0 1rem;padding-left:1.35rem;max-width:70ch}
li{margin:.22rem 0}
a{color:var(--ink);text-decoration:underline;text-underline-offset:3px;
text-decoration-color:var(--line-hi)}
a:hover{text-decoration-color:var(--ink)}
a:focus-visible{outline:1px solid var(--ink);outline-offset:3px}
code{font-family:var(--mono);font-size:.85em;background:var(--panel);
border:1px solid var(--line);border-radius:5px;padding:.1em .36em}
pre{background:var(--panel);border:1px solid var(--line);padding:1rem 1.1rem;
border-radius:var(--r-soft);overflow-x:auto;margin:0 0 1.25rem}
pre code{background:none;border:0;padding:0;font-size:.82rem;line-height:1.7}
blockquote{margin:0 0 1rem;padding-left:1rem;border-left:1px solid var(--line-hi);color:var(--soft)}
hr{border:0;border-top:1px solid var(--line);margin:2.5rem 0}
table{border-collapse:collapse;width:100%;margin:0 0 1.25rem;font-size:.92rem;
display:block;overflow-x:auto}
th,td{border:1px solid var(--line);padding:.5rem .7rem;text-align:left}
.curl{display:block;background:var(--panel);border:1px solid var(--line);
border-radius:var(--r-soft);padding:.95rem 1.1rem;font-family:var(--mono);
font-size:.8rem;color:var(--soft);overflow-x:auto;margin:0 0 2.5rem;
white-space:pre;line-height:1.9}
.curl b{color:var(--ink);font-weight:400}
.curl i{font-style:normal;color:var(--faint)}
/* index */
#q{width:100%;padding:.85rem 1.2rem;font-family:var(--mono);font-size:.85rem;color:var(--ink);
background:var(--panel);border:1px solid var(--line);border-radius:var(--r);margin:0 0 .6rem}
#q::placeholder{color:var(--faint);text-transform:uppercase;letter-spacing:.1em;font-size:.72rem}
#q:focus{outline:none;border-color:var(--soft)}
#count{color:var(--faint);font-family:var(--mono);font-size:.72rem;letter-spacing:.1em;
text-transform:uppercase;margin:0 0 2rem}
.cat{margin:2.5rem 0 .5rem;font-family:var(--mono);font-size:.7rem;letter-spacing:.16em;
text-transform:uppercase;color:var(--faint);border-bottom:1px solid var(--line);
padding-bottom:.45rem}
.row{display:block;padding:.6rem .9rem;margin:0 -.9rem;border-bottom:1px solid var(--line);
border-radius:var(--r-soft);text-decoration:none}
.row:hover{background:#0a0a0a}
.nm{font-family:var(--mono);font-size:.84rem;color:var(--ink);display:block}
.ds{color:var(--soft);font-size:.9rem;display:block;margin-top:.12rem}
footer{margin-top:4rem;padding:2rem 0 4rem;border-top:1px solid var(--line);
color:var(--faint);font-size:.85rem}
footer a{color:var(--soft)}
.fmap{display:grid;gap:1.75rem;grid-template-columns:repeat(auto-fit,minmax(11rem,1fr));
margin:0 0 2rem;font-size:.85rem}
.fmap b{display:block;font-family:var(--mono);font-size:.68rem;letter-spacing:.16em;
text-transform:uppercase;color:var(--faint);font-weight:400;margin:0 0 .6rem;
border-bottom:1px solid var(--line);padding-bottom:.4rem}
.fmap a{display:inline-block;color:var(--ink);text-decoration:none;margin:.3rem .35rem 0 0}
.fmap a:hover{text-decoration:underline}
.fmap div{color:var(--faint);line-height:1.5}
.fend{border-top:1px solid var(--line);padding-top:1.25rem;font-size:.82rem}
@media (max-width:640px){
.hero .wrap{padding-top:3.25rem;padding-bottom:2.75rem}
.btns{gap:.5rem}
.btn{padding:.75rem 1.15rem;font-size:.68rem}
.copy,.copy .btn{width:100%;justify-content:center}
.menu{left:0;right:0;min-width:0}
.strip .in{flex-direction:column;align-items:flex-start;gap:.6rem;padding:.9rem 1.75rem}
.strip b{border-right:0;padding-right:0}
.bar{width:100%}
}
</style>
<link rel="alternate" type="text/markdown" href="/research-research-design.md">
<link rel="icon" href="/favicon.svg">
<link rel="canonical" href="https://hanzoskills.com/research-research-design">
</head>
<!-- Reading this with a machine? Every page here is also plain markdown.
This one: https://hanzoskills.com/research-research-design.md
Index: https://hanzoskills.com/llms.txt
Corpus: https://hanzoskills.com/all.md
Append .md to any path for the markdown twin. -->
<body>
<nav><div class="in">
<a class="brand" href="/"><svg class="mark" viewBox="0 0 67 67" fill="currentColor" aria-hidden="true"><path d="M22.21 67V44.6369H0V67H22.21Z"/><path d="M66.7038 22.3184H22.2534L0.0878906 44.6367H44.4634L66.7038 22.3184Z"/><path d="M22.21 0H0V22.3184H22.21V0Z"/><path d="M66.7198 0H44.5098V22.3184H66.7198V0Z"/><path d="M66.7198 67V44.6369H44.5098V67H66.7198Z"/></svg> HANZO<i>SKILLS</i></a>
<a class="up" href="/">All skills</a>
</div></nav>
<div class="wrap">
<h1>research-research-design</h1><p class="lede">Master research study design including hypothesis formation, validity, sampling strategies, and experimental control</p><h1>Research Design Skill</h1>
<h2>When to Use This Skill</h2>
<p>Use this skill when you need to:</p>
<ul><li>Plan a research study from scratch</li><li>Choose appropriate research methodology</li><li>Formulate testable hypotheses</li><li>Design sampling strategies</li><li>Ensure validity and reliability</li><li>Control for confounding variables</li><li>Balance internal and external validity</li><li>Navigate ethical considerations</li></ul>
<h2>Core Design Elements</h2>
<h3>1. Research Questions and Hypotheses</h3>
<p><strong>FINER Criteria for Research Questions</strong>:</p>
<pre><code>F - Feasible: Can you actually do this?
I - Interesting: Does it matter?
N - Novel: Is it new?
E - Ethical: Is it responsible?
R - Relevant: Will it impact the field?</code></pre>
<p><strong>Question to Hypothesis Framework</strong>:</p>
<pre><code class="lang-python">from dataclasses import dataclass
from typing import List, Optional
from enum import Enum
class HypothesisType(Enum):
DIRECTIONAL = &quot;directional&quot;
NON_DIRECTIONAL = &quot;non-directional&quot;
NULL = &quot;null&quot;
@dataclass
class ResearchQuestion:
&quot;&quot;&quot;Structure a research question&quot;&quot;&quot;
question: str
population: str
variables: List[str]
relationship_type: str # &#x27;difference&#x27;, &#x27;relationship&#x27;, &#x27;prediction&#x27;
def to_hypothesis(self, hypothesis_type: HypothesisType,
expected_direction: Optional[str] = None):
&quot;&quot;&quot;Convert research question to testable hypothesis&quot;&quot;&quot;
if hypothesis_type == HypothesisType.NULL:
return f&quot;There is no {self.relationship_type} between &quot; \
f&quot;{&#x27; and &#x27;.join(self.variables)} in {self.population}.&quot;
elif hypothesis_type == HypothesisType.DIRECTIONAL:
if not expected_direction:
raise ValueError(&quot;Directional hypothesis requires expected_direction&quot;)
return f&quot;There is a {expected_direction} {self.relationship_type} &quot; \
f&quot;between {&#x27; and &#x27;.join(self.variables)} in {self.population}.&quot;
else: # NON_DIRECTIONAL
return f&quot;There is a {self.relationship_type} between &quot; \
f&quot;{&#x27; and &#x27;.join(self.variables)} in {self.population}.&quot;
# Example
rq = ResearchQuestion(
question=&quot;Does exercise frequency affect anxiety levels in college students?&quot;,
population=&quot;college students&quot;,
variables=[&quot;exercise frequency&quot;, &quot;anxiety levels&quot;],
relationship_type=&quot;relationship&quot;
)
print(&quot;Null:&quot;, rq.to_hypothesis(HypothesisType.NULL))
print(&quot;Directional:&quot;, rq.to_hypothesis(HypothesisType.DIRECTIONAL,
&quot;negative&quot;))
print(&quot;Non-directional:&quot;, rq.to_hypothesis(HypothesisType.NON_DIRECTIONAL))</code></pre>
<h3>2. Validity Considerations</h3>
<p><strong>Validity Framework</strong>:</p>
<pre><code class="lang-python">from typing import List, Dict
from enum import Enum
class ValidityThreat(Enum):
# Internal validity
HISTORY = &quot;history&quot;
MATURATION = &quot;maturation&quot;
TESTING = &quot;testing&quot;
INSTRUMENTATION = &quot;instrumentation&quot;
REGRESSION = &quot;regression_to_mean&quot;
SELECTION = &quot;selection_bias&quot;
ATTRITION = &quot;attrition&quot;
# External validity
INTERACTION_SELECTION = &quot;selection_treatment_interaction&quot;
SETTING = &quot;setting_effects&quot;
HISTORY_TREATMENT = &quot;history_treatment_interaction&quot;
# Construct validity
HYPOTHESIS_GUESSING = &quot;hypothesis_guessing&quot;
EVALUATION_APPREHENSION = &quot;evaluation_apprehension&quot;
EXPERIMENTER_EFFECTS = &quot;experimenter_effects&quot;
MONO_OPERATION = &quot;mono_operation_bias&quot;
MONO_METHOD = &quot;mono_method_bias&quot;
class ValidityAnalysis:
&quot;&quot;&quot;Analyze validity threats and controls&quot;&quot;&quot;
def __init__(self, study_design: str):
self.design = study_design
self.threats = []
self.controls = {}
def add_threat(self, threat: ValidityThreat, description: str,
severity: str):
&quot;&quot;&quot;Identify potential validity threat&quot;&quot;&quot;
self.threats.append({
&#x27;threat&#x27;: threat.value,
&#x27;description&#x27;: description,
&#x27;severity&#x27;: severity # &#x27;low&#x27;, &#x27;medium&#x27;, &#x27;high&#x27;
})
def add_control(self, threat: ValidityThreat, control: str):
&quot;&quot;&quot;Document how threat is controlled&quot;&quot;&quot;
self.controls[threat.value] = control
def generate_validity_table(self):
&quot;&quot;&quot;Create validity threat and control matrix&quot;&quot;&quot;
import pandas as pd
data = []
for threat_info in self.threats:
threat_name = threat_info[&#x27;threat&#x27;]
data.append({
&#x27;Threat&#x27;: threat_name,
&#x27;Description&#x27;: threat_info[&#x27;description&#x27;],
&#x27;Severity&#x27;: threat_info[&#x27;severity&#x27;],
&#x27;Control&#x27;: self.controls.get(threat_name, &#x27;None specified&#x27;)
})
return pd.DataFrame(data)
# Example
validity = ValidityAnalysis(&quot;Pre-post intervention with control group&quot;)
validity.add_threat(
ValidityThreat.HISTORY,
&quot;External events during study period may affect outcomes&quot;,
severity=&quot;medium&quot;
)
validity.add_control(
ValidityThreat.HISTORY,
&quot;Control group experiencing same time period&quot;
)
validity.add_threat(
ValidityThreat.ATTRITION,
&quot;Differential dropout between treatment and control&quot;,
severity=&quot;high&quot;
)
validity.add_control(
ValidityThreat.ATTRITION,
&quot;Intent-to-treat analysis; track and report attrition rates; &quot;
&quot;compare completers vs dropouts on baseline characteristics&quot;
)
print(validity.generate_validity_table())</code></pre>
<h3>3. Sampling Strategies</h3>
<p><strong>Sampling Design Framework</strong>:</p>
<pre><code class="lang-python">from enum import Enum
import numpy as np
from typing import Optional
class SamplingMethod(Enum):
# Probability sampling
SIMPLE_RANDOM = &quot;simple_random&quot;
SYSTEMATIC = &quot;systematic&quot;
STRATIFIED = &quot;stratified&quot;
CLUSTER = &quot;cluster&quot;
MULTISTAGE = &quot;multistage&quot;
# Non-probability sampling
CONVENIENCE = &quot;convenience&quot;
PURPOSIVE = &quot;purposive&quot;
QUOTA = &quot;quota&quot;
SNOWBALL = &quot;snowball&quot;
class SamplingDesign:
&quot;&quot;&quot;Design and document sampling strategy&quot;&quot;&quot;
def __init__(self, method: SamplingMethod, population_size: int,
target_sample_size: int):
self.method = method
self.N = population_size
self.n = target_sample_size
self.sampling_frame = None
self.strata = None
def simple_random_sample(self, population_ids: list, seed: int = 42):
&quot;&quot;&quot;Draw simple random sample&quot;&quot;&quot;
np.random.seed(seed)
return np.random.choice(population_ids, size=self.n, replace=False)
def stratified_sample(self, strata_dict: dict, proportional: bool = True):
&quot;&quot;&quot;Draw stratified sample
Args:
strata_dict: {stratum_name: [ids in stratum]}
proportional: If True, proportional allocation; else equal
&quot;&quot;&quot;
samples = []
if proportional:
# Proportional to stratum size
for stratum_name, stratum_ids in strata_dict.items():
stratum_n = int(self.n * len(stratum_ids) / self.N)
stratum_sample = np.random.choice(stratum_ids,
size=stratum_n,
replace=False)
samples.extend(stratum_sample)
else:
# Equal allocation
n_per_stratum = self.n // len(strata_dict)
for stratum_ids in strata_dict.values():
stratum_sample = np.random.choice(stratum_ids,
size=n_per_stratum,
replace=False)
samples.extend(stratum_sample)
return samples
def calculate_sampling_error(self, std_dev: float,
finite_correction: bool = True):
&quot;&quot;&quot;Calculate standard error of the mean&quot;&quot;&quot;
if finite_correction and self.N &gt; 0:
fpc = np.sqrt((self.N - self.n) / (self.N - 1))
se = (std_dev / np.sqrt(self.n)) * fpc
else:
se = std_dev / np.sqrt(self.n)
return se
def required_sample_size(self, std_dev: float, margin_error: float,
confidence_level: float = 0.95,
finite_correction: bool = True):
&quot;&quot;&quot;Calculate required sample size for desired precision&quot;&quot;&quot;
from scipy import stats
# Z-score for confidence level
z = stats.norm.ppf((1 + confidence_level) / 2)
# Required n without finite population correction
n_0 = (z * std_dev / margin_error) ** 2
if finite_correction and self.N &gt; 0:
# Adjust for finite population
n = n_0 / (1 + (n_0 - 1) / self.N)
else:
n = n_0
return int(np.ceil(n))
def document_sampling(self):
&quot;&quot;&quot;Generate sampling documentation&quot;&quot;&quot;
doc = f&quot;&quot;&quot;
# Sampling Design Documentation
## Method
{self.method.value}
## Population and Sample
- Population size (N): {self.N if self.N else &#x27;Unknown&#x27;}
- Target sample size (n): {self.n}
- Sampling fraction: {(self.n/self.N)*100:.1f}% (if N known)
## Sampling Procedure
[Describe step-by-step how sampling was conducted]
## Rationale
[Justify why this method is appropriate for research question]
## Limitations
[Discuss potential sampling biases and limitations]
&quot;&quot;&quot;
return doc
# Example
design = SamplingDesign(
method=SamplingMethod.STRATIFIED,
population_size=10000,
target_sample_size=400
)
# Calculate required n for desired precision
required_n = design.required_sample_size(
std_dev=15,
margin_error=2,
confidence_level=0.95
)
print(f&quot;Required sample size: {required_n}&quot;)
# Draw stratified sample
strata = {
&#x27;stratum_A&#x27;: list(range(0, 5000)),
&#x27;stratum_B&#x27;: list(range(5000, 10000))
}
sample = design.stratified_sample(strata, proportional=True)
print(f&quot;Sample drawn: {len(sample)} participants&quot;)</code></pre>
<h3>4. Experimental Control</h3>
<p><strong>Control Strategies</strong>:</p>
<pre><code class="lang-python">from dataclasses import dataclass
from typing import List, Dict, Optional
from enum import Enum
class ControlMethod(Enum):
RANDOMIZATION = &quot;randomization&quot;
MATCHING = &quot;matching&quot;
BLOCKING = &quot;blocking&quot;
STATISTICAL_CONTROL = &quot;statistical_control&quot;
STANDARDIZATION = &quot;standardization&quot;
@dataclass
class ConfoundingVariable:
&quot;&quot;&quot;Define potential confounding variable&quot;&quot;&quot;
name: str
relationship_to_iv: str
relationship_to_dv: str
control_method: ControlMethod
control_procedure: str
class ExperimentalControl:
&quot;&quot;&quot;Design experimental controls&quot;&quot;&quot;
def __init__(self):
self.confounds = {}
self.design_features = []
def identify_confound(self, confound: ConfoundingVariable):
&quot;&quot;&quot;Add confounding variable with control plan&quot;&quot;&quot;
self.confounds[confound.name] = confound
def add_design_feature(self, feature: str, purpose: str):
&quot;&quot;&quot;Document design feature for control&quot;&quot;&quot;
self.design_features.append({
&#x27;feature&#x27;: feature,
&#x27;purpose&#x27;: purpose
})
def random_assignment(self, participants: list,
n_groups: int, seed: int = 42):
&quot;&quot;&quot;Randomly assign participants to groups&quot;&quot;&quot;
np.random.seed(seed)
shuffled = np.random.permutation(participants)
groups = np.array_split(shuffled, n_groups)
return [list(g) for g in groups]
def matched_assignment(self, participants_df,
matching_vars: List[str],
n_groups: int):
&quot;&quot;&quot;Create matched groups based on variables&quot;&quot;&quot;
import pandas as pd
from sklearn.preprocessing import StandardScaler
# Standardize matching variables
scaler = StandardScaler()
X = scaler.fit_transform(participants_df[matching_vars])
# Cluster into n_groups using K-means
from sklearn.cluster import KMeans
kmeans = KMeans(n_clusters=n_groups, random_state=42)
participants_df[&#x27;match_group&#x27;] = kmeans.fit_predict(X)
# Assign one from each cluster to each condition
groups = [[] for _ in range(n_groups)]
for cluster in range(n_groups):
cluster_members = participants_df[
participants_df[&#x27;match_group&#x27;] == cluster
].index.tolist()
np.random.shuffle(cluster_members)
for i, member in enumerate(cluster_members):
groups[i % n_groups].append(member)
return groups
def generate_control_plan(self):
&quot;&quot;&quot;Create documented control plan&quot;&quot;&quot;
plan = &quot;# Experimental Control Plan\n\n&quot;
plan += &quot;## Identified Confounding Variables\n\n&quot;
for name, confound in self.confounds.items():
plan += f&quot;### {name}\n&quot;
plan += f&quot;- **Relationship to IV**: {confound.relationship_to_iv}\n&quot;
plan += f&quot;- **Relationship to DV**: {confound.relationship_to_dv}\n&quot;
plan += f&quot;- **Control Method**: {confound.control_method.value}\n&quot;
plan += f&quot;- **Procedure**: {confound.control_procedure}\n\n&quot;
plan += &quot;## Design Features for Control\n\n&quot;
for feature in self.design_features:
plan += f&quot;- **{feature[&#x27;feature&#x27;]}**: {feature[&#x27;purpose&#x27;]}\n&quot;
return plan
# Example
control = ExperimentalControl()
# Identify confounds
control.identify_confound(ConfoundingVariable(
name=&#x27;Prior Experience&#x27;,
relationship_to_iv=&#x27;More experienced participants may seek treatment&#x27;,
relationship_to_dv=&#x27;Experience directly affects performance&#x27;,
control_method=ControlMethod.RANDOMIZATION,
control_procedure=&#x27;Random assignment to conditions distributes experience equally&#x27;
))
control.identify_confound(ConfoundingVariable(
name=&#x27;Time of Day&#x27;,
relationship_to_iv=&#x27;Treatment sessions at different times&#x27;,
relationship_to_dv=&#x27;Cognitive performance varies by time&#x27;,
control_method=ControlMethod.STANDARDIZATION,
control_procedure=&#x27;All sessions conducted between 9-11am&#x27;
))
# Add design features
control.add_design_feature(
&#x27;Double-blind procedure&#x27;,
&#x27;Prevent experimenter bias and demand characteristics&#x27;
)
control.add_design_feature(
&#x27;Standardized protocols&#x27;,
&#x27;Ensure consistent treatment delivery&#x27;
)
print(control.generate_control_plan())</code></pre>
<h2>Design Patterns</h2>
<h3>Strong Research Design</h3>
<pre><code>✓ Clear, testable hypotheses
✓ Appropriate methodology for question
✓ Threats to validity identified and controlled
✓ Adequate sample size (power analysis)
✓ Random sampling or assignment when possible
✓ Multiple measures/methods (triangulation)
✓ Pilot testing conducted
✓ Pre-registration of hypotheses and methods</code></pre>
<h3>Weak Research Design</h3>
<pre><code>✗ Vague or non-testable hypotheses
✗ Method-question mismatch
✗ Uncontrolled confounds
✗ Convenience sample assumed representative
✗ Underpowered study
✗ Single method/measure
✗ Post-hoc hypothesizing (HARKing)
✗ P-hacking through multiple analyses</code></pre>
<h2>Research Design Types</h2>
<h3>1. Experimental Designs</h3>
<p><strong>True Experiment</strong>:</p>
<pre><code>- Random assignment to conditions
- Manipulation of IV
- Control group
- Maximum internal validity</code></pre>
<p><strong>Quasi-Experiment</strong>:</p>
<pre><code>- No random assignment
- Manipulation of IV or natural variation
- Comparison group
- Moderate internal validity</code></pre>
<p><strong>Single-Case Design</strong>:</p>
<pre><code>- Individual as own control
- Repeated measures over time
- Experimental control through replication
- Good for clinical intervention research</code></pre>
<h3>2. Non-Experimental Designs</h3>
<p><strong>Correlational</strong>:</p>
<pre><code>- Examine relationships between variables
- No manipulation
- Cannot infer causation
- Useful for prediction</code></pre>
<p><strong>Survey</strong>:</p>
<pre><code>- Describe population characteristics
- No manipulation
- Generalization to population
- Good for attitudes, beliefs, behaviors</code></pre>
<p><strong>Observational</strong>:</p>
<pre><code>- Observe naturally occurring behavior
- No manipulation
- High ecological validity
- Good for exploratory research</code></pre>
<h2>Best Practices</h2>
<h3>1. Planning Phase</h3>
<ul><li>Start with clear research question</li><li>Review literature thoroughly</li><li>Choose design matching question and resources</li><li>Conduct power analysis</li><li>Create detailed protocol</li><li>Pre-register study</li><li>Get IRB approval</li></ul>
<h3>2. Design Phase</h3>
<ul><li>Map causal model (DAG)</li><li>Identify all confounds</li><li>Choose appropriate controls</li><li>Balance internal and external validity</li><li>Plan for attrition</li><li>Build in manipulation checks</li><li>Design pilot study</li></ul>
<h3>3. Ethical Considerations</h3>
<ul><li>Assess risk-benefit ratio</li><li>Ensure informed consent</li><li>Protect participant privacy</li><li>Plan for adverse events</li><li>Consider vulnerable populations</li><li>Ensure equitable participant selection</li><li>Plan data security</li></ul>
<h3>4. Documentation</h3>
<ul><li>Write detailed protocol</li><li>Create decision tree for procedures</li><li>Document all changes from protocol</li><li>Maintain audit trail</li><li>Archive all materials</li><li>Enable reproducibility</li></ul>
<h2>Common Design Mistakes</h2>
<ol><li><strong>Confusing Correlation and Causation</strong></li></ol>
<ul><li>Problem: Inferring causation from correlational design</li><li>Solution: Use causal language only with experimental designs</li></ul>
<ol><li><strong>Insufficient Power</strong></li></ol>
<ul><li>Problem: Sample too small to detect real effects</li><li>Solution: Conduct a priori power analysis</li></ul>
<ol><li><strong>Unmeasured Confounds</strong></li></ol>
<ul><li>Problem: Alternative explanations not ruled out</li><li>Solution: Create comprehensive causal diagram</li></ul>
<ol><li><strong>Convenience Sampling Generalization</strong></li></ol>
<ul><li>Problem: Assuming convenience sample represents population</li><li>Solution: Acknowledge limitations; use probability sampling</li></ul>
<ol><li><strong>Demand Characteristics</strong></li></ol>
<ul><li>Problem: Participants guess hypothesis and act accordingly</li><li>Solution: Use blind procedures; cover story; implicit measures</li></ul>
<ol><li><strong>Experimenter Bias</strong></li></ol>
<ul><li>Problem: Researcher expectations influence results</li><li>Solution: Double-blind design; automated procedures</li></ul>
<h2>Related Skills</h2>
<ul><li><strong>quantitative-methods</strong>: Statistical analysis for designed studies</li><li><strong>qualitative-methods</strong>: Alternative research approaches</li><li><strong>data-collection</strong>: Implementing research protocols</li><li><strong>research-synthesis</strong>: Learning from existing research</li><li><strong>research-writing</strong>: Documenting research design</li></ul>
<h2>Quick Reference</h2>
<h3>Design Selection Matrix</h3>
<pre><code>Question Type → Design
-------------------------------------
Causation → True experiment
Association → Correlational
Prevalence → Cross-sectional survey
Change over time → Longitudinal
Lived experience → Phenomenology
Process/meaning → Grounded theory
Bounded system → Case study</code></pre>
<h3>Internal Validity Hierarchy</h3>
<pre><code>Highest: Randomized controlled trial
Quasi-experiment with matching
Pre-post with control
Post-only with control
Lowest: One-group pre-post
Cross-sectional correlation</code></pre>
<h3>Sample Size Quick Rules</h3>
<pre><code>Simple comparison: 50-100 per group
Multiple regression: 104 + k (k=predictors)
Factor analysis: 300+ or 10× variables
Structural equation: 200+ minimum
Qualitative: Until saturation (varies)</code></pre>
<footer>
<div class="fmap">
<div><b>Build</b>
<a href="https://hanzo.app">hanzo.app</a> app builder
<a href="https://hanzo.codes">hanzo.codes</a> code with an agent
<a href="https://hanzoskills.com">hanzoskills.com</a> what an agent reads</div>
<div><b>Run</b>
<a href="https://console.hanzo.ai">console.hanzo.ai</a> cloud console
<a href="https://api.hanzo.ai">api.hanzo.ai</a> the API</div>
<div><b>Account</b>
<a href="https://hanzo.id">hanzo.id</a> sign in
<a href="https://pay.hanzo.ai">pay.hanzo.ai</a> top up
<a href="https://billing.hanzo.ai">billing.hanzo.ai</a> invoices and usage</div>
<div><b>Company</b>
<a href="https://hanzo.ai">hanzo.ai</a> canonical
<a href="https://hanzo.ventures">hanzo.ventures</a> ventures</div>
</div>
<div class="fend"><a href="/research-research-design.md">This page as plain text</a> · <a href="/llms.txt">Index for agents</a></div>
</footer>
</div></body></html>