mirror of
https://github.com/hanzoskill/causal-inference.git
synced 2026-10-30 21:08:40 +00:00
3.6 KiB
3.6 KiB
Treatment Effect Estimation
Practical methods for estimating causal effects from observational data.
Average Treatment Effect (ATE)
ATE = E[Y(1)] - E[Y(0)]
= E[Y | do(T=1)] - E[Y | do(T=0)]
Where T is treatment (e.g., send_time=morning vs evening).
Method 1: Difference in Means (Randomized)
If you can randomize (e.g., A/B test email timing):
ate = mean(Y[T==1]) - mean(Y[T==0])
se = sqrt(var(Y[T==1])/n1 + var(Y[T==0])/n0)
Use when: You control the intervention and can randomize.
Method 2: Regression Adjustment
Control for confounders via linear regression:
# Y = β0 + β1*T + β2*X1 + β3*X2 + ε
# β1 is the treatment effect, adjusted for confounders X1, X2
import statsmodels.api as sm
model = sm.OLS(Y, sm.add_constant(df[['treatment', 'confounder1', 'confounder2']]))
results = model.fit()
ate = results.params['treatment']
se = results.bse['treatment']
Use when: Linear relationships, known confounders.
Method 3: Propensity Score Matching
Match treated/control units with similar propensity scores:
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import NearestNeighbors
# 1. Estimate propensity scores
ps_model = LogisticRegression().fit(X_confounders, T)
propensity = ps_model.predict_proba(X_confounders)[:, 1]
# 2. Match on propensity
treated_idx = np.where(T == 1)[0]
control_idx = np.where(T == 0)[0]
nn = NearestNeighbors(n_neighbors=1)
nn.fit(propensity[control_idx].reshape(-1, 1))
matches = nn.kneighbors(propensity[treated_idx].reshape(-1, 1))[1]
# 3. Estimate ATE from matched pairs
matched_control_idx = control_idx[matches.flatten()]
ate = mean(Y[treated_idx]) - mean(Y[matched_control_idx])
Use when: Many confounders, non-linear relationships.
Method 4: Inverse Propensity Weighting (IPW)
Weight observations by inverse propensity:
# Treated: weight = 1/P(T=1|X)
# Control: weight = 1/P(T=0|X) = 1/(1-P(T=1|X))
weights_treated = 1 / propensity[T == 1]
weights_control = 1 / (1 - propensity[T == 0])
ate = (
np.average(Y[T == 1], weights=weights_treated) -
np.average(Y[T == 0], weights=weights_control)
)
Use when: Want population-level estimates, can model propensity well.
Method 5: Doubly Robust Estimation
Combines regression and IPW — consistent if either model is correct:
from econml.dr import DRLearner
dr = DRLearner()
dr.fit(Y, T, X_confounders)
ate = dr.ate(X_confounders)
Use when: Uncertain about model specification, want robustness.
Confidence Intervals
Always report uncertainty:
# 95% CI
ci_lower = ate - 1.96 * se
ci_upper = ate + 1.96 * se
# For decision-making, use conservative bound
if ci_lower > 0:
print("Confident positive effect")
elif ci_upper < 0:
print("Confident negative effect")
else:
print("Effect uncertain — may need more data or user confirmation")
Practical Guidelines for Bot
- Start with regression — simple, interpretable, usually sufficient
- Log everything — you need outcome data to estimate effects
- Wait for sufficient n — at least 30-50 observations per treatment level
- Check overlap — propensity methods fail if treatment groups don't overlap
- Validate with holdout — reserve recent data for prediction validation
Quick Decision Matrix
| Situation | Method |
|---|---|
| Can randomize | Difference in means |
| Few confounders, linear | Regression |
| Many confounders | Propensity matching/IPW |
| Uncertain about model | Doubly robust |
| Time series | Interrupted time series / diff-in-diff |