|
195 | 195 | "id": "cell-05-90139e87", |
196 | 196 | "metadata": {}, |
197 | 197 | "source": [ |
198 | | - "**About the normalization warning:** You'll see `pweight weights normalized to mean=1` throughout this tutorial. Survey weights are inverse selection probabilities -- they rarely have mean=1 out of the box. The library rescales them internally so that weighted estimators are numerically stable. This is standard practice (Lumley 2004, \u00a72.2). The warning confirms rescaling occurred; it is not an error." |
| 198 | + "**About the normalization warning:** You'll see `pweight weights normalized to mean=1` throughout this tutorial. Survey weights are inverse selection probabilities -- they rarely have mean=1 out of the box. The library rescales them internally so that weighted estimators are numerically stable. This is standard practice (Lumley 2004, §2.2). The warning confirms rescaling occurred; it is not an error." |
199 | 199 | ] |
200 | 200 | }, |
201 | 201 | { |
|
1087 | 1087 | "cell_type": "markdown", |
1088 | 1088 | "id": "cell-35-f1ef376c", |
1089 | 1089 | "metadata": {}, |
1090 | | - "source": [ |
1091 | | - "## 9. Which Estimators Support Survey Design?\n", |
1092 | | - "\n", |
1093 | | - "`diff-diff` supports survey design across all estimators, though the level of support varies:\n", |
1094 | | - "\n", |
1095 | | - "| Estimator | Weights | Strata/PSU/FPC (TSL) | Replicate Weights | Survey-Aware Bootstrap |\n", |
1096 | | - "|-----------|---------|---------------------|-------------------|------------------------|\n", |
1097 | | - "| **DifferenceInDifferences** | Full | Full | -- | -- |\n", |
1098 | | - "| **TwoWayFixedEffects** | Full | Full | -- | -- |\n", |
1099 | | - "| **MultiPeriodDiD** | Full | Full | -- | -- |\n", |
1100 | | - "| **CallawaySantAnna** | pweight only | Full | Full | Multiplier at PSU |\n", |
1101 | | - "| **TripleDifference** | pweight only | Full | Full (analytical) | -- |\n", |
1102 | | - "| **StaggeredTripleDifference** | pweight only | Full | Full | Multiplier at PSU |\n", |
1103 | | - "| **SunAbraham** | Full | Full | -- | Rao-Wu rescaled |\n", |
1104 | | - "| **StackedDiD** | pweight only | Full (pweight only) | -- | -- |\n", |
1105 | | - "| **ImputationDiD** | pweight only | Partial (no FPC) | -- | Multiplier at PSU |\n", |
1106 | | - "| **TwoStageDiD** | pweight only | Partial (no FPC) | -- | Multiplier at PSU |\n", |
1107 | | - "| **ContinuousDiD** | Full | Full | Full (analytical) | Multiplier at PSU |\n", |
1108 | | - "| **EfficientDiD** | Full | Full | Full (analytical) | Multiplier at PSU |\n", |
1109 | | - "| **SyntheticDiD** | pweight only | -- | -- | Rao-Wu rescaled |\n", |
1110 | | - "| **TROP** | pweight only | -- | -- | Rao-Wu rescaled |\n", |
1111 | | - "| **BaconDecomposition** | Diagnostic | Diagnostic | -- | -- |\n", |
1112 | | - "\n", |
1113 | | - "**Legend:**\n", |
1114 | | - "- **Full**: All weight types (pweight/fweight/aweight) + strata/PSU/FPC + Taylor Series Linearization variance\n", |
1115 | | - "- **Full (pweight only)**: Full TSL support with strata/PSU/FPC, but only accepts `pweight` weight type (`fweight`/`aweight` rejected because Q-weight composition changes their semantics)\n", |
1116 | | - "- **Partial (no FPC)**: Weights + strata (for df) + PSU (for clustering); FPC raises `NotImplementedError`\n", |
1117 | | - "- **pweight only** (Weights column): Only `pweight` accepted; `fweight`/`aweight` raise an error\n", |
1118 | | - "- **pweight only** (TSL column): Sampling weights for point estimates; no strata/PSU/FPC design elements\n", |
1119 | | - "- **Diagnostic**: Weighted descriptive statistics only (no inference)\n", |
1120 | | - "- **--**: Not supported\n", |
1121 | | - "\n", |
1122 | | - "**Note:** `EfficientDiD` supports `covariates` and `survey_design` simultaneously. The doubly-robust (DR) path threads survey weights through WLS outcome regression, weighted sieve propensity ratios, and survey-weighted kernel smoothing.\n", |
1123 | | - "\n", |
1124 | | - "For full details, see `docs/survey-roadmap.md`." |
1125 | | - ] |
| 1090 | + "source": "## 9. Which Estimators Support Survey Design?\n\n`diff-diff` supports survey design across all estimators, though the level of support varies:\n\n| Estimator | Weights | Strata/PSU/FPC (TSL) | Replicate Weights | Survey-Aware Bootstrap |\n|-----------|---------|---------------------|-------------------|------------------------|\n| **DifferenceInDifferences** | Full | Full | -- | -- |\n| **TwoWayFixedEffects** | Full | Full | -- | -- |\n| **MultiPeriodDiD** | Full | Full | -- | -- |\n| **CallawaySantAnna** | pweight only | Full | Full | Multiplier at PSU |\n| **TripleDifference** | pweight only | Full | Full (analytical) | -- |\n| **StaggeredTripleDifference** | pweight only | Full | Full | Multiplier at PSU |\n| **SunAbraham** | Full | Full | -- | Rao-Wu rescaled |\n| **StackedDiD** | pweight only | Full (pweight only) | -- | -- |\n| **ImputationDiD** | pweight only | Partial (no FPC) | -- | Multiplier at PSU |\n| **TwoStageDiD** | pweight only | Partial (no FPC) | -- | Multiplier at PSU |\n| **ContinuousDiD** | Full | Full | Full (analytical) | Multiplier at PSU |\n| **EfficientDiD** | Full | Full | Full (analytical) | Multiplier at PSU |\n| **SyntheticDiD** | pweight only (placebo / jackknife) | -- | -- | -- |\n| **TROP** | pweight only | -- | -- | Rao-Wu rescaled |\n| **BaconDecomposition** | Diagnostic | Diagnostic | -- | -- |\n\n**Legend:**\n- **Full**: All weight types (pweight/fweight/aweight) + strata/PSU/FPC + Taylor Series Linearization variance\n- **Full (pweight only)**: Full TSL support with strata/PSU/FPC, but only accepts `pweight` weight type (`fweight`/`aweight` rejected because Q-weight composition changes their semantics)\n- **Partial (no FPC)**: Weights + strata (for df) + PSU (for clustering); FPC raises `NotImplementedError`\n- **pweight only** (Weights column): Only `pweight` accepted; `fweight`/`aweight` raise an error\n- **pweight only** (TSL column): Sampling weights for point estimates; no strata/PSU/FPC design elements\n- **Diagnostic**: Weighted descriptive statistics only (no inference)\n- **--**: Not supported\n\n**Note on SyntheticDiD:** `variance_method=\"placebo\"` and `variance_method=\"jackknife\"` support pweight-only survey designs. `variance_method=\"bootstrap\"` rejects every survey design (including pweight-only) because the paper-faithful refit bootstrap composed with Rao-Wu rescaled weights requires a weighted-Frank-Wolfe derivation that is not yet implemented. Strata/PSU/FPC are not supported by any SDID variance method in this release. The weighted-FW + Rao-Wu composition follow-up is tracked in `TODO.md`; see `docs/methodology/REGISTRY.md` §SyntheticDiD for the deferred-composition sketch.\n\n**Note:** `EfficientDiD` supports `covariates` and `survey_design` simultaneously. The doubly-robust (DR) path threads survey weights through WLS outcome regression, weighted sieve propensity ratios, and survey-weighted kernel smoothing.\n\nFor full details, see `docs/survey-roadmap.md`." |
1126 | 1091 | }, |
1127 | 1092 | { |
1128 | 1093 | "cell_type": "markdown", |
|
1137 | 1102 | "\n", |
1138 | 1103 | "**Policy background.** The Affordable Care Act's dependent coverage provision, effective\n", |
1139 | 1104 | "September 2010, allowed young adults to remain on their parents' health insurance until age 26.\n", |
1140 | | - "This created a natural experiment: adults aged 19\u201325 gained coverage access (treatment group),\n", |
1141 | | - "while adults aged 27\u201334 \u2014 similar demographics but ineligible \u2014 serve as controls. This is one\n", |
| 1105 | + "This created a natural experiment: adults aged 19–25 gained coverage access (treatment group),\n", |
| 1106 | + "while adults aged 27–34 — similar demographics but ineligible — serve as controls. This is one\n", |
1142 | 1107 | "of the most widely studied DiD natural experiments in health economics.\n", |
1143 | 1108 | "\n", |
1144 | 1109 | "> Antwi, Y.A., Moriya, A.S. & Simon, K. (2013). \"Effects of Federal Policy to Insure Young\n", |
1145 | 1110 | "> Adults: Evidence from the 2010 Affordable Care Act's Dependent-Coverage Mandate.\"\n", |
1146 | | - "> *American Economic Journal: Economic Policy* 5(4): 1\u201328.\n", |
| 1111 | + "> *American Economic Journal: Economic Policy* 5(4): 1–28.\n", |
1147 | 1112 | "\n", |
1148 | | - "We use two NHANES cycles: **2007\u20132008** (pre-ACA) and **2015\u20132016** (post-ACA), with health\n", |
| 1113 | + "We use two NHANES cycles: **2007–2008** (pre-ACA) and **2015–2016** (post-ACA), with health\n", |
1149 | 1114 | "insurance coverage as the outcome (binary, modeled as a linear probability model). NHANES uses\n", |
1150 | 1115 | "a complex multi-stage probability sampling design with **masked pseudo-strata** (`SDMVSTRA`),\n", |
1151 | 1116 | "**masked pseudo-PSUs** (`SDMVPSU`), and **exam weights** (`WTMEC2YR`). Because PSU IDs are\n", |
|
1351 | 1316 | "source": [ |
1352 | 1317 | "With real survey data, **both the point estimate and standard error change** when we account\n", |
1353 | 1318 | "for the survey design. The ATT shifts from 0.065 (unweighted) to 0.097 (weighted) because\n", |
1354 | | - "NHANES uses unequal selection probabilities \u2014 the weighted estimate is population-representative,\n", |
| 1319 | + "NHANES uses unequal selection probabilities — the weighted estimate is population-representative,\n", |
1355 | 1320 | "while the unweighted one over- or under-represents certain demographic groups. The SE also\n", |
1356 | 1321 | "increases because NHANES clusters individuals within PSUs, and people from the same geographic\n", |
1357 | 1322 | "area have correlated insurance status.\n", |
1358 | 1323 | "\n", |
1359 | 1324 | "The survey-corrected estimate of **~9.7 percentage points** suggests that the ACA provisions\n", |
1360 | 1325 | "meaningfully increased insurance coverage among young adults. This is consistent with the\n", |
1361 | 1326 | "published literature: studies measuring only the 2010 dependent coverage mandate find\n", |
1362 | | - "3\u20136 pp effects (Antwi et al., 2013; Sommers, 2012), while studies spanning the full\n", |
1363 | | - "ACA implementation \u2014 including the 2014 marketplace, individual mandate, and Medicaid\n", |
1364 | | - "expansion \u2014 find 8\u201313 pp (Kaestner et al., 2017; Courtemanche et al., 2017). Our\n", |
1365 | | - "2007\u201308 vs. 2015\u201316 window captures all of these provisions, placing the 9.7 pp\n", |
| 1327 | + "3–6 pp effects (Antwi et al., 2013; Sommers, 2012), while studies spanning the full\n", |
| 1328 | + "ACA implementation — including the 2014 marketplace, individual mandate, and Medicaid\n", |
| 1329 | + "expansion — find 8–13 pp (Kaestner et al., 2017; Courtemanche et al., 2017). Our\n", |
| 1330 | + "2007–08 vs. 2015–16 window captures all of these provisions, placing the 9.7 pp\n", |
1366 | 1331 | "estimate squarely in the expected range.\n", |
1367 | 1332 | "\n", |
1368 | | - "The survey degrees of freedom (31 = n_PSU \u2212 n_strata) reflect the actual number\n", |
| 1333 | + "The survey degrees of freedom (31 = n_PSU − n_strata) reflect the actual number\n", |
1369 | 1334 | "of independent sampling units, not the number of individuals. This is why the\n", |
1370 | 1335 | "confidence interval [0.006, 0.187] is wide despite nearly 3,000 observations.\n", |
1371 | 1336 | "\n", |
|
0 commit comments