Skip to content

Commit 9010ef7

Browse files
igerberclaude
andcommitted
Update survey roadmap: mark 10b shipped, note conditional PT gap for 10e
- Mark Phase 10b (research-grade DGP) as shipped with parameter summary - Add conditional PT as required DGP extension for position paper (10e) simulation study — unconditional PT fails, conditional PT holds after covariate adjustment, DR/IPW recovers truth - Genericize co-authorship suggestion (no specific names) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent e123338 commit 9010ef7

1 file changed

Lines changed: 28 additions & 18 deletions

File tree

docs/survey-roadmap.md

Lines changed: 28 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -137,22 +137,18 @@ immediately.
137137
Survey Data." *JASA* 83(401).
138138
- Shao, J. (1996). "Resampling Methods in Sample Surveys." *Statistics* 27.
139139

140-
### 10b. Survey Simulation DGP (HIGH priority)
140+
### 10b. Survey Simulation DGP (HIGH priority)
141141

142-
Build a research-grade DGP that generates realistic complex survey data
143-
over a staggered treatment adoption panel. The existing `generate_survey_did_data()`
144-
tests code correctness but lacks the properties needed for statistical
145-
coverage studies and compelling tutorials. The new DGP needs:
142+
Enhanced `generate_survey_did_data()` with 8 research-grade parameters:
143+
`icc`, `weight_cv`, `informative_sampling`, `heterogeneous_te_by_strata`,
144+
`te_covariate_interaction`, `covariate_effects`, `strata_sizes`, and
145+
`return_true_population_att`. All backward-compatible. Supports panel
146+
and repeated cross-section modes.
146147

147-
- Known stratified cluster structure with varying PSU sizes
148-
- Controllable intra-cluster correlation (so true DEFF is known)
149-
- Known treatment effects (so coverage of 95% CIs can be measured)
150-
- Enough design complexity to show where flat weights fail (clustering
151-
inflates variance, stratification reduces it, FPC matters for small
152-
populations)
153-
154-
This is a dependency for both 10c (tutorial) and 10d (paper simulation
155-
study). Add to `diff_diff.prep` alongside the existing DGP functions.
148+
**Remaining gap for 10e:** Conditional parallel trends — the DGP has
149+
unconditional PT by construction. A `conditional_pt` parameter is needed
150+
before the simulation study so that unconditional PT fails but conditional
151+
PT holds after covariate adjustment (DR/IPW recovers truth).
156152

157153
### 10c. Expand R Validation Coverage (HIGH priority)
158154

@@ -191,10 +187,24 @@ arXiv. Theory (~5pp), simulation study using DGP from 10b (~8pp),
191187
empirical illustration with NHANES ACA data (~3pp), software section
192188
(~2pp).
193189

194-
**Ideal co-author:** Pedro Sant'Anna — derived the IFs in CS/DRDID and
195-
can vouch they are valid under survey weighting. The survey statistics
196-
(Binder 1983, Rao & Wu 1988) are established and don't need a survey
197-
methodologist to co-sign.
190+
**Simulation study scenarios** (minimum):
191+
1. Unconditional PT with complex survey — coverage of TSL vs flat-weight SEs
192+
2. Informative sampling + heterogeneous TE — weighted ATT bias correction
193+
3. Panel vs repeated cross-section — both design types
194+
4. **Conditional PT** — unconditional PT fails (differential pre-trends
195+
correlated with X), conditional PT holds after covariate adjustment.
196+
DR/IPW with covariates recovers truth; no-covariate estimator is biased.
197+
This is the most novel claim — survey-weighted nuisance estimation
198+
(propensity scores, outcome regression) produces valid IFs under complex
199+
sampling. **Requires DGP extension**: add a `conditional_pt` parameter
200+
to `generate_survey_did_data()` that makes the time trend
201+
X-dependent (e.g., `trend_i = 0.5*t + delta * x1_i * t`).
202+
203+
**Co-authorship:** A co-author from the DiD methodology community would
204+
strengthen credibility — someone who can vouch that the IFs are valid
205+
under survey weighting. The survey statistics side (Binder 1983, Rao &
206+
Wu 1988) is established and doesn't need a survey methodologist to
207+
co-sign.
198208

199209
### 10f. WooldridgeDiD Survey Support (MEDIUM priority)
200210

0 commit comments

Comments
 (0)