Skip to content

Commit b634294

Browse files
igerberclaude
andcommitted
Add survey real-data validation against R using federal survey datasets
Validate diff-diff's survey variance estimation against R's survey package using three real-world datasets: California API (strata+FPC), NHANES ACA young adult coverage (strata+PSU+nest), and RECS 2020 (JK1 replicate weights). All 15 tests match R to machine precision (<1e-10 differences). Includes R benchmark scripts, Python download scripts, golden value JSON files, and a real-data section in the survey tutorial demonstrating the ACA dependent coverage provision DiD on actual CDC data. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 1f6ade8 commit b634294

15 files changed

Lines changed: 270621 additions & 81 deletions

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@ Thumbs.db
5555

5656
# Benchmarks - generated data and results (can be regenerated)
5757
benchmarks/data/synthetic/*.csv
58+
benchmarks/data/real/raw/
5859
benchmarks/results/
5960

6061
# Rust build artifacts

CHANGELOG.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
### Added
11+
- **Survey real-data validation** (Phase 9) — 15 cross-validation tests against R's `survey` package using three real federal survey datasets:
12+
- **API** (R `survey` package): TSL variance with strata, FPC, subpopulations, covariates, and Fay's BRR replicates
13+
- **NHANES** (CDC/NCHS): TSL variance with strata + PSU + nest=TRUE, validating the ACA young adult coverage provision DiD
14+
- **RECS 2020** (U.S. EIA): JK1 replicate weight variance with 60 pre-computed replicate columns
15+
- All metrics (ATT, SE, df, CI) match R to machine precision (< 1e-10 differences)
16+
1017
## [2.8.4] - 2026-04-04
1118

1219
### Added

0 commit comments

Comments
 (0)