Skip to content

Commit d7ec694

Browse files
igerberclaude
andcommitted
Add webb bootstrap weight tests, document TROP n_bootstrap constraint for PR #165 round 4
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 98029d1 commit d7ec694

3 files changed

Lines changed: 32 additions & 0 deletions

File tree

docs/methodology/REGISTRY.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -941,6 +941,7 @@ Q(λ) = Σ_{j,s: D_js=0} [τ̂_js^loocv(λ)]²
941941
- **n_post_periods metadata**: Counts periods where D=1 is actually observed (at least one unit has D=1), not calendar periods from first treatment. In unbalanced panels where treated units are missing in some post-treatment periods, only periods with observed D=1 values are counted.
942942
- Wrong D specification: if user provides event-style D (only first treatment period),
943943
the absorbing-state validation will raise ValueError with helpful guidance
944+
- **Bootstrap minimum**: `n_bootstrap` must be >= 2 (enforced via `ValueError`). TROP uses bootstrap for all variance estimation — there is no analytical SE formula.
944945
- **LOOCV failure metadata**: When LOOCV fits fail in the Rust backend, the first failed observation coordinates (t, i) are returned to Python for informative warning messages
945946
- **Inference CI distribution**: After `safe_inference()` migration, CI uses t-distribution (df = max(1, n_treated_obs - 1)), consistent with p_value. Previously CI used normal-distribution while p_value used t-distribution (inconsistent). This is a minor behavioral change; CIs may be slightly wider for small n_treated_obs.
946947

tests/test_imputation.py

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1104,6 +1104,21 @@ def test_bootstrap_weights_mammen(self, ci_params):
11041104
assert br.overall_att_se > 0
11051105
assert np.isfinite(br.overall_att_p_value)
11061106

1107+
def test_bootstrap_weights_webb(self, ci_params):
1108+
"""Bootstrap with webb weights should produce valid results."""
1109+
data = generate_test_data()
1110+
n_boot = ci_params.bootstrap(50)
1111+
est = ImputationDiD(n_bootstrap=n_boot, bootstrap_weights="webb", seed=42)
1112+
results = est.fit(
1113+
data, outcome="outcome", unit="unit", time="time", first_treat="first_treat",
1114+
)
1115+
1116+
br = results.bootstrap_results
1117+
assert br is not None
1118+
assert br.weight_type == "webb"
1119+
assert br.overall_att_se > 0
1120+
assert np.isfinite(br.overall_att_p_value)
1121+
11071122
def test_bootstrap_with_covariates(self, ci_params):
11081123
"""Bootstrap should work with covariates."""
11091124
data = generate_test_data()

tests/test_two_stage.py

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1076,6 +1076,22 @@ def test_bootstrap_weights_mammen(self, ci_params):
10761076
assert br.overall_att_se > 0
10771077
assert np.isfinite(br.overall_att_p_value)
10781078

1079+
def test_bootstrap_weights_webb(self, ci_params):
1080+
"""Bootstrap with webb weights should produce valid results."""
1081+
data = generate_test_data()
1082+
n_boot = ci_params.bootstrap(50)
1083+
results = TwoStageDiD(
1084+
n_bootstrap=n_boot, bootstrap_weights="webb", seed=42
1085+
).fit(
1086+
data, outcome="outcome", unit="unit", time="time", first_treat="first_treat"
1087+
)
1088+
1089+
br = results.bootstrap_results
1090+
assert br is not None
1091+
assert br.weight_type == "webb"
1092+
assert br.overall_att_se > 0
1093+
assert np.isfinite(br.overall_att_p_value)
1094+
10791095

10801096
# =============================================================================
10811097
# TestTwoStageDiDConvenience

0 commit comments

Comments
 (0)