Skip to content

Commit 826b1a8

Browse files
igerberclaude
andcommitted
Address PR #351 R5 P3: test silent-column-pickup contract
The previous rewrite compared two semantically identical pweight-only SurveyDesign constructions, which can't catch the contract the docstring claimed — if a future change silently picked up `stratum` or `psu` by name, both fits would pick them up identically and the test would still pass. Rewrite to compare a fit on the original DataFrame (with `stratum` / `psu` columns present) against a fit on the same data with those columns physically dropped. If the estimator ever silently reads those columns by naming convention the two fits would diverge and the abs=1e-12 ATT check would fail. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent ec92d39 commit 826b1a8

1 file changed

Lines changed: 19 additions & 21 deletions

File tree

tests/test_survey_phase5.py

Lines changed: 19 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -227,26 +227,23 @@ def test_full_design_jackknife_raises(self, sdid_survey_data, survey_design_full
227227
def test_placebo_with_pweight_only_full_design_stripped_att_match(
228228
self, sdid_survey_data
229229
):
230-
"""Placebo ATT with pweight-only matches ATT when strata/PSU are
231-
stripped (never attached to the SurveyDesign).
230+
"""Placebo ATT with pweight-only is unchanged when stratum/psu
231+
columns are physically dropped from the input DataFrame.
232232
233233
Point estimates depend only on the pseudo-population weights, not on
234234
the strata/PSU structure — full-design bootstrap previously exploited
235235
that structure via Rao-Wu rescaling and is now rejected upstream (see
236236
``test_full_design_bootstrap_raises`` /
237-
``test_full_design_placebo_raises``). This asserts ATT-equivalence
238-
between an explicit pweight-only design and an otherwise identical
239-
pweight-only design — i.e., the strata/psu columns in the DataFrame
240-
are not tacitly read by the estimator unless the SurveyDesign
241-
references them.
237+
``test_full_design_placebo_raises``). A silent pickup of ``stratum``
238+
or ``psu`` by the estimator (e.g., by name-matching a convention
239+
column) would cause the two fits to diverge, so comparing a
240+
DataFrame with those columns present against one with them dropped
241+
is the real contract.
242242
"""
243243
sd_pweight_only = SurveyDesign(weights="weight")
244-
# Same pweight-only design, but constructed via the full helper shape
245-
# to confirm no silent column pickup:
246-
sd_pweight_explicit = SurveyDesign(weights="weight", weight_type="pweight")
247244
est = SyntheticDiD(variance_method="placebo", n_bootstrap=100, seed=42)
248245

249-
result_a = est.fit(
246+
result_with_cols = est.fit(
250247
sdid_survey_data,
251248
outcome="outcome",
252249
treatment="treated",
@@ -255,21 +252,22 @@ def test_placebo_with_pweight_only_full_design_stripped_att_match(
255252
post_periods=[6, 7, 8, 9],
256253
survey_design=sd_pweight_only,
257254
)
258-
result_b = est.fit(
259-
sdid_survey_data,
255+
sdid_survey_data_stripped = sdid_survey_data.drop(columns=["stratum", "psu"])
256+
result_stripped = est.fit(
257+
sdid_survey_data_stripped,
260258
outcome="outcome",
261259
treatment="treated",
262260
unit="unit",
263261
time="time",
264262
post_periods=[6, 7, 8, 9],
265-
survey_design=sd_pweight_explicit,
266-
)
267-
assert np.isfinite(result_a.att)
268-
assert np.isfinite(result_a.se)
269-
assert result_a.se > 0
270-
# ATT is a deterministic function of the pweight-only design — the
271-
# two equivalent pweight-only constructions must agree bit-for-bit.
272-
assert result_a.att == pytest.approx(result_b.att, abs=1e-12)
263+
survey_design=sd_pweight_only,
264+
)
265+
assert np.isfinite(result_with_cols.att)
266+
assert np.isfinite(result_with_cols.se)
267+
assert result_with_cols.se > 0
268+
# ATT depends only on pweight — silent pickup of stratum/psu would
269+
# make the with-columns fit differ from the stripped fit.
270+
assert result_with_cols.att == pytest.approx(result_stripped.att, abs=1e-12)
273271

274272
def test_fweight_aweight_raises(self, sdid_survey_data):
275273
"""Non-pweight raises ValueError."""

0 commit comments

Comments
 (0)