Skip to content

Commit 054e2ee

Browse files
igerberclaude
andcommitted
Codex CI R11 P3: gate MPD wild-bootstrap fallback warning on non-Conley path + harmonize doc restriction bullets
Two P3 doc-and-UX items from R11 (✅ Looks good overall). P3 #1 (Code Quality) — contradictory warning + error on MPD+Conley +wild_bootstrap `MultiPeriodDiD.fit()` had an unconditional UserWarning at the top of fit() saying wild_bootstrap "is not yet supported, using analytical instead" — but the Conley validator below raises NotImplementedError on `vcov_type="conley", inference="wild_bootstrap"`. Users hitting that combination got "warn we're falling back" followed immediately by "actually we're raising," which is contradictory guidance on the same call. Fix: gate the warning on `self.vcov_type != "conley"`. The Conley validator's raise takes precedence. Adds regression `test_mpd_conley_wild_bootstrap_raises_without_warning` asserting the NotImplementedError fires with no analytical-fallback warning. P3 #2 (Documentation/Tests) — restriction bullets understated MPD's reject surface - `llms-full.txt:L2026` only named DiD/TWFE for the `inference="wild_bootstrap"` reject; MPD also rejects this combo via the shared validator. Updated to enumerate all three. - `llms-full.txt:L2027` only named DiD for the estimator-level `survey_design=` reject; MPD and TWFE both raise the same on this combo. Updated to enumerate all three. - `docs/methodology/REGISTRY.md:L3188` had the same drift; updated in parallel and added a parenthetical noting the MPD warning suppression. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 3b05cd2 commit 054e2ee

4 files changed

Lines changed: 67 additions & 5 deletions

File tree

diff_diff/estimators.py

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1249,9 +1249,14 @@ def fit( # type: ignore[override]
12491249
If required parameters are missing or data validation fails.
12501250
"""
12511251
# Fall back to analytical inference if wild bootstrap requested
1252-
# (must happen before _resolve_survey_for_fit which rejects bootstrap+survey)
1252+
# (must happen before _resolve_survey_for_fit which rejects bootstrap+survey).
1253+
# SKIP the warning on the Conley path — the Conley validator below
1254+
# raises NotImplementedError for wild_bootstrap + Conley, so emitting
1255+
# the analytical-fallback warning first would produce contradictory
1256+
# guidance on the same call (warn "falling back" + raise "not
1257+
# supported"). The Conley raise takes precedence. Codex CI R11 P3.
12531258
effective_inference = self.inference
1254-
if self.inference == "wild_bootstrap":
1259+
if self.inference == "wild_bootstrap" and self.vcov_type != "conley":
12551260
warnings.warn(
12561261
"Wild bootstrap inference is not yet supported for MultiPeriodDiD. "
12571262
"Using analytical inference instead.",

diff_diff/guides/llms-full.txt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2023,8 +2023,8 @@ SE range.
20232023
- `DifferenceInDifferences` / `MultiPeriodDiD` / `TwoWayFixedEffects` `+ vcov_type="conley"` without `conley_lag_cutoff` → `ValueError` (no defensible default; explicit user choice required).
20242024
- `DifferenceInDifferences` / `MultiPeriodDiD` `+ vcov_type="conley"` without `unit=` at fit-time → `ValueError`.
20252025
- Combining `vcov_type="conley"` with explicit `cluster=<col>` applies the combined spatial + cluster product kernel (Wave A #119). On the panel path the cluster must be constant within each unit across periods (validator raises `ValueError` otherwise). TWFE's default auto-cluster on the Conley path is silently dropped; users opt into the combined kernel explicitly.
2026-
- `TwoWayFixedEffects` / `DifferenceInDifferences` `(vcov_type="conley", inference="wild_bootstrap")` → `NotImplementedError` (wild bootstrap does not consume the analytical sandwich).
2027-
- `DifferenceInDifferences(vcov_type="conley")` + `survey_design=` → `NotImplementedError` (Bertanha-Imbens 2014 follow-up).
2026+
- `DifferenceInDifferences` / `MultiPeriodDiD` / `TwoWayFixedEffects` `(vcov_type="conley", inference="wild_bootstrap")` → `NotImplementedError` (wild bootstrap does not consume the analytical sandwich).
2027+
- `DifferenceInDifferences` / `MultiPeriodDiD` / `TwoWayFixedEffects` `(vcov_type="conley")` + `survey_design=` → `NotImplementedError` (Bertanha-Imbens 2014 follow-up).
20282028
- `SyntheticDiD(vcov_type="conley")` → `TypeError` (uses bootstrap, not analytical sandwich).
20292029
- Any-mode `vcov_type="conley"` `+ weights=` / `survey_design=` on `LinearRegression` / `compute_robust_vcov` → `NotImplementedError` (Bertanha-Imbens 2014 weighted-Conley deferred to follow-up PR).
20302030
- A sparse k-d-tree fast path auto-activates for `n > 5_000` with `conley_kernel="bartlett"` AND `conley_metric` in `{"haversine", "euclidean"}` (Wave A #120); callable metrics and uniform kernel fall back to the dense path. `n > 20_000` with the dense fallback still emits a memory-OOM `UserWarning`.

docs/methodology/REGISTRY.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3185,7 +3185,8 @@ metrics (`"haversine"`, `"euclidean"`) satisfy this by construction.
31853185
- `MultiPeriodDiD` / `DifferenceInDifferences` `(vcov_type="conley")` without `unit=` at fit-time → `ValueError`.
31863186
- `TwoWayFixedEffects(vcov_type="conley", cluster=<col>)` is supported (Wave A #119): combined spatial + cluster product kernel applies. The cluster must be time-invariant within each unit on the panel path (validator-enforced). TWFE's default auto-cluster is silently dropped on the Conley path; explicit cluster is required to opt in.
31873187
- `DifferenceInDifferences(vcov_type="conley", cluster=<col>)`: combined kernel applies; same time-invariance contract on the panel path. DiD has no auto-cluster, so the cluster choice is fully explicit.
3188-
- `TwoWayFixedEffects(vcov_type="conley", inference="wild_bootstrap")` / `DifferenceInDifferences(vcov_type="conley", inference="wild_bootstrap")``NotImplementedError`.
3188+
- `DifferenceInDifferences` / `MultiPeriodDiD` / `TwoWayFixedEffects` `(vcov_type="conley", inference="wild_bootstrap")``NotImplementedError`. (MPD's pre-Conley analytical-fallback `UserWarning` is suppressed when `vcov_type="conley"` so the user gets one consistent error message.)
3189+
- `DifferenceInDifferences` / `MultiPeriodDiD` / `TwoWayFixedEffects` `(vcov_type="conley")` + `survey_design=``NotImplementedError` at the estimator level (deferred to Bertanha-Imbens 2014 weighted-Conley follow-up).
31893190
- `SyntheticDiD(vcov_type="conley")``TypeError` (SyntheticDiD uses bootstrap/jackknife/placebo variance, not the analytical sandwich; tracked in TODO.md).
31903191
- Any-mode `vcov_type="conley"` `+ weights=` / `survey_design=``NotImplementedError` (Bertanha-Imbens 2014 territory; Phase 5 follow-up).
31913192
- Panel path: partial `conley_time` / `conley_unit` / `conley_lag_cutoff` (not all three set) → `ValueError` at validator.

tests/test_conley_vcov.py

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1259,6 +1259,62 @@ def test_twfe_conley_unknown_cluster_column_raises(self):
12591259
conley_lag_cutoff=1,
12601260
).fit(df, outcome="y", treatment="treated", time="time", unit="unit")
12611261

1262+
def test_mpd_conley_wild_bootstrap_raises_without_warning(self):
1263+
"""MPD + Conley + inference='wild_bootstrap' raises NotImplementedError
1264+
cleanly. The pre-Conley analytical-fallback UserWarning is suppressed
1265+
on this combination so the user gets one consistent error message
1266+
instead of "warn then raise". Codex CI R11 P3 #1.
1267+
"""
1268+
import pandas as _pd
1269+
1270+
from diff_diff import MultiPeriodDiD
1271+
1272+
rng = np.random.default_rng(seed=211)
1273+
rows = []
1274+
for u in range(8):
1275+
lat = rng.uniform(-30, 30)
1276+
lon = rng.uniform(-100, 100)
1277+
for t in range(3):
1278+
rows.append(
1279+
{
1280+
"unit": u,
1281+
"time": t,
1282+
"y": rng.standard_normal(),
1283+
"treated": int(u >= 4),
1284+
"lat": lat,
1285+
"lon": lon,
1286+
}
1287+
)
1288+
df = _pd.DataFrame(rows)
1289+
with warnings.catch_warnings(record=True) as w:
1290+
warnings.simplefilter("always")
1291+
with pytest.raises(NotImplementedError, match="wild_bootstrap"):
1292+
MultiPeriodDiD(
1293+
vcov_type="conley",
1294+
inference="wild_bootstrap",
1295+
conley_coords=("lat", "lon"),
1296+
conley_cutoff_km=2000.0,
1297+
conley_lag_cutoff=1,
1298+
).fit(
1299+
df,
1300+
outcome="y",
1301+
treatment="treated",
1302+
time="time",
1303+
unit="unit",
1304+
post_periods=[1, 2],
1305+
reference_period=0,
1306+
)
1307+
fallback_warnings = [
1308+
msg
1309+
for msg in w
1310+
if "falling back to analytical" in str(msg.message)
1311+
or "Wild bootstrap inference is not yet supported" in str(msg.message)
1312+
]
1313+
assert len(fallback_warnings) == 0, (
1314+
"Got the analytical-fallback warning on a Conley fit that will "
1315+
"raise NotImplementedError — contradictory guidance."
1316+
)
1317+
12621318
def test_did_conley_malformed_coord_tuple_raises(self, two_period_panel):
12631319
"""vcov_type='conley' with a malformed conley_coords (wrong arity or
12641320
non-string elements) raises ValueError before downstream access.

0 commit comments

Comments
 (0)