Skip to content

Commit 14891c9

Browse files
igerberclaude
andcommitted
Add per-path joint sup-t bands to ChaisemartinDHaultfoeuille.by_path
When `n_bootstrap > 0` is set with `by_path=k`, per-path joint sup-t simultaneous confidence bands are now computed across horizons `1..L_max` within each path. A single shared `(n_bootstrap, n_eligible)` Hall-Mammen multiplier weight matrix is drawn per path and broadcast across all valid horizons, producing correlated bootstrap distributions. The path-specific critical value `c_p = quantile(max_l |t_l|, 1-α)` is applied per horizon as `cband_conf_int = (eff - c_p·se, eff + c_p·se)`, and surfaced at top level as `results.path_sup_t_bands[path]`. Closes Wave 2 #4 of the by_path follow-up sequence (PR #357 foundation, PR #360 R-parity, PR #364 bootstrap, PR #371 placebos). Methodology asymmetry vs OVERALL `event_study_sup_t_bands` (intentional, documented): per-path sup-t draws fresh shared weights AFTER the per-path SE bootstrap block has populated `path_ses` via independent per-(path, horizon) draws. Asymptotically equivalent to OVERALL's self-consistent reuse but NOT bit-identical. Preserves RNG-state isolation for existing per-path SE seed-reproducibility tests. Gates mirror OVERALL: a path needs >=2 valid horizons (finite bootstrap SE > 0) AND >=50% finite sup-t draws to receive a band; otherwise the path is absent from `path_sup_t_bands` (NaN-on-invalid absent-key pattern). Deviation from R: `did_multiplegt_dyn` provides no joint / sup-t bands at any surface — Python-only methodology extension consistent with the existing OVERALL sup-t bands. Inherits the cross-path cohort-sharing SE deviation from R documented for `path_effects` (the bootstrap SE used in the t-stat denominator carries the same deviation). Bundled pre-audit fix (sibling-surface check): the existing OVERALL `sup_t_bands` field's stale "Phase 2 placeholder" docstring updated to the actual contract description. Tests: new `TestByPathSupTBands` class with 11 tests covering none-when-no-bootstrap, none-when-no-by_path, keys-match-path-effects, band-wider-than-pointwise, crit-finite-and-positive, seed-reproducibility, single-horizon-path-skip, L_max=1-skip, n_valid_horizons-matches, absent-path-no-cband-keys, summary-renders. All `@pytest.mark.slow`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 8596f51 commit 14891c9

7 files changed

Lines changed: 544 additions & 2 deletions

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
88
## [Unreleased]
99

1010
### Added
11+
- **`ChaisemartinDHaultfoeuille.by_path` + `n_bootstrap > 0` joint sup-t bands** — per-path joint sup-t simultaneous confidence intervals across horizons `1..L_max` within each path. A single shared `(n_bootstrap, n_eligible)` Hall-Mammen multiplier weight matrix is drawn per path and broadcast across all horizons of that path, producing correlated bootstrap distributions across horizons. The path-specific critical value `c_p = quantile(max_l |t_l|, 1 - α)` is used to construct symmetric joint bands `effect_l ± c_p · se_l` per horizon. Surfaced on `results.path_sup_t_bands` (dict keyed by path tuple, each entry with `crit_value / alpha / n_bootstrap / method / n_valid_horizons`) and as `cband_conf_int` per horizon entry on `path_effects[path]["horizons"][l]`. Gates: a path needs `>= 2` valid horizons (finite bootstrap SE > 0) AND `>= 50%` finite sup-t draws to receive a band. Empty-state contract: `path_sup_t_bands is None` when not requested; `{}` when requested but no path passes both gates. **Methodology asymmetry vs OVERALL `event_study_sup_t_bands`:** the per-path sup-t draws a fresh shared weight matrix per path AFTER the per-path SE bootstrap block has already populated `results.path_ses` via independent per-(path, horizon) draws — asymptotically equivalent to OVERALL's self-consistent reuse but NOT bit-identical. Documented intentional choice to preserve RNG-state isolation for existing per-path SE seed-reproducibility tests. Inherits the cross-path cohort-sharing SE deviation from R documented for `path_effects`. **Deviation from R:** `did_multiplegt_dyn` does not provide joint / sup-t bands at any surface — this is a Python-only methodology extension consistent with the existing OVERALL sup-t bands (also Python-only). Bands cover joint inference WITHIN a single path across horizons; they do NOT provide simultaneous coverage across paths. Pre-audit fix bundled: stale "Phase 2 placeholder" docstring on the existing `sup_t_bands` field updated to the actual contract description. Tests at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathSupTBands` (`@pytest.mark.slow`). See `docs/methodology/REGISTRY.md` §ChaisemartinDHaultfoeuille `Note (Phase 3 by_path per-path joint sup-t bands)` for the full contract.
1112
- **`ChaisemartinDHaultfoeuille.by_path` + `placebo=True`** — per-path backward-horizon placebos `DID^{pl}_{path, l}` for `l = 1..L_max`. The same per-path SE convention used for the event-study (joiners/leavers IF precedent: switcher-side contributions zeroed for non-path groups; cohort structure and control pool unchanged; plug-in SE with path-specific divisor `N^{pl}_{l, path}`) is applied to backward horizons via the new `switcher_subset_mask` parameter on `_compute_per_group_if_placebo_horizon`. Surfaced on `results.path_placebo_event_study[path][-l]` (negative-int inner keys mirroring `placebo_event_study`); `summary()` renders the rows alongside per-path event-study horizons; `to_dataframe(level="by_path")` emits negative-horizon rows alongside the existing positive-horizon rows. **Bootstrap** (when `n_bootstrap > 0`) propagates per-`(path, lag)` percentile CI / p-value through the same `_bootstrap_one_target` dispatch as the per-path event-study, with the canonical NaN-on-invalid contract enforced on the new surface (PR #364 library-wide invariant). **SE inherits the cross-path cohort-sharing deviation from R** documented for `path_effects` (full-panel cohort-centered plug-in vs R's per-path re-run): tracks R within tolerance on single-path-cohort panels, diverges materially on cohort-mixed panels — the bootstrap SE is a Monte Carlo analog of the analytical SE and inherits the same deviation. R-parity confirmed at `tests/test_chaisemartin_dhaultfoeuille_parity.py::TestDCDHDynRParityByPathPlacebo` on the new `multi_path_reversible_by_path_placebo` scenario (point estimates exact match; SE within Phase-2 envelope rtol ≤ 5%); positive analytical + bootstrap invariants at `tests/test_chaisemartin_dhaultfoeuille.py::TestByPathPlacebo` (and the gated `::TestBootstrap` subclass). See `docs/methodology/REGISTRY.md` §ChaisemartinDHaultfoeuille `Note (Phase 3 by_path ...)` → "Per-path placebos" for the full contract.
1213
- **Tutorial 19: dCDH for Marketing Pulse Campaigns** (`docs/tutorials/19_dcdh_marketing_pulse.ipynb`) — end-to-end practitioner walkthrough on a 60-market reversible-treatment panel covering the TWFE decomposition diagnostic (`twowayfeweights`), `DCDH` Phase 1 (DID_M, joiners-vs-leavers, single-lag placebo), the `L_max` multi-horizon event study with multiplier bootstrap, a stakeholder communication template, and drift guards. README listing for Tutorial 17 (Brand Awareness Survey) backfilled in the same edit. Cross-link from `docs/practitioner_decision_tree.rst` § "Reversible Treatment" added.
1314

diff_diff/chaisemartin_dhaultfoeuille.py

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -431,6 +431,21 @@ class ChaisemartinDHaultfoeuille(ChaisemartinDHaultfoeuilleBootstrapMixin):
431431
cross-path cohort-sharing deviation from R is inherited from
432432
the analytical event-study path.
433433
434+
With ``n_bootstrap > 0``, per-path joint sup-t simultaneous
435+
confidence bands are also computed across horizons
436+
``1..L_max`` within each path. A path-specific critical value
437+
``c_p`` (constructed from a fresh shared-weights multiplier-
438+
bootstrap draw per path) is surfaced at top level as
439+
``results.path_sup_t_bands[path] = {"crit_value", "alpha",
440+
"n_bootstrap", "method", "n_valid_horizons"}`` and applied
441+
per-horizon as ``cband_conf_int`` on
442+
``path_effects[path]["horizons"][l]``. Bands cover joint
443+
inference WITHIN a single path across horizons; they do NOT
444+
provide simultaneous coverage across paths. Python-only
445+
library extension; R ``did_multiplegt_dyn`` provides no joint
446+
bands at any surface. See REGISTRY.md ``Note (Phase 3 by_path
447+
per-path joint sup-t bands)``.
448+
434449
SE convention: per-path IF parallels the joiners / leavers
435450
construction — the switcher-side contribution is zeroed for
436451
groups not in the selected path, and the cohort structure and
@@ -2986,6 +3001,33 @@ def fit(
29863001
path_placebos[path_key][neg_key]["conf_int"] = (np.nan, np.nan)
29873002
path_placebos[path_key][neg_key]["t_stat"] = np.nan
29883003

3004+
# Phase 3: propagate per-path sup-t critical values to per-
3005+
# horizon `cband_conf_int` entries on path_effects (by_path +
3006+
# n_bootstrap > 0). Sibling of the OVERALL event-study cband
3007+
# propagation at `:2865-2875`. For each path with a finite
3008+
# crit, write `cband_conf_int = (eff - c_p*se, eff + c_p*se)`
3009+
# into each horizon's dict whose bootstrap-replaced SE is
3010+
# finite > 0. Mirror the OVERALL absent-key pattern: non-finite
3011+
# SE horizons simply don't get the `cband_conf_int` key.
3012+
if (
3013+
bootstrap_results is not None
3014+
and bootstrap_results.path_cband_crit_values is not None
3015+
and path_effects is not None
3016+
):
3017+
for path_key, crit in bootstrap_results.path_cband_crit_values.items():
3018+
if path_key not in path_effects:
3019+
continue
3020+
if not np.isfinite(crit):
3021+
continue
3022+
for l_h, h_dict in path_effects[path_key]["horizons"].items():
3023+
se = h_dict.get("se", np.nan)
3024+
eff = h_dict.get("effect", np.nan)
3025+
if np.isfinite(se) and se > 0:
3026+
h_dict["cband_conf_int"] = (
3027+
eff - crit * se,
3028+
eff + crit * se,
3029+
)
3030+
29893031
# When L_max >= 1 and the per-group path is active, sync
29903032
# overall_* from event_study_effects[1] AFTER bootstrap propagation
29913033
# so that bootstrap SE/p/CI flow to the top-level surface.
@@ -3618,6 +3660,28 @@ def fit(
36183660
),
36193661
path_effects=path_effects,
36203662
path_placebo_event_study=path_placebos,
3663+
path_sup_t_bands=(
3664+
{
3665+
path_key: {
3666+
"crit_value": crit,
3667+
"alpha": self.alpha,
3668+
"n_bootstrap": self.n_bootstrap,
3669+
"method": "multiplier_bootstrap",
3670+
"n_valid_horizons": (
3671+
bootstrap_results.path_cband_n_valid_horizons.get(path_key, 0)
3672+
if bootstrap_results.path_cband_n_valid_horizons is not None
3673+
else 0
3674+
),
3675+
}
3676+
for path_key, crit in bootstrap_results.path_cband_crit_values.items()
3677+
if np.isfinite(crit)
3678+
}
3679+
if (
3680+
bootstrap_results is not None
3681+
and bootstrap_results.path_cband_crit_values is not None
3682+
)
3683+
else None
3684+
),
36213685
survey_metadata=survey_metadata,
36223686
_estimator_ref=self,
36233687
)

diff_diff/chaisemartin_dhaultfoeuille_bootstrap.py

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -778,6 +778,90 @@ def _compute_dcdh_bootstrap(
778778
results.path_placebo_cis = path_pl_cis
779779
results.path_placebo_p_values = path_pl_pvals
780780

781+
# --- Phase 3: Per-path joint sup-t (by_path + n_bootstrap > 0) ---
782+
# Sibling of the OVERALL event-study sup-t at the multi-horizon
783+
# block above (`:599-614`). Per-path joint simultaneous
784+
# confidence bands across horizons 1..L_max within each path:
785+
# one shared (n_bootstrap, n_eligible) Hall-Mammen weight matrix
786+
# per path is broadcast across all valid horizons of that path,
787+
# producing correlated bootstrap distributions across horizons.
788+
# The path-specific critical value
789+
# `c_p = quantile(max_l |t_l|, 1-alpha)` is the band half-width
790+
# multiplier applied to each horizon's bootstrap SE in fit().
791+
#
792+
# Note (asymmetry vs OVERALL): this draws a FRESH shared-weights
793+
# matrix per path AFTER the per-path SE block above has populated
794+
# results.path_ses via independent per-(path, horizon) draws.
795+
# Numerator: fresh shared draws; denominator: bootstrap SEs from
796+
# the earlier independent draws. Asymptotically equivalent to
797+
# OVERALL's self-consistent reuse, but NOT bit-identical. The
798+
# fresh draw is intentional: it preserves RNG-state isolation
799+
# for existing per-path SE seed-reproducibility tests.
800+
#
801+
# Gates: a path needs >=2 valid horizons (finite bootstrap SE>0)
802+
# AND >=50% finite sup-t draws to receive a band. Otherwise the
803+
# path is absent from path_cband_crit_values (mirrors OVERALL
804+
# absent-key pattern at `:605,612`).
805+
if path_bootstrap_inputs is not None and results.path_ses:
806+
path_cband_crits: Dict[Tuple[int, ...], float] = {}
807+
path_cband_n_valid: Dict[Tuple[int, ...], int] = {}
808+
809+
for path_key, horizon_inputs in path_bootstrap_inputs.items():
810+
bs_ses_for_path = results.path_ses.get(path_key, {})
811+
valid_horizons = []
812+
for l_h, (u_h, n_h, eff_h, _u_pp_h) in sorted(horizon_inputs.items()):
813+
if u_h.size == 0 or n_h <= 0:
814+
continue
815+
bs_se = bs_ses_for_path.get(l_h, np.nan)
816+
if not np.isfinite(bs_se) or bs_se <= 0:
817+
continue
818+
valid_horizons.append((l_h, u_h, n_h, eff_h, bs_se))
819+
820+
if len(valid_horizons) < 2:
821+
continue
822+
823+
# All horizons within a path use the same n_eligible
824+
# (variance-eligible group ordering enforced by
825+
# _collect_path_bootstrap_inputs's use of
826+
# eligible_mask_var for cohort-recentering); use the
827+
# first valid horizon's IF size as the shared dim.
828+
n_dim = valid_horizons[0][1].size
829+
map_path = _map_for_target(
830+
n_dim,
831+
group_id_to_psu_code,
832+
eligible_group_ids,
833+
)
834+
with np.errstate(invalid="ignore", divide="ignore"):
835+
shared_weights = _generate_psu_or_group_weights(
836+
n_bootstrap=self.n_bootstrap,
837+
n_groups_target=n_dim,
838+
weight_type=self.bootstrap_weights,
839+
rng=rng,
840+
group_to_psu_map=map_path,
841+
)
842+
es_dists_path = []
843+
for _l_h, u_h, n_h, eff_h, _bs_se in valid_horizons:
844+
deviations = (shared_weights @ u_h) / n_h
845+
es_dists_path.append(eff_h + deviations)
846+
boot_matrix = np.asarray(es_dists_path)
847+
effects_vec = np.array([v[3] for v in valid_horizons])
848+
ses_vec = np.array([v[4] for v in valid_horizons])
849+
t_stats = np.abs((boot_matrix - effects_vec[:, None]) / ses_vec[:, None])
850+
sup_t_dist = np.max(t_stats, axis=0)
851+
finite_mask = np.isfinite(sup_t_dist)
852+
if finite_mask.sum() <= 0.5 * self.n_bootstrap:
853+
continue
854+
crit_p = float(np.quantile(sup_t_dist[finite_mask], 1.0 - self.alpha))
855+
856+
if not np.isfinite(crit_p):
857+
continue
858+
859+
path_cband_crits[path_key] = crit_p
860+
path_cband_n_valid[path_key] = len(valid_horizons)
861+
862+
results.path_cband_crit_values = path_cband_crits
863+
results.path_cband_n_valid_horizons = path_cband_n_valid
864+
781865
return results
782866

783867

0 commit comments

Comments
 (0)