Skip to content

Commit 846fd8f

Browse files
igerberclaude
andcommitted
Add Tutorial 20: HAD for National Brand Campaign with Regional Spend Intensity
Practitioner walkthrough for HeterogeneousAdoptionDiD on the no-untreated-controls case: every market got the campaign at varying intensity and there is no clean comparison group. Fills the structural gap T14 (ContinuousDiD) cannot address. Notebook scope (23 cells, 13 markdown / 10 code, mirrors T19's structure): - Sections 1-3: framing the no-untreated-controls measurement problem, setup imports, synthetic 60-DMA / 8-week panel with Uniform[$5K, $50K] regional add-on spend (every DMA participates, no DMA at $0). DGP is internally consistent: outcomes are generated from the dose values HAD then sees, no post-hoc relabeling. - Section 4: overall WAS_d_lower fit on a 2-period (pre/post mean) collapse - HAD's overall mode requires exactly 2 periods (had.py:952-959). Locked headline: per-$1K marginal effect of 100 weekly visits per DMA above the boundary spend (95% CI [98.6, 101.4]) with design auto-detection landing on `continuous_near_d_lower` (Design 1) and target `WAS_d_lower`. Surfaces the Assumption 5/6 advisory the library fires for Design 1 and explains why it holds in this DGP (linear by construction). - Section 5: multi-week event-study fit on the 8-week panel, per-week WAS_d_lower for e=0..3 (~100 each, CIs cover truth) and pre-launch placebos at e=-2..-4 sitting on zero. - Section 6: stakeholder communication template (T18/T19 markdown blockquote pattern), per-DMA dollar-lift interpretation `(actual_dose - d_lower) * WAS_d_lower`, Assumption 6 caveat. - Section 7: extensions (population-weighted/survey path, composite pretest workflow described accurately as QUG support-infimum test + linearity tests, mass-point design path), related-tutorials cross-links (T01, T02, T14, T17, T18, T19), summary checklist. Drift detection: companion tests/test_t20_had_brand_campaign_drift.py (13 tests, 0.06s, mirrors T19's test-file-only pattern - T19's notebook itself has zero in-notebook asserts). Pins panel composition including sample median, design auto-detection / target / d_lower, overall WAS_d_lower / SE / CI endpoints to one-decimal display, dose mean, n_units, full event-study horizon presence (e=-4..-2, 0..3), per-week post-launch coverage of TRUE_SLOPE=100 and zero coverage at every placebo horizon (|placebo_att| < 0.1). Tight `round(_, 1) == X.X` pins throughout - HAD's analytical SE path is bit-identical regardless of backend env (no Rust kernel involved). Locked DGP seed: MAIN_SEED=87. Documentation integration: - docs/tutorials/README.md: new T20 entry following T18/T19's 5-bullet pattern. - docs/doc-deps.yaml: T20 added to the existing diff_diff/had.py entry; cross-link to docs/practitioner_decision_tree.rst added. - docs/practitioner_decision_tree.rst: `.. tip::` block at the end of `section-no-untreated` (Universal Rollout - landed on main via PR #389) cross-links to T20 for the full walkthrough. - CHANGELOG.md: new ### Added bullet under [Unreleased]. Out of scope (queued in project_had_followups.md memory): - _handle_had in practitioner.py:_HANDLERS map. - HAD entries in llms-full.txt / choosing_estimator.rst. - Pretest workflow tutorial, weighted/survey HAD tutorial, mass-point design demo. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 8e1282b commit 846fd8f

6 files changed

Lines changed: 735 additions & 0 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1010
### Added
1111
- **HAD `trends_lin=True` linear-trend detrending mode** on `HeterogeneousAdoptionDiD.fit(aggregate="event_study")`, `joint_pretrends_test`, and `joint_homogeneity_test`. Mirrors R `DIDHAD::did_had(..., trends_lin=TRUE)` (paper Eq. 17 / Eq. 18 / page 32 joint-Stute homogeneity-with-trends). Per-group linear-trend slope estimated as `Y[g, F-1] - Y[g, F-2]` and applied as `(t - base) × slope` adjustment to per-event-time outcome evolutions. Requires F ≥ 3 (panel must contain F-2). The "consumed" placebo at our event-time `e=-2` is auto-dropped (R reduces max placebo lag by 1 with the same effect). Mutually exclusive with survey weighting (`survey_design` / `survey` / `weights`): raises `NotImplementedError` per `feedback_per_method_survey_element_contract` (weighted slope estimator not derived from paper; tracked in TODO.md as a follow-up). Bit-exact backcompat for `trends_lin=False` (default). Patch-level (additive keyword-only kwarg).
1212
- **HAD R-package end-to-end parity test** vs `DIDHAD` v2.0.0 (`Credible-Answers/did_had`) on the **`design="continuous_at_zero"` (Design 1') surface**. New parity fixture `benchmarks/data/did_had_golden.json` generated by `benchmarks/R/generate_did_had_golden.R` covers 3 paper-derived synthetic DGPs (Uniform, Beta(2,2), Beta(0.5,1)) × 5 method combinations (overall, event-study, placebo, yatchew, trends_lin). The harness explicitly forces `HeterogeneousAdoptionDiD(design="continuous_at_zero")` because R `did_had` always evaluates the local-linear at `d=0` regardless of dose distribution; our default `design="auto"` may legitimately choose `continuous_near_d_lower` or `mass_point` on dose distributions with boundary density bounded away from zero (e.g., Beta(2,2)) and thereby diverge from R numerically — that divergence is methodologically defensible but out of scope for this parity test. Python parity test `tests/test_did_had_parity.py` asserts point estimate / SE / CI bounds at `atol=1e-8` and Yatchew T-stat at `atol=1e-10` after a documented `× G/(G-1)` finite-sample convention shift. Two intentional convention deviations from R, documented in `docs/methodology/REGISTRY.md`: (a) we report the bias-corrected point estimate (modern CCF 2018 convention; R's `Estimate` column reports the conventional estimate with the bias-corrected CI separately — our `att` matches R's CI midpoint); (b) Yatchew uses paper Appendix E's literal (1/G) variance-denominator convention while R uses base-R `var()`'s (1/(N-1)) sample-variance convention (parity is bit-exact after the `× G/(G-1)` shift). Yatchew on placebos with R's mean-independence null (`order=0`) is not yet exposed in our `yatchew_hr_test` (we currently only support the linearity null) and is skipped in the parity test; tracked as TODO follow-up.
13+
- **Tutorial 20: HAD for National Brand Campaign with Regional Spend Intensity** (`docs/tutorials/20_had_brand_campaign.ipynb`) — end-to-end practitioner walkthrough for `HeterogeneousAdoptionDiD` on a 60-DMA panel where every market is treated at a different dose level and no never-treated unit exists; comparison comes from dose variation across markets, not from an untreated holdout. The DGP uses Uniform[\$5K, \$50K] regional add-on spend per DMA (every DMA participates, no DMA at exactly \$0), so `design="auto"` resolves to `continuous_near_d_lower` (Design 1) with target `WAS_d_lower` — interpreted as the average per-dollar marginal effect of regional spend above the lightest-touch DMA's spend (`d_lower` ≈ \$5K). Covers the headline `WAS_d_lower` fit on a 2-period collapse, the multi-week event study with per-week pointwise CIs and pre-launch placebos, and a stakeholder communication template that flags the Assumption 5/6 caveat (non-testable local-linearity at the boundary). Companion drift-test file `tests/test_t20_had_brand_campaign_drift.py` (13 tests pinning panel composition / sample median, design auto-detection / target / `d_lower`, overall `WAS_d_lower`, CI endpoints, dose mean, n_units, full event-study horizon presence, and per-horizon coverage). T20 wired into the existing `had.py` entry in `docs/doc-deps.yaml`; cross-link added from `docs/practitioner_decision_tree.rst` § "Universal Rollout (No Untreated Markets)" via a `.. tip::` block.
1314

1415
### Changed
1516
- **Rust dependency upgrades**: bumped `rand` 0.8 → 0.10 and `rand_xoshiro` 0.6 → 0.8 in the Rust backend (the two crates are coupled through `rand_core` and must move together). MSRV bumped from Rust 1.84 → 1.85 to satisfy the new dependency requirements. Three call sites in `rust/src/bootstrap.rs` updated for the `rand 0.9` API rename: `gen::<bool>()` → `random::<bool>()`, `gen::<f64>()` → `random::<f64>()`, `gen_range(0..6)` → `random_range(0..6)`. **Webb wild bootstrap byte stream shifted** as a side effect: `rand 0.9` reworked the internal algorithm for `random_range` (improved rejection sampling), so `Xoshiro256PlusPlus::seed_from_u64(seed)` followed by `random_range(0..6)` consumes RNG bytes differently than the old `gen_range(0..6)` did. Distributional properties of Webb weights are unchanged (still uniform over the 6-point support); aggregate inference (SE, p-values, CI) converges to the same values for any reasonable `n_bootstrap`. Rademacher and Mammen byte streams are bit-identical to the prior release. Anyone with a saved Rust+Webb baseline pinning specific seeded results will see different numbers; the regression test suite uses within-build seed-reproducibility (not cross-version baselines) so all internal tests pass unchanged. New regression guard `TestRustBackend::test_bootstrap_weights_bit_identity_snapshot` pins fixed-seed weights for all three weight types, so any future RNG drift fails loudly with a localized error message.

docs/doc-deps.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -371,6 +371,8 @@ sources:
371371
type: methodology
372372
- path: docs/api/had.rst
373373
type: api_reference
374+
- path: docs/tutorials/20_had_brand_campaign.ipynb
375+
type: tutorial
374376
- path: README.md
375377
section: "Estimators (one-line catalog entry)"
376378
type: user_guide
@@ -379,6 +381,10 @@ sources:
379381
- path: diff_diff/guides/llms.txt
380382
section: "Estimators"
381383
type: user_guide
384+
- path: docs/practitioner_decision_tree.rst
385+
section: "Universal Rollout (No Untreated Markets)"
386+
type: user_guide
387+
note: "Tip cross-link to T20 in the no-untreated section"
382388
# Note: llms-full.txt does not yet have a HeterogeneousAdoptionDiD section
383389
# (deferred to TODO.md Phase 5 follow-up); the dependency mapping will be
384390
# added when that section lands.

docs/practitioner_decision_tree.rst

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -310,6 +310,13 @@ identification rests on stronger structural assumptions (Design 1).
310310
:doc:`api/had` for the inference contract (three SE regimes; pointwise CIs;
311311
sup-t bands only on the weighted event-study path).
312312

313+
.. tip::
314+
315+
For a full walkthrough including data setup, the design auto-detection
316+
diagnostic, the multi-week event study, and a stakeholder communication
317+
template, see `Tutorial 20: HAD for National Brand Campaign with Regional
318+
Spend Intensity <tutorials/20_had_brand_campaign.ipynb>`_.
319+
313320

314321
.. _section-few-markets:
315322

0 commit comments

Comments
 (0)