|
2 | 2 | "cells": [ |
3 | 3 | { |
4 | 4 | "cell_type": "markdown", |
5 | | - "id": "9e25598f", |
| 5 | + "id": "2c409551", |
6 | 6 | "metadata": {}, |
7 | 7 | "source": [ |
8 | 8 | "# Tutorial 21: HAD Pre-test Workflow - Running the Pre-test Diagnostics on the Brand Campaign Panel\n", |
|
14 | 14 | }, |
15 | 15 | { |
16 | 16 | "cell_type": "markdown", |
17 | | - "id": "0cc1feee", |
| 17 | + "id": "1ccaad91", |
18 | 18 | "metadata": {}, |
19 | 19 | "source": [ |
20 | 20 | "## 1. The Pre-test Battery\n", |
|
31 | 31 | }, |
32 | 32 | { |
33 | 33 | "cell_type": "markdown", |
34 | | - "id": "9ac9f15b", |
| 34 | + "id": "c110fe0e", |
35 | 35 | "metadata": {}, |
36 | 36 | "source": [ |
37 | 37 | "## 2. The Panel\n", |
|
42 | 42 | { |
43 | 43 | "cell_type": "code", |
44 | 44 | "execution_count": 1, |
45 | | - "id": "4ced81a7", |
| 45 | + "id": "05269d1c", |
46 | 46 | "metadata": { |
47 | 47 | "execution": { |
48 | | - "iopub.execute_input": "2026-05-09T23:46:44.813436Z", |
49 | | - "iopub.status.busy": "2026-05-09T23:46:44.813125Z", |
50 | | - "iopub.status.idle": "2026-05-09T23:46:45.859473Z", |
51 | | - "shell.execute_reply": "2026-05-09T23:46:45.859187Z" |
| 48 | + "iopub.execute_input": "2026-05-10T00:17:36.394301Z", |
| 49 | + "iopub.status.busy": "2026-05-10T00:17:36.394076Z", |
| 50 | + "iopub.status.idle": "2026-05-10T00:17:37.818650Z", |
| 51 | + "shell.execute_reply": "2026-05-10T00:17:37.818348Z" |
52 | 52 | } |
53 | 53 | }, |
54 | 54 | "outputs": [ |
|
116 | 116 | }, |
117 | 117 | { |
118 | 118 | "cell_type": "markdown", |
119 | | - "id": "53772584", |
| 119 | + "id": "91811549", |
120 | 120 | "metadata": {}, |
121 | 121 | "source": [ |
122 | 122 | "## 3. Step 1: The Overall Workflow (Two-Period Path)\n", |
|
129 | 129 | { |
130 | 130 | "cell_type": "code", |
131 | 131 | "execution_count": 2, |
132 | | - "id": "1d7d1a0e", |
| 132 | + "id": "cbda5c0c", |
133 | 133 | "metadata": { |
134 | 134 | "execution": { |
135 | | - "iopub.execute_input": "2026-05-09T23:46:45.860769Z", |
136 | | - "iopub.status.busy": "2026-05-09T23:46:45.860646Z", |
137 | | - "iopub.status.idle": "2026-05-09T23:46:45.902629Z", |
138 | | - "shell.execute_reply": "2026-05-09T23:46:45.902302Z" |
| 135 | + "iopub.execute_input": "2026-05-10T00:17:37.819909Z", |
| 136 | + "iopub.status.busy": "2026-05-10T00:17:37.819802Z", |
| 137 | + "iopub.status.idle": "2026-05-10T00:17:37.858844Z", |
| 138 | + "shell.execute_reply": "2026-05-10T00:17:37.858574Z" |
139 | 139 | } |
140 | 140 | }, |
141 | 141 | "outputs": [ |
|
188 | 188 | }, |
189 | 189 | { |
190 | 190 | "cell_type": "markdown", |
191 | | - "id": "bbc73e9e", |
| 191 | + "id": "9452bc09", |
192 | 192 | "metadata": {}, |
193 | 193 | "source": [ |
194 | 194 | "**Reading the overall verdict.** Three things to note.\n", |
|
203 | 203 | { |
204 | 204 | "cell_type": "code", |
205 | 205 | "execution_count": 3, |
206 | | - "id": "d009ea15", |
| 206 | + "id": "7dca161a", |
207 | 207 | "metadata": { |
208 | 208 | "execution": { |
209 | | - "iopub.execute_input": "2026-05-09T23:46:45.904054Z", |
210 | | - "iopub.status.busy": "2026-05-09T23:46:45.903927Z", |
211 | | - "iopub.status.idle": "2026-05-09T23:46:45.906185Z", |
212 | | - "shell.execute_reply": "2026-05-09T23:46:45.905937Z" |
| 209 | + "iopub.execute_input": "2026-05-10T00:17:37.860034Z", |
| 210 | + "iopub.status.busy": "2026-05-10T00:17:37.859953Z", |
| 211 | + "iopub.status.idle": "2026-05-10T00:17:37.861749Z", |
| 212 | + "shell.execute_reply": "2026-05-10T00:17:37.861541Z" |
213 | 213 | } |
214 | 214 | }, |
215 | 215 | "outputs": [ |
|
269 | 269 | }, |
270 | 270 | { |
271 | 271 | "cell_type": "markdown", |
272 | | - "id": "d6258552", |
| 272 | + "id": "bb4d7ef5", |
273 | 273 | "metadata": {}, |
274 | 274 | "source": [ |
275 | 275 | "A note on the Yatchew row. The `T_hr` statistic is **very large and negative** (~-35,000). That looks alarming but is correct here: under perfectly linear dose-response with very heterogeneous doses (Uniform[\\$0.01K, \\$50K]) and 60 sorted-by-dose units, the differencing variance `sigma2_diff` (which captures the squared gap between adjacent-by-dose units' `dy` values) is much larger than the OLS residual variance `sigma2_lin`. The formula `T_hr = sqrt(G) * (sigma2_lin - sigma2_diff) / sigma2_W` then goes massively negative, p-value rounds to 1.0, and we comfortably fail to reject linearity. (For a different way to look at this same test, see the Yatchew side panel later in the notebook.)\n" |
276 | 276 | ] |
277 | 277 | }, |
278 | 278 | { |
279 | 279 | "cell_type": "markdown", |
280 | | - "id": "9b25fada", |
| 280 | + "id": "0bb3c4e3", |
281 | 281 | "metadata": {}, |
282 | 282 | "source": [ |
283 | 283 | "## 4. Step 2: Upgrade to the Event-Study Workflow\n", |
|
296 | 296 | { |
297 | 297 | "cell_type": "code", |
298 | 298 | "execution_count": 4, |
299 | | - "id": "6dd7f0f3", |
| 299 | + "id": "e4903c58", |
300 | 300 | "metadata": { |
301 | 301 | "execution": { |
302 | | - "iopub.execute_input": "2026-05-09T23:46:45.907227Z", |
303 | | - "iopub.status.busy": "2026-05-09T23:46:45.907141Z", |
304 | | - "iopub.status.idle": "2026-05-09T23:46:46.040067Z", |
305 | | - "shell.execute_reply": "2026-05-09T23:46:46.039690Z" |
| 302 | + "iopub.execute_input": "2026-05-10T00:17:37.862773Z", |
| 303 | + "iopub.status.busy": "2026-05-10T00:17:37.862692Z", |
| 304 | + "iopub.status.idle": "2026-05-10T00:17:37.988346Z", |
| 305 | + "shell.execute_reply": "2026-05-10T00:17:37.988066Z" |
306 | 306 | } |
307 | 307 | }, |
308 | 308 | "outputs": [ |
|
342 | 342 | }, |
343 | 343 | { |
344 | 344 | "cell_type": "markdown", |
345 | | - "id": "5f12d7aa", |
| 345 | + "id": "b820c289", |
346 | 346 | "metadata": {}, |
347 | 347 | "source": [ |
348 | 348 | "**Reading the event-study verdict.** Now the verdict reads `\"QUG, joint pre-trends, and joint linearity diagnostics fail-to-reject (TWFE admissible under Section 4 assumptions)\"`. The `\"deferred\"` caveat from the overall path is gone because the joint pre-trends and joint homogeneity diagnostics now ran. The structural fields confirm: `pretrends_joint` and `homogeneity_joint` are both populated.\n", |
|
355 | 355 | { |
356 | 356 | "cell_type": "code", |
357 | 357 | "execution_count": 5, |
358 | | - "id": "cfaa750b", |
| 358 | + "id": "cd1d2dde", |
359 | 359 | "metadata": { |
360 | 360 | "execution": { |
361 | | - "iopub.execute_input": "2026-05-09T23:46:46.041790Z", |
362 | | - "iopub.status.busy": "2026-05-09T23:46:46.041665Z", |
363 | | - "iopub.status.idle": "2026-05-09T23:46:46.043716Z", |
364 | | - "shell.execute_reply": "2026-05-09T23:46:46.043421Z" |
| 361 | + "iopub.execute_input": "2026-05-10T00:17:37.989443Z", |
| 362 | + "iopub.status.busy": "2026-05-10T00:17:37.989364Z", |
| 363 | + "iopub.status.idle": "2026-05-10T00:17:37.991250Z", |
| 364 | + "shell.execute_reply": "2026-05-10T00:17:37.990991Z" |
365 | 365 | } |
366 | 366 | }, |
367 | 367 | "outputs": [ |
|
434 | 434 | }, |
435 | 435 | { |
436 | 436 | "cell_type": "markdown", |
437 | | - "id": "b95cbac1", |
| 437 | + "id": "072e39d8", |
438 | 438 | "metadata": {}, |
439 | 439 | "source": [ |
440 | 440 | "The pre-trends p-value (~0.07) sits close to the conventional alpha = 0.05 threshold. The test does not reject at alpha = 0.05, but the near-threshold p-value warrants scrutiny - the diagnostic is not failing in a clearly-far-from-rejection regime. In a real analysis this would warrant a closer look at the per-horizon CvM contributions (visible in `per_horizon_stats`) and possibly a Pierce-Schott-style linear-trend detrending via `trends_lin=True` (an extension we do not demonstrate here; see `did_had_pretest_workflow`'s docstring).\n", |
|
446 | 446 | }, |
447 | 447 | { |
448 | 448 | "cell_type": "markdown", |
449 | | - "id": "c0d6ddbb", |
| 449 | + "id": "bba51a15", |
450 | 450 | "metadata": {}, |
451 | 451 | "source": [ |
452 | 452 | "## 5. Side Panel: Yatchew-HR Null Modes\n", |
|
462 | 462 | { |
463 | 463 | "cell_type": "code", |
464 | 464 | "execution_count": 6, |
465 | | - "id": "c231b096", |
| 465 | + "id": "d0d4807d", |
466 | 466 | "metadata": { |
467 | 467 | "execution": { |
468 | | - "iopub.execute_input": "2026-05-09T23:46:46.045080Z", |
469 | | - "iopub.status.busy": "2026-05-09T23:46:46.044960Z", |
470 | | - "iopub.status.idle": "2026-05-09T23:46:46.050811Z", |
471 | | - "shell.execute_reply": "2026-05-09T23:46:46.050511Z" |
| 468 | + "iopub.execute_input": "2026-05-10T00:17:37.992213Z", |
| 469 | + "iopub.status.busy": "2026-05-10T00:17:37.992138Z", |
| 470 | + "iopub.status.idle": "2026-05-10T00:17:37.996905Z", |
| 471 | + "shell.execute_reply": "2026-05-10T00:17:37.996663Z" |
472 | 472 | } |
473 | 473 | }, |
474 | 474 | "outputs": [ |
|
530 | 530 | }, |
531 | 531 | { |
532 | 532 | "cell_type": "markdown", |
533 | | - "id": "f0a34622", |
| 533 | + "id": "45254f92", |
534 | 534 | "metadata": {}, |
535 | 535 | "source": [ |
536 | 536 | "**Reading the side-panel comparison.**\n", |
|
545 | 545 | }, |
546 | 546 | { |
547 | 547 | "cell_type": "markdown", |
548 | | - "id": "fa7bcd99", |
| 548 | + "id": "6bdc1f7d", |
549 | 549 | "metadata": {}, |
550 | 550 | "source": [ |
551 | 551 | "## 6. Communicating the Diagnostics to Leadership\n", |
|
565 | 565 | }, |
566 | 566 | { |
567 | 567 | "cell_type": "markdown", |
568 | | - "id": "0126ef99", |
| 568 | + "id": "d866da6c", |
569 | 569 | "metadata": {}, |
570 | 570 | "source": [ |
571 | 571 | "## 7. Extensions\n", |
572 | 572 | "\n", |
573 | | - "This tutorial covered the composite pre-test workflow on a single Design 1 panel. A few directions we did not exercise here:\n", |
| 573 | + "This tutorial covered the composite pre-test workflow on a single panel where QUG led the workflow to select the `continuous_at_zero` (Design 1) identification path. A few directions we did not exercise here:\n", |
574 | 574 | "\n", |
575 | 575 | "- **Survey-weighted / population-weighted inference** - HAD's pre-test workflow accepts `survey_design=` (or the deprecated `survey=` / `weights=` aliases) for design-based inference. The QUG step is permanently deferred under survey weighting (extreme-value theory under complex sampling is not a settled toolkit); the linearity family runs with PSU-level Mammen multiplier bootstrap (Stute and joint variants) and weighted OLS + weighted variance components (Yatchew). A follow-up tutorial covers this path end-to-end.\n", |
576 | 576 | "- **`trends_lin=True` (Pierce-Schott Eq 17 / 18 detrending)** - mirrors R `DIDHAD::did_had(..., trends_lin=TRUE)`. Forwards into both joint pre-trends and joint homogeneity wrappers; consumes the placebo at `base_period - 1` and skips Step 2 if no earlier placebo survives the drop. Useful when you suspect linear time trends correlated with dose but want to keep the joint-Stute machinery.\n", |
|
587 | 587 | }, |
588 | 588 | { |
589 | 589 | "cell_type": "markdown", |
590 | | - "id": "cad9c1d7", |
| 590 | + "id": "0105ae26", |
591 | 591 | "metadata": {}, |
592 | 592 | "source": [ |
593 | 593 | "## 8. Summary Checklist\n", |
|
0 commit comments