|
2 | 2 | "cells": [ |
3 | 3 | { |
4 | 4 | "cell_type": "markdown", |
5 | | - "id": "02f317b7", |
| 5 | + "id": "6623db00", |
6 | 6 | "metadata": {}, |
7 | 7 | "source": [ |
8 | 8 | "# Tutorial 21: HAD Pre-test Workflow - Did the Brand Campaign Satisfy the Identifying Assumptions?\n", |
|
14 | 14 | }, |
15 | 15 | { |
16 | 16 | "cell_type": "markdown", |
17 | | - "id": "47b10255", |
| 17 | + "id": "baf69855", |
18 | 18 | "metadata": {}, |
19 | 19 | "source": [ |
20 | 20 | "## 1. The Pre-test Battery\n", |
|
31 | 31 | }, |
32 | 32 | { |
33 | 33 | "cell_type": "markdown", |
34 | | - "id": "b39cfcc4", |
| 34 | + "id": "587dd5ae", |
35 | 35 | "metadata": {}, |
36 | 36 | "source": [ |
37 | 37 | "## 2. The Panel\n", |
|
42 | 42 | { |
43 | 43 | "cell_type": "code", |
44 | 44 | "execution_count": 1, |
45 | | - "id": "e7d08b12", |
| 45 | + "id": "4169ccef", |
46 | 46 | "metadata": { |
47 | 47 | "execution": { |
48 | | - "iopub.execute_input": "2026-05-09T23:26:36.985852Z", |
49 | | - "iopub.status.busy": "2026-05-09T23:26:36.985618Z", |
50 | | - "iopub.status.idle": "2026-05-09T23:26:37.598610Z", |
51 | | - "shell.execute_reply": "2026-05-09T23:26:37.598289Z" |
| 48 | + "iopub.execute_input": "2026-05-09T23:37:56.569904Z", |
| 49 | + "iopub.status.busy": "2026-05-09T23:37:56.569787Z", |
| 50 | + "iopub.status.idle": "2026-05-09T23:37:57.370321Z", |
| 51 | + "shell.execute_reply": "2026-05-09T23:37:57.370029Z" |
52 | 52 | } |
53 | 53 | }, |
54 | 54 | "outputs": [ |
|
116 | 116 | }, |
117 | 117 | { |
118 | 118 | "cell_type": "markdown", |
119 | | - "id": "3a9b551b", |
| 119 | + "id": "c21196bd", |
120 | 120 | "metadata": {}, |
121 | 121 | "source": [ |
122 | 122 | "## 3. Step 1: The Overall Workflow (Two-Period Path)\n", |
|
129 | 129 | { |
130 | 130 | "cell_type": "code", |
131 | 131 | "execution_count": 2, |
132 | | - "id": "b4057d6a", |
| 132 | + "id": "f57f8c97", |
133 | 133 | "metadata": { |
134 | 134 | "execution": { |
135 | | - "iopub.execute_input": "2026-05-09T23:26:37.599845Z", |
136 | | - "iopub.status.busy": "2026-05-09T23:26:37.599739Z", |
137 | | - "iopub.status.idle": "2026-05-09T23:26:37.634021Z", |
138 | | - "shell.execute_reply": "2026-05-09T23:26:37.633738Z" |
| 135 | + "iopub.execute_input": "2026-05-09T23:37:57.371617Z", |
| 136 | + "iopub.status.busy": "2026-05-09T23:37:57.371488Z", |
| 137 | + "iopub.status.idle": "2026-05-09T23:37:57.410189Z", |
| 138 | + "shell.execute_reply": "2026-05-09T23:37:57.409927Z" |
139 | 139 | } |
140 | 140 | }, |
141 | 141 | "outputs": [ |
|
188 | 188 | }, |
189 | 189 | { |
190 | 190 | "cell_type": "markdown", |
191 | | - "id": "8994fa7c", |
| 191 | + "id": "a37bb4f5", |
192 | 192 | "metadata": {}, |
193 | 193 | "source": [ |
194 | 194 | "**Reading the overall verdict.** Three things to note.\n", |
|
203 | 203 | { |
204 | 204 | "cell_type": "code", |
205 | 205 | "execution_count": 3, |
206 | | - "id": "89e549ef", |
| 206 | + "id": "78aaa722", |
207 | 207 | "metadata": { |
208 | 208 | "execution": { |
209 | | - "iopub.execute_input": "2026-05-09T23:26:37.635153Z", |
210 | | - "iopub.status.busy": "2026-05-09T23:26:37.635075Z", |
211 | | - "iopub.status.idle": "2026-05-09T23:26:37.636869Z", |
212 | | - "shell.execute_reply": "2026-05-09T23:26:37.636643Z" |
| 209 | + "iopub.execute_input": "2026-05-09T23:37:57.411584Z", |
| 210 | + "iopub.status.busy": "2026-05-09T23:37:57.411480Z", |
| 211 | + "iopub.status.idle": "2026-05-09T23:37:57.413454Z", |
| 212 | + "shell.execute_reply": "2026-05-09T23:37:57.413187Z" |
213 | 213 | } |
214 | 214 | }, |
215 | 215 | "outputs": [ |
|
269 | 269 | }, |
270 | 270 | { |
271 | 271 | "cell_type": "markdown", |
272 | | - "id": "892978cd", |
| 272 | + "id": "aaa21a26", |
273 | 273 | "metadata": {}, |
274 | 274 | "source": [ |
275 | 275 | "A note on the Yatchew row. The `T_hr` statistic is **very large and negative** (~-35,000). That looks alarming but is correct here: under perfectly linear dose-response with very heterogeneous doses (Uniform[\\$0.01K, \\$50K]) and 60 sorted-by-dose units, the differencing variance `sigma2_diff` (which captures the squared gap between adjacent-by-dose units' `dy` values) is much larger than the OLS residual variance `sigma2_lin`. The formula `T_hr = sqrt(G) * (sigma2_lin - sigma2_diff) / sigma2_W` then goes massively negative, p-value rounds to 1.0, and we comfortably fail to reject linearity. (For a different way to look at this same test, see the Yatchew side panel later in the notebook.)\n" |
276 | 276 | ] |
277 | 277 | }, |
278 | 278 | { |
279 | 279 | "cell_type": "markdown", |
280 | | - "id": "461e877c", |
| 280 | + "id": "09b0f2a3", |
281 | 281 | "metadata": {}, |
282 | 282 | "source": [ |
283 | 283 | "## 4. Step 2: Upgrade to the Event-Study Workflow\n", |
|
296 | 296 | { |
297 | 297 | "cell_type": "code", |
298 | 298 | "execution_count": 4, |
299 | | - "id": "23b947ad", |
| 299 | + "id": "d94b8cbf", |
300 | 300 | "metadata": { |
301 | 301 | "execution": { |
302 | | - "iopub.execute_input": "2026-05-09T23:26:37.637903Z", |
303 | | - "iopub.status.busy": "2026-05-09T23:26:37.637825Z", |
304 | | - "iopub.status.idle": "2026-05-09T23:26:37.760056Z", |
305 | | - "shell.execute_reply": "2026-05-09T23:26:37.759776Z" |
| 302 | + "iopub.execute_input": "2026-05-09T23:37:57.414542Z", |
| 303 | + "iopub.status.busy": "2026-05-09T23:37:57.414461Z", |
| 304 | + "iopub.status.idle": "2026-05-09T23:37:57.539317Z", |
| 305 | + "shell.execute_reply": "2026-05-09T23:37:57.539034Z" |
306 | 306 | } |
307 | 307 | }, |
308 | 308 | "outputs": [ |
|
342 | 342 | }, |
343 | 343 | { |
344 | 344 | "cell_type": "markdown", |
345 | | - "id": "78d6b00d", |
| 345 | + "id": "ebb7378f", |
346 | 346 | "metadata": {}, |
347 | 347 | "source": [ |
348 | 348 | "**Reading the event-study verdict.** Now the verdict reads `\"QUG, joint pre-trends, and joint linearity diagnostics fail-to-reject (TWFE admissible under Section 4 assumptions)\"`. The `\"deferred\"` caveat from the overall path is gone - all three paper steps closed jointly. The structural fields confirm: `pretrends_joint` and `homogeneity_joint` are both populated.\n", |
|
353 | 353 | { |
354 | 354 | "cell_type": "code", |
355 | 355 | "execution_count": 5, |
356 | | - "id": "dfaf3133", |
| 356 | + "id": "4c0f47d0", |
357 | 357 | "metadata": { |
358 | 358 | "execution": { |
359 | | - "iopub.execute_input": "2026-05-09T23:26:37.761340Z", |
360 | | - "iopub.status.busy": "2026-05-09T23:26:37.761247Z", |
361 | | - "iopub.status.idle": "2026-05-09T23:26:37.763147Z", |
362 | | - "shell.execute_reply": "2026-05-09T23:26:37.762905Z" |
| 359 | + "iopub.execute_input": "2026-05-09T23:37:57.540476Z", |
| 360 | + "iopub.status.busy": "2026-05-09T23:37:57.540385Z", |
| 361 | + "iopub.status.idle": "2026-05-09T23:37:57.542348Z", |
| 362 | + "shell.execute_reply": "2026-05-09T23:37:57.542100Z" |
363 | 363 | } |
364 | 364 | }, |
365 | 365 | "outputs": [ |
|
432 | 432 | }, |
433 | 433 | { |
434 | 434 | "cell_type": "markdown", |
435 | | - "id": "acab854c", |
| 435 | + "id": "f28a7820", |
436 | 436 | "metadata": {}, |
437 | 437 | "source": [ |
438 | 438 | "The pre-trends p-value (~0.07) sits close to the conventional alpha = 0.05 threshold - the test is not vacuous, it is informative. It is consistent with parallel pre-trends but not by a wide margin. In a real analysis this would warrant a closer look at the per-horizon CvM contributions (visible in `per_horizon_stats`) and possibly a Pierce-Schott-style linear-trend detrending via `trends_lin=True` (an extension we do not demonstrate here; see `did_had_pretest_workflow`'s docstring).\n", |
|
444 | 444 | }, |
445 | 445 | { |
446 | 446 | "cell_type": "markdown", |
447 | | - "id": "ba3a0c3c", |
| 447 | + "id": "b805e082", |
448 | 448 | "metadata": {}, |
449 | 449 | "source": [ |
450 | 450 | "## 5. Side Panel: Yatchew-HR Null Modes\n", |
451 | 451 | "\n", |
452 | 452 | "The Yatchew-HR test exposes two `null=` modes (the second was added in 2026-04 for parity with the R `YatchewTest` package).\n", |
453 | 453 | "\n", |
454 | 454 | "- `null=\"linearity\"` (default; paper Theorem 7): tests `H0: E[dY | D]` is linear in `D`. Residuals come from OLS `dy ~ 1 + d`. This is what `did_had_pretest_workflow` calls under the hood.\n", |
455 | | - "- `null=\"mean_independence\"` (PR #400, 2026-04, Phase 4 R-parity): tests the stricter `H0: E[dY | D] = E[dY]`, i.e. `dY` is mean-independent of `D`. Residuals come from intercept-only OLS `dy ~ 1`. Mirrors R `YatchewTest::yatchew_test(order=0)`.\n", |
| 455 | + "- `null=\"mean_independence\"` (added 2026-04-26 in PR #397, Phase 4 R-parity): tests the stricter `H0: E[dY | D] = E[dY]`, i.e. `dY` is mean-independent of `D`. Residuals come from intercept-only OLS `dy ~ 1`. Mirrors R `YatchewTest::yatchew_test(order=0)`.\n", |
456 | 456 | "\n", |
457 | 457 | "The mean-independence mode is typically used on **placebo (pre-treatment) data** to test parallel pre-trends as a non-parametric mean-independence assertion. Below we construct an illustrative input - the within-pre-period first-difference `dy = Y[week=4] - Y[week=3]` paired with each DMA's actual post-period dose - and run both modes side by side. Both should fail to reject on this clean linear DGP; the contrast is in the residual structure.\n" |
458 | 458 | ] |
459 | 459 | }, |
460 | 460 | { |
461 | 461 | "cell_type": "code", |
462 | 462 | "execution_count": 6, |
463 | | - "id": "319a1c0c", |
| 463 | + "id": "63fa34db", |
464 | 464 | "metadata": { |
465 | 465 | "execution": { |
466 | | - "iopub.execute_input": "2026-05-09T23:26:37.764116Z", |
467 | | - "iopub.status.busy": "2026-05-09T23:26:37.764040Z", |
468 | | - "iopub.status.idle": "2026-05-09T23:26:37.768553Z", |
469 | | - "shell.execute_reply": "2026-05-09T23:26:37.768342Z" |
| 466 | + "iopub.execute_input": "2026-05-09T23:37:57.543368Z", |
| 467 | + "iopub.status.busy": "2026-05-09T23:37:57.543298Z", |
| 468 | + "iopub.status.idle": "2026-05-09T23:37:57.547774Z", |
| 469 | + "shell.execute_reply": "2026-05-09T23:37:57.547551Z" |
470 | 470 | } |
471 | 471 | }, |
472 | 472 | "outputs": [ |
|
528 | 528 | }, |
529 | 529 | { |
530 | 530 | "cell_type": "markdown", |
531 | | - "id": "d142244a", |
| 531 | + "id": "da899c45", |
532 | 532 | "metadata": {}, |
533 | 533 | "source": [ |
534 | 534 | "**Reading the side-panel comparison.**\n", |
|
543 | 543 | }, |
544 | 544 | { |
545 | 545 | "cell_type": "markdown", |
546 | | - "id": "5c5d2b18", |
| 546 | + "id": "c98e7202", |
547 | 547 | "metadata": {}, |
548 | 548 | "source": [ |
549 | 549 | "## 6. Communicating the Validation to Leadership\n", |
|
563 | 563 | }, |
564 | 564 | { |
565 | 565 | "cell_type": "markdown", |
566 | | - "id": "0d0c55b3", |
| 566 | + "id": "56246bf2", |
567 | 567 | "metadata": {}, |
568 | 568 | "source": [ |
569 | 569 | "## 7. Extensions\n", |
|
585 | 585 | }, |
586 | 586 | { |
587 | 587 | "cell_type": "markdown", |
588 | | - "id": "36453f8b", |
| 588 | + "id": "ca588d0c", |
589 | 589 | "metadata": {}, |
590 | 590 | "source": [ |
591 | 591 | "## 8. Summary Checklist\n", |
|
0 commit comments