Skip to content

Commit 62b756a

Browse files
igerberclaude
andcommitted
T21 P3 wording cleanups: align Section 7 + drift docstring with revised tutorial
Two stale shorthand phrasings inconsistent with the revised methodology framing: - Section 7 Extensions: "single Design 1 panel" → "single panel where QUG led the workflow to select the continuous_at_zero (Design 1) identification path" (matches the corrected Section 2 wording). - `test_event_study_pretrends_fails_to_reject` docstring quoted "close to alpha = 0.05 but conclusive"; the user-facing text now says "warrants scrutiny" - update internal docstring to match. No methodology change, no new pins; all 15 drift tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 6c281d1 commit 62b756a

2 files changed

Lines changed: 47 additions & 46 deletions

File tree

docs/tutorials/21_had_pretest_workflow.ipynb

Lines changed: 45 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
"cells": [
33
{
44
"cell_type": "markdown",
5-
"id": "9e25598f",
5+
"id": "2c409551",
66
"metadata": {},
77
"source": [
88
"# Tutorial 21: HAD Pre-test Workflow - Running the Pre-test Diagnostics on the Brand Campaign Panel\n",
@@ -14,7 +14,7 @@
1414
},
1515
{
1616
"cell_type": "markdown",
17-
"id": "0cc1feee",
17+
"id": "1ccaad91",
1818
"metadata": {},
1919
"source": [
2020
"## 1. The Pre-test Battery\n",
@@ -31,7 +31,7 @@
3131
},
3232
{
3333
"cell_type": "markdown",
34-
"id": "9ac9f15b",
34+
"id": "c110fe0e",
3535
"metadata": {},
3636
"source": [
3737
"## 2. The Panel\n",
@@ -42,13 +42,13 @@
4242
{
4343
"cell_type": "code",
4444
"execution_count": 1,
45-
"id": "4ced81a7",
45+
"id": "05269d1c",
4646
"metadata": {
4747
"execution": {
48-
"iopub.execute_input": "2026-05-09T23:46:44.813436Z",
49-
"iopub.status.busy": "2026-05-09T23:46:44.813125Z",
50-
"iopub.status.idle": "2026-05-09T23:46:45.859473Z",
51-
"shell.execute_reply": "2026-05-09T23:46:45.859187Z"
48+
"iopub.execute_input": "2026-05-10T00:17:36.394301Z",
49+
"iopub.status.busy": "2026-05-10T00:17:36.394076Z",
50+
"iopub.status.idle": "2026-05-10T00:17:37.818650Z",
51+
"shell.execute_reply": "2026-05-10T00:17:37.818348Z"
5252
}
5353
},
5454
"outputs": [
@@ -116,7 +116,7 @@
116116
},
117117
{
118118
"cell_type": "markdown",
119-
"id": "53772584",
119+
"id": "91811549",
120120
"metadata": {},
121121
"source": [
122122
"## 3. Step 1: The Overall Workflow (Two-Period Path)\n",
@@ -129,13 +129,13 @@
129129
{
130130
"cell_type": "code",
131131
"execution_count": 2,
132-
"id": "1d7d1a0e",
132+
"id": "cbda5c0c",
133133
"metadata": {
134134
"execution": {
135-
"iopub.execute_input": "2026-05-09T23:46:45.860769Z",
136-
"iopub.status.busy": "2026-05-09T23:46:45.860646Z",
137-
"iopub.status.idle": "2026-05-09T23:46:45.902629Z",
138-
"shell.execute_reply": "2026-05-09T23:46:45.902302Z"
135+
"iopub.execute_input": "2026-05-10T00:17:37.819909Z",
136+
"iopub.status.busy": "2026-05-10T00:17:37.819802Z",
137+
"iopub.status.idle": "2026-05-10T00:17:37.858844Z",
138+
"shell.execute_reply": "2026-05-10T00:17:37.858574Z"
139139
}
140140
},
141141
"outputs": [
@@ -188,7 +188,7 @@
188188
},
189189
{
190190
"cell_type": "markdown",
191-
"id": "bbc73e9e",
191+
"id": "9452bc09",
192192
"metadata": {},
193193
"source": [
194194
"**Reading the overall verdict.** Three things to note.\n",
@@ -203,13 +203,13 @@
203203
{
204204
"cell_type": "code",
205205
"execution_count": 3,
206-
"id": "d009ea15",
206+
"id": "7dca161a",
207207
"metadata": {
208208
"execution": {
209-
"iopub.execute_input": "2026-05-09T23:46:45.904054Z",
210-
"iopub.status.busy": "2026-05-09T23:46:45.903927Z",
211-
"iopub.status.idle": "2026-05-09T23:46:45.906185Z",
212-
"shell.execute_reply": "2026-05-09T23:46:45.905937Z"
209+
"iopub.execute_input": "2026-05-10T00:17:37.860034Z",
210+
"iopub.status.busy": "2026-05-10T00:17:37.859953Z",
211+
"iopub.status.idle": "2026-05-10T00:17:37.861749Z",
212+
"shell.execute_reply": "2026-05-10T00:17:37.861541Z"
213213
}
214214
},
215215
"outputs": [
@@ -269,15 +269,15 @@
269269
},
270270
{
271271
"cell_type": "markdown",
272-
"id": "d6258552",
272+
"id": "bb4d7ef5",
273273
"metadata": {},
274274
"source": [
275275
"A note on the Yatchew row. The `T_hr` statistic is **very large and negative** (~-35,000). That looks alarming but is correct here: under perfectly linear dose-response with very heterogeneous doses (Uniform[\\$0.01K, \\$50K]) and 60 sorted-by-dose units, the differencing variance `sigma2_diff` (which captures the squared gap between adjacent-by-dose units' `dy` values) is much larger than the OLS residual variance `sigma2_lin`. The formula `T_hr = sqrt(G) * (sigma2_lin - sigma2_diff) / sigma2_W` then goes massively negative, p-value rounds to 1.0, and we comfortably fail to reject linearity. (For a different way to look at this same test, see the Yatchew side panel later in the notebook.)\n"
276276
]
277277
},
278278
{
279279
"cell_type": "markdown",
280-
"id": "9b25fada",
280+
"id": "0bb3c4e3",
281281
"metadata": {},
282282
"source": [
283283
"## 4. Step 2: Upgrade to the Event-Study Workflow\n",
@@ -296,13 +296,13 @@
296296
{
297297
"cell_type": "code",
298298
"execution_count": 4,
299-
"id": "6dd7f0f3",
299+
"id": "e4903c58",
300300
"metadata": {
301301
"execution": {
302-
"iopub.execute_input": "2026-05-09T23:46:45.907227Z",
303-
"iopub.status.busy": "2026-05-09T23:46:45.907141Z",
304-
"iopub.status.idle": "2026-05-09T23:46:46.040067Z",
305-
"shell.execute_reply": "2026-05-09T23:46:46.039690Z"
302+
"iopub.execute_input": "2026-05-10T00:17:37.862773Z",
303+
"iopub.status.busy": "2026-05-10T00:17:37.862692Z",
304+
"iopub.status.idle": "2026-05-10T00:17:37.988346Z",
305+
"shell.execute_reply": "2026-05-10T00:17:37.988066Z"
306306
}
307307
},
308308
"outputs": [
@@ -342,7 +342,7 @@
342342
},
343343
{
344344
"cell_type": "markdown",
345-
"id": "5f12d7aa",
345+
"id": "b820c289",
346346
"metadata": {},
347347
"source": [
348348
"**Reading the event-study verdict.** Now the verdict reads `\"QUG, joint pre-trends, and joint linearity diagnostics fail-to-reject (TWFE admissible under Section 4 assumptions)\"`. The `\"deferred\"` caveat from the overall path is gone because the joint pre-trends and joint homogeneity diagnostics now ran. The structural fields confirm: `pretrends_joint` and `homogeneity_joint` are both populated.\n",
@@ -355,13 +355,13 @@
355355
{
356356
"cell_type": "code",
357357
"execution_count": 5,
358-
"id": "cfaa750b",
358+
"id": "cd1d2dde",
359359
"metadata": {
360360
"execution": {
361-
"iopub.execute_input": "2026-05-09T23:46:46.041790Z",
362-
"iopub.status.busy": "2026-05-09T23:46:46.041665Z",
363-
"iopub.status.idle": "2026-05-09T23:46:46.043716Z",
364-
"shell.execute_reply": "2026-05-09T23:46:46.043421Z"
361+
"iopub.execute_input": "2026-05-10T00:17:37.989443Z",
362+
"iopub.status.busy": "2026-05-10T00:17:37.989364Z",
363+
"iopub.status.idle": "2026-05-10T00:17:37.991250Z",
364+
"shell.execute_reply": "2026-05-10T00:17:37.990991Z"
365365
}
366366
},
367367
"outputs": [
@@ -434,7 +434,7 @@
434434
},
435435
{
436436
"cell_type": "markdown",
437-
"id": "b95cbac1",
437+
"id": "072e39d8",
438438
"metadata": {},
439439
"source": [
440440
"The pre-trends p-value (~0.07) sits close to the conventional alpha = 0.05 threshold. The test does not reject at alpha = 0.05, but the near-threshold p-value warrants scrutiny - the diagnostic is not failing in a clearly-far-from-rejection regime. In a real analysis this would warrant a closer look at the per-horizon CvM contributions (visible in `per_horizon_stats`) and possibly a Pierce-Schott-style linear-trend detrending via `trends_lin=True` (an extension we do not demonstrate here; see `did_had_pretest_workflow`'s docstring).\n",
@@ -446,7 +446,7 @@
446446
},
447447
{
448448
"cell_type": "markdown",
449-
"id": "c0d6ddbb",
449+
"id": "bba51a15",
450450
"metadata": {},
451451
"source": [
452452
"## 5. Side Panel: Yatchew-HR Null Modes\n",
@@ -462,13 +462,13 @@
462462
{
463463
"cell_type": "code",
464464
"execution_count": 6,
465-
"id": "c231b096",
465+
"id": "d0d4807d",
466466
"metadata": {
467467
"execution": {
468-
"iopub.execute_input": "2026-05-09T23:46:46.045080Z",
469-
"iopub.status.busy": "2026-05-09T23:46:46.044960Z",
470-
"iopub.status.idle": "2026-05-09T23:46:46.050811Z",
471-
"shell.execute_reply": "2026-05-09T23:46:46.050511Z"
468+
"iopub.execute_input": "2026-05-10T00:17:37.992213Z",
469+
"iopub.status.busy": "2026-05-10T00:17:37.992138Z",
470+
"iopub.status.idle": "2026-05-10T00:17:37.996905Z",
471+
"shell.execute_reply": "2026-05-10T00:17:37.996663Z"
472472
}
473473
},
474474
"outputs": [
@@ -530,7 +530,7 @@
530530
},
531531
{
532532
"cell_type": "markdown",
533-
"id": "f0a34622",
533+
"id": "45254f92",
534534
"metadata": {},
535535
"source": [
536536
"**Reading the side-panel comparison.**\n",
@@ -545,7 +545,7 @@
545545
},
546546
{
547547
"cell_type": "markdown",
548-
"id": "fa7bcd99",
548+
"id": "6bdc1f7d",
549549
"metadata": {},
550550
"source": [
551551
"## 6. Communicating the Diagnostics to Leadership\n",
@@ -565,12 +565,12 @@
565565
},
566566
{
567567
"cell_type": "markdown",
568-
"id": "0126ef99",
568+
"id": "d866da6c",
569569
"metadata": {},
570570
"source": [
571571
"## 7. Extensions\n",
572572
"\n",
573-
"This tutorial covered the composite pre-test workflow on a single Design 1 panel. A few directions we did not exercise here:\n",
573+
"This tutorial covered the composite pre-test workflow on a single panel where QUG led the workflow to select the `continuous_at_zero` (Design 1) identification path. A few directions we did not exercise here:\n",
574574
"\n",
575575
"- **Survey-weighted / population-weighted inference** - HAD's pre-test workflow accepts `survey_design=` (or the deprecated `survey=` / `weights=` aliases) for design-based inference. The QUG step is permanently deferred under survey weighting (extreme-value theory under complex sampling is not a settled toolkit); the linearity family runs with PSU-level Mammen multiplier bootstrap (Stute and joint variants) and weighted OLS + weighted variance components (Yatchew). A follow-up tutorial covers this path end-to-end.\n",
576576
"- **`trends_lin=True` (Pierce-Schott Eq 17 / 18 detrending)** - mirrors R `DIDHAD::did_had(..., trends_lin=TRUE)`. Forwards into both joint pre-trends and joint homogeneity wrappers; consumes the placebo at `base_period - 1` and skips Step 2 if no earlier placebo survives the drop. Useful when you suspect linear time trends correlated with dose but want to keep the joint-Stute machinery.\n",
@@ -587,7 +587,7 @@
587587
},
588588
{
589589
"cell_type": "markdown",
590-
"id": "cad9c1d7",
590+
"id": "0105ae26",
591591
"metadata": {},
592592
"source": [
593593
"## 8. Summary Checklist\n",

tests/test_t21_had_pretest_workflow_drift.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -270,7 +270,8 @@ def test_event_study_homogeneity_horizons_correct(event_study_report):
270270

271271
def test_event_study_pretrends_fails_to_reject(event_study_report):
272272
"""Section 4 narrative quotes the pre-trends p-value as 'close to
273-
alpha = 0.05 but conclusive' (~0.07 from numbers.json). Use binary
273+
alpha = 0.05 ... warrants scrutiny' (~0.07 from numbers.json) -
274+
non-rejection is not a clean pass at this margin. Use binary
274275
fail-to-reject + a wide abs tolerance band - bootstrap p-values
275276
near alpha are the most sensitive to RNG path differences."""
276277
pj = event_study_report.pretrends_joint

0 commit comments

Comments
 (0)