Skip to content

Commit e00113b

Browse files
igerberclaude
andcommitted
Clarify METHODOLOGY_REVIEW.md: DeltaSD uses proper optimal FLCI
The outstanding concerns and deviations now accurately reflect that: - DeltaSD uses the paper's optimal FLCI (Section 4.1) — not a deviation - DeltaRM uses naive FLCI — documented deviation from R's ARP hybrid - Note format matches project's reviewer-recognized labels Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 652bc5f commit e00113b

1 file changed

Lines changed: 10 additions & 9 deletions

File tree

METHODOLOGY_REVIEW.md

Lines changed: 10 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -677,19 +677,20 @@ variables appear to the left of the `|` separator.
677677
looser Nelder-Mead tolerances.
678678

679679
**Outstanding Concerns:**
680-
- ARP hybrid confidence sets for Delta^RM: infrastructure implemented (`_arp_confidence_set`,
681-
`_enumerate_vertices`, `_compute_arp_test`) but disabled pending calibration of the moment
682-
inequality transformation. Currently uses conservative naive FLCI for RM CIs.
680+
- **Delta^RM CI**: uses naive FLCI (conservative) instead of the paper's ARP conditional/hybrid
681+
confidence sets. ARP infrastructure exists but moment inequality transformation needs
682+
calibration. Tracked in TODO.md.
683683
- R benchmark comparison not yet run (Python benchmark needs API update)
684684
- Combined method uses single M for both SD and RM (DeltaSDRM dataclass has separate M/Mbar)
685685

686686
**Deviations from R's HonestDiD:**
687-
1. **Delta^RM CI**: R uses full ARP conditional/hybrid confidence sets. Python uses naive FLCI
688-
(conservative — wider CIs, valid coverage). ARP implementation exists but needs calibration.
689-
2. **Optimal FLCI**: R uses the same approach (Armstrong & Kolesar 2018). Python implementation
690-
matches the methodology but uses Nelder-Mead optimization vs R's custom solver. Numerical
691-
differences expected at tolerance level.
692-
3. **Base period handling**: Python warns (doesn't error) when CallawaySantAnna results use
687+
1. **Deviation from R:** Delta^RM CIs use naive FLCI (`lb - z*se, ub + z*se`) instead of ARP
688+
conditional/hybrid. Conservative (wider CIs, valid coverage). ARP deferred.
689+
2. **Note:** Delta^SD optimal FLCI matches the paper's Section 4.1 methodology: first-difference
690+
reparameterization, slope weights with sum(w)=1 constraint, bias LP in fd-space, folded
691+
normal (or folded non-central t for survey df). Nelder-Mead optimizer vs R's custom solver
692+
may produce numerical differences at tolerance level.
693+
3. **Note (deviation from R):** Python warns (doesn't error) when CallawaySantAnna results use
693694
`base_period != "universal"`. R's HonestDiD requires universal base period.
694695

695696
---

0 commit comments

Comments
 (0)