Wave D fail-closed HC1/CR1 multiplier on saturated stage-2 designs

When the kept stage-2 design saturates the sample (n_obs == effective_rank
after rank-deficient drops), the HC1 multiplier n/(n-p) and the CR1
multiplier (n-1)/(n-p) are mathematically undefined. The original Wave D
helper used `max(n - p_2, 1)` to clamp the denominator, which silently
fabricated finite multipliers on underdetermined fits — `result.se` and
per-coefficient SEs could stay finite even when only `t_stat`/`p_value`/CI
were NaN-gated via `df_resid=0`. That violates the no-silent-failures
contract.

Fix: when n - p_2 <= 0, return NaN meat with an explicit UserWarning
so the SE surface NaN-propagates consistently with the inference fields.
The Conley path is unaffected (no finite-sample multiplier on that
branch by convention).

Tests: new `test_saturated_design_yields_nan_se_not_finite` in
TestSpilloverDiDWaveDPublicVarianceContract exercises both the HC1 and
CR1 paths on a synthetic n=p_2=4 Psi fixture; asserts NaN meat AND
the saturation warning fires.

Docs: replaced "Wave B MVP limitations" section heading at
docs/api/spillover.rst with "Restrictions and follow-ups" (the
section now describes the shipped Wave D variance + remaining
limitations); updated the SpilloverDiD vs TwoStageDiD comparison table
to label the Conley and cluster rows "(Wave D GMM-corrected sandwich)"
instead of "(via solve_ols at stage 2)".

All 224 spillover tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: igerber

diff_diff/two_stage.py

@@ -208,8 +208,23 @@ def _compute_gmm_corrected_meat(
     # 3. Kernel dispatch.
     if vcov_type == "hc1":
         # K = I_n: meat = Psi' Psi with HC1 finite-sample multiplier.
+        # Fail closed when n - p_2 <= 0 (saturated design — every degree
+        # of freedom consumed by the stage-2 design): the multiplier
+        # n / (n - p_2) is undefined, so NaN-propagate per
+        # `feedback_no_silent_failures` rather than clamping the
+        # denominator and emitting finite SE on an underdetermined fit.
+        if n - p_2 <= 0:
+            warnings.warn(
+                "SpilloverDiD Wave D HC1 sandwich: saturated stage-2 design "
+                f"(n_obs={n}, effective_rank={p_2}, n-p_2={n - p_2} <= 0). "
+                "The HC1 finite-sample multiplier n/(n-p) is undefined. "
+                "Returning NaN meat so downstream inference NaN-propagates.",
+                UserWarning,
+                stacklevel=2,
+            )
+            return np.full((p_2, p_2), np.nan)
         meat_unscaled = Psi.T @ Psi
-        multiplier = n / max(n - p_2, 1)
+        multiplier = n / (n - p_2)
         meat = multiplier * meat_unscaled
     elif vcov_type == "cluster":
         if cluster_ids is None:
@@ -223,16 +238,29 @@ def _compute_gmm_corrected_meat(
         G = len(unique_clusters)
         # Mirror linalg.py:1942 — reject G<2 so the CR1 finite-sample
         # multiplier G/(G-1) doesn't fabricate finite output on a degenerate
-        # one-cluster sample. (Round 1 codex P0 fix.)
+        # one-cluster sample.
         if G < 2:
             raise ValueError(f"Need at least 2 clusters for cluster-robust SEs, got {G}")
+        # Fail closed on saturated design (n - p_2 <= 0). The CR1
+        # multiplier (n-1)/(n-p) is undefined; emitting finite SE here
+        # would be silently wrong.
+        if n - p_2 <= 0:
+            warnings.warn(
+                "SpilloverDiD Wave D CR1 sandwich: saturated stage-2 design "
+                f"(n_obs={n}, effective_rank={p_2}, n-p_2={n - p_2} <= 0). "
+                "The CR1 finite-sample multiplier (n-1)/(n-p) is undefined. "
+                "Returning NaN meat so downstream inference NaN-propagates.",
+                UserWarning,
+                stacklevel=2,
+            )
+            return np.full((p_2, p_2), np.nan)
         S_cluster = np.zeros((G, p_2))
         for j in range(p_2):
             np.add.at(S_cluster[:, j], cluster_indices, Psi[:, j])
         meat_unscaled = S_cluster.T @ S_cluster
         # CR1 finite-sample multiplier: G/(G-1) * (n-1)/(n-p_2). Standard
         # cluster-robust convention (Stata, R `sandwich::vcovCL(type='CR1')`).
-        multiplier = (G / (G - 1)) * ((n - 1) / max(n - p_2, 1))
+        multiplier = (G / (G - 1)) * ((n - 1) / (n - p_2))
         meat = multiplier * meat_unscaled
     elif vcov_type == "conley":
         if conley_coords is None or conley_cutoff_km is None or conley_metric is None:

docs/api/spillover.rst

@@ -160,19 +160,19 @@ Estimator Comparison
      - ``D=0`` (untreated)
      - N/A (single stage)
    * - Conley spatial-HAC SE
-     - Yes (via solve_ols at stage 2)
+     - Yes (Wave D GMM-corrected sandwich)
      - Not yet supported
      - Yes
    * - Cluster-robust SE
-     - Yes (HC1 + CR1 via solve_ols)
+     - Yes (HC1 + CR1, Wave D GMM-corrected sandwich)
      - Yes (GMM sandwich + clusters)
      - Yes
 
-Wave B MVP limitations
-----------------------
+Restrictions and follow-ups
+---------------------------
 
-The current implementation has the following documented limitations,
-planned as follow-up enhancements:
+The current implementation has the following documented restrictions
+and planned follow-up enhancements:
 
 - **Gardner GMM first-stage correction at stage 2** — SHIPPED in Wave D.
   Stage-2 variance now applies the influence-function-based correction

tests/test_spillover.py

@@ -4458,6 +4458,54 @@ def test_single_cluster_sample_raises(self):
             with pytest.raises(ValueError, match="at least 2 clusters"):
                 est.fit(df, outcome="y", unit="unit", time="time", treatment="D")
 
+    def test_saturated_design_yields_nan_se_not_finite(self):
+        """`n_obs == p_2` saturated stage-2 design: HC1 multiplier
+        ``n/(n-p)`` is undefined. Wave D fails closed by returning NaN
+        meat → NaN SE downstream, rather than clamping the denominator
+        to 1 and emitting a finite SE on an underdetermined fit.
+        """
+        from scipy import sparse
+
+        from diff_diff.two_stage import _compute_gmm_corrected_meat
+
+        # Construct a saturated synthetic Psi fixture directly through the
+        # helper (avoids manufacturing a real saturated SpilloverDiD panel,
+        # which is constrained by the validator). n_obs == p_2 == 4.
+        n, p_1, p_2 = 4, 3, 4
+        rng = np.random.default_rng(0)
+        X_1 = sparse.csr_matrix(rng.standard_normal((n, p_1)))
+        X_10 = sparse.csr_matrix(rng.standard_normal((n, p_1)))
+        eps_10 = rng.standard_normal(n)
+        X_2 = rng.standard_normal((n, p_2))
+        eps_2 = rng.standard_normal(n)
+
+        import warnings as _w
+
+        for vmode, kwargs in [
+            ("hc1", {}),
+            ("cluster", {"cluster_ids": np.array([0, 0, 1, 1])}),
+        ]:
+            with _w.catch_warnings(record=True) as caught:
+                _w.simplefilter("always")
+                meat = _compute_gmm_corrected_meat(
+                    X_1_sparse=X_1,
+                    X_10_sparse=X_10,
+                    eps_10=eps_10,
+                    X_2=X_2,
+                    eps_2=eps_2,
+                    vcov_type=vmode,
+                    **kwargs,
+                )
+            assert np.all(np.isnan(meat)), (
+                f"vcov_type={vmode!r} saturated design (n=p_2={n}) returned "
+                f"finite meat instead of NaN: {meat!r}"
+            )
+            saturation_warning_fired = any("saturated" in str(w.message) for w in caught)
+            assert saturation_warning_fired, (
+                f"vcov_type={vmode!r} saturated design did not emit the "
+                f"expected saturation warning"
+            )
+
     def test_classical_vcov_raises_with_clear_message(self):
         """`vcov_type="classical"` raises NotImplementedError upfront with a
         clear remediation message rather than failing deep inside the GMM