Skip to content

test(e2e): isolate iOS acceptance on a fresh host (experiment concluded) - #103

Closed
leon-zym wants to merge 6 commits into
mainfrom
agent/issue-101-isolated-ios-e2e
Closed

test(e2e): isolate iOS acceptance on a fresh host (experiment concluded)#103
leon-zym wants to merge 6 commits into
mainfrom
agent/issue-101-isolated-ios-e2e

Conversation

@leon-zym

@leon-zym leon-zym commented Aug 15, 2026

Copy link
Copy Markdown
Owner

Related to #101.

状态:实验已结束。此 PR 必须保持 Draft,且不得合并。

结论

本 PR 检验“把 iOS acceptance 放到全新宿主,能否稳定消除托管环境长尾”的假设。有效配对样本没有达到采纳门槛:

  • control 组能够进入 UI 自动化,但暴露出文本提交断言可命中仍打开输入框的测试伪阳性;
  • fresh-host 组停在 CoreSimulator 初始化阶段,未能稳定进入 App acceptance;
  • 因此该隔离 seam 没有证明能带来可重复、可接受的稳定性提升。

这不是对产品代码的归因,也不应靠 retry、sleep 或放宽生产 acceptance deadline 来掩盖。不要继续扩样或自动补跑;ADR 0042 继续有效。

已独立提取的可合入修复

不依赖本实验的修复已推送到 agent/issue-101-e2e-failure-identity

  1. 区分 Maestro/CocoaPods 冷启动超时与 PATH 缺失;
  2. 保留命令超时作为 primary error,终止失败仅保留为次级证据;
  3. 文本提交流程先验证编辑面板关闭,避免输入框文本造成伪阳性。

与 main 的完整比较 已通过 pnpm verify

后续决策

要提升 iOS L4 的真实稳定性,需要选择更可靠的执行面,例如专用/更大规格 runner 或设备云。若暂时不能提供该执行面,完整 iOS L4 应作为定时或手动证据,而不是不稳定的 PR 硬门禁。

@leon-zym leon-zym changed the title test(e2e): isolate iOS acceptance on a fresh host test(e2e): isolate iOS acceptance on a fresh host (experiment concluded) Aug 16, 2026

Copy link
Copy Markdown
Owner Author

实验结论已写入 PR 描述:fresh-host 隔离未达到采纳门槛,因此此 PR 保持 Draft 且不得合并。可独立合入的探针、失败身份与文本断言修复已提取到 agent/issue-101-e2e-failure-identity,并通过 pnpm verify

Copy link
Copy Markdown
Owner Author

Closing without merge

This experiment has concluded and did not meet its adoption threshold. It is closed without merge.

@leon-zym leon-zym closed this Aug 16, 2026
leon-zym added a commit that referenced this pull request Aug 16, 2026
## What changed

This PR extracts the independently valid fixes discovered while
evaluating #101 / #103:

- distinguish bounded Maestro and CocoaPods cold-start timeouts from a
missing executable;
- preserve the original command timeout as the primary E2E failure when
process finalization also fails;
- require the text editor panel to close after confirmation, preventing
an input-field false positive in the E2E flow;
- give a redaction-only orchestration test appropriate scheduling
headroom.

## Why

The fresh-host experiment in #103 is not being adopted, but these
corrections improve the truthfulness and diagnosability of the existing
E2E runner without changing its execution plane.

## Scope boundary

- No sealed-package or cross-host orchestration from #103.
- No production acceptance retry, device rebuild, fixed sleep, or broad
timeout expansion.
- No change to the current L4 gate policy; #104 owns that future
decision.

## Validation

```text
pnpm verify
- scenario verification: 104 Scenarios, 165 bindings
- orchestration: 198 passed, 1 platform skip
- app: 412 passed
- render: 85 passed
```

Related to #101 and #104.
@leon-zym
leon-zym deleted the agent/issue-101-isolated-ios-e2e branch August 16, 2026 07:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant