Skip to content

Improve app_wait_for stability windows and timeout diagnostics #745

@shaun0927

Description

@shaun0927

#745 Improve app_wait_for stability windows and timeout diagnostics

Why OpenSafari should reflect this

WebdriverIO and Playwright reduce flakiness with auto-wait semantics and detailed timeout diagnostics. OpenSafari already has app_wait_for, but timeout failures currently return only the query, elapsed time, and poll count. For native and Flutter apps, reviewers need to know whether an element was absent, present-but-hidden, present-but-disabled, unstable during animation, or found under a mismatched app context.

This should be reflected because OpenSafari drives real simulator UI via AX/SimulatorKit. Better wait diagnostics improve stability without changing tap/type behavior.

Scope / how to implement

  • Extend app_wait_for with backward-compatible optional diagnostics:
    • stable_ms: require the condition to hold continuously before success.
    • timeout response includes last match count and a small bounded sample of nearest/last candidates with role, label, identifier, visible/enabled, frame, and path.
    • include context metadata already produced by native context classification.
  • Preserve existing defaults: condition=exists, timeout 10s, interval 500ms, no stability requirement unless requested.
  • Add unit tests around condition/stability evaluation using pure helpers.

Decisions needed before implementation

  1. Maximum number of candidate samples in timeout output (suggest 5).
  2. Whether stable_ms should apply to not_exists as well as positive conditions.
  3. Whether candidate sampling should query broad tree only on timeout or during every poll.
  4. Naming: stable_ms vs stability_ms.

Success criteria

  • Existing app_wait_for calls remain compatible.
  • Optional stability window prevents success until the condition has held for the requested duration.
  • Timeout JSON explains why the condition was not met using bounded non-sensitive AX metadata.
  • Unit tests cover positive condition, not_exists, visible/enabled failures, and candidate truncation.

Post-merge OpenSafari live validation

  • Against a booted simulator app, wait for a visible button with stable_ms=500 and confirm success includes elapsed/polls/stable duration.
  • Wait for a nonexistent label and confirm timeout output includes query, polls, context, and bounded candidate evidence.
  • Confirm a normal existing app_wait_for call still returns the previous core fields.

Ambiguity review

This issue does not add a new locator system or action retry engine. It only enriches the existing wait tool with opt-in stability and diagnostics.

Direction and necessity review (2026 OSS comparison)

  • Aligned: yes — improves existing native/Flutter wait behavior while preserving direct AX/SimulatorKit automation.
  • Necessary: yes — wait timeouts are a primary source of flakiness and need actionable evidence.
  • Minimal first PR: add stable_ms, bounded timeout samples, and pure unit tests; no locator rewrite or action retry engine.

Metadata

Metadata

Assignees

No one assigned

    Labels

    automation-roadmapOpenSafari automation roadmap work itemsenhancementNew feature or requestflutterFlutter app QA supportreliabilityReliability and stability

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions