Skip to content

tests: replace timing sleeps with condition-based waits - #1252

Merged
Carter Tinney (cartertinney) merged 15 commits into
mainfrom
agents/unit-test-failure-analysis-python312
Sep 1, 2026
Merged

tests: replace timing sleeps with condition-based waits#1252
Carter Tinney (cartertinney) merged 15 commits into
mainfrom
agents/unit-test-failure-analysis-python312

Conversation

@cartertinney

@cartertinney Carter Tinney (cartertinney) commented Sep 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

What changed

  • add reusable poll_until and async_poll_until fixtures with required explicit timeouts
  • preserve the original 100 ms expectation for single handler operations and use 500 ms only for five-item batches
  • replace post-removal observation sleeps with direct assertions that the handler and runner are cleared before adding new items
  • keep five-second bounds only for deliberately paused cleanup/deadlock-safety orchestration
  • add assert_call_blocks for blocking sync API assertions
  • replace historical Janus teardown sleeps with explicit Queue.aclose()

Why

A Python 3.12 unit-test run observed only 2 of 5 handler invocations after a fixed 100 ms sleep. Positive completion tests now poll for outcomes while retaining meaningful timing bounds. Negative post-removal behavior is verified structurally: the setter does not return until the runner exits, after which both handler and runner references must be None.

Validation

  • full Python 3.12.14 unit suite with coverage: 5437 passed, 6 skipped
  • sleep-free handler-removal cases: 100 repeated runs, 900/900 tests passed
  • refined handler deadlines: 100 repeated runs, 7500/7500 tests passed
  • Black and Ruff passed

@cartertinney Carter Tinney (cartertinney) mentioned this pull request Sep 1, 2026
Merged
@cartertinney
Carter Tinney (cartertinney) changed the base branch from main to ct/stabilize-uv-cache September 1, 2026 13:17
@cartertinney
Carter Tinney (cartertinney) force-pushed the agents/unit-test-failure-analysis-python312 branch from 97a22f5 to a8070b5 Compare September 1, 2026 13:17
Copilot AI reviewed Sep 1, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Replaces timing sleeps with bounded, condition-based synchronization to make unit tests more deterministic.

Changes:

  • Adds reusable polling, thread-running, and blocking-call fixtures.
  • Uses explicit completion waits and Janus queue cleanup.
  • Strengthens handler-removal and alarm synchronization tests.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/conftest.py Adds synchronization fixtures.
tests/unit/helpers.py Defines shared timeout constants.
tests/unit/iothub/test_sync_inbox.py Tests blocking inbox retrieval deterministically.
tests/unit/iothub/test_sync_handler_manager.py Replaces handler timing sleeps.
tests/unit/iothub/test_sync_clients.py Uses blocking-call assertions.
tests/unit/iothub/aio/test_async_inbox.py Adds bounded waits and queue cleanup.
tests/unit/iothub/aio/test_async_handler_manager.py Adds outcome-based handler waits.
tests/unit/iothub/aio/test_async_clients.py Bounds asynchronous receive operations.
tests/unit/common/test_async_adapter.py Awaits callback completion directly.
tests/unit/common/test_alarm.py Waits on alarm completion events.
tests/unit/common/pipeline/test_pipeline_stages_base.py Polls for event-handler invocation.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tests/unit/iothub/aio/test_async_handler_manager.py
@cartertinney
Carter Tinney (cartertinney) force-pushed the agents/unit-test-failure-analysis-python312 branch from 619cfd7 to f08deb3 Compare September 1, 2026 17:26
@cartertinney

Carter Tinney (cartertinney) commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

/azp run Python E2E

@azure-pipelines

azure-pipelines Bot commented Sep 1, 2026

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Base automatically changed from ct/stabilize-uv-cache to main September 1, 2026 19:29
@cartertinney
tests: replace timing sleeps with condition-based waits
3e38098 
Avoid flaky assertions that assume cross-thread and cross-event-loop work completes within a fixed delay. Wait for observable outcomes instead, directly await callback futures, and explicitly synchronize blocking inbox tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cartertinney
tests: preserve timing semantics in hardened tests
5cbd0db 
Retain not-before and negative observation assertions where elapsed time is part of the behavior. Use explicit runner barriers to prove handler removal occurs while inbox work is still pending.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cartertinney
tests: bound asynchronous and blocking waits
4b50f70 
Use local deadlines for prompt and cross-loop async completion. Run deliberately blocking synchronous calls in daemon-backed futures so timeout failures cannot hang executor shutdown.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cartertinney
tests: simplify bounded async assertions
8b52fa3 
Delegate async condition deadlines to asyncio.wait_for and remove future-state assertions already guaranteed by successful bounded awaits. Keep direct exception-state checks where storing the exact exception is under test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cartertinney
tests: clarify polling helper names
f72d971 
Rename predicate-polling fixtures to poll_until and async_poll_until so they are clearly distinct from asyncio.wait_for, which applies a deadline to an awaitable.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cartertinney
tests: document synchronization flow
ac411bf 
Add succinct setup, action, and state comments around the non-obvious polling, blocking, and runner-control mechanics introduced by the test hardening.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cartertinney
tests: assert prompt async completion
71f0c6d 
Use explicit 100 ms deadlines for inbox reads and receive operations arranged to complete immediately. This preserves promptness expectations while still allowing the cross-loop scheduling required by AsyncClientInbox.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cartertinney
tests: centralize prompt timeout
d5077cf 
Define the 100 ms prompt-completion policy once and reuse it across async adapter, inbox, and receive-client tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cartertinney
tests: explicitly close Janus inbox queues
0641d26 
Replace historical teardown sleeps with the Janus 2.0 aclose API, scheduled on the internal loop where each queue's async side is bound.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cartertinney
tests: consolidate blocking call assertions
70e8490 
Replace the repeated call-start tracking and daemon-thread fixture protocol with one assert_call_blocks helper that synchronizes, releases, and propagates results with bounded waits.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cartertinney
tests: require explicit polling timeouts
cd4eb0c 
Make every poll_until and async_poll_until call state its timeout so bounded waiting is visible at the assertion site rather than hidden by a helper default.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cartertinney
tests: calibrate polling deadlines
7fdf782 
Use the 100 ms prompt deadline for single-hop pipeline callbacks and a one-second eventual deadline for handler work crossing queues, event loops, and worker threads. Keep five-second bounds only for deliberate deadlock-safety orchestration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cartertinney
tests: preserve prompt handler expectations
f47002b 
Use the original 100 ms expectation for single handler operations. Reserve a 500 ms batch deadline for five-item tests, including the case observed exceeding 100 ms in CI.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cartertinney
tests: remove redundant handler sleeps
d23480f 
After handler removal returns, directly assert that the handler and runner have been cleared before queuing new items. This proves no worker can invoke the removed handler without an observation sleep.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cartertinney
tests: await pending coroutine handlers
88fa1e2 
Receiver runner removal only guarantees coroutine handlers were scheduled on the handler loop. Wait for those invocations to complete before asserting the final call count.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cartertinney
Carter Tinney (cartertinney) force-pushed the agents/unit-test-failure-analysis-python312 branch from f08deb3 to 88fa1e2 Compare September 1, 2026 19:29
@cartertinney
Carter Tinney (cartertinney) merged commit 4196881 into main Sep 1, 2026
10 of 17 checks passed
@cartertinney
Carter Tinney (cartertinney) deleted the agents/unit-test-failure-analysis-python312 branch September 1, 2026 19:29
Carter Tinney (cartertinney) added a commit that referenced this pull request Sep 1, 2026
@cartertinney
e2e: replace timing waits with bounded synchronization (#1253)
fefdb59 
## Stack
- Stacked on #1252; review this PR after or against that branch.

## What changed
- replace one-second connection polling and unbounded event/future waits
with bounded condition-based synchronization
- remove method subscription sleeps because receive-handler setters
synchronously wait for feature subscription
- signal Event Hub readiness after the first active receive cycle
instead of sleeping three seconds
- start each helper from its construction timestamp so later partitions
retain new events without replaying prior test traffic
- filter raw-string telemetry waits by expected payload so delayed
unrelated events cannot satisfy them
- guarantee service-helper shutdown if readiness fails before fixture
yield
- run the Event Hub receiver on a daemon-backed future so its 30-second
shutdown timeout cannot be bypassed
- bound twin matching/retry loops with absolute deadlines
- remove import-time iptables mutation; existing per-test setup restores
network state
- co-locate focused wait-helper and Event Hub lifecycle tests under the
IoT Hub E2E suite

## Validation
- Python 3.12 unit suite: 5437 passed, 6 skipped
- E2E infrastructure tests: 13 passed
- Black and Ruff passed
- all 132 IoT Hub E2E tests collected without cloud access
- previous natural build, Python E2E, DPS E2E, and Horton E2E gates
passed

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@ewertons Ewerton Scaboro da Silva (ewertons) ewertons approved these changes
@avishekpant Avishek (avishekpant) avishekpant approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@cartertinney @ewertons @avishekpant