Skip to content

Add: ProfilerBase::quiesce() to drain without retiring the threads - #2091

Merged
ChaoWao merged 1 commit into
hw-native-sys:mainfrom
ChaoWao:profiler-quiesce
Sep 1, 2026
Merged

Add: ProfilerBase::quiesce() to drain without retiring the threads#2091
ChaoWao merged 1 commit into
hw-native-sys:mainfrom
ChaoWao:profiler-quiesce

Conversation

@ChaoWao

@ChaoWao ChaoWao commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

stop() couples two things: draining the collector pipeline and retiring its
worker threads. Its drain guarantee is in fact paid for by the threads
exiting — joining mgmt is what proves its final sweep landed in the host ready
queue shards, and joining the collectors is what proves those shards were
consumed. Its own comment spells out the ordering that depends on this.

That coupling is why the DFX collectors are started and joined per run
(#2078): there is no way to reach a known-empty state without tearing them
down. quiesce() supplies the missing half — the same guarantee, threads
intact
— using a two-phase epoch handshake in place of the joins.

void quiesce();   // both queues empty, every entry through on_buffer_collected
void stop();      // unchanged

stop()'s implementation is untouched; the only lines this PR removes from it
are comment text. So nothing that exists today changes behaviour, and this
lands as a standalone primitive ahead of the collector-residency work.

Two things the design is forced into

The phases are ordered, not concurrent. A collector reporting its shard
empty before mgmt finished sweeping would be reporting on a queue mgmt is about
to push into — so collect_quiesce_epoch_ is published only once every drain
ack for that epoch has landed.

The alternative — having the caller drain the device-side ring itself — is not
available: mgmt is already consuming it, and a second consumer breaks the SPSC
invariant. What makes a drain worker's "one full sweep found nothing"
conclusive is the caller having stopped the device-side producers, which is
already true where the DFX teardown runs.

The collector ack sits above the has_seen_buffer guard. A shard that
never received a buffer is a valid run shape (the loop's existing comment says
so), and an ack behind that guard would leave quiesce() waiting on a silent
subsystem forever.

Testing

  • test_profiler_base 9/9 (was 6) — three new cases: drain-without-retire,
    silent-collector completion, and no-op before start() / after stop()
  • Negative control run for the guard-placement claim: moving the ack
    below has_seen_buffer and rebuilding makes QuiesceCompletesOnASilentCollector
    hang (ctest 30s timeout) while the preceding case still passes — so the test
    targets that failure specifically rather than passing incidentally
  • Full cpput 128/128 on this base
  • clang-format clean
  • No scene tests — nothing calls quiesce() yet; stop() is unchanged, so
    the runtime paths are untouched by this PR

QuiesceDrainsWithoutRetiringThreads deliberately does not poll with
wait_for_collected: quiesce() must have delivered everything by the time it
returns, so polling would hide a handshake that reports too early.

Also

Drops the L1/L2 shorthand from this file's comments. It denoted the
device-side ring and the host ready queue shards, but L1 and L2 are already
taken by the hierarchy model (docs/hierarchical-level-runtime.md: L1 = chip
die, L2 = chip runtime), so the same coordinates carried two unrelated
meanings. Comment-only; the file now has zero L1/L2 matches.

Step 1 of the plan in
#2078 (comment).

stop() couples two things: draining the pipeline and retiring the worker
threads. Its drain guarantee is in fact *paid for* by the threads exiting
— joining mgmt is what proves its final sweep landed in the host shards,
and joining the collectors is what proves those shards were consumed.

That coupling is why the DFX collectors are started and joined per run:
there is no way to reach a known-empty state without tearing them down.
quiesce() supplies the missing half — the same guarantee, threads intact
— using a two-phase epoch handshake in place of the joins.

The phases are ordered rather than concurrent. A collector reporting its
shard empty before mgmt has finished sweeping would be reporting on a
queue mgmt is about to push into, so collect_quiesce_epoch_ is published
only once every drain ack for that epoch has landed. The caller having
stopped the device-side producers is what makes a drain worker's "one
full sweep found nothing" conclusive; that already holds where the DFX
teardown runs.

The collector-side ack sits above the has_seen_buffer guard on purpose:
a shard that never received a buffer is a valid run shape, and an ack
behind that guard would leave quiesce() waiting on a silent subsystem
forever. The new silent-collector test hangs if it is moved.

stop()'s implementation is unchanged, so no existing behavior moves.

Also drops the L1/L2 shorthand from this file's comments. It denoted the
device-side ring and the host ready queue shards, but L1 and L2 are
already taken by the hierarchy model (die and chip runtime), so the same
coordinates meant two unrelated things.
@coderabbitai

coderabbitai Bot commented Sep 1, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 45 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: a3c476fc-c510-4653-a8ea-188811421fea

📥 Commits

Reviewing files that changed from the base of the PR and between ae90918 and 749dd05.

📒 Files selected for processing (2)
  • src/common/platform/include/host/profiler_base.h
  • tests/ut/cpp/common/test_profiler_base.cpp

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ChaoWao
ChaoWao merged commit 33078b0 into hw-native-sys:main Sep 1, 2026
20 checks passed
@ChaoWao
ChaoWao deleted the profiler-quiesce branch September 1, 2026 12:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant