Skip to content

Refactor bluesky-queueserver support#266

Open
thopkins32 wants to merge 14 commits intomainfrom
qserver-refactor
Open

Refactor bluesky-queueserver support#266
thopkins32 wants to merge 14 commits intomainfrom
qserver-refactor

Conversation

@thopkins32
Copy link
Collaborator

@thopkins32 thopkins32 commented Mar 9, 2026

Summary

Refactors the bluesky-queueserver integration to be more modular.

Closes #265

User-facing Changes

  • blop.queueserver.ConsumerCallback now checks for a "blop_correlation_uid" key in the start document so we can properly filter queue items and get only the Bluesky run UID that we care about. Before, we did the same injection but left the filtering up to the evaluation function via searching the Tiled catalog.
  • blop.ax.QueueserverAgent has a run method for running the standard optimization loop for multiple iterations
  • blop.ax.QueueserverAgent has a submit_suggestions method for sampling specific points manually
  • Enables the specification of actuators, sensors, and acquisition plans as strings rather than instances of objects, since these instances will no longer exist in the process running Blop, but instead will exist in the queueserver process.

Technical Changes

  • Adds blop/queueserver.py which specifies a QueueserverClient, ConsumerCallback, and QueueserverOptimizationRunner
    • QueueserverClient handles the REManagerAPI instance, ZMQ remote disptach, and the callback thread
    • QueueserverOptimizationRunner contains the optimization loop and tracks state, using the client to submit suggestions to run
    • ConsumerCallback filters to find the appropriate RunStart and RunStop document of the submitted plan
  • Adds blop/tests/test_queueserver,py for unit tests
  • Adds QueueserverAgent to blop/ax/agent.py which provides a nicer interface for Ax-based optimization with queueserver
  • Adds tests for QueueserverAgent to blop/tests/ax/test_agent.py
  • Create _AxAgentMixin to provide common Ax functionality across Agents
  • blop.protocols.BaseOptimizationProblem abstraction to enforce type-safety using in-process vs out-of-process device and plan references
    • blop.protocols.OptimizationProblem should be used for in-process Blop with the RunEngine
    • blop.protocols.QueueserverOptimizationProblem should be used for Blop with queueserver

Other Changes

  • Lowered the type constraint on Actuator to be MovableHasName instead of NamedMovable. The distinction is documented in the code. This is required due to ophyd-async not requiring HasHints for signals.

Follow-up Work

  • Integration test as a tutorial, run queueserver & ZMQ proxy in separate processes, then go through a full optimization

@thopkins32 thopkins32 changed the title Qserver refactor Refactor bluesky-queueserver support Mar 9, 2026
@thopkins32 thopkins32 marked this pull request as ready for review March 11, 2026 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refactor bluesky-queueserver integration to work with any optimizer

2 participants