Skip to content

feat: optionally let jobs run while an ad hoc command uses the inventory - #1208

Open
fernandorocagonzalez wants to merge 1 commit into
ctrliq:mainfrom
fernandorocagonzalez:feat-inv-adhoc-no-block
Open

fernandorocagonzalez wants to merge 1 commit into
ctrliq:mainfrom
fernandorocagonzalez:feat-inv-adhoc-no-block

Conversation

@fernandorocagonzalez

@fernandorocagonzalez fernandorocagonzalez commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor
SUMMARY

While an ad hoc command is running against an inventory, every job and every other ad hoc command launched against that inventory stays pending until it finishes:

waiting for adhoccommand-1234 to finish

That's the right default when the ad hoc command changes things under whatever comes next, like a reboot. But many ad hoc commands are long and harmless for everything else, a dnf update across a few hundred hosts for example, and while they run they hold back scheduled jobs that have nothing to do with them.

This adds an opt-in flag on the inventory, allow_jobs_while_ad_hoc, that lifts that block. It's set by whoever administers the inventory, the same as allow_jobs_while_syncing from #1054, so jobs and their variables have no say in it.

How it works:

  • New field Inventory.allow_jobs_while_ad_hoc, default false, so nothing changes for existing inventories. It's on the inventory and constructed inventory serializers and it's copied along with the inventory.
  • In DependencyGraph.inventory_blocked_by, a running ad hoc command no longer blocks when the inventory has the flag. It lets through both jobs and other ad hoc commands, since the reason to block them is the same.
  • Syncs are not affected. An ad hoc command never blocked a sync (inventory_update_blocked_by only looks at other syncs of the same source), and a running sync still blocks jobs and ad hoc commands unless allow_jobs_while_syncing is also set. Each flag only lifts its own block. There are tests for both so this doesn't change by accident.
  • The graph now reads both inventory flags with a single query per busy inventory and scheduling cycle, replacing the allow_jobs_while_syncing cache from feat: let jobs run against an inventory while it is being synced #1054. The query count doesn't grow with the number of pending tasks, and there's a test that checks it.
  • Migration 0235, depending on 0233. Optionally keep only inventory variables provided by inventory sources #1202 uses 0234, so whichever of the two goes in second needs renumbering, I'll take care of it.

UI: an Allow Jobs While Ad Hoc Commands Run checkbox under Options in the inventory form and the constructed inventory form, listed under Enabled Options in both detail views, with help text. Spanish translation added, the other locales re-extracted.

Docs: docs/inventory/allow_jobs_while_ad_hoc.md, and allow_jobs_while_syncing.md now points to it where it says ad hoc commands still block.

ISSUE TYPE
  • New or Enhanced Feature
COMPONENT NAME
  • API
  • UI
  • Docs
ASCENDER VERSION
ascender: 25.6.4.dev37+gb414367f6
ADDITIONAL INFORMATION

Tests added:

  • TestAllowJobsWhileAdHoc in test_scheduler.py: an ad hoc command blocks jobs and other ad hoc commands by default (with the right job explanation); with the flag both start; the flag of another inventory doesn't count; a sync starts while an ad hoc command runs, with the flag on or off; a running sync still blocks without allow_jobs_while_syncing, and allow_jobs_while_syncing alone doesn't let tasks past an ad hoc command; the flags are read once per cycle.
  • API test: the field is editable on regular and constructed inventories. The copy test checks the field is preserved.
  • Vitest tests for both form checkboxes and both detail views.

Checks from .github/copilot-instructions.md, run inside the dev container:

make test                 5434 passed, 6 skipped
make test_migrations      2 passed
check_migrations          No changes detected
check_settings            Every registered setting has a default its field accepts.
ruff format / ruff check / yamllint -s .   (tox linters pins)   clean
make ui-lint / ui-type-check               clean
make ui-test-screens      1980 passed
make ui-test-general      1180 passed
make ui-api-types         only the new field (9 schemas)

End to end on the docker-compose dev environment, with an inventory holding one local host, a sleep 150 ad hoc command and a ping job template:

  1. Flag off: launching the job and a second ad hoc command while the long one runs leaves both pending with waiting for adhoccommand-5 to finish. They start as soon as it ends.
  2. Turning the flag on from the inventory form saves it and shows it under Enabled Options.
  3. Flag on: the job and the second ad hoc command start right away and finish while the long one is still running.
  4. A constructed inventory over the same inventory, with a sleep 45 ad hoc command running on it: a sync of its source starts and finishes during the ad hoc command, with the flag off and with it on, the same as before this change.
  5. The checkbox and the Enabled Options entry also show on the constructed inventory form and details.

A running ad hoc command keeps every job and every other ad hoc command
against the same inventory pending until it finishes. With a long ad hoc
command, like a package update across many hosts, that holds back work
that has nothing to do with it.

The new allow_jobs_while_ad_hoc flag on the inventory lifts that block
for both jobs and ad hoc commands. It defaults to false, so nothing
changes for existing inventories. Syncs are not affected: an ad hoc
command never blocked a sync, and a running sync still blocks jobs
unless allow_jobs_while_syncing is also set.

The scheduler reads both inventory flags in a single query, once per
busy inventory and cycle, so the number of queries does not grow with
the number of pending tasks.

Signed-off-by: fernandorocagonzalez <fernandorocagonzalez@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant