Skip to content

L2 dispatcher passes xdist options to a child with xdist disabled #1993

Description

@doraemonmj

Platform

a2a3sim (the command-construction bug is platform-independent)

Runtime

Multiple L2 runtimes in one pytest invocation, for example host_build_graph and tensormap_and_ringbuffer

Description

When the top-level pytest invocation disables xdist with -p no:xdist, the L2 phase dispatcher inherits that option into each child command. If the effective --max-parallel value is greater than 1 and the xdist package is importable, the dispatcher also appends -n <N> --dist loadfile.

The resulting child command both disables xdist and uses xdist-only CLI options. Pytest then exits during argument parsing with rc=4.

This is easy to miss because a collection containing only L2 items from one runtime takes the direct-run fast path and does not invoke the L2 dispatcher. A collection spanning multiple runtimes does invoke it and fails deterministically.

Steps to reproduce

From the repository root with the test dependencies installed:

python -m pytest \
  tests/st/a2a3/tensormap_and_ringbuffer/dfx/args_dump/test_args_dump.py \
  tests/st/a2a3/host_build_graph/vector_example/test_vector_example.py \
  --platform a2a3sim \
  --device 0-1 \
  -p no:xdist \
  --max-parallel 2 \
  --manual include \
  --pto-session-timeout 600

No device allocation is required to observe the failure because both child processes fail while parsing arguments.

Expected behavior

The dispatcher must not construct a child command that disables xdist while also passing xdist-only options.

The intended semantics of -p no:xdist should be made explicit. Either:

  1. Treat it as authoritative for all phases and run L2 children serially without -n/--dist; or
  2. Treat it as applying only to the outer orchestrator and remove only the no:xdist plugin-disable option from L2 child arguments before adding -n/--dist.

Actual behavior

Both runtime children fail before collection:

::group::L2 host_build_graph [-n 2]
ERROR: ... unrecognized arguments: -n 2 --dist loadfile
--- L2 host_build_graph: FAIL rc=4 ---

::group::L2 tensormap_and_ringbuffer [-n 2]
ERROR: ... unrecognized arguments: -n 2 --dist loadfile
--- L2 tensormap_and_ringbuffer: FAIL rc=4 ---

Root cause

_base_pytest_argv() in conftest.py inherits the original invocation arguments. The L2 path only strips --exclude-level, so -p no:xdist remains in base_args.

_dispatch_test_phases() independently checks whether import xdist succeeds when max_parallel > 1. It does not check whether the plugin was explicitly disabled, and then appends -n <N> --dist loadfile.

Importability is therefore being used as a proxy for plugin availability in the child, but those are not equivalent when -p no:xdist is inherited.

Suggested regression coverage

Add a dispatcher command-construction test with:

  • multiple L2 runtimes so the dispatcher path is taken;
  • -p no:xdist in the inherited invocation arguments;
  • --max-parallel 2; and
  • pytest-xdist installed.

Assert that a child command never contains both no:xdist and -n/--dist, and add an end-to-end subprocess assertion that the invocation does not exit with pytest usage error rc=4.

Commit

58ca976f97c7ac9461f8c0ae8f2db3b7fe4f6f78

Host platform

Linux aarch64, Python 3.10.19, pytest 8.4.2, pytest-xdist 3.8.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingcode healthTechnical debt, robustness, code quality

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions