Parallel execution (--parallel) broken in 25.2.0

[osksa]

Prerequisites

• This was not already reported in the past (duplicate check)
• It does reproduce it with code from main branch (latest unreleased version)
• I include a minimal example for reproducing the bug
• The bug is not trivial, as for those a direct pull-request is preferred
• Running pip check does not report any conflicts
• I was able to reproduce the issue on a different machine
• The issue is not specific to any driver other than 'default' one

Environment

molecule 25.2.0 using python 3.11
ansible:2.18.3
podman:2.0.3 from molecule_podman requiring collections: containers.podman>=1.7.0 ansible.posix>=1.3.0
default:25.2.0 from molecule

Running on Almalinux 9.5.

What happened

I use Molecule with Podman as driver. When I use Molecule 25.1.0, I can run many Molecule test runs on the same machine using --parallel. After upgrade to 25.2.0 or later, it no longer works. What happens then is that when the first molecule process which finishes its test run exits, all other Molecule processes on the machine fails with

`Traceback (most recent call last):
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/molecule/command/base.py", line 153, in execute_cmdline_scenarios
execute_scenario(scenario)
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/molecule/command/base.py", line 207, in execute_scenario
execute_subcommand(scenario.config, action)
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/molecule/command/base.py", line 197, in execute_subcommand
return command(current_config).execute(args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/molecule/logger.py", line 248, in wrapper
rt = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/molecule/command/idempotence.py", line 60, in execute
output = self._config.provisioner.converge()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/molecule/provisioner/ansible.py", line 711, in converge
return pb.execute()
^^^^^^^^^^^^
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/molecule/provisioner/ansible_playbook.py", line 139, in execute
util.sysexit_with_message(
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/molecule/util.py", line 151, in sysexit_with_message
sysexit(code)
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/molecule/util.py", line 125, in sysexit
sys.exit(code)
SystemExit: 2

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/os-sar/repos/orange-env/venv/bin/molecule", line 8, in
sys.exit(main())
^^^^^^
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/click/core.py", line 1161, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/click/core.py", line 1082, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/click/core.py", line 1697, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/click/core.py", line 1443, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/click/core.py", line 788, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/molecule/command/test.py", line 134, in test
base.execute_cmdline_scenarios(scenario_name, args, command_args, ansible_args)
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/molecule/command/base.py", line 163, in execute_cmdline_scenarios
execute_subcommand(scenario.config, "cleanup")
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/molecule/command/base.py", line 197, in execute_subcommand
return command(current_config).execute(args)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/molecule/command/base.py", line 74, in init
self._setup()
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/molecule/command/base.py", line 95, in _setup
self._config.write()
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/molecule/config.py", line 136, in write
util.write_file(self.config_file, util.safe_dump(self.config))
File "/home/os-sar/repos/orange-env/venv/lib64/python3.11/site-packages/molecule/util.py", line 216, in write_file
filename.write_text(content)
File "/usr/lib64/python3.11/pathlib.py", line 1078, in write_text
with self.open(mode='w', encoding=encoding, errors=errors, newline=newline) as f:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/pathlib.py", line 1044, in open
return io.open(self, mode, buffering, encoding, errors, newline)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/home/os-sar/.ansible/tmp/molecule.Iw-e.default/molecule.yml'`

I think the reason is that in 25.0.2, when Molecule finishes, it deletes a directory also used by the other processes. On 25.0.2 I see, as last output from Molecule when exiting

INFO Removing /home/os-sar/.ansible/tmp.

On 25.0.1, instead I see

INFO Removing /home/os-sar/.cache/molecule_parallel/realm-e00b2fe7-bb30-497a-8752-4cc515181204

(The name of the tested role here is "realm". )

Reproducing example

molecule.yml:


`dependency:
  name: galaxy
driver:
  name: podman
platforms:
  - name: rocky9
    image: docker.io/geerlingguy/docker-rockylinux9-ansible:latest
    pre_build_image: true
    privileged: true
    # These options are needed to run services using systemd inside the container
    command: /sbin/init
    cgroup_manager: cgroupfs
    storage_opt: overlay.mount_program=/usr/bin/fuse-overlayfs
    storage_driver: overlay
    security_opt: seccomp=unconfined
    cgroupns_mode: host
    volumes:
      - /sys/fs/cgroup:/sys/fs/cgroup:rw
provisioner:
  name: ansible
  config_options:
    defaults:
      roles_path: "$MOLECULE_PROJECT_DIRECTORY/.."
  inventory:
    group_vars:
      all:
        ansible_python_interpreter: /usr/bin/python3
verifier:
  name: testinfra
` 

command: `molecule test --parallel`