Molecule 4.0.0 not running additional roles

[austincloudguru]

I have a vault role (https://github.com/austincloudguru/ansible-role-vault) and as part of the testing, it first installs my consul role (for the backend). In version 3.6.1, everything works as expected. In 4.0.0, it completely ignores my consul role (although it does download it). It just seems to skip right over it. No errors, no nothing. Any ideas what could be going on? Did I miss something on the new release?

[zhan9san]

Hi @austincloudguru

The role defined in molecule/<scenario_name>/requirements.yml will be installed in an ephemeral dir which can be found when we run 'molecule test'.

Does the 'molecule test' succeed?
If yes, it matches what I say above.

If another role is required when we run our own role, the another role should be defined in galaxy metadata dependency.

But I noticed the dependency is commented in your source code.

Reference: https://galaxy.ansible.com/docs/using/installing.html#dependencies

Let me know what you think.

[austincloudguru]

A molecule test never even tries to run the consul role in 4.0.0. It just acts like it is not even there. No errors, no nothing. The dependency is commented out because I was trying all different ways to get it to work, and just forgot to uncomment it and return it to normal before I merged it. The master branch runs perfectly in 3.6.1 (the requirements file is now included in the source directory). But if I run the same code w/ 4.0.0, it never runs the role.

[zhan9san]

Sorry to introduce any inconvenience.

Would you mind providing a minimal-reproducible-example with actual behavior and expected behavior?

[austincloudguru]

I've created https://github.com/AustinCloudGuru/ansible_role_example that is a pretty empty role that also tries to install my consul role. If you create a virtual environment with the requirements-3.6.1.txt it runs the consul role just fine. If you use the 4.0.0 requirements file, it doesn't do anything and fails the verify.

[zhan9san]

@austincloudguru

Sorry for the late response and any inconvenience introduced.

Thanks for your patience. The issue can be reproduced.

In short

There is a conflict between ansible-lint and molecule==4.0.0. I'll try to fix it ASAP.

In detail

what ansible-lint does

Both ansible-lint==6.0.2 in requirements-3.6.1.txt and ansible-lint==6.3.0 in requirements-4.0.0.txt create empty directories under ~/.cache/ansible-compat/0ad53a/roles as below.

$ ls -l /Users/jackzhang/.cache/ansible-compat/0ad53a/roles/austincloudguru.consul
total 0
$ ls -l /Users/jackzhang/.cache/ansible-compat/0ad53a/roles/austincloudguru.vault
total 0

As austincloudguru.consul is set in https://github.com/AustinCloudGuru/ansible_role_example/blob/5ec807e7400d41dcfce0266bde8ac1869fe6bfec/.ansible-lint#L12

mock_roles:
  - austincloudguru.vault
  - austincloudguru.consul

molecule test contains lint as below. And ansible-lint is set in https://github.com/AustinCloudGuru/ansible_role_example/blob/5ec807e7400d41dcfce0266bde8ac1869fe6bfec/molecule/default/molecule.yml#L9

molecule matrix test
INFO     Test matrix
---
default:
  - dependency
  - lint
  - cleanup
  - destroy
  - syntax
  - create
  - prepare
  - converge
  - idempotence
  - side_effect
  - verify
  - cleanup
  - destroy

what molecule does

molecule test did install requirements austincloudguru.consul, but under another directory ~/.cache/molecule/ansible_role_example/default/roles

$ molecule dependency
...
Starting galaxy role install process
- changing role austincloudguru.consul from v1.9.13 to unspecified
- downloading role 'consul', owned by austincloudguru
- downloading role from https://github.com/AustinCloudGuru/ansible-role-consul/archive/v1.9.13.tar.gz
- extracting austincloudguru.consul to /Users/jackzhang/.cache/molecule/ansible_role_example/default/roles/austincloudguru.consul
- austincloudguru.consul (v1.9.13) was installed successfully
INFO     Dependency completed successfully.
WARNING  Skipping, missing the requirements file.

The result of conflict

When molecule runs, it will set ANSIBLE_ROLES_PATH=$runtime_cache_dir/roles:$ephemeral_directory/roles/:$project_directory/../:~/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles referring to ANSIBLE_ROLES_PATH.

The first path $runtime_cache_dir/roles in ANSIBLE_ROLES_PATH is ~/.cache/ansible-compat/0ad53a/roles while the second $ephemeral_directory/roles/ is ~/.cache/molecule/ansible_role_example/default/roles.

As we know, the order of ANSIBLE_ROLES_PATH matters, and austincloudguru.consul does exist in first role path, so ansible will run it without error if it is an empty role.

why does it work in molecule==3.6.1

In molecule==3.6.1, $runtime_cache_dir/roles(~/.cache/ansible-compat/0ad53a/roles) is not in ANSIBLE_ROLES_PATH.

You can verify it by adding the following code snippet in converge.yml.

    - name: "test"
      ansible.builtin.debug:
        msg: "{{ lookup('env','ANSIBLE_ROLES_PATH') }}"

Hope I express myself clear.

[zhan9san]

@austincloudguru

Let us wait for next release of ansible-lint with which your role may work with latest molecule

ansible/ansible-lint#2235

And if you are interested in this change, you can try it as below.

pip install molecule==4.0.0
pip install git+https://github.com/ansible/ansible-lint
molecule test

[austincloudguru]

Thanks. Sorry for the slow response, I'm in the process of moving right now. I'll test the ansible-lint when it's updated.