Skip to content

Python code coverage not working for multiprocessing inside site-packages #1968

Open
@sebsken

Description

@sebsken

Describe the bug
Code coverage does not work when testing installed library.

To Reproduce
How can we reproduce the problem? Please be specific. Don't link to a failing CI job. Answer the questions below:

  1. What version of Python are you using? 3.10
  2. What version of coverage.py shows the problem? The output of coverage debug sys is helpful. 7.5.1
  3. What versions of what packages do you have installed? The output of pip freeze is helpful.
  4. What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix.
  5. What commands should we run to reproduce the problem? Be specific. Include everything, even git clone, pip install, and so on. Explain like we're five!

Folder architecture:

pytest.ini
production_code_folder
- pyproject.toml
- pxz_dataset
      - sample.py
      - __init__.py
test_code_folder
- test.py

pyproject.toml:

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[project]
name = "pxz_dataset"
version = "1.0.0"

__init__.py:

from .sample import big_run

sample.py:

import multiprocessing

def f(x):
    return x*x

def big_run(n):
    with multiprocessing.Pool(5) as p:
        a = p.map(f, list(range(n)))
        p.close()
        p.join()

test.py:

from pxz_dataset import big_run

def test_main():
    big_run(10)

pytest.ini

[pytest]
python_files = test*.py
required_plugins = pytest-cov pytest-xdist pytest-env

[coverage:run]
concurrency = multiprocessing
branch = true
parallel = true
sigterm = true

command line to run

cd path/to/production_code_folder
python -m build
pip install --find-links=dist/ pxz_dataset
cd ../test_code_folder
pytest . --cov=pxz_dataset --cov-branch --cov-report=term --cov-config=../pytest.ini

And this prints

---------- coverage: platform linux, python 3.10.6-final-0 -----------
Name                                                                                   Stmts   Miss Branch BrPart  Cover
------------------------------------------------------------------------------------------------------------------------
/mnt/data/global_envs/seb_py310/lib/python3.10/site-packages/pxz_dataset/__init__.py       1      0      0      0   100%
/mnt/data/global_envs/seb_py310/lib/python3.10/site-packages/pxz_dataset/sample.py         8      1      2      0    90%
------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                      9      1      2      0    91%

Expected behavior
I should have 100% code coverage.
We can test this by doing the following

pip uninstall pxz_dataset
cp -r ../production_code_folder/pxz_dataset .
pytest . --cov=pxz_dataset --cov-branch --cov-report=term --cov-config=../pytest.ini

And this prints

---------- coverage: platform linux, python 3.10.6-final-0 -----------
Name                      Stmts   Miss Branch BrPart  Cover
-----------------------------------------------------------
pxz_dataset/__init__.py       1      0      0      0   100%
pxz_dataset/sample.py         8      0      2      0   100%
-----------------------------------------------------------
TOTAL                         9      0      2      0   100%

Additional context
I tried simply copy pasting the folder anywhere else in the system and there is no problem.
I tried duplicating site-packages anywhere else in the system and putting the folder pxz_dataset in it and no problem
I tried creating a folder /mnt/data/global_envs/seb_py310/lib/python3.10/sites and putting pxz_dataset in it and also no problem.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions