Coverage miss in Python 3.10

[Nik-09]

Describe the bug
While upgrading the Python version for the Oppia Foundation organization from 3.9.20 to 3.10.16, I noticed that some backend files are missing coverage for lines that are otherwise covered in Python 3.9.

Upon investigation, I found that these lines are generally trivial (e.g., simple assignments) and should not be excluded. Approximately 23 files show this behavior. I’m sharing a single instance here for clarity.

To Reproduce

What version of Python are you using? Python 3.10.16

What version of coverage.py shows the problem? The output of coverage debug sys is helpful.

Coverage version: 7.8.0

Output of coverage debug sys

-- sys -------------------------------------------------------
               coverage_version: 7.8.0
                coverage_module: /home/nikhil/oppia_clone/.direnv/python-3.10.16/lib/python3.10/site-packages/coverage/__init__.py
                           core: -none-
                        CTracer: available
           plugins.file_tracers: -none-
            plugins.configurers: -none-
      plugins.context_switchers: -none-
              configs_attempted: /home/nikhil/oppia_clone/oppia/.coveragerc
                   configs_read: /home/nikhil/oppia_clone/oppia/.coveragerc
                    config_file: /home/nikhil/oppia_clone/oppia/.coveragerc
                config_contents: b"# .coveragerc to control coverage.py\n[run]\nomit =\n    */oppia_tools/*\n    */third_party/*\n    */usr/*\n    *_test.py\n    *core/tests/*\n    *scripts/linters/test_files/*\n    *__init__.py\n    */vm_deps/*\n    */scripts/build.py\n    */scripts/install_python_dev_dependencies.py\n    */scripts/install_python_prod_dependencies.py\n    */scripts/install_third_party_libs.py\n    */scripts/run_acceptance_tests.py\n# The above files in the scripts folder will be deleted once the migration to Docker is complete.\n# Currently, the backend tests for these files are failing when runnning within the Docker container.\n\n# The 'exclude_lines' is used to skip a particular clause in coverage.\n# We exclude any line with a comment of \xe2\x80\x9cpragma: no cover\xe2\x80\x9d as this is the default option\n# provided by the coverage library.\n# We exclude any line with a comment of \xe2\x80\x9cdocker: no cover\xe2\x80\x9d as this is used to skip\n# lines that will be removed after the migration to Docker.\n# We also exclude @overload, because it is used for defining multiple MyPy definitions \n# of a function and cannot be tested by Python tests.\n[report]\nexclude_lines =\n    pragma: no cover\n    @overload\n    docker: no cover\n"
                      data_file: -none-
                         python: 3.10.16 (main, Mar 30 2025, 13:20:54) [GCC 13.3.0]
                       platform: Linux-6.11.0-26-generic-x86_64-with-glibc2.39
                 implementation: CPython
                    gil_enabled: True
                     executable: /home/nikhil/oppia_clone/.direnv/python-3.10.16/bin/python
                   def_encoding: utf-8
                    fs_encoding: utf-8
                            pid: 37337
                            cwd: /home/nikhil/oppia_clone/oppia
                           path: /home/nikhil/oppia_clone/.direnv/python-3.10.16/bin
                                 /home/nikhil/.pyenv/versions/3.10.16/lib/python310.zip
                                 /home/nikhil/.pyenv/versions/3.10.16/lib/python3.10
                                 /home/nikhil/.pyenv/versions/3.10.16/lib/python3.10/lib-dynload
                                 /home/nikhil/oppia_clone/.direnv/python-3.10.16/lib/python3.10/site-packages
                    environment: COVERAGE_PROCESS_START = .coveragerc
                                 HOME = /home/nikhil
                                 PYENV_ROOT = /home/nikhil/.pyenv
                                 PYENV_SHELL = bash
                                 PYENV_VIRTUALENV_INIT = 1
                   command_line: /home/nikhil/oppia_clone/.direnv/python-3.10.16/bin/coverage debug sys
         sqlite3_sqlite_version: 3.45.1
             sqlite3_temp_store: 0
        sqlite3_compile_options: ATOMIC_INTRINSICS=1, COMPILER=gcc-13.3.0, DEFAULT_AUTOVACUUM,
                                 DEFAULT_CACHE_SIZE=-2000, DEFAULT_FILE_FORMAT=4,
                                 DEFAULT_JOURNAL_SIZE_LIMIT=-1, DEFAULT_MMAP_SIZE=0, DEFAULT_PAGE_SIZE=4096,
                                 DEFAULT_PCACHE_INITSZ=20, DEFAULT_RECURSIVE_TRIGGERS,
                                 DEFAULT_SECTOR_SIZE=4096, DEFAULT_SYNCHRONOUS=2,
                                 DEFAULT_WAL_AUTOCHECKPOINT=1000, DEFAULT_WAL_SYNCHRONOUS=2,
                                 DEFAULT_WORKER_THREADS=0, DIRECT_OVERFLOW_READ, ENABLE_COLUMN_METADATA,
                                 ENABLE_DBSTAT_VTAB, ENABLE_FTS3, ENABLE_FTS3_PARENTHESIS,
                                 ENABLE_FTS3_TOKENIZER, ENABLE_FTS4, ENABLE_FTS5, ENABLE_LOAD_EXTENSION,
                                 ENABLE_MATH_FUNCTIONS, ENABLE_PREUPDATE_HOOK, ENABLE_RTREE, ENABLE_SESSION,
                                 ENABLE_STMTVTAB, ENABLE_UNLOCK_NOTIFY, ENABLE_UPDATE_DELETE_LIMIT,
                                 HAVE_ISNAN, LIKE_DOESNT_MATCH_BLOBS, MALLOC_SOFT_LIMIT=1024,
                                 MAX_ATTACHED=10, MAX_COLUMN=2000, MAX_COMPOUND_SELECT=500,
                                 MAX_DEFAULT_PAGE_SIZE=32768, MAX_EXPR_DEPTH=1000, MAX_FUNCTION_ARG=127,
                                 MAX_LENGTH=1000000000, MAX_LIKE_PATTERN_LENGTH=50000,
                                 MAX_MMAP_SIZE=0x7fff0000, MAX_PAGE_COUNT=0xfffffffe, MAX_PAGE_SIZE=65536,
                                 MAX_SCHEMA_RETRY=25, MAX_SQL_LENGTH=1000000000, MAX_TRIGGER_DEPTH=1000,
                                 MAX_VARIABLE_NUMBER=250000, MAX_VDBE_OP=250000000, MAX_WORKER_THREADS=8,
                                 MUTEX_PTHREADS, SECURE_DELETE, SOUNDEX, SYSTEM_MALLOC, TEMP_STORE=1,
                                 THREADSAFE=1, USE_URI

What versions of what packages do you have installed? The output of pip freeze is helpful.1. What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix.

Output of pip freeze

astroid==2.15.6
beautifulsoup4==4.12.2
build==1.2.2.post1
certifi==2023.7.22
click==8.1.8
coverage==7.8.0
dill==0.3.6
esprima==4.0.1
filetype==1.2.0
isort==5.12.0
lazy-object-proxy==1.9.0
mccabe==0.7.0
mypy==0.812
mypy_extensions==0.4.4
packaging==25.0
Pillow==10.1.0
pip-tools==7.4.1
platformdirs==3.9.1
psutil==5.9.5
pycodestyle==2.10.0
pylint==2.17.2
pylint-quotes==0.2.3
pyproject_hooks==1.2.0
PyYAML==6.0
rcssmin==1.1.1
soupsieve==2.4.1
tomli==2.0.1
tomlkit==0.11.8
typed_ast==1.4.3
typing_extensions==4.5.0
waitress==2.1.2
WebOb==1.8.7
WebTest==3.0.0
wrapt==1.15.0
xmltodict==0.13.0

Missing code lines: link

translation_reviewer_stats = list(translation_reviewer_stats)
language_code, reviewer_user_id = keys

What commands should we run to reproduce the problem?

1. Create and navigate to a new directory (e.g., test_oppia)
2. Clone the relevant branch:

git clone -b python_migration https://github.com/Nik-09/oppia.git

3. Navigate to oppia cd oppia.
4. Start Docker Desktop.
5. Build the project: make build
6. Run the backend test: make run_tests.backend PYTHON_ARGS="--test_target=core.jobs.batch_jobs.contributor_admin_stats_jobs_test --generate_coverage_report --skip-install"
7. Check for the missing coverage: View CLI output or navigate to the HTML report: cd htmlcov then open index.html.

Expected behavior
The coverage report should match the behavior observed with Python 3.9.20, where all relevant lines (including trivial ones) are marked as covered.

Python 3.10.16 and Code coverage for Python, version 7.8.0 with C extension:

Python 3.9.20 and Code coverage for Python, version 7.2.3 with C extension:

Additional context
This issue appears to affect only Apache Beam job files, which are asynchronous pipeline jobs. It’s unclear if the async behavior is affecting the coverage collection in Python 3.10.

[Nik-09]

Adding @nedbat to this issue for his insights. Thank you!