Introduction

pysource_codegen is able to generate random python code which can be compiled. The compiled code should not be executed.

This is still a very early version, but it does its job. It is general useful to test tools like formatters, linters or anything which operates on python code.

Install:

pip install pysource-codegen

Usage:

The tool can be used over the CLI:

pysource-codegen --seed 42

or as a library:

from pysource_codegen import generate

seed = 42
print(generate(seed))

You might find pysource-minimize also useful to reduce the generated code which triggers your bug down to a minimal code snipped, which can be used to fix the issue.

from pysource_codegen import generate
from pysource_minimize import minimize


def contains_bug(code):
    """
    returns True if the code triggers a bug and False otherwise
    """
    try:
        test_something_with(code)  # might throw

        if "bug" in code:  # maybe other checks
            return True
    except:
        return True
    return False


def find_issue():
    for seed in range(0, 10000):
        code = generate(seed)

        if contains_bug(code):
            new_code = minimize(code, contains_bug)

            print("the following code triggers a bug")
            print(new_code)

            return


find_issue()

Bugs found in other projects:

black

• psf/black#3676
• psf/black#3678
• psf/black#3677

cpython

3.12

• python/cpython#109219
• python/cpython#109823
• python/cpython#109719
• python/cpython#109627
• python/cpython#109219
• python/cpython#109118
• python/cpython#109114
• python/cpython#109889

3.13

• python/cpython#120367
• python/cpython#120225
• python/cpython#124746
• python/cpython#124871

Todo:

• refactor the existing code
  • add better context information to probability()
  • remove fix() function and move code into probability()
• use probabilities for the ast-nodes from existing python code (use markov chains)
• support older python versions
• allow to customize the probabilities to generate code to test specific language features
• hypothesis support