Additional newline when printing inside of a test

[MarcinKonowalczyk]

What's the problem this feature will solve?

I, as i'm sure many others, sometimes use printing inside of the tests for quick debugging. The pattern is something like pytest -k <my-specific-test> -s -x (aka "run a subset of tests, don't capture prints and fail fast"). The issue is very minor: the prints look a bit ugly when in the middle of the pytest command-line output.

Describe the solution you'd like / Alternative Solutions

I've come up with the following:

import pytest
import builtins


@pytest.fixture(autouse=True)
def add_space_before_first_print():
    _print = builtins.print

    def patched_print(*args, **kwargs):
        # _print("CALLING PRINT", end="")
        _print(end="\n")  # Add a newline
        _print(*args, **kwargs)  # Do the print
        builtins.print = _print  # Restore the original after first print

    # Patch the print
    builtins.print = patched_print
    try:
        yield
    finally:
        # Make sure to restore the original print
        builtins.print = _print


def test_with_prints_1():
    print("test_with_prints_1 says hello")
    assert True


def test_without_prints_1(subtests):
    for _ in range(10):
        with subtests.test():
            assert True


def test_with_prints_2():
    print("test_with_prints_2 says hello")
    assert True


def test_without_prints_2(subtests):
    for _ in range(10):
        with subtests.test():
            assert True

which gives

test_with_prints_1 says hello
.,,,,,,,,,,.
test_with_prints_2 says hello
.,,,,,,,,,,.

with autose=False we get

tests/test_pytest_print_hack.py test_with_prints_1 says hello
.,,,,,,,,,,.test_with_prints_2 says hello
.,,,,,,,,,,.

Should purest do this by default?

Notes

1. I use subtests just to make purest print more markers to make demonstration clearer.
2. Same problem happens in -v mode and same solution also works.
3. Without -s the patched prints don't fire, as expected.

[The-Compiler]

IMHO, pytest monkey-patching Python built-ins by default is a big no-go.

(and yes, it does that with sys.stdout / sys.stderr - but I mean in a way that changes their behavior significantly)

[MarcinKonowalczyk]

I agree that touching the expected behaviour of print is a bad idea. Pytest is, however, already printing things itself, so how about making the newline character part of output there. Basically, if we're in -s mode, the test marker would become '.\n' if something was printed in the test. Else just '.'. (or, what's probably a better idea, there would be a post-marker which is normally '' and then selectively '\n'. This way we don't break any code which redefines the marker for whatever reason (i'm thinking e.g. subtests)

(I have not looked at the internals of pytest so i don't actually know how are things implemented)

[The-Compiler]

If you turn off output capturing, it follows that pytest has no way of knowing if something was printed before, as far as I'm aware.

Maybe an option to show the captured output for all passed tests would make sense? I thought we had something like that already, but doesn't look like it.

[MarcinKonowalczyk]

For the first bit: How about just peeking at the output capture (but still letting it go to wherever it would be going to otherwise)? When the -s flag is enabled, pytest would patch the sys.stdout/sys.stderr (which pytest does without the -s flag anyway) to just peek at whether there is any activity there. There could then be an -ss or -S flag for the edgewise where someone really really does not want pytest to touch sys.stdout/sys.stderr at all, even just to see whether there is something there.

As to the second bit: there is the --show-capture mode but it controls the print only for failed tests. Even if it did work, i think this is a subtly different use case and/or it would be good if the print in the middle of tests (which is what i'm imagining someone unfamiliar with pytest would reach for first) would work nicely too.

On this note a peripherally related idea: Currently, if all the test pass, pytest captures silently. Would it make sense to add a short line / warning saying that stdout has been captured and can be shown with a -s option?... Just thinking out loud here. I'm not sure this is a good idea...

[The-Compiler]

For the first bit: How about just peeking at the output capture (but still letting it go to wherever it would be going to otherwise)? When the -s flag is enabled, pytest would patch the sys.stdout/sys.stderr (which pytest does without the -s flag anyway) to just peek at whether there is any activity there. There could then be an -ss or -S flag for the edgewise where someone really really does not want pytest to touch sys.stdout/sys.stderr at all, even just to see whether there is something there.

I'm a bit lukewarm on the idea. It seems a very small benefit to me, in exchange for a somewhat surprising (and harder to explain) behavior.

As to the second bit: there is the --show-capture mode but it controls the print only for failed tests. Even if it did work, i think this is a subtly different use case and/or it would be good if the print in the middle of tests (which is what i'm imagining someone unfamiliar with pytest would reach for first) would work nicely too.

I don't think someone new to pytest would be confused by the output being shown at the end. I've seen the opposite (people being confused about output in the middle of pytest output), and I'm not sure if an additional newline would change anything.

On this note a peripherally related idea: Currently, if all the test pass, pytest captures silently. Would it make sense to add a short line / warning saying that stdout has been captured and can be shown with a -s option?... Just thinking out loud here. I'm not sure this is a good idea...

That seems like adding a lot of noise to me. Ideally, most of the time a testsuite passes - but with logging etc., it's rather normal for there to be some kind of output, so this doesn't seem anything worth printing a note about, IMHO (since it essentially would just ~always be printed).