`@ParameterizedTest` that accepts no arguments should fail by default

[pfichtner]

A @ParameterizedTest that takes no argument should "fail".

If a @ParameterizedTest does not have any source it fails:

PreconditionViolationException: Configuration error: You must configure at least one arguments source for this @ParameterizedTest

This @ParameterizedTest passes without taking any argument:

@ParameterizedTest
@ValueSource(strings = { "foo", "bar" })
void methodDoesNotTakeAnyParameter() {}

I suggest a @ParameterizedTest to fail if it takes no arguments at all with a PreconditionViolationException or similar, or is there any valid reason to have a @ParameterizedTest without an argument?

[pfichtner]

Background: I saw a test where the developer thought they were testing all combinations of the parameter, but hadn’t actually done so.
Of course, one could also overlook this if one accepts the argument and does not use it in any of the AAA sections.

[marcphilipp]

Thanks for the report!

This is a special case of parameterized test allowing more arguments than their parameter list supports which we originally decided to make it reuse to use parameter lists for different tests.

However, I think this case looks wrong and we should think about making it an error. I've added it to our team discussion backlog.

[codingkiddo]

I’d be interested in helping with this once the team agrees on the desired behavior.

From a quick look, @ParameterizedTest currently validates that at least one argument source is configured and that at least one invocation is produced, but it does not seem to validate that the test method declares at least one parameter/aggregator to consume the supplied arguments.

Would the preferred behavior be to fail during parameterized test discovery/context creation when a @ParameterizedTest method has no formal parameters, or should this only be enforced when an argument source actually produces arguments?

[sbrannen]

A @ParameterizedTest that takes no argument should "fail".

That's arguably correct, but for historical reasons that is not the case.

However, the behavior is configurable since JUnit 5.12.

The following...

@ParameterizedTest(argumentCountValidation = ArgumentCountValidationMode.STRICT)
@ValueSource(strings = { "foo", "bar" })
void methodDoesNotTakeAnyParameter() {
}

... fails as follows for the first invocation and similarly for the second invocation.

org.junit.platform.commons.PreconditionViolationException: Configuration error: @ParameterizedTest consumes 0 parameters but there was 1 argument provided.
Note: the provided arguments were [foo]

Does ArgumentCountValidationMode.STRICT meet your needs?

[pfichtner]

Thanks for the answer @sbrannen
It seems it would fix the issue, but if someone forgets to consume the parameter, they most likely won't have set ArgumentCountValidationMode.STRICT as an option.

So I'd vote for a breaking change in JUnit by switching the default to ArgumentCountValidationMode.STRICT so if someone needs to have an ParameterizedTest without any argument to set the option to something like ArgumentCountValidationMode.NONE or ArgumentCountValidationMode.NONSTRICT

[mpkorstanje]

Note for team discussion: If we don't change the default argumentCountValidation due to concerns about backwards compatibility in v6, we can put it on the backlog for v7

[marcphilipp]

Thanks for chiming in, @sbrannen!

It seems it would fix the issue, but if someone forgets to consume the parameter, they most likely won't have set ArgumentCountValidationMode.STRICT as an option.

You can set it "globally" in your project's src/test/resources/junit-platform.properties like this:

junit.jupiter.params.argumentCountValidation = STRICT

[pfichtner]

That's a great option (really!!!) but still "just" a project default.
Please discuss, as suggested, whether this should be the default setting for JUnit in general.