Skip to content

Commit cfa0c09

Browse files
authored
Clarify the outcome of applying a timeLimit trait to a suite. (#1225)
Make it clearer that the time limit applies to each test or case in the suite, not to the overall runtime of the test suite. ### Motivation: I received feedback that a developer couldn't work out whether `.timeLimit(_:)` on a suite limits each test, or the whole suite. ### Modifications: - Add documentation that explains that the time limit applies to each test or case in a parameterized test, when you apply it to a suite. - Remove some passive-voice sentences in the same documents so that it's clearer what a developer does, and what the testing library does for them. ### Checklist: - [X] Code and documentation should follow the style of the [Style Guide](https://github.com/apple/swift-testing/blob/main/Documentation/StyleGuide.md). - [X] If public symbols are renamed or modified, DocC references should be updated.
1 parent bc74ac0 commit cfa0c09

File tree

2 files changed

+30
-9
lines changed

2 files changed

+30
-9
lines changed

Sources/Testing/Testing.docc/LimitingExecutionTime.md

Lines changed: 16 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -40,22 +40,29 @@ hour (60 x 60 seconds) to execute, the task in which it's running is
4040
and the test fails with an issue of kind
4141
``Issue/Kind-swift.enum/timeLimitExceeded(timeLimitComponents:)``.
4242

43-
- Note: If multiple time limit traits apply to a test, the shortest time limit
44-
is used.
43+
- Note: If multiple time limit traits apply to a test, the testing library uses
44+
the shortest time limit.
4545

4646
The testing library may adjust the specified time limit for performance reasons
4747
or to ensure tests have enough time to run. In particular, a granularity of (by
4848
default) one minute is applied to tests. The testing library can also be
4949
configured with a maximum time limit per test that overrides any applied time
5050
limit traits.
5151

52-
### Time limits applied to test suites
52+
### Apply time limits to test suites
5353

54-
When a time limit is applied to a test suite, it's recursively applied to all
55-
test functions and child test suites within that suite.
54+
When you apply a time limit to a test suite, the testing library recursively
55+
applies it to all test functions and child test suites within that suite.
56+
The time limit applies to each test in the test suite and any child test suites,
57+
or each test case for parameterized tests.
5658

57-
### Time limits applied to parameterized tests
59+
For example, if a suite contains five tests and you apply a time limit trait
60+
with a duration of one minute, then each test in the suite may run for up to
61+
one minute.
5862

59-
When a time limit is applied to a parameterized test function, it's applied to
60-
each invocation _separately_ so that if only some arguments cause failures, then
61-
successful arguments aren't incorrectly marked as failing too.
63+
### Apply time limits to parameterized tests
64+
65+
When you apply a time limit to a parameterized test function, the testing
66+
library applies it to each invocation _separately_ so that if only some
67+
cases cause failures due to timeouts, then the testing library doesn't
68+
incorrectly mark successful cases as failing.

Sources/Testing/Traits/TimeLimitTrait.swift

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,13 @@ extension Trait where Self == TimeLimitTrait {
8383
/// test cases individually. If a test has more than one time limit associated
8484
/// with it, the shortest one is used. A test run may also be configured with
8585
/// a maximum time limit per test case.
86+
///
87+
/// If you apply this trait to a test suite, then it sets the time limit for
88+
/// each test in the suite, or each test case in parameterized tests in the
89+
/// suite.
90+
/// For example, if a suite contains five tests and you apply a time limit trait
91+
/// with a duration of one minute, then each test in the suite may run for up to
92+
/// one minute.
8693
@_spi(Experimental)
8794
public static func timeLimit(_ timeLimit: Duration) -> Self {
8895
return Self(timeLimit: timeLimit)
@@ -116,6 +123,13 @@ extension Trait where Self == TimeLimitTrait {
116123
/// If a test is parameterized, this time limit is applied to each of its
117124
/// test cases individually. If a test has more than one time limit associated
118125
/// with it, the testing library uses the shortest time limit.
126+
///
127+
/// If you apply this trait to a test suite, then it sets the time limit for
128+
/// each test in the suite, or each test case in parameterized tests in the
129+
/// suite.
130+
/// For example, if a suite contains five tests and you apply a time limit trait
131+
/// with a duration of one minute, then each test in the suite may run for up to
132+
/// one minute.
119133
public static func timeLimit(_ timeLimit: Self.Duration) -> Self {
120134
return Self(timeLimit: timeLimit.underlyingDuration)
121135
}

0 commit comments

Comments
 (0)