Skip to content

Commit 215095e

Browse files
armcknightarmcknight
authored
meta: update contributing readme to point to test plans for skipping (#5140)
1 parent ea44e7f commit 215095e

File tree

6 files changed

+3
-88
lines changed

6 files changed

+3
-88
lines changed

‎CONTRIBUTING.md

Lines changed: 3 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -48,17 +48,13 @@ make test
4848

4949
### Flaky tests
5050

51-
If you see a test being flaky, you should ideally fix it immediately. If that's not feasible, you can disable the test in the test scheme by unchecking it in the Test action:
51+
If you see a test being flaky, you should ideally fix it immediately. If that's not feasible, you can disable the test in the test scheme by unchecking it in the associated test plan:
5252

53-
![Disabling test cases via the Xcode scheme](./develop-docs/disabling_tests_xcode_scheme.png)
54-
55-
or by right-clicking it in the Tests Navigator (⌘6):
56-
57-
![Disabling test cases via the Xcode Tests navigator](./develop-docs/disabling_tests_xcode_tests_navigator.png)
53+
![Disabling test cases via the Xcode Tests navigator](./develop-docs/disabling_tests_xcode_test_plan.png)
5854

5955
Then create a GH issue with the [flaky test issue template](https://github.com/getsentry/sentry-cocoa/issues/new?assignees=&labels=Platform%3A+Cocoa%2CType%3A+Flaky+Test&template=flaky-test.yml).
6056

61-
Disabling the test in the scheme has the advantage that the test report will state "X tests passed, Y tests failed, Z tests skipped", as well as maintaining a centralized list of skipped tests (look in Sentry.xcscheme) and they will be grayed out when viewing in the Xcode Tests Navigator (⌘6):
57+
Disabling the test in the test plan has the advantage that the test report will state "X tests passed, Y tests failed, Z tests skipped", as well as maintaining a centralized list of skipped tests (look in the associated .xctestplan file source in //Plans/) and they will be grayed out when viewing in the Xcode Tests Navigator (⌘6):
6258

6359
![How Xcode displays skipped tests in the Tests Navigator](./develop-docs/xcode_tests_navigator_with_skipped_test.png)
6460

‎Samples/iOS-Swift/iOS-Swift.xcodeproj/xcshareddata/xcschemes/iOS-Swift.xcscheme

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -44,14 +44,6 @@
4444
BlueprintName = "iOS-Swift-UITests"
4545
ReferencedContainer = "container:iOS-Swift.xcodeproj">
4646
</BuildableReference>
47-
<SkippedTests>
48-
<Test
49-
Identifier = "SentryFileIOTrackingIntegrationTests/test_DataConsistency_readPath_disabled()">
50-
</Test>
51-
<Test
52-
Identifier = "SentryFileIOTrackingIntegrationTests/test_DataConsistency_readUrl_disabled()">
53-
</Test>
54-
</SkippedTests>
5547
</TestableReference>
5648
</Testables>
5749
</TestAction>

‎Sentry.xcodeproj/xcshareddata/xcschemes/Sentry.xcscheme

Lines changed: 0 additions & 73 deletions
Original file line numberDiff line numberDiff line change
@@ -60,74 +60,6 @@
6060
BlueprintName = "SentryTests"
6161
ReferencedContainer = "container:Sentry.xcodeproj">
6262
</BuildableReference>
63-
<SkippedTests>
64-
<Test
65-
Identifier = "SentryANRTrackerTests/testANRButAppInBackground_NoANR()">
66-
</Test>
67-
<Test
68-
Identifier = "SentryANRTrackerTests/testMultipleListeners()">
69-
</Test>
70-
<Test
71-
Identifier = "SentryCoreDataTrackerTests/testFetchRequestBackgroundThread()">
72-
</Test>
73-
<Test
74-
Identifier = "SentryCoreDataTrackerTests/testSaveBackgroundThread()">
75-
</Test>
76-
<Test
77-
Identifier = "SentryCrashIntegrationTests/testStartUpCrash_CallsFlush()">
78-
</Test>
79-
<Test
80-
Identifier = "SentryCrashReportStore_Tests/testCrashReportCount1">
81-
</Test>
82-
<Test
83-
Identifier = "SentryCrashReportStore_Tests/testDeleteAllReports">
84-
</Test>
85-
<Test
86-
Identifier = "SentryCrashReportStore_Tests/testStoresLoadsMultipleReports">
87-
</Test>
88-
<Test
89-
Identifier = "SentryHttpTransportTests/testFlush_WhenNoInternet_BlocksAndFinishes()">
90-
</Test>
91-
<Test
92-
Identifier = "SentryNetworkTrackerIntegrationTests/testGetCaptureFailedRequestsEnabled()">
93-
</Test>
94-
<Test
95-
Identifier = "SentryNetworkTrackerIntegrationTests/testGetRequest_CompareSentryTraceHeader()">
96-
</Test>
97-
<Test
98-
Identifier = "SentryOnDemandReplayTests/testAddFrameIsThreadSafe()">
99-
</Test>
100-
<Test
101-
Identifier = "SentryProfilerSwiftTests/testConcurrentSpansWithTimeout()">
102-
</Test>
103-
<Test
104-
Identifier = "SentryReachabilityTest/testMultipleReachabilityObservers">
105-
</Test>
106-
<Test
107-
Identifier = "SentrySDKIntegrationTestsBase">
108-
</Test>
109-
<Test
110-
Identifier = "SentrySDKTests/testStartOnTheMainThread()">
111-
</Test>
112-
<Test
113-
Identifier = "SentrySessionGeneratorTests/testSendSessions()">
114-
</Test>
115-
<Test
116-
Identifier = "SentrySubClassFinderTests/testActOnSubclassesOfViewController()">
117-
</Test>
118-
<Test
119-
Identifier = "SentryThreadInspectorTests/testGetCurrentThreadsWithStacktrace_WithSymbolication()">
120-
</Test>
121-
<Test
122-
Identifier = "SentryThreadInspectorTests/testStacktraceHasFrames_forEveryThread()">
123-
</Test>
124-
<Test
125-
Identifier = "SentryThreadWrapperTests/testOnMainThreadFromNonMainContext()">
126-
</Test>
127-
<Test
128-
Identifier = "SentryTracerTests/testDeadlineTimer_StartedAndCancelledOnMainThread()">
129-
</Test>
130-
</SkippedTests>
13163
</TestableReference>
13264
<TestableReference
13365
skipped = "NO">
@@ -138,11 +70,6 @@
13870
BlueprintName = "SentryProfilerTests"
13971
ReferencedContainer = "container:Sentry.xcodeproj">
14072
</BuildableReference>
141-
<SkippedTests>
142-
<Test
143-
Identifier = "SentryProfilerTests/testProfilerMutationDuringSlicing">
144-
</Test>
145-
</SkippedTests>
14673
</TestableReference>
14774
</Testables>
14875
</TestAction>

‎develop-docs/disabling_tests_xcode_scheme.png

-432 KB
Binary file not shown.

‎develop-docs/disabling_tests_xcode_test_plan.png

148 KB
Loading

‎develop-docs/disabling_tests_xcode_tests_navigator.png

-298 KB
Binary file not shown.

0 commit comments

Comments
 (0)