Skip to content

Commit 3c745a9

Browse files
committed
ref(snapshot-tests): Invoke CLI subprocess uniformly and expand MCP fixture coverage
Replace direct module imports in the snapshot harness with CLI subprocess invocations so every test exercises the real transport path. The resource harness now connects via MCP SDK client instead of importing handler modules directly. - Remove invokeDirect path and shutdownAllSimulatorsExcept - Add createTemporarySimulator/deleteSimulator/shutdownSimulator helpers - Add output-parsers for extracting app paths, PIDs, simulator entries - Resource harness uses StdioClientTransport + MCP Client - Add normalizers for MCP process IDs and OSLog session arrays - Remove CLI session-management fixtures (MCP-only tools) - Add missing MCP fixtures for device, macos, simulator, swift-package - Update all suites to use CLI-based harness patterns
1 parent 8b4e72d commit 3c745a9

77 files changed

Lines changed: 1525 additions & 1371 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

src/snapshot-tests/__fixtures__/cli/coverage/get-file-coverage--success.txt

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,10 +6,11 @@
66

77
File: example_projects/iOS_Calculator/CalculatorAppPackage/Sources/CalculatorAppFeature/CalculatorService.swift
88

9-
ℹ️ Coverage: 83.1% (157/189 lines)
9+
ℹ️ Coverage: 77.8% (147/189 lines)
1010

11-
🔴 Not Covered (7 functions, 22 lines)
11+
🔴 Not Covered (8 functions, 27 lines)
1212
L159 CalculatorService.deleteLastDigit() -- 0/16 lines
13+
L178 CalculatorService.setError(_:) -- 0/5 lines
1314
L58 implicit closure #2 in CalculatorService.inputNumber(_:) -- 0/1 lines
1415
L98 implicit closure #3 in CalculatorService.calculate() -- 0/1 lines
1516
L99 implicit closure #4 in CalculatorService.calculate() -- 0/1 lines
@@ -18,12 +19,12 @@ File: example_projects/iOS_Calculator/CalculatorAppPackage/Sources/CalculatorApp
1819
L214 implicit closure #4 in CalculatorService.formatNumber(_:) -- 0/1 lines
1920

2021
🟡 Partial Coverage (4 functions)
22+
L63 CalculatorService.inputDecimal() -- 71.4% (10/14 lines)
2123
L184 CalculatorService.updateExpressionDisplay() -- 80.0% (8/10 lines)
24+
L93 CalculatorService.calculate() -- 84.2% (32/38 lines)
2225
L195 CalculatorService.formatNumber(_:) -- 85.7% (18/21 lines)
23-
L93 CalculatorService.calculate() -- 89.5% (34/38 lines)
24-
L63 CalculatorService.inputDecimal() -- 92.9% (13/14 lines)
2526

26-
🟢 Full Coverage (28 functions) -- all at 100%
27+
🟢 Full Coverage (27 functions) -- all at 100%
2728

2829
Next steps:
2930
1. View overall coverage: xcodebuildmcp coverage get-coverage-report --xcresult-path "<TMPDIR>/TestResults.xcresult"

src/snapshot-tests/__fixtures__/cli/debugging/attach--success-continue.txt

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,6 @@
77
└ Execution: Execution is running. App is responsive to UI interaction.
88

99
Next steps:
10-
1. Add a breakpoint: debug_breakpoint_add({ debugSessionId: "<UUID>", file: "...", line: 123 })
11-
2. Continue execution: debug_continue({ debugSessionId: "<UUID>" })
12-
3. Show call stack: debug_stack({ debugSessionId: "<UUID>" })
10+
1. Add a breakpoint: xcodebuildmcp debugging add-breakpoint --debug-session-id "<UUID>" --file "..." --line "123"
11+
2. Continue execution: xcodebuildmcp debugging continue --debug-session-id "<UUID>"
12+
3. Show call stack: xcodebuildmcp debugging stack --debug-session-id "<UUID>"

src/snapshot-tests/__fixtures__/cli/debugging/attach--success.txt

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,6 @@
77
└ Execution: Execution is paused. Use debug_continue to resume before UI automation.
88

99
Next steps:
10-
1. Add a breakpoint: debug_breakpoint_add({ debugSessionId: "<UUID>", file: "...", line: 123 })
11-
2. Continue execution: debug_continue({ debugSessionId: "<UUID>" })
12-
3. Show call stack: debug_stack({ debugSessionId: "<UUID>" })
10+
1. Add a breakpoint: xcodebuildmcp debugging add-breakpoint --debug-session-id "<UUID>" --file "..." --line "123"
11+
2. Continue execution: xcodebuildmcp debugging continue --debug-session-id "<UUID>"
12+
3. Show call stack: xcodebuildmcp debugging stack --debug-session-id "<UUID>"
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
2+
🚀 Build & Run
3+
4+
Scheme: NONEXISTENT
5+
Workspace: example_projects/iOS_Calculator/CalculatorApp.xcworkspace
6+
Configuration: Debug
7+
Platform: iOS
8+
Device: <DEVICE> (<UUID>)
9+
Derived Data: <HOME>/Library/Developer/XcodeBuildMCP/DerivedData
10+
11+
Errors (1):
12+
13+
✗ The workspace named "CalculatorApp" does not contain a scheme named "NONEXISTENT". The "-list" option can be used to find the names of the schemes in the workspace.
14+
15+
❌ Build failed. (⏱️ <DURATION>)
16+
└ Build Logs: <HOME>/Library/Developer/XcodeBuildMCP/logs/build_run_device_<TIMESTAMP>_pid<PID>.log

src/snapshot-tests/__fixtures__/cli/device/test--failure.txt

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,15 @@
77
Device: <DEVICE> (<UUID>)
88
Derived Data: <HOME>/Library/Developer/XcodeBuildMCP/DerivedData
99

10+
Discovered 52 test(s):
11+
CalculatorAppFeatureTests/CalculatorBasicTests/testClear
12+
CalculatorAppFeatureTests/CalculatorBasicTests/testInitialState
13+
CalculatorAppFeatureTests/CalculatorBasicTests/testIntentionalFailure
14+
CalculatorAppFeatureTests/CalculatorIntegrationTests/testChainCalculations
15+
CalculatorAppFeatureTests/CalculatorIntegrationTests/testComplexCalculation
16+
CalculatorAppFeatureTests/CalculatorIntegrationTests/testExpressionDisplay
17+
(...and 46 more)
18+
1019
CalculatorAppTests
1120
✗ testCalculatorServiceFailure:
1221
- XCTAssertEqual failed: ("0") is not equal to ("999") - This test should fail - display should be 0, not 999
@@ -17,5 +26,5 @@ IntentionalFailureTests
1726
- XCTAssertTrue failed - This test should fail to verify error reporting
1827
example_projects/iOS_Calculator/CalculatorAppTests/CalculatorAppTests.swift:286
1928

20-
2 tests failed, 21 passed, 0 skipped (⏱️ <DURATION>)
29+
<FAIL_COUNT> tests failed, <PASS_COUNT> passed, <SKIP_COUNT> skipped (⏱️ <DURATION>)
2130
└ Build Logs: <HOME>/Library/Developer/XcodeBuildMCP/logs/test_device_<TIMESTAMP>_pid<PID>.log

src/snapshot-tests/__fixtures__/cli/device/test--success.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,5 +9,8 @@
99
Selective Testing:
1010
CalculatorAppTests/CalculatorAppTests/testAddition
1111

12+
Discovered 1 test(s):
13+
CalculatorAppTests/CalculatorAppTests/testAddition
14+
1215
✅ 1 test passed, 0 skipped (⏱️ <DURATION>)
1316
└ Build Logs: <HOME>/Library/Developer/XcodeBuildMCP/logs/test_device_<TIMESTAMP>_pid<PID>.log

src/snapshot-tests/__fixtures__/cli/session-management/session-clear-defaults--success.txt

Lines changed: 0 additions & 6 deletions
This file was deleted.

src/snapshot-tests/__fixtures__/cli/session-management/session-set-defaults--success.txt

Lines changed: 0 additions & 24 deletions
This file was deleted.

src/snapshot-tests/__fixtures__/cli/session-management/session-show-defaults--success.txt

Lines changed: 0 additions & 38 deletions
This file was deleted.

src/snapshot-tests/__fixtures__/cli/session-management/session-sync-xcode-defaults--success.txt

Lines changed: 0 additions & 6 deletions
This file was deleted.

0 commit comments

Comments
 (0)