The mutation harness was last run 63 commits ago. Re-running it on the current tree gives 40 CAUGHT, 1 MISSED, 8 SKIP, 1 INVALID over 1038 tests, and surfaces three things worth separate attention.
21 mutations are caught by a test other than the one named to catch it
The harness flags a CAUGHT result when none of the mutation's expectFail tests actually failed. That flag fired 24 times. Splitting them:
- 3 are stale metadata: the named test was renamed or removed.
- 21 are cases where the named test still exists and did not fail, while a sibling caught the mutation instead.
Examples: deleting the remembered-port shortcut does not fail returns the same port on the next cold start, it fails does not drift back to a lower port once it has moved. Replacing the token generator with a counter does not fail generates non-null token, it fails different instances generate different tokens.
The suite is not blind here, since something caught every one. What is true is that 21 tests do not guard the property their names claim. The harness README is explicit that no verdict can decide this class and a human has to read them, so this is 21 to read rather than 21 defects.
8 mutations no longer address the tree
--validate reports them: 5 in ProcessManager.kt, plus SecurityManager.kt and SafSyncEngine.kt. Some find text is gone, some is now ambiguous. They need re-authoring against the current code or retiring.
1 MISSED, 1 INVALID
- MISSED:
all path methods return absolute paths. 1038 tests ran, none failed.
- INVALID:
getProjectsDir falls back to internal storage. Zero XML written, so the run broke rather than the test.
Already resolved
The previous run's only MISSED was the exit-code-137 classification in the watchdog. It is now CAUGHT.
The mutation harness was last run 63 commits ago. Re-running it on the current tree gives 40 CAUGHT, 1 MISSED, 8 SKIP, 1 INVALID over 1038 tests, and surfaces three things worth separate attention.
21 mutations are caught by a test other than the one named to catch it
The harness flags a
CAUGHTresult when none of the mutation'sexpectFailtests actually failed. That flag fired 24 times. Splitting them:Examples: deleting the remembered-port shortcut does not fail
returns the same port on the next cold start, it failsdoes not drift back to a lower port once it has moved. Replacing the token generator with a counter does not failgenerates non-null token, it failsdifferent instances generate different tokens.The suite is not blind here, since something caught every one. What is true is that 21 tests do not guard the property their names claim. The harness README is explicit that no verdict can decide this class and a human has to read them, so this is 21 to read rather than 21 defects.
8 mutations no longer address the tree
--validatereports them: 5 inProcessManager.kt, plusSecurityManager.ktandSafSyncEngine.kt. Some find text is gone, some is now ambiguous. They need re-authoring against the current code or retiring.1 MISSED, 1 INVALID
all path methods return absolute paths. 1038 tests ran, none failed.getProjectsDir falls back to internal storage. Zero XML written, so the run broke rather than the test.Already resolved
The previous run's only MISSED was the exit-code-137 classification in the watchdog. It is now CAUGHT.