scalasem: read Scala 3.10 builds, skip stale output, add an accuracy check

docs/SCALASEM.md

@@ -58,7 +58,11 @@ scalasem "$(pwd)" slices.json
 The debug printer behind `scalac -print-tasty` is never used: it crashes on TASTy newer than
 the compiler that reads it, and one crash ends the whole batch. The TASTy Inspector API reads
 through the compiler's real unpickler instead. That is why the compiler that wrote the files is
-resolved first; its version is recorded in every TASTy header.
+resolved first; its version is recorded in every TASTy header. The helper is compiled against
+the compiler's full runtime classpath, which is read from the compiler's own POM in the local
+caches and resolved from those caches the way Maven resolves it, so a release that changes the
+artifacts it needs is followed without a new list; when the caches cannot satisfy the POM, the
+throwaway sbt project completes them.
 
 ## SemanticDB
 
@@ -124,7 +128,11 @@ source layout rather than the compiler's trees, and every call graph edge they p
   previous build exported next to it (sbt 1 `<module>/target/streams`, sbt 2
   `target/out/<platform>/scala-<v>/<module>/streams`, Mill `out/<module>/*.json`). When a cross
   build left several Scala 3 trees for one module, the newest is read and the choice is
-  recorded as a `no-build-scala-version` diagnostic.
+  recorded as a `no-build-scala-version` diagnostic. An sbt build that keeps the output of its
+  modules under `target/out` at the build root declares them: that output is read, whatever
+  Scala version it was built with, and the trees an earlier build left beside the sources are
+  skipped as `stale-output`, with the directory, its Scala version and the winning one named.
+  A directory that belongs to no module of the build is ignored.
 
 Only the class directories the build reports are read. A cross build keeps several
 `target/scala-<v>` trees side by side, and a directory walk would mix them; the build says
@@ -343,9 +351,10 @@ where one did:
 | `build-compile-failed`, `scala-cli-compile-failed`, `maven-classpath-failed`   | The build tool failed to compile or to print the classpath.                                    |
 | `semanticdb-unavailable`, `semanticdb-missing`, `semanticdb-compile-failed`    | No plugin release for the Scala version, or a compile that wrote no SemanticDB.                |
 | `scala2-unsupported`                                                           | A Scala 2 module with no SemanticDB to read.                                                   |
-| `compiler-unavailable`, `tasty-inspector-unavailable`, `helper-compile-failed` | The TASTy of a compiler release could not be read.                                             |
+| `compiler-unavailable`, `tasty-inspector-unavailable`, `helper-compile-failed` | The TASTy of a compiler release could not be read. A failed helper compile names the class its compiler could not load in `missingClass`.                            |
 | `unreadable-tasty`                                                             | TASTy files the compiler could not load.                                                       |
 | `scala-version-mismatch`, `no-build-scala-version`                             | `SCALA_VERSION` names a release the build output does not hold, or a cross build left several. |
+| `stale-output`                                                                 | Output of a module that an earlier build left elsewhere was skipped; the entry names the source root, the skipped directory with its Scala version, and the directory the module was read from. |
 | `unresolved-symbols`, `walker-failed`                                          | Counts of symbols the helper could not resolve and files its walker could not finish.          |
 
 ## Caching

docs/TESTING.md

@@ -21,6 +21,25 @@ The Ruby and PHP suites skip themselves cleanly when the plugin bundle or the la
 
 The single most useful thing to know locally: `ci/verify-packed-tarball.sh` is the release check, and it runs the same way on a laptop as in CI. It packs the tarball, installs it into a scratch project, and parses fixtures with the installed copy.
 
+## The local accuracy check
+
+The fixture suites run scalasem over small projects, which no new Scala or sbt release can be trusted against alone. `test-fixtures/scalasem-accuracy.js` is the by-hand check over a large, current Scala code base, for example over a checkout of [chen](https://github.com/AppThreat/chen); run it before a release and after a change to the inventory, the compiler handling or the derivation rules. It is not part of CI and not part of `npm test`.
+
+```shell
+node test-fixtures/scalasem-accuracy.js ~/work/chen --out accuracy.json
+node test-fixtures/scalasem-accuracy.js ~/work/chen --no-build \
+    --baseline accuracy.json --out accuracy-new.json
+```
+
+The script builds the project with sbt unless `--no-build` is given (the sbt build may need `GITHUB_TOKEN`), runs scalasem, and scores the report against facts that do not come from scalasem:
+
+- **Coverage** per module is the share of main source files that have facts in the report.
+- **Diagnostics** are all listed; a `helper-compile-failed` or `stale-output` diagnostic counts as a failure and fails the run.
+- **Library usage** is read from the class files themselves: a constant pool reader collects the classes every compiled class file names, they are mapped to the jars of the module's classpath through the jars' zip central directories, and that set is compared against the types, owners and symbols the report names. A library the class files use but the report never mentions is a miss; one the report names without a class file reference lands in a review list. Inline methods and constants folded at compile time make the class files an imperfect ground truth, which is why the review list is not a failure. The project's own upstream modules are not libraries and stay out of the scoring.
+- **Call sites** are spot checked: a sample of the reported calls is checked for the called name appearing on the reported line of the source file.
+
+The numbers, the commit of the project and the scalasem version go to the `--out` JSON file, and a summary goes to stdout. With `--baseline`, the file of an earlier run, every number is shown against that run and a regression — less coverage, fewer calls, a diagnostic that appears — is called out.
+
 ## What the tests assert
 
 The suites deliberately assert what a consumer depends on rather than that a command ran. A few examples make the style concrete.

docs/TROUBLESHOOTING.md

@@ -18,14 +18,18 @@ Count first, then explain. Both batch tools write a manifest with `files_parsed`
 | astgen: `node_modules` bundles missing                                   | bundles are opt-in; `ASTGEN_INCLUDE_NODE_MODULES_BUNDLES=true`                                                            |
 | phpastgen: `vendor/` or `tests/` absent                                  | the default `--exclude` regex; pass `-e NOMATCH` to disable                                                               |
 | rbastgen: `Gemfile`, `Rakefile` missing from a _different_ tool's output | basename-discovered DSL files are a rbastgen feature; check you ran rbastgen and that the manifest did not exclude them   |
-| scalasem: entries missing                                                | `scalac -print-tasty` failed for that file; check `ATOM_MAX_BUFFER` against very large class files, and stderr in the log |
+| scalasem: entries missing                                                | the helper or the build failed for that module; check the `diagnostics` array of the report, and stderr in the log       |
 
 ## A specific file produces no AST
 
 Per-file failures are never fatal in batch mode. Look in the diagnostics side-record (`phpastgen_diagnostics.jsonl`, `ruby_astgen_diagnostics.jsonl`) for the message, line, and column. Remember the naming contract: diagnostics exist only for the latest run, and a clean run deletes the file, so a stale diagnostics file on disk means something failed in the run you just did.
 
 Two file-level behaviors are sometimes mistaken for bugs. Non-UTF-8 sources are scrubbed and marked `encoding_scrubbed: true` rather than dropped. Trees deeper than `--max-depth` (default 250) are truncated with a `truncated_nodes` count rather than emitted whole or dropped.
 
+## A scalasem report covers fewer modules than the build has
+
+Read the `diagnostics` array of the report first; it names the module and the reason per case. A `helper-compile-failed` entry carries the Scala version and, when the compiler could not load a class, that class in `missingClass` — usually a runtime artifact of the compiler release that the local caches do not hold, which a run with installs allowed resolves through sbt. A `stale-output` entry says the module was read from its current output and an older output directory, built at another Scala version, was skipped; the report then describes the current build, not the older one. See [SCALASEM.md](SCALASEM.md) for the full diagnostics list.
+
 ## Output differs between machines
 
 Compare provenance before debugging parsers. `rbastgen --parser-info` shows which `parser` and `prism` copies a machine loads; the Ruby grammar ceiling follows the runtime's prism (grammar 3.5 on Ruby 3.4 versus 4.1 on Ruby 4.0 at the time of writing), and a newer machine-installed prism raises it because `GEM_PATH` is preserved. Per-file JSON records `parser_backend`, `generator_version`, and `ruby_version` for exactly this comparison. On the PHP side, pin `--target-version` if you need byte-identical trees across PHP runtimes. On the astgen side, remember that `ASTGEN_TYPE_WORKERS` can reorder members of inferred union types; unset it when output is diffed or cached.

lib/scalasem/build.js

@@ -1371,6 +1371,11 @@ function noBuildInventory(dir, opts, diagnostics, tool) {
       roots.set(root, leftoverModule(dir, root));
     }
   }
+  // Only a build whose definition can be read tells the current output of a module from a
+  // stale one; everywhere else every tree stands on its own, as before.
+  if (tool === "sbt" || detectBuildTool(dir).tool === "sbt") {
+    applyDeclaredOutput(dir, roots, diagnostics);
+  }
   const chosen = new Map();
   for (const [root, info] of roots) {
     const key = `${info.id}:${info.platform}`;
@@ -1419,6 +1424,90 @@ function noBuildInventory(dir, opts, diagnostics, tool) {
   return { modules };
 }
 
+// `target/out/jvm/scala-3.10.0/backend/classes` at the build root: platform, Scala version, project.
+const SBT2_OUTPUT =
+  /(?:^|\/)target\/out\/([^/]+)\/scala-([^/]+)\/([^/]+)\/classes$/;
+
+/**
+ * Reconcile the class roots against the output of the current sbt build. sbt 2 keeps the
+ * output of every module under `<build>/target/out`, so the directories there name the modules
+ * of the build and their current Scala version. An output of such a module that an earlier
+ * build left anywhere else, mostly the sbt 1 trees beside the sources, is skipped with a
+ * `stale-output` diagnostic, and a directory no module of the build claims is ignored. A build
+ * that left no `target/out` at all declares nothing here, so every tree keeps standing on its
+ * own.
+ *
+ * @param {string} projectDir Build directory
+ * @param {Map<string, Object>} roots Class root to its module information
+ * @param {Object[]} diagnostics Diagnostics of the inventory
+ */
+function applyDeclaredOutput(projectDir, roots, diagnostics) {
+  const relOf = (root) => relative(projectDir, root).replaceAll("\\", "/");
+  const winners = new Map();
+  for (const [root, info] of roots) {
+    const match = SBT2_OUTPUT.exec(relOf(root));
+    if (!match) {
+      continue;
+    }
+    const key = `${match[3]}:${info.platform}`;
+    const winner = winners.get(key);
+    if (!winner || compareDotted(match[2], winner.scalaVersion) > 0) {
+      winners.set(key, {
+        root,
+        id: match[3],
+        scalaVersion: match[2],
+        platform: info.platform
+      });
+    }
+  }
+  if (!winners.size) {
+    return;
+  }
+  for (const [root, info] of [...roots]) {
+    const rel = relOf(root);
+    const match = SBT2_OUTPUT.exec(rel);
+    if (match) {
+      const owner = winners.get(`${match[3]}:${info.platform}`);
+      if (owner?.root === root) {
+        continue;
+      }
+      roots.delete(root);
+      if (owner) {
+        diagnostics.push({
+          code: "stale-output",
+          module: owner.id,
+          sourceRoot: ".",
+          skippedDir: rel,
+          skippedVersion: match[2],
+          usedDir: relOf(owner.root),
+          usedVersion: owner.scalaVersion
+        });
+      }
+      continue;
+    }
+    // An sbt 1 tree belongs to the module whose directory it sits in: the path before its
+    // `target`, or the build root for the root project.
+    const moduleDir = /^(.+?)\/target\//.exec(rel)?.[1];
+    const owner = moduleDir
+      ? winners.get(`${basename(moduleDir)}:${info.platform}`)
+      : winners.get(`${basename(projectDir)}:${info.platform}`);
+    roots.delete(root);
+    if (!owner) {
+      // A tree that belongs to no module the output names is not a project of the build.
+      continue;
+    }
+    diagnostics.push({
+      code: "stale-output",
+      module: owner.id,
+      sourceRoot: moduleDir || ".",
+      skippedDir: rel,
+      skippedVersion: info.scalaVersion,
+      usedDir: relOf(owner.root),
+      usedVersion: owner.scalaVersion
+    });
+  }
+}
+
 /** The class directory a package directory belongs to: the nearest `classes` ancestor. */
 function classRootOf(projectDir, packageDir) {
   let d = packageDir;
@@ -1524,13 +1613,63 @@ function platformOfDir(classDir) {
   return "jvm";
 }
 
-/** The module directory that owns a leftover class directory: the path before its `target`. */
+/**
+ * The source roots of a leftover class directory: the module directory before its `target`,
+ * or, for the output sbt 2 writes under the build root, the directory of the project the
+ * output is named after. An ambiguous or absent project name leaves the whole build as the
+ * root, which is all that is known for certain.
+ */
 function guessSourceRoots(projectDir, classDir) {
   const rel = relative(projectDir, classDir).replaceAll("\\", "/");
+  const sbt2 = SBT2_OUTPUT.exec(rel)?.[3];
+  if (sbt2) {
+    const moduleDir = findModuleDir(projectDir, sbt2);
+    if (moduleDir) {
+      return [moduleDir];
+    }
+  }
   const prefix = /^(.+?)\/(?:target|\.scala-build)\//.exec(rel)?.[1];
   return [prefix ? join(projectDir, prefix) : projectDir];
 }
 
+/**
+ * The directory of the project an sbt 2 output directory is named after: the project lives in
+ * a directory of that name under the build root, a few levels deep at most. One clear match
+ * answers; none or several leaves the question open.
+ */
+function findModuleDir(projectDir, id) {
+  const matches = [];
+  const visit = (d, depth) => {
+    if (depth > 4 || matches.length > 1) {
+      return;
+    }
+    if (basename(d) === id) {
+      matches.push(d);
+      return;
+    }
+    let entries;
+    try {
+      entries = readdirSync(d, { withFileTypes: true });
+    } catch (_err) {
+      return;
+    }
+    for (const entry of entries) {
+      if (
+        !entry.isDirectory() ||
+        entry.name === "target" ||
+        entry.name === "out" ||
+        entry.name === "project" ||
+        entry.name.startsWith(".")
+      ) {
+        continue;
+      }
+      visit(join(d, entry.name), depth + 1);
+    }
+  };
+  visit(projectDir, 0);
+  return matches.length === 1 ? matches[0] : undefined;
+}
+
 /**
  * The dependency classpath the last build exported for a class directory: next to it for
  * sbt 2 (`<module>/streams`), under the module's `target/streams` for sbt 1, else the Mill