tbrun.mjs

// Build a twinBASIC probe and capture what it writes to the DEBUG CONSOLE.
//
//     node scripts/tbrun.mjs <source-dir> [options]
//
//       --port <n>        DevTools port to start the IDE on (default: one Windows picks)
//       --arch <target>   win32 or win64 (default win32): the target to build
//                         for, and so the process the probe runs in -- a win64
//                         probe runs in the IDE's 64-bit compiler
//       --compile-timeout <secs>  the compile's limit, from the project reported
//                         open to the compile settled
//       --build-timeout <secs>  the build's limit, from the press to [BUILD] Executing
//       --compiler-run-timeout <secs>  the probe's run's limit, from Executing
//       --compiler-run-quiet <secs>  the run is over after this long with no
//                         output, counted from Executing or the last output
//       --exe-run-timeout <secs>  with --exe, the exe's limit
//       --timeout <secs>  both --compiler-run-timeout and --exe-run-timeout; with
//                         --tests, each case's limit. Each limit's default is
//                         test/harness.json's (test/harness.md), or its
//                         TB_<PHASE> variable
//       --json            emit one JSON object instead of text
//       --raw             do not strip the console's timestamp column
//       --keep            leave the IDE running afterwards, and print its pid
//                         (implies --no-reap)
//       --no-reap         do not harvest automation servers the probe left behind
//       --reap-images     comma-separated image names to harvest
//                         (default: the Office suite -- see REAP_IMAGES)
//       --show / --hide   as tbbuild's
//       --llvm            compile the whole project with LLVM: the same as
//                         --compiler-options +llvm
//       --compiler-options <s>  the project's compiler options, for the run
//                         and for the exe (compiler.debugOptions and
//                         compiler.buildOptions); an option string with +llvm
//                         is refused on a Community or Personal licence
//       --exe             run the built exe as well, on a private desktop, and
//                         capture what it writes with TbRun.Out, its exit code,
//                         the fault the event log records for it, and the boxes
//                         it opens, each closed with OK
//       --tests           do not build: judge the project's compile cases and
//                         run its [TestCase] Subs, printing PASS or FAIL for
//                         each (lib/tb-tests.mjs)
//       --only <names>    with --tests, judge and run only these cases: a
//                         comma-separated list of Fixture.Sub names and compile
//                         case names; a case that never returns is run alone
//                         this way (scripts/package_tests.mjs)
//
// Exit: 0 captured output, 1 the project has compile errors, 2 the harness
// could not run (a refused command line included), a compile never settled, or
// it crashed -- a build that fails after a clean compile included, and a
// [RunAfterBuild] Sub that fails code generation, since the probe never runs,
// and a procedure the probe calls that fails it, since the probe stops at the
// call, an LLVM refusal anywhere in the log, a build that met its limit, and a
// test/harness.json missing or malformed -- 3 no output: the build produced none
// in the console, or the probe ran and printed none after its last Debug.Cls -- 4 the
// compiler crashed, or restarted twice, while compiling the project -- 5 the
// probe ended before it returned, or stopped at a run-time error it does not
// handle, its output printed all the same -- 6 with
// --exe, the exe exited with a code other than 0, the event log records that it
// faulted, it opened a box, or it was still running at its limit; its
// output, exit code, fault and boxes printed all the same -- 7 with --tests, a
// case failed.
//
// ---------------------------------------------------------------- why
//
// tbbuild.mjs answers "does this compile". It cannot answer "what does this
// print", and some questions only runtime can settle -- the one that prompted
// this script was the width of a Debug.Print print zone, which no shipped
// source demonstrates and no amount of reading the documentation establishes.
// (14 characters, and positive numbers carry a leading sign space, so values
// land at columns 1, 15, 29 rather than 0, 14, 28.)
//
// The mechanism is the [RunAfterBuild] attribute: a Sub marked with it runs in
// the IDE once the exe is built, so anything it writes with Debug.Print lands
// in the IDE's DEBUG CONSOLE, where this script reads it back over CDP.
//
// It starts and reads the IDE through scripts/lib/tb-ide.mjs, the same code
// tbbuild uses, rather than by running tbbuild as a child process. As a child
// it was handed neither --ide nor the IDE this script had packed with, so the
// two could be different installs, and the pid it had to kill came back as a
// line of text to parse.
//
// ----------------------------------------------- seven things it gets right
//
// Each of these cost an hour when the probe was first done by hand.
//
//  1. THE BUILD PATH MUST BE IN AN EXPLICIT FOLDER. A project whose
//     `project.buildPath` is still the default `${SourcePath}\Build\...`
//     template opens a native Save dialog on the build -- and because the
//     IDE runs on a private desktop, that dialog is invisible, takes no
//     input, and the build simply never happens. Nothing reports it: the
//     WebView2 renderer stays responsive, so even a CDP health check says the
//     IDE is fine. This script therefore owns the tree and pins buildPath to
//     its own out folder before importing, which makes the trap unreachable
//     (lib/tb-project.mjs, shared with the add-in harness). The file name is
//     the IDE's own `${ProjectName}_${Architecture}.${FileExtension}`, so it
//     says what was built: on BETA 1010 those variables in an explicit
//     folder open no dialog, and a win32 build of the staged project gives
//     DocSamples_win32.exe.
//  2. A JAVASCRIPT .click() ON THE BUILD BUTTON DOES NOTHING. `#buildIcon` is
//     a plain DIV wired through the IDE's own pointer handling; it needs real
//     CDP Input.dispatchMouseEvent presses at its centre (tb-click's click,
//     which also checks that nothing covers it).
//  3. READ THE CONSOLE'S BACKING ARRAY, NOT THE PANE. The DEBUG CONSOLE is a
//     virtualised list view: only the rows that fit are in the DOM, so an
//     `.innerText` scrape of it returns the tail of a long probe and looks
//     exactly like a complete capture. Measured against the old reader: a
//     probe printing 120 lines came back with 11. tb-ide's readConsole reads
//     `debugConsoleContent.dataNodes`, the whole log, and explains the rest.
//  4. START THE PROBE WITH Debug.Cls. The DEBUG CONSOLE is also where the IDE
//     writes its own build log, and the linker writes there after the build --
//     so without a clear, a probe's output comes back interleaved with
//     [LINKER] lines. The script warns when a probe omits it. The clear can
//     erase a failure as well: a procedure the probe calls that fails code
//     generation is reported before the probe's first statement runs. So the
//     script wraps the page's clearDebugConsole() before the build, keeps
//     what each clear erases, and looks there too (tb-ide's keepClears).
//  5. QUIET-PERIOD, AND A MARKER THE PROBE NEVER WRITES. The run is over when
//     the console stops changing, which works for any probe. But a probe that
//     stops early also stops changing the console: End does, and so does an
//     error raised in LLVM-compiled code with no handler, which ends the run
//     without a word (BETA 1010, plain code too). So the staged copy calls the
//     probe's Sub from a wrapper that prints a sentinel once it returns
//     (lib/tb-probe.mjs). Its absence is exit 5; its arrival ends the wait
//     without the quiet period. Only the IDE's line about forms still loaded
//     may follow it (sentinelIndex).
//  6. EVERY RUN OWNS ITS OWN WORKSPACE AND KILLS ONLY ITS OWN IDE. Both were
//     shared, and both broke concurrency in ways that looked like something
//     else. The staging directory was a fixed %TEMP%/tbrun/src, so a second
//     run rmSync'd the first one's tree out from under it -- observed as an
//     EPERM from a script that had touched no such path. Worse, shutdown was
//     `taskkill /F /T /IM twinBASIC.exe`, which is machine-wide: it ended
//     every concurrent run's IDE, and the IDE you had open yourself. The work
//     directory and the project.id are now keyed to the run's slot from
//     tbbroker (WIP.Broker.md), and the IDE is killed by the pid its launch
//     returned.
//  7. A COM SERVER THE PROBE STARTED IS NOT A CHILD OF ANYTHING WE OWN.
//     CreateObject("Excel.Application") is activated by DCOM, so the EXCEL.EXE
//     that appears has svchost.exe for a parent -- measured. No tree kill can
//     reach it, and a probe that throws before app.Quit leaves it running
//     forever. Harvesting it therefore has to be a before/after diff, which is
//     a blunt enough instrument to need the guard rails in reapOrphans().

import { execFileSync } from "node:child_process";
import { existsSync, readFileSync, mkdirSync, statSync, readdirSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import path from "node:path";
import {
  CliError,
  choiceOption,
  die,
  exitOnCrash,
  numberOption,
  parseCli,
  printHelpAndExit,
  refuseTogether,
  withUsageError,
} from "../lib/cli.mjs";
import { HarnessError, optionSeconds, secondsText, timeoutMs } from "./lib/harness-settings.mjs";
import { findIde, unsupportedIde } from "./lib/tb-install.mjs";
import {
  TARGETS,
  attachIde,
  compileOutcome,
  keptIdeLines,
  killTree,
  launchIde,
  launchOnDesktop,
  leaseSlot,
  llvmLicence,
  setBuildTarget,
  shutdownIde,
  summaryLine,
  waitForCompile,
  wantShow,
} from "./lib/tb-ide.mjs";
import { readConsole } from "./lib/tb-ide-console.mjs";
import { asksForLlvm, holdLlvmCache, leaseReport } from "./lib/tb-llvm-cache.mjs";
import { sentinelIndex, wrapProbe } from "./lib/tb-probe.mjs";
import { laneProjectId, stageProject } from "./lib/tb-project.mjs";
import { finishTidy, startTidy } from "./lib/tb-registry.mjs";
import { buildTimeoutMessage, captureRun, checkCapture, strip } from "./lib/tb-run.mjs";
import { caseLine, findCases, judgeCompiles, runTest } from "./lib/tb-tests.mjs";
import { faultText, recordedFault } from "./lib/win-fault.mjs";

exitOnCrash();

const USAGE = `usage: node scripts/tbrun.mjs <source-dir> [--ide <twinBASIC.exe>] [--port N] [--arch win32|win64] [--compile-timeout S] [--build-timeout S] [--compiler-run-timeout S] [--compiler-run-quiet S] [--exe-run-timeout S] [--timeout S] [--json] [--raw] [--keep] [--no-reap] [--reap-images a,b] [--show|--hide] [--llvm | --compiler-options S] [--exe | --tests [--only <names>]] [-h, --help]

Builds an exported twinBASIC source tree in the IDE, runs it, and prints what it
writes to the DEBUG CONSOLE. With --tests it builds nothing, and runs the
project's own test cases instead.

  --ide <path>        as tbbuild's
  --port <n>          DevTools port to start the IDE on (default: one Windows
                      picks, so runs at once never meet on a port)
  --arch <target>     win32 or win64 (default win32)
  --compile-timeout <secs>
                      the compile's limit, from the project reported open to
                      the compile settled (default: test/harness.json's
                      compile, or TB_COMPILE_TIMEOUT)
  --build-timeout <secs>
                      the build's limit, from the press to [BUILD] Executing
                      (default: its build, or TB_BUILD_TIMEOUT)
  --compiler-run-timeout <secs>
                      the probe's run's limit, from Executing (default: its
                      compilerRun, or TB_COMPILER_RUN_TIMEOUT); with --tests,
                      how long each case may run: one still running then is
                      reported "no answer", and the cases after it "not run",
                      since the compiler cannot stop it
  --compiler-run-quiet <secs>
                      the run is over after this long with no output, counted
                      from Executing or the last output, when the probe has not
                      returned (default: its compilerRunQuiet, or
                      TB_COMPILER_RUN_QUIET)
  --exe-run-timeout <secs>
                      with --exe, the exe's limit (default: its exeRun, or
                      TB_EXE_RUN_TIMEOUT)
  --timeout <secs>    both --compiler-run-timeout and --exe-run-timeout
  --json              emit one JSON object instead of text
  --raw               do not strip the console's timestamp column
  --keep              leave the IDE running afterwards, and print its pid
                      (implies --no-reap)
  --no-reap           do not harvest automation servers the probe left behind
  --reap-images a,b   comma-separated image names to harvest (default: the
                      Office suite)
  --show, --hide      as tbbuild's
  --llvm              compile the whole project with LLVM, as
                      --compiler-options +llvm
  --compiler-options <s>
                      the project's compiler options, for the run and the exe;
                      +llvm is refused on a Community or Personal licence
  --exe               also run the built exe on a private desktop, and print
                      what it writes with TbRun.Out, its exit code, the fault
                      the event log records for it, and the boxes it opens,
                      each closed with OK
  --tests             compile the project and do not build it: print PASS or
                      FAIL for each compile case (a line ending in the comment
                      ' CASE <Name>: <codes>, ' CASE <Name>: error or
                      ' CASE <Name>: none, which passes when the IDE reports
                      those diagnostic codes on that line, any error, or
                      nothing on it) and for each [TestCase] Sub of each
                      [TestFixture] Module, run in the compiler's test mode
  --only <names>      with --tests, only these cases: a comma-separated list of
                      Fixture.Sub names and compile case names
  -h, --help          print this text and exit

Exit codes:
  0  the probe ran and its output was captured
  1  the project has compile errors; the diagnostics are printed
  2  a refused command line (a source folder that is missing or has no Settings file
     included), no IDE, an IDE that did not start, a compile that never
     settled or met its limit, a build that failed after a clean compile (an LLVM
     refusal anywhere in its log, an exe linked or not, and a build that met its
     limit included), a probe that never ran or stopped at a procedure that failed
     code generation, test/harness.json missing or malformed, a TB_<PHASE> variable
     that is not a number of seconds greater than 0, an --llvm run on a
     Community or Personal licence, an --exe run with no exe built, --tests on a
     project with no case or with a [TestCase] it cannot call, or a crash
  3  no output: the console held none, or the probe printed none
     after its last Debug.Cls
  4  the compiler crashed, or restarted twice, while compiling the project
  5  the probe ended before it returned (End, or an error that ended the run), or
     stopped at a run-time error it does not handle, which is named; what it
     printed is printed all the same
  6  --exe: the exe exited with a code other than 0, the event log records that it
     faulted, it opened a box (closed by tbrun, as an unhandled error opens one), or it
     was still running at its limit and was ended; its output, exit code, fault and
     boxes are printed all the same
  7  --tests: a case failed; every case's line is printed all the same`;

const { values, positionals } = withUsageError(
  () =>
    parseCli(process.argv.slice(2), {
      options: {
        port: { type: "string" },
        arch: { type: "string" },
        timeout: { type: "string" },
        "compile-timeout": { type: "string" },
        "build-timeout": { type: "string" },
        "compiler-run-timeout": { type: "string" },
        "compiler-run-quiet": { type: "string" },
        "exe-run-timeout": { type: "string" },
        ide: { type: "string" },
        "reap-images": { type: "string" },
        "compiler-options": { type: "string" },
        llvm: { type: "boolean", default: false },
        exe: { type: "boolean", default: false },
        tests: { type: "boolean", default: false },
        only: { type: "string" },
        json: { type: "boolean", default: false },
        raw: { type: "boolean", default: false },
        keep: { type: "boolean", default: false },
        "no-reap": { type: "boolean", default: false },
        show: { type: "boolean", default: false },
        hide: { type: "boolean", default: false },
        help: { type: "boolean", short: "h", default: false },
      },
      positionals: { min: 0, max: 1 },
      stopAt: ["help"],
    }),
  { format: (err) => `${err.message}\n${USAGE}` },
);
if (values.help) printHelpAndExit(USAGE);

// The values are read before anything starts.
const { port, arch, seconds } = withUsageError(
  () => {
    refuseTogether(values, ["show", "hide"]);
    refuseTogether(values, ["timeout", "compiler-run-timeout"]);
    refuseTogether(values, ["timeout", "exe-run-timeout"]);
    refuseTogether(values, ["llvm", "compiler-options"]);
    refuseTogether(values, ["exe", "tests"]);
    if (values.only !== undefined && !values.tests) throw new CliError("needs-option", "--only needs --tests");
    return {
      port:
        values.port === undefined
          ? null
          : numberOption(values.port, { option: "--port", integer: true, min: 1, max: 65535 }),
      arch: choiceOption(values.arch ?? TARGETS[0], { option: "--arch", choices: TARGETS }),
      seconds: {
        compile: optionSeconds(values, ["compile-timeout"]),
        build: optionSeconds(values, ["build-timeout"]),
        compilerRun: optionSeconds(values, ["compiler-run-timeout", "timeout"]),
        compilerRunQuiet: optionSeconds(values, ["compiler-run-quiet"]),
        exeRun: optionSeconds(values, ["exe-run-timeout", "timeout"]),
      },
    };
  },
  { format: (err) => `${err.message}\n${USAGE}` },
);

// The limits, in ms: the option, else the TB_<PHASE> variable, else
// test/harness.json, whose faults are exit 2 (lib/harness-settings.mjs).
let limits;
try {
  limits = Object.fromEntries(Object.entries(seconds).map(([key, s]) => [key, timeoutMs(key, s)]));
} catch (e) {
  if (!(e instanceof HarnessError)) throw e;
  die(2, e.message);
}

if (!positionals.length) die(2, USAGE);

const srcDir = path.resolve(positionals[0]);
if (!existsSync(srcDir) || !statSync(srcDir).isDirectory()) {
  die(
    2,
    `not a directory: ${srcDir}\n` +
      `tbrun takes an exported source tree (the folder holding Sources/ and Settings), ` +
      `because it has to pin the build path before packing. Export one with:\n` +
      `  node scripts/impexp.mjs export <project.twinproj> <dir>`,
  );
}

const settingsPath = path.join(srcDir, "Settings");
if (!existsSync(settingsPath)) die(2, `no Settings file in ${srcDir}`);

// Images a probe can leave behind through COM activation. Office is the set that
// prompted this; --reap-images replaces the list for anything else. Only out-of-
// process (LocalServer32) servers can outlive the probe at all -- an in-process
// one dies with it -- so this list is short by nature rather than by omission.
const REAP_IMAGES = ["excel", "winword", "powerpnt", "msaccess", "outlook", "onenote", "mspub", "visio", "winproj"];

// ---------------------------------------------------------------- the IDE

// One install builds the tree. --ide, then TB_IDE, then the newest BETA on the
// Desktop, as for every other tool here.
const ide = findIde(values.ide || undefined);
const tooOld = unsupportedIde(ide);
if (tooOld) die(2, tooOld);
if (!ide || !existsSync(ide)) {
  die(2, "no twinBASIC IDE found. Pass --ide <twinBASIC.exe> or set TB_IDE.");
}

// ------------------------------------------------- pin the build output (1)

// (6) The workspace is keyed to the run's slot (tbbroker, WIP.Broker.md), which no
// other run on the machine holds while this one does; the IDE's private desktop
// and WebView2 user data folder are the slot's too. Sharing one %TEMP%/tbrun/src
// meant the second run deleted the first one's tree. The built file's path is in
// the --json report (`exe`) for a caller that wants it.
let slot;
try {
  slot = await leaseSlot({ tag: `tbrun ${path.basename(srcDir)}` });
} catch (e) {
  die(2, `tbrun: ${e.message}`);
}
const runKey = String(slot.slot);
const work = path.join(tmpdir(), "tbrun", runKey);
rmSync(work, { recursive: true, force: true });
mkdirSync(work, { recursive: true });

// Staged into a temp copy rather than edited in place, with buildPath pinned to
// the run's own out folder and a project.id keyed to the slot
// (lib/tb-project.mjs). The file is named as the IDE's own template names it,
// so the build type and the target are in the name (1).
const stage = path.join(work, "src");
const outDir = path.join(work, "out");
mkdirSync(outDir, { recursive: true });
const buildPath = path.join(outDir, "${ProjectName}_${Architecture}.${FileExtension}");
const projPath = path.join(work, "tbrun-probe.twinproj");

// Every .twin under dir, in its subfolders too, named by its path there with forward
// slashes, as a diagnostic row ends: "Tests/Probe.twin" for /<Project>/Sources/Tests/Probe.twin.
function twinFiles(dir) {
  return readdirSync(dir, { recursive: true })
    .filter((rel) => rel.toLowerCase().endsWith(".twin"))
    .sort()
    .map((rel) => ({ name: rel.split(path.sep).join("/"), text: readFileSync(path.join(dir, rel), "utf8") }));
}

const sourceFiles = (() => {
  const dir = path.join(srcDir, "Sources");
  if (!existsSync(dir)) return [];
  return twinFiles(dir);
})();
const sourceText = sourceFiles.map((f) => f.text).join(String.fromCharCode(10));
const hasHook = /\[RunAfterBuild\]/i.test(sourceText);
const hasCls = /Debug\s*\.\s*Cls/i.test(sourceText);
// --tests: the cases are found before an IDE starts, so a project with none, or
// with a [TestCase] the runner cannot call, costs nothing.
const cases = values.tests ? findCases(sourceFiles) : null;
if (cases) {
  if (cases.problems.length) die(2, `tbrun --tests:\n${cases.problems.map((p) => `  ${p}`).join("\n")}`);
  if (values.only !== undefined) {
    const wanted = new Set(values.only.split(",").map((n) => n.trim()));
    const known = new Set([...cases.tests.map((t) => `${t.fixture}.${t.name}`), ...cases.compiles.map((c) => c.name)]);
    const unknown = [...wanted].filter((n) => !known.has(n));
    if (unknown.length) die(2, `tbrun --tests --only: no such case: ${unknown.join(", ")}`);
    cases.tests = cases.tests.filter((t) => wanted.has(`${t.fixture}.${t.name}`));
    cases.compiles = cases.compiles.filter((c) => wanted.has(c.name));
  }
  if (!cases.tests.length && !cases.compiles.length) {
    die(
      2,
      "tbrun --tests: Sources/ holds no [TestCase] Sub in a [TestFixture] Module, " +
        "and no line ending in a ' CASE <Name>: <codes> comment",
    );
  }
}
// The console also carries the IDE's own build log, and the linker writes to it
// AFTER the build -- so a probe that does not clear the console first comes back
// with its output mixed into [LINKER] chatter. Debug.Cls as the probe's first
// statement is what makes the capture clean, and it is cheap to check for.
if (values.tests) {
  // nothing is built, so nothing runs after a build
} else if (!hasHook) {
  console.error(
    "warning: no [RunAfterBuild] in Sources/ -- nothing of yours will " +
      "run after the build, so you will capture the IDE's build log and nothing else.",
  );
} else if (!hasCls) {
  console.error(
    "warning: the probe does not call Debug.Cls -- the IDE's build log will be " +
      "mixed into the captured output. Make Debug.Cls the first statement.",
  );
}

// ------------------------------------------------------------------- pack

// A packing failure is the harness's, exit 2.
// The project's compiler options: compiler.debugOptions are what the
// [RunAfterBuild] run is compiled with, and compiler.buildOptions what the exe
// is -- on BETA 1010, +llvm in the build options alone leaves the run's
// Debug.Assert evaluated.
const compilerOptions = values.llvm ? "+llvm" : values.compilerOptions;
let wasTemplate = false,
  projectName = "",
  usesLlvm = false,
  wrap = null;
try {
  const staged = stageProject({
    src: srcDir,
    stage,
    project: projPath,
    settings: {
      "project.buildPath": buildPath,
      "project.id": laneProjectId(0, slot.slot),
      ...(compilerOptions === undefined
        ? {}
        : { "compiler.debugOptions": compilerOptions, "compiler.buildOptions": compilerOptions }),
    },
    prepare: (dir) => {
      const sources = path.join(dir, "Sources");
      const files = existsSync(sources) ? twinFiles(sources) : [];
      wrap = wrapProbe(files);
      for (const f of wrap.files) writeFileSync(path.join(sources, f.name), f.text, "utf8");
    },
  });
  wasTemplate = /\$\{/.test(staged.original["project.buildPath"] ?? "");
  projectName = String(staged.settings["project.name"] ?? "");
  // Any LLVM in the run: the project's options, or a procedure's own.
  usesLlvm = asksForLlvm(staged.settings, sourceText);
} catch (e) {
  die(2, e.message);
}
if (hasHook && !wrap.wrapped && !values.tests) {
  console.error(
    `warning: ${wrap.why} -- so a probe that ends before it returns cannot be told from one that finished.`,
  );
}

// What the build wrote: the IDE expands the template, so the name is looked for
// rather than assumed -- ${FileExtension} follows the build type -- and the
// binary is told from anything written beside it by its "MZ" header.
function builtFile() {
  const stem = `${projectName}_${arch}.`.toLowerCase();
  for (const f of readdirSync(outDir).filter((n) => n.toLowerCase().startsWith(stem))) {
    const file = path.join(outDir, f);
    try {
      if (readFileSync(file).subarray(0, 2).toString("latin1") === "MZ") return file;
    } catch {
      /* gone */
    }
  }
  return null;
}

// ---------------------------------------------------------------- compile

// A run with LLVM in it holds a lease on the LLVM cache server from before its
// IDE starts until that IDE has ended (shutdown), so the server is never one its
// IDE's compiler starts in the IDE's job, which would end with it and take the
// cache from every other run on the machine (lib/tb-llvm-cache.mjs).
let llvmLease = null;
if (usesLlvm) {
  try {
    llvmLease = await holdLlvmCache(ide);
  } catch (e) {
    die(2, `tbrun: ${e.message}`);
  }
}

// (7) Taken before the IDE starts, so anything in it is somebody else's and is
// never a candidate for harvesting. Cheap enough to be unconditional (~0.4 s
// against a ~10 s run) and skipping it under --no-reap would only make the two
// paths differ in a way nobody would remember.
const processesBefore = snapshotProcesses();

// Everything this run opens is under `work`, so the IDE's registry entries for
// it -- its saved project state, see lib/tb-registry.mjs --
// is swept by that folder once the IDE has exited, along with any an earlier
// run in this slot left behind. Not under --keep: a kept IDE is still writing.
const tidy = values.keep ? null : startTidy({ prefixes: [work] });

let ideRun = null;
// A failure before the console is read: said on stdout, as it was when this
// phase was tbbuild's output relayed, and ended with tbbuild's meaning of 1
// (compile errors) or 4 (the compiler crashed), and 2 for anything else.
function failBuild(code, text) {
  process.stdout.write(text + "\n");
  shutdown();
  process.exit(code);
}

try {
  ideRun = await launchIde({
    exe: ide,
    project: projPath,
    slot,
    port,
    keep: values.keep,
    show: wantShow({ show: values.show, hide: values.hide }),
  });
} catch (e) {
  failBuild(2, e.message);
}
// A kept IDE keeps the slot after this run has ended, until it is closed.
if (values.keep) await slot.handTo(ideRun.pid);

const cdp = await attachIde(ideRun.port);
if (!cdp) failBuild(2, "the IDE never exposed a debug port");

let outcome = compileOutcome(await waitForCompile(cdp, { project: projPath, timeout: limits.compile }), {
  name: projPath,
});
if (!outcome.ok) failBuild(outcome.code === 4 ? 4 : 2, outcome.message);

// The target, set on every run, win32 included (setBuildTarget says why). The
// probe runs in the compiler that builds it, so under win64 it runs in the
// IDE's 64-bit compiler, twinBASIC_win64_noDEP.exe, as a 64-bit process:
// on BETA 1010 LenB of a LongPtr is 8 and PROCESSOR_ARCHITECTURE reads AMD64.
try {
  const target = await setBuildTarget(cdp, arch, { project: projPath, timeout: limits.compile });
  // Only a target the IDE remembered is worth a word: a new path opens in win32.
  if (target.from !== TARGETS[0]) {
    console.error(`note: the IDE remembered ${target.from} for this path; the probe is built for ${arch}`);
  }
  if (target.waited) {
    outcome = compileOutcome(target.waited, { name: projPath });
    if (!outcome.ok) failBuild(outcome.code === 4 ? 4 : 2, outcome.message);
  }
} catch (e) {
  failBuild(2, e.message);
}
// --tests judges the diagnostics itself: a compile case may expect an error.
if (outcome.counts[0] > 0 && !values.tests) {
  failBuild(1, [...outcome.rows, summaryLine(outcome.counts)].join("\n"));
}

// The licence, for a run with LLVM in it: llvmLicence says why a Community or
// Personal one measures nothing.
let licence = null;
if (usesLlvm) {
  const found = await llvmLicence(cdp);
  licence = found.licence;
  if (found.refusal) failBuild(2, `tbrun: ${found.refusal}`);
}

if (values.tests) await runCases();

// --------------------------------------------- build the exe, read the console

let captured = null,
  capture = null,
  shown = null,
  erased = null,
  failure = null;
try {
  // (4) the clears are kept, (2) the build button is pressed, (5) and the wait
  // settles on the sentinel, or else a quiet period: lib/tb-run.mjs.
  const run = await captureRun(cdp, {
    done: (lines) => wrap.wrapped && sentinelIndex(lines) >= 0,
    buildMs: limits.build,
    runMs: limits.compilerRun,
    quietMs: limits.compilerRunQuiet,
  });
  captured = run.captured;
  capture = run;
  erased = run.erased;
  // --raw changes what is printed, never what is checked. BUILD_FAILED needs a
  // line that starts where the console's text does, and a line holding only a
  // timestamp is never blank, so every check reads without the column; under
  // --raw the lines printed are the same entries, read again with it.
  shown = values.raw ? strip(run.last, await readConsole(cdp, { timestamps: true })) : captured;
  cdp.close();
} catch (e) {
  failure = e.message;
}

const reaped = shutdown();

if (failure) die(2, `tbrun: ${failure}`);
// A build that fails after a clean compile never runs the probe, and leaves the
// IDE's own build log in the console. The probe's first statement is Debug.Cls,
// which would have erased that log, so its survival means the capture is not the
// probe's output. Returned as output, a failed build's log would exit 0.
// A [RunAfterBuild] Sub
// that fails code generation leaves the log too: the build succeeds, and then
// nothing in the Sub runs, Debug.Cls included. BUILD_FAILED is buildProject's
// list of failure lines, the code-generation one among them.
const { started, buildFailed, lost } = checkCapture(captured, erased);
// A build that met its limit failed, and says so as a timeout.
const buildTimedOut = buildTimeoutMessage(capture);
if (buildTimedOut && buildFailed === undefined && !lost) {
  die(
    2,
    `tbrun: ${buildTimedOut}` + (shown.length ? `; the console held:\n${shown.map((l) => `  ${l}`).join("\n")}` : ""),
  );
}
if (buildFailed !== undefined) {
  die(
    2,
    "tbrun: the build or the probe's code generation failed, so the probe never ran. " +
      "The console holds the IDE's build log, not the probe's output:\n" +
      shown.map((l) => `  ${l}`).join("\n"),
  );
}
// A procedure the probe calls that fails code generation is reported straight
// after the "[BUILD] Executing '<project>.<module>.<Sub>'..." line, before the
// probe's first statement runs. So Debug.Cls erases the report, the probe stops
// where it calls that procedure, and the console holds only what it printed
// before then -- which tbrun once returned as the whole output, exit 0. A failure line among what the clears erased counts only after the
// last Executing line: before it is the build's own log, which ended in success
// or the probe would not have run (checkCapture).
if (lost) {
  die(
    2,
    "tbrun: the probe's Debug.Cls erased a failure the IDE reported as the probe started:\n" +
      `  ${lost}\n` +
      "What the probe printed, which stops where it called the procedure that failed:\n" +
      (shown.length ? shown.map((l) => `  ${l}`).join("\n") : "  (nothing)"),
  );
}
// (5) The wrapper's sentinel says the probe returned. It is the wrapper's line,
// not the probe's, so it is never printed. The IDE's own lines after it stay:
// one says the probe left a form loaded.
const at = wrap.wrapped ? sentinelIndex(captured) : -1;
const returned = wrap.wrapped ? at >= 0 : null;
if (returned) {
  // Without --raw, shown is captured itself.
  if (shown !== captured) shown.splice(at, 1);
  captured.splice(at, 1);
}
// A probe that ran leaves its Executing line among what its Debug.Cls erased,
// so an empty console then means it printed nothing after its last clear, not
// that it never ran.
const printed = returned ? at : captured.length;
if (!printed && started >= 0 && returned !== false) {
  die(3, `tbrun: the probe ran (${erased[started]}) but printed nothing after its last Debug.Cls.`);
}
if (!captured.length && started < 0) {
  die(
    3,
    "tbrun: the build produced no console output.\n" +
      (hasHook
        ? "  The [RunAfterBuild] Sub may not have run -- check the IDE for a modal."
        : "  There is no [RunAfterBuild] Sub to produce any.") +
      (wasTemplate ? "" : "\n  buildPath was already explicit, so a Save dialog is unlikely."),
  );
}

const exeRun = values.exe ? await runExe() : null;

if (values.json) {
  console.log(
    JSON.stringify(
      {
        exe: builtFile(),
        arch,
        lines: shown,
        returned,
        stopped: capture.stopped,
        licence,
        llvmCache: leaseReport(llvmLease),
        exeRun,
        idePid: ideRun?.pid ?? null,
        reaped,
      },
      null,
      2,
    ),
  );
} else {
  for (const l of shown) console.log(l);
  if (exeRun) {
    console.log(
      `--- exe: ${exeRun.timedOut ? `still running after ${secondsText(limits.exeRun)}, ended` : `exit ${exitText(exeRun.exitCode)}`}`,
    );
    if (exeRun.fault) console.log(`--- exe: ${faultText(exeRun.fault)}`);
    for (const d of exeRun.dialogs) console.log(`--- exe: a box, closed: ${dialogText(d)}`);
    for (const l of exeRun.lines) console.log(l);
  }
  // Only under --keep, where the pid is still alive and therefore actionable.
  if (values.keep && ideRun?.pid) for (const l of keptIdeLines(ideRun.pid)) console.log(l);
}
// An error the probe does not handle stops it at the IDE's error panel, which waits for a
// person, so the probe never returns; the panel's text says why (captureRun).
if (capture.stopped && returned !== true) {
  const s = capture.stopped;
  const where = s.file ? `, in ${s.file}${s.line === null ? "" : ` at line ${s.line}`}` : "";
  die(5, `tbrun: the probe stopped at a run-time error it does not handle${where}: ${s.header}: ${s.error}`);
}
if (returned === false) {
  die(
    5,
    `tbrun: the probe ended before it returned${started >= 0 ? "" : ", and the run's start was not seen"}: ` +
      "End, or an error that ended the run without a report -- as one raised with no handler in " +
      "LLVM-compiled code does. Or it was still running, silent, after the run's quiet period " +
      `(${secondsText(limits.compilerRunQuiet)}) or its limit (${secondsText(limits.compilerRun)}); ` +
      "raise --compiler-run-quiet or --compiler-run-timeout for a slow probe.",
  );
}
if (exeRun?.timedOut) {
  die(
    6,
    `tbrun: timeout: the exe was still running after ${secondsText(limits.exeRun)}, and was ended ` +
      "(test/harness.json's exeRun; --exe-run-timeout or TB_EXE_RUN_TIMEOUT overrides it).",
  );
}
if (exeRun && exeRun.exitCode !== 0) die(6, `tbrun: the exe exited with code ${exitText(exeRun.exitCode)}.`);
if (exeRun?.fault) die(6, `tbrun: the exe exited with code 0, and the event log records that it faulted.`);
if (exeRun?.dialogs.length) die(6, `tbrun: the exe opened a box, which tbrun closed: an unhandled error opens one.`);
// Explicitly: under --keep the launcher, and the pipes to it, live as long as the
// IDE, and would keep this process waiting for it.
process.exit(0);

// ------------------------------------------------------------------ helpers

// --tests: the compile cases judged by the diagnostics the compile reported, and
// each [TestCase] Sub run in the compiler's test mode (lib/tb-tests.mjs), one at a
// time, the compilerRun limit each. A project with errors runs none of them, and says so for
// each. A case that is still running at that limit is reported "no answer" and the
// cases after it are reported "not run", not sent: the compiler cannot cancel it.
// The diagnostics follow the cases' lines, which are what a reproducer's expect
// reads. Never returns.
async function runCases() {
  const results = judgeCompiles(cases.compiles, outcome.rows).map((r) => ({ ...r }));
  const errors = outcome.counts[0] > 0;
  try {
    // A case still running at its limit cannot be stopped, and the compiler's main thread refuses every
    // request until it ends ("main thread is busy"), so the cases after it are not sent.
    let stalled = null;
    for (const t of cases.tests) {
      const name = `${t.fixture}.${t.name}`;
      if (errors) results.push({ name, pass: false, detail: "not run: the project has compile errors" });
      else if (stalled) results.push({ name, pass: false, detail: `not run: ${stalled} did not return` });
      else {
        const r = await runTest(cdp, t, limits.compilerRun);
        results.push({ name, ...r });
        if (r.timedOut) stalled = name;
      }
    }
    cdp.close();
  } catch (e) {
    shutdown();
    die(2, `tbrun: ${e.message}`);
  }
  const reaped = shutdown();
  const failed = results.filter((r) => !r.pass).length;
  if (values.json) {
    console.log(
      JSON.stringify(
        {
          arch,
          cases: results,
          diagnostics: outcome.rows,
          licence,
          llvmCache: leaseReport(llvmLease),
          idePid: ideRun?.pid ?? null,
          reaped,
        },
        null,
        2,
      ),
    );
  } else {
    for (const r of results) console.log(caseLine(r.name, r));
    if (outcome.rows.length) {
      console.log("diagnostics:");
      for (const row of outcome.rows) console.log(`  ${row}`);
    }
    if (values.keep && ideRun?.pid) for (const l of keptIdeLines(ideRun.pid)) console.log(l);
  }
  if (failed) die(7, `tbrun: ${failed} of ${results.length} case${results.length === 1 ? "" : "s"} failed.`);
  process.exit(0);
}

// --exe: the built exe, started as the IDE is, on a private desktop and inside
// a kill-on-close job, so a window it opens -- a MsgBox -- is on no desktop
// anyone uses, and nothing it starts outlives it. It inherits no handles, and
// Debug.Print writes nothing in an exe, so what it printed is what TbRun.Out
// appended to the file TBRUN_OUT names.
//
// An unhandled error opens a box and waits for it, so the launcher closes every
// box the exe opens and records it; without that the exe sat until its limit.
// The box is the only sign of the error: in BETA 1010 a plain exe exits with
// code 0 once it is closed ("Run-time error '11' Division by zero"), and an
// LLVM-compiled one ("_TB_ERROR_HANDLER", "unhandled error in ProbeMod.Oops")
// then dies of an access violation. The event log's record of a fault is read
// for the same reason: an exit code need not say it (lib/win-fault.mjs).
async function runExe() {
  const file = builtFile();
  if (!file || !/\.exe$/i.test(file)) {
    die(2, `tbrun: --exe, but the build wrote no exe to ${outDir}${file ? ` (it wrote ${path.basename(file)})` : ""}`);
  }
  const outFile = path.join(work, "exe-out.txt");
  rmSync(outFile, { force: true });
  const since = Date.now();
  let run;
  try {
    run = await launchOnDesktop({
      exe: file,
      tag: `tbrun --exe ${path.basename(file)}`,
      env: { ...process.env, TBRUN_OUT: outFile },
      dialogs: "close",
    });
  } catch (e) {
    die(2, `tbrun: could not start the exe on a private desktop: ${e.message}`);
  }
  let timer;
  const timedOut = await Promise.race([
    run.exited.then(() => false),
    new Promise((r) => {
      timer = setTimeout(() => r(true), limits.exeRun);
    }),
  ]);
  clearTimeout(timer);
  if (timedOut) {
    killTree(run.pid);
    run.launcher.kill();
  }
  const { code: exitCode, dialogs } = await run.finished;
  const text = existsSync(outFile) ? readFileSync(outFile, "utf8") : "";
  const fault = timedOut ? null : recordedFault(file, since);
  return { file, exitCode: timedOut ? null : exitCode, timedOut, dialogs, fault, lines: strip(text) };
}

/** An exit code, with the hex of a code that is an NTSTATUS, such as 0xC0000005 for an access violation. */
function exitText(code) {
  return code !== null && (code < 0 || code > 0xffff)
    ? `${code} (0x${(code >>> 0).toString(16).toUpperCase().padStart(8, "0")})`
    : String(code);
}

/** A box the exe opened, on one line: its title and its text. */
function dialogText({ title, text }) {
  return `${title}: ${text.replace(/\s*\r?\n\s*/g, " ")}`;
}

// (6) End OUR IDE by pid, never by image name. The tree kill takes the probe exe
// and anything it spawned with CreateProcess; what it cannot take is a COM
// server, which is what reapOrphans is for.
function shutdown() {
  if (values.keep) return null; // the IDE is the caller's problem now
  shutdownIde(ideRun);
  llvmLease?.releaseSync();
  finishTidy(tidy);
  return values.noReap ? null : reapOrphans();
}

// Identity is pid + start time, because a pid alone is reused and a run that
// reaped a recycled pid would be killing a stranger.
function snapshotProcesses() {
  const ps =
    "Get-Process | Select-Object Id, ProcessName, " +
    "@{n='Start';e={try{$_.StartTime.ToFileTimeUtc()}catch{0}}}, " +
    "@{n='Win';e={$_.MainWindowTitle}} | ConvertTo-Json -Compress";
  try {
    const out = execFileSync("powershell", ["-NoProfile", "-NonInteractive", "-Command", ps], {
      encoding: "utf8",
      stdio: ["ignore", "pipe", "ignore"],
      maxBuffer: 32 * 1024 * 1024,
    });
    const parsed = JSON.parse(out);
    return new Map((Array.isArray(parsed) ? parsed : [parsed]).map((p) => [`${p.Id}:${p.Start}`, p]));
  } catch {
    return null; // reaping degrades to off, never to guessing
  }
}

// A COM server started by the probe has svchost.exe for a parent (measured), so
// there is no ancestry to walk and this has to be a before/after diff. Three
// guard rails, because a diff over a live machine is a blunt instrument:
//
//   - it must be NEW -- present now, absent from the pre-launch snapshot;
//   - its image must be on REAP_IMAGES, so an unrelated process that happened
//     to start during the run is never a candidate;
//   - it must have NO main window, which is what separates a server the probe
//     activated from the copy of Excel the user opened to look at a spreadsheet.
//
// Anything new and on the list but WINDOWED is reported and left alone. That is
// the case where the evidence is ambiguous, and killing it could discard
// somebody's unsaved work.
//
// Known limit: two concurrent runs both driving Excel cannot tell their servers
// apart, so whichever finishes first harvests both. Pass --no-reap for that and
// sweep once at the end of the batch.
function reapOrphans() {
  if (!processesBefore) return null;
  const after = snapshotProcesses();
  if (!after) return null;

  const images = new Set(
    (values.reapImages || "")
      .split(",")
      .map((s) =>
        s
          .trim()
          .toLowerCase()
          .replace(/\.exe$/, ""),
      )
      .filter(Boolean),
  );
  const wanted = images.size ? images : new Set(REAP_IMAGES);

  const killed = [],
    skipped = [];
  for (const [key, p] of after) {
    if (processesBefore.has(key)) continue;
    if (!wanted.has(String(p.ProcessName).toLowerCase())) continue;
    if (p.Win && String(p.Win).trim()) {
      skipped.push(p);
      continue;
    }
    killTree(p.Id);
    killed.push({ pid: p.Id, image: p.ProcessName });
  }

  for (const p of skipped) {
    console.error(
      `note: ${p.ProcessName} (pid ${p.Id}) started during this run but has a ` +
        `window open, so it was left alone -- close it yourself if it is a leak.`,
    );
  }
  if (killed.length) {
    console.error("reaped: " + killed.map((k) => `${k.image} (pid ${k.pid})`).join(", "));
  }
  return killed;
}