Skip to content

feat: store temporary ID maps to file and include in safe outputs artifact#25828

Merged
pelikhan merged 1 commit intomainfrom
copilot/store-temporary-ids-in-safe-outputs
Apr 11, 2026
Merged

feat: store temporary ID maps to file and include in safe outputs artifact#25828
pelikhan merged 1 commit intomainfrom
copilot/store-temporary-ids-in-safe-outputs

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 11, 2026

Summary

Store the temporary ID maps in safe outputs to a file and include it in the safe outputs artifact for review and audit.

During safe output processing, the handler manager now writes the complete temporary ID map (aw_*{repo, number}) to /tmp/gh-aw/temporary-id-map.json. This file is uploaded alongside the existing safe-output-items.jsonl manifest in the safe-outputs-items artifact.

Changes

JavaScript (actions/setup/js/)

  • constants.cjs: Added TEMPORARY_ID_MAP_FILE_PATH constant (/tmp/gh-aw/temporary-id-map.json)
  • safe_output_manifest.cjs: Added writeTemporaryIdMapFile() function that writes the temporary ID map as pretty-printed JSON to a file, with directory creation and error handling
  • safe_output_handler_manager.cjs: After exporting the temporary ID map via core.setOutput(), also writes it to disk via writeTemporaryIdMapFile() (skipped in staged mode)
  • constants.test.cjs: Added test for the new TEMPORARY_ID_MAP_FILE_PATH constant
  • safe_output_manifest.test.cjs: Added tests for writeTemporaryIdMapFile() covering empty maps, maps with entries, directory creation, file overwriting, and error handling

Go (pkg/)

  • pkg/constants/job_constants.go: Added TemporaryIdMapFilename constant (temporary-id-map.json)
  • pkg/constants/constants_test.go: Added test assertion for the new constant
  • pkg/workflow/compiler_safe_outputs_job.go: Updated buildSafeOutputItemsManifestUploadStep() to include both safe-output-items.jsonl and temporary-id-map.json in the artifact upload using multiline path syntax

Compiled Workflows (.github/workflows/*.lock.yml)

All lock files recompiled to reflect the updated artifact upload path.

File produced at runtime

// /tmp/gh-aw/temporary-id-map.json
{
  "aw_abc123": {
    "repo": "owner/repo",
    "number": 42
  },
  "aw_def456": {
    "repo": "owner/repo",
    "number": 100
  }
}

Testing

  • ✅ Go constants tests pass (TestConstantValues/TemporaryIdMapFilename)
  • ✅ Go compiler tests pass (TestBuildConsolidatedSafeOutputsJob, TestJobOutputs, TestStepOrder*)
  • ✅ JavaScript functionality manually verified (file writing, pretty-printing, directory creation)
  • make fmt and make lint pass
  • make recompile succeeds
  • ✅ Code review passed with no issues

…ifact

Write the temporary ID map (aw_* → {repo, number}) to
/tmp/gh-aw/temporary-id-map.json during safe output processing.
Include this file alongside the existing safe-output-items.jsonl
in the safe-outputs-items artifact for review and audit.

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/a69d61aa-3e4f-4aa3-8672-06f44bed5bb5

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI requested a review from pelikhan April 11, 2026 15:40
@pelikhan pelikhan marked this pull request as ready for review April 11, 2026 15:43
Copilot AI review requested due to automatic review settings April 11, 2026 15:43
@pelikhan pelikhan merged commit 525b5b7 into main Apr 11, 2026
70 of 71 checks passed
@pelikhan pelikhan deleted the copilot/store-temporary-ids-in-safe-outputs branch April 11, 2026 15:46
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds persistence of the safe-outputs “temporary ID map” to disk and includes it in the safe-outputs-items artifact to enable post-run review/audit without relying on step outputs.

Changes:

  • Add a JS constant for /tmp/gh-aw/temporary-id-map.json and implement writeTemporaryIdMapFile() to write the full map as pretty-printed JSON.
  • Update the safe output handler manager to write the temporary ID map file (non-staged mode) in addition to exporting it as an action output.
  • Update the Go workflow compiler to upload both safe-output-items.jsonl and temporary-id-map.json in the safe-outputs-items artifact (and recompile workflow lock files accordingly).
Show a summary per file
File Description
actions/setup/js/constants.cjs Defines TEMPORARY_ID_MAP_FILE_PATH under /tmp/gh-aw/ for runtime output.
actions/setup/js/constants.test.cjs Adds coverage ensuring the new constant is exported and under TMP_GH_AW_PATH.
actions/setup/js/safe_output_manifest.cjs Implements writeTemporaryIdMapFile() and exports the new helper/constant.
actions/setup/js/safe_output_manifest.test.cjs Adds tests for writing the temp ID map file (empty/non-empty, mkdirs, overwrite, error path).
actions/setup/js/safe_output_handler_manager.cjs Writes the temp ID map JSON file after processing (skipped in staged mode).
pkg/constants/job_constants.go Introduces TemporaryIdMapFilename used for workflow artifact upload paths.
pkg/constants/constants_test.go Adds assertion for TemporaryIdMapFilename.
pkg/workflow/compiler_safe_outputs_job.go Updates artifact upload step to include both manifest and temp ID map via multiline path.
.github/workflows/*.lock.yml Recompiled workflow lock files to upload the additional temporary-id-map.json file.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 187/187 changed files
  • Comments generated: 4

Comment on lines +122 to +126
// TemporaryIdMapFilename is the filename of the temporary ID map JSON file written to /tmp/gh-aw/.
// This file contains a JSON object mapping temporary IDs (e.g., aw_abc123) to their resolved
// GitHub resource references ({repo, number}) for review and audit purposes.
// It is uploaded alongside the safe-output-items.jsonl manifest in the safe-outputs-items artifact.
const TemporaryIdMapFilename = "temporary-id-map.json"
Copy link

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment for TemporaryIdMapFilename says the JSON maps temporary IDs to {repo, number} pairs, but the JS implementation stores other shapes in the temporary ID map too (e.g. { projectUrl } in actions/setup/js/create_project.cjs:438-443 and { draftItemId } in actions/setup/js/update_project.cjs:836-841). Please update this docstring to reflect that values can be different resolved reference types (or explicitly constrain what gets written to this file).

Copilot uses AI. Check for mistakes.
Comment on lines +146 to +152
* The file contains a pretty-printed JSON object mapping temporary IDs to their resolved
* GitHub resource references for review and audit purposes.
*
* @param {Object} temporaryIdMap - The temporary ID map object (keys are temp IDs, values are {repo, number})
* @param {string} [filePath] - Path to the output file (defaults to TEMPORARY_ID_MAP_FILE_PATH)
*/
function writeTemporaryIdMapFile(temporaryIdMap, filePath = TEMPORARY_ID_MAP_FILE_PATH) {
Copy link

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

writeTemporaryIdMapFile docstring/param typing says map values are {repo, number}, but the temporary ID map can contain other resolved reference shapes (e.g. { projectUrl } / { draftItemId }). This makes the artifact format misleading for auditors/consumers. Please relax/clarify the JSDoc to match actual contents (or filter the map before writing if only repo/number pairs are intended).

Copilot uses AI. Check for mistakes.
Comment on lines +1162 to +1167
// Write temporary ID map to file for inclusion in the safe-outputs-items artifact.
// This allows reviewers and auditors to inspect the full map of temporary IDs
// to resolved GitHub resources (issue numbers, repos) without parsing step outputs.
if (!isStaged) {
writeTemporaryIdMapFile(processingResult.temporaryIdMap);
core.info(`Wrote temporary ID map to file for artifact upload`);
Copy link

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The inline comment describes the temporary ID map as mapping to resolved issue {repo, number} references, but the map can also contain non-issue mappings (e.g. project URLs, draft item IDs) depending on which handlers ran. Consider rewording this comment (and the core.info message if needed) to avoid implying the artifact only contains issue mappings.

Copilot uses AI. Check for mistakes.
Comment on lines +352 to +362
it("should create parent directories if they do not exist", () => {
const deepPath = `/tmp/test-temp-id-deep-${Math.random().toString(36).substring(7)}/a/b/c/map.json`;
writeTemporaryIdMapFile({ aw_test: { repo: "o/r", number: 1 } }, deepPath);

expect(fs.existsSync(deepPath)).toBe(true);
const parsed = JSON.parse(fs.readFileSync(deepPath, "utf8"));
expect(parsed.aw_test).toEqual({ repo: "o/r", number: 1 });

// Cleanup
fs.rmSync(path.dirname(deepPath).split("/a/")[0] + "/a", { recursive: true, force: true });
});
Copy link

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test creates a deep temp directory under /tmp/test-temp-id-deep-*, but the manual cleanup only removes the /a subdirectory, leaving the parent directory behind. Over many test runs this can leak directories in /tmp and make failures harder to debug. Prefer deleting the top-level temp directory you created (e.g., the /tmp/test-temp-id-deep-* prefix) instead of reconstructing a partial path with split("/a/").

Copilot uses AI. Check for mistakes.
@github-actions github-actions bot mentioned this pull request Apr 11, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

Test Quality Score: 65/100

⚠️ Acceptable, with suggestions

Metric Value
New/modified tests analyzed 8
✅ Design tests (behavioral contracts) 6 (75%)
⚠️ Implementation tests (low value) 2 (25%)
Tests with error/edge cases 4 (50%)
Duplicate test clusters 0
Test inflation detected ⚠️ YES — safe_output_manifest.test.cjs (3.1:1 ratio)
🚨 Coding-guideline violations 0

Test Classification Details

View all 8 test cases
Test File Classification Issues Detected
should export TEMPORARY_ID_MAP_FILE_PATH under TMP_GH_AW_PATH constants.test.cjs:44 ✅ Design Verifies structural invariant: path is under TMP_GH_AW_PATH
should be the expected default path safe_output_manifest.test.cjs ⚠️ Implementation Pure constant value check; partially duplicates the constants.test.cjs assertion
should write an empty map as pretty-printed JSON safe_output_manifest.test.cjs ✅ Design Edge case: empty map; verifies file I/O output format
should write a map with entries as pretty-printed JSON safe_output_manifest.test.cjs ✅ Design Verifies JSON structure, indentation, and trailing newline
should create parent directories if they do not exist safe_output_manifest.test.cjs ✅ Design Edge case: deep nested path; verifies mkdir-p behaviour
should overwrite an existing file safe_output_manifest.test.cjs ✅ Design Edge case: overwrite; verifies idempotent write semantics
should throw when the file cannot be written safe_output_manifest.test.cjs ✅ Design Error path: asserts the wrapped error message propagates correctly
TestConstantValuesTemporaryIdMapFilename row pkg/constants/constants_test.go:240 ⚠️ Implementation Constant snapshot check only; no structural invariant

Flagged Tests — Suggestions

⚠️ should be the expected default path (safe_output_manifest.test.cjs)

Classification: Implementation test
Issue: Asserts TEMPORARY_ID_MAP_FILE_PATH === "/tmp/gh-aw/temporary-id-map.json" — a constant snapshot that is already covered (with an additional structural invariant) in constants.test.cjs.
What design invariant does this test enforce? None beyond the identical check in the other file.
What would break if deleted? Nothing that isn't already caught by constants.test.cjs.
Suggested improvement: Either remove this isolated check (trust the canonical constant test) or elevate it by adding the startsWith(TMP_GH_AW_PATH) guard like its sister test, so it enforces the "must live under the temp directory" invariant.


⚠️ TestConstantValuesTemporaryIdMapFilename row (pkg/constants/constants_test.go:240)

Classification: Implementation test
Issue: Checks that TemporaryIdMapFilename == "temporary-id-map.json". This is a constant snapshot with no structural constraint.
What design invariant does this test enforce? None — it will only fail if the literal string changes.
What would break if deleted? Only a rename of the constant value; no observable system behaviour would regress.
Suggested improvement: This is fine as a regression guard, but adding a check that the filename has no path separators (i.e., it is a plain filename, not a path) would enforce a real design invariant and increase the value of the test row.


Test Inflation Note

safe_output_manifest.test.cjs added 78 lines against 25 lines in the production file — a 3.1:1 ratio (threshold: 2:1). This is not inherently harmful; the extra test lines come from five meaningful behavioural scenarios plus proper beforeEach/afterEach teardown. No action required, but worth noting.


Language Support

Tests analyzed:

  • 🐹 Go (*_test.go): 1 new test case (table row in TestConstantValues, //go:build !integration ✅)
  • 🟨 JavaScript (*.test.cjs): 7 tests (vitest, no mock libraries used)

Verdict

Check passed. 25% of new tests are implementation tests (threshold: 30%). The writeTemporaryIdMapFile suite in safe_output_manifest.test.cjs is particularly strong — it covers empty inputs, nested directory creation, overwrite semantics, and error propagation with real file I/O.


📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

  • Assert on observable outputs, return values, or state changes
  • Cover error paths and boundary conditions
  • Would catch a behavioral regression if deleted
  • Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

  • Assert on internal function calls (mocking internals)
  • Only test the happy path with typical inputs
  • Break during legitimate refactoring even when behavior is correct
  • Give false assurance: they pass even when the system is wrong

Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.

🧪 Test quality analysis by Test Quality Sentinel · ● 685K ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 65/100. Test quality is acceptable — 25% of new tests are implementation tests (threshold: 30%). The writeTemporaryIdMapFile behavioural suite is well-structured with real file I/O, covering empty inputs, nested directories, overwrite semantics, and error propagation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants