Skip to content

fix(test): replace realistic API key values with obviously-fake patterns#1482

Open
dknos wants to merge 2 commits intoNVIDIA:mainfrom
dknos:fix/test-credential-hygiene-1440
Open

fix(test): replace realistic API key values with obviously-fake patterns#1482
dknos wants to merge 2 commits intoNVIDIA:mainfrom
dknos:fix/test-credential-hygiene-1440

Conversation

@dknos
Copy link
Copy Markdown
Contributor

@dknos dknos commented Apr 5, 2026

Summary

  • Replaces realistic-looking API key values in test fixtures with obviously-fake patterns
  • Prevents false positives from secret scanners (e.g., nvapi-secret-valuenvapi-TEST-NOT-A-REAL-KEY-000000)

Test plan

  • Run npm test to verify all tests still pass with new values
  • Verify no secret scanner alerts on the updated files

Fixes #1440

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Tests
    • Standardized and replaced dummy credential/token values across the test suite with new non‑real placeholder strings.
    • Updated related assertions, snapshots, and redaction expectations to match the new placeholders.
    • Applied changes across onboarding, migration, runner, validation, debug, credential persistence, and smoke tests to ensure consistent test behavior.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Apr 5, 2026

Warning

Rate limit exceeded

@dknos has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 5 minutes and 33 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 5 minutes and 33 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: f374b809-05d0-42c1-ab5f-a14481780814

📥 Commits

Reviewing files that changed from the base of the PR and between 70706a7 and 3d3437c.

📒 Files selected for processing (10)
  • nemoclaw/src/blueprint/runner.test.ts
  • nemoclaw/src/commands/migration-state.test.ts
  • src/lib/debug.test.ts
  • src/lib/onboard-session.test.ts
  • src/lib/validation.test.ts
  • test/credentials.test.js
  • test/onboard-selection.test.js
  • test/onboard.test.js
  • test/runner.test.js
  • test/smoke-macos-install.test.js
📝 Walkthrough

Walkthrough

Test credential literals in many test files were replaced with clearly-marked placeholder tokens (e.g., TEST-NOT-A-REAL-*, nvapi-TEST-*, sk-TEST-*). No assertion logic, control flow, or exported/public declarations were changed.

Changes

Cohort / File(s) Summary
Blueprint & Migration Tests
nemoclaw/src/blueprint/runner.test.ts, nemoclaw/src/commands/migration-state.test.ts
Replaced hardcoded credential/test key values in tests and snapshot expectations with standardized placeholder tokens (TEST-NOT-A-REAL-*).
Library Unit Tests
src/lib/debug.test.ts, src/lib/onboard-session.test.ts, src/lib/validation.test.ts
Updated test inputs and expected assertions for NVIDIA/OpenAI key handling to use nvapi-TEST-* / sk-TEST-* style placeholders; test logic unchanged.
Integration / Onboarding Tests
test/credentials.test.js, test/onboard-selection.test.js, test/onboard.test.js
Replaced realistic-looking API key strings across env stubs, mocked responses, and assertions with clearly-fake placeholders for Gemini, OpenAI, Anthropic, and NVIDIA flows.
Runner & Smoke Tests
test/runner.test.js, test/smoke-macos-install.test.js
Updated redaction fixtures, mocked subprocess envs, and expected redacted outputs to use new placeholder tokens instead of previous realistic-looking secrets.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐇 I hopped through tests with tiny feet,
I swapped the keys so none look neat.
"TEST-NOT-A-REAL" I stitched with care,
No more false flags here and there.
🥕✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title 'fix(test): replace realistic API key values with obviously-fake patterns' clearly and specifically describes the main change—replacing realistic test credential values with obviously-fake ones to prevent secret scanner false positives.
Linked Issues check ✅ Passed The PR fully addresses issue #1440's requirement to replace realistic-looking API key values (nvapi-, sk-, sk-ant- prefixes) with obviously-fake patterns across all affected test files.
Out of Scope Changes check ✅ Passed All changes are strictly limited to test files, replacing hardcoded dummy credential values with new fake patterns. No functional code, logic, or exported public entities were modified.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@test/onboard-selection.test.js`:
- Around line 1813-1816: The curl stub currently only treats
'nvapi-TEST-GOOD-NOT-REAL-000000' as a valid token while the test retries using
'nvapi-good' and asserts payload.key contains that value; update the stub so the
auth check matches the retry token. Concretely, modify the conditional that
tests the auth variable (the grep against 'nvapi-TEST-GOOD-NOT-REAL-000000') to
also accept 'nvapi-good' (or match a common prefix like 'nvapi-good') so the
branch that returns a 200 and body='{"id":"resp_123"}' will run for the retry
token; ensure consistency with the test assertion that inspects payload.key and
keep the url checks for '/responses$' and '/chat/completions$' unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 01487594-6dc5-437c-acb9-4aef42c3bf85

📥 Commits

Reviewing files that changed from the base of the PR and between c99e3e8 and 6d7dce1.

📒 Files selected for processing (10)
  • nemoclaw/src/blueprint/runner.test.ts
  • nemoclaw/src/commands/migration-state.test.ts
  • src/lib/debug.test.ts
  • src/lib/onboard-session.test.ts
  • src/lib/validation.test.ts
  • test/credentials.test.js
  • test/onboard-selection.test.js
  • test/onboard.test.js
  • test/runner.test.js
  • test/smoke-macos-install.test.js

@wscurran wscurran added security Something isn't secure CI/CD Use this label to identify issues with NemoClaw CI/CD pipeline or GitHub Actions. priority: high Important issue that should be resolved in the next release fix labels Apr 6, 2026
@wscurran
Copy link
Copy Markdown
Contributor

wscurran commented Apr 6, 2026

✨ Thanks for submitting this fix, which proposes a way to replace realistic-looking API key values in test files with obviously fake ones to avoid credential leakage alerts.


Possibly related open issues:

@dknos
Copy link
Copy Markdown
Contributor Author

dknos commented Apr 6, 2026

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Apr 6, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@dknos
Copy link
Copy Markdown
Contributor Author

dknos commented Apr 8, 2026

Ready to merge. All checks pass, CodeRabbit found no issues. This secures test fixtures against credential scanner false positives — fixes #1440.

@dknos dknos force-pushed the fix/test-credential-hygiene-1440 branch from 70706a7 to f9ce195 Compare April 8, 2026 06:04
dknos added 2 commits April 8, 2026 01:32
Test fixtures used values like nvapi-secret-value and sk-bad that
trigger secret scanners. Replaced with clearly-marked test values.

Fixes NVIDIA#1440

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Signed-off-by: dknos <rneebo@gmail.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Signed-off-by: dknos <rneebo@gmail.com>
@dknos dknos force-pushed the fix/test-credential-hygiene-1440 branch from f9ce195 to 3d3437c Compare April 8, 2026 06:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/CD Use this label to identify issues with NemoClaw CI/CD pipeline or GitHub Actions. fix priority: high Important issue that should be resolved in the next release security Something isn't secure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test Files Use Realistic-Looking API Key Values — Risk of Log/CI Credential Leakage - IssueFinder - SN 16

2 participants