Repository navigation
redact process stdout/stderr writes in-process (opt-in) - #1181
Conversation
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
varlock-website | a33978e | Commit Preview URL Branch Preview URL |
Oct 07 2026, 10:46 PM |
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
Important
Held output can be lost when a patched stream is ended before the holdback timer flushes.
Reviewed changes across the in-process stdout/stderr redaction path, its schema and serialized settings, parent-child handoff, shared redactor optimizations, reveal handling, tests, docs, and benchmark tooling.
- Stream interception: Adds opt-in patching for Node process streams and Bun writes, including UTF-8 handling and split-secret holdback.
- Configuration: Adds independent
consoleandstdoutcontrols plus per-item log-redaction exemptions. - Parent handoff: Marks streams already redacted by
varlock runorvarlock proxy runto avoid duplicate processing. - Shared redaction: Optimizes regex and holdback lookup behavior and moves unmask-marker removal to the final output layer.
- Coverage and guidance: Adds unit and smoke tests, reference documentation, a changeset, and a manual benchmark.
gpt-5.6-sol ｜ 𝕏
📦 Bundle size
dist/ only; native binaries are versioned separately and not counted here. |
There was a problem hiding this comment.
Caution
This update can leave leak-detected secrets in client-facing Next.js artifacts.
Reviewed changes since the prior Pullfrog review: the per-item log-redaction exemption was extended to the public redaction helper while leak-prevention scrubbing received a separate all-values path.
- Redaction semantics: Changed
redactSensitiveConfig()to honor@sensitive={redactLogs=false}and addedredactAllSensitiveValues()for contexts that must scrub every leak-scanned value. - Response protection: Updated patched server responses to use all-values scrubbing and added coverage for log-exempt secrets.
- Documentation: Clarified that
redactLogs=falsealso affectsredactSensitiveConfig()but not response leak detection.
gpt-5.6-sol ｜ 𝕏
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
@varlock/ci-env-info
varlock
@varlock/astro-integration
@varlock/cloudflare-integration
@varlock/nextjs-integration
@varlock/nuxt-integration
@varlock/vite-integration
@varlock/1password-plugin
@varlock/aws-secrets-plugin
@varlock/aws-sigv4-plugin
@varlock/azure-key-vault-plugin
@varlock/bitwarden-plugin
@varlock/dashlane-plugin
@varlock/doppler-plugin
@varlock/google-secret-manager-plugin
@varlock/hashicorp-vault-plugin
@varlock/infisical-plugin
@varlock/keepass-plugin
@varlock/keeper-plugin
@varlock/kubernetes-plugin
@varlock/pass-plugin
@varlock/passbolt-plugin
@varlock/proton-pass-plugin
@varlock/native-helper-darwin
@varlock/native-helper-linux-arm64
@varlock/native-helper-linux-x64
@varlock/native-helper-win32-x64
commit: |
|
The changes in this PR will be included in the next version bump.
|
|
@pullfrog - review this |
There was a problem hiding this comment.
Caution
The new stream protection still leaks complete secrets in two reproducible cases. Pending write errors and framework initialization also need attention before merging.
Reviewed changes across the process-stream patch, configuration serialization, console hooks, leak remediation, documentation, benchmarks, and tests.
- Process-stream redaction: Opt-in stdout/stderr patching, UTF-8 buffering, callbacks, final flushing, Bun writes, and parent-redaction markers.
- Configuration: Independent console/stdout settings and per-item log-redaction exemptions, while retaining leak detection.
- Output handling: Indexed holdback matching, faster replacement, reveal-marker handling, and Next.js artifact scrubbing.
- Verification: The package build and 128 focused tests passed; separate reproductions exposed the inline findings. Smoke tests could not run because the standalone smoke package lacks installed Vitest (
vitest: command not found).
⚠️ Vite SSR initialization never enables the new protection
The advertised framework support is incomplete: Vite's init-only and resolved-env SSR initialization calls the existing patches but never patchProcessStreams(). Setting @redactLogs={stdout=true} therefore leaves direct stream writes unprotected on these paths, including Astro configurations that select resolved-env.
Technical details
# Missing framework stream initialization
## Affected sites
- `packages/integrations/vite/src/index.ts:344`: `init-only` is the default SSR injection mode.
- `packages/integrations/vite/src/index.ts:466-493`: generated non-auto-load initialization imports/calls console and response patches, but no process-stream patch.
- `packages/integrations/astro/src/index.ts:128,146`: configurations selecting `resolved-env` use this generated path.
## Required outcome
- Non-edge SSR initialization must enable process-stream protection when the schema or environment opts in, without requiring an additional application import.
- Keep edge initialization free of Node-only stream dependencies.
- Cover the generated initialization path with a test that verifies direct stdout output, not just console output.
## Suggested approach
- Wire the stream patch into Vite's non-edge initialization with a lightweight published runtime entry point, consistent with its existing patch imports.gpt-6.1-sol ｜ 𝕏
There was a problem hiding this comment.
Important
Mixed Bun and process-stream writes can report callback success before an earlier write fails.
Reviewed changes since 7f4412f3, focusing on the shared streaming state, callback error handling, and framework initialization.
- Preserved complete matches: Added boundary selection that avoids cutting complete sensitive values, including self-overlap, in both stream writers.
- Shared Bun holdback: Routed Bun text writes through the process-stream holdback state and extended split-write smoke scenarios.
- Observed prefix errors: Added error callbacks for emitted prefixes and real-Writable tests for both auto-destroy modes.
- Unified initialization: Moved process-stream patching into
patchGlobalConsole()so existing framework initialization enables it, without a static filesystem import.
The package build and 52 focused tests passed. Built-package Bun split-write checks also passed; the smoke suite could not load because vitest/config is missing from its local dependency resolution.
gpt-6.1-sol ｜ 𝕏
There was a problem hiding this comment.
Important
The new write route fixes the earlier callback-transfer bug, but introduces two error-handling regressions that should be addressed before merging.
Reviewed changes since 80837bad, focusing on Bun text routing and the revised acknowledgement of held writes.
- Routed Bun text writes through the process-stream writer instead of an independent Bun promise.
- Changed write acknowledgement to complete after the emitted portion, allowing sequential awaited writes to retain split-secret protection.
- Added regression coverage for mixed-API errors and sequential split writes, and updated held-write callback assertions.
The package build and all 54 focused tests passed. Separate Node and real-Bun reproductions exposed the inline findings.
gpt-6.1-sol ｜ 𝕏
Adds @redactLogs={console, stdout} options. stdout=true patches
process.stdout/stderr.write (and Bun.write to them) in auto-load and
init-server, redacting non-TTY streams like varlock run does. Opt-in for
now via the schema or _VARLOCK_REDACT_STDOUT=1.
Also adds @sensitive={redactLogs=false} to exempt a value from output
redaction, skips the in-process patch on streams a varlock run parent
already redacts, and makes the streaming holdback check much faster.
Fixes #1149
bench:redaction measures per-write CPU cost and end-to-end cost under node and bun for no redaction, console only, in-process stdout, and varlock run, with a realistic 10-secret schema generated at run time. The holdback check now only tries suffix lengths whose end char can match, which cuts it from ~3.5us to ~0.05us per log line under node.
Node's console writes through process.stdout/stderr, so with the stream patch active the kWriteToConsole hook redacted every line twice. Also make the benchmark realistic: secrets appear in one line per run by default, with --secret-every N for a worst case.
Optional unmask-marker groups around the secret alternation made every scan 2-5x slower; check the markers in the replace fn instead, and wrap the alternation in a single group (faster in JSC). Also pass a write callback straight through when nothing is held back.
The markers were never stripped, so revealed values printed as "👁 value 👁", and on bun the value was not marked at all (and got redacted). The layer that writes output last now strips the markers: the stream patch, node's console internals, or the console method wrapper where nothing runs after it.
- accept "true"/"false" strings for dynamic @redactLogs values (env overrides resolve to strings), instead of failing the load - tag the parent redaction marker with varlock run's pid, so a grandchild piping elsewhere (node app | tee log) redacts its own output - keep redactLogs=false values in response scrubbing and the public redactSensitiveConfig; only log/stdout redaction skips them - an opt-out on one item no longer weakens a value it shares with another - hold back a trailing unmask marker so split revealed values print - drop settings that disappear on env reload
The public redact fn now leaves log-exempt values alone; only leak prevention's response scrubbing still redacts them.
…facts - the stream patch now writes held-back text before end(), so it is not lost or written after the stream closes - Next.js leak remediation for built files uses all-values redaction, so a redactLogs=false value caught by the leak scan is not written back - remove a stray file committed by accident
…reams from patchGlobalConsole - the stream holdback never cuts through a complete value; a value whose ending is also the start of a value (e.g. secret-token-s) was emitted in two unredacted pieces (in-process and in varlock run) - Bun.write to stdout/stderr shares holdback state with process.stdout.write, so values split across writes of either kind are caught - a write error on already-emitted text reaches callbacks waiting on held text - patchGlobalConsole() also patches the streams, so integrations that only call it (vite/astro SSR init) get stdout redaction; the stream patch no longer imports node:fs (uses process.getBuiltinModule) so it is edge-safe
Bun.write text now goes through the patched process stream, so it shares the write queue (ordering and errors) as well as the holdback state. Write callbacks fire once the writable part is out instead of waiting on held-back text. Waiting made writers that await each write (e.g. awaited Bun.write calls) force every held prefix out by timeout, unredacted, before the rest of the value arrived. Held text still goes out on the next write, end(), the flush timer, or process exit.
74c5148 to
c57e6fd
Compare
- Bun.write shares holdback state with the process stream but writes with Bun's original writer again, so a failure (e.g. EPIPE) only rejects the promise instead of also emitting an unhandled stream error - writes to an ended, destroyed or errored stream pass straight to the original write, so they report ERR_STREAM_WRITE_AFTER_END etc. instead of being buffered and acknowledged
There was a problem hiding this comment.
Important
No new issues found, but the existing Bun failure-handling concern is only partially addressed: a caught failed write with a held suffix still causes an uncaught EPIPE during the later flush. Remaining prior feedback prevents approval.
Reviewed changes since the prior substantive Pullfrog review, including the mechanical intermediate update and the latest writer/error-handling fixes.
- Restored Bun's original writer while retaining shared split-secret holdback with process-stream writes.
- Delegated closed process-stream writes to the original writer before buffering, preserving ended/destroyed errors.
- Added regression coverage for Bun promise-only errors and writes after stream closure.
The package build and all 56 focused tests passed. Real-Bun closed-pipe checks confirmed that plain-text promise rejection is fixed, but the existing held-tail failure persists. The closed-process-stream thread was resolved.
gpt-6.1-sol ｜ 𝕏
The in-process stream patch no longer buffers partial matches between writes. Each write (and end()'s final chunk) is redacted and passed straight to the original write with the same arguments, callback and return value, so stream semantics are untouched by construction. A value split across writes is still detected: the stream remembers the end of the previous write that could start a value, masks the part that arrives in the next write, and prints a one-time warning naming the item, since the first part was already written. varlock run keeps its holdback buffering.
There was a problem hiding this comment.
Caution
The new buffer handling can emit a complete secret unchanged when a write boundary falls inside one of its UTF-8 characters.
Reviewed changes since 8518662f, focusing on the no-holdback rewrite and its updated output contract.
- Replaced delayed output buffering with immediate redaction that forwards each call to the original writer, including
end()final chunks. - Added split-value detection using shared carry across process-stream and Bun writes, masking the later portion and issuing a one-time warning.
- Revised UTF-8 handling and coverage to pass boundary bytes through immediately, and updated tests and documentation for partial exposure.
The package build and all 54 focused tests passed. Separate source and built-package reproductions confirmed the inline leak; the earlier Bun and callback error threads were resolved.
gpt-6.1-sol ｜ 𝕏
| const start = leadingContinuationBytes(bytes); | ||
| const end = bytes.length - incompleteUtf8TailLength(bytes.subarray(start)); | ||
| let text: string; | ||
| try { | ||
| text = utf8Decoder.decode(bytes.subarray(start, end)); |
There was a problem hiding this comment.
[P1] A character split across buffers is excluded from both decoded chunks, so a sensitive value containing that character never matches even though all its bytes are printed. Registering super-sécret-value-12345 and splitting its buffer between é's C3 and A9 bytes emits the entire secret unchanged, with no split warning, rather than masking the later portion.
Technical details
# Split UTF-8 characters bypass carry detection
## Affected sites
- `packages/varlock/src/runtime/patch-process-streams.ts:114-125`: excludes incomplete trailing bytes and leading continuation bytes from decoded matching text, then forwards both buffers unchanged when no match is found.
- `packages/varlock/src/runtime/env.ts:533-545`: searches `carry + text`, which is missing the split character.
- `packages/varlock/src/runtime/test/patch-process-streams.test.ts:132-137`: splits a multibyte character outside the secret, leaving the ASCII secret complete in the second buffer.
## Reproduction
- Register sensitive value `super-sécret-value-12345` and patch a recording stream.
- Create `Buffer.from(secret + '\n')` and split at `bytes.indexOf(0xC3) + 1`.
- Write each slice and concatenate the emitted buffers.
- Actual output is the complete secret followed by a newline, with no warning. Matching instead sees `super-scret-value-12345`.
- Confirmed against source under Bun and the built package under Node. A boundary immediately before `é` is correctly detected.
## Required outcome
- Carry detection must include multibyte characters completed across buffer boundaries so the later portion is masked and the warning is emitted.
- Preserve immediate output and native callback/error behavior without restoring delayed flushes.
- Add regression coverage with the boundary inside a multibyte character that belongs to the registered sensitive value.… output When a chunk ends with what may be the start of a sensitive value, varlock run holds it back for up to 100ms. After that it was written as-is and forgotten, so if the child paused mid-value (e.g. block-buffered piped output) the rest arrived unrecognized and the whole value was printed. The writer now remembers the flushed fragment; when the next chunk completes a value across it, that part is masked and a one-time warning names the item. The held text is still flushed on time, so prompts are not stalled, and a flushed lookalike that never completes a value is left alone.
There was a problem hiding this comment.
Important
No new issues found in this delta. The existing split UTF-8 secret leak remains reproducible and unresolved, so approval is still blocked.
Reviewed changes since 716010cb, focusing on CLI redaction after timeout flushes.
- Added CLI carry tracking so
varlock runmasks the remainder of a value whose prefix was already printed after the 100ms timeout, with a one-time warning. - Extracted shared matching helpers for split-value completion and trailing partial-value carry without changing the in-process output contract.
- Added timeout regression coverage for delayed completions, multiple pauses, and lookalike prefixes, and documented the behavior in the secrets guide.
All 57 focused tests passed. An additional exact-output check passed all 253 three-chunk flush partitions of the test secret. Build and smoke tests were not rerun for this delta.
gpt-6.1-sol ｜ 𝕏
… stream code out of edge bundles - Remove @sensitive={redactLogs=false} for now (and with it the second redaction regex, the module-level all-values toggle, redactAllSensitiveValues and the Next.js leak-scrub shim). It only matters once in-process stdout redaction is the default, so it will ship with that change. - @redactLogs takes only {stdout=...}; console redaction is off only via @redactLogs=false. On Node stdout=true already covers console output. - patchGlobalConsole no longer patches the process streams. patchProcessStreams is exported from varlock/patch-console and called by auto-load, init-server and the node integrations, so init-edge and the Cloudflare init stay flat. - Move the streaming helpers (redactStreamWrite, getStreamHoldbackLength, ...) out of env.ts into runtime/lib/redact-stream.ts. - Holdback index uses two Maps instead of parallel ascii-array/Map buckets. - Benchmark uses Math.random (closes the CodeQL biased-random alert). - Docs: note the one split case in-process redaction cannot detect (a multi-byte character split across byte writes inside a sensitive value).
There was a problem hiding this comment.
Important
No new issues found in this delta. The existing split UTF-8 secret leak is now documented but remains reproducible, so that thread remains open and approval is still blocked.
Reviewed changes since the prior substantive review at dd657f77, including the documentation-only update at 19b53452 and the latest scope reduction.
- Removed per-item log exemptions and the separate all-values scrubbing helper, restoring one redactor for logs and leak remediation.
- Simplified configuration to the
stdoutoption, with console redaction disabled only by@redactLogs=false. - Separated stream initialization from
patchGlobalConsole()and added explicit calls in Node entry points and framework integrations, including generated non-edge SSR code. - Consolidated streaming helpers and simplified the holdback index to Map-based buckets.
- Documented split-write limitations, exposure guidance, and process-level stdout opt-outs.
Runtime and modified integration builds passed, along with 210 focused tests. Edge-bundle checks found no new runtime failure. Smoke tests and the website build were not rerun.
gpt-6.1-sol ｜ 𝕏
…s=false}
The console option is not redundant with stdout: stream redaction skips
interactive terminals, so console redaction is the only thing hiding values
from a human at the keyboard. {console=false, stdout=true} is a coherent
setting (see secrets in your terminal, never in piped output).
The per-item opt-out keeps an item sensitive (leak-scanned, kept out of
client bundles) while letting a program print it. Leak remediation scrubs
these values too via redactAllSensitiveValues. Reimplemented by passing the
active find/replace down through the redaction walk instead of a
module-level toggle.
There was a problem hiding this comment.
Caution
Restoring per-item log exemptions introduces a reproducible leak of a different, non-exempt secret across in-process writes. The existing split UTF-8 leak also remains unresolved.
Reviewed changes since 34827597, focusing on the restored configuration options and separation of output redaction from leak remediation.
- Restored independent console controls alongside the opt-in stdout/stderr setting.
- Restored per-item log exemptions in schema validation, serialization, and recursive output redaction, while retaining leak scanning.
- Restored all-values scrubbing for server responses and Next.js build artifacts, including older-peer fallback.
- Extended tests and documentation for exempt values, configuration independence, and intentional output.
Runtime and Next.js integration builds passed, along with 179 focused tests. Independent source and built-runtime reproductions confirmed the new inline finding. Smoke tests and the website build were not rerun.
gpt-6.1-sol ｜ 𝕏
The decorator now covers stdout data and varlock run output, not just console logs, and will cover everything a process emits once stream redaction becomes the default. @redact is the name for that. - @redactLogs still works and warns; using both is an error. - @sensitive={redact=false} replaces redactLogs=false on items, with the old option name accepted as a deprecated alias. - The serialized settings key stays redactLogs so blobs stay readable across CLI/runtime versions. - Docs, CLI help, VSCode catalog and the smoke test use the new name.
theoephraim
changed the title
There was a problem hiding this comment.
Important
No new issues found in this delta. The existing overlapping-value leak and split UTF-8 leak remain reproducible, so approval is still blocked.
Reviewed changes since ec2b186e, focusing on the decorator rename and deprecated aliases.
- Renamed the root decorator to
@redact, retaining@redactLogswith a warning and rejecting schemas that use both names. - Renamed the per-item output option and serialized item metadata to
redact, retainingredactLogsas a deprecated schema option while preserving the shippedsettings.redactLogsroot key. - Updated documentation and tooling across CLI help, VS Code completion, benchmarks, fixtures, and regression tests to use the new names.
Runtime, Next.js integration, and website builds passed, along with 168 focused tests. Source reproductions confirmed both existing leak findings; smoke tests were not rerun.
gpt-6.1-sol ｜ 𝕏
Mirrors the decorator one to one and folds in the never-shipped redactStdout key. settings.redactLogs is still written as a mirror of redact.console for runtimes older than 1.22, and the runtime falls back to it when a blob from an older CLI has no redact key. Both go away in the next major with the @redactLogs alias.
There was a problem hiding this comment.
Important
No new issues found in this delta. The existing overlapping-value leak and split UTF-8 leak remain reproducible, so approval is still blocked.
Reviewed changes since 7affeb71, focusing on root-setting serialization and its consumers.
- Serialized root redaction options as
settings.redact = { console, stdout? }, preserving the distinct defaults whenstdoutis omitted. - Retained older-runtime compatibility through the boolean
settings.redactLogsmirror and a runtime fallback for older CLI blobs. - Updated console, process-stream, and CLI consumers to read the new object, and adjusted schema, stream, and telemetry assertions.
The runtime build and all 143 focused tests passed. Independent checks verified legacy fallback, new-setting precedence, stale-setting cleanup on reload, and both existing leak cases. Smoke tests and website/integration builds were not rerun.
gpt-6.1-sol ｜ 𝕏
The completion and boundary-crossing checks walked the redaction regex, whose exec() only yields non-overlapping matches: a complete value inside already-written text hid an overlapping occurrence extending into the current write. With values aabaa (protected) and aabaab (log-exempt), writing aabaabaa then baa printed the protected value intact. The same happens with two protected values, so this was not specific to the exemption. Both checks now test each candidate start position directly (bounded by the longest value). varlock run's holdback also now holds a value that would be cut by the boundary back whole, instead of emitting it early: an occurrence overlapping an emitted one could otherwise be completed by the next chunk unnoticed. Benchmark uses crypto.randomInt, which is unbiased and keeps CodeQL quiet about secret-looking variable names.
There was a problem hiding this comment.
Important
The revised CLI holdback can retain continuously arriving overlapping output without limit. The existing split UTF-8 leak also remains reproducible and unresolved.
Reviewed changes since 4ac44aa8, focusing on overlapping-value matching and CLI emission boundaries.
- Replaced regex traversal with candidate-start checks that detect overlapping protected values, including overlaps with exempt values.
- Changed CLI holdback to retain complete values crossed by an emission boundary instead of emitting them early.
- Added overlap regressions for in-process and CLI writers, and switched benchmark value generation to
crypto.randomInt.
The runtime build and all 71 focused tests passed. Independent reproductions verified the overlap fix, the remaining UTF-8 leak, and the new holdback issue. Smoke tests and website/integration builds were not rerun.
gpt-6.1-sol ｜ 𝕏
Random overlapping secrets over a two-letter alphabet, random chunking, fixed seeds. The varlock run writer must reproduce single-shot redaction exactly when chunks arrive back to back, and must never emit a protected value when its flush timeout fires between chunks. The in-process patch must never emit a protected value for any split. Re-introducing either of the two bugs fixed in the previous commit fails these tests.
There was a problem hiding this comment.
Important
No new issues found in this delta. The existing unbounded CLI holdback and split UTF-8 leak remain reproducible and unresolved, so approval is still blocked.
Reviewed changes since d525334f, focusing on the new randomized streaming regression coverage. No runtime code changed.
- Added single-shot equivalence checks for back-to-back CLI chunks and whole-write in-process output.
- Added split-output leak checks for CLI timeout flushes and immediate in-process writes, using fixed seeds, overlapping ASCII values, and output exemptions across 12,000 randomized rounds.
All 60 focused tests passed, including the four new fuzz tests, and lint of the new file passed. Independent source checks confirmed both existing findings. Builds and smoke tests were not rerun for this tests-only delta.
gpt-6.1-sol ｜ 𝕏
Still fails on both re-introduced bugs; runs in about 130ms.
theoephraim
changed the title
Holding a value that would be cut by the holdback boundary back whole means an overlap chain (secret-token-s, ecret-token-s, ecret-token-s, ...) can never be cut without splitting a value. With the flush timer reset on every write, continuous output of that shape was buffered until the child went quiet. The timer is now a deadline from the first held byte, so held output is bounded by one interval of output and the whole chain goes through single-shot redaction, which masks overlapping occurrences correctly.
There was a problem hiding this comment.
Important
No new issues found in this delta. The existing split UTF-8 leak remains reproducible and unresolved, so approval is still blocked.
Reviewed changes since the previously reviewed version at 926d1b36, focusing on sustained CLI output and flush timing.
- Changed CLI flushing from a per-write debounce to a deadline from the first held text, allowing continuous overlapping output to progress.
- Added timer regressions for sustained overlap chains and writes that arrive before the original deadline.
The runtime build and all 62 focused tests passed. An independent real-time reproduction confirmed output progress without a complete-value leak. Smoke tests and website/integration builds were not rerun.
gpt-6.1-sol ｜ 𝕏
A Buffer write that ends mid-character left the dangling bytes out of the text used for matching on both sides, so a sensitive value containing an accented letter or emoji, split exactly inside that character, printed whole with no warning. The incomplete sequence is now kept, completed by the next write, and fed to the matcher as already-written text (like the carry), so the rest of the value is masked and the one-time warning fires. Output bytes are unchanged. Drops the doc caveat for this case.
There was a problem hiding this comment.
Important
No new issues found, but the existing split UTF-8 leak is only partially addressed. Splitting super-secret-value-12345é between the final character's C3 and A9 bytes still emits the complete secret unchanged, without a warning, so that thread remains open and approval is blocked.
Reviewed changes since e4758c6e, focusing on UTF-8 reconstruction across immediate byte writes.
- Added pending-byte reconstruction so characters completed by later writes participate in split-value matching without delaying output.
- Added exact-output regressions for splits inside non-final two-byte and three-byte characters, including a character spread over three writes.
- Removed the documented multibyte limitation from the secrets guide.
The runtime build and all 67 focused tests passed. Independent source partition checks and built-export Node reproductions confirmed the original internal-character fix and the remaining final-character leak: the completed character is added to already-written carry, but matching requires the value to extend beyond that boundary. Smoke tests and website/integration builds were not rerun.
gpt-6.1-sol ｜ 𝕏
A guide section and a SECURITY.md scope list stating what redaction and leak prevention are (guardrails against accidental exposure by trusted code) and are not (a sandbox), with the known limitations spelled out: uncovered write paths, transformed values, TTY output, split values, overlapping values, encodings, opt-outs, the visible two-char prefix, and what leak prevention does not scan. Reports that reduce to one of these are handled as issues, not advisories.
Fixes #1149.
varlock/auto-loadonly redactedconsoleoutput, so anything writing toprocess.stdout/process.stderrdirectly (pino, CLI frameworks,child.stdout.pipe(process.stdout),Bun.write(Bun.stdout)) leaked secrets unless the app ran undervarlock run.What changed
@redactLogsrenamed to@redact, with an options object:@redact={console=true, stdout=true}. The decorator now covers stdout data andvarlock runoutput, not just console logs, so the old name was misleading.@redactLogskeeps working as a deprecated alias (loads with a warning; using both is an error). The blob carries it assettings.redact = { console, stdout? }, mirroring the decorator;settings.redactLogsis still written as a mirror ofredact.consolefor runtimes older than 1.22, and new runtimes fall back to it when a blob from an older CLI has noredactkey. Alias and mirror both go away in the next major.stdout=truepatchesprocess.stdout.write/process.stderr.write(andBun.writeto them), redacting streams that are not a TTY, the same rulevarlock runuses. One setting now controls both in-process andvarlock runredaction;@redact=falsestill turns everything off, and@redact={stdout=false}turns off only stream redaction (for a process whose stdout is a data channel).consoleis separate because console redaction also applies on an interactive terminal, so{console=false, stdout=true}shows values in your own terminal while redacting anything piped.stdout=trueor_VARLOCK_REDACT_STDOUT=1.varlock runkeeps redacting by default. The plan is to make in-process redaction the default in the next major, since it can change output that is data rather than logs ($(node print-token.js), generators writing to a file).@sensitive={redact=false}exempts a value that is meant to be printed from console/stdout redaction andredactSensitiveConfig(), while keeping it sensitive: it is still leak-scanned, kept out of client bundles, and still scrubbed from responses and Next.js build output when a leak is caught (redactAllSensitiveValues).patchProcessStreamsis exported fromvarlock/patch-consoleand called next topatchGlobalConsolebyauto-load,init-server, and the Vite, Next.js and Expo integrations. It is deliberately not called frompatchGlobalConsoleitself, so the edge bundles (init-edge, the Cloudflare init) do not carry node-only stream code.end()'s final chunk) is redacted and passed straight to the originalwritewith the same arguments, callback and return value. Nothing is held back, so stream behavior is unchanged. A value split across separate writes (including mixedprocess.stdout.write/Bun.write) is caught when its second part arrives: that part is masked and a one-time warning names the item, since the first part was already printed. Binary buffers and non-UTF-8 encodings pass through.varlock runstill holds back partial matches, and tags its child with__VARLOCK_REDACTED_STREAMS, so its direct child skips streams the parent already redacts.varlock runand response scanning:varlock run's holdback never cuts through a complete value (a value likesecret-token-sused to print in two unredacted pieces);varlock runflushes a held partial match after its 100ms timeout, it now remembers it, so if the child pauses mid-value the rest is masked when it arrives (with a one-time warning) instead of the whole value printing;revealSensitiveConfig()fix: it printed its 👁 markers around the value and did not work on Bun. The layer that writes output last now strips them.bun run bench:redaction(manual, not in CI) measures the cost with a realistic 10-secret schema under Node and Bun.Cost
With realistic output (secrets almost never present), in-process stdout redaction adds roughly 0.5-1µs per log line on Node and Bun, and processes bulk output at roughly 0.4-1GB/s. See
packages/varlock/scripts/bench-stdout-redaction.ts.Scope
The secrets guide gains a Scope and known limitations section and
SECURITY.mda matching scope list: redaction and leak prevention are guardrails against accidental exposure by trusted code, not a sandbox, and the known limitations (uncovered write paths, transformed values, TTY output, split values, overlapping values, encodings, opt-outs, the visible two-character prefix, what leak prevention does not scan) are written down so they are triaged as issues rather than advisories.Not covered in-process
Writes that bypass the stream objects (
fs.writeSync(1, ...),pino.destination(), worker-thread transports) are only redacted undervarlock run.