Repository navigation
fix(analyzer): treat quoted $(printf ... | ...) as runtime data, not a parse limit - #1109
Varshith-Kali wants to merge 4 commits into
Conversation
Backtick substitutions keep their runtime-command gap: a backtick printf piped into another command is the runtime command itself, so its failure to resolve statically is still a coverage gap. The $(...) form is runtime-dependent data, as before. Fixes the two test-unit failures on NVIDIA#1109 in test_table_column_lookahead_preserves_runtime_command_gaps (backtick printf 'x' | cat in a markdown table cell). Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
|
Fixed the 2 test-unit failures on this head in fc47c50. Root cause: the pipeline treatment in the #1107 fix also applied to backtick substitutions, so a backtick |
yashrajp22
left a comment
There was a problem hiding this comment.
The quoted pipeline case is handled, but executable substitutions can now lose their coverage warning, and the new operator scan bypasses cancellation. Please keep the exemption scoped to proven data arguments and preserve the runtime check.
| recognized, _ = _printf_invocation_arguments( | ||
| inner, check_runtime=runtime_check, diagnostics=invocation_diagnostics | ||
| ) | ||
| if recognized and not backtick and _printf_invocation_has_following_operator(inner): |
There was a problem hiding this comment.
Could we limit this exemption to proven quoted data arguments? It also applies when the substitution is the command itself. For example, $(printf '%s%s' r 'm -rf /' | cat) supplies the command and all its arguments through word splitting. This early return drops the reconstruction diagnostic, while the outer-operand check has nothing left to inspect. Please retain the coverage warning for executable substitutions while allowing the quoted printf argument from the issue.
There was a problem hiding this comment.
Good catch, fixed in bbf57b7. The exemption is now scoped to proven quoted data arguments: _is_printf_substitution and _static_printf_substitution only take it when the caller proves the substitution sits inside a double-quoted span, via the quoted_result flag and the quote-limits map for candidates the word reader starts on directly. An unquoted $(printf ... | cat) in command position keeps its reconstruction diagnostic, pinned by a new regression test using your rm -rf / example. The quoted case from #1107 still completes, and backtick substitutions keep their runtime-command gap.
| """ | ||
| cursor = 0 | ||
| while True: | ||
| word, cursor, limited = _next_shell_invocation_word(inner, cursor, lambda: None) |
There was a problem hiding this comment.
Please pass the caller's runtime check through this scan. _is_printf_substitution can reach it with input larger than _PRINTF_STATIC_CHARS, and quoted words have no _SHELL_COMMAND_WORD_CHARS cap. Using lambda: None makes this added full pass ignore cancellation and the analyzer deadline while reading a long quoted argument.
There was a problem hiding this comment.
Fixed in bbf57b7. _printf_invocation_has_following_operator now takes the caller runtime check and passes it to the word reader instead of lambda: None, threaded through from every call site. Added a regression test with a 9000-character quoted argument that raises on cancellation.
The NVIDIA#1107 exemption treated any $(printf ... | ...) substitution as runtime data, including one in command position: $(printf '%s%s' r 'm -rf /' | cat) supplies the command and its arguments through word splitting, and the early return dropped the reconstruction diagnostic while the outer-operand check had nothing left to inspect. Scope the exemption to proven quoted data arguments: _is_printf_substitution and _static_printf_substitution now take it only when the caller proves the substitution sits inside a double-quoted span, via the quoted_result flag and the quote-limits map for candidates the word reader starts on directly. A bare substitution keeps its coverage warning, and backtick substitutions keep their runtime-command gap. Thread the caller's runtime check through the operator scan as well: _printf_invocation_has_following_operator took lambda: None, so the added full pass ignored cancellation and the analyzer deadline while reading a long quoted argument, which has no _SHELL_COMMAND_WORD_CHARS cap. It now receives the caller's check from every call site. Addresses review feedback on NVIDIA#1109. Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Decision — Request changes.
Value and readiness — The cron quoting idiom should complete static inspection when the runtime value is data. The current revision preserves the unquoted-command diagnostic and threads the runtime check through the added scan. It still treats quote provenance as proof of argument position.
Material findings
static_patterns_tool_misuse.py:2523–2541allows the exemption whenquoted_resultis true. A shell can execute a double-quoted substitution as the command name. For example,"$(printf '%s' "$tool_name" | cat)"uses the resulting single word as the executable. The quoted-word reader and quote-limits fallback both take the exemption here, and there is no outer operand to trigger the fallback diagnostic. Require a proven non-command data position separately from quoting, and add the quoted-command counterpart to the existing unquoted-command regression. Keep the original outer-printffixture completing. Exemption · Quoted word reader
PIC tradeoffs — A data-position requirement removes spurious coverage gaps for the original use case while keeping a coverage warning when the executable cannot be determined. The revised scan's cooperative cancellation preserves the existing deadline behavior.
Verification and gaps — Reviewed the complete diff, relevant parser paths, prior findings and replies, and the regression fixtures. All six recorded checks succeeded for this head. The PR remains conflicting, and the current executable-substitution fixture covers only the unquoted case. Static review only.
Reviewed head: bbf57b7212c161de715c529a6adb6d7b1f02f72f
…a parse limit A POSIX shell printf whose argument is a double-quoted command substitution piping printf '%s' "$1" into another command made static_patterns_tool_misuse record static_parse_limit (printf_format / unsupported_syntax), so the file counted as partially inspected. The inner substitution's output comes from the downstream command, not from printf, so it is runtime-dependent data in an argument position of the outer printf, as for the echo and assignment variants. Add _printf_invocation_has_following_operator, which walks the substitution with the shell word reader and reports whether a pipeline, command list, or redirection follows the printf words. The quoted-result branch of _static_printf_substitution returns unresolved without a diagnostic in that case, and _is_printf_substitution no longer treats it as a bounded printf substitution. A bare printf with a runtime argument stays fail-closed, a pipe quoted inside a printf argument still does not count as a pipeline, and literal printf handling (NVIDIA#694) is unchanged. Fixes NVIDIA#1107 Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
Backtick substitutions keep their runtime-command gap: a backtick printf piped into another command is the runtime command itself, so its failure to resolve statically is still a coverage gap. The $(...) form is runtime-dependent data, as before. Fixes the two test-unit failures on NVIDIA#1109 in test_table_column_lookahead_preserves_runtime_command_gaps (backtick printf 'x' | cat in a markdown table cell). Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
The NVIDIA#1107 exemption treated any $(printf ... | ...) substitution as runtime data, including one in command position: $(printf '%s%s' r 'm -rf /' | cat) supplies the command and its arguments through word splitting, and the early return dropped the reconstruction diagnostic while the outer-operand check had nothing left to inspect. Scope the exemption to proven quoted data arguments: _is_printf_substitution and _static_printf_substitution now take it only when the caller proves the substitution sits inside a double-quoted span, via the quoted_result flag and the quote-limits map for candidates the word reader starts on directly. A bare substitution keeps its coverage warning, and backtick substitutions keep their runtime-command gap. Thread the caller's runtime check through the operator scan as well: _printf_invocation_has_following_operator took lambda: None, so the added full pass ignored cancellation and the analyzer deadline while reading a long quoted argument, which has no _SHELL_COMMAND_WORD_CHARS cap. It now receives the caller's check from every call site. Addresses review feedback on NVIDIA#1109. Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
…xemption A shell executes a double-quoted substitution in command position as the command name, so quoting alone never proves a data argument: "$(printf '%s' "$tool_name" | cat)" on its own line lost its coverage warning, with no outer operand left to trigger the fallback diagnostic. The pipeline exemption now also needs a proven non-command data position, via the file's own _is_shell_command_position test: the word reader computes it from the enclosing word's start, the outer-operand fallback checks the candidate word, and the quote-limits fallback receives it from the full source since a fragment alone proves nothing. A native argument is proven data by its caller, and the argument tokenizer only scans argument words. The position travels as a separate quoted_data_position flag so a literal printf in command position still resolves to its value and keeps its destructive-head finding. The original outer-printf fixture still completes, and the quoted-command case is pinned by a new regression test alongside the unquoted one. Addresses rng1995's review on NVIDIA#1109. Signed-off-by: Varshith Puli <pulivarshit@gmail.com>
bbf57b7 to
5f60df5
Compare
|
Fixed in 5f60df5, and rebased onto current main (48e0f94), so the conflict is gone. The exemption now needs a proven non-command data position on top of quoting, using the file's own _is_shell_command_position test. The word reader computes it from the enclosing word's start, the outer-operand fallback checks the candidate word, and the quote-limits fallback receives it from the full source, since a fragment alone proves nothing. A native argument is proven data by its caller, and the argument tokenizer only scans argument words. The position travels as a separate quoted_data_position flag, so a literal printf in command position still resolves to its value and keeps its destructive-head finding. That split mattered: gating the shared quoted_result flag broke the TM1 HIGH finding for "$(printf "%s\n" rm)" -rf /, now restored. Added the quoted-command counterpart to the unquoted regression: your "$(printf '%s' "$tool_name" | cat)" example, the rm variant, the quoted command printf wrapper case, and the same word after ; and after | all keep the coverage warning. The original outer-printf fixture still completes. Test files green: 864 passed, 1 skipped across the 1107, 901, literals, and static-patterns files. One known limitation: a word after an assignment prefix (FOO=bar "$(...)") still reads as data, since the position test only sees separators and line starts. Say the word and I will tighten that too, it needs a small assignment-prefix walk. |
Fixes #1107.
Problem
A POSIX shell printf whose argument is a double-quoted command substitution piping printf '%s' "$1" into another command made static_patterns_tool_misuse record static_parse_limit (printf_format / unsupported_syntax), so the file counted as partially inspected. This is the common quote-a-value-for-sh/cron idiom, for example cronq() { printf "'%s'" "$(printf '%s' "$1" | sed ...)"; }.
Root cause
The quoted-result branch of _static_printf_substitution failed whenever a quoted substitution started with printf and _has_literal_printf_arguments was false, even when the substitution was a pipeline whose output another command produces. _is_printf_substitution then treated that failure as a coverage gap, so the outer printf argument parse was marked limited.
Fix
Tests
New tests/nodes/analyzers/test_static_parse_limit_issue_1107.py (11 tests): all 5 line variants from the issue (the two failing printf-inner cases plus the three passing controls), ledger-level COMPLETED assertions for the two failing cases and the verbatim cronq idiom, a #694 literal $(printf '\177ELF\001') test asserting the static value is still 0x7f ELF 0x01, and scope guards asserting a bare runtime-arg printf and a quoted pipe in an argument stay partial.
Verification
static_parse_limitfrom a backtick-quoted word in a shell comment, or an apostrophe after inline code, turns clean files into AE1 HIGH #901, bounded parser regressions, static_patterns); 3150 passed (dependency sources, SC2 boundaries, shell quote ownership, shell generated sources, prohibition suites, command wrapper, eval command strings, host language shell ownership, documentation reconstruction, shell artifacts); 2860 passed (shell source effects, security end-to-end, security reconstruction, prose shell context, E1 boundaries, markdown ownership composition, unit patterns). 7017 passed in total, 0 failed.