Repository navigation
Conversation
yashrajp22
left a comment
There was a problem hiding this comment.
The wrapped inline-tag continuation is handled, but a new list item can inherit the preceding paragraph's inline-code state and mask literal HTML backticks. Please preserve that block boundary.
| html_open = re.match(r"<(?:[A-Za-z][A-Za-z0-9-]*(?=[\s/>]|$)|[!?/])", leading) | ||
| if ( | ||
| html_open | ||
| and paragraph_open |
There was a problem hiding this comment.
Please keep HTML-block detection active when this line starts a new list item. For example:
Intro `open
- <custom>
`rm -rf /`Here paragraph_open and runs still belong to Intro, so the guard clears html_open before the new list marker closes that paragraph. <custom> starts a real type-7 HTML block inside the new item, whose body backticks must remain literal. Instead, they are masked and registered as inline-code ranges, changing the shell parser's input. Please limit this guard to continuation of the same paragraph, excluding a new list item.
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Decision: Changes Requested
Value and readiness
The wrapped-code-span fix addresses a real completeness false positive in ordinary Markdown, and the added tests cover the issue fixture through projection, ledger and CLI output. One block-boundary regression still needs correction before this is ready.
Material findings
Preserve new-list HTML block ownership. In src/skillspector/nodes/analyzers/static_patterns_tool_misuse.py:9068, the new guard uses paragraph_open and runs from the preceding paragraph after the current list marker has already been stripped. For the existing reviewer example:
Intro `open
- <custom>
`rm -rf /`has_list_marker is true, but the guard clears html_open. The later new-item branch clears the old runs without entering the <custom> HTML block, so the following raw HTML-body backticks are processed and masked as inline-code delimiters. This changes the security parser's input and grants ownership that the rendered block does not establish. Restrict this exception to continuation of the same paragraph, excluding a new list item, and add a regression asserting the HTML-body ticks remain literal. This confirms the still-open current-head thread; there is no author fix or reply in the frozen inventory.
PIC tradeoffs
No new product decision is needed. The documented second issue variant can remain outside this patch; preserving actual Markdown block boundaries is required for the chosen conservative projection.
Verification and gaps
Reviewed the complete two-file diff, exact-head surrounding paragraph/list/HTML state transitions, all seven added tests, linked issue #1023, the complete available review/comment/thread records and all six successful hosted check results. Pre-fetch, post-fetch and post-review heads match the frozen inventory. The review thread remains unresolved and the other review requests changes. No contributor code, tests, imports, builds, hooks, scanner, regex probes or model calls were executed.
Reviewed head: fbc54d17bfb35bc19b06ba065ffe8b5e720af043.
A list marker ends the preceding paragraph, so its content starts a fresh block where an inline tag can open a real HTML block. The wrapped-span guard now applies only to same-paragraph continuation, as requested in review on NVIDIA#1085. Signed-off-by: Koloyaaa <yakamozzz@sjtu.edu.cn>
|
Updated in 6a21283: the guard now also requires the line to continue the same paragraph ( Re-verified: focused file 8 passed; planned Markdown/parser subset 1,356 passed / 1 deselected; security/parse-limit group 1,036 passed; ruff check and format clean. The #1023 fixture still scans complete at 100% coverage, score 0. |
yashrajp22
left a comment
There was a problem hiding this comment.
The new-list HTML boundary is addressed. One remaining case lets a literal HTML-attribute backtick create a false inline-code range and combine separate lines into an unsafe-option match.
| if ( | ||
| html_open | ||
| and paragraph_open | ||
| and runs |
There was a problem hiding this comment.
Please verify that runs contains a real open code span before using it to clear html_open. For example:
Intro <a title="`">curl
<custom> --insecure
`done`The first backtick belongs to the quoted HTML attribute, but it remains in runs. This guard then clears html_open on line 2, and paragraph pairing matches that attribute tick with the opener before done, leaving the real closing tick unmatched. The resulting inline_regions also makes _option_command_matches join curl with --insecure across the intervening markup. Please exclude HTML-owned literal ticks from the pending-span proof, or keep the conservative behavior when ownership is uncertain.
|
Pushed a follow-up revision. Backticks inside complete inline HTML tags no longer count as evidence of a pending code span, and a tag whose attributes wrap within the same paragraph still owns its ticks. Your example and its wrapped variant now pass through unchanged, with no inline region and no An unclosed tag still leaves its backticks as code delimiters; I checked that against markdown-it-py. 11 focused tests pass, plus the 1,348-file Markdown/parser subset, the 1,036-test security group, and ruff. I left the review thread unresolved so you can verify. |
yashrajp22
left a comment
There was a problem hiding this comment.
The list boundary and original attribute example are handled. The earlier attribute issue remains for attributes on later lines; the inline comments below cover two additional problems with code-span precedence and repeated paragraph scans.
| tag whose attributes wrap within a paragraph still owns its ticks. | ||
| """ | ||
| limit = len(text) if end is None else end | ||
| tags = _markdown_html_tag_ranges(text, start, limit if tag_limit is None else tag_limit) |
There was a problem hiding this comment.
Please preserve code-span ownership before excluding HTML-looking ranges. For example:
Run `curl
--insecure <name`>.The code span owns both backticks, but this pass treats <name through > as a tag and discards the closing tick. No inline region is produced, so _option_command_matches treats the newline as a command boundary and misses curl --insecure. The same ordering problem can also extend a span past its real closer into later prose. Please recognize inline constructs in source order and require valid, unescaped HTML before granting it ownership.
| for match in backtick_runs.finditer(line): | ||
| if paragraph_tag_limit is None: | ||
| paragraph_tag_limit = _markdown_paragraph_limit(content, offset) | ||
| for start, end in _markdown_backtick_runs_outside_html( |
There was a problem hiding this comment.
Please avoid rescanning the remaining paragraph for every physical line. paragraph_tag_limit stays at the paragraph end, and _markdown_backtick_runs_outside_html builds all tag ranges up to that limit before looking for any backticks. A paragraph of 10,000 a <b> lines therefore visits 50,005,000 tags, even with no backticks, turning this previously linear projection into quadratic work. This consumes the scanner's shared runtime budget and can leave ordinary Markdown incompletely inspected. Please compute ownership once per paragraph or carry a forward parser state, with runtime checks inside long scans.
|
Pushed a revision that covers both points. Ownership now happens in source order. A code span that opens before a tag keeps its body, so the closing tick in this example is honored again: It produces the region Tags need to match CommonMark's inline HTML grammar and be unescaped before they can own a tick. An unclosed or malformed The per-line paragraph scan is gone. Tag ranges are computed once per paragraph, and a 40,000-line paragraph of 14 focused tests pass, along with the 1,348-file Markdown/parser subset, the 1,036-test security group, and ruff. I left the review thread open so you can check it. |
|
CI found five failing cases in the new dense-reference test, all from the same paragraph scan, so I pushed a follow-up. That test separates four-line blocks with headings and no blank lines. The scan used to stop only at blank lines, so it walked to the end of the file for each paragraph. At 256 KB that became quadratic and the ledger reported The 256 KB case now scans 220,682 of 262,144 characters in about 0.28s, and the dense test passes 15/15 here. The focused tests, the Markdown/parser subset, and the security group still pass, and the earlier code-span cases are unchanged. I left the review thread open. |
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Decision — Request changes.
Value and readiness — The current revision preserves the wrapped placeholder span and addresses the prior list-boundary, HTML-attribute, and code-span precedence examples. Computing the HTML ranges once removes the earlier per-line parse, but the subsequent filtering still repeats work.
Material findings
static_patterns_tool_misuse.py:8965–8968restartstag_indexat zero for every physical line in a paragraph. In a long paragraph with one inline tag and one complete inline code span per line, each line with backticks walks all tag ranges from earlier lines. The visits grow quadratically, and the inner loop contains no runtime check. The new call-count test uses lines without backticks, so that scan never runs there. Carry a forward range cursor through the paragraph or bisect sorted tag endpoints at the line's start. Add a dense tag-and-backtick control that measures this filtering work and checks that the projection and ledger remain complete. Tag filtering · Per-line caller
PIC tradeoffs — Retaining source-order ownership is valuable for accurate findings. Indexing the already computed ranges keeps the precision gain while guarding the analyzer's coverage budget on reference-heavy Markdown.
Verification and gaps — Reviewed the complete diff, paragraph and list state, ownership/filtering paths, fixtures, and prior threads and replies. All six recorded checks succeeded for this head, and GitHub reports a merge conflict. Static review only.
Reviewed head: c3554b53cd361bf8b3767d45bf35dc9faba0cd31
Current main already pairs the wrapped span correctly, so this adds the issue fixture and its joined control as a regression test. The test fails on bb991c5 and passes on current main. Signed-off-by: Koloyaaa <yakamozzz@sjtu.edu.cn>
c3554b5 to
3ca711b
Compare
|
I reduced this PR to a regression test. Current main already handles the reported wrapped span: the issue fixture scans at 100% coverage, complete, score 0, SAFE. The earlier 394-line patch was duplicating that fix, so I removed it. The branch now adds one test file for #1023 and its joined control. The test fails 4/5 on bb991c5 and passes 5/5 on current main, so it still detects the regression. The Markdown and parse-limit subset passes (1,289 tests), and ruff is clean. The per-line tag-filter finding no longer applies: that filter is not part of this diff. I left the earlier review threads open because they refer to code that has been removed. |
Fixes #1023.
Impact: silent-wrong-result
Who reaches this: any
skillspector scanof a Markdown file with an inline code span that wraps across a soft line break, where the continuation line starts with an inline-HTML-looking tag such as<path>.Root cause:
_markdown_shell_texttreats any line that begins with<plus a tag-name prefix as an HTML block start. In the issue's fixture, line 2 begins with<path>, so the projection treats the rest of the paragraph as an HTML body, leaves every backtick visible, and the shell tokenizer reads the closing delimiter at offset 177 as an unmatched substitution opener. The scan reportsstatic_parse_limit:references/guide.mdis 50% covered and the linkingSKILL.mdgets AE1 HIGH (score 25, CAUTION).Fix: a
<...continuation line may interrupt an open paragraph only when it is a real CommonMark HTML block start — a raw-text tag, comment, processing instruction, declaration, CDATA section, or one of the type-6 block tags. Otherwise the line stays in the paragraph and backtick runs pair across the soft break. The type-6 tag list mirrors the repository'ssecurity_reconstruction._HTML_BLOCK_TAG_RE.Tests:
tests/nodes/analyzers/test_static_parse_limit_issue_1023.pycovers projection masking, bounded-parse completion, ledgerCOMPLETED, the joined control, preserved<pre>and<div>blocks, and an end-to-end CLI scan of the issue fixture. Before the fix the fixture scans at 50% coverage with AE1 HIGH; after the fix it scans at 100%, complete, score 0, SAFE, including with--fail-on-incomplete.Verification: focused file 7 passed; planned Markdown/parser subset 1,355 passed, 1 deselected (POSIX-only file-output test); security-reconstruction, false-positive, and parse-limit group 1,036 passed;
ruff checkandruff format --checkclean.make testis not available on the Windows host used here because the integration target may call an LLM provider; CI will run the full suite.Known limitation: the guard requires a pending inline-code run, so the no-pending-run
<path>shape remains an HTML block. The second double-backtick case from the issue is not part of this change.Signed-off-by: Koloyaaa yakamozzz@sjtu.edu.cn