Skip to content

feat(grammars): apex/html/xml/dart/clojure search grammars (→ 93.4%) #531

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
peaktwilight merged 1 commit into from
Jun 18, 2026
Merged

feat(grammars): apex/html/xml/dart/clojure search grammars (→ 93.4%) #531

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,11 @@ tree-sitter-ocaml = "0.25.0"
tree-sitter-scala = "0.26.0"
tree-sitter-elixir = "0.3.5"
tree-sitter-json = "0.24.8"
tree-sitter-html = "0.23"
tree-sitter-xml = "0.7"
tree-sitter-dart = "0.2"
tree-sitter-clojure-orchard = "0.2.5"
tree-sitter-sfapex = "3.0.0"

[features]
default = []
Expand Down
50 changes: 18 additions & 32 deletions docs/parity/registry-coverage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,27 +12,22 @@ Measures how well foxguard's existing Semgrep-compat YAML loader (`src/rules/sem
| Rule files scanned | 2070 |
| Files with YAML parse errors | 0 |
| Total rules | 2144 |
| Rules loaded OK | 1982 (92.4%) |
| Rules skipped | 162 (7.6%) |
| Rules loaded OK | 2002 (93.4%) |
| Rules skipped | 142 (6.6%) |

**Headline load rate: 92.4%** (1982 / 2144 rules).
**Headline load rate: 93.4%** (2002 / 2144 rules).

## Skip-reason histogram

Sorted by frequency. The reason names the operator/key that blocks the rule today.

| Skip reason | Rules | % of skipped | % of all rules |
|---|---:|---:|---:|
| `mode: taint (unsupported shape)` | 126 | 77.8% | 5.9% |
| `unsupported language: apex` | 9 | 5.6% | 0.4% |
| `generic mode (languages: [generic])` | 7 | 4.3% | 0.3% |
| `taint: pattern-propagators` | 5 | 3.1% | 0.2% |
| `unsupported language: clojure` | 5 | 3.1% | 0.2% |
| `unsupported language: html` | 4 | 2.5% | 0.2% |
| `mode: taint (unsupported language: apex)` | 3 | 1.9% | 0.1% |
| `mode: taint (unsupported language: swift)` | 1 | 0.6% | 0.0% |
| `unsupported language: dart` | 1 | 0.6% | 0.0% |
| `unsupported language: xml` | 1 | 0.6% | 0.0% |
| `mode: taint (unsupported shape)` | 126 | 88.7% | 5.9% |
| `generic mode (languages: [generic])` | 7 | 4.9% | 0.3% |
| `taint: pattern-propagators` | 5 | 3.5% | 0.2% |
| `mode: taint (unsupported language: apex)` | 3 | 2.1% | 0.1% |
| `mode: taint (unsupported language: swift)` | 1 | 0.7% | 0.0% |

## Priority order — operator/feature backlog

Expand All @@ -51,16 +46,11 @@ Rules foxguard cannot run because it has no tree-sitter grammar for the target l

| Rank | Language to add | Rules unlocked |
|---:|---|---:|
| 1 | `unsupported language: apex` | 9 |
| 2 | `generic mode (languages: [generic])` | 7 |
| 3 | `unsupported language: clojure` | 5 |
| 4 | `unsupported language: html` | 4 |
| 5 | `mode: taint (unsupported language: apex)` | 3 |
| 6 | `mode: taint (unsupported language: swift)` | 1 |
| 7 | `unsupported language: dart` | 1 |
| 8 | `unsupported language: xml` | 1 |
| 1 | `generic mode (languages: [generic])` | 7 |
| 2 | `mode: taint (unsupported language: apex)` | 3 |
| 3 | `mode: taint (unsupported language: swift)` | 1 |

Missing-grammar gaps account for **31 rules** (1.4% of all rules).
Missing-grammar gaps account for **11 rules** (0.5% of all rules).
Comment on lines +49 to +53

@coderabbitai coderabbitai Bot Jun 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Section labeling is now misleading for these entries.

This table is under “missing language grammars,” but the listed reasons are generic-mode and taint-mode support gaps, not strictly grammar gaps. Please retitle/reword this section so prioritization isn’t skewed.

Suggested doc tweak 
-## Priority order — missing language grammars
+## Priority order — remaining language-capability gaps

-Missing-grammar gaps account for **11 rules** (0.5% of all rules).
+These language-capability gaps account for **11 rules** (0.5% of all rules).
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/parity/registry-coverage.md` around lines 49 - 53, The section heading
in the registry-coverage.md file is labeled as "missing language grammars" but
the table entries (generic mode, mode: taint with unsupported languages)
represent mode and language support gaps rather than strictly grammar gaps.
Retitle or reword the section heading to accurately reflect that it covers mode
and language support gaps, not just grammar gaps, so the content aligns with the
heading and prevents misrepresentation during prioritization.


## Per-language breakdown

Expand All @@ -86,34 +76,30 @@ Language is the rule's first declared language (js/ts/jsx/tsx collapsed to `java
| c | 16 | 16 | 0 | 100.0% |
| kotlin | 15 | 15 | 0 | 100.0% |
| bash | 13 | 13 | 0 | 100.0% |
| apex | 12 | 0 | 12 | 0.0% |
| apex | 12 | 9 | 3 | 75.0% |
| rust | 10 | 10 | 0 | 100.0% |
| elixir | 7 | 7 | 0 | 100.0% |
| json | 7 | 7 | 0 | 100.0% |
| swift | 6 | 5 | 1 | 83.3% |
| clojure | 5 | 0 | 5 | 0.0% |
| clojure | 5 | 5 | 0 | 100.0% |
| terraform | 5 | 5 | 0 | 100.0% |
| html | 4 | 0 | 4 | 0.0% |
| dart | 1 | 0 | 1 | 0.0% |
| xml | 1 | 0 | 1 | 0.0% |
| html | 4 | 4 | 0 | 100.0% |
| dart | 1 | 1 | 0 | 100.0% |
| xml | 1 | 1 | 0 | 100.0% |

## Top skip reasons per language

- **apex**: `unsupported language: apex` (9), `mode: taint (unsupported language: apex)` (3)
- **clojure**: `unsupported language: clojure` (5)
- **apex**: `mode: taint (unsupported language: apex)` (3)
- **csharp**: `mode: taint (unsupported shape)` (9), `taint: pattern-propagators` (1)
- **dart**: `unsupported language: dart` (1)
- **generic**: `generic mode (languages: [generic])` (7)
- **go**: `mode: taint (unsupported shape)` (13)
- **html**: `unsupported language: html` (4)
- **java**: `mode: taint (unsupported shape)` (17), `taint: pattern-propagators` (3)
- **javascript**: `mode: taint (unsupported shape)` (35)
- **php**: `mode: taint (unsupported shape)` (14)
- **python**: `mode: taint (unsupported shape)` (22)
- **ruby**: `mode: taint (unsupported shape)` (15), `taint: pattern-propagators` (1)
- **solidity**: `mode: taint (unsupported shape)` (1)
- **swift**: `mode: taint (unsupported language: swift)` (1)
- **xml**: `unsupported language: xml` (1)

---

Expand Down
15 changes: 15 additions & 0 deletions src/bin/gen_rules_ts.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,11 @@ fn language_slug(language: Language) -> &'static str {
Language::Scala => "scala",
Language::Elixir => "elixir",
Language::Json => "json",
Language::Apex => "apex",
Language::Clojure => "clojure",
Language::Html => "html",
Language::Xml => "xml",
Language::Dart => "dart",
// Regex-mode rules fan out per language; this binary generates rules
// for concrete languages only.
Language::Regex => "regex",
Expand Down Expand Up @@ -90,6 +95,11 @@ fn language_display_name(language: Language) -> &'static str {
Language::Scala => "Scala",
Language::Elixir => "Elixir",
Language::Json => "JSON",
Language::Apex => "Apex",
Language::Clojure => "Clojure",
Language::Html => "HTML",
Language::Xml => "XML",
Language::Dart => "Dart",
Language::Regex => "Regex",
}
}
Expand Down Expand Up @@ -120,6 +130,11 @@ fn language_array_name(language: Language) -> &'static str {
Language::Scala => "scalaRules",
Language::Elixir => "elixirRules",
Language::Json => "jsonRules",
Language::Apex => "apexRules",
Language::Clojure => "clojureRules",
Language::Html => "htmlRules",
Language::Xml => "xmlRules",
Language::Dart => "dartRules",
Language::Regex => "regexRules",
}
}
Expand Down
39 changes: 35 additions & 4 deletions src/bin/registry_coverage.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,20 @@ fn language_supported(lang: &str) -> bool {
| "exs"
// JSON grammar (tree-sitter-json).
| "json"
// Apex grammar (tree-sitter-sfapex).
| "apex"
// Clojure grammar (tree-sitter-clojure-orchard).
| "clojure"
| "clj"
| "cljs"
| "cljc"
// HTML grammar (tree-sitter-html).
| "html"
| "htm"
// XML grammar (tree-sitter-xml).
| "xml"
// Dart grammar (tree-sitter-dart).
| "dart"
)
}

Expand Down Expand Up @@ -915,23 +929,40 @@ rules:

#[test]
fn unsupported_language_is_skipped() {
// Apex has no supported grammar — it must be skipped.
// COBOL has no supported grammar — it must be skipped.
let r = rule(
r#"
rules:
- id: apex-rule
- id: cobol-rule
pattern: foo
message: m
severity: INFO
languages: [apex]
languages: [cobol]
"#,
);
match classify_rule(&r) {
Outcome::Skipped(reason) => assert_eq!(reason, "unsupported language: apex"),
Outcome::Skipped(reason) => assert_eq!(reason, "unsupported language: cobol"),
other => panic!("expected skip, got {other:?}"),
}
}

#[test]
fn apex_language_loads() {
// Apex now has a real grammar (tree-sitter-sfapex), so search-mode
// Apex rules must load.
let r = rule(
r#"
rules:
- id: apex-rule
pattern-regex: 'System\.debug'
message: m
severity: INFO
languages: [apex]
"#,
);
assert!(matches!(classify_rule(&r), Outcome::Loaded));
}

#[test]
fn yaml_language_loads() {
let r = rule(
Expand Down
51 changes: 51 additions & 0 deletions src/engine/parser.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,11 @@
Language::Scala => tree_sitter_scala::LANGUAGE.into(),
Language::Elixir => tree_sitter_elixir::LANGUAGE.into(),
Language::Json => tree_sitter_json::LANGUAGE.into(),
Language::Apex => tree_sitter_sfapex::apex::LANGUAGE.into(),
Language::Clojure => tree_sitter_clojure_orchard::LANGUAGE.into(),
Language::Html => tree_sitter_html::LANGUAGE.into(),
Language::Xml => tree_sitter_xml::LANGUAGE_XML.into(),
Language::Dart => tree_sitter_dart::LANGUAGE.into(),
// Regex-mode rules never use a tree-sitter parser — they match raw text
// only. Return `None` immediately so the scanner skips the tree build.
Language::Regex => return None,
Expand Down Expand Up @@ -147,6 +152,52 @@
assert!(!tree.root_node().has_error());
}

#[test]
fn parses_apex_without_errors() {
let source =
"public class Hello {\n public void greet() {\n String x = 'hi';\n }\n}\n";
let tree = parse_path(source, Language::Apex, Path::new("Hello.cls"))

@foxguard-app foxguard-app Bot Jun 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

foxguard · MEDIUM · rs/no-unwrap-in-lib (CWE-248)

.expect() can panic at runtime — use proper error handling with ? or match

.expect("Apex parser should produce a tree");

Check warning on line 160 in src/engine/parser.rs

View check run for this annotation

Foxguard App / foxguard

rs/no-unwrap-in-lib (medium) 

.expect() can panic at runtime — use proper error handling with ? or match
Raw output 
        let tree = parse_path(source, Language::Apex, Path::new("Hello.cls"))

assert!(!tree.root_node().has_error());
}

#[test]
fn parses_clojure_without_errors() {
let source = "(defn greet [name]\n (println \"hello\" name))\n";
let tree = parse_path(source, Language::Clojure, Path::new("core.clj"))

@foxguard-app foxguard-app Bot Jun 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

foxguard · MEDIUM · rs/no-unwrap-in-lib (CWE-248)

.expect() can panic at runtime — use proper error handling with ? or match

.expect("Clojure parser should produce a tree");

Check warning on line 169 in src/engine/parser.rs

View check run for this annotation

Foxguard App / foxguard

rs/no-unwrap-in-lib (medium) 

.expect() can panic at runtime — use proper error handling with ? or match
Raw output 
        let tree = parse_path(source, Language::Clojure, Path::new("core.clj"))

assert!(!tree.root_node().has_error());
}

#[test]
fn parses_html_without_errors() {
let source = "<html>\n <body>\n <a href=\"/x\">link</a>\n </body>\n</html>\n";
let tree = parse_path(source, Language::Html, Path::new("index.html"))

@foxguard-app foxguard-app Bot Jun 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

foxguard · MEDIUM · rs/no-unwrap-in-lib (CWE-248)

.expect() can panic at runtime — use proper error handling with ? or match

.expect("HTML parser should produce a tree");

Check warning on line 178 in src/engine/parser.rs

View check run for this annotation

Foxguard App / foxguard

rs/no-unwrap-in-lib (medium) 

.expect() can panic at runtime — use proper error handling with ? or match
Raw output 
        let tree = parse_path(source, Language::Html, Path::new("index.html"))

assert!(!tree.root_node().has_error());
}

#[test]
fn parses_xml_without_errors() {
let source = "<?xml version=\"1.0\"?>\n<root>\n <child id=\"1\">text</child>\n</root>\n";
let tree = parse_path(source, Language::Xml, Path::new("data.xml"))

@foxguard-app foxguard-app Bot Jun 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

foxguard · MEDIUM · rs/no-unwrap-in-lib (CWE-248)

.expect() can panic at runtime — use proper error handling with ? or match

.expect("XML parser should produce a tree");

Check warning on line 187 in src/engine/parser.rs

View check run for this annotation

Foxguard App / foxguard

rs/no-unwrap-in-lib (medium) 

.expect() can panic at runtime — use proper error handling with ? or match
Raw output 
        let tree = parse_path(source, Language::Xml, Path::new("data.xml"))

assert!(!tree.root_node().has_error());
}

#[test]
fn parses_dart_without_errors() {
let source = "void main() {\n print('hello');\n}\n";
let tree = parse_path(source, Language::Dart, Path::new("main.dart"))

@foxguard-app foxguard-app Bot Jun 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

foxguard · MEDIUM · rs/no-unwrap-in-lib (CWE-248)

.expect() can panic at runtime — use proper error handling with ? or match

.expect("Dart parser should produce a tree");

Check warning on line 196 in src/engine/parser.rs

View check run for this annotation

Foxguard App / foxguard

rs/no-unwrap-in-lib (medium) 

.expect() can panic at runtime — use proper error handling with ? or match
Raw output 
        let tree = parse_path(source, Language::Dart, Path::new("main.dart"))

assert!(!tree.root_node().has_error());
}

#[test]
fn parses_ocaml_without_errors() {
let source = "let () = print_endline \"Hello, world\"\n";
Expand Down
Loading