Skip to content

docs(benchmarks): verify crystal, elm, and ponyc targets#1311

Merged
mstykow merged 3 commits into
mainfrom
verify/crystal-elm-ponyc
Jul 19, 2026
Merged

docs(benchmarks): verify crystal, elm, and ponyc targets#1311
mstykow merged 3 commits into
mainfrom
verify/crystal-elm-ponyc

Conversation

@mstykow

@mstykow mstykow commented Jul 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Verify three new compare-outputs repository targets not already in docs/BENCHMARKS.md: crystal-lang/crystal, elm/compiler, and ponylang/ponyc.
  • Record end-state timings and Provenant-vs-ScanCode advantages; regenerate the benchmark chart (255 runs).
  • Drive-by: reject changelog copyright-year bump prose and related holder/author junk exposed while triaging Crystal (start-anchored so real notices with embedded “update copyright year” instructions are kept).
  • compare-outputs triage surface: flat ScanCode/Provenant-favored review queues, review_queue_summary, stdout triage pointer, and run-manifest.json duration fields — so agents inspect artifacts instead of writing ad-hoc scripts.

Scope and exclusions

  • Included: benchmark entries for the three targets; changelog/contribution-guide junk filtering in copyright refine + final refine path; compare-outputs review queues + duration parsing; focused unit coverage; verify-benchmark-target skill pointer updates.
  • Explicit exclusions: no Crystal shard.yml package parser (tracked in Support parsing: Crystal shard.yml #1312; ScanCode also emits 0 packages on this snapshot); no broader literary-quote author cleanup on Elm's Quotes.hs.

How to verify

  • Spot-check the three new docs/BENCHMARKS.md entries for present-tense end-state wording and alphabetical placement under Julia / Nix / Haskell / other ecosystems.
  • Optional: rerun one cached compare, e.g. cargo run --manifest-path xtask/Cargo.toml --bin compare-outputs -- --repo-url https://github.com/crystal-lang/crystal.git --repo-ref 82b5094b716b1e9a3f4fe6c601fd48272fc35d01 --profile common, and confirm (1) changelog copyright-year bullets stay absent and (2) comparison/samples/scancode_favored_review_queue.json plus run-manifest.json *.duration_secs are present.

Follow-up work

Expected-output fixture changes

  • Files changed: none (junk filtering and compare artifacts covered by unit tests; no golden/fixture updates).

Compare runs (local):

  • Crystal: .provenant/compare-runs/20260719T171119Z-crystal-9555
  • Elm: .provenant/compare-runs/20260719T171137Z-compiler-10473
  • Ponyc: .provenant/compare-runs/20260719T171145Z-ponyc-11639

Generated with Cursor.

Made with Cursor

Record compare-outputs results for three new repository targets and
tighten changelog copyright/holder/author junk filtering exposed while
triaging crystal.

Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@greptile-apps

greptile-apps Bot commented Jul 19, 2026

Copy link
Copy Markdown

Greptile Summary

This PR updates benchmark documentation and compare-output triage artifacts. The main changes are:

  • Adds Crystal, Elm, and Ponyc benchmark entries and refreshes the chart.
  • Adds flat ScanCode-favored and Provenant-favored review queues to compare artifacts.
  • Records optional scanner duration fields in compare run manifests.
  • Narrows changelog copyright-year junk filtering and adds focused tests.

Confidence Score: 5/5

This looks safe to merge.

  • No blocking issues found in the changed code.

Important Files Changed

Filename Overview
src/copyright/refiner/copyrights_junk_patterns.rs Adds narrower changelog copyright-year junk filters with coverage for real notices that contain update-year prose.
src/copyright/refiner/holders_junk_patterns.rs Adds targeted filters for changelog and PR holder fragments.
src/copyright/refiner/authors_junk_patterns.rs Adds filters for contribution-guide and month prose mistaken as authors.
src/compare.rs Adds diagnostic review queue artifact generation and summary pointers.
xtask/src/bin/compare_outputs.rs Adds optional duration parsing for compare run manifests.
xtask/src/manifests.rs Extends scanner manifest records with optional duration fields.
docs/BENCHMARKS.md Adds new benchmark rows and updates aggregate counts.

Reviews (4): Last reviewed commit: "fix(copyright): keep junk filter out of ..." | Re-trigger Greptile

Comment thread src/copyright/refiner/copyrights_junk_patterns.rs Outdated
Surface ScanCode/Provenant-favored file-metric triage queues and
run-manifest durations from compare-outputs, start-anchor changelog
copyright-year junk filtering, and keep the verify-benchmark-target
skill pointed at those artifacts.

Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@mstykow

mstykow commented Jul 19, 2026

Copy link
Copy Markdown
Collaborator Author

@greptileai review

@mstykow
mstykow enabled auto-merge (squash) July 19, 2026 17:23
Applying is_junk_copyright in refine_final_copyrights dropped many
golden-expected notices that already match historical junk patterns.
Changelog year-bump filtering still works via the existing detector
junk path and the start-anchored patterns.

Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@mstykow
mstykow merged commit e4ee203 into main Jul 19, 2026
13 checks passed
@mstykow
mstykow deleted the verify/crystal-elm-ponyc branch July 19, 2026 17:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant