fix(xtask): make validate-urls a blocking check and harden probing

[mstykow]

Summary

• Turn CI URL validation into a blocking check (drop continue-on-error in check.yml) and remove the false-positive/transient failures that made it unfit to gate on.
• Validator hardening (xtask/src/bin/validate_urls.rs):
  • Probe with HTTP HEAD first, falling back to a single-byte ranged GET — large artifacts (release tarballs, package files) are no longer downloaded in full and no longer time out. GET is authoritative on any non-2xx/3xx HEAD (some hosts 404 HEAD but serve GET).
  • Retry transient failures (--retry 2 --retry-connrefused) so a network blip on one of ~460 URLs doesn't fail the whole check; genuine 4xx still fails fast.
  • Skip loopback/example hosts (127.0.0.1, localhost, ::1) and the … (U+2026) placeholder alongside the existing ... rule.
• Fixed the two genuinely-broken benchmark links in docs/BENCHMARKS.md:
  • FreeBSD bash row re-pointed to its committed testdata/ fixture (the upstream .pkg URL is non-durable per its SOURCE.md).
  • restic row removed — its SHA does not exist upstream, no scan input is committed, so the row is not reproducible. Regenerated the benchmark chart + headline.
• src/utils/font.rs: appended … to the deliberately-corrupted run-on example in a doc comment so it is skipped rather than probed as a live URL.
• Updated xtask/README.md and both generated-docs-maintenance skill mirrors to describe the check as blocking.

Scope and exclusions

• Included: validator behavior, the two broken benchmark links, CI gating, and doc/skill notes.
• Explicit exclusions: no change to which files/URLs are scanned or to other benchmark rows.

How to verify

• cargo run --quiet --manifest-path xtask/Cargo.toml --bin validate-urls -- --root . → Failed: 0 (447 passed, 15 skipped).
• Note the residual risk of a hard gate: probing ~460 external URLs each run means a benchmark artifact that rots later (or a host that starts rejecting CI IPs) will block unrelated PRs. HEAD + retry + the crates.io allowlist reduce flake but don't eliminate external rot; the fallback is to allowlist volatile artifact hosts to non-fatal.

Expected-output fixture changes

• docs/scan-duration-vs-files.svg and the docs/BENCHMARKS.md headline stats regenerated via generate-benchmark-chart after removing the non-reproducible restic row (254 runs, 19.7× median).

🤖 Generated with Claude Code

[greptile-apps]

Greptile Summary

This PR makes URL validation a blocking check and updates the related docs and benchmark data.

• CI now fails when URL validation fails.
• URL probing now uses a ranged GET first, with HEAD fallback for transport or range-specific failures.
• Loopback hosts and Unicode ellipsis placeholders are skipped.
• Benchmark docs and the generated SVG were refreshed after removing or repointing non-durable links.
• Maintainer docs and skill mirrors now describe URL validation as blocking.

Confidence Score: 5/5

This looks safe to merge.

• No blocking issues found in the changed code.

Important Files Changed

Filename	Overview
xtask/src/bin/validate_urls.rs	URL probing now uses ranged GET with HEAD fallback for transport and range-specific failures, plus retries and new skip rules.
.github/workflows/check.yml	URL validation now blocks the CI check.
docs/BENCHMARKS.md	Benchmark totals and entries were updated after removing one row and repointing one fixture link.
xtask/README.md	The maintainer workflow docs now describe the blocking URL check and probing behavior.

_{Reviews (3): Last reviewed commit: "fix(xtask): make validate-urls a blockin..." | Re-trigger Greptile}