Skip to content

feat(php): consolidated PHP tooling (php, artisan, phpunit, phpstan, pest, paratest, ecs, pint)#1649

Open
iliaal wants to merge 7 commits into
rtk-ai:developfrom
iliaal:feat/php-tooling
Open

feat(php): consolidated PHP tooling (php, artisan, phpunit, phpstan, pest, paratest, ecs, pint)#1649
iliaal wants to merge 7 commits into
rtk-ai:developfrom
iliaal:feat/php-tooling

Conversation

@iliaal

@iliaal iliaal commented Apr 30, 2026

Copy link
Copy Markdown

Summary

Consolidates the open and closed PHP-tooling work into a single PR against develop, omitting .phpt (which has its own PR #1503).

This PR adds eight rtk subcommands for PHP test runners and code tooling: php, artisan, phpunit, phpstan, pest, paratest, ecs, pint. Output compression in the 60% to 95% range per tool, structured parsers where possible (PHPUnit state machine, PHPStan typed JSON, Pint per-file rule counts).

Credits to original authors

  • @aaronflorey (rtk-ai/rtk#1246, self-closed 2026-04-26): php, artisan, ecs, pest, paratest, test_output, utils, composer_bin_dirs, registry normalization for Composer custom-bin-dir layouts.
  • @Beninho (rtk-ai/rtk#874, open): phpunit state-machine parser. I ported it into the consolidated module, switched it to runner::run_filtered, and stripped emoji from output.
  • @LucianoVandi (rtk-ai/rtk#1110, open): phpstan typed serde::Deserialize parser for --error-format=json. Groups errors by file, sorts by count descending; passes utility commands (--version, list, clear-result-cache) through unchanged. I stripped emoji from the success output.
  • New work: pint (Laravel Pint code-style fixer) using --format=json for structured per-file rule counts.

If the maintainers prefer this consolidation, close the three separate PRs in favor of this one. If they prefer the originals, close this PR and let the per-tool PRs continue independently. Either is fine with me.

Composer custom-bin-dir support

composer_bin_dirs() reads COMPOSER_BIN_DIR and composer.json's config.bin-dir, so tools/bin/phpunit classifies identically to vendor/bin/phpunit. registry.rs normalizes tool paths before matching, so a single rule covers the standard Composer layouts.

@pszymkowiak pszymkowiak added effort-large Plusieurs jours, nouveau module enhancement New feature or request labels Apr 30, 2026
@pszymkowiak

Copy link
Copy Markdown
Collaborator

[w] wshm · Automated triage by AI

📊 Automated PR Analysis

Type feature
🟡 Risk medium

Summary

Adds eight new rtk subcommands for PHP tooling (php, artisan, phpunit, phpstan, pest, paratest, ecs, pint) with structured output parsers and compression. Consolidates work from three prior PRs by different authors into a single module, including shared test output filtering, Composer custom-bin-dir support, and registry normalization.

Review Checklist

  • Tests present
  • Breaking change
  • Docs updated

Linked issues: #1503, #1246, #874, #1110


Analyzed automatically by wshm · This is an automated analysis, not a human review.

@pierresh

pierresh commented May 1, 2026

Copy link
Copy Markdown

Hi,
Thanks a lot for this consolidated PHP tooling. I tested it, and it works well. Just 2 suggestions: to add Behat and Rector as they are both popular tools in the PHP world.

@iliaal

iliaal commented May 2, 2026

Copy link
Copy Markdown
Author

Hi, Thanks a lot for this consolidated PHP tooling. I tested it, and it works well. Just 2 suggestions: to add Behat and Rector as they are both popular tools in the PHP world.

Can be added maybe as a follow-up, PR is already quite big, but 100% a good idea.

@iliaal

iliaal commented May 3, 2026

Copy link
Copy Markdown
Author

Added f9e139a on top of this branch: swaps the make filter from max_lines = 50 (head-only) to head_lines = 10 + tail_lines = 40, so make test runs (e.g. PHPT, which puts ~17000 progress lines before the pass/fail summary) keep both the prologue and the trailing summary instead of dropping the latter.

Companion fix at the recovery layer: #1696 — same head-bias bug in write_tee_file, where the tee log capped at 1MB by keeping raw[..1MB]. With both landed, tail-heavy output survives at the filter and at the fallback log.

@iliaal iliaal force-pushed the feat/php-tooling branch from f9e139a to a944865 Compare May 17, 2026 15:58
@iliaal iliaal force-pushed the feat/php-tooling branch from a944865 to 6ba52ae Compare May 30, 2026 01:38
@webagil-kevin

Copy link
Copy Markdown

Tested this branch on a production Symfony monorepo (27 packages, PHP 8.4/8.5, Pest 4.6.3 + PHPStan level 9), built from source and wired through the Claude Code PreToolUse hook.

Works well across the toolchain:

  • rtk pest / rtk phpunit — failures + summary only, drops the passing noise while preserving diffs and file:line
  • rtk phpstan — compact file:line errors instead of the full table + preamble
  • rtk pint, rtk php artisan test — resolve and filter as expected
  • vendor/bin/* rewrites and php artisan* routing all behave correctly
  • cargo test is green on the branch (2035 passed)

Coverage matches our PHP stack exactly and the filtered output stays faithful to each tool's native format. Would love to see this land — happy to help test further. Thanks for the thorough work here.

@EliW

EliW commented Jun 9, 2026

Copy link
Copy Markdown

Really looking forward to this landing as part of rtk. It's needed to get real benefits from rtk for those of us on PHP monoliths at the moment :)

Pretty please?

iliaal added 3 commits June 9, 2026 11:24
…pest, paratest, ecs, pint)

Consolidates the PHP-tooling work from three upstream PRs plus a new
Pint module, leaving phpt to its own PR (rtk-ai#1503).

- rtk php / rtk artisan: syntax check (-l) and Laravel artisan wrapper.
- rtk phpunit: structured-state parser, aggregate counts, bounded
  failure list. Uses runner::run_filtered.
- rtk phpstan: typed serde::Deserialize parser for --error-format=json,
  groups errors by file, sorts by count desc. Utility commands
  (--version, list, clear-result-cache) pass through unchanged.
- rtk pest / rtk paratest: shared test_output helper.
- rtk ecs / rtk pint: code-style fixers; pint uses --format=json for
  structured per-file rule counts.

Composer custom-bin-dir detection: composer_bin_dirs() reads
COMPOSER_BIN_DIR and composer.json config.bin-dir, so tools/bin/phpunit
classifies identically to vendor/bin/phpunit. registry.rs normalizes
tool paths before matching.

Sources:
- rtk-ai#1246 (aaronflorey, self-closed): php, artisan, ecs, pest,
  paratest, test_output, utils, composer_bin_dirs, registry normalization.
- rtk-ai#874 (Beninho, open): phpunit state-machine parser.
- rtk-ai#1110 (LucianoVandi, open): phpstan typed parser.
- New: pint_cmd.rs.

Tests: discover::registry 253 pass; cmds::php 36 pass; cargo build
--release 0 errors; cargo fmt --check clean.
The make filter used max_lines=50, implemented as head-only truncation in
toml_filter. For commands like 'make test' running PHPT (~17000 progress
lines followed by a pass/fail summary), the cap discarded exactly the
useful part — the tail.

Switch to head_lines=10 + tail_lines=40 so both the build prologue and
the final summary survive, with a '... (N lines omitted)' marker in the
middle. Token savings on long outputs remain >60%; existing inline tests
still pass and a new tail-preservation test guards the behaviour.
Upstream develop changed `rewrite_command` to take 3 args
(cmd, excluded, transparent_prefixes). The PHP tooling tests that
don't care about transparent prefixes now use the existing
`rewrite_command_no_prefixes` helper instead.
@iliaal iliaal force-pushed the feat/php-tooling branch from 6ba52ae to 16c12bc Compare June 9, 2026 15:26
@KuSh

KuSh commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

Hi @iliaal,

Since you're aggregating contributions from others in this PR, please ensure you include the "Co-authored-by:" trailer in the commit message to properly credit them.

Example:

Co-authored-by: Name <email@example.com>

This helps maintain proper attribution for all contributors. Thanks!

EliW and others added 2 commits June 18, 2026 11:36
The PHP runners (phpunit, pest/paratest, ecs, phpstan, pint) only
applied their compact filters when rtk itself launched the process.
Output produced elsewhere — most commonly a tool run inside a Docker
container and piped back to the host — bypassed them entirely, since
the visible command is `docker ...`, not the PHP tool.

Wire the existing filter functions into `rtk pipe`:

- resolve_filter: phpunit, pest|paratest|php-test, ecs, phpstan, pint
- phpstan/pint pipe wrappers sniff JSON-vs-text by content, since the
  runners force --format=json but piped output may be either
- auto_detect_filter: route the "by Sebastian Bergmann" banner to the
  phpunit filter (no -f needed)
- bump the four backing fns to pub(crate)

Also fix filter_phpstan_text matching the summary line case-sensitively
("found"), which missed phpstan's actual "[ERROR] Found N errors".

Tests: phpunit banner auto-detect, phpstan case-insensitive summary.
feat(pipe): expose PHP tool filters as stdin pipe filters
@CLAassistant

CLAassistant commented Jun 18, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@iliaal iliaal closed this Jun 18, 2026
@iliaal iliaal deleted the feat/php-tooling branch June 18, 2026 16:16
@iliaal iliaal restored the feat/php-tooling branch June 18, 2026 16:21
@iliaal iliaal reopened this Jun 18, 2026
@evaldnet

Copy link
Copy Markdown

Tried this branch locally against current PHP tool versions (Pint 1.27.1, PHPStan 2.1.40, ECS 13.0.4) — great work, the savings are substantial. I hit two parser bugs where the expected schema no longer matches what current tooling emits. Both stem from test fixtures using an older schema, so the suite stays green. Details + fixes below.

1. pint — JSON field names changed (name/appliedFixerspath/fixers)

PintFile requires name + appliedFixers, but Pint ≥ ~1.14 (confirmed on 1.27.1) emits:

{"result":"fail","files":[{"path":"app/Foo.php","fixers":["concat_space"]}]}

The fields are required with no aliases, so serde rejects it and filter_pint_json falls back to raw output — no compression. Fix is backward-compatible aliases:

#[serde(alias = "path")]
name: String,
#[serde(rename = "appliedFixers", alias = "fixers")]
applied_fixers: Vec<String>,

2. phpstan — reports phpstan: ok while errors exist ⚠️

filter_phpstan_json gates the "ok" case on totals.errors == 0, but PHPStan puts per-file errors in totals.file_errors and leaves totals.errors (global/config errors only) at 0. A real run with 78 file errors — {"totals":{"errors":0,"file_errors":78}, ...} — returns phpstan: ok, silently hiding the failures (the summary line under-reports for the same reason). Note file_errors is currently #[allow(dead_code)]. Fix:

if phpstan.totals.file_errors == 0 && phpstan.totals.errors == 0 { return "phpstan: ok".to_string(); }
// and report phpstan.totals.file_errors in the summary line

Both fixtures set errors == file_errors (phpstan) / use the old field names (pint), which is why tests don't catch these. Happy to open a PR against this branch with both fixes + regression tests using the current-version schemas if that's useful.

pint: Pint >=1.14 renamed JSON keys name->path and appliedFixers->fixers.
The struct fields were required with no aliases, so serde rejected current
output and the filter fell back to raw (no compression). Add backward-
compatible aliases so both schemas parse.

phpstan: the "ok" gate and summary line read totals.errors, which counts
only non-file-specific (global) errors. A normal failing run reports
errors=0 with the count in file_errors, so runs with real errors were
reported as "phpstan: ok", silently hiding failures. Gate on both counts
and report file_errors in the summary.

Both regressions slipped past the suite because the fixtures set
errors == file_errors (phpstan) and used the old key names (pint). Added
regression tests using the current-version schemas.

Reported by @evaldnet (verified against Pint 1.27.1, PHPStan 2.1.40).

Co-authored-by: Aaron Florey <azza@jcaks.net>
Co-authored-by: Eli White <1153183+EliW@users.noreply.github.com>
Co-authored-by: Benjamin LETELLIER <bletellier@audencia.com>
Co-authored-by: Luciano <vandi.luciano@gmail.com>
@iliaal

iliaal commented Jun 22, 2026

Copy link
Copy Markdown
Author

Thanks for the precise report, @evaldnet — both confirmed and fixed on this branch (184ced3).

  • pint: added backward-compatible serde aliases (path/fixers) so current Pint output compresses again instead of falling back to raw.
  • phpstan: the ok gate and summary now read file_errors, so runs with file-level errors no longer report phpstan: ok.

Both ship with regression tests on the current-version schemas — the old fixtures set errors == file_errors and used the old pint keys, which is why the suite stayed green.

@EliW

EliW commented Jun 22, 2026

Copy link
Copy Markdown

@evaldnet Thanks for finding those. --- out of curiousity @iliaal --- Does this mean that the code here now may depend on which version of phpstan you are using for example? Or did I grok that incorrectly. Some folks end up stuck on older phpstan's due to framework incompatibilities at times...

@iliaal

iliaal commented Jun 22, 2026

Copy link
Copy Markdown
Author

@EliW No. Both old and current tool output parse; I checked each schema against the new code.

pint: the fix adds serde aliases, not a rename. The struct accepts both the old keys (name/appliedFixers) and the current ones (path/fixers), so it parses Pint from either side of the ~1.14 rename. Before, only the old schema parsed and current Pint fell back to raw. Confirmed: old-key and new-key JSON both compress.

phpstan: no new version dependency. totals.errors and totals.file_errors have both been in the --error-format=json output since at least 0.12, unchanged through 1.x and current 2.1.40. errors counts non-file-specific errors, file_errors counts file-specific ones. The parser already required file_errors, it was just unused. The bug read errors where it should read file_errors, which is correct on old and current PHPStan alike.

Safety net under both: if the JSON ever fails to parse, rtk falls back to raw tool output. You lose the compression for that run, not the result. So anyone stuck on an older phpstan or pint won't get broken output; worst case is unfiltered passthrough.

@evaldnet

Copy link
Copy Markdown

Rebuilt on 184ced3 and re-ran against current versions. Both fixes hold:

  • phpstan (2.1.40): file-level errors now report instead of "phpstan: ok".
  • pint (1.29.1, the new path/fixers keys): an 8-fixer file compresses to "pint: 8 changes in 1 files" with the rule list.

One separate thing I noticed while testing: the leading ./ form isn't matched for most tools, so it runs raw with no compression.

  • ./vendor/bin/pint runs raw, while vendor/bin/pint rewrites to rtk pint. Same for pest, paratest, ecs, phpunit.
  • The phpstan pattern already handles this with an optional \.?/? prefix (it also requires the analyse subcommand). The other five patterns are ^(?:vendor/bin/)?<tool>, so a leading ./ defeats the match.

Since ./vendor/bin/... is how most people invoke these in Laravel, adding the same \.?/? prefix would close it.

…ecs/pint

`./vendor/bin/<tool>` is the common Laravel invocation form. classify_command
normalizes the leading `./`, so these classify as supported, but the rewrite
strips literal `rewrite_prefixes` from the raw command and the five rules only
carried `vendor/bin/<tool>` and bare `<tool>`. So `./vendor/bin/pint` ran raw
with no compression while `vendor/bin/pint` rewrote to `rtk pint`. phpstan
already carried `./vendor/bin/phpstan`. Add the `./vendor/bin/` prefix to the
other five, with a regression test.

Reported by @evaldnet (verified against pint 1.29.1, phpstan 2.1.40).

Co-authored-by: Aaron Florey <azza@jcaks.net>
Co-authored-by: Eli White <1153183+EliW@users.noreply.github.com>
Co-authored-by: Benjamin LETELLIER <bletellier@audencia.com>
Co-authored-by: Luciano <vandi.luciano@gmail.com>
@iliaal

iliaal commented Jun 23, 2026

Copy link
Copy Markdown
Author

Fixed on the branch (bd91048). One correction on the mechanism: the regex isn't the blocker. classify_command already normalizes a leading ./ (normalize_php_tool_path strips it), so ./vendor/bin/pint classifies fine without \.?/? in the pattern. The gap is the rewrite step, which strips literal rewrite_prefixes from the raw command. The five rules only carried vendor/bin/<tool> and bare <tool>; phpstan works because ./vendor/bin/phpstan is in its prefix list, not its regex. Fix adds ./vendor/bin/<tool> to the five lists, with a regression test covering ./vendor/bin/{pint,pest,paratest,ecs,phpunit}.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort-large Plusieurs jours, nouveau module enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants