Skip to content

Add support to detect Assisted-By trailer#33

Open
omkar-foss wants to merge 4 commits into
chaoss:mainfrom
omkar-foss:support-assistedby-trailer
Open

Add support to detect Assisted-By trailer#33
omkar-foss wants to merge 4 commits into
chaoss:mainfrom
omkar-foss:support-assistedby-trailer

Conversation

@omkar-foss

@omkar-foss omkar-foss commented May 25, 2026

Copy link
Copy Markdown
Contributor

Closes #28 and #53.

This PR adds support for detecting one or more Assisted-By trailers in git commit message. Also includes tests for supported cases.

Changes in this PR:

@MoralCode MoralCode left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change diff feels like it is rewriting lots of the other functions that are checking for signs of other AI related uses besides Assisted-By trailiers.

I like the additional unit tests, but If theres additional formatting/refactoring changes that need to be made, I'd like those to be explained in the PR description or discussed beforehand.

Is there anything preventing this change from being a handful of lines of new unit tests and a smaller handful of lines to add a new entry to the commitMessagePatterns list that satisfies the unit tests?

@omkar-foss

omkar-foss commented May 26, 2026

Copy link
Copy Markdown
Contributor Author

Thanks Adrian, I've added a Changes list to PR description above, which should hopefully answer all questions related to other function changes. Kindly go through it and let me know if you've any follow-up questions.

@omkar-foss omkar-foss requested a review from MoralCode May 26, 2026 05:34
@andrew

andrew commented May 26, 2026

Copy link
Copy Markdown
Contributor

Thanks for the detailed writeup. I agree with Adrian that the diff is larger than it needs to be.

The root issue is that Finding.Tool is being used to carry a comma-joined list which then gets split apart again in scan.go and output.go. Detect() already returns []Finding, so the cleaner fix is to emit one finding per Assisted-By line. That removes the scan.go and output.go changes entirely and means the existing checkers don't need to be rewritten.

A few other things:

  • Assisted-By: Kimi K2.6 is silently dropped because it's not in SupportedTools. An explicit Assisted-By trailer is itself the disclosure, so we should report it with the verbatim tool name even if we don't recognise it.
  • message importing toolmention couples two sibling detectors. If the tool list needs sharing it should move somewhere neutral first, but per the point above I'm not sure it's needed here at all.
  • Adding bare "Gemini" to the tools list changes the behaviour of the toolmention detector, which is out of scope for this PR.
  • Minor: var getMatchedTools = func(...) can be a plain func; matches == nil || len(matches) == 0 reduces to len(matches) == 0; strings.Index(toolName, ",") >= 1 should be strings.Contains.

I'd suggest making Assisted-By its own detector package alongside coauthor, emitting one finding per trailer, and not gating on a known-tools list. That should come out to roughly 40 lines plus the tests you've already written, with no changes to the existing checkers.

@omkar-foss

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed review, I'll update the PR accordingly.

@omkar-foss omkar-foss force-pushed the support-assistedby-trailer branch from c707d90 to e59e9dc Compare May 26, 2026 17:21
@omkar-foss

Copy link
Copy Markdown
Contributor Author

@andrew I've updated the PR as per your suggestions here. Please find all changes in description here. Thanks.

@MoralCode

Copy link
Copy Markdown
Contributor

i see this new version adds an entirely new type of detector for assisted by trailers. I dont see why this needs to be a whole new detector.

Assisted-By is a trailer, which means its something that is added to the end of a commit message. Id like to reuse the existing detector we have for commit messages.

@omkar-foss omkar-foss force-pushed the support-assistedby-trailer branch from e59e9dc to d90cf1e Compare May 26, 2026 17:36

@MoralCode MoralCode left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not require a new detector, we already have a detector for commit messages

Comment thread scan/scan_test.go Outdated
@omkar-foss

Copy link
Copy Markdown
Contributor Author

This does not require a new detector, we already have a detector for commit messages

I see two main reasons why we should keep it in a separate detector:

  1. separate detector design has a cleaner way to have one Finding per tool detected. adding it to commit messages' detector is cluttering things and increasing diff, as we saw in my previous changes in this PR.
  2. we'll be able to keep scoring separate for each detector as we discussed few days back in Refactor confidence scoring to be based on a numeric system #12.

Also this was suggested in @andrew's suggestions comment which I've fully incorporated:

I'd suggest making Assisted-By its own detector package alongside coauthor, emitting one finding per trailer, and not gating on a known-tools list. That should come out to roughly 40 lines plus the tests you've already written, with no changes to the existing checkers.

@omkar-foss

omkar-foss commented May 27, 2026

Copy link
Copy Markdown
Contributor Author

Additionally I think Assisted-By and Co-authored-by should have their own detectors since they're more widely available trailers and becoming standard (and so are more prone to format iterations in near future). While the others (Replit, EntireIO, etc) are tool-specific.

So these are separate concerns and we should try to keep them separate to encourage the fact that newcomers can easily develop and maintain them separately.

Signed-off-by: Omkar P <45419097+omkar-foss@users.noreply.github.com>
@omkar-foss omkar-foss force-pushed the support-assistedby-trailer branch from d90cf1e to 7fd2837 Compare June 25, 2026 11:08
@omkar-foss

Copy link
Copy Markdown
Contributor Author

PR rebased with main, conflicts resolved, ready to review. Please have a look when possible, thanks.

Signed-off-by: Omkar P <45419097+omkar-foss@users.noreply.github.com>
@omkar-foss

omkar-foss commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

@MoralCode Hey, as per our discussion in #49, I've got the trailer-based detection logic together. Adding it to this PR since the discussion started here with Assisted-By addition.

So now we've only 1 package trailer for Co-Authored-By, Assisted-By and all other trailer based detection which we had in message package previously.

PR is ready to review from my side, please have a look. Since this is a big change, I'll merge this only after approval from @andrew or you. Thanks.

Issue for reference: #53

@omkar-foss omkar-foss requested a review from andrew July 8, 2026 09:23

@andrew andrew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for reworking this. The consolidation into one package is fine and tests pass locally. A few things before merging:

detection/constants.go extracts constants that aren't shared. Every value in that file is used by exactly one detector: KnownAgentCommitters/GithubNoReplyEmailSuffix only by committer, SupportedToolsInMentions only by toolmention, GitNotesAuthorshipPrefix only by gitnotes, and the rest only by trailer. Nothing is reused across packages, so this is pure indirection: to change the Replit regex you now edit a different file from the code that runs it. I'd keep each constant next to its sole consumer and only lift things into detection when a second package actually needs them. That also drops the committer.go, gitnotes.go and toolmention.go diffs from this PR entirely.

extractToolsFromText rewrites the tool name instead of reporting it verbatim. Assisted-By: gpt-4o comes out as Gpt-4o, llama.cpp as Llama.cpp. The trailer is the disclosure, so we should report what the user wrote. Related:

  • Assisted-By: <noreply@anthropic.com> (email only, no name) yields zero findings, so an explicit disclosure is silently dropped.
  • strings.Split(text, "\n") is dead code: AssistedByPattern captures [^\r\n]+, so match[1] never contains a newline.

I'd reduce the function to: strip a trailing <…>, strip a trailing (…), TrimSpace, and if that leaves an empty string fall back to the raw trailer value so nothing is dropped.

No dedup on Assisted-By. Two Assisted-By: Claude lines produce two identical findings. detectTrailerCoauthoredBy already dedupes via a seen map; Assisted-By should do the same.

Duplicate subtest names in trailer_test.go. "Claude trailer with Opus model", "Cursor trailer", "multiple AI trailers", "case variation" etc. each appear twice. Go appends #01 so they still run, but you can't -run a specific one. Prefixing with coauthor: / assistedby: would fix it.

Smaller bits:

  • copilot@github.com was added to KnownCoAuthorEmails. Fine, but worth its own commit or a mention in the description.
  • detectCustomTrailers handles the aider: subject prefix and Generated with Claude Code body text, neither of which is a trailer. Something like detectMessagePatterns would be more accurate.
  • In Detect, var findings; findings = slices.Concat(...); return findings can be return slices.Concat(...).

The constants extraction is the main one; the rest are small.

Signed-off-by: Omkar P <45419097+omkar-foss@users.noreply.github.com>
Signed-off-by: Omkar P <45419097+omkar-foss@users.noreply.github.com>
@omkar-foss

Copy link
Copy Markdown
Contributor Author

Hi @andrew thanks for your reviewing, I've updated the PR as per your comments, my comments below quoting yours:

detection/constants.go extracts constants that aren't shared. Every value in that file is used by exactly one detector: KnownAgentCommitters/GithubNoReplyEmailSuffix only by committer, SupportedToolsInMentions only by toolmention, GitNotesAuthorshipPrefix only by gitnotes, and the rest only by trailer. Nothing is reused across packages, so this is pure indirection: to change the Replit regex you now edit a different file from the code that runs it. I'd keep each constant next to its sole consumer and only lift things into detection when a second package actually needs them. That also drops the committer.go, gitnotes.go and toolmention.go diffs from this PR entirely.

It'll be great if we could retain this. Idea is to keep all the detection related constants together so we can incrementally replace them with more dynamic lists, see: #42

extractToolsFromText rewrites the tool name instead of reporting it verbatim. Assisted-By: gpt-4o comes out as Gpt-4o, llama.cpp as Llama.cpp. The trailer is the disclosure, so we should report what the user wrote. Related:
Assisted-By: noreply@anthropic.com (email only, no name) yields zero findings, so an explicit disclosure is silently dropped.
strings.Split(text, "\n") is dead code: AssistedByPattern captures [^\r\n]+, so match[1] never contains a newline.

This is expected behaviour, I'm capitalizing the tool name for readability. Also Assisted-By: <noreply@anthropic.com> doesn't seem to be a valid trailer so zero findings is the correct result. The strings.Split(text, "\n") is absolutely required here since there could be cases where the Assisted-By trailer line itself could be multiline, for ref, see this trailer in test.

No dedup on Assisted-By. Two Assisted-By: Claude lines produce two identical findings. detectTrailerCoauthoredBy already dedupes via a seen map; Assisted-By should do the same.

Done, added dedupe logic with corresponding test. Thanks.

Duplicate subtest names in trailer_test.go. "Claude trailer with Opus model", "Cursor trailer", "multiple AI trailers", "case variation" etc. each appear twice. Go appends #1 so they still run, but you can't -run a specific one. Prefixing with coauthor: / assistedby: would fix it.

Done, added the prefixes. Thanks.

Smaller bits:
copilot@github.com was added to KnownCoAuthorEmails. Fine, but worth its own commit or a mention in the description.
detectCustomTrailers handles the aider: subject prefix and Generated with Claude Code body text, neither of which is a trailer. Something like detectMessagePatterns would be more accurate.
In Detect, var findings; findings = slices.Concat(...); return findings can be return slices.Concat(...).

Thanks, have added a line about KnownCoAuthorEmails to PR description above (see last point under changes). Also have renamed the function and returned findings accordingly. All set!

@omkar-foss omkar-foss requested a review from andrew July 8, 2026 10:36

@andrew andrew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates. Dedup, test names, rename and the return simplification all look good.

On constants.go: #42 is really about the tool-name lists rather than parsing patterns like ReplitAttributionRegex or GitNotesAuthorshipPrefix, but I'm happy to leave it and revisit when #42 lands.

One factual note on the \n split, non-blocking: AssistedByPattern's capture group is [^\r\n]+?, so match[1] is always a single line. For the multiline test case the regex returns "Claude 4.7 Opus" and the indented (logic...) continuation is never passed to extractToolsFromText — the test passes because the continuation is ignored, not because the split handles it. Fine to leave as-is.

LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

Add Support for Assisted-by commit trailer

3 participants