Add Trustabl Agent Scanner to CI - #261
Conversation
|
|
|
CLA Not Signed The Contributor License Agreement (CLA) check is currently pending on this PR ( @joshua-trustabl please sign the CLA via the CLA assistant badge in the comment above, or visit https://cla-assistant.io/alibaba/loongsuite-python. Once signed, the Automated check by github-manager-bot |
ralf0131
left a comment
ralf0131
left a comment
There was a problem hiding this comment.
Summary
This PR adds a Trustabl Agent Scanner GitHub Action workflow to detect agent runtime reliability issues. The change is small (17 lines, single workflow file), but there are security concerns that should be addressed before merging.
Verdict: The action reference must be pinned to a commit SHA, and permissions should follow least-privilege. Additionally, CLA must be signed before this can be approved.
Findings
- [Critical]
.github/workflows/trustabl.yml:16— Action not pinned to commit SHA (supply chain risk) - [Warning]
.github/workflows/trustabl.yml:10— Overly broad permissions (pull-requests: write, security-events: write)
Suggestions
- Pin the action to a full commit SHA:
trustabl/trustabl-action@<sha> - Reduce permissions to only what the scanner actually needs
- Please sign the CLA (see the CLA bot comment above) — required before this PR can be merged
- Consider adding a brief description of what data the scanner collects and whether it sends data externally
Automated review by github-manager-bot
| scan: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 |
There was a problem hiding this comment.
Security: Action not pinned to commit SHA. trustabl/trustabl-action@v0 uses a mutable tag reference. GitHub Actions security best practice requires pinning third-party actions to a full commit SHA (e.g., trustabl/trustabl-action@<40-char-sha>) to prevent supply chain attacks via tag mutation. See GitHub Security Hardening.
| permissions: | ||
| contents: read | ||
| security-events: write | ||
| pull-requests: write |
There was a problem hiding this comment.
Overly broad permissions. The workflow grants pull-requests: write and security-events: write, but a read-only scanner typically only needs contents: read. Consider applying the principle of least privilege and removing unnecessary write permissions. If the action does need to post findings as PR comments, document why in the PR description.
ralf0131
left a comment
ralf0131
left a comment
There was a problem hiding this comment.
Summary
Adds Trustabl Agent Scanner CI workflow — a lightweight GitHub Action that scans for AI agent reliability issues. The workflow configuration is clean, permissions are appropriately scoped, and the change is non-intrusive.
LGTM — welcome contribution from a first-time contributor! 🎉
Automated review by github-manager-bot
|
Automated notice by github-manager-bot |
ralf0131
left a comment
ralf0131
left a comment
There was a problem hiding this comment.
Summary
This PR adds a Trustabl Agent Scanner GitHub Action workflow to CI. The workflow configuration looks reasonable in terms of permissions scope.
Note: This PR cannot be approved until the contributor signs the CLA (see the CLA reminder comment above).
Findings
- [Warning] .github/workflows/trustabl.yml:16 — Using a floating tag @v0 is a supply-chain risk; consider pinning to a commit SHA.
Automated review by github-manager-bot
| scan: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 |
There was a problem hiding this comment.
[Security] Using a floating tag trustabl/trustabl-action@v0 is a supply-chain risk — tags can be moved to point to different commits. Consider pinning to a specific commit SHA for reproducibility.
ralf0131
left a comment
ralf0131
left a comment
There was a problem hiding this comment.
Summary
This PR adds a Trustabl Agent Scanner CI workflow to detect reliability and security issues in agent code. The change is small (single workflow file, 17 lines) and non-intrusive.
Two minor suggestions: pin the third-party action to a commit SHA for supply-chain security, and review whether pull-requests: write permission is necessary.
Note: CLA must be signed before this PR can be merged.
Automated review by github-manager-bot
| scan: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 |
There was a problem hiding this comment.
[Security] Third-party action pinned to mutable tag @v0. Best practice is to pin to a full commit SHA to prevent supply-chain attacks (the tag can be force-pushed to a different commit). Consider using - uses: trustabl/trustabl-action@<commit-sha> instead. Additionally, v0 indicates pre-1.0 — breaking changes may occur without notice.
| contents: read | ||
| security-events: write | ||
| pull-requests: write | ||
|
|
There was a problem hiding this comment.
[Permissions] pull-requests: write is broader than needed for a read-only scanner. If the action only posts SARIF results, security-events: write suffices. If it posts PR comments, this is justified but worth documenting why.
3 participants
We came across your repo and we like how you're providing instrumentation for Python applications to contribute to Alibaba's unified observability data collection suite. We scanned the repo, and noticed agent runtime reliability findings that might be worth reviewing.
[HIGH] Claude subagent is granted filesystem-write built-ins
File: instrumentation-genai/opentelemetry-instrumentation-claude-agent-sdk/examples/manual/main.py
What it means: This AgentDefinition grants a filesystem-write built-in (Write, Edit, MultiEdit, or NotebookEdit), letting the subagent modify files on disk.
[HIGH] Claude subagent is granted filesystem-write built-ins
File: instrumentation-genai/opentelemetry-instrumentation-claude-agent-sdk/examples/zero-code/main.py
What it means: This AgentDefinition grants a filesystem-write built-in (Write, Edit, MultiEdit, or NotebookEdit), letting the subagent modify files on disk.
Recommendations are based on our understanding of agent runtime reliability, some findings may be intentional. Please let us know if this was intentional or if our findings are helpful so we can improve the accuracy of the scanner.
Best,
Trustabl.ai
Open-source AI agent reliability scanner (runs locally, GitHub Action)