Modeling AI-agent threats as a Gemara threat catalogue: capability granularity and a small slice to validate

[eeee2345]

Hello. I maintain Agent Threat Rules (ATR), an open MIT-licensed detection ruleset for AI agent attacks: over 640 machine-readable rules across 10 attack categories, each already cross-referenced to OWASP Agentic and LLM, MITRE ATLAS, NIST AI RMF, EU AI Act, and ISO 42001.

I would like to explore expressing these AI-agent threats as a Gemara threat catalogue, and I want to get the modeling right before proposing anything substantial.

Two questions:

1. capabilities is required on a Threat. For agent-attack threats (for example tool-response injection, or cross-agent privilege escalation), what granularity do you intend for capabilities? The Capability Profiles work in [Question] Should we support Capability Profiles? #419 looks directly related. Is there guidance or an example I should follow so a catalogue fits that direction?

2. Would a small slice of 5 to 10 threats be the right way to validate the modeling first, before a fuller catalogue?

Happy to follow the ADR process. I am not looking to drop a full catalogue; I would rather align on the model first.

[jpower432]

Hi @eeee2345. Welcome! Really glad to see interest in modeling AI agent threats with Gemara, and ATR's existing cross-references to OWASP, MITRE ATLAS, etc. would map well to Gemara's #MappingDocument artifacts too.

On question 1: One clarification -- Gemara defines the schema, but does not store the content. Your ATR catalog would live in your repository, validated against the Gemara schemas. However, we are working on tooling to help with distribution, specifically via OCI artfiacts 👉 https://github.com/gemaraproj/gemara-publish-action.

capabilities on a #Threat are a reference to entries in a separate #CapabilityCatalog. You'd author an AI agent #CapabilityCatalog describing what agent systems can do, things like tool invocation, cross-agent message passing, then reference those from your threats. A capability should describe a feature or function that creates the opportunity for the threat. FINOS Common Cloud Controls catalogs are a good example to look at.

The Capability Profiles work in #419 is about how external CUE modules can extend #Capability with domain-specific fields. A plain #CapabilityCatalog can work today, but if you're interested in building an AI agent Capability profile, I'd be happy to work on that with you. An ADR would be the right vehicle for proposing it.

On question 2: Yes, a small slice is the right approach. 5-10 threats with a companion capability catalog would be a great way to validate and we have a tutorial that may be helpful there. Happy to help you work through it -- feel free to drop examples here and we can iterate together.

[eeee2345]

Thanks @jpower432 — that clears up the split: I author and host the content (an AI-agent #CapabilityCatalog + the ATR threat mappings) in the ATR repo, validated against Gemara's schemas, with gemara-publish-action for distribution. That's a clean fit — ATR is already a versioned, independent corpus, so keeping the content on our side and validating against your schema is exactly the model we'd want.

For the #CapabilityCatalog, I'd describe the agent capabilities that create the attack surface — tool invocation, MCP / cross-agent message passing, skill/plugin loading, memory read+write — and reference those from the threats. I'll draft the catalog plus a first batch of ATR #Threat mappings, validate against the schemas, and share the repo link here for review. Does that granularity for capabilities match what you'd want, or would you scope them finer/coarser?

[jpower432]

Hi @eeee2345. Sounds good, I think we are on the same page. The capabilities granularity sounds right to me. I will start reviewing your PR and hopefully can provide more concrete feedback there.