Skip to content

Conversation

imays11
Copy link
Contributor

@imays11 imays11 commented Aug 25, 2025

Pull Request

Issue link(s):

Summary - What I changed

Rule is executing as expected, however it is alerting on failed requests. Low alert telemetry.

This tuning:

  • removed markdown and edited description to be more specific
  • reduced execution window for 1 min lookback
  • name change to add AWS consistent with all other rules
  • added references that reflect in the wild threats and persistence usage
  • increased risk_score and severity to medium accounting for usage as persistence mechanism in the wild
  • added Persistence tag and Mitre tactic, technique, subtechnique
  • added event.outcome: success criteria to query
  • edited investigation guide to be more accurate reflection of steps required for investigating alert, including appropriate response action
  • added highlighted fields

Note: only IAMUser and Root user identities can call this actions so we can use aws.cloudtrail.user_identity.arn as the new terms field without worrying about Role vs Role + Session issue seen with other new_terms rules

How To Test

To test you can use this script which will create a new user, long term credentials, authenticate and run GetFederationToken API. To test manually, use an existing user identity with long term credentials and run the following or similar command (you must attach a policy):

aws sts get-federation-token --name UnauthorizedUser --policy-arns arn=arn:aws:iam::aws:policy/AdministratorAccess --duration-seconds 43200 
Screenshot 2025-08-25 at 4 47 30 PM

Rule is executing as expected, however it is alerting on failed requests. Low alert telemetry.

This tuning:
- removed markdown and edited description to be more specific
- reduced execution window for 1 min lookback
- name change to add `AWS` consistent with all other rules
- added references that reflect in the wild threats and persistence usage
- increased risk_score and severity to medium accounting for usage as persistence mechanism in the wild
- added Persistence tag and Mitre tactic, technique, subtechnique
- added `event.outcome: success` criteria to query
- edited investigation guide to be more accurate reflection of steps required for investigating alert, including appropriate response action
- added highlighted fields
** Note: only IAMUser and Root user identities can call this actions so we can use `aws.cloudtrail.user_identity.arn` as the new terms field without worrying about Role vs Role + Session issue seen with other new_terms rules
@imays11 imays11 self-assigned this Aug 25, 2025
@imays11 imays11 added Rule: Tuning tweaking or tuning an existing rule Team: TRADE and removed backport: auto labels Aug 25, 2025
Copy link
Contributor

Rule: Tuning - Guidelines

These guidelines serve as a reminder set of considerations when tuning an existing rule.

Documentation and Context

  • Detailed description of the suggested changes.
  • Provide example JSON data or screenshots.
  • Provide evidence of reducing benign events mistakenly identified as threats (False Positives).
  • Provide evidence of enhancing detection of true threats that were previously missed (False Negatives).
  • Provide evidence of optimizing resource consumption and execution time of detection rules (Performance).
  • Provide evidence of specific environment factors influencing customized rule tuning (Contextual Tuning).
  • Provide evidence of improvements made by modifying sensitivity by changing alert triggering thresholds (Threshold Adjustments).
  • Provide evidence of refining rules to better detect deviations from typical behavior (Behavioral Tuning).
  • Provide evidence of improvements of adjusting rules based on time-based patterns (Temporal Tuning).
  • Provide reasoning of adjusting priority or severity levels of alerts (Severity Tuning).
  • Provide evidence of improving quality integrity of our data used by detection rules (Data Quality).
  • Ensure the tuning includes necessary updates to the release documentation and versioning.

Rule Metadata Checks

  • updated_date matches the date of tuning PR merged.
  • min_stack_version should support the widest stack versions.
  • name and description should be descriptive and not include typos.
  • query should be inclusive, not overly exclusive. Review to ensure the original intent of the rule is maintained.

Testing and Validation

  • Validate that the tuned rule's performance is satisfactory and does not negatively impact the stack.
  • Ensure that the tuned rule has a low false positive rate.

@imays11 imays11 merged commit 590cc9c into main Aug 29, 2025
14 of 21 checks passed
@imays11 imays11 deleted the tuning_federated_session_token branch August 29, 2025 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants