Skip to content

[DOCS-11316] Add individual detection rules docs #31199

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 18 commits into
base: may/cloud-siem-nav-restructure
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 31 additions & 1 deletion config/_default/menus/main.en.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6035,11 +6035,41 @@ menu:
parent: cloud_siem_detect_and_monitor
identifier: cloud_siem_custom_detection_rules
weight: 201
- name: Threshold
url: security/cloud_siem/detect_and_monitor/custom_detection_rules/threshold
parent: cloud_siem_custom_detection_rules
identifier: cloud_siem_threshold_rule
weight: 2011
- name: New Value
url: security/cloud_siem/detect_and_monitor/custom_detection_rules/new_value
parent: cloud_siem_custom_detection_rules
identifier: cloud_siem_new_value_rule
weight: 2012
- name: Anomaly
url: security/cloud_siem/detect_and_monitor/custom_detection_rules/anomaly
parent: cloud_siem_custom_detection_rules
identifier: cloud_siem_anomaly_rule
weight: 2103
- name: Content Anomaly
url: security/cloud_siem/detect_and_monitor/custom_detection_rules/content_anomaly
parent: cloud_siem_custom_detection_rules
identifier: cloud_siem_content_anomaly_rule
weight: 2104
- name: Impossible Travel
url: security/cloud_siem/detect_and_monitor/custom_detection_rules/impossible_travel
parent: cloud_siem_custom_detection_rules
identifier: cloud_siem_impossible_travel_rule
weight: 2105
- name: Third Party
url: security/cloud_siem/detect_and_monitor/custom_detection_rules/third_party
parent: cloud_siem_custom_detection_rules
identifier: cloud_siem_third_party_rule
weight: 2106
- name: Signal Correlation
url: security/cloud_siem/detect_and_monitor/custom_detection_rules/signal_correlation_rules
parent: cloud_siem_custom_detection_rules
identifier: cloud_siem_signal_correlation_rules
weight: 2101
weight: 2107
- name: OOTB Rules
url: /security/default_rules/#cat-cloud-siem-log-detection
parent: cloud_siem_detect_and_monitor
Expand Down
9 changes: 9 additions & 0 deletions content/.gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,12 @@

<<<<<<< HEAD
# THIS IS A GENERATED FILE. Manual edits will be overwritten.

# To ignore a content file manually, add it to the .gitignore file in the root of the documentation repository: https://github.com/DataDog/documentation/blob/master/.gitignore

# This file lists compiled Cdocs files to keep them out of version control. For more information, see the internal Cdocs documentation: https://datadoghq.atlassian.net/wiki/spaces/docs4docs/pages/4898063037/Cdocs+Build

=======
# This file lists compiled Cdocs files to keep them out of version control. For more information, see the internal Cdocs documentation: https://datadoghq.atlassian.net/wiki/spaces/docs4docs/pages/4898063037/Cdocs+Build

# For the list of files to ignore in the documentation repo, see the version in the root of the documentation repository: https://github.com/DataDog/documentation/blob/master/.gitignore
Expand All @@ -19,6 +27,7 @@
# For the list of files to ignore in the documentation repo, see the version in the root of the documentation repository: https://github.com/DataDog/documentation/blob/master/.gitignore


>>>>>>> may/cloud-siem-nav-restructure
/en/product_analytics/session_replay/mobile/setup_and_configuration.md
/en/real_user_monitoring/guide/proxy-mobile-rum-data.md
/en/real_user_monitoring/guide/proxy-rum-data.md
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -245,9 +245,7 @@ To use unit testing:

### Trigger

{{< img src="security/security_monitoring/detection_rules/define_rule_case2.png" alt="The set rule case section showing the default settings" style="width:80%;" >}}

Enable **Create rules cases with the Then operator** if you want to trigger a signal for the example: If query A occurs and then query B occurs. The `then` operator can only be used on a single rule case.

All rule cases are evaluated as case statements. Thus, the order of the cases affects which notifications are sent because the first case to match generates the signal. Click and drag your rule cases to change their ordering.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
---
title: Anomaly
disable_toc: false
---

## Overview

When configuring a specific threshold isn't an option, you can define an anomaly detection rule instead. With anomaly detection, a dynamic threshold is automatically derived from the past observations of the events.

## Create a rule

To create a threshold detection rule or job, navigate to the [Detection Rules][1] page and click **+ New Rule**.

### Create a New Rule

Select a **Real-Time Rule**, **Scheduled Rule** or a **Historical Job**.

### Define your rule or historical job

If you are creating a historical job, select the logs index and time range for the job.

Select the **Anomaly** tile.

### Define search queries

{{< img src="security/security_monitoring/detection_rules/threshold_20250310.png" alt="Define the search query" style="width:100%;" >}}

Cloud SIEM can analyze logs, Audit Trail events, and events from Event Management. To search Audit Trail or events from Events Management, click the down arrow next to **Logs** and select **Audit Trail** or **Events**. Construct a search query for your logs or events using the [Log Explorer search syntax][2].

Optionally, define a unique count and signal grouping. Count the number of unique values observed for an attribute in a given time frame. The defined `group by` generates a signal for each `group by` value. Typically, the `group by` is an entity (like user, or IP). The Group By is also used to [join the queries together](#joining-queries).

Anomaly detection inspects how the `group by` attribute has behaved in the past. If a `group by` attribute is seen for the first time (for example, the first time an IP is communicating with your system) and is anomalous, it does not generate a security signal because the anomaly detection algorithm has no historical data to base its decision on.

**Note**: The query applies to all ingested logs and events.

#### Filter logs based on Reference Tables

{{% filter_by_reference_tables %}}

{{< img src="/security/security_monitoring/detection_rules/filter-by-reference-table.png" alt="The log detection rule query editor with the reference table search options highlighted" style="width:100%;" >}}

#### Unit testing

{{% cloud_siem/unit_test %}}

To finish setting up the detection rule, select the type of rule you are creating and follow the instructions.

{{< tabs >}}
{{% tab "Real-time rule" %}}

### Set conditions

#### Severity and notification

{{% security-rule-severity-notification %}}

#### Time windows

Datadog automatically detects the seasonality of the data and generates a security signal when the data is determined to be anomalous.

After a signal is generated, the signal remains "open" if the data remains anomalous and the last updated timestamp is updated for the anomalous duration.

A signal "closes" after the time period exceeds the maximum signal duration, regardless of whether or not the anomaly is still anomalous. This time is calculated from the first seen timestamp.

#### Other parameters

In the **Rule multi-triggering behavior** section, select how often you want to keep updating the same signal if new values are detected.

Toggle **Decrease severity for non-production environment** if you want to prioritize production environment signals over non-production signals. See [Decreasing non-production severity](#decreasing-non-production-severity) for more information.

Toggle **Enable Optional Group By** section, if you want to group events even when values are missing. If there is a missing value, a sample value is generated to avoid selection exclusion.

##### Decreasing non-production severity

{{% cloud_siem/decreasing_non_prod_severity %}}

### Describe your playbook

{{% security-rule-say-whats-happening %}}

### Create a suppression

{{% cloud_siem/create_suppression %}}

{{% /tab %}}
{{% tab "Scheduled rule" %}}

### Set conditions

#### Severity and notification

{{% security-rule-severity-notification %}}

#### Time windows

Datadog automatically detects the seasonality of the data and generates a security signal when the data is determined to be anomalous.

After a signal is generated, the signal remains "open" if the data remains anomalous and the last updated timestamp is updated for the anomalous duration.

A signal "closes" after the time period exceeds the maximum signal duration, regardless of whether or not the anomaly is still anomalous. This time is calculated from the first seen timestamp.

#### Other parameters

In the **Rule multi-triggering behavior** section, select how often you want to keep updating the same signal if new values are detected.

Toggle **Decrease severity for non-production environment** if you want to prioritize production environment signals over non-production signals. See [Decreasing non-production severity](#decreasing-non-production-severity) for more information.

Toggle **Enable Optional Group By** section, if you want to group events even when values are missing. If there is a missing value, a sample value is generated to avoid selection exclusion.

### Add custom schedule

{{% cloud_siem/add_custom_schedule %}}

### Describe your playbook

{{% security-rule-say-whats-happening %}}

{{% /tab %}}
{{% tab "Historical job" %}}

### Set conditions

#### Other parameters

In the **Job multi-triggering behavior** section, select how often you want to keep updating the same signal if new values are detected within a specified time frame. For example, the same signal updates if any new value is detected within 1 hour, for a maximum duration of 24 hours.

**Note**: If a unique signal is required for every new value, configure this value to `0` minutes.

Toggle **Enable Optional Group By** section, if you want to group events even when values are missing. If there is a missing value, a sample value is generated to avoid selection exclusion.

### Notify when job is complete

{{% cloud_siem/notify_when_job_complete %}}

### Describe your playbook

{{% security-rule-say-whats-happening %}}

Click **Save Rule**.

{{% /tab %}}
{{< /tabs >}}

[1]: https://app.datadoghq.com/security/configuration/siem/rules
[2]: /logs/search_syntax/
Loading
Loading