Install prebuilt rules #1296

tehbooom · 2025-09-10T18:16:33Z

This PR adds a resource to install and enable prebuilt rules. It will also update the prebuilt rules if it detects there is an update from Elastic.

This will install and enabled rules with the following tags

resource "elasticstack_kibana_prebuilt_rule" "example" {
  tags = [
    "OS: Linux",
    "OS: Windows",
    "Data Source: GCP"
  ]
}

This will only install rules and not enable any of them

resource "elasticstack_kibana_prebuilt_rule" "example" {
  tags = []
}

The initial issue wanted to support enabling all rules at once but this is not feasible and highly unlikely that a user will want to enable 1500 rules. Thus, one can only enable 1000 rules at a time.

Set the minimum version to 8.0.0 as I don't think this was supported in version 7.X and the EOL for 7.17.x is coming soon.

This PR will close #750

tobio

tags = ["OS: Linux", "OS: Windows"]

I'm not sure how I feel about this resource as it's written.

We're kind of implementing two things (install, and enable) within a single resource. That's not necessarily bad, but I wonder if it would be simpler to have two independent resources in this case.

There's also a reasonable amount of complexity in managing the tag set, and I'm not sure there's a tangible benefit for users in supporting multiple tags within a single resource. If we reduced this to a single tag then Terraform would be able to fully manage the state transitions for us.

WDYT about something like:

resource "elasticstack_kibana_install_prebuilt_rules" "rules" {}

resource "elasticstack_kibana_enable_prebuilt_rules" "linux" {
  key = "OS"
  value = "Linux"
}

resource "elasticstack_kibana_enable_prebuilt_rules" "windows" {
  key = "OS"
  value = "Windows"
}

tobio · 2025-09-10T22:07:34Z

internal/kibana/prebuilt_rules/acc_test.go

+	})
+}
+
+func testAccPrebuiltRuleDestroy(s *terraform.State) error {


If this is a no-op we can just omit it entirely from the test definition.

tobio · 2025-09-10T22:09:32Z

internal/kibana/prebuilt_rules/models.go

+	TimelinesNotUpdated   int `json:"timelines_not_updated"`
+}
+
+func (model *prebuiltRuleModel) populateFromStatus(ctx context.Context, status *prebuiltRulesStatus) diag.Diagnostics {


Suggested change

func (model *prebuiltRuleModel) populateFromStatus(ctx context.Context, status *prebuiltRulesStatus) diag.Diagnostics {

func (model *prebuiltRuleModel) populateFromStatus(ctx context.Context, status *prebuiltRulesStatus) {

This function never returns a non-nil diagnostic

tobio · 2025-09-10T22:11:30Z

internal/kibana/prebuilt_rules/models.go

+	var resp *kbapi.ReadPrebuiltRulesAndTimelinesStatusResponse
+	var err error
+
+	if spaceID != "default" {
+		resp, err = client.API.ReadPrebuiltRulesAndTimelinesStatusWithResponse(ctx, func(ctx context.Context, req *http.Request) error {
+			req.Header.Set("kbn-space-id", spaceID)
+			return nil
+		})
+	} else {
+		resp, err = client.API.ReadPrebuiltRulesAndTimelinesStatusWithResponse(ctx)
+	}


Suggested change

var resp *kbapi.ReadPrebuiltRulesAndTimelinesStatusResponse

var err error

if spaceID != "default" {

resp, err = client.API.ReadPrebuiltRulesAndTimelinesStatusWithResponse(ctx, func(ctx context.Context, req *http.Request) error {

req.Header.Set("kbn-space-id", spaceID)

return nil

})

} else {

resp, err = client.API.ReadPrebuiltRulesAndTimelinesStatusWithResponse(ctx)

}

resp, err = client.API.ReadPrebuiltRulesAndTimelinesStatusWithResponse(ctx, func(ctx context.Context, req *http.Request) error {

if spaceID != "default" {

req.Header.Set("kbn-space-id", spaceID)

return nil

}

})

tobio · 2025-09-10T22:41:38Z

internal/kibana/prebuilt_rules/schema.go

+				},
+			},
+			"tags": schema.ListAttribute{
+				Description: "A list of tag names to filter prebuilt rules for enabling/disabling. Use ['all'] to enable all prebuilt rules, or an empty list to install but not enable any rules.",


Suggested change

Description: "A list of tag names to filter prebuilt rules for enabling/disabling. Use ['all'] to enable all prebuilt rules, or an empty list to install but not enable any rules.",

Description: "A list of tag names to filter prebuilt rules for enabling/disabling.",

We're explicitly returning an error for all, it doesn't seem helpful to suggest it in the docs.
Is there a reason we expect this to be set to an empty list to install but not enable, the attribute is marked as optional so I'd expect that just omitting it entirely should be fine?

tobio · 2025-09-10T22:43:53Z

internal/kibana/prebuilt_rules/schema.go

+				ElementType: types.StringType,
+				Optional:    true,
+				Validators: []validator.List{
+					listvalidator.SizeAtLeast(0),


Suggested change

listvalidator.SizeAtLeast(0),

listvalidator.SizeAtLeast(1),

Linked to the above comment, I'd expect that if a user just wants to install the rules then they can leave tags undefined, and if they want to enable rules then they should include some tags. I don't really feel strongly about enforcing a non-empty list, but if we're going to maintain support for an empty list of tags then we can just remove this validator altogether, it's impossible to have a list with a negative size.

tobio · 2025-09-10T22:54:41Z

internal/kibana/prebuilt_rules/models.go

+	diags := performBulkActionByTags(ctx, client, spaceID, "enable", tags)
+	if diags.HasError() {
+		return diags
+	}
+
+	return nil


Suggested change

diags := performBulkActionByTags(ctx, client, spaceID, "enable", tags)

if diags.HasError() {

return diags

}

return nil

return performBulkActionByTags(ctx, client, spaceID, "enable", tags)

In theory there could be warning diags that are getting dropped here.

tobio · 2025-09-10T22:58:31Z

internal/kibana/prebuilt_rules/models.go

+	}
+
+	// Enable rules for added tags
+	if len(addedTags) > 0 {


We only enable newly added tags here, is it possible this misses potential new rules:

User create this resource with os:Windows

New rules are added matching os:Windows (Stack release? do we push rulesets outside of Stack releases IDK?)

User re-runs this resource with the same tag set. The new rules are presumably installed, but they'll never be updated because no tags were added.

tobio · 2025-09-10T23:33:21Z

internal/kibana/prebuilt_rules/models.go

+
+	var resp *kbapi.PerformRulesBulkActionResponse
+
+	if spaceID != "default" {


Same, can we move the if inside the request editor

tobio · 2025-09-10T23:37:06Z

examples/resources/elasticstack_kibana_prebuilt_rule/import.sh

@@ -0,0 +1 @@
+terraform import elasticstack_kibana_prebuilt_rule.enable "default"


The resource doesn't implement an import handler, so this command will fail.

IMO I'm not sure import actually makes sense for this resource though, it's quite difficult for us to know that all rules matching a given tag set have been enabled. I don't think there's a real impact to just running create again if the rules have been installed and the expected rules are already enabled?

tobio · 2025-09-10T23:50:12Z

internal/kibana/prebuilt_rules/models.go

+	return nil
+}
+
+func performBulkActionByTags(ctx context.Context, client *kibana_oapi.Client, spaceID, action string, tags []string) diag.Diagnostics {


I'm certainly unfamiliar with this API, so apologies if I'm missing something obvious. What prevents this resource from operating on user defined rules that also match the given tag set? That is, what restricts the resource to only enabling 'pre-built' rules, and not user defined rules which also match a given tag?

tehbooom · 2025-09-11T01:26:01Z

tags = ["OS: Linux", "OS: Windows"]

I'm not sure how I feel about this resource as it's written.

We're kind of implementing two things (install, and enable) within a single resource. That's not necessarily bad, but I wonder if it would be simpler to have two independent resources in this case.

There's also a reasonable amount of complexity in managing the tag set, and I'm not sure there's a tangible benefit for users in supporting multiple tags within a single resource. If we reduced this to a single tag then Terraform would be able to fully manage the state transitions for us.

WDYT about something like:
resource "elasticstack_kibana_install_prebuilt_rules" "rules" {}



resource "elasticstack_kibana_enable_prebuilt_rules" "linux" {

  key = "OS"

  value = "Linux"

}



resource "elasticstack_kibana_enable_prebuilt_rules" "windows" {

  key = "OS"

  value = "Windows"

}

I think this is the best way to go. One resource to install and ensure the latest prebuilt rules exist on the cluster. And another resource that enables rules based on one tag.

This also makes sense since a user may not want the prebuilt rules but wants to have terraform enable rules for them. Having everything under one resource forces the user to install the prebuilt rules.

Do we want to split this into two separate PRs for the resources? Thanks for the review! @tobio

tobio · 2025-09-11T03:12:33Z

Do we want to split this into two separate PRs for the resources?

Yea, that would be simplest IMO. It's not a hill I'm going to die on though :)

Thanks for the review! @tobio

Absolutely no problem, great to see some new additions here!

tehbooom and others added 9 commits September 10, 2025 09:57

feat: Add prebuilt_rules resource

25c08c8

fix: ensure minimum versions is satisfied

a2e726e

fix diags

8e4e945

fix versions

1f12585

fix verions

1c98f32

skip versions not 8.x+

c59ae60

fix docs with new generation

2f579a8

Merge branch 'elastic:main' into prebuilt-rules

188d206

update changelog

fad6305

tobio reviewed Sep 10, 2025

View reviewed changes

tehbooom added 2 commits September 11, 2025 11:02

fix: Only install prebuilt rules

13eea2b

fix resource name in test

517a262

tehbooom changed the title ~~Add prebuilt rules resource~~ Install prebuilt rules Oct 2, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Install prebuilt rules #1296

Install prebuilt rules #1296

Uh oh!

tehbooom commented Sep 10, 2025

Uh oh!

tobio left a comment

Uh oh!

tobio Sep 10, 2025

Uh oh!

tobio Sep 10, 2025

Uh oh!

tobio Sep 10, 2025

Uh oh!

tobio Sep 10, 2025

Uh oh!

tobio Sep 10, 2025

Uh oh!

tobio Sep 10, 2025

Uh oh!

tobio Sep 10, 2025

Uh oh!

tobio Sep 10, 2025

Uh oh!

tobio Sep 10, 2025

Uh oh!

tobio Sep 10, 2025

Uh oh!

tehbooom commented Sep 11, 2025

Uh oh!

tobio commented Sep 11, 2025

Uh oh!

Uh oh!

	func (model prebuiltRuleModel) populateFromStatus(ctx context.Context, status prebuiltRulesStatus) diag.Diagnostics {
	func (model prebuiltRuleModel) populateFromStatus(ctx context.Context, status prebuiltRulesStatus) {

	Description: "A list of tag names to filter prebuilt rules for enabling/disabling. Use ['all'] to enable all prebuilt rules, or an empty list to install but not enable any rules.",
	Description: "A list of tag names to filter prebuilt rules for enabling/disabling.",


		var resp *kbapi.PerformRulesBulkActionResponse

		if spaceID != "default" {

		@@ -0,0 +1 @@
		terraform import elasticstack_kibana_prebuilt_rule.enable "default"

Install prebuilt rules #1296

Are you sure you want to change the base?

Install prebuilt rules #1296

Uh oh!

Conversation

tehbooom commented Sep 10, 2025

Uh oh!

tobio left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tehbooom commented Sep 11, 2025

Uh oh!

tobio commented Sep 11, 2025

Uh oh!

Uh oh!