Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Workload Management] Add rule schema for workload management #17238

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

ruai0511
Copy link
Contributor

@ruai0511 ruai0511 commented Feb 3, 2025

Description

This PR introduces the schema for Rule object used in the workload management feature

{
    "_id": "fwhbuib397u4o03=="
    "index_pattern": ["logs123", "user*"],
    "query_group": "dev_query_group_id",
    "updated_at": "01-10-2025T21:23:456Z"
}

Related issues:
RFC: #16797
#16813

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

Copy link
Contributor

github-actions bot commented Feb 4, 2025

✅ Gradle check result for f6a4a28: SUCCESS

Copy link

codecov bot commented Feb 4, 2025

Codecov Report

Attention: Patch coverage is 82.60870% with 24 lines in your changes missing coverage. Please review.

Project coverage is 72.44%. Comparing base (faabd10) to head (bb8d06c).
Report is 19 commits behind head on main.

Files with missing lines Patch % Lines
server/src/main/java/org/opensearch/wlm/Rule.java 82.60% 9 Missing and 15 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #17238      +/-   ##
============================================
+ Coverage     72.36%   72.44%   +0.07%     
- Complexity    65733    65825      +92     
============================================
  Files          5318     5319       +1     
  Lines        305674   305812     +138     
  Branches      44349    44376      +27     
============================================
+ Hits         221197   221535     +338     
+ Misses        66341    66128     -213     
- Partials      18136    18149      +13     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

CHANGELOG.md Outdated Show resolved Hide resolved
server/src/main/java/org/opensearch/wlm/Rule.java Outdated Show resolved Hide resolved
server/src/main/java/org/opensearch/wlm/Rule.java Outdated Show resolved Hide resolved
server/src/main/java/org/opensearch/wlm/Rule.java Outdated Show resolved Hide resolved
server/src/main/java/org/opensearch/wlm/Rule.java Outdated Show resolved Hide resolved
server/src/main/java/org/opensearch/wlm/Rule.java Outdated Show resolved Hide resolved
server/src/main/java/org/opensearch/wlm/Rule.java Outdated Show resolved Hide resolved
server/src/main/java/org/opensearch/wlm/Rule.java Outdated Show resolved Hide resolved
server/src/main/java/org/opensearch/wlm/Rule.java Outdated Show resolved Hide resolved
server/src/main/java/org/opensearch/wlm/Rule.java Outdated Show resolved Hide resolved
@ruai0511 ruai0511 added the backport 2.x Backport to 2.x branch label Feb 7, 2025
Copy link
Contributor

github-actions bot commented Feb 7, 2025

❕ Gradle check result for bb8d06c: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Comment on lines +37 to +38
* "_id": "fwehf8302582mglfio349==",
* "index_pattern": ["logs123", "user*"],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also have customer friendly - name/description fields for a rule?

Comment on lines +166 to +169
QUERY_GROUP("query_group", Set.of(RuleAttribute.INDEX_PATTERN));

private final String name;
private final Set<RuleAttribute> allowedAttributes;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This design of using same rules for multiple features becomes really tricky if they have different allowed attribute values. Having more attributes present than the ones that apply to a rule can be very confusing and if we are not allowing that, the possible attributes is the intersection of allowed attributes for all the features.

Comment on lines +39 to +41
* "query_group": "dev_query_group_id",
* "updated_at": "01-10-2025T21:23:21.456Z"
* }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ruai0511 / @kaushalmahi12 - Can we add some more examples of how these rules will look if we have multiple features and multiple attributes tomorrow? Especially when the allowed attributes for those features are not common

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A single Rule entity will always map to a single feature. Hence each feature will define their set of allowed attributes.

For example since the slow logs are defined at index level. It doesn't make sense for s.ow log feature to consider index_pattern as attribute. It can define next layer of attributes to provide more specific details about the slow logs,

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah a single rule will only have one feature but could have multiple attributes.
For example,

{
    "_id": "fwhbuib397u4o03=="
    "attribute_1": ["logs123", "user*"],
    "attribute_2": ["logs123", "user*"],
    "query_group": "dev_query_group_id",
    "updated_at": "01-10-2025T21:23:456Z"
}

Here the feature is query_group, and corresponds to the tag dev_query_group_id

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should move this file under a new package e,g; auto_tagging since the construct is meant to be used in generic context but the structure hints a specific use.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants