Updating Test Benchmark to Sysbench #1476

shekhar316 · 2025-01-24T09:35:55Z

Description

This PR replaces TFB benchmark to sysbench for local monitoring use case.

How has this been tested?

Openshift Cluster

Test Configuration

Kubernetes clusters tested on: Openshift

Checklist 🎯

Additional information

Include any additional information such as links, test results, screenshots here

chandrams · 2025-02-18T09:13:34Z

@shekhar316 - Can you rebase the PR and update the status on this one please

Signed-off-by: Shekhar Saxena <[email protected]>

shreyabiradar07 · 2025-02-18T09:49:17Z

@shekhar316 please fix the syntax error

<frozen importlib._bootstrap>:671: in _load_unlocked
    ???
/home/runner/.local/lib/python3.8/site-packages/_pytest/assertion/rewrite.py:175: in exec_module
    source_stat, co = _rewrite_test(fn, self.config)
/home/runner/.local/lib/python3.8/site-packages/_pytest/assertion/rewrite.py:355: in _rewrite_test
    tree = ast.parse(source, filename=strfn)
/usr/lib/python3.8/ast.py:47: in parse
    return compile(source, filename, mode, flags,
E     File "/home/runner/work/autotune/autotune/tests/scripts/local_monitoring_tests/rest_apis/test_local_monitoring_e2e_workflow.py", line 165
E       <<<<<<< HEAD
E       ^
E   SyntaxError: invalid syntax

Signed-off-by: Shekhar Saxena <[email protected]>

shekhar316 · 2025-02-18T09:53:42Z

@shekhar316 please fix the syntax error

<frozen importlib._bootstrap>:671: in _load_unlocked
    ???
/home/runner/.local/lib/python3.8/site-packages/_pytest/assertion/rewrite.py:175: in exec_module
    source_stat, co = _rewrite_test(fn, self.config)
/home/runner/.local/lib/python3.8/site-packages/_pytest/assertion/rewrite.py:355: in _rewrite_test
    tree = ast.parse(source, filename=strfn)
/usr/lib/python3.8/ast.py:47: in parse
    return compile(source, filename, mode, flags,
E     File "/home/runner/work/autotune/autotune/tests/scripts/local_monitoring_tests/rest_apis/test_local_monitoring_e2e_workflow.py", line 165
E       <<<<<<< HEAD
E       ^
E   SyntaxError: invalid syntax

Missed this conflict. Resolved now.
I'll add the test results also.

shreyabiradar07 · 2025-02-18T11:17:38Z

@shekhar316 looks like test_crc_manifest_local_monitoring PR check is failing during recommendation validation check due to "There is not enough data available to generate a recommendation."

Is 4 mins threshold enough for short_term recommendation generation or should the measurement_duration be reduced?

Signed-off-by: Shekhar Saxena <[email protected]>

chandrams · 2025-02-19T06:29:38Z

tests/scripts/local_monitoring_tests/rest_apis/test_local_monitoring_e2e_workflow.py

-    # Wait for the container to complete
-    wait_for_container_to_complete(container_id)
+    # Wait for the threshold for short term recommendations
+    time.sleep(180)


Please ensure the test behavior is consistent

chandrams · 2025-02-19T06:30:37Z

tests/scripts/local_monitoring_tests/rest_apis/test_namespace_reco_e2e_workflow.py

-    wait_for_container_to_complete(container_id3)
-
+    # Wait for the threshold for short term recommendations
+    time.sleep(180)


Please ensure the test behavior is consistent and short recommendations are generated after this

@shekhar316 - Any updates on this?

Signed-off-by: Shekhar Saxena <[email protected]>

shekhar316 · 2025-03-04T07:26:38Z

Test Results

Local Monitoring e2e Test:
e2e_lm_report.html.pdf

~/redhat/codebase/testKruize/autotune/tests/scripts/local_monitoring_tests/rest_apis git:[newTests]
pytest -vv --html=tmp/report.html --cluster_type openshift test_local_monitoring_e2e_workflow.py
================================================================================== test session starts ==================================================================================
platform linux -- Python 3.12.4, pytest-8.0.0, pluggy-1.4.0 -- /usr/bin/python3
cachedir: .pytest_cache
metadata: {'Python': '3.12.4', 'Platform': 'Linux-6.10.3-200.fc40.x86_64-x86_64-with-glibc2.39', 'Packages': {'pytest': '8.0.0', 'pluggy': '1.4.0'}, 'Plugins': {'metadata': '3.1.0', 'html': '3.2.0'}, 'JAVA_HOME': '/opt/jdk-21.0.1+12'}
rootdir: /home/shesaxen/redhat/codebase/testKruize/autotune/tests/scripts/local_monitoring_tests
configfile: pytest.ini
plugins: metadata-3.1.0, html-3.2.0
collected 1 item                                                                                                                                                                        

test_local_monitoring_e2e_workflow.py::test_list_recommendations_multiple_exps_for_datasource_workloads PASSED                                                                    [100%]

--------------------- generated html file: file:///home/shesaxen/redhat/codebase/testKruize/autotune/tests/scripts/local_monitoring_tests/rest_apis/tmp/report.html ---------------------
============================================================================= 1 passed in 311.69s (0:05:11) =============================================================================

Namespace e2e Test:
e2e_ns_report.html.pdf

~/redhat/codebase/testKruize/autotune/tests/scripts/local_monitoring_tests/rest_apis git:[newTests]
pytest -vv --html=tmp/report.html --cluster_type openshift test_namespace_reco_e2e_workflow.py
================================================================================== test session starts ==================================================================================
platform linux -- Python 3.12.4, pytest-8.0.0, pluggy-1.4.0 -- /usr/bin/python3
cachedir: .pytest_cache
metadata: {'Python': '3.12.4', 'Platform': 'Linux-6.10.3-200.fc40.x86_64-x86_64-with-glibc2.39', 'Packages': {'pytest': '8.0.0', 'pluggy': '1.4.0'}, 'Plugins': {'metadata': '3.1.0', 'html': '3.2.0'}, 'JAVA_HOME': '/opt/jdk-21.0.1+12'}
rootdir: /home/shesaxen/redhat/codebase/testKruize/autotune/tests/scripts/local_monitoring_tests
configfile: pytest.ini
plugins: metadata-3.1.0, html-3.2.0
collected 1 item                                                                                                                                                                        

test_namespace_reco_e2e_workflow.py::test_list_recommendations_namespace_exps PASSED                                                                                              [100%]

=================================================================================== warnings summary ====================================================================================
rest_apis/test_namespace_reco_e2e_workflow.py::test_list_recommendations_namespace_exps
  /usr/lib/python3.12/site-packages/urllib3/connectionpool.py:1063: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.shekhar.lab.psi.pnq2.redhat.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
--------------------- generated html file: file:///home/shesaxen/redhat/codebase/testKruize/autotune/tests/scripts/local_monitoring_tests/rest_apis/tmp/report.html ---------------------
======================================================================= 1 passed, 1 warning in 317.60s (0:05:17) ========================================================================

e2e_ns_report.html.pdf

tests/scripts/helpers/utils.py

Signed-off-by: Shekhar Saxena <[email protected]>

shreyabiradar07

LGTM

chandrams

LGTM

dinogun · 2025-03-11T14:17:32Z

@shekhar316 I see a couple of sleep(300), are we planning to use this in PR checks?

dinogun · 2025-03-17T09:52:16Z

Adding this to PR check will happen in a separate PR. This is ready for merge

shekhar316 force-pushed the newTests branch from 5569f52 to fc55191 Compare February 5, 2025 05:43

chandrams assigned chandrams and shekhar316 and unassigned chandrams Feb 18, 2025

chandrams added this to the Kruize 0.5 Release milestone Feb 18, 2025

shekhar316 added 3 commits February 18, 2025 15:04

updating namespace e2e

17ee6e5

Signed-off-by: Shekhar Saxena <[email protected]>

updating test for e2e lm

7169309

Signed-off-by: Shekhar Saxena <[email protected]>

updating sleep time

69eb719

Signed-off-by: Shekhar Saxena <[email protected]>

shekhar316 force-pushed the newTests branch from fc55191 to 69eb719 Compare February 18, 2025 09:37

updating sleep time

845b7cc

Signed-off-by: Shekhar Saxena <[email protected]>

shekhar316 marked this pull request as ready for review February 18, 2025 09:41

shekhar316 requested review from shreyabiradar07 and chandrams February 18, 2025 09:42

shekhar316 changed the title ~~Updating Test Benchmark to Sysbench [wip]~~ Updating Test Benchmark to Sysbench Feb 18, 2025

adding rm=false in del exp

c99abdb

Signed-off-by: Shekhar Saxena <[email protected]>

updating measurement duration

37a837f

Signed-off-by: Shekhar Saxena <[email protected]>

chandrams reviewed Feb 19, 2025

View reviewed changes

updating measurement duration and sleep time

71fc960

Signed-off-by: Shekhar Saxena <[email protected]>

shreyabiradar07 requested changes Mar 5, 2025

View reviewed changes

tests/scripts/helpers/utils.py Outdated Show resolved Hide resolved

updating print statement

08a37c4

Signed-off-by: Shekhar Saxena <[email protected]>

shekhar316 requested a review from shreyabiradar07 March 5, 2025 13:24

shreyabiradar07 approved these changes Mar 6, 2025

View reviewed changes

chandrams approved these changes Mar 6, 2025

View reviewed changes

dinogun merged commit 96bc66e into kruize:mvp_demo Mar 17, 2025
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updating Test Benchmark to Sysbench #1476

Updating Test Benchmark to Sysbench #1476

shekhar316 commented Jan 24, 2025

chandrams commented Feb 18, 2025

shreyabiradar07 commented Feb 18, 2025

shekhar316 commented Feb 18, 2025

shreyabiradar07 commented Feb 18, 2025

chandrams Feb 19, 2025

chandrams Feb 19, 2025

chandrams Mar 3, 2025

shekhar316 commented Mar 4, 2025

shreyabiradar07 left a comment

chandrams left a comment

dinogun commented Mar 11, 2025

dinogun commented Mar 17, 2025

Updating Test Benchmark to Sysbench #1476

Updating Test Benchmark to Sysbench #1476

Conversation

shekhar316 commented Jan 24, 2025

Description

How has this been tested?

Checklist 🎯

Additional information

chandrams commented Feb 18, 2025

shreyabiradar07 commented Feb 18, 2025

shekhar316 commented Feb 18, 2025

shreyabiradar07 commented Feb 18, 2025

chandrams Feb 19, 2025

Choose a reason for hiding this comment

chandrams Feb 19, 2025

Choose a reason for hiding this comment

chandrams Mar 3, 2025

Choose a reason for hiding this comment

shekhar316 commented Mar 4, 2025

Test Results

shreyabiradar07 left a comment

Choose a reason for hiding this comment

chandrams left a comment

Choose a reason for hiding this comment

dinogun commented Mar 11, 2025

dinogun commented Mar 17, 2025