api,server: allow updating hypervisor capabilities with hypervisor and version by shwstppr · Pull Request #8475 · apache/cloudstack

shwstppr · 2024-01-09T09:18:31Z

Description

hypervisor and hypervisorversion parameters are added to the updateHypervisorCapabilities API.
This would allow updating capabilities by specifying the hypervisor type and its version. This would also allow creating new hypervisor capabilities entries in the database when they doesn't exist for a given hypervisor version.

Related to #8412

New parameters as their description suggests are not meant to change the value of hypervisor and hypervisorversion of an existing capability. Rather they will be used to find the capabilities entry in the database and update other params such as maxdatavolumeslimit, maxguestslimit, maxhostspercluster, securitygroupenabled, storagemotionenabled, vmsnapshotenabled.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

(localcloud) 🐱 > list hypervisorcapabilities hypervisor=VMware filter=hypervisorversion,maxdatavolumeslimit
{
  "count": 21,
  "hypervisorCapabilities": [
    {
      "hypervisorversion": "default",
      "maxdatavolumeslimit": 13
    },
    {
      "hypervisorversion": "4.0",
      "maxdatavolumeslimit": 13
    },
    {
      "hypervisorversion": "4.1",
      "maxdatavolumeslimit": 13
    },
    {
      "hypervisorversion": "5.0",
      "maxdatavolumeslimit": 13
    },
    {
      "hypervisorversion": "5.1",
      "maxdatavolumeslimit": 13
    },
    {
      "hypervisorversion": "5.5",
      "maxdatavolumeslimit": 13
    },
    {
      "hypervisorversion": "6.0",
      "maxdatavolumeslimit": 59
    },
    {
      "hypervisorversion": "6.5",
      "maxdatavolumeslimit": 59
    },
    {
      "hypervisorversion": "6.7",
      "maxdatavolumeslimit": 59
    },
    {
      "hypervisorversion": "6.7.1",
      "maxdatavolumeslimit": 59
    },
    {
      "hypervisorversion": "6.7.2",
      "maxdatavolumeslimit": 59
    },
    {
      "hypervisorversion": "6.7.3",
      "maxdatavolumeslimit": 59
    },
    {
      "hypervisorversion": "7.0",
      "maxdatavolumeslimit": 59
    },
    {
      "hypervisorversion": "7.0.1.0",
      "maxdatavolumeslimit": 59
    },
    {
      "hypervisorversion": "7.0.2.0",
      "maxdatavolumeslimit": 59
    },
    {
      "hypervisorversion": "7.0.3.0",
      "maxdatavolumeslimit": 59
    },
    {
      "hypervisorversion": "8.0",
      "maxdatavolumeslimit": 59
    },
    {
      "hypervisorversion": "8.0.0.1",
      "maxdatavolumeslimit": 59
    },
    {
      "hypervisorversion": "8.0.1",
      "maxdatavolumeslimit": 59
    },
    {
      "hypervisorversion": "8.0.0.2",
      "maxdatavolumeslimit": 59
    },
    {
      "hypervisorversion": "8.0.0.3",
      "maxdatavolumeslimit": 59
    }
  ]
}

Update a hypervisorcapability by using hypervisor and hypervisorversion

(localcloud) 🐱 > update hypervisorcapabilities hypervisor=VMware hypervisorversion="8.0.0.3" maxdatavolumeslimit=100
{
  "hypervisorcapabilities": {
    "hypervisor": "VMware",
    "hypervisorversion": "8.0.0.3",
    "id": "761b1e90-22af-11ef-b7e0-a02942fcdd70",
    "maxdatavolumeslimit": 100,
    "maxguestslimit": 1024,
    "maxhostspercluster": 64,
    "securitygroupenabled": false,
    "storagemotionenabled": true,
    "vmsnapshotenabled": true
  }
}

Add capabilities for a patch version which does not exist (specifying one capability, other will be copied from the parent version)

(localcloud) 🐱 > update hypervisorcapabilities hypervisor=VMware hypervisorversion="8.0.0.4" maxdatavolumeslimit=64
{
  "hypervisorcapabilities": {
    "hypervisor": "VMware",
    "hypervisorversion": "8.0.0.4",
    "id": "9cca3a28-da41-4a9b-9dbc-86db434ebbf8",
    "maxdatavolumeslimit": 64,
    "maxguestslimit": 1024,
    "maxhostspercluster": 64,
    "securitygroupenabled": false,
    "storagemotionenabled": true,
    "vmsnapshotenabled": true
  }
}
(localcloud) 🐱 > list hypervisorcapabilities id=9cca3a28-da41-4a9b-9dbc-86db434ebbf8
{
  "count": 1,
  "hypervisorCapabilities": [
    {
      "hypervisor": "VMware",
      "hypervisorversion": "8.0.0.4",
      "id": "9cca3a28-da41-4a9b-9dbc-86db434ebbf8",
      "maxdatavolumeslimit": 64,
      "maxguestslimit": 1024,
      "maxhostspercluster": 64,
      "securitygroupenabled": false,
      "storagemotionenabled": true,
      "vmsnapshotenabled": true
    }
  ]
}

How did you try to break this feature and the system with this change?

codecov · 2024-01-09T09:27:03Z

Codecov Report

Attention: Patch coverage is 0% with 49 lines in your changes missing coverage. Please review.

Project coverage is 14.95%. Comparing base (ca59720) to head (ff9a5f5).
Report is 69 commits behind head on 4.19.

Files	Patch %	Lines
...in/java/com/cloud/server/ManagementServerImpl.java	0.00%	32 Missing ⚠️
...com/cloud/hypervisor/HypervisorCapabilitiesVO.java	0.00%	11 Missing ⚠️
.../admin/config/UpdateHypervisorCapabilitiesCmd.java	0.00%	6 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##              4.19    #8475       +/-   ##
============================================
+ Coverage     4.30%   14.95%   +10.65%     
- Complexity       0    11016    +11016     
============================================
  Files          363     5382     +5019     
  Lines        29312   470175   +440863     
  Branches      5118    60220    +55102     
============================================
+ Hits          1261    70321    +69060     
- Misses       27908   392065   +364157     
- Partials       143     7789     +7646

Flag	Coverage Δ
uitests	`4.28% <ø> (-0.02%)`	⬇️
unittests	`15.66% <0.00%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

server/src/main/java/com/cloud/server/ManagementServerImpl.java

sureshanaparti · 2024-03-12T09:52:11Z

Hi @shwstppr Is this PR targeted for 4.19.1? If so, please rebase and update target branch.

version hypervisor and hypervisorversion parameter added to the updateHypervisorCapabilities API. Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr · 2024-04-18T11:59:52Z

@sureshanaparti done

@blueorangutan package

blueorangutan · 2024-04-18T12:02:03Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-04-18T13:13:02Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9311

shwstppr · 2024-06-12T07:30:04Z

@blueorangutan package

blueorangutan · 2024-06-12T07:32:04Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

sureshanaparti

clgtm

blueorangutan · 2024-06-12T08:32:40Z

Packaging result [SF]: ✔️ el7 ✖️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 9882

kiranchavala · 2024-06-12T12:19:39Z

@blueorangutan package

blueorangutan · 2024-06-12T12:20:04Z

@kiranchavala a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

server/src/main/java/com/cloud/server/ManagementServerImpl.java

blueorangutan · 2024-06-12T13:23:44Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9900

JoaoJandre

CLGTM, did not test it

DaanHoogland

just one remark about the parameter description, looks good otherwise

...ain/java/org/apache/cloudstack/api/command/admin/config/UpdateHypervisorCapabilitiesCmd.java

Co-authored-by: dahn <daan.hoogland@gmail.com>

DaanHoogland

tlgtm

...ain/java/org/apache/cloudstack/api/command/admin/config/UpdateHypervisorCapabilitiesCmd.java

shwstppr · 2024-06-13T12:39:41Z

@kiranchavala added some tests I did with the API in the PR description

server/src/main/java/com/cloud/server/ManagementServerImpl.java

Co-authored-by: Henrique Sato <henriquesato2003@gmail.com>

DaanHoogland · 2024-06-17T08:39:40Z

@blueorangutan package

blueorangutan · 2024-06-17T08:42:06Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-06-17T09:47:28Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9980

DaanHoogland · 2024-06-17T10:04:57Z

@blueorangutan test

blueorangutan · 2024-06-17T10:06:03Z

@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2024-06-17T10:25:22Z

[SF] Trillian Build Failed (tid-10471)

vishesh92 · 2024-06-21T08:06:40Z

@blueorangutan test

blueorangutan · 2024-06-21T08:08:04Z

@vishesh92 a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2024-06-21T20:15:57Z

[SF] Trillian test result (tid-10544)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 42176 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8475-t10544-kvm-centos7.zip
Smoke tests completed. 130 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_02_trigger_shutdown	`Failure`	341.67	test_safe_shutdown.py

vishesh92 · 2024-06-21T20:33:33Z

server/src/main/java/com/cloud/server/ManagementServerImpl.java

+        }

-        hpvCapabilities = _hypervisorCapabilitiesDao.createForUpdate(id);
+        hpvCapabilities = _hypervisorCapabilitiesDao.createForUpdate(hpvCapabilities.getId());


Need to set id = hpvCapabilities.getId().
id is being used in L5156 to update the hypervisor capabilities. If id is not passed in the request, update is not getting triggered if the value already exists because id is null. This is also resulting in not getting any output.

(localcloud) __ > update hypervisorcapabilities hypervisor=VMware hypervisorversion="8.0.0.15" maxdatavolumeslimit=640 __ Error: (HTTP 530, error code 9999) Failed to update hypervisor capabilities

Executing this for the first time creates an entry which is a duplicate of the parent hypervisor. And subsequent command just fails.

server/src/main/java/com/cloud/server/ManagementServerImpl.java

Co-authored-by: Vishesh <vishesh92@gmail.com>

shwstppr · 2024-06-24T07:02:37Z

@blueorangutan package

blueorangutan · 2024-06-24T07:04:03Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

server/src/main/java/com/cloud/server/ManagementServerImpl.java

vishesh92 · 2024-06-24T07:13:18Z

@blueorangutan package

blueorangutan · 2024-06-24T07:14:03Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

vishesh92

code looks good to me.
Tested locally with Simulator. API is working as expected.

blueorangutan · 2024-06-24T08:19:28Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10084

…d version (apache#8475) * api,server: allow updating hypervisor capabilities with hypervisor and version hypervisor and hypervisorversion parameter added to the updateHypervisorCapabilities API. * param description * Update server/src/main/java/com/cloud/server/ManagementServerImpl.java Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> Co-authored-by: dahn <daan.hoogland@gmail.com> Co-authored-by: Henrique Sato <henriquesato2003@gmail.com> Co-authored-by: Vishesh <vishesh92@gmail.com>

shwstppr added this to the 4.19.1.0 milestone Jan 9, 2024

boring-cyborg bot added component:api component:management-server labels Jan 9, 2024

JoaoJandre reviewed Feb 8, 2024

View reviewed changes

server/src/main/java/com/cloud/server/ManagementServerImpl.java Outdated Show resolved Hide resolved

api,server: allow updating hypervisor capabilities with hypervisor and

308819c

version hypervisor and hypervisorversion parameter added to the updateHypervisorCapabilities API. Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

shwstppr changed the base branch from main to 4.19 April 18, 2024 11:57

shwstppr force-pushed the add-updhycapapi-param branch from 9503ce3 to 308819c Compare April 18, 2024 11:59

Merge branch '4.19' into add-updhycapapi-param

1e39514

shwstppr marked this pull request as ready for review June 12, 2024 07:29

sureshanaparti approved these changes Jun 12, 2024

View reviewed changes

sureshanaparti requested a review from DaanHoogland June 12, 2024 07:37

sureshanaparti added the status:needs-testing label Jun 12, 2024

kiranchavala self-assigned this Jun 12, 2024

JoaoJandre reviewed Jun 12, 2024

View reviewed changes

server/src/main/java/com/cloud/server/ManagementServerImpl.java Show resolved Hide resolved

JoaoJandre approved these changes Jun 12, 2024

View reviewed changes

DaanHoogland reviewed Jun 13, 2024

View reviewed changes

...ain/java/org/apache/cloudstack/api/command/admin/config/UpdateHypervisorCapabilitiesCmd.java Outdated Show resolved Hide resolved

param description

96c662d

Co-authored-by: dahn <daan.hoogland@gmail.com>

DaanHoogland approved these changes Jun 13, 2024

View reviewed changes

shwstppr commented Jun 13, 2024

View reviewed changes

...ain/java/org/apache/cloudstack/api/command/admin/config/UpdateHypervisorCapabilitiesCmd.java Outdated Show resolved Hide resolved

param description

98cbf5f

hsato03 reviewed Jun 14, 2024

View reviewed changes

server/src/main/java/com/cloud/server/ManagementServerImpl.java Outdated Show resolved Hide resolved

Update server/src/main/java/com/cloud/server/ManagementServerImpl.java

1a85d32

Co-authored-by: Henrique Sato <henriquesato2003@gmail.com>

vishesh92 reviewed Jun 21, 2024

View reviewed changes

vishesh92 reviewed Jun 24, 2024

View reviewed changes

server/src/main/java/com/cloud/server/ManagementServerImpl.java Outdated Show resolved Hide resolved

Update server/src/main/java/com/cloud/server/ManagementServerImpl.java

a381322

Co-authored-by: Vishesh <vishesh92@gmail.com>

vishesh92 reviewed Jun 24, 2024

View reviewed changes

server/src/main/java/com/cloud/server/ManagementServerImpl.java Outdated Show resolved Hide resolved

Update server/src/main/java/com/cloud/server/ManagementServerImpl.java

ff9a5f5

vishesh92 approved these changes Jun 24, 2024

View reviewed changes

sureshanaparti removed the status:needs-testing label Jun 24, 2024

sureshanaparti merged commit de266a4 into apache:4.19 Jun 24, 2024

sureshanaparti deleted the add-updhycapapi-param branch June 24, 2024 09:11

Conversation

shwstppr commented Jan 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

codecov bot commented Jan 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

sureshanaparti commented Mar 12, 2024

Uh oh!

shwstppr commented Apr 18, 2024

Uh oh!

blueorangutan commented Apr 18, 2024

Uh oh!

blueorangutan commented Apr 18, 2024

Uh oh!

shwstppr commented Jun 12, 2024

Uh oh!

blueorangutan commented Jun 12, 2024

Uh oh!

sureshanaparti left a comment

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Jun 12, 2024

Uh oh!

kiranchavala commented Jun 12, 2024

Uh oh!

blueorangutan commented Jun 12, 2024

Uh oh!

Uh oh!

blueorangutan commented Jun 12, 2024

Uh oh!

JoaoJandre left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

shwstppr commented Jun 13, 2024

Uh oh!

Uh oh!

DaanHoogland commented Jun 17, 2024

Uh oh!

blueorangutan commented Jun 17, 2024

Uh oh!

blueorangutan commented Jun 17, 2024

Uh oh!

DaanHoogland commented Jun 17, 2024

Uh oh!

blueorangutan commented Jun 17, 2024

Uh oh!

blueorangutan commented Jun 17, 2024

Uh oh!

vishesh92 commented Jun 21, 2024

Uh oh!

blueorangutan commented Jun 21, 2024

Uh oh!

blueorangutan commented Jun 21, 2024

Uh oh!

vishesh92 Jun 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

shwstppr commented Jun 24, 2024

Uh oh!

shwstppr commented Jan 9, 2024 •

edited

Loading

codecov bot commented Jan 9, 2024 •

edited

Loading

vishesh92 Jun 21, 2024 •

edited

Loading