[Quota] Update pre-populated Quota tariffs' type by BryanMLima · Pull Request #8591 · apache/cloudstack

BryanMLima · 2024-02-01T13:12:30Z

Description

The PR #7152 changed the unit of the Quota types VM_DISK_IO_READ, VM_DISK_IO_WRITE, VM_DISK_BYTES_READ and VM_DISK_BYTES_WRITE. However, the pre-populated tariffs were not considered, and their unit type is still with the old value. The unit was only displayed incorrectly, as the calculation utilizes the correct unit. Their unit types were updated to the correct ones.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI

Feature/Enhancement Scale or Bug Severity

Bug Severity

How Has This Been Tested?

Update script

2024-02-05 14:42:03,890 INFO  [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) DB version = 4.19.0.0 Code Version = 4.19.1.0
2024-02-05 14:42:03,894 INFO  [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Database upgrade must be performed from 4.19.0.0 to 4.19.1.0
2024-02-05 14:42:03,949 DEBUG [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Running upgrade Upgrade41900to41910 to upgrade from 4.19.0.0-4.19.1.0 to 4.19.1.0
2024-02-05 14:42:03,957 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Licensed to the Apache Software Foundation (ASF) under one
2024-02-05 14:42:03,958 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- or more contributor license agreements.  See the NOTICE file
2024-02-05 14:42:03,958 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- distributed with this work for additional information
2024-02-05 14:42:03,958 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- regarding copyright ownership.  The ASF licenses this file
2024-02-05 14:42:03,959 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- to you under the Apache License, Version 2.0 (the
2024-02-05 14:42:03,959 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- "License"); you may not use this file except in compliance
2024-02-05 14:42:03,959 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- with the License.  You may obtain a copy of the License at
2024-02-05 14:42:03,959 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --
2024-02-05 14:42:03,959 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --   http://www.apache.org/licenses/LICENSE-2.0
2024-02-05 14:42:03,960 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --
2024-02-05 14:42:03,960 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Unless required by applicable law or agreed to in writing,
2024-02-05 14:42:03,960 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- software distributed under the License is distributed on an
2024-02-05 14:42:03,960 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
2024-02-05 14:42:03,963 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- KIND, either express or implied.  See the License for the
2024-02-05 14:42:03,963 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- specific language governing permissions and limitations
2024-02-05 14:42:03,963 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- under the License.
2024-02-05 14:42:03,964 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --;
2024-02-05 14:42:03,964 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Schema upgrade from 4.19.0.0 to 4.19.1.0
2024-02-05 14:42:03,965 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --;
2024-02-05 14:42:03,965 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Updates the populated Quota tariff's types VM_DISK_BYTES_READ, VM_DISK_BYTES_WRITE, VM_DISK_IO_READ and VM_DISK_IO_WRITE to the correct unit.
2024-02-05 14:42:03,966 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) UPDATE cloud_usage.quota_tariff SET usage_unit = 'Bytes', updated_on = NOW() WHERE effective_on = '2010-05-04 00:00:00' AND name IN ('VM_DISK_BYTES_READ', 'VM_DISK_BYTES_WRITE') 
2024-02-05 14:42:03,976 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) UPDATE cloud_usage.quota_tariff SET usage_unit = 'IOPS', updated_on = NOW() WHERE effective_on = '2010-05-04 00:00:00' AND name IN ('VM_DISK_IO_READ', 'VM_DISK_IO_WRITE') 
2024-02-05 14:42:04,091 INFO  [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Cleanup upgrade Upgrade41900to41910 to upgrade from 4.19.0.0-4.19.1.0 to 4.19.1.0
2024-02-05 14:42:04,103 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Licensed to the Apache Software Foundation (ASF) under one
2024-02-05 14:42:04,103 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- or more contributor license agreements.  See the NOTICE file
2024-02-05 14:42:04,103 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- distributed with this work for additional information
2024-02-05 14:42:04,104 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- regarding copyright ownership.  The ASF licenses this file
2024-02-05 14:42:04,104 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- to you under the Apache License, Version 2.0 (the
2024-02-05 14:42:04,104 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- "License"); you may not use this file except in compliance
2024-02-05 14:42:04,105 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- with the License.  You may obtain a copy of the License at
2024-02-05 14:42:04,105 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --
2024-02-05 14:42:04,105 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --   http://www.apache.org/licenses/LICENSE-2.0
2024-02-05 14:42:04,105 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --
2024-02-05 14:42:04,105 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Unless required by applicable law or agreed to in writing,
2024-02-05 14:42:04,106 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- software distributed under the License is distributed on an
2024-02-05 14:42:04,106 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
2024-02-05 14:42:04,106 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- KIND, either express or implied.  See the License for the
2024-02-05 14:42:04,106 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- specific language governing permissions and limitations
2024-02-05 14:42:04,106 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- under the License.
2024-02-05 14:42:04,107 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --;
2024-02-05 14:42:04,107 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) -- Schema upgrade cleanup from 4.19.0.0 to 4.19.1.0
2024-02-05 14:42:04,108 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --;
2024-02-05 14:42:04,115 DEBUG [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Cleanup script Upgrade41900to41910 is executed successfully
2024-02-05 14:42:04,225 DEBUG [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Upgrade completed for version 4.19.1.0

After the update script, the usage units were updated accordingly.

|id|usage_type|usage_name|usage_unit|usage_discriminator|currency_value|effective_on|updated_on|updated_by|uuid|name|description|activation_rule|removed|end_date|
|--|----------|----------|----------|-------------------|--------------|------------|----------|----------|----|----|-----------|---------------|-------|--------|
|18|21|VM_DISK_IO_READ|IOPS|1|0.00000|2010-05-04 00:00:00|2024-01-31 20:51:49|1|9f3f2dc7-771e-11ee-8e59-5254003754dc|VM_DISK_IO_READ|||||
|19|22|VM_DISK_IO_WRITE|IOPS|1|0.00000|2010-05-04 00:00:00|2024-01-31 20:51:49|1|9f3f2ead-771e-11ee-8e59-5254003754dc|VM_DISK_IO_WRITE|||||
|20|23|VM_DISK_BYTES_READ|Bytes|1|0.00000|2010-05-04 00:00:00|2024-01-31 20:51:49|1|9f3f2f84-771e-11ee-8e59-5254003754dc|VM_DISK_BYTES_READ|||||
|21|24|VM_DISK_BYTES_WRITE|Bytes|1|0.00000|2010-05-04 00:00:00|2024-01-31 20:51:49|1|9f3f304f-771e-11ee-8e59-5254003754dc|VM_DISK_BYTES_WRITE|||||

How did you try to break this feature and the system with this change?

BryanMLima · 2024-02-01T13:14:16Z

@JoaoJandre, do you think this can go to the 4.18.2.0 release, as it is a small fix?

BryanMLima · 2024-02-01T13:14:41Z

@blueorangutan package

blueorangutan · 2024-02-01T13:16:03Z

@BryanMLima a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

codecov · 2024-02-01T14:03:17Z

Codecov Report

Attention: Patch coverage is 18.75000% with 13 lines in your changes are missing coverage. Please review.

Project coverage is 31.49%. Comparing base (c795547) to head (3658e5b).
Report is 47 commits behind head on 4.19.

Files	Patch %	Lines
...ava/com/cloud/upgrade/dao/Upgrade41900to41910.java	13.33%	13 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##               4.19    #8591       +/-   ##
=============================================
+ Coverage     20.99%   31.49%   +10.50%     
- Complexity    19683    35081    +15398     
=============================================
  Files          4849     5354      +505     
  Lines        324441   385758    +61317     
  Branches      45573    58895    +13322     
=============================================
+ Hits          68121   121507    +53386     
- Misses       246563   248242     +1679     
- Partials       9757    16009     +6252

Flag	Coverage Δ
simulator-marvin-tests	`25.46% <12.50%> (+4.47%)`	⬆️
uitests	`4.32% <ø> (?)`
unit-tests	`16.60% <18.75%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

blueorangutan · 2024-02-01T14:19:56Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 8489

JoaoJandre · 2024-02-02T16:48:24Z

@DaanHoogland, as 4.18.2 is going to be released after 4.19.0, and therefore users might upgrade to 4.19 without going through the 4.18.2 schema upgrade, how should we go about making sure that this update is executed for users in this situation? When merging 4.18 forward I should also add the same update to the 4.20 schema upgrade?

weizhouapache · 2024-02-02T17:17:44Z

@DaanHoogland, as 4.18.2 is going to be released after 4.19.0, and therefore users might upgrade to 4.19 without going through the 4.18.2 schema upgrade, how should we go about making sure that this update is executed for users in this situation? When merging 4.18 forward I should also add the same update to the 4.20 schema upgrade?

@JoaoJandre
You need to add this to the upgrade schema of both 4.18.1-4.18.2 and 4.19.0-4.19.1

Considering this is a minor fix and this is the only database change between 4.18.1 and 4.18.2, can we retarget to 4.19.1 and add it to the upgrade schema of 4.19.0-4.19.1 only ?

BryanMLima · 2024-02-05T14:50:19Z

@DaanHoogland, as 4.18.2 is going to be released after 4.19.0, and therefore users might upgrade to 4.19 without going through the 4.18.2 schema upgrade, how should we go about making sure that this update is executed for users in this situation? When merging 4.18 forward I should also add the same update to the 4.20 schema upgrade?

@JoaoJandre You need to add this to the upgrade schema of both 4.18.1-4.18.2 and 4.19.0-4.19.1

Considering this is a minor fix and this is the only database change between 4.18.1 and 4.18.2, can we retarget to 4.19.1 and add it to the upgrade schema of 4.19.0-4.19.1 only ?

Retargeted to 4.19.1, as this is a minor fix.

BryanMLima · 2024-02-05T14:50:27Z

@blueorangutan package

blueorangutan · 2024-02-05T14:52:04Z

@BryanMLima a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-02-05T15:57:31Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 8533

GutoVeronezi

CLGTM

DaanHoogland

clgtm

DaanHoogland · 2024-02-16T13:02:20Z

@blueorangutan test

blueorangutan · 2024-02-16T13:04:06Z

@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2024-02-17T05:16:39Z

[SF] Trillian test result (tid-9257)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 56799 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8591-t9257-kvm-centos7.zip
Smoke tests completed. 120 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_list_volumes_metrics_history	`Error`	3607.52	test_metrics_api.py
test_02_trigger_shutdown	`Failure`	336.64	test_safe_shutdown.py

DaanHoogland · 2024-02-20T10:25:35Z

some CI failures, reopening.

DaanHoogland · 2024-02-20T13:28:33Z

still doesn't pass :( though the error is in a library download. Applying insanity

DaanHoogland · 2024-03-04T09:25:51Z

@blueorangutan package

blueorangutan · 2024-03-04T09:28:03Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-03-04T10:28:18Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 8844

DaanHoogland · 2024-03-04T11:37:54Z

@blueorangutan test

blueorangutan · 2024-03-04T11:40:03Z

@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2024-03-05T00:23:28Z

[SF] Trillian test result (tid-9380)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 44248 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8591-t9380-kvm-centos7.zip
Smoke tests completed. 129 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

Update pre-populated quota tariffs' unit types

3658e5b

BryanMLima force-pushed the update-quota-types-unit branch from 1e25180 to 3658e5b Compare February 5, 2024 14:46

boring-cyborg bot added component:agent component:api component:build component:integration labels Feb 5, 2024

BryanMLima changed the base branch from 4.18 to 4.19 February 5, 2024 14:46

BryanMLima added component:quota and removed component:api component:agent component:build component:integration labels Feb 5, 2024

BryanMLima added this to the 4.19.1.0 milestone Feb 5, 2024

GutoVeronezi approved these changes Feb 14, 2024

View reviewed changes

yadvr approved these changes Feb 16, 2024

View reviewed changes

yadvr requested a review from DaanHoogland February 16, 2024 05:55

DaanHoogland approved these changes Feb 16, 2024

View reviewed changes

DaanHoogland closed this Feb 20, 2024

DaanHoogland reopened this Feb 20, 2024

DaanHoogland mentioned this pull request Feb 20, 2024

field enlarged and db upgrade #8675

Merged

13 tasks

DaanHoogland closed this Feb 20, 2024

DaanHoogland reopened this Feb 20, 2024

BryanMLima closed this Feb 29, 2024

BryanMLima reopened this Feb 29, 2024

DaanHoogland merged commit b82ea3d into apache:4.19 Mar 6, 2024

dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Mar 21, 2024

Update pre-populated quota tariffs' unit types (apache#8591)

47669b2

Conversation

BryanMLima commented Feb 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Bug Severity

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

BryanMLima commented Feb 1, 2024

Uh oh!

BryanMLima commented Feb 1, 2024

Uh oh!

blueorangutan commented Feb 1, 2024

Uh oh!

codecov bot commented Feb 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

blueorangutan commented Feb 1, 2024

Uh oh!

JoaoJandre commented Feb 2, 2024

Uh oh!

weizhouapache commented Feb 2, 2024

Uh oh!

BryanMLima commented Feb 5, 2024

Uh oh!

BryanMLima commented Feb 5, 2024

Uh oh!

blueorangutan commented Feb 5, 2024

Uh oh!

blueorangutan commented Feb 5, 2024

Uh oh!

GutoVeronezi left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland commented Feb 16, 2024

Uh oh!

blueorangutan commented Feb 16, 2024

Uh oh!

blueorangutan commented Feb 17, 2024

Uh oh!

DaanHoogland commented Feb 20, 2024

Uh oh!

DaanHoogland commented Feb 20, 2024

Uh oh!

DaanHoogland commented Mar 4, 2024

Uh oh!

blueorangutan commented Mar 4, 2024

Uh oh!

blueorangutan commented Mar 4, 2024

Uh oh!

DaanHoogland commented Mar 4, 2024

Uh oh!

blueorangutan commented Mar 4, 2024

Uh oh!

blueorangutan commented Mar 5, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

BryanMLima commented Feb 1, 2024 •

edited

Loading

codecov bot commented Feb 1, 2024 •

edited

Loading