Skip to content

Add available memory threshold to preload event#710

Merged
marmarek merged 1 commit intoQubesOS:mainfrom
ben-grande:preload-mem-treshold
Aug 6, 2025
Merged

Add available memory threshold to preload event#710
marmarek merged 1 commit intoQubesOS:mainfrom
ben-grande:preload-mem-treshold

Conversation

@ben-grande
Copy link
Contributor

@ben-grande
Copy link
Contributor Author

I did consider a dynamic management depending on the threshold but I thought that would be too much. The management would be done on domain-feature-delete and domain-feature-set to dynamically add or reduce depending on the threshold. It will not be easy because it involves checking for every dvm template that has the preload feature and set some heuristics of what to do in each case... therefore, I went with the simpler option.

The openqa test low_mem is working.

@codecov
Copy link

codecov bot commented Jul 30, 2025

Codecov Report

❌ Patch coverage is 84.61538% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.49%. Comparing base (d95888b) to head (6c3c634).
⚠️ Report is 8 commits behind head on main.

Files with missing lines Patch % Lines
qubes/vm/mix/dvmtemplate.py 75.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #710      +/-   ##
==========================================
- Coverage   70.53%   70.49%   -0.04%     
==========================================
  Files          61       61              
  Lines       13546    13557      +11     
==========================================
+ Hits         9554     9557       +3     
- Misses       3992     4000       +8     
Flag Coverage Δ
unittests 70.49% <84.61%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ben-grande ben-grande force-pushed the preload-mem-treshold branch from 344f9d4 to 6c3c634 Compare July 30, 2025 15:28
ben-grande added a commit to ben-grande/qubes-core-admin-client that referenced this pull request Jul 31, 2025
@qubesos-bot
Copy link

qubesos-bot commented Aug 4, 2025

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025080604-4.3&flavor=pull-requests

Test run included the following:

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025061004-4.3&flavor=update

  • system_tests_extra

    • TC_00_QVCTest_whonix-gateway-17: test_010_screenshare (failure)
      ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 0 == 0
  • system_tests_qwt_win10_seamless@hw13

    • windows_clipboard_and_filecopy: unnamed test (unknown)
    • windows_clipboard_and_filecopy: Failed (test died)
      # Test died: no candidate needle with tag(s) 'windows-Edge-address-...
  • system_tests_qwt_win11@hw13

    • windows_install: wait_serial (wait serial expected)
      # wait_serial expected: qr/qDqV_-\d+-/...

    • windows_install: Failed (test died + timed out)
      # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...

  • system_tests_dispvm_perf@hw7

Failed tests

9 failures
  • system_tests_extra

    • TC_00_QVCTest_whonix-gateway-17: test_010_screenshare (failure)
      ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 0 == 0

    • TC_00_QVCTest_whonix-workstation-17: test_010_screenshare (failure)
      AssertionError: 1 != 0 : Timeout waiting for /dev/video0 in test-in...

  • system_tests_kde_gui_interactive

    • gui_keyboard_layout: wait_serial (wait serial expected)
      # wait_serial expected: "echo -e '[Layout]\nLayoutList=us,de' | sud...

    • gui_keyboard_layout: Failed (test died)
      # Test died: command 'test "$(cd ~user;ls e1*)" = "$(qvm-run -p wor...

  • system_tests_qwt_win10_seamless@hw13

    • windows_clipboard_and_filecopy: unnamed test (unknown)
    • windows_clipboard_and_filecopy: Failed (test died)
      # Test died: no candidate needle with tag(s) 'windows-Edge-address-...
  • system_tests_qwt_win11@hw13

    • windows_install: wait_serial (wait serial expected)
      # wait_serial expected: qr/qDqV_-\d+-/...

    • windows_install: Failed (test died + timed out)
      # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...

  • system_tests_dispvm_perf@hw7

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/142375#dependencies

10 fixed

Unstable tests

Details

Performance Tests

Performance degradation:

9 performance degradations
  • debian-12-xfce_exec-data-simplex: 74.46 🔺 ( previous job: 65.51, degradation: 113.65%)
  • debian-12-xfce_exec-data-duplex-root: 84.31 🔺 ( previous job: 70.01, degradation: 120.43%)
  • dom0_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 13726.00 :small_red_triangle: ( previous job: 17102.00, degradation: 80.26%)
  • dom0_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 524.00 :small_red_triangle: ( previous job: 1091.00, degradation: 48.03%)
  • dom0_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 14.00 :small_red_triangle: ( previous job: 1840.00, degradation: 0.76%)
  • dom0_varlibqubes_seq1m_q8t1_read 3:read_bandwidth_kb: 61207.00 :small_red_triangle: ( previous job: 289182.00, degradation: 21.17%)
  • dom0_varlibqubes_seq1m_q8t1_write 3:write_bandwidth_kb: 78728.00 :small_red_triangle: ( previous job: 122848.00, degradation: 64.09%)
  • dom0_varlibqubes_rnd4k_q32t1_write 3:write_bandwidth_kb: 5543.00 :small_red_triangle: ( previous job: 8874.00, degradation: 62.46%)
  • dom0_varlibqubes_rnd4k_q1t1_write 3:write_bandwidth_kb: 3403.00 :small_red_triangle: ( previous job: 4420.00, degradation: 76.99%)

Remaining performance tests:

164 tests
  • debian-12-xfce_exec: 6.91 🟢 ( previous job: 8.63, improvement: 80.04%)
  • debian-12-xfce_exec-root: 27.96 🟢 ( previous job: 29.44, improvement: 94.98%)
  • debian-12-xfce_socket: 7.90 🟢 ( previous job: 8.50, improvement: 92.97%)
  • debian-12-xfce_socket-root: 8.38 🔺 ( previous job: 8.31, degradation: 100.81%)
  • debian-12-xfce_exec-data-duplex: 70.88 🟢 ( previous job: 73.55, improvement: 96.38%)
  • debian-12-xfce_socket-data-duplex: 157.66 🟢 ( previous job: 161.35, improvement: 97.71%)
  • fedora-42-xfce_exec: 9.04
  • fedora-42-xfce_exec-root: 60.22
  • fedora-42-xfce_socket: 8.59
  • fedora-42-xfce_socket-root: 7.61
  • fedora-42-xfce_exec-data-simplex: 66.57
  • fedora-42-xfce_exec-data-duplex: 69.19
  • fedora-42-xfce_exec-data-duplex-root: 95.26
  • fedora-42-xfce_socket-data-duplex: 140.03
  • whonix-gateway-17_exec: 7.84 🔺 ( previous job: 7.34, degradation: 106.77%)
  • whonix-gateway-17_exec-root: 41.95 🔺 ( previous job: 39.57, degradation: 105.99%)
  • whonix-gateway-17_socket: 7.68 🟢 ( previous job: 7.85, improvement: 97.78%)
  • whonix-gateway-17_socket-root: 8.51 🔺 ( previous job: 7.89, degradation: 107.84%)
  • whonix-gateway-17_exec-data-simplex: 75.55 🟢 ( previous job: 77.76, improvement: 97.16%)
  • whonix-gateway-17_exec-data-duplex: 78.31 🟢 ( previous job: 78.39, improvement: 99.90%)
  • whonix-gateway-17_exec-data-duplex-root: 89.55 🟢 ( previous job: 90.74, improvement: 98.69%)
  • whonix-gateway-17_socket-data-duplex: 167.16 🔺 ( previous job: 161.95, degradation: 103.21%)
  • whonix-workstation-17_exec: 8.25 🟢 ( previous job: 8.27, improvement: 99.70%)
  • whonix-workstation-17_exec-root: 53.08 🟢 ( previous job: 57.61, improvement: 92.14%)
  • whonix-workstation-17_socket: 7.96 🟢 ( previous job: 8.97, improvement: 88.73%)
  • whonix-workstation-17_socket-root: 8.38 🟢 ( previous job: 9.46, improvement: 88.62%)
  • whonix-workstation-17_exec-data-simplex: 75.94 🔺 ( previous job: 74.54, degradation: 101.88%)
  • whonix-workstation-17_exec-data-duplex: 76.30 🔺 ( previous job: 74.84, degradation: 101.95%)
  • whonix-workstation-17_exec-data-duplex-root: 87.69 🔺 ( previous job: 86.00, degradation: 101.97%)
  • whonix-workstation-17_socket-data-duplex: 161.39 🔺 ( previous job: 160.20, degradation: 100.74%)
  • dom0_root_seq1m_q8t1_read 3:read_bandwidth_kb: 462131.00 :green_circle: ( previous job: 289982.00, improvement: 159.37%)
  • dom0_root_seq1m_q8t1_write 3:write_bandwidth_kb: 204191.00 :green_circle: ( previous job: 101988.00, improvement: 200.21%)
  • dom0_root_seq1m_q1t1_read 3:read_bandwidth_kb: 107886.00 :green_circle: ( previous job: 14284.00, improvement: 755.29%)
  • dom0_root_seq1m_q1t1_write 3:write_bandwidth_kb: 49852.00 :green_circle: ( previous job: 32696.00, improvement: 152.47%)
  • dom0_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 11447.00 :green_circle: ( previous job: 11086.00, improvement: 103.26%)
  • dom0_varlibqubes_seq1m_q1t1_read 3:read_bandwidth_kb: 390385.00 :small_red_triangle: ( previous job: 433654.00, degradation: 90.02%)
  • dom0_varlibqubes_seq1m_q1t1_write 3:write_bandwidth_kb: 194580.00 :green_circle: ( previous job: 167872.00, improvement: 115.91%)
  • dom0_varlibqubes_rnd4k_q32t1_read 3:read_bandwidth_kb: 104720.00 :small_red_triangle: ( previous job: 108760.00, degradation: 96.29%)
  • dom0_varlibqubes_rnd4k_q1t1_read 3:read_bandwidth_kb: 8279.00 :green_circle: ( previous job: 6356.00, improvement: 130.25%)
  • fedora-42-xfce_root_seq1m_q8t1_read 3:read_bandwidth_kb: 322738.00
  • fedora-42-xfce_root_seq1m_q8t1_write 3:write_bandwidth_kb: 335436.00
  • fedora-42-xfce_root_seq1m_q1t1_read 3:read_bandwidth_kb: 302270.00
  • fedora-42-xfce_root_seq1m_q1t1_write 3:write_bandwidth_kb: 81141.00
  • fedora-42-xfce_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 85333.00
  • fedora-42-xfce_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 2998.00
  • fedora-42-xfce_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 7650.00
  • fedora-42-xfce_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 1025.00
  • fedora-42-xfce_private_seq1m_q8t1_read 3:read_bandwidth_kb: 361827.00
  • fedora-42-xfce_private_seq1m_q8t1_write 3:write_bandwidth_kb: 254323.00
  • fedora-42-xfce_private_seq1m_q1t1_read 3:read_bandwidth_kb: 331617.00
  • fedora-42-xfce_private_seq1m_q1t1_write 3:write_bandwidth_kb: 66249.00
  • fedora-42-xfce_private_rnd4k_q32t1_read 3:read_bandwidth_kb: 88727.00
  • fedora-42-xfce_private_rnd4k_q32t1_write 3:write_bandwidth_kb: 3624.00
  • fedora-42-xfce_private_rnd4k_q1t1_read 3:read_bandwidth_kb: 8568.00
  • fedora-42-xfce_private_rnd4k_q1t1_write 3:write_bandwidth_kb: 495.00
  • fedora-42-xfce_volatile_seq1m_q8t1_read 3:read_bandwidth_kb: 368179.00
  • fedora-42-xfce_volatile_seq1m_q8t1_write 3:write_bandwidth_kb: 106908.00
  • fedora-42-xfce_volatile_seq1m_q1t1_read 3:read_bandwidth_kb: 300107.00
  • fedora-42-xfce_volatile_seq1m_q1t1_write 3:write_bandwidth_kb: 72763.00
  • fedora-42-xfce_volatile_rnd4k_q32t1_read 3:read_bandwidth_kb: 84116.00
  • fedora-42-xfce_volatile_rnd4k_q32t1_write 3:write_bandwidth_kb: 2608.00
  • fedora-42-xfce_volatile_rnd4k_q1t1_read 3:read_bandwidth_kb: 8286.00
  • fedora-42-xfce_volatile_rnd4k_q1t1_write 3:write_bandwidth_kb: 812.00
  • debian-12-xfce_dispvm(avg:6.195): 99.12
  • debian-12-xfce_dispvm-gui(avg:7.118): 113.89
  • debian-12-xfce_dispvm-concurrent(avg:2.810): 44.96
  • debian-12-xfce_dispvm-gui-concurrent(avg:3.941): 63.05
  • debian-12-xfce_dispvm-dom0(avg:6.864): 109.83
  • debian-12-xfce_dispvm-dom0-gui(avg:7.858): 125.72
  • debian-12-xfce_dispvm-dom0-concurrent(avg:3.236): 51.77
  • debian-12-xfce_dispvm-dom0-gui-concurrent(avg:4.075): 65.20
  • debian-12-xfce_dispvm-preload(avg:3.439): 55.02
  • debian-12-xfce_dispvm-preload-gui(avg:4.570): 73.13
  • debian-12-xfce_dispvm-preload-concurrent(avg:2.576): 41.22
  • debian-12-xfce_dispvm-preload-gui-concurrent(avg:3.146): 50.33
  • debian-12-xfce_dispvm-preload-dom0(avg:3.539): 56.62
  • debian-12-xfce_dispvm-preload-dom0-gui(avg:5.176): 82.81
  • debian-12-xfce_dispvm-preload-dom0-concurrent(avg:2.962): 47.40
  • debian-12-xfce_dispvm-preload-dom0-gui-concurrent(avg:3.686): 58.97
  • debian-12-xfce_dispvm-api(avg:6.765): 108.24
  • debian-12-xfce_dispvm-gui-api(avg:7.861): 125.78
  • debian-12-xfce_dispvm-concurrent-api(avg:3.116): 49.85
  • debian-12-xfce_dispvm-gui-concurrent-api(avg:3.853): 61.64
  • debian-12-xfce_dispvm-preload-api(avg:3.646): 58.34
  • debian-12-xfce_dispvm-preload-less-api(avg:6.217): 99.47
  • debian-12-xfce_dispvm-preload-more-api(avg:3.572): 57.15
  • debian-12-xfce_dispvm-preload-gui-api(avg:5.269): 84.30
  • debian-12-xfce_dispvm-preload-concurrent-api(avg:3.091): 49.46
  • debian-12-xfce_vm(avg:0.413): 6.60
  • debian-12-xfce_vm-gui(avg:0.449): 7.19
  • debian-12-xfce_vm-concurrent(avg:0.376): 6.02
  • debian-12-xfce_vm-gui-concurrent(avg:0.458): 7.33
  • debian-12-xfce_vm-api(avg:0.083): 1.33
  • debian-12-xfce_vm-gui-api(avg:0.121): 1.93
  • debian-12-xfce_vm-concurrent-api(avg:0.048): 0.77
  • debian-12-xfce_vm-gui-concurrent-api(avg:0.105): 1.68
  • fedora-42-xfce_dispvm(avg:6.987): 111.80
  • fedora-42-xfce_dispvm-gui(avg:8.364): 133.82
  • fedora-42-xfce_dispvm-concurrent(avg:3.622): 57.96
  • fedora-42-xfce_dispvm-gui-concurrent(avg:4.388): 70.20
  • fedora-42-xfce_dispvm-dom0(avg:7.806): 124.89
  • fedora-42-xfce_dispvm-dom0-gui(avg:9.207): 147.31
  • fedora-42-xfce_dispvm-dom0-concurrent(avg:3.969): 63.51
  • fedora-42-xfce_dispvm-dom0-gui-concurrent(avg:4.738): 75.81
  • fedora-42-xfce_dispvm-preload(avg:4.256): 68.09
  • fedora-42-xfce_dispvm-preload-gui(avg:4.816): 77.06
  • fedora-42-xfce_dispvm-preload-concurrent(avg:6.462): 103.39
  • fedora-42-xfce_dispvm-preload-gui-concurrent(avg:3.900): 62.41
  • fedora-42-xfce_dispvm-preload-dom0(avg:4.371): 69.94
  • fedora-42-xfce_dispvm-preload-dom0-gui(avg:5.628): 90.04
  • fedora-42-xfce_dispvm-preload-dom0-concurrent(avg:3.505): 56.08
  • fedora-42-xfce_dispvm-preload-dom0-gui-concurrent(avg:4.294): 68.70
  • fedora-42-xfce_dispvm-api(avg:7.895): 126.32
  • fedora-42-xfce_dispvm-gui-api(avg:9.287): 148.59
  • fedora-42-xfce_dispvm-concurrent-api(avg:3.781): 60.50
  • fedora-42-xfce_dispvm-gui-concurrent-api(avg:4.663): 74.61
  • fedora-42-xfce_dispvm-preload-api(avg:4.434): 70.95
  • fedora-42-xfce_dispvm-preload-less-api(avg:7.255): 116.08
  • fedora-42-xfce_dispvm-preload-more-api(avg:4.116): 65.85
  • fedora-42-xfce_dispvm-preload-gui-api(avg:5.361): 85.78
  • fedora-42-xfce_dispvm-preload-concurrent-api(avg:3.525): 56.39
  • fedora-42-xfce_dispvm-preload-gui-concurrent-api(avg:4.614): 73.83
  • fedora-42-xfce_vm(avg:0.555): 8.88
  • fedora-42-xfce_vm-gui(avg:0.562): 8.99
  • fedora-42-xfce_vm-concurrent(avg:0.509): 8.14
  • fedora-42-xfce_vm-gui-concurrent(avg:0.555): 8.89
  • fedora-42-xfce_vm-api(avg:0.141): 2.25
  • fedora-42-xfce_vm-gui-api(avg:0.152): 2.43
  • fedora-42-xfce_vm-concurrent-api(avg:0.096): 1.54
  • fedora-42-xfce_vm-gui-concurrent-api(avg:0.150): 2.39
  • whonix-workstation-17_dispvm(avg:7.748): 123.97
  • whonix-workstation-17_dispvm-gui(avg:8.910): 142.56
  • whonix-workstation-17_dispvm-concurrent(avg:4.383): 70.13
  • whonix-workstation-17_dispvm-gui-concurrent(avg:4.859): 77.75
  • whonix-workstation-17_dispvm-dom0(avg:8.546): 136.73
  • whonix-workstation-17_dispvm-dom0-gui(avg:9.637): 154.20
  • whonix-workstation-17_dispvm-dom0-concurrent(avg:4.429): 70.87
  • whonix-workstation-17_dispvm-dom0-gui-concurrent(avg:5.238): 83.81
  • whonix-workstation-17_dispvm-preload(avg:7.638): 122.21
  • whonix-workstation-17_dispvm-preload-gui(avg:8.550): 136.79
  • whonix-workstation-17_dispvm-preload-concurrent(avg:4.085): 65.36
  • whonix-workstation-17_dispvm-preload-gui-concurrent(avg:4.821): 77.14
  • whonix-workstation-17_dispvm-preload-dom0(avg:4.905): 78.48
  • whonix-workstation-17_dispvm-preload-dom0-gui(avg:5.853): 93.65
  • whonix-workstation-17_dispvm-preload-dom0-concurrent(avg:4.048): 64.77
  • whonix-workstation-17_dispvm-preload-dom0-gui-concurrent(avg:4.908): 78.53
  • whonix-workstation-17_dispvm-api(avg:8.388): 134.21
  • whonix-workstation-17_dispvm-gui-api(avg:9.526): 152.42
  • whonix-workstation-17_dispvm-concurrent-api(avg:4.219): 67.51
  • whonix-workstation-17_dispvm-gui-concurrent-api(avg:4.915): 78.65
  • whonix-workstation-17_dispvm-preload-api(avg:5.136): 82.17
  • whonix-workstation-17_dispvm-preload-less-api(avg:7.765): 124.24
  • whonix-workstation-17_dispvm-preload-more-api(avg:5.283): 84.52
  • whonix-workstation-17_dispvm-preload-gui-api(avg:6.296): 100.73
  • whonix-workstation-17_dispvm-preload-concurrent-api(avg:4.236): 67.78
  • whonix-workstation-17_dispvm-preload-gui-concurrent-api(avg:5.260): 84.16
  • whonix-workstation-17_vm(avg:0.583): 9.32
  • whonix-workstation-17_vm-gui(avg:0.584): 9.35
  • whonix-workstation-17_vm-concurrent(avg:0.554): 8.87
  • whonix-workstation-17_vm-gui-concurrent(avg:0.578): 9.24
  • whonix-workstation-17_vm-api(avg:0.145): 2.32
  • whonix-workstation-17_vm-gui-api(avg:0.162): 2.59
  • whonix-workstation-17_vm-concurrent-api(avg:0.115): 1.84
  • whonix-workstation-17_vm-gui-concurrent-api(avg:0.154): 2.47

@marmarek marmarek merged commit aebac0e into QubesOS:main Aug 6, 2025
3 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants