Skip to content

Add threshold preload feature#377

Merged
marmarek merged 1 commit intoQubesOS:mainfrom
ben-grande:preload-mem-threshold
Aug 6, 2025
Merged

Add threshold preload feature#377
marmarek merged 1 commit intoQubesOS:mainfrom
ben-grande:preload-mem-threshold

Conversation

@ben-grande
Copy link
Contributor

@codecov
Copy link

codecov bot commented Jul 30, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.11%. Comparing base (cdf85fa) to head (ba79054).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #377      +/-   ##
==========================================
- Coverage   76.13%   76.11%   -0.03%     
==========================================
  Files          53       53              
  Lines        9222     9222              
==========================================
- Hits         7021     7019       -2     
- Misses       2201     2203       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ben-grande ben-grande force-pushed the preload-mem-threshold branch from 4542f6e to ba79054 Compare July 31, 2025 04:44

Memory size in MiB to deduct (on calculation only) from system's available
memory. The resulting value is considered before trying to preload to avoid
exhausting system's memory.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This description isn't quite clear what that feature does. Maybe better something like "Memory size in MiB to leave free when preloading disposables. If preloading a disposable would leave less memory, it's skipped".

@andrewdavidwong any better idea?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or maybe something based on comments in the other PR (QubesOS/qubes-desktop-linux-manager#272)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about: "The amount of free system memory that will never be used for preloading disposables. Ensures preloaded disposables do not consume all available memory."

@qubesos-bot
Copy link

qubesos-bot commented Aug 4, 2025

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025080604-4.3&flavor=pull-requests

Test run included the following:

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025061004-4.3&flavor=update

  • system_tests_extra

    • TC_00_QVCTest_whonix-gateway-17: test_010_screenshare (failure)
      ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 0 == 0
  • system_tests_qwt_win10_seamless@hw13

    • windows_clipboard_and_filecopy: unnamed test (unknown)
    • windows_clipboard_and_filecopy: Failed (test died)
      # Test died: no candidate needle with tag(s) 'windows-Edge-address-...
  • system_tests_qwt_win11@hw13

    • windows_install: wait_serial (wait serial expected)
      # wait_serial expected: qr/qDqV_-\d+-/...

    • windows_install: Failed (test died + timed out)
      # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...

  • system_tests_dispvm_perf@hw7

Failed tests

9 failures
  • system_tests_extra

    • TC_00_QVCTest_whonix-gateway-17: test_010_screenshare (failure)
      ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 0 == 0

    • TC_00_QVCTest_whonix-workstation-17: test_010_screenshare (failure)
      AssertionError: 1 != 0 : Timeout waiting for /dev/video0 in test-in...

  • system_tests_kde_gui_interactive

    • gui_keyboard_layout: wait_serial (wait serial expected)
      # wait_serial expected: "echo -e '[Layout]\nLayoutList=us,de' | sud...

    • gui_keyboard_layout: Failed (test died)
      # Test died: command 'test "$(cd ~user;ls e1*)" = "$(qvm-run -p wor...

  • system_tests_qwt_win10_seamless@hw13

    • windows_clipboard_and_filecopy: unnamed test (unknown)
    • windows_clipboard_and_filecopy: Failed (test died)
      # Test died: no candidate needle with tag(s) 'windows-Edge-address-...
  • system_tests_qwt_win11@hw13

    • windows_install: wait_serial (wait serial expected)
      # wait_serial expected: qr/qDqV_-\d+-/...

    • windows_install: Failed (test died + timed out)
      # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...

  • system_tests_dispvm_perf@hw7

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/142375#dependencies

10 fixed

Unstable tests

Details

Performance Tests

Performance degradation:

9 performance degradations
  • debian-12-xfce_exec-data-simplex: 74.46 🔺 ( previous job: 65.51, degradation: 113.65%)
  • debian-12-xfce_exec-data-duplex-root: 84.31 🔺 ( previous job: 70.01, degradation: 120.43%)
  • dom0_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 13726.00 :small_red_triangle: ( previous job: 17102.00, degradation: 80.26%)
  • dom0_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 524.00 :small_red_triangle: ( previous job: 1091.00, degradation: 48.03%)
  • dom0_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 14.00 :small_red_triangle: ( previous job: 1840.00, degradation: 0.76%)
  • dom0_varlibqubes_seq1m_q8t1_read 3:read_bandwidth_kb: 61207.00 :small_red_triangle: ( previous job: 289182.00, degradation: 21.17%)
  • dom0_varlibqubes_seq1m_q8t1_write 3:write_bandwidth_kb: 78728.00 :small_red_triangle: ( previous job: 122848.00, degradation: 64.09%)
  • dom0_varlibqubes_rnd4k_q32t1_write 3:write_bandwidth_kb: 5543.00 :small_red_triangle: ( previous job: 8874.00, degradation: 62.46%)
  • dom0_varlibqubes_rnd4k_q1t1_write 3:write_bandwidth_kb: 3403.00 :small_red_triangle: ( previous job: 4420.00, degradation: 76.99%)

Remaining performance tests:

164 tests
  • debian-12-xfce_exec: 6.91 🟢 ( previous job: 8.63, improvement: 80.04%)
  • debian-12-xfce_exec-root: 27.96 🟢 ( previous job: 29.44, improvement: 94.98%)
  • debian-12-xfce_socket: 7.90 🟢 ( previous job: 8.50, improvement: 92.97%)
  • debian-12-xfce_socket-root: 8.38 🔺 ( previous job: 8.31, degradation: 100.81%)
  • debian-12-xfce_exec-data-duplex: 70.88 🟢 ( previous job: 73.55, improvement: 96.38%)
  • debian-12-xfce_socket-data-duplex: 157.66 🟢 ( previous job: 161.35, improvement: 97.71%)
  • fedora-42-xfce_exec: 9.04
  • fedora-42-xfce_exec-root: 60.22
  • fedora-42-xfce_socket: 8.59
  • fedora-42-xfce_socket-root: 7.61
  • fedora-42-xfce_exec-data-simplex: 66.57
  • fedora-42-xfce_exec-data-duplex: 69.19
  • fedora-42-xfce_exec-data-duplex-root: 95.26
  • fedora-42-xfce_socket-data-duplex: 140.03
  • whonix-gateway-17_exec: 7.84 🔺 ( previous job: 7.34, degradation: 106.77%)
  • whonix-gateway-17_exec-root: 41.95 🔺 ( previous job: 39.57, degradation: 105.99%)
  • whonix-gateway-17_socket: 7.68 🟢 ( previous job: 7.85, improvement: 97.78%)
  • whonix-gateway-17_socket-root: 8.51 🔺 ( previous job: 7.89, degradation: 107.84%)
  • whonix-gateway-17_exec-data-simplex: 75.55 🟢 ( previous job: 77.76, improvement: 97.16%)
  • whonix-gateway-17_exec-data-duplex: 78.31 🟢 ( previous job: 78.39, improvement: 99.90%)
  • whonix-gateway-17_exec-data-duplex-root: 89.55 🟢 ( previous job: 90.74, improvement: 98.69%)
  • whonix-gateway-17_socket-data-duplex: 167.16 🔺 ( previous job: 161.95, degradation: 103.21%)
  • whonix-workstation-17_exec: 8.25 🟢 ( previous job: 8.27, improvement: 99.70%)
  • whonix-workstation-17_exec-root: 53.08 🟢 ( previous job: 57.61, improvement: 92.14%)
  • whonix-workstation-17_socket: 7.96 🟢 ( previous job: 8.97, improvement: 88.73%)
  • whonix-workstation-17_socket-root: 8.38 🟢 ( previous job: 9.46, improvement: 88.62%)
  • whonix-workstation-17_exec-data-simplex: 75.94 🔺 ( previous job: 74.54, degradation: 101.88%)
  • whonix-workstation-17_exec-data-duplex: 76.30 🔺 ( previous job: 74.84, degradation: 101.95%)
  • whonix-workstation-17_exec-data-duplex-root: 87.69 🔺 ( previous job: 86.00, degradation: 101.97%)
  • whonix-workstation-17_socket-data-duplex: 161.39 🔺 ( previous job: 160.20, degradation: 100.74%)
  • dom0_root_seq1m_q8t1_read 3:read_bandwidth_kb: 462131.00 :green_circle: ( previous job: 289982.00, improvement: 159.37%)
  • dom0_root_seq1m_q8t1_write 3:write_bandwidth_kb: 204191.00 :green_circle: ( previous job: 101988.00, improvement: 200.21%)
  • dom0_root_seq1m_q1t1_read 3:read_bandwidth_kb: 107886.00 :green_circle: ( previous job: 14284.00, improvement: 755.29%)
  • dom0_root_seq1m_q1t1_write 3:write_bandwidth_kb: 49852.00 :green_circle: ( previous job: 32696.00, improvement: 152.47%)
  • dom0_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 11447.00 :green_circle: ( previous job: 11086.00, improvement: 103.26%)
  • dom0_varlibqubes_seq1m_q1t1_read 3:read_bandwidth_kb: 390385.00 :small_red_triangle: ( previous job: 433654.00, degradation: 90.02%)
  • dom0_varlibqubes_seq1m_q1t1_write 3:write_bandwidth_kb: 194580.00 :green_circle: ( previous job: 167872.00, improvement: 115.91%)
  • dom0_varlibqubes_rnd4k_q32t1_read 3:read_bandwidth_kb: 104720.00 :small_red_triangle: ( previous job: 108760.00, degradation: 96.29%)
  • dom0_varlibqubes_rnd4k_q1t1_read 3:read_bandwidth_kb: 8279.00 :green_circle: ( previous job: 6356.00, improvement: 130.25%)
  • fedora-42-xfce_root_seq1m_q8t1_read 3:read_bandwidth_kb: 322738.00
  • fedora-42-xfce_root_seq1m_q8t1_write 3:write_bandwidth_kb: 335436.00
  • fedora-42-xfce_root_seq1m_q1t1_read 3:read_bandwidth_kb: 302270.00
  • fedora-42-xfce_root_seq1m_q1t1_write 3:write_bandwidth_kb: 81141.00
  • fedora-42-xfce_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 85333.00
  • fedora-42-xfce_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 2998.00
  • fedora-42-xfce_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 7650.00
  • fedora-42-xfce_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 1025.00
  • fedora-42-xfce_private_seq1m_q8t1_read 3:read_bandwidth_kb: 361827.00
  • fedora-42-xfce_private_seq1m_q8t1_write 3:write_bandwidth_kb: 254323.00
  • fedora-42-xfce_private_seq1m_q1t1_read 3:read_bandwidth_kb: 331617.00
  • fedora-42-xfce_private_seq1m_q1t1_write 3:write_bandwidth_kb: 66249.00
  • fedora-42-xfce_private_rnd4k_q32t1_read 3:read_bandwidth_kb: 88727.00
  • fedora-42-xfce_private_rnd4k_q32t1_write 3:write_bandwidth_kb: 3624.00
  • fedora-42-xfce_private_rnd4k_q1t1_read 3:read_bandwidth_kb: 8568.00
  • fedora-42-xfce_private_rnd4k_q1t1_write 3:write_bandwidth_kb: 495.00
  • fedora-42-xfce_volatile_seq1m_q8t1_read 3:read_bandwidth_kb: 368179.00
  • fedora-42-xfce_volatile_seq1m_q8t1_write 3:write_bandwidth_kb: 106908.00
  • fedora-42-xfce_volatile_seq1m_q1t1_read 3:read_bandwidth_kb: 300107.00
  • fedora-42-xfce_volatile_seq1m_q1t1_write 3:write_bandwidth_kb: 72763.00
  • fedora-42-xfce_volatile_rnd4k_q32t1_read 3:read_bandwidth_kb: 84116.00
  • fedora-42-xfce_volatile_rnd4k_q32t1_write 3:write_bandwidth_kb: 2608.00
  • fedora-42-xfce_volatile_rnd4k_q1t1_read 3:read_bandwidth_kb: 8286.00
  • fedora-42-xfce_volatile_rnd4k_q1t1_write 3:write_bandwidth_kb: 812.00
  • debian-12-xfce_dispvm(avg:6.195): 99.12
  • debian-12-xfce_dispvm-gui(avg:7.118): 113.89
  • debian-12-xfce_dispvm-concurrent(avg:2.810): 44.96
  • debian-12-xfce_dispvm-gui-concurrent(avg:3.941): 63.05
  • debian-12-xfce_dispvm-dom0(avg:6.864): 109.83
  • debian-12-xfce_dispvm-dom0-gui(avg:7.858): 125.72
  • debian-12-xfce_dispvm-dom0-concurrent(avg:3.236): 51.77
  • debian-12-xfce_dispvm-dom0-gui-concurrent(avg:4.075): 65.20
  • debian-12-xfce_dispvm-preload(avg:3.439): 55.02
  • debian-12-xfce_dispvm-preload-gui(avg:4.570): 73.13
  • debian-12-xfce_dispvm-preload-concurrent(avg:2.576): 41.22
  • debian-12-xfce_dispvm-preload-gui-concurrent(avg:3.146): 50.33
  • debian-12-xfce_dispvm-preload-dom0(avg:3.539): 56.62
  • debian-12-xfce_dispvm-preload-dom0-gui(avg:5.176): 82.81
  • debian-12-xfce_dispvm-preload-dom0-concurrent(avg:2.962): 47.40
  • debian-12-xfce_dispvm-preload-dom0-gui-concurrent(avg:3.686): 58.97
  • debian-12-xfce_dispvm-api(avg:6.765): 108.24
  • debian-12-xfce_dispvm-gui-api(avg:7.861): 125.78
  • debian-12-xfce_dispvm-concurrent-api(avg:3.116): 49.85
  • debian-12-xfce_dispvm-gui-concurrent-api(avg:3.853): 61.64
  • debian-12-xfce_dispvm-preload-api(avg:3.646): 58.34
  • debian-12-xfce_dispvm-preload-less-api(avg:6.217): 99.47
  • debian-12-xfce_dispvm-preload-more-api(avg:3.572): 57.15
  • debian-12-xfce_dispvm-preload-gui-api(avg:5.269): 84.30
  • debian-12-xfce_dispvm-preload-concurrent-api(avg:3.091): 49.46
  • debian-12-xfce_vm(avg:0.413): 6.60
  • debian-12-xfce_vm-gui(avg:0.449): 7.19
  • debian-12-xfce_vm-concurrent(avg:0.376): 6.02
  • debian-12-xfce_vm-gui-concurrent(avg:0.458): 7.33
  • debian-12-xfce_vm-api(avg:0.083): 1.33
  • debian-12-xfce_vm-gui-api(avg:0.121): 1.93
  • debian-12-xfce_vm-concurrent-api(avg:0.048): 0.77
  • debian-12-xfce_vm-gui-concurrent-api(avg:0.105): 1.68
  • fedora-42-xfce_dispvm(avg:6.987): 111.80
  • fedora-42-xfce_dispvm-gui(avg:8.364): 133.82
  • fedora-42-xfce_dispvm-concurrent(avg:3.622): 57.96
  • fedora-42-xfce_dispvm-gui-concurrent(avg:4.388): 70.20
  • fedora-42-xfce_dispvm-dom0(avg:7.806): 124.89
  • fedora-42-xfce_dispvm-dom0-gui(avg:9.207): 147.31
  • fedora-42-xfce_dispvm-dom0-concurrent(avg:3.969): 63.51
  • fedora-42-xfce_dispvm-dom0-gui-concurrent(avg:4.738): 75.81
  • fedora-42-xfce_dispvm-preload(avg:4.256): 68.09
  • fedora-42-xfce_dispvm-preload-gui(avg:4.816): 77.06
  • fedora-42-xfce_dispvm-preload-concurrent(avg:6.462): 103.39
  • fedora-42-xfce_dispvm-preload-gui-concurrent(avg:3.900): 62.41
  • fedora-42-xfce_dispvm-preload-dom0(avg:4.371): 69.94
  • fedora-42-xfce_dispvm-preload-dom0-gui(avg:5.628): 90.04
  • fedora-42-xfce_dispvm-preload-dom0-concurrent(avg:3.505): 56.08
  • fedora-42-xfce_dispvm-preload-dom0-gui-concurrent(avg:4.294): 68.70
  • fedora-42-xfce_dispvm-api(avg:7.895): 126.32
  • fedora-42-xfce_dispvm-gui-api(avg:9.287): 148.59
  • fedora-42-xfce_dispvm-concurrent-api(avg:3.781): 60.50
  • fedora-42-xfce_dispvm-gui-concurrent-api(avg:4.663): 74.61
  • fedora-42-xfce_dispvm-preload-api(avg:4.434): 70.95
  • fedora-42-xfce_dispvm-preload-less-api(avg:7.255): 116.08
  • fedora-42-xfce_dispvm-preload-more-api(avg:4.116): 65.85
  • fedora-42-xfce_dispvm-preload-gui-api(avg:5.361): 85.78
  • fedora-42-xfce_dispvm-preload-concurrent-api(avg:3.525): 56.39
  • fedora-42-xfce_dispvm-preload-gui-concurrent-api(avg:4.614): 73.83
  • fedora-42-xfce_vm(avg:0.555): 8.88
  • fedora-42-xfce_vm-gui(avg:0.562): 8.99
  • fedora-42-xfce_vm-concurrent(avg:0.509): 8.14
  • fedora-42-xfce_vm-gui-concurrent(avg:0.555): 8.89
  • fedora-42-xfce_vm-api(avg:0.141): 2.25
  • fedora-42-xfce_vm-gui-api(avg:0.152): 2.43
  • fedora-42-xfce_vm-concurrent-api(avg:0.096): 1.54
  • fedora-42-xfce_vm-gui-concurrent-api(avg:0.150): 2.39
  • whonix-workstation-17_dispvm(avg:7.748): 123.97
  • whonix-workstation-17_dispvm-gui(avg:8.910): 142.56
  • whonix-workstation-17_dispvm-concurrent(avg:4.383): 70.13
  • whonix-workstation-17_dispvm-gui-concurrent(avg:4.859): 77.75
  • whonix-workstation-17_dispvm-dom0(avg:8.546): 136.73
  • whonix-workstation-17_dispvm-dom0-gui(avg:9.637): 154.20
  • whonix-workstation-17_dispvm-dom0-concurrent(avg:4.429): 70.87
  • whonix-workstation-17_dispvm-dom0-gui-concurrent(avg:5.238): 83.81
  • whonix-workstation-17_dispvm-preload(avg:7.638): 122.21
  • whonix-workstation-17_dispvm-preload-gui(avg:8.550): 136.79
  • whonix-workstation-17_dispvm-preload-concurrent(avg:4.085): 65.36
  • whonix-workstation-17_dispvm-preload-gui-concurrent(avg:4.821): 77.14
  • whonix-workstation-17_dispvm-preload-dom0(avg:4.905): 78.48
  • whonix-workstation-17_dispvm-preload-dom0-gui(avg:5.853): 93.65
  • whonix-workstation-17_dispvm-preload-dom0-concurrent(avg:4.048): 64.77
  • whonix-workstation-17_dispvm-preload-dom0-gui-concurrent(avg:4.908): 78.53
  • whonix-workstation-17_dispvm-api(avg:8.388): 134.21
  • whonix-workstation-17_dispvm-gui-api(avg:9.526): 152.42
  • whonix-workstation-17_dispvm-concurrent-api(avg:4.219): 67.51
  • whonix-workstation-17_dispvm-gui-concurrent-api(avg:4.915): 78.65
  • whonix-workstation-17_dispvm-preload-api(avg:5.136): 82.17
  • whonix-workstation-17_dispvm-preload-less-api(avg:7.765): 124.24
  • whonix-workstation-17_dispvm-preload-more-api(avg:5.283): 84.52
  • whonix-workstation-17_dispvm-preload-gui-api(avg:6.296): 100.73
  • whonix-workstation-17_dispvm-preload-concurrent-api(avg:4.236): 67.78
  • whonix-workstation-17_dispvm-preload-gui-concurrent-api(avg:5.260): 84.16
  • whonix-workstation-17_vm(avg:0.583): 9.32
  • whonix-workstation-17_vm-gui(avg:0.584): 9.35
  • whonix-workstation-17_vm-concurrent(avg:0.554): 8.87
  • whonix-workstation-17_vm-gui-concurrent(avg:0.578): 9.24
  • whonix-workstation-17_vm-api(avg:0.145): 2.32
  • whonix-workstation-17_vm-gui-api(avg:0.162): 2.59
  • whonix-workstation-17_vm-concurrent-api(avg:0.115): 1.84
  • whonix-workstation-17_vm-gui-concurrent-api(avg:0.154): 2.47

@marmarek marmarek merged commit fd3b8f4 into QubesOS:main Aug 6, 2025
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants