Skip to content

Conversation

@marmarek
Copy link
Member

@marmarek marmarek commented Apr 3, 2025

No description provided.

@codecov
Copy link

codecov bot commented Apr 3, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.19%. Comparing base (8a93ad3) to head (20aa549).
Report is 8 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #670      +/-   ##
==========================================
- Coverage   70.24%   70.19%   -0.05%     
==========================================
  Files          59       59              
  Lines       12739    12739              
==========================================
- Hits         8948     8942       -6     
- Misses       3791     3797       +6     
Flag Coverage Δ
unittests 70.19% <ø> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@qubesos-bot
Copy link

qubesos-bot commented Apr 3, 2025

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025041215-4.3&flavor=pull-requests

Test run included the following:

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025031804-4.3&flavor=update

  • system_tests_qwt_win11@hw13

    • windows_install: wait_serial (wait serial expected)
      # wait_serial expected: qr/dcWzE-\d+-/...

    • windows_install: Failed (test died + timed out)
      # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...

  • system_tests_suspend@hw1

    • suspend: unnamed test (unknown)
    • suspend: Failed (test died)
      # Test died: no candidate needle with tag(s) 'SUSPEND-FAILED' match...
  • system_tests_whonix

    • whonixcheck: unnamed test (unknown)

    • whonixcheck: wait_serial (wait serial expected)
      # wait_serial expected: qr/TBvsr-\d+-/...

    • whonixcheck: Failed (test died + timed out)
      # Test died: command 'qvm-run -ap whonix-gateway-17 'LC_ALL=C whon...

  • system_tests_gui_tools

    • qui_widgets_update: unnamed test (unknown)
    • qui_widgets_update: Failed (test died)
      # Test died: no candidate needle with tag(s) 'qui-updates-widget-op...
  • system_tests_suspend

    • suspend: wait_serial (wait serial expected)
      # wait_serial expected: qr/wG5VX-\d+-/...
  • system_tests_extra

    • TC_00_QVCTest_whonix-workstation-17: test_010_screenshare (failure + cleanup)
      ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 0 == 0
  • system_tests_qrexec

  • system_tests_kde_gui_interactive

    • gui_keyboard_layout: wait_serial (wait serial expected)
      # wait_serial expected: "echo -e '[Layout]\nLayoutList=us,de' | sud...

    • gui_keyboard_layout: Failed (test died)
      # Test died: command 'test "$(cd ~user;ls e1*)" = "$(qvm-run -p wor...

  • system_tests_pvgrub_salt_storage

    • TC_10_VMSalt_fedora-41-xfce: test_003_update (failure)
      AssertionError: Salt command '['qubesctl', '--skip-dom0', '--show-o...
  • system_tests_audio

  • system_tests_guivm_vnc_gui_interactive

    • guivm_manager: unnamed test (unknown)
    • guivm_manager: Failed (test died)
      # Test died: no candidate needle with tag(s) 'menu-qubes-tools-subm...
  • system_tests_qwt_win10@hw13

    • windows_install: wait_serial (wait serial expected)
      # wait_serial expected: qr/LDWI1-\d+-/...

    • windows_install: Failed (test died + timed out)
      # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...

  • system_tests_qwt_win10_seamless@hw13

    • windows_clipboard_and_filecopy: unnamed test (unknown)
    • windows_clipboard_and_filecopy: Failed (test died)
      # Test died: no candidate needle with tag(s) 'windows-Explorer-empt...

Failed tests

37 failures
  • system_tests_whonix@hw7

    • whonixcheck: fail (unknown)
      Whonixcheck for sys-whonix failed...

    • whonixcheck: unnamed test (unknown)

  • system_tests_qwt_win11@hw13

    • windows_install: wait_serial (wait serial expected)
      # wait_serial expected: qr/dcWzE-\d+-/...

    • windows_install: Failed (test died + timed out)
      # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...

  • system_tests_suspend@hw1

    • suspend: unnamed test (unknown)
    • suspend: Failed (test died)
      # Test died: no candidate needle with tag(s) 'SUSPEND-FAILED' match...
  • system_tests_whonix

    • whonixcheck: fail (unknown)
      Whonixcheck for anon-whonix failed...

    • whonixcheck: unnamed test (unknown)

    • whonixcheck: wait_serial (wait serial expected)
      # wait_serial expected: qr/TBvsr-\d+-/...

    • whonixcheck: Failed (test died + timed out)
      # Test died: command 'qvm-run -ap whonix-gateway-17 'LC_ALL=C whon...

  • system_tests_gui_tools

    • qui_widgets_update: unnamed test (unknown)
    • qui_widgets_update: Failed (test died)
      # Test died: no candidate needle with tag(s) 'qui-updates-widget-op...
  • system_tests_suspend

    • suspend: wait_serial (wait serial expected)
      # wait_serial expected: qr/wG5VX-\d+-/...

    • suspend: Failed (test died + timed out)
      # Test died: command 'qvm-run -ap sys-whonix 'LC_ALL=C whonixcheck ...

  • system_tests_extra

    • TC_00_QVCTest_whonix-workstation-17: test_010_screenshare (failure + cleanup)
      ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 0 == 0
  • system_tests_qrexec

  • system_tests_kde_gui_interactive

    • gui_keyboard_layout: wait_serial (wait serial expected)
      # wait_serial expected: "echo -e '[Layout]\nLayoutList=us,de' | sud...

    • gui_keyboard_layout: Failed (test died)
      # Test died: command 'test "$(cd ~user;ls e1*)" = "$(qvm-run -p wor...

  • system_tests_pvgrub_salt_storage

    • TC_10_VMSalt_fedora-41-xfce: test_003_update (failure)
      AssertionError: Salt command '['qubesctl', '--skip-dom0', '--show-o...
  • system_tests_audio

  • system_tests_guivm_vnc_gui_interactive

    • guivm_manager: unnamed test (unknown)
    • guivm_manager: Failed (test died)
      # Test died: no candidate needle with tag(s) 'menu-qubes-tools-subm...
  • system_tests_qwt_win10@hw13

    • windows_install: wait_serial (wait serial expected)
      # wait_serial expected: qr/LDWI1-\d+-/...

    • windows_install: Failed (test died + timed out)
      # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...

  • system_tests_qwt_win10_seamless@hw13

    • windows_clipboard_and_filecopy: unnamed test (unknown)
    • windows_clipboard_and_filecopy: Failed (test died)
      # Test died: no candidate needle with tag(s) 'windows-Explorer-empt...

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/132953#dependencies

9 fixed
  • system_tests_whonix

  • system_tests_suspend

  • system_tests_basic_vm_qrexec_gui

  • system_tests_kde_gui_interactive

    • clipboard_and_web: unnamed test (unknown)

    • clipboard_and_web: Failed (test died)
      # Test died: no candidate needle with tag(s) 'qubes-website' matche...

    • clipboard_and_web: wait_serial (wait serial expected)
      # wait_serial expected: "lspci; echo 2E8vz-\$?-"...

  • system_tests_audio

  • system_tests_guivm_vnc_gui_interactive

    • gui_filecopy: unnamed test (unknown)
    • gui_filecopy: Failed (test died)
      # Test died: no candidate needle with tag(s) 'files-work' matched...

Unstable tests

Details

Performance Tests

Performance degradation:

16 performance degradations
  • debian-12-xfce_exec: 8.37 :small_red_triangle_up: ( previous job: 7.12, degradation: 117.52%)
  • whonix-gateway-17_exec: 8.90 :small_red_triangle_up: ( previous job: 6.82, degradation: 130.50%)
  • whonix-gateway-17_socket-root: 8.50 :small_red_triangle_up: ( previous job: 7.65, degradation: 111.16%)
  • whonix-workstation-17_socket-root: 9.04 :small_red_triangle_up: ( previous job: 8.13, degradation: 111.15%)
  • dom0_root_seq1m_q8t1_read 3:read_bandwidth_kb: 116150.00 :small_red_triangle_up: ( previous job: 446963.00, degradation: 25.99%)
  • dom0_root_seq1m_q1t1_read 3:read_bandwidth_kb: 31030.00 :small_red_triangle_up: ( previous job: 294295.00, degradation: 10.54%)
  • dom0_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 24355.00 :small_red_triangle_up: ( previous job: 79803.00, degradation: 30.52%)
  • dom0_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 950.00 :small_red_triangle_up: ( previous job: 6149.00, degradation: 15.45%)
  • dom0_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 2198.00 :small_red_triangle_up: ( previous job: 4826.00, degradation: 45.54%)
  • dom0_varlibqubes_seq1m_q8t1_write 3:write_bandwidth_kb: 149908.00 :small_red_triangle_up: ( previous job: 250795.00, degradation: 59.77%)
  • dom0_varlibqubes_rnd4k_q32t1_write 3:write_bandwidth_kb: 5338.00 :small_red_triangle_up: ( previous job: 6479.00, degradation: 82.39%)
  • dom0_varlibqubes_rnd4k_q1t1_write 3:write_bandwidth_kb: 3556.00 :small_red_triangle_up: ( previous job: 4903.00, degradation: 72.53%)
  • fedora-41-xfce_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 2140.00 :small_red_triangle_up: ( previous job: 3599.00, degradation: 59.46%)
  • fedora-41-xfce_private_seq1m_q1t1_read 3:read_bandwidth_kb: 275000.00 :small_red_triangle_up: ( previous job: 334687.00, degradation: 82.17%)
  • fedora-41-xfce_volatile_rnd4k_q32t1_write 3:write_bandwidth_kb: 1888.00 :small_red_triangle_up: ( previous job: 5672.00, degradation: 33.29%)
  • fedora-41-xfce_volatile_rnd4k_q1t1_write 3:write_bandwidth_kb: 1478.00 :small_red_triangle_up: ( previous job: 1953.00, degradation: 75.68%)

Remaining performance tests:

56 tests
  • debian-12-xfce_exec-root: 28.79 :small_red_triangle_up: ( previous job: 28.65, degradation: 100.45%)
  • debian-12-xfce_socket: 8.23 🟢 ( previous job: 8.60, improvement: 95.65%)
  • debian-12-xfce_socket-root: 8.92 :small_red_triangle_up: ( previous job: 8.52, degradation: 104.63%)
  • debian-12-xfce_exec-data-simplex: 69.28 🟢 ( previous job: 71.62, improvement: 96.72%)
  • debian-12-xfce_exec-data-duplex: 68.10 🟢 ( previous job: 70.34, improvement: 96.81%)
  • debian-12-xfce_exec-data-duplex-root: 86.76 :small_red_triangle_up: ( previous job: 82.72, degradation: 104.88%)
  • debian-12-xfce_socket-data-duplex: 163.59 :small_red_triangle_up: ( previous job: 156.96, degradation: 104.23%)
  • fedora-41-xfce_exec: 9.33 :small_red_triangle_up: ( previous job: 9.27, degradation: 100.66%)
  • fedora-41-xfce_exec-root: 63.23 :small_red_triangle_up: ( previous job: 61.51, degradation: 102.80%)
  • fedora-41-xfce_socket: 8.33 🟢 ( previous job: 8.63, improvement: 96.50%)
  • fedora-41-xfce_socket-root: 9.00 :small_red_triangle_up: ( previous job: 8.71, degradation: 103.34%)
  • fedora-41-xfce_exec-data-simplex: 64.75 🟢 ( previous job: 75.53, improvement: 85.73%)
  • fedora-41-xfce_exec-data-duplex: 66.18 🟢 ( previous job: 71.56, improvement: 92.48%)
  • fedora-41-xfce_exec-data-duplex-root: 105.60 🟢 ( previous job: 109.13, improvement: 96.77%)
  • fedora-41-xfce_socket-data-duplex: 149.04 🟢 ( previous job: 150.61, improvement: 98.96%)
  • whonix-gateway-17_exec-root: 40.34 🟢 ( previous job: 40.43, improvement: 99.76%)
  • whonix-gateway-17_socket: 7.41 :small_red_triangle_up: ( previous job: 7.24, degradation: 102.40%)
  • whonix-gateway-17_exec-data-simplex: 70.65 🟢 ( previous job: 78.32, improvement: 90.22%)
  • whonix-gateway-17_exec-data-duplex: 75.15 🟢 ( previous job: 76.65, improvement: 98.04%)
  • whonix-gateway-17_exec-data-duplex-root: 92.29 :small_red_triangle_up: ( previous job: 88.52, degradation: 104.26%)
  • whonix-gateway-17_socket-data-duplex: 170.79 🟢 ( previous job: 171.76, improvement: 99.43%)
  • whonix-workstation-17_exec: 8.03 :small_red_triangle_up: ( previous job: 7.67, degradation: 104.72%)
  • whonix-workstation-17_exec-root: 62.43 :small_red_triangle_up: ( previous job: 58.26, degradation: 107.15%)
  • whonix-workstation-17_socket: 8.50 :small_red_triangle_up: ( previous job: 8.19, degradation: 103.77%)
  • whonix-workstation-17_exec-data-simplex: 76.51 :small_red_triangle_up: ( previous job: 74.99, degradation: 102.02%)
  • whonix-workstation-17_exec-data-duplex: 64.36 🟢 ( previous job: 72.71, improvement: 88.52%)
  • whonix-workstation-17_exec-data-duplex-root: 102.71 :small_red_triangle_up: ( previous job: 99.82, degradation: 102.89%)
  • whonix-workstation-17_socket-data-duplex: 165.93 🟢 ( previous job: 169.50, improvement: 97.89%)
  • dom0_root_seq1m_q8t1_write 3:write_bandwidth_kb: 168170.00 :green_circle: ( previous job: 129298.00, improvement: 130.06%)
  • dom0_root_seq1m_q1t1_write 3:write_bandwidth_kb: 104179.00 :green_circle: ( previous job: 95454.00, improvement: 109.14%)
  • dom0_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 11162.00 :green_circle: ( previous job: 10795.00, improvement: 103.40%)
  • dom0_varlibqubes_seq1m_q8t1_read 3:read_bandwidth_kb: 466033.00 :green_circle: ( previous job: 382273.00, improvement: 121.91%)
  • dom0_varlibqubes_seq1m_q1t1_read 3:read_bandwidth_kb: 438734.00 :green_circle: ( previous job: 437636.00, improvement: 100.25%)
  • dom0_varlibqubes_seq1m_q1t1_write 3:write_bandwidth_kb: 180273.00 :small_red_triangle_up: ( previous job: 184752.00, degradation: 97.58%)
  • dom0_varlibqubes_rnd4k_q32t1_read 3:read_bandwidth_kb: 102966.00 :green_circle: ( previous job: 62195.00, improvement: 165.55%)
  • dom0_varlibqubes_rnd4k_q1t1_read 3:read_bandwidth_kb: 7688.00 :green_circle: ( previous job: 7669.00, improvement: 100.25%)
  • fedora-41-xfce_root_seq1m_q8t1_read 3:read_bandwidth_kb: 367277.00 :small_red_triangle_up: ( previous job: 368309.00, degradation: 99.72%)
  • fedora-41-xfce_root_seq1m_q8t1_write 3:write_bandwidth_kb: 280143.00 :green_circle: ( previous job: 162081.00, improvement: 172.84%)
  • fedora-41-xfce_root_seq1m_q1t1_read 3:read_bandwidth_kb: 323434.00 :green_circle: ( previous job: 318716.00, improvement: 101.48%)
  • fedora-41-xfce_root_seq1m_q1t1_write 3:write_bandwidth_kb: 111370.00 :green_circle: ( previous job: 87940.00, improvement: 126.64%)
  • fedora-41-xfce_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 93009.00 :green_circle: ( previous job: 82694.00, improvement: 112.47%)
  • fedora-41-xfce_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 7792.00 :small_red_triangle_up: ( previous job: 8485.00, degradation: 91.83%)
  • fedora-41-xfce_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 906.00 :green_circle: ( previous job: 542.00, improvement: 167.16%)
  • fedora-41-xfce_private_seq1m_q8t1_read 3:read_bandwidth_kb: 393609.00 :green_circle: ( previous job: 373957.00, improvement: 105.26%)
  • fedora-41-xfce_private_seq1m_q8t1_write 3:write_bandwidth_kb: 174844.00 :green_circle: ( previous job: 170062.00, improvement: 102.81%)
  • fedora-41-xfce_private_seq1m_q1t1_write 3:write_bandwidth_kb: 89487.00 :green_circle: ( previous job: 61534.00, improvement: 145.43%)
  • fedora-41-xfce_private_rnd4k_q32t1_read 3:read_bandwidth_kb: 74301.00 :small_red_triangle_up: ( previous job: 80283.00, degradation: 92.55%)
  • fedora-41-xfce_private_rnd4k_q32t1_write 3:write_bandwidth_kb: 3220.00 :green_circle: ( previous job: 2215.00, improvement: 145.37%)
  • fedora-41-xfce_private_rnd4k_q1t1_read 3:read_bandwidth_kb: 8522.00 :green_circle: ( previous job: 7540.00, improvement: 113.02%)
  • fedora-41-xfce_private_rnd4k_q1t1_write 3:write_bandwidth_kb: 1452.00 :green_circle: ( previous job: 1130.00, improvement: 128.50%)
  • fedora-41-xfce_volatile_seq1m_q8t1_read 3:read_bandwidth_kb: 356416.00 :small_red_triangle_up: ( previous job: 369868.00, degradation: 96.36%)
  • fedora-41-xfce_volatile_seq1m_q8t1_write 3:write_bandwidth_kb: 230456.00 :green_circle: ( previous job: 179949.00, improvement: 128.07%)
  • fedora-41-xfce_volatile_seq1m_q1t1_read 3:read_bandwidth_kb: 356173.00 :green_circle: ( previous job: 324737.00, improvement: 109.68%)
  • fedora-41-xfce_volatile_seq1m_q1t1_write 3:write_bandwidth_kb: 46638.00 :green_circle: ( previous job: 17567.00, improvement: 265.49%)
  • fedora-41-xfce_volatile_rnd4k_q32t1_read 3:read_bandwidth_kb: 75436.00 :small_red_triangle_up: ( previous job: 79021.00, degradation: 95.46%)
  • fedora-41-xfce_volatile_rnd4k_q1t1_read 3:read_bandwidth_kb: 8299.00 :green_circle: ( previous job: 7867.00, improvement: 105.49%)

marmarek added 2 commits April 4, 2025 04:35
In Whonix, setup wizard shows up shortly after startup. When that
happens just between capturing screenshot in dom0 and VM, xterm will
loose focus, causing different cursor and thus screenshots not matching.

Avoid this issue by disabling whonix setup wizard by pretending it was
completed already.
Generally most tests should do that, but there are few exceptions, so
deal with it in a more generic way. At least for now.
pipewire-utils in Fedora 42 no longer plays raw audio with --format etc
options unless --raw is added too. But older pipewire-utils doesn't have
the --raw option, so it must be added only when it's actually supported.
Furthermore, --raw option ignores the file name argument (but it's still
mandatory) and always uses stdin/out. Change pw-record call to always
use stdout (by using '-' file name) and redirection, so it works in both
cases.

QubesOS/qubes-issues#9807
@marmarek marmarek merged commit 41e596a into QubesOS:main Apr 12, 2025
3 of 5 checks passed
@marmarek marmarek deleted the devel20250403 branch July 19, 2025 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants