Skip to content

Add type hints to app.py#438

Merged
marmarek merged 2 commits intoQubesOS:mainfrom
hippalectryon-0:typing-app
Mar 15, 2026
Merged

Add type hints to app.py#438
marmarek merged 2 commits intoQubesOS:mainfrom
hippalectryon-0:typing-app

Conversation

@hippalectryon-0
Copy link

Follows #437

Comments / Questions

_vm_list

  • _vm_list is no a list - it's a dict. I renamed it accordingly.
  • I added a new _vm_dict_initialized variable that explicitly tracks the initialisation rather than relying on is None. (which avoids putting assert is not None everywhere)
  • (not done in this PR) I believe we should add a comment / docstring to state clearly what's supposed to be in both _vm_list and _vm_objects. Would be happy to add it to the PR if I get some suggestions.

deviceclass

def list_deviceclass(self) -> list[str]: 

I would like to make deviceclass a Literal list of allowed strings, but I'm not sure what the exhaustive list of allowed classes are ?

Safe asserts is not None

  • What guarantees below that volume.name is not None ?
default_pool = getattr(
                    self.app, "default_pool_" + volume.name, volume.pool
                )
  • Same for dst_volume.name a bit after:
src_volume = src_vm.volumes[dst_volume.name]
  • in qubesd_call we have
    def qubesd_call(
        self, dest, method, arg=None, payload=None, payload_stream=None
    ):
        if payload_stream:
            method_path = os.path.join(
                qubesadmin.config.QREXEC_SERVICES_DIR, method
            )
            if not os.path.exists(method_path):
                raise qubesadmin.exc.QubesDaemonCommunicationError(
                    "{} not found".format(method_path)
                )
            command = [
                "env",
                "QREXEC_REMOTE_DOMAIN=dom0",
                "QREXEC_REQUESTED_TARGET=" + dest,
                method_path,
                arg,
            ]
          self._call_with_stream(
                command, payload, payload_stream
            )
  • This will fail if dest=None but this is not mentioned in the docstring
  • This should also fail if arg is None since _call_with_stream excepts a list[str], not list[str|None]. Is that OK ?

@hippalectryon-0 hippalectryon-0 force-pushed the typing-app branch 2 times, most recently from 052b35b to 2a42699 Compare February 28, 2026 10:21
@codecov
Copy link

codecov bot commented Feb 28, 2026

Codecov Report

❌ Patch coverage is 98.63014% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 76.36%. Comparing base (1efaa45) to head (8fa5e85).
⚠️ Report is 7 commits behind head on main.

Files with missing lines Patch % Lines
qubesadmin/app.py 98.52% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #438      +/-   ##
==========================================
+ Coverage   76.30%   76.36%   +0.06%     
==========================================
  Files          53       53              
  Lines        9363     9388      +25     
==========================================
+ Hits         7144     7169      +25     
  Misses       2219     2219              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

# TODO what if `dest` remains None here ?
# qubesd_call expects a non-None `dest` arg
dest: QubesVM | None = getattr(self.app, "default_dispvm", None)
assert dest is not None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be None, please remove this assertion. Let qubesd reply with the exception so the caller knows what happened.

Copy link
Author

@hippalectryon-0 hippalectryon-0 Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this comment belongs to #436 but we can have the conversation here.

Just to be clear: I also had to add an assert is not None in qubesd_call when payload_stream is not None, otherwise we raise a TypeError: unsupported operand type(s) for +: 'NoneType' ...'

Can you confirm that what you're suggesting is to remove the above-mentioned first assert dest is no None (L97), but not the second one (L884) ?

Note: even if payload_stream is None, dest is then used in call_header = "{}+{} dom0 name {}\0".format(method, arg or "", dest), and I would be quite surprised if a cast to "None" rather than to "" is the expected behavior here, can you confirm ?

Copy link
Contributor

@ben-grande ben-grande Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some logging:

In qubesadmin module:

diff --git a/qubesadmin/base.py b/qubesadmin/base.py
index b07af14..91253a1 100644
--- a/qubesadmin/base.py
+++ b/qubesadmin/base.py
@@ -84,6 +84,7 @@ class PropertyHolder:
                 if dest:
                     dest = dest.name
         # have the actual implementation at Qubes() instance
+        self.app.log.warning("TTT: dest=%s method=%s arg=%s", dest, method, arg)
         return self.app.qubesd_call(dest, method, arg, payload,
             payload_stream)

In qubes module:

diff --git a/qubes/api/admin.py b/qubes/api/admin.py
index 2b86b4ce..599a636a 100644
--- a/qubes/api/admin.py
+++ b/qubes/api/admin.py
@@ -1337,6 +1337,7 @@ class QubesAdminAPI(qubes.api.AbstractQubesAPI):
             appvm = self.src.default_dispvm
         else:
             appvm = self.dest
+        self.app.log.warning("TTT: src=%s dest=%s appvm=%s", self.src, self.dest, appvm)

         self.fire_event_for_permission(dispvm_template=appvm)
         if preload:
diff --git a/qubes/vm/dispvm.py b/qubes/vm/dispvm.py
index 2f8edcd4..06ec54f8 100644
--- a/qubes/vm/dispvm.py
+++ b/qubes/vm/dispvm.py
@@ -786,7 +786,7 @@ class DispVM(qubes.vm.qubesvm.QubesVM):
         if not getattr(appvm, "template_for_dispvms", False):
             raise qubes.exc.QubesException(
                 "Refusing to create disposable out of app qube which has "
-                "template_for_dispvms=False"
+                "template_for_dispvms=False out of %s" % appvm
             )
         if preload and not appvm.can_preload():
             # Using an exception clutters the log when 'used' event is

Unset the dom0 disposable template:

qvm-prefs dom0 default_dispvm ''

Unset the global disposable template (just to avoid confusion):

qubes-prefs default_dispvm ''

And then:

% qvm-run -p --dispvm -- echo hey
app: TTT: dest=None method=admin.vm.feature.CheckWithTemplate arg=vmexec
@dispvm: Refusing to create disposable out of app qube which has template_for_dispvms=False out of None
qubesd[261121]: WARNING: TTT: src=dom0 dest=dom0 appvm=None

So, the client passing None, the server translates to dom0 somewhere. So I am unsure if you should block it or not. I guess OpenQA will tell.

However, in this particular case, I think I made a mistake, it shouldn't try to get self.app.default_dispvm, but instead, should be "dom0", as a string, not a qube object.

We can't pass dest="", it may fail depending if we are using VMExec or VMShell, the interaction is a bit weird because of the different calls it needs to make in qubesadmin.tools.qvm_run:

% qvm-run -p --dispvm -- echo hey
app: TTT: dest= method=admin.vm.feature.CheckWithTemplate arg=vmexec
@dispvm: VM name contains illegal characters
zsh: exit 255   qvm-run -p --dispvm -- echo hey

% qvm-run -p --dispvm -- 'echo hey'
app: TTT: dest=disp5555 method=admin.vm.property.Get arg=qrexec_timeout
app: TTT: dest=disp5555 method=admin.vm.feature.CheckWithTemplate arg=os
hey
app: TTT: dest=disp5555 method=admin.vm.Kill arg=None

We can pass dest=None:

% qvm-run -p --dispvm -- echo hey
app: TTT: dest=None method=admin.vm.feature.CheckWithTemplate arg=vmexec
app: TTT: dest=disp6178 method=admin.vm.property.Get arg=qrexec_timeout
app: TTT: dest=disp6178 method=admin.vm.feature.CheckWithTemplate arg=os
hey
app: TTT: dest=disp6178 method=admin.vm.Kill arg=None

% qvm-run -p --dispvm -- 'echo hey'
app: TTT: dest=disp4672 method=admin.vm.property.Get arg=qrexec_timeout
app: TTT: dest=disp4672 method=admin.vm.feature.CheckWithTemplate arg=os
hey
app: TTT: dest=disp4672 method=admin.vm.Kill arg=None

And we can pass dest=dom0:

% qvm-run -p --dispvm -- echo hey
app: TTT: dest=dom0 method=admin.vm.feature.CheckWithTemplate arg=vmexec
app: TTT: dest=disp8458 method=admin.vm.property.Get arg=qrexec_timeout
app: TTT: dest=disp8458 method=admin.vm.feature.CheckWithTemplate arg=os
hey
app: TTT: dest=disp8458 method=admin.vm.Kill arg=None

% qvm-run -p --dispvm -- 'echo hey'
app: TTT: dest=disp4995 method=admin.vm.property.Get arg=qrexec_timeout
app: TTT: dest=disp4995 method=admin.vm.feature.CheckWithTemplate arg=os
hey
app: TTT: dest=disp4995 method=admin.vm.Kill arg=None

However, in this particular case, I think I made a mistake, it shouldn't try to get self.app.default_dispvm, but instead, should be "dom0", as a string, not a qube object.

I will explain this a bit more in depth. The client doesn't need to know the system/global default disposable template, they just need to destine their call to dom0. It is better to use dest="dom0" then, because of how it is handled on the server:

https://github.com/QubesOS/qubes-core-admin/blob/53ca30fc3257ac8174ae6a6d37dce70f72e788d5/qubes/api/admin.py#L1336

Using the caller (src) default disposable template.


Not to the point of call_header, it seems that it is currently passed as None and works, if this is correct and if the server does some transformation, I don't know, but it works:

% qvm-run -p --dispvm -- echo hey
app: TTT: dest=None method=admin.vm.feature.CheckWithTemplate arg=vmexec
app: TTT: QubesLocal.qubesd_call header: admin.vm.feature.CheckWithTemplate+vmexec dom0 name None
app: TTT: QubesLocal.qubesd_call header: admin.vm.CreateDisposable+ dom0 name dom0
app: TTT: dest=disp220 method=admin.vm.property.Get arg=qrexec_timeout
app: TTT: QubesLocal.qubesd_call header: admin.vm.property.Get+qrexec_timeout dom0 name disp220
app: TTT: QubesLocal.qubesd_call header: admin.vm.Start+ dom0 name disp220
app: TTT: dest=disp220 method=admin.vm.feature.CheckWithTemplate arg=os
app: TTT: QubesLocal.qubesd_call header: admin.vm.feature.CheckWithTemplate+os dom0 name disp220
hey
app: TTT: dest=disp220 method=admin.vm.Kill arg=None
app: TTT: QubesLocal.qubesd_call header: admin.vm.Kill+ dom0 name disp220

% qvm-run -p --dispvm -- 'echo hey'
app: TTT: QubesLocal.qubesd_call header: admin.vm.CreateDisposable+ dom0 name dom0
app: TTT: dest=disp7281 method=admin.vm.property.Get arg=qrexec_timeout
app: TTT: QubesLocal.qubesd_call header: admin.vm.property.Get+qrexec_timeout dom0 name disp7281
app: TTT: QubesLocal.qubesd_call header: admin.vm.Start+ dom0 name disp7281
app: TTT: dest=disp7281 method=admin.vm.feature.CheckWithTemplate arg=os
app: TTT: QubesLocal.qubesd_call header: admin.vm.feature.CheckWithTemplate+os dom0 name disp7281
hey
app: TTT: dest=disp7281 method=admin.vm.Kill arg=None
app: TTT: QubesLocal.qubesd_call header: admin.vm.Kill+ dom0 name disp7281

If I use on the call_header, dest or "", it doesn't work always, e.g. with VMExec it fails, with VMShell, it never reached that point, so it was not impacted, but if it reached, it would most likely break also:

% qvm-run -p --dispvm -- echo hey
app: TTT: dest=None method=admin.vm.feature.CheckWithTemplate arg=vmexec
app: TTT: QubesLocal.qubesd_call header: admin.vm.feature.CheckWithTemplate+vmexec dom0 name
@dispvm: VM name contains illegal characters
zsh: exit 255   qvm-run -p --dispvm -- echo hey

% qvm-run -p --dispvm -- 'echo hey'
app: TTT: QubesLocal.qubesd_call header: admin.vm.CreateDisposable+ dom0 name dom0
app: TTT: dest=disp4346 method=admin.vm.property.Get arg=qrexec_timeout
app: TTT: QubesLocal.qubesd_call header: admin.vm.property.Get+qrexec_timeout dom0 name disp4346
app: TTT: QubesLocal.qubesd_call header: admin.vm.Start+ dom0 name disp4346
app: TTT: dest=disp4346 method=admin.vm.feature.CheckWithTemplate arg=os
app: TTT: QubesLocal.qubesd_call header: admin.vm.feature.CheckWithTemplate+os dom0 name disp4346
hey
app: TTT: dest=disp4346 method=admin.vm.Kill arg=None
app: TTT: QubesLocal.qubesd_call header: admin.vm.Kill+ dom0 name disp4346

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tl;dr: app.qubesd_call() should never get dest=None (it's an error, should raise an exception). It's okay to have dest=None at vm.qubesd_call(), which will translate it to vm._method_dest.

The thing that happens above is bizarre, and "thanks" to lack of enforcement of dest!=None, it ends up producing weird results. Specifically:

  1. Note this is about admin.vm.feature.CheckWithTemplate method, so it got called from QubesVM.check_with_template(). It will be important later.
  2. QubesLocal.qubesd_call() will actually send call with str(None) as destination.
  3. Qubesd will get it, and will (unsuccessfully) try to find such VM.
  4. Qubesd will respond with QubesVMNotFoundError("None").
  5. Now, back in QubesVM.check_with_template() you get QubesVMNotFoundError("None"), which happen to inherit from KeyError, so gets handled as "no such feature" and default value is used.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this particular case, the issue is in PropertyHolder.qubesd_call() - it should verify if default_dispvm it got is not None, and refuse to continue otherwise (not sure what exception would be most appropriate, but the user should get a message about default dispvm not set).

Copy link
Contributor

@ben-grande ben-grande Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is hardcoding dom0 because of this: https://github.com/QubesOS/qubes-core-admin/blob/53ca30fc3257ac8174ae6a6d37dce70f72e788d5/qubes/api/admin.py#L1336

Calling @dispvm with dest=dom0 gets the dom0.default_dispvm, which defaults to app.default_dispvm.

I will use local_name, I will use dom0, as local is the current qube that loaded the qubesadmin, which might not be dom0.


But then, the message should be clearer why this happened, maybe "Requested default disposable, but 'default_dispvm' property is empty"?
As for the exception, maybe just QubesVMNotFoundError? Or if going with a new one, sometime like QubesDefaultNotSetError?

I don't like QubesDefaultNotSetError the second because the dom0.default_dispvm can use a literal qube reference instead of a default value to use app.deault_dispvm.

I think I will settle for QubesVMNotFoundError, I just wanted to do a better exception.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Calling @dispvm with dest=dom0 gets the dom0.default_dispvm, which defaults to app.default_dispvm.

Correction, it gets src.default_dispvm, which depends on the calling qube property..., so yes, indeed local_name fits better.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR to qubes-builderv2 for reference:

https://github.com/QubesOS/qubes-builderv2/pull/135/changes

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some weird recursion error when using self.app.domains[self.local_name] inside qubesd_call...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that should be self.app.local_name...

@ben-grande
Copy link
Contributor

deviceclass

def list_deviceclass(self) -> list[str]: 

I would like to make deviceclass a Literal list of allowed strings, but I'm not sure what the exhaustive list of allowed classes are ?

Replied in the review above.

Safe asserts is not None

* What guarantees below that `volume.name` is not `None` ?
default_pool = getattr(
                    self.app, "default_pool_" + volume.name, volume.pool
                )

qubesadmin.storage.Volume.name --> self._vm_name -> __init__ --> if vm is not None and vm_name is None. So, as it is looping src_vm.volume.values(), it is guaranteed to not be None.

* Same for `dst_volume.name` a bit after:
src_volume = src_vm.volumes[dst_volume.name]

Same response as above.

* in `qubesd_call` we have
    def qubesd_call(
        self, dest, method, arg=None, payload=None, payload_stream=None
    ):
        if payload_stream:
            method_path = os.path.join(
                qubesadmin.config.QREXEC_SERVICES_DIR, method
            )
            if not os.path.exists(method_path):
                raise qubesadmin.exc.QubesDaemonCommunicationError(
                    "{} not found".format(method_path)
                )
            command = [
                "env",
                "QREXEC_REMOTE_DOMAIN=dom0",
                "QREXEC_REQUESTED_TARGET=" + dest,
                method_path,
                arg,
            ]
          self._call_with_stream(
                command, payload, payload_stream
            )
* This will fail if `dest=None` but this is not mentioned in the docstring

Yes, I don't think dest can ever be None.

* This should also fail if `arg` is `None` since _call_with_stream excepts a `list[str]`, not `list[str|None]`. Is that OK ?

I don't think it should fail, but instead omit arg if it is None.

@hippalectryon-0 hippalectryon-0 force-pushed the typing-app branch 3 times, most recently from d1485e6 to 133e678 Compare March 4, 2026 23:42
@hippalectryon-0
Copy link
Author

hippalectryon-0 commented Mar 4, 2026

Yes, I don't think dest can ever be None.

Not sure I understand if you mean that it is not allowed to ever be None (in which case we should raise an explicit error, or simply remove the | None from the signature, but your above comments seem to indicate you'd rather raise an exception) or if in practice you believe it's not ever called with a None value (which I'd argue is not true as seen in #438 (comment))

I don't think it should fail, but instead omit arg if it is None.

133e678

@ben-grande
Copy link
Contributor

Yes, I don't think dest can ever be None.

Not sure I understand if you mean that it is not allowed to ever be None (in which case we should raise an explicit error, or simply remove the | None from the signature, but your above comments seem to indicate you'd rather raise an exception) or if in practice you believe it's not ever called with a None value (which I'd argue is not true as seen in #438 (comment))

It seems that dest can be None: #438 (comment)

@marmarek
Copy link
Member

I would like to make deviceclass a Literal list of allowed strings, but I'm not sure what the exhaustive list of allowed classes are ?

Technically, the list can be dynamic. Add-on to core-admin can introduce new device classes - for example once you install qubes-video-companion-dom0, you'll get webcam class, but without that package it isn't known. There can be other classes introduced by other packages (for example https://github.com/QubesOS-contrib/qubes-core-admin-addon-bridge-device adds another one).

  • (not done in this PR) I believe we should add a comment / docstring to state clearly what's supposed to be in both _vm_list and _vm_objects. Would be happy to add it to the PR if I get some suggestions.

_vm_list (now _vm_dict) is a cache of known VMs, usually obtained by earlier iteration over the collection (either implicit or explicit). This avoids calling the admin.vm.List method over and over.
The _vm_objects dict is there to ensure you get the same object when do app.domains["some-name"] twice. If you get different objects for the same VM, weird things happen (including outdated cache in some of them...). Note the _vm_objects dict is intentionally not cleared if you clear cache, so even after force-refreshing VMs list, you still get the same objects for VMs that still exist.

@hippalectryon-0
Copy link
Author

hippalectryon-0 commented Mar 12, 2026

Technically, the list can be dynamic.

OK

_vm_list (now _vm_dict) is a cache of known VMs ...

Imo should be "fixed" by #445 so not blocking anymore for this PR

@hippalectryon-0
Copy link
Author

Regarding the discussion around dest=None - thank you for your clarifications. Can you confirm whether you still request dest=None-related changes on this PR ?

@marmarek
Copy link
Member

It can be in separate PR (adjusting type hint, adding assert, and handling the case of default_dispvm=None).

@hippalectryon-0 hippalectryon-0 force-pushed the typing-app branch 2 times, most recently from a1e67cd to 5c4e103 Compare March 12, 2026 04:29
@hippalectryon-0
Copy link
Author

I believe I resolved the comments above

Copy link
Member

@marmarek marmarek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please squash "Cleanup dynamic types" commit into the typing one, as it simply undoes some changes.

@marmarek marmarek merged commit 71fab5b into QubesOS:main Mar 15, 2026
4 of 5 checks passed
@hippalectryon-0 hippalectryon-0 deleted the typing-app branch March 15, 2026 13:19
marmarek added a commit that referenced this pull request Mar 15, 2026
* origin/pr/440:
  type-check devices.py exc.py features.py

Pull request description:

Follows #438
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants