Qubes update cannot handle hvms with attached pci devices

[RandyTheOtter]

How to file a helpful issue

Qubes OS release

4.2.4, 4.3

Brief summary

Qubes update GUI (with concurrency set to 1) and qubes-vm-update --max-concurrency=1 are unable to handle multiple hvms that operate at least one same pcie device. Next qube in the queue is started without waiting for previous one to shut down completely, which causes pcie device conflicts.

Steps to reproduce

1. Create at least two hvms that have at least one same pcie device attached to all of them
2. Attempt to update all of them
   • Set concurrency to 1 for this step

Expected behavior

If currently updated vm has at least one same pcie device attached to it as the next vm in queue, wait for complete shutdown before starting the next vm.

Actual behavior

Updating mechanism attempts to boot the second hvm immediately after initiating shutdown of the first hvm, which results in a pcie device conflict.

Additional information

It would be nice to see this fixed for higher concurrency settings as well :)

[Atrate]

This might affect 4.3 as well, I haven't tested it there.

Yes, it does affect r4.3 as well.

[Jayant-kernel]

Hi, @RandyTheOtter @ben-grande @andrewdavidwong
I’d like to take this issue. I’ll start by reproducing it and narrowing down which component is responsible, then I’ll report findings and open a PR if needed.

[Jayant-kernel]

@andrewdavidwong
I reproduced the logic path and opened a fix PR here: QubesOS/qubes-core-admin-linux#208

Main change is to wait for full VM shutdown before moving to the next queued VM, so shared PCI devices are released first.