s390x/pci: Unregister listeners before destroying IOMMU address space
PATCH 1/1 s390x/pci: Unregister listeners before destroying IOMMU address space
Hot-unplugging a vfio-pci device on s390x causes a QEMU crash:
qemu-system-s390x: ../softmmu/memory.c:2772:
do_address_space_destroy: Assertion `QTAILQ_EMPTY(&as->listeners)' failed.
In s390, the IOMMU address space is freed during device unplug but the associated vfio-pci device may not yet be finalized and therefore may still have a listener registered to the IOMMU address space.
Commit a2166410ad74 ("spapr_pci: Unregister listeners before destroying the IOMMU address space") previously resolved this issue for spapr_pci. We are now seeing this in s390x; it would seem the possibility for this issue was already present but based on a bisect commit 2d24a6466154 ("device-core: use RCU for list of children of a bus") has now changed the timing such that it is now readily reproducible.
Add logic to ensure listeners are removed before destroying the address space.
--- hw/s390x/s390-pci-bus.c | 7 +++++++ 1 file changed, 7 insertions(+)
hw/s390x/s390-pci-bus.c +7 -0
@@ -732,6 +732,13 @@ static void s390_pci_iommu_free(S390pciState *s, PCIBus *bus, int32_t devfn) table->iommu[PCI_SLOT (devfn)] =NULL ;g_hash_table_destroy (iommu->iotlb); +/* +* An attached PCI device may have memory listeners, eg. VFIO PCI. +* The associated subregion will already have been unmapped in +* s390_pci_iommu_disable in response to the guest deconfigure request. +* Remove the listeners now before destroying the address space. +*/ +address_space_remove_listeners (&iommu->as);address_space_destroy (&iommu->as);object_unparent (OBJECT (&iommu->mr));object_unparent (OBJECT (iommu));
Signature · 1 line
1.8.3.1
Quoted 36 lines · Matthew Rosato
Hot-unplugging a vfio-pci device on s390x causes a QEMU crash:
qemu-system-s390x:
../softmmu/memory.c:2772:
do_address_space_destroy: Assertion `QTAILQ_EMPTY(&as->listeners)' failed.In s390, the IOMMU address space is freed during device unplug but the associated vfio-pci device may not yet be finalized and therefore may still have a listener registered to the IOMMU address space.
Commit
a2166410ad74("spapr_pci: Unregister listeners before destroying the IOMMU address space") previously resolved this issue forspapr_pci. We are now seeing this in s390x; it would seem the possibility for this issue was already present but based on a bisect commit2d24a6466154("device-core: use RCU for list of children of a bus") has now changed the timing such that it is now readily reproducible.Add logic to ensure listeners are removed before destroying the address space.
Reported-by Niklas Schnelle <[email protected]>Signed-off-by Matthew Rosato <[email protected]>Reviewed-by Pierre Morel <[email protected]>Tested-by Niklas Schnelle <[email protected]>--- hw/s390x/s390-pci-bus.c | 7 +++++++ 1 file changed, 7 insertions(+)hw/s390x/s390-pci-bus.c +0
-0@@ -732,6 +732,13 @@ static void s390_pci_iommu_free(S390pciState *s, PCIBus *bus, int32_t devfn) table->iommu[PCI_SLOT (devfn)] =NULL ;g_hash_table_destroy (iommu->iotlb);
hw/s390x/s390-pci-bus.c static void s390_pci_iommu_free(S390pciState *s, PCIBus *bus, int32_t devfn) · line 744
Quoted 10 lines
hw/s390x/s390-pci-bus.c +7
-0+/* +* An attached PCI device may have memory listeners, eg. VFIO PCI. +* The associated subregion will already have been unmapped in +* s390_pci_iommu_disable in response to the guest deconfigure request. +* Remove the listeners now before destroying the address space. +*/ +address_space_remove_listeners (&iommu->as);address_space_destroy (&iommu->as);object_unparent (OBJECT (&iommu->mr));object_unparent (OBJECT (iommu));
Looks reasonable to me.
Quoted 19 lines · On Mon, 16 Nov 2020 16:42:35 -0500 Matthew Rosato <[email protected]>…
Hot-unplugging a vfio-pci device on s390x causes a QEMU crash:
qemu-system-s390x:
../softmmu/memory.c:2772:
do_address_space_destroy: Assertion `QTAILQ_EMPTY(&as->listeners)' failed.In s390, the IOMMU address space is freed during device unplug but the associated vfio-pci device may not yet be finalized and therefore may still have a listener registered to the IOMMU address space.
Commit
a2166410ad74("spapr_pci: Unregister listeners before destroying the IOMMU address space") previously resolved this issue forspapr_pci. We are now seeing this in s390x; it would seem the possibility for this issue was already present but based on a bisect commit2d24a6466154("device-core: use RCU for list of children of a bus") has now changed the timing such that it is now readily reproducible.Add logic to ensure listeners are removed before destroying the address space.
Quoted 7 lines
Reported-by Niklas Schnelle <[email protected]>Signed-off-by Matthew Rosato <[email protected]>Reviewed-by Pierre Morel <[email protected]>Tested-by Niklas Schnelle <[email protected]>--- hw/s390x/s390-pci-bus.c | 7 +++++++ 1 file changed, 7 insertions(+)
Thanks, queued to s390-fixes.