Skip to contentSkip to conversation

spapr_pci: remove all child functions in function zero unplug

PATCH In review Daniel Henrique Barboza 3 participants 4 messages Aug 22, 2019 – Aug 23, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 spapr_pci: remove all child functions in function zero unplug

There is nothing wrong with how sPAPR handles multifunction PCI hot unplugs. The problem is that x86 does it simpler. Instead of removing each non-zero function and then removing function zero, x86 can remove any function of the slot to trigger the hot unplug.

Libvirt will be directly impacted by this difference, in the (hopefully soon) PCI Multifunction hot plug/unplug support. For hot plugs, both x86 and sPAPR will operate the same way: a XML with all desired functions to be added, then consecutive hotplugs of all non-zero functions first, zero last. For hot unplugs, at least in the current state, a XML with the devices to be removed must also be provided because of how sPAPR operates - x86 does not need it - since any function unplug will unplug the whole PCIe slot. This difference puts extra strain in the management layer, which needs to either handle both archs differently in the unplug scenario or choose treat x86 like sPAPR, forcing x86 users to cope with sPAPR internals.

This patch changes spapr_pci_unplug_request to handle the unplug of function zero differently. When removing function zero, instead of error-ing out if there are any remaining function DRCs which needs detaching, detach those. This has no effect in any existing scripts that are detaching the non-zero functions before function zero, and can be used by management as a shortcut to remove the whole PCI multifunction device without specifying each child function.

---
 hw/ppc/spapr_pci.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)
hw/ppc/spapr_pci.c +7 -5
index deb0b0c80c..9f176f463e 100644
@@ -1700,11 +1700,13 @@ static void spapr_pci_unplug_request(HotplugHandler *plug_handler,
                 state = func_drck->dr_entity_sense(func_drc);
                 if (state == SPAPR_DR_ENTITY_SENSE_PRESENT
                     && !spapr_drc_unplug_requested(func_drc)) {
-                    error_setg(errp,
-                               "PCI: slot %d, function %d still present. "
-                               "Must unplug all non-0 functions first.",
-                               slotnr, i);
-                    return;
+                    /*
+                     * Attempting to remove function 0 of a multifunction
+                     * device will will cascade into removing all child
+                     * functions, even if their unplug weren't requested
+                     * beforehand.
+                     */
+                    spapr_drc_detach(func_drc);
                 }
             }
         }
Signature · 1 line
2.21.0
#1 lore raw
Quoted 10 lines · Daniel Henrique Barboza

There is nothing wrong with how sPAPR handles multifunction PCI hot unplugs. The problem is that x86 does it simpler. Instead of removing each non-zero function and then removing function zero, x86 can remove any function of the slot to trigger the hot unplug.

+++ b/hw/ppc/spapr_pci.c

hw/ppc/spapr_pci.c +0 -0
@@ -1700,11 +1700,13 @@ static void spapr_pci_unplug_request(HotplugHandler *plug_handler,
                 state = func_drck->dr_entity_sense(func_drc);
                 if (state == SPAPR_DR_ENTITY_SENSE_PRESENT

hw/ppc/spapr_pci.c static void spapr_pci_unplug_request(HotplugHandler *plug_handler, · line 1706

Quoted 10 lines
hw/ppc/spapr_pci.c +4 -5
                     && !spapr_drc_unplug_requested(func_drc)) {
-                    error_setg(errp,
-                               "PCI: slot %d, function %d still present. "
-                               "Must unplug all non-0 functions first.",
-                               slotnr, i);
-                    return;
+                    /*
+                     * Attempting to remove function 0 of a multifunction
+                     * device will will cascade into removing all child
+                     * functions, even if their unplug weren't requested

s/weren't/wasn't/

Quoted 7 lines
hw/ppc/spapr_pci.c +3 -0
+                     * beforehand.
+                     */
+                    spapr_drc_detach(func_drc);
                 }
             }
         }
Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#2 lore raw
Quoted 7 lines · Eric Blake
Quoted 5 lines · Daniel Henrique Barboza

There is nothing wrong with how sPAPR handles multifunction PCI hot unplugs. The problem is that x86 does it simpler. Instead of removing each non-zero function and then removing function zero, x86 can remove any function of the slot to trigger the hot unplug.

Quoted 16 lines
Quoted 4 lines

+++ b/hw/ppc/spapr_pci.c

hw/ppc/spapr_pci.c +0 -0
@@ -1700,11 +1700,13 @@ static void spapr_pci_unplug_request(HotplugHandler *plug_handler,
                 state = func_drck->dr_entity_sense(func_drc);
                 if (state == SPAPR_DR_ENTITY_SENSE_PRESENT

hw/ppc/spapr_pci.c static void spapr_pci_unplug_request(HotplugHandler *plug_handler, · line 1706

Quoted 10 lines
hw/ppc/spapr_pci.c +4 -5
                     && !spapr_drc_unplug_requested(func_drc)) {
-                    error_setg(errp,
-                               "PCI: slot %d, function %d still present. "
-                               "Must unplug all non-0 functions first.",
-                               slotnr, i);
-                    return;
+                    /*
+                     * Attempting to remove function 0 of a multifunction
+                     * device will will cascade into removing all child
+                     * functions, even if their unplug weren't requested

s/weren't/wasn't/

Actually, I think this is technically a subjunctive, which would make it "were" rather than "was". Modern English usage doesn't really do the subjunctive, though.
</more-pedantic-than-thou>

With my maintainer hat on, rather than pedant hat, the meaning is clear so I really don't care, especially from a contributer whose first language isn't English.

Quoted 9 lines
Quoted 8 lines
hw/ppc/spapr_pci.c +3 -0
+                     * beforehand.
+                     */
+                    spapr_drc_detach(func_drc);
                 }
             }
         }
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#3 lore raw
In reply to Daniel Henrique Barboza
Quoted 18 lines · Daniel Henrique Barboza

There is nothing wrong with how sPAPR handles multifunction PCI hot unplugs. The problem is that x86 does it simpler. Instead of removing each non-zero function and then removing function zero, x86 can remove any function of the slot to trigger the hot unplug.

Libvirt will be directly impacted by this difference, in the (hopefully soon) PCI Multifunction hot plug/unplug support. For hot plugs, both x86 and sPAPR will operate the same way: a XML with all desired functions to be added, then consecutive hotplugs of all non-zero functions first, zero last. For hot unplugs, at least in the current state, a XML with the devices to be removed must also be provided because of how sPAPR operates - x86 does not need it - since any function unplug will unplug the whole PCIe slot. This difference puts extra strain in the management layer, which needs to either handle both archs differently in the unplug scenario or choose treat x86 like sPAPR, forcing x86 users to cope with sPAPR internals.

Quoted 8 lines

This patch changes spapr_pci_unplug_request to handle the unplug of function zero differently. When removing function zero, instead of error-ing out if there are any remaining function DRCs which needs detaching, detach those. This has no effect in any existing scripts that are detaching the non-zero functions before function zero, and can be used by management as a shortcut to remove the whole PCI multifunction device without specifying each child function.

Makes sense to me, applied to ppc-for-4.2.

Quoted 29 lines
---
 hw/ppc/spapr_pci.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)
hw/ppc/spapr_pci.c +7 -5
index deb0b0c80c..9f176f463e 100644
@@ -1700,11 +1700,13 @@ static void spapr_pci_unplug_request(HotplugHandler *plug_handler,
                 state = func_drck->dr_entity_sense(func_drc);
                 if (state == SPAPR_DR_ENTITY_SENSE_PRESENT
                     && !spapr_drc_unplug_requested(func_drc)) {
-                    error_setg(errp,
-                               "PCI: slot %d, function %d still present. "
-                               "Must unplug all non-0 functions first.",
-                               slotnr, i);
-                    return;
+                    /*
+                     * Attempting to remove function 0 of a multifunction
+                     * device will will cascade into removing all child
+                     * functions, even if their unplug weren't requested
+                     * beforehand.
+                     */
+                    spapr_drc_detach(func_drc);
                 }
             }
         }
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson