Skip to contentSkip to conversation

hw/pci: enforce use of slot only slot 0 when devices have an upstream PCIE port

PATCH In review Ani Sinha 3 participants 11 messages Jun 20, 2023 – Jun 22, 2023

lore
Share & tools
lore

Ani Sinha

author
#0 lore raw

PATCH v4 1/1 hw/pci: enforce use of slot only slot 0 when devices have an upstream PCIE port

When a device has an upstream PCIE port, we can only use slot 0. Non-zero slots are invalid. This change ensures that we throw an error if the user tries to hotplug a device with an upstream PCIE port to a non-zero slot.

CC: [email protected]
CC: [email protected]
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2128929

---
 hw/pci/pci.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

changelog:
v2: addressed issue with multifunction pcie root ports. Should allow hotplug on functions other than function 0.
v3: improved commit message.
v4: improve commit message and code comments further. Some more improvements might come in v5. No claims made here that this is the final one :-)

hw/pci/pci.c +18 -0
index bf38905b7d..30ce6a78cb 100644
@@ -64,6 +64,7 @@ bool pci_available = true;
 static char *pcibus_get_dev_path(DeviceState *dev);
 static char *pcibus_get_fw_dev_path(DeviceState *dev);
 static void pcibus_reset(BusState *qbus);
+static bool pcie_has_upstream_port(PCIDevice *dev);
 
 static Property pci_props[] = {
     DEFINE_PROP_PCI_DEVFN("addr", PCIDevice, devfn, -1),
@@ -1182,6 +1183,11 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
     } else if (dev->hotplugged &&
                !pci_is_vf(pci_dev) &&
                pci_get_function_0(pci_dev)) {
+        /*
+         * populating function 0 triggers a bus scan from the guest that
+         * exposes other non-zero functions. Hence we need to ensure that
+         * function 0 wasn't added yet.
+         */
         error_setg(errp, "PCI: slot %d function 0 already occupied by %s,"
                    " new func %s cannot be exposed to guest.",
                    PCI_SLOT(pci_get_function_0(pci_dev)->devfn),
@@ -1189,6 +1195,18 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
                    name);
 
        return NULL;
+    } else if (dev->hotplugged &&
+               !pci_is_vf(pci_dev) &&
+               pcie_has_upstream_port(pci_dev) && PCI_SLOT(devfn)) {
+        /*
+         * If the device has an upstream PCIE port, like a pcie root port,
+         * we only support functions on slot 0.
+         */
+        error_setg(errp, "PCI: slot %d is not valid for %s,"
+                   " only functions on slot 0 is supported for devices"
+                   " with an upstream PCIE port.",
+                   PCI_SLOT(devfn), name);
+        return NULL;
     }
 
     pci_dev->devfn = devfn;
Signature · 1 line
2.39.1
#1 lore raw
Quoted 4 lines · On Tue, 20 Jun 2023 12:48:05 +0530 Ani Sinha <[email protected]> wrote:

When a device has an upstream PCIE port, we can only use slot 0. Non-zero slots are invalid.
This change ensures that we throw an error if the user
tries to hotplug a device with an upstream PCIE port to a non-zero slot.

Isn't the same true for coldplugged devices?
Why you limit it only to hotplug?

Quoted 45 lines

CC: [email protected]
CC: [email protected]
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2128929

---
 hw/pci/pci.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

changelog:
v2: addressed issue with multifunction pcie root ports. Should allow hotplug on functions other than function 0.
v3: improved commit message.
v4: improve commit message and code comments further. Some more improvements might come in v5. No claims made here that this is the final one :-)

hw/pci/pci.c +6 -0
index bf38905b7d..30ce6a78cb 100644
@@ -64,6 +64,7 @@ bool pci_available = true;
 static char *pcibus_get_dev_path(DeviceState *dev);
 static char *pcibus_get_fw_dev_path(DeviceState *dev);
 static void pcibus_reset(BusState *qbus);
+static bool pcie_has_upstream_port(PCIDevice *dev);
 
 static Property pci_props[] = {
     DEFINE_PROP_PCI_DEVFN("addr", PCIDevice, devfn, -1),
@@ -1182,6 +1183,11 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
     } else if (dev->hotplugged &&
                !pci_is_vf(pci_dev) &&
                pci_get_function_0(pci_dev)) {
+        /*
+         * populating function 0 triggers a bus scan from the guest that
+         * exposes other non-zero functions. Hence we need to ensure that
+         * function 0 wasn't added yet.
+         */
         error_setg(errp, "PCI: slot %d function 0 already occupied by %s,"
                    " new func %s cannot be exposed to guest.",
                    PCI_SLOT(pci_get_function_0(pci_dev)->devfn),
@@ -1189,6 +1195,18 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
                    name);
 
        return NULL;

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1207

Quoted 10 lines
hw/pci/pci.c +10 -0
+    } else if (dev->hotplugged &&
+               !pci_is_vf(pci_dev) &&
+               pcie_has_upstream_port(pci_dev) && PCI_SLOT(devfn)) {
+        /*
+         * If the device has an upstream PCIE port, like a pcie root port,
+         * we only support functions on slot 0.
+         */
+        error_setg(errp, "PCI: slot %d is not valid for %s,"
+                   " only functions on slot 0 is supported for devices"
+                   " with an upstream PCIE port.",

upstream port language is confusing here and elsewhere you mention it. It would be better to use root-port instead.

Quoted 5 lines
hw/pci/pci.c +2 -0
+                   PCI_SLOT(devfn), name);
+        return NULL;
     }
 
     pci_dev->devfn = devfn;
#2 lore raw
Quoted 55 lines · Igor Mammedov
Quoted 4 lines · On Tue, 20 Jun 2023 12:48:05 +0530 Ani Sinha <[email protected]> wrote:

When a device has an upstream PCIE port, we can only use slot 0. Non-zero slots are invalid.
This change ensures that we throw an error if the user
tries to hotplug a device with an upstream PCIE port to a non-zero slot.

Isn't the same true for coldplugged devices?
Why you limit it only to hotplug?

Quoted 44 lines

CC: [email protected]
CC: [email protected]
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2128929

---
 hw/pci/pci.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

changelog:
v2: addressed issue with multifunction pcie root ports. Should allow hotplug on functions other than function 0.
v3: improved commit message.
v4: improve commit message and code comments further. Some more improvements might come in v5. No claims made here that this is the final one :-)

hw/pci/pci.c +6 -0
index bf38905b7d..30ce6a78cb 100644
@@ -64,6 +64,7 @@ bool pci_available = true;
 static char *pcibus_get_dev_path(DeviceState *dev);
 static char *pcibus_get_fw_dev_path(DeviceState *dev);
 static void pcibus_reset(BusState *qbus);
+static bool pcie_has_upstream_port(PCIDevice *dev);
 
 static Property pci_props[] = {
     DEFINE_PROP_PCI_DEVFN("addr", PCIDevice, devfn, -1),
@@ -1182,6 +1183,11 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
     } else if (dev->hotplugged &&
                !pci_is_vf(pci_dev) &&
                pci_get_function_0(pci_dev)) {
+        /*
+         * populating function 0 triggers a bus scan from the guest that
+         * exposes other non-zero functions. Hence we need to ensure that
+         * function 0 wasn't added yet.
+         */
         error_setg(errp, "PCI: slot %d function 0 already occupied by %s,"
                    " new func %s cannot be exposed to guest.",
                    PCI_SLOT(pci_get_function_0(pci_dev)->devfn),
@@ -1189,6 +1195,18 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
                    name);
 
Quoted 14 lines

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1207

Quoted 11 lines
hw/pci/pci.c +10 -0
        return NULL;
+    } else if (dev->hotplugged &&
+               !pci_is_vf(pci_dev) &&
+               pcie_has_upstream_port(pci_dev) && PCI_SLOT(devfn)) {
+        /*
+         * If the device has an upstream PCIE port, like a pcie root port,
+         * we only support functions on slot 0.
+         */
+        error_setg(errp, "PCI: slot %d is not valid for %s,"
+                   " only functions on slot 0 is supported for devices"
+                   " with an upstream PCIE port.",

upstream port language is confusing here and elsewhere you mention it. It would be better to use root-port instead.

No i do not think this is specific to root ports.
it is technically any non-integrated express device but we also plug pci devices into express ports as a hack.
so checking where device is plugged (this is what pcie_has_upstream_port does) seems like a reasonable approach.

Quoted 5 lines
Quoted 5 lines
hw/pci/pci.c +2 -0
+                   PCI_SLOT(devfn), name);
+        return NULL;
     }
 
     pci_dev->devfn = devfn;
#3 lore raw
In reply to Ani Sinha
Quoted 1 line · Ani Sinha

When a device has an upstream PCIE port, we can only use slot 0.

Actually, it's when device is plugged into a PCIE port. So maybe:

PCI Express ports only have one slot, so PCI Express devices can only be plugged into slot 0 on a PCIE port

Quoted 3 lines

Non-zero slots
are invalid. This change ensures that we throw an error if the user tries to hotplug a device with an upstream PCIE port to a non-zero slot.

it also adds a comment explaining why function 0 must not exist when function != 0 is added. or maybe split that part out.

Quoted 29 lines

CC: [email protected]
CC: [email protected]
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2128929

---
 hw/pci/pci.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

changelog:
v2: addressed issue with multifunction pcie root ports. Should allow hotplug on functions other than function 0.
v3: improved commit message.
v4: improve commit message and code comments further. Some more improvements might come in v5. No claims made here that this is the final one :-)

hw/pci/pci.c +1 -0
index bf38905b7d..30ce6a78cb 100644
@@ -64,6 +64,7 @@ bool pci_available = true;
 static char *pcibus_get_dev_path(DeviceState *dev);
 static char *pcibus_get_fw_dev_path(DeviceState *dev);
 static void pcibus_reset(BusState *qbus);
+static bool pcie_has_upstream_port(PCIDevice *dev);
 
 static Property pci_props[] = {
     DEFINE_PROP_PCI_DEVFN("addr", PCIDevice, devfn, -1),

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1190

Quoted 8 lines
hw/pci/pci.c +5 -0
@@ -1182,6 +1183,11 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
     } else if (dev->hotplugged &&
                !pci_is_vf(pci_dev) &&
                pci_get_function_0(pci_dev)) {
+        /*
+         * populating function 0 triggers a bus scan from the guest that
+         * exposes other non-zero functions. Hence we need to ensure that
+         * function 0 wasn't added yet.
+         */

Pls capitalize populating. Also, comments like this should come before the logic they document, not after. By the way it doesn't have to be a bus scan - I'd just say "a scan" - with ACPI guest knows what was added and can just probe the device functions.

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1198

Quoted 8 lines
hw/pci/pci.c +1 -0
         error_setg(errp, "PCI: slot %d function 0 already occupied by %s,"
                    " new func %s cannot be exposed to guest.",
                    PCI_SLOT(pci_get_function_0(pci_dev)->devfn),
@@ -1189,6 +1195,18 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
                    name);
 
        return NULL;
+    } else if (dev->hotplugged &&

why hotplugged? Doesn't the same rule apply to all devices?

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1199

Quoted 1 line
hw/pci/pci.c +1 -0
+               !pci_is_vf(pci_dev) &&

Hmm. I think you copied it from here:

    } else if (dev->hotplugged &&
               !pci_is_vf(pci_dev) &&
               pci_get_function_0(pci_dev)) {

it makes sense there because VFs are added later
after PF exists.

But here it makes no sense that I can see.

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1202

Quoted 3 lines
hw/pci/pci.c +3 -0
+               pcie_has_upstream_port(pci_dev) && PCI_SLOT(devfn)) {
+        /*
+         * If the device has an upstream PCIE port, like a pcie root port,

no, a root port can not be an upstream port.

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1207

Quoted 5 lines
hw/pci/pci.c +5 -0
+         * we only support functions on slot 0.
+         */
+        error_setg(errp, "PCI: slot %d is not valid for %s,"
+                   " only functions on slot 0 is supported for devices"
+                   " with an upstream PCIE port.",

something like:

error_setg(errp, "PCI: slot %d is not valid for %s:"
" PCI Express devices can only be plugged into slot 0")

and then you don't really need a comment.

Quoted 7 lines
hw/pci/pci.c +2 -0
+                   PCI_SLOT(devfn), name);
+        return NULL;
     }
 
     pci_dev->devfn = devfn;
Signature · 1 line
2.39.1
author
#4 lore raw
Quoted 77 lines · Michael S. Tsirkin
Quoted 1 line · Ani Sinha

When a device has an upstream PCIE port, we can only use slot 0.

Actually, it's when device is plugged into a PCIE port. So maybe:

PCI Express ports only have one slot, so PCI Express devices can only be plugged into slot 0 on a PCIE port

Quoted 3 lines

Non-zero slots
are invalid. This change ensures that we throw an error if the user tries to hotplug a device with an upstream PCIE port to a non-zero slot.

it also adds a comment explaining why function 0 must not exist when function != 0 is added. or maybe split that part out.

Quoted 29 lines

CC: [email protected]
CC: [email protected]
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2128929

---
hw/pci/pci.c | 18 ++++++++++++++++++

1 file changed, 18 insertions(+)

changelog:
v2: addressed issue with multifunction pcie root ports. Should allow hotplug on functions other than function 0.
v3: improved commit message.
v4: improve commit message and code comments further. Some more improvements might come in v5. No claims made here that this is the final one :-)

hw/pci/pci.c +1 -0
index bf38905b7d..30ce6a78cb 100644
@@ -64,6 +64,7 @@ bool pci_available = true;
 static char *pcibus_get_dev_path(DeviceState *dev);
 static char *pcibus_get_fw_dev_path(DeviceState *dev);
 static void pcibus_reset(BusState *qbus);
+static bool pcie_has_upstream_port(PCIDevice *dev);
 
 static Property pci_props[] = {
    DEFINE_PROP_PCI_DEVFN("addr", PCIDevice, devfn, -1),

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1190

Quoted 8 lines
hw/pci/pci.c +5 -0
@@ -1182,6 +1183,11 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
    } else if (dev->hotplugged &&
               !pci_is_vf(pci_dev) &&
               pci_get_function_0(pci_dev)) {
+        /*
+         * populating function 0 triggers a bus scan from the guest that
+         * exposes other non-zero functions. Hence we need to ensure that
+         * function 0 wasn't added yet.
+         */

Pls capitalize populating. Also, comments like this should come before the logic they document, not after. By the way it doesn't have to be a bus scan - I'd just say "a scan" - with ACPI guest knows what was added and can just probe the device functions.

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1198

Quoted 8 lines
hw/pci/pci.c +1 -0
        error_setg(errp, "PCI: slot %d function 0 already occupied by %s,"
                   " new func %s cannot be exposed to guest.",
                   PCI_SLOT(pci_get_function_0(pci_dev)->devfn),
@@ -1189,6 +1195,18 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
                   name);
 
       return NULL;
+    } else if (dev->hotplugged &&

why hotplugged? Doesn't the same rule apply to all devices?

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1199

Quoted 1 line
hw/pci/pci.c +1 -0
+               !pci_is_vf(pci_dev) &&
Quoted 7 lines

Hmm. I think you copied it from here:

   } else if (dev->hotplugged &&
              !pci_is_vf(pci_dev) &&
              pci_get_function_0(pci_dev)) {

it makes sense there because VFs are added later
after PF exists.

I thought PFs are handled only in the host OS and only VFs are passthrough into the guest? I thought this check was because VFs have a different domain address separate from other non-vf devices in the guest PCI tree.

Quoted 33 lines

But here it makes no sense that I can see.

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1202

Quoted 3 lines
hw/pci/pci.c +3 -0
+               pcie_has_upstream_port(pci_dev) && PCI_SLOT(devfn)) {
+        /*
+         * If the device has an upstream PCIE port, like a pcie root port,

no, a root port can not be an upstream port.

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1207

Quoted 5 lines
hw/pci/pci.c +5 -0
+         * we only support functions on slot 0.
+         */
+        error_setg(errp, "PCI: slot %d is not valid for %s,"
+                   " only functions on slot 0 is supported for devices"
+                   " with an upstream PCIE port.",

something like:

error_setg(errp, "PCI: slot %d is not valid for %s:"
" PCI Express devices can only be plugged into slot 0")

and then you don't really need a comment.

Quoted 7 lines
hw/pci/pci.c +2 -0
+                   PCI_SLOT(devfn), name);
+        return NULL;
    }
 
    pci_dev->devfn = devfn;
Signature · 1 line
2.39.1
#5 lore raw
Quoted 82 lines · Ani Sinha
Quoted 82 lines · Michael S. Tsirkin
Quoted 1 line · Ani Sinha

When a device has an upstream PCIE port, we can only use slot 0.

Actually, it's when device is plugged into a PCIE port. So maybe:

PCI Express ports only have one slot, so PCI Express devices can only be plugged into slot 0 on a PCIE port

Quoted 3 lines

Non-zero slots
are invalid. This change ensures that we throw an error if the user tries to hotplug a device with an upstream PCIE port to a non-zero slot.

it also adds a comment explaining why function 0 must not exist when function != 0 is added. or maybe split that part out.

Quoted 29 lines

CC: [email protected]
CC: [email protected]
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2128929

---
hw/pci/pci.c | 18 ++++++++++++++++++

1 file changed, 18 insertions(+)

changelog:
v2: addressed issue with multifunction pcie root ports. Should allow hotplug on functions other than function 0.
v3: improved commit message.
v4: improve commit message and code comments further. Some more improvements might come in v5. No claims made here that this is the final one :-)

hw/pci/pci.c +1 -0
index bf38905b7d..30ce6a78cb 100644
@@ -64,6 +64,7 @@ bool pci_available = true;
 static char *pcibus_get_dev_path(DeviceState *dev);
 static char *pcibus_get_fw_dev_path(DeviceState *dev);
 static void pcibus_reset(BusState *qbus);
+static bool pcie_has_upstream_port(PCIDevice *dev);
 
 static Property pci_props[] = {
    DEFINE_PROP_PCI_DEVFN("addr", PCIDevice, devfn, -1),

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1190

Quoted 8 lines
hw/pci/pci.c +5 -0
@@ -1182,6 +1183,11 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
    } else if (dev->hotplugged &&
               !pci_is_vf(pci_dev) &&
               pci_get_function_0(pci_dev)) {
+        /*
+         * populating function 0 triggers a bus scan from the guest that
+         * exposes other non-zero functions. Hence we need to ensure that
+         * function 0 wasn't added yet.
+         */

Pls capitalize populating. Also, comments like this should come before the logic they document, not after. By the way it doesn't have to be a bus scan - I'd just say "a scan" - with ACPI guest knows what was added and can just probe the device functions.

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1198

Quoted 8 lines
hw/pci/pci.c +1 -0
        error_setg(errp, "PCI: slot %d function 0 already occupied by %s,"
                   " new func %s cannot be exposed to guest.",
                   PCI_SLOT(pci_get_function_0(pci_dev)->devfn),
@@ -1189,6 +1195,18 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
                   name);
 
       return NULL;
+    } else if (dev->hotplugged &&

why hotplugged? Doesn't the same rule apply to all devices?

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1199

Quoted 1 line
hw/pci/pci.c +1 -0
+               !pci_is_vf(pci_dev) &&

Hmm. I think you copied it from here:

   } else if (dev->hotplugged &&
              !pci_is_vf(pci_dev) &&
              pci_get_function_0(pci_dev)) {
Quoted 5 lines
Quoted 2 lines

it makes sense there because VFs are added later
after PF exists.

I thought PFs are handled only in the host OS and only VFs are passthrough into the guest?

This is emulated SRIOV. host and guest would be nested L2.

Quoted 3 lines

I thought this check was because VFs have
a different domain address separate from other non-vf devices in the guest PCI tree.

Maybe take a look at the SRIOV spec then.

Quoted 33 lines
Quoted 33 lines

But here it makes no sense that I can see.

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1202

Quoted 3 lines
hw/pci/pci.c +3 -0
+               pcie_has_upstream_port(pci_dev) && PCI_SLOT(devfn)) {
+        /*
+         * If the device has an upstream PCIE port, like a pcie root port,

no, a root port can not be an upstream port.

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1207

Quoted 5 lines
hw/pci/pci.c +5 -0
+         * we only support functions on slot 0.
+         */
+        error_setg(errp, "PCI: slot %d is not valid for %s,"
+                   " only functions on slot 0 is supported for devices"
+                   " with an upstream PCIE port.",

something like:

error_setg(errp, "PCI: slot %d is not valid for %s:"
" PCI Express devices can only be plugged into slot 0")

and then you don't really need a comment.

Quoted 7 lines
hw/pci/pci.c +2 -0
+                   PCI_SLOT(devfn), name);
+        return NULL;
    }
 
    pci_dev->devfn = devfn;
Signature · 1 line
2.39.1
author
#6 lore raw
In reply to Michael S. Tsirkin
Quoted 78 lines · Michael S. Tsirkin
Quoted 1 line · Ani Sinha

When a device has an upstream PCIE port, we can only use slot 0.

Actually, it's when device is plugged into a PCIE port. So maybe:

PCI Express ports only have one slot, so PCI Express devices can only be plugged into slot 0 on a PCIE port

Quoted 3 lines

Non-zero slots
are invalid. This change ensures that we throw an error if the user tries to hotplug a device with an upstream PCIE port to a non-zero slot.

it also adds a comment explaining why function 0 must not exist when function != 0 is added. or maybe split that part out.

Quoted 29 lines

CC: [email protected]
CC: [email protected]
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2128929

---
hw/pci/pci.c | 18 ++++++++++++++++++

1 file changed, 18 insertions(+)

changelog:
v2: addressed issue with multifunction pcie root ports. Should allow hotplug on functions other than function 0.
v3: improved commit message.
v4: improve commit message and code comments further. Some more improvements might come in v5. No claims made here that this is the final one :-)

hw/pci/pci.c +1 -0
index bf38905b7d..30ce6a78cb 100644
@@ -64,6 +64,7 @@ bool pci_available = true;
 static char *pcibus_get_dev_path(DeviceState *dev);
 static char *pcibus_get_fw_dev_path(DeviceState *dev);
 static void pcibus_reset(BusState *qbus);
+static bool pcie_has_upstream_port(PCIDevice *dev);
 
 static Property pci_props[] = {
    DEFINE_PROP_PCI_DEVFN("addr", PCIDevice, devfn, -1),

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1190

Quoted 8 lines
hw/pci/pci.c +5 -0
@@ -1182,6 +1183,11 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
    } else if (dev->hotplugged &&
               !pci_is_vf(pci_dev) &&
               pci_get_function_0(pci_dev)) {
+        /*
+         * populating function 0 triggers a bus scan from the guest that
+         * exposes other non-zero functions. Hence we need to ensure that
+         * function 0 wasn't added yet.
+         */

Pls capitalize populating. Also, comments like this should come before the logic they document, not after. By the way it doesn't have to be a bus scan - I'd just say "a scan" - with ACPI guest knows what was added and can just probe the device functions.

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1198

Quoted 8 lines
hw/pci/pci.c +1 -0
        error_setg(errp, "PCI: slot %d function 0 already occupied by %s,"
                   " new func %s cannot be exposed to guest.",
                   PCI_SLOT(pci_get_function_0(pci_dev)->devfn),
@@ -1189,6 +1195,18 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
                   name);
 
       return NULL;
+    } else if (dev->hotplugged &&

why hotplugged? Doesn't the same rule apply to all devices?

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1199

Quoted 1 line
hw/pci/pci.c +1 -0
+               !pci_is_vf(pci_dev) &&

Hmm. I think you copied it from here:

Quoted 8 lines
   } else if (dev->hotplugged &&
              !pci_is_vf(pci_dev) &&
              pci_get_function_0(pci_dev)) {

it makes sense there because VFs are added later
after PF exists.

But here it makes no sense that I can see.

This patch with these changes causes failures in bios-tables-test which can be fixed easily. It also breaks hd-geo-test and I do not know enough of this test to fix them.

I think I will drop this patch for now.

Quoted 31 lines

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1202

Quoted 5 lines
hw/pci/pci.c +3 -0
+               pcie_has_upstream_port(pci_dev) && PCI_SLOT(devfn)) {
+        /*
+         * If the device has an upstream PCIE port, like a pcie root port,

no, a root port can not be an upstream port.

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1207

Quoted 5 lines
hw/pci/pci.c +5 -0
+         * we only support functions on slot 0.
+         */
+        error_setg(errp, "PCI: slot %d is not valid for %s,"
+                   " only functions on slot 0 is supported for devices"
+                   " with an upstream PCIE port.",

something like:

error_setg(errp, "PCI: slot %d is not valid for %s:"
" PCI Express devices can only be plugged into slot 0")

and then you don't really need a comment.

Quoted 7 lines
hw/pci/pci.c +2 -0
+                   PCI_SLOT(devfn), name);
+        return NULL;
    }
 
    pci_dev->devfn = devfn;
Signature · 1 line
2.39.1
author
#7 lore raw
Quoted 86 lines · Ani Sinha
Quoted 83 lines · Michael S. Tsirkin
Quoted 1 line · Ani Sinha

When a device has an upstream PCIE port, we can only use slot 0.

Actually, it's when device is plugged into a PCIE port. So maybe:

PCI Express ports only have one slot, so PCI Express devices can only be plugged into slot 0 on a PCIE port

Quoted 3 lines

Non-zero slots
are invalid. This change ensures that we throw an error if the user tries to hotplug a device with an upstream PCIE port to a non-zero slot.

it also adds a comment explaining why function 0 must not exist when function != 0 is added. or maybe split that part out.

Quoted 29 lines

CC: [email protected]
CC: [email protected]
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2128929

---
hw/pci/pci.c | 18 ++++++++++++++++++

1 file changed, 18 insertions(+)

changelog:
v2: addressed issue with multifunction pcie root ports. Should allow hotplug on functions other than function 0.
v3: improved commit message.
v4: improve commit message and code comments further. Some more improvements might come in v5. No claims made here that this is the final one :-)

hw/pci/pci.c +1 -0
index bf38905b7d..30ce6a78cb 100644
@@ -64,6 +64,7 @@ bool pci_available = true;
 static char *pcibus_get_dev_path(DeviceState *dev);
 static char *pcibus_get_fw_dev_path(DeviceState *dev);
 static void pcibus_reset(BusState *qbus);
+static bool pcie_has_upstream_port(PCIDevice *dev);
 
 static Property pci_props[] = {
   DEFINE_PROP_PCI_DEVFN("addr", PCIDevice, devfn, -1),

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1190

Quoted 8 lines
hw/pci/pci.c +5 -0
@@ -1182,6 +1183,11 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
   } else if (dev->hotplugged &&
              !pci_is_vf(pci_dev) &&
              pci_get_function_0(pci_dev)) {
+        /*
+         * populating function 0 triggers a bus scan from the guest that
+         * exposes other non-zero functions. Hence we need to ensure that
+         * function 0 wasn't added yet.
+         */

Pls capitalize populating. Also, comments like this should come before the logic they document, not after. By the way it doesn't have to be a bus scan - I'd just say "a scan" - with ACPI guest knows what was added and can just probe the device functions.

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1198

Quoted 8 lines
hw/pci/pci.c +1 -0
       error_setg(errp, "PCI: slot %d function 0 already occupied by %s,"
                  " new func %s cannot be exposed to guest.",
                  PCI_SLOT(pci_get_function_0(pci_dev)->devfn),
@@ -1189,6 +1195,18 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
                  name);
 
      return NULL;
+    } else if (dev->hotplugged &&

why hotplugged? Doesn't the same rule apply to all devices?

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1199

Quoted 1 line
hw/pci/pci.c +1 -0
+               !pci_is_vf(pci_dev) &&

Hmm. I think you copied it from here:
} else if (dev->hotplugged &&

             !pci_is_vf(pci_dev) &&
             pci_get_function_0(pci_dev)) {
Quoted 6 lines
Quoted 4 lines

it makes sense there because VFs are added later
after PF exists.

But here it makes no sense that I can see.

This patch with these changes causes failures in bios-tables-test which can be fixed easily. It also breaks hd-geo-test and I do not know enough of this test to fix them.

Specifically it breaks test_override_scsi_q35() and test_override_virtio_blk_q35().
I think these tests were wrong to begin with since they attach a pcie-to-pci bridge on a pcie root port and then attach a scsi controller not on the pcie-to-pci bridge but on the root port (which is not possible because we can only attach one device on a non-multifunction pcie root port). Even if I fix them, its failing somewhere else.

I have added Thomas and Laurent, maybe they can help fix these in this test.
I have pushed my patch here: https://gitlab.com/anisinha/qemu/-/commit/24b3eddb968a0739bff222bdf781f722365cc9b2

Quoted 34 lines

I think I will drop this patch for now.

Quoted 31 lines

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1202

Quoted 5 lines
hw/pci/pci.c +3 -0
+               pcie_has_upstream_port(pci_dev) && PCI_SLOT(devfn)) {
+        /*
+         * If the device has an upstream PCIE port, like a pcie root port,

no, a root port can not be an upstream port.

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1207

Quoted 5 lines
hw/pci/pci.c +5 -0
+         * we only support functions on slot 0.
+         */
+        error_setg(errp, "PCI: slot %d is not valid for %s,"
+                   " only functions on slot 0 is supported for devices"
+                   " with an upstream PCIE port.",

something like:

error_setg(errp, "PCI: slot %d is not valid for %s:"
" PCI Express devices can only be plugged into slot 0")

and then you don't really need a comment.

Quoted 7 lines
hw/pci/pci.c +2 -0
+                   PCI_SLOT(devfn), name);
+        return NULL;
   }
 
   pci_dev->devfn = devfn;
Signature · 1 line
2.39.1
author
#8 lore raw
Quoted 55 lines · Ani Sinha
Quoted 52 lines · Ani Sinha
Quoted 48 lines · Michael S. Tsirkin
Quoted 1 line · Ani Sinha

When a device has an upstream PCIE port, we can only use slot 0.

Actually, it's when device is plugged into a PCIE port. So maybe:

PCI Express ports only have one slot, so PCI Express devices can only be plugged into slot 0 on a PCIE port

Quoted 3 lines

Non-zero slots
are invalid. This change ensures that we throw an error if the user tries to hotplug a device with an upstream PCIE port to a non-zero slot.

it also adds a comment explaining why function 0 must not exist when function != 0 is added. or maybe split that part out.

Quoted 26 lines

CC: [email protected]
CC: [email protected]
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2128929

---
hw/pci/pci.c | 18 ++++++++++++++++++

1 file changed, 18 insertions(+)

changelog:
v2: addressed issue with multifunction pcie root ports. Should allow hotplug on functions other than function 0.
v3: improved commit message.
v4: improve commit message and code comments further. Some more improvements might come in v5. No claims made here that this is the final one :-)

hw/pci/pci.c +1 -0
index bf38905b7d..30ce6a78cb 100644
@@ -64,6 +64,7 @@ bool pci_available = true;
 static char *pcibus_get_dev_path(DeviceState *dev);
 static char *pcibus_get_fw_dev_path(DeviceState *dev);
 static void pcibus_reset(BusState *qbus);
+static bool pcie_has_upstream_port(PCIDevice *dev);
 
Quoted 16 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
hw/pci/pci.c +2 -0
 static Property pci_props[] = {
  DEFINE_PROP_PCI_DEVFN("addr", PCIDevice, devfn, -1),
@@ -1182,6 +1183,11 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
  } else if (dev->hotplugged &&
             !pci_is_vf(pci_dev) &&
             pci_get_function_0(pci_dev)) {
+        /*
+         * populating function 0 triggers a bus scan from the guest that
Quoted 8 lines
Quoted 8 lines

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1190

Quoted 3 lines
hw/pci/pci.c +3 -0
+         * exposes other non-zero functions. Hence we need to ensure that
+         * function 0 wasn't added yet.
+         */

Pls capitalize populating. Also, comments like this should come before the logic they document, not after. By the way it doesn't have to be a bus scan - I'd just say "a scan" - with ACPI guest knows what was added and can just probe the device functions.

This commit essentially needs fixing:

commit c46b126088b5616d8b7cd3ff83aaf5d097c36633

Author: Michael Labiuk <[email protected]>
Date:   Fri Sep 30 01:35:42 2022 +0300

tests/x86: Add 'q35' machine type to override-tests in hd-geo-test

Message-Id: <[email protected]>

Quoted 67 lines
Quoted 19 lines
Quoted 19 lines

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1198

Quoted 9 lines
hw/pci/pci.c +1 -0
      error_setg(errp, "PCI: slot %d function 0 already occupied by %s,"
                 " new func %s cannot be exposed to guest.",
                 PCI_SLOT(pci_get_function_0(pci_dev)->devfn),
@@ -1189,6 +1195,18 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
                 name);
 
     return NULL;
+    } else if (dev->hotplugged &&

why hotplugged? Doesn't the same rule apply to all devices?

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1199

Quoted 1 line
hw/pci/pci.c +1 -0
+               !pci_is_vf(pci_dev) &&

Hmm. I think you copied it from here:
} else if (dev->hotplugged &&

            !pci_is_vf(pci_dev) &&
            pci_get_function_0(pci_dev)) {
Quoted 6 lines
Quoted 4 lines

it makes sense there because VFs are added later
after PF exists.

But here it makes no sense that I can see.

This patch with these changes causes failures in bios-tables-test which can be fixed easily. It also breaks hd-geo-test and I do not know enough of this test to fix them.

Specifically it breaks test_override_scsi_q35() and test_override_virtio_blk_q35().
I think these tests were wrong to begin with since they attach a pcie-to-pci bridge on a pcie root port and then attach a scsi controller not on the pcie-to-pci bridge but on the root port (which is not possible because we can only attach one device on a non-multifunction pcie root port). Even if I fix them, its failing somewhere else.

I have added Thomas and Laurent, maybe they can help fix these in this test.
I have pushed my patch here: https://gitlab.com/anisinha/qemu/-/commit/24b3eddb968a0739bff222bdf781f722365cc9b2

Quoted 34 lines

I think I will drop this patch for now.

Quoted 31 lines

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1202

Quoted 5 lines
hw/pci/pci.c +3 -0
+               pcie_has_upstream_port(pci_dev) && PCI_SLOT(devfn)) {
+        /*
+         * If the device has an upstream PCIE port, like a pcie root port,

no, a root port can not be an upstream port.

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1207

Quoted 5 lines
hw/pci/pci.c +5 -0
+         * we only support functions on slot 0.
+         */
+        error_setg(errp, "PCI: slot %d is not valid for %s,"
+                   " only functions on slot 0 is supported for devices"
+                   " with an upstream PCIE port.",

something like:

error_setg(errp, "PCI: slot %d is not valid for %s:"
" PCI Express devices can only be plugged into slot 0")

and then you don't really need a comment.

Quoted 7 lines
hw/pci/pci.c +2 -0
+                   PCI_SLOT(devfn), name);
+        return NULL;
  }
 
  pci_dev->devfn = devfn;
Signature · 1 line
2.39.1
author
#9 lore raw
Quoted 79 lines · Ani Sinha
Quoted 56 lines · Ani Sinha
Quoted 52 lines · Ani Sinha
Quoted 48 lines · Michael S. Tsirkin
Quoted 1 line · Ani Sinha

When a device has an upstream PCIE port, we can only use slot 0.

Actually, it's when device is plugged into a PCIE port. So maybe:

PCI Express ports only have one slot, so PCI Express devices can only be plugged into slot 0 on a PCIE port

Quoted 3 lines

Non-zero slots
are invalid. This change ensures that we throw an error if the user tries to hotplug a device with an upstream PCIE port to a non-zero slot.

it also adds a comment explaining why function 0 must not exist when function != 0 is added. or maybe split that part out.

Quoted 26 lines

CC: [email protected]
CC: [email protected]
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2128929

---
hw/pci/pci.c | 18 ++++++++++++++++++

1 file changed, 18 insertions(+)

changelog:
v2: addressed issue with multifunction pcie root ports. Should allow hotplug on functions other than function 0.
v3: improved commit message.
v4: improve commit message and code comments further. Some more improvements might come in v5. No claims made here that this is the final one :-)

hw/pci/pci.c +1 -0
index bf38905b7d..30ce6a78cb 100644
@@ -64,6 +64,7 @@ bool pci_available = true;
 static char *pcibus_get_dev_path(DeviceState *dev);
 static char *pcibus_get_fw_dev_path(DeviceState *dev);
 static void pcibus_reset(BusState *qbus);
+static bool pcie_has_upstream_port(PCIDevice *dev);
 
Quoted 16 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
hw/pci/pci.c +2 -0
 static Property pci_props[] = {
 DEFINE_PROP_PCI_DEVFN("addr", PCIDevice, devfn, -1),
@@ -1182,6 +1183,11 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
 } else if (dev->hotplugged &&
            !pci_is_vf(pci_dev) &&
            pci_get_function_0(pci_dev)) {
+        /*
+         * populating function 0 triggers a bus scan from the guest that
Quoted 8 lines
Quoted 8 lines

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1190

Quoted 3 lines
hw/pci/pci.c +3 -0
+         * exposes other non-zero functions. Hence we need to ensure that
+         * function 0 wasn't added yet.
+         */

Pls capitalize populating. Also, comments like this should come before the logic they document, not after. By the way it doesn't have to be a bus scan - I'd just say "a scan" - with ACPI guest knows what was added and can just probe the device functions.

This commit essentially needs fixing:

commit c46b126088b5616d8b7cd3ff83aaf5d097c36633

Quoted 8 lines
Author: Michael Labiuk <[email protected]>
Date:   Fri Sep 30 01:35:42 2022 +0300

tests/x86: Add 'q35' machine type to override-tests in hd-geo-test

Message-Id: <[email protected]>

I have sent a patch series with a proposed fix for this test (hd-geo-test) as well as fixes for bios-tables-test. The series also includes my QEMU patch that enforces proper PCIE slot usage. The patches are part of one series and should be applied in the same sequence - test fixes first then the QEMU fix so that the QEMU fix does not break the unit tests. Please review.

Quoted 68 lines
Quoted 68 lines
Quoted 20 lines
Quoted 20 lines

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1198

Quoted 10 lines
hw/pci/pci.c +1 -0
     error_setg(errp, "PCI: slot %d function 0 already occupied by %s,"
                " new func %s cannot be exposed to guest.",
                PCI_SLOT(pci_get_function_0(pci_dev)->devfn),
@@ -1189,6 +1195,18 @@ static PCIDevice *do_pci_register_device(PCIDevice *pci_dev,
                name);
 
    return NULL;
+    } else if (dev->hotplugged &&

why hotplugged? Doesn't the same rule apply to all devices?

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1199

Quoted 1 line
hw/pci/pci.c +1 -0
+               !pci_is_vf(pci_dev) &&

Hmm. I think you copied it from here:
} else if (dev->hotplugged &&

           !pci_is_vf(pci_dev) &&
           pci_get_function_0(pci_dev)) {
Quoted 6 lines
Quoted 4 lines

it makes sense there because VFs are added later
after PF exists.

But here it makes no sense that I can see.

This patch with these changes causes failures in bios-tables-test which can be fixed easily. It also breaks hd-geo-test and I do not know enough of this test to fix them.

Specifically it breaks test_override_scsi_q35() and test_override_virtio_blk_q35().
I think these tests were wrong to begin with since they attach a pcie-to-pci bridge on a pcie root port and then attach a scsi controller not on the pcie-to-pci bridge but on the root port (which is not possible because we can only attach one device on a non-multifunction pcie root port). Even if I fix them, its failing somewhere else.

I have added Thomas and Laurent, maybe they can help fix these in this test.
I have pushed my patch here: https://gitlab.com/anisinha/qemu/-/commit/24b3eddb968a0739bff222bdf781f722365cc9b2

Quoted 34 lines

I think I will drop this patch for now.

Quoted 31 lines

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1202

Quoted 5 lines
hw/pci/pci.c +3 -0
+               pcie_has_upstream_port(pci_dev) && PCI_SLOT(devfn)) {
+        /*
+         * If the device has an upstream PCIE port, like a pcie root port,

no, a root port can not be an upstream port.

hw/pci/pci.c static PCIDevice *do_pci_register_device(PCIDevice *pci_dev, · line 1207

Quoted 5 lines
hw/pci/pci.c +5 -0
+         * we only support functions on slot 0.
+         */
+        error_setg(errp, "PCI: slot %d is not valid for %s,"
+                   " only functions on slot 0 is supported for devices"
+                   " with an upstream PCIE port.",

something like:

error_setg(errp, "PCI: slot %d is not valid for %s:"
" PCI Express devices can only be plugged into slot 0")

and then you don't really need a comment.

Quoted 7 lines
hw/pci/pci.c +2 -0
+                   PCI_SLOT(devfn), name);
+        return NULL;
 }
 
 pci_dev->devfn = devfn;
Signature · 1 line
2.39.1