Skip to contentSkip to conversation

Expose and manage PCI device reset

PATCH Superseded Amey Narkhede 5 participants 45 messages Jul 9, 2021 – Jul 31, 2021

Subsystems (8)
Network drivers PowerPC ACPI PCI +4 more subsystems

v10 of 16 Superseded Latest is v16 →

lore
Share & tools
lore
#0 lore raw

v10 cover Expose and manage PCI device reset

PCI and PCIe devices may support a number of possible reset mechanisms for example Function Level Reset (FLR) provided via Advanced Feature or PCIe capabilities, Power Management reset, bus reset, or device specific reset. Currently the PCI subsystem creates a policy prioritizing these reset methods which provides neither visibility nor control to userspace.

Expose the reset methods available per device to userspace, via sysfs and allow an administrative user or device owner to have ability to manage per device reset method priorities or exclusions. This feature aims to allow greater control of a device for use cases as device assignment, where specific device or platform issues may interact poorly with a given reset method, and for which device specific quirks have not been developed.

Changes in v10:
- Fix build error on ppc as reported by build bot

Changes in v9:
- Renamed has_flr bitfield to has_pcie_flr and restored use of PCI_DEV_FLAGS_NO_FLR_RESET in quirk_no_flr()
- Cleaned up sysfs code

Changes in v8:
- Added has_flr bitfield to struct pci_dev to cache flr capability
- Updated encoding scheme used in reset_methods array as per Bjorn's suggestion
- Updated Shanker's ACPI patches

Changes in v7:
- Fix the pci_dev_acpi_reset() prototype mismatch in case of CONFIG_ACPI=n

Changes in v6:
- Address Bjorn's and Krzysztof's review comments
- Add Shanker's updated patches along with new "PCI: Setup ACPI_COMPANION early" patch

Changes in v5:
- Rebase the series over pci/reset branch of Bjorn's pci tree to avoid merge conflicts caused by recent changes in existing reset sysfs attribute

Changes in v4:
- Change the order or strlen and strim in reset_method_store function to avoid extra strlen call.
- Use consistent terminology in new pci_reset_mode enum and rename the probe argument of reset functions.

Changes in v3:
- Dropped "PCI: merge slot and bus reset implementations" which was already accepted separately
- Grammar fixes
- Added Shanker's patches which were rebased on v2 of this series
- Added "PCI: Change the type of probe argument in reset functions" and additional user input sanitization code in reset_method_store function per review feedback from Krzysztof

Changes in v2:
- Use byte array instead of bitmap to keep track of ordering of reset methods
- Fix incorrect use of reset_fn field in octeon driver
- Allow writing comma separated list of names of supported reset methods to reset_method sysfs attribute
- Writing empty string instead of "none" to reset_method attribute disables ability of reset the device

Amey Narkhede (5):
PCI: Add pcie_reset_flr to follow calling convention of other reset
methods
PCI: Add new array for keeping track of ordering of reset methods
PCI: Remove reset_fn field from pci_dev
PCI/sysfs: Allow userspace to query and set device reset mechanism
PCI: Change the type of probe argument in reset functions

Shanker Donthineni (3):
PCI: Define a function to set ACPI_COMPANION in pci_dev
PCI: Setup ACPI fwnode early and at the same time with OF
PCI: Add support for ACPI _RST reset method

 Documentation/ABI/testing/sysfs-bus-pci       |  19 ++
 drivers/crypto/cavium/nitrox/nitrox_main.c    |   4 +-
 .../ethernet/cavium/liquidio/lio_vf_main.c    |   2 +-
 drivers/pci/hotplug/pciehp.h                  |   2 +-
 drivers/pci/hotplug/pciehp_hpc.c              |   4 +-
 drivers/pci/hotplug/pnv_php.c                 |   4 +-
 drivers/pci/pci-acpi.c                        |  38 +++-
 drivers/pci/pci-sysfs.c                       | 107 ++++++++-
 drivers/pci/pci.c                             | 215 ++++++++++--------
 drivers/pci/pci.h                             |  23 +-
 drivers/pci/pcie/aer.c                        |  12 +-
 drivers/pci/probe.c                           |  17 +-
 drivers/pci/quirks.c                          |  42 ++--
 drivers/pci/remove.c                          |   1 -
 include/linux/pci.h                           |  17 +-
 include/linux/pci_hotplug.h                   |   2 +-
 16 files changed, 363 insertions(+), 146 deletions(-)

--
2.32.0

#1 lore raw

PATCH v10 1/8 PCI: Add pcie_reset_flr to follow calling convention of other reset methods

Add has_pcie_flr bitfield in struct pci_dev to indicate support for PCIe FLR to avoid reading PCI_EXP_DEVCAP multiple times.

Currently there is separate function pcie_has_flr() to probe if PCIe FLR is supported by the device which does not match the calling convention followed by reset methods which use second function argument to decide whether to probe or not. Add new function pcie_reset_flr() that follows the calling convention of reset methods.

---
 drivers/crypto/cavium/nitrox/nitrox_main.c |  4 +-
 drivers/pci/pci.c                          | 59 +++++++++++-----------
 drivers/pci/pcie/aer.c                     | 12 ++---
 drivers/pci/probe.c                        |  6 ++-
 drivers/pci/quirks.c                       |  9 ++--
 include/linux/pci.h                        |  3 +-
 6 files changed, 45 insertions(+), 48 deletions(-)
drivers/crypto/cavium/nitrox/nitrox_main.c +1 -3
index facc8e6bc..15d6c8452 100644
@@ -306,9 +306,7 @@ static int nitrox_device_flr(struct pci_dev *pdev)
 		return -ENOMEM;
 	}
 
-	/* check flr support */
-	if (pcie_has_flr(pdev))
-		pcie_flr(pdev);
+	pcie_reset_flr(pdev, 0);
 
 	pci_restore_state(pdev);
 
drivers/pci/pci.c +30 -29

87 lines not on this page · raw message

drivers/pci/pcie/aer.c +5 -7
index ec943cee5..98077595a 100644
@@ -1405,13 +1405,11 @@ static pci_ers_result_t aer_root_reset(struct pci_dev *dev)
 	}
 
 	if (type == PCI_EXP_TYPE_RC_EC || type == PCI_EXP_TYPE_RC_END) {
-		if (pcie_has_flr(dev)) {
-			rc = pcie_flr(dev);
-			pci_info(dev, "has been reset (%d)\n", rc);
-		} else {
-			pci_info(dev, "not reset (no FLR support)\n");
-			rc = -ENOTTY;
-		}
+		rc = pcie_reset_flr(dev, 0);
+		if (!rc)
+			pci_info(dev, "has been reset\n");
+		else
+			pci_info(dev, "not reset (no FLR support: %d)\n", rc);
 	} else {
 		rc = pci_bus_error_reset(dev);
 		pci_info(dev, "%s Port link has been reset (%d)\n",
drivers/pci/probe.c +4 -2
index 3a62d09b8..072a3d4dc 100644
@@ -1487,6 +1487,7 @@ void set_pcie_port_type(struct pci_dev *pdev)
 {
 	int pos;
 	u16 reg16;
+	u32 reg32;
 	int type;
 	struct pci_dev *parent;
 
@@ -1497,8 +1498,9 @@ void set_pcie_port_type(struct pci_dev *pdev)
 	pdev->pcie_cap = pos;
 	pci_read_config_word(pdev, pos + PCI_EXP_FLAGS, &reg16);
 	pdev->pcie_flags_reg = reg16;
-	pci_read_config_word(pdev, pos + PCI_EXP_DEVCAP, &reg16);
-	pdev->pcie_mpss = reg16 & PCI_EXP_DEVCAP_PAYLOAD;
+	pci_read_config_dword(pdev, pos + PCI_EXP_DEVCAP, &reg32);
+	pdev->pcie_mpss = reg32 & PCI_EXP_DEVCAP_PAYLOAD;
+	pdev->has_pcie_flr = !!(reg32 & PCI_EXP_DEVCAP_FLR);
 
 	parent = pci_upstream_bridge(pdev);
 	if (!parent)
drivers/pci/quirks.c +3 -6
index d85914afe..f977ba79a 100644
@@ -3819,7 +3819,7 @@ static int nvme_disable_and_flr(struct pci_dev *dev, int probe)
 	u32 cfg;
 
 	if (dev->class != PCI_CLASS_STORAGE_EXPRESS ||
-	    !pcie_has_flr(dev) || !pci_resource_start(dev, 0))
+	    pcie_reset_flr(dev, 1) || !pci_resource_start(dev, 0))
 		return -ENOTTY;
 
 	if (probe)
@@ -3888,13 +3888,10 @@ static int nvme_disable_and_flr(struct pci_dev *dev, int probe)
  */
 static int delay_250ms_after_flr(struct pci_dev *dev, int probe)
 {
-	if (!pcie_has_flr(dev))
-		return -ENOTTY;
+	int ret = pcie_reset_flr(dev, probe);
 
 	if (probe)
-		return 0;
-
-	pcie_flr(dev);
+		return ret;
 
 	msleep(250);
 
include/linux/pci.h +2 -1
index c20211e59..d432428fd 100644
@@ -337,6 +337,7 @@ struct pci_dev {
 	u8		msi_cap;	/* MSI capability offset */
 	u8		msix_cap;	/* MSI-X capability offset */
 	u8		pcie_mpss:3;	/* PCIe Max Payload Size Supported */
+	u8		has_pcie_flr:1; /* PCIe FLR supported */
 	u8		rom_base_reg;	/* Config register controlling ROM */
 	u8		pin;		/* Interrupt pin this device uses */
 	u16		pcie_flags_reg;	/* Cached PCIe Capabilities Register */
@@ -1225,7 +1226,7 @@ u32 pcie_bandwidth_available(struct pci_dev *dev, struct pci_dev **limiting_dev,
 			     enum pci_bus_speed *speed,
 			     enum pcie_link_width *width);
 void pcie_print_link_status(struct pci_dev *dev);
-bool pcie_has_flr(struct pci_dev *dev);
+int pcie_reset_flr(struct pci_dev *dev, int probe);
 int pcie_flr(struct pci_dev *dev);
 int __pci_reset_function_locked(struct pci_dev *dev);
 int pci_reset_function(struct pci_dev *dev);
Signature · 1 line
2.32.0
Reviewed-by on v10 1/8
#2 lore raw

Re: [PATCH v10 1/8] PCI: Add pcie_reset_flr to follow calling convention of other reset methods

Quoted 10 lines · On Fri, 9 Jul 2021 18:08:06 +0530 Amey Narkhede <[email protected]>…

Add has_pcie_flr bitfield in struct pci_dev to indicate support for PCIe FLR to avoid reading PCI_EXP_DEVCAP multiple times.

Currently there is separate function pcie_has_flr() to probe if PCIe FLR is supported by the device which does not match the calling convention followed by reset methods which use second function argument to decide whether to probe or not. Add new function pcie_reset_flr() that follows the calling convention of reset methods.

Quoted 8 lines
---
 drivers/crypto/cavium/nitrox/nitrox_main.c |  4 +-
 drivers/pci/pci.c                          | 59 +++++++++++-----------
 drivers/pci/pcie/aer.c                     | 12 ++---
 drivers/pci/probe.c                        |  6 ++-
 drivers/pci/quirks.c                       |  9 ++--
 include/linux/pci.h                        |  3 +-
 6 files changed, 45 insertions(+), 48 deletions(-)

Looks good to me,

#3 lore raw

Re: [PATCH v10 1/8] PCI: Add pcie_reset_flr to follow calling convention of other reset methods

In reply to Amey Narkhede
Quoted 24 lines · Amey Narkhede

Add has_pcie_flr bitfield in struct pci_dev to indicate support for PCIe FLR to avoid reading PCI_EXP_DEVCAP multiple times.

Currently there is separate function pcie_has_flr() to probe if PCIe FLR is supported by the device which does not match the calling convention followed by reset methods which use second function argument to decide whether to probe or not. Add new function pcie_reset_flr() that follows the calling convention of reset methods.

---
 drivers/crypto/cavium/nitrox/nitrox_main.c |  4 +-
 drivers/pci/pci.c                          | 59 +++++++++++-----------
 drivers/pci/pcie/aer.c                     | 12 ++---
 drivers/pci/probe.c                        |  6 ++-
 drivers/pci/quirks.c                       |  9 ++--
 include/linux/pci.h                        |  3 +-
 6 files changed, 45 insertions(+), 48 deletions(-)
drivers/crypto/cavium/nitrox/nitrox_main.c +0 -0
index facc8e6bc..15d6c8452 100644

drivers/crypto/cavium/nitrox/nitrox_main.c static int nitrox_device_flr(struct pci_dev *pdev) · line 309

Quoted 7 lines
drivers/crypto/cavium/nitrox/nitrox_main.c +1 -3
@@ -306,9 +306,7 @@ static int nitrox_device_flr(struct pci_dev *pdev)
 		return -ENOMEM;
 	}
 
-	/* check flr support */
-	if (pcie_has_flr(pdev))
-		pcie_flr(pdev);
+	pcie_reset_flr(pdev, 0);

I'm not really a fan of exposing the "probe" argument outside drivers/pci/. I think this would be the only occurrence. Is there a way to avoid that?

Can we just make pcie_flr() do the equivalent of pcie_has_flr() internally?

drivers/crypto/cavium/nitrox/nitrox_main.c static int nitrox_device_flr(struct pci_dev *pdev)

Quoted 13 lines
drivers/crypto/cavium/nitrox/nitrox_main.c +2 -5
 static int delay_250ms_after_flr(struct pci_dev *dev, int probe)
 {
-	if (!pcie_has_flr(dev))
-		return -ENOTTY;
+	int ret = pcie_reset_flr(dev, probe);
 
 	if (probe)
-		return 0;
-
-	pcie_flr(dev);
+		return ret;
 
 	msleep(250);

Can we structure this like the following? I think it's easier to understand.

  if (probe)
    return pcie_reset_flr(dev, 1);

  pcie_reset_flr(dev, 0);
  msleep(250);
  return 0;

include/linux/pci.h struct pci_dev { · line 340

Quoted 9 lines
include/linux/pci.h +1 -0
index c20211e59..d432428fd 100644
@@ -337,6 +337,7 @@ struct pci_dev {
 	u8		msi_cap;	/* MSI capability offset */
 	u8		msix_cap;	/* MSI-X capability offset */
 	u8		pcie_mpss:3;	/* PCIe Max Payload Size Supported */
+	u8		has_pcie_flr:1; /* PCIe FLR supported */

Let's add a devcap member instead. Then we can use it for some ASPM-related things as well. We *could* use it to replace pcie_mpss, since that comes from PCI_EXP_DEVCAP, too, but for now I think it's easier to just keep it because it's encoded, and some drivers and quirks use it so it would be a fair amount of work to change that. Example patch below that could become the first in the series.

drivers/pci/pci.c +1 -2
index aacf575c15cf..5a99061ea53a 100644
@@ -4635,8 +4635,7 @@ bool pcie_has_flr(struct pci_dev *dev)
 	if (dev->dev_flags & PCI_DEV_FLAGS_NO_FLR_RESET)
 		return false;
 
-	pcie_capability_read_dword(dev, PCI_EXP_DEVCAP, &cap);
-	return cap & PCI_EXP_DEVCAP_FLR;
+	return dev->devcap & PCI_EXP_DEVCAP_FLR;
 }
 EXPORT_SYMBOL_GPL(pcie_has_flr);
 
drivers/pci/probe.c +2 -2
index 79177ac37880..52ae26bcc68c 100644
@@ -1498,8 +1498,8 @@ void set_pcie_port_type(struct pci_dev *pdev)
 	pdev->pcie_cap = pos;
 	pci_read_config_word(pdev, pos + PCI_EXP_FLAGS, &reg16);
 	pdev->pcie_flags_reg = reg16;
-	pci_read_config_word(pdev, pos + PCI_EXP_DEVCAP, &reg16);
-	pdev->pcie_mpss = reg16 & PCI_EXP_DEVCAP_PAYLOAD;
+	pci_read_config_dword(pdev, pos + PCI_EXP_DEVCAP, &pdev->devcap);
+	pdev->pcie_mpss = pdev->devcap & PCI_EXP_DEVCAP_PAYLOAD;
 
 	parent = pci_upstream_bridge(pdev);
 	if (!parent)
include/linux/pci.h +1 -0
index 540b377ca8f6..294d1c857a57 100644
@@ -334,6 +334,7 @@ struct pci_dev {
 	struct pci_dev  *rcec;          /* Associated RCEC device */
 #endif
 	u8		pcie_cap;	/* PCIe capability offset */
+	u32		devcap;		/* PCIe Device Capabilities */
 	u8		msi_cap;	/* MSI capability offset */
 	u8		msix_cap;	/* MSI-X capability offset */
 	u8		pcie_mpss:3;	/* PCIe Max Payload Size Supported */
#4 lore raw

Re: [PATCH v10 1/8] PCI: Add pcie_reset_flr to follow calling convention of other reset methods

Hi Bjorn,

Quoted 39 lines · Bjorn Helgaas

External email: Use caution opening links or attachments

Quoted 24 lines · Amey Narkhede

Add has_pcie_flr bitfield in struct pci_dev to indicate support for PCIe FLR to avoid reading PCI_EXP_DEVCAP multiple times.

Currently there is separate function pcie_has_flr() to probe if PCIe FLR is supported by the device which does not match the calling convention followed by reset methods which use second function argument to decide whether to probe or not. Add new function pcie_reset_flr() that follows the calling convention of reset methods.

---
 drivers/crypto/cavium/nitrox/nitrox_main.c |  4 +-
 drivers/pci/pci.c                          | 59 +++++++++++-----------
 drivers/pci/pcie/aer.c                     | 12 ++---
 drivers/pci/probe.c                        |  6 ++-
 drivers/pci/quirks.c                       |  9 ++--
 include/linux/pci.h                        |  3 +-
 6 files changed, 45 insertions(+), 48 deletions(-)
drivers/crypto/cavium/nitrox/nitrox_main.c +0 -0
index facc8e6bc..15d6c8452 100644

drivers/crypto/cavium/nitrox/nitrox_main.c static int nitrox_device_flr(struct pci_dev *pdev) · line 309

Quoted 7 lines
drivers/crypto/cavium/nitrox/nitrox_main.c +1 -3
@@ -306,9 +306,7 @@ static int nitrox_device_flr(struct pci_dev *pdev)
              return -ENOMEM;
      }
 
-     /* check flr support */
-     if (pcie_has_flr(pdev))
-             pcie_flr(pdev);
+     pcie_reset_flr(pdev, 0);

I'm not really a fan of exposing the "probe" argument outside drivers/pci/. I think this would be the only occurrence. Is there a way to avoid that?

Quoted 3 lines

Can we just make pcie_flr() do the equivalent of pcie_has_flr() internally?

I like your suggestion don't change the existing definition of pcie_has_flr()/pcie_flr() and define a new function pcie_reset_flr() to fit into the reset framework. This way no need to modify logic/drivers outside of driver/pci/xxx.

int pcie_reset_flr(struct pci_dev *dev, int probe) {
        if (!pcie_has_flr(dev))
                return -ENOTTY;

        if (probe)
                return 0;

        return pcie_flr(dev);
}

And add a new patch to begging of the series for caching 'devcap' in pci_dev structure.

include/linux/pci.h +0 -0
@@ -333,6 +333,7 @@ struct pci_dev {

        struct rcec_ea  *rcec_ea;       /* RCEC cached endpoint association */ struct pci_dev  *rcec;          /* Associated RCEC device */ #endif
+       u32             devcap;         /* Cached PCIe device capabilities */ u8              pcie_cap;       /* PCIe capability offset */ u8              msi_cap;        /* MSI capability offset */ u8              msix_cap;       /* MSI-X capability offset */

drivers/pci/pci.c +0 -0
@@ -31,6 +31,7 @@

 #include <linux/vmalloc.h>
 #include <asm/dma.h>
 #include <linux/aer.h>
+#include <linux/bitfield.h>
 #include "pci.h"

 DEFINE_MUTEX(pci_slot_mutex);
@@ -4630,13 +4631,10 @@ EXPORT_SYMBOL(pci_wait_for_pending_transaction); */
 bool pcie_has_flr(struct pci_dev *dev)
 {
-       u32 cap;
-
        if (dev->dev_flags & PCI_DEV_FLAGS_NO_FLR_RESET) return false;

-       pcie_capability_read_dword(dev, PCI_EXP_DEVCAP, &cap); -       return cap & PCI_EXP_DEVCAP_FLR;
+       return !!FIELD_GET(PCI_EXP_DEVCAP_FLR, dev->devcap); }

drivers/pci/probe.c +0 -0
@@ -19,6 +19,7 @@

 #include <linux/hypervisor.h>
 #include <linux/irqdomain.h>
 #include <linux/pm_runtime.h>
+#include <linux/bitfield.h>
 #include "pci.h"

 #define CARDBUS_LATENCY_TIMER  176     /* secondary latency timer */ @@ -1498,8 +1499,8 @@ void set_pcie_port_type(struct pci_dev *pdev) pdev->pcie_cap = pos;
        pci_read_config_word(pdev, pos + PCI_EXP_FLAGS, &reg16); pdev->pcie_flags_reg = reg16;
-       pci_read_config_word(pdev, pos + PCI_EXP_DEVCAP, &reg16); -       pdev->pcie_mpss = reg16 & PCI_EXP_DEVCAP_PAYLOAD;
+       pci_read_config_dword(pdev, pos + PCI_EXP_DEVCAP, &pdev->devcap); +       pdev->pcie_mpss = FIELD_GET(PCI_EXP_DEVCAP_PAYLOAD, pdev->devcap);

#5 lore raw

Re: [PATCH v10 1/8] PCI: Add pcie_reset_flr to follow calling convention of other reset methods

Quoted 50 lines · Shanker R Donthineni
Quoted 36 lines · Bjorn Helgaas
Quoted 24 lines · Amey Narkhede

Add has_pcie_flr bitfield in struct pci_dev to indicate support for PCIe FLR to avoid reading PCI_EXP_DEVCAP multiple times.

Currently there is separate function pcie_has_flr() to probe if PCIe FLR is supported by the device which does not match the calling convention followed by reset methods which use second function argument to decide whether to probe or not. Add new function pcie_reset_flr() that follows the calling convention of reset methods.

---
 drivers/crypto/cavium/nitrox/nitrox_main.c |  4 +-
 drivers/pci/pci.c                          | 59 +++++++++++-----------
 drivers/pci/pcie/aer.c                     | 12 ++---
 drivers/pci/probe.c                        |  6 ++-
 drivers/pci/quirks.c                       |  9 ++--
 include/linux/pci.h                        |  3 +-
 6 files changed, 45 insertions(+), 48 deletions(-)
drivers/crypto/cavium/nitrox/nitrox_main.c +0 -0
index facc8e6bc..15d6c8452 100644

drivers/crypto/cavium/nitrox/nitrox_main.c static int nitrox_device_flr(struct pci_dev *pdev) · line 309

Quoted 7 lines
drivers/crypto/cavium/nitrox/nitrox_main.c +1 -3
@@ -306,9 +306,7 @@ static int nitrox_device_flr(struct pci_dev *pdev)
              return -ENOMEM;
      }
 
-     /* check flr support */
-     if (pcie_has_flr(pdev))
-             pcie_flr(pdev);
+     pcie_reset_flr(pdev, 0);

I'm not really a fan of exposing the "probe" argument outside drivers/pci/. I think this would be the only occurrence. Is there a way to avoid that?

Quoted 3 lines

Can we just make pcie_flr() do the equivalent of pcie_has_flr() internally?

I like your suggestion don't change the existing definition of pcie_has_flr()/pcie_flr() and define a new function pcie_reset_flr() to fit into the reset framework. This way no need to modify logic/drivers outside of driver/pci/xxx.

int pcie_reset_flr(struct pci_dev *dev, int probe) {
        if (!pcie_has_flr(dev))
                return -ENOTTY;

Quoted 5 lines

        if (probe)
                return 0;

        return pcie_flr(dev);
}

Can't remember the whole context of this in the series, but I assume this would be static?

Quoted 27 lines

And add a new patch to begging of the series for caching 'devcap' in pci_dev structure.

include/linux/pci.h +0 -0
@@ -333,6 +333,7 @@ struct pci_dev {

        struct rcec_ea  *rcec_ea;       /* RCEC cached endpoint association */ struct pci_dev  *rcec;          /* Associated RCEC device */ #endif
+       u32             devcap;         /* Cached PCIe device capabilities */ u8              pcie_cap;       /* PCIe capability offset */ u8              msi_cap;        /* MSI capability offset */ u8              msix_cap;       /* MSI-X capability offset */

drivers/pci/pci.c +1 -0
@@ -31,6 +31,7 @@
  #include <linux/vmalloc.h>
  #include <asm/dma.h>
  #include <linux/aer.h>
+#include <linux/bitfield.h>
  #include "pci.h"
 
  DEFINE_MUTEX(pci_slot_mutex);
@@ -4630,13 +4631,10 @@ EXPORT_SYMBOL(pci_wait_for_pending_transaction);
   */

drivers/pci/pci.c EXPORT_SYMBOL(pci_wait_for_pending_transaction); · line 4637

Quoted 10 lines
drivers/pci/pci.c +1 -4
  bool pcie_has_flr(struct pci_dev *dev)
  {
-       u32 cap;
-
         if (dev->dev_flags & PCI_DEV_FLAGS_NO_FLR_RESET)
                 return false;
 
-       pcie_capability_read_dword(dev, PCI_EXP_DEVCAP, &cap);
-       return cap & PCI_EXP_DEVCAP_FLR;
+       return !!FIELD_GET(PCI_EXP_DEVCAP_FLR, dev->devcap);

Nice, thanks for reminding me of FIELD_GET(). I like how that works without having to #define *_SHIFT values. I personally don't care for "!!" and would probably write something like:

  return FIELD_GET(PCI_EXP_DEVCAP_FLR, dev->devcap) == 1;
Quoted 21 lines
drivers/pci/pci.c +0 -0
  }
 
drivers/pci/probe.c +3 -2
@@ -19,6 +19,7 @@
  #include <linux/hypervisor.h>
  #include <linux/irqdomain.h>
  #include <linux/pm_runtime.h>
+#include <linux/bitfield.h>
  #include "pci.h"
 
  #define CARDBUS_LATENCY_TIMER  176     /* secondary latency timer */
@@ -1498,8 +1499,8 @@ void set_pcie_port_type(struct pci_dev *pdev)
         pdev->pcie_cap = pos;
         pci_read_config_word(pdev, pos + PCI_EXP_FLAGS, &reg16);
         pdev->pcie_flags_reg = reg16;
-       pci_read_config_word(pdev, pos + PCI_EXP_DEVCAP, &reg16);
-       pdev->pcie_mpss = reg16 & PCI_EXP_DEVCAP_PAYLOAD;
+       pci_read_config_dword(pdev, pos + PCI_EXP_DEVCAP, &pdev->devcap);
+       pdev->pcie_mpss = FIELD_GET(PCI_EXP_DEVCAP_PAYLOAD, pdev->devcap);
 
#6 lore raw

Re: [PATCH v10 1/8] PCI: Add pcie_reset_flr to follow calling convention of other reset methods

Hi Bjorn,

Quoted 54 lines · Bjorn Helgaas

External email: Use caution opening links or attachments

Quoted 50 lines · Shanker R Donthineni
Quoted 36 lines · Bjorn Helgaas
Quoted 24 lines · Amey Narkhede

Add has_pcie_flr bitfield in struct pci_dev to indicate support for PCIe FLR to avoid reading PCI_EXP_DEVCAP multiple times.

Currently there is separate function pcie_has_flr() to probe if PCIe FLR is supported by the device which does not match the calling convention followed by reset methods which use second function argument to decide whether to probe or not. Add new function pcie_reset_flr() that follows the calling convention of reset methods.

---
 drivers/crypto/cavium/nitrox/nitrox_main.c |  4 +-
 drivers/pci/pci.c                          | 59 +++++++++++-----------
 drivers/pci/pcie/aer.c                     | 12 ++---
 drivers/pci/probe.c                        |  6 ++-
 drivers/pci/quirks.c                       |  9 ++--
 include/linux/pci.h                        |  3 +-
 6 files changed, 45 insertions(+), 48 deletions(-)
drivers/crypto/cavium/nitrox/nitrox_main.c +0 -0
index facc8e6bc..15d6c8452 100644

drivers/crypto/cavium/nitrox/nitrox_main.c static int nitrox_device_flr(struct pci_dev *pdev) · line 309

Quoted 7 lines
drivers/crypto/cavium/nitrox/nitrox_main.c +1 -3
@@ -306,9 +306,7 @@ static int nitrox_device_flr(struct pci_dev *pdev)
              return -ENOMEM;
      }
 
-     /* check flr support */
-     if (pcie_has_flr(pdev))
-             pcie_flr(pdev);
+     pcie_reset_flr(pdev, 0);

I'm not really a fan of exposing the "probe" argument outside drivers/pci/. I think this would be the only occurrence. Is there a way to avoid that?

Quoted 3 lines

Can we just make pcie_flr() do the equivalent of pcie_has_flr() internally?

I like your suggestion don't change the existing definition of pcie_has_flr()/pcie_flr() and define a new function pcie_reset_flr() to fit into the reset framework. This way no need to modify logic/drivers outside of driver/pci/xxx.

int pcie_reset_flr(struct pci_dev *dev, int probe)

{
        if (!pcie_has_flr(dev))
                return -ENOTTY;
Quoted 7 lines
Quoted 5 lines
        if (probe)
                return 0;

        return pcie_flr(dev);
}

Can't remember the whole context of this in the series, but I assume this would be static?

It should be static since it's referenced in driver/pci/qrirk.c and aer.c.

Quoted 34 lines
Quoted 34 lines

And add a new patch to begging of the series for caching 'devcap' in pci_dev structure.

include/linux/pci.h +1 -0
@@ -333,6 +333,7 @@ struct pci_dev {
        struct rcec_ea  *rcec_ea;       /* RCEC cached endpoint association */
        struct pci_dev  *rcec;          /* Associated RCEC device */
 #endif
+       u32             devcap;         /* Cached PCIe device capabilities */
        u8              pcie_cap;       /* PCIe capability offset */
        u8              msi_cap;        /* MSI capability offset */
        u8              msix_cap;       /* MSI-X capability offset */
drivers/pci/pci.c +1 -2
@@ -31,6 +31,7 @@
 #include <linux/vmalloc.h>
 #include <asm/dma.h>
 #include <linux/aer.h>
+#include <linux/bitfield.h>
 #include "pci.h"
 
 DEFINE_MUTEX(pci_slot_mutex);
@@ -4630,13 +4631,10 @@ EXPORT_SYMBOL(pci_wait_for_pending_transaction);
  */
 bool pcie_has_flr(struct pci_dev *dev)
 {
-       u32 cap;
-
        if (dev->dev_flags & PCI_DEV_FLAGS_NO_FLR_RESET)
                return false;
 
Quoted 8 lines

drivers/pci/pci.c EXPORT_SYMBOL(pci_wait_for_pending_transaction); · line 4637

Quoted 3 lines
drivers/pci/pci.c +1 -2
-       pcie_capability_read_dword(dev, PCI_EXP_DEVCAP, &cap);
-       return cap & PCI_EXP_DEVCAP_FLR;
+       return !!FIELD_GET(PCI_EXP_DEVCAP_FLR, dev->devcap);

Nice, thanks for reminding me of FIELD_GET(). I like how that works without having to #define *_SHIFT values. I personally don't care for "!!" and would probably write something like:

  return FIELD_GET(PCI_EXP_DEVCAP_FLR, dev->devcap) == 1;

Both are same since FLR is a single bit value.

Quoted 21 lines
Quoted 21 lines
drivers/pci/pci.c +0 -0
 }
 
drivers/pci/probe.c +3 -2
@@ -19,6 +19,7 @@
 #include <linux/hypervisor.h>
 #include <linux/irqdomain.h>
 #include <linux/pm_runtime.h>
+#include <linux/bitfield.h>
 #include "pci.h"
 
 #define CARDBUS_LATENCY_TIMER  176     /* secondary latency timer */
@@ -1498,8 +1499,8 @@ void set_pcie_port_type(struct pci_dev *pdev)
        pdev->pcie_cap = pos;
        pci_read_config_word(pdev, pos + PCI_EXP_FLAGS, &reg16);
        pdev->pcie_flags_reg = reg16;
-       pci_read_config_word(pdev, pos + PCI_EXP_DEVCAP, &reg16);
-       pdev->pcie_mpss = reg16 & PCI_EXP_DEVCAP_PAYLOAD;
+       pci_read_config_dword(pdev, pos + PCI_EXP_DEVCAP, &pdev->devcap);
+       pdev->pcie_mpss = FIELD_GET(PCI_EXP_DEVCAP_PAYLOAD, pdev->devcap);
 
#7 lore raw

Re: [PATCH v10 1/8] PCI: Add pcie_reset_flr to follow calling convention of other reset methods

Quoted 60 lines · Shanker R Donthineni

External email: Use caution opening links or attachments

Hi Bjorn,

Quoted 54 lines · Bjorn Helgaas

External email: Use caution opening links or attachments

Quoted 50 lines · Shanker R Donthineni
Quoted 36 lines · Bjorn Helgaas
Quoted 24 lines · Amey Narkhede

Add has_pcie_flr bitfield in struct pci_dev to indicate support for PCIe FLR to avoid reading PCI_EXP_DEVCAP multiple times.

Currently there is separate function pcie_has_flr() to probe if PCIe FLR is supported by the device which does not match the calling convention followed by reset methods which use second function argument to decide whether to probe or not. Add new function pcie_reset_flr() that follows the calling convention of reset methods.

---
 drivers/crypto/cavium/nitrox/nitrox_main.c |  4 +-
 drivers/pci/pci.c                          | 59 +++++++++++-----------
 drivers/pci/pcie/aer.c                     | 12 ++---
 drivers/pci/probe.c                        |  6 ++-
 drivers/pci/quirks.c                       |  9 ++--
 include/linux/pci.h                        |  3 +-
 6 files changed, 45 insertions(+), 48 deletions(-)
drivers/crypto/cavium/nitrox/nitrox_main.c +0 -0
index facc8e6bc..15d6c8452 100644

drivers/crypto/cavium/nitrox/nitrox_main.c static int nitrox_device_flr(struct pci_dev *pdev) · line 309

Quoted 7 lines
drivers/crypto/cavium/nitrox/nitrox_main.c +1 -3
@@ -306,9 +306,7 @@ static int nitrox_device_flr(struct pci_dev *pdev)
              return -ENOMEM;
      }
 
-     /* check flr support */
-     if (pcie_has_flr(pdev))
-             pcie_flr(pdev);
+     pcie_reset_flr(pdev, 0);

I'm not really a fan of exposing the "probe" argument outside drivers/pci/. I think this would be the only occurrence. Is there a way to avoid that?

Quoted 3 lines

Can we just make pcie_flr() do the equivalent of pcie_has_flr() internally?

I like your suggestion don't change the existing definition of pcie_has_flr()/pcie_flr() and define a new function pcie_reset_flr() to fit into the reset framework. This way no need to modify logic/drivers outside of driver/pci/xxx.

int pcie_reset_flr(struct pci_dev *dev, int probe)

{
        if (!pcie_has_flr(dev))
                return -ENOTTY;
Quoted 9 lines
Quoted 7 lines
Quoted 5 lines
        if (probe)
                return 0;

        return pcie_flr(dev);
}

Can't remember the whole context of this in the series, but I assume this would be static?

It should be static since it's referenced in driver/pci/qrirk.c and aer.c.

Sorry it should not be static since it's referenced in driver/pci/quirk.c and aer.c

#8 lore raw

Re: [PATCH v10 1/8] PCI: Add pcie_reset_flr to follow calling convention of other reset methods

In reply to Shanker R Donthineni
Quoted 11 lines · Shanker R Donthineni
Quoted 10 lines · Bjorn Helgaas
Quoted 8 lines · Shanker R Donthineni
Quoted 3 lines · Bjorn Helgaas
Quoted 2 lines · Amey Narkhede

Add has_pcie_flr bitfield in struct pci_dev to indicate support for PCIe FLR to avoid reading PCI_EXP_DEVCAP multiple times.

(quoted diff) +1 -2
-       pcie_capability_read_dword(dev, PCI_EXP_DEVCAP, &cap);
-       return cap & PCI_EXP_DEVCAP_FLR;
+       return !!FIELD_GET(PCI_EXP_DEVCAP_FLR, dev->devcap);
Quoted 7 lines
Quoted 5 lines

Nice, thanks for reminding me of FIELD_GET(). I like how that works without having to #define *_SHIFT values. I personally don't care for "!!" and would probably write something like:

  return FIELD_GET(PCI_EXP_DEVCAP_FLR, dev->devcap) == 1;

Both are same since FLR is a single bit value.

Right; this is a style preference, not a correctness question.

#9 lore raw

PATCH v10 2/8 PCI: Add new array for keeping track of ordering of reset methods

In reply to Amey Narkhede

Introduce a new array reset_methods in struct pci_dev to keep track of reset mechanisms supported by the device and their ordering.

Also refactor probing and reset functions to take advantage of calling convention of reset functions.

---
 drivers/pci/pci.c   | 92 ++++++++++++++++++++++++++-------------------
 drivers/pci/pci.h   |  9 ++++-
 drivers/pci/probe.c |  5 +--
 include/linux/pci.h |  7 ++++
 4 files changed, 70 insertions(+), 43 deletions(-)
drivers/pci/pci.c +53 -39
index fefa6d7b3..42440cb10 100644
@@ -72,6 +72,14 @@ static void pci_dev_d3_sleep(struct pci_dev *dev)
 		msleep(delay);
 }
 
+int pci_reset_supported(struct pci_dev *dev)
+{
+	u8 null_reset_methods[PCI_NUM_RESET_METHODS] = { 0 };
+
+	return memcmp(null_reset_methods,
+		      dev->reset_methods, sizeof(null_reset_methods));
+}
+
 #ifdef CONFIG_PCI_DOMAINS
 int pci_domains_supported = 1;
 #endif
@@ -5104,6 +5112,15 @@ static void pci_dev_restore(struct pci_dev *dev)
 		err_handler->reset_done(dev);
 }
 
+const struct pci_reset_fn_method pci_reset_fn_methods[] = {
+	{ },
+	{ &pci_dev_specific_reset, .name = "device_specific" },
+	{ &pcie_reset_flr, .name = "flr" },
+	{ &pci_af_flr, .name = "af_flr" },
+	{ &pci_pm_reset, .name = "pm" },
+	{ &pci_reset_bus_function, .name = "bus" },
+};
+
 /**
  * __pci_reset_function_locked - reset a PCI device function while holding
  * the @dev mutex lock.
@@ -5126,65 +5143,62 @@ static void pci_dev_restore(struct pci_dev *dev)
  */
 int __pci_reset_function_locked(struct pci_dev *dev)
 {
-	int rc;
+	int i, m, rc = -ENOTTY;
 
 	might_sleep();
 
 	/*
-	 * A reset method returns -ENOTTY if it doesn't support this device
-	 * and we should try the next method.
+	 * A reset method returns -ENOTTY if it doesn't support this device and
+	 * we should try the next method.
 	 *
-	 * If it returns 0 (success), we're finished.  If it returns any
-	 * other error, we're also finished: this indicates that further
-	 * reset mechanisms might be broken on the device.
+	 * If it returns 0 (success), we're finished.  If it returns any other
+	 * error, we're also finished: this indicates that further reset
+	 * mechanisms might be broken on the device.
 	 */
-	rc = pci_dev_specific_reset(dev, 0);
-	if (rc != -ENOTTY)
-		return rc;
-	rc = pcie_reset_flr(dev, 0);
-	if (rc != -ENOTTY)
-		return rc;
-	rc = pci_af_flr(dev, 0);
-	if (rc != -ENOTTY)
-		return rc;
-	rc = pci_pm_reset(dev, 0);
-	if (rc != -ENOTTY)
-		return rc;
-	return pci_reset_bus_function(dev, 0);
+	for (i = 0; i <  PCI_NUM_RESET_METHODS && (m = dev->reset_methods[i]); i++) {
+		rc = pci_reset_fn_methods[m].reset_fn(dev, 0);
+		if (!rc)
+			return 0;
+		if (rc != -ENOTTY)
+			return rc;
+	}
+
+	return -ENOTTY;
 }
 EXPORT_SYMBOL_GPL(__pci_reset_function_locked);
 
 /**
- * pci_probe_reset_function - check whether the device can be safely reset
- * @dev: PCI device to reset
+ * pci_init_reset_methods - check whether device can be safely reset
+ * and store supported reset mechanisms.
+ * @dev: PCI device to check for reset mechanisms
  *
  * Some devices allow an individual function to be reset without affecting
  * other functions in the same device.  The PCI device must be responsive
- * to PCI config space in order to use this function.
+ * to reads and writes to its PCI config space in order to use this function.
  *
- * Returns 0 if the device function can be reset or negative if the
- * device doesn't support resetting a single function.
+ * Stores reset mechanisms supported by device in reset_methods byte array
+ * which is a member of struct pci_dev.
  */
-int pci_probe_reset_function(struct pci_dev *dev)
+void pci_init_reset_methods(struct pci_dev *dev)
 {
-	int rc;
+	int i, n, rc;
+	u8 reset_methods[PCI_NUM_RESET_METHODS] = { 0 };
+
+	n = 0;
+
+	BUILD_BUG_ON(ARRAY_SIZE(pci_reset_fn_methods) != PCI_NUM_RESET_METHODS);
 
 	might_sleep();
 
-	rc = pci_dev_specific_reset(dev, 1);
-	if (rc != -ENOTTY)
-		return rc;
-	rc = pcie_reset_flr(dev, 1);
-	if (rc != -ENOTTY)
-		return rc;
-	rc = pci_af_flr(dev, 1);
-	if (rc != -ENOTTY)
-		return rc;
-	rc = pci_pm_reset(dev, 1);
-	if (rc != -ENOTTY)
-		return rc;
+	for (i = 1; i < PCI_NUM_RESET_METHODS; i++) {
+		rc = pci_reset_fn_methods[i].reset_fn(dev, 1);
+		if (!rc)
+			reset_methods[n++] = i;
+		else if (rc != -ENOTTY)
+			break;
+	}
 
-	return pci_reset_bus_function(dev, 1);
+	memcpy(dev->reset_methods, reset_methods, sizeof(reset_methods));
 }
 
 /**
drivers/pci/pci.h +8 -1
index 37c913bbc..db1ad94e7 100644
@@ -33,7 +33,8 @@ enum pci_mmap_api {
 int pci_mmap_fits(struct pci_dev *pdev, int resno, struct vm_area_struct *vmai,
 		  enum pci_mmap_api mmap_api);
 
-int pci_probe_reset_function(struct pci_dev *dev);
+int pci_reset_supported(struct pci_dev *dev);
+void pci_init_reset_methods(struct pci_dev *dev);
 int pci_bridge_secondary_bus_reset(struct pci_dev *dev);
 int pci_bus_error_reset(struct pci_dev *dev);
 
@@ -606,6 +607,12 @@ struct pci_dev_reset_methods {
 	int (*reset)(struct pci_dev *dev, int probe);
 };
 
+struct pci_reset_fn_method {
+	int (*reset_fn)(struct pci_dev *pdev, int probe);
+	char *name;
+};
+
+extern const struct pci_reset_fn_method pci_reset_fn_methods[];
 #ifdef CONFIG_PCI_QUIRKS
 int pci_dev_specific_reset(struct pci_dev *dev, int probe);
 #else
drivers/pci/probe.c +2 -3
index 072a3d4dc..bc4af914a 100644
@@ -2406,9 +2406,8 @@ static void pci_init_capabilities(struct pci_dev *dev)
 	pci_rcec_init(dev);		/* Root Complex Event Collector */
 
 	pcie_report_downtraining(dev);
-
-	if (pci_probe_reset_function(dev) == 0)
-		dev->reset_fn = 1;
+	pci_init_reset_methods(dev);
+	dev->reset_fn = pci_reset_supported(dev);
 }
 
 /*
include/linux/pci.h +7 -0
index d432428fd..9f3e85f33 100644
@@ -49,6 +49,9 @@
 			       PCI_STATUS_SIG_TARGET_ABORT | \
 			       PCI_STATUS_PARITY)
 
+/* Number of reset methods used in pci_reset_fn_methods array in pci.c */
+#define PCI_NUM_RESET_METHODS 6
+
 /*
  * The PCI interface treats multi-function devices as independent
  * devices.  The slot/function address of each device is encoded
@@ -506,6 +509,10 @@ struct pci_dev {
 	char		*driver_override; /* Driver name to force a match */
 
 	unsigned long	priv_flags;	/* Private flags for the PCI driver */
+	/*
+	 * See pci_reset_fn_methods array in pci.c for ordering.
+	 */
+	u8 reset_methods[PCI_NUM_RESET_METHODS];	/* Reset methods ordered by priority */
 };
 
 static inline struct pci_dev *pci_physfn(struct pci_dev *dev)
Signature · 1 line
2.32.0
#10 lore raw

Re: [PATCH v10 2/8] PCI: Add new array for keeping track of ordering of reset methods

Quoted 21 lines · Amey Narkhede

Introduce a new array reset_methods in struct pci_dev to keep track of reset mechanisms supported by the device and their ordering.

Also refactor probing and reset functions to take advantage of calling convention of reset functions.

---
 drivers/pci/pci.c   | 92 ++++++++++++++++++++++++++-------------------
 drivers/pci/pci.h   |  9 ++++-
 drivers/pci/probe.c |  5 +--
 include/linux/pci.h |  7 ++++
 4 files changed, 70 insertions(+), 43 deletions(-)
drivers/pci/pci.c +0 -0
index fefa6d7b3..42440cb10 100644

drivers/pci/pci.c static void pci_dev_d3_sleep(struct pci_dev *dev) · line 80

Quoted 9 lines
drivers/pci/pci.c +6 -0
@@ -72,6 +72,14 @@ static void pci_dev_d3_sleep(struct pci_dev *dev)
 		msleep(delay);
 }
 
+int pci_reset_supported(struct pci_dev *dev)
+{
+	u8 null_reset_methods[PCI_NUM_RESET_METHODS] = { 0 };
+
+	return memcmp(null_reset_methods,
+		      dev->reset_methods, sizeof(null_reset_methods));

I think "return dev->reset_methods[0] != 0;" is sufficient, isn't it?

Quoted 7 lines
drivers/pci/pci.c +2 -0
+}
+
 #ifdef CONFIG_PCI_DOMAINS
 int pci_domains_supported = 1;
 #endif
@@ -5104,6 +5112,15 @@ static void pci_dev_restore(struct pci_dev *dev)
 		err_handler->reset_done(dev);

drivers/pci/pci.c static void pci_dev_restore(struct pci_dev *dev) · line 5122

Quoted 10 lines
drivers/pci/pci.c +8 -0
 }
 
+const struct pci_reset_fn_method pci_reset_fn_methods[] = {
+	{ },
+	{ &pci_dev_specific_reset, .name = "device_specific" },
+	{ &pcie_reset_flr, .name = "flr" },
+	{ &pci_af_flr, .name = "af_flr" },
+	{ &pci_pm_reset, .name = "pm" },
+	{ &pci_reset_bus_function, .name = "bus" },
+};

No need for "&" before the function names.

This should be static until it's needed outside this file. You can remove the "static" in the patch that adds the use in another file.

Looks like the only use is in pci-sysfs.c. I think it might be better to move the pci_dev_reset_method_attr_group and related stuff here to pci.c and add an extern for it, as is done for aspm_ctrl_attr_group.

I know there's some trick that relies on the first element being empty, and it's probably worth a one-line comment there to point it out. Maybe something like this?

/* dev->reset_methods[] is a 0-terminated list of indices into this */

Quoted 29 lines
drivers/pci/pci.c +6 -10
 /**
  * __pci_reset_function_locked - reset a PCI device function while holding
  * the @dev mutex lock.
@@ -5126,65 +5143,62 @@ static void pci_dev_restore(struct pci_dev *dev)
  */
 int __pci_reset_function_locked(struct pci_dev *dev)
 {
-	int rc;
+	int i, m, rc = -ENOTTY;
 
 	might_sleep();
 
 	/*
-	 * A reset method returns -ENOTTY if it doesn't support this device
-	 * and we should try the next method.
+	 * A reset method returns -ENOTTY if it doesn't support this device and
+	 * we should try the next method.
 	 *
-	 * If it returns 0 (success), we're finished.  If it returns any
-	 * other error, we're also finished: this indicates that further
-	 * reset mechanisms might be broken on the device.
+	 * If it returns 0 (success), we're finished.  If it returns any other
+	 * error, we're also finished: this indicates that further reset
+	 * mechanisms might be broken on the device.
 	 */
-	rc = pci_dev_specific_reset(dev, 0);
-	if (rc != -ENOTTY)
-		return rc;
-	rc = pcie_reset_flr(dev, 0);

drivers/pci/pci.c static void pci_dev_restore(struct pci_dev *dev) · line 5158

Quoted 10 lines
drivers/pci/pci.c +1 -9
-	if (rc != -ENOTTY)
-		return rc;
-	rc = pci_af_flr(dev, 0);
-	if (rc != -ENOTTY)
-		return rc;
-	rc = pci_pm_reset(dev, 0);
-	if (rc != -ENOTTY)
-		return rc;
-	return pci_reset_bus_function(dev, 0);
+	for (i = 0; i <  PCI_NUM_RESET_METHODS && (m = dev->reset_methods[i]); i++) {

I'm not a huge fan of using assignments as a loop condition because it's a little subtle. Is there a way to restructure this, e.g.,

  for (i = 0; i < PCI_NUM_RESET_METHODS; i++) {
    m = dev->reset_methods[i];
    if (!m)
      return -ENOTTY;
    rc = pci_reset_fn_methods[m].reset_fn(dev, 0);
Quoted 12 lines
drivers/pci/pci.c +8 -0
+		rc = pci_reset_fn_methods[m].reset_fn(dev, 0);
+		if (!rc)
+			return 0;
+		if (rc != -ENOTTY)
+			return rc;
+	}
+
+	return -ENOTTY;
 }
 EXPORT_SYMBOL_GPL(__pci_reset_function_locked);
 
 /**

drivers/pci/pci.c static void pci_dev_restore(struct pci_dev *dev) · line 5177

Quoted 10 lines
drivers/pci/pci.c +4 -3
- * pci_probe_reset_function - check whether the device can be safely reset
- * @dev: PCI device to reset
+ * pci_init_reset_methods - check whether device can be safely reset
+ * and store supported reset mechanisms.
+ * @dev: PCI device to check for reset mechanisms
  *
  * Some devices allow an individual function to be reset without affecting
  * other functions in the same device.  The PCI device must be responsive
- * to PCI config space in order to use this function.
+ * to reads and writes to its PCI config space in order to use this function.

Since you're reworking this comment anyway, maybe update the "must be responsive to PCI config space" comment to explicitly say the device must be in D0-D3hot.

drivers/pci/pci.c static void pci_dev_restore(struct pci_dev *dev) · line 5187

Quoted 13 lines
drivers/pci/pci.c +7 -4
- * Returns 0 if the device function can be reset or negative if the
- * device doesn't support resetting a single function.
+ * Stores reset mechanisms supported by device in reset_methods byte array
+ * which is a member of struct pci_dev.
  */
-int pci_probe_reset_function(struct pci_dev *dev)
+void pci_init_reset_methods(struct pci_dev *dev)
 {
-	int rc;
+	int i, n, rc;
+	u8 reset_methods[PCI_NUM_RESET_METHODS] = { 0 };
+
+	n = 0;

Move this init down to just before the loop that uses it.

Quoted 10 lines
drivers/pci/pci.c +1 -6
+	BUILD_BUG_ON(ARRAY_SIZE(pci_reset_fn_methods) != PCI_NUM_RESET_METHODS);
 
 	might_sleep();
 
-	rc = pci_dev_specific_reset(dev, 1);
-	if (rc != -ENOTTY)
-		return rc;
-	rc = pcie_reset_flr(dev, 1);
-	if (rc != -ENOTTY)
-		return rc;

drivers/pci/pci.c static void pci_dev_restore(struct pci_dev *dev) · line 5196

Quoted 10 lines
drivers/pci/pci.c +4 -6
-	rc = pci_af_flr(dev, 1);
-	if (rc != -ENOTTY)
-		return rc;
-	rc = pci_pm_reset(dev, 1);
-	if (rc != -ENOTTY)
-		return rc;
+	for (i = 1; i < PCI_NUM_RESET_METHODS; i++) {
+		rc = pci_reset_fn_methods[i].reset_fn(dev, 1);
+		if (!rc)
+			reset_methods[n++] = i;

Why do we need this local reset_methods[] array? Can we just fill in dev->reset_methods[] directly and skip the memcpy() below?

Quoted 80 lines
drivers/pci/pci.c +4 -1
+		else if (rc != -ENOTTY)
+			break;
+	}
 
-	return pci_reset_bus_function(dev, 1);
+	memcpy(dev->reset_methods, reset_methods, sizeof(reset_methods));
 }
 
 /**
drivers/pci/pci.h +8 -1
index 37c913bbc..db1ad94e7 100644
@@ -33,7 +33,8 @@ enum pci_mmap_api {
 int pci_mmap_fits(struct pci_dev *pdev, int resno, struct vm_area_struct *vmai,
 		  enum pci_mmap_api mmap_api);
 
-int pci_probe_reset_function(struct pci_dev *dev);
+int pci_reset_supported(struct pci_dev *dev);
+void pci_init_reset_methods(struct pci_dev *dev);
 int pci_bridge_secondary_bus_reset(struct pci_dev *dev);
 int pci_bus_error_reset(struct pci_dev *dev);
 
@@ -606,6 +607,12 @@ struct pci_dev_reset_methods {
 	int (*reset)(struct pci_dev *dev, int probe);
 };
 
+struct pci_reset_fn_method {
+	int (*reset_fn)(struct pci_dev *pdev, int probe);
+	char *name;
+};
+
+extern const struct pci_reset_fn_method pci_reset_fn_methods[];
 #ifdef CONFIG_PCI_QUIRKS
 int pci_dev_specific_reset(struct pci_dev *dev, int probe);
 #else
drivers/pci/probe.c +2 -3
index 072a3d4dc..bc4af914a 100644
@@ -2406,9 +2406,8 @@ static void pci_init_capabilities(struct pci_dev *dev)
 	pci_rcec_init(dev);		/* Root Complex Event Collector */
 
 	pcie_report_downtraining(dev);
-
-	if (pci_probe_reset_function(dev) == 0)
-		dev->reset_fn = 1;
+	pci_init_reset_methods(dev);
+	dev->reset_fn = pci_reset_supported(dev);
 }
 
 /*
include/linux/pci.h +7 -0
index d432428fd..9f3e85f33 100644
@@ -49,6 +49,9 @@
 			       PCI_STATUS_SIG_TARGET_ABORT | \
 			       PCI_STATUS_PARITY)
 
+/* Number of reset methods used in pci_reset_fn_methods array in pci.c */
+#define PCI_NUM_RESET_METHODS 6
+
 /*
  * The PCI interface treats multi-function devices as independent
  * devices.  The slot/function address of each device is encoded
@@ -506,6 +509,10 @@ struct pci_dev {
 	char		*driver_override; /* Driver name to force a match */
 
 	unsigned long	priv_flags;	/* Private flags for the PCI driver */
+	/*
+	 * See pci_reset_fn_methods array in pci.c for ordering.
+	 */
+	u8 reset_methods[PCI_NUM_RESET_METHODS];	/* Reset methods ordered by priority */
 };
 
 static inline struct pci_dev *pci_physfn(struct pci_dev *dev)
Signature · 1 line
2.32.0
#11 lore raw

Re: [PATCH v10 2/8] PCI: Add new array for keeping track of ordering of reset methods

Quoted 18 lines · Bjorn Helgaas
Quoted 16 lines · Amey Narkhede

Introduce a new array reset_methods in struct pci_dev to keep track of reset mechanisms supported by the device and their ordering.

Also refactor probing and reset functions to take advantage of calling convention of reset functions.

---
 drivers/pci/pci.c   | 92 ++++++++++++++++++++++++++-------------------
 drivers/pci/pci.h   |  9 ++++-
 drivers/pci/probe.c |  5 +--
 include/linux/pci.h |  7 ++++
 4 files changed, 70 insertions(+), 43 deletions(-)

[...]

Quoted 15 lines
Quoted 15 lines
drivers/pci/pci.c +1 -11
+	BUILD_BUG_ON(ARRAY_SIZE(pci_reset_fn_methods) != PCI_NUM_RESET_METHODS);
 
 	might_sleep();
 
-	rc = pci_dev_specific_reset(dev, 1);
-	if (rc != -ENOTTY)
-		return rc;
-	rc = pcie_reset_flr(dev, 1);
-	if (rc != -ENOTTY)
-		return rc;
-	rc = pci_af_flr(dev, 1);
-	if (rc != -ENOTTY)
-		return rc;
-	rc = pci_pm_reset(dev, 1);
-	if (rc != -ENOTTY)
Quoted 9 lines

drivers/pci/pci.c static void pci_dev_restore(struct pci_dev *dev) · line 5196

Quoted 5 lines
drivers/pci/pci.c +4 -1
-		return rc;
+	for (i = 1; i < PCI_NUM_RESET_METHODS; i++) {
+		rc = pci_reset_fn_methods[i].reset_fn(dev, 1);
+		if (!rc)
+			reset_methods[n++] = i;

Why do we need this local reset_methods[] array? Can we just fill in dev->reset_methods[] directly and skip the memcpy() below?

This is for avoiding caching of previously supported reset methods. Is it okay if I use memset(dev->reset_methods, 0, sizeof(dev->reset_methods)) instead to clear the values in dev->reset_methods?

Thanks,
Amey

Quoted 9 lines
Quoted 9 lines
drivers/pci/pci.c +4 -1
+		else if (rc != -ENOTTY)
+			break;
+	}
 
-	return pci_reset_bus_function(dev, 1);
+	memcpy(dev->reset_methods, reset_methods, sizeof(reset_methods));
 }
 
 /**

[...]

#12 lore raw

Re: [PATCH v10 2/8] PCI: Add new array for keeping track of ordering of reset methods

Quoted 44 lines · Amey Narkhede
Quoted 18 lines · Bjorn Helgaas
Quoted 16 lines · Amey Narkhede

Introduce a new array reset_methods in struct pci_dev to keep track of reset mechanisms supported by the device and their ordering.

Also refactor probing and reset functions to take advantage of calling convention of reset functions.

---
 drivers/pci/pci.c   | 92 ++++++++++++++++++++++++++-------------------
 drivers/pci/pci.h   |  9 ++++-
 drivers/pci/probe.c |  5 +--
 include/linux/pci.h |  7 ++++
 4 files changed, 70 insertions(+), 43 deletions(-)

[...]

Quoted 15 lines
Quoted 15 lines
drivers/pci/pci.c +1 -11
+	BUILD_BUG_ON(ARRAY_SIZE(pci_reset_fn_methods) != PCI_NUM_RESET_METHODS);
 
 	might_sleep();
 
-	rc = pci_dev_specific_reset(dev, 1);
-	if (rc != -ENOTTY)
-		return rc;
-	rc = pcie_reset_flr(dev, 1);
-	if (rc != -ENOTTY)
-		return rc;
-	rc = pci_af_flr(dev, 1);
-	if (rc != -ENOTTY)
-		return rc;
-	rc = pci_pm_reset(dev, 1);
-	if (rc != -ENOTTY)
Quoted 9 lines

drivers/pci/pci.c static void pci_dev_restore(struct pci_dev *dev) · line 5196

Quoted 5 lines
drivers/pci/pci.c +4 -1
-		return rc;
+	for (i = 1; i < PCI_NUM_RESET_METHODS; i++) {
+		rc = pci_reset_fn_methods[i].reset_fn(dev, 1);
+		if (!rc)
+			reset_methods[n++] = i;

Why do we need this local reset_methods[] array? Can we just fill in dev->reset_methods[] directly and skip the memcpy() below?

Quoted 4 lines

This is for avoiding caching of previously supported reset methods. Is it okay if I use memset(dev->reset_methods, 0, sizeof(dev->reset_methods)) instead to clear the values in dev->reset_methods?

I don't think there's ever a case where you look at a dev->reset_methods[] element past a zero value, so we shouldn't care about any previously-supported methods left in the array.

If we *do* look at something past a zero value, why do we do that? It sounds like it would be a bug.

Quoted 10 lines
Quoted 9 lines
Quoted 9 lines
drivers/pci/pci.c +4 -1
+		else if (rc != -ENOTTY)
+			break;
+	}
 
-	return pci_reset_bus_function(dev, 1);
+	memcpy(dev->reset_methods, reset_methods, sizeof(reset_methods));
 }
 
 /**

[...]

#13 lore raw

Re: [PATCH v10 2/8] PCI: Add new array for keeping track of ordering of reset methods

Hi Bjorn,

Quoted 50 lines · Bjorn Helgaas

External email: Use caution opening links or attachments

Quoted 39 lines · Amey Narkhede
Quoted 17 lines · Bjorn Helgaas
Quoted 16 lines · Amey Narkhede

Introduce a new array reset_methods in struct pci_dev to keep track of reset mechanisms supported by the device and their ordering.

Also refactor probing and reset functions to take advantage of calling convention of reset functions.

---
 drivers/pci/pci.c   | 92 ++++++++++++++++++++++++++-------------------
 drivers/pci/pci.h   |  9 ++++-
 drivers/pci/probe.c |  5 +--
 include/linux/pci.h |  7 ++++
 4 files changed, 70 insertions(+), 43 deletions(-)

[...]

Quoted 20 lines
Quoted 20 lines

+ BUILD_BUG_ON(ARRAY_SIZE(pci_reset_fn_methods) != PCI_NUM_RESET_METHODS);

  might_sleep();

- rc = pci_dev_specific_reset(dev, 1);
- if (rc != -ENOTTY)

drivers/pci/pci.c +4 -10
-         return rc;
- rc = pcie_reset_flr(dev, 1);
- if (rc != -ENOTTY)
-         return rc;
- rc = pci_af_flr(dev, 1);
- if (rc != -ENOTTY)
-         return rc;
- rc = pci_pm_reset(dev, 1);
- if (rc != -ENOTTY)
-         return rc;
+ for (i = 1; i < PCI_NUM_RESET_METHODS; i++) {
+         rc = pci_reset_fn_methods[i].reset_fn(dev, 1);
+         if (!rc)
+                 reset_methods[n++] = i;
Quoted 7 lines
Quoted 3 lines

Why do we need this local reset_methods[] array? Can we just fill in dev->reset_methods[] directly and skip the memcpy() below?

This is for avoiding caching of previously supported reset methods. Is it okay if I use memset(dev->reset_methods, 0, sizeof(dev->reset_methods)) instead to clear the values in dev->reset_methods?

Quoted 7 lines

I don't think there's ever a case where you look at a dev->reset_methods[] element past a zero value, so we shouldn't care about any previously-supported methods left in the array.

If we *do* look at something past a zero value, why do we do that? It sounds like it would be a bug.

I think either we need memset or clear 0th/last element. Can we set the last index explicitly to zero?

void pci_init_reset_methods(struct pci_dev *dev)
{
        int i, n, rc;

        BUILD_BUG_ON(ARRAY_SIZE(pci_reset_fn_methods) != PCI_NUM_RESET_METHODS);

        might_sleep();

        n = 0;
        for (i = 1; i < PCI_NUM_RESET_METHODS; i++) { rc = pci_reset_fn_methods[i].reset_fn(dev, 1); if (!rc)
                        dev->reset_methods[n++] = i; else if (rc != -ENOTTY)
                        break;
        }
        dev->reset_methods[n] = 0;
}

#14 lore raw

Re: [PATCH v10 2/8] PCI: Add new array for keeping track of ordering of reset methods

In reply to Amey Narkhede

Hi Amey,

Quoted 20 lines · Amey Narkhede
Quoted 20 lines
Quoted 20 lines
drivers/pci/pci.c +5 -12
+   BUILD_BUG_ON(ARRAY_SIZE(pci_reset_fn_methods) != PCI_NUM_RESET_METHODS);
 
    might_sleep();
 
-   rc = pci_dev_specific_reset(dev, 1);
-   if (rc != -ENOTTY)
-           return rc;
-   rc = pcie_reset_flr(dev, 1);
-   if (rc != -ENOTTY)
-           return rc;
-   rc = pci_af_flr(dev, 1);
-   if (rc != -ENOTTY)
-           return rc;
-   rc = pci_pm_reset(dev, 1);
-   if (rc != -ENOTTY)
-           return rc;
+   for (i = 1; i < PCI_NUM_RESET_METHODS; i++) {
+           rc = pci_reset_fn_methods[i].reset_fn(dev, 1);
+           if (!rc)
+                   reset_methods[n++] = i;
Quoted 7 lines
Quoted 3 lines

Why do we need this local reset_methods[] array? Can we just fill in dev->reset_methods[] directly and skip the memcpy() below?

This is for avoiding caching of previously supported reset methods. Is it okay if I use memset(dev->reset_methods, 0, sizeof(dev->reset_methods)) instead to clear the values in dev->reset_methods?

Clearing the array before the loop might a better option, we can get rid of a local variable.

void pci_init_reset_methods(struct pci_dev *dev)
{
        int i, n, rc;

        BUILD_BUG_ON(ARRAY_SIZE(pci_reset_fn_methods) != PCI_NUM_RESET_METHODS);

        might_sleep();

        memset(dev->reset_methods, 0x0, sizeof(reset_methods));

        n = 0;
        for (i = 1; i < PCI_NUM_RESET_METHODS; i++) { rc = pci_reset_fn_methods[i].reset_fn(dev, 1); if (!rc)
                        dev->reset_methods[n++] = i; else if (rc != -ENOTTY)
                        break;
        }
}

#15 lore raw

Re: [PATCH v10 2/8] PCI: Add new array for keeping track of ordering of reset methods

In reply to Bjorn Helgaas

Hi Bjorn,

Quoted 8 lines · Bjorn Helgaas
Quoted 8 lines
drivers/pci/pci.c +0 -0
index fefa6d7b3..42440cb10 100644
@@ -72,6 +72,14 @@ static void pci_dev_d3_sleep(struct pci_dev *dev)
              msleep(delay);
 }
 
Quoted 8 lines

drivers/pci/pci.c static void pci_dev_d3_sleep(struct pci_dev *dev) · line 80

Quoted 6 lines
drivers/pci/pci.c +6 -0
+int pci_reset_supported(struct pci_dev *dev)
+{
+     u8 null_reset_methods[PCI_NUM_RESET_METHODS] = { 0 };
+
+     return memcmp(null_reset_methods,
+                   dev->reset_methods, sizeof(null_reset_methods));

I think "return dev->reset_methods[0] != 0;" is sufficient, isn't it?

Agree with you, it simplifies code logic and can be moved to "include/linux/pci.h" with inline definition. Can we change return type to 'bool' instead of 'int' ?

static inline bool pci_reset_supported(struct pci_dev *dev) {
    return !!dev->reset_methods[0];
}

#16 lore raw

Re: [PATCH v10 2/8] PCI: Add new array for keeping track of ordering of reset methods

Quoted 20 lines · Shanker R Donthineni

Hi Bjorn,

Quoted 8 lines · Bjorn Helgaas
Quoted 8 lines
drivers/pci/pci.c +0 -0
index fefa6d7b3..42440cb10 100644
@@ -72,6 +72,14 @@ static void pci_dev_d3_sleep(struct pci_dev *dev)
              msleep(delay);
 }
 
Quoted 8 lines

drivers/pci/pci.c static void pci_dev_d3_sleep(struct pci_dev *dev) · line 80

Quoted 6 lines
drivers/pci/pci.c +6 -0
+int pci_reset_supported(struct pci_dev *dev)
+{
+     u8 null_reset_methods[PCI_NUM_RESET_METHODS] = { 0 };
+
+     return memcmp(null_reset_methods,
+                   dev->reset_methods, sizeof(null_reset_methods));

I think "return dev->reset_methods[0] != 0;" is sufficient, isn't it?

Quoted 3 lines

Agree with you, it simplifies code logic and can be moved to "include/linux/pci.h" with inline definition. Can we change return type to 'bool' instead of 'int' ?

Does it need to be exported to drivers? Looks like it's only used inside drivers/pci/, so it shouldn't be in include/linux/pci.h.

Making it bool is fine with me.

Quoted 5 lines

static inline bool pci_reset_supported(struct pci_dev *dev) {
    return !!dev->reset_methods[0];
}

#17 lore raw

Re: [PATCH v10 2/8] PCI: Add new array for keeping track of ordering of reset methods

Quoted 23 lines · Bjorn Helgaas

External email: Use caution opening links or attachments

Quoted 19 lines · Shanker R Donthineni

Hi Bjorn,

Quoted 16 lines · Bjorn Helgaas
Quoted 5 lines
drivers/pci/pci.c +0 -0
index fefa6d7b3..42440cb10 100644

drivers/pci/pci.c static void pci_dev_d3_sleep(struct pci_dev *dev) · line 80

Quoted 9 lines
drivers/pci/pci.c +6 -0
@@ -72,6 +72,14 @@ static void pci_dev_d3_sleep(struct pci_dev *dev)
              msleep(delay);
 }
 
+int pci_reset_supported(struct pci_dev *dev)
+{
+     u8 null_reset_methods[PCI_NUM_RESET_METHODS] = { 0 };
+
+     return memcmp(null_reset_methods,
+                   dev->reset_methods, sizeof(null_reset_methods));

I think "return dev->reset_methods[0] != 0;" is sufficient, isn't it?

Quoted 5 lines
Quoted 3 lines

Agree with you, it simplifies code logic and can be moved to "include/linux/pci.h" with inline definition. Can we change return type to 'bool' instead of 'int' ?

Does it need to be exported to drivers? Looks like it's only used inside drivers/pci/, so it shouldn't be in include/linux/pci.h.

Yes, you're right can be moved to driver/pci/pci.h.

Quoted 7 lines

Making it bool is fine with me.

Quoted 5 lines

static inline bool pci_reset_supported(struct pci_dev *dev)

{  
    return !!dev->reset_methods[0];
}
#18 lore raw

PATCH v10 3/8 PCI: Remove reset_fn field from pci_dev

In reply to Amey Narkhede

reset_fn field is used to indicate whether the device supports any reset mechanism or not. Remove the use of reset_fn in favor of new reset_methods array which can be used to keep track of all supported reset mechanisms of a device and their ordering.

The octeon driver is incorrectly using reset_fn field to detect if the device supports FLR or not. Use pcie_reset_flr() to probe whether it supports FLR or not.

---
 drivers/net/ethernet/cavium/liquidio/lio_vf_main.c | 2 +-
 drivers/pci/pci-sysfs.c                            | 2 +-
 drivers/pci/pci.c                                  | 6 +++---
 drivers/pci/probe.c                                | 1 -
 drivers/pci/quirks.c                               | 2 +-
 drivers/pci/remove.c                               | 1 -
 include/linux/pci.h                                | 1 -
 7 files changed, 6 insertions(+), 9 deletions(-)
drivers/net/ethernet/cavium/liquidio/lio_vf_main.c +1 -1
index 516f166ce..336d149ee 100644
@@ -526,7 +526,7 @@ static void octeon_destroy_resources(struct octeon_device *oct)
 			oct->irq_name_storage = NULL;
 		}
 		/* Soft reset the octeon device before exiting */
-		if (oct->pci_dev->reset_fn)
+		if (!pcie_reset_flr(oct->pci_dev, 1))
 			octeon_pci_flr(oct);
 		else
 			cn23xx_vf_ask_pf_to_do_flr(oct);
drivers/pci/pci-sysfs.c +1 -1
index beb8d1f4f..316f70c3e 100644
@@ -1367,7 +1367,7 @@ static umode_t pci_dev_reset_attr_is_visible(struct kobject *kobj,
 {
 	struct pci_dev *pdev = to_pci_dev(kobj_to_dev(kobj));
 
-	if (!pdev->reset_fn)
+	if (!pci_reset_supported(pdev))
 		return 0;
 
 	return a->mode;
drivers/pci/pci.c +3 -3
index 42440cb10..d5c16492c 100644
@@ -5221,7 +5221,7 @@ int pci_reset_function(struct pci_dev *dev)
 {
 	int rc;
 
-	if (!dev->reset_fn)
+	if (!pci_reset_supported(dev))
 		return -ENOTTY;
 
 	pci_dev_lock(dev);
@@ -5257,7 +5257,7 @@ int pci_reset_function_locked(struct pci_dev *dev)
 {
 	int rc;
 
-	if (!dev->reset_fn)
+	if (!pci_reset_supported(dev))
 		return -ENOTTY;
 
 	pci_dev_save_and_disable(dev);
@@ -5280,7 +5280,7 @@ int pci_try_reset_function(struct pci_dev *dev)
 {
 	int rc;
 
-	if (!dev->reset_fn)
+	if (!pci_reset_supported(dev))
 		return -ENOTTY;
 
 	if (!pci_dev_trylock(dev))
drivers/pci/probe.c +0 -1
index bc4af914a..c272e23db 100644
@@ -2407,7 +2407,6 @@ static void pci_init_capabilities(struct pci_dev *dev)
 
 	pcie_report_downtraining(dev);
 	pci_init_reset_methods(dev);
-	dev->reset_fn = pci_reset_supported(dev);
 }
 
 /*
drivers/pci/quirks.c +1 -1
index f977ba79a..e86cf4a3b 100644
@@ -5589,7 +5589,7 @@ static void quirk_reset_lenovo_thinkpad_p50_nvgpu(struct pci_dev *pdev)
 
 	if (pdev->subsystem_vendor != PCI_VENDOR_ID_LENOVO ||
 	    pdev->subsystem_device != 0x222e ||
-	    !pdev->reset_fn)
+	    !pci_reset_supported(pdev))
 		return;
 
 	if (pci_enable_device_mem(pdev))
drivers/pci/remove.c +0 -1
index dd12c2fcc..4c54c7505 100644
@@ -19,7 +19,6 @@ static void pci_stop_dev(struct pci_dev *dev)
 	pci_pme_active(dev, false);
 
 	if (pci_dev_is_added(dev)) {
-		dev->reset_fn = 0;
 
 		device_release_driver(&dev->dev);
 		pci_proc_detach_device(dev);
include/linux/pci.h +0 -1
index 9f3e85f33..f34563831 100644
@@ -431,7 +431,6 @@ struct pci_dev {
 	unsigned int	state_saved:1;
 	unsigned int	is_physfn:1;
 	unsigned int	is_virtfn:1;
-	unsigned int	reset_fn:1;
 	unsigned int	is_hotplug_bridge:1;
 	unsigned int	shpc_managed:1;		/* SHPC owned by shpchp */
 	unsigned int	is_thunderbolt:1;	/* Thunderbolt controller */
Signature · 1 line
2.32.0
#19 lore raw

PATCH v10 4/8 PCI/sysfs: Allow userspace to query and set device reset mechanism

In reply to Amey Narkhede

Add reset_method sysfs attribute to enable user to query and set user preferred device reset methods and their ordering.

---
 Documentation/ABI/testing/sysfs-bus-pci |  19 +++++
 drivers/pci/pci-sysfs.c                 | 103 ++++++++++++++++++++++++
 2 files changed, 122 insertions(+)
Documentation/ABI/testing/sysfs-bus-pci +19 -0
index ef00fada2..43f4e33c7 100644
@@ -121,6 +121,25 @@ Description:
 		child buses, and re-discover devices removed earlier
 		from this part of the device tree.
 
+What:		/sys/bus/pci/devices/.../reset_method
+Date:		March 2021
+Contact:	Amey Narkhede <[email protected]>
+Description:
+		Some devices allow an individual function to be reset
+		without affecting other functions in the same slot.
+
+		For devices that have this support, a file named
+		reset_method will be present in sysfs. Initially reading
+		this file will give names of the device supported reset
+		methods and their ordering. After write, this file will
+		give names and ordering of currently enabled reset methods.
+		Writing the name or comma separated list of names of any of
+		the device supported reset methods to this file will set
+		the reset methods and their ordering to be used when
+		resetting the device. Writing empty string to this file
+		will disable ability to reset the device and writing
+		"default" will return to the original value.
+
 What:		/sys/bus/pci/devices/.../reset
 Date:		July 2009
 Contact:	Michael S. Tsirkin <[email protected]>
drivers/pci/pci-sysfs.c +103 -0

115 lines not on this page · raw message

Signature · 1 line
2.32.0
#20 lore raw

Re: [PATCH v10 4/8] PCI/sysfs: Allow userspace to query and set device reset mechanism

Quoted 50 lines · Amey Narkhede

Add reset_method sysfs attribute to enable user to query and set user preferred device reset methods and their ordering.

---
 Documentation/ABI/testing/sysfs-bus-pci |  19 +++++
 drivers/pci/pci-sysfs.c                 | 103 ++++++++++++++++++++++++
 2 files changed, 122 insertions(+)
Documentation/ABI/testing/sysfs-bus-pci +19 -0
index ef00fada2..43f4e33c7 100644
@@ -121,6 +121,25 @@ Description:
 		child buses, and re-discover devices removed earlier
 		from this part of the device tree.
 
+What:		/sys/bus/pci/devices/.../reset_method
+Date:		March 2021
+Contact:	Amey Narkhede <[email protected]>
+Description:
+		Some devices allow an individual function to be reset
+		without affecting other functions in the same slot.
+
+		For devices that have this support, a file named
+		reset_method will be present in sysfs. Initially reading
+		this file will give names of the device supported reset
+		methods and their ordering. After write, this file will
+		give names and ordering of currently enabled reset methods.
+		Writing the name or comma separated list of names of any of
+		the device supported reset methods to this file will set
+		the reset methods and their ordering to be used when
+		resetting the device. Writing empty string to this file
+		will disable ability to reset the device and writing
+		"default" will return to the original value.
+
 What:		/sys/bus/pci/devices/.../reset
 Date:		July 2009
 Contact:	Michael S. Tsirkin <[email protected]>
drivers/pci/pci-sysfs.c +1 -0
index 316f70c3e..8a740e211 100644
@@ -1334,6 +1334,108 @@ static const struct attribute_group pci_dev_rom_attr_group = {
 	.is_bin_visible = pci_dev_rom_attr_is_visible,
 };
 
+static ssize_t reset_method_show(struct device *dev,

drivers/pci/pci-sysfs.c static const struct attribute_group pci_dev_rom_attr_group = { · line 1347

Quoted 10 lines
drivers/pci/pci-sysfs.c +10 -0
+				 struct device_attribute *attr,
+				 char *buf)
+{
+	struct pci_dev *pdev = to_pci_dev(dev);
+	ssize_t len = 0;
+	int i, idx;
+
+	for (i = 0; i < PCI_NUM_RESET_METHODS; i++) {
+		idx = pdev->reset_methods[i];
+		if (!idx)

I know it's totally OCD, but can you use the same "m" for indexing pci_reset_fn_methods[] that you used in __pci_reset_function_locked() and ... oops, I meant to say pci_init_reset_methods(), but there you used "i". Maybe pci_init_reset_methods() could use "m" as well, and maybe it could use "i" instead of "n" to index dev->reset_methods[]?

drivers/pci/pci-sysfs.c static const struct attribute_group pci_dev_rom_attr_group = { · line 1351

Quoted 4 lines
drivers/pci/pci-sysfs.c +4 -0
+			break;
+
+		len += sysfs_emit_at(buf, len, "%s%s", len ? "," : "",
+				     pci_reset_fn_methods[idx].name);

Maybe separate with spaces instead of commas? E.g.,

device_specific flr pm bus

instead of

device_specific,flr,pm,bus

I think the former is less error-prone when parsing. Easier to split in a shell script, for example.

Quoted 9 lines
drivers/pci/pci-sysfs.c +9 -0
+	}
+
+	if (len)
+		len += sysfs_emit_at(buf, len, "\n");
+
+	return len;
+}
+
+static ssize_t reset_method_store(struct device *dev,

drivers/pci/pci-sysfs.c static const struct attribute_group pci_dev_rom_attr_group = { · line 1370

Quoted 10 lines
drivers/pci/pci-sysfs.c +10 -0
+				  struct device_attribute *attr,
+				  const char *buf, size_t count)
+{
+	struct pci_dev *pdev = to_pci_dev(dev);
+	int n = 0;
+	char *name, *options = NULL;
+	u8 reset_methods[PCI_NUM_RESET_METHODS] = { 0 };
+
+	if (count >= (PAGE_SIZE - 1))
+		return -EINVAL;

I'm not the sysfs expert, but surely the sysfs infrastructure already guarantees this?

drivers/pci/pci-sysfs.c static const struct attribute_group pci_dev_rom_attr_group = { · line 1373

Quoted 3 lines
drivers/pci/pci-sysfs.c +3 -0
+
+	if (sysfs_streq(buf, "")) {
+		pci_warn(pdev, "All device reset methods disabled by user");

Can we just do:

  pdev->reset_methods[0] = 0;

here and dispense with the memcpy()?

drivers/pci/pci-sysfs.c static const struct attribute_group pci_dev_rom_attr_group = { · line 1382

Quoted 9 lines
drivers/pci/pci-sysfs.c +9 -0
+		goto set_reset_methods;
+	}
+
+	if (sysfs_streq(buf, "default")) {
+		pci_init_reset_methods(pdev);
+		return count;
+	}
+
+	options = kstrndup(buf, count, GFP_KERNEL);

I assume the kstrndup() is because strsep() writes into the buffer? Aren't we allowed to write into the buffer we get from sysfs? Does the user ever see the buffer contents again? I would think sysfs would have already done a copy_from_user() or whatever.

drivers/pci/pci-sysfs.c static const struct attribute_group pci_dev_rom_attr_group = { · line 1386

Quoted 4 lines
drivers/pci/pci-sysfs.c +4 -0
+	if (!options)
+		return -ENOMEM;
+
+	while ((name = strsep(&options, ",")) != NULL) {

*If* you change the show to use spaces, obviously this would have to change as well (as well as the doc).

drivers/pci/pci-sysfs.c static const struct attribute_group pci_dev_rom_attr_group = { · line 1397

Quoted 11 lines
drivers/pci/pci-sysfs.c +11 -0
+		int i;
+
+		if (sysfs_streq(name, ""))
+			continue;
+
+		name = strim(name);
+
+		for (i = 1; i < PCI_NUM_RESET_METHODS; i++) {
+			if (sysfs_streq(name, pci_reset_fn_methods[i].name) &&
+			    !pci_reset_fn_methods[i].reset_fn(pdev, 1)) {
+				reset_methods[n++] = i;

Can we build this directly in pdev->reset_methods[] so we don't need the memcpy() below?

drivers/pci/pci-sysfs.c static const struct attribute_group pci_dev_rom_attr_group = { · line 1358

Quoted 3 lines
drivers/pci/pci-sysfs.c +3 -0
+				break;
+			}
+		}

Same "m" for indexing pci_reset_fn_methods[], "i" for pdev->reset_methods[] comment here.

drivers/pci/pci-sysfs.c static const struct attribute_group pci_dev_rom_attr_group = { · line 1409

Quoted 8 lines
drivers/pci/pci-sysfs.c +8 -0
+		if (i == PCI_NUM_RESET_METHODS) {
+			kfree(options);
+			return -EINVAL;
+		}
+	}
+
+	if (!pci_reset_fn_methods[1].reset_fn(pdev, 1) && reset_methods[0] != 1)
+		pci_warn(pdev, "Device specific reset disabled/de-prioritized by user");

Hmmm. I sort of see the point here, but I wish we didn't have the implicit dependency on pci_reset_fn_methods[1] being pci_dev_specific_reset().

I know we've talked about this before. I'm still not 100% sure either of these warnings is worthwhile, especially since we're not *using* the reset here. It might be useful at the point where we try to *do* a reset. I dunno. Maybe this is the best place since this is where the user potentially screwed up.

Quoted 42 lines
drivers/pci/pci-sysfs.c +29 -0
+set_reset_methods:
+	memcpy(pdev->reset_methods, reset_methods, sizeof(reset_methods));
+	kfree(options);
+	return count;
+}
+static DEVICE_ATTR_RW(reset_method);
+
+static struct attribute *pci_dev_reset_method_attrs[] = {
+	&dev_attr_reset_method.attr,
+	NULL,
+};
+
+static umode_t pci_dev_reset_method_attr_is_visible(struct kobject *kobj,
+						    struct attribute *a, int n)
+{
+	struct pci_dev *pdev = to_pci_dev(kobj_to_dev(kobj));
+
+	if (!pci_reset_supported(pdev))
+		return 0;
+
+	return a->mode;
+}
+
+static const struct attribute_group pci_dev_reset_method_attr_group = {
+	.attrs = pci_dev_reset_method_attrs,
+	.is_visible = pci_dev_reset_method_attr_is_visible,
+};
+
 static ssize_t reset_store(struct device *dev, struct device_attribute *attr,
 			   const char *buf, size_t count)
 {
@@ -1491,6 +1593,7 @@ const struct attribute_group *pci_dev_groups[] = {
 	&pci_dev_config_attr_group,
 	&pci_dev_rom_attr_group,
 	&pci_dev_reset_attr_group,
+	&pci_dev_reset_method_attr_group,
 	&pci_dev_vpd_attr_group,
 #ifdef CONFIG_DMI
 	&pci_dev_smbios_attr_group,
Signature · 1 line
2.32.0
#21 lore raw

Re: [PATCH v10 4/8] PCI/sysfs: Allow userspace to query and set device reset mechanism

Hi Bjorn,

[...]

Quoted 5 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+	if (count >= (PAGE_SIZE - 1))
+		return -EINVAL;

I'm not the sysfs expert, but surely the sysfs infrastructure already guarantees this?

We don't need to store any value, since we are processing the input from the userspace, thus ensuring that we have room for the newline is not needed, especially since the show() function dynamically builds the content to show, so indeed this check can be dropped.

To add, there aren't any guarantees other from sysfs than we get a up to a PAGE_SIZE worth of data in the buffer.

[...]

Quoted 3 lines

drivers/pci/pci-sysfs.c static const struct attribute_group pci_dev_rom_attr_group = { · line 1382

Quoted 1 line
drivers/pci/pci-sysfs.c +1 -0
+	options = kstrndup(buf, count, GFP_KERNEL);

I assume the kstrndup() is because strsep() writes into the buffer?

Yes, Amey added kstrndup() in v6 following my recommendation as per:

https://lore.kernel.org/linux-pci/[email protected]/

This was to avoid removing the const quantifier through a type cast given that the signature of the function denotes that the buffer is a pointer to immutable string, as per:

https://elixir.bootlin.com/linux/v5.14-rc3/source/include/linux/device/driver.h#L137

Some other sysfs users do employ the cast when using strtok() and I am not so such it's the right way to do it, as per:

  drivers/s390/net/qeth_l3_sys.c
  232:	tmp = strsep((char **)&buf, "\n");
  
  drivers/media/rc/rc-main.c
  1167:	while ((tmp = strsep((char **)&buf, " \n")) != NULL) {
Quoted 3 lines

Aren't we allowed to write into the buffer we get from sysfs? Does the user ever see the buffer contents again? I would think sysfs would have already done a copy_from_user() or whatever.

I might be wrong about this, but I suppose this might be to stop people from accidentally freeing the buffer as kernfs_fop_write_iter() would do it after all the internal housekeeping is done, provided that someone pays attention to compile time warnings.

Krzysztof

#22 lore raw

Re: [PATCH v10 4/8] PCI/sysfs: Allow userspace to query and set device reset mechanism

Quoted 6 lines · Krzysztof Wilczyński
Quoted 3 lines

drivers/pci/pci-sysfs.c static const struct attribute_group pci_dev_rom_attr_group = { · line 1382

Quoted 1 line
drivers/pci/pci-sysfs.c +1 -0
+	options = kstrndup(buf, count, GFP_KERNEL);

I assume the kstrndup() is because strsep() writes into the buffer?

Yes, Amey added kstrndup() in v6 following my recommendation as per:

Quoted 7 lines

https://lore.kernel.org/linux-pci/[email protected]/

This was to avoid removing the const quantifier through a type cast given that the signature of the function denotes that the buffer is a pointer to immutable string, as per:

https://elixir.bootlin.com/linux/v5.14-rc3/source/include/linux/device/driver.h#L137

Ah, right, thanks! Definitely prefer not to cast away the constness.

I guess the strings here are short (<100 chars max), so no big deal to duplicate them. Sorry for the noise!

#23 lore raw

Re: [PATCH v10 4/8] PCI/sysfs: Allow userspace to query and set device reset mechanism

In reply to Bjorn Helgaas
Quoted 43 lines · Bjorn Helgaas
Quoted 41 lines · Amey Narkhede

Add reset_method sysfs attribute to enable user to query and set user preferred device reset methods and their ordering.

---
 Documentation/ABI/testing/sysfs-bus-pci |  19 +++++
 drivers/pci/pci-sysfs.c                 | 103 ++++++++++++++++++++++++
 2 files changed, 122 insertions(+)
Documentation/ABI/testing/sysfs-bus-pci +19 -0
index ef00fada2..43f4e33c7 100644
@@ -121,6 +121,25 @@ Description:
 		child buses, and re-discover devices removed earlier
 		from this part of the device tree.
 
+What:		/sys/bus/pci/devices/.../reset_method
+Date:		March 2021
+Contact:	Amey Narkhede <[email protected]>
+Description:
+		Some devices allow an individual function to be reset
+		without affecting other functions in the same slot.
+
+		For devices that have this support, a file named
+		reset_method will be present in sysfs. Initially reading
+		this file will give names of the device supported reset
+		methods and their ordering. After write, this file will
+		give names and ordering of currently enabled reset methods.
+		Writing the name or comma separated list of names of any of
+		the device supported reset methods to this file will set
+		the reset methods and their ordering to be used when
+		resetting the device. Writing empty string to this file
+		will disable ability to reset the device and writing
+		"default" will return to the original value.
+
 What:		/sys/bus/pci/devices/.../reset
 Date:		July 2009
 Contact:	Michael S. Tsirkin <[email protected]>

[...]

Quoted 6 lines
Quoted 6 lines
Documentation/ABI/testing/sysfs-bus-pci +6 -0
+		int i;
+
+		if (sysfs_streq(name, ""))
+			continue;
+
+		name = strim(name);
Quoted 9 lines

Documentation/ABI/testing/sysfs-bus-pci

Quoted 5 lines
Documentation/ABI/testing/sysfs-bus-pci +5 -0
+
+		for (i = 1; i < PCI_NUM_RESET_METHODS; i++) {
+			if (sysfs_streq(name, pci_reset_fn_methods[i].name) &&
+			    !pci_reset_fn_methods[i].reset_fn(pdev, 1)) {
+				reset_methods[n++] = i;

Can we build this directly in pdev->reset_methods[] so we don't need the memcpy() below?

This is to avoid writing partial values directly to dev->reset_methods. So for example if user writes flr,unsupported_value then dev->reset_methods should not be modified even though flr is valid reset method in this example to avoid partial writes. Either operation(in this case writing supported reset methods to reset_method attr) succeeds completely or it fails othewise.

Thanks,
Amey

[...]

#24 lore raw

Re: [PATCH v10 4/8] PCI/sysfs: Allow userspace to query and set device reset mechanism

Quoted 61 lines · Amey Narkhede
Quoted 43 lines · Bjorn Helgaas
Quoted 41 lines · Amey Narkhede

Add reset_method sysfs attribute to enable user to query and set user preferred device reset methods and their ordering.

---
 Documentation/ABI/testing/sysfs-bus-pci |  19 +++++
 drivers/pci/pci-sysfs.c                 | 103 ++++++++++++++++++++++++
 2 files changed, 122 insertions(+)
Documentation/ABI/testing/sysfs-bus-pci +19 -0
index ef00fada2..43f4e33c7 100644
@@ -121,6 +121,25 @@ Description:
 		child buses, and re-discover devices removed earlier
 		from this part of the device tree.
 
+What:		/sys/bus/pci/devices/.../reset_method
+Date:		March 2021
+Contact:	Amey Narkhede <[email protected]>
+Description:
+		Some devices allow an individual function to be reset
+		without affecting other functions in the same slot.
+
+		For devices that have this support, a file named
+		reset_method will be present in sysfs. Initially reading
+		this file will give names of the device supported reset
+		methods and their ordering. After write, this file will
+		give names and ordering of currently enabled reset methods.
+		Writing the name or comma separated list of names of any of
+		the device supported reset methods to this file will set
+		the reset methods and their ordering to be used when
+		resetting the device. Writing empty string to this file
+		will disable ability to reset the device and writing
+		"default" will return to the original value.
+
 What:		/sys/bus/pci/devices/.../reset
 Date:		July 2009
 Contact:	Michael S. Tsirkin <[email protected]>

[...]

Quoted 6 lines
Quoted 6 lines
Documentation/ABI/testing/sysfs-bus-pci +6 -0
+		int i;
+
+		if (sysfs_streq(name, ""))
+			continue;
+
+		name = strim(name);
Quoted 9 lines

Documentation/ABI/testing/sysfs-bus-pci

Quoted 5 lines
Documentation/ABI/testing/sysfs-bus-pci +5 -0
+
+		for (i = 1; i < PCI_NUM_RESET_METHODS; i++) {
+			if (sysfs_streq(name, pci_reset_fn_methods[i].name) &&
+			    !pci_reset_fn_methods[i].reset_fn(pdev, 1)) {
+				reset_methods[n++] = i;

Can we build this directly in pdev->reset_methods[] so we don't need the memcpy() below?

Quoted 6 lines

This is to avoid writing partial values directly to dev->reset_methods. So for example if user writes flr,unsupported_value then dev->reset_methods should not be modified even though flr is valid reset method in this example to avoid partial writes. Either operation(in this case writing supported reset methods to reset_method attr) succeeds completely or it fails othewise.

I guess the question is, if somebody writes

flr junk bus

and we set the supported methods to "flr bus", is that OK? It seems OK to me; obviously we have to protect ourselves from attack, but over-zealous checking can make things unnecessarily complicated.

#25 lore raw

Re: [PATCH v10 4/8] PCI/sysfs: Allow userspace to query and set device reset mechanism

Quoted 69 lines · Bjorn Helgaas
Quoted 61 lines · Amey Narkhede
Quoted 43 lines · Bjorn Helgaas
Quoted 41 lines · Amey Narkhede

Add reset_method sysfs attribute to enable user to query and set user preferred device reset methods and their ordering.

---
 Documentation/ABI/testing/sysfs-bus-pci |  19 +++++
 drivers/pci/pci-sysfs.c                 | 103 ++++++++++++++++++++++++
 2 files changed, 122 insertions(+)
Documentation/ABI/testing/sysfs-bus-pci +19 -0
index ef00fada2..43f4e33c7 100644
@@ -121,6 +121,25 @@ Description:
 		child buses, and re-discover devices removed earlier
 		from this part of the device tree.
 
+What:		/sys/bus/pci/devices/.../reset_method
+Date:		March 2021
+Contact:	Amey Narkhede <[email protected]>
+Description:
+		Some devices allow an individual function to be reset
+		without affecting other functions in the same slot.
+
+		For devices that have this support, a file named
+		reset_method will be present in sysfs. Initially reading
+		this file will give names of the device supported reset
+		methods and their ordering. After write, this file will
+		give names and ordering of currently enabled reset methods.
+		Writing the name or comma separated list of names of any of
+		the device supported reset methods to this file will set
+		the reset methods and their ordering to be used when
+		resetting the device. Writing empty string to this file
+		will disable ability to reset the device and writing
+		"default" will return to the original value.
+
 What:		/sys/bus/pci/devices/.../reset
 Date:		July 2009
 Contact:	Michael S. Tsirkin <[email protected]>

[...]

Quoted 6 lines
Quoted 6 lines
Documentation/ABI/testing/sysfs-bus-pci +6 -0
+		int i;
+
+		if (sysfs_streq(name, ""))
+			continue;
+
+		name = strim(name);
Quoted 9 lines

Documentation/ABI/testing/sysfs-bus-pci

Quoted 5 lines
Documentation/ABI/testing/sysfs-bus-pci +5 -0
+
+		for (i = 1; i < PCI_NUM_RESET_METHODS; i++) {
+			if (sysfs_streq(name, pci_reset_fn_methods[i].name) &&
+			    !pci_reset_fn_methods[i].reset_fn(pdev, 1)) {
+				reset_methods[n++] = i;

Can we build this directly in pdev->reset_methods[] so we don't need the memcpy() below?

Quoted 6 lines

This is to avoid writing partial values directly to dev->reset_methods. So for example if user writes flr,unsupported_value then dev->reset_methods should not be modified even though flr is valid reset method in this example to avoid partial writes. Either operation(in this case writing supported reset methods to reset_method attr) succeeds completely or it fails othewise.

Quoted 7 lines

I guess the question is, if somebody writes

flr junk bus

and we set the supported methods to "flr bus", is that OK? It seems OK to me; obviously we have to protect ourselves from attack, but over-zealous checking can make things unnecessarily complicated.

The problem is once we encounter "junk" we return -EINVAL so in your example we only set flr.

Thanks,
Amey

#26 lore raw

Re: [PATCH v10 4/8] PCI/sysfs: Allow userspace to query and set device reset mechanism

Quoted 72 lines · Amey Narkhede
Quoted 71 lines · Bjorn Helgaas
Quoted 61 lines · Amey Narkhede
Quoted 43 lines · Bjorn Helgaas
Quoted 41 lines · Amey Narkhede

Add reset_method sysfs attribute to enable user to query and set user preferred device reset methods and their ordering.

---
 Documentation/ABI/testing/sysfs-bus-pci |  19 +++++
 drivers/pci/pci-sysfs.c                 | 103 ++++++++++++++++++++++++
 2 files changed, 122 insertions(+)
Documentation/ABI/testing/sysfs-bus-pci +19 -0
index ef00fada2..43f4e33c7 100644
@@ -121,6 +121,25 @@ Description:
 		child buses, and re-discover devices removed earlier
 		from this part of the device tree.
 
+What:		/sys/bus/pci/devices/.../reset_method
+Date:		March 2021
+Contact:	Amey Narkhede <[email protected]>
+Description:
+		Some devices allow an individual function to be reset
+		without affecting other functions in the same slot.
+
+		For devices that have this support, a file named
+		reset_method will be present in sysfs. Initially reading
+		this file will give names of the device supported reset
+		methods and their ordering. After write, this file will
+		give names and ordering of currently enabled reset methods.
+		Writing the name or comma separated list of names of any of
+		the device supported reset methods to this file will set
+		the reset methods and their ordering to be used when
+		resetting the device. Writing empty string to this file
+		will disable ability to reset the device and writing
+		"default" will return to the original value.
+
 What:		/sys/bus/pci/devices/.../reset
 Date:		July 2009
 Contact:	Michael S. Tsirkin <[email protected]>

[...]

Quoted 6 lines
Quoted 6 lines
Documentation/ABI/testing/sysfs-bus-pci +6 -0
+		int i;
+
+		if (sysfs_streq(name, ""))
+			continue;
+
+		name = strim(name);
Quoted 9 lines

Documentation/ABI/testing/sysfs-bus-pci

Quoted 5 lines
Documentation/ABI/testing/sysfs-bus-pci +5 -0
+
+		for (i = 1; i < PCI_NUM_RESET_METHODS; i++) {
+			if (sysfs_streq(name, pci_reset_fn_methods[i].name) &&
+			    !pci_reset_fn_methods[i].reset_fn(pdev, 1)) {
+				reset_methods[n++] = i;

Can we build this directly in pdev->reset_methods[] so we don't need the memcpy() below?

Quoted 6 lines

This is to avoid writing partial values directly to dev->reset_methods. So for example if user writes flr,unsupported_value then dev->reset_methods should not be modified even though flr is valid reset method in this example to avoid partial writes. Either operation(in this case writing supported reset methods to reset_method attr) succeeds completely or it fails othewise.

I guess the question is, if somebody writes

Quoted 8 lines
Quoted 5 lines

flr junk bus

and we set the supported methods to "flr bus", is that OK? It seems OK to me; obviously we have to protect ourselves from attack, but over-zealous checking can make things unnecessarily complicated.

The problem is once we encounter "junk" we return -EINVAL so in your example we only set flr.

We're still talking about whether we need the memcpy(). We can decide whether it's the right thing to return -EINVAL if we encounter "junk".

Maybe it's OK if we set it to "flr" and ignore everything after "junk"? Or maybe it's OK to ignore unsupported values and set it to "flr bus"?

#27 lore raw

Re: [PATCH v10 4/8] PCI/sysfs: Allow userspace to query and set device reset mechanism

In reply to Bjorn Helgaas
Quoted 12 lines · Bjorn Helgaas
Quoted 11 lines · Amey Narkhede

Add reset_method sysfs attribute to enable user to query and set user preferred device reset methods and their ordering.

---
 Documentation/ABI/testing/sysfs-bus-pci |  19 +++++
 drivers/pci/pci-sysfs.c                 | 103 ++++++++++++++++++++++++
 2 files changed, 122 insertions(+)

[...]

Quoted 13 lines

drivers/pci/pci-sysfs.c static const struct attribute_group pci_dev_rom_attr_group = { · line 1409

Quoted 8 lines
drivers/pci/pci-sysfs.c +8 -0
+		if (i == PCI_NUM_RESET_METHODS) {
+			kfree(options);
+			return -EINVAL;
+		}
+	}
+
+	if (!pci_reset_fn_methods[1].reset_fn(pdev, 1) && reset_methods[0] != 1)
+		pci_warn(pdev, "Device specific reset disabled/de-prioritized by user");

Hmmm. I sort of see the point here, but I wish we didn't have the implicit dependency on pci_reset_fn_methods[1] being pci_dev_specific_reset().

Quoted 6 lines

I know we've talked about this before. I'm still not 100% sure either of these warnings is worthwhile, especially since we're not *using* the reset here. It might be useful at the point where we try to *do* a reset. I dunno. Maybe this is the best place since this is where the user potentially screwed up.

I agree this is the best place for the warning as this where potentially broken reset methods may get called/prioritized. We can move this check to __pci_reset_function_locked() if you want.

Quoted 42 lines
Quoted 42 lines
drivers/pci/pci-sysfs.c +29 -1
+set_reset_methods:
+	memcpy(pdev->reset_methods, reset_methods, sizeof(reset_methods));
+	kfree(options);
+	return count;
+}
+static DEVICE_ATTR_RW(reset_method);
+
+static struct attribute *pci_dev_reset_method_attrs[] = {
+	&dev_attr_reset_method.attr,
+	NULL,
+};
+
+static umode_t pci_dev_reset_method_attr_is_visible(struct kobject *kobj,
+						    struct attribute *a, int n)
+{
+	struct pci_dev *pdev = to_pci_dev(kobj_to_dev(kobj));
+
+	if (!pci_reset_supported(pdev))
+		return 0;
+
+	return a->mode;
+}
+
+static const struct attribute_group pci_dev_reset_method_attr_group = {
+	.attrs = pci_dev_reset_method_attrs,
+	.is_visible = pci_dev_reset_method_attr_is_visible,
+};
+
 static ssize_t reset_store(struct device *dev, struct device_attribute *attr,
 			   const char *buf, size_t count)
 {
@@ -1491,6 +1593,7 @@ const struct attribute_group *pci_dev_groups[] = {
 	&pci_dev_config_attr_group,
 	&pci_dev_rom_attr_group,
 	&pci_dev_reset_attr_group,
+	&pci_dev_reset_method_attr_group,
 	&pci_dev_vpd_attr_group,
 #ifdef CONFIG_DMI
 	&pci_dev_smbios_attr_group,
--

2.32.0

#28 lore raw

Re: [PATCH v10 4/8] PCI/sysfs: Allow userspace to query and set device reset mechanism

In reply to Amey Narkhede

[+cc Serge, linux-security-module: should we check CAP_SYS_ADMIN or similar for changing PCI reset mechanisms for a device?]

Quoted 70 lines · Amey Narkhede

Add reset_method sysfs attribute to enable user to query and set user preferred device reset methods and their ordering.

---
 Documentation/ABI/testing/sysfs-bus-pci |  19 +++++
 drivers/pci/pci-sysfs.c                 | 103 ++++++++++++++++++++++++
 2 files changed, 122 insertions(+)
Documentation/ABI/testing/sysfs-bus-pci +19 -0
index ef00fada2..43f4e33c7 100644
@@ -121,6 +121,25 @@ Description:
 		child buses, and re-discover devices removed earlier
 		from this part of the device tree.
 
+What:		/sys/bus/pci/devices/.../reset_method
+Date:		March 2021
+Contact:	Amey Narkhede <[email protected]>
+Description:
+		Some devices allow an individual function to be reset
+		without affecting other functions in the same slot.
+
+		For devices that have this support, a file named
+		reset_method will be present in sysfs. Initially reading
+		this file will give names of the device supported reset
+		methods and their ordering. After write, this file will
+		give names and ordering of currently enabled reset methods.
+		Writing the name or comma separated list of names of any of
+		the device supported reset methods to this file will set
+		the reset methods and their ordering to be used when
+		resetting the device. Writing empty string to this file
+		will disable ability to reset the device and writing
+		"default" will return to the original value.
+
 What:		/sys/bus/pci/devices/.../reset
 Date:		July 2009
 Contact:	Michael S. Tsirkin <[email protected]>
drivers/pci/pci-sysfs.c +21 -0
index 316f70c3e..8a740e211 100644
@@ -1334,6 +1334,108 @@ static const struct attribute_group pci_dev_rom_attr_group = {
 	.is_bin_visible = pci_dev_rom_attr_is_visible,
 };
 
+static ssize_t reset_method_show(struct device *dev,
+				 struct device_attribute *attr,
+				 char *buf)
+{
+	struct pci_dev *pdev = to_pci_dev(dev);
+	ssize_t len = 0;
+	int i, idx;
+
+	for (i = 0; i < PCI_NUM_RESET_METHODS; i++) {
+		idx = pdev->reset_methods[i];
+		if (!idx)
+			break;
+
+		len += sysfs_emit_at(buf, len, "%s%s", len ? "," : "",
+				     pci_reset_fn_methods[idx].name);
+	}
+
+	if (len)
+		len += sysfs_emit_at(buf, len, "\n");
+
+	return len;

drivers/pci/pci-sysfs.c static const struct attribute_group pci_dev_rom_attr_group = { · line 1367

Quoted 10 lines
drivers/pci/pci-sysfs.c +10 -0
+}
+
+static ssize_t reset_method_store(struct device *dev,
+				  struct device_attribute *attr,
+				  const char *buf, size_t count)
+{
+	struct pci_dev *pdev = to_pci_dev(dev);
+	int n = 0;
+	char *name, *options = NULL;
+	u8 reset_methods[PCI_NUM_RESET_METHODS] = { 0 };

Should this check "capable(CAP_SYS_ADMIN)" or similar? The sysfs file is mode 0644, so writable only by root.

I do note that Documentation/process/adding-syscalls.rst suggests "avoid adding new uses of the already overly-general CAP_SYS_ADMIN capability." But CAP_SYS_ADMIN is used for all the other checks in pci-sysfs.c.

Quoted 84 lines
drivers/pci/pci-sysfs.c +71 -0
+	if (count >= (PAGE_SIZE - 1))
+		return -EINVAL;
+
+	if (sysfs_streq(buf, "")) {
+		pci_warn(pdev, "All device reset methods disabled by user");
+		goto set_reset_methods;
+	}
+
+	if (sysfs_streq(buf, "default")) {
+		pci_init_reset_methods(pdev);
+		return count;
+	}
+
+	options = kstrndup(buf, count, GFP_KERNEL);
+	if (!options)
+		return -ENOMEM;
+
+	while ((name = strsep(&options, ",")) != NULL) {
+		int i;
+
+		if (sysfs_streq(name, ""))
+			continue;
+
+		name = strim(name);
+
+		for (i = 1; i < PCI_NUM_RESET_METHODS; i++) {
+			if (sysfs_streq(name, pci_reset_fn_methods[i].name) &&
+			    !pci_reset_fn_methods[i].reset_fn(pdev, 1)) {
+				reset_methods[n++] = i;
+				break;
+			}
+		}
+
+		if (i == PCI_NUM_RESET_METHODS) {
+			kfree(options);
+			return -EINVAL;
+		}
+	}
+
+	if (!pci_reset_fn_methods[1].reset_fn(pdev, 1) && reset_methods[0] != 1)
+		pci_warn(pdev, "Device specific reset disabled/de-prioritized by user");
+
+set_reset_methods:
+	memcpy(pdev->reset_methods, reset_methods, sizeof(reset_methods));
+	kfree(options);
+	return count;
+}
+static DEVICE_ATTR_RW(reset_method);
+
+static struct attribute *pci_dev_reset_method_attrs[] = {
+	&dev_attr_reset_method.attr,
+	NULL,
+};
+
+static umode_t pci_dev_reset_method_attr_is_visible(struct kobject *kobj,
+						    struct attribute *a, int n)
+{
+	struct pci_dev *pdev = to_pci_dev(kobj_to_dev(kobj));
+
+	if (!pci_reset_supported(pdev))
+		return 0;
+
+	return a->mode;
+}
+
+static const struct attribute_group pci_dev_reset_method_attr_group = {
+	.attrs = pci_dev_reset_method_attrs,
+	.is_visible = pci_dev_reset_method_attr_is_visible,
+};
+
 static ssize_t reset_store(struct device *dev, struct device_attribute *attr,
 			   const char *buf, size_t count)
 {
@@ -1491,6 +1593,7 @@ const struct attribute_group *pci_dev_groups[] = {
 	&pci_dev_config_attr_group,
 	&pci_dev_rom_attr_group,
 	&pci_dev_reset_attr_group,
+	&pci_dev_reset_method_attr_group,
 	&pci_dev_vpd_attr_group,
 #ifdef CONFIG_DMI
 	&pci_dev_smbios_attr_group,
Signature · 1 line
2.32.0
#29 lore raw

PATCH v10 5/8 PCI: Define a function to set ACPI_COMPANION in pci_dev

In reply to Amey Narkhede

From: Shanker Donthineni <[email protected]>

Move the existing code logic from acpi_pci_bridge_d3() to a separate function pci_set_acpi_fwnode() to set the ACPI fwnode.

No functional change with this patch.

---
 drivers/pci/pci-acpi.c | 12 ++++++++----
 drivers/pci/pci.h      |  2 ++
 2 files changed, 10 insertions(+), 4 deletions(-)
drivers/pci/pci-acpi.c +8 -4
index 36bc23e21..eaddbf701 100644
@@ -934,6 +934,13 @@ static pci_power_t acpi_pci_choose_state(struct pci_dev *pdev)
 
 static struct acpi_device *acpi_pci_find_companion(struct device *dev);
 
+void pci_set_acpi_fwnode(struct pci_dev *dev)
+{
+	if (!ACPI_COMPANION(&dev->dev) && !pci_dev_is_added(dev))
+		ACPI_COMPANION_SET(&dev->dev,
+				   acpi_pci_find_companion(&dev->dev));
+}
+
 static bool acpi_pci_bridge_d3(struct pci_dev *dev)
 {
 	const struct fwnode_handle *fwnode;
@@ -945,11 +952,8 @@ static bool acpi_pci_bridge_d3(struct pci_dev *dev)
 		return false;
 
 	/* Assume D3 support if the bridge is power-manageable by ACPI. */
+	pci_set_acpi_fwnode(dev);
 	adev = ACPI_COMPANION(&dev->dev);
-	if (!adev && !pci_dev_is_added(dev)) {
-		adev = acpi_pci_find_companion(&dev->dev);
-		ACPI_COMPANION_SET(&dev->dev, adev);
-	}
 
 	if (adev && acpi_device_power_manageable(adev))
 		return true;
drivers/pci/pci.h +2 -0
index db1ad94e7..990b73e90 100644
@@ -704,7 +704,9 @@ static inline int pci_aer_raw_clear_status(struct pci_dev *dev) { return -EINVAL
 #ifdef CONFIG_ACPI
 int pci_acpi_program_hp_params(struct pci_dev *dev);
 extern const struct attribute_group pci_dev_acpi_attr_group;
+void pci_set_acpi_fwnode(struct pci_dev *dev);
 #else
+static inline void pci_set_acpi_fwnode(struct pci_dev *dev) {}
 static inline int pci_acpi_program_hp_params(struct pci_dev *dev)
 {
 	return -ENODEV;
Signature · 1 line
2.32.0
Reviewed-by on v10 5/8
#30 lore raw

Re: [PATCH v10 5/8] PCI: Define a function to set ACPI_COMPANION in pci_dev

Quoted 5 lines · On Fri, 9 Jul 2021 18:08:10 +0530 Amey Narkhede <[email protected]>…

From: Shanker Donthineni <[email protected]>

Move the existing code logic from acpi_pci_bridge_d3() to a separate function pci_set_acpi_fwnode() to set the ACPI fwnode.

Quoted 7 lines

No functional change with this patch.

---
 drivers/pci/pci-acpi.c | 12 ++++++++----
 drivers/pci/pci.h      |  2 ++
 2 files changed, 10 insertions(+), 4 deletions(-)
#31 lore raw

PATCH v10 6/8 PCI: Setup ACPI fwnode early and at the same time with OF

In reply to Amey Narkhede

From: Shanker Donthineni <[email protected]>

The pci_dev objects are created through two mechanisms 1) during PCI bus scan and 2) from I/O Virtualization. The fwnode in pci_dev object is being set at different places depends on the type of firmware used, device creation mechanism, and acpi_pci_bridge_d3() WAR.

The software features which have a dependency on ACPI fwnode properties and need to be handled before device_add() will not work. One use case, the software has to check the existence of _RST method to support ACPI based reset method.

This patch does the two changes in order to provide fwnode consistently.
- Set ACPI and OF fwnodes from pci_setup_device().
- Remove pci_set_acpi_fwnode() in acpi_pci_bridge_d3().

After this patch, ACPI/OF firmware properties are visible at the same time during the early stage of pci_dev setup. And also call sites should be able to use firmware agnostic functions device_property_xxx() for the early PCI quirks in the future.

---
 drivers/pci/pci-acpi.c | 1 -
 drivers/pci/probe.c    | 7 ++++---
 2 files changed, 4 insertions(+), 4 deletions(-)
drivers/pci/pci-acpi.c +0 -1
index eaddbf701..dae021322 100644
@@ -952,7 +952,6 @@ static bool acpi_pci_bridge_d3(struct pci_dev *dev)
 		return false;
 
 	/* Assume D3 support if the bridge is power-manageable by ACPI. */
-	pci_set_acpi_fwnode(dev);
 	adev = ACPI_COMPANION(&dev->dev);
 
 	if (adev && acpi_device_power_manageable(adev))
drivers/pci/probe.c +4 -3
index c272e23db..c911d6a5c 100644
@@ -1790,6 +1790,9 @@ int pci_setup_device(struct pci_dev *dev)
 	dev->error_state = pci_channel_io_normal;
 	set_pcie_port_type(dev);
 
+	pci_set_of_node(dev);
+	pci_set_acpi_fwnode(dev);
+
 	pci_dev_assign_slot(dev);
 
 	/*
@@ -1925,6 +1928,7 @@ int pci_setup_device(struct pci_dev *dev)
 	default:				    /* unknown header */
 		pci_err(dev, "unknown header type %02x, ignoring device\n",
 			dev->hdr_type);
+		pci_release_of_node(dev);
 		return -EIO;
 
 	bad:
@@ -2352,10 +2356,7 @@ static struct pci_dev *pci_scan_device(struct pci_bus *bus, int devfn)
 	dev->vendor = l & 0xffff;
 	dev->device = (l >> 16) & 0xffff;
 
-	pci_set_of_node(dev);
-
 	if (pci_setup_device(dev)) {
-		pci_release_of_node(dev);
 		pci_bus_put(dev->bus);
 		kfree(dev);
 		return NULL;
Signature · 1 line
2.32.0
Reviewed-by on v10 6/8
#32 lore raw

Re: [PATCH v10 6/8] PCI: Setup ACPI fwnode early and at the same time with OF

Quoted 21 lines · On Fri, 9 Jul 2021 18:08:11 +0530 Amey Narkhede <[email protected]>…

From: Shanker Donthineni <[email protected]>

The pci_dev objects are created through two mechanisms 1) during PCI bus scan and 2) from I/O Virtualization. The fwnode in pci_dev object is being set at different places depends on the type of firmware used, device creation mechanism, and acpi_pci_bridge_d3() WAR.

The software features which have a dependency on ACPI fwnode properties and need to be handled before device_add() will not work. One use case, the software has to check the existence of _RST method to support ACPI based reset method.

This patch does the two changes in order to provide fwnode consistently.
- Set ACPI and OF fwnodes from pci_setup_device().
- Remove pci_set_acpi_fwnode() in acpi_pci_bridge_d3().

After this patch, ACPI/OF firmware properties are visible at the same time during the early stage of pci_dev setup. And also call sites should be able to use firmware agnostic functions device_property_xxx() for the early PCI quirks in the future.

Quoted 5 lines
---
 drivers/pci/pci-acpi.c | 1 -
 drivers/pci/probe.c    | 7 ++++---
 2 files changed, 4 insertions(+), 4 deletions(-)

Looks ok to me.

#33 lore raw

Re: [PATCH v10 6/8] PCI: Setup ACPI fwnode early and at the same time with OF

In reply to Amey Narkhede
Quoted 6 lines · Amey Narkhede

From: Shanker Donthineni <[email protected]>

The pci_dev objects are created through two mechanisms 1) during PCI bus scan and 2) from I/O Virtualization. The fwnode in pci_dev object is being set at different places depends on the type of firmware used, device creation mechanism, and acpi_pci_bridge_d3() WAR.

WAR?

Quoted 68 lines

The software features which have a dependency on ACPI fwnode properties and need to be handled before device_add() will not work. One use case, the software has to check the existence of _RST method to support ACPI based reset method.

This patch does the two changes in order to provide fwnode consistently.
- Set ACPI and OF fwnodes from pci_setup_device().
- Remove pci_set_acpi_fwnode() in acpi_pci_bridge_d3().

After this patch, ACPI/OF firmware properties are visible at the same time during the early stage of pci_dev setup. And also call sites should be able to use firmware agnostic functions device_property_xxx() for the early PCI quirks in the future.

---
 drivers/pci/pci-acpi.c | 1 -
 drivers/pci/probe.c    | 7 ++++---
 2 files changed, 4 insertions(+), 4 deletions(-)
drivers/pci/pci-acpi.c +0 -1
index eaddbf701..dae021322 100644
@@ -952,7 +952,6 @@ static bool acpi_pci_bridge_d3(struct pci_dev *dev)
 		return false;
 
 	/* Assume D3 support if the bridge is power-manageable by ACPI. */
-	pci_set_acpi_fwnode(dev);
 	adev = ACPI_COMPANION(&dev->dev);
 
 	if (adev && acpi_device_power_manageable(adev))
drivers/pci/probe.c +4 -3
index c272e23db..c911d6a5c 100644
@@ -1790,6 +1790,9 @@ int pci_setup_device(struct pci_dev *dev)
 	dev->error_state = pci_channel_io_normal;
 	set_pcie_port_type(dev);
 
+	pci_set_of_node(dev);
+	pci_set_acpi_fwnode(dev);
+
 	pci_dev_assign_slot(dev);
 
 	/*
@@ -1925,6 +1928,7 @@ int pci_setup_device(struct pci_dev *dev)
 	default:				    /* unknown header */
 		pci_err(dev, "unknown header type %02x, ignoring device\n",
 			dev->hdr_type);
+		pci_release_of_node(dev);
 		return -EIO;
 
 	bad:
@@ -2352,10 +2356,7 @@ static struct pci_dev *pci_scan_device(struct pci_bus *bus, int devfn)
 	dev->vendor = l & 0xffff;
 	dev->device = (l >> 16) & 0xffff;
 
-	pci_set_of_node(dev);
-
 	if (pci_setup_device(dev)) {
-		pci_release_of_node(dev);
 		pci_bus_put(dev->bus);
 		kfree(dev);
 		return NULL;
Signature · 1 line
2.32.0
#34 lore raw

Re: [PATCH v10 6/8] PCI: Setup ACPI fwnode early and at the same time with OF

Hi Bjorn

Quoted 8 lines · Bjorn Helgaas
Quoted 6 lines

From: Shanker Donthineni <[email protected]>

The pci_dev objects are created through two mechanisms 1) during PCI bus scan and 2) from I/O Virtualization. The fwnode in pci_dev object is being set at different places depends on the type of firmware used, device creation mechanism, and acpi_pci_bridge_d3() WAR.

WAR?

Thanks for catching a mistake, will remove it in the next patch.

-Shanker

#35 lore raw

PATCH v10 7/8 PCI: Add support for ACPI _RST reset method

In reply to Amey Narkhede

From: Shanker Donthineni <[email protected]>

The _RST is a standard method specified in the ACPI specification. It provides a function level reset when it is described in the acpi_device context associated with PCI-device. Implement a new reset function pci_dev_acpi_reset() for probing RST method and execute if it is defined in the firmware.

The default priority of the ACPI reset is set to below device-specific and above hardware resets.

---
 drivers/pci/pci-acpi.c | 23 +++++++++++++++++++++++
 drivers/pci/pci.c      |  1 +
 drivers/pci/pci.h      |  6 ++++++
 include/linux/pci.h    |  2 +-
 4 files changed, 31 insertions(+), 1 deletion(-)
drivers/pci/pci-acpi.c +23 -0
index dae021322..b6de71d15 100644
@@ -941,6 +941,29 @@ void pci_set_acpi_fwnode(struct pci_dev *dev)
 				   acpi_pci_find_companion(&dev->dev));
 }
 
+/**
+ * pci_dev_acpi_reset - do a function level reset using _RST method
+ * @dev: device to reset
+ * @probe: check if _RST method is included in the acpi_device context.
+ */
+int pci_dev_acpi_reset(struct pci_dev *dev, int probe)
+{
+	acpi_handle handle = ACPI_HANDLE(&dev->dev);
+
+	if (!handle || !acpi_has_method(handle, "_RST"))
+		return -ENOTTY;
+
+	if (probe)
+		return 0;
+
+	if (ACPI_FAILURE(acpi_evaluate_object(handle, "_RST", NULL, NULL))) {
+		pci_warn(dev, "ACPI _RST failed\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
 static bool acpi_pci_bridge_d3(struct pci_dev *dev)
 {
 	const struct fwnode_handle *fwnode;
drivers/pci/pci.c +1 -0
index d5c16492c..1e64dbd3e 100644
@@ -5115,6 +5115,7 @@ static void pci_dev_restore(struct pci_dev *dev)
 const struct pci_reset_fn_method pci_reset_fn_methods[] = {
 	{ },
 	{ &pci_dev_specific_reset, .name = "device_specific" },
+	{ &pci_dev_acpi_reset, .name = "acpi" },
 	{ &pcie_reset_flr, .name = "flr" },
 	{ &pci_af_flr, .name = "af_flr" },
 	{ &pci_pm_reset, .name = "pm" },
drivers/pci/pci.h +6 -0
index 990b73e90..2c12017ed 100644
@@ -705,7 +705,13 @@ static inline int pci_aer_raw_clear_status(struct pci_dev *dev) { return -EINVAL
 int pci_acpi_program_hp_params(struct pci_dev *dev);
 extern const struct attribute_group pci_dev_acpi_attr_group;
 void pci_set_acpi_fwnode(struct pci_dev *dev);
+int pci_dev_acpi_reset(struct pci_dev *dev, int probe);
 #else
+static inline int pci_dev_acpi_reset(struct pci_dev *dev, int probe)
+{
+	return -ENOTTY;
+}
+
 static inline void pci_set_acpi_fwnode(struct pci_dev *dev) {}
 static inline int pci_acpi_program_hp_params(struct pci_dev *dev)
 {
include/linux/pci.h +1 -1
index f34563831..c3b0d771c 100644
@@ -50,7 +50,7 @@
 			       PCI_STATUS_PARITY)
 
 /* Number of reset methods used in pci_reset_fn_methods array in pci.c */
-#define PCI_NUM_RESET_METHODS 6
+#define PCI_NUM_RESET_METHODS 7
 
 /*
  * The PCI interface treats multi-function devices as independent
Signature · 1 line
2.32.0
#36 lore raw

Re: [PATCH v10 7/8] PCI: Add support for ACPI _RST reset method

Quoted 37 lines · On Fri, 9 Jul 2021 18:08:12 +0530 Amey Narkhede <[email protected]>…

From: Shanker Donthineni <[email protected]>

The _RST is a standard method specified in the ACPI specification. It provides a function level reset when it is described in the acpi_device context associated with PCI-device. Implement a new reset function pci_dev_acpi_reset() for probing RST method and execute if it is defined in the firmware.

The default priority of the ACPI reset is set to below device-specific and above hardware resets.

---
 drivers/pci/pci-acpi.c | 23 +++++++++++++++++++++++
 drivers/pci/pci.c      |  1 +
 drivers/pci/pci.h      |  6 ++++++
 include/linux/pci.h    |  2 +-
 4 files changed, 31 insertions(+), 1 deletion(-)
drivers/pci/pci-acpi.c +8 -0
index dae021322..b6de71d15 100644
@@ -941,6 +941,29 @@ void pci_set_acpi_fwnode(struct pci_dev *dev)
 				   acpi_pci_find_companion(&dev->dev));
 }
 
+/**
+ * pci_dev_acpi_reset - do a function level reset using _RST method
+ * @dev: device to reset
+ * @probe: check if _RST method is included in the acpi_device context.
+ */
+int pci_dev_acpi_reset(struct pci_dev *dev, int probe)
+{
+	acpi_handle handle = ACPI_HANDLE(&dev->dev);

drivers/pci/pci-acpi.c void pci_set_acpi_fwnode(struct pci_dev *dev) · line 961

Quoted 10 lines
drivers/pci/pci-acpi.c +10 -0
+
+	if (!handle || !acpi_has_method(handle, "_RST"))
+		return -ENOTTY;
+
+	if (probe)
+		return 0;
+
+	if (ACPI_FAILURE(acpi_evaluate_object(handle, "_RST", NULL, NULL))) {
+		pci_warn(dev, "ACPI _RST failed\n");
+		return -EINVAL;

Should we return -ENOTTY here instead to give a possible secondary reset method a chance? Thanks,

Alex

Quoted 51 lines
drivers/pci/pci-acpi.c +5 -0
+	}
+
+	return 0;
+}
+
 static bool acpi_pci_bridge_d3(struct pci_dev *dev)
 {
 	const struct fwnode_handle *fwnode;
drivers/pci/pci.c +1 -0
index d5c16492c..1e64dbd3e 100644
@@ -5115,6 +5115,7 @@ static void pci_dev_restore(struct pci_dev *dev)
 const struct pci_reset_fn_method pci_reset_fn_methods[] = {
 	{ },
 	{ &pci_dev_specific_reset, .name = "device_specific" },
+	{ &pci_dev_acpi_reset, .name = "acpi" },
 	{ &pcie_reset_flr, .name = "flr" },
 	{ &pci_af_flr, .name = "af_flr" },
 	{ &pci_pm_reset, .name = "pm" },
drivers/pci/pci.h +6 -0
index 990b73e90..2c12017ed 100644
@@ -705,7 +705,13 @@ static inline int pci_aer_raw_clear_status(struct pci_dev *dev) { return -EINVAL
 int pci_acpi_program_hp_params(struct pci_dev *dev);
 extern const struct attribute_group pci_dev_acpi_attr_group;
 void pci_set_acpi_fwnode(struct pci_dev *dev);
+int pci_dev_acpi_reset(struct pci_dev *dev, int probe);
 #else
+static inline int pci_dev_acpi_reset(struct pci_dev *dev, int probe)
+{
+	return -ENOTTY;
+}
+
 static inline void pci_set_acpi_fwnode(struct pci_dev *dev) {}
 static inline int pci_acpi_program_hp_params(struct pci_dev *dev)
 {
include/linux/pci.h +1 -1
index f34563831..c3b0d771c 100644
@@ -50,7 +50,7 @@
 			       PCI_STATUS_PARITY)
 
 /* Number of reset methods used in pci_reset_fn_methods array in pci.c */
-#define PCI_NUM_RESET_METHODS 6
+#define PCI_NUM_RESET_METHODS 7
 
 /*
  * The PCI interface treats multi-function devices as independent
#37 lore raw

Re: [PATCH v10 7/8] PCI: Add support for ACPI _RST reset method

Hi Alex,

Quoted 12 lines · Alex Williamson
Quoted 12 lines
drivers/pci/pci-acpi.c +12 -0
+/**
+ * pci_dev_acpi_reset - do a function level reset using _RST method
+ * @dev: device to reset
+ * @probe: check if _RST method is included in the acpi_device context.
+ */
+int pci_dev_acpi_reset(struct pci_dev *dev, int probe)
+{
+     acpi_handle handle = ACPI_HANDLE(&dev->dev);
+
+     if (!handle || !acpi_has_method(handle, "_RST"))
+             return -ENOTTY;
+
Quoted 8 lines

drivers/pci/pci-acpi.c void pci_set_acpi_fwnode(struct pci_dev *dev) · line 961

Quoted 6 lines
drivers/pci/pci-acpi.c +6 -0
+     if (probe)
+             return 0;
+
+     if (ACPI_FAILURE(acpi_evaluate_object(handle, "_RST", NULL, NULL))) {
+             pci_warn(dev, "ACPI _RST failed\n");
+             return -EINVAL;

Should we return -ENOTTY here instead to give a possible secondary reset method a chance? Thanks,

Thanks for reviewing patches.

ACPI/AML _RST method type is VOID. The only possibility of failure would be either system is running out of memory or bugs in ACPICA. There is no strong reason not to return -NOTTY.

I'll fix in the next version. Is there opportunity to include reset feature in v5.14-rc2?

Can I add your reviewed-by since no other comments to this patch?

-Shanker

Reviewed-by
#38 lore raw

Re: [PATCH v10 7/8] PCI: Add support for ACPI _RST reset method

Quoted 23 lines · On Mon, 12 Jul 2021 19:51:41 -0500 Shanker R Donthineni…

Hi Alex,

Quoted 12 lines · Alex Williamson
Quoted 12 lines
drivers/pci/pci-acpi.c +12 -0
+/**
+ * pci_dev_acpi_reset - do a function level reset using _RST method
+ * @dev: device to reset
+ * @probe: check if _RST method is included in the acpi_device context.
+ */
+int pci_dev_acpi_reset(struct pci_dev *dev, int probe)
+{
+     acpi_handle handle = ACPI_HANDLE(&dev->dev);
+
+     if (!handle || !acpi_has_method(handle, "_RST"))
+             return -ENOTTY;
+
Quoted 8 lines

drivers/pci/pci-acpi.c void pci_set_acpi_fwnode(struct pci_dev *dev) · line 961

Quoted 6 lines
drivers/pci/pci-acpi.c +6 -0
+     if (probe)
+             return 0;
+
+     if (ACPI_FAILURE(acpi_evaluate_object(handle, "_RST", NULL, NULL))) {
+             pci_warn(dev, "ACPI _RST failed\n");
+             return -EINVAL;  

Should we return -ENOTTY here instead to give a possible secondary reset method a chance? Thanks,

Quoted 7 lines

Thanks for reviewing patches.

ACPI/AML _RST method type is VOID. The only possibility of failure would be either system is running out of memory or bugs in ACPICA. There is no strong reason not to return -NOTTY.

I'll fix in the next version. Is there opportunity to include reset feature in v5.14-rc2?

Sounds good, it's a corner case but since we've got a series of methods we can try and part of the point of Amey's series is giving the user control of the order and methods to try, we might as well make use of it. I think there's also some precedence in the quirks that they can fail and fall through to standard resets.

I'll leave any upstream timing questions to Bjorn, but we've passed the v5.14 merge window when new functionality is generally accepted.

Quoted 1 line

Can I add your reviewed-by since no other comments to this patch?

Yeah, s/-EINVAL/-ENOTTY/

Thanks!

#39 lore raw

PATCH v10 8/8 PCI: Change the type of probe argument in reset functions

In reply to Amey Narkhede

Introduce a new enum pci_reset_mode_t to make the context of probe argument in reset functions clear and the code easier to read. Change the type of probe argument in functions which implement reset methods from int to pci_reset_mode_t to make the intent clear.

Add a new line in return statement of pci_reset_bus_function().

---
 drivers/crypto/cavium/nitrox/nitrox_main.c    |  2 +-
 .../ethernet/cavium/liquidio/lio_vf_main.c    |  2 +-
 drivers/pci/hotplug/pciehp.h                  |  2 +-
 drivers/pci/hotplug/pciehp_hpc.c              |  4 +-
 drivers/pci/hotplug/pnv_php.c                 |  4 +-
 drivers/pci/pci-acpi.c                        | 10 ++-
 drivers/pci/pci-sysfs.c                       |  6 +-
 drivers/pci/pci.c                             | 85 ++++++++++++-------
 drivers/pci/pci.h                             | 12 +--
 drivers/pci/pcie/aer.c                        |  2 +-
 drivers/pci/quirks.c                          | 37 ++++----
 include/linux/pci.h                           |  8 +-
 include/linux/pci_hotplug.h                   |  2 +-
 13 files changed, 107 insertions(+), 69 deletions(-)
drivers/crypto/cavium/nitrox/nitrox_main.c +1 -1
index 15d6c8452..f97fa8e99 100644
@@ -306,7 +306,7 @@ static int nitrox_device_flr(struct pci_dev *pdev)
 		return -ENOMEM;
 	}
 
-	pcie_reset_flr(pdev, 0);
+	pcie_reset_flr(pdev, PCI_RESET_DO_RESET);
 
 	pci_restore_state(pdev);
 
drivers/net/ethernet/cavium/liquidio/lio_vf_main.c +1 -1
index 336d149ee..6e666be69 100644
@@ -526,7 +526,7 @@ static void octeon_destroy_resources(struct octeon_device *oct)
 			oct->irq_name_storage = NULL;
 		}
 		/* Soft reset the octeon device before exiting */
-		if (!pcie_reset_flr(oct->pci_dev, 1))
+		if (!pcie_reset_flr(oct->pci_dev, PCI_RESET_PROBE))
 			octeon_pci_flr(oct);
 		else
 			cn23xx_vf_ask_pf_to_do_flr(oct);
drivers/pci/hotplug/pciehp.h +1 -1
index 4fd200d8b..87da03adc 100644
@@ -181,7 +181,7 @@ void pciehp_release_ctrl(struct controller *ctrl);
 
 int pciehp_sysfs_enable_slot(struct hotplug_slot *hotplug_slot);
 int pciehp_sysfs_disable_slot(struct hotplug_slot *hotplug_slot);
-int pciehp_reset_slot(struct hotplug_slot *hotplug_slot, int probe);
+int pciehp_reset_slot(struct hotplug_slot *hotplug_slot, pci_reset_mode_t mode);
 int pciehp_get_attention_status(struct hotplug_slot *hotplug_slot, u8 *status);
 int pciehp_set_raw_indicator_status(struct hotplug_slot *h_slot, u8 status);
 int pciehp_get_raw_indicator_status(struct hotplug_slot *h_slot, u8 *status);
drivers/pci/hotplug/pciehp_hpc.c +2 -2
index fb3840e22..24b3c8787 100644
@@ -834,14 +834,14 @@ void pcie_disable_interrupt(struct controller *ctrl)
  * momentarily, if we see that they could interfere. Also, clear any spurious
  * events after.
  */
-int pciehp_reset_slot(struct hotplug_slot *hotplug_slot, int probe)
+int pciehp_reset_slot(struct hotplug_slot *hotplug_slot, pci_reset_mode_t mode)
 {
 	struct controller *ctrl = to_ctrl(hotplug_slot);
 	struct pci_dev *pdev = ctrl_dev(ctrl);
 	u16 stat_mask = 0, ctrl_mask = 0;
 	int rc;
 
-	if (probe)
+	if (mode == PCI_RESET_PROBE)
 		return 0;
 
 	down_write(&ctrl->reset_lock);
drivers/pci/hotplug/pnv_php.c +2 -2
index 04565162a..77df598c8 100644
@@ -526,7 +526,7 @@ static int pnv_php_enable(struct pnv_php_slot *php_slot, bool rescan)
 	return 0;
 }
 
-static int pnv_php_reset_slot(struct hotplug_slot *slot, int probe)
+static int pnv_php_reset_slot(struct hotplug_slot *slot, pci_reset_mode_t mode)
 {
 	struct pnv_php_slot *php_slot = to_pnv_php_slot(slot);
 	struct pci_dev *bridge = php_slot->pdev;
@@ -537,7 +537,7 @@ static int pnv_php_reset_slot(struct hotplug_slot *slot, int probe)
 	 * which don't have a bridge. Only claim to support
 	 * reset_slot() if we have a bridge device (for now...)
 	 */
-	if (probe)
+	if (mode == PCI_RESET_PROBE)
 		return !bridge;
 
 	/* mask our interrupt while resetting the bridge */
drivers/pci/pci-acpi.c +7 -3
index b6de71d15..a92ed574a 100644
@@ -944,16 +944,20 @@ void pci_set_acpi_fwnode(struct pci_dev *dev)
 /**
  * pci_dev_acpi_reset - do a function level reset using _RST method
  * @dev: device to reset
- * @probe: check if _RST method is included in the acpi_device context.
+ * @probe: If PCI_RESET_PROBE, check whether _RST method is included
+ *         in the acpi_device context.
  */
-int pci_dev_acpi_reset(struct pci_dev *dev, int probe)
+int pci_dev_acpi_reset(struct pci_dev *dev, pci_reset_mode_t mode)
 {
 	acpi_handle handle = ACPI_HANDLE(&dev->dev);
 
+	if (mode >= PCI_RESET_MODE_MAX)
+		return -EINVAL;
+
 	if (!handle || !acpi_has_method(handle, "_RST"))
 		return -ENOTTY;
 
-	if (probe)
+	if (mode == PCI_RESET_PROBE)
 		return 0;
 
 	if (ACPI_FAILURE(acpi_evaluate_object(handle, "_RST", NULL, NULL))) {
drivers/pci/pci-sysfs.c +4 -2
index 8a740e211..dcf19f6d6 100644
@@ -1393,7 +1393,8 @@ static ssize_t reset_method_store(struct device *dev,
 
 		for (i = 1; i < PCI_NUM_RESET_METHODS; i++) {
 			if (sysfs_streq(name, pci_reset_fn_methods[i].name) &&
-			    !pci_reset_fn_methods[i].reset_fn(pdev, 1)) {
+			    !pci_reset_fn_methods[i].reset_fn(pdev,
+							      PCI_RESET_PROBE)) {
 				reset_methods[n++] = i;
 				break;
 			}
@@ -1405,7 +1406,8 @@ static ssize_t reset_method_store(struct device *dev,
 		}
 	}
 
-	if (!pci_reset_fn_methods[1].reset_fn(pdev, 1) && reset_methods[0] != 1)
+	if (!pci_reset_fn_methods[1].reset_fn(pdev, PCI_RESET_PROBE) &&
+	    reset_methods[0] != 1)
 		pci_warn(pdev, "Device specific reset disabled/de-prioritized by user");
 
 set_reset_methods:
drivers/pci/pci.c +53 -32

266 lines not on this page · raw message

drivers/pci/pci.h +6 -6
index 2c12017ed..06be9e6fa 100644
@@ -604,19 +604,19 @@ static inline int pci_enable_ptm(struct pci_dev *dev, u8 *granularity)
 struct pci_dev_reset_methods {
 	u16 vendor;
 	u16 device;
-	int (*reset)(struct pci_dev *dev, int probe);
+	int (*reset)(struct pci_dev *dev, pci_reset_mode_t mode);
 };
 
 struct pci_reset_fn_method {
-	int (*reset_fn)(struct pci_dev *pdev, int probe);
+	int (*reset_fn)(struct pci_dev *pdev, pci_reset_mode_t mode);
 	char *name;
 };
 
 extern const struct pci_reset_fn_method pci_reset_fn_methods[];
 #ifdef CONFIG_PCI_QUIRKS
-int pci_dev_specific_reset(struct pci_dev *dev, int probe);
+int pci_dev_specific_reset(struct pci_dev *dev, pci_reset_mode_t mode);
 #else
-static inline int pci_dev_specific_reset(struct pci_dev *dev, int probe)
+static inline int pci_dev_specific_reset(struct pci_dev *dev, pci_reset_mode_t mode)
 {
 	return -ENOTTY;
 }
@@ -705,9 +705,9 @@ static inline int pci_aer_raw_clear_status(struct pci_dev *dev) { return -EINVAL
 int pci_acpi_program_hp_params(struct pci_dev *dev);
 extern const struct attribute_group pci_dev_acpi_attr_group;
 void pci_set_acpi_fwnode(struct pci_dev *dev);
-int pci_dev_acpi_reset(struct pci_dev *dev, int probe);
+int pci_dev_acpi_reset(struct pci_dev *dev, pci_reset_mode_t mode);
 #else
-static inline int pci_dev_acpi_reset(struct pci_dev *dev, int probe)
+static inline int pci_dev_acpi_reset(struct pci_dev *dev, pci_reset_mode_t mode)
 {
 	return -ENOTTY;
 }
drivers/pci/pcie/aer.c +1 -1
index 98077595a..cfa7a1775 100644
@@ -1405,7 +1405,7 @@ static pci_ers_result_t aer_root_reset(struct pci_dev *dev)
 	}
 
 	if (type == PCI_EXP_TYPE_RC_EC || type == PCI_EXP_TYPE_RC_END) {
-		rc = pcie_reset_flr(dev, 0);
+		rc = pcie_reset_flr(dev, PCI_RESET_DO_RESET);
 		if (!rc)
 			pci_info(dev, "has been reset\n");
 		else
drivers/pci/quirks.c +21 -16
index e86cf4a3b..e7f15fc02 100644
@@ -3669,7 +3669,7 @@ DECLARE_PCI_FIXUP_SUSPEND_LATE(PCI_VENDOR_ID_INTEL,
  * reset a single function if other methods (e.g. FLR, PM D0->D3) are
  * not available.
  */
-static int reset_intel_82599_sfp_virtfn(struct pci_dev *dev, int probe)
+static int reset_intel_82599_sfp_virtfn(struct pci_dev *dev, pci_reset_mode_t mode)
 {
 	/*
 	 * http://www.intel.com/content/dam/doc/datasheet/82599-10-gbe-controller-datasheet.pdf
@@ -3679,7 +3679,7 @@ static int reset_intel_82599_sfp_virtfn(struct pci_dev *dev, int probe)
 	 * Thus we must call pcie_flr() directly without first checking if it is
 	 * supported.
 	 */
-	if (!probe)
+	if (mode == PCI_RESET_DO_RESET)
 		pcie_flr(dev);
 	return 0;
 }
@@ -3691,13 +3691,13 @@ static int reset_intel_82599_sfp_virtfn(struct pci_dev *dev, int probe)
 #define NSDE_PWR_STATE		0xd0100
 #define IGD_OPERATION_TIMEOUT	10000     /* set timeout 10 seconds */
 
-static int reset_ivb_igd(struct pci_dev *dev, int probe)
+static int reset_ivb_igd(struct pci_dev *dev, pci_reset_mode_t mode)
 {
 	void __iomem *mmio_base;
 	unsigned long timeout;
 	u32 val;
 
-	if (probe)
+	if (mode == PCI_RESET_PROBE)
 		return 0;
 
 	mmio_base = pci_iomap(dev, 0, 0);
@@ -3734,7 +3734,7 @@ static int reset_ivb_igd(struct pci_dev *dev, int probe)
 }
 
 /* Device-specific reset method for Chelsio T4-based adapters */
-static int reset_chelsio_generic_dev(struct pci_dev *dev, int probe)
+static int reset_chelsio_generic_dev(struct pci_dev *dev, pci_reset_mode_t mode)
 {
 	u16 old_command;
 	u16 msix_flags;
@@ -3750,7 +3750,7 @@ static int reset_chelsio_generic_dev(struct pci_dev *dev, int probe)
 	 * If this is the "probe" phase, return 0 indicating that we can
 	 * reset this device.
 	 */
-	if (probe)
+	if (mode == PCI_RESET_PROBE)
 		return 0;
 
 	/*
@@ -3812,17 +3812,17 @@ static int reset_chelsio_generic_dev(struct pci_dev *dev, int probe)
  *    Chapter 3: NVMe control registers
  *    Chapter 7.3: Reset behavior
  */
-static int nvme_disable_and_flr(struct pci_dev *dev, int probe)
+static int nvme_disable_and_flr(struct pci_dev *dev, pci_reset_mode_t mode)
 {
 	void __iomem *bar;
 	u16 cmd;
 	u32 cfg;
 
 	if (dev->class != PCI_CLASS_STORAGE_EXPRESS ||
-	    pcie_reset_flr(dev, 1) || !pci_resource_start(dev, 0))
+	    pcie_reset_flr(dev, PCI_RESET_PROBE) || !pci_resource_start(dev, 0))
 		return -ENOTTY;
 
-	if (probe)
+	if (mode == PCI_RESET_PROBE)
 		return 0;
 
 	bar = pci_iomap(dev, 0, NVME_REG_CC + sizeof(cfg));
@@ -3886,11 +3886,13 @@ static int nvme_disable_and_flr(struct pci_dev *dev, int probe)
  * device too soon after FLR.  A 250ms delay after FLR has heuristically
  * proven to produce reliably working results for device assignment cases.
  */
-static int delay_250ms_after_flr(struct pci_dev *dev, int probe)
+static int delay_250ms_after_flr(struct pci_dev *dev, pci_reset_mode_t mode)
 {
-	int ret = pcie_reset_flr(dev, probe);
+	int ret;
+
+	ret = pcie_reset_flr(dev, mode);
 
-	if (probe)
+	if (ret || mode == PCI_RESET_PROBE)
 		return ret;
 
 	msleep(250);
@@ -3906,13 +3908,13 @@ static int delay_250ms_after_flr(struct pci_dev *dev, int probe)
 #define HINIC_OPERATION_TIMEOUT     15000	/* 15 seconds */
 
 /* Device-specific reset method for Huawei Intelligent NIC virtual functions */
-static int reset_hinic_vf_dev(struct pci_dev *pdev, int probe)
+static int reset_hinic_vf_dev(struct pci_dev *pdev, pci_reset_mode_t mode)
 {
 	unsigned long timeout;
 	void __iomem *bar;
 	u32 val;
 
-	if (probe)
+	if (mode == PCI_RESET_PROBE)
 		return 0;
 
 	bar = pci_iomap(pdev, 0, 0);
@@ -3983,16 +3985,19 @@ static const struct pci_dev_reset_methods pci_dev_reset_methods[] = {
  * because when a host assigns a device to a guest VM, the host may need
  * to reset the device but probably doesn't have a driver for it.
  */
-int pci_dev_specific_reset(struct pci_dev *dev, int probe)
+int pci_dev_specific_reset(struct pci_dev *dev, pci_reset_mode_t mode)
 {
 	const struct pci_dev_reset_methods *i;
 
+	if (mode >= PCI_RESET_MODE_MAX)
+		return -EINVAL;
+
 	for (i = pci_dev_reset_methods; i->reset; i++) {
 		if ((i->vendor == dev->vendor ||
 		     i->vendor == (u16)PCI_ANY_ID) &&
 		    (i->device == dev->device ||
 		     i->device == (u16)PCI_ANY_ID))
-			return i->reset(dev, probe);
+			return i->reset(dev, mode);
 	}
 
 	return -ENOTTY;
include/linux/pci.h +7 -1
index c3b0d771c..0d650c873 100644
@@ -52,6 +52,12 @@
 /* Number of reset methods used in pci_reset_fn_methods array in pci.c */
 #define PCI_NUM_RESET_METHODS 7
 
+typedef enum pci_reset_mode {
+	PCI_RESET_DO_RESET,
+	PCI_RESET_PROBE,
+	PCI_RESET_MODE_MAX,
+} pci_reset_mode_t;
+
 /*
  * The PCI interface treats multi-function devices as independent
  * devices.  The slot/function address of each device is encoded
@@ -1232,7 +1238,7 @@ u32 pcie_bandwidth_available(struct pci_dev *dev, struct pci_dev **limiting_dev,
 			     enum pci_bus_speed *speed,
 			     enum pcie_link_width *width);
 void pcie_print_link_status(struct pci_dev *dev);
-int pcie_reset_flr(struct pci_dev *dev, int probe);
+int pcie_reset_flr(struct pci_dev *dev, pci_reset_mode_t mode);
 int pcie_flr(struct pci_dev *dev);
 int __pci_reset_function_locked(struct pci_dev *dev);
 int pci_reset_function(struct pci_dev *dev);
include/linux/pci_hotplug.h +1 -1
index b482e42d7..9e8da46e7 100644
@@ -44,7 +44,7 @@ struct hotplug_slot_ops {
 	int (*get_attention_status)	(struct hotplug_slot *slot, u8 *value);
 	int (*get_latch_status)		(struct hotplug_slot *slot, u8 *value);
 	int (*get_adapter_status)	(struct hotplug_slot *slot, u8 *value);
-	int (*reset_slot)		(struct hotplug_slot *slot, int probe);
+	int (*reset_slot)		(struct hotplug_slot *slot, pci_reset_mode_t mode);
 };
 
 /**
Signature · 1 line
2.32.0
Reviewed-by on v10 8/8
#40 lore raw

Re: [PATCH v10 8/8] PCI: Change the type of probe argument in reset functions

Quoted 10 lines · On Fri, 9 Jul 2021 18:08:13 +0530 Amey Narkhede <[email protected]>…

Introduce a new enum pci_reset_mode_t to make the context of probe argument in reset functions clear and the code easier to read. Change the type of probe argument in functions which implement reset methods from int to pci_reset_mode_t to make the intent clear.

Add a new line in return statement of pci_reset_bus_function().

Quoted 15 lines
---
 drivers/crypto/cavium/nitrox/nitrox_main.c    |  2 +-
 .../ethernet/cavium/liquidio/lio_vf_main.c    |  2 +-
 drivers/pci/hotplug/pciehp.h                  |  2 +-
 drivers/pci/hotplug/pciehp_hpc.c              |  4 +-
 drivers/pci/hotplug/pnv_php.c                 |  4 +-
 drivers/pci/pci-acpi.c                        | 10 ++-
 drivers/pci/pci-sysfs.c                       |  6 +-
 drivers/pci/pci.c                             | 85 ++++++++++++-------
 drivers/pci/pci.h                             | 12 +--
 drivers/pci/pcie/aer.c                        |  2 +-
 drivers/pci/quirks.c                          | 37 ++++----
 include/linux/pci.h                           |  8 +-
 include/linux/pci_hotplug.h                   |  2 +-
 13 files changed, 107 insertions(+), 69 deletions(-)
#41 lore raw

Re: [PATCH v10 8/8] PCI: Change the type of probe argument in reset functions

In reply to Amey Narkhede
Quoted 4 lines · Amey Narkhede

Introduce a new enum pci_reset_mode_t to make the context of probe argument in reset functions clear and the code easier to read. Change the type of probe argument in functions which implement reset methods from int to pci_reset_mode_t to make the intent clear.

Not sure adding an enum and a PCI_RESET_MODE_MAX seems worth it to me. It's really a boolean parameter, and I'd be happy to change it to a bool. But I don't think it's worth checking against PCI_RESET_MODE_MAX unless we need more than two options.

Quoted 1 line

Add a new line in return statement of pci_reset_bus_function().

Drop this line, since I don't think it's correct and there's no need to describe whitespace changes anyway.

Quoted 29 lines
drivers/crypto/cavium/nitrox/nitrox_main.c +7 -2
@@ -306,7 +306,7 @@ static int nitrox_device_flr(struct pci_dev *pdev)
 		return -ENOMEM;
 	}
 
-	pcie_reset_flr(pdev, 0);
+	pcie_reset_flr(pdev, PCI_RESET_DO_RESET);
 
 	pci_restore_state(pdev);
 
@@ -526,7 +526,7 @@ static void octeon_destroy_resources(struct octeon_device *oct)
 			oct->irq_name_storage = NULL;
 		}
 		/* Soft reset the octeon device before exiting */
-		if (!pcie_reset_flr(oct->pci_dev, 1))
+		if (!pcie_reset_flr(oct->pci_dev, PCI_RESET_PROBE))
 			octeon_pci_flr(oct);
 		else
 			cn23xx_vf_ask_pf_to_do_flr(oct);
 
+typedef enum pci_reset_mode {
+	PCI_RESET_DO_RESET,
+	PCI_RESET_PROBE,
+	PCI_RESET_MODE_MAX,
+} pci_reset_mode_t;
#42 lore raw

Re: [PATCH v10 8/8] PCI: Change the type of probe argument in reset functions

Quoted 11 lines · Bjorn Helgaas
Quoted 4 lines · Amey Narkhede

Introduce a new enum pci_reset_mode_t to make the context of probe argument in reset functions clear and the code easier to read. Change the type of probe argument in functions which implement reset methods from int to pci_reset_mode_t to make the intent clear.

Not sure adding an enum and a PCI_RESET_MODE_MAX seems worth it to me. It's really a boolean parameter, and I'd be happy to change it to a bool. But I don't think it's worth checking against PCI_RESET_MODE_MAX unless we need more than two options.

Is it okay to use PCI_RESET_PROBE and PCI_RESET_DO_RESET as bool. That would be less confusing than directly using true/false.

Thanks,
Amey

Quoted 34 lines
Quoted 1 line

Add a new line in return statement of pci_reset_bus_function().

Drop this line, since I don't think it's correct and there's no need to describe whitespace changes anyway.

Quoted 29 lines
drivers/crypto/cavium/nitrox/nitrox_main.c +7 -2
@@ -306,7 +306,7 @@ static int nitrox_device_flr(struct pci_dev *pdev)
 		return -ENOMEM;
 	}
 
-	pcie_reset_flr(pdev, 0);
+	pcie_reset_flr(pdev, PCI_RESET_DO_RESET);
 
 	pci_restore_state(pdev);
 
@@ -526,7 +526,7 @@ static void octeon_destroy_resources(struct octeon_device *oct)
 			oct->irq_name_storage = NULL;
 		}
 		/* Soft reset the octeon device before exiting */
-		if (!pcie_reset_flr(oct->pci_dev, 1))
+		if (!pcie_reset_flr(oct->pci_dev, PCI_RESET_PROBE))
 			octeon_pci_flr(oct);
 		else
 			cn23xx_vf_ask_pf_to_do_flr(oct);
 
+typedef enum pci_reset_mode {
+	PCI_RESET_DO_RESET,
+	PCI_RESET_PROBE,
+	PCI_RESET_MODE_MAX,
+} pci_reset_mode_t;
#43 lore raw

Re: [PATCH v10 8/8] PCI: Change the type of probe argument in reset functions

Quoted 7 lines · Amey Narkhede
Quoted 6 lines · Bjorn Helgaas
Quoted 4 lines · Amey Narkhede

Introduce a new enum pci_reset_mode_t to make the context of probe argument in reset functions clear and the code easier to read. Change the type of probe argument in functions which implement reset methods from int to pci_reset_mode_t to make the intent clear.

Quoted 7 lines
Quoted 5 lines

Not sure adding an enum and a PCI_RESET_MODE_MAX seems worth it to me. It's really a boolean parameter, and I'd be happy to change it to a bool. But I don't think it's worth checking against PCI_RESET_MODE_MAX unless we need more than two options.

Is it okay to use PCI_RESET_PROBE and PCI_RESET_DO_RESET as bool. That would be less confusing than directly using true/false.

You mean like this?

  #define PCI_RESET_DO_RESET  false
  #define PCI_RESET_PROBE     true

I don't think there's a huge amount of value, but I guess that's OK as long as it's confined to drivers/pci/, i.e., not exposed via include/linux/pci.h.

#44 lore raw
In reply to Amey Narkhede
Quoted 14 lines · Amey Narkhede

PCI and PCIe devices may support a number of possible reset mechanisms for example Function Level Reset (FLR) provided via Advanced Feature or PCIe capabilities, Power Management reset, bus reset, or device specific reset. Currently the PCI subsystem creates a policy prioritizing these reset methods which provides neither visibility nor control to userspace.

Expose the reset methods available per device to userspace, via sysfs and allow an administrative user or device owner to have ability to manage per device reset method priorities or exclusions. This feature aims to allow greater control of a device for use cases as device assignment, where specific device or platform issues may interact poorly with a given reset method, and for which device specific quirks have not been developed.

Quoted 3 lines

Changes in v10:
- Fix build error on ppc as reported by build bot

Aplogies for late response. For some reason I did not get email from test bot. I checked spam folder too. Not sure if gmail messed something up.

[...]
Amey