Skip to contentSkip to conversation

pci: New PCI-E reset API

PATCH In review Brian King 2 participants 6 messages Feb 1, 2007 – Feb 9, 2007

Subsystems (5)
lore
Share & tools
lore

Brian King

author
#0 lore raw

PATCH 1/3 pci: New PCI-E reset API

Adds a new API which can be used to issue various types of PCI-E reset, including PCI-E warm reset and PCI-E hot reset. This is needed for an ipr PCI-E adapter which does not properly implement BIST. Running BIST on this adapter results in PCI-E errors. The only reliable reset mechanism that exists on this hardware is PCI Fundamental reset (warm reset). Since driving this type of reset is architecture unique, this provides the necessary hooks for architectures to add this support.

---

 linux-2.6-bjking1/drivers/pci/pci.c   |   29 +++++++++++++++++++++++++++++
 linux-2.6-bjking1/include/linux/pci.h |   14 ++++++++++++++
 2 files changed, 43 insertions(+)
drivers/pci/pci.c renamed from drivers/pci/pci.c~pci_pci_reset_api +29 -0
diff -puN drivers/pci/pci.c~pci_pci_reset_api drivers/pci/pci.c
@@ -791,6 +791,34 @@ pci_disable_device(struct pci_dev *dev)
 }
 
 /**
+ * pcibios_set_pcie_reset_state - set reset state for device dev
+ * @dev: the PCI-E device reset
+ * @state: Reset state to enter into
+ *
+ *
+ * Sets the PCI-E reset state for the device. This is the default
+ * implementation. Architecture implementations can override this.
+ */
+int __attribute__ ((weak)) pcibios_set_pcie_reset_state(struct pci_dev *dev,
+							enum pcie_reset_state state)
+{
+	return -EINVAL;
+}
+
+/**
+ * pci_set_pcie_reset_state - set reset state for device dev
+ * @dev: the PCI-E device reset
+ * @state: Reset state to enter into
+ *
+ *
+ * Sets the PCI reset state for the device.
+ */
+int pci_set_pcie_reset_state(struct pci_dev *dev, enum pcie_reset_state state)
+{
+	return pcibios_set_pcie_reset_state(dev, state);
+}
+
+/**
  * pci_enable_wake - enable device to generate PME# when suspended
  * @dev: - PCI device to operate on
  * @state: - Current state of device.
@@ -1210,6 +1238,7 @@ EXPORT_SYMBOL(pci_set_power_state);
 EXPORT_SYMBOL(pci_save_state);
 EXPORT_SYMBOL(pci_restore_state);
 EXPORT_SYMBOL(pci_enable_wake);
+EXPORT_SYMBOL_GPL(pci_set_pcie_reset_state);
 
 /* Quirk info */
 
include/linux/pci.h renamed from include/linux/pci.h~pci_pci_reset_api +14 -0
diff -puN include/linux/pci.h~pci_pci_reset_api include/linux/pci.h
@@ -96,6 +96,19 @@ enum pci_channel_state {
 	pci_channel_io_perm_failure = (__force pci_channel_state_t) 3,
 };
 
+typedef unsigned int __bitwise pcie_reset_state_t;
+
+enum pcie_reset_state {
+	/* Reset is NOT asserted (Use to deassert reset) */
+	pci_reset_normal = (__force pcie_reset_state_t) 1,
+
+	/* Use #PERST to reset PCI-E device */
+	pci_reset_pcie_warm_reset = (__force pcie_reset_state_t) 2,
+
+	/* Use PCI-E Hot Reset to reset device */
+	pci_reset_pcie_hot_reset = (__force pcie_reset_state_t) 3
+};
+
 typedef unsigned short __bitwise pci_bus_flags_t;
 enum pci_bus_flags {
 	PCI_BUS_FLAGS_NO_MSI = (__force pci_bus_flags_t) 1,
@@ -523,6 +536,7 @@ int __must_check pci_enable_device(struc
 int __must_check pci_enable_device_bars(struct pci_dev *dev, int mask);
 void pci_disable_device(struct pci_dev *dev);
 void pci_set_master(struct pci_dev *dev);
+int pci_set_pcie_reset_state(struct pci_dev *dev, enum pcie_reset_state state);
 #define HAVE_PCI_SET_MWI
 int __must_check pci_set_mwi(struct pci_dev *dev);
 void pci_clear_mwi(struct pci_dev *dev);

_

author
#1 lore raw

PATCH 2/3 powerpc: Add powerpc PCI-E reset API implementation

Adds the pSeries platform implementation for a new PCI API which can be used to issue various types of PCI-E reset, including PCI-E warm reset and PCI-E hot reset. This is needed for an ipr PCI-E adapter which does not properly implement BIST. Running BIST on this adapter results in PCI-E errors. The only reliable reset mechanism that exists on this hardware is PCI Fundamental reset (warm reset).

---

 linux-2.6-bjking1/arch/powerpc/platforms/pseries/eeh.c |   30 +++++++++++++++++
 1 files changed, 30 insertions(+)
arch/powerpc/platforms/pseries/eeh.c renamed from arch/powerpc/platforms/pseries/eeh.c~powerpc_slot_reset_api3 +30 -0
diff -puN arch/powerpc/platforms/pseries/eeh.c~powerpc_slot_reset_api3 arch/powerpc/platforms/pseries/eeh.c
@@ -561,6 +561,36 @@ rtas_pci_slot_reset(struct pci_dn *pdn, 
 }
 
 /**
+ * pcibios_set_pcie_slot_reset - Set PCI-E reset state
+ * @dev:	pci device struct
+ * @state:	reset state to enter
+ *
+ * Return value:
+ * 	0 if success
+ **/
+int pcibios_set_pcie_reset_state(struct pci_dev *dev, enum pcie_reset_state state)
+{
+	struct device_node *dn = pci_device_to_OF_node(dev);
+	struct pci_dn *pdn = PCI_DN(dn);
+
+	switch (state) {
+	case pci_reset_normal:
+		rtas_pci_slot_reset(pdn, 0);
+		break;
+	case pci_reset_pcie_hot_reset:
+		rtas_pci_slot_reset(pdn, 1);
+		break;
+	case pci_reset_pcie_warm_reset:
+		rtas_pci_slot_reset(pdn, 3);
+		break;
+	default:
+		return -EINVAL;
+	};
+
+	return 0;
+}
+
+/**
  * rtas_set_slot_reset -- assert the pci #RST line for 1/4 second
  * @pdn: pci device node to be reset.
  *

_

author
#2 lore raw

PATCH 3/3 ipr: Use PCI-E reset API for new ipr adapter

In reply to Brian King

Use a newly added PCI API to issue a PCI Fundamental reset (warm reset) to a new ipr PCI-E adapter. Typically, the ipr adapter uses the start BIST bit in config space to reset an adapter. Issuing start BIST on this particular adapter results in the PCI-E logic on the card losing sync, which causes PCI-E errors, making the card unusable. The only reset mechanism that exists on this hardware that does not have this problem is PCI Fundamental reset (warm reset).

---

 linux-2.6-bjking1/drivers/scsi/ipr.c |   61 ++++++++++++++++++++++++++++++++---
 linux-2.6-bjking1/drivers/scsi/ipr.h |    7 ++--
 2 files changed, 61 insertions(+), 7 deletions(-)
drivers/scsi/ipr.h renamed from drivers/scsi/ipr.h~ipr_pci_reset6 +5 -2
diff -puN drivers/scsi/ipr.h~ipr_pci_reset6 drivers/scsi/ipr.h
@@ -37,8 +37,8 @@
 /*
  * Literals
  */
-#define IPR_DRIVER_VERSION "2.3.1"
-#define IPR_DRIVER_DATE "(January 23, 2007)"
+#define IPR_DRIVER_VERSION "2.3.2"
+#define IPR_DRIVER_DATE "(January 26, 2007)"
 
 /*
  * IPR_MAX_CMD_PER_LUN: This defines the maximum number of outstanding
@@ -182,6 +182,7 @@
 #define IPR_WAIT_FOR_RESET_TIMEOUT		(2 * HZ)
 #define IPR_CHECK_FOR_RESET_TIMEOUT		(HZ / 10)
 #define IPR_WAIT_FOR_BIST_TIMEOUT		(2 * HZ)
+#define IPR_PCI_RESET_TIMEOUT			(HZ / 2)
 #define IPR_DUMP_TIMEOUT			(15 * HZ)
 
 /*
@@ -1058,6 +1059,7 @@ struct ipr_ioa_cfg {
 	u8 allow_cmds:1;
 	u8 allow_ml_add_del:1;
 	u8 needs_hard_reset:1;
+	u8 needs_warm_reset:1;
 
 	enum ipr_cache_state cache_state;
 	u16 type; /* CCIN of the card */
@@ -1150,6 +1152,7 @@ struct ipr_ioa_cfg {
 	struct pci_pool *ipr_cmd_pool;
 
 	struct ipr_cmnd *reset_cmd;
+	int (*reset) (struct ipr_cmnd *);
 
 	struct ata_host ata_host;
 	char ipr_cmd_label[8];
drivers/scsi/ipr.c renamed from drivers/scsi/ipr.c~ipr_pci_reset6 +56 -5

97 lines not on this page · raw message

_

#3 lore raw
In reply to Brian King
Quoted 8 lines · Brian King

Adds a new API which can be used to issue various types of PCI-E reset, including PCI-E warm reset and PCI-E hot reset. This is needed for an ipr PCI-E adapter which does not properly implement BIST. Running BIST on this adapter results in PCI-E errors. The only reliable reset mechanism that exists on this hardware is PCI Fundamental reset (warm reset). Since driving this type of reset is architecture unique, this provides the necessary hooks for architectures to add this support.

A few points ...

- When doing a warm reset, you reset the entire device not just the function (== pci_dev) that gets passed in. How happy are drivers for the other functions going to be about this?

- You've missed the requirement:

"To allow components to perform internal initialization, system software must wait for at least 100 ms from the end of a Conventional Reset of one or more devices before it is permitted to issue Configuration Requests to those devices."

To fix this, we need to call pci_block_user_cfg_access() before calling the pcibios function, then msleep(100) after calling it, then call pci_unblock_user_cfg_access().

- There's no attempt to support either cold or function-level reset in this patch.

I suspect the Right Way of handling hot/warm/cold reset is going to be some kind of integration with error handling. This driver understands about slots being different from functions, and has the ability to notify drivers of other functions that a reset is happening.

To a certain extent, what's going on with IPR here *is* an error condition, it's just that we can recover from it with a warm reset rather than a hot reset.

author
#4 lore raw
Quoted 10 lines · Matthew Wilcox
Quoted 8 lines · Brian King

Adds a new API which can be used to issue various types of PCI-E reset, including PCI-E warm reset and PCI-E hot reset. This is needed for an ipr PCI-E adapter which does not properly implement BIST. Running BIST on this adapter results in PCI-E errors. The only reliable reset mechanism that exists on this hardware is PCI Fundamental reset (warm reset). Since driving this type of reset is architecture unique, this provides the necessary hooks for architectures to add this support.

Quoted 5 lines

A few points ...

- When doing a warm reset, you reset the entire device not just the function (== pci_dev) that gets passed in. How happy are drivers for the other functions going to be about this?

I guess I don't see how a warm reset could be issued to a single function of a PCI device. I would argue that for a multi-function device, you would have to use function level reset.

Quoted 10 lines

- You've missed the requirement:

"To allow components to perform internal initialization, system software must wait for at least 100 ms from the end of a Conventional Reset of one or more devices before it is permitted to issue Configuration Requests to those devices."

To fix this, we need to call pci_block_user_cfg_access() before calling the pcibios function, then msleep(100) after calling it, then call pci_unblock_user_cfg_access().

What I've done is to provide a very low-level API that can be used to accomplish this. In my implementation, the ipr driver is the one doing all the required delays and calling pci_block_user_cfg_access, since it already was doing that in order to run BIST on the adapter.

Quoted 2 lines

- There's no attempt to support either cold or function-level reset in this patch.

Correct. I had no requirement to implement this. It can always be added if there is a need. A function level reset can be performed by simply writing a bit in config space, so *technically* we wouldn't need an API to do that for us, but it could certainly be added here.

Quoted 4 lines

I suspect the Right Way of handling hot/warm/cold reset is going to be some kind of integration with error handling. This driver understands about slots being different from functions, and has the ability to notify drivers of other functions that a reset is happening.

Perhaps. It would require a way for the adapter device driver to indicate what type of reset(s) will work for a particular pci device. It would also require a method for a device driver to invoke a reset, which does not currently exist today. I think it would be the first case of the device driver invoking pci error recovery, so I'm not sure how difficult that would be to do with the current code.

I actually thought this API might be used by PCI error recovery code, since it may need to perform these sorts of functions.

CC'ing Linas Vepstas since he wrote the powerpc pci recovery code.

Brian

Signature · 3 lines
Brian King
eServer Storage I/O
IBM Linux Technology Center
author
#5 lore raw

Matthew,

Any further comments on this?

Thanks,

Brian

Quoted 65 lines · Brian King
Quoted 9 lines · Matthew Wilcox
Quoted 8 lines · Brian King

Adds a new API which can be used to issue various types of PCI-E reset, including PCI-E warm reset and PCI-E hot reset. This is needed for an ipr PCI-E adapter which does not properly implement BIST. Running BIST on this adapter results in PCI-E errors. The only reliable reset mechanism that exists on this hardware is PCI Fundamental reset (warm reset). Since driving this type of reset is architecture unique, this provides the necessary hooks for architectures to add this support.

Quoted 5 lines

A few points ...

- When doing a warm reset, you reset the entire device not just the function (== pci_dev) that gets passed in. How happy are drivers for the other functions going to be about this?

I guess I don't see how a warm reset could be issued to a single function of a PCI device. I would argue that for a multi-function device, you would have to use function level reset.

Quoted 10 lines

- You've missed the requirement:

"To allow components to perform internal initialization, system software must wait for at least 100 ms from the end of a Conventional Reset of one or more devices before it is permitted to issue Configuration Requests to those devices."

To fix this, we need to call pci_block_user_cfg_access() before calling the pcibios function, then msleep(100) after calling it, then call pci_unblock_user_cfg_access().

What I've done is to provide a very low-level API that can be used to accomplish this. In my implementation, the ipr driver is the one doing all the required delays and calling pci_block_user_cfg_access, since it already was doing that in order to run BIST on the adapter.

Quoted 2 lines

- There's no attempt to support either cold or function-level reset in this patch.

Correct. I had no requirement to implement this. It can always be added if there is a need. A function level reset can be performed by simply writing a bit in config space, so *technically* we wouldn't need an API to do that for us, but it could certainly be added here.

Quoted 4 lines

I suspect the Right Way of handling hot/warm/cold reset is going to be some kind of integration with error handling. This driver understands about slots being different from functions, and has the ability to notify drivers of other functions that a reset is happening.

Perhaps. It would require a way for the adapter device driver to indicate what type of reset(s) will work for a particular pci device. It would also require a method for a device driver to invoke a reset, which does not currently exist today. I think it would be the first case of the device driver invoking pci error recovery, so I'm not sure how difficult that would be to do with the current code.

I actually thought this API might be used by PCI error recovery code, since it may need to perform these sorts of functions.

CC'ing Linas Vepstas since he wrote the powerpc pci recovery code.

Brian

Signature · 3 lines
Brian King
eServer Storage I/O
IBM Linux Technology Center