Skip to contentSkip to conversation

Support for PCI Express reset type

PATCH In review Mike Mason 5 participants 6 messages Jul 30, 2009 – Aug 14, 2009

Subsystems (4)
lore
Share & tools
lore

Mike Mason

author
#0 lore raw

PATCH 1/3 Support for PCI Express reset type

This is the first of three patches that implement a bit field that PCI Express device drivers can use to indicate they need a fundamental reset during error recovery.

By default, the EEH framework on powerpc does what's known as a "hot reset" during recovery of a PCI Express device. We've found a case where the device needs a "fundamental reset" to recover properly. The current PCI error recovery and EEH frameworks do not support this distinction.

The attached patch (courtesy of Richard Lary) adds a bit field to pci_dev that indicates whether the device requires a fundamental reset during recovery.

These patches supersede the previously submitted patch that implemented a fundamental reset bit field.

Please review and let me know of any concerns.

include/linux/pci.h +1 -0
diff -uNrp a/include/linux/pci.h b/include/linux/pci.h
@@ -273,6 +273,7 @@ struct pci_dev {
 	unsigned int	ari_enabled:1;	/* ARI forwarding */
 	unsigned int	is_managed:1;
 	unsigned int	is_pcie:1;
+	unsigned int    needs_freset:1; /* Dev requires fundamental reset */
 	unsigned int	state_saved:1;
 	unsigned int	is_physfn:1;
 	unsigned int	is_virtfn:1;
#1 lore raw
Quoted 5 lines · Mike Mason

These patches supersede the previously submitted patch that implemented a fundamental reset bit field.

Please review and let me know of any concerns.

Any plans to implement that for x86 too? Right now it seems to be a PPC specific hack. And where is the driver that is using it?

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
#2 lore raw
Quoted 9 lines · Andi Kleen
Quoted 5 lines · Mike Mason

These patches supersede the previously submitted patch that implemented a fundamental reset bit field.

Please review and let me know of any concerns.

Any plans to implement that for x86 too? Right now it seems to be a PPC specific hack.

Are there any non-PPC platforms which support EEH slot-reset?

Quoted 1 line

And where is the driver that is using it?

That would be the qla2xxx (FC/FCoE storage) and qlge (10gb NIC ) drivers. Changes to the drivers could look something like:

drivers/net/qlge/qlge_main.c +3 -0
Index: b/drivers/net/qlge/qlge_main.c
===================================================================
@@ -4289,6 +4289,9 @@ static int __devinit qlge_probe(struct p
 		return err;
 	}
 
+	/* Set EEH reset type to fundamental for this device */
+	pdev->needs_freset = 1;
+
 	qdev = netdev_priv(ndev);
 	SET_NETDEV_DEV(ndev, &pdev->dev);
 	ndev->features = (0
drivers/scsi/qla2xxx/qla_os.c +4 -0
Index: b/drivers/scsi/qla2xxx/qla_os.c
===================================================================
@@ -1773,6 +1773,10 @@ qla2x00_probe_one(struct pci_dev *pdev, 
 	/* Set ISP-type information. */
 	qla2x00_set_isp_flags(ha);
 
+	/* Set EEH reset type to fundamental if required by hba  */
+	if (IS_QLA24XX(ha) || IS_QLA25XX(ha) || IS_QLA81XX(ha))
+		pdev->needs_freset = 1;
+
 	/* Configure PCI I/O space */
 	ret = qla2x00_iospace_config(ha);
 	if (ret)

once the infrastructure changes were hashed out.

-- av

#3 lore raw
In reply to Andi Kleen

Hi Andi,

2009/7/31 Andi Kleen <[email protected]>:

Quoted 9 lines
Quoted 5 lines · Mike Mason

These patches supersede the previously submitted patch that implemented a fundamental reset bit field.

Please review and let me know of any concerns.

Any plans to implement that for x86 too? Right now it seems to be a PPC specific hack.

I've found the PCIE chipsepc somewhat daunting, but was under the impression that much if not most of what was needed was specified there.

See, for example:
Documentation/PCI/pcieaer-howto.txt

which states:

|||   The PCI Express Advanced Error Reporting Driver Guide HOWTO
|||                T. Long Nguyen  <[email protected]>
|||                Yanmin Zhang    <[email protected]>
|||                                07/29/2006
[..]
||| The PCI Express AER driver provides the infrastructure to support PCI
||| Express Advanced Error Reporting capability. The PCI Express AER
||| driver provides three basic functions:
|||
||| -       Gathers the comprehensive error information if errors occurred.
||| -       Reports error to the users.
||| -       Performs error recovery actions.

I presume the last bullet point means that the AER code works and actually does more or less the same thing as the PPC EEH code, but in a more architecture-independent way, as it only assumes that PCI AER is there (and is correctly implemented in the CPI chipset) The AER code uses the same core infrastructure as the EEH code, at the time, I did exchange emails w/ the above authors discussing this stuff.

As to whether the x86 server vendors are actually selling something with AER in it, and whether any of them are actually testing this stuff is unclear.

FWIW IBM has pretty much no incentive to lobby other server vendors to get on the ball ...as this is viewed as one of those things that lets IBM charge premium prices for PPC hardware.

--linas

#4 lore raw
In reply to Mike Mason

2009/7/30 Mike Mason <[email protected]>:

Quoted 5 lines

This is the first of three patches that implement a bit field that PCI Express device drivers can use to indicate they need a fundamental reset during error recovery.

By default, the EEH framework on powerpc does what's known as a "hot rese=

t"

Quoted 1 line

during recovery of a PCI Express device. =C2=A0We've found a case where t=

he

Quoted 1 line

device needs a "fundamental reset" to recover properly. =C2=A0The current=

PCI

Quoted 6 lines

error recovery and EEH frameworks do not support this distinction.

The attached patch (courtesy of Richard Lary) adds a bit field to pci_dev that indicates whether the device requires a fundamental reset during recovery.

Quoted 6 lines

These patches supersede the previously submitted patch that implemented a fundamental reset bit field.
Please review and let me know of any concerns.

#5 lore raw
In reply to Mike Mason
Quoted 14 lines · On Thu, 30 Jul 2009 15:33:21 -0700 Mike Mason <[email protected]> wrote:

This is the first of three patches that implement a bit field that PCI Express device drivers can use to indicate they need a fundamental reset during error recovery.

By default, the EEH framework on powerpc does what's known as a "hot reset" during recovery of a PCI Express device. We've found a case where the device needs a "fundamental reset" to recover properly. The current PCI error recovery and EEH frameworks do not support this distinction.

The attached patch (courtesy of Richard Lary) adds a bit field to pci_dev that indicates whether the device requires a fundamental reset during recovery.

Quoted 7 lines

These patches supersede the previously submitted patch that implemented a fundamental reset bit field.

Please review and let me know of any concerns.

Ok, applied this series to my linux-next branch, it looks pretty reasonable to me.

For future patches, please cc me, and include the subsystem in the subject, along with a specific description of the patch, e.g. "PCI: add PCIe fundamental reset interface", "PCI: document PCIe fundamental reset", or for arch specific patches, "PCI/powerpc: implement support for PCIe fundamental reset".

Thanks,

Signature · 1 line
Jesse Barnes, Intel Open Source Technology Center