PCIe Hotplug Slot Emulation driver
Subsystems (3)
cover PCIe Hotplug Slot Emulation driver
This set adds an emulation driver for PCIe Hotplug. There may be platforms with specific configurations that can support hotplug but don't provide the logical slot hotplug hardware. For instance, the platform may use an electrically-tolerant interposer between the slot and the device.
This driver utilizes the pci-bridge-emul architecture to manage register reads and writes. The underlying functionality of the hotplug emulation driver uses the Data Link Layer Link Active Reporting mechanism in a polling loop, but can tolerate other event sources such as AER or DPC.
When enabled and a slot is managed by the driver, all port services are managed by the kernel. This is done to ensure that firmware hotplug and error architecture does not (correctly) halt/machine check the system when hotplug is performed on a non-hotplug slot.
The driver offers two active mode: Auto and Force.
auto: The driver will bind to non-hotplug slots
force: The driver will bind to all slots and overrides the slot's services
There are three kernel params:
pciehp.pciehp_emul_mode={off, auto, force} pciehp.pciehp_emul_time=<msecs polling time> (def 1000, min 100, max 60000) pciehp.pciehp_emul_ports=<PCI [S]BDF/ID format string>
The pciehp_emul_ports kernel parameter takes a semi-colon tokenized string representing PCI [S]BDFs and IDs. The pciehp_emul_mode will then be applied to only those slots, leaving other slots unmanaged by pciehp_emul.
The string follows the pci_dev_str_match() format:
[<domain>:]<bus>:<device>.<func>[/<device>.<func>]* pci:<vendor>:<device>[:<subvendor>:<subdevice>]
When using the path format, the path for the device can be obtained using 'lspci -t' and further specified using the upstream bridge and the downstream port's device-function to be more robust against bus renumbering.
When using the vendor-device format, a value of '0' in any field acts as a wildcard for that field, matching all values.
The driver is enabled with CONFIG_HOTPLUG_PCI_PCIE_EMUL=y.
The driver should be considered 'use at own risk' unless the platform/hardware vendor recommends this mode.
Jon Derrick (9):
PCI: pci-bridge-emul: Update PCIe register behaviors
PCI: pci-bridge-emul: Eliminate reserved member
PCI: pci-bridge-emul: Provide a helper to set behavior
PCI: pciehp: Indirect slot register operations
PCI: Add pcie_port_slot_emulated stub
PCI: pciehp: Expose the poll loop to other drivers
PCI: Move pci_dev_str_match to search.c
PCI: pciehp: Add hotplug slot emulation driver
PCI: pciehp: Wire up pcie_port_emulate_slot and pciehp_emul
drivers/pci/hotplug/Makefile | 4 + drivers/pci/hotplug/pciehp.h | 28 +++ drivers/pci/hotplug/pciehp_emul.c | 378 ++++++++++++++++++++++++++++++++++++++ drivers/pci/hotplug/pciehp_hpc.c | 136 ++++++++++---- drivers/pci/pci-acpi.c | 3 + drivers/pci/pci-bridge-emul.c | 95 +++++----- drivers/pci/pci-bridge-emul.h | 10 + drivers/pci/pci.c | 163 ---------------- drivers/pci/pcie/Kconfig | 14 ++ drivers/pci/pcie/portdrv_core.c | 14 +- drivers/pci/probe.c | 2 +- drivers/pci/search.c | 162 ++++++++++++++++ include/linux/pci.h | 8 + 13 files changed, 775 insertions(+), 242 deletions(-) create mode 100644 drivers/pci/hotplug/pciehp_emul.c
Signature · 1 line
1.8.3.1
RFC 1/9 PCI: pci-bridge-emul: Update PCIe register behaviors
Update the PCIe register behaviors and comments for PCIe v5.0. Replace the specific bit definitions with BIT and GENMASK to make updating easier in the future.
--- drivers/pci/pci-bridge-emul.c | 54 +++++++++++++++++++++---------------------- 1 file changed, 27 insertions(+), 27 deletions(-)
drivers/pci/pci-bridge-emul.c +27 -27
@@ -191,12 +191,12 @@ struct pci_bridge_reg_behavior { .rw = GENMASK(15, 0),/* -* Device status register has 4 bits W1C, then 2 bits -* RO, the rest is reserved +* Device status register has bits 6 and [3:0] W1C, [5:4] RO, +* the rest is reserved */ - .w1c = GENMASK(19, 16), - .ro = GENMASK(20, 19), - .rsvd = GENMASK(31, 21), + .w1c = (BIT(6) | GENMASK(3, 0)) << 16, + .ro = GENMASK(5, 4) << 16, + .rsvd = GENMASK(15, 7) << 16, }, [PCI_EXP_LNKCAP / 4] = {
@@ -207,15 +207,16 @@ struct pci_bridge_reg_behavior { [PCI_EXP_LNKCTL / 4] = {/* -* Link control has bits [1:0] and [11:3] RW, the -* other bits are reserved. -* Link status has bits [13:0] RO, and bits [14:15] +* Link control has bits [15:14], [11:3] and [1:0] RW, the +* rest is reserved. +* +* Link status has bits [13:0] RO, and bits [15:14] * W1C. */ - .rw = GENMASK(11, 3) | GENMASK(1, 0), + .rw = GENMASK(15, 14) | GENMASK(11, 3) | GENMASK(1, 0), .ro = GENMASK(13, 0) << 16, .w1c = GENMASK(15, 14) << 16, - .rsvd = GENMASK(15, 12) | BIT(2), + .rsvd = GENMASK(13, 12) | BIT(2), }, [PCI_EXP_SLTCAP / 4] = {
@@ -224,19 +225,16 @@ struct pci_bridge_reg_behavior { [PCI_EXP_SLTCTL / 4] = {/* -* Slot control has bits [12:0] RW, the rest is +* Slot control has bits [14:0] RW, the rest is * reserved. * -* Slot status has a mix of W1C and RO bits, as well -* as reserved bits. +* Slot status has bits 8 and [4:0] W1C, bits [7:5] RO, the +* rest is reserved. */ - .rw = GENMASK(12, 0), - .w1c = (PCI_EXP_SLTSTA_ABP |PCI_EXP_SLTSTA_PFD | -PCI_EXP_SLTSTA_MRLSC |PCI_EXP_SLTSTA_PDC | - PCI_EXP_SLTSTA_CC | PCI_EXP_SLTSTA_DLLSC) << 16, - .ro = (PCI_EXP_SLTSTA_MRLSS |PCI_EXP_SLTSTA_PDS | - PCI_EXP_SLTSTA_EIS) << 16, - .rsvd = GENMASK(15, 12) | (GENMASK(15, 9) << 16), + .rw = GENMASK(14, 0), + .w1c = (BIT(8) | GENMASK(4, 0)) << 16, + .ro = GENMASK(7, 5) << 16, + .rsvd = BIT(15) | (GENMASK(15, 9) << 16), }, [PCI_EXP_RTCTL / 4] = {
@@ -244,18 +242,20 @@ struct pci_bridge_reg_behavior {* Root control has bits [4:0] RW, the rest is * reserved. * -* Root status has bit 0 RO, the rest is reserved. +* Root capabilities has bit 0 RO, the rest is reserved. */ - .rw = (PCI_EXP_RTCTL_SECEE | PCI_EXP_RTCTL_SENFEE | -PCI_EXP_RTCTL_SEFEE |PCI_EXP_RTCTL_PMEIE | - PCI_EXP_RTCTL_CRSSVE), - .ro = PCI_EXP_RTCAP_CRSVIS << 16, + .rw = GENMASK(4, 0), + .ro = BIT(0) << 16, .rsvd = GENMASK(15, 5) | (GENMASK(15, 1) << 16), }, [PCI_EXP_RTSTA / 4] = { - .ro = GENMASK(15, 0) | PCI_EXP_RTSTA_PENDING, - .w1c = PCI_EXP_RTSTA_PME, +/* +* Root status has bits 17 and [15:0] RO, bit 16 W1C, the rest +* is reserved. +*/ + .ro = BIT(17) | GENMASK(15, 0), + .w1c = BIT(16), .rsvd = GENMASK(31, 18), }, };
Signature · 1 line
1.8.3.1
Re: [RFC 1/9] PCI: pci-bridge-emul: Update PCIe register behaviors
Quoted 2 lines · Jon Derrick
Update the PCIe register behaviors and comments for PCIe v5.0.
I think you may elaborate the changes here, like a summary.
Quoted 2 lines
Replace the specific bit definitions with BIT and GENMASK to make updating easier in the future.
...
drivers/pci/pci-bridge-emul.c struct pci_bridge_reg_behavior { · line 195
Quoted 2 lines
drivers/pci/pci-bridge-emul.c +2
-0+* Device status register has bits 6 and [3:0] W1C, [5:4] RO, +* the rest is reserved
Perhaps it makes sense to add '.' (period) to the end of sentence. And the same for the rest comments.
...
drivers/pci/pci-bridge-emul.c struct pci_bridge_reg_behavior {
Quoted 8 lines
drivers/pci/pci-bridge-emul.c +2
-5- .w1c = (PCI_EXP_SLTSTA_ABP | PCI_EXP_SLTSTA_PFD | -PCI_EXP_SLTSTA_MRLSC |PCI_EXP_SLTSTA_PDC | - PCI_EXP_SLTSTA_CC | PCI_EXP_SLTSTA_DLLSC) << 16, - .ro = (PCI_EXP_SLTSTA_MRLSS |PCI_EXP_SLTSTA_PDS | - PCI_EXP_SLTSTA_EIS) << 16, + .w1c = (BIT(8) | GENMASK(4, 0)) << 16, + .ro = GENMASK(7, 5) << 16,
I'm not sure the new one is better. Perhaps you need to add description for the new bits and, if you consider it's needed, add a picture of the bits in the comment, like XXXX rwX1 XXrX XwXX, where r = ro, w = rw, 1 = w1c, X = rsvd. But it's up to Bjorn and you, I have no strong opinion here.
Same for the rest similar changes.
...
drivers/pci/pci-bridge-emul.c struct pci_bridge_reg_behavior { · line 248
Quoted 6 lines
drivers/pci/pci-bridge-emul.c +2
-4- .rw = (PCI_EXP_RTCTL_SECEE | PCI_EXP_RTCTL_SENFEE | -PCI_EXP_RTCTL_SEFEE |PCI_EXP_RTCTL_PMEIE | - PCI_EXP_RTCTL_CRSSVE), - .ro = PCI_EXP_RTCAP_CRSVIS << 16, + .rw = GENMASK(4, 0), + .ro = BIT(0) << 16,
Bit 0 in both rw and ro -- is it correct?
Signature · 2 lines
With Best Regards, Andy Shevchenko
Re: [RFC 1/9] PCI: pci-bridge-emul: Update PCIe register behaviors
In reply to Jon Derrick[+cc Thomas, Russell]
Quoted 3 lines · Jon Derrick
Update the PCIe register behaviors and comments for PCIe v5.0. Replace the specific bit definitions with BIT and GENMASK to make updating easier in the future.
I think this patch makes sense on its own, independent of the rest of the series. I *would* like to see it split into (a) a "no changes" patch that converts to BIT/GENMASK, and (b) another patch that contains only the PCIe r5.0 updates.
Quoted 107 lines
Signed-off-by Jon Derrick <[email protected]>--- drivers/pci/pci-bridge-emul.c | 54 +++++++++++++++++++++---------------------- 1 file changed, 27 insertions(+), 27 deletions(-)drivers/pci/pci-bridge-emul.c +27
-27@@ -191,12 +191,12 @@ struct pci_bridge_reg_behavior { .rw = GENMASK(15, 0),/* -* Device status register has 4 bits W1C, then 2 bits -* RO, the rest is reserved +* Device status register has bits 6 and [3:0] W1C, [5:4] RO, +* the rest is reserved */ - .w1c = GENMASK(19, 16), - .ro = GENMASK(20, 19), - .rsvd = GENMASK(31, 21), + .w1c = (BIT(6) | GENMASK(3, 0)) << 16, + .ro = GENMASK(5, 4) << 16, + .rsvd = GENMASK(15, 7) << 16, }, [PCI_EXP_LNKCAP / 4] = {@@ -207,15 +207,16 @@ struct pci_bridge_reg_behavior { [PCI_EXP_LNKCTL / 4] = {/* -* Link control has bits [1:0] and [11:3] RW, the -* other bits are reserved. -* Link status has bits [13:0] RO, and bits [14:15] +* Link control has bits [15:14], [11:3] and [1:0] RW, the +* rest is reserved. +* +* Link status has bits [13:0] RO, and bits [15:14] * W1C. */ - .rw = GENMASK(11, 3) | GENMASK(1, 0), + .rw = GENMASK(15, 14) | GENMASK(11, 3) | GENMASK(1, 0), .ro = GENMASK(13, 0) << 16, .w1c = GENMASK(15, 14) << 16, - .rsvd = GENMASK(15, 12) | BIT(2), + .rsvd = GENMASK(13, 12) | BIT(2), }, [PCI_EXP_SLTCAP / 4] = {@@ -224,19 +225,16 @@ struct pci_bridge_reg_behavior { [PCI_EXP_SLTCTL / 4] = {/* -* Slot control has bits [12:0] RW, the rest is +* Slot control has bits [14:0] RW, the rest is * reserved. * -* Slot status has a mix of W1C and RO bits, as well -* as reserved bits. +* Slot status has bits 8 and [4:0] W1C, bits [7:5] RO, the +* rest is reserved. */ - .rw = GENMASK(12, 0), - .w1c = (PCI_EXP_SLTSTA_ABP |PCI_EXP_SLTSTA_PFD | -PCI_EXP_SLTSTA_MRLSC |PCI_EXP_SLTSTA_PDC | - PCI_EXP_SLTSTA_CC | PCI_EXP_SLTSTA_DLLSC) << 16, - .ro = (PCI_EXP_SLTSTA_MRLSS |PCI_EXP_SLTSTA_PDS | - PCI_EXP_SLTSTA_EIS) << 16, - .rsvd = GENMASK(15, 12) | (GENMASK(15, 9) << 16), + .rw = GENMASK(14, 0), + .w1c = (BIT(8) | GENMASK(4, 0)) << 16, + .ro = GENMASK(7, 5) << 16, + .rsvd = BIT(15) | (GENMASK(15, 9) << 16), }, [PCI_EXP_RTCTL / 4] = {@@ -244,18 +242,20 @@ struct pci_bridge_reg_behavior {* Root control has bits [4:0] RW, the rest is * reserved. * -* Root status has bit 0 RO, the rest is reserved. +* Root capabilities has bit 0 RO, the rest is reserved. */ - .rw = (PCI_EXP_RTCTL_SECEE | PCI_EXP_RTCTL_SENFEE | -PCI_EXP_RTCTL_SEFEE |PCI_EXP_RTCTL_PMEIE | - PCI_EXP_RTCTL_CRSSVE), - .ro = PCI_EXP_RTCAP_CRSVIS << 16, + .rw = GENMASK(4, 0), + .ro = BIT(0) << 16, .rsvd = GENMASK(15, 5) | (GENMASK(15, 1) << 16), }, [PCI_EXP_RTSTA / 4] = { - .ro = GENMASK(15, 0) | PCI_EXP_RTSTA_PENDING, - .w1c = PCI_EXP_RTSTA_PME, +/* +* Root status has bits 17 and [15:0] RO, bit 16 W1C, the rest +* is reserved. +*/ + .ro = BIT(17) | GENMASK(15, 0), + .w1c = BIT(16), .rsvd = GENMASK(31, 18), }, };Signature · 1 line
1.8.3.1
RFC 2/9 PCI: pci-bridge-emul: Eliminate reserved member
In reply to Jon DerrickAssume any bit not in the Read-Only, Read-Write, or Write-1-to-Clear behavior masks is a Reserved bit and should return 0 on reads.
--- drivers/pci/pci-bridge-emul.c | 28 +++++++++++----------------- 1 file changed, 11 insertions(+), 17 deletions(-)
drivers/pci/pci-bridge-emul.c +11 -17
@@ -24,6 +24,15 @@#define PCI_CAP_PCIE_START PCI_BRIDGE_CONF_END#define PCI_CAP_PCIE_END (PCI_CAP_PCIE_START + PCI_EXP_SLTSTA2 + 2) +/** +* struct pci_bridge_reg_behavior - register bits behaviors +* @ro: Read-Only bits +* @rw: Read-Write bits +* @w1c: Write-1-to-Clear bits +* +* Reads and Writes will be filtered by specified behavior. All other bits not +* declared are assumed 'Reserved' and will return 0 on reads. +*/ struct pci_bridge_reg_behavior {/* Read-only bits */ u32 ro;
@@ -33,9 +42,6 @@ struct pci_bridge_reg_behavior {/* Write-1-to-clear bits */ u32 w1c; - -/* Reserved bits (hardwired to 0) */ -u32 rsvd; };static const struct pci_bridge_reg_behavior pci_regs_behavior[] = {
@@ -49,7 +55,6 @@ struct pci_bridge_reg_behavior { PCI_COMMAND_FAST_BACK) | (PCI_STATUS_CAP_LIST |PCI_STATUS_66MHZ | PCI_STATUS_FAST_BACK | PCI_STATUS_DEVSEL_MASK) << 16), - .rsvd = GENMASK(15, 10) | ((BIT(6) | GENMASK(3, 0)) << 16), .w1c = (PCI_STATUS_PARITY | PCI_STATUS_SIG_TARGET_ABORT | PCI_STATUS_REC_TARGET_ABORT |
@@ -106,8 +111,6 @@ struct pci_bridge_reg_behavior {PCI_STATUS_REC_MASTER_ABORT |PCI_STATUS_SIG_SYSTEM_ERROR | PCI_STATUS_DETECTED_PARITY) << 16, - - .rsvd = ((BIT(6) | GENMASK(4, 0)) << 16), }, [PCI_MEMORY_BASE / 4] = {
@@ -140,12 +143,10 @@ struct pci_bridge_reg_behavior { [PCI_CAPABILITY_LIST / 4] = { .ro = GENMASK(7, 0), - .rsvd = GENMASK(31, 8), }, [PCI_ROM_ADDRESS1 / 4] = { .rw = GENMASK(31, 11) | BIT(0), - .rsvd = GENMASK(10, 1), },/*
@@ -168,8 +169,6 @@ struct pci_bridge_reg_behavior { .ro = (GENMASK(15, 8) | ((PCI_BRIDGE_CTL_FAST_BACK) << 16)), .w1c = BIT(10) << 16, - - .rsvd = (GENMASK(15, 12) | BIT(4)) << 16, }, };
@@ -196,13 +195,11 @@ struct pci_bridge_reg_behavior {*/ .w1c = (BIT(6) | GENMASK(3, 0)) << 16, .ro = GENMASK(5, 4) << 16, - .rsvd = GENMASK(15, 7) << 16, }, [PCI_EXP_LNKCAP / 4] = {/* All bits are RO, except bit 23 which is reserved */ .ro = lower_32_bits(~BIT(23)), - .rsvd = BIT(23), }, [PCI_EXP_LNKCTL / 4] = {
@@ -216,7 +213,6 @@ struct pci_bridge_reg_behavior { .rw = GENMASK(15, 14) | GENMASK(11, 3) | GENMASK(1, 0), .ro = GENMASK(13, 0) << 16, .w1c = GENMASK(15, 14) << 16, - .rsvd = GENMASK(13, 12) | BIT(2), }, [PCI_EXP_SLTCAP / 4] = {
@@ -234,7 +230,6 @@ struct pci_bridge_reg_behavior { .rw = GENMASK(14, 0), .w1c = (BIT(8) | GENMASK(4, 0)) << 16, .ro = GENMASK(7, 5) << 16, - .rsvd = BIT(15) | (GENMASK(15, 9) << 16), }, [PCI_EXP_RTCTL / 4] = {
@@ -246,7 +241,6 @@ struct pci_bridge_reg_behavior {*/ .rw = GENMASK(4, 0), .ro = BIT(0) << 16, - .rsvd = GENMASK(15, 5) | (GENMASK(15, 1) << 16), }, [PCI_EXP_RTSTA / 4] = {
@@ -256,7 +250,6 @@ struct pci_bridge_reg_behavior {*/ .ro = BIT(17) | GENMASK(15, 0), .w1c = BIT(16), - .rsvd = GENMASK(31, 18), }, };
@@ -364,7 +357,8 @@ int pci_bridge_emul_conf_read(struct pci_bridge_emul *bridge, int where,* Make sure we never return any reserved bit with a value * different from 0. */ - *value &= ~behavior[reg /4 ].rsvd; + *value &= behavior[reg /4 ].ro | behavior[reg /4 ].rw | + behavior[reg /4 ].w1c;if (size ==1 ) *value = (*value >> (8 * (where &3 ))) &0xff ;
Signature · 1 line
1.8.3.1
Re: [RFC 2/9] PCI: pci-bridge-emul: Eliminate reserved member
[+cc Thomas, Russell]
Quoted 2 lines · Jon Derrick
Assume any bit not in the Read-Only, Read-Write, or Write-1-to-Clear behavior masks is a Reserved bit and should return 0 on reads.
This also seems like it makes sense on its own, especially if you can include a spec citation.
Quoted 133 lines
Signed-off-by Jon Derrick <[email protected]>--- drivers/pci/pci-bridge-emul.c | 28 +++++++++++----------------- 1 file changed, 11 insertions(+), 17 deletions(-)drivers/pci/pci-bridge-emul.c +11
-17@@ -24,6 +24,15 @@#define PCI_CAP_PCIE_START PCI_BRIDGE_CONF_END#define PCI_CAP_PCIE_END (PCI_CAP_PCIE_START + PCI_EXP_SLTSTA2 + 2) +/** +* struct pci_bridge_reg_behavior - register bits behaviors +* @ro: Read-Only bits +* @rw: Read-Write bits +* @w1c: Write-1-to-Clear bits +* +* Reads and Writes will be filtered by specified behavior. All other bits not +* declared are assumed 'Reserved' and will return 0 on reads. +*/ struct pci_bridge_reg_behavior {/* Read-only bits */ u32 ro;@@ -33,9 +42,6 @@ struct pci_bridge_reg_behavior {/* Write-1-to-clear bits */ u32 w1c; - -/* Reserved bits (hardwired to 0) */ -u32 rsvd; };static const struct pci_bridge_reg_behavior pci_regs_behavior[] = {@@ -49,7 +55,6 @@ struct pci_bridge_reg_behavior { PCI_COMMAND_FAST_BACK) | (PCI_STATUS_CAP_LIST |PCI_STATUS_66MHZ | PCI_STATUS_FAST_BACK | PCI_STATUS_DEVSEL_MASK) << 16), - .rsvd = GENMASK(15, 10) | ((BIT(6) | GENMASK(3, 0)) << 16), .w1c = (PCI_STATUS_PARITY | PCI_STATUS_SIG_TARGET_ABORT | PCI_STATUS_REC_TARGET_ABORT |@@ -106,8 +111,6 @@ struct pci_bridge_reg_behavior {PCI_STATUS_REC_MASTER_ABORT |PCI_STATUS_SIG_SYSTEM_ERROR | PCI_STATUS_DETECTED_PARITY) << 16, - - .rsvd = ((BIT(6) | GENMASK(4, 0)) << 16), }, [PCI_MEMORY_BASE / 4] = {@@ -140,12 +143,10 @@ struct pci_bridge_reg_behavior { [PCI_CAPABILITY_LIST / 4] = { .ro = GENMASK(7, 0), - .rsvd = GENMASK(31, 8), }, [PCI_ROM_ADDRESS1 / 4] = { .rw = GENMASK(31, 11) | BIT(0), - .rsvd = GENMASK(10, 1), },/* @@ -168,8 +169,6 @@ struct pci_bridge_reg_behavior { .ro = (GENMASK(15, 8) | ((PCI_BRIDGE_CTL_FAST_BACK) << 16)), .w1c = BIT(10) << 16, - - .rsvd = (GENMASK(15, 12) | BIT(4)) << 16, }, };@@ -196,13 +195,11 @@ struct pci_bridge_reg_behavior {*/ .w1c = (BIT(6) | GENMASK(3, 0)) << 16, .ro = GENMASK(5, 4) << 16, - .rsvd = GENMASK(15, 7) << 16, }, [PCI_EXP_LNKCAP / 4] = {/* All bits are RO, except bit 23 which is reserved */ .ro = lower_32_bits(~BIT(23)), - .rsvd = BIT(23), }, [PCI_EXP_LNKCTL / 4] = {@@ -216,7 +213,6 @@ struct pci_bridge_reg_behavior { .rw = GENMASK(15, 14) | GENMASK(11, 3) | GENMASK(1, 0), .ro = GENMASK(13, 0) << 16, .w1c = GENMASK(15, 14) << 16, - .rsvd = GENMASK(13, 12) | BIT(2), }, [PCI_EXP_SLTCAP / 4] = {@@ -234,7 +230,6 @@ struct pci_bridge_reg_behavior { .rw = GENMASK(14, 0), .w1c = (BIT(8) | GENMASK(4, 0)) << 16, .ro = GENMASK(7, 5) << 16, - .rsvd = BIT(15) | (GENMASK(15, 9) << 16), }, [PCI_EXP_RTCTL / 4] = {@@ -246,7 +241,6 @@ struct pci_bridge_reg_behavior {*/ .rw = GENMASK(4, 0), .ro = BIT(0) << 16, - .rsvd = GENMASK(15, 5) | (GENMASK(15, 1) << 16), }, [PCI_EXP_RTSTA / 4] = {@@ -256,7 +250,6 @@ struct pci_bridge_reg_behavior {*/ .ro = BIT(17) | GENMASK(15, 0), .w1c = BIT(16), - .rsvd = GENMASK(31, 18), }, };@@ -364,7 +357,8 @@ int pci_bridge_emul_conf_read(struct pci_bridge_emul *bridge, int where,* Make sure we never return any reserved bit with a value * different from 0. */ - *value &= ~behavior[reg /4 ].rsvd; + *value &= behavior[reg /4 ].ro | behavior[reg /4 ].rw | + behavior[reg /4 ].w1c;if (size ==1 ) *value = (*value >> (8 * (where &3 ))) &0xff ;Signature · 1 line
1.8.3.1
RFC 3/9 PCI: pci-bridge-emul: Provide a helper to set behavior
In reply to Jon DerrickAdd a handler to set behavior of a PCI or PCIe register. Add the appropriate enums to specify the register's Read-Only, Read-Write, and Write-1-to-Clear behaviors.
--- drivers/pci/pci-bridge-emul.c | 19 +++++++++++++++++++ drivers/pci/pci-bridge-emul.h | 10 ++++++++++ 2 files changed, 29 insertions(+)
drivers/pci/pci-bridge-emul.c +19 -0
@@ -44,6 +44,25 @@ struct pci_bridge_reg_behavior {u32 w1c; }; +void pci_bridge_emul_set_reg_behavior (struct pci_bridge_emul *bridge, +bool pcie,int reg,u32 val, +enum pci_bridge_emul_reg_behavior type) +{ +struct pci_bridge_reg_behavior *behavior; + +if (pcie) + behavior = &bridge->pcie_cap_regs_behavior[reg /4 ]; +else + behavior = &bridge->pci_regs_behavior[reg /4 ]; + +if (type ==PCI_BRIDGE_EMUL_REG_BEHAVIOR_RO ) + behavior->ro = val; +else if (type ==PCI_BRIDGE_EMUL_REG_BEHAVIOR_RW ) + behavior->rw = val; +else /* PCI_BRIDGE_EMUL_REG_BEHAVIOR_W1C */ + behavior->w1c = val; +} +static const struct pci_bridge_reg_behavior pci_regs_behavior[] = { [PCI_VENDOR_ID /4 ] = { .ro = ~0 }, [PCI_COMMAND /4 ] = {
drivers/pci/pci-bridge-emul.h +10 -0
@@ -72,6 +72,12 @@ struct pci_bridge_emul_pcie_conf {typedef enum {PCI_BRIDGE_EMUL_HANDLED ,PCI_BRIDGE_EMUL_NOT_HANDLED }pci_bridge_emul_read_status_t ; +enum pci_bridge_emul_reg_behavior { +PCI_BRIDGE_EMUL_REG_BEHAVIOR_RO , +PCI_BRIDGE_EMUL_REG_BEHAVIOR_RW , +PCI_BRIDGE_EMUL_REG_BEHAVIOR_W1C , +}; +struct pci_bridge_emul_ops {/* * Called when reading from the regular PCI bridge
@@ -132,4 +138,8 @@ int pci_bridge_emul_conf_read(struct pci_bridge_emul *bridge, int where,int pci_bridge_emul_conf_write (struct pci_bridge_emul *bridge,int where,int size,u32 value); +void pci_bridge_emul_set_reg_behavior (struct pci_bridge_emul *bridge, +bool pcie,int reg,u32 val, +enum pci_bridge_emul_reg_behavior type); +#endif /* __PCI_BRIDGE_EMUL_H__ */
Signature · 1 line
1.8.3.1
RFC 4/9 PCI: pciehp: Indirect slot register operations
In reply to Jon DerrickAllow another driver to provide alternative operations when doing capability register reads and writes to the Slot Capabilities, Slot Control, and Slot Status registers. Add handlers to pciehp to read/write the slot registers.
--- drivers/pci/hotplug/pciehp.h | 8 +++ drivers/pci/hotplug/pciehp_hpc.c | 116 ++++++++++++++++++++++++++++----------- 2 files changed, 93 insertions(+), 31 deletions(-)
drivers/pci/hotplug/pciehp.h +8 -0
@@ -28,6 +28,12 @@extern bool pciehp_poll_mode;extern int pciehp_poll_time; +/* Capability reads/writes but only for SLTCAP, SLTCTL, and SLTSTA. */ +struct slot_ecap_ops { +int (*read)(struct pci_dev *dev,int pos,int len,u32 *val); +int (*write)(struct pci_dev *dev,int pos,int len,u32 val); +}; +/* * Set CONFIG_DYNAMIC_DEBUG=y and boot with 'dyndbg="file pciehp* +p"' to * enable debug messages.
@@ -76,6 +82,7 @@* @request_result: result of last user request submitted to the IRQ thread * @requester: wait queue to wake up on completion of user request, * used for synchronous slot enable/disable request via sysfs +* @slot_ops: slot config registers read/write operations * * PCIe hotplug has a 1:1 relationship between controller and slot, hence * unlike other drivers, the two aren't represented by separate structures.
@@ -105,6 +112,7 @@ struct controller {unsigned int ist_running;int request_result;wait_queue_head_t requester; +struct slot_ecap_ops *slot_ops; };/**
drivers/pci/hotplug/pciehp_hpc.c +85 -31
260 lines not on this page · raw message
Signature · 1 line
1.8.3.1
RFC 5/9 PCI: Add pcie_port_slot_emulated stub
In reply to Jon DerrickAdd the checks to allow an emulated slot. An emulated slot will use native Hotplug, AER, and PME services. It also needs to specify itself as a hotplug bridge in order for bridge sizing to account for hotplug reserved windows.
--- drivers/pci/pci-acpi.c | 3 +++ drivers/pci/pcie/portdrv_core.c | 11 ++++++++--- drivers/pci/probe.c | 2 +- include/linux/pci.h | 2 ++ 4 files changed, 14 insertions(+), 4 deletions(-)
drivers/pci/pci-acpi.c +3 -0
@@ -796,6 +796,9 @@ bool pciehp_is_native(struct pci_dev *bridge)if (!IS_ENABLED (CONFIG_HOTPLUG_PCI_PCIE ))return false; +if (pcie_port_slot_emulated (bridge)) +return true; +pcie_capability_read_dword (bridge,PCI_EXP_SLTCAP , &slot_cap);if (!(slot_cap &PCI_EXP_SLTCAP_HPC ))return false;
drivers/pci/pcie/portdrv_core.c +8 -3
@@ -193,6 +193,11 @@ static int pcie_init_service_irqs(struct pci_dev *dev, int *irqs, int mask)return 0 ; } +static inline bool is_port_native_or_emulated (struct pci_dev *dev) +{ +return pcie_ports_native ||pcie_port_slot_emulated (dev); +} +/** * get_port_device_capability - discover capabilities of a PCI Express port * @dev: PCI Express port to examine
@@ -209,7 +214,7 @@ static int get_port_device_capability(struct pci_dev *dev)int services =0 ;if (dev->is_hotplug_bridge && - (pcie_ports_native || host->native_pcie_hotplug)) { + (is_port_native_or_emulated (dev) || host->native_pcie_hotplug)) { services |=PCIE_PORT_SERVICE_HP ;/*
@@ -222,7 +227,7 @@ static int get_port_device_capability(struct pci_dev *dev)#ifdef CONFIG_PCIEAER if (dev->aer_cap &&pci_aer_available () && - (pcie_ports_native || host->native_aer)) { + (is_port_native_or_emulated (dev) || host->native_aer)) { services |=PCIE_PORT_SERVICE_AER ;/*
@@ -239,7 +244,7 @@ static int get_port_device_capability(struct pci_dev *dev)* those yet. */ if (pci_pcie_type (dev) ==PCI_EXP_TYPE_ROOT_PORT && - (pcie_ports_native || host->native_pme)) { + (is_port_native_or_emulated (dev) || host->native_pme)) { services |=PCIE_PORT_SERVICE_PME ;/*
drivers/pci/probe.c +1 -1
@@ -1477,7 +1477,7 @@ void set_pcie_hotplug_bridge(struct pci_dev *pdev)u32 reg32;pcie_capability_read_dword (pdev,PCI_EXP_SLTCAP , ®32); -if (reg32 &PCI_EXP_SLTCAP_HPC ) +if (reg32 &PCI_EXP_SLTCAP_HPC ||pcie_port_slot_emulated (pdev)) pdev->is_hotplug_bridge =1 ; }
include/linux/pci.h +2 -0
@@ -1536,6 +1536,8 @@ static inline int pci_irqd_intx_xlate(struct irq_domain *d,#define pcie_ports_native false#endif +#define pcie_port_slot_emulated (dev) false +#define PCIE_LINK_STATE_L0S BIT(0)#define PCIE_LINK_STATE_L1 BIT(1)#define PCIE_LINK_STATE_CLKPM BIT(2)
Signature · 1 line
1.8.3.1
RFC 6/9 PCI: pciehp: Expose the poll loop to other drivers
In reply to Jon DerrickAbstract the poll loop into an 'events pending' check and a 'handle events' method in order to allow another driver to call the polling loop.
--- drivers/pci/hotplug/pciehp.h | 2 ++ drivers/pci/hotplug/pciehp_hpc.c | 16 +++++++++++++--- 2 files changed, 15 insertions(+), 3 deletions(-)
drivers/pci/hotplug/pciehp.h +2 -0
@@ -160,6 +160,8 @@ struct controller {#define NO_CMD_CMPL (ctrl) ((ctrl)->slot_cap & PCI_EXP_SLTCAP_NCCS)#define PSN (ctrl) (((ctrl)->slot_cap & PCI_EXP_SLTCAP_PSN) >> 19) +bool pciehp_events_pending (struct controller *ctrl); +void pciehp_handle_events (struct controller *ctrl);void pciehp_request (struct controller *ctrl,int action);void pciehp_handle_button_press (struct controller *ctrl);void pciehp_handle_disable_request (struct controller *ctrl);
drivers/pci/hotplug/pciehp_hpc.c +13 -3
@@ -708,6 +708,17 @@ static irqreturn_t pciehp_ist(int irq, void *dev_id)return IRQ_HANDLED ; } +bool pciehp_events_pending (struct controller *ctrl) +{ +return pciehp_isr (IRQ_NOTCONNECTED , ctrl) ==IRQ_WAKE_THREAD || +atomic_read (&ctrl->pending_events); +} + +void pciehp_handle_events (struct controller *ctrl) +{ +pciehp_ist (IRQ_NOTCONNECTED , ctrl); +} +static int pciehp_poll (void *data) {struct controller *ctrl = data;
@@ -716,9 +727,8 @@ static int pciehp_poll(void *data)while (!kthread_should_stop ()) {/* poll for interrupt events or user requests */ -while (pciehp_isr (IRQ_NOTCONNECTED , ctrl) ==IRQ_WAKE_THREAD || -atomic_read (&ctrl->pending_events)) -pciehp_ist (IRQ_NOTCONNECTED , ctrl); +while (pciehp_events_pending (ctrl)) +pciehp_handle_events (ctrl);if (pciehp_poll_time <=0 || pciehp_poll_time >60 ) pciehp_poll_time =2 ;/* clamp to sane value */
Signature · 1 line
1.8.3.1
RFC 7/9 PCI: Move pci_dev_str_match to search.c
In reply to Jon DerrickThe method which extracts a string descriptor of one or more PCI devices and matches to a struct pci_dev is useful in general to other subsystems needing to match parameters or sysfs strings. Move this function to search.c for general use.
No functional changes.
--- drivers/pci/pci.c | 163 --------------------------------------------------- drivers/pci/search.c | 162 ++++++++++++++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 2 + 3 files changed, 164 insertions(+), 163 deletions(-)
drivers/pci/pci.c +0 -163
@@ -203,169 +203,6 @@ void __iomem *pci_ioremap_wc_bar(struct pci_dev *pdev, int bar)EXPORT_SYMBOL_GPL (pci_ioremap_wc_bar);#endif -/** -* pci_dev_str_match_path - test if a path string matches a device -* @dev: the PCI device to test -* @path: string to match the device against -* @endptr: pointer to the string after the match -* -* Test if a string (typically from a kernel parameter) formatted as a -* path of device/function addresses matches a PCI device. The string must -* be of the form: -* -* [<domain>:]<bus>:<device>.<func>[/<device>.<func>]* -* -* A path for a device can be obtained using 'lspci -t'. Using a path -* is more robust against bus renumbering than using only a single bus, -* device and function address. -* -* Returns 1 if the string matches the device, 0 if it does not and -* a negative error code if it fails to parse the string. -*/ -static int pci_dev_str_match_path (struct pci_dev *dev,const char *path, -const char **endptr) -{ -int ret; -int seg, bus, slot, func; -char *wpath, *p; -char end; - - *endptr =strchrnul (path,';' ); - - wpath =kmemdup_nul (path, *endptr - path,GFP_KERNEL ); -if (!wpath) -return -ENOMEM ; - -while (1 ) { - p =strrchr (wpath,'/' ); -if (!p) -break ; - ret =sscanf (p,"/%x.%x%c" , &slot, &func, &end); -if (ret !=2 ) { - ret = -EINVAL ; - goto free_and_exit; - } - -if (dev->devfn !=PCI_DEVFN (slot, func)) { - ret =0 ; - goto free_and_exit; - } - -/* -* Note: we don't need to get a reference to the upstream -* bridge because we hold a reference to the top level -* device which should hold a reference to the bridge, -* and so on. -*/ - dev =pci_upstream_bridge (dev); -if (!dev) { - ret =0 ; - goto free_and_exit; - } - - *p =0 ; - } - - ret =sscanf (wpath,"%x:%x:%x.%x%c" , &seg, &bus, &slot, - &func, &end); -if (ret !=4 ) { - seg =0 ; - ret =sscanf (wpath,"%x:%x.%x%c" , &bus, &slot, &func, &end); -if (ret !=3 ) { - ret = -EINVAL ; - goto free_and_exit; - } - } - - ret = (seg ==pci_domain_nr (dev->bus) && - bus == dev->bus->number && - dev->devfn ==PCI_DEVFN (slot, func)); - -free_and_exit: -kfree (wpath); -return ret; -} - -/** -* pci_dev_str_match - test if a string matches a device -* @dev: the PCI device to test -* @p: string to match the device against -* @endptr: pointer to the string after the match -* -* Test if a string (typically from a kernel parameter) matches a specified -* PCI device. The string may be of one of the following formats: -* -* [<domain>:]<bus>:<device>.<func>[/<device>.<func>]* -* pci:<vendor>:<device>[:<subvendor>:<subdevice>] -* -* The first format specifies a PCI bus/device/function address which -* may change if new hardware is inserted, if motherboard firmware changes, -* or due to changes caused in kernel parameters. If the domain is -* left unspecified, it is taken to be 0. In order to be robust against -* bus renumbering issues, a path of PCI device/function numbers may be used -* to address the specific device. The path for a device can be determined -* through the use of 'lspci -t'. -* -* The second format matches devices using IDs in the configuration -* space which may match multiple devices in the system. A value of 0 -* for any field will match all devices. (Note: this differs from -* in-kernel code that uses PCI_ANY_ID which is ~0; this is for -* legacy reasons and convenience so users don't have to specify -* FFFFFFFFs on the command line.) -* -* Returns 1 if the string matches the device, 0 if it does not and -* a negative error code if the string cannot be parsed. -*/ -static int pci_dev_str_match (struct pci_dev *dev,const char *p, -const char **endptr) -{ -int ret; -int count; -unsigned short vendor, device, subsystem_vendor, subsystem_device; - -if (strncmp (p,"pci:" ,4 ) ==0 ) { -/* PCI vendor/device (subvendor/subdevice) IDs are specified */ - p +=4 ; - ret =sscanf (p,"%hx:%hx:%hx:%hx%n" , &vendor, &device, - &subsystem_vendor, &subsystem_device, &count); -if (ret !=4 ) { - ret =sscanf (p,"%hx:%hx%n" , &vendor, &device, &count); -if (ret !=2 ) -return -EINVAL ; - - subsystem_vendor =0 ; - subsystem_device =0 ; - } - - p += count; - -if ((!vendor || vendor == dev->vendor) && - (!device || device == dev->device) && - (!subsystem_vendor || - subsystem_vendor == dev->subsystem_vendor) && - (!subsystem_device || - subsystem_device == dev->subsystem_device)) - goto found; - }else { -/* -* PCI Bus, Device, Function IDs are specified -* (optionally, may include a path of devfns following it) -*/ - ret =pci_dev_str_match_path (dev, p, &p); -if (ret <0 ) -return ret; -else if (ret) - goto found; - } - - *endptr = p; -return 0 ; - -found: - *endptr = p; -return 1 ; -} -static int __pci_find_next_cap_ttl (struct pci_bus *bus,unsigned int devfn,u8 pos,int cap,int *ttl) {
drivers/pci/search.c +162 -0
165 lines not on this page · raw message
include/linux/pci.h +2 -0
@@ -1056,6 +1056,8 @@ struct pci_dev *pci_get_domain_bus_and_slot(int domain, unsigned int bus,struct pci_dev *pci_get_class (unsigned int class,struct pci_dev *from);int pci_dev_present (const struct pci_device_id *ids); +int pci_dev_str_match (struct pci_dev *dev,const char *p,const char **endptr); +int pci_bus_read_config_byte (struct pci_bus *bus,unsigned int devfn,int where,u8 *val); int pci_bus_read_config_word(struct pci_bus *bus, unsigned int devfn,
Signature · 1 line
1.8.3.1
RFC 8/9 PCI: pciehp: Add hotplug slot emulation driver
In reply to Jon DerrickThis adds the PCIe Hotplug emulation driver. There may be platforms with specific configurations that can support hotplug but don't provide the logical slot hotplug hardware. For instance, the platform may use an electrically-tolerant interposer between the slot and the device.
This driver utilizes the pci-bridge-emul architecture to manage register reads and writes. The underlying functionality of the hotplug emulation driver uses the Data Link Layer Link Active Reporting mechanism in a polling loop, but can tolerate other event sources such as AER or DPC.
When enabled and a slot is managed by the driver, all PCIe Port services are managed by the kernel. This is done to ensure that firmware hotplug and error architecture does not (correctly) machine check the system when hotplug is performed on a non-hotplug slot.
The driver offers two active mode: auto and force.
auto: Bind to non-hotplug slots
force: Bind to all slots and overrides the slot's services
There are three kernel parameters:
pciehp.pciehp_emul_mode={off, auto, force} pciehp.pciehp_emul_time=<msecs poll time> (def 1000, min 100, max 60000) pciehp.pciehp_emul_ports=<PCI [S]BDF/ID format string>
The pciehp_emul_ports parameter takes a semi-colon tokenized string representing PCI [S]BDFs and IDs. The pciehp_emul_mode will then be applied to only those slots, leaving other slots unmanaged by pciehp_emul.
The string follows the pci_dev_str_match() format:
[<domain>:]<bus>:<device>.<func>[/<device>.<func>]* pci:<vendor>:<device>[:<subvendor>:<subdevice>]
When using the path format, the path for the device can be obtained using 'lspci -t' and further specified using the upstream bridge and the downstream port's device-function to be more robust against bus renumbering.
When using the vendor-device format, a value of '0' in any field acts as a wildcard for that field, matching all values.
The driver is enabled with CONFIG_HOTPLUG_PCI_PCIE_EMUL=y.
--- drivers/pci/hotplug/Makefile | 4 + drivers/pci/hotplug/pciehp_emul.c | 373 ++++++++++++++++++++++++++++++++++++++ drivers/pci/pcie/Kconfig | 14 ++ 3 files changed, 391 insertions(+) create mode 100644 drivers/pci/hotplug/pciehp_emul.c
drivers/pci/hotplug/Makefile +4 -0
@@ -65,6 +65,10 @@ pciehp-objs := pciehp_core.o \ pciehp_pci.o \ pciehp_hpc.o +ifdef CONFIG_HOTPLUG_PCI_PCIE_EMUL +pciehp-objs += pciehp_emul.o +endif + shpchp-objs := shpchp_core.o \ shpchp_ctrl.o \ shpchp_pci.o \
drivers/pci/hotplug/pciehp_emul.c new file +373 -0
373 lines not on this page · raw message
drivers/pci/pcie/Kconfig +14 -0
@@ -21,6 +21,20 @@ config HOTPLUG_PCI_PCIE When in doubt, say N. +config HOTPLUG_PCI_PCIE_EMUL + bool "PCI Express Hotplug Emulation driver" + depends on HOTPLUG_PCI_PCIE + select PCI_BRIDGE_EMUL + select PCIEAER + help + This enables a PCI Express Native Hotplug emulation driver for + non-hotplug-capable PCI Express slots. Hotplug is determined using + link events and this driver can be disabled, or manage either only + non-hotplug-capable slots or both non-hotplug and hotplug slots. + Specific slots may also be targeted. + + When in doubt, say N. + config PCIEAER bool "PCI Express Advanced Error Reporting support" depends on PCIEPORTBUS
Signature · 1 line
1.8.3.1
RFC 9/9 PCI: pciehp: Wire up pcie_port_emulate_slot and pciehp_emul
In reply to Jon DerrickAdd pcie_port_slot_emulated hook and wire it up to pciehp_emul. Add pciehp_emul_{attach,detach} calls in pciehp and move the management check into the attach caller.
--- drivers/pci/hotplug/pciehp.h | 18 ++++++++++++++++++ drivers/pci/hotplug/pciehp_emul.c | 11 ++++++++--- drivers/pci/hotplug/pciehp_hpc.c | 4 ++++ drivers/pci/pcie/portdrv_core.c | 3 +++ include/linux/pci.h | 4 ++++ 5 files changed, 37 insertions(+), 3 deletions(-)
drivers/pci/hotplug/pciehp.h +18 -0
@@ -197,6 +197,24 @@ struct controller {int pciehp_set_raw_indicator_status (struct hotplug_slot *h_slot,u8 status);int pciehp_get_raw_indicator_status (struct hotplug_slot *h_slot,u8 *status); +#ifdef CONFIG_HOTPLUG_PCI_PCIE_EMUL +bool pciehp_emul_will_manage (struct pci_dev *dev); +int pciehp_emul_attach (struct controller *ctrl); +void pciehp_emul_detach (struct controller *ctrl); +#else +static inline bool pciehp_emul_will_manage (struct pci_dev *dev) +{ +return false; +}; + +static inline int pciehp_emul_attach (struct controller *ctrl) +{ +return 0 ; +}; + +static inline void pciehp_emul_detach (struct controller *ctrl) {}; +#endif +static inline const char *slot_name (struct controller *ctrl) {return hotplug_slot_name (&ctrl->hotplug_slot);
drivers/pci/hotplug/pciehp_emul.c +8 -3
@@ -281,9 +281,6 @@ int pciehp_emul_attach(struct controller *ctrl)u32 slotcap =0 ;int ret; -if (!pciehp_emul_will_manage (dev)) -return 0 ; - emul =kzalloc (sizeof (*emul),GFP_KERNEL );if (!emul)return -ENOMEM ;
@@ -371,3 +368,11 @@ void pciehp_emul_detach(struct controller *ctrl)kfree (emul); } + +static int __init pciehp_emul_init (void ) +{ + pcie_port_slot_emulated = &pciehp_emul_will_manage; + +return 0 ; +} +postcore_initcall (pciehp_emul_init);
drivers/pci/hotplug/pciehp_hpc.c +4 -0
@@ -922,6 +922,10 @@ struct controller *pcie_init(struct pcie_device *dev) ctrl->pcie = dev; ctrl->slot_ops = &pciehp_default_slot_ops; + +if (pciehp_emul_will_manage (pdev) &&pciehp_emul_attach (ctrl)) +return NULL ; +pcie_read_slot_capabilities (ctrl, &slot_cap);if (pdev->hotplug_user_indicators)
drivers/pci/pcie/portdrv_core.c +3 -0
@@ -19,6 +19,9 @@#include "../pci.h" #include "portdrv.h" +/* Hook for hotplug emulation driver */ +bool (*pcie_port_slot_emulated)(struct pci_dev *dev); +struct portdrv_service_data {struct pcie_port_service_driver *drv;struct device *dev;
include/linux/pci.h +4 -0
@@ -1538,7 +1538,11 @@ static inline int pci_irqd_intx_xlate(struct irq_domain *d,#define pcie_ports_native false#endif +#ifdef CONFIG_HOTPLUG_PCI_PCIE_EMUL +extern bool (*pcie_port_slot_emulated)(struct pci_dev *dev); +#else #define pcie_port_slot_emulated (dev) false +#endif #define PCIE_LINK_STATE_L0S BIT(0)#define PCIE_LINK_STATE_L1 BIT(1)
Signature · 1 line
1.8.3.1
Quoted 4 lines · Jon Derrick
This set adds an emulation driver for PCIe Hotplug. There may be platforms with specific configurations that can support hotplug but don't provide the logical slot hotplug hardware. For instance, the platform may use an electrically-tolerant interposer between the slot and the device.
The code seems like one giant hack to me. What is the real life use case for this? Another Intel chipset fuckup like vmd or the ahci remapping?
Quoted 10 lines · Christoph Hellwig
Quoted 4 lines · Jon Derrick
This set adds an emulation driver for PCIe Hotplug. There may be platforms with specific configurations that can support hotplug but don't provide the logical slot hotplug hardware. For instance, the platform may use an electrically-tolerant interposer between the slot and the device.
The code seems like one giant hack to me. What is the real life use case for this? Another Intel chipset fuckup like vmd or the ahci remapping?
Exactly as the cover letter describes. An interposer being used on a non-hotplug slot.
Quoted 6 lines · Jonathan
Quoted 4 lines
The code seems like one giant hack to me. What is the real life use case for this? Another Intel chipset fuckup like vmd or the ahci remapping?
Exactly as the cover letter describes. An interposer being used on a non-hotplug slot.
That isn't a use a case, that iѕ a description of the implementation. Why would you want this code?
Quoted 10 lines · On Mon, 2020-02-10 at 17:58 +0100, [email protected] wrote:
Quoted 6 lines · Jonathan
Quoted 4 lines
The code seems like one giant hack to me. What is the real life use case for this? Another Intel chipset fuckup like vmd or the ahci remapping?
Exactly as the cover letter describes. An interposer being used on a non-hotplug slot.
That isn't a use a case, that iѕ a description of the implementation. Why would you want this code?
It allows non-hotplug slots to take advantage of the kernel's robust hotplug ecosystem, if the platform configuration can tolerate the events. This could also reduce BOM cost by eliminating some slot controllers. Say you had something that only needed to be hotplugged very infrequently, like RAIDed OS drives, versus something needing to be hotplugged very frequently like data drives.
Granted it probably could be fit into pciehp_poll, but it seemed to have a different objective (emulating slot)
[+cc Stuart, Lukas]
Quoted 38 lines · Jon Derrick
This set adds an emulation driver for PCIe Hotplug. There may be platforms with specific configurations that can support hotplug but don't provide the logical slot hotplug hardware. For instance, the platform may use an electrically-tolerant interposer between the slot and the device.
This driver utilizes the pci-bridge-emul architecture to manage register reads and writes. The underlying functionality of the hotplug emulation driver uses the Data Link Layer Link Active Reporting mechanism in a polling loop, but can tolerate other event sources such as AER or DPC.
When enabled and a slot is managed by the driver, all port services are managed by the kernel. This is done to ensure that firmware hotplug and error architecture does not (correctly) halt/machine check the system when hotplug is performed on a non-hotplug slot.
The driver offers two active mode: Auto and Force.
auto: The driver will bind to non-hotplug slots
force: The driver will bind to all slots and overrides the slot's servicesThere are three kernel params:
pciehp.pciehp_emul_mode={off, auto, force} pciehp.pciehp_emul_time=<msecs polling time> (def 1000, min 100, max 60000) pciehp.pciehp_emul_ports=<PCI [S]BDF/ID format string>The
pciehp_emul_portskernel parameter takes a semi-colon tokenized string representing PCI [S]BDFs and IDs. Thepciehp_emul_modewill then be applied to only those slots, leaving other slots unmanaged bypciehp_emul.The string follows the
pci_dev_str_match()format:[<domain>:]<bus>:<device>.<func>[/<device>.<func>]* pci:<vendor>:<device>[:<subvendor>:<subdevice>]
When using the path format, the path for the device can be obtained using 'lspci -t' and further specified using the upstream bridge and the downstream port's device-function to be more robust against bus renumbering.
Quoted 7 lines
When using the vendor-device format, a value of '0' in any field acts as a wildcard for that field, matching all values.
The driver is enabled with
CONFIG_HOTPLUG_PCI_PCIE_EMUL=y.The driver should be considered 'use at own risk' unless the platform/hardware vendor recommends this mode.
There's a lot of good work in here, and I don't claim to understand the use case and all the benefits.
But it seems like quite a lot of additional code and complexity in an area that's already pretty subtle, so I'm not yet convinced that it's all worthwhile.
It seems like this would rely on Data Link Layer Link Active Reporting. Is that something we could add to pciehp as a generic feature without making a separate driver for it? I haven't looked at this for a while, but I would assume that if we find out that a link went down, pciehp could/should be smart enough to notice that even if it didn't come via the usual pciehp Slot Status path.
Quoted 30 lines
Jon Derrick (9):
PCI: pci-bridge-emul: Update PCIe register behaviors
PCI: pci-bridge-emul: Eliminate reserved member
PCI: pci-bridge-emul: Provide a helper to set behavior
PCI: pciehp: Indirect slot register operations
PCI: Addpcie_port_slot_emulatedstub
PCI: pciehp: Expose the poll loop to other drivers
PCI: Movepci_dev_str_matchtosearch.c
PCI: pciehp: Add hotplug slot emulation driver
PCI: pciehp: Wire uppcie_port_emulate_slotandpciehp_emuldrivers/pci/hotplug/Makefile | 4 + drivers/pci/hotplug/pciehp.h | 28 +++ drivers/pci/hotplug/pciehp_emul.c | 378 ++++++++++++++++++++++++++++++++++++++ drivers/pci/hotplug/pciehp_hpc.c | 136 ++++++++++---- drivers/pci/pci-acpi.c | 3 + drivers/pci/pci-bridge-emul.c | 95 +++++----- drivers/pci/pci-bridge-emul.h | 10 + drivers/pci/pci.c | 163 ---------------- drivers/pci/pcie/Kconfig | 14 ++ drivers/pci/pcie/portdrv_core.c | 14 +- drivers/pci/probe.c | 2 +- drivers/pci/search.c | 162 ++++++++++++++++ include/linux/pci.h | 8 + 13 files changed, 775 insertions(+), 242 deletions(-) create mode 100644 drivers/pci/hotplug/pciehp_emul.cSignature · 1 line
1.8.3.1
Hi Bjorn,
Quoted 44 lines · Bjorn Helgaas
[+cc Stuart, Lukas]
Quoted 41 lines · Jon Derrick
This set adds an emulation driver for PCIe Hotplug. There may be platforms with specific configurations that can support hotplug but don't provide the logical slot hotplug hardware. For instance, the platform may use an electrically-tolerant interposer between the slot and the device.
This driver utilizes the pci-bridge-emul architecture to manage register reads and writes. The underlying functionality of the hotplug emulation driver uses the Data Link Layer Link Active Reporting mechanism in a polling loop, but can tolerate other event sources such as AER or DPC.
When enabled and a slot is managed by the driver, all port services are managed by the kernel. This is done to ensure that firmware hotplug and error architecture does not (correctly) halt/machine check the system when hotplug is performed on a non-hotplug slot.
The driver offers two active mode: Auto and Force.
auto: The driver will bind to non-hotplug slots
force: The driver will bind to all slots and overrides the slot's servicesThere are three kernel params:
pciehp.pciehp_emul_mode={off, auto, force} pciehp.pciehp_emul_time=<msecs polling time> (def 1000, min 100, max 60000) pciehp.pciehp_emul_ports=<PCI [S]BDF/ID format string>The
pciehp_emul_portskernel parameter takes a semi-colon tokenized string representing PCI [S]BDFs and IDs. Thepciehp_emul_modewill then be applied to only those slots, leaving other slots unmanaged bypciehp_emul.The string follows the
pci_dev_str_match()format:[<domain>:]<bus>:<device>.<func>[/<device>.<func>]* pci:<vendor>:<device>[:<subvendor>:<subdevice>]
When using the path format, the path for the device can be obtained using 'lspci -t' and further specified using the upstream bridge and the downstream port's device-function to be more robust against bus renumbering.
When using the vendor-device format, a value of '0' in any field acts as a wildcard for that field, matching all values.
Quoted 7 lines
Quoted 4 lines
The driver is enabled with
CONFIG_HOTPLUG_PCI_PCIE_EMUL=y.The driver should be considered 'use at own risk' unless the platform/hardware vendor recommends this mode.
There's a lot of good work in here, and I don't claim to understand the use case and all the benefits.
I've received more info that the customer use case is an AIC that breaks out 1-4 M.2 cards which have been made hotplug tolerant.
Quoted 11 lines
But it seems like quite a lot of additional code and complexity in an area that's already pretty subtle, so I'm not yet convinced that it's all worthwhile.
It seems like this would rely on Data Link Layer Link Active Reporting. Is that something we could add to pciehp as a generic feature without making a separate driver for it? I haven't looked at this for a while, but I would assume that if we find out that a link went down, pciehp could/should be smart enough to notice that even if it didn't come via the usual pciehp Slot Status path.
I had a plan to do V2 by intercepting bus_ops rather than indirecting slot_ops in pciehp. That should touch /a lot/ less code.
The problem I saw with adding DLLLA as a primary signal in pciehp is that most of the pciehp boilerplate relies on valid Slot register logic. I don't know how reliable pciehp will be if there's no backing slot register logic, emulated or real. Consider how many slot capability reads are in hpc.
I could add a non-slot flag check to each of those callers, but it might be worse than the emulation alternative.
What do you think?
Thanks
Quoted 31 lines
Quoted 31 lines
Jon Derrick (9):
PCI: pci-bridge-emul: Update PCIe register behaviors
PCI: pci-bridge-emul: Eliminate reserved member
PCI: pci-bridge-emul: Provide a helper to set behavior
PCI: pciehp: Indirect slot register operations
PCI: Addpcie_port_slot_emulatedstub
PCI: pciehp: Expose the poll loop to other drivers
PCI: Movepci_dev_str_matchtosearch.c
PCI: pciehp: Add hotplug slot emulation driver
PCI: pciehp: Wire uppcie_port_emulate_slotandpciehp_emuldrivers/pci/hotplug/Makefile | 4 + drivers/pci/hotplug/pciehp.h | 28 +++ drivers/pci/hotplug/pciehp_emul.c | 378 ++++++++++++++++++++++++++++++++++++++ drivers/pci/hotplug/pciehp_hpc.c | 136 ++++++++++---- drivers/pci/pci-acpi.c | 3 + drivers/pci/pci-bridge-emul.c | 95 +++++----- drivers/pci/pci-bridge-emul.h | 10 + drivers/pci/pci.c | 163 ---------------- drivers/pci/pcie/Kconfig | 14 ++ drivers/pci/pcie/portdrv_core.c | 14 +- drivers/pci/probe.c | 2 +- drivers/pci/search.c | 162 ++++++++++++++++ include/linux/pci.h | 8 + 13 files changed, 775 insertions(+), 242 deletions(-) create mode 100644 drivers/pci/hotplug/pciehp_emul.cSignature · 1 line
1.8.3.1
Quoted 4 lines · Jonathan
Quoted 2 lines
There's a lot of good work in here, and I don't claim to understand the use case and all the benefits.
I've received more info that the customer use case is an AIC that breaks out 1-4 M.2 cards which have been made hotplug tolerant.
Which sounds completely bogus. M.2 cards are eletrically not designed for this at all, and neither is the AIC. If people want to support similar use cases they need to get them standardized by PCI SIG and handled by hotplug capable slots.
Quoted 17 lines · Jonathan
Quoted 9 lines · Bjorn Helgaas
Quoted 5 lines · Jon Derrick
This set adds an emulation driver for PCIe Hotplug. There may be platforms with specific configurations that can support hotplug but don't provide the logical slot hotplug hardware. For instance, the platform may use an electrically-tolerant interposer between the slot and the device.
...There's a lot of good work in here, and I don't claim to understand the use case and all the benefits.
I've received more info that the customer use case is an AIC that breaks out 1-4 M.2 cards which have been made hotplug tolerant.
Quoted 4 lines
But it seems like quite a lot of additional code and complexity in an area that's already pretty subtle, so I'm not yet convinced that it's all worthwhile.
Quoted 8 lines
Quoted 6 lines
It seems like this would rely on Data Link Layer Link Active Reporting. Is that something we could add to pciehp as a generic feature without making a separate driver for it? I haven't looked at this for a while, but I would assume that if we find out that a link went down, pciehp could/should be smart enough to notice that even if it didn't come via the usual pciehp Slot Status path.
I had a plan to do V2 by intercepting
bus_opsrather than indirectingslot_opsin pciehp. That should touch /a lot/ less code.
I assume this is something like pci_bus_set_ops() or pci_bus_set_aer_ops()? Probably touches less code, but I'm not really a fan of either of those current situations because they make things magical -- there's a lot of stuff going on behind the curtain, and it makes another thing to consider when we evaluate every pciehp change.
Quoted 8 lines
The problem I saw with adding DLLLA as a primary signal in pciehp is that most of the pciehp boilerplate relies on valid Slot register logic. I don't know how reliable pciehp will be if there's no backing slot register logic, emulated or real. Consider how many slot capability reads are in hpc.
I could add a non-slot flag check to each of those callers, but it might be worse than the emulation alternative.
I see what you mean -- there are quite a few reads of PCI_EXP_SLTSTA. I'm not 100% sure all of those really need to exist. I would expect that we'd read it once in the ISR and then operate on that value. So maybe there's some middle ground of restructuring to remove some of those reads and making the remaining few more generic.
All that being said, I'm also sympathetic to Christoph's concern about cluttering pciehp to deal with non-standard topologies. At some point if you need to do non-standard things you may have to write and maintain your own drivers. I don't know where that point is yet.
Bjorn