PCI: Add Secondary Bus Reset (SBR) support for CXL
Subsystems (5)
Dave Jiang
maintainerv6 cover PCI: Add Secondary Bus Reset (SBR) support for CXL
Hi Bjorn,
Please consider this series for the next appropriate kernel merge window. The CXL part has review tags from Dan and Jonathan. It can go through your tree. The series attempt to add secondary bus reset (SBR) support to CXL. By default, SBR for CXL is masked. Per CXL specification r3.1 8.1.5.2, the Port Control Extensions register bit 0 (Unmask SBR) in the host bridge controls the masking of CXL SBR. "When 0, SBR bit in Bridge Control register of this Port has no effect. When 1, the Port shall generate hot reset when SBR in Bridge Control gets set to 1. Default value of this bit is 0. When the Port is operating in PCIe mode or RCD mode, this field has no effect on SBR functionality and Port shall follow PCIe Base Specification."
v6:
- Split out locking of upstream bridge to a separate patch and add lockdep support. (Dan)
- Added Dan's review tags for last 2 patches.
v5:
- Move bridge locking to pci_reset_function(). (Dan)
- Simplify retrieval of cxld. (Dan)
- Add lock to device to prevent racing of disabled cxlmd. (Dan)
- Promote dev_warn() to dev_crit() (Dan)
- Remove LOCKDEP_NOT_UNRELIABLE flag. (Dan)
v4:
- Return u16 for cxl_port_dvsec(). (Lukas)
- Use pci_dev_lock guard() on bridge. (Lukas)
- Clarify commit subject on 4/4. (Jonathan)
v3:
- Move PCI_DVSEC_VENDOR_ID_CXL to PCI_VENDOR_ID_CXL in pci_ids.h (Bjorn)
- Remove of CXL device checking. (Bjorn)
- Rename defines to PCI_DVSEC_CXL_PORT_*. (Bjorn)
- Fix SBR define in commit log. (Bjorn)
- Update comment on dvsec not found. (Dan)
- Check return of dvsec value read for error. (Dan)
- Move cxl_port_dvsec() to an earlier patch. (Dan)
- Add pci_cfg_access_lock() for bridge access. (Dan)
- Change cxl_bus_force method to cxl_bus. (Dan)
- Rename decoder_hw_mismatch() to __cxl_endpoint_decoder_reset_detected(). (Dan)
- Move CXL register access function to core/pci.c. (Dan)
- Add kernel taint to decoder reset warning. (Dan)
v2:
- Use pci_upstream_bridge() instead of dev->bus->self. (Lukas)
- Rename is_cxl_device() to pci_is_cxl(). (Lukas)
- Return -ENOTTY on error. (Lukas)
Patch 1:
Move PCI_DVSEC_VENDOR_ID_CXL to PCI_VENDOR_ID_CXL in pci_ids.h and adjust related CXL bits.
Patch 2:
Add locking of the upstream bridge to keep configuration of the bridge consistent.
Patch 3:
Add check to PCI bus_reset path for CXL device and return with error if "Unmask SBR" bit is set to 0. This allows user to realize that SBR is masked for this CXL device. However, if the user sets the "Unmask SBR" bit via a tool such as setpci, then the bus_reset will proceed.
Patch4:
Add a new PCI reset method "cxl_bus" in order to allow the user an intetional way to perform SBR. The code will set the "Unmask SBR" bit to 1 and proceed with bus_reset. The original value of the bit will be restored after the reset operation.
Patch5:
CXL driver change that provides a ->reset_done() callback. It compares the hardware decoder settings with the existing software configuration and emit warning if they differ. The difference indicates that decoders were programmed before the reset and are now cleared after reset. There may be onlined system memory backed by CXL memory device that are now violently ripped away from kernel mapping.
Patch series stemmed from this [1] patch. With comments [2] from Bjorn.
[1]: https://lore.kernel.org/linux-cxl/[email protected]/
[2]: https://lore.kernel.org/linux-cxl/20240220203956.GA1502351@bhelgaas/
Dave Jiang
maintainerPATCH v6 1/5 PCI/cxl: Move PCI CXL vendor Id to a common location from CXL subsystem
Move PCI_DVSEC_VENDOR_ID_CXL in CXL private code to PCI_VENDOR_ID_CXL in pci_ids.h in order to be utilized in PCI subsystem.
When uplevelling PCI_DVSEC_VENDOR_ID_CXL to a common locatoin Bjorn suggested making it a proper PCI_VENDOR_ID_* define in include/linux/pci_ids.h. While it is not in the PCI IDs database it is a reserved value and Linux treats it as a 'vendor id' for all intents and purposes [1].
--- drivers/cxl/core/pci.c | 6 +++--- drivers/cxl/core/regs.c | 2 +- drivers/cxl/cxlpci.h | 1 - drivers/cxl/pci.c | 2 +- drivers/perf/cxl_pmu.c | 2 +- include/linux/pci_ids.h | 2 ++ 6 files changed, 8 insertions(+), 7 deletions(-)
drivers/cxl/core/pci.c +3 -3
@@ -525,7 +525,7 @@ static int cxl_cdat_get_length(struct device *dev,__le32 response[2 ];int rc; - rc =pci_doe (doe_mb,PCI_DVSEC_VENDOR_ID_CXL , + rc =pci_doe (doe_mb,PCI_VENDOR_ID_CXL ,CXL_DOE_PROTOCOL_TABLE_ACCESS , &request,sizeof (request), &response,sizeof (response));
@@ -555,7 +555,7 @@ static int cxl_cdat_read_table(struct device *dev,__le32 request =CDAT_DOE_REQ (entry_handle);int rc; - rc =pci_doe (doe_mb,PCI_DVSEC_VENDOR_ID_CXL , + rc =pci_doe (doe_mb,PCI_VENDOR_ID_CXL ,CXL_DOE_PROTOCOL_TABLE_ACCESS , &request,sizeof (request), rsp,sizeof (*rsp) + remaining);
@@ -640,7 +640,7 @@ void read_cdat_data(struct cxl_port *port)if (!pdev)return ; - doe_mb =pci_find_doe_mailbox (pdev,PCI_DVSEC_VENDOR_ID_CXL , + doe_mb =pci_find_doe_mailbox (pdev,PCI_VENDOR_ID_CXL ,CXL_DOE_PROTOCOL_TABLE_ACCESS );if (!doe_mb) {dev_dbg (dev,"No CDAT mailbox\n" );
drivers/cxl/core/regs.c +1 -1
@@ -314,7 +314,7 @@ int cxl_find_regblock_instance(struct pci_dev *pdev, enum cxl_regloc_type type, .resource = CXL_RESOURCE_NONE, }; - regloc =pci_find_dvsec_capability (pdev,PCI_DVSEC_VENDOR_ID_CXL , + regloc =pci_find_dvsec_capability (pdev,PCI_VENDOR_ID_CXL ,CXL_DVSEC_REG_LOCATOR );if (!regloc)return -ENXIO ;
drivers/cxl/cxlpci.h +0 -1
@@ -13,7 +13,6 @@* "DVSEC" redundancies removed. When obvious, abbreviations may be used. */ #define PCI_DVSEC_HEADER1_LENGTH_MASK GENMASK(31, 20) -#define PCI_DVSEC_VENDOR_ID_CXL 0x1E98/* CXL 2.0 8.1.3: PCIe DVSEC for CXL Device */ #define CXL_DVSEC_PCIE_DEVICE 0
drivers/cxl/pci.c +1 -1
@@ -817,7 +817,7 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) cxlds->rcd =is_cxl_restricted (pdev); cxlds->serial =pci_get_dsn (pdev); cxlds->cxl_dvsec =pci_find_dvsec_capability ( - pdev,PCI_DVSEC_VENDOR_ID_CXL ,CXL_DVSEC_PCIE_DEVICE ); + pdev,PCI_VENDOR_ID_CXL ,CXL_DVSEC_PCIE_DEVICE );if (!cxlds->cxl_dvsec)dev_warn (&pdev->dev,"Device DVSEC not present, skip CXL.mem init\n" );
drivers/perf/cxl_pmu.c +1 -1
@@ -345,7 +345,7 @@ static ssize_t cxl_pmu_event_sysfs_show(struct device *dev,/* For CXL spec defined events */ #define CXL_PMU_EVENT_CXL_ATTR (_name, _gid, _msk) \ - CXL_PMU_EVENT_ATTR(_name, PCI_DVSEC_VENDOR_ID_CXL, _gid, _msk) + CXL_PMU_EVENT_ATTR(_name, PCI_VENDOR_ID_CXL, _gid, _msk)static struct attribute *cxl_pmu_event_attrs[] = {CXL_PMU_EVENT_CXL_ATTR (clock_ticks,CXL_PMU_GID_CLOCK_TICKS ,BIT (0 )),
include/linux/pci_ids.h +2 -0
@@ -2607,6 +2607,8 @@#define PCI_VENDOR_ID_ALIBABA 0x1ded +#define PCI_VENDOR_ID_CXL 0x1e98 +#define PCI_VENDOR_ID_TEHUTI 0x1fc9#define PCI_DEVICE_ID_TEHUTI_3009 0x3009#define PCI_DEVICE_ID_TEHUTI_3010 0x3010
Signature · 1 line
2.44.0
Dave Jiang
maintainerPATCH v6 2/5 PCI: Add locking of upstream bridge for pci_reset_function()
In reply to Dave JiangFix a long standing locking gap for missing pci_cfg_access_lock() while manipulating bridge reset registers and configuration during pci_reset_bus_function(). Add calling of pci_dev_lock() against the bridge device before locking the device. The locking is conditional depending on whether the trigger device has an upstream bridge. If the device is a root port then there would be no upstream bridge and thus the locking of the bridge is unnecessary. As part of calling pci_dev_lock(), pci_cfg_access_lock() happens and blocks the writing of PCI config space by user space.
Add lockdep assertion via pci_dev->cfg_access_lock in order to verify pci_dev->block_cfg_access is set.
--- drivers/pci/access.c | 4 ++++ drivers/pci/pci.c | 13 +++++++++++++ drivers/pci/probe.c | 3 +++ include/linux/lockdep.h | 5 +++++ include/linux/pci.h | 2 ++ 5 files changed, 27 insertions(+)
drivers/pci/access.c +4 -0
@@ -275,6 +275,8 @@ void pci_cfg_access_lock(struct pci_dev *dev) {might_sleep (); +lock_map_acquire (&dev->cfg_access_lock); +raw_spin_lock_irq (&pci_lock);if (dev->block_cfg_access)pci_wait_cfg (dev);
@@ -329,6 +331,8 @@ void pci_cfg_access_unlock(struct pci_dev *dev)raw_spin_unlock_irqrestore (&pci_lock, flags);wake_up_all (&pci_cfg_wait); + +lock_map_release (&dev->cfg_access_lock); }EXPORT_SYMBOL_GPL (pci_cfg_access_unlock);
drivers/pci/pci.c +13 -0
@@ -4879,6 +4879,7 @@ void __weak pcibios_reset_secondary_bus(struct pci_dev *dev)*/ int pci_bridge_secondary_bus_reset (struct pci_dev *dev) { +lock_map_assert_held (&dev->cfg_access_lock);pcibios_reset_secondary_bus (dev);return pci_bridge_wait_for_secondary_bus (dev,"bus reset" );
@@ -5245,11 +5246,20 @@ void pci_init_reset_methods(struct pci_dev *dev)*/ int pci_reset_function (struct pci_dev *dev) { +struct pci_dev *bridge;int rc;if (!pci_reset_supported (dev))return -ENOTTY ; + bridge =pci_upstream_bridge (dev); +/* +* If there's no upstream bridge, then no locking is needed since there is no +* upstream bridge configuration to hold consistent. +*/ +if (bridge) +pci_dev_lock (bridge); +pci_dev_lock (dev);pci_dev_save_and_disable (dev);
@@ -5258,6 +5268,9 @@ int pci_reset_function(struct pci_dev *dev)pci_dev_restore (dev);pci_dev_unlock (dev); +if (bridge) +pci_dev_unlock (bridge); +return rc; }EXPORT_SYMBOL_GPL (pci_reset_function);
drivers/pci/probe.c +3 -0
@@ -2543,6 +2543,9 @@ void pci_device_add(struct pci_dev *dev, struct pci_bus *bus) dev->dev.dma_mask = &dev->dma_mask; dev->dev.dma_parms = &dev->dma_parms; dev->dev.coherent_dma_mask =0xffffffffull ; +lockdep_register_key (&dev->cfg_access_key); +lockdep_init_map (&dev->cfg_access_lock,dev_name (&dev->dev), + &dev->cfg_access_key,0 );dma_set_max_seg_size (&dev->dev,65536 );dma_set_seg_boundary (&dev->dev,0xffffffff );
include/linux/lockdep.h +5 -0
@@ -297,6 +297,9 @@ extern void lock_unpin_lock(struct lockdep_map *lock, struct pin_cookie); .wait_type_inner = _wait_type, \ .lock_type = LD_LOCK_WAIT_OVERRIDE, } +#define lock_map_assert_held (l) \ + lockdep_assert(lock_is_held(l) != LOCK_STATE_NOT_HELD) +#else /* !CONFIG_LOCKDEP */ static inline void lockdep_init_task (struct task_struct *task)
@@ -388,6 +391,8 @@ extern int lockdep_is_held(const void *);#define DEFINE_WAIT_OVERRIDE_MAP (_name, _wait_type) \ struct lockdep_map __maybe_unused _name = {} +#define lock_map_assert_held (l) do { (void)(l); } while (0) +#endif /* !LOCKDEP */ #ifdef CONFIG_PROVE_LOCKING
include/linux/pci.h +2 -0
@@ -413,6 +413,8 @@ struct pci_dev {struct resource driver_exclusive_resource;/* driver exclusive resource ranges */ bool match_driver;/* Skip attaching driver */ +struct lock_class_key cfg_access_key; +struct lockdep_map cfg_access_lock; unsigned int transparent:1; /* Subtractive decode bridge */ unsigned int io_window:1; /* Bridge has I/O window */
Signature · 1 line
2.44.0
Re: [PATCH v6 2/5] PCI: Add locking of upstream bridge for pci_reset_function()
Quoted 38 lines · On Thu, 2 May 2024 09:57:31 -0700 Dave Jiang <[email protected]> wrote:
Fix a long standing locking gap for missing
pci_cfg_access_lock()while manipulating bridge reset registers and configuration duringpci_reset_bus_function(). Add calling ofpci_dev_lock()against the bridge device before locking the device. The locking is conditional depending on whether the trigger device has an upstream bridge. If the device is a root port then there would be no upstream bridge and thus the locking of the bridge is unnecessary. As part of callingpci_dev_lock(),pci_cfg_access_lock()happens and blocks the writing of PCI config space by user space.Add lockdep assertion via
pci_dev->cfg_access_lockin order to verifypci_dev->block_cfg_accessis set.Co-developed-by Dan Williams <[email protected]>Signed-off-by Dan Williams <[email protected]>Signed-off-by Dave Jiang <[email protected]>--- drivers/pci/access.c | 4 ++++ drivers/pci/pci.c | 13 +++++++++++++ drivers/pci/probe.c | 3 +++ include/linux/lockdep.h | 5 +++++ include/linux/pci.h | 2 ++ 5 files changed, 27 insertions(+)drivers/pci/access.c +2
-0@@ -275,6 +275,8 @@ void pci_cfg_access_lock(struct pci_dev *dev) {might_sleep (); +lock_map_acquire (&dev->cfg_access_lock); +raw_spin_lock_irq (&pci_lock);if (dev->block_cfg_access)pci_wait_cfg (dev);
drivers/pci/access.c void pci_cfg_access_unlock(struct pci_dev *dev) · line 335
Quoted 5 lines
drivers/pci/access.c +2
-0@@ -329,6 +331,8 @@ void pci_cfg_access_unlock(struct pci_dev *dev)raw_spin_unlock_irqrestore (&pci_lock, flags);wake_up_all (&pci_cfg_wait); + +lock_map_release (&dev->cfg_access_lock);
This doesn't account for config access locks acquired via pci_cfg_access_trylock(), such as the pci_dev_trylock() through pci_try_reset_function() resulting in a new lockdep warning for vfio-pci when we try to release a lock that was never acquired. Thanks,
Alex
Quoted 96 lines
drivers/pci/access.c +0
-0}EXPORT_SYMBOL_GPL (pci_cfg_access_unlock);drivers/pci/pci.c +13
-0@@ -4879,6 +4879,7 @@ void __weak pcibios_reset_secondary_bus(struct pci_dev *dev)*/ int pci_bridge_secondary_bus_reset (struct pci_dev *dev) { +lock_map_assert_held (&dev->cfg_access_lock);pcibios_reset_secondary_bus (dev);return pci_bridge_wait_for_secondary_bus (dev,"bus reset" );@@ -5245,11 +5246,20 @@ void pci_init_reset_methods(struct pci_dev *dev)*/ int pci_reset_function (struct pci_dev *dev) { +struct pci_dev *bridge;int rc;if (!pci_reset_supported (dev))return -ENOTTY ; + bridge =pci_upstream_bridge (dev); +/* +* If there's no upstream bridge, then no locking is needed since there is no +* upstream bridge configuration to hold consistent. +*/ +if (bridge) +pci_dev_lock (bridge); +pci_dev_lock (dev);pci_dev_save_and_disable (dev);@@ -5258,6 +5268,9 @@ int pci_reset_function(struct pci_dev *dev)pci_dev_restore (dev);pci_dev_unlock (dev); +if (bridge) +pci_dev_unlock (bridge); +return rc; }EXPORT_SYMBOL_GPL (pci_reset_function);drivers/pci/probe.c +3
-0@@ -2543,6 +2543,9 @@ void pci_device_add(struct pci_dev *dev, struct pci_bus *bus) dev->dev.dma_mask = &dev->dma_mask; dev->dev.dma_parms = &dev->dma_parms; dev->dev.coherent_dma_mask =0xffffffffull ; +lockdep_register_key (&dev->cfg_access_key); +lockdep_init_map (&dev->cfg_access_lock,dev_name (&dev->dev), + &dev->cfg_access_key,0 );dma_set_max_seg_size (&dev->dev,65536 );dma_set_seg_boundary (&dev->dev,0xffffffff );include/linux/lockdep.h +5
-0@@ -297,6 +297,9 @@ extern void lock_unpin_lock(struct lockdep_map *lock, struct pin_cookie); .wait_type_inner = _wait_type, \ .lock_type = LD_LOCK_WAIT_OVERRIDE, } +#define lock_map_assert_held (l) \ + lockdep_assert(lock_is_held(l) != LOCK_STATE_NOT_HELD) +#else /* !CONFIG_LOCKDEP */ static inline void lockdep_init_task (struct task_struct *task)@@ -388,6 +391,8 @@ extern int lockdep_is_held(const void *);#define DEFINE_WAIT_OVERRIDE_MAP (_name, _wait_type) \ struct lockdep_map __maybe_unused _name = {} +#define lock_map_assert_held (l) do { (void)(l); } while (0) +#endif /* !LOCKDEP */ #ifdef CONFIG_PROVE_LOCKING include/linux/pci.h +2
-0@@ -413,6 +413,8 @@ struct pci_dev {struct resource driver_exclusive_resource;/* driver exclusive resource ranges */ bool match_driver;/* Skip attaching driver */ +struct lock_class_key cfg_access_key; +struct lockdep_map cfg_access_lock; unsigned int transparent:1; /* Subtractive decode bridge */ unsigned int io_window:1; /* Bridge has I/O window */
Re: [PATCH v6 2/5] PCI: Add locking of upstream bridge for pci_reset_function()
Quoted 20 lines · Alex Williamson
Quoted 17 lines · On Thu, 2 May 2024 09:57:31 -0700 Dave Jiang <[email protected]> wrote:
Fix a long standing locking gap for missing
pci_cfg_access_lock()while manipulating bridge reset registers and configuration duringpci_reset_bus_function(). Add calling ofpci_dev_lock()against the bridge device before locking the device. The locking is conditional depending on whether the trigger device has an upstream bridge. If the device is a root port then there would be no upstream bridge and thus the locking of the bridge is unnecessary. As part of callingpci_dev_lock(),pci_cfg_access_lock()happens and blocks the writing of PCI config space by user space.Add lockdep assertion via
pci_dev->cfg_access_lockin order to verifypci_dev->block_cfg_accessis set.Co-developed-by Dan Williams <[email protected]>Signed-off-by Dan Williams <[email protected]>Signed-off-by Dave Jiang <[email protected]>---
[..]
Quoted 16 lines
Quoted 16 lines
drivers/pci/access.c +2
-0@@ -275,6 +275,8 @@ void pci_cfg_access_lock(struct pci_dev *dev) {might_sleep (); +lock_map_acquire (&dev->cfg_access_lock); +raw_spin_lock_irq (&pci_lock);if (dev->block_cfg_access)pci_wait_cfg (dev);@@ -329,6 +331,8 @@ void pci_cfg_access_unlock(struct pci_dev *dev)raw_spin_unlock_irqrestore (&pci_lock, flags);
Quoted 10 lines
drivers/pci/access.c void pci_cfg_access_unlock(struct pci_dev *dev) · line 335
Quoted 3 lines
drivers/pci/access.c +2
-0wake_up_all (&pci_cfg_wait); + +lock_map_release (&dev->cfg_access_lock);This doesn't account for config access locks acquired via
pci_cfg_access_trylock(), such as thepci_dev_trylock()throughpci_try_reset_function()resulting in a new lockdep warning for vfio-pci when we try to release a lock that was never acquired. Thanks,
Hey Alex, sorry about that, does this fix it up for you? Note I move the lock_map_acquire() relative to the global pci_lock just for symmetry purposes.
-- >8 --
drivers/pci/access.c +5 -5
@@ -289,11 +289,10 @@ void pci_cfg_access_lock(struct pci_dev *dev) {might_sleep (); -lock_map_acquire (&dev->cfg_access_lock); -raw_spin_lock_irq (&pci_lock);if (dev->block_cfg_access)pci_wait_cfg (dev); +lock_map_acquire (&dev->cfg_access_lock); dev->block_cfg_access =1 ;raw_spin_unlock_irq (&pci_lock); }
@@ -315,8 +314,10 @@ bool pci_cfg_access_trylock(struct pci_dev *dev)raw_spin_lock_irqsave (&pci_lock, flags);if (dev->block_cfg_access) locked = false; -else +else { +lock_map_acquire (&dev->cfg_access_lock); dev->block_cfg_access =1 ; + }raw_spin_unlock_irqrestore (&pci_lock, flags);return locked;
@@ -342,11 +343,10 @@ void pci_cfg_access_unlock(struct pci_dev *dev)WARN_ON (!dev->block_cfg_access); dev->block_cfg_access =0 ; +lock_map_release (&dev->cfg_access_lock);raw_spin_unlock_irqrestore (&pci_lock, flags);wake_up_all (&pci_cfg_wait); - -lock_map_release (&dev->cfg_access_lock); }EXPORT_SYMBOL_GPL (pci_cfg_access_unlock);
Re: [PATCH v6 2/5] PCI: Add locking of upstream bridge for pci_reset_function()
Quoted 49 lines · On Thu, 23 May 2024 14:17:58 -0700 Dan Williams <[email protected]>…
Quoted 20 lines · Alex Williamson
Quoted 17 lines · On Thu, 2 May 2024 09:57:31 -0700 Dave Jiang <[email protected]> wrote:
Fix a long standing locking gap for missing
pci_cfg_access_lock()while manipulating bridge reset registers and configuration duringpci_reset_bus_function(). Add calling ofpci_dev_lock()against the bridge device before locking the device. The locking is conditional depending on whether the trigger device has an upstream bridge. If the device is a root port then there would be no upstream bridge and thus the locking of the bridge is unnecessary. As part of callingpci_dev_lock(),pci_cfg_access_lock()happens and blocks the writing of PCI config space by user space.Add lockdep assertion via
pci_dev->cfg_access_lockin order to verifypci_dev->block_cfg_accessis set.Co-developed-by Dan Williams <[email protected]>Signed-off-by Dan Williams <[email protected]>Signed-off-by Dave Jiang <[email protected]>---
[..]
Quoted 16 lines
Quoted 16 lines
drivers/pci/access.c +2
-0@@ -275,6 +275,8 @@ void pci_cfg_access_lock(struct pci_dev *dev) {might_sleep (); +lock_map_acquire (&dev->cfg_access_lock); +raw_spin_lock_irq (&pci_lock);if (dev->block_cfg_access)pci_wait_cfg (dev);@@ -329,6 +331,8 @@ void pci_cfg_access_unlock(struct pci_dev *dev)raw_spin_unlock_irqrestore (&pci_lock, flags);Quoted 10 lines
drivers/pci/access.c void pci_cfg_access_unlock(struct pci_dev *dev) · line 335
Quoted 3 lines
drivers/pci/access.c +2
-0wake_up_all (&pci_cfg_wait); + +lock_map_release (&dev->cfg_access_lock);This doesn't account for config access locks acquired via
pci_cfg_access_trylock(), such as thepci_dev_trylock()throughpci_try_reset_function()resulting in a new lockdep warning for vfio-pci when we try to release a lock that was never acquired. Thanks,
Quoted 3 lines
Hey Alex, sorry about that, does this fix it up for you? Note I move the
lock_map_acquire()relative to the globalpci_lockjust for symmetry purposes.
Thanks, Dan! Yes, this fixes it. Please feel free to add Reported/Tested-by tags when you post it. Thanks!
Alex
Quoted 44 lines
-- >8 --
drivers/pci/access.c +5
-5@@ -289,11 +289,10 @@ void pci_cfg_access_lock(struct pci_dev *dev) {might_sleep (); -lock_map_acquire (&dev->cfg_access_lock); -raw_spin_lock_irq (&pci_lock);if (dev->block_cfg_access)pci_wait_cfg (dev); +lock_map_acquire (&dev->cfg_access_lock); dev->block_cfg_access =1 ;raw_spin_unlock_irq (&pci_lock); }@@ -315,8 +314,10 @@ bool pci_cfg_access_trylock(struct pci_dev *dev)raw_spin_lock_irqsave (&pci_lock, flags);if (dev->block_cfg_access) locked = false; -else +else { +lock_map_acquire (&dev->cfg_access_lock); dev->block_cfg_access =1 ; + }raw_spin_unlock_irqrestore (&pci_lock, flags);return locked;@@ -342,11 +343,10 @@ void pci_cfg_access_unlock(struct pci_dev *dev)WARN_ON (!dev->block_cfg_access); dev->block_cfg_access =0 ; +lock_map_release (&dev->cfg_access_lock);raw_spin_unlock_irqrestore (&pci_lock, flags);wake_up_all (&pci_cfg_wait); - -lock_map_release (&dev->cfg_access_lock); }EXPORT_SYMBOL_GPL (pci_cfg_access_unlock);
Dave Jiang
maintainerPATCH v6 3/5 PCI: Add check for CXL Secondary Bus Reset
In reply to Dave JiangPer CXL spec r3.1 8.1.5.2, Secondary Bus Reset (SBR) is masked unless the "Unmask SBR" bit is set. Add a check to the PCI secondary bus reset path to fail the CXL SBR request if the "Unmask SBR" bit is clear in the CXL Port Control Extensions register by returning -ENOTTY.
Otherwise when the "Unmask SBR" bit is set to 0 (default), the bus_reset would appear to have executed successfully. However the operation is actually masked. The intention is to inform the user that SBR for the CXL device is masked and will not go through.
If the "Unmask SBR" bit is set to 1, then the bus reset will execute successfully.
Add the locking of the upstream bridge in pci_reset_function() to ensure the locking order of locking the bridge then locking the device. The bridge configuration needs to be consistent for a CXL device. This should not impact PCI devices.
---
v6:
- Split out upstream bridge locking. (Dan)
--- drivers/pci/pci.c | 45 +++++++++++++++++++++++++++++++++++ include/uapi/linux/pci_regs.h | 5 ++++ 2 files changed, 50 insertions(+)
drivers/pci/pci.c +45 -0
@@ -4928,10 +4928,55 @@ static int pci_dev_reset_slot_function(struct pci_dev *dev, bool probe)return pci_reset_hotplug_slot (dev->slot->hotplug, probe); } +static u16 cxl_port_dvsec (struct pci_dev *dev) +{ +return pci_find_dvsec_capability (dev,PCI_VENDOR_ID_CXL , +PCI_DVSEC_CXL_PORT ); +} + +static bool cxl_sbr_masked (struct pci_dev *dev) +{ +u16 dvsec, reg; +int rc; + +/* +* No DVSEC found, either is not a CXL port, or not connected in which +* case mask state is a nop (CXL r3.1 sec 9.12.3 "Enumerating CXL RPs +* and DSPs" +*/ + dvsec =cxl_port_dvsec (dev); +if (!dvsec) +return false; + + rc =pci_read_config_word (dev, dvsec +PCI_DVSEC_CXL_PORT_CTL , ®); +if (rc ||PCI_POSSIBLE_ERROR (reg)) +return false; + +/* +* CXL spec r3.1 8.1.5.2 +* When 0, SBR bit in Bridge Control register of this Port has no effect. +* When 1, the Port shall generate hot reset when SBR bit in Bridge +* Control gets set to 1. +*/ +if (reg &PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR ) +return false; + +return true; +} +static int pci_reset_bus_function (struct pci_dev *dev,bool probe) { +struct pci_dev *bridge =pci_upstream_bridge (dev);int rc; +/* If it's a CXL port and the SBR control is masked, fail the SBR */ +if (bridge &&cxl_sbr_masked (bridge)) { +if (probe) +return 0 ; + +return -ENOTTY ; + } + rc =pci_dev_reset_slot_function (dev, probe);if (rc != -ENOTTY )return rc;
include/uapi/linux/pci_regs.h +5 -0
@@ -1148,4 +1148,9 @@#define PCI_DOE_DATA_OBJECT_DISC_RSP_3_PROTOCOL 0x00ff0000#define PCI_DOE_DATA_OBJECT_DISC_RSP_3_NEXT_INDEX 0xff000000 +/* Compute Express Link (CXL) */ +#define PCI_DVSEC_CXL_PORT 3 +#define PCI_DVSEC_CXL_PORT_CTL 0x0c +#define PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR 0x00000001 +#endif /* LINUX_PCI_REGS_H */
Signature · 1 line
2.44.0
Dave Jiang
maintainerPATCH v6 4/5 PCI: Create new reset method to force SBR for CXL
In reply to Dave JiangCXL spec r3.1 8.1.5.2
By default Secondary Bus Reset (SBR) is masked for CXL ports. Introduce a new PCI reset method "cxl_bus" to force SBR on CXL ports by setting the unmask SBR bit in the CXL DVSEC port control register before performing the bus reset and restore the original value of the bit post reset. The new reset method allows the user to intentionally perform SBR on a CXL device without needing to set the "Unmask SBR" bit via a user tool.
--- drivers/pci/pci.c | 38 ++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 2 +- 2 files changed, 39 insertions(+), 1 deletion(-)
drivers/pci/pci.c +38 -0
@@ -4983,6 +4983,43 @@ static int pci_reset_bus_function(struct pci_dev *dev, bool probe)return pci_parent_bus_reset (dev, probe); } +static int cxl_reset_bus_function (struct pci_dev *dev,bool probe) +{ +struct pci_dev *bridge; +u16 dvsec, reg, val; +int rc; + + bridge =pci_upstream_bridge (dev); +if (!bridge) +return -ENOTTY ; + + dvsec =cxl_port_dvsec (bridge); +if (!dvsec) +return -ENOTTY ; + +if (probe) +return 0 ; + + rc =pci_read_config_word (bridge, dvsec +PCI_DVSEC_CXL_PORT_CTL , ®); +if (rc) +return -ENOTTY ; + +if (!(reg &PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR )) { + val = reg |PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR ; +pci_write_config_word (bridge, dvsec +PCI_DVSEC_CXL_PORT_CTL , + val); + }else { + val = reg; + } + + rc =pci_reset_bus_function (dev, probe); + +if (reg != val) +pci_write_config_word (bridge, dvsec +PCI_DVSEC_CXL_PORT_CTL , reg); + +return rc; +} +void pci_dev_lock (struct pci_dev *dev) {/* block PM suspend, driver probe, etc. */
@@ -5067,6 +5104,7 @@ static const struct pci_reset_fn_method pci_reset_fn_methods[] = { { pci_af_flr, .name = "af_flr" }, { pci_pm_reset, .name = "pm" }, { pci_reset_bus_function, .name = "bus" }, + { cxl_reset_bus_function, .name = "cxl_bus" }, }; static ssize_t reset_method_show(struct device *dev,
include/linux/pci.h +1 -1
@@ -51,7 +51,7 @@ PCI_STATUS_PARITY)/* Number of reset methods used in pci_reset_fn_methods array in pci.c */ -#define PCI_NUM_RESET_METHODS 7 +#define PCI_NUM_RESET_METHODS 8#define PCI_RESET_PROBE true#define PCI_RESET_DO_RESET false
Signature · 1 line
2.44.0
Dave Jiang
maintainerPATCH v6 5/5 cxl: Add post reset warning if reset results in loss of previously committed HDM decoders
In reply to Dave JiangSBR is equivalent to a device been hot removed and inserted again. Doing a SBR on a CXL type 3 device is problematic if the exported device memory is part of system memory that cannot be offlined. The event is equivalent to violently ripping out that range of memory from the kernel. While the hardware requires the "Unmask SBR" bit set in the Port Control Extensions register and the kernel currently does not unmask it, user can unmask this bit via setpci or similar tool.
The driver does not have a way to detect whether a reset coming from the PCI subsystem is a Function Level Reset (FLR) or SBR. The only way to detect is to note if a decoder is marked as enabled in software but the decoder control register indicates it's not committed.
A helper function is added to find discrepancy between the decoder software state versus the hardware register state.
--- drivers/cxl/core/pci.c | 29 +++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 2 ++ drivers/cxl/pci.c | 22 ++++++++++++++++++++++ 3 files changed, 53 insertions(+)
drivers/cxl/core/pci.c +29 -0
@@ -1045,3 +1045,32 @@ long cxl_pci_get_latency(struct pci_dev *pdev)return cxl_flit_size (pdev) *MEGA / bw; } + +static int __cxl_endpoint_decoder_reset_detected (struct device *dev,void *data) +{ +struct cxl_port *port = data; +struct cxl_decoder *cxld; +struct cxl_hdm *cxlhdm; +void __iomem *hdm; +u32 ctrl; + +if (!is_endpoint_decoder (dev)) +return 0 ; + + cxld =to_cxl_decoder (dev); +if ((cxld->flags &CXL_DECODER_F_ENABLE ) ==0 ) +return 0 ; + + cxlhdm =dev_get_drvdata (&port->dev); + hdm = cxlhdm->regs.hdm_decoder; + ctrl =readl (hdm +CXL_HDM_DECODER0_CTRL_OFFSET (cxld->id)); + +return !FIELD_GET (CXL_HDM_DECODER0_CTRL_COMMITTED , ctrl); +} + +bool cxl_endpoint_decoder_reset_detected (struct cxl_port *port) +{ +return device_for_each_child (&port->dev, port, + __cxl_endpoint_decoder_reset_detected); +} +EXPORT_SYMBOL_NS_GPL (cxl_endpoint_decoder_reset_detected,CXL );
drivers/cxl/cxl.h +2 -0
@@ -891,6 +891,8 @@ void cxl_coordinates_combine(struct access_coordinate *out,struct access_coordinate *c1, struct access_coordinate *c2); +bool cxl_endpoint_decoder_reset_detected (struct cxl_port *port); +/* * Unit test builds overrides this to __weak, find the 'strong' version * of these symbols in tools/testing/cxl/.
drivers/cxl/pci.c +22 -0
@@ -957,11 +957,33 @@ static void cxl_error_resume(struct pci_dev *pdev) dev->driver ? "successful" : "failed"); } +static void cxl_reset_done (struct pci_dev *pdev) +{ +struct cxl_dev_state *cxlds =pci_get_drvdata (pdev); +struct cxl_memdev *cxlmd = cxlds->cxlmd; +struct device *dev = &pdev->dev; + +/* +* FLR does not expect to touch the HDM decoders and related registers. +* SBR however will wipe all device configurations. +* Issue warning if there was active decoder before reset that no +* longer exists. +*/ +guard (device)(&cxlmd->dev); +if (cxlmd->endpoint && +cxl_endpoint_decoder_reset_detected (cxlmd->endpoint)) { +dev_crit (dev,"SBR happened without memory regions removal.\n" ); +dev_crit (dev,"System may be unstable if regions hosted system memory.\n" ); +add_taint (TAINT_USER ,LOCKDEP_STILL_OK ); + } +} +static const struct pci_error_handlers cxl_error_handlers = { .error_detected = cxl_error_detected, .slot_reset = cxl_slot_reset, .resume = cxl_error_resume, .cor_error_detected = cxl_cor_error_detected, + .reset_done = cxl_reset_done, };static struct pci_driver cxl_pci_driver = {
Signature · 1 line
2.44.0
Quoted 11 lines · Dave Jiang
Hi Bjorn,
Please consider this series for the next appropriate kernel merge window. The CXL part has review tags from Dan and Jonathan. It can go through your tree. The series attempt to add secondary bus reset (SBR) support to CXL. By default, SBR for CXL is masked. Per CXL specification r3.1 8.1.5.2, the Port Control Extensions register bit 0 (Unmask SBR) in the host bridge controls the masking of CXL SBR. "When 0, SBR bit in Bridge Control register of this Port has no effect. When 1, the Port shall generate hot reset when SBR in Bridge Control gets set to 1. Default value of this bit is 0. When the Port is operating in PCIe mode or RCD mode, this field has no effect on SBR functionality and Port shall follow PCIe Base Specification."
Applied with minor cleanups to pci/cxl for v6.10, thanks!
Quoted 68 lines
v6:
- Split out locking of upstream bridge to a separate patch and add lockdep support. (Dan)
- Added Dan's review tags for last 2 patches.v5:
- Move bridge locking topci_reset_function(). (Dan)
- Simplify retrieval of cxld. (Dan)
- Add lock to device to prevent racing of disabled cxlmd. (Dan)
- Promotedev_warn()todev_crit()(Dan)
- RemoveLOCKDEP_NOT_UNRELIABLEflag. (Dan)v4:
- Return u16 forcxl_port_dvsec(). (Lukas)
- Usepci_dev_lockguard()on bridge. (Lukas)
- Clarify commit subject on 4/4. (Jonathan)v3:
- MovePCI_DVSEC_VENDOR_ID_CXLtoPCI_VENDOR_ID_CXLinpci_ids.h(Bjorn)
- Remove of CXL device checking. (Bjorn)
- Rename defines to PCI_DVSEC_CXL_PORT_*. (Bjorn)
- Fix SBR define in commit log. (Bjorn)
- Update comment on dvsec not found. (Dan)
- Check return of dvsec value read for error. (Dan)
- Movecxl_port_dvsec()to an earlier patch. (Dan)
- Addpci_cfg_access_lock()for bridge access. (Dan)
- Changecxl_bus_forcemethod tocxl_bus. (Dan)
- Renamedecoder_hw_mismatch()to__cxl_endpoint_decoder_reset_detected(). (Dan)
- Move CXL register access function tocore/pci.c. (Dan)
- Add kernel taint to decoder reset warning. (Dan)v2:
- Usepci_upstream_bridge()instead ofdev->bus->self. (Lukas)
- Renameis_cxl_device()topci_is_cxl(). (Lukas)
- Return-ENOTTYon error. (Lukas)Patch 1:
MovePCI_DVSEC_VENDOR_ID_CXLtoPCI_VENDOR_ID_CXLinpci_ids.hand adjust related CXL bits.Patch 2:
Add locking of the upstream bridge to keep configuration of the bridge consistent.Patch 3:
Add check to PCIbus_resetpath for CXL device and return with error if "Unmask SBR" bit is set to 0. This allows user to realize that SBR is masked for this CXL device. However, if the user sets the "Unmask SBR" bit via a tool such as setpci, then thebus_resetwill proceed.Patch4:
Add a new PCI reset method "cxl_bus" in order to allow the user an intetional way to perform SBR. The code will set the "Unmask SBR" bit to 1 and proceed withbus_reset. The original value of the bit will be restored after the reset operation.Patch5:
CXL driver change that provides a ->reset_done() callback. It compares the hardware decoder settings with the existing software configuration and emit warning if they differ. The difference indicates that decoders were programmed before the reset and are now cleared after reset. There may be onlined system memory backed by CXL memory device that are now violently ripped away from kernel mapping.Patch series stemmed from this [1] patch. With comments [2] from Bjorn.
[1]: https://lore.kernel.org/linux-cxl/[email protected]/
[2]: https://lore.kernel.org/linux-cxl/20240220203956.GA1502351@bhelgaas/