Enable CXL PCIe Port Protocol Error handling and logging
Subsystems (6)
v14 cover Enable CXL PCIe Port Protocol Error handling and logging
This patchset updates CXL Protocol Error handling for CXL Ports and CXL Endpoints(EP). Previous version of this series can be found here:
https://lore.kernel.org/linux-cxl/[email protected]/
Patches 1-3 introduce pcie_is_cxl(). This includes moving the DVSEC definitions to uapi/linux/pci_regs.h and then updating formating to match existing definitions.
Patches 4-12 improve maintainability by separating CXL RAS logic from AER and CXL PCI logic. 3 new files are created as a result. CXL RCH code in the AER driver is moved to new file, pci/pcie/aer_cxl_rch.c. The CXL driver's RAS related code is moved to cxl/core/ras.c, also a new file. cxl/core/ras_rch.c is introduced to holf the CXL drivers RCH logic. These moves allow the existing #ifdef conditional compilation to be removed.
Patches 13-18 are general changes needed for CXL handling. These replace locks with guard(), add bus type to AER logging, add AER documentation, and replace kernel config CONFIG_PCIEAER_CXL for CONFIG_CXL_RAS.
Patches 19-29 update the CXL Port and Downstream Port discovery. This set of patches moves Endpoint RAS registers to the parent CXL Endpoint Port, updates find_cxl_port_by_uport() to work with EP's, and moves dport RAS setup to EP Port initialization. This subset of patches also moves the AER driver's remaining CXL logic into pci/pcie/aer_cxl_vh.c.
Patches 30-34 introduce AER-CXL kfifo dequeue logic and add CXL Port protocol handlers for CXL Ports and EPs. CXL EPs will also include PCI protocol error handlers. Recent changes in this patch subset include only logging and handling the the error source device instead of recursing children devices.
@Bjorn - The patches requiring PCI approval use a "PCI" prefixed title.
== Testing ==
Below are the testing results while using QEMU. The QEMU testing uses a CXL Root Port, CXL Upstream Switch Port, CXL Downstream Switch Port and CXL Endpoint as given below. I've attached the QEMU startup commandline used. This testing uses protocol error injection at all the devices.
The sub-topology for the QEMU testing is:
---------------------
| CXL RP - 0C:00.0 |
---------------------
|
---------------------
| CXL USP - 0D:00.0 |
---------------------
|
---------------------
| CXL DSP - 0E:00.0 |
---------------------
|
---------------------
| CXL EP - 0F:00.0 |
---------------------root@tbowman-cxl:~# lspci -t -+-[0000:00]- -00.0 | +-01.0 | +-02.0 | +-03.0 | +-1f.0 | +-1f.2 | \-1f.3 \-[0000:0c]---00.0-[0d-0f]----00.0-[0e-0f]----00.0-[0f]----00.0
The topology was created with:
${qemu} -boot menu=on \
-cpu host \
-nographic \
-monitor telnet:127.0.0.1:1234,server,nowait \
-M virt,cxl=on \
-chardev stdio,id=s1,signal=off,mux=on -serial none \
-device isa-serial,chardev=s1 -mon chardev=s1,mode=readline \
-machine q35,cxl=on \
-m 16G,maxmem=24G,slots=8 \
-cpu EPYC-v3 \
-smp 32 \
-accel kvm \
-drive file=${img},format=raw,index=0,media=disk \
-device e1000,netdev=user.0 \
-netdev user,id=user.0,hostfwd=tcp::5555-:22 \
-object memory-backend-file,id=cxl-mem0,share=on,mem-path=/tmp/cxltest.raw,size=256M \
-object memory-backend-file,id=cxl-mem1,share=on,mem-path=/tmp/cxltest1.raw,size=256M \
-object memory-backend-file,id=cxl-mem2,share=on,mem-path=/tmp/cxltest2.raw,size=256M \
-object memory-backend-file,id=cxl-mem3,share=on,mem-path=/tmp/cxltest3.raw,size=256M \
-object memory-backend-file,id=cxl-lsa0,share=on,mem-path=/tmp/lsa0.raw,size=256M \
-object memory-backend-file,id=cxl-lsa1,share=on,mem-path=/tmp/lsa1.raw,size=256M \
-object memory-backend-file,id=cxl-lsa2,share=on,mem-path=/tmp/lsa2.raw,size=256M \
-object memory-backend-file,id=cxl-lsa3,share=on,mem-path=/tmp/lsa3.raw,size=256M \
-device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
-device cxl-rp,port=0,bus=cxl.1,id=root_port0,chassis=0,slot=0 \
-device cxl-upstream,bus=root_port0,id=us0 \
-device cxl-downstream,port=0,bus=us0,id=swport0,chassis=0,slot=4 \
-device cxl-type3,bus=swport0,volatile-memdev=cxl-mem0,lsa=cxl-lsa0,id=cxl-vmem0 \
-device cxl-downstream,port=1,bus=us0,id=swport1,chassis=0,slot=5 \
-device cxl-type3,bus=swport1,volatile-memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-vmem1 \
-device cxl-downstream,port=2,bus=us0,id=swport2,chassis=0,slot=6 \
-device cxl-type3,bus=swport2,volatile-memdev=cxl-mem2,lsa=cxl-lsa2,id=cxl-vmem2 \
-device cxl-downstream,port=3,bus=us0,id=swport3,chassis=0,slot=7 \
-device cxl-type3,bus=swport3,volatile-memdev=cxl-mem3,lsa=cxl-lsa3,id=cxl-vmem3 \
-M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k=== Root Port ===
root@tbowman-cxl:~/aer-inject# ./root-ce-inject.sh pcieport 0000:0c:00.0: aer_inject: Injecting errors 00004000/00000000 into device 0000:0c:00.0 pcieport 0000:0c:00.0: AER: Correctable error message received from 0000:0c:00.0 aer_event: 0000:0c:00.0 CXL Bus Error: severity=Corrected, Corrected Internal Error, TLP Header=Not available pcieport 0000:0c:00.0: CXL Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID) pcieport 0000:0c:00.0: device [8086:7075] error status/mask=00004000/0000a000 pcieport 0000:0c:00.0: [14] CorrIntErr cxl_port_aer_correctable_error: device=0000:0c:00.0 host=pci0000:0c status='CRC Threshold Hit'
root@tbowman-cxl:~/aer-inject# ./root-uce-inject.sh pcieport 0000:0c:00.0: aer_inject: Injecting errors 00000000/00400000 into device 0000:0c:00.0 pcieport 0000:0c:00.0: AER: Uncorrectable (Fatal) error message received from 0000:0c:00.0 aer_event: 0000:0c:00.0 CXL Bus Error: severity=Fatal, Uncorrectable Internal Error, TLP Header=Not available pcieport 0000:0c:00.0: CXL Bus Error: severity=Uncorrectable (Fatal), type=Transaction Layer, (Receiver ID) pcieport 0000:0c:00.0: device [8086:7075] error status/mask=00400000/02000000 pcieport 0000:0c:00.0: [22] UncorrIntErr cxl_port_aer_uncorrectable_error: device=0000:0c:00.0 host=pci0000:0c status: 'Cache Byte Enable Parity Error' first_error: 'Cache Byte Enable Parity Error' Kernel panic - not syncing: CXL cachemem error. CPU: 26 UID: 0 PID: 176 Comm: kworker/26:0 Tainted: G E 6.19.0-rc4-gd2320443c4cf #222 PREEMPT(voluntary) Tainted: [E]=UNSIGNED_MODULE Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 Workqueue: events cxl_proto_err_work_fn [cxl_core] Call Trace: <TASK> dump_stack_lvl+0x26/0xc0 dump_stack+0x10/0x20 vpanic+0x35e/0x3b0 panic+0x57/0x60 cxl_proto_err_work_fn+0x1fc/0x210 [cxl_core] process_one_work+0x22b/0x600 worker_thread+0x195/0x350 kthread+0x119/0x230 ? __pfx_worker_thread+0x10/0x10 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x261/0x2e0 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK> Kernel Offset: disabled ---[ end Kernel panic - not syncing: CXL cachemem error. ]---
=== Upstream Port ===
root@tbowman-cxl:~/aer-inject# ./us-ce-inject.sh pcieport 0000:0c:00.0: aer_inject: Injecting errors 00004000/00000000 into device 0000:0d:00.0 pcieport 0000:0c:00.0: AER: Correctable error message received from 0000:0d:00.0 aer_event: 0000:0d:00.0 CXL Bus Error: severity=Corrected, Corrected Internal Error, TLP Header=Not available pcieport 0000:0d:00.0: CXL Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID) pcieport 0000:0d:00.0: device [19e5:a128] error status/mask=00004000/0000a000 pcieport 0000:0d:00.0: [14] CorrIntErr cxl_port_aer_correctable_error: device=0000:0d:00.0 host=0000:0c:00.0 status='CRC Threshold Hit'
root@tbowman-cxl:~/aer-inject# ./us-uce-inject.sh pcieport 0000:0c:00.0: aer_inject: Injecting errors 00000000/00400000 into device 0000:0d:00.0 pcieport 0000:0c:00.0: AER: Uncorrectable (Fatal) error message received from 0000:0d:00.0 aer_event: 0000:0d:00.0 CXL Bus Error: severity=Fatal, , TLP Header=Not available pcieport 0000:0d:00.0: AER: CXL Bus Error: severity=Uncorrectable (Fatal), type=Inaccessible, (Unregistered Agent ID) cxl_port_aer_uncorrectable_error: device=0000:0f:00.0 host=0000:0e:00.0 status: 'Cache Byte Enable Parity Error' first_error: 'Cache Byte Enable Parity Error' Kernel panic - not syncing: CXL cachemem error. CPU: 26 UID: 0 PID: 247 Comm: irq/24-aerdrv Tainted: G E 6.19.0-rc4-gd2320443c4cf #222 PREEMPT(voluntary) Tainted: [E]=UNSIGNED_MODULE Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x26/0xc0 dump_stack+0x10/0x20 vpanic+0x35e/0x3b0 panic+0x57/0x60 cxl_pci_error_detected+0xbb/0xc0 [cxl_core] report_error_detected+0xda/0x1d0 ? __pfx_report_frozen_detected+0x10/0x10 __pci_walk_bus+0x51/0x80 ? __pfx_report_frozen_detected+0x10/0x10 __pci_walk_bus+0x39/0x80 ? __pfx_report_frozen_detected+0x10/0x10 __pci_walk_bus+0x39/0x80 ? __pfx_report_frozen_detected+0x10/0x10 pci_walk_bus+0x32/0x50 pcie_do_recovery+0x302/0x450 ? __pfx_aer_root_reset+0x10/0x10 aer_isr_one_error_type+0x18e/0x330 aer_isr_one_error+0x124/0x150 aer_isr+0x4c/0x80 irq_thread_fn+0x28/0x70 irq_thread+0x19a/0x2a0 ? __pfx_irq_thread_fn+0x10/0x10 ? __pfx_irq_thread_dtor+0x10/0x10 kthread+0x119/0x230 ? __pfx_irq_thread+0x10/0x10 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x261/0x2e0 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK> Kernel Offset: disabled ---[ end Kernel panic - not syncing: CXL cachemem error. ]---
=== Downstream Port ===
root@tbowman-cxl:~/aer-inject# ./ds-ce-inject.sh pcieport 0000:0c:00.0: aer_inject: Injecting errors 00004000/00000000 into device 0000:0e:00.0 pcieport 0000:0c:00.0: AER: Correctable error message received from 0000:0e:00.0 aer_event: 0000:0e:00.0 CXL Bus Error: severity=Corrected, Corrected Internal Error, TLP Header=Not available pcieport 0000:0e:00.0: CXL Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID) pcieport 0000:0e:00.0: device [19e5:a129] error status/mask=00004000/0000a000 pcieport 0000:0e:00.0: [14] CorrIntErr cxl_port_aer_correctable_error: device=0000:0e:00.0 host=0000:0d:00.0 status='CRC Threshold Hit'
root@tbowman-cxl:~/aer-inject# ./ds-uce-inject.sh pcieport 0000:0c:00.0: aer_inject: Injecting errors 00000000/00400000 into device 0000:0e:00.0 pcieport 0000:0c:00.0: AER: Uncorrectable (Fatal) error message received from 0000:0e:00.0 aer_event: 0000:0e:00.0 CXL Bus Error: severity=Fatal, Uncorrectable Internal Error, TLP Header=Not available pcieport 0000:0e:00.0: CXL Bus Error: severity=Uncorrectable (Fatal), type=Transaction Layer, (Receiver ID) pcieport 0000:0e:00.0: device [19e5:a129] error status/mask=00400000/02000000 pcieport 0000:0e:00.0: [22] UncorrIntErr cxl_port_aer_uncorrectable_error: device=0000:0e:00.0 host=0000:0d:00.0 status: 'Cache Byte Enable Parity Error' first_error: 'Cache Byte Enable Parity Error' Kernel panic - not syncing: CXL cachemem error. CPU: 26 UID: 0 PID: 176 Comm: kworker/26:0 Tainted: G E 6.19.0-rc4-gd2320443c4cf #222 PREEMPT(voluntary) Tainted: [E]=UNSIGNED_MODULE Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 Workqueue: events cxl_proto_err_work_fn [cxl_core] Call Trace: <TASK> dump_stack_lvl+0x26/0xc0 dump_stack+0x10/0x20 vpanic+0x35e/0x3b0 panic+0x57/0x60 cxl_proto_err_work_fn+0x1fc/0x210 [cxl_core] process_one_work+0x22b/0x600 worker_thread+0x195/0x350 kthread+0x119/0x230 ? __pfx_worker_thread+0x10/0x10 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x261/0x2e0 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK> Kernel Offset: disabled ---[ end Kernel panic - not syncing: CXL cachemem error. ]---
=== Endpoint ====
root@tbowman-cxl:~/aer-inject# ./ep-ce-inject.sh pcieport 0000:0c:00.0: aer_inject: Injecting errors 00004000/00000000 into device 0000:0f:00.0 pcieport 0000:0c:00.0: AER: Correctable error message received from 0000:0f:00.0 aer_event: 0000:0f:00.0 CXL Bus Error: severity=Corrected, Corrected Internal Error, TLP Header=Not available cxl_pci 0000:0f:00.0: CXL Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID) cxl_pci 0000:0f:00.0: device [8086:0d93] error status/mask=00004000/0000a000 cxl_pci 0000:0f:00.0: [14] CorrIntErr cxl_port_aer_correctable_error: device=0000:0f:00.0 host=0000:0e:00.0 status='CRC Threshold Hit'
root@tbowman-cxl:~/aer-inject# ./ep-uce-inject.sh pcieport 0000:0c:00.0: aer_inject: Injecting errors 00000000/00400000 into device 0000:0f:00.0 pcieport 0000:0c:00.0: AER: Uncorrectable (Fatal) error message received from 0000:0f:00.0 aer_event: 0000:0f:00.0 CXL Bus Error: severity=Fatal, , TLP Header=Not available cxl_pci 0000:0f:00.0: AER: CXL Bus Error: severity=Uncorrectable (Fatal), type=Inaccessible, (Unregistered Agent ID) cxl_port_aer_uncorrectable_error: device=0000:0f:00.0 host=0000:0e:00.0 status: 'Cache Byte Enable Parity Error' first_error: 'Cache Byte Enable Parity Error' Kernel panic - not syncing: CXL cachemem error. CPU: 26 UID: 0 PID: 247 Comm: irq/24-aerdrv Tainted: G E 6.19.0-rc4-gd2320443c4cf #222 PREEMPT(voluntary) Tainted: [E]=UNSIGNED_MODULE Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x26/0xc0 dump_stack+0x10/0x20 vpanic+0x35e/0x3b0 panic+0x57/0x60 cxl_pci_error_detected+0xbb/0xc0 [cxl_core] report_error_detected+0xda/0x1d0 ? __pfx_report_frozen_detected+0x10/0x10 report_frozen_detected+0x16/0x20 __pci_walk_bus+0x51/0x80 ? __pfx_report_frozen_detected+0x10/0x10 pci_walk_bus+0x32/0x50 pcie_do_recovery+0x302/0x450 ? __pfx_aer_root_reset+0x10/0x10 aer_isr_one_error_type+0x18e/0x330 aer_isr_one_error+0x124/0x150 aer_isr+0x4c/0x80 irq_thread_fn+0x28/0x70 irq_thread+0x19a/0x2a0 ? __pfx_irq_thread_fn+0x10/0x10 ? __pfx_irq_thread_dtor+0x10/0x10 kthread+0x119/0x230 ? __pfx_irq_thread+0x10/0x10 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x261/0x2e0 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK> Kernel Offset: disabled ---[ end Kernel panic - not syncing: CXL cachemem error. ]---
== Changes ==
Changes in v13->v14:
PCI: Move CXL DVSEC definitions into uapi/linux/pci_regs.h
- Add Jonathan's and Dan's review-by
- Update commit title prefix (Bjorn)
- Revert format fix for cxl_sbr_masked() (Jonathan)
- Update 'Compute Express Link' comment block (Jonathan)
- Move PCI_DVSEC_CXL_FLEXBUS definitions to later patch where used (Jonathan)
- Removed stray change (Bjorn)
PCI: Update CXL DVSEC definitions
- New patch. Split from previous patch such that there is now a separate move patch and a format fix patch.
- Formatting update requested (Bjorn)
- Remove PCI_DVSEC_HEADER1_LENGTH_MASK because it duplicates PCI_DVSEC_HEADER1_LEN() (Bjorn)
- Add Dan's review-by
PCI: Introduce pcie_is_cxl()
- Move FLEXBUS_STATUS DVSEC here (Jonathan)
- Remove check for EP and USP (Dan)
- Update commit message (Bjorn)
- Fix writing past 80 columns (Bjorn)
- Add pci_is_pcie() parent bridge check at beginning of function (Bjorn)
PCI: Replace cxl_error_is_native() with pcie_aer_is_native()
- New commit
cxl/pci: Move CXL driver's RCH error handling into core/ras_rch.c
- Add sign-off for Dan and Jonathan
- Revert inadvertent formatting of cxl_dport_map_rch_aer() (Jonathan)
- Remove default value for CXL_RCH_RAS (Dan)
- Remove unnecessary pci.h include in core.h & ras_rch.c (Jonathan)
- Add linux/types.h include in ras_rch.c (Jonathan)
- Change CONFIG_CXL_RCH_RAS -> CONFIG_CXL_RAS (Dan)
PCI/AER: Export pci_aer_unmask_internal_errors
- New commit. Bjorn requested separating out and adding immediatetly before being used. This is called from cxl_rch_enable_rcec() in following patch.
PCI/AER: Update is_internal_error() to be non-static is_aer_internal_error()
- New commit
PCI/AER: Move CXL RCH error handling to aer_cxl_rch.c
- Add review-by and signed-off for Dan
- Commit message fixup (Dan)
- Update commit message with use-case description (Dan, Lukas)
- Make cxl_error_is_native() static (Dan)
- Make is_internal_error() non-static, non-export (Terry) PCI/AER: Use guard() in cxl_rch_handle_error_iter()
- Add review-by for Jonathan, Dave Jiang, Dan WIlliams, and Bjorn
- Remove cleanup.h (Jonathan)
- Reverted comment removal (Bjorn)
- Move this patch after pci/pcie/aer_cxl_rch.c creation (Bjorn) PCI/AER: Replace PCIEAER_CXL symbol with CXL_RAS
- New commit
PCI/AER: Report CXL or PCIe bus type in AER trace logging
- Merged with Dan's commit. Changes are moving bus_type the last parameter in function calls (Dan)
- Removed all DCOs because of changes (Terry)
- Update commit message (Bjorn)
- Add Bjorn's ack-by
PCI/AER: Update struct aer_err_info with kernel-doc formatting
- New commit
cxl/mem: Clarify @host for devm_cxl_add_nvdimm()
- New commit
cxl/port: Remove "enumerate dports" helpers
- New commit
cxl/port: Fix devm resource leaks around with dport management
- New commit
cxl/port: Move dport operations to a driver event
- New commit
cxl/port: Move dport RAS reporting to a port resource
- New commit
cxl: Map CXL Endpoint Port and CXL Switch Port RAS registers
- Correct message spelling (Terry)
cxl/port: Move endpoint component register management to cxl_port
- Correct message spelling (Terry)
cxl/port: Map Port component registers before switchport init
- Updates to use cxl_port_setup_regs() (Dan)
cxl: Change CXL handlers to use guard() instead of scoped_guard()
- Add reviewed-by for Jonathan and Dave Jiang
PCI/ERR: Introduce PCI_ERS_RESULT_PANIC
- Add review-by for Dan
- Update Title prefix (Bjorn)
- Removed merge_result. Only logging error for device reporting the error (Dan)
- Remove PCI_ERS_RESULT_PANIC paragraph in pci-error-recovery.rst (Bjorn) PCI/AER: Move AER driver's CXL VH handling to pcie/aer_cxl_vh.c
- Replaced workqueue_types.h include with 'struct work_struct' predeclaration (Bjorn)
- Update error message (Bjorn)
- Reordered 'struct cxl_proto_err_work_data' (Bjorn)
- Remove export of cxl_error_is_native() here (Bjorn)
cxl/port: Unify endpoint and switch port lookup
- New patch
PCI/AER: Dequeue forwarded CXL error
- Update commit title's prefix (Bjorn)
- Add pdev ref get in AER driver before enqueue and add pdev ref put in CXL driver after dequeue and handling (Dan)
- Removed handling to simplify patch context (Terry)
PCI: Introduce CXL Port protocol error handlers
- Add Dave Jiang's review-by
- Update commit message & headline (Bjorn)
- Refactor cxl_port_error_detected()/cxl_port_cor_error_detected() to one line (Jonathan)
- Remove cxl_walk_port(). Only log the erroring device. No port walking. (Dan)
- Remove cxl_pci_drv_bound(). Check for 'is_cxl' parent port is sufficient (Dan)
- Remove device_lock_if()
- Combine CE and UCE here (Terry)
cxl: Update Endpoint uncorrectable protocol error handling
- Update commit headline (Bjorn)
- Rename pci_error_detected()/pci_cor_error_detected() -> cxl_pci_error_detected/cxl_pci_cor_error_detected() (Jonathan)
- Remove now-invalid comment in cxl_error_detected() (Jonathan)
- Split into separate patches for UCE and CE (Terry)
cxl: Update Endpoint correctable protocol error handling
- New commit
- Change cxl_cor_error_detected() parameter to &pdev->dev device from memdev device. (Terry)
cxl: Enable CXL protocol errors during CXL Port probe
- Update commit title's prefix (Bjorn)
Changes in v12->v13:
CXL/PCI: Move CXL DVSEC definitions into uapi/linux/pci_regs.h
- Add Dave Jiang's reviewed-by
- Remove changes to existing PCI_DVSEC_CXL_PORT* defines. Update commit message. (Jonathan)
PCI/CXL: Introduce pcie_is_cxl()
- Add Ben's "reviewed-by"
cxl/pci: Remove unnecessary CXL Endpoint handling helper functions
- None
cxl/pci: Remove unnecessary CXL RCH handling helper functions
- None
cxl: Remove CXL VH handling in CONFIG_PCIEAER_CXL conditional blocks from core
- None
cxl: Move CXL driver's RCH error handling into core/ras_rch.c
- None
CXL/AER: Replace device_lock() in cxl_rch_handle_error_iter() with guard() lock
- New patch
CXL/AER: Move AER drivers RCH error handling into pcie/aer_cxl_rch.c
- Add forward declararation of 'struct aer_err_info' in pci/pci.h (Terry)
- Changed copyright date from 2025 to 2023 (Jonathan)
- Add David Jiang's, Jonathan's, and Ben's review-by
- Readd 'struct aer_err_info' (Bot)
PCI/AER: Report CXL or PCIe bus error type in trace logging
- Remove duplicated aer_err_info inline comments. Is already in the kernel-doc header (Ben)
cxl/pci: Update RAS handler interfaces to also support CXL Ports
- None
cxl/pci: Log message if RAS registers are unmapped
- Added Bens review-by
cxl/pci: Unify CXL trace logging for CXL Endpoints and CXL Ports
- Added Dave Jiang's review-by
cxl/pci: Update cxl_handle_cor_ras() to return early if no RAS errors
- Add Ben's review-by
cxl/pci: Map CXL Endpoint Port and CXL Switch Port RAS registers
- Change as result of dport delay fix. No longer need switchport and endport approach. Refactor. (Terry)
CXL/PCI: Introduce PCI_ERS_RESULT_PANIC
- Add Dave Jiang's, Jonathan's, Ben's review-by
- Typo fix (Ben)
CXL/AER: Introduce pcie/aer_cxl_vh.c in AER driver for forwarding CXL errors
- Add Dave Jiang's review-by
- Update error message (Ben)
cxl: Introduce cxl_pci_drv_bound() to check for bound driver
- Add Dave Jiang's review-by.
cxl: Change CXL handlers to use guard() instead of scoped_guard()
- New patch
cxl/pci: Introduce CXL protocol error handlers for endpoints
- Updated all the implemetnation and commit message. (Terry)
- Refactored cxl_cor_error_detected()/cxl_error_detected() to remove pdev (Dave Jiang)
CXL/PCI: Introduce CXL Port protocol error handlers
- Move get_pci_cxl_host_dev() and cxl_handle_proto_error() to Dequeue patch (Terry)
- Remove EP case in cxl_get_ras_base(), not used. (Terry)
- Remove check for dport->dport_dev (Dave)
- Remove whitespace (Terry)
PCI/AER: Dequeue forwarded CXL error
- Rewrite cxl_handle_proto_error() and cxl_proto_err_work_fn() (Terry)
- Rename get_cxl_host dev() to be get_cxl_port() (Terry)
- Remove exporting of unused function, pci_aer_clear_fatal_status() (Dave Jiang)
- Change pr_err() calls to ratelimited. (Terry)
- Update commit message. (Terry)
- Remove namespace qualifier from pcie_clear_device_status()
export (Dave Jiang)
- Move locks into cxl_proto_err_work_fn() (Dave)
- Update log messages in cxl_forward_error() (Ben)
CXL/PCI: Export and rename merge_result() to pci_ers_merge_result()
- Renamed pci_ers_merge_result() to pcie_ers_merge_result(). pci_ers_merge_result() is already used in eeh driver. (Bot)
CXL/PCI: Introduce CXL uncorrectable protocol error recovery
- Rewrite report_error_detected() and cxl_walk_port (Terry)
- Add guard() before calling cxl_pci_drv_bound() (Dave Jiang)
- Add guard() calls for EP (cxlds->cxlmd->dev & pdev->dev) and ports (pdev->dev & parent cxl_port) in cxl_report_error_detected() and cxl_handle_proto_error() (Terry)
- Remove unnecessary check for endpoint port. (Dave Jiang)
- Remove check for RCIEP EP in cxl_report_error_detected() (Terry)
CXL/PCI: Enable CXL protocol errors during CXL Port probe
- Add dev and dev_is_pci() NULL checks in cxl_unmask_proto_interrupts() (Terry)
- Add Dave Jiang's and Ben's review-by
CXL/PCI: Disable CXL protocol error interrupts during CXL Port cleanup
- Added dev and dev_is_pci() checks in cxl_mask_proto_interrupts() (Terry)
Changes in v11 -> v12:
cxl/pci: Remove unnecessary CXL Endpoint handling helper functions
- Added Dave Jiang's review by
- Moved to front of series
cxl/pci: Remove unnecessary CXL RCH handling helper functions
- Add reviewed-by for Alejandro & Dave Jiang
- Moved to front of series
cxl: Remove ifdef blocks of CONFIG_PCIEAER_CXL from core/pci.c
- Update CONFIG_CXL_RAS in CXL Kconfig to have CXL_PCI dependency (Terry)
CXL/AER: Remove CONFIG_PCIEAER_CXL and replace with CONFIG_CXL_RAS
- Added review-by for Sathyanarayanan
- Changed Kconfig dependency from PCIEAER_CXL to PCIEAER. Moved
this backwards into this patch.
cxl: Move CXL driver RCH error handling into CONFIG_CXL_RCH_RAS conditio
- Moved CXL_RCH_RAS Kconfig definition here from following commit
CXL/AER: Introduce aer_cxl_rch.c into AER driver for handling CXL RCH errors
- Rename drivers/pci/pcie/cxl_rch.c to drivers/pci/pcie/aer_cxl_rch.c (Lukas)
- Removed forward declararation of 'struct aer_err_info' in pci/pci.h (Terry)
CXL/PCI: Move CXL DVSEC definitions into uapi/linux/pci_regs.h
- Change formatting to be same as existing definitions
- Change GENMASK() -> __GENMASK() and BIT() to _BITUL()
PCI/CXL: Introduce pcie_is_cxl()
- Add review-by for Alejandro
- Add comment in set_pcie_cxl() explaining why updating parent status.
PCI/AER: Report CXL or PCIe bus error type in trace logging
- Change aer_err_info::is_cxl to be bool a bitfield. Update structure padding. (Lukas)
- Add kernel-doc for 'struct aer_err_info' (Lukas)
cxl/pci: Unify CXL trace logging for CXL Endpoints and CXL Ports
- Correct parameters to call trace_cxl_aer_correctable_error() (Shiju)
- Add reviewed-by for Jonathan and Shiju
cxl/pci: Map CXL Endpoint Port and CXL Switch Port RAS registers
- Add check for dport_parent->rch before calling cxl_dport_init_ras_reporting().
- RCH dports are initialized from cxl_dport_init_ras_reporting cxl_mem_probe().
CXL/PCI: Introduce PCI_ERS_RESULT_PANIC
- Documentation requested by (Lukas)
CXL/AER: Introduce aer_cxl_vh.c in AER driver for forwarding CXL errors
- Rename drivers/pci/pcie/cxl_aer.c to drivers/pci/pcie/aer_cxl_vh.c (Lukas)
cxl: Introduce cxl_pci_drv_bound() to check for bound driver
- New patch
PCI/AER: Dequeue forwarded CXL error
- Add guard for CE case in cxl_handle_proto_error() (Dave)
- Updated commit message (Terry)
CXL/PCI: Introduce CXL Port protocol error handlers
- Add call to cxl_pci_drv_bound() in cxl_handle_proto_error() and pci_to_cxl_dev() (Lukas)
- Change cxl_error_detected() -> cxl_cor_error_detected() (Terry)
- Remove NULL variable assignments (Jonathan)
- Replace bus_find_device() with find_cxl_port_by_uport() for upstream port searches. (Dave)
CXL/PCI: Export and rename merge_result() to pci_ers_merge_result()
- Remove static inline pci_ers_merge_result() definition for !CONFIG_PCIEAER. Is not needed. (Lukas)
CXL/PCI: Introduce CXL uncorrectable protocol error recovery
- Clean up port discovery in cxl_do_recovery() (Dave)
- Add PCI_EXP_TYPE_RC_END to type check in cxl_report_error_detected()
Changes in v10 -> v11:
cxl: Remove ifdef blocks of CONFIG_PCIEAER_CXL from core/pci.c
- New patch
CXL/AER: Remove CONFIG_PCIEAER_CXL and replace with CONFIG_CXL_RAS
- New patch
cxl/pci: Remove unnecessary CXL RCH handling helper functions
- New patch
cxl: Move CXL driver RCH error handling into CONFIG_CXL_RCH_RAS conditional block
- New patch
CXL/AER: Introduce rch_aer.c into AER driver for handling CXL RCH errors
- Remove changes in code-split and move to earlier, new patch
- Add #include <linux/bitfield.h> to cxl_ras.c
- Move cxl_rch_handle_error() & cxl_rch_enable_rcec() declarations from pci.h to aer.h, more localized.
- Introduce CONFIG_CXL_RCH_RAS, includes Makefile changes, ras.c ifdef changes CXL/PCI: Move CXL DVSEC definitions into uapi/linux/pci_regs.h
- New patch
PCI/CXL: Introduce pcie_is_cxl()
- Amended set_pcie_cxl() to check for Upstream Port's and EP's parent downstream port by calling set_pcie_cxl(). (Dan)
- Retitle patch: 'Add' -> 'Introduce'
- Add check for CXL.mem and CXL.cache (Alejandro, Dan)
PCI/AER: Report CXL or PCIe bus error type in trace logging
- Remove duplicate call to trace_aer_event() (Shiju)
- Added Dan William's and Dave Jiang's reviewed-by
CXL/AER: Update PCI class code check to use FIELD_GET()
- Add #include <linux/bitfield.h> to cxl_ras.c (Terry)
- Removed line wrapping at "(CXL 3.2, 8.1.12.1)". (Jonathan)
cxl/pci: Log message if RAS registers are unmapped
- Added Dave Jiang's review-by (Terry)
cxl/pci: Unify CXL trace logging for CXL Endpoints and CXL Ports
- Updated CE and UCE trace routines to maintian consistent TP_Struct ABI and unchanged TP_printk() logging. (Shiju, Alison)
cxl/pci: Update cxl_handle_cor_ras() to return early if no RAS errors
- Added Dave Jiang and Jonathan Cameron's review-by
- Changes moved to core/ras.c
cxl/pci: Map CXL Endpoint Port and CXL Switch Port RAS registers
- Use local pointer for readability in cxl_switch_port_init_ras() (Jonathan Cameron)
- Rename port to be ep in cxl_endpoint_port_init_ras() (Dave Jiang)
- Rename dport to be parent_dport in cxl_endpoint_port_init_ras()
and cxl_switch_port_init_ras() (Dave Jiang)
- Port helper changes were in cxl/port.c, now in core/ras.c (Dave Jiang) cxl/pci: Introduce CXL Endpoint protocol error handlers
- cxl_error_detected() - Change handlers' scoped_guard() to guard() (Jonathan)
- cxl_error_detected() - Remove extra line (Shiju)
- Changes moved to core/ras.c (Terry)
- cxl_error_detected(), remove 'ue' and return with function call. (Jonathan)
- Remove extra space in documentation for PCI_ERS_RESULT_PANIC definition
- Move #include "pci.h from cxl.h to core.h (Terry)
- Remove unnecessary includes of cxl.h and core.h in mem.c (Terry) CXL/AER: Introduce cxl_aer.c into AER driver for forwarding CXL errors
- Move RCH implementation to cxl_rch.c and RCH declarations to pci/pci.h. (Terry)
- Introduce 'struct cxl_proto_err_kfifo' containing semaphore, fifo, and work struct. (Dan)
- Remove embedded struct from cxl_proto_err_work (Dan)
- Make 'struct work_struct *cxl_proto_err_work' definition static (Jonathan)
- Add check for NULL cxl_proto_err_kfifo to determine if CXL driver is not registered for workqueue. (Dan)
PCI/AER: Dequeue forwarded CXL error
- Reword patch commit message to remove RCiEP details (Jonathan)
- Add #include <linux/bitfield.h> (Terry)
- is_cxl_rcd() - Fix short comment message wrap (Jonathan)
- is_cxl_rcd() - Combine return calls into 1 (Jonathan)
- cxl_handle_proto_error() - Move comment earlier (Jonathan)
- Usse FIELD_GET() in discovering class code (Jonathan)
- Remove BDF from cxl_proto_err_work_data. Use 'struct pci_dev *' (Dan) CXL/PCI: Introduce CXL Port protocol error handlers
- Removed check for PCI_EXP_TYPE_RC_END in cxl_report_error_detected() (Terry)
- Update is_cxl_error() to check for acceptable PCI EP and port types CXL/PCI: Export and rename merge_result() to pci_ers_merge_result()
- pci_ers_merge_result() - Change export to non-namespace and rename to be pci_ers_merge_result() (Jonathan)
- Move pci_ers_merge_result() definition to pci.h. Needs pci_ers_result (Terry) CXL/PCI: Introduce CXL uncorrectable protocol error recovery
- pci_ers_merge_results() - Move to earlier patch
CXL/PCI: Disable CXL protocol error interrupts during CXL Port cleanup
- Remove guard() in cxl_mask_proto_interrupts(). Observed device lockup/block during testing. (Terry)
Changes in v9 -> v10:
- Add drivers/pci/pcie/cxl_aer.c
- Add drivers/cxl/core/native_ras.c
- Change cxl_register_prot_err_work()/cxl_unregister_prot_err_work to return void
- Check for pcie_ports_native in cxl_do_recovery()
- Remove debug logging in cxl_do_recovery()
- Update PCI_ERS_RESULT_PANIC definition to indicate is CXL specific
- Revert trace logging changes: name,parent -> memdev,host.
- Use FIELD_GET() to check for EP class code (cxl_aer.c & native_ras.c).
- Change _prot_ to _proto_ everywhere
- cxl_rch_handle_error_iter(), check if driver is cxl_pci_driver
- Remove cxl_create_prot_error_info(). Move logic into forward_cxl_error()
- Remove sbdf_to_pci() and move logic into cxl_handle_proto_error()
- Simplify/refactor get_pci_cxl_host_dev()
- Simplify/refactor cxl_get_ras_base()
- Move patch 'Remove unnecessary CXL Endpoint handling helper functions' to front
- Update description for 'CXL/PCI: Introduce CXL Port protocol error handlers' with why state is not used to determine handling
- Introduce cxl_pci_drv_bound() and call from cxl_rch_handle_error_iter()
Changes in v8 -> v9:
- Updated reference counting to use pci_get_device()/pci_put_device() in cxl_disable_prot_errors()/cxl_enable_prot_errors
- Refactored cxl_create_prot_err_info() to fix reference counting
- Removed 'struct cxl_port' driver changes for error handler. Instead check for CXL device type (EP or Port device) and call handler
- Make pcie_is_cxl() static inline in include/linux/linux.h
- Remove NULL check in create_prot_err_info()
- Change success return in cxl_ras_init() to use hardcoded 0
- Changed 'struct work_struct cxl_prot_err_work' declaration to static
- Change to use rate limited log with dev anchor in forward_cxl_error()
- Refactored forward-cxl_error() to remove severity auto variable
- Changed pci_aer_clear_nonfatal_status() to be static inline for !(CONFIG_PCIEAER)
- Renamed merge_result() to be cxl_merge_result()
- Removed 'ue' condition in cxl_error_detected()
- Updated 2nd parameter in call to __cxl_handle_cor_ras()/__cxl_handle_ras() in unify patch
- Added log message for failure while assigning interrupt disable callback
- Updated pci_aer_mask_internal_errors() to use pci_clear_and_set_config_dword()
- Simplified patch titles for clarity
- Moved CXL error interrupt disabling into cxl/core/port.c with CXL Port teardown
- Updated 'struct cxl_port_err_info' to only contain sbdf and severity Removed everything else.
- Added pdev and CXL device get_device()/put_device() before calling handlers
Changes in v7 -> v8:
[Dan] Use kfifo. Move handling to CXL driver. AER forwards error to CXL driver
[Dan] Add device reference incrementors where needed throughout [Dan] Initiate CXL Port RAS init from Switch Port and Endpoint Port init [Dan] Combine CXL Port and CXL Endpoint trace routine
[Dan] Introduce aer_info::is_cxl. Use to indicate CXL or PCI errors [Jonathan] Add serial number for all devices in trace [DaveJ] Move find_cxl_port() change into patch using it [Terry] Move CXL Port RAS init into cxl/port.c
[Terry] Moved kfifo functions into cxl/core/ras.c
Changes in v6 -> v7:
[Terry] Move updated trace routine call to later patch. Was causing build error.
Changes in v5 -> v6:
[Ira] Move pcie_is_cxl(dev) define to a inline function
[Ira] Update returning value from pcie_is_cxl_port() to bool w/o cast
[Ira] Change cxl_report_error_detected() cleanup to return correct bool
[Ira] Introduce and use PCI_ERS_RESULT_PANIC
[Ira] Reuse comment for PCIe and CXL recovery paths
[Jonathan] Add type check in for cxl_handle_cor_ras() and cxl_handle_ras() [Jonathan] cxl_uport/dport_init_ras_reporting(), added a mutex.
[Jonathan] Add logging example to patches updating trace output
[Jonathan] Make parameter 'const' to eliminate for cast in match_uport() [Jonathan] Use __free() in cxl_pci_port_ras()
[Terry] Add patch to log the PCIe SBDF along with CXL device name
[Terry] Add patch to handle CXL endpoint and RCH DP errors as CXL errors [Terry] Remove patch w USP UCE fatal support @ aer_get_device_error_info() [Terry] Rebase to cxl/next commit 5585e342e8d3 ("cxl/memdev: Remove unused partition values") [Gregory] Pre-initialize pointer to NULL in cxl_pci_port_ras()
[Gregory] Move AER driver bus name detection to a static function
Changes in v4 -> v5:
[Alejandro] Refactor cxl_walk_bridge to simplify 'status' variable usage [Alejandro] Add WARN_ONCE() in __cxl_handle_ras() and cxl_handle_cor_ras() [Ming] Remove unnecessary NULL check in cxl_pci_port_ras() [Terry] Add failure check for call to to_cxl_port() in cxl_pci_port_ras() [Ming] Use port->dev for call to devm_add_action_or_reset() in cxl_dport_init_ras_reporting() and cxl_uport_init_ras_reporting() [Jonathan] Use get_device()/put_device() to prevent race condition in cxl_clear_port_error_handlers() and cxl_clear_port_error_handlers() [Terry] Commit message cleanup. Capitalize keywords from CXL and PCI specifications
Changes in v3 -> v4:
[Lukas] Capitalize PCIe and CXL device names as in specifications [Lukas] Move call to pcie_is_cxl() into cxl_port_devsec()
[Lukas] Correct namespace spelling
[Lukas] Removed export from pcie_is_cxl_port()
[Lukas] Simplify 'if' blocks in cxl_handle_error()
[Lukas] Change panic message to remove redundant 'panic' text
[Ming] Update to call cxl_dport_init_ras_reporting() in RCH case [lkp@intel] 'host' parameter is already removed. Remove parameter description too. [Terry] Added field description for cxl_err_handlers in pci.h comment block
Changes in v1 -> v2:
[Jonathan] Remove extra NULL check and cleanup in cxl_pci_port_ras() [Jonathan] Update description to DSP map patch description
[Jonathan] Update cxl_pci_port_ras() to check for NULL port [Jonathan] Dont call handler before handler port changes are present (patch order) [Bjorn] Fix linebreak in cover sheet URL
[Bjorn] Remove timestamps from test logs in cover sheet
[Bjorn] Retitle AER commits to use "PCI/AER:"
[Bjorn] Retitle patch#3 to use renaming instead of refactoring [Bjorn] Fix base commit-id on cover sheet
[Bjorn] Add VH spec reference/citation
[Terry] Removed last 2 patches to enable internal errors. Is not needed because internal errors are enabled in AER driver.
[Dan] Create cxl_do_recovery() and pci_driver::cxl_err_handlers. [Dan] Use kernel panic in CXL recovery
[Dan] cxl_port_hndlrs -> cxl_port_error_handlers
Dan Williams (8):
PCI/AER: Replace PCIEAER_CXL symbol with CXL_RAS cxl/mem: Clarify @host for devm_cxl_add_nvdimm() cxl/port: Remove "enumerate dports" helpers
cxl/port: Fix devm resource leaks around with dport management cxl/port: Move dport operations to a driver event cxl/port: Move dport RAS reporting to a port resource cxl/port: Move endpoint component register management to cxl_port cxl/port: Unify endpoint and switch port lookup
Dave Jiang (1):
cxl/pci: Remove CXL VH handling in CONFIG_PCIEAER_CXL conditional
blocks from core/pci.c
Terry Bowman (25):
PCI: Move CXL DVSEC definitions into uapi/linux/pci_regs.h
PCI: Update CXL DVSEC definitions
PCI: Introduce pcie_is_cxl()
cxl/pci: Remove unnecessary CXL Endpoint handling helper functions cxl/pci: Remove unnecessary CXL RCH handling helper functions
PCI: Replace cxl_error_is_native() with pcie_aer_is_native() cxl/pci: Move CXL driver's RCH error handling into core/ras_rch.c PCI/AER: Export pci_aer_unmask_internal_errors() PCI/AER: Update is_internal_error() to be non-static
is_aer_internal_error()
PCI/AER: Move CXL RCH error handling to aer_cxl_rch.c PCI/AER: Use guard() in cxl_rch_handle_error_iter() PCI/AER: Report CXL or PCIe bus type in AER trace logging PCI/AER: Update struct aer_err_info with kernel-doc formatting
cxl: Update RAS handler interfaces to also support CXL Ports
cxl: Update CXL Endpoint tracing
cxl: Map CXL Endpoint Port and CXL Switch Port RAS registers cxl/port: Map Port component registers before switchport init
cxl: Change CXL handlers to use guard() instead of scoped_guard() PCI/ERR: Introduce PCI_ERS_RESULT_PANIC
PCI/AER: Move AER driver's CXL VH handling to pcie/aer_cxl_vh.c PCI/AER: Dequeue forwarded CXL error
PCI: Introduce CXL Port protocol error handlers
cxl: Update Endpoint uncorrectable protocol error handling
cxl: Update Endpoint correctable protocol error handling
cxl: Enable CXL protocol errors during CXL Port probe
Documentation/PCI/pci-error-recovery.rst | 2 + drivers/cxl/Kconfig | 4 + drivers/cxl/acpi.c | 11 +- drivers/cxl/core/Makefile | 3 +- drivers/cxl/core/core.h | 26 ++ drivers/cxl/core/hdm.c | 6 +- drivers/cxl/core/pci.c | 402 +++------------------- drivers/cxl/core/pmem.c | 13 +- drivers/cxl/core/port.c | 245 +++++++------- drivers/cxl/core/ras.c | 405 ++++++++++++++++++++++- drivers/cxl/core/ras_rch.c | 121 +++++++ drivers/cxl/core/regs.c | 14 +- drivers/cxl/core/trace.h | 25 +- drivers/cxl/cxl.h | 53 +-- drivers/cxl/cxlmem.h | 4 +- drivers/cxl/cxlpci.h | 93 +++--- drivers/cxl/mem.c | 4 +- drivers/cxl/pci.c | 73 +--- drivers/cxl/port.c | 187 ++++++++++- drivers/pci/pci.c | 1 + drivers/pci/pci.h | 39 ++- drivers/pci/pcie/Kconfig | 9 - drivers/pci/pcie/Makefile | 2 + drivers/pci/pcie/aer.c | 144 ++------ drivers/pci/pcie/aer_cxl_rch.c | 104 ++++++ drivers/pci/pcie/aer_cxl_vh.c | 82 +++++ drivers/pci/pcie/portdrv.h | 16 + drivers/pci/probe.c | 31 ++ include/linux/aer.h | 26 ++ include/linux/pci.h | 11 + include/ras/ras_event.h | 12 +- include/uapi/linux/pci_regs.h | 64 +++- tools/testing/cxl/Kbuild | 11 +- tools/testing/cxl/cxl_core_exports.c | 22 -- tools/testing/cxl/exports.h | 13 - tools/testing/cxl/test/cxl.c | 6 +- tools/testing/cxl/test/mock.c | 54 +-- tools/testing/cxl/test/mock.h | 4 +- 38 files changed, 1435 insertions(+), 907 deletions(-) create mode 100644 drivers/cxl/core/ras_rch.c create mode 100644 drivers/pci/pcie/aer_cxl_rch.c create mode 100644 drivers/pci/pcie/aer_cxl_vh.c delete mode 100644 tools/testing/cxl/exports.h
base-commit: 0f61b1860cc3f52aef9036d7235ed1f017632193
Signature · 1 line
2.34.1
PATCH v14 1/34 PCI: Move CXL DVSEC definitions into uapi/linux/pci_regs.h
The CXL DVSECs are currently defined in cxl/core/cxlpci.h. These are not accessible to other subsystems. Move these to uapi/linux/pci_regs.h.
The CXL DVSEC definitions will be renamed and reformatted to fit better with existing defines.
----
Changes in v13->v14:
- Add Jonathan's and Dan's review-by
- Update commit title prefix (Bjorn)
- Revert format fix for cxl_sbr_masked() (Jonathan)
- Update 'Compute Express Link' comment block (Jonathan)
- Move PCI_DVSEC_CXL_FLEXBUS definitions to later patch where used (Jonathan)
- Removed stray change (Bjorn)
Changes in v12->v13:
- Add Dave Jiang's reviewed-by
- Remove changes to existing PCI_DVSEC_CXL_PORT* defines. Update commit message. (Jonathan)
Changes in v11 -> v12:
- Change formatting to be same as existing definitions
- Change GENMASK() -> __GENMASK() and BIT() to _BITUL()
Changes in v10 -> v11:
- New commit
--- drivers/cxl/cxlpci.h | 53 ----------------------------- include/uapi/linux/pci_regs.h | 64 ++++++++++++++++++++++++++++++++--- 2 files changed, 59 insertions(+), 58 deletions(-)
drivers/cxl/cxlpci.h +0 -53
@@ -7,59 +7,6 @@#define CXL_MEMORY_PROGIF 0x10 -/* -* See section 8.1 Configuration Space Registers in the CXL 2.0 -* Specification. Names are taken straight from the specification with "CXL" and -* "DVSEC" redundancies removed. When obvious, abbreviations may be used. -*/ -#define PCI_DVSEC_HEADER1_LENGTH_MASK GENMASK(31, 20) - -/* CXL 2.0 8.1.3: PCIe DVSEC for CXL Device */ -#define CXL_DVSEC_PCIE_DEVICE 0 -#define CXL_DVSEC_CAP_OFFSET 0xA -#define CXL_DVSEC_MEM_CAPABLE BIT(2) -#define CXL_DVSEC_HDM_COUNT_MASK GENMASK(5, 4) -#define CXL_DVSEC_CTRL_OFFSET 0xC -#define CXL_DVSEC_MEM_ENABLE BIT(2) -#define CXL_DVSEC_RANGE_SIZE_HIGH (i) (0x18 + (i * 0x10)) -#define CXL_DVSEC_RANGE_SIZE_LOW (i) (0x1C + (i * 0x10)) -#define CXL_DVSEC_MEM_INFO_VALID BIT(0) -#define CXL_DVSEC_MEM_ACTIVE BIT(1) -#define CXL_DVSEC_MEM_SIZE_LOW_MASK GENMASK(31, 28) -#define CXL_DVSEC_RANGE_BASE_HIGH (i) (0x20 + (i * 0x10)) -#define CXL_DVSEC_RANGE_BASE_LOW (i) (0x24 + (i * 0x10)) -#define CXL_DVSEC_MEM_BASE_LOW_MASK GENMASK(31, 28) - -#define CXL_DVSEC_RANGE_MAX 2 - -/* CXL 2.0 8.1.4: Non-CXL Function Map DVSEC */ -#define CXL_DVSEC_FUNCTION_MAP 2 - -/* CXL 2.0 8.1.5: CXL 2.0 Extensions DVSEC for Ports */ -#define CXL_DVSEC_PORT_EXTENSIONS 3 - -/* CXL 2.0 8.1.6: GPF DVSEC for CXL Port */ -#define CXL_DVSEC_PORT_GPF 4 -#define CXL_DVSEC_PORT_GPF_PHASE_1_CONTROL_OFFSET 0x0C -#define CXL_DVSEC_PORT_GPF_PHASE_1_TMO_BASE_MASK GENMASK(3, 0) -#define CXL_DVSEC_PORT_GPF_PHASE_1_TMO_SCALE_MASK GENMASK(11, 8) -#define CXL_DVSEC_PORT_GPF_PHASE_2_CONTROL_OFFSET 0xE -#define CXL_DVSEC_PORT_GPF_PHASE_2_TMO_BASE_MASK GENMASK(3, 0) -#define CXL_DVSEC_PORT_GPF_PHASE_2_TMO_SCALE_MASK GENMASK(11, 8) - -/* CXL 2.0 8.1.7: GPF DVSEC for CXL Device */ -#define CXL_DVSEC_DEVICE_GPF 5 - -/* CXL 2.0 8.1.8: PCIe DVSEC for Flex Bus Port */ -#define CXL_DVSEC_PCIE_FLEXBUS_PORT 7 - -/* CXL 2.0 8.1.9: Register Locator DVSEC */ -#define CXL_DVSEC_REG_LOCATOR 8 -#define CXL_DVSEC_REG_LOCATOR_BLOCK1_OFFSET 0xC -#define CXL_DVSEC_REG_LOCATOR_BIR_MASK GENMASK(2, 0) -#define CXL_DVSEC_REG_LOCATOR_BLOCK_ID_MASK GENMASK(15, 8) -#define CXL_DVSEC_REG_LOCATOR_BLOCK_OFF_LOW_MASK GENMASK(31, 16) -/* * NOTE: Currently all the functions which are enabled for CXL require their * vectors to be in the first 16. Use this as the default max.
include/uapi/linux/pci_regs.h +59 -5
74 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH v14 01/34] PCI: Move CXL DVSEC definitions into uapi/linux/pci_regs.h
Quoted 2 lines · Terry Bowman
The CXL DVSECs are currently defined in
cxl/core/cxlpci.h. These are not accessible to other subsystems. Move these touapi/linux/pci_regs.h.
I'm looking a little bit askance at adding things to uapi/linux/pci_regs.h and then renaming them. I know it's OCD to worry about that momentary blip, but changes in uapi potentially break userspace.
Maybe we could rename them first, then move them to pci_regs.h?
Either way:
Quoted 182 lines
The CXL DVSEC definitions will be renamed and reformatted to fit better with existing defines.
Signed-off-by Terry Bowman <[email protected]>Reviewed-by Dave Jiang <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Dan Williams <[email protected]>----
Changes in
v13->v14:
- Add Jonathan's and Dan's review-by
- Update commit title prefix (Bjorn)
- Revert format fix forcxl_sbr_masked()(Jonathan)
- Update 'Compute Express Link' comment block (Jonathan)
- MovePCI_DVSEC_CXL_FLEXBUSdefinitions to later patch where used (Jonathan)
- Removed stray change (Bjorn)Changes in
v12->v13:
- Add Dave Jiang's reviewed-by
- Remove changes to existing PCI_DVSEC_CXL_PORT* defines. Update commit message. (Jonathan)Changes in v11 -> v12:
- Change formatting to be same as existing definitions
- ChangeGENMASK()->__GENMASK()andBIT()to_BITUL()Changes in v10 -> v11:
- New commit--- drivers/cxl/cxlpci.h | 53 ----------------------------- include/uapi/linux/pci_regs.h | 64 ++++++++++++++++++++++++++++++++--- 2 files changed, 59 insertions(+), 58 deletions(-)drivers/cxl/cxlpci.h +0
-53@@ -7,59 +7,6 @@#define CXL_MEMORY_PROGIF 0x10 -/* -* See section 8.1 Configuration Space Registers in the CXL 2.0 -* Specification. Names are taken straight from the specification with "CXL" and -* "DVSEC" redundancies removed. When obvious, abbreviations may be used. -*/ -#define PCI_DVSEC_HEADER1_LENGTH_MASK GENMASK(31, 20) - -/* CXL 2.0 8.1.3: PCIe DVSEC for CXL Device */ -#define CXL_DVSEC_PCIE_DEVICE 0 -#define CXL_DVSEC_CAP_OFFSET 0xA -#define CXL_DVSEC_MEM_CAPABLE BIT(2) -#define CXL_DVSEC_HDM_COUNT_MASK GENMASK(5, 4) -#define CXL_DVSEC_CTRL_OFFSET 0xC -#define CXL_DVSEC_MEM_ENABLE BIT(2) -#define CXL_DVSEC_RANGE_SIZE_HIGH (i) (0x18 + (i * 0x10)) -#define CXL_DVSEC_RANGE_SIZE_LOW (i) (0x1C + (i * 0x10)) -#define CXL_DVSEC_MEM_INFO_VALID BIT(0) -#define CXL_DVSEC_MEM_ACTIVE BIT(1) -#define CXL_DVSEC_MEM_SIZE_LOW_MASK GENMASK(31, 28) -#define CXL_DVSEC_RANGE_BASE_HIGH (i) (0x20 + (i * 0x10)) -#define CXL_DVSEC_RANGE_BASE_LOW (i) (0x24 + (i * 0x10)) -#define CXL_DVSEC_MEM_BASE_LOW_MASK GENMASK(31, 28) - -#define CXL_DVSEC_RANGE_MAX 2 - -/* CXL 2.0 8.1.4: Non-CXL Function Map DVSEC */ -#define CXL_DVSEC_FUNCTION_MAP 2 - -/* CXL 2.0 8.1.5: CXL 2.0 Extensions DVSEC for Ports */ -#define CXL_DVSEC_PORT_EXTENSIONS 3 - -/* CXL 2.0 8.1.6: GPF DVSEC for CXL Port */ -#define CXL_DVSEC_PORT_GPF 4 -#define CXL_DVSEC_PORT_GPF_PHASE_1_CONTROL_OFFSET 0x0C -#define CXL_DVSEC_PORT_GPF_PHASE_1_TMO_BASE_MASK GENMASK(3, 0) -#define CXL_DVSEC_PORT_GPF_PHASE_1_TMO_SCALE_MASK GENMASK(11, 8) -#define CXL_DVSEC_PORT_GPF_PHASE_2_CONTROL_OFFSET 0xE -#define CXL_DVSEC_PORT_GPF_PHASE_2_TMO_BASE_MASK GENMASK(3, 0) -#define CXL_DVSEC_PORT_GPF_PHASE_2_TMO_SCALE_MASK GENMASK(11, 8) - -/* CXL 2.0 8.1.7: GPF DVSEC for CXL Device */ -#define CXL_DVSEC_DEVICE_GPF 5 - -/* CXL 2.0 8.1.8: PCIe DVSEC for Flex Bus Port */ -#define CXL_DVSEC_PCIE_FLEXBUS_PORT 7 - -/* CXL 2.0 8.1.9: Register Locator DVSEC */ -#define CXL_DVSEC_REG_LOCATOR 8 -#define CXL_DVSEC_REG_LOCATOR_BLOCK1_OFFSET 0xC -#define CXL_DVSEC_REG_LOCATOR_BIR_MASK GENMASK(2, 0) -#define CXL_DVSEC_REG_LOCATOR_BLOCK_ID_MASK GENMASK(15, 8) -#define CXL_DVSEC_REG_LOCATOR_BLOCK_OFF_LOW_MASK GENMASK(31, 16) -/* * NOTE: Currently all the functions which are enabled for CXL require their * vectors to be in the first 16. Use this as the default max. include/uapi/linux/pci_regs.h +59
-5@@ -1253,11 +1253,6 @@#define PCI_DEV3_STA 0x0c/* Device 3 Status Register */ #define PCI_DEV3_STA_SEGMENT 0x8/* Segment Captured (end-to-end flit-mode detected) */ -/* Compute Express Link (CXL r3.1, sec 8.1.5) */ -#define PCI_DVSEC_CXL_PORT 3 -#define PCI_DVSEC_CXL_PORT_CTL 0x0c -#define PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR 0x00000001 -/* Integrity and Data Encryption Extended Capability */ #define PCI_IDE_CAP 0x04#define PCI_IDE_CAP_LINK 0x1/* Link IDE Stream Supported */ @@ -1338,4 +1333,63 @@#define PCI_IDE_SEL_ADDR_3 (x) (28 + (x) * PCI_IDE_SEL_ADDR_BLOCK_SIZE)#define PCI_IDE_SEL_BLOCK_SIZE (nr_assoc) (20 + PCI_IDE_SEL_ADDR_BLOCK_SIZE * (nr_assoc)) +/* Compute Express Link (CXL r3.1, sec 8.1.5) */ +#define PCI_DVSEC_CXL_PORT 3 +#define PCI_DVSEC_CXL_PORT_CTL 0x0c +#define PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR 0x00000001 + +/* +* Compute Express Link (CXL r3.2, sec 8.1) +* +* Note that CXL DVSEC id 3 and 7 to be ignored when the CXL link state +* is "disconnected" (CXL r3.2, sec 9.12.3). Re-enumerate these +* registers on downstream link-up events. +*/ +#define PCI_DVSEC_HEADER1_LENGTH_MASK __GENMASK(31, 20) + +/* CXL 3.2 8.1.3: PCIe DVSEC for CXL Device */ +#define CXL_DVSEC_PCIE_DEVICE 0 +#define CXL_DVSEC_CAP_OFFSET 0xA +#define CXL_DVSEC_MEM_CAPABLE _BITUL(2) +#define CXL_DVSEC_HDM_COUNT_MASK __GENMASK(5, 4) +#define CXL_DVSEC_CTRL_OFFSET 0xC +#define CXL_DVSEC_MEM_ENABLE _BITUL(2) +#define CXL_DVSEC_RANGE_SIZE_HIGH (i) (0x18 + (i * 0x10)) +#define CXL_DVSEC_RANGE_SIZE_LOW (i) (0x1C + (i * 0x10)) +#define CXL_DVSEC_MEM_INFO_VALID _BITUL(0) +#define CXL_DVSEC_MEM_ACTIVE _BITUL(1) +#define CXL_DVSEC_MEM_SIZE_LOW_MASK __GENMASK(31, 28) +#define CXL_DVSEC_RANGE_BASE_HIGH (i) (0x20 + (i * 0x10)) +#define CXL_DVSEC_RANGE_BASE_LOW (i) (0x24 + (i * 0x10)) +#define CXL_DVSEC_MEM_BASE_LOW_MASK __GENMASK(31, 28) + +#define CXL_DVSEC_RANGE_MAX 2 + +/* CXL 3.2 8.1.4: Non-CXL Function Map DVSEC */ +#define CXL_DVSEC_FUNCTION_MAP 2 + +/* CXL 3.2 8.1.5: Extensions DVSEC for Ports */ +#define CXL_DVSEC_PORT 3 +#define CXL_DVSEC_PORT_CTL 0x0c +#define CXL_DVSEC_PORT_CTL_UNMASK_SBR 0x00000001 + +/* CXL 3.2 8.1.6: GPF DVSEC for CXL Port */ +#define CXL_DVSEC_PORT_GPF 4 +#define CXL_DVSEC_PORT_GPF_PHASE_1_CONTROL_OFFSET 0x0C +#define CXL_DVSEC_PORT_GPF_PHASE_1_TMO_BASE_MASK __GENMASK(3, 0) +#define CXL_DVSEC_PORT_GPF_PHASE_1_TMO_SCALE_MASK __GENMASK(11, 8) +#define CXL_DVSEC_PORT_GPF_PHASE_2_CONTROL_OFFSET 0xE +#define CXL_DVSEC_PORT_GPF_PHASE_2_TMO_BASE_MASK __GENMASK(3, 0) +#define CXL_DVSEC_PORT_GPF_PHASE_2_TMO_SCALE_MASK __GENMASK(11, 8) + +/* CXL 3.2 8.1.7: GPF DVSEC for CXL Device */ +#define CXL_DVSEC_DEVICE_GPF 5 + +/* CXL 3.2 8.1.9: Register Locator DVSEC */ +#define CXL_DVSEC_REG_LOCATOR 8 +#define CXL_DVSEC_REG_LOCATOR_BLOCK1_OFFSET 0xC +#define CXL_DVSEC_REG_LOCATOR_BIR_MASK __GENMASK(2, 0) +#define CXL_DVSEC_REG_LOCATOR_BLOCK_ID_MASK __GENMASK(15, 8) +#define CXL_DVSEC_REG_LOCATOR_BLOCK_OFF_LOW_MASK __GENMASK(31, 16) +#endif /* LINUX_PCI_REGS_H */ Signature · 1 line
2.34.1
Re: [PATCH v14 01/34] PCI: Move CXL DVSEC definitions into uapi/linux/pci_regs.h
Quoted 9 lines · Bjorn Helgaas
Quoted 2 lines · Terry Bowman
The CXL DVSECs are currently defined in
cxl/core/cxlpci.h. These are not accessible to other subsystems. Move these touapi/linux/pci_regs.h.I'm looking a little bit askance at adding things to
uapi/linux/pci_regs.hand then renaming them. I know it's OCD to worry about that momentary blip, but changes in uapi potentially break userspace.
Quoted 6 lines
Maybe we could rename them first, then move them to
pci_regs.h?Either way:
Acked-by Bjorn Helgaas <[email protected]>
Ok, I'll update the naming before moving.
Thanks for reviewing.
-Terry
Quoted 182 lines
Quoted 182 lines
The CXL DVSEC definitions will be renamed and reformatted to fit better with existing defines.
Signed-off-by Terry Bowman <[email protected]>Reviewed-by Dave Jiang <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Dan Williams <[email protected]>----
Changes in
v13->v14:
- Add Jonathan's and Dan's review-by
- Update commit title prefix (Bjorn)
- Revert format fix forcxl_sbr_masked()(Jonathan)
- Update 'Compute Express Link' comment block (Jonathan)
- MovePCI_DVSEC_CXL_FLEXBUSdefinitions to later patch where used (Jonathan)
- Removed stray change (Bjorn)Changes in
v12->v13:
- Add Dave Jiang's reviewed-by
- Remove changes to existing PCI_DVSEC_CXL_PORT* defines. Update commit message. (Jonathan)Changes in v11 -> v12:
- Change formatting to be same as existing definitions
- ChangeGENMASK()->__GENMASK()andBIT()to_BITUL()Changes in v10 -> v11:
- New commit--- drivers/cxl/cxlpci.h | 53 ----------------------------- include/uapi/linux/pci_regs.h | 64 ++++++++++++++++++++++++++++++++--- 2 files changed, 59 insertions(+), 58 deletions(-)drivers/cxl/cxlpci.h +0
-53@@ -7,59 +7,6 @@#define CXL_MEMORY_PROGIF 0x10 -/* -* See section 8.1 Configuration Space Registers in the CXL 2.0 -* Specification. Names are taken straight from the specification with "CXL" and -* "DVSEC" redundancies removed. When obvious, abbreviations may be used. -*/ -#define PCI_DVSEC_HEADER1_LENGTH_MASK GENMASK(31, 20) - -/* CXL 2.0 8.1.3: PCIe DVSEC for CXL Device */ -#define CXL_DVSEC_PCIE_DEVICE 0 -#define CXL_DVSEC_CAP_OFFSET 0xA -#define CXL_DVSEC_MEM_CAPABLE BIT(2) -#define CXL_DVSEC_HDM_COUNT_MASK GENMASK(5, 4) -#define CXL_DVSEC_CTRL_OFFSET 0xC -#define CXL_DVSEC_MEM_ENABLE BIT(2) -#define CXL_DVSEC_RANGE_SIZE_HIGH (i) (0x18 + (i * 0x10)) -#define CXL_DVSEC_RANGE_SIZE_LOW (i) (0x1C + (i * 0x10)) -#define CXL_DVSEC_MEM_INFO_VALID BIT(0) -#define CXL_DVSEC_MEM_ACTIVE BIT(1) -#define CXL_DVSEC_MEM_SIZE_LOW_MASK GENMASK(31, 28) -#define CXL_DVSEC_RANGE_BASE_HIGH (i) (0x20 + (i * 0x10)) -#define CXL_DVSEC_RANGE_BASE_LOW (i) (0x24 + (i * 0x10)) -#define CXL_DVSEC_MEM_BASE_LOW_MASK GENMASK(31, 28) - -#define CXL_DVSEC_RANGE_MAX 2 - -/* CXL 2.0 8.1.4: Non-CXL Function Map DVSEC */ -#define CXL_DVSEC_FUNCTION_MAP 2 - -/* CXL 2.0 8.1.5: CXL 2.0 Extensions DVSEC for Ports */ -#define CXL_DVSEC_PORT_EXTENSIONS 3 - -/* CXL 2.0 8.1.6: GPF DVSEC for CXL Port */ -#define CXL_DVSEC_PORT_GPF 4 -#define CXL_DVSEC_PORT_GPF_PHASE_1_CONTROL_OFFSET 0x0C -#define CXL_DVSEC_PORT_GPF_PHASE_1_TMO_BASE_MASK GENMASK(3, 0) -#define CXL_DVSEC_PORT_GPF_PHASE_1_TMO_SCALE_MASK GENMASK(11, 8) -#define CXL_DVSEC_PORT_GPF_PHASE_2_CONTROL_OFFSET 0xE -#define CXL_DVSEC_PORT_GPF_PHASE_2_TMO_BASE_MASK GENMASK(3, 0) -#define CXL_DVSEC_PORT_GPF_PHASE_2_TMO_SCALE_MASK GENMASK(11, 8) - -/* CXL 2.0 8.1.7: GPF DVSEC for CXL Device */ -#define CXL_DVSEC_DEVICE_GPF 5 - -/* CXL 2.0 8.1.8: PCIe DVSEC for Flex Bus Port */ -#define CXL_DVSEC_PCIE_FLEXBUS_PORT 7 - -/* CXL 2.0 8.1.9: Register Locator DVSEC */ -#define CXL_DVSEC_REG_LOCATOR 8 -#define CXL_DVSEC_REG_LOCATOR_BLOCK1_OFFSET 0xC -#define CXL_DVSEC_REG_LOCATOR_BIR_MASK GENMASK(2, 0) -#define CXL_DVSEC_REG_LOCATOR_BLOCK_ID_MASK GENMASK(15, 8) -#define CXL_DVSEC_REG_LOCATOR_BLOCK_OFF_LOW_MASK GENMASK(31, 16) -/* * NOTE: Currently all the functions which are enabled for CXL require their * vectors to be in the first 16. Use this as the default max. include/uapi/linux/pci_regs.h +59
-5@@ -1253,11 +1253,6 @@#define PCI_DEV3_STA 0x0c/* Device 3 Status Register */ #define PCI_DEV3_STA_SEGMENT 0x8/* Segment Captured (end-to-end flit-mode detected) */ -/* Compute Express Link (CXL r3.1, sec 8.1.5) */ -#define PCI_DVSEC_CXL_PORT 3 -#define PCI_DVSEC_CXL_PORT_CTL 0x0c -#define PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR 0x00000001 -/* Integrity and Data Encryption Extended Capability */ #define PCI_IDE_CAP 0x04#define PCI_IDE_CAP_LINK 0x1/* Link IDE Stream Supported */ @@ -1338,4 +1333,63 @@#define PCI_IDE_SEL_ADDR_3 (x) (28 + (x) * PCI_IDE_SEL_ADDR_BLOCK_SIZE)#define PCI_IDE_SEL_BLOCK_SIZE (nr_assoc) (20 + PCI_IDE_SEL_ADDR_BLOCK_SIZE * (nr_assoc)) +/* Compute Express Link (CXL r3.1, sec 8.1.5) */ +#define PCI_DVSEC_CXL_PORT 3 +#define PCI_DVSEC_CXL_PORT_CTL 0x0c +#define PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR 0x00000001 + +/* +* Compute Express Link (CXL r3.2, sec 8.1) +* +* Note that CXL DVSEC id 3 and 7 to be ignored when the CXL link state +* is "disconnected" (CXL r3.2, sec 9.12.3). Re-enumerate these +* registers on downstream link-up events. +*/ +#define PCI_DVSEC_HEADER1_LENGTH_MASK __GENMASK(31, 20) + +/* CXL 3.2 8.1.3: PCIe DVSEC for CXL Device */ +#define CXL_DVSEC_PCIE_DEVICE 0 +#define CXL_DVSEC_CAP_OFFSET 0xA +#define CXL_DVSEC_MEM_CAPABLE _BITUL(2) +#define CXL_DVSEC_HDM_COUNT_MASK __GENMASK(5, 4) +#define CXL_DVSEC_CTRL_OFFSET 0xC +#define CXL_DVSEC_MEM_ENABLE _BITUL(2) +#define CXL_DVSEC_RANGE_SIZE_HIGH (i) (0x18 + (i * 0x10)) +#define CXL_DVSEC_RANGE_SIZE_LOW (i) (0x1C + (i * 0x10)) +#define CXL_DVSEC_MEM_INFO_VALID _BITUL(0) +#define CXL_DVSEC_MEM_ACTIVE _BITUL(1) +#define CXL_DVSEC_MEM_SIZE_LOW_MASK __GENMASK(31, 28) +#define CXL_DVSEC_RANGE_BASE_HIGH (i) (0x20 + (i * 0x10)) +#define CXL_DVSEC_RANGE_BASE_LOW (i) (0x24 + (i * 0x10)) +#define CXL_DVSEC_MEM_BASE_LOW_MASK __GENMASK(31, 28) + +#define CXL_DVSEC_RANGE_MAX 2 + +/* CXL 3.2 8.1.4: Non-CXL Function Map DVSEC */ +#define CXL_DVSEC_FUNCTION_MAP 2 + +/* CXL 3.2 8.1.5: Extensions DVSEC for Ports */ +#define CXL_DVSEC_PORT 3 +#define CXL_DVSEC_PORT_CTL 0x0c +#define CXL_DVSEC_PORT_CTL_UNMASK_SBR 0x00000001 + +/* CXL 3.2 8.1.6: GPF DVSEC for CXL Port */ +#define CXL_DVSEC_PORT_GPF 4 +#define CXL_DVSEC_PORT_GPF_PHASE_1_CONTROL_OFFSET 0x0C +#define CXL_DVSEC_PORT_GPF_PHASE_1_TMO_BASE_MASK __GENMASK(3, 0) +#define CXL_DVSEC_PORT_GPF_PHASE_1_TMO_SCALE_MASK __GENMASK(11, 8) +#define CXL_DVSEC_PORT_GPF_PHASE_2_CONTROL_OFFSET 0xE +#define CXL_DVSEC_PORT_GPF_PHASE_2_TMO_BASE_MASK __GENMASK(3, 0) +#define CXL_DVSEC_PORT_GPF_PHASE_2_TMO_SCALE_MASK __GENMASK(11, 8) + +/* CXL 3.2 8.1.7: GPF DVSEC for CXL Device */ +#define CXL_DVSEC_DEVICE_GPF 5 + +/* CXL 3.2 8.1.9: Register Locator DVSEC */ +#define CXL_DVSEC_REG_LOCATOR 8 +#define CXL_DVSEC_REG_LOCATOR_BLOCK1_OFFSET 0xC +#define CXL_DVSEC_REG_LOCATOR_BIR_MASK __GENMASK(2, 0) +#define CXL_DVSEC_REG_LOCATOR_BLOCK_ID_MASK __GENMASK(15, 8) +#define CXL_DVSEC_REG_LOCATOR_BLOCK_OFF_LOW_MASK __GENMASK(31, 16) +#endif /* LINUX_PCI_REGS_H */ Signature · 1 line
2.34.1
PATCH v14 2/34 PCI: Update CXL DVSEC definitions
In reply to Terry BowmanCXL DVSEC definitions were recently moved into uapi/pci_regs.h, but the newly added macros do not follow the file's existing naming conventions. The current format uses CXL_DVSEC_XYZ, while the new CXL entries must instead use the PCI_DVSEC_CXL_XYZ prefix to match the conventions already established in pci_regs.h.
The new CXL DVSEC macros also introduce _MASK and _OFFSET suffixes, which are not used anywhere else in the file. These suffixes lengthen the identifiers and reduce readability. Remove _MASK and _OFFSET from the recently added definitions.
Additionally, remove PCI_DVSEC_HEADER1_LENGTH, as it duplicates the existing PCI_DVSEC_HEADER1_LEN() macro.
Update all existing references to use the new macro names.
Finally, update the inline documentation to reference the latest revision of the CXL specification.
---
Changes in v13->v14:
- New patch. Split from previous patch such that there is now a separate move patch and a format fix patch.
- Formatting update requested (Bjorn)
- Remove PCI_DVSEC_HEADER1_LENGTH_MASK because it duplicates PCI_DVSEC_HEADER1_LEN() (Bjorn)
- Add Dan's review-by
--- drivers/cxl/core/pci.c | 58 ++++++++++----------- drivers/cxl/core/regs.c | 14 +++--- drivers/cxl/pci.c | 2 +- include/uapi/linux/pci_regs.h | 94 ++++++++++++++++------------------- 4 files changed, 81 insertions(+), 87 deletions(-)
drivers/cxl/core/pci.c +29 -29
@@ -86,12 +86,12 @@ static int cxl_dvsec_mem_range_valid(struct cxl_dev_state *cxlds, int id) i =1 ; do { rc = pci_read_config_dword(pdev, - d + CXL_DVSEC_RANGE_SIZE_LOW(id), + d + PCI_DVSEC_CXL_RANGE_SIZE_LOW(id), &temp); if (rc) return rc; - valid = FIELD_GET(CXL_DVSEC_MEM_INFO_VALID, temp); + valid = FIELD_GET(PCI_DVSEC_CXL_MEM_INFO_VALID, temp); if (valid) break; msleep(1000);
@@ -121,11 +121,11 @@ static int cxl_dvsec_mem_range_active(struct cxl_dev_state *cxlds, int id)/* Check MEM ACTIVE bit, up to 60s timeout by default */ for (i = media_ready_timeout; i; i--) { rc =pci_read_config_dword ( - pdev, d +CXL_DVSEC_RANGE_SIZE_LOW (id), &temp); + pdev, d +PCI_DVSEC_CXL_RANGE_SIZE_LOW (id), &temp);if (rc)return rc; - active =FIELD_GET (CXL_DVSEC_MEM_ACTIVE , temp); + active =FIELD_GET (PCI_DVSEC_CXL_MEM_ACTIVE , temp);if (active)break ;msleep (1000 );
@@ -154,11 +154,11 @@ int cxl_await_media_ready(struct cxl_dev_state *cxlds)u16 cap; rc =pci_read_config_word (pdev, - d +CXL_DVSEC_CAP_OFFSET , &cap); + d +PCI_DVSEC_CXL_CAP , &cap);if (rc)return rc; - hdm_count =FIELD_GET (CXL_DVSEC_HDM_COUNT_MASK , cap); + hdm_count =FIELD_GET (PCI_DVSEC_CXL_HDM_COUNT , cap);for (i =0 ; i < hdm_count; i++) { rc =cxl_dvsec_mem_range_valid (cxlds, i);if (rc)
@@ -186,16 +186,16 @@ static int cxl_set_mem_enable(struct cxl_dev_state *cxlds, u16 val)u16 ctrl;int rc; - rc =pci_read_config_word (pdev, d +CXL_DVSEC_CTRL_OFFSET , &ctrl); + rc =pci_read_config_word (pdev, d +PCI_DVSEC_CXL_CTRL , &ctrl);if (rc <0 )return rc; -if ((ctrl &CXL_DVSEC_MEM_ENABLE ) == val) +if ((ctrl &PCI_DVSEC_CXL_MEM_ENABLE ) == val)return 1 ; - ctrl &= ~CXL_DVSEC_MEM_ENABLE ; + ctrl &= ~PCI_DVSEC_CXL_MEM_ENABLE ; ctrl |= val; - rc =pci_write_config_word (pdev, d +CXL_DVSEC_CTRL_OFFSET , ctrl); + rc =pci_write_config_word (pdev, d +PCI_DVSEC_CXL_CTRL , ctrl);if (rc <0 )return rc;
@@ -211,7 +211,7 @@ static int devm_cxl_enable_mem(struct device *host, struct cxl_dev_state *cxlds) {int rc; - rc =cxl_set_mem_enable (cxlds,CXL_DVSEC_MEM_ENABLE ); + rc =cxl_set_mem_enable (cxlds,PCI_DVSEC_CXL_MEM_ENABLE );if (rc <0 )return rc;if (rc >0 )
@@ -273,11 +273,11 @@ int cxl_dvsec_rr_decode(struct cxl_dev_state *cxlds,return -ENXIO ; } - rc =pci_read_config_word (pdev, d +CXL_DVSEC_CAP_OFFSET , &cap); + rc =pci_read_config_word (pdev, d +PCI_DVSEC_CXL_CAP , &cap);if (rc)return rc; -if (!(cap &CXL_DVSEC_MEM_CAPABLE )) { +if (!(cap &PCI_DVSEC_CXL_MEM_CAPABLE )) {dev_dbg (dev,"Not MEM Capable\n" );return -ENXIO ; }
@@ -288,7 +288,7 @@ int cxl_dvsec_rr_decode(struct cxl_dev_state *cxlds,* driver is for a spec defined class code which must be CXL.mem * capable, there is no point in continuing to enable CXL.mem. */ - hdm_count =FIELD_GET (CXL_DVSEC_HDM_COUNT_MASK , cap); + hdm_count =FIELD_GET (PCI_DVSEC_CXL_HDM_COUNT , cap);if (!hdm_count || hdm_count >2 )return -EINVAL ;
@@ -297,11 +297,11 @@ int cxl_dvsec_rr_decode(struct cxl_dev_state *cxlds,* disabled, and they will remain moot after the HDM Decoder * capability is enabled. */ - rc =pci_read_config_word (pdev, d +CXL_DVSEC_CTRL_OFFSET , &ctrl); + rc =pci_read_config_word (pdev, d +PCI_DVSEC_CXL_CTRL , &ctrl);if (rc)return rc; - info->mem_enabled =FIELD_GET (CXL_DVSEC_MEM_ENABLE , ctrl); + info->mem_enabled =FIELD_GET (PCI_DVSEC_CXL_MEM_ENABLE , ctrl);if (!info->mem_enabled)return 0 ;
@@ -314,35 +314,35 @@ int cxl_dvsec_rr_decode(struct cxl_dev_state *cxlds,return rc; rc =pci_read_config_dword ( - pdev, d +CXL_DVSEC_RANGE_SIZE_HIGH (i), &temp); + pdev, d +PCI_DVSEC_CXL_RANGE_SIZE_HIGH (i), &temp);if (rc)return rc; size = (u64 )temp <<32 ; rc =pci_read_config_dword ( - pdev, d +CXL_DVSEC_RANGE_SIZE_LOW (i), &temp); + pdev, d +PCI_DVSEC_CXL_RANGE_SIZE_LOW (i), &temp);if (rc)return rc; - size |= temp &CXL_DVSEC_MEM_SIZE_LOW_MASK ; + size |= temp &PCI_DVSEC_CXL_MEM_SIZE_LOW ;if (!size) {continue ; } rc =pci_read_config_dword ( - pdev, d +CXL_DVSEC_RANGE_BASE_HIGH (i), &temp); + pdev, d +PCI_DVSEC_CXL_RANGE_BASE_HIGH (i), &temp);if (rc)return rc; base = (u64 )temp <<32 ; rc =pci_read_config_dword ( - pdev, d +CXL_DVSEC_RANGE_BASE_LOW (i), &temp); + pdev, d +PCI_DVSEC_CXL_RANGE_BASE_LOW (i), &temp);if (rc)return rc; - base |= temp &CXL_DVSEC_MEM_BASE_LOW_MASK ; + base |= temp &PCI_DVSEC_CXL_MEM_BASE_LOW ; info->dvsec_range[ranges++] = (struct range ) { .start = base,
@@ -1068,7 +1068,7 @@ u16 cxl_gpf_get_dvsec(struct device *dev) is_port = false; dvsec =pci_find_dvsec_capability (pdev,PCI_VENDOR_ID_CXL , - is_port ?CXL_DVSEC_PORT_GPF :CXL_DVSEC_DEVICE_GPF ); + is_port ?PCI_DVSEC_CXL_PORT_GPF :PCI_DVSEC_CXL_DEVICE_GPF );if (!dvsec)dev_warn (dev,"%s GPF DVSEC not present\n" , is_port ?"Port" :"Device" );
@@ -1084,14 +1084,14 @@ static int update_gpf_port_dvsec(struct pci_dev *pdev, int dvsec, int phase)switch (phase) {case 1 : - offset =CXL_DVSEC_PORT_GPF_PHASE_1_CONTROL_OFFSET ; - base =CXL_DVSEC_PORT_GPF_PHASE_1_TMO_BASE_MASK ; - scale =CXL_DVSEC_PORT_GPF_PHASE_1_TMO_SCALE_MASK ; + offset =PCI_DVSEC_CXL_PORT_GPF_PHASE_1_CONTROL ; + base =PCI_DVSEC_CXL_PORT_GPF_PHASE_1_TMO_BASE ; + scale =PCI_DVSEC_CXL_PORT_GPF_PHASE_1_TMO_SCALE ;break ;case 2 : - offset =CXL_DVSEC_PORT_GPF_PHASE_2_CONTROL_OFFSET ; - base =CXL_DVSEC_PORT_GPF_PHASE_2_TMO_BASE_MASK ; - scale =CXL_DVSEC_PORT_GPF_PHASE_2_TMO_SCALE_MASK ; + offset =PCI_DVSEC_CXL_PORT_GPF_PHASE_2_CONTROL ; + base =PCI_DVSEC_CXL_PORT_GPF_PHASE_2_TMO_BASE ; + scale =PCI_DVSEC_CXL_PORT_GPF_PHASE_2_TMO_SCALE ;break ;default :return -EINVAL ;
drivers/cxl/core/regs.c +7 -7
@@ -271,10 +271,10 @@ EXPORT_SYMBOL_NS_GPL(cxl_map_device_regs, "CXL");static bool cxl_decode_regblock (struct pci_dev *pdev,u32 reg_lo,u32 reg_hi,struct cxl_register_map *map) { -u8 reg_type =FIELD_GET (CXL_DVSEC_REG_LOCATOR_BLOCK_ID_MASK , reg_lo); -int bar =FIELD_GET (CXL_DVSEC_REG_LOCATOR_BIR_MASK , reg_lo); +u8 reg_type =FIELD_GET (PCI_DVSEC_CXL_REG_LOCATOR_BLOCK_ID , reg_lo); +int bar =FIELD_GET (PCI_DVSEC_CXL_REG_LOCATOR_BIR , reg_lo);u64 offset = ((u64 )reg_hi <<32 ) | - (reg_lo &CXL_DVSEC_REG_LOCATOR_BLOCK_OFF_LOW_MASK ); + (reg_lo &PCI_DVSEC_CXL_REG_LOCATOR_BLOCK_OFF_LOW );if (offset >pci_resource_len (pdev, bar)) { dev_warn(&pdev->dev,
@@ -311,15 +311,15 @@ static int __cxl_find_regblock_instance(struct pci_dev *pdev, enum cxl_regloc_ty }; regloc =pci_find_dvsec_capability (pdev,PCI_VENDOR_ID_CXL , -CXL_DVSEC_REG_LOCATOR ); +PCI_DVSEC_CXL_REG_LOCATOR );if (!regloc)return -ENXIO ;pci_read_config_dword (pdev, regloc +PCI_DVSEC_HEADER1 , ®loc_size); - regloc_size =FIELD_GET (PCI_DVSEC_HEADER1_LENGTH_MASK , regloc_size); + regloc_size =PCI_DVSEC_HEADER1_LEN (regloc_size); - regloc +=CXL_DVSEC_REG_LOCATOR_BLOCK1_OFFSET ; - regblocks = (regloc_size -CXL_DVSEC_REG_LOCATOR_BLOCK1_OFFSET ) /8 ; + regloc +=PCI_DVSEC_CXL_REG_LOCATOR_BLOCK1 ; + regblocks = (regloc_size -PCI_DVSEC_CXL_REG_LOCATOR_BLOCK1 ) /8 ;for (i =0 ; i < regblocks; i++, regloc +=8 ) {u32 reg_lo, reg_hi;
drivers/cxl/pci.c +1 -1
@@ -933,7 +933,7 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) cxlds->rcd =is_cxl_restricted (pdev); cxlds->serial =pci_get_dsn (pdev); cxlds->cxl_dvsec =pci_find_dvsec_capability ( - pdev,PCI_VENDOR_ID_CXL ,CXL_DVSEC_PCIE_DEVICE ); + pdev,PCI_VENDOR_ID_CXL ,PCI_DVSEC_CXL_DEVICE );if (!cxlds->cxl_dvsec)dev_warn (&pdev->dev,"Device DVSEC not present, skip CXL.mem init\n" );
include/uapi/linux/pci_regs.h +44 -50
107 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH v14 02/34] PCI: Update CXL DVSEC definitions
Quoted 24 lines · On Wed, 14 Jan 2026 12:20:23 -0600 Terry Bowman <[email protected]> wrote:
CXL DVSEC definitions were recently moved into
uapi/pci_regs.h, but the newly added macros do not follow the file's existing naming conventions. The current format usesCXL_DVSEC_XYZ, while the new CXL entries must instead use thePCI_DVSEC_CXL_XYZprefix to match the conventions already established inpci_regs.h.The new CXL DVSEC macros also introduce
_MASKand_OFFSETsuffixes, which are not used anywhere else in the file. These suffixes lengthen the identifiers and reduce readability. Remove_MASKand_OFFSETfrom the recently added definitions.Additionally, remove
PCI_DVSEC_HEADER1_LENGTH, as it duplicates the existingPCI_DVSEC_HEADER1_LEN()macro.Update all existing references to use the new macro names.
Finally, update the inline documentation to reference the latest revision of the CXL specification.
Signed-off-by Terry Bowman <[email protected]>Reviewed-by Dan Williams <[email protected]>---
Quoted 7 lines
Changes in
v13->v14:
- New patch. Split from previous patch such that there is now a separate move patch and a format fix patch.
- Formatting update requested (Bjorn)
- RemovePCI_DVSEC_HEADER1_LENGTH_MASKbecause it duplicatesPCI_DVSEC_HEADER1_LEN()(Bjorn)
- Add Dan's review-by
A couple of name choices don't quite work to my reading. I care more about the _DEVICE_ one than the other.
If consensus is that it is fine with out then I could probably be persuaded.
include/uapi/linux/pci_regs.h line 1346
Quoted 9 lines
include/uapi/linux/pci_regs.h +3
-0+/* CXL r4.0, 8.1.3: PCIe DVSEC for CXL Device */ +#define PCI_DVSEC_CXL_DEVICE 0 +#define PCI_DVSEC_CXL_CAP 0xA
Why drop the _DEVICE_ bit of these I'd kind of expect #define PCI_DVSEC_CXL_DEVICE_CAP
to indicate which DVSEC it is in.
Quoted 14 lines
include/uapi/linux/pci_regs.h +12
-0+#define PCI_DVSEC_CXL_MEM_CAPABLE _BITUL(2) +#define PCI_DVSEC_CXL_HDM_COUNT __GENMASK(5, 4) +#define PCI_DVSEC_CXL_CTRL 0xC +#define PCI_DVSEC_CXL_MEM_ENABLE _BITUL(2) +#define PCI_DVSEC_CXL_RANGE_SIZE_HIGH (i) (0x18 + (i * 0x10)) +#define PCI_DVSEC_CXL_RANGE_SIZE_LOW (i) (0x1C + (i * 0x10)) +#define PCI_DVSEC_CXL_MEM_INFO_VALID _BITUL(0) +#define PCI_DVSEC_CXL_MEM_ACTIVE _BITUL(1) +#define PCI_DVSEC_CXL_MEM_SIZE_LOW __GENMASK(31, 28) +#define PCI_DVSEC_CXL_RANGE_BASE_HIGH (i) (0x20 + (i * 0x10)) +#define PCI_DVSEC_CXL_RANGE_BASE_LOW (i) (0x24 + (i * 0x10)) +#define PCI_DVSEC_CXL_MEM_BASE_LOW __GENMASK(31, 28)
include/uapi/linux/pci_regs.h line 1371
Quoted 10 lines
include/uapi/linux/pci_regs.h +10
-0+/* CXL r4.0, 8.1.4: Non-CXL Function Map DVSEC */ +#define PCI_DVSEC_CXL_FUNCTION_MAP 2 + +/* CXL r4.0, 8.1.5: Extensions DVSEC for Ports */ +#define PCI_DVSEC_CXL_PORT 3 +#define PCI_DVSEC_CXL_PORT_CTL 0x0c +#define PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR 0x00000001 + +/* CXL r4.0, 8.1.6: GPF DVSEC for CXL Port */ +#define PCI_DVSEC_CXL_PORT_GPF 4
Nothing like ambiguous naming in the CXL spec as the following fields sound like they are in the CXL_PORT dvsec but they aren't. Well the spec avoids it with GPF_FOR_PORT but we don't want to go there. I wonder... PCI_DVSEC_CXL_PORTGPF maybe to avoid that?
Sigh. It's probably not worth it and does look horrible, so stick with these.
Quoted 18 lines
include/uapi/linux/pci_regs.h +16
-0+#define PCI_DVSEC_CXL_PORT_GPF_PHASE_1_CONTROL 0x0C +#define PCI_DVSEC_CXL_PORT_GPF_PHASE_1_TMO_BASE __GENMASK(3, 0) +#define PCI_DVSEC_CXL_PORT_GPF_PHASE_1_TMO_SCALE __GENMASK(11, 8) +#define PCI_DVSEC_CXL_PORT_GPF_PHASE_2_CONTROL 0xE +#define PCI_DVSEC_CXL_PORT_GPF_PHASE_2_TMO_BASE __GENMASK(3, 0) +#define PCI_DVSEC_CXL_PORT_GPF_PHASE_2_TMO_SCALE __GENMASK(11, 8) + +/* CXL r4.0, 8.1.7: GPF DVSEC for CXL Device */ +#define PCI_DVSEC_CXL_DEVICE_GPF 5 + +/* CXL r4.0, 8.1.9: Register Locator DVSEC */ +#define PCI_DVSEC_CXL_REG_LOCATOR 8 +#define PCI_DVSEC_CXL_REG_LOCATOR_BLOCK1 0xC +#define PCI_DVSEC_CXL_REG_LOCATOR_BIR __GENMASK(2, 0) +#define PCI_DVSEC_CXL_REG_LOCATOR_BLOCK_ID __GENMASK(15, 8) +#define PCI_DVSEC_CXL_REG_LOCATOR_BLOCK_OFF_LOW __GENMASK(31, 16)#endif /* LINUX_PCI_REGS_H */
Re: [PATCH v14 02/34] PCI: Update CXL DVSEC definitions
Jonathan Cameron wrote:
[..]
Quoted 6 lines
Quoted 4 lines
include/uapi/linux/pci_regs.h +0
-0
Quoted 7 lines
include/uapi/linux/pci_regs.h line 1346
Quoted 3 lines
include/uapi/linux/pci_regs.h +3
-0+/* CXL r4.0, 8.1.3: PCIe DVSEC for CXL Device */ +#define PCI_DVSEC_CXL_DEVICE 0 +#define PCI_DVSEC_CXL_CAP 0xAWhy drop the _DEVICE_ bit of these I'd kind of expect #define
PCI_DVSEC_CXL_DEVICE_CAP
to indicate which DVSEC it is in.
We got by without the redundant _DEVICE_ in the name to date, the port DVSEC has the _PORT_ differentiation. In the interest of not needing to review another version of this simple patch I vote leave well enough alone. Will leave it to Bjorn if he wants to override.
[..]
Quoted 11 lines
include/uapi/linux/pci_regs.h line 1371
Quoted 2 lines
include/uapi/linux/pci_regs.h +2
-0+/* CXL r4.0, 8.1.6: GPF DVSEC for CXL Port */ +#define PCI_DVSEC_CXL_PORT_GPF 4Nothing like ambiguous naming in the CXL spec as the following fields sound like they are in the
CXL_PORTdvsec but they aren't. Well the spec avoids it withGPF_FOR_PORTbut we don't want to go there. I wonder...PCI_DVSEC_CXL_PORTGPFmaybe to avoid that?Sigh. It's probably not worth it and does look horrible, so stick with these.
Not seeing a siginficant improvement in the suggestion.
Re: [PATCH v14 02/34] PCI: Update CXL DVSEC definitions
In reply to Terry BowmanQuoted 14 lines · Terry Bowman
CXL DVSEC definitions were recently moved into
uapi/pci_regs.h, but the newly added macros do not follow the file's existing naming conventions. The current format usesCXL_DVSEC_XYZ, while the new CXL entries must instead use thePCI_DVSEC_CXL_XYZprefix to match the conventions already established inpci_regs.h.The new CXL DVSEC macros also introduce
_MASKand_OFFSETsuffixes, which are not used anywhere else in the file. These suffixes lengthen the identifiers and reduce readability. Remove_MASKand_OFFSETfrom the recently added definitions.Additionally, remove
PCI_DVSEC_HEADER1_LENGTH, as it duplicates the existingPCI_DVSEC_HEADER1_LEN()macro.
Quoted 7 lines
Update all existing references to use the new macro names.
Finally, update the inline documentation to reference the latest revision of the CXL specification.
Signed-off-by Terry Bowman <[email protected]>Reviewed-by Dan Williams <[email protected]>
I do agree that PCI_DVSEC_CXL_CAP seems possibly a little too generic given that there may be more CXL-related DVSECs, so I like Jonathan's PCI_DVSEC_CXL_DEVICE_CAP idea.
Keep my ack either way.
Quoted 368 lines
---
Changes in
v13->v14:
- New patch. Split from previous patch such that there is now a separate move patch and a format fix patch.
- Formatting update requested (Bjorn)
- RemovePCI_DVSEC_HEADER1_LENGTH_MASKbecause it duplicatesPCI_DVSEC_HEADER1_LEN()(Bjorn)
- Add Dan's review-by--- drivers/cxl/core/pci.c | 58 ++++++++++----------- drivers/cxl/core/regs.c | 14 +++--- drivers/cxl/pci.c | 2 +- include/uapi/linux/pci_regs.h | 94 ++++++++++++++++------------------- 4 files changed, 81 insertions(+), 87 deletions(-)drivers/cxl/core/pci.c +29
-29@@ -86,12 +86,12 @@ static int cxl_dvsec_mem_range_valid(struct cxl_dev_state *cxlds, int id) i =1 ; do { rc = pci_read_config_dword(pdev, - d + CXL_DVSEC_RANGE_SIZE_LOW(id), + d + PCI_DVSEC_CXL_RANGE_SIZE_LOW(id), &temp); if (rc) return rc; - valid = FIELD_GET(CXL_DVSEC_MEM_INFO_VALID, temp); + valid = FIELD_GET(PCI_DVSEC_CXL_MEM_INFO_VALID, temp); if (valid) break; msleep(1000);@@ -121,11 +121,11 @@ static int cxl_dvsec_mem_range_active(struct cxl_dev_state *cxlds, int id)/* Check MEM ACTIVE bit, up to 60s timeout by default */ for (i = media_ready_timeout; i; i--) { rc =pci_read_config_dword ( - pdev, d +CXL_DVSEC_RANGE_SIZE_LOW (id), &temp); + pdev, d +PCI_DVSEC_CXL_RANGE_SIZE_LOW (id), &temp);if (rc)return rc; - active =FIELD_GET (CXL_DVSEC_MEM_ACTIVE , temp); + active =FIELD_GET (PCI_DVSEC_CXL_MEM_ACTIVE , temp);if (active)break ;msleep (1000 );@@ -154,11 +154,11 @@ int cxl_await_media_ready(struct cxl_dev_state *cxlds)u16 cap; rc =pci_read_config_word (pdev, - d +CXL_DVSEC_CAP_OFFSET , &cap); + d +PCI_DVSEC_CXL_CAP , &cap);if (rc)return rc; - hdm_count =FIELD_GET (CXL_DVSEC_HDM_COUNT_MASK , cap); + hdm_count =FIELD_GET (PCI_DVSEC_CXL_HDM_COUNT , cap);for (i =0 ; i < hdm_count; i++) { rc =cxl_dvsec_mem_range_valid (cxlds, i);if (rc)@@ -186,16 +186,16 @@ static int cxl_set_mem_enable(struct cxl_dev_state *cxlds, u16 val)u16 ctrl;int rc; - rc =pci_read_config_word (pdev, d +CXL_DVSEC_CTRL_OFFSET , &ctrl); + rc =pci_read_config_word (pdev, d +PCI_DVSEC_CXL_CTRL , &ctrl);if (rc <0 )return rc; -if ((ctrl &CXL_DVSEC_MEM_ENABLE ) == val) +if ((ctrl &PCI_DVSEC_CXL_MEM_ENABLE ) == val)return 1 ; - ctrl &= ~CXL_DVSEC_MEM_ENABLE ; + ctrl &= ~PCI_DVSEC_CXL_MEM_ENABLE ; ctrl |= val; - rc =pci_write_config_word (pdev, d +CXL_DVSEC_CTRL_OFFSET , ctrl); + rc =pci_write_config_word (pdev, d +PCI_DVSEC_CXL_CTRL , ctrl);if (rc <0 )return rc;@@ -211,7 +211,7 @@ static int devm_cxl_enable_mem(struct device *host, struct cxl_dev_state *cxlds) {int rc; - rc =cxl_set_mem_enable (cxlds,CXL_DVSEC_MEM_ENABLE ); + rc =cxl_set_mem_enable (cxlds,PCI_DVSEC_CXL_MEM_ENABLE );if (rc <0 )return rc;if (rc >0 )@@ -273,11 +273,11 @@ int cxl_dvsec_rr_decode(struct cxl_dev_state *cxlds,return -ENXIO ; } - rc =pci_read_config_word (pdev, d +CXL_DVSEC_CAP_OFFSET , &cap); + rc =pci_read_config_word (pdev, d +PCI_DVSEC_CXL_CAP , &cap);if (rc)return rc; -if (!(cap &CXL_DVSEC_MEM_CAPABLE )) { +if (!(cap &PCI_DVSEC_CXL_MEM_CAPABLE )) {dev_dbg (dev,"Not MEM Capable\n" );return -ENXIO ; }@@ -288,7 +288,7 @@ int cxl_dvsec_rr_decode(struct cxl_dev_state *cxlds,* driver is for a spec defined class code which must be CXL.mem * capable, there is no point in continuing to enable CXL.mem. */ - hdm_count =FIELD_GET (CXL_DVSEC_HDM_COUNT_MASK , cap); + hdm_count =FIELD_GET (PCI_DVSEC_CXL_HDM_COUNT , cap);if (!hdm_count || hdm_count >2 )return -EINVAL ;@@ -297,11 +297,11 @@ int cxl_dvsec_rr_decode(struct cxl_dev_state *cxlds,* disabled, and they will remain moot after the HDM Decoder * capability is enabled. */ - rc =pci_read_config_word (pdev, d +CXL_DVSEC_CTRL_OFFSET , &ctrl); + rc =pci_read_config_word (pdev, d +PCI_DVSEC_CXL_CTRL , &ctrl);if (rc)return rc; - info->mem_enabled =FIELD_GET (CXL_DVSEC_MEM_ENABLE , ctrl); + info->mem_enabled =FIELD_GET (PCI_DVSEC_CXL_MEM_ENABLE , ctrl);if (!info->mem_enabled)return 0 ;@@ -314,35 +314,35 @@ int cxl_dvsec_rr_decode(struct cxl_dev_state *cxlds,return rc; rc =pci_read_config_dword ( - pdev, d +CXL_DVSEC_RANGE_SIZE_HIGH (i), &temp); + pdev, d +PCI_DVSEC_CXL_RANGE_SIZE_HIGH (i), &temp);if (rc)return rc; size = (u64 )temp <<32 ; rc =pci_read_config_dword ( - pdev, d +CXL_DVSEC_RANGE_SIZE_LOW (i), &temp); + pdev, d +PCI_DVSEC_CXL_RANGE_SIZE_LOW (i), &temp);if (rc)return rc; - size |= temp &CXL_DVSEC_MEM_SIZE_LOW_MASK ; + size |= temp &PCI_DVSEC_CXL_MEM_SIZE_LOW ;if (!size) {continue ; } rc =pci_read_config_dword ( - pdev, d +CXL_DVSEC_RANGE_BASE_HIGH (i), &temp); + pdev, d +PCI_DVSEC_CXL_RANGE_BASE_HIGH (i), &temp);if (rc)return rc; base = (u64 )temp <<32 ; rc =pci_read_config_dword ( - pdev, d +CXL_DVSEC_RANGE_BASE_LOW (i), &temp); + pdev, d +PCI_DVSEC_CXL_RANGE_BASE_LOW (i), &temp);if (rc)return rc; - base |= temp &CXL_DVSEC_MEM_BASE_LOW_MASK ; + base |= temp &PCI_DVSEC_CXL_MEM_BASE_LOW ; info->dvsec_range[ranges++] = (struct range ) { .start = base,@@ -1068,7 +1068,7 @@ u16 cxl_gpf_get_dvsec(struct device *dev) is_port = false; dvsec =pci_find_dvsec_capability (pdev,PCI_VENDOR_ID_CXL , - is_port ?CXL_DVSEC_PORT_GPF :CXL_DVSEC_DEVICE_GPF ); + is_port ?PCI_DVSEC_CXL_PORT_GPF :PCI_DVSEC_CXL_DEVICE_GPF );if (!dvsec)dev_warn (dev,"%s GPF DVSEC not present\n" , is_port ?"Port" :"Device" );@@ -1084,14 +1084,14 @@ static int update_gpf_port_dvsec(struct pci_dev *pdev, int dvsec, int phase)switch (phase) {case 1 : - offset =CXL_DVSEC_PORT_GPF_PHASE_1_CONTROL_OFFSET ; - base =CXL_DVSEC_PORT_GPF_PHASE_1_TMO_BASE_MASK ; - scale =CXL_DVSEC_PORT_GPF_PHASE_1_TMO_SCALE_MASK ; + offset =PCI_DVSEC_CXL_PORT_GPF_PHASE_1_CONTROL ; + base =PCI_DVSEC_CXL_PORT_GPF_PHASE_1_TMO_BASE ; + scale =PCI_DVSEC_CXL_PORT_GPF_PHASE_1_TMO_SCALE ;break ;case 2 : - offset =CXL_DVSEC_PORT_GPF_PHASE_2_CONTROL_OFFSET ; - base =CXL_DVSEC_PORT_GPF_PHASE_2_TMO_BASE_MASK ; - scale =CXL_DVSEC_PORT_GPF_PHASE_2_TMO_SCALE_MASK ; + offset =PCI_DVSEC_CXL_PORT_GPF_PHASE_2_CONTROL ; + base =PCI_DVSEC_CXL_PORT_GPF_PHASE_2_TMO_BASE ; + scale =PCI_DVSEC_CXL_PORT_GPF_PHASE_2_TMO_SCALE ;break ;default :return -EINVAL ;drivers/cxl/core/regs.c +7
-7@@ -271,10 +271,10 @@ EXPORT_SYMBOL_NS_GPL(cxl_map_device_regs, "CXL");static bool cxl_decode_regblock (struct pci_dev *pdev,u32 reg_lo,u32 reg_hi,struct cxl_register_map *map) { -u8 reg_type =FIELD_GET (CXL_DVSEC_REG_LOCATOR_BLOCK_ID_MASK , reg_lo); -int bar =FIELD_GET (CXL_DVSEC_REG_LOCATOR_BIR_MASK , reg_lo); +u8 reg_type =FIELD_GET (PCI_DVSEC_CXL_REG_LOCATOR_BLOCK_ID , reg_lo); +int bar =FIELD_GET (PCI_DVSEC_CXL_REG_LOCATOR_BIR , reg_lo);u64 offset = ((u64 )reg_hi <<32 ) | - (reg_lo &CXL_DVSEC_REG_LOCATOR_BLOCK_OFF_LOW_MASK ); + (reg_lo &PCI_DVSEC_CXL_REG_LOCATOR_BLOCK_OFF_LOW );if (offset >pci_resource_len (pdev, bar)) { dev_warn(&pdev->dev,@@ -311,15 +311,15 @@ static int __cxl_find_regblock_instance(struct pci_dev *pdev, enum cxl_regloc_ty }; regloc =pci_find_dvsec_capability (pdev,PCI_VENDOR_ID_CXL , -CXL_DVSEC_REG_LOCATOR ); +PCI_DVSEC_CXL_REG_LOCATOR );if (!regloc)return -ENXIO ;pci_read_config_dword (pdev, regloc +PCI_DVSEC_HEADER1 , ®loc_size); - regloc_size =FIELD_GET (PCI_DVSEC_HEADER1_LENGTH_MASK , regloc_size); + regloc_size =PCI_DVSEC_HEADER1_LEN (regloc_size); - regloc +=CXL_DVSEC_REG_LOCATOR_BLOCK1_OFFSET ; - regblocks = (regloc_size -CXL_DVSEC_REG_LOCATOR_BLOCK1_OFFSET ) /8 ; + regloc +=PCI_DVSEC_CXL_REG_LOCATOR_BLOCK1 ; + regblocks = (regloc_size -PCI_DVSEC_CXL_REG_LOCATOR_BLOCK1 ) /8 ;for (i =0 ; i < regblocks; i++, regloc +=8 ) {u32 reg_lo, reg_hi;drivers/cxl/pci.c +1
-1@@ -933,7 +933,7 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) cxlds->rcd =is_cxl_restricted (pdev); cxlds->serial =pci_get_dsn (pdev); cxlds->cxl_dvsec =pci_find_dvsec_capability ( - pdev,PCI_VENDOR_ID_CXL ,CXL_DVSEC_PCIE_DEVICE ); + pdev,PCI_VENDOR_ID_CXL ,PCI_DVSEC_CXL_DEVICE );if (!cxlds->cxl_dvsec)dev_warn (&pdev->dev,"Device DVSEC not present, skip CXL.mem init\n" );include/uapi/linux/pci_regs.h +44
-50@@ -1333,63 +1333,57 @@#define PCI_IDE_SEL_ADDR_3 (x) (28 + (x) * PCI_IDE_SEL_ADDR_BLOCK_SIZE)#define PCI_IDE_SEL_BLOCK_SIZE (nr_assoc) (20 + PCI_IDE_SEL_ADDR_BLOCK_SIZE * (nr_assoc)) -/* Compute Express Link (CXL r3.1, sec 8.1.5) */ -#define PCI_DVSEC_CXL_PORT 3 -#define PCI_DVSEC_CXL_PORT_CTL 0x0c -#define PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR 0x00000001 -/* -* Compute Express Link (CXL r3.2, sec 8.1) +* Compute Express Link (CXL r4.0, sec 8.1) * * Note that CXL DVSEC id 3 and 7 to be ignored when the CXL link state -* is "disconnected" (CXL r3.2, sec 9.12.3). Re-enumerate these +* is "disconnected" (CXL r4.0, sec 9.12.3). Re-enumerate these * registers on downstream link-up events. */ -#define PCI_DVSEC_HEADER1_LENGTH_MASK __GENMASK(31, 20) - -/* CXL 3.2 8.1.3: PCIe DVSEC for CXL Device */ -#define CXL_DVSEC_PCIE_DEVICE 0 -#define CXL_DVSEC_CAP_OFFSET 0xA -#define CXL_DVSEC_MEM_CAPABLE _BITUL(2) -#define CXL_DVSEC_HDM_COUNT_MASK __GENMASK(5, 4) -#define CXL_DVSEC_CTRL_OFFSET 0xC -#define CXL_DVSEC_MEM_ENABLE _BITUL(2) -#define CXL_DVSEC_RANGE_SIZE_HIGH (i) (0x18 + (i * 0x10)) -#define CXL_DVSEC_RANGE_SIZE_LOW (i) (0x1C + (i * 0x10)) -#define CXL_DVSEC_MEM_INFO_VALID _BITUL(0) -#define CXL_DVSEC_MEM_ACTIVE _BITUL(1) -#define CXL_DVSEC_MEM_SIZE_LOW_MASK __GENMASK(31, 28) -#define CXL_DVSEC_RANGE_BASE_HIGH (i) (0x20 + (i * 0x10)) -#define CXL_DVSEC_RANGE_BASE_LOW (i) (0x24 + (i * 0x10)) -#define CXL_DVSEC_MEM_BASE_LOW_MASK __GENMASK(31, 28) + +/* CXL r4.0, 8.1.3: PCIe DVSEC for CXL Device */ +#define PCI_DVSEC_CXL_DEVICE 0 +#define PCI_DVSEC_CXL_CAP 0xA +#define PCI_DVSEC_CXL_MEM_CAPABLE _BITUL(2) +#define PCI_DVSEC_CXL_HDM_COUNT __GENMASK(5, 4) +#define PCI_DVSEC_CXL_CTRL 0xC +#define PCI_DVSEC_CXL_MEM_ENABLE _BITUL(2) +#define PCI_DVSEC_CXL_RANGE_SIZE_HIGH (i) (0x18 + (i * 0x10)) +#define PCI_DVSEC_CXL_RANGE_SIZE_LOW (i) (0x1C + (i * 0x10)) +#define PCI_DVSEC_CXL_MEM_INFO_VALID _BITUL(0) +#define PCI_DVSEC_CXL_MEM_ACTIVE _BITUL(1) +#define PCI_DVSEC_CXL_MEM_SIZE_LOW __GENMASK(31, 28) +#define PCI_DVSEC_CXL_RANGE_BASE_HIGH (i) (0x20 + (i * 0x10)) +#define PCI_DVSEC_CXL_RANGE_BASE_LOW (i) (0x24 + (i * 0x10)) +#define PCI_DVSEC_CXL_MEM_BASE_LOW __GENMASK(31, 28)#define CXL_DVSEC_RANGE_MAX 2 -/* CXL 3.2 8.1.4: Non-CXL Function Map DVSEC */ -#define CXL_DVSEC_FUNCTION_MAP 2 - -/* CXL 3.2 8.1.5: Extensions DVSEC for Ports */ -#define CXL_DVSEC_PORT 3 -#define CXL_DVSEC_PORT_CTL 0x0c -#define CXL_DVSEC_PORT_CTL_UNMASK_SBR 0x00000001 - -/* CXL 3.2 8.1.6: GPF DVSEC for CXL Port */ -#define CXL_DVSEC_PORT_GPF 4 -#define CXL_DVSEC_PORT_GPF_PHASE_1_CONTROL_OFFSET 0x0C -#define CXL_DVSEC_PORT_GPF_PHASE_1_TMO_BASE_MASK __GENMASK(3, 0) -#define CXL_DVSEC_PORT_GPF_PHASE_1_TMO_SCALE_MASK __GENMASK(11, 8) -#define CXL_DVSEC_PORT_GPF_PHASE_2_CONTROL_OFFSET 0xE -#define CXL_DVSEC_PORT_GPF_PHASE_2_TMO_BASE_MASK __GENMASK(3, 0) -#define CXL_DVSEC_PORT_GPF_PHASE_2_TMO_SCALE_MASK __GENMASK(11, 8) - -/* CXL 3.2 8.1.7: GPF DVSEC for CXL Device */ -#define CXL_DVSEC_DEVICE_GPF 5 - -/* CXL 3.2 8.1.9: Register Locator DVSEC */ -#define CXL_DVSEC_REG_LOCATOR 8 -#define CXL_DVSEC_REG_LOCATOR_BLOCK1_OFFSET 0xC -#define CXL_DVSEC_REG_LOCATOR_BIR_MASK __GENMASK(2, 0) -#define CXL_DVSEC_REG_LOCATOR_BLOCK_ID_MASK __GENMASK(15, 8) -#define CXL_DVSEC_REG_LOCATOR_BLOCK_OFF_LOW_MASK __GENMASK(31, 16) +/* CXL r4.0, 8.1.4: Non-CXL Function Map DVSEC */ +#define PCI_DVSEC_CXL_FUNCTION_MAP 2 + +/* CXL r4.0, 8.1.5: Extensions DVSEC for Ports */ +#define PCI_DVSEC_CXL_PORT 3 +#define PCI_DVSEC_CXL_PORT_CTL 0x0c +#define PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR 0x00000001 + +/* CXL r4.0, 8.1.6: GPF DVSEC for CXL Port */ +#define PCI_DVSEC_CXL_PORT_GPF 4 +#define PCI_DVSEC_CXL_PORT_GPF_PHASE_1_CONTROL 0x0C +#define PCI_DVSEC_CXL_PORT_GPF_PHASE_1_TMO_BASE __GENMASK(3, 0) +#define PCI_DVSEC_CXL_PORT_GPF_PHASE_1_TMO_SCALE __GENMASK(11, 8) +#define PCI_DVSEC_CXL_PORT_GPF_PHASE_2_CONTROL 0xE +#define PCI_DVSEC_CXL_PORT_GPF_PHASE_2_TMO_BASE __GENMASK(3, 0) +#define PCI_DVSEC_CXL_PORT_GPF_PHASE_2_TMO_SCALE __GENMASK(11, 8) + +/* CXL r4.0, 8.1.7: GPF DVSEC for CXL Device */ +#define PCI_DVSEC_CXL_DEVICE_GPF 5 + +/* CXL r4.0, 8.1.9: Register Locator DVSEC */ +#define PCI_DVSEC_CXL_REG_LOCATOR 8 +#define PCI_DVSEC_CXL_REG_LOCATOR_BLOCK1 0xC +#define PCI_DVSEC_CXL_REG_LOCATOR_BIR __GENMASK(2, 0) +#define PCI_DVSEC_CXL_REG_LOCATOR_BLOCK_ID __GENMASK(15, 8) +#define PCI_DVSEC_CXL_REG_LOCATOR_BLOCK_OFF_LOW __GENMASK(31, 16)#endif /* LINUX_PCI_REGS_H */ Signature · 1 line
2.34.1
PATCH v14 3/34 PCI: Introduce pcie_is_cxl()
In reply to Terry BowmanCXL and AER drivers need the ability to identify CXL devices.
Introduce set_pcie_cxl() with logic checking for CXL.mem or CXL.cache status in the CXL Flex Bus DVSEC status register. The CXL Flex Bus DVSEC presence is used because it is required for all the CXL PCIe devices.[1]
Add boolean 'struct pci_dev::is_cxl' with the purpose to cache the CXL CXL.cache and CXl.mem status.
Call set_pcie_cxl() for the parent bridge. Once a device is created there is a possibility the parent training or CXL state was updated as well. This will make certain the correct parent CXL state is cached.
Add function pcie_is_cxl() to return 'struct pci_dev::is_cxl'.
[1] CXL 3.1 Spec, 8.1.1 PCIe Designated Vendor-Specific Extended Capability (DVSEC) ID Assignment, Table 8-2
---
Changes in v13->v14:
- Move FLEXBUS_STATUS DVSEC here (Jonathan)
- Remove check for EP and USP (Dan)
- Update commit message (Bjorn)
- Fix writing past 80 columns (Bjorn)
- Add pci_is_pcie() parent bridge check at beginning of function (Bjorn)
Changes in v12->v13:
- Add Ben's "reviewed-by"
Changes in v11->v12:
- Add review-by for Alejandro
- Add comment in set_pcie_cxl() explaining why updating parent status.
Changes in v10->v11:
- Amend set_pcie_cxl() to check for Upstream Port's and EP's parent downstream port by calling set_pcie_cxl(). (Dan)
- Retitle patch: 'Add' -> 'Introduce'
- Add check for CXL.mem and CXL.cache (Alejandro, Dan)
--- drivers/pci/probe.c | 31 +++++++++++++++++++++++++++++++ include/linux/pci.h | 6 ++++++ include/uapi/linux/pci_regs.h | 6 ++++++ 3 files changed, 43 insertions(+)
drivers/pci/probe.c +31 -0
@@ -1735,6 +1735,35 @@ static void set_pcie_thunderbolt(struct pci_dev *dev) dev->is_thunderbolt =1 ; } +static void set_pcie_cxl (struct pci_dev *dev) +{ +struct pci_dev *bridge; +u16 dvsec, cap; + +if (!pci_is_pcie (dev)) +return ; + +/* +* Update parent's CXL state because alternate protocol training +* may have changed +*/ + bridge =pci_upstream_bridge (dev); +if (bridge) +set_pcie_cxl (bridge); + + dvsec =pci_find_dvsec_capability (dev,PCI_VENDOR_ID_CXL , +PCI_DVSEC_CXL_FLEXBUS_PORT ); +if (!dvsec) +return ; + +pci_read_config_word (dev, dvsec +PCI_DVSEC_CXL_FLEXBUS_PORT_STATUS , + &cap); + + dev->is_cxl =FIELD_GET (PCI_DVSEC_CXL_FLEXBUS_PORT_STATUS_CACHE , cap) || +FIELD_GET (PCI_DVSEC_CXL_FLEXBUS_PORT_STATUS_MEM , cap); + +} +static void set_pcie_untrusted (struct pci_dev *dev) {struct pci_dev *parent =pci_upstream_bridge (dev);
@@ -2065,6 +2094,8 @@ int pci_setup_device(struct pci_dev *dev)/* Need to have dev->cfg_size ready */ set_pcie_thunderbolt (dev); +set_pcie_cxl (dev); +set_pcie_untrusted (dev);if (pci_is_pcie (dev))
include/linux/pci.h +6 -0
@@ -463,6 +463,7 @@ struct pci_dev { unsigned int is_pciehp:1; unsigned int shpc_managed:1; /* SHPC owned by shpchp */ unsigned int is_thunderbolt:1; /* Thunderbolt controller */ + unsigned int is_cxl:1; /* Compute Express Link (CXL) *//* * Devices marked being untrusted are the ones that can potentially * execute DMA attacks and similar. They are typically connected
@@ -791,6 +792,11 @@ static inline bool pci_is_display(struct pci_dev *pdev)return (pdev->class >>16 ) ==PCI_BASE_CLASS_DISPLAY ; } +static inline bool pcie_is_cxl (struct pci_dev *pci_dev) +{ +return pci_dev->is_cxl; +} +#define for_each_pci_bridge (dev, bus) \ list_for_each_entry(dev, &bus->devices, bus_list) \ if (!pci_is_bridge(dev)) {} else
include/uapi/linux/pci_regs.h +6 -0
@@ -1379,6 +1379,12 @@/* CXL r4.0, 8.1.7: GPF DVSEC for CXL Device */ #define PCI_DVSEC_CXL_DEVICE_GPF 5 +/* CXL r4.0, 8.1.8: Flex Bus DVSEC */ +#define PCI_DVSEC_CXL_FLEXBUS_PORT 7 +#define PCI_DVSEC_CXL_FLEXBUS_PORT_STATUS 0xE +#define PCI_DVSEC_CXL_FLEXBUS_PORT_STATUS_CACHE _BITUL(0) +#define PCI_DVSEC_CXL_FLEXBUS_PORT_STATUS_MEM _BITUL(2) +/* CXL r4.0, 8.1.9: Register Locator DVSEC */ #define PCI_DVSEC_CXL_REG_LOCATOR 8#define PCI_DVSEC_CXL_REG_LOCATOR_BLOCK1 0xC
Signature · 1 line
2.34.1
Re: [PATCH v14 03/34] PCI: Introduce pcie_is_cxl()
Quoted 9 lines · Terry Bowman
CXL and AER drivers need the ability to identify CXL devices.
Introduce
set_pcie_cxl()with logic checking for CXL.mem or CXL.cache status in the CXL Flex Bus DVSEC status register. The CXL Flex Bus DVSEC presence is used because it is required for all the CXL PCIe devices.[1]Add boolean 'struct
pci_dev::is_cxl' with the purpose to cache the CXL CXL.cache and CXl.mem status.
Quoted 8 lines
Call
set_pcie_cxl()for the parent bridge. Once a device is created there is a possibility the parent training or CXL state was updated as well. This will make certain the correct parent CXL state is cached.Add function
pcie_is_cxl()to return 'structpci_dev::is_cxl'.[1] CXL 3.1 Spec, 8.1.1 PCIe Designated Vendor-Specific Extended Capability (DVSEC) ID Assignment, Table 8-2
Hi Bjorn, this is probably the highest impact change to the PCI core to enable the rest of the series that does not yet have your ack.
I think the changelog could use some enhancement to highlight more of the "why" than the "what":
---
CXL is a protocol that runs on top of PCIe electricals. Its error model also runs on top of the PCIe AER error model by standardizing "internal" errors as "CXL" errors. Linux has historically ignored internal errors.
CXL protocol error handling is then a task of enhancing the PCIe AER core to understand that PCIe ports (upstream and downstream) and endpoints may throw internal errors that represent standard CXL protocol errors.
The proposed method to make that determination is to teach 'struct pci_dev' to cache when its link has trained the CXL.mem and/or CXL.cache protocols and then treat all internal errors as CXL errors. A design goal is to not burden the PCIe AER core with CXL knowledge beyond just enough to forward error notifications to the CXL RAS core. The forwarded notification looks up a 'struct cxl_port' or 'struct cxl_dport' companion device to the PCI device.
Introduce set_pcie_cxl() with logic checking for CXL.mem or CXL.cache status in the CXL Flex Bus DVSEC status register. The CXL Flex Bus DVSEC presence is used because it is required for all the CXL PCIe devices.[1]
[1] CXL 3.1 Spec, 8.1.1 PCIe Designated Vendor-Specific Extended Capability (DVSEC) ID Assignment, Table 8-2
---
Re: [PATCH v14 03/34] PCI: Introduce pcie_is_cxl()
In reply to Terry BowmanQuoted 18 lines · Terry Bowman
CXL and AER drivers need the ability to identify CXL devices.
Introduce
set_pcie_cxl()with logic checking for CXL.mem or CXL.cache status in the CXL Flex Bus DVSEC status register. The CXL Flex Bus DVSEC presence is used because it is required for all the CXL PCIe devices.[1]Add boolean 'struct
pci_dev::is_cxl' with the purpose to cache the CXL CXL.cache and CXl.mem status.Call
set_pcie_cxl()for the parent bridge. Once a device is created there is a possibility the parent training or CXL state was updated as well. This will make certain the correct parent CXL state is cached.Add function
pcie_is_cxl()to return 'structpci_dev::is_cxl'.[1] CXL 3.1 Spec, 8.1.1 PCIe Designated Vendor-Specific Extended Capability (DVSEC) ID Assignment, Table 8-2
Quoted 8 lines
Signed-off-by Terry Bowman <[email protected]>Reviewed-by Ira Weiny <[email protected]>Reviewed-by Kuppuswamy Sathyanarayanan <[email protected]>Reviewed-by Dave Jiang <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Alejandro Lucero <[email protected]>Reviewed-by Ben Cheatham <[email protected]>Reviewed-by Dan Williams <[email protected]>
Quoted 120 lines
---
Changes in
v13->v14:
- MoveFLEXBUS_STATUSDVSEC here (Jonathan)
- Remove check for EP and USP (Dan)
- Update commit message (Bjorn)
- Fix writing past 80 columns (Bjorn)
- Addpci_is_pcie()parent bridge check at beginning of function (Bjorn)Changes in
v12->v13:
- Add Ben's "reviewed-by"Changes in
v11->v12:
- Add review-by for Alejandro
- Add comment inset_pcie_cxl()explaining why updating parent status.Changes in
v10->v11:
- Amendset_pcie_cxl()to check for Upstream Port's and EP's parent downstream port by callingset_pcie_cxl(). (Dan)
- Retitle patch: 'Add' -> 'Introduce'
- Add check for CXL.mem and CXL.cache (Alejandro, Dan)--- drivers/pci/probe.c | 31 +++++++++++++++++++++++++++++++ include/linux/pci.h | 6 ++++++ include/uapi/linux/pci_regs.h | 6 ++++++ 3 files changed, 43 insertions(+)drivers/pci/probe.c +31
-0@@ -1735,6 +1735,35 @@ static void set_pcie_thunderbolt(struct pci_dev *dev) dev->is_thunderbolt =1 ; } +static void set_pcie_cxl (struct pci_dev *dev) +{ +struct pci_dev *bridge; +u16 dvsec, cap; + +if (!pci_is_pcie (dev)) +return ; + +/* +* Update parent's CXL state because alternate protocol training +* may have changed +*/ + bridge =pci_upstream_bridge (dev); +if (bridge) +set_pcie_cxl (bridge); + + dvsec =pci_find_dvsec_capability (dev,PCI_VENDOR_ID_CXL , +PCI_DVSEC_CXL_FLEXBUS_PORT ); +if (!dvsec) +return ; + +pci_read_config_word (dev, dvsec +PCI_DVSEC_CXL_FLEXBUS_PORT_STATUS , + &cap); + + dev->is_cxl =FIELD_GET (PCI_DVSEC_CXL_FLEXBUS_PORT_STATUS_CACHE , cap) || +FIELD_GET (PCI_DVSEC_CXL_FLEXBUS_PORT_STATUS_MEM , cap); + +} +static void set_pcie_untrusted (struct pci_dev *dev) {struct pci_dev *parent =pci_upstream_bridge (dev);@@ -2065,6 +2094,8 @@ int pci_setup_device(struct pci_dev *dev)/* Need to have dev->cfg_size ready */ set_pcie_thunderbolt (dev); +set_pcie_cxl (dev); +set_pcie_untrusted (dev);if (pci_is_pcie (dev))include/linux/pci.h +6
-0@@ -463,6 +463,7 @@ struct pci_dev { unsigned int is_pciehp:1; unsigned int shpc_managed:1; /* SHPC owned by shpchp */ unsigned int is_thunderbolt:1; /* Thunderbolt controller */ + unsigned int is_cxl:1; /* Compute Express Link (CXL) *//* * Devices marked being untrusted are the ones that can potentially * execute DMA attacks and similar. They are typically connected @@ -791,6 +792,11 @@ static inline bool pci_is_display(struct pci_dev *pdev)return (pdev->class >>16 ) ==PCI_BASE_CLASS_DISPLAY ; } +static inline bool pcie_is_cxl (struct pci_dev *pci_dev) +{ +return pci_dev->is_cxl; +} +#define for_each_pci_bridge (dev, bus) \ list_for_each_entry(dev, &bus->devices, bus_list) \ if (!pci_is_bridge(dev)) {} elseinclude/uapi/linux/pci_regs.h +6
-0@@ -1379,6 +1379,12 @@/* CXL r4.0, 8.1.7: GPF DVSEC for CXL Device */ #define PCI_DVSEC_CXL_DEVICE_GPF 5 +/* CXL r4.0, 8.1.8: Flex Bus DVSEC */ +#define PCI_DVSEC_CXL_FLEXBUS_PORT 7 +#define PCI_DVSEC_CXL_FLEXBUS_PORT_STATUS 0xE +#define PCI_DVSEC_CXL_FLEXBUS_PORT_STATUS_CACHE _BITUL(0) +#define PCI_DVSEC_CXL_FLEXBUS_PORT_STATUS_MEM _BITUL(2) +/* CXL r4.0, 8.1.9: Register Locator DVSEC */ #define PCI_DVSEC_CXL_REG_LOCATOR 8#define PCI_DVSEC_CXL_REG_LOCATOR_BLOCK1 0xCSignature · 1 line
2.34.1
PATCH v14 4/34 cxl/pci: Remove unnecessary CXL Endpoint handling helper functions
In reply to Terry BowmanThe CXL driver's cxl_handle_endpoint_cor_ras()/cxl_handle_endpoint_ras() are unnecessary helper functions used only for Endpoints. Remove these functions as they are not common for all CXL devices and do not provide value for EP handling.
Rename __cxl_handle_ras to cxl_handle_ras() and __cxl_handle_cor_ras() to cxl_handle_cor_ras().
---
Changes in v13->v14:
- None
Changes in v12->v13:
- None
Changes in v11->v12:
- Added Dave Jiang's review by
- Moved to front of series
Changes in v10->v11:
- None
--- drivers/cxl/core/pci.c | 26 ++++++++------------------ 1 file changed, 8 insertions(+), 18 deletions(-)
drivers/cxl/core/pci.c +8 -18
@@ -632,8 +632,8 @@ void read_cdat_data(struct cxl_port *port) }EXPORT_SYMBOL_NS_GPL (read_cdat_data,"CXL" ); -static void __cxl_handle_cor_ras (struct cxl_dev_state *cxlds, -void __iomem *ras_base) +static void cxl_handle_cor_ras (struct cxl_dev_state *cxlds, +void __iomem *ras_base) {void __iomem *addr;u32 status;
@@ -649,11 +649,6 @@ static void __cxl_handle_cor_ras(struct cxl_dev_state *cxlds, } } -static void cxl_handle_endpoint_cor_ras (struct cxl_dev_state *cxlds) -{ -return __cxl_handle_cor_ras (cxlds, cxlds->regs.ras); -} -/* CXL spec rev3.0 8.2.4.16.1 */ static void header_log_copy (void __iomem *ras_base,u32 *log) {
@@ -675,8 +670,8 @@ static void header_log_copy(void __iomem *ras_base, u32 *log)* Log the state of the RAS status registers and prepare them to log the * next error status. Return 1 if reset needed. */ -static bool __cxl_handle_ras (struct cxl_dev_state *cxlds, -void __iomem *ras_base) +static bool cxl_handle_ras (struct cxl_dev_state *cxlds, +void __iomem *ras_base) {u32 hl[CXL_HEADERLOG_SIZE_U32 ];void __iomem *addr;
@@ -709,11 +704,6 @@ static bool __cxl_handle_ras(struct cxl_dev_state *cxlds,return true; } -static bool cxl_handle_endpoint_ras (struct cxl_dev_state *cxlds) -{ -return __cxl_handle_ras (cxlds, cxlds->regs.ras); -} -#ifdef CONFIG_PCIEAER_CXL static void cxl_dport_map_rch_aer (struct cxl_dport *dport)
@@ -792,13 +782,13 @@ EXPORT_SYMBOL_NS_GPL(cxl_dport_init_ras_reporting, "CXL");static void cxl_handle_rdport_cor_ras (struct cxl_dev_state *cxlds,struct cxl_dport *dport) { -return __cxl_handle_cor_ras (cxlds, dport->regs.ras); +return cxl_handle_cor_ras (cxlds, dport->regs.ras); }static bool cxl_handle_rdport_ras (struct cxl_dev_state *cxlds,struct cxl_dport *dport) { -return __cxl_handle_ras (cxlds, dport->regs.ras); +return cxl_handle_ras (cxlds, dport->regs.ras); }/*
@@ -895,7 +885,7 @@ void cxl_cor_error_detected(struct pci_dev *pdev)if (cxlds->rcd)cxl_handle_rdport_errors (cxlds); -cxl_handle_endpoint_cor_ras (cxlds); +cxl_handle_cor_ras (cxlds, cxlds->regs.ras); } }EXPORT_SYMBOL_NS_GPL (cxl_cor_error_detected,"CXL" );
@@ -924,7 +914,7 @@ pci_ers_result_t cxl_error_detected(struct pci_dev *pdev,* chance the situation is recoverable dump the status of the RAS * capability registers and bounce the active state of the memdev. */ - ue =cxl_handle_endpoint_ras (cxlds); + ue =cxl_handle_ras (cxlds, cxlds->regs.ras); }
Signature · 1 line
2.34.1
PATCH v14 5/34 cxl/pci: Remove unnecessary CXL RCH handling helper functions
In reply to Terry Bowmancxl_handle_rdport_cor_ras() and cxl_handle_rdport_ras() are specific to Restricted CXL Host (RCH) handling. Improve readability and maintainability by replacing these and instead using the common cxl_handle_cor_ras() and cxl_handle_ras() functions.
---
Changes in v13->v14:
- None
Changes in v12->v13:
- None
Changes in v11->v12:
- Add reviewed-by for Alejandro & Dave Jiang
- Moved to front of series
Changes in v10->v11:
- New patch
--- drivers/cxl/core/pci.c | 16 ++-------------- 1 file changed, 2 insertions(+), 14 deletions(-)
drivers/cxl/core/pci.c +2 -14
@@ -779,18 +779,6 @@ void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host) }EXPORT_SYMBOL_NS_GPL (cxl_dport_init_ras_reporting,"CXL" ); -static void cxl_handle_rdport_cor_ras (struct cxl_dev_state *cxlds, -struct cxl_dport *dport) -{ -return cxl_handle_cor_ras (cxlds, dport->regs.ras); -} - -static bool cxl_handle_rdport_ras (struct cxl_dev_state *cxlds, -struct cxl_dport *dport) -{ -return cxl_handle_ras (cxlds, dport->regs.ras); -} -/* * Copy the AER capability registers using 32 bit read accesses. * This is necessary because RCRB AER capability is MMIO mapped. Clear the
@@ -860,9 +848,9 @@ static void cxl_handle_rdport_errors(struct cxl_dev_state *cxlds)pci_print_aer (pdev, severity, &aer_regs);if (severity ==AER_CORRECTABLE ) -cxl_handle_rdport_cor_ras (cxlds, dport); +cxl_handle_cor_ras (cxlds, dport->regs.ras);else - cxl_handle_rdport_ras(cxlds, dport); + cxl_handle_ras(cxlds, dport->regs.ras); }#else
Signature · 1 line
2.34.1
PATCH v14 6/34 PCI: Replace cxl_error_is_native() with pcie_aer_is_native()
In reply to Terry BowmanThe AER driver includes a CXL support function cxl_error_is_native(). This function adds no additional value from pcie_aer_is_native().
Simplify the codebase by removing cxl_error_is_native() and replace occurrences of cxl_error_is_native() with pcie_aer_is_native().
---
Changes in v13->v14:
- New commit (Dan)
--- drivers/pci/pcie/aer.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-)
drivers/pci/pcie/aer.c +2 -9
@@ -1166,13 +1166,6 @@ static bool is_cxl_mem_dev(struct pci_dev *dev)return true; } -static bool cxl_error_is_native (struct pci_dev *dev) -{ -struct pci_host_bridge *host =pci_find_host_bridge (dev->bus); - -return (pcie_ports_native || host->native_aer); -} -static bool is_internal_error (struct aer_err_info *info) {if (info->severity ==AER_CORRECTABLE )
@@ -1186,7 +1179,7 @@ static int cxl_rch_handle_error_iter(struct pci_dev *dev, void *data)struct aer_err_info *info = (struct aer_err_info *)data;const struct pci_error_handlers *err_handler; -if (!is_cxl_mem_dev (dev) || !cxl_error_is_native (dev)) +if (!is_cxl_mem_dev (dev) || !pcie_aer_is_native (dev))return 0 ;/* Protect dev->driver */
@@ -1227,7 +1220,7 @@ static int handles_cxl_error_iter(struct pci_dev *dev, void *data)bool *handles_cxl = data;if (!*handles_cxl) - *handles_cxl =is_cxl_mem_dev (dev) &&cxl_error_is_native (dev); + *handles_cxl =is_cxl_mem_dev (dev) &&pcie_aer_is_native (dev);/* Non-zero terminates iteration */ return *handles_cxl;
Signature · 1 line
2.34.1
Re: [PATCH v14 06/34] PCI: Replace cxl_error_is_native() with pcie_aer_is_native()
Quoted 8 lines · On Wed, 14 Jan 2026 12:20:27 -0600 Terry Bowman <[email protected]> wrote:
The AER driver includes a CXL support function
cxl_error_is_native(). This function adds no additional value frompcie_aer_is_native().Simplify the codebase by removing
cxl_error_is_native()and replace occurrences ofcxl_error_is_native()withpcie_aer_is_native().Signed-off-by Terry Bowman <[email protected]>Reviewed-by Dan Williams <[email protected]>
Dave, if for any reason the rest gets delayed, nice if we can pick this one up anyway.
Dave Jiang
maintainerRe: [PATCH v14 06/34] PCI: Replace cxl_error_is_native() with pcie_aer_is_native()
Quoted 9 lines · Jonathan Cameron
Quoted 6 lines · On Wed, 14 Jan 2026 12:20:27 -0600 Terry Bowman <[email protected]> wrote:
The AER driver includes a CXL support function
cxl_error_is_native(). This function adds no additional value frompcie_aer_is_native().Simplify the codebase by removing
cxl_error_is_native()and replace occurrences ofcxl_error_is_native()withpcie_aer_is_native().
Quoted 6 lines
Quoted 2 lines
Signed-off-by Terry Bowman <[email protected]>Reviewed-by Dan Williams <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Dave, if for any reason the rest gets delayed, nice if we can pick this one up anyway.
Needs an Ack from Bjorn one way or another.
Re: [PATCH v14 06/34] PCI: Replace cxl_error_is_native() with pcie_aer_is_native()
In reply to Terry BowmanQuoted 8 lines · Terry Bowman
The AER driver includes a CXL support function
cxl_error_is_native(). This function adds no additional value frompcie_aer_is_native().Simplify the codebase by removing
cxl_error_is_native()and replace occurrences ofcxl_error_is_native()withpcie_aer_is_native().Signed-off-by Terry Bowman <[email protected]>Reviewed-by Dan Williams <[email protected]>
Quoted 45 lines
---
Changes in
v13->v14:
- New commit (Dan)--- drivers/pci/pcie/aer.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-)drivers/pci/pcie/aer.c +2
-9@@ -1166,13 +1166,6 @@ static bool is_cxl_mem_dev(struct pci_dev *dev)return true; } -static bool cxl_error_is_native (struct pci_dev *dev) -{ -struct pci_host_bridge *host =pci_find_host_bridge (dev->bus); - -return (pcie_ports_native || host->native_aer); -} -static bool is_internal_error (struct aer_err_info *info) {if (info->severity ==AER_CORRECTABLE )@@ -1186,7 +1179,7 @@ static int cxl_rch_handle_error_iter(struct pci_dev *dev, void *data)struct aer_err_info *info = (struct aer_err_info *)data;const struct pci_error_handlers *err_handler; -if (!is_cxl_mem_dev (dev) || !cxl_error_is_native (dev)) +if (!is_cxl_mem_dev (dev) || !pcie_aer_is_native (dev))return 0 ;/* Protect dev->driver */ @@ -1227,7 +1220,7 @@ static int handles_cxl_error_iter(struct pci_dev *dev, void *data)bool *handles_cxl = data;if (!*handles_cxl) - *handles_cxl =is_cxl_mem_dev (dev) &&cxl_error_is_native (dev); + *handles_cxl =is_cxl_mem_dev (dev) &&pcie_aer_is_native (dev);/* Non-zero terminates iteration */ return *handles_cxl;
Re: [PATCH v14 06/34] PCI: Replace cxl_error_is_native() with pcie_aer_is_native()
In reply to Terry BowmanQuoted 8 lines · Terry Bowman
The AER driver includes a CXL support function
cxl_error_is_native(). This function adds no additional value frompcie_aer_is_native().Simplify the codebase by removing
cxl_error_is_native()and replace occurrences ofcxl_error_is_native()withpcie_aer_is_native().Signed-off-by Terry Bowman <[email protected]>Reviewed-by Dan Williams <[email protected]>
Quoted 47 lines
---
Changes in
v13->v14:
- New commit (Dan)--- drivers/pci/pcie/aer.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-)drivers/pci/pcie/aer.c +2
-9@@ -1166,13 +1166,6 @@ static bool is_cxl_mem_dev(struct pci_dev *dev)return true; } -static bool cxl_error_is_native (struct pci_dev *dev) -{ -struct pci_host_bridge *host =pci_find_host_bridge (dev->bus); - -return (pcie_ports_native || host->native_aer); -} -static bool is_internal_error (struct aer_err_info *info) {if (info->severity ==AER_CORRECTABLE )@@ -1186,7 +1179,7 @@ static int cxl_rch_handle_error_iter(struct pci_dev *dev, void *data)struct aer_err_info *info = (struct aer_err_info *)data;const struct pci_error_handlers *err_handler; -if (!is_cxl_mem_dev (dev) || !cxl_error_is_native (dev)) +if (!is_cxl_mem_dev (dev) || !pcie_aer_is_native (dev))return 0 ;/* Protect dev->driver */ @@ -1227,7 +1220,7 @@ static int handles_cxl_error_iter(struct pci_dev *dev, void *data)bool *handles_cxl = data;if (!*handles_cxl) - *handles_cxl =is_cxl_mem_dev (dev) &&cxl_error_is_native (dev); + *handles_cxl =is_cxl_mem_dev (dev) &&pcie_aer_is_native (dev);/* Non-zero terminates iteration */ return *handles_cxl;Signature · 1 line
2.34.1
PATCH v14 7/34 cxl/pci: Remove CXL VH handling in CONFIG_PCIEAER_CXL conditional blocks from core/pci.c
In reply to Terry BowmanFrom: Dave Jiang <[email protected]>
Create new config CONFIG_CXL_RAS and put all CXL RAS items behind the config. The config will depend on CPER and PCIE AER to build. Move the related VH RAS code from core/pci.c to core/ras.c.
Restricted CXL host (RCH) RAS functions will be moved in a future patch.
---
Changes in v13->v14:
- None
Changes in v12->v13:
- None
Changes in v11->v12:
- None
Changes in v10->v11:
- New patch
- Updated by Terry Bowman to use (ACPI_APEI_GHES && PCIEAER_CXL) dependency in Kconfig. Otherwise checks will be reauired for CONFIG_PCIEAER because AER driver functions are called.
--- drivers/cxl/Kconfig | 4 + drivers/cxl/core/Makefile | 2 +- drivers/cxl/core/core.h | 31 +++++++ drivers/cxl/core/pci.c | 189 +------------------------------------- drivers/cxl/core/ras.c | 176 +++++++++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 8 -- drivers/cxl/cxlpci.h | 16 ++++ tools/testing/cxl/Kbuild | 2 +- 8 files changed, 233 insertions(+), 195 deletions(-)
drivers/cxl/Kconfig +4 -0
@@ -233,4 +233,8 @@ config CXL_MCE def_bool y depends on X86_MCE && MEMORY_FAILURE +config CXL_RAS + def_bool y + depends on ACPI_APEI_GHES && PCIEAER && CXL_PCI + endif
drivers/cxl/core/Makefile +1 -1
@@ -14,9 +14,9 @@ cxl_core-y += pci.o cxl_core-y += hdm.o cxl_core-y += pmu.o cxl_core-y += cdat.o -cxl_core-y += ras.o cxl_core-$(CONFIG_TRACING) += trace.o cxl_core-$(CONFIG_CXL_REGION) += region.o cxl_core-$(CONFIG_CXL_MCE) += mce.o cxl_core-$(CONFIG_CXL_FEATURES) += features.o cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += edac.o +cxl_core-$(CONFIG_CXL_RAS) += ras.o
drivers/cxl/core/core.h +31 -0
@@ -144,8 +144,39 @@ int cxl_pci_get_bandwidth(struct pci_dev *pdev, struct access_coordinate *c);int cxl_port_get_switch_dport_bandwidth (struct cxl_port *port,struct access_coordinate *c); +#ifdef CONFIG_CXL_RAS int cxl_ras_init (void );void cxl_ras_exit (void ); +bool cxl_handle_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base); +void cxl_handle_cor_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base); +#else +static inline int cxl_ras_init (void ) +{ +return 0 ; +} + +static inline void cxl_ras_exit (void ) +{ +} + +static inline bool cxl_handle_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base) +{ +return false; +} +static inline void cxl_handle_cor_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base) { } +#endif /* CONFIG_CXL_RAS */ + +/* Restricted CXL Host specific RAS functions */ +#ifdef CONFIG_CXL_RAS +void cxl_dport_map_rch_aer (struct cxl_dport *dport); +void cxl_disable_rch_root_ints (struct cxl_dport *dport); +void cxl_handle_rdport_errors (struct cxl_dev_state *cxlds); +#else +static inline void cxl_dport_map_rch_aer (struct cxl_dport *dport) { } +static inline void cxl_disable_rch_root_ints (struct cxl_dport *dport) { } +static inline void cxl_handle_rdport_errors (struct cxl_dev_state *cxlds) { } +#endif /* CONFIG_CXL_RAS */ +int cxl_gpf_port_setup (struct cxl_dport *dport);struct cxl_hdm ;
drivers/cxl/core/pci.c +4 -185
221 lines not on this page · raw message
drivers/cxl/core/ras.c +176 -0
185 lines not on this page · raw message
drivers/cxl/cxl.h +0 -8
@@ -803,14 +803,6 @@ struct cxl_dport *devm_cxl_add_rch_dport(struct cxl_port *port, struct device *dport_dev, int port_id, resource_size_t rcrb); -#ifdef CONFIG_PCIEAER_CXL -void cxl_setup_parent_dport (struct device *host,struct cxl_dport *dport); -void cxl_dport_init_ras_reporting (struct cxl_dport *dport,struct device *host); -#else -static inline void cxl_dport_init_ras_reporting (struct cxl_dport *dport, -struct device *host) { } -#endif -struct cxl_decoder *to_cxl_decoder (struct device *dev);struct cxl_root_decoder *to_cxl_root_decoder (struct device *dev);struct cxl_switch_decoder *to_cxl_switch_decoder (struct device *dev);
drivers/cxl/cxlpci.h +16 -0
@@ -76,7 +76,23 @@ static inline bool cxl_pci_flit_256(struct pci_dev *pdev)struct cxl_dev_state ;void read_cdat_data (struct cxl_port *port); + +#ifdef CONFIG_CXL_RAS void cxl_cor_error_detected (struct pci_dev *pdev);pci_ers_result_t cxl_error_detected (struct pci_dev *pdev,pci_channel_state_t state); +void cxl_dport_init_ras_reporting (struct cxl_dport *dport,struct device *host); +#else +static inline void cxl_cor_error_detected (struct pci_dev *pdev) { } + +static inline pci_ers_result_t cxl_error_detected (struct pci_dev *pdev, +pci_channel_state_t state) +{ +return PCI_ERS_RESULT_NONE ; +} + +static inline void cxl_dport_init_ras_reporting (struct cxl_dport *dport, +struct device *host) { } +#endif +#endif /* __CXL_PCI_H__ */
tools/testing/cxl/Kbuild +1 -1
@@ -57,12 +57,12 @@ cxl_core-y += $(CXL_CORE_SRC)/pci.o cxl_core-y += $(CXL_CORE_SRC)/hdm.o cxl_core-y += $(CXL_CORE_SRC)/pmu.o cxl_core-y += $(CXL_CORE_SRC)/cdat.o -cxl_core-y += $(CXL_CORE_SRC)/ras.o cxl_core-$(CONFIG_TRACING) += $(CXL_CORE_SRC)/trace.o cxl_core-$(CONFIG_CXL_REGION) += $(CXL_CORE_SRC)/region.o cxl_core-$(CONFIG_CXL_MCE) += $(CXL_CORE_SRC)/mce.o cxl_core-$(CONFIG_CXL_FEATURES) += $(CXL_CORE_SRC)/features.o cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += $(CXL_CORE_SRC)/edac.o +cxl_core-$(CONFIG_CXL_RAS) += $(CXL_CORE_SRC)/ras.o cxl_core-y += config_check.o cxl_core-y += cxl_core_test.o cxl_core-y += cxl_core_exports.o
Signature · 1 line
2.34.1
Dave Jiang
maintainerRe: [PATCH v14 07/34] cxl/pci: Remove CXL VH handling in CONFIG_PCIEAER_CXL conditional blocks from core/pci.c
Quoted 8 lines · Terry Bowman
From: Dave Jiang <[email protected]>
Create new config
CONFIG_CXL_RASand put all CXL RAS items behind the config. The config will depend on CPER and PCIE AER to build. Move the related VH RAS code fromcore/pci.ctocore/ras.c.Restricted CXL host (RCH) RAS functions will be moved in a future patch.
Quoted 8 lines
Cc Robert Richter <[email protected]>Cc Terry Bowman <[email protected]>Reviewed-by Joshua Hahn <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Signed-off-by Dave Jiang <[email protected]>Reviewed-by Alison Schofield <[email protected]>Co-developed-by Terry Bowman <[email protected]>Reviewed-by Dan Williams <[email protected]>
Terry, missing your sign off as well.
Quoted 586 lines
---
Changes in
v13->v14:
- NoneChanges in
v12->v13:
- NoneChanges in
v11->v12:
- NoneChanges in
v10->v11:
- New patch
- Updated by Terry Bowman to use (ACPI_APEI_GHES&&PCIEAER_CXL) dependency inKconfig. Otherwise checks will be reauired forCONFIG_PCIEAERbecause AER driver functions are called.--- drivers/cxl/Kconfig | 4 + drivers/cxl/core/Makefile | 2 +- drivers/cxl/core/core.h | 31 +++++++ drivers/cxl/core/pci.c | 189 +------------------------------------- drivers/cxl/core/ras.c | 176 +++++++++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 8 -- drivers/cxl/cxlpci.h | 16 ++++ tools/testing/cxl/Kbuild | 2 +- 8 files changed, 233 insertions(+), 195 deletions(-)drivers/cxl/Kconfig +4
-0@@ -233,4 +233,8 @@ config CXL_MCE def_bool y depends on X86_MCE && MEMORY_FAILURE +config CXL_RAS + def_bool y + depends on ACPI_APEI_GHES && PCIEAER && CXL_PCI + endifdrivers/cxl/core/Makefile +1
-1@@ -14,9 +14,9 @@ cxl_core-y += pci.o cxl_core-y += hdm.o cxl_core-y += pmu.o cxl_core-y += cdat.o -cxl_core-y += ras.o cxl_core-$(CONFIG_TRACING) += trace.o cxl_core-$(CONFIG_CXL_REGION) += region.o cxl_core-$(CONFIG_CXL_MCE) += mce.o cxl_core-$(CONFIG_CXL_FEATURES) += features.o cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += edac.o +cxl_core-$(CONFIG_CXL_RAS) += ras.odrivers/cxl/core/core.h +31
-0@@ -144,8 +144,39 @@ int cxl_pci_get_bandwidth(struct pci_dev *pdev, struct access_coordinate *c);int cxl_port_get_switch_dport_bandwidth (struct cxl_port *port,struct access_coordinate *c); +#ifdef CONFIG_CXL_RAS int cxl_ras_init (void );void cxl_ras_exit (void ); +bool cxl_handle_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base); +void cxl_handle_cor_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base); +#else +static inline int cxl_ras_init (void ) +{ +return 0 ; +} + +static inline void cxl_ras_exit (void ) +{ +} + +static inline bool cxl_handle_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base) +{ +return false; +} +static inline void cxl_handle_cor_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base) { } +#endif /* CONFIG_CXL_RAS */ + +/* Restricted CXL Host specific RAS functions */ +#ifdef CONFIG_CXL_RAS +void cxl_dport_map_rch_aer (struct cxl_dport *dport); +void cxl_disable_rch_root_ints (struct cxl_dport *dport); +void cxl_handle_rdport_errors (struct cxl_dev_state *cxlds); +#else +static inline void cxl_dport_map_rch_aer (struct cxl_dport *dport) { } +static inline void cxl_disable_rch_root_ints (struct cxl_dport *dport) { } +static inline void cxl_handle_rdport_errors (struct cxl_dev_state *cxlds) { } +#endif /* CONFIG_CXL_RAS */ +int cxl_gpf_port_setup (struct cxl_dport *dport);struct cxl_hdm ;drivers/cxl/core/pci.c +4
-185@@ -632,81 +632,8 @@ void read_cdat_data(struct cxl_port *port) }EXPORT_SYMBOL_NS_GPL (read_cdat_data,"CXL" ); -static void cxl_handle_cor_ras (struct cxl_dev_state *cxlds, -void __iomem *ras_base) -{ -void __iomem *addr; -u32 status; - -if (!ras_base) -return ; - - addr = ras_base +CXL_RAS_CORRECTABLE_STATUS_OFFSET ; - status =readl (addr); -if (status &CXL_RAS_CORRECTABLE_STATUS_MASK ) { -writel (status &CXL_RAS_CORRECTABLE_STATUS_MASK , addr); -trace_cxl_aer_correctable_error (cxlds->cxlmd, status); - } -} - -/* CXL spec rev3.0 8.2.4.16.1 */ -static void header_log_copy (void __iomem *ras_base,u32 *log) -{ -void __iomem *addr; -u32 *log_addr; -int i, log_u32_size =CXL_HEADERLOG_SIZE /sizeof (u32); - - addr = ras_base +CXL_RAS_HEADER_LOG_OFFSET ; - log_addr = log; - -for (i =0 ; i < log_u32_size; i++) { - *log_addr =readl (addr); - log_addr++; - addr +=sizeof (u32); - } -} - -/* -* Log the state of the RAS status registers and prepare them to log the -* next error status. Return 1 if reset needed. -*/ -static bool cxl_handle_ras (struct cxl_dev_state *cxlds, -void __iomem *ras_base) -{ -u32 hl[CXL_HEADERLOG_SIZE_U32 ]; -void __iomem *addr; -u32 status; -u32 fe; - -if (!ras_base) -return false; - - addr = ras_base +CXL_RAS_UNCORRECTABLE_STATUS_OFFSET ; - status =readl (addr); -if (!(status &CXL_RAS_UNCORRECTABLE_STATUS_MASK )) -return false; - -/* If multiple errors, log header points to first error from ctrl reg */ -if (hweight32 (status) >1 ) { -void __iomem *rcc_addr = - ras_base +CXL_RAS_CAP_CONTROL_OFFSET ; - - fe =BIT (FIELD_GET (CXL_RAS_CAP_CONTROL_FE_MASK , -readl (rcc_addr))); - }else { - fe = status; - } - -header_log_copy (ras_base, hl); -trace_cxl_aer_uncorrectable_error (cxlds->cxlmd, status, fe, hl); -writel (status &CXL_RAS_UNCORRECTABLE_STATUS_MASK , addr); - -return true; -} - -#ifdef CONFIG_PCIEAER_CXL - -static void cxl_dport_map_rch_aer (struct cxl_dport *dport) +#ifdef CONFIG_CXL_RAS +void cxl_dport_map_rch_aer (struct cxl_dport *dport) {resource_size_t aer_phys;struct device *host;@@ -721,19 +648,7 @@ static void cxl_dport_map_rch_aer(struct cxl_dport *dport) } } -static void cxl_dport_map_ras (struct cxl_dport *dport) -{ -struct cxl_register_map *map = &dport->reg_map; -struct device *dev = dport->dport_dev; - -if (!map->component_map.ras.valid) -dev_dbg (dev,"RAS registers not found\n" ); -else if (cxl_map_component_regs (map, &dport->regs.component, -BIT (CXL_CM_CAP_CAP_ID_RAS ))) -dev_dbg (dev,"Failed to map RAS capability.\n" ); -} - -static void cxl_disable_rch_root_ints (struct cxl_dport *dport) +void cxl_disable_rch_root_ints (struct cxl_dport *dport) {void __iomem *aer_base = dport->regs.dport_aer;u32 aer_cmd_mask, aer_cmd;@@ -757,28 +672,6 @@ static void cxl_disable_rch_root_ints(struct cxl_dport *dport)writel (aer_cmd, aer_base +PCI_ERR_ROOT_COMMAND ); } -/** -* cxl_dport_init_ras_reporting - Setup CXL RAS report on this dport -* @dport: the cxl_dport that needs to be initialized -* @host: host device for devm operations -*/ -void cxl_dport_init_ras_reporting (struct cxl_dport *dport,struct device *host) -{ - dport->reg_map.host = host; -cxl_dport_map_ras (dport); - -if (dport->rch) { -struct pci_host_bridge *host_bridge =to_pci_host_bridge (dport->dport_dev); - -if (!host_bridge->native_aer) -return ; - -cxl_dport_map_rch_aer (dport); -cxl_disable_rch_root_ints (dport); - } -} -EXPORT_SYMBOL_NS_GPL (cxl_dport_init_ras_reporting,"CXL" ); -/* * Copy the AER capability registers using 32 bit read accesses. * This is necessary because RCRB AER capability is MMIO mapped. Clear the @@ -827,7 +720,7 @@ static bool cxl_rch_get_aer_severity(struct aer_capability_regs *aer_regs,return false; } -static void cxl_handle_rdport_errors (struct cxl_dev_state *cxlds) +void cxl_handle_rdport_errors (struct cxl_dev_state *cxlds) {struct pci_dev *pdev =to_pci_dev (cxlds->dev);struct aer_capability_regs aer_regs;@@ -852,82 +745,8 @@ static void cxl_handle_rdport_errors(struct cxl_dev_state *cxlds) else cxl_handle_ras(cxlds, dport->regs.ras); } - -#else -static void cxl_handle_rdport_errors (struct cxl_dev_state *cxlds) { }#endif -void cxl_cor_error_detected (struct pci_dev *pdev) -{ -struct cxl_dev_state *cxlds =pci_get_drvdata (pdev); -struct device *dev = &cxlds->cxlmd->dev; - - scoped_guard(device, dev) { -if (!dev->driver) { -dev_warn (&pdev->dev, -"%s: memdev disabled, abort error handling\n" , -dev_name (dev)); -return ; - } - -if (cxlds->rcd) -cxl_handle_rdport_errors (cxlds); - -cxl_handle_cor_ras (cxlds, cxlds->regs.ras); - } -} -EXPORT_SYMBOL_NS_GPL (cxl_cor_error_detected,"CXL" ); - -pci_ers_result_t cxl_error_detected (struct pci_dev *pdev, -pci_channel_state_t state) -{ -struct cxl_dev_state *cxlds =pci_get_drvdata (pdev); -struct cxl_memdev *cxlmd = cxlds->cxlmd; -struct device *dev = &cxlmd->dev; -bool ue; - - scoped_guard(device, dev) { -if (!dev->driver) { -dev_warn (&pdev->dev, -"%s: memdev disabled, abort error handling\n" , -dev_name (dev)); -return PCI_ERS_RESULT_DISCONNECT ; - } - -if (cxlds->rcd) -cxl_handle_rdport_errors (cxlds); -/* -* A frozen channel indicates an impending reset which is fatal to -* CXL.mem operation, and will likely crash the system. On the off -* chance the situation is recoverable dump the status of the RAS -* capability registers and bounce the active state of the memdev. -*/ - ue =cxl_handle_ras (cxlds, cxlds->regs.ras); - } - - -switch (state) { -case pci_channel_io_normal: -if (ue) { -device_release_driver (dev); -return PCI_ERS_RESULT_NEED_RESET ; - } -return PCI_ERS_RESULT_CAN_RECOVER ; -case pci_channel_io_frozen: -dev_warn (&pdev->dev, -"%s: frozen state error detected, disable CXL.mem\n" , -dev_name (dev)); -device_release_driver (dev); -return PCI_ERS_RESULT_NEED_RESET ; -case pci_channel_io_perm_failure: -dev_warn (&pdev->dev, -"failure state error detected, request disconnect\n" ); -return PCI_ERS_RESULT_DISCONNECT ; - } -return PCI_ERS_RESULT_NEED_RESET ; -} -EXPORT_SYMBOL_NS_GPL (cxl_error_detected,"CXL" ); -static int cxl_flit_size (struct pci_dev *pdev) {if (cxl_pci_flit_256 (pdev))drivers/cxl/core/ras.c +176
-0@@ -5,6 +5,7 @@#include <linux/aer.h> #include <cxl/event.h> #include <cxlmem.h> +#include <cxlpci.h> #include "trace.h" static void cxl_cper_trace_corr_port_prot_err(struct pci_dev *pdev,@@ -124,3 +125,178 @@ void cxl_ras_exit(void)cxl_cper_unregister_prot_err_work (&cxl_cper_prot_err_work);cancel_work_sync (&cxl_cper_prot_err_work); } + +static void cxl_dport_map_ras (struct cxl_dport *dport) +{ +struct cxl_register_map *map = &dport->reg_map; +struct device *dev = dport->dport_dev; + +if (!map->component_map.ras.valid) +dev_dbg (dev,"RAS registers not found\n" ); +else if (cxl_map_component_regs (map, &dport->regs.component, +BIT (CXL_CM_CAP_CAP_ID_RAS ))) +dev_dbg (dev,"Failed to map RAS capability.\n" ); +} + +/** +* cxl_dport_init_ras_reporting - Setup CXL RAS report on this dport +* @dport: the cxl_dport that needs to be initialized +* @host: host device for devm operations +*/ +void cxl_dport_init_ras_reporting (struct cxl_dport *dport,struct device *host) +{ + dport->reg_map.host = host; +cxl_dport_map_ras (dport); + +if (dport->rch) { +struct pci_host_bridge *host_bridge =to_pci_host_bridge (dport->dport_dev); + +if (!host_bridge->native_aer) +return ; + +cxl_dport_map_rch_aer (dport); +cxl_disable_rch_root_ints (dport); + } +} +EXPORT_SYMBOL_NS_GPL (cxl_dport_init_ras_reporting,"CXL" ); + +void cxl_handle_cor_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base) +{ +void __iomem *addr; +u32 status; + +if (!ras_base) +return ; + + addr = ras_base +CXL_RAS_CORRECTABLE_STATUS_OFFSET ; + status =readl (addr); +if (status &CXL_RAS_CORRECTABLE_STATUS_MASK ) { +writel (status &CXL_RAS_CORRECTABLE_STATUS_MASK , addr); +trace_cxl_aer_correctable_error (cxlds->cxlmd, status); + } +} + +/* CXL spec rev3.0 8.2.4.16.1 */ +static void header_log_copy (void __iomem *ras_base,u32 *log) +{ +void __iomem *addr; +u32 *log_addr; +int i, log_u32_size =CXL_HEADERLOG_SIZE /sizeof (u32); + + addr = ras_base +CXL_RAS_HEADER_LOG_OFFSET ; + log_addr = log; + +for (i =0 ; i < log_u32_size; i++) { + *log_addr =readl (addr); + log_addr++; + addr +=sizeof (u32); + } +} + +/* +* Log the state of the RAS status registers and prepare them to log the +* next error status. Return 1 if reset needed. +*/ +bool cxl_handle_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base) +{ +u32 hl[CXL_HEADERLOG_SIZE_U32 ]; +void __iomem *addr; +u32 status; +u32 fe; + +if (!ras_base) +return false; + + addr = ras_base +CXL_RAS_UNCORRECTABLE_STATUS_OFFSET ; + status =readl (addr); +if (!(status &CXL_RAS_UNCORRECTABLE_STATUS_MASK )) +return false; + +/* If multiple errors, log header points to first error from ctrl reg */ +if (hweight32 (status) >1 ) { +void __iomem *rcc_addr = + ras_base +CXL_RAS_CAP_CONTROL_OFFSET ; + + fe =BIT (FIELD_GET (CXL_RAS_CAP_CONTROL_FE_MASK , +readl (rcc_addr))); + }else { + fe = status; + } + +header_log_copy (ras_base, hl); +trace_cxl_aer_uncorrectable_error (cxlds->cxlmd, status, fe, hl); +writel (status &CXL_RAS_UNCORRECTABLE_STATUS_MASK , addr); + +return true; +} + +void cxl_cor_error_detected (struct pci_dev *pdev) +{ +struct cxl_dev_state *cxlds =pci_get_drvdata (pdev); +struct device *dev = &cxlds->cxlmd->dev; + + scoped_guard(device, dev) { +if (!dev->driver) { +dev_warn (&pdev->dev, +"%s: memdev disabled, abort error handling\n" , +dev_name (dev)); +return ; + } + +if (cxlds->rcd) +cxl_handle_rdport_errors (cxlds); + +cxl_handle_cor_ras (cxlds, cxlds->regs.ras); + } +} +EXPORT_SYMBOL_NS_GPL (cxl_cor_error_detected,"CXL" ); + +pci_ers_result_t cxl_error_detected (struct pci_dev *pdev, +pci_channel_state_t state) +{ +struct cxl_dev_state *cxlds =pci_get_drvdata (pdev); +struct cxl_memdev *cxlmd = cxlds->cxlmd; +struct device *dev = &cxlmd->dev; +bool ue; + + scoped_guard(device, dev) { +if (!dev->driver) { +dev_warn (&pdev->dev, +"%s: memdev disabled, abort error handling\n" , +dev_name (dev)); +return PCI_ERS_RESULT_DISCONNECT ; + } + +if (cxlds->rcd) +cxl_handle_rdport_errors (cxlds); +/* +* A frozen channel indicates an impending reset which is fatal to +* CXL.mem operation, and will likely crash the system. On the off +* chance the situation is recoverable dump the status of the RAS +* capability registers and bounce the active state of the memdev. +*/ + ue =cxl_handle_ras (cxlds, cxlds->regs.ras); + } + + +switch (state) { +case pci_channel_io_normal: +if (ue) { +device_release_driver (dev); +return PCI_ERS_RESULT_NEED_RESET ; + } +return PCI_ERS_RESULT_CAN_RECOVER ; +case pci_channel_io_frozen: +dev_warn (&pdev->dev, +"%s: frozen state error detected, disable CXL.mem\n" , +dev_name (dev)); +device_release_driver (dev); +return PCI_ERS_RESULT_NEED_RESET ; +case pci_channel_io_perm_failure: +dev_warn (&pdev->dev, +"failure state error detected, request disconnect\n" ); +return PCI_ERS_RESULT_DISCONNECT ; + } +return PCI_ERS_RESULT_NEED_RESET ; +} +EXPORT_SYMBOL_NS_GPL (cxl_error_detected,"CXL" );drivers/cxl/cxl.h +0
-8@@ -803,14 +803,6 @@ struct cxl_dport *devm_cxl_add_rch_dport(struct cxl_port *port, struct device *dport_dev, int port_id, resource_size_t rcrb); -#ifdef CONFIG_PCIEAER_CXL -void cxl_setup_parent_dport (struct device *host,struct cxl_dport *dport); -void cxl_dport_init_ras_reporting (struct cxl_dport *dport,struct device *host); -#else -static inline void cxl_dport_init_ras_reporting (struct cxl_dport *dport, -struct device *host) { } -#endif -struct cxl_decoder *to_cxl_decoder (struct device *dev);struct cxl_root_decoder *to_cxl_root_decoder (struct device *dev);struct cxl_switch_decoder *to_cxl_switch_decoder (struct device *dev);drivers/cxl/cxlpci.h +16
-0@@ -76,7 +76,23 @@ static inline bool cxl_pci_flit_256(struct pci_dev *pdev)struct cxl_dev_state ;void read_cdat_data (struct cxl_port *port); + +#ifdef CONFIG_CXL_RAS void cxl_cor_error_detected (struct pci_dev *pdev);pci_ers_result_t cxl_error_detected (struct pci_dev *pdev,pci_channel_state_t state); +void cxl_dport_init_ras_reporting (struct cxl_dport *dport,struct device *host); +#else +static inline void cxl_cor_error_detected (struct pci_dev *pdev) { } + +static inline pci_ers_result_t cxl_error_detected (struct pci_dev *pdev, +pci_channel_state_t state) +{ +return PCI_ERS_RESULT_NONE ; +} + +static inline void cxl_dport_init_ras_reporting (struct cxl_dport *dport, +struct device *host) { } +#endif +#endif /* __CXL_PCI_H__ */ tools/testing/cxl/Kbuild +1
-1@@ -57,12 +57,12 @@ cxl_core-y += $(CXL_CORE_SRC)/pci.o cxl_core-y += $(CXL_CORE_SRC)/hdm.o cxl_core-y += $(CXL_CORE_SRC)/pmu.o cxl_core-y += $(CXL_CORE_SRC)/cdat.o -cxl_core-y += $(CXL_CORE_SRC)/ras.o cxl_core-$(CONFIG_TRACING) += $(CXL_CORE_SRC)/trace.o cxl_core-$(CONFIG_CXL_REGION) += $(CXL_CORE_SRC)/region.o cxl_core-$(CONFIG_CXL_MCE) += $(CXL_CORE_SRC)/mce.o cxl_core-$(CONFIG_CXL_FEATURES) += $(CXL_CORE_SRC)/features.o cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += $(CXL_CORE_SRC)/edac.o +cxl_core-$(CONFIG_CXL_RAS) += $(CXL_CORE_SRC)/ras.o cxl_core-y += config_check.o cxl_core-y += cxl_core_test.o cxl_core-y += cxl_core_exports.o
PATCH v14 8/34 cxl/pci: Move CXL driver's RCH error handling into core/ras_rch.c
In reply to Terry BowmanRestricted CXL Host (RCH) protocol error handling uses a procedure distinct from the CXL Virtual Hierarchy (VH) handling. This is because of the differences in the RCH and VH topologies. Improve the maintainability and add ability to enable/disable RCH handling.
Move and combine the RCH handling code into a single block conditionally compiled with the CONFIG_CXL_RCH_RAS kernel config.
---
Changes in v13->v14:
- Add sign-off for Dan and Jonathan
- Revert inadvertent formatting of cxl_dport_map_rch_aer() (Jonathan)
- Remove default value for CXL_RCH_RAS (Dan)
- Remove unnecessary pci.h include in core.h & ras_rch.c (Jonathan)
- Add linux/types.h include in ras_rch.c (Jonathan)
- Change CONFIG_CXL_RCH_RAS -> CONFIG_CXL_RAS (Dan)
Changes in v12->v13:
- None
Changes v11->v12:
- Moved CXL_RCH_RAS Kconfig definition here from following commit.
Changes v10->v11:
- New patch
--- drivers/cxl/core/Makefile | 1 + drivers/cxl/core/core.h | 11 +--- drivers/cxl/core/pci.c | 115 ----------------------------------- drivers/cxl/core/ras_rch.c | 121 +++++++++++++++++++++++++++++++++++++ tools/testing/cxl/Kbuild | 1 + 5 files changed, 126 insertions(+), 123 deletions(-) create mode 100644 drivers/cxl/core/ras_rch.c
drivers/cxl/core/Makefile +1 -0
@@ -20,3 +20,4 @@ cxl_core-$(CONFIG_CXL_MCE) += mce.o cxl_core-$(CONFIG_CXL_FEATURES) += features.o cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += edac.o cxl_core-$(CONFIG_CXL_RAS) += ras.o +cxl_core-$(CONFIG_CXL_RAS) += ras_rch.o
drivers/cxl/core/core.h +3 -8
@@ -149,6 +149,9 @@ int cxl_ras_init(void);void cxl_ras_exit (void );bool cxl_handle_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base);void cxl_handle_cor_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base); +void cxl_dport_map_rch_aer (struct cxl_dport *dport); +void cxl_disable_rch_root_ints (struct cxl_dport *dport); +void cxl_handle_rdport_errors (struct cxl_dev_state *cxlds);#else static inline int cxl_ras_init (void ) {
@@ -164,14 +167,6 @@ static inline bool cxl_handle_ras(struct cxl_dev_state *cxlds, void __iomem *rasreturn false; }static inline void cxl_handle_cor_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base) { } -#endif /* CONFIG_CXL_RAS */ - -/* Restricted CXL Host specific RAS functions */ -#ifdef CONFIG_CXL_RAS -void cxl_dport_map_rch_aer (struct cxl_dport *dport); -void cxl_disable_rch_root_ints (struct cxl_dport *dport); -void cxl_handle_rdport_errors (struct cxl_dev_state *cxlds); -#else static inline void cxl_dport_map_rch_aer (struct cxl_dport *dport) { }static inline void cxl_disable_rch_root_ints (struct cxl_dport *dport) { }static inline void cxl_handle_rdport_errors (struct cxl_dev_state *cxlds) { }
drivers/cxl/core/pci.c +0 -115
121 lines not on this page · raw message
drivers/cxl/core/ras_rch.c new file +121 -0
121 lines not on this page · raw message
tools/testing/cxl/Kbuild +1 -0
@@ -63,6 +63,7 @@ cxl_core-$(CONFIG_CXL_MCE) += $(CXL_CORE_SRC)/mce.o cxl_core-$(CONFIG_CXL_FEATURES) += $(CXL_CORE_SRC)/features.o cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += $(CXL_CORE_SRC)/edac.o cxl_core-$(CONFIG_CXL_RAS) += $(CXL_CORE_SRC)/ras.o +cxl_core-$(CONFIG_CXL_RAS) += $(CXL_CORE_SRC)/ras_rch.o cxl_core-y += config_check.o cxl_core-y += cxl_core_test.o cxl_core-y += cxl_core_exports.o
Signature · 1 line
2.34.1
Re: [PATCH v14 08/34] cxl/pci: Move CXL driver's RCH error handling into core/ras_rch.c
Quoted 5 lines · Terry Bowman
Restricted CXL Host (RCH) protocol error handling uses a procedure distinct from the CXL Virtual Hierarchy (VH) handling. This is because of the differences in the RCH and VH topologies. Improve the maintainability and add ability to enable/disable RCH handling.
Quoted 7 lines
Move and combine the RCH handling code into a single block conditionally compiled with the
CONFIG_CXL_RCH_RASkernel config.Signed-off-by Terry Bowman <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Signed-off-by Dan Williams <[email protected]>
Quoted 330 lines
---
Changes in
v13->v14:
- Add sign-off for Dan and Jonathan
- Revert inadvertent formatting ofcxl_dport_map_rch_aer()(Jonathan)
- Remove default value forCXL_RCH_RAS(Dan)
- Remove unnecessarypci.hinclude incore.h&ras_rch.c(Jonathan)
- Addlinux/types.hinclude inras_rch.c(Jonathan)
- ChangeCONFIG_CXL_RCH_RAS->CONFIG_CXL_RAS(Dan)Changes in
v12->v13:
- NoneChanges
v11->v12:
- MovedCXL_RCH_RASKconfigdefinition here from following commit.Changes
v10->v11:
- New patch--- drivers/cxl/core/Makefile | 1 + drivers/cxl/core/core.h | 11 +--- drivers/cxl/core/pci.c | 115 ----------------------------------- drivers/cxl/core/ras_rch.c | 121 +++++++++++++++++++++++++++++++++++++ tools/testing/cxl/Kbuild | 1 + 5 files changed, 126 insertions(+), 123 deletions(-) create mode 100644 drivers/cxl/core/ras_rch.cdrivers/cxl/core/Makefile +1
-0@@ -20,3 +20,4 @@ cxl_core-$(CONFIG_CXL_MCE) += mce.o cxl_core-$(CONFIG_CXL_FEATURES) += features.o cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += edac.o cxl_core-$(CONFIG_CXL_RAS) += ras.o +cxl_core-$(CONFIG_CXL_RAS) += ras_rch.odrivers/cxl/core/core.h +3
-8@@ -149,6 +149,9 @@ int cxl_ras_init(void);void cxl_ras_exit (void );bool cxl_handle_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base);void cxl_handle_cor_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base); +void cxl_dport_map_rch_aer (struct cxl_dport *dport); +void cxl_disable_rch_root_ints (struct cxl_dport *dport); +void cxl_handle_rdport_errors (struct cxl_dev_state *cxlds);#else static inline int cxl_ras_init (void ) {@@ -164,14 +167,6 @@ static inline bool cxl_handle_ras(struct cxl_dev_state *cxlds, void __iomem *rasreturn false; }static inline void cxl_handle_cor_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base) { } -#endif /* CONFIG_CXL_RAS */ - -/* Restricted CXL Host specific RAS functions */ -#ifdef CONFIG_CXL_RAS -void cxl_dport_map_rch_aer (struct cxl_dport *dport); -void cxl_disable_rch_root_ints (struct cxl_dport *dport); -void cxl_handle_rdport_errors (struct cxl_dev_state *cxlds); -#else static inline void cxl_dport_map_rch_aer (struct cxl_dport *dport) { }static inline void cxl_disable_rch_root_ints (struct cxl_dport *dport) { }static inline void cxl_handle_rdport_errors (struct cxl_dev_state *cxlds) { }drivers/cxl/core/pci.c +0
-115@@ -632,121 +632,6 @@ void read_cdat_data(struct cxl_port *port) }EXPORT_SYMBOL_NS_GPL (read_cdat_data,"CXL" ); -#ifdef CONFIG_CXL_RAS -void cxl_dport_map_rch_aer (struct cxl_dport *dport) -{ -resource_size_t aer_phys; -struct device *host; -u16 aer_cap; - - aer_cap =cxl_rcrb_to_aer (dport->dport_dev, dport->rcrb.base); -if (aer_cap) { - host = dport->reg_map.host; - aer_phys = aer_cap + dport->rcrb.base; - dport->regs.dport_aer =devm_cxl_iomap_block (host, aer_phys, -sizeof (struct aer_capability_regs )); - } -} - -void cxl_disable_rch_root_ints (struct cxl_dport *dport) -{ -void __iomem *aer_base = dport->regs.dport_aer; -u32 aer_cmd_mask, aer_cmd; - -if (!aer_base) -return ; - -/* -* Disable RCH root port command interrupts. -* CXL 3.0 12.2.1.1 - RCH Downstream Port-detected Errors -* -* This sequence may not be necessary. CXL spec states disabling -* the root cmd register's interrupts is required. But, PCI spec -* shows these are disabled by default on reset. -*/ - aer_cmd_mask = (PCI_ERR_ROOT_CMD_COR_EN | -PCI_ERR_ROOT_CMD_NONFATAL_EN | -PCI_ERR_ROOT_CMD_FATAL_EN ); - aer_cmd =readl (aer_base +PCI_ERR_ROOT_COMMAND ); - aer_cmd &= ~aer_cmd_mask; -writel (aer_cmd, aer_base +PCI_ERR_ROOT_COMMAND ); -} - -/* -* Copy the AER capability registers using 32 bit read accesses. -* This is necessary because RCRB AER capability is MMIO mapped. Clear the -* status after copying. -* -* @aer_base: base address of AER capability block in RCRB -* @aer_regs: destination for copying AER capability -*/ -static bool cxl_rch_get_aer_info (void __iomem *aer_base, -struct aer_capability_regs *aer_regs) -{ -int read_cnt =sizeof (struct aer_capability_regs ) /sizeof (u32); -u32 *aer_regs_buf = (u32 *)aer_regs; -int n; - -if (!aer_base) -return false; - -/* Use readl() to guarantee 32-bit accesses */ -for (n =0 ; n < read_cnt; n++) - aer_regs_buf[n] =readl (aer_base + n *sizeof (u32)); - -writel (aer_regs->uncor_status, aer_base +PCI_ERR_UNCOR_STATUS ); -writel (aer_regs->cor_status, aer_base +PCI_ERR_COR_STATUS ); - -return true; -} - -/* Get AER severity. Return false if there is no error. */ -static bool cxl_rch_get_aer_severity (struct aer_capability_regs *aer_regs, -int *severity) -{ -if (aer_regs->uncor_status & ~aer_regs->uncor_mask) { -if (aer_regs->uncor_status &PCI_ERR_ROOT_FATAL_RCV ) - *severity =AER_FATAL ; -else - *severity =AER_NONFATAL ; -return true; - } - -if (aer_regs->cor_status & ~aer_regs->cor_mask) { - *severity =AER_CORRECTABLE ; -return true; - } - -return false; -} - -void cxl_handle_rdport_errors (struct cxl_dev_state *cxlds) -{ -struct pci_dev *pdev =to_pci_dev (cxlds->dev); -struct aer_capability_regs aer_regs; -struct cxl_dport *dport; -int severity; - - struct cxl_port *port __free(put_cxl_port) = -cxl_pci_find_port (pdev, &dport); -if (!port) -return ; - -if (!cxl_rch_get_aer_info (dport->regs.dport_aer, &aer_regs)) -return ; - -if (!cxl_rch_get_aer_severity (&aer_regs, &severity)) -return ; - -pci_print_aer (pdev, severity, &aer_regs); - -if (severity ==AER_CORRECTABLE ) -cxl_handle_cor_ras (cxlds, dport->regs.ras); -else -cxl_handle_ras (cxlds, dport->regs.ras); -} -#endif -static int cxl_flit_size (struct pci_dev *pdev) {if (cxl_pci_flit_256 (pdev))drivers/cxl/core/ras_rch.c new file +121
-0@@ -0,0 +1,121 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(c) 2025 AMD Corporation. All rights reserved. */ + +#include <linux/types.h> +#include <linux/aer.h> +#include "cxl.h" +#include "core.h" +#include "cxlmem.h" + +void cxl_dport_map_rch_aer (struct cxl_dport *dport) +{ +resource_size_t aer_phys; +struct device *host; +u16 aer_cap; + + aer_cap =cxl_rcrb_to_aer (dport->dport_dev, dport->rcrb.base); +if (aer_cap) { + host = dport->reg_map.host; + aer_phys = aer_cap + dport->rcrb.base; + dport->regs.dport_aer = +devm_cxl_iomap_block (host, aer_phys, +sizeof (struct aer_capability_regs )); + } +} + +void cxl_disable_rch_root_ints (struct cxl_dport *dport) +{ +void __iomem *aer_base = dport->regs.dport_aer; +u32 aer_cmd_mask, aer_cmd; + +if (!aer_base) +return ; + +/* +* Disable RCH root port command interrupts. +* CXL 3.0 12.2.1.1 - RCH Downstream Port-detected Errors +* +* This sequence may not be necessary. CXL spec states disabling +* the root cmd register's interrupts is required. But, PCI spec +* shows these are disabled by default on reset. +*/ + aer_cmd_mask = (PCI_ERR_ROOT_CMD_COR_EN | +PCI_ERR_ROOT_CMD_NONFATAL_EN | +PCI_ERR_ROOT_CMD_FATAL_EN ); + aer_cmd =readl (aer_base +PCI_ERR_ROOT_COMMAND ); + aer_cmd &= ~aer_cmd_mask; +writel (aer_cmd, aer_base +PCI_ERR_ROOT_COMMAND ); +} + +/* +* Copy the AER capability registers using 32 bit read accesses. +* This is necessary because RCRB AER capability is MMIO mapped. Clear the +* status after copying. +* +* @aer_base: base address of AER capability block in RCRB +* @aer_regs: destination for copying AER capability +*/ +static bool cxl_rch_get_aer_info (void __iomem *aer_base, +struct aer_capability_regs *aer_regs) +{ +int read_cnt =sizeof (struct aer_capability_regs ) /sizeof (u32); +u32 *aer_regs_buf = (u32 *)aer_regs; +int n; + +if (!aer_base) +return false; + +/* Use readl() to guarantee 32-bit accesses */ +for (n =0 ; n < read_cnt; n++) + aer_regs_buf[n] =readl (aer_base + n *sizeof (u32)); + +writel (aer_regs->uncor_status, aer_base +PCI_ERR_UNCOR_STATUS ); +writel (aer_regs->cor_status, aer_base +PCI_ERR_COR_STATUS ); + +return true; +} + +/* Get AER severity. Return false if there is no error. */ +static bool cxl_rch_get_aer_severity (struct aer_capability_regs *aer_regs, +int *severity) +{ +if (aer_regs->uncor_status & ~aer_regs->uncor_mask) { +if (aer_regs->uncor_status &PCI_ERR_ROOT_FATAL_RCV ) + *severity =AER_FATAL ; +else + *severity =AER_NONFATAL ; +return true; + } + +if (aer_regs->cor_status & ~aer_regs->cor_mask) { + *severity =AER_CORRECTABLE ; +return true; + } + +return false; +} + +void cxl_handle_rdport_errors (struct cxl_dev_state *cxlds) +{ +struct pci_dev *pdev =to_pci_dev (cxlds->dev); +struct aer_capability_regs aer_regs; +struct cxl_dport *dport; +int severity; + + struct cxl_port *port __free(put_cxl_port) = +cxl_pci_find_port (pdev, &dport); +if (!port) +return ; + +if (!cxl_rch_get_aer_info (dport->regs.dport_aer, &aer_regs)) +return ; + +if (!cxl_rch_get_aer_severity (&aer_regs, &severity)) +return ; + +pci_print_aer (pdev, severity, &aer_regs); +if (severity ==AER_CORRECTABLE ) +cxl_handle_cor_ras (cxlds, dport->regs.ras); +else +cxl_handle_ras (cxlds, dport->regs.ras); +}tools/testing/cxl/Kbuild +1
-0@@ -63,6 +63,7 @@ cxl_core-$(CONFIG_CXL_MCE) += $(CXL_CORE_SRC)/mce.o cxl_core-$(CONFIG_CXL_FEATURES) += $(CXL_CORE_SRC)/features.o cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += $(CXL_CORE_SRC)/edac.o cxl_core-$(CONFIG_CXL_RAS) += $(CXL_CORE_SRC)/ras.o +cxl_core-$(CONFIG_CXL_RAS) += $(CXL_CORE_SRC)/ras_rch.o cxl_core-y += config_check.o cxl_core-y += cxl_core_test.o cxl_core-y += cxl_core_exports.o
PATCH v14 9/34 PCI/AER: Export pci_aer_unmask_internal_errors()
In reply to Terry BowmanInternal PCIe errors are not enabled by default during initialization. This creates a problem for CXL drivers, which rely on PCIe Correctable and Uncorrectable Internal Errors to receive CXL protocol error notifications.
Export pci_aer_unmask_internal_errors() so CXL and other drivers can enable internal PCIe errors.
---
Changes in v13->v14:
- New commit. Bjorn requested separating out and adding immediatetly before being used. This is called from cxl_rch_enable_rcec() in following patch.
--- drivers/pci/pcie/aer.c | 6 +++--- include/linux/aer.h | 2 ++ 2 files changed, 5 insertions(+), 3 deletions(-)
drivers/pci/pcie/aer.c +3 -3
@@ -1120,8 +1120,6 @@ static bool find_source_device(struct pci_dev *parent,return true; } -#ifdef CONFIG_PCIEAER_CXL -/** * pci_aer_unmask_internal_errors - unmask internal errors * @dev: pointer to the pci_dev data structure
@@ -1132,7 +1130,7 @@ static bool find_source_device(struct pci_dev *parent,* Note: AER must be enabled and supported by the device which must be * checked in advance, e.g. with pcie_aer_is_native(). */ -static void pci_aer_unmask_internal_errors (struct pci_dev *dev) +void pci_aer_unmask_internal_errors (struct pci_dev *dev) {int aer = dev->aer_cap;u32 mask;
@@ -1145,7 +1143,9 @@ static void pci_aer_unmask_internal_errors(struct pci_dev *dev) mask &= ~PCI_ERR_COR_INTERNAL ;pci_write_config_dword (dev, aer +PCI_ERR_COR_MASK , mask); } +EXPORT_SYMBOL_GPL (pci_aer_unmask_internal_errors); +#ifdef CONFIG_PCIEAER_CXL static bool is_cxl_mem_dev (struct pci_dev *dev) {/*
include/linux/aer.h +2 -0
@@ -56,12 +56,14 @@ struct aer_capability_regs {#if defined(CONFIG_PCIEAER )int pci_aer_clear_nonfatal_status (struct pci_dev *dev);int pcie_aer_is_native (struct pci_dev *dev); +void pci_aer_unmask_internal_errors (struct pci_dev *dev);#else static inline int pci_aer_clear_nonfatal_status (struct pci_dev *dev) {return -EINVAL ; }static inline int pcie_aer_is_native (struct pci_dev *dev) {return 0 ; } +static inline void pci_aer_unmask_internal_errors (struct pci_dev *dev) { }#endif void pci_print_aer(struct pci_dev *dev, int aer_severity,
Signature · 1 line
2.34.1
Re: [PATCH v14 09/34] PCI/AER: Export pci_aer_unmask_internal_errors()
Quoted 8 lines · On Wed, 14 Jan 2026 12:20:30 -0600 Terry Bowman <[email protected]> wrote:
Internal PCIe errors are not enabled by default during initialization. This creates a problem for CXL drivers, which rely on PCIe Correctable and Uncorrectable Internal Errors to receive CXL protocol error notifications.
Export
pci_aer_unmask_internal_errors()so CXL and other drivers can enable internal PCIe errors.Signed-off-by Terry Bowman <[email protected]>
Re: [PATCH v14 09/34] PCI/AER: Export pci_aer_unmask_internal_errors()
In reply to Terry BowmanQuoted 10 lines · Terry Bowman
Internal PCIe errors are not enabled by default during initialization. This creates a problem for CXL drivers, which rely on PCIe Correctable and Uncorrectable Internal Errors to receive CXL protocol error notifications.
Export
pci_aer_unmask_internal_errors()so CXL and other drivers can enable internal PCIe errors.Signed-off-by Terry Bowman <[email protected]>---
Quoted 61 lines
Changes in
v13->v14:
- New commit. Bjorn requested separating out and adding immediatetly before being used. This is called fromcxl_rch_enable_rcec()in following patch.--- drivers/pci/pcie/aer.c | 6 +++--- include/linux/aer.h | 2 ++ 2 files changed, 5 insertions(+), 3 deletions(-)drivers/pci/pcie/aer.c +3
-3@@ -1120,8 +1120,6 @@ static bool find_source_device(struct pci_dev *parent,return true; } -#ifdef CONFIG_PCIEAER_CXL -/** * pci_aer_unmask_internal_errors - unmask internal errors * @dev: pointer to the pci_dev data structure @@ -1132,7 +1130,7 @@ static bool find_source_device(struct pci_dev *parent,* Note: AER must be enabled and supported by the device which must be * checked in advance, e.g. with pcie_aer_is_native(). */ -static void pci_aer_unmask_internal_errors (struct pci_dev *dev) +void pci_aer_unmask_internal_errors (struct pci_dev *dev) {int aer = dev->aer_cap;u32 mask;@@ -1145,7 +1143,9 @@ static void pci_aer_unmask_internal_errors(struct pci_dev *dev) mask &= ~PCI_ERR_COR_INTERNAL ;pci_write_config_dword (dev, aer +PCI_ERR_COR_MASK , mask); } +EXPORT_SYMBOL_GPL (pci_aer_unmask_internal_errors); +#ifdef CONFIG_PCIEAER_CXL static bool is_cxl_mem_dev (struct pci_dev *dev) {/* include/linux/aer.h +2
-0@@ -56,12 +56,14 @@ struct aer_capability_regs {#if defined(CONFIG_PCIEAER )int pci_aer_clear_nonfatal_status (struct pci_dev *dev);int pcie_aer_is_native (struct pci_dev *dev); +void pci_aer_unmask_internal_errors (struct pci_dev *dev);#else static inline int pci_aer_clear_nonfatal_status (struct pci_dev *dev) {return -EINVAL ; }static inline int pcie_aer_is_native (struct pci_dev *dev) {return 0 ; } +static inline void pci_aer_unmask_internal_errors (struct pci_dev *dev) { }#endif void pci_print_aer(struct pci_dev *dev, int aer_severity,
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [PATCH v14 09/34] PCI/AER: Export pci_aer_unmask_internal_errors()
In reply to Terry BowmanQuoted 8 lines · Terry Bowman
Internal PCIe errors are not enabled by default during initialization. This creates a problem for CXL drivers, which rely on PCIe Correctable and Uncorrectable Internal Errors to receive CXL protocol error notifications.
Export
pci_aer_unmask_internal_errors()so CXL and other drivers can enable internal PCIe errors.Signed-off-by Terry Bowman <[email protected]>
Quoted 63 lines
---
Changes in
v13->v14:
- New commit. Bjorn requested separating out and adding immediatetly before being used. This is called fromcxl_rch_enable_rcec()in following patch.--- drivers/pci/pcie/aer.c | 6 +++--- include/linux/aer.h | 2 ++ 2 files changed, 5 insertions(+), 3 deletions(-)drivers/pci/pcie/aer.c +3
-3@@ -1120,8 +1120,6 @@ static bool find_source_device(struct pci_dev *parent,return true; } -#ifdef CONFIG_PCIEAER_CXL -/** * pci_aer_unmask_internal_errors - unmask internal errors * @dev: pointer to the pci_dev data structure @@ -1132,7 +1130,7 @@ static bool find_source_device(struct pci_dev *parent,* Note: AER must be enabled and supported by the device which must be * checked in advance, e.g. with pcie_aer_is_native(). */ -static void pci_aer_unmask_internal_errors (struct pci_dev *dev) +void pci_aer_unmask_internal_errors (struct pci_dev *dev) {int aer = dev->aer_cap;u32 mask;@@ -1145,7 +1143,9 @@ static void pci_aer_unmask_internal_errors(struct pci_dev *dev) mask &= ~PCI_ERR_COR_INTERNAL ;pci_write_config_dword (dev, aer +PCI_ERR_COR_MASK , mask); } +EXPORT_SYMBOL_GPL (pci_aer_unmask_internal_errors); +#ifdef CONFIG_PCIEAER_CXL static bool is_cxl_mem_dev (struct pci_dev *dev) {/* include/linux/aer.h +2
-0@@ -56,12 +56,14 @@ struct aer_capability_regs {#if defined(CONFIG_PCIEAER )int pci_aer_clear_nonfatal_status (struct pci_dev *dev);int pcie_aer_is_native (struct pci_dev *dev); +void pci_aer_unmask_internal_errors (struct pci_dev *dev);#else static inline int pci_aer_clear_nonfatal_status (struct pci_dev *dev) {return -EINVAL ; }static inline int pcie_aer_is_native (struct pci_dev *dev) {return 0 ; } +static inline void pci_aer_unmask_internal_errors (struct pci_dev *dev) { }#endif void pci_print_aer(struct pci_dev *dev, int aer_severity,
[email protected]
Reviewed-by Dave Jiang Reviewed-by Kuppuswamy Sathyanarayanan Reviewed-by Jonathan Cameron on v14 9/34Re: [PATCH v14 09/34] PCI/AER: Export pci_aer_unmask_internal_errors()
In reply to Terry BowmanQuoted 6 lines · Terry Bowman
Internal PCIe errors are not enabled by default during initialization. This creates a problem for CXL drivers, which rely on PCIe Correctable and Uncorrectable Internal Errors to receive CXL protocol error notifications.
Export
pci_aer_unmask_internal_errors()so CXL and other drivers can enable internal PCIe errors.
I folded in the following to this patch because opening up internal errors for PCIe drivers in general is not a goal.
1: cb9a15481d8c ! 1: 7433e0204753 PCI/AER: Export pci_aer_unmask_internal_errors()
@@ Metadata
## Commit message ##
PCI/AER: Export pci_aer_unmask_internal_errors()
- Internal PCIe errors are not enabled by default during initialization. This
- creates a problem for CXL drivers, which rely on PCIe Correctable and
- Uncorrectable Internal Errors to receive CXL protocol error notifications.
+ Internal PCIe errors are not enabled by default during initialization
+ because their behavior is too device-specific and there is no standard way
+ to reason about them. However, for CXL an internal error is the standard
+ mechanism for conveying CXL protocol errors.
- Export pci_aer_unmask_internal_errors() so CXL and other drivers can
- enable internal PCIe errors.
+ Export pci_aer_unmask_internal_errors() for CXL, but make it clear that
+ they are only meant for CXL and the status quo for leaving them masked for
+ PCIe in general remains.
+ Co-developed-by: Dan Williams <[email protected]>
## include/linux/aer.h ##
@@ drivers/pci/pcie/aer.c: static bool find_source_device(struct pci_dev *parent,
int aer = dev->aer_cap;
u32 mask;
@@ drivers/pci/pcie/aer.c: static void pci_aer_unmask_internal_errors(struct pci_dev *dev)
- mask &= ~PCI_ERR_COR_INTERNAL;
pci_write_config_dword(dev, aer + PCI_ERR_COR_MASK, mask);
}
-+EXPORT_SYMBOL_GPL(pci_aer_unmask_internal_errors);
++/*
++ * Internal errors are too device-specific to enable generally, however for CXL ++ * their behavior is standardized for conveying CXL protocol errors. ++ */
++EXPORT_SYMBOL_FOR_MODULES(pci_aer_unmask_internal_errors, "cxl_core"); ++
+#ifdef CONFIG_PCIEAER_CXL
static bool is_cxl_mem_dev(struct pci_dev *dev)
{
Re: [PATCH v14 09/34] PCI/AER: Export pci_aer_unmask_internal_errors()
Quoted 10 lines · On Mon, Jan 19, 2026 at 06:09:39PM -0800, [email protected] wrote:
Quoted 6 lines · Terry Bowman
Internal PCIe errors are not enabled by default during initialization. This creates a problem for CXL drivers, which rely on PCIe Correctable and Uncorrectable Internal Errors to receive CXL protocol error notifications.
Export
pci_aer_unmask_internal_errors()so CXL and other drivers can enable internal PCIe errors.I folded in the following to this patch because opening up internal errors for PCIe drivers in general is not a goal.
As said, the "xe" driver needs to unmask Internal Errors and could take advantage of this helper, so I'd call opening this up for PCI drivers if not a goal then at least a "desirable side effect". ;)
https://lore.kernel.org/all/[email protected]/
Quoted 3 lines
+ Internal PCIe errors are not enabled by default during initialization
+ because their behavior is too device-specific and there is no standard way
+ to reason about them.Well, they're not enabled by default because per the spec they're masked in the Uncorrectable Error Mask and Correctable Error Mask Registers. It's up to drivers to unmask them if they know the hardware signals them. CXL just happens be one of those drivers.
Quoted 4 lines
@@
drivers/pci/pcie/aer.c: static voidpci_aer_unmask_internal_errors(struct pci_dev *dev)
- mask &= ~PCI_ERR_COR_INTERNAL;
pci_write_config_dword(dev, aer + PCI_ERR_COR_MASK, mask);
}
Unexplained change vis-à-vis Terry's submission. It seems you're reading the Correctable Error Mask Register and writing the same value back. That's doesn't seem to make sense.
Quoted 8 lines
-+EXPORT_SYMBOL_GPL(pci_aer_unmask_internal_errors);
++/*
++ * Internal errors are too device-specific to enable generally, however for CXL
++ * their behavior is standardized for conveying CXL protocol errors.
++ */
++EXPORT_SYMBOL_FOR_MODULES(pci_aer_unmask_internal_errors, "cxl_core");
++This change will require touching aer.c every time a driver (such as xe) has the need to unmask Internal Errors. Not sure if that's such a good idea...
Thanks,
Lukas
Re: [PATCH v14 09/34] PCI/AER: Export pci_aer_unmask_internal_errors()
Quoted 9 lines · Lukas Wunner
Quoted 8 lines · On Mon, Jan 19, 2026 at 06:09:39PM -0800, [email protected] wrote:
Quoted 6 lines · Terry Bowman
Internal PCIe errors are not enabled by default during initialization. This creates a problem for CXL drivers, which rely on PCIe Correctable and Uncorrectable Internal Errors to receive CXL protocol error notifications.
Export
pci_aer_unmask_internal_errors()so CXL and other drivers can enable internal PCIe errors.
Quoted 8 lines
Quoted 2 lines
I folded in the following to this patch because opening up internal errors for PCIe drivers in general is not a goal.
As said, the "xe" driver needs to unmask Internal Errors and could take advantage of this helper, so I'd call opening this up for PCI drivers if not a goal then at least a "desirable side effect". ;)
I missed that earlier. How did xe manage to be the only device in the history of Linux that needs internal errors unmasked?
What happens if Linux says "no, that error model has never been supported and it creates in ongoing mental / maintenance load of internal errors do not matter for PCIe, only CXL, (except xe)."
Quoted 9 lines
Quoted 3 lines
+ Internal PCIe errors are not enabled by default during initialization + because their behavior is too device-specific and there is no standard way + to reason about them.Well, they're not enabled by default because per the spec they're masked in the Uncorrectable Error Mask and Correctable Error Mask Registers. It's up to drivers to unmask them if they know the hardware signals them. CXL just happens be one of those drivers.
Quoted 8 lines
Quoted 4 lines
@@
drivers/pci/pcie/aer.c: static voidpci_aer_unmask_internal_errors(struct pci_dev *dev)
- mask &= ~PCI_ERR_COR_INTERNAL;
pci_write_config_dword(dev, aer + PCI_ERR_COR_MASK, mask);
}Unexplained change vis-à-vis Terry's submission. It seems you're reading the Correctable Error Mask Register and writing the same value back. That's doesn't seem to make sense.
No, sorry, this an interdiff so that change was just a change in context.
It also caused me to do a double-take until I realized it was a pure hunk context change.
Quoted 10 lines
Quoted 9 lines
-+EXPORT_SYMBOL_GPL(pci_aer_unmask_internal_errors); ++/* ++ * Internal errors are too device-specific to enable generally, however for CXL ++ * their behavior is standardized for conveying CXL protocol errors. ++ */ ++EXPORT_SYMBOL_FOR_MODULES(pci_aer_unmask_internal_errors, "cxl_core"); ++
Quoted 3 lines
This change will require touching
aer.cevery time a driver (such as xe) has the need to unmask Internal Errors. Not sure if that's such a good idea...
The xe driver can always come back and change this to plain EXPORT_SYMBOL_GPL() once the clear the hurdle above of, "please reconsider your error model to not require this never needed before feature of AER".
Re: [PATCH v14 09/34] PCI/AER: Export pci_aer_unmask_internal_errors()
Quoted 5 lines · On Thu, Jan 22, 2026 at 08:48:36AM -0800, [email protected] wrote:
Quoted 4 lines · Lukas Wunner
As said, the "xe" driver needs to unmask Internal Errors and could take advantage of this helper, so I'd call opening this up for PCI drivers if not a goal then at least a "desirable side effect". ;)
Quoted 8 lines
Quoted 1 line
I missed that earlier. How did xe manage to be the only device in the history of Linux that needs internal errors unmasked?
What happens if Linux says "no, that error model has never been supported and it creates in ongoing mental / maintenance load of internal errors do not matter for PCIe, only CXL, (except xe)."
Every spec-defined feature is fair game to be used by drivers and denying them support would seem unreasonable.
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
-+EXPORT_SYMBOL_GPL(pci_aer_unmask_internal_errors); ++/* ++ * Internal errors are too device-specific to enable generally, however for CXL ++ * their behavior is standardized for conveying CXL protocol errors. ++ */ ++EXPORT_SYMBOL_FOR_MODULES(pci_aer_unmask_internal_errors, "cxl_core"); ++
Quoted 8 lines
Quoted 3 lines
This change will require touching
aer.cevery time a driver (such as xe) has the need to unmask Internal Errors. Not sure if that's such a good idea...The xe driver can always come back and change this to plain
EXPORT_SYMBOL_GPL()once the clear the hurdle above of, "please reconsider your error model to not require this never needed before feature of AER".
Of course, but it may annoy Bjorn that he'll have to deal with an amendment of this macro for each individual driver that needs it, or at least for the *next* driver that needs it.
Thanks,
Lukas
Re: [PATCH v14 09/34] PCI/AER: Export pci_aer_unmask_internal_errors()
In reply to Terry BowmanQuoted 8 lines · Terry Bowman
Internal PCIe errors are not enabled by default during initialization. This creates a problem for CXL drivers, which rely on PCIe Correctable and Uncorrectable Internal Errors to receive CXL protocol error notifications.
Export
pci_aer_unmask_internal_errors()so CXL and other drivers can enable internal PCIe errors.Signed-off-by Terry Bowman <[email protected]>
Quoted 65 lines
---
Changes in
v13->v14:
- New commit. Bjorn requested separating out and adding immediatetly before being used. This is called fromcxl_rch_enable_rcec()in following patch.--- drivers/pci/pcie/aer.c | 6 +++--- include/linux/aer.h | 2 ++ 2 files changed, 5 insertions(+), 3 deletions(-)drivers/pci/pcie/aer.c +3
-3@@ -1120,8 +1120,6 @@ static bool find_source_device(struct pci_dev *parent,return true; } -#ifdef CONFIG_PCIEAER_CXL -/** * pci_aer_unmask_internal_errors - unmask internal errors * @dev: pointer to the pci_dev data structure @@ -1132,7 +1130,7 @@ static bool find_source_device(struct pci_dev *parent,* Note: AER must be enabled and supported by the device which must be * checked in advance, e.g. with pcie_aer_is_native(). */ -static void pci_aer_unmask_internal_errors (struct pci_dev *dev) +void pci_aer_unmask_internal_errors (struct pci_dev *dev) {int aer = dev->aer_cap;u32 mask;@@ -1145,7 +1143,9 @@ static void pci_aer_unmask_internal_errors(struct pci_dev *dev) mask &= ~PCI_ERR_COR_INTERNAL ;pci_write_config_dword (dev, aer +PCI_ERR_COR_MASK , mask); } +EXPORT_SYMBOL_GPL (pci_aer_unmask_internal_errors); +#ifdef CONFIG_PCIEAER_CXL static bool is_cxl_mem_dev (struct pci_dev *dev) {/* include/linux/aer.h +2
-0@@ -56,12 +56,14 @@ struct aer_capability_regs {#if defined(CONFIG_PCIEAER )int pci_aer_clear_nonfatal_status (struct pci_dev *dev);int pcie_aer_is_native (struct pci_dev *dev); +void pci_aer_unmask_internal_errors (struct pci_dev *dev);#else static inline int pci_aer_clear_nonfatal_status (struct pci_dev *dev) {return -EINVAL ; }static inline int pcie_aer_is_native (struct pci_dev *dev) {return 0 ; } +static inline void pci_aer_unmask_internal_errors (struct pci_dev *dev) { }#endif void pci_print_aer(struct pci_dev *dev, int aer_severity,Signature · 1 line
2.34.1
PATCH v14 10/34 PCI/AER: Update is_internal_error() to be non-static is_aer_internal_error()
In reply to Terry BowmanThe AER driver includes significant logic for handling CXL protocol errors. The AER driver will be updated in the future to separate the AER and CXL logic.
Rename the is_internal_error() function to is_aer_internal_error() as it gives a more precise indication of the purpose. Make is_aer_internal_error() non-static to allow for other PCI drivers to access.
---
Changes in v13->v14:
- New patch
--- drivers/pci/pcie/aer.c | 4 ++-- drivers/pci/pcie/portdrv.h | 9 +++++++++ 2 files changed, 11 insertions(+), 2 deletions(-)
drivers/pci/pcie/aer.c +2 -2
@@ -1166,7 +1166,7 @@ static bool is_cxl_mem_dev(struct pci_dev *dev)return true; } -static bool is_internal_error (struct aer_err_info *info) +bool is_aer_internal_error (struct aer_err_info *info) {if (info->severity ==AER_CORRECTABLE )return info->status &PCI_ERR_COR_INTERNAL ;
@@ -1211,7 +1211,7 @@ static void cxl_rch_handle_error(struct pci_dev *dev, struct aer_err_info *info)* device driver. */ if (pci_pcie_type (dev) ==PCI_EXP_TYPE_RC_EC && -is_internal_error (info)) +is_aer_internal_error (info))pcie_walk_rcec (dev, cxl_rch_handle_error_iter, info); }
drivers/pci/pcie/portdrv.h +9 -0
@@ -123,4 +123,13 @@ static inline void pcie_pme_interrupt_enable(struct pci_dev *dev, bool en) {}#endif /* !CONFIG_PCIE_PME */ struct device *pcie_port_find_device (struct pci_dev *dev,u32 service); + +struct aer_err_info ; + +#ifdef CONFIG_PCIEAER_CXL +bool is_aer_internal_error (struct aer_err_info *info); +#else +static inline bool is_aer_internal_error (struct aer_err_info *info) {return false; } +#endif /* CONFIG_PCIEAER_CXL */ +#endif /* _PORTDRV_H_ */
Signature · 1 line
2.34.1
Re: [PATCH v14 10/34] PCI/AER: Update is_internal_error() to be non-static is_aer_internal_error()
Quoted 9 lines · On Wed, 14 Jan 2026 12:20:31 -0600 Terry Bowman <[email protected]> wrote:
The AER driver includes significant logic for handling CXL protocol errors. The AER driver will be updated in the future to separate the AER and CXL logic.
Rename the
is_internal_error()function tois_aer_internal_error()as it gives a more precise indication of the purpose. Makeis_aer_internal_error()non-static to allow for other PCI drivers to access.Signed-off-by Terry Bowman <[email protected]>
Hi Terry,
I don't see it as sensible to have is_aer_internal_error() return false if CXL is not built. That question has nothing to do with CXL. Hence if we are doing generic naming, I think we should just always have the function available. Gating on CXL belongs at whatever called it. Which is the case already for cxl_rch_handle_error() which has a stub that doesn't call this for when CXL stuff isn't built.
Should just be a case of moving out of if the ifdef in aer.c as part of this patch.
Jonathan
Quoted 37 lines
---
Changes in
v13->v14:
- New patch--- drivers/pci/pcie/aer.c | 4 ++-- drivers/pci/pcie/portdrv.h | 9 +++++++++ 2 files changed, 11 insertions(+), 2 deletions(-)drivers/pci/pcie/aer.c +2
-2@@ -1166,7 +1166,7 @@ static bool is_cxl_mem_dev(struct pci_dev *dev)return true; } -static bool is_internal_error (struct aer_err_info *info) +bool is_aer_internal_error (struct aer_err_info *info) {if (info->severity ==AER_CORRECTABLE )return info->status &PCI_ERR_COR_INTERNAL ;@@ -1211,7 +1211,7 @@ static void cxl_rch_handle_error(struct pci_dev *dev, struct aer_err_info *info)* device driver. */ if (pci_pcie_type (dev) ==PCI_EXP_TYPE_RC_EC && -is_internal_error (info)) +is_aer_internal_error (info))pcie_walk_rcec (dev, cxl_rch_handle_error_iter, info); }drivers/pci/pcie/portdrv.h +0
-0
drivers/pci/pcie/portdrv.h static inline void pcie_pme_interrupt_enable(struct pci_dev *dev, bool en) {} · line 132
Quoted 10 lines
drivers/pci/pcie/portdrv.h +7
-0@@ -123,4 +123,13 @@ static inline void pcie_pme_interrupt_enable(struct pci_dev *dev, bool en) {}#endif /* !CONFIG_PCIE_PME */ struct device *pcie_port_find_device (struct pci_dev *dev,u32 service); + +struct aer_err_info ; + +#ifdef CONFIG_PCIEAER_CXL +bool is_aer_internal_error (struct aer_err_info *info); +#else +static inline bool is_aer_internal_error (struct aer_err_info *info) {return false; }
This seems odd. It's either an AER internal error or it isn't, whether or not CXL is enabled. That stubbing out should I think go up to the caller that can decide whether it cares or not.
Quoted 3 lines
drivers/pci/pcie/portdrv.h +2
-0+#endif /* CONFIG_PCIEAER_CXL */ +#endif /* _PORTDRV_H_ */
Re: [PATCH v14 10/34] PCI/AER: Update is_internal_error() to be non-static is_aer_internal_error()
Quoted 22 lines · Jonathan Cameron
Quoted 9 lines · On Wed, 14 Jan 2026 12:20:31 -0600 Terry Bowman <[email protected]> wrote:
The AER driver includes significant logic for handling CXL protocol errors. The AER driver will be updated in the future to separate the AER and CXL logic.
Rename the
is_internal_error()function tois_aer_internal_error()as it gives a more precise indication of the purpose. Makeis_aer_internal_error()non-static to allow for other PCI drivers to access.Signed-off-by Terry Bowman <[email protected]>Hi Terry,
I don't see it as sensible to have
is_aer_internal_error()return false if CXL is not built. That question has nothing to do with CXL. Hence if we are doing generic naming, I think we should just always have the function available. Gating on CXL belongs at whatever called it. Which is the case already forcxl_rch_handle_error()which has a stub that doesn't call this for when CXL stuff isn't built.
Quoted 2 lines
Should just be a case of moving out of if the ifdef in
aer.cas part of this patch.
I agree with the general sentiment, but not the conclusion, especially because this is a private detail. Linux has long ignored internal errors. The only reason to consider them now is because CXL decided to multiplex its error model on top of this oft-ignored feature of PCIe AER.
Specifically, portdrv.h is not in the global include namespace, this is a private detail of the only conumer of internal errors:drivers/pci/pcie/aer_cxl_{rch,vh}.c
At most we should have this as a comment to clarify:
/*
* Note, internal errors are only considered for the CXL error model,
* not for other implementations.
*/
...and the pci_aer_unmask_internal_errors() export should be:
EXPORT_SYMBOL_FOR_MODULES(pci_aer_unmask_internal_errors, "cxl_core")
...for the same reason. Steer folks away from thinking that it is open season for adding more internal error support.
Re: [PATCH v14 10/34] PCI/AER: Update is_internal_error() to be non-static is_aer_internal_error()
Quoted 17 lines · On Thu, Jan 15, 2026 at 12:42:36PM -0800, [email protected] wrote:
I agree with the general sentiment, but not the conclusion, especially because this is a private detail. Linux has long ignored internal errors. The only reason to consider them now is because CXL decided to multiplex its error model on top of this oft-ignored feature of PCIe AER.
Specifically,
portdrv.his not in the global include namespace, this is a private detail of the only conumer of internal errors:drivers/pci/pcie/aer_cxl_{rch,vh}.cAt most we should have this as a comment to clarify:
/*
* Note, internal errors are only considered for the CXL error model,
* not for other implementations.
*/
Quoted 6 lines
...and the
pci_aer_unmask_internal_errors()export should be:
EXPORT_SYMBOL_FOR_MODULES(pci_aer_unmask_internal_errors, "cxl_core")...for the same reason. Steer folks away from thinking that it is open season for adding more internal error support.
It's not like Internal Errors are a bad thing per se. They're a way to signal "other" errors besides the spec-defined ones.
As an example, and I'm keeping this in general terms to avoid devulging information about future products, a device possessing ECC RAM may raise a Correctable Internal Error when ECC successfully recovers from flipped bits because it allows alerting the user in advance that the device might need to be replaced in the near future. If ECC recovery fails, the device might try to use a reserved spare portion of RAM in lieu of the failing one and instruct the AER driver to recover through a bus reset. Such errors are not covered by the spec-defined types. Using the Internal Error type is the only possibility it seems.
My point is, there are valid (upcoming, not theoretical) use cases for Internal Errors and creating infrastructure in the kernel to take advantage of them is a good thing. Hence my continued pushing back on hiding or discouraging their use.
Thanks,
Lukas
Re: [PATCH v14 10/34] PCI/AER: Update is_internal_error() to be non-static is_aer_internal_error()
Quoted 28 lines · Lukas Wunner
Quoted 17 lines · On Thu, Jan 15, 2026 at 12:42:36PM -0800, [email protected] wrote:
I agree with the general sentiment, but not the conclusion, especially because this is a private detail. Linux has long ignored internal errors. The only reason to consider them now is because CXL decided to multiplex its error model on top of this oft-ignored feature of PCIe AER.
Specifically,
portdrv.his not in the global include namespace, this is a private detail of the only conumer of internal errors:drivers/pci/pcie/aer_cxl_{rch,vh}.cAt most we should have this as a comment to clarify:
/*
* Note, internal errors are only considered for the CXL error model,
* not for other implementations.
*/Quoted 6 lines
...and the
pci_aer_unmask_internal_errors()export should be:
EXPORT_SYMBOL_FOR_MODULES(pci_aer_unmask_internal_errors, "cxl_core")...for the same reason. Steer folks away from thinking that it is open season for adding more internal error support.
It's not like Internal Errors are a bad thing per se. They're a way to signal "other" errors besides the spec-defined ones.
Quoted 9 lines
As an example, and I'm keeping this in general terms to avoid devulging information about future products, a device possessing ECC RAM may raise a Correctable Internal Error when ECC successfully recovers from flipped bits because it allows alerting the user in advance that the device might need to be replaced in the near future. If ECC recovery fails, the device might try to use a reserved spare portion of RAM in lieu of the failing one and instruct the AER driver to recover through a bus reset. Such errors are not covered by the spec-defined types. Using the Internal Error type is the only possibility it seems.
The Internal Error type is a poor fit for that. This ECC RAM scenario is simply an internal device event, not a PCIe visible error case. Consider that CXL Memory Expanders are nothing if not "devices possessing ECC RAM" that may encounter correctable errors in that RAM. Yes, the user has need for those correctable errors to be reported, and no, PCIe AER has no reason to care about conveying those reports. CXL bypasses AER for internal ECC RAM events.
PCIe AER only notices device-internal ECC RAM events in the case where a PCIe transaction encounters an error. For example, a completer abort attempting to pull from bad RAM.
So if CXL saw no need to architect internal ECC events into AER, why does Xe think it is special in this regard?
The CXL solution is simply a typical device interrupt that notifies new entries in the device event log. See trace_cxl_dram() and trace_cxl_general_media() for that event handling.
Quoted 4 lines
My point is, there are valid (upcoming, not theoretical) use cases for Internal Errors and creating infrastructure in the kernel to take advantage of them is a good thing. Hence my continued pushing back on hiding or discouraging their use.
It is fine to look ahead, but I would not go so far as to pull in future requirements into a present patch set. Especially when those future requirements are suspect.
Re: [PATCH v14 10/34] PCI/AER: Update is_internal_error() to be non-static is_aer_internal_error()
Quoted 10 lines · On Thu, Jan 22, 2026 at 11:09:48AM -0800, [email protected] wrote:
Quoted 8 lines · Lukas Wunner
a device possessing ECC RAM may raise
a Correctable Internal Error when ECC successfully recovers from flipped bits because it allows alerting the user in advance that the device might need to be replaced in the near future. If ECC recovery fails, the device might try to use a reserved spare portion of RAM in lieu of the failing one and instruct the AER driver to recover through a bus reset. Such errors are not covered by the spec-defined types. Using the Internal Error type is the only possibility it seems.
Quoted 6 lines
The Internal Error type is a poor fit for that. This ECC RAM scenario is simply an internal device event, not a PCIe visible error case. Consider that CXL Memory Expanders are nothing if not "devices possessing ECC RAM" that may encounter correctable errors in that RAM. Yes, the user has need for those correctable errors to be reported, and no, PCIe AER has no reason to care about conveying those reports.
I'm not aware of a better PCIe spec-defined mechanism to report such errors besides AER (Advanced Error *Reporting*), so I'm not sure why you consider it a poor fit.
However, reporting corrected ECC errors is only half of the equation. As stated above, if the ECC error is not correctable, the device may choose to replace the faulty memory region with reserved spare memory, but then a reset is required to recover from the error. Precisely what the AER driver provides, so again I'm not sure why it's a poor fit.
Quoted 2 lines
So if CXL saw no need to architect internal ECC events into AER, why does Xe think it is special in this regard?
The most charitable interpretation is that it's just the first mover and others will follow. Well actually CXL is the first mover. ;)
Quoted 3 lines
The CXL solution is simply a typical device interrupt that notifies new entries in the device event log. See
trace_cxl_dram()andtrace_cxl_general_media()for that event handling.
This seems to be based on CPER, which is not part of the PCIe Base Spec. I can only guess that xe devices are intended to be used on non-ACPI platforms as well, which may have led to the decision to use a PCIe spec-defined mechanism.
Thanks,
Lukas
Re: [PATCH v14 10/34] PCI/AER: Update is_internal_error() to be non-static is_aer_internal_error()
Quoted 18 lines · Lukas Wunner
Quoted 10 lines · On Thu, Jan 22, 2026 at 11:09:48AM -0800, [email protected] wrote:
Quoted 8 lines · Lukas Wunner
a device possessing ECC RAM may raise
a Correctable Internal Error when ECC successfully recovers from flipped bits because it allows alerting the user in advance that the device might need to be replaced in the near future. If ECC recovery fails, the device might try to use a reserved spare portion of RAM in lieu of the failing one and instruct the AER driver to recover through a bus reset. Such errors are not covered by the spec-defined types. Using the Internal Error type is the only possibility it seems.Quoted 6 lines
The Internal Error type is a poor fit for that. This ECC RAM scenario is simply an internal device event, not a PCIe visible error case. Consider that CXL Memory Expanders are nothing if not "devices possessing ECC RAM" that may encounter correctable errors in that RAM. Yes, the user has need for those correctable errors to be reported, and no, PCIe AER has no reason to care about conveying those reports.
Quoted 3 lines
I'm not aware of a better PCIe spec-defined mechanism to report such errors besides AER (Advanced Error *Reporting*), so I'm not sure why you consider it a poor fit.
PCIe spec has no role defining the internal error model of devices. Linux has reason to not endorse a blurring of the lines of where the PCIe error model ends and the device-specific error model begins. CXL respects those boundaries, Xe is pushing the boundary.
Quoted 5 lines
However, reporting corrected ECC errors is only half of the equation. As stated above, if the ECC error is not correctable, the device may choose to replace the faulty memory region with reserved spare memory, but then a reset is required to recover from the error. Precisely what the AER driver provides, so again I'm not sure why it's a poor fit.
Again CXL has a model for this, those are the "post-package repair" events handled internally to the device / driver either transparently or user coordinated. No AER needed. In general devices have plenty of reasons that the driver determines they need to be reset, they do not need AER core help to reset themselves on error.
AER is there for link recovery.
Quoted 5 lines
Quoted 2 lines
So if CXL saw no need to architect internal ECC events into AER, why does Xe think it is special in this regard?
The most charitable interpretation is that it's just the first mover and others will follow. Well actually CXL is the first mover. ;)
...first mover that helps clarify the role of AER that just happens to match the status quo that PCIe AER core ignore internal errors.
Quoted 8 lines
Quoted 3 lines
The CXL solution is simply a typical device interrupt that notifies new entries in the device event log. See
trace_cxl_dram()andtrace_cxl_general_media()for that event handling.This seems to be based on CPER, which is not part of the PCIe Base Spec. I can only guess that xe devices are intended to be used on non-ACPI platforms as well, which may have led to the decision to use a PCIe spec-defined mechanism.
CPER is compatibility hack for operating systems that do not have native CXL drivers. The native support is just an interrupt fronting an event log retrieved with mailbox commands.
Re: [PATCH v14 10/34] PCI/AER: Update is_internal_error() to be non-static is_aer_internal_error()
Quoted 19 lines · On Thu, 22 Jan 2026 13:32:08 -0800 [email protected] wrote:
Quoted 18 lines · Lukas Wunner
Quoted 16 lines · On Thu, Jan 22, 2026 at 11:09:48AM -0800, [email protected] wrote:
Quoted 8 lines · Lukas Wunner
a device possessing ECC RAM may raise
a Correctable Internal Error when ECC successfully recovers from flipped bits because it allows alerting the user in advance that the device might need to be replaced in the near future. If ECC recovery fails, the device might try to use a reserved spare portion of RAM in lieu of the failing one and instruct the AER driver to recover through a bus reset. Such errors are not covered by the spec-defined types. Using the Internal Error type is the only possibility it seems.The Internal Error type is a poor fit for that. This ECC RAM scenario is simply an internal device event, not a PCIe visible error case. Consider that CXL Memory Expanders are nothing if not "devices possessing ECC RAM" that may encounter correctable errors in that RAM. Yes, the user has need for those correctable errors to be reported, and no, PCIe AER has no reason to care about conveying those reports.
Quoted 8 lines
Quoted 3 lines
I'm not aware of a better PCIe spec-defined mechanism to report such errors besides AER (Advanced Error *Reporting*), so I'm not sure why you consider it a poor fit.
PCIe spec has no role defining the internal error model of devices. Linux has reason to not endorse a blurring of the lines of where the PCIe error model ends and the device-specific error model begins. CXL respects those boundaries, Xe is pushing the boundary.
FWIW we have a bunch of older hardware where we could report this sort of error either via AER or via an MSI. After some push back years ago, we flipped them all to the MSI path. That includes stuff that triggers device resets. I don't think it caused us too much trouble to make that switch.
Quoted 28 lines
Quoted 6 lines
However, reporting corrected ECC errors is only half of the equation. As stated above, if the ECC error is not correctable, the device may choose to replace the faulty memory region with reserved spare memory, but then a reset is required to recover from the error. Precisely what the AER driver provides, so again I'm not sure why it's a poor fit.
Again CXL has a model for this, those are the "post-package repair" events handled internally to the device / driver either transparently or user coordinated. No AER needed. In general devices have plenty of reasons that the driver determines they need to be reset, they do not need AER core help to reset themselves on error.
AER is there for link recovery.
Quoted 5 lines
Quoted 2 lines
So if CXL saw no need to architect internal ECC events into AER, why does Xe think it is special in this regard?
The most charitable interpretation is that it's just the first mover and others will follow. Well actually CXL is the first mover. ;)
...first mover that helps clarify the role of AER that just happens to match the status quo that PCIe AER core ignore internal errors.
Quoted 4 lines
Quoted 3 lines
The CXL solution is simply a typical device interrupt that notifies new entries in the device event log. See
trace_cxl_dram()andtrace_cxl_general_media()for that event handling.
Quoted 8 lines
Quoted 4 lines
This seems to be based on CPER, which is not part of the PCIe Base Spec. I can only guess that xe devices are intended to be used on non-ACPI platforms as well, which may have led to the decision to use a PCIe spec-defined mechanism.
CPER is compatibility hack for operating systems that do not have native CXL drivers. The native support is just an interrupt fronting an event log retrieved with mailbox commands.
Just as a side note, CXL also has FW specific interrupts with a negotation process for whether they are used, or MSI-X is used for event queues.
Jonathan
Re: [PATCH v14 10/34] PCI/AER: Update is_internal_error() to be non-static is_aer_internal_error()
In reply to Terry BowmanQuoted 7 lines · Terry Bowman
The AER driver includes significant logic for handling CXL protocol errors. The AER driver will be updated in the future to separate the AER and CXL logic.
Rename the
is_internal_error()function tois_aer_internal_error()as it gives a more precise indication of the purpose. Makeis_aer_internal_error()non-static to allow for other PCI drivers to access.
Not even sure this rename is needed given that it is private to drivers/pci/pcie/ and the sharing is only for cxl_{rch,vh}.c, not for "other PCI drivers". Consistent with the idea that internal errors are not going to become a first-class citizen let us keep this a CXL-only consideration.
I'll update the changelog to drop the "other PCI drivers" comment.
Re: [PATCH v14 10/34] PCI/AER: Update is_internal_error() to be non-static is_aer_internal_error()
Quoted 9 lines · On 1/19/2026 8:20 PM, [email protected] wrote:
Quoted 7 lines · Terry Bowman
The AER driver includes significant logic for handling CXL protocol errors. The AER driver will be updated in the future to separate the AER and CXL logic.
Rename the
is_internal_error()function tois_aer_internal_error()as it gives a more precise indication of the purpose. Makeis_aer_internal_error()non-static to allow for other PCI drivers to access.
Quoted 7 lines
Not even sure this rename is needed given that it is private to
drivers/pci/pcie/and the sharing is only for cxl_{rch,vh}.c, not for "other PCI drivers". Consistent with the idea that internal errors are not going to become a first-class citizen let us keep this a CXL-only consideration.I'll update the changelog to drop the "other PCI drivers" comment.
The name choice was addressed by Bjorn here:
https://lore.kernel.org/linux-cxl/20251208180624.GA3300935@bhelgaas/
Terry
Re: [PATCH v14 10/34] PCI/AER: Update is_internal_error() to be non-static is_aer_internal_error()
Quoted 16 lines · Terry
Quoted 15 lines · On 1/19/2026 8:20 PM, [email protected] wrote:
Quoted 7 lines · Terry Bowman
The AER driver includes significant logic for handling CXL protocol errors. The AER driver will be updated in the future to separate the AER and CXL logic.
Rename the
is_internal_error()function tois_aer_internal_error()as it gives a more precise indication of the purpose. Makeis_aer_internal_error()non-static to allow for other PCI drivers to access.Not even sure this rename is needed given that it is private to
drivers/pci/pcie/and the sharing is only for cxl_{rch,vh}.c, not for "other PCI drivers". Consistent with the idea that internal errors are not going to become a first-class citizen let us keep this a CXL-only consideration.
Quoted 5 lines
Quoted 1 line
I'll update the changelog to drop the "other PCI drivers" comment.
The name choice was addressed by Bjorn here:
https://lore.kernel.org/linux-cxl/20251208180624.GA3300935@bhelgaas/
Thanks, yes, I only folded in the following changes to the changelog:
10: 417535d35e9f ! 11: 098f14e1d884 PCI/AER: Update is_internal_error() to be non-static is_aer_internal_error()
@@ Commit message
logic.
Rename the is_internal_error() function to is_aer_internal_error() as it
- gives a more precise indication of the purpose. Make is_aer_internal_error()
- non-static to allow for other PCI drivers to access.
+ gives a more precise indication of the purpose. Make
+ is_aer_internal_error() non-static to allow for the 2 different CXL
+ topology error model implementations (RCH and VH) to share this helper.
-
- ---
-
- Changes in v13->v14:
- - New patch
+ Link: https://patch.msgid.link/[email protected]
+ Signed-off-by: Dan Williams <[email protected]>
Re: [PATCH v14 10/34] PCI/AER: Update is_internal_error() to be non-static is_aer_internal_error()
In reply to Terry BowmanQuoted 9 lines · Terry Bowman
The AER driver includes significant logic for handling CXL protocol errors. The AER driver will be updated in the future to separate the AER and CXL logic.
Rename the
is_internal_error()function tois_aer_internal_error()as it gives a more precise indication of the purpose. Makeis_aer_internal_error()non-static to allow for other PCI drivers to access.Signed-off-by Terry Bowman <[email protected]>
Personally I would put "aer_" at the beginning, i.e., "aer_is_internal_error()" to match other AER functions. But either is OK.
Quoted 52 lines
---
Changes in
v13->v14:
- New patch--- drivers/pci/pcie/aer.c | 4 ++-- drivers/pci/pcie/portdrv.h | 9 +++++++++ 2 files changed, 11 insertions(+), 2 deletions(-)drivers/pci/pcie/aer.c +2
-2@@ -1166,7 +1166,7 @@ static bool is_cxl_mem_dev(struct pci_dev *dev)return true; } -static bool is_internal_error (struct aer_err_info *info) +bool is_aer_internal_error (struct aer_err_info *info) {if (info->severity ==AER_CORRECTABLE )return info->status &PCI_ERR_COR_INTERNAL ;@@ -1211,7 +1211,7 @@ static void cxl_rch_handle_error(struct pci_dev *dev, struct aer_err_info *info)* device driver. */ if (pci_pcie_type (dev) ==PCI_EXP_TYPE_RC_EC && -is_internal_error (info)) +is_aer_internal_error (info))pcie_walk_rcec (dev, cxl_rch_handle_error_iter, info); }drivers/pci/pcie/portdrv.h +9
-0@@ -123,4 +123,13 @@ static inline void pcie_pme_interrupt_enable(struct pci_dev *dev, bool en) {}#endif /* !CONFIG_PCIE_PME */ struct device *pcie_port_find_device (struct pci_dev *dev,u32 service); + +struct aer_err_info ; + +#ifdef CONFIG_PCIEAER_CXL +bool is_aer_internal_error (struct aer_err_info *info); +#else +static inline bool is_aer_internal_error (struct aer_err_info *info) {return false; } +#endif /* CONFIG_PCIEAER_CXL */ +#endif /* _PORTDRV_H_ */ Signature · 1 line
2.34.1
Re: [PATCH v14 10/34] PCI/AER: Update is_internal_error() to be non-static is_aer_internal_error()
In reply to Terry BowmanQuoted 5 lines · Terry Bowman
+++
b/drivers/pci/pcie/portdrv.hdrivers/pci/pcie/portdrv.h +0
-0@@ -123,4 +123,13 @@ static inline void pcie_pme_interrupt_enable(struct pci_dev *dev, bool en) {}#endif /* !CONFIG_PCIE_PME */ struct device *pcie_port_find_device (struct pci_dev *dev,u32 service);
drivers/pci/pcie/portdrv.h static inline void pcie_pme_interrupt_enable(struct pci_dev *dev, bool en) {} · line 135
Quoted 10 lines
drivers/pci/pcie/portdrv.h +9
-0+ +struct aer_err_info ; + +#ifdef CONFIG_PCIEAER_CXL +bool is_aer_internal_error (struct aer_err_info *info); +#else +static inline bool is_aer_internal_error (struct aer_err_info *info) {return false; } +#endif /* CONFIG_PCIEAER_CXL */ +#endif /* _PORTDRV_H_ */
This has no relation to portdrv and so it's unclear and confusing why it was placed in portdrv.h.
In your pending CXL AER v20 series, you're adding more declarations here.
We keep declarations that are internal to the PCI core in drivers/pci/pci.h. However since this is just an interface between aer.c and aer_cxl_rch.c, maybe drivers/pci/pcie/aer_cxl.h would make sense.
Thanks,
Lukas
Re: [PATCH v14 10/34] PCI/AER: Update is_internal_error() to be non-static is_aer_internal_error()
Quoted 22 lines · Lukas Wunner
Quoted 5 lines · Terry Bowman
+++
b/drivers/pci/pcie/portdrv.hdrivers/pci/pcie/portdrv.h +0
-0@@ -123,4 +123,13 @@ static inline void pcie_pme_interrupt_enable(struct pci_dev *dev, bool en) {}#endif /* !CONFIG_PCIE_PME */ struct device *pcie_port_find_device (struct pci_dev *dev,u32 service);drivers/pci/pcie/portdrv.h static inline void pcie_pme_interrupt_enable(struct pci_dev *dev, bool en) {} · line 135
Quoted 10 lines
drivers/pci/pcie/portdrv.h +9
-0+ +struct aer_err_info ; + +#ifdef CONFIG_PCIEAER_CXL +bool is_aer_internal_error (struct aer_err_info *info); +#else +static inline bool is_aer_internal_error (struct aer_err_info *info) {return false; } +#endif /* CONFIG_PCIEAER_CXL */ +#endif /* _PORTDRV_H_ */ This has no relation to portdrv and so it's unclear and confusing why it was placed in
portdrv.h.In your pending CXL AER v20 series, you're adding more declarations here.
Quoted 8 lines
We keep declarations that are internal to the PCI core in
drivers/pci/pci.h. However since this is just an interface betweenaer.candaer_cxl_rch.c, maybedrivers/pci/pcie/aer_cxl.hwould make sense.Thanks,
Lukas
Agreed, it doesn't belong in portdrv.h. I'll introduce drivers/pci/pcie/aer_cxl.h for the aer.c <-> aer_cxl_rch.c interface and move is_aer_internal_error() there.
I'll also move the additional declarations that the v20 series currently adds to portdrv.h into the same header so they don't reintroduce this.
-Terry
PATCH v14 11/34 PCI/AER: Move CXL RCH error handling to aer_cxl_rch.c
In reply to Terry BowmanThe Restricted CXL Host (RCH) AER error handling logic currently resides in the AER driver file, aer.c. CXL specific changes conditionally compiled using #ifdefs.
Improve the AER driver maintainability by separating the RCH specific logic from the AER driver's core functionality and removing the ifdefs. Introduce drivers/pci/pcie/aer_cxl_rch.c for moving the RCH AER logic into. Conditionally compile the file using the CONFIG_CXL_RCH_RAS Kconfig.
Move the CXL logic into the new file but leave CXL helper function is_internal_error() in aer.c for now as it will be moved in future patch for CXL Virtual Hierarchy handling.
To maintain compilation after the move other changes are required. Change cxl_rch_handle_error(), cxl_rch_enable_rcec(), and is_internal_error() to be non-static inorder for accessing from the AER driver.
Update the new file with the SPDX and 2023 AMD copyright notations because
the RCH bits were initially contributed in 2023 by AMD. See commit:
commit 0a867568bb0d ("PCI/AER: Forward RCH downstream port-detected errors to the CXL.mem dev handler")
---
Changes in v13->v14:
- Add review-by and signed-off for Dan
- Commit message fixup (Dan)
- Update commit message with use-case description (Dan, Lukas)
- Make cxl_error_is_native() static (Dan)
Changes in v12->v13:
- Add forward declararation of 'struct aer_err_info' in pci/pci.h (Terry)
- Changed copyright date from 2025 to 2023 (Jonathan)
- Add David Jiang's, Jonathan's, and Ben's review-by
- Re-add 'struct aer_err_info' (Bot)
Changes in v11->v12:
- Rename drivers/pci/pcie/cxl_rch.c to drivers/pci/pcie/aer_cxl_rch.c (Lukas)
- Removed forward declararation of 'struct aer_err_info' in pci/pci.h (Terry)
Changes in v10->v11:
- Remove changes in code-split and move to earlier, new patch
- Add #include <linux/bitfield.h> to cxl_ras.c
- Move cxl_rch_handle_error() & cxl_rch_enable_rcec() declarations from pci.h to aer.h, more localized.
- Introduce CONFIG_CXL_RCH_RAS, includes Makefile changes, ras.c ifdef changes
--- drivers/pci/pcie/Makefile | 1 + drivers/pci/pcie/aer.c | 99 +----------------------------- drivers/pci/pcie/aer_cxl_rch.c | 106 +++++++++++++++++++++++++++++++++ drivers/pci/pcie/portdrv.h | 9 ++- 4 files changed, 114 insertions(+), 101 deletions(-) create mode 100644 drivers/pci/pcie/aer_cxl_rch.c
drivers/pci/pcie/Makefile +1 -0
@@ -8,6 +8,7 @@ obj-$(CONFIG_PCIEPORTBUS) += pcieportdrv.o bwctrl.o obj-y += aspm.o obj-$(CONFIG_PCIEAER) += aer.o err.o tlp.o +obj-$(CONFIG_CXL_RAS) += aer_cxl_rch.o obj-$(CONFIG_PCIEAER_INJECT) += aer_inject.o obj-$(CONFIG_PCIE_PME) += pme.o obj-$(CONFIG_PCIE_DPC) += dpc.o
drivers/pci/pcie/aer.c +1 -98
111 lines not on this page · raw message
drivers/pci/pcie/aer_cxl_rch.c new file +106 -0
106 lines not on this page · raw message
drivers/pci/pcie/portdrv.h +6 -3
@@ -126,10 +126,13 @@ struct device *pcie_port_find_device(struct pci_dev *dev, u32 service);struct aer_err_info ; -#ifdef CONFIG_PCIEAER_CXL +#ifdef CONFIG_CXL_RAS bool is_aer_internal_error (struct aer_err_info *info); +void cxl_rch_handle_error (struct pci_dev *dev,struct aer_err_info *info); +void cxl_rch_enable_rcec (struct pci_dev *rcec);#else static inline bool is_aer_internal_error (struct aer_err_info *info) {return false; } -#endif /* CONFIG_PCIEAER_CXL */ - +static inline void cxl_rch_handle_error (struct pci_dev *dev,struct aer_err_info *info) { } +static inline void cxl_rch_enable_rcec (struct pci_dev *rcec) { } +#endif /* CONFIG_CXL_RAS */ #endif /* _PORTDRV_H_ */
Signature · 1 line
2.34.1
Re: [PATCH v14 11/34] PCI/AER: Move CXL RCH error handling to aer_cxl_rch.c
…
Quoted 2 lines
+++
b/drivers/accel/thames/thames_gem.c(quoted diff) +0
-0@@ -0,0 +1,353 @@
…
Quoted 2 lines
(quoted diff) +2
-0+static int cxl_rch_handle_error_iter(struct pci_dev *dev, void *data) +{
…
Quoted 3 lines
drivers/pci/pcie/aer_cxl_rch.c new file +3
-0+device_lock (&dev->dev); + + err_handler = dev->driver ? dev->driver->err_handler :NULL ;
…
drivers/pci/pcie/aer_cxl_rch.c line 63
Quoted 4 lines
drivers/pci/pcie/aer_cxl_rch.c new file +4
-0+out: +device_unlock (&dev->dev); +return 0 ; +}
…
Under which circumstances would you become interested to apply a statement like “guard(device)(&dev->dev);”?
https://elixir.bootlin.com/linux/v6.19-rc5/source/include/linux/device.h#L913
Regards,
Markus
Re: [PATCH v14 11/34] PCI/AER: Move CXL RCH error handling to aer_cxl_rch.c
Quoted 17 lines · Markus Elfring
…
Quoted 2 lines
+++
b/drivers/accel/thames/thames_gem.c(quoted diff) +0
-0@@ -0,0 +1,353 @@…
Quoted 2 lines
(quoted diff) +2
-0+static int cxl_rch_handle_error_iter(struct pci_dev *dev, void *data) +{…
Quoted 3 lines
drivers/pci/pcie/aer_cxl_rch.c new file +3
-0+device_lock (&dev->dev); + + err_handler = dev->driver ? dev->driver->err_handler :NULL ;…
Quoted 4 lines
drivers/pci/pcie/aer_cxl_rch.c new file +4
-0+out: +device_unlock (&dev->dev); +return 0 ; +}…
Quoted 6 lines
Under which circumstances would you become interested to apply a statement like “guard(device)(&dev->dev);”?
https://elixir.bootlin.com/linux/v6.19-rc5/source/include/linux/device.h#L913Regards,
Markus
Hi Markus,
This patch was a move (leaving the lock/unlock calls intact). I change the lock/unlock to guard() in the following patch, after the move:
PCI/AER: Use guard() in cxl_rch_handle_error_iter()
-Terry
Re: [PATCH v14 11/34] PCI/AER: Move CXL RCH error handling to aer_cxl_rch.c
In reply to Terry BowmanQuoted 9 lines · Terry Bowman
The Restricted CXL Host (RCH) AER error handling logic currently resides in the AER driver file,
aer.c. CXL specific changes conditionally compiled using #ifdefs.Improve the AER driver maintainability by separating the RCH specific logic from the AER driver's core functionality and removing the ifdefs. Introduce
drivers/pci/pcie/aer_cxl_rch.cfor moving the RCH AER logic into. Conditionally compile the file using theCONFIG_CXL_RCH_RASKconfig.
Quoted 7 lines
Move the CXL logic into the new file but leave CXL helper function
is_internal_error()inaer.cfor now as it will be moved in future patch for CXL Virtual Hierarchy handling.To maintain compilation after the move other changes are required. Change
cxl_rch_handle_error(),cxl_rch_enable_rcec(), andis_internal_error()to be non-static inorder for accessing from the AER driver.
s/inorder for accessing from the/so they can be used by the/
Quoted 9 lines
Update the new file with the SPDX and 2023 AMD copyright notations because
the RCH bits were initially contributed in 2023 by AMD. See commit:
commit0a867568bb0d("PCI/AER: Forward RCH downstream port-detected errors to the CXL.mem dev handler")Signed-off-by Terry Bowman <[email protected]>Reviewed-by Dave Jiang <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Ben Cheatham <[email protected]>Reviewed-by Dan Williams <[email protected]>
Quoted 298 lines
---
Changes in
v13->v14:
- Add review-by and signed-off for Dan
- Commit message fixup (Dan)
- Update commit message with use-case description (Dan, Lukas)
- Makecxl_error_is_native()static (Dan)Changes in
v12->v13:
- Add forward declararation of 'structaer_err_info' inpci/pci.h(Terry)
- Changed copyright date from 2025 to 2023 (Jonathan)
- Add David Jiang's, Jonathan's, and Ben's review-by
- Re-add 'structaer_err_info' (Bot)Changes in
v11->v12:
- Renamedrivers/pci/pcie/cxl_rch.ctodrivers/pci/pcie/aer_cxl_rch.c(Lukas)
- Removed forward declararation of 'structaer_err_info' inpci/pci.h(Terry)Changes in
v10->v11:
- Remove changes in code-split and move to earlier, new patch
- Add #include <linux/bitfield.h> tocxl_ras.c
- Movecxl_rch_handle_error()&cxl_rch_enable_rcec()declarations frompci.htoaer.h, more localized.
- IntroduceCONFIG_CXL_RCH_RAS, includesMakefilechanges,ras.cifdef changes--- drivers/pci/pcie/Makefile | 1 + drivers/pci/pcie/aer.c | 99 +----------------------------- drivers/pci/pcie/aer_cxl_rch.c | 106 +++++++++++++++++++++++++++++++++ drivers/pci/pcie/portdrv.h | 9 ++- 4 files changed, 114 insertions(+), 101 deletions(-) create mode 100644 drivers/pci/pcie/aer_cxl_rch.cdrivers/pci/pcie/Makefile +1
-0@@ -8,6 +8,7 @@ obj-$(CONFIG_PCIEPORTBUS) += pcieportdrv.o bwctrl.o obj-y += aspm.o obj-$(CONFIG_PCIEAER) += aer.o err.o tlp.o +obj-$(CONFIG_CXL_RAS) += aer_cxl_rch.o obj-$(CONFIG_PCIEAER_INJECT) += aer_inject.o obj-$(CONFIG_PCIE_PME) += pme.o obj-$(CONFIG_PCIE_DPC) += dpc.odrivers/pci/pcie/aer.c +1
-98@@ -1145,27 +1145,7 @@ void pci_aer_unmask_internal_errors(struct pci_dev *dev) }EXPORT_SYMBOL_GPL (pci_aer_unmask_internal_errors); -#ifdef CONFIG_PCIEAER_CXL -static bool is_cxl_mem_dev (struct pci_dev *dev) -{ -/* -* The capability, status, and control fields in Device 0, -* Function 0 DVSEC control the CXL functionality of the -* entire device (CXL 3.0, 8.1.3). -*/ -if (dev->devfn !=PCI_DEVFN (0 ,0 )) -return false; - -/* -* CXL Memory Devices must have the 502h class code set (CXL -* 3.0, 8.1.12.1). -*/ -if ((dev->class >>8 ) !=PCI_CLASS_MEMORY_CXL ) -return false; - -return true; -} - +#ifdef CONFIG_CXL_RAS bool is_aer_internal_error (struct aer_err_info *info) {if (info->severity ==AER_CORRECTABLE )@@ -1173,83 +1153,6 @@ bool is_aer_internal_error(struct aer_err_info *info) return info->status & PCI_ERR_UNC_INTN; } - -static int cxl_rch_handle_error_iter (struct pci_dev *dev,void *data) -{ -struct aer_err_info *info = (struct aer_err_info *)data; -const struct pci_error_handlers *err_handler; - -if (!is_cxl_mem_dev (dev) || !pcie_aer_is_native (dev)) -return 0 ; - -/* Protect dev->driver */ -device_lock (&dev->dev); - - err_handler = dev->driver ? dev->driver->err_handler :NULL ; -if (!err_handler) - goto out; - -if (info->severity ==AER_CORRECTABLE ) { -if (err_handler->cor_error_detected) - err_handler->cor_error_detected (dev); - }else if (err_handler->error_detected) { -if (info->severity ==AER_NONFATAL ) - err_handler->error_detected (dev, pci_channel_io_normal); -else if (info->severity ==AER_FATAL ) - err_handler->error_detected (dev, pci_channel_io_frozen); - } -out: -device_unlock (&dev->dev); -return 0 ; -} - -static void cxl_rch_handle_error (struct pci_dev *dev,struct aer_err_info *info) -{ -/* -* Internal errors of an RCEC indicate an AER error in an -* RCH's downstream port. Check and handle them in the CXL.mem -* device driver. -*/ -if (pci_pcie_type (dev) ==PCI_EXP_TYPE_RC_EC && -is_aer_internal_error (info)) -pcie_walk_rcec (dev, cxl_rch_handle_error_iter, info); -} - -static int handles_cxl_error_iter (struct pci_dev *dev,void *data) -{ -bool *handles_cxl = data; - -if (!*handles_cxl) - *handles_cxl =is_cxl_mem_dev (dev) &&pcie_aer_is_native (dev); - -/* Non-zero terminates iteration */ -return *handles_cxl; -} - -static bool handles_cxl_errors (struct pci_dev *rcec) -{ -bool handles_cxl = false; - -if (pci_pcie_type (rcec) ==PCI_EXP_TYPE_RC_EC && -pcie_aer_is_native (rcec)) -pcie_walk_rcec (rcec, handles_cxl_error_iter, &handles_cxl); - -return handles_cxl; -} - -static void cxl_rch_enable_rcec (struct pci_dev *rcec) -{ -if (!handles_cxl_errors (rcec)) -return ; - -pci_aer_unmask_internal_errors (rcec); -pci_info (rcec,"CXL: Internal errors unmasked" ); -} - -#else -static inline void cxl_rch_enable_rcec (struct pci_dev *dev) { } -static inline void cxl_rch_handle_error (struct pci_dev *dev, -struct aer_err_info *info) { }#endif /** drivers/pci/pcie/aer_cxl_rch.c new file +106
-0@@ -0,0 +1,106 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(c) 2023 AMD Corporation. All rights reserved. */ + +#include <linux/pci.h> +#include <linux/aer.h> +#include <linux/bitfield.h> +#include "../pci.h" +#include "portdrv.h" + +static bool is_cxl_mem_dev (struct pci_dev *dev) +{ +/* +* The capability, status, and control fields in Device 0, +* Function 0 DVSEC control the CXL functionality of the +* entire device (CXL 3.0, 8.1.3). +*/ +if (dev->devfn !=PCI_DEVFN (0 ,0 )) +return false; + +/* +* CXL Memory Devices must have the 502h class code set (CXL +* 3.0, 8.1.12.1). +*/ +if ((dev->class >>8 ) !=PCI_CLASS_MEMORY_CXL ) +return false; + +return true; +} + +static bool cxl_error_is_native (struct pci_dev *dev) +{ +struct pci_host_bridge *host =pci_find_host_bridge (dev->bus); + +return (pcie_ports_native || host->native_aer); +} + +static int cxl_rch_handle_error_iter (struct pci_dev *dev,void *data) +{ +struct aer_err_info *info = (struct aer_err_info *)data; +const struct pci_error_handlers *err_handler; + +if (!is_cxl_mem_dev (dev) || !cxl_error_is_native (dev)) +return 0 ; + +device_lock (&dev->dev); + + err_handler = dev->driver ? dev->driver->err_handler :NULL ; +if (!err_handler) + goto out; + +if (info->severity ==AER_CORRECTABLE ) { +if (err_handler->cor_error_detected) + err_handler->cor_error_detected (dev); + }else if (err_handler->error_detected) { +if (info->severity ==AER_NONFATAL ) + err_handler->error_detected (dev, pci_channel_io_normal); +else if (info->severity ==AER_FATAL ) + err_handler->error_detected (dev, pci_channel_io_frozen); + } +out: +device_unlock (&dev->dev); +return 0 ; +} + +void cxl_rch_handle_error (struct pci_dev *dev,struct aer_err_info *info) +{ +/* +* Internal errors of an RCEC indicate an AER error in an +* RCH's downstream port. Check and handle them in the CXL.mem +* device driver. +*/ +if (pci_pcie_type (dev) ==PCI_EXP_TYPE_RC_EC && +is_aer_internal_error (info)) +pcie_walk_rcec (dev, cxl_rch_handle_error_iter, info); +} + +static int handles_cxl_error_iter (struct pci_dev *dev,void *data) +{ +bool *handles_cxl = data; + +if (!*handles_cxl) + *handles_cxl =is_cxl_mem_dev (dev) &&cxl_error_is_native (dev); + +/* Non-zero terminates iteration */ +return *handles_cxl; +} + +static bool handles_cxl_errors (struct pci_dev *rcec) +{ +bool handles_cxl = false; + +if (pci_pcie_type (rcec) ==PCI_EXP_TYPE_RC_EC && +pcie_aer_is_native (rcec)) +pcie_walk_rcec (rcec, handles_cxl_error_iter, &handles_cxl); + +return handles_cxl; +} + +void cxl_rch_enable_rcec (struct pci_dev *rcec) +{ +if (!handles_cxl_errors (rcec)) +return ; + +pci_aer_unmask_internal_errors (rcec); +pci_info (rcec,"CXL: Internal errors unmasked" ); +}drivers/pci/pcie/portdrv.h +6
-3@@ -126,10 +126,13 @@ struct device *pcie_port_find_device(struct pci_dev *dev, u32 service);struct aer_err_info ; -#ifdef CONFIG_PCIEAER_CXL +#ifdef CONFIG_CXL_RAS bool is_aer_internal_error (struct aer_err_info *info); +void cxl_rch_handle_error (struct pci_dev *dev,struct aer_err_info *info); +void cxl_rch_enable_rcec (struct pci_dev *rcec);#else static inline bool is_aer_internal_error (struct aer_err_info *info) {return false; } -#endif /* CONFIG_PCIEAER_CXL */ - +static inline void cxl_rch_handle_error (struct pci_dev *dev,struct aer_err_info *info) { } +static inline void cxl_rch_enable_rcec (struct pci_dev *rcec) { } +#endif /* CONFIG_CXL_RAS */ #endif /* _PORTDRV_H_ */ Signature · 1 line
2.34.1
PATCH v14 12/34 PCI/AER: Use guard() in cxl_rch_handle_error_iter()
In reply to Terry Bowmancxl_rch_handle_error_iter() includes a call to device_lock() using a goto for multiple return paths. Improve readability and maintainability by using the guard() lock variant.
---
Changes in v13 -> v14:
- Add review-by for Jonathan, Dave Jiang, Dan WIlliams, and Bjorn
- Remove cleanup.h (Jonathan)
- Reverted comment removal (Bjorn)
- Move this patch after pci/pcie/aer_cxl_rch.c creation (Bjorn)
Changes in v12 -> v13:
- New patch
--- drivers/pci/pcie/aer_cxl_rch.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
drivers/pci/pcie/aer_cxl_rch.c +2 -4
@@ -42,11 +42,11 @@ static int cxl_rch_handle_error_iter(struct pci_dev *dev, void *data)if (!is_cxl_mem_dev (dev) || !cxl_error_is_native (dev))return 0 ; -device_lock (&dev->dev); +guard (device)(&dev->dev); err_handler = dev->driver ? dev->driver->err_handler :NULL ;if (!err_handler) - goto out; +return 0 ;if (info->severity ==AER_CORRECTABLE ) {if (err_handler->cor_error_detected)
@@ -57,8 +57,6 @@ static int cxl_rch_handle_error_iter(struct pci_dev *dev, void *data)else if (info->severity ==AER_FATAL ) err_handler->error_detected (dev, pci_channel_io_frozen); } -out: -device_unlock (&dev->dev);return 0 ; }
Signature · 1 line
2.34.1
Re: [PATCH v14 12/34] PCI/AER: Use guard() in cxl_rch_handle_error_iter()
Quoted 10 lines · On Wed, 14 Jan 2026 12:20:33 -0600 Terry Bowman <[email protected]> wrote:
cxl_rch_handle_error_iter()includes a call todevice_lock()using a goto for multiple return paths. Improve readability and maintainability by using theguard()lock variant.Signed-off-by Terry Bowman <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Dave Jiang <[email protected]>Reviewed-by Dan Williams <[email protected]>Acked-by Bjorn Helgaas <[email protected]>
Quoted 5 lines
---
Changes in v13 -> v14:
- Add review-by for Jonathan, Dave Jiang, Dan WIlliams, and Bjorn
- Removecleanup.h(Jonathan)
I'm confused. I asked you to add the include (it wasn't there to be removed!)
Quoted 36 lines
- Reverted comment removal (Bjorn)
- Move this patch afterpci/pcie/aer_cxl_rch.ccreation (Bjorn)Changes in v12 -> v13:
- New patch--- drivers/pci/pcie/aer_cxl_rch.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)drivers/pci/pcie/aer_cxl_rch.c +2
-4@@ -42,11 +42,11 @@ static int cxl_rch_handle_error_iter(struct pci_dev *dev, void *data)if (!is_cxl_mem_dev (dev) || !cxl_error_is_native (dev))return 0 ; -device_lock (&dev->dev); +guard (device)(&dev->dev); err_handler = dev->driver ? dev->driver->err_handler :NULL ;if (!err_handler) - goto out; +return 0 ;if (info->severity ==AER_CORRECTABLE ) {if (err_handler->cor_error_detected)@@ -57,8 +57,6 @@ static int cxl_rch_handle_error_iter(struct pci_dev *dev, void *data)else if (info->severity ==AER_FATAL ) err_handler->error_detected (dev, pci_channel_io_frozen); } -out: -device_unlock (&dev->dev);return 0 ; }
PATCH v14 13/34 PCI/AER: Replace PCIEAER_CXL symbol with CXL_RAS
In reply to Terry BowmanFrom: Dan Williams <[email protected]>
One of the primary reasons for the CXL driver to exist is to perform error handling. If both PCIEAER and CXL are enabled then light up CXL error handling as well. The work to remove CONFIG_PCIEAER_CXL started in:
commit 4ae6ae66649c ("cxl/pci: Remove CXL VH handling in CONFIG_PCIEAER_CXL conditional blocks from core/pci.c")
Finish that off with conditionally compiling all CXL RAS related helpers with CONFIG_CXL_RAS.
----
Changes in v13->v14:
- New commit
--- drivers/cxl/Kconfig | 2 +- drivers/pci/pcie/Kconfig | 9 --------- 2 files changed, 1 insertion(+), 10 deletions(-)
drivers/cxl/Kconfig +1 -1
@@ -235,6 +235,6 @@ config CXL_MCE config CXL_RAS def_bool y - depends on ACPI_APEI_GHES && PCIEAER && CXL_PCI + depends on ACPI_APEI_GHES && PCIEAER && CXL_BUS endif
drivers/pci/pcie/Kconfig +0 -9
@@ -49,15 +49,6 @@ config PCIEAER_INJECT gotten from: https://github.com/intel/aer-inject.git -config PCIEAER_CXL - bool "PCI Express CXL RAS support" - default y - depends on PCIEAER && CXL_PCI - help - Enables CXL error handling. - - If unsure, say Y. - # # PCI Express ECRC #
Signature · 1 line
2.34.1
Re: [PATCH v14 13/34] PCI/AER: Replace PCIEAER_CXL symbol with CXL_RAS
Quoted 6 lines · On Wed, 14 Jan 2026 12:20:34 -0600 Terry Bowman <[email protected]> wrote:
From: Dan Williams <[email protected]>
One of the primary reasons for the CXL driver to exist is to perform error handling. If both PCIEAER and CXL are enabled then light up CXL error handling as well. The work to remove
CONFIG_PCIEAER_CXLstarted in:
Quoted 7 lines
commit
4ae6ae66649c("cxl/pci: Remove CXL VH handling inCONFIG_PCIEAER_CXLconditional blocks fromcore/pci.c")Finish that off with conditionally compiling all CXL RAS related helpers with
CONFIG_CXL_RAS.Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>
Re: [PATCH v14 13/34] PCI/AER: Replace PCIEAER_CXL symbol with CXL_RAS
In reply to Terry BowmanQuoted 6 lines · Terry Bowman
From: Dan Williams <[email protected]>
One of the primary reasons for the CXL driver to exist is to perform error handling. If both PCIEAER and CXL are enabled then light up CXL error handling as well. The work to remove
CONFIG_PCIEAER_CXLstarted in:
Quoted 7 lines
commit
4ae6ae66649c("cxl/pci: Remove CXL VH handling inCONFIG_PCIEAER_CXLconditional blocks fromcore/pci.c")Finish that off with conditionally compiling all CXL RAS related helpers with
CONFIG_CXL_RAS.Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>
Quoted 42 lines
----
Changes in
v13->v14:
- New commit--- drivers/cxl/Kconfig | 2 +- drivers/pci/pcie/Kconfig | 9 --------- 2 files changed, 1 insertion(+), 10 deletions(-)drivers/cxl/Kconfig +1
-1@@ -235,6 +235,6 @@ config CXL_MCE config CXL_RAS def_bool y - depends on ACPI_APEI_GHES && PCIEAER && CXL_PCI + depends on ACPI_APEI_GHES && PCIEAER && CXL_BUS endifdrivers/pci/pcie/Kconfig +0
-9@@ -49,15 +49,6 @@ config PCIEAER_INJECT gotten from: https://github.com/intel/aer-inject.git -config PCIEAER_CXL - bool "PCI Express CXL RAS support" - default y - depends on PCIEAER && CXL_PCI - help - Enables CXL error handling. - - If unsure, say Y. - # # PCI Express ECRC #
Dave Jiang
maintainerRe: [PATCH v14 13/34] PCI/AER: Replace PCIEAER_CXL symbol with CXL_RAS
In reply to Terry BowmanQuoted 6 lines · Terry Bowman
From: Dan Williams <[email protected]>
One of the primary reasons for the CXL driver to exist is to perform error handling. If both PCIEAER and CXL are enabled then light up CXL error handling as well. The work to remove
CONFIG_PCIEAER_CXLstarted in:
Quoted 7 lines
commit
4ae6ae66649c("cxl/pci: Remove CXL VH handling inCONFIG_PCIEAER_CXLconditional blocks fromcore/pci.c")Finish that off with conditionally compiling all CXL RAS related helpers with
CONFIG_CXL_RAS.Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>
Terry, if you are including this patch from Dan in your series, you need to sign off on it.
Quoted 42 lines
----
Changes in
v13->v14:
- New commit--- drivers/cxl/Kconfig | 2 +- drivers/pci/pcie/Kconfig | 9 --------- 2 files changed, 1 insertion(+), 10 deletions(-)drivers/cxl/Kconfig +1
-1@@ -235,6 +235,6 @@ config CXL_MCE config CXL_RAS def_bool y - depends on ACPI_APEI_GHES && PCIEAER && CXL_PCI + depends on ACPI_APEI_GHES && PCIEAER && CXL_BUS endifdrivers/pci/pcie/Kconfig +0
-9@@ -49,15 +49,6 @@ config PCIEAER_INJECT gotten from: https://github.com/intel/aer-inject.git -config PCIEAER_CXL - bool "PCI Express CXL RAS support" - default y - depends on PCIEAER && CXL_PCI - help - Enables CXL error handling. - - If unsure, say Y. - # # PCI Express ECRC #
Re: [PATCH v14 13/34] PCI/AER: Replace PCIEAER_CXL symbol with CXL_RAS
In reply to Terry BowmanQuoted 6 lines · Terry Bowman
From: Dan Williams <[email protected]>
One of the primary reasons for the CXL driver to exist is to perform error handling. If both PCIEAER and CXL are enabled then light up CXL error handling as well. The work to remove
CONFIG_PCIEAER_CXLstarted in:
Quoted 7 lines
commit
4ae6ae66649c("cxl/pci: Remove CXL VH handling inCONFIG_PCIEAER_CXLconditional blocks fromcore/pci.c")Finish that off with conditionally compiling all CXL RAS related helpers with
CONFIG_CXL_RAS.Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>
Quoted 45 lines
----
Changes in
v13->v14:
- New commit--- drivers/cxl/Kconfig | 2 +- drivers/pci/pcie/Kconfig | 9 --------- 2 files changed, 1 insertion(+), 10 deletions(-)drivers/cxl/Kconfig +1
-1@@ -235,6 +235,6 @@ config CXL_MCE config CXL_RAS def_bool y - depends on ACPI_APEI_GHES && PCIEAER && CXL_PCI + depends on ACPI_APEI_GHES && PCIEAER && CXL_BUS endifdrivers/pci/pcie/Kconfig +0
-9@@ -49,15 +49,6 @@ config PCIEAER_INJECT gotten from: https://github.com/intel/aer-inject.git -config PCIEAER_CXL - bool "PCI Express CXL RAS support" - default y - depends on PCIEAER && CXL_PCI - help - Enables CXL error handling. - - If unsure, say Y. - # # PCI Express ECRC #Signature · 1 line
2.34.1
PATCH v14 14/34 PCI/AER: Report CXL or PCIe bus type in AER trace logging
In reply to Terry BowmanThe AER service driver and aer_event tracing currently log 'PCIe Bus Type' for all errors. Update the driver and aer_event tracing to log 'CXL Bus Type' for CXL device errors.
This requires that AER can identify and distinguish between PCIe errors and CXL errors.
Introduce boolean 'is_cxl' to 'struct aer_err_info'. Add assignment in aer_get_device_error_info() and pci_print_aer().
Update the aer_event trace routine to accept a bus type string parameter.
---
Changes in v13->v14:
- Merged with Dan's commit. Changes are moving bus_type the last parameter in function calls (Dan)
- Removed all DCOs because of changes (Terry)
- Update commit message (Bjorn)
- Add Bjorn's ack-by
Changes in v12->v13:
- Remove duplicated aer_err_info inline comments. Is already in the kernel-doc header (Ben)
Changes in v11->v12:
- Change aer_err_info::is_cxl to be bool a bitfield. Update structure padding. (Lukas)
- Add kernel-doc for 'struct aer_err_info' (Lukas)
Changes in v10->v11:
- Remove duplicate call to trace_aer_event() (Shiju)
- Added Dan William's and Dave Jiang's reviewed-by
--- drivers/pci/pci.h | 8 +++++++- drivers/pci/pcie/aer.c | 20 +++++++++++++------- include/ras/ras_event.h | 12 ++++++++---- 3 files changed, 28 insertions(+), 12 deletions(-)
drivers/pci/pci.h +7 -1
@@ -738,7 +738,8 @@ struct aer_err_info { unsigned int multi_error_valid:1; unsigned int first_error:5; - unsigned int __pad2:2; + unsigned int __pad2:1; + unsigned int is_cxl:1; unsigned int tlp_header_valid:1;unsigned int status;/* COR/UNCOR Error Status */
@@ -749,6 +750,11 @@ struct aer_err_info {int aer_get_device_error_info (struct aer_err_info *info,int i);void aer_print_error (struct aer_err_info *info,int i); +static inline const char *aer_err_bus (struct aer_err_info *info) +{ +return info->is_cxl ?"CXL" :"PCIe" ; +} +int pcie_read_tlp_log (struct pci_dev *dev,int where,int where2,unsigned int tlp_len,bool flit,struct pcie_tlp_log *log);
drivers/pci/pcie/aer.c +13 -7
@@ -870,6 +870,7 @@ void aer_print_error(struct aer_err_info *info, int i)struct pci_dev *dev;int layer, agent, id;const char *level = info->level; +const char *bus_type =aer_err_bus (info);if (WARN_ON_ONCE (i >=AER_MAX_MULTI_ERR_DEVICES ))return ;
@@ -879,22 +880,22 @@ void aer_print_error(struct aer_err_info *info, int i)pci_dev_aer_stats_incr (dev, info);trace_aer_event (pci_name (dev), (info->status & ~info->mask), - info->severity, info->tlp_header_valid, &info->tlp); + info->severity, info->tlp_header_valid, &info->tlp, bus_type);if (!info->ratelimit_print[i])return ;if (!info->status) { -pci_err (dev,"PCIe Bus Error: severity=%s, type=Inaccessible, (Unregistered Agent ID)\n" , - aer_error_severity_string[info->severity]); +pci_err (dev,"%s Bus Error: severity=%s, type=Inaccessible, (Unregistered Agent ID)\n" , + bus_type, aer_error_severity_string[info->severity]); goto out; } layer =AER_GET_LAYER_ERROR (info->severity, info->status); agent =AER_GET_AGENT (info->severity, info->status); -aer_printk (level, dev,"PCIe Bus Error: severity=%s, type=%s, (%s)\n" , - aer_error_severity_string[info->severity], +aer_printk (level, dev,"%s Bus Error: severity=%s, type=%s, (%s)\n" , + bus_type, aer_error_severity_string[info->severity], aer_error_layer[layer], aer_agent_string[agent]); aer_printk(level, dev, " device [%04x:%04x] error status/mask=%08x/%08x\n",
@@ -928,6 +929,7 @@ EXPORT_SYMBOL_GPL(cper_severity_to_aer);void pci_print_aer (struct pci_dev *dev,int aer_severity,struct aer_capability_regs *aer) { +const char *bus_type;int layer, agent, tlp_header_valid =0 ;u32 status, mask;struct aer_err_info info = {
@@ -948,10 +950,13 @@ void pci_print_aer(struct pci_dev *dev, int aer_severity, info.status = status; info.mask = mask; + info.is_cxl =pcie_is_cxl (dev); + + bus_type =aer_err_bus (&info);pci_dev_aer_stats_incr (dev, &info); -trace_aer_event (pci_name (dev), (status & ~mask), - aer_severity, tlp_header_valid, &aer->header_log); +trace_aer_event (pci_name (dev), (status & ~mask), aer_severity, + tlp_header_valid, &aer->header_log, bus_type);if (!aer_ratelimit (dev, info.severity))return ;
@@ -1301,6 +1306,7 @@ int aer_get_device_error_info(struct aer_err_info *info, int i)/* Must reset in this function */ info->status =0 ; info->tlp_header_valid =0 ; + info->is_cxl =pcie_is_cxl (dev);/* The device might not support AER */ if (!aer)
include/ras/ras_event.h +8 -4
@@ -339,9 +339,11 @@ TRACE_EVENT(aer_event,const u32 status, const u8 severity, const u8 tlp_header_valid, - struct pcie_tlp_log *tlp), + struct pcie_tlp_log *tlp, + const char *bus_type), -TP_ARGS (dev_name ,status ,severity ,tlp_header_valid ,tlp ), + +TP_ARGS (dev_name ,status ,severity ,tlp_header_valid ,tlp ,bus_type ),TP_STRUCT__entry ( __string( dev_name, dev_name )
@@ -349,10 +351,12 @@ TRACE_EVENT(aer_event, __field( u8, severity ) __field( u8, tlp_header_valid) __array( u32, tlp_header, PCIE_STD_MAX_TLP_HEADERLOG) +__string ( bus_type, bus_type ) ),TP_fast_assign ( __assign_str(dev_name); +__assign_str (bus_type); __entry->status = status; __entry->severity = severity; __entry->tlp_header_valid = tlp_header_valid;
@@ -364,8 +368,8 @@ TRACE_EVENT(aer_event, } ), - TP_printk("%s PCIe Bus Error: severity=%s, %s, TLP Header=%s\n", - __get_str(dev_name), + TP_printk("%s %s Bus Error: severity=%s, %s, TLP Header=%s\n", + __get_str(dev_name), __get_str(bus_type), __entry->severity == AER_CORRECTABLE ? "Corrected" : __entry->severity == AER_FATAL ? "Fatal" : "Uncorrected, non-fatal",
Signature · 1 line
2.34.1
Re: [PATCH v14 14/34] PCI/AER: Report CXL or PCIe bus type in AER trace logging
Quoted 10 lines · On Wed, 14 Jan 2026 12:20:35 -0600 Terry Bowman <[email protected]> wrote:
The AER service driver and
aer_eventtracing currently log 'PCIe Bus Type' for all errors. Update the driver andaer_eventtracing to log 'CXL Bus Type' for CXL device errors.This requires that AER can identify and distinguish between PCIe errors and CXL errors.
Introduce boolean '
is_cxl' to 'structaer_err_info'. Add assignment inaer_get_device_error_info()andpci_print_aer().
Quoted 6 lines
Update the
aer_eventtrace routine to accept a bus type string parameter.Signed-off-by Terry Bowman <[email protected]>Co-developed-by Dan Williams <[email protected]>Signed-off-by Dan Williams <[email protected]>Acked-by Bjorn Helgaas <[email protected]>
I wonder if it is worth using __print_symbolic() etc and an integer storage rather than a string for in the tracepoints. However, not really that important to me as the strings are small anyway and there is no precedence of this in ras trace events.
RE: [PATCH v14 14/34] PCI/AER: Report CXL or PCIe bus type in AER trace logging
Quoted 17 lines
Quoted 10 lines
The AER service driver and
aer_eventtracing currently log 'PCIe Bus Type' for all errors. Update the driver andaer_eventtracing to log 'CXL Bus Type' for CXL device errors.This requires that AER can identify and distinguish between PCIe errors and CXL errors.
Introduce boolean '
is_cxl' to 'structaer_err_info'. Add assignment inaer_get_device_error_info()andpci_print_aer().Quoted 6 lines
Update the
aer_eventtrace routine to accept a bus type string parameter.Signed-off-by Terry Bowman <[email protected]>Co-developed-by Dan Williams <[email protected]>Signed-off-by Dan Williams <[email protected]>Acked-by Bjorn Helgaas <[email protected]>
Quoted 4 lines
I wonder if it is worth using
__print_symbolic()etc and an integer storage rather than a string for in the tracepoints. However, not really that important to me as the strings are small anyway and there is no precedence of this in ras trace events.Reviewed-by Jonathan Cameron <[email protected]>
It would be a lot better to pass integer values instead of strings. Right now, I'm working on a way to have a CI to check if Kernel + rasdaemon is doing the right thing. The idea is to inject errors on QEMU via QMP interface (main patches already there at QEMU tree).
By using integers, it sounds easier to veriy if everything was properly handled, as we can ignore __print_symbolic at rasdaemon, picking the actual values directly.
Regards,
Mauro
Re: [PATCH v14 14/34] PCI/AER: Report CXL or PCIe bus type in AER trace logging
In reply to Terry BowmanQuoted 10 lines · Terry Bowman
The AER service driver and
aer_eventtracing currently log 'PCIe Bus Type' for all errors. Update the driver andaer_eventtracing to log 'CXL Bus Type' for CXL device errors.This requires that AER can identify and distinguish between PCIe errors and CXL errors.
Introduce boolean '
is_cxl' to 'structaer_err_info'. Add assignment inaer_get_device_error_info()andpci_print_aer().
Quoted 6 lines
Update the
aer_eventtrace routine to accept a bus type string parameter.Signed-off-by Terry Bowman <[email protected]>Co-developed-by Dan Williams <[email protected]>Signed-off-by Dan Williams <[email protected]>Acked-by Bjorn Helgaas <[email protected]>
Quoted 168 lines
---
Changes in
v13->v14:
- Merged with Dan's commit. Changes are movingbus_typethe last parameter in function calls (Dan)
- Removed all DCOs because of changes (Terry)
- Update commit message (Bjorn)
- Add Bjorn's ack-byChanges in
v12->v13:
- Remove duplicatedaer_err_infoinline comments. Is already in the kernel-doc header (Ben)Changes in
v11->v12:
- Changeaer_err_info::is_cxlto be bool a bitfield. Update structure padding. (Lukas)
- Add kernel-doc for 'structaer_err_info' (Lukas)Changes in
v10->v11:
- Remove duplicate call totrace_aer_event()(Shiju)
- Added Dan William's and Dave Jiang's reviewed-by--- drivers/pci/pci.h | 8 +++++++- drivers/pci/pcie/aer.c | 20 +++++++++++++------- include/ras/ras_event.h | 12 ++++++++---- 3 files changed, 28 insertions(+), 12 deletions(-)drivers/pci/pci.h +7
-1@@ -738,7 +738,8 @@ struct aer_err_info { unsigned int multi_error_valid:1; unsigned int first_error:5; - unsigned int __pad2:2; + unsigned int __pad2:1; + unsigned int is_cxl:1; unsigned int tlp_header_valid:1;unsigned int status;/* COR/UNCOR Error Status */ @@ -749,6 +750,11 @@ struct aer_err_info {int aer_get_device_error_info (struct aer_err_info *info,int i);void aer_print_error (struct aer_err_info *info,int i); +static inline const char *aer_err_bus (struct aer_err_info *info) +{ +return info->is_cxl ?"CXL" :"PCIe" ; +} +int pcie_read_tlp_log (struct pci_dev *dev,int where,int where2,unsigned int tlp_len,bool flit,struct pcie_tlp_log *log);drivers/pci/pcie/aer.c +13
-7@@ -870,6 +870,7 @@ void aer_print_error(struct aer_err_info *info, int i)struct pci_dev *dev;int layer, agent, id;const char *level = info->level; +const char *bus_type =aer_err_bus (info);if (WARN_ON_ONCE (i >=AER_MAX_MULTI_ERR_DEVICES ))return ;@@ -879,22 +880,22 @@ void aer_print_error(struct aer_err_info *info, int i)pci_dev_aer_stats_incr (dev, info);trace_aer_event (pci_name (dev), (info->status & ~info->mask), - info->severity, info->tlp_header_valid, &info->tlp); + info->severity, info->tlp_header_valid, &info->tlp, bus_type);if (!info->ratelimit_print[i])return ;if (!info->status) { -pci_err (dev,"PCIe Bus Error: severity=%s, type=Inaccessible, (Unregistered Agent ID)\n" , - aer_error_severity_string[info->severity]); +pci_err (dev,"%s Bus Error: severity=%s, type=Inaccessible, (Unregistered Agent ID)\n" , + bus_type, aer_error_severity_string[info->severity]); goto out; } layer =AER_GET_LAYER_ERROR (info->severity, info->status); agent =AER_GET_AGENT (info->severity, info->status); -aer_printk (level, dev,"PCIe Bus Error: severity=%s, type=%s, (%s)\n" , - aer_error_severity_string[info->severity], +aer_printk (level, dev,"%s Bus Error: severity=%s, type=%s, (%s)\n" , + bus_type, aer_error_severity_string[info->severity], aer_error_layer[layer], aer_agent_string[agent]); aer_printk(level, dev, " device [%04x:%04x] error status/mask=%08x/%08x\n",@@ -928,6 +929,7 @@ EXPORT_SYMBOL_GPL(cper_severity_to_aer);void pci_print_aer (struct pci_dev *dev,int aer_severity,struct aer_capability_regs *aer) { +const char *bus_type;int layer, agent, tlp_header_valid =0 ;u32 status, mask;struct aer_err_info info = {@@ -948,10 +950,13 @@ void pci_print_aer(struct pci_dev *dev, int aer_severity, info.status = status; info.mask = mask; + info.is_cxl =pcie_is_cxl (dev); + + bus_type =aer_err_bus (&info);pci_dev_aer_stats_incr (dev, &info); -trace_aer_event (pci_name (dev), (status & ~mask), - aer_severity, tlp_header_valid, &aer->header_log); +trace_aer_event (pci_name (dev), (status & ~mask), aer_severity, + tlp_header_valid, &aer->header_log, bus_type);if (!aer_ratelimit (dev, info.severity))return ;@@ -1301,6 +1306,7 @@ int aer_get_device_error_info(struct aer_err_info *info, int i)/* Must reset in this function */ info->status =0 ; info->tlp_header_valid =0 ; + info->is_cxl =pcie_is_cxl (dev);/* The device might not support AER */ if (!aer)include/ras/ras_event.h +8
-4@@ -339,9 +339,11 @@ TRACE_EVENT(aer_event,const u32 status, const u8 severity, const u8 tlp_header_valid, - struct pcie_tlp_log *tlp), + struct pcie_tlp_log *tlp, + const char *bus_type), -TP_ARGS (dev_name ,status ,severity ,tlp_header_valid ,tlp ), + +TP_ARGS (dev_name ,status ,severity ,tlp_header_valid ,tlp ,bus_type ),TP_STRUCT__entry ( __string( dev_name, dev_name )@@ -349,10 +351,12 @@ TRACE_EVENT(aer_event, __field( u8, severity ) __field( u8, tlp_header_valid) __array( u32, tlp_header, PCIE_STD_MAX_TLP_HEADERLOG) +__string ( bus_type, bus_type ) ),TP_fast_assign ( __assign_str(dev_name); +__assign_str (bus_type); __entry->status = status; __entry->severity = severity; __entry->tlp_header_valid = tlp_header_valid;@@ -364,8 +368,8 @@ TRACE_EVENT(aer_event, } ), - TP_printk("%s PCIe Bus Error: severity=%s, %s, TLP Header=%s\n", - __get_str(dev_name), + TP_printk("%s %s Bus Error: severity=%s, %s, TLP Header=%s\n", + __get_str(dev_name), __get_str(bus_type), __entry->severity == AER_CORRECTABLE ? "Corrected" : __entry->severity == AER_FATAL ? "Fatal" : "Uncorrected, non-fatal",
PATCH v14 15/34 PCI/AER: Update struct aer_err_info with kernel-doc formatting
In reply to Terry BowmanUpdate the existing 'struct aer_err_info' definition to use kernel-doc formatting. Remove the inline comments to reduce noise and do not introduce functional changes. This will improve readability and maintainability.
---
Changes in v13->v14:
- New commit
--- drivers/pci/pci.h | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-)
drivers/pci/pci.h +23 -6
@@ -724,16 +724,33 @@ static inline bool pci_dev_binding_disallowed(struct pci_dev *dev)#define AER_MAX_MULTI_ERR_DEVICES 5/* Not likely to have more */ +/** +* struct aer_err_info - AER Error Information +* @dev: Devices reporting error +* @ratelimit_print: Flag to log or not log the devices' error. 0=NotLog/1=Log +* @error_dev_num: Number of devices reporting an error +* @level: printk level to use in logging +* @id: Value from register PCI_ERR_ROOT_ERR_SRC +* @severity: AER severity, 0-UNCOR Non-fatal, 1-UNCOR fatal, 2-COR +* @root_ratelimit_print: Flag to log or not log the root's error. 0=NotLog/1=Log +* @multi_error_valid: If multiple errors are reported +* @first_error: First reported error +* @is_cxl: Bus type error: 0-PCI Bus error, 1-CXL Bus error +* @tlp_header_valid: Indicates if TLP field contains error information +* @status: COR/UNCOR error status +* @mask: COR/UNCOR mask +* @tlp: Transaction packet information +*/ struct aer_err_info {struct pci_dev *dev[AER_MAX_MULTI_ERR_DEVICES ];int ratelimit_print[AER_MAX_MULTI_ERR_DEVICES ];int error_dev_num; -const char *level;/* printk level */ +const char *level;unsigned int id:16 ; -unsigned int severity:2 ;/* 0:NONFATAL | 1:FATAL | 2:COR */ -unsigned int root_ratelimit_print:1 ;/* 0=skip, 1=print */ +unsigned int severity:2 ; +unsigned int root_ratelimit_print:1 ;unsigned int __pad1:4 ;unsigned int multi_error_valid:1 ;
@@ -742,9 +759,9 @@ struct aer_err_info { unsigned int is_cxl:1; unsigned int tlp_header_valid:1; -unsigned int status;/* COR/UNCOR Error Status */ -unsigned int mask;/* COR/UNCOR Error Mask */ -struct pcie_tlp_log tlp;/* TLP Header */ +unsigned int status; +unsigned int mask; +struct pcie_tlp_log tlp; };int aer_get_device_error_info (struct aer_err_info *info,int i);
Signature · 1 line
2.34.1
Re: [PATCH v14 15/34] PCI/AER: Update struct aer_err_info with kernel-doc formatting
Quoted 6 lines · On Wed, 14 Jan 2026 12:20:36 -0600 Terry Bowman <[email protected]> wrote:
Update the existing 'struct
aer_err_info' definition to use kernel-doc formatting. Remove the inline comments to reduce noise and do not introduce functional changes. This will improve readability and maintainability.Signed-off-by Terry Bowman <[email protected]>Reviewed-by Dan Williams <[email protected]>
Hi Terry.
I didn't check but I think kernel-doc script will complain about partial docs. Other than that possibly needing fixing with a trivial entry for __pad1
Quoted 16 lines
---
Changes in
v13->v14:
- New commit--- drivers/pci/pci.h | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-)drivers/pci/pci.h +0
-0@@ -724,16 +724,33 @@ static inline bool pci_dev_binding_disallowed(struct pci_dev *dev)#define AER_MAX_MULTI_ERR_DEVICES 5/* Not likely to have more */
drivers/pci/pci.h static inline bool pci_dev_binding_disallowed(struct pci_dev *dev) · line 735
Quoted 10 lines
drivers/pci/pci.h +9
-0+/** +* struct aer_err_info - AER Error Information +* @dev: Devices reporting error +* @ratelimit_print: Flag to log or not log the devices' error. 0=NotLog/1=Log +* @error_dev_num: Number of devices reporting an error +* @level: printk level to use in logging +* @id: Value from register PCI_ERR_ROOT_ERR_SRC +* @severity: AER severity, 0-UNCOR Non-fatal, 1-UNCOR fatal, 2-COR +* @root_ratelimit_print: Flag to log or not log the root's error. 0=NotLog/1=Log
The kernel-doc scripts are annoying fussy. Do they not moan about __pad1 being undocumented?
Quoted 37 lines
drivers/pci/pci.h +14
-6+* @multi_error_valid: If multiple errors are reported +* @first_error: First reported error +* @is_cxl: Bus type error: 0-PCI Bus error, 1-CXL Bus error +* @tlp_header_valid: Indicates if TLP field contains error information +* @status: COR/UNCOR error status +* @mask: COR/UNCOR mask +* @tlp: Transaction packet information +*/ struct aer_err_info {struct pci_dev *dev[AER_MAX_MULTI_ERR_DEVICES ];int ratelimit_print[AER_MAX_MULTI_ERR_DEVICES ];int error_dev_num; -const char *level;/* printk level */ +const char *level;unsigned int id:16 ; -unsigned int severity:2 ;/* 0:NONFATAL | 1:FATAL | 2:COR */ -unsigned int root_ratelimit_print:1 ;/* 0=skip, 1=print */ +unsigned int severity:2 ; +unsigned int root_ratelimit_print:1 ;unsigned int __pad1:4 ;unsigned int multi_error_valid:1 ;@@ -742,9 +759,9 @@ struct aer_err_info { unsigned int is_cxl:1; unsigned int tlp_header_valid:1; -unsigned int status;/* COR/UNCOR Error Status */ -unsigned int mask;/* COR/UNCOR Error Mask */ -struct pcie_tlp_log tlp;/* TLP Header */ +unsigned int status; +unsigned int mask; +struct pcie_tlp_log tlp; };int aer_get_device_error_info (struct aer_err_info *info,int i);
Re: [PATCH v14 15/34] PCI/AER: Update struct aer_err_info with kernel-doc formatting
Quoted 7 lines · Jonathan Cameron
Quoted 4 lines · On Wed, 14 Jan 2026 12:20:36 -0600 Terry Bowman <[email protected]> wrote:
Update the existing 'struct
aer_err_info' definition to use kernel-doc formatting. Remove the inline comments to reduce noise and do not introduce functional changes. This will improve readability and maintainability.
Quoted 7 lines
Quoted 2 lines
Signed-off-by Terry Bowman <[email protected]>Reviewed-by Dan Williams <[email protected]>Hi Terry.
I didn't check but I think kernel-doc script will complain about partial docs. Other than that possibly needing fixing with a trivial entry for
__pad1
It does:
Warning: drivers/pci/pci.h:764 struct member '__pad1' not described in 'aer_err_info'
Warning: drivers/pci/pci.h:764 struct member '__pad2' not described in 'aer_err_info'
...those are the only warnings in this set. Btw, this is my hacky script for checking for new kdoc errors introduced in a patch series. I assume the 0day robot has something similar. Maybe something to cleanup and contribute to checkpatch:
KDOC=~/git/linux/scripts/kernel-doc
for p in $(stg series -A --noprefix)
do
echo KERNELDOC $p
stg goto $p >/dev/null
for i in $(stg files --bare $p)
do
# only show the new errors relative to the contents of
# the file in the previous commit
f1=$(mktemp)
if [ git show HEAD^:$i >$f1 2>/dev/null ]; then
f2=$(mktemp)
f3=$(mktemp)
$KDOC $f1 2>$f2 >/dev/null
$KDOC $i 2>$f3 >/dev/null
diff -u $f2 $f3
rm $f2 $f3
else
$KDOC $i 2>&1 1>/dev/null
fi
rm $f1
donedone
Re: [PATCH v14 15/34] PCI/AER: Update struct aer_err_info with kernel-doc formatting
In reply to Terry BowmanQuoted 6 lines · Terry Bowman
Update the existing 'struct
aer_err_info' definition to use kernel-doc formatting. Remove the inline comments to reduce noise and do not introduce functional changes. This will improve readability and maintainability.Signed-off-by Terry Bowman <[email protected]>Reviewed-by Dan Williams <[email protected]>
Quoted 63 lines
---
Changes in
v13->v14:
- New commit--- drivers/pci/pci.h | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-)drivers/pci/pci.h +23
-6@@ -724,16 +724,33 @@ static inline bool pci_dev_binding_disallowed(struct pci_dev *dev)#define AER_MAX_MULTI_ERR_DEVICES 5/* Not likely to have more */ +/** +* struct aer_err_info - AER Error Information +* @dev: Devices reporting error +* @ratelimit_print: Flag to log or not log the devices' error. 0=NotLog/1=Log +* @error_dev_num: Number of devices reporting an error +* @level: printk level to use in logging +* @id: Value from register PCI_ERR_ROOT_ERR_SRC +* @severity: AER severity, 0-UNCOR Non-fatal, 1-UNCOR fatal, 2-COR +* @root_ratelimit_print: Flag to log or not log the root's error. 0=NotLog/1=Log +* @multi_error_valid: If multiple errors are reported +* @first_error: First reported error +* @is_cxl: Bus type error: 0-PCI Bus error, 1-CXL Bus error +* @tlp_header_valid: Indicates if TLP field contains error information +* @status: COR/UNCOR error status +* @mask: COR/UNCOR mask +* @tlp: Transaction packet information +*/ struct aer_err_info {struct pci_dev *dev[AER_MAX_MULTI_ERR_DEVICES ];int ratelimit_print[AER_MAX_MULTI_ERR_DEVICES ];int error_dev_num; -const char *level;/* printk level */ +const char *level;unsigned int id:16 ; -unsigned int severity:2 ;/* 0:NONFATAL | 1:FATAL | 2:COR */ -unsigned int root_ratelimit_print:1 ;/* 0=skip, 1=print */ +unsigned int severity:2 ; +unsigned int root_ratelimit_print:1 ;unsigned int __pad1:4 ;unsigned int multi_error_valid:1 ;@@ -742,9 +759,9 @@ struct aer_err_info { unsigned int is_cxl:1; unsigned int tlp_header_valid:1; -unsigned int status;/* COR/UNCOR Error Status */ -unsigned int mask;/* COR/UNCOR Error Mask */ -struct pcie_tlp_log tlp;/* TLP Header */ +unsigned int status; +unsigned int mask; +struct pcie_tlp_log tlp; };int aer_get_device_error_info (struct aer_err_info *info,int i);
Re: [PATCH v14 15/34] PCI/AER: Update struct aer_err_info with kernel-doc formatting
In reply to Terry BowmanQuoted 6 lines · Terry Bowman
Update the existing 'struct
aer_err_info' definition to use kernel-doc formatting. Remove the inline comments to reduce noise and do not introduce functional changes. This will improve readability and maintainability.Signed-off-by Terry Bowman <[email protected]>Reviewed-by Dan Williams <[email protected]>
Quoted 65 lines
---
Changes in
v13->v14:
- New commit--- drivers/pci/pci.h | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-)drivers/pci/pci.h +23
-6@@ -724,16 +724,33 @@ static inline bool pci_dev_binding_disallowed(struct pci_dev *dev)#define AER_MAX_MULTI_ERR_DEVICES 5/* Not likely to have more */ +/** +* struct aer_err_info - AER Error Information +* @dev: Devices reporting error +* @ratelimit_print: Flag to log or not log the devices' error. 0=NotLog/1=Log +* @error_dev_num: Number of devices reporting an error +* @level: printk level to use in logging +* @id: Value from register PCI_ERR_ROOT_ERR_SRC +* @severity: AER severity, 0-UNCOR Non-fatal, 1-UNCOR fatal, 2-COR +* @root_ratelimit_print: Flag to log or not log the root's error. 0=NotLog/1=Log +* @multi_error_valid: If multiple errors are reported +* @first_error: First reported error +* @is_cxl: Bus type error: 0-PCI Bus error, 1-CXL Bus error +* @tlp_header_valid: Indicates if TLP field contains error information +* @status: COR/UNCOR error status +* @mask: COR/UNCOR mask +* @tlp: Transaction packet information +*/ struct aer_err_info {struct pci_dev *dev[AER_MAX_MULTI_ERR_DEVICES ];int ratelimit_print[AER_MAX_MULTI_ERR_DEVICES ];int error_dev_num; -const char *level;/* printk level */ +const char *level;unsigned int id:16 ; -unsigned int severity:2 ;/* 0:NONFATAL | 1:FATAL | 2:COR */ -unsigned int root_ratelimit_print:1 ;/* 0=skip, 1=print */ +unsigned int severity:2 ; +unsigned int root_ratelimit_print:1 ;unsigned int __pad1:4 ;unsigned int multi_error_valid:1 ;@@ -742,9 +759,9 @@ struct aer_err_info { unsigned int is_cxl:1; unsigned int tlp_header_valid:1; -unsigned int status;/* COR/UNCOR Error Status */ -unsigned int mask;/* COR/UNCOR Error Mask */ -struct pcie_tlp_log tlp;/* TLP Header */ +unsigned int status; +unsigned int mask; +struct pcie_tlp_log tlp; };int aer_get_device_error_info (struct aer_err_info *info,int i);Signature · 1 line
2.34.1
PATCH v14 16/34 cxl/mem: Clarify @host for devm_cxl_add_nvdimm()
In reply to Terry BowmanFrom: Dan Williams <[email protected]>
The convention for devm_ helpers in the CXL driver is that the first argument is the @host for the operation (locked driver::probe() context).
---
Changes in v13 -> v14:
- New patch
--- drivers/cxl/core/pmem.c | 13 +++++++------ drivers/cxl/cxl.h | 3 ++- drivers/cxl/mem.c | 2 +- 3 files changed, 10 insertions(+), 8 deletions(-)
drivers/cxl/core/pmem.c +7 -6
@@ -237,12 +237,13 @@ static void cxlmd_release_nvdimm(void *_cxlmd)/** * devm_cxl_add_nvdimm() - add a bridge between a cxl_memdev and an nvdimm -* @parent_port: parent port for the (to be added) @cxlmd endpoint port -* @cxlmd: cxl_memdev instance that will perform LIBNVDIMM operations +* @host: host device for devm operations +* @port: any port in the CXL topology to find the nvdimm-bridge device +* @cxlmd: parent of the to be created cxl_nvdimm device * * Return: 0 on success negative error code on failure. */ -int devm_cxl_add_nvdimm (struct cxl_port *parent_port, +int devm_cxl_add_nvdimm (struct device *host,struct cxl_port *port,struct cxl_memdev *cxlmd) {struct cxl_nvdimm_bridge *cxl_nvb;
@@ -250,7 +251,7 @@ int devm_cxl_add_nvdimm(struct cxl_port *parent_port,struct device *dev;int rc; - cxl_nvb =cxl_find_nvdimm_bridge (parent_port); + cxl_nvb =cxl_find_nvdimm_bridge (port);if (!cxl_nvb)return -ENODEV ;
@@ -270,10 +271,10 @@ int devm_cxl_add_nvdimm(struct cxl_port *parent_port,if (rc) goto err; -dev_dbg (&cxlmd->dev,"register %s\n" ,dev_name (dev)); +dev_dbg (host,"register %s\n" ,dev_name (dev));/* @cxlmd carries a reference on @cxl_nvb until cxlmd_release_nvdimm */ -return devm_add_action_or_reset (&cxlmd->dev, cxlmd_release_nvdimm, cxlmd); +return devm_add_action_or_reset (host, cxlmd_release_nvdimm, cxlmd); err:put_device (dev);
drivers/cxl/cxl.h +2 -1
@@ -887,7 +887,8 @@ struct cxl_nvdimm_bridge *devm_cxl_add_nvdimm_bridge(struct device *host, struct cxl_port *port);struct cxl_nvdimm *to_cxl_nvdimm (struct device *dev);bool is_cxl_nvdimm (struct device *dev); -int devm_cxl_add_nvdimm (struct cxl_port *parent_port,struct cxl_memdev *cxlmd); +int devm_cxl_add_nvdimm (struct device *host,struct cxl_port *port, +struct cxl_memdev *cxlmd);struct cxl_nvdimm_bridge *cxl_find_nvdimm_bridge (struct cxl_port *port);#ifdef CONFIG_CXL_REGION
drivers/cxl/mem.c +1 -1
@@ -153,7 +153,7 @@ static int cxl_mem_probe(struct device *dev) }if (cxl_pmem_size (cxlds) &&IS_ENABLED (CONFIG_CXL_PMEM )) { - rc =devm_cxl_add_nvdimm (parent_port, cxlmd); + rc =devm_cxl_add_nvdimm (dev, parent_port, cxlmd);if (rc) {if (rc == -ENODEV )dev_info (dev,"PMEM disabled by platform\n" );
Signature · 1 line
2.34.1
Re: [PATCH v14 16/34] cxl/mem: Clarify @host for devm_cxl_add_nvdimm()
Quoted 7 lines · On Wed, 14 Jan 2026 12:20:37 -0600 Terry Bowman <[email protected]> wrote:
From: Dan Williams <[email protected]>
The convention for
devm_helpers in the CXL driver is that the first argument is the @host for the operation (lockeddriver::probe()context).Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>
Re: [PATCH v14 16/34] cxl/mem: Clarify @host for devm_cxl_add_nvdimm()
In reply to Terry BowmanQuoted 7 lines · Terry Bowman
From: Dan Williams <[email protected]>
The convention for
devm_helpers in the CXL driver is that the first argument is the @host for the operation (lockeddriver::probe()context).Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>
A nit below
Quoted 19 lines
---
Changes in v13 -> v14:
- New patch--- drivers/cxl/core/pmem.c | 13 +++++++------ drivers/cxl/cxl.h | 3 ++- drivers/cxl/mem.c | 2 +- 3 files changed, 10 insertions(+), 8 deletions(-)drivers/cxl/core/pmem.c +0
-0@@ -237,12 +237,13 @@ static void cxlmd_release_nvdimm(void *_cxlmd)/** * devm_cxl_add_nvdimm() - add a bridge between a cxl_memdev and an nvdimm
drivers/cxl/core/pmem.c static void cxlmd_release_nvdimm(void *_cxlmd) · line 246
Quoted 10 lines
drivers/cxl/core/pmem.c +4
-3-* @parent_port: parent port for the (to be added) @cxlmd endpoint port -* @cxlmd: cxl_memdev instance that will perform LIBNVDIMM operations +* @host: host device for devm operations +* @port: any port in the CXL topology to find the nvdimm-bridge device +* @cxlmd: parent of the to be created cxl_nvdimm device * * Return: 0 on success negative error code on failure. */ -int devm_cxl_add_nvdimm(struct cxl_port *parent_port, +int devm_cxl_add_nvdimm(struct device *host, struct cxl_port *port,
s/port/parent_port/ to maintain clarity of the port
DJ
Quoted 52 lines
drivers/cxl/core/pmem.c +3
-3struct cxl_memdev *cxlmd) {struct cxl_nvdimm_bridge *cxl_nvb;@@ -250,7 +251,7 @@ int devm_cxl_add_nvdimm(struct cxl_port *parent_port,struct device *dev;int rc; - cxl_nvb =cxl_find_nvdimm_bridge (parent_port); + cxl_nvb =cxl_find_nvdimm_bridge (port);if (!cxl_nvb)return -ENODEV ;@@ -270,10 +271,10 @@ int devm_cxl_add_nvdimm(struct cxl_port *parent_port,if (rc) goto err; -dev_dbg (&cxlmd->dev,"register %s\n" ,dev_name (dev)); +dev_dbg (host,"register %s\n" ,dev_name (dev));/* @cxlmd carries a reference on @cxl_nvb until cxlmd_release_nvdimm */ -return devm_add_action_or_reset (&cxlmd->dev, cxlmd_release_nvdimm, cxlmd); +return devm_add_action_or_reset (host, cxlmd_release_nvdimm, cxlmd); err:put_device (dev);drivers/cxl/cxl.h +2
-1@@ -887,7 +887,8 @@ struct cxl_nvdimm_bridge *devm_cxl_add_nvdimm_bridge(struct device *host, struct cxl_port *port);struct cxl_nvdimm *to_cxl_nvdimm (struct device *dev);bool is_cxl_nvdimm (struct device *dev); -int devm_cxl_add_nvdimm (struct cxl_port *parent_port,struct cxl_memdev *cxlmd); +int devm_cxl_add_nvdimm (struct device *host,struct cxl_port *port, +struct cxl_memdev *cxlmd);struct cxl_nvdimm_bridge *cxl_find_nvdimm_bridge (struct cxl_port *port);#ifdef CONFIG_CXL_REGION drivers/cxl/mem.c +1
-1@@ -153,7 +153,7 @@ static int cxl_mem_probe(struct device *dev) }if (cxl_pmem_size (cxlds) &&IS_ENABLED (CONFIG_CXL_PMEM )) { - rc =devm_cxl_add_nvdimm (parent_port, cxlmd); + rc =devm_cxl_add_nvdimm (dev, parent_port, cxlmd);if (rc) {if (rc == -ENODEV )dev_info (dev,"PMEM disabled by platform\n" );
Re: [PATCH v14 16/34] cxl/mem: Clarify @host for devm_cxl_add_nvdimm()
Quoted 32 lines · Dave Jiang
Quoted 7 lines · Terry Bowman
From: Dan Williams <[email protected]>
The convention for
devm_helpers in the CXL driver is that the first argument is the @host for the operation (lockeddriver::probe()context).Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>Reviewed-by Dave Jiang <[email protected]>A nit below
Quoted 17 lines
---
Changes in v13 -> v14:
- New patch--- drivers/cxl/core/pmem.c | 13 +++++++------ drivers/cxl/cxl.h | 3 ++- drivers/cxl/mem.c | 2 +- 3 files changed, 10 insertions(+), 8 deletions(-)drivers/cxl/core/pmem.c +0
-0@@ -237,12 +237,13 @@ static void cxlmd_release_nvdimm(void *_cxlmd)
Quoted 14 lines
drivers/cxl/core/pmem.c static void cxlmd_release_nvdimm(void *_cxlmd) · line 246
Quoted 12 lines
drivers/cxl/core/pmem.c +4
-3/** * devm_cxl_add_nvdimm() - add a bridge between a cxl_memdev and an nvdimm -* @parent_port: parent port for the (to be added) @cxlmd endpoint port -* @cxlmd: cxl_memdev instance that will perform LIBNVDIMM operations +* @host: host device for devm operations +* @port: any port in the CXL topology to find the nvdimm-bridge device +* @cxlmd: parent of the to be created cxl_nvdimm device * * Return: 0 on success negative error code on failure. */ -int devm_cxl_add_nvdimm(struct cxl_port *parent_port, +int devm_cxl_add_nvdimm(struct device *host, struct cxl_port *port,s/port/parent_port/ to maintain clarity of the port
...but it is not used as a "parent" port in this function. Any port in the topology will do. The reason a port argument is needed is disambiguate when there are multiple CXL root devices. That currently only happens when cxl_test is loaded.
However, after writing that, it may make more sense to make that semantic explicit and just have the caller responsible for passing in an @cxl_root argument.
A change for not this series.
Dave Jiang
maintainerRe: [PATCH v14 16/34] cxl/mem: Clarify @host for devm_cxl_add_nvdimm()
Quoted 53 lines · On 1/15/26 8:07 PM, [email protected] wrote:
Quoted 32 lines · Dave Jiang
Quoted 7 lines · Terry Bowman
From: Dan Williams <[email protected]>
The convention for
devm_helpers in the CXL driver is that the first argument is the @host for the operation (lockeddriver::probe()context).Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>Reviewed-by Dave Jiang <[email protected]>A nit below
Quoted 17 lines
---
Changes in v13 -> v14:
- New patch--- drivers/cxl/core/pmem.c | 13 +++++++------ drivers/cxl/cxl.h | 3 ++- drivers/cxl/mem.c | 2 +- 3 files changed, 10 insertions(+), 8 deletions(-)drivers/cxl/core/pmem.c +0
-0@@ -237,12 +237,13 @@ static void cxlmd_release_nvdimm(void *_cxlmd)Quoted 14 lines
drivers/cxl/core/pmem.c static void cxlmd_release_nvdimm(void *_cxlmd) · line 246
Quoted 12 lines
drivers/cxl/core/pmem.c +4
-3/** * devm_cxl_add_nvdimm() - add a bridge between a cxl_memdev and an nvdimm -* @parent_port: parent port for the (to be added) @cxlmd endpoint port -* @cxlmd: cxl_memdev instance that will perform LIBNVDIMM operations +* @host: host device for devm operations +* @port: any port in the CXL topology to find the nvdimm-bridge device +* @cxlmd: parent of the to be created cxl_nvdimm device * * Return: 0 on success negative error code on failure. */ -int devm_cxl_add_nvdimm(struct cxl_port *parent_port, +int devm_cxl_add_nvdimm(struct device *host, struct cxl_port *port,s/port/parent_port/ to maintain clarity of the port
...but it is not used as a "parent" port in this function. Any port in the topology will do. The reason a port argument is needed is disambiguate when there are multiple CXL root devices. That currently only happens when
cxl_testis loaded.
Quoted 5 lines
However, after writing that, it may make more sense to make that semantic explicit and just have the caller responsible for passing in an @cxl_root argument.
A change for not this series.
I'll make a note in the backlog.
PATCH v14 17/34 cxl: Update RAS handler interfaces to also support CXL Ports
In reply to Terry BowmanCXL PCIe Port Protocol Error handling support will be added to the CXL drivers in the future. In preparation, rename the existing interfaces to support handling all CXL PCIe Port Protocol Errors.
The driver's RAS support functions currently rely on a 'struct cxl_dev_state' type parameter, which is not available for CXL Port devices. However, since the same CXL RAS capability structure is needed across most CXL components and devices, a common handling approach should be adopted.
To accommodate this, update the __cxl_handle_cor_ras() and __cxl_handle_ras() functions to use a struct device instead of struct cxl_dev_state.
No functional changes are introduced.
[1] CXL 3.1 Spec, 8.2.4 CXL.cache and CXL.mem Registers
---
Changes in v13->v14:
- None
--- drivers/cxl/core/core.h | 14 +++++--------- drivers/cxl/core/ras.c | 12 ++++++------ drivers/cxl/core/ras_rch.c | 4 ++-- 3 files changed, 13 insertions(+), 17 deletions(-)
drivers/cxl/core/core.h +5 -9
@@ -147,8 +147,8 @@ int cxl_port_get_switch_dport_bandwidth(struct cxl_port *port,#ifdef CONFIG_CXL_RAS int cxl_ras_init (void );void cxl_ras_exit (void ); -bool cxl_handle_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base); -void cxl_handle_cor_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base); +bool cxl_handle_ras (struct device *dev,void __iomem *ras_base); +void cxl_handle_cor_ras (struct device *dev,void __iomem *ras_base);void cxl_dport_map_rch_aer (struct cxl_dport *dport);void cxl_disable_rch_root_ints (struct cxl_dport *dport);void cxl_handle_rdport_errors (struct cxl_dev_state *cxlds);
@@ -157,16 +157,12 @@ static inline int cxl_ras_init(void) {return 0 ; } - -static inline void cxl_ras_exit (void ) -{ -} - -static inline bool cxl_handle_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base) +static inline void cxl_ras_exit (void ) { } +static inline bool cxl_handle_ras (struct device *dev,void __iomem *ras_base) {return false; } -static inline void cxl_handle_cor_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base) { } +static inline void cxl_handle_cor_ras (struct device *dev,void __iomem *ras_base) { }static inline void cxl_dport_map_rch_aer (struct cxl_dport *dport) { }static inline void cxl_disable_rch_root_ints (struct cxl_dport *dport) { }static inline void cxl_handle_rdport_errors (struct cxl_dev_state *cxlds) { }
drivers/cxl/core/ras.c +6 -6
@@ -160,7 +160,7 @@ void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host) }EXPORT_SYMBOL_NS_GPL (cxl_dport_init_ras_reporting,"CXL" ); -void cxl_handle_cor_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base) +void cxl_handle_cor_ras (struct device *dev,void __iomem *ras_base) {void __iomem *addr;u32 status;
@@ -172,7 +172,7 @@ void cxl_handle_cor_ras(struct cxl_dev_state *cxlds, void __iomem *ras_base) status =readl (addr);if (status &CXL_RAS_CORRECTABLE_STATUS_MASK ) {writel (status &CXL_RAS_CORRECTABLE_STATUS_MASK , addr); -trace_cxl_aer_correctable_error (cxlds->cxlmd, status); +trace_cxl_aer_correctable_error (to_cxl_memdev (dev), status); } }
@@ -197,7 +197,7 @@ static void header_log_copy(void __iomem *ras_base, u32 *log)* Log the state of the RAS status registers and prepare them to log the * next error status. Return 1 if reset needed. */ -bool cxl_handle_ras (struct cxl_dev_state *cxlds,void __iomem *ras_base) +bool cxl_handle_ras (struct device *dev,void __iomem *ras_base) {u32 hl[CXL_HEADERLOG_SIZE_U32 ];void __iomem *addr;
@@ -224,7 +224,7 @@ bool cxl_handle_ras(struct cxl_dev_state *cxlds, void __iomem *ras_base) }header_log_copy (ras_base, hl); -trace_cxl_aer_uncorrectable_error (cxlds->cxlmd, status, fe, hl); +trace_cxl_aer_uncorrectable_error (to_cxl_memdev (dev), status, fe, hl);writel (status &CXL_RAS_UNCORRECTABLE_STATUS_MASK , addr);return true;
@@ -246,7 +246,7 @@ void cxl_cor_error_detected(struct pci_dev *pdev)if (cxlds->rcd)cxl_handle_rdport_errors (cxlds); -cxl_handle_cor_ras (cxlds, cxlds->regs.ras); +cxl_handle_cor_ras (&cxlds->cxlmd->dev, cxlds->regs.ras); } }EXPORT_SYMBOL_NS_GPL (cxl_cor_error_detected,"CXL" );
@@ -275,7 +275,7 @@ pci_ers_result_t cxl_error_detected(struct pci_dev *pdev,* chance the situation is recoverable dump the status of the RAS * capability registers and bounce the active state of the memdev. */ - ue =cxl_handle_ras (cxlds, cxlds->regs.ras); + ue =cxl_handle_ras (&cxlds->cxlmd->dev, cxlds->regs.ras); }
drivers/cxl/core/ras_rch.c +2 -2
@@ -115,7 +115,7 @@ void cxl_handle_rdport_errors(struct cxl_dev_state *cxlds)pci_print_aer (pdev, severity, &aer_regs);if (severity ==AER_CORRECTABLE ) -cxl_handle_cor_ras (cxlds, dport->regs.ras); +cxl_handle_cor_ras (&cxlds->cxlmd->dev, dport->regs.ras);else -cxl_handle_ras (cxlds, dport->regs.ras); +cxl_handle_ras (&cxlds->cxlmd->dev, dport->regs.ras); }
Signature · 1 line
2.34.1
PATCH v14 18/34 cxl/port: Remove "enumerate dports" helpers
In reply to Terry BowmanFrom: Dan Williams <[email protected]>
Now that cxl_switch_port_probe() no longer walks potential dports, because they are enumerated dynamically on descendant endpoint arrival, remove the dead code.
---
Changes in v13 -> v14:
- New patch
--- drivers/cxl/core/pci.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
drivers/cxl/core/pci.c +8 -8
@@ -71,6 +71,14 @@ struct cxl_dport *__devm_cxl_add_dport_by_dev(struct cxl_port *port, }EXPORT_SYMBOL_NS_GPL (__devm_cxl_add_dport_by_dev,"CXL" ); +struct cxl_walk_context { +struct pci_bus *bus; +struct cxl_port *port; +int type; +int error; +int count; +}; +static int cxl_dvsec_mem_range_valid (struct cxl_dev_state *cxlds,int id) {struct pci_dev *pdev =to_pci_dev (cxlds->dev);
@@ -820,14 +828,6 @@ int cxl_gpf_port_setup(struct cxl_dport *dport)return 0 ; } -struct cxl_walk_context { -struct pci_bus *bus; -struct cxl_port *port; -int type; -int error; -int count; -}; -static int count_dports (struct pci_dev *pdev,void *data) {struct cxl_walk_context *ctx = data;
Signature · 1 line
2.34.1
Re: [PATCH v14 18/34] cxl/port: Remove "enumerate dports" helpers
Quoted 8 lines · On Wed, 14 Jan 2026 12:20:39 -0600 Terry Bowman <[email protected]> wrote:
From: Dan Williams <[email protected]>
Now that
cxl_switch_port_probe()no longer walks potential dports, because they are enumerated dynamically on descendant endpoint arrival, remove the dead code.Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>
Patch description doesn't match patch.
Quoted 43 lines
---
Changes in v13 -> v14:
- New patch--- drivers/cxl/core/pci.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)drivers/cxl/core/pci.c +8
-8@@ -71,6 +71,14 @@ struct cxl_dport *__devm_cxl_add_dport_by_dev(struct cxl_port *port, }EXPORT_SYMBOL_NS_GPL (__devm_cxl_add_dport_by_dev,"CXL" ); +struct cxl_walk_context { +struct pci_bus *bus; +struct cxl_port *port; +int type; +int error; +int count; +}; +static int cxl_dvsec_mem_range_valid (struct cxl_dev_state *cxlds,int id) {struct pci_dev *pdev =to_pci_dev (cxlds->dev);@@ -820,14 +828,6 @@ int cxl_gpf_port_setup(struct cxl_dport *dport)return 0 ; } -struct cxl_walk_context { -struct pci_bus *bus; -struct cxl_port *port; -int type; -int error; -int count; -}; -static int count_dports (struct pci_dev *pdev,void *data) {struct cxl_walk_context *ctx = data;
Dave Jiang
maintainerRe: [PATCH v14 18/34] cxl/port: Remove "enumerate dports" helpers
Quoted 5 lines · Jonathan Cameron
Quoted 2 lines · On Wed, 14 Jan 2026 12:20:39 -0600 Terry Bowman <[email protected]> wrote:
From: Dan Williams <[email protected]>
Quoted 8 lines
Quoted 6 lines
Now that
cxl_switch_port_probe()no longer walks potential dports, because they are enumerated dynamically on descendant endpoint arrival, remove the dead code.Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>Patch description doesn't match patch.
Looks like it's the clean up of this commit that's upstream.
Quoted 45 lines
Quoted 44 lines
---
Changes in v13 -> v14:
- New patch--- drivers/cxl/core/pci.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)drivers/cxl/core/pci.c +8
-8@@ -71,6 +71,14 @@ struct cxl_dport *__devm_cxl_add_dport_by_dev(struct cxl_port *port, }EXPORT_SYMBOL_NS_GPL (__devm_cxl_add_dport_by_dev,"CXL" ); +struct cxl_walk_context { +struct pci_bus *bus; +struct cxl_port *port; +int type; +int error; +int count; +}; +static int cxl_dvsec_mem_range_valid (struct cxl_dev_state *cxlds,int id) {struct pci_dev *pdev =to_pci_dev (cxlds->dev);@@ -820,14 +828,6 @@ int cxl_gpf_port_setup(struct cxl_dport *dport)return 0 ; } -struct cxl_walk_context { -struct pci_bus *bus; -struct cxl_port *port; -int type; -int error; -int count; -}; -static int count_dports (struct pci_dev *pdev,void *data) {struct cxl_walk_context *ctx = data;
Re: [PATCH v14 18/34] cxl/port: Remove "enumerate dports" helpers
In reply to Jonathan CameronQuoted 5 lines · Jonathan Cameron
Quoted 2 lines · On Wed, 14 Jan 2026 12:20:39 -0600 Terry Bowman <[email protected]> wrote:
From: Dan Williams <[email protected]>
Quoted 8 lines
Quoted 6 lines
Now that
cxl_switch_port_probe()no longer walks potential dports, because they are enumerated dynamically on descendant endpoint arrival, remove the dead code.Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>Patch description doesn't match patch.
Yeah, something got frobbed from the reference patch:
https://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git/commit/?id=ac97e6edd792
Dave Jiang
maintainerRe: [PATCH v14 18/34] cxl/port: Remove "enumerate dports" helpers
In reply to Terry BowmanQuoted 8 lines · Terry Bowman
From: Dan Williams <[email protected]>
Now that
cxl_switch_port_probe()no longer walks potential dports, because they are enumerated dynamically on descendant endpoint arrival, remove the dead code.Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>
Missing your sign off tag
Quoted 43 lines
---
Changes in v13 -> v14:
- New patch--- drivers/cxl/core/pci.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)drivers/cxl/core/pci.c +8
-8@@ -71,6 +71,14 @@ struct cxl_dport *__devm_cxl_add_dport_by_dev(struct cxl_port *port, }EXPORT_SYMBOL_NS_GPL (__devm_cxl_add_dport_by_dev,"CXL" ); +struct cxl_walk_context { +struct pci_bus *bus; +struct cxl_port *port; +int type; +int error; +int count; +}; +static int cxl_dvsec_mem_range_valid (struct cxl_dev_state *cxlds,int id) {struct pci_dev *pdev =to_pci_dev (cxlds->dev);@@ -820,14 +828,6 @@ int cxl_gpf_port_setup(struct cxl_dport *dport)return 0 ; } -struct cxl_walk_context { -struct pci_bus *bus; -struct cxl_port *port; -int type; -int error; -int count; -}; -static int count_dports (struct pci_dev *pdev,void *data) {struct cxl_walk_context *ctx = data;
Re: [PATCH v14 18/34] cxl/port: Remove "enumerate dports" helpers
In reply to Terry BowmanQuoted 8 lines · Terry Bowman
From: Dan Williams <[email protected]>
Now that
cxl_switch_port_probe()no longer walks potential dports, because they are enumerated dynamically on descendant endpoint arrival, remove the dead code.Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>
Terry, the reference patch I sent was based on 6.18-rc4. That means it was missing this change:
3f5b8f7f34f6 cxl/port: Remove devm_cxl_port_enumerate_dports()
This cxl_walk_context move, which was not part of my original patch, also does not appear to be needed, so this patch can simply be dropped.
PATCH v14 19/34 cxl/port: Fix devm resource leaks around with dport management
In reply to Terry BowmanFrom: Dan Williams <[email protected]>
With dport addition moving out of cxl_switch_port_probe() it is no longer the case that a single dport-add failure will cause all dport resources to be automatically unwound.
devm still helps all dport resources get cleaned up when the port is detached, but setup now needs to avoid leaking resources if an early exit occurs during setup.
Convert from a "devm add" model, to an "auto remove" model that makes the caller responsible for registering devm reclaim after the object is fully instantiated.
As a side of effect of this reorganization port->nr_dports is now always consistent with the number of entries in the port->dports xarray, and this can stop playing games with ida_is_empty() which is unreliable as a detector of whether decoders are setup. I.e. consider how CONFIG_DEBUG_KOBJECT_RELEASE might wreak havoc with this approach.
---
Changes in v13 -> v14:
- New patch
--- drivers/cxl/acpi.c | 11 +- drivers/cxl/core/pci.c | 10 +- drivers/cxl/core/port.c | 225 ++++++++++++++++----------- drivers/cxl/cxl.h | 23 +-- drivers/cxl/port.c | 8 +- tools/testing/cxl/Kbuild | 3 +- tools/testing/cxl/cxl_core_exports.c | 13 +- tools/testing/cxl/exports.h | 4 +- tools/testing/cxl/test/cxl.c | 6 +- tools/testing/cxl/test/mock.c | 25 ++- tools/testing/cxl/test/mock.h | 4 +- 11 files changed, 188 insertions(+), 144 deletions(-)
drivers/cxl/acpi.c +7 -4
@@ -679,16 +679,19 @@ static int add_host_bridge_dport(struct device *match, void *arg)if (ctx.cxl_version ==ACPI_CEDT_CHBS_VERSION_CXL11 ) {dev_dbg (match,"RCRB found for UID %lld: %pa\n" , ctx.uid, &ctx.base); - dport =devm_cxl_add_rch_dport (root_port, bridge, ctx.uid, - ctx.base); + dport =cxl_add_rch_dport (root_port, bridge, ctx.uid, ctx.base); }else { - dport =devm_cxl_add_dport (root_port, bridge, ctx.uid, -CXL_RESOURCE_NONE ); + dport =cxl_add_dport (root_port, bridge, ctx.uid, +CXL_RESOURCE_NONE ); }if (IS_ERR (dport))return PTR_ERR (dport); + ret =cxl_dport_autoremove (dport); +if (ret) +return ret; + ret =get_genport_coordinates (match, dport);if (ret)dev_dbg (match,"Failed to get generic port perf coordinates.\n" );
drivers/cxl/core/pci.c +5 -5
@@ -41,14 +41,14 @@ static int pci_get_port_num(struct pci_dev *pdev) }/** -* __devm_cxl_add_dport_by_dev - allocate a dport by dport device +* __cxl_add_dport_by_dev - allocate a dport by dport device * @port: cxl_port that hosts the dport * @dport_dev: 'struct device' of the dport * * Returns the allocated dport on success or ERR_PTR() of -errno on error */ -struct cxl_dport *__devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev) +struct cxl_dport *__cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev) {struct cxl_register_map map;struct pci_dev *pdev;
@@ -67,9 +67,9 @@ struct cxl_dport *__devm_cxl_add_dport_by_dev(struct cxl_port *port,return ERR_PTR (rc);device_lock_assert (&port->dev); -return devm_cxl_add_dport (port, dport_dev, port_num, map.resource); +return cxl_add_dport (port, dport_dev, port_num, map.resource); } -EXPORT_SYMBOL_NS_GPL (__devm_cxl_add_dport_by_dev,"CXL" ); +EXPORT_SYMBOL_NS_GPL (__cxl_add_dport_by_dev,"CXL" );struct cxl_walk_context {struct pci_bus *bus;
drivers/cxl/core/port.c +131 -94
353 lines not on this page · raw message
drivers/cxl/cxl.h +12 -11
@@ -796,12 +796,12 @@ struct cxl_port *cxl_mem_find_port(struct cxl_memdev *cxlmd, struct cxl_dport **dport);bool schedule_cxl_memdev_detach (struct cxl_memdev *cxlmd); -struct cxl_dport *devm_cxl_add_dport (struct cxl_port *port, -struct device *dport,int port_id, -resource_size_t component_reg_phys); -struct cxl_dport *devm_cxl_add_rch_dport (struct cxl_port *port, -struct device *dport_dev,int port_id, -resource_size_t rcrb); +struct cxl_dport *cxl_add_dport (struct cxl_port *port,struct device *dport, +int port_id, +resource_size_t component_reg_phys); +struct cxl_dport *cxl_add_rch_dport (struct cxl_port *port, +struct device *dport_dev,int port_id, +resource_size_t rcrb);struct cxl_decoder *to_cxl_decoder (struct device *dev);struct cxl_root_decoder *to_cxl_root_decoder (struct device *dev);
@@ -824,6 +824,7 @@ static inline int cxl_root_decoder_autoremove(struct device *host,return cxl_decoder_autoremove (host, &cxlrd->cxlsd.cxld); }int cxl_endpoint_autoremove (struct cxl_memdev *cxlmd,struct cxl_port *endpoint); +int cxl_dport_autoremove (struct cxl_dport *dport);/** * struct cxl_endpoint_dvsec_info - Cached DVSEC info
@@ -937,10 +938,10 @@ void cxl_coordinates_combine(struct access_coordinate *out, struct access_coordinate *c2);bool cxl_endpoint_decoder_reset_detected (struct cxl_port *port); -struct cxl_dport *devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev); -struct cxl_dport *__devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev); +struct cxl_dport *cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev); +struct cxl_dport *__cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev);/* * Unit test builds overrides this to __weak, find the 'strong' version
@@ -964,7 +965,7 @@ u16 cxl_gpf_get_dvsec(struct device *dev);*/ #ifndef CXL_TEST_ENABLE #define DECLARE_TESTABLE (x) __##x -#define devm_cxl_add_dport_by_dev DECLARE_TESTABLE(devm_cxl_add_dport_by_dev) +#define cxl_add_dport_by_dev DECLARE_TESTABLE(cxl_add_dport_by_dev)#define devm_cxl_switch_port_decoders_setup DECLARE_TESTABLE(devm_cxl_switch_port_decoders_setup)#endif
drivers/cxl/port.c +6 -2
@@ -59,8 +59,12 @@ static int discover_region(struct device *dev, void *unused)static int cxl_switch_port_probe (struct cxl_port *port) { -/* Reset nr_dports for rebind of driver */ - port->nr_dports =0 ; +/* +* Unfortunately, typical driver operations like "find and map +* registers", can not be done at port device attach time and must wait +* for dport arrival. See cxl_port_add_dport() and the comments in +* add_dport() for details. +*/ /* Cache the data early to ensure is_visible() works */ read_cdat_data (port);
tools/testing/cxl/Kbuild +2 -1
@@ -5,7 +5,8 @@ ldflags-y += --wrap=acpi_evaluate_integer ldflags-y += --wrap=acpi_pci_find_root ldflags-y += --wrap=nvdimm_bus_register ldflags-y += --wrap=cxl_await_media_ready -ldflags-y += --wrap=devm_cxl_add_rch_dport +ldflags-y += --wrap=cxl_add_rch_dport +ldflags-y += --wrap=cxl_rcd_component_reg_phys ldflags-y += --wrap=cxl_endpoint_parse_cdat ldflags-y += --wrap=cxl_dport_init_ras_reporting ldflags-y += --wrap=devm_cxl_endpoint_decoders_setup
tools/testing/cxl/cxl_core_exports.c +6 -7
@@ -7,16 +7,15 @@/* Exporting of cxl_core symbols that are only used by cxl_test */ EXPORT_SYMBOL_NS_GPL (cxl_num_decoders_committed,"CXL" ); -cxl_add_dport_by_dev_fn _devm_cxl_add_dport_by_dev = - __devm_cxl_add_dport_by_dev; -EXPORT_SYMBOL_NS_GPL (_devm_cxl_add_dport_by_dev,"CXL" ); +cxl_add_dport_by_dev_fn _cxl_add_dport_by_dev = __cxl_add_dport_by_dev; +EXPORT_SYMBOL_NS_GPL (_cxl_add_dport_by_dev,"CXL" ); -struct cxl_dport *devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev) +struct cxl_dport *cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev) { -return _devm_cxl_add_dport_by_dev (port, dport_dev); +return _cxl_add_dport_by_dev (port, dport_dev); } -EXPORT_SYMBOL_NS_GPL (devm_cxl_add_dport_by_dev,"CXL" ); +EXPORT_SYMBOL_NS_GPL (cxl_add_dport_by_dev,"CXL" );cxl_switch_decoders_setup_fn _devm_cxl_switch_port_decoders_setup = __devm_cxl_switch_port_decoders_setup;
tools/testing/cxl/exports.h +2 -2
@@ -4,8 +4,8 @@#define __MOCK_CXL_EXPORTS_H_typedef struct cxl_dport *(*cxl_add_dport_by_dev_fn )(struct cxl_port *port, -struct device *dport_dev); -extern cxl_add_dport_by_dev_fn _devm_cxl_add_dport_by_dev; +struct device *dport_dev); +extern cxl_add_dport_by_dev_fn _cxl_add_dport_by_dev;typedef int (*cxl_switch_decoders_setup_fn )(struct cxl_port *port);extern cxl_switch_decoders_setup_fn _devm_cxl_switch_port_decoders_setup;
tools/testing/cxl/test/cxl.c +3 -3
@@ -1060,8 +1060,8 @@ static struct cxl_dport *mock_cxl_add_dport_by_dev(struct cxl_port *port,if (&pdev->dev != dport_dev)continue ; -return devm_cxl_add_dport (port, &pdev->dev, pdev->id, -CXL_RESOURCE_NONE ); +return cxl_add_dport (port, &pdev->dev, pdev->id, +CXL_RESOURCE_NONE ); }return ERR_PTR (-ENODEV );
@@ -1126,9 +1126,9 @@ static struct cxl_mock_ops cxl_mock_ops = { .devm_cxl_switch_port_decoders_setup = mock_cxl_switch_port_decoders_setup, .devm_cxl_endpoint_decoders_setup = mock_cxl_endpoint_decoders_setup, .cxl_endpoint_parse_cdat = mock_cxl_endpoint_parse_cdat, - .devm_cxl_add_dport_by_dev = mock_cxl_add_dport_by_dev, .hmat_get_extended_linear_cache_size = mock_hmat_get_extended_linear_cache_size, + .cxl_add_dport_by_dev = mock_cxl_add_dport_by_dev, .list = LIST_HEAD_INIT(cxl_mock_ops.list), };
tools/testing/cxl/test/mock.c +12 -13
@@ -15,14 +15,13 @@static LIST_HEAD (mock);static struct cxl_dport * -redirect_devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev); +redirect_cxl_add_dport_by_dev (struct cxl_port *port,struct device *dport_dev);static int redirect_devm_cxl_switch_port_decoders_setup (struct cxl_port *port);void register_cxl_mock_ops (struct cxl_mock_ops *ops) {list_add_rcu (&ops->list, &mock); - _devm_cxl_add_dport_by_dev = redirect_devm_cxl_add_dport_by_dev; + _cxl_add_dport_by_dev = redirect_cxl_add_dport_by_dev; _devm_cxl_switch_port_decoders_setup = redirect_devm_cxl_switch_port_decoders_setup; }
@@ -34,7 +33,7 @@ void unregister_cxl_mock_ops(struct cxl_mock_ops *ops) { _devm_cxl_switch_port_decoders_setup = __devm_cxl_switch_port_decoders_setup; - _devm_cxl_add_dport_by_dev = __devm_cxl_add_dport_by_dev; + _cxl_add_dport_by_dev = __cxl_add_dport_by_dev;list_del_rcu (&ops->list);synchronize_srcu (&cxl_mock_srcu); }
@@ -207,7 +206,7 @@ int __wrap_cxl_await_media_ready(struct cxl_dev_state *cxlds) }EXPORT_SYMBOL_NS_GPL (__wrap_cxl_await_media_ready,"CXL" ); -struct cxl_dport *__wrap_devm_cxl_add_rch_dport (struct cxl_port *port, +struct cxl_dport *__wrap_cxl_add_rch_dport (struct cxl_port *port,struct device *dport_dev,int port_id,resource_size_t rcrb)
@@ -217,19 +216,19 @@ struct cxl_dport *__wrap_devm_cxl_add_rch_dport(struct cxl_port *port,struct cxl_mock_ops *ops =get_cxl_mock_ops (&index);if (ops && ops->is_mock_port (dport_dev)) { - dport =devm_cxl_add_dport (port, dport_dev, port_id, -CXL_RESOURCE_NONE ); + dport =cxl_add_dport (port, dport_dev, port_id, +CXL_RESOURCE_NONE );if (!IS_ERR (dport)) { dport->rcrb.base = rcrb; dport->rch = true; } }else - dport =devm_cxl_add_rch_dport (port, dport_dev, port_id, rcrb); + dport =cxl_add_rch_dport (port, dport_dev, port_id, rcrb);put_cxl_mock_ops (index);return dport; } -EXPORT_SYMBOL_NS_GPL (__wrap_devm_cxl_add_rch_dport,"CXL" ); +EXPORT_SYMBOL_NS_GPL (__wrap_cxl_add_rch_dport,"CXL" );void __wrap_cxl_endpoint_parse_cdat (struct cxl_port *port) {
@@ -257,17 +256,17 @@ void __wrap_cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device }EXPORT_SYMBOL_NS_GPL (__wrap_cxl_dport_init_ras_reporting,"CXL" ); -struct cxl_dport *redirect_devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev) +struct cxl_dport *redirect_cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev) {int index;struct cxl_mock_ops *ops =get_cxl_mock_ops (&index);struct cxl_dport *dport;if (ops && ops->is_mock_port (port->uport_dev)) - dport = ops->devm_cxl_add_dport_by_dev (port, dport_dev); + dport = ops->cxl_add_dport_by_dev (port, dport_dev);else - dport =__devm_cxl_add_dport_by_dev (port, dport_dev); + dport =__cxl_add_dport_by_dev (port, dport_dev);put_cxl_mock_ops (index);return dport;
tools/testing/cxl/test/mock.h +2 -2
@@ -22,8 +22,8 @@ struct cxl_mock_ops {int (*devm_cxl_switch_port_decoders_setup)(struct cxl_port *port);int (*devm_cxl_endpoint_decoders_setup)(struct cxl_port *port);void (*cxl_endpoint_parse_cdat)(struct cxl_port *port); -struct cxl_dport *(*devm_cxl_add_dport_by_dev)(struct cxl_port *port, -struct device *dport_dev); +struct cxl_dport *(*cxl_add_dport_by_dev)(struct cxl_port *port, +struct device *dport_dev);int (*hmat_get_extended_linear_cache_size)(struct resource *backing_res,int nid,resource_size_t *cache_size);
Signature · 1 line
2.34.1
Re: [PATCH v14 19/34] cxl/port: Fix devm resource leaks around with dport management
Quoted 14 lines · Terry Bowman
From: Dan Williams <[email protected]>
With dport addition moving out of
cxl_switch_port_probe()it is no longer the case that a single dport-add failure will cause all dport resources to be automatically unwound.devm still helps all dport resources get cleaned up when the port is detached, but setup now needs to avoid leaking resources if an early exit occurs during setup.
Convert from a "devm add" model, to an "auto remove" model that makes the caller responsible for registering devm reclaim after the object is fully instantiated.
Quoted 8 lines
As a side of effect of this reorganization
port->nr_dportsis now always consistent with the number of entries in theport->dportsxarray, and this can stop playing games withida_is_empty()which is unreliable as a detector of whether decoders are setup. I.e. consider howCONFIG_DEBUG_KOBJECT_RELEASEmight wreak havoc with this approach.Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>
Missing sign off tag
Quoted 706 lines
---
Changes in v13 -> v14:
- New patch--- drivers/cxl/acpi.c | 11 +- drivers/cxl/core/pci.c | 10 +- drivers/cxl/core/port.c | 225 ++++++++++++++++----------- drivers/cxl/cxl.h | 23 +-- drivers/cxl/port.c | 8 +- tools/testing/cxl/Kbuild | 3 +- tools/testing/cxl/cxl_core_exports.c | 13 +- tools/testing/cxl/exports.h | 4 +- tools/testing/cxl/test/cxl.c | 6 +- tools/testing/cxl/test/mock.c | 25 ++- tools/testing/cxl/test/mock.h | 4 +- 11 files changed, 188 insertions(+), 144 deletions(-)drivers/cxl/acpi.c +7
-4@@ -679,16 +679,19 @@ static int add_host_bridge_dport(struct device *match, void *arg)if (ctx.cxl_version ==ACPI_CEDT_CHBS_VERSION_CXL11 ) {dev_dbg (match,"RCRB found for UID %lld: %pa\n" , ctx.uid, &ctx.base); - dport =devm_cxl_add_rch_dport (root_port, bridge, ctx.uid, - ctx.base); + dport =cxl_add_rch_dport (root_port, bridge, ctx.uid, ctx.base); }else { - dport =devm_cxl_add_dport (root_port, bridge, ctx.uid, -CXL_RESOURCE_NONE ); + dport =cxl_add_dport (root_port, bridge, ctx.uid, +CXL_RESOURCE_NONE ); }if (IS_ERR (dport))return PTR_ERR (dport); + ret =cxl_dport_autoremove (dport); +if (ret) +return ret; + ret =get_genport_coordinates (match, dport);if (ret)dev_dbg (match,"Failed to get generic port perf coordinates.\n" );drivers/cxl/core/pci.c +5
-5@@ -41,14 +41,14 @@ static int pci_get_port_num(struct pci_dev *pdev) }/** -* __devm_cxl_add_dport_by_dev - allocate a dport by dport device +* __cxl_add_dport_by_dev - allocate a dport by dport device * @port: cxl_port that hosts the dport * @dport_dev: 'struct device' of the dport * * Returns the allocated dport on success or ERR_PTR() of -errno on error */ -struct cxl_dport *__devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev) +struct cxl_dport *__cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev) {struct cxl_register_map map;struct pci_dev *pdev;@@ -67,9 +67,9 @@ struct cxl_dport *__devm_cxl_add_dport_by_dev(struct cxl_port *port,return ERR_PTR (rc);device_lock_assert (&port->dev); -return devm_cxl_add_dport (port, dport_dev, port_num, map.resource); +return cxl_add_dport (port, dport_dev, port_num, map.resource); } -EXPORT_SYMBOL_NS_GPL (__devm_cxl_add_dport_by_dev,"CXL" ); +EXPORT_SYMBOL_NS_GPL (__cxl_add_dport_by_dev,"CXL" );struct cxl_walk_context {struct pci_bus *bus;drivers/cxl/core/port.c +131
-94@@ -1051,7 +1051,8 @@ static struct cxl_dport *find_dport(struct cxl_port *port, int id)return NULL ; } -static int add_dport (struct cxl_port *port,struct cxl_dport *dport) +static struct cxl_dport *add_dport (struct cxl_port *port, +struct cxl_dport *dport) {struct cxl_dport *dup;int rc;@@ -1063,16 +1064,33 @@ static int add_dport(struct cxl_port *port, struct cxl_dport *dport)"unable to add dport%d-%s non-unique port id (%s)\n" , dport->port_id,dev_name (dport->dport_dev), dev_name(dup->dport_dev)); -return -EBUSY ; +return ERR_PTR (-EBUSY ); + } + +/* +* Unlike CXL switch upstream ports where it can train a CXL link +* independent of its downstream ports, a host bridge upstream port may +* not enable CXL registers until at least one downstream port (root +* port) trains CXL. Enumerate registers once when the number of dports +* transitions from zero to one. +*/ +if (!port->nr_dports) { + rc =cxl_port_setup_regs (port, port->component_reg_phys); +if (rc) +return ERR_PTR (rc); } +/* Arrange for dport_dev to be valid through remove_dport() */ + struct device *dev __free(put_device) = get_device(dport->dport_dev); + rc =xa_insert (&port->dports, (unsigned long )dport->dport_dev, dport,GFP_KERNEL );if (rc) -return rc; +return ERR_PTR (rc); +retain_and_null_ptr (dev); port->nr_dports++; -return 0 ; +return dport; }/* @@ -1094,51 +1112,32 @@ static void cond_cxl_root_unlock(struct cxl_port *port)device_unlock (&port->dev); } -static void cxl_dport_remove (void *data) +static void remove_dport (struct cxl_dport *dport) { -struct cxl_dport *dport = data;struct cxl_port *port = dport->port; + port->nr_dports--;xa_erase (&port->dports, (unsigned long ) dport->dport_dev);put_device (dport->dport_dev); } -static void cxl_dport_unlink (void *data) -{ -struct cxl_dport *dport = data; -struct cxl_port *port = dport->port; -char link_name[CXL_TARGET_STRLEN ]; +DEFINE_FREE(remove_dport, struct cxl_dport *, + if (!IS_ERR_OR_NULL(_T)) remove_dport(_T)) -sprintf (link_name,"dport%d" , dport->port_id); -sysfs_remove_link (&port->dev.kobj, link_name); -} - -static struct cxl_dport * -__devm_cxl_add_dport (struct cxl_port *port,struct device *dport_dev, -int port_id,resource_size_t component_reg_phys, -resource_size_t rcrb) +static struct cxl_dport *__cxl_add_dport (struct cxl_port *port, +struct device *dport_dev,int port_id, +resource_size_t component_reg_phys, +resource_size_t rcrb) {char link_name[CXL_TARGET_STRLEN ]; -struct cxl_dport *dport; -struct device *host;int rc; -if (is_cxl_root (port)) - host = port->uport_dev; -else - host = &port->dev; - -if (!host->driver) { -dev_WARN_ONCE (&port->dev,1 ,"dport:%s bad devm context\n" , -dev_name (dport_dev)); -return ERR_PTR (-ENXIO ); - } -if (snprintf (link_name,CXL_TARGET_STRLEN ,"dport%d" , port_id) >=CXL_TARGET_STRLEN )return ERR_PTR (-EINVAL ); - dport =devm_kzalloc (host,sizeof (*dport),GFP_KERNEL ); + struct cxl_dport *dport __free(kfree) = +kzalloc (sizeof (*dport),GFP_KERNEL );if (!dport)return ERR_PTR (-ENOMEM );@@ -1176,48 +1175,27 @@ __devm_cxl_add_dport(struct cxl_port *port, struct device *dport_dev, &component_reg_phys);cond_cxl_root_lock (port); - rc =add_dport (port, dport); + struct cxl_dport *dport_add __free(remove_dport) = +add_dport (port, dport);cond_cxl_root_unlock (port); -if (rc) -return ERR_PTR (rc); - -/* -* Setup port register if this is the first dport showed up. Having -* a dport also means that there is at least 1 active link. -*/ -if (port->nr_dports ==1 && - port->component_reg_phys !=CXL_RESOURCE_NONE ) { - rc =cxl_port_setup_regs (port, port->component_reg_phys); -if (rc) { -xa_erase (&port->dports, (unsigned long )dport->dport_dev); -return ERR_PTR (rc); - } - port->component_reg_phys =CXL_RESOURCE_NONE ; - } +if (IS_ERR (dport_add)) +return dport_add; -get_device (dport_dev); - rc =devm_add_action_or_reset (host, cxl_dport_remove, dport); -if (rc) -return ERR_PTR (rc); +if (dev_is_pci (dport_dev)) + dport->link_latency =cxl_pci_get_latency (to_pci_dev (dport_dev)); rc =sysfs_create_link (&port->dev.kobj, &dport_dev->kobj, link_name);if (rc)return ERR_PTR (rc); - rc =devm_add_action_or_reset (host, cxl_dport_unlink, dport); -if (rc) -return ERR_PTR (rc); - -if (dev_is_pci (dport_dev)) - dport->link_latency =cxl_pci_get_latency (to_pci_dev (dport_dev)); -cxl_debugfs_create_dport_dir (dport); -return dport; +retain_and_null_ptr (dport_add); +return no_free_ptr (dport); }/** -* devm_cxl_add_dport - append VH downstream port data to a cxl_port +* cxl_add_dport - append VH downstream port data to a cxl_port * @port: the cxl_port that references this dport * @dport_dev: firmware or PCI device representing the dport * @port_id: identifier for this dport in a decoder's target list @@ -1227,14 +1205,13 @@ __devm_cxl_add_dport(struct cxl_port *port, struct device *dport_dev,* either the port's host (for root ports), or the port itself (for * switch ports) */ -struct cxl_dport *devm_cxl_add_dport (struct cxl_port *port, -struct device *dport_dev,int port_id, -resource_size_t component_reg_phys) +struct cxl_dport *cxl_add_dport (struct cxl_port *port,struct device *dport_dev, +int port_id,resource_size_t component_reg_phys) {struct cxl_dport *dport; - dport =__devm_cxl_add_dport (port, dport_dev, port_id, - component_reg_phys,CXL_RESOURCE_NONE ); + dport =__cxl_add_dport (port, dport_dev, port_id, component_reg_phys, +CXL_RESOURCE_NONE );if (IS_ERR (dport)) {dev_dbg (dport_dev,"failed to add dport to %s: %ld\n" ,dev_name (&port->dev),PTR_ERR (dport));@@ -1245,10 +1222,10 @@ struct cxl_dport *devm_cxl_add_dport(struct cxl_port *port,return dport; } -EXPORT_SYMBOL_NS_GPL (devm_cxl_add_dport,"CXL" ); +EXPORT_SYMBOL_NS_GPL (cxl_add_dport,"CXL" );/** -* devm_cxl_add_rch_dport - append RCH downstream port data to a cxl_port +* cxl_add_rch_dport - append RCH downstream port data to a cxl_port * @port: the cxl_port that references this dport * @dport_dev: firmware or PCI device representing the dport * @port_id: identifier for this dport in a decoder's target list @@ -1256,9 +1233,9 @@ EXPORT_SYMBOL_NS_GPL(devm_cxl_add_dport, "CXL");* * See CXL 3.0 9.11.8 CXL Devices Attached to an RCH */ -struct cxl_dport *devm_cxl_add_rch_dport (struct cxl_port *port, -struct device *dport_dev,int port_id, -resource_size_t rcrb) +struct cxl_dport *cxl_add_rch_dport (struct cxl_port *port, +struct device *dport_dev,int port_id, +resource_size_t rcrb) {struct cxl_dport *dport;@@ -1267,8 +1244,8 @@ struct cxl_dport *devm_cxl_add_rch_dport(struct cxl_port *port,return ERR_PTR (-EINVAL ); } - dport =__devm_cxl_add_dport (port, dport_dev, port_id, -CXL_RESOURCE_NONE , rcrb); + dport =__cxl_add_dport (port, dport_dev, port_id,CXL_RESOURCE_NONE , + rcrb);if (IS_ERR (dport)) {dev_dbg (dport_dev,"failed to add RCH dport to %s: %ld\n" ,dev_name (&port->dev),PTR_ERR (dport));@@ -1279,7 +1256,7 @@ struct cxl_dport *devm_cxl_add_rch_dport(struct cxl_port *port,return dport; } -EXPORT_SYMBOL_NS_GPL (devm_cxl_add_rch_dport,"CXL" ); +EXPORT_SYMBOL_NS_GPL (cxl_add_rch_dport,"CXL" );static int add_ep (struct cxl_ep *new) {@@ -1439,13 +1416,42 @@ static void delete_switch_port(struct cxl_port *port)devm_release_action (port->dev.parent, unregister_port, port); } +static void unlink_dport (void *data) +{ +struct cxl_dport *dport = data; +struct cxl_port *port = dport->port; +char link_name[CXL_TARGET_STRLEN ]; + +sprintf (link_name,"dport%d" , dport->port_id); +sysfs_remove_link (&port->dev.kobj, link_name); +remove_dport (dport); +kfree (dport); +} + +int cxl_dport_autoremove (struct cxl_dport *dport) +{ +struct cxl_port *port = dport->port; +struct device *host; + +if (is_cxl_root (port)) + host = port->uport_dev; +else + host = &port->dev; + +return devm_add_action_or_reset (host, unlink_dport, dport); +} +EXPORT_SYMBOL_NS_GPL (cxl_dport_autoremove,"CXL" ); + +/* +* Note: this only services dynamic removal of mid-level ports, root ports are +* always removed by the platform driver (e.g. cxl_acpi). @host can be +* hard-coded to &port->dev. +*/ static void del_dport (struct cxl_dport *dport) {struct cxl_port *port = dport->port; -devm_release_action (&port->dev, cxl_dport_unlink, dport); -devm_release_action (&port->dev, cxl_dport_remove, dport); -devm_kfree (&port->dev, dport); +devm_release_action (&port->dev, unlink_dport, dport); }static void del_dports (struct cxl_port *port)@@ -1597,10 +1603,24 @@ static int update_decoder_targets(struct device *dev, void *data)return 0 ; } -DEFINE_FREE(del_cxl_dport, struct cxl_dport *, if (!IS_ERR_OR_NULL(_T)) del_dport(_T)) +static struct cxl_port *cxl_port_devres_group (struct cxl_port *port) +{ +if (!devres_open_group (&port->dev, port,GFP_KERNEL )) +return ERR_PTR (-ENOMEM ); +return port; +} +DEFINE_FREE(cxl_port_group_free, struct cxl_port *, + if (!IS_ERR_OR_NULL(_T)) devres_release_group(&(_T)->dev, _T)) + +static void cxl_port_group_close (struct cxl_port *port) +{ +devres_remove_group (&port->dev, port); +} +static struct cxl_dport *cxl_port_add_dport (struct cxl_port *port,struct device *dport_dev) { +struct cxl_dport *new_dport;struct cxl_dport *dport;int rc;@@ -1615,29 +1635,46 @@ static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port,return ERR_PTR (-EBUSY ); } - struct cxl_dport *new_dport __free(del_cxl_dport) = -devm_cxl_add_dport_by_dev (port, dport_dev); -if (IS_ERR (new_dport)) -return new_dport; - -cxl_switch_parse_cdat (new_dport); +/* +* With the first dport arrival it is now safe to start looking at +* component registers. Be careful to not strand resources if dport +* creation ultimately fails. +*/ + struct cxl_port *port_group __free(cxl_port_group_free) = +cxl_port_devres_group (port); +if (IS_ERR (port_group)) +return ERR_CAST (port_group); -if (ida_is_empty (&port->decoder_ida)) { +if (port->nr_dports ==0 ) { rc =devm_cxl_switch_port_decoders_setup (port);if (rc)return ERR_PTR (rc); -dev_dbg (&port->dev,"first dport%d:%s added with decoders\n" , - new_dport->port_id,dev_name (dport_dev)); -return no_free_ptr (new_dport); +/* +* Note, when nr_dports returns to zero the port is unregistered +* and triggers cleanup. I.e. no need for open-coded release +* action on dport removal. See cxl_detach_ep() for that logic. +*/ } + new_dport =cxl_add_dport_by_dev (port, dport_dev); +if (IS_ERR (new_dport)) +return new_dport; + + rc =cxl_dport_autoremove (new_dport); +if (rc) +return ERR_PTR (rc); + +cxl_switch_parse_cdat (new_dport); + +cxl_port_group_close (no_free_ptr (port_group)); + +dev_dbg (&port->dev,"dport[%d] id:%d dport_dev: %s added\n" , + port->nr_dports -1 , new_dport->port_id,dev_name (dport_dev)); +/* New dport added, update the decoder targets */ device_for_each_child (&port->dev, new_dport, update_decoder_targets); -dev_dbg (&port->dev,"dport%d:%s added\n" , new_dport->port_id, -dev_name (dport_dev)); - -return no_free_ptr (new_dport); +return new_dport; } static struct cxl_dport *devm_cxl_create_port(struct device *ep_dev,drivers/cxl/cxl.h +12
-11@@ -796,12 +796,12 @@ struct cxl_port *cxl_mem_find_port(struct cxl_memdev *cxlmd, struct cxl_dport **dport);bool schedule_cxl_memdev_detach (struct cxl_memdev *cxlmd); -struct cxl_dport *devm_cxl_add_dport (struct cxl_port *port, -struct device *dport,int port_id, -resource_size_t component_reg_phys); -struct cxl_dport *devm_cxl_add_rch_dport (struct cxl_port *port, -struct device *dport_dev,int port_id, -resource_size_t rcrb); +struct cxl_dport *cxl_add_dport (struct cxl_port *port,struct device *dport, +int port_id, +resource_size_t component_reg_phys); +struct cxl_dport *cxl_add_rch_dport (struct cxl_port *port, +struct device *dport_dev,int port_id, +resource_size_t rcrb);struct cxl_decoder *to_cxl_decoder (struct device *dev);struct cxl_root_decoder *to_cxl_root_decoder (struct device *dev);@@ -824,6 +824,7 @@ static inline int cxl_root_decoder_autoremove(struct device *host,return cxl_decoder_autoremove (host, &cxlrd->cxlsd.cxld); }int cxl_endpoint_autoremove (struct cxl_memdev *cxlmd,struct cxl_port *endpoint); +int cxl_dport_autoremove (struct cxl_dport *dport);/** * struct cxl_endpoint_dvsec_info - Cached DVSEC info @@ -937,10 +938,10 @@ void cxl_coordinates_combine(struct access_coordinate *out, struct access_coordinate *c2);bool cxl_endpoint_decoder_reset_detected (struct cxl_port *port); -struct cxl_dport *devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev); -struct cxl_dport *__devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev); +struct cxl_dport *cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev); +struct cxl_dport *__cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev);/* * Unit test builds overrides this to __weak, find the 'strong' version @@ -964,7 +965,7 @@ u16 cxl_gpf_get_dvsec(struct device *dev);*/ #ifndef CXL_TEST_ENABLE #define DECLARE_TESTABLE (x) __##x -#define devm_cxl_add_dport_by_dev DECLARE_TESTABLE(devm_cxl_add_dport_by_dev) +#define cxl_add_dport_by_dev DECLARE_TESTABLE(cxl_add_dport_by_dev)#define devm_cxl_switch_port_decoders_setup DECLARE_TESTABLE(devm_cxl_switch_port_decoders_setup)#endif drivers/cxl/port.c +6
-2@@ -59,8 +59,12 @@ static int discover_region(struct device *dev, void *unused)static int cxl_switch_port_probe (struct cxl_port *port) { -/* Reset nr_dports for rebind of driver */ - port->nr_dports =0 ; +/* +* Unfortunately, typical driver operations like "find and map +* registers", can not be done at port device attach time and must wait +* for dport arrival. See cxl_port_add_dport() and the comments in +* add_dport() for details. +*/ /* Cache the data early to ensure is_visible() works */ read_cdat_data (port);tools/testing/cxl/Kbuild +2
-1@@ -5,7 +5,8 @@ ldflags-y += --wrap=acpi_evaluate_integer ldflags-y += --wrap=acpi_pci_find_root ldflags-y += --wrap=nvdimm_bus_register ldflags-y += --wrap=cxl_await_media_ready -ldflags-y += --wrap=devm_cxl_add_rch_dport +ldflags-y += --wrap=cxl_add_rch_dport +ldflags-y += --wrap=cxl_rcd_component_reg_phys ldflags-y += --wrap=cxl_endpoint_parse_cdat ldflags-y += --wrap=cxl_dport_init_ras_reporting ldflags-y += --wrap=devm_cxl_endpoint_decoders_setuptools/testing/cxl/cxl_core_exports.c +6
-7@@ -7,16 +7,15 @@/* Exporting of cxl_core symbols that are only used by cxl_test */ EXPORT_SYMBOL_NS_GPL (cxl_num_decoders_committed,"CXL" ); -cxl_add_dport_by_dev_fn _devm_cxl_add_dport_by_dev = - __devm_cxl_add_dport_by_dev; -EXPORT_SYMBOL_NS_GPL (_devm_cxl_add_dport_by_dev,"CXL" ); +cxl_add_dport_by_dev_fn _cxl_add_dport_by_dev = __cxl_add_dport_by_dev; +EXPORT_SYMBOL_NS_GPL (_cxl_add_dport_by_dev,"CXL" ); -struct cxl_dport *devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev) +struct cxl_dport *cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev) { -return _devm_cxl_add_dport_by_dev (port, dport_dev); +return _cxl_add_dport_by_dev (port, dport_dev); } -EXPORT_SYMBOL_NS_GPL (devm_cxl_add_dport_by_dev,"CXL" ); +EXPORT_SYMBOL_NS_GPL (cxl_add_dport_by_dev,"CXL" );cxl_switch_decoders_setup_fn _devm_cxl_switch_port_decoders_setup = __devm_cxl_switch_port_decoders_setup;tools/testing/cxl/exports.h +2
-2@@ -4,8 +4,8 @@#define __MOCK_CXL_EXPORTS_H_typedef struct cxl_dport *(*cxl_add_dport_by_dev_fn )(struct cxl_port *port, -struct device *dport_dev); -extern cxl_add_dport_by_dev_fn _devm_cxl_add_dport_by_dev; +struct device *dport_dev); +extern cxl_add_dport_by_dev_fn _cxl_add_dport_by_dev;typedef int (*cxl_switch_decoders_setup_fn )(struct cxl_port *port);extern cxl_switch_decoders_setup_fn _devm_cxl_switch_port_decoders_setup;tools/testing/cxl/test/cxl.c +3
-3@@ -1060,8 +1060,8 @@ static struct cxl_dport *mock_cxl_add_dport_by_dev(struct cxl_port *port,if (&pdev->dev != dport_dev)continue ; -return devm_cxl_add_dport (port, &pdev->dev, pdev->id, -CXL_RESOURCE_NONE ); +return cxl_add_dport (port, &pdev->dev, pdev->id, +CXL_RESOURCE_NONE ); }return ERR_PTR (-ENODEV );@@ -1126,9 +1126,9 @@ static struct cxl_mock_ops cxl_mock_ops = { .devm_cxl_switch_port_decoders_setup = mock_cxl_switch_port_decoders_setup, .devm_cxl_endpoint_decoders_setup = mock_cxl_endpoint_decoders_setup, .cxl_endpoint_parse_cdat = mock_cxl_endpoint_parse_cdat, - .devm_cxl_add_dport_by_dev = mock_cxl_add_dport_by_dev, .hmat_get_extended_linear_cache_size = mock_hmat_get_extended_linear_cache_size, + .cxl_add_dport_by_dev = mock_cxl_add_dport_by_dev, .list = LIST_HEAD_INIT(cxl_mock_ops.list), };tools/testing/cxl/test/mock.c +12
-13@@ -15,14 +15,13 @@static LIST_HEAD (mock);static struct cxl_dport * -redirect_devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev); +redirect_cxl_add_dport_by_dev (struct cxl_port *port,struct device *dport_dev);static int redirect_devm_cxl_switch_port_decoders_setup (struct cxl_port *port);void register_cxl_mock_ops (struct cxl_mock_ops *ops) {list_add_rcu (&ops->list, &mock); - _devm_cxl_add_dport_by_dev = redirect_devm_cxl_add_dport_by_dev; + _cxl_add_dport_by_dev = redirect_cxl_add_dport_by_dev; _devm_cxl_switch_port_decoders_setup = redirect_devm_cxl_switch_port_decoders_setup; }@@ -34,7 +33,7 @@ void unregister_cxl_mock_ops(struct cxl_mock_ops *ops) { _devm_cxl_switch_port_decoders_setup = __devm_cxl_switch_port_decoders_setup; - _devm_cxl_add_dport_by_dev = __devm_cxl_add_dport_by_dev; + _cxl_add_dport_by_dev = __cxl_add_dport_by_dev;list_del_rcu (&ops->list);synchronize_srcu (&cxl_mock_srcu); }@@ -207,7 +206,7 @@ int __wrap_cxl_await_media_ready(struct cxl_dev_state *cxlds) }EXPORT_SYMBOL_NS_GPL (__wrap_cxl_await_media_ready,"CXL" ); -struct cxl_dport *__wrap_devm_cxl_add_rch_dport (struct cxl_port *port, +struct cxl_dport *__wrap_cxl_add_rch_dport (struct cxl_port *port,struct device *dport_dev,int port_id,resource_size_t rcrb)@@ -217,19 +216,19 @@ struct cxl_dport *__wrap_devm_cxl_add_rch_dport(struct cxl_port *port,struct cxl_mock_ops *ops =get_cxl_mock_ops (&index);if (ops && ops->is_mock_port (dport_dev)) { - dport =devm_cxl_add_dport (port, dport_dev, port_id, -CXL_RESOURCE_NONE ); + dport =cxl_add_dport (port, dport_dev, port_id, +CXL_RESOURCE_NONE );if (!IS_ERR (dport)) { dport->rcrb.base = rcrb; dport->rch = true; } }else - dport =devm_cxl_add_rch_dport (port, dport_dev, port_id, rcrb); + dport =cxl_add_rch_dport (port, dport_dev, port_id, rcrb);put_cxl_mock_ops (index);return dport; } -EXPORT_SYMBOL_NS_GPL (__wrap_devm_cxl_add_rch_dport,"CXL" ); +EXPORT_SYMBOL_NS_GPL (__wrap_cxl_add_rch_dport,"CXL" );void __wrap_cxl_endpoint_parse_cdat (struct cxl_port *port) {@@ -257,17 +256,17 @@ void __wrap_cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device }EXPORT_SYMBOL_NS_GPL (__wrap_cxl_dport_init_ras_reporting,"CXL" ); -struct cxl_dport *redirect_devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev) +struct cxl_dport *redirect_cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev) {int index;struct cxl_mock_ops *ops =get_cxl_mock_ops (&index);struct cxl_dport *dport;if (ops && ops->is_mock_port (port->uport_dev)) - dport = ops->devm_cxl_add_dport_by_dev (port, dport_dev); + dport = ops->cxl_add_dport_by_dev (port, dport_dev);else - dport =__devm_cxl_add_dport_by_dev (port, dport_dev); + dport =__cxl_add_dport_by_dev (port, dport_dev);put_cxl_mock_ops (index);return dport;tools/testing/cxl/test/mock.h +2
-2@@ -22,8 +22,8 @@ struct cxl_mock_ops {int (*devm_cxl_switch_port_decoders_setup)(struct cxl_port *port);int (*devm_cxl_endpoint_decoders_setup)(struct cxl_port *port);void (*cxl_endpoint_parse_cdat)(struct cxl_port *port); -struct cxl_dport *(*devm_cxl_add_dport_by_dev)(struct cxl_port *port, -struct device *dport_dev); +struct cxl_dport *(*cxl_add_dport_by_dev)(struct cxl_port *port, +struct device *dport_dev);int (*hmat_get_extended_linear_cache_size)(struct resource *backing_res,int nid,resource_size_t *cache_size);
Re: [PATCH v14 19/34] cxl/port: Fix devm resource leaks around with dport management
In reply to Terry BowmanQuoted 20 lines · On Wed, 14 Jan 2026 12:20:40 -0600 Terry Bowman <[email protected]> wrote:
From: Dan Williams <[email protected]>
With dport addition moving out of
cxl_switch_port_probe()it is no longer the case that a single dport-add failure will cause all dport resources to be automatically unwound.devm still helps all dport resources get cleaned up when the port is detached, but setup now needs to avoid leaking resources if an early exit occurs during setup.
Convert from a "devm add" model, to an "auto remove" model that makes the caller responsible for registering devm reclaim after the object is fully instantiated.
As a side of effect of this reorganization
port->nr_dportsis now always consistent with the number of entries in theport->dportsxarray, and this can stop playing games withida_is_empty()which is unreliable as a detector of whether decoders are setup. I.e. consider howCONFIG_DEBUG_KOBJECT_RELEASEmight wreak havoc with this approach.
Quoted 7 lines
Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>---
Changes in v13 -> v14:
- New patch
Hi Dan, Terry,
I think this needs a little reorganization to ensure we don't have dport and dport_add both being the same pointer for different free reasons. Adding a helper and we can combine them with a clear hand over of ownership.
Wrapping devres_remove_group() in a function that is called close_group() rings alarm bells.
Jonathan
Quoted 41 lines
drivers/cxl/core/port.c +6
-18-static struct cxl_dport * -__devm_cxl_add_dport (struct cxl_port *port,struct device *dport_dev, -int port_id,resource_size_t component_reg_phys, -resource_size_t rcrb) +static struct cxl_dport *__cxl_add_dport (struct cxl_port *port, +struct device *dport_dev,int port_id, +resource_size_t component_reg_phys, +resource_size_t rcrb) {char link_name[CXL_TARGET_STRLEN ]; -struct cxl_dport *dport; -struct device *host;int rc; -if (is_cxl_root (port)) - host = port->uport_dev; -else - host = &port->dev; - -if (!host->driver) { -dev_WARN_ONCE (&port->dev,1 ,"dport:%s bad devm context\n" , -dev_name (dport_dev)); -return ERR_PTR (-ENXIO ); - } -if (snprintf (link_name,CXL_TARGET_STRLEN ,"dport%d" , port_id) >=CXL_TARGET_STRLEN )return ERR_PTR (-EINVAL ); - dport =devm_kzalloc (host,sizeof (*dport),GFP_KERNEL ); + struct cxl_dport *dport __free(kfree) = +kzalloc (sizeof (*dport),GFP_KERNEL );if (!dport)return ERR_PTR (-ENOMEM );
drivers/cxl/core/port.c __devm_cxl_add_dport(struct cxl_port *port, struct device *dport_dev, · line 1179
Quoted 6 lines
drivers/cxl/core/port.c +2
-1@@ -1176,48 +1175,27 @@ __devm_cxl_add_dport(struct cxl_port *port, struct device *dport_dev, &component_reg_phys);cond_cxl_root_lock (port); - rc =add_dport (port, dport); + struct cxl_dport *dport_add __free(remove_dport) = +add_dport (port, dport);
This pattern of having both dport and dport_add effectively pointing to the same pointer concerns me from a readability / maintainability point of view. We've often made use of helper functions to avoid doing this and I think that would make sense here as well.
Take everything down to and including dport_add() as a helper called something like (naming needs work!)
struct dport_dev *dport __free(remove_and_free_dport) = add_dport_wrapper();
With the __free doing the kfree as well as remove.
Quoted 69 lines
drivers/cxl/core/port.c +13
-32cond_cxl_root_unlock (port); -if (rc) -return ERR_PTR (rc); - -/* -* Setup port register if this is the first dport showed up. Having -* a dport also means that there is at least 1 active link. -*/ -if (port->nr_dports ==1 && - port->component_reg_phys !=CXL_RESOURCE_NONE ) { - rc =cxl_port_setup_regs (port, port->component_reg_phys); -if (rc) { -xa_erase (&port->dports, (unsigned long )dport->dport_dev); -return ERR_PTR (rc); - } - port->component_reg_phys =CXL_RESOURCE_NONE ; - } +if (IS_ERR (dport_add)) +return dport_add; -get_device (dport_dev); - rc =devm_add_action_or_reset (host, cxl_dport_remove, dport); -if (rc) -return ERR_PTR (rc); +if (dev_is_pci (dport_dev)) + dport->link_latency =cxl_pci_get_latency (to_pci_dev (dport_dev)); rc =sysfs_create_link (&port->dev.kobj, &dport_dev->kobj, link_name);if (rc)return ERR_PTR (rc); - rc =devm_add_action_or_reset (host, cxl_dport_unlink, dport); -if (rc) -return ERR_PTR (rc); - -if (dev_is_pci (dport_dev)) - dport->link_latency =cxl_pci_get_latency (to_pci_dev (dport_dev)); -cxl_debugfs_create_dport_dir (dport); -return dport; +retain_and_null_ptr (dport_add); +return no_free_ptr (dport); } + +/* +* Note: this only services dynamic removal of mid-level ports, root ports are +* always removed by the platform driver (e.g. cxl_acpi). @host can be +* hard-coded to &port->dev. +*/ static void del_dport (struct cxl_dport *dport) {struct cxl_port *port = dport->port; -devm_release_action (&port->dev, cxl_dport_unlink, dport); -devm_release_action (&port->dev, cxl_dport_remove, dport); -devm_kfree (&port->dev, dport); +devm_release_action (&port->dev, unlink_dport, dport); }static void del_dports (struct cxl_port *port)@@ -1597,10 +1603,24 @@ static int update_decoder_targets(struct device *dev, void *data)return 0 ; } -DEFINE_FREE(del_cxl_dport, struct cxl_dport *, if (!IS_ERR_OR_NULL(_T)) del_dport(_T))
drivers/cxl/core/port.c static int update_decoder_targets(struct device *dev, void *data) · line 1615
Quoted 10 lines
drivers/cxl/core/port.c +10
-0+static struct cxl_port *cxl_port_devres_group (struct cxl_port *port) +{ +if (!devres_open_group (&port->dev, port,GFP_KERNEL )) +return ERR_PTR (-ENOMEM ); +return port; +} +DEFINE_FREE(cxl_port_group_free, struct cxl_port *, + if (!IS_ERR_OR_NULL(_T)) devres_release_group(&(_T)->dev, _T)) + +static void cxl_port_group_close (struct cxl_port *port)
This feels like misleading naming and I'm not sure what intent is. Would have expected it to call devres_close_group()
Quoted 47 lines
drivers/cxl/core/port.c +21
-10+{ +devres_remove_group (&port->dev, port); +} +static struct cxl_dport *cxl_port_add_dport (struct cxl_port *port,struct device *dport_dev) { +struct cxl_dport *new_dport;struct cxl_dport *dport;int rc;@@ -1615,29 +1635,46 @@ static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port,return ERR_PTR (-EBUSY ); } - struct cxl_dport *new_dport __free(del_cxl_dport) = -devm_cxl_add_dport_by_dev (port, dport_dev); -if (IS_ERR (new_dport)) -return new_dport; - -cxl_switch_parse_cdat (new_dport); +/* +* With the first dport arrival it is now safe to start looking at +* component registers. Be careful to not strand resources if dport +* creation ultimately fails. +*/ + struct cxl_port *port_group __free(cxl_port_group_free) = +cxl_port_devres_group (port); +if (IS_ERR (port_group)) +return ERR_CAST (port_group); -if (ida_is_empty (&port->decoder_ida)) { +if (port->nr_dports ==0 ) { rc =devm_cxl_switch_port_decoders_setup (port);if (rc)return ERR_PTR (rc); -dev_dbg (&port->dev,"first dport%d:%s added with decoders\n" , - new_dport->port_id,dev_name (dport_dev)); -return no_free_ptr (new_dport); +/* +* Note, when nr_dports returns to zero the port is unregistered +* and triggers cleanup. I.e. no need for open-coded release +* action on dport removal. See cxl_detach_ep() for that logic. +*/ } + new_dport =cxl_add_dport_by_dev (port, dport_dev);
drivers/cxl/core/port.c static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port, · line 1669
Quoted 10 lines
drivers/cxl/core/port.c +10
-0+if (IS_ERR (new_dport)) +return new_dport; + + rc =cxl_dport_autoremove (new_dport); +if (rc) +return ERR_PTR (rc); + +cxl_switch_parse_cdat (new_dport); + +cxl_port_group_close (no_free_ptr (port_group));
Give name vs what it does I'm not sure how this currently works.
Quoted 15 lines
drivers/cxl/core/port.c +5
-4+ +dev_dbg (&port->dev,"dport[%d] id:%d dport_dev: %s added\n" , + port->nr_dports -1 , new_dport->port_id,dev_name (dport_dev)); +/* New dport added, update the decoder targets */ device_for_each_child (&port->dev, new_dport, update_decoder_targets); -dev_dbg (&port->dev,"dport%d:%s added\n" , new_dport->port_id, -dev_name (dport_dev)); - -return no_free_ptr (new_dport); +return new_dport; } static struct cxl_dport *devm_cxl_create_port(struct device *ep_dev,
Re: [PATCH v14 19/34] cxl/port: Fix devm resource leaks around with dport management
Quoted 41 lines · Jonathan Cameron
Quoted 20 lines · On Wed, 14 Jan 2026 12:20:40 -0600 Terry Bowman <[email protected]> wrote:
From: Dan Williams <[email protected]>
With dport addition moving out of
cxl_switch_port_probe()it is no longer the case that a single dport-add failure will cause all dport resources to be automatically unwound.devm still helps all dport resources get cleaned up when the port is detached, but setup now needs to avoid leaking resources if an early exit occurs during setup.
Convert from a "devm add" model, to an "auto remove" model that makes the caller responsible for registering devm reclaim after the object is fully instantiated.
As a side of effect of this reorganization
port->nr_dportsis now always consistent with the number of entries in theport->dportsxarray, and this can stop playing games withida_is_empty()which is unreliable as a detector of whether decoders are setup. I.e. consider howCONFIG_DEBUG_KOBJECT_RELEASEmight wreak havoc with this approach.Quoted 7 lines
Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>---
Changes in v13 -> v14:
- New patchHi Dan, Terry,
I think this needs a little reorganization to ensure we don't have dport and
dport_addboth being the same pointer for different free reasons. Adding a helper and we can combine them with a clear hand over of ownership.Wrapping
devres_remove_group()in a function that is calledclose_group()rings alarm bells.Jonathan
[..]
Quoted 4 lines
Quoted 4 lines
(quoted diff) +0
-0@@ -1176,48 +1175,27 @@ __devm_cxl_add_dport(struct cxl_port *port, struct device *dport_dev, &component_reg_phys);
Quoted 9 lines
Review __devm_cxl_add_dport(struct cxl_port *port, struct device *dport_dev, · line 1179
Quoted 4 lines
(quoted diff) +2
-1cond_cxl_root_lock(port); - rc = add_dport(port, dport); + struct cxl_dport *dport_add __free(remove_dport) = + add_dport(port, dport);This pattern of having both dport and
dport_addeffectively pointing to the same pointer concerns me from a readability / maintainability point of view. We've often made use of helper functions to avoid doing this and I think that would make sense here as well.
Yeah, while I do think the multi-variable pattern is useful for many-step object construction, I can usually easily be persuaded to consider a helper function.
Quoted 4 lines
Take everything down to and including
dport_add()as a helper called something like (naming needs work!)struct dport_dev *dport __free(remove_and_free_dport) = add_dport_wrapper();
I ended up with the patch below which is similar in spirit to this without a new DEFINE_FREE().
Quoted 69 lines
Quoted 69 lines
(quoted diff) +13
-31cond_cxl_root_unlock(port); - if (rc) - return ERR_PTR(rc); - - /* - * Setup port register if this is the first dport showed up. Having - * a dport also means that there is at least 1 active link. - */ - if (port->nr_dports == 1 && - port->component_reg_phys != CXL_RESOURCE_NONE) { - rc = cxl_port_setup_regs(port, port->component_reg_phys); - if (rc) { - xa_erase(&port->dports, (unsigned long)dport->dport_dev); - return ERR_PTR(rc); - } - port->component_reg_phys = CXL_RESOURCE_NONE; - } + if (IS_ERR(dport_add)) + return dport_add; - get_device(dport_dev); - rc = devm_add_action_or_reset(host, cxl_dport_remove, dport); - if (rc) - return ERR_PTR(rc); + if (dev_is_pci(dport_dev)) + dport->link_latency = cxl_pci_get_latency(to_pci_dev(dport_dev)); rc = sysfs_create_link(&port->dev.kobj, &dport_dev->kobj, link_name); if (rc) return ERR_PTR(rc); - rc = devm_add_action_or_reset(host, cxl_dport_unlink, dport); - if (rc) - return ERR_PTR(rc); - - if (dev_is_pci(dport_dev)) - dport->link_latency = cxl_pci_get_latency(to_pci_dev(dport_dev)); - cxl_debugfs_create_dport_dir(dport); - return dport; + retain_and_null_ptr(dport_add); + return no_free_ptr(dport); } + +/* + * Note: this only services dynamic removal of mid-level ports, root ports are + * always removed by the platform driver (e.g. cxl_acpi). @host can be + * hard-coded to &port->dev. + */ static void del_dport(struct cxl_dport *dport) { struct cxl_port *port = dport->port; - devm_release_action(&port->dev, cxl_dport_unlink, dport); - devm_release_action(&port->dev, cxl_dport_remove, dport); - devm_kfree(&port->dev, dport); + devm_release_action(&port->dev, unlink_dport, dport); } static void del_dports(struct cxl_port *port)@@ -1597,10 +1603,24 @@ static int update_decoder_targets(struct device *dev, void *data) return 0; }
Quoted 14 lines
Review static int update_decoder_targets(struct device *dev, void *data) · line 1615
Quoted 11 lines
(quoted diff) +10
-1-DEFINE_FREE(del_cxl_dport, struct cxl_dport *, if (!IS_ERR_OR_NULL(_T)) del_dport(_T)) +static struct cxl_port *cxl_port_devres_group(struct cxl_port *port) +{ + if (!devres_open_group(&port->dev, port, GFP_KERNEL)) + return ERR_PTR(-ENOMEM); + return port; +} +DEFINE_FREE(cxl_port_group_free, struct cxl_port *, + if (!IS_ERR_OR_NULL(_T)) devres_release_group(&(_T)->dev, _T)) + +static void cxl_port_group_close(struct cxl_port *port)This feels like misleading naming and I'm not sure what intent is. Would have expected it to call
devres_close_group()
Agree. The hastiness of this patch shows. Switched all the naming to not be surprising. The flow is:
cxl_port_open_group(): start recording devres resource acquisition cxl_port_remove_group(): on success, stop tracking the group, leave the resources cxl_port_release_group(): on failure, destroy the group, free the resources
New patch, added a Fixes: tag.
-- 8< --
From 9731bb6cb5638a0d2141dc072f90db0d00400680 Mon Sep 17 00:00:00 2001
From: Dan Williams <[email protected]>
Date: Wed, 14 Jan 2026 12:20:40 -0600
Subject: [PATCH] cxl/port: Fix devm resource leaks with dport management
With dport addition moving out of cxl_switch_port_probe() it is no longer the case that a single dport-add failure will cause all dport resources to be automatically unwound.
devm still helps all dport resources get cleaned up when the port is detached, but setup now needs to avoid leaking resources if an early exit occurs during setup.
Convert from a "devm add" model, to an "auto remove" model that makes the caller responsible for registering devm reclaim after the object is fully instantiated.
As a side of effect of this reorganization port->nr_dports is now always consistent with the number of entries in the port->dports xarray, and this can stop playing games with ida_is_empty() which is unreliable as a detector of whether decoders are setup. I.e. consider how CONFIG_DEBUG_KOBJECT_RELEASE might wreak havoc with this approach.
--- drivers/cxl/cxl.h | 23 +-- tools/testing/cxl/exports.h | 4 +- tools/testing/cxl/test/mock.h | 4 +- drivers/cxl/acpi.c | 11 +- drivers/cxl/core/pci.c | 10 +- drivers/cxl/core/port.c | 252 ++++++++++++++++----------- drivers/cxl/port.c | 8 +- tools/testing/cxl/cxl_core_exports.c | 13 +- tools/testing/cxl/test/cxl.c | 6 +- tools/testing/cxl/test/mock.c | 25 ++- tools/testing/cxl/Kbuild | 3 +- 11 files changed, 209 insertions(+), 150 deletions(-)
drivers/cxl/cxl.h +12 -11
@@ -796,12 +796,12 @@ struct cxl_port *cxl_mem_find_port(struct cxl_memdev *cxlmd, struct cxl_dport **dport);bool schedule_cxl_memdev_detach (struct cxl_memdev *cxlmd); -struct cxl_dport *devm_cxl_add_dport (struct cxl_port *port, -struct device *dport,int port_id, -resource_size_t component_reg_phys); -struct cxl_dport *devm_cxl_add_rch_dport (struct cxl_port *port, -struct device *dport_dev,int port_id, -resource_size_t rcrb); +struct cxl_dport *cxl_add_dport (struct cxl_port *port,struct device *dport, +int port_id, +resource_size_t component_reg_phys); +struct cxl_dport *cxl_add_rch_dport (struct cxl_port *port, +struct device *dport_dev,int port_id, +resource_size_t rcrb);struct cxl_decoder *to_cxl_decoder (struct device *dev);struct cxl_root_decoder *to_cxl_root_decoder (struct device *dev);
@@ -824,6 +824,7 @@ static inline int cxl_root_decoder_autoremove(struct device *host,return cxl_decoder_autoremove (host, &cxlrd->cxlsd.cxld); }int cxl_endpoint_autoremove (struct cxl_memdev *cxlmd,struct cxl_port *endpoint); +int cxl_dport_autoremove (struct cxl_dport *dport);/** * struct cxl_endpoint_dvsec_info - Cached DVSEC info
@@ -937,10 +938,10 @@ void cxl_coordinates_combine(struct access_coordinate *out, struct access_coordinate *c2);bool cxl_endpoint_decoder_reset_detected (struct cxl_port *port); -struct cxl_dport *devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev); -struct cxl_dport *__devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev); +struct cxl_dport *cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev); +struct cxl_dport *__cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev);/* * Unit test builds overrides this to __weak, find the 'strong' version
@@ -964,7 +965,7 @@ u16 cxl_gpf_get_dvsec(struct device *dev);*/ #ifndef CXL_TEST_ENABLE #define DECLARE_TESTABLE (x) __##x -#define devm_cxl_add_dport_by_dev DECLARE_TESTABLE(devm_cxl_add_dport_by_dev) +#define cxl_add_dport_by_dev DECLARE_TESTABLE(cxl_add_dport_by_dev)#define devm_cxl_switch_port_decoders_setup DECLARE_TESTABLE(devm_cxl_switch_port_decoders_setup)#endif
tools/testing/cxl/exports.h +2 -2
@@ -4,8 +4,8 @@#define __MOCK_CXL_EXPORTS_H_typedef struct cxl_dport *(*cxl_add_dport_by_dev_fn )(struct cxl_port *port, -struct device *dport_dev); -extern cxl_add_dport_by_dev_fn _devm_cxl_add_dport_by_dev; +struct device *dport_dev); +extern cxl_add_dport_by_dev_fn _cxl_add_dport_by_dev;typedef int (*cxl_switch_decoders_setup_fn )(struct cxl_port *port);extern cxl_switch_decoders_setup_fn _devm_cxl_switch_port_decoders_setup;
tools/testing/cxl/test/mock.h +2 -2
@@ -22,8 +22,8 @@ struct cxl_mock_ops {int (*devm_cxl_switch_port_decoders_setup)(struct cxl_port *port);int (*devm_cxl_endpoint_decoders_setup)(struct cxl_port *port);void (*cxl_endpoint_parse_cdat)(struct cxl_port *port); -struct cxl_dport *(*devm_cxl_add_dport_by_dev)(struct cxl_port *port, -struct device *dport_dev); +struct cxl_dport *(*cxl_add_dport_by_dev)(struct cxl_port *port, +struct device *dport_dev);int (*hmat_get_extended_linear_cache_size)(struct resource *backing_res,int nid,resource_size_t *cache_size);
drivers/cxl/acpi.c +7 -4
@@ -679,16 +679,19 @@ static int add_host_bridge_dport(struct device *match, void *arg)if (ctx.cxl_version ==ACPI_CEDT_CHBS_VERSION_CXL11 ) {dev_dbg (match,"RCRB found for UID %lld: %pa\n" , ctx.uid, &ctx.base); - dport =devm_cxl_add_rch_dport (root_port, bridge, ctx.uid, - ctx.base); + dport =cxl_add_rch_dport (root_port, bridge, ctx.uid, ctx.base); }else { - dport =devm_cxl_add_dport (root_port, bridge, ctx.uid, -CXL_RESOURCE_NONE ); + dport =cxl_add_dport (root_port, bridge, ctx.uid, +CXL_RESOURCE_NONE ); }if (IS_ERR (dport))return PTR_ERR (dport); + ret =cxl_dport_autoremove (dport); +if (ret) +return ret; + ret =get_genport_coordinates (match, dport);if (ret)dev_dbg (match,"Failed to get generic port perf coordinates.\n" );
drivers/cxl/core/pci.c +5 -5
@@ -41,14 +41,14 @@ static int pci_get_port_num(struct pci_dev *pdev) }/** -* __devm_cxl_add_dport_by_dev - allocate a dport by dport device +* __cxl_add_dport_by_dev - allocate a dport by dport device * @port: cxl_port that hosts the dport * @dport_dev: 'struct device' of the dport * * Returns the allocated dport on success or ERR_PTR() of -errno on error */ -struct cxl_dport *__devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev) +struct cxl_dport *__cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev) {struct cxl_register_map map;struct pci_dev *pdev;
@@ -67,9 +67,9 @@ struct cxl_dport *__devm_cxl_add_dport_by_dev(struct cxl_port *port,return ERR_PTR (rc);device_lock_assert (&port->dev); -return devm_cxl_add_dport (port, dport_dev, port_num, map.resource); +return cxl_add_dport (port, dport_dev, port_num, map.resource); } -EXPORT_SYMBOL_NS_GPL (__devm_cxl_add_dport_by_dev,"CXL" ); +EXPORT_SYMBOL_NS_GPL (__cxl_add_dport_by_dev,"CXL" );static int cxl_dvsec_mem_range_valid (struct cxl_dev_state *cxlds,int id) {
drivers/cxl/core/port.c +152 -100
363 lines not on this page · raw message
drivers/cxl/port.c +6 -2
@@ -59,8 +59,12 @@ static int discover_region(struct device *dev, void *unused)static int cxl_switch_port_probe (struct cxl_port *port) { -/* Reset nr_dports for rebind of driver */ - port->nr_dports =0 ; +/* +* Unfortunately, typical driver operations like "find and map +* registers", can not be done at port device attach time and must wait +* for dport arrival. See cxl_port_add_dport() and the comments in +* add_dport() for details. +*/ /* Cache the data early to ensure is_visible() works */ read_cdat_data (port);
tools/testing/cxl/cxl_core_exports.c +6 -7
@@ -7,16 +7,15 @@/* Exporting of cxl_core symbols that are only used by cxl_test */ EXPORT_SYMBOL_NS_GPL (cxl_num_decoders_committed,"CXL" ); -cxl_add_dport_by_dev_fn _devm_cxl_add_dport_by_dev = - __devm_cxl_add_dport_by_dev; -EXPORT_SYMBOL_NS_GPL (_devm_cxl_add_dport_by_dev,"CXL" ); +cxl_add_dport_by_dev_fn _cxl_add_dport_by_dev = __cxl_add_dport_by_dev; +EXPORT_SYMBOL_NS_GPL (_cxl_add_dport_by_dev,"CXL" ); -struct cxl_dport *devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev) +struct cxl_dport *cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev) { -return _devm_cxl_add_dport_by_dev (port, dport_dev); +return _cxl_add_dport_by_dev (port, dport_dev); } -EXPORT_SYMBOL_NS_GPL (devm_cxl_add_dport_by_dev,"CXL" ); +EXPORT_SYMBOL_NS_GPL (cxl_add_dport_by_dev,"CXL" );cxl_switch_decoders_setup_fn _devm_cxl_switch_port_decoders_setup = __devm_cxl_switch_port_decoders_setup;
tools/testing/cxl/test/cxl.c +3 -3
@@ -1060,8 +1060,8 @@ static struct cxl_dport *mock_cxl_add_dport_by_dev(struct cxl_port *port,if (&pdev->dev != dport_dev)continue ; -return devm_cxl_add_dport (port, &pdev->dev, pdev->id, -CXL_RESOURCE_NONE ); +return cxl_add_dport (port, &pdev->dev, pdev->id, +CXL_RESOURCE_NONE ); }return ERR_PTR (-ENODEV );
@@ -1126,9 +1126,9 @@ static struct cxl_mock_ops cxl_mock_ops = { .devm_cxl_switch_port_decoders_setup = mock_cxl_switch_port_decoders_setup, .devm_cxl_endpoint_decoders_setup = mock_cxl_endpoint_decoders_setup, .cxl_endpoint_parse_cdat = mock_cxl_endpoint_parse_cdat, - .devm_cxl_add_dport_by_dev = mock_cxl_add_dport_by_dev, .hmat_get_extended_linear_cache_size = mock_hmat_get_extended_linear_cache_size, + .cxl_add_dport_by_dev = mock_cxl_add_dport_by_dev, .list = LIST_HEAD_INIT(cxl_mock_ops.list), };
tools/testing/cxl/test/mock.c +12 -13
@@ -15,14 +15,13 @@static LIST_HEAD (mock);static struct cxl_dport * -redirect_devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev); +redirect_cxl_add_dport_by_dev (struct cxl_port *port,struct device *dport_dev);static int redirect_devm_cxl_switch_port_decoders_setup (struct cxl_port *port);void register_cxl_mock_ops (struct cxl_mock_ops *ops) {list_add_rcu (&ops->list, &mock); - _devm_cxl_add_dport_by_dev = redirect_devm_cxl_add_dport_by_dev; + _cxl_add_dport_by_dev = redirect_cxl_add_dport_by_dev; _devm_cxl_switch_port_decoders_setup = redirect_devm_cxl_switch_port_decoders_setup; }
@@ -34,7 +33,7 @@ void unregister_cxl_mock_ops(struct cxl_mock_ops *ops) { _devm_cxl_switch_port_decoders_setup = __devm_cxl_switch_port_decoders_setup; - _devm_cxl_add_dport_by_dev = __devm_cxl_add_dport_by_dev; + _cxl_add_dport_by_dev = __cxl_add_dport_by_dev;list_del_rcu (&ops->list);synchronize_srcu (&cxl_mock_srcu); }
@@ -207,7 +206,7 @@ int __wrap_cxl_await_media_ready(struct cxl_dev_state *cxlds) }EXPORT_SYMBOL_NS_GPL (__wrap_cxl_await_media_ready,"CXL" ); -struct cxl_dport *__wrap_devm_cxl_add_rch_dport (struct cxl_port *port, +struct cxl_dport *__wrap_cxl_add_rch_dport (struct cxl_port *port,struct device *dport_dev,int port_id,resource_size_t rcrb)
@@ -217,19 +216,19 @@ struct cxl_dport *__wrap_devm_cxl_add_rch_dport(struct cxl_port *port,struct cxl_mock_ops *ops =get_cxl_mock_ops (&index);if (ops && ops->is_mock_port (dport_dev)) { - dport =devm_cxl_add_dport (port, dport_dev, port_id, -CXL_RESOURCE_NONE ); + dport =cxl_add_dport (port, dport_dev, port_id, +CXL_RESOURCE_NONE );if (!IS_ERR (dport)) { dport->rcrb.base = rcrb; dport->rch = true; } }else - dport =devm_cxl_add_rch_dport (port, dport_dev, port_id, rcrb); + dport =cxl_add_rch_dport (port, dport_dev, port_id, rcrb);put_cxl_mock_ops (index);return dport; } -EXPORT_SYMBOL_NS_GPL (__wrap_devm_cxl_add_rch_dport,"CXL" ); +EXPORT_SYMBOL_NS_GPL (__wrap_cxl_add_rch_dport,"CXL" );void __wrap_cxl_endpoint_parse_cdat (struct cxl_port *port) {
@@ -257,17 +256,17 @@ void __wrap_cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device }EXPORT_SYMBOL_NS_GPL (__wrap_cxl_dport_init_ras_reporting,"CXL" ); -struct cxl_dport *redirect_devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev) +struct cxl_dport *redirect_cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev) {int index;struct cxl_mock_ops *ops =get_cxl_mock_ops (&index);struct cxl_dport *dport;if (ops && ops->is_mock_port (port->uport_dev)) - dport = ops->devm_cxl_add_dport_by_dev (port, dport_dev); + dport = ops->cxl_add_dport_by_dev (port, dport_dev);else - dport =__devm_cxl_add_dport_by_dev (port, dport_dev); + dport =__cxl_add_dport_by_dev (port, dport_dev);put_cxl_mock_ops (index);return dport;
tools/testing/cxl/Kbuild +2 -1
@@ -5,7 +5,8 @@ ldflags-y += --wrap=acpi_evaluate_integer ldflags-y += --wrap=acpi_pci_find_root ldflags-y += --wrap=nvdimm_bus_register ldflags-y += --wrap=cxl_await_media_ready -ldflags-y += --wrap=devm_cxl_add_rch_dport +ldflags-y += --wrap=cxl_add_rch_dport +ldflags-y += --wrap=cxl_rcd_component_reg_phys ldflags-y += --wrap=cxl_endpoint_parse_cdat ldflags-y += --wrap=cxl_dport_init_ras_reporting ldflags-y += --wrap=devm_cxl_endpoint_decoders_setup
Signature · 1 line
2.52.0
Re: [PATCH v14 19/34] cxl/port: Fix devm resource leaks around with dport management
Quoted 154 lines · On Thu, 15 Jan 2026 20:45:20 -0800 [email protected] wrote:
Quoted 41 lines · Jonathan Cameron
Quoted 20 lines · On Wed, 14 Jan 2026 12:20:40 -0600 Terry Bowman <[email protected]> wrote:
From: Dan Williams <[email protected]>
With dport addition moving out of
cxl_switch_port_probe()it is no longer the case that a single dport-add failure will cause all dport resources to be automatically unwound.devm still helps all dport resources get cleaned up when the port is detached, but setup now needs to avoid leaking resources if an early exit occurs during setup.
Convert from a "devm add" model, to an "auto remove" model that makes the caller responsible for registering devm reclaim after the object is fully instantiated.
As a side of effect of this reorganization
port->nr_dportsis now always consistent with the number of entries in theport->dportsxarray, and this can stop playing games withida_is_empty()which is unreliable as a detector of whether decoders are setup. I.e. consider howCONFIG_DEBUG_KOBJECT_RELEASEmight wreak havoc with this approach.Quoted 7 lines
Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>---
Changes in v13 -> v14:
- New patchHi Dan, Terry,
I think this needs a little reorganization to ensure we don't have dport and
dport_addboth being the same pointer for different free reasons. Adding a helper and we can combine them with a clear hand over of ownership.Wrapping
devres_remove_group()in a function that is calledclose_group()rings alarm bells.Jonathan
[..]
Quoted 4 lines
Quoted 4 lines
drivers/cxl/core/port.c +0
-0@@ -1176,48 +1175,27 @@ __devm_cxl_add_dport(struct cxl_port *port, struct device *dport_dev, &component_reg_phys);Quoted 9 lines
Review __devm_cxl_add_dport(struct cxl_port *port, struct device *dport_dev, · line 1179
Quoted 4 lines
(quoted diff) +2
-1cond_cxl_root_lock(port); - rc = add_dport(port, dport); + struct cxl_dport *dport_add __free(remove_dport) = + add_dport(port, dport);This pattern of having both dport and
dport_addeffectively pointing to the same pointer concerns me from a readability / maintainability point of view. We've often made use of helper functions to avoid doing this and I think that would make sense here as well.Yeah, while I do think the multi-variable pattern is useful for many-step object construction, I can usually easily be persuaded to consider a helper function.
Quoted 4 lines
Take everything down to and including
dport_add()as a helper called something like (naming needs work!)struct dport_dev *dport __free(remove_and_free_dport) = add_dport_wrapper();I ended up with the patch below which is similar in spirit to this without a new
DEFINE_FREE().Quoted 69 lines
Quoted 69 lines
drivers/cxl/core/port.c +13
-31cond_cxl_root_unlock (port); -if (rc) -return ERR_PTR (rc); - -/* -* Setup port register if this is the first dport showed up. Having -* a dport also means that there is at least 1 active link. -*/ -if (port->nr_dports ==1 && - port->component_reg_phys !=CXL_RESOURCE_NONE ) { - rc =cxl_port_setup_regs (port, port->component_reg_phys); -if (rc) { -xa_erase (&port->dports, (unsigned long )dport->dport_dev); -return ERR_PTR (rc); - } - port->component_reg_phys =CXL_RESOURCE_NONE ; - } +if (IS_ERR (dport_add)) +return dport_add; -get_device (dport_dev); - rc =devm_add_action_or_reset (host, cxl_dport_remove, dport); -if (rc) -return ERR_PTR (rc); +if (dev_is_pci (dport_dev)) + dport->link_latency =cxl_pci_get_latency (to_pci_dev (dport_dev)); rc =sysfs_create_link (&port->dev.kobj, &dport_dev->kobj, link_name);if (rc)return ERR_PTR (rc); - rc =devm_add_action_or_reset (host, cxl_dport_unlink, dport); -if (rc) -return ERR_PTR (rc); - -if (dev_is_pci (dport_dev)) - dport->link_latency =cxl_pci_get_latency (to_pci_dev (dport_dev)); -cxl_debugfs_create_dport_dir (dport); -return dport; +retain_and_null_ptr (dport_add); +return no_free_ptr (dport); } + +/* +* Note: this only services dynamic removal of mid-level ports, root ports are +* always removed by the platform driver (e.g. cxl_acpi). @host can be +* hard-coded to &port->dev. +*/ static void del_dport (struct cxl_dport *dport) {struct cxl_port *port = dport->port; -devm_release_action (&port->dev, cxl_dport_unlink, dport); -devm_release_action (&port->dev, cxl_dport_remove, dport); -devm_kfree (&port->dev, dport); +devm_release_action (&port->dev, unlink_dport, dport); }static void del_dports (struct cxl_port *port)@@ -1597,10 +1603,24 @@ static int update_decoder_targets(struct device *dev, void *data)return 0 ; }Quoted 14 lines
drivers/cxl/core/port.c static int update_decoder_targets(struct device *dev, void *data)
Quoted 11 lines
drivers/cxl/core/port.c +10
-1-DEFINE_FREE(del_cxl_dport, struct cxl_dport *, if (!IS_ERR_OR_NULL(_T)) del_dport(_T)) +static struct cxl_port *cxl_port_devres_group (struct cxl_port *port) +{ +if (!devres_open_group (&port->dev, port,GFP_KERNEL )) +return ERR_PTR (-ENOMEM ); +return port; +} +DEFINE_FREE(cxl_port_group_free, struct cxl_port *, + if (!IS_ERR_OR_NULL(_T)) devres_release_group(&(_T)->dev, _T)) + +static void cxl_port_group_close (struct cxl_port *port)This feels like misleading naming and I'm not sure what intent is. Would have expected it to call
devres_close_group()
Quoted 6 lines
Agree. The hastiness of this patch shows. Switched all the naming to not be surprising. The flow is:
cxl_port_open_group(): start recording devres resource acquisitioncxl_port_remove_group(): on success, stop tracking the group, leave the resourcescxl_port_release_group(): on failure, destroy the group, free the resources
Hi Dan, thanks for getting back on this so quickly!
Ok. So I'd misunderstood intent. If we don't have the option of close_group() then these are just for temporary tracking of potential cleanup stuff rather than because we want to optionally roll back part of the main devres stuff (the bit that gets cleaned up on driver unbind / just after remove())
I was thinking this was odd usage, but it is documented in devres.rst as one or the two group examples, so I'm less concerned about that. Maybe sprinkle a comment or two on the temporary nature of the devres group?
However, this makes me wonder why the other model in devres.rst
https://elixir.bootlin.com/linux/v6.19-rc5/source/Documentation/driver-api/driver-model/devres.rst#L187
int my_midlayer_create_something()
{
if (!devres_open_group(dev, my_midlayer_create_something, GFP_KERNEL))
return -ENOMEM;
...
devres_close_group(dev, my_midlayer_create_something);
return 0;}
void my_midlayer_destroy_something()
{
devres_release_group(dev, my_midlayer_create_something);}
isn't more appropriate here.
Did you give that approach a go? Assuming unlink_dport() cleans up all the same stuff as was covered by the group (it doesn't quite because of the last few things that can't fail) it should be a much less invasive change. A small complexity is you'd need group to be created on the right dev so that it matches what is done in the new autoremove code.
I'll give this a go, but might take a while so sending this in the meantime.
Anyhow, some of the comments that follow are on what you have done, and a few others are on what the devres_close_group approach would look like.
This might the hardest to review patch I've looked at in a while... Not sure what you could do about that though!
Quoted 21 lines
New patch, added a Fixes: tag.
-- 8< --
From9731bb6cb5638a0d2141dc072f90db0d00400680Mon Sep 17 00:00:00 2001
From: Dan Williams <[email protected]>
Date: Wed, 14 Jan 2026 12:20:40 -0600
Subject: [PATCH] cxl/port: Fix devm resource leaks with dport managementWith dport addition moving out of
cxl_switch_port_probe()it is no longer the case that a single dport-add failure will cause all dport resources to be automatically unwound.devm still helps all dport resources get cleaned up when the port is detached, but setup now needs to avoid leaking resources if an early exit occurs during setup.
Convert from a "devm add" model, to an "auto remove" model that makes the caller responsible for registering devm reclaim after the object is fully instantiated.
Quoted 5 lines
As a side of effect of this reorganization
port->nr_dportsis now always consistent with the number of entries in theport->dportsxarray, and this can stop playing games withida_is_empty()which is unreliable as a detector of whether decoders are setup. I.e. consider howCONFIG_DEBUG_KOBJECT_RELEASEmight wreak havoc with this approach.
Given complexity of ownership, can we have a flow chart of who owns what when?
Quoted 123 lines
Cc <[email protected]>Fixes 4f06d81e7c6a ("cxl: Defer dport allocation for switch ports")Signed-off-by Dan Williams <[email protected]>drivers/cxl/core/pci.c +5
-5@@ -41,14 +41,14 @@ static int pci_get_port_num(struct pci_dev *pdev) }/** -* __devm_cxl_add_dport_by_dev - allocate a dport by dport device +* __cxl_add_dport_by_dev - allocate a dport by dport device * @port: cxl_port that hosts the dport * @dport_dev: 'struct device' of the dport * * Returns the allocated dport on success or ERR_PTR() of -errno on error */ -struct cxl_dport *__devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev) +struct cxl_dport *__cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev) {struct cxl_register_map map;struct pci_dev *pdev;@@ -67,9 +67,9 @@ struct cxl_dport *__devm_cxl_add_dport_by_dev(struct cxl_port *port,return ERR_PTR (rc);device_lock_assert (&port->dev); -return devm_cxl_add_dport (port, dport_dev, port_num, map.resource); +return cxl_add_dport (port, dport_dev, port_num, map.resource); } -EXPORT_SYMBOL_NS_GPL (__devm_cxl_add_dport_by_dev,"CXL" ); +EXPORT_SYMBOL_NS_GPL (__cxl_add_dport_by_dev,"CXL" );static int cxl_dvsec_mem_range_valid (struct cxl_dev_state *cxlds,int id) {drivers/cxl/core/port.c +31
-20@@ -1066,11 +1066,28 @@ static int add_dport(struct cxl_port *port, struct cxl_dport *dport)return -EBUSY ; } +/* +* Unlike CXL switch upstream ports where it can train a CXL link +* independent of its downstream ports, a host bridge upstream port may +* not enable CXL registers until at least one downstream port (root +* port) trains CXL. Enumerate registers once when the number of dports +* transitions from zero to one. +*/ +if (!port->nr_dports) { + rc =cxl_port_setup_regs (port, port->component_reg_phys); +if (rc) +return rc; + } + +/* Arrange for dport_dev to be valid through remove_dport() */ + struct device *dev __free(put_device) = get_device(dport->dport_dev); + rc =xa_insert (&port->dports, (unsigned long )dport->dport_dev, dport,GFP_KERNEL );if (rc)return rc; +retain_and_null_ptr (dev); port->nr_dports++;return 0 ; }@@ -1094,51 +1111,64 @@ static void cond_cxl_root_unlock(struct cxl_port *port)device_unlock (&port->dev); } -static void cxl_dport_remove (void *data) +static void remove_dport (struct cxl_dport *dport) { -struct cxl_dport *dport = data;struct cxl_port *port = dport->port; + port->nr_dports--;xa_erase (&port->dports, (unsigned long ) dport->dport_dev);put_device (dport->dport_dev); } -static void cxl_dport_unlink (void *data) +static struct cxl_dport *__register_dport (struct cxl_dport *dport) { -struct cxl_dport *dport = data; -struct cxl_port *port = dport->port; +int rc;char link_name[CXL_TARGET_STRLEN ]; +struct cxl_port *port = dport->port; +struct device *dport_dev = dport->dport_dev; -sprintf (link_name,"dport%d" , dport->port_id); -sysfs_remove_link (&port->dev.kobj, link_name); -} +if (snprintf (link_name,CXL_TARGET_STRLEN ,"dport%d" , dport->port_id) >= +CXL_TARGET_STRLEN ) +return ERR_PTR (-EINVAL ); -static struct cxl_dport * -__devm_cxl_add_dport (struct cxl_port *port,struct device *dport_dev, -int port_id,resource_size_t component_reg_phys, -resource_size_t rcrb) -{ -char link_name[CXL_TARGET_STRLEN ]; -struct cxl_dport *dport; -struct device *host; -int rc; +cond_cxl_root_lock (port); + rc =add_dport (port, dport); +cond_cxl_root_unlock (port); +if (rc) +return ERR_PTR (rc); -if (is_cxl_root (port)) - host = port->uport_dev; -else
drivers/cxl/core/port.c static void cond_cxl_root_unlock(struct cxl_port *port) · line 1144
Quoted 10 lines
drivers/cxl/core/port.c +4
-5- host = &port->dev; +if (dev_is_pci (dport_dev)) + dport->link_latency =cxl_pci_get_latency (to_pci_dev (dport_dev)); -if (!host->driver) { -dev_WARN_ONCE (&port->dev,1 ,"dport:%s bad devm context\n" , -dev_name (dport_dev)); -return ERR_PTR (-ENXIO ); + rc =sysfs_create_link (&port->dev.kobj, &dport_dev->kobj, link_name); +if (rc) {
There are several operations in here that get undone in unlink_dport() I'd wrap those up in an __unregister_dport() function so it is easy to see what pairs with what.
Quoted 18 lines
drivers/cxl/core/port.c +5
-4+remove_dport (dport); +return ERR_PTR (rc); } -if (snprintf (link_name,CXL_TARGET_STRLEN ,"dport%d" , port_id) >= -CXL_TARGET_STRLEN ) -return ERR_PTR (-EINVAL ); +cxl_debugfs_create_dport_dir (dport); - dport =devm_kzalloc (host,sizeof (*dport),GFP_KERNEL ); +return dport; +}@@ -1439,13 +1430,42 @@ static void delete_switch_port(struct cxl_port *port)devm_release_action (port->dev.parent, unregister_port, port); }
drivers/cxl/core/port.c static void delete_switch_port(struct cxl_port *port) · line 1442
Quoted 10 lines
drivers/cxl/core/port.c +10
-0+static void unlink_dport (void *data) +{ +struct cxl_dport *dport = data; +struct cxl_port *port = dport->port; +char link_name[CXL_TARGET_STRLEN ]; + +sprintf (link_name,"dport%d" , dport->port_id); +sysfs_remove_link (&port->dev.kobj, link_name); +remove_dport (dport); +kfree (dport);
To me this removes half the advantage of devres which is that we don't need to be careful to remove things in the right order. Ah well, perhaps a price we need to pay.
Quoted 19 lines
drivers/cxl/core/port.c +19
-0+} + +int cxl_dport_autoremove (struct cxl_dport *dport) +{ +struct cxl_port *port = dport->port; +struct device *host; + +if (is_cxl_root (port)) + host = port->uport_dev; +else + host = &port->dev; + +return devm_add_action_or_reset (host, unlink_dport, dport); +} +EXPORT_SYMBOL_NS_GPL (cxl_dport_autoremove,"CXL" ); + +/* +* Note: this only services dynamic removal of mid-level ports, root ports are +* always removed by the platform driver (e.g. cxl_acpi). @host can be
drivers/cxl/core/port.c static void delete_switch_port(struct cxl_port *port) · line 1468
Quoted 10 lines
drivers/cxl/core/port.c +3
-3+* hard-coded to &port->dev. +*/ static void del_dport (struct cxl_dport *dport) {struct cxl_port *port = dport->port; -devm_release_action (&port->dev, cxl_dport_unlink, dport); -devm_release_action (&port->dev, cxl_dport_remove, dport); -devm_kfree (&port->dev, dport); +devm_release_action (&port->dev, unlink_dport, dport);
If you did go with the devres_close_group() suggestion I think you could then call devres_remove_group() to undo the dport stuff leaving the rest of the devres on port-dev in place.
drivers/cxl/core/port.c static int update_decoder_targets(struct device *dev, void *data) · line 1622
Quoted 11 lines
drivers/cxl/core/port.c +3
-1}static void del_dports (struct cxl_port *port)@@ -1597,10 +1617,24 @@ static int update_decoder_targets(struct device *dev, void *data)return 0 ; } -DEFINE_FREE(del_cxl_dport, struct cxl_dport *, if (!IS_ERR_OR_NULL(_T)) del_dport(_T)) +static struct cxl_port *cxl_port_open_group (struct cxl_port *port) +{ +if (!devres_open_group (&port->dev, port,GFP_KERNEL ))
The use of port as the ID is tiny bit nasty but necessary I guess for the DEFINE_FREE to work (you could use &port->dev) but that doesn't really help.
Disadvantage is you can't stack these groups without breaking the advice in devres not to reuse IDs. In practice I think that actually works but it's the sort of advice comment that makes me think it might not always do so!
Quoted 6 lines
drivers/cxl/core/port.c +6
-0+return ERR_PTR (-ENOMEM ); +return port; +} +DEFINE_FREE(cxl_port_release_group, struct cxl_port *, + if (!IS_ERR_OR_NULL(_T)) devres_release_group(&(_T)->dev, _T)) +
drivers/cxl/core/port.c static int update_decoder_targets(struct device *dev, void *data) · line 1638
Quoted 10 lines
drivers/cxl/core/port.c +6
-0+static void cxl_port_remove_group (struct cxl_port *port) +{ +devres_remove_group (&port->dev, port); +} +static struct cxl_dport *cxl_port_add_dport (struct cxl_port *port,struct device *dport_dev) { +struct cxl_dport *new_dport;struct cxl_dport *dport;
Too many dports. Given the existing one is only used for a sanity check, can we have a precursor that gets rid of it via a helper
int check_no_existing_dport(struct cxl_dport *port, struct device *dport_dev)
{
struct cxl_dport *dport = cxl_find_dport_by_dev(port, dport_dev);
if (dport) {
dev_dbg(&port->dev, "dport%d:%s already exists\n",
dport->port_id, dev_name(dport_dev));
return -EBUSY;
}
return 0;
}...
rc = check_no_existing_dport(port, dport_dev); if (rc) return ERR_PTR(rc);
then can rename new_dport to dport for this patch.
Or don't bother hiding it and just use dport variable for both.
Quoted 9 lines
drivers/cxl/core/port.c +0
-3int rc;@@ -1615,29 +1649,47 @@ static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port,return ERR_PTR (-EBUSY ); } - struct cxl_dport *new_dport __free(del_cxl_dport) = -devm_cxl_add_dport_by_dev (port, dport_dev); -if (IS_ERR (new_dport))
drivers/cxl/core/port.c static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port, · line 1658
Quoted 10 lines
drivers/cxl/core/port.c +7
-3-return new_dport; - -cxl_switch_parse_cdat (new_dport); +/* +* With the first dport arrival it is now safe to start looking at +* component registers. Be careful to not strand resources if dport +* creation ultimately fails. +*/ + struct cxl_port *port_group __free(cxl_port_release_group) = +cxl_port_open_group (port);
So this is relies on everything being registered against port->dev, whereas for root ports this then gets handed off to port->uport_dev. That's a bit obscure - hence request for some patch description text on who owns what resources + when.
Quoted 20 lines
drivers/cxl/core/port.c +10
-4+if (IS_ERR (port_group)) +return ERR_CAST (port_group); -if (ida_is_empty (&port->decoder_ida)) { +if (port->nr_dports ==0 ) { rc =devm_cxl_switch_port_decoders_setup (port);if (rc)return ERR_PTR (rc); -dev_dbg (&port->dev,"first dport%d:%s added with decoders\n" , - new_dport->port_id,dev_name (dport_dev)); -return no_free_ptr (new_dport); +/* +* Note, when nr_dports returns to zero the port is unregistered +* and triggers cleanup. I.e. no need for open-coded release +* action on dport removal. See cxl_detach_ep() for that logic. +*/ } + new_dport =cxl_add_dport_by_dev (port, dport_dev); +if (IS_ERR (new_dport))
drivers/cxl/core/port.c static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port, · line 1684
Quoted 10 lines
drivers/cxl/core/port.c +10
-0+return new_dport; + + rc =cxl_dport_autoremove (new_dport); +if (rc) +return ERR_PTR (rc); + +cxl_switch_parse_cdat (new_dport); + +/* group tracking no longer needed, dport successfully added */ +cxl_port_remove_group (no_free_ptr (port_group));
I think this is a tiny bit too late (though my head hurts so could be wrong). If we hit the error condition just above, we already freed the stuff that this group controls.
I'd be tempted to have a helper for the entire region the group is held for
helper()
{
struct cxl_port *port_group __free(cxl_port_release_group) =
cxl_port_open_group(port);
if (IS_ERR(port_group))
return ERR_CAST(port_group);
...
cxl_port_remove_group(no_free_ptr(port_group));
return something good;
} so that the scope is clear.
Quoted 15 lines
drivers/cxl/core/port.c +5
-4+ +dev_dbg (&port->dev,"dport[%d] id:%d dport_dev: %s added\n" , + port->nr_dports -1 , new_dport->port_id,dev_name (dport_dev)); +/* New dport added, update the decoder targets */ device_for_each_child (&port->dev, new_dport, update_decoder_targets); -dev_dbg (&port->dev,"dport%d:%s added\n" , new_dport->port_id, -dev_name (dport_dev)); - -return no_free_ptr (new_dport); +return new_dport; } static struct cxl_dport *devm_cxl_create_port(struct device *ep_dev,
Re: [PATCH v14 19/34] cxl/port: Fix devm resource leaks around with dport management
Quoted 34 lines
rc = check_no_existing_dport(port, dport_dev); if (rc) return ERR_PTR(rc);then can rename
new_dportto dport for this patch.
Or don't bother hiding it and just use dport variable for both.Quoted 9 lines
drivers/cxl/core/port.c +0
-3int rc;@@ -1615,29 +1649,47 @@ static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port,return ERR_PTR (-EBUSY ); } - struct cxl_dport *new_dport __free(del_cxl_dport) = -devm_cxl_add_dport_by_dev (port, dport_dev); -if (IS_ERR (new_dport))drivers/cxl/core/port.c static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port, · line 1658
Quoted 10 lines
drivers/cxl/core/port.c +7
-3-return new_dport; - -cxl_switch_parse_cdat (new_dport); +/* +* With the first dport arrival it is now safe to start looking at +* component registers. Be careful to not strand resources if dport +* creation ultimately fails. +*/ + struct cxl_port *port_group __free(cxl_port_release_group) = +cxl_port_open_group (port);So this is relies on everything being registered against
port->dev, whereas for root ports this then gets handed off toport->uport_dev. That's a bit obscure - hence request for some patch description text on who owns what resources + when.
Quoted 8 lines
drivers/cxl/core/port.c static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port, · line 1665
Quoted 8 lines
drivers/cxl/core/port.c +3
-1+if (IS_ERR (port_group)) +return ERR_CAST (port_group); -if (ida_is_empty (&port->decoder_ida)) { +if (port->nr_dports ==0 ) { rc =devm_cxl_switch_port_decoders_setup (port);if (rc)return ERR_PTR (rc);
I'm not totally sure I have an appropriate base for messing with this but with just this patch on top of cxl/for-7.0/cxl-init I'm getting problems with the reorder here as the devm_cxl_switch_port fails to map HDM decoders. On a simple single switch couple of devices test.
I'm probably suffering Friday afternoon syndrome (and naughty testing just a middle patch in a series)
My 'guess' is that it's because we no longer add the port first.
I couldn't spot any other patch messing with related logic earlier in Terry's series (and wanted to keep what I was messing with for trying alternative fixes to a minimum)
Any thoughts on what I'm missing?
Jonathan
Quoted 22 lines
Quoted 22 lines
drivers/cxl/core/port.c +17
-3-dev_dbg (&port->dev,"first dport%d:%s added with decoders\n" , - new_dport->port_id,dev_name (dport_dev)); -return no_free_ptr (new_dport); +/* +* Note, when nr_dports returns to zero the port is unregistered +* and triggers cleanup. I.e. no need for open-coded release +* action on dport removal. See cxl_detach_ep() for that logic. +*/ } + new_dport =cxl_add_dport_by_dev (port, dport_dev); +if (IS_ERR (new_dport)) +return new_dport; + + rc =cxl_dport_autoremove (new_dport); +if (rc) +return ERR_PTR (rc); + +cxl_switch_parse_cdat (new_dport); + +/* group tracking no longer needed, dport successfully added */ +cxl_port_remove_group (no_free_ptr (port_group));
Re: [PATCH v14 19/34] cxl/port: Fix devm resource leaks around with dport management
Jonathan Cameron wrote:
[..]
Quoted 1 line
Any thoughts on what I'm missing?
Probably:
cxl/port: Map Port component registers before switchport init
...which really should not be a distinct patch from the one that changes the ordering. I will send out a more patient series to settle this so Terry does not need to keep carrying it.
Re: [PATCH v14 19/34] cxl/port: Fix devm resource leaks around with dport management
Quoted 4 lines · On Mon, 19 Jan 2026 15:02:09 -0800 [email protected] wrote:
Jonathan Cameron wrote:
[..]Quoted 1 line
Any thoughts on what I'm missing?
Quoted 8 lines
Probably:
cxl/port: Map Port component registers before switchport init
...which really should not be a distinct patch from the one that changes the ordering. I will send out a more patient series to settle this so Terry does not need to keep carrying it.
Ah. I stopped looking when I got to this patch :) Spot on - thanks!
Below is my suggestion of another approach. I think it ends up a fair bit simpler, but you know this code a lot better than I do, so I may be missing some problems! I like the simpler reaping. That approach looks like it can be used elsewhere in this file.
Testing so far is one representative config only.
The patch you highlight above is squashed into this. At least one comment I made on your patch applies here too (naughty me :)
From a95567d9e2e3809ca1953a9aec24324ae13f6145 Mon Sep 17 00:00:00 2001
From: Jonathan Cameron <[email protected]>
Date: Tue, 20 Jan 2026 10:53:14 +0000
Subject: [PATCH] Alternative to Dan's approach for managing dport resources.
Create a devres group to manage a subset of resources. On success, close the devres group, but also stash a copy in struct cxl_dport so we can use it for dport reaping.
On error, when group open release it.
Somewhat tested, but not what I'd consider exhaustive yet! Used a 2 port switch, 2 EP test config.
1. Unbind endpoints, checked reap happened.
2. Unbind port1, check tear down happened.
3. Add some errors so some calls fail (stepping through each group that was created) Some cases get setup by
it having another try based on a different downstream device arrival triggering the flow, but it seems fine.
--- drivers/cxl/core/port.c | 76 ++++++++++++++++++++--------------- drivers/cxl/cxl.h | 2 + drivers/cxl/cxlpci.h | 4 ++ tools/testing/cxl/test/mock.c | 1 + 4 files changed, 51 insertions(+), 32 deletions(-)
drivers/cxl/core/port.c +44 -32
@@ -778,7 +778,7 @@ static int cxl_setup_comp_regs(struct device *host, struct cxl_register_map *mapreturn cxl_setup_regs (map); } -static int cxl_port_setup_regs (struct cxl_port *port, +int cxl_port_setup_regs (struct cxl_port *port,resource_size_t component_reg_phys) {if (dev_is_platform (port->uport_dev))
@@ -786,6 +786,7 @@ static int cxl_port_setup_regs(struct cxl_port *port,return cxl_setup_comp_regs (&port->dev, &port->reg_map, component_reg_phys); } +EXPORT_SYMBOL_NS_GPL (cxl_port_setup_regs,"CXL" );static int cxl_dport_setup_regs (struct device *host,struct cxl_dport *dport,resource_size_t component_reg_phys)
@@ -1065,7 +1066,12 @@ static int add_dport(struct cxl_port *port, struct cxl_dport *dport) dev_name(dup->dport_dev));return -EBUSY ; } - +//comment from original patch here. +if (!port->nr_dports) { + rc =cxl_port_setup_regs (port, port->component_reg_phys); +if (rc) +return rc; + } rc =xa_insert (&port->dports, (unsigned long )dport->dport_dev, dport,GFP_KERNEL );if (rc)
@@ -1181,20 +1187,6 @@ __devm_cxl_add_dport(struct cxl_port *port, struct device *dport_dev,if (rc)return ERR_PTR (rc); -/* -* Setup port register if this is the first dport showed up. Having -* a dport also means that there is at least 1 active link. -*/ -if (port->nr_dports ==1 && - port->component_reg_phys !=CXL_RESOURCE_NONE ) { - rc =cxl_port_setup_regs (port, port->component_reg_phys); -if (rc) { -xa_erase (&port->dports, (unsigned long )dport->dport_dev); -return ERR_PTR (rc); - } - port->component_reg_phys =CXL_RESOURCE_NONE ; - } -get_device (dport_dev); rc =devm_add_action_or_reset (host, cxl_dport_remove, dport);if (rc)
@@ -1441,11 +1433,7 @@ static void delete_switch_port(struct cxl_port *port)static void del_dport (struct cxl_dport *dport) { -struct cxl_port *port = dport->port; - -devm_release_action (&port->dev, cxl_dport_unlink, dport); -devm_release_action (&port->dev, cxl_dport_remove, dport); -devm_kfree (&port->dev, dport); +devres_release_group (&dport->port->dev, dport->devres_group); }static void del_dports (struct cxl_port *port)
@@ -1602,6 +1590,9 @@ static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port, struct device *dport_dev) {struct cxl_dport *dport; +struct cxl_dport *new_dport; +struct device *host; +void *devres_group;int rc;device_lock_assert (&port->dev);
@@ -1615,21 +1606,31 @@ static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port,return ERR_PTR (-EBUSY ); } - struct cxl_dport *new_dport __free(del_cxl_dport) = -devm_cxl_add_dport_by_dev (port, dport_dev); -if (IS_ERR (new_dport)) -return new_dport; +if (is_cxl_root (port)) + host = port->uport_dev; +else + host = &port->dev; + + devres_group =devres_open_group (host,NULL ,GFP_KERNEL ); +if (!devres_group) +return ERR_PTR (-ENOMEM ); -cxl_switch_parse_cdat (new_dport); +if (port->nr_dports ==0 ) { + rc =cxl_port_setup_regs (port, port->component_reg_phys); +if (rc) + goto release_group; -if (ida_is_empty (&port->decoder_ida)) { rc =devm_cxl_switch_port_decoders_setup (port);if (rc) -return ERR_PTR (rc); -dev_dbg (&port->dev,"first dport%d:%s added with decoders\n" , - new_dport->port_id,dev_name (dport_dev)); -return no_free_ptr (new_dport); + goto release_group; + } + + new_dport =devm_cxl_add_dport_by_dev (port, dport_dev); +if (IS_ERR (new_dport)) { + rc =PTR_ERR (new_dport); + goto release_group; } +cxl_switch_parse_cdat (new_dport);/* New dport added, update the decoder targets */ device_for_each_child (&port->dev, new_dport, update_decoder_targets);
@@ -1637,7 +1638,18 @@ static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port,dev_dbg (&port->dev,"dport%d:%s added\n" , new_dport->port_id,dev_name (dport_dev)); -return no_free_ptr (new_dport); +/* +* Stash the group for use during reaping when all downstream devices +* go away. +*/ + new_dport->devres_group = devres_group; +devres_close_group (host, devres_group); + +return new_dport; + +release_group: +devres_release_group (host, devres_group); +return ERR_PTR (rc); } static struct cxl_dport *devm_cxl_create_port(struct device *ep_dev,
drivers/cxl/cxl.h +2 -0
@@ -688,6 +688,7 @@ struct cxl_rcrb_info {* @coord: access coordinates (bandwidth and latency performance attributes) * @link_latency: calculated PCIe downstream latency * @gpf_dvsec: Cached GPF port DVSEC +* @devres_group: Used to simplify reaping ports. */ struct cxl_dport {struct device *dport_dev;
@@ -700,6 +701,7 @@ struct cxl_dport {struct access_coordinate coord[ACCESS_COORDINATE_MAX ];long link_latency;int gpf_dvsec; +void *devres_group; };/**
drivers/cxl/cxlpci.h +4 -0
@@ -132,4 +132,8 @@ void read_cdat_data(struct cxl_port *port);void cxl_cor_error_detected (struct pci_dev *pdev);pci_ers_result_t cxl_error_detected (struct pci_dev *pdev,pci_channel_state_t state); + +int cxl_port_setup_regs (struct cxl_port *port, +resource_size_t component_reg_phys); +#endif /* __CXL_PCI_H__ */
Re: [PATCH v14 19/34] cxl/port: Fix devm resource leaks around with dport management
In reply to Jonathan CameronJonathan Cameron wrote:
[..]
Quoted 1 line
This might the hardest to review patch I've looked at in a while...
Well, that *is* fatal feedback. I think this simply needs to be broken up into smaller to digest pieces. Will send that shortly.
PATCH v14 20/34 cxl/port: Move dport operations to a driver event
In reply to Terry BowmanFrom: Dan Williams <[email protected]>
In preparation for adding more register setup to the cxl_port_add_dport() path (for RAS register mapping), move the dport creation event to a driver callback. This achieves 2 things it puts driver operations logically where they belong, in a driver, and it obviates the gymnastics of DECLARE_TESTABLE() which just makes a mess of grepping for CXL symbols.
In other words, a driver callback is less of an ongoing maintenance burden than this DECLARE_TESTABLE arrangement that does not scale and diminishes the grep-ability of the codebase.
cxl_port_add_dport() moves mostly unmodified from drivers/cxl/core/port.c. The only deliberate change is that it now assumes that the device_lock is held on entry and the driver is attached (just like cxl_port_probe()).
---
Changes in v13 -> v14:
- New patch
--- drivers/cxl/core/hdm.c | 6 +-- drivers/cxl/core/pci.c | 8 +-- drivers/cxl/core/port.c | 79 ++++++---------------------- drivers/cxl/cxl.h | 23 ++------ drivers/cxl/port.c | 71 +++++++++++++++++++++++++ tools/testing/cxl/Kbuild | 2 + tools/testing/cxl/cxl_core_exports.c | 21 -------- tools/testing/cxl/exports.h | 13 ----- tools/testing/cxl/test/mock.c | 23 +++----- 9 files changed, 107 insertions(+), 139 deletions(-) delete mode 100644 tools/testing/cxl/exports.h
drivers/cxl/core/hdm.c +3 -3
@@ -1219,12 +1219,12 @@ static int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm, }/** -* __devm_cxl_switch_port_decoders_setup - allocate and setup switch decoders +* devm_cxl_switch_port_decoders_setup - allocate and setup switch decoders * @port: CXL port context * * Return 0 or -errno on error */ -int __devm_cxl_switch_port_decoders_setup (struct cxl_port *port) +int devm_cxl_switch_port_decoders_setup (struct cxl_port *port) {struct cxl_hdm *cxlhdm;
@@ -1248,7 +1248,7 @@ int __devm_cxl_switch_port_decoders_setup(struct cxl_port *port)dev_err (&port->dev,"HDM decoder capability not found\n" );return -ENXIO ; } -EXPORT_SYMBOL_NS_GPL (__devm_cxl_switch_port_decoders_setup,"CXL" ); +EXPORT_SYMBOL_NS_GPL (devm_cxl_switch_port_decoders_setup,"CXL" );/** * devm_cxl_endpoint_decoders_setup - allocate and setup endpoint decoders
drivers/cxl/core/pci.c +4 -4
@@ -41,14 +41,14 @@ static int pci_get_port_num(struct pci_dev *pdev) }/** -* __cxl_add_dport_by_dev - allocate a dport by dport device +* cxl_add_dport_by_dev - allocate a dport by dport device * @port: cxl_port that hosts the dport * @dport_dev: 'struct device' of the dport * * Returns the allocated dport on success or ERR_PTR() of -errno on error */ -struct cxl_dport *__cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev) +struct cxl_dport *cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev) {struct cxl_register_map map;struct pci_dev *pdev;
@@ -69,7 +69,7 @@ struct cxl_dport *__cxl_add_dport_by_dev(struct cxl_port *port,device_lock_assert (&port->dev);return cxl_add_dport (port, dport_dev, port_num, map.resource); } -EXPORT_SYMBOL_NS_GPL (__cxl_add_dport_by_dev,"CXL" ); +EXPORT_SYMBOL_NS_GPL (cxl_add_dport_by_dev,"CXL" );struct cxl_walk_context {struct pci_bus *bus;
drivers/cxl/core/port.c +16 -63
115 lines not on this page · raw message
drivers/cxl/cxl.h +4 -19
@@ -841,8 +841,9 @@ struct cxl_endpoint_dvsec_info { };int devm_cxl_switch_port_decoders_setup (struct cxl_port *port); -int __devm_cxl_switch_port_decoders_setup (struct cxl_port *port);int devm_cxl_endpoint_decoders_setup (struct cxl_port *port); +void cxl_port_update_decoder_targets (struct cxl_port *port, +struct cxl_dport *dport);struct cxl_dev_state ; int cxl_dvsec_rr_decode(struct cxl_dev_state *cxlds,
@@ -856,6 +857,8 @@ struct cxl_driver {const char *name;int (*probe)(struct device *dev);void (*remove)(struct device *dev); +struct cxl_dport *(*add_dport)(struct cxl_port *port, +struct device *dport_dev);struct device_driver drv;int id; };
@@ -940,8 +943,6 @@ void cxl_coordinates_combine(struct access_coordinate *out,bool cxl_endpoint_decoder_reset_detected (struct cxl_port *port);struct cxl_dport *cxl_add_dport_by_dev (struct cxl_port *port,struct device *dport_dev); -struct cxl_dport *__cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev);/* * Unit test builds overrides this to __weak, find the 'strong' version
@@ -953,20 +954,4 @@ struct cxl_dport *__cxl_add_dport_by_dev(struct cxl_port *port,u16 cxl_gpf_get_dvsec (struct device *dev); -/* -* Declaration for functions that are mocked by cxl_test that are called by -* cxl_core. The respective functions are defined as __foo() and called by -* cxl_core as foo(). The macros below ensures that those functions would -* exist as foo(). See tools/testing/cxl/cxl_core_exports.c and -* tools/testing/cxl/exports.h for setting up the mock functions. The dance -* is done to avoid a circular dependency where cxl_core calls a function that -* ends up being a mock function and goes to * cxl_test where it calls a -* cxl_core function. -*/ -#ifndef CXL_TEST_ENABLE -#define DECLARE_TESTABLE (x) __##x -#define cxl_add_dport_by_dev DECLARE_TESTABLE(cxl_add_dport_by_dev) -#define devm_cxl_switch_port_decoders_setup DECLARE_TESTABLE(devm_cxl_switch_port_decoders_setup) -#endif -#endif /* __CXL_H__ */
drivers/cxl/port.c +71 -0
80 lines not on this page · raw message
tools/testing/cxl/Kbuild +2 -0
@@ -11,6 +11,8 @@ ldflags-y += --wrap=cxl_endpoint_parse_cdat ldflags-y += --wrap=cxl_dport_init_ras_reporting ldflags-y += --wrap=devm_cxl_endpoint_decoders_setup ldflags-y += --wrap=hmat_get_extended_linear_cache_size +ldflags-y += --wrap=cxl_add_dport_by_dev +ldflags-y += --wrap=devm_cxl_switch_port_decoders_setup DRIVERS := ../../../drivers CXL_SRC := $(DRIVERS)/cxl
tools/testing/cxl/cxl_core_exports.c +0 -21
@@ -2,27 +2,6 @@/* Copyright(c) 2022 Intel Corporation. All rights reserved. */ #include "cxl.h" -#include "exports.h" /* Exporting of cxl_core symbols that are only used by cxl_test */ EXPORT_SYMBOL_NS_GPL (cxl_num_decoders_committed,"CXL" ); - -cxl_add_dport_by_dev_fn _cxl_add_dport_by_dev = __cxl_add_dport_by_dev; -EXPORT_SYMBOL_NS_GPL (_cxl_add_dport_by_dev,"CXL" ); - -struct cxl_dport *cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev) -{ -return _cxl_add_dport_by_dev (port, dport_dev); -} -EXPORT_SYMBOL_NS_GPL (cxl_add_dport_by_dev,"CXL" ); - -cxl_switch_decoders_setup_fn _devm_cxl_switch_port_decoders_setup = - __devm_cxl_switch_port_decoders_setup; -EXPORT_SYMBOL_NS_GPL (_devm_cxl_switch_port_decoders_setup,"CXL" ); - -int devm_cxl_switch_port_decoders_setup (struct cxl_port *port) -{ -return _devm_cxl_switch_port_decoders_setup (port); -} -EXPORT_SYMBOL_NS_GPL (devm_cxl_switch_port_decoders_setup,"CXL" );
tools/testing/cxl/exports.h deleted +0 -13
@@ -1,13 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* Copyright(c) 2025 Intel Corporation */ -#ifndef __MOCK_CXL_EXPORTS_H_ -#define __MOCK_CXL_EXPORTS_H_ - -typedef struct cxl_dport *(*cxl_add_dport_by_dev_fn )(struct cxl_port *port, -struct device *dport_dev); -extern cxl_add_dport_by_dev_fn _cxl_add_dport_by_dev; - -typedef int (*cxl_switch_decoders_setup_fn )(struct cxl_port *port); -extern cxl_switch_decoders_setup_fn _devm_cxl_switch_port_decoders_setup; - -#endif
tools/testing/cxl/test/mock.c +7 -16
@@ -10,20 +10,12 @@#include <cxlmem.h> #include <cxlpci.h> #include "mock.h" -#include "../exports.h" static LIST_HEAD (mock); -static struct cxl_dport * -redirect_cxl_add_dport_by_dev (struct cxl_port *port,struct device *dport_dev); -static int redirect_devm_cxl_switch_port_decoders_setup (struct cxl_port *port); -void register_cxl_mock_ops (struct cxl_mock_ops *ops) {list_add_rcu (&ops->list, &mock); - _cxl_add_dport_by_dev = redirect_cxl_add_dport_by_dev; - _devm_cxl_switch_port_decoders_setup = - redirect_devm_cxl_switch_port_decoders_setup; }EXPORT_SYMBOL_GPL (register_cxl_mock_ops);
@@ -31,9 +23,6 @@ DEFINE_STATIC_SRCU(cxl_mock_srcu);void unregister_cxl_mock_ops (struct cxl_mock_ops *ops) { - _devm_cxl_switch_port_decoders_setup = - __devm_cxl_switch_port_decoders_setup; - _cxl_add_dport_by_dev = __cxl_add_dport_by_dev;list_del_rcu (&ops->list);synchronize_srcu (&cxl_mock_srcu); }
@@ -162,7 +151,7 @@ __wrap_nvdimm_bus_register(struct device *dev, }EXPORT_SYMBOL_GPL (__wrap_nvdimm_bus_register); -int redirect_devm_cxl_switch_port_decoders_setup (struct cxl_port *port) +int __wrap_devm_cxl_switch_port_decoders_setup (struct cxl_port *port) {int rc, index;struct cxl_mock_ops *ops =get_cxl_mock_ops (&index);
@@ -170,11 +159,12 @@ int redirect_devm_cxl_switch_port_decoders_setup(struct cxl_port *port)if (ops && ops->is_mock_port (port->uport_dev)) rc = ops->devm_cxl_switch_port_decoders_setup (port);else - rc =__devm_cxl_switch_port_decoders_setup (port); + rc =devm_cxl_switch_port_decoders_setup (port);put_cxl_mock_ops (index);return rc; } +EXPORT_SYMBOL_NS_GPL (__wrap_devm_cxl_switch_port_decoders_setup,"CXL" );int __wrap_devm_cxl_endpoint_decoders_setup (struct cxl_port *port) {
@@ -256,8 +246,8 @@ void __wrap_cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device }EXPORT_SYMBOL_NS_GPL (__wrap_cxl_dport_init_ras_reporting,"CXL" ); -struct cxl_dport *redirect_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev) +struct cxl_dport *__wrap_cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev) {int index;struct cxl_mock_ops *ops =get_cxl_mock_ops (&index);
@@ -266,11 +256,12 @@ struct cxl_dport *redirect_cxl_add_dport_by_dev(struct cxl_port *port,if (ops && ops->is_mock_port (port->uport_dev)) dport = ops->cxl_add_dport_by_dev (port, dport_dev);else - dport =__cxl_add_dport_by_dev (port, dport_dev); + dport =cxl_add_dport_by_dev (port, dport_dev);put_cxl_mock_ops (index);return dport; } +EXPORT_SYMBOL_NS_GPL (__wrap_cxl_add_dport_by_dev,"CXL" );MODULE_LICENSE ("GPL v2" );MODULE_DESCRIPTION ("cxl_test: emulation module" );
Signature · 1 line
2.34.1
Re: [PATCH v14 20/34] cxl/port: Move dport operations to a driver event
Quoted 12 lines · Terry Bowman
From: Dan Williams <[email protected]>
In preparation for adding more register setup to the
cxl_port_add_dport()path (for RAS register mapping), move the dport creation event to a driver callback. This achieves 2 things it puts driver operations logically where they belong, in a driver, and it obviates the gymnastics ofDECLARE_TESTABLE()which just makes a mess of grepping for CXL symbols.In other words, a driver callback is less of an ongoing maintenance burden than this
DECLARE_TESTABLEarrangement that does not scale and diminishes the grep-ability of the codebase.
Quoted 6 lines
cxl_port_add_dport()moves mostly unmodified fromdrivers/cxl/core/port.c. The only deliberate change is that it now assumes that thedevice_lockis held on entry and the driver is attached (just likecxl_port_probe()).Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>
Missing sign off
Quoted 486 lines
---
Changes in v13 -> v14:
- New patch--- drivers/cxl/core/hdm.c | 6 +-- drivers/cxl/core/pci.c | 8 +-- drivers/cxl/core/port.c | 79 ++++++---------------------- drivers/cxl/cxl.h | 23 ++------ drivers/cxl/port.c | 71 +++++++++++++++++++++++++ tools/testing/cxl/Kbuild | 2 + tools/testing/cxl/cxl_core_exports.c | 21 -------- tools/testing/cxl/exports.h | 13 ----- tools/testing/cxl/test/mock.c | 23 +++----- 9 files changed, 107 insertions(+), 139 deletions(-) delete mode 100644 tools/testing/cxl/exports.hdrivers/cxl/core/hdm.c +3
-3@@ -1219,12 +1219,12 @@ static int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm, }/** -* __devm_cxl_switch_port_decoders_setup - allocate and setup switch decoders +* devm_cxl_switch_port_decoders_setup - allocate and setup switch decoders * @port: CXL port context * * Return 0 or -errno on error */ -int __devm_cxl_switch_port_decoders_setup (struct cxl_port *port) +int devm_cxl_switch_port_decoders_setup (struct cxl_port *port) {struct cxl_hdm *cxlhdm;@@ -1248,7 +1248,7 @@ int __devm_cxl_switch_port_decoders_setup(struct cxl_port *port)dev_err (&port->dev,"HDM decoder capability not found\n" );return -ENXIO ; } -EXPORT_SYMBOL_NS_GPL (__devm_cxl_switch_port_decoders_setup,"CXL" ); +EXPORT_SYMBOL_NS_GPL (devm_cxl_switch_port_decoders_setup,"CXL" );/** * devm_cxl_endpoint_decoders_setup - allocate and setup endpoint decoders drivers/cxl/core/pci.c +4
-4@@ -41,14 +41,14 @@ static int pci_get_port_num(struct pci_dev *pdev) }/** -* __cxl_add_dport_by_dev - allocate a dport by dport device +* cxl_add_dport_by_dev - allocate a dport by dport device * @port: cxl_port that hosts the dport * @dport_dev: 'struct device' of the dport * * Returns the allocated dport on success or ERR_PTR() of -errno on error */ -struct cxl_dport *__cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev) +struct cxl_dport *cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev) {struct cxl_register_map map;struct pci_dev *pdev;@@ -69,7 +69,7 @@ struct cxl_dport *__cxl_add_dport_by_dev(struct cxl_port *port,device_lock_assert (&port->dev);return cxl_add_dport (port, dport_dev, port_num, map.resource); } -EXPORT_SYMBOL_NS_GPL (__cxl_add_dport_by_dev,"CXL" ); +EXPORT_SYMBOL_NS_GPL (cxl_add_dport_by_dev,"CXL" );struct cxl_walk_context {struct pci_bus *bus;drivers/cxl/core/port.c +16
-63@@ -1603,78 +1603,31 @@ static int update_decoder_targets(struct device *dev, void *data)return 0 ; } -static struct cxl_port *cxl_port_devres_group (struct cxl_port *port) +void cxl_port_update_decoder_targets (struct cxl_port *port, +struct cxl_dport *dport) { -if (!devres_open_group (&port->dev, port,GFP_KERNEL )) -return ERR_PTR (-ENOMEM ); -return port; +device_for_each_child (&port->dev, dport, update_decoder_targets); } +EXPORT_SYMBOL_NS_GPL (cxl_port_update_decoder_targets,"CXL" ); + DEFINE_FREE(cxl_port_group_free, struct cxl_port *, if (!IS_ERR_OR_NULL(_T)) devres_release_group(&(_T)->dev, _T)) -static void cxl_port_group_close (struct cxl_port *port) -{ -devres_remove_group (&port->dev, port); -} - -static struct cxl_dport *cxl_port_add_dport (struct cxl_port *port, -struct device *dport_dev) +static struct cxl_dport *probe_dport (struct cxl_port *port, +struct device *dport_dev) { -struct cxl_dport *new_dport; -struct cxl_dport *dport; -int rc; +struct cxl_driver *drv;device_lock_assert (&port->dev);if (!port->dev.driver)return ERR_PTR (-ENXIO ); - dport =cxl_find_dport_by_dev (port, dport_dev); -if (dport) { -dev_dbg (&port->dev,"dport%d:%s already exists\n" , - dport->port_id,dev_name (dport_dev)); -return ERR_PTR (-EBUSY ); - } - -/* -* With the first dport arrival it is now safe to start looking at -* component registers. Be careful to not strand resources if dport -* creation ultimately fails. -*/ - struct cxl_port *port_group __free(cxl_port_group_free) = -cxl_port_devres_group (port); -if (IS_ERR (port_group)) -return ERR_CAST (port_group); - -if (port->nr_dports ==0 ) { - rc =devm_cxl_switch_port_decoders_setup (port); -if (rc) -return ERR_PTR (rc); -/* -* Note, when nr_dports returns to zero the port is unregistered -* and triggers cleanup. I.e. no need for open-coded release -* action on dport removal. See cxl_detach_ep() for that logic. -*/ - } - - new_dport =cxl_add_dport_by_dev (port, dport_dev); -if (IS_ERR (new_dport)) -return new_dport; - - rc =cxl_dport_autoremove (new_dport); -if (rc) -return ERR_PTR (rc); - -cxl_switch_parse_cdat (new_dport); - -cxl_port_group_close (no_free_ptr (port_group)); - -dev_dbg (&port->dev,"dport[%d] id:%d dport_dev: %s added\n" , - port->nr_dports -1 , new_dport->port_id,dev_name (dport_dev)); - -/* New dport added, update the decoder targets */ -device_for_each_child (&port->dev, new_dport, update_decoder_targets); + drv = container_of(port->dev.driver, struct cxl_driver, drv); +if (!drv->add_dport) +return ERR_PTR (-ENXIO ); -return new_dport; +/* see cxl_port_add_dport() */ +return drv->add_dport (port, dport_dev); } static struct cxl_dport *devm_cxl_create_port(struct device *ep_dev,@@ -1721,7 +1674,7 @@ static struct cxl_dport *devm_cxl_create_port(struct device *ep_dev, }guard (device)(&port->dev); -return cxl_port_add_dport (port, dport_dev); +return probe_dport (port, dport_dev); } static int add_port_attach_ep(struct cxl_memdev *cxlmd,@@ -1753,7 +1706,7 @@ static int add_port_attach_ep(struct cxl_memdev *cxlmd, scoped_guard(device, &parent_port->dev) { parent_dport =cxl_find_dport_by_dev (parent_port, dparent);if (!parent_dport) { - parent_dport =cxl_port_add_dport (parent_port, dparent); + parent_dport =probe_dport (parent_port, dparent);if (IS_ERR (parent_dport))return PTR_ERR (parent_dport); }@@ -1789,7 +1742,7 @@ static struct cxl_dport *find_or_add_dport(struct cxl_port *port,device_lock_assert (&port->dev); dport =cxl_find_dport_by_dev (port, dport_dev);if (!dport) { - dport =cxl_port_add_dport (port, dport_dev); + dport =probe_dport (port, dport_dev);if (IS_ERR (dport))return dport;drivers/cxl/cxl.h +4
-19@@ -841,8 +841,9 @@ struct cxl_endpoint_dvsec_info { };int devm_cxl_switch_port_decoders_setup (struct cxl_port *port); -int __devm_cxl_switch_port_decoders_setup (struct cxl_port *port);int devm_cxl_endpoint_decoders_setup (struct cxl_port *port); +void cxl_port_update_decoder_targets (struct cxl_port *port, +struct cxl_dport *dport);struct cxl_dev_state ; int cxl_dvsec_rr_decode(struct cxl_dev_state *cxlds,@@ -856,6 +857,8 @@ struct cxl_driver {const char *name;int (*probe)(struct device *dev);void (*remove)(struct device *dev); +struct cxl_dport *(*add_dport)(struct cxl_port *port, +struct device *dport_dev);struct device_driver drv;int id; };@@ -940,8 +943,6 @@ void cxl_coordinates_combine(struct access_coordinate *out,bool cxl_endpoint_decoder_reset_detected (struct cxl_port *port);struct cxl_dport *cxl_add_dport_by_dev (struct cxl_port *port,struct device *dport_dev); -struct cxl_dport *__cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev);/* * Unit test builds overrides this to __weak, find the 'strong' version @@ -953,20 +954,4 @@ struct cxl_dport *__cxl_add_dport_by_dev(struct cxl_port *port,u16 cxl_gpf_get_dvsec (struct device *dev); -/* -* Declaration for functions that are mocked by cxl_test that are called by -* cxl_core. The respective functions are defined as __foo() and called by -* cxl_core as foo(). The macros below ensures that those functions would -* exist as foo(). See tools/testing/cxl/cxl_core_exports.c and -* tools/testing/cxl/exports.h for setting up the mock functions. The dance -* is done to avoid a circular dependency where cxl_core calls a function that -* ends up being a mock function and goes to * cxl_test where it calls a -* cxl_core function. -*/ -#ifndef CXL_TEST_ENABLE -#define DECLARE_TESTABLE (x) __##x -#define cxl_add_dport_by_dev DECLARE_TESTABLE(cxl_add_dport_by_dev) -#define devm_cxl_switch_port_decoders_setup DECLARE_TESTABLE(devm_cxl_switch_port_decoders_setup) -#endif -#endif /* __CXL_H__ */ drivers/cxl/port.c +71
-0@@ -155,9 +155,80 @@ static const struct attribute_group *cxl_port_attribute_groups[] = { NULL, }; +static struct cxl_port *cxl_port_devres_group (struct cxl_port *port) +{ +if (!devres_open_group (&port->dev, port,GFP_KERNEL )) +return ERR_PTR (-ENOMEM ); +return port; +} +DEFINE_FREE(cxl_port_group_free, struct cxl_port *, + if (!IS_ERR_OR_NULL(_T)) devres_release_group(&(_T)->dev, _T)) + +static void cxl_port_group_close (struct cxl_port *port) +{ +devres_remove_group (&port->dev, port); +} + +static struct cxl_dport *cxl_port_add_dport (struct cxl_port *port, +struct device *dport_dev) +{ +struct cxl_dport *new_dport; +struct cxl_dport *dport; +int rc; + + dport =cxl_find_dport_by_dev (port, dport_dev); +if (dport) { +dev_dbg (&port->dev,"dport%d:%s already exists\n" , + dport->port_id,dev_name (dport_dev)); +return ERR_PTR (-EBUSY ); + } + +/* +* With the first dport arrival it is now safe to start looking at +* component registers. Be careful to not strand resources if dport +* creation ultimately fails. +*/ + struct cxl_port *port_group __free(cxl_port_group_free) = +cxl_port_devres_group (port); +if (IS_ERR (port_group)) +return ERR_CAST (port_group); + +if (port->nr_dports ==0 ) { + rc =devm_cxl_switch_port_decoders_setup (port); +if (rc) +return ERR_PTR (rc); +/* +* Note, when nr_dports returns to zero the port is unregistered +* and triggers cleanup. I.e. no need for open-coded release +* action on dport removal. See cxl_detach_ep() for that logic. +*/ + } + + new_dport =cxl_add_dport_by_dev (port, dport_dev); +if (IS_ERR (new_dport)) +return new_dport; + + rc =cxl_dport_autoremove (new_dport); +if (rc) +return ERR_PTR (rc); + +cxl_switch_parse_cdat (new_dport); + +cxl_port_group_close (no_free_ptr (port_group)); + +dev_dbg (&port->dev,"dport[%d] id:%d dport_dev: %s added\n" , + port->nr_dports -1 , new_dport->port_id,dev_name (dport_dev)); + +/* New dport added, update the decoder targets */ +cxl_port_update_decoder_targets (port, new_dport); + +return new_dport; +} +static struct cxl_driver cxl_port_driver = { .name ="cxl_port" , .probe = cxl_port_probe, + .add_dport = cxl_port_add_dport, .id =CXL_DEVICE_PORT , .drv = { .dev_groups = cxl_port_attribute_groups,tools/testing/cxl/Kbuild +2
-0@@ -11,6 +11,8 @@ ldflags-y += --wrap=cxl_endpoint_parse_cdat ldflags-y += --wrap=cxl_dport_init_ras_reporting ldflags-y += --wrap=devm_cxl_endpoint_decoders_setup ldflags-y += --wrap=hmat_get_extended_linear_cache_size +ldflags-y += --wrap=cxl_add_dport_by_dev +ldflags-y += --wrap=devm_cxl_switch_port_decoders_setup DRIVERS := ../../../drivers CXL_SRC := $(DRIVERS)/cxltools/testing/cxl/cxl_core_exports.c +0
-21@@ -2,27 +2,6 @@/* Copyright(c) 2022 Intel Corporation. All rights reserved. */ #include "cxl.h" -#include "exports.h" /* Exporting of cxl_core symbols that are only used by cxl_test */ EXPORT_SYMBOL_NS_GPL (cxl_num_decoders_committed,"CXL" ); - -cxl_add_dport_by_dev_fn _cxl_add_dport_by_dev = __cxl_add_dport_by_dev; -EXPORT_SYMBOL_NS_GPL (_cxl_add_dport_by_dev,"CXL" ); - -struct cxl_dport *cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev) -{ -return _cxl_add_dport_by_dev (port, dport_dev); -} -EXPORT_SYMBOL_NS_GPL (cxl_add_dport_by_dev,"CXL" ); - -cxl_switch_decoders_setup_fn _devm_cxl_switch_port_decoders_setup = - __devm_cxl_switch_port_decoders_setup; -EXPORT_SYMBOL_NS_GPL (_devm_cxl_switch_port_decoders_setup,"CXL" ); - -int devm_cxl_switch_port_decoders_setup (struct cxl_port *port) -{ -return _devm_cxl_switch_port_decoders_setup (port); -} -EXPORT_SYMBOL_NS_GPL (devm_cxl_switch_port_decoders_setup,"CXL" );tools/testing/cxl/exports.h deleted +0
-13@@ -1,13 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* Copyright(c) 2025 Intel Corporation */ -#ifndef __MOCK_CXL_EXPORTS_H_ -#define __MOCK_CXL_EXPORTS_H_ - -typedef struct cxl_dport *(*cxl_add_dport_by_dev_fn )(struct cxl_port *port, -struct device *dport_dev); -extern cxl_add_dport_by_dev_fn _cxl_add_dport_by_dev; - -typedef int (*cxl_switch_decoders_setup_fn )(struct cxl_port *port); -extern cxl_switch_decoders_setup_fn _devm_cxl_switch_port_decoders_setup; - -#endif tools/testing/cxl/test/mock.c +7
-16@@ -10,20 +10,12 @@#include <cxlmem.h> #include <cxlpci.h> #include "mock.h" -#include "../exports.h" static LIST_HEAD (mock); -static struct cxl_dport * -redirect_cxl_add_dport_by_dev (struct cxl_port *port,struct device *dport_dev); -static int redirect_devm_cxl_switch_port_decoders_setup (struct cxl_port *port); -void register_cxl_mock_ops (struct cxl_mock_ops *ops) {list_add_rcu (&ops->list, &mock); - _cxl_add_dport_by_dev = redirect_cxl_add_dport_by_dev; - _devm_cxl_switch_port_decoders_setup = - redirect_devm_cxl_switch_port_decoders_setup; }EXPORT_SYMBOL_GPL (register_cxl_mock_ops);@@ -31,9 +23,6 @@ DEFINE_STATIC_SRCU(cxl_mock_srcu);void unregister_cxl_mock_ops (struct cxl_mock_ops *ops) { - _devm_cxl_switch_port_decoders_setup = - __devm_cxl_switch_port_decoders_setup; - _cxl_add_dport_by_dev = __cxl_add_dport_by_dev;list_del_rcu (&ops->list);synchronize_srcu (&cxl_mock_srcu); }@@ -162,7 +151,7 @@ __wrap_nvdimm_bus_register(struct device *dev, }EXPORT_SYMBOL_GPL (__wrap_nvdimm_bus_register); -int redirect_devm_cxl_switch_port_decoders_setup (struct cxl_port *port) +int __wrap_devm_cxl_switch_port_decoders_setup (struct cxl_port *port) {int rc, index;struct cxl_mock_ops *ops =get_cxl_mock_ops (&index);@@ -170,11 +159,12 @@ int redirect_devm_cxl_switch_port_decoders_setup(struct cxl_port *port)if (ops && ops->is_mock_port (port->uport_dev)) rc = ops->devm_cxl_switch_port_decoders_setup (port);else - rc =__devm_cxl_switch_port_decoders_setup (port); + rc =devm_cxl_switch_port_decoders_setup (port);put_cxl_mock_ops (index);return rc; } +EXPORT_SYMBOL_NS_GPL (__wrap_devm_cxl_switch_port_decoders_setup,"CXL" );int __wrap_devm_cxl_endpoint_decoders_setup (struct cxl_port *port) {@@ -256,8 +246,8 @@ void __wrap_cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device }EXPORT_SYMBOL_NS_GPL (__wrap_cxl_dport_init_ras_reporting,"CXL" ); -struct cxl_dport *redirect_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev) +struct cxl_dport *__wrap_cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev) {int index;struct cxl_mock_ops *ops =get_cxl_mock_ops (&index);@@ -266,11 +256,12 @@ struct cxl_dport *redirect_cxl_add_dport_by_dev(struct cxl_port *port,if (ops && ops->is_mock_port (port->uport_dev)) dport = ops->cxl_add_dport_by_dev (port, dport_dev);else - dport =__cxl_add_dport_by_dev (port, dport_dev); + dport =cxl_add_dport_by_dev (port, dport_dev);put_cxl_mock_ops (index);return dport; } +EXPORT_SYMBOL_NS_GPL (__wrap_cxl_add_dport_by_dev,"CXL" );MODULE_LICENSE ("GPL v2" );MODULE_DESCRIPTION ("cxl_test: emulation module" );
Re: [PATCH v14 20/34] cxl/port: Move dport operations to a driver event
In reply to Terry BowmanQuoted 12 lines · On Wed, 14 Jan 2026 12:20:41 -0600 Terry Bowman <[email protected]> wrote:
From: Dan Williams <[email protected]>
In preparation for adding more register setup to the
cxl_port_add_dport()path (for RAS register mapping), move the dport creation event to a driver callback. This achieves 2 things it puts driver operations logically where they belong, in a driver, and it obviates the gymnastics ofDECLARE_TESTABLE()which just makes a mess of grepping for CXL symbols.In other words, a driver callback is less of an ongoing maintenance burden than this
DECLARE_TESTABLEarrangement that does not scale and diminishes the grep-ability of the codebase.
Quoted 6 lines
cxl_port_add_dport()moves mostly unmodified fromdrivers/cxl/core/port.c. The only deliberate change is that it now assumes that thedevice_lockis held on entry and the driver is attached (just likecxl_port_probe()).Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>
Subject to carrying fixes from earlier review forwards in the code movement, this looks fine to me.
PATCH v14 21/34 cxl/port: Move dport RAS reporting to a port resource
In reply to Terry BowmanFrom: Dan Williams <[email protected]>
Towards the end goal of making all CXL RAS capability handling uniform across upstream host bridges, upstream switch ports, and upstream endpoint ports, move dport RAS setup to cxl_endpoint_port_probe(). Rename the RAS setup helper to devm_cxl_dport_ras_setup() for symmetry with devm_cxl_switch_port_decoders_setup().
---
Changes in v13 -> v14:
- New patch
--- drivers/cxl/core/ras.c | 12 ++++++------ drivers/cxl/cxlpci.h | 8 ++++---- drivers/cxl/mem.c | 2 -- drivers/cxl/port.c | 12 ++++++++++++ tools/testing/cxl/Kbuild | 2 +- tools/testing/cxl/test/mock.c | 6 +++--- 6 files changed, 26 insertions(+), 16 deletions(-)
drivers/cxl/core/ras.c +6 -6
@@ -139,17 +139,17 @@ static void cxl_dport_map_ras(struct cxl_dport *dport) }/** -* cxl_dport_init_ras_reporting - Setup CXL RAS report on this dport +* devm_cxl_dport_ras_setup - Setup CXL RAS report on this dport * @dport: the cxl_dport that needs to be initialized -* @host: host device for devm operations */ -void cxl_dport_init_ras_reporting (struct cxl_dport *dport,struct device *host) +void devm_cxl_dport_ras_setup (struct cxl_dport *dport) { - dport->reg_map.host = host; + dport->reg_map.host = &dport->port->dev;cxl_dport_map_ras (dport);if (dport->rch) { -struct pci_host_bridge *host_bridge =to_pci_host_bridge (dport->dport_dev); +struct pci_host_bridge *host_bridge = +to_pci_host_bridge (dport->dport_dev);if (!host_bridge->native_aer)return ;
@@ -158,7 +158,7 @@ void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host)cxl_disable_rch_root_ints (dport); } } -EXPORT_SYMBOL_NS_GPL (cxl_dport_init_ras_reporting,"CXL" ); +EXPORT_SYMBOL_NS_GPL (devm_cxl_dport_ras_setup,"CXL" );void cxl_handle_cor_ras (struct device *dev,void __iomem *ras_base) {
drivers/cxl/cxlpci.h +4 -4
@@ -81,7 +81,7 @@ void read_cdat_data(struct cxl_port *port);void cxl_cor_error_detected (struct pci_dev *pdev);pci_ers_result_t cxl_error_detected (struct pci_dev *pdev,pci_channel_state_t state); -void cxl_dport_init_ras_reporting (struct cxl_dport *dport,struct device *host); +void devm_cxl_dport_ras_setup (struct cxl_dport *dport);#else static inline void cxl_cor_error_detected (struct pci_dev *pdev) { }
@@ -90,9 +90,9 @@ static inline pci_ers_result_t cxl_error_detected(struct pci_dev *pdev, {return PCI_ERS_RESULT_NONE ; } - -static inline void cxl_dport_init_ras_reporting (struct cxl_dport *dport, -struct device *host) { } +static inline void devm_cxl_dport_ras_setup (struct cxl_dport *dport) +{ +}#endif #endif /* __CXL_PCI_H__ */
drivers/cxl/mem.c +0 -2
@@ -166,8 +166,6 @@ static int cxl_mem_probe(struct device *dev)else endpoint_parent = &parent_port->dev; -cxl_dport_init_ras_reporting (dport, dev); - scoped_guard(device, endpoint_parent) {if (!endpoint_parent->driver) { dev_err(dev, "CXL port topology %s not enabled\n",
drivers/cxl/port.c +12 -0
@@ -75,6 +75,7 @@ static int cxl_switch_port_probe(struct cxl_port *port)static int cxl_endpoint_port_probe (struct cxl_port *port) {struct cxl_memdev *cxlmd =to_cxl_memdev (port->uport_dev); +struct cxl_dport *dport = port->parent_dport;int rc;/* Cache the data early to ensure is_visible() works */
@@ -90,6 +91,17 @@ static int cxl_endpoint_port_probe(struct cxl_port *port)if (rc)return rc; +/* +* With VH (CXL Virtual Host) topology the cxl_port::add_dport() method +* handles RAS setup for downstream ports. With RCH (CXL Restricted CXL +* Host) topologies the downstream port is enumerated early by platform +* firmware, but the RCRB (root complex register block) is not mapped +* until after the cxl_pci driver attaches to the RCIeP (root complex +* integrated endpoint). +*/ +if (dport->rch) +devm_cxl_dport_ras_setup (dport); +/* * Now that all endpoint decoders are successfully enumerated, try to * assemble regions from committed decoders
tools/testing/cxl/Kbuild +1 -1
@@ -8,7 +8,7 @@ ldflags-y += --wrap=cxl_await_media_ready ldflags-y += --wrap=cxl_add_rch_dport ldflags-y += --wrap=cxl_rcd_component_reg_phys ldflags-y += --wrap=cxl_endpoint_parse_cdat -ldflags-y += --wrap=cxl_dport_init_ras_reporting +ldflags-y += --wrap=devm_cxl_dport_ras_setup ldflags-y += --wrap=devm_cxl_endpoint_decoders_setup ldflags-y += --wrap=hmat_get_extended_linear_cache_size ldflags-y += --wrap=cxl_add_dport_by_dev
tools/testing/cxl/test/mock.c +3 -3
@@ -234,17 +234,17 @@ void __wrap_cxl_endpoint_parse_cdat(struct cxl_port *port) }EXPORT_SYMBOL_NS_GPL (__wrap_cxl_endpoint_parse_cdat,"CXL" ); -void __wrap_cxl_dport_init_ras_reporting (struct cxl_dport *dport,struct device *host) +void __wrap_devm_cxl_dport_ras_setup (struct cxl_dport *dport) {int index;struct cxl_mock_ops *ops =get_cxl_mock_ops (&index);if (!ops || !ops->is_mock_port (dport->dport_dev)) -cxl_dport_init_ras_reporting (dport, host); +devm_cxl_dport_ras_setup (dport);put_cxl_mock_ops (index); } -EXPORT_SYMBOL_NS_GPL (__wrap_cxl_dport_init_ras_reporting,"CXL" ); +EXPORT_SYMBOL_NS_GPL (__wrap_devm_cxl_dport_ras_setup,"CXL" );struct cxl_dport *__wrap_cxl_add_dport_by_dev (struct cxl_port *port,struct device *dport_dev)
Signature · 1 line
2.34.1
Re: [PATCH v14 21/34] cxl/port: Move dport RAS reporting to a port resource
Quoted 10 lines · Terry Bowman
From: Dan Williams <[email protected]>
Towards the end goal of making all CXL RAS capability handling uniform across upstream host bridges, upstream switch ports, and upstream endpoint ports, move dport RAS setup to
cxl_endpoint_port_probe(). Rename the RAS setup helper todevm_cxl_dport_ras_setup()for symmetry withdevm_cxl_switch_port_decoders_setup().Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>
missing sign off tag
Quoted 157 lines
---
Changes in v13 -> v14:
- New patch--- drivers/cxl/core/ras.c | 12 ++++++------ drivers/cxl/cxlpci.h | 8 ++++---- drivers/cxl/mem.c | 2 -- drivers/cxl/port.c | 12 ++++++++++++ tools/testing/cxl/Kbuild | 2 +- tools/testing/cxl/test/mock.c | 6 +++--- 6 files changed, 26 insertions(+), 16 deletions(-)drivers/cxl/core/ras.c +6
-6@@ -139,17 +139,17 @@ static void cxl_dport_map_ras(struct cxl_dport *dport) }/** -* cxl_dport_init_ras_reporting - Setup CXL RAS report on this dport +* devm_cxl_dport_ras_setup - Setup CXL RAS report on this dport * @dport: the cxl_dport that needs to be initialized -* @host: host device for devm operations */ -void cxl_dport_init_ras_reporting (struct cxl_dport *dport,struct device *host) +void devm_cxl_dport_ras_setup (struct cxl_dport *dport) { - dport->reg_map.host = host; + dport->reg_map.host = &dport->port->dev;cxl_dport_map_ras (dport);if (dport->rch) { -struct pci_host_bridge *host_bridge =to_pci_host_bridge (dport->dport_dev); +struct pci_host_bridge *host_bridge = +to_pci_host_bridge (dport->dport_dev);if (!host_bridge->native_aer)return ;@@ -158,7 +158,7 @@ void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host)cxl_disable_rch_root_ints (dport); } } -EXPORT_SYMBOL_NS_GPL (cxl_dport_init_ras_reporting,"CXL" ); +EXPORT_SYMBOL_NS_GPL (devm_cxl_dport_ras_setup,"CXL" );void cxl_handle_cor_ras (struct device *dev,void __iomem *ras_base) {drivers/cxl/cxlpci.h +4
-4@@ -81,7 +81,7 @@ void read_cdat_data(struct cxl_port *port);void cxl_cor_error_detected (struct pci_dev *pdev);pci_ers_result_t cxl_error_detected (struct pci_dev *pdev,pci_channel_state_t state); -void cxl_dport_init_ras_reporting (struct cxl_dport *dport,struct device *host); +void devm_cxl_dport_ras_setup (struct cxl_dport *dport);#else static inline void cxl_cor_error_detected (struct pci_dev *pdev) { }@@ -90,9 +90,9 @@ static inline pci_ers_result_t cxl_error_detected(struct pci_dev *pdev, {return PCI_ERS_RESULT_NONE ; } - -static inline void cxl_dport_init_ras_reporting (struct cxl_dport *dport, -struct device *host) { } +static inline void devm_cxl_dport_ras_setup (struct cxl_dport *dport) +{ +}#endif #endif /* __CXL_PCI_H__ */ drivers/cxl/mem.c +0
-2@@ -166,8 +166,6 @@ static int cxl_mem_probe(struct device *dev)else endpoint_parent = &parent_port->dev; -cxl_dport_init_ras_reporting (dport, dev); - scoped_guard(device, endpoint_parent) {if (!endpoint_parent->driver) { dev_err(dev, "CXL port topology %s not enabled\n",drivers/cxl/port.c +12
-0@@ -75,6 +75,7 @@ static int cxl_switch_port_probe(struct cxl_port *port)static int cxl_endpoint_port_probe (struct cxl_port *port) {struct cxl_memdev *cxlmd =to_cxl_memdev (port->uport_dev); +struct cxl_dport *dport = port->parent_dport;int rc;/* Cache the data early to ensure is_visible() works */ @@ -90,6 +91,17 @@ static int cxl_endpoint_port_probe(struct cxl_port *port)if (rc)return rc; +/* +* With VH (CXL Virtual Host) topology the cxl_port::add_dport() method +* handles RAS setup for downstream ports. With RCH (CXL Restricted CXL +* Host) topologies the downstream port is enumerated early by platform +* firmware, but the RCRB (root complex register block) is not mapped +* until after the cxl_pci driver attaches to the RCIeP (root complex +* integrated endpoint). +*/ +if (dport->rch) +devm_cxl_dport_ras_setup (dport); +/* * Now that all endpoint decoders are successfully enumerated, try to * assemble regions from committed decoders tools/testing/cxl/Kbuild +1
-1@@ -8,7 +8,7 @@ ldflags-y += --wrap=cxl_await_media_ready ldflags-y += --wrap=cxl_add_rch_dport ldflags-y += --wrap=cxl_rcd_component_reg_phys ldflags-y += --wrap=cxl_endpoint_parse_cdat -ldflags-y += --wrap=cxl_dport_init_ras_reporting +ldflags-y += --wrap=devm_cxl_dport_ras_setup ldflags-y += --wrap=devm_cxl_endpoint_decoders_setup ldflags-y += --wrap=hmat_get_extended_linear_cache_size ldflags-y += --wrap=cxl_add_dport_by_devtools/testing/cxl/test/mock.c +3
-3@@ -234,17 +234,17 @@ void __wrap_cxl_endpoint_parse_cdat(struct cxl_port *port) }EXPORT_SYMBOL_NS_GPL (__wrap_cxl_endpoint_parse_cdat,"CXL" ); -void __wrap_cxl_dport_init_ras_reporting (struct cxl_dport *dport,struct device *host) +void __wrap_devm_cxl_dport_ras_setup (struct cxl_dport *dport) {int index;struct cxl_mock_ops *ops =get_cxl_mock_ops (&index);if (!ops || !ops->is_mock_port (dport->dport_dev)) -cxl_dport_init_ras_reporting (dport, host); +devm_cxl_dport_ras_setup (dport);put_cxl_mock_ops (index); } -EXPORT_SYMBOL_NS_GPL (__wrap_cxl_dport_init_ras_reporting,"CXL" ); +EXPORT_SYMBOL_NS_GPL (__wrap_devm_cxl_dport_ras_setup,"CXL" );struct cxl_dport *__wrap_cxl_add_dport_by_dev (struct cxl_port *port,struct device *dport_dev)
Re: [PATCH v14 21/34] cxl/port: Move dport RAS reporting to a port resource
In reply to Terry BowmanQuoted 10 lines · On Wed, 14 Jan 2026 12:20:42 -0600 Terry Bowman <[email protected]> wrote:
From: Dan Williams <[email protected]>
Towards the end goal of making all CXL RAS capability handling uniform across upstream host bridges, upstream switch ports, and upstream endpoint ports, move dport RAS setup to
cxl_endpoint_port_probe(). Rename the RAS setup helper todevm_cxl_dport_ras_setup()for symmetry withdevm_cxl_switch_port_decoders_setup().Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>
One trivial thing inline.
Quoted 19 lines
---
Changes in v13 -> v14:
- New patch--- drivers/cxl/core/ras.c | 12 ++++++------ drivers/cxl/cxlpci.h | 8 ++++---- drivers/cxl/mem.c | 2 -- drivers/cxl/port.c | 12 ++++++++++++ tools/testing/cxl/Kbuild | 2 +- tools/testing/cxl/test/mock.c | 6 +++--- 6 files changed, 26 insertions(+), 16 deletions(-)drivers/cxl/core/ras.c +0
-0
drivers/cxl/core/ras.c static void cxl_dport_map_ras(struct cxl_dport *dport) · line 145
Quoted 10 lines
drivers/cxl/core/ras.c +2
-3@@ -139,17 +139,17 @@ static void cxl_dport_map_ras(struct cxl_dport *dport) }/** -* cxl_dport_init_ras_reporting - Setup CXL RAS report on this dport +* devm_cxl_dport_ras_setup - Setup CXL RAS report on this dport * @dport: the cxl_dport that needs to be initialized -* @host: host device for devm operations */ -void cxl_dport_init_ras_reporting (struct cxl_dport *dport,struct device *host) +void devm_cxl_dport_ras_setup (struct cxl_dport *dport)
Not a thing for this patch set, but might be nice to at somepoint prefix all the functions that have devm_ registrations in them so it is obvious where those are hiding. I had to dig down a few levels to find the call.
drivers/cxl/core/ras.c static void cxl_dport_map_ras(struct cxl_dport *dport) · line 152
Quoted 9 lines
drivers/cxl/core/ras.c +3
-2{ - dport->reg_map.host = host; + dport->reg_map.host = &dport->port->dev;cxl_dport_map_ras (dport);if (dport->rch) { -struct pci_host_bridge *host_bridge =to_pci_host_bridge (dport->dport_dev); +struct pci_host_bridge *host_bridge = +to_pci_host_bridge (dport->dport_dev);
Unrelated change. This series is complex, so this sort of noise is not helpful to reviewability!
Quoted 12 lines
drivers/cxl/core/ras.c +1
-1if (!host_bridge->native_aer)return ;@@ -158,7 +158,7 @@ void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host)cxl_disable_rch_root_ints (dport); } } -EXPORT_SYMBOL_NS_GPL (cxl_dport_init_ras_reporting,"CXL" ); +EXPORT_SYMBOL_NS_GPL (devm_cxl_dport_ras_setup,"CXL" );void cxl_handle_cor_ras (struct device *dev,void __iomem *ras_base) {
PATCH v14 22/34 cxl: Update CXL Endpoint tracing
In reply to Terry BowmanCXL protocol error handling will be expanded to soon include CXL Port support along with existing Endpoint support. 2 updates are needed first:
- Update calling interfaces to use 'struct device*'
- Log serial number
Add serial number parameter to the trace logging. This is used for EPs and 0 is provided for CXL port devices without a serial number.
Leave the correctable and uncorrectable trace routines' TP_STRUCT__entry() unchanged with respect to member data types and order.
Below is output of correctable and uncorrectable protocol error logging. CXL Root Port and CXL Endpoint examples are included below.
Root Port:cxl_aer_correctable_error: device=0000:0c:00.0 host=pci0000:0c serial: 0 status='CRC Threshold Hit'cxl_aer_uncorrectable_error: device=0000:0c:00.0 host=pci0000:0c serial: 0 status: 'Cache Byte Enable Parity Error' first_error: 'Cache Byte Enable Parity Error'
Endpoint:cxl_aer_correctable_error: memdev=mem3 host=0000:0f:00.0 serial=0 status='CRC Threshold Hit'cxl_aer_uncorrectable_error: memdev=mem3 host=0000:0f:00.0 serial: 0 status: 'Cache Byte Enable Parity Error' first_error: 'Cache Byte Enable Parity Error'
---
Changes in v13->v14:
- Update commit headline (Bjorn)
Changes in v12->v13:
- Added Dave Jiang's review-by
Changes in v11 -> v12:
- Correct parameters to call trace_cxl_aer_correctable_error()
- Add reviewed-by for Jonathan and Shiju
Changes in v10->v11:
- Updated CE and UCE trace routines to maintain consistent TP_Struct ABI and unchanged TP_printk() logging.
--- drivers/cxl/core/core.h | 4 ++-- drivers/cxl/core/ras.c | 35 ++++++++++++++++++++--------------- drivers/cxl/core/ras_rch.c | 4 ++-- drivers/cxl/core/trace.h | 25 +++++++++++++------------ 4 files changed, 37 insertions(+), 31 deletions(-)
drivers/cxl/core/core.h +2 -2
@@ -147,8 +147,8 @@ int cxl_port_get_switch_dport_bandwidth(struct cxl_port *port,#ifdef CONFIG_CXL_RAS int cxl_ras_init (void );void cxl_ras_exit (void ); -bool cxl_handle_ras (struct device *dev,void __iomem *ras_base); -void cxl_handle_cor_ras (struct device *dev,void __iomem *ras_base); +bool cxl_handle_ras (struct device *dev,u64 serial,void __iomem *ras_base); +void cxl_handle_cor_ras (struct device *dev,u64 serial,void __iomem *ras_base);void cxl_dport_map_rch_aer (struct cxl_dport *dport);void cxl_disable_rch_root_ints (struct cxl_dport *dport);void cxl_handle_rdport_errors (struct cxl_dev_state *cxlds);
drivers/cxl/core/ras.c +20 -15
@@ -13,7 +13,7 @@ static void cxl_cper_trace_corr_port_prot_err(struct pci_dev *pdev, {u32 status = ras_cap.cor_status & ~ras_cap.cor_mask; -trace_cxl_port_aer_correctable_error (&pdev->dev, status); +trace_cxl_aer_correctable_error (&pdev->dev, status,0 ); } static void cxl_cper_trace_uncorr_port_prot_err(struct pci_dev *pdev,
@@ -28,8 +28,8 @@ static void cxl_cper_trace_uncorr_port_prot_err(struct pci_dev *pdev,else fe = status; -trace_cxl_port_aer_uncorrectable_error (&pdev->dev, status, fe, - ras_cap.header_log); +trace_cxl_aer_uncorrectable_error (&pdev->dev, status, fe, + ras_cap.header_log,0 ); } static void cxl_cper_trace_corr_prot_err(struct cxl_memdev *cxlmd,
@@ -37,7 +37,7 @@ static void cxl_cper_trace_corr_prot_err(struct cxl_memdev *cxlmd, {u32 status = ras_cap.cor_status & ~ras_cap.cor_mask; -trace_cxl_aer_correctable_error (cxlmd, status); +trace_cxl_aer_correctable_error (&cxlmd->dev, status, cxlmd->cxlds->serial); } static void
@@ -45,6 +45,7 @@ cxl_cper_trace_uncorr_prot_err(struct cxl_memdev *cxlmd, struct cxl_ras_capability_regs ras_cap) {u32 status = ras_cap.uncor_status & ~ras_cap.uncor_mask; +struct cxl_dev_state *cxlds = cxlmd->cxlds;u32 fe;if (hweight32 (status) >1 )
@@ -53,8 +54,9 @@ cxl_cper_trace_uncorr_prot_err(struct cxl_memdev *cxlmd,else fe = status; -trace_cxl_aer_uncorrectable_error (cxlmd, status, fe, - ras_cap.header_log); +trace_cxl_aer_uncorrectable_error (&cxlmd->dev, status, fe, + ras_cap.header_log, + cxlds->serial); }static int match_memdev_by_parent (struct device *dev,const void *uport)
@@ -160,7 +162,7 @@ void devm_cxl_dport_ras_setup(struct cxl_dport *dport) }EXPORT_SYMBOL_NS_GPL (devm_cxl_dport_ras_setup,"CXL" ); -void cxl_handle_cor_ras (struct device *dev,void __iomem *ras_base) +void cxl_handle_cor_ras (struct device *dev,u64 serial,void __iomem *ras_base) {void __iomem *addr;u32 status;
@@ -170,10 +172,11 @@ void cxl_handle_cor_ras(struct device *dev, void __iomem *ras_base) addr = ras_base +CXL_RAS_CORRECTABLE_STATUS_OFFSET ; status =readl (addr); -if (status &CXL_RAS_CORRECTABLE_STATUS_MASK ) { -writel (status &CXL_RAS_CORRECTABLE_STATUS_MASK , addr); -trace_cxl_aer_correctable_error (to_cxl_memdev (dev), status); - } +if (!(status &CXL_RAS_CORRECTABLE_STATUS_MASK )) +return ; +writel (status &CXL_RAS_CORRECTABLE_STATUS_MASK , addr); + +trace_cxl_aer_correctable_error (dev, status, serial); }/* CXL spec rev3.0 8.2.4.16.1 */
@@ -197,7 +200,7 @@ static void header_log_copy(void __iomem *ras_base, u32 *log)* Log the state of the RAS status registers and prepare them to log the * next error status. Return 1 if reset needed. */ -bool cxl_handle_ras (struct device *dev,void __iomem *ras_base) +bool cxl_handle_ras (struct device *dev,u64 serial,void __iomem *ras_base) {u32 hl[CXL_HEADERLOG_SIZE_U32 ];void __iomem *addr;
@@ -224,7 +227,7 @@ bool cxl_handle_ras(struct device *dev, void __iomem *ras_base) }header_log_copy (ras_base, hl); -trace_cxl_aer_uncorrectable_error (to_cxl_memdev (dev), status, fe, hl); +trace_cxl_aer_uncorrectable_error (dev, status, fe, hl, serial);writel (status &CXL_RAS_UNCORRECTABLE_STATUS_MASK , addr);return true;
@@ -246,7 +249,8 @@ void cxl_cor_error_detected(struct pci_dev *pdev)if (cxlds->rcd)cxl_handle_rdport_errors (cxlds); -cxl_handle_cor_ras (&cxlds->cxlmd->dev, cxlds->regs.ras); +cxl_handle_cor_ras (&cxlds->cxlmd->dev, cxlds->serial, + cxlds->regs.ras); } }EXPORT_SYMBOL_NS_GPL (cxl_cor_error_detected,"CXL" );
@@ -275,7 +279,8 @@ pci_ers_result_t cxl_error_detected(struct pci_dev *pdev,* chance the situation is recoverable dump the status of the RAS * capability registers and bounce the active state of the memdev. */ - ue =cxl_handle_ras (&cxlds->cxlmd->dev, cxlds->regs.ras); + ue =cxl_handle_ras (&cxlds->cxlmd->dev, cxlds->serial, + cxlds->regs.ras); }
drivers/cxl/core/ras_rch.c +2 -2
@@ -115,7 +115,7 @@ void cxl_handle_rdport_errors(struct cxl_dev_state *cxlds)pci_print_aer (pdev, severity, &aer_regs);if (severity ==AER_CORRECTABLE ) -cxl_handle_cor_ras (&cxlds->cxlmd->dev, dport->regs.ras); +cxl_handle_cor_ras (&cxlds->cxlmd->dev, cxlds->serial, dport->regs.ras);else -cxl_handle_ras (&cxlds->cxlmd->dev, dport->regs.ras); +cxl_handle_ras (&cxlds->cxlmd->dev, cxlds->serial, dport->regs.ras); }
drivers/cxl/core/trace.h +13 -12
@@ -77,11 +77,12 @@ TRACE_EVENT(cxl_port_aer_uncorrectable_error, ); TRACE_EVENT(cxl_aer_uncorrectable_error, - TP_PROTO(const struct cxl_memdev *cxlmd, u32 status, u32 fe, u32 *hl), - TP_ARGS(cxlmd, status, fe, hl), + TP_PROTO(const struct device *cxlmd, u32 status, u32 fe, u32 *hl, + u64 serial), + TP_ARGS(cxlmd, status, fe, hl, serial), TP_STRUCT__entry( - __string(memdev, dev_name(&cxlmd->dev)) - __string(host, dev_name(cxlmd->dev.parent)) + __string(memdev, dev_name(cxlmd)) + __string(host, dev_name(cxlmd->parent)) __field(u64, serial) __field(u32, status) __field(u32, first_error)
@@ -90,7 +91,7 @@ TRACE_EVENT(cxl_aer_uncorrectable_error,TP_fast_assign ( __assign_str(memdev);__assign_str (host); - __entry->serial = cxlmd->cxlds->serial; + __entry->serial = serial; __entry->status = status; __entry->first_error = fe;/*
@@ -138,24 +139,24 @@ TRACE_EVENT(cxl_port_aer_correctable_error, __entry->status = status; ),TP_printk ("device=%s host=%s status='%s'" , -__get_str (device),__get_str (host), -show_ce_errs (__entry->status) +__get_str (device),__get_str (host), +show_ce_errs (__entry->status) ) ); TRACE_EVENT(cxl_aer_correctable_error, - TP_PROTO(const struct cxl_memdev *cxlmd, u32 status), - TP_ARGS(cxlmd, status), + TP_PROTO(const struct device *cxlmd, u32 status, u64 serial), + TP_ARGS(cxlmd, status, serial), TP_STRUCT__entry( - __string(memdev, dev_name(&cxlmd->dev)) - __string(host, dev_name(cxlmd->dev.parent)) + __string(memdev, dev_name(cxlmd)) + __string(host, dev_name(cxlmd->parent)) __field(u64, serial) __field(u32, status) ), TP_fast_assign( __assign_str(memdev); __assign_str(host); - __entry->serial = cxlmd->cxlds->serial; + __entry->serial = serial; __entry->status = status; ), TP_printk("memdev=%s host=%s serial=%lld: status: '%s'",
Signature · 1 line
2.34.1
PATCH v14 23/34 cxl: Map CXL Endpoint Port and CXL Switch Port RAS registers
In reply to Terry BowmanIn preparation for CXL VH (Virtual Host) topology protocol error handling, add RAS capability registered mapping for all ports in a CXL VH topology. This includes the RAS capabilities of Switch Upstream Ports, Switch Downstream Ports, Host Bridge Ports ("upstream"), and Root Ports ("downstream")
Update cxl_port_add_dport() to map the upstream RAS capability on first 'dport' attach, and downstream RAS capability on each 'dport' attach. Arrange for dport mappings to be released at del_dport() time.
---
Changes in v13->v14:
- Correct message spelling (Terry)
--- drivers/cxl/core/port.c | 2 +- drivers/cxl/core/ras.c | 11 +++++++++++ drivers/cxl/cxl.h | 2 ++ drivers/cxl/cxlpci.h | 4 ++++ drivers/cxl/port.c | 37 +++++++++++++++++++++++++++++++++++ tools/testing/cxl/Kbuild | 1 + tools/testing/cxl/test/mock.c | 12 ++++++++++++ 7 files changed, 68 insertions(+), 1 deletion(-)
drivers/cxl/core/port.c +1 -1
@@ -1451,7 +1451,7 @@ static void del_dport(struct cxl_dport *dport) {struct cxl_port *port = dport->port; -devm_release_action (&port->dev, unlink_dport, dport); +devres_release_group (&port->dev, dport); }static void del_dports (struct cxl_port *port)
drivers/cxl/core/ras.c +11 -0
@@ -162,6 +162,17 @@ void devm_cxl_dport_ras_setup(struct cxl_dport *dport) }EXPORT_SYMBOL_NS_GPL (devm_cxl_dport_ras_setup,"CXL" ); +void devm_cxl_port_ras_setup (struct cxl_port *port) +{ +struct cxl_register_map *map = &port->reg_map; + + map->host = &port->dev; +if (cxl_map_component_regs (map, &port->regs, +BIT (CXL_CM_CAP_CAP_ID_RAS ))) +dev_dbg (&port->dev,"Failed to map RAS capability\n" ); +} +EXPORT_SYMBOL_NS_GPL (devm_cxl_port_ras_setup,"CXL" ); +void cxl_handle_cor_ras (struct device *dev,u64 serial,void __iomem *ras_base) {void __iomem *addr;
drivers/cxl/cxl.h +2 -0
@@ -607,6 +607,7 @@ struct cxl_dax_region {* @parent_dport: dport that points to this port in the parent * @decoder_ida: allocator for decoder ids * @reg_map: component and ras register mapping parameters +* @regs: mapped component registers * @nr_dports: number of entries in @dports * @hdm_end: track last allocated HDM decoder instance for allocation ordering * @commit_end: cursor to track highest committed decoder for commit ordering
@@ -628,6 +629,7 @@ struct cxl_port {struct cxl_dport *parent_dport;struct ida decoder_ida;struct cxl_register_map reg_map; +struct cxl_component_regs regs;int nr_dports;int hdm_end;int commit_end;
drivers/cxl/cxlpci.h +4 -0
@@ -82,6 +82,7 @@ void cxl_cor_error_detected(struct pci_dev *pdev);pci_ers_result_t cxl_error_detected (struct pci_dev *pdev,pci_channel_state_t state);void devm_cxl_dport_ras_setup (struct cxl_dport *dport); +void devm_cxl_port_ras_setup (struct cxl_port *port);#else static inline void cxl_cor_error_detected (struct pci_dev *pdev) { }
@@ -93,6 +94,9 @@ static inline pci_ers_result_t cxl_error_detected(struct pci_dev *pdev,static inline void devm_cxl_dport_ras_setup (struct cxl_dport *dport) { } +static inline void devm_cxl_port_ras_setup (struct cxl_port *port) +{ +}#endif #endif /* __CXL_PCI_H__ */
drivers/cxl/port.c +37 -0
@@ -176,11 +176,29 @@ static struct cxl_port *cxl_port_devres_group(struct cxl_port *port) DEFINE_FREE(cxl_port_group_free, struct cxl_port *, if (!IS_ERR_OR_NULL(_T)) devres_release_group(&(_T)->dev, _T)) +static struct cxl_dport *cxl_dport_devres_group (struct cxl_dport *dport) +{ +if (!devres_open_group (&dport->port->dev, dport,GFP_KERNEL )) +return ERR_PTR (-ENOMEM ); +return dport; +} +DEFINE_FREE(cxl_dport_group_free, struct cxl_dport *, + if (!IS_ERR_OR_NULL(_T)) devres_release_group(&(_T)->port->dev, _T)) +static void cxl_port_group_close (struct cxl_port *port) {devres_remove_group (&port->dev, port); } +/* +* Unlike the port group, that just facilitates unwind of setup failures, the +* dport group needs to stay live for del_dport() to reference. +*/ +static void cxl_dport_group_close (struct cxl_dport *dport) +{ +devres_close_group (&dport->port->dev, dport); +} +static struct cxl_dport *cxl_port_add_dport (struct cxl_port *port,struct device *dport_dev) {
@@ -209,6 +227,13 @@ static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port, rc =devm_cxl_switch_port_decoders_setup (port);if (rc)return ERR_PTR (rc); + +/* +* RAS setup is optional, either driver operation can continue +* on failure, or the device does not implement RAS registers. +*/ +devm_cxl_port_ras_setup (port); +/* * Note, when nr_dports returns to zero the port is unregistered * and triggers cleanup. I.e. no need for open-coded release
@@ -220,12 +245,24 @@ static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port,if (IS_ERR (new_dport))return new_dport; +/* +* Establish a group for all dport resources that need to be released +* when the dport is deleted. +*/ + struct cxl_dport *dport_group __free(cxl_dport_group_free) = +cxl_dport_devres_group (new_dport); +if (IS_ERR (dport_group)) +return ERR_CAST (dport_group); + rc =cxl_dport_autoremove (new_dport);if (rc)return ERR_PTR (rc); +devm_cxl_dport_ras_setup (new_dport); +cxl_switch_parse_cdat (new_dport); +cxl_dport_group_close (no_free_ptr (dport_group));cxl_port_group_close (no_free_ptr (port_group)); dev_dbg(&port->dev, "dport[%d] id:%d dport_dev: %s added\n",
tools/testing/cxl/Kbuild +1 -0
@@ -13,6 +13,7 @@ ldflags-y += --wrap=devm_cxl_endpoint_decoders_setup ldflags-y += --wrap=hmat_get_extended_linear_cache_size ldflags-y += --wrap=cxl_add_dport_by_dev ldflags-y += --wrap=devm_cxl_switch_port_decoders_setup +ldflags-y += --wrap=devm_cxl_port_ras_setup DRIVERS := ../../../drivers CXL_SRC := $(DRIVERS)/cxl
tools/testing/cxl/test/mock.c +12 -0
@@ -246,6 +246,18 @@ void __wrap_devm_cxl_dport_ras_setup(struct cxl_dport *dport) }EXPORT_SYMBOL_NS_GPL (__wrap_devm_cxl_dport_ras_setup,"CXL" ); +void __wrap_devm_cxl_port_ras_setup (struct cxl_port *port) +{ +int index; +struct cxl_mock_ops *ops =get_cxl_mock_ops (&index); + +if (!ops || !ops->is_mock_port (port->uport_dev)) +devm_cxl_port_ras_setup (port); + +put_cxl_mock_ops (index); +} +EXPORT_SYMBOL_NS_GPL (__wrap_devm_cxl_port_ras_setup,"CXL" ); +struct cxl_dport *__wrap_cxl_add_dport_by_dev (struct cxl_port *port,struct device *dport_dev) {
Signature · 1 line
2.34.1
Dave Jiang
maintainerRe: [PATCH v14 23/34] cxl: Map CXL Endpoint Port and CXL Switch Port RAS registers
Quoted 6 lines · Terry Bowman
In preparation for CXL VH (Virtual Host) topology protocol error handling, add RAS capability registered mapping for all ports in a CXL VH topology. This includes the RAS capabilities of Switch Upstream Ports, Switch Downstream Ports, Host Bridge Ports ("upstream"), and Root Ports ("downstream")
Quoted 8 lines
Update
cxl_port_add_dport()to map the upstream RAS capability on first 'dport' attach, and downstream RAS capability on each 'dport' attach. Arrange for dport mappings to be released atdel_dport()time.Signed-off-by Terry Bowman <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Dave Jiang <[email protected]>[djbw: reword changelog, fix devm handling]
drop the line above
DJ
Quoted 202 lines
Co-developed-by Dan Williams <[email protected]>Signed-off-by Dan Williams <[email protected]>---
Changes in
v13->v14:
- Correct message spelling (Terry)--- drivers/cxl/core/port.c | 2 +- drivers/cxl/core/ras.c | 11 +++++++++++ drivers/cxl/cxl.h | 2 ++ drivers/cxl/cxlpci.h | 4 ++++ drivers/cxl/port.c | 37 +++++++++++++++++++++++++++++++++++ tools/testing/cxl/Kbuild | 1 + tools/testing/cxl/test/mock.c | 12 ++++++++++++ 7 files changed, 68 insertions(+), 1 deletion(-)drivers/cxl/core/port.c +1
-1@@ -1451,7 +1451,7 @@ static void del_dport(struct cxl_dport *dport) {struct cxl_port *port = dport->port; -devm_release_action (&port->dev, unlink_dport, dport); +devres_release_group (&port->dev, dport); }static void del_dports (struct cxl_port *port)drivers/cxl/core/ras.c +11
-0@@ -162,6 +162,17 @@ void devm_cxl_dport_ras_setup(struct cxl_dport *dport) }EXPORT_SYMBOL_NS_GPL (devm_cxl_dport_ras_setup,"CXL" ); +void devm_cxl_port_ras_setup (struct cxl_port *port) +{ +struct cxl_register_map *map = &port->reg_map; + + map->host = &port->dev; +if (cxl_map_component_regs (map, &port->regs, +BIT (CXL_CM_CAP_CAP_ID_RAS ))) +dev_dbg (&port->dev,"Failed to map RAS capability\n" ); +} +EXPORT_SYMBOL_NS_GPL (devm_cxl_port_ras_setup,"CXL" ); +void cxl_handle_cor_ras (struct device *dev,u64 serial,void __iomem *ras_base) {void __iomem *addr;drivers/cxl/cxl.h +2
-0@@ -607,6 +607,7 @@ struct cxl_dax_region {* @parent_dport: dport that points to this port in the parent * @decoder_ida: allocator for decoder ids * @reg_map: component and ras register mapping parameters +* @regs: mapped component registers * @nr_dports: number of entries in @dports * @hdm_end: track last allocated HDM decoder instance for allocation ordering * @commit_end: cursor to track highest committed decoder for commit ordering @@ -628,6 +629,7 @@ struct cxl_port {struct cxl_dport *parent_dport;struct ida decoder_ida;struct cxl_register_map reg_map; +struct cxl_component_regs regs;int nr_dports;int hdm_end;int commit_end;drivers/cxl/cxlpci.h +4
-0@@ -82,6 +82,7 @@ void cxl_cor_error_detected(struct pci_dev *pdev);pci_ers_result_t cxl_error_detected (struct pci_dev *pdev,pci_channel_state_t state);void devm_cxl_dport_ras_setup (struct cxl_dport *dport); +void devm_cxl_port_ras_setup (struct cxl_port *port);#else static inline void cxl_cor_error_detected (struct pci_dev *pdev) { }@@ -93,6 +94,9 @@ static inline pci_ers_result_t cxl_error_detected(struct pci_dev *pdev,static inline void devm_cxl_dport_ras_setup (struct cxl_dport *dport) { } +static inline void devm_cxl_port_ras_setup (struct cxl_port *port) +{ +}#endif #endif /* __CXL_PCI_H__ */ drivers/cxl/port.c +37
-0@@ -176,11 +176,29 @@ static struct cxl_port *cxl_port_devres_group(struct cxl_port *port) DEFINE_FREE(cxl_port_group_free, struct cxl_port *, if (!IS_ERR_OR_NULL(_T)) devres_release_group(&(_T)->dev, _T)) +static struct cxl_dport *cxl_dport_devres_group (struct cxl_dport *dport) +{ +if (!devres_open_group (&dport->port->dev, dport,GFP_KERNEL )) +return ERR_PTR (-ENOMEM ); +return dport; +} +DEFINE_FREE(cxl_dport_group_free, struct cxl_dport *, + if (!IS_ERR_OR_NULL(_T)) devres_release_group(&(_T)->port->dev, _T)) +static void cxl_port_group_close (struct cxl_port *port) {devres_remove_group (&port->dev, port); } +/* +* Unlike the port group, that just facilitates unwind of setup failures, the +* dport group needs to stay live for del_dport() to reference. +*/ +static void cxl_dport_group_close (struct cxl_dport *dport) +{ +devres_close_group (&dport->port->dev, dport); +} +static struct cxl_dport *cxl_port_add_dport (struct cxl_port *port,struct device *dport_dev) {@@ -209,6 +227,13 @@ static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port, rc =devm_cxl_switch_port_decoders_setup (port);if (rc)return ERR_PTR (rc); + +/* +* RAS setup is optional, either driver operation can continue +* on failure, or the device does not implement RAS registers. +*/ +devm_cxl_port_ras_setup (port); +/* * Note, when nr_dports returns to zero the port is unregistered * and triggers cleanup. I.e. no need for open-coded release @@ -220,12 +245,24 @@ static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port,if (IS_ERR (new_dport))return new_dport; +/* +* Establish a group for all dport resources that need to be released +* when the dport is deleted. +*/ + struct cxl_dport *dport_group __free(cxl_dport_group_free) = +cxl_dport_devres_group (new_dport); +if (IS_ERR (dport_group)) +return ERR_CAST (dport_group); + rc =cxl_dport_autoremove (new_dport);if (rc)return ERR_PTR (rc); +devm_cxl_dport_ras_setup (new_dport); +cxl_switch_parse_cdat (new_dport); +cxl_dport_group_close (no_free_ptr (dport_group));cxl_port_group_close (no_free_ptr (port_group)); dev_dbg(&port->dev, "dport[%d] id:%d dport_dev: %s added\n",tools/testing/cxl/Kbuild +1
-0@@ -13,6 +13,7 @@ ldflags-y += --wrap=devm_cxl_endpoint_decoders_setup ldflags-y += --wrap=hmat_get_extended_linear_cache_size ldflags-y += --wrap=cxl_add_dport_by_dev ldflags-y += --wrap=devm_cxl_switch_port_decoders_setup +ldflags-y += --wrap=devm_cxl_port_ras_setup DRIVERS := ../../../drivers CXL_SRC := $(DRIVERS)/cxltools/testing/cxl/test/mock.c +12
-0@@ -246,6 +246,18 @@ void __wrap_devm_cxl_dport_ras_setup(struct cxl_dport *dport) }EXPORT_SYMBOL_NS_GPL (__wrap_devm_cxl_dport_ras_setup,"CXL" ); +void __wrap_devm_cxl_port_ras_setup (struct cxl_port *port) +{ +int index; +struct cxl_mock_ops *ops =get_cxl_mock_ops (&index); + +if (!ops || !ops->is_mock_port (port->uport_dev)) +devm_cxl_port_ras_setup (port); + +put_cxl_mock_ops (index); +} +EXPORT_SYMBOL_NS_GPL (__wrap_devm_cxl_port_ras_setup,"CXL" ); +struct cxl_dport *__wrap_cxl_add_dport_by_dev (struct cxl_port *port,struct device *dport_dev) {
Re: [PATCH v14 23/34] cxl: Map CXL Endpoint Port and CXL Switch Port RAS registers
In reply to Terry BowmanQuoted 10 lines · On Wed, 14 Jan 2026 12:20:44 -0600 Terry Bowman <[email protected]> wrote:
In preparation for CXL VH (Virtual Host) topology protocol error handling, add RAS capability registered mapping for all ports in a CXL VH topology. This includes the RAS capabilities of Switch Upstream Ports, Switch Downstream Ports, Host Bridge Ports ("upstream"), and Root Ports ("downstream")
Update
cxl_port_add_dport()to map the upstream RAS capability on first 'dport' attach, and downstream RAS capability on each 'dport' attach. Arrange for dport mappings to be released atdel_dport()time.
Quoted 7 lines
Signed-off-by Terry Bowman <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Dave Jiang <[email protected]>[djbw: reword changelog, fix devm handling]Co-developed-by Dan Williams <[email protected]>Signed-off-by Dan Williams <[email protected]>
One comment inline on which level we handle failures in ras setup at but that's already true so things aren't made worse by this.
I'm not particularly keen on failing to pass errors up to callers of devm_cxl_dport_ras_setup() which could then cleanly ignore them with comments saying why. However, that predates this anyway so a question for another day.
PATCH v14 24/34 cxl/port: Move endpoint component register management to cxl_port
In reply to Terry BowmanFrom: Dan Williams <[email protected]>
In preparation for generic protocol error handling across CXL endpoints, whether they be memory expander class devices or accelerators, drop the endpoint component management from cxl_dev_state.
Organize all CXL port component management through the common cxl_port driver.
Note that the end game is that drivers/cxl/core/ras.c loses all dependencies on a 'struct cxl_dev_state' parameter and operates only on port resources. The removal of component register mapping from cxl_pci is an incremental step towards that.
---
Changes in v13 -> v14:
- New patch
- Update log message for cxl_ras_unmask() failure (Dan)
--- drivers/cxl/core/ras.c | 6 ++-- drivers/cxl/cxlmem.h | 4 +-- drivers/cxl/pci.c | 63 +----------------------------------------- drivers/cxl/port.c | 54 ++++++++++++++++++++++++++++++++++++ 4 files changed, 60 insertions(+), 67 deletions(-)
drivers/cxl/core/ras.c +3 -3
@@ -247,6 +247,7 @@ bool cxl_handle_ras(struct device *dev, u64 serial, void __iomem *ras_base)void cxl_cor_error_detected (struct pci_dev *pdev) {struct cxl_dev_state *cxlds =pci_get_drvdata (pdev); +struct cxl_memdev *cxlmd = cxlds->cxlmd;struct device *dev = &cxlds->cxlmd->dev; scoped_guard(device, dev) {
@@ -261,7 +262,7 @@ void cxl_cor_error_detected(struct pci_dev *pdev)cxl_handle_rdport_errors (cxlds);cxl_handle_cor_ras (&cxlds->cxlmd->dev, cxlds->serial, - cxlds->regs.ras); + cxlmd->endpoint->regs.ras); } }EXPORT_SYMBOL_NS_GPL (cxl_cor_error_detected,"CXL" );
@@ -291,10 +292,9 @@ pci_ers_result_t cxl_error_detected(struct pci_dev *pdev,* capability registers and bounce the active state of the memdev. */ ue =cxl_handle_ras (&cxlds->cxlmd->dev, cxlds->serial, - cxlds->regs.ras); + cxlmd->endpoint->regs.ras); } -switch (state) {case pci_channel_io_normal:if (ue) {
drivers/cxl/cxlmem.h +2 -2
@@ -415,7 +415,7 @@ struct cxl_dpa_partition {* @dev: The device associated with this CXL state * @cxlmd: The device representing the CXL.mem capabilities of @dev * @reg_map: component and ras register mapping parameters -* @regs: Parsed register blocks +* @regs: Class device "Device" registers * @cxl_dvsec: Offset to the PCIe device DVSEC * @rcd: operating in RCD mode (CXL 3.0 9.11.8 CXL Devices Attached to an RCH) * @media_ready: Indicate whether the device media is usable
@@ -431,7 +431,7 @@ struct cxl_dev_state {struct device *dev;struct cxl_memdev *cxlmd;struct cxl_register_map reg_map; -struct cxl_regs regs; +struct cxl_device_regs regs;int cxl_dvsec;bool rcd;bool media_ready;
drivers/cxl/pci.c +1 -62
93 lines not on this page · raw message
drivers/cxl/port.c +54 -0
71 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH v14 24/34] cxl/port: Move endpoint component register management to cxl_port
Quoted 9 lines · Terry Bowman
From: Dan Williams <[email protected]>
In preparation for generic protocol error handling across CXL endpoints, whether they be memory expander class devices or accelerators, drop the endpoint component management from
cxl_dev_state.Organize all CXL port component management through the common
cxl_portdriver.
Quoted 7 lines
Note that the end game is that
drivers/cxl/core/ras.closes all dependencies on a 'structcxl_dev_state' parameter and operates only on port resources. The removal of component register mapping fromcxl_pciis an incremental step towards that.Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>
missing sign off
Quoted 248 lines
---
Changes in v13 -> v14:
- New patch
- Update log message forcxl_ras_unmask()failure (Dan)--- drivers/cxl/core/ras.c | 6 ++-- drivers/cxl/cxlmem.h | 4 +-- drivers/cxl/pci.c | 63 +----------------------------------------- drivers/cxl/port.c | 54 ++++++++++++++++++++++++++++++++++++ 4 files changed, 60 insertions(+), 67 deletions(-)drivers/cxl/core/ras.c +3
-3@@ -247,6 +247,7 @@ bool cxl_handle_ras(struct device *dev, u64 serial, void __iomem *ras_base)void cxl_cor_error_detected (struct pci_dev *pdev) {struct cxl_dev_state *cxlds =pci_get_drvdata (pdev); +struct cxl_memdev *cxlmd = cxlds->cxlmd;struct device *dev = &cxlds->cxlmd->dev; scoped_guard(device, dev) {@@ -261,7 +262,7 @@ void cxl_cor_error_detected(struct pci_dev *pdev)cxl_handle_rdport_errors (cxlds);cxl_handle_cor_ras (&cxlds->cxlmd->dev, cxlds->serial, - cxlds->regs.ras); + cxlmd->endpoint->regs.ras); } }EXPORT_SYMBOL_NS_GPL (cxl_cor_error_detected,"CXL" );@@ -291,10 +292,9 @@ pci_ers_result_t cxl_error_detected(struct pci_dev *pdev,* capability registers and bounce the active state of the memdev. */ ue =cxl_handle_ras (&cxlds->cxlmd->dev, cxlds->serial, - cxlds->regs.ras); + cxlmd->endpoint->regs.ras); } -switch (state) {case pci_channel_io_normal:if (ue) {drivers/cxl/cxlmem.h +2
-2@@ -415,7 +415,7 @@ struct cxl_dpa_partition {* @dev: The device associated with this CXL state * @cxlmd: The device representing the CXL.mem capabilities of @dev * @reg_map: component and ras register mapping parameters -* @regs: Parsed register blocks +* @regs: Class device "Device" registers * @cxl_dvsec: Offset to the PCIe device DVSEC * @rcd: operating in RCD mode (CXL 3.0 9.11.8 CXL Devices Attached to an RCH) * @media_ready: Indicate whether the device media is usable @@ -431,7 +431,7 @@ struct cxl_dev_state {struct device *dev;struct cxl_memdev *cxlmd;struct cxl_register_map reg_map; -struct cxl_regs regs; +struct cxl_device_regs regs;int cxl_dvsec;bool rcd;bool media_ready;drivers/cxl/pci.c +1
-62@@ -535,52 +535,6 @@ static int cxl_pci_setup_regs(struct pci_dev *pdev, enum cxl_regloc_type type,return cxl_setup_regs (map); } -static int cxl_pci_ras_unmask (struct pci_dev *pdev) -{ -struct cxl_dev_state *cxlds =pci_get_drvdata (pdev); -void __iomem *addr; -u32 orig_val, val, mask; -u16 cap; -int rc; - -if (!cxlds->regs.ras) { -dev_dbg (&pdev->dev,"No RAS registers.\n" ); -return 0 ; - } - -/* BIOS has PCIe AER error control */ -if (!pcie_aer_is_native (pdev)) -return 0 ; - - rc =pcie_capability_read_word (pdev,PCI_EXP_DEVCTL , &cap); -if (rc) -return rc; - -if (cap &PCI_EXP_DEVCTL_URRE ) { - addr = cxlds->regs.ras +CXL_RAS_UNCORRECTABLE_MASK_OFFSET ; - orig_val =readl (addr); - - mask =CXL_RAS_UNCORRECTABLE_MASK_MASK | -CXL_RAS_UNCORRECTABLE_MASK_F256B_MASK ; - val = orig_val & ~mask; -writel (val, addr); -dev_dbg (&pdev->dev, -"Uncorrectable RAS Errors Mask: %#x -> %#x\n" , - orig_val, val); - } - -if (cap &PCI_EXP_DEVCTL_CERE ) { - addr = cxlds->regs.ras +CXL_RAS_CORRECTABLE_MASK_OFFSET ; - orig_val =readl (addr); - val = orig_val & ~CXL_RAS_CORRECTABLE_MASK_MASK ; -writel (val, addr); -dev_dbg (&pdev->dev,"Correctable RAS Errors Mask: %#x -> %#x\n" , - orig_val, val); - } - -return 0 ; -} -static void free_event_buf (void *buf) {kvfree (buf);@@ -912,13 +866,6 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)unsigned int i;bool irq_avail; -/* -* Double check the anonymous union trickery in struct cxl_regs -* FIXME switch to struct_group() -*/ -BUILD_BUG_ON (offsetof(struct cxl_regs , memdev) != - offsetof(struct cxl_regs, device_regs.memdev)); - rc =pcim_enable_device (pdev);if (rc)return rc;@@ -942,7 +889,7 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)if (rc)return rc; - rc =cxl_map_device_regs (&map, &cxlds->regs.device_regs); + rc =cxl_map_device_regs (&map, &cxlds->regs);if (rc)return rc;@@ -957,11 +904,6 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)else if (!cxlds->reg_map.component_map.ras.valid)dev_dbg (&pdev->dev,"RAS registers not found\n" ); - rc =cxl_map_component_regs (&cxlds->reg_map, &cxlds->regs.component, -BIT (CXL_CM_CAP_CAP_ID_RAS )); -if (rc) -dev_dbg (&pdev->dev,"Failed to map RAS capability.\n" ); - rc =cxl_pci_type3_init_mailbox (cxlds);if (rc)return rc;@@ -1052,9 +994,6 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)if (rc)return rc; -if (cxl_pci_ras_unmask (pdev)) -dev_dbg (&pdev->dev,"No RAS reporting unmasked\n" ); -pci_save_state (pdev);return rc;drivers/cxl/port.c +54
-0@@ -1,5 +1,6 @@// SPDX-License-Identifier: GPL-2.0-only /* Copyright(c) 2022 Intel Corporation. All rights reserved. */ +#include <linux/aer.h> #include <linux/device.h> #include <linux/module.h> #include <linux/slab.h> @@ -72,6 +73,55 @@ static int cxl_switch_port_probe(struct cxl_port *port)return 0 ; } +static int cxl_ras_unmask (struct cxl_port *port) +{ +struct pci_dev *pdev; +void __iomem *addr; +u32 orig_val, val, mask; +u16 cap; +int rc; + +if (!dev_is_pci (port->uport_dev)) +return 0 ; + pdev =to_pci_dev (port->uport_dev); + +if (!port->regs.ras) { +pci_dbg (pdev,"No RAS registers.\n" ); +return 0 ; + } + +/* BIOS has PCIe AER error control */ +if (!pcie_aer_is_native (pdev)) +return 0 ; + + rc =pcie_capability_read_word (pdev,PCI_EXP_DEVCTL , &cap); +if (rc) +return rc; + +if (cap &PCI_EXP_DEVCTL_URRE ) { + addr = port->regs.ras +CXL_RAS_UNCORRECTABLE_MASK_OFFSET ; + orig_val =readl (addr); + + mask =CXL_RAS_UNCORRECTABLE_MASK_MASK | +CXL_RAS_UNCORRECTABLE_MASK_F256B_MASK ; + val = orig_val & ~mask; +writel (val, addr); +pci_dbg (pdev,"Uncorrectable RAS Errors Mask: %#x -> %#x\n" , + orig_val, val); + } + +if (cap &PCI_EXP_DEVCTL_CERE ) { + addr = port->regs.ras +CXL_RAS_CORRECTABLE_MASK_OFFSET ; + orig_val =readl (addr); + val = orig_val & ~CXL_RAS_CORRECTABLE_MASK_MASK ; +writel (val, addr); +pci_dbg (pdev,"Correctable RAS Errors Mask: %#x -> %#x\n" , + orig_val, val); + } + +return 0 ; +} +static int cxl_endpoint_port_probe (struct cxl_port *port) {struct cxl_memdev *cxlmd =to_cxl_memdev (port->uport_dev);@@ -102,6 +152,10 @@ static int cxl_endpoint_port_probe(struct cxl_port *port)if (dport->rch)devm_cxl_dport_ras_setup (dport); +devm_cxl_port_ras_setup (port); +if (cxl_ras_unmask (port)) +dev_dbg (&port->dev,"failed to unmask RAS interrupts\n" ); +/* * Now that all endpoint decoders are successfully enumerated, try to * assemble regions from committed decoders
Re: [PATCH v14 24/34] cxl/port: Move endpoint component register management to cxl_port
In reply to Terry BowmanQuoted 9 lines · On Wed, 14 Jan 2026 12:20:45 -0600 Terry Bowman <[email protected]> wrote:
From: Dan Williams <[email protected]>
In preparation for generic protocol error handling across CXL endpoints, whether they be memory expander class devices or accelerators, drop the endpoint component management from
cxl_dev_state.Organize all CXL port component management through the common
cxl_portdriver.
Quoted 7 lines
Note that the end game is that
drivers/cxl/core/ras.closes all dependencies on a 'structcxl_dev_state' parameter and operates only on port resources. The removal of component register mapping fromcxl_pciis an incremental step towards that.Signed-off-by Dan Williams <[email protected]>Reviewed-by Terry Bowman <[email protected]>
PATCH v14 25/34 cxl/port: Map Port component registers before switchport init
In reply to Terry BowmanPort HDM registers must be mapped before calling devm_cxl_switch_port_decoders_setup(). Invoke a call to this function in cxl_port_add_dport().
--- drivers/cxl/core/port.c | 3 ++- drivers/cxl/cxlpci.h | 3 +++ drivers/cxl/port.c | 5 +++++ 3 files changed, 10 insertions(+), 1 deletion(-)
drivers/cxl/core/port.c +2 -1
@@ -778,7 +778,7 @@ static int cxl_setup_comp_regs(struct device *host, struct cxl_register_map *mapreturn cxl_setup_regs (map); } -static int cxl_port_setup_regs (struct cxl_port *port, +int cxl_port_setup_regs (struct cxl_port *port,resource_size_t component_reg_phys) {if (dev_is_platform (port->uport_dev))
@@ -786,6 +786,7 @@ static int cxl_port_setup_regs(struct cxl_port *port,return cxl_setup_comp_regs (&port->dev, &port->reg_map, component_reg_phys); } +EXPORT_SYMBOL_NS_GPL (cxl_port_setup_regs,"CXL" );static int cxl_dport_setup_regs (struct device *host,struct cxl_dport *dport,resource_size_t component_reg_phys)
drivers/cxl/cxlpci.h +3 -0
@@ -99,4 +99,7 @@ static inline void devm_cxl_port_ras_setup(struct cxl_port *port) }#endif +int cxl_port_setup_regs (struct cxl_port *port, +resource_size_t component_reg_phys); +#endif /* __CXL_PCI_H__ */
drivers/cxl/port.c +5 -0
@@ -278,6 +278,11 @@ static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port,return ERR_CAST (port_group);if (port->nr_dports ==0 ) { + + rc =cxl_port_setup_regs (port, port->component_reg_phys); +if (rc) +return ERR_PTR (rc); + rc =devm_cxl_switch_port_decoders_setup (port);if (rc)return ERR_PTR (rc);
Signature · 1 line
2.34.1
Re: [PATCH v14 25/34] cxl/port: Map Port component registers before switchport init
Quoted 3 lines · Terry Bowman
Port HDM registers must be mapped before calling
devm_cxl_switch_port_decoders_setup(). Invoke a call to this function incxl_port_add_dport().
s/Invoke.../Map the per port component registers when the first dport is being enumerated./
Quoted 2 lines
Signed-off-by Terry Bowman <[email protected]>
Quoted 55 lines
--- drivers/cxl/core/port.c | 3 ++- drivers/cxl/cxlpci.h | 3 +++ drivers/cxl/port.c | 5 +++++ 3 files changed, 10 insertions(+), 1 deletion(-)drivers/cxl/core/port.c +2
-1@@ -778,7 +778,7 @@ static int cxl_setup_comp_regs(struct device *host, struct cxl_register_map *mapreturn cxl_setup_regs (map); } -static int cxl_port_setup_regs (struct cxl_port *port, +int cxl_port_setup_regs (struct cxl_port *port,resource_size_t component_reg_phys) {if (dev_is_platform (port->uport_dev))@@ -786,6 +786,7 @@ static int cxl_port_setup_regs(struct cxl_port *port,return cxl_setup_comp_regs (&port->dev, &port->reg_map, component_reg_phys); } +EXPORT_SYMBOL_NS_GPL (cxl_port_setup_regs,"CXL" );static int cxl_dport_setup_regs (struct device *host,struct cxl_dport *dport,resource_size_t component_reg_phys)drivers/cxl/cxlpci.h +3
-0@@ -99,4 +99,7 @@ static inline void devm_cxl_port_ras_setup(struct cxl_port *port) }#endif +int cxl_port_setup_regs (struct cxl_port *port, +resource_size_t component_reg_phys); +#endif /* __CXL_PCI_H__ */ drivers/cxl/port.c +5
-0@@ -278,6 +278,11 @@ static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port,return ERR_CAST (port_group);if (port->nr_dports ==0 ) { + + rc =cxl_port_setup_regs (port, port->component_reg_phys); +if (rc) +return ERR_PTR (rc); + rc =devm_cxl_switch_port_decoders_setup (port);if (rc)return ERR_PTR (rc);
Re: [PATCH v14 25/34] cxl/port: Map Port component registers before switchport init
In reply to Terry BowmanQuoted 3 lines · On Wed, 14 Jan 2026 12:20:46 -0600 Terry Bowman <[email protected]> wrote:
Port HDM registers must be mapped before calling
devm_cxl_switch_port_decoders_setup(). Invoke a call to this function incxl_port_add_dport().
As I read that description, there is a bisection break before this in that if you build up to patch 24, they won't be mapped before it is called.
Maybe this needs squashing with an earlier patch, or if it is for some reason safe, then add a comment here on why.
Quoted 57 lines
Signed-off-by Terry Bowman <[email protected]>--- drivers/cxl/core/port.c | 3 ++- drivers/cxl/cxlpci.h | 3 +++ drivers/cxl/port.c | 5 +++++ 3 files changed, 10 insertions(+), 1 deletion(-)drivers/cxl/core/port.c +2
-1@@ -778,7 +778,7 @@ static int cxl_setup_comp_regs(struct device *host, struct cxl_register_map *mapreturn cxl_setup_regs (map); } -static int cxl_port_setup_regs (struct cxl_port *port, +int cxl_port_setup_regs (struct cxl_port *port,resource_size_t component_reg_phys) {if (dev_is_platform (port->uport_dev))@@ -786,6 +786,7 @@ static int cxl_port_setup_regs(struct cxl_port *port,return cxl_setup_comp_regs (&port->dev, &port->reg_map, component_reg_phys); } +EXPORT_SYMBOL_NS_GPL (cxl_port_setup_regs,"CXL" );static int cxl_dport_setup_regs (struct device *host,struct cxl_dport *dport,resource_size_t component_reg_phys)drivers/cxl/cxlpci.h +3
-0@@ -99,4 +99,7 @@ static inline void devm_cxl_port_ras_setup(struct cxl_port *port) }#endif +int cxl_port_setup_regs (struct cxl_port *port, +resource_size_t component_reg_phys); +#endif /* __CXL_PCI_H__ */ drivers/cxl/port.c +5
-0@@ -278,6 +278,11 @@ static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port,return ERR_CAST (port_group);if (port->nr_dports ==0 ) { + + rc =cxl_port_setup_regs (port, port->component_reg_phys); +if (rc) +return ERR_PTR (rc); + rc =devm_cxl_switch_port_decoders_setup (port);if (rc)return ERR_PTR (rc);
PATCH v14 26/34 cxl: Change CXL handlers to use guard() instead of scoped_guard()
In reply to Terry BowmanThe CXL protocol error handlers use scoped_guard() to guarantee access to the underlying CXL memory device. Improve readability and reduce complexity by changing the current scoped_guard() to be guard().
---
Changes in v13->v14:
- Add reviewed-by for Jonathan and Dave Jiang
Changes in v12->v13:
- New patch
--- drivers/cxl/core/ras.c | 58 +++++++++++++++++++++--------------------- 1 file changed, 29 insertions(+), 29 deletions(-)
drivers/cxl/core/ras.c +29 -29
@@ -250,20 +250,20 @@ void cxl_cor_error_detected(struct pci_dev *pdev)struct cxl_memdev *cxlmd = cxlds->cxlmd;struct device *dev = &cxlds->cxlmd->dev; - scoped_guard(device, dev) { -if (!dev->driver) { -dev_warn (&pdev->dev, -"%s: memdev disabled, abort error handling\n" , -dev_name (dev)); -return ; - } +guard (device)(dev); -if (cxlds->rcd) -cxl_handle_rdport_errors (cxlds); - -cxl_handle_cor_ras (&cxlds->cxlmd->dev, cxlds->serial, - cxlmd->endpoint->regs.ras); +if (!dev->driver) { +dev_warn (&pdev->dev, +"%s: memdev disabled, abort error handling\n" , +dev_name (dev)); +return ; } + +if (cxlds->rcd) +cxl_handle_rdport_errors (cxlds); + +cxl_handle_cor_ras (&cxlmd->dev, cxlds->serial, + cxlmd->endpoint->regs.ras); }EXPORT_SYMBOL_NS_GPL (cxl_cor_error_detected,"CXL" );
@@ -275,26 +275,26 @@ pci_ers_result_t cxl_error_detected(struct pci_dev *pdev,struct device *dev = &cxlmd->dev;bool ue; - scoped_guard(device, dev) { -if (!dev->driver) { -dev_warn (&pdev->dev, -"%s: memdev disabled, abort error handling\n" , -dev_name (dev)); -return PCI_ERS_RESULT_DISCONNECT ; - } +guard (device)(dev); -if (cxlds->rcd) -cxl_handle_rdport_errors (cxlds); -/* -* A frozen channel indicates an impending reset which is fatal to -* CXL.mem operation, and will likely crash the system. On the off -* chance the situation is recoverable dump the status of the RAS -* capability registers and bounce the active state of the memdev. -*/ - ue =cxl_handle_ras (&cxlds->cxlmd->dev, cxlds->serial, - cxlmd->endpoint->regs.ras); +if (!dev->driver) { +dev_warn (&pdev->dev, +"%s: memdev disabled, abort error handling\n" , +dev_name (dev)); +return PCI_ERS_RESULT_DISCONNECT ; } +if (cxlds->rcd) +cxl_handle_rdport_errors (cxlds); +/* +* A frozen channel indicates an impending reset which is fatal to +* CXL.mem operation, and will likely crash the system. On the off +* chance the situation is recoverable dump the status of the RAS +* capability registers and bounce the active state of the memdev. +*/ + ue =cxl_handle_ras (&cxlmd->dev, cxlds->serial, + cxlmd->endpoint->regs.ras); +switch (state) {case pci_channel_io_normal:if (ue) {
Signature · 1 line
2.34.1
Re: [PATCH v14 26/34] cxl: Change CXL handlers to use guard() instead of scoped_guard()
Quoted 3 lines
The CXL protocol error handlers use
scoped_guard()to guarantee access to the underlying CXL memory device. Improve readability and reduce complexity by changing the currentscoped_guard()to beguard().
Would it be worth to mention that this adjustment can only be performed because lock scopes can be kept the same for two function implementations?
Regards,
Markus
PATCH v14 27/34 PCI/ERR: Introduce PCI_ERS_RESULT_PANIC
In reply to Terry BowmanThe CXL driver's error handling for uncorrectable errors (UCE) will be updated in the future. A required change is for the error handlers to to force a system panic when a UCE is detected.
Introduce PCI_ERS_RESULT_PANIC as a 'enum pci_ers_result' type. This will be used by CXL UCE fatal and non-fatal recovery in future patches. Update PCIe recovery documentation with details of PCI_ERS_RESULT_PANIC.
---
Changes in v13 -> v14:
- Add review-by for Dan
- Update Title prefix (Bjorn)
- Removed merge_result. Only logging error for device reporting the error (Dan)
Changes in v12->v13:
- Add Dave Jiang's, Jonathan's, Ben's review-by
- Typo fix (Ben)
Changes v11 -> v12:
- Documentation requested (Lukas)
--- Documentation/PCI/pci-error-recovery.rst | 2 ++ include/linux/pci.h | 3 +++ 2 files changed, 5 insertions(+)
Documentation/PCI/pci-error-recovery.rst +2 -0
@@ -102,6 +102,8 @@ Possible return values are:: PCI_ERS_RESULT_NEED_RESET, /* Device driver wants slot to be reset. */ PCI_ERS_RESULT_DISCONNECT, /* Device has completely failed, is unrecoverable */ PCI_ERS_RESULT_RECOVERED, /* Device driver is fully recovered and operational */ + PCI_ERS_RESULT_NO_AER_DRIVER, /* No AER capabilities registered for the driver */ + PCI_ERS_RESULT_PANIC, /* System is unstable, panic. Is CXL specific */ }; A driver does not have to implement all of these callbacks; however,
include/linux/pci.h +3 -0
@@ -921,6 +921,9 @@ enum pci_ers_result {/* No AER capabilities registered for the driver */ PCI_ERS_RESULT_NO_AER_DRIVER = (__force pci_ers_result_t )6 , + +/* System is unstable, panic. Is CXL specific */ + PCI_ERS_RESULT_PANIC = (__force pci_ers_result_t) 7, };/* PCI bus error event callbacks */
Signature · 1 line
2.34.1
Re: [PATCH v14 27/34] PCI/ERR: Introduce PCI_ERS_RESULT_PANIC
Hi,
Quoted 8 lines · Terry Bowman
The CXL driver's error handling for uncorrectable errors (UCE) will be updated in the future. A required change is for the error handlers to to force a system panic when a UCE is detected.
Introduce
PCI_ERS_RESULT_PANICas a 'enumpci_ers_result' type. This will be used by CXL UCE fatal and non-fatal recovery in future patches. Update PCIe recovery documentation with details ofPCI_ERS_RESULT_PANIC.
Quoted 6 lines
Signed-off-by Terry Bowman <[email protected]>Reviewed-by Dave Jiang <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Ben Cheatham <[email protected]>Reviewed-by Dan Williams <[email protected]>
Quoted 24 lines
---
Changes in v13 -> v14:
- Add review-by for Dan
- Update Title prefix (Bjorn)
- Removedmerge_result. Only logging error for device reporting the error (Dan)Changes in
v12->v13:
- Add Dave Jiang's, Jonathan's, Ben's review-by
- Typo fix (Ben)Changes v11 -> v12:
- Documentation requested (Lukas)--- Documentation/PCI/pci-error-recovery.rst | 2 ++ include/linux/pci.h | 3 +++ 2 files changed, 5 insertions(+)Documentation/PCI/pci-error-recovery.rst +0
-0
Documentation/PCI/pci-error-recovery.rst Possible return values are:: · line 107
Quoted 6 lines
Documentation/PCI/pci-error-recovery.rst +2
-0@@ -102,6 +102,8 @@ Possible return values are:: PCI_ERS_RESULT_NEED_RESET, /* Device driver wants slot to be reset. */ PCI_ERS_RESULT_DISCONNECT, /* Device has completely failed, is unrecoverable */ PCI_ERS_RESULT_RECOVERED, /* Device driver is fully recovered and operational */ + PCI_ERS_RESULT_NO_AER_DRIVER, /* No AER capabilities registered for the driver */ + PCI_ERS_RESULT_PANIC, /* System is unstable, panic. Is CXL specific */ };
I think you also need to update the "Detailed Steps" section of this document to include details on when these new values should be returned and how they affect the recovery flow.
Quoted 16 lines
Documentation/PCI/pci-error-recovery.rst +0
-0A driver does not have to implement all of these callbacks; however,include/linux/pci.h +3
-0@@ -921,6 +921,9 @@ enum pci_ers_result {/* No AER capabilities registered for the driver */ PCI_ERS_RESULT_NO_AER_DRIVER = (__force pci_ers_result_t )6 , + +/* System is unstable, panic. Is CXL specific */ + PCI_ERS_RESULT_PANIC = (__force pci_ers_result_t) 7, };/* PCI bus error event callbacks */
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [PATCH v14 27/34] PCI/ERR: Introduce PCI_ERS_RESULT_PANIC
Quoted 39 lines · Kuppuswamy Sathyanarayanan
Hi,
Quoted 8 lines · Terry Bowman
The CXL driver's error handling for uncorrectable errors (UCE) will be updated in the future. A required change is for the error handlers to to force a system panic when a UCE is detected.
Introduce
PCI_ERS_RESULT_PANICas a 'enumpci_ers_result' type. This will be used by CXL UCE fatal and non-fatal recovery in future patches. Update PCIe recovery documentation with details ofPCI_ERS_RESULT_PANIC.Quoted 6 lines
Signed-off-by Terry Bowman <[email protected]>Reviewed-by Dave Jiang <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Ben Cheatham <[email protected]>Reviewed-by Dan Williams <[email protected]>Reviewed-by Kuppuswamy Sathyanarayanan <[email protected]>Quoted 19 lines
---
Changes in v13 -> v14:
- Add review-by for Dan
- Update Title prefix (Bjorn)
- Removedmerge_result. Only logging error for device reporting the error (Dan)Changes in
v12->v13:
- Add Dave Jiang's, Jonathan's, Ben's review-by
- Typo fix (Ben)Changes v11 -> v12:
- Documentation requested (Lukas)--- Documentation/PCI/pci-error-recovery.rst | 2 ++ include/linux/pci.h | 3 +++ 2 files changed, 5 insertions(+)
Quoted 16 lines
Documentation/PCI/pci-error-recovery.rst Possible return values are:: · line 107
Quoted 11 lines
Documentation/PCI/pci-error-recovery.rst +2
-0@@ -102,6 +102,8 @@ Possible return values are:: PCI_ERS_RESULT_NEED_RESET, /* Device driver wants slot to be reset. */ PCI_ERS_RESULT_DISCONNECT, /* Device has completely failed, is unrecoverable */ PCI_ERS_RESULT_RECOVERED, /* Device driver is fully recovered and operational */ + PCI_ERS_RESULT_NO_AER_DRIVER, /* No AER capabilities registered for the driver */ + PCI_ERS_RESULT_PANIC, /* System is unstable, panic. Is CXL specific */ };I think you also need to update the "Detailed Steps" section of this document to include details on when these new values should be returned and how they affect the recovery flow.
I had details about PCI_ERS_RESULT_PANIC you mention in v13. Bjorne asked me to remove.
-Terry
Quoted 17 lines
Quoted 16 lines
Documentation/PCI/pci-error-recovery.rst +0
-0A driver does not have to implement all of these callbacks; however,include/linux/pci.h +3
-0@@ -921,6 +921,9 @@ enum pci_ers_result {/* No AER capabilities registered for the driver */ PCI_ERS_RESULT_NO_AER_DRIVER = (__force pci_ers_result_t )6 , + +/* System is unstable, panic. Is CXL specific */ + PCI_ERS_RESULT_PANIC = (__force pci_ers_result_t) 7, };/* PCI bus error event callbacks */
Re: [PATCH v14 27/34] PCI/ERR: Introduce PCI_ERS_RESULT_PANIC
Hi,
Quoted 57 lines · Terry
Quoted 39 lines · Kuppuswamy Sathyanarayanan
Hi,
Quoted 8 lines · Terry Bowman
The CXL driver's error handling for uncorrectable errors (UCE) will be updated in the future. A required change is for the error handlers to to force a system panic when a UCE is detected.
Introduce
PCI_ERS_RESULT_PANICas a 'enumpci_ers_result' type. This will be used by CXL UCE fatal and non-fatal recovery in future patches. Update PCIe recovery documentation with details ofPCI_ERS_RESULT_PANIC.Quoted 6 lines
Signed-off-by Terry Bowman <[email protected]>Reviewed-by Dave Jiang <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Ben Cheatham <[email protected]>Reviewed-by Dan Williams <[email protected]>Reviewed-by Kuppuswamy Sathyanarayanan <[email protected]>Quoted 19 lines
---
Changes in v13 -> v14:
- Add review-by for Dan
- Update Title prefix (Bjorn)
- Removedmerge_result. Only logging error for device reporting the error (Dan)Changes in
v12->v13:
- Add Dave Jiang's, Jonathan's, Ben's review-by
- Typo fix (Ben)Changes v11 -> v12:
- Documentation requested (Lukas)--- Documentation/PCI/pci-error-recovery.rst | 2 ++ include/linux/pci.h | 3 +++ 2 files changed, 5 insertions(+)Quoted 16 lines
Documentation/PCI/pci-error-recovery.rst Possible return values are:: · line 107
Quoted 11 lines
Documentation/PCI/pci-error-recovery.rst +2
-0@@ -102,6 +102,8 @@ Possible return values are:: PCI_ERS_RESULT_NEED_RESET, /* Device driver wants slot to be reset. */ PCI_ERS_RESULT_DISCONNECT, /* Device has completely failed, is unrecoverable */ PCI_ERS_RESULT_RECOVERED, /* Device driver is fully recovered and operational */ + PCI_ERS_RESULT_NO_AER_DRIVER, /* No AER capabilities registered for the driver */ + PCI_ERS_RESULT_PANIC, /* System is unstable, panic. Is CXL specific */ };I think you also need to update the "Detailed Steps" section of this document to include details on when these new values should be returned and how they affect the recovery flow.
Quoted 1 line
I had details about
PCI_ERS_RESULT_PANICyou mention in v13. Bjorne asked me to remove.
Sorry, I did not check the previous version.
What about PCI_ERS_RESULT_NO_AER_DRIVER? I think it needs to be included part of STEP 1 details, but likely as a separate patch since it is unrelated to the CXL changes in this series.
Quoted 21 lines
-Terry
Quoted 17 lines
Quoted 16 lines
Documentation/PCI/pci-error-recovery.rst +0
-0A driver does not have to implement all of these callbacks; however,include/linux/pci.h +3
-0@@ -921,6 +921,9 @@ enum pci_ers_result {/* No AER capabilities registered for the driver */ PCI_ERS_RESULT_NO_AER_DRIVER = (__force pci_ers_result_t )6 , + +/* System is unstable, panic. Is CXL specific */ + PCI_ERS_RESULT_PANIC = (__force pci_ers_result_t) 7, };/* PCI bus error event callbacks */
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer