cxl: Delay HB port and switch dport probing until endpoint dev probe
Dave Jiang
maintainerv9 cover cxl: Delay HB port and switch dport probing until endpoint dev probe
v9:
- Reworked the port enumeration iteration loop. (Robert)
All please re-review "cxl: Defer dport allocation for switch ports"
- Created helper functions for dealing with switch and endpoint decoder enumeration. Main goal is to reduce cxl_test interface burden. (Robert)
- Dropped cxl_test changes for decoder functions
- Dropped the cxl_test for region replay. It's not 100% ready and can be submitted later. (Alison)
- See specific commits for more detailed changes.
v8:
- A bit of changes from Dan and Robert's comments. Main change is moving the port MMIO register probing to after the first dport shows up. This resulted with decoder allocation happens after the register probe.
- See specific commits for more detailed changes.
v7:
- Remove -EEXIST to simplify error flow. (Ming)
- Set dport to NULL during declare. (Jonathan)
v6:
- Return -EEXIST when a dport already exists. (Jonathan)
- Fix checking wrong port for NULL. (Ming)
- Check host_bridge and call devm_cxl_add_dport_by_uport() directly vs add_port_attach_ep(). (Ming)
- Set dport to NULL during declaration. (Jonathan)
v5:
- Return dport instead of errno with dport pointer as output param. (Jonathan)
- Consolidate common code in cxl_test. (Jonathan)
- Rename cxl_port_get_total_dports() to cxl_port_update_total_dports(). (Jonathan)
v4:
- Push dport allocation to when they are discovered. (Robert)
- Drop linux id for dport with above changes.
v3:
- Main changes revolve around improving naming of hostbridge uport and dport (Gregory)
- See specific patches for detailed change log
This series attempts to delay the allocation of dports until when the endpoint device (memdev) are being probed. At this point, the CXL link is established and all the devices along the CXL link path up to the Root Port (RP) should be active.
And hopefully this help a bit with Robert's issue raised in the "Inactive downstream port handling" series [1]. Testing would be appreicated. Thank you!
[1]: https://lore.kernel.org/linux-cxl/[email protected]/
Dave Jiang (10):
cxl: Add helper to detect top of CXL device topology
cxl: Add helper to reap dport
cxl: Add a cached copy of target_map to cxl_decoder
cxl: Move port register setup to first dport appear cxl/test: Refactor decoder setup to reduce cxl_test burden
cxl: Defer dport allocation for switch ports cxl/test: Add mock version of devm_cxl_add_dport_by_dev() cxl/test: Adjust the mock version of
devm_cxl_switch_port_decoders_setup()
cxl/test: Setup target_map for cxl_test decoder initialization
cxl: Change sslbis handler to only handle single dport
drivers/cxl/acpi.c | 7 +- drivers/cxl/core/cdat.c | 25 +-- drivers/cxl/core/core.h | 5 + drivers/cxl/core/hdm.c | 105 ++++++--- drivers/cxl/core/pci.c | 89 ++++++++ drivers/cxl/core/port.c | 317 ++++++++++++++++++++------- drivers/cxl/core/region.c | 4 +- drivers/cxl/cxl.h | 44 +++- drivers/cxl/cxlpci.h | 2 - drivers/cxl/port.c | 47 +--- tools/testing/cxl/Kbuild | 7 +- tools/testing/cxl/cxl_core_exports.c | 22 ++ tools/testing/cxl/exports.h | 13 ++ tools/testing/cxl/test/cxl.c | 115 ++++++++-- tools/testing/cxl/test/mock.c | 96 +++----- tools/testing/cxl/test/mock.h | 9 +- 16 files changed, 642 insertions(+), 265 deletions(-) create mode 100644 tools/testing/cxl/exports.h
base-commit: 8f5ae30d69d7543eee0d70083daf4de8fe15d585
Signature · 1 line
2.50.1
Dave Jiang
maintainerPATCH v9 1/10 cxl: Add helper to detect top of CXL device topology
Add a helper to replace the open code detection of CXL device hierarchy root, or the host bridge. The helper will be used for delayed downstream port (dport) creation.
--- drivers/cxl/core/port.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-)
drivers/cxl/core/port.c +11 -6
@@ -33,6 +33,15 @@static DEFINE_IDA (cxl_port_ida);static DEFINE_XARRAY (cxl_root_buses); +/* +* The terminal device in PCI is NULL and @platform_bus +* for platform devices (for cxl_test) +*/ +static bool is_cxl_host_bridge (struct device *dev) +{ +return (!dev || dev == &platform_bus); +} +int cxl_num_decoders_committed (struct cxl_port *port) {lockdep_assert_held (&cxl_rwsem.region);
@@ -1541,7 +1550,7 @@ static int add_port_attach_ep(struct cxl_memdev *cxlmd,resource_size_t component_reg_phys;int rc; -if (!dparent) { +if (is_cxl_host_bridge (dparent)) {/* * The iteration reached the topology root without finding the * CXL-root 'cxl_port' on a previous iteration, fail for now to
@@ -1629,11 +1638,7 @@ int devm_cxl_enumerate_ports(struct cxl_memdev *cxlmd)struct device *uport_dev;struct cxl_dport *dport; -/* -* The terminal "grandparent" in PCI is NULL and @platform_bus -* for platform devices -*/ -if (!dport_dev || dport_dev == &platform_bus) +if (is_cxl_host_bridge (dport_dev))return 0 ; uport_dev = dport_dev->parent;
Signature · 1 line
2.50.1
Dave Jiang
maintainerPATCH v9 2/10 cxl: Add helper to reap dport
In reply to Dave JiangRefactor the code in reap_dports() out to provide a helper function that reaps a single dport. This will be used later in the cleanup path for allocating a dport. Renaming to del_port() and del_dports() to mirror devm_cxl_add_dport().
--- drivers/cxl/core/port.c | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-)
drivers/cxl/core/port.c +14 -8
@@ -1432,7 +1432,7 @@ EXPORT_SYMBOL_NS_GPL(cxl_endpoint_autoremove, "CXL");* through ->remove(). This "bottom-up" removal selectively removes individual * child ports manually. This depends on devm_cxl_add_port() to not change is * devm action registration order, and for dports to have already been -* destroyed by reap_dports(). +* destroyed by del_dports(). */ static void delete_switch_port (struct cxl_port *port) {
@@ -1441,18 +1441,24 @@ static void delete_switch_port(struct cxl_port *port)devm_release_action (port->dev.parent, unregister_port, port); } -static void reap_dports (struct cxl_port *port) +static void del_dport (struct cxl_dport *dport) +{ +struct cxl_port *port = dport->port; + +devm_release_action (&port->dev, cxl_dport_unlink, dport); +devm_release_action (&port->dev, cxl_dport_remove, dport); +devm_kfree (&port->dev, dport); +} + +static void del_dports (struct cxl_port *port) {struct cxl_dport *dport;unsigned long index;device_lock_assert (&port->dev); -xa_for_each (&port->dports, index, dport) { -devm_release_action (&port->dev, cxl_dport_unlink, dport); -devm_release_action (&port->dev, cxl_dport_remove, dport); -devm_kfree (&port->dev, dport); - } +xa_for_each (&port->dports, index, dport) +del_dport (dport); }struct detach_ctx {
@@ -1510,7 +1516,7 @@ static void cxl_detach_ep(void *data)*/ died = true; port->dead = true; -reap_dports (port); +del_dports (port); }device_unlock (&port->dev);
Signature · 1 line
2.50.1
Re: [PATCH v9 02/10] cxl: Add helper to reap dport
Quoted 4 lines · Dave Jiang
Refactor the code in
reap_dports()out to provide a helper function that reaps a single dport. This will be used later in the cleanup path for allocating a dport. Renaming todel_port()anddel_dports()to mirrordevm_cxl_add_dport().
In case there is another version, reword the subject too.
-Robert
Quoted 10 lines
Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Li Ming <[email protected]>Reviewed-by Alison Schofield <[email protected]>Reviewed-by Dan Williams <[email protected]>Tested-by Robert Richter <[email protected]>Signed-off-by Dave Jiang <[email protected]>--- drivers/cxl/core/port.c | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-)
Dave Jiang
maintainerPATCH v9 3/10 cxl: Add a cached copy of target_map to cxl_decoder
In reply to Dave JiangAdd a cached copy of the hardware port-id list that is available at init before all @dport objects have been instantiated. Change is in preparation of delayed dport instantiation.
---
v9:
- Add region target_map update for manual assemble
--- drivers/cxl/acpi.c | 7 +++---- drivers/cxl/core/hdm.c | 20 ++++++++------------ drivers/cxl/core/port.c | 22 +++++++--------------- drivers/cxl/core/region.c | 4 +++- drivers/cxl/cxl.h | 8 ++++++-- tools/testing/cxl/test/cxl.c | 8 ++++---- 6 files changed, 31 insertions(+), 38 deletions(-)
drivers/cxl/acpi.c +3 -4
@@ -398,7 +398,6 @@ DEFINE_FREE(del_cxl_resource, struct resource *, if (_T) del_cxl_resource(_T))static int __cxl_parse_cfmws (struct acpi_cedt_cfmws *cfmws,struct cxl_cfmws_context *ctx) { -int target_map[CXL_DECODER_MAX_INTERLEAVE ];struct cxl_port *root_port = ctx->root_port;struct cxl_cxims_context cxims_ctx;struct device *dev = ctx->dev;
@@ -416,8 +415,6 @@ static int __cxl_parse_cfmws(struct acpi_cedt_cfmws *cfmws, rc =eig_to_granularity (cfmws->granularity, &ig);if (rc)return rc; -for (i =0 ; i < ways; i++) - target_map[i] = cfmws->interleave_targets[i]; struct resource *res __free(del_cxl_resource) = alloc_cxl_resource( cfmws->base_hpa, cfmws->window_size, ctx->id++);
@@ -443,6 +440,8 @@ static int __cxl_parse_cfmws(struct acpi_cedt_cfmws *cfmws, .end = cfmws->base_hpa + cfmws->window_size - 1, }; cxld->interleave_ways = ways; +for (i =0 ; i < ways; i++) + cxld->target_map[i] = cfmws->interleave_targets[i];/* * Minimize the x1 granularity to advertise support for any * valid region granularity
@@ -475,7 +474,7 @@ static int __cxl_parse_cfmws(struct acpi_cedt_cfmws *cfmws,if (cfmws->interleave_arithmetic ==ACPI_CEDT_CFMWS_ARITHMETIC_XOR ) cxlrd->hpa_to_spa = cxl_xor_hpa_to_spa; - rc =cxl_decoder_add (cxld, target_map); + rc =cxl_decoder_add (cxld);if (rc)return rc;
drivers/cxl/core/hdm.c +8 -12
@@ -21,12 +21,11 @@ struct cxl_rwsem cxl_rwsem = { .dpa = __RWSEM_INITIALIZER(cxl_rwsem.dpa), }; -static int add_hdm_decoder (struct cxl_port *port,struct cxl_decoder *cxld, -int *target_map) +static int add_hdm_decoder (struct cxl_port *port,struct cxl_decoder *cxld) {int rc; - rc =cxl_decoder_add_locked (cxld, target_map); + rc =cxl_decoder_add_locked (cxld);if (rc) {put_device (&cxld->dev);dev_err (&port->dev,"Failed to add decoder\n" );
@@ -54,7 +53,6 @@ int devm_cxl_add_passthrough_decoder(struct cxl_port *port) {struct cxl_switch_decoder *cxlsd;struct cxl_dport *dport =NULL ; -int single_port_map[1 ];unsigned long index;struct cxl_hdm *cxlhdm =dev_get_drvdata (&port->dev);
@@ -73,9 +71,9 @@ int devm_cxl_add_passthrough_decoder(struct cxl_port *port)xa_for_each (&port->dports, index, dport)break ; - single_port_map[0 ] = dport->port_id; + cxlsd->cxld.target_map[0 ] = dport->port_id; -return add_hdm_decoder (port, &cxlsd->cxld, single_port_map); +return add_hdm_decoder (port, &cxlsd->cxld); }EXPORT_SYMBOL_NS_GPL (devm_cxl_add_passthrough_decoder,"CXL" );
@@ -984,7 +982,7 @@ static int cxl_setup_hdm_decoder_from_dvsec( }static int init_hdm_decoder (struct cxl_port *port,struct cxl_decoder *cxld, -int *target_map,void __iomem *hdm,int which, +void __iomem *hdm,int which,u64 *dpa_base,struct cxl_endpoint_dvsec_info *info) {struct cxl_endpoint_decoder *cxled =NULL ;
@@ -1103,7 +1101,7 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, hi =readl (hdm +CXL_HDM_DECODER0_TL_HIGH (which)); target_list.value = (hi <<32 ) + lo;for (i =0 ; i < cxld->interleave_ways; i++) - target_map[i] = target_list.target_id[i]; + cxld->target_map[i] = target_list.target_id[i];return 0 ; }
@@ -1179,7 +1177,6 @@ int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm,cxl_settle_decoders (cxlhdm);for (i =0 ; i < cxlhdm->decoder_count; i++) { -int target_map[CXL_DECODER_MAX_INTERLEAVE ] = {0 };int rc, target_count = cxlhdm->target_count;struct cxl_decoder *cxld;
@@ -1207,8 +1204,7 @@ int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm, cxld = &cxlsd->cxld; } - rc =init_hdm_decoder (port, cxld, target_map, hdm, i, - &dpa_base, info); + rc =init_hdm_decoder (port, cxld, hdm, i, &dpa_base, info);if (rc) { dev_warn(&port->dev, "Failed to initialize decoder%d.%d\n",
@@ -1216,7 +1212,7 @@ int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm,put_device (&cxld->dev);return rc; } - rc =add_hdm_decoder (port, cxld, target_map); + rc =add_hdm_decoder (port, cxld);if (rc) {dev_warn (&port->dev,"Failed to add decoder%d.%d\n" , port->id, i);
drivers/cxl/core/port.c +7 -15
@@ -1715,13 +1715,11 @@ struct cxl_port *cxl_mem_find_port(struct cxl_memdev *cxlmd,EXPORT_SYMBOL_NS_GPL (cxl_mem_find_port,"CXL" );static int decoder_populate_targets (struct cxl_switch_decoder *cxlsd, -struct cxl_port *port,int *target_map) +struct cxl_port *port) { +struct cxl_decoder *cxld = &cxlsd->cxld;int i; -if (!target_map) -return 0 ; -device_lock_assert (&port->dev);if (xa_empty (&port->dports))
@@ -1729,7 +1727,7 @@ static int decoder_populate_targets(struct cxl_switch_decoder *cxlsd,guard (rwsem_write)(&cxl_rwsem.region);for (i =0 ; i < cxlsd->cxld.interleave_ways; i++) { -struct cxl_dport *dport =find_dport (port, target_map[i]); +struct cxl_dport *dport =find_dport (port, cxld->target_map[i]);if (!dport)return -ENXIO ;
@@ -1921,9 +1919,6 @@ EXPORT_SYMBOL_NS_GPL(cxl_endpoint_decoder_alloc, "CXL");/** * cxl_decoder_add_locked - Add a decoder with targets * @cxld: The cxl decoder allocated by cxl_<type>_decoder_alloc() -* @target_map: A list of downstream ports that this decoder can direct memory -* traffic to. These numbers should correspond with the port number -* in the PCIe Link Capabilities structure. * * Certain types of decoders may not have any targets. The main example of this * is an endpoint device. A more awkward example is a hostbridge whose root
@@ -1937,7 +1932,7 @@ EXPORT_SYMBOL_NS_GPL(cxl_endpoint_decoder_alloc, "CXL");* Return: Negative error code if the decoder wasn't properly configured; else * returns 0. */ -int cxl_decoder_add_locked (struct cxl_decoder *cxld,int *target_map) +int cxl_decoder_add_locked (struct cxl_decoder *cxld) {struct cxl_port *port;struct device *dev;
@@ -1958,7 +1953,7 @@ int cxl_decoder_add_locked(struct cxl_decoder *cxld, int *target_map)if (!is_endpoint_decoder (dev)) {struct cxl_switch_decoder *cxlsd =to_cxl_switch_decoder (dev); - rc =decoder_populate_targets (cxlsd, port, target_map); + rc =decoder_populate_targets (cxlsd, port);if (rc && (cxld->flags &CXL_DECODER_F_ENABLE )) {dev_err (&port->dev,"Failed to populate active decoder targets\n" );
@@ -1977,9 +1972,6 @@ EXPORT_SYMBOL_NS_GPL(cxl_decoder_add_locked, "CXL");/** * cxl_decoder_add - Add a decoder with targets * @cxld: The cxl decoder allocated by cxl_<type>_decoder_alloc() -* @target_map: A list of downstream ports that this decoder can direct memory -* traffic to. These numbers should correspond with the port number -* in the PCIe Link Capabilities structure. * * This is the unlocked variant of cxl_decoder_add_locked(). * See cxl_decoder_add_locked().
@@ -1987,7 +1979,7 @@ EXPORT_SYMBOL_NS_GPL(cxl_decoder_add_locked, "CXL");* Context: Process context. Takes and releases the device lock of the port that * owns the @cxld. */ -int cxl_decoder_add (struct cxl_decoder *cxld,int *target_map) +int cxl_decoder_add (struct cxl_decoder *cxld) {struct cxl_port *port;
@@ -2000,7 +1992,7 @@ int cxl_decoder_add(struct cxl_decoder *cxld, int *target_map) port =to_cxl_port (cxld->dev.parent);guard (device)(&port->dev); -return cxl_decoder_add_locked (cxld, target_map); +return cxl_decoder_add_locked (cxld); }EXPORT_SYMBOL_NS_GPL (cxl_decoder_add,"CXL" );
drivers/cxl/core/region.c +3 -1
@@ -1510,8 +1510,10 @@ static int cxl_port_setup_targets(struct cxl_port *port, cxl_rr->nr_targets_set);return -ENXIO ; } - }else + }else { cxlsd->target[cxl_rr->nr_targets_set] = ep->dport; + cxlsd->cxld.target_map[cxl_rr->nr_targets_set] = ep->dport->port_id; + } inc =1 ; out_target_set: cxl_rr->nr_targets_set += inc;
drivers/cxl/cxl.h +6 -2
@@ -357,6 +357,9 @@ enum cxl_decoder_type {* @target_type: accelerator vs expander (type2 vs type3) selector * @region: currently assigned region for this decoder * @flags: memory type capabilities and locking +* @target_map: cached copy of hardware port-id list, available at init +* before all @dport objects have been instantiated. While +* dport id is 8bit, CFMWS interleave targets are 32bits. * @commit: device/decoder-type specific callback to commit settings to hw * @reset: device/decoder-type specific callback to reset hw settings */
@@ -369,6 +372,7 @@ struct cxl_decoder {enum cxl_decoder_type target_type;struct cxl_region *region;unsigned long flags; +u32 target_map[CXL_DECODER_MAX_INTERLEAVE ];int (*commit)(struct cxl_decoder *cxld);void (*reset)(struct cxl_decoder *cxld); };
@@ -781,9 +785,9 @@ struct cxl_root_decoder *cxl_root_decoder_alloc(struct cxl_port *port, unsigned int nr_targets);struct cxl_switch_decoder *cxl_switch_decoder_alloc (struct cxl_port *port,unsigned int nr_targets); -int cxl_decoder_add (struct cxl_decoder *cxld,int *target_map); +int cxl_decoder_add (struct cxl_decoder *cxld);struct cxl_endpoint_decoder *cxl_endpoint_decoder_alloc (struct cxl_port *port); -int cxl_decoder_add_locked (struct cxl_decoder *cxld,int *target_map); +int cxl_decoder_add_locked (struct cxl_decoder *cxld);int cxl_decoder_autoremove (struct device *host,struct cxl_decoder *cxld);static inline int cxl_root_decoder_autoremove (struct device *host,struct cxl_root_decoder *cxlrd)
tools/testing/cxl/test/cxl.c +4 -4
@@ -651,7 +651,7 @@ static int mock_cxl_add_passthrough_decoder(struct cxl_port *port)struct target_map_ctx { -int *target_map; +u32 *target_map;int index;int target_count; };
@@ -863,9 +863,7 @@ static int mock_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm, target_count =NR_CXL_SWITCH_PORTS ;for (i =0 ; i <NR_CXL_PORT_DECODERS ; i++) { -int target_map[CXL_DECODER_MAX_INTERLEAVE ] = {0 };struct target_map_ctx ctx = { - .target_map = target_map, .target_count = target_count, };struct cxl_decoder *cxld;
@@ -894,6 +892,8 @@ static int mock_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm, cxld = &cxled->cxld; } + ctx.target_map = cxld->target_map; +mock_init_hdm_decoder (cxld);if (target_count) {
@@ -905,7 +905,7 @@ static int mock_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm, } } - rc =cxl_decoder_add_locked (cxld, target_map); + rc =cxl_decoder_add_locked (cxld);if (rc) {put_device (&cxld->dev);dev_err (&port->dev,"Failed to add decoder\n" );
Signature · 1 line
2.50.1
Re: [PATCH v9 03/10] cxl: Add a cached copy of target_map to cxl_decoder
Quoted 3 lines · Dave Jiang
Add a cached copy of the hardware port-id list that is available at init before all @dport objects have been instantiated. Change is in preparation of delayed dport instantiation.
Re: [PATCH v9 03/10] cxl: Add a cached copy of target_map to cxl_decoder
In reply to Dave JiangQuoted 21 lines · Dave Jiang
Add a cached copy of the hardware port-id list that is available at init before all @dport objects have been instantiated. Change is in preparation of delayed dport instantiation.
Reviewed-by Robert Richter <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Tested-by Robert Richter <[email protected]>Signed-off-by Dave Jiang <[email protected]>---
v9:
- Add regiontarget_mapupdate for manual assemble--- drivers/cxl/acpi.c | 7 +++---- drivers/cxl/core/hdm.c | 20 ++++++++------------ drivers/cxl/core/port.c | 22 +++++++--------------- drivers/cxl/core/region.c | 4 +++- drivers/cxl/cxl.h | 8 ++++++-- tools/testing/cxl/test/cxl.c | 8 ++++---- 6 files changed, 31 insertions(+), 38 deletions(-)
drivers/cxl/core/hdm.c static int cxl_setup_hdm_decoder_from_dvsec( · line 986
Quoted 6 lines
drivers/cxl/core/hdm.c +1
-1@@ -984,7 +982,7 @@ static int cxl_setup_hdm_decoder_from_dvsec( }static int init_hdm_decoder (struct cxl_port *port,struct cxl_decoder *cxld, -int *target_map,void __iomem *hdm,int which, +void __iomem *hdm,int which,u64 *dpa_base,struct cxl_endpoint_dvsec_info *info)
Stumbled over that again in v9: Use 80 char limit here.
Anyway, not a show-stopper.
-Robert
Dave Jiang
maintainerPATCH v9 4/10 cxl: Move port register setup to first dport appear
In reply to Dave JiangThis patch moves the port register setup to when the first dport appears via the memdev probe path. At this point, the CXL link should be established and the register access is expected to succeed. This change addresses an error message observed when PCIe hotplug is enabled on an Intel platform. The error messages "cxl portN: Couldn't locate the CXL.cache and CXL.mem capability array header" is observed for the hostbridge during cxl_acpi driver probe. If the cxl_acpi module probe is running before the CXL link between the endpoint device and the RP is established, then the platform may not have exposed DVSEC ID 3 and/or DVSEC ID 7 blocks which will trigger the error message. This behavior is defined by the spec and not a hardware quirk.
This change also needs the dport enumeration to be moved to the memdev probe path in order to address the issue. This change is just part of the code refactoring and is not a wholly contained fix itself.
--- drivers/cxl/core/port.c | 16 +++++++++++++--- drivers/cxl/cxl.h | 2 ++ 2 files changed, 15 insertions(+), 3 deletions(-)
drivers/cxl/core/port.c +13 -3
@@ -867,9 +867,7 @@ static int cxl_port_add(struct cxl_port *port,if (rc)return rc; - rc =cxl_port_setup_regs (port, component_reg_phys); -if (rc) -return rc; + port->component_reg_phys = component_reg_phys; }else { rc =dev_set_name (dev,"root%d" , port->id);if (rc)
@@ -1200,6 +1198,18 @@ __devm_cxl_add_dport(struct cxl_port *port, struct device *dport_dev,cxl_debugfs_create_dport_dir (dport); +/* +* Setup port register if this is the first dport showed up. Having +* a dport also means that there is at least 1 active link. +*/ +if (port->nr_dports ==1 && + port->component_reg_phys !=CXL_RESOURCE_NONE ) { + rc =cxl_port_setup_regs (port, port->component_reg_phys); +if (rc) +return ERR_PTR (rc); + port->component_reg_phys =CXL_RESOURCE_NONE ; + } +return dport; }
drivers/cxl/cxl.h +2 -0
@@ -599,6 +599,7 @@ struct cxl_dax_region {* @cdat: Cached CDAT data * @cdat_available: Should a CDAT attribute be available in sysfs * @pci_latency: Upstream latency in picoseconds +* @component_reg_phys: Physical address of component register */ struct cxl_port {struct device dev;
@@ -622,6 +623,7 @@ struct cxl_port { } cdat;bool cdat_available;long pci_latency; +resource_size_t component_reg_phys; };/**
Signature · 1 line
2.50.1
Re: [PATCH v9 04/10] cxl: Move port register setup to first dport appear
Quoted 12 lines · Dave Jiang
This patch moves the port register setup to when the first dport appears via the memdev probe path. At this point, the CXL link should be established and the register access is expected to succeed. This change addresses an error message observed when PCIe hotplug is enabled on an Intel platform. The error messages "cxl portN: Couldn't locate the CXL.cache and CXL.mem capability array header" is observed for the hostbridge during
cxl_acpidriver probe. If thecxl_acpimodule probe is running before the CXL link between the endpoint device and the RP is established, then the platform may not have exposed DVSEC ID 3 and/or DVSEC ID 7 blocks which will trigger the error message. This behavior is defined by the spec and not a hardware quirk.
Quoted 3 lines
This change also needs the dport enumeration to be moved to the memdev probe path in order to address the issue. This change is just part of the code refactoring and is not a wholly contained fix itself.
Dave Jiang
maintainerPATCH v9 5/10 cxl/test: Refactor decoder setup to reduce cxl_test burden
In reply to Dave JiangGroup the decoder setup code in switch and endpoint port probe into a single function for each to reduce the number of functions to be mocked in cxl_test. Introduce devm_cxl_switch_port_decoders_setup() and devm_cxl_endpoint_decoders_setup(). These two functions will be mocked instead with some functions optimized out since the mock version does not do anything. Remove devm_cxl_setup_hdm(), devm_cxl_add_passthrough_decoder(), and devm_cxl_enumerate_decoders() in cxl_test mock code. In turn, mock_cxl_add_passthrough_decoder() can be removed since cxl_test does not setup passthrough decoders. __wrap_cxl_hdm_decode_init() and __wrap_cxl_dvsec_rr_decode() can be removed as well since they only return 0 when called.
--- drivers/cxl/core/core.h | 5 +++ drivers/cxl/core/hdm.c | 81 +++++++++++++++++++++++++++++++---- drivers/cxl/core/pci.c | 42 ++++++++++++++++++ drivers/cxl/cxl.h | 10 ++--- drivers/cxl/cxlpci.h | 2 - drivers/cxl/port.c | 38 +--------------- tools/testing/cxl/Kbuild | 7 +-- tools/testing/cxl/test/cxl.c | 42 +++++++++++++----- tools/testing/cxl/test/mock.c | 69 ++++------------------------- tools/testing/cxl/test/mock.h | 7 +-- 10 files changed, 170 insertions(+), 133 deletions(-)
drivers/cxl/core/core.h +5 -0
@@ -147,6 +147,11 @@ int cxl_ras_init(void);void cxl_ras_exit (void );int cxl_gpf_port_setup (struct cxl_dport *dport); +struct cxl_hdm ; +int cxl_hdm_decode_init (struct cxl_dev_state *cxlds,struct cxl_hdm *cxlhdm, +struct cxl_endpoint_dvsec_info *info); +int cxl_port_get_possible_dports (struct cxl_port *port); +#ifdef CONFIG_CXL_FEATURES struct cxl_feat_entry *cxl_feature_info (struct cxl_features_state *cxlfs,const uuid_t *uuid);
drivers/cxl/core/hdm.c +73 -8
114 lines not on this page · raw message
drivers/cxl/core/pci.c +42 -0
45 lines not on this page · raw message
drivers/cxl/cxl.h +4 -6
@@ -812,12 +812,10 @@ struct cxl_endpoint_dvsec_info {struct range dvsec_range[2 ]; }; -struct cxl_hdm ; -struct cxl_hdm *devm_cxl_setup_hdm (struct cxl_port *port, -struct cxl_endpoint_dvsec_info *info); -int devm_cxl_enumerate_decoders (struct cxl_hdm *cxlhdm, -struct cxl_endpoint_dvsec_info *info); -int devm_cxl_add_passthrough_decoder (struct cxl_port *port); +struct cxl_port ; +int devm_cxl_switch_port_decoders_setup (struct cxl_port *port); +int devm_cxl_endpoint_decoders_setup (struct cxl_port *port); +struct cxl_dev_state ;int cxl_dvsec_rr_decode (struct cxl_dev_state *cxlds,struct cxl_endpoint_dvsec_info *info);
drivers/cxl/cxlpci.h +0 -2
@@ -129,8 +129,6 @@ static inline bool cxl_pci_flit_256(struct pci_dev *pdev)int devm_cxl_port_enumerate_dports (struct cxl_port *port);struct cxl_dev_state ; -int cxl_hdm_decode_init (struct cxl_dev_state *cxlds,struct cxl_hdm *cxlhdm, -struct cxl_endpoint_dvsec_info *info);void read_cdat_data (struct cxl_port *port);void cxl_cor_error_detected (struct pci_dev *pdev); pci_ers_result_t cxl_error_detected(struct pci_dev *pdev,
drivers/cxl/port.c +2 -36
@@ -59,7 +59,6 @@ static int discover_region(struct device *dev, void *unused)static int cxl_switch_port_probe (struct cxl_port *port) { -struct cxl_hdm *cxlhdm;int rc;/* Cache the data early to ensure is_visible() works */
@@ -71,43 +70,14 @@ static int cxl_switch_port_probe(struct cxl_port *port)cxl_switch_parse_cdat (port); - cxlhdm =devm_cxl_setup_hdm (port,NULL ); -if (!IS_ERR (cxlhdm)) -return devm_cxl_enumerate_decoders (cxlhdm,NULL ); - -if (PTR_ERR (cxlhdm) != -ENODEV ) { -dev_err (&port->dev,"Failed to map HDM decoder capability\n" ); -return PTR_ERR (cxlhdm); - } - -if (rc ==1 ) { -dev_dbg (&port->dev,"Fallback to passthrough decoder\n" ); -return devm_cxl_add_passthrough_decoder (port); - } - -dev_err (&port->dev,"HDM decoder capability not found\n" ); -return -ENXIO ; +return devm_cxl_switch_port_decoders_setup (port); }static int cxl_endpoint_port_probe (struct cxl_port *port) { -struct cxl_endpoint_dvsec_info info = { .port = port };struct cxl_memdev *cxlmd =to_cxl_memdev (port->uport_dev); -struct cxl_dev_state *cxlds = cxlmd->cxlds; -struct cxl_hdm *cxlhdm;int rc; - rc =cxl_dvsec_rr_decode (cxlds, &info); -if (rc <0 ) -return rc; - - cxlhdm =devm_cxl_setup_hdm (port, &info); -if (IS_ERR (cxlhdm)) { -if (PTR_ERR (cxlhdm) == -ENODEV ) -dev_err (&port->dev,"HDM decoder registers not found\n" ); -return PTR_ERR (cxlhdm); - } -/* Cache the data early to ensure is_visible() works */ read_cdat_data (port);cxl_endpoint_parse_cdat (port);
@@ -117,11 +87,7 @@ static int cxl_endpoint_port_probe(struct cxl_port *port)if (rc)return rc; - rc =cxl_hdm_decode_init (cxlds, cxlhdm, &info); -if (rc) -return rc; - - rc =devm_cxl_enumerate_decoders (cxlhdm, &info); + rc =devm_cxl_endpoint_decoders_setup (port);if (rc)return rc;
tools/testing/cxl/Kbuild +2 -5
@@ -5,16 +5,13 @@ ldflags-y += --wrap=acpi_evaluate_integer ldflags-y += --wrap=acpi_pci_find_root ldflags-y += --wrap=nvdimm_bus_register ldflags-y += --wrap=devm_cxl_port_enumerate_dports -ldflags-y += --wrap=devm_cxl_setup_hdm -ldflags-y += --wrap=devm_cxl_add_passthrough_decoder -ldflags-y += --wrap=devm_cxl_enumerate_decoders ldflags-y += --wrap=cxl_await_media_ready -ldflags-y += --wrap=cxl_hdm_decode_init -ldflags-y += --wrap=cxl_dvsec_rr_decode ldflags-y += --wrap=devm_cxl_add_rch_dport ldflags-y += --wrap=cxl_rcd_component_reg_phys ldflags-y += --wrap=cxl_endpoint_parse_cdat ldflags-y += --wrap=cxl_dport_init_ras_reporting +ldflags-y += --wrap=devm_cxl_switch_port_decoders_setup +ldflags-y += --wrap=devm_cxl_endpoint_decoders_setup DRIVERS := ../../../drivers CXL_SRC := $(DRIVERS)/cxl
tools/testing/cxl/test/cxl.c +32 -10
61 lines not on this page · raw message
tools/testing/cxl/test/mock.c +8 -61
102 lines not on this page · raw message
tools/testing/cxl/test/mock.h +2 -5
@@ -20,11 +20,8 @@ struct cxl_mock_ops {bool (*is_mock_port)(struct device *dev);bool (*is_mock_dev)(struct device *dev);int (*devm_cxl_port_enumerate_dports)(struct cxl_port *port); -struct cxl_hdm *(*devm_cxl_setup_hdm)( -struct cxl_port *port,struct cxl_endpoint_dvsec_info *info); -int (*devm_cxl_add_passthrough_decoder)(struct cxl_port *port); -int (*devm_cxl_enumerate_decoders)( -struct cxl_hdm *hdm,struct cxl_endpoint_dvsec_info *info); +int (*devm_cxl_switch_port_decoders_setup)(struct cxl_port *port); +int (*devm_cxl_endpoint_decoders_setup)(struct cxl_port *port);void (*cxl_endpoint_parse_cdat)(struct cxl_port *port); };
Signature · 1 line
2.50.1
Re: [PATCH v9 05/10] cxl/test: Refactor decoder setup to reduce cxl_test burden
Quoted 12 lines · On Fri, 29 Aug 2025 11:09:23 -0700 Dave Jiang <[email protected]> wrote:
Group the decoder setup code in switch and endpoint port probe into a single function for each to reduce the number of functions to be mocked in
cxl_test. Introducedevm_cxl_switch_port_decoders_setup()anddevm_cxl_endpoint_decoders_setup(). These two functions will be mocked instead with some functions optimized out since the mock version does not do anything. Removedevm_cxl_setup_hdm(),devm_cxl_add_passthrough_decoder(), anddevm_cxl_enumerate_decoders()incxl_testmock code. In turn,mock_cxl_add_passthrough_decoder()can be removed sincecxl_testdoes not setup passthrough decoders.__wrap_cxl_hdm_decode_init()and__wrap_cxl_dvsec_rr_decode()can be removed as well since they only return 0 when called.
Quoted 2 lines
Suggested-by Robert Richter <[email protected]>Signed-off-by Dave Jiang <[email protected]>
There was a bit of code reordering in here that made it a little hard to check but I can't see an easy way to make that more obvious. A precursor patch shuffling the code would perhaps allow it to be discussed in the patch description, but perhaps that's overkill
Anyhow, I'm fairly sure it is fine.
Quoted 29 lines
drivers/cxl/port.c +0
-14static int cxl_endpoint_port_probe (struct cxl_port *port) { -struct cxl_endpoint_dvsec_info info = { .port = port };struct cxl_memdev *cxlmd =to_cxl_memdev (port->uport_dev); -struct cxl_dev_state *cxlds = cxlmd->cxlds; -struct cxl_hdm *cxlhdm;int rc; - rc =cxl_dvsec_rr_decode (cxlds, &info); -if (rc <0 ) -return rc; - - cxlhdm =devm_cxl_setup_hdm (port, &info); -if (IS_ERR (cxlhdm)) { -if (PTR_ERR (cxlhdm) == -ENODEV ) -dev_err (&port->dev,"HDM decoder registers not found\n" ); -return PTR_ERR (cxlhdm); - } -/* Cache the data early to ensure is_visible() works */ read_cdat_data (port);cxl_endpoint_parse_cdat (port);
drivers/cxl/port.c static int cxl_endpoint_port_probe(struct cxl_port *port) · line 88
Quoted 2 lines
drivers/cxl/port.c +0
-0@@ -117,11 +87,7 @@ static int cxl_endpoint_port_probe(struct cxl_port *port)if (rc)return rc;
Hmm. The reordering makes me a little nervous but I 'think' it's ok as the code in between is all related to the cxlmd which none of the code above is directly related to.
Quoted 9 lines
drivers/cxl/port.c +1
-5- rc =cxl_hdm_decode_init (cxlds, cxlhdm, &info); -if (rc) -return rc; - - rc =devm_cxl_enumerate_decoders (cxlhdm, &info); + rc =devm_cxl_endpoint_decoders_setup (port);if (rc)return rc;
Re: [PATCH v9 05/10] cxl/test: Refactor decoder setup to reduce cxl_test burden
In reply to Dave JiangQuoted 11 lines · Dave Jiang
Group the decoder setup code in switch and endpoint port probe into a single function for each to reduce the number of functions to be mocked in
cxl_test. Introducedevm_cxl_switch_port_decoders_setup()anddevm_cxl_endpoint_decoders_setup(). These two functions will be mocked instead with some functions optimized out since the mock version does not do anything. Removedevm_cxl_setup_hdm(),devm_cxl_add_passthrough_decoder(), anddevm_cxl_enumerate_decoders()incxl_testmock code. In turn,mock_cxl_add_passthrough_decoder()can be removed sincecxl_testdoes not setup passthrough decoders.__wrap_cxl_hdm_decode_init()and__wrap_cxl_dvsec_rr_decode()can be removed as well since they only return 0 when called.
Nice refactor!
Re: [PATCH v9 05/10] cxl/test: Refactor decoder setup to reduce cxl_test burden
In reply to Dave JiangQuoted 12 lines · Dave Jiang
Group the decoder setup code in switch and endpoint port probe into a single function for each to reduce the number of functions to be mocked in
cxl_test. Introducedevm_cxl_switch_port_decoders_setup()anddevm_cxl_endpoint_decoders_setup(). These two functions will be mocked instead with some functions optimized out since the mock version does not do anything. Removedevm_cxl_setup_hdm(),devm_cxl_add_passthrough_decoder(), anddevm_cxl_enumerate_decoders()incxl_testmock code. In turn,mock_cxl_add_passthrough_decoder()can be removed sincecxl_testdoes not setup passthrough decoders.__wrap_cxl_hdm_decode_init()and__wrap_cxl_dvsec_rr_decode()can be removed as well since they only return 0 when called.
Quoted 2 lines
Suggested-by Robert Richter <[email protected]>Signed-off-by Dave Jiang <[email protected]>
Great rework. I saw you applied it already, but in case you rebase again:
One small comment:
Quoted 5 lines
drivers/cxl/cxl.h +0
-0
drivers/cxl/cxl.h struct cxl_endpoint_dvsec_info { · line 815
Quoted 10 lines
drivers/cxl/cxl.h +1
-6@@ -812,12 +812,10 @@ struct cxl_endpoint_dvsec_info {struct range dvsec_range[2 ]; }; -struct cxl_hdm ; -struct cxl_hdm *devm_cxl_setup_hdm (struct cxl_port *port, -struct cxl_endpoint_dvsec_info *info); -int devm_cxl_enumerate_decoders (struct cxl_hdm *cxlhdm, -struct cxl_endpoint_dvsec_info *info); -int devm_cxl_add_passthrough_decoder (struct cxl_port *port); +struct cxl_port ;
cxl_port is already defined here. It can be dropped.
Feel free to fold the change in. On top patch fine with me too.
Thanks,
-Robert
Quoted 6 lines
drivers/cxl/cxl.h +3
-0+int devm_cxl_switch_port_decoders_setup (struct cxl_port *port); +int devm_cxl_endpoint_decoders_setup (struct cxl_port *port); +struct cxl_dev_state ;int cxl_dvsec_rr_decode (struct cxl_dev_state *cxlds,struct cxl_endpoint_dvsec_info *info);
Dave Jiang
maintainerPATCH v9 6/10 cxl: Defer dport allocation for switch ports
In reply to Dave JiangThe current implementation enumerates the dports during the cxl_port driver probe. Without an endpoint connected, the dport may not be active during port probe. This scheme may prevent a valid hardware dport id to be retrieved and MMIO registers to be read when an endpoint is hot-plugged. Move the dport allocation and setup to behind memdev probe so the endpoint is guaranteed to be connected.
In the original enumeration behavior, there are 3 phases (or 2 if no CXL switches) for port creation. cxl_acpi() creates a Root Port (RP) from the ACPI0017.N device. Through that it enumerates downstream ports composed of ACPI0016.N devices through add_host_bridge_dport(). Once done, it uses add_host_bridge_uport() to create the ports that enumerate the PCI RPs as the dports of these ports. Every time a port is created, the port driver is attached, cxl_switch_porbe_probe() is called and devm_cxl_port_enumerate_dports() is invoked to enumerate and probe the dports.
The second phase is if there are any CXL switches. When the pci endpoint device driver (cxl_pci) calls probe, it will add a mem device and triggers the cxl_mem_probe(). cxl_mem_probe() calls devm_cxl_enumerate_ports() and attempts to discovery and create all the ports represent CXL switches. During this phase, a port is created per switch and the attached dports are also enumerated and probed.
The last phase is creating endpoint port which happens for all endpoint devices.
The new sequence is instead of creating all possible dports at initial port creation, defer port instantiation until a memdev beneath that dport arrives. Introduce devm_cxl_create_or_extend_port() to centralize the creation and extension of ports with new dports as memory devices arrive. As part of this rework, switch decoder target list is amended at runtime as dports show up.
While the decoders are allocated during the port driver probe, The decoders must also be updated since previously it's all done when all the dports are setup and now every time a dport is setup per endpoint, the switch target listing need to be updated with new dport. A guard(rwsem_write) is used to update decoder targets. This is similar to when decoder_populate_target() is called and the decoder programming must be protected.
Also the port registers are probed the first time when the first dport shows up. This ensures that the CXL link is established when the port registers are probed.
---
v9:
- Dropped tested-by tag by Robert since significant code changes. Also dropped review tags.
- Rework iteration loop. (Robert)
- Dropped possible_ports in 'struct cxl_port' (Robert)
- Stop the update_decoder_targets() iteration when found. (Robert)
- Use number of decoders to gate decoder allocation.
- cxl_port_get_or_add_dport() ->cxl_port_add_dport(). (Robert)
- Unfold cxl_decoders_dport_update() (Robert)
- Remove devm_cxl_add_dport_by_uport() (Robert)
--- drivers/cxl/core/cdat.c | 2 +- drivers/cxl/core/core.h | 2 + drivers/cxl/core/hdm.c | 6 - drivers/cxl/core/pci.c | 46 ++++++++ drivers/cxl/core/port.c | 240 ++++++++++++++++++++++++++++++++-------- drivers/cxl/port.c | 11 +- 6 files changed, 247 insertions(+), 60 deletions(-)
drivers/cxl/core/cdat.c +1 -1
@@ -338,7 +338,7 @@ static int match_cxlrd_hb(struct device *dev, void *data)guard (rwsem_read)(&cxl_rwsem.region);for (int i =0 ; i < cxlsd->nr_targets; i++) { -if (host_bridge == cxlsd->target[i]->dport_dev) +if (cxlsd->target[i] && host_bridge == cxlsd->target[i]->dport_dev)return 1 ; }
drivers/cxl/core/core.h +2 -0
@@ -146,6 +146,8 @@ int cxl_port_get_switch_dport_bandwidth(struct cxl_port *port,int cxl_ras_init (void );void cxl_ras_exit (void );int cxl_gpf_port_setup (struct cxl_dport *dport); +struct cxl_dport *devm_cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev);struct cxl_hdm ; int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm,
drivers/cxl/core/hdm.c +0 -6
@@ -52,8 +52,6 @@ static int add_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld)static int devm_cxl_add_passthrough_decoder (struct cxl_port *port) {struct cxl_switch_decoder *cxlsd; -struct cxl_dport *dport =NULL ; -unsigned long index;struct cxl_hdm *cxlhdm =dev_get_drvdata (&port->dev);/*
@@ -69,10 +67,6 @@ static int devm_cxl_add_passthrough_decoder(struct cxl_port *port)device_lock_assert (&port->dev); -xa_for_each (&port->dports, index, dport) -break ; - cxlsd->cxld.target_map[0 ] = dport->port_id; -return add_hdm_decoder (port, &cxlsd->cxld); }
drivers/cxl/core/pci.c +46 -0
52 lines not on this page · raw message
drivers/cxl/core/port.c +195 -45
303 lines not on this page · raw message
drivers/cxl/port.c +3 -8
@@ -59,18 +59,13 @@ static int discover_region(struct device *dev, void *unused)static int cxl_switch_port_probe (struct cxl_port *port) { -int rc; +/* Reset nr_dports for rebind of driver */ + port->nr_dports =0 ;/* Cache the data early to ensure is_visible() works */ read_cdat_data (port); - rc =devm_cxl_port_enumerate_dports (port); -if (rc <0 ) -return rc; - -cxl_switch_parse_cdat (port); - -return devm_cxl_switch_port_decoders_setup (port); +return 0 ; }static int cxl_endpoint_port_probe (struct cxl_port *port)
Signature · 1 line
2.50.1
Re: [PATCH v9 06/10] cxl: Defer dport allocation for switch ports
Quoted 34 lines · On Fri, 29 Aug 2025 11:09:24 -0700 Dave Jiang <[email protected]> wrote:
The current implementation enumerates the dports during the
cxl_portdriver probe. Without an endpoint connected, the dport may not be active during port probe. This scheme may prevent a valid hardware dport id to be retrieved and MMIO registers to be read when an endpoint is hot-plugged. Move the dport allocation and setup to behind memdev probe so the endpoint is guaranteed to be connected.In the original enumeration behavior, there are 3 phases (or 2 if no CXL switches) for port creation.
cxl_acpi()creates a Root Port (RP) from the ACPI0017.N device. Through that it enumerates downstream ports composed of ACPI0016.N devices throughadd_host_bridge_dport(). Once done, it usesadd_host_bridge_uport()to create the ports that enumerate the PCI RPs as the dports of these ports. Every time a port is created, the port driver is attached,cxl_switch_porbe_probe()is called anddevm_cxl_port_enumerate_dports()is invoked to enumerate and probe the dports.The second phase is if there are any CXL switches. When the pci endpoint device driver (
cxl_pci) calls probe, it will add a mem device and triggers thecxl_mem_probe().cxl_mem_probe()callsdevm_cxl_enumerate_ports()and attempts to discovery and create all the ports represent CXL switches. During this phase, a port is created per switch and the attached dports are also enumerated and probed.The last phase is creating endpoint port which happens for all endpoint devices.
The new sequence is instead of creating all possible dports at initial port creation, defer port instantiation until a memdev beneath that dport arrives. Introduce
devm_cxl_create_or_extend_port()to centralize the creation and extension of ports with new dports as memory devices arrive. As part of this rework, switch decoder target list is amended at runtime as dports show up.
Quoted 2 lines
While the decoders are allocated during the port driver probe, The decoders must also be updated since previously it's all done when all
it was all done?
Quoted 6 lines
the dports are setup and now every time a dport is setup per endpoint, the switch target listing need to be updated with new dport. A
guard(rwsem_write)is used to update decoder targets. This is similar to whendecoder_populate_target()is called and the decoder programming must be protected.
Quoted 6 lines
Also the port registers are probed the first time when the first dport shows up. This ensures that the CXL link is established when the port registers are probed.
Hi Dave,
This isn't one of my more confident reviews but I think this looks ok. Definitely could do with more eyes though. In the ideal world I'd emulate a bunch of test cases to poke this with, but I'm snowed under for a while so hopefully testing on the platforms that exhibit the problem will be enough.
One trivial comment inline.
Quoted 41 lines
---
v9:
- Dropped tested-by tag by Robert since significant code changes. Also dropped review tags.
- Rework iteration loop. (Robert)
- Droppedpossible_portsin 'structcxl_port' (Robert)
- Stop theupdate_decoder_targets()iteration when found. (Robert)
- Use number of decoders to gate decoder allocation.
-cxl_port_get_or_add_dport()->cxl_port_add_dport(). (Robert)
- Unfoldcxl_decoders_dport_update()(Robert)
- Removedevm_cxl_add_dport_by_uport()(Robert)drivers/cxl/core/pci.c +0
-0struct cxl_walk_context {struct pci_bus *bus;struct cxl_port *port;drivers/cxl/core/port.c +14
-0+static struct cxl_dport *devm_cxl_create_port (struct device *ep_dev, +struct cxl_port *parent_port, +struct cxl_dport *parent_dport, +struct device *uport_dev, +struct device *dport_dev) +{ +resource_size_t component_reg_phys; + +device_lock_assert (&parent_port->dev); +if (!parent_port->dev.driver) { +dev_warn (ep_dev, +"port %s:%s:%s disabled, failed to enumerate CXL.mem\n" , +dev_name (&parent_port->dev),dev_name (uport_dev), +dev_name (dport_dev));
drivers/cxl/core/port.c static resource_size_t find_component_registers(struct device *dev) · line 1663
Quoted 10 lines
drivers/cxl/core/port.c +10
-0+ } + + struct cxl_port *port __free(put_cxl_port) = +find_cxl_port_by_uport (uport_dev); +if (!port) { + component_reg_phys =find_component_registers (uport_dev); + port =devm_cxl_add_port (&parent_port->dev, uport_dev, + component_reg_phys, parent_dport); +if (IS_ERR (port)) +return (struct cxl_dport *)port;
return ERR_CAST(port);
Just to make it even more obvious what is going on.
Quoted 22 lines
drivers/cxl/core/port.c +22
-0+ +/* +* retry to make sure a port is found. a port device +* reference is taken. +*/ + port =find_cxl_port_by_uport (uport_dev); +if (!port) +return ERR_PTR (-ENODEV ); + +dev_dbg (ep_dev,"created port %s:%s\n" , +dev_name (&port->dev),dev_name (port->uport_dev)); + }else { +/* +* Port was created before right before this function is +* called. Signal the caller to deal with it. +*/ +return ERR_PTR (-EAGAIN ); + } + +guard (device)(&port->dev); +return cxl_port_add_dport (port, dport_dev); +}
Re: [PATCH v9 06/10] cxl: Defer dport allocation for switch ports
In reply to Dave JiangQuoted 42 lines · Dave Jiang
The current implementation enumerates the dports during the
cxl_portdriver probe. Without an endpoint connected, the dport may not be active during port probe. This scheme may prevent a valid hardware dport id to be retrieved and MMIO registers to be read when an endpoint is hot-plugged. Move the dport allocation and setup to behind memdev probe so the endpoint is guaranteed to be connected.In the original enumeration behavior, there are 3 phases (or 2 if no CXL switches) for port creation.
cxl_acpi()creates a Root Port (RP) from the ACPI0017.N device. Through that it enumerates downstream ports composed of ACPI0016.N devices throughadd_host_bridge_dport(). Once done, it usesadd_host_bridge_uport()to create the ports that enumerate the PCI RPs as the dports of these ports. Every time a port is created, the port driver is attached,cxl_switch_porbe_probe()is called anddevm_cxl_port_enumerate_dports()is invoked to enumerate and probe the dports.The second phase is if there are any CXL switches. When the pci endpoint device driver (
cxl_pci) calls probe, it will add a mem device and triggers thecxl_mem_probe().cxl_mem_probe()callsdevm_cxl_enumerate_ports()and attempts to discovery and create all the ports represent CXL switches. During this phase, a port is created per switch and the attached dports are also enumerated and probed.The last phase is creating endpoint port which happens for all endpoint devices.
The new sequence is instead of creating all possible dports at initial port creation, defer port instantiation until a memdev beneath that dport arrives. Introduce
devm_cxl_create_or_extend_port()to centralize the creation and extension of ports with new dports as memory devices arrive. As part of this rework, switch decoder target list is amended at runtime as dports show up.While the decoders are allocated during the port driver probe, The decoders must also be updated since previously it's all done when all the dports are setup and now every time a dport is setup per endpoint, the switch target listing need to be updated with new dport. A
guard(rwsem_write)is used to update decoder targets. This is similar to whendecoder_populate_target()is called and the decoder programming must be protected.
Quoted 6 lines
Also the port registers are probed the first time when the first dport shows up. This ensures that the CXL link is established when the port registers are probed.
Dave Jiang
maintainerPATCH v9 7/10 cxl/test: Add mock version of devm_cxl_add_dport_by_dev()
In reply to Dave Jiangdevm_cxl_add_dport_by_dev() outside of cxl_test is done through PCI hierarchy. However with cxl_test, it needs to be done through the platform device hierarchy. Add the mock function for devm_cxl_add_dport_by_dev().
When cxl_core calls a cxl_core exported function and that function is mocked by cxl_test, the call chain causes a circular dependency issue. Dan provided a workaround to avoid this issue. Apply the method to changes from the late dport allocation changes in order to enable cxl-test.
In cxl_core they are defined with "__" added in front of the function. A macro is used to define the original function names for when non-test version of the kernel is built. A bit of macros and typedefs are used to allow mocking of those functions in cxl_test.
--- drivers/cxl/core/core.h | 2 -- drivers/cxl/core/pci.c | 7 ++-- drivers/cxl/cxl.h | 20 +++++++++++ tools/testing/cxl/Kbuild | 1 + tools/testing/cxl/cxl_core_exports.c | 12 +++++++ tools/testing/cxl/exports.h | 10 ++++++ tools/testing/cxl/test/cxl.c | 53 ++++++++++++++++++++++++++-- tools/testing/cxl/test/mock.c | 23 ++++++++++++ tools/testing/cxl/test/mock.h | 2 ++ 9 files changed, 123 insertions(+), 7 deletions(-) create mode 100644 tools/testing/cxl/exports.h
drivers/cxl/core/core.h +0 -2
@@ -146,8 +146,6 @@ int cxl_port_get_switch_dport_bandwidth(struct cxl_port *port,int cxl_ras_init (void );void cxl_ras_exit (void );int cxl_gpf_port_setup (struct cxl_dport *dport); -struct cxl_dport *devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev);struct cxl_hdm ; int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm,
drivers/cxl/core/pci.c +4 -3
@@ -41,14 +41,14 @@ static int pci_get_port_num(struct pci_dev *pdev) }/** -* devm_cxl_add_dport_by_dev - allocate a dport by the dport device +* __devm_cxl_add_dport_by_dev - allocate a dport by dport device * @port: cxl_port that hosts the dport * @dport_dev: 'struct device' of the dport * * Returns the allocated dport on success or ERR_PTR() of -errno on error */ -struct cxl_dport *devm_cxl_add_dport_by_dev (struct cxl_port *port, -struct device *dport_dev) +struct cxl_dport *__devm_cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev) {struct cxl_register_map map;struct pci_dev *pdev;
@@ -69,6 +69,7 @@ struct cxl_dport *devm_cxl_add_dport_by_dev(struct cxl_port *port,device_lock_assert (&port->dev);return devm_cxl_add_dport (port, dport_dev, port_num, map.resource); } +EXPORT_SYMBOL_NS_GPL (__devm_cxl_add_dport_by_dev,"CXL" );struct cxl_walk_context {struct pci_bus *bus;
drivers/cxl/cxl.h +20 -0
@@ -909,6 +909,10 @@ void cxl_coordinates_combine(struct access_coordinate *out, struct access_coordinate *c2);bool cxl_endpoint_decoder_reset_detected (struct cxl_port *port); +struct cxl_dport *devm_cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev); +struct cxl_dport *__devm_cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev);/* * Unit test builds overrides this to __weak, find the 'strong' version
@@ -919,4 +923,20 @@ bool cxl_endpoint_decoder_reset_detected(struct cxl_port *port);#endif u16 cxl_gpf_get_dvsec (struct device *dev); + +/* +* Declaration for functions that are mocked by cxl_test that are called by +* cxl_core. The respective functions are defined as __foo() and called by +* cxl_core as foo(). The macros below ensures that those functions would +* exist as foo(). See tools/testing/cxl/cxl_core_exports.c and +* tools/testing/cxl/exports.h for setting up the mock functions. The dance +* is done to avoid a circular dependency where cxl_core calls a function that +* ends up being a mock function and goes to * cxl_test where it calls a +* cxl_core function. +*/ +#ifndef CXL_TEST_ENABLE +#define DECLARE_TESTABLE (x) __##x +#define devm_cxl_add_dport_by_dev DECLARE_TESTABLE(devm_cxl_add_dport_by_dev) +#endif +#endif /* __CXL_H__ */
tools/testing/cxl/Kbuild +1 -0
@@ -18,6 +18,7 @@ CXL_SRC := $(DRIVERS)/cxl CXL_CORE_SRC := $(DRIVERS)/cxl/core ccflags-y := -I$(srctree)/drivers/cxl/ ccflags-y += -D__mock=__weak +ccflags-y += -DCXL_TEST_ENABLE=1 ccflags-y += -DTRACE_INCLUDE_PATH=$(CXL_CORE_SRC) -I$(srctree)/drivers/cxl/core/ obj-m += cxl_acpi.o
tools/testing/cxl/cxl_core_exports.c +12 -0
@@ -2,6 +2,18 @@/* Copyright(c) 2022 Intel Corporation. All rights reserved. */ #include "cxl.h" +#include "exports.h" /* Exporting of cxl_core symbols that are only used by cxl_test */ EXPORT_SYMBOL_NS_GPL (cxl_num_decoders_committed,"CXL" ); + +cxl_add_dport_by_dev_fn _devm_cxl_add_dport_by_dev = + __devm_cxl_add_dport_by_dev; +EXPORT_SYMBOL_NS_GPL (_devm_cxl_add_dport_by_dev,"CXL" ); + +struct cxl_dport *devm_cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev) +{ +return _devm_cxl_add_dport_by_dev (port, dport_dev); +} +EXPORT_SYMBOL_NS_GPL (devm_cxl_add_dport_by_dev,"CXL" );
tools/testing/cxl/exports.h new file +10 -0
@@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2025 Intel Corporation */ +#ifndef __MOCK_CXL_EXPORTS_H_ +#define __MOCK_CXL_EXPORTS_H_ + +typedef struct cxl_dport *(*cxl_add_dport_by_dev_fn )(struct cxl_port *port, +struct device *dport_dev); +extern cxl_add_dport_by_dev_fn _devm_cxl_add_dport_by_dev; + +#endif
tools/testing/cxl/test/cxl.c +51 -2
79 lines not on this page · raw message
tools/testing/cxl/test/mock.c +23 -0
@@ -10,12 +10,18 @@#include <cxlmem.h> #include <cxlpci.h> #include "mock.h" +#include "../exports.h" static LIST_HEAD (mock); +static struct cxl_dport * +redirect_devm_cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev); +void register_cxl_mock_ops (struct cxl_mock_ops *ops) {list_add_rcu (&ops->list, &mock); + _devm_cxl_add_dport_by_dev = redirect_devm_cxl_add_dport_by_dev; }EXPORT_SYMBOL_GPL (register_cxl_mock_ops);
@@ -23,6 +29,7 @@ DEFINE_STATIC_SRCU(cxl_mock_srcu);void unregister_cxl_mock_ops (struct cxl_mock_ops *ops) { + _devm_cxl_add_dport_by_dev = __devm_cxl_add_dport_by_dev;list_del_rcu (&ops->list);synchronize_srcu (&cxl_mock_srcu); }
@@ -258,6 +265,22 @@ void __wrap_cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device }EXPORT_SYMBOL_NS_GPL (__wrap_cxl_dport_init_ras_reporting,"CXL" ); +struct cxl_dport *redirect_devm_cxl_add_dport_by_dev (struct cxl_port *port, +struct device *dport_dev) +{ +int index; +struct cxl_mock_ops *ops =get_cxl_mock_ops (&index); +struct cxl_dport *dport; + +if (ops && ops->is_mock_port (port->uport_dev)) + dport = ops->devm_cxl_add_dport_by_dev (port, dport_dev); +else + dport =__devm_cxl_add_dport_by_dev (port, dport_dev); +put_cxl_mock_ops (index); + +return dport; +} +MODULE_LICENSE ("GPL v2" );MODULE_DESCRIPTION ("cxl_test: emulation module" );MODULE_IMPORT_NS ("ACPI" );
tools/testing/cxl/test/mock.h +2 -0
@@ -23,6 +23,8 @@ struct cxl_mock_ops {int (*devm_cxl_switch_port_decoders_setup)(struct cxl_port *port);int (*devm_cxl_endpoint_decoders_setup)(struct cxl_port *port);void (*cxl_endpoint_parse_cdat)(struct cxl_port *port); +struct cxl_dport *(*devm_cxl_add_dport_by_dev)(struct cxl_port *port, +struct device *dport_dev); };void register_cxl_mock_ops (struct cxl_mock_ops *ops);
Signature · 1 line
2.50.1
Dave Jiang
maintainerPATCH v9 8/10 cxl/test: Adjust the mock version of devm_cxl_switch_port_decoders_setup()
In reply to Dave JiangWith devm_cxl_switch_port_decoders_setup() being called within cxl_core instead of by the port driver probe, adjustments are needed to deal with circular symbol dependency when this function is being mock'd. Add the appropriate changes to get around the circular dependency.
--- drivers/cxl/core/hdm.c | 6 +++--- drivers/cxl/cxl.h | 2 ++ tools/testing/cxl/Kbuild | 1 - tools/testing/cxl/cxl_core_exports.c | 10 ++++++++++ tools/testing/cxl/exports.h | 3 +++ tools/testing/cxl/test/mock.c | 10 +++++++--- 6 files changed, 25 insertions(+), 7 deletions(-)
drivers/cxl/core/hdm.c +3 -3
@@ -1216,12 +1216,12 @@ static int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm, }/** -* devm_cxl_switch_port_decoders_setup - allocate and setup switch decoders +* __devm_cxl_switch_port_decoders_setup - allocate and setup switch decoders * @port: CXL port context * * Return 0 or -errno on error */ -int devm_cxl_switch_port_decoders_setup (struct cxl_port *port) +int __devm_cxl_switch_port_decoders_setup (struct cxl_port *port) {struct cxl_hdm *cxlhdm;
@@ -1245,7 +1245,7 @@ int devm_cxl_switch_port_decoders_setup(struct cxl_port *port)dev_err (&port->dev,"HDM decoder capability not found\n" );return -ENXIO ; } -EXPORT_SYMBOL_NS_GPL (devm_cxl_switch_port_decoders_setup,"CXL" ); +EXPORT_SYMBOL_NS_GPL (__devm_cxl_switch_port_decoders_setup,"CXL" );/** * devm_cxl_endpoint_decoders_setup - allocate and setup endpoint decoders
drivers/cxl/cxl.h +2 -0
@@ -814,6 +814,7 @@ struct cxl_endpoint_dvsec_info {struct cxl_port ;int devm_cxl_switch_port_decoders_setup (struct cxl_port *port); +int __devm_cxl_switch_port_decoders_setup (struct cxl_port *port);int devm_cxl_endpoint_decoders_setup (struct cxl_port *port);struct cxl_dev_state ;
@@ -937,6 +938,7 @@ u16 cxl_gpf_get_dvsec(struct device *dev);#ifndef CXL_TEST_ENABLE #define DECLARE_TESTABLE (x) __##x#define devm_cxl_add_dport_by_dev DECLARE_TESTABLE(devm_cxl_add_dport_by_dev) +#define devm_cxl_switch_port_decoders_setup DECLARE_TESTABLE(devm_cxl_switch_port_decoders_setup)#endif #endif /* __CXL_H__ */
tools/testing/cxl/Kbuild +0 -1
@@ -10,7 +10,6 @@ ldflags-y += --wrap=devm_cxl_add_rch_dport ldflags-y += --wrap=cxl_rcd_component_reg_phys ldflags-y += --wrap=cxl_endpoint_parse_cdat ldflags-y += --wrap=cxl_dport_init_ras_reporting -ldflags-y += --wrap=devm_cxl_switch_port_decoders_setup ldflags-y += --wrap=devm_cxl_endpoint_decoders_setup DRIVERS := ../../../drivers
tools/testing/cxl/cxl_core_exports.c +10 -0
@@ -17,3 +17,13 @@ struct cxl_dport *devm_cxl_add_dport_by_dev(struct cxl_port *port,return _devm_cxl_add_dport_by_dev (port, dport_dev); }EXPORT_SYMBOL_NS_GPL (devm_cxl_add_dport_by_dev,"CXL" ); + +cxl_switch_decoders_setup_fn _devm_cxl_switch_port_decoders_setup = + __devm_cxl_switch_port_decoders_setup; +EXPORT_SYMBOL_NS_GPL (_devm_cxl_switch_port_decoders_setup,"CXL" ); + +int devm_cxl_switch_port_decoders_setup (struct cxl_port *port) +{ +return _devm_cxl_switch_port_decoders_setup (port); +} +EXPORT_SYMBOL_NS_GPL (devm_cxl_switch_port_decoders_setup,"CXL" );
tools/testing/cxl/exports.h +3 -0
@@ -7,4 +7,7 @@ typedef struct cxl_dport *(*cxl_add_dport_by_dev_fn)(struct cxl_port *port, struct device *dport_dev);extern cxl_add_dport_by_dev_fn _devm_cxl_add_dport_by_dev; +typedef int (*cxl_switch_decoders_setup_fn )(struct cxl_port *port); +extern cxl_switch_decoders_setup_fn _devm_cxl_switch_port_decoders_setup; +#endif
tools/testing/cxl/test/mock.c +7 -3
@@ -17,11 +17,14 @@ static LIST_HEAD(mock);static struct cxl_dport *redirect_devm_cxl_add_dport_by_dev (struct cxl_port *port,struct device *dport_dev); +static int redirect_devm_cxl_switch_port_decoders_setup (struct cxl_port *port);void register_cxl_mock_ops (struct cxl_mock_ops *ops) {list_add_rcu (&ops->list, &mock); _devm_cxl_add_dport_by_dev = redirect_devm_cxl_add_dport_by_dev; + _devm_cxl_switch_port_decoders_setup = + redirect_devm_cxl_switch_port_decoders_setup; }EXPORT_SYMBOL_GPL (register_cxl_mock_ops);
@@ -29,6 +32,8 @@ DEFINE_STATIC_SRCU(cxl_mock_srcu);void unregister_cxl_mock_ops (struct cxl_mock_ops *ops) { + _devm_cxl_switch_port_decoders_setup = + __devm_cxl_switch_port_decoders_setup; _devm_cxl_add_dport_by_dev = __devm_cxl_add_dport_by_dev;list_del_rcu (&ops->list);synchronize_srcu (&cxl_mock_srcu);
@@ -138,7 +143,7 @@ __wrap_nvdimm_bus_register(struct device *dev, }EXPORT_SYMBOL_GPL (__wrap_nvdimm_bus_register); -int __wrap_devm_cxl_switch_port_decoders_setup (struct cxl_port *port) +int redirect_devm_cxl_switch_port_decoders_setup (struct cxl_port *port) {int rc, index;struct cxl_mock_ops *ops =get_cxl_mock_ops (&index);
@@ -146,12 +151,11 @@ int __wrap_devm_cxl_switch_port_decoders_setup(struct cxl_port *port)if (ops && ops->is_mock_port (port->uport_dev)) rc = ops->devm_cxl_switch_port_decoders_setup (port);else - rc =devm_cxl_switch_port_decoders_setup (port); + rc =__devm_cxl_switch_port_decoders_setup (port);put_cxl_mock_ops (index);return rc; } -EXPORT_SYMBOL_NS_GPL (__wrap_devm_cxl_switch_port_decoders_setup,"CXL" );int __wrap_devm_cxl_endpoint_decoders_setup (struct cxl_port *port) {
Signature · 1 line
2.50.1
Re: [PATCH v9 08/10] cxl/test: Adjust the mock version of devm_cxl_switch_port_decoders_setup()
Quoted 6 lines · On Fri, 29 Aug 2025 11:09:26 -0700 Dave Jiang <[email protected]> wrote:
With
devm_cxl_switch_port_decoders_setup()being called withincxl_coreinstead of by the port driver probe, adjustments are needed to deal with circular symbol dependency when this function is being mock'd. Add the appropriate changes to get around the circular dependency.Signed-off-by Dave Jiang <[email protected]>
Looks fairly standard.
Re: [PATCH v9 08/10] cxl/test: Adjust the mock version of devm_cxl_switch_port_decoders_setup()
In reply to Dave JiangQuoted 4 lines · Dave Jiang
With
devm_cxl_switch_port_decoders_setup()being called withincxl_coreinstead of by the port driver probe, adjustments are needed to deal with circular symbol dependency when this function is being mock'd. Add the appropriate changes to get around the circular dependency.
snip
Dave Jiang
maintainerPATCH v9 9/10 cxl/test: Setup target_map for cxl_test decoder initialization
In reply to Dave Jiangcxl_test uses mock functions for decoder enumaration. Add initialization of the cxld->target_map[] for cxl_test based decoders in the mock functions.
--- tools/testing/cxl/test/cxl.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-)
tools/testing/cxl/test/cxl.c +9 -3
@@ -811,15 +811,21 @@ static void mock_init_hdm_decoder(struct cxl_decoder *cxld)*/ if (WARN_ON (!dev))continue ; + cxlsd =to_cxl_switch_decoder (dev);if (i ==0 ) {/* put cxl_mem.4 second in the decode order */ -if (pdev->id ==4 ) +if (pdev->id ==4 ) { cxlsd->target[1 ] = dport; -else + cxld->target_map[1 ] = dport->port_id; + }else { cxlsd->target[0 ] = dport; - }else + cxld->target_map[0 ] = dport->port_id; + } + }else { cxlsd->target[0 ] = dport; + cxld->target_map[0 ] = dport->port_id; + } cxld = &cxlsd->cxld; cxld->target_type =CXL_DECODER_HOSTONLYMEM ; cxld->flags =CXL_DECODER_F_ENABLE ;
Signature · 1 line
2.50.1
Re: [PATCH v9 09/10] cxl/test: Setup target_map for cxl_test decoder initialization
Quoted 3 lines · Dave Jiang
cxl_testuses mock functions for decoder enumaration. Add initialization of the cxld->target_map[] forcxl_testbased decoders in the mock functions.
Quoted 4 lines
Reviewed-by Jonathan Cameron <[email protected]>Tested-by Robert Richter <[email protected]>Signed-off-by Dave Jiang <[email protected]>
Dave Jiang
maintainerPATCH v9 10/10 cxl: Change sslbis handler to only handle single dport
In reply to Dave JiangWhile cxl_switch_parse_cdat() is harmless to be run multiple times, it is not efficient in the current scheme where one dport is being updated at a time by the memdev probe path. Change the input parameter to the specific dport being updated to pick up the SSLBIS information for just that dport.
--- drivers/cxl/core/cdat.c | 23 ++++++++++------------- drivers/cxl/core/port.c | 2 +- drivers/cxl/cxl.h | 2 +- 3 files changed, 12 insertions(+), 15 deletions(-)
drivers/cxl/core/cdat.c +10 -13
@@ -440,8 +440,8 @@ static int cdat_sslbis_handler(union acpi_subtable_headers *header, void *arg, } *tbl = (struct acpi_cdat_sslbis_table *)header;int size =sizeof (header->cdat) +sizeof (tbl->sslbis_header);struct acpi_cdat_sslbis *sslbis; -struct cxl_port *port = arg; -struct device *dev = &port->dev; +struct cxl_dport *dport = arg; +struct device *dev = &dport->port->dev;int remain, entries, i;u16 len;
@@ -467,8 +467,6 @@ static int cdat_sslbis_handler(union acpi_subtable_headers *header, void *arg,u16 y =le16_to_cpu ((__force __le16 )tbl->entries[i].porty_id);__le64 le_base;__le16 le_val; -struct cxl_dport *dport; -unsigned long index;u16 dsp_id;u64 val;
@@ -499,28 +497,27 @@ static int cdat_sslbis_handler(union acpi_subtable_headers *header, void *arg, val =cdat_normalize (le16_to_cpu (le_val),le64_to_cpu (le_base), sslbis->data_type); -xa_for_each (&port->dports, index, dport) { -if (dsp_id ==ACPI_CDAT_SSLBIS_ANY_PORT || - dsp_id == dport->port_id) { -cxl_access_coordinate_set (dport->coord, - sslbis->data_type, - val); - } +if (dsp_id ==ACPI_CDAT_SSLBIS_ANY_PORT || + dsp_id == dport->port_id) { +cxl_access_coordinate_set (dport->coord, + sslbis->data_type, val); +return 0 ; } }return 0 ; } -void cxl_switch_parse_cdat (struct cxl_port *port) +void cxl_switch_parse_cdat (struct cxl_dport *dport) { +struct cxl_port *port = dport->port;int rc;if (!port->cdat.table)return ; rc =cdat_table_parse (ACPI_CDAT_TYPE_SSLBIS , cdat_sslbis_handler, - port, port->cdat.table, port->cdat.length); + dport, port->cdat.table, port->cdat.length); rc =cdat_table_parse_output (rc);if (rc)dev_dbg (&port->dev,"Failed to parse SSLBIS: %d\n" , rc);
drivers/cxl/core/port.c +1 -1
@@ -1617,7 +1617,7 @@ static struct cxl_dport *cxl_port_add_dport(struct cxl_port *port,if (IS_ERR (new_dport))return new_dport; -cxl_switch_parse_cdat (port); +cxl_switch_parse_cdat (new_dport);if (ida_is_empty (&port->decoder_ida)) { rc =devm_cxl_switch_port_decoders_setup (port);
drivers/cxl/cxl.h +1 -1
@@ -895,7 +895,7 @@ static inline u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint,#endif void cxl_endpoint_parse_cdat (struct cxl_port *port); -void cxl_switch_parse_cdat (struct cxl_port *port); +void cxl_switch_parse_cdat (struct cxl_dport *dport);int cxl_endpoint_get_perf_coordinates (struct cxl_port *port,struct access_coordinate *coord);
Signature · 1 line
2.50.1
Quoted 80 lines · Dave Jiang
v9:
- Reworked the port enumeration iteration loop. (Robert)
All please re-review "cxl: Defer dport allocation for switch ports"
- Created helper functions for dealing with switch and endpoint decoder enumeration. Main goal is to reducecxl_testinterface burden. (Robert)
- Droppedcxl_testchanges for decoder functions
- Dropped thecxl_testfor region replay. It's not 100% ready and can be submitted later. (Alison)
- See specific commits for more detailed changes.v8:
- A bit of changes from Dan and Robert's comments. Main change is moving the port MMIO register probing to after the first dport shows up. This resulted with decoder allocation happens after the register probe.
- See specific commits for more detailed changes.v7:
- Remove-EEXISTto simplify error flow. (Ming)
- Set dport to NULL during declare. (Jonathan)v6:
- Return-EEXISTwhen a dport already exists. (Jonathan)
- Fix checking wrong port for NULL. (Ming)
- Checkhost_bridgeand calldevm_cxl_add_dport_by_uport()directly vsadd_port_attach_ep(). (Ming)
- Set dport to NULL during declaration. (Jonathan)v5:
- Return dport instead of errno with dport pointer as output param. (Jonathan)
- Consolidate common code incxl_test. (Jonathan)
- Renamecxl_port_get_total_dports()tocxl_port_update_total_dports(). (Jonathan)v4:
- Push dport allocation to when they are discovered. (Robert)
- Drop linux id for dport with above changes.v3:
- Main changes revolve around improving naming of hostbridge uport and dport (Gregory)
- See specific patches for detailed change logThis series attempts to delay the allocation of dports until when the endpoint device (memdev) are being probed. At this point, the CXL link is established and all the devices along the CXL link path up to the Root Port (RP) should be active.
And hopefully this help a bit with Robert's issue raised in the "Inactive downstream port handling" series [1]. Testing would be appreicated. Thank you!
[1]: https://lore.kernel.org/linux-cxl/[email protected]/
Dave Jiang (10):
cxl: Add helper to detect top of CXL device topology
cxl: Add helper to reap dport
cxl: Add a cached copy oftarget_maptocxl_decoder
cxl: Move port register setup to first dport appear cxl/test: Refactor decoder setup to reducecxl_testburden
cxl: Defer dport allocation for switch ports cxl/test: Add mock version ofdevm_cxl_add_dport_by_dev()cxl/test: Adjust the mock version of
devm_cxl_switch_port_decoders_setup()
cxl/test: Setuptarget_mapforcxl_testdecoder initialization
cxl: Change sslbis handler to only handle single dportdrivers/cxl/acpi.c | 7 +- drivers/cxl/core/cdat.c | 25 +-- drivers/cxl/core/core.h | 5 + drivers/cxl/core/hdm.c | 105 ++++++--- drivers/cxl/core/pci.c | 89 ++++++++ drivers/cxl/core/port.c | 317 ++++++++++++++++++++------- drivers/cxl/core/region.c | 4 +- drivers/cxl/cxl.h | 44 +++- drivers/cxl/cxlpci.h | 2 - drivers/cxl/port.c | 47 +--- tools/testing/cxl/Kbuild | 7 +- tools/testing/cxl/cxl_core_exports.c | 22 ++ tools/testing/cxl/exports.h | 13 ++ tools/testing/cxl/test/cxl.c | 115 ++++++++-- tools/testing/cxl/test/mock.c | 96 +++----- tools/testing/cxl/test/mock.h | 9 +- 16 files changed, 642 insertions(+), 265 deletions(-) create mode 100644 tools/testing/cxl/exports.h
Quoted 1 line
base-commit:
8f5ae30d69d7543eee0d70083daf4de8fe15d585
Applied to cxl/next with minor fixes from comments:14868cf9e4b8cefac0fb9eea4993ac3f863a8c0d