cxl: Fixes for hdm decoder initialization from DVSEC ranges
v4 cover cxl: Fixes for hdm decoder initialization from DVSEC ranges
The first and third patch are intended to fix potential issues regarding to retrieve and record DVSEC ranges. The second and fourth are cleanup.
v3->v4:
- improved the commit messages for patch2 and patch3 based on Jonathan's comments.
- collected "Reviewed-by" from Jonathan v3:https://lore.kernel.org/all/[email protected]/T/
v2->v3:
- improved the commit message of patch1 to indicate potential impact of the change. (Dan)
- Dropped un-appropriate "Fixes" tag. (Dan)
- Dropped the patch2 which is a code movement in original patchset.
- Separated the original patch3 into cleanup one and logic change one which are corresponding to patch2 and patch3 in current patchset. (Dan)
v2:https://lore.kernel.org/linux-cxl/[email protected]/T/#t
Yanfei Xu (4):
cxl/pci: Fix to record only non-zero ranges
cxl/pci: Remove duplicated implementation of waiting for
memory_info_valid
cxl/pci: Check Mem_info_valid bit for each applicable DVSEC cxl/pci: simplify the check of mem_enabled in cxl_hdm_decode_init()
drivers/cxl/core/pci.c | 74 +++++++++-------------------------- drivers/cxl/cxl.h | 2 +- drivers/cxl/port.c | 2 +- tools/testing/cxl/test/mock.c | 4 +- 4 files changed, 23 insertions(+), 59 deletions(-)
Signature · 1 line
2.39.2
PATCH v4 1/4 cxl/pci: Fix to record only non-zero ranges
The function cxl_dvsec_rr_decode() retrieves and records DVSEC ranges into info->dvsec_range[], regardless of whether it is non-zero range, and the variable info->ranges indicates the number of non-zero ranges. However, in cxl_hdm_decode_init(), the validation for info->dvsec_range[] occurs in a for loop that iterates based on info->ranges. It may result in zero range to be validated but non-zero range not be validated, in turn, the number of allowed ranges is to be
0. Address it by only record non-zero ranges.
This fix is not urgent as it requires a configuration that zeroes out the first dvsec range while populating the second. This has not been observed, but it is theoretically possible. If this gets picked up for -stable, no harm done, but there is no urgency to backport.
--- drivers/cxl/core/pci.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-)
drivers/cxl/core/pci.c +1 -7
@@ -390,10 +390,6 @@ int cxl_dvsec_rr_decode(struct device *dev, int d, size |= temp &CXL_DVSEC_MEM_SIZE_LOW_MASK ;if (!size) { - info->dvsec_range[i] = (struct range ) { - .start =0 , - .end =CXL_RESOURCE_NONE , - };continue ; }
@@ -411,12 +407,10 @@ int cxl_dvsec_rr_decode(struct device *dev, int d, base |= temp &CXL_DVSEC_MEM_BASE_LOW_MASK ; - info->dvsec_range[i] = (struct range ) { + info->dvsec_range[ranges++] = (struct range ) { .start = base, .end = base + size -1 }; - - ranges++; } info->ranges = ranges;
Signature · 1 line
2.39.2
Re: [v4 1/4] cxl/pci: Fix to record only non-zero ranges
Quoted 9 lines · Yanfei Xu
The function
cxl_dvsec_rr_decode()retrieves and records DVSEC ranges into info->dvsec_range[], regardless of whether it is non-zero range, and the variableinfo->rangesindicates the number of non-zero ranges. However, incxl_hdm_decode_init(), the validation for info->dvsec_range[] occurs in a for loop that iterates based oninfo->ranges. It may result in zero range to be validated but non-zero range not be validated, in turn, the number of allowed ranges is to be
0. Address it by only record non-zero ranges.
Quoted 5 lines
This fix is not urgent as it requires a configuration that zeroes out the first dvsec range while populating the second. This has not been observed, but it is theoretically possible. If this gets picked up for -stable, no harm done, but there is no urgency to backport.
I'd prefer the commit msg be:
cxl/pci: Record only non-zero DVSEC ranges
but either way -
Quoted 6 lines
Fixes 560f78559006 ("cxl/pci: Retrieve CXL DVSEC memory info")Reviewed-by Jonathan Cameron <[email protected]>Signed-off-by Yanfei Xu <[email protected]>--- drivers/cxl/core/pci.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-)
snip
PATCH v4 2/4 cxl/pci: Remove duplicated implementation of waiting for memory_info_valid
In reply to Yanfei Xucommit ce17ad0d5498 ("cxl: Wait Memory_Info_Valid before access memory related info") added another implementation, which is cxl_dvsec_mem_range_valid(), of waiting for memory_info_valid without realizing it duplicated wait_for_valid(). Remove wait_for_valid() and retain cxl_dvsec_mem_range_valid() as the former is hardcoded to check only the Memory_Info_Valid bit of DVSEC range 1, while the latter allows for selection between DVSEC range 1 or 2 via parameter.
--- drivers/cxl/core/pci.c | 41 +++++------------------------------ drivers/cxl/cxl.h | 2 +- drivers/cxl/port.c | 2 +- tools/testing/cxl/test/mock.c | 4 ++-- 4 files changed, 9 insertions(+), 40 deletions(-)
drivers/cxl/core/pci.c +5 -36
@@ -211,37 +211,6 @@ int cxl_await_media_ready(struct cxl_dev_state *cxlds) }EXPORT_SYMBOL_NS_GPL (cxl_await_media_ready,CXL ); -static int wait_for_valid (struct pci_dev *pdev,int d) -{ -u32 val; -int rc; - -/* -* Memory_Info_Valid: When set, indicates that the CXL Range 1 Size high -* and Size Low registers are valid. Must be set within 1 second of -* deassertion of reset to CXL device. Likely it is already set by the -* time this runs, but otherwise give a 1.5 second timeout in case of -* clock skew. -*/ - rc =pci_read_config_dword (pdev, d +CXL_DVSEC_RANGE_SIZE_LOW (0 ), &val); -if (rc) -return rc; - -if (val &CXL_DVSEC_MEM_INFO_VALID ) -return 0 ; - -msleep (1500 ); - - rc =pci_read_config_dword (pdev, d +CXL_DVSEC_RANGE_SIZE_LOW (0 ), &val); -if (rc) -return rc; - -if (val &CXL_DVSEC_MEM_INFO_VALID ) -return 0 ; - -return -ETIMEDOUT ; -} -static int cxl_set_mem_enable (struct cxl_dev_state *cxlds,u16 val) {struct pci_dev *pdev =to_pci_dev (cxlds->dev);
@@ -322,11 +291,13 @@ static int devm_cxl_enable_hdm(struct device *host, struct cxl_hdm *cxlhdm)return devm_add_action_or_reset (host, disable_hdm, cxlhdm); } -int cxl_dvsec_rr_decode (struct device *dev,int d, +int cxl_dvsec_rr_decode (struct device *dev,struct cxl_port *port,struct cxl_endpoint_dvsec_info *info) {struct pci_dev *pdev =to_pci_dev (dev); +struct cxl_dev_state *cxlds =pci_get_drvdata (pdev);int hdm_count, rc, i, ranges =0 ; +int d = cxlds->cxl_dvsec;u16 cap, ctrl;if (!d) {
@@ -353,11 +324,9 @@ int cxl_dvsec_rr_decode(struct device *dev, int d,if (!hdm_count || hdm_count >2 )return -EINVAL ; - rc =wait_for_valid (pdev, d); -if (rc) { -dev_dbg (dev,"Failure awaiting MEM_INFO_VALID (%d)\n" , rc); + rc =cxl_dvsec_mem_range_valid (cxlds,0 ); +if (rc)return rc; - }/* * The current DVSEC values are moot if the memory capability is
drivers/cxl/cxl.h +1 -1
@@ -809,7 +809,7 @@ struct cxl_hdm *devm_cxl_setup_hdm(struct cxl_port *port,int devm_cxl_enumerate_decoders (struct cxl_hdm *cxlhdm,struct cxl_endpoint_dvsec_info *info);int devm_cxl_add_passthrough_decoder (struct cxl_port *port); -int cxl_dvsec_rr_decode (struct device *dev,int dvsec, +int cxl_dvsec_rr_decode (struct device *dev,struct cxl_port *port,struct cxl_endpoint_dvsec_info *info);bool is_cxl_region (struct device *dev);
drivers/cxl/port.c +1 -1
@@ -98,7 +98,7 @@ static int cxl_endpoint_port_probe(struct cxl_port *port)struct cxl_port *root;int rc; - rc =cxl_dvsec_rr_decode (cxlds->dev, cxlds->cxl_dvsec, &info); + rc =cxl_dvsec_rr_decode (cxlds->dev, port, &info);if (rc <0 )return rc;
tools/testing/cxl/test/mock.c +2 -2
@@ -228,7 +228,7 @@ int __wrap_cxl_hdm_decode_init(struct cxl_dev_state *cxlds, }EXPORT_SYMBOL_NS_GPL (__wrap_cxl_hdm_decode_init,CXL ); -int __wrap_cxl_dvsec_rr_decode (struct device *dev,int dvsec, +int __wrap_cxl_dvsec_rr_decode (struct device *dev,struct cxl_port *port,struct cxl_endpoint_dvsec_info *info) {int rc =0 , index;
@@ -237,7 +237,7 @@ int __wrap_cxl_dvsec_rr_decode(struct device *dev, int dvsec,if (ops && ops->is_mock_dev (dev)) rc =0 ;else - rc =cxl_dvsec_rr_decode (dev, dvsec, info); + rc =cxl_dvsec_rr_decode (dev, port, info);put_cxl_mock_ops (index);return rc;
Signature · 1 line
2.39.2
Re: [v4 2/4] cxl/pci: Remove duplicated implementation of waiting for memory_info_valid
Quoted 8 lines · Yanfei Xu
commit
ce17ad0d5498("cxl: WaitMemory_Info_Validbefore access memory related info") added another implementation, which iscxl_dvsec_mem_range_valid(), of waiting formemory_info_validwithout realizing it duplicatedwait_for_valid(). Removewait_for_valid()and retaincxl_dvsec_mem_range_valid()as the former is hardcoded to check only theMemory_Info_Validbit of DVSEC range 1, while the latter allows for selection between DVSEC range 1 or 2 via parameter.
Quoted 4 lines
Suggested-by Dan Williams <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Signed-off-by Yanfei Xu <[email protected]>---
snip
PATCH v4 3/4 cxl/pci: Check Mem_info_valid bit for each applicable DVSEC
In reply to Yanfei XuIn theory a device might set the mem_info_valid bit for a first range after it is ready but before as second range has reached that state. Therefore, the correct approach is to check the Mem_info_valid bit for each applicable DVSEC range against HDM_COUNT, rather than only for the DVSEC range 1. Consequently, let's move the check into the "for loop" that handles each DVSEC range.
--- drivers/cxl/core/pci.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/cxl/core/pci.c +4 -4
@@ -324,10 +324,6 @@ int cxl_dvsec_rr_decode(struct device *dev, struct cxl_port *port,if (!hdm_count || hdm_count >2 )return -EINVAL ; - rc =cxl_dvsec_mem_range_valid (cxlds,0 ); -if (rc) -return rc; -/* * The current DVSEC values are moot if the memory capability is * disabled, and they will remain moot after the HDM Decoder
@@ -345,6 +341,10 @@ int cxl_dvsec_rr_decode(struct device *dev, struct cxl_port *port,u64 base, size;u32 temp; + rc =cxl_dvsec_mem_range_valid (cxlds, i); +if (rc) +return rc; + rc =pci_read_config_dword ( pdev, d +CXL_DVSEC_RANGE_SIZE_HIGH (i), &temp);if (rc)
Signature · 1 line
2.39.2
Re: [v4 3/4] cxl/pci: Check Mem_info_valid bit for each applicable DVSEC
Quoted 6 lines · Yanfei Xu
In theory a device might set the
mem_info_validbit for a first range after it is ready but before as second range has reached that state. Therefore, the correct approach is to check theMem_info_validbit for each applicable DVSEC range againstHDM_COUNT, rather than only for the DVSEC range 1. Consequently, let's move the check into the "for loop" that handles each DVSEC range.
Quoted 36 lines
Reviewed-by Jonathan Cameron <[email protected]>Signed-off-by Yanfei Xu <[email protected]>--- drivers/cxl/core/pci.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)drivers/cxl/core/pci.c +4
-4@@ -324,10 +324,6 @@ int cxl_dvsec_rr_decode(struct device *dev, struct cxl_port *port,if (!hdm_count || hdm_count >2 )return -EINVAL ; - rc =cxl_dvsec_mem_range_valid (cxlds,0 ); -if (rc) -return rc; -/* * The current DVSEC values are moot if the memory capability is * disabled, and they will remain moot after the HDM Decoder @@ -345,6 +341,10 @@ int cxl_dvsec_rr_decode(struct device *dev, struct cxl_port *port,u64 base, size;u32 temp; + rc =cxl_dvsec_mem_range_valid (cxlds, i); +if (rc) +return rc; + rc =pci_read_config_dword ( pdev, d +CXL_DVSEC_RANGE_SIZE_HIGH (i), &temp);if (rc)Signature · 1 line
2.39.2
PATCH v4 4/4 cxl/pci: simplify the check of mem_enabled in cxl_hdm_decode_init()
In reply to Yanfei XuCases can be divided into two categories which are DVSEC range enabled and not enabled when HDM decoders exist but is not enabled. To avoid checking info->mem_enabled, which indicates the enablement of DVSEC range, every time, we can check !info->mem_enabled once in advance. This simplification can make the code clearer.
No functional change intended.
--- drivers/cxl/core/pci.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-)
drivers/cxl/core/pci.c +11 -10
@@ -426,7 +426,15 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm,return -ENODEV ; } -for (i =0 , allowed =0 ; info->mem_enabled && i < info->ranges; i++) { +if (!info->mem_enabled) { + rc =devm_cxl_enable_hdm (&port->dev, cxlhdm); +if (rc) +return rc; + +return devm_cxl_enable_mem (&port->dev, cxlds); + } + +for (i =0 , allowed =0 ; i < info->ranges; i++) {struct device *cxld_dev; cxld_dev = device_find_child(&root->dev, &info->dvsec_range[i],
@@ -440,7 +448,7 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, allowed++; } -if (!allowed && info->mem_enabled) { +if (!allowed) {dev_err (dev,"Range register decodes outside platform defined CXL ranges.\n" );return -ENXIO ; }
@@ -454,14 +462,7 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm,* match. If at least one DVSEC range is enabled and allowed, skip HDM * Decoder Capability Enable. */ -if (info->mem_enabled) -return 0 ; - - rc =devm_cxl_enable_hdm (&port->dev, cxlhdm); -if (rc) -return rc; - -return devm_cxl_enable_mem (&port->dev, cxlds); +return 0 ; }EXPORT_SYMBOL_NS_GPL (cxl_hdm_decode_init,CXL );
Signature · 1 line
2.39.2
Re: [v4 4/4] cxl/pci: simplify the check of mem_enabled in cxl_hdm_decode_init()
Quoted 7 lines · Yanfei Xu
Cases can be divided into two categories which are DVSEC range enabled and not enabled when HDM decoders exist but is not enabled. To avoid checking
info->mem_enabled, which indicates the enablement of DVSEC range, every time, we can check !info->mem_enabled once in advance. This simplification can make the code clearer.No functional change intended.
Maybe s/simplify/Simplify in commit message upon applying.
Quoted 4 lines
Reviewed-by Jonathan Cameron <[email protected]>Signed-off-by Yanfei Xu <[email protected]>---
snip