Skip to contentSkip to conversation

cxl/test: Update mock memdev array before calling platform_device_add()

PATCH Superseded Li Ming 3 participants 3 messages Mar 6 – May 20 CXL

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Li Ming

author reviewer
#0 lore raw

PATCH 1/1 cxl/test: Update mock memdev array before calling platform_device_add()

CXL test environment hits the following error sometimes.

cxl_mem mem9: endpoint7 failed probe

All mock memdevs are platform firmware devices added by cxl_test module, and cxl_test module also provides a platform device driver for them to create a memdev device to CXL subsystem. cxl_test module uses cxl_rcd/mem_single/mem arrays to store different types of mock memdevs. CXL drivers calls registered mock functions for a mock memdev by checking if a given memdev is in these arrays.

When cxl_test module adds these mock memdevs, it always calls platform_device_add() before adding them to a suitable mock memdev array. However, there is a small window where CXL drivers calls mock function for a added memdev before it added to a mock memdev array. In above case, cxl endpoint driver considers a added memdev was not a mock memdev, then calling devm_cxl_endpoint_decoders_setup() for it rather than mock_endpoint_decoders_setup().

An appropriate solution is that adding a new mock memdev to a mock memdev array before calling platform_device_add() for it. It can guarantee the new mock memdev is visible to CXL subsystem.

Fixes 3a2b97b3210b ("cxl/test: Improve init-order fidelity relative to real-world systems")
---
 tools/testing/cxl/test/cxl.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
tools/testing/cxl/test/cxl.c +6 -3
index 81e2aef3627a..2664c9de8dd6 100644
@@ -1356,12 +1356,13 @@ static int cxl_mem_init(void)
 		pdev->dev.parent = &dport->dev;
 		set_dev_node(&pdev->dev, i % 2);
 
+		cxl_mem[i] = pdev;
 		rc = platform_device_add(pdev);
 		if (rc) {
 			platform_device_put(pdev);
+			cxl_mem[i] = NULL;
 			goto err_mem;
 		}
-		cxl_mem[i] = pdev;
 	}
 
 	for (i = 0; i < ARRAY_SIZE(cxl_mem_single); i++) {
@@ -1374,12 +1375,13 @@ static int cxl_mem_init(void)
 		pdev->dev.parent = &dport->dev;
 		set_dev_node(&pdev->dev, i % 2);
 
+		cxl_mem_single[i] = pdev;
 		rc = platform_device_add(pdev);
 		if (rc) {
 			platform_device_put(pdev);
+			cxl_mem_single[i] = NULL;
 			goto err_single;
 		}
-		cxl_mem_single[i] = pdev;
 	}
 
 	for (i = 0; i < ARRAY_SIZE(cxl_rcd); i++) {
@@ -1393,12 +1395,13 @@ static int cxl_mem_init(void)
 		pdev->dev.parent = &rch->dev;
 		set_dev_node(&pdev->dev, i % 2);
 
+		cxl_rcd[i] = pdev;
 		rc = platform_device_add(pdev);
 		if (rc) {
 			platform_device_put(pdev);
+			cxl_rcd[i] = NULL;
 			goto err_rcd;
 		}
-		cxl_rcd[i] = pdev;
 	}
 
 	return 0;

---
base-commit: 19d2f0b97a131198efc2c4ca3eb7f980bba8c2b4
change-id: 20260306-update_array_before_adding_mock_memdev-89673b4d9118

Best regards,

Signature · 1 line
Li Ming <[email protected]>
maintainer Reviewed-by on v1 1/1
#1 lore raw
Quoted 19 lines · Li Ming

CXL test environment hits the following error sometimes.

cxl_mem mem9: endpoint7 failed probe

All mock memdevs are platform firmware devices added by cxl_test module, and cxl_test module also provides a platform device driver for them to create a memdev device to CXL subsystem. cxl_test module uses cxl_rcd/mem_single/mem arrays to store different types of mock memdevs. CXL drivers calls registered mock functions for a mock memdev by checking if a given memdev is in these arrays.

When cxl_test module adds these mock memdevs, it always calls platform_device_add() before adding them to a suitable mock memdev array. However, there is a small window where CXL drivers calls mock function for a added memdev before it added to a mock memdev array. In above case, cxl endpoint driver considers a added memdev was not a mock memdev, then calling devm_cxl_endpoint_decoders_setup() for it rather than mock_endpoint_decoders_setup().

Quoted 6 lines

An appropriate solution is that adding a new mock memdev to a mock memdev array before calling platform_device_add() for it. It can guarantee the new mock memdev is visible to CXL subsystem.

Fixes 3a2b97b3210b ("cxl/test: Improve init-order fidelity relative to real-world systems")

Makes sense.

Quoted 59 lines
---
 tools/testing/cxl/test/cxl.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
tools/testing/cxl/test/cxl.c +6 -4
index 81e2aef3627a..2664c9de8dd6 100644
@@ -1356,12 +1356,13 @@ static int cxl_mem_init(void)
 		pdev->dev.parent = &dport->dev;
 		set_dev_node(&pdev->dev, i % 2);
 
+		cxl_mem[i] = pdev;
 		rc = platform_device_add(pdev);
 		if (rc) {
 			platform_device_put(pdev);
+			cxl_mem[i] = NULL;
 			goto err_mem;
 		}
-		cxl_mem[i] = pdev;
 	}
 
 	for (i = 0; i < ARRAY_SIZE(cxl_mem_single); i++) {
@@ -1374,12 +1375,13 @@ static int cxl_mem_init(void)
 		pdev->dev.parent = &dport->dev;
 		set_dev_node(&pdev->dev, i % 2);
 
+		cxl_mem_single[i] = pdev;
 		rc = platform_device_add(pdev);
 		if (rc) {
 			platform_device_put(pdev);
+			cxl_mem_single[i] = NULL;
 			goto err_single;
 		}
-		cxl_mem_single[i] = pdev;
 	}
 
 	for (i = 0; i < ARRAY_SIZE(cxl_rcd); i++) {
@@ -1393,12 +1395,13 @@ static int cxl_mem_init(void)
 		pdev->dev.parent = &rch->dev;
 		set_dev_node(&pdev->dev, i % 2);
 
+		cxl_rcd[i] = pdev;
 		rc = platform_device_add(pdev);
 		if (rc) {
 			platform_device_put(pdev);
+			cxl_rcd[i] = NULL;
 			goto err_rcd;
 		}
-		cxl_rcd[i] = pdev;
 	}
 
 	return 0;
 
---

base-commit: 19d2f0b97a131198efc2c4ca3eb7f980bba8c2b4
change-id: 20260306-update_array_before_adding_mock_memdev-89673b4d9118

Best regards,

maintainer Reviewed-by Tested-by on v1 1/1
#2 lore raw
In reply to Li Ming
Quoted 19 lines · Li Ming

CXL test environment hits the following error sometimes.

cxl_mem mem9: endpoint7 failed probe

All mock memdevs are platform firmware devices added by cxl_test module, and cxl_test module also provides a platform device driver for them to create a memdev device to CXL subsystem. cxl_test module uses cxl_rcd/mem_single/mem arrays to store different types of mock memdevs. CXL drivers calls registered mock functions for a mock memdev by checking if a given memdev is in these arrays.

When cxl_test module adds these mock memdevs, it always calls platform_device_add() before adding them to a suitable mock memdev array. However, there is a small window where CXL drivers calls mock function for a added memdev before it added to a mock memdev array. In above case, cxl endpoint driver considers a added memdev was not a mock memdev, then calling devm_cxl_endpoint_decoders_setup() for it rather than mock_endpoint_decoders_setup().

Quoted 3 lines

An appropriate solution is that adding a new mock memdev to a mock memdev array before calling platform_device_add() for it. It can guarantee the new mock memdev is visible to CXL subsystem.

Hi Ming,

Thanks, I was able to reproduce this as well while testing a new configuration with 12 memdevs, and your fix resolved it, so...

However, since this is a low frequency nuisance now, and given that it now touches all three mock memdev arrays, and I am heading towards adding a fourth (which is how I hit this), it would be nice to tidy this code up before fixing. Refactor the repeated platform_device_alloc()/platform_device_add() pattern into a helper. That would let the ordering rule live in one place rather than 3 or 4.

-- Alison

Quoted 65 lines
Fixes 3a2b97b3210b ("cxl/test: Improve init-order fidelity relative to real-world systems")
---
 tools/testing/cxl/test/cxl.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
tools/testing/cxl/test/cxl.c +6 -4
index 81e2aef3627a..2664c9de8dd6 100644
@@ -1356,12 +1356,13 @@ static int cxl_mem_init(void)
 		pdev->dev.parent = &dport->dev;
 		set_dev_node(&pdev->dev, i % 2);
 
+		cxl_mem[i] = pdev;
 		rc = platform_device_add(pdev);
 		if (rc) {
 			platform_device_put(pdev);
+			cxl_mem[i] = NULL;
 			goto err_mem;
 		}
-		cxl_mem[i] = pdev;
 	}
 
 	for (i = 0; i < ARRAY_SIZE(cxl_mem_single); i++) {
@@ -1374,12 +1375,13 @@ static int cxl_mem_init(void)
 		pdev->dev.parent = &dport->dev;
 		set_dev_node(&pdev->dev, i % 2);
 
+		cxl_mem_single[i] = pdev;
 		rc = platform_device_add(pdev);
 		if (rc) {
 			platform_device_put(pdev);
+			cxl_mem_single[i] = NULL;
 			goto err_single;
 		}
-		cxl_mem_single[i] = pdev;
 	}
 
 	for (i = 0; i < ARRAY_SIZE(cxl_rcd); i++) {
@@ -1393,12 +1395,13 @@ static int cxl_mem_init(void)
 		pdev->dev.parent = &rch->dev;
 		set_dev_node(&pdev->dev, i % 2);
 
+		cxl_rcd[i] = pdev;
 		rc = platform_device_add(pdev);
 		if (rc) {
 			platform_device_put(pdev);
+			cxl_rcd[i] = NULL;
 			goto err_rcd;
 		}
-		cxl_rcd[i] = pdev;
 	}
 
 	return 0;
 
---

base-commit: 19d2f0b97a131198efc2c4ca3eb7f980bba8c2b4
change-id: 20260306-update_array_before_adding_mock_memdev-89673b4d9118

Best regards,

Signature · 1 line
Li Ming <[email protected]>