cxl/test: Update mock memdev array before calling platform_device_add()
Li Ming
reviewerPATCH 1/1 cxl/test: Update mock memdev array before calling platform_device_add()
CXL test environment hits the following error sometimes.
cxl_mem mem9: endpoint7 failed probe
All mock memdevs are platform firmware devices added by cxl_test module, and cxl_test module also provides a platform device driver for them to create a memdev device to CXL subsystem. cxl_test module uses cxl_rcd/mem_single/mem arrays to store different types of mock memdevs. CXL drivers calls registered mock functions for a mock memdev by checking if a given memdev is in these arrays.
When cxl_test module adds these mock memdevs, it always calls platform_device_add() before adding them to a suitable mock memdev array. However, there is a small window where CXL drivers calls mock function for a added memdev before it added to a mock memdev array. In above case, cxl endpoint driver considers a added memdev was not a mock memdev, then calling devm_cxl_endpoint_decoders_setup() for it rather than mock_endpoint_decoders_setup().
An appropriate solution is that adding a new mock memdev to a mock memdev array before calling platform_device_add() for it. It can guarantee the new mock memdev is visible to CXL subsystem.
--- tools/testing/cxl/test/cxl.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
tools/testing/cxl/test/cxl.c +6 -3
@@ -1356,12 +1356,13 @@ static int cxl_mem_init(void) pdev->dev.parent = &dport->dev;set_dev_node (&pdev->dev, i %2 ); + cxl_mem[i] = pdev; rc =platform_device_add (pdev);if (rc) {platform_device_put (pdev); + cxl_mem[i] =NULL ; goto err_mem; } - cxl_mem[i] = pdev; }for (i =0 ; i <ARRAY_SIZE (cxl_mem_single); i++) {
@@ -1374,12 +1375,13 @@ static int cxl_mem_init(void) pdev->dev.parent = &dport->dev;set_dev_node (&pdev->dev, i %2 ); + cxl_mem_single[i] = pdev; rc =platform_device_add (pdev);if (rc) {platform_device_put (pdev); + cxl_mem_single[i] =NULL ; goto err_single; } - cxl_mem_single[i] = pdev; }for (i =0 ; i <ARRAY_SIZE (cxl_rcd); i++) {
@@ -1393,12 +1395,13 @@ static int cxl_mem_init(void) pdev->dev.parent = &rch->dev;set_dev_node (&pdev->dev, i %2 ); + cxl_rcd[i] = pdev; rc =platform_device_add (pdev);if (rc) {platform_device_put (pdev); + cxl_rcd[i] =NULL ; goto err_rcd; } - cxl_rcd[i] = pdev; }return 0 ;
---
base-commit: 19d2f0b97a131198efc2c4ca3eb7f980bba8c2b4
change-id: 20260306-update_array_before_adding_mock_memdev-89673b4d9118
Best regards,
Signature · 1 line
Li Ming <[email protected]>
Quoted 19 lines · Li Ming
CXL test environment hits the following error sometimes.
cxl_memmem9: endpoint7 failed probeAll mock memdevs are platform firmware devices added by
cxl_testmodule, andcxl_testmodule also provides a platform device driver for them to create a memdev device to CXL subsystem.cxl_testmodule uses cxl_rcd/mem_single/mem arrays to store different types of mock memdevs. CXL drivers calls registered mock functions for a mock memdev by checking if a given memdev is in these arrays.When
cxl_testmodule adds these mock memdevs, it always callsplatform_device_add()before adding them to a suitable mock memdev array. However, there is a small window where CXL drivers calls mock function for a added memdev before it added to a mock memdev array. In above case, cxl endpoint driver considers a added memdev was not a mock memdev, then callingdevm_cxl_endpoint_decoders_setup()for it rather thanmock_endpoint_decoders_setup().
Quoted 6 lines
An appropriate solution is that adding a new mock memdev to a mock memdev array before calling
platform_device_add()for it. It can guarantee the new mock memdev is visible to CXL subsystem.Fixes 3a2b97b3210b ("cxl/test: Improve init-order fidelity relative to real-world systems")Signed-off-by Li Ming <[email protected]>
Makes sense.
Quoted 59 lines
--- tools/testing/cxl/test/cxl.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)tools/testing/cxl/test/cxl.c +6
-4@@ -1356,12 +1356,13 @@ static int cxl_mem_init(void) pdev->dev.parent = &dport->dev;set_dev_node (&pdev->dev, i %2 ); + cxl_mem[i] = pdev; rc =platform_device_add (pdev);if (rc) {platform_device_put (pdev); + cxl_mem[i] =NULL ; goto err_mem; } - cxl_mem[i] = pdev; }for (i =0 ; i <ARRAY_SIZE (cxl_mem_single); i++) {@@ -1374,12 +1375,13 @@ static int cxl_mem_init(void) pdev->dev.parent = &dport->dev;set_dev_node (&pdev->dev, i %2 ); + cxl_mem_single[i] = pdev; rc =platform_device_add (pdev);if (rc) {platform_device_put (pdev); + cxl_mem_single[i] =NULL ; goto err_single; } - cxl_mem_single[i] = pdev; }for (i =0 ; i <ARRAY_SIZE (cxl_rcd); i++) {@@ -1393,12 +1395,13 @@ static int cxl_mem_init(void) pdev->dev.parent = &rch->dev;set_dev_node (&pdev->dev, i %2 ); + cxl_rcd[i] = pdev; rc =platform_device_add (pdev);if (rc) {platform_device_put (pdev); + cxl_rcd[i] =NULL ; goto err_rcd; } - cxl_rcd[i] = pdev; }return 0 ; ---base-commit:
19d2f0b97a131198efc2c4ca3eb7f980bba8c2b4
change-id: 20260306-update_array_before_adding_mock_memdev-89673b4d9118Best regards,
Quoted 19 lines · Li Ming
CXL test environment hits the following error sometimes.
cxl_memmem9: endpoint7 failed probeAll mock memdevs are platform firmware devices added by
cxl_testmodule, andcxl_testmodule also provides a platform device driver for them to create a memdev device to CXL subsystem.cxl_testmodule uses cxl_rcd/mem_single/mem arrays to store different types of mock memdevs. CXL drivers calls registered mock functions for a mock memdev by checking if a given memdev is in these arrays.When
cxl_testmodule adds these mock memdevs, it always callsplatform_device_add()before adding them to a suitable mock memdev array. However, there is a small window where CXL drivers calls mock function for a added memdev before it added to a mock memdev array. In above case, cxl endpoint driver considers a added memdev was not a mock memdev, then callingdevm_cxl_endpoint_decoders_setup()for it rather thanmock_endpoint_decoders_setup().
Quoted 3 lines
An appropriate solution is that adding a new mock memdev to a mock memdev array before calling
platform_device_add()for it. It can guarantee the new mock memdev is visible to CXL subsystem.
Hi Ming,
Thanks, I was able to reproduce this as well while testing a new configuration with 12 memdevs, and your fix resolved it, so...
However, since this is a low frequency nuisance now, and given that it now touches all three mock memdev arrays, and I am heading towards adding a fourth (which is how I hit this), it would be nice to tidy this code up before fixing. Refactor the repeated platform_device_alloc()/platform_device_add() pattern into a helper. That would let the ordering rule live in one place rather than 3 or 4.
-- Alison
Quoted 65 lines
Fixes 3a2b97b3210b ("cxl/test: Improve init-order fidelity relative to real-world systems")Signed-off-by Li Ming <[email protected]>--- tools/testing/cxl/test/cxl.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)tools/testing/cxl/test/cxl.c +6
-4@@ -1356,12 +1356,13 @@ static int cxl_mem_init(void) pdev->dev.parent = &dport->dev;set_dev_node (&pdev->dev, i %2 ); + cxl_mem[i] = pdev; rc =platform_device_add (pdev);if (rc) {platform_device_put (pdev); + cxl_mem[i] =NULL ; goto err_mem; } - cxl_mem[i] = pdev; }for (i =0 ; i <ARRAY_SIZE (cxl_mem_single); i++) {@@ -1374,12 +1375,13 @@ static int cxl_mem_init(void) pdev->dev.parent = &dport->dev;set_dev_node (&pdev->dev, i %2 ); + cxl_mem_single[i] = pdev; rc =platform_device_add (pdev);if (rc) {platform_device_put (pdev); + cxl_mem_single[i] =NULL ; goto err_single; } - cxl_mem_single[i] = pdev; }for (i =0 ; i <ARRAY_SIZE (cxl_rcd); i++) {@@ -1393,12 +1395,13 @@ static int cxl_mem_init(void) pdev->dev.parent = &rch->dev;set_dev_node (&pdev->dev, i %2 ); + cxl_rcd[i] = pdev; rc =platform_device_add (pdev);if (rc) {platform_device_put (pdev); + cxl_rcd[i] =NULL ; goto err_rcd; } - cxl_rcd[i] = pdev; }return 0 ; ---base-commit:
19d2f0b97a131198efc2c4ca3eb7f980bba8c2b4
change-id: 20260306-update_array_before_adding_mock_memdev-89673b4d9118Best regards,
Signature · 1 line
Li Ming <[email protected]>