Skip to contentSkip to conversation

CXL/region : commit reset of out of order region appears to succeed.

DISCUSSION Jonathan Cameron 2 participants 2 messages Mar 16, 2023 – Jun 17, 2023

lore
Share & tools
lore
#0 lore raw

Ran into this whilst testing fix for QEMU uncommit handling.

To replicate.
1) Setup two regions on a direct connected Type 3 and commit them both.
2) Uncommit the first region once. (it fails with an out of order message) Note that from here on the sysfs commit attribute reads as 0.
3) Uncommit that first region again. It appears to succeed.

Reason is easy to track down:
https://elixir.bootlin.com/linux/v6.3-rc2/source/drivers/cxl/core/region.c#L257

commit_store() of 0 unconditionally sets the state to CXL_CONFIG_RESET_PENDING

When the decoder reset fails, that is left set.
Hence next call drops straight through.

Whilst it's easy to 'fix' the superficial issue by reseting the state to the previous value on error, I'm not sure that's sufficient or race free.

Hence report rather than a patch. I can look into this in more depth, but a few other things come before it in my list.

Thanks,

Jonathan

p.s. I hope to send the qemu fix for uncommit fairly soon.

#1 lore raw
Quoted 11 lines · Jonathan Cameron

Ran into this whilst testing fix for QEMU uncommit handling.

To replicate.
1) Setup two regions on a direct connected Type 3 and commit them both.
2) Uncommit the first region once. (it fails with an out of order message) Note that from here on the sysfs commit attribute reads as 0.
3) Uncommit that first region again. It appears to succeed.

Reason is easy to track down:
https://elixir.bootlin.com/linux/v6.3-rc2/source/drivers/cxl/core/region.c#L257

Quoted 7 lines

commit_store() of 0 unconditionally sets the state to CXL_CONFIG_RESET_PENDING

When the decoder reset fails, that is left set.
Hence next call drops straight through.

Whilst it's easy to 'fix' the superficial issue by reseting the state to the previous value on error, I'm not sure that's sufficient or race free.

I think it is sufficient because the state transition is happening under the lock and RESET_PENDING > ACTIVE. So any paths that depend on the region not being active will be protected.

On the other side, if someone races to commit the region via another thread while the lock is dropped they will either successfully transition the region back to the COMMIT state, or will re-attempt the reset. When both those threads re-acquire the lock one of them will see that the reset state can advance back to ACTIVE, or will see that someone snuck in and committed the region again while the lock was dropped.