Skip to contentSkip to conversation

nvme: ensure reset state check ordering

PATCH In review Keith Busch 3 participants 5 messages Oct 27, 2023 – Nov 20, 2023 NVMe

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 nvme: ensure reset state check ordering

From: Keith Busch <[email protected]>

A different CPU may be setting the ctrl->state value while flushing the reset work, so ensure proper barriers to prevent reading a stale state. Reading the wrong state can report unexpected ENETRESET errors when everything was successful.

Fixes 8000d1fdb07e36 ("nvme-rdma: fix sysfs invoked reset_ctrl error flow")
---
 drivers/nvme/host/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/nvme/host/core.c +1 -1
index 62612f87aafa2..9ec7ce8cc802e 100644
@@ -201,7 +201,7 @@ int nvme_reset_ctrl_sync(struct nvme_ctrl *ctrl)
 	ret = nvme_reset_ctrl(ctrl);
 	if (!ret) {
 		flush_work(&ctrl->reset_work);
-		if (ctrl->state != NVME_CTRL_LIVE)
+		if (smp_load_acquire(&ctrl->state) != NVME_CTRL_LIVE)
 			ret = -ENETRESET;
 	}
 
Signature · 1 line
2.34.1
#1 lore raw
Quoted 6 lines · Keith Busch

From: Keith Busch <[email protected]>

A different CPU may be setting the ctrl->state value while flushing the reset work, so ensure proper barriers to prevent reading a stale state. Reading the wrong state can report unexpected ENETRESET errors when everything was successful.

smp_load_acquire needs to be paired with a smp_store_release.

We should either always hold the lock or use WRITE_ONCE/READ_ONCE

maintainer
#2 lore raw
Quoted 11 lines
Quoted 7 lines

From: Keith Busch <[email protected]>

A different CPU may be setting the ctrl->state value while flushing the reset work, so ensure proper barriers to prevent reading a stale state. Reading the wrong state can report unexpected ENETRESET errors when everything was successful.

smp_load_acquire needs to be paired with a smp_store_release.

We should either always hold the lock or use WRITE_ONCE/READ_ONCE

which lock?

#3 lore raw
Quoted 8 lines · Sagi Grimberg
Quoted 8 lines
Quoted 7 lines

From: Keith Busch <[email protected]>

A different CPU may be setting the ctrl->state value while flushing the reset work, so ensure proper barriers to prevent reading a stale state. Reading the wrong state can report unexpected ENETRESET errors when everything was successful.

Quoted 5 lines
Quoted 3 lines

smp_load_acquire needs to be paired with a smp_store_release.

We should either always hold the lock or use WRITE_ONCE/READ_ONCE

which lock?

in this case: ctrl->lock.

maintainer
#4 lore raw
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

From: Keith Busch <[email protected]>

A different CPU may be setting the ctrl->state value while flushing the reset work, so ensure proper barriers to prevent reading a stale state. Reading the wrong state can report unexpected ENETRESET errors when everything was successful.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

smp_load_acquire needs to be paired with a smp_store_release.

We should either always hold the lock or use WRITE_ONCE/READ_ONCE

which lock?

in this case: ctrl->lock.

Yes, we can do either. READ/WRITE_ONCE seems more appropriate. Worth a comment because we don't use it in other places where it is advisory.