Skip to contentSkip to conversation

nvme: generate uevent once a multipath namespace is operational again

PATCH Superseded Hannes Reinecke 2 participants 3 messages May 5, 2021 – May 6, 2021 NVMe

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 nvme: generate uevent once a multipath namespace is operational again

In an all paths down scenario I/O will be requeued or aborted, so no further I/O will be ongoing on this namespace.
This leaves upper layers like MD unable to determine if the namespace becomes operational again after a successful controller reset. This patch will send an uevent per multipathed namespace once the underlying controller is LIVE, allowing MD to start resync.

---
 drivers/nvme/host/multipath.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/nvme/host/multipath.c +6 -2
index 0551796517e6..f099897aea59 100644
@@ -100,8 +100,12 @@ void nvme_kick_requeue_lists(struct nvme_ctrl *ctrl)
 
 	down_read(&ctrl->namespaces_rwsem);
 	list_for_each_entry(ns, &ctrl->namespaces, list) {
-		if (ns->head->disk)
-			kblockd_schedule_work(&ns->head->requeue_work);
+		if (!ns->head->disk)
+			continue;
+		kblockd_schedule_work(&ns->head->requeue_work);
+		if (ctrl->state == NVME_CTRL_LIVE)
+			kobject_uevent(&disk_to_dev(ns->head->disk)->kobj,
+				       KOBJ_CHANGE);
 	}
 	up_read(&ctrl->namespaces_rwsem);
 }
Signature · 7 lines
2.29.2


_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
#1 lore raw
Quoted 6 lines · Hannes Reinecke

In an all paths down scenario I/O will be requeued or aborted, so no further I/O will be ongoing on this namespace.
This leaves upper layers like MD unable to determine if the namespace becomes operational again after a successful controller reset. This patch will send an uevent per multipathed namespace once the underlying controller is LIVE, allowing MD to start resync.

Do we have any documentation or other exampes for this KOBJ_CHANGED magic? I've seen it in a few places, but it always seemed rather cargo cult to me. If you have a more insights any chance you could document it?

drivers/nvme/host/multipath.c void nvme_kick_requeue_lists(struct nvme_ctrl *ctrl) · line 108

Quoted 3 lines
drivers/nvme/host/multipath.c +3 -0
+		if (ctrl->state == NVME_CTRL_LIVE)
+			kobject_uevent(&disk_to_dev(ns->head->disk)->kobj,
+				       KOBJ_CHANGE);

Also this should probably use disk_uevent to also notify partitions. Maybe also for other existing callers.

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#2 lore raw
Quoted 8 lines · Christoph Hellwig
Quoted 6 lines · Hannes Reinecke

In an all paths down scenario I/O will be requeued or aborted, so no further I/O will be ongoing on this namespace.
This leaves upper layers like MD unable to determine if the namespace becomes operational again after a successful controller reset. This patch will send an uevent per multipathed namespace once the underlying controller is LIVE, allowing MD to start resync.

Quoted 5 lines

Do we have any documentation or other exampes for this KOBJ_CHANGED magic? I've seen it in a few places, but it always seemed rather cargo cult to me. If you have a more insights any chance you could document it?

It's precisely cargo cult, but definitely not well documented.

Currently there is an ambiguity between 'KOBJ_ADD' and 'KOBJ_CHANGED'; some devices will only send KOBJ_ADD, other (most notably S/390 DASDs and device-mapper devices) will send KOBJ_CHANGED to indicate that this device is now live and ready for use.
Sadly there is no indicator telling you if that particular device implements KOBJ_CHANGED at all, so one really has to know what to look out for.
But the general rule is that 'KOBJ_CHANGED' indicates that a device is not ready for use; KOBJ_ADD indicates that a device has been added to the system and _might_ indicate that the device is ready to use.

Quoted 7 lines

drivers/nvme/host/multipath.c void nvme_kick_requeue_lists(struct nvme_ctrl *ctrl) · line 108

Quoted 3 lines
drivers/nvme/host/multipath.c +3 -0
+		if (ctrl->state == NVME_CTRL_LIVE)
+			kobject_uevent(&disk_to_dev(ns->head->disk)->kobj,
+				       KOBJ_CHANGE);

Also this should probably use disk_uevent to also notify partitions. Maybe also for other existing callers.

Indeed, you are correct. Will be fixing it.

Cheers,

Hannes

Signature · 9 lines
Dr. Hannes Reinecke		        Kernel Storage Architect
[email protected]			               +49 911 74053 688
SUSE Software Solutions Germany GmbH, 90409 Nürnberg
GF: F. Imendörffer, HRB 36809 (AG Nürnberg)

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme