Skip to contentSkip to conversation

Ensure ordered namespace registration during async scan

PATCH Superseded Maurizio Lombardi 2 participants 7 messages Feb 24

Subsystems (3)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover Ensure ordered namespace registration during async scan

The NVMe fully asynchronous namespace scanning introduced in commit 4e893ca81170 ("nvme-core: scan namespaces asynchronously") significantly improved discovery times. However, it also introduced non-deterministic ordering for namespace registration.

While kernel device names (/dev/nvmeXnY) are not guaranteed to be stable across reboots, this unpredictable ordering has caused considerable user confusion and has been perceived as a regression, leading to multiple bug reports.

This series introduces a solution to enforce strict sequential registration based on NSID order, entirely preserving the performance benefits of the asynchronous scan approach.

Instead of adding an NVMe-specific hack, this series abstracts the serialization mechanism currently open-coded in the SCSI subsystem (drivers/scsi/scsi_scan.c) into a generic library helper called the completion chain (compl_chain).

By enforcing a strict First-In, First-Out (FIFO) completion order for asynchronous tasks, we can ensure that namespaces are allocated and registered sequentially without blocking the underlying parallel discovery processes.

PATCH 3 Refactors the existing SCSI asynchronous scanning implementation to use the new compl_chain helper, stripping out the custom, open-coded task list and reducing code duplication.

Original code:

$ nvme list
Node                  Generic               Namespace
--------------------- --------------------- ----------
/dev/nvme0n1          /dev/ng0n1            0x2
/dev/nvme0n2          /dev/ng0n2            0x1
/dev/nvme0n3          /dev/ng0n3            0x5
/dev/nvme0n4          /dev/ng0n4            0x3
/dev/nvme0n5          /dev/ng0n5            0x4
[...]
/dev/nvme0n10         /dev/ng0n10           0xa
/dev/nvme0n11         /dev/ng0n11           0x8
/dev/nvme0n12         /dev/ng0n12           0x12
/dev/nvme0n13         /dev/ng0n13           0x17
/dev/nvme0n14         /dev/ng0n14           0xc
/dev/nvme0n15         /dev/ng0n15           0x11
/dev/nvme0n16         /dev/ng0n16           0x14
/dev/nvme0n17         /dev/ng0n17           0x13
/dev/nvme0n18         /dev/ng0n18           0xe
/dev/nvme0n19         /dev/ng0n19           0xf

With this patch:

$ nvme list
Node                  Generic               Namespace
--------------------- --------------------- ----------
/dev/nvme0n1          /dev/ng0n1            0x1
/dev/nvme0n2          /dev/ng0n2            0x2
/dev/nvme0n3          /dev/ng0n3            0x3
/dev/nvme0n4          /dev/ng0n4            0x4
/dev/nvme0n5          /dev/ng0n5            0x5
/dev/nvme0n6          /dev/ng0n6            0x6
[...]
/dev/nvme0n10         /dev/ng0n10           0xa
/dev/nvme0n11         /dev/ng0n11           0xb
/dev/nvme0n12         /dev/ng0n12           0xc
/dev/nvme0n13         /dev/ng0n13           0xd
/dev/nvme0n14         /dev/ng0n14           0xe
/dev/nvme0n15         /dev/ng0n15           0xf
/dev/nvme0n16         /dev/ng0n16           0x10
/dev/nvme0n17         /dev/ng0n17           0x11
/dev/nvme0n18         /dev/ng0n18           0x12
/dev/nvme0n19         /dev/ng0n19           0x13

V2: create the compl_chain helper that both SCSI and NVMe can share

Maurizio Lombardi (3):
lib: Introduce completion chain helper
nvme-core: register namespaces in order during async scan
scsi: Convert async scanning to use the completion chain helper

 drivers/nvme/host/core.c    |  94 ++++++++++++++++++----------
 drivers/nvme/host/nvme.h    |   2 +
 drivers/scsi/scsi_priv.h    |   2 +-
 drivers/scsi/scsi_scan.c    |  68 +++-----------------
 include/linux/compl_chain.h |  35 +++++++++++
 lib/Makefile                |   2 +-
 lib/compl_chain.c           | 121 ++++++++++++++++++++++++++++++++++++
 7 files changed, 228 insertions(+), 96 deletions(-)
 create mode 100644 include/linux/compl_chain.h
 create mode 100644 lib/compl_chain.c
Signature · 1 line
2.53.0
#1 lore raw

PATCH v2 1/3 lib: Introduce completion chain helper

Introduce a new helper library, the completion chain, designed to serialize asynchronous operations that must execute in a strict First-In, First-Out (FIFO) order.

Certain workflows, particularly in storage drivers, require operations to complete in the same sequence they were submitted.
This helper provides a generic mechanism to enforce this ordering.

compl_chain: The main structure representing the queue of operations
compl_chain_entry: An entry embedded in a per-operation structure

The typical usage pattern is:

* An operation is enqueued by calling compl_chain_add().

* The worker thread for the operation calls compl_chain_wait(), which blocks until the previously enqueued operation has finished.

* After the work is done, the thread calls compl_chain_complete(). This signals the next operation in the chain that it can now proceed and removes the current entry from the list.

---
 include/linux/compl_chain.h |  35 +++++++++++
 lib/Makefile                |   2 +-
 lib/compl_chain.c           | 120 ++++++++++++++++++++++++++++++++++++
 3 files changed, 156 insertions(+), 1 deletion(-)
 create mode 100644 include/linux/compl_chain.h
 create mode 100644 lib/compl_chain.c
include/linux/compl_chain.h new file +35 -0
new file mode 100644
index 000000000000..a2bf271144e0
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_COMPLETION_CHAIN_H
+#define _LINUX_COMPLETION_CHAIN_H
+
+#include <linux/list.h>
+#include <linux/completion.h>
+#include <linux/spinlock.h>
+
+struct compl_chain {
+	spinlock_t lock;
+	struct list_head list;
+};
+
+#define COMPL_CHAIN_INIT(name) \
+	{ .lock = __SPIN_LOCK_UNLOCKED((name).lock), \
+	  .list = LIST_HEAD_INIT((name).list) }
+
+#define DEFINE_COMPL_CHAIN(name) \
+	struct compl_chain name = COMPL_CHAIN_INIT(name)
+
+struct compl_chain_entry {
+	struct compl_chain *chain;
+	struct list_head list;
+	struct completion prev_finished;
+};
+
+void compl_chain_init(struct compl_chain *chain);
+void compl_chain_add(struct compl_chain *chain,
+			struct compl_chain_entry *entry);
+void compl_chain_wait(struct compl_chain_entry *entry);
+void compl_chain_complete(struct compl_chain_entry *entry);
+bool compl_chain_pending(struct compl_chain_entry *entry);
+void compl_chain_flush(struct compl_chain *chain);
+
+#endif /* _LINUX_COMPLETION_CHAIN_H */
lib/Makefile +1 -1
index 1b9ee167517f..c3ccd82bb190 100644
@@ -58,7 +58,7 @@ obj-y += bcd.o sort.o parser.o debug_locks.o random32.o \
 	 bsearch.o find_bit.o llist.o lwq.o memweight.o kfifo.o \
 	 percpu-refcount.o rhashtable.o base64.o \
 	 once.o refcount.o rcuref.o usercopy.o errseq.o bucket_locks.o \
-	 generic-radix-tree.o bitmap-str.o
+	 generic-radix-tree.o bitmap-str.o compl_chain.o
 obj-y += string_helpers.o
 obj-y += hexdump.o
 obj-$(CONFIG_TEST_HEXDUMP) += test_hexdump.o
lib/compl_chain.c new file +120 -0

120 lines not on this page · raw message

Signature · 1 line
2.53.0
#3 lore raw

PATCH v2 2/3 nvme-core: register namespaces in order during async scan

In reply to Maurizio Lombardi

The fully asynchronous namespace scanning, while fast, can result in namespaces being allocated and registered out of order. This leads to unpredictable device naming across reboots which can be confusing for users.

To solve this, introduce a serialization mechanism for the asynchronous namespace scan. This is achieved by using the generic compl_chain helper, which ensures that the initialization of one namespace (nvme_alloc_ns) completes before the next one begins.

This approach preserves the performance benefits of asynchronous identification while guaranteeing that the final device registration occurs in the correct order.

Performance testing shows that this change has no noticeable impact on scan times compared to the fully asynchronous method.

High latency NVMe/TCP, ~150ms ping, 100 namespaces

Synchronous namespace scan (RHEL-10.1): 32375ms
Fully async namespace scan (7.0-rc1): 2543ms
Async namespace scan with dependency chain (7.0-rc1): 2431ms

Low latency NVMe/TCP, ~0.2ms ping, 100 namespaces

Synchronous namespace scan (RHEL-10.1): 352ms Fully async namespace scan (7.0-rc1): 248ms
Async namespace scan with dependency chain (7.0-rc1): 191ms

---
 drivers/nvme/host/core.c | 94 +++++++++++++++++++++++++---------------
 drivers/nvme/host/nvme.h |  2 +
 2 files changed, 62 insertions(+), 34 deletions(-)
drivers/nvme/host/core.c +60 -34
index f5ebcaa2f859..24c62bdf8abc 100644
@@ -4105,13 +4105,27 @@ static void nvme_ns_add_to_ctrl_list(struct nvme_ns *ns)
 	list_add_rcu(&ns->list, &ns->ctrl->namespaces);
 }
 
-static void nvme_alloc_ns(struct nvme_ctrl *ctrl, struct nvme_ns_info *info)
+/**
+ * struct async_scan_task - keeps track of controller & NSID to scan
+ * @entry               link to the completion chain list
+ * @ctrl:		Controller on which namespaces are being scanned
+ * @nsid:		The NSID to scan
+ */
+struct async_scan_task {
+	struct compl_chain_entry chain_entry;
+	struct nvme_ctrl *ctrl;
+	u32 nsid;
+};
+
+static void nvme_alloc_ns(struct nvme_ctrl *ctrl, struct nvme_ns_info *info,
+				struct compl_chain_entry *cc_entry)
 {
 	struct queue_limits lim = { };
 	struct nvme_ns *ns;
 	struct gendisk *disk;
 	int node = ctrl->numa_node;
 	bool last_path = false;
+	int r;
 
 	ns = kzalloc_node(sizeof(*ns), GFP_KERNEL, node);
 	if (!ns)
@@ -4134,7 +4148,19 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, struct nvme_ns_info *info)
 	ns->ctrl = ctrl;
 	kref_init(&ns->kref);
 
-	if (nvme_init_ns_head(ns, info))
+	/*
+	 * Wait for the previous async task to finish before
+	 * allocating the namespace.
+	 */
+	if (cc_entry)
+		compl_chain_wait(cc_entry);
+
+	r = nvme_init_ns_head(ns, info);
+
+	if (cc_entry)
+		compl_chain_complete(cc_entry);
+
+	if (r)
 		goto out_cleanup_disk;
 
 	/*
@@ -4309,7 +4335,8 @@ static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_info *info)
 		nvme_ns_remove(ns);
 }
 
-static void nvme_scan_ns(struct nvme_ctrl *ctrl, unsigned nsid)
+static void nvme_scan_ns(struct nvme_ctrl *ctrl, unsigned int nsid,
+				struct compl_chain_entry *cc_entry)
 {
 	struct nvme_ns_info info = { .nsid = nsid };
 	struct nvme_ns *ns;
@@ -4348,40 +4375,30 @@ static void nvme_scan_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 
 	ns = nvme_find_get_ns(ctrl, nsid);
 	if (ns) {
+		/* Release the chain early so the next task can proceed */
+		if (cc_entry)
+			compl_chain_complete(cc_entry);
 		nvme_validate_ns(ns, &info);
 		nvme_put_ns(ns);
 	} else {
-		nvme_alloc_ns(ctrl, &info);
+		nvme_alloc_ns(ctrl, &info, cc_entry);
 	}
 }
 
-/**
- * struct async_scan_info - keeps track of controller & NSIDs to scan
- * @ctrl:	Controller on which namespaces are being scanned
- * @next_nsid:	Index of next NSID to scan in ns_list
- * @ns_list:	Pointer to list of NSIDs to scan
- *
- * Note: There is a single async_scan_info structure shared by all instances
- * of nvme_scan_ns_async() scanning a given controller, so the atomic
- * operations on next_nsid are critical to ensure each instance scans a unique
- * NSID.
- */
-struct async_scan_info {
-	struct nvme_ctrl *ctrl;
-	atomic_t next_nsid;
-	__le32 *ns_list;
-};
-
 static void nvme_scan_ns_async(void *data, async_cookie_t cookie)
 {
-	struct async_scan_info *scan_info = data;
-	int idx;
-	u32 nsid;
+	struct async_scan_task *task = data;
+
+	nvme_scan_ns(task->ctrl, task->nsid, &task->chain_entry);
 
-	idx = (u32)atomic_fetch_inc(&scan_info->next_nsid);
-	nsid = le32_to_cpu(scan_info->ns_list[idx]);
+	/*
+	 * If the task failed early and returned without completing the
+	 * chain entry, ensure the chain progresses safely.
+	 */
+	if (compl_chain_pending(&task->chain_entry))
+		compl_chain_complete(&task->chain_entry);
 
-	nvme_scan_ns(scan_info->ctrl, nsid);
+	kfree(task);
 }
 
 static void nvme_remove_invalid_namespaces(struct nvme_ctrl *ctrl,
@@ -4411,14 +4428,12 @@ static int nvme_scan_ns_list(struct nvme_ctrl *ctrl)
 	u32 prev = 0;
 	int ret = 0, i;
 	ASYNC_DOMAIN(domain);
-	struct async_scan_info scan_info;
+	struct async_scan_task *task;
 
 	ns_list = kzalloc(NVME_IDENTIFY_DATA_SIZE, GFP_KERNEL);
 	if (!ns_list)
 		return -ENOMEM;
 
-	scan_info.ctrl = ctrl;
-	scan_info.ns_list = ns_list;
 	for (;;) {
 		struct nvme_command cmd = {
 			.identify.opcode	= nvme_admin_identify,
@@ -4434,20 +4449,30 @@ static int nvme_scan_ns_list(struct nvme_ctrl *ctrl)
 			goto free;
 		}
 
-		atomic_set(&scan_info.next_nsid, 0);
 		for (i = 0; i < nr_entries; i++) {
 			u32 nsid = le32_to_cpu(ns_list[i]);
 
 			if (!nsid)	/* end of the list? */
 				goto out;
-			async_schedule_domain(nvme_scan_ns_async, &scan_info,
+
+			task = kmalloc_obj(*task);
+			if (!task) {
+				ret = -ENOMEM;
+				goto out;
+			}
+
+			task->nsid = nsid;
+			task->ctrl = ctrl;
+			compl_chain_add(&ctrl->scan_chain, &task->chain_entry);
+
+			async_schedule_domain(nvme_scan_ns_async, task,
 						&domain);
 			while (++prev < nsid)
 				nvme_ns_remove_by_nsid(ctrl, prev);
 		}
-		async_synchronize_full_domain(&domain);
 	}
  out:
+	async_synchronize_full_domain(&domain);
 	nvme_remove_invalid_namespaces(ctrl, prev);
  free:
 	async_synchronize_full_domain(&domain);
@@ -4466,7 +4491,7 @@ static void nvme_scan_ns_sequential(struct nvme_ctrl *ctrl)
 	kfree(id);
 
 	for (i = 1; i <= nn; i++)
-		nvme_scan_ns(ctrl, i);
+		nvme_scan_ns(ctrl, i, NULL);
 
 	nvme_remove_invalid_namespaces(ctrl, nn);
 }
@@ -5094,6 +5119,7 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
 
 	mutex_init(&ctrl->scan_lock);
 	INIT_LIST_HEAD(&ctrl->namespaces);
+	compl_chain_init(&ctrl->scan_chain);
 	xa_init(&ctrl->cels);
 	ctrl->dev = dev;
 	ctrl->ops = ops;
drivers/nvme/host/nvme.h +2 -0
index 9a5f28c5103c..95f8c40ec86b 100644
@@ -17,6 +17,7 @@
 #include <linux/wait.h>
 #include <linux/t10-pi.h>
 #include <linux/ratelimit_types.h>
+#include <linux/compl_chain.h>
 
 #include <trace/events/block.h>
 
@@ -294,6 +295,7 @@ struct nvme_ctrl {
 	struct blk_mq_tag_set *tagset;
 	struct blk_mq_tag_set *admin_tagset;
 	struct list_head namespaces;
+	struct compl_chain scan_chain;
 	struct mutex namespaces_lock;
 	struct srcu_struct srcu;
 	struct device ctrl_device;
Signature · 1 line
2.53.0
#5 lore raw

PATCH v2 3/3 scsi: Convert async scanning to use the completion chain helper

In reply to Maurizio Lombardi

The asynchronous host scanning logic in scsi_scan.c uses a custom, open-coded implementation to serialize scans. This involves a manually managed list of tasks, each with its own completion, to ensure that hosts are scanned and added to the system in a deterministic order.

Refactors the SCSI async scanning implementation to use the new compl_async_chain helper. This simplifies the scsi_scan.c code and makes the serialization logic more readable.

---
 drivers/scsi/scsi_priv.h |  2 +-
 drivers/scsi/scsi_scan.c | 68 +++++-----------------------------------
 2 files changed, 9 insertions(+), 61 deletions(-)
drivers/scsi/scsi_priv.h +1 -1
index 7a193cc04e5b..d8a157bc9078 100644
@@ -132,7 +132,7 @@ extern void scsi_exit_procfs(void);
 
 /* scsi_scan.c */
 void scsi_enable_async_suspend(struct device *dev);
-extern int scsi_complete_async_scans(void);
+extern void scsi_complete_async_scans(void);
 extern int scsi_scan_host_selected(struct Scsi_Host *, unsigned int,
 				   unsigned int, u64, enum scsi_scan_mode);
 extern void scsi_forget_host(struct Scsi_Host *);
drivers/scsi/scsi_scan.c +8 -60

115 lines not on this page · raw message

Signature · 1 line
2.53.0
#6 lore raw

Re: [PATCH V2 3/3] scsi: Convert async scanning to use the completion chain helper

drivers/scsi/scsi_priv.h extern void scsi_exit_procfs(void); · line 135

Quoted 10 lines · Maurizio Lombardi
drivers/scsi/scsi_priv.h +1 -1
index 7a193cc04e5b..d8a157bc9078 100644
@@ -132,7 +132,7 @@ extern void scsi_exit_procfs(void);
 
 /* scsi_scan.c */
 void scsi_enable_async_suspend(struct device *dev);
-extern int scsi_complete_async_scans(void);
+extern void scsi_complete_async_scans(void);

No need for the extern here.

drivers/scsi/scsi_priv.h extern void scsi_exit_procfs(void); · line 136

Quoted 1 line
drivers/scsi/scsi_priv.h +1 -0
+DEFINE_COMPL_CHAIN(scanning_hosts);

This looks like it should be marked static.