Skip to contentSkip to conversation

nvme: Do not reject dynamic controller cntlid

RFC In review Daniel Wagner 3 participants 5 messages Jan 27, 2022 – Jan 28, 2022 NVMe

lore
Share & tools
lore
#0 lore raw

RFC 1/1 nvme: Do not reject dynamic controller cntlid

The NVME spec states in 3.1.1 Controller Model there are two types
controllers: dynamic and static.

The nvme_validate_cntlid() check ensures that the newly discovered controller does not have the same cntlid as a known controller. For the dynamic controller model this check doesn't make sense as all cntlid will be 0xffff. In this case ignore this check.

---

I got a bug report with this:

nvme nvme10: pci function 0000:00:19.0
nvme 0000:00:19.0: enabling device (0000 -> 0002) nvme nvme10: Duplicate cntlid 65535 with nvme7, rejecting nvme nvme10: Removing after probe failure status: -22

From reading the spec I got the impression that it should be enough to ignore the cntlid check if the cntlid is 0xffff. Though I might be completely wrong here.

 drivers/nvme/host/core.c | 2 +-
 drivers/nvme/host/nvme.h | 5 +++++
 2 files changed, 6 insertions(+), 1 deletion(-)
drivers/nvme/host/core.c +1 -1
index dd18861f77c0..918abb54771c 100644
@@ -2701,7 +2701,7 @@ static bool nvme_validate_cntlid(struct nvme_subsystem *subsys,
 		if (nvme_state_terminal(tmp))
 			continue;
 
-		if (tmp->cntlid == ctrl->cntlid) {
+		if (!nvme_ctrl_dynamic(tmp) && tmp->cntlid == ctrl->cntlid) {
 			dev_err(ctrl->device,
 				"Duplicate cntlid %u with %s, subsys %s, rejecting\n",
 				ctrl->cntlid, dev_name(tmp->device),
drivers/nvme/host/nvme.h +5 -0
index a162f6c6da6e..ed75245263f5 100644
@@ -539,6 +539,11 @@ static inline struct request *nvme_cid_to_rq(struct blk_mq_tags *tags,
 	return blk_mq_tag_to_rq(tags, nvme_tag_from_cid(command_id));
 }
 
+static inline bool nvme_ctrl_dynamic(struct nvme_ctrl *ctrl)
+{
+	return ctrl->cntlid == 0xffff;
+}
+
 #ifdef CONFIG_FAULT_INJECTION_DEBUG_FS
 void nvme_fault_inject_init(struct nvme_fault_inject *fault_inj,
 			    const char *dev_name);
Signature · 1 line
2.29.2
maintainer
#1 lore raw
Quoted 17 lines · Daniel Wagner

index dd18861f77c0..918abb54771c 100644

drivers/nvme/host/core.c +1 -1
@@ -2701,7 +2701,7 @@ static bool nvme_validate_cntlid(struct nvme_subsystem *subsys,
 		if (nvme_state_terminal(tmp))
 			continue;
 
-		if (tmp->cntlid == ctrl->cntlid) {
+		if (!nvme_ctrl_dynamic(tmp) && tmp->cntlid == ctrl->cntlid) {
 			dev_err(ctrl->device,
 				"Duplicate cntlid %u with %s, subsys %s, rejecting\n",
 				ctrl->cntlid, dev_name(tmp->device),
drivers/nvme/host/nvme.h +0 -0
index a162f6c6da6e..ed75245263f5 100644

drivers/nvme/host/nvme.h static inline struct request *nvme_cid_to_rq(struct blk_mq_tags *tags, · line 545

Quoted 7 lines
drivers/nvme/host/nvme.h +4 -0
@@ -539,6 +539,11 @@ static inline struct request *nvme_cid_to_rq(struct blk_mq_tags *tags,
 	return blk_mq_tag_to_rq(tags, nvme_tag_from_cid(command_id));
 }
 
+static inline bool nvme_ctrl_dynamic(struct nvme_ctrl *ctrl)
+{
+	return ctrl->cntlid == 0xffff;
+}

It's probably safe to assume 0xffff is dynamic, but spec suggests we check ID_CTRL.FCATT bit 0.

#2 lore raw
Quoted 7 lines · Keith Busch

drivers/nvme/host/nvme.h static inline struct request *nvme_cid_to_rq(struct blk_mq_tags *tags, · line 545

Quoted 4 lines
drivers/nvme/host/nvme.h +4 -0
+static inline bool nvme_ctrl_dynamic(struct nvme_ctrl *ctrl)
+{
+	return ctrl->cntlid == 0xffff;
+}

It's probably safe to assume 0xffff is dynamic, but spec suggests we check ID_CTRL.FCATT bit 0.

Okay, but this one is only defined for fabrics. I haven't found anything so far which is equivalent to FCATT bit 0 for memory based transport.

#3 lore raw
Quoted 11 lines · Daniel Wagner
Quoted 7 lines · Keith Busch

drivers/nvme/host/nvme.h static inline struct request *nvme_cid_to_rq(struct blk_mq_tags *tags, · line 545

Quoted 4 lines
drivers/nvme/host/nvme.h +4 -0
+static inline bool nvme_ctrl_dynamic(struct nvme_ctrl *ctrl)
+{
+	return ctrl->cntlid == 0xffff;
+}

It's probably safe to assume 0xffff is dynamic, but spec suggests we check ID_CTRL.FCATT bit 0.

Okay, but this one is only defined for fabrics. I haven't found anything so far which is equivalent to FCATT bit 0 for memory based transport.

Never mind. After discussing it with Hannes it turns out there is no problem here. I didn't understand the spec correctly (it's difficult to get an exact answer) on this topic.

#4 lore raw
Quoted 10 lines · Daniel Wagner
Quoted 9 lines · Daniel Wagner
Quoted 7 lines · Keith Busch

drivers/nvme/host/nvme.h static inline struct request *nvme_cid_to_rq(struct blk_mq_tags *tags, · line 545

Quoted 4 lines
drivers/nvme/host/nvme.h +4 -0
+static inline bool nvme_ctrl_dynamic(struct nvme_ctrl *ctrl)
+{
+	return ctrl->cntlid == 0xffff;
+}

It's probably safe to assume 0xffff is dynamic, but spec suggests we check ID_CTRL.FCATT bit 0.

Quoted 7 lines
Quoted 2 lines

Okay, but this one is only defined for fabrics. I haven't found anything so far which is equivalent to FCATT bit 0 for memory based transport.

Never mind. After discussing it with Hannes it turns out there is no problem here. I didn't understand the spec correctly (it's difficult to get an exact answer) on this topic.

Weasely wording in the spec again.
It talks at great length on controller IDs for fabrics, which values are allowed and which not, and how one should do dynamic controller ids _on fabrics_, but is strangely silent for memory-based (ie PCI) transports. There, apparently, 0xFFFF _is_ a valid controller id, and the only restriction is that the controller ID must be unique.
And even that is never stated directly, but must be inferred from the various command payload descriptions.
Guess it's a topic for the fmds call.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke                Kernel Storage Architect
[email protected]                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer