Skip to contentSkip to conversation

nvme: retrigger ANA log update if group descriptor isn't found

PATCH Applied Hannes Reinecke 5 participants 13 messages Dec 5, 2020 – Apr 2, 2021 NVMe

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 nvme: retrigger ANA log update if group descriptor isn't found

If ANA is enabled but no ANA group descriptor is found when creating a new namespace the ANA log is most likely out of date, so trigger a re-read. The namespace will be tagged with the NS_ANA_PENDING flag to exclude it from path selection until the ANA log has been re-read.

---
 drivers/nvme/host/multipath.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/nvme/host/multipath.c +4 -0
index 74896be40c17..ee4a42dc99a0 100644
@@ -667,6 +667,10 @@ void nvme_mpath_add_disk(struct nvme_ns *ns, struct nvme_id_ns *id)
 		if (desc.state) {
 			/* found the group desc: update */
 			nvme_update_ns_ana_state(&desc, ns);
+		} else {
+			/* group desc not found: trigger a re-read */
+			set_bit(NVME_NS_ANA_PENDING, &ns->flags);
+			queue_work(nvme_wq, &ns->ctrl->ana_work);
 		}
 	} else {
 		ns->ana_state = NVME_ANA_OPTIMIZED; 
Signature · 7 lines
2.26.2


_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
maintainer Reviewed-by on v1 1/1
#1 lore raw
Quoted 7 lines · Hannes Reinecke

If ANA is enabled but no ANA group descriptor is found when creating a new namespace the ANA log is most likely out of date, so trigger a re-read. The namespace will be tagged with the NS_ANA_PENDING flag to exclude it from path selection until the ANA log has been re-read.

This looks good to me.

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#2 lore raw
Quoted 6 lines · Keith Busch
Quoted 5 lines · Hannes Reinecke

If ANA is enabled but no ANA group descriptor is found when creating a new namespace the ANA log is most likely out of date, so trigger a re-read. The namespace will be tagged with the NS_ANA_PENDING flag to exclude it from path selection until the ANA log has been re-read.

Quoted 6 lines
Quoted 2 lines

This looks good to me.

But as I just outlined it just papers over buggy controllers. I really don't think we should just silently do that.

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

maintainer
#3 lore raw
Quoted 10 lines · Christoph Hellwig
Quoted 9 lines · Keith Busch
Quoted 7 lines · Hannes Reinecke

If ANA is enabled but no ANA group descriptor is found when creating a new namespace the ANA log is most likely out of date, so trigger a re-read. The namespace will be tagged with the NS_ANA_PENDING flag to exclude it from path selection until the ANA log has been re-read.

Quoted 6 lines
Quoted 3 lines

This looks good to me.

But as I just outlined it just papers over buggy controllers. I really don't think we should just silently do that.

Okay, that's fine with me too. I agree with you on what the correct event sequence should be, but I just thought this looks like a fairly harmless work-around.

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#7 lore raw
In reply to Christoph Hellwig
Quoted 10 lines · Christoph Hellwig
Quoted 9 lines · Keith Busch
Quoted 7 lines · Hannes Reinecke

If ANA is enabled but no ANA group descriptor is found when creating a new namespace the ANA log is most likely out of date, so trigger a re-read. The namespace will be tagged with the NS_ANA_PENDING flag to exclude it from path selection until the ANA log has been re-read.

Quoted 7 lines
Quoted 3 lines

This looks good to me.

But as I just outlined it just papers over buggy controllers. I really don't think we should just silently do that.

What would be the downside of taking this patch? Personally, I'd rather be lenient and allow to interoperate...

Cheers,

Hannes

Signature · 9 lines
Dr. Hannes Reinecke                Kernel Storage Architect
[email protected]                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
#4 lore raw
In reply to Keith Busch
Quoted 6 lines · Keith Busch
Quoted 2 lines

But as I just outlined it just papers over buggy controllers. I really don't think we should just silently do that.

Okay, that's fine with me too. I agree with you on what the correct event sequence should be, but I just thought this looks like a fairly harmless work-around.

I see three major downsides:

(1) it papers over our host side bug where we do not process the different AENs in the order that the controller generated them
(2) the code is black magic as there is no indicator why this condition would happen

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

maintainer
#5 lore raw
Quoted 9 lines · Christoph Hellwig
Quoted 6 lines · Keith Busch
Quoted 2 lines

But as I just outlined it just papers over buggy controllers. I really don't think we should just silently do that.

Okay, that's fine with me too. I agree with you on what the correct event sequence should be, but I just thought this looks like a fairly harmless work-around.

I see three major downsides:

You only listed two. :)

Quoted 2 lines

(1) it papers over our host side bug where we do not process the different AENs in the order that the controller generated them

We only have one AEN outstanding at a time, though, so we can't process them in a different order than the controller sent them, right?

Quoted 2 lines

(2) the code is black magic as there is no indicator why this condition would happen

Would it help if Hannes adds a dev_warn() that this log refresh is occuring from a controller's inconsistent reporting?

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#6 lore raw
Quoted 5 lines · Keith Busch
Quoted 4 lines · Christoph Hellwig
Quoted 3 lines · Keith Busch
Quoted 2 lines

But as I just outlined it just papers over buggy controllers. I really don't think we should just silently do that.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

Okay, that's fine with me too. I agree with you on what the correct event sequence should be, but I just thought this looks like a fairly harmless work-around.

I see three major downsides:

You only listed two. :)

Yeah.

Quoted 6 lines
Quoted 3 lines

(1) it papers over our host side bug where we do not process the different AENs in the order that the controller generated them

We only have one AEN outstanding at a time, though, so we can't process them in a different order than the controller sent them, right?

In a more theoretical than practical race the next AEN could have completed befoe the workqueue that executes the previous one has run.

Quoted 5 lines
Quoted 2 lines

(2) the code is black magic as there is no indicator why this condition would happen

Would it help if Hannes adds a dev_warn() that this log refresh is occuring from a controller's inconsistent reporting?

I really want to settle the discussion first as Fred has an extremely strange interpretation of the spec.

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

maintainer Reviewed-by on v1 1/1
#8 lore raw
In reply to Hannes Reinecke
Quoted 17 lines

If ANA is enabled but no ANA group descriptor is found when creating a new namespace the ANA log is most likely out of date, so trigger a re-read. The namespace will be tagged with the NS_ANA_PENDING flag to exclude it from path selection until the ANA log has been re-read.

---
  drivers/nvme/host/multipath.c | 4 ++++
  1 file changed, 4 insertions(+)
drivers/nvme/host/multipath.c +0 -0
index 74896be40c17..ee4a42dc99a0 100644

drivers/nvme/host/multipath.c void nvme_mpath_add_disk(struct nvme_ns *ns, struct nvme_id_ns *id) · line 673

Quoted 7 lines
drivers/nvme/host/multipath.c +4 -0
@@ -667,6 +667,10 @@ void nvme_mpath_add_disk(struct nvme_ns *ns, struct nvme_id_ns *id)
  		if (desc.state) {
  			/* found the group desc: update */
  			nvme_update_ns_ana_state(&desc, ns);
+		} else {
+			/* group desc not found: trigger a re-read */
+			set_bit(NVME_NS_ANA_PENDING, &ns->flags);
+			queue_work(nvme_wq, &ns->ctrl->ana_work);

btw, I just commented on the original thread, but I do like this one better.

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#9 lore raw
Quoted 33 lines · Sagi Grimberg
Quoted 21 lines

If ANA is enabled but no ANA group descriptor is found when creating
a new namespace the ANA log is most likely out of date, so trigger a re-read. The namespace will be tagged with the NS_ANA_PENDING flag
to exclude it from path selection until the ANA log has been re- read.

---
  drivers/nvme/host/multipath.c | 4 ++++
  1 file changed, 4 insertions(+)
drivers/nvme/host/multipath.c +0 -0
index 74896be40c17..ee4a42dc99a0 100644

drivers/nvme/host/multipath.c void nvme_mpath_add_disk(struct nvme_ns *ns, · line 674

Quoted 8 lines
drivers/nvme/host/multipath.c +4 -0
@@ -667,6 +667,10 @@ void nvme_mpath_add_disk(struct nvme_ns *ns,
 struct nvme_id_ns *id)
              if (desc.state) {
                      /* found the group desc: update */
                      nvme_update_ns_ana_state(&desc, ns);
+             } else {
+                     /* group desc not found: trigger a re-read */
+                     set_bit(NVME_NS_ANA_PENDING, &ns->flags);
+                     queue_work(nvme_wq, &ns->ctrl->ana_work);

btw, I just commented on the original thread, but I do like this one better.

Quoted 2 lines

We now have a ratified ECN which clarifies that a NVME_AEN_CFG_NS_ATTR is sufficient for announcing a new namespace along with a new ANA group, as opposed to additionally raising a NVME_AEN_CFG_ANA_CHANGE as well for the latter - available at
https://nvmexpress.org/wp-content/uploads/NVM-Express-1.4-Ratified-ECN.zip
. I hope that finally clears the confusion on this topic.

So could we now have this patch pulled in please?

-Martin
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

maintainer Reviewed-by
#10 lore raw
Quoted 37 lines · Martin
Quoted 33 lines · Sagi Grimberg
Quoted 21 lines

If ANA is enabled but no ANA group descriptor is found when creating
a new namespace the ANA log is most likely out of date, so trigger a re-read. The namespace will be tagged with the NS_ANA_PENDING flag
to exclude it from path selection until the ANA log has been re- read.

---
  drivers/nvme/host/multipath.c | 4 ++++
  1 file changed, 4 insertions(+)
drivers/nvme/host/multipath.c +0 -0
index 74896be40c17..ee4a42dc99a0 100644

drivers/nvme/host/multipath.c void nvme_mpath_add_disk(struct nvme_ns *ns, · line 674

Quoted 8 lines
drivers/nvme/host/multipath.c +4 -0
@@ -667,6 +667,10 @@ void nvme_mpath_add_disk(struct nvme_ns *ns,
 struct nvme_id_ns *id)
              if (desc.state) {
                      /* found the group desc: update */
                      nvme_update_ns_ana_state(&desc, ns);
+             } else {
+                     /* group desc not found: trigger a re-read */
+                     set_bit(NVME_NS_ANA_PENDING, &ns->flags);
+                     queue_work(nvme_wq, &ns->ctrl->ana_work);

btw, I just commented on the original thread, but I do like this one better.

Quoted 2 lines
Quoted 8 lines

We now have a ratified ECN which clarifies that a NVME_AEN_CFG_NS_ATTR is sufficient for announcing a new namespace along with a new ANA group, as opposed to additionally raising a NVME_AEN_CFG_ANA_CHANGE as well for the latter - available at
https://nvmexpress.org/wp-content/uploads/NVM-Express-1.4-Ratified-ECN.zip
. I hope that finally clears the confusion on this topic.

So could we now have this patch pulled in please?

Yeah, I was fine with the patch initiailly since it is harmless and provides higher tolerance to unclear behavior. The ECN essentially requires this patch now, so again:

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

maintainer Reviewed-by
#11 lore raw
Quoted 8 lines
Quoted 2 lines

So could we now have this patch pulled in please?

Yeah, I was fine with the patch initiailly since it is harmless and provides higher tolerance to unclear behavior. The ECN essentially requires this patch now, so again:

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme