Skip to contentSkip to conversation

nvme: do not ignore nvme status in nvme_set_queue_count()

PATCH In review Hannes Reinecke 3 participants 7 messages Jan 21, 2021 – Jan 26, 2021 NVMe

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 nvme: do not ignore nvme status in nvme_set_queue_count()

If the call to nvme_set_queue_count() fails with a status we should not ignore it but rather pass it on to the caller.
It's then up to the transport to decide whether to ignore it (like PCI does) or to reset the connection (as would be appropriate for fabrics).

---
 drivers/nvme/host/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/nvme/host/core.c +1 -1
index ce1b61519441..ddf32f5b4534 100644
@@ -1486,7 +1486,7 @@ int nvme_set_queue_count(struct nvme_ctrl *ctrl, int *count)
 		*count = min(*count, nr_io_queues);
 	}
 
-	return 0;
+	return status;
 }
 EXPORT_SYMBOL_GPL(nvme_set_queue_count);
 
Signature · 7 lines
2.26.2


_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Reviewed-by on v1 1/1
#1 lore raw
Quoted 16 lines · Hannes Reinecke

If the call to nvme_set_queue_count() fails with a status we should not ignore it but rather pass it on to the caller.
It's then up to the transport to decide whether to ignore it (like PCI does) or to reset the connection (as would be appropriate for fabrics).

---
 drivers/nvme/host/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/nvme/host/core.c +0 -0
index ce1b61519441..ddf32f5b4534 100644

drivers/nvme/host/core.c int nvme_set_queue_count(struct nvme_ctrl *ctrl, int *count) · line 1492

Quoted 8 lines
drivers/nvme/host/core.c +1 -1
@@ -1486,7 +1486,7 @@ int nvme_set_queue_count(struct nvme_ctrl *ctrl, int *count)
 		*count = min(*count, nr_io_queues);
 	}
 
-	return 0;
+	return status;
 }
 EXPORT_SYMBOL_GPL(nvme_set_queue_count);
 

Looks good.

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

maintainer
#2 lore raw
In reply to Hannes Reinecke
Quoted 5 lines · Hannes Reinecke

If the call to nvme_set_queue_count() fails with a status we should not ignore it but rather pass it on to the caller.
It's then up to the transport to decide whether to ignore it (like PCI does) or to reset the connection (as would be appropriate for fabrics).

Instead of checking the error, wouldn't checking the number of created queues be sufficient? What handling difference do you expect to occur between getting a success with 0 queues, vs getting an error?

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#3 lore raw
Quoted 11 lines · Keith Busch
Quoted 5 lines · Hannes Reinecke

If the call to nvme_set_queue_count() fails with a status we should not ignore it but rather pass it on to the caller.
It's then up to the transport to decide whether to ignore it (like PCI does) or to reset the connection (as would be appropriate for fabrics).

Instead of checking the error, wouldn't checking the number of created queues be sufficient? What handling difference do you expect to occur between getting a success with 0 queues, vs getting an error?

The difference is that an error will (re-)start recovery, 0 queues won't.
But the problem here is that nvme_set_queue_count() is being called during reconnection, ie during the recovery process itself.
And this command is returned with a timeout, which in any other case is being treated as a fatal error. Plus we have been sending this command on the admin queue, so a timeout on the admin queue pretty much _is_ a fatal error. So we should be terminating the current recovery and reconnect. None of that will happen if we return '0' queues.

Cheers,

Hannes

Signature · 9 lines
Dr. Hannes Reinecke                Kernel Storage Architect
[email protected]                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
maintainer
#4 lore raw
Quoted 12 lines · Hannes Reinecke
Quoted 11 lines · Keith Busch
Quoted 5 lines · Hannes Reinecke

If the call to nvme_set_queue_count() fails with a status we should not ignore it but rather pass it on to the caller.
It's then up to the transport to decide whether to ignore it (like PCI does) or to reset the connection (as would be appropriate for fabrics).

Instead of checking the error, wouldn't checking the number of created queues be sufficient? What handling difference do you expect to occur between getting a success with 0 queues, vs getting an error?

Quoted 8 lines

The difference is that an error will (re-)start recovery, 0 queues won't. But the problem here is that nvme_set_queue_count() is being called during reconnection, ie during the recovery process itself.
And this command is returned with a timeout, which in any other case is being treated as a fatal error. Plus we have been sending this command on the admin queue, so a timeout on the admin queue pretty much _is_ a fatal error. So we should be terminating the current recovery and reconnect. None of that will happen if we return '0' queues.

You should already be getting an error return status if a timeout occurs for nvme_set_queue_count(), specifically -EINTR. Are you getting success for some reason?

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#5 lore raw
Quoted 22 lines · Keith Busch
Quoted 12 lines · Hannes Reinecke
Quoted 11 lines · Keith Busch
Quoted 5 lines · Hannes Reinecke

If the call to nvme_set_queue_count() fails with a status we should not ignore it but rather pass it on to the caller.
It's then up to the transport to decide whether to ignore it (like PCI does) or to reset the connection (as would be appropriate for fabrics).

Instead of checking the error, wouldn't checking the number of created queues be sufficient? What handling difference do you expect to occur between getting a success with 0 queues, vs getting an error?

Quoted 8 lines

The difference is that an error will (re-)start recovery, 0 queues won't. But the problem here is that nvme_set_queue_count() is being called during reconnection, ie during the recovery process itself.
And this command is returned with a timeout, which in any other case is being treated as a fatal error. Plus we have been sending this command on the admin queue, so a timeout on the admin queue pretty much _is_ a fatal error. So we should be terminating the current recovery and reconnect. None of that will happen if we return '0' queues.

Quoted 4 lines

You should already be getting an error return status if a timeout occurs for nvme_set_queue_count(), specifically -EINTR. Are you getting success for some reason?

-EINTR (which translates to 'nvme_req(req)->flags & NVME_REQ_CANCELLED') will only ever be returned on pci; fabrics doesn't set this flag, so we're never getting an -EINTR.

Cheers,

Hannes

Signature · 9 lines
Dr. Hannes Reinecke                Kernel Storage Architect
[email protected]                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
maintainer
#6 lore raw
Quoted 23 lines · Hannes Reinecke
Quoted 22 lines · Keith Busch
Quoted 20 lines · Hannes Reinecke
Quoted 11 lines · Keith Busch
Quoted 5 lines · Hannes Reinecke

If the call to nvme_set_queue_count() fails with a status we should not ignore it but rather pass it on to the caller.
It's then up to the transport to decide whether to ignore it (like PCI does) or to reset the connection (as would be appropriate for fabrics).

Instead of checking the error, wouldn't checking the number of created queues be sufficient? What handling difference do you expect to occur between getting a success with 0 queues, vs getting an error?

The difference is that an error will (re-)start recovery, 0 queues won't. But the problem here is that nvme_set_queue_count() is being called during reconnection, ie during the recovery process itself.
And this command is returned with a timeout, which in any other case is being treated as a fatal error. Plus we have been sending this command on the admin queue, so a timeout on the admin queue pretty much _is_ a fatal error. So we should be terminating the current recovery and reconnect. None of that will happen if we return '0' queues.

Quoted 7 lines
Quoted 4 lines

You should already be getting an error return status if a timeout occurs for nvme_set_queue_count(), specifically -EINTR. Are you getting success for some reason?

-EINTR (which translates to 'nvme_req(req)->flags & NVME_REQ_CANCELLED') will only ever be returned on pci; fabrics doesn't set this flag, so we're never getting an -EINTR.

Sounds like that's the problem that needs to be fixed.

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme