nvme-fabrics: short-circuit connect retries
v3 cover nvme-fabrics: short-circuit connect retries
I've picked up Hannes' DNR patches. In short the make the transports behave the same way when the DNR bit set on a re-connect attempt. We had a discussion this topic in the past and if I got this right we all agreed is that the host should honor the DNR bit on a connect attempt [1]
The nvme/045 test case (authentication tests) in blktests is a good test case for this after extending it slightly. TCP and RDMA try to reconnect with an invalid key over and over again, while loop and FC stop after the first fail.
[1] https://lore.kernel.org/linux-nvme/[email protected]/
changes:
v3:
- added my SOB tag
- fixed indention
v2:
- refresh/rebase on current head
- extended blktests (nvme/045) to cover this case
(see separate post)
- https://lore.kernel.org/linux-nvme/[email protected]/
v1:
- initial version
- https://lore.kernel.org/linux-nvme/[email protected]/
Hannes Reinecke (2):
nvme-tcp: short-circuit reconnect retries
nvme-rdma: short-circuit reconnect retries
drivers/nvme/host/rdma.c | 22 +++++++++++++++------- drivers/nvme/host/tcp.c | 23 +++++++++++++++-------- 2 files changed, 30 insertions(+), 15 deletions(-)
Signature · 1 line
2.44.0
PATCH v3 1/2 nvme-tcp: short-circuit reconnect retries
From: Hannes Reinecke <[email protected]>
Returning an nvme status from nvme_tcp_setup_ctrl() indicates that the association was established and we have received a status from the controller; consequently we should honour the DNR bit. If not any future reconnect attempts will just return the same error, so we can short-circuit the reconnect attempts and fail the connection directly.
--- drivers/nvme/host/tcp.c | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-)
drivers/nvme/host/tcp.c +15 -8
@@ -2149,9 +2149,11 @@ static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,nvme_tcp_destroy_io_queues (ctrl, remove); } -static void nvme_tcp_reconnect_or_remove (struct nvme_ctrl *ctrl) +static void nvme_tcp_reconnect_or_remove (struct nvme_ctrl *ctrl, +int status) {enum nvme_ctrl_state state =nvme_ctrl_state (ctrl); +bool recon = status >0 && (status &NVME_SC_DNR ) ? false : true;/* If we are resetting/deleting then do nothing */ if (state !=NVME_CTRL_CONNECTING ) {
@@ -2159,13 +2161,14 @@ static void nvme_tcp_reconnect_or_remove(struct nvme_ctrl *ctrl)return ; } -if (nvmf_should_reconnect (ctrl)) { +if (recon &&nvmf_should_reconnect (ctrl)) {dev_info (ctrl->device,"Reconnecting in %d seconds...\n" , ctrl->opts->reconnect_delay);queue_delayed_work (nvme_wq, &to_tcp_ctrl (ctrl)->connect_work, ctrl->opts->reconnect_delay *HZ ); }else { -dev_info (ctrl->device,"Removing controller...\n" ); +dev_info (ctrl->device,"Removing controller (%d)...\n" , + status);nvme_delete_ctrl (ctrl); } }
@@ -2246,10 +2249,12 @@ static void nvme_tcp_reconnect_ctrl_work(struct work_struct *work)struct nvme_tcp_ctrl *tcp_ctrl =container_of (to_delayed_work (work), struct nvme_tcp_ctrl, connect_work);struct nvme_ctrl *ctrl = &tcp_ctrl->ctrl; +int ret; ++ctrl->nr_reconnects; -if (nvme_tcp_setup_ctrl (ctrl, false)) + ret =nvme_tcp_setup_ctrl (ctrl, false); +if (ret) goto requeue; dev_info(ctrl->device, "Successfully reconnected (%d attempt)\n",
@@ -2262,7 +2267,7 @@ static void nvme_tcp_reconnect_ctrl_work(struct work_struct *work) requeue:dev_info (ctrl->device,"Failed reconnect attempt %d\n" , ctrl->nr_reconnects); -nvme_tcp_reconnect_or_remove (ctrl); +nvme_tcp_reconnect_or_remove (ctrl, ret); }static void nvme_tcp_error_recovery_work (struct work_struct *work)
@@ -2289,7 +2294,7 @@ static void nvme_tcp_error_recovery_work(struct work_struct *work)return ; } -nvme_tcp_reconnect_or_remove (ctrl); +nvme_tcp_reconnect_or_remove (ctrl, -ENOTCONN ); }static void nvme_tcp_teardown_ctrl (struct nvme_ctrl *ctrl,bool shutdown)
@@ -2309,6 +2314,7 @@ static void nvme_reset_ctrl_work(struct work_struct *work) {struct nvme_ctrl *ctrl = container_of(work, struct nvme_ctrl, reset_work); +int ret;nvme_stop_ctrl (ctrl);nvme_tcp_teardown_ctrl (ctrl, false);
@@ -2322,14 +2328,15 @@ static void nvme_reset_ctrl_work(struct work_struct *work)return ; } -if (nvme_tcp_setup_ctrl (ctrl, false)) + ret =nvme_tcp_setup_ctrl (ctrl, false); +if (ret) goto out_fail;return ; out_fail: ++ctrl->nr_reconnects; -nvme_tcp_reconnect_or_remove (ctrl); +nvme_tcp_reconnect_or_remove (ctrl, ret); }static void nvme_tcp_stop_ctrl (struct nvme_ctrl *ctrl)
Signature · 1 line
2.44.0
Re: [PATCH v3 1/2] nvme-tcp: short-circuit reconnect retries
Quoted 20 lines · Daniel Wagner
From: Hannes Reinecke <[email protected]>
Returning an nvme status from
nvme_tcp_setup_ctrl()indicates that the association was established and we have received a status from the controller; consequently we should honour the DNR bit. If not any future reconnect attempts will just return the same error, so we can short-circuit the reconnect attempts and fail the connection directly.Signed-off-by Hannes Reinecke <[email protected]>Signed-off-by Daniel Wagner <[email protected]>--- drivers/nvme/host/tcp.c | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-)drivers/nvme/host/tcp.c +0
-0
drivers/nvme/host/tcp.c static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl, · line 2157
Quoted 10 lines
drivers/nvme/host/tcp.c +3
-1@@ -2149,9 +2149,11 @@ static void nvme_tcp_teardown_io_queues(struct nvme_ctrl *ctrl,nvme_tcp_destroy_io_queues (ctrl, remove); } -static void nvme_tcp_reconnect_or_remove (struct nvme_ctrl *ctrl) +static void nvme_tcp_reconnect_or_remove (struct nvme_ctrl *ctrl, +int status) {enum nvme_ctrl_state state =nvme_ctrl_state (ctrl); +bool recon = status >0 && (status &NVME_SC_DNR ) ? false : true;
hmmm bunch of nvme command calls may have DNR set ...
Looks good.
-ck
PATCH v3 2/2 nvme-rdma: short-circuit reconnect retries
In reply to Daniel WagnerFrom: Hannes Reinecke <[email protected]>
Returning an nvme status from nvme_rdma_setup_ctrl() indicates that the association was established and we have received a status from the controller; consequently we should honour the DNR bit. If not any future reconnect attempts will just return the same error, so we can short-circuit the reconnect attempts and fail the connection directly.
--- drivers/nvme/host/rdma.c | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-)
drivers/nvme/host/rdma.c +15 -7
@@ -982,9 +982,11 @@ static void nvme_rdma_free_ctrl(struct nvme_ctrl *nctrl)kfree (ctrl); } -static void nvme_rdma_reconnect_or_remove (struct nvme_rdma_ctrl *ctrl) +static void nvme_rdma_reconnect_or_remove (struct nvme_rdma_ctrl *ctrl, +int status) {enum nvme_ctrl_state state =nvme_ctrl_state (&ctrl->ctrl); +bool recon = status >0 && (status &NVME_SC_DNR ) ? false : true;/* If we are resetting/deleting then do nothing */ if (state !=NVME_CTRL_CONNECTING ) {
@@ -992,12 +994,14 @@ static void nvme_rdma_reconnect_or_remove(struct nvme_rdma_ctrl *ctrl)return ; } -if (nvmf_should_reconnect (&ctrl->ctrl)) { +if (recon &&nvmf_should_reconnect (&ctrl->ctrl)) {dev_info (ctrl->ctrl.device,"Reconnecting in %d seconds...\n" , ctrl->ctrl.opts->reconnect_delay);queue_delayed_work (nvme_wq, &ctrl->reconnect_work, ctrl->ctrl.opts->reconnect_delay *HZ ); }else { +dev_info (ctrl->ctrl.device,"Removing controller (%d)...\n" , + status);nvme_delete_ctrl (&ctrl->ctrl); } }
@@ -1098,10 +1102,12 @@ static void nvme_rdma_reconnect_ctrl_work(struct work_struct *work) {struct nvme_rdma_ctrl *ctrl =container_of (to_delayed_work (work), struct nvme_rdma_ctrl, reconnect_work); +int ret; ++ctrl->ctrl.nr_reconnects; -if (nvme_rdma_setup_ctrl (ctrl, false)) + ret =nvme_rdma_setup_ctrl (ctrl, false); +if (ret) goto requeue; dev_info(ctrl->ctrl.device, "Successfully reconnected (%d attempts)\n",
@@ -1114,7 +1120,7 @@ static void nvme_rdma_reconnect_ctrl_work(struct work_struct *work) requeue:dev_info (ctrl->ctrl.device,"Failed reconnect attempt %d\n" , ctrl->ctrl.nr_reconnects); -nvme_rdma_reconnect_or_remove (ctrl); +nvme_rdma_reconnect_or_remove (ctrl, ret); }static void nvme_rdma_error_recovery_work (struct work_struct *work)
@@ -1139,7 +1145,7 @@ static void nvme_rdma_error_recovery_work(struct work_struct *work)return ; } -nvme_rdma_reconnect_or_remove (ctrl); +nvme_rdma_reconnect_or_remove (ctrl, -ENOTCONN ); }static void nvme_rdma_error_recovery (struct nvme_rdma_ctrl *ctrl)
@@ -2163,6 +2169,7 @@ static void nvme_rdma_reset_ctrl_work(struct work_struct *work) {struct nvme_rdma_ctrl *ctrl = container_of(work, struct nvme_rdma_ctrl, ctrl.reset_work); +int ret;nvme_stop_ctrl (&ctrl->ctrl);nvme_rdma_shutdown_ctrl (ctrl, false);
@@ -2173,14 +2180,15 @@ static void nvme_rdma_reset_ctrl_work(struct work_struct *work)return ; } -if (nvme_rdma_setup_ctrl (ctrl, false)) + ret =nvme_rdma_setup_ctrl (ctrl, false); +if (ret) goto out_fail;return ; out_fail: ++ctrl->ctrl.nr_reconnects; -nvme_rdma_reconnect_or_remove (ctrl); +nvme_rdma_reconnect_or_remove (ctrl, ret); }static const struct nvme_ctrl_ops nvme_rdma_ctrl_ops = {
Signature · 1 line
2.44.0
Re: [PATCH v3 2/2] nvme-rdma: short-circuit reconnect retries
Quoted 9 lines · Daniel Wagner
From: Hannes Reinecke <[email protected]>
Returning an nvme status from
nvme_rdma_setup_ctrl()indicates that the association was established and we have received a status from the controller; consequently we should honour the DNR bit. If not any future reconnect attempts will just return the same error, so we can short-circuit the reconnect attempts and fail the connection directly.
Quoted 4 lines
Signed-off-by Hannes Reinecke <[email protected]>Signed-off-by Daniel Wagner <[email protected]>---
same here, do we need a same fix for FC as well ? just curious ...
Looks good.
-ck
Re: [PATCH v3 2/2] nvme-rdma: short-circuit reconnect retries
Quoted 14 lines · Chaitanya Kulkarni
Quoted 9 lines · Daniel Wagner
From: Hannes Reinecke <[email protected]>
Returning an nvme status from
nvme_rdma_setup_ctrl()indicates that the association was established and we have received a status from the controller; consequently we should honour the DNR bit. If not any future reconnect attempts will just return the same error, so we can short-circuit the reconnect attempts and fail the connection directly.Quoted 4 lines
Signed-off-by Hannes Reinecke <[email protected]>Signed-off-by Daniel Wagner <[email protected]>---
Quoted 8 lines
same here, do we need a same fix for FC as well ? just curious ...
Looks good.
Reviewed-by Chaitanya Kulkarni <[email protected]>-ck
fc already handles DNR in nvme_fc_reconnect_or_delete().
-ck
Sagi Grimberg
maintainerQuoted 4 lines · Daniel Wagner
I've picked up Hannes' DNR patches. In short the make the transports behave the same way when the DNR bit set on a re-connect attempt. We had a discussion this topic in the past and if I got this right we all agreed is that the host should honor the DNR bit on a connect attempt [1]
Umm, I don't recall this being conclusive though. The spec ought to be clearer here I think.
Quoted 4 lines
The nvme/045 test case (authentication tests) in blktests is a good test case for this after extending it slightly. TCP and RDMA try to reconnect with an invalid key over and over again, while loop and FC stop after the first fail.
Who says that invalid key is a permanent failure though?
Quoted 6 lines · Sagi Grimberg
Quoted 3 lines · Daniel Wagner
I've picked up Hannes' DNR patches. In short the make the transports behave the same way when the DNR bit set on a re-connect attempt. We had a discussion this
topic in the past and if I got this right we all agreed is that the host should honor the DNR bit on a connect attempt [1]Umm, I don't recall this being conclusive though. The spec ought to be clearer here I think.
I've asked the NVMexpress fmds group, and the response was pretty unanimous that the DNR bit on connect should be evaluated.
Quoted 7 lines
Quoted 4 lines
The nvme/045 test case (authentication tests) in blktests is a good test case for this after extending it slightly. TCP and RDMA try to reconnect with an
invalid key over and over again, while loop and FC stop after the first fail.Who says that invalid key is a permanent failure though?
See the response to the other patchset.
'Invalid key' in this context means that the _client_ evaluated the key as invalid, ie the key is unusable for the client. As the key is passed in via the commandline there is no way the client
can ever change the value here, and no amount of retry will change things here. That's what we try to fix.
The controller surely can return an invalid key, but that's part of the authenticaion protocol and will be evaluated differently.
Cheers,
Hannes
Sagi Grimberg
maintainerQuoted 9 lines · Hannes Reinecke
Quoted 6 lines · Sagi Grimberg
Quoted 3 lines · Daniel Wagner
I've picked up Hannes' DNR patches. In short the make the transports behave the same way when the DNR bit set on a re-connect attempt. We had a discussion this
topic in the past and if I got this right we all agreed is that the host should honor the DNR bit on a connect attempt [1]Umm, I don't recall this being conclusive though. The spec ought to be clearer here I think.
I've asked the NVMexpress fmds group, and the response was pretty unanimous that the DNR bit on connect should be evaluated.
OK.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
The nvme/045 test case (authentication tests) in blktests is a good test case for this after extending it slightly. TCP and RDMA try to reconnect with an
invalid key over and over again, while loop and FC stop after the first fail.
Quoted 6 lines
Quoted 2 lines
Who says that invalid key is a permanent failure though?
See the response to the other patchset.
'Invalid key' in this context means that the _client_ evaluated the key as invalid, ie the key is unusable for the client. As the key is passed in via the commandline there is no way the client
can ever change the value here, and no amount of retry will change things here. That's what we try to fix.
Where is this retried today, I don't see where connect failure is retried, outside of a periodic reconnect. Maybe I'm missing where what is the actual failure here.
Quoted 23 lines · Sagi Grimberg
Quoted 9 lines · Hannes Reinecke
Quoted 6 lines · Sagi Grimberg
Quoted 3 lines · Daniel Wagner
I've picked up Hannes' DNR patches. In short the make the transports behave the same way when the DNR bit set on a re-connect attempt. We had a discussion this
topic in the past and if I got this right we all agreed is that the host should honor the DNR bit on a connect attempt [1]Umm, I don't recall this being conclusive though. The spec ought to be clearer here I think.
I've asked the NVMexpress fmds group, and the response was pretty unanimous that the DNR bit on connect should be evaluated.
OK.
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
The nvme/045 test case (authentication tests) in blktests is a good test case for this after extending it slightly. TCP and RDMA try to reconnect with an
invalid key over and over again, while loop and FC stop after the first fail.Who says that invalid key is a permanent failure though?
Quoted 7 lines
Quoted 4 lines
See the response to the other patchset.
'Invalid key' in this context means that the _client_ evaluated the key as invalid, ie the key is unusable for the client. As the key is passed in via the commandline there is no way the client
can ever change the value here, and no amount of retry will change things here. That's what we try to fix.Where is this retried today, I don't see where connect failure is retried, outside of a periodic reconnect. Maybe I'm missing where what is the actual failure here.
static void nvme_tcp_reconnect_ctrl_work(struct work_struct *work)
{
struct nvme_tcp_ctrl *tcp_ctrl =
container_of(to_delayed_work(work),
struct nvme_tcp_ctrl, connect_work);
struct nvme_ctrl *ctrl = &tcp_ctrl->ctrl;
++ctrl->nr_reconnects;
if (nvme_tcp_setup_ctrl(ctrl, false))
goto requeue;
dev_info(ctrl->device, "Successfully reconnected (%d attempt)\n",
ctrl->nr_reconnects);
ctrl->nr_reconnects = 0;
return;requeue:
dev_info(ctrl->device, "Failed reconnect attempt %d\n",
and nvme_tcp_setup_ctrl() returns either a negative errno or an NVMe status code (which might include the DNR bit).
Cheers,
Hannes
Sagi Grimberg
maintainerQuoted 47 lines · Hannes Reinecke
Quoted 23 lines · Sagi Grimberg
Quoted 9 lines · Hannes Reinecke
Quoted 6 lines · Sagi Grimberg
Quoted 3 lines · Daniel Wagner
I've picked up Hannes' DNR patches. In short the make the transports behave the same way when the DNR bit set on a re-connect attempt. We had a discussion this
topic in the past and if I got this right we all agreed is that the host should honor the DNR bit on a connect attempt [1]Umm, I don't recall this being conclusive though. The spec ought to be clearer here I think.
I've asked the NVMexpress fmds group, and the response was pretty unanimous that the DNR bit on connect should be evaluated.
OK.
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
The nvme/045 test case (authentication tests) in blktests is a good test case for this after extending it slightly. TCP and RDMA try to reconnect with an
invalid key over and over again, while loop and FC stop after the first fail.Who says that invalid key is a permanent failure though?
Quoted 7 lines
Quoted 4 lines
See the response to the other patchset.
'Invalid key' in this context means that the _client_ evaluated the key as invalid, ie the key is unusable for the client. As the key is passed in via the commandline there is no way the client
can ever change the value here, and no amount of retry will change things here. That's what we try to fix.Where is this retried today, I don't see where connect failure is retried, outside of a periodic reconnect. Maybe I'm missing where what is the actual failure here.
static void
nvme_tcp_reconnect_ctrl_work(struct work_struct *work){
structnvme_tcp_ctrl*tcp_ctrl=container_of(to_delayed_work(work), struct nvme_tcp_ctrl, connect_work); structnvme_ctrl*ctrl =&tcp_ctrl->ctrl;++ctrl->nr_reconnects;
if (
nvme_tcp_setup_ctrl(ctrl, false))
goto requeue;
dev_info(ctrl->device, "Successfully reconnected (%d attempt)\n", ctrl->nr_reconnects);
Quoted 8 lines
ctrl->nr_reconnects= 0;return;
requeue:
dev_info(ctrl->device, "Failed reconnect attempt %d\n",and
nvme_tcp_setup_ctrl()returns either a negative errno or an NVMe status code (which might include the DNR bit).
I thought this is about the initialization. yes today we ignore the status in re-connection assuming that whatever happened, may (or may not) resolve itself. The basis for this assumption is that if we managed to connect the first time there is no reason to assume that connecting again should fail persistently.
If there is a consensus that we should not assume it, its a valid argument. I didn't see where this happens with respect to authentication though.
Quoted 5 lines · Sagi Grimberg
Quoted 2 lines · Hannes Reinecke
Quoted 1 line · Sagi Grimberg
[ .. ]
Quoted 23 lines
Quoted 23 lines
Quoted 3 lines
Where is this retried today, I don't see where connect failure is retried, outside of a periodic reconnect. Maybe I'm missing where what is the actual failure here.
static void
nvme_tcp_reconnect_ctrl_work(struct work_struct *work){
structnvme_tcp_ctrl*tcp_ctrl=container_of(to_delayed_work(work), struct nvme_tcp_ctrl, connect_work); structnvme_ctrl*ctrl =&tcp_ctrl->ctrl;++ctrl->nr_reconnects;
if (
nvme_tcp_setup_ctrl(ctrl, false))
goto requeue;
dev_info(ctrl->device, "Successfully reconnected (%d attempt)\n", ctrl->nr_reconnects);
ctrl->nr_reconnects= 0;return;
Quoted 9 lines
Quoted 4 lines
requeue:
dev_info(ctrl->device, "Failed reconnect attempt %d\n",and
nvme_tcp_setup_ctrl()returns either a negative errno or an NVMe status code (which might include the DNR bit).I thought this is about the initialization. yes today we ignore the status in re-connection assuming that whatever happened, may (or may not) resolve itself. The basis for this assumption is that if we managed to connect the first time there is no reason to assume that connecting again should fail persistently.
And that is another issue where I'm not really comfortable with.
While it would make sense to have the connect functionality to be
one-shot, and let userspace retry if needed, the problem is that we
don't have a means of transporting that information to userspace.
The only thing which we can transport is an error number, which
could be anything and mean anything.
If we had a defined way stating: 'This is a retryable, retry with the same options.' vs 'This is retryable error, retry with modified options.' vs 'This a non-retryable error, don't bother.' I'd be fine with delegating retries to userspace.
But currently we don't.
Quoted 2 lines
If there is a consensus that we should not assume it, its a valid argument. I didn't see where this happens with respect to authentication though.
nvmf_connect_admin_queue():
/* Authentication required */
ret = nvme_auth_negotiate(ctrl, 0);
if (ret) {
dev_warn(ctrl->device,
"qid 0: authentication setup failed\n");
ret = NVME_SC_AUTH_REQUIRED;
goto out_free_data;
}
ret = nvme_auth_wait(ctrl, 0);
if (ret)
dev_warn(ctrl->device,
"qid 0: authentication failed\n");
else
dev_info(ctrl->device,
"qid 0: authenticated\n");The first call to 'nvme_auth_negotiate()' is just for setting up the negotiation context and start the protocol. So if we get an error here it's pretty much non-retryable as it's completely controlled by the fabrics options.nvme_auth_wait(), OTOH, contains the actual result from the negotiation, so there we might or might not retry, depending on the value of 'ret'.
Cheers,
Hannes
Sagi Grimberg
maintainerQuoted 39 lines · Hannes Reinecke
Quoted 5 lines · Sagi Grimberg
Quoted 2 lines · Hannes Reinecke
Quoted 1 line · Sagi Grimberg
[ .. ]
Quoted 23 lines
Quoted 23 lines
Quoted 3 lines
Where is this retried today, I don't see where connect failure is retried, outside of a periodic reconnect. Maybe I'm missing where what is the actual failure here.
static void
nvme_tcp_reconnect_ctrl_work(struct work_struct *work){
structnvme_tcp_ctrl*tcp_ctrl=container_of(to_delayed_work(work), struct nvme_tcp_ctrl, connect_work); structnvme_ctrl*ctrl =&tcp_ctrl->ctrl;++ctrl->nr_reconnects;
if (
nvme_tcp_setup_ctrl(ctrl, false))
goto requeue;
dev_info(ctrl->device, "Successfully reconnected (%d attempt)\n", ctrl->nr_reconnects);
ctrl->nr_reconnects= 0;return;
Quoted 9 lines
Quoted 4 lines
requeue:
dev_info(ctrl->device, "Failed reconnect attempt %d\n",and
nvme_tcp_setup_ctrl()returns either a negative errno or an NVMe status code (which might include the DNR bit).I thought this is about the initialization. yes today we ignore the status in re-connection assuming that whatever happened, may (or may not) resolve itself. The basis for this assumption is that if we managed to connect the first time there is no reason to assume that connecting again should fail persistently.
Quoted 6 lines
And that is another issue where I'm not really comfortable with. While it would make sense to have the connect functionality to be one-shot, and let userspace retry if needed, the problem is that we don't have a means of transporting that information to userspace. The only thing which we can transport is an error number, which could be anything and mean anything.
Not necessarily. error codes semantics exists for a reason.
I just really don't think that doing reconnects on a user-driven initialization is a good idea at all unlike the case where the controller was connected and got disrupted, this is not user driven and hence makes sense.
Quoted 3 lines
If we had a defined way stating: 'This is a retryable, retry with the same options.' vs 'This is retryable error, retry with modified options.' vs 'This a non-retryable error, don't bother.' I'd be fine with delegating retries to userspace.
But currently we don't.
Well, TBH I don't know if userspace even needs it. Most likely what a user would want is to define a number of retries and give up if they expire. Adding the intelligence for what connect is retry-able or not does not seem all that useful to me.
Quoted 33 lines
Quoted 3 lines
If there is a consensus that we should not assume it, its a valid argument. I didn't see where this happens with respect to authentication though.
nvmf_connect_admin_queue():/* Authentication required */
ret =nvme_auth_negotiate(ctrl, 0);
if (ret) {
dev_warn(ctrl->device,
"qid 0: authentication setup failed\n"); ret =NVME_SC_AUTH_REQUIRED;
gotoout_free_data;
}
ret =nvme_auth_wait(ctrl, 0);
if (ret)
dev_warn(ctrl->device,
"qid 0: authentication failed\n"); else
dev_info(ctrl->device,
"qid 0: authenticated\n");The first call to '
nvme_auth_negotiate()' is just for setting up the negotiation context and start the protocol. So if we get an error here it's pretty much non-retryable as it's completely controlled by the fabrics options.nvme_auth_wait(), OTOH, contains the actual result from the negotiation, so there we might or might not retry, depending on the value of 'ret'.Cheers,
Hannes