nvme-multipath: fixes for non-optimized paths
cover nvme-multipath: fixes for non-optimized paths
Hi all,
one of our partners informed us that the round-robin path selector didn't do the right thing when only one optimized path (or, indeed, none at all) are present.
These patches will fixup nvme_round_robin() to handle the case when less than two optimized (but several non-optimized) paths are present.
These pathes are a rework of the earlier patches send by Martin Wilck.
As usual, comments and reviews are welcome.
Hannes Reinecke (1):
nvme-multipath: do not fall back to __nvme_find_path() for
non-optimized paths
Martin Wilck (1):
nvme-multipath: fix logic for non-optimized paths
drivers/nvme/host/multipath.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
Signature · 7 lines
2.16.4 _______________________________________________ Linux-nvme mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-nvme
PATCH 1/2 nvme-multipath: fix logic for non-optimized paths
From: Martin Wilck <[email protected]>
Handle the special case where we have exactly one optimized path, which we should keep using in this case.
Signed off-by: Martin Wilck <[email protected]>
--- drivers/nvme/host/multipath.c | 6 ++++++ 1 file changed, 6 insertions(+)
drivers/nvme/host/multipath.c +6 -0
@@ -246,6 +246,12 @@ static struct nvme_ns *nvme_round_robin_path(struct nvme_ns_head *head, fallback = ns; } +/* No optimized path found, re-check the current path */ +if (!nvme_path_is_disabled (old) && + old->ana_state ==NVME_ANA_OPTIMIZED ) { + found = old; + goto out; + }if (!fallback)return NULL ; found = fallback;
Signature · 7 lines
2.16.4 _______________________________________________ Linux-nvme mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-nvme
Re: [PATCH 1/2] nvme-multipath: fix logic for non-optimized paths
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
PATCH 2/2 nvme-multipath: do not fall back to __nvme_find_path() for non-optimized paths
In reply to Hannes ReineckeWhen nvme_round_robin_path() finds a valid namespace we should be using it; falling back to __nvme_find_path() for non-optimized paths will cause the result from nvme_round_robin_path() to be ignored for non-optimized paths.
--- drivers/nvme/host/multipath.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/nvme/host/multipath.c +4 -1
@@ -272,8 +272,11 @@ inline struct nvme_ns *nvme_find_path(struct nvme_ns_head *head)struct nvme_ns *ns; ns =srcu_dereference (head->current_path[node], &head->srcu); -if (READ_ONCE (head->subsys->iopolicy) ==NVME_IOPOLICY_RR && ns) +if (READ_ONCE (head->subsys->iopolicy) ==NVME_IOPOLICY_RR && ns) { ns =nvme_round_robin_path (head, node, ns); +if (ns) +return ns; + }if (unlikely (!ns || !nvme_path_is_optimized (ns))) ns =__nvme_find_path (head, node);return ns;
Signature · 7 lines
2.16.4 _______________________________________________ Linux-nvme mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-nvme
Sagi Grimberg
maintainerRe: [PATCH 2/2] nvme-multipath: do not fall back to __nvme_find_path() for non-optimized paths
Quoted 16 lines · Hannes Reinecke
When
nvme_round_robin_path()finds a valid namespace we should be using it; falling back to__nvme_find_path()for non-optimized paths will cause the result fromnvme_round_robin_path()to be ignored for non-optimized paths.Signed-off-by Martin Wilck <[email protected]>Signed-off-by Hannes Reinecke <[email protected]>--- drivers/nvme/host/multipath.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)drivers/nvme/host/multipath.c +0
-0@@ -272,8 +272,11 @@ inline struct nvme_ns *nvme_find_path(struct nvme_ns_head *head)struct nvme_ns *ns;
drivers/nvme/host/multipath.c inline struct nvme_ns *nvme_find_path(struct nvme_ns_head *head) · line 281
Quoted 10 lines
drivers/nvme/host/multipath.c +4
-1ns =srcu_dereference (head->current_path[node], &head->srcu); -if (READ_ONCE (head->subsys->iopolicy) ==NVME_IOPOLICY_RR && ns) +if (READ_ONCE (head->subsys->iopolicy) ==NVME_IOPOLICY_RR && ns) { ns =nvme_round_robin_path (head, node, ns); +if (ns) +return ns; + }if (unlikely (!ns || !nvme_path_is_optimized (ns))) ns =__nvme_find_path (head, node);
Maybe instead of the early return put the following condition on else?
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Re: [PATCH 2/2] nvme-multipath: do not fall back to __nvme_find_path() for non-optimized paths
Quoted 23 lines · Sagi Grimberg
Quoted 20 lines · Hannes Reinecke
When
nvme_round_robin_path()finds a valid namespace we should be using it; falling back to__nvme_find_path()for non-optimized paths will cause the result fromnvme_round_robin_path()to be ignored for non-optimized paths.Signed-off-by Martin Wilck <[email protected]>Signed-off-by Hannes Reinecke <[email protected]>---
drivers/nvme/host/multipath.c| 5 ++++-1 file changed, 4 insertions(+), 1 deletion(-)drivers/nvme/host/multipath.c +1
-1@@ -272,8 +272,11 @@ inline struct nvme_ns *nvme_find_path(struct nvme_ns_head *head) struct nvme_ns *ns; ns = srcu_dereference(head->current_path[node], &head->srcu); - if (READ_ONCE(head->subsys->iopolicy) == NVME_IOPOLICY_RR && ns) + if (READ_ONCE(head->subsys->iopolicy) == NVME_IOPOLICY_RR && ns) {
Quoted 8 lines
drivers/nvme/host/multipath.c inline struct nvme_ns *nvme_find_path(struct nvme_ns_head *head) · line 281
Quoted 6 lines
drivers/nvme/host/multipath.c +3
-0ns = nvme_round_robin_path(head, node, ns); + if (ns) + return ns; + } if (unlikely(!ns || !nvme_path_is_optimized(ns))) ns = __nvme_find_path(head, node);Maybe instead of the early return put the following condition on else?
That depends on whether we want to have a fallback to __nvme_find_path() or not. With your suggestion we would lose that; I'd rather keep it.
Cheers,
Hannes
Signature · 9 lines
Dr. Hannes Reinecke Teamlead Storage & Networking [email protected] +49 911 74053 688 SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer _______________________________________________ Linux-nvme mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-nvme
Re: [PATCH 2/2] nvme-multipath: do not fall back to __nvme_find_path() for non-optimized paths
Quoted 30 lines
Quoted 21 lines
Quoted 21 lines
When
nvme_round_robin_path()finds a valid namespace we should be using it; falling back to__nvme_find_path()for non-optimized paths will cause the result fromnvme_round_robin_path()to be ignored for non-optimized paths.Signed-off-by Martin Wilck <[email protected]>Signed-off-by Hannes Reinecke <[email protected]>---
drivers/nvme/host/multipath.c| 5 ++++-1 file changed, 4 insertions(+), 1 deletion(-)drivers/nvme/host/multipath.c +1
-1@@ -272,8 +272,11 @@ inline struct nvme_ns *nvme_find_path(struct nvme_ns_head *head) struct nvme_ns *ns; ns = srcu_dereference(head->current_path[node], &head->srcu); - if (READ_ONCE(head->subsys->iopolicy) == NVME_IOPOLICY_RR && ns) + if (READ_ONCE(head->subsys->iopolicy) == NVME_IOPOLICY_RR && ns) {Quoted 8 lines
drivers/nvme/host/multipath.c inline struct nvme_ns *nvme_find_path(struct nvme_ns_head *head) · line 281
Quoted 6 lines
drivers/nvme/host/multipath.c +3
-0ns = nvme_round_robin_path(head, node, ns); + if (ns) + return ns; + } if (unlikely(!ns || !nvme_path_is_optimized(ns))) ns = __nvme_find_path(head, node);Maybe instead of the early return put the following condition on else?
Quoted 1 line
That depends on whether we want to have a fallback to
__nvme_find_path()or not. With your suggestion we would lose that; I'd rather keep it.
OK...
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Re: [PATCH 2/2] nvme-multipath: do not fall back to __nvme_find_path() for non-optimized paths
In reply to Hannes ReineckeQuoted 4 lines · Hannes Reinecke
Quoted 1 line
Maybe instead of the early return put the following condition on else?
That depends on whether we want to have a fallback to
__nvme_find_path()or not. With your suggestion we would lose that; I'd rather keep it.
Why would we want to fall back? nvme_round_robin_path only returns NULL in case there is a single disabled path, in which case __nvme_find_path won't find anything either. I'd go for something like this:
---
From 4f578364a3d15898c7d715315a3371b2f71db416 Mon Sep 17 00:00:00 2001
From: Hannes Reinecke <[email protected]>
Date: Mon, 27 Jul 2020 18:08:03 +0200
Subject: nvme-multipath: do not fall back to __nvme_find_path() for
non-optimized paths
When nvme_round_robin_path() finds a valid namespace we should be using it; falling back to __nvme_find_path() for non-optimized paths will cause the result from nvme_round_robin_path() to be ignored for non-optimized paths.
--- drivers/nvme/host/multipath.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
drivers/nvme/host/multipath.c +7 -4
@@ -281,10 +281,13 @@ inline struct nvme_ns *nvme_find_path(struct nvme_ns_head *head)struct nvme_ns *ns; ns =srcu_dereference (head->current_path[node], &head->srcu); -if (READ_ONCE (head->subsys->iopolicy) ==NVME_IOPOLICY_RR && ns) - ns =nvme_round_robin_path (head, node, ns); -if (unlikely (!ns || !nvme_path_is_optimized (ns))) - ns =__nvme_find_path (head, node); +if (unlikely (!ns)) +return __nvme_find_path (head, node); + +if (READ_ONCE (head->subsys->iopolicy) ==NVME_IOPOLICY_RR ) +return nvme_round_robin_path (head, node, ns); +if (unlikely (!nvme_path_is_optimized (ns))) +return __nvme_find_path (head, node);return ns; }
Signature · 7 lines
2.27.0 _______________________________________________ Linux-nvme mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-nvme
Re: [PATCH 2/2] nvme-multipath: do not fall back to __nvme_find_path() for non-optimized paths
Quoted 43 lines · Christoph Hellwig
Quoted 4 lines · Hannes Reinecke
Quoted 1 line
Maybe instead of the early return put the following condition on else?
That depends on whether we want to have a fallback to
__nvme_find_path()or not. With your suggestion we would lose that; I'd rather keep it.Why would we want to fall back?
nvme_round_robin_pathonly returns NULL in case there is a single disabled path, in which case__nvme_find_pathwon't find anything either. I'd go for something like this:---
From4f578364a3d15898c7d715315a3371b2f71db416Mon Sep 17 00:00:00 2001
From: Hannes Reinecke <[email protected]>
Date: Mon, 27 Jul 2020 18:08:03 +0200
Subject: nvme-multipath: do not fall back to__nvme_find_path()for
non-optimized pathsWhen
nvme_round_robin_path()finds a valid namespace we should be using it; falling back to__nvme_find_path()for non-optimized paths will cause the result fromnvme_round_robin_path()to be ignored for non-optimized paths.Signed-off-by Martin Wilck <[email protected]>Signed-off-by Hannes Reinecke <[email protected]>Reviewed-by Sagi Grimberg <[email protected]>Signed-off-by Christoph Hellwig <[email protected]>--- drivers/nvme/host/multipath.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)drivers/nvme/host/multipath.c +0
-4@@ -281,10 +281,13 @@ inline struct nvme_ns *nvme_find_path(struct nvme_ns_head *head)struct nvme_ns *ns; ns =srcu_dereference (head->current_path[node], &head->srcu); -if (READ_ONCE (head->subsys->iopolicy) ==NVME_IOPOLICY_RR && ns) - ns =nvme_round_robin_path (head, node, ns); -if (unlikely (!ns || !nvme_path_is_optimized (ns))) - ns =__nvme_find_path (head, node);
drivers/nvme/host/multipath.c inline struct nvme_ns *nvme_find_path(struct nvme_ns_head *head) · line 293
Quoted 10 lines
drivers/nvme/host/multipath.c +7
-0+if (unlikely (!ns)) +return __nvme_find_path (head, node); + +if (READ_ONCE (head->subsys->iopolicy) ==NVME_IOPOLICY_RR ) +return nvme_round_robin_path (head, node, ns); +if (unlikely (!nvme_path_is_optimized (ns))) +return __nvme_find_path (head, node);return ns; }
Yes, that would work, too.
Should I resend?
Cheers,
Hannes
Signature · 9 lines
Dr. Hannes Reinecke Kernel Storage Architect [email protected] +49 911 74053 688 SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg HRB 36809 (AG Nürnberg), GF: Felix Imendörffer _______________________________________________ Linux-nvme mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-nvme
Christoph Hellwig
maintainerRe: [PATCH 2/2] nvme-multipath: do not fall back to __nvme_find_path() for non-optimized paths
Quoted 57 lines · Hannes Reinecke
Quoted 43 lines · Christoph Hellwig
Quoted 4 lines · Hannes Reinecke
Quoted 1 line
Maybe instead of the early return put the following condition on else?
That depends on whether we want to have a fallback to
__nvme_find_path()or not. With your suggestion we would lose that; I'd rather keep it.Why would we want to fall back?
nvme_round_robin_pathonly returns NULL in case there is a single disabled path, in which case__nvme_find_pathwon't find anything either. I'd go for something like this:---
From4f578364a3d15898c7d715315a3371b2f71db416Mon Sep 17 00:00:00 2001
From: Hannes Reinecke <[email protected]>
Date: Mon, 27 Jul 2020 18:08:03 +0200
Subject: nvme-multipath: do not fall back to__nvme_find_path()for
non-optimized pathsWhen
nvme_round_robin_path()finds a valid namespace we should be using it; falling back to__nvme_find_path()for non-optimized paths will cause the result fromnvme_round_robin_path()to be ignored for non-optimized paths.Signed-off-by Martin Wilck <[email protected]>Signed-off-by Hannes Reinecke <[email protected]>Reviewed-by Sagi Grimberg <[email protected]>Signed-off-by Christoph Hellwig <[email protected]>--- drivers/nvme/host/multipath.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)drivers/nvme/host/multipath.c +0
-4@@ -281,10 +281,13 @@ inline struct nvme_ns *nvme_find_path(struct nvme_ns_head *head)struct nvme_ns *ns; ns =srcu_dereference (head->current_path[node], &head->srcu); -if (READ_ONCE (head->subsys->iopolicy) ==NVME_IOPOLICY_RR && ns) - ns =nvme_round_robin_path (head, node, ns); -if (unlikely (!ns || !nvme_path_is_optimized (ns))) - ns =__nvme_find_path (head, node);drivers/nvme/host/multipath.c inline struct nvme_ns *nvme_find_path(struct nvme_ns_head *head) · line 293
Quoted 10 lines
drivers/nvme/host/multipath.c +7
-0+if (unlikely (!ns)) +return __nvme_find_path (head, node); + +if (READ_ONCE (head->subsys->iopolicy) ==NVME_IOPOLICY_RR ) +return nvme_round_robin_path (head, node, ns); +if (unlikely (!nvme_path_is_optimized (ns))) +return __nvme_find_path (head, node);return ns; }Yes, that would work, too.
Quoted 1 line
Should I resend?
No need, I can apply it with the modification. But if you have a
RR test setup it would be great to run it through that again, just in case.
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Re: [PATCH 2/2] nvme-multipath: do not fall back to __nvme_find_path() for non-optimized paths
Quoted 58 lines · Christoph Hellwig
Quoted 57 lines · Hannes Reinecke
Quoted 43 lines · Christoph Hellwig
Quoted 4 lines · Hannes Reinecke
Quoted 1 line
Maybe instead of the early return put the following condition on else?
That depends on whether we want to have a fallback to
__nvme_find_path()or not. With your suggestion we would lose that; I'd rather keep it.Why would we want to fall back?
nvme_round_robin_pathonly returns NULL in case there is a single disabled path, in which case__nvme_find_pathwon't find anything either. I'd go for something like this:---
From4f578364a3d15898c7d715315a3371b2f71db416Mon Sep 17 00:00:00 2001
From: Hannes Reinecke <[email protected]>
Date: Mon, 27 Jul 2020 18:08:03 +0200
Subject: nvme-multipath: do not fall back to__nvme_find_path()for
non-optimized pathsWhen
nvme_round_robin_path()finds a valid namespace we should be using it; falling back to__nvme_find_path()for non-optimized paths will cause the result fromnvme_round_robin_path()to be ignored for non-optimized paths.Signed-off-by Martin Wilck <[email protected]>Signed-off-by Hannes Reinecke <[email protected]>Reviewed-by Sagi Grimberg <[email protected]>Signed-off-by Christoph Hellwig <[email protected]>--- drivers/nvme/host/multipath.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)drivers/nvme/host/multipath.c +0
-4@@ -281,10 +281,13 @@ inline struct nvme_ns *nvme_find_path(struct nvme_ns_head *head)struct nvme_ns *ns; ns =srcu_dereference (head->current_path[node], &head->srcu); -if (READ_ONCE (head->subsys->iopolicy) ==NVME_IOPOLICY_RR && ns) - ns =nvme_round_robin_path (head, node, ns); -if (unlikely (!ns || !nvme_path_is_optimized (ns))) - ns =__nvme_find_path (head, node);drivers/nvme/host/multipath.c inline struct nvme_ns *nvme_find_path(struct nvme_ns_head *head) · line 293
Quoted 10 lines
drivers/nvme/host/multipath.c +7
-0+if (unlikely (!ns)) +return __nvme_find_path (head, node); + +if (READ_ONCE (head->subsys->iopolicy) ==NVME_IOPOLICY_RR ) +return nvme_round_robin_path (head, node, ns); +if (unlikely (!nvme_path_is_optimized (ns))) +return __nvme_find_path (head, node);return ns; }Yes, that would work, too.
Quoted 6 lines
Quoted 1 line
Should I resend?
No need, I can apply it with the modification. But if you have a
RR test setup it would be great to run it through that again, just in case.
I'm at it.
Cheers,
Hannes
Signature · 9 lines
Dr. Hannes Reinecke Kernel Storage Architect [email protected] +49 911 74053 688 SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg HRB 36809 (AG Nürnberg), GF: Felix Imendörffer _______________________________________________ Linux-nvme mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-nvme
Applied to nvme-5.9 with the tweak discussed earlier.
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme