Skip to contentSkip to conversation

nvme-multipath: sanitize nvme_update_ana_state()

PATCH New Hannes Reinecke 3 participants 6 messages Jul 16, 2019 – Jul 19, 2019 NVMe

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 nvme-multipath: sanitize nvme_update_ana_state()

Commit 04e70bd4a026 ("nvme-multipath: do not select namespaces which are about to be removed") introduced checks when traversing the list of namespaces to avoid tripping over invalid namespaces. A similar check is needed in nvme_update_ana_state() to skip updates for any namespaces which will be removed.
With that we should also remove the WARN_ON() at the end of the iteration; this will also be errorneously triggered if ANA Change AENS are received during scanning.

Signed-off-by Hannes Reinecke <hare at suse.com>
---
 drivers/nvme/host/multipath.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/nvme/host/multipath.c +2 -1
index a9a927677970..78415f2909cd 100644
@@ -449,6 +449,8 @@ static int nvme_update_ana_state(struct nvme_ctrl *ctrl,
 
 	down_write(&ctrl->namespaces_rwsem);
 	list_for_each_entry(ns, &ctrl->namespaces, list) {
+		if (test_bit(NVME_NS_REMOVING, &ns->flags))
+			continue;
 		if (ns->head->ns_id != le32_to_cpu(desc->nsids[n]))
 			continue;
 		nvme_update_ns_ana_state(desc, ns);
@@ -456,7 +458,6 @@ static int nvme_update_ana_state(struct nvme_ctrl *ctrl,
 			break;
 	}
 	up_write(&ctrl->namespaces_rwsem);
-	WARN_ON_ONCE(n < nr_nsids);
 	return 0;
 }
 
Signature · 1 line
2.16.4
#1 lore raw

PATCH 1/1 nvme-multipath: sanitize nvme_update_ana_state()

Quoted 8 lines · Hannes Reinecke

Commit 04e70bd4a026 ("nvme-multipath: do not select namespaces which are about to be removed") introduced checks when traversing the list of namespaces to avoid tripping over invalid namespaces. A similar check is needed in nvme_update_ana_state() to skip updates for any namespaces which will be removed.
With that we should also remove the WARN_ON() at the end of the iteration; this will also be errorneously triggered if ANA Change AENS are received during scanning.

What is the problem we are trying to fix here? Just that we should not call nvme_mpath_set_live? In that case it might make more sense to move the removed check to just guard that call and at least keep the values in struct nvme_ns uptodate.

#2 lore raw

PATCH 1/1 nvme-multipath: sanitize nvme_update_ana_state()

Quoted 10 lines · Christoph Hellwig
Quoted 8 lines · Hannes Reinecke

Commit 04e70bd4a026 ("nvme-multipath: do not select namespaces which are about to be removed") introduced checks when traversing the list of namespaces to avoid tripping over invalid namespaces. A similar check is needed in nvme_update_ana_state() to skip updates for any namespaces which will be removed.
With that we should also remove the WARN_ON() at the end of the iteration; this will also be errorneously triggered if ANA Change AENS are received during scanning.

Quoted 5 lines

What is the problem we are trying to fix here? Just that we should not call nvme_mpath_set_live? In that case it might make more sense to move the removed check to just guard that call and at least keep the values in struct nvme_ns uptodate.

The underlying problem is the 'WARN_ON()', which actually can (and is) triggered when ANA change AENs are received during controller (re-) connect. At the same time it's quite pointless to update the ANA state for namespaces which are on their way out; we _still_ seeing odd behaviour during simultaneous rescan/reset, and this is just another safeguard.

But for now I'd be happy to drop the check for NS_REMOVING and just delete the WARN_ON().

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG N?rnberg)
#3 lore raw

PATCH 1/1 nvme-multipath: sanitize nvme_update_ana_state()

Quoted 8 lines · Hannes Reinecke

The underlying problem is the 'WARN_ON()', which actually can (and is) triggered when ANA change AENs are received during controller (re-) connect. At the same time it's quite pointless to update the ANA state for namespaces which are on their way out; we _still_ seeing odd behaviour during simultaneous rescan/reset, and this is just another safeguard.

But for now I'd be happy to drop the check for NS_REMOVING and just delete the WARN_ON().

Well, we should certainly skip the nvme_mpath_set_live as well for a removing namespace. But yes, I'd prefer to move the check there and then remove the WARN_ON over the big hammer.

#4 lore raw

PATCH 1/1 nvme-multipath: sanitize nvme_update_ana_state()

Quoted 11 lines · Christoph Hellwig
Quoted 6 lines · Hannes Reinecke

The underlying problem is the 'WARN_ON()', which actually can (and is) triggered when ANA change AENs are received during controller (re-) connect. At the same time it's quite pointless to update the ANA state for namespaces which are on their way out; we _still_ seeing odd behaviour during simultaneous rescan/reset, and this is just another safeguard.

Quoted 7 lines
Quoted 2 lines

But for now I'd be happy to drop the check for NS_REMOVING and just delete the WARN_ON().

Well, we should certainly skip the nvme_mpath_set_live as well for a removing namespace. But yes, I'd prefer to move the check there and then remove the WARN_ON over the big hammer.

Hi
I happen to be seeing this WARNING in a port we are using while doing failover testing . If the WARNING is removed , should there be another logging message of a similar nature that alerts of a configuration issue has been detected ? Or is this a transient condition ?

#5 lore raw

PATCH 1/1 nvme-multipath: sanitize nvme_update_ana_state()

Quoted 19 lines · John Donnelly
Quoted 11 lines · Christoph Hellwig
Quoted 6 lines · Hannes Reinecke

The underlying problem is the 'WARN_ON()', which actually can (and is) triggered when ANA change AENs are received during controller (re-) connect. At the same time it's quite pointless to update the ANA state for namespaces which are on their way out; we _still_ seeing odd behaviour during simultaneous rescan/reset, and this is just another safeguard.

Quoted 7 lines
Quoted 2 lines

But for now I'd be happy to drop the check for NS_REMOVING and just delete the WARN_ON().

Well, we should certainly skip the nvme_mpath_set_live as well for a removing namespace. But yes, I'd prefer to move the check there and then remove the WARN_ON over the big hammer.

Quoted 6 lines

Hi
I happen to be seeing this WARNING in a port we are using while doing
failover testing . If the WARNING is removed , should there be another logging message of a similar nature that alerts of a configuration issue has been detected ? Or is this a transient condition ?

This is typically a transient condition.
If we receive an ANA chane AEN while scanning is in progress the ANA log page will contain all namespaces, yet the scanning thread is still processing and hasn't connected all namespaces. Hence we see this error.

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG N?rnberg)