Skip to contentSkip to conversation

libmultipath: multipath active paths count optimization

PATCH In review Chongyun Wu 2 participants 3 messages Sep 5, 2019 Device mapper

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 libmultipath: multipath active paths count optimization

Hi Martin, Ben and other viewers

Cloud you help to view below patch which try to deal with a multipath active paths count not right issue, thanks a lot.

Quoted 1 line

From deee7196ece43b01b8ee635e60ce465080905b5e Mon Sep 17 00:00:00 2001

From: Chongyun Wu <[email protected]>
Date: Tue, 27 Aug 2019 13:58:33 +0800
Subject: [PATCH] libmultipath: multipath active paths count optimization

Really count multipath active paths not use mpp->nr_active++ or mpp->nr_active--, because there are other places might call pathinfo to change path state not only in check_path, if other places detect and changed path state but not do mpp->nr_active++ or mpp->nr_active--, the active paths might not right any more.

We meet an issue which actually have three paths but after all paths down syslog report have three paths remaining so multipathd not send disable queueing to dm and this dm device been blocked. This patch might fix this issue.

---
 libmultipath/structs_vec.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
libmultipath/structs_vec.c +6 -2
index c43b58f..7e2441b 100644
@@ -450,7 +450,8 @@ int verify_paths(struct multipath *mpp, struct vectors *vecs)
  */
 void update_queue_mode_del_path(struct multipath *mpp)
 {
-	if (--mpp->nr_active == 0) {
+	mpp->nr_active = pathcount(mpp, PATH_UP) + pathcount(mpp, PATH_GHOST);
+	if (mpp->nr_active == 0) {
 		if (mpp->no_path_retry > 0)
 			enter_recovery_mode(mpp);
 		else if (mpp->no_path_retry != NO_PATH_RETRY_QUEUE)
@@ -461,7 +462,10 @@ void update_queue_mode_del_path(struct multipath *mpp)
 
 void update_queue_mode_add_path(struct multipath *mpp)
 {
-	if (mpp->nr_active++ == 0 && mpp->no_path_retry > 0) {
+	int old_active_paths = mpp->nr_active;
+	mpp->nr_active = pathcount(mpp, PATH_UP) + pathcount(mpp, PATH_GHOST);
+	if ((old_active_paths == 0 && mpp->nr_active == 1)
+		&& mpp->no_path_retry > 0) {
 		/* come back to normal mode from retry mode */
 		mpp->retry_tick = 0;
 		dm_queue_if_no_path(mpp->alias, 1);
Signature · 8 lines
Best Regards,
Chongyun Wu


--
dm-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/dm-devel
#1 lore raw

Hi Chongyun,

Quoted 24 lines · Chongyun Wu

Hi Martin, Ben and other viewers

Cloud you help to view below patch which try to deal with a multipath active paths
count not right issue, thanks a lot.

From deee7196ece43b01b8ee635e60ce465080905b5e Mon Sep 17 00:00:00 2001
From: Chongyun Wu <[email protected]>
Date: Tue, 27 Aug 2019 13:58:33 +0800
Subject: [PATCH] libmultipath: multipath active paths count optimization

Really count multipath active paths not use mpp->nr_active++ or mpp->nr_active--, because there are other places might call pathinfo to change path state not only in check_path, if other places detect and changed path state but not do mpp->nr_active++ or mpp->nr_active--, the active paths might not right any more.

We meet an issue which actually have three paths but after all paths down syslog report have three paths remaining so multipathd not send disable queueing to dm and this dm device been blocked. This patch might fix this issue.

Quoted 4 lines
---
 libmultipath/structs_vec.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

Thanks a lot for your patch. We've discussed this previously, and in general, there's little reason not to do it - pathcount() is fast, and could be made even faster. But if we do, we should ditch the nr_active field altogether - no need to carry it around if we re-calculate it anyway when we need it.

However, it worries me a bit that nr_active may go wrong. Both Ben and I have reviewed the code and we thought the nr_active tracking was correct. Something seems to happen in our code that we don't understand.

- can you please confirm that you are using the latest code,
containing e224d57 "libmutipath: continue to use old state on PATH_PENDING", 9b715bf "multipathd: Fix miscounting active paths" and (in case you're using the marginal_paths options) also 7d4b40f and 058df77 ?

- If you have a reliable reproducer, would you mind adding log
messages to the code you just submitted, so that we can observe how nr_active evolves in time, and perhaps understand why it's going wrong?

Regards
Martin

#2 lore raw

Hi Martin,

Thanks a lot.
I think patch e224d57 "libmutipath: continue to use old state on PATH_PENDING" might fix our issue compare to our syslog output, I will cherry-pick it. And also need cherry-pick 9b715bf "multipathd: Fix miscounting active paths" for good.

Regards
Chongyun Wu

-----邮件原件-----
发件人: Martin Wilck [mailto:[email protected]]
发送时间: 2019年9月5日 15:37
收件人: [email protected]; [email protected]; wuchongyun (Cloud) <[email protected]>
抄送: guozhonghua (Cloud) <[email protected]>; likai (Cloud) <[email protected]>; renqinwei (Cloud) <[email protected]>; liliang (Cloud) <[email protected]>; liucong (Cloud) <[email protected]>; wangyong (Cloud) <[email protected]>; changlimin (Cloud) <[email protected]>; zhangduan (Cloud) <[email protected]>; huangyong (Cloud) <[email protected]>; wangxibo (Cloud) <[email protected]>; zhangguanghui (Cloud) <[email protected]>; zhangcheng (Cloud) <[email protected]> 主题: Re: [dm-devel] [PATCH] libmultipath: multipath active paths count optimization

Hi Chongyun,

Quoted 23 lines · Chongyun Wu

Hi Martin, Ben and other viewers

Cloud you help to view below patch which try to deal with a multipath active paths count not right issue, thanks a lot.

From deee7196ece43b01b8ee635e60ce465080905b5e Mon Sep 17 00:00:00 2001
From: Chongyun Wu <[email protected]>
Date: Tue, 27 Aug 2019 13:58:33 +0800
Subject: [PATCH] libmultipath: multipath active paths count optimization

Really count multipath active paths not use mpp->nr_active++ or mpp->nr_active--, because there are other places might call pathinfo to change path state not only in check_path, if other places detect and changed path state but not do mpp->nr_active++ or mpp->nr_active--, the active paths might not right any more.

We meet an issue which actually have three paths but after all paths down syslog report have three paths remaining so multipathd not send disable queueing to dm and this dm device been blocked. This patch might fix this issue.

Quoted 4 lines
---
 libmultipath/structs_vec.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

Thanks a lot for your patch. We've discussed this previously, and in general, there's little reason not to do it - pathcount() is fast, and could be made even faster. But if we do, we should ditch the nr_active field altogether - no need to carry it around if we re-calculate it anyway when we need it.

However, it worries me a bit that nr_active may go wrong. Both Ben and I have reviewed the code and we thought the nr_active tracking was correct. Something seems to happen in our code that we don't understand.

- can you please confirm that you are using the latest code, containing e224d57 "libmutipath: continue to use old state on PATH_PENDING", 9b715bf "multipathd: Fix miscounting active paths" and (in case you're using the marginal_paths options) also 7d4b40f and
058df77 ?

- If you have a reliable reproducer, would you mind adding log messages to the code you just submitted, so that we can observe how nr_active evolves in time, and perhaps understand why it's going wrong?

Regards
Martin

--
dm-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/dm-devel