Skip to contentSkip to conversation

multipath-tools: update no_path_retry value for IBM/2145

PATCH In review Xose Vazquez Perez 3 participants 5 messages Aug 25, 2021 – Feb 13, 2024 Device mapper

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 multipath-tools: update no_path_retry value for IBM/2145

Based on current configs: https://www.ibm.com/docs/en/flashsystem-9x00/8.4.x?topic=system-settings-linux-hosts

Cc Martin Wilck <[email protected]>
Cc Benjamin Marzinski <[email protected]>
Cc Christophe Varoqui <[email protected]>
Cc DM-DEVEL ML <[email protected]>
---
 libmultipath/hwtable.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
libmultipath/hwtable.c +1 -1
index 2a896440..58554cbb 100644
@@ -662,7 +662,7 @@ static struct hwentry default_hw[] = {
 		/* Storwize family / SAN Volume Controller / Flex System V7000 / FlashSystem V840/V9000/9100 */
 		.vendor        = "IBM",
 		.product       = "^2145",
-		.no_path_retry = NO_PATH_RETRY_QUEUE,
+		.no_path_retry = 5,
 		.pgpolicy      = GROUP_BY_PRIO,
 		.pgfailback    = -FAILBACK_IMMEDIATE,
 		.prio_name     = PRIO_ALUA,
Signature · 6 lines
2.32.0

--
dm-devel mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/dm-devel
#1 lore raw
Quoted 18 lines · Xose Vazquez Perez

Based on current configs:
https://www.ibm.com/docs/en/flashsystem-9x00/8.4.x?topic=system-settings-linux-hosts

Cc Martin Wilck <[email protected]>
Cc Benjamin Marzinski <[email protected]>
Cc Christophe Varoqui <[email protected]>
Cc DM-DEVEL ML <[email protected]>

---
 libmultipath/hwtable.c | 2 +-

 1 file changed, 1 insertion(+), 1 deletion(-)
libmultipath/hwtable.c +0 -0
index 2a896440..58554cbb 100644
@@ -662,7 +662,7 @@ static struct hwentry default_hw[] = {
                 /* Storwize family / SAN Volume Controller / Flex
Quoted 8 lines

System V7000 / FlashSystem V840/V9000/9100 */ .vendor        = "IBM", .product       = "^2145",
-               .no_path_retry = NO_PATH_RETRY_QUEUE, +               .no_path_retry = 5,
                .pgpolicy      = GROUP_BY_PRIO, .pgfailback    = -FAILBACK_IMMEDIATE, .prio_name     = PRIO_ALUA,

Ref: https://github.com/opensvc/multipath-tools/issues/6

The question is on which basis IBM came up with this recommendation. 5 (aka 25s) is a rather low value. Some users may encounter unpleasant surprises if we change the default this way, as it used to be infinite before.

Using 5, the IBS 2145 would have the 2nd-lowest default in hwtable.c after Dell PowerStore (3). Symmetrix has 6; all other arrays default to 10 or higher, many default to "queue".

Observing that the above is the documentation for the *Flashsystem* 9200, I consider it likely that the value ".no_path_retry = 5" would apply to flash-based IBM storage products, but not to the older products such as the V7000, which unfortunately use the same device ID.

It'd be helpful if someone from IBM could jump in here...

Pondering the pros and cons, I vote for keeping the current defaults for now.

Martin

--
dm-devel mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/dm-devel

#2 lore raw
Quoted 44 lines · Martin Wilck
Quoted 18 lines · Xose Vazquez Perez

Based on current configs:
https://www.ibm.com/docs/en/flashsystem-9x00/8.4.x?topic=system-settings-linux-hosts

Cc Martin Wilck <[email protected]>
Cc Benjamin Marzinski <[email protected]>
Cc Christophe Varoqui <[email protected]>
Cc DM-DEVEL ML <[email protected]>
---
  libmultipath/hwtable.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
libmultipath/hwtable.c +0 -0
index 2a896440..58554cbb 100644
@@ -662,7 +662,7 @@ static struct hwentry default_hw[] = {
                 /* Storwize family / SAN Volume Controller / Flex
Quoted 8 lines

System V7000 / FlashSystem V840/V9000/9100 */
                .vendor        = "IBM", .product       = "^2145",
-               .no_path_retry = NO_PATH_RETRY_QUEUE, +               .no_path_retry = 5,
                .pgpolicy      = GROUP_BY_PRIO, .pgfailback    = -FAILBACK_IMMEDIATE, .prio_name     = PRIO_ALUA,

Ref: https://github.com/opensvc/multipath-tools/issues/6

The question is on which basis IBM came up with this recommendation. 5 (aka 25s) is a rather low value. Some users may encounter unpleasant surprises if we change the default this way, as it used to be infinite before.

Using 5, the IBS 2145 would have the 2nd-lowest default in hwtable.c after Dell PowerStore (3). Symmetrix has 6; all other arrays default to 10 or higher, many default to "queue".

Observing that the above is the documentation for the *Flashsystem* 9200, I consider it likely that the value ".no_path_retry = 5" would apply to flash-based IBM storage products, but not to the older products such as the V7000, which unfortunately use the same device ID.

Quoted 4 lines

It'd be helpful if someone from IBM could jump in here...

Pondering the pros and cons, I vote for keeping the current defaults for now.

+1

I think this depends on host and workload requirements and maybe other things. There might not be one simple answer.

FWIW, from a zfcp point of view: https://public.dhe.ibm.com/software/dw/linux390/lvc/zFCP_Best_Practices-BB-Webcast_201805.pdf#page=19
Distributed/parallel file systems with shared volumes might have their own requirement.
YMMV

We also have our opinion on dev_loss_tmo and fast_io_fail_tmo, but that's a different story [https://public.dhe.ibm.com/software/dw/linux390/lvc/zFCP_Best_Practices-BB-Webcast_201805.pdf#page=18].

Signature · 17 lines
Mit freundlichen Gruessen / Kind regards
Steffen Maier

Linux on IBM Z and LinuxONE

https://www.ibm.com/privacy/us/en/
IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Gregor Pillen
Geschaeftsfuehrung: Dirk Wittkopp
Sitz der Gesellschaft: Boeblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294


--
dm-devel mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/dm-devel
#3 lore raw
In reply to Martin Wilck

On 8/26/21 8:47 AM, Martin Wilck wrote:
^^^^^^^
It is never too late!

Quoted 44 lines · Xose Vazquez Perez
Quoted 18 lines

Based on current configs:
https://www.ibm.com/docs/en/flashsystem-9x00/8.4.x?topic=system-settings-linux-hosts

Cc Martin Wilck <[email protected]>
Cc Benjamin Marzinski <[email protected]>
Cc Christophe Varoqui <[email protected]>
Cc DM-DEVEL ML <[email protected]>
---
  libmultipath/hwtable.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
libmultipath/hwtable.c +0 -0
index 2a896440..58554cbb 100644
@@ -662,7 +662,7 @@ static struct hwentry default_hw[] = {
                 /* Storwize family / SAN Volume Controller / Flex
Quoted 8 lines

System V7000 / FlashSystem V840/V9000/9100 */
                .vendor        = "IBM", .product       = "^2145",
-               .no_path_retry = NO_PATH_RETRY_QUEUE, +               .no_path_retry = 5,
                .pgpolicy      = GROUP_BY_PRIO, .pgfailback    = -FAILBACK_IMMEDIATE, .prio_name     = PRIO_ALUA,

Ref: https://github.com/opensvc/multipath-tools/issues/6

The question is on which basis IBM came up with this recommendation. 5 (aka 25s) is a rather low value. Some users may encounter unpleasant surprises if we change the default this way, as it used to be infinite before.

Using 5, the IBS 2145 would have the 2nd-lowest default in hwtable.c after Dell PowerStore (3). Symmetrix has 6; all other arrays default to 10 or higher, many default to "queue".

Observing that the above is the documentation for the *Flashsystem* 9200, I consider it likely that the value ".no_path_retry = 5" would apply to flash-based IBM storage products, but not to the older products such as the V7000, which unfortunately use the same device ID.

Quoted 6 lines

It'd be helpful if someone from IBM could jump in here...

Pondering the pros and cons, I vote for keeping the current defaults for now.

Martin

Some history:

first commit 3eb8c380a :
{
/* IBM SAN Volume Controller */
.vendor = "IBM",
.product = "2145",
.getuid = DEFAULT_GETUID,
.getprio = "mpath_prio_alua /dev/%n",
.features = "1 queue_if_no_path",
.hwhandler = DEFAULT_HWHANDLER,
.selector = DEFAULT_SELECTOR,
.pgpolicy = GROUP_BY_PRIO,
.pgfailback = -FAILBACK_IMMEDIATE,
.rr_weight = RR_WEIGHT_NONE,
.no_path_retry = NO_PATH_RETRY_UNDEF,
.minio = DEFAULT_MINIO,
.checker_name = TUR,
},

NO_PATH_RETRY_UNDEF was removed in b7c3cf014 because it was the default value, and later "1 queue_if_no_path" was replaced by NO_PATH_RETRY_QUEUE in 87ea76f99

IBM docs recommends:
no_path_retry 5 # or no_path_retry "fail" for some current linux distros

IBM Storage FlashSystem 5200, 5000, 5100, Storwize V5100 and V5000E:
https://www.ibm.com/docs/en/flashsystem-5x00/8.6.x?topic=system-settings-linux-hosts

IBM Storage FlashSystem 7300, 7200 and Storwize V7000:
https://www.ibm.com/docs/en/flashsystem-7x00/8.6.x?topic=system-settings-linux-hosts

IBM FlashSystem V9000:
https://www.ibm.com/docs/en/flashsystem-v9000/8.3.x?topic=system-settings-linux-hosts

IBM Storage FlashSystem 9500, 9200 and 9100:
https://www.ibm.com/docs/en/flashsystem-9x00/8.6.x?topic=system-settings-linux-hosts

Therefore, we should change this value.

#4 lore raw
Quoted 3 lines · Xose Vazquez Perez

On 8/26/21 8:47 AM, Martin Wilck wrote:
    ^^^^^^^
It is never too late!

:-)

Quoted 20 lines

Some history:

first commit 3eb8c380a :
        {
                /* IBM SAN Volume Controller */ .vendor        = "IBM",
                .product       = "2145",
                .getuid        = DEFAULT_GETUID, .getprio       = "mpath_prio_alua /dev/%n", .features      = "1 queue_if_no_path", .hwhandler     = DEFAULT_HWHANDLER, .selector      = DEFAULT_SELECTOR, .pgpolicy      = GROUP_BY_PRIO, .pgfailback    = -FAILBACK_IMMEDIATE, .rr_weight     = RR_WEIGHT_NONE, .no_path_retry = NO_PATH_RETRY_UNDEF, .minio         = DEFAULT_MINIO, .checker_name  = TUR,
        },

Quoted 4 lines

NO_PATH_RETRY_UNDEF was removed in b7c3cf014 because it was the default value,
and later "1 queue_if_no_path" was replaced by NO_PATH_RETRY_QUEUE in 87ea76f99

... which shows that the default has been "queue" for almost 18 years.

Quoted 10 lines

IBM docs recommends:
no_path_retry 5 # or no_path_retry "fail" for some current linux distros

IBM Storage FlashSystem 5200, 5000, 5100, Storwize V5100 and V5000E:
https://www.ibm.com/docs/en/flashsystem-5x00/8.6.x?topic=system-settings-linux-hosts

IBM Storage FlashSystem 7300, 7200 and Storwize V7000:
https://www.ibm.com/docs/en/flashsystem-7x00/8.6.x?topic=system-settings-linux-hosts

Quoted 7 lines

IBM FlashSystem V9000:
https://www.ibm.com/docs/en/flashsystem-v9000/8.3.x?topic=system-settings-linux-hosts

IBM Storage FlashSystem 9500, 9200 and 9100:
https://www.ibm.com/docs/en/flashsystem-9x00/8.6.x?topic=system-settings-linux-hosts

Therefore, we should change this value.

I tend to disagree. It's true that we usually follow vendor recommendations. But in this case, I think the change would do more harm than good, because we've defaulted to "queue" basically forever for this product. Suddenly switching to a rather short no_path_retry value might come as a unpleasant surprise for users. Users who follow the IBM recommendations (using explicit multipath.conf settings) won't notice the change anyway, but those who rely on our defaults might even loose data.

In general, I believe vendors recommendations about "no_path_retry" don't mean much. This setting doesn't depend on the properties of the hardware, it's rather the preference of the end customer [*]. IMHO "fail" or low numeric values of no_path_retry mainly make sense in cluster configurations. Unfortunately, IBM gives no rationale for this recommendation in its manuals [+].

But I'm not religious on the matter; more opinions welcome.

Martin

[*] Vendors can recommend a lower limit for no_path_retry, in the sense "with this product, it can happen that zero paths are available for N seconds during a firmware update", but a fixed no_path_retry value acts as an upper limit.
[+] I suspect that the recommendations in the current IBM manuals have just been copy/pasted from earlier ones, without much consideration.