Skip to contentSkip to conversation

LVM + Multipathing

linux-lvm Ritesh Raj Sarraf 5 participants 8 messages Feb 17, 2007 – Feb 28, 2007

lore
Share & tools
lore
#0 lore raw

Hi,

I have a question regarding the combination of LVM + Multipathing in a SAN environment.

I have a LUN mapped to a host using iSCSI. I have two two paths to the LUN. This setup, using multipathing, provides me failover functionality.

My question is how does LVM react to path failures ?
In a failover environment, multipathing can take upto 120 seconds to detect that the active path to the LUN is no more available and then switch to the secondary path. During this 120 seconds, how does LVM react ? Does it really sense that the device is offlined ? Or that information is never passed to LVM and it just allows I/O to happen leaving that responsibility to the multipathing layer ?

Thanks,
Ritesh

Signature · 6 lines
Ritesh Raj Sarraf
RESEARCHUT - http://www.researchut.com
"Necessity is the mother of invention."
"Stealing logic from one person is plagiarism, stealing from many is research."
"The great are those who achieve the impossible, the petty are those who
cannot - rrs"
#1 lore raw
Quoted 8 lines · Ritesh Raj Sarraf

Hi,

I have a question regarding the combination of LVM + Multipathing in a SAN environment.

I have a LUN mapped to a host using iSCSI. I have two two paths to the LUN. This setup, using multipathing, provides me failover functionality.

Quoted 7 lines

My question is how does LVM react to path failures ?
In a failover environment, multipathing can take upto 120 seconds to detect that the active path to the LUN is no more available and then switch to the secondary path. During this 120 seconds, how does LVM react ? Does it really sense that the device is offlined ? Or that information is never passed to LVM and it just allows I/O to happen leaving that responsibility to the multipathing layer ?

i believe LVM does not react at all.

L.

Signature · 6 lines
Luca Berra -- [email protected]
       Communication Media & Services S.r.l.
/"\
\ /     ASCII RIBBON CAMPAIGN
 X        AGAINST HTML MAIL
/ \
#2 lore raw
Quoted 17 lines · Luca Berra
Quoted 8 lines · Ritesh Raj Sarraf

Hi,

I have a question regarding the combination of LVM + Multipathing in a SAN environment.

I have a LUN mapped to a host using iSCSI. I have two two paths to the LUN. This setup, using multipathing, provides me failover functionality.

Quoted 7 lines

My question is how does LVM react to path failures ?
In a failover environment, multipathing can take upto 120 seconds to detect that the active path to the LUN is no more available and then switch to the secondary path. During this 120 seconds, how does LVM react ? Does it really sense that the device is offlined ? Or that information is never passed to LVM and it just allows I/O to happen leaving that responsibility to the multipathing layer ?

Quoted 2 lines

i believe LVM does not react at all.

I brought this question because I'm seeing Filesystem READONLY issues when doing a takeover/giveback.

My understanding is that a filesystem usually goes read-only, only when it senses errors in the drive. Now I believe LVM might the culprit because beneath the filesystem and above the block device, LVM is the only layer.

Thanks,
Ritesh

Signature · 6 lines
Ritesh Raj Sarraf
RESEARCHUT - http://www.researchut.com
"Necessity is the mother of invention."
"Stealing logic from one person is plagiarism, stealing from many is research."
"The great are those who achieve the impossible, the petty are those who
cannot - rrs"
#3 lore raw
Quoted 22 lines · Ritesh Raj Sarraf
Quoted 17 lines · Luca Berra
Quoted 8 lines · Ritesh Raj Sarraf

Hi,

I have a question regarding the combination of LVM + Multipathing in a SAN environment.

I have a LUN mapped to a host using iSCSI. I have two two paths to the LUN. This setup, using multipathing, provides me failover functionality.

Quoted 7 lines

My question is how does LVM react to path failures ?
In a failover environment, multipathing can take upto 120 seconds to detect that the active path to the LUN is no more available and then switch to the secondary path. During this 120 seconds, how does LVM react ? Does it really sense that the device is offlined ? Or that information is never passed to LVM and it just allows I/O to happen leaving that responsibility to the multipathing layer ?

Quoted 2 lines

i believe LVM does not react at all.

Quoted 6 lines

I brought this question because I'm seeing Filesystem READONLY issues when doing a takeover/giveback.

My understanding is that a filesystem usually goes read-only, only when it senses errors in the drive. Now I believe LVM might the culprit because beneath the filesystem and above the block device, LVM is the only layer.

to clarify my point above:
i meant that LVM (actually device-mapper) does not do anything special. it just passes the IO requests from the fs layer to the underlying block device and if the underlyng block device returns an IO error then it is passed back to the fs. which will cause ext2 to remount the filesystem readonly.

L.

Signature · 6 lines
Luca Berra -- [email protected]
       Communication Media & Services S.r.l.
/"\
\ /     ASCII RIBBON CAMPAIGN
 X        AGAINST HTML MAIL
/ \
#4 lore raw
Quoted 5 lines · Luca Berra

i meant that LVM (actually device-mapper) does not do anything special. it just passes the IO requests from the fs layer to the underlying block device and if the underlyng block device returns an IO error then it is passed back to the fs. which will cause ext2 to remount the filesystem readonly.

Hi,

Thanks for clarifying.

In my case, the block device is hidden by the multipathing layer. It is something like:
(Block Device(Multipathing(LVM(Filesystem) ) ) )

Now during takeover/giveback, the multipathing layer is intelligent enough to wait till <120 seconds before declaring that the path has really gone offline and no more paths are available.
If within the 120 seconds time span, the takeover succeeds, the path is back online and everything works fine in a non-LVM setup.

It is only in an LVM setup that a takeover/giveback ends up with the host OS having a filesystem read-only problem.
Now if I go with your explanation, I shouldn't have had the filesystem read-only problem since the I/O is being passed on to the multipathing layer which is intelligent enough to wait for N seconds before really sending an I/O Error.

Are there any timeout options in LVM to allow it to wait for N seconds before sending out an error ?
(I understand that LVM might not be involved, but just wondering).

Thanks,
Ritesh

Signature · 6 lines
Ritesh Raj Sarraf
RESEARCHUT - http://www.researchut.com
"Necessity is the mother of invention."
"Stealing logic from one person is plagiarism, stealing from many is research."
"The great are those who achieve the impossible, the petty are those who
cannot - rrs"
#5 lore raw
Quoted 16 lines · Ritesh Raj Sarraf
Quoted 5 lines · Luca Berra

i meant that LVM (actually device-mapper) does not do anything special. it just passes the IO requests from the fs layer to the underlying block device and if the underlyng block device returns an IO error then it is passed back to the fs. which will cause ext2 to remount the filesystem readonly.

Hi,

Thanks for clarifying.

In my case, the block device is hidden by the multipathing layer. It is something like:
(Block Device(Multipathing(LVM(Filesystem) ) ) )

Quoted 8 lines

Now during takeover/giveback, the multipathing layer is intelligent enough to wait till <120 seconds before declaring that the path has really gone offline and no more paths are available.
If within the 120 seconds time span, the takeover succeeds, the path is back online and everything works fine in a non-LVM setup.

It is only in an LVM setup that a takeover/giveback ends up with the host OS having a filesystem read-only problem.

Interesting.

Quoted 8 lines

Now if I go with your explanation, I shouldn't have had the filesystem read-only problem since the I/O is being passed on to the multipathing layer which is intelligent enough to wait for N seconds before really sending an I/O Error.

Are there any timeout options in LVM to allow it to wait for N seconds before sending out an error ?
(I understand that LVM might not be involved, but just wondering).

There are no LVM timeouts like you are suggesting. LVM just does I/O to devices like any other application. The timeout settings you're looking for should be in the layers below LVM.

What does your /etc/multipath.conf file look like? Do you have "no_path_retry" set and/or "queue_if_no_path"? Both of these settings will affect how multipath deals with a "no paths available" situation.

There are also settings below multipath, in the low-level driver(s). Since you are using iscsi, I assume you've got open-iscsi? You can set node.session.timeo.replacement_timeout to a lower value (it is 120 by default in /etc/iscsi/iscsid.conf) if you want faster path failure detection and faster failover.

In a lot of cases, people set the low level driver timeouts smaller (< 30 sec) to allow for quicker failure detection and failover, but a higher or infinite value for the "no paths available" situation.

You might try open-iscsi.org and/or dm-devel lists.

Quoted 3 lines

Thanks,
Ritesh

#6 lore raw
In reply to Luca Berra

Your problem might be related to this bug
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=213921

Quoted 52 lines · Luca Berra
Quoted 22 lines · Ritesh Raj Sarraf
Quoted 17 lines · Luca Berra
Quoted 8 lines · Ritesh Raj Sarraf

Hi,

I have a question regarding the combination of LVM + Multipathing in a SAN environment.

I have a LUN mapped to a host using iSCSI. I have two two paths to the LUN. This setup, using multipathing, provides me failover functionality.

Quoted 7 lines

My question is how does LVM react to path failures ?
In a failover environment, multipathing can take upto 120 seconds to detect that the active path to the LUN is no more available and then switch to the secondary path. During this 120 seconds, how does LVM react ? Does it really sense that the device is offlined ? Or that information is never passed to LVM and it just allows I/O to happen leaving that responsibility to the multipathing layer ?

Quoted 2 lines

i believe LVM does not react at all.

Quoted 6 lines

I brought this question because I'm seeing Filesystem READONLY issues when doing a takeover/giveback.

My understanding is that a filesystem usually goes read-only, only when it senses errors in the drive. Now I believe LVM might the culprit because beneath the filesystem and above the block device, LVM is the only layer.

to clarify my point above:
i meant that LVM (actually device-mapper) does not do anything special. it just passes the IO requests from the fs layer to the underlying block device and if the underlyng block device returns an IO error then it is passed back to the fs. which will cause ext2 to remount the filesystem readonly.

L.

--
Luca Berra -- [email protected]
Communication Media & Services S.r.l.
/"\
\ / ASCII RIBBON CAMPAIGN
X AGAINST HTML MAIL
/ \

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

#7 lore raw

Using device-mapper-multipath to create a device alias (/dev/mapper/<...>) which you would then use as a PV in your LVM setup. Device-mapper-multipath can indeed handle multi-pathingsuch that a failure of one one of the path would not lead to any IO issues and LVm would just keep on ticking.

"Matt P"
<slarty.tj@gmail.
com> To
Sent by: [email protected]
linux-lvm-bounces cc
@redhat.com
Subject
Re: [linux-lvm] Re: LVM +
20.02.2007 18:03 Multipathing

Please respond to
LVM general
discussion and
development
<linux-lvm@redhat
.com>

Your problem might be related to this bug
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=213921

Quoted 7 lines · Luca Berra
Quoted 6 lines · Ritesh Raj Sarraf
Quoted 4 lines · Luca Berra
Quoted 3 lines · Ritesh Raj Sarraf

Hi,

I have a question regarding the combination of LVM + Multipathing in a

SAN

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

environment.

I have a LUN mapped to a host using iSCSI. I have two two paths to the

LUN.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

This setup, using multipathing, provides me failover functionality.

My question is how does LVM react to path failures ?
In a failover environment, multipathing can take upto 120 seconds to

detect

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

that the active path to the LUN is no more available and then switch

to the

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

secondary path. During this 120 seconds, how does LVM react ? Does it

really

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

sense that the device is offlined ? Or that information is never

passed to LVM

Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

and it just allows I/O to happen leaving that responsibility to the multipathing layer ?

i believe LVM does not react at all.

I brought this question because I'm seeing Filesystem READONLY issues

when doing

Quoted 3 lines
Quoted 3 lines

a takeover/giveback.

My understanding is that a filesystem usually goes read-only, only when

it

Quoted 1 line
Quoted 1 line

senses errors in the drive. Now I believe LVM might the culprit because

beneath

Quoted 18 lines
Quoted 1 line

the filesystem and above the block device, LVM is the only layer.

to clarify my point above:
i meant that LVM (actually device-mapper) does not do anything special. it just passes the IO requests from the fs layer to the underlying block device and if the underlyng block device returns an IO error then it is passed back to the fs. which will cause ext2 to remount the filesystem readonly.

L.

--
Luca Berra -- [email protected]
Communication Media & Services S.r.l.
/"\
\ / ASCII RIBBON CAMPAIGN
X AGAINST HTML MAIL
/ \

Quoted 6 lines

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/