Skip to contentSkip to conversation

problems with raid=noautodetect

DISCUSSION Florian Dazinger 6 participants 13 messages May 22, 2006 – May 31, 2006 MD RAID

lore
Share & tools
lore
#0 lore raw

hi list,
I read somewhere that it would be better not to rely on the autodetect-mechanism in the kernel at boot time, but rather to set up /etc/mdadm.conf accordingly and boot with raid=noautodetect. Well, I tried that :)

I set up /etc/mdadm.conf for my 2 raid5 arrays:

---- snip ----
# mountpoint: /home/media
ARRAY /dev/md0
level=raid5
UUID=86ed1434:43380717:4abf124e:970d843a
devices=/dev/sda1,/dev/sdb1,/dev/sdd3

# mountpoint: /mnt/raid
ARRAY /dev/md1
level=raid5
UUID=baf59fb5:f4805e7a:91a77644:af3dde17
# devices=/dev/sda2,/dev/sdb2,/dev/sdd2
---- snap ----

and rebooted with raid=noautodetect. It booted fine, but the 3rd disks from each array (/dev/sdd2 and /dev/sdd3) were removed, so I had 2 degraded raid5 arrays. It was possible to readd them with sth. like:

mdadm /dev/md0 -a /dev/sdd3
(synced and /proc/mdstat showed [UUU])

but after the next reboot, the two partitions were again removed ([UU_])?! This was a reproducible error, I tried it several times with different /etc/mdadm.conf settings (ARRAY-statement with UUID=, devices=, UUID+devices, etc.).

I´m now running autodetect again, all raid arrays are working fine, but can anyone explain this strange behaviour? (kernel-2.6.16.14, amd64)

thanks,
florian

PS: please cc me, as I´m not subscribed to the list
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#1 lore raw
Quoted 15 lines · On Monday May 22, [email protected] wrote:

hi list,
I read somewhere that it would be better not to rely on the autodetect-mechanism in the kernel at boot time, but rather to set up /etc/mdadm.conf accordingly and boot with raid=noautodetect. Well, I tried that :)

I set up /etc/mdadm.conf for my 2 raid5 arrays:

---- snip ----
# mountpoint: /home/media
ARRAY /dev/md0

    level=raid5
    UUID=86ed1434:43380717:4abf124e:970d843a
    devices=/dev/sda1,/dev/sdb1,/dev/sdd3
Quoted 6 lines

# mountpoint: /mnt/raid
ARRAY /dev/md1

    level=raid5
    UUID=baf59fb5:f4805e7a:91a77644:af3dde17

# devices=/dev/sda2,/dev/sdb2,/dev/sdd2
---- snap ----

Presumably you have a 'DEVICE' line in mdadm.conf too? What is it. My first guess is that it isn't listing /dev/sdd? somehow.

Otherwise, can you add a '-v' to the mdadm command that assembles the array, and capture the output. That might be helpful.

NeilBrown

Quoted 17 lines

and rebooted with raid=noautodetect. It booted fine, but the 3rd disks from each array (/dev/sdd2 and /dev/sdd3) were removed, so I had 2 degraded raid5 arrays. It was possible to readd them with sth. like:

mdadm /dev/md0 -a /dev/sdd3
(synced and /proc/mdstat showed [UUU])

but after the next reboot, the two partitions were again removed ([UU_])?! This was a reproducible error, I tried it several times with different /etc/mdadm.conf settings (ARRAY-statement with UUID=, devices=, UUID+devices, etc.).

I´m now running autodetect again, all raid arrays are working fine, but can anyone explain this strange behaviour? (kernel-2.6.16.14, amd64)

Quoted 8 lines

thanks,
florian

PS: please cc me, as I´m not subscribed to the list
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#2 lore raw

Re: problems with raid=noautodetect - solved

Quoted 8 lines · Neil Brown

Presumably you have a 'DEVICE' line in mdadm.conf too? What is it. My first guess is that it isn't listing /dev/sdd? somehow.

Otherwise, can you add a '-v' to the mdadm command that assembles the array, and capture the output. That might be helpful.

NeilBrown

stupid me! I had a DEVICE section, but somehow forgot about my /dev/sdd drive.

sorry for the noise..

thanks,
florian

#3 lore raw

Re: problems with raid=noautodetect - solved

On 24 May 2006, Florian Dazinger uttered the following:

Quoted 8 lines
Quoted 6 lines · Neil Brown

Presumably you have a 'DEVICE' line in mdadm.conf too? What is it. My first guess is that it isn't listing /dev/sdd? somehow. Otherwise, can you add a '-v' to the mdadm command that assembles the array, and capture the output. That might be helpful. NeilBrown

stupid me! I had a DEVICE section, but somehow forgot about my /dev/sdd drive.

`DEVICE partitions' is generally preferable for that reason, unless you have entries in /proc/partitions which you explicitly want to exclude from scanning for RAID superblocks.

Signature · 3 lines
`On a scale of 1-10, X's "brokenness rating" is 1.1, but that's only
 because bringing Windows into the picture rescaled "brokenness" by
 a factor of 10.' --- Peter da Silva
#4 lore raw
In reply to Neil Brown
Quoted 2 lines · Neil Brown

Presumably you have a 'DEVICE' line in mdadm.conf too? What is it. My first guess is that it isn't listing /dev/sdd? somehow.

Neil,
i am seeing a lot of people that fall in this same error, and i would propose a way of avoiding this problem

1) make "DEVICE partitions" the default if no device line is specified.
2) deprecate the "DEVICE" keyword issuing a warning when it is found in the configuration file
3) introduce "DEVICEFILTER" or similar keyword with the same meaning at the actual "DEVICE" keyboard
4) optionally add an "EXCLUDEDEVICE" keyword with the opposite meaning.

L.

Signature · 6 lines
Luca Berra -- [email protected]
       Communication Media & Services S.r.l.
/"\
\ /     ASCII RIBBON CAMPAIGN
 X        AGAINST HTML MAIL
/ \
#5 lore raw
Quoted 9 lines · Luca Berra
Quoted 2 lines · Neil Brown

Presumably you have a 'DEVICE' line in mdadm.conf too? What is it. My first guess is that it isn't listing /dev/sdd? somehow.

Neil,
i am seeing a lot of people that fall in this same error, and i would propose a way of avoiding this problem

1) make "DEVICE partitions" the default if no device line is specified.

oops,
just read your 2.5 announce, you already did that :)

Quoted 6 lines

2) deprecate the "DEVICE" keyword issuing a warning when it is found in the configuration file
3) introduce "DEVICEFILTER" or similar keyword with the same meaning at the actual "DEVICE" keyboard
4) optionally add an "EXCLUDEDEVICE" keyword with the opposite meaning.

Signature · 6 lines
Luca Berra -- [email protected]
       Communication Media & Services S.r.l.
/"\
\ /     ASCII RIBBON CAMPAIGN
 X        AGAINST HTML MAIL
/ \
#6 lore raw
In reply to Luca Berra
Quoted 9 lines · On Friday May 26, [email protected] wrote:
Quoted 2 lines · Neil Brown

Presumably you have a 'DEVICE' line in mdadm.conf too? What is it. My first guess is that it isn't listing /dev/sdd? somehow.

Neil,
i am seeing a lot of people that fall in this same error, and i would propose a way of avoiding this problem

1) make "DEVICE partitions" the default if no device line is specified.

As you note, we think alike on this :-)

Quoted 2 lines

2) deprecate the "DEVICE" keyword issuing a warning when it is found in the configuration file

Not sure I'm so keen on that, at least not in the near term.

Quoted 2 lines

3) introduce "DEVICEFILTER" or similar keyword with the same meaning at the actual "DEVICE" keyboard

If it has the same meaning, why not leave it called 'DEVICE'???

However, there is at least the beginnings of a good idea here.

If we assume there is a list of devices provided by a (possibly default) 'DEVICE' line, then

DEVICEFILTER !pattern1 !pattern2 pattern3 pattern4

could mean that any device in that list which matches pattern 1 or 2 is immediately discarded, and remaining device that matches patterns 3 or 4 are included, and the remainder are discard.

The rule could be that the default is to include any devices that don't match a !pattern, unless there is a pattern without a '!', in which case the default is to reject non-accepted patterns. Is that straight forward enough, or do I need an
order allow,deny
like apache has?

Thanks for the suggestion.

NeilBrown

#7 lore raw
Quoted 4 lines · Neil Brown
Quoted 2 lines

3) introduce "DEVICEFILTER" or similar keyword with the same meaning at the actual "DEVICE" keyboard

If it has the same meaning, why not leave it called 'DEVICE'???

the idea was to warn people that write

DEVICE /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1
ARRAY /dev/md0 .......

that it might break since disk naming is not guaranteed to be constant.

Quoted 11 lines

However, there is at least the beginnings of a good idea here.

If we assume there is a list of devices provided by a (possibly default) 'DEVICE' line, then

DEVICEFILTER !pattern1 !pattern2 pattern3 pattern4

could mean that any device in that list which matches pattern 1 or 2 is immediately discarded, and remaining device that matches patterns 3 or 4 are included, and the remainder are discard.

Quoted 7 lines

The rule could be that the default is to include any devices that don't match a !pattern, unless there is a pattern without a '!', in which case the default is to reject non-accepted patterns. Is that straight forward enough, or do I need an
order allow,deny
like apache has?

I think that documenting the feature would be enough L.

Signature · 6 lines
Luca Berra -- [email protected]
       Communication Media & Services S.r.l.
/"\
\ /     ASCII RIBBON CAMPAIGN
 X        AGAINST HTML MAIL
/ \
#8 lore raw
In reply to Neil Brown
Quoted 1 line · Neil Brown

On Friday May 26, [email protected] wrote:

[]

Quoted 9 lines

If we assume there is a list of devices provided by a (possibly default) 'DEVICE' line, then

DEVICEFILTER !pattern1 !pattern2 pattern3 pattern4

could mean that any device in that list which matches pattern 1 or 2 is immediately discarded, and remaining device that matches patterns 3 or 4 are included, and the remainder are discard.

Quoted 6 lines

The rule could be that the default is to include any devices that don't match a !pattern, unless there is a pattern without a '!', in which case the default is to reject non-accepted patterns. Is that straight forward enough, or do I need an
order allow,deny
like apache has?

I'd suggest the following.

"All the other devices" are included or excluded from the list of devices to consider based on the last component in the DEVICE line. Ie. if it ends up at !dev, all the rest of devices are included. If it ends up at dev (w/o !), all the rest are excluded. If memory serves me right, it's how squid ACLs works.

There's no need to introduce new keyword. Given this rule, a line

DEVICE a b c

will do exactly as it does now. Line

DEVICE a b c !d

is somewhat redundant - it's the same as DEVICE !d
Ie, if the list ends up at !stuff, append `partitions' (or *) to it.

Ofcourse mixing !s and !!s is useful, like to say use all sda* but not sda1:

DEVICE !sda1 sda*

(and nothing else).

And the default is to have `DEVICE partitions'.

The only possible issue I see here is that with udev, it's possible to use, say, /dev/disk/by-id/*-like stuff (don't remember exact directory layout) -- symlinked to /dev/sd* according to the disk serial number or something like that -- for this to work, mdadm needs to use glob() internally.

/mjt

#9 lore raw
Quoted 11 lines · Michael Tokarev
Quoted 1 line · Neil Brown

On Friday May 26, [email protected] wrote:

I'd suggest the following.

"All the other devices" are included or excluded from the list of devices to consider based on the last component in the DEVICE line. Ie. if it ends up at !dev, all the rest of devices are included. If it ends up at dev (w/o !), all the rest are excluded. If memory serves me right, it's how squid ACLs works.

There's no need to introduce new keyword. Given this rule, a line

as i said the new keyword is to warn on configurations that do not account for changing device-ids, and if we change the syntax a new keyword would make it clearer. In case the user tries to use a new configuration on an old mdadm.

Quoted 5 lines

The only possible issue I see here is that with udev, it's possible to use, say, /dev/disk/by-id/*-like stuff (don't remember exact directory layout) -- symlinked to /dev/sd* according to the disk serial number or something like that -- for this to work, mdadm needs to use glob() internally.

uhm
i think that we would better translate any device found on a DEVICE (or DEVICEFILTER) line to the corresponding major/minor number and blacklist based on that.
nothing prevents someone to have an udev rule that creates a device node, instead of symlinking.

L.

Signature · 6 lines
Luca Berra -- [email protected]
       Communication Media & Services S.r.l.
/"\
\ /     ASCII RIBBON CAMPAIGN
 X        AGAINST HTML MAIL
/ \
#10 lore raw
In reply to Neil Brown
Quoted 22 lines · Neil Brown
Quoted 7 lines · On Friday May 26, [email protected] wrote:
Quoted 4 lines · Neil Brown

Presumably you have a 'DEVICE' line in mdadm.conf too? What is it. My first guess is that it isn't listing /dev/sdd? somehow.

Quoted 7 lines

Neil,
i am seeing a lot of people that fall in this same error, and i would propose a way of avoiding this problem

1) make "DEVICE partitions" the default if no device line is specified.

As you note, we think alike on this :-)

Quoted 7 lines
Quoted 4 lines

2) deprecate the "DEVICE" keyword issuing a warning when it is found in the configuration file

Not sure I'm so keen on that, at least not in the near term.

Let's not start warning and depreciating powerful features because they can be misused... If I wanted someone to make decisions for me I would be using this software at all.

Signature · 3 lines
bill davidsen <[email protected]>
 CTO TMR Associates, Inc
 Doing interesting things with small computers since 1979
#11 lore raw
Quoted 6 lines · Bill Davidsen
Quoted 5 lines
Quoted 2 lines

2) deprecate the "DEVICE" keyword issuing a warning when it is found in the configuration file

Not sure I'm so keen on that, at least not in the near term.

Let's not start warning and depreciating powerful features because they can be misused... If I wanted someone to make decisions for me I would be using this software at all.

you cut the rest of the mail.
i did not propose to deprecate the feature,
just the keyword.

but, ok,
just go on writing DEVICE /dev/sda1
DEVICE /dev/sdb1
ARRAY /dev/md0 devices=/dev/sda1,/dev/sdb1

then come on the list and complain when it stops working.

L.

Signature · 6 lines
Luca Berra -- [email protected]
       Communication Media & Services S.r.l.
/"\
\ /     ASCII RIBBON CAMPAIGN
 X        AGAINST HTML MAIL
/ \
#12 lore raw
Quoted 6 lines · Luca Berra
Quoted 4 lines · Bill Davidsen
Quoted 4 lines
Quoted 2 lines

2) deprecate the "DEVICE" keyword issuing a warning when it is found in the configuration file

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

Not sure I'm so keen on that, at least not in the near term.

Let's not start warning and depreciating powerful features because they can be misused... If I wanted someone to make decisions for me I would be using this software at all.

you cut the rest of the mail.

Trimming the part about which I make no comment is usually a good thing.

Quoted 2 lines

i did not propose to deprecate the feature,
just the keyword.

"A rose by any other name would still smell as sweet." In other words, the capability is still able to be misused, and changing the name or generating error messages will only cause work and concern for people using the feature.

Quoted 7 lines

but, ok,
just go on writing DEVICE /dev/sda1
DEVICE /dev/sdb1
ARRAY /dev/md0 devices=/dev/sda1,/dev/sdb1

then come on the list and complain when it stops working.

What I suggest is that the feature keep working, and no one will complain. If there is a missing partition the error messages are clear. The feature is mainly used when there are partitions or drives which should not be examined, and "stops working" only when a hardware config has changed.

Signature · 3 lines
bill davidsen <[email protected]>
 CTO TMR Associates, Inc
 Doing interesting things with small computers since 1979