Skip to contentSkip to conversation

linux raid wiki - backup files

DISCUSSION Wols Lists 3 participants 10 messages Nov 19, 2016 – Nov 22, 2016 MD RAID

lore
Share & tools
lore

Wols Lists

author
#0 lore raw

I'm updating the mdadm overview at the moment, and I'm giving examples for changing raid levels etc.

Is a backup file still required? I get the impression with current kernels and mdadm, any required backup is put in the space left by "data offset", or if you're adding a disk it stores it in the space on that disk.

Okay, I'll have to document that older versions still need it, but if that's not needed any more, I'd rather not emphasize it.

I'd also like to confirm my understanding of how a resize works ...

If the number of raid devices is increased, am I right in thinking that a new stripe one is created from the old stripes one and two, then written, then the new stripe two is created from the old two and three, etc etc? In effect, the data slumps down from the old array into the new?

And of course the reverse when a device is removed - it starts with the highest stripe, which gets pulled upwards so the old array gets pulled up into the new?

(And during the reshape, you have a marker of the current stripe being updated, so any stripes above that get read from one array, and below they get read from the other.)

Cheers,
Wol

#1 lore raw
Quoted 6 lines · Wols Lists

I'm updating the mdadm overview at the moment, and I'm giving examples for changing raid levels etc.

Is a backup file still required? I get the impression with current kernels and mdadm, any required backup is put in the space left by "data offset", or if you're adding a disk it stores it in the space on that disk.

You can't change the data offset of zero for metadata v0.90 and v1.0, the latter of which isn't obsolete. So a backup file would be needed for many --grow operations on those arrays.

Quoted 1 line

I'd also like to confirm my understanding of how a resize works ...

Just to clarify: --grow with a change in number of devices or layout is called a reshape, not a resize, even though the size may change. Just changing the amount of space used per device is a "resize". A simple resize never requires a backup file, as no chunks will move.

Quoted 8 lines

If the number of raid devices is increased, am I right in thinking that a new stripe one is created from the old stripes one and two, then written, then the new stripe two is created from the old two and three, etc etc? In effect, the data slumps down from the old array into the new?

And of course the reverse when a device is removed - it starts with the highest stripe, which gets pulled upwards so the old array gets pulled up into the new?

Yes. But the new stripes lay on top of the old stripes, unless you move the data offset. Which is why a backup file holds the old stripe just in case. If you can move the offset, you use the lower offset for the lower addresses in the array, and the higher offset for the higher addresses, on either side of the reshape position.

Quoted 3 lines

(And during the reshape, you have a marker of the current stripe being updated, so any stripes above that get read from one array, and below they get read from the other.)

Yes. And the stripe being worked on is "frozen" to prevent an upper layer from interfering.

Phil

author
#2 lore raw
Quoted 5 lines · Phil Turmel

Yes. But the new stripes lay on top of the old stripes, unless you move the data offset. Which is why a backup file holds the old stripe just in case. If you can move the offset, you use the lower offset for the lower addresses in the array, and the higher offset for the higher addresses, on either side of the reshape position.

Okay, understood. So v0.9 and v1.0 always need a backup for a reshape.

But if we have a data offset with v1.2, a reshape will use that space if it can rather than needing a backup file?

(And changing topic slightly, that space is also used for an internal bitmap?)

Cheers,
Wol

#3 lore raw
Quoted 11 lines · Wols Lists
Quoted 5 lines · Phil Turmel

Yes. But the new stripes lay on top of the old stripes, unless you move the data offset. Which is why a backup file holds the old stripe just in case. If you can move the offset, you use the lower offset for the lower addresses in the array, and the higher offset for the higher addresses, on either side of the reshape position.

Okay, understood. So v0.9 and v1.0 always need a backup for a reshape.

But if we have a data offset with v1.2, a reshape will use that space if it can rather than needing a backup file?

If there's room to move the entire data area (direction varies with operation), yes.

Quoted 2 lines

(And changing topic slightly, that space is also used for an internal bitmap?)

And bad block log.

Phil

#4 lore raw
Quoted 9 lines · Phil Turmel
Quoted 8 lines · Wols Lists
Quoted 5 lines · Phil Turmel

Yes. But the new stripes lay on top of the old stripes, unless you move the data offset. Which is why a backup file holds the old stripe just in case. If you can move the offset, you use the lower offset for the lower addresses in the array, and the higher offset for the higher addresses, on either side of the reshape position.

Okay, understood. So v0.9 and v1.0 always need a backup for a reshape.

Having looked at the man page, this now seems obvious - the superblock is at the end, so the data offset is 0. But for a 1.0 array, could we create a data offset?

(So, if we created a data offset, we could then move the superblock and convert a 1.0 to 1.1 or 1.2? Okay, it can't do it now, but it looks to me like it shouldn't be that hard ... ?)

Quoted 4 lines
Quoted 3 lines

But if we have a data offset with v1.2, a reshape will use that space if it can rather than needing a backup file?

I'm guessing that 1.0 and 1.1 defaulted to no data offset to speak of? And if we (can) create a decent data offset, we can then use that in exactly the same way as with v1.2?

Cheers,
Wol

#5 lore raw
Quoted 7 lines · Anthony Youngman

Having looked at the man page, this now seems obvious - the superblock is at the end, so the data offset is 0. But for a 1.0 array, could we create a data offset?

(So, if we created a data offset, we could then move the superblock and convert a 1.0 to 1.1 or 1.2? Okay, it can't do it now, but it looks to me like it shouldn't be that hard ... ?)

I suppose it would be possible, but defeats the purpose of having data offset == 0: making non-parity array contents directly readable outside the array. Commonly used to raid boot partitions but still have grub able to read them directly. Less of an issue today with grub2, I suppose (but I don't use bootloaders anymore, so I'm not a good resource for that).

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

But if we have a data offset with v1.2, a reshape will use that space if it can rather than needing a backup file?

I'm guessing that 1.0 and 1.1 defaulted to no data offset to speak of? And if we (can) create a decent data offset, we can then use that in exactly the same way as with v1.2?

V1.1 and v1.2 are identical except for the superblock offset (one 4k block difference). v1.1 reshapes just like v1.2.

Phil

#6 lore raw
Quoted 9 lines · Phil Turmel
Quoted 7 lines · Anthony Youngman

Having looked at the man page, this now seems obvious - the superblock is at the end, so the data offset is 0. But for a 1.0 array, could we create a data offset?

(So, if we created a data offset, we could then move the superblock and convert a 1.0 to 1.1 or 1.2? Okay, it can't do it now, but it looks to me like it shouldn't be that hard ... ?)

Quoted 6 lines

I suppose it would be possible, but defeats the purpose of having data offset == 0: making non-parity array contents directly readable outside the array. Commonly used to raid boot partitions but still have grub able to read them directly. Less of an issue today with grub2, I suppose (but I don't use bootloaders anymore, so I'm not a good resource for that).

Only works for mirrors ...

So basically, there's no point using 1.0 for any form of parity raid. In which case, if you convert a mirror to parity, it would be nice to be able to create said offset and move the superblock.

Quoted 11 lines
Quoted 7 lines
Quoted 4 lines
Quoted 3 lines

But if we have a data offset with v1.2, a reshape will use that space if it can rather than needing a backup file?

I'm guessing that 1.0 and 1.1 defaulted to no data offset to speak of? And if we (can) create a decent data offset, we can then use that in exactly the same way as with v1.2?

V1.1 and v1.2 are identical except for the superblock offset (one 4k block difference). v1.1 reshapes just like v1.2.

Any reason for that difference? How big is the superblock? Did v1.1 fill most of the first 4K and just leave not much room with a default 4K data offset initially?

Quoted 2 lines

Phil

Cheers,
Wol

#7 lore raw
Quoted 3 lines · Anthony Youngman

So basically, there's no point using 1.0 for any form of parity raid. In which case, if you convert a mirror to parity, it would be nice to be able to create said offset and move the superblock.

Not for new parity arrays, no. v1.0 is what you get if you convert an old v0.90 array to v1.x using --assemble --update=metadata. I simply don't know if a v1.0 superblock will run with a non-zero data offset. That would have to be an intermediate step before the superblock and the rest of the metadata could be relocated.

Quoted 6 lines
Quoted 3 lines

V1.1 and v1.2 are identical except for the superblock offset (one 4k block difference). v1.1 reshapes just like v1.2.

Any reason for that difference? How big is the superblock? Did v1.1 fill most of the first 4K and just leave not much room with a default 4K data offset initially?

v1.1 places the superblock in the first sector of the device. v1.2 skips the first 4k, making such arrays resistant to destruction by accidental creation of MBR partition tables or similar crises.

Phil

#8 lore raw
In reply to Anthony Youngman
Quoted 9 lines · Anthony Youngman
Quoted 8 lines · Phil Turmel
Quoted 7 lines · Wols Lists
Quoted 5 lines · Phil Turmel

Yes. But the new stripes lay on top of the old stripes, unless you move the data offset. Which is why a backup file holds the old stripe just in case. If you can move the offset, you use the lower offset for the lower addresses in the array, and the higher offset for the higher addresses, on either side of the reshape position.

Quoted 5 lines
Quoted 1 line
Quoted 1 line

Okay, understood. So v0.9 and v1.0 always need a backup for a reshape.

Having looked at the man page, this now seems obvious - the superblock is at the end, so the data offset is 0. But for a 1.0 array, could we create a data offset?

Yes. But usually the purpose for using 1.0 is to have data_offset == 0, so you might not want to.

Quoted 4 lines

(So, if we created a data offset, we could then move the superblock and convert a 1.0 to 1.1 or 1.2? Okay, it can't do it now, but it looks to me like it shouldn't be that hard ... ?)

It would be quite easy to extend "--update=metadata" to change the version once the data_offset had been changed.

Quoted 7 lines
Quoted 4 lines
Quoted 3 lines

But if we have a data offset with v1.2, a reshape will use that space if it can rather than needing a backup file?

I'm guessing that 1.0 and 1.1 defaulted to no data offset to speak of? And if we (can) create a decent data offset, we can then use that in exactly the same way as with v1.2?

1.0 defaults to no data_offset.
1.1 uses the safe choice function as 1.2.

NeilBrown

#9 lore raw

Okay, after all that I've ended up not mentioning backup files at all. I get the feeling that in pretty much all modern scenarios they are not required. If we have v1.1 or v1.2 there should be enough spare metadata space, and a modern system shouldn't be using v1.0. It's only of interest if you are doing a bios boot off a mirror with no initramfs. Not a likely option for a system being set up today. And from the man page, mdadm leaves 128K at the end of each device - would a v1.0 array have that? Would that be used for backup?

I probably ought to work out how I can get it to fit neatly in my mdadm page, especially if it's only needed for older setups.

Can you take a look at the page and make sure it's not got anything obviously wrong ...

https://raid.wiki.kernel.org/index.php/A_guide_to_mdadm

And then I'll try and sort out a page on finding lost arrays - we've had a few on the list recently ...

Cheers,
Wol

Quoted 41 lines · NeilBrown
Quoted 9 lines · Anthony Youngman
Quoted 8 lines · Phil Turmel
Quoted 7 lines · Wols Lists
Quoted 5 lines · Phil Turmel

Yes. But the new stripes lay on top of the old stripes, unless you move the data offset. Which is why a backup file holds the old stripe just in case. If you can move the offset, you use the lower offset for the lower addresses in the array, and the higher offset for the higher addresses, on either side of the reshape position.

Quoted 5 lines
Quoted 1 line
Quoted 1 line

Okay, understood. So v0.9 and v1.0 always need a backup for a reshape.

Having looked at the man page, this now seems obvious - the superblock is at the end, so the data offset is 0. But for a 1.0 array, could we create a data offset?

Yes. But usually the purpose for using 1.0 is to have data_offset == 0, so you might not want to.

Quoted 4 lines

(So, if we created a data offset, we could then move the superblock and convert a 1.0 to 1.1 or 1.2? Okay, it can't do it now, but it looks to me like it shouldn't be that hard ... ?)

It would be quite easy to extend "--update=metadata" to change the version once the data_offset had been changed.

Quoted 7 lines
Quoted 4 lines
Quoted 3 lines

But if we have a data offset with v1.2, a reshape will use that space if it can rather than needing a backup file?

I'm guessing that 1.0 and 1.1 defaulted to no data offset to speak of? And if we (can) create a decent data offset, we can then use that in exactly the same way as with v1.2?

1.0 defaults to no data_offset.
1.1 uses the safe choice function as 1.2.

NeilBrown