pro/cons of raid1 with mdadm/lvm2

Hello,

recently I was migrating from Debian to openSUSE and wanted to make it smooth by dismantling my 2x1TB raid-1 array, install SUSE on the 1st disk, cp /home data, check everything is OK and then add 2nd disk containing Debian into raid-1 array.

However, in orde to accomplish it I learnt that one cannot simply degrade mount and use such array like with mdadm, but I had to convert system with -dconvert=single -mconvert=single which takes some time.

That's why I'm considering to put all my partitions (swap, root with several subvolumes, home) under LVM2 volumes and then create raid-1 array with mdadm since that would enable to me more easy and quickly temporarily dismantle raid-1 array, do some data manipulation from one disk to another (sometimes I use 2nd disk as temporatily storage when restoring some archived data from tapes etc.) and then resilver raid-1 array.

However, I wonder if there are some 'cons' in having raid-1 partition under mdadm and not using native mirroring capabilities of btrfs fs?

Let me add that I also want to take advantage of using SUSE's snapshots features, but I hope that's not the obstacle for the above-mentioned layout - I'd still use btrfs' snapshot facility.

Sincerely,
Gour

Whenever and wherever there is a decline in religious practice, 
O descendant of Bharata, and a predominant rise of irreligion — 
at that time I descend Myself.

When the 2 disks have different data mdadm has no way of knowing which one is correct and has a 50% chance of overwriting good data. But BTRFS does checksums on all reads and solves the problem of corrupt data - as long as you don't have 2 corrupt sectors in matching blocks.

Sent from my Samsung Galaxy Note 3 with K-9 Mail.

Yeah. I'm not sure though if openSUSE 13.2 prevents users from creating btrfs raid1 volumes entirely, or if it's just an install time limitation.

I know that Fedora's installer won't allow the user to create Btrfs on LVM, and it probably doesn't allow it on md raid either.

Chris Murphy

With BTRFS you can make it RAID-1 afterwards. The possibility of data loss during system install usually isn't something you are concerned about so this shouldn't be a problem.

For LVM that's reasonable, for MD-RAID that would be a bug IMHO.

True. But that's just a lack of performance tuning in the current code, it will be fixed at some future time.

My Main Blog         http://etbe.coker.com.au/
My Documents Blog    http://doc.coker.com.au/

I was able to create btrfs raid1 volumes under lvm, but installer failed at installing Grub2.

Don't know if it is installer limitation and decided to use btrfs' native raid1, at least for root. Will see whether to use it for /home as well or use XFS as suggested by SUSE.

Sincerely,
Gour

In the material world, one who is unaffected by whatever good 
or evil he may obtain, neither praising it nor despising it, 
is firmly fixed in perfect knowledge.

-- 
The humble sages, by virtue of true knowledge, see with equal 
vision a learned and gentle brāhmana, a cow, an elephant, a dog 
and a dog-eater.

http://www.atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810

Hmm, this is very interesting and valuable info. Thank you.

Sincerely,
Gour

Many, many births both you and I have passed. I can remember 
all of them, but you cannot, O subduer of the enemy!

-- 
One is understood to be in full knowledge whose every endeavor 
is devoid of desire for sense gratification. He is said by sages 
to be a worker for whom the reactions of work have been burned 
up by the fire of perfect knowledge.

http://www.atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810

Don't get too excited. If the data disagrees in either system because of partial starts (e.g. one mount /dev/sda1 is hot, the next mount /dev/sdb1 is hot) leaving both disks with equally valid but disagreeing data, "magical guessing" _will_ ensue.

In the BTRFS case the system will guess based on generation numbers.

In the MDADM case the system will guess based on the write intent bitmaps.

In both cases, you are _way_ better off invalidating any missing array elements if you find yourself reaching a write-available event with said missing elements.

BTRFS checksums really help when a write goes to both devices (e.g. /dev/sda1 and /dev/sdb1) and the disk subsystem silently fails one of the necessary writes, creating a disagreement between the checksum and the data block on the drive in question.

It's an outstanding feature, but it doesn't protect you from weak maintenance after a partial start.

Pros:

* mdadm RAID has much better read balancing;
Btrfs reads are satisfied from what's in effect a random drive (PID-based balancing of threads to drives), mdadm reads from the less-loaded drive. Also mdadm has a way to specify some RAID1 array members as to be never used for reads if at all possible ("write-mostly"), which helps in RAID1 of HDD and SSD.

* mdadm RAID has much better write submission;
In my experience [1] Btrfs RAID1 on heavy write operations first writes to one drive, then to another. The whole process takes up to 2x longer than with a single drive. On the other hand mdadm writes to both drives simultaneously.

[1] https://www.mail-archive.com/[email protected]/msg34103.html

Con:

* You only get the ability to recover from a checksum failure with Btrfs RAID1, not with mdadm RAID1 (see Russell's reply).

With respect,
Roman

[...]

For the reasons you mentioned I'll keep my root under btrfs' native raid and do some more thinking in regard to /home and possibility to use XFS for it.

Sincerely,
Gour

Abandoning all attachment to the results of his activities, 
ever satisfied and independent, he performs no fruitive action, 
although engaged in all kinds of undertakings.