Skip to contentSkip to conversation

Confining scrub to a subvolume

DISCUSSION Sree Harsha Totakura 4 participants 10 messages Dec 30, 2015 – Jan 6, 2016 Btrfs file system

lore
Share & tools
lore
#0 lore raw

Hi,

Is it possible to confine scrubbing to a subvolume instead of the whole file system?

The problem I am facing is that I have a 5 TB btrfs FS. On it I have created subvolumes for weekly backups, personal photos, music, and documents. Obviously, I am more concerned about my photos and documents than my backups and music. Therefore, I would like to scrub the photos and documents subvolumes more often than the backups subvolume. Would this be possible with the current tools?

Regards,
Sree

#1 lore raw
Quoted 2 lines · Sree Harsha Totakura

Is it possible to confine scrubbing to a subvolume instead of the whole file system?

No. Srub reads the blocks from devices (without knowing which files own them) and compares them to the stored checksums.

Quoted 3 lines

[...] Therefore, I would like to scrub the photos
and documents subvolumes more often than the backups subvolume. Would this be possible with the current tools?

The closest would be to read the files and look for any reported errors.

#2 lore raw

David Sterba posted on Wed, 30 Dec 2015 18:39:49 +0100 as excerpted:

Quoted 6 lines · Sree Harsha Totakura
Quoted 2 lines

Is it possible to confine scrubbing to a subvolume instead of the whole file system?

No. Srub reads the blocks from devices (without knowing which files own them) and compares them to the stored checksums.

Of course if like me you prefer not to have all your data eggs in one filesystem basket and have used partitions (or LVM) and multiple independent btrfs, in which case you scrub the filesystem you want, and don't worry about the others. =:^)

It definitely helps with maintenance time -- on SSDs with all under 50 GiB partitions, scrub times per btrfs are typically under a minute, and btrfs check and balance times are similarly short. Plus, arranging to have additional partitions exactly the same size to use as backups works pretty nicely as well. =:^) OTOH, people routinely report days for multi-terabyte btrfs maintenance commands on spinning rust. =:^(

Tho I do still have my media partition, along with backups, on reiserfs on spinning rust. I should think about switching that over one of these days...

Quoted 5 lines
Quoted 3 lines

[...] Therefore, I would like to scrub the photos and documents subvolumes more often than the backups subvolume. Would this be possible with the current tools?

The closest would be to read the files and look for any reported errors.

That should work. Cat the files to /dev/null and check dmesg. For single mode it should check the only copy. For raid1/10 or dup, running two checks, ensuring one is even-PID while the other is odd-PID, should work to check both copies, since the read-scheduler assigns copy based on even/odd PID. Errors will show up in dmesg, as well as cat's STDERR.

Pretty clever thinking there. =:^)

Signature · 3 lines
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman
#8 lore raw
In reply to David Sterba
Quoted 2 lines · David Sterba

The closest would be to read the files and look for any reported errors.

Doesn't that fail for any multi-device setup, in which case btrfs reads the blocks only from one device, and if that verifies, doesn't check the other?

Cheers,
Chris.

#3 lore raw
In reply to Duncan
Quoted 8 lines · Duncan

That should work.  Cat the files to /dev/null and check dmesg.  For single mode it should check the only copy.  For raid1/10 or dup, running
two checks, ensuring one is even-PID while the other is odd-PID, should
work to check both copies, since the read-scheduler assigns copy based on
even/odd PID.  Errors will show up in dmesg, as well as cat's STDERR.

That doesn't seem very reliable to me, to be honest... plus it wouldn't work in any RAID56 or dupN (with n!=2) case, when that gets sooner or later implemented.

Also, I'd kinda guess (or better said: hope) that the kernel's cache would destroy these efforts, at least when the two reads happen mostly in parallel.

Cheers,
Chris.

#4 lore raw

Christoph Anton Mitterer posted on Wed, 30 Dec 2015 20:28:00 +0100 as excerpted:

Quoted 11 lines · Duncan
Quoted 6 lines

That should work.  Cat the files to /dev/null and check dmesg.  For single mode it should check the only copy.  For raid1/10 or dup, running two checks, ensuring one is even-PID while the other is odd-PID, should work to check both copies, since the read-scheduler assigns copy based on even/odd PID.  Errors will show up in dmesg, as well as cat's STDERR.

That doesn't seem very reliable to me, to be honest... plus it wouldn't work in any RAID56 or dupN (with n!=2) case, when that gets sooner or later implemented.

Well, yes, but right now except on raid56... and there's a good chance it'll work for a year at least, as I've seen no first-patches yet to implement n-way (which I'm sure looking forward to), after which perhaps he'll have implemented the multi-btrfs on partitions or lvm thing that I actually prefer, myself.

Meanwhile, it's a pretty clever solution, I think. =:^)

Quoted 3 lines

Also, I'd kinda guess (or better said: hope) that the kernel's cache would destroy these efforts, at least when the two reads happen mostly in parallel.

I was thinking run them in parallel, but you're right, you'd have to run them serially and dumpcache between runs.

Signature · 3 lines
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman
#5 lore raw
Quoted 1 line · Duncan

Meanwhile, it's a pretty clever solution, I think. =:^)

Well the problem with such workaround-solutions is... end-users get used to it, rely on it, and suddenly they don't work anymore (which the user wouldn't probably notice, though).

Quoted 3 lines

I was thinking run them in parallel, but you're right, you'd have to run
them serially and dumpcache between runs.

And, you'd need to clear the cache, at least for filesystems < memory but even if not, I wouldn't be 100% that one can safely do without clearing the cache.

Cheers,
Chris.

#6 lore raw
In reply to Duncan
Quoted 6 lines · Duncan

David Sterba posted on Wed, 30 Dec 2015 18:39:49 +0100 as excerpted:

Quoted 4 lines · Sree Harsha Totakura
Quoted 2 lines

Is it possible to confine scrubbing to a subvolume instead of the whole file system?

Quoted 7 lines
Quoted 2 lines

No. Srub reads the blocks from devices (without knowing which files own them) and compares them to the stored checksums.

Of course if like me you prefer not to have all your data eggs in one filesystem basket and have used partitions (or LVM) and multiple independent btrfs, in which case you scrub the filesystem you want, and don't worry about the others. =:^)

I considered it, but after reading somewhere (couldn't find the source) that having a single btrfs could be beneficial, I decided not to. Clearly, it doesn't seem to be true in this case.

Thank you all for your comments and suggestions. I will be using different btrfs file systems based.

Regards,
Sree Harsha Totakura

#9 lore raw
In reply to Christoph Anton Mitterer
Quoted 6 lines · Christoph Anton Mitterer
Quoted 2 lines · David Sterba

The closest would be to read the files and look for any reported errors.

Doesn't that fail for any multi-device setup, in which case btrfs reads the blocks only from one device, and if that verifies, doesn't check the other?

That's right, it's not equivalent to the scrub.

#7 lore raw
In reply to Sree Harsha Totakura

Sree Harsha Totakura posted on Mon, 04 Jan 2016 12:01:58 +0100 as excerpted:

Quoted 10 lines · Duncan
Quoted 9 lines

David Sterba posted on Wed, 30 Dec 2015 18:39:49 +0100 as excerpted:

Quoted 6 lines · Sree Harsha Totakura
Quoted 2 lines

Is it possible to confine scrubbing to a subvolume instead of the whole file system?

No. Srub reads the blocks from devices (without knowing which files own them) and compares them to the stored checksums.

Quoted 8 lines
Quoted 4 lines

Of course if like me you prefer not to have all your data eggs in one filesystem basket and have used partitions (or LVM) and multiple independent btrfs, in which case you scrub the filesystem you want, and don't worry about the others. =:^)

I considered it, but after reading somewhere (couldn't find the source) that having a single btrfs could be beneficial, I decided not to. Clearly, it doesn't seem to be true in this case.

It depends very much on your viewpoint and use-case.

Arguably, btrfs should /eventually/ provide a more flexible alternative to lvm as a volume manager, letting you do subvolumes, restrict them if desired to some maximum size via quotas (which remain buggy and unreliable on btrfs ATM so don't try to use them for that yet), and let you "magically" add and remove devices from a single btrfs storage pool, changing quota settings as desired, without the hassle of managing individual partitions and multiple filesystems.

But IMO at least, and I'd guess in the opinion of most on the list, btrfs at its present "still stabilizing, not yet fully stable and mature" status, remains at least partially unsuited to that. Besides general stability, some options, including quotas, simply don't work reliably yet, and other tools and features have yet to be developed.

But even in the future, when btrfs is stable and many are using these sorts of logical volume management and integrated filesystem features to do everything on a single filesystem, I'm still very likely to consider that a higher risk than I'm willing to take, because it'll still be ultimately putting all those data eggs in the filesystem basket, which if the bottom falls out...

In addition, I actually tried, for instance, big partitioned mdraids, with several filesystems each on their own partition of that mdraid, and I eventually came to the conclusion that at some point, they simply get too big, and maintenance simply takes too long, to be practical for me. When adding a multi-TB device to a big mdraid takes days... I'd much rather have multiple much smaller mdraids, or now, btrfs raids, and perhaps still take days overall to do the same thing if I'm doing it to all of them, but in increments of a few hours each on multiple much smaller capacity ones, rather than a single-shot instance that takes days.

Meanwhile, my current btrfs layout is multiple mostly raid1 btrfs, on a pair of partitioned SSDs, which each partition under 50 GiB, under 100 GiB for the raid1 filesystem, 50 on each device. On that, scrubs normally take, literally, under a minute, full balances well under ten, per filesystem. Sure, to do every single filesystem might still take say a half hour, but most of the time, not all filesystems are even mounted, and most of the time, I only need to scrub or balance perhaps three of them, so while if they were all in one I might do it in say 20 minutes, and if I had to do all of them it might take me 30 because I have to repeatedly type in the command for each one, because I have to do only three of them, it's done in five minutes or less.

That's in addition to the fact that if a filesystem dies, I've only a fraction of the data to btrfs restore or to copy over from backup, because most of it was on other filesystems, many of which weren't even mounted or in some cases (my /) were mounted read-only, so they're just fine and I don't have to btrfs restore or copy back over from backup, for them.

These are lessons I've learned in a quarter century of working with computers, about a decade on MS, a decade and a half later this year, on Linux. They may not always apply to everyone, but I've definitely learned how to spare myself unnecessary pain, as I've learned how they apply to me. =:^)

Signature · 3 lines
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman