Does data checksumming remain for files with No_COW file attribute?
There is an unanswered question at stackexchange:
http://unix.stackexchange.com/q/303130/41572
Assume that we don't invoke mount options like nodatacow and nodatasum - operate only at file attributes level, which are controlled with lsattr/chattr. So does setting No_COW on a file (chattr +C) imply disabling data checksumming on it? The same way as mount options behave (nodatacow implies nodatasum)? Moreover may it disable checksumming only for newly written extents and keep for reading existing ones?
Quoted 4 lines · Alexander Tomokhov
There is an unanswered question at stackexchange:
http://unix.stackexchange.com/q/303130/41572Assume that we don't invoke mount options like nodatacow and nodatasum - operate only at file attributes level, which are controlled with lsattr/chattr. So does setting
No_COWon a file (chattr +C) imply disabling data checksumming on it? The same way as mount options behave (nodatacow implies nodatasum)? Moreover may it disable checksumming only for newly written extents and keep for reading existing ones?
Yes, it does imply disabling checksums.
If you have extents in the file when +C is set, then the flag is
ignored (even for new extents, as far as I know): you can only usefully set +C on a file with no extents.
So, to use +C, you either need to touch the file first, then set +C
on it, then open the file and write to it, or you need to have +C set on the parent directory.
Hugo.
Quoted 4 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 4 lines
Hugo Mills | Comic Sans goes into a bar, and the barman says, "We hugo@... carfax.org.uk | don't serve your type here." http://carfax.org.uk/ | PGP: E2AB1DE4 |
Ok, so data checksumming does not remain for newly created empty files with No_COW attribute. I think it's an important trait of Btrfs behavior and should be added to wiki. So that users are informed that disabling CoW on a per-file basis also loses checksum correctness of such file.
24.09.2016, 15:37, "Hugo Mills" <[email protected]>:
Quoted 28 lines · Alexander Tomokhov
Quoted 4 lines
There is an unanswered question at stackexchange:
http://unix.stackexchange.com/q/303130/41572Assume that we don't invoke mount options like nodatacow and nodatasum - operate only at file attributes level, which are controlled with lsattr/chattr. So does setting
No_COWon a file (chattr +C) imply disabling data checksumming on it? The same way as mount options behave (nodatacow implies nodatasum)? Moreover may it disable checksumming only for newly written extents and keep for reading existing ones?Yes, it does imply disabling checksums.
If you have extents in the file when +C is set, then the flag is ignored (even for new extents, as far as I know): you can only usefully set +C on a file with no extents.
So, to use +C, you either need to touch the file first, then set +C on it, then open the file and write to it, or you need to have +C set on the parent directory.
Hugo.
Quoted 4 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html--
Hugo Mills | Comic Sans goes into a bar, and the barman says, "We hugo@... carfax.org.uk | don't serve your type here."
http://carfax.org.uk/ |
PGP: E2AB1DE4 |
Quoted 4 lines · Alexander Tomokhov
Ok, so data checksumming does not remain for newly created empty files with
No_COWattribute. I think it's an important trait of Btrfs behavior and should be added to wiki. So that users are informed that disabling CoW on a per-file basis also loses checksum correctness of such file.
Actually, it disables pretty much all btrfs features except for... CoW.
You lose:
* checksums
* compression
* safety against power loss (torn writes, etc)
* transactions (not that anyone uses them...)
* etc
But, CoW is still there. Try it: make a subvolume, create a FS_NO_COW file (preferably one big enough), snapshot the subvolume, filefrag -v both copies. Write to one of them, changing only a part of file. Wait for writeout, filefrag -v them again.
Signature · 1 line
An imaginary friend squared is a real enemy.
Adam Borowski posted on Sun, 25 Sep 2016 01:50:14 +0200 as excerpted:
Quoted 16 lines · Alexander Tomokhov
Quoted 5 lines
Ok, so data checksumming does not remain for newly created empty files with
No_COWattribute. I think it's an important trait of Btrfs behavior and should be added to wiki. So that users are informed that disabling CoW on a per-file basis also loses checksum correctness of such file.Actually, it disables pretty much all btrfs features except for... CoW.
You lose:
* checksums
* compression
* safety against power loss (torn writes, etc)
* transactions (not that anyone uses them...)
* etc
Quoted 6 lines
But, CoW is still there.
Try it: make a subvolume, create a
FS_NO_COWfile (preferably one big enough), snapshot the subvolume, filefrag -v both copies. Write to one of them, changing only a part of file. Wait for writeout, filefrag -v them again.
That's because snapshots depend on COW. If you don't snapshot the file (or otherwise create additional reflinks to it, using cp --reflink=always, for instance), it'll be NOCOW. But because snapshots (and other forms of multiple reflink) depend on COW, taking a snapshot (or otherwise multi- reflinking) and then writing to one copy forces what has been referred to on this list as COW1, a single COW to break the multi-reflink.
However, COW1 doesn't change the NOCOW attribute, and further writes to the same block of the NOCOW file will overwrite the now-current block, instead of COWING it... until the next snapshot (or another multi-reflink operation) locks it too in place, of course, after which another COW1 will be required.
Which means otherwise NOCOW files that are both repeatedly overwritten and repeatedly snapshotted, with both happening at about the same rate or snapshots happening more frequently than rewrites, will tend to fragment almost as fast as if they hadn't been set NOCOW in the first place.
So NOCOW still has an effect -- as long as rewrites are coming in more frequently than snapshots. However, if the file is repeatedly snapshotted at the same or faster rate than it is rewritten, all those COW1s due to the repeated snapshotting will pretty effectively nullify the NOCOW setting, even if it otherwise remains valid.
The other alternative, of course, is to avoid snapshotting your NOCOW files (which of course means losing send/receive, since send requires a read-only snapshot). You can choose one or the other, but can't have both without one, NOCOW, yielding to the other.
Signature · 3 lines
Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Quoted 19 lines · Duncan
Adam Borowski posted on Sun, 25 Sep 2016 01:50:14 +0200 as excerpted:
Quoted 10 lines
Actually, it disables pretty much all btrfs features except for... CoW.
You lose:
* checksums
* compression
* safety against power loss (torn writes, etc)
* transactions (not that anyone uses them...)
* etcBut, CoW is still there.
That's because snapshots depend on COW. If you don't snapshot the file (or otherwise create additional reflinks to it, using cp --reflink=always, for instance), it'll be NOCOW. But because snapshots (and other forms of multiple reflink) depend on COW, taking a snapshot (or otherwise multi- reflinking) and then writing to one copy forces what has been referred to on this list as COW1, a single COW to break the multi-reflink.
Quoted 5 lines
However, COW1 doesn't change the NOCOW attribute, and further writes to the same block of the NOCOW file will overwrite the now-current block, instead of COWING it... until the next snapshot (or another multi-reflink operation) locks it too in place, of course, after which another COW1 will be required.
There is no CoW unless you have more than one ref to the extent, as there is no copying involved (other than when you write less than a whole block[1], but that's the case for any filesystem). Depending on FS_NOCOW_FL, you either overwrite in place or write elsewhere then switch refs, but at no point there is more than one live version. You could call the latter "move on write", perhaps. On the other hand, with >1 reference, there's always CoW.
[1]. The exact definition of "block" here varies.
Signature · 1 line
An imaginary friend squared is a real enemy.
Quoted 1 line · On Sat, 24 Sep 2016 15:26:14 +0300 Alexander Tomokhov <[email protected]> wrote:
does setting
No_COWon a file (chattr +C) imply disabling data checksumming on it?
Yes. IIRC the reasoning was that it's more difficult to track checksums of data which is being overwritten in-place (as opposed to CoW).
Quoted 1 line
may it disable checksumming only for newly written extents and keep for reading existing ones?
You can't apply chattr +C to any files of non-zero length, so by definition there won't be any pre-existing checksummed extents in that file.
Signature · 2 lines
With respect, Roman
Quoted 7 lines · Roman Mamedov
Quoted 1 line · On Sat, 24 Sep 2016 15:26:14 +0300 Alexander Tomokhov <[email protected]> wrote:
does setting
No_COWon a file (chattr +C) imply disabling data checksumming on it?Yes. IIRC the reasoning was that it's more difficult to track checksums of data which is being overwritten in-place (as opposed to CoW).
It's because you can't update the data and the checksum atomically
-- at some point in the writing process, they must be inconsistent. This is considered a Bad Thing.
Hugo.
Quoted 8 lines
Quoted 1 line
may it disable checksumming only for newly written extents and keep for reading existing ones?
You can't apply chattr +C to any files of non-zero length, so by definition there won't be any pre-existing checksummed extents in that file.
Signature · 2 lines
With respect, Roman
Signature · 4 lines
Hugo Mills | Comic Sans goes into a bar, and the barman says, "We hugo@... carfax.org.uk | don't serve your type here." http://carfax.org.uk/ | PGP: E2AB1DE4 |
Quoted 3 lines · Hugo Mills
It's because you can't update the data and the checksum atomically -- at some point in the writing process, they must be inconsistent. This is considered a Bad Thing.
It's not worse at all than simply not cheksuming... in both cases you have no guarantee whether the data is bogus or not, but in with csums, you can at least tell - only at the little cost, of false positives when the data was fully and correctly written, but the csums not.
Cheers.
Quoted 10 lines · Christoph Anton Mitterer
Quoted 3 lines · Hugo Mills
It's because you can't update the data and the checksum atomically
-- at some point in the writing process, they must be inconsistent. This is considered a Bad Thing.It's not worse at all than simply not cheksuming... in both cases you have no guarantee whether the data is bogus or not, but in with csums, you can at least tell - only at the little cost, of false positives when the data was fully and correctly written, but the csums not.
I think that the bigger cost is the lower performance due to the write of checksums.
Quoted 3 lines
Cheers.
Signature · 2 lines
gpg @keyserver.linux.it: Goffredo Baroncelli <kreijackATinwind.it> Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5
Quoted 2 lines · Goffredo Baroncelli
I think that the bigger cost is the lower performance due to the write of checksums.
Which would make btrfs default mode of CoW+checksumming also unusable...
If for anyone checksumming comes at a too high cost, he can simply use nodatasum.
Cheers,
Chris.
Quoted 9 lines · Christoph Anton Mitterer
Quoted 3 lines · Hugo Mills
It's because you can't update the data and the checksum atomically
-- at some point in the writing process, they must be inconsistent. This is considered a Bad Thing.It's not worse at all than simply not cheksuming... in both cases you have no guarantee whether the data is bogus or not, but in with csums, you can at least tell - only at the little cost, of false positives when the data was fully and correctly written, but the csums not.
Except that without checksums, you don't lose all the data in that block when torn write happens, with checksums without COW, you do. Also, IIUC, due to how BTRFS implements checksums, you have a bad block for the entire duration of the write, because the stored checksum is for the old block, not the current one. Whether or not that matters is of course usage dependent, but it's still worth noting. Personally, I'd rather be able to read the file and see that it's corrupted instead of getting -EIO trying to do so (I also would rather we had something other than -EIO to return when checksum verification fails, but that's a separate issue).
Quoted 3 lines · Roman Mamedov
Yes. IIRC the reasoning was that it's more difficult to track checksums of
data which is being overwritten in-place (as opposed to CoW).
AFAIU it wouldn't be more difficult, since the meta-data itself is still subject to CoW...
There's just no guarantee in the case of a crash, that checksum and data match (which is IMO however a small price to pay - especially as the data is in that case and without CoW anyway not guaranteed to be valid - compared to all sorts of other silent corruptions against which checksums protect... not to talk about the ability to actually repair files in case of RAID inconsistencies.
Quoted 3 lines
You can't apply chattr +C to any files of non-zero length, so by definition
there won't be any pre-existing checksummed extents in that file.
Speaking of which,... can't one modify chattr properly so that it gives an error message and $? != 0 in that case?
Cheers,
Chris.
Quoted 6 lines · Christoph Anton Mitterer
Quoted 2 lines · Roman Mamedov
Yes. IIRC the reasoning was that it's more difficult to track checksums of data which is being overwritten in-place (as opposed to CoW).
AFAIU it wouldn't be more difficult, since the meta-data itself is still subject to CoW...
Quoted 6 lines
There's just no guarantee in the case of a crash, that checksum and data match (which is IMO however a small price to pay - especially as the data is in that case and without CoW anyway not guaranteed to be valid - compared to all sorts of other silent corruptions against which checksums protect... not to talk about the ability to actually repair files in case of RAID inconsistencies.
This would require teaching btrfs that, in some cases, a csum mismatch is no big thing and it can legitimately return junk data (like most other filesystems) rather than complaining. Same for scrub and btrfs check. For now, there's a nice simple rule: bad csum is bad.
Quoted 6 lines
Quoted 3 lines
You can't apply chattr +C to any files of non-zero length, so by definition there won't be any pre-existing checksummed extents in that file.
Speaking of which,... can't one modify chattr properly so that it gives an error message and $? != 0 in that case?
chattr has no knowledge of btrfs whatsoever, and has no filesystem-specific code. It comes from e2fsprogs and from its internal naming of flags, I see it believes everything to be ext* -- but it doesn't actually have anything specific to ext*, either.
Thus, any other filesystem can make use of +C (FS_NOCOW_FL), and there's no reason the "+C doesn't work on non-empty files" rule needs to hold on btrfs in the future.
Meow!
Signature · 1 line
An imaginary friend squared is a real enemy.
Quoted 5 lines · Adam Borowski
This would require teaching btrfs that, in some cases, a csum mismatch is no
big thing and it can legitimately return junk data (like most other filesystems) rather than complaining. Same for scrub and btrfs check.
Well, I see no point in returning data that is not "guaranteed" to be valid.
In case of a crash, one must assume that the data is bogus, and that's what the program should get.
Quoted 8 lines
chattr has no knowledge of btrfs whatsoever, and has no filesystem- specific
code. It comes from e2fsprogs and from its internal naming of flags, I see
it believes everything to be ext* -- but it doesn't actually have anything
specific to ext*, either.
Quoted 5 lines
Thus, any other filesystem can make use of +C (
FS_NOCOW_FL), and there's no
reason the "+C doesn't work on non-empty files" rule needs to hold on btrfs
in the future.
Well chattr must probably call some btrfs ioctl or so... and that should simply give an error back... and chattr at least print some generic error message.
Or this should be rather replaced with a btrfs specific tool.
Cheers,
Chris.