Meaning of "no_csum" field when scrubbing with -R option
Hello everyone,
I have a question: What exactly does the value for "no_csum" mean when doing a scrub with the -R option? Example output:
Quoted 1 line
sudo btrfs scrub start -BR /
scrub done for ...
...
csum_errors: 0
verify_errors: 0
no_csum: 70517
csum_discards: 87381
super_errors: 0
...
In the btrfs header, I found the following comment for the "no_csum" field of the btrfs_scrub_progress struct:
"# of 4k data block for which no csum is present, probably the result of data written with nodatasum"
So my question is, why does scrub show a high (i.e. non-zero) value for no_csum? I never enabled nodatasum or a similar option.
Best regards
Sebastian
Sebastian Ochmann posted on Wed, 19 Feb 2014 13:58:17 +0100 as excerpted:
Quoted 2 lines
So my question is, why does scrub show a high (i.e. non-zero) value for
no_csum? I never enabled nodatasum or a similar option.
I've no answer but have wondered that myself. So hopefully you get an answer I can read too. =:^)
Signature · 3 lines
Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Quoted 4 lines · Duncan
Sebastian Ochmann posted on Wed, 19 Feb 2014 13:58:17 +0100 as excerpted:
Quoted 2 lines
So my question is, why does scrub show a high (i.e. non-zero) value for
no_csum? I never enabled nodatasum or a similar option.
Did you enable nodatacow option? if nodatacow option is enabled, data checksums will be also disabled at the same time.
Thanks,
Wang
Quoted 3 lines
I've no answer but have wondered that myself. So hopefully you get an answer I can read too. =:^)
Wang Shilong posted on Thu, 20 Feb 2014 18:51:10 +0800 as excerpted:
Quoted 9 lines · Duncan
Quoted 5 lines
Sebastian Ochmann posted on Wed, 19 Feb 2014 13:58:17 +0100 as excerpted:
Quoted 2 lines
So my question is, why does scrub show a high (i.e. non-zero) value for
no_csum? I never enabled nodatasum or a similar option.Did you enable nodatacow option? if nodatacow option is enabled, data checksums will be also disabled at the same time.
I have not. Everything here is standard checksummed COW, which is why I wondered why I was seeing the no_csum results.
Tho in my case there's few enough I thought it might have been triggered by some other abnormality.
(In particular, I was suspending-to-ram for awhile, and if I left the system suspended too long, the disks wouldn't wake up fast enough on resume and one of the raid1 pair would often drop out, forcing the filesystem to read-only and generally a pretty quick reboot, after which I'd have to do a scrub to sync the raid1 back up. I had guessed the no- csum instances might have been half-written at the suspend, but it bothered me.
That was a couple kernels ago, tho. I decided to quit risking the suspend-to-ram since I was sometimes having to reboot anyway and I did end up with a file or two corrupted (even after scrub) as a result, so I've not had that issue or had any other scrub errors in some months now. I've thought I should try it again and see if it works better now, but I haven't done so yet.)
Signature · 3 lines
Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Re: Meaning of \"no_csum\" field when scrubbing with -R option
In reply to Wang ShilongHello,
Quoted 8 lines
Sebastian Ochmann posted on Wed, 19 Feb 2014 13:58:17 +0100 as excerpted:
So my question is, why does scrub show a high (i.e. non-zero) value for
no_csum? I never enabled nodatasum or a similar option.Did you enable nodatacow option? if nodatacow option is enabled, data checksums will be also disabled at the same time.
No, never, not even on single files. Some additional info: The filesystem is only a few weeks old (even though I see similar results on an older filesystem as well), it's my root filesystem, and as mount options I use "rw,noatime,ssd,discard,space_cache" (it's on a SSD). Kernel version is 3.12.9.
Best regards,
Sebastian
Re: Meaning of \"no_csum\" field when scrubbing with -R option
Quoted 12 lines · Sebastian Ochmann
Hello,
Quoted 10 lines
Sebastian Ochmann posted on Wed, 19 Feb 2014 13:58:17 +0100 asexcerpted:
So my question is, why does scrub show a high (i.e. non-zero)
value for
no_csum? I never enabled nodatasum or a similar option.Did you enable nodatacow option? if nodatacow option is enabled, data checksums will be also disabled at the same time.
[SNIP]
So i have found why we have such strange things from debugging, you can have a try as the following steps:
# mkfs.btrfs -f /dev/sda9 # mount /dev/sda9 /mnt # btrfs scrub start -BR /mnt
scrub done for 02dd3326-959f-4602-9baa-aa9ed99ac2e5
scrub started at Thu Feb 20 20:31:46 2014 and finished after 0 seconds
data_extents_scrubbed: 1
tree_extents_scrubbed: 16
data_bytes_scrubbed: 65536
tree_bytes_scrubbed: 262144
read_errors: 0
csum_errors: 0
verify_errors: 0
no_csum: 16 ---------------------->not equal 0 for a newly mkfs.
csum_discards: 0
super_errors: 0
malloc_errors: 0
uncorrectable_errors: 0
unverified_errors: 0
corrected_errors: 0
last_physical: 467140608# btrfs-debug-tree /dev/sda9
By debuging tree, we found there is a EXTENT_ITEM in extent tree for newly mkfs filesystem which we have written anything yet.
item 2 key (12582912 EXTENT_ITEM 65536) itemoff 16182 itemsize 53
extent refs 1 gen 5 flags 1
extent data backref root 1 objectid 256 offset 0 count 1
item 3 key (12582912 BLOCK_GROUP_ITEM 8388608) itemoff 16158 itemsize 24
At the same time, Let's see Csum tree debug output:
checksum tree key (CSUM_TREE ROOT_ITEM 0)
leaf 29425664 items 0 free space 16283 generation 4 owner 7 fs uuid 02dd3326-959f-4602-9baa-aa9ed99ac2e5
So there is not corresponding checksum item for that DATA extent item.. This can explain why scrub output no_sum count!
For reasons, It may be reserved data space without checksum for other purpose! So if this is true, i don't think this is harm for common users unless it is designed unexpectedly!
Thanks,
Wang
Quoted 7 lines
Signature · 4 lines
-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Meaning of \"no_csum\" field when scrubbing with -R option
Quoted 10 lines · Wang Shilong
Quoted 9 lines · Sebastian Ochmann
Hello,
Quoted 7 lines
Sebastian Ochmann posted on Wed, 19 Feb 2014 13:58:17 +0100 asexcerpted:
So my question is, why does scrub show a high (i.e.
non-zero) value for
no_csum? I never enabled nodatasum or a similar option.
This should be related to btrfs free space cache, it is designed as nocow without checksums.
Thanks,
Wang
Quoted 69 lines
Quoted 3 lines
Quoted 3 lines
Did you enable nodatacow option? if nodatacow option is enabled, data checksums will be also disabled at the same time.
[SNIP]
So i have found why we have such strange things from debugging, you can have a try as the following steps:
# mkfs.btrfs -f /dev/sda9 # mount /dev/sda9 /mnt # btrfs scrub start -BR /mntscrub done for 02dd3326-959f-4602-9baa-aa9ed99ac2e5
scrub started at Thu Feb 20 20:31:46 2014 and finished after 0
secondsdata_extents_scrubbed: 1 tree_extents_scrubbed: 16 data_bytes_scrubbed: 65536 tree_bytes_scrubbed: 262144 read_errors: 0 csum_errors: 0 verify_errors: 0 no_csum: 16 ---------------------->not equal 0 for a newly mkfs. csum_discards: 0 super_errors: 0 malloc_errors: 0 uncorrectable_errors: 0 unverified_errors: 0 corrected_errors: 0 last_physical: 467140608# btrfs-debug-tree /dev/sda9By debuging tree, we found there is a
EXTENT_ITEMin extent tree for newly mkfs filesystem which we have written anything yet.item 2 key (12582912
EXTENT_ITEM65536) itemoff 16182 itemsize 53
extent refs 1 gen 5 flags 1
extent data backref root 1 objectid 256 offset 0 count 1
item 3 key (12582912BLOCK_GROUP_ITEM8388608) itemoff 16158 itemsize 24At the same time, Let's see Csum tree debug output:
checksum tree key (
CSUM_TREEROOT_ITEM0)
leaf 29425664 items 0 free space 16283 generation 4 owner 7 fs uuid 02dd3326-959f-4602-9baa-aa9ed99ac2e5So there is not corresponding checksum item for that DATA extent item.. This can explain why scrub output
no_sumcount!For reasons, It may be reserved data space without checksum for other purpose! So if this is true, i don't think this is harm for common users unless it is designed unexpectedly!
Thanks,
WangQuoted 7 lines
Signature · 4 lines
-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.htmlSignature · 3 lines
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Meaning of \"no_csum\" field when scrubbing with -R option
Wang Shilong posted on Fri, 21 Feb 2014 10:30:25 +0800 as excerpted:
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
So my question is, why does scrub show a high (i.e. non-zero) value for
no_csum? I never enabled nodatasum or a similar option.This should be related to btrfs free space cache, it is designed as nocow without checksums.
That's a reasonable explanation. Thanks. =:^)
(And anyway, if the space-cache gets corrupted, there are mount options to clear it, etc, and it's easily rebuilt even if it takes long enough keeping the cache is useful in general, so it's not a huge deal needing checksummed.)
Signature · 3 lines
Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman