Skip to contentSkip to conversation

Meaning of "no_csum" field when scrubbing with -R option

DISCUSSION Sebastian Ochmann 3 participants 8 messages Feb 19, 2014 – Feb 21, 2014 Btrfs file system

lore
Share & tools
lore
#0 lore raw

Hello everyone,

I have a question: What exactly does the value for "no_csum" mean when doing a scrub with the -R option? Example output:

Quoted 1 line

sudo btrfs scrub start -BR /

scrub done for ...
...
csum_errors: 0
verify_errors: 0
no_csum: 70517
csum_discards: 87381
super_errors: 0
...

In the btrfs header, I found the following comment for the "no_csum" field of the btrfs_scrub_progress struct:

"# of 4k data block for which no csum is present, probably the result of data written with nodatasum"

So my question is, why does scrub show a high (i.e. non-zero) value for no_csum? I never enabled nodatasum or a similar option.

Best regards
Sebastian

#1 lore raw

Sebastian Ochmann posted on Wed, 19 Feb 2014 13:58:17 +0100 as excerpted:

Quoted 2 lines

So my question is, why does scrub show a high (i.e. non-zero) value for no_csum? I never enabled nodatasum or a similar option.

I've no answer but have wondered that myself. So hopefully you get an answer I can read too. =:^)

Signature · 3 lines
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman
#2 lore raw
Quoted 4 lines · Duncan

Sebastian Ochmann posted on Wed, 19 Feb 2014 13:58:17 +0100 as excerpted:

Quoted 2 lines

So my question is, why does scrub show a high (i.e. non-zero) value for no_csum? I never enabled nodatasum or a similar option.

Did you enable nodatacow option? if nodatacow option is enabled, data checksums will be also disabled at the same time.

Thanks,
Wang

Quoted 3 lines

I've no answer but have wondered that myself. So hopefully you get an answer I can read too. =:^)

#3 lore raw

Wang Shilong posted on Thu, 20 Feb 2014 18:51:10 +0800 as excerpted:

Quoted 9 lines · Duncan
Quoted 5 lines

Sebastian Ochmann posted on Wed, 19 Feb 2014 13:58:17 +0100 as excerpted:

Quoted 2 lines

So my question is, why does scrub show a high (i.e. non-zero) value for no_csum? I never enabled nodatasum or a similar option.

Did you enable nodatacow option? if nodatacow option is enabled, data checksums will be also disabled at the same time.

I have not. Everything here is standard checksummed COW, which is why I wondered why I was seeing the no_csum results.

Tho in my case there's few enough I thought it might have been triggered by some other abnormality.

(In particular, I was suspending-to-ram for awhile, and if I left the system suspended too long, the disks wouldn't wake up fast enough on resume and one of the raid1 pair would often drop out, forcing the filesystem to read-only and generally a pretty quick reboot, after which I'd have to do a scrub to sync the raid1 back up. I had guessed the no- csum instances might have been half-written at the suspend, but it bothered me.

That was a couple kernels ago, tho. I decided to quit risking the suspend-to-ram since I was sometimes having to reboot anyway and I did end up with a file or two corrupted (even after scrub) as a result, so I've not had that issue or had any other scrub errors in some months now. I've thought I should try it again and see if it works better now, but I haven't done so yet.)

Signature · 3 lines
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman
#4 lore raw

Re: Meaning of \"no_csum\" field when scrubbing with -R option

In reply to Wang Shilong

Hello,

Quoted 8 lines

Sebastian Ochmann posted on Wed, 19 Feb 2014 13:58:17 +0100 as excerpted:

So my question is, why does scrub show a high (i.e. non-zero) value for no_csum? I never enabled nodatasum or a similar option.

Did you enable nodatacow option? if nodatacow option is enabled, data checksums will be also disabled at the same time.

No, never, not even on single files. Some additional info: The filesystem is only a few weeks old (even though I see similar results on an older filesystem as well), it's my root filesystem, and as mount options I use "rw,noatime,ssd,discard,space_cache" (it's on a SSD). Kernel version is 3.12.9.

Best regards,
Sebastian

#5 lore raw

Re: Meaning of \"no_csum\" field when scrubbing with -R option

Quoted 12 lines · Sebastian Ochmann

Hello,

Quoted 10 lines
    Sebastian Ochmann posted on Wed, 19 Feb 2014 13:58:17 +0100 as 

excerpted:

So my question is, why does scrub show a high (i.e. non-zero)
value for
no_csum? I never enabled nodatasum or a similar option.

Did you enable nodatacow option? if nodatacow option is enabled, data checksums will be also disabled at the same time.

[SNIP]

So i have found why we have such strange things from debugging, you can have a try as the following steps:

# mkfs.btrfs -f /dev/sda9
# mount /dev/sda9 /mnt
# btrfs scrub start -BR /mnt

scrub done for 02dd3326-959f-4602-9baa-aa9ed99ac2e5

    scrub started at Thu Feb 20 20:31:46 2014 and finished after 0 seconds
    data_extents_scrubbed: 1
    tree_extents_scrubbed: 16
    data_bytes_scrubbed: 65536
    tree_bytes_scrubbed: 262144
    read_errors: 0
    csum_errors: 0
    verify_errors: 0
    no_csum: 16 ---------------------->not equal 0 for a newly mkfs.
    csum_discards: 0
    super_errors: 0
    malloc_errors: 0
    uncorrectable_errors: 0
    unverified_errors: 0
    corrected_errors: 0
    last_physical: 467140608
# btrfs-debug-tree /dev/sda9

By debuging tree, we found there is a EXTENT_ITEM in extent tree for newly mkfs filesystem which we have written anything yet.

item 2 key (12582912 EXTENT_ITEM 65536) itemoff 16182 itemsize 53
extent refs 1 gen 5 flags 1
extent data backref root 1 objectid 256 offset 0 count 1
item 3 key (12582912 BLOCK_GROUP_ITEM 8388608) itemoff 16158 itemsize 24

At the same time, Let's see Csum tree debug output:

checksum tree key (CSUM_TREE ROOT_ITEM 0)
leaf 29425664 items 0 free space 16283 generation 4 owner 7 fs uuid 02dd3326-959f-4602-9baa-aa9ed99ac2e5

So there is not corresponding checksum item for that DATA extent item.. This can explain why scrub output no_sum count!

For reasons, It may be reserved data space without checksum for other purpose! So if this is true, i don't think this is harm for common users unless it is designed unexpectedly!

Thanks,
Wang

Quoted 7 lines
Signature · 4 lines
-- 
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#6 lore raw

Re: Meaning of \"no_csum\" field when scrubbing with -R option

Quoted 10 lines · Wang Shilong
Quoted 9 lines · Sebastian Ochmann

Hello,

Quoted 7 lines
    Sebastian Ochmann posted on Wed, 19 Feb 2014 13:58:17 +0100 as 

excerpted:

So my question is, why does scrub show a high (i.e.
non-zero) value for
no_csum? I never enabled nodatasum or a similar option.

This should be related to btrfs free space cache, it is designed as nocow without checksums.

Thanks,
Wang

Quoted 69 lines
Quoted 3 lines
Quoted 3 lines

Did you enable nodatacow option? if nodatacow option is enabled, data checksums will be also disabled at the same time.

[SNIP]

So i have found why we have such strange things from debugging, you can have a try as the following steps:

# mkfs.btrfs -f /dev/sda9
# mount /dev/sda9 /mnt
# btrfs scrub start -BR /mnt

scrub done for 02dd3326-959f-4602-9baa-aa9ed99ac2e5
scrub started at Thu Feb 20 20:31:46 2014 and finished after 0
seconds

    data_extents_scrubbed: 1
    tree_extents_scrubbed: 16
    data_bytes_scrubbed: 65536
    tree_bytes_scrubbed: 262144
    read_errors: 0
    csum_errors: 0
    verify_errors: 0
    no_csum: 16 ---------------------->not equal 0 for a newly mkfs.
    csum_discards: 0
    super_errors: 0
    malloc_errors: 0
    uncorrectable_errors: 0
    unverified_errors: 0
    corrected_errors: 0
    last_physical: 467140608
# btrfs-debug-tree /dev/sda9

By debuging tree, we found there is a EXTENT_ITEM in extent tree for newly mkfs filesystem which we have written anything yet.

item 2 key (12582912 EXTENT_ITEM 65536) itemoff 16182 itemsize 53
extent refs 1 gen 5 flags 1
extent data backref root 1 objectid 256 offset 0 count 1
item 3 key (12582912 BLOCK_GROUP_ITEM 8388608) itemoff 16158 itemsize 24

At the same time, Let's see Csum tree debug output:

checksum tree key (CSUM_TREE ROOT_ITEM 0)
leaf 29425664 items 0 free space 16283 generation 4 owner 7 fs uuid 02dd3326-959f-4602-9baa-aa9ed99ac2e5

So there is not corresponding checksum item for that DATA extent item.. This can explain why scrub output no_sum count!

For reasons, It may be reserved data space without checksum for other purpose! So if this is true, i don't think this is harm for common users unless it is designed unexpectedly!

Thanks,
Wang

Quoted 7 lines
Signature · 4 lines
-- 
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 3 lines
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#7 lore raw

Re: Meaning of \"no_csum\" field when scrubbing with -R option

Wang Shilong posted on Fri, 21 Feb 2014 10:30:25 +0800 as excerpted:

Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

So my question is, why does scrub show a high (i.e. non-zero) value for no_csum? I never enabled nodatasum or a similar option.

This should be related to btrfs free space cache, it is designed as nocow without checksums.

That's a reasonable explanation. Thanks. =:^)

(And anyway, if the space-cache gets corrupted, there are mount options to clear it, etc, and it's easily rebuilt even if it takes long enough keeping the cache is useful in general, so it's not a huge deal needing checksummed.)

Signature · 3 lines
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman