What about storing more crc32 in the unused csum size for metadata?

[Old layout]
Btrfs csum size for tree block is 32 bytes, and currently, only the first 4 bytes(sector 0) is used, storing the crc32 for the whole leaf/node.

Takes 4K as example for leafsize.
Sectors:
|---0---|---1---|---2---|---3---|---4---|---5---|---6---|---7---| Sector 0: Csum of the leaf/node (32 ~ 4K)
Sector 1~7: Not used

So, what about restoring more crc32 in other sectors in the following backward-compatible way?

[New layout]
Take leafsize as 4K for example.

Sectors:
|---0---|---1---|---2---|---3---|---4---|---5---|---6---|---7---| Sector 0: Csum of the leaf/node (32~4K)(not changed) Sector 1: Csum of the first eighths of the leaf/node (32~512) Sector 2: Csum of the second eighths of the leaf/node (512~1024)
....
Sector 7: Csum of the last eighths of the leaf/node (3584 ~ 4096)

And due to the fact
crc(sector 0 ~ sector 7) = crc(crc(sector 0), crc(sector 1), ...)

We won't waste too much CPU time, and since we keep the behavior of sector 0, so it is completely backward compatible.

[Advantage]
The advantage usage of the new layout is more accuracy in scrub and btrfsck.
For scrub, even all duplication is broken, there is still chance for scrub to rebuild the block if corruptions occurs in different sector.

And for btrfsck --repair, if node is corrupted, we don't need to drop all the nodes/leaves belong to the nodes, but only drops nodes/leaves belong to the sector.

Qu Wenruo posted on Mon, 12 Jan 2015 10:32:30 +0800 as excerpted:

OK, I've been up too long and should be sleeping, and it may be that's affecting my logic, but AFAICT either you're missing something obvious, or I am.

There are eight eights, and eight "sectors", but the first one is already used, thus leaving seven available. You have the first eighth (ending at byte 512*1) in sector one (zero-based so the second sector), the second (ending at byte 512*2=1024) in sector two... the last (eighth) eighth (ending at byte 512*8=4096) in sector seven.

Of eighths 3-7 in that ..., which eighth do you skip csumming, or is one of them a full quarter instead of an eighth?

Of course if we could make them sevenths... But making it four quarters instead of 7/8 is perhaps more reasonable, with the remaining three sectors remaining unused, and if only one fails csum we still save 75% of the leaf/node, instead of none of it, currently.

But maybe it makes complete sense as-is and I should be sleeping instead of trying to make sense of what is clearly not making sense to me at this point...

Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-------- Original Message --------
Subject: Re: What about storing more crc32 in the unused csum size for metadata?
From: Duncan <[email protected]>
To: <[email protected]>
Date: 2015年01月12日 12:52

Oh, my fault, sector 1 should contains csum for the second eighth of the leaf/node... Since the first eighth csum can be calculated vice verse.

Thanks,
Qu

Qu Wenruo posted on Mon, 12 Jan 2015 12:56:14 +0800 as excerpted:

Thanks indeed.

/Now/ it makes sense. Skip the one and reverse-calculate it. (Which you had actually implied and I guess it doesn't matter which one as long as it's /only/ one, but given my general unfamiliarity with crc at a practical level and impaired logic state, and your hiding it in the ... gap, I needed a bit more help to make the leap. =:^)

Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman