CRC support for superblock, ag headers, log and btree blocks

Here's an updated crc series for various metadata structures. We're still looking at the magic number in the log recovery case because getting down a buffer type for the other structures wasn't quite as easy as for the btree block. I'll probably look into that again once we're done with all data structures.

FWIW, I note that you haven't done anything with the AGFL yet. I was considering 2 different approaches. The first was to put the CRC for the AGFL into the AGF so that the internals of the AGFL do not change. However, that means we have interesting synchronisation problems with the CRC, so I don't think that will fly.

The other option was to reduce the number of slots in the AGFL by one and use the last slot to hold the CRC. I think that is OK to do; the AGFL can change size with different secotr size support, so I don't see this being a big problem. Nor do I think that we ever fill the AGFL up to it's full 128 slots during any transaction, so reducing the size by one should be safe.

Cheers,

Dave.

Dave Chinner
[email protected]

Yes, my plan was to remove one slot, but use the first one for the crc. Last one might actually be smarter.

On Fri, Sep 26, 2008 at 12:56:13AM +0200, Christoph Hellwig wrote:

| Here's an updated crc series for various metadata structures.  We're still
| looking at the magic number in the log recovery case because getting down
| a buffer type for the other structures wasn't quite as easy as for the
| btree block.  I'll probably look into that again once we're done with all
| data structures.

Just to be clear, crc only applies to the metadata structures listed in the subject line, correct? It wasn't clear to me where you were with the "other structures". Maybe it would be more clear if you could provide a simple table listing the structures and whether or not crc applies (yet).

Bill

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

On Wed, Dec 03, 2008 at 05:44:09AM -0500, Christoph Hellwig wrote:

| On Mon, Dec 01, 2008 at 05:10:05PM -0600, Bill O'Donnell wrote:
| > On Fri, Sep 26, 2008 at 12:56:13AM +0200, Christoph Hellwig wrote:
| > | Here's an updated crc series for various metadata structures.  We're still
| > | looking at the magic number in the log recovery case because getting down
| > | a buffer type for the other structures wasn't quite as easy as for the
| > | btree block.  I'll probably look into that again once we're done with all
| > | data structures.
| > 
| > Just to be clear, crc only applies to the metadata structures listed in the
| > subject line, correct?  It wasn't clear to me where you were with the "other
| > structures".  Maybe it would be more clear if you could provide a simple
| > table listing the structures and whether or not crc applies (yet).
| 
| The patchset sent applies to
| 
| sb
| agi
| agf
| btree_block
| log buffer
| 
| next patchset later this week will add the inode

OK, thanks Christoph-
I'll look for it later in the week.
BTW, there seems to be some aliasing issue with xfs-oss list at least with my mailer. I dunno, but its converting [email protected] into [email protected]. The upshot is that although you cc'd the list in your replies to my posts, those replies aren't getting posted.
Cheers-
Bill

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

On Fri, Sep 26, 2008 at 12:56:13AM +0200, Christoph Hellwig wrote:

| Here's an updated crc series for various metadata structures.  We're still
| looking at the magic number in the log recovery case because getting down
| a buffer type for the other structures wasn't quite as easy as for the
| btree block.  I'll probably look into that again once we're done with all
| data structures.

IIUC, this is the latest crc series, right?
(I had once thought, perhaps mistakenly, that there would be a refreshed patchset subsequent to this one).

Thanks,
Bill

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

On Fri, Sep 26, 2008 at 12:56:13AM +0200, Christoph Hellwig wrote:

| Here's an updated crc series for various metadata structures.  We're still
| looking at the magic number in the log recovery case because getting down
| a buffer type for the other structures wasn't quite as easy as for the
| btree block.  I'll probably look into that again once we're done with all
| data structures.

Christoph-
Yet another question as I revisit this ;) ...
What is to be done with the other tarball on your site ( xfs-cmds-crcs.tgz )? Is there a separate posting on this list that I may have missed earlier?

Bill

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

On Fri, Sep 26, 2008 at 12:56:13AM +0200, Christoph Hellwig wrote:

| Here's an updated crc series for various metadata structures.  We're still
| looking at the magic number in the log recovery case because getting down
| a buffer type for the other structures wasn't quite as easy as for the
| btree block.  I'll probably look into that again once we're done with all
| data structures.

I've carried out cursory build and test of this series. I'm concerned that my testing (and anyone else's testing) has not addressed performance hits being introduced with crc on metadata. What level of testing have you done, and can you recommend some testing that goes beyond the QA suite that we have in place. I did run the btree test harness on that earlier series, but it appeared to me that that was a go/no-go test and unless I'm missing something, I've not seen tests that might help quantify performance hits.

Thanks,
Bill

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs