Skip to contentSkip to conversation

review: hopefully final attr2 corruption fixes

PATCH David Chatterton 1 participant 1 message Dec 15, 2006 XFS filesystem

lore
Share & tools
lore
#0 lore raw

-------- Original Message --------
Subject: RE: attr2
Date: Fri, 15 Dec 2006 16:33:04 +1100
From: Barry Naujok <[email protected]>
To: 'Russell Cattelan' <[email protected]>
CC: 'Timothy Shimmin' <[email protected]>,
<[email protected]>, 'Timothy Shimmin' <[email protected]>,
<[email protected]>, 'David Chinner' <[email protected]>

I think I have it cracked now :)

I've cleaned up a bit more as per Russell's comments.

I also found another case of btree corruption when the last attribute is deleted (an attr2 optimization is to completely remove the attr fork when the last attribute is deleted).

Also, the root btree is not modified anymore when an attribute goes just beyond the space the rest of the code thinks it needs (XFS_BMAP_ROOT_SPACE macro).

Barry.

Quoted 271 lines

-----Original Message-----
From: Russell Cattelan [mailto:[email protected]]
Sent: Friday, 15 December 2006 2:31 PM
To: Barry Naujok
Cc: 'Timothy Shimmin'; [email protected]; 'Timothy Shimmin'; [email protected]; 'David Chinner'
Subject: Re: attr2

Barry Naujok wrote:

sorry just getting time to look at this again

So looking over the latest patch I'm trying to understand what is different from the original patch I sent out.

I guess there is the addition of the calculation of number of btree recs in the case of a the inode being converted to an attr capable format.
I know we had talked about optimizing that space but it leads into the guessing game we were talking about. Will giving more space to attr be better or will leaving more space for btree block be better.

I would lean on the side of attrs are infrequent updates/additions but data blocks are not. So opting for an attr1 half and half split once
the inode goes btree for either fork seems simpler to me. And keeps the code out of the guessing game of which fork is better optimized by having more space.

My style point comment:
The current patch is a bit busy and has more calls to MAX, the maxforkoff is a bit confusing since it contains bytes during the calculation then converted back to the >> 3 value. My orginal patch has the dsize variable to help clarify bytes vs forkoff.
Hmm looking closer I also think there is replicated checks for does the attr fit, the case statement doesn't need to deal with "does the attr fit" since that is handled later. Nothing really wrong with it but I just like the less line of code approach.

Quoted 22 lines
Quoted 12 lines

-----Original Message-----
From: Timothy Shimmin [mailto:[email protected]]
Sent: Friday, 15 December 2006 1:08 PM
To: Barry Naujok
Cc: Russell Cattelan; [email protected]; Timothy Shimmin; [email protected]; David Chinner
Subject: Re: attr2

Quoted 9 lines

Hi Barry,

I don't have the tree locally (can't do an xdiff) but if we have data btree and existing forkoff then you just set the minforkoff to the existing forkoff which won't guarantee that the forkoff is not moved - previously I was returning the forkoff in that case.

Quoted 1 line

Yes, good point. I was only thinking of the case of attrs

trying to get

Quoted 12 lines

enlarged. I don't believe forkoff can be increased with the existing code, but with the attached change, it's now guaranteed.

Quoted 7 lines

Also comments could be fixed up a bit.

The XFS_DINODE_FMT_EXTENTS comment has problems:
"in in the extents form will migrate to btree" maybe you mean:
"in the data extents form migrating to btree"

Quoted 6 lines
Quoted 4 lines

The XFS_DINODE_FMT_BTREE comment talks about setting the forkoff but it is actually setting minforkoff.

Hehe, that comment is unchanged from the previous patch, but

I've fixed

Quoted 3 lines

it.

There still seems to be an issue with a non-attribute

data-btree case,

Quoted 1 line

where adding a big-enough attribute will still cause the

btree to grow

Quoted 150 lines

another level (not corrupted though), so it seems minforkoff in that case is not quite right.

Quoted 145 lines

--Tim

Quoted 10 lines · Barry Naujok
Quoted 10 lines

-----Original Message-----
From: Timothy Shimmin [mailto:[email protected]]
Sent: Thursday, 14 December 2006 11:57 PM
To: Barry Naujok
Cc: Russell Cattelan; [email protected]; Timothy Shimmin; [email protected]; David Chinner
Subject: Re: attr2

Quoted 8 lines
Quoted 8 lines

Hi,

Quoted 3 lines · Barry Naujok

Made one interesting observation with the attr2 with a

inode full of

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

extents that must be migrated to btree format, if the

xattr is big

Quoted 8 lines
Quoted 8 lines
Quoted 3 lines

enough to go beyond the default forkoff (ie. < 15 in 256

byte inode) but

Quoted 2 lines

still stay inline, it won't, but go to extents.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Eg:

makeextents -b4096 -n 9 <foo>
setfattr -n user.<32 chars> <foo>

forkoff stays at 15, the data extents are btree as

expected, and the

Quoted 52 lines
Quoted 52 lines
Quoted 7 lines

xattr is extents rather than local (shortform).

I think a test is needed when creating the initial attr fork:

xfs_bmap_add_attrfork() calls xfs_attr_shortform_bytesfit()

to determine

Quoted 3 lines

the desired forkoff. As the inode is full at this stage

without an attr

Quoted 7 lines

fork, it will definitely go btree.

  minforkoff = MAX(extents used, XFS_BMDR_SPACE_CALC());

I'm thinking if the inode doesn't have attrs at this stage,

it should

Quoted 3 lines

base the minimum on XFS_BMDR_SPACE_CALC, rather than the

space currently

Quoted 13 lines

used by extents.

eg:

       minforkoff = XFS_BMDR_SPACE_CALC(MINDBTPTRS);
       if (dp->i_d.di_forkoff && xfs_ifork_dsize_used(dp)
minforkoff)
               minforkoff = xfs_ifork_dsize_used(dp);

Thoughts?

Did you mean this just if the data extents are in btree format? You don't seem to be codifying the notion of
"full of extents that must be migrated to btree format". No comparison of requested attr offset with the data space used which would tell if we were going to have to force data extents to btree.

Quoted 7 lines
Quoted 7 lines

I guess there is no point in both going out of line if they don't have to.
However, should we favour one over the other (EA over data). I thought we were doing 1st come 1st served.
If data extents fit inline currently and then they still

fit inline

Quoted 5 lines
Quoted 5 lines

after EA added but
with EA out of line, then data wins.
But if data extents have to go out of line and we can have attr inline then keep attr inline.

Quoted 8 lines
Quoted 5 lines

It's late so I'm probably not thinking clearly ;-)

--Tim

Ok, I have implemented and tested the attached patch. I've

merged the

Quoted 5 lines

function back in, distilling the core requirements.

So, the main change from the previous patch is with data

extents format

Quoted 3 lines

and no attr, it checks to see if adding the default attr

size (forkoff =

Quoted 2 lines

15 for 256 byte inodes) will cause the data fork to be converted to btree format. If so, it will then set the minimum offset

to the base

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

btree root size.

So, with 7 extents, forkoff is allowed to go down to 14.

After that,

Quoted 20 lines
Quoted 20 lines
Quoted 5 lines

forkoff will be back at 15 and the attrs go out-of-line.

With 8 extents, adding any attr at all will cause the data

extents to go

Quoted 3 lines

btree. This allows the attr to use up all the inode space

it can up to

Quoted 6 lines

the btree root.

Signature · 3 lines
David Chatterton
XFS Engineering Manager
SGI Australia

===========================================================================
fs/xfs/xfs_attr.c
===========================================================================

fs/xfs/xfs_attr.c +11 -8
@@ -199,18 +199,14 @@ xfs_attr_set_int(xfs_inode_t *dp, const 
 		return (error);
 
 	/*
-	 * Determine space new attribute will use, and if it would be
-	 * "local" or "remote" (note: local != inline).
-	 */
-	size = xfs_attr_leaf_newentsize(namelen, valuelen,
-					mp->m_sb.sb_blocksize, &local);
-
-	/*
 	 * If the inode doesn't have an attribute fork, add one.
 	 * (inode must not be locked when we call this routine)
 	 */
 	if (XFS_IFORK_Q(dp) == 0) {
-		if ((error = xfs_bmap_add_attrfork(dp, size, rsvd)))
+		int sf_size = sizeof(xfs_attr_sf_hdr_t) +
+			      XFS_ATTR_SF_ENTSIZE_BYNAME(namelen, valuelen);
+
+		if ((error = xfs_bmap_add_attrfork(dp, sf_size, rsvd)))
 			return(error);
 	}
 
@@ -231,6 +227,13 @@ xfs_attr_set_int(xfs_inode_t *dp, const 
 	args.addname = 1;
 	args.oknoent = 1;
 
+	/*
+	 * Determine space new attribute will use, and if it would be
+	 * "local" or "remote" (note: local != inline).
+	 */
+	size = xfs_attr_leaf_newentsize(namelen, valuelen,
+					mp->m_sb.sb_blocksize, &local);
+
 	nblks = XFS_DAENTER_SPACE_RES(mp, XFS_ATTR_FORK);
 	if (local) {
 		if (size > (mp->m_sb.sb_blocksize >> 1)) {

===========================================================================
fs/xfs/xfs_attr_leaf.c
===========================================================================

fs/xfs/xfs_attr_leaf.c +42 -3

75 lines not on this page · raw message

===========================================================================
fs/xfs/xfs_bmap.c
===========================================================================

fs/xfs/xfs_bmap.c +1 -0
@@ -3543,6 +3543,7 @@ xfs_bmap_forkoff_reset(
 	if (whichfork == XFS_ATTR_FORK &&
 	    (ip->i_d.di_format != XFS_DINODE_FMT_DEV) &&
 	    (ip->i_d.di_format != XFS_DINODE_FMT_UUID) &&
+	    (ip->i_d.di_format != XFS_DINODE_FMT_BTREE) &&
 	    ((mp->m_attroffset >> 3) > ip->i_d.di_forkoff)) {
 		ip->i_d.di_forkoff = mp->m_attroffset >> 3;
 		ip->i_df.if_ext_max = XFS_IFORK_DSIZE(ip) /