Skip to contentSkip to conversation

REVIEW XFSQA test out xfs_attr_shortform_bytesfit attr2/attr1 fix

PATCH Timothy Shimmin 3 participants 7 messages Apr 9, 2008 – Apr 14, 2008

lore
Share & tools
lore
#0 lore raw

Hi there,

A test to test out Eric's fix for xfs_attr_shortform_bytesfit bug when going from attr2 to attr1.

With TOT kernel, without patch, one can see the corrupted inline dirents. With patch, all is well.

The 186.out _should_ be output'ing ATTR2 for the db version command but I'm awaiting Barry's xfsprogs checkin to fix that one - and then I will regenerate it.

186 does have some superfluous functionality: _cleanup, _rmv_eas but it may be handy to have for more attr2 testing later.

--Tim

===========================================================================
Index: xfstests/186
===========================================================================

xfstests/186 +165 -0
@@ -0,0 +1,165 @@
+#! /bin/sh
+# FS QA Test No. 186
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2008 Silicon Graphics, Inc.  All Rights Reserved.
+#-----------------------------------------------------------------------
+#
+# Test out:
+# pv#979606: xfs bug in going from attr2 back to attr1
+#
+# Test bug in going from attr2 back to attr1 where xfs
+# (due to xfs_attr_shortform_bytesfit)
+# would reset the di_forkoff to the m_offset instead of
+# leaving the di_forkoff alone as was intended.
+#
+# We create enough dirents to push us past m_attroffset,
+# and create an EA so we have a fork offset
+# and then turn on attr1 and add one more EA which
+# will write over the shortform dirents.
+#
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+_create_dirents()
+{
+	start_num=$1
+	end_num=$2
+	cd $fork_dir
+	for i in `seq $start_num $end_num`; do
+		touch file.$i
+	done
+}
+
+_create_eas()
+{
+	start_num=$1
+	end_num=$2
+	for i in `seq $start_num $end_num`; do
+		setfattr -n user.$i -v 0xbabe $fork_dir
+	done
+}
+
+_rmv_eas()
+{
+	start_num=$1
+	end_num=$2
+	for i in `seq $start_num $end_num`; do
+		setfattr -x user.$i $fork_dir
+	done
+}
+
+_filter_inode()
+{
+	egrep '^u.sfdir2|^a.sfattr|core.forkoff' |\
+	egrep -v 'inumber|parent'
+}
+
+_filter_version()
+{
+	tr ',' '\n' | grep ATTR
+}
+
+_print_inode()
+{
+	echo ""
+	echo "================================="
+	$XFS_DB_PROG -c "version" $SCRATCH_DEV 2>&1 | _filter_version
+	$XFS_DB_PROG -c "inode $inum" -c p $SCRATCH_DEV 2>&1 | _filter_inode
+	echo "================================="
+}
+
+_do_eas()
+{
+	echo ""
+	_scratch_mount
+	if [ $1 = "-r" ]; then
+		echo "*** remove EAs start $2 end $3 ***"
+		_rmv_eas $2 $3
+	else
+		echo "*** create EAs start $2 end $3 ***"
+		_create_eas $2 $3
+	fi
+	echo ""
+	cd /; $UMOUNT_PROG $SCRATCH_MNT
+	_print_inode
+}
+
+_do_dirents()
+{
+	num=`expr $2 - $1 + 1`
+	echo ""
+	echo "*** create $num dirents ***"
+	echo ""
+	_scratch_mount
+	_create_dirents $1 $2
+	cd /; $UMOUNT_PROG $SCRATCH_MNT
+	_print_inode
+}
+
+_changeto_attr1()
+{
+	echo ""
+	echo "Try setting attr1 by db"
+	echo ""
+	$XFS_DB_PROG -x -c "version attr1" $SCRATCH_DEV | _filter_version
+}
+
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs xfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs -i attr=2,size=512 >/dev/null 2>&1
+
+# set inum to root dir ino
+# we'll add in dirents and EAs into the root directory
+eval `$XFS_DB_PROG -r -c 'sb 0' -c 'p rootino' $SCRATCH_DEV | $SED_PROG 's/ //g'`
+inum=$rootino
+fork_dir=$SCRATCH_MNT
+_print_inode
+
+# add enough dirents to be inline but more
+# than will fit for m_attroffset for 512b inodes
+# for attr2 this is not a problem
+_do_dirents 1 25
+
+# add 1 ea so we get our forkoff happening
+_do_eas -c 1 1
+
+# now change back to attr1 where forkoff is constant now
+_changeto_attr1
+
+# now add another EA
+# for a bug in xfs_add_shortform_bytesfit
+# where it reset the forkoff to m_attroffset>>3 instead of
+# leaving as di_forkoff
+# If it resets to m_attroffset which is in the middle of
+# the dirents then they will get corrupted
+_do_eas -c 2 2
+
+# success, all done
+status=0
+exit

===========================================================================
Index: xfstests/186.out
===========================================================================

xfstests/186.out +288 -0

288 lines not on this page · raw message

===========================================================================
Index: xfstests/group
===========================================================================

xfstests/group +1 -0
@@ -273,3 +273,4 @@ filestreams	[email protected]
 183 rw other auto
 184 metadata auto
 185 dmapi auto
+186 attr auto
#1 lore raw
Quoted 11 lines · Timothy Shimmin

Hi there,

A test to test out Eric's fix for xfs_attr_shortform_bytesfit bug when going from attr2 to attr1.

With TOT kernel, without patch, one can see the corrupted inline dirents. With patch, all is well.

The 186.out _should_ be output'ing ATTR2 for the db version command but I'm awaiting Barry's xfsprogs checkin to fix that one - and then I will regenerate it.

Really? I'm seeing it fail with ATTR2 in the xfs_db output...

dgc@budgie:~/xfstests$ sudo ./check 186
FSTYP         -- xfs (debug)
PLATFORM      -- Linux/ia64 budgie 2.6.25-rc3-dgc-xfs
MKFS_OPTIONS  -- -f -bsize=4096 /dev/sdb6
MOUNT_OPTIONS -- /dev/sdb6 /mnt/scratch

186 - output mismatch (see 186.out.bad)
13a14

Quoted 1 line

ATTR2

98a100

Quoted 1 line

ATTR2

Failures: 186
Failed 1 of 1 tests
dgc@budgie:~/xfstests$

So the first is supposed to be there, but the second shows:

....
=================================
ATTR
ATTR2
core.forkoff = 47
....

Are both supposed to be present?

Cheers,

Dave.

Signature · 3 lines
Dave Chinner
Principal Engineer
SGI Australian Software Group
#2 lore raw

Hi Dave and Barry,

Quoted 6 lines · David Chinner
Quoted 5 lines · Timothy Shimmin

Hi there,

A test to test out Eric's fix for xfs_attr_shortform_bytesfit bug when going from attr2 to attr1.

Quoted 9 lines
Quoted 6 lines

With TOT kernel, without patch, one can see the corrupted inline dirents. With patch, all is well.

The 186.out _should_ be output'ing ATTR2 for the db version command but I'm awaiting Barry's xfsprogs checkin to fix that one - and then I will regenerate it.

Really? I'm seeing it fail with ATTR2 in the xfs_db output...

That's what I used to see until I updated to TOT kernel. That's weird.
And I don't know why ATTR and ATTR2 is coming out either. I assume it is all about the recent version 2 changes, and what was done in userspace and in the kernel. Barry, can you enlighten us here?
Do we have latest changes for xfs_db and mkfs now? That's why I was waiting to ensure I had the latest user changes; it feels like xfs_db is getting things wrong as the mkfs and kernel behaviour seems to be behaving like I expected.

--Tim

Quoted 29 lines
dgc@budgie:~/xfstests$ sudo ./check 186
FSTYP         -- xfs (debug)
PLATFORM      -- Linux/ia64 budgie 2.6.25-rc3-dgc-xfs
MKFS_OPTIONS  -- -f -bsize=4096 /dev/sdb6
MOUNT_OPTIONS -- /dev/sdb6 /mnt/scratch

186 - output mismatch (see 186.out.bad)
13a14

Quoted 1 line

ATTR2

98a100

Quoted 1 line

ATTR2

Failures: 186
Failed 1 of 1 tests
dgc@budgie:~/xfstests$

So the first is supposed to be there, but the second shows:

....
=================================
ATTR
ATTR2
core.forkoff = 47
....

Are both supposed to be present?

Cheers,

Dave.

#3 lore raw
Quoted 17 lines · Timothy Shimmin

Hi Dave and Barry,

Quoted 6 lines · David Chinner
Quoted 5 lines · Timothy Shimmin

Hi there,

A test to test out Eric's fix for xfs_attr_shortform_bytesfit bug when going from attr2 to attr1.

Quoted 8 lines
Quoted 6 lines

With TOT kernel, without patch, one can see the corrupted inline dirents. With patch, all is well.

The 186.out _should_ be output'ing ATTR2 for the db version command but I'm awaiting Barry's xfsprogs checkin to fix that one - and then I will regenerate it.

Really? I'm seeing it fail with ATTR2 in the xfs_db output...

Quoted 10 lines

That's what I used to see until I updated to TOT kernel. That's weird.
And I don't know why ATTR and ATTR2 is coming out either. I assume it is all about the recent version 2 changes, and what was done in userspace and in the kernel. Barry, can you enlighten us here?
Do we have latest changes for xfs_db and mkfs now? That's why I was waiting to ensure I had the latest user changes; it feels like xfs_db is getting things wrong as the mkfs and kernel behaviour seems to be behaving like I expected.

I'll now checkin the latest sb_bad_features2 patches for user-space. With Dave's kernel changes, should be quite compatible now.

Quoted 26 lines

--Tim

Quoted 23 lines
dgc@budgie:~/xfstests$ sudo ./check 186
FSTYP         -- xfs (debug)
PLATFORM      -- Linux/ia64 budgie 2.6.25-rc3-dgc-xfs
MKFS_OPTIONS  -- -f -bsize=4096 /dev/sdb6
MOUNT_OPTIONS -- /dev/sdb6 /mnt/scratch
 186      - output mismatch (see 186.out.bad)
13a14
Quoted 1 line

ATTR2

98a100

Quoted 1 line

ATTR2

Failures: 186
Failed 1 of 1 tests
dgc@budgie:~/xfstests$
So the first is supposed to be there, but the second shows:
....
=================================
ATTR
ATTR2
core.forkoff = 47
....
Are both supposed to be present?
Cheers,
Dave.

#4 lore raw
In reply to Timothy Shimmin
Quoted 12 lines · Timothy Shimmin

Hi Dave and Barry,

Quoted 9 lines · David Chinner
Quoted 8 lines · Timothy Shimmin

Hi there,

A test to test out Eric's fix for xfs_attr_shortform_bytesfit bug when going from attr2 to attr1.

With TOT kernel, without patch, one can see the corrupted inline dirents. With patch, all is well.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

The 186.out _should_ be output'ing ATTR2 for the db version command but I'm awaiting Barry's xfsprogs checkin to fix that one - and then I will regenerate it.

Really? I'm seeing it fail with ATTR2 in the xfs_db output...

That's what I used to see until I updated to TOT kernel. That's weird.

It's likely due to Eric's change to make sb_badfeatures2 = sb_features2. That will make userspace pick up the attr2 feature properly regardless of where it is. Perhaps xfs_db is b0rked w.r.t. reporting attr vs attr2....

Cheers,

Dave.

Signature · 3 lines
Dave Chinner
Principal Engineer
SGI Australian Software Group
#5 lore raw

Hi Dave,

Quoted 21 lines · David Chinner
Quoted 12 lines · Timothy Shimmin

Hi Dave and Barry,

Quoted 9 lines · David Chinner
Quoted 8 lines · Timothy Shimmin

Hi there,

A test to test out Eric's fix for xfs_attr_shortform_bytesfit bug when going from attr2 to attr1.

With TOT kernel, without patch, one can see the corrupted inline dirents. With patch, all is well.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

The 186.out _should_ be output'ing ATTR2 for the db version command but I'm awaiting Barry's xfsprogs checkin to fix that one - and then I will regenerate it.

Really? I'm seeing it fail with ATTR2 in the xfs_db output...

That's what I used to see until I updated to TOT kernel. That's weird.

Quoted 8 lines

It's likely due to Eric's change to make sb_badfeatures2 = sb_features2. That will make userspace pick up the attr2 feature properly regardless of where it is. Perhaps xfs_db is b0rked w.r.t. reporting attr vs attr2....

Cheers,

Dave.

I'm not sure, but with Barry's latest checkins, all seems well and I'm now getting the attr2 output as expected.

Which just leaves the ATTR and ATTR2 output when I add an EA to a file on an ATTR2 file-system.
It is naturally the kernel that is doing this.
The existing behaviour prior to ATTR2, was to
turn on ATTR as soon as one added an EA to a file (added by doucette).

Quoted 1 line

xfs_bmap_add_attrfork

....

Quoted 5 lines
        if (!xfs_sb_version_hasattr(&mp->m_sb) ||
           (!xfs_sb_version_hasattr2(&mp->m_sb) && version == 2)) {
                __int64_t sbfields = 0;

                spin_lock(&mp->m_sb_lock);
Quoted 8 lines
                if (!xfs_sb_version_hasattr(&mp->m_sb)) {
                        xfs_sb_version_addattr(&mp->m_sb);
                        sbfields |= XFS_SB_VERSIONNUM;
                }
                if (!xfs_sb_version_hasattr2(&mp->m_sb) && version == 2) {
                        xfs_sb_version_addattr2(&mp->m_sb);
                        sbfields |= (XFS_SB_VERSIONNUM | XFS_SB_FEATURES2);
                }

So I would say it is intentional and we are preserving the behaviour of knowing if we have EAs in use or not.
I guess it seems a bit redundant but it may well cause less problems in doing so and doesn't hurt.

--Tim

#6 lore raw

Updated 186 output...

--Tim

===========================================================================
Index: xfstests/186
===========================================================================

xfstests/186 +166 -0
@@ -0,0 +1,166 @@
+#! /bin/sh
+# FS QA Test No. 186
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2008 Silicon Graphics, Inc.  All Rights Reserved.
+#-----------------------------------------------------------------------
+#
+# Test out:
+# pv#979606: xfs bug in going from attr2 back to attr1
+#
+# Test bug in going from attr2 back to attr1 where xfs
+# (due to xfs_attr_shortform_bytesfit)
+# would reset the di_forkoff to the m_offset instead of
+# leaving the di_forkoff alone as was intended.
+#
+# We create enough dirents to push us past m_attroffset,
+# and create an EA so we have a fork offset
+# and then turn on attr1 and add one more EA which
+# will write over the shortform dirents.
+#
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+_create_dirents()
+{
+	start_num=$1
+	end_num=$2
+	cd $fork_dir
+	for i in `seq $start_num $end_num`; do
+		touch file.$i
+	done
+}
+
+_create_eas()
+{
+	start_num=$1
+	end_num=$2
+	for i in `seq $start_num $end_num`; do
+		setfattr -n user.$i -v 0xbabe $fork_dir
+	done
+}
+
+_rmv_eas()
+{
+	start_num=$1
+	end_num=$2
+	for i in `seq $start_num $end_num`; do
+		setfattr -x user.$i $fork_dir
+	done
+}
+
+_filter_inode()
+{
+	tee -a $seq.full | egrep '^u.sfdir2|^a.sfattr|core.forkoff' |\
+	egrep -v 'inumber|parent'
+}
+
+_filter_version()
+{
+	tee -a $seq.full | tr ',' '\n' | grep ATTR
+}
+
+_print_inode()
+{
+	echo ""
+	echo "================================="
+	$XFS_DB_PROG -c "version" $SCRATCH_DEV 2>&1 | _filter_version
+	$XFS_DB_PROG -c "inode $inum" -c p $SCRATCH_DEV 2>&1 | _filter_inode
+	echo "================================="
+}
+
+_do_eas()
+{
+	echo ""
+	_scratch_mount
+	if [ $1 = "-r" ]; then
+		echo "*** remove EAs start $2 end $3 ***"
+		_rmv_eas $2 $3
+	else
+		echo "*** create EAs start $2 end $3 ***"
+		_create_eas $2 $3
+	fi
+	echo ""
+	cd /; $UMOUNT_PROG $SCRATCH_MNT
+	_print_inode
+}
+
+_do_dirents()
+{
+	num=`expr $2 - $1 + 1`
+	echo ""
+	echo "*** create $num dirents ***"
+	echo ""
+	_scratch_mount
+	_create_dirents $1 $2
+	cd /; $UMOUNT_PROG $SCRATCH_MNT
+	_print_inode
+}
+
+_changeto_attr1()
+{
+	echo ""
+	echo "Try setting attr1 by db"
+	echo ""
+	$XFS_DB_PROG -x -c "version attr1" $SCRATCH_DEV | _filter_version
+}
+
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs xfs
+_supported_os Linux
+_require_scratch
+rm -f $seq.full
+
+_scratch_mkfs -i attr=2,size=512 >/dev/null 2>&1
+
+# set inum to root dir ino
+# we'll add in dirents and EAs into the root directory
+eval `$XFS_DB_PROG -r -c 'sb 0' -c 'p rootino' $SCRATCH_DEV | $SED_PROG 's/ //g'`
+inum=$rootino
+fork_dir=$SCRATCH_MNT
+_print_inode
+
+# add enough dirents to be inline but more
+# than will fit for m_attroffset for 512b inodes
+# for attr2 this is not a problem
+_do_dirents 1 25
+
+# add 1 ea so we get our forkoff happening
+_do_eas -c 1 1
+
+# now change back to attr1 where forkoff is constant now
+_changeto_attr1
+
+# now add another EA
+# for a bug in xfs_add_shortform_bytesfit
+# where it reset the forkoff to m_attroffset>>3 instead of
+# leaving as di_forkoff
+# If it resets to m_attroffset which is in the middle of
+# the dirents then they will get corrupted
+_do_eas -c 2 2
+
+# success, all done
+status=0
+exit

===========================================================================
Index: xfstests/186.out
===========================================================================

xfstests/186.out +290 -0

290 lines not on this page · raw message

===========================================================================
Index: xfstests/group
===========================================================================

xfstests/group +1 -0
@@ -273,3 +273,4 @@ filestreams	[email protected]
 183 rw other auto
 184 metadata auto
 185 dmapi auto
+186 attr auto