Skip to contentSkip to conversation

xfstests: Test xfs_fsr / swapext management of di_forkoff w/ selinux

PATCH In review Eric Sandeen 4 participants 7 messages Oct 18, 2013 – Oct 22, 2013

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xfstests: Test xfs_fsr / swapext management of di_forkoff w/ selinux

In some cases selinux's creation of an xattr on the temporary fd creates a local xattr, but the file we are trying to defragment has attrs in extent format, and the forkoff mismatch will cause xfs_fsr to fail. This test demonstrates it; I have old patches sent to the list long ago that should fix it. I'll resend them soon.

---

tests/xfs/351 new file +89 -0
new file mode 100755
index 0000000..b21ba7a
@@ -0,0 +1,89 @@
+#! /bin/bash
+# FS QA Test No. 350
+#
+# Test xfs_fsr / swapext management of di_forkoff w/ selinux
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2013 Red Hat, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+_require_scratch
+
+# real QA test starts here
+
+_supported_fs xfs
+_supported_os Linux
+
+getenforce | grep -q "Enforcing\|Permissive" || _notrun "SELinux not enabled"
+[ "$XFS_FSR_PROG" = "" ] && _notrun "xfs_fsr not found"
+
+rm -f $seqres.full
+
+_scratch_mkfs_xfs -i size=256 >> $seq.full 2>&1 || _fail "mkfs failed"
+
+# Manually mount to avoid fs-wide context set by default in xfstests
+mount $SCRATCH_DEV $SCRATCH_MNT
+
+touch $SCRATCH_MNT/$seq.test
+
+# Create an extents-format attribute on a file
+xfs_io -f -c "pwrite -S 0x63 0 4096" $SCRATCH_MNT/attrvals 2>&1 &> $seqres.full
+cat $SCRATCH_MNT/attrvals | attr -s name $SCRATCH_MNT/$seq.test 2>&1 &> $seqres.full
+
+# Fragment the file by writing backwards
+for I in `seq 6 -1 0`; do
+	dd if=/dev/zero of=$SCRATCH_MNT/$seq.test seek=$I bs=4k \
+	   oflag=direct count=1 conv=notrunc 2>&1 &> $seqres.full
+done
+
+# See if xfs_fsr works.
+
+# The target file now has a non-local attribute, but the
+# fsr temp file gets assigned a local attribute on creation,
+# due to selinux.
+    
+# In this case, the large-ish selinux attr will create a forkoff
+# in the temp file smaller than the forkoff in the target file,
+# because the FMT_EXTENTS attr takes up less space.  If there is
+# no mechanism to grow the temp forkoff to match, we can end up
+# failing to swap these 2 inodes if the result is not enough
+# data space in the temp inode as a result.
+
+$XFS_FSR_PROG -v $SCRATCH_MNT/$seq.test | _filter_scratch
+
+# success, all done
+status=0
+exit
tests/xfs/351.out new file +3 -0
new file mode 100644
index 0000000..86cbe5d
@@ -0,0 +1,3 @@
+QA output created by 351
+SCRATCH_MNT/351.test
+extents before:6 after:1 DONE SCRATCH_MNT/351.test
tests/xfs/group +1 -1
index acb92ac..da76770 100644
@@ -180,4 +180,4 @@
 298 auto attr symlink quick
 299 auto quota
 350 auto dump quota
-
+351 auto fsr

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#1 lore raw
Quoted 29 lines · Eric Sandeen

In some cases selinux's creation of an xattr on the temporary fd creates a local xattr, but the file we are trying to defragment has attrs in extent format, and the forkoff mismatch will cause xfs_fsr to fail. This test demonstrates it; I have old patches sent to the list long ago that should fix it. I'll resend them soon.

---

tests/xfs/351 new file +13 -0
new file mode 100755
index 0000000..b21ba7a
@@ -0,0 +1,89 @@
+#! /bin/bash
+# FS QA Test No. 350
+#
+# Test xfs_fsr / swapext management of di_forkoff w/ selinux
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2013 Red Hat, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,

tests/xfs/351 line 23

Quoted 10 lines
tests/xfs/351 new file +10 -0
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator

Seems we don't need "creator" in comment now.

tests/xfs/351 line 25

Quoted 2 lines
tests/xfs/351 new file +2 -0
+
+seq=`basename $0`

Forgot about "seqres=$RESULT_DIR/$seq" ?

Quoted 28 lines
tests/xfs/351 new file +28 -0
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+_require_scratch
+
+# real QA test starts here
+
+_supported_fs xfs
+_supported_os Linux
+
+getenforce | grep -q "Enforcing\|Permissive" || _notrun "SELinux not enabled"
+[ "$XFS_FSR_PROG" = "" ] && _notrun "xfs_fsr not found"
+
+rm -f $seqres.full

tests/xfs/351 line 63

Quoted 10 lines
tests/xfs/351 new file +10 -0
+
+_scratch_mkfs_xfs -i size=256 >> $seq.full 2>&1 || _fail "mkfs failed"
+
+# Manually mount to avoid fs-wide context set by default in xfstests
+mount $SCRATCH_DEV $SCRATCH_MNT
+
+touch $SCRATCH_MNT/$seq.test
+
+# Create an extents-format attribute on a file
+xfs_io -f -c "pwrite -S 0x63 0 4096" $SCRATCH_MNT/attrvals 2>&1 &> $seqres.full

Use $XFS_IO_PROG instead?

And seems "2>&1" is redundant, "&>" can do the work.

Thanks,
Eryu Guan

Quoted 47 lines
tests/xfs/351 new file +26 -0
+cat $SCRATCH_MNT/attrvals | attr -s name $SCRATCH_MNT/$seq.test 2>&1 &> $seqres.full
+
+# Fragment the file by writing backwards
+for I in `seq 6 -1 0`; do
+	dd if=/dev/zero of=$SCRATCH_MNT/$seq.test seek=$I bs=4k \
+	   oflag=direct count=1 conv=notrunc 2>&1 &> $seqres.full
+done
+
+# See if xfs_fsr works.
+
+# The target file now has a non-local attribute, but the
+# fsr temp file gets assigned a local attribute on creation,
+# due to selinux.
+    
+# In this case, the large-ish selinux attr will create a forkoff
+# in the temp file smaller than the forkoff in the target file,
+# because the FMT_EXTENTS attr takes up less space.  If there is
+# no mechanism to grow the temp forkoff to match, we can end up
+# failing to swap these 2 inodes if the result is not enough
+# data space in the temp inode as a result.
+
+$XFS_FSR_PROG -v $SCRATCH_MNT/$seq.test | _filter_scratch
+
+# success, all done
+status=0
+exit
tests/xfs/351.out new file +3 -0
new file mode 100644
index 0000000..86cbe5d
@@ -0,0 +1,3 @@
+QA output created by 351
+SCRATCH_MNT/351.test
+extents before:6 after:1 DONE SCRATCH_MNT/351.test
tests/xfs/group +1 -1
index acb92ac..da76770 100644
@@ -180,4 +180,4 @@
 298 auto attr symlink quick
 299 auto quota
 350 auto dump quota
-
+351 auto fsr
Quoted 4 lines

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#2 lore raw
Quoted 34 lines · Eryu Guan
Quoted 33 lines · Eric Sandeen

In some cases selinux's creation of an xattr on the temporary fd creates a local xattr, but the file we are trying to defragment has attrs in extent format, and the forkoff mismatch will cause xfs_fsr to fail. This test demonstrates it; I have old patches sent to the list long ago that should fix it. I'll resend them soon.

---

tests/xfs/351 new file +17 -0
new file mode 100755
index 0000000..b21ba7a
@@ -0,0 +1,89 @@
+#! /bin/bash
+# FS QA Test No. 350
+#
+# Test xfs_fsr / swapext management of di_forkoff w/ selinux
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2013 Red Hat, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
Quoted 8 lines

tests/xfs/351 line 23

Quoted 6 lines
tests/xfs/351 new file +6 -0
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator

Seems we don't need "creator" in comment now.

yup

Quoted 4 lines

tests/xfs/351 line 25

Quoted 2 lines
tests/xfs/351 new file +2 -0
+
+seq=`basename $0`

Forgot about "seqres=$RESULT_DIR/$seq" ?

crud. :)

...

Quoted 4 lines

tests/xfs/351 line 63

Quoted 2 lines
tests/xfs/351 new file +2 -0
+# Create an extents-format attribute on a file
+xfs_io -f -c "pwrite -S 0x63 0 4096" $SCRATCH_MNT/attrvals 2>&1 &> $seqres.full

Use $XFS_IO_PROG instead?

yup

Quoted 1 line

And seems "2>&1" is redundant, "&>" can do the work.

heh, yeah. Ok, will tidy up and resend tomorrow.

Thanks!

-Eric

Quoted 50 lines

Thanks,
Eryu Guan

Quoted 47 lines
tests/xfs/351 new file +26 -0
+cat $SCRATCH_MNT/attrvals | attr -s name $SCRATCH_MNT/$seq.test 2>&1 &> $seqres.full
+
+# Fragment the file by writing backwards
+for I in `seq 6 -1 0`; do
+	dd if=/dev/zero of=$SCRATCH_MNT/$seq.test seek=$I bs=4k \
+	   oflag=direct count=1 conv=notrunc 2>&1 &> $seqres.full
+done
+
+# See if xfs_fsr works.
+
+# The target file now has a non-local attribute, but the
+# fsr temp file gets assigned a local attribute on creation,
+# due to selinux.
+    
+# In this case, the large-ish selinux attr will create a forkoff
+# in the temp file smaller than the forkoff in the target file,
+# because the FMT_EXTENTS attr takes up less space.  If there is
+# no mechanism to grow the temp forkoff to match, we can end up
+# failing to swap these 2 inodes if the result is not enough
+# data space in the temp inode as a result.
+
+$XFS_FSR_PROG -v $SCRATCH_MNT/$seq.test | _filter_scratch
+
+# success, all done
+status=0
+exit
tests/xfs/351.out new file +3 -0
new file mode 100644
index 0000000..86cbe5d
@@ -0,0 +1,3 @@
+QA output created by 351
+SCRATCH_MNT/351.test
+extents before:6 after:1 DONE SCRATCH_MNT/351.test
tests/xfs/group +1 -1
index acb92ac..da76770 100644
@@ -180,4 +180,4 @@
 298 auto attr symlink quick
 299 auto quota
 350 auto dump quota
-
+351 auto fsr
Quoted 10 lines
Quoted 4 lines

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#3 lore raw

PATCH v2 1/1 xfstests: Test xfs_fsr / swapext management of di_forkoff w/ selinux

In reply to Eric Sandeen

In some cases selinux's creation of an xattr on the temporary fd creates a local xattr, but the file we are trying to defragment has attrs in extent format, and the forkoff mismatch will cause xfs_fsr to fail. This test demonstrates it; I have old patches sent to the list long ago that should fix it. I'll resend them soon.

---

V2: Address Eryu's review

tests/xfs/351 new file +88 -0
new file mode 100755
index 0000000..13c7629
@@ -0,0 +1,88 @@
+#! /bin/bash
+# FS QA Test No. 351
+#
+# Test xfs_fsr / swapext management of di_forkoff w/ selinux
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2013 Red Hat, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+_require_scratch
+
+# real QA test starts here
+
+_supported_fs xfs
+_supported_os Linux
+
+getenforce | grep -q "Enforcing\|Permissive" || _notrun "SELinux not enabled"
+[ "$XFS_FSR_PROG" = "" ] && _notrun "xfs_fsr not found"
+
+rm -f $seqres.full
+
+_scratch_mkfs_xfs -i size=256 >> $seq.full 2>&1 || _fail "mkfs failed"
+
+# Manually mount to avoid fs-wide context set by default in xfstests
+mount $SCRATCH_DEV $SCRATCH_MNT
+
+touch $SCRATCH_MNT/$seq.test
+
+# Create an extents-format attribute on a file
+$XFS_IO_PROG -f -c "pwrite -S 0x63 0 4096" $SCRATCH_MNT/attrvals >> $seqres.full 2>&1
+cat $SCRATCH_MNT/attrvals | attr -s name $SCRATCH_MNT/$seq.test >> $seqres.full 2>&1
+
+# Fragment the file by writing backwards
+for I in `seq 6 -1 0`; do
+	dd if=/dev/zero of=$SCRATCH_MNT/$seq.test seek=$I bs=4k \
+	   oflag=direct count=1 conv=notrunc >> $seqres.full 2>&1
+done
+
+# See if xfs_fsr works.
+
+# The target file now has a non-local attribute, but the
+# fsr temp file gets assigned a local attribute on creation,
+# due to selinux.
+    
+# In this case, the large-ish selinux attr will create a forkoff
+# in the temp file smaller than the forkoff in the target file,
+# because the FMT_EXTENTS attr takes up less space.  If there is
+# no mechanism to grow the temp forkoff to match, we can end up
+# failing to swap these 2 inodes if the result is not enough
+# data space in the temp inode as a result.
+
+$XFS_FSR_PROG -v $SCRATCH_MNT/$seq.test | _filter_scratch
+
+# success, all done
+status=0
+exit
tests/xfs/351.out new file +3 -0
new file mode 100644
index 0000000..86cbe5d
@@ -0,0 +1,3 @@
+QA output created by 351
+SCRATCH_MNT/351.test
+extents before:6 after:1 DONE SCRATCH_MNT/351.test
tests/xfs/group +1 -1
index acb92ac..da76770 100644
@@ -180,4 +180,4 @@
 298 auto attr symlink quick
 299 auto quota
 350 auto dump quota
-
+351 auto fsr

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Reviewed-by on v2 1/1
#4 lore raw
Quoted 84 lines · Eric Sandeen

In some cases selinux's creation of an xattr on the temporary fd creates a local xattr, but the file we are trying to defragment has attrs in extent format, and the forkoff mismatch will cause xfs_fsr to fail. This test demonstrates it; I have old patches sent to the list long ago that should fix it. I'll resend them soon.

---

V2: Address Eryu's review

tests/xfs/351 new file +66 -0
new file mode 100755
index 0000000..13c7629
@@ -0,0 +1,88 @@
+#! /bin/bash
+# FS QA Test No. 351
+#
+# Test xfs_fsr / swapext management of di_forkoff w/ selinux
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2013 Red Hat, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+_require_scratch
+
+# real QA test starts here
+
+_supported_fs xfs
+_supported_os Linux
+
+getenforce | grep -q "Enforcing\|Permissive" || _notrun "SELinux not enabled"
+[ "$XFS_FSR_PROG" = "" ] && _notrun "xfs_fsr not found"
+
+rm -f $seqres.full
+
+_scratch_mkfs_xfs -i size=256 >> $seq.full 2>&1 || _fail "mkfs failed"
+
+# Manually mount to avoid fs-wide context set by default in xfstests
+mount $SCRATCH_DEV $SCRATCH_MNT
+
+touch $SCRATCH_MNT/$seq.test
+
+# Create an extents-format attribute on a file
+$XFS_IO_PROG -f -c "pwrite -S 0x63 0 4096" $SCRATCH_MNT/attrvals >> $seqres.full 2>&1
+cat $SCRATCH_MNT/attrvals | attr -s name $SCRATCH_MNT/$seq.test >> $seqres.full 2>&1
+
+# Fragment the file by writing backwards
+for I in `seq 6 -1 0`; do

tests/xfs/351 line 76

Quoted 10 lines
tests/xfs/351 new file +10 -0
+	dd if=/dev/zero of=$SCRATCH_MNT/$seq.test seek=$I bs=4k \
+	   oflag=direct count=1 conv=notrunc >> $seqres.full 2>&1
+done
+
+# See if xfs_fsr works.
+
+# The target file now has a non-local attribute, but the
+# fsr temp file gets assigned a local attribute on creation,
+# due to selinux.
+    

There's trailing whitespaces issue here, otherwise looks good to me.

I suppose the seq number can be fixed at commit time, as well as the trailing whitespaces.

Thanks!
Eryu

BTW, test fails on 3.12-rc5 kernel

root/xfstests/results//xfs/351.out.bad 2013-10-18 13:24:38.014147342 -0400 renamed from xfs/351.out 2013-10-18 13:23:10.219847555 -0400 +1 -1
@@ -1,3 +1,3 @@
 QA output created by 351
 SCRATCH_MNT/351.test
-extents before:6 after:1 DONE SCRATCH_MNT/351.test
+XFS_IOC_SWAPEXT failed: SCRATCH_MNT/351.test: Invalid argument
Quoted 32 lines
tests/xfs/351 new file +12 -0
+# In this case, the large-ish selinux attr will create a forkoff
+# in the temp file smaller than the forkoff in the target file,
+# because the FMT_EXTENTS attr takes up less space.  If there is
+# no mechanism to grow the temp forkoff to match, we can end up
+# failing to swap these 2 inodes if the result is not enough
+# data space in the temp inode as a result.
+
+$XFS_FSR_PROG -v $SCRATCH_MNT/$seq.test | _filter_scratch
+
+# success, all done
+status=0
+exit
tests/xfs/351.out new file +3 -0
new file mode 100644
index 0000000..86cbe5d
@@ -0,0 +1,3 @@
+QA output created by 351
+SCRATCH_MNT/351.test
+extents before:6 after:1 DONE SCRATCH_MNT/351.test
tests/xfs/group +1 -1
index acb92ac..da76770 100644
@@ -180,4 +180,4 @@
 298 auto attr symlink quick
 299 auto quota
 350 auto dump quota
-
+351 auto fsr
Quoted 4 lines

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#5 lore raw

On 10/18/13 12:26 PM, Eryu Guan wrote:

...

Quoted 11 lines

There's trailing whitespaces issue here, otherwise looks good to me.

I suppose the seq number can be fixed at commit time, as well as the trailing whitespaces.

Thanks!
Eryu

BTW, test fails on 3.12-rc5 kernel

Yup. (actually it's a userspace bug)

-Eric

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs