Skip to contentSkip to conversation

xfstests: v2 xfs directory unbalance assert test

PATCH Superseded Mark Tinguely 4 participants 9 messages Sep 17, 2013 – Sep 23, 2013

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xfstests: v2 xfs directory unbalance assert test

This tests triggers an assert in the XFS directory unbalance code. This test originally written by Brian Foster and suggestions from Micheal Semon.

---
v1->v2:
remove clean up.
remove $seqres.full before test.
remove scratch mount.
11g -> calculate 11g bytes.
check for failing mkfs and mount.
add a safer removal of the scratch files.

put into auto group and "long" group so people can opt out:
this test almost an hour on my test box.

 tests/generic/319     |   56 ++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/319.out |    2 +
 tests/generic/group   |    1 
 3 files changed, 59 insertions(+)
tests/generic/319 new file +56 -0
Index: b/tests/generic/319
===================================================================
@@ -0,0 +1,56 @@
+#! /bin/bash
+# FS QA Test No. 319
+#
+# Test directory code correctly handles fsstress filling the filesystem
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2013 SGI.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+_require_scratch
+
+# real QA test starts here
+
+_supported_fs generic
+_supported_os IRIX Linux
+
+rm -f $seqres.full
+_scratch_mkfs_sized $((11 * 1024 * 1024 * 1024)) >> $seqres.full 2>&1 \
+	|| _fail "mkfs failed"
+run_check _scratch_mount >> $seqres.full
+
+# Fill the filesystem.
+FSSTRESS_ARGS="-z -s 1378390208 -fsymlink=1 -n7500000 -p4 -d $SCRATCH_MNT"
+$FSSTRESS_PROG $FSSTRESS_ARGS >> $seqres.full 2>&1
+
+# A debug XFS may assert in the remove due to a directory bug.
+rm -rf $SCRATCH_MNT/p?
+echo "--- silence is golden ---"
+status=0
+exit
tests/generic/319.out new file +2 -0
Index: b/tests/generic/319.out
===================================================================
@@ -0,0 +1,2 @@
+QA output created by 319
+--- silence is golden ---
tests/generic/group +1 -0
Index: b/tests/generic/group
===================================================================
@@ -121,3 +121,4 @@
 316 auto quick
 317 auto metadata quick
 318 acl attr auto quick
+319 auto stress long

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Reviewed-by on v1 1/1
#1 lore raw
Quoted 14 lines · Mark Tinguely

This tests triggers an assert in the XFS directory unbalance code. This test originally written by Brian Foster and suggestions from Micheal Semon.

---
v1->v2:
remove clean up.
remove $seqres.full before test.
remove scratch mount.
11g -> calculate 11g bytes.
check for failing mkfs and mount.
add a safer removal of the scratch files.

Quoted 2 lines

put into auto group and "long" group so people can opt out:
this test almost an hour on my test box.

Is there any way to do a more targeted test than 1hr of fsstress? :(

(TBH I didn't follow the bug, so not sure what it takes to repro)

Otherwise seems fine.

Quoted 84 lines
 tests/generic/319     |   56 ++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/319.out |    2 +
 tests/generic/group   |    1 
 3 files changed, 59 insertions(+)

Index: b/tests/generic/319
===================================================================

tests/generic/319 new file +56 -0
@@ -0,0 +1,56 @@
+#! /bin/bash
+# FS QA Test No. 319
+#
+# Test directory code correctly handles fsstress filling the filesystem
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2013 SGI.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+_require_scratch
+
+# real QA test starts here
+
+_supported_fs generic
+_supported_os IRIX Linux
+
+rm -f $seqres.full
+_scratch_mkfs_sized $((11 * 1024 * 1024 * 1024)) >> $seqres.full 2>&1 \
+	|| _fail "mkfs failed"
+run_check _scratch_mount >> $seqres.full
+
+# Fill the filesystem.
+FSSTRESS_ARGS="-z -s 1378390208 -fsymlink=1 -n7500000 -p4 -d $SCRATCH_MNT"
+$FSSTRESS_PROG $FSSTRESS_ARGS >> $seqres.full 2>&1
+
+# A debug XFS may assert in the remove due to a directory bug.
+rm -rf $SCRATCH_MNT/p?
+echo "--- silence is golden ---"
+status=0
+exit

Index: b/tests/generic/319.out
===================================================================

tests/generic/319.out new file +2 -0
@@ -0,0 +1,2 @@
+QA output created by 319
+--- silence is golden ---

Index: b/tests/generic/group
===================================================================

tests/generic/group +1 -0
@@ -121,3 +121,4 @@
 316 auto quick
 317 auto metadata quick
 318 acl attr auto quick
+319 auto stress long
Quoted 5 lines

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#2 lore raw
Quoted 15 lines · Eric Sandeen
Quoted 14 lines · Mark Tinguely

This tests triggers an assert in the XFS directory unbalance code. This test originally written by Brian Foster and suggestions from Micheal Semon.

---
v1->v2:
remove clean up.
remove $seqres.full before test.
remove scratch mount.
11g -> calculate 11g bytes.
check for failing mkfs and mount.
add a safer removal of the scratch files.

Quoted 4 lines
Quoted 2 lines

put into auto group and "long" group so people can opt out:
this test almost an hour on my test box.

Is there any way to do a more targeted test than 1hr of fsstress? :(

No, I pruned the test down from a 5 hr test to 1 hr test.

Quoted 6 lines

(TBH I didn't follow the bug, so not sure what it takes to repro)

Otherwise seems fine.

Thanks.

--Mark.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#3 lore raw
In reply to Mark Tinguely
Quoted 14 lines · Mark Tinguely

This tests triggers an assert in the XFS directory unbalance code. This test originally written by Brian Foster and suggestions from Micheal Semon.

---
v1->v2:
remove clean up.
remove $seqres.full before test.
remove scratch mount.
11g -> calculate 11g bytes.
check for failing mkfs and mount.
add a safer removal of the scratch files.

Quoted 2 lines

put into auto group and "long" group so people can opt out:
this test almost an hour on my test box.

45m is too long for the auto group. The entire auto group runs in 50-55 minutes on my test VMs, so doubling that for one test is not really a good idea.

The usual rule of thumb for the auto group is that the test should take less than 5 minutes to run. For the quick group, it should take less than 30-60s to run.

If you want to make this test run in the auto group, scale it for stress tests via the use of LOAD_FACTOR and TIME_FACTOR, and add it to the stress group.

Cheers,

Dave.

Signature · 7 lines
Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#4 lore raw
Quoted 15 lines · Dave Chinner
Quoted 14 lines · Mark Tinguely

This tests triggers an assert in the XFS directory unbalance code. This test originally written by Brian Foster and suggestions from Micheal Semon.

---
v1->v2:
remove clean up.
remove $seqres.full before test.
remove scratch mount.
11g -> calculate 11g bytes.
check for failing mkfs and mount.
add a safer removal of the scratch files.

Quoted 6 lines
Quoted 2 lines

put into auto group and "long" group so people can opt out:
this test almost an hour on my test box.

45m is too long for the auto group. The entire auto group runs in 50-55 minutes on my test VMs, so doubling that for one test is not really a good idea.

Ok, sorry, my fault. Fix on commit?

-Eric

Quoted 4 lines

The usual rule of thumb for the auto group is that the test should take less than 5 minutes to run. For the quick group, it should take less than 30-60s to run.

Quoted 8 lines

If you want to make this test run in the auto group, scale it for stress tests via the use of LOAD_FACTOR and TIME_FACTOR, and add it to the stress group.

Cheers,

Dave.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#5 lore raw
Quoted 19 lines · Eric Sandeen
Quoted 18 lines · Dave Chinner
Quoted 16 lines · Mark Tinguely

This tests triggers an assert in the XFS directory unbalance code. This test originally written by Brian Foster and suggestions from Micheal Semon.

---
v1->v2:
remove clean up.
remove $seqres.full before test.
remove scratch mount.
11g -> calculate 11g bytes.
check for failing mkfs and mount.
add a safer removal of the scratch files.

put into auto group and "long" group so people can opt out:
this test almost an hour on my test box.

Quoted 7 lines
Quoted 3 lines

45m is too long for the auto group. The entire auto group runs in 50-55 minutes on my test VMs, so doubling that for one test is not really a good idea.

Ok, sorry, my fault. Fix on commit?

-Eric

We can't remove much more and still hit the assert, so I am happy to remove it from the auto group.

--Mark.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#6 lore raw
In reply to Mark Tinguely

Looks good. I tried it on an 11-GB partition with an unpatched git kernel 3.12.0 + xfs-oss/master, coupled with the latest git xfsprogs. It took a reasonable amount of time, relatively speaking. It put forth the error I was expecting, and a new 32-bit core is available upon your request but not uploaded yet.

As for the assert itself, according to kgdb, these are the first, last, and BBTOB(bp->b_length) numbers for this run:

1544 4591 4096

Good work!

Michael

Quoted 101 lines · Mark Tinguely

This tests triggers an assert in the XFS directory unbalance code. This test originally written by Brian Foster and suggestions from Micheal Semon.

---
v1->v2:
remove clean up.
remove $seqres.full before test.
remove scratch mount.
11g -> calculate 11g bytes.
check for failing mkfs and mount.
add a safer removal of the scratch files.

put into auto group and "long" group so people can opt out:
this test almost an hour on my test box.

 tests/generic/319     |   56 ++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/319.out |    2 +
 tests/generic/group   |    1
 3 files changed, 59 insertions(+)

Index: b/tests/generic/319
===================================================================

tests/generic/319 new file +56 -0
@@ -0,0 +1,56 @@
+#! /bin/bash
+# FS QA Test No. 319
+#
+# Test directory code correctly handles fsstress filling the filesystem
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2013 SGI.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1       # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+_require_scratch
+
+# real QA test starts here
+
+_supported_fs generic
+_supported_os IRIX Linux
+
+rm -f $seqres.full
+_scratch_mkfs_sized $((11 * 1024 * 1024 * 1024)) >> $seqres.full 2>&1 \
+       || _fail "mkfs failed"
+run_check _scratch_mount >> $seqres.full
+
+# Fill the filesystem.
+FSSTRESS_ARGS="-z -s 1378390208 -fsymlink=1 -n7500000 -p4 -d $SCRATCH_MNT"
+$FSSTRESS_PROG $FSSTRESS_ARGS >> $seqres.full 2>&1
+
+# A debug XFS may assert in the remove due to a directory bug.
+rm -rf $SCRATCH_MNT/p?
+echo "--- silence is golden ---"
+status=0
+exit

Index: b/tests/generic/319.out
===================================================================

tests/generic/319.out new file +2 -0
@@ -0,0 +1,2 @@
+QA output created by 319
+--- silence is golden ---

Index: b/tests/generic/group
===================================================================

tests/generic/group +1 -0
@@ -121,3 +121,4 @@
 316 auto quick
 317 auto metadata quick
 318 acl attr auto quick
+319 auto stress long
Quoted 4 lines

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#7 lore raw
Quoted 6 lines · Michael L. Semon

Looks good. I tried it on an 11-GB partition with an unpatched git kernel 3.12.0 + xfs-oss/master, coupled with the latest git xfsprogs. It took a reasonable amount of time, relatively speaking. It put forth the error I was expecting, and a new 32-bit core is available upon your request but not uploaded yet.

Quoted 8 lines

As for the assert itself, according to kgdb, these are the first, last, and BBTOB(bp->b_length) numbers for this run:

1544 4591 4096

Good work!

Michael

Thanks for the feedback.

last > BBTOB(bp->b_length) was also true for the longer version of the test. The only difference is that it happens earlier in the remove.

I take a closer look at the code.

--Mark.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#8 lore raw
In reply to Mark Tinguely
Quoted 5 lines · Mark Tinguely

This tests triggers an assert in the XFS directory unbalance code. This test originally written by Brian Foster and suggestions from Micheal Semon.

A recent comment about the _cleanup funtion being removed made me look at this again....

Where's the trap function to catch a ctrl-c and clean up the test when that happens?

i.e. at minimum, the trap and cleanup function is used to ensure the test is marked as failed if it is interrupted, and in this case it probably needs to kill all the fsstress processes and wait for them to complete, as well as remove

Quoted 2 lines

put into auto group and "long" group so people can opt out:
this test almost an hour on my test box.

The group for this is called "stress" - there's no need to introduce a new group for the same test criteria.

Cheers,

Dave.

Signature · 7 lines
Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs