Skip to contentSkip to conversation

xfstests 285: verify extN statfs f_blocks

PATCH In review Eric Sandeen 2 participants 3 messages Sep 5, 2012 – Sep 6, 2012

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xfstests 285: verify extN statfs f_blocks

extN can report f_blocks in statfs in 2 different ways, based on whether or not metadata overhead is counted. This has broken in the past, so here's a test for it.

It looks at dumpe2fs output to get total blocks and free blocks right after mkfs. The difference should be, by definition, the exact amount of metadata overhead.

It then compares this to what's reported via stat -f for f_blocks. For "minix" df, it should be exactly equal to the total blocks, and for "bsd" df, it should be total blocks less overhead.

It tests that the latter is accurate to within a 1% tolerance.

Today the journal doesn't count as overhead in the statfs call; this should be fixed kernelside but for many filesystems it'll be within the threshold anyway.

---

285 new file +97 -0
new file mode 100755
index 0000000..cda8531
@@ -0,0 +1,97 @@
+#! /bin/bash
+# FS QA Test No. 286
+#
+# Test overhead & df output for extN filesystems
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2012 Red Hat, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs ext2 ext3 ext4
+_supported_os Linux
+_require_scratch
+
+rm -f $seq.full
+
+_scratch_mkfs >> $seq.full 2>&1
+
+TOTAL_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+		| awk '/Block count:/{print $3}'`
+
+FREE_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+		| awk '/Free blocks:/{print $3}'`
+
+# nb: kernels today don't count journal blocks  as overhead, but should.
+# For most fileystems this will still be within tolerance.
+OVERHEAD=$(($TOTAL_BLOCKS-$FREE_BLOCKS))
+
+#  bsddf|minixdf
+#         Set the behaviour  for  the  statfs  system  call.  The  minixdf
+#         behaviour is to return in the f_blocks field the total number of
+#         blocks of the filesystem, while the bsddf  behaviour  (which  is
+#         the default) is to subtract the overhead blocks used by the ext2
+#         filesystem and not available for file storage.
+
+# stat -f output looks like; we get f_blocks from that, which
+# varies depending on the df mount options used below:
+
+# Filesystem           4K-blocks      Used Available Use% Mounted on
+# /dev/sda8              2405312   2208933    196379  92% /
+
+_scratch_mount "-o minixdf"
+MINIX_F_BLOCKS=`stat -f $SCRATCH_MNT | awk '/^Blocks/{print $3}'`
+umount $SCRATCH_MNT
+
+_scratch_mount "-o bsddf"
+BSD_F_BLOCKS=`stat -f $SCRATCH_MNT | awk '/^Blocks/{print $3}'`
+umount $SCRATCH_MNT
+
+echo "Overhead is $OVERHEAD out of $TOTAL_BLOCKS" >> $seq.full
+echo "BSD blocks $BSD_F_BLOCKS" >> $seq.full
+echo "MINIX blocks $MINIX_F_BLOCKS" >> $seq.full
+
+# minix should be exactly equal (hence 0)
+_within_tolerance "minix f_blocks" $MINIX_F_BLOCKS $TOTAL_BLOCKS 0 -v
+# bsd should be within ... we'll say 1%
+_within_tolerance "bsd f_blocks" $BSD_F_BLOCKS $(($TOTAL_BLOCKS-$OVERHEAD)) 1% -v
+
+# success, all done
+status=0
+exit
285.out new file +3 -0
new file mode 100644
index 0000000..2075e21
@@ -0,0 +1,3 @@
+QA output created by 285
+minix f_blocks is in range
+bsd f_blocks is in range
group +1 -0
index 104ed35..0b33178 100644
@@ -403,3 +403,4 @@ deprecated
 282 dump ioctl auto quick
 283 dump ioctl auto quick
 284 auto
+285 auto

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#1 lore raw
Quoted 27 lines · Eric Sandeen

extN can report f_blocks in statfs in 2 different ways, based on whether or not metadata overhead is counted. This has broken in the past, so here's a test for it.

It looks at dumpe2fs output to get total blocks and free blocks right after mkfs. The difference should be, by definition, the exact amount of metadata overhead.

It then compares this to what's reported via stat -f for f_blocks. For "minix" df, it should be exactly equal to the total blocks, and for "bsd" df, it should be total blocks less overhead.

It tests that the latter is accurate to within a 1% tolerance.

Today the journal doesn't count as overhead in the statfs call; this should be fixed kernelside but for many filesystems it'll be within the threshold anyway.

---

285 new file +0 -0
new file mode 100755
index 0000000..cda8531

285 line 2

Quoted 2 lines
285 new file +2 -0
@@ -0,0 +1,97 @@
+#! /bin/bash
+# FS QA Test No. 286

The Test No. should be 285 here?

Quoted 47 lines
285 new file +47 -0
+#
+# Test overhead & df output for extN filesystems
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2012 Red Hat, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1       # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs ext2 ext3 ext4
+_supported_os Linux
+_require_scratch

285 line 59

Quoted 10 lines
285 new file +10 -0
+
+rm -f $seq.full
+
+_scratch_mkfs >> $seq.full 2>&1
+
+TOTAL_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+               | awk '/Block count:/{print $3}'`
+
+FREE_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+               | awk '/Free blocks:/{print $3}'`

I guess TOTAL_BLOCKS and FREE_BLOCKS will be wrong when testing ext4 on RHEL5 which has a separate dumpe4fs for ext4.

Thanks,
Eryu Guan

Quoted 58 lines
285 new file +38 -0
+
+# nb: kernels today don't count journal blocks  as overhead, but should.
+# For most fileystems this will still be within tolerance.
+OVERHEAD=$(($TOTAL_BLOCKS-$FREE_BLOCKS))
+
+#  bsddf|minixdf
+#         Set the behaviour  for  the  statfs  system  call.  The  minixdf
+#         behaviour is to return in the f_blocks field the total number of
+#         blocks of the filesystem, while the bsddf  behaviour  (which  is
+#         the default) is to subtract the overhead blocks used by the ext2
+#         filesystem and not available for file storage.
+
+# stat -f output looks like; we get f_blocks from that, which
+# varies depending on the df mount options used below:
+
+# Filesystem           4K-blocks      Used Available Use% Mounted on
+# /dev/sda8              2405312   2208933    196379  92% /
+
+_scratch_mount "-o minixdf"
+MINIX_F_BLOCKS=`stat -f $SCRATCH_MNT | awk '/^Blocks/{print $3}'`
+umount $SCRATCH_MNT
+
+_scratch_mount "-o bsddf"
+BSD_F_BLOCKS=`stat -f $SCRATCH_MNT | awk '/^Blocks/{print $3}'`
+umount $SCRATCH_MNT
+
+echo "Overhead is $OVERHEAD out of $TOTAL_BLOCKS" >> $seq.full
+echo "BSD blocks $BSD_F_BLOCKS" >> $seq.full
+echo "MINIX blocks $MINIX_F_BLOCKS" >> $seq.full
+
+# minix should be exactly equal (hence 0)
+_within_tolerance "minix f_blocks" $MINIX_F_BLOCKS $TOTAL_BLOCKS 0 -v
+# bsd should be within ... we'll say 1%
+_within_tolerance "bsd f_blocks" $BSD_F_BLOCKS $(($TOTAL_BLOCKS-$OVERHEAD)) 1% -v
+
+# success, all done
+status=0
+exit
285.out new file +3 -0
new file mode 100644
index 0000000..2075e21
@@ -0,0 +1,3 @@
+QA output created by 285
+minix f_blocks is in range
+bsd f_blocks is in range
group +1 -1
index 104ed35..0b33178 100644
@@ -403,3 +403,4 @@ deprecated
 282 dump ioctl auto quick
 283 dump ioctl auto quick
 284 auto
+285 auto
 
--
Quoted 3 lines

To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#2 lore raw
Quoted 22 lines · Eryu Guan
Quoted 21 lines · Eric Sandeen

extN can report f_blocks in statfs in 2 different ways, based on whether or not metadata overhead is counted. This has broken in the past, so here's a test for it.

It looks at dumpe2fs output to get total blocks and free blocks right after mkfs. The difference should be, by definition, the exact amount of metadata overhead.

It then compares this to what's reported via stat -f for f_blocks. For "minix" df, it should be exactly equal to the total blocks, and for "bsd" df, it should be total blocks less overhead.

It tests that the latter is accurate to within a 1% tolerance.

Today the journal doesn't count as overhead in the statfs call; this should be fixed kernelside but for many filesystems it'll be within the threshold anyway.

---

Quoted 10 lines

285 line 2

Quoted 8 lines
285 new file +2 -0
new file mode 100755
index 0000000..cda8531
@@ -0,0 +1,97 @@
+#! /bin/bash
+# FS QA Test No. 286

The Test No. should be 285 here?

Oh, yes. I figure I'll have to renumber before commit anyway ;)

Quoted 52 lines
Quoted 52 lines
285 new file +52 -0
+#
+# Test overhead & df output for extN filesystems
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2012 Red Hat, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1       # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs ext2 ext3 ext4
+_supported_os Linux
+_require_scratch
+
+rm -f $seq.full
+
+_scratch_mkfs >> $seq.full 2>&1
+
Quoted 8 lines

285 line 59

Quoted 5 lines
285 new file +5 -0
+TOTAL_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+               | awk '/Block count:/{print $3}'`
+
+FREE_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+               | awk '/Free blocks:/{print $3}'`

I guess TOTAL_BLOCKS and FREE_BLOCKS will be wrong when testing ext4 on RHEL5 which has a separate dumpe4fs for ext4.

dumpe2fs would fail, yes ... I don't know if we really want to special case rhel5's weirdness in xfstests or not ...

-Eric

Quoted 59 lines

Thanks,
Eryu Guan

Quoted 57 lines
285 new file +38 -0
+
+# nb: kernels today don't count journal blocks  as overhead, but should.
+# For most fileystems this will still be within tolerance.
+OVERHEAD=$(($TOTAL_BLOCKS-$FREE_BLOCKS))
+
+#  bsddf|minixdf
+#         Set the behaviour  for  the  statfs  system  call.  The  minixdf
+#         behaviour is to return in the f_blocks field the total number of
+#         blocks of the filesystem, while the bsddf  behaviour  (which  is
+#         the default) is to subtract the overhead blocks used by the ext2
+#         filesystem and not available for file storage.
+
+# stat -f output looks like; we get f_blocks from that, which
+# varies depending on the df mount options used below:
+
+# Filesystem           4K-blocks      Used Available Use% Mounted on
+# /dev/sda8              2405312   2208933    196379  92% /
+
+_scratch_mount "-o minixdf"
+MINIX_F_BLOCKS=`stat -f $SCRATCH_MNT | awk '/^Blocks/{print $3}'`
+umount $SCRATCH_MNT
+
+_scratch_mount "-o bsddf"
+BSD_F_BLOCKS=`stat -f $SCRATCH_MNT | awk '/^Blocks/{print $3}'`
+umount $SCRATCH_MNT
+
+echo "Overhead is $OVERHEAD out of $TOTAL_BLOCKS" >> $seq.full
+echo "BSD blocks $BSD_F_BLOCKS" >> $seq.full
+echo "MINIX blocks $MINIX_F_BLOCKS" >> $seq.full
+
+# minix should be exactly equal (hence 0)
+_within_tolerance "minix f_blocks" $MINIX_F_BLOCKS $TOTAL_BLOCKS 0 -v
+# bsd should be within ... we'll say 1%
+_within_tolerance "bsd f_blocks" $BSD_F_BLOCKS $(($TOTAL_BLOCKS-$OVERHEAD)) 1% -v
+
+# success, all done
+status=0
+exit
285.out new file +3 -0
new file mode 100644
index 0000000..2075e21
@@ -0,0 +1,3 @@
+QA output created by 285
+minix f_blocks is in range
+bsd f_blocks is in range
group +1 -0
index 104ed35..0b33178 100644
@@ -403,3 +403,4 @@ deprecated
 282 dump ioctl auto quick
 283 dump ioctl auto quick
 284 auto
+285 auto
Quoted 4 lines
Quoted 4 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs