Skip to contentSkip to conversation

Count journal size in test 289

PATCH In review Jan Kara 3 participants 4 messages Mar 20, 2013

lore
Share & tools
lore

Jan Kara

author
#0 lore raw

PATCH v2 1/1 Count journal size in test 289

Test 289 ignored the fact that historically journal is not accounted as fs overhead in ext3. For larger filesystems it is hidden in 1% tolerance but for filesystems smaller than 12G the test fails. So make the counting precise to work everywhere.

CC Eric Sandeen <[email protected]>
---
 289           |   16 ++++++++++++----
 common.filter |   15 +++++++++++++++
 2 files changed, 27 insertions(+), 4 deletions(-)
289 +12 -4
index b057c20..eb5c63b 100755
@@ -59,10 +59,18 @@ TOTAL_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
 FREE_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
 		| awk '/Free blocks:/{print $3}'`
 
-# nb: kernels today don't count journal blocks  as overhead, but should.
-# For most filesystems this will still be within tolerance.
-# Overhead is all the blocks (already) used by the fs itself:
-OVERHEAD=$(($TOTAL_BLOCKS-$FREE_BLOCKS))
+# ext3 doesn't count journal blocks as overhead, ext4 does.
+if [ $FSTYP = "ext3" ]; then
+	JOURNAL_SIZE=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+		| awk '/Journal size:/{print $3}' | _filter_size_to_bytes`
+	BLOCK_SIZE=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+		| awk '/Block size:/{print $3}'`
+	JOURNAL_BLOCKS=$(($JOURNAL_SIZE/$BLOCK_SIZE))
+else
+	JOURNAL_BLOCKS=0
+fi
+
+OVERHEAD=$(($TOTAL_BLOCKS-$FREE_BLOCKS-$JOURNAL_BLOCKS))
 
 #  bsddf|minixdf
 #         Set the behaviour  for  the  statfs  system  call.  The  minixdf
common.filter +15 -0
index f0f6076..fcd7589 100644
@@ -229,5 +229,20 @@ _filter_spaces()
        sed -e 's/ [ ]*/ /g'
 }
 
+# Convert string read from stdin like 128K to bytes and print it to stdout
+_filter_size_to_bytes()
+{
+	read size
+	suffix=${size:${#size}-1}
+	mul=1
+	case $suffix in
+		K) mul=1024 ;;
+		M) mul=$((1024*1024)) ;;
+		G) mul=$((1024*1024*1024)) ;;
+		T) mul=$((1024*1024*1024*1024)) ;;
+	esac
+	echo $((${size:0:${#size}-1}*$mul))
+}
+
 # make sure this script returns success
 /bin/true
Signature · 6 lines
1.7.1

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Reviewed-by on v2 1/1
#1 lore raw
Quoted 7 lines · Jan Kara

Test 289 ignored the fact that historically journal is not accounted as fs overhead in ext3. For larger filesystems it is hidden in 1% tolerance but for filesystems smaller than 12G the test fails. So make the counting precise to work everywhere.

CC Eric Sandeen <[email protected]>

Looks fine, thanks.

p.s. - sorry, didn't catch this the first time:

_filter_size_to_bytes won't work for lowercase units, which might be nice. How about:

Review

Quoted 6 lines
(quoted diff) +6 -0
+	case $suffix in
+		k|K) mul=1024 ;;
+		m|M) mul=$((1024*1024)) ;;
+		g|G) mul=$((1024*1024*1024)) ;;
+		t|T) mul=$((1024*1024*1024*1024)) ;;
+	esac

SGI guys - maybe could do that as a small fix-up on commit. Otherwise, if anyone ever needs lower case they could just add it at the same time, so no big deal.

-Eric

Quoted 48 lines
---
 289           |   16 ++++++++++++----
 common.filter |   15 +++++++++++++++
 2 files changed, 27 insertions(+), 4 deletions(-)
289 +12 -4
index b057c20..eb5c63b 100755
@@ -59,10 +59,18 @@ TOTAL_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
 FREE_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
 		| awk '/Free blocks:/{print $3}'`
 
-# nb: kernels today don't count journal blocks  as overhead, but should.
-# For most filesystems this will still be within tolerance.
-# Overhead is all the blocks (already) used by the fs itself:
-OVERHEAD=$(($TOTAL_BLOCKS-$FREE_BLOCKS))
+# ext3 doesn't count journal blocks as overhead, ext4 does.
+if [ $FSTYP = "ext3" ]; then
+	JOURNAL_SIZE=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+		| awk '/Journal size:/{print $3}' | _filter_size_to_bytes`
+	BLOCK_SIZE=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+		| awk '/Block size:/{print $3}'`
+	JOURNAL_BLOCKS=$(($JOURNAL_SIZE/$BLOCK_SIZE))
+else
+	JOURNAL_BLOCKS=0
+fi
+
+OVERHEAD=$(($TOTAL_BLOCKS-$FREE_BLOCKS-$JOURNAL_BLOCKS))
 
 #  bsddf|minixdf
 #         Set the behaviour  for  the  statfs  system  call.  The  minixdf
common.filter +7 -0
index f0f6076..fcd7589 100644
@@ -229,5 +229,20 @@ _filter_spaces()
        sed -e 's/ [ ]*/ /g'
 }
 
+# Convert string read from stdin like 128K to bytes and print it to stdout
+_filter_size_to_bytes()
+{
+	read size
+	suffix=${size:${#size}-1}
+	mul=1
+	case $suffix in

common.filter _filter_spaces() · line 248

Quoted 10 lines
common.filter +8 -0
+		K) mul=1024 ;;
+		M) mul=$((1024*1024)) ;;
+		G) mul=$((1024*1024*1024)) ;;
+		T) mul=$((1024*1024*1024*1024)) ;;
+	esac
+	echo $((${size:0:${#size}-1}*$mul))
+}
+
 # make sure this script returns success
 /bin/true

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#2 lore raw
Quoted 19 lines · Eric Sandeen
Quoted 7 lines · Jan Kara

Test 289 ignored the fact that historically journal is not accounted as fs overhead in ext3. For larger filesystems it is hidden in 1% tolerance but for filesystems smaller than 12G the test fails. So make the counting precise to work everywhere.

CC Eric Sandeen <[email protected]>

Looks fine, thanks.

p.s. - sorry, didn't catch this the first time:

_filter_size_to_bytes won't work for lowercase units, which might be nice. How about:

Quoted 9 lines

Review

Quoted 6 lines
(quoted diff) +6 -0
+	case $suffix in
+		k|K) mul=1024 ;;
+		m|M) mul=$((1024*1024)) ;;
+		g|G) mul=$((1024*1024*1024)) ;;
+		t|T) mul=$((1024*1024*1024*1024)) ;;
+	esac

SGI guys - maybe could do that as a small fix-up on commit. Otherwise, if anyone ever needs lower case they could just add it at the same time, so no big deal.

Will do thanks for the review.

--Rich

Quoted 2 lines

-Eric

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Reviewed-by Eric Sandeen on v2 1/1
#3 lore raw
In reply to Jan Kara

Thanks Jan for submitting this patch, it has been committed with Eric suggestions.

Thanks
--Rich

commit 3574531af49bdde338aff0131100852e87e9d0fd

Author: Jan Kara <[email protected]>
Date:   Wed Mar 20 10:50:48 2013 +0000

xfstests: count journal size in test 289

Test 289 ignored the fact that historically journal is not accounted as fs overhead in ext3. For larger filesystems it is hidden in 1%
tolerance
but for filesystems smaller than 12G the test fails. So make the counting precise to work everywhere.

CC Eric Sandeen <[email protected]>
[[email protected]: add lower case units to filter]

+ read size

289 +12 -4
index b057c20..eb5c63b 100755
@@ -59,10 +59,18 @@ TOTAL_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
 FREE_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
 		| awk '/Free blocks:/{print $3}'`
 
-# nb: kernels today don't count journal blocks  as overhead, but should.
-# For most filesystems this will still be within tolerance.
-# Overhead is all the blocks (already) used by the fs itself:
-OVERHEAD=$(($TOTAL_BLOCKS-$FREE_BLOCKS))
+# ext3 doesn't count journal blocks as overhead, ext4 does.
+if [ $FSTYP = "ext3" ]; then
+	JOURNAL_SIZE=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+		| awk '/Journal size:/{print $3}' | _filter_size_to_bytes`
+	BLOCK_SIZE=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+		| awk '/Block size:/{print $3}'`
+	JOURNAL_BLOCKS=$(($JOURNAL_SIZE/$BLOCK_SIZE))
+else
+	JOURNAL_BLOCKS=0
+fi
+
+OVERHEAD=$(($TOTAL_BLOCKS-$FREE_BLOCKS-$JOURNAL_BLOCKS))
 
 #  bsddf|minixdf
 #         Set the behaviour  for  the  statfs  system  call.  The  minixdf
common.filter +15 -0
index da5675f..bdd6427 100644
@@ -265,5 +265,20 @@ _filter_size()
 	sed -e "s/[0-9\.]\+\s\?[b|k|m|g|t][b]\?/<SIZE>/ig"
 }
 
+# Convert string read from stdin like 128K to bytes and print it to stdout
+_filter_size_to_bytes()
+{
+	read size
+	suffix=${size:${#size}-1}
+	mul=1
+	case $suffix in
+		k|K) mul=1024 ;;
+		m|M) mul=$((1024*1024)) ;;
+		g|G) mul=$((1024*1024*1024)) ;;
+		t|T) mul=$((1024*1024*1024*1024)) ;;
+	esac
+	echo $((${size:0:${#size}-1}*$mul))
+}
+
 # make sure this script returns success
 /bin/true

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs