Skip to contentSkip to conversation

xfstests: Count journal size in test 289

PATCH Superseded Jan Kara 2 participants 4 messages Mar 19, 2013 – Mar 20, 2013

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Jan Kara

author
#0 lore raw

PATCH 1/1 xfstests: Count journal size in test 289

Test 289 ignored the fact that historically journal is not accounted as fs overhead. For larger filesystems it is hidden in 1% tolerance but for filesystems smaller than 12G the test fails. So make the counting precise to work everywhere.

---
 289           |   12 ++++++++----
 common.filter |   15 +++++++++++++++
 2 files changed, 23 insertions(+), 4 deletions(-)
289 +8 -4
index b057c20..9bba144 100755
@@ -59,10 +59,14 @@ TOTAL_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
 FREE_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
 		| awk '/Free blocks:/{print $3}'`
 
-# nb: kernels today don't count journal blocks  as overhead, but should.
-# For most filesystems this will still be within tolerance.
-# Overhead is all the blocks (already) used by the fs itself:
-OVERHEAD=$(($TOTAL_BLOCKS-$FREE_BLOCKS))
+JOURNAL_SIZE=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+		| awk '/Journal size:/{print $3}' | _filter_size_to_bytes`
+BLOCK_SIZE=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+		| awk '/Block size:/{print $3}'`
+JOURNAL_BLOCKS=$(($JOURNAL_SIZE/$BLOCK_SIZE))
+
+# kernels today don't count journal blocks as overhead, but should.
+OVERHEAD=$(($TOTAL_BLOCKS-$FREE_BLOCKS-$JOURNAL_BLOCKS))
 
 #  bsddf|minixdf
 #         Set the behaviour  for  the  statfs  system  call.  The  minixdf
common.filter +15 -0
index f0f6076..fcd7589 100644
@@ -229,5 +229,20 @@ _filter_spaces()
        sed -e 's/ [ ]*/ /g'
 }
 
+# Convert string read from stdin like 128K to bytes and print it to stdout
+_filter_size_to_bytes()
+{
+	read size
+	suffix=${size:${#size}-1}
+	mul=1
+	case $suffix in
+		K) mul=1024 ;;
+		M) mul=$((1024*1024)) ;;
+		G) mul=$((1024*1024*1024)) ;;
+		T) mul=$((1024*1024*1024*1024)) ;;
+	esac
+	echo $((${size:0:${#size}-1}*$mul))
+}
+
 # make sure this script returns success
 /bin/true
Signature · 6 lines
1.7.1

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#1 lore raw
Quoted 4 lines · Jan Kara

Test 289 ignored the fact that historically journal is not accounted as fs overhead. For larger filesystems it is hidden in 1% tolerance but for filesystems smaller than 12G the test fails. So make the counting precise to work everywhere.

Thanks.

0875a2b448fcaba67010850cf9649293a5ef653d ext4: include journal blocks in df overhead calcs

changed this again, right - so will this change work in both cases?

Also: is using "Journal length: " any simpler?

Although I do like the new filter ;)

-Eric

Quoted 45 lines
---
 289           |   12 ++++++++----
 common.filter |   15 +++++++++++++++
 2 files changed, 23 insertions(+), 4 deletions(-)
289 +8 -4
index b057c20..9bba144 100755
@@ -59,10 +59,14 @@ TOTAL_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
 FREE_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
 		| awk '/Free blocks:/{print $3}'`
 
-# nb: kernels today don't count journal blocks  as overhead, but should.
-# For most filesystems this will still be within tolerance.
-# Overhead is all the blocks (already) used by the fs itself:
-OVERHEAD=$(($TOTAL_BLOCKS-$FREE_BLOCKS))
+JOURNAL_SIZE=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+		| awk '/Journal size:/{print $3}' | _filter_size_to_bytes`
+BLOCK_SIZE=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+		| awk '/Block size:/{print $3}'`
+JOURNAL_BLOCKS=$(($JOURNAL_SIZE/$BLOCK_SIZE))
+
+# kernels today don't count journal blocks as overhead, but should.
+OVERHEAD=$(($TOTAL_BLOCKS-$FREE_BLOCKS-$JOURNAL_BLOCKS))
 
 #  bsddf|minixdf
 #         Set the behaviour  for  the  statfs  system  call.  The  minixdf
common.filter +7 -0
index f0f6076..fcd7589 100644
@@ -229,5 +229,20 @@ _filter_spaces()
        sed -e 's/ [ ]*/ /g'
 }
 
+# Convert string read from stdin like 128K to bytes and print it to stdout
+_filter_size_to_bytes()
+{
+	read size
+	suffix=${size:${#size}-1}
+	mul=1
+	case $suffix in

common.filter _filter_spaces() · line 248

Quoted 10 lines
common.filter +8 -0
+		K) mul=1024 ;;
+		M) mul=$((1024*1024)) ;;
+		G) mul=$((1024*1024*1024)) ;;
+		T) mul=$((1024*1024*1024*1024)) ;;
+	esac
+	echo $((${size:0:${#size}-1}*$mul))
+}
+
 # make sure this script returns success
 /bin/true

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

author
#2 lore raw
Quoted 9 lines · Eric Sandeen
Quoted 4 lines · Jan Kara

Test 289 ignored the fact that historically journal is not accounted as fs overhead. For larger filesystems it is hidden in 1% tolerance but for filesystems smaller than 12G the test fails. So make the counting precise to work everywhere.

Thanks.

0875a2b448fcaba67010850cf9649293a5ef653d ext4: include journal blocks in df overhead calcs

Bah, I missed that. But ext3 definitely needs this (and I'm not sure we
want to change that behavior after so many years - it's mostly cosmetic anyway so the possibility of breaking some userspace seems a bad tradeoff).

Quoted 1 line

changed this again, right - so will this change work in both cases?

No, it will now fail for small ext4 filesystems instead of small ext3
filesystems :). So we will count journal blocks only for ext3, ok?

Quoted 1 line

Also: is using "Journal length: " any simpler?

It would be, but older versions of dumpe2fs don't have that (it happened
somewhere between 1.41.9 and 1.41.11) so I figured I will use a more generic approach (too lazy to fix my test machine ;).

Honza

Quoted 56 lines
Quoted 55 lines
---
 289           |   12 ++++++++----
 common.filter |   15 +++++++++++++++
 2 files changed, 23 insertions(+), 4 deletions(-)
289 +8 -4
index b057c20..9bba144 100755
@@ -59,10 +59,14 @@ TOTAL_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
 FREE_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
 		| awk '/Free blocks:/{print $3}'`
 
-# nb: kernels today don't count journal blocks  as overhead, but should.
-# For most filesystems this will still be within tolerance.
-# Overhead is all the blocks (already) used by the fs itself:
-OVERHEAD=$(($TOTAL_BLOCKS-$FREE_BLOCKS))
+JOURNAL_SIZE=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+		| awk '/Journal size:/{print $3}' | _filter_size_to_bytes`
+BLOCK_SIZE=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+		| awk '/Block size:/{print $3}'`
+JOURNAL_BLOCKS=$(($JOURNAL_SIZE/$BLOCK_SIZE))
+
+# kernels today don't count journal blocks as overhead, but should.
+OVERHEAD=$(($TOTAL_BLOCKS-$FREE_BLOCKS-$JOURNAL_BLOCKS))
 
 #  bsddf|minixdf
 #         Set the behaviour  for  the  statfs  system  call.  The  minixdf
common.filter +15 -0
index f0f6076..fcd7589 100644
@@ -229,5 +229,20 @@ _filter_spaces()
        sed -e 's/ [ ]*/ /g'
 }
 
+# Convert string read from stdin like 128K to bytes and print it to stdout
+_filter_size_to_bytes()
+{
+	read size
+	suffix=${size:${#size}-1}
+	mul=1
+	case $suffix in
+		K) mul=1024 ;;
+		M) mul=$((1024*1024)) ;;
+		G) mul=$((1024*1024*1024)) ;;
+		T) mul=$((1024*1024*1024*1024)) ;;
+	esac
+	echo $((${size:0:${#size}-1}*$mul))
+}
+
 # make sure this script returns success
 /bin/true
Signature · 7 lines
Jan Kara <[email protected]>
SUSE Labs, CR

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#3 lore raw
Quoted 7 lines · Jan Kara
Quoted 6 lines · Eric Sandeen
Quoted 4 lines · Jan Kara

Test 289 ignored the fact that historically journal is not accounted as fs overhead. For larger filesystems it is hidden in 1% tolerance but for filesystems smaller than 12G the test fails. So make the counting precise to work everywhere.

Quoted 6 lines
Quoted 3 lines

Thanks.

0875a2b448fcaba67010850cf9649293a5ef653d ext4: include journal blocks in df overhead calcs

Bah, I missed that. But ext3 definitely needs this (and I'm not sure we
want to change that behavior after so many years - it's mostly cosmetic anyway so the possibility of breaking some userspace seems a bad tradeoff).

Sure, that's fine. Ext4 has munged df reporting a few times recently anyway, so figured it was worth sneaking in something to make it more accurate.

Quoted 3 lines
Quoted 1 line

changed this again, right - so will this change work in both cases?

No, it will now fail for small ext4 filesystems instead of small ext3
filesystems :). So we will count journal blocks only for ext3, ok?

Sounds like a plan.

Quoted 4 lines
Quoted 1 line

Also: is using "Journal length: " any simpler?

It would be, but older versions of dumpe2fs don't have that (it happened
somewhere between 1.41.9 and 1.41.11) so I figured I will use a more generic approach (too lazy to fix my test machine ;).

Makes sense :)

Thanks,
-Eric

Quoted 58 lines

Honza

Quoted 56 lines
Quoted 55 lines
---
 289           |   12 ++++++++----
 common.filter |   15 +++++++++++++++
 2 files changed, 23 insertions(+), 4 deletions(-)
289 +8 -4
index b057c20..9bba144 100755
@@ -59,10 +59,14 @@ TOTAL_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
 FREE_BLOCKS=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
 		| awk '/Free blocks:/{print $3}'`
 
-# nb: kernels today don't count journal blocks  as overhead, but should.
-# For most filesystems this will still be within tolerance.
-# Overhead is all the blocks (already) used by the fs itself:
-OVERHEAD=$(($TOTAL_BLOCKS-$FREE_BLOCKS))
+JOURNAL_SIZE=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+		| awk '/Journal size:/{print $3}' | _filter_size_to_bytes`
+BLOCK_SIZE=`dumpe2fs -h $SCRATCH_DEV 2>/dev/null \
+		| awk '/Block size:/{print $3}'`
+JOURNAL_BLOCKS=$(($JOURNAL_SIZE/$BLOCK_SIZE))
+
+# kernels today don't count journal blocks as overhead, but should.
+OVERHEAD=$(($TOTAL_BLOCKS-$FREE_BLOCKS-$JOURNAL_BLOCKS))
 
 #  bsddf|minixdf
 #         Set the behaviour  for  the  statfs  system  call.  The  minixdf
common.filter +15 -0
index f0f6076..fcd7589 100644
@@ -229,5 +229,20 @@ _filter_spaces()
        sed -e 's/ [ ]*/ /g'
 }
 
+# Convert string read from stdin like 128K to bytes and print it to stdout
+_filter_size_to_bytes()
+{
+	read size
+	suffix=${size:${#size}-1}
+	mul=1
+	case $suffix in
+		K) mul=1024 ;;
+		M) mul=$((1024*1024)) ;;
+		G) mul=$((1024*1024*1024)) ;;
+		T) mul=$((1024*1024*1024*1024)) ;;
+	esac
+	echo $((${size:0:${#size}-1}*$mul))
+}
+
 # make sure this script returns success
 /bin/true

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs