Review: fix test 004 to account for reserved space
With the changes to use some space by default in only in memory as a reserved pool, df and statfs will now output a fre block count that is slightly different to what is held in the superblock.
Update the qa test to account for this change.
Cheers,
Dave.
Signature · 59 lines
Dave Chinner
Principal Engineer
SGI Australian Software Group
---
xfstests/004 | 35 +++++++++++++++++++++++++----------
1 file changed, 25 insertions(+), 10 deletions(-)
Index: xfs-cmds/xfstests/004
===================================================================
--- xfs-cmds.orig/xfstests/004 2006-11-14 19:57:39.000000000 +1100
+++ xfs-cmds/xfstests/004 2007-05-04 16:38:03.957537306 +1000
@@ -67,21 +67,36 @@ xfs_db -r -c "freesp -s" $SCRATCH_DEV >$
echo "xfs_db for $SCRATCH_DEV" >>$seq.full
cat $tmp.xfs_db >>$seq.full
+eval `$XFS_IO_PROG -x -c resblks $SCRATCH_MNT 2>&1 \
+ | $AWK_PROG '/available/ { printf "resblks=%u\n", $5 }'`
+echo "resblks gave: resblks=$resblks" >>$seq.full
+
# check the 'blocks' field from freesp command is OK
# since 2.6.18, df does not report the 4 blocks per AG that cannot
# be allocated, hence we check for that exact mismatch.
+# since ~2.6.22, reserved blocks are used by default and df does
+# not report them, hence check for an exact mismatch.
perl -ne '
- BEGIN { $avail ='$avail' * 512;
- $answer="(no xfs_db free blocks line?)" }
- /free blocks (\d+)$/ || next;
- $freesp = $1 * '$dbsize';
- if ($freesp == $avail) { $answer = "yes"; }
- else {
+ BEGIN { $avail ='$avail' * 512;
+ $answer="(no xfs_db free blocks line?)" }
+ /free blocks (\d+)$/ || next;
+ $freesp = $1 * '$dbsize';
+ if ($freesp == $avail) {
+ $answer = "yes";
+ } else {
$avail = $avail + (('$agcount' + 1) * '$dbsize' * 4);
- if ($freesp == $avail) { $answer = "yes"; }
- else { $answer = "no ($freesp != $avail)"; }
- }
- END { print "$answer\n" }
+ if ($freesp == $avail) {
+ $answer = "yes";
+ } else {
+ $avail = $avail + ('$resblks' * '$dbsize');
+ if ($freesp == $avail) {
+ $answer = "yes";
+ } else {
+ $answer = "no ($freesp != $avail)";
+ }
+ }
+ }
+ END { print "$answer\n" }
' <$tmp.xfs_db >$tmp.ans
ans="`cat $tmp.ans`"
echo "Checking blocks column same as df: $ans"Ping?
Quoted 69 lines · David Chinner
With the changes to use some space by default in only in memory as a reserved pool, df and statfs will now output a fre block count that is slightly different to what is held in the superblock.
Update the qa test to account for this change.
Cheers,
Dave.
Signature · 10 lines
Dave Chinner Principal Engineer SGI Australian Software Group --- xfstests/004 | 35 +++++++++++++++++++++++++---------- 1 file changed, 25 insertions(+), 10 deletions(-) Index: xfs-cmds/xfstests/004 ===================================================================xfstests/004 +25
-10@@ -67,21 +67,36 @@ xfs_db -r -c "freesp -s" $SCRATCH_DEV >$ echo "xfs_db for $SCRATCH_DEV" >>$seq.full cat $tmp.xfs_db >>$seq.full +eval `$XFS_IO_PROG -x -c resblks $SCRATCH_MNT 2>&1 \ + | $AWK_PROG '/available/ { printf "resblks=%u\n", $5 }'` +echo "resblks gave: resblks=$resblks" >>$seq.full + # check the 'blocks' field from freesp command is OK # since 2.6.18, df does not report the 4 blocks per AG that cannot # be allocated, hence we check for that exact mismatch. +# since ~2.6.22, reserved blocks are used by default and df does +# not report them, hence check for an exact mismatch. perl -ne ' - BEGIN { $avail ='$avail' * 512; - $answer="(no xfs_db free blocks line?)" } - /free blocks (\d+)$/ || next; - $freesp = $1 * '$dbsize'; - if ($freesp == $avail) { $answer = "yes"; } - else { + BEGIN { $avail ='$avail' * 512; + $answer="(no xfs_db free blocks line?)" } + /free blocks (\d+)$/ || next; + $freesp = $1 * '$dbsize'; + if ($freesp == $avail) { + $answer = "yes"; + } else { $avail = $avail + (('$agcount' + 1) * '$dbsize' * 4); - if ($freesp == $avail) { $answer = "yes"; } - else { $answer = "no ($freesp != $avail)"; } - } - END { print "$answer\n" } + if ($freesp == $avail) { + $answer = "yes"; + } else { + $avail = $avail + ('$resblks' * '$dbsize'); + if ($freesp == $avail) { + $answer = "yes"; + } else { + $answer = "no ($freesp != $avail)"; + } + } + } + END { print "$answer\n" } ' <$tmp.xfs_db >$tmp.ans ans="`cat $tmp.ans`" echo "Checking blocks column same as df: $ans"
Signature · 3 lines
Dave Chinner Principal Engineer SGI Australian Software Group
I have to go home now, but I'll look at this one soon.
--Tim
Quoted 73 lines · David Chinner
Ping?
Quoted 69 lines · David Chinner
With the changes to use some space by default in only in memory as a reserved pool, df and statfs will now output a fre block count that is slightly different to what is held in the superblock.
Update the qa test to account for this change.
Cheers,
Dave.
Signature · 10 lines
Dave Chinner Principal Engineer SGI Australian Software Group --- xfstests/004 | 35 +++++++++++++++++++++++++---------- 1 file changed, 25 insertions(+), 10 deletions(-) Index: xfs-cmds/xfstests/004 ===================================================================xfstests/004 +25
-10@@ -67,21 +67,36 @@ xfs_db -r -c "freesp -s" $SCRATCH_DEV >$ echo "xfs_db for $SCRATCH_DEV" >>$seq.full cat $tmp.xfs_db >>$seq.full +eval `$XFS_IO_PROG -x -c resblks $SCRATCH_MNT 2>&1 \ + | $AWK_PROG '/available/ { printf "resblks=%u\n", $5 }'` +echo "resblks gave: resblks=$resblks" >>$seq.full + # check the 'blocks' field from freesp command is OK # since 2.6.18, df does not report the 4 blocks per AG that cannot # be allocated, hence we check for that exact mismatch. +# since ~2.6.22, reserved blocks are used by default and df does +# not report them, hence check for an exact mismatch. perl -ne ' - BEGIN { $avail ='$avail' * 512; - $answer="(no xfs_db free blocks line?)" } - /free blocks (\d+)$/ || next; - $freesp = $1 * '$dbsize'; - if ($freesp == $avail) { $answer = "yes"; } - else { + BEGIN { $avail ='$avail' * 512; + $answer="(no xfs_db free blocks line?)" } + /free blocks (\d+)$/ || next; + $freesp = $1 * '$dbsize'; + if ($freesp == $avail) { + $answer = "yes"; + } else { $avail = $avail + (('$agcount' + 1) * '$dbsize' * 4); - if ($freesp == $avail) { $answer = "yes"; } - else { $answer = "no ($freesp != $avail)"; } - } - END { print "$answer\n" } + if ($freesp == $avail) { + $answer = "yes"; + } else { + $avail = $avail + ('$resblks' * '$dbsize'); + if ($freesp == $avail) { + $answer = "yes"; + } else { + $answer = "no ($freesp != $avail)"; + } + } + } + END { print "$answer\n" } ' <$tmp.xfs_db >$tmp.ans ans="`cat $tmp.ans`" echo "Checking blocks column same as df: $ans"
Hi Dave,
Yes that looks not bad.
However, your last proposal, prompted from Nathan, was to adjust f_bavail differently than bfree to account for how much was available in the reserved block space.
struct statfs {
long f_type; /* type of filesystem (see below) */ long f_bsize; /* optimal transfer block size */ long f_blocks; /* total data blocks in file system */ long f_bfree; /* free blocks in fs */ <------ includes reserved long f_bavail; /* free blocks avail to non-superuser */ <------ doesn't include reserved long f_files; /* total file nodes in file system */ long f_ffree; /* free file nodes in fs */ fsid_t f_fsid; /* file system id */ long f_namelen; /* maximum length of filenames */ long f_spare[6]; /* spare for later */
};
And looking at df code it uses bfree in calculation of space used... the 3rd field of df output.
df code:
---------------------------------------------------------------
total = fsu.fsu_blocks;
available = fsu.fsu_bavail;
negate_available = (fsu.fsu_bavail_top_bit_set
& (available != UINTMAX_MAX));
available_to_root = fsu.fsu_bfree;}
used = UINTMAX_MAX;negate_used = false;
if (total != UINTMAX_MAX && available_to_root != UINTMAX_MAX)
{
used = total - available_to_root;
negate_used = (total < available_to_root);}
printf (" %*s %*s %*s ",
width + col1_adjustment,
df_readable (false, total,
buf[0], input_units, output_units),
width, df_readable (negate_used, used,
buf[1], input_units, output_units),
width, df_readable (negate_available, available,
buf[2], input_units, output_units));
-----------------------------------------------------------------
example df output:
Filesystem 1K-blocks Used Available Use% Mounted on/dev/sda3 18864128 7476720 11387408 40% /
-------------------------------------------------------------------
And indeed we set "used" as a shell variable in test 004 from df but don't seem to do anything with it.
I think we should check it to verify that we have set bfree correctly.
Cheers,
Tim.
Quoted 76 lines · David Chinner
Ping?
Quoted 70 lines · David Chinner
With the changes to use some space by default in only in memory as a reserved pool, df and statfs will now output a fre block count that is slightly different to what is held in the superblock.
Update the qa test to account for this change.
Cheers,
Dave.
Signature · 10 lines
Dave Chinner Principal Engineer SGI Australian Software Group --- xfstests/004 | 35 +++++++++++++++++++++++++---------- 1 file changed, 25 insertions(+), 10 deletions(-) Index: xfs-cmds/xfstests/004 ===================================================================xfstests/004 +25
-10@@ -67,21 +67,36 @@ xfs_db -r -c "freesp -s" $SCRATCH_DEV >$ echo "xfs_db for $SCRATCH_DEV" >>$seq.full cat $tmp.xfs_db >>$seq.full +eval `$XFS_IO_PROG -x -c resblks $SCRATCH_MNT 2>&1 \ + | $AWK_PROG '/available/ { printf "resblks=%u\n", $5 }'` +echo "resblks gave: resblks=$resblks" >>$seq.full + # check the 'blocks' field from freesp command is OK # since 2.6.18, df does not report the 4 blocks per AG that cannot # be allocated, hence we check for that exact mismatch. +# since ~2.6.22, reserved blocks are used by default and df does +# not report them, hence check for an exact mismatch. perl -ne ' - BEGIN { $avail ='$avail' * 512; - $answer="(no xfs_db free blocks line?)" } - /free blocks (\d+)$/ || next; - $freesp = $1 * '$dbsize'; - if ($freesp == $avail) { $answer = "yes"; } - else { + BEGIN { $avail ='$avail' * 512; + $answer="(no xfs_db free blocks line?)" } + /free blocks (\d+)$/ || next; + $freesp = $1 * '$dbsize'; + if ($freesp == $avail) { + $answer = "yes"; + } else { $avail = $avail + (('$agcount' + 1) * '$dbsize' * 4); - if ($freesp == $avail) { $answer = "yes"; } - else { $answer = "no ($freesp != $avail)"; } - } - END { print "$answer\n" } + if ($freesp == $avail) { + $answer = "yes"; + } else { + $avail = $avail + ('$resblks' * '$dbsize'); + if ($freesp == $avail) { + $answer = "yes"; + } else { + $answer = "no ($freesp != $avail)"; + } + } + } + END { print "$answer\n" } ' <$tmp.xfs_db >$tmp.ans ans="`cat $tmp.ans`" echo "Checking blocks column same as df: $ans"
Quoted 5 lines · David Chinner
With the changes to use some space by default in only in memory as a reserved pool, df and statfs will now output a fre block count that is slightly different to what is held in the superblock.
Update the qa test to account for this change.
I think we should rather subtract the amount of internally reserved blocks from the return value in xfs_statvfs.
Quoted 9 lines · Christoph Hellwig
Quoted 5 lines · David Chinner
With the changes to use some space by default in only in memory as a reserved pool, df and statfs will now output a fre block count that is slightly different to what is held in the superblock.
Update the qa test to account for this change.
I think we should rather subtract the amount of internally reserved blocks from the return value in
xfs_statvfs.
Which return value?
With this patch:
--- fs/xfs/xfs_vfsops.c | 1 + 1 file changed, 1 insertion(+)
fs/xfs/xfs_vfsops.c +1 -0
@@ -876,6 +876,7 @@ xfs_statvfs( statp->f_blocks = sbp->sb_dblocks - lsize; statp->f_bfree = statp->f_bavail = sbp->sb_fdblocks -XFS_ALLOC_SET_ASIDE (mp); + statp->f_bfree += mp->m_resblks_avail; fakeinos = statp->f_bfree << sbp->sb_inopblog;#if XFS_BIG_INUMS fakeinos += mp->m_inoadd;
An strace of df --block-size=4k gives:
statfs("/mnt/test", {f_type=0x58465342, f_bsize=4096, f_blocks=1048616, f_bfree=874158, f_bavail=873134, f_files=4204672, f_ffree=4191008, f_fsid={2072, 0}, f_namelen=255, f_frsize=4096}) = 0
write(1, "/dev/sdb8 1048616 "..., 66/dev/sdb8 1048616 174458 873134 17% /mnt/test
) = 66
statfs("/mnt/scratch", {f_type=0x58465342, f_bsize=4096, f_blocks=1248496, f_bfree=1248392, f_bavail=1247368, f_files=5004224, f_ffree=5004220, f_fsid={2073, 0}, f_namelen=255, f_frsize=4096}) = 0
write(1, "/dev/sdb9 1248496 "..., 69/dev/sdb9 1248496 104 1247368 1% /mnt/scratch
Is this what you were thinking of?
Note that this still requires the fix to the qa test because the value in the on disk superblock matches f_bfree, not f_bavail and df appears to output f_bavail....
Cheers,
Dave.
Signature · 3 lines
Dave Chinner Principal Engineer SGI Australian Software Group