xfstests: several 274 fixups

PATCH Superseded Eric Sandeen 4 participants 9 messages Jan 30, 2012 – Apr 6, 2012

lore

Share & tools

lore

Eric Sandeen

author

2012-01-30 22:27 UTC #0 lore raw

PATCH 1/1 xfstests: several 274 fixups

This changes quite a few things about 274 to make it more robust and useful.

* More comments
* Use xfs_io for falloc (not all systems have /usr/bin/fallocate)
* use _require_xfs_io_falloc to be sure system & fs support preallocation
* Do not remove all of the files in $SCRATCH_MNT/ post-mkfs
* Do not remove all of the files in $SCRATCH_MNT/ on completion (this breaks e2fsck when lost+found/ goes missing)
* Don't cd into $SCRATCH_MNT
* Try harder to completely fill the fs
* Use a larger preallocated space, and write into all of it (hopefully avoid just accidentally succeeding by writing into fs reserved space that may be there)
* Save more output in $seq.full instead of /dev/null

Signed-off-by Eric Sandeen <[email protected]>

---

274 +22 ~~-12~~

index b658004..acf4543 100755
@@ -1,7 +1,9 @@
 #! /bin/bash
 # FS QA Test No. 274
 #
-# preallocation test
+# preallocation test:
+# Preallocate space to a file, and fill the rest of the fs to 100%.
+# Then test a write into that preallocated space, which should succeed.
 #
 #-----------------------------------------------------------------------
 # Copyright (c) 2011-2012 Fujitsu, Inc.  All Rights Reserved.

@@ -35,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _cleanup()
 {
 	cd /
-	rm -f $SCRATCH_MNT/* $tmp.*
+	rm -f $tmp.*
 	_scratch_unmount
 }

@@ -46,6 +48,7 @@ _cleanup()
 _supported_fs generic
 _supported_os IRIX Linux
 _require_scratch
+_require_xfs_io_falloc
 
 echo "------------------------------"
 echo "preallocation test"

@@ -57,32 +60,39 @@ umount $SCRATCH_DEV 2>/dev/null
 _scratch_mkfs_sized $((1 * 1024 * 1024 * 1024)) >>$seq.full 2>&1
 _scratch_mount
 
-rm -rf $SCRATCH_MNT/*
-cd $SCRATCH_MNT
-dd if=/dev/zero of=test bs=4K count=1 >/dev/null 2>&1
+# Create a 4k file
+dd if=/dev/zero of=$SCRATCH_MNT/test bs=4K count=1 >>$seq.full 2>&1
 if [ $? -ne 0 ]
 then
-	echo "create file err"
+	echo "create file error"
 	status=1
 	exit
 fi
 
-fallocate -n -o 4K -l 1M test >/dev/null 2>&1
+# Allocate 4M past EOF on that file
+xfs_io -F -c "falloc -k 4k 4m" $SCRATCH_MNT/test >>$seq.full 2>&1
 if [ $? -ne 0 ]
 then
-	echo "fallocate file err"
+	echo "fallocate file error"
 	status=1
 	exit
 fi
 
-dd if=/dev/zero of=tmp1 bs=1M >/dev/null 2>&1
-dd if=/dev/zero of=tmp2 bs=4K >/dev/null 2>&1
+# Fill the rest of the fs completely
+dd if=/dev/zero of=$SCRATCH_MNT/tmp1 bs=1M >>$seq.full 2>&1
+dd if=/dev/zero of=$SCRATCH_MNT/tmp2 bs=4K >>$seq.full 2>&1
 sync
+# Last effort, use O_SYNC
+dd if=/dev/zero of=$SCRATCH_MNT/tmp3 bs=4K oflag=sync >>$seq.full 2>&1
+# Save space usage info
+echo "Post-fill space:" >> $seq.full
+df $SCRATCH_MNT >>$seq.full 2>&1
 
-dd if=/dev/zero of=test seek=1 bs=4K count=2 conv=notrunc >/dev/null 2>&1
+# Now attempt a write into all of the preallocated space 
+dd if=/dev/zero of=$SCRATCH_MNT/test seek=1 bs=4K count=1024 conv=notrunc >>$seq.full 2>&1
 if [ $? -ne 0 ]
 then
-	echo "fill prealloc range err"
+	echo "fill prealloc range error"
 	status=1
 	exit
 fi

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Dave Chinner

2012-01-31 22:41 UTC #1 lore raw

Quoted 9 lines · Eric Sandeen

This changes quite a few things about 274 to make it more robust and useful.
* More comments
* Use xfs_io for falloc (not all systems have /usr/bin/fallocate)
* use _require_xfs_io_falloc to be sure system & fs support preallocation
* Do not remove all of the files in $SCRATCH_MNT/ post-mkfs
* Do not remove all of the files in $SCRATCH_MNT/ on completion (this breaks e2fsck when lost+found/ goes missing)

FWIW, can't e2fsck be fixed to handle this case?

......

Quoted 33 lines

_cleanup()
{

274 +5 -5

 	cd /
-	rm -f $SCRATCH_MNT/* $tmp.*
+	rm -f $tmp.*
 	_scratch_unmount
 }

@@ -46,6 +48,7 @@ _cleanup()
 _supported_fs generic
 _supported_os IRIX Linux
 _require_scratch
+_require_xfs_io_falloc
 
 echo "------------------------------"
 echo "preallocation test"

@@ -57,32 +60,39 @@ umount $SCRATCH_DEV 2>/dev/null
 _scratch_mkfs_sized $((1 * 1024 * 1024 * 1024)) >>$seq.full 2>&1
 _scratch_mount
 
-rm -rf $SCRATCH_MNT/*
-cd $SCRATCH_MNT
-dd if=/dev/zero of=test bs=4K count=1 >/dev/null 2>&1
+# Create a 4k file
+dd if=/dev/zero of=$SCRATCH_MNT/test bs=4K count=1 >>$seq.full 2>&1
 if [ $? -ne 0 ]
 then
-	echo "create file err"
+	echo "create file error"
 	status=1
 	exit
 fi

274 umount $SCRATCH_DEV 2>/dev/null · line 79

Quoted 10 lines

274 +3 -2

-fallocate -n -o 4K -l 1M test >/dev/null 2>&1
+# Allocate 4M past EOF on that file
+xfs_io -F -c "falloc -k 4k 4m" $SCRATCH_MNT/test >>$seq.full 2>&1
 if [ $? -ne 0 ]
 then
-	echo "fallocate file err"
+	echo "fallocate file error"
 	status=1
 	exit
 fi

That whole create and falloc step can be done with one command:

xfs_io -F -c "write 0 4k" -c "falloc -k 4k 4m" $SCRATCH_MNT/test

The output of xfs_io will tell us what failed if it does.

Also, failure handling is as simple as appending:

	|| _fail "failure string"

to the command. It handles setting status appropriately, tees the failure string to $seq.full, and tells the user to go look at $seq.full for why the test failed. hence that 16 lines of script can be simply replaced with these 2 lines:

xfs_io -F -c "write 0 4k" -c "falloc -k 4k 4m" $SCRATCH_MNT/test \
	>>$seq.full 2>&1 || _fail "failed to create test file"

Quoted 12 lines

274 +8 -2

-dd if=/dev/zero of=tmp1 bs=1M >/dev/null 2>&1
-dd if=/dev/zero of=tmp2 bs=4K >/dev/null 2>&1
+# Fill the rest of the fs completely
+dd if=/dev/zero of=$SCRATCH_MNT/tmp1 bs=1M >>$seq.full 2>&1
+dd if=/dev/zero of=$SCRATCH_MNT/tmp2 bs=4K >>$seq.full 2>&1
 sync
+# Last effort, use O_SYNC
+dd if=/dev/zero of=$SCRATCH_MNT/tmp3 bs=4K oflag=sync >>$seq.full 2>&1
+# Save space usage info
+echo "Post-fill space:" >> $seq.full
+df $SCRATCH_MNT >>$seq.full 2>&1

274 umount $SCRATCH_DEV 2>/dev/null · line 98

Quoted 10 lines

274 +3 -2

-dd if=/dev/zero of=test seek=1 bs=4K count=2 conv=notrunc >/dev/null 2>&1
+# Now attempt a write into all of the preallocated space 
+dd if=/dev/zero of=$SCRATCH_MNT/test seek=1 bs=4K count=1024 conv=notrunc >>$seq.full 2>&1
 if [ $? -ne 0 ]
 then
-	echo "fill prealloc range err"
+	echo "fill prealloc range error"
 	status=1
 	exit
 fi

I'd still like to see this write attempt to trigger nasty behaviours like needing to allocate a metadata block for the extent list. I suggested randholes, but perhaps this would be easier:

for i in seq 1 2 1023; do

	dd if=/dev/zero of=$SCRATCH_MNT/test seek=$i bs=4K count=1 conv=notrunc \
		>>$seq.full 2>&1 || _fail "failed to write test file"

done

which will write every second block and so only convert every second block from unwritten to written and hence blow out the size of the extent list and require extent map block allocation and potentially trigger ENOSPC that way....

Cheers,

Dave.

Signature · 7 lines

Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Eric Sandeen

author

2012-02-01 02:27 UTC #2 lore raw

Quoted 4 lines · Dave Chinner

Quoted 3 lines · Eric Sandeen
This changes quite a few things about 274 to make it more robust and useful.

Quoted 8 lines

Quoted 6 lines
* More comments
* Use xfs_io for falloc (not all systems have /usr/bin/fallocate)
* use _require_xfs_io_falloc to be sure system & fs support preallocation
* Do not remove all of the files in $SCRATCH_MNT/ post-mkfs
* Do not remove all of the files in $SCRATCH_MNT/ on completion (this breaks e2fsck when lost+found/ goes missing)
FWIW, can't e2fsck be fixed to handle this case?

perhaps some day.

but it'd return "modified" if it adds lost+found ...

I suppose it could just be left missing unless otherwise needed.

Quoted 55 lines

......

Quoted 33 lines

_cleanup()
{

274 +5 -5

 	cd /
-	rm -f $SCRATCH_MNT/* $tmp.*
+	rm -f $tmp.*
 	_scratch_unmount
 }

@@ -46,6 +48,7 @@ _cleanup()
 _supported_fs generic
 _supported_os IRIX Linux
 _require_scratch
+_require_xfs_io_falloc
 
 echo "------------------------------"
 echo "preallocation test"

@@ -57,32 +60,39 @@ umount $SCRATCH_DEV 2>/dev/null
 _scratch_mkfs_sized $((1 * 1024 * 1024 * 1024)) >>$seq.full 2>&1
 _scratch_mount
 
-rm -rf $SCRATCH_MNT/*
-cd $SCRATCH_MNT
-dd if=/dev/zero of=test bs=4K count=1 >/dev/null 2>&1
+# Create a 4k file
+dd if=/dev/zero of=$SCRATCH_MNT/test bs=4K count=1 >>$seq.full 2>&1
 if [ $? -ne 0 ]
 then
-	echo "create file err"
+	echo "create file error"
 	status=1
 	exit
 fi

274 umount $SCRATCH_DEV 2>/dev/null · line 79

Quoted 10 lines

274 +3 -2

-fallocate -n -o 4K -l 1M test >/dev/null 2>&1
+# Allocate 4M past EOF on that file
+xfs_io -F -c "falloc -k 4k 4m" $SCRATCH_MNT/test >>$seq.full 2>&1
 if [ $? -ne 0 ]
 then
-	echo "fallocate file err"
+	echo "fallocate file error"
 	status=1
 	exit
 fi

That whole create and falloc step can be done with one command:

xfs_io -F -c "write 0 4k" -c "falloc -k 4k 4m" $SCRATCH_MNT/test

The output of xfs_io will tell us what failed if it does.

Also, failure handling is as simple as appending:

	|| _fail "failure string"

Quoted 7 lines

to the command. It handles setting status appropriately, tees the failure string to $seq.full, and tells the user to go look at $seq.full for why the test failed. hence that 16 lines of script can be simply replaced with these 2 lines:
xfs_io -F -c "write 0 4k" -c "falloc -k 4k 4m" $SCRATCH_MNT/test \
	>>$seq.full 2>&1 || _fail "failed to create test file"

Ok, that's fine.

Quoted 28 lines

Quoted 12 lines

274 +8 -2

-dd if=/dev/zero of=tmp1 bs=1M >/dev/null 2>&1
-dd if=/dev/zero of=tmp2 bs=4K >/dev/null 2>&1
+# Fill the rest of the fs completely
+dd if=/dev/zero of=$SCRATCH_MNT/tmp1 bs=1M >>$seq.full 2>&1
+dd if=/dev/zero of=$SCRATCH_MNT/tmp2 bs=4K >>$seq.full 2>&1
 sync
+# Last effort, use O_SYNC
+dd if=/dev/zero of=$SCRATCH_MNT/tmp3 bs=4K oflag=sync >>$seq.full 2>&1
+# Save space usage info
+echo "Post-fill space:" >> $seq.full
+df $SCRATCH_MNT >>$seq.full 2>&1

274 umount $SCRATCH_DEV 2>/dev/null · line 98

Quoted 10 lines

274 +3 -2

-dd if=/dev/zero of=test seek=1 bs=4K count=2 conv=notrunc >/dev/null 2>&1
+# Now attempt a write into all of the preallocated space 
+dd if=/dev/zero of=$SCRATCH_MNT/test seek=1 bs=4K count=1024 conv=notrunc >>$seq.full 2>&1
 if [ $? -ne 0 ]
 then
-	echo "fill prealloc range err"
+	echo "fill prealloc range error"
 	status=1
 	exit
 fi

I'd still like to see this write attempt to trigger nasty behaviours like needing to allocate a metadata block for the extent list. I suggested randholes, but perhaps this would be easier:

for i in seq 1 2 1023; do

Quoted 8 lines

	dd if=/dev/zero of=$SCRATCH_MNT/test seek=$i bs=4K count=1 conv=notrunc \
		>>$seq.full 2>&1 || _fail "failed to write test file"
done
which will write every second block and so only convert every second block from unwritten to written and hence blow out the size of the extent list and require extent map block allocation and potentially trigger ENOSPC that way....

but will also only write half the actual preallocated space. I guess it depends on what we are trying to test - reservations for metadata or proper accounting of prealloc'd space itself...?

(and really, what is the proper amount of preallocated metadata space; if I subbed 2G for 4M, would anything survive the test above?)

-Eric

Quoted 3 lines

Cheers,
Dave.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Dave Chinner

2012-02-01 04:07 UTC #3 lore raw

Quoted 34 lines · Eric Sandeen

Quoted 33 lines · Dave Chinner

Quoted 12 lines · Eric Sandeen

274 +8 -2

-dd if=/dev/zero of=tmp1 bs=1M >/dev/null 2>&1
-dd if=/dev/zero of=tmp2 bs=4K >/dev/null 2>&1
+# Fill the rest of the fs completely
+dd if=/dev/zero of=$SCRATCH_MNT/tmp1 bs=1M >>$seq.full 2>&1
+dd if=/dev/zero of=$SCRATCH_MNT/tmp2 bs=4K >>$seq.full 2>&1
 sync
+# Last effort, use O_SYNC
+dd if=/dev/zero of=$SCRATCH_MNT/tmp3 bs=4K oflag=sync >>$seq.full 2>&1
+# Save space usage info
+echo "Post-fill space:" >> $seq.full
+df $SCRATCH_MNT >>$seq.full 2>&1

274 umount $SCRATCH_DEV 2>/dev/null · line 98

Quoted 10 lines

274 +3 -2

-dd if=/dev/zero of=test seek=1 bs=4K count=2 conv=notrunc >/dev/null 2>&1
+# Now attempt a write into all of the preallocated space 
+dd if=/dev/zero of=$SCRATCH_MNT/test seek=1 bs=4K count=1024 conv=notrunc >>$seq.full 2>&1
 if [ $? -ne 0 ]
 then
-	echo "fill prealloc range err"
+	echo "fill prealloc range error"
 	status=1
 	exit
 fi

I'd still like to see this write attempt to trigger nasty behaviours like needing to allocate a metadata block for the extent list. I suggested randholes, but perhaps this would be easier:

for i in seq 1 2 1023; do

	dd if=/dev/zero of=$SCRATCH_MNT/test seek=$i bs=4K count=1 conv=notrunc \
		>>$seq.full 2>&1 || _fail "failed to write test file"

done

Quoted 8 lines

Quoted 4 lines
which will write every second block and so only convert every second block from unwritten to written and hence blow out the size of the extent list and require extent map block allocation and potentially trigger ENOSPC that way....
but will also only write half the actual preallocated space. I guess it depends on what we are trying to test - reservations for metadata or proper accounting of prealloc'd space itself...?

So then go back and fill in the unwritten blocks. i.e:

for i in seq 1 2 1023; do

	dd if=/dev/zero of=$SCRATCH_MNT/test seek=$i bs=4K count=1 conv=notrunc \
		>>$seq.full 2>&1 || _fail "failed to write test file"
done
sync
for i in `seq 2 2 1023`; do
	dd if=/dev/zero of=$SCRATCH_MNT/test seek=$i bs=4K count=1 conv=notrunc \
		>>$seq.full 2>&1 || _fail "failed to fill test file"

done
sync

Quoted 2 lines

(and really, what is the proper amount of preallocated metadata space; if I subbed 2G for 4M, would anything survive the test above?)

Given that the test is only using a 1GB filesystem.... ;)

But even if it was 2GB, then I'm pretty sure that XFS would handle it fine. Take a long to run, though, with all those individual 4k IOs. My point was simply to determine if the filesystem will be able to hold up the guarantees posix_fallocate() tries to give applications, and so enough extents simple to force a single metadata block allocation is all that is required. Having a 1000 extent state changes shoul dbe sufficient to cause that....

Cheers,

Dave.

Signature · 7 lines

Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Eric Sandeen

author

2012-03-08 21:55 UTC #4 lore raw

In reply to Dave Chinner

Quoted 60 lines · Dave Chinner

Quoted 9 lines · Eric Sandeen
This changes quite a few things about 274 to make it more robust and useful.
* More comments
* Use xfs_io for falloc (not all systems have /usr/bin/fallocate)
* use _require_xfs_io_falloc to be sure system & fs support preallocation
* Do not remove all of the files in $SCRATCH_MNT/ post-mkfs
* Do not remove all of the files in $SCRATCH_MNT/ on completion (this breaks e2fsck when lost+found/ goes missing)
FWIW, can't e2fsck be fixed to handle this case?
......
Quoted 33 lines
_cleanup()
{
274 +5 -5
 	cd /
-	rm -f $SCRATCH_MNT/* $tmp.*
+	rm -f $tmp.*
 	_scratch_unmount
 }
@@ -46,6 +48,7 @@ _cleanup()
 _supported_fs generic
 _supported_os IRIX Linux
 _require_scratch
+_require_xfs_io_falloc
 
 echo "------------------------------"
 echo "preallocation test"
@@ -57,32 +60,39 @@ umount $SCRATCH_DEV 2>/dev/null
 _scratch_mkfs_sized $((1 * 1024 * 1024 * 1024)) >>$seq.full 2>&1
 _scratch_mount
 
-rm -rf $SCRATCH_MNT/*
-cd $SCRATCH_MNT
-dd if=/dev/zero of=test bs=4K count=1 >/dev/null 2>&1
+# Create a 4k file
+dd if=/dev/zero of=$SCRATCH_MNT/test bs=4K count=1 >>$seq.full 2>&1
 if [ $? -ne 0 ]
 then
-	echo "create file err"
+	echo "create file error"
 	status=1
 	exit
 fi
 
274 umount $SCRATCH_DEV 2>/dev/null · line 79
Quoted 10 lines
274 +3 -2
-fallocate -n -o 4K -l 1M test >/dev/null 2>&1
+# Allocate 4M past EOF on that file
+xfs_io -F -c "falloc -k 4k 4m" $SCRATCH_MNT/test >>$seq.full 2>&1
 if [ $? -ne 0 ]
 then
-	echo "fallocate file err"
+	echo "fallocate file error"
 	status=1
 	exit
 fi
That whole create and falloc step can be done with one command:

Quoted 3 lines

xfs_io -F -c "write 0 4k" -c "falloc -k 4k 4m" $SCRATCH_MNT/test
The output of xfs_io will tell us what failed if it does.

too bad that doesn't quite fail properly ;)

[root@inode xfstests-dev]# xfs_io -F -f -c "write 0 4k" -c "falloc -k 4k 4m" foobar command "write" not found
[root@inode xfstests-dev]# echo $?
0

I'll fix that up too I guess.

--Eric

Quoted 44 lines

Also, failure handling is as simple as appending:
	|| _fail "failure string"
to the command. It handles setting status appropriately, tees the failure string to $seq.full, and tells the user to go look at $seq.full for why the test failed. hence that 16 lines of script can be simply replaced with these 2 lines:
xfs_io -F -c "write 0 4k" -c "falloc -k 4k 4m" $SCRATCH_MNT/test \
	>>$seq.full 2>&1 || _fail "failed to create test file"
Quoted 12 lines
274 +8 -2
-dd if=/dev/zero of=tmp1 bs=1M >/dev/null 2>&1
-dd if=/dev/zero of=tmp2 bs=4K >/dev/null 2>&1
+# Fill the rest of the fs completely
+dd if=/dev/zero of=$SCRATCH_MNT/tmp1 bs=1M >>$seq.full 2>&1
+dd if=/dev/zero of=$SCRATCH_MNT/tmp2 bs=4K >>$seq.full 2>&1
 sync
+# Last effort, use O_SYNC
+dd if=/dev/zero of=$SCRATCH_MNT/tmp3 bs=4K oflag=sync >>$seq.full 2>&1
+# Save space usage info
+echo "Post-fill space:" >> $seq.full
+df $SCRATCH_MNT >>$seq.full 2>&1
 
274 umount $SCRATCH_DEV 2>/dev/null · line 98
Quoted 10 lines
274 +3 -2
-dd if=/dev/zero of=test seek=1 bs=4K count=2 conv=notrunc >/dev/null 2>&1
+# Now attempt a write into all of the preallocated space 
+dd if=/dev/zero of=$SCRATCH_MNT/test seek=1 bs=4K count=1024 conv=notrunc >>$seq.full 2>&1
 if [ $? -ne 0 ]
 then
-	echo "fill prealloc range err"
+	echo "fill prealloc range error"
 	status=1
 	exit
 fi
I'd still like to see this write attempt to trigger nasty behaviours like needing to allocate a metadata block for the extent list. I suggested randholes, but perhaps this would be easier:
for i in seq 1 2 1023; do
	dd if=/dev/zero of=$SCRATCH_MNT/test seek=$i bs=4K count=1 conv=notrunc \
		>>$seq.full 2>&1 || _fail "failed to write test file"
done

Quoted 8 lines

which will write every second block and so only convert every second block from unwritten to written and hence blow out the size of the extent list and require extent map block allocation and potentially trigger ENOSPC that way....
Cheers,
Dave.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Dave Chinner

2012-03-08 23:34 UTC #5 lore raw

Quoted 10 lines · Eric Sandeen

Quoted 7 lines · Dave Chinner
On Mon, Jan 30, 2012 at 04:27:51PM -0600, Eric Sandeen wrote:
That whole create and falloc step can be done with one command:
xfs_io -F -c "write 0 4k" -c "falloc -k 4k 4m" $SCRATCH_MNT/test
The output of xfs_io will tell us what failed if it does.
too bad that doesn't quite fail properly ;)

Just a small typo - the command is pwrite ;)

Cheers,

Dave.

Signature · 7 lines

Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Eric Sandeen

author

2012-03-08 22:23 UTC #6 lore raw

PATCH v2 1/1 xfstests: several 274 fixups

In reply to Eric Sandeen

This changes quite a few things about 274 to make it more robust and useful.

Signed-off-by Eric Sandeen <[email protected]>

---

V2: Several fixes suggested by Dave.
Also fills prealloc space diabolically. This fails on ext4.

274 +38 ~~-28~~

index b658004..bba5eb4 100755
@@ -1,7 +1,9 @@
 #! /bin/bash
 # FS QA Test No. 274
 #
-# preallocation test
+# preallocation test:
+# Preallocate space to a file, and fill the rest of the fs to 100%.
+# Then test a write into that preallocated space, which should succeed.
 #
 #-----------------------------------------------------------------------
 # Copyright (c) 2011-2012 Fujitsu, Inc.  All Rights Reserved.

@@ -35,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _cleanup()
 {
 	cd /
-	rm -f $SCRATCH_MNT/* $tmp.*
+	rm -f $tmp.*
 	_scratch_unmount
 }

@@ -46,6 +48,7 @@ _cleanup()
 _supported_fs generic
 _supported_os IRIX Linux
 _require_scratch
+_require_xfs_io_falloc
 
 echo "------------------------------"
 echo "preallocation test"

@@ -57,35 +60,42 @@ umount $SCRATCH_DEV 2>/dev/null
 _scratch_mkfs_sized $((1 * 1024 * 1024 * 1024)) >>$seq.full 2>&1
 _scratch_mount
 
-rm -rf $SCRATCH_MNT/*
-cd $SCRATCH_MNT
-dd if=/dev/zero of=test bs=4K count=1 >/dev/null 2>&1
-if [ $? -ne 0 ]
-then
-	echo "create file err"
-	status=1
-	exit
-fi
+# Create a 4k file and Allocate 4M past EOF on that file
+xfs_io -F -f -c "pwrite 0 4k" -c "falloc -k 4k 4m" $SCRATCH_MNT/test \
+	>>$seq.full 2>&1 || _fail "failed to create test file"
 
-fallocate -n -o 4K -l 1M test >/dev/null 2>&1
-if [ $? -ne 0 ]
-then
-	echo "fallocate file err"
-	status=1
-	exit
-fi
-
-dd if=/dev/zero of=tmp1 bs=1M >/dev/null 2>&1
-dd if=/dev/zero of=tmp2 bs=4K >/dev/null 2>&1
+# Fill the rest of the fs completely
+# Note, this will show ENOSPC errors in $seq.full, that's ok.
+echo "Fill fs with 1M IOs; EIO expected" >> $seq.full
+dd if=/dev/zero of=$SCRATCH_MNT/tmp1 bs=1M >>$seq.full 2>&1
+echo "Fill fs with 4K IOs; EIO expected" >> $seq.full
+dd if=/dev/zero of=$SCRATCH_MNT/tmp2 bs=4K >>$seq.full 2>&1
 sync
+# Last effort, use O_SYNC
+echo "Fill fs with 4K DIOs; EIO expected" >> $seq.full
+dd if=/dev/zero of=$SCRATCH_MNT/tmp3 bs=4K oflag=sync >>$seq.full 2>&1
+# Save space usage info
+echo "Post-fill space:" >> $seq.full
+df $SCRATCH_MNT >>$seq.full 2>&1
 
-dd if=/dev/zero of=test seek=1 bs=4K count=2 conv=notrunc >/dev/null 2>&1
-if [ $? -ne 0 ]
-then
-	echo "fill prealloc range err"
-	status=1
-	exit
-fi
+# Now attempt a write into all of the preallocated space -
+# in a very nasty way, badly fragmenting it and then filling it in.
+echo "Fill in prealloc space; fragment at offsets:" >> $seq.full
+for i in `seq 1 2 1023`; do
+	echo -n "$i " >> $seq.full
+	dd if=/dev/zero of=$SCRATCH_MNT/test seek=$i bs=4K count=1 conv=notrunc \
+		>>$seq.full 2>/dev/null || _fail "failed to write to test file"
+done
+sync
+echo >> $seq.full
+echo "Fill in prealloc space; fill holes at offsets:" >> $seq.full
+for i in `seq 2 2 1023`; do
+	echo -n "$i " >> $seq.full
+	dd if=/dev/zero of=$SCRATCH_MNT/test seek=$i bs=4K count=1 conv=notrunc \
+		>>$seq.full 2>/dev/null || _fail "failed to fill test file"
+done
+sync
+echo >> $seq.full
 
 echo "done"
 exit

Mark Tinguely

Reviewed-by on v2 1/1

2012-04-06 16:12 UTC #7 lore raw

Quoted 17 lines · Eric Sandeen

This changes quite a few things about 274 to make it more robust and useful.
* More comments
* Use xfs_io for falloc (not all systems have /usr/bin/fallocate)
* use _require_xfs_io_falloc to be sure system& fs support preallocation
* Do not remove all of the files in $SCRATCH_MNT/ post-mkfs
* Do not remove all of the files in $SCRATCH_MNT/ on completion (this breaks e2fsck when lost+found/ goes missing)
* Don't cd into $SCRATCH_MNT
* Try harder to completely fill the fs
* Use a larger preallocated space, and write into all of it (hopefully avoid just accidentally succeeding by writing into fs reserved space that may be there)
* Save more output in $seq.full instead of /dev/null
* Fill preallocated space diabolically

Quoted 2 lines

Signed-off-by Eric Sandeen<[email protected]>
---

Looks very nice.

Reviewed-by Mark Tinguely <[email protected]>

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Christoph Hellwig

2012-03-31 16:00 UTC #8 lore raw

In reply to Eric Sandeen

Do you plan to resend this one with the fixups pointed out by Dave?

Quoted 102 lines · Eric Sandeen

This changes quite a few things about 274 to make it more robust and useful.

Signed-off-by Eric Sandeen <[email protected]>

---

274 +22 ~~-12~~

index b658004..acf4543 100755
@@ -1,7 +1,9 @@
 #! /bin/bash
 # FS QA Test No. 274
 #
-# preallocation test
+# preallocation test:
+# Preallocate space to a file, and fill the rest of the fs to 100%.
+# Then test a write into that preallocated space, which should succeed.
 #
 #-----------------------------------------------------------------------
 # Copyright (c) 2011-2012 Fujitsu, Inc.  All Rights Reserved.

@@ -35,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _cleanup()
 {
 	cd /
-	rm -f $SCRATCH_MNT/* $tmp.*
+	rm -f $tmp.*
 	_scratch_unmount
 }

@@ -46,6 +48,7 @@ _cleanup()
 _supported_fs generic
 _supported_os IRIX Linux
 _require_scratch
+_require_xfs_io_falloc
 
 echo "------------------------------"
 echo "preallocation test"

@@ -57,32 +60,39 @@ umount $SCRATCH_DEV 2>/dev/null
 _scratch_mkfs_sized $((1 * 1024 * 1024 * 1024)) >>$seq.full 2>&1
 _scratch_mount
 
-rm -rf $SCRATCH_MNT/*
-cd $SCRATCH_MNT
-dd if=/dev/zero of=test bs=4K count=1 >/dev/null 2>&1
+# Create a 4k file
+dd if=/dev/zero of=$SCRATCH_MNT/test bs=4K count=1 >>$seq.full 2>&1
 if [ $? -ne 0 ]
 then
-	echo "create file err"
+	echo "create file error"
 	status=1
 	exit
 fi
 
-fallocate -n -o 4K -l 1M test >/dev/null 2>&1
+# Allocate 4M past EOF on that file
+xfs_io -F -c "falloc -k 4k 4m" $SCRATCH_MNT/test >>$seq.full 2>&1
 if [ $? -ne 0 ]
 then
-	echo "fallocate file err"
+	echo "fallocate file error"
 	status=1
 	exit
 fi
 
-dd if=/dev/zero of=tmp1 bs=1M >/dev/null 2>&1
-dd if=/dev/zero of=tmp2 bs=4K >/dev/null 2>&1
+# Fill the rest of the fs completely
+dd if=/dev/zero of=$SCRATCH_MNT/tmp1 bs=1M >>$seq.full 2>&1
+dd if=/dev/zero of=$SCRATCH_MNT/tmp2 bs=4K >>$seq.full 2>&1
 sync
+# Last effort, use O_SYNC
+dd if=/dev/zero of=$SCRATCH_MNT/tmp3 bs=4K oflag=sync >>$seq.full 2>&1
+# Save space usage info
+echo "Post-fill space:" >> $seq.full
+df $SCRATCH_MNT >>$seq.full 2>&1
 
-dd if=/dev/zero of=test seek=1 bs=4K count=2 conv=notrunc >/dev/null 2>&1
+# Now attempt a write into all of the preallocated space 
+dd if=/dev/zero of=$SCRATCH_MNT/test seek=1 bs=4K count=1024 conv=notrunc >>$seq.full 2>&1
 if [ $? -ne 0 ]
 then
-	echo "fill prealloc range err"
+	echo "fill prealloc range error"
 	status=1
 	exit
 fi

Quoted 4 lines

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

---end quoted text---

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs