tst_fill_fs: enhance the filesystem filling routine
PATCH 1/1 tst_fill_fs: enhance the filesystem filling routine
Do more tries with size in half when write() getting ENOSPC, until the size is less than the filesystem block size. Though we can't really fill a filesystem full, this could make the routine more robust.
--- lib/tst_fill_fs.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
lib/tst_fill_fs.c +9 -1
@@ -6,6 +6,7 @@#include <errno.h> #include <stdio.h> #include <stdlib.h> +#include <sys/statvfs.h> #define TST_NO_DEFAULT_MAIN #include "tst_test.h"
@@ -19,6 +20,8 @@ void tst_fill_fs(const char *path, int verbose)size_t len;ssize_t ret;int fd; +struct statvfs fi; +statvfs (path, &fi);for (;;) { len =random () % (1024 *102400 );
@@ -37,10 +40,15 @@ void tst_fill_fs(const char *path, int verbose)return ; } -while (len) { +while (len >= fi.f_bsize) { ret =write (fd, buf,MIN (len,sizeof (buf)));if (ret <0 ) { +if (errno ==ENOSPC ) { + len /=2 ; +continue ; + } +SAFE_CLOSE (fd);if (errno !=ENOSPC )
Signature · 1 line
2.20.1
PATCH 1/1 tst_fill_fs: enhance the filesystem filling routine
----- Original Message -----
lib/tst_fill_fs.c void tst_fill_fs(const char *path, int verbose) · line 43
Quoted 2 lines
lib/tst_fill_fs.c +1
-1-while (len) { +while (len >= fi.f_bsize) {
Let's say f_bsize is 512, and len is 1023. We hit ENOSPC for 1023, len for next round is 511. Loop ends, but we never tried write with f_bsize or smaller value.
I think we should try len < fi.f_bsize@least once.
lib/tst_fill_fs.c void tst_fill_fs(const char *path, int verbose) · line 50
Quoted 7 lines
lib/tst_fill_fs.c +4
-0ret =write (fd, buf,MIN (len,sizeof (buf)));if (ret <0 ) { +if (errno ==ENOSPC ) { + len /=2 ; +continue ; + }
The return below (in original code) doesn't appear to be reachable now.
PATCH 1/1 tst_fill_fs: enhance the filesystem filling routine
Quoted 12 lines · Jan Stancek
----- Original Message -----
lib/tst_fill_fs.c void tst_fill_fs(const char *path, int verbose) · line 43
Quoted 2 lines
lib/tst_fill_fs.c +1
-1-while (len) { +while (len >= fi.f_bsize) {Let's say
f_bsizeis 512, and len is 1023. We hit ENOSPC for 1023, len for next round is 511. Loop ends, but we never tried write withf_bsizeor smaller value.I think we should try len < fi.f_bsize@least once.
Good point. At least for len = fi.f_bsize.
Quoted 11 lines
lib/tst_fill_fs.c void tst_fill_fs(const char *path, int verbose) · line 50
Quoted 8 lines
ret = write(fd, buf, MIN(len, sizeof(buf))); if (ret < 0) {lib/tst_fill_fs.c +4
-0+if (errno ==ENOSPC ) { + len /=2 ; +continue ; + }The return below (in original code) doesn't appear to be reachable now.
Yes, but it doesn't make much sense there, maybe we could remove that part directly?
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20191119/e94973b4/attachment.htm>
PATCH 1/1 tst_fill_fs: enhance the filesystem filling routine
Quoted 7 lines · Li Wang
On Tue, Nov 19, 2019 at 7:07 PM Jan Stancek <[email protected] <mailto:[email protected]>> wrote:
----- Original Message -----
> -? ? ? ? ? ? ?while (len) {
> +? ? ? ? ? ? ?while (len >= fi.f_bsize) {
Quoted 8 lines
Let's say
f_bsizeis 512, and len is 1023. We hit ENOSPC for 1023, len for next round is 511. Loop ends, but we never tried write withf_bsizeor smaller value.I think we should try len < fi.f_bsize@least once.
Good point. At least for len = fi.f_bsize.
You don't need to reduce the write size in the first place. Kernel will happily do short writes for you if the file system somehow finds some free blocks. The more important question is how to trigger garbage collection of internal file system book-keeping data.
Signature · 7 lines
Martin Doucha [email protected] QA Engineer for Software Maintenance SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic
PATCH 1/1 tst_fill_fs: enhance the filesystem filling routine
Quoted 12 lines · Martin Doucha
Quoted 11 lines · Li Wang
On Tue, Nov 19, 2019 at 7:07 PM Jan Stancek <[email protected] <mailto:[email protected]>> wrote:
----- Original Message -----
> - while (len) {
> + while (len >= fi.f_bsize) {Let's say
f_bsizeis 512, and len is 1023. We hit ENOSPC for 1023, len for next round is 511. Loop ends, but we never tried write withf_bsizeor smaller value.
Quoted 8 lines
Quoted 4 lines
I think we should try len < fi.f_bsize@least once.
Good point. At least for len = fi.f_bsize.
You don't need to reduce the write size in the first place. Kernel will happily do short writes for you if the file system somehow finds some
You're probably right, but here I just want to enhance the tst_fill_fs routine, it obviously couldn't be guaranteed way to perfectly fill a filesystem, this action from userspace is what I can think of.
Quoted 3 lines
free blocks. The more important question is how to trigger garbage collection of internal file system book-keeping data.
Okay. I'm not good at FS issues. Or, do you have a better idea for that?
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20191120/12246ea0/attachment.htm>
PATCH 1/1 tst_fill_fs: enhance the filesystem filling routine
----- Original Message -----
Quoted 9 lines
Quoted 3 lines
You don't need to reduce the write size in the first place. Kernel will happily do short writes for you if the file system somehow finds some
You're probably right, but here I just want to enhance the
tst_fill_fsroutine, it obviously couldn't be guaranteed way to perfectly fill a filesystem, this action from userspace is what I can think of.
Quoted 5 lines
Quoted 3 lines
free blocks. The more important question is how to trigger garbage collection of internal file system book-keeping data.
Okay. I'm not good at FS issues. Or, do you have a better idea for that?
FS devs mentioned couple ideas. Repeated writes (this thread), fsync, mount+umount. But all suggestions were from "better" category, don't think we have 100% reliable one.
PATCH 1/1 tst_fill_fs: enhance the filesystem filling routine
In reply to Li WangQuoted 7 lines · Li Wang
You don't need to reduce the write size in the first place. Kernel will happily do short writes for you if the file system somehow finds some
You're probably right, but here I just want to enhance the
tst_fill_fsroutine, it obviously couldn't? be guaranteed way to perfectly fill a filesystem, this action from userspace is what I can think of.
I was also talking about enhancing tst_fill_fs(). You can simply repeat write()s with the maximum size until you're sure the file system is really full. Reducing the write size and then also using the write size as iteration counter creates some really nasty edge cases that can be easily avoided.
Signature · 7 lines
Martin Doucha [email protected] QA Engineer for Software Maintenance SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic
PATCH 1/1 tst_fill_fs: enhance the filesystem filling routine
----- Original Message -----
Quoted 5 lines · Li Wang
Quoted 4 lines
You don't need to reduce the write size in the first place. Kernel will happily do short writes for you if the file system somehow finds some
Quoted 7 lines
Quoted 3 lines
You're probably right, but here I just want to enhance the
tst_fill_fsroutine, it obviously couldn't? be guaranteed way to perfectly fill a filesystem, this action from userspace is what I can think of.I was also talking about enhancing
tst_fill_fs(). You can simply repeat write()s with the maximum size until you're sure the file system is really full.
I'd rather we mirror xfstests, which halves the length on ENOSPC.
Quoted 2 lines
Reducing the write size and then also using the write size as iteration counter
We could do fixed number of retries. In the end it still feels like rain dance.
PATCH 1/1 tst_fill_fs: enhance the filesystem filling routine
In reply to Martin DouchaHello,
here's something fun which I've discovered while playing with fallocate05 test on XFS:
- fallocate(x blocks)
- tst_fill_fs()
- write(x blocks) - OK
- write(1 block) - ENOSPC
- fallocate(1 block) - OK
- write(1 block) - OK
- fallocate(1 block) - ENOSPC
Somehow, fallocate() can find another free block on XFS which wasn't available to write() at first. But then that block becomes write()able.
Signature · 7 lines
Martin Doucha [email protected] QA Engineer for Software Maintenance SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic
PATCH 1/1 tst_fill_fs: enhance the filesystem filling routine
Quoted 11 lines · Martin Doucha
Hello,
here's something fun which I've discovered while playing with fallocate05 test on XFS:
- fallocate(x blocks)
-tst_fill_fs()
- write(x blocks) - OK
- write(1 block) - ENOSPC
- fallocate(1 block) - OK
- write(1 block) - OK
- fallocate(1 block) - ENOSPC
Quoted 3 lines
Somehow,
fallocate()can find another free block on XFS which wasn't available towrite()at first. But then that block becomes write()able.
That's interesting.
It does prove that an xfs filesystem can't be really fill full. As FS dev says though a filesystem return -ENOSPC, after some internal bookkeeping, flushing, then have space available again.
And that also means to increases FALLOCATE_SIZE is a wise way to avoid this problem.
Quoted 11 lines
--
Martin Doucha [email protected]
QA Engineer for Software Maintenance
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20191128/5a04406e/attachment-0001.htm>