Skip to contentSkip to conversation

fallocate05: increase FALLOCATE_SIZE

PATCH New Jan Stancek 4 participants 9 messages Nov 18, 2019 – Nov 19, 2019 Library code

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 fallocate05: increase FALLOCATE_SIZE

write() returning ENOSPC doesn't guarantee that filesystem after some internal book-keeping, flushing, finishing transactions, etc. won't still find some extra space.

Increase FALLOCATE_SIZE to minimize chance of hitting sporadic failures when that happens.

Thanks to Carlos Maiolino and Eric Sandeen for their comments and suggestions.

Fixes #610

---
 testcases/kernel/syscalls/fallocate/fallocate05.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
testcases/kernel/syscalls/fallocate/fallocate05.c +1 -1
index 50c610c448ba..17034e5b11e7 100644
@@ -17,7 +17,7 @@
 #include "lapi/fallocate.h"
 
 #define MNTPOINT "mntpoint"
-#define FALLOCATE_SIZE 8192
+#define FALLOCATE_SIZE (1024*1024)
 #define TESTED_FLAGS "fallocate(FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE)"
 
 static int fd;
Signature · 1 line
1.8.3.1
#1 lore raw

PATCH 1/1 fallocate05: increase FALLOCATE_SIZE

Quoted 6 lines · Jan Stancek

write() returning ENOSPC doesn't guarantee that filesystem after some internal book-keeping, flushing, finishing transactions, etc. won't still find some extra space.

Increase FALLOCATE_SIZE to minimize chance of hitting sporadic failures when that happens.

We're planning to rewrite fallocate05 this week and FALLOCATE_SIZE will be removed entirely. The test must use a multiple of the real file system block size, otherwise it'll test different things on different platforms.

Signature · 7 lines
Martin Doucha   [email protected]
QA Engineer for Software Maintenance
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic
#2 lore raw

PATCH 1/1 fallocate05: increase FALLOCATE_SIZE

Quoted 5 lines · Martin Doucha
Quoted 4 lines · Jan Stancek

write() returning ENOSPC doesn't guarantee that filesystem after some internal book-keeping, flushing, finishing transactions, etc. won't still find some extra space.

Quoted 8 lines
Quoted 2 lines

Increase FALLOCATE_SIZE to minimize chance of hitting sporadic failures when that happens.

We're planning to rewrite fallocate05 this week and FALLOCATE_SIZE will be removed entirely. The test must use a multiple of the real file system block size, otherwise it'll test different things on different platforms.

Sounds good. Thanks!

But it'd be better to merge Jan's patch first because there will still need time for new patch reviewing.

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20191119/2362b5a9/attachment.htm>
#3 lore raw

PATCH 1/1 fallocate05: increase FALLOCATE_SIZE

In reply to Jan Stancek

Hi!

Quoted 9 lines

write() returning ENOSPC doesn't guarantee that filesystem after some internal book-keeping, flushing, finishing transactions, etc. won't still find some extra space.

Increase FALLOCATE_SIZE to minimize chance of hitting sporadic failures when that happens.

Thanks to Carlos Maiolino and Eric Sandeen for their comments and suggestions.

Acked.

Signature · 2 lines
Cyril Hrubis
[email protected]
#4 lore raw

PATCH 1/1 fallocate05: increase FALLOCATE_SIZE

In reply to Jan Stancek

Hi Jan,

Quoted 4 lines · Jan Stancek

write() returning ENOSPC doesn't guarantee that filesystem after some internal book-keeping, flushing, finishing transactions, etc. won't still find some extra space.

Thanks for the patch, I have drafted a similar one but you sent out first :).

Another patch I was thinking is to enhance the tst_fill_fs routine, which as Eric suggested, makes more reliably to get to a full filesystem. Something like what xfstest does to cut the trial write size in half and try again until the size is less than the filesystem block size.

Comments?

lib/tst_fill_fs.c +9 -0
@@ -6,6 +6,7 @@
 #include <errno.h>
 #include <stdio.h>
 #include <stdlib.h>
+#include <sys/statvfs.h>
 
 #define TST_NO_DEFAULT_MAIN
 #include "tst_test.h"
@@ -19,6 +20,8 @@ void tst_fill_fs(const char *path, int verbose)
        size_t len;
        ssize_t ret;
        int fd;
+       struct statvfs fi;
+       statvfs(path, &fi);
 
        for (;;) {
                len = random() % (1024 * 102400);
@@ -41,6 +44,12 @@ void tst_fill_fs(const char *path, int verbose)
                        ret = write(fd, buf, MIN(len, sizeof(buf)));
 
                        if (ret < 0) {
+                               if (errno == ENOSPC) {
+                                       len /= 2;
+                                       if (len >= fi.f_bsize)
+                                               continue;
+                               }
+
                                SAFE_CLOSE(fd);
 
                                if (errno != ENOSPC)
Quoted 10 lines

Increase FALLOCATE_SIZE to minimize chance of hitting sporadic failures when that happens.

Thanks to Carlos Maiolino and Eric Sandeen for their comments and suggestions.

Fixes #610

Quoted 23 lines
---
 testcases/kernel/syscalls/fallocate/fallocate05.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
testcases/kernel/syscalls/fallocate/fallocate05.c +1 -1
index 50c610c448ba..17034e5b11e7 100644
@@ -17,7 +17,7 @@
 #include "lapi/fallocate.h"
 
 #define MNTPOINT "mntpoint"
-#define FALLOCATE_SIZE 8192
+#define FALLOCATE_SIZE (1024*1024)
 #define TESTED_FLAGS "fallocate(FALLOC_FL_PUNCH_HOLE |

FALLOC_FL_KEEP_SIZE)"

static int fd;
--
1.8.3.1

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20191119/9aa93bfa/attachment.htm>
#5 lore raw

PATCH 1/1 fallocate05: increase FALLOCATE_SIZE

----- Original Message -----

Quoted 6 lines

Another patch I was thinking is to enhance the tst_fill_fs routine, which as Eric suggested, makes more reliably to get to a full filesystem. Something like what xfstest does to cut the trial write size in half and try again until the size is less than the filesystem block size.

Comments?

fallocate05 seems to be the only test using it, but in general I think we can do that too. Assuming this alone would be reliable, is there any advantage of running test with small FALLOCATE_SIZE?

Quoted 40 lines
lib/tst_fill_fs.c +9 -0
@@ -6,6 +6,7 @@
 #include <errno.h>
 #include <stdio.h>
 #include <stdlib.h>
+#include <sys/statvfs.h>
 
 #define TST_NO_DEFAULT_MAIN
 #include "tst_test.h"
@@ -19,6 +20,8 @@ void tst_fill_fs(const char *path, int verbose)
        size_t len;
        ssize_t ret;
        int fd;
+       struct statvfs fi;
+       statvfs(path, &fi);
 
        for (;;) {
                len = random() % (1024 * 102400);
@@ -41,6 +44,12 @@ void tst_fill_fs(const char *path, int verbose)
                        ret = write(fd, buf, MIN(len, sizeof(buf)));
 
                        if (ret < 0) {
+                               if (errno == ENOSPC) {
+                                       len /= 2;
+                                       if (len >= fi.f_bsize)
+                                               continue;
+                               }
+
                                SAFE_CLOSE(fd);
 
                                if (errno != ENOSPC)
Quoted 7 lines

Increase FALLOCATE_SIZE to minimize chance of hitting sporadic failures when that happens.

Quoted 7 lines
Quoted 6 lines

Thanks to Carlos Maiolino and Eric Sandeen for their comments and suggestions.

Fixes #610

Thanks, I pushed this patch for now.

#6 lore raw

PATCH 1/1 fallocate05: increase FALLOCATE_SIZE

Hi,

Quoted 12 lines · Jan Stancek

----- Original Message -----

Quoted 6 lines

Another patch I was thinking is to enhance the tst_fill_fs routine, which as Eric suggested, makes more reliably to get to a full filesystem. Something like what xfstest does to cut the trial write size in half and try again until the size is less than the filesystem block size.

Comments?

fallocate05 seems to be the only test using it, but in general I think we

Thanks, I will format a patch.

Quoted 3 lines

can do that too. Assuming this alone would be reliable, is there any advantage of running test with small FALLOCATE_SIZE?

Hmm, maybe no, my purpose to shrink the FALLOCATE_SIZE is just to reproduce the problem. I thought that if the filesystem is not 'really' full, then the fallocate() should succeed with a smaller size, only for degging.

Quoted 52 lines
Quoted 41 lines
lib/tst_fill_fs.c +9 -0
@@ -6,6 +6,7 @@
 #include <errno.h>
 #include <stdio.h>
 #include <stdlib.h>
+#include <sys/statvfs.h>
 
 #define TST_NO_DEFAULT_MAIN
 #include "tst_test.h"
@@ -19,6 +20,8 @@ void tst_fill_fs(const char *path, int verbose)
        size_t len;
        ssize_t ret;
        int fd;
+       struct statvfs fi;
+       statvfs(path, &fi);
 
        for (;;) {
                len = random() % (1024 * 102400);
@@ -41,6 +44,12 @@ void tst_fill_fs(const char *path, int verbose)
                        ret = write(fd, buf, MIN(len, sizeof(buf)));
 
                        if (ret < 0) {
+                               if (errno == ENOSPC) {
+                                       len /= 2;
+                                       if (len >= fi.f_bsize)
+                                               continue;
+                               }
+
                                SAFE_CLOSE(fd);
 
                                if (errno != ENOSPC)
Quoted 7 lines

Increase FALLOCATE_SIZE to minimize chance of hitting sporadic failures when that happens.

Quoted 7 lines
Quoted 6 lines

Thanks to Carlos Maiolino and Eric Sandeen for their comments and suggestions.

Fixes #610

Thanks, I pushed this patch for now.

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20191119/539ca4f6/attachment-0001.htm>
#7 lore raw

PATCH 1/1 fallocate05: increase FALLOCATE_SIZE

In reply to Jan Stancek
Quoted 8 lines · Jan Stancek

----- Original Message -----

Quoted 5 lines

Another patch I was thinking is to enhance the tst_fill_fs routine, which as Eric suggested, makes more reliably to get to a full filesystem. Something like what xfstest does to cut the trial write size in half and try again until the size is less than the filesystem block size.

Quoted 5 lines
Quoted 1 line

Comments?

fallocate05 seems to be the only test using it, but in general I think we can do that too. Assuming this alone would be reliable, is there any advantage of running test with small FALLOCATE_SIZE?

Note that simply increasing FALLOCATE_SIZE will not fix an incorrect pass when the file system wasn't completely full. Here's the code that checks whether some blocks were properly fallocate()d:

tst_fill_fs(MNTPOINT, 1);
ret = write(fd, buf, sizeof(buf));
if (ret < 0)

	tst_res(TFAIL | TERRNO, "write() failed unexpectedly");
else
	tst_res(TPASS, "write() wrote %zu bytes", ret);

If the file system somehow finds a few free blocks after tst_fill_fs() returns, short write() will still count as a pass.

Signature · 7 lines
Martin Doucha   [email protected]
QA Engineer for Software Maintenance
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic
#8 lore raw

PATCH 1/1 fallocate05: increase FALLOCATE_SIZE

----- Original Message -----

Quoted 12 lines
Quoted 4 lines

----- Original Message -----
fallocate05 seems to be the only test using it, but in general I think we can do that too. Assuming this alone would be reliable, is there any advantage of running test with small FALLOCATE_SIZE?

Note that simply increasing FALLOCATE_SIZE will not fix an incorrect pass when the file system wasn't completely full. Here's the code that checks whether some blocks were properly fallocate()d:

tst_fill_fs(MNTPOINT, 1);
ret = write(fd, buf, sizeof(buf));
if (ret < 0)

Quoted 6 lines
	tst_res(TFAIL | TERRNO, "write() failed unexpectedly");
else
	tst_res(TPASS, "write() wrote %zu bytes", ret);

If the file system somehow finds a few free blocks after tst_fill_fs() returns, short write() will still count as a pass.

That is good point, but that seems like issue that existed even with 8k FALLOCATE_SIZE, right?