Skip to contentSkip to conversation

LTP random fixes for xfs and btrfs

PATCH In review Zorro Lang 6 participants 22 messages Dec 1, 2024 – Dec 11, 2024

Subsystems (2)
lore
Share & tools
lore

Zorro Lang

author
#0 lore raw

cover LTP random fixes for xfs and btrfs

[PATCH 1/3] ioctl_ficlone02.c: set all_filesystems to zero

It doesn't skip filesystems as its plan, fix it.

[PATCH 2/3] stat04+lstat03: fix bad blocksize mkfs option for xfs

mkfs.xfs doesn't support "-b 1024", needs "-b size=1024"

[PATCH 3/3] stat04+lstat03: skip test on btrfs

The "-b" option of mkfs.btrfs isn't a blocksize option, there's not blocksize option in mkfs.btrfs. So I'd like to skip this test for btrfs. But I'm not sure if there's better way, so CC *btrfs list* to get more review points for that.
(BTW, better to have a common helper to deal with different filesystems'
blocksize options in the future)

author
#1 lore raw

PATCH 1/3 ioctl_ficlone02.c: set all_filesystems to zero

This test need to skip test known filesystems, but according to below code logic (in lib/tst_test.c):

  if (!tst_test->all_filesystems && tst_test->skip_filesystems) {
        long fs_type = tst_fs_type(".");
        const char *fs_name = tst_fs_type_name(fs_type);
        if (tst_fs_in_skiplist(fs_name, tst_test->skip_filesystems)) {
            tst_brk(TCONF, "%s is not supported by the test",
            fs_name);
        }

        tst_res(TINFO, "%s is supported by the test", fs_name);

}

if all_filesystems is 1, the skip_filesystems doesn't work. So set all_filesystems to 0.

---
 testcases/kernel/syscalls/ioctl/ioctl_ficlone02.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
testcases/kernel/syscalls/ioctl/ioctl_ficlone02.c +1 -1
index fab0daaee..b7f676ec7 100644
@@ -57,7 +57,7 @@ static struct tst_test test = {
 	.needs_root = 1,
 	.mount_device = 1,
 	.mntpoint = MNTPOINT,
-	.all_filesystems = 1,
+	.all_filesystems = 0,
 	.skip_filesystems = (const char *[]) {
 		"bcachefs",
 		"btrfs",
Signature · 1 line
2.45.2
author
#16 lore raw

PATCH 2/3 stat04+lstat03: fix bad blocksize mkfs option for xfs

In reply to Zorro Lang

Not all filesystems use "-b 1024" to set its blocksize. XFS uses "-b size=1024", so this test fails as "unknown option -b 1024" on xfs.

---
 testcases/kernel/syscalls/lstat/lstat03.c | 8 ++++++--
 testcases/kernel/syscalls/stat/stat04.c   | 8 ++++++--
 2 files changed, 12 insertions(+), 4 deletions(-)
testcases/kernel/syscalls/lstat/lstat03.c +6 -2
index d48af180b..675fb56f4 100644
@@ -44,8 +44,9 @@ static void run(void)
 
 static void setup(void)
 {
+	char *opt_name="-b";
 	char opt_bsize[32];
-	const char *const fs_opts[] = {opt_bsize, NULL};
+	const char *const fs_opts[] = {opt_name, opt_bsize, NULL};
 	struct stat sb;
 	int pagesize;
 	int fd;
@@ -54,7 +55,10 @@ static void setup(void)
 	SAFE_STAT(".", &sb);
 	pagesize = sb.st_blksize == 4096 ? 1024 : 4096;
 
-	snprintf(opt_bsize, sizeof(opt_bsize), "-b %i", pagesize);
+	if (strcmp(tst_device->fs_type, "xfs") == 0)
+		snprintf(opt_bsize, sizeof(opt_bsize), "size=%i", pagesize);
+	else
+		snprintf(opt_bsize, sizeof(opt_bsize), "%i", pagesize);
 	SAFE_MKFS(tst_device->dev, tst_device->fs_type, fs_opts, NULL);
 	SAFE_MOUNT(tst_device->dev, MNTPOINT, tst_device->fs_type, 0, 0);
 
testcases/kernel/syscalls/stat/stat04.c +6 -2
index 05ace606a..2a17cc7d7 100644
@@ -43,8 +43,9 @@ static void run(void)
 
 static void setup(void)
 {
+	char *opt_name="-b";
 	char opt_bsize[32];
-	const char *const fs_opts[] = {opt_bsize, NULL};
+	const char *const fs_opts[] = {opt_name, opt_bsize, NULL};
 	struct stat sb;
 	int pagesize;
 	int fd;
@@ -56,7 +57,10 @@ static void setup(void)
 	SAFE_STAT(".", &sb);
 	pagesize = sb.st_blksize == 4096 ? 1024 : 4096;
 
-	snprintf(opt_bsize, sizeof(opt_bsize), "-b %i", pagesize);
+	if (strcmp(tst_device->fs_type, "xfs") == 0)
+		snprintf(opt_bsize, sizeof(opt_bsize), "size=%i", pagesize);
+	else
+		snprintf(opt_bsize, sizeof(opt_bsize), "%i", pagesize);
 	SAFE_MKFS(tst_device->dev, tst_device->fs_type, fs_opts, NULL);
 	SAFE_MOUNT(tst_device->dev, MNTPOINT, tst_device->fs_type, 0, 0);
 
Signature · 1 line
2.45.2
author
#18 lore raw

PATCH 3/3 stat04+lstat03: skip test on btrfs

In reply to Zorro Lang

The "-b" option for mkfs.btrfs isn't a blocksize option, it does "specify the size of each device as seen by the filesystem" for btrfs. There's not an blocksize mkfs option for btrfs, so skip this test.

---
 testcases/kernel/syscalls/lstat/lstat03.c | 2 ++
 testcases/kernel/syscalls/stat/stat04.c   | 2 ++
 2 files changed, 4 insertions(+)
testcases/kernel/syscalls/lstat/lstat03.c +2 -0
index 675fb56f4..f7346893d 100644
@@ -57,6 +57,8 @@ static void setup(void)
 
 	if (strcmp(tst_device->fs_type, "xfs") == 0)
 		snprintf(opt_bsize, sizeof(opt_bsize), "size=%i", pagesize);
+	else if (strcmp(tst_device->fs_type, "btrfs") == 0)
+		tst_brk(TCONF, "btrfs is not supported");
 	else
 		snprintf(opt_bsize, sizeof(opt_bsize), "%i", pagesize);
 	SAFE_MKFS(tst_device->dev, tst_device->fs_type, fs_opts, NULL);
testcases/kernel/syscalls/stat/stat04.c +2 -0
index 2a17cc7d7..3c4f1a6b4 100644
@@ -59,6 +59,8 @@ static void setup(void)
 
 	if (strcmp(tst_device->fs_type, "xfs") == 0)
 		snprintf(opt_bsize, sizeof(opt_bsize), "size=%i", pagesize);
+	else if (strcmp(tst_device->fs_type, "btrfs") == 0)
+		tst_brk(TCONF, "btrfs is not supported");
 	else
 		snprintf(opt_bsize, sizeof(opt_bsize), "%i", pagesize);
 	SAFE_MKFS(tst_device->dev, tst_device->fs_type, fs_opts, NULL);
Signature · 1 line
2.45.2
#20 lore raw
In reply to Zorro Lang
Quoted 8 lines · Zorro Lang

[PATCH 1/3] ioctl_ficlone02.c: set all_filesystems to zero

It doesn't skip filesystems as its plan, fix it.

[PATCH 2/3] stat04+lstat03: fix bad blocksize mkfs option for xfs

   mkfs.xfs doesn't support "-b 1024", needs "-b size=1024"
Quoted 9 lines

[PATCH 3/3] stat04+lstat03: skip test on btrfs

The "-b" option of mkfs.btrfs isn't a blocksize option, there's not blocksize option in mkfs.btrfs. So I'd like to skip this test for btrfs. But I'm not sure if there's better way, so CC *btrfs list* to get more review points for that.
(BTW, better to have a common helper to deal with different filesystems'
blocksize options in the future)

Well, I'd say Wilcox is kinda correct here.

Btrfs uses the name "sector size" to indicate the minimal unit, aka, the blocksize of all the other fses.

Not sure if we will even rename the whole sector size to block size in the future, it looks like a huge work to do.

However there is another problem related to the btrfs block size (aka, "sector size").
Our block size starts at 4K, ends at 64K, and we do not yet support block size > page size.

For systems with 64K page size, although we have the support for block size < page size, I have added an artificial limits, that only 4K block size and page size is supported.

So even if we added btrfs block size support, it won't really work except 4K and page size (at least for now).

Thanks,
Qu

#2 lore raw

Re: [LTP] [PATCH 1/3] ioctl_ficlone02.c: set all_filesystems to zero

In reply to Zorro Lang

Hi!

Quoted 13 lines

This test need to skip test known filesystems, but according to below code logic (in lib/tst_test.c):

  if (!tst_test->all_filesystems && tst_test->skip_filesystems) {
        long fs_type = tst_fs_type(".");
        const char *fs_name = tst_fs_type_name(fs_type);
        if (tst_fs_in_skiplist(fs_name, tst_test->skip_filesystems)) {
            tst_brk(TCONF, "%s is not supported by the test",
            fs_name);
        }

        tst_res(TINFO, "%s is supported by the test", fs_name);
Quoted 4 lines

}

if all_filesystems is 1, the skip_filesystems doesn't work. So set all_filesystems to 0.

The code to skip filesystems in the case of all filesystems is in the run_tcase_per_fs() function:

static int run_tcases_per_fs(void)

{
        int ret = 0;
        unsigned int i;
        const char *const *filesystems = tst_get_supported_fs_types(tst_test->skip_filesystems);

The skip_filesystems array is passed to the tst_get_supporte_fs_types() function which filters out them.

Signature · 2 lines
Cyril Hrubis
[email protected]
#19 lore raw

Re: [LTP] [PATCH 3/3] stat04+lstat03: skip test on btrfs

In reply to Zorro Lang

Hi!

Quoted 16 lines

The "-b" option for mkfs.btrfs isn't a blocksize option, it does "specify the size of each device as seen by the filesystem" for btrfs. There's not an blocksize mkfs option for btrfs, so skip this test.

---
 testcases/kernel/syscalls/lstat/lstat03.c | 2 ++
 testcases/kernel/syscalls/stat/stat04.c   | 2 ++
 2 files changed, 4 insertions(+)
testcases/kernel/syscalls/lstat/lstat03.c +0 -0
index 675fb56f4..f7346893d 100644

testcases/kernel/syscalls/lstat/lstat03.c static void setup(void) · line 61

Quoted 5 lines
testcases/kernel/syscalls/lstat/lstat03.c +2 -0
@@ -57,6 +57,8 @@ static void setup(void)
 
 	if (strcmp(tst_device->fs_type, "xfs") == 0)
 		snprintf(opt_bsize, sizeof(opt_bsize), "size=%i", pagesize);
+	else if (strcmp(tst_device->fs_type, "btrfs") == 0)
+		tst_brk(TCONF, "btrfs is not supported");

This is overkill, all we need to skip on Btrfs is the st_blksize test. We need to set a flag in the test setup and then skip the corresponding TST_EXP_EXPR().

Signature · 2 lines
Cyril Hrubis
[email protected]
#17 lore raw

Re: [LTP] [PATCH 2/3] stat04+lstat03: fix bad blocksize mkfs option for xfs

In reply to Zorro Lang

Hi!

Quoted 15 lines

Not all filesystems use "-b 1024" to set its blocksize. XFS uses "-b size=1024", so this test fails as "unknown option -b 1024" on xfs.

---
 testcases/kernel/syscalls/lstat/lstat03.c | 8 ++++++--
 testcases/kernel/syscalls/stat/stat04.c   | 8 ++++++--
 2 files changed, 12 insertions(+), 4 deletions(-)
testcases/kernel/syscalls/lstat/lstat03.c +0 -0
index d48af180b..675fb56f4 100644

testcases/kernel/syscalls/lstat/lstat03.c static void run(void) · line 49

Quoted 7 lines
testcases/kernel/syscalls/lstat/lstat03.c +2 -1
@@ -44,8 +44,9 @@ static void run(void)
 
 static void setup(void)
 {
+	char *opt_name="-b";
 	char opt_bsize[32];
-	const char *const fs_opts[] = {opt_bsize, NULL};
+	const char *const fs_opts[] = {opt_name, opt_bsize, NULL};

^
You can just add "-b" here instead of creating a variable.

testcases/kernel/syscalls/lstat/lstat03.c static void setup(void) · line 58

Quoted 9 lines
testcases/kernel/syscalls/lstat/lstat03.c +1 -1
 	struct stat sb;
 	int pagesize;
 	int fd;
@@ -54,7 +55,10 @@ static void setup(void)
 	SAFE_STAT(".", &sb);
 	pagesize = sb.st_blksize == 4096 ? 1024 : 4096;
 
-	snprintf(opt_bsize, sizeof(opt_bsize), "-b %i", pagesize);
+	if (strcmp(tst_device->fs_type, "xfs") == 0)

^
The more common style is if (!strcmp(...))

testcases/kernel/syscalls/lstat/lstat03.c static void setup(void) · line 61

Quoted 3 lines
testcases/kernel/syscalls/lstat/lstat03.c +3 -0
+		snprintf(opt_bsize, sizeof(opt_bsize), "size=%i", pagesize);
+	else
+		snprintf(opt_bsize, sizeof(opt_bsize), "%i", pagesize);

Otherwise it looks good.

Signature · 2 lines
Cyril Hrubis
[email protected]
#3 lore raw

Re: [LTP] [PATCH 1/3] ioctl_ficlone02.c: set all_filesystems to zero

In reply to Cyril Hrubis

Hi!

Quoted 11 lines

The code to skip filesystems in the case of all filesystems is in the run_tcase_per_fs() function:

static int run_tcases_per_fs(void)

{
        int ret = 0;
        unsigned int i;
        const char *const *filesystems = tst_get_supported_fs_types(tst_test->skip_filesystems);

The skip_filesystems array is passed to the tst_get_supporte_fs_types() function which filters out them.

Perhaps you mean that the skiplist does not work with .all_filesystems _and_ the LTP_SINGLE_FS_TYPE environment variable?

I guess that we need:

lib/tst_supported_fs_types.c +4 -0
index bbbb8df19..49b1d7205 100644
@@ -159,6 +159,10 @@ const char **tst_get_supported_fs_types(const char *const *skiplist)
 
        if (only_fs) {
                tst_res(TINFO, "WARNING: testing only %s", only_fs);
+
+               if (tst_fs_in_skiplist(only_fs, skiplist))
+                       tst_brk(TCONF, "Requested filesystems is in test skiplist");
+
                if (tst_fs_is_supported(only_fs))
                        fs_types[0] = only_fs;
                return fs_types;
Signature · 2 lines
Cyril Hrubis
[email protected]
#4 lore raw

Re: [LTP] [PATCH 1/3] ioctl_ficlone02.c: set all_filesystems to zero

Quoted 23 lines

Hi!

Quoted 11 lines

The code to skip filesystems in the case of all filesystems is in the run_tcase_per_fs() function:

static int run_tcases_per_fs(void)

{
        int ret = 0;
        unsigned int i;
        const char *const *filesystems = tst_get_supported_fs_types(tst_test->skip_filesystems);

The skip_filesystems array is passed to the tst_get_supporte_fs_types() function which filters out them.

Perhaps you mean that the skiplist does not work with .all_filesystems _and_ the LTP_SINGLE_FS_TYPE environment variable?

I guess that we need:

lib/tst_supported_fs_types.c +0 -0
index bbbb8df19..49b1d7205 100644

lib/tst_supported_fs_types.c const char **tst_get_supported_fs_types(const char *const *skiplist) · line 165

Quoted 7 lines
lib/tst_supported_fs_types.c +4 -0
@@ -159,6 +159,10 @@ const char **tst_get_supported_fs_types(const char *const *skiplist)
 
        if (only_fs) {
                tst_res(TINFO, "WARNING: testing only %s", only_fs);
+
+               if (tst_fs_in_skiplist(only_fs, skiplist))
+                       tst_brk(TCONF, "Requested filesystems is in test skiplist");
+

It's a nice feature to be able to force testing on filesystem even it's set to be skipped without need to manually enable the filesystem and recompile. (It helps testing with LTP compiled as a package without need to compile LTP.) Therefore I would avoid this.

@Zorro Lang or are you testing whole syscalls on particular filesystem via LTP_SINGLE_FS_TYPE=xfs ?

Kind regards,
Petr

Quoted 3 lines
lib/tst_supported_fs_types.c +0 -0
                if (tst_fs_is_supported(only_fs))
                        fs_types[0] = only_fs;
                return fs_types;
#5 lore raw

Re: [LTP] [PATCH 1/3] ioctl_ficlone02.c: set all_filesystems to zero

Hi!

Quoted 4 lines

It's a nice feature to be able to force testing on filesystem even it's set to be skipped without need to manually enable the filesystem and recompile. (It helps testing with LTP compiled as a package without need to compile LTP.) Therefore I would avoid this.

I guess that this should be another env variable e.g. LTP_FORCE_SINGLE_FS_TYPE and it should print a big fat warning that it's only for development purposes.

Signature · 2 lines
Cyril Hrubis
[email protected]
#6 lore raw

Re: [LTP] [PATCH 1/3] ioctl_ficlone02.c: set all_filesystems to zero

Quoted 9 lines

Hi!

Quoted 4 lines

It's a nice feature to be able to force testing on filesystem even it's set to be skipped without need to manually enable the filesystem and recompile. (It helps testing with LTP compiled as a package without need to compile LTP.) Therefore I would avoid this.

I guess that this should be another env variable e.g. LTP_FORCE_SINGLE_FS_TYPE and it should print a big fat warning that it's only for development purposes.

Well, LTP_SINGLE_FS_TYPE already has "Testing only" and that was the reason it just forced filesystem regardless "skip" setup. Sure, we can turn it into "normal" variable and introduce LTP_FORCE_SINGLE_FS_TYPE if it's needed. But that would be an use case if anybody uses LTP really to test particular filesystem. And it would affect only .all_filesystem tests (e.g. user's responsibility would be to point TMPDIR to that particular system on non- .all_filesystem tests).

@Zorro was it meant like this?

Kind regards,
Petr

#7 lore raw

Re: [LTP] [PATCH 1/3] ioctl_ficlone02.c: set all_filesystems to zero

Hi!

Quoted 10 lines
Quoted 8 lines
Quoted 4 lines

It's a nice feature to be able to force testing on filesystem even it's set to be skipped without need to manually enable the filesystem and recompile. (It helps testing with LTP compiled as a package without need to compile LTP.) Therefore I would avoid this.

I guess that this should be another env variable e.g. LTP_FORCE_SINGLE_FS_TYPE and it should print a big fat warning that it's only for development purposes.

Well, LTP_SINGLE_FS_TYPE already has "Testing only"

That was maybe how we intended it but we exposed the API and it seems that there is a usecase for it so people are possibly using it.

Quoted 7 lines

and that was the reason it just forced filesystem regardless "skip" setup. Sure, we can turn it into "normal" variable and introduce LTP_FORCE_SINGLE_FS_TYPE if it's needed. But that would be an use case if anybody uses LTP really to test particular filesystem. And it would affect only .all_filesystem tests (e.g. user's responsibility would be to point TMPDIR to that particular system on non- .all_filesystem tests).

There is a usecase we haven't figured out, when you want to test a single filesystem, you do not want to waste time on the rest of the filesystems and LTP_SINGlE_FS_TYPE nearly works for that usecase.

Signature · 2 lines
Cyril Hrubis
[email protected]
#8 lore raw

Re: [LTP] [PATCH 1/3] ioctl_ficlone02.c: set all_filesystems to zero

Quoted 23 lines

Hi!

Quoted 10 lines
Quoted 8 lines
Quoted 4 lines

It's a nice feature to be able to force testing on filesystem even it's set to be skipped without need to manually enable the filesystem and recompile. (It helps testing with LTP compiled as a package without need to compile LTP.) Therefore I would avoid this.

I guess that this should be another env variable e.g. LTP_FORCE_SINGLE_FS_TYPE and it should print a big fat warning that it's only for development purposes.

Well, LTP_SINGLE_FS_TYPE already has "Testing only"

That was maybe how we intended it but we exposed the API and it seems that there is a usecase for it so people are possibly using it.

Quoted 7 lines

and that was the reason it just forced filesystem regardless "skip" setup. Sure, we can turn it into "normal" variable and introduce LTP_FORCE_SINGLE_FS_TYPE if it's needed. But that would be an use case if anybody uses LTP really to test particular filesystem. And it would affect only .all_filesystem tests (e.g. user's responsibility would be to point TMPDIR to that particular system on non- .all_filesystem tests).

Quoted 3 lines

There is a usecase we haven't figured out, when you want to test a single filesystem, you do not want to waste time on the rest of the filesystems and LTP_SINGlE_FS_TYPE nearly works for that usecase.

Yes, I understood this use case, this was probably the reason of this PR. I'd appreciate Zorro's confirmation we understood him properly (e.g. that somebody really needs this). I suppose you vote for this feature anyway and it's fairly simple, thus I can prepare it.

Kind regards,
Petr

#21 lore raw
In reply to Qu Wenruo
Quoted 21 lines · Qu Wenruo
Quoted 8 lines · Zorro Lang

[PATCH 1/3] ioctl_ficlone02.c: set all_filesystems to zero

It doesn't skip filesystems as its plan, fix it.

[PATCH 2/3] stat04+lstat03: fix bad blocksize mkfs option for xfs

   mkfs.xfs doesn't support "-b 1024", needs "-b size=1024"
Quoted 9 lines

[PATCH 3/3] stat04+lstat03: skip test on btrfs

The "-b" option of mkfs.btrfs isn't a blocksize option, there's not blocksize option in mkfs.btrfs. So I'd like to skip this test for btrfs. But I'm not sure if there's better way, so CC *btrfs list* to get more review points for that.
(BTW, better to have a common helper to deal with different filesystems'
blocksize options in the future)

Quoted 7 lines

Well, I'd say Wilcox is kinda correct here.

Btrfs uses the name "sector size" to indicate the minimal unit, aka, the blocksize of all the other fses.

Not sure if we will even rename the whole sector size to block size in the future, it looks like a huge work to do.

Well, I think we can at least add an alias blocksize to sectorsize to mkfs. We don't have a time machine to change the initial confusing naming, but can slightly improve the user convenience. Internally in the code we can keep sectorsize or incrementally rename it to blocksize. What matters more here is the user intrface, i.e. the mkfs options.

author
#9 lore raw

Re: [LTP] [PATCH 1/3] ioctl_ficlone02.c: set all_filesystems to zero

In reply to Petr Vorel
Quoted 36 lines · Petr Vorel
Quoted 23 lines

Hi!

Quoted 11 lines

The code to skip filesystems in the case of all filesystems is in the run_tcase_per_fs() function:

static int run_tcases_per_fs(void)

{
        int ret = 0;
        unsigned int i;
        const char *const *filesystems = tst_get_supported_fs_types(tst_test->skip_filesystems);

The skip_filesystems array is passed to the tst_get_supporte_fs_types() function which filters out them.

Perhaps you mean that the skiplist does not work with .all_filesystems _and_ the LTP_SINGLE_FS_TYPE environment variable?

I guess that we need:

lib/tst_supported_fs_types.c +0 -0
index bbbb8df19..49b1d7205 100644

lib/tst_supported_fs_types.c const char **tst_get_supported_fs_types(const char *const *skiplist) · line 165

Quoted 7 lines
lib/tst_supported_fs_types.c +4 -0
@@ -159,6 +159,10 @@ const char **tst_get_supported_fs_types(const char *const *skiplist)
 
        if (only_fs) {
                tst_res(TINFO, "WARNING: testing only %s", only_fs);
+
+               if (tst_fs_in_skiplist(only_fs, skiplist))
+                       tst_brk(TCONF, "Requested filesystems is in test skiplist");
+

It's a nice feature to be able to force testing on filesystem even it's set to be skipped without need to manually enable the filesystem and recompile. (It helps testing with LTP compiled as a package without need to compile LTP.) Therefore I would avoid this.

Quoted 2 lines

@Zorro Lang or are you testing whole syscalls on particular filesystem via LTP_SINGLE_FS_TYPE=xfs ?

Oh, yes, I always use LTP with different LTP_SINGLE_FS_TYPE. So that's might be the problem?

Thanks,
Zorro

Quoted 11 lines

Kind regards,
Petr

Quoted 3 lines
lib/tst_supported_fs_types.c +0 -0
                if (tst_fs_is_supported(only_fs))
                        fs_types[0] = only_fs;
                return fs_types;
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#10 lore raw

Re: [LTP] [PATCH 1/3] ioctl_ficlone02.c: set all_filesystems to zero

Quoted 37 lines · Petr Vorel
Quoted 35 lines
Quoted 23 lines

Hi!

Quoted 11 lines

The code to skip filesystems in the case of all filesystems is in the run_tcase_per_fs() function:

static int run_tcases_per_fs(void)

{
        int ret = 0;
        unsigned int i;
        const char *const *filesystems = tst_get_supported_fs_types(tst_test->skip_filesystems);

The skip_filesystems array is passed to the tst_get_supporte_fs_types() function which filters out them.

Perhaps you mean that the skiplist does not work with .all_filesystems _and_ the LTP_SINGLE_FS_TYPE environment variable?

I guess that we need:

lib/tst_supported_fs_types.c +0 -0
index bbbb8df19..49b1d7205 100644

lib/tst_supported_fs_types.c const char **tst_get_supported_fs_types(const char *const *skiplist) · line 165

Quoted 7 lines
lib/tst_supported_fs_types.c +4 -0
@@ -159,6 +159,10 @@ const char **tst_get_supported_fs_types(const char *const *skiplist)
 
        if (only_fs) {
                tst_res(TINFO, "WARNING: testing only %s", only_fs);
+
+               if (tst_fs_in_skiplist(only_fs, skiplist))
+                       tst_brk(TCONF, "Requested filesystems is in test skiplist");
+

It's a nice feature to be able to force testing on filesystem even it's set to be skipped without need to manually enable the filesystem and recompile. (It helps testing with LTP compiled as a package without need to compile LTP.) Therefore I would avoid this.

Quoted 5 lines
Quoted 2 lines

@Zorro Lang or are you testing whole syscalls on particular filesystem via LTP_SINGLE_FS_TYPE=xfs ?

Oh, yes, I always use LTP with different LTP_SINGLE_FS_TYPE. So that's might be the problem?

Thanks for confirming your use case.

Well, "Testing only" in the help (-h) was added there to suggest it's for testing/debugging LTP, not a production testing. But newer mind, I'll implement Cyril's suggestion, real usage justify it. + I'll add LTP_FORCE_SINGLE_FS_TYPE.

We could allow more filesystems, e.g. instead of running LTP few times with different LTP_SINGLE_FS_TYPE value: e.g.

for fs in ext4 xfs btrfs; do LTP_SINGLE_FS_TYPE=fs ioctl_ficlone02; done

we could introduce support for particular filesystems LTP_FILESYSTEMS="ext4,xfs,btrfs" ioctl_ficlone02

(Probably define new variable because "SINGLE" is misleading when supporting more filesystems. Also when we touch it, I would consider renaming variable FILESYSTEMS is more obvious for newcomers than "FS_TYPE".)

WDYT?

Kind regards,
Petr

Quoted 2 lines

Thanks,
Zorro

#11 lore raw

Re: [LTP] [PATCH 1/3] ioctl_ficlone02.c: set all_filesystems to zero

Quoted 54 lines · Petr Vorel
Quoted 37 lines · Petr Vorel
Quoted 35 lines
Quoted 23 lines

Hi!

Quoted 11 lines

The code to skip filesystems in the case of all filesystems is in the run_tcase_per_fs() function:

static int run_tcases_per_fs(void)

{
        int ret = 0;
        unsigned int i;
        const char *const *filesystems = tst_get_supported_fs_types(tst_test->skip_filesystems);

The skip_filesystems array is passed to the tst_get_supporte_fs_types() function which filters out them.

Perhaps you mean that the skiplist does not work with .all_filesystems _and_ the LTP_SINGLE_FS_TYPE environment variable?

I guess that we need:

lib/tst_supported_fs_types.c +0 -0
index bbbb8df19..49b1d7205 100644

lib/tst_supported_fs_types.c const char **tst_get_supported_fs_types(const char *const *skiplist) · line 165

Quoted 7 lines
lib/tst_supported_fs_types.c +4 -0
@@ -159,6 +159,10 @@ const char **tst_get_supported_fs_types(const char *const *skiplist)
 
        if (only_fs) {
                tst_res(TINFO, "WARNING: testing only %s", only_fs);
+
+               if (tst_fs_in_skiplist(only_fs, skiplist))
+                       tst_brk(TCONF, "Requested filesystems is in test skiplist");
+

It's a nice feature to be able to force testing on filesystem even it's set to be skipped without need to manually enable the filesystem and recompile. (It helps testing with LTP compiled as a package without need to compile LTP.) Therefore I would avoid this.

Quoted 5 lines
Quoted 2 lines

@Zorro Lang or are you testing whole syscalls on particular filesystem via LTP_SINGLE_FS_TYPE=xfs ?

Oh, yes, I always use LTP with different LTP_SINGLE_FS_TYPE. So that's might be the problem?

Thanks for confirming your use case.

Well, "Testing only" in the help (-h) was added there to suggest it's for testing/debugging LTP, not a production testing. But newer mind, I'll implement Cyril's suggestion, real usage justify it. + I'll add LTP_FORCE_SINGLE_FS_TYPE.

We could allow more filesystems, e.g. instead of running LTP few times with different LTP_SINGLE_FS_TYPE value: e.g.

for fs in ext4 xfs btrfs; do LTP_SINGLE_FS_TYPE=fs ioctl_ficlone02; done

Quoted 8 lines

we could introduce support for particular filesystems LTP_FILESYSTEMS="ext4,xfs,btrfs" ioctl_ficlone02

(Probably define new variable because "SINGLE" is misleading when supporting more filesystems. Also when we touch it, I would consider renaming variable FILESYSTEMS is more obvious for newcomers than "FS_TYPE".)

WDYT?

I generally mount a known filesystem on /mnt/ltpdir, and specify "-d /mnt/ltpdir" (to be TMPDIR). Then set (export)
LTP_SINGLE_FS_TYPE
LTP_DEV_FS_TYPE
LTP_BIG_DEV_FS_TYPE
to be the same fs with the one mounted on $TMPDIR. Then run ltp more likes:

for cmdfile in fs fs_perms_simple syscalls fs_bind fsx mm commands fcntl-locktests fs_readonly ipc io containers cve smoketest syscalls-ipc dio ltp-aiodio.part1 ltp-aiodio.part2 ltp-aiodio.part3 ltp-aiodio.part4 ltp-aio-stress; do
runltp -p -d /mnt/ltpdir -b /dev/loop0 -B xfs -z /dev/loop1 -Z xfs -f $cmdfile ...
done

I do this to test each filesystem in different test jobs. Each test job test one fs. I think that might make sure all system calls/IOs test on same filesystem, no matter on TMPDIR or someone device. Am I right? Or I missunderstand something?

Thanks,
Zorro

Quoted 10 lines

Kind regards,
Petr

Quoted 2 lines

Thanks,
Zorro

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#12 lore raw

Re: [LTP] [PATCH 1/3] ioctl_ficlone02.c: set all_filesystems to zero

Quoted 71 lines · Petr Vorel
Quoted 54 lines
Quoted 37 lines · Petr Vorel
Quoted 35 lines
Quoted 23 lines

Hi!

Quoted 11 lines

The code to skip filesystems in the case of all filesystems is in the run_tcase_per_fs() function:

static int run_tcases_per_fs(void)

{
        int ret = 0;
        unsigned int i;
        const char *const *filesystems = tst_get_supported_fs_types(tst_test->skip_filesystems);

The skip_filesystems array is passed to the tst_get_supporte_fs_types() function which filters out them.

Perhaps you mean that the skiplist does not work with .all_filesystems _and_ the LTP_SINGLE_FS_TYPE environment variable?

I guess that we need:

lib/tst_supported_fs_types.c +0 -0
index bbbb8df19..49b1d7205 100644

lib/tst_supported_fs_types.c const char **tst_get_supported_fs_types(const char *const *skiplist) · line 165

Quoted 7 lines
lib/tst_supported_fs_types.c +4 -0
@@ -159,6 +159,10 @@ const char **tst_get_supported_fs_types(const char *const *skiplist)
 
        if (only_fs) {
                tst_res(TINFO, "WARNING: testing only %s", only_fs);
+
+               if (tst_fs_in_skiplist(only_fs, skiplist))
+                       tst_brk(TCONF, "Requested filesystems is in test skiplist");
+

It's a nice feature to be able to force testing on filesystem even it's set to be skipped without need to manually enable the filesystem and recompile. (It helps testing with LTP compiled as a package without need to compile LTP.) Therefore I would avoid this.

Quoted 5 lines
Quoted 2 lines

@Zorro Lang or are you testing whole syscalls on particular filesystem via LTP_SINGLE_FS_TYPE=xfs ?

Oh, yes, I always use LTP with different LTP_SINGLE_FS_TYPE. So that's might be the problem?

Thanks for confirming your use case.

Well, "Testing only" in the help (-h) was added there to suggest it's for testing/debugging LTP, not a production testing. But newer mind, I'll implement Cyril's suggestion, real usage justify it. + I'll add LTP_FORCE_SINGLE_FS_TYPE.

We could allow more filesystems, e.g. instead of running LTP few times with different LTP_SINGLE_FS_TYPE value: e.g.

for fs in ext4 xfs btrfs; do LTP_SINGLE_FS_TYPE=fs ioctl_ficlone02; done

Quoted 8 lines

we could introduce support for particular filesystems LTP_FILESYSTEMS="ext4,xfs,btrfs" ioctl_ficlone02

(Probably define new variable because "SINGLE" is misleading when supporting more filesystems. Also when we touch it, I would consider renaming variable FILESYSTEMS is more obvious for newcomers than "FS_TYPE".)

WDYT?

I generally mount a known filesystem on /mnt/ltpdir, and specify "-d /mnt/ltpdir" (to be TMPDIR). Then set (export)
LTP_SINGLE_FS_TYPE
LTP_DEV_FS_TYPE
LTP_BIG_DEV_FS_TYPE
to be the same fs with the one mounted on $TMPDIR. Then run ltp more likes:

Quoted 7 lines

for cmdfile in fs fs_perms_simple syscalls fs_bind fsx mm commands fcntl-locktests fs_readonly ipc io containers cve smoketest syscalls-ipc dio ltp-aiodio.part1 ltp-aiodio.part2 ltp-aiodio.part3 ltp-aiodio.part4 ltp-aio-stress; do
runltp -p -d /mnt/ltpdir -b /dev/loop0 -B xfs -z /dev/loop1 -Z xfs -f $cmdfile ...
done

I do this to test each filesystem in different test jobs. Each test job test one fs. I think that might make sure all system calls/IOs test on same filesystem, no matter on TMPDIR or someone device. Am I right? Or I missunderstand something?

IMHO we intended slightly different behavior:

point TMPDIR to the filesystem you are interested in (e.g. /var/tmp for openSUSE to test Btrfs instead of tmpfs).
Most of the tests are filesystem agnostic (not related to file/filesystem or using common VFS code), thus this setup is ok (just to avoid testing tmpfs which is in RAM instead of real block device). Running these tests "manually" for all filesystems should be waste of time. If you see different results, it might mean that some system should use .all_filesystems = 1.

These tests where filesystem matters should use .all_filesystems = 1 setup, thus they run on all available filesystems. You just need to make sure that SUT has required dependencies (mkfs.* binary and filesystem configured in kernel).

Kind regards,
Petr

Quoted 12 lines

Thanks,
Zorro

Quoted 8 lines

Kind regards,
Petr

Quoted 2 lines

Thanks,
Zorro

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#13 lore raw

Re: [LTP] [PATCH 1/3] ioctl_ficlone02.c: set all_filesystems to zero

In reply to Zorro Lang via ltp
Quoted 71 lines · Petr Vorel
Quoted 54 lines
Quoted 37 lines · Petr Vorel
Quoted 35 lines
Quoted 23 lines

Hi!

Quoted 11 lines

The code to skip filesystems in the case of all filesystems is in the run_tcase_per_fs() function:

static int run_tcases_per_fs(void)

{
        int ret = 0;
        unsigned int i;
        const char *const *filesystems = tst_get_supported_fs_types(tst_test->skip_filesystems);

The skip_filesystems array is passed to the tst_get_supporte_fs_types() function which filters out them.

Perhaps you mean that the skiplist does not work with .all_filesystems _and_ the LTP_SINGLE_FS_TYPE environment variable?

I guess that we need:

lib/tst_supported_fs_types.c +0 -0
index bbbb8df19..49b1d7205 100644

lib/tst_supported_fs_types.c const char **tst_get_supported_fs_types(const char *const *skiplist) · line 165

Quoted 7 lines
lib/tst_supported_fs_types.c +4 -0
@@ -159,6 +159,10 @@ const char **tst_get_supported_fs_types(const char *const *skiplist)
 
        if (only_fs) {
                tst_res(TINFO, "WARNING: testing only %s", only_fs);
+
+               if (tst_fs_in_skiplist(only_fs, skiplist))
+                       tst_brk(TCONF, "Requested filesystems is in test skiplist");
+

It's a nice feature to be able to force testing on filesystem even it's set to be skipped without need to manually enable the filesystem and recompile. (It helps testing with LTP compiled as a package without need to compile LTP.) Therefore I would avoid this.

Quoted 5 lines
Quoted 2 lines

@Zorro Lang or are you testing whole syscalls on particular filesystem via LTP_SINGLE_FS_TYPE=xfs ?

Oh, yes, I always use LTP with different LTP_SINGLE_FS_TYPE. So that's might be the problem?

Thanks for confirming your use case.

Well, "Testing only" in the help (-h) was added there to suggest it's for testing/debugging LTP, not a production testing. But newer mind, I'll implement Cyril's suggestion, real usage justify it. + I'll add LTP_FORCE_SINGLE_FS_TYPE.

We could allow more filesystems, e.g. instead of running LTP few times with different LTP_SINGLE_FS_TYPE value: e.g.

for fs in ext4 xfs btrfs; do LTP_SINGLE_FS_TYPE=fs ioctl_ficlone02; done

Quoted 8 lines

we could introduce support for particular filesystems LTP_FILESYSTEMS="ext4,xfs,btrfs" ioctl_ficlone02

(Probably define new variable because "SINGLE" is misleading when supporting more filesystems. Also when we touch it, I would consider renaming variable FILESYSTEMS is more obvious for newcomers than "FS_TYPE".)

WDYT?

I generally mount a known filesystem on /mnt/ltpdir, and specify "-d /mnt/ltpdir" (to be TMPDIR). Then set (export)
LTP_SINGLE_FS_TYPE
LTP_DEV_FS_TYPE
LTP_BIG_DEV_FS_TYPE
to be the same fs with the one mounted on $TMPDIR. Then run ltp more likes:

Quoted 3 lines

for cmdfile in fs fs_perms_simple syscalls fs_bind fsx mm commands fcntl-locktests fs_readonly ipc io containers cve smoketest syscalls-ipc dio ltp-aiodio.part1 ltp-aiodio.part2 ltp-aiodio.part3 ltp-aiodio.part4 ltp-aio-stress; do
runltp -p -d /mnt/ltpdir -b /dev/loop0 -B xfs -z /dev/loop1 -Z xfs -f $cmdfile ...
done

NOTE: I recommend you to test kirk [1], the runltp replacement. Output is in JSON, which is much easier to parse. It also has more features. We should add some documentation about kirk into LTP documentation.

Kind regards,
Petr

[1] https://github.com/linux-test-project/kirk

Quoted 16 lines

I do this to test each filesystem in different test jobs. Each test job test one fs. I think that might make sure all system calls/IOs test on same filesystem, no matter on TMPDIR or someone device. Am I right? Or I missunderstand something?

Thanks,
Zorro

Quoted 8 lines

Kind regards,
Petr

Quoted 2 lines

Thanks,
Zorro

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#14 lore raw

Re: [LTP] [PATCH 1/3] ioctl_ficlone02.c: set all_filesystems to zero

In reply to Petr Vorel

Hi!

Quoted 11 lines

Well, "Testing only" in the help (-h) was added there to suggest it's for testing/debugging LTP, not a production testing. But newer mind, I'll implement Cyril's suggestion, real usage justify it. + I'll add LTP_FORCE_SINGLE_FS_TYPE.

We could allow more filesystems, e.g. instead of running LTP few times with different LTP_SINGLE_FS_TYPE value: e.g.

for fs in ext4 xfs btrfs; do LTP_SINGLE_FS_TYPE=fs ioctl_ficlone02; done

we could introduce support for particular filesystems LTP_FILESYSTEMS="ext4,xfs,btrfs" ioctl_ficlone02

That is stil not equivalent if you run it with a whole LTP. We would have to run each test that uses device for all LTP_FILESYSTEMS, since many of the testcases does use device but does not use .all_filesystems.

So all in all I think that LTP_SINGLE_FS_TYPE is good enough solution.

Or we can try to rething the whole thing, but it's getting quite complicated with all the options we have.

Signature · 2 lines
Cyril Hrubis
[email protected]
#15 lore raw

Re: [LTP] [PATCH 1/3] ioctl_ficlone02.c: set all_filesystems to zero

Quoted 13 lines

Hi!

Quoted 11 lines

Well, "Testing only" in the help (-h) was added there to suggest it's for testing/debugging LTP, not a production testing. But newer mind, I'll implement Cyril's suggestion, real usage justify it. + I'll add LTP_FORCE_SINGLE_FS_TYPE.

We could allow more filesystems, e.g. instead of running LTP few times with different LTP_SINGLE_FS_TYPE value: e.g.

for fs in ext4 xfs btrfs; do LTP_SINGLE_FS_TYPE=fs ioctl_ficlone02; done

we could introduce support for particular filesystems LTP_FILESYSTEMS="ext4,xfs,btrfs" ioctl_ficlone02

Quoted 8 lines

That is stil not equivalent if you run it with a whole LTP. We would have to run each test that uses device for all LTP_FILESYSTEMS, since many of the testcases does use device but does not use .all_filesystems.

So all in all I think that LTP_SINGLE_FS_TYPE is good enough solution.

Or we can try to rething the whole thing, but it's getting quite complicated with all the options we have.

I guess LTP_SINGLE_FS_TYPE + LTP_FORCE_SINGLE_FS_TYPE LGTM.

What bothers me more how much time we spent with formatting loop device (done for each test with .all_filesystems several times). Running all tests on single filesystem, then reformat and run all of them on the other filesystem would be faster. The only thing which is better on this is theoretical chance that filesystem gets corrupted by some test, then other tests would be somehow influenced.

Kind regards,
Petr