Skip to contentSkip to conversation

lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

PATCH New Zorro Lang 3 participants 22 messages Mar 8, 2016 – Mar 14, 2016 Library code

lore
Share & tools
lore

Zorro Lang

author
#0 lore raw

PATCH 1/2 lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

If the device which pointed by -b or -z options is too large, mkfs on it will cost lots of time, some cases even return ETIMEDOUT error(e.g. mmap16). So I do below change:

Change tst_mkfs() function to tst_mkfs_sized(), add two new parameters:
fssize and blocksize. tst_mkfs_sized() can create an appointed size fs with appointed block size, or just point block size.

tst_mkfs() still can be used. It's defined as tst_mkfs_sized(..., NULL, NULL).

Until now tst_mkfs_sized() only support extX, xfs and btrfs, if we point fs size or block size parameters.

---
 include/test.h |  10 ++++--
 lib/tst_mkfs.c | 109 +++++++++++++++++++++++++++++++++++++++++++++++++++------
 2 files changed, 106 insertions(+), 13 deletions(-)
include/test.h +8 -2
index 5c78b1d..25d14d9 100644
@@ -317,9 +317,15 @@ int tst_system(const char *command);
  * @dev: path to a device
  * @fs_type: filesystem type
  * @fs_opts: NULL or NULL terminated array of extra mkfs options
+ * @fssize: fs size, can use k,m,g,t,p,e as unit. NULL can be accepted
+ * @blocksize: fs block size, can use k,m as unit. NULL can be accepted
  */
-void tst_mkfs(void (cleanup_fn)(void), const char *dev,
-	      const char *fs_type, const char *const fs_opts[]);
+void tst_mkfs_sized(void (cleanup_fn)(void), const char *dev,
+                    const char *fs_type, const char *const fs_opts[],
+                    const char *fssize, const char *blocksize);
+
+#define tst_mkfs(cleanup_fn, dev, fs_type, fs_opts) \
+	tst_mkfs_sized(cleanup_fn, dev, fs_type, fs_opts, NULL, NULL)
 
 /*
  * Returns filesystem type to be used for the testing. Unless your test is
lib/tst_mkfs.c +98 -11

146 lines not on this page · raw message

Signature · 1 line
2.5.0
author
#1 lore raw

PATCH 2/2 mmap16: fix ETIMEDOUT error if test device is too large

mmap16 will wait DEFAULT_MSEC_TIMEOUT=10000 msec, for parent process full the test device(-b $DEVICE). But if the device size is too large, this case will hit ETIMEDOUT error.

For reproduce the bug of mmap16, it don't need too large device. So I limit the fs size in 100M.

---
 testcases/kernel/syscalls/mmap/mmap16.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
testcases/kernel/syscalls/mmap/mmap16.c +1 -3
index c5828ea..f8fbd50 100644
@@ -143,8 +143,6 @@ static void do_test(void)
 
 static void setup(void)
 {
-	const char *fs_opts[3] = {"-b", "1024", NULL};
-
 	tst_sig(FORK, DEF_HANDLER, NULL);
 	tst_require_root();
 
@@ -158,7 +156,7 @@ static void setup(void)
 	device = tst_acquire_device(cleanup);
 	if (!device)
 		tst_brkm(TCONF, cleanup, "Failed to obtain block device");
-	tst_mkfs(cleanup, device, fs_type, fs_opts);
+	tst_mkfs(cleanup, device, fs_type, NULL, "100m", "1024");
 
 	SAFE_MKDIR(cleanup, MNTPOINT, 0755);
 	/*
Signature · 1 line
2.5.0
#2 lore raw

PATCH 2/2 mmap16: fix ETIMEDOUT error if test device is too large

Quoted 26 lines · Zorro Lang

mmap16 will wait DEFAULT_MSEC_TIMEOUT=10000 msec, for parent process full the test device(-b $DEVICE). But if the device size is too large, this case will hit ETIMEDOUT error.

For reproduce the bug of mmap16, it don't need too large device. So I limit the fs size in 100M.

---
 testcases/kernel/syscalls/mmap/mmap16.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
testcases/kernel/syscalls/mmap/mmap16.c +0 -2
index c5828ea..f8fbd50 100644
@@ -143,8 +143,6 @@ static void do_test(void)
 
 static void setup(void)
 {
-	const char *fs_opts[3] = {"-b", "1024", NULL};
-
 	tst_sig(FORK, DEF_HANDLER, NULL);
 	tst_require_root();
 

testcases/kernel/syscalls/mmap/mmap16.c static void setup(void) · line 159

Quoted 5 lines
testcases/kernel/syscalls/mmap/mmap16.c +1 -1
@@ -158,7 +156,7 @@ static void setup(void)
 	device = tst_acquire_device(cleanup);
 	if (!device)
 		tst_brkm(TCONF, cleanup, "Failed to obtain block device");
-	tst_mkfs(cleanup, device, fs_type, fs_opts);
+	tst_mkfs(cleanup, device, fs_type, NULL, "100m", "1024");

Should be calling tst_mkfs_sized() instead?

Thanks,
Eryu

Quoted 9 lines
testcases/kernel/syscalls/mmap/mmap16.c +0 -0
 	SAFE_MKDIR(cleanup, MNTPOINT, 0755);
 	/*
Signature · 5 lines
2.5.0


-- 
Mailing list info: http://lists.linux.it/listinfo/ltp
#3 lore raw

PATCH 2/2 mmap16: fix ETIMEDOUT error if test device is too large

----- 原始邮件 -----

Quoted 33 lines

发件人: "Eryu Guan" <[email protected]>
收件人: "Zorro Lang" <[email protected]>
抄送: [email protected]
发送时间: 星期三, 2016年 3 月 09日 上午 10:22:00
主题: Re: [LTP] [PATCH 2/2] mmap16: fix ETIMEDOUT error if test device is too large

Quoted 26 lines · Zorro Lang

mmap16 will wait DEFAULT_MSEC_TIMEOUT=10000 msec, for parent process full the test device(-b $DEVICE). But if the device size is too large, this case will hit ETIMEDOUT error.

For reproduce the bug of mmap16, it don't need too large device. So I limit the fs size in 100M.

---
 testcases/kernel/syscalls/mmap/mmap16.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
testcases/kernel/syscalls/mmap/mmap16.c +0 -2
index c5828ea..f8fbd50 100644
@@ -143,8 +143,6 @@ static void do_test(void)
 
 static void setup(void)
 {
-	const char *fs_opts[3] = {"-b", "1024", NULL};
-
 	tst_sig(FORK, DEF_HANDLER, NULL);
 	tst_require_root();
 
Quoted 8 lines

testcases/kernel/syscalls/mmap/mmap16.c static void setup(void) · line 159

Quoted 6 lines
testcases/kernel/syscalls/mmap/mmap16.c +1 -1
@@ -158,7 +156,7 @@ static void setup(void)
 	device = tst_acquire_device(cleanup);
 	if (!device)
 		tst_brkm(TCONF, cleanup, "Failed to obtain block device");
-	tst_mkfs(cleanup, device, fs_type, fs_opts);
+	tst_mkfs(cleanup, device, fs_type, NULL, "100m", "1024");

Should be calling tst_mkfs_sized() instead?

Wow! Yes, my mistake. Sorry, it's should be:
tst_mkfs_sized(cleanup, device, fs_type, NULL, "100m", "1024");

I will wait for the patch about tst_mkfs_sized() be reviewed, then change this patch.

My original idea is: after tst_mkfs_sized() be accepted, I should try to find all cases use tst_mkfs(), then check/fix if large device maybe cause them hit TIMEOUT error or run too long time.

This patch about mmap16 is only a demo, I want to follow the maintainer's suggestions. If I do all those works now, but LTP don't want that, that's waste time:)

Thanks,
Zorro

Quoted 13 lines

Thanks,
Eryu

Quoted 9 lines
testcases/kernel/syscalls/mmap/mmap16.c +0 -1
 	SAFE_MKDIR(cleanup, MNTPOINT, 0755);
 	/*
--

2.5.0

--
Mailing list info: http://lists.linux.it/listinfo/ltp

#4 lore raw

PATCH 1/2 lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

In reply to Zorro Lang

Hi!

Quoted 22 lines

#include "test.h"
#include "ltp_priv.h"

#define OPTS_MAX 32

lib/tst_mkfs.c +15 -2
-void tst_mkfs(void (cleanup_fn)(void), const char *dev,
-	      const char *fs_type, const char *const fs_opts[])
+long long cvtnum(const char *s)
+{
+	long long i;
+	char *sp = NULL;
+
+	i = strtoll(s, &sp, 0);
+	if (i == 0 && sp == s)
+		return -1LL;
+	if (*sp == '\0')
+		return i;
+
+	if (*sp == 'k' && sp[1] == '\0')
+		return 1024LL * i;
+	if (*sp == 'm' && sp[1] == '\0')
+		return 1024LL * 1024LL * i;

lib/tst_mkfs.c line 49

Quoted 10 lines
lib/tst_mkfs.c +10 -0
+	if (*sp == 'g' && sp[1] == '\0')
+		return 1024LL * 1024LL * 1024LL * i;
+	if (*sp == 't' && sp[1] == '\0')
+		return 1024LL * 1024LL * 1024LL * 1024LL * i;
+	if (*sp == 'p' && sp[1] == '\0')
+		return 1024LL * 1024LL * 1024LL * 1024LL * 1024LL * i;
+	if (*sp == 'e' && sp[1] == '\0')
+		return 1024LL * 1024LL * 1024LL * 1024LL * 1024LL * 1024LL * i;
+	return -1LL;
+}

We allready have bytes_by_prefix() in LTP library.

lib/tst_mkfs.c line 53

Quoted 3 lines
lib/tst_mkfs.c +3 -0
+void tst_mkfs_sized(void (cleanup_fn)(void), const char *dev,
+                    const char *fs_type, const char *const fs_opts[],
+                    const char *fssize, const char *blocksize)

What about passing the size parameter as we do in tst_fs_has_free()?

I.e. with two paramters where one is size and the second is units?

Also why have you added the blocksize parameter as well? The block size is needed only for the mmap16 test while limiting filesystem size seems like good addition to the library?

Quoted 94 lines

{

lib/tst_mkfs.c +61 -7
 	int i, pos = 3;
 	const char *argv[OPTS_MAX] = {"mkfs", "-t", fs_type};
 	char fs_opts_str[1024] = "";
+	char blk_size_str[256] = "";
+	char fs_size_str[256] = "";
+	long long fssz = 0;
+	unsigned int blsz= 0;
 
 	if (!fs_type)
 		tst_brkm(TBROK, cleanup_fn, "No fs_type specified");
 
-	/*
-	 * mkfs.xfs and mkfs.btrfs aborts if it finds a filesystem
-	 * superblock on the device, which is the case here as we
-	 * reuse one device for all tests.
-	 */
 	if (!strcmp(fs_type, "xfs")) {
+		/*
+		 * mkfs.xfs and mkfs.btrfs aborts if it finds a filesystem
+		 * superblock on the device, which is the case here as we
+		 * reuse one device for all tests.
+		 */
 		tst_resm(TINFO, "Appending '-f' flag to mkfs.%s", fs_type);
 		argv[pos++] = "-f";
-	}
+		if (blocksize) {
+			argv[pos++] = "-b";
+			strcat(blk_size_str, "size=");
+			strcat(blk_size_str, blocksize);
+			argv[pos++] = blk_size_str;
+		}
 
-	if (!strcmp(fs_type, "btrfs")) {
+		if (!fssize) {
+			argv[pos++] = "-d";
+			strcat(fs_size_str, "size=");
+			strcat(fs_size_str, fssize);
+			argv[pos++] = fs_size_str;
+		}
+	} else if (!strncmp(fs_type, "ext", 3)) {
+		if (blocksize) {
+			argv[pos++] = "-b";
+			argv[pos++] = blocksize;
+		}
+		/*
+		 * If fs is extX, the fs_size should be set behind device name.
+		 * Not set at here.
+		 */
+	} else if (!strcmp(fs_type, "btrfs")) {
 		/*
 		 * The -f option was added to btrfs-progs v3.12
 		 */
@@ -50,6 +102,21 @@ void tst_mkfs(void (cleanup_fn)(void), const char *dev,
 				fs_type);
 			argv[pos++] = "-f";
 		}
+		if (fssize) {
+			/*
+			 * The recommended size for the mixed mode is for filesystems less than 1GiB
+			 */
+			if (cvtnum(fssize) < 1024 * 1024 * 1024)
+				argv[pos++] = "--mixed";
+			argv[pos++] = "-b";
+			argv[pos++] = fssize;
+		}
+	} else if (fssize || blocksize) {
+		/*
+		 * Can't set fs size or block size for others fs,
+		 * except add new fs support as above.
+		 */
+		tst_brkm(TBROK, cleanup_fn, "tst_mkfs_sized don't support '%s' fs, please add this fs as a new feature", fs_type);
 	}
 
 	if (fs_opts) {
@@ -68,10 +135,30 @@ void tst_mkfs(void (cleanup_fn)(void), const char *dev,
 	}
 
 	argv[pos++] = dev;
+
+	/*
+	 * According to mke2fs(8) manual, fs-size need be behind the device
+	 * parameter. So add fs size into argv[pos++] after dev name.
+	 */
+	if (!strncmp(fs_type, "ext", 3) && fssize && blocksize) {
+		fssz = cvtnum(fssize);
+		blsz = cvtnum(blocksize);
+		if (fssz > 0 && blsz > 0){
+			sprintf(fs_size_str, "%llu", fssz / blsz);
+		} else {
+			tst_brkm(TBROK, cleanup_fn, "No suitable filesystem/block size specified");
+		}
+		argv[pos++] = fs_size_str;
+	}
Signature · 2 lines
Cyril Hrubis
[email protected]
#5 lore raw

PATCH 1/2 lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

----- 原始邮件 -----

Quoted 33 lines

发件人: "Cyril Hrubis" <[email protected]>
收件人: "Zorro Lang" <[email protected]>
抄送: [email protected]
发送时间: 星期三, 2016年 3 月 09日 下午 9:07:10
主题: Re: [PATCH 1/2] lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

Hi!

Quoted 26 lines

#include "test.h"
#include "ltp_priv.h"

#define OPTS_MAX 32

lib/tst_mkfs.c +19 -2
-void tst_mkfs(void (cleanup_fn)(void), const char *dev,
-	      const char *fs_type, const char *const fs_opts[])
+long long cvtnum(const char *s)
+{
+	long long i;
+	char *sp = NULL;
+
+	i = strtoll(s, &sp, 0);
+	if (i == 0 && sp == s)
+		return -1LL;
+	if (*sp == '\0')
+		return i;
+
+	if (*sp == 'k' && sp[1] == '\0')
+		return 1024LL * i;
+	if (*sp == 'm' && sp[1] == '\0')
+		return 1024LL * 1024LL * i;
+	if (*sp == 'g' && sp[1] == '\0')
+		return 1024LL * 1024LL * 1024LL * i;
+	if (*sp == 't' && sp[1] == '\0')
+		return 1024LL * 1024LL * 1024LL * 1024LL * i;
Quoted 8 lines

lib/tst_mkfs.c line 49

Quoted 6 lines
lib/tst_mkfs.c +6 -0
+	if (*sp == 'p' && sp[1] == '\0')
+		return 1024LL * 1024LL * 1024LL * 1024LL * 1024LL * i;
+	if (*sp == 'e' && sp[1] == '\0')
+		return 1024LL * 1024LL * 1024LL * 1024LL * 1024LL * 1024LL * i;
+	return -1LL;
+}

We allready have bytes_by_prefix() in LTP library.

Oh, I just noticed that function. The difference is bytes_by_prefix() run an int value, so it can't bigger than 2^31. But some FS, likes XFS can support very huge size, even in RHEL-7, XFS support 500T. it's bigger than 2^31.

But generally we won't run LTP on a 500T device:) So if you don't want to change that, we should say mkfs size can't bigger than 2^31 bytes, or the result will be unexpected.

Quoted 8 lines

lib/tst_mkfs.c line 53

Quoted 4 lines
lib/tst_mkfs.c +3 -0
+void tst_mkfs_sized(void (cleanup_fn)(void), const char *dev,
+                    const char *fs_type, const char *const fs_opts[],
+                    const char *fssize, const char *blocksize)

What about passing the size parameter as we do in tst_fs_has_free()?

I.e. with two paramters where one is size and the second is units?

Oh, I don't know why we need two parameters to describe one size. I use "char *fssize" due to tst_mkfs() use string paramters directly, it don't need an int value(but int value is OK too).

Quoted 4 lines

Also why have you added the blocksize parameter as well? The block size is needed only for the mmap16 test while limiting filesystem size seems like good addition to the library?

Block size is needed because some fs need to know the block count, if you want to create a sized fs. For example if we want to make 512m ext4 and xfs:

xfs can directly use mkfs -t xfs -d size=512m device

but mkfs.ext4 need to do like: mkfs -t ext4 -b $block_size device 512*1024*1024/$block_size

So we need to know block size for some fs, as I know udf fs need to know block count too.

Quoted 104 lines
Quoted 89 lines

{

lib/tst_mkfs.c +34 -7
 	int i, pos = 3;
 	const char *argv[OPTS_MAX] = {"mkfs", "-t", fs_type};
 	char fs_opts_str[1024] = "";
+	char blk_size_str[256] = "";
+	char fs_size_str[256] = "";
+	long long fssz = 0;
+	unsigned int blsz= 0;
 
 	if (!fs_type)
 		tst_brkm(TBROK, cleanup_fn, "No fs_type specified");
 
-	/*
-	 * mkfs.xfs and mkfs.btrfs aborts if it finds a filesystem
-	 * superblock on the device, which is the case here as we
-	 * reuse one device for all tests.
-	 */
 	if (!strcmp(fs_type, "xfs")) {
+		/*
+		 * mkfs.xfs and mkfs.btrfs aborts if it finds a filesystem
+		 * superblock on the device, which is the case here as we
+		 * reuse one device for all tests.
+		 */
 		tst_resm(TINFO, "Appending '-f' flag to mkfs.%s", fs_type);
 		argv[pos++] = "-f";
-	}
+		if (blocksize) {
+			argv[pos++] = "-b";
+			strcat(blk_size_str, "size=");
+			strcat(blk_size_str, blocksize);
+			argv[pos++] = blk_size_str;
+		}
 
-	if (!strcmp(fs_type, "btrfs")) {
+		if (!fssize) {
+			argv[pos++] = "-d";
+			strcat(fs_size_str, "size=");
+			strcat(fs_size_str, fssize);
+			argv[pos++] = fs_size_str;
+		}
+	} else if (!strncmp(fs_type, "ext", 3)) {
+		if (blocksize) {
+			argv[pos++] = "-b";
+			argv[pos++] = blocksize;
+		}
+		/*
+		 * If fs is extX, the fs_size should be set behind device name.
+		 * Not set at here.
+		 */
+	} else if (!strcmp(fs_type, "btrfs")) {
 		/*
 		 * The -f option was added to btrfs-progs v3.12
 		 */
@@ -50,6 +102,21 @@ void tst_mkfs(void (cleanup_fn)(void), const char *dev,
 				fs_type);
 			argv[pos++] = "-f";
 		}
+		if (fssize) {
+			/*
+			 * The recommended size for the mixed mode is for filesystems less than

1GiB

lib/tst_mkfs.c +12 -0
+			 */
+			if (cvtnum(fssize) < 1024 * 1024 * 1024)
+				argv[pos++] = "--mixed";
+			argv[pos++] = "-b";
+			argv[pos++] = fssize;
+		}
+	} else if (fssize || blocksize) {
+		/*
+		 * Can't set fs size or block size for others fs,
+		 * except add new fs support as above.
+		 */
+		tst_brkm(TBROK, cleanup_fn, "tst_mkfs_sized don't support '%s' fs,

please add this fs as a new feature", fs_type);

lib/tst_mkfs.c +6 -0
 	}
 
 	if (fs_opts) {
@@ -68,10 +135,30 @@ void tst_mkfs(void (cleanup_fn)(void), const char
 *dev,
 	}
 
 	argv[pos++] = dev;
+
+	/*
+	 * According to mke2fs(8) manual, fs-size need be behind the device
+	 * parameter. So add fs size into argv[pos++] after dev name.
+	 */
+	if (!strncmp(fs_type, "ext", 3) && fssize && blocksize) {

lib/tst_mkfs.c void tst_mkfs(void (cleanup_fn)(void), const char · line 154

Quoted 10 lines
lib/tst_mkfs.c +9 -0
+		fssz = cvtnum(fssize);
+		blsz = cvtnum(blocksize);
+		if (fssz > 0 && blsz > 0){
+			sprintf(fs_size_str, "%llu", fssz / blsz);
+		} else {
+			tst_brkm(TBROK, cleanup_fn, "No suitable filesystem/block size
 specified");
+		}
+		argv[pos++] = fs_size_str;
+	}

--
Cyril Hrubis
[email protected]

#6 lore raw

PATCH 1/2 lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

Hi!

Quoted 4 lines
Quoted 3 lines

Also why have you added the blocksize parameter as well? The block size is needed only for the mmap16 test while limiting filesystem size seems like good addition to the library?

Quoted 8 lines

Block size is needed because some fs need to know the block count, if you want to create a sized fs. For example if we want to make 512m ext4 and xfs:

xfs can directly use mkfs -t xfs -d size=512m device

but mkfs.ext4 need to do like: mkfs -t ext4 -b $block_size device 512*1024*1024/$block_size

So we need to know block size for some fs, as I know udf fs need to know block count too.

Looking at the a few different mkfs programs there is no unified way to pass the size parameter. And looks like only mke2fs and mkfs.fat supports one paramter after the device.

So after all the easiest solution may be just to add one more const char* fs-size parameter to tst_mkfs that could be used to pass block count to these two.

Signature · 2 lines
Cyril Hrubis
[email protected]
#7 lore raw

PATCH 1/2 lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

----- 原始邮件 -----

Quoted 23 lines

发件人: "Cyril Hrubis" <[email protected]>
收件人: "Zirong Lang" <[email protected]>
抄送: [email protected]
发送时间: 星期三, 2016年 3 月 09日 下午 11:52:58
主题: Re: [PATCH 1/2] lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

Hi!

Quoted 8 lines
Quoted 3 lines

Also why have you added the blocksize parameter as well? The block size is needed only for the mmap16 test while limiting filesystem size seems like good addition to the library?

Block size is needed because some fs need to know the block count, if you want to create a sized fs. For example if we want to make 512m ext4 and xfs:

Quoted 7 lines

xfs can directly use mkfs -t xfs -d size=512m device

but mkfs.ext4 need to do like: mkfs -t ext4 -b $block_size device 512*1024*1024/$block_size

So we need to know block size for some fs, as I know udf fs need to know block count too.

Quoted 7 lines

Looking at the a few different mkfs programs there is no unified way to pass the size parameter. And looks like only mke2fs and mkfs.fat supports one paramter after the device.

So after all the easiest solution may be just to add one more const char* fs-size parameter to tst_mkfs that could be used to pass block count to these two.

Yes, this likes what I just said to you. add another parameter char *fs_opts[], which store all parameters for mkfs need to behind device name.

Or maybe my "|" method is the simplest, even it's ugly:)

Thanks,
Zorro

Quoted 5 lines

--
Cyril Hrubis
[email protected]

#8 lore raw

PATCH 1/2 lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

Hi!

Quoted 2 lines

Yes, this likes what I just said to you. add another parameter char *fs_opts[], which store all parameters for mkfs need to behind device name.

Given that all mkfs programs that I have installed have either one (fs-size) or zero paremeters it does not have to be array at all.

We can just do:

void safe_mkfs(const int lineno, const char *fname, const char *dev,
const char *fs_type, const char *fs_size, const char *const fs_opts[]);

And append fs_size after device if it's non-NULL.

Signature · 2 lines
Cyril Hrubis
[email protected]
#9 lore raw

PATCH 1/2 lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

----- 原始邮件 -----

Quoted 14 lines

发件人: "Cyril Hrubis" <[email protected]>
收件人: "Zirong Lang" <[email protected]>
抄送: [email protected]
发送时间: 星期四, 2016年 3 月 10日 上午 12:09:25
主题: Re: [PATCH 1/2] lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

Hi!

Quoted 3 lines

Yes, this likes what I just said to you. add another parameter char *fs_opts[], which
store all parameters for mkfs need to behind device name.

Given that all mkfs programs that I have installed have either one (fs-size) or zero paremeters it does not have to be array at all.

Quoted 7 lines

We can just do:

void safe_mkfs(const int lineno, const char *fname, const char *dev,
const char *fs_type, const char *fs_size, const char *const fs_opts[]);

And append fs_size after device if it's non-NULL.

Hi,

Sure, it's OK for me. And I think if we call it *fs_size, maybe make the user feel confused. The truth is it's the count of blocks, and only used for some fs. So maybe we can call it *extra_opts, means used after device name?

So I will do this patch:

1. change tst_mkfs to
void safe_mkfs(const int lineno, const char *fname, const char *dev,
const char *fs_type, const char *const fs_opts[], const char *extra_opts)

2. add tst_mkfs into test.h:
static inline void tst_mkfs(const int lineno, const char *fname, const char *dev,

                             const char *fs_type, const char *const fs_opts[])
{
    safe_mkfs(lineno, fname, dev, fs_type, fs_opts, NULL);
}

Does this you want?

Thanks,
Zorro

Quoted 5 lines

--
Cyril Hrubis
[email protected]

#10 lore raw

PATCH 1/2 lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

Hi!

Quoted 3 lines

Sure, it's OK for me. And I think if we call it *fs_size, maybe make the user feel confused. The truth is it's the count of blocks, and only used for some fs. So maybe we can call it *extra_opts, means used after device name?

That is true, well the 'man mkfs' is confusing as well since it looks like the size argument is supported generally by mkfs binaries...

And we should call it extra_opt, since it's just one option.

Quoted 7 lines

So I will do this patch:

1. change tst_mkfs to
void safe_mkfs(const int lineno, const char *fname, const char *dev,
const char *fs_type, const char *const fs_opts[], const char *extra_opts)

Quoted 8 lines

2. add tst_mkfs into test.h:
static inline void tst_mkfs(const int lineno, const char *fname, const char *dev,

                             const char *fs_type, const char *const fs_opts[])
{
    safe_mkfs(lineno, fname, dev, fs_type, fs_opts, NULL);
}

Does this you want?

Having two nearly identical *_mkfs() functions would be confusing as well. I would have just changed the function prototype and fixed all the testcases that use it (36 testcases if I'm counting right).

Signature · 2 lines
Cyril Hrubis
[email protected]
#11 lore raw

PATCH 1/2 lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

----- 原始邮件 -----

Quoted 34 lines

发件人: "Cyril Hrubis" <[email protected]>
收件人: "Zirong Lang" <[email protected]>
抄送: [email protected]
发送时间: 星期四, 2016年 3 月 10日 上午 1:43:37
主题: Re: [PATCH 1/2] lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

Hi!

Quoted 5 lines

Sure, it's OK for me. And I think if we call it *fs_size, maybe make the user
feel confused. The truth is it's the count of blocks, and only used for some
fs. So maybe we can call it *extra_opts, means used after device name?

That is true, well the 'man mkfs' is confusing as well since it looks like the size argument is supported generally by mkfs binaries...

And we should call it extra_opt, since it's just one option.

Quoted 16 lines

So I will do this patch:

1. change tst_mkfs to
void safe_mkfs(const int lineno, const char *fname, const char *dev,
const char *fs_type, const char *const fs_opts[], const char *extra_opts)

2. add tst_mkfs into test.h:
static inline void tst_mkfs(const int lineno, const char *fname, const char *dev,
const char *fs_type, const char *const fs_opts[])

{
    safe_mkfs(lineno, fname, dev, fs_type, fs_opts, NULL);
}
Quoted 5 lines
Quoted 1 line

Does this you want?

Having two nearly identical *_mkfs() functions would be confusing as well. I would have just changed the function prototype and fixed all the testcases that use it (36 testcases if I'm counting right).

OK...

But actually I want to check all testcases which use tst_mkfs(), and give them a fs_size limit if big device will effect them in my 2nd patch, after the new "fs size" parameter be accepted. I think it looks clearly.

Anyway, if you think it's OK, I can do them in one patch:)

Thanks,
Zorro

Quoted 5 lines

--
Cyril Hrubis
[email protected]

#12 lore raw

PATCH 1/2 lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

Hi!

Quoted 6 lines

OK...

But actually I want to check all testcases which use tst_mkfs(), and give them a fs_size limit if big device will effect them in my 2nd patch, after the new "fs size" parameter be accepted. I think it looks clearly.

This begs a question what exactly is your motivation? Why do you pass a big device in LTP_DEV in the first place? The testcases that use the device just need it to be big enough to be formatted with a filesystem which is about 100MB at the moment.

Signature · 2 lines
Cyril Hrubis
[email protected]
#13 lore raw

PATCH 1/2 lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

Hi,

----- 原始邮件 -----

Quoted 14 lines

发件人: "Cyril Hrubis" <[email protected]>
收件人: "Zirong Lang" <[email protected]>
抄送: [email protected]
发送时间: 星期四, 2016年 3 月 10日 下午 6:04:25
主题: Re: [PATCH 1/2] lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

Hi!

Quoted 6 lines

OK...

But actually I want to check all testcases which use tst_mkfs(), and give them a fs_size limit if big device will effect them in my 2nd patch, after the new "fs size" parameter be accepted. I think it looks clearly.

Quoted 4 lines

This begs a question what exactly is your motivation? Why do you pass a big device in LTP_DEV in the first place? The testcases that use the device just need it to be big enough to be formatted with a filesystem which is about 100MB at the moment.

I don't mean give size limit to all cases use tst_mkfs. I mean the limited size used for cases which will effected by test device size. Actually, I just pass size limit to mmap16 until now:) I don't know which case need it or will need it too.

I found that mmap16 ETIMEDOUT problem by test on a 1G /dev/loop0. I don't think it's too big as a general test device.

I think make an appointed size fs can be used for 3 parts:
1) If someone case need a specially appointed fs size, it can do it. e.g. quota test.
2) Generally most QE will prepare some partitions or some special device(FC, SAS...) in his test machine, and use them for many different test suit(LTP, xfstests ...). These partitions or devices maybe 15G, 80G, 1T, 2T... they always pass them as test devices directly. I think we shouldn't say "NO, you can't pass a device more than 1G to LTP, that will cause failures.", I think LTP should deal with big test device problem, so mkfs_sized is needed.
3) If someone case try to full the whole test fs, and it don't need big size fs, it can use mkfs_sized to prevent this case run too long time.

I think the reason that LTP don't care make a sized fs is LTP doesn't do many fs test, if LTP have more and more fs test cases, it will feel mkfs_sized is important.

Thanks,
Zorro

Quoted 5 lines

--
Cyril Hrubis
[email protected]

#14 lore raw

PATCH 1/2 lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

Hi!

Quoted 6 lines

I don't mean give size limit to all cases use tst_mkfs. I mean the limited size used for cases which will effected by test device size. Actually, I just pass size limit to mmap16 until now:) I don't know which case need it or will need it too.

I found that mmap16 ETIMEDOUT problem by test on a 1G /dev/loop0. I don't think it's too big as a general test device.

Well that could be fixed easily by the extra blocksize parameter.

And I guess that the testcase can also run faster if we limit the fs size to minimal ext4 size. So this looks like a good idea.

Are there any other testcases that fail with a device of size 1GB or less?

Quoted 2 lines

I think make an appointed size fs can be used for 3 parts:
1) If someone case need a specially appointed fs size, it can do it. e.g. quota test.

We do not have this need at the moment.

Are you about the write such testcase?

Quoted 6 lines

2) Generally most QE will prepare some partitions or some special device(FC, SAS...) in his test machine, and use them for many different test suit(LTP, xfstests ...). These partitions or devices maybe 15G, 80G, 1T, 2T... they always pass them as test devices directly. I think we shouldn't say "NO, you can't pass a device more than 1G to LTP, that will cause failures.", I think LTP should deal with big test device problem, so mkfs_sized is needed.

I do not agree. Since if you do not pass any device to LTP, it will simply create small enough loopback device. So in this case the solution is simply not passing any device to LTP at all.

It's not that LTP does any device specific testing or I/O stress on the LTP_DEV so it does not matter that we use loopback device or some real peartition. These testcases just need a device because the underlying syscall needs to work with one or we need to simluate read only mounted fs, etc. So there is absolutly _no_ reason to pass terabyte sized device to these testcases.

Quoted 2 lines

3) If someone case try to full the whole test fs, and it don't need big size fs, it can use mkfs_sized to prevent this case run too long time.

Again passing too big device to the test ends means that the test will take a long time to run. There is nothing unexpected on this.

Quoted 2 lines

I think the reason that LTP don't care make a sized fs is LTP doesn't do many fs test, if LTP have more and more fs test cases, it will feel mkfs_sized is important.

I'm against adding functions that _may_ be needed in the future. If you are writing a testcase that needs some functionality just clearly describe what exactly do you need.

Signature · 2 lines
Cyril Hrubis
[email protected]
#15 lore raw

PATCH 1/2 lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

----- 原始邮件 -----

Quoted 18 lines

发件人: "Cyril Hrubis" <[email protected]>
收件人: "Zirong Lang" <[email protected]>
抄送: [email protected]
发送时间: 星期四, 2016年 3 月 10日 下午 8:19:14
主题: Re: [PATCH 1/2] lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

Hi!

Quoted 10 lines

I don't mean give size limit to all cases use tst_mkfs. I mean the limited size
used for cases which will effected by test device size. Actually, I just pass
size limit to mmap16 until now:) I don't know which case need it or will need it too.

I found that mmap16 ETIMEDOUT problem by test on a 1G /dev/loop0. I don't think
it's too big as a general test device.

Quoted 7 lines

Well that could be fixed easily by the extra blocksize parameter.

And I guess that the testcase can also run faster if we limit the fs size to minimal ext4 size. So this looks like a good idea.

Are there any other testcases that fail with a device of size 1GB or less?

I just run some LTP cases about FS, not all LTP cases. Until now I haven't found any others similar fail.

Quoted 8 lines
Quoted 4 lines

I think make an appointed size fs can be used for 3 parts:
1) If someone case need a specially appointed fs size, it can do it. e.g. quota test.

We do not have this need at the moment.

Are you about the write such testcase?

Sorry, no this plan until now.

Quoted 13 lines
Quoted 12 lines

2) Generally most QE will prepare some partitions or some special device(FC, SAS...)
in his test machine, and use them for many different test suit(LTP, xfstests ...).
These partitions or devices maybe 15G, 80G, 1T, 2T... they always pass them as test
devices directly. I think we shouldn't say "NO, you can't pass a device more than
1G to LTP, that will cause failures.", I think LTP should deal with big test device
problem, so mkfs_sized is needed.

Quoted 3 lines

I do not agree. Since if you do not pass any device to LTP, it will simply create small enough loopback device. So in this case the solution is simply not passing any device to LTP at all.

Oh, sorry I don't know LTP will create small loop device by itself. So do you still want to fix this ETIMEDOUT problem? Or we just say "please don't give outside device to LTP?" or "please give a small enough device to LTP -b device"

BTW, if LTP will create small loop device by itself, I can't make mmap16 to create a 100M ext4. I think it maybe too big for LTP small loop device. I will test and change my recent patch

Thanks,
Zorro

#16 lore raw

PATCH 1/2 lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

Hi!

Quoted 12 lines
Quoted 12 lines
Quoted 11 lines

2) Generally most QE will prepare some partitions or some special device(FC, SAS...)
in his test machine, and use them for many different test suit(LTP, xfstests ...).
These partitions or devices maybe 15G, 80G, 1T, 2T... they always pass them as test
devices directly. I think we shouldn't say "NO, you can't pass a device more than
1G to LTP, that will cause failures.", I think LTP should deal with big test device
problem, so mkfs_sized is needed.

Quoted 7 lines
Quoted 3 lines

I do not agree. Since if you do not pass any device to LTP, it will simply create small enough loopback device. So in this case the solution is simply not passing any device to LTP at all.

Oh, sorry I don't know LTP will create small loop device by itself. So do you still want to fix this ETIMEDOUT problem? Or we just say "please don't give outside device to LTP?" or "please give a small enough device to LTP -b device"

I still do, since it's reasonably easy to fix and 1GB block device is not that big to begin with.

Quoted 3 lines

BTW, if LTP will create small loop device by itself, I can't make mmap16 to create a 100M ext4. I think it maybe too big for LTP small loop device. I will test and change my recent patch

Ah looks like there is a bug. Since the default LTP device size was changed to 100MB in lib/tst_device.c and in testcases/lib/test.sh but not in the runltp (which is because new enough btrfs needs at least 100MB for filesystem).

But the mmap16 test should work with as small device as possible, which would be 20MB.

Signature · 2 lines
Cyril Hrubis
[email protected]
#17 lore raw

PATCH 1/2 lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

----- 原始邮件 -----

Quoted 39 lines

发件人: "Cyril Hrubis" <[email protected]>
收件人: "Zirong Lang" <[email protected]>
抄送: [email protected]
发送时间: 星期四, 2016年 3 月 10日 下午 10:06:53
主题: Re: [PATCH 1/2] lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

Hi!

Quoted 17 lines
Quoted 13 lines
Quoted 4 lines

2) Generally most QE will prepare some partitions or some special device(FC, SAS...)
in his test machine, and use them for many different test suit(LTP, xfstests ...).

Quoted 8 lines

These partitions or devices maybe 15G, 80G, 1T, 2T... they always pass them
as test
devices directly. I think we shouldn't say "NO, you can't pass a device more than
1G to LTP, that will cause failures.", I think LTP should deal with big test device
problem, so mkfs_sized is needed.

Quoted 3 lines

I do not agree. Since if you do not pass any device to LTP, it will simply create small enough loopback device. So in this case the solution is simply not passing any device to LTP at all.

Quoted 5 lines

Oh, sorry I don't know LTP will create small loop device by itself. So do you
still want to fix this ETIMEDOUT problem? Or we just say "please don't give outside device to LTP?" or "please give a small enough device to LTP -b device"

I still do, since it's reasonably easy to fix and 1GB block device is not that big to begin with.

Quoted 5 lines

BTW, if LTP will create small loop device by itself, I can't make mmap16 to create
a 100M ext4. I think it maybe too big for LTP small loop device. I will test and
change my recent patch

Quoted 4 lines

Ah looks like there is a bug. Since the default LTP device size was changed to 100MB in lib/tst_device.c and in testcases/lib/test.sh but not in the runltp (which is because new enough btrfs needs at least 100MB for filesystem).

Oh, if you want to mkfs.btrfs on a device less than 100MB, you can use --mixed option. I know it can support small device less than 100MB, but I haven't try the smallest size.

Quoted 8 lines

But the mmap16 test should work with as small device as possible, which would be 20MB.

--
Cyril Hrubis
[email protected]

#18 lore raw

PATCH 1/2 lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

Hi!

Quoted 3 lines

Oh, if you want to mkfs.btrfs on a device less than 100MB, you can use --mixed option. I know it can support small device less than 100MB, but I haven't try the smallest size.

I would rather use the default options since that is what most of the users will use anyway and since 100Mb is not that much either.

Signature · 2 lines
Cyril Hrubis
[email protected]
#19 lore raw

PATCH 1/2 lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

In reply to Zirong Lang

----- 原始邮件 -----

Quoted 83 lines

发件人: "Zirong Lang" <[email protected]>
收件人: "Cyril Hrubis" <[email protected]>
抄送: [email protected]
发送时间: 星期三, 2016年 3 月 09日 下午 11:31:32
主题: Re: [LTP] [PATCH 1/2] lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

----- 原始邮件 -----

Quoted 34 lines

发件人: "Cyril Hrubis" <[email protected]>
收件人: "Zorro Lang" <[email protected]>
抄送: [email protected]
发送时间: 星期三, 2016年 3 月 09日 下午 9:07:10
主题: Re: [PATCH 1/2] lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

Hi!

Quoted 26 lines

#include "test.h"
#include "ltp_priv.h"

#define OPTS_MAX 32

lib/tst_mkfs.c +19 -2
-void tst_mkfs(void (cleanup_fn)(void), const char *dev,
-	      const char *fs_type, const char *const fs_opts[])
+long long cvtnum(const char *s)
+{
+	long long i;
+	char *sp = NULL;
+
+	i = strtoll(s, &sp, 0);
+	if (i == 0 && sp == s)
+		return -1LL;
+	if (*sp == '\0')
+		return i;
+
+	if (*sp == 'k' && sp[1] == '\0')
+		return 1024LL * i;
+	if (*sp == 'm' && sp[1] == '\0')
+		return 1024LL * 1024LL * i;
+	if (*sp == 'g' && sp[1] == '\0')
+		return 1024LL * 1024LL * 1024LL * i;
+	if (*sp == 't' && sp[1] == '\0')
+		return 1024LL * 1024LL * 1024LL * 1024LL * i;
Quoted 8 lines

lib/tst_mkfs.c line 49

Quoted 6 lines
lib/tst_mkfs.c +6 -0
+	if (*sp == 'p' && sp[1] == '\0')
+		return 1024LL * 1024LL * 1024LL * 1024LL * 1024LL * i;
+	if (*sp == 'e' && sp[1] == '\0')
+		return 1024LL * 1024LL * 1024LL * 1024LL * 1024LL * 1024LL * i;
+	return -1LL;
+}

We allready have bytes_by_prefix() in LTP library.

Oh, I just noticed that function. The difference is bytes_by_prefix() run an int value, so it can't bigger than 2^31. But some FS, likes XFS can support very huge size, even in RHEL-7, XFS support 500T. it's bigger than 2^31.

But generally we won't run LTP on a 500T device:) So if you don't want to change
that, we should say mkfs size can't bigger than 2^31 bytes, or the result will
be unexpected.

Quoted 8 lines

lib/tst_mkfs.c line 53

Quoted 4 lines
lib/tst_mkfs.c +3 -0
+void tst_mkfs_sized(void (cleanup_fn)(void), const char *dev,
+                    const char *fs_type, const char *const fs_opts[],
+                    const char *fssize, const char *blocksize)

What about passing the size parameter as we do in tst_fs_has_free()?

I.e. with two paramters where one is size and the second is units?

Oh, I don't know why we need two parameters to describe one size. I use "char *fssize" due to tst_mkfs() use string paramters directly, it don't need an int value(but int value is OK too).

Quoted 4 lines

Also why have you added the blocksize parameter as well? The block size is needed only for the mmap16 test while limiting filesystem size seems like good addition to the library?

Block size is needed because some fs need to know the block count, if you want to create a sized fs. For example if we want to make 512m ext4 and xfs:

Quoted 8 lines

xfs can directly use mkfs -t xfs -d size=512m device

but mkfs.ext4 need to do like: mkfs -t ext4 -b $block_size device 512*1024*1024/$block_size

So we need to know block size for some fs, as I know udf fs need to know block count too.

And even if not for extX fs, I don't need tst_mkfs_sized() function. If for XFS, the fs_opts parameter of tst_mkfs() already can do that by:
const char *fs_opts[5] = {"-b", "size=1024", "-d", "size=100m" NULL};

But make a sized extX fs, need to know how many block counts (fs_size/block_size), and this block counts need to behind the device name, e.g.
mkfs -t ext4 -b 1024 device 100*1024*1024/1024

I can't just write likes above:
const char *fs_opts[4] = {"-b", "1024", "102400" NULL};

So at first demo patch, I add a "|" to separate the parameters which need to before device name or behind device name.

Or I said another method, add a new parameter for tst_mkfs(), named "fs_opts2", which store all paramters for mkfs and need to behind the device name.

Thanks,
Zorro

#20 lore raw

PATCH 1/2 lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

In reply to Zorro Lang

Hi!

include/test.h int tst_system(const char *command); · line 328

Quoted 6 lines
include/test.h +6 -0
+void tst_mkfs_sized(void (cleanup_fn)(void), const char *dev,
+                    const char *fs_type, const char *const fs_opts[],
+                    const char *fssize, const char *blocksize);
+
+#define tst_mkfs(cleanup_fn, dev, fs_type, fs_opts) \
+	tst_mkfs_sized(cleanup_fn, dev, fs_type, fs_opts, NULL, NULL)

Also make this static inline function instead. We only use such macros for SAFE_MACROS() where we have to pass correct __FILE__ and __LINE__, which is very special occasion.

Signature · 2 lines
Cyril Hrubis
[email protected]
#21 lore raw

PATCH 1/2 lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

----- 原始邮件 -----

Quoted 7 lines

发件人: "Cyril Hrubis" <[email protected]>
收件人: "Zorro Lang" <[email protected]>
抄送: [email protected]
发送时间: 星期三, 2016年 3 月 09日 下午 9:09:21
主题: Re: [PATCH 1/2] lib/tst_mkfs: new tst_mkfs_sized function for create appointed size fs

Hi!

Quoted 10 lines

include/test.h int tst_system(const char *command); · line 328

Quoted 6 lines
include/test.h +6 -0
+void tst_mkfs_sized(void (cleanup_fn)(void), const char *dev,
+                    const char *fs_type, const char *const fs_opts[],
+                    const char *fssize, const char *blocksize);
+
+#define tst_mkfs(cleanup_fn, dev, fs_type, fs_opts) \
+	tst_mkfs_sized(cleanup_fn, dev, fs_type, fs_opts, NULL, NULL)

Also make this static inline function instead. We only use such macros for SAFE_MACROS() where we have to pass correct __FILE__ and __LINE__, which is very special occasion.

OK, I will change it to inline function.

Quoted 5 lines

--
Cyril Hrubis
[email protected]