Skip to contentSkip to conversation

tst_device: do sync() before reading test block device stat file

PATCH New Li Wang 3 participants 13 messages Jan 2, 2020 – Jan 7, 2020

lore
Share & tools
lore

Li Wang

author
#0 lore raw

PATCH 1/1 tst_device: do sync() before reading test block device stat file

To avoid FS deferred IO metadata/cache interferes test result, so we do sync simply before the tst_dev_bytes_written invocation.

Cc Sumit Garg <[email protected]>
Cc Yang Xu <[email protected]>
---
 testcases/kernel/syscalls/fdatasync/fdatasync03.c | 2 ++
 testcases/kernel/syscalls/fsync/fsync04.c         | 2 ++
 testcases/kernel/syscalls/sync/sync03.c           | 2 ++
 testcases/kernel/syscalls/syncfs/syncfs01.c       | 2 ++
 4 files changed, 8 insertions(+)
testcases/kernel/syscalls/fdatasync/fdatasync03.c +2 -0
index ee50e75c9..032ac4b58 100644
@@ -32,6 +32,8 @@ static void verify_fdatasync(void)
 
 	fd = SAFE_OPEN(FNAME, O_RDWR|O_CREAT, MODE);
 
+	sync();
+
 	tst_dev_bytes_written(tst_device->dev);
 
 	tst_fill_fd(fd, 0, TST_MB, FILE_SIZE_MB);
testcases/kernel/syscalls/fsync/fsync04.c +2 -0
index c67fc5692..3c1f45e94 100644
@@ -32,6 +32,8 @@ static void verify_fsync(void)
 
 	fd = SAFE_OPEN(FNAME, O_RDWR|O_CREAT, MODE);
 
+	sync();
+
 	tst_dev_bytes_written(tst_device->dev);
 
 	tst_fill_fd(fd, 0, TST_MB, FILE_SIZE_MB);
testcases/kernel/syscalls/sync/sync03.c +2 -0
index a6f72d2ed..085ccfdeb 100644
@@ -32,6 +32,8 @@ static void verify_sync(void)
 
 	fd = SAFE_OPEN(FNAME, O_RDWR|O_CREAT, MODE);
 
+	sync();
+
 	tst_dev_bytes_written(tst_device->dev);
 
 	tst_fill_fd(fd, 0, TST_MB, FILE_SIZE_MB);
testcases/kernel/syscalls/syncfs/syncfs01.c +2 -0
index 051a19ea6..3cf404450 100644
@@ -33,6 +33,8 @@ static void verify_syncfs(void)
 
 	fd = SAFE_OPEN(FNAME, O_RDWR|O_CREAT, MODE);
 
+	sync();
+
 	tst_dev_bytes_written(tst_device->dev);
 
 	tst_fill_fd(fd, 0, TST_MB, FILE_SIZE_MB);
Signature · 1 line
2.20.1
#1 lore raw

PATCH 1/1 tst_device: do sync() before reading test block device stat file

Hi Li

Quoted 3 lines

To avoid FS deferred IO metadata/cache interferes test result, so we do sync simply before the tst_dev_bytes_written invocation.

Looks good, acked. Also, I think we can mention it in doc/test-writing-guidelines.txt when using this api.

Best Regards
Yang Xu

Quoted 63 lines
Cc Sumit Garg <[email protected]>
Cc Yang Xu <[email protected]>
---
  testcases/kernel/syscalls/fdatasync/fdatasync03.c | 2 ++
  testcases/kernel/syscalls/fsync/fsync04.c         | 2 ++
  testcases/kernel/syscalls/sync/sync03.c           | 2 ++
  testcases/kernel/syscalls/syncfs/syncfs01.c       | 2 ++
  4 files changed, 8 insertions(+)
testcases/kernel/syscalls/fdatasync/fdatasync03.c +2 -0
index ee50e75c9..032ac4b58 100644
@@ -32,6 +32,8 @@ static void verify_fdatasync(void)
 
  	fd = SAFE_OPEN(FNAME, O_RDWR|O_CREAT, MODE);
 
+	sync();
+
  	tst_dev_bytes_written(tst_device->dev);
 
  	tst_fill_fd(fd, 0, TST_MB, FILE_SIZE_MB);
testcases/kernel/syscalls/fsync/fsync04.c +2 -0
index c67fc5692..3c1f45e94 100644
@@ -32,6 +32,8 @@ static void verify_fsync(void)
 
  	fd = SAFE_OPEN(FNAME, O_RDWR|O_CREAT, MODE);
 
+	sync();
+
  	tst_dev_bytes_written(tst_device->dev);
 
  	tst_fill_fd(fd, 0, TST_MB, FILE_SIZE_MB);
testcases/kernel/syscalls/sync/sync03.c +2 -0
index a6f72d2ed..085ccfdeb 100644
@@ -32,6 +32,8 @@ static void verify_sync(void)
 
  	fd = SAFE_OPEN(FNAME, O_RDWR|O_CREAT, MODE);
 
+	sync();
+
  	tst_dev_bytes_written(tst_device->dev);
 
  	tst_fill_fd(fd, 0, TST_MB, FILE_SIZE_MB);
testcases/kernel/syscalls/syncfs/syncfs01.c +2 -0
index 051a19ea6..3cf404450 100644
@@ -33,6 +33,8 @@ static void verify_syncfs(void)
 
  	fd = SAFE_OPEN(FNAME, O_RDWR|O_CREAT, MODE);
 
+	sync();
+
  	tst_dev_bytes_written(tst_device->dev);
 
  	tst_fill_fd(fd, 0, TST_MB, FILE_SIZE_MB);
author
#2 lore raw

PATCH 1/1 tst_device: do sync() before reading test block device stat file

Quoted 9 lines · Yang Xu

Hi Li

Quoted 3 lines

To avoid FS deferred IO metadata/cache interferes test result, so we do sync simply before the tst_dev_bytes_written invocation.

Looks good, acked. Also, I think we can mention it in doc/test-writing-guidelines.txt when using this api.

Ok, I will append one line as:

doc/test-writing-guidelines.txt +0 -0
@@ -1072,7 +1072,9 @@ unsigned long tst_dev_bytes_written(const char *dev);
 -------------------------------------------------------------------------------
 
 This function reads test block device stat file (/sys/block/<device>/stat)

and
-returns the bytes written since the last invocation of this function. +returns the bytes written since the last invocation of this function. To avoid
+FS deferred IO metadata/cache interferes the test result, we suggest doing sync()
+before the tst_dev_bytes_written first invocation.

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20200102/bbed2fbd/attachment-0001.htm>
#3 lore raw

PATCH 1/1 tst_device: do sync() before reading test block device stat file

Quoted 25 lines · Li Wang

On Thu, Jan 2, 2020 at 10:10 AM Yang Xu <[email protected] <mailto:[email protected]>> wrote:

Hi Li
> To avoid FS deferred IO metadata/cache interferes test result, so we > do sync simply before the tst_dev_bytes_written invocation. >
Looks good, acked. Also, I think we can mention it in doc/test-writing-guidelines.txt when using this api.

Ok, I will append one line as:

doc/test-writing-guidelines.txt +0 -0
@@ -1072,7 +1072,9 @@ unsigned long tst_dev_bytes_written(const char *dev);
 ?-------------------------------------------------------------------------------
 
 ?This function reads test block device stat file 

(/sys/block/<device>/stat) and

doc/test-writing-guidelines.txt +1 -1
-returns the bytes written since the last invocation of this function.
+returns the bytes written since the last invocation of this function. 
Quoted 4 lines

To avoid
+FS deferred IO metadata/cache interferes the test result, we suggest doing sync()
+before the tst_dev_bytes_written first invocation.

OK.

I also meet another problem when using this api on old kernel.

tst_device.c:395: CONF: Test device stat file: /sys/block/loop0/stat broken

/sys/block/loop0/stat is all 0 and case reports TCONF. on new kernel, this value is normal. This is a block layer or loop device driver modifition several yeas ago?

ps:I know ltp used LOOP_SET_FD to make loop device simulate other filesystems. I am trying to find a generic way about this api.

Best Regards
Yang Xu

Quoted 4 lines
Signature · 3 lines
-- 
Regards,
Li Wang
author
#4 lore raw

PATCH 1/1 tst_device: do sync() before reading test block device stat file

Quoted 37 lines · Yang Xu
Quoted 9 lines · Li Wang

On Thu, Jan 2, 2020 at 10:10 AM Yang Xu <[email protected] <mailto:[email protected]>> wrote:

Hi Li
> To avoid FS deferred IO metadata/cache interferes test result, so

we

doc/test-writing-guidelines.txt unsigned long tst_dev_bytes_written(const char

Quoted 10 lines

> do sync simply before the tst_dev_bytes_written invocation. >
Looks good, acked. Also, I think we can mention it in doc/test-writing-guidelines.txt when using this api.

Ok, I will append one line as:

doc/test-writing-guidelines.txt +0 -0
@@ -1072,7 +1072,9 @@ unsigned long tst_dev_bytes_written(const char

*dev);

Quoted 1 line

-------------------------------------------------------------------------------

Quoted 9 lines

This function reads test block device stat file
(/sys/block/<device>/stat) and

doc/test-writing-guidelines.txt +1 -1
-returns the bytes written since the last invocation of this function.
+returns the bytes written since the last invocation of this function.

To avoid
+FS deferred IO metadata/cache interferes the test result, we suggest doing sync()
+before the tst_dev_bytes_written first invocation.

OK.

Quoted 8 lines

I also meet another problem when using this api on old kernel.

tst_device.c:395: CONF: Test device stat file: /sys/block/loop0/stat broken

/sys/block/loop0/stat is all 0 and case reports TCONF. on new kernel, this value is normal. This is a block layer or loop device driver modifition several yeas ago?

It sounds like a kernel issue, can you tell which kernel version you did test?

Quoted 13 lines

ps:I know ltp used LOOP_SET_FD to make loop device simulate other filesystems. I am trying to find a generic way about this api.

Best Regards
Yang Xu

Quoted 4 lines

--
Regards,
Li Wang

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20200103/30712709/attachment.htm>
#5 lore raw

PATCH 1/1 tst_device: do sync() before reading test block device stat file

Quoted 57 lines · Li Wang

On Thu, Jan 2, 2020 at 2:46 PM Yang Xu <[email protected] <mailto:[email protected]>> wrote:

    on 2020/01/02 14:31, Li Wang wrote:
     >
     >
     > On Thu, Jan 2, 2020 at 10:10 AM Yang Xu
    <[email protected] <mailto:[email protected]>
     > <mailto:[email protected]
    <mailto:[email protected]>>> wrote:
     >
     >
     >
     >? ? ?Hi Li
     >? ? ? > To avoid FS deferred IO metadata/cache interferes test
    result, so we
     >? ? ? > do sync simply before the tst_dev_bytes_written invocation.
     >? ? ? >
     >? ? ?Looks good, acked. Also, I think we can mention it in
     >? ? ?doc/test-writing-guidelines.txt when using this api.
     >
     >
     > Ok, I will append one line as:
     > --- a/doc/test-writing-guidelines.txt
     > +++ b/doc/test-writing-guidelines.txt
     > @@ -1072,7 +1072,9 @@ unsigned long tst_dev_bytes_written(const
    char *dev);
     > 
     ?-------------------------------------------------------------------------------
     >
     >? ?This function reads test block device stat file
     > (/sys/block/<device>/stat) and
     > -returns the bytes written since the last invocation of this
    function.
     > +returns the bytes written since the last invocation of this
    function.
     > To avoid
     > +FS deferred IO metadata/cache interferes the test result, we
    suggest
     > doing sync()
     > +before the tst_dev_bytes_written first invocation.
    OK.

    I also meet another problem when using this api on old kernel.

    tst_device.c:395: CONF: Test device stat file: /sys/block/loop0/stat
    broken

    /sys/block/loop0/stat is all 0 and case reports TCONF. on new kernel,
    this value is normal. This is a block layer or loop device driver
    modifition several yeas ago?
Quoted 3 lines

It sounds like a kernel issue, can you tell which kernel version you did test?

I test this on 3.10.0-1101.el7.x86_64. It also reports TCONF on old fedora such as fedora 20,21. I guess loop driver modification(I am verifying ...) causes it.

Quoted 17 lines

ps:I know ltp used LOOP_SET_FD to make loop device simulate other filesystems. I am trying to find a generic way about this api.

Best Regards
Yang Xu
>
> --
> Regards,
> Li Wang

Signature · 2 lines
Regards,
Li Wang
#6 lore raw

PATCH 1/1 tst_device: do sync() before reading test block device stat file

Hi Li

Quoted 61 lines
Quoted 58 lines · Li Wang

On Thu, Jan 2, 2020 at 2:46 PM Yang Xu <[email protected] <mailto:[email protected]>> wrote:

??? on 2020/01/02 14:31, Li Wang wrote:
???? >
???? >
???? > On Thu, Jan 2, 2020 at 10:10 AM Yang Xu
??? <[email protected] <mailto:[email protected]> ???? > <mailto:[email protected]
??? <mailto:[email protected]>>> wrote:
???? >
???? >
???? >
???? >? ? ?Hi Li
???? >? ? ? > To avoid FS deferred IO metadata/cache interferes test ??? result, so we
???? >? ? ? > do sync simply before the tst_dev_bytes_written invocation. ???? >? ? ? >
???? >? ? ?Looks good, acked. Also, I think we can mention it in ???? >? ? ?doc/test-writing-guidelines.txt when using this api. ???? >
???? >
???? > Ok, I will append one line as:
???? > --- a/doc/test-writing-guidelines.txt
???? > +++ b/doc/test-writing-guidelines.txt
???? > @@ -1072,7 +1072,9 @@ unsigned long tst_dev_bytes_written(const ??? char *dev);
???? > ?-------------------------------------------------------------------------------

???? >
???? >? ?This function reads test block device stat file ???? > (/sys/block/<device>/stat) and
???? > -returns the bytes written since the last invocation of this ??? function.
???? > +returns the bytes written since the last invocation of this ??? function.
???? > To avoid
???? > +FS deferred IO metadata/cache interferes the test result, we ??? suggest
???? > doing sync()
???? > +before the tst_dev_bytes_written first invocation. ??? OK.

??? I also meet another problem when using this api on old kernel.

??? tst_device.c:395: CONF: Test device stat file: /sys/block/loop0/stat ??? broken

??? /sys/block/loop0/stat is all 0 and case reports TCONF. on new kernel, ??? this value is normal. This is a block layer or loop device driver ??? modifition several yeas ago?

Quoted 6 lines
Quoted 3 lines

It sounds like a kernel issue, can you tell which kernel version you did test?

I test this on 3.10.0-1101.el7.x86_64. It also reports TCONF on old fedora such as fedora 20,21. I guess loop driver modification(I am verifying ...) causes it.

Before kernel commit b5dd2f6047ca[1] (block: loop: improve performance via blk-mq), this /sys/block/loop0/stat value is all 0 for loop device because loop device uses its own make requestion function but not blk end request (so these value can not be counted). After onversion of loop to blk-mq, this value is normal. This patch was merged into upstream kernel since v4.0. When using tst_dev_bytes_written api(using loop device as block device) on older kernel, this /sys/block/loop0/stat value is all 0 and we should mention user to export $LTP_DEV to test. Also, we can do it in other patch.

[1]https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/block/loop.c?id=b5dd2f6047ca108001328aac0e8588edd15f1778

ps:I don't know block device(bio,request) or loop driver very much. If I was wrong, please feel free to correct me.

Best Regards
Yang Xu

Quoted 20 lines
Quoted 17 lines

??? ps:I know ltp used LOOP_SET_FD to make loop device simulate other ??? filesystems. I am trying to find a generic way about this api.

??? Best Regards
??? Yang Xu
???? >
???? > --
???? > Regards,
???? > Li Wang

Signature · 2 lines
Regards,
Li Wang
#7 lore raw

PATCH 1/1 tst_device: do sync() before reading test block device stat file

In reply to Li Wang

Hi!

Quoted 2 lines

To avoid FS deferred IO metadata/cache interferes test result, so we do sync simply before the tst_dev_bytes_written invocation.

Can we do fsync() on the fd instead of full sync()? That should be slightly faster.

Signature · 2 lines
Cyril Hrubis
[email protected]
author
#8 lore raw

PATCH 1/1 tst_device: do sync() before reading test block device stat file

Hi Cyril,

Quoted 7 lines · Cyril Hrubis

Hi!

Quoted 2 lines

To avoid FS deferred IO metadata/cache interferes test result, so we do sync simply before the tst_dev_bytes_written invocation.

Can we do fsync() on the fd instead of full sync()? That should be slightly faster.

Probably you miss the previous discussed [1], we use sync() here because we do want to make sure all FS metadata/cache being written back before the testing since there is no obtainable file descriptor 'fd' for the ext4 deferred IO (e.g. initialize the journal and inode tables).

If I was wrong here, feel free to correct me.

[1] http://lists.linux.it/pipermail/ltp/2019-December/014792.html

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20200103/afa5ca78/attachment.htm>
#9 lore raw

PATCH 1/1 tst_device: do sync() before reading test block device stat file

Hi!

Quoted 11 lines
Quoted 6 lines
Quoted 2 lines

To avoid FS deferred IO metadata/cache interferes test result, so we do sync simply before the tst_dev_bytes_written invocation.

Can we do fsync() on the fd instead of full sync()? That should be slightly faster.

Probably you miss the previous discussed [1], we use sync() here because we do want to make sure all FS metadata/cache being written back before the testing since there is no obtainable file descriptor 'fd' for the ext4 deferred IO (e.g. initialize the journal and inode tables).

Ah, right, we measure I/O to the whole device, so we would have to sync the device in question. Then syncfs() on the fd we got should work right? And it should avoid syncing unrelated filesystems as well.

Signature · 2 lines
Cyril Hrubis
[email protected]
#10 lore raw

PATCH 1/1 tst_device: do sync() before reading test block device stat file

Hi!

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 2 lines

To avoid FS deferred IO metadata/cache interferes test result, so we do sync simply before the tst_dev_bytes_written invocation.

Can we do fsync() on the fd instead of full sync()? That should be slightly faster.

Quoted 8 lines
Quoted 4 lines

Probably you miss the previous discussed [1], we use sync() here because we do want to make sure all FS metadata/cache being written back before the testing since there is no obtainable file descriptor 'fd' for the ext4 deferred IO (e.g. initialize the journal and inode tables).

Ah, right, we measure I/O to the whole device, so we would have to sync the device in question. Then syncfs() on the fd we got should work right? And it should avoid syncing unrelated filesystems as well.

It wouldn't probably harm to create an inline function in the test library just for a sake of documentation, something as:

static inline void tst_dev_sync(int fd)

{
	syncfs(fd);
}

Then we end up calling:

	tst_dev_sync(fd);
	tst_dev_bytes_written(device);
Signature · 2 lines
Cyril Hrubis
[email protected]
author
#11 lore raw

PATCH 1/1 tst_device: do sync() before reading test block device stat file

Quoted 29 lines · Cyril Hrubis

Hi!

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

To avoid FS deferred IO metadata/cache interferes test result, so

we

Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 1 line

do sync simply before the tst_dev_bytes_written invocation.

Can we do fsync() on the fd instead of full sync()? That should be slightly faster.

Probably you miss the previous discussed [1], we use sync() here

because we

Quoted 1 line
Quoted 1 line

do want to make sure all FS metadata/cache being written back before

the

Quoted 6 lines
Quoted 2 lines

testing since there is no obtainable file descriptor 'fd' for the ext4 deferred IO (e.g. initialize the journal and inode tables).

Ah, right, we measure I/O to the whole device, so we would have to sync the device in question. Then syncfs() on the fd we got should work right? And it should avoid syncing unrelated filesystems as well.

It wouldn't probably harm to create an inline function in the test library just for a sake of documentation, something as:

static inline void tst_dev_sync(int fd)

{
        syncfs(fd);
}
Quoted 5 lines

Then we end up calling:

        tst_dev_sync(fd);
        tst_dev_bytes_written(device);

Agree.

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20200107/465be886/attachment.htm>
author
#12 lore raw

PATCH 1/1 tst_device: do sync() before reading test block device stat file

In reply to Cyril Hrubis

Hi Cyril,

Quoted 9 lines · Cyril Hrubis

Hi!

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

To avoid FS deferred IO metadata/cache interferes test result, so we do sync simply before the tst_dev_bytes_written invocation.

Can we do fsync() on the fd instead of full sync()? That should be slightly faster.

Probably you miss the previous discussed [1], we use sync() here because

Quoted 9 lines

we

Quoted 3 lines

do want to make sure all FS metadata/cache being written back before the testing since there is no obtainable file descriptor 'fd' for the ext4 deferred IO (e.g. initialize the journal and inode tables).

Ah, right, we measure I/O to the whole device, so we would have to sync the device in question. Then syncfs() on the fd we got should work right? And it should avoid syncing unrelated filesystems as well.

Good point, I double read the fsync() and syncfs() manual page. It looks like syncfs() is not the same as the previous function, it synchronizes the file system containing file referred to by the open file descriptor fd, I mixed them before, to be honest.

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20200107/c94dff56/attachment.htm>