tst_device: do sync() before reading test block device stat file
PATCH 1/1 tst_device: do sync() before reading test block device stat file
To avoid FS deferred IO metadata/cache interferes test result, so we do sync simply before the tst_dev_bytes_written invocation.
--- testcases/kernel/syscalls/fdatasync/fdatasync03.c | 2 ++ testcases/kernel/syscalls/fsync/fsync04.c | 2 ++ testcases/kernel/syscalls/sync/sync03.c | 2 ++ testcases/kernel/syscalls/syncfs/syncfs01.c | 2 ++ 4 files changed, 8 insertions(+)
testcases/kernel/syscalls/fdatasync/fdatasync03.c +2 -0
@@ -32,6 +32,8 @@ static void verify_fdatasync(void) fd =SAFE_OPEN (FNAME ,O_RDWR |O_CREAT ,MODE ); +sync (); +tst_dev_bytes_written (tst_device->dev);tst_fill_fd (fd,0 ,TST_MB ,FILE_SIZE_MB );
testcases/kernel/syscalls/fsync/fsync04.c +2 -0
@@ -32,6 +32,8 @@ static void verify_fsync(void) fd =SAFE_OPEN (FNAME ,O_RDWR |O_CREAT ,MODE ); +sync (); +tst_dev_bytes_written (tst_device->dev);tst_fill_fd (fd,0 ,TST_MB ,FILE_SIZE_MB );
testcases/kernel/syscalls/sync/sync03.c +2 -0
@@ -32,6 +32,8 @@ static void verify_sync(void) fd =SAFE_OPEN (FNAME ,O_RDWR |O_CREAT ,MODE ); +sync (); +tst_dev_bytes_written (tst_device->dev);tst_fill_fd (fd,0 ,TST_MB ,FILE_SIZE_MB );
testcases/kernel/syscalls/syncfs/syncfs01.c +2 -0
@@ -33,6 +33,8 @@ static void verify_syncfs(void) fd =SAFE_OPEN (FNAME ,O_RDWR |O_CREAT ,MODE ); +sync (); +tst_dev_bytes_written (tst_device->dev);tst_fill_fd (fd,0 ,TST_MB ,FILE_SIZE_MB );
Signature · 1 line
2.20.1
PATCH 1/1 tst_device: do sync() before reading test block device stat file
Hi Li
Quoted 3 lines
To avoid FS deferred IO metadata/cache interferes test result, so we do sync simply before the
tst_dev_bytes_writteninvocation.
Looks good, acked. Also, I think we can mention it in doc/test-writing-guidelines.txt when using this api.
Best Regards
Yang Xu
Quoted 63 lines
Signed-off-by Li Wang <[email protected]>Cc Sumit Garg <[email protected]>Cc Yang Xu <[email protected]>--- testcases/kernel/syscalls/fdatasync/fdatasync03.c | 2 ++ testcases/kernel/syscalls/fsync/fsync04.c | 2 ++ testcases/kernel/syscalls/sync/sync03.c | 2 ++ testcases/kernel/syscalls/syncfs/syncfs01.c | 2 ++ 4 files changed, 8 insertions(+)testcases/kernel/syscalls/fdatasync/fdatasync03.c +2
-0@@ -32,6 +32,8 @@ static void verify_fdatasync(void) fd =SAFE_OPEN (FNAME ,O_RDWR |O_CREAT ,MODE ); +sync (); +tst_dev_bytes_written (tst_device->dev);tst_fill_fd (fd,0 ,TST_MB ,FILE_SIZE_MB );testcases/kernel/syscalls/fsync/fsync04.c +2
-0@@ -32,6 +32,8 @@ static void verify_fsync(void) fd =SAFE_OPEN (FNAME ,O_RDWR |O_CREAT ,MODE ); +sync (); +tst_dev_bytes_written (tst_device->dev);tst_fill_fd (fd,0 ,TST_MB ,FILE_SIZE_MB );testcases/kernel/syscalls/sync/sync03.c +2
-0@@ -32,6 +32,8 @@ static void verify_sync(void) fd =SAFE_OPEN (FNAME ,O_RDWR |O_CREAT ,MODE ); +sync (); +tst_dev_bytes_written (tst_device->dev);tst_fill_fd (fd,0 ,TST_MB ,FILE_SIZE_MB );testcases/kernel/syscalls/syncfs/syncfs01.c +2
-0@@ -33,6 +33,8 @@ static void verify_syncfs(void) fd =SAFE_OPEN (FNAME ,O_RDWR |O_CREAT ,MODE ); +sync (); +tst_dev_bytes_written (tst_device->dev);tst_fill_fd (fd,0 ,TST_MB ,FILE_SIZE_MB );
PATCH 1/1 tst_device: do sync() before reading test block device stat file
Quoted 9 lines · Yang Xu
Hi Li
Quoted 3 lines
To avoid FS deferred IO metadata/cache interferes test result, so we do sync simply before the
tst_dev_bytes_writteninvocation.Looks good, acked. Also, I think we can mention it in
doc/test-writing-guidelines.txtwhen using this api.
Ok, I will append one line as:
doc/test-writing-guidelines.txt +0 -0
@@ -1072,7 +1072,9 @@ unsigned long tst_dev_bytes_written(const char *dev); ------------------------------------------------------------------------------- This function reads test block device stat file (/sys/block/<device>/stat)
and
-returns the bytes written since the last invocation of this function. +returns the bytes written since the last invocation of this function. To avoid
+FS deferred IO metadata/cache interferes the test result, we suggest doing sync()
+before the tst_dev_bytes_written first invocation.
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20200102/bbed2fbd/attachment-0001.htm>
PATCH 1/1 tst_device: do sync() before reading test block device stat file
Quoted 25 lines · Li Wang
On Thu, Jan 2, 2020 at 10:10 AM Yang Xu <[email protected] <mailto:[email protected]>> wrote:
Hi Li
> To avoid FS deferred IO metadata/cache interferes test result, so we > do sync simply before thetst_dev_bytes_writteninvocation. >
Looks good, acked. Also, I think we can mention it indoc/test-writing-guidelines.txtwhen using this api.Ok, I will append one line as:
doc/test-writing-guidelines.txt +0
-0@@ -1072,7 +1072,9 @@ unsigned long tst_dev_bytes_written(const char *dev); ?------------------------------------------------------------------------------- ?This function reads test block device stat file(/sys/block/<device>/stat) and
doc/test-writing-guidelines.txt +1
-1-returns the bytes written since the last invocation of this function. +returns the bytes written since the last invocation of this function.
Quoted 4 lines
To avoid
+FS deferred IO metadata/cache interferes the test result, we suggest doingsync()
+before thetst_dev_bytes_writtenfirst invocation.
OK.
I also meet another problem when using this api on old kernel.
tst_device.c:395: CONF: Test device stat file: /sys/block/loop0/stat broken
/sys/block/loop0/stat is all 0 and case reports TCONF. on new kernel, this value is normal. This is a block layer or loop device driver modifition several yeas ago?
ps:I know ltp used LOOP_SET_FD to make loop device simulate other filesystems. I am trying to find a generic way about this api.
Best Regards
Yang Xu
Quoted 4 lines
Signature · 3 lines
-- Regards, Li Wang
PATCH 1/1 tst_device: do sync() before reading test block device stat file
Quoted 37 lines · Yang Xu
Quoted 9 lines · Li Wang
On Thu, Jan 2, 2020 at 10:10 AM Yang Xu <[email protected] <mailto:[email protected]>> wrote:
Hi Li
> To avoid FS deferred IO metadata/cache interferes test result, sowe
doc/test-writing-guidelines.txt unsigned long tst_dev_bytes_written(const char
Quoted 10 lines
> do sync simply before the
tst_dev_bytes_writteninvocation. >
Looks good, acked. Also, I think we can mention it indoc/test-writing-guidelines.txtwhen using this api.Ok, I will append one line as:
doc/test-writing-guidelines.txt +0
-0@@ -1072,7 +1072,9 @@ unsigned long tst_dev_bytes_written(const char*dev);
Quoted 1 line
-------------------------------------------------------------------------------
Quoted 9 lines
This function reads test block device stat file
(/sys/block/<device>/stat) anddoc/test-writing-guidelines.txt +1
-1-returns the bytes written since the last invocation of this function. +returns the bytes written since the last invocation of this function.To avoid
+FS deferred IO metadata/cache interferes the test result, we suggest doingsync()
+before thetst_dev_bytes_writtenfirst invocation.OK.
Quoted 8 lines
I also meet another problem when using this api on old kernel.
tst_device.c:395: CONF: Test device stat file:
/sys/block/loop0/statbroken
/sys/block/loop0/statis all 0 and case reports TCONF. on new kernel, this value is normal. This is a block layer or loop device driver modifition several yeas ago?
It sounds like a kernel issue, can you tell which kernel version you did test?
Quoted 13 lines
ps:I know ltp used
LOOP_SET_FDto make loop device simulate other filesystems. I am trying to find a generic way about this api.Best Regards
Yang XuQuoted 4 lines
--
Regards,
Li Wang
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20200103/30712709/attachment.htm>
PATCH 1/1 tst_device: do sync() before reading test block device stat file
Quoted 57 lines · Li Wang
On Thu, Jan 2, 2020 at 2:46 PM Yang Xu <[email protected] <mailto:[email protected]>> wrote:
on 2020/01/02 14:31, Li Wang wrote: > > > On Thu, Jan 2, 2020 at 10:10 AM Yang Xu <[email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]>>> wrote: > > > >? ? ?Hi Li >? ? ? > To avoid FS deferred IO metadata/cache interferes test result, so we >? ? ? > do sync simply before the tst_dev_bytes_written invocation. >? ? ? > >? ? ?Looks good, acked. Also, I think we can mention it in >? ? ?doc/test-writing-guidelines.txt when using this api. > > > Ok, I will append one line as: > --- a/doc/test-writing-guidelines.txt > +++ b/doc/test-writing-guidelines.txt > @@ -1072,7 +1072,9 @@ unsigned long tst_dev_bytes_written(const char *dev); > ?------------------------------------------------------------------------------- > >? ?This function reads test block device stat file > (/sys/block/<device>/stat) and > -returns the bytes written since the last invocation of this function. > +returns the bytes written since the last invocation of this function. > To avoid > +FS deferred IO metadata/cache interferes the test result, we suggest > doing sync() > +before the tst_dev_bytes_written first invocation. OK. I also meet another problem when using this api on old kernel. tst_device.c:395: CONF: Test device stat file: /sys/block/loop0/stat broken /sys/block/loop0/stat is all 0 and case reports TCONF. on new kernel, this value is normal. This is a block layer or loop device driver modifition several yeas ago?
Quoted 3 lines
It sounds like a kernel issue, can you tell which kernel version you did test?
I test this on 3.10.0-1101.el7.x86_64. It also reports TCONF on old fedora such as fedora 20,21. I guess loop driver modification(I am verifying ...) causes it.
Quoted 17 lines
ps:I know ltp used
LOOP_SET_FDto make loop device simulate other filesystems. I am trying to find a generic way about this api.Best Regards
Yang Xu
>
> --
> Regards,
> Li WangSignature · 2 lines
Regards, Li Wang
PATCH 1/1 tst_device: do sync() before reading test block device stat file
Hi Li
Quoted 61 lines
Quoted 58 lines · Li Wang
On Thu, Jan 2, 2020 at 2:46 PM Yang Xu <[email protected] <mailto:[email protected]>> wrote:
??? on 2020/01/02 14:31, Li Wang wrote:
???? >
???? >
???? > On Thu, Jan 2, 2020 at 10:10 AM Yang Xu
??? <[email protected] <mailto:[email protected]> ???? > <mailto:[email protected]
??? <mailto:[email protected]>>> wrote:
???? >
???? >
???? >
???? >? ? ?Hi Li
???? >? ? ? > To avoid FS deferred IO metadata/cache interferes test ??? result, so we
???? >? ? ? > do sync simply before thetst_dev_bytes_writteninvocation. ???? >? ? ? >
???? >? ? ?Looks good, acked. Also, I think we can mention it in ???? >? ? ?doc/test-writing-guidelines.txt when using this api. ???? >
???? >
???? > Ok, I will append one line as:
???? > ---a/doc/test-writing-guidelines.txt
???? > +++b/doc/test-writing-guidelines.txt
???? > @@ -1072,7 +1072,9 @@ unsigned longtst_dev_bytes_written(const ??? char *dev);
???? > ?-------------------------------------------------------------------------------???? >
???? >? ?This function reads test block device stat file ???? > (/sys/block/<device>/stat) and
???? > -returns the bytes written since the last invocation of this ??? function.
???? > +returns the bytes written since the last invocation of this ??? function.
???? > To avoid
???? > +FS deferred IO metadata/cache interferes the test result, we ??? suggest
???? > doingsync()
???? > +before thetst_dev_bytes_writtenfirst invocation. ??? OK.??? I also meet another problem when using this api on old kernel.
??? tst_device.c:395: CONF: Test device stat file:
/sys/block/loop0/stat??? broken???
/sys/block/loop0/statis all 0 and case reports TCONF. on new kernel, ??? this value is normal. This is a block layer or loop device driver ??? modifition several yeas ago?
Quoted 6 lines
Quoted 3 lines
It sounds like a kernel issue, can you tell which kernel version you did test?
I test this on 3.10.0-1101.el7.x86_64. It also reports TCONF on old fedora such as fedora 20,21. I guess loop driver modification(I am verifying ...) causes it.
Before kernel commit b5dd2f6047ca[1] (block: loop: improve performance via blk-mq), this /sys/block/loop0/stat value is all 0 for loop device because loop device uses its own make requestion function but not blk end request (so these value can not be counted). After onversion of loop to blk-mq, this value is normal. This patch was merged into upstream kernel since v4.0. When using tst_dev_bytes_written api(using loop device as block device) on older kernel, this /sys/block/loop0/stat value is all 0 and we should mention user to export $LTP_DEV to test. Also, we can do it in other patch.
ps:I don't know block device(bio,request) or loop driver very much. If I was wrong, please feel free to correct me.
Best Regards
Yang Xu
Quoted 20 lines
Quoted 17 lines
??? ps:I know ltp used
LOOP_SET_FDto make loop device simulate other ??? filesystems. I am trying to find a generic way about this api.??? Best Regards
??? Yang Xu
???? >
???? > --
???? > Regards,
???? > Li WangSignature · 2 lines
Regards, Li Wang
PATCH 1/1 tst_device: do sync() before reading test block device stat file
In reply to Li WangHi!
Quoted 2 lines
To avoid FS deferred IO metadata/cache interferes test result, so we do sync simply before the
tst_dev_bytes_writteninvocation.
Can we do fsync() on the fd instead of full sync()? That should be slightly faster.
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH 1/1 tst_device: do sync() before reading test block device stat file
Hi Cyril,
Quoted 7 lines · Cyril Hrubis
Hi!
Quoted 2 lines
To avoid FS deferred IO metadata/cache interferes test result, so we do sync simply before the
tst_dev_bytes_writteninvocation.Can we do
fsync()on the fd instead of fullsync()? That should be slightly faster.
Probably you miss the previous discussed [1], we use sync() here because we do want to make sure all FS metadata/cache being written back before the testing since there is no obtainable file descriptor 'fd' for the ext4 deferred IO (e.g. initialize the journal and inode tables).
If I was wrong here, feel free to correct me.
[1] http://lists.linux.it/pipermail/ltp/2019-December/014792.html
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20200103/afa5ca78/attachment.htm>
PATCH 1/1 tst_device: do sync() before reading test block device stat file
Hi!
Quoted 11 lines
Quoted 6 lines
Quoted 2 lines
To avoid FS deferred IO metadata/cache interferes test result, so we do sync simply before the
tst_dev_bytes_writteninvocation.Can we do
fsync()on the fd instead of fullsync()? That should be slightly faster.Probably you miss the previous discussed [1], we use
sync()here because we do want to make sure all FS metadata/cache being written back before the testing since there is no obtainable file descriptor 'fd' for the ext4 deferred IO (e.g. initialize the journal and inode tables).
Ah, right, we measure I/O to the whole device, so we would have to sync the device in question. Then syncfs() on the fd we got should work right? And it should avoid syncing unrelated filesystems as well.
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH 1/1 tst_device: do sync() before reading test block device stat file
Hi!
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 2 lines
To avoid FS deferred IO metadata/cache interferes test result, so we do sync simply before the
tst_dev_bytes_writteninvocation.Can we do
fsync()on the fd instead of fullsync()? That should be slightly faster.
Quoted 8 lines
Quoted 4 lines
Probably you miss the previous discussed [1], we use
sync()here because we do want to make sure all FS metadata/cache being written back before the testing since there is no obtainable file descriptor 'fd' for the ext4 deferred IO (e.g. initialize the journal and inode tables).Ah, right, we measure I/O to the whole device, so we would have to sync the device in question. Then
syncfs()on the fd we got should work right? And it should avoid syncing unrelated filesystems as well.
It wouldn't probably harm to create an inline function in the test library just for a sake of documentation, something as:
static inline void tst_dev_sync(int fd)
{
syncfs(fd);
}Then we end up calling:
tst_dev_sync(fd); tst_dev_bytes_written(device);
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH 1/1 tst_device: do sync() before reading test block device stat file
Quoted 29 lines · Cyril Hrubis
Hi!
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
To avoid FS deferred IO metadata/cache interferes test result, so
we
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
do sync simply before the
tst_dev_bytes_writteninvocation.Can we do
fsync()on the fd instead of fullsync()? That should be slightly faster.Probably you miss the previous discussed [1], we use
sync()herebecause we
Quoted 1 line
Quoted 1 line
do want to make sure all FS metadata/cache being written back before
the
Quoted 6 lines
Quoted 2 lines
testing since there is no obtainable file descriptor 'fd' for the ext4 deferred IO (e.g. initialize the journal and inode tables).
Ah, right, we measure I/O to the whole device, so we would have to sync the device in question. Then
syncfs()on the fd we got should work right? And it should avoid syncing unrelated filesystems as well.It wouldn't probably harm to create an inline function in the test library just for a sake of documentation, something as:
static inline void
tst_dev_sync(int fd){ syncfs(fd); }
Quoted 5 lines
Then we end up calling:
tst_dev_sync(fd); tst_dev_bytes_written(device);
Agree.
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20200107/465be886/attachment.htm>
PATCH 1/1 tst_device: do sync() before reading test block device stat file
In reply to Cyril HrubisHi Cyril,
Quoted 9 lines · Cyril Hrubis
Hi!
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
To avoid FS deferred IO metadata/cache interferes test result, so we do sync simply before the
tst_dev_bytes_writteninvocation.Can we do
fsync()on the fd instead of fullsync()? That should be slightly faster.Probably you miss the previous discussed [1], we use
sync()here because
Quoted 9 lines
we
Quoted 3 lines
do want to make sure all FS metadata/cache being written back before the testing since there is no obtainable file descriptor 'fd' for the ext4 deferred IO (e.g. initialize the journal and inode tables).
Ah, right, we measure I/O to the whole device, so we would have to sync the device in question. Then
syncfs()on the fd we got should work right? And it should avoid syncing unrelated filesystems as well.
Good point, I double read the fsync() and syncfs() manual page. It looks like syncfs() is not the same as the previous function, it synchronizes the file system containing file referred to by the open file descriptor fd, I mixed them before, to be honest.
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20200107/c94dff56/attachment.htm>