sync_file_range02: remove the toplimit of write back
RFC PATCH 1/1 sync_file_range02: remove the toplimit of write back
Test tries to sync a range of $BYTES bytes, and then makes sure that between $BYTES and $BYTES+10% was written to disk. But sometimes, more than $BYTES+10% hit the disk: "Synced 39843840, expected 33554432" so it failed as below.
tst_test.c:1179: INFO: Testing on ext4 sync_file_range02.c:74: FAIL: Sync equals write: Synced 39843840,
expected 33554432
sync_file_range02.c:74: FAIL: Sync inside of write: Synced 18612224,
expected 16777216From FS dev's view:
" The test's assumptions are fundamentally false; it thinks it can look at IO counters (tst_dev_bytes_written) for a disk before and after a system call, and attribute all of the IO seen to the system call that was made - this isn't necessarily correct. Other processes may generate IO in the background.
ext4 defers a lot of IO on a freshly made filesystem to the kernel - for example it will initialize the journal and inode tables after the mount, and this will cause extra IO.
Creating ext4 filesystems with the options: "-E lazy_itable_init=0, lazy_journal_init=0" might help.
Another option would be to raise the threshold. Essentially, the report here is that the test is failing because the filesystem wrote "too much" as a result of the sync. How much is "too much?" ..."
Let's remove the toplimit of write back, and think as long as we synced at least the expected amount, the test passes. The +10% limit seems arbitrary.
--- testcases/kernel/syscalls/sync_file_range/sync_file_range02.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
testcases/kernel/syscalls/sync_file_range/sync_file_range02.c +1 -2
@@ -66,8 +66,7 @@ static void verify_sync_file_range(struct testcase *tc)SAFE_CLOSE (fd); -if ((written >= tc->exp_sync_size) && - (written <= (tc->exp_sync_size + tc->exp_sync_size/10 ))) +if ((written >= tc->exp_sync_size))tst_res (TPASS ,"%s" , tc->desc);else tst_res(TFAIL, "%s: Synced %li, expected %li", tc->desc,
Signature · 1 line
2.20.1
RFC PATCH 1/1 sync_file_range02: remove the toplimit of write back
----- Original Message -----
Quoted 5 lines
" The test's assumptions are fundamentally false; it thinks it can look at IO counters (
tst_dev_bytes_written) for a disk before and after a system call, and attribute all of the IO seen to the system call that was made - this isn't necessarily correct. Other processes may generate IO in the background.
We create our own block device, so there shouldn't be other processes writing to it.
Quoted 3 lines
ext4 defers a lot of IO on a freshly made filesystem to the kernel - for example it will initialize the journal and inode tables after the mount
Journal was my guess as well.
Quoted 3 lines
Let's remove the toplimit of write back, and think as long as we synced at least the expected amount, the test passes. The +10% limit seems arbitrary.
I think this is reasonable approach until we find better way to measure what was synced.
RFC PATCH 1/1 sync_file_range02: remove the toplimit of write back
+ Cyril (who originally proposed upper bound check)
Quoted 9 lines · Jan Stancek
----- Original Message -----
Quoted 5 lines
" The test's assumptions are fundamentally false; it thinks it can look at IO counters (
tst_dev_bytes_written) for a disk before and after a system call, and attribute all of the IO seen to the system call that was made - this isn't necessarily correct. Other processes may generate IO in the background.
Quoted 8 lines
We create our own block device, so there shouldn't be other processes writing to it.
Quoted 3 lines
ext4 defers a lot of IO on a freshly made filesystem to the kernel - for example it will initialize the journal and inode tables after the mount
Journal was my guess as well.
To avoid similar scenarios, I suggested to add a "sync()" call just prior to test here [1]. And I couldn't reproduce the failure in 1000-times run with 4.19 kernel.
Also, I think having a "sync()" call becomes more important in case we remove the upper bound otherwise test might not provide reliable results.
[1] http://lists.linux.it/pipermail/ltp/2019-October/014157.html
-Sumit
Quoted 13 lines
Quoted 4 lines
Let's remove the toplimit of write back, and think as long as we synced at least the expected amount, the test passes. The +10% limit seems arbitrary.
I think this is reasonable approach until we find better way to measure what was synced.
Acked-by Jan Stancek <[email protected]>--
Mailing list info: https://lists.linux.it/listinfo/ltp
RFC PATCH 1/1 sync_file_range02: remove the toplimit of write back
Hi Sumit,
Quoted 19 lines · Sumit Garg
+ Cyril (who originally proposed upper bound check)
Quoted 16 lines · Jan Stancek
----- Original Message -----
Quoted 5 lines
" The test's assumptions are fundamentally false; it thinks it can look at IO counters (
tst_dev_bytes_written) for a disk before and after a system call, and attribute all of the IO seen to the system call that was made - this isn't necessarily correct. Other processes may generate IO in the background.We create our own block device, so there shouldn't be other processes writing to it.
Quoted 3 lines
ext4 defers a lot of IO on a freshly made filesystem to the kernel - for example it will initialize the journal and inode tables after the mount
Quoted 6 lines
Quoted 1 line
Journal was my guess as well.
To avoid similar scenarios, I suggested to add a "
sync()" call just prior to test here [1]. And I couldn't reproduce the failure in 1000-times run with 4.19 kernel.
Yes, that makes sense to me.
Quoted 24 lines
Also, I think having a "
sync()" call becomes more important in case we remove the upper bound otherwise test might not provide reliable results.[1] http://lists.linux.it/pipermail/ltp/2019-October/014157.html
-Sumit
Quoted 13 lines
Quoted 4 lines
Let's remove the toplimit of write back, and think as long as we synced at least the expected amount, the test passes. The +10% limit seems arbitrary.
I think this is reasonable approach until we find better way to measure what was synced.
Acked-by Jan Stancek <[email protected]>--
Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20191219/d21b6bec/attachment.htm>
RFC PATCH 1/1 sync_file_range02: remove the toplimit of write back
Hi Li,
Quoted 24 lines · Li Wang
Hi Sumit,
Quoted 20 lines · Sumit Garg
+ Cyril (who originally proposed upper bound check)
Quoted 16 lines · Jan Stancek
----- Original Message -----
Quoted 5 lines
" The test's assumptions are fundamentally false; it thinks it can look at IO counters (
tst_dev_bytes_written) for a disk before and after a system call, and attribute all of the IO seen to the system call that was made - this isn't necessarily correct. Other processes may generate IO in the background.We create our own block device, so there shouldn't be other processes writing to it.
Quoted 3 lines
ext4 defers a lot of IO on a freshly made filesystem to the kernel - for example it will initialize the journal and inode tables after the mount
Quoted 9 lines
Quoted 5 lines
Quoted 1 line
Journal was my guess as well.
To avoid similar scenarios, I suggested to add a "
sync()" call just prior to test here [1]. And I couldn't reproduce the failure in 1000-times run with 4.19 kernel.Yes, that makes sense to me.
Would you mind to give it a try and check if you could reproduce the failure?
-Sumit
Quoted 29 lines
Quoted 24 lines
Also, I think having a "
sync()" call becomes more important in case we remove the upper bound otherwise test might not provide reliable results.[1] http://lists.linux.it/pipermail/ltp/2019-October/014157.html
-Sumit
Quoted 13 lines
Quoted 4 lines
Let's remove the toplimit of write back, and think as long as we synced at least the expected amount, the test passes. The +10% limit seems arbitrary.
I think this is reasonable approach until we find better way to measure what was synced.
Acked-by Jan Stancek <[email protected]>--
Mailing list info: https://lists.linux.it/listinfo/ltp--
Regards,
Li Wang
RFC PATCH 1/1 sync_file_range02: remove the toplimit of write back
Quoted 6 lines · Sumit Garg
...
Quoted 5 lines
Quoted 3 lines
To avoid similar scenarios, I suggested to add a "
sync()" call just prior to test here [1]. And I couldn't reproduce the failure in 1000-times run with 4.19 kernel.
Quoted 6 lines
Quoted 2 lines
Yes, that makes sense to me.
Would you mind to give it a try and check if you could reproduce the failure?
Actually, I haven't had one time to reproduce it. This failure was reported by CI jobs and very low frequency occurred. The fix is based on a discussion with FS developer analysis.
But I would have a try with this patch(+ sync()).
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20191219/b20347a7/attachment.htm>
RFC PATCH 1/1 sync_file_range02: remove the toplimit of write back
Hi,
Quoted 17 lines · Li Wang
Quoted 6 lines · Sumit Garg
...
Quoted 5 lines
Quoted 3 lines
To avoid similar scenarios, I suggested to add a "
sync()" call just prior to test here [1]. And I couldn't reproduce the failure in 1000-times run with 4.19 kernel.Quoted 6 lines
Quoted 2 lines
Yes, that makes sense to me.
Would you mind to give it a try and check if you could reproduce the failure?
Quoted 6 lines
Actually, I haven't had one time to reproduce it. This failure was reported by CI jobs and very low frequency occurred. The fix is based on a discussion with FS developer analysis.
But I would have a try with this patch(+
sync()).
I have tried this for many different systems, and it doesn't hit the failure at least one time. So if no more comments, I would merge the patch as below in the next step.
testcases/kernel/syscalls/sync_file_range/sync_file_range02.c +3 -2
@@ -48,6 +48,8 @@ static void verify_sync_file_range(struct testcase *tc)lseek (fd, tc->write_off,SEEK_SET ); +sync (); +tst_dev_bytes_written (tst_device->dev);tst_fill_fd (fd,0 ,TST_MB , tc->write_size_mb);
@@ -66,8 +68,7 @@ static void verify_sync_file_range(struct testcase *tc)SAFE_CLOSE (fd); -if ((written >= tc->exp_sync_size) && - (written <= (tc->exp_sync_size + tc->exp_sync_size/10 ))) +if (written >= tc->exp_sync_size)tst_res (TPASS ,"%s" , tc->desc);else tst_res(TFAIL, "%s: Synced %li, expected %li", tc->desc,
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20191231/886c1803/attachment-0001.htm>
RFC PATCH 1/1 sync_file_range02: remove the toplimit of write back
Quoted 41 lines · Li Wang
Hi,
Quoted 14 lines · Li Wang
Quoted 10 lines · Sumit Garg
...
Quoted 7 lines
Quoted 3 lines
To avoid similar scenarios, I suggested to add a "
sync()" call just prior to test here [1]. And I couldn't reproduce the failure in 1000-times run with 4.19 kernel.Yes, that makes sense to me.
Quoted 6 lines
Quoted 1 line
Would you mind to give it a try and check if you could reproduce the failure?
Actually, I haven't had one time to reproduce it. This failure was reported by CI jobs and very low frequency occurred. The fix is based on a discussion with FS developer analysis.
But I would have a try with this patch(+
sync()).I have tried this for many different systems, and it doesn't hit the failure at least one time. So if no more comments, I would merge the patch as below in the next step.
testcases/kernel/syscalls/sync_file_range/sync_file_range02.c +2
-0@@ -48,6 +48,8 @@ static void verify_sync_file_range(struct testcase *tc)lseek (fd, tc->write_off,SEEK_SET ); +sync (); +tst_dev_bytes_written (tst_device->dev);tst_fill_fd (fd,0 ,TST_MB , tc->write_size_mb);
testcases/kernel/syscalls/sync_file_range/sync_file_range02.c static void verify_sync_file_range(struct testcase *tc) · line 74
Quoted 9 lines
testcases/kernel/syscalls/sync_file_range/sync_file_range02.c +1
-2@@ -66,8 +68,7 @@ static void verify_sync_file_range(struct testcase *tc)SAFE_CLOSE (fd); -if ((written >= tc->exp_sync_size) && - (written <= (tc->exp_sync_size + tc->exp_sync_size/10 ))) +if (written >= tc->exp_sync_size)tst_res (TPASS ,"%s" , tc->desc);else tst_res(TFAIL, "%s: Synced %li, expected %li", tc->desc,
Quoted 3 lines
--
Regards,
Li Wang
RFC PATCH 1/1 sync_file_range02: remove the toplimit of write back
Quoted 4 lines · Sumit Garg
...
Reviewed-by Sumit Garg <[email protected]>
Pushed.
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20200102/7be7051c/attachment.htm>
RFC PATCH 1/1 sync_file_range02: remove the toplimit of write back
In reply to Li WangHi Li
Quoted 28 lines · Li Wang
Hi,
On Thu, Dec 19, 2019 at 5:15 PM Li Wang <[email protected] <mailto:[email protected]>> wrote:
On Thu, Dec 19, 2019 at 4:52 PM Sumit Garg <[email protected] <mailto:[email protected]>> wrote:
...
>> To avoid similar scenarios, I suggested to add a "sync()" call just
>> prior to test here [1]. And I couldn't reproduce the failure in >> 1000-times run with 4.19 kernel.
>
>
> Yes, that makes sense to me.
>Would you mind to give it a try and check if you could reproduce the failure?
Actually, I haven't had one time to reproduce it. This failure was reported by CI jobs and very low frequency occurred. The fix is based on a discussion?with FS developer analysis.
Quoted 6 lines
But I would have a try with this patch(+
sync()).I have tried this for many different systems, and it doesn't hit the failure at least one time. So if no more comments, I would merge the patch as below in the next step.
I have a question.we must call sync()? I think syncfs is more accurate.
Best Regards
Yang Xu
Quoted 20 lines
testcases/kernel/syscalls/sync_file_range/sync_file_range02.c +3
-2@@ -48,6 +48,8 @@ static void verify_sync_file_range(struct testcase *tc) ? ? ? ? lseek(fd, tc->write_off, SEEK_SET); + ? ? ? sync(); + ? ? ? ? tst_dev_bytes_written(tst_device->dev); ? ? ? ? tst_fill_fd(fd, 0, TST_MB, tc->write_size_mb);@@ -66,8 +68,7 @@ static void verify_sync_file_range(struct testcase *tc) ? ? ? ? SAFE_CLOSE(fd); - ? ? ? if ((written >= tc->exp_sync_size) && - ? ? ? ? ? (written <= (tc->exp_sync_size + tc->exp_sync_size/10))) + ? ? ? if (written >= tc->exp_sync_size) ? ? ? ? ? ? ? ? tst_res(TPASS, "%s", tc->desc);
Quoted 7 lines
? ? ? ? else
? ? ? ? ? ? ? ? tst_res(TFAIL, "%s: Synced %li, expected %li",tc->desc,Signature · 2 lines
Regards, Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20191231/eb79f954/attachment.htm>
RFC PATCH 1/1 sync_file_range02: remove the toplimit of write back
Hi Yang,
Quoted 10 lines · Yang Xu
...
I have tried this for many different systems, and it doesn't hit the failure at least one time. So if no more comments, I would merge the patch as below in the next step.
I have a question.we must call
sync()? I think syncfs is more accurate.
Here we use sync() is to make fs metadata/cache being written back before testing because there is no obtainable file descriptor 'fd' for the ext4 deferred IO (e.g. initialize the journal and inode tables).
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20191231/4dd2b6ca/attachment.htm>
RFC PATCH 1/1 sync_file_range02: remove the toplimit of write back
Hi Li
Quoted 12 lines
Hi Yang,
Yang Xu <[email protected]
<mailto:[email protected]>> wrote:...Quoted 4 lines
I have tried this for many different systems, and it doesn't hit the failure at least one time. So if no more comments, I would merge the patch as below in the next step.
Quoted 6 lines
I have a question.we must call
sync()? I think syncfs is more accurate.Here we use
sync()is to make fs metadata/cache being written back before testing because there is no obtainable?file descriptor 'fd' for the ext4 deferred?IO (e.g. initialize the journal and inode tables).
I see. For other test cases using tst_dev_bytes_written api such as fdatasync03.c, we also need to call sync() to make this written value more accurate. What do you think about it?
Best Regards
Yang Xu
Quoted 4 lines
Signature · 3 lines
-- Regards, Li Wang
RFC PATCH 1/1 sync_file_range02: remove the toplimit of write back
Quoted 15 lines · Yang Xu
...
Quoted 5 lines
Quoted 3 lines
I have tried this for many different systems, and it doesn't hit the failure at least one time. So if no more comments, I would merge the patch as below in the next step.
I have a question.we must call
sync()? I think syncfs is moreaccurate.
Quoted 5 lines
Here we use
sync()is to make fs metadata/cache being written back before testing because there is no obtainable file descriptor 'fd' for the ext4 deferred IO (e.g. initialize the journal and inode tables).
Quoted 4 lines
I see. For other test cases using
tst_dev_bytes_writtenapi such asfdatasync03.c, we also need to callsync()to make this written value more accurate. What do you think about it?
Good point. I think we should take care of them as we do for sync_file_range02 too. It will more easily report fail in the case of a situation that synced data is less than expected.
$ git grep tst_dev_bytes_written kernel/syscalls/fdatasync/fdatasync03.c: tst_dev_bytes_written(tst_device->dev); kernel/syscalls/fdatasync/fdatasync03.c: written = tst_dev_bytes_written(tst_device->dev); kernel/syscalls/fsync/fsync04.c: tst_dev_bytes_written(tst_device->dev); kernel/syscalls/fsync/fsync04.c: written = tst_dev_bytes_written(tst_device->dev); kernel/syscalls/sync/sync03.c: tst_dev_bytes_written(tst_device->dev); kernel/syscalls/sync/sync03.c: written = tst_dev_bytes_written(tst_device->dev); kernel/syscalls/sync_file_range/sync_file_range02.c: tst_dev_bytes_written(tst_device->dev); kernel/syscalls/sync_file_range/sync_file_range02.c: written = tst_dev_bytes_written(tst_device->dev); kernel/syscalls/syncfs/syncfs01.c: tst_dev_bytes_written(tst_device->dev); kernel/syscalls/syncfs/syncfs01.c: written = tst_dev_bytes_written(tst_device->dev);
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20191231/73f7871b/attachment.htm>
RFC PATCH 1/1 sync_file_range02: remove the toplimit of write back
Quoted 26 lines · Li Wang
Quoted 15 lines · Yang Xu
...
Quoted 10 lines
Quoted 3 lines
I have tried this for many different systems, and it doesn't hit the failure at least one time. So if no more comments, I would merge the patch as below in the next step.
I have a question.we must call
sync()? I think syncfs is more accurate.Here we use
sync()is to make fs metadata/cache being written back before testing because there is no obtainable file descriptor 'fd' for the ext4 deferred IO (e.g. initialize the journal and inode tables).Quoted 3 lines
I see. For other test cases using
tst_dev_bytes_writtenapi such asfdatasync03.c, we also need to callsync()to make this written value more accurate. What do you think about it?Good point. I think we should take care of them as we do for
sync_file_range02too. It will more easily report fail in the case of a situation that synced data is less than expected.
Quoted 12 lines
$ git grep tst_dev_bytes_written kernel/syscalls/fdatasync/fdatasync03.c: tst_dev_bytes_written(tst_device->dev); kernel/syscalls/fdatasync/fdatasync03.c: written = tst_dev_bytes_written(tst_device->dev); kernel/syscalls/fsync/fsync04.c: tst_dev_bytes_written(tst_device->dev); kernel/syscalls/fsync/fsync04.c: written = tst_dev_bytes_written(tst_device->dev); kernel/syscalls/sync/sync03.c: tst_dev_bytes_written(tst_device->dev); kernel/syscalls/sync/sync03.c: written = tst_dev_bytes_written(tst_device->dev); kernel/syscalls/sync_file_range/sync_file_range02.c: tst_dev_bytes_written(tst_device->dev); kernel/syscalls/sync_file_range/sync_file_range02.c: written = tst_dev_bytes_written(tst_device->dev); kernel/syscalls/syncfs/syncfs01.c: tst_dev_bytes_written(tst_device->dev); kernel/syscalls/syncfs/syncfs01.c: written = tst_dev_bytes_written(tst_device->dev);
Agree.
-Sumit
Quoted 3 lines
--
Regards,
Li Wang