Skip to contentSkip to conversation

readahead02.c: Use fsync instead of sync

PATCH In review Wei Gao via ltp 3 participants 6 messages Jan 16, 2023 – Jan 17, 2023

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 readahead02.c: Use fsync instead of sync

Use fsync on test file instead of sync should faster than syncing whole system.

---
 .../kernel/syscalls/readahead/readahead02.c     | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)
testcases/kernel/syscalls/readahead/readahead02.c +13 -4
index 7acf4bb18..e04046bc3 100644
@@ -99,6 +99,17 @@ static void drop_caches(void)
 	SAFE_FILE_PRINTF(DROP_CACHES_FNAME, "1");
 }
 
+static void sync_drop_caches(void)
+{
+	int fd;
+
+	fd  = SAFE_OPEN(testfile, O_RDONLY);
+	if (fsync(fd) == -1)
+		tst_brk(TBROK | TERRNO, "fsync()");
+	SAFE_CLOSE(fd);
+	drop_caches();
+}
+
 static unsigned long get_bytes_read(void)
 {
 	unsigned long ret;
@@ -233,8 +244,7 @@ static void test_readahead(unsigned int n)
 	read_testfile(tc, 0, testfile, testfile_size, &read_bytes, &usec,
 		      &cached);
 	cached_high = get_cached_size();
-	sync();
-	drop_caches();
+	sync_drop_caches();
 	cached_low = get_cached_size();
 	cached_max = MAX(cached_max, cached_high - cached_low);
 
@@ -246,8 +256,7 @@ static void test_readahead(unsigned int n)
 	else
 		cached = 0;
 
-	sync();
-	drop_caches();
+	sync_drop_caches();
 	cached_low = get_cached_size();
 	tst_res(TINFO, "read_testfile(1)");
 	ret = read_testfile(tc, 1, testfile, testfile_size, &read_bytes_ra,
Signature · 5 lines
2.35.3


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#1 lore raw

Hello,

Quoted 2 lines · Wei Gao via ltp

Use fsync on test file instead of sync should faster than syncing whole system.

The test completes in less than a second in OpenQA. We don't want to risk introducing a regression or spend time reviewing changes unless the performance improvement solves a timeout.

I suggest you convert dup06 to the new API (for example) or investigate a test failure.

Quoted 50 lines
---
 .../kernel/syscalls/readahead/readahead02.c     | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)
testcases/kernel/syscalls/readahead/readahead02.c +13 -4
index 7acf4bb18..e04046bc3 100644
@@ -99,6 +99,17 @@ static void drop_caches(void)
 	SAFE_FILE_PRINTF(DROP_CACHES_FNAME, "1");
 }
 
+static void sync_drop_caches(void)
+{
+	int fd;
+
+	fd  = SAFE_OPEN(testfile, O_RDONLY);
+	if (fsync(fd) == -1)
+		tst_brk(TBROK | TERRNO, "fsync()");
+	SAFE_CLOSE(fd);
+	drop_caches();
+}
+
 static unsigned long get_bytes_read(void)
 {
 	unsigned long ret;
@@ -233,8 +244,7 @@ static void test_readahead(unsigned int n)
 	read_testfile(tc, 0, testfile, testfile_size, &read_bytes, &usec,
 		      &cached);
 	cached_high = get_cached_size();
-	sync();
-	drop_caches();
+	sync_drop_caches();
 	cached_low = get_cached_size();
 	cached_max = MAX(cached_max, cached_high - cached_low);
 
@@ -246,8 +256,7 @@ static void test_readahead(unsigned int n)
 	else
 		cached = 0;
 
-	sync();
-	drop_caches();
+	sync_drop_caches();
 	cached_low = get_cached_size();
 	tst_res(TINFO, "read_testfile(1)");
 	ret = read_testfile(tc, 1, testfile, testfile_size, &read_bytes_ra,
Signature · 1 line
2.35.3
Signature · 5 lines
Thank you,
Richard.

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#2 lore raw
Quoted 7 lines · Richard Palethorpe

Hello,

Quoted 2 lines · Wei Gao via ltp

Use fsync on test file instead of sync should faster than syncing whole system.

Quoted 7 lines

The test completes in less than a second in OpenQA. We don't want to risk introducing a regression or spend time reviewing changes unless the performance improvement solves a timeout.

I suggest you convert dup06 to the new API (for example) or investigate a test failure.

The motivation of this change is base on the https://github.com/linux-test-project/ltp/issues/972
which give following suggestion:
"As we run the test inside a loop device I guess that we can also
sync and drop caches just for the device, which should be faster
than syncing and dropping the whole system. Possibly we just need
to umount it and mount it again."

But currently i can not find any API to sync and drop caches just ONLY for device, so base my view just replace sync whole system to single file also can make a small help.

Quoted 55 lines
Quoted 50 lines
---
 .../kernel/syscalls/readahead/readahead02.c     | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)
testcases/kernel/syscalls/readahead/readahead02.c +13 -4
index 7acf4bb18..e04046bc3 100644
@@ -99,6 +99,17 @@ static void drop_caches(void)
 	SAFE_FILE_PRINTF(DROP_CACHES_FNAME, "1");
 }
 
+static void sync_drop_caches(void)
+{
+	int fd;
+
+	fd  = SAFE_OPEN(testfile, O_RDONLY);
+	if (fsync(fd) == -1)
+		tst_brk(TBROK | TERRNO, "fsync()");
+	SAFE_CLOSE(fd);
+	drop_caches();
+}
+
 static unsigned long get_bytes_read(void)
 {
 	unsigned long ret;
@@ -233,8 +244,7 @@ static void test_readahead(unsigned int n)
 	read_testfile(tc, 0, testfile, testfile_size, &read_bytes, &usec,
 		      &cached);
 	cached_high = get_cached_size();
-	sync();
-	drop_caches();
+	sync_drop_caches();
 	cached_low = get_cached_size();
 	cached_max = MAX(cached_max, cached_high - cached_low);
 
@@ -246,8 +256,7 @@ static void test_readahead(unsigned int n)
 	else
 		cached = 0;
 
-	sync();
-	drop_caches();
+	sync_drop_caches();
 	cached_low = get_cached_size();
 	tst_res(TINFO, "read_testfile(1)");
 	ret = read_testfile(tc, 1, testfile, testfile_size, &read_bytes_ra,
Signature · 1 line
2.35.3
Signature · 2 lines
Thank you,
Richard.
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#3 lore raw

Hello,

Quoted 15 lines · Wei Gao
Quoted 7 lines · Richard Palethorpe

Hello,

Quoted 2 lines · Wei Gao via ltp

Use fsync on test file instead of sync should faster than syncing whole system.

Quoted 7 lines

The test completes in less than a second in OpenQA. We don't want to risk introducing a regression or spend time reviewing changes unless the performance improvement solves a timeout.

I suggest you convert dup06 to the new API (for example) or investigate a test failure.

Quoted 6 lines

The motivation of this change is base on the https://github.com/linux-test-project/ltp/issues/972
which give following suggestion:
"As we run the test inside a loop device I guess that we can also
sync and drop caches just for the device, which should be faster
than syncing and dropping the whole system. Possibly we just need
to umount it and mount it again."

I see. Well unless Cyril can show that the test is actually failing somewhere (or there is a strong logical argument this will cause a failure). Then this task is still valid, but low priority IMO.

Quoted 4 lines

But currently i can not find any API to sync and drop caches just ONLY for device, so base my view just replace sync whole system to single file also can make a small help.

If we don't have one or more concrete failures to focus on then we really have to research whether fsync (or syncfs FYI) or unmounting the device are the correct thing to do. They will all have subtly different effects.

Quoted 56 lines
Quoted 56 lines
Quoted 51 lines
---
 .../kernel/syscalls/readahead/readahead02.c     | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)
testcases/kernel/syscalls/readahead/readahead02.c +13 -4
index 7acf4bb18..e04046bc3 100644
@@ -99,6 +99,17 @@ static void drop_caches(void)
 	SAFE_FILE_PRINTF(DROP_CACHES_FNAME, "1");
 }
 
+static void sync_drop_caches(void)
+{
+	int fd;
+
+	fd  = SAFE_OPEN(testfile, O_RDONLY);
+	if (fsync(fd) == -1)
+		tst_brk(TBROK | TERRNO, "fsync()");
+	SAFE_CLOSE(fd);
+	drop_caches();
+}
+
 static unsigned long get_bytes_read(void)
 {
 	unsigned long ret;
@@ -233,8 +244,7 @@ static void test_readahead(unsigned int n)
 	read_testfile(tc, 0, testfile, testfile_size, &read_bytes, &usec,
 		      &cached);
 	cached_high = get_cached_size();
-	sync();
-	drop_caches();
+	sync_drop_caches();
 	cached_low = get_cached_size();
 	cached_max = MAX(cached_max, cached_high - cached_low);
 
@@ -246,8 +256,7 @@ static void test_readahead(unsigned int n)
 	else
 		cached = 0;
 
-	sync();
-	drop_caches();
+	sync_drop_caches();
 	cached_low = get_cached_size();
 	tst_res(TINFO, "read_testfile(1)");
 	ret = read_testfile(tc, 1, testfile, testfile_size, &read_bytes_ra,
Signature · 1 line
2.35.3
Signature · 2 lines
Thank you,
Richard.
Signature · 5 lines
Thank you,
Richard.

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#4 lore raw

Hi!

Quoted 10 lines
Quoted 6 lines

The motivation of this change is base on the https://github.com/linux-test-project/ltp/issues/972
which give following suggestion:
"As we run the test inside a loop device I guess that we can also
sync and drop caches just for the device, which should be faster
than syncing and dropping the whole system. Possibly we just need
to umount it and mount it again."

I see. Well unless Cyril can show that the test is actually failing somewhere (or there is a strong logical argument this will cause a failure). Then this task is still valid, but low priority IMO.

We do sync more than needed here, since we are looking at the per device counters we have to sync just the device we mount for the test, so this is optimization for the case that the system has many dirty cases and will need seconds or a minute to write them to the pernament storage.

Quoted 8 lines
Quoted 3 lines

But currently i can not find any API to sync and drop caches just ONLY for device, so base my view just replace sync whole system to single file also can make a small help.

If we don't have one or more concrete failures to focus on then we really have to research whether fsync (or syncfs FYI) or unmounting the device are the correct thing to do. They will all have subtly different effects.

Looking at the code closely I'm starting to think that the sync is not required at all. What we do in the test is that we create file and sync it to the external storage. Then we read it a few times and mesure differences in cache. As far as I can tell we just need to drop the page cache after we have read the file. What do you think?

In any case I would avoid changing the test before the release, but it's certainly something we can look at after that.

Signature · 5 lines
Cyril Hrubis
[email protected]

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#5 lore raw

Hello,

Quoted 26 lines · Cyril Hrubis

Hi!

Quoted 10 lines
Quoted 6 lines

The motivation of this change is base on the https://github.com/linux-test-project/ltp/issues/972
which give following suggestion:
"As we run the test inside a loop device I guess that we can also
sync and drop caches just for the device, which should be faster
than syncing and dropping the whole system. Possibly we just need
to umount it and mount it again."

I see. Well unless Cyril can show that the test is actually failing somewhere (or there is a strong logical argument this will cause a failure). Then this task is still valid, but low priority IMO.

We do sync more than needed here, since we are looking at the per device counters we have to sync just the device we mount for the test, so this is optimization for the case that the system has many dirty cases and will need seconds or a minute to write them to the pernament storage.

Quoted 8 lines
Quoted 3 lines

But currently i can not find any API to sync and drop caches just ONLY for device, so base my view just replace sync whole system to single file also can make a small help.

If we don't have one or more concrete failures to focus on then we really have to research whether fsync (or syncfs FYI) or unmounting the device are the correct thing to do. They will all have subtly different effects.

Quoted 8 lines

Looking at the code closely I'm starting to think that the sync is not required at all. What we do in the test is that we create file and sync it to the external storage. Then we read it a few times and mesure differences in cache. As far as I can tell we just need to drop the page cache after we have read the file. What do you think?

In any case I would avoid changing the test before the release, but it's certainly something we can look at after that.

I still think same as before. It may be valid to drop sync or whatever, but it's just not important compared to actively failing tests.

Signature · 5 lines
Thank you,
Richard.

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp