Skip to contentSkip to conversation

readahead02: estimate max readahead size

PATCH New Jan Stancek 2 participants 4 messages Oct 10, 2016

lore
Share & tools
lore

Jan Stancek

author
#0 lore raw

PATCH 1/1 readahead02: estimate max readahead size

Max readahead size is kernel implementation detail, which can and already has changed in past (and probably will again [1]). Futher, current (4.8) implementation defines it as block device's read_ahead_kb, which means its value varies based on storage/fs setup.

This patch estimates max readahead size based on cache increase from first readahead call and then advances offset for subsequent calls by that amount. It also makes sure it's > 0 to guarantee, that it eventually reaches end of file.

[1] https://lkml.org/lkml/2016/7/25/308

Signed-off-by Jan Stancek <[email protected]>
---
 testcases/kernel/syscalls/readahead/readahead02.c | 64 ++++++++++-------------
 1 file changed, 29 insertions(+), 35 deletions(-)
testcases/kernel/syscalls/readahead/readahead02.c +29 -35
index 2517a333680e..7fef9fa3fc39 100644
@@ -59,6 +59,8 @@ static int opt_fsize;
 static char *opt_fsizestr;
 static int pagesize;
 
+#define MIN_SANE_READAHEAD (4 * 1024)
+
 option_t options[] = {
 	{"s:", &opt_fsize, &opt_fsizestr},
 	{NULL, NULL, NULL}

@@ -184,26 +186,6 @@ static void create_testfile(void)
 	free(tmp);
 }
 
-static long get_device_readahead(const char *fname)
-{
-	struct stat st;
-	unsigned long ra_kb = 0;
-	char buf[256];
-
-	if (stat(fname, &st) == -1)
-		tst_brkm(TBROK | TERRNO, cleanup, "stat");
-	snprintf(buf, sizeof(buf), "/sys/dev/block/%d:%d/queue/read_ahead_kb",
-		 major(st.st_dev), minor(st.st_dev));
-	if (access(buf, F_OK)) {
-		snprintf(buf, sizeof(buf),
-			"/sys/dev/block/%d:%d/../queue/read_ahead_kb",
-			major(st.st_dev), minor(st.st_dev));
-	}
-	tst_resm(TINFO, "Reading %s", buf);
-	SAFE_FILE_SCANF(cleanup, buf, "%ld", &ra_kb);
-
-	return ra_kb * 1024;
-}
 
 /* read_testfile - mmap testfile and read every page.
  * This functions measures how many I/O and time it takes to fully

@@ -221,32 +203,44 @@ static void read_testfile(int do_readahead, const char *fname, size_t fsize,
 			  unsigned long *cached)
 {
 	int fd;
-	size_t i;
+	size_t i = 0;
 	long read_bytes_start;
 	unsigned char *p, tmp;
 	unsigned long time_start_usec, time_end_usec;
-	off_t offset;
+	unsigned long cached_start, max_ra_estimate = 0;
+	off_t offset = 0;
 	struct timeval now;
-	long readahead_size;
-
-	/* use bdi limit for 4.4 and older, otherwise default to 2M */
-	if ((tst_kvercmp(4, 4, 0)) >= 0)
-		readahead_size = get_device_readahead(fname);
-	else
-		readahead_size = 2 * 1024 * 1024;
-	tst_resm(TINFO, "max readahead size is: %ld", readahead_size);
 
 	fd = open(fname, O_RDONLY);
 	if (fd < 0)
 		tst_brkm(TBROK | TERRNO, cleanup, "Failed to open %s", fname);
 
 	if (do_readahead) {
-		for (i = 0; i < fsize; i += readahead_size) {
-			TEST(readahead(fd, (off64_t) i, readahead_size));
-			if (TEST_RETURN != 0)
+		cached_start = get_cached_size();
+		do {
+			TEST(readahead(fd, offset, fsize - offset));
+			if (TEST_RETURN != 0) {
+				check_ret(0);
 				break;
-		}
-		check_ret(0);
+			}
+
+			/* estimate max readahead size based on first call */
+			if (!max_ra_estimate) {
+				*cached = get_cached_size();
+				if (*cached > cached_start) {
+					max_ra_estimate = ((*cached - cached_start)
+						* 1024 & (~(pagesize - 1)));
+					tst_resm(TINFO, "max ra estimate: %lu",
+						max_ra_estimate);
+				}
+				max_ra_estimate = MAX(max_ra_estimate,
+					MIN_SANE_READAHEAD);
+			}
+
+			i++;
+			offset += max_ra_estimate;
+		} while ((size_t)offset < fsize);
+		tst_resm(TINFO, "readahead calls made: %zu", i);
 		*cached = get_cached_size();
 
 		/* offset of file shouldn't change after readahead */
Signature · 1 line
1.8.3.1
#1 lore raw

PATCH 1/1 readahead02: estimate max readahead size

Hi!

Quoted 5 lines
testcases/kernel/syscalls/readahead/readahead02.c +4 -0
+		do {
+			TEST(readahead(fd, offset, fsize - offset));
+			if (TEST_RETURN != 0) {
+				check_ret(0);
 				break;

testcases/kernel/syscalls/readahead/readahead02.c static void read_testfile(int do_readahead, const char *fname, size_t fsize, · line 232

Quoted 10 lines
testcases/kernel/syscalls/readahead/readahead02.c +8 -2
-		}
-		check_ret(0);
+			}
+
+			/* estimate max readahead size based on first call */
+			if (!max_ra_estimate) {
+				*cached = get_cached_size();
+				if (*cached > cached_start) {
+					max_ra_estimate = ((*cached - cached_start)
+						* 1024 & (~(pagesize - 1)));

I'm just curious, why do we round the value down to pagesize?

I've tried this on the /tmp on Btrfs and got max_ra_estimage nearly ten times greater than the sysfs value. But apart from that the measured times are pretty much comparable with the previous version, which suggests that maximal readahead size on Btrfs is set to ~4M and the limit per device is not applicable.

Did you get the estimate close to the value of the sysfs file in your environment?

Your version:

readahead02    0  TINFO  :  creating test file of size: 67108864
readahead02    0  TINFO  :  read_testfile(0)
readahead02    0  TINFO  :  read_testfile(1)
readahead02    0  TINFO  :  max ra estimate: 4177920
readahead02    0  TINFO  :  readahead calls made: 17
readahead02    1  TPASS  :  offset is still at 0 as expected
readahead02    0  TINFO  :  read_testfile(0) took: 103324 usec
readahead02    0  TINFO  :  read_testfile(1) took: 22291 usec
readahead02    0  TINFO  :  read_testfile(0) read: 67190784 bytes
readahead02    0  TINFO  :  read_testfile(1) read: 0 bytes
readahead02    2  TPASS  :  readahead saved some I/O
readahead02    0  TINFO  :  cache can hold at least: 75140 kB
readahead02    0  TINFO  :  read_testfile(0) used cache: 65592 kB
readahead02    0  TINFO  :  read_testfile(1) used cache: 65592 kB
readahead02    3  TPASS  :  using cache as expected

With my patch for Btrfs:

readahead02    0  TINFO  :  creating test file of size: 67108864
readahead02    0  TINFO  :  Looking for device in '/sys/fs/btrfs/60cb755a-ae5c-4059-ae0f-7a11ee9d0e9d/devices/'
readahead02    0  TINFO  :  Reading /sys/fs/btrfs/60cb755a-ae5c-4059-ae0f-7a11ee9d0e9d/devices//sda2/../queue/read_ahead_kb
readahead02    0  TINFO  :  max readahead size is: 524288
readahead02    0  TINFO  :  read_testfile(0)
readahead02    0  TINFO  :  Looking for device in '/sys/fs/btrfs/60cb755a-ae5c-4059-ae0f-7a11ee9d0e9d/devices/'
readahead02    0  TINFO  :  Reading /sys/fs/btrfs/60cb755a-ae5c-4059-ae0f-7a11ee9d0e9d/devices//sda2/../queue/read_ahead_kb
readahead02    0  TINFO  :  max readahead size is: 524288
readahead02    0  TINFO  :  read_testfile(1)
readahead02    0  TINFO  :  Looking for device in '/sys/fs/btrfs/60cb755a-ae5c-4059-ae0f-7a11ee9d0e9d/devices/'
readahead02    0  TINFO  :  Reading /sys/fs/btrfs/60cb755a-ae5c-4059-ae0f-7a11ee9d0e9d/devices//sda2/../queue/read_ahead_kb
readahead02    0  TINFO  :  max readahead size is: 524288
readahead02    1  TPASS  :  expected ret success - returned value = 0
readahead02    2  TPASS  :  offset is still at 0 as expected
readahead02    0  TINFO  :  read_testfile(0) took: 117244 usec
readahead02    0  TINFO  :  read_testfile(1) took: 34219 usec
readahead02    0  TINFO  :  read_testfile(0) read: 67190784 bytes
readahead02    0  TINFO  :  read_testfile(1) read: 0 bytes
readahead02    3  TPASS  :  readahead saved some I/O
readahead02    0  TINFO  :  cache can hold at least: 67068 kB
readahead02    0  TINFO  :  read_testfile(0) used cache: 65592 kB
readahead02    0  TINFO  :  read_testfile(1) used cache: 65592 kB
readahead02    4  TPASS  :  using cache as expected

Btw the test should probably be cleaned up a bit, there are couple of places that could use SAFE_FILE_SCANF/PRINTF and it should also use monotonic timers instead of gettimeofday as well...

Signature · 2 lines
Cyril Hrubis
[email protected]
author
#2 lore raw

PATCH 1/1 readahead02: estimate max readahead size

----- Original Message -----

Quoted 16 lines

From: "Cyril Hrubis" <[email protected]>
To: "Jan Stancek" <[email protected]>
Cc: [email protected]
Sent: Monday, 10 October, 2016 4:16:52 PM
Subject: Re: [PATCH] readahead02: estimate max readahead size

Hi!

Quoted 9 lines
testcases/kernel/syscalls/readahead/readahead02.c +6 -2
+		do {
+			TEST(readahead(fd, offset, fsize - offset));
+			if (TEST_RETURN != 0) {
+				check_ret(0);
 				break;
-		}
-		check_ret(0);
+			}
+
Quoted 8 lines

testcases/kernel/syscalls/readahead/readahead02.c static void read_testfile(int do_readahead, const char *fname, size_t fsize, · line 232

Quoted 6 lines
testcases/kernel/syscalls/readahead/readahead02.c +6 -0
+			/* estimate max readahead size based on first call */
+			if (!max_ra_estimate) {
+				*cached = get_cached_size();
+				if (*cached > cached_start) {
+					max_ra_estimate = ((*cached - cached_start)
+						* 1024 & (~(pagesize - 1)));

I'm just curious, why do we round the value down to pagesize?

Not sure why I left it in, my previous approach was checking increase in each loop, but that had issues, because it varied a lot among iteration (whole system contributes to that number).
Should be safe to drop.

Quoted 10 lines

I've tried this on the /tmp on Btrfs and got max_ra_estimage nearly ten times greater than the sysfs value. But apart from that the measured times are pretty
much comparable with the previous version, which suggests that maximal readahead size on Btrfs is set to ~4M and the limit per device is not applicable.

Quoted 2 lines

Did you get the estimate close to the value of the sysfs file in your environment?

I do (on 4.8 x86 KVM guest):

# cat /sys/devices/virtual/bdi/253:1/read_ahead_kb
4096
readahead02    0  TINFO  :  max ra estimate: 4145152

Are you using upstream kernel or a patched one?

Regards,
Jan

Quoted 62 lines

Your version:

readahead02    0  TINFO  :  creating test file of size: 67108864
readahead02    0  TINFO  :  read_testfile(0)
readahead02    0  TINFO  :  read_testfile(1)
readahead02    0  TINFO  :  max ra estimate: 4177920
readahead02    0  TINFO  :  readahead calls made: 17
readahead02    1  TPASS  :  offset is still at 0 as expected
readahead02    0  TINFO  :  read_testfile(0) took: 103324 usec
readahead02    0  TINFO  :  read_testfile(1) took: 22291 usec
readahead02    0  TINFO  :  read_testfile(0) read: 67190784 bytes
readahead02    0  TINFO  :  read_testfile(1) read: 0 bytes
readahead02    2  TPASS  :  readahead saved some I/O
readahead02    0  TINFO  :  cache can hold at least: 75140 kB
readahead02    0  TINFO  :  read_testfile(0) used cache: 65592 kB
readahead02    0  TINFO  :  read_testfile(1) used cache: 65592 kB
readahead02    3  TPASS  :  using cache as expected

With my patch for Btrfs:

readahead02    0  TINFO  :  creating test file of size: 67108864
readahead02    0  TINFO  :  Looking for device in
'/sys/fs/btrfs/60cb755a-ae5c-4059-ae0f-7a11ee9d0e9d/devices/'
readahead02    0  TINFO  :  Reading
/sys/fs/btrfs/60cb755a-ae5c-4059-ae0f-7a11ee9d0e9d/devices//sda2/../queue/read_ahead_kb
readahead02    0  TINFO  :  max readahead size is: 524288
readahead02    0  TINFO  :  read_testfile(0)
readahead02    0  TINFO  :  Looking for device in
'/sys/fs/btrfs/60cb755a-ae5c-4059-ae0f-7a11ee9d0e9d/devices/'
readahead02    0  TINFO  :  Reading
/sys/fs/btrfs/60cb755a-ae5c-4059-ae0f-7a11ee9d0e9d/devices//sda2/../queue/read_ahead_kb
readahead02    0  TINFO  :  max readahead size is: 524288
readahead02    0  TINFO  :  read_testfile(1)
readahead02    0  TINFO  :  Looking for device in
'/sys/fs/btrfs/60cb755a-ae5c-4059-ae0f-7a11ee9d0e9d/devices/'
readahead02    0  TINFO  :  Reading
/sys/fs/btrfs/60cb755a-ae5c-4059-ae0f-7a11ee9d0e9d/devices//sda2/../queue/read_ahead_kb
readahead02    0  TINFO  :  max readahead size is: 524288
readahead02    1  TPASS  :  expected ret success - returned value = 0
readahead02    2  TPASS  :  offset is still at 0 as expected
readahead02    0  TINFO  :  read_testfile(0) took: 117244 usec
readahead02    0  TINFO  :  read_testfile(1) took: 34219 usec
readahead02    0  TINFO  :  read_testfile(0) read: 67190784 bytes
readahead02    0  TINFO  :  read_testfile(1) read: 0 bytes
readahead02    3  TPASS  :  readahead saved some I/O
readahead02    0  TINFO  :  cache can hold at least: 67068 kB
readahead02    0  TINFO  :  read_testfile(0) used cache: 65592 kB
readahead02    0  TINFO  :  read_testfile(1) used cache: 65592 kB
readahead02    4  TPASS  :  using cache as expected

Btw the test should probably be cleaned up a bit, there are couple of places that could use SAFE_FILE_SCANF/PRINTF and it should also use monotonic timers instead of gettimeofday as well...

--
Cyril Hrubis
[email protected]

#3 lore raw

PATCH 1/1 readahead02: estimate max readahead size

Hi!

Quoted 10 lines
Quoted 9 lines

I've tried this on the /tmp on Btrfs and got max_ra_estimage nearly ten times greater than the sysfs value. But apart from that the measured times are pretty
much comparable with the previous version, which suggests that maximal readahead size on Btrfs is set to ~4M and the limit per device is not applicable.

Did you get the estimate close to the value of the sysfs file in your environment?

Quoted 6 lines

I do (on 4.8 x86 KVM guest):

# cat /sys/devices/virtual/bdi/253:1/read_ahead_kb
4096
readahead02    0  TINFO  :  max ra estimate: 4145152

Are you using upstream kernel or a patched one?

That is Btrfs on SLES12-SP2 RC with kernel 4.4.21, but it seems to be pretty much Btrfs specific. If I point TMPDIR to ext3 fs I got estimated ra 507904 while the read_ahead_kb for the device is set to 512.

Signature · 2 lines
Cyril Hrubis
[email protected]