Skip to contentSkip to conversation

fio: add support for POSIX_FADV_NOREUSE

PATCH Applied Yuanchu Xie 2 participants 5 messages Mar 31, 2023

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/2 fio: add support for POSIX_FADV_NOREUSE

As of Linux kernel commit 17e810229cb3 ("mm: support POSIX_FADV_NOREUSE"), POSIX_FADV_NOREUSE hints at the LRU algorithm to ignore accesses to mapped files with this flag. Previously, it was a no-op.

Add it in fio as an fadvise_hint option to test the new behavior.

---
 fio.h       | 1 +
 ioengines.c | 2 ++
 options.c   | 4 ++++
 3 files changed, 7 insertions(+)
fio.h +1 -0
index 32535517..f2acd430 100644
@@ -163,6 +163,7 @@ enum {
 	F_ADV_TYPE,
 	F_ADV_RANDOM,
 	F_ADV_SEQUENTIAL,
+	F_ADV_NOREUSE,
 };
 
 /*
ioengines.c +2 -0
index e2316ee4..56759c53 100644
@@ -565,6 +565,8 @@ int td_io_open_file(struct thread_data *td, struct fio_file *f)
 			flags = POSIX_FADV_RANDOM;
 		else if (td->o.fadvise_hint == F_ADV_SEQUENTIAL)
 			flags = POSIX_FADV_SEQUENTIAL;
+		else if (td->o.fadvise_hint == F_ADV_NOREUSE)
+			flags = POSIX_FADV_NOREUSE;
 		else {
 			log_err("fio: unknown fadvise type %d\n",
 							td->o.fadvise_hint);
options.c +4 -0
index 18857795..fe580ebb 100644
@@ -2740,6 +2740,10 @@ struct fio_option fio_options[FIO_MAX_OPTS] = {
 			    .oval = F_ADV_SEQUENTIAL,
 			    .help = "Advise using FADV_SEQUENTIAL",
 			  },
+			  { .ival = "noreuse",
+			    .oval = F_ADV_NOREUSE,
+			    .help = "Advise using FADV_NOREUSE",
+			  },
 		},
 		.help	= "Use fadvise() to advise the kernel on IO pattern",
 		.def	= "1",
Signature · 1 line
2.40.0.348.gf938b09366-goog
#1 lore raw

PATCH v2 2/2 docs: add noreuse fadvise_hint option

noreuse was added as an fadvise_hint option to apply POSIX_FADV_NOREUSE, so we add it to the docs as well.

---
 HOWTO.rst | 5 +++++
 fio.1     | 5 +++++
 2 files changed, 10 insertions(+)
HOWTO.rst +5 -0
index 5240f9da..cb0f9834 100644
@@ -1308,6 +1308,11 @@ I/O type
 		**random**
 			Advise using **FADV_RANDOM**.
 
+		**noreuse**
+			Advise using **FADV_NOREUSE**. This may be a no-op on older Linux
+			kernels. Since Linux 6.3, it provides a hint to the LRU algorithm.
+			See the :manpage:`posix_fadvise(2)` man page.
+
 .. option:: write_hint=str
 
 	Use :manpage:`fcntl(2)` to advise the kernel what life time to expect
fio.1 +5 -0
index e2db3a3f..311b16d8 100644
@@ -1098,6 +1098,11 @@ Advise using FADV_SEQUENTIAL.
 .TP
 .B random
 Advise using FADV_RANDOM.
+.TP
+.B noreuse
+Advise using FADV_NOREUSE. This may be a no-op on older Linux
+kernels. Since Linux 6.3, it provides a hint to the LRU algorithm.
+See the \fBposix_fadvise\fR\|(2) man page.
 .RE
 .RE
 .TP
Signature · 1 line
2.40.0.348.gf938b09366-goog
#2 lore raw
In reply to Yuanchu Xie
Quoted 6 lines · Yuanchu Xie

As of Linux kernel commit 17e810229cb3 ("mm: support POSIX_FADV_NOREUSE"), POSIX_FADV_NOREUSE hints at the LRU algorithm to ignore accesses to mapped files with this flag. Previously, it was a no-op.

Add it in fio as an fadvise_hint option to test the new behavior.

I just applied this, but then I pondered if we'll be breaking normal installs that don't have POSIX_FADV_NOREUSE available? Maybe the libc define predates the kernel actually implementing it by long enough that this doesn't matter? Let's see what happens...

Signature · 1 line
Jens Axboe
#3 lore raw
Quoted 6 lines · Jens Axboe
Quoted 5 lines · Yuanchu Xie

As of Linux kernel commit 17e810229cb3 ("mm: support POSIX_FADV_NOREUSE"), POSIX_FADV_NOREUSE hints at the LRU algorithm to ignore accesses to mapped files with this flag. Previously, it was a no-op.

Quoted 6 lines
Quoted 1 line

Add it in fio as an fadvise_hint option to test the new behavior.

I just applied this, but then I pondered if we'll be breaking normal installs that don't have POSIX_FADV_NOREUSE available? Maybe the libc define predates the kernel actually implementing it by long enough that this doesn't matter? Let's see what happens...

Somewhat unsurprisingly, this did break the build on OSX. I added a patch to make it conditionally available.

Signature · 1 line
Jens Axboe
Applied
#4 lore raw
In reply to Yuanchu Xie
Quoted 8 lines · Yuanchu Xie

As of Linux kernel commit 17e810229cb3 ("mm: support POSIX_FADV_NOREUSE"), POSIX_FADV_NOREUSE hints at the LRU algorithm to ignore accesses to mapped files with this flag. Previously, it was a no-op.

Add it in fio as an fadvise_hint option to test the new behavior.

[...]

Applied, thanks!

[1/2] fio: add support for POSIX_FADV_NOREUSE

      commit: 109aad505302d2924396774f267eca7451f8fb14
[2/2] docs: add noreuse fadvise_hint option
      commit: 109aad505302d2924396774f267eca7451f8fb14

Best regards,

Signature · 1 line
Jens Axboe