Skip to contentSkip to conversation

Introduce a page_fault ioengine for MM workflows

PATCH Superseded Nico Pache 3 participants 7 messages Mar 11 – Mar 12 Kbuild

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Nico Pache

author
#0 lore raw

v2 cover Introduce a page_fault ioengine for MM workflows

This series introduces a new page_fault ioengine for Anonymous memory testing. This enables using fio’s existing framework and job files for memory management style workloads without relying on a filesystem. An example job file is included to demonstrate usage.

The first patch adds the mmap‑backed ioengine that allocates anonymous memory and copies data on read/write to intentionally trigger faults.

The second patch updates the documentation to guide usage of the new feature.

V2:
- drop the khugepaged_delay, and all other additional features. This provides a clean initial implmentation. Further features will be added in future series as we keep testing and discover what will be useful for functional and performance testing.
- Changed the byte-by-byte r/w to leverage memcpy
- provide a number of test cases
- hardened the engine by limiting what parameters it can accept, preventing undefined or broken behavior
- Use clang-format to adhere to formatting conventions

Nico Pache (2):
page_fault: add mmap-backed ioengine for anonymous faults
Documentation: update the HOWTO to include the page_fault engine

 HOWTO.rst               |   5 ++
 Makefile                |   2 +-
 engines/page_fault.c    | 116 ++++++++++++++++++++++++++++++++++++++++
 examples/page_fault.fio |  60 +++++++++++++++++++++
 4 files changed, 182 insertions(+), 1 deletion(-)
 create mode 100644 engines/page_fault.c
 create mode 100644 examples/page_fault.fio
Signature · 1 line
2.53.0
author
#1 lore raw

PATCH v2 1/2 page_fault: add mmap-backed ioengine for anonymous faults

Introduce a new ioengine that mmaps anonymous memory and copies data on read/write to trigger page faults. This allows us to leverage FIOs powerful framework for MM related testing, and will ideally allow us to quickly expand testing, by leveraging previously FS related fio scripts.

---
 Makefile                |   2 +-
 engines/page_fault.c    | 116 ++++++++++++++++++++++++++++++++++++++++
 examples/page_fault.fio |  60 +++++++++++++++++++++
 3 files changed, 177 insertions(+), 1 deletion(-)
 create mode 100644 engines/page_fault.c
 create mode 100644 examples/page_fault.fio
Makefile +1 -1
index 0337e8fe..099e2f94 100644
@@ -57,7 +57,7 @@ SOURCE :=	$(sort $(patsubst $(SRCDIR)/%,%,$(wildcard $(SRCDIR)/crc/*.c)) \
 		smalloc.c filehash.c profile.c debug.c engines/cpu.c \
 		engines/mmap.c engines/sync.c engines/null.c engines/net.c \
 		engines/ftruncate.c engines/fileoperations.c \
-		engines/exec.c \
+		engines/exec.c engines/page_fault.c \
 		server.c client.c iolog.c backend.c libfio.c flow.c cconv.c \
 		gettime-thread.c helpers.c json.c idletime.c td_error.c \
 		profiles/tiobench.c profiles/act.c io_u_queue.c filelock.c \
engines/page_fault.c new file +116 -0

117 lines not on this page · raw message

examples/page_fault.fio new file +60 -0

60 lines not on this page · raw message

Signature · 1 line
2.53.0
#2 lore raw

Re: [PATCH v2 1/2] page_fault: add mmap-backed ioengine for anonymous faults

Quoted 40 lines · Nico Pache

Introduce a new ioengine that mmaps anonymous memory and copies data on read/write to trigger page faults. This allows us to leverage FIOs powerful framework for MM related testing, and will ideally allow us to quickly expand testing, by leveraging previously FS related fio scripts.

---
 Makefile                |   2 +-
 engines/page_fault.c    | 116 ++++++++++++++++++++++++++++++++++++++++
 examples/page_fault.fio |  60 +++++++++++++++++++++
 3 files changed, 177 insertions(+), 1 deletion(-)
 create mode 100644 engines/page_fault.c
 create mode 100644 examples/page_fault.fio
Makefile +1 -1
index 0337e8fe..099e2f94 100644
@@ -57,7 +57,7 @@ SOURCE :=     $(sort $(patsubst $(SRCDIR)/%,%,$(wildcard $(SRCDIR)/crc/*.c)) \
                smalloc.c filehash.c profile.c debug.c engines/cpu.c \
                engines/mmap.c engines/sync.c engines/null.c engines/net.c \
                engines/ftruncate.c engines/fileoperations.c \
-               engines/exec.c \
+               engines/exec.c engines/page_fault.c \
                server.c client.c iolog.c backend.c libfio.c flow.c cconv.c \
                gettime-thread.c helpers.c json.c idletime.c td_error.c \
                profiles/tiobench.c profiles/act.c io_u_queue.c filelock.c \
engines/page_fault.c new file +6 -0
new file mode 100644
index 00000000..88108644
@@ -0,0 +1,116 @@
+/*
+ * page_fault engine
+ *
+ * IO engine that reads/writes directly to/from anonymous memory
+ * by triggering page faults.
+ */

engines/page_fault.c line 16

Quoted 10 lines
engines/page_fault.c new file +10 -0
+#include "fio.h"
+#include "ioengines.h"
+#include <sys/mman.h>
+
+struct fio_page_fault_data {
+  void *mmap_ptr;
+  size_t mmap_sz;
+};
+
+static int fio_page_fault_init(struct thread_data *td) {

The style we follow is to put the { on a new line. Also please use tabs. We basically follow the kernel's style conventions. Did you use the default LLVM conventions with clang-format?

engines/page_fault.c line 21

Quoted 5 lines
engines/page_fault.c new file +5 -0
+  size_t total_io_size;
+  struct fio_page_fault_data *fpd;
+
+  if (td->o.nr_files > 1) {
+    log_err("page_fault engine does not support multiple files\n");

How about, "fio: page_fault ioengine ..."?

engines/page_fault.c line 23

Quoted 2 lines
engines/page_fault.c new file +2 -0
+    return 1;
+  }

Consider checking that td->o.offset == 0 and emitting an error message saying that the offset option must be zero for this ioengine to work.

Quoted 6 lines
engines/page_fault.c new file +6 -0
+
+  fpd = calloc(1, sizeof(*fpd));
+  if (!fpd)
+    return 1;
+
+  total_io_size = td->o.size;

engines/page_fault.c line 39

Quoted 10 lines
engines/page_fault.c new file +10 -0
+  fpd->mmap_sz = total_io_size;
+  fpd->mmap_ptr = mmap(NULL, total_io_size, PROT_READ | PROT_WRITE,
+                       MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+  if (fpd->mmap_ptr == MAP_FAILED) {
+    free(fpd);
+    return 1;
+  }
+
+  td->io_ops_data = fpd;
+  FILE_SET_ENG_DATA(td->files[0], fpd);

I would avoid directly manipulating files[] here. Since there is only a single instance of fpd, why not omit this and just access fpd via td->io_ops_data in queue()?

Vincent

author
#3 lore raw

Re: [PATCH v2 1/2] page_fault: add mmap-backed ioengine for anonymous faults

Quoted 48 lines · Vincent Fu
Quoted 46 lines · Nico Pache

Introduce a new ioengine that mmaps anonymous memory and copies data on read/write to trigger page faults. This allows us to leverage FIOs powerful framework for MM related testing, and will ideally allow us to quickly expand testing, by leveraging previously FS related fio scripts.

---
 Makefile                |   2 +-
 engines/page_fault.c    | 116 ++++++++++++++++++++++++++++++++++++++++
 examples/page_fault.fio |  60 +++++++++++++++++++++
 3 files changed, 177 insertions(+), 1 deletion(-)
 create mode 100644 engines/page_fault.c
 create mode 100644 examples/page_fault.fio
Makefile +1 -1
index 0337e8fe..099e2f94 100644
@@ -57,7 +57,7 @@ SOURCE :=     $(sort $(patsubst $(SRCDIR)/%,%,$(wildcard $(SRCDIR)/crc/*.c)) \
                smalloc.c filehash.c profile.c debug.c engines/cpu.c \
                engines/mmap.c engines/sync.c engines/null.c engines/net.c \
                engines/ftruncate.c engines/fileoperations.c \
-               engines/exec.c \
+               engines/exec.c engines/page_fault.c \
                server.c client.c iolog.c backend.c libfio.c flow.c cconv.c \
                gettime-thread.c helpers.c json.c idletime.c td_error.c \
                profiles/tiobench.c profiles/act.c io_u_queue.c filelock.c \
engines/page_fault.c new file +12 -0
new file mode 100644
index 00000000..88108644
@@ -0,0 +1,116 @@
+/*
+ * page_fault engine
+ *
+ * IO engine that reads/writes directly to/from anonymous memory
+ * by triggering page faults.
+ */
+#include "fio.h"
+#include "ioengines.h"
+#include <sys/mman.h>
+
+struct fio_page_fault_data {
+  void *mmap_ptr;
Quoted 8 lines

engines/page_fault.c line 16

Quoted 4 lines
engines/page_fault.c new file +4 -0
+  size_t mmap_sz;
+};
+
+static int fio_page_fault_init(struct thread_data *td) {

The style we follow is to put the { on a new line. Also please use tabs. We basically follow the kernel's style conventions. Did you use the default LLVM conventions with clang-format?

Hmm perhaps I called clang-format incorrectly. I assumed it would read the format file automatically but that is not the case. Sorry, first time using the clang-format function.

Quoted 8 lines

engines/page_fault.c line 21

Quoted 6 lines
engines/page_fault.c new file +5 -0
+  size_t total_io_size;
+  struct fio_page_fault_data *fpd;
+
+  if (td->o.nr_files > 1) {
+    log_err("page_fault engine does not support multiple files\n");

How about, "fio: page_fault ioengine ..."?

Sounds good, yeah I wasnt thinking about the reporting format you guys use. Ill keep that in mind.

Quoted 6 lines

engines/page_fault.c line 23

Quoted 3 lines
engines/page_fault.c new file +2 -0
+    return 1;
+  }

Consider checking that td->o.offset == 0 and emitting an error message saying that the offset option must be zero for this ioengine to work.

Hmm, I didnt even notice, but while doing my cleanup and ripping out the other features I added I accidently removed the offset...

Ill add that back. Thanks :)

Quoted 13 lines
Quoted 13 lines
engines/page_fault.c new file +12 -0
+
+  fpd = calloc(1, sizeof(*fpd));
+  if (!fpd)
+    return 1;
+
+  total_io_size = td->o.size;
+  fpd->mmap_sz = total_io_size;
+  fpd->mmap_ptr = mmap(NULL, total_io_size, PROT_READ | PROT_WRITE,
+                       MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+  if (fpd->mmap_ptr == MAP_FAILED) {
+    free(fpd);
+    return 1;
Quoted 8 lines

engines/page_fault.c line 39

Quoted 4 lines
engines/page_fault.c new file +4 -0
+  }
+
+  td->io_ops_data = fpd;
+  FILE_SET_ENG_DATA(td->files[0], fpd);

I would avoid directly manipulating files[] here. Since there is only a single instance of fpd, why not omit this and just access fpd via td->io_ops_data in queue()?

Yeah I had to switch to using io_ops_data directly and use the cleanup hook (rather than close_file) because of a bug I found, but now it just looks messy. Ill clean it up as you suggested! thanks.

Cheers,
-- Nico

Quoted 3 lines

Vincent

author
#4 lore raw

PATCH v2 2/2 Documentation: update the HOWTO to include the page_fault engine

In reply to Nico Pache

Document the new page fault engine.

---
 HOWTO.rst | 5 +++++
 1 file changed, 5 insertions(+)
HOWTO.rst +5 -0
index e712ee2e..421e6408 100644
@@ -2376,6 +2376,11 @@ I/O engine
 			several instances to access the same device or file
 			simultaneously, but allow it for threads.
 
+		**page_fault**
+			I/O engine that uses an mmap region to simulate data transfer by
+			allocating anonymous memory and copying data on read/write to
+			intentionally trigger page faults.
+
 	File/directory operation engines define how the job operates file or directory. The
 	following types are defined:
 
Signature · 1 line
2.53.0
#5 lore raw

Re: [PATCH v2 2/2] Documentation: update the HOWTO to include the page_fault engine

Quoted 25 lines · Nico Pache

Document the new page fault engine.

---
 HOWTO.rst | 5 +++++
 1 file changed, 5 insertions(+)
HOWTO.rst +5 -1
index e712ee2e..421e6408 100644
@@ -2376,6 +2376,11 @@ I/O engine
                        several instances to access the same device or file
                        simultaneously, but allow it for threads.
 
+               **page_fault**
+                       I/O engine that uses an mmap region to simulate data transfer by
+                       allocating anonymous memory and copying data on read/write to
+                       intentionally trigger page faults.
+
        File/directory operation engines define how the job operates file or directory. The
        following types are defined:
 
--
Quoted 2 lines

2.53.0

Please also make the same change to fio.1

Vincent