Skip to contentSkip to conversation

page_fault: add mmap-backed ioengine for anonymous faults

PATCH Applied Nico Pache 3 participants 6 messages Apr 8 Kbuild

lore
Share & tools
lore

Nico Pache

author
#0 lore raw

v3 cover page_fault: add mmap-backed ioengine for anonymous faults

This series introduces a new page_fault ioengine for Anonymous memory testing. This enables using fio’s existing framework and job files for memory management style workloads without relying on a filesystem. An example job file is included to demonstrate usage and lays the groundwork for how we plan on utilizing fio to test a number of MM related workloads.

The first patch adds the mmap‑backed ioengine that allocates anonymous memory and copies data on read/write to intentionally trigger faults.

The second patch updates the documentation to guide usage of the new feature.

V3:
- Correctly use clang-format to adhere to formatting conventions
- check (and fail/warn) if user sets the start_offset
- directly utilize io_ops_data as there is no files
- updated error/print to follow proper format
- Copy docs update to fio.1

V2: https://lore.kernel.org/fio/[email protected]/
V1: https://lore.kernel.org/fio/[email protected]/

Nico Pache (2):
page_fault: add mmap-backed ioengine for anonymous faults
Documentation: update the documentation to include the page_fault
engine

 HOWTO.rst               |   5 ++
 Makefile                |   2 +-
 engines/page_fault.c    | 125 ++++++++++++++++++++++++++++++++++++++++
 examples/page_fault.fio |  60 +++++++++++++++++++
 fio.1                   |   5 ++
 5 files changed, 196 insertions(+), 1 deletion(-)
 create mode 100644 engines/page_fault.c
 create mode 100644 examples/page_fault.fio
Signature · 1 line
2.53.0
author
#1 lore raw

PATCH v3 1/2 page_fault: add mmap-backed ioengine for anonymous faults

Introduce a new ioengine that mmaps anonymous memory and copies data on read/write to trigger page faults. This allows us to leverage FIOs powerful framework for MM related testing, and will ideally allow us to quickly expand testing, by leveraging previously FS related fio scripts.

---
 Makefile                |   2 +-
 engines/page_fault.c    | 125 ++++++++++++++++++++++++++++++++++++++++
 examples/page_fault.fio |  60 +++++++++++++++++++
 3 files changed, 186 insertions(+), 1 deletion(-)
 create mode 100644 engines/page_fault.c
 create mode 100644 examples/page_fault.fio
Makefile +1 -1
index 0337e8fe..099e2f94 100644
@@ -57,7 +57,7 @@ SOURCE :=	$(sort $(patsubst $(SRCDIR)/%,%,$(wildcard $(SRCDIR)/crc/*.c)) \
 		smalloc.c filehash.c profile.c debug.c engines/cpu.c \
 		engines/mmap.c engines/sync.c engines/null.c engines/net.c \
 		engines/ftruncate.c engines/fileoperations.c \
-		engines/exec.c \
+		engines/exec.c engines/page_fault.c \
 		server.c client.c iolog.c backend.c libfio.c flow.c cconv.c \
 		gettime-thread.c helpers.c json.c idletime.c td_error.c \
 		profiles/tiobench.c profiles/act.c io_u_queue.c filelock.c \
engines/page_fault.c new file +125 -0

126 lines not on this page · raw message

examples/page_fault.fio new file +60 -0

60 lines not on this page · raw message

Signature · 1 line
2.53.0
author
#2 lore raw

PATCH v3 2/2 Documentation: update the documentation to include the page_fault engine

In reply to Nico Pache

Document the new page fault engine.

---
 HOWTO.rst | 5 +++++
 fio.1     | 5 +++++
 2 files changed, 10 insertions(+)
HOWTO.rst +5 -0
index 5e3266df..bb97ee22 100644
@@ -2379,6 +2379,11 @@ I/O engine
 			several instances to access the same device or file
 			simultaneously, but allow it for threads.
 
+		**page_fault**
+			I/O engine that uses an mmap region to simulate data transfer by
+			allocating anonymous memory and copying data on read/write to
+			intentionally trigger page faults.
+
 	File/directory operation engines define how the job operates file or directory. The
 	following types are defined:
 
fio.1 +5 -0
index 664d7e3b..14ea6427 100644
@@ -2168,6 +2168,11 @@ instance is used per process, so all jobs setting option \fBthread\fR will share
 a single instance (with one queue per thread) and must specify compatible
 options. Note that some drivers don't allow several instances to access the same
 device or file simultaneously, but allow it for threads.
+.TP
+.B page_fault
+I/O engine that uses an mmap region to simulate data transfer by allocating
+anonymous memory and copying data on read/write to intentionally trigger page
+faults.
 .RE
 .P
 File/directory operation engines define how the job operates file or directory.
Signature · 1 line
2.53.0
Applied
#3 lore raw
In reply to Nico Pache
Quoted 10 lines · Nico Pache

This series introduces a new page_fault ioengine for Anonymous memory testing. This enables using fio’s existing framework and job files for memory management style workloads without relying on a filesystem. An example job file is included to demonstrate usage and lays the groundwork for how we plan on utilizing fio to test a number of MM related workloads.

The first patch adds the mmap‑backed ioengine that allocates anonymous memory and copies data on read/write to intentionally trigger faults.

[...]

Applied, thanks!

[1/2] page_fault: add mmap-backed ioengine for anonymous faults

      commit: f5ea7d9e1d7c07053f23e764b281a5e47bf01ba3
[2/2] Documentation: update the documentation to include the page_fault engine
      commit: dceeff3a1dd87b5194c84845d77096b3cdb3cee3

Best regards,

Signature · 1 line
Jens Axboe
author
#4 lore raw
Quoted 9 lines · Jens Axboe
Quoted 6 lines · Nico Pache

This series introduces a new page_fault ioengine for Anonymous memory testing. This enables using fio’s existing framework and job files for memory management style workloads without relying on a filesystem. An example job file is included to demonstrate usage and lays the groundwork for how we plan on utilizing fio to test a number of MM related workloads.

Quoted 6 lines
Quoted 4 lines

The first patch adds the mmap‑backed ioengine that allocates anonymous memory and copies data on read/write to intentionally trigger faults.

[...]

Applied, thanks!

Thank you! I have more ideas in mind but they will come in waves given my current workload. Please let me know if you spot any errors, bugs, or have suggestions for features to add.

-- Nico

Quoted 12 lines

[1/2] page_fault: add mmap-backed ioengine for anonymous faults

      commit: f5ea7d9e1d7c07053f23e764b281a5e47bf01ba3
[2/2] Documentation: update the documentation to include the page_fault engine
      commit: dceeff3a1dd87b5194c84845d77096b3cdb3cee3

Best regards,
--
Jens Axboe