page_fault: add mmap-backed ioengine for anonymous faults
v3 cover page_fault: add mmap-backed ioengine for anonymous faults
This series introduces a new page_fault ioengine for Anonymous memory testing. This enables using fio’s existing framework and job files for memory management style workloads without relying on a filesystem. An example job file is included to demonstrate usage and lays the groundwork for how we plan on utilizing fio to test a number of MM related workloads.
The first patch adds the mmap‑backed ioengine that allocates anonymous memory and copies data on read/write to intentionally trigger faults.
The second patch updates the documentation to guide usage of the new feature.
V3:
- Correctly use clang-format to adhere to formatting conventions
- check (and fail/warn) if user sets the start_offset
- directly utilize io_ops_data as there is no files
- updated error/print to follow proper format
- Copy docs update to fio.1
V2: https://lore.kernel.org/fio/[email protected]/
V1: https://lore.kernel.org/fio/[email protected]/
Nico Pache (2):
page_fault: add mmap-backed ioengine for anonymous faults
Documentation: update the documentation to include the page_fault
engine
HOWTO.rst | 5 ++ Makefile | 2 +- engines/page_fault.c | 125 ++++++++++++++++++++++++++++++++++++++++ examples/page_fault.fio | 60 +++++++++++++++++++ fio.1 | 5 ++ 5 files changed, 196 insertions(+), 1 deletion(-) create mode 100644 engines/page_fault.c create mode 100644 examples/page_fault.fio
Signature · 1 line
2.53.0
PATCH v3 1/2 page_fault: add mmap-backed ioengine for anonymous faults
Introduce a new ioengine that mmaps anonymous memory and copies data on read/write to trigger page faults. This allows us to leverage FIOs powerful framework for MM related testing, and will ideally allow us to quickly expand testing, by leveraging previously FS related fio scripts.
--- Makefile | 2 +- engines/page_fault.c | 125 ++++++++++++++++++++++++++++++++++++++++ examples/page_fault.fio | 60 +++++++++++++++++++ 3 files changed, 186 insertions(+), 1 deletion(-) create mode 100644 engines/page_fault.c create mode 100644 examples/page_fault.fio
Makefile +1 -1
@@ -57,7 +57,7 @@ SOURCE := $(sort $(patsubst $(SRCDIR)/%,%,$(wildcard $(SRCDIR)/crc/*.c)) \ smalloc.c filehash.c profile.c debug.c engines/cpu.c \ engines/mmap.c engines/sync.c engines/null.c engines/net.c \ engines/ftruncate.c engines/fileoperations.c \ - engines/exec.c \ + engines/exec.c engines/page_fault.c \ server.c client.c iolog.c backend.c libfio.c flow.c cconv.c \ gettime-thread.c helpers.c json.c idletime.c td_error.c \ profiles/tiobench.c profiles/act.c io_u_queue.c filelock.c \
engines/page_fault.c new file +125 -0
126 lines not on this page · raw message
examples/page_fault.fio new file +60 -0
60 lines not on this page · raw message
Signature · 1 line
2.53.0
PATCH v3 2/2 Documentation: update the documentation to include the page_fault engine
In reply to Nico PacheDocument the new page fault engine.
--- HOWTO.rst | 5 +++++ fio.1 | 5 +++++ 2 files changed, 10 insertions(+)
HOWTO.rst +5 -0
@@ -2379,6 +2379,11 @@ I/O engine several instances to access the same device or file simultaneously, but allow it for threads. + **page_fault** + I/O engine that uses an mmap region to simulate data transfer by + allocating anonymous memory and copying data on read/write to + intentionally trigger page faults. + File/directory operation engines define how the job operates file or directory. The following types are defined:
fio.1 +5 -0
@@ -2168,6 +2168,11 @@ instance is used per process, so all jobs setting option \fBthread\fR will share a single instance (with one queue per thread) and must specify compatible options. Note that some drivers don't allow several instances to access the same device or file simultaneously, but allow it for threads. +.TP +.B page_fault +I/O engine that uses an mmap region to simulate data transfer by allocating +anonymous memory and copying data on read/write to intentionally trigger page +faults. .RE .P File/directory operation engines define how the job operates file or directory.
Signature · 1 line
2.53.0
Jens Axboe
AppliedQuoted 10 lines · Nico Pache
This series introduces a new
page_faultioengine for Anonymous memory testing. This enables using fio’s existing framework and job files for memory management style workloads without relying on a filesystem. An example job file is included to demonstrate usage and lays the groundwork for how we plan on utilizing fio to test a number of MM related workloads.The first patch adds the mmap‑backed ioengine that allocates anonymous memory and copies data on read/write to intentionally trigger faults.
[...]
Applied, thanks!
[1/2] page_fault: add mmap-backed ioengine for anonymous faults
commit: f5ea7d9e1d7c07053f23e764b281a5e47bf01ba3
[2/2] Documentation: update the documentation to include the page_fault engine
commit: dceeff3a1dd87b5194c84845d77096b3cdb3cee3Best regards,
Signature · 1 line
Jens Axboe
The result of fio's continuous integration tests was: success
For more details see https://github.com/fiotestbot/fio/actions/runs/24112588366
Quoted 9 lines · Jens Axboe
Quoted 6 lines · Nico Pache
This series introduces a new
page_faultioengine for Anonymous memory testing. This enables using fio’s existing framework and job files for memory management style workloads without relying on a filesystem. An example job file is included to demonstrate usage and lays the groundwork for how we plan on utilizing fio to test a number of MM related workloads.
Quoted 6 lines
Quoted 4 lines
The first patch adds the mmap‑backed ioengine that allocates anonymous memory and copies data on read/write to intentionally trigger faults.
[...]
Applied, thanks!
Thank you! I have more ideas in mind but they will come in waves given my current workload. Please let me know if you spot any errors, bugs, or have suggestions for features to add.
-- Nico
Quoted 12 lines
[1/2]
page_fault: add mmap-backed ioengine for anonymous faultscommit: f5ea7d9e1d7c07053f23e764b281a5e47bf01ba3 [2/2] Documentation: update the documentation to include the page_fault engine commit: dceeff3a1dd87b5194c84845d77096b3cdb3cee3Best regards,
--
Jens Axboe