Introduce a page_fault ioengine for MM workflows
cover Introduce a page_fault ioengine for MM workflows
This series introduces a new page_fault ioengine and a follow‑up option to make khugepaged behavior more deterministic for page fault testing.
The first patch adds an mmap‑backed ioengine that allocates anonymous memory and copies data on read/write to intentionally trigger faults. This enables using fio’s existing framework and job files for memory management style workloads without relying on a filesystem. An example job file is included to demonstrate usage.
The second patch adds a hugepage_delay option to the new engine. It initially maps memory with MADV_NOHUGEPAGE, then (after a configurable delay) switches to MADV_HUGEPAGE via a helper thread. This makes khugepaged candidates reproducible, allows for easier khugepaged testing and improves repeatability for page_fault tests.
Nico Pache (2):
page_fault: add mmap-backed ioengine for anonymous faults
page_fault: add hugepage_delay option for delayed MADV_HUGEPAGE
Makefile | 2 +- cconv.c | 3 + engines/page_fault.c | 185 ++++++++++++++++++++++++++++++++++++++++ examples/page_fault.fio | 9 ++ options.c | 10 +++ thread_options.h | 3 + 6 files changed, 211 insertions(+), 1 deletion(-) create mode 100644 engines/page_fault.c create mode 100644 examples/page_fault.fio
Signature · 1 line
2.52.0
RFC 1/2 page_fault: add mmap-backed ioengine for anonymous faults
Introduce a new ioengine that mmaps anonymous memory and copies data on read/write to trigger page faults. This allows us to leverage FIOs powerful framework for MM related testing, and will ideally allow us to quickly expand testing, by leveraging previously FS related fio scripts.
--- Makefile | 2 +- engines/page_fault.c | 105 ++++++++++++++++++++++++++++++++++++++++ examples/page_fault.fio | 9 ++++ 3 files changed, 115 insertions(+), 1 deletion(-) create mode 100644 engines/page_fault.c create mode 100644 examples/page_fault.fio
Makefile +1 -1
@@ -57,7 +57,7 @@ SOURCE := $(sort $(patsubst $(SRCDIR)/%,%,$(wildcard $(SRCDIR)/crc/*.c)) \ smalloc.c filehash.c profile.c debug.c engines/cpu.c \ engines/mmap.c engines/sync.c engines/null.c engines/net.c \ engines/ftruncate.c engines/fileoperations.c \ - engines/exec.c \ + engines/exec.c engines/page_fault.c \ server.c client.c iolog.c backend.c libfio.c flow.c cconv.c \ gettime-thread.c helpers.c json.c idletime.c td_error.c \ profiles/tiobench.c profiles/act.c io_u_queue.c filelock.c \
engines/page_fault.c new file +105 -0
106 lines not on this page · raw message
examples/page_fault.fio new file +9 -0
@@ -0,0 +1,9 @@ +[global] +bs=4k + +[page_fault] +ioengine=page_fault +size=2g +rw=randrw +rwmixread=50 +verify=crc32c
Signature · 1 line
2.52.0
RFC 2/2 page_fault: add hugepage_delay option for delayed MADV_HUGEPAGE
In reply to Nico PacheIntroduce hugepage_delay to map with MADV_NOHUGEPAGE first, then madvise MADV_HUGEPAGE after a configurable delay via a helper thread. This makes khugepaged candidates reproducible for page_fault tests.
--- cconv.c | 3 ++ engines/page_fault.c | 94 ++++++++++++++++++++++++++++++++++++++++---- options.c | 10 +++++ thread_options.h | 3 ++ 4 files changed, 103 insertions(+), 7 deletions(-)
cconv.c +3 -0
@@ -242,6 +242,7 @@ int convert_thread_options_to_cpu(struct thread_options *o, o->random_center.u.f =fio_uint64_to_double (le64_to_cpu (top->random_center.u.i)); o->random_generator =le32_to_cpu (top->random_generator); o->hugepage_size =le32_to_cpu (top->hugepage_size); + o->hugepage_delay =le32_to_cpu (top->hugepage_delay); o->rw_min_bs =le64_to_cpu (top->rw_min_bs); o->thinkcycles =le32_to_cpu (top->thinkcycles); o->thinktime =le32_to_cpu (top->thinktime);
@@ -494,6 +495,8 @@ void convert_thread_options_to_net(struct thread_options_pack *top, top->random_center.u.i =__cpu_to_le64 (fio_double_to_uint64 (o->random_center.u.f)); top->random_generator =cpu_to_le32 (o->random_generator); top->hugepage_size =cpu_to_le32 (o->hugepage_size); + top->hugepage_delay =cpu_to_le32 (o->hugepage_delay); + top->hugepage_delay_pad =0 ; top->rw_min_bs =__cpu_to_le64 (o->rw_min_bs); top->thinkcycles =cpu_to_le32 (o->thinkcycles); top->thinktime =cpu_to_le32 (o->thinktime);
engines/page_fault.c +87 -7
125 lines not on this page · raw message
options.c +10 -0
@@ -5490,6 +5490,16 @@ struct fio_option fio_options[FIO_MAX_OPTS] = { .category = FIO_OPT_C_GENERAL, .group = FIO_OPT_G_INVALID, }, + { + .name = "hugepage_delay", + .lname = "Hugepage delay", + .type = FIO_OPT_INT, + .off1 = offsetof(struct thread_options, hugepage_delay), + .help = "For mmap, map with MADV_NOHUGEPAGE then MADV_HUGEPAGE after delay (in ms)", + .def = "0", + .category = FIO_OPT_C_GENERAL, + .group = FIO_OPT_G_INVALID, + }, { .name = "flow_id", .lname = "I/O flow ID",
thread_options.h +3 -0
@@ -203,6 +203,7 @@ struct thread_options {unsigned int perc_rand[DDIR_RWDIR_CNT ];unsigned int hugepage_size; +unsigned int hugepage_delay;unsigned long long rw_min_bs;unsigned int fsync_blocks;unsigned int fdatasync_blocks;
@@ -539,6 +540,8 @@ struct thread_options_pack {uint32_t perc_rand[DDIR_RWDIR_CNT ];uint32_t hugepage_size; +uint32_t hugepage_delay; +uint32_t hugepage_delay_pad;uint64_t rw_min_bs;uint32_t fsync_blocks;uint32_t fdatasync_blocks;
Signature · 1 line
2.52.0
Re: [RFC 2/2] page_fault: add hugepage_delay option for delayed MADV_HUGEPAGE
Quoted 7 lines · Nico Pache
Introduce
hugepage_delayto map withMADV_NOHUGEPAGEfirst, then madviseMADV_HUGEPAGEafter a configurable delay via a helper thread. This makes khugepaged candidates reproducible forpage_faulttests.Signed-off-by Nico Pache <[email protected]>---
I would add this option as an *ioengine* option instead of a general FIo option. There are examples in sync.c, io_uring.c, etc.
This option also needs to be added to the documentation.
Vincent
Re: [RFC 2/2] page_fault: add hugepage_delay option for delayed MADV_HUGEPAGE
Quoted 7 lines · Vincent Fu
Quoted 5 lines · Nico Pache
Introduce
hugepage_delayto map withMADV_NOHUGEPAGEfirst, then madviseMADV_HUGEPAGEafter a configurable delay via a helper thread. This makes khugepaged candidates reproducible forpage_faulttests.
Quoted 7 lines
Quoted 2 lines
Signed-off-by Nico Pache <[email protected]>---
I would add this option as an *ioengine* option instead of a general FIo option. There are examples in
sync.c,io_uring.c, etc.This option also needs to be added to the documentation.
Hey Vincent,
Thanks for the review. I will switch over to ioengine options and add some documentation!
Cheers,
-- Nico
Quoted 3 lines
Vincent
The result of fio's continuous integration tests was: success
For more details see https://github.com/fiotestbot/fio/actions/runs/21529724218
Quoted 9 lines · Nico Pache
This series introduces a new
page_faultioengine and a follow?up option to make khugepaged behavior more deterministic for page fault testing.The first patch adds an mmap?backed ioengine that allocates anonymous memory and copies data on read/write to intentionally trigger faults. This enables using fio?s existing framework and job files for memory management style workloads without relying on a filesystem. An example job file is included to demonstrate usage.
Quoted 5 lines
The second patch adds a
hugepage_delayoption to the new engine. It initially maps memory withMADV_NOHUGEPAGE, then (after a configurable delay) switches toMADV_HUGEPAGEvia a helper thread. This makes khugepaged candidates reproducible, allows for easier khugepaged testing and improves repeatability forpage_faulttests.
Looks pretty clean - echo what Vincent added on the option that should be an engine specific option, but outside of that, the formatting of the code is all wrong. Please follow the style of the code that's in the repo already, which means:
1) Placement of braces
2) Tabs are... tabs, not 4 spaces. The code is a mix and match of this.
3) Appropriate empty lines after variable declarations
4) Style of switch case (indentation and braces)
etc. It's a good start to just use the clang format from the kernel, that should get you basically to where you need to be.
Signature · 1 line
Jens Axboe
Quoted 17 lines · Jens Axboe
Quoted 9 lines · Nico Pache
This series introduces a new
page_faultioengine and a follow?up option to make khugepaged behavior more deterministic for page fault testing.The first patch adds an mmap?backed ioengine that allocates anonymous memory and copies data on read/write to intentionally trigger faults. This enables using fio?s existing framework and job files for memory management style workloads without relying on a filesystem. An example job file is included to demonstrate usage.
Quoted 5 lines
The second patch adds a
hugepage_delayoption to the new engine. It initially maps memory withMADV_NOHUGEPAGE, then (after a configurable delay) switches toMADV_HUGEPAGEvia a helper thread. This makes khugepaged candidates reproducible, allows for easier khugepaged testing and improves repeatability forpage_faulttests.
Quoted 4 lines
Looks pretty clean - echo what Vincent added on the option that should be an engine specific option, but outside of that, the formatting of the code is all wrong. Please follow the style of the code that's in the repo already, which means:
Thanks! Yeah the engine specific option makes total sense, I will fix that. That should also allow me to add some other functionality to this engine :)
Quoted 8 lines
1) Placement of braces
2) Tabs are... tabs, not 4 spaces. The code is a mix and match of this.
3) Appropriate empty lines after variable declarations
4) Style of switch case (indentation and braces)etc. It's a good start to just use the clang format from the kernel, that should get you basically to where you need to be.
Sounds good! yeah sorry, didn't really consider the formatting at all for the RFC, was just trying to get the idea out, and make sure there were no glaring issues!
Thanks for the review!
-- Nico
Quoted 4 lines
--
Jens Axboe