fdp fixes, random placment id and xnvme support
A message that is not in the archive (sent off-list or not yet received): CGME20230712050110epcas5p1282a4e40485cb37545ec7cb55a3dcff1@epcas5p1.samsung.com
v2 cover fdp fixes, random placment id and xnvme support
This series now enables us to either roundrobin or select a random placement ID from the available ones. In future a non-unifrom placement id selection such as zipf, pareto etc can be added.
The current boundary check for input placment ids was not correct. Added a fix for that.
This series also adds FDP support to the xnvme I/O engine.
Changes since v1:
- Split the macro change and placement id check fix into separate patches as suggested by Vincent
Ankit Kumar (4):
fdp: use macros
fdp: fix placement id check
fdp: support random placement id selection
engines/xnvme: add support for fdp
HOWTO.rst | 19 ++++++++-- configure | 2 +- engines/io_uring.c | 2 +- engines/xnvme.c | 78 +++++++++++++++++++++++++++++++++++++++++- examples/xnvme-fdp.fio | 36 +++++++++++++++++++ fdp.c | 22 +++++++----- fdp.h | 13 +++++++ fio.1 | 20 +++++++++-- fio.h | 2 ++ init.c | 2 ++ options.c | 20 +++++++++++ thread_options.h | 2 ++ 12 files changed, 203 insertions(+), 15 deletions(-) create mode 100644 examples/xnvme-fdp.fio
Signature · 1 line
2.25.1
PATCH v2 1/4 fdp: use macros
use macros for directive type and max ruhs.
--- engines/io_uring.c | 2 +- fdp.c | 8 ++++---- fdp.h | 3 +++ 3 files changed, 8 insertions(+), 5 deletions(-)
engines/io_uring.c +1 -1
@@ -1310,7 +1310,7 @@ static int fio_ioring_cmd_fetch_ruhs(struct thread_data *td, struct fio_file *f,struct nvme_fdp_ruh_status *ruhs;int bytes, ret, i; - bytes =sizeof (*ruhs) +128 *sizeof (struct nvme_fdp_ruh_status_desc ); + bytes =sizeof (*ruhs) +FDP_MAX_RUHS *sizeof (struct nvme_fdp_ruh_status_desc ); ruhs =scalloc (1 , bytes);if (!ruhs)return -ENOMEM ;
fdp.c +4 -4
@@ -45,7 +45,7 @@ static int init_ruh_info(struct thread_data *td, struct fio_file *f)struct fio_ruhs_info *ruhs, *tmp;int i, ret; - ruhs =scalloc (1 ,sizeof (*ruhs) +128 *sizeof (*ruhs->plis)); + ruhs =scalloc (1 ,sizeof (*ruhs) +FDP_MAX_RUHS *sizeof (*ruhs->plis));if (!ruhs)return -ENOMEM ;
@@ -56,8 +56,8 @@ static int init_ruh_info(struct thread_data *td, struct fio_file *f) goto out; } -if (ruhs->nr_ruhs >128 ) - ruhs->nr_ruhs =128 ; +if (ruhs->nr_ruhs >FDP_MAX_RUHS ) + ruhs->nr_ruhs =FDP_MAX_RUHS ;if (td->o.fdp_nrpli ==0 ) { f->ruhs_info = ruhs;
@@ -123,6 +123,6 @@ void fdp_fill_dspec_data(struct thread_data *td, struct io_u *io_u) ruhs->pli_loc =0 ; dspec = ruhs->plis[ruhs->pli_loc++]; - io_u->dtype =2 ; + io_u->dtype =FDP_DIR_DTYPE ; io_u->dspec = dspec; }
fdp.h +3 -0
@@ -3,6 +3,9 @@#include "io_u.h" +#define FDP_DIR_DTYPE 2 +#define FDP_MAX_RUHS 128 +struct fio_ruhs_info {uint32_t nr_ruhs;uint32_t pli_loc;
Signature · 1 line
2.25.1
PATCH v2 2/4 fdp: fix placement id check
In reply to Ankit KumarNumber of reclaim unit handle descriptors are 1 based, whereas the input placement id index / indices are 0 based. Add the correct check for that.
--- fdp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fdp.c +1 -1
@@ -65,7 +65,7 @@ static int init_ruh_info(struct thread_data *td, struct fio_file *f) }for (i =0 ; i < td->o.fdp_nrpli; i++) { -if (td->o.fdp_plis[i] > ruhs->nr_ruhs) { +if (td->o.fdp_plis[i] >= ruhs->nr_ruhs) { ret = -EINVAL ; goto out; }
Signature · 1 line
2.25.1
PATCH v2 3/4 fdp: support random placement id selection
In reply to Ankit KumarAllow user to either roundrobin or select random placement ID from the available placement IDs.
--- HOWTO.rst | 15 +++++++++++++++ fdp.c | 12 +++++++++--- fdp.h | 10 ++++++++++ fio.1 | 16 ++++++++++++++++ fio.h | 2 ++ init.c | 2 ++ options.c | 20 ++++++++++++++++++++ thread_options.h | 2 ++ 8 files changed, 76 insertions(+), 3 deletions(-)
HOWTO.rst +15 -0
@@ -2435,6 +2435,21 @@ with the caveat that when used on the command line, they must come after the Enable Flexible Data Placement mode for write commands. +.. option:: fdp_pli_select=str : [io_uring_cmd] + + Defines how fio decides which placement ID to use next. The following + types are defined: + + **random** + Choose a placement ID at random (uniform). + + **roundrobin** + Round robin over available placement IDs. This is the + default. + + The available placement ID index/indices is defined by the option + :option:`fdp_pli`. + .. option:: fdp_pli=str : [io_uring_cmd] Select which Placement ID Index/Indicies this job is allowed to use for
fdp.c +9 -3
@@ -119,10 +119,16 @@ void fdp_fill_dspec_data(struct thread_data *td, struct io_u *io_u)return ; } -if (ruhs->pli_loc >= ruhs->nr_ruhs) - ruhs->pli_loc =0 ; +if (td->o.fdp_pli_select ==FIO_FDP_RR ) { +if (ruhs->pli_loc >= ruhs->nr_ruhs) + ruhs->pli_loc =0 ; + + dspec = ruhs->plis[ruhs->pli_loc++]; + }else { + ruhs->pli_loc =rand_between (&td->fdp_state,0 , ruhs->nr_ruhs -1 ); + dspec = ruhs->plis[ruhs->pli_loc]; + } - dspec = ruhs->plis[ruhs->pli_loc++]; io_u->dtype =FDP_DIR_DTYPE ; io_u->dspec = dspec; }
fdp.h +10 -0
@@ -6,6 +6,16 @@#define FDP_DIR_DTYPE 2#define FDP_MAX_RUHS 128 +/* +* How fio chooses what placement identifier to use next. Choice of +* uniformly random, or roundrobin. +*/ + +enum { +FIO_FDP_RANDOM =0x1 , +FIO_FDP_RR =0x2 , +}; +struct fio_ruhs_info {uint32_t nr_ruhs;uint32_t pli_loc;
fio.1 +16 -0
@@ -2195,6 +2195,22 @@ file blocks are fully allocated and the disk request could be issued immediately .BI (io_uring_cmd)fdp \fR=\fPbool Enable Flexible Data Placement mode for write commands. .TP +.BI (io_uring_cmd)fdp_pli_select \fR=\fPstr +Defines how fio decides which placement ID to use next. The following types +are defined: +.RS +.RS +.TP +.B random +Choose a placement ID at random (uniform). +.TP +.B roundrobin +Round robin over available placement IDs. This is the default. +.RE +.P +The available placement ID index/indices is defined by \fBfdp_pli\fR option. +.RE +.TP .BI (io_uring_cmd)fdp_pli \fR=\fPstr Select which Placement ID Index/Indicies this job is allowed to use for writes. By default, the job will cycle through all available Placement IDs, so use this
fio.h +2 -0
@@ -144,6 +144,7 @@ enum {FIO_RAND_POISSON3_OFF ,FIO_RAND_PRIO_CMDS ,FIO_RAND_DEDUPE_WORKING_SET_IX , +FIO_RAND_FDP_OFF , FIO_RAND_NR_OFFS, };
@@ -262,6 +263,7 @@ struct thread_data {struct frand_state verify_state_last_do_io;struct frand_state trim_state;struct frand_state delay_state; +struct frand_state fdp_state;struct frand_state buf_state;struct frand_state buf_state_prev;
init.c +2 -0
@@ -1082,6 +1082,8 @@ void td_fill_rand_seeds(struct thread_data *td)init_rand_seed (&td->buf_state, td->rand_seeds[FIO_RAND_BUF_OFF ], use64);frand_copy (&td->buf_state_prev, &td->buf_state); + +init_rand_seed (&td->fdp_state, td->rand_seeds[FIO_RAND_FDP_OFF ], use64); }static int setup_random_seeds (struct thread_data *td)
options.c +20 -0
@@ -3679,6 +3679,26 @@ struct fio_option fio_options[FIO_MAX_OPTS] = { .category = FIO_OPT_C_IO, .group = FIO_OPT_G_INVALID, }, + { + .name = "fdp_pli_select", + .lname = "FDP Placement ID select", + .type = FIO_OPT_STR, + .off1 = offsetof(struct thread_options, fdp_pli_select), + .help = "Select which FDP placement ID to use next", + .def = "roundrobin", + .category = FIO_OPT_C_IO, + .group = FIO_OPT_G_INVALID, + .posval = { + { .ival = "random", + .oval = FIO_FDP_RANDOM, + .help = "Choose a Placement ID at random (uniform)", + }, + { .ival = "roundrobin", + .oval = FIO_FDP_RR, + .help = "Round robin select Placement IDs", + }, + }, + }, { .name = "fdp_pli", .lname = "FDP Placement ID indicies",
thread_options.h +2 -0
@@ -388,6 +388,7 @@ struct thread_options {#define FIO_MAX_PLIS 16unsigned int fdp; +unsigned int fdp_pli_select;unsigned int fdp_plis[FIO_MAX_PLIS ];unsigned int fdp_nrpli;
@@ -703,6 +704,7 @@ struct thread_options_pack {uint32_t log_prio;uint32_t fdp; +uint32_t fdp_pli_select;uint32_t fdp_plis[FIO_MAX_PLIS ];uint32_t fdp_nrpli;
Signature · 1 line
2.25.1
PATCH v2 4/4 engines/xnvme: add support for fdp
In reply to Ankit KumarAdd FDP support to xnvme I/O engine. This support can be used only with nvme-ns generic character device (/dev/ngXnY). The available backends are --xnvme_async=io_uring_cmd and --xnvme_sync=nvme.
Add a xnvme-fdp config example file.
Update the minimum required xnvme version to 0.7.0
--- HOWTO.rst | 6 ++-- configure | 2 +- engines/xnvme.c | 78 +++++++++++++++++++++++++++++++++++++++++- examples/xnvme-fdp.fio | 36 +++++++++++++++++++ fio.1 | 6 ++-- 5 files changed, 120 insertions(+), 8 deletions(-) create mode 100644 examples/xnvme-fdp.fio
HOWTO.rst +3 -3
@@ -2431,11 +2431,11 @@ with the caveat that when used on the command line, they must come after the For direct I/O, requests will only succeed if cache invalidation isn't required, file blocks are fully allocated and the disk request could be issued immediately. -.. option:: fdp=bool : [io_uring_cmd] +.. option:: fdp=bool : [io_uring_cmd] [xnvme] Enable Flexible Data Placement mode for write commands. -.. option:: fdp_pli_select=str : [io_uring_cmd] +.. option:: fdp_pli_select=str : [io_uring_cmd] [xnvme] Defines how fio decides which placement ID to use next. The following types are defined:
@@ -2450,7 +2450,7 @@ with the caveat that when used on the command line, they must come after the The available placement ID index/indices is defined by the option :option:`fdp_pli`. -.. option:: fdp_pli=str : [io_uring_cmd] +.. option:: fdp_pli=str : [io_uring_cmd] [xnvme] Select which Placement ID Index/Indicies this job is allowed to use for writes. By default, the job will cycle through all available Placement
configure +1 -1
@@ -2651,7 +2651,7 @@ fi ########################################## # Check if we have xnvme if test "$xnvme" != "no" ; then - if check_min_lib_version xnvme 0.2.0; then + if check_min_lib_version xnvme 0.7.0; then xnvme="yes" xnvme_cflags=$(pkg-config --cflags xnvme) xnvme_libs=$(pkg-config --libs xnvme)
engines/xnvme.c +77 -1
114 lines not on this page · raw message
examples/xnvme-fdp.fio new file +36 -0
@@ -0,0 +1,36 @@ +; README +; +; This job-file is intended to be used either as: +; +; # Use the xNVMe io-engine engine io_uring_cmd async. impl. +; fio examples/xnvme-fdp.fio \ +; --section=default \ +; --ioengine=xnvme \ +; --xnvme_async=io_uring_cmd \ +; --filename=/dev/ng0n1 +; +; # Use the xNVMe io-engine engine with nvme sync. impl. +; fio examples/xnvme-fdp.fio \ +; --section=default \ +; --ioengine=xnvme \ +; --xnvme_sync=nvme \ +; --filename=/dev/ng0n1 +; +; FIO_BS="512" FIO_RW="read" FIO_IODEPTH=16 fio examples/xnvme-fdp.fio \ +; --section=override --ioengine=xnvme --xnvme_sync=nvme --filename=/dev/ng0n1 +; +[global] +rw=randwrite +size=2M +iodepth=1 +bs=4K +thread=1 +fdp=1 +fdp_pli=4,5 + +[default] + +[override] +rw=${FIO_RW} +iodepth=${FIO_IODEPTH} +bs=${FIO_BS}
fio.1 +3 -3
@@ -2192,10 +2192,10 @@ cached data. Currently the RWF_NOWAIT flag does not supported for cached write. For direct I/O, requests will only succeed if cache invalidation isn't required, file blocks are fully allocated and the disk request could be issued immediately. .TP -.BI (io_uring_cmd)fdp \fR=\fPbool +.BI (io_uring_cmd,xnvme)fdp \fR=\fPbool Enable Flexible Data Placement mode for write commands. .TP -.BI (io_uring_cmd)fdp_pli_select \fR=\fPstr +.BI (io_uring_cmd,xnvme)fdp_pli_select \fR=\fPstr Defines how fio decides which placement ID to use next. The following types are defined: .RS
@@ -2211,7 +2211,7 @@ Round robin over available placement IDs. This is the default. The available placement ID index/indices is defined by \fBfdp_pli\fR option. .RE .TP -.BI (io_uring_cmd)fdp_pli \fR=\fPstr +.BI (io_uring_cmd,xnvme)fdp_pli \fR=\fPstr Select which Placement ID Index/Indicies this job is allowed to use for writes. By default, the job will cycle through all available Placement IDs, so use this to isolate these identifiers to specific jobs. If you want fio to use placement
Signature · 1 line
2.25.1
Vincent Fu
AppliedQuoted 32 lines · Ankit Kumar
This series now enables us to either roundrobin or select a random placement ID from the available ones. In future a non-unifrom placement id selection such as zipf, pareto etc can be added.
The current boundary check for input placment ids was not correct. Added a fix for that.
This series also adds FDP support to the xnvme I/O engine.
Changes since v1:
- Split the macro change and placement id check fix into separate patches as suggested by VincentAnkit Kumar (4):
fdp: use macros
fdp: fix placement id check
fdp: support random placement id selection
engines/xnvme: add support for fdpHOWTO.rst | 19 ++++++++-- configure | 2 +- engines/io_uring.c | 2 +- engines/xnvme.c | 78 +++++++++++++++++++++++++++++++++++++++++- examples/xnvme-fdp.fio | 36 +++++++++++++++++++ fdp.c | 22 +++++++----- fdp.h | 13 +++++++ fio.1 | 20 +++++++++-- fio.h | 2 ++ init.c | 2 ++ options.c | 20 +++++++++++ thread_options.h | 2 ++ 12 files changed, 203 insertions(+), 15 deletions(-)
Quoted 2 lines
create mode 100644 examples/xnvme-fdp.fio
Applied, thanks.
Vincent