Skip to contentSkip to conversation

v3 Patchset : Zone Append command support

PATCH In review Krishna Kanth Reddy 2 participants 5 messages Jul 28, 2020 1 message not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): CGME20200728124000epcas5p4fa4a9eed2e1f8d3e20f36ede60a63708@epcas5p4.samsung.com

#1 lore raw

cover v3 Patchset : Zone Append command support

This is a v3 Patchset for the Zone Append command support The v1 Pull Request is https://github.com/axboe/fio/pull/1021
The v2 Pull Request is https://github.com/axboe/fio/pull/1026

Review comments for the v2 Patchset have been taken care of in the v3 Patchset. Corresponding Linux Kernel Patchset for the Zone Append command support :
https://lore.kernel.org/linux-block/[email protected]/

1. Added a new FIO option zone_append.
When zone_append option is enabled, for Zone mode ZBD,
the existing write path will send Zone Append command with offset as start of the Zone.
2. libaio: support for Zone Append command in libaio IO engine
3. t/zbd: Add support to verify Zone Append command with libaio IO engine tests

This is a RFC and the Linux Kernel io_uring interface is being discussed in the LKML.
io_uring engine's Zone Append command support will be submitted for review after the kernel interface is finalized.

Feedback / Comments will help in improving this further.

Ankit Kumar (2):
Add Zone Append command support
libaio: support for Zone Append command

Krishna Kanth Reddy (1):
t/zbd: Add support to verify Zone Append command with libaio tests

 HOWTO                  |  7 +++++++
 engines/libaio.c       | 32 ++++++++++++++++++++++++++++----
 fio.1                  |  7 +++++++
 ioengines.c            |  6 ++++++
 ioengines.h            |  3 ++-
 options.c              | 10 ++++++++++
 t/zbd/test-zbd-support | 29 +++++++++++++++++++++++++++++
 thread_options.h       |  2 ++
 8 files changed, 91 insertions(+), 5 deletions(-)
Signature · 1 line
2.17.1
#2 lore raw

PATCH 1/3 Add Zone Append command support

From: Ankit Kumar <[email protected]>

Added a new FIO option zone_append. In ZBD zone mode, when zone_append option is enabled, the existing write path will send Zone Append command with offset as start of the Zone. Added a new io engine flag FIO_ZONE_APPEND that can be used in engine declaration to indicate that it supports zone append.

---
 HOWTO            |  7 +++++++
 fio.1            |  7 +++++++
 ioengines.c      |  6 ++++++
 ioengines.h      |  3 ++-
 options.c        | 10 ++++++++++
 thread_options.h |  2 ++
 6 files changed, 34 insertions(+), 1 deletion(-)
HOWTO +7 -0
index 35ead0cb..a46b9da0 100644
@@ -1022,6 +1022,13 @@ Target file/device
 	:option:`zonesize` bytes of data have been transferred. This parameter
 	must be zero for :option:`zonemode` =zbd.
 
+.. option:: zone_append=bool
+
+	For :option:`rw` =write or :option: `rw` =randwrite and
+	:option:`zonemode` =zbd, if zone_append is enabled the offset to write
+	will be the starting offset of a zone. On successful completion the device
+	returns the offset where the data has been placed.
+
 .. option:: read_beyond_wp=bool
 
 	This parameter applies to :option:`zonemode` =zbd only.
fio.1 +7 -0
index a3d348b2..00122808 100644
@@ -790,6 +790,13 @@ zone have been read (read workloads). This parameter is valid only for
 sequential workloads and ignored for random workloads. For read workloads,
 see also \fBread_beyond_wp\fR.
 
+.TP
+.BI zone_append
+For :option:`rw` =write or :option: `rw` =randwrite and for
+:option:`zonemode` =zbd, if zone_append is enabled the offset to write will be
+the starting offset of a zone. On successful completion the device returns the
+offset where the data has been placed.
+
 .TP
 .BI read_beyond_wp \fR=\fPbool
 This parameter applies to \fBzonemode=zbd\fR only.
ioengines.c +6 -0
index 1c5970a4..288306a2 100644
@@ -448,6 +448,12 @@ int td_io_init(struct thread_data *td)
 			td->error = ret;
 	}
 
+	if (td->o.zone_append && !td_ioengine_flagged(td, FIO_ZONE_APPEND)) {
+		log_err("fio: io engine %s doesn't support zone append\n",
+			td->io_ops->name);
+		return 1;
+	}
+
 	return ret;
 }
 
ioengines.h +2 -1
index 54dadba2..5ba6b7f2 100644
@@ -77,7 +77,8 @@ enum fio_ioengine_flags {
 	FIO_NOSTATS	= 1 << 12,	/* don't do IO stats */
 	FIO_NOFILEHASH	= 1 << 13,	/* doesn't hash the files for lookup later. */
 	FIO_ASYNCIO_SYNC_TRIM
-			= 1 << 14	/* io engine has async ->queue except for trim */
+			= 1 << 14,	/* io engine has async ->queue except for trim */
+	FIO_ZONE_APPEND = 1 << 15	/* engine supports zone append */
 };
 
 /*
options.c +10 -0
index 251ad2c1..0c9cd910 100644
@@ -3316,6 +3316,16 @@ struct fio_option fio_options[FIO_MAX_OPTS] = {
 			   },
 		},
 	},
+	{
+		.name	= "zone_append",
+		.lname	= "zone_append",
+		.type	= FIO_OPT_BOOL,
+		.off1	= offsetof(struct thread_options, zone_append),
+		.help	= "Use zone append for writing zones of a zoned block device",
+		.def	= "0",
+		.category = FIO_OPT_C_IO,
+		.group	= FIO_OPT_G_ZONE,
+	},
 	{
 		.name	= "zonesize",
 		.lname	= "Zone size",
thread_options.h +2 -0
index 3fe48ecc..b03b35a7 100644
@@ -196,6 +196,7 @@ struct thread_options {
 	unsigned long long zone_capacity;
 	unsigned long long zone_skip;
 	enum fio_zone_mode zone_mode;
+	unsigned int zone_append;
 	unsigned long long lockmem;
 	enum fio_memtype mem_type;
 	unsigned int mem_align;
@@ -633,6 +634,7 @@ struct thread_options_pack {
 	uint32_t allow_mounted_write;
 
 	uint32_t zone_mode;
+	uint32_t zone_append;
 } __attribute__((packed));
 
 extern void convert_thread_options_to_cpu(struct thread_options *o, struct thread_options_pack *top);
Signature · 1 line
2.17.1
#3 lore raw

PATCH 2/3 libaio: support for Zone Append command

In reply to Krishna Kanth Reddy

From: Ankit Kumar <[email protected]>

The zone append command uses the write path with offset as start of the zone, when ZBD zone mode is enabled. Upon successful completion, offset of the location where the data has been placed is returned in res2 field of the io event. If data verification is enabled, the offset returned is used for read and verify.

---
 engines/libaio.c | 32 ++++++++++++++++++++++++++++----
 1 file changed, 28 insertions(+), 4 deletions(-)
engines/libaio.c +28 -4
index b909b79e..5c10f69d 100644
@@ -12,6 +12,7 @@
 #include <sys/resource.h>
 
 #include "../fio.h"
+#include "../zbd.h"
 #include "../lib/pow2.h"
 #include "../optgroup.h"
 #include "../lib/memalign.h"
@@ -123,7 +124,14 @@ static int fio_libaio_prep(struct thread_data *td, struct io_u *io_u)
 		if (o->nowait)
 			iocb->aio_rw_flags |= RWF_NOWAIT;
 	} else if (io_u->ddir == DDIR_WRITE) {
-		io_prep_pwrite(iocb, f->fd, io_u->xfer_buf, io_u->xfer_buflen, io_u->offset);
+		if ((td->o.zone_mode == ZONE_MODE_ZBD) && td->o.zone_append && f->zbd_info->model != ZBD_NONE) {
+			unsigned long long zone_start_offset = io_u->offset & ~(f->zbd_info->zone_size - 1);
+			io_prep_pwrite(iocb, f->fd, io_u->xfer_buf, io_u->xfer_buflen, zone_start_offset);
+		}
+		else
+			io_prep_pwrite(iocb, f->fd, io_u->xfer_buf, io_u->xfer_buflen, io_u->offset);
+		if (td->o.zone_append)
+			iocb->aio_rw_flags |= RWF_APPEND;
 		if (o->nowait)
 			iocb->aio_rw_flags |= RWF_NOWAIT;
 	} else if (ddir_sync(io_u->ddir))
@@ -229,9 +237,25 @@ static int fio_libaio_getevents(struct thread_data *td, unsigned int min,
 			r = io_getevents(ld->aio_ctx, actual_min,
 				max, ld->aio_events + events, lt);
 		}
-		if (r > 0)
+		if (r > 0) {
+			if ((td->o.zone_mode == ZONE_MODE_ZBD) && td->o.zone_append
+			    && td->o.do_verify && td->o.verify) {
+				struct io_event *ev;
+				struct io_u *io_u;
+				struct fio_file *f;
+				unsigned event;
+
+				for (event = 0; event < r; event++) {
+					ev = ld->aio_events + event;
+					io_u = container_of(ev->obj, struct io_u, iocb);
+					f = io_u->file;
+					if ((io_u->ddir == DDIR_WRITE)
+					    && (f->zbd_info->model != ZBD_NONE))
+						io_u->ipo->offset = ev->res2;
+				}
+			}
 			events += r;
-		else if ((min && r == 0) || r == -EAGAIN) {
+		} else if ((min && r == 0) || r == -EAGAIN) {
 			fio_libaio_commit(td);
 			if (actual_min)
 				usleep(10);
@@ -448,7 +472,7 @@ static int fio_libaio_init(struct thread_data *td)
 FIO_STATIC struct ioengine_ops ioengine = {
 	.name			= "libaio",
 	.version		= FIO_IOOPS_VERSION,
-	.flags			= FIO_ASYNCIO_SYNC_TRIM,
+	.flags			= FIO_ASYNCIO_SYNC_TRIM | FIO_ZONE_APPEND,
 	.init			= fio_libaio_init,
 	.post_init		= fio_libaio_post_init,
 	.prep			= fio_libaio_prep,
Signature · 1 line
2.17.1
#4 lore raw

PATCH 3/3 t/zbd: Add support to verify Zone Append command with libaio tests

In reply to Krishna Kanth Reddy

Added a new FIO option zone_append.
When zone_append option is enabled and when the zonemode is ZBD,
the existing write path will send Zone Append command with offset as start of the Zone.

---
 t/zbd/test-zbd-support | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)
t/zbd/test-zbd-support +29 -0
index 471a3487..048daaf5 100755
@@ -860,6 +860,35 @@ test49() {
     check_read $((capacity * 2)) || return $?
 }
 
+# Zone append to sequential zones, libaio, 1 job, queue depth 1
+test50() {
+    local size capacity
+
+    [ -n "$use_libzbc" ] && return 0
+    size=$((8 * zone_size))
+    off=$((first_sequential_zone_sector * 512))
+    capacity=$(total_zone_capacity 8 $off $dev)
+
+    run_fio_on_seq --ioengine=libaio --iodepth=1 --rw=write --zone_append=1 \
+                   --bs=65536 --size=$size --do_verify=1 --verify=md5 \
+                   >>"${logfile}.${test_number}" 2>&1 || return $?
+    check_written $capacity || return $?
+    check_read $capacity || return $?
+}
+
+# Random zone append to sequential zones, libaio, 8 jobs, queue depth 64 per job
+test51() {
+    local capacity
+
+    [ -n "$use_libzbc" ] && return 0
+    off=$((first_sequential_zone_sector * 512))
+    capacity=$(total_zone_capacity 4 $off $dev)
+    run_fio_on_seq --ioengine=libaio --iodepth=64 --rw=randwrite \
+		   --bs=4096 --group_reporting=1 --numjobs=8 --zone_append=1 \
+                   >> "${logfile}.${test_number}" 2>&1 || return $?
+    check_written $((capacity * 8)) || return $?
+}
+
 tests=()
 dynamic_analyzer=()
 reset_all_zones=
Signature · 1 line
2.17.1
#5 lore raw
In reply to Krishna Kanth Reddy
Quoted 14 lines · Krishna Kanth Reddy

This is a v3 Patchset for the Zone Append command support The v1 Pull Request is https://github.com/axboe/fio/pull/1021
The v2 Pull Request is https://github.com/axboe/fio/pull/1026

Review comments for the v2 Patchset have been taken care of in the v3 Patchset. Corresponding Linux Kernel Patchset for the Zone Append command support :
https://lore.kernel.org/linux-block/[email protected]/

1. Added a new FIO option zone_append.
When zone_append option is enabled, for Zone mode ZBD,
the existing write path will send Zone Append command with offset as start of the Zone.
2. libaio: support for Zone Append command in libaio IO engine
3. t/zbd: Add support to verify Zone Append command with libaio IO engine tests

Quoted 4 lines

This is a RFC and the Linux Kernel io_uring interface is being discussed in the LKML.
io_uring engine's Zone Append command support will be submitted for review after the kernel interface is finalized.

Feedback / Comments will help in improving this further.

Krishna,

The kernel side interface for zone append to raw block devices is not yet sorted out. That interface needs to be fixed first and we can then think of fio patching. Without a clear idea of what the kernel interface and its semantic will be, I do not think that commenting on your current code will be useful.

Quoted 18 lines

Ankit Kumar (2):
Add Zone Append command support
libaio: support for Zone Append command

Krishna Kanth Reddy (1):
t/zbd: Add support to verify Zone Append command with libaio tests

 HOWTO                  |  7 +++++++
 engines/libaio.c       | 32 ++++++++++++++++++++++++++++----
 fio.1                  |  7 +++++++
 ioengines.c            |  6 ++++++
 ioengines.h            |  3 ++-
 options.c              | 10 ++++++++++
 t/zbd/test-zbd-support | 29 +++++++++++++++++++++++++++++
 thread_options.h       |  2 ++
 8 files changed, 91 insertions(+), 5 deletions(-)
Signature · 2 lines
Damien Le Moal
Western Digital Research