Skip to contentSkip to conversation

v1 Patchset : Zone Explicit Open support

PATCH In review Krishna Kanth Reddy 2 participants 7 messages Sep 29, 2020 – Oct 6, 2020 1 message not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): CGME20200929130315epcas5p4e6a11cc77fb06a64c31e57740d2e0e31@epcas5p4.samsung.com

#1 lore raw

cover v1 Patchset : Zone Explicit Open support

1. Added a new FIO option zone_open.
When it is enabled, Zones will be opened explicitly before sending I/Os.
2. Add support for open_zones in libzbc I/O engine
3. t/zbd: Add support to verify Zone Explicit Open

This is a RFC.
Feedback / Comments will help in improving this further.

Ankit Kumar (2):
Add support for Explicit open zones
Add support for open_zones in libzbc I/O engine.

Krishna Kanth Reddy (1):
t/zbd: Add support to verify Zone Explicit Open

 HOWTO                       | 13 ++++++++
 engines/libzbc.c            | 42 +++++++++++++++++++++++++
 engines/skeleton_external.c | 12 ++++++++
 fio.1                       | 11 +++++++
 ioengines.h                 |  2 ++
 options.c                   | 20 ++++++++++++
 oslib/blkzoned.h            |  7 +++++
 oslib/linux-blkzoned.c      | 26 ++++++++++++++++
 t/zbd/test-zbd-support      | 27 ++++++++++++++++
 thread_options.h            |  7 +++++
 zbd.c                       | 61 +++++++++++++++++++++++++++++++++++++
 11 files changed, 228 insertions(+)
Signature · 1 line
2.17.1
#2 lore raw

PATCH 1/3 Add support for Explicit open zones

From: Ankit Kumar <[email protected]>

Added a new FIO option zone_open. When it is enabled, Zones will be opened explicitly before sending I/Os.

---
 HOWTO                       | 13 ++++++++
 engines/skeleton_external.c | 12 ++++++++
 fio.1                       | 11 +++++++
 ioengines.h                 |  2 ++
 options.c                   | 20 ++++++++++++
 oslib/blkzoned.h            |  7 +++++
 oslib/linux-blkzoned.c      | 26 ++++++++++++++++
 thread_options.h            |  7 +++++
 zbd.c                       | 61 +++++++++++++++++++++++++++++++++++++
 9 files changed, 159 insertions(+)
HOWTO +13 -0
index 2d8c7a02..320a1083 100644
@@ -1022,6 +1022,19 @@ Target file/device
 	:option:`zonesize` bytes of data have been transferred. This parameter
 	must be zero for :option:`zonemode` =zbd.
 
+.. option:: zone_open=str
+
+	This parameter applies to :option:`zonemode` =zbd only.
+
+	Accepted values are:
+
+		**implicit**
+				Zones are opened implicitly before sending
+				the I/Os.
+		**explicit**
+				Zones are opened explicitly before sending
+				the I/Os.
+
 .. option:: read_beyond_wp=bool
 
 	This parameter applies to :option:`zonemode` =zbd only.
engines/skeleton_external.c +12 -0
index 7f3e4cb3..828668b8 100644
@@ -193,6 +193,17 @@ static int fio_skeleton_reset_wp(struct thread_data *td, struct fio_file *f,
 	return 0;
 }
 
+/*
+ * Hook called for explicitly opening zones of a ZBD_HOST_AWARE or
+ * ZBD_HOST_MANAGED zoned block device. All the zones in the range
+ * @offset..@offset + @length must be opened.
+ */
+static int fio_skeleton_open_zones(struct thread_data *td, struct fio_file *f,
+				   uint64_t offset, uint64_t length)
+{
+	return 0;
+}
+
 /*
  * Note that the structure is exported, so that fio can get it via
  * dlsym(..., "ioengine"); for (and only for) external engines.
@@ -212,6 +223,7 @@ struct ioengine_ops ioengine = {
 	.get_zoned_model = fio_skeleton_get_zoned_model,
 	.report_zones	= fio_skeleton_report_zones,
 	.reset_wp	= fio_skeleton_reset_wp,
+	.open_zones	= fio_skeleton_open_zones,
 	.options	= options,
 	.option_struct_size	= sizeof(struct fio_skeleton_options),
 };
fio.1 +11 -0
index a881277c..de69f7af 100644
@@ -789,6 +789,17 @@ once a zone is fully written (write workloads) or all written data in the
 zone have been read (read workloads). This parameter is valid only for
 sequential workloads and ignored for random workloads. For read workloads,
 see also \fBread_beyond_wp\fR.
+.TP
+.BI zone_open\fR=\fPstr
+For \fBzonemode\fR=zbd the accepted values are
+.RS
+.TP
+.B explicit
+Zones are opened Explicitly before sending I/Os.
+.TP
+.B implicit
+Zones are opened Implicitly before sending I/Os.
+.RE
 
 .TP
 .BI read_beyond_wp \fR=\fPbool
ioengines.h +2 -0
index 54dadba2..8caac373 100644
@@ -57,6 +57,8 @@ struct ioengine_ops {
 			    uint64_t, struct zbd_zone *, unsigned int);
 	int (*reset_wp)(struct thread_data *, struct fio_file *,
 			uint64_t, uint64_t);
+	int (*open_zones)(struct thread_data *, const struct fio_file *,
+			  uint64_t, uint64_t);
 	int option_struct_size;
 	struct fio_option *options;
 };
options.c +20 -0
index b497d973..2b2bbb6e 100644
@@ -3332,6 +3332,26 @@ struct fio_option fio_options[FIO_MAX_OPTS] = {
 			   },
 		},
 	},
+	{
+		.name	= "zone_open",
+		.lname	= "zone_open",
+		.type	= FIO_OPT_STR,
+		.off1	= offsetof(struct thread_options, zone_open),
+		.help	= "Zone Open Mode (Implicit / Explicit) for Zoned Block Devices",
+		.def	= "implicit",
+		.category = FIO_OPT_C_IO,
+		.group	= FIO_OPT_G_ZONE,
+		.posval	= {
+			   { .ival = "implicit",
+			     .oval = ZONE_OPEN_IMPLICIT,
+			     .help = "Open Zones Implicitly",
+			   },
+			   { .ival = "explicit",
+			     .oval = ZONE_OPEN_EXPLICIT,
+			     .help = "Open Zones Explicitly",
+			   },
+		},
+	},
 	{
 		.name	= "zonesize",
 		.lname	= "Zone size",
oslib/blkzoned.h +7 -0
index 4cc071dc..9c1422d3 100644
@@ -16,6 +16,8 @@ extern int blkzoned_report_zones(struct thread_data *td,
 				struct zbd_zone *zones, unsigned int nr_zones);
 extern int blkzoned_reset_wp(struct thread_data *td, struct fio_file *f,
 				uint64_t offset, uint64_t length);
+extern int blkzoned_open_zone(struct thread_data *td, const struct fio_file *f,
+				uint64_t offset, uint64_t length);
 #else
 /*
  * Define stubs for systems that do not have zoned block device support.
@@ -44,6 +46,11 @@ static inline int blkzoned_reset_wp(struct thread_data *td, struct fio_file *f,
 {
 	return -EIO;
 }
+static inline int blkzoned_open_zone(struct thread_data *td, const struct fio_file *f,
+				     uint64_t offset, uint64_t length)
+{
+	return -EIO;
+}
 #endif
 
 #endif /* FIO_BLKZONED_H */
oslib/linux-blkzoned.c +26 -0
index 0a8a577a..f8cdce3b 100644
@@ -240,3 +240,29 @@ int blkzoned_reset_wp(struct thread_data *td, struct fio_file *f,
 
 	return ret;
 }
+
+int blkzoned_open_zone(struct thread_data *td, const struct fio_file *f,
+		       uint64_t offset, uint64_t length)
+{
+	struct blk_zone_range zr = {
+		.sector         = offset >> 9,
+		.nr_sectors     = length >> 9,
+	};
+	int fd, ret = 0;
+
+	/* If the file is not yet opened, open it for this function. */
+	fd = f->fd;
+	if (fd < 0) {
+		fd = open(f->file_name, O_RDWR | O_LARGEFILE);
+		if (fd < 0)
+			return -errno;
+	}
+
+	if (ioctl(fd, BLKOPENZONE, &zr) < 0)
+		return -errno;
+
+	if (f->fd < 0)
+		close(fd);
+
+	return ret;
+}
thread_options.h +7 -0
index 97c400fe..d8f352b7 100644
@@ -18,6 +18,11 @@ enum fio_zone_mode {
 	ZONE_MODE_ZBD		= 3,
 };
 
+enum fio_zone_open {
+	ZONE_OPEN_IMPLICIT	= 0, /* Open Zones Implicitly */
+	ZONE_OPEN_EXPLICIT	= 1, /* Open Zones Explicitly */
+};
+
 /*
  * What type of allocation to use for io buffers
  */
@@ -197,6 +202,7 @@ struct thread_options {
 	unsigned long long zone_capacity;
 	unsigned long long zone_skip;
 	enum fio_zone_mode zone_mode;
+	enum fio_zone_open zone_open;
 	unsigned long long lockmem;
 	enum fio_memtype mem_type;
 	unsigned int mem_align;
@@ -640,6 +646,7 @@ struct thread_options_pack {
 	uint32_t allow_mounted_write;
 
 	uint32_t zone_mode;
+	uint32_t zone_open;
 } __attribute__((packed));
 
 extern void convert_thread_options_to_cpu(struct thread_options *o, struct thread_options_pack *top);
zbd.c +61 -0

97 lines not on this page · raw message

Signature · 1 line
2.17.1
#3 lore raw

PATCH 2/3 Add support for open_zones in libzbc I/O engine.

In reply to Krishna Kanth Reddy

From: Ankit Kumar <[email protected]>

---
 engines/libzbc.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)
engines/libzbc.c +42 -0
index 4b900233..4b742d24 100644
@@ -334,6 +334,47 @@ err:
 	return -ret;
 }
 
+static int libzbc_open_zones(struct thread_data *td, const struct fio_file *f,
+			     uint64_t offset, uint64_t length)
+{
+	struct libzbc_data *ld = td->io_ops_data;
+	uint64_t sector = offset >> 9;
+	uint64_t end_sector = (offset + length) >> 9;
+	unsigned int nr_zones;
+	struct zbc_errno err;
+	int i, ret;
+
+	assert(ld);
+	assert(ld->zdev);
+
+	nr_zones = (length + td->o.zone_size - 1) / td->o.zone_size;
+	if (!sector && end_sector >= ld->nr_sectors) {
+		/* Open all zones */
+		ret = zbc_open_zone(ld->zdev, 0, ZBC_OP_ALL_ZONES);
+		if (ret)
+			goto err;
+
+		return 0;
+	}
+
+	for (i = 0; i < nr_zones; i++, sector += td->o.zone_size >> 9) {
+		ret = zbc_open_zone(ld->zdev, sector, 0);
+		if (ret)
+			goto err;
+	}
+
+	return 0;
+
+err:
+	zbc_errno(ld->zdev, &err);
+	td_verror(td, errno, "zbc_open_zone failed");
+	if (err.sk)
+		log_err("%s: open failed %s:%s\n",
+			f->file_name,
+			zbc_sk_str(err.sk), zbc_asc_ascq_str(err.asc_ascq));
+	return -ret;
+}
+
 ssize_t libzbc_rw(struct thread_data *td, struct io_u *io_u)
 {
 	struct libzbc_data *ld = td->io_ops_data;
@@ -413,6 +454,7 @@ FIO_STATIC struct ioengine_ops ioengine = {
 	.get_zoned_model	= libzbc_get_zoned_model,
 	.report_zones		= libzbc_report_zones,
 	.reset_wp		= libzbc_reset_wp,
+	.open_zones		= libzbc_open_zones,
 	.queue			= libzbc_queue,
 	.flags			= FIO_SYNCIO | FIO_NOEXTEND | FIO_RAWIO,
 };
Signature · 1 line
2.17.1
#4 lore raw

PATCH 3/3 t/zbd: Add support to verify Zone Explicit Open

In reply to Krishna Kanth Reddy

Modify the test-zbd-support script to verify the Zone Explicit Open. Added a new FIO option zone_open.
When zone_open option is set to explicit, zones are opened explicitly. If it is set to implicit, zones are opened implicitly. Default value is implicit for the zone_open option.

---
 t/zbd/test-zbd-support | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
t/zbd/test-zbd-support +27 -0
index 248423bb..c1f35165 100755
@@ -948,6 +948,33 @@ test49() {
     check_read $((capacity * 2)) || return $?
 }
 
+# Explicit zone open to sequential zones, libaio, 1 job, queue depth 1
+test50() {
+    local size capacity
+
+    size=$((8 * zone_size))
+    off=$((first_sequential_zone_sector * 512))
+    capacity=$(total_zone_capacity 8 $off $dev)
+
+    run_fio_on_seq --ioengine=libaio --iodepth=1 --rw=write --zone_open=explicit \
+                   --bs=65536 --size=$size --do_verify=1 --verify=md5 \
+                   >>"${logfile}.${test_number}" 2>&1 || return $?
+    check_written $capacity || return $?
+    check_read $capacity || return $?
+}
+
+# Random zone write to sequential zones, libaio, 8 jobs, queue depth 64 per job, Explicit open
+test51() {
+    local capacity
+
+    off=$((first_sequential_zone_sector * 512))
+    capacity=$(total_zone_capacity 4 $off $dev)
+    run_fio_on_seq --ioengine=libaio --iodepth=64 --rw=randwrite \
+		   --bs=4096 --group_reporting=1 --numjobs=8 --zone_open=explicit \
+                   >> "${logfile}.${test_number}" 2>&1 || return $?
+    check_written $((capacity * 8)) || return $?
+}
+
 tests=()
 dynamic_analyzer=()
 reset_all_zones=
Signature · 1 line
2.17.1
#5 lore raw
In reply to Krishna Kanth Reddy
Quoted 7 lines · Krishna Kanth Reddy

1. Added a new FIO option zone_open.
When it is enabled, Zones will be opened explicitly before sending I/Os.
2. Add support for open_zones in libzbc I/O engine
3. t/zbd: Add support to verify Zone Explicit Open

This is a RFC.
Feedback / Comments will help in improving this further.

My main question is why ? There is no performance benefit on the drive side from doing explicit zone open. On the contrary, performance may go down due to the added command overhead. So why do this ? What is your goal ?

Using the max_open_zones=N option, fio writes will be limited to a set of N zones, implicitly open, until the zones are full (not open anymore) and new zones chosen (becoming implicitly open with writes). Adding explicit zone open will not give you anything more with this option used. And if you do not use it, performance with random writes will significantly go down.

So please try to give convincing reasons for this.

Quoted 21 lines

Ankit Kumar (2):
Add support for Explicit open zones
Add support for open_zones in libzbc I/O engine.

Krishna Kanth Reddy (1):
t/zbd: Add support to verify Zone Explicit Open

 HOWTO                       | 13 ++++++++
 engines/libzbc.c            | 42 +++++++++++++++++++++++++
 engines/skeleton_external.c | 12 ++++++++
 fio.1                       | 11 +++++++
 ioengines.h                 |  2 ++
 options.c                   | 20 ++++++++++++
 oslib/blkzoned.h            |  7 +++++
 oslib/linux-blkzoned.c      | 26 ++++++++++++++++
 t/zbd/test-zbd-support      | 27 ++++++++++++++++
 thread_options.h            |  7 +++++
 zbd.c                       | 61 +++++++++++++++++++++++++++++++++++++
 11 files changed, 228 insertions(+)
Signature · 2 lines
Damien Le Moal
Western Digital Research
#6 lore raw
Quoted 13 lines · Damien Le Moal
Quoted 7 lines · Krishna Kanth Reddy

1. Added a new FIO option zone_open.
When it is enabled, Zones will be opened explicitly before sending I/Os.
2. Add support for open_zones in libzbc I/O engine
3. t/zbd: Add support to verify Zone Explicit Open

This is a RFC.
Feedback / Comments will help in improving this further.

My main question is why ? There is no performance benefit on the drive side from doing explicit zone open. On the contrary, performance may go down due to the added command overhead. So why do this ? What is your goal ?

Quoted 8 lines

Using the max_open_zones=N option, fio writes will be limited to a set of N zones, implicitly open, until the zones are full (not open anymore) and new zones chosen (becoming implicitly open with writes). Adding explicit zone open will not give you anything more with this option used. And if you do not use it, performance with random writes will significantly go down.

So please try to give convincing reasons for this.

It is a building block for future TPs.
We would like to do the pre-work and add the right option abstraction. Zone open is 'implicit' by default.
Hence there won't be a performance drop with the newly added code.

Quoted 27 lines
Quoted 21 lines

Ankit Kumar (2):
Add support for Explicit open zones
Add support for open_zones in libzbc I/O engine.

Krishna Kanth Reddy (1):
t/zbd: Add support to verify Zone Explicit Open

 HOWTO                       | 13 ++++++++
 engines/libzbc.c            | 42 +++++++++++++++++++++++++
 engines/skeleton_external.c | 12 ++++++++
 fio.1                       | 11 +++++++
 ioengines.h                 |  2 ++
 options.c                   | 20 ++++++++++++
 oslib/blkzoned.h            |  7 +++++
 oslib/linux-blkzoned.c      | 26 ++++++++++++++++
 t/zbd/test-zbd-support      | 27 ++++++++++++++++
 thread_options.h            |  7 +++++
 zbd.c                       | 61 +++++++++++++++++++++++++++++++++++++
 11 files changed, 228 insertions(+)
Signature · 2 lines
Damien Le Moal
Western Digital Research
#7 lore raw
Quoted 20 lines · Krishna Kanth Reddy
Quoted 19 lines · Damien Le Moal
Quoted 7 lines · Krishna Kanth Reddy

1. Added a new FIO option zone_open.
When it is enabled, Zones will be opened explicitly before sending I/Os.
2. Add support for open_zones in libzbc I/O engine
3. t/zbd: Add support to verify Zone Explicit Open

This is a RFC.
Feedback / Comments will help in improving this further.

My main question is why ? There is no performance benefit on the drive side from doing explicit zone open. On the contrary, performance may go down due to the added command overhead. So why do this ? What is your goal ?

Using the max_open_zones=N option, fio writes will be limited to a set of N zones, implicitly open, until the zones are full (not open anymore) and new zones chosen (becoming implicitly open with writes). Adding explicit zone open will not give you anything more with this option used. And if you do not use it, performance with random writes will significantly go down.

Quoted 3 lines
Quoted 2 lines

So please try to give convincing reasons for this.

It is a building block for future TPs.

Which one ? Without that justification, I do not see the point of introducing this change right now.

Quoted 3 lines

We would like to do the pre-work and add the right option abstraction. Zone open is 'implicit' by default.
Hence there won't be a performance drop with the newly added code.

Sure, but there will be a drop for regular zone block devices with some workloads when that option is used. So if explicit zone open is needed by some future TP feature, it may be better to integrate explicit open with that feature rather than making it a generic fio option that users may mistakenly enable and end up wondering why they see lower performance with their device...

Quoted 28 lines
Quoted 28 lines
Quoted 22 lines

Ankit Kumar (2):
Add support for Explicit open zones
Add support for open_zones in libzbc I/O engine.

Krishna Kanth Reddy (1):
t/zbd: Add support to verify Zone Explicit Open

 HOWTO                       | 13 ++++++++
 engines/libzbc.c            | 42 +++++++++++++++++++++++++
 engines/skeleton_external.c | 12 ++++++++
 fio.1                       | 11 +++++++
 ioengines.h                 |  2 ++
 options.c                   | 20 ++++++++++++
 oslib/blkzoned.h            |  7 +++++
 oslib/linux-blkzoned.c      | 26 ++++++++++++++++
 t/zbd/test-zbd-support      | 27 ++++++++++++++++
 thread_options.h            |  7 +++++
 zbd.c                       | 61 +++++++++++++++++++++++++++++++++++++
 11 files changed, 228 insertions(+)
Signature · 2 lines
Damien Le Moal
Western Digital Research
Signature · 2 lines
Damien Le Moal
Western Digital Research