three patches

From: Vincent Fu <[email protected]>

Jens, please consider this patch series.

The LFSR error message and test script error handling changes I believe should be uncontroversial.

Regarding the zonemode=strided patch, it will break job files that depend on the old buggy behavior, but that behavior is so inconsistent with the documentation that I think this fix is warranted.

Vincent

Vincent Fu (3):
filesetup: improve LFSR init failure error message
io_u: move to next zone even if zoneskip is unset t/run-fio-tests: improve error handling

 filesetup.c        |  3 +++
 io_u.c             |  3 +--
 t/run-fio-tests.py | 10 ++++++----
 t/strided.py       |  3 +--
 4 files changed, 11 insertions(+), 8 deletions(-)

2.17.1

From: Vincent Fu <[email protected]>

Especially with small sample spaces, the LFSR random generator occasionally fails to initialize successfully. When this occurs, the error message refers to problems allocating a random map. Change the error message to explicitly mention the LFSR failure.

OLD BEHAVIOR

$ ./fio --name=test --ioengine=null --size=4k --random_generator=lfsr --rw=randread --randrepeat=0
test: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=null, iodepth=1
fio-3.16-13-g998b-dirty
Starting 1 process
fio: failed allocating random map. If running a large number of jobs, try the 'norandommap' option or set 'softrandommap'. Or give a larger --alloc-size to fio.

NEW BEHAVIOR

$ ./fio --name=test --ioengine=null --size=4k --random_generator=lfsr --rw=randread --randrepeat=0
test: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=null, iodepth=1
fio-3.16-37-g65ee
Starting 1 process
fio: failed initializing LFSR
---
 filesetup.c | 3 +++
 1 file changed, 3 insertions(+)

index 1d3094c1..7d54c9f1 100644
@@ -1354,6 +1354,9 @@ bool init_random_map(struct thread_data *td)
 			if (!lfsr_init(&f->lfsr, blocks, seed, 0)) {
 				fio_file_set_lfsr(f);
 				continue;
+			} else {
+				log_err("fio: failed initializing LFSR\n");
+				return false;
 			}
 		} else if (!td->o.norandommap) {
 			f->io_axmap = axmap_new(blocks);

2.17.1

From: Vincent Fu <[email protected]>

This patch makes fio's behavior under zonemode=strided conform to the documentation:

I/O happens in a single zone until zonesize bytes have been transferred. After that number of bytes has been transferred processing of the next zone starts.

With zonemode=strided, before commit 35f561eb, fio would only move to the next zone when zoneskip > 0 and zonesize bytes were written. There would always be zoneskip bytes between the end of one zone and the beginning of the next zone. If zoneskip was not set or set to 0, all IO would happen in the first zone.

Commit 35f561eb changed this so that fio would move to the next zone upon writing zonesize bytes if zoneskip was explicitly set to a value >=
0. This option made it possible for zones to be contiguous. The documentation was not updated to reflect the new behavior.

I originally intended to submit a patch to update fio's documentation, but upon further reflection it seems better to change fio's behavior and have a clean user interface than to change the documentation to note that zoneskip must be explciitly set in order for fio to move to the next zone.

This patch also updates t/strided.py to reflect the new behavior.

---
 io_u.c       | 3 +--
 t/strided.py | 3 +--
 2 files changed, 2 insertions(+), 4 deletions(-)

index 5cbbe85a..b5c31335 100644
@@ -850,8 +850,7 @@ static void setup_strided_zone_mode(struct thread_data *td, struct io_u *io_u)
 	/*
 	 * See if it's time to switch to a new zone
 	 */
-	if (td->zone_bytes >= td->o.zone_size &&
-			fio_option_is_set(&td->o, zone_skip)) {
+	if (td->zone_bytes >= td->o.zone_size) {
 		td->zone_bytes = 0;
 		f->file_offset += td->o.zone_range + td->o.zone_skip;
 

index c159dc0b..aac15d10 100755
@@ -22,7 +22,7 @@
 #
 # ===TEST MATRIX===
 #
-# --zonemode=strided, zoneskip >= 0
+# --zonemode=strided, zoneskip unset
 #   w/ randommap and LFSR
 #       zonesize=zonerange  all blocks in zonerange touched
 #       zonesize>zonerange  all blocks touched and roll-over back into zone

@@ -57,7 +57,6 @@ def run_fio(fio, test, index):
                 "--log_offset=1",
                 "--randrepeat=0",
                 "--rw=randread",
-                "--zoneskip=0",
                 "--write_iops_log={0}{1:03d}".format(filename, index),
                 "--output={0}{1:03d}.out".format(filename, index),
                 "--zonerange={zonerange}".format(**test),

2.17.1

From: Vincent Fu <[email protected]>

More gracefully handle errors when running a test. Also change the EXAMPLE to refer to fio's canonical git repository.

---
 t/run-fio-tests.py | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

index 1b8ca0a2..cf8de093 100755
@@ -14,7 +14,7 @@
 #
 #
 # EXAMPLE
-# # git clone [fio-repository]
+# # git clone git://git.kernel.dk/fio.git
 # # cd fio
 # # make -j
 # # python3 t/run-fio-tests.py

@@ -123,6 +123,7 @@ class FioExeTest(FioTest):
         stderr_file = open(self.stderr_file, "w+")
         exticode_file = open(self.exticode_file, "w+")
         try:
+            proc = None
             # Avoid using subprocess.run() here because when a timeout occurs,
             # fio will be stopped with SIGKILL. This does not give fio a
             # chance to clean up and means that child processes may continue

@@ -142,9 +143,10 @@ class FioExeTest(FioTest):
             assert proc.poll()
             self.output['failure'] = 'timeout'
         except Exception:
-            if not proc.poll():
-                proc.terminate()
-                proc.communicate()
+            if proc:
+                if not proc.poll():
+                    proc.terminate()
+                    proc.communicate()
             self.output['failure'] = 'exception'
             self.output['exc_info'] = sys.exc_info()
         finally:

2.17.1

Agree, and I don't think that's a high risk.

Applied, thanks.

Jens Axboe