Skip to contentSkip to conversation

nbd: reduce max_block restrictions

PATCH Superseded Vladimir Sementsov-Ogievskiy 2 participants 6 messages Jun 10, 2020 Block layer

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover nbd: reduce max_block restrictions

Recent changes in NBD protocol allowed to use some commands without max_block restriction. Let's drop the restrictions.

NBD change is here:
https://github.com/NetworkBlockDevice/nbd/commit/9f30fedb8699f151e7ef4ccc07e624330be3316b#diff-762fb7c670348da69cc9050ef58fe3ae

v3: first two patches from v2 was merged. Let's continue with the rest.

Vladimir Sementsov-Ogievskiy (4):
block: add max_pwrite_zeroes_fast to BlockLimits block/nbd: define new max_write_zero_fast limit block/io: refactor bdrv_co_do_pwrite_zeroes head calculation block/io: auto-no-fallback for write-zeroes

 include/block/block_int.h |  8 ++++++++
 block/io.c                | 39 +++++++++++++++++++++++++++++++++------
 block/nbd.c               |  1 +
 3 files changed, 42 insertions(+), 6 deletions(-)
Signature · 1 line
2.21.0
#1 lore raw

PATCH v3 1/4 block: add max_pwrite_zeroes_fast to BlockLimits

The NBD spec was recently updated to clarify that max_block doesn't relate to NBD_CMD_WRITE_ZEROES with NBD_CMD_FLAG_FAST_ZERO (which mirrors Qemu flag BDRV_REQ_NO_FALLBACK). To drop the restriction we need new max_pwrite_zeroes_fast.

Default value of new max_pwrite_zeroes_fast is zero and it means use max_pwrite_zeroes. So this commit semantically changes nothing.

---
 include/block/block_int.h |  8 ++++++++
 block/io.c                | 17 ++++++++++++-----
 2 files changed, 20 insertions(+), 5 deletions(-)
include/block/block_int.h +8 -0
index 791de6a59c..277e32fe31 100644
@@ -626,6 +626,14 @@ typedef struct BlockLimits {
      * pwrite_zeroes_alignment. May be 0 if no inherent 32-bit limit */
     int32_t max_pwrite_zeroes;
 
+    /*
+     * Maximum number of bytes that can zeroed at once if flag
+     * BDRV_REQ_NO_FALLBACK specified. Must be multiple of
+     * pwrite_zeroes_alignment.
+     * If 0, max_pwrite_zeroes is used for no-fallback case.
+     */
+    int64_t max_pwrite_zeroes_fast;
+
     /* Optimal alignment for write zeroes requests in bytes. A power
      * of 2 is best but not mandatory.  Must be a multiple of
      * bl.request_alignment, and must be less than max_pwrite_zeroes
block/io.c +12 -5
index df8f2a98d4..0af62a53fd 100644
@@ -1774,12 +1774,13 @@ static int coroutine_fn bdrv_co_do_pwrite_zeroes(BlockDriverState *bs,
     bool need_flush = false;
     int head = 0;
     int tail = 0;
-
-    int max_write_zeroes = MIN_NON_ZERO(bs->bl.max_pwrite_zeroes, INT_MAX);
+    int max_write_zeroes;
     int alignment = MAX(bs->bl.pwrite_zeroes_alignment,
                         bs->bl.request_alignment);
     int max_transfer = MIN_NON_ZERO(bs->bl.max_transfer, MAX_BOUNCE_BUFFER);
 
+    assert(alignment % bs->bl.request_alignment == 0);
+
     if (!drv) {
         return -ENOMEDIUM;
     }
@@ -1788,12 +1789,18 @@ static int coroutine_fn bdrv_co_do_pwrite_zeroes(BlockDriverState *bs,
         return -ENOTSUP;
     }
 
-    assert(alignment % bs->bl.request_alignment == 0);
-    head = offset % alignment;
-    tail = (offset + bytes) % alignment;
+    if ((flags & BDRV_REQ_NO_FALLBACK) && bs->bl.max_pwrite_zeroes_fast) {
+        max_write_zeroes = bs->bl.max_pwrite_zeroes_fast;
+    } else {
+        max_write_zeroes = bs->bl.max_pwrite_zeroes;
+    }
+    max_write_zeroes = MIN_NON_ZERO(max_write_zeroes, INT_MAX);
     max_write_zeroes = QEMU_ALIGN_DOWN(max_write_zeroes, alignment);
     assert(max_write_zeroes >= bs->bl.request_alignment);
 
+    head = offset % alignment;
+    tail = (offset + bytes) % alignment;
+
     while (bytes > 0 && !ret) {
         int num = bytes;
 
Signature · 1 line
2.21.0
#2 lore raw

PATCH v3 2/4 block/nbd: define new max_write_zero_fast limit

In reply to Vladimir Sementsov-Ogievskiy

The NBD spec was recently updated to clarify that max_block doesn't relate to NBD_CMD_WRITE_ZEROES with NBD_CMD_FLAG_FAST_ZERO (which mirrors Qemu flag BDRV_REQ_NO_FALLBACK).

bs->bl.max_write_zero_fast is zero by default which means using max_pwrite_zeroes. Update nbd driver to allow larger requests with BDRV_REQ_NO_FALLBACK.

---
 block/nbd.c | 1 +
 1 file changed, 1 insertion(+)
block/nbd.c +1 -0
index 4ac23c8f62..b0584cf68d 100644
@@ -1956,6 +1956,7 @@ static void nbd_refresh_limits(BlockDriverState *bs, Error **errp)
 
     bs->bl.request_alignment = min;
     bs->bl.max_pdiscard = QEMU_ALIGN_DOWN(INT_MAX, min);
+    bs->bl.max_pwrite_zeroes_fast = bs->bl.max_pdiscard;
     bs->bl.max_pwrite_zeroes = max;
     bs->bl.max_transfer = max;
 
Signature · 1 line
2.21.0
#3 lore raw

PATCH v3 3/4 block/io: refactor bdrv_co_do_pwrite_zeroes head calculation

In reply to Vladimir Sementsov-Ogievskiy

It's wrong to update head using num in this place, as num may be reduced during the iteration (seems it doesn't, but it's not obvious), and we'll have wrong head value on next iteration.

Instead update head at iteration end.

---
 block/io.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
block/io.c +3 -1
index 0af62a53fd..3fae97da2d 100644
@@ -1813,7 +1813,6 @@ static int coroutine_fn bdrv_co_do_pwrite_zeroes(BlockDriverState *bs,
              * convenience, limit this request to max_transfer even if
              * we don't need to fall back to writes.  */
             num = MIN(MIN(bytes, max_transfer), alignment - head);
-            head = (head + num) % alignment;
             assert(num < max_write_zeroes);
         } else if (tail && num > alignment) {
             /* Shorten the request to the last aligned sector.  */
@@ -1872,6 +1871,9 @@ static int coroutine_fn bdrv_co_do_pwrite_zeroes(BlockDriverState *bs,
 
         offset += num;
         bytes -= num;
+        if (head) {
+            head = offset % alignment;
+        }
     }
 
 fail:
Signature · 1 line
2.21.0
#4 lore raw

PATCH v3 4/4 block/io: auto-no-fallback for write-zeroes

In reply to Vladimir Sementsov-Ogievskiy

When BDRV_REQ_NO_FALLBACK is supported, the NBD driver supports a larger request size. Add code to try large zero requests with a NO_FALLBACK request prior to having to split a request into chunks according to max_pwrite_zeroes.

---
 block/io.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
block/io.c +18 -0
index 3fae97da2d..9a6dabb595 100644
@@ -1778,6 +1778,7 @@ static int coroutine_fn bdrv_co_do_pwrite_zeroes(BlockDriverState *bs,
     int alignment = MAX(bs->bl.pwrite_zeroes_alignment,
                         bs->bl.request_alignment);
     int max_transfer = MIN_NON_ZERO(bs->bl.max_transfer, MAX_BOUNCE_BUFFER);
+    bool auto_no_fallback;
 
     assert(alignment % bs->bl.request_alignment == 0);
 
@@ -1785,6 +1786,16 @@ static int coroutine_fn bdrv_co_do_pwrite_zeroes(BlockDriverState *bs,
         return -ENOMEDIUM;
     }
 
+    if (!(flags & BDRV_REQ_NO_FALLBACK) &&
+        (bs->supported_zero_flags & BDRV_REQ_NO_FALLBACK) &&
+        bs->bl.max_pwrite_zeroes && bs->bl.max_pwrite_zeroes < bytes &&
+        bs->bl.max_pwrite_zeroes < bs->bl.max_pwrite_zeroes_fast)
+    {
+        assert(drv->bdrv_co_pwrite_zeroes);
+        flags |= BDRV_REQ_NO_FALLBACK;
+        auto_no_fallback = true;
+    }
+
     if ((flags & ~bs->supported_zero_flags) & BDRV_REQ_NO_FALLBACK) {
         return -ENOTSUP;
     }
@@ -1829,6 +1840,13 @@ static int coroutine_fn bdrv_co_do_pwrite_zeroes(BlockDriverState *bs,
         if (drv->bdrv_co_pwrite_zeroes) {
             ret = drv->bdrv_co_pwrite_zeroes(bs, offset, num,
                                              flags & bs->supported_zero_flags);
+            if (ret == -ENOTSUP && auto_no_fallback) {
+                flags &= ~BDRV_REQ_NO_FALLBACK;
+                max_write_zeroes =
+                    QEMU_ALIGN_DOWN(MIN_NON_ZERO(bs->bl.max_pwrite_zeroes,
+                                                 INT_MAX), alignment);
+                continue;
+            }
             if (ret != -ENOTSUP && (flags & BDRV_REQ_FUA) &&
                 !(bs->supported_zero_flags & BDRV_REQ_FUA)) {
                 need_flush = true;
Signature · 1 line
2.21.0
#5 lore raw
In reply to Vladimir Sementsov-Ogievskiy
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1 === TEST SCRIPT END ===

tmp/qemu-test/build/tests/qemu-iotests/251.out.bad 2020-06-10 20:24:40.007412790 +0000 renamed from tmp/qemu-test/src/tests/qemu-iotests/251.out 2020-06-10 18:56:36.000000000 +0000 +0 -3
@@ -18,26 +18,16 @@
 qemu-img: warning: error while reading offset read_fail_offset_8: Input/output error
 qemu-img: warning: error while reading offset read_fail_offset_9: Input/output error
 
-wrote 512/512 bytes at offset read_fail_offset_0
-512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
---

Not run: 259
Failures: 033 034 154 177 251
Failed 5 of 119 iotests
make: *** [check-tests/check-block.sh] Error 1
make: *** Waiting for unfinished jobs....
TEST check-qtest-aarch64: tests/qtest/qos-test
Traceback (most recent call last):
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=faa1edfc69684422b314343cf30174a5', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-9egury9q/src/docker-src.2020-06-10-16.12.07.11584:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2. filter=--filter=label=com.qemu.instance.uuid=faa1edfc69684422b314343cf30174a5
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-9egury9q/src'
make: *** [docker-run-test-quick@centos7] Error 2

real 13m12.298s
user 0m8.670s

The full log is available at
http://patchew.org/logs/[email protected]/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]