Skip to contentSkip to conversation

nbd: accept a large NBD_CMD_CACHE, cover the transmission commands

PATCH Superseded Denis V. Lunev 3 participants 13 messages Aug 26 – Aug 27

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover nbd: accept a large NBD_CMD_CACHE, cover the transmission commands

NBD_CMD_CACHE carries no payload in either direction, yet the server rejects any effect length above the advertised maximum payload size. The specification explicitly allows the oversize request, and we already accept it for NBD_CMD_TRIM and NBD_CMD_WRITE_ZEROES. Patch 2 drops the check.

The rest of the series adds tests/qemu-iotests/tests/nbd-commands, a small client which speaks the NBD transmission phase itself. Nothing in the tree does that today, so the protocol surface of the server - reply modes, command flags, error codes, sparse and extended replies - is covered only as far as our own client happens to exercise it. It is useful both as coverage and as a regression tool.

CC Eric Blake <[email protected]>
CC Vladimir Sementsov-Ogievskiy <[email protected]>

Denis V. Lunev (5):
iotests: add coverage for NBD transmission commands nbd/server: accept NBD_CMD_CACHE above the maximum payload size iotests/nbd-commands: exercise the simple and structured reply modes iotests/nbd-commands: cover NBD_CMD_BLOCK_STATUS with a payload iotests/nbd-commands: cover the command flags and sparse replies

 nbd/server.c                              |   4 +-
 tests/qemu-iotests/tests/nbd-commands     | 269 ++++++++++++++++++++++
 tests/qemu-iotests/tests/nbd-commands.out |   5 +
 3 files changed, 275 insertions(+), 3 deletions(-)
 create mode 100755 tests/qemu-iotests/tests/nbd-commands
 create mode 100644 tests/qemu-iotests/tests/nbd-commands.out

base-commit: bde2492aace2b5acb755a5b057013e915163a77f

Signature · 1 line
2.53.0
#1 lore raw

PATCH 1/5 iotests: add coverage for NBD transmission commands

From: Denis V. Lunev <[email protected]>

NBD_CMD_CACHE has no coverage anywhere in the tree. Nothing ever sends it: our own NBD client does not implement the command, and neither qemu-io nor 'qemu-nbd --list' can issue one, so the only clients reaching this server path are external ones.

Add a test driven by libnbd, gated the way nbd-multiconn already is, and start it with the case the command exists for. The export is a qcow2 image over a fully written backing file, so a prefetch has visible work to do.

CC Eric Blake <[email protected]>
CC Vladimir Sementsov-Ogievskiy <[email protected]>
---
 tests/qemu-iotests/tests/nbd-commands     | 136 ++++++++++++++++++++++
 tests/qemu-iotests/tests/nbd-commands.out |   5 +
 2 files changed, 141 insertions(+)
 create mode 100755 tests/qemu-iotests/tests/nbd-commands
 create mode 100644 tests/qemu-iotests/tests/nbd-commands.out
tests/qemu-iotests/tests/nbd-commands new file +136 -0
new file mode 100755
index 0000000000..73baeb236f
@@ -0,0 +1,136 @@
+#!/usr/bin/env python3
+# group: rw auto quick
+#
+# Test NBD transmission commands against a qemu NBD export
+#
+# Copyright (C) 2026 Virtuozzo International GmbH
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+import os
+from contextlib import contextmanager
+from types import ModuleType
+
+import iotests
+from iotests import qemu_img_create, qemu_img_map, qemu_io
+
+
+base = os.path.join(iotests.test_dir, 'base')
+top = os.path.join(iotests.test_dir, 'top')
+# Larger than the maximum payload size an export can advertise
+size = 64 * 1024 * 1024
+pattern = 0xa5
+nbd_sock = os.path.join(iotests.sock_dir, 'nbd_sock')
+nbd_uri = 'nbd+unix:///exp?socket=' + nbd_sock
+nbd: ModuleType
+
+DEPTH_LOCAL = 1
+DEPTH_BACKING = 2
+
+
+class TestNbdCommands(iotests.QMPTestCase):
+    def setUp(self):
+        qemu_img_create('-f', iotests.imgfmt, base, str(size))
+        qemu_io('-c', f'write -P {pattern} 0 {size}', base)
+        qemu_img_create('-f', iotests.imgfmt, '-b', base,
+                        '-F', iotests.imgfmt, top, str(size))
+
+        self.vm = iotests.VM()
+        self.vm.launch()
+        self.vm.cmd('blockdev-add', {
+            'driver': iotests.imgfmt,
+            'node-name': 'n',
+            'file': {'driver': 'file', 'filename': top},
+            'backing': {
+                'driver': iotests.imgfmt,
+                'node-name': 'base',
+                'file': {'driver': 'file', 'filename': base},
+            },
+        })
+        self.vm.cmd('nbd-server-start', {
+            'addr': {'type': 'unix', 'data': {'path': nbd_sock}}
+        })
+        self.vm.cmd('block-export-add', {
+            'type': 'nbd',
+            'id': 'exp',
+            'node-name': 'n',
+            'name': 'exp',
+            'writable': True,
+            'allocation-depth': True,
+        })
+
+    def tearDown(self):
+        self.vm.shutdown()
+        for f in (top, base, nbd_sock):
+            try:
+                os.remove(f)
+            except OSError:
+                pass
+
+    @contextmanager
+    def open_nbd(self, structured=True, extended=True):
+        h = nbd.NBD()
+        h.set_request_structured_replies(structured)
+        h.set_request_extended_headers(extended)
+        h.add_meta_context('base:allocation')
+        h.add_meta_context('qemu:allocation-depth')
+        # Let the server, not libnbd, reject the out of range requests below
+        h.set_strict_mode(h.get_strict_mode() &
+                          ~(nbd.STRICT_BOUNDS | nbd.STRICT_PAYLOAD))
+        try:
+            h.connect_uri(nbd_uri)
+            self.assertEqual(h.get_structured_replies_negotiated(), structured)
+            self.assertEqual(h.get_extended_headers_negotiated(), extended)
+            yield h
+        finally:
+            h.shutdown()
+
+    def allocation_depth(self, h):
+        extents = []
+
+        def cb(meta, offset, entries, err):
+            if meta == 'qemu:allocation-depth':
+                extents.extend(zip(entries[0::2], entries[1::2]))
+
+        h.block_status(size, 0, cb)
+        return extents
+
+    def top_extents(self):
+        """Which parts of the top image are local, once qemu has let go."""
+        self.vm.shutdown()
+        return [(e['start'], e['length'], e['depth'])
+                for e in qemu_img_map(top)]
+
+    def test_cache_copies_on_read(self):
+        with self.open_nbd() as h:
+            maximum = h.get_block_size(nbd.SIZE_MAXIMUM)
+            self.assertLess(maximum, size)
+            self.assertEqual(self.allocation_depth(h),
+                             [(size, DEPTH_BACKING)])
+
+            h.cache(maximum, 0)
+
+        self.assertEqual(self.top_extents(),
+                         [(0, maximum, 0), (maximum, size - maximum, 1)])
+        qemu_io('-c', f'read -P {pattern} 0 {size}', top)
+
+    def test_cache_past_end_of_export(self):
+        with self.open_nbd() as h:
+            self.assertRaises(nbd.Error, h.cache, size + 1, 0)
+
+    def test_read_bound_by_max_payload(self):
+        with self.open_nbd() as h:
+            maximum = h.get_block_size(nbd.SIZE_MAXIMUM)
+            self.assertRaises(nbd.Error, h.pread, maximum + 65536, 0)
+
+
+if __name__ == '__main__':
+    try:
+        # Easier to use libnbd than to try and set up parallel
+        # 'qemu-nbd --list' or 'qemu-io' processes, but not all systems
+        # have libnbd installed.
+        import nbd  # type: ignore
+
+        iotests.main(supported_fmts=['qcow2'])
+    except ImportError:
+        iotests.notrun('Python bindings to libnbd are not installed')
tests/qemu-iotests/tests/nbd-commands.out new file +5 -0
new file mode 100644
index 0000000000..8d7e996700
@@ -0,0 +1,5 @@
+...
+----------------------------------------------------------------------
+Ran 3 tests
+
+OK
Signature · 1 line
2.53.0
Reviewed-by on v1 1/5
#2 lore raw

Re: [PATCH 1/5] iotests: add coverage for NBD transmission commands

Quoted 7 lines · Denis V. Lunev

From: Denis V. Lunev<[email protected]>

NBD_CMD_CACHE has no coverage anywhere in the tree. Nothing ever sends it: our own NBD client does not implement the command, and neither qemu-io nor 'qemu-nbd --list' can issue one, so the only clients reaching this server path are external ones.

Quoted 8 lines

Add a test driven by libnbd, gated the way nbd-multiconn already is, and start it with the case the command exists for. The export is a qcow2 image over a fully written backing file, so a prefetch has visible work to do.

CC Eric Blake<[email protected]>
CC Vladimir Sementsov-Ogievskiy<[email protected]>
Signature · 2 lines
Best regards,
Vladimir
#3 lore raw

Re: [PATCH 1/5] iotests: add coverage for NBD transmission commands

In reply to Denis V. Lunev
Quoted 8 lines · Denis V. Lunev
tests/qemu-iotests/tests/nbd-commands new file +8 -0
+
+    @contextmanager
+    def open_nbd(self, structured=True, extended=True):
+        h = nbd.NBD()
+        h.set_request_structured_replies(structured)
+        h.set_request_extended_headers(extended)
+        h.add_meta_context('base:allocation')
+        h.add_meta_context('qemu:allocation-depth')

tests/qemu-iotests/tests/nbd-commands line 86

Quoted 10 lines
tests/qemu-iotests/tests/nbd-commands new file +10 -0
+        # Let the server, not libnbd, reject the out of range requests below
+        h.set_strict_mode(h.get_strict_mode() &
+                          ~(nbd.STRICT_BOUNDS | nbd.STRICT_PAYLOAD))
+        try:
+            h.connect_uri(nbd_uri)
+            self.assertEqual(h.get_structured_replies_negotiated(), structured)
+            self.assertEqual(h.get_extended_headers_negotiated(), extended)
+            yield h
+        finally:
+            h.shutdown()

Why not initialize self.h = nbd.NBD() in setUp(), and call self.h.shutdown() in tearDown()? This way we get rid of "with self.open_nbd() as h:" and extra indent in each testcase.

Signature · 2 lines
Best regards,
Vladimir
#4 lore raw

Re: [PATCH 1/5] iotests: add coverage for NBD transmission commands

Quoted 19 lines · Vladimir Sementsov-Ogievskiy
Quoted 8 lines · Denis V. Lunev
tests/qemu-iotests/tests/nbd-commands new file +8 -0
+
+    @contextmanager
+    def open_nbd(self, structured=True, extended=True):
+        h = nbd.NBD()
+        h.set_request_structured_replies(structured)
+        h.set_request_extended_headers(extended)
+        h.add_meta_context('base:allocation')
+        h.add_meta_context('qemu:allocation-depth')

tests/qemu-iotests/tests/nbd-commands line 86

Quoted 10 lines
tests/qemu-iotests/tests/nbd-commands new file +10 -0
+        # Let the server, not libnbd, reject the out of range requests below
+        h.set_strict_mode(h.get_strict_mode() &
+                          ~(nbd.STRICT_BOUNDS | nbd.STRICT_PAYLOAD))
+        try:
+            h.connect_uri(nbd_uri)
+            self.assertEqual(h.get_structured_replies_negotiated(), structured)
+            self.assertEqual(h.get_extended_headers_negotiated(), extended)
+            yield h
+        finally:
+            h.shutdown()
Quoted 4 lines

Why not initialize self.h = nbd.NBD() in setUp(), and call self.h.shutdown() in tearDown()? This way we get rid of "with self.open_nbd() as h:" and extra indent in each testcase.

I will do that with in v2, will also fix nit for patch 5.

That is not a problem, really.

Thank you for a review.

#5 lore raw

PATCH 2/5 nbd/server: accept NBD_CMD_CACHE above the maximum payload size

In reply to Denis V. Lunev

From: Denis V. Lunev <[email protected]>

NBD_CMD_CACHE carries no payload in either direction. The request is a header only, nbd_do_cmd_cache() passes a NULL qiov to blk_co_preadv(), and the reply is a bare status. Still the server rejects any effect length above NBD_MAX_BUFFER_SIZE with EINVAL, which forces a client to split a large prefetch into 32 MiB pieces.

The specification does not ask for this. The constraint was renamed from "maximum block size" to "maximum payload size" precisely to separate payload length from effect length, and it says:

For commands that do not require a payload in either direction (such as NBD_CMD_TRIM or NBD_CMD_WRITE_ZEROES), the client MAY request an effect length larger than the maximum payload size; the server SHOULD NOT disconnect, but MAY reply with an NBD_EOVERFLOW or NBD_EINVAL error if the oversize request would require too many server resources when compared to the same command with an effect length limited to the maximum payload size (such as an implementation of NBD_CMD_WRITE_ZEROES that utilizes a scratch buffer).

We already follow that for NBD_CMD_TRIM and NBD_CMD_WRITE_ZEROES, which carry no length check at all, and our client assumes a server supporting extended headers takes unlimited zero and trim lengths. Handle NBD_CMD_CACHE in the same way.

CC Eric Blake <[email protected]>
CC Vladimir Sementsov-Ogievskiy <[email protected]>
---
 nbd/server.c                          | 4 +---
 tests/qemu-iotests/tests/nbd-commands | 5 ++---
 2 files changed, 3 insertions(+), 6 deletions(-)
nbd/server.c +1 -3
index 78ec984409..e6c47f8c2e 100644
@@ -2718,7 +2718,6 @@ static int coroutine_fn nbd_co_receive_request(NBDRequestData *req,
         break;
 
     case NBD_CMD_CACHE:
-        check_length = true;
         break;
 
     case NBD_CMD_WRITE_ZEROES:
@@ -2752,7 +2751,7 @@ static int coroutine_fn nbd_co_receive_request(NBDRequestData *req,
         req->complete = true;
     }
     if (check_length && request->len > NBD_MAX_BUFFER_SIZE) {
-        /* READ, WRITE, CACHE */
+        /* READ, WRITE */
         error_setg(errp, "len (%" PRIu64 ") is larger than max len (%u)",
                    request->len, NBD_MAX_BUFFER_SIZE);
         return -EINVAL;
@@ -2908,7 +2907,6 @@ static coroutine_fn int nbd_do_cmd_cache(NBDClient *client, NBDRequest *request,
     NBDExport *exp = client->exp;
 
     assert(request->type == NBD_CMD_CACHE);
-    assert(request->len <= NBD_MAX_BUFFER_SIZE);
 
     ret = blk_co_preadv(exp->common.blk, request->from, request->len,
                         NULL, BDRV_REQ_COPY_ON_READ | BDRV_REQ_PREFETCH);
tests/qemu-iotests/tests/nbd-commands +2 -3
index 73baeb236f..d2715b59a1 100755
@@ -108,10 +108,9 @@ class TestNbdCommands(iotests.QMPTestCase):
             self.assertEqual(self.allocation_depth(h),
                              [(size, DEPTH_BACKING)])
 
-            h.cache(maximum, 0)
+            h.cache(size, 0)
 
-        self.assertEqual(self.top_extents(),
-                         [(0, maximum, 0), (maximum, size - maximum, 1)])
+        self.assertEqual(self.top_extents(), [(0, size, 0)])
         qemu_io('-c', f'read -P {pattern} 0 {size}', top)
 
     def test_cache_past_end_of_export(self):
Signature · 1 line
2.53.0
Reviewed-by on v1 2/5
#6 lore raw

Re: [PATCH 2/5] nbd/server: accept NBD_CMD_CACHE above the maximum payload size

Quoted 22 lines · Denis V. Lunev

From: Denis V. Lunev<[email protected]>

NBD_CMD_CACHE carries no payload in either direction. The request is a header only, nbd_do_cmd_cache() passes a NULL qiov to blk_co_preadv(), and the reply is a bare status. Still the server rejects any effect length above NBD_MAX_BUFFER_SIZE with EINVAL, which forces a client to split a large prefetch into 32 MiB pieces.

The specification does not ask for this. The constraint was renamed from "maximum block size" to "maximum payload size" precisely to separate payload length from effect length, and it says:

For commands that do not require a payload in either direction (such as NBD_CMD_TRIM or NBD_CMD_WRITE_ZEROES), the client MAY request an effect length larger than the maximum payload size; the server SHOULD NOT disconnect, but MAY reply with an NBD_EOVERFLOW or NBD_EINVAL error if the oversize request would require too many server resources when compared to the same command with an effect length limited to the maximum payload size (such as an implementation of NBD_CMD_WRITE_ZEROES that utilizes a scratch buffer).

Quoted 8 lines

We already follow that for NBD_CMD_TRIM and NBD_CMD_WRITE_ZEROES, which carry no length check at all, and our client assumes a server supporting extended headers takes unlimited zero and trim lengths. Handle NBD_CMD_CACHE in the same way.

CC Eric Blake<[email protected]>
CC Vladimir Sementsov-Ogievskiy<[email protected]>
Signature · 2 lines
Best regards,
Vladimir
#7 lore raw

PATCH 3/5 iotests/nbd-commands: exercise the simple and structured reply modes

In reply to Denis V. Lunev

From: Denis V. Lunev <[email protected]>

Every NBD request the iotests send arrives in NBD_MODE_EXTENDED, so the server paths kept for older clients are never taken. Instrumenting nbd_co_receive_request() over the whole auto group and every test that touches NBD gives mode 4 for all of them, which leaves nbd_co_send_simple_reply() and the compact header handling dead under test. A regression there would only show against a third party client.

libnbd can negotiate down, so run the same command set three times, once per mode, and assert the mode that was actually reached. Block status is limited to the two modes that can negotiate a meta context.

CC Eric Blake <[email protected]>
CC Vladimir Sementsov-Ogievskiy <[email protected]>
---
 tests/qemu-iotests/tests/nbd-commands     | 28 +++++++++++++++++++++++
 tests/qemu-iotests/tests/nbd-commands.out |  4 ++--
 2 files changed, 30 insertions(+), 2 deletions(-)
tests/qemu-iotests/tests/nbd-commands +28 -0
index d2715b59a1..f0b40feb6b 100755
@@ -113,6 +113,34 @@ class TestNbdCommands(iotests.QMPTestCase):
         self.assertEqual(self.top_extents(), [(0, size, 0)])
         qemu_io('-c', f'read -P {pattern} 0 {size}', top)
 
+    def check_commands(self, structured, extended):
+        with self.open_nbd(structured, extended) as h:
+            self.assertEqual(h.pread(4096, 4096), bytes([pattern]) * 4096)
+
+            h.cache(size, 0)
+
+            h.pwrite(b'x' * 4096, 4096)
+            h.flush()
+            self.assertEqual(h.pread(4096, 4096), b'x' * 4096)
+
+            h.zero(4096, 8192)
+            self.assertEqual(h.pread(4096, 8192), bytes(4096))
+
+            h.trim(4096, 16384)
+
+            if structured:
+                self.assertEqual(self.allocation_depth(h),
+                                 [(size, DEPTH_LOCAL)])
+
+    def test_commands_simple_replies(self):
+        self.check_commands(structured=False, extended=False)
+
+    def test_commands_structured_replies(self):
+        self.check_commands(structured=True, extended=False)
+
+    def test_commands_extended_headers(self):
+        self.check_commands(structured=True, extended=True)
+
     def test_cache_past_end_of_export(self):
         with self.open_nbd() as h:
             self.assertRaises(nbd.Error, h.cache, size + 1, 0)
tests/qemu-iotests/tests/nbd-commands.out +2 -2
index 8d7e996700..3f8a935a08 100644
@@ -1,5 +1,5 @@
-...
+......
 ----------------------------------------------------------------------
-Ran 3 tests
+Ran 6 tests
 
 OK
Signature · 1 line
2.53.0
Reviewed-by on v1 3/5
#8 lore raw

Re: [PATCH 3/5] iotests/nbd-commands: exercise the simple and structured reply modes

Quoted 9 lines · Denis V. Lunev

From: Denis V. Lunev<[email protected]>

Every NBD request the iotests send arrives in NBD_MODE_EXTENDED, so the server paths kept for older clients are never taken. Instrumenting nbd_co_receive_request() over the whole auto group and every test that touches NBD gives mode 4 for all of them, which leaves nbd_co_send_simple_reply() and the compact header handling dead under test. A regression there would only show against a third party client.

Quoted 7 lines

libnbd can negotiate down, so run the same command set three times, once per mode, and assert the mode that was actually reached. Block status is limited to the two modes that can negotiate a meta context.

CC Eric Blake<[email protected]>
CC Vladimir Sementsov-Ogievskiy<[email protected]>
Signature · 2 lines
Best regards,
Vladimir
#9 lore raw

PATCH 4/5 iotests/nbd-commands: cover NBD_CMD_BLOCK_STATUS with a payload

In reply to Denis V. Lunev

From: Denis V. Lunev <[email protected]>

With extended headers a client may name the meta contexts it wants in a request payload. Our own client never sends one, so the server side of it, nbd_co_block_status_payload_read(), is not exercised anywhere.

Export two meta contexts and ask libnbd for each of them in turn, which is only answerable through that path.

CC Eric Blake <[email protected]>
CC Vladimir Sementsov-Ogievskiy <[email protected]>
---
 tests/qemu-iotests/tests/nbd-commands     | 20 ++++++++++++++++++++
 tests/qemu-iotests/tests/nbd-commands.out |  4 ++--
 2 files changed, 22 insertions(+), 2 deletions(-)
tests/qemu-iotests/tests/nbd-commands +20 -0
index f0b40feb6b..683a8303e3 100755
@@ -141,6 +141,26 @@ class TestNbdCommands(iotests.QMPTestCase):
     def test_commands_extended_headers(self):
         self.check_commands(structured=True, extended=True)
 
+    def contexts_reported(self, h, wanted=None):
+        seen = []
+
+        def cb(meta, offset, entries, err):
+            seen.append(meta)
+
+        if wanted is None:
+            h.block_status(size, 0, cb)
+        else:
+            h.block_status_filter(size, 0, wanted, cb)
+        return sorted(seen)
+
+    def test_block_status_payload_filter(self):
+        with self.open_nbd() as h:
+            self.assertEqual(self.contexts_reported(h),
+                             ['base:allocation', 'qemu:allocation-depth'])
+
+            for wanted in (['base:allocation'], ['qemu:allocation-depth']):
+                self.assertEqual(self.contexts_reported(h, wanted), wanted)
+
     def test_cache_past_end_of_export(self):
         with self.open_nbd() as h:
             self.assertRaises(nbd.Error, h.cache, size + 1, 0)
tests/qemu-iotests/tests/nbd-commands.out +2 -2
index 3f8a935a08..2f7d3902f2 100644
@@ -1,5 +1,5 @@
-......
+.......
 ----------------------------------------------------------------------
-Ran 6 tests
+Ran 7 tests
 
 OK
Signature · 1 line
2.53.0
Reviewed-by on v1 4/5
#10 lore raw

Re: [PATCH 4/5] iotests/nbd-commands: cover NBD_CMD_BLOCK_STATUS with a payload

Quoted 6 lines · Denis V. Lunev

From: Denis V. Lunev<[email protected]>

With extended headers a client may name the meta contexts it wants in a request payload. Our own client never sends one, so the server side of it, nbd_co_block_status_payload_read(), is not exercised anywhere.

Quoted 6 lines

Export two meta contexts and ask libnbd for each of them in turn, which is only answerable through that path.

CC Eric Blake<[email protected]>
CC Vladimir Sementsov-Ogievskiy<[email protected]>
Signature · 2 lines
Best regards,
Vladimir
#11 lore raw

PATCH 5/5 iotests/nbd-commands: cover the command flags and sparse replies

In reply to Denis V. Lunev

From: Denis V. Lunev <[email protected]>

No test sends the NBD command flags, so the server paths behind them go unexercised. Send them from the new client.

A structured read answers a hole with an offset and a length rather than a cluster of zeroes. Lay out data, a hole and data again, and read the three arrangements a hole can appear in, since a leading hole and a hole between two extents take different turns through nbd_co_send_sparse_read(). NBD_CMD_FLAG_DF asks for one chunk instead, which is the same layout sent as real zeroes. Check the chunk boundaries in both forms, and that the data still reads back.

NBD_CMD_FLAG_REQ_ONE caps the extent array at one entry, so the block status reply covers only the first cluster of the three asked about. That a reply may describe less than was requested is also how the server keeps NBD_MAX_BLOCK_STATUS_EXTENTS from being exceeded on a long fragmented range, and a client assuming full coverage believes stale status.

NBD_CMD_FLAG_FAST_ZERO becomes BDRV_REQ_NO_FALLBACK. A cluster aligned zero can be done by marking the cluster, while zeroing part of a cluster over a backing file needs the read modify write the flag forbids, so check both the success and the ENOTSUP.

NBD_CMD_FLAG_FUA is added to a write and a trim.

CC Eric Blake <[email protected]>
CC Vladimir Sementsov-Ogievskiy <[email protected]>
---
 tests/qemu-iotests/tests/nbd-commands     | 86 +++++++++++++++++++++++
 tests/qemu-iotests/tests/nbd-commands.out |  4 +-
 2 files changed, 88 insertions(+), 2 deletions(-)
tests/qemu-iotests/tests/nbd-commands +86 -0
index 683a8303e3..f727ea3161 100755
@@ -26,6 +26,7 @@ nbd: ModuleType
 
 DEPTH_LOCAL = 1
 DEPTH_BACKING = 2
+CLUSTER = 65536
 
 
 class TestNbdCommands(iotests.QMPTestCase):
@@ -161,6 +162,91 @@ class TestNbdCommands(iotests.QMPTestCase):
             for wanted in (['base:allocation'], ['qemu:allocation-depth']):
                 self.assertEqual(self.contexts_reported(h, wanted), wanted)
 
+    def read_chunks(self, h, count, offset, flags=0):
+        chunks = []
+
+        def cb(subbuf, off, status, err):
+            chunks.append((off, len(subbuf), status))
+
+        h.pread_structured(count, offset, cb, flags)
+        return chunks
+
+    @staticmethod
+    def make_sparse(h):
+        """Lay out data, a hole and data again, one cluster each."""
+        h.pwrite(b'z' * CLUSTER, 0)
+        h.zero(CLUSTER, CLUSTER)
+        h.pwrite(b'z' * CLUSTER, 2 * CLUSTER)
+
+    def test_read_sparse_chunks(self):
+        with self.open_nbd() as h:
+            self.make_sparse(h)
+
+            first = (0, CLUSTER, nbd.READ_DATA)
+            hole = (CLUSTER, CLUSTER, nbd.READ_HOLE)
+            second = (2 * CLUSTER, CLUSTER, nbd.READ_DATA)
+
+            # A hole is an offset and a length, not a cluster of zeroes,
+            # wherever it falls in the reply
+            self.assertEqual(self.read_chunks(h, 2 * CLUSTER, 0),
+                             [first, hole])
+            self.assertEqual(self.read_chunks(h, 2 * CLUSTER, CLUSTER),
+                             [hole, second])
+            self.assertEqual(self.read_chunks(h, 3 * CLUSTER, 0),
+                             [first, hole, second])
+
+            self.assertEqual(h.pread(CLUSTER, CLUSTER), bytes(CLUSTER))
+            self.assertEqual(h.pread(CLUSTER, 0), b'z' * CLUSTER)
+
+    def test_read_dont_fragment(self):
+        with self.open_nbd() as h:
+            self.make_sparse(h)
+
+            self.assertEqual(self.read_chunks(h, 3 * CLUSTER, 0,
+                                              nbd.CMD_FLAG_DF),
+                             [(0, 3 * CLUSTER, nbd.READ_DATA)])
+
+    def status_extents(self, h, count, flags=0):
+        extents = []
+
+        def cb(meta, offset, entries, err):
+            if meta == 'base:allocation':
+                extents.extend(zip(entries[0::2], entries[1::2]))
+
+        h.block_status(count, 0, cb, flags)
+        return extents
+
+    def test_block_status_req_one(self):
+        hole = nbd.STATE_HOLE | nbd.STATE_ZERO
+
+        with self.open_nbd() as h:
+            self.make_sparse(h)
+
+            self.assertEqual(self.status_extents(h, 3 * CLUSTER),
+                             [(CLUSTER, 0), (CLUSTER, hole), (CLUSTER, 0)])
+
+            # One extent, so the reply covers less than was asked for and
+            # the client has to come back for the rest
+            self.assertEqual(self.status_extents(h, 3 * CLUSTER,
+                                                 nbd.CMD_FLAG_REQ_ONE),
+                             [(CLUSTER, 0)])
+
+    def test_write_and_trim_fua(self):
+        with self.open_nbd() as h:
+            h.pwrite(b'y' * 4096, 4096, nbd.CMD_FLAG_FUA)
+            self.assertEqual(h.pread(4096, 4096), b'y' * 4096)
+            h.trim(4096, 4096, nbd.CMD_FLAG_FUA)
+
+    def test_fast_zero(self):
+        with self.open_nbd() as h:
+            h.zero(CLUSTER, CLUSTER, nbd.CMD_FLAG_FAST_ZERO)
+            self.assertEqual(h.pread(CLUSTER, CLUSTER), bytes(CLUSTER))
+
+            # Zeroing part of a cluster needs the fallback the flag forbids
+            with self.assertRaises(nbd.Error) as caught:
+                h.zero(4096, 4096, nbd.CMD_FLAG_FAST_ZERO)
+            self.assertEqual(caught.exception.errno, 'ENOTSUP')
+
     def test_cache_past_end_of_export(self):
         with self.open_nbd() as h:
             self.assertRaises(nbd.Error, h.cache, size + 1, 0)
tests/qemu-iotests/tests/nbd-commands.out +2 -2
index 2f7d3902f2..281b69efea 100644
@@ -1,5 +1,5 @@
-.......
+............
 ----------------------------------------------------------------------
-Ran 7 tests
+Ran 12 tests
 
 OK
Signature · 1 line
2.53.0
Reviewed-by on v1 5/5
#12 lore raw

Re: [PATCH 5/5] iotests/nbd-commands: cover the command flags and sparse replies

Quoted 39 lines · Denis V. Lunev

From: Denis V. Lunev <[email protected]>

No test sends the NBD command flags, so the server paths behind them go unexercised. Send them from the new client.

A structured read answers a hole with an offset and a length rather than a cluster of zeroes. Lay out data, a hole and data again, and read the three arrangements a hole can appear in, since a leading hole and a hole between two extents take different turns through nbd_co_send_sparse_read(). NBD_CMD_FLAG_DF asks for one chunk instead, which is the same layout sent as real zeroes. Check the chunk boundaries in both forms, and that the data still reads back.

NBD_CMD_FLAG_REQ_ONE caps the extent array at one entry, so the block status reply covers only the first cluster of the three asked about. That a reply may describe less than was requested is also how the server keeps NBD_MAX_BLOCK_STATUS_EXTENTS from being exceeded on a long fragmented range, and a client assuming full coverage believes stale status.

NBD_CMD_FLAG_FAST_ZERO becomes BDRV_REQ_NO_FALLBACK. A cluster aligned zero can be done by marking the cluster, while zeroing part of a cluster over a backing file needs the read modify write the flag forbids, so check both the success and the ENOTSUP.

NBD_CMD_FLAG_FUA is added to a write and a trim.

CC Eric Blake <[email protected]>
CC Vladimir Sementsov-Ogievskiy <[email protected]>
---
  tests/qemu-iotests/tests/nbd-commands     | 86 +++++++++++++++++++++++
  tests/qemu-iotests/tests/nbd-commands.out |  4 +-
  2 files changed, 88 insertions(+), 2 deletions(-)
tests/qemu-iotests/tests/nbd-commands +0 -0
index 683a8303e3..f727ea3161 100755

[..]

tests/qemu-iotests/tests/nbd-commands class TestNbdCommands(iotests.QMPTestCase): · line 217

Quoted 10 lines
tests/qemu-iotests/tests/nbd-commands +10 -0
+
+    def status_extents(self, h, count, flags=0):
+        extents = []
+
+        def cb(meta, offset, entries, err):
+            if meta == 'base:allocation':
+                extents.extend(zip(entries[0::2], entries[1::2]))
+
+        h.block_status(count, 0, cb, flags)
+        return extents

duplicates allocation_depth() functions except for meta name and flags passing. code may be shared.

anyway,

Signature · 2 lines
Best regards,
Vladimir