Skip to contentSkip to conversation

nbd: s/handle/cookie/

PATCH Applied Eric Blake 2 participants 7 messages Apr 10, 2023 – Apr 28, 2023

Subsystems (4)
lore
Share & tools
lore

Eric Blake

author
#0 lore raw

v3 cover nbd: s/handle/cookie/

v2 was here: https://lkml.org/lkml/2023/3/17/1107
since then:
- squash patch 2/5 and 3/5 into 3/4 [Ming]
- add Josef's R-b
- tweak commit messages to match commits in userspace NBD (code itself is unchanged, modulo the patch squash)

Eric Blake (4):
uapi nbd: improve doc links to userspace spec uapi nbd: add cookie alias to handle
block nbd: use req.cookie instead of req.handle docs nbd: userspace NBD now favors github over sourceforge

 Documentation/admin-guide/blockdev/nbd.rst |  2 +-
 drivers/block/nbd.c                        |  6 +++---
 include/uapi/linux/nbd.h                   | 25 +++++++++++++++++-----
 3 files changed, 24 insertions(+), 9 deletions(-)

base-commit: 09a9639e56c01c7a00d6c0ca63f4c7c41abe075d

Signature · 1 line
2.39.2
author
#1 lore raw

PATCH v3 1/4 uapi nbd: improve doc links to userspace spec

The uapi <linux/nbd.h> header intentionally documents only the NBD server features that the kernel module will utilize as a client. But while it already had one mention of skipped bits due to userspace extensions, it did not actually direct the reader to the canonical source to learn about those extensions.

While touching comments, fix an outdated reference that listed only READ and WRITE as commands.

---
v2: Split change to sourceforge link to separate patch
v3: Add R-b

---
 include/uapi/linux/nbd.h | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
include/uapi/linux/nbd.h +12 -3
index 20d6cc91435d..8797387caaf7 100644
@@ -11,6 +11,8 @@
  *            Cleanup PARANOIA usage & code.
  * 2004/02/19 Paul Clements
  *            Removed PARANOIA, plus various cleanup and comments
+ * 2023 Copyright Red Hat
+ *            Link to userspace extensions.
  */
 
 #ifndef _UAPILINUX_NBD_H
@@ -30,12 +32,18 @@
 #define NBD_SET_TIMEOUT _IO( 0xab, 9 )
 #define NBD_SET_FLAGS   _IO( 0xab, 10)
 
+/*
+ * See also https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md
+ * for additional userspace extensions not yet utilized in the kernel module.
+ */
+
 enum {
 	NBD_CMD_READ = 0,
 	NBD_CMD_WRITE = 1,
 	NBD_CMD_DISC = 2,
 	NBD_CMD_FLUSH = 3,
 	NBD_CMD_TRIM = 4
+	/* userspace defines additional extension commands */
 };
 
 /* values for flags field, these are server interaction specific. */
@@ -64,14 +72,15 @@ enum {
 #define NBD_REQUEST_MAGIC 0x25609513
 #define NBD_REPLY_MAGIC 0x67446698
 /* Do *not* use magics: 0x12560953 0x96744668. */
+/* magic 0x668e33ef for structured reply not supported by kernel yet */
 
 /*
  * This is the packet used for communication between client and
  * server. All data are in network byte order.
  */
 struct nbd_request {
-	__be32 magic;
-	__be32 type;	/* == READ || == WRITE 	*/
+	__be32 magic;	/* NBD_REQUEST_MAGIC	*/
+	__be32 type;	/* See NBD_CMD_*	*/
 	char handle[8];
 	__be64 from;
 	__be32 len;
@@ -82,7 +91,7 @@ struct nbd_request {
  * it has completed an I/O request (or an error occurs).
  */
 struct nbd_reply {
-	__be32 magic;
+	__be32 magic;		/* NBD_REPLY_MAGIC	*/
 	__be32 error;		/* 0 = ok, else error	*/
 	char handle[8];		/* handle you got from request	*/
 };
Signature · 1 line
2.39.2
author
#2 lore raw

PATCH v3 2/4 uapi nbd: add cookie alias to handle

In reply to Eric Blake

The uapi <linux/nbd.h> header declares a 'char handle[8]' per request; which is overloaded in English (are you referring to "handle" the verb, such as handling a signal or writing a callback handler, or "handle" the noun, the value used in a lookup table to correlate a response back to the request). Many user-space NBD implementations (both servers and clients) have instead used 'uint64_t cookie' or similar, as it is easier to directly assign an integer than to futz around with memcpy. In fact, upstream documentation is now encouraging this shift in terminology:
https://github.com/NetworkBlockDevice/nbd/commit/ca4392eb2b

Accomplish this by use of an anonymous union to provide the alias for anyone getting the definition from the uapi; this does not break existing clients, while exposing the nicer name for those who prefer it. Note that block/nbd.c still uses the term handle (in fact, it actually combines a 32-bit cookie and a 32-bit tag into the 64-bit handle), but that internal usage is not changed by the public uapi, since no compliant NBD server has any reason to inspect or alter the 64 bits sent over the socket.

---
v2: swap order of anonymous union and add comments to list favored name
v3: tweak link in commit message, add R-b

---
 include/uapi/linux/nbd.h | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
include/uapi/linux/nbd.h +9 -3
index 8797387caaf7..80ce0ef43afd 100644
@@ -12,7 +12,7 @@
  * 2004/02/19 Paul Clements
  *            Removed PARANOIA, plus various cleanup and comments
  * 2023 Copyright Red Hat
- *            Link to userspace extensions.
+ *            Link to userspace extensions, favor cookie over handle.
  */
 
 #ifndef _UAPILINUX_NBD_H
@@ -81,7 +81,10 @@ enum {
 struct nbd_request {
 	__be32 magic;	/* NBD_REQUEST_MAGIC	*/
 	__be32 type;	/* See NBD_CMD_*	*/
-	char handle[8];
+	union {
+		__be64 cookie;	/* Opaque identifier for request	*/
+		char handle[8];	/* older spelling of cookie		*/
+	};
 	__be64 from;
 	__be32 len;
 } __attribute__((packed));
@@ -93,6 +96,9 @@ struct nbd_request {
 struct nbd_reply {
 	__be32 magic;		/* NBD_REPLY_MAGIC	*/
 	__be32 error;		/* 0 = ok, else error	*/
-	char handle[8];		/* handle you got from request	*/
+	union {
+		__be64 cookie;	/* Opaque identifier from request	*/
+		char handle[8];	/* older spelling of cookie		*/
+	};
 };
 #endif /* _UAPILINUX_NBD_H */
Signature · 1 line
2.39.2
author
#3 lore raw

PATCH v3 3/4 block nbd: use req.cookie instead of req.handle

In reply to Eric Blake

The NBD spec was recently changed [1] to refer to the opaque client identifier as a 'cookie' rather than a 'handle', but has for a much longer time listed it as a 64-bit value, and declares that all values in the NBD protocol are sent in network byte order (big-endian).

Because the value is opaque to the server, it doesn't usually matter what endianness we send as the client - as long as we are consistent that either we byte-swap on both write and read, or on neither, then we can match server replies back to our requests. That said, our internal use of the cookie is as a 64-bit number (well, as two 32-bit numbers concatenated together), rather than as 8 individual bytes; so prior to this commit, we ARE leaking the native endianness of our internals as a client out to the server. We don't know of any server that will actually inspect the opaque value and behave differently depending on whether a little-endian or big-endian client is sending requests, but since we DO log the cookie value, a wireshark capture of the network traffic is easier to correlate back to the kernel traffic of a big-endian host (where the u64 and char[8] representations are the same) than of a little-endian host (where if wireshark honors the NBD spec and displays a u64 in network byte order, it is byte-swapped from what the kernel logged).

The fix in this patch is thus two-part: it now consistently uses network byte order for the opaque value (no difference to a big-endian machine, but an extra byteswap on a little-endian machine; probably in the noise compared to the overhead of network traffic in general), and now uses a 64-bit integer instead of char[8] as its preferred access to the opaque value (direct assignment instead of memcpy()).

---

v3: squash original 2/5 and 4/5 into one patch, to avoid intermediate duplication through a tmp variable [Ming]. Josef's R-b added, since the final outcome of the squash is unchanged.

---
 drivers/block/nbd.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/block/nbd.c +3 -3
index 592cfa8b765a..94ae85400b46 100644
@@ -606,7 +606,7 @@ static int nbd_send_cmd(struct nbd_device *nbd, struct nbd_cmd *cmd, int index)
 		request.len = htonl(size);
 	}
 	handle = nbd_cmd_handle(cmd);
-	memcpy(request.handle, &handle, sizeof(handle));
+	request.cookie = cpu_to_be64(handle);
 
 	trace_nbd_send_request(&request, nbd->index, blk_mq_rq_from_pdu(cmd));
 
@@ -618,7 +618,7 @@ static int nbd_send_cmd(struct nbd_device *nbd, struct nbd_cmd *cmd, int index)
 	trace_nbd_header_sent(req, handle);
 	if (result < 0) {
 		if (was_interrupted(result)) {
-			/* If we havne't sent anything we can just return BUSY,
+			/* If we haven't sent anything we can just return BUSY,
 			 * however if we have sent something we need to make
 			 * sure we only allow this req to be sent until we are
 			 * completely done.
@@ -732,7 +732,7 @@ static struct nbd_cmd *nbd_handle_reply(struct nbd_device *nbd, int index,
 	u32 tag;
 	int ret = 0;
 
-	memcpy(&handle, reply->handle, sizeof(handle));
+	handle = be64_to_cpu(reply->cookie);
 	tag = nbd_handle_to_tag(handle);
 	hwq = blk_mq_unique_tag_to_hwq(tag);
 	if (hwq < nbd->tag_set.nr_hw_queues)
Signature · 1 line
2.39.2
author
#4 lore raw

PATCH v3 4/4 docs nbd: userspace NBD now favors github over sourceforge

In reply to Eric Blake

While the sourceforge site for userspace NBD still exists, the code repository moved to github several years ago. Then with a recent patch[1], the github landing page contains just as much information as the sourceforge page, so we might as well point to a single location that also provides the code.

[1] https://lists.debian.org/nbd/2023/03/msg00051.html

---
v2: split into its own patch

---
 Documentation/admin-guide/blockdev/nbd.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/admin-guide/blockdev/nbd.rst +1 -1
index d78dfe559dcf..faf2ac4b1509 100644
@@ -14,7 +14,7 @@ to borrow disk space from another computer.
 Unlike NFS, it is possible to put any filesystem on it, etc.
 
 For more information, or to download the nbd-client and nbd-server
-tools, go to http://nbd.sf.net/.
+tools, go to https://github.com/NetworkBlockDevice/nbd.
 
 The nbd kernel module need only be installed on the client
 system, as the nbd-server is completely in userspace. In fact,
Signature · 1 line
2.39.2
author
#5 lore raw
In reply to Eric Blake

ping

Quoted 23 lines · Eric Blake

v2 was here: https://lkml.org/lkml/2023/3/17/1107
since then:
- squash patch 2/5 and 3/5 into 3/4 [Ming]
- add Josef's R-b
- tweak commit messages to match commits in userspace NBD (code itself is unchanged, modulo the patch squash)

Eric Blake (4):
uapi nbd: improve doc links to userspace spec uapi nbd: add cookie alias to handle
block nbd: use req.cookie instead of req.handle docs nbd: userspace NBD now favors github over sourceforge

 Documentation/admin-guide/blockdev/nbd.rst |  2 +-
 drivers/block/nbd.c                        |  6 +++---
 include/uapi/linux/nbd.h                   | 25 +++++++++++++++++-----
 3 files changed, 24 insertions(+), 9 deletions(-)

base-commit: 09a9639e56c01c7a00d6c0ca63f4c7c41abe075d

Signature · 1 line
2.39.2
Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org
maintainer Applied
#6 lore raw
In reply to Eric Blake
Quoted 8 lines · Eric Blake

v2 was here: https://lkml.org/lkml/2023/3/17/1107
since then:
- squash patch 2/5 and 3/5 into 3/4 [Ming]
- add Josef's R-b
- tweak commit messages to match commits in userspace NBD (code itself is unchanged, modulo the patch squash)

[...]

Applied, thanks!

[1/4] uapi nbd: improve doc links to userspace spec

      commit: daf376a366fd2d469d66ab83dfdc074777462bab
[2/4] uapi nbd: add cookie alias to handle
      commit: 2686eb845da7762ee98b17e578b0c081aafb77b9
[3/4] block nbd: use req.cookie instead of req.handle
      commit: bd9e9916c32fd4b4fb4e879e05bd1568ee02ec93
[4/4] docs nbd: userspace NBD now favors github over sourceforge
      commit: 952aa344bf4305ab6fa0d9962ef8c2caa2afef4c

Best regards,

Signature · 1 line
Jens Axboe