Skip to contentSkip to conversation

rbd: Fix .bdrv_get_specific_info implementation

PATCH In review Kevin Wolf 2 participants 3 messages Aug 11, 2025 – Aug 12, 2025 Block layer

lore
Share & tools
lore

Kevin Wolf

author
#0 lore raw

PATCH 1/1 rbd: Fix .bdrv_get_specific_info implementation

qemu_rbd_get_specific_info() has at least two problems:

The first is that it issues a blocking rbd_read() call in order to probe the encryption format for the image while querying the node. This means that if the connection to the server goes down, not only I/O is stuck (which is unavoidable), but query-names-block-nodes will actually make the whole QEMU instance unresponsive. .bdrv_get_specific_info implementations shouldn't perform blocking operations, but only return what is already known.

The second is that the information returned isn't even correct. If the image is already opened with encryption enabled at the RBD level, we'll probe for "double encryption", i.e. if the encrypted data contains another encryption header. If it doesn't (which is the normal case), we won't return the encryption format. If it does, we return misleading information because it looks like we're talking about the outer level (the encryption format of the image itself) while the information is about an encryption header in the guest data.

Fix this by storing the encryption format in BDRVRBDState when the image is opened (and we do blocking operations anyway) and returning only the stored information in qemu_rbd_get_specific_info().

The information we'll store is either the actual encryption format that we enabled on the RBD level, or if the image is unencrypted, the result of the same probing as we previously did when querying the node. Probing image formats based on content that can be modified by the guest has long been known as problematic, but as long as we only output it to the user instead of making decisions based on it, it should be okay. It is undoubtedly useful in the context of 'qemu-img info' when you're trying to figure out which encryption options you have to use to open the image successfully.

Fixes 42e4ac9ef5a6 ("block/rbd: Add support for rbd image encryption")
---
 qapi/block-core.json |   9 +++-
 block/rbd.c          | 103 ++++++++++++++++++++++++++++---------------
 2 files changed, 75 insertions(+), 37 deletions(-)
qapi/block-core.json +8 -1
index ebbe95b3d8..cbd2bc84fe 100644
@@ -159,7 +159,14 @@
 ##
 # @ImageInfoSpecificRbd:
 #
-# @encryption-format: Image encryption format
+# @encryption-format: Image encryption format. If encryption is enabled for the
+#     image (see encrpyted in BlockNodeInfo), this is the actual format in which the
+#     image is accessed. If encryption is not enabled, this is the result of
+#     probing when the image was opened, to give a suggestion which encryption
+#     format could be enabled. Note that probing results can be changed by the
+#     guest by writing a (possibly partial) encryption format header to the
+#     image, so don't treat this information as trusted if the guest is not
+#     trusted.
 #
 # Since: 6.1
 ##
block/rbd.c +67 -36

178 lines not on this page · raw message

Signature · 1 line
2.50.1
Reviewed-by on v1 1/1
#1 lore raw
Quoted 9 lines · Kevin Wolf

qemu_rbd_get_specific_info() has at least two problems:

The first is that it issues a blocking rbd_read() call in order to probe the encryption format for the image while querying the node. This means that if the connection to the server goes down, not only I/O is stuck (which is unavoidable), but query-names-block-nodes will actually make the whole QEMU instance unresponsive. .bdrv_get_specific_info implementations shouldn't perform blocking operations, but only return what is already known.

Maybe we should put a “must not block” warning on BlockDriver.bdrv_get_specific_info()?

Quoted 23 lines

The second is that the information returned isn't even correct. If the image is already opened with encryption enabled at the RBD level, we'll probe for "double encryption", i.e. if the encrypted data contains another encryption header. If it doesn't (which is the normal case), we won't return the encryption format. If it does, we return misleading information because it looks like we're talking about the outer level (the encryption format of the image itself) while the information is about an encryption header in the guest data.

Fix this by storing the encryption format in BDRVRBDState when the image is opened (and we do blocking operations anyway) and returning only the stored information in qemu_rbd_get_specific_info().

The information we'll store is either the actual encryption format that we enabled on the RBD level, or if the image is unencrypted, the result of the same probing as we previously did when querying the node. Probing image formats based on content that can be modified by the guest has long been known as problematic, but as long as we only output it to the user instead of making decisions based on it, it should be okay. It is undoubtedly useful in the context of 'qemu-img info' when you're trying to figure out which encryption options you have to use to open the image successfully.

Quoted 7 lines
Fixes 42e4ac9ef5a6 ("block/rbd: Add support for rbd image encryption")
---
  qapi/block-core.json |   9 +++-
  block/rbd.c          | 103 ++++++++++++++++++++++++++++---------------
  2 files changed, 75 insertions(+), 37 deletions(-)

Looks good to me, just found two spelling/wording nits below.  With those fixed:

(I think the .bdrv_get_specific_info() warning would be nice, but optional.)

qapi/block-core.json line 163

Quoted 11 lines
qapi/block-core.json +2 -1
index ebbe95b3d8..cbd2bc84fe 100644
@@ -159,7 +159,14 @@
  ##
  # @ImageInfoSpecificRbd:
  #
-# @encryption-format: Image encryption format
+# @encryption-format: Image encryption format. If encryption is enabled for the
+#     image (see encrpyted in BlockNodeInfo), this is the actual format in which the

*encrypted

Quoted 73 lines
qapi/block-core.json +6 -0
+#     image is accessed. If encryption is not enabled, this is the result of
+#     probing when the image was opened, to give a suggestion which encryption
+#     format could be enabled. Note that probing results can be changed by the
+#     guest by writing a (possibly partial) encryption format header to the
+#     image, so don't treat this information as trusted if the guest is not
+#     trusted.
  #
  # Since: 6.1
  ##
block/rbd.c +18 -2
index 951cd63f9a..8582bc8360 100644
@@ -99,6 +99,14 @@ typedef struct BDRVRBDState {
      char *namespace;
      uint64_t image_size;
      uint64_t object_size;
+
+    /*
+     * If @bs->encrypted is true, this is the encryption format actually loaded
+     * at the librbd level. If it is false, it is the result of probing.
+     * RBD_IMAGE_ENCRYPTION_FORMAT__MAX means that encryption is not enabled and
+     * probing didn't find any known encryption header either.
+     */
+    RbdImageEncryptionFormat encryption_format;
  } BDRVRBDState;
 
  typedef struct RBDTask {
@@ -470,10 +478,12 @@ static int qemu_rbd_encryption_format(rbd_image_t image,
      return 0;
  }
 
-static int qemu_rbd_encryption_load(rbd_image_t image,
+static int qemu_rbd_encryption_load(BlockDriverState *bs,
+                                    rbd_image_t image,
                                      RbdEncryptionOptions *encrypt,
                                      Error **errp)
  {
+    BDRVRBDState *s = bs->opaque;
      int r = 0;
      g_autofree char *passphrase = NULL;
      rbd_encryption_luks1_format_options_t luks_opts;
@@ -544,15 +554,19 @@ static int qemu_rbd_encryption_load(rbd_image_t image,
          error_setg_errno(errp, -r, "encryption load fail");
          return r;
      }
+    bs->encrypted = true;
+    s->encryption_format = encrypt->format;
 
      return 0;
  }
 
  #ifdef LIBRBD_SUPPORTS_ENCRYPTION_LOAD2
-static int qemu_rbd_encryption_load2(rbd_image_t image,
+static int qemu_rbd_encryption_load2(BlockDriverState *bs,
+                                     rbd_image_t image,
                                       RbdEncryptionOptions *encrypt,
                                       Error **errp)
  {
+    BDRVRBDState *s = bs->opaque;
      int r = 0;
      int encrypt_count = 1;
      int i;
@@ -638,6 +652,8 @@ static int qemu_rbd_encryption_load2(rbd_image_t image,
          error_setg_errno(errp, -r, "layered encryption load fail");
          goto exit;
      }
+    bs->encrypted = true;
+    s->encryption_format = encrypt->format;
 
  exit:
      for (i = 0; i < encrypt_count; ++i) {

block/rbd.c exit: · line 695

Quoted 9 lines
block/rbd.c +6 -0
@@ -671,6 +687,44 @@ exit:
  #endif
  #endif
 
+/*
+ * For an image without encryption enabled on the rbd layer, probe the start of
+ * the image if it could be opened as an encrypted image so that we can display
+ * it when the user queries the node (most importantly in qemu-img).
+ *
+ * If the guest writes an encryption header to its disk after this probing, but

Either there’s something missing after the comma (e.g. “it wouldn’t be reflected when queried”), or the “but” is too much.

Quoted 118 lines
block/rbd.c +43 -34
+ * that's okay. There is no reason why the user should want to apply encryption
+ * at the rbd level while the image is still in use. This is just guest data.
+ */
+static void qemu_rbd_encryption_probe(BlockDriverState *bs)
+{
+    BDRVRBDState *s = bs->opaque;
+    char buf[RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {0};
+    int r;
+
+    assert(s->encryption_format == RBD_IMAGE_ENCRYPTION_FORMAT__MAX);
+
+    r = rbd_read(s->image, 0,
+                 RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN, buf);
+    if (r < RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN) {
+        return;
+    }
+
+    if (memcmp(buf, rbd_luks_header_verification,
+               RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN) == 0) {
+        s->encryption_format = RBD_IMAGE_ENCRYPTION_FORMAT_LUKS;
+    } else if (memcmp(buf, rbd_luks2_header_verification,
+               RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN) == 0) {
+        s->encryption_format = RBD_IMAGE_ENCRYPTION_FORMAT_LUKS2;
+    } else if (memcmp(buf, rbd_layered_luks_header_verification,
+               RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN) == 0) {
+        s->encryption_format = RBD_IMAGE_ENCRYPTION_FORMAT_LUKS;
+    } else if (memcmp(buf, rbd_layered_luks2_header_verification,
+               RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN) == 0) {
+        s->encryption_format = RBD_IMAGE_ENCRYPTION_FORMAT_LUKS2;
+    }
+}
+
  /* FIXME Deprecate and remove keypairs or make it available in QMP. */
  static int qemu_rbd_do_create(BlockdevCreateOptions *options,
                                const char *keypairs, const char *password_secret,
@@ -1133,17 +1187,18 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags,
          goto failed_open;
      }
 
+    s->encryption_format = RBD_IMAGE_ENCRYPTION_FORMAT__MAX;
      if (opts->encrypt) {
  #ifdef LIBRBD_SUPPORTS_ENCRYPTION
          if (opts->encrypt->parent) {
  #ifdef LIBRBD_SUPPORTS_ENCRYPTION_LOAD2
-            r = qemu_rbd_encryption_load2(s->image, opts->encrypt, errp);
+            r = qemu_rbd_encryption_load2(bs, s->image, opts->encrypt, errp);
  #else
              r = -ENOTSUP;
              error_setg(errp, "RBD library does not support layered encryption");
  #endif
          } else {
-            r = qemu_rbd_encryption_load(s->image, opts->encrypt, errp);
+            r = qemu_rbd_encryption_load(bs, s->image, opts->encrypt, errp);
          }
          if (r < 0) {
              goto failed_post_open;
@@ -1153,6 +1208,8 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags,
          error_setg(errp, "RBD library does not support image encryption");
          goto failed_post_open;
  #endif
+    } else {
+        qemu_rbd_encryption_probe(bs);
      }
 
      r = rbd_stat(s->image, &info, sizeof(info));
@@ -1412,17 +1469,6 @@ static ImageInfoSpecific *qemu_rbd_get_specific_info(BlockDriverState *bs,
  {
      BDRVRBDState *s = bs->opaque;
      ImageInfoSpecific *spec_info;
-    char buf[RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {0};
-    int r;
-
-    if (s->image_size >= RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN) {
-        r = rbd_read(s->image, 0,
-                     RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN, buf);
-        if (r < 0) {
-            error_setg_errno(errp, -r, "cannot read image start for probe");
-            return NULL;
-        }
-    }
 
      spec_info = g_new(ImageInfoSpecific, 1);
      *spec_info = (ImageInfoSpecific){
@@ -1430,28 +1476,13 @@ static ImageInfoSpecific *qemu_rbd_get_specific_info(BlockDriverState *bs,
          .u.rbd.data = g_new0(ImageInfoSpecificRbd, 1),
      };
 
-    if (memcmp(buf, rbd_luks_header_verification,
-               RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN) == 0) {
-        spec_info->u.rbd.data->encryption_format =
-                RBD_IMAGE_ENCRYPTION_FORMAT_LUKS;
-        spec_info->u.rbd.data->has_encryption_format = true;
-    } else if (memcmp(buf, rbd_luks2_header_verification,
-               RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN) == 0) {
-        spec_info->u.rbd.data->encryption_format =
-                RBD_IMAGE_ENCRYPTION_FORMAT_LUKS2;
-        spec_info->u.rbd.data->has_encryption_format = true;
-    } else if (memcmp(buf, rbd_layered_luks_header_verification,
-               RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN) == 0) {
-        spec_info->u.rbd.data->encryption_format =
-                RBD_IMAGE_ENCRYPTION_FORMAT_LUKS;
-        spec_info->u.rbd.data->has_encryption_format = true;
-    } else if (memcmp(buf, rbd_layered_luks2_header_verification,
-               RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN) == 0) {
-        spec_info->u.rbd.data->encryption_format =
-                RBD_IMAGE_ENCRYPTION_FORMAT_LUKS2;
-        spec_info->u.rbd.data->has_encryption_format = true;
+    if (s->encryption_format == RBD_IMAGE_ENCRYPTION_FORMAT__MAX) {
+        assert(!bs->encrypted);
      } else {
-        spec_info->u.rbd.data->has_encryption_format = false;
+        ImageInfoSpecificRbd *rbd_info = spec_info->u.rbd.data;
+
+        rbd_info->has_encryption_format = true;
+        rbd_info->encryption_format = s->encryption_format;
      }
 
      return spec_info;
author
#2 lore raw

Am 12.08.2025 um 09:46 hat Hanna Czenczek geschrieben:

Quoted 11 lines · Kevin Wolf
Quoted 9 lines

qemu_rbd_get_specific_info() has at least two problems:

The first is that it issues a blocking rbd_read() call in order to probe the encryption format for the image while querying the node. This means that if the connection to the server goes down, not only I/O is stuck (which is unavoidable), but query-names-block-nodes will actually make the whole QEMU instance unresponsive. .bdrv_get_specific_info implementations shouldn't perform blocking operations, but only return what is already known.

Quoted 2 lines

Maybe we should put a “must not block” warning on
BlockDriver.bdrv_get_specific_info()?

I'm not sure. In theory, nothing that is called from the main loop may block. We're violating this all the time, of course, but it has the same effects as violating it here. I expect that doing blockdev-add for a server that just doesn't respond will hang QEMU the exact same way.

So making .bdrv_get_specific_info implementations not block may be easier, but I don't know if I want to make a distinction that would basically imply that blocking in .bdrv_open is not a longstanding problem, but actually fine.

Quoted 123 lines
Quoted 23 lines

The second is that the information returned isn't even correct. If the image is already opened with encryption enabled at the RBD level, we'll probe for "double encryption", i.e. if the encrypted data contains another encryption header. If it doesn't (which is the normal case), we won't return the encryption format. If it does, we return misleading information because it looks like we're talking about the outer level (the encryption format of the image itself) while the information is about an encryption header in the guest data.

Fix this by storing the encryption format in BDRVRBDState when the image is opened (and we do blocking operations anyway) and returning only the stored information in qemu_rbd_get_specific_info().

The information we'll store is either the actual encryption format that we enabled on the RBD level, or if the image is unencrypted, the result of the same probing as we previously did when querying the node. Probing image formats based on content that can be modified by the guest has long been known as problematic, but as long as we only output it to the user instead of making decisions based on it, it should be okay. It is undoubtedly useful in the context of 'qemu-img info' when you're trying to figure out which encryption options you have to use to open the image successfully.

Quoted 7 lines
Fixes 42e4ac9ef5a6 ("block/rbd: Add support for rbd image encryption")
---
  qapi/block-core.json |   9 +++-
  block/rbd.c          | 103 ++++++++++++++++++++++++++++---------------
  2 files changed, 75 insertions(+), 37 deletions(-)

Looks good to me, just found two spelling/wording nits below.  With those fixed:

(I think the .bdrv_get_specific_info() warning would be nice, but optional.)

qapi/block-core.json line 163

Quoted 11 lines
qapi/block-core.json +2 -1
index ebbe95b3d8..cbd2bc84fe 100644
@@ -159,7 +159,14 @@
  ##
  # @ImageInfoSpecificRbd:
  #
-# @encryption-format: Image encryption format
+# @encryption-format: Image encryption format. If encryption is enabled for the
+#     image (see encrpyted in BlockNodeInfo), this is the actual format in which the

*encrypted

Quoted 71 lines
qapi/block-core.json +6 -0
+#     image is accessed. If encryption is not enabled, this is the result of
+#     probing when the image was opened, to give a suggestion which encryption
+#     format could be enabled. Note that probing results can be changed by the
+#     guest by writing a (possibly partial) encryption format header to the
+#     image, so don't treat this information as trusted if the guest is not
+#     trusted.
  #
  # Since: 6.1
  ##
block/rbd.c +19 -2
index 951cd63f9a..8582bc8360 100644
@@ -99,6 +99,14 @@ typedef struct BDRVRBDState {
      char *namespace;
      uint64_t image_size;
      uint64_t object_size;
+
+    /*
+     * If @bs->encrypted is true, this is the encryption format actually loaded
+     * at the librbd level. If it is false, it is the result of probing.
+     * RBD_IMAGE_ENCRYPTION_FORMAT__MAX means that encryption is not enabled and
+     * probing didn't find any known encryption header either.
+     */
+    RbdImageEncryptionFormat encryption_format;
  } BDRVRBDState;
  typedef struct RBDTask {
@@ -470,10 +478,12 @@ static int qemu_rbd_encryption_format(rbd_image_t image,
      return 0;
  }
-static int qemu_rbd_encryption_load(rbd_image_t image,
+static int qemu_rbd_encryption_load(BlockDriverState *bs,
+                                    rbd_image_t image,
                                      RbdEncryptionOptions *encrypt,
                                      Error **errp)
  {
+    BDRVRBDState *s = bs->opaque;
      int r = 0;
      g_autofree char *passphrase = NULL;
      rbd_encryption_luks1_format_options_t luks_opts;
@@ -544,15 +554,19 @@ static int qemu_rbd_encryption_load(rbd_image_t image,
          error_setg_errno(errp, -r, "encryption load fail");
          return r;
      }
+    bs->encrypted = true;
+    s->encryption_format = encrypt->format;
      return 0;
  }
  #ifdef LIBRBD_SUPPORTS_ENCRYPTION_LOAD2
-static int qemu_rbd_encryption_load2(rbd_image_t image,
+static int qemu_rbd_encryption_load2(BlockDriverState *bs,
+                                     rbd_image_t image,
                                       RbdEncryptionOptions *encrypt,
                                       Error **errp)
  {
+    BDRVRBDState *s = bs->opaque;
      int r = 0;
      int encrypt_count = 1;
      int i;
@@ -638,6 +652,8 @@ static int qemu_rbd_encryption_load2(rbd_image_t image,
          error_setg_errno(errp, -r, "layered encryption load fail");
          goto exit;
      }
+    bs->encrypted = true;
+    s->encryption_format = encrypt->format;
  exit:
      for (i = 0; i < encrypt_count; ++i) {
@@ -671,6 +687,44 @@ exit:
  #endif
  #endif
+/*
Quoted 8 lines

block/rbd.c exit: · line 695

Quoted 5 lines
block/rbd.c +5 -0
+ * For an image without encryption enabled on the rbd layer, probe the start of
+ * the image if it could be opened as an encrypted image so that we can display
+ * it when the user queries the node (most importantly in qemu-img).
+ *
+ * If the guest writes an encryption header to its disk after this probing, but

Either there’s something missing after the comma (e.g. “it wouldn’t be reflected when queried”), or the “but” is too much.

I think I had the longer version in mind, I'll add it while applying. Thanks for catching this.

Quoted 4 lines

block/rbd.c exit: · line 699

Quoted 4 lines
block/rbd.c +4 -0
+ * that's okay. There is no reason why the user should want to apply encryption
+ * at the rbd level while the image is still in use. This is just guest data.
+ */
+static void qemu_rbd_encryption_probe(BlockDriverState *bs)

Kevin