Skip to contentSkip to conversation

migration/dirty-bitmaps: change bitmap enumeration method

PATCH In review John Snow 3 participants 13 messages May 14, 2019 – Dec 9, 2019

lore
Share & tools
lore

John Snow

author
#0 lore raw

PATCH v2 1/1 migration/dirty-bitmaps: change bitmap enumeration method

Shift from looking at every root BDS to *every* BDS. This will migrate bitmaps that are attached to blockdev created nodes instead of just ones attached to emulated storage devices.

Note that this will not migrate anonymous or internal-use bitmaps, as those are defined as having no name.

This will also fix the Coverity issues Peter Maydell has been asking about for the past several releases, as well as fixing a real bug.

Reported-by Coverity 😅
Fixes Coverity CID 1390625
CC Stefan Hajnoczi <[email protected]>
---
 migration/block-dirty-bitmap.c | 14 ++++----------
 1 file changed, 4 insertions(+), 10 deletions(-)
migration/block-dirty-bitmap.c +4 -10
index d1bb863cb6..4a896a09eb 100644
@@ -273,7 +273,6 @@ static int init_dirty_bitmap_migration(void)
     BlockDriverState *bs;
     BdrvDirtyBitmap *bitmap;
     DirtyBitmapMigBitmapState *dbms;
-    BdrvNextIterator it;
     Error *local_err = NULL;
 
     dirty_bitmap_mig_state.bulk_completed = false;
@@ -281,13 +280,8 @@ static int init_dirty_bitmap_migration(void)
     dirty_bitmap_mig_state.prev_bitmap = NULL;
     dirty_bitmap_mig_state.no_bitmaps = false;
 
-    for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) {
-        const char *drive_name = bdrv_get_device_or_node_name(bs);
-
-        /* skip automatically inserted nodes */
-        while (bs && bs->drv && bs->implicit) {
-            bs = backing_bs(bs);
-        }
+    for (bs = bdrv_next_all_states(NULL); bs; bs = bdrv_next_all_states(bs)) {
+        const char *name = bdrv_get_device_or_node_name(bs);
 
         for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap;
              bitmap = bdrv_dirty_bitmap_next(bs, bitmap))
@@ -296,7 +290,7 @@ static int init_dirty_bitmap_migration(void)
                 continue;
             }
 
-            if (drive_name == NULL) {
+            if (!name || strcmp(name, "") == 0) {
                 error_report("Found bitmap '%s' in unnamed node %p. It can't "
                              "be migrated", bdrv_dirty_bitmap_name(bitmap), bs);
                 goto fail;
@@ -313,7 +307,7 @@ static int init_dirty_bitmap_migration(void)
 
             dbms = g_new0(DirtyBitmapMigBitmapState, 1);
             dbms->bs = bs;
-            dbms->node_name = drive_name;
+            dbms->node_name = name;
             dbms->bitmap = bitmap;
             dbms->total_sectors = bdrv_nb_sectors(bs);
             dbms->sectors_per_chunk = CHUNK_SIZE * 8 *
Signature · 1 line
2.20.1
Reviewed-by on v2 1/1
#1 lore raw
Quoted 34 lines · John Snow

Shift from looking at every root BDS to *every* BDS. This will migrate bitmaps that are attached to blockdev created nodes instead of just ones attached to emulated storage devices.

Note that this will not migrate anonymous or internal-use bitmaps, as those are defined as having no name.

This will also fix the Coverity issues Peter Maydell has been asking about for the past several releases, as well as fixing a real bug.

Reported-by Coverity 😅
Fixes Coverity CID 1390625
CC Stefan Hajnoczi <[email protected]>
---
  migration/block-dirty-bitmap.c | 14 ++++----------
  1 file changed, 4 insertions(+), 10 deletions(-)
migration/block-dirty-bitmap.c +0 -1
index d1bb863cb6..4a896a09eb 100644
@@ -273,7 +273,6 @@ static int init_dirty_bitmap_migration(void)
      BlockDriverState *bs;
      BdrvDirtyBitmap *bitmap;
      DirtyBitmapMigBitmapState *dbms;
-    BdrvNextIterator it;
      Error *local_err = NULL;
 
      dirty_bitmap_mig_state.bulk_completed = false;

migration/block-dirty-bitmap.c static int init_dirty_bitmap_migration(void) · line 290

Quoted 10 lines
migration/block-dirty-bitmap.c +0 -7
@@ -281,13 +280,8 @@ static int init_dirty_bitmap_migration(void)
      dirty_bitmap_mig_state.prev_bitmap = NULL;
      dirty_bitmap_mig_state.no_bitmaps = false;
 
-    for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) {
-        const char *drive_name = bdrv_get_device_or_node_name(bs);
-
-        /* skip automatically inserted nodes */
-        while (bs && bs->drv && bs->implicit) {
-            bs = backing_bs(bs);
-        }

hm, so, after the patch, for implicitly-filtered nodes we'll have node_name instead of device name..

But, on the other, hand, if we have implicitly-filtered node on target, we were doing wrong thing anyway, as dirty_bitmap_load_header don't skip implicit nodes.

migration/block-dirty-bitmap.c static int init_dirty_bitmap_migration(void) · line 283

Quoted 1 line
migration/block-dirty-bitmap.c +1 -0
+    for (bs = bdrv_next_all_states(NULL); bs; bs = bdrv_next_all_states(bs)) {

As I understand, difference with bdrv_next_node is that we don't skip unnamed nodes [...]

migration/block-dirty-bitmap.c static int init_dirty_bitmap_migration(void) · line 293

Quoted 10 lines
migration/block-dirty-bitmap.c +2 -1
+        const char *name = bdrv_get_device_or_node_name(bs);
 
          for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap;
               bitmap = bdrv_dirty_bitmap_next(bs, bitmap))
@@ -296,7 +290,7 @@ static int init_dirty_bitmap_migration(void)
                  continue;
              }
 
-            if (drive_name == NULL) {
+            if (!name || strcmp(name, "") == 0) {

[...] to do this (may be paranoiac, but why not?) check

migration/block-dirty-bitmap.c static int init_dirty_bitmap_migration(void) · line 313

Quoted 13 lines
migration/block-dirty-bitmap.c +1 -1
                  error_report("Found bitmap '%s' in unnamed node %p. It can't "
                               "be migrated", bdrv_dirty_bitmap_name(bitmap), bs);
                  goto fail;
@@ -313,7 +307,7 @@ static int init_dirty_bitmap_migration(void)
 
              dbms = g_new0(DirtyBitmapMigBitmapState, 1);
              dbms->bs = bs;
-            dbms->node_name = drive_name;
+            dbms->node_name = name;
              dbms->bitmap = bitmap;
              dbms->total_sectors = bdrv_nb_sectors(bs);
              dbms->sectors_per_chunk = CHUNK_SIZE * 8 *

There is still some mess about device name vs node name, and I don't know, could we somehow solve it, but patch looks OK for me anyway:

Signature · 2 lines
Best regards,
Vladimir
author
#2 lore raw
Quoted 38 lines · Vladimir Sementsov-Ogievskiy
Quoted 37 lines · John Snow

Shift from looking at every root BDS to *every* BDS. This will migrate bitmaps that are attached to blockdev created nodes instead of just ones attached to emulated storage devices.

Note that this will not migrate anonymous or internal-use bitmaps, as those are defined as having no name.

This will also fix the Coverity issues Peter Maydell has been asking about for the past several releases, as well as fixing a real bug.

Reported-by Coverity 😅
Fixes Coverity CID 1390625
CC Stefan Hajnoczi <[email protected]>
---
  migration/block-dirty-bitmap.c | 14 ++++----------
  1 file changed, 4 insertions(+), 10 deletions(-)
migration/block-dirty-bitmap.c +0 -1
index d1bb863cb6..4a896a09eb 100644
@@ -273,7 +273,6 @@ static int init_dirty_bitmap_migration(void)
      BlockDriverState *bs;
      BdrvDirtyBitmap *bitmap;
      DirtyBitmapMigBitmapState *dbms;
-    BdrvNextIterator it;
      Error *local_err = NULL;
 
      dirty_bitmap_mig_state.bulk_completed = false;
@@ -281,13 +280,8 @@ static int init_dirty_bitmap_migration(void)
      dirty_bitmap_mig_state.prev_bitmap = NULL;
      dirty_bitmap_mig_state.no_bitmaps = false;
 
Quoted 10 lines

migration/block-dirty-bitmap.c static int init_dirty_bitmap_migration(void) · line 290

Quoted 7 lines
migration/block-dirty-bitmap.c +0 -7
-    for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) {
-        const char *drive_name = bdrv_get_device_or_node_name(bs);
-
-        /* skip automatically inserted nodes */
-        while (bs && bs->drv && bs->implicit) {
-            bs = backing_bs(bs);
-        }

hm, so, after the patch, for implicitly-filtered nodes we'll have node_name instead of device name..

Oh, I see -- this does change what we send over the wire for interior/leaf nodes; that was unintentional on my part.

After my patch, this requires that if you have a manually constructed tree such that you have attached a bitmap to an interior or leaf node, you *need* to name that node so that it can be consistently reconstructed at the target.

I think that's a reasonable requirement and is actually superior to re-attaching all bitmaps to the root on migration (which would have happened before.)

Codewise, what we have currently (both before and after this patch) is:

    if (flags & DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME) {
        qemu_put_counted_string(f, dbms->node_name);
    }

So we named the constant "DEVICE_NAME", but the field was already named node_name, so this seems fine on the sending end. In practice, pre-patch we sent a device_name for any node that was the root attached to a device. Post-patch, that doesn't change because I am using the same API call to retrieve the name.

For interior/leaf nodes, we now send the node-name specifically instead of the name of the device root. This requires identically constructed (or at least compatibly named) graphs on the source and destination, which is a reasonable requirement for migration.

On the receiving end, we have this code:

    if (s->flags & DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME) {
        if (!qemu_get_counted_string(f, s->node_name)) {
            error_report("Unable to read node name string");
            return -EINVAL;
	}
        s->bs = bdrv_lookup_bs(s->node_name, s->node_name, &local_err);

which looks like a correct mapping. I think this is a safe change, even though I made it somewhat unintentionally.

Quoted 7 lines

But, on the other, hand, if we have implicitly-filtered node on target, we were doing wrong thing anyway, as dirty_bitmap_load_header don't skip implicit nodes.

migration/block-dirty-bitmap.c static int init_dirty_bitmap_migration(void) · line 283

Quoted 1 line
migration/block-dirty-bitmap.c +1 -0
+    for (bs = bdrv_next_all_states(NULL); bs; bs = bdrv_next_all_states(bs)) {

As I understand, difference with bdrv_next_node is that we don't skip unnamed nodes [...]

The difference is that we iterate over states that aren't roots of trees; so not just unnamed nodes but rather intermediate nodes as well as nodes not attached to a storage device.

bdrv_next says: Iterates over all top-level BlockDriverStates, i.e. BDSs that are owned by the monitor or attached to a BlockBackend

So this loop is going to iterate over *everything*, not just top-level nodes. This lets me skip the tree-crawling loop that didn't work quite right.

Quoted 8 lines
Quoted 8 lines
migration/block-dirty-bitmap.c +1 -0
+        const char *name = bdrv_get_device_or_node_name(bs);
 
          for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap;
               bitmap = bdrv_dirty_bitmap_next(bs, bitmap))
@@ -296,7 +290,7 @@ static int init_dirty_bitmap_migration(void)
                  continue;
              }
 
Quoted 5 lines

migration/block-dirty-bitmap.c static int init_dirty_bitmap_migration(void) · line 293

Quoted 2 lines
migration/block-dirty-bitmap.c +1 -1
-            if (drive_name == NULL) {
+            if (!name || strcmp(name, "") == 0) {

[...] to do this (may be paranoiac, but why not?) check

Because bdrv_get_device_or_node_name technically has the capability to return an empty string, though I believe in contemporary block layer that we always generate a node-name. So it might be paranoid, but it matches the API I'm calling as documented.

Quoted 5 lines
Quoted 5 lines
migration/block-dirty-bitmap.c +0 -0
                  error_report("Found bitmap '%s' in unnamed node %p. It can't "
                               "be migrated", bdrv_dirty_bitmap_name(bitmap), bs);
                  goto fail;
@@ -313,7 +307,7 @@ static int init_dirty_bitmap_migration(void)
 
Quoted 12 lines

migration/block-dirty-bitmap.c static int init_dirty_bitmap_migration(void) · line 313

Quoted 8 lines
migration/block-dirty-bitmap.c +1 -1
              dbms = g_new0(DirtyBitmapMigBitmapState, 1);
              dbms->bs = bs;
-            dbms->node_name = drive_name;
+            dbms->node_name = name;
              dbms->bitmap = bitmap;
              dbms->total_sectors = bdrv_nb_sectors(bs);
              dbms->sectors_per_chunk = CHUNK_SIZE * 8 *

There is still some mess about device name vs node name, and I don't know, could we somehow solve it, but patch looks OK for me anyway:

Yes, there's more mess, but I think this is Less Wrong™️. I will try to combat some of this confusion soon; but I'll look at your cross-node merge patch first.

Quoted 2 lines

Does this review still stand after my clarifications?

--js

#3 lore raw
Quoted 105 lines · John Snow
Quoted 38 lines · Vladimir Sementsov-Ogievskiy
Quoted 37 lines · John Snow

Shift from looking at every root BDS to *every* BDS. This will migrate bitmaps that are attached to blockdev created nodes instead of just ones attached to emulated storage devices.

Note that this will not migrate anonymous or internal-use bitmaps, as those are defined as having no name.

This will also fix the Coverity issues Peter Maydell has been asking about for the past several releases, as well as fixing a real bug.

Reported-by Coverity 😅
Fixes Coverity CID 1390625
CC Stefan Hajnoczi <[email protected]>
---
   migration/block-dirty-bitmap.c | 14 ++++----------
   1 file changed, 4 insertions(+), 10 deletions(-)
migration/block-dirty-bitmap.c +0 -1
index d1bb863cb6..4a896a09eb 100644
@@ -273,7 +273,6 @@ static int init_dirty_bitmap_migration(void)
       BlockDriverState *bs;
       BdrvDirtyBitmap *bitmap;
       DirtyBitmapMigBitmapState *dbms;
-    BdrvNextIterator it;
       Error *local_err = NULL;
 
       dirty_bitmap_mig_state.bulk_completed = false;
@@ -281,13 +280,8 @@ static int init_dirty_bitmap_migration(void)
       dirty_bitmap_mig_state.prev_bitmap = NULL;
       dirty_bitmap_mig_state.no_bitmaps = false;
 
Quoted 10 lines

migration/block-dirty-bitmap.c static int init_dirty_bitmap_migration(void) · line 290

Quoted 7 lines
migration/block-dirty-bitmap.c +0 -7
-    for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) {
-        const char *drive_name = bdrv_get_device_or_node_name(bs);
-
-        /* skip automatically inserted nodes */
-        while (bs && bs->drv && bs->implicit) {
-            bs = backing_bs(bs);
-        }

hm, so, after the patch, for implicitly-filtered nodes we'll have node_name instead of device name..

Oh, I see -- this does change what we send over the wire for interior/leaf nodes; that was unintentional on my part.

After my patch, this requires that if you have a manually constructed tree such that you have attached a bitmap to an interior or leaf node, you *need* to name that node so that it can be consistently reconstructed at the target.

I think that's a reasonable requirement and is actually superior to re-attaching all bitmaps to the root on migration (which would have happened before.)

Codewise, what we have currently (both before and after this patch) is:

     if (flags & DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME) {
         qemu_put_counted_string(f, dbms->node_name);
     }

So we named the constant "DEVICE_NAME", but the field was already named node_name, so this seems fine on the sending end. In practice, pre-patch we sent a device_name for any node that was the root attached to a device. Post-patch, that doesn't change because I am using the same API call to retrieve the name.

For interior/leaf nodes, we now send the node-name specifically instead of the name of the device root. This requires identically constructed (or at least compatibly named) graphs on the source and destination, which is a reasonable requirement for migration.

On the receiving end, we have this code:

     if (s->flags & DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME) {
         if (!qemu_get_counted_string(f, s->node_name)) {
             error_report("Unable to read node name string");
             return -EINVAL;
	}
         s->bs = bdrv_lookup_bs(s->node_name, s->node_name, &local_err);

which looks like a correct mapping. I think this is a safe change, even though I made it somewhat unintentionally.

Quoted 7 lines

But, on the other, hand, if we have implicitly-filtered node on target, we were doing wrong thing anyway, as dirty_bitmap_load_header don't skip implicit nodes.

migration/block-dirty-bitmap.c static int init_dirty_bitmap_migration(void) · line 283

Quoted 1 line
migration/block-dirty-bitmap.c +1 -0
+    for (bs = bdrv_next_all_states(NULL); bs; bs = bdrv_next_all_states(bs)) {

As I understand, difference with bdrv_next_node is that we don't skip unnamed nodes [...]

The difference is that we iterate over states that aren't roots of trees; so not just unnamed nodes but rather intermediate nodes as well as nodes not attached to a storage device.

Quoted 6 lines

bdrv_next says: Iterates over all top-level BlockDriverStates, i.e. BDSs that are owned by the monitor or attached to a BlockBackend

So this loop is going to iterate over *everything*, not just top-level nodes. This lets me skip the tree-crawling loop that didn't work quite right.

I meant not bdrv_next but bdrv_next_node, which iterates through graph nodes.. What is real difference between graph_bdrv_states and all_bdrv_states ?

Node is inserted to graph_bdrv_states in bdrv_assign_node_name(), and to all_bdrv_states in bdrv_new().

Three calls to bdrv_new:

bdrv_new_open_driver, calls bdrv_new and then bdrv_open_driver, which calls bdrv_assign_node_name, and if it fails new created node is released.

bdrv_open_inherit

    bdrv_new
    ..
    bdrv_open_common
       bdrv_open_driver
           bdrv_assign_node_name

iscsi_co_create_opts
bdrv_new

... hmm.. looks like it a kind of temporary unnamed bs

So, now, I'm not sure. Possibly we'd better use bdrv_next_node().

Kevin introduced all_bdrv_states in 0f12264e7a4145 , to use in drain instead of bdrv_next... But I don't understand, why he didn't use graph_bdrv_states and corresponding bdrv_next_node(), which is only skips some temporary or under-constuction nodes..

Quoted 38 lines
Quoted 9 lines
Quoted 9 lines
migration/block-dirty-bitmap.c +1 -0
+        const char *name = bdrv_get_device_or_node_name(bs);
 
           for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap;
                bitmap = bdrv_dirty_bitmap_next(bs, bitmap))
@@ -296,7 +290,7 @@ static int init_dirty_bitmap_migration(void)
                   continue;
               }
 
Quoted 5 lines

migration/block-dirty-bitmap.c static int init_dirty_bitmap_migration(void) · line 293

Quoted 2 lines
migration/block-dirty-bitmap.c +1 -1
-            if (drive_name == NULL) {
+            if (!name || strcmp(name, "") == 0) {

[...] to do this (may be paranoiac, but why not?) check

Because bdrv_get_device_or_node_name technically has the capability to return an empty string, though I believe in contemporary block layer that we always generate a node-name. So it might be paranoid, but it matches the API I'm calling as documented.

Quoted 5 lines
Quoted 5 lines
migration/block-dirty-bitmap.c +0 -0
                   error_report("Found bitmap '%s' in unnamed node %p. It can't "
                                "be migrated", bdrv_dirty_bitmap_name(bitmap), bs);
                   goto fail;
@@ -313,7 +307,7 @@ static int init_dirty_bitmap_migration(void)
 
Quoted 12 lines

migration/block-dirty-bitmap.c static int init_dirty_bitmap_migration(void) · line 313

Quoted 8 lines
migration/block-dirty-bitmap.c +1 -1
               dbms = g_new0(DirtyBitmapMigBitmapState, 1);
               dbms->bs = bs;
-            dbms->node_name = drive_name;
+            dbms->node_name = name;
               dbms->bitmap = bitmap;
               dbms->total_sectors = bdrv_nb_sectors(bs);
               dbms->sectors_per_chunk = CHUNK_SIZE * 8 *

There is still some mess about device name vs node name, and I don't know, could we somehow solve it, but patch looks OK for me anyway:

Quoted 9 lines

Yes, there's more mess, but I think this is Less Wrong™️. I will try to combat some of this confusion soon; but I'll look at your cross-node merge patch first.

Quoted 2 lines

Does this review still stand after my clarifications?

Let's figure out graph_bdrv_states vs all_bdrv_states first.

Signature · 2 lines
Best regards,
Vladimir
#4 lore raw

Am 17.05.2019 um 12:22 hat Vladimir Sementsov-Ogievskiy geschrieben:

Quoted 17 lines
Quoted 16 lines · John Snow
Quoted 7 lines · Vladimir Sementsov-Ogievskiy

But, on the other, hand, if we have implicitly-filtered node on target, we were doing wrong thing anyway, as dirty_bitmap_load_header don't skip implicit nodes.

Quoted 1 line

+ for (bs = bdrv_next_all_states(NULL); bs; bs = bdrv_next_all_states(bs)) {

As I understand, difference with bdrv_next_node is that we don't skip unnamed nodes [...]

The difference is that we iterate over states that aren't roots of trees; so not just unnamed nodes but rather intermediate nodes as well as nodes not attached to a storage device.

bdrv_next says: Iterates over all top-level BlockDriverStates, i.e. BDSs that are owned by the monitor or attached to a BlockBackend

Quoted 6 lines
Quoted 3 lines

So this loop is going to iterate over *everything*, not just top-level nodes. This lets me skip the tree-crawling loop that didn't work quite right.

I meant not bdrv_next but bdrv_next_node, which iterates through graph nodes.. What is real difference between graph_bdrv_states and all_bdrv_states ?

I don't think there is any relevant difference any more since commit 15489c769b9 ('block: auto-generated node-names'). We can only see a difference if a BDS was created, but never opened. This means either that we are still in the process of opening an image or that we have a bug somewhere.

Maybe we should remove graph_bdrv_states and replace all of its uses with the more obviously correct all_bdrv_states (if we are sure that all users aren't called between creating and opening a BDS).

Quoted 16 lines

Node is inserted to graph_bdrv_states in bdrv_assign_node_name(), and to all_bdrv_states in bdrv_new().

Three calls to bdrv_new:

bdrv_new_open_driver, calls bdrv_new and then bdrv_open_driver, which calls bdrv_assign_node_name, and if it fails new created node is released.

bdrv_open_inherit

    bdrv_new
    ..
    bdrv_open_common
       bdrv_open_driver
           bdrv_assign_node_name
Quoted 6 lines

iscsi_co_create_opts
bdrv_new

... hmm.. looks like it a kind of temporary unnamed bs

So, now, I'm not sure. Possibly we'd better use bdrv_next_node().

I wonder if the iscsi one can't be replaced with bdrv_new_open_driver(). Manually building a half-opened BDS like it does currently looks suspicious.

Quoted 4 lines

Kevin introduced all_bdrv_states in 0f12264e7a4145 , to use in drain instead of bdrv_next... But I don't understand, why he didn't use graph_bdrv_states and corresponding bdrv_next_node(), which is only skips some temporary or under-constuction nodes..

I probably just didn't realise that graph_bdrv_states exists and is effectively the same. I knew that all_bdrv_states contains what I want, so I just wanted to access that.

But if anonymous BDSes did actually still exist, drain would want to wait for those as well, so it's the more natural choice anyway.

Kevin

#5 lore raw
Quoted 67 lines · Kevin Wolf

Am 17.05.2019 um 12:22 hat Vladimir Sementsov-Ogievskiy geschrieben:

Quoted 17 lines
Quoted 16 lines · John Snow
Quoted 7 lines · Vladimir Sementsov-Ogievskiy

But, on the other, hand, if we have implicitly-filtered node on target, we were doing wrong thing anyway, as dirty_bitmap_load_header don't skip implicit nodes.

Quoted 1 line

+ for (bs = bdrv_next_all_states(NULL); bs; bs = bdrv_next_all_states(bs)) {

As I understand, difference with bdrv_next_node is that we don't skip unnamed nodes [...]

The difference is that we iterate over states that aren't roots of trees; so not just unnamed nodes but rather intermediate nodes as well as nodes not attached to a storage device.

bdrv_next says: Iterates over all top-level BlockDriverStates, i.e. BDSs that are owned by the monitor or attached to a BlockBackend

Quoted 6 lines
Quoted 3 lines

So this loop is going to iterate over *everything*, not just top-level nodes. This lets me skip the tree-crawling loop that didn't work quite right.

I meant not bdrv_next but bdrv_next_node, which iterates through graph nodes.. What is real difference between graph_bdrv_states and all_bdrv_states ?

I don't think there is any relevant difference any more since commit 15489c769b9 ('block: auto-generated node-names'). We can only see a difference if a BDS was created, but never opened. This means either that we are still in the process of opening an image or that we have a bug somewhere.

Maybe we should remove graph_bdrv_states and replace all of its uses with the more obviously correct all_bdrv_states (if we are sure that all users aren't called between creating and opening a BDS).

Quoted 16 lines

Node is inserted to graph_bdrv_states in bdrv_assign_node_name(), and to all_bdrv_states in bdrv_new().

Three calls to bdrv_new:

bdrv_new_open_driver, calls bdrv_new and then bdrv_open_driver, which calls bdrv_assign_node_name, and if it fails new created node is released.

bdrv_open_inherit

     bdrv_new
     ..
     bdrv_open_common
        bdrv_open_driver
            bdrv_assign_node_name
Quoted 6 lines

iscsi_co_create_opts
bdrv_new

... hmm.. looks like it a kind of temporary unnamed bs

So, now, I'm not sure. Possibly we'd better use bdrv_next_node().

I wonder if the iscsi one can't be replaced with bdrv_new_open_driver(). Manually building a half-opened BDS like it does currently looks suspicious.

Quoted 4 lines

Kevin introduced all_bdrv_states in 0f12264e7a4145 , to use in drain instead of bdrv_next... But I don't understand, why he didn't use graph_bdrv_states and corresponding bdrv_next_node(), which is only skips some temporary or under-constuction nodes..

Quoted 9 lines

I probably just didn't realise that graph_bdrv_states exists and is effectively the same. I knew that all_bdrv_states contains what I want, so I just wanted to access that.

But if anonymous BDSes did actually still exist, drain would want to wait for those as well, so it's the more natural choice anyway.

Kevin

Thank you for clarification. Then, my r-b still stand here, and fixing/refacting graph_nodes vs all_states should be a separate thing.

Signature · 2 lines
Best regards,
Vladimir
author
#6 lore raw
Quoted 75 lines · Vladimir Sementsov-Ogievskiy
Quoted 74 lines · Kevin Wolf

Am 17.05.2019 um 12:22 hat Vladimir Sementsov-Ogievskiy geschrieben:

Quoted 17 lines
Quoted 16 lines · John Snow
Quoted 7 lines · Vladimir Sementsov-Ogievskiy

But, on the other, hand, if we have implicitly-filtered node on target, we were doing wrong thing anyway, as dirty_bitmap_load_header don't skip implicit nodes.

Quoted 1 line

+ for (bs = bdrv_next_all_states(NULL); bs; bs = bdrv_next_all_states(bs)) {

As I understand, difference with bdrv_next_node is that we don't skip unnamed nodes [...]

The difference is that we iterate over states that aren't roots of trees; so not just unnamed nodes but rather intermediate nodes as well as nodes not attached to a storage device.

bdrv_next says: Iterates over all top-level BlockDriverStates, i.e. BDSs that are owned by the monitor or attached to a BlockBackend

Quoted 6 lines
Quoted 3 lines

So this loop is going to iterate over *everything*, not just top-level nodes. This lets me skip the tree-crawling loop that didn't work quite right.

I meant not bdrv_next but bdrv_next_node, which iterates through graph nodes.. What is real difference between graph_bdrv_states and all_bdrv_states ?

I don't think there is any relevant difference any more since commit 15489c769b9 ('block: auto-generated node-names'). We can only see a difference if a BDS was created, but never opened. This means either that we are still in the process of opening an image or that we have a bug somewhere.

Maybe we should remove graph_bdrv_states and replace all of its uses with the more obviously correct all_bdrv_states (if we are sure that all users aren't called between creating and opening a BDS).

Quoted 16 lines

Node is inserted to graph_bdrv_states in bdrv_assign_node_name(), and to all_bdrv_states in bdrv_new().

Three calls to bdrv_new:

bdrv_new_open_driver, calls bdrv_new and then bdrv_open_driver, which calls bdrv_assign_node_name, and if it fails new created node is released.

bdrv_open_inherit

     bdrv_new
     ..
     bdrv_open_common
        bdrv_open_driver
            bdrv_assign_node_name
Quoted 6 lines

iscsi_co_create_opts
bdrv_new

... hmm.. looks like it a kind of temporary unnamed bs

So, now, I'm not sure. Possibly we'd better use bdrv_next_node().

I wonder if the iscsi one can't be replaced with bdrv_new_open_driver(). Manually building a half-opened BDS like it does currently looks suspicious.

Quoted 4 lines

Kevin introduced all_bdrv_states in 0f12264e7a4145 , to use in drain instead of bdrv_next... But I don't understand, why he didn't use graph_bdrv_states and corresponding bdrv_next_node(), which is only skips some temporary or under-constuction nodes..

I probably just didn't realise that graph_bdrv_states exists and is effectively the same. I knew that all_bdrv_states contains what I want, so I just wanted to access that.

But if anonymous BDSes did actually still exist, drain would want to wait for those as well, so it's the more natural choice anyway.

Quoted 8 lines
Quoted 2 lines

Kevin

Thank you for clarification. Then, my r-b still stand here, and fixing/refacting graph_nodes vs all_states should be a separate thing.

Great, thanks all :)

I think I will stage this through my tree -- on the premise that David Gilbert won't want to stage a block patch, and that since it's not Kevin's tree, he won't mind either.

--js

#7 lore raw
In reply to John Snow
Quoted 4 lines · John Snow

Shift from looking at every root BDS to *every* BDS. This will migrate bitmaps that are attached to blockdev created nodes instead of just ones attached to emulated storage devices.

Quoted 8 lines

Note that this will not migrate anonymous or internal-use bitmaps, as those are defined as having no name.

This will also fix the Coverity issues Peter Maydell has been asking about for the past several releases, as well as fixing a real bug.

Reported-by Coverity 😅

What was the coverity number (I don't believe that it was smile:)?

Do someone know, that this patch fixes very-very-very terrible bug?

Before this patch, here were bdrv_next-based loop, with exists from it, but not using bdrv_next_cleanup(). This leads to leaked (incremented) refcnt of bds on any failure during this loop!

Now we faced this bug, in Rhel-based Qemu, so I strongly recommend to fix it in Rhel.

Quoted 56 lines
---
  migration/block-dirty-bitmap.c | 14 ++++----------
  1 file changed, 4 insertions(+), 10 deletions(-)
migration/block-dirty-bitmap.c +4 -10
index d1bb863cb6..4a896a09eb 100644
@@ -273,7 +273,6 @@ static int init_dirty_bitmap_migration(void)
      BlockDriverState *bs;
      BdrvDirtyBitmap *bitmap;
      DirtyBitmapMigBitmapState *dbms;
-    BdrvNextIterator it;
      Error *local_err = NULL;
 
      dirty_bitmap_mig_state.bulk_completed = false;
@@ -281,13 +280,8 @@ static int init_dirty_bitmap_migration(void)
      dirty_bitmap_mig_state.prev_bitmap = NULL;
      dirty_bitmap_mig_state.no_bitmaps = false;
 
-    for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) {
-        const char *drive_name = bdrv_get_device_or_node_name(bs);
-
-        /* skip automatically inserted nodes */
-        while (bs && bs->drv && bs->implicit) {
-            bs = backing_bs(bs);
-        }
+    for (bs = bdrv_next_all_states(NULL); bs; bs = bdrv_next_all_states(bs)) {
+        const char *name = bdrv_get_device_or_node_name(bs);
 
          for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap;
               bitmap = bdrv_dirty_bitmap_next(bs, bitmap))
@@ -296,7 +290,7 @@ static int init_dirty_bitmap_migration(void)
                  continue;
              }
 
-            if (drive_name == NULL) {
+            if (!name || strcmp(name, "") == 0) {
                  error_report("Found bitmap '%s' in unnamed node %p. It can't "
                               "be migrated", bdrv_dirty_bitmap_name(bitmap), bs);
                  goto fail;
@@ -313,7 +307,7 @@ static int init_dirty_bitmap_migration(void)
 
              dbms = g_new0(DirtyBitmapMigBitmapState, 1);
              dbms->bs = bs;
-            dbms->node_name = drive_name;
+            dbms->node_name = name;
              dbms->bitmap = bitmap;
              dbms->total_sectors = bdrv_nb_sectors(bs);
              dbms->sectors_per_chunk = CHUNK_SIZE * 8 *
Signature · 2 lines
Best regards,
Vladimir
author Reviewed-by Vladimir Sementsov-Ogievskiy
#8 lore raw
Quoted 8 lines · Vladimir Sementsov-Ogievskiy
Quoted 7 lines · John Snow

Shift from looking at every root BDS to *every* BDS. This will migrate bitmaps that are attached to blockdev created nodes instead of just ones attached to emulated storage devices.

Note that this will not migrate anonymous or internal-use bitmaps, as those are defined as having no name.

Quoted 8 lines
Quoted 5 lines

This will also fix the Coverity issues Peter Maydell has been asking about for the past several releases, as well as fixing a real bug.

Reported-by Coverity 😅

What was the coverity number (I don't believe that it was smile:)?

Message-id: [email protected]

Fixes Coverity CID 1390625
CC Stefan Hajnoczi <[email protected]>
Quoted 7 lines

Do someone know, that this patch fixes very-very-very terrible bug?

Before this patch, here were bdrv_next-based loop, with exists from it, but not using bdrv_next_cleanup(). This leads to leaked (incremented) refcnt of bds on any failure during this loop!

Now we faced this bug, in Rhel-based Qemu, so I strongly recommend to fix it in Rhel.

OK, this was fixed for 4.1, and was introduced in b35ebdf076d for 2.12.0, so all versions between have the problem.

Quoted 59 lines
Quoted 57 lines
---
  migration/block-dirty-bitmap.c | 14 ++++----------
  1 file changed, 4 insertions(+), 10 deletions(-)
migration/block-dirty-bitmap.c +4 -10
index d1bb863cb6..4a896a09eb 100644
@@ -273,7 +273,6 @@ static int init_dirty_bitmap_migration(void)
      BlockDriverState *bs;
      BdrvDirtyBitmap *bitmap;
      DirtyBitmapMigBitmapState *dbms;
-    BdrvNextIterator it;
      Error *local_err = NULL;
 
      dirty_bitmap_mig_state.bulk_completed = false;
@@ -281,13 +280,8 @@ static int init_dirty_bitmap_migration(void)
      dirty_bitmap_mig_state.prev_bitmap = NULL;
      dirty_bitmap_mig_state.no_bitmaps = false;
 
-    for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) {
-        const char *drive_name = bdrv_get_device_or_node_name(bs);
-
-        /* skip automatically inserted nodes */
-        while (bs && bs->drv && bs->implicit) {
-            bs = backing_bs(bs);
-        }
+    for (bs = bdrv_next_all_states(NULL); bs; bs = bdrv_next_all_states(bs)) {
+        const char *name = bdrv_get_device_or_node_name(bs);
 
          for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap;
               bitmap = bdrv_dirty_bitmap_next(bs, bitmap))
@@ -296,7 +290,7 @@ static int init_dirty_bitmap_migration(void)
                  continue;
              }
 
-            if (drive_name == NULL) {
+            if (!name || strcmp(name, "") == 0) {
                  error_report("Found bitmap '%s' in unnamed node %p. It can't "
                               "be migrated", bdrv_dirty_bitmap_name(bitmap), bs);
                  goto fail;
@@ -313,7 +307,7 @@ static int init_dirty_bitmap_migration(void)
 
              dbms = g_new0(DirtyBitmapMigBitmapState, 1);
              dbms->bs = bs;
-            dbms->node_name = drive_name;
+            dbms->node_name = name;
              dbms->bitmap = bitmap;
              dbms->total_sectors = bdrv_nb_sectors(bs);
              dbms->sectors_per_chunk = CHUNK_SIZE * 8 *
author
#9 lore raw

(off list)

Quoted 34 lines · John Snow
Quoted 8 lines · Vladimir Sementsov-Ogievskiy
Quoted 7 lines · John Snow

Shift from looking at every root BDS to *every* BDS. This will migrate bitmaps that are attached to blockdev created nodes instead of just ones attached to emulated storage devices.

Note that this will not migrate anonymous or internal-use bitmaps, as those are defined as having no name.

Quoted 8 lines
Quoted 5 lines

This will also fix the Coverity issues Peter Maydell has been asking about for the past several releases, as well as fixing a real bug.

Reported-by Coverity 😅

What was the coverity number (I don't believe that it was smile:)?

Message-id: [email protected]

Fixes Coverity CID 1390625
CC Stefan Hajnoczi <[email protected]>
Quoted 2 lines

Do someone know, that this patch fixes very-very-very terrible bug?

Quoted 9 lines
Quoted 5 lines

Before this patch, here were bdrv_next-based loop, with exists from it, but not using bdrv_next_cleanup(). This leads to leaked (incremented) refcnt of bds on any failure during this loop!

Now we faced this bug, in Rhel-based Qemu, so I strongly recommend to fix it in Rhel.

OK, this was fixed for 4.1, and was introduced in b35ebdf076d for 2.12.0, so all versions between have the problem.

As far as I know, we don't "support" incremental backup for RHEL based packages, because we only support what you can do directly through libvirt. And since RHEL libvirt doesn't have incremental backup, ...

I can try to fix it anyway, though, if it makes your life easier especially.

Which version(s) are you using? I'll try to target a fix for that version, but it will likely be a special fix that just fixes the leak without changing the enumeration method, to keep migration ABI consistent with what we expect from the different versions.

--js

author
#10 lore raw
Quoted 1 line · John Snow

(off list)

lol, or not. No big deal. all the code goes upstream anyway (:

#11 lore raw
In reply to John Snow
Quoted 31 lines · John Snow

(off list)

Quoted 28 lines · John Snow
Quoted 8 lines · Vladimir Sementsov-Ogievskiy
Quoted 7 lines · John Snow

Shift from looking at every root BDS to *every* BDS. This will migrate bitmaps that are attached to blockdev created nodes instead of just ones attached to emulated storage devices.

Note that this will not migrate anonymous or internal-use bitmaps, as those are defined as having no name.

Quoted 8 lines
Quoted 5 lines

This will also fix the Coverity issues Peter Maydell has been asking about for the past several releases, as well as fixing a real bug.

Reported-by Coverity 😅

What was the coverity number (I don't believe that it was smile:)?

Message-id: [email protected]
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1652490
Fixes: Coverity CID 1390625

Ah, missed it, sorry.

Quoted 16 lines
Quoted 6 lines
CC Stefan Hajnoczi <[email protected]>
Quoted 2 lines

Do someone know, that this patch fixes very-very-very terrible bug?

Quoted 9 lines
Quoted 5 lines

Before this patch, here were bdrv_next-based loop, with exists from it, but not using bdrv_next_cleanup(). This leads to leaked (incremented) refcnt of bds on any failure during this loop!

Now we faced this bug, in Rhel-based Qemu, so I strongly recommend to fix it in Rhel.

OK, this was fixed for 4.1, and was introduced in b35ebdf076d for 2.12.0, so all versions between have the problem.

Quoted 5 lines

As far as I know, we don't "support" incremental backup for RHEL based packages, because we only support what you can do directly through libvirt. And since RHEL libvirt doesn't have incremental backup, ...

I can try to fix it anyway, though, if it makes your life easier especially.

No, actually, it's no real difference, now we fixed it in our branch. So, it's up to you.

Quoted 2 lines

Which version(s) are you using?

2.12.0-33

Quoted 7 lines

I'll try to target a fix for that
version, but it will likely be a special fix that just fixes the leak without changing the enumeration method, to keep migration ABI consistent with what we expect from the different versions.

--js

Signature · 2 lines
Best regards,
Vladimir
author
#12 lore raw
Quoted 3 lines · Vladimir Sementsov-Ogievskiy
Quoted 2 lines · John Snow

(off list)

Well, it isn't now, but oh well. :)

Quoted 30 lines
Quoted 21 lines · John Snow
Quoted 20 lines · Vladimir Sementsov-Ogievskiy
Quoted 16 lines
Quoted 4 lines · John Snow

Shift from looking at every root BDS to *every* BDS. This will migrate bitmaps that are attached to blockdev created nodes instead of just ones attached to emulated storage devices.

Quoted 8 lines

Note that this will not migrate anonymous or internal-use bitmaps, as those are defined as having no name.

This will also fix the Coverity issues Peter Maydell has been asking about for the past several releases, as well as fixing a real bug.

Reported-by Coverity 😅

What was the coverity number (I don't believe that it was smile:)?

Quoted 8 lines
Quoted 8 lines

Message-id: [email protected]
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1652490
Fixes: Coverity CID 1390625

Quoted 2 lines

Ah, missed it, sorry.

No problem -- it got added for the PR instead of the patch.

Quoted 16 lines
Quoted 16 lines
Quoted 15 lines
CC Stefan Hajnoczi <[email protected]>
Quoted 7 lines

Do someone know, that this patch fixes very-very-very terrible bug?

Before this patch, here were bdrv_next-based loop, with exists from it, but not using bdrv_next_cleanup(). This leads to leaked (incremented) refcnt of bds on any failure during this loop!

Now we faced this bug, in Rhel-based Qemu, so I strongly recommend to fix it in Rhel.

OK, this was fixed for 4.1, and was introduced in b35ebdf076d for 2.12.0, so all versions between have the problem.

Quoted 9 lines
Quoted 5 lines

As far as I know, we don't "support" incremental backup for RHEL based packages, because we only support what you can do directly through libvirt. And since RHEL libvirt doesn't have incremental backup, ...

I can try to fix it anyway, though, if it makes your life easier especially.

No, actually, it's no real difference, now we fixed it in our branch. So, it's up to you.

OK, thanks for the report, then!

--js