Skip to contentSkip to conversation

qcow2: Save another common flush

PATCH In review Kevin Wolf 2 participants 8 messages Sep 17, 2010 Block layer

lore
Share & tools
lore

Kevin Wolf

author
#0 lore raw

cover qcow2: Save another common flush

For copy on write (this includes any cluster allocations that don't fill the whole cluster with one request), what qcow2 does looks like this:

1. Allocate new clusters (increase refcounts)
2. bdrv_flush
3. Copy sectors before the first touched one
4. bdrv_flush
5. Copy sectors after the last touched one
6. bdrv_flush
7. Update the L2 table to point to the new clusters

Step 2 and 4 are not necessary. This series moves flushes around to get all of these three bdrv_flush calls merged into one.

Kevin Wolf (4):
qcow2: Move sync out of write_refcount_block_entries
qcow2: Move sync out of update_refcount
qcow2: Move sync out of qcow2_alloc_clusters
qcow2: Get rid of additional sync on COW

 block/qcow2-cluster.c  |   11 ++++++++++-
 block/qcow2-refcount.c |   13 ++++++++++++-
 block/qcow2-snapshot.c |    2 ++
 3 files changed, 24 insertions(+), 2 deletions(-)
Signature · 1 line
1.7.2.2
author
#1 lore raw

PATCH 1/4 qcow2: Move sync out of write_refcount_block_entries

---
 block/qcow2-refcount.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)
block/qcow2-refcount.c +3 -1
index 4c19e7e..7dc75d1 100644
@@ -444,7 +444,7 @@ static int write_refcount_block_entries(BlockDriverState *bs,
     size = (last_index - first_index) << REFCOUNT_SHIFT;
 
     BLKDBG_EVENT(bs->file, BLKDBG_REFBLOCK_UPDATE_PART);
-    ret = bdrv_pwrite_sync(bs->file,
+    ret = bdrv_pwrite(bs->file,
         refcount_block_offset + (first_index << REFCOUNT_SHIFT),
         &s->refcount_block_cache[first_index], size);
     if (ret < 0) {
@@ -551,6 +551,8 @@ fail:
         dummy = update_refcount(bs, offset, cluster_offset - offset, -addend);
     }
 
+    bdrv_flush(bs->file);
+
     return ret;
 }
 
Signature · 1 line
1.7.2.2
author
#2 lore raw

PATCH 2/4 qcow2: Move sync out of update_refcount

In reply to Kevin Wolf

Note that the flush is omitted intentionally in qcow2_free_clusters. If anything, we can leak clusters here if we lose the writes.

---
 block/qcow2-refcount.c |   13 +++++++++++--
 1 files changed, 11 insertions(+), 2 deletions(-)
block/qcow2-refcount.c +11 -2
index 7dc75d1..4fc3f80 100644
@@ -261,6 +261,8 @@ static int64_t alloc_refcount_block(BlockDriverState *bs, int64_t cluster_index)
             goto fail_block;
         }
 
+        bdrv_flush(bs->file);
+
         /* Initialize the new refcount block only after updating its refcount,
          * update_refcount uses the refcount cache itself */
         memset(s->refcount_block_cache, 0, s->cluster_size);
@@ -551,8 +553,6 @@ fail:
         dummy = update_refcount(bs, offset, cluster_offset - offset, -addend);
     }
 
-    bdrv_flush(bs->file);
-
     return ret;
 }
 
@@ -575,6 +575,8 @@ static int update_cluster_refcount(BlockDriverState *bs,
         return ret;
     }
 
+    bdrv_flush(bs->file);
+
     return get_refcount(bs, cluster_index);
 }
 
@@ -626,6 +628,9 @@ int64_t qcow2_alloc_clusters(BlockDriverState *bs, int64_t size)
     if (ret < 0) {
         return ret;
     }
+
+    bdrv_flush(bs->file);
+
     return offset;
 }
 
@@ -803,6 +808,10 @@ int qcow2_update_snapshot_refcount(BlockDriverState *bs,
                             if (ret < 0) {
                                 goto fail;
                             }
+
+                            /* TODO Flushing once for the whole function should
+                             * be enough */
+                            bdrv_flush(bs->file);
                         }
                         /* compressed clusters are never modified */
                         refcount = 2;
Signature · 1 line
1.7.2.2
#3 lore raw

Re: [Qemu-devel] [PATCH 2/4] qcow2: Move sync out of update_refcount

Quoted 5 lines · Kevin Wolf

Note that the flush is omitted intentionally in qcow2_free_clusters. If anything, we can leak clusters here if we lose the writes.

Cluster leaking gets picked up by bdrv_check though, right?

I think I've convinced myself that leaking clusters is not an acceptable behavior from a security perspective but as long as it's detectable via bdrv_check, qcow2 could implement an online check to address it.

Regards,

Anthony Liguori

Quoted 57 lines
---
  block/qcow2-refcount.c |   13 +++++++++++--
  1 files changed, 11 insertions(+), 2 deletions(-)
block/qcow2-refcount.c +11 -2
index 7dc75d1..4fc3f80 100644
@@ -261,6 +261,8 @@ static int64_t alloc_refcount_block(BlockDriverState *bs, int64_t cluster_index)
              goto fail_block;
          }
 
+        bdrv_flush(bs->file);
+
          /* Initialize the new refcount block only after updating its refcount,
           * update_refcount uses the refcount cache itself */
          memset(s->refcount_block_cache, 0, s->cluster_size);
@@ -551,8 +553,6 @@ fail:
          dummy = update_refcount(bs, offset, cluster_offset - offset, -addend);
      }
 
-    bdrv_flush(bs->file);
-
      return ret;
  }
 
@@ -575,6 +575,8 @@ static int update_cluster_refcount(BlockDriverState *bs,
          return ret;
      }
 
+    bdrv_flush(bs->file);
+
      return get_refcount(bs, cluster_index);
  }
 
@@ -626,6 +628,9 @@ int64_t qcow2_alloc_clusters(BlockDriverState *bs, int64_t size)
      if (ret<  0) {
          return ret;
      }
+
+    bdrv_flush(bs->file);
+
      return offset;
  }
 
@@ -803,6 +808,10 @@ int qcow2_update_snapshot_refcount(BlockDriverState *bs,
                              if (ret<  0) {
                                  goto fail;
                              }
+
+                            /* TODO Flushing once for the whole function should
+                             * be enough */
+                            bdrv_flush(bs->file);
                          }
                          /* compressed clusters are never modified */
                          refcount = 2;
author
#4 lore raw

Re: [Qemu-devel] [PATCH 2/4] qcow2: Move sync out of update_refcount

Am 17.09.2010 19:06, schrieb Anthony Liguori:

Quoted 12 lines · Kevin Wolf
Quoted 5 lines

Note that the flush is omitted intentionally in qcow2_free_clusters. If anything, we can leak clusters here if we lose the writes.

Cluster leaking gets picked up by bdrv_check though, right?

I think I've convinced myself that leaking clusters is not an acceptable behavior from a security perspective but as long as it's detectable via bdrv_check, qcow2 could implement an online check to address it.

Leaking clusters on crashes is unavoidable. But yes, qemu-img check does detect this.

Kevin

author
#5 lore raw

PATCH 3/4 qcow2: Move sync out of qcow2_alloc_clusters

In reply to Kevin Wolf
---
 block/qcow2-cluster.c  |    3 +++
 block/qcow2-refcount.c |    4 ++--
 block/qcow2-snapshot.c |    2 ++
 3 files changed, 7 insertions(+), 2 deletions(-)
block/qcow2-cluster.c +3 -0
index 91fa9ac..66969c2 100644
@@ -60,6 +60,7 @@ int qcow2_grow_l1_table(BlockDriverState *bs, int min_size)
         qemu_free(new_l1_table);
         return new_l1_table_offset;
     }
+    bdrv_flush(bs->file);
 
     BLKDBG_EVENT(bs->file, BLKDBG_L1_GROW_WRITE_TABLE);
     for(i = 0; i < s->l1_size; i++)
@@ -244,6 +245,7 @@ static int l2_allocate(BlockDriverState *bs, int l1_index, uint64_t **table)
     if (l2_offset < 0) {
         return l2_offset;
     }
+    bdrv_flush(bs->file);
 
     /* allocate a new entry in the l2 cache */
 
@@ -870,6 +872,7 @@ int qcow2_alloc_cluster_offset(BlockDriverState *bs, uint64_t offset,
         QLIST_REMOVE(m, next_in_flight);
         return cluster_offset;
     }
+    bdrv_flush(bs->file);
 
     /* save info needed for meta data update */
     m->offset = offset;
block/qcow2-refcount.c +2 -2
index 4fc3f80..7082601 100644
@@ -629,8 +629,6 @@ int64_t qcow2_alloc_clusters(BlockDriverState *bs, int64_t size)
         return ret;
     }
 
-    bdrv_flush(bs->file);
-
     return offset;
 }
 
@@ -678,6 +676,8 @@ int64_t qcow2_alloc_bytes(BlockDriverState *bs, int size)
             goto redo;
         }
     }
+
+    bdrv_flush(bs->file);
     return offset;
 }
 
block/qcow2-snapshot.c +2 -0
index e663e8b..e429920 100644
@@ -138,6 +138,7 @@ static int qcow_write_snapshots(BlockDriverState *bs)
     snapshots_size = offset;
 
     snapshots_offset = qcow2_alloc_clusters(bs, snapshots_size);
+    bdrv_flush(bs->file);
     offset = snapshots_offset;
     if (offset < 0) {
         return offset;
@@ -271,6 +272,7 @@ int qcow2_snapshot_create(BlockDriverState *bs, QEMUSnapshotInfo *sn_info)
     if (l1_table_offset < 0) {
         goto fail;
     }
+    bdrv_flush(bs->file);
 
     sn->l1_table_offset = l1_table_offset;
     sn->l1_size = s->l1_size;
Signature · 1 line
1.7.2.2
author
#6 lore raw

PATCH 4/4 qcow2: Get rid of additional sync on COW

In reply to Kevin Wolf

We always have a sync for the refcount update when a new cluster is allocated. If we move this past the COW, we can save an additional sync.

---
 block/qcow2-cluster.c |   10 ++++++++--
 1 files changed, 8 insertions(+), 2 deletions(-)
block/qcow2-cluster.c +8 -2
index 66969c2..634ccef 100644
@@ -422,7 +422,7 @@ static int copy_sectors(BlockDriverState *bs, uint64_t start_sect,
                         &s->aes_encrypt_key);
     }
     BLKDBG_EVENT(bs->file, BLKDBG_COW_WRITE);
-    ret = bdrv_write_sync(bs->file, (cluster_offset >> 9) + n_start,
+    ret = bdrv_write(bs->file, (cluster_offset >> 9) + n_start,
         s->cluster_data, n);
     if (ret < 0)
         return ret;
@@ -721,6 +721,13 @@ int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, QCowL2Meta *m)
                     (i << s->cluster_bits)) | QCOW_OFLAG_COPIED);
      }
 
+    /*
+     * Before we update the L2 table to actually point to the new cluster, we
+     * need to be sure that the refcounts have been increased and COW was
+     * handled.
+     */
+    bdrv_flush(bs->file);
+
     ret = write_l2_entries(bs, l2_table, l2_offset, l2_index, m->nb_clusters);
     if (ret < 0) {
         qcow2_l2_cache_reset(bs);
@@ -872,7 +879,6 @@ int qcow2_alloc_cluster_offset(BlockDriverState *bs, uint64_t offset,
         QLIST_REMOVE(m, next_in_flight);
         return cluster_offset;
     }
-    bdrv_flush(bs->file);
 
     /* save info needed for meta data update */
     m->offset = offset;
Signature · 1 line
1.7.2.2
#7 lore raw
In reply to Kevin Wolf
Quoted 11 lines · Kevin Wolf

For copy on write (this includes any cluster allocations that don't fill the whole cluster with one request), what qcow2 does looks like this:

1. Allocate new clusters (increase refcounts)
2. bdrv_flush
3. Copy sectors before the first touched one
4. bdrv_flush
5. Copy sectors after the last touched one
6. bdrv_flush
7. Update the L2 table to point to the new clusters

Quoted 3 lines

Step 2 and 4 are not necessary. This series moves flushes around to get all of these three bdrv_flush calls merged into one.

Makes sense to me.

Regards,

Anthony Liguori

Quoted 12 lines

Kevin Wolf (4):
qcow2: Move sync out of write_refcount_block_entries
qcow2: Move sync out of update_refcount
qcow2: Move sync out of qcow2_alloc_clusters
qcow2: Get rid of additional sync on COW

  block/qcow2-cluster.c  |   11 ++++++++++-
  block/qcow2-refcount.c |   13 ++++++++++++-
  block/qcow2-snapshot.c |    2 ++
  3 files changed, 24 insertions(+), 2 deletions(-)