Skip to contentSkip to conversation

block/stream:add flush l2_table_cache, ensure data integrity

PATCH New Evanzhang 3 participants 6 messages Jul 24, 2023 – Jul 26, 2023 Block layer

lore
Share & tools
lore
author
#1 lore raw

PATCH 1/1 block/stream:add flush l2_table_cache, ensure data integrity

In reply to Evanzhang

block_stream will not actively flush l2_table_cache,when qemu process exception exit,causing disk data loss

---
 block/stream.c | 6 ++++++
 1 file changed, 6 insertions(+)
block/stream.c +6 -0
index e522bbd..a5e08da 100644
@@ -207,6 +207,12 @@ static int coroutine_fn stream_run(Job *job, Error **errp)
         }
     }
 
+    /*
+     * Complete stream_populate,force flush l2_table_cache,to
+     * avoid unexpected termination of process, l2_table loss
+     */
+    qcow2_cache_flush(bs, ((BDRVQcow2State *)bs->opaque)->l2_table_cache);
+
     /* Do not remove the backing file if an error was there but ignored. */
     return error;
 }
Signature · 1 line
2.9.5

Evanzhang

author
#0 lore raw

PATCH 1/1 block/stream:add flush l2_table_cache, ensure data integrity

Verification steps:

1.qemu-img  create -f  qcow2 data1.img 10M
  qemu-img  create -f  qcow2 -b data1.img data1.qcow2

2.write 1M data to data1.img

3.create vm use data1.qcow2 as the data disk
/usr/libexec/qemu-kvm -M pc,accel=kvm -smp 4 -cpu host -m 4g -drive file=./centos_7.5_64_20200603.qcow2,if=none,id=drive-virtio0,cache=none, aio=native -device virtio-blk-pci,drive=drive-virtio0,id=virtio0 -drive file=./data1.qcow2,if=none,id=drive-virtio1,cache=none,l2-cache-size= 1048576,aio=native -device virtio-blk-pci,drive=drive-virtio1,id=virtio1 -vnc :101 -qmp stdio

4.{"execute": "block-stream", "arguments":{"device":"drive-virtio1"}}

5.kill -9 $(pidof qemu-kvm)

6.md5sum data1.img data1.qcow2,check if it is the same

Evanzhang (1):
block/stream:add flush l2_table_cache,ensure data integrity

 block/stream.c | 6 ++++++
 1 file changed, 6 insertions(+)
Signature · 1 line
2.9.5
#2 lore raw

Re: [PATCH v1] block/stream:add flush l2_table_cache,ensure data integrity

In reply to Evanzhang
Quoted 15 lines · Evanzhang

block_stream will not actively flush l2_table_cache,when qemu process exception exit,causing disk data loss

---
  block/stream.c | 6 ++++++
  1 file changed, 6 insertions(+)
block/stream.c +0 -0
index e522bbd..a5e08da 100644
@@ -207,6 +207,12 @@ static int coroutine_fn stream_run(Job *job, Error **errp)
          }
      }

block/stream.c static int coroutine_fn stream_run(Job *job, Error **errp) · line 218

Quoted 10 lines
block/stream.c +6 -0
 
+    /*
+     * Complete stream_populate,force flush l2_table_cache,to
+     * avoid unexpected termination of process, l2_table loss
+     */
+    qcow2_cache_flush(bs, ((BDRVQcow2State *)bs->opaque)->l2_table_cache);
+
      /* Do not remove the backing file if an error was there but ignored. */
      return error;
  }

Hi!

I think, it's more correct just call bdrv_co_flush(bs), which should do all the job. Also, stream_run() should fail if flush fails.

Also, I remember I've done it for all (or at least several) blockjobs generically, so that any blockjob must succesfully flush target to report success.. But now I can find neither my patches nor the code :( Den, Kevin, Hanna, don't you remember this topic?

Signature · 2 lines
Best regards,
Vladimir
#3 lore raw

Re: [PATCH v1] block/stream:add flush l2_table_cache,ensure data integrity

Quoted 30 lines · Vladimir Sementsov-Ogievskiy
Quoted 18 lines · Evanzhang

block_stream will not actively flush l2_table_cache,when qemu process exception exit,causing disk data loss

---
  block/stream.c | 6 ++++++

  1 file changed, 6 insertions(+)
block/stream.c +0 -0
index e522bbd..a5e08da 100644
@@ -207,6 +207,12 @@ static int coroutine_fn stream_run(Job *job, 
 Error **errp)
           }
       }

  +    /*
+     * Complete stream_populate,force flush l2_table_cache,to

Quoted 8 lines

+     * avoid unexpected termination of process, l2_table loss
+     */
+    qcow2_cache_flush(bs, ((BDRVQcow2State *)bs->opaque)->l2_table_cache);
+
      /* Do not remove the backing file if an error was there but ignored. */ return error;
  }

Hi!

Quoted 4 lines

I think, it's more correct just call bdrv_co_flush(bs), which should do all the job. Also, stream_run() should fail if flush fails.

Also, I remember I've done it for all (or at least several) blockjobs generically, so that any blockjob must succesfully flush target to report success.. But now I can find neither my patches nor the code :( Den, Kevin, Hanna, don't you remember this topic?

This was a part of compressed write cache series, which was postponed.

https://lore.kernel.org/all/[email protected]/T/#m87315593ed5ab16e5d0e4e7a5ae6d776fbbaec77

We have it ported to 7.0 QEMU.

Not a problem to port to master and resend.
Will this make a sense?

Den

#4 lore raw

Re: [PATCH v1] block/stream:add flush l2_table_cache,ensure data integrity

Quoted 33 lines · Denis V. Lunev
Quoted 28 lines · Vladimir Sementsov-Ogievskiy
Quoted 16 lines · Evanzhang

block_stream will not actively flush l2_table_cache,when qemu process exception exit,causing disk data loss

---
  block/stream.c | 6 ++++++

  1 file changed, 6 insertions(+)
block/stream.c +0 -0
index e522bbd..a5e08da 100644
@@ -207,6 +207,12 @@ static int coroutine_fn stream_run(Job *job, Error **errp)
           }
       }

  +    /*

Quoted 8 lines

+     * Complete stream_populate,force flush l2_table_cache,to
+     * avoid unexpected termination of process, l2_table loss
+     */
+    qcow2_cache_flush(bs, ((BDRVQcow2State *)bs->opaque)->l2_table_cache); +
      /* Do not remove the backing file if an error was there but ignored. */ return error;
  }

Hi!

Quoted 4 lines

I think, it's more correct just call bdrv_co_flush(bs), which should do all the job. Also, stream_run() should fail if flush fails.

Also, I remember I've done it for all (or at least several) blockjobs generically, so that any blockjob must succesfully flush target to report success.. But now I can find neither my patches nor the code :( Den, Kevin, Hanna, don't you remember this topic?

Quoted 9 lines

This was a part of compressed write cache series, which was postponed.

https://lore.kernel.org/all/[email protected]/T/#m87315593ed5ab16e5d0e4e7a5ae6d776fbbaec77

We have it ported to 7.0 QEMU.

Not a problem to port to master and resend.
Will this make a sense?

O, thanks! Patch 01 applies with a little conflict to master, so I'll just resend it myself.

Signature · 2 lines
Best regards,
Vladimir
author
#5 lore raw

回复: [PATCH v1] block/stream:add flush l2_table_cache,ensure data integrity

Quoted 43 lines · Denis V. Lunev
Quoted 42 lines · Vladimir Sementsov-Ogievskiy
Quoted 32 lines · Evanzhang
Quoted 24 lines · Vladimir Sementsov-Ogievskiy

block_stream will not actively flush l2_table_cache,when qemu process exception exit,causing disk data loss

---
  block/stream.c | 6 ++++++

  1 file changed, 6 insertions(+)
block/stream.c index e522bbd..a5e08da renamed from block/stream.c +0 -0

100644

block/stream.c +0 -0
@@ -207,6 +207,12 @@ static int coroutine_fn stream_run(Job *job, 
 Error **errp)
           }
       }

  +    /*
+     * Complete stream_populate,force flush l2_table_cache,to +     * avoid unexpected termination of process, l2_table loss +     */
+    qcow2_cache_flush(bs, ((BDRVQcow2State

block/stream.c +2 -0
+*)bs->opaque)->l2_table_cache);
+
Quoted 4 lines

      /* Do not remove the backing file if an error was there but ignored. */
      return error;
  }

Hi!

Quoted 4 lines

I think, it's more correct just call bdrv_co_flush(bs), which should do all the job. Also, stream_run() should fail if flush fails.

Also, I remember I've done it for all (or at least several) blockjobs generically, so that any blockjob must succesfully flush target to report success.. But now I can find neither my patches nor the code :( Den, Kevin, Hanna, don't you remember this topic?

This was a part of compressed write cache series, which was postponed.

https://lore.kernel.org/all/20210305173507.393137-1-vsementsov@virtuoz
zo.com/T/#m87315593ed5ab16e5d0e4e7a5ae6d776fbbaec77

Quoted 8 lines
Quoted 5 lines

We have it ported to 7.0 QEMU.

Not a problem to port to master and resend.
Will this make a sense?

O, thanks! Patch 01 applies with a little conflict to master, so I'll just resend it myself.

Thanks all !
With best regards !