Implement new cache mode "target"
PATCH 1/1 Implement new cache mode "target"
There is an issue with databases in VM that perform too slow on generic SAN storages. The key point is fdatasync that flushes disk on SCSI target.
The QEMU blockdev "target" cache mode intended to be used with SAN storages and is a mix of "none" by using direct I/O and "unsafe" that omit device flush.
Such storages has its own data integrity protection and can be operated with direct I/O without additional fdatasyc().
With generic SCSI targets like LIO or SCST it boost performance up to 100% on some profiles like database with transaction journal (postrgesql/mssql/oracle etc) or virtualized SDS (ceph/rook inside VMs) which performs block device cache flush on journal record.
--- block.c | 4 ++++ qemu-options.hx | 3 ++- tests/qemu-iotests/026 | 2 +- tests/qemu-iotests/091 | 2 +- 4 files changed, 8 insertions(+), 3 deletions(-)
block.c +4 -0
@@ -884,6 +884,10 @@ int bdrv_parse_cache_mode(const char *mode, int *flags, bool *writethrough) }else if (!strcmp (mode,"unsafe" )) { *writethrough = false; *flags |=BDRV_O_NO_FLUSH ; + }else if (!strcmp (mode,"target" )) { + *writethrough = false; + *flags |=BDRV_O_NOCACHE ; + *flags |=BDRV_O_NO_FLUSH ; }else if (!strcmp (mode,"writethrough" )) { *writethrough = true; }else {
qemu-options.hx +2 -1
@@ -1065,7 +1065,7 @@ This option defines the type of the media: disk or cdrom. @var{snapshot} is "on" or "off" and controls snapshot mode for the given drive (see @option{-snapshot}). @item cache=@var{cache} -@var{cache} is "none", "writeback", "unsafe", "directsync" or "writethrough" +@var{cache} is "none", "writeback", "unsafe", "target", "directsync" or "writethrough" and controls how the host cache is used to access block data. This is a shortcut that sets the @option{cache.direct} and @option{cache.no-flush} options (as in @option{-blockdev}), and additionally @option{cache.writeback},
@@ -1084,6 +1084,7 @@ none │ on on off writethrough │ off off off directsync │ off on off unsafe │ on off on +target │ on on on @end example The default mode is @option{cache=writeback}.
tests/qemu-iotests/026 +1 -1
@@ -42,7 +42,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto file _default_cache_mode "writethrough" -_supported_cache_modes "writethrough" "none" +_supported_cache_modes "writethrough" "none" "target" # The refcount table tests expect a certain minimum width for refcount entries # (so that the refcount table actually needs to grow); that minimum is 16 bits, # being the default refcount entry width.
tests/qemu-iotests/091 +1 -1
@@ -47,7 +47,7 @@ _supported_fmt qcow2 _supported_proto file _supported_os Linux _default_cache_mode "none" -_supported_cache_modes "writethrough" "none" "writeback" +_supported_cache_modes "writethrough" "none" "writeback" "target" size=1G
Signature · 1 line
2.21.0
On Wed, Aug 07, 2019 at 04:09:54PM +0300, Artemy Kapitula wrote:
Hi,
Please use "scripts/get_maintainer.pl -f block.c" to find out which maintainers to email. [email protected] is a high-traffic list and patches not CCed to the right maintainer may not get quick review.
Quoted 8 lines
There is an issue with databases in VM that perform too slow on generic SAN storages. The key point is fdatasync that flushes disk on SCSI target.
The QEMU blockdev "target" cache mode intended to be used with SAN storages and is a mix of "none" by using direct I/O and "unsafe" that omit device flush.
Quoted 7 lines
Such storages has its own data integrity protection and can be operated with direct I/O without additional
fdatasyc().With generic SCSI targets like LIO or SCST it boost performance up to 100% on some profiles like database with transaction journal (postrgesql/mssql/oracle etc) or virtualized SDS (ceph/rook inside VMs) which performs block device cache flush on journal record.
If the physical storage controller has a Battery Backed Unit (BBU) or similar then flush requests are not required with O_DIRECT. This has been a common enterprise storage configuration for many years and is already supported in QEMU today:
Configure the guest with cache=none and disable the emulated storage controller's write cache (e.g. -device virtio-blk-pci,write-cache=off). Inside the guest /sys/block/$BLKDEV/queue/write_cache should show "write through".
I think this patch is not necessary since write-cache=off already exists. cache=target is also slower since the guest sends unnecessary flush requests to the emulated storage controller.
Thanks,
Stefan
Quoted 72 lines
Signed-off-by Artemy Kapitula <[email protected]>--- block.c | 4 ++++ qemu-options.hx | 3 ++- tests/qemu-iotests/026 | 2 +- tests/qemu-iotests/091 | 2 +- 4 files changed, 8 insertions(+), 3 deletions(-)block.c +4
-0@@ -884,6 +884,10 @@ int bdrv_parse_cache_mode(const char *mode, int *flags, bool *writethrough) }else if (!strcmp (mode,"unsafe" )) { *writethrough = false; *flags |=BDRV_O_NO_FLUSH ; + }else if (!strcmp (mode,"target" )) { + *writethrough = false; + *flags |=BDRV_O_NOCACHE ; + *flags |=BDRV_O_NO_FLUSH ; }else if (!strcmp (mode,"writethrough" )) { *writethrough = true; }else {qemu-options.hx +2
-1@@ -1065,7 +1065,7 @@ This option defines the type of the media: disk or cdrom. @var{snapshot} is "on" or "off" and controls snapshot mode for the given drive (see @option{-snapshot}). @item cache=@var{cache} -@var{cache} is "none", "writeback", "unsafe", "directsync" or "writethrough" +@var{cache} is "none", "writeback", "unsafe", "target", "directsync" or "writethrough" and controls how the host cache is used to access block data. This is a shortcut that sets the @option{cache.direct} and @option{cache.no-flush} options (as in @option{-blockdev}), and additionally @option{cache.writeback},@@ -1084,6 +1084,7 @@ none │ on on off writethrough │ off off off directsync │ off on off unsafe │ on off on +target │ on on on @end example The default mode is @option{cache=writeback}.tests/qemu-iotests/026 +1
-1@@ -42,7 +42,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto file _default_cache_mode "writethrough" -_supported_cache_modes "writethrough" "none" +_supported_cache_modes "writethrough" "none" "target" # The refcount table tests expect a certain minimum width for refcount entries # (so that the refcount table actually needs to grow); that minimum is 16 bits, # being the default refcount entry width.tests/qemu-iotests/091 +1
-1@@ -47,7 +47,7 @@ _supported_fmt qcow2 _supported_proto file _supported_os Linux _default_cache_mode "none" -_supported_cache_modes "writethrough" "none" "writeback" +_supported_cache_modes "writethrough" "none" "writeback" "target" size=1GSignature · 1 line
2.21.0
Am 15.08.2019 um 15:53 hat Stefan Hajnoczi geschrieben:
Quoted 28 lines · Artemy Kapitula
Hi,
Please use "scripts/get_maintainer.pl-fblock.c" to find out which maintainers to email. [email protected] is a high-traffic list and patches not CCed to the right maintainer may not get quick review.Quoted 8 lines
There is an issue with databases in VM that perform too slow on generic SAN storages. The key point is fdatasync that flushes disk on SCSI target.
The QEMU blockdev "target" cache mode intended to be used with SAN storages and is a mix of "none" by using direct I/O and "unsafe" that omit device flush.
Quoted 7 lines
Such storages has its own data integrity protection and can be operated with direct I/O without additional
fdatasyc().With generic SCSI targets like LIO or SCST it boost performance up to 100% on some profiles like database with transaction journal (postrgesql/mssql/oracle etc) or virtualized SDS (ceph/rook inside VMs) which performs block device cache flush on journal record.
If the physical storage controller has a Battery Backed Unit (BBU) or similar then flush requests are not required with
O_DIRECT. This has been a common enterprise storage configuration for many years and is already supported in QEMU today:
Quoted 8 lines
Configure the guest with cache=none and disable the emulated storage controller's write cache (e.g. -device virtio-blk-pci,write-cache=off). Inside the guest
/sys/block/$BLKDEV/queue/write_cacheshould show "write through".I think this patch is not necessary since write-cache=off already exists. cache=target is also slower since the guest sends unnecessary flush requests to the emulated storage controller.
Two more comments:
1. The proposed cache mode can already be configured as cache.direct=on,cache.no-flush=on. I don't think we intend to add new aliases for combinations of these options. The existing aliases exist for compatibility reasons.
2. If fdatasync() takes noticable time on such storage, this is a host kernel problem. If we know that there is nothing to be synced, the kernel should just return immediately without involving any I/O.
Kevin