block: Avoid unecessary drv->bdrv_getlength() calls
PATCH 1/1 block: Avoid unecessary drv->bdrv_getlength() calls
The block layer generally keeps the size of an image cached in bs->total_sectors so that it doesn't have to perform expensive operations to get the size whenever it needs it.
This doesn't work however when using a backend that can change its size without qemu being aware of it, i.e. passthrough of removable media like CD-ROMs or floppy disks. For this reason, the caching is disabled when a removable device is used.
It is obvious that checking whether the _guest_ device has removable media isn't the right thing to do when we want to know whether the size of the host backend can change. To make things worse, non-top-level BlockDriverStates never have any device attached, which makes qemu assume they are removable, so drv->bdrv_getlength() is always called on the protocol layer. In the case of raw-posix, this causes unnecessary lseek() system calls, which turned out to be rather expensive.
This patch completely changes the logic and disables bs->total_sectors caching only for certain block driver types, for which a size change is
expected: host_cdrom and host_floppy; also the raw format in case it sits on top of one of these protocols, but in the common case the nested bdrv_getlength() call on the protocol driver will use the cache again and avoid an expensive drv->bdrv_getlength() call.
--- block.c | 7 ++++--- block/raw-posix.c | 9 ++++++--- block/raw_bsd.c | 1 + include/block/block_int.h | 3 +++ 4 files changed, 14 insertions(+), 6 deletions(-)
block.c +4 -3
@@ -2868,9 +2868,10 @@ int64_t bdrv_getlength(BlockDriverState *bs)if (!drv)return -ENOMEDIUM ; -if (bdrv_dev_has_removable_media (bs)) { -if (drv->bdrv_getlength) { -return drv->bdrv_getlength (bs); +if (drv->has_variable_length) { +int ret =refresh_total_sectors (bs, bs->total_sectors); +if (ret <0 ) { +return ret; } }return bs->total_sectors *BDRV_SECTOR_SIZE ;
block/raw-posix.c +6 -3
@@ -1715,7 +1715,8 @@ static BlockDriver bdrv_host_floppy = { .bdrv_aio_flush = raw_aio_flush, .bdrv_truncate = raw_truncate, - .bdrv_getlength = raw_getlength, + .bdrv_getlength = raw_getlength, + .has_variable_length = true, .bdrv_get_allocated_file_size = raw_get_allocated_file_size,
@@ -1824,7 +1825,8 @@ static BlockDriver bdrv_host_cdrom = { .bdrv_aio_flush = raw_aio_flush, .bdrv_truncate = raw_truncate, - .bdrv_getlength = raw_getlength, + .bdrv_getlength = raw_getlength, + .has_variable_length = true, .bdrv_get_allocated_file_size = raw_get_allocated_file_size,
@@ -1951,7 +1953,8 @@ static BlockDriver bdrv_host_cdrom = { .bdrv_aio_flush = raw_aio_flush, .bdrv_truncate = raw_truncate, - .bdrv_getlength = raw_getlength, + .bdrv_getlength = raw_getlength, + .has_variable_length = true, .bdrv_get_allocated_file_size = raw_get_allocated_file_size,
block/raw_bsd.c +1 -0
@@ -178,6 +178,7 @@ static BlockDriver bdrv_raw = { .bdrv_co_get_block_status = &raw_co_get_block_status, .bdrv_truncate = &raw_truncate, .bdrv_getlength = &raw_getlength, + .has_variable_length = true, .bdrv_get_info = &raw_get_info, .bdrv_is_inserted = &raw_is_inserted, .bdrv_media_changed = &raw_media_changed,
include/block/block_int.h +3 -0
@@ -156,8 +156,11 @@ struct BlockDriver {const char *protocol_name;int (*bdrv_truncate)(BlockDriverState *bs,int64_t offset); +int64_t (*bdrv_getlength)(BlockDriverState *bs); +bool has_variable_length;int64_t (*bdrv_get_allocated_file_size)(BlockDriverState *bs); +int (*bdrv_write_compressed)(BlockDriverState *bs,int64_t sector_num,const uint8_t *buf,int nb_sectors);
Signature · 1 line
1.8.1.4
Il 29/10/2013 12:35, Kevin Wolf ha scritto:
Quoted 103 lines
The block layer generally keeps the size of an image cached in
bs->total_sectorsso that it doesn't have to perform expensive operations to get the size whenever it needs it.This doesn't work however when using a backend that can change its size without qemu being aware of it, i.e. passthrough of removable media like CD-ROMs or floppy disks. For this reason, the caching is disabled when a removable device is used.
It is obvious that checking whether the _guest_ device has removable media isn't the right thing to do when we want to know whether the size of the host backend can change. To make things worse, non-top-level BlockDriverStates never have any device attached, which makes qemu assume they are removable, so
drv->bdrv_getlength()is always called on the protocol layer. In the case of raw-posix, this causes unnecessarylseek()system calls, which turned out to be rather expensive.This patch completely changes the logic and disables
bs->total_sectorscaching only for certain block driver types, for which a size change is
expected:host_cdromandhost_floppy; also the raw format in case it sits on top of one of these protocols, but in the common case the nestedbdrv_getlength()call on the protocol driver will use the cache again and avoid an expensivedrv->bdrv_getlength()call.Signed-off-by Kevin Wolf <[email protected]>--- block.c | 7 ++++--- block/raw-posix.c | 9 ++++++--- block/raw_bsd.c | 1 + include/block/block_int.h | 3 +++ 4 files changed, 14 insertions(+), 6 deletions(-)block.c +4
-3@@ -2868,9 +2868,10 @@ int64_t bdrv_getlength(BlockDriverState *bs)if (!drv)return -ENOMEDIUM ; -if (bdrv_dev_has_removable_media (bs)) { -if (drv->bdrv_getlength) { -return drv->bdrv_getlength (bs); +if (drv->has_variable_length) { +int ret =refresh_total_sectors (bs, bs->total_sectors); +if (ret <0 ) { +return ret; } }return bs->total_sectors *BDRV_SECTOR_SIZE ;block/raw-posix.c +6
-3@@ -1715,7 +1715,8 @@ static BlockDriver bdrv_host_floppy = { .bdrv_aio_flush = raw_aio_flush, .bdrv_truncate = raw_truncate, - .bdrv_getlength = raw_getlength, + .bdrv_getlength = raw_getlength, + .has_variable_length = true, .bdrv_get_allocated_file_size = raw_get_allocated_file_size,@@ -1824,7 +1825,8 @@ static BlockDriver bdrv_host_cdrom = { .bdrv_aio_flush = raw_aio_flush, .bdrv_truncate = raw_truncate, - .bdrv_getlength = raw_getlength, + .bdrv_getlength = raw_getlength, + .has_variable_length = true, .bdrv_get_allocated_file_size = raw_get_allocated_file_size,@@ -1951,7 +1953,8 @@ static BlockDriver bdrv_host_cdrom = { .bdrv_aio_flush = raw_aio_flush, .bdrv_truncate = raw_truncate, - .bdrv_getlength = raw_getlength, + .bdrv_getlength = raw_getlength, + .has_variable_length = true, .bdrv_get_allocated_file_size = raw_get_allocated_file_size,block/raw_bsd.c +1
-0@@ -178,6 +178,7 @@ static BlockDriver bdrv_raw = { .bdrv_co_get_block_status = &raw_co_get_block_status, .bdrv_truncate = &raw_truncate, .bdrv_getlength = &raw_getlength, + .has_variable_length = true, .bdrv_get_info = &raw_get_info, .bdrv_is_inserted = &raw_is_inserted, .bdrv_media_changed = &raw_media_changed,include/block/block_int.h +0
-0@@ -156,8 +156,11 @@ struct BlockDriver {
include/block/block_int.h struct BlockDriver { · line 166
Quoted 10 lines
include/block/block_int.h +3
-0const char *protocol_name;int (*bdrv_truncate)(BlockDriverState *bs,int64_t offset); +int64_t (*bdrv_getlength)(BlockDriverState *bs); +bool has_variable_length;int64_t (*bdrv_get_allocated_file_size)(BlockDriverState *bs); +int (*bdrv_write_compressed)(BlockDriverState *bs,int64_t sector_num,const uint8_t *buf,int nb_sectors);
raw-win32.c probably needs to have a .has_variable_length=true in bdrv_host_device. Apart from that,
Am 29.10.2013 um 13:02 hat Paolo Bonzini geschrieben:
Quoted 25 lines
Il 29/10/2013 12:35, Kevin Wolf ha scritto:
Quoted 24 lines
The block layer generally keeps the size of an image cached in
bs->total_sectorsso that it doesn't have to perform expensive operations to get the size whenever it needs it.This doesn't work however when using a backend that can change its size without qemu being aware of it, i.e. passthrough of removable media like CD-ROMs or floppy disks. For this reason, the caching is disabled when a removable device is used.
It is obvious that checking whether the _guest_ device has removable media isn't the right thing to do when we want to know whether the size of the host backend can change. To make things worse, non-top-level BlockDriverStates never have any device attached, which makes qemu assume they are removable, so
drv->bdrv_getlength()is always called on the protocol layer. In the case of raw-posix, this causes unnecessarylseek()system calls, which turned out to be rather expensive.This patch completely changes the logic and disables
bs->total_sectorscaching only for certain block driver types, for which a size change is
expected:host_cdromandhost_floppy; also the raw format in case it sits on top of one of these protocols, but in the common case the nestedbdrv_getlength()call on the protocol driver will use the cache again and avoid an expensivedrv->bdrv_getlength()call.
Quoted 6 lines
Quoted 1 line
Signed-off-by Kevin Wolf <[email protected]>
raw-win32.cprobably needs to have a .has_variable_length=true inbdrv_host_device. Apart from that,Reviewed-by Paolo Bonzini <[email protected]>
Thanks, good catch. I've added this now.
Kevin
Quoted 26 lines · Kevin Wolf
Am 29.10.2013 um 13:02 hat Paolo Bonzini geschrieben:
Quoted 25 lines
Il 29/10/2013 12:35, Kevin Wolf ha scritto:
Quoted 24 lines
The block layer generally keeps the size of an image cached in
bs->total_sectorsso that it doesn't have to perform expensive operations to get the size whenever it needs it.This doesn't work however when using a backend that can change its size without qemu being aware of it, i.e. passthrough of removable media like CD-ROMs or floppy disks. For this reason, the caching is disabled when a removable device is used.
It is obvious that checking whether the _guest_ device has removable media isn't the right thing to do when we want to know whether the size of the host backend can change. To make things worse, non-top-level BlockDriverStates never have any device attached, which makes qemu assume they are removable, so
drv->bdrv_getlength()is always called on the protocol layer. In the case of raw-posix, this causes unnecessarylseek()system calls, which turned out to be rather expensive.This patch completely changes the logic and disables
bs->total_sectorscaching only for certain block driver types, for which a size change is
expected:host_cdromandhost_floppy; also the raw format in case it sits on top of one of these protocols, but in the common case the nestedbdrv_getlength()call on the protocol driver will use the cache again and avoid an expensivedrv->bdrv_getlength()call.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Signed-off-by Kevin Wolf <[email protected]>
raw-win32.cprobably needs to have a .has_variable_length=true inbdrv_host_device. Apart from that,Reviewed-by Paolo Bonzini <[email protected]>Thanks, good catch. I've added this now.
This breaks VMDK because it can't read description file: buffer is empty in bdrv_probe, when the file size is <512 bytes. (for raw-posix).
Fam
Am 04.11.2013 um 08:24 hat Fam Zheng geschrieben:
Quoted 34 lines · Kevin Wolf
Quoted 31 lines
Am 29.10.2013 um 13:02 hat Paolo Bonzini geschrieben:
Quoted 30 lines
Il 29/10/2013 12:35, Kevin Wolf ha scritto:
Quoted 17 lines
The block layer generally keeps the size of an image cached in
bs->total_sectorsso that it doesn't have to perform expensive operations to get the size whenever it needs it.This doesn't work however when using a backend that can change its size without qemu being aware of it, i.e. passthrough of removable media like CD-ROMs or floppy disks. For this reason, the caching is disabled when a removable device is used.
It is obvious that checking whether the _guest_ device has removable media isn't the right thing to do when we want to know whether the size of the host backend can change. To make things worse, non-top-level BlockDriverStates never have any device attached, which makes qemu assume they are removable, so
drv->bdrv_getlength()is always called on the protocol layer. In the case of raw-posix, this causes unnecessarylseek()system calls, which turned out to be rather expensive.Quoted 8 lines
This patch completely changes the logic and disables
bs->total_sectorscaching only for certain block driver types, for which a size change is
expected:host_cdromandhost_floppy; also the raw format in case it sits on top of one of these protocols, but in the common case the nestedbdrv_getlength()call on the protocol driver will use the cache again and avoid an expensivedrv->bdrv_getlength()call.Signed-off-by Kevin Wolf <[email protected]>
raw-win32.cprobably needs to have a .has_variable_length=true inbdrv_host_device. Apart from that,
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
Reviewed-by Paolo Bonzini <[email protected]>Thanks, good catch. I've added this now.
This breaks VMDK because it can't read description file: buffer is empty in
bdrv_probe, when the file size is <512 bytes. (for raw-posix).
Perhaps we should replace bs->total_sectors with bs->total_byts one day... For now, how about introducing a bdrv_getlength_bytes() that always calls the driver and never converts bytes to sectors?
Kevin
Quoted 33 lines · Kevin Wolf
Am 04.11.2013 um 08:24 hat Fam Zheng geschrieben:
Quoted 32 lines · Kevin Wolf
Quoted 30 lines
Am 29.10.2013 um 13:02 hat Paolo Bonzini geschrieben:
Quoted 29 lines
Il 29/10/2013 12:35, Kevin Wolf ha scritto:
Quoted 17 lines
The block layer generally keeps the size of an image cached in
bs->total_sectorsso that it doesn't have to perform expensive operations to get the size whenever it needs it.This doesn't work however when using a backend that can change its size without qemu being aware of it, i.e. passthrough of removable media like CD-ROMs or floppy disks. For this reason, the caching is disabled when a removable device is used.
It is obvious that checking whether the _guest_ device has removable media isn't the right thing to do when we want to know whether the size of the host backend can change. To make things worse, non-top-level BlockDriverStates never have any device attached, which makes qemu assume they are removable, so
drv->bdrv_getlength()is always called on the protocol layer. In the case of raw-posix, this causes unnecessarylseek()system calls, which turned out to be rather expensive.Quoted 8 lines
This patch completely changes the logic and disables
bs->total_sectorscaching only for certain block driver types, for which a size change is
expected:host_cdromandhost_floppy; also the raw format in case it sits on top of one of these protocols, but in the common case the nestedbdrv_getlength()call on the protocol driver will use the cache again and avoid an expensivedrv->bdrv_getlength()call.Signed-off-by Kevin Wolf <[email protected]>
raw-win32.cprobably needs to have a .has_variable_length=true inbdrv_host_device. Apart from that,
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Quoted 1 line
Reviewed-by Paolo Bonzini <[email protected]>Thanks, good catch. I've added this now.
This breaks VMDK because it can't read description file: buffer is empty in
bdrv_probe, when the file size is <512 bytes. (for raw-posix).Perhaps we should replace
bs->total_sectorswithbs->total_bytsone day... For now, how about introducing abdrv_getlength_bytes()that always calls the driver and never converts bytes to sectors?
I think it will work. And is it correct to round up total_bytes to BDRV_SECTOR_SIZE before storing to total_sectors? From my view it's better than losing some bytes by rounding down.
Fam