Block patches
Details on the patches:
1. Fam Zheng's VMDK zeroed-grain GTEs implements zero cluster support in VMDK. We need this to correctly read files containing zero clusters - it's essentially a bugfix.
2. Jeff Cody's VHDX series implements read-only support for the new Hyper-V image format. The series has been on the list for a while and stripped down to make it mergable for QEMU 1.5. Not all image files are supported yet but this already allows for new v2v migrations.
3. Kevin Wolf's qmp_block_resize error clarification.
4. My NBD fix for new Linux nbd drivers that can send >1 MB requests.
The following changes since commit 8ca27ce2e1150486ea2db4116a03706b28294f16:
Merge remote-tracking branch 'afaerber/qom-cpu' into staging (2013-05-02 10:57:01 -0500)
are available in the git repository at:
git://github.com/stefanha/qemu.git block
for you to fetch changes up to 86abefd61e23325162e59e5bfb8f0346eda62541:
qemu-iotests: Filter out 'adapter_type' (2013-05-03 13:06:22 +0200)
----------------------------------------------------------------
Fam Zheng (8):
vmdk: named return code.
vmdk: add support for “zeroed‐grain” GTE
vmdk: Add option to create zeroed-grain image
vmdk: change magic number to macro
vmdk: store fields of VmdkMetaData in cpu endian
vmdk: add bdrv_co_write_zeroes
qemu-iotests: Filter out vmdk creation options
qemu-iotests: Filter out 'adapter_type'
Jeff Cody (4):
qemu: add castagnoli crc32c checksum algorithm
block: vhdx header for the QEMU support of VHDX images
block: initial VHDX driver support framework - supports open and probe
block: add read-only support to VHDX image format.
Kevin Wolf (1):
blockdev: Replace "undefined error" in qmp_block_resize
Stefan Hajnoczi (2):
nbd: use g_slice_new() instead of a freelist
nbd: support large NBD requests
block/Makefile.objs | 1 + block/vhdx.c | 972 +++++++++++++++++++++++++++++++++++++++++++ block/vhdx.h | 325 +++++++++++++++ block/vmdk.c | 208 ++++++--- blockdev.c | 6 +- include/block/nbd.h | 3 +- include/qemu/crc32c.h | 35 ++ nbd.c | 36 +- tests/qemu-iotests/common.rc | 3 + util/Makefile.objs | 1 + util/crc32c.c | 115 +++++ 11 files changed, 1618 insertions(+), 87 deletions(-) create mode 100644 block/vhdx.c create mode 100644 block/vhdx.h create mode 100644 include/qemu/crc32c.h create mode 100644 util/crc32c.c
Signature · 1 line
1.8.1.4
PATCH 1/15 qemu: add castagnoli crc32c checksum algorithm
From: Jeff Cody <[email protected]>
This adds the Castagnoli CRC32C algorithm, using the 0x11EDC6F41 polynomial.
This is extracted from the linux kernel cryptographic crc32.c module.
The algorithm is based on:
Castagnoli93: Guy Castagnoli and Stefan Braeuer and Martin Herrman
"Optimization of Cyclic Redundancy-Check Codes with 24
and 32 Parity Bits", IEEE Transactions on Communication, Volume 41, Number 6, June 1993
--- include/qemu/crc32c.h | 35 +++++++++++++++ util/Makefile.objs | 1 + util/crc32c.c | 115 ++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 151 insertions(+) create mode 100644 include/qemu/crc32c.h create mode 100644 util/crc32c.c
include/qemu/crc32c.h new file +35 -0
@@ -0,0 +1,35 @@ +/* +* Castagnoli CRC32C Checksum Algorithm +* +* Polynomial: 0x11EDC6F41 +* +* Castagnoli93: Guy Castagnoli and Stefan Braeuer and Martin Herrman +* "Optimization of Cyclic Redundancy-Check Codes with 24 +* and 32 Parity Bits",IEEE Transactions on Communication, +* Volume 41, Number 6, June 1993 +* +* Copyright (c) 2013 Red Hat, Inc., +* +* Authors: +* Jeff Cody <[email protected]> +* +* Based on the Linux kernel cryptographic crc32c module, +* +* Copyright (c) 2004 Cisco Systems, Inc. +* Copyright (c) 2008 Herbert Xu <[email protected]> +* +* This program is free software; you can redistribute it and/or modify it +* under the terms of the GNU General Public License as published by the Free +* Software Foundation; either version 2 of the License, or (at your option) +* any later version. +* +*/ + +#ifndef QEMU_CRC32_H +#define QEMU_CRC32_H + +#include "qemu-common.h" + +uint32_t crc32c (uint32_t crc,const uint8_t *data,unsigned int length); + +#endif
util/Makefile.objs +1 -0
@@ -10,3 +10,4 @@ util-obj-$(CONFIG_POSIX) += compatfd.o util-obj-y += iov.o aes.o qemu-config.o qemu-sockets.o uri.o notify.o util-obj-y += qemu-option.o qemu-progress.o util-obj-y += hexdump.o +util-obj-y += crc32c.o
util/crc32c.c new file +115 -0
115 lines not on this page · raw message
Signature · 1 line
1.8.1.4
PATCH 2/15 block: vhdx header for the QEMU support of VHDX images
In reply to Stefan HajnocziFrom: Jeff Cody <[email protected]>
This is based on Microsoft's VHDX specification:
"VHDX Format Specification v0.95", published 4/12/2012
https://www.microsoft.com/en-us/download/details.aspx?id=29681These structures define the various header, metadata, and other block structures defined in the VHDX specification.
--- block/vhdx.h | 311 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 311 insertions(+) create mode 100644 block/vhdx.h
block/vhdx.h new file +311 -0
@@ -0,0 +1,311 @@ +/* +* Block driver for Hyper-V VHDX Images +* +* Copyright (c) 2013 Red Hat, Inc., +* +* Authors: +* Jeff Cody <[email protected]> +* +* This is based on the "VHDX Format Specification v0.95", published 4/12/2012 +* by Microsoft: +* https://www.microsoft.com/en-us/download/details.aspx?id=29681 +* +* This work is licensed under the terms of the GNU LGPL, version 2 or later. +* See the COPYING.LIB file in the top-level directory. +* +*/ + +#ifndef BLOCK_VHDX_H +#define BLOCK_VHDX_H + +/* Structures and fields present in the VHDX file */ + +/* The header section has the following blocks, +* each block is 64KB: +* +* _____________________________________________________________________________ +* | File Id. | Header 1 | Header 2 | Region Table | Reserved (768KB) | +* |----------|---------------|------------|--------------|--------------------| +* | | | | | | +* 0.........64KB...........128KB........192KB..........256KB................1MB +*/ + +#define VHDX_HEADER_BLOCK_SIZE (64*1024) + +#define VHDX_FILE_ID_OFFSET 0 +#define VHDX_HEADER1_OFFSET (VHDX_HEADER_BLOCK_SIZE*1) +#define VHDX_HEADER2_OFFSET (VHDX_HEADER_BLOCK_SIZE*2) +#define VHDX_REGION_TABLE_OFFSET (VHDX_HEADER_BLOCK_SIZE*3) + + +/* +* A note on the use of MS-GUID fields. For more details on the GUID, +* please see: https://en.wikipedia.org/wiki/Globally_unique_identifier. +* +* The VHDX specification only states that these are MS GUIDs, and which +* bytes are data1-data4. It makes no mention of what algorithm should be used +* to generate the GUID, nor what standard. However, looking at the specified +* known GUID fields, it appears the GUIDs are: +* Standard/DCE GUID type (noted by 10b in the MSB of byte 0 of .data4) +* Random algorithm (noted by 0x4XXX for .data3) +*/ + +/* ---- HEADER SECTION STRUCTURES ---- */ + +/* These structures are ones that are defined in the VHDX specification +* document */ + +typedef struct VHDXFileIdentifier { +uint64_t signature;/* "vhdxfile" in ASCII */ +uint16_t creator[256 ];/* optional; utf-16 string to identify +the vhdx file creator. Diagnotistic +only */ +}VHDXFileIdentifier ; + + +/* the guid is a 16 byte unique ID - the definition for this used by +* Microsoft is not just 16 bytes though - it is a structure that is defined, +* so we need to follow it here so that endianness does not trip us up */ + +typedef struct MSGUID { +uint32_t data1; +uint16_t data2; +uint16_t data3; +uint8_t data4[8 ]; +}MSGUID ; + +#define guid_eq (a, b) \ + (memcmp(&(a), &(b), sizeof(MSGUID)) == 0) + +#define VHDX_HEADER_SIZE (4*1024)/* although the vhdx_header struct in disk +is only 582 bytes, for purposes of crc +the header is the first 4KB of the 64KB +block */ + +/* The full header is 4KB, although the actual header data is much smaller. +* But for the checksum calculation, it is over the entire 4KB structure, +* not just the defined portion of it */ +typedef struct QEMU_PACKED VHDXHeader { +uint32_t signature;/* "head" in ASCII */ +uint32_t checksum;/* CRC-32C hash of the whole header */ +uint64_t sequence_number;/* Seq number of this header. Each +VHDX file has 2 of these headers, +and only the header with the highest +sequence number is valid */ +MSGUID file_write_guid;/* 128 bit unique identifier. Must be +updated to new, unique value before +the first modification is made to +file */ +MSGUID data_write_guid;/* 128 bit unique identifier. Must be +updated to new, unique value before +the first modification is made to +visible data. Visbile data is +defined as: +- system & user metadata +- raw block data +- disk size +- any change that will +cause the virtual disk +sector read to differ + +This does not need to change if +blocks are re-arranged */ +MSGUID log_guid;/* 128 bit unique identifier. If zero, +there is no valid log. If non-zero, +log entries with this guid are +valid. */ +uint16_t log_version;/* version of the log format. Mustn't be +zero, unless log_guid is also zero */ +uint16_t version;/* version of th evhdx file. Currently, +only supported version is "1" */ +uint32_t log_length;/* length of the log. Must be multiple +of 1MB */ +uint64_t log_offset;/* byte offset in the file of the log. +Must also be a multiple of 1MB */ +} VHDXHeader; + +/* Header for the region table block */ +typedef struct QEMU_PACKED VHDXRegionTableHeader { +uint32_t signature;/* "regi" in ASCII */ +uint32_t checksum;/* CRC-32C hash of the 64KB table */ +uint32_t entry_count;/* number of valid entries */ +uint32_t reserved; +} VHDXRegionTableHeader; + +/* Individual region table entry. There may be a maximum of 2047 of these +* +* There are two known region table properties. Both are required. +* BAT (block allocation table): 2DC27766F62342009D64115E9BFD4A08 +* Metadata: 8B7CA20647904B9AB8FE575F050F886E +*/ +#define VHDX_REGION_ENTRY_REQUIRED 0x01/* if set, parser must understand +this entry in order to open +file */ +typedef struct QEMU_PACKED VHDXRegionTableEntry { +MSGUID guid;/* 128-bit unique identifier */ +uint64_t file_offset;/* offset of the object in the file. +Must be multiple of 1MB */ +uint32_t length;/* length, in bytes, of the object */ +uint32_t data_bits; +} VHDXRegionTableEntry; + + +/* ---- LOG ENTRY STRUCTURES ---- */ +#define VHDX_LOG_HDR_SIZE 64 +typedef struct QEMU_PACKED VHDXLogEntryHeader { +uint32_t signature;/* "loge" in ASCII */ +uint32_t checksum;/* CRC-32C hash of the 64KB table */ +uint32_t entry_length;/* length in bytes, multiple of 1MB */ +uint32_t tail;/* byte offset of first log entry of a +seq, where this entry is the last +entry */ +uint64_t sequence_number;/* incremented with each log entry. +May not be zero. */ +uint32_t descriptor_count;/* number of descriptors in this log +entry, must be >= 0 */ +uint32_t reserved; +MSGUID log_guid;/* value of the log_guid from +vhdx_header. If not found in +vhdx_header, it is invalid */ +uint64_t flushed_file_offset;/* see spec for full details - this +sould be vhdx file size in bytes */ +uint64_t last_file_offset;/* size in bytes that all allocated +file structures fit into */ +} VHDXLogEntryHeader; + +#define VHDX_LOG_DESC_SIZE 32 + +typedef struct QEMU_PACKED VHDXLogDescriptor { +uint32_t signature;/* "zero" or "desc" in ASCII */ +union { +uint32_t reserved;/* zero desc */ +uint32_t trailing_bytes;/* data desc: bytes 4092-4096 of the +data sector */ + }; +union { +uint64_t zero_length;/* zero desc: length of the section to +zero */ +uint64_t leading_bytes;/* data desc: bytes 0-7 of the data +sector */ + }; +uint64_t file_offset;/* file offset to write zeros - multiple +of 4kB */ +uint64_t sequence_number;/* must match same field in +vhdx_log_entry_header */ +} VHDXLogDescriptor; + +typedef struct QEMU_PACKED VHDXLogDataSector { +uint32_t data_signature;/* "data" in ASCII */ +uint32_t sequence_high;/* 4 MSB of 8 byte sequence_number */ +uint8_t data[4084 ];/* raw data, bytes 8-4091 (inclusive). +see the data descriptor field for the +other mising bytes */ +uint32_t sequence_low;/* 4 LSB of 8 byte sequence_number */ +} VHDXLogDataSector; + + + +/* block states - different state values depending on whether it is a +* payload block, or a sector block. */ + +#define PAYLOAD_BLOCK_NOT_PRESENT 0 +#define PAYLOAD_BLOCK_UNDEFINED 1 +#define PAYLOAD_BLOCK_ZERO 2 +#define PAYLOAD_BLOCK_UNMAPPED 5 +#define PAYLOAD_BLOCK_FULL_PRESENT 6 +#define PAYLOAD_BLOCK_PARTIALLY_PRESENT 7 + +#define SB_BLOCK_NOT_PRESENT 0 +#define SB_BLOCK_PRESENT 6 + +/* per the spec */ +#define VHDX_MAX_SECTORS_PER_BLOCK (1<<23) + +/* upper 44 bits are the file offset in 1MB units lower 3 bits are the state +other bits are reserved */ +#define VHDX_BAT_STATE_BIT_MASK 0x07 +#define VHDX_BAT_FILE_OFF_BITS (64-44) +typedef uint64_t VHDXBatEntry ; + +/* ---- METADATA REGION STRUCTURES ---- */ + +#define VHDX_METADATA_ENTRY_SIZE 32 +#define VHDX_METADATA_MAX_ENTRIES 2047/* not including the header */ +#define VHDX_METADATA_TABLE_MAX_SIZE \ + (VHDX_METADATA_ENTRY_SIZE * (VHDX_METADATA_MAX_ENTRIES+1)) +typedef struct QEMU_PACKED VHDXMetadataTableHeader { +uint64_t signature;/* "metadata" in ASCII */ +uint16_t reserved; +uint16_t entry_count;/* number table entries. <= 2047 */ +uint32_t reserved2[5 ]; +} VHDXMetadataTableHeader; + +#define VHDX_META_FLAGS_IS_USER 0x01/* max 1024 entries */ +#define VHDX_META_FLAGS_IS_VIRTUAL_DISK 0x02/* virtual disk metadata if set, +otherwise file metdata */ +#define VHDX_META_FLAGS_IS_REQUIRED 0x04/* parse must understand this +entry to open the file */ +typedef struct QEMU_PACKED VHDXMetadataTableEntry { +MSGUID item_id;/* 128-bit identifier for metadata */ +uint32_t offset;/* byte offset of the metadata. At +least 64kB. Relative to start of +metadata region */ +/* note: if length = 0, so is offset */ +uint32_t length;/* length of metadata. <= 1MB. */ +uint32_t data_bits;/* least-significant 3 bits are flags, the +rest are reserved (see above) */ +uint32_t reserved2; +} VHDXMetadataTableEntry; + +#define VHDX_PARAMS_LEAVE_BLOCKS_ALLOCED 0x01/* Do not change any blocks to +be BLOCK_NOT_PRESENT. +If set indicates a fixed +size VHDX file */ +#define VHDX_PARAMS_HAS_PARENT 0x02/* has parent / backing file */ +typedef struct QEMU_PACKED VHDXFileParameters { +uint32_t block_size;/* size of each payload block, always +power of 2, <= 256MB and >= 1MB. */ +uint32_t data_bits;/* least-significant 2 bits are flags, the rest +are reserved (see above) */ +} VHDXFileParameters; + +typedef struct QEMU_PACKED VHDXVirtualDiskSize { +uint64_t virtual_disk_size;/* Size of the virtual disk, in bytes. +Must be multiple of the sector size, +max of 64TB */ +} VHDXVirtualDiskSize; + +typedef struct QEMU_PACKED VHDXPage83Data { +MSGUID page_83_data[16 ];/* unique id for scsi devices that +support page 0x83 */ +} VHDXPage83Data; + +typedef struct QEMU_PACKED VHDXVirtualDiskLogicalSectorSize { +uint32_t logical_sector_size;/* virtual disk sector size (in bytes). +Can only be 512 or 4096 bytes */ +} VHDXVirtualDiskLogicalSectorSize; + +typedef struct QEMU_PACKED VHDXVirtualDiskPhysicalSectorSize { +uint32_t physical_sector_size;/* physical sector size (in bytes). +Can only be 512 or 4096 bytes */ +} VHDXVirtualDiskPhysicalSectorSize; + +typedef struct QEMU_PACKED VHDXParentLocatorHeader { +MSGUID locator_type[16 ];/* type of the parent virtual disk. */ +uint16_t reserved; +uint16_t key_value_count;/* number of key/value pairs for this +locator */ +} VHDXParentLocatorHeader; + +/* key and value strings are UNICODE strings, UTF-16 LE encoding, no NULs */ +typedef struct QEMU_PACKED VHDXParentLocatorEntry { +uint32_t key_offset;/* offset in metadata for key, > 0 */ +uint32_t value_offset;/* offset in metadata for value, >0 */ +uint16_t key_length;/* length of entry key, > 0 */ +uint16_t value_length;/* length of entry value, > 0 */ +} VHDXParentLocatorEntry; + + +/* ----- END VHDX SPECIFICATION STRUCTURES ---- */ + +#endif
Signature · 1 line
1.8.1.4
PATCH 3/15 block: initial VHDX driver support framework - supports open and probe
In reply to Stefan HajnocziFrom: Jeff Cody <[email protected]>
This is the initial block driver framework for VHDX image support (i.e. Hyper-V image file formats), that supports opening VHDX files, and parsing the headers.
This commit does not yet enable:
- reading
- writing
- updating the header
- differencing files (images with parents)
- log replay / dirty logs (only clean images)
This is based on Microsoft's VHDX specification:
"VHDX Format Specification v0.95", published 4/12/2012
https://www.microsoft.com/en-us/download/details.aspx?id=29681--- block/Makefile.objs | 1 + block/vhdx.c | 853 ++++++++++++++++++++++++++++++++++++++++++++++++++++ block/vhdx.h | 14 + 3 files changed, 868 insertions(+) create mode 100644 block/vhdx.c
block/Makefile.objs +1 -0
@@ -2,6 +2,7 @@ block-obj-y += raw.o cow.o qcow.o vdi.o vmdk.o cloop.o dmg.o bochs.o vpc.o vvfat block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o block-obj-y += qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o block-obj-y += qed-check.o +block-obj-y += vhdx.o block-obj-y += parallels.o blkdebug.o blkverify.o block-obj-$(CONFIG_WIN32) += raw-win32.o win32-aio.o block-obj-$(CONFIG_POSIX) += raw-posix.o
block/vhdx.c new file +853 -0
853 lines not on this page · raw message
block/vhdx.h +14 -0
@@ -308,4 +308,18 @@ typedef struct QEMU_PACKED VHDXParentLocatorEntry {/* ----- END VHDX SPECIFICATION STRUCTURES ---- */ + +uint32_t vhdx_checksum_calc (uint32_t crc,uint8_t *buf,size_t size, +int crc_offset); + +bool vhdx_checksum_is_valid (uint8_t *buf,size_t size,int crc_offset); + + +static void leguid_to_cpus (MSGUID *guid) +{ +le32_to_cpus (&guid->data1); +le16_to_cpus (&guid->data2); +le16_to_cpus (&guid->data3); +} +#endif
Signature · 1 line
1.8.1.4
PATCH 4/15 block: add read-only support to VHDX image format.
In reply to Stefan HajnocziFrom: Jeff Cody <[email protected]>
This adds in read-only support to the VHDX image format. This supports reads for fixed-size, and dynamic sized VHDX images.
Differencing files are still unsupported.
The image must be opened without BDRV_O_RDWR set, because we do not yet update the headers. I.e., pass 'readonly=on' in the drive image options from the QEMU commandline.
--- block/vhdx.c | 123 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 121 insertions(+), 2 deletions(-)
block/vhdx.c +121 -2
@@ -114,6 +114,17 @@ typedef struct VHDXMetadataEntries { } VHDXMetadataEntries; +typedef struct VHDXSectorInfo { +uint32_t bat_idx;/* BAT entry index */ +uint32_t sectors_avail;/* sectors available in payload block */ +uint32_t bytes_left;/* bytes left in the block after data to r/w */ +uint32_t bytes_avail;/* bytes available in payload block */ +uint64_t file_offset;/* absolute offset in bytes, in file */ +uint64_t block_offset;/* block offset, in bytes */ +}VHDXSectorInfo ; + + + typedef struct BDRVVHDXState { CoMutex lock;
@@ -792,7 +803,7 @@ static int vhdx_open(BlockDriverState *bs, QDict *options, int flags) goto fail; } -/* TODO: differencing files, read, write */ +/* TODO: differencing files, write */ return 0 ; fail:
@@ -810,10 +821,118 @@ static int vhdx_reopen_prepare(BDRVReopenState *state, } +/* +* Perform sector to block offset translations, to get various +* sector and file offsets into the image. See VHDXSectorInfo +*/ +static void vhdx_block_translate (BDRVVHDXState *s,int64_t sector_num, +int nb_sectors,VHDXSectorInfo *sinfo) +{ +uint32_t block_offset; + + sinfo->bat_idx = sector_num >> s->sectors_per_block_bits; +/* effectively a modulo - this gives us the offset into the block +* (in sector sizes) for our sector number */ + block_offset = sector_num - (sinfo->bat_idx << s->sectors_per_block_bits); +/* the chunk ratio gives us the interleaving of the sector +* bitmaps, so we need to advance our page block index by the +* sector bitmaps entry number */ + sinfo->bat_idx += sinfo->bat_idx >> s->chunk_ratio_bits; + +/* the number of sectors we can read/write in this cycle */ + sinfo->sectors_avail = s->sectors_per_block - block_offset; + + sinfo->bytes_left = sinfo->sectors_avail << s->logical_sector_size_bits; + +if (sinfo->sectors_avail > nb_sectors) { + sinfo->sectors_avail = nb_sectors; + } + + sinfo->bytes_avail = sinfo->sectors_avail << s->logical_sector_size_bits; + + sinfo->file_offset = s->bat[sinfo->bat_idx] >>VHDX_BAT_FILE_OFF_BITS ; + + sinfo->block_offset = block_offset << s->logical_sector_size_bits; + +/* The file offset must be past the header section, so must be > 0 */ +if (sinfo->file_offset ==0 ) { +return ; + } + +/* block offset is the offset in vhdx logical sectors, in +* the payload data block. Convert that to a byte offset +* in the block, and add in the payload data block offset +* in the file, in bytes, to get the final read address */ + + sinfo->file_offset <<=20 ;/* now in bytes, rather than 1MB units */ + sinfo->file_offset += sinfo->block_offset; +} + + + static coroutine_fn int vhdx_co_readv(BlockDriverState *bs, int64_t sector_num,int nb_sectors,QEMUIOVector *qiov) { -return -ENOTSUP ; +BDRVVHDXState *s = bs->opaque; +int ret =0 ; +VHDXSectorInfo sinfo; +uint64_t bytes_done =0 ; +QEMUIOVector hd_qiov; + +qemu_iovec_init (&hd_qiov, qiov->niov); + +qemu_co_mutex_lock (&s->lock); + +while (nb_sectors >0 ) { +/* We are a differencing file, so we need to inspect the sector bitmap +* to see if we have the data or not */ +if (s->params.data_bits &VHDX_PARAMS_HAS_PARENT ) { +/* not supported yet */ + ret = -ENOTSUP ; + goto exit; + }else { +vhdx_block_translate (s, sector_num, nb_sectors, &sinfo); + +qemu_iovec_reset (&hd_qiov); +qemu_iovec_concat (&hd_qiov, qiov, bytes_done, sinfo.bytes_avail); + +/* check the payload block state */ +switch (s->bat[sinfo.bat_idx] &VHDX_BAT_STATE_BIT_MASK ) { +case PAYLOAD_BLOCK_NOT_PRESENT :/* fall through */ +case PAYLOAD_BLOCK_UNDEFINED :/* fall through */ +case PAYLOAD_BLOCK_UNMAPPED :/* fall through */ +case PAYLOAD_BLOCK_ZERO : +/* return zero */ +qemu_iovec_memset (&hd_qiov,0 ,0 , sinfo.bytes_avail); +break ; +case PAYLOAD_BLOCK_FULL_PRESENT : +qemu_co_mutex_unlock (&s->lock); + ret =bdrv_co_readv (bs->file, + sinfo.file_offset >>BDRV_SECTOR_BITS , + sinfo.sectors_avail, &hd_qiov); +qemu_co_mutex_lock (&s->lock); +if (ret <0 ) { + goto exit; + } +break ; +case PAYLOAD_BLOCK_PARTIALLY_PRESENT : +/* we don't yet support difference files, fall through +* to error */ +default : + ret = -EIO ; + goto exit; +break ; + } + nb_sectors -= sinfo.sectors_avail; + sector_num += sinfo.sectors_avail; + bytes_done += sinfo.bytes_avail; + } + } + ret =0 ; +exit: +qemu_co_mutex_unlock (&s->lock); +qemu_iovec_destroy (&hd_qiov); +return ret; }
Signature · 1 line
1.8.1.4
PATCH 5/15 blockdev: Replace "undefined error" in qmp_block_resize
In reply to Stefan HajnocziFrom: Kevin Wolf <[email protected]>
We have an errno value that can be displayed, so we should just do that. An easy way to reproduce this case is to resize a raw image to a size that is too large for the host file system.
--- blockdev.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
blockdev.c +4 -2
@@ -1118,6 +1118,7 @@ int do_drive_del(Monitor *mon, const QDict *qdict, QObject **ret_data)void qmp_block_resize (const char *device,int64_t size,Error **errp) {BlockDriverState *bs; +int ret; bs =bdrv_find (device);if (!bs) {
@@ -1133,7 +1134,8 @@ void qmp_block_resize(const char *device, int64_t size, Error **errp)/* complete all in-flight operations before resizing the device */ bdrv_drain_all (); -switch (bdrv_truncate (bs, size)) { + ret =bdrv_truncate (bs, size); +switch (ret) {case 0 :break ;case -ENOMEDIUM :
@@ -1149,7 +1151,7 @@ void qmp_block_resize(const char *device, int64_t size, Error **errp)error_set (errp,QERR_DEVICE_IN_USE , device);break ;default : -error_set (errp,QERR_UNDEFINED_ERROR ); +error_setg_errno (errp, -ret,"Could not resize" );break ; } }
Signature · 1 line
1.8.1.4
PATCH 6/15 vmdk: named return code.
In reply to Stefan HajnocziFrom: Fam Zheng <[email protected]>
Internal routines in vmdk.c previously return -1 on error and 0 on success. More return values are useful for future changes such as zeroed-grain GTE. Change all the magic return 0 and return -1 to macro names:
* VMDK_OK 0
* VMDK_ERROR (-1)
* VMDK_UNALLOC (-2)
* VMDK_ZEROED (-3)
--- block/vmdk.c | 60 ++++++++++++++++++++++++++++++++++-------------------------- 1 file changed, 34 insertions(+), 26 deletions(-)
block/vmdk.c +34 -26
@@ -37,6 +37,14 @@#define VMDK4_FLAG_MARKER (1 << 17)#define VMDK4_GD_AT_END 0xffffffffffffffffULL + +/* VMDK internal error codes */ +#define VMDK_OK 0 +#define VMDK_ERROR (-1) +/* Cluster not allocated */ +#define VMDK_UNALLOC (-2) +#define VMDK_ZEROED (-3) + typedef struct { uint32_t version; uint32_t flags;
@@ -578,22 +586,22 @@ static int vmdk_parse_description(const char *desc, const char *opt_name, opt_pos =strstr (desc, opt_name);if (!opt_pos) { -return -1 ; +return VMDK_ERROR ; }/* Skip "=\"" following opt_name */ opt_pos +=strlen (opt_name) +2 ;if (opt_pos >= end) { -return -1 ; +return VMDK_ERROR ; } opt_end = opt_pos;while (opt_end < end && *opt_end !='"' ) { opt_end++; }if (opt_end == end || buf_size < opt_end - opt_pos +1 ) { -return -1 ; +return VMDK_ERROR ; }pstrcpy (buf, opt_end - opt_pos +1 , opt_pos); -return 0 ; +return VMDK_OK ; }/* Open an extent file and append to bs array */
@@ -772,7 +780,7 @@ static int get_whole_cluster(BlockDriverState *bs,int ret;if (!vmdk_is_cid_valid (bs)) { -return -1 ; +return VMDK_ERROR ; }/* floor offset to cluster */
@@ -780,17 +788,17 @@ static int get_whole_cluster(BlockDriverState *bs, ret =bdrv_read (bs->backing_hd, offset >>9 , whole_grain, extent->cluster_sectors);if (ret <0 ) { -return -1 ; +return VMDK_ERROR ; }/* Write grain only into the active image */ ret =bdrv_write (extent->file, cluster_offset, whole_grain, extent->cluster_sectors);if (ret <0 ) { -return -1 ; +return VMDK_ERROR ; } } -return 0 ; +return VMDK_OK ; }static int vmdk_L2update (VmdkExtent *extent,VmdkMetaData *m_data)
@@ -803,7 +811,7 @@ static int vmdk_L2update(VmdkExtent *extent, VmdkMetaData *m_data) &(m_data->offset), sizeof(m_data->offset) ) < 0) { -return -1 ; +return VMDK_ERROR ; }/* update backup L2 table */ if (extent->l1_backup_table_offset !=0 ) {
@@ -814,11 +822,11 @@ static int vmdk_L2update(VmdkExtent *extent, VmdkMetaData *m_data) + (m_data->l2_index * sizeof(m_data->offset)), &(m_data->offset), sizeof(m_data->offset) ) < 0) { -return -1 ; +return VMDK_ERROR ; } } -return 0 ; +return VMDK_OK ; } static int get_cluster_offset(BlockDriverState *bs,
@@ -837,17 +845,17 @@ static int get_cluster_offset(BlockDriverState *bs, }if (extent->flat) { *cluster_offset = extent->flat_start_offset; -return 0 ; +return VMDK_OK ; } offset -= (extent->end_sector - extent->sectors) *SECTOR_SIZE ; l1_index = (offset >>9 ) / extent->l1_entry_sectors;if (l1_index >= extent->l1_size) { -return -1 ; +return VMDK_ERROR ; } l2_offset = extent->l1_table[l1_index];if (!l2_offset) { -return -1 ; +return VMDK_UNALLOC ; }for (i =0 ; i <L2_CACHE_SIZE ; i++) {if (l2_offset == extent->l2_cache_offsets[i]) {
@@ -877,7 +885,7 @@ static int get_cluster_offset(BlockDriverState *bs, l2_table, extent->l2_size * sizeof(uint32_t) ) != extent->l2_size * sizeof(uint32_t)) { -return -1 ; +return VMDK_ERROR ; } extent->l2_cache_offsets[min_index] = l2_offset;
@@ -888,7 +896,7 @@ static int get_cluster_offset(BlockDriverState *bs,if (!*cluster_offset) {if (!allocate) { -return -1 ; +return VMDK_UNALLOC ; }/* Avoid the L2 tables update for the images that have snapshots. */
@@ -911,7 +919,7 @@ static int get_cluster_offset(BlockDriverState *bs,*/ if (get_whole_cluster ( bs, extent, *cluster_offset, offset, allocate) ==-1 ) { -return -1 ; +return VMDK_ERROR ; }if (m_data) {
@@ -923,7 +931,7 @@ static int get_cluster_offset(BlockDriverState *bs, } } *cluster_offset <<=9 ; -return 0 ; +return VMDK_OK ; } static VmdkExtent *find_extent(BDRVVmdkState *s,
@@ -1173,7 +1181,7 @@ static int vmdk_write(BlockDriverState *bs, int64_t sector_num, sector_num <<9 , !extent->compressed, &cluster_offset);if (extent->compressed) { -if (ret ==0 ) { +if (ret ==VMDK_OK ) {/* Refuse write to allocated cluster for streamOptimized */ fprintf (stderr, "VMDK: can't write to allocated cluster"
@@ -1357,7 +1365,7 @@ static int filename_decompose(const char *filename, char *path, char *prefix,if (filename ==NULL || !strlen (filename)) {fprintf (stderr,"Vmdk: no filename provided.\n" ); -return -1 ; +return VMDK_ERROR ; } p =strrchr (filename,'/' );if (p ==NULL ) {
@@ -1369,7 +1377,7 @@ static int filename_decompose(const char *filename, char *path, char *prefix,if (p !=NULL ) { p++;if (p - filename >= buf_len) { -return -1 ; +return VMDK_ERROR ; }pstrcpy (path, p - filename +1 , filename); }else {
@@ -1382,12 +1390,12 @@ static int filename_decompose(const char *filename, char *path, char *prefix, postfix[0 ] ='\0' ; }else {if (q - p >= buf_len) { -return -1 ; +return VMDK_ERROR ; }pstrcpy (prefix, q - p +1 , p);pstrcpy (postfix, buf_len, q); } -return 0 ; +return VMDK_OK ; } static int relative_path(char *dest, int dest_size,
@@ -1403,11 +1411,11 @@ static int relative_path(char *dest, int dest_size,#endif if (!(dest && base && target)) { -return -1 ; +return VMDK_ERROR ; }if (path_is_absolute (target)) {pstrcpy (dest, dest_size, target); -return 0 ; +return VMDK_OK ; }while (base[i] == target[i]) { i++;
@@ -1426,7 +1434,7 @@ static int relative_path(char *dest, int dest_size,pstrcat (dest, dest_size, sep); }pstrcat (dest, dest_size, q); -return 0 ; +return VMDK_OK ; }static int vmdk_create (const char *filename,QEMUOptionParameter *options)
Signature · 1 line
1.8.1.4
PATCH 7/15 vmdk: add support for “zeroed‐grain” GTE
In reply to Stefan HajnocziFrom: Fam Zheng <[email protected]>
Introduced support for zeroed-grain GTE, as specified in Virtual Disk Format 5.0[1].
Recent VMware hosted platform products support a new “zeroed‐grain” grain table entry (GTE). The zeroed‐grain GTE returns all zeros on read. In other words, the zeroed‐grain GTE indicates that a grain in the child disk is zero‐filled but does not actually occupy space in storage. A sparse extent with zeroed‐grain GTE has the following in its header:
* SparseExtentHeader.version = 2
* SparseExtentHeader.flags has bit 2 set
Other than the new flag and the possibly zeroed‐grain GTE, version 2 sparse extents are identical to version 1. Also, a zeroed‐grain GTE has value 0x1 in the GT table.
[1] Virtual Disk Format 5.0, http://www.vmware.com/support/developer/vddk/vmdk_50_technote.pdf?src=vmdk
--- block/vmdk.c | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-)
block/vmdk.c +18 -6
@@ -33,10 +33,13 @@#define VMDK4_MAGIC (('K' << 24) | ('D' << 16) | ('M' << 8) | 'V')#define VMDK4_COMPRESSION_DEFLATE 1#define VMDK4_FLAG_RGD (1 << 1) +/* Zeroed-grain enable bit */ +#define VMDK4_FLAG_ZERO_GRAIN (1 << 2)#define VMDK4_FLAG_COMPRESS (1 << 16)#define VMDK4_FLAG_MARKER (1 << 17)#define VMDK4_GD_AT_END 0xffffffffffffffffULL +#define VMDK_GTE_ZEROED 0x1/* VMDK internal error codes */ #define VMDK_OK 0
@@ -81,6 +84,8 @@ typedef struct VmdkExtent {bool flat;bool compressed;bool has_marker; +bool has_zero_grain; +int version;int64_t sectors;int64_t end_sector;int64_t flat_start_offset;
@@ -569,6 +574,8 @@ static int vmdk_open_vmdk4(BlockDriverState *bs, extent->compressed =le16_to_cpu (header.compressAlgorithm) ==VMDK4_COMPRESSION_DEFLATE ; extent->has_marker =le32_to_cpu (header.flags) &VMDK4_FLAG_MARKER ; + extent->version =le32_to_cpu (header.version); + extent->has_zero_grain =le32_to_cpu (header.flags) &VMDK4_FLAG_ZERO_GRAIN ; ret =vmdk_init_tables (bs, extent);if (ret) {/* free extent allocated by vmdk_add_extent */
@@ -839,6 +846,7 @@ static int get_cluster_offset(BlockDriverState *bs,unsigned int l1_index, l2_offset, l2_index;int min_index, i, j;uint32_t min_count, *l2_table, tmp =0 ; +bool zeroed = false;if (m_data) { m_data->valid =0 ;
@@ -894,9 +902,13 @@ static int get_cluster_offset(BlockDriverState *bs, l2_index = ((offset >>9 ) / extent->cluster_sectors) % extent->l2_size; *cluster_offset =le32_to_cpu (l2_table[l2_index]); -if (!*cluster_offset) { +if (extent->has_zero_grain && *cluster_offset ==VMDK_GTE_ZEROED ) { + zeroed = true; + } + +if (!*cluster_offset || zeroed) {if (!allocate) { -return VMDK_UNALLOC ; +return zeroed ?VMDK_ZEROED :VMDK_UNALLOC ; }/* Avoid the L2 tables update for the images that have snapshots. */
@@ -967,8 +979,8 @@ static int coroutine_fn vmdk_co_is_allocated(BlockDriverState *bs, ret =get_cluster_offset (bs, extent,NULL , sector_num *512 ,0 , &offset);qemu_co_mutex_unlock (&s->lock); -/* get_cluster_offset returning 0 means success */ - ret = !ret; + + ret = (ret ==VMDK_OK || ret ==VMDK_ZEROED ); index_in_cluster = sector_num % extent->cluster_sectors; n = extent->cluster_sectors - index_in_cluster;
@@ -1111,9 +1123,9 @@ static int vmdk_read(BlockDriverState *bs, int64_t sector_num,if (n > nb_sectors) { n = nb_sectors; } -if (ret) { +if (ret !=VMDK_OK ) {/* if not allocated, try to read from parent image, if exist */ -if (bs->backing_hd) { +if (bs->backing_hd && ret !=VMDK_ZEROED ) {if (!vmdk_is_cid_valid (bs)) {return -EINVAL ; }
Signature · 1 line
1.8.1.4
PATCH 8/15 vmdk: Add option to create zeroed-grain image
In reply to Stefan HajnocziFrom: Fam Zheng <[email protected]>
Add image create option "zeroed-grain" to enable zeroed-grain GTE feature of vmdk sparse extents. When this option is on, header version of newly created extent will be 2 and VMDK4_FLAG_ZERO_GRAIN flag bit will be set.
--- block/vmdk.c | 22 +++++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-)
block/vmdk.c +17 -5
@@ -48,6 +48,8 @@#define VMDK_UNALLOC (-2)#define VMDK_ZEROED (-3) +#define BLOCK_OPT_ZEROED_GRAIN "zeroed_grain" + typedef struct { uint32_t version; uint32_t flags;
@@ -1262,7 +1264,7 @@ static coroutine_fn int vmdk_co_write(BlockDriverState *bs, int64_t sector_num,static int vmdk_create_extent (const char *filename,int64_t filesize, -bool flat,bool compress) +bool flat,bool compress,bool zeroed_grain) {int ret, i;int fd =0 ;
@@ -1284,9 +1286,10 @@ static int vmdk_create_extent(const char *filename, int64_t filesize, } magic =cpu_to_be32 (VMDK4_MAGIC );memset (&header,0 ,sizeof (header)); - header.version =1 ; - header.flags = -3 | (compress ?VMDK4_FLAG_COMPRESS |VMDK4_FLAG_MARKER :0 ); + header.version = zeroed_grain ?2 :1 ; + header.flags =3 + | (compress ?VMDK4_FLAG_COMPRESS |VMDK4_FLAG_MARKER :0 ) + | (zeroed_grain ?VMDK4_FLAG_ZERO_GRAIN :0 ); header.compressAlgorithm = compress ?VMDK4_COMPRESSION_DEFLATE :0 ; header.capacity = filesize /512 ; header.granularity =128 ;
@@ -1467,6 +1470,7 @@ static int vmdk_create(const char *filename, QEMUOptionParameter *options)char parent_desc_line[BUF_SIZE ] ="" ;uint32_t parent_cid =0xffffffff ;uint32_t number_heads =16 ; +bool zeroed_grain = false;const char desc_template[] ="# Disk DescriptorFile\n" "version=1\n"
@@ -1502,6 +1506,8 @@ static int vmdk_create(const char *filename, QEMUOptionParameter *options) flags |= options->value.n ?BLOCK_FLAG_COMPAT6 :0 ; }else if (!strcmp (options->name,BLOCK_OPT_SUBFMT )) { fmt = options->value.s; + }else if (!strcmp (options->name,BLOCK_OPT_ZEROED_GRAIN )) { + zeroed_grain |= options->value.n; } options++; }
@@ -1588,7 +1594,8 @@ static int vmdk_create(const char *filename, QEMUOptionParameter *options)snprintf (ext_filename,sizeof (ext_filename),"%s%s" , path, desc_filename); -if (vmdk_create_extent (ext_filename, size, flat, compress)) { +if (vmdk_create_extent (ext_filename, size, + flat, compress, zeroed_grain)) {return -EINVAL ; } filesize -= size;
@@ -1714,6 +1721,11 @@ static QEMUOptionParameter vmdk_create_options[] = {"VMDK flat extent format, can be one of " "{monolithicSparse (default) | monolithicFlat | twoGbMaxExtentSparse | twoGbMaxExtentFlat | streamOptimized} " }, + { + .name = BLOCK_OPT_ZEROED_GRAIN, + .type = OPT_FLAG, + .help ="Enable efficient zero writes using the zeroed-grain GTE feature" + }, { NULL } };
Signature · 1 line
1.8.1.4
PATCH 9/15 vmdk: change magic number to macro
In reply to Stefan HajnocziFrom: Fam Zheng <[email protected]>
Two hard coded flag bits are changed to macros.
--- block/vmdk.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
block/vmdk.c +2 -1
@@ -32,6 +32,7 @@#define VMDK3_MAGIC (('C' << 24) | ('O' << 16) | ('W' << 8) | 'D')#define VMDK4_MAGIC (('K' << 24) | ('D' << 16) | ('M' << 8) | 'V')#define VMDK4_COMPRESSION_DEFLATE 1 +#define VMDK4_FLAG_NL_DETECT (1 << 0)#define VMDK4_FLAG_RGD (1 << 1)/* Zeroed-grain enable bit */ #define VMDK4_FLAG_ZERO_GRAIN (1 << 2)
@@ -1287,7 +1288,7 @@ static int vmdk_create_extent(const char *filename, int64_t filesize, magic =cpu_to_be32 (VMDK4_MAGIC );memset (&header,0 ,sizeof (header)); header.version = zeroed_grain ?2 :1 ; - header.flags =3 + header.flags =VMDK4_FLAG_RGD |VMDK4_FLAG_NL_DETECT | (compress ?VMDK4_FLAG_COMPRESS |VMDK4_FLAG_MARKER :0 ) | (zeroed_grain ?VMDK4_FLAG_ZERO_GRAIN :0 ); header.compressAlgorithm = compress ?VMDK4_COMPRESSION_DEFLATE :0 ;
Signature · 1 line
1.8.1.4
PATCH 10/15 vmdk: store fields of VmdkMetaData in cpu endian
In reply to Stefan HajnocziFrom: Fam Zheng <[email protected]>
Previously VmdkMetaData.offset is stored little endian while other fields are cpu endian. This changes offset to cpu endian and convert before writing to image.
--- block/vmdk.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-)
block/vmdk.c +8 -9
@@ -813,14 +813,15 @@ static int get_whole_cluster(BlockDriverState *bs,static int vmdk_L2update (VmdkExtent *extent,VmdkMetaData *m_data) { +uint32_t offset; +QEMU_BUILD_BUG_ON (sizeof (offset) !=sizeof (m_data->offset)); + offset =cpu_to_le32 (m_data->offset);/* update L2 table */ if (bdrv_pwrite_sync ( extent->file, ((int64_t )m_data->l2_offset *512 ) + (m_data->l2_index *sizeof (m_data->offset)), - &(m_data->offset), -sizeof (m_data->offset) - ) <0 ) { + &offset,sizeof (offset)) <0 ) {return VMDK_ERROR ; }/* update backup L2 table */
@@ -830,8 +831,7 @@ static int vmdk_L2update(VmdkExtent *extent, VmdkMetaData *m_data) extent->file, ((int64_t)m_data->l2_offset * 512) + (m_data->l2_index * sizeof(m_data->offset)), - &(m_data->offset), sizeof(m_data->offset) - ) < 0) { + &offset, sizeof(offset)) < 0) {return VMDK_ERROR ; } }
@@ -848,7 +848,7 @@ static int get_cluster_offset(BlockDriverState *bs, {unsigned int l1_index, l2_offset, l2_index;int min_index, i, j; -uint32_t min_count, *l2_table, tmp =0 ; +uint32_t min_count, *l2_table;bool zeroed = false;if (m_data) {
@@ -924,8 +924,7 @@ static int get_cluster_offset(BlockDriverState *bs, } *cluster_offset >>=9 ; - tmp =cpu_to_le32 (*cluster_offset); - l2_table[l2_index] = tmp; + l2_table[l2_index] =cpu_to_le32 (*cluster_offset);/* First of all we write grain itself, to avoid race condition * that may to corrupt the image.
@@ -938,7 +937,7 @@ static int get_cluster_offset(BlockDriverState *bs, }if (m_data) { - m_data->offset = tmp; + m_data->offset = *cluster_offset; m_data->l1_index = l1_index; m_data->l2_index = l2_index; m_data->l2_offset = l2_offset;
Signature · 1 line
1.8.1.4
PATCH 11/15 vmdk: add bdrv_co_write_zeroes
In reply to Stefan HajnocziFrom: Fam Zheng <[email protected]>
Use special offset to write zeroes efficiently, when zeroed-grain GTE is available. If zero-write an allocated cluster, cluster is leaked because its offset pointer is overwritten by "0x1".
--- block/vmdk.c | 86 +++++++++++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 68 insertions(+), 18 deletions(-)
block/vmdk.c +68 -18
@@ -124,6 +124,7 @@ typedef struct VmdkMetaData {unsigned int l2_index;unsigned int l2_offset;int valid; +uint32_t *l2_cache_entry; } VmdkMetaData; typedef struct VmdkGrainMarker {
@@ -835,6 +836,9 @@ static int vmdk_L2update(VmdkExtent *extent, VmdkMetaData *m_data)return VMDK_ERROR ; } } +if (m_data->l2_cache_entry) { + *m_data->l2_cache_entry = offset; + }return VMDK_OK ; }
@@ -905,6 +909,14 @@ static int get_cluster_offset(BlockDriverState *bs, l2_index = ((offset >>9 ) / extent->cluster_sectors) % extent->l2_size; *cluster_offset =le32_to_cpu (l2_table[l2_index]); +if (m_data) { + m_data->valid =1 ; + m_data->l1_index = l1_index; + m_data->l2_index = l2_index; + m_data->offset = *cluster_offset; + m_data->l2_offset = l2_offset; + m_data->l2_cache_entry = &l2_table[l2_index]; + }if (extent->has_zero_grain && *cluster_offset ==VMDK_GTE_ZEROED ) { zeroed = true; }
@@ -938,10 +950,6 @@ static int get_cluster_offset(BlockDriverState *bs,if (m_data) { m_data->offset = *cluster_offset; - m_data->l1_index = l1_index; - m_data->l2_index = l2_index; - m_data->l2_offset = l2_offset; - m_data->valid =1 ; } } *cluster_offset <<=9 ;
@@ -1164,8 +1172,17 @@ static coroutine_fn int vmdk_co_read(BlockDriverState *bs, int64_t sector_num,return ret; } +/** +* vmdk_write: +* @zeroed: buf is ignored (data is zero), use zeroed_grain GTE feature +* if possible, otherwise return -ENOTSUP. +* @zero_dry_run: used for zeroed == true only, don't update L2 table, just +* +* Returns: error code with 0 for success. +*/ static int vmdk_write (BlockDriverState *bs,int64_t sector_num, -const uint8_t *buf,int nb_sectors) +const uint8_t *buf,int nb_sectors, +bool zeroed,bool zero_dry_run) {BDRVVmdkState *s = bs->opaque;VmdkExtent *extent =NULL ;
@@ -1211,7 +1228,7 @@ static int vmdk_write(BlockDriverState *bs, int64_t sector_num, &cluster_offset); } } -if (ret) { +if (ret ==VMDK_ERROR ) {return -EINVAL ; } extent_begin_sector = extent->end_sector - extent->sectors;
@@ -1221,17 +1238,34 @@ static int vmdk_write(BlockDriverState *bs, int64_t sector_num,if (n > nb_sectors) { n = nb_sectors; } - - ret =vmdk_write_extent (extent, - cluster_offset, index_in_cluster *512 , - buf, n, sector_num); -if (ret) { -return ret; - } -if (m_data.valid) { -/* update L2 tables */ -if (vmdk_L2update (extent, &m_data) ==-1 ) { -return -EIO ; +if (zeroed) { +/* Do zeroed write, buf is ignored */ +if (extent->has_zero_grain && + index_in_cluster ==0 && + n >= extent->cluster_sectors) { + n = extent->cluster_sectors; +if (!zero_dry_run) { + m_data.offset =VMDK_GTE_ZEROED ; +/* update L2 tables */ +if (vmdk_L2update (extent, &m_data) !=VMDK_OK ) { +return -EIO ; + } + } + }else { +return -ENOTSUP ; + } + }else { + ret =vmdk_write_extent (extent, + cluster_offset, index_in_cluster *512 , + buf, n, sector_num); +if (ret) { +return ret; + } +if (m_data.valid) { +/* update L2 tables */ +if (vmdk_L2update (extent, &m_data) !=VMDK_OK ) { +return -EIO ; + } } } nb_sectors -= n;
@@ -1257,7 +1291,22 @@ static coroutine_fn int vmdk_co_write(BlockDriverState *bs, int64_t sector_num,int ret;BDRVVmdkState *s = bs->opaque;qemu_co_mutex_lock (&s->lock); - ret =vmdk_write (bs, sector_num, buf, nb_sectors); + ret =vmdk_write (bs, sector_num, buf, nb_sectors, false, false); +qemu_co_mutex_unlock (&s->lock); +return ret; +} + +static int coroutine_fn vmdk_co_write_zeroes(BlockDriverState *bs, +int64_t sector_num, +int nb_sectors) +{ +int ret; +BDRVVmdkState *s = bs->opaque; +qemu_co_mutex_lock (&s->lock); + ret =vmdk_write (bs, sector_num,NULL , nb_sectors, true, true); +if (!ret) { + ret =vmdk_write (bs, sector_num,NULL , nb_sectors, true, false); + }qemu_co_mutex_unlock (&s->lock);return ret; }
@@ -1737,6 +1786,7 @@ static BlockDriver bdrv_vmdk = { .bdrv_reopen_prepare = vmdk_reopen_prepare, .bdrv_read = vmdk_co_read, .bdrv_write = vmdk_co_write, + .bdrv_co_write_zeroes = vmdk_co_write_zeroes, .bdrv_close = vmdk_close, .bdrv_create = vmdk_create, .bdrv_co_flush_to_disk = vmdk_co_flush,
Signature · 1 line
1.8.1.4
PATCH 12/15 qemu-iotests: Filter out vmdk creation options
In reply to Stefan HajnocziFrom: Fam Zheng <[email protected]>
Cover new image creation options for vmdk, so we can use '-o zeroed_grain=XXX' and '-o subformat=XXX' to run the tests successfully.
--- tests/qemu-iotests/common.rc | 2 ++ 1 file changed, 2 insertions(+)
tests/qemu-iotests/common.rc +2 -0
@@ -127,6 +127,8 @@ _make_test_img() -e "s# compat='[^']*'##g" \ -e "s# compat6=\\(on\\|off\\)##g" \ -e "s# static=\\(on\\|off\\)##g" \ + -e "s# zeroed_grain=\\(on\\|off\\)##g" \ + -e "s# subformat='[^']*'##g" \ -e "s# lazy_refcounts=\\(on\\|off\\)##g" # Start an NBD server on the image file, which is what we'll be talking to
Signature · 1 line
1.8.1.4
PATCH 13/15 nbd: use g_slice_new() instead of a freelist
In reply to Stefan HajnocziUse GLib's efficient slice allocator instead of open-coding the request freelist. This patch simplifies the NBDRequest code.
Now we qemu_blockalign() the req->data buffer each time but the next patch switches from a fixed size buffer to a dynamic size anyway.
--- nbd.c | 23 ++++++----------------- 1 file changed, 6 insertions(+), 17 deletions(-)
nbd.c +6 -17
@@ -98,7 +98,6 @@ struct NBDExport {off_t size;uint32_t nbdflags; QTAILQ_HEAD(, NBDClient) clients; - QSIMPLEQ_HEAD(, NBDRequest) requests;QTAILQ_ENTRY (NBDExport) next; };
@@ -850,13 +849,8 @@ static NBDRequest *nbd_request_get(NBDClient *client)assert (client->nb_requests <=MAX_NBD_REQUESTS -1 ); client->nb_requests++; -if (QSIMPLEQ_EMPTY (&exp->requests)) { - req =g_malloc0 (sizeof (NBDRequest)); - req->data =qemu_blockalign (exp->bs,NBD_BUFFER_SIZE ); - }else { - req =QSIMPLEQ_FIRST (&exp->requests); -QSIMPLEQ_REMOVE_HEAD (&exp->requests, entry); - } + req =g_slice_new0 (NBDRequest); + req->data =qemu_blockalign (exp->bs,NBD_BUFFER_SIZE );nbd_client_get (client); req->client = client;return req;
@@ -865,7 +859,10 @@ static NBDRequest *nbd_request_get(NBDClient *client)static void nbd_request_put (NBDRequest *req) {NBDClient *client = req->client; -QSIMPLEQ_INSERT_HEAD (&client->exp->requests, req, entry); + +qemu_vfree (req->data); +g_slice_free (NBDRequest, req); +if (client->nb_requests-- ==MAX_NBD_REQUESTS ) {qemu_notify_event (); }
@@ -877,7 +874,6 @@ NBDExport *nbd_export_new(BlockDriverState *bs, off_t dev_offset, void (*close)(NBDExport *)) {NBDExport *exp =g_malloc0 (sizeof (NBDExport)); -QSIMPLEQ_INIT (&exp->requests); exp->refcount =1 ;QTAILQ_INIT (&exp->clients); exp->bs = bs;
@@ -953,13 +949,6 @@ void nbd_export_put(NBDExport *exp) exp->close (exp); } -while (!QSIMPLEQ_EMPTY (&exp->requests)) { -NBDRequest *first =QSIMPLEQ_FIRST (&exp->requests); -QSIMPLEQ_REMOVE_HEAD (&exp->requests, entry); -qemu_vfree (first->data); -g_free (first); - } -g_free (exp); } }
Signature · 1 line
1.8.1.4
PATCH 14/15 nbd: support large NBD requests
In reply to Stefan HajnocziThe Linux nbd driver recently increased the maximum supported request size up to 32 MB:
commit 078be02b80359a541928c899c2631f39628f56df
Author: Michal Belczyk <[email protected]>
Date: Tue Apr 30 15:28:28 2013 -0700
nbd: increase default and max request sizes
Raise the default max request size for nbd to 128KB (from 127KB) to get it 4KB aligned. This patch also allows the max request size to be increased (via /sys/block/nbd<x>/queue/max_sectors_kb) to 32MB.
QEMU's 1 MB buffers are too small to handle these requests.
This patch allocates data buffers dynamically and allows up to 32 MB per request.
--- include/block/nbd.h | 3 ++- nbd.c | 17 +++++++++++------ 2 files changed, 13 insertions(+), 7 deletions(-)
include/block/nbd.h +2 -1
@@ -58,7 +58,8 @@ enum {#define NBD_DEFAULT_PORT 10809 -#define NBD_BUFFER_SIZE (1024*1024) +/* Maximum size of a single READ/WRITE data buffer */ +#define NBD_MAX_BUFFER_SIZE (32 * 1024 * 1024)ssize_t nbd_wr_sync (int fd,void *buffer,size_t size,bool do_read);int tcp_socket_incoming (const char *address,uint16_t port);
nbd.c +11 -6
@@ -844,13 +844,11 @@ void nbd_client_close(NBDClient *client)static NBDRequest *nbd_request_get (NBDClient *client) {NBDRequest *req; -NBDExport *exp = client->exp;assert (client->nb_requests <=MAX_NBD_REQUESTS -1 ); client->nb_requests++; req =g_slice_new0 (NBDRequest); - req->data =qemu_blockalign (exp->bs,NBD_BUFFER_SIZE );nbd_client_get (client); req->client = client;return req;
@@ -860,7 +858,9 @@ static void nbd_request_put(NBDRequest *req) {NBDClient *client = req->client; -qemu_vfree (req->data); +if (req->data) { +qemu_vfree (req->data); + }g_slice_free (NBDRequest, req);if (client->nb_requests-- ==MAX_NBD_REQUESTS ) {
@@ -1007,6 +1007,7 @@ static ssize_t nbd_co_receive_request(NBDRequest *req, struct nbd_request *reque {NBDClient *client = req->client;int csock = client->sock; +uint32_t command;ssize_t rc; client->recv_coroutine =qemu_coroutine_self ();
@@ -1018,9 +1019,9 @@ static ssize_t nbd_co_receive_request(NBDRequest *req, struct nbd_request *reque goto out; } -if (request->len >NBD_BUFFER_SIZE ) { +if (request->len >NBD_MAX_BUFFER_SIZE ) {LOG ("len (%u) is larger than max len (%u)" , - request->len,NBD_BUFFER_SIZE ); + request->len,NBD_MAX_BUFFER_SIZE ); rc = -EINVAL ; goto out; }
@@ -1034,7 +1035,11 @@ static ssize_t nbd_co_receive_request(NBDRequest *req, struct nbd_request *requeTRACE ("Decoding type" ); -if ((request->type &NBD_CMD_MASK_COMMAND ) ==NBD_CMD_WRITE ) { + command = request->type &NBD_CMD_MASK_COMMAND ; +if (command ==NBD_CMD_READ || command ==NBD_CMD_WRITE ) { + req->data =qemu_blockalign (client->exp->bs, request->len); + } +if (command ==NBD_CMD_WRITE ) {TRACE ("Reading %u byte(s)" , request->len);if (qemu_co_recv (csock, req->data, request->len) != request->len) {
Signature · 1 line
1.8.1.4
PATCH 15/15 qemu-iotests: Filter out 'adapter_type'
In reply to Stefan HajnocziFrom: Fam Zheng <[email protected]>
Filter out vmdk creation option 'adapter_type' for vmdk. So that tests with an explicit './check -o adapter_type=XXX' will not fail.
--- tests/qemu-iotests/common.rc | 1 + 1 file changed, 1 insertion(+)
tests/qemu-iotests/common.rc +1 -0
@@ -129,6 +129,7 @@ _make_test_img() -e "s# static=\\(on\\|off\\)##g" \ -e "s# zeroed_grain=\\(on\\|off\\)##g" \ -e "s# subformat='[^']*'##g" \ + -e "s# adapter_type='[^']*'##g" \ -e "s# lazy_refcounts=\\(on\\|off\\)##g" # Start an NBD server on the image file, which is what we'll be talking to
Signature · 1 line
1.8.1.4