Initial VHDX support
v3 cover Initial VHDX support
Differences from v2:
Patch 2/4: changed 2 uint8_t[16] to MSGUID (Kevin)
renamed all strucs to proper style (Kevin) indentations / style (Kevin) Removed/fixed outdated comments (Stefan) dropped vhdx_header_padded (Kevin)
Patch 3/4: removed hunk that was meant for old patch 5/5 (Fam)
check VHDX header version field (Fam)
validate file type id in open and not just probe (Kevin) allow headers with seq # of 0 if other header is invalid (Kevin) fixed masking typo for parent meta present (Kevin) removed error check for 512-byte sectors (Kevin) verify region and metadata table guid are unique (Kevin) removed signed/unsigned comparisons (Stefan)
sanity checking on rt.entry_count (Stefan)
use clz_() instead of inline code (Stefan) bat_rt.length sanity check - compute it rather
than trust the entry length (Stefan)
Patch 5 (v2): Dropped
This adds the initial support for VHDX image files.
It currently only supports read operations of VHDX, for fixed and dynamic files.
Notably, the following is not yet supported:
* Differencing files
* Log replay (so we will refuse to open any images that are not 'clean')
* .bdrv_create()
* write operations other than to the header
Jeff Cody (4):
qemu: add castagnoli crc32c checksum algorithm
block: vhdx header for the QEMU support of VHDX images
block: initial VHDX driver support framework - supports open and probe
block: add read-only support to VHDX image format.
block/Makefile.objs | 1 + block/vhdx.c | 972 ++++++++++++++++++++++++++++++++++++++++++++++++++ block/vhdx.h | 325 +++++++++++++++++ include/qemu/crc32c.h | 35 ++ util/Makefile.objs | 1 + util/crc32c.c | 115 ++++++ 6 files changed, 1449 insertions(+) create mode 100644 block/vhdx.c create mode 100644 block/vhdx.h create mode 100644 include/qemu/crc32c.h create mode 100644 util/crc32c.c
Signature · 1 line
1.8.1.4
PATCH v3 1/4 qemu: add castagnoli crc32c checksum algorithm
This adds the Castagnoli CRC32C algorithm, using the 0x11EDC6F41 polynomial.
This is extracted from the linux kernel cryptographic crc32.c module.
The algorithm is based on:
Castagnoli93: Guy Castagnoli and Stefan Braeuer and Martin Herrman
"Optimization of Cyclic Redundancy-Check Codes with 24
and 32 Parity Bits", IEEE Transactions on Communication, Volume 41, Number 6, June 1993
--- include/qemu/crc32c.h | 35 +++++++++++++++ util/Makefile.objs | 1 + util/crc32c.c | 115 ++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 151 insertions(+) create mode 100644 include/qemu/crc32c.h create mode 100644 util/crc32c.c
include/qemu/crc32c.h new file +35 -0
@@ -0,0 +1,35 @@ +/* +* Castagnoli CRC32C Checksum Algorithm +* +* Polynomial: 0x11EDC6F41 +* +* Castagnoli93: Guy Castagnoli and Stefan Braeuer and Martin Herrman +* "Optimization of Cyclic Redundancy-Check Codes with 24 +* and 32 Parity Bits",IEEE Transactions on Communication, +* Volume 41, Number 6, June 1993 +* +* Copyright (c) 2013 Red Hat, Inc., +* +* Authors: +* Jeff Cody <[email protected]> +* +* Based on the Linux kernel cryptographic crc32c module, +* +* Copyright (c) 2004 Cisco Systems, Inc. +* Copyright (c) 2008 Herbert Xu <[email protected]> +* +* This program is free software; you can redistribute it and/or modify it +* under the terms of the GNU General Public License as published by the Free +* Software Foundation; either version 2 of the License, or (at your option) +* any later version. +* +*/ + +#ifndef QEMU_CRC32_H +#define QEMU_CRC32_H + +#include "qemu-common.h" + +uint32_t crc32c (uint32_t crc,const uint8_t *data,unsigned int length); + +#endif
util/Makefile.objs +1 -0
@@ -10,3 +10,4 @@ util-obj-$(CONFIG_POSIX) += compatfd.o util-obj-y += iov.o aes.o qemu-config.o qemu-sockets.o uri.o notify.o util-obj-y += qemu-option.o qemu-progress.o util-obj-y += hexdump.o +util-obj-y += crc32c.o
util/crc32c.c new file +115 -0
115 lines not on this page · raw message
Signature · 1 line
1.8.1.4
PATCH v3 2/4 block: vhdx header for the QEMU support of VHDX images
In reply to Jeff CodyThis is based on Microsoft's VHDX specification:
"VHDX Format Specification v0.95", published 4/12/2012
https://www.microsoft.com/en-us/download/details.aspx?id=29681These structures define the various header, metadata, and other block structures defined in the VHDX specification.
--- block/vhdx.h | 311 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 311 insertions(+) create mode 100644 block/vhdx.h
block/vhdx.h new file +311 -0
@@ -0,0 +1,311 @@ +/* +* Block driver for Hyper-V VHDX Images +* +* Copyright (c) 2013 Red Hat, Inc., +* +* Authors: +* Jeff Cody <[email protected]> +* +* This is based on the "VHDX Format Specification v0.95", published 4/12/2012 +* by Microsoft: +* https://www.microsoft.com/en-us/download/details.aspx?id=29681 +* +* This work is licensed under the terms of the GNU LGPL, version 2 or later. +* See the COPYING.LIB file in the top-level directory. +* +*/ + +#ifndef BLOCK_VHDX_H +#define BLOCK_VHDX_H + +/* Structures and fields present in the VHDX file */ + +/* The header section has the following blocks, +* each block is 64KB: +* +* _____________________________________________________________________________ +* | File Id. | Header 1 | Header 2 | Region Table | Reserved (768KB) | +* |----------|---------------|------------|--------------|--------------------| +* | | | | | | +* 0.........64KB...........128KB........192KB..........256KB................1MB +*/ + +#define VHDX_HEADER_BLOCK_SIZE (64*1024) + +#define VHDX_FILE_ID_OFFSET 0 +#define VHDX_HEADER1_OFFSET (VHDX_HEADER_BLOCK_SIZE*1) +#define VHDX_HEADER2_OFFSET (VHDX_HEADER_BLOCK_SIZE*2) +#define VHDX_REGION_TABLE_OFFSET (VHDX_HEADER_BLOCK_SIZE*3) + + +/* +* A note on the use of MS-GUID fields. For more details on the GUID, +* please see: https://en.wikipedia.org/wiki/Globally_unique_identifier. +* +* The VHDX specification only states that these are MS GUIDs, and which +* bytes are data1-data4. It makes no mention of what algorithm should be used +* to generate the GUID, nor what standard. However, looking at the specified +* known GUID fields, it appears the GUIDs are: +* Standard/DCE GUID type (noted by 10b in the MSB of byte 0 of .data4) +* Random algorithm (noted by 0x4XXX for .data3) +*/ + +/* ---- HEADER SECTION STRUCTURES ---- */ + +/* These structures are ones that are defined in the VHDX specification +* document */ + +typedef struct VHDXFileIdentifier { +uint64_t signature;/* "vhdxfile" in ASCII */ +uint16_t creator[256 ];/* optional; utf-16 string to identify +the vhdx file creator. Diagnotistic +only */ +}VHDXFileIdentifier ; + + +/* the guid is a 16 byte unique ID - the definition for this used by +* Microsoft is not just 16 bytes though - it is a structure that is defined, +* so we need to follow it here so that endianness does not trip us up */ + +typedef struct MSGUID { +uint32_t data1; +uint16_t data2; +uint16_t data3; +uint8_t data4[8 ]; +}MSGUID ; + +#define guid_eq (a, b) \ + (memcmp(&(a), &(b), sizeof(MSGUID)) == 0) + +#define VHDX_HEADER_SIZE (4*1024)/* although the vhdx_header struct in disk +is only 582 bytes, for purposes of crc +the header is the first 4KB of the 64KB +block */ + +/* The full header is 4KB, although the actual header data is much smaller. +* But for the checksum calculation, it is over the entire 4KB structure, +* not just the defined portion of it */ +typedef struct QEMU_PACKED VHDXHeader { +uint32_t signature;/* "head" in ASCII */ +uint32_t checksum;/* CRC-32C hash of the whole header */ +uint64_t sequence_number;/* Seq number of this header. Each +VHDX file has 2 of these headers, +and only the header with the highest +sequence number is valid */ +MSGUID file_write_guid;/* 128 bit unique identifier. Must be +updated to new, unique value before +the first modification is made to +file */ +MSGUID data_write_guid;/* 128 bit unique identifier. Must be +updated to new, unique value before +the first modification is made to +visible data. Visbile data is +defined as: +- system & user metadata +- raw block data +- disk size +- any change that will +cause the virtual disk +sector read to differ + +This does not need to change if +blocks are re-arranged */ +MSGUID log_guid;/* 128 bit unique identifier. If zero, +there is no valid log. If non-zero, +log entries with this guid are +valid. */ +uint16_t log_version;/* version of the log format. Mustn't be +zero, unless log_guid is also zero */ +uint16_t version;/* version of th evhdx file. Currently, +only supported version is "1" */ +uint32_t log_length;/* length of the log. Must be multiple +of 1MB */ +uint64_t log_offset;/* byte offset in the file of the log. +Must also be a multiple of 1MB */ +} VHDXHeader; + +/* Header for the region table block */ +typedef struct QEMU_PACKED VHDXRegionTableHeader { +uint32_t signature;/* "regi" in ASCII */ +uint32_t checksum;/* CRC-32C hash of the 64KB table */ +uint32_t entry_count;/* number of valid entries */ +uint32_t reserved; +} VHDXRegionTableHeader; + +/* Individual region table entry. There may be a maximum of 2047 of these +* +* There are two known region table properties. Both are required. +* BAT (block allocation table): 2DC27766F62342009D64115E9BFD4A08 +* Metadata: 8B7CA20647904B9AB8FE575F050F886E +*/ +#define VHDX_REGION_ENTRY_REQUIRED 0x01/* if set, parser must understand +this entry in order to open +file */ +typedef struct QEMU_PACKED VHDXRegionTableEntry { +MSGUID guid;/* 128-bit unique identifier */ +uint64_t file_offset;/* offset of the object in the file. +Must be multiple of 1MB */ +uint32_t length;/* length, in bytes, of the object */ +uint32_t data_bits; +} VHDXRegionTableEntry; + + +/* ---- LOG ENTRY STRUCTURES ---- */ +#define VHDX_LOG_HDR_SIZE 64 +typedef struct QEMU_PACKED VHDXLogEntryHeader { +uint32_t signature;/* "loge" in ASCII */ +uint32_t checksum;/* CRC-32C hash of the 64KB table */ +uint32_t entry_length;/* length in bytes, multiple of 1MB */ +uint32_t tail;/* byte offset of first log entry of a +seq, where this entry is the last +entry */ +uint64_t sequence_number;/* incremented with each log entry. +May not be zero. */ +uint32_t descriptor_count;/* number of descriptors in this log +entry, must be >= 0 */ +uint32_t reserved; +MSGUID log_guid;/* value of the log_guid from +vhdx_header. If not found in +vhdx_header, it is invalid */ +uint64_t flushed_file_offset;/* see spec for full details - this +sould be vhdx file size in bytes */ +uint64_t last_file_offset;/* size in bytes that all allocated +file structures fit into */ +} VHDXLogEntryHeader; + +#define VHDX_LOG_DESC_SIZE 32 + +typedef struct QEMU_PACKED VHDXLogDescriptor { +uint32_t signature;/* "zero" or "desc" in ASCII */ +union { +uint32_t reserved;/* zero desc */ +uint32_t trailing_bytes;/* data desc: bytes 4092-4096 of the +data sector */ + }; +union { +uint64_t zero_length;/* zero desc: length of the section to +zero */ +uint64_t leading_bytes;/* data desc: bytes 0-7 of the data +sector */ + }; +uint64_t file_offset;/* file offset to write zeros - multiple +of 4kB */ +uint64_t sequence_number;/* must match same field in +vhdx_log_entry_header */ +} VHDXLogDescriptor; + +typedef struct QEMU_PACKED VHDXLogDataSector { +uint32_t data_signature;/* "data" in ASCII */ +uint32_t sequence_high;/* 4 MSB of 8 byte sequence_number */ +uint8_t data[4084 ];/* raw data, bytes 8-4091 (inclusive). +see the data descriptor field for the +other mising bytes */ +uint32_t sequence_low;/* 4 LSB of 8 byte sequence_number */ +} VHDXLogDataSector; + + + +/* block states - different state values depending on whether it is a +* payload block, or a sector block. */ + +#define PAYLOAD_BLOCK_NOT_PRESENT 0 +#define PAYLOAD_BLOCK_UNDEFINED 1 +#define PAYLOAD_BLOCK_ZERO 2 +#define PAYLOAD_BLOCK_UNMAPPED 5 +#define PAYLOAD_BLOCK_FULL_PRESENT 6 +#define PAYLOAD_BLOCK_PARTIALLY_PRESENT 7 + +#define SB_BLOCK_NOT_PRESENT 0 +#define SB_BLOCK_PRESENT 6 + +/* per the spec */ +#define VHDX_MAX_SECTORS_PER_BLOCK (1<<23) + +/* upper 44 bits are the file offset in 1MB units lower 3 bits are the state +other bits are reserved */ +#define VHDX_BAT_STATE_BIT_MASK 0x07 +#define VHDX_BAT_FILE_OFF_BITS (64-44) +typedef uint64_t VHDXBatEntry ; + +/* ---- METADATA REGION STRUCTURES ---- */ + +#define VHDX_METADATA_ENTRY_SIZE 32 +#define VHDX_METADATA_MAX_ENTRIES 2047/* not including the header */ +#define VHDX_METADATA_TABLE_MAX_SIZE \ + (VHDX_METADATA_ENTRY_SIZE * (VHDX_METADATA_MAX_ENTRIES+1)) +typedef struct QEMU_PACKED VHDXMetadataTableHeader { +uint64_t signature;/* "metadata" in ASCII */ +uint16_t reserved; +uint16_t entry_count;/* number table entries. <= 2047 */ +uint32_t reserved2[5 ]; +} VHDXMetadataTableHeader; + +#define VHDX_META_FLAGS_IS_USER 0x01/* max 1024 entries */ +#define VHDX_META_FLAGS_IS_VIRTUAL_DISK 0x02/* virtual disk metadata if set, +otherwise file metdata */ +#define VHDX_META_FLAGS_IS_REQUIRED 0x04/* parse must understand this +entry to open the file */ +typedef struct QEMU_PACKED VHDXMetadataTableEntry { +MSGUID item_id;/* 128-bit identifier for metadata */ +uint32_t offset;/* byte offset of the metadata. At +least 64kB. Relative to start of +metadata region */ +/* note: if length = 0, so is offset */ +uint32_t length;/* length of metadata. <= 1MB. */ +uint32_t data_bits;/* least-significant 3 bits are flags, the +rest are reserved (see above) */ +uint32_t reserved2; +} VHDXMetadataTableEntry; + +#define VHDX_PARAMS_LEAVE_BLOCKS_ALLOCED 0x01/* Do not change any blocks to +be BLOCK_NOT_PRESENT. +If set indicates a fixed +size VHDX file */ +#define VHDX_PARAMS_HAS_PARENT 0x02/* has parent / backing file */ +typedef struct QEMU_PACKED VHDXFileParameters { +uint32_t block_size;/* size of each payload block, always +power of 2, <= 256MB and >= 1MB. */ +uint32_t data_bits;/* least-significant 2 bits are flags, the rest +are reserved (see above) */ +} VHDXFileParameters; + +typedef struct QEMU_PACKED VHDXVirtualDiskSize { +uint64_t virtual_disk_size;/* Size of the virtual disk, in bytes. +Must be multiple of the sector size, +max of 64TB */ +} VHDXVirtualDiskSize; + +typedef struct QEMU_PACKED VHDXPage83Data { +MSGUID page_83_data[16 ];/* unique id for scsi devices that +support page 0x83 */ +} VHDXPage83Data; + +typedef struct QEMU_PACKED VHDXVirtualDiskLogicalSectorSize { +uint32_t logical_sector_size;/* virtual disk sector size (in bytes). +Can only be 512 or 4096 bytes */ +} VHDXVirtualDiskLogicalSectorSize; + +typedef struct QEMU_PACKED VHDXVirtualDiskPhysicalSectorSize { +uint32_t physical_sector_size;/* physical sector size (in bytes). +Can only be 512 or 4096 bytes */ +} VHDXVirtualDiskPhysicalSectorSize; + +typedef struct QEMU_PACKED VHDXParentLocatorHeader { +MSGUID locator_type[16 ];/* type of the parent virtual disk. */ +uint16_t reserved; +uint16_t key_value_count;/* number of key/value pairs for this +locator */ +} VHDXParentLocatorHeader; + +/* key and value strings are UNICODE strings, UTF-16 LE encoding, no NULs */ +typedef struct QEMU_PACKED VHDXParentLocatorEntry { +uint32_t key_offset;/* offset in metadata for key, > 0 */ +uint32_t value_offset;/* offset in metadata for value, >0 */ +uint16_t key_length;/* length of entry key, > 0 */ +uint16_t value_length;/* length of entry value, > 0 */ +} VHDXParentLocatorEntry; + + +/* ----- END VHDX SPECIFICATION STRUCTURES ---- */ + +#endif
Signature · 1 line
1.8.1.4
PATCH v3 3/4 block: initial VHDX driver support framework - supports open and probe
In reply to Jeff CodyThis is the initial block driver framework for VHDX image support (i.e. Hyper-V image file formats), that supports opening VHDX files, and parsing the headers.
This commit does not yet enable:
- reading
- writing
- updating the header
- differencing files (images with parents)
- log replay / dirty logs (only clean images)
This is based on Microsoft's VHDX specification:
"VHDX Format Specification v0.95", published 4/12/2012
https://www.microsoft.com/en-us/download/details.aspx?id=29681--- block/Makefile.objs | 1 + block/vhdx.c | 853 ++++++++++++++++++++++++++++++++++++++++++++++++++++ block/vhdx.h | 14 + 3 files changed, 868 insertions(+) create mode 100644 block/vhdx.c
block/Makefile.objs +1 -0
@@ -2,6 +2,7 @@ block-obj-y += raw.o cow.o qcow.o vdi.o vmdk.o cloop.o dmg.o bochs.o vpc.o vvfat block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o block-obj-y += qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o block-obj-y += qed-check.o +block-obj-y += vhdx.o block-obj-y += parallels.o blkdebug.o blkverify.o block-obj-$(CONFIG_WIN32) += raw-win32.o win32-aio.o block-obj-$(CONFIG_POSIX) += raw-posix.o
block/vhdx.c new file +853 -0
853 lines not on this page · raw message
block/vhdx.h +14 -0
@@ -308,4 +308,18 @@ typedef struct QEMU_PACKED VHDXParentLocatorEntry {/* ----- END VHDX SPECIFICATION STRUCTURES ---- */ + +uint32_t vhdx_checksum_calc (uint32_t crc,uint8_t *buf,size_t size, +int crc_offset); + +bool vhdx_checksum_is_valid (uint8_t *buf,size_t size,int crc_offset); + + +static void leguid_to_cpus (MSGUID *guid) +{ +le32_to_cpus (&guid->data1); +le16_to_cpus (&guid->data2); +le16_to_cpus (&guid->data3); +} +#endif
Signature · 1 line
1.8.1.4
PATCH v3 4/4 block: add read-only support to VHDX image format.
In reply to Jeff CodyThis adds in read-only support to the VHDX image format. This supports reads for fixed-size, and dynamic sized VHDX images.
Differencing files are still unsupported.
The image must be opened without BDRV_O_RDWR set, because we do not yet update the headers. I.e., pass 'readonly=on' in the drive image options from the QEMU commandline.
--- block/vhdx.c | 123 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 121 insertions(+), 2 deletions(-)
block/vhdx.c +121 -2
@@ -114,6 +114,17 @@ typedef struct VHDXMetadataEntries { } VHDXMetadataEntries; +typedef struct VHDXSectorInfo { +uint32_t bat_idx;/* BAT entry index */ +uint32_t sectors_avail;/* sectors available in payload block */ +uint32_t bytes_left;/* bytes left in the block after data to r/w */ +uint32_t bytes_avail;/* bytes available in payload block */ +uint64_t file_offset;/* absolute offset in bytes, in file */ +uint64_t block_offset;/* block offset, in bytes */ +}VHDXSectorInfo ; + + + typedef struct BDRVVHDXState { CoMutex lock;
@@ -792,7 +803,7 @@ static int vhdx_open(BlockDriverState *bs, QDict *options, int flags) goto fail; } -/* TODO: differencing files, read, write */ +/* TODO: differencing files, write */ return 0 ; fail:
@@ -810,10 +821,118 @@ static int vhdx_reopen_prepare(BDRVReopenState *state, } +/* +* Perform sector to block offset translations, to get various +* sector and file offsets into the image. See VHDXSectorInfo +*/ +static void vhdx_block_translate (BDRVVHDXState *s,int64_t sector_num, +int nb_sectors,VHDXSectorInfo *sinfo) +{ +uint32_t block_offset; + + sinfo->bat_idx = sector_num >> s->sectors_per_block_bits; +/* effectively a modulo - this gives us the offset into the block +* (in sector sizes) for our sector number */ + block_offset = sector_num - (sinfo->bat_idx << s->sectors_per_block_bits); +/* the chunk ratio gives us the interleaving of the sector +* bitmaps, so we need to advance our page block index by the +* sector bitmaps entry number */ + sinfo->bat_idx += sinfo->bat_idx >> s->chunk_ratio_bits; + +/* the number of sectors we can read/write in this cycle */ + sinfo->sectors_avail = s->sectors_per_block - block_offset; + + sinfo->bytes_left = sinfo->sectors_avail << s->logical_sector_size_bits; + +if (sinfo->sectors_avail > nb_sectors) { + sinfo->sectors_avail = nb_sectors; + } + + sinfo->bytes_avail = sinfo->sectors_avail << s->logical_sector_size_bits; + + sinfo->file_offset = s->bat[sinfo->bat_idx] >>VHDX_BAT_FILE_OFF_BITS ; + + sinfo->block_offset = block_offset << s->logical_sector_size_bits; + +/* The file offset must be past the header section, so must be > 0 */ +if (sinfo->file_offset ==0 ) { +return ; + } + +/* block offset is the offset in vhdx logical sectors, in +* the payload data block. Convert that to a byte offset +* in the block, and add in the payload data block offset +* in the file, in bytes, to get the final read address */ + + sinfo->file_offset <<=20 ;/* now in bytes, rather than 1MB units */ + sinfo->file_offset += sinfo->block_offset; +} + + + static coroutine_fn int vhdx_co_readv(BlockDriverState *bs, int64_t sector_num,int nb_sectors,QEMUIOVector *qiov) { -return -ENOTSUP ; +BDRVVHDXState *s = bs->opaque; +int ret =0 ; +VHDXSectorInfo sinfo; +uint64_t bytes_done =0 ; +QEMUIOVector hd_qiov; + +qemu_iovec_init (&hd_qiov, qiov->niov); + +qemu_co_mutex_lock (&s->lock); + +while (nb_sectors >0 ) { +/* We are a differencing file, so we need to inspect the sector bitmap +* to see if we have the data or not */ +if (s->params.data_bits &VHDX_PARAMS_HAS_PARENT ) { +/* not supported yet */ + ret = -ENOTSUP ; + goto exit; + }else { +vhdx_block_translate (s, sector_num, nb_sectors, &sinfo); + +qemu_iovec_reset (&hd_qiov); +qemu_iovec_concat (&hd_qiov, qiov, bytes_done, sinfo.bytes_avail); + +/* check the payload block state */ +switch (s->bat[sinfo.bat_idx] &VHDX_BAT_STATE_BIT_MASK ) { +case PAYLOAD_BLOCK_NOT_PRESENT :/* fall through */ +case PAYLOAD_BLOCK_UNDEFINED :/* fall through */ +case PAYLOAD_BLOCK_UNMAPPED :/* fall through */ +case PAYLOAD_BLOCK_ZERO : +/* return zero */ +qemu_iovec_memset (&hd_qiov,0 ,0 , sinfo.bytes_avail); +break ; +case PAYLOAD_BLOCK_FULL_PRESENT : +qemu_co_mutex_unlock (&s->lock); + ret =bdrv_co_readv (bs->file, + sinfo.file_offset >>BDRV_SECTOR_BITS , + sinfo.sectors_avail, &hd_qiov); +qemu_co_mutex_lock (&s->lock); +if (ret <0 ) { + goto exit; + } +break ; +case PAYLOAD_BLOCK_PARTIALLY_PRESENT : +/* we don't yet support difference files, fall through +* to error */ +default : + ret = -EIO ; + goto exit; +break ; + } + nb_sectors -= sinfo.sectors_avail; + sector_num += sinfo.sectors_avail; + bytes_done += sinfo.bytes_avail; + } + } + ret =0 ; +exit: +qemu_co_mutex_unlock (&s->lock); +qemu_iovec_destroy (&hd_qiov); +return ret; }
Signature · 1 line
1.8.1.4
Stefan Hajnoczi
AppliedQuoted 42 lines · Jeff Cody
Differences from v2:
Patch 2/4: changed 2 uint8_t[16] to MSGUID (Kevin)
renamed all strucs to proper style (Kevin) indentations / style (Kevin) Removed/fixed outdated comments (Stefan) droppedvhdx_header_padded(Kevin)Patch 3/4: removed hunk that was meant for old patch 5/5 (Fam)
check VHDX header version field (Fam)
validate file type id in open and not just probe (Kevin) allow headers with seq # of 0 if other header is invalid (Kevin) fixed masking typo for parent meta present (Kevin) removed error check for 512-byte sectors (Kevin) verify region and metadata table guid are unique (Kevin) removed signed/unsigned comparisons (Stefan)
sanity checking on rt.entry_count (Stefan)
useclz_()instead of inline code (Stefan) bat_rt.length sanity check - compute it rather
than trust the entry length (Stefan)Patch 5 (v2): Dropped
This adds the initial support for VHDX image files.
It currently only supports read operations of VHDX, for fixed and dynamic files.
Notably, the following is not yet supported:
* Differencing files
* Log replay (so we will refuse to open any images that are not 'clean')
* .bdrv_create()
* write operations other than to the headerJeff Cody (4):
qemu: add castagnoli crc32c checksum algorithm
block: vhdx header for the QEMU support of VHDX images
block: initial VHDX driver support framework - supports open and probe
block: add read-only support to VHDX image format.
Quoted 11 lines
block/Makefile.objs | 1 + block/vhdx.c | 972 ++++++++++++++++++++++++++++++++++++++++++++++++++ block/vhdx.h | 325 +++++++++++++++++ include/qemu/crc32c.h | 35 ++ util/Makefile.objs | 1 + util/crc32c.c | 115 ++++++ 6 files changed, 1449 insertions(+) create mode 100644 block/vhdx.c create mode 100644 block/vhdx.h create mode 100644 include/qemu/crc32c.h create mode 100644 util/crc32c.c
Do you have links to vhdx files I can test?
Thanks, applied to my block tree:
https://github.com/stefanha/qemu/commits/block
Stefan
Quoted 55 lines · Stefan Hajnoczi
Quoted 42 lines · Jeff Cody
Differences from v2:
Patch 2/4: changed 2 uint8_t[16] to MSGUID (Kevin)
renamed all strucs to proper style (Kevin) indentations / style (Kevin) Removed/fixed outdated comments (Stefan) droppedvhdx_header_padded(Kevin)Patch 3/4: removed hunk that was meant for old patch 5/5 (Fam)
check VHDX header version field (Fam)
validate file type id in open and not just probe (Kevin) allow headers with seq # of 0 if other header is invalid (Kevin) fixed masking typo for parent meta present (Kevin) removed error check for 512-byte sectors (Kevin) verify region and metadata table guid are unique (Kevin) removed signed/unsigned comparisons (Stefan)
sanity checking on rt.entry_count (Stefan)
useclz_()instead of inline code (Stefan) bat_rt.length sanity check - compute it rather
than trust the entry length (Stefan)Patch 5 (v2): Dropped
This adds the initial support for VHDX image files.
It currently only supports read operations of VHDX, for fixed and dynamic files.
Notably, the following is not yet supported:
* Differencing files
* Log replay (so we will refuse to open any images that are not 'clean')
* .bdrv_create()
* write operations other than to the headerJeff Cody (4):
qemu: add castagnoli crc32c checksum algorithm
block: vhdx header for the QEMU support of VHDX images
block: initial VHDX driver support framework - supports open and probe
block: add read-only support to VHDX image format.Quoted 11 lines
block/Makefile.objs | 1 + block/vhdx.c | 972 ++++++++++++++++++++++++++++++++++++++++++++++++++ block/vhdx.h | 325 +++++++++++++++++ include/qemu/crc32c.h | 35 ++ util/Makefile.objs | 1 + util/crc32c.c | 115 ++++++ 6 files changed, 1449 insertions(+) create mode 100644 block/vhdx.c create mode 100644 block/vhdx.h create mode 100644 include/qemu/crc32c.h create mode 100644 util/crc32c.c
Quoted 2 lines
Do you have links to vhdx files I can test?
I created a new VHDX file, because the ones I've been testing with have grown too large to share easily. It is a 32GB dynamic image that contains a single EXT4 partition:
https://www.dropbox.com/s/52q84zy2n59ix6h/TestDisk32GBDynamic-ext4.vhdx.7z
I'll keep this link around for a while, but it will eventually go away.
Inside the ext4 partition there are 10 image files generated from /dev/urandom, and corresponding SHA1 hashes in a single SHA1SUM file, for verification.
Also, if you wish to check the SHA1SUM of the entire image from the view of the guest (e.g. sha1sum /dev/vdb), the resulting hash should be:
f897e5cadd3a0b1e776625caf79ed0eaf4baa3d8 /dev/vdb
(The above has was calculated from a Linux guest under Hyper-V 2012, and verified with these patches and QEMU).
Quoted 3 lines
Thanks, applied to my block tree:
https://github.com/stefanha/qemu/commits/block
Thanks!
Jeff
Quoted 72 lines · Jeff Cody
Quoted 55 lines · Stefan Hajnoczi
Quoted 42 lines · Jeff Cody
Differences from v2:
Patch 2/4: changed 2 uint8_t[16] to MSGUID (Kevin)
renamed all strucs to proper style (Kevin) indentations / style (Kevin) Removed/fixed outdated comments (Stefan) droppedvhdx_header_padded(Kevin)Patch 3/4: removed hunk that was meant for old patch 5/5 (Fam)
check VHDX header version field (Fam)
validate file type id in open and not just probe (Kevin) allow headers with seq # of 0 if other header is invalid (Kevin) fixed masking typo for parent meta present (Kevin) removed error check for 512-byte sectors (Kevin) verify region and metadata table guid are unique (Kevin) removed signed/unsigned comparisons (Stefan)
sanity checking on rt.entry_count (Stefan)
useclz_()instead of inline code (Stefan) bat_rt.length sanity check - compute it rather
than trust the entry length (Stefan)Patch 5 (v2): Dropped
This adds the initial support for VHDX image files.
It currently only supports read operations of VHDX, for fixed and dynamic files.
Notably, the following is not yet supported:
* Differencing files
* Log replay (so we will refuse to open any images that are not 'clean')
* .bdrv_create()
* write operations other than to the headerJeff Cody (4):
qemu: add castagnoli crc32c checksum algorithm
block: vhdx header for the QEMU support of VHDX images
block: initial VHDX driver support framework - supports open and probe
block: add read-only support to VHDX image format.Quoted 11 lines
block/Makefile.objs | 1 + block/vhdx.c | 972 ++++++++++++++++++++++++++++++++++++++++++++++++++ block/vhdx.h | 325 +++++++++++++++++ include/qemu/crc32c.h | 35 ++ util/Makefile.objs | 1 + util/crc32c.c | 115 ++++++ 6 files changed, 1449 insertions(+) create mode 100644 block/vhdx.c create mode 100644 block/vhdx.h create mode 100644 include/qemu/crc32c.h create mode 100644 util/crc32c.cQuoted 2 lines
Do you have links to vhdx files I can test?
I created a new VHDX file, because the ones I've been testing with have grown too large to share easily. It is a 32GB dynamic image that contains a single EXT4 partition:
https://www.dropbox.com/s/52q84zy2n59ix6h/TestDisk32GBDynamic-ext4.vhdx.7z
I'll keep this link around for a while, but it will eventually go away.
Inside the ext4 partition there are 10 image files generated from
/dev/urandom, and corresponding SHA1 hashes in a single SHA1SUM file, for verification.
Quoted 8 lines
Also, if you wish to check the SHA1SUM of the entire image from the view of the guest (e.g. sha1sum
/dev/vdb), the resulting hash should be:
f897e5cadd3a0b1e776625caf79ed0eaf4baa3d8/dev/vdb(The above has was calculated from a Linux guest under Hyper-V 2012, and verified with these patches and QEMU).
Cool, thanks for making this available!
Stefan