Skip to contentSkip to conversation

QEMU online guest disk resize wrt host block devices

Qemu-devel Superseded Daniel P. Berrange 3 participants 10 messages Sep 1, 2011 – Sep 9, 2011 Block layer

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

We want to support online resize of guest disks in the libvirt API for QEMU, and had intended to use the fairly recently added 'block_resize' command:

commit db97ee6a976bacbb0d18818e951cfc41b39269a7
Author: Christoph Hellwig <[email protected]>
Date: Mon Jan 24 13:32:41 2011 +0100

block: tell drivers about an image resize

commit 6d4a2b3a47959f02e7f307f50396e70e8464f95e
Author: Christoph Hellwig <[email protected]>
Date: Mon Jan 24 13:32:33 2011 +0100

block: add block_resize monitor command

There is unfortunately a problem with the design of this monitor command because it has tied the task of resizing the host backing file for the virtual disk, together with the task of notifying the guest device of the block resize. This means that we can only use 'block_resize' for disks that are backed by image files (raw, qcow2, etc) that QEMU can actually resize.

To be able to support online resize of virtual disks backed by host block devices, LVM volumes, SCSI LUNs, iSCSI LUNs, etc we need to be able to decouple these tasks. The resize of the block device needs to be done outside QEMU, either by privileged tools on the host, or by a remote SAN administrator, or a combo of both.

IOW, we need a way to tell QEMU to merely re-read the host file/block size to detect an externally performed resize, and then just notify the guest. While this would also be sufficient for raw file images, obviously we don't want todo this with any of the magic formats where QEMU has to remain in charge of updating metadata while active.

I see two likely approaches:

1. Add a parameter to the existing 'block_resize' command 'refreshonly=true|false'

2. Add a separate command 'block_refresh'

Both of these options would internally just re-read the capacity of the BlockDriver image, and then notify the guest device.

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#1 lore raw
Quoted 6 lines · Daniel P. Berrange

I see two likely approaches:

1. Add a parameter to the existing 'block_resize' command 'refreshonly=true|false'

2. Add a separate command 'block_refresh'

I think all that you need is a to implement a bdrv_truncate method for host devices that simply checks if the new size smaller or equal to the device size, and return 0 in that case or an error otherwise.

#7 lore raw
In reply to Daniel P. Berrange
Quoted 14 lines · Daniel P. Berrange

We want to support online resize of guest disks in the libvirt API for QEMU, and had intended to use the fairly recently added 'block_resize' command:

commit db97ee6a976bacbb0d18818e951cfc41b39269a7
Author: Christoph Hellwig <[email protected]>
Date: Mon Jan 24 13:32:41 2011 +0100

block: tell drivers about an image resize

commit 6d4a2b3a47959f02e7f307f50396e70e8464f95e
Author: Christoph Hellwig <[email protected]>
Date: Mon Jan 24 13:32:33 2011 +0100

Quoted 8 lines

block: add block_resize monitor command

There is unfortunately a problem with the design of this monitor command because it has tied the task of resizing the host backing file for the virtual disk, together with the task of notifying the guest device of the block resize. This means that we can only use 'block_resize' for disks that are backed by image files (raw, qcow2, etc) that QEMU can actually resize.

One other question too, when creating a qcow2 image via 'qemu-img create' you can specify a 'prealloc' option to require metadata to be allocated at time of creation.

Should we have the same control at time of resize too. If the app had originally created the qcow2 image with preallocated metadata, then I'd expect they want to pre-allocate metadata when extending it too, or is there no additional metadata allocation required when extending an image ?

Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#2 lore raw
In reply to Christoph Hellwig
Quoted 11 lines · Christoph Hellwig
Quoted 6 lines · Daniel P. Berrange

I see two likely approaches:

1. Add a parameter to the existing 'block_resize' command 'refreshonly=true|false'

2. Add a separate command 'block_refresh'

I think all that you need is a to implement a bdrv_truncate method for host devices that simply checks if the new size smaller or equal to the device size, and return 0 in that case or an error otherwise.

Hmm, yes that is another option. I'll see if I can cook up a patch for doing that.

Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#3 lore raw

Please try the attached (untested) patch:

block/raw-posix.c +20 -3
Index: qemu/block/raw-posix.c
===================================================================
@@ -645,10 +645,23 @@ static void raw_close(BlockDriverState *
 static int raw_truncate(BlockDriverState *bs, int64_t offset)
 {
     BDRVRawState *s = bs->opaque;
-    if (s->type != FTYPE_FILE)
-        return -ENOTSUP;
-    if (ftruncate(s->fd, offset) < 0)
+    struct stat st;
+
+    if (fstat(s->fd, &st))
         return -errno;
+
+    if (S_ISREG(st.st_mode)) {
+        if (ftruncate(s->fd, offset) < 0)
+            return -errno;
+    } else if (S_ISCHR(st.st_mode) || S_ISBLK(st.st_mode)) {
+       if (offset > raw_getlength(bs)) {
+           return -EINVAL;
+       }
+       return 0;
+
+    } else {
+        return -ENOTSUP;
+    }
     return 0;
 }
 
@@ -1167,6 +1180,7 @@ static BlockDriver bdrv_host_device = {
 
     .bdrv_read          = raw_read,
     .bdrv_write         = raw_write,
+    .bdrv_truncate = raw_truncate,
     .bdrv_getlength	= raw_getlength,
     .bdrv_get_allocated_file_size
                         = raw_get_allocated_file_size,
@@ -1288,6 +1302,7 @@ static BlockDriver bdrv_host_floppy = {
 
     .bdrv_read          = raw_read,
     .bdrv_write         = raw_write,
+    .bdrv_truncate = raw_truncate,
     .bdrv_getlength	= raw_getlength,
     .bdrv_get_allocated_file_size
                         = raw_get_allocated_file_size,
@@ -1389,6 +1404,7 @@ static BlockDriver bdrv_host_cdrom = {
 
     .bdrv_read          = raw_read,
     .bdrv_write         = raw_write,
+    .bdrv_truncate = raw_truncate,
     .bdrv_getlength     = raw_getlength,
     .bdrv_get_allocated_file_size
                         = raw_get_allocated_file_size,
@@ -1510,6 +1526,7 @@ static BlockDriver bdrv_host_cdrom = {
 
     .bdrv_read          = raw_read,
     .bdrv_write         = raw_write,
+    .bdrv_truncate = raw_truncate,
     .bdrv_getlength     = raw_getlength,
     .bdrv_get_allocated_file_size
                         = raw_get_allocated_file_size,
#8 lore raw
In reply to Daniel P. Berrange
Quoted 9 lines · Daniel P. Berrange

One other question too, when creating a qcow2 image via 'qemu-img create' you can specify a 'prealloc' option to require metadata to be allocated at time of creation.

Should we have the same control at time of resize too. If the app had originally created the qcow2 image with preallocated metadata, then I'd expect they want to pre-allocate metadata when extending it too, or is there no additional metadata allocation required when extending an image ?

Sounds reasonable. Keving, is there a sane way to implement this?

Quoted 8 lines

Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

---end quoted text---

#4 lore raw
In reply to Christoph Hellwig
Quoted 1 line · Christoph Hellwig

Please try the attached (untested) patch:

Yes, this patch worked succesfully with the following test case:

  $ dd if=/dev/zero of=loop.img bs=1M seek=100 count=0
  $ losetup -f loop.img
  $ ./x86_64-softmmu/qemu-system-x86_64 -hda /dev/loop0 -monitor stdio
  (qemu) info block
  ide0-hd0: removable=0 file=/dev/loop0 sectors=204800 ro=0 drv=raw encrypted=0
  ide1-cd0: removable=1 locked=0 [not inserted]
  floppy0: removable=1 locked=0 [not inserted]
  sd0: removable=1 locked=0 [not inserted]
  $ dd if=/dev/zero of=loop.img bs=1M seek=200 count=0
  $ losetup -c /dev/loop0 

(qemu) block_resize ide0-hd0 200
(qemu) info block
ide0-hd0: removable=0 file=/dev/loop0 sectors=409600 ro=0 drv=raw encrypted=0
ide1-cd0: removable=1 locked=0 [not inserted]
floppy0: removable=1 locked=0 [not inserted]
sd0: removable=1 locked=0 [not inserted]

Regards,
Daniel

Quoted 63 lines

Index: qemu/block/raw-posix.c
===================================================================

block/raw-posix.c +20 -3
@@ -645,10 +645,23 @@ static void raw_close(BlockDriverState *
 static int raw_truncate(BlockDriverState *bs, int64_t offset)
 {
     BDRVRawState *s = bs->opaque;
-    if (s->type != FTYPE_FILE)
-        return -ENOTSUP;
-    if (ftruncate(s->fd, offset) < 0)
+    struct stat st;
+
+    if (fstat(s->fd, &st))
         return -errno;
+
+    if (S_ISREG(st.st_mode)) {
+        if (ftruncate(s->fd, offset) < 0)
+            return -errno;
+    } else if (S_ISCHR(st.st_mode) || S_ISBLK(st.st_mode)) {
+       if (offset > raw_getlength(bs)) {
+           return -EINVAL;
+       }
+       return 0;
+
+    } else {
+        return -ENOTSUP;
+    }
     return 0;
 }
 
@@ -1167,6 +1180,7 @@ static BlockDriver bdrv_host_device = {
 
     .bdrv_read          = raw_read,
     .bdrv_write         = raw_write,
+    .bdrv_truncate = raw_truncate,
     .bdrv_getlength	= raw_getlength,
     .bdrv_get_allocated_file_size
                         = raw_get_allocated_file_size,
@@ -1288,6 +1302,7 @@ static BlockDriver bdrv_host_floppy = {
 
     .bdrv_read          = raw_read,
     .bdrv_write         = raw_write,
+    .bdrv_truncate = raw_truncate,
     .bdrv_getlength	= raw_getlength,
     .bdrv_get_allocated_file_size
                         = raw_get_allocated_file_size,
@@ -1389,6 +1404,7 @@ static BlockDriver bdrv_host_cdrom = {
 
     .bdrv_read          = raw_read,
     .bdrv_write         = raw_write,
+    .bdrv_truncate = raw_truncate,
     .bdrv_getlength     = raw_getlength,
     .bdrv_get_allocated_file_size
                         = raw_get_allocated_file_size,
@@ -1510,6 +1526,7 @@ static BlockDriver bdrv_host_cdrom = {
 
     .bdrv_read          = raw_read,
     .bdrv_write         = raw_write,
+    .bdrv_truncate = raw_truncate,
     .bdrv_getlength     = raw_getlength,
     .bdrv_get_allocated_file_size
                         = raw_get_allocated_file_size,
#9 lore raw
In reply to Christoph Hellwig

Am 01.09.2011 17:56, schrieb Christoph Hellwig:

Quoted 5 lines · Daniel P. Berrange
Quoted 4 lines

One other question too, when creating a qcow2 image via 'qemu-img create' you can specify a 'prealloc' option to require metadata to be allocated at time of creation.

Quoted 7 lines
Quoted 5 lines

Should we have the same control at time of resize too. If the app had originally created the qcow2 image with preallocated metadata, then I'd expect they want to pre-allocate metadata when extending it too, or is there no additional metadata allocation required when extending an image ?

Sounds reasonable. Keving, is there a sane way to implement this?

Implementing the functionality itself shouldn't be a big problem. The big question is what the right interface would look like.

We have driver specific preallocation options, so we would end up with something like this: bdrv_truncate(bs, int64_t size, char* prealloc_mode). Not exactly nice. And is preallocation the only thing or would we need to pass a whole option list like for image creation?

Of course, if this is a real requirement and not only a random thought, we can always introduce a specific flag for the current use case and add the generic thing only later when we have thought a bit more about it.

Kevin

#5 lore raw

PATCH 1/1 block: allow resizing of images residing on host devices

In reply to Christoph Hellwig

Allow to resize images that reside on host devices up to the available space. This allows to grow images after resizing the device manually or vice versa.

block/raw-posix.c +20 -3
Index: qemu/block/raw-posix.c
===================================================================
@@ -645,10 +645,23 @@ static void raw_close(BlockDriverState *
 static int raw_truncate(BlockDriverState *bs, int64_t offset)
 {
     BDRVRawState *s = bs->opaque;
-    if (s->type != FTYPE_FILE)
-        return -ENOTSUP;
-    if (ftruncate(s->fd, offset) < 0)
+    struct stat st;
+
+    if (fstat(s->fd, &st))
         return -errno;
+
+    if (S_ISREG(st.st_mode)) {
+        if (ftruncate(s->fd, offset) < 0)
+            return -errno;
+    } else if (S_ISCHR(st.st_mode) || S_ISBLK(st.st_mode)) {
+       if (offset > raw_getlength(bs)) {
+           return -EINVAL;
+       }
+       return 0;
+
+    } else {
+        return -ENOTSUP;
+    }
     return 0;
 }
 
@@ -1167,6 +1180,7 @@ static BlockDriver bdrv_host_device = {
 
     .bdrv_read          = raw_read,
     .bdrv_write         = raw_write,
+    .bdrv_truncate      = raw_truncate,
     .bdrv_getlength	= raw_getlength,
     .bdrv_get_allocated_file_size
                         = raw_get_allocated_file_size,
@@ -1288,6 +1302,7 @@ static BlockDriver bdrv_host_floppy = {
 
     .bdrv_read          = raw_read,
     .bdrv_write         = raw_write,
+    .bdrv_truncate      = raw_truncate,
     .bdrv_getlength	= raw_getlength,
     .bdrv_get_allocated_file_size
                         = raw_get_allocated_file_size,
@@ -1389,6 +1404,7 @@ static BlockDriver bdrv_host_cdrom = {
 
     .bdrv_read          = raw_read,
     .bdrv_write         = raw_write,
+    .bdrv_truncate      = raw_truncate,
     .bdrv_getlength     = raw_getlength,
     .bdrv_get_allocated_file_size
                         = raw_get_allocated_file_size,
@@ -1510,6 +1526,7 @@ static BlockDriver bdrv_host_cdrom = {
 
     .bdrv_read          = raw_read,
     .bdrv_write         = raw_write,
+    .bdrv_truncate      = raw_truncate,
     .bdrv_getlength     = raw_getlength,
     .bdrv_get_allocated_file_size
                         = raw_get_allocated_file_size,
#6 lore raw

Re: [Qemu-devel] [PATCH] block: allow resizing of images residing on host devices

Am 07.09.2011 13:02, schrieb Christoph Hellwig:

Quoted 5 lines

Allow to resize images that reside on host devices up to the available space. This allows to grow images after resizing the device manually or vice versa.

You mean Signed-off-by, no?

Quoted 5 lines

Index: qemu/block/raw-posix.c
===================================================================

block/raw-posix.c static void raw_close(BlockDriverState * · line 651

Quoted 10 lines
block/raw-posix.c +3 -3
@@ -645,10 +645,23 @@ static void raw_close(BlockDriverState *
 static int raw_truncate(BlockDriverState *bs, int64_t offset)
 {
     BDRVRawState *s = bs->opaque;
-    if (s->type != FTYPE_FILE)
-        return -ENOTSUP;
-    if (ftruncate(s->fd, offset) < 0)
+    struct stat st;
+
+    if (fstat(s->fd, &st))
         return -errno;

Coding style requires braces.

block/raw-posix.c static void raw_close(BlockDriverState * · line 655

Quoted 4 lines
block/raw-posix.c +4 -0
+
+    if (S_ISREG(st.st_mode)) {
+        if (ftruncate(s->fd, offset) < 0)
+            return -errno;

Here as well.

block/raw-posix.c static void raw_close(BlockDriverState * · line 660

Quoted 5 lines
block/raw-posix.c +5 -0
+    } else if (S_ISCHR(st.st_mode) || S_ISBLK(st.st_mode)) {
+       if (offset > raw_getlength(bs)) {
+           return -EINVAL;
+       }
+       return 0;

This return isn't needed and it's not there in the S_ISREG case. It should be symmetrical at least.

Semantically the patch looks okay.

Kevin