Skip to contentSkip to conversation

KVM call agenda for Oct 19

DISCUSSION Juan Quintela 6 participants 13 messages Oct 18, 2010 – Oct 19, 2010 KVM

lore
Share & tools
lore
#0 lore raw

Please send in any agenda items you are interested in covering.

thanks,

Juan.

#1 lore raw
Quoted 2 lines · * Juan Quintela ([email protected]) wrote:

Please send in any agenda items you are interested in covering.

- 0.13.X -stable handoff
- 0.14 planning
- threadlet work
- virtfs proposals

#2 lore raw
Quoted 9 lines · Chris Wright
Quoted 2 lines · * Juan Quintela ([email protected]) wrote:

Please send in any agenda items you are interested in covering.

- 0.13.X -stable handoff
- 0.14 planning
- threadlet work
- virtfs proposals

- Live snapshots
- We were asked to add this feature for external qcow2 images. Will simple approach of fsync + tracking each requested backing file (it can be per vDisk) and re-open the new image would be accepted?
- Integration with FS freeze for consistent guest app snapshot Many apps do not sync their ram state to disk correctly or frequent enough. Physical world backup software calls fs freeze on xfs and VSS for windows to make the backup consistent.
In order to integrated this with live snapshots we need a guest agent to trigger the guest fs freeze.
We can either have qemu communicate with the agent directly through virtio-serial or have a mgmt daemon use virtio-serial to communicate with the guest in addition to QMP messages about the live snapshot state.
Preferences? The first solution complicates qemu while the second complicates mgmt.

#3 lore raw
Quoted 20 lines · Dor Laor
Quoted 9 lines · Chris Wright
Quoted 2 lines · * Juan Quintela ([email protected]) wrote:

Please send in any agenda items you are interested in covering.

- 0.13.X -stable handoff
- 0.14 planning
- threadlet work
- virtfs proposals

- Live snapshots
- We were asked to add this feature for external qcow2 images. Will simple approach of fsync + tracking each requested backing file (it can be per vDisk) and re-open the new image would be accepted?
- Integration with FS freeze for consistent guest app snapshot Many apps do not sync their ram state to disk correctly or frequent enough. Physical world backup software calls fs freeze on xfs and VSS for windows to make the backup consistent.

Quoted 8 lines

In order to integrated this with live snapshots we need a guest agent to trigger the guest fs freeze.
We can either have qemu communicate with the agent directly through virtio-serial or have a mgmt daemon use virtio-serial to communicate with the guest in addition to QMP messages about the live snapshot state.
Preferences? The first solution complicates qemu while the second complicates mgmt.

Third option, make the freeze path management -> qemu -> virtio-blk -> guest kernel -> file systems. The advantage is that it's easy to associate file systems with a block device this way.

Signature · 1 line
error compiling committee.c: too many arguments to function
#4 lore raw
Quoted 30 lines · Avi Kivity
Quoted 20 lines · Dor Laor
Quoted 9 lines · Chris Wright
Quoted 2 lines · * Juan Quintela ([email protected]) wrote:

Please send in any agenda items you are interested in covering.

- 0.13.X -stable handoff
- 0.14 planning
- threadlet work
- virtfs proposals

- Live snapshots
- We were asked to add this feature for external qcow2 images. Will simple approach of fsync + tracking each requested backing file (it can be per vDisk) and re-open the new image would be accepted?
- Integration with FS freeze for consistent guest app snapshot Many apps do not sync their ram state to disk correctly or frequent enough. Physical world backup software calls fs freeze on xfs and VSS for windows to make the backup consistent.

Quoted 8 lines

In order to integrated this with live snapshots we need a guest agent to trigger the guest fs freeze.
We can either have qemu communicate with the agent directly through virtio-serial or have a mgmt daemon use virtio-serial to communicate with the guest in addition to QMP messages about the live snapshot state.
Preferences? The first solution complicates qemu while the second complicates mgmt.

Quoted 3 lines

Third option, make the freeze path management -> qemu -> virtio-blk -> guest kernel -> file systems. The advantage is that it's easy to associate file systems with a block device this way.

OTH the userspace freeze path already exist and now you create another path. What about FS that span over LVM with multiple drives? IDE/SCSI?

#5 lore raw
Quoted 31 lines · Dor Laor
Quoted 30 lines · Avi Kivity
Quoted 28 lines · Dor Laor
Quoted 9 lines · Chris Wright
Quoted 2 lines · * Juan Quintela ([email protected]) wrote:

Please send in any agenda items you are interested in covering.

- 0.13.X -stable handoff
- 0.14 planning
- threadlet work
- virtfs proposals

- Live snapshots
- We were asked to add this feature for external qcow2 images. Will simple approach of fsync + tracking each requested backing file (it can be per vDisk) and re-open the new image would be accepted?
- Integration with FS freeze for consistent guest app snapshot Many apps do not sync their ram state to disk correctly or frequent enough. Physical world backup software calls fs freeze on xfs and VSS for windows to make the backup consistent.
In order to integrated this with live snapshots we need a guest agent to trigger the guest fs freeze.
We can either have qemu communicate with the agent directly through virtio-serial or have a mgmt daemon use virtio-serial to communicate with the guest in addition to QMP messages about the live snapshot state.
Preferences? The first solution complicates qemu while the second complicates mgmt.

Quoted 5 lines
Quoted 3 lines

Third option, make the freeze path management -> qemu -> virtio-blk -> guest kernel -> file systems. The advantage is that it's easy to associate file systems with a block device this way.

OTH the userspace freeze path already exist and now you create another path.

I guess we would still have a userspace daemon; instead of talking to virtio-serial it talks to virtio-blk. So:

management -> qemu -> virtio-blk -> guest driver -> kernel fs
resolver -> daemon -> apps

Yuck.

Quoted 1 line

What about FS that span over LVM with multiple drives? IDE/SCSI?

Good points.

Signature · 1 line
error compiling committee.c: too many arguments to function
#6 lore raw
Quoted 36 lines · Avi Kivity
Quoted 35 lines · Dor Laor
Quoted 33 lines · Avi Kivity
Quoted 20 lines · Dor Laor
Quoted 9 lines · Chris Wright
Quoted 2 lines · * Juan Quintela ([email protected]) wrote:

Please send in any agenda items you are interested in covering.

- 0.13.X -stable handoff
- 0.14 planning
- threadlet work
- virtfs proposals

- Live snapshots
- We were asked to add this feature for external qcow2 images. Will simple approach of fsync + tracking each requested backing file (it can be per vDisk) and re-open the new image would be accepted?
- Integration with FS freeze for consistent guest app snapshot Many apps do not sync their ram state to disk correctly or frequent enough. Physical world backup software calls fs freeze on xfs and VSS for windows to make the backup consistent.

Quoted 8 lines

In order to integrated this with live snapshots we need a guest agent to trigger the guest fs freeze.
We can either have qemu communicate with the agent directly through virtio-serial or have a mgmt daemon use virtio-serial to communicate with the guest in addition to QMP messages about the live snapshot state.
Preferences? The first solution complicates qemu while the second complicates mgmt.

Third option, make the freeze path management -> qemu -> virtio-blk -> guest kernel -> file systems. The advantage is that it's easy to associate file systems with a block device this way.

Quoted 8 lines
Quoted 1 line

OTH the userspace freeze path already exist and now you create another path.

I guess we would still have a userspace daemon; instead of talking to virtio-serial it talks to virtio-blk. So:

management -> qemu -> virtio-blk -> guest driver -> kernel fs
resolver -> daemon -> apps

Yuck.

Yeah, in Windows, I'm pretty sure the freeze API is a userspace concept. Various apps can hook into it to serialize their state.

At the risk of stealing Mike's thunder, we've actually been working on a simple guest agent exactly for this type of task. Mike's planning an RFC for later this week but for those that are interested the repo is at http://repo.or.cz/w/qemu/mdroth.git

Regards,

Anthony Liguori

Quoted 5 lines
Quoted 2 lines

What about FS that span over LVM with multiple drives? IDE/SCSI?

Good points.

#7 lore raw
In reply to Dor Laor
Quoted 11 lines · Dor Laor
Quoted 9 lines · Chris Wright
Quoted 2 lines · * Juan Quintela ([email protected]) wrote:

Please send in any agenda items you are interested in covering.

- 0.13.X -stable handoff
- 0.14 planning
- threadlet work
- virtfs proposals

Quoted 5 lines

- Live snapshots
- We were asked to add this feature for external qcow2 images. Will simple approach of fsync + tracking each requested backing file (it can be per vDisk) and re-open the new image would be accepted?

I had assumed that this would involve:

qemu -hda windows.img

(qemu) snapshot ide0-disk0 snap0.img

1) create snap0.img internally by doing the equivalent of `qemu-img create -f qcow2 -b windows.img snap0.img'
2) bdrv_flush('ide0-disk0')
3) bdrv_open(snap0.img)
4) bdrv_close(windows.img)
5) rename('windows.img', 'windows.img.tmp')
6) rename('snap0.img', 'windows.img')
7) rename('windows.img.tmp', 'snap0.img')

Regards,

Anthony Liguori

Quoted 16 lines

- Integration with FS freeze for consistent guest app snapshot Many apps do not sync their ram state to disk correctly or frequent enough. Physical world backup software calls fs freeze on xfs and VSS for windows to make the backup consistent.
In order to integrated this with live snapshots we need a guest agent to trigger the guest fs freeze.
We can either have qemu communicate with the agent directly through virtio-serial or have a mgmt daemon use virtio-serial to communicate with the guest in addition to QMP messages about the live snapshot state.
Preferences? The first solution complicates qemu while the second complicates mgmt.

Signature · 3 lines
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#8 lore raw
Quoted 7 lines · Anthony Liguori

I had assumed that this would involve:

qemu -hda windows.img

(qemu) snapshot ide0-disk0 snap0.img

Quoted 8 lines

1) create snap0.img internally by doing the equivalent of `qemu-img create -f qcow2 -b windows.img snap0.img'
2) bdrv_flush('ide0-disk0')
3) bdrv_open(snap0.img)
4) bdrv_close(windows.img)
5) rename('windows.img', 'windows.img.tmp')
6) rename('snap0.img', 'windows.img')
7) rename('windows.img.tmp', 'snap0.img')

Looks reasonable.

Would be interesting to look at this as a use case for the threading work. We should eventually be able to create a snapshot without stalling vcpus (stalling I/O of course allowed).

Signature · 1 line
error compiling committee.c: too many arguments to function
#9 lore raw
Quoted 17 lines · Avi Kivity
Quoted 7 lines · Anthony Liguori

I had assumed that this would involve:

qemu -hda windows.img

(qemu) snapshot ide0-disk0 snap0.img

Quoted 8 lines

1) create snap0.img internally by doing the equivalent of `qemu-img create -f qcow2 -b windows.img snap0.img'
2) bdrv_flush('ide0-disk0')
3) bdrv_open(snap0.img)
4) bdrv_close(windows.img)
5) rename('windows.img', 'windows.img.tmp')
6) rename('snap0.img', 'windows.img')
7) rename('windows.img.tmp', 'snap0.img')

Quoted 3 lines

Looks reasonable.

Would be interesting to look at this as a use case for the threading work. We should eventually be able to create a snapshot without stalling vcpus (stalling I/O of course allowed).

If we had another block-level command, like bdrv_aio_freeze(), that queued all pending requests until the given callback completed, it would be very easy to do this entirely asynchronously. For instance:

bdrv_aio_freeze(create_snapshot)

create_snapshot():
bdrv_aio_flush(done_flush)

done_flush():
bdrv_open(...)
bdrv_close(...)
...

Of course, closing a device while it's being frozen is probably a recipe for disaster but you get the idea :-)

Regards,

Anthony Liguori

#10 lore raw
Quoted 35 lines · Anthony Liguori
Quoted 19 lines · Avi Kivity
Quoted 7 lines · Anthony Liguori

I had assumed that this would involve:

qemu -hda windows.img

(qemu) snapshot ide0-disk0 snap0.img

Quoted 9 lines

1) create snap0.img internally by doing the equivalent of `qemu-img create -f qcow2 -b windows.img snap0.img'
2) bdrv_flush('ide0-disk0')
3) bdrv_open(snap0.img)
4) bdrv_close(windows.img)
5) rename('windows.img', 'windows.img.tmp')
6) rename('snap0.img', 'windows.img')
7) rename('windows.img.tmp', 'snap0.img')

Quoted 5 lines

Looks reasonable.

Would be interesting to look at this as a use case for the threading work. We should eventually be able to create a snapshot without stalling vcpus (stalling I/O of course allowed).

If we had another block-level command, like bdrv_aio_freeze(), that queued all pending requests until the given callback completed, it would be very easy to do this entirely asynchronously.  For instance:

bdrv_aio_freeze(create_snapshot)

create_snapshot():
 bdrv_aio_flush(done_flush)

Quoted 7 lines

done_flush():
 bdrv_open(...)
 bdrv_close(...)
 ...

Of course, closing a device while it's being frozen is probably a recipe for disaster but you get the idea :-)

bdrv_aio_freeze() or any mechanism to deal with pending requests in the generic block code would be a good step for future "live" support of other operations like truncate.

Stefan

#11 lore raw
Quoted 3 lines · Stefan Hajnoczi

bdrv_aio_freeze() or any mechanism to deal with pending requests in the generic block code would be a good step for future "live" support of other operations like truncate.

+ logical disk grow, etc.

Signature · 1 line
error compiling committee.c: too many arguments to function
#12 lore raw
In reply to Dor Laor
Quoted 20 lines · Dor Laor
Quoted 9 lines · Chris Wright
Quoted 2 lines · * Juan Quintela ([email protected]) wrote:

Please send in any agenda items you are interested in covering.

- 0.13.X -stable handoff
- 0.14 planning
- threadlet work
- virtfs proposals

- Live snapshots
- We were asked to add this feature for external qcow2 images. Will simple approach of fsync + tracking each requested backing file (it can be per vDisk) and re-open the new image would be accepted?
- Integration with FS freeze for consistent guest app snapshot Many apps do not sync their ram state to disk correctly or frequent enough. Physical world backup software calls fs freeze on xfs and VSS for windows to make the backup consistent.

Quoted 8 lines

In order to integrated this with live snapshots we need a guest agent to trigger the guest fs freeze.
We can either have qemu communicate with the agent directly through virtio-serial or have a mgmt daemon use virtio-serial to communicate with the guest in addition to QMP messages about the live snapshot state.
Preferences? The first solution complicates qemu while the second complicates mgmt.

- usb-ccid (aka external device modules)

We probably won't get to it for today's call, but we should try to queue this topic up for discussion. We have a similar situation with vtpm (existing device model that wants to integrate with QEMU). My position so far has been that we should avoid external device models because of difficulty integrating QEMU features with external device models.

However, I'd like to hear opinions from a wider audience.

Regards,

Anthony Liguori

Quoted 4 lines
Signature · 3 lines
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html