Fake dirty loggin when it's not there
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 11 lines · Anthony Liguori
From: Alexander Graf <[email protected]>
Some KVM platforms don't support dirty logging yet, like IA64 and PPC, so in order to still have screen updates on those, we need to fake it.
This patch just tells the getter function for dirty bitmaps, that all pages within a slot are dirty when the slot has dirty logging enabled.
That way we can implement dirty logging on those platforms sometime when it drags down performance, but share the rest of the code with dirty logging capable platforms.
Sorry to disturb the merge, but my remarks remained unpatched. Would be nice if they are considered in some follow-up patch... :)
Quoted 17 lines
Signed-off-by Alexander Graf <[email protected]>Signed-off-by Anthony Liguori <aliguori-r/[email protected]>kvm-all.c +1
-0@@ -300,6 +300,7 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr,KVMDirtyLog d;KVMSlot *mem;int ret =0 ; +int r; d.dirty_bitmap =NULL ;while (start_addr < end_addr) {
kvm-all.c int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, · line 316
Quoted 8 lines
kvm-all.c +5
-0@@ -308,6 +309,11 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr,break ; } +/* We didn't activate dirty logging? Don't care then. */ +if (!(mem->flags &KVM_MEM_LOG_DIRTY_PAGES )) { +continue ; + } +
According to Alex' reply [1], I think this is probably better expressed as an assert() than a silent skip. It indicates an error at higher level, no?
kvm-all.c int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, · line 332
Quoted 13 lines
kvm-all.c +2
-1size = ((mem->memory_size >>TARGET_PAGE_BITS ) +7 ) /8 ;if (!d.dirty_bitmap) { d.dirty_bitmap =qemu_malloc (size);@@ -319,7 +325,8 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, d.slot = mem->slot; -if (kvm_vm_ioctl (s,KVM_GET_DIRTY_LOG , &d) ==-1 ) { + r =kvm_vm_ioctl (s,KVM_GET_DIRTY_LOG , &d); +if (r == -EINVAL ) {dprintf ("ioctl failed %d\n" , errno); ret =-1 ;break ;
My remark still stands: This is an "optimistic" assumption that only EINVAL is a "real" error. If KVM is buggy /wrt a specific invalid return value, work around that particular bug (and we should also fix the kernel, but that's a different topic).
Jan
[1] http://permalink.gmane.org/gmane.comp.emulators.qemu/47996
Quoted 19 lines · Jan Kiszka
Quoted 19 lines
Signed-off-by Alexander Graf <[email protected]>Signed-off-by Anthony Liguori <aliguori-r/[email protected]>kvm-all.c +1
-0@@ -300,6 +300,7 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr,KVMDirtyLog d;KVMSlot *mem;int ret =0 ; +int r; d.dirty_bitmap =NULL ;while (start_addr < end_addr) {@@ -308,6 +309,11 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr,break ; }
Quoted 11 lines
kvm-all.c int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, · line 317
Quoted 6 lines
kvm-all.c +5
-0+/* We didn't activate dirty logging? Don't care then. */ +if (!(mem->flags &KVM_MEM_LOG_DIRTY_PAGES )) { +continue ; + } +According to Alex' reply [1], I think this is probably better expressed as an
assert()than a silent skip. It indicates an error at higher level, no?
I don't have a strong opinion either way.
Quoted 7 lines
Quoted 7 lines
kvm-all.c +0
-0size = ((mem->memory_size >>TARGET_PAGE_BITS ) +7 ) /8 ;if (!d.dirty_bitmap) {d.dirty_bitmap = qemu_malloc(size);kvm-all.c +0
-0@@ -319,7 +325,8 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, d.slot = mem->slot;
Quoted 13 lines
kvm-all.c int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, · line 332
Quoted 7 lines
kvm-all.c +2
-1-if (kvm_vm_ioctl (s,KVM_GET_DIRTY_LOG , &d) ==-1 ) { + r =kvm_vm_ioctl (s,KVM_GET_DIRTY_LOG , &d); +if (r == -EINVAL ) {dprintf ("ioctl failed %d\n" , errno); ret =-1 ;break ;My remark still stands: This is an "optimistic" assumption that only EINVAL is a "real" error. If KVM is buggy /wrt a specific invalid return value, work around that particular bug (and we should also fix the kernel, but that's a different topic).
The behavior is identical to what it was before. It's just a stylistic change.
I agree that this needs to be properly fixed in KVM and then in QEMU but I also think that that is best done as a separate set of patches.
Regards,
Anthony Liguori
PATCH 1/1 kvm: Fix error detection for KVM_GET_DIRTY_LOG
Quoted 59 lines · Anthony Liguori
Quoted 22 lines · Jan Kiszka
Quoted 22 lines
Signed-off-by Alexander Graf <[email protected]>Signed-off-by Anthony Liguori<aliguori-r/[email protected]>
kvm-all.c +1
-0@@ -300,6 +300,7 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, KVMDirtyLog d;KVMSlot *mem;int ret =0 ; +int r; d.dirty_bitmap =NULL ;while (start_addr < end_addr) {kvm-all.c +0
-0@@ -308,6 +309,11 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, break; }Quoted 11 lines
kvm-all.c int · line 318
Quoted 6 lines
kvm-all.c +5
-0+/* We didn't activate dirty logging? Don't care then. */ +if (!(mem->flags &KVM_MEM_LOG_DIRTY_PAGES )) { +continue ; + } +According to Alex' reply [1], I think this is probably better expressed as an
assert()than a silent skip. It indicates an error at higher level, no?I don't have a strong opinion either way.
Quoted 8 lines
Quoted 8 lines
kvm-all.c +0
-0size = ((mem->memory_size >>TARGET_PAGE_BITS ) +7 ) /8 ;if (!d.dirty_bitmap) { d.dirty_bitmap = qemu_malloc(size);kvm-all.c +0
-0@@ -319,7 +325,8 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, d.slot = mem->slot;Quoted 13 lines
Quoted 7 lines
kvm-all.c +2
-1-if (kvm_vm_ioctl (s,KVM_GET_DIRTY_LOG , &d) ==-1 ) { + r =kvm_vm_ioctl (s,KVM_GET_DIRTY_LOG , &d); +if (r == -EINVAL ) {dprintf ("ioctl failed %d\n" , errno); ret =-1 ;break;My remark still stands: This is an "optimistic" assumption that only EINVAL is a "real" error. If KVM is buggy /wrt a specific invalid return value, work around that particular bug (and we should also fix the kernel, but that's a different topic).
Quoted 2 lines
The behavior is identical to what it was before. It's just a stylistic change.
Not really (EINVAL != 1). But, yes, it's incorrect in similar way. :)
Jan
----------->
---
kvm-all.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-)
kvm-all.c +3 -1
@@ -348,7 +348,9 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, d.slot = mem->slot; r =kvm_vm_ioctl (s,KVM_GET_DIRTY_LOG , &d); -if (r == -EINVAL ) { +/* KVM for PowerPC returns illicit -ENOTSUPP (-524) which we handle +* below. */ +if (r <0 && r !=-524 ) {dprintf ("ioctl failed %d\n" , errno); ret =-1 ;break ;
[Qemu-devel] Re: [PATCH] kvm: Fix error detection for KVM_GET_DIRTY_LOG
Quoted 13 lines · Jan Kiszka
----------->
Signed-off-by Jan Kiszka <[email protected]>---
kvm-all.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-)kvm-all.c +0
-0
kvm-all.c int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, · line 354
Quoted 8 lines
kvm-all.c +3
-1@@ -348,7 +348,9 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, d.slot = mem->slot; r =kvm_vm_ioctl (s,KVM_GET_DIRTY_LOG , &d); -if (r == -EINVAL ) { +/* KVM for PowerPC returns illicit -ENOTSUPP (-524) which we handle +* below. */ +if (r <0 && r !=-524 ) {
That makes me uncomfortable. Shouldn't we make kvm return something that's exposed to userspace?
Signature · 3 lines
Regards, Anthony Liguori
[Qemu-devel] Re: [PATCH] kvm: Fix error detection for KVM_GET_DIRTY_LOG
Quoted 17 lines · Anthony Liguori
Quoted 16 lines · Jan Kiszka
----------->
Signed-off-by Jan Kiszka <[email protected]>---
kvm-all.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-)kvm-all.c +0
-0@@ -348,7 +348,9 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, d.slot = mem->slot;
Quoted 11 lines
kvm-all.c int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, · line 353
Quoted 7 lines
kvm-all.c +1
-1r =kvm_vm_ioctl (s,KVM_GET_DIRTY_LOG , &d); -if (r == -EINVAL ) { +/* KVM for PowerPC returns illicit -ENOTSUPP (-524) which we handle
kvm-all.c +2
-0+* below. */ +if (r <0 && r !=-524 ) {That makes me uncomfortable. Shouldn't we make kvm return something that's exposed to userspace?
Yes, but we can't do this from user space :) (or in other words: there are already kernels out there which return this invalid code).
The situation would only be different if Alex said that it takes further kernel patches anyway to make his PowerPC targets work. Dunno.
Jan
[Qemu-devel] Re: [PATCH] kvm: Fix error detection for KVM_GET_DIRTY_LOG
Quoted 8 lines · Jan Kiszka
Quoted 4 lines
That makes me uncomfortable. Shouldn't we make kvm return something that's exposed to userspace?
Yes, but we can't do this from user space :) (or in other words: there are already kernels out there which return this invalid code).
But since it's not symbolic, at some point in time the meaning of 524 can potentially change and introduce a very, very subtle bug.
Quoted 3 lines
The situation would only be different if Alex said that it takes further kernel patches anyway to make his PowerPC targets work. Dunno.
Certainly, his PPC target is not in any released kernel version so there's time to fix things properly.
Quoted 3 lines
Jan
Signature · 3 lines
Regards, Anthony Liguori
[Qemu-devel] Re: [PATCH] kvm: Fix error detection for KVM_GET_DIRTY_LOG
Quoted 6 lines · Anthony Liguori
Quoted 5 lines · Jan Kiszka
Quoted 4 lines
That makes me uncomfortable. Shouldn't we make kvm return something that's exposed to userspace?
Quoted 6 lines
Quoted 3 lines
Yes, but we can't do this from user space :) (or in other words: there are already kernels out there which return this invalid code).
But since it's not symbolic, at some point in time the meaning of 524 can potentially change and introduce a very, very subtle bug.
I won't change in old kernel versions, and I expect someone from the PowerPC folks to fix it for new version fairly soon.
Quoted 7 lines
Quoted 4 lines
The situation would only be different if Alex said that it takes further kernel patches anyway to make his PowerPC targets work. Dunno.
Certainly, his PPC target is not in any released kernel version so there's time to fix things properly.
Ah, ok. Then let's do this (Alex can carry a temporary workaround locally IMHO):
------------->
---
kvm-all.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
kvm-all.c +1 -1
@@ -348,7 +348,7 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, d.slot = mem->slot; r =kvm_vm_ioctl (s,KVM_GET_DIRTY_LOG , &d); -if (r == -EINVAL ) { +if (r <0 ) {dprintf ("ioctl failed %d\n" , errno); ret =-1 ;break ;
[Qemu-devel] Re: [PATCH] kvm: Fix error detection for KVM_GET_DIRTY_LOG
Am 22.07.2009 um 22:07 schrieb Jan Kiszka <[email protected]>:
Quoted 40 lines
Quoted 6 lines · Anthony Liguori
Quoted 5 lines · Jan Kiszka
Quoted 4 lines
That makes me uncomfortable. Shouldn't we make kvm return something that's exposed to userspace?
Quoted 6 lines
Quoted 3 lines
Yes, but we can't do this from user space :) (or in other words: there are already kernels out there which return this invalid code).
But since it's not symbolic, at some point in time the meaning of 524 can potentially change and introduce a very, very subtle bug.
I won't change in old kernel versions, and I expect someone from the PowerPC folks to fix it for new version fairly soon.
Quoted 7 lines
Quoted 4 lines
The situation would only be different if Alex said that it takes further kernel patches anyway to make his PowerPC targets work. Dunno.
Certainly, his PPC target is not in any released kernel version so there's time to fix things properly.
Ah, ok. Then let's do this (Alex can carry a temporary workaround locally IMHO):
------------->
Signed-off-by Jan Kiszka <[email protected]>---
kvm-all.c| 2 +-1 files changed, 1 insertions(+), 1 deletions(-)kvm-all.c +0
-0
kvm-all.c int kvm_physical_sync_dirty_bitmap · line 352
Quoted 6 lines
kvm-all.c +1
-1@@ -348,7 +348,7 @@ int kvm_physical_sync_dirty_bitmap (target_phys_addr_t start_addr, d.slot = mem->slot; r =kvm_vm_ioctl (s,KVM_GET_DIRTY_LOG , &d); -if (r == -EINVAL ) { +if (r <0 ) {
That's worse as it essentially removes the not implemented case.
The only thing I'd agree with as a good idea instead of the version I sent is an explicit switch statement with an include of errno.h with __KERNEL__ set.
Alex
Quoted 4 lines
kvm-all.c +0
-0dprintf ("ioctl failed %d\n" , errno); ret =-1 ;break;
[Qemu-devel] Re: [PATCH] kvm: Fix error detection for KVM_GET_DIRTY_LOG
Quoted 51 lines · Alexander Graf
Am 22.07.2009 um 22:07 schrieb Jan Kiszka <[email protected]>:
Quoted 45 lines
Quoted 6 lines · Anthony Liguori
Quoted 5 lines · Jan Kiszka
Quoted 4 lines
That makes me uncomfortable. Shouldn't we make kvm return something that's exposed to userspace?
Quoted 6 lines
Quoted 3 lines
Yes, but we can't do this from user space :) (or in other words: there are already kernels out there which return this invalid code).
But since it's not symbolic, at some point in time the meaning of 524 can potentially change and introduce a very, very subtle bug.
I won't change in old kernel versions, and I expect someone from the PowerPC folks to fix it for new version fairly soon.
Quoted 8 lines
Quoted 5 lines
The situation would only be different if Alex said that it takes further
kernel patches anyway to make his PowerPC targets work. Dunno.Certainly, his PPC target is not in any released kernel version so there's time to fix things properly.
Ah, ok. Then let's do this (Alex can carry a temporary workaround locally IMHO):
------------->
Signed-off-by Jan Kiszka <[email protected]>---
kvm-all.c| 2 +-1 files changed, 1 insertions(+), 1 deletions(-)kvm-all.c +0
-0@@ -348,7 +348,7 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, d.slot = mem->slot;
Quoted 5 lines
kvm-all.c int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, · line 351
Quoted 3 lines
kvm-all.c +1
-1r =kvm_vm_ioctl (s,KVM_GET_DIRTY_LOG , &d); -if (r == -EINVAL ) { +if (r <0 ) {That's worse as it essentially removes the not implemented case.
You can add the valid return code of a fixed powerpc version of kvm_vm_ioctl_get_dirty_log here.
Quoted 4 lines
The only thing I'd agree with as a good idea instead of the version I sent is an explicit switch statement with an include of
errno.hwith__KERNEL__set.
Sorry, that's also horrible.
Again the question: Is there already some official, unpatched kernel version which would work with QEMU if we only handle that borken ENOTSUPP case?
If not, I see no point in handling it here. Please fix KVM and provide a patch that checks for the fixed, valid error code.
If there is such a kernel, I see no problem with hard-coding ENOTSUPP's value now, as we are working around a hard-coded bug that will no longer exist in the future (where ENOTSUPP may change its value - theoretically).
Jan
[Qemu-devel] Re: [PATCH] kvm: Fix error detection for KVM_GET_DIRTY_LOG
Am 22.07.2009 um 23:14 schrieb Jan Kiszka <[email protected]>:
Quoted 66 lines
Quoted 51 lines · Alexander Graf
Am 22.07.2009 um 22:07 schrieb Jan Kiszka <[email protected]>:
Quoted 45 lines
Quoted 6 lines · Anthony Liguori
Quoted 5 lines · Jan Kiszka
Quoted 4 lines
That makes me uncomfortable. Shouldn't we make kvm return something that's exposed to userspace?
Quoted 6 lines
Quoted 3 lines
Yes, but we can't do this from user space :) (or in other words: there are already kernels out there which return this invalid code).
But since it's not symbolic, at some point in time the meaning of 524 can potentially change and introduce a very, very subtle bug.
I won't change in old kernel versions, and I expect someone from the PowerPC folks to fix it for new version fairly soon.
Quoted 8 lines
Quoted 5 lines
The situation would only be different if Alex said that it takes further
kernel patches anyway to make his PowerPC targets work. Dunno.Certainly, his PPC target is not in any released kernel version so there's time to fix things properly.
Ah, ok. Then let's do this (Alex can carry a temporary workaround locally IMHO):
------------->
Signed-off-by Jan Kiszka <[email protected]>---
kvm-all.c| 2 +-1 files changed, 1 insertions(+), 1 deletions(-)kvm-all.c +0
-0@@ -348,7 +348,7 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, d.slot = mem->slot;Quoted 5 lines
kvm-all.c int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, · line 351
Quoted 3 lines
kvm-all.c +1
-1r =kvm_vm_ioctl (s,KVM_GET_DIRTY_LOG , &d); -if (r == -EINVAL ) { +if (r <0 ) {That's worse as it essentially removes the not implemented case.
You can add the valid return code of a fixed powerpc version of
kvm_vm_ioctl_get_dirty_loghere.Quoted 4 lines
The only thing I'd agree with as a good idea instead of the version I sent is an explicit switch statement with an include of
errno.hwith__KERNEL__set.
Quoted 5 lines
Sorry, that's also horrible.
Again the question: Is there already some official, unpatched kernel version which would work with QEMU if we only handle that borken ENOTSUPP case?
No, but I haven't implemented dirty logging at all yet.
I guess the real question is what does IA64 do and check for that.
Alex
Quoted 10 lines
If not, I see no point in handling it here. Please fix KVM and provide a patch that checks for the fixed, valid error code.
If there is such a kernel, I see no problem with hard-coding ENOTSUPP's value now, as we are working around a hard-coded bug that will no longer exist in the future (where ENOTSUPP may change its value - theoretically).
Jan
[Qemu-devel] Re: [PATCH] kvm: Fix error detection for KVM_GET_DIRTY_LOG
In reply to Alexander GrafQuoted 2 lines · Alexander Graf
The only thing I'd agree with as a good idea instead of the version I sent is an explicit switch statement with an include of
errno.hwith__KERNEL__set.
No way :-)
What's the problem with fixing the kernel to return a proper errno?
We can introduce a CAP if necessary to gracefully detect support for proper errno return.
Signature · 3 lines
Regards, Anthony Liguori
[Qemu-devel] Re: [PATCH] kvm: Fix error detection for KVM_GET_DIRTY_LOG
Am 22.07.2009 um 23:15 schrieb Anthony Liguori <[email protected]>:
Quoted 7 lines
Quoted 2 lines · Alexander Graf
The only thing I'd agree with as a good idea instead of the version I sent is an explicit switch statement with an include of
errno.hwith__KERNEL__set.No way :-)
What's the problem with fixing the kernel to return a proper errno?
I'd rather do a proper dirty log implementation than that :). But yes, you convinced me. I'll just do dirtylogging for ppc.
Now what about ia64?
Alex
[Qemu-devel] Re: [PATCH] kvm: Fix error detection for KVM_GET_DIRTY_LOG
Quoted 9 lines · Alexander Graf
Am 22.07.2009 um 23:15 schrieb Anthony Liguori <[email protected]>:
Quoted 6 lines
Quoted 4 lines · Alexander Graf
The only thing I'd agree with as a good idea instead of the version I sent is an explicit switch statement with an include of
errno.hwith__KERNEL__set.
Quoted 6 lines
Quoted 3 lines
No way :-)
What's the problem with fixing the kernel to return a proper errno?
I'd rather do a proper dirty log implementation than that :). But yes, you convinced me. I'll just do dirtylogging for ppc.
Puh... :)
Quoted 2 lines
Now what about ia64?
There is a non-empty kvm_vm_ioctl_get_dirty_log() in kvm-ia64.c.
So I think we can revert your patch and just merge the tiny error-check fix, right?
Jan
[Qemu-devel] Re: [PATCH] kvm: Fix error detection for KVM_GET_DIRTY_LOG
In reply to Alexander GrafQuoted 1 line · Alexander Graf
I'd rather do a proper dirty log implementation than that :). But yes, you convinced me. I'll just do dirtylogging for ppc.
Okay, then I'll take Jan's patch that just checks for if (ret < 0).
Quoted 2 lines
Now what about ia64?
Out of sight, out of mind.
I have no idea what they do in qemu-kvm. There is no ia64 support in upstream QEMU.
Signature · 3 lines
Regards, Anthony Liguori
[Qemu-devel] Re: [PATCH] kvm: Fix error detection for KVM_GET_DIRTY_LOG
Quoted 4 lines · Anthony Liguori
Quoted 1 line · Alexander Graf
I'd rather do a proper dirty log implementation than that :). But yes, you convinced me. I'll just do dirtylogging for ppc.
Okay, then I'll take Jan's patch that just checks for if (ret < 0).
Sounds good. If anyone needs it they get fake dirty logging without needing to implement it then.
Quoted 6 lines
Quoted 2 lines
Now what about ia64?
Out of sight, out of mind.
I have no idea what they do in qemu-kvm. There is no ia64 support in upstream QEMU.
*shrug*. I just figured it might be worth considering, since the plan was to get qemu as capable as qemu-kvm, no?
Alex
[Qemu-devel] Re: [PATCH] kvm: Fix error detection for KVM_GET_DIRTY_LOG
Quoted 8 lines · Alexander Graf
Quoted 6 lines
Quoted 2 lines
Now what about ia64?
Out of sight, out of mind.
I have no idea what they do in qemu-kvm. There is no ia64 support in upstream QEMU.
*shrug*. I just figured it might be worth considering, since the plan was to get qemu as capable as qemu-kvm, no?
I do, but there's a lot of work to get ia64 there primarily due to the fact that there is no target support for ia64 in qemu. The upstream implementation is going to end up significantly different than the qemu-kvm implementation.
Quoted 2 lines
Alex
Regards,
Anthony Liguori