Skip to contentSkip to conversation

Add preliminary KVM support for non-embedded PPC v3

PATCH In review Alexander Graf 6 participants 32 messages Jul 17, 2009 – Jul 22, 2009

lore
Share & tools
lore
#0 lore raw

cover Add preliminary KVM support for non-embedded PPC v3

With KVM support emerging for Book3S (PPC64), let's add support for it in Qemu too, so we have a user of this awesome new infrastructure.

This patchset enabled building of Qemu on KVM, though it does not work completely due to

1) Brokenness of Qemu (PPC32 and PPC64 targets are broken for me atm)
2) Brokenness in Slot management (guest breaks in Linux's PCI init code)

Nevertheless, this is a good starting point for anyone who wants to get involved with KVM on PowerPC!

V1 -> V2

- remove unnecessary code for MP patch
- include SoB

V2 -> V3

- update to new configure
- don't break 440
- relax slot setting
- fake dirty log when it's not available

Alexander Graf (7):
Enable PPC KVM for non-embedded
Set PVR in sregs
Add mp_state to PPC CPU struct
Fix warning in kvm-all.c
Use correct input constant
Set slots more carefully
Fake dirty loggin when it's not there

 configure        |    3 ++-
 kvm-all.c        |   17 ++++++++++++++---
 target-ppc/cpu.h |    4 ++++
 target-ppc/kvm.c |   18 ++++++++++++++++--
 4 files changed, 36 insertions(+), 6 deletions(-)
#1 lore raw

PATCH 1/7 Enable PPC KVM for non-embedded

We now have KVM on PPC64 too and might get it on PPC32 as well, as soon as someone writes it.

So let's enable KVM for PPC32 and PPC64 targets.

---
 configure |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)
configure +2 -1
index 2a6ae40..753061b 100755
@@ -2016,11 +2016,12 @@ case "$target_arch2" in
     fi
 esac
 case "$target_arch2" in
-  i386|x86_64|ppcemb)
+  i386|x86_64|ppcemb|ppc|ppc64)
     # Make sure the target and host cpus are compatible
     if test "$kvm" = "yes" -a "$target_softmmu" = "yes" -a \
       \( "$target_arch2" = "$cpu" -o \
       \( "$target_arch2" = "ppcemb" -a "$cpu" = "ppc" \) -o \
+      \( "$target_arch2" = "ppc64"  -a "$cpu" = "ppc" \) -o \
       \( "$target_arch2" = "x86_64" -a "$cpu" = "i386"   \) -o \
       \( "$target_arch2" = "i386"   -a "$cpu" = "x86_64" \) \) ; then
       echo "CONFIG_KVM=y" >> $config_mak
Signature · 1 line
1.6.0.2
#2 lore raw

PATCH 2/7 Set PVR in sregs

We need to tell the kernel about some initial CPU state we don't have yet, so let's use the "sregs" IOCTL for that and simply put the Processor Version Register in there.

Now the kernel knows which guest CPU to virtualize.

---
 target-ppc/kvm.c |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)
target-ppc/kvm.c +7 -1
index acbb1ab..04bb305 100644
@@ -44,7 +44,13 @@ int kvm_arch_init(KVMState *s, int smp_cpus)
 
 int kvm_arch_init_vcpu(CPUState *cenv)
 {
-    return 0;
+    int ret = 0;
+    struct kvm_sregs sregs;
+
+    sregs.pvr = cenv->spr[SPR_PVR];
+    ret = kvm_vcpu_ioctl(cenv, KVM_SET_SREGS, &sregs);
+
+    return ret;
 }
 
 int kvm_arch_put_registers(CPUState *env)
Signature · 1 line
1.6.0.2
#3 lore raw

PATCH 3/7 Add mp_state to PPC CPU struct

Some generic code tries to access env->mp_state. Let's just make it happy and provide it.

---
 target-ppc/cpu.h |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)
target-ppc/cpu.h +4 -0
index 69c1d58..39b0d82 100644
@@ -659,6 +659,10 @@ struct CPUPPCState {
     target_ulong hreset_vector;
 #endif
 
+#ifdef CONFIG_KVM
+    uint32_t mp_state;
+#endif
+
     /* Those resources are used only during code translation */
     /* Next instruction pointer */
     target_ulong nip;
Signature · 1 line
1.6.0.2
#4 lore raw

PATCH 4/7 Fix warning in kvm-all.c

This fixes a warning I stumbled across while compiling qemu on PPC64.

---
 kvm-all.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
kvm-all.c +1 -1
index 8567ac9..961fa32 100644
@@ -226,7 +226,7 @@ static int kvm_dirty_pages_log_change(target_phys_addr_t phys_addr,
     if (mem == NULL)  {
             fprintf(stderr, "BUG: %s: invalid parameters " TARGET_FMT_plx "-"
                     TARGET_FMT_plx "\n", __func__, phys_addr,
-                    phys_addr + size - 1);
+                    (target_phys_addr_t)(phys_addr + size - 1));
             return -EINVAL;
     }
 
Signature · 1 line
1.6.0.2
#5 lore raw

PATCH 5/7 Use correct input constant

440 and desktop codes use different input constants for interrupt indication.

Let's use the respective ones for KVM.

---
 target-ppc/kvm.c |   10 +++++++++-
 1 files changed, 9 insertions(+), 1 deletions(-)
target-ppc/kvm.c +9 -1
index 04bb305..b53d6e9 100644
@@ -124,6 +124,14 @@ int kvm_arch_get_registers(CPUState *env)
     return 0;
 }
 
+#if defined(TARGET_PPCEMB)
+#define PPC_INPUT_INT PPC40x_INPUT_INT
+#elif defined(TARGET_PPC64)
+#define PPC_INPUT_INT PPC970_INPUT_INT
+#else
+#define PPC_INPUT_INT PPC6xx_INPUT_INT
+#endif
+
 int kvm_arch_pre_run(CPUState *env, struct kvm_run *run)
 {
     int r;
@@ -133,7 +141,7 @@ int kvm_arch_pre_run(CPUState *env, struct kvm_run *run)
      * interrupt, reset, etc) in PPC-specific env->irq_input_state. */
     if (run->ready_for_interrupt_injection &&
         (env->interrupt_request & CPU_INTERRUPT_HARD) &&
-        (env->irq_input_state & (1<<PPC40x_INPUT_INT)))
+        (env->irq_input_state & (1<<PPC_INPUT_INT)))
     {
         /* For now KVM disregards the 'irq' argument. However, in the
          * future KVM could cache it in-kernel to avoid a heavyweight exit
Signature · 1 line
1.6.0.2
#6 lore raw

PATCH 6/7 Set slots more carefully

KVM only supports slot sizes of PAGE_SIZE granilarity. On PPC the OS sets the framebuffer to some odd size though, causing the current code to simply abort().

So let's bet graceful here. We can just allocate memory sizes that are of PAGE_SIZE granularity and everything that exceeds that comes in as MMIO and gets handled too - just slower.

---
 kvm-all.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
kvm-all.c +1 -1
index 961fa32..60b76cf 100644
@@ -134,7 +134,7 @@ static int kvm_set_user_memory_region(KVMState *s, KVMSlot *slot)
 
     mem.slot = slot->slot;
     mem.guest_phys_addr = slot->start_addr;
-    mem.memory_size = slot->memory_size;
+    mem.memory_size = slot->memory_size & ~(TARGET_PAGE_SIZE - 1);
     mem.userspace_addr = (unsigned long)qemu_get_ram_ptr(slot->phys_offset);
     mem.flags = slot->flags;
     if (s->migration_log) {
Signature · 1 line
1.6.0.2
#7 lore raw

PATCH 7/7 Fake dirty loggin when it's not there

Some KVM platforms don't support dirty logging yet, like IA64 and PPC, so in order to still have screen updates on those, we need to fake it.

This patch just tells the getter function for dirty bitmaps, that all pages within a slot are dirty when the slot has dirty logging enabled.

That way we can implement dirty logging on those platforms sometime when it drags down performance, but share the rest of the code with dirty logging capable platforms.

---
 kvm-all.c |   13 ++++++++++++-
 1 files changed, 12 insertions(+), 1 deletions(-)
kvm-all.c +12 -1
index 60b76cf..72b7935 100644
@@ -300,6 +300,7 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr,
     KVMDirtyLog d;
     KVMSlot *mem;
     int ret = 0;
+    int r;
 
     d.dirty_bitmap = NULL;
     while (start_addr < end_addr) {
@@ -308,6 +309,11 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr,
             break;
         }
 
+        /* We didn't activate dirty logging? Don't care then. */
+        if(!(mem->flags & KVM_MEM_LOG_DIRTY_PAGES)) {
+            continue;
+        }
+
         size = ((mem->memory_size >> TARGET_PAGE_BITS) + 7) / 8;
         if (!d.dirty_bitmap) {
             d.dirty_bitmap = qemu_malloc(size);
@@ -319,7 +325,8 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr,
 
         d.slot = mem->slot;
 
-        if (kvm_vm_ioctl(s, KVM_GET_DIRTY_LOG, &d) == -1) {
+        r = kvm_vm_ioctl(s, KVM_GET_DIRTY_LOG, &d);
+        if (r == -EINVAL) {
             dprintf("ioctl failed %d\n", errno);
             ret = -1;
             break;
@@ -335,6 +342,10 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr,
 
             if ((bitmap[word] >> bit) & 1) {
                 cpu_physical_memory_set_dirty(addr);
+            } else if (r < 0) {
+                /* When our KVM implementation doesn't know about dirty logging
+                 * we can just assume it's always dirty and be fine. */
+                cpu_physical_memory_set_dirty(addr);
             }
         }
         start_addr = phys_addr;
Signature · 1 line
1.6.0.2
#8 lore raw

Re: [Qemu-devel] [PATCH 7/7] Fake dirty loggin when it's not there

Quoted 9 lines · Alexander Graf

Some KVM platforms don't support dirty logging yet, like IA64 and PPC, so in order to still have screen updates on those, we need to fake it.

This patch just tells the getter function for dirty bitmaps, that all pages within a slot are dirty when the slot has dirty logging enabled.

That way we can implement dirty logging on those platforms sometime when it drags down performance, but share the rest of the code with dirty logging capable platforms.

Isn't it going to be very very slow?
I think a memcmp based approach as a fallback would be a better idea, and if we could make this generic it would be even better :)

#9 lore raw

Re: [Qemu-devel] [PATCH 7/7] Fake dirty loggin when it's not there

Quoted 7 lines · Stefano Stabellini
Quoted 6 lines · Alexander Graf

Some KVM platforms don't support dirty logging yet, like IA64 and PPC, so in order to still have screen updates on those, we need to fake it.

This patch just tells the getter function for dirty bitmaps, that all pages within a slot are dirty when the slot has dirty logging enabled.

Quoted 7 lines
Quoted 3 lines

That way we can implement dirty logging on those platforms sometime when it drags down performance, but share the rest of the code with dirty logging capable platforms.

Isn't it going to be very very slow?
I think a memcmp based approach as a fallback would be a better idea, and if we could make this generic it would be even better :)

I just ran a guest with it and it's not really bad.
It's really only about setting the dirty flag. The memory is written to directly because it's a slot.

Alex

#10 lore raw

Re: [Qemu-devel] [PATCH 7/7] Fake dirty loggin when it's not there

In reply to Stefano Stabellini
Quoted 7 lines · Stefano Stabellini
Quoted 6 lines · Alexander Graf

Some KVM platforms don't support dirty logging yet, like IA64 and PPC, so in order to still have screen updates on those, we need to fake it.

This patch just tells the getter function for dirty bitmaps, that all pages within a slot are dirty when the slot has dirty logging enabled.

Quoted 7 lines
Quoted 3 lines

That way we can implement dirty logging on those platforms sometime when it drags down performance, but share the rest of the code with dirty logging capable platforms.

Isn't it going to be very very slow?
I think a memcmp based approach as a fallback would be a better idea, and if we could make this generic it would be even better :)

Oh memcmp not memcpy - sorry I misread that.

Yeah, one could do memcmp there. I don't know if it's worth it though. At least the VNC target does a dirty verification itself again anyways.

Alex

#11 lore raw

Re: [Qemu-devel] [PATCH 7/7] Fake dirty loggin when it's not there

Quoted 4 lines · Alexander Graf

Oh memcmp not memcpy - sorry I misread that.

Yeah, one could do memcmp there. I don't know if it's worth it though. At least the VNC target does a dirty verification itself again anyways.

Yes, but it does that once per client connected, I wouldn't want to rely on that...

#12 lore raw

Re: [Qemu-devel] [PATCH 7/7] Fake dirty loggin when it's not there

In reply to Alexander Graf
Quoted 14 lines · Alexander Graf
Quoted 11 lines · Stefano Stabellini
Quoted 9 lines · Alexander Graf

Some KVM platforms don't support dirty logging yet, like IA64 and PPC, so in order to still have screen updates on those, we need to fake it.

This patch just tells the getter function for dirty bitmaps, that all pages within a slot are dirty when the slot has dirty logging enabled.

That way we can implement dirty logging on those platforms sometime when it drags down performance, but share the rest of the code with dirty logging capable platforms.

Quoted 7 lines
Quoted 3 lines

Isn't it going to be very very slow?
I think a memcmp based approach as a fallback would be a better idea, and if we could make this generic it would be even better :)

Oh memcmp not memcpy - sorry I misread that.

Yeah, one could do memcmp there. I don't know if it's worth it though. At least the VNC target does a dirty verification itself again anyways.

Thinking about this again, you'd have to copy it somewhere before. So on every dirty check you'd have to copy all memory off to "somewhere" to be able to check for it again later.

Sounds feasible, but is it really worth the effort? Can Xen do dirty logging?

Alex

#13 lore raw

Re: [Qemu-devel] [PATCH 7/7] Fake dirty loggin when it's not there

Quoted 6 lines · Alexander Graf

Thinking about this again, you'd have to copy it somewhere before. So on every dirty check you'd have to copy all memory off to "somewhere" to be able to check for it again later.

Sounds feasible, but is it really worth the effort? Can Xen do dirty logging?

Sure it can now, but it used to have a memcmp based mechanism before, and it had decent performances, while setting everything as dirty every time it is going to be a killer (at least if you do this under xen it is).

Besides I don't know how the dirty tracking is implemented in kvm, but it is not so good from a performance point of view in the HAP case, because you have to set the videoram as read only and catch every write (this how we do it at least) so it would be interesting to compare the two approaches.

#14 lore raw

Re: [Qemu-devel] [PATCH 7/7] Fake dirty loggin when it's not there

Quoted 11 lines · Stefano Stabellini
Quoted 6 lines · Alexander Graf

Thinking about this again, you'd have to copy it somewhere before. So on every dirty check you'd have to copy all memory off to "somewhere" to be able to check for it again later.

Sounds feasible, but is it really worth the effort? Can Xen do dirty logging?

Sure it can now, but it used to have a memcmp based mechanism before, and it had decent performances, while setting everything as dirty every time it is going to be a killer (at least if you do this under xen it is).

Hum, strage. For now the PPC target has bigger (performance) issues than dirty logging though :-).

Quoted 5 lines

Besides I don't know how the dirty tracking is implemented in kvm, but it is not so good from a performance point of view in the HAP case, because you have to set the videoram as read only and catch every write (this how we do it at least) so it would be interesting to compare the two approaches.

The current implementation does the same. Videoram is set r/o and on first write the log gets filled.

I was thinking of implementing a lazy dirty log that would check the pagetable's dirty bits when the dirty log is requested. That way we might save some time. We'd need to invalidate the entries in the current scheme anyways, so why not just read dirty and set it to 0?

On PPC this gets even more complicated, because nobody guarantees that the page you mapped is still in the "pagetable". So it might as well only live in the TLB or not at all anymore.

But then again I need to get dirty logging implemented for PPC at all first :-)

Alex

#15 lore raw

[Qemu-devel] Re: [PATCH 7/7] Fake dirty loggin when it's not there

In reply to Alexander Graf
Quoted 28 lines · Alexander Graf

Some KVM platforms don't support dirty logging yet, like IA64 and PPC, so in order to still have screen updates on those, we need to fake it.

This patch just tells the getter function for dirty bitmaps, that all pages within a slot are dirty when the slot has dirty logging enabled.

That way we can implement dirty logging on those platforms sometime when it drags down performance, but share the rest of the code with dirty logging capable platforms.

---
 kvm-all.c |   13 ++++++++++++-
 1 files changed, 12 insertions(+), 1 deletions(-)
kvm-all.c +1 -0
index 60b76cf..72b7935 100644
@@ -300,6 +300,7 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr,
     KVMDirtyLog d;
     KVMSlot *mem;
     int ret = 0;
+    int r;
 
     d.dirty_bitmap = NULL;
     while (start_addr < end_addr) {

kvm-all.c int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, · line 316

Quoted 8 lines
kvm-all.c +5 -0
@@ -308,6 +309,11 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr,
             break;
         }
 
+        /* We didn't activate dirty logging? Don't care then. */
+        if(!(mem->flags & KVM_MEM_LOG_DIRTY_PAGES)) {
+            continue;
+        }
+

Isn't this worth a separate patch? Do we already run into this patch on dirty-logging archs (ie. x86)? I'm still wondering why it's needed.

kvm-all.c int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, · line 329

Quoted 10 lines
kvm-all.c +2 -1
         size = ((mem->memory_size >> TARGET_PAGE_BITS) + 7) / 8;
         if (!d.dirty_bitmap) {
             d.dirty_bitmap = qemu_malloc(size);
@@ -319,7 +325,8 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr,
 
         d.slot = mem->slot;
 
-        if (kvm_vm_ioctl(s, KVM_GET_DIRTY_LOG, &d) == -1) {
+        r = kvm_vm_ioctl(s, KVM_GET_DIRTY_LOG, &d);
+        if (r == -EINVAL) {

I would say, better exclude 0 and the error you for missing support here than to assume only EINVAL is a "real" error.

Quoted 4 lines
kvm-all.c +0 -0
             dprintf("ioctl failed %d\n", errno);
             ret = -1;
             break;

kvm-all.c int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, · line 351

Quoted 10 lines
kvm-all.c +4 -0
@@ -335,6 +342,10 @@ int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr,
 
             if ((bitmap[word] >> bit) & 1) {
                 cpu_physical_memory_set_dirty(addr);
+            } else if (r < 0) {
+                /* When our KVM implementation doesn't know about dirty logging
+                 * we can just assume it's always dirty and be fine. */
+                cpu_physical_memory_set_dirty(addr);
             }
         }
         start_addr = phys_addr;

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
#16 lore raw

[Qemu-devel] Re: [PATCH 7/7] Fake dirty loggin when it's not there

Quoted 34 lines · Jan Kiszka
Quoted 33 lines · Alexander Graf

Some KVM platforms don't support dirty logging yet, like IA64 and PPC, so in order to still have screen updates on those, we need to fake it.

This patch just tells the getter function for dirty bitmaps, that all pages within a slot are dirty when the slot has dirty logging enabled.

That way we can implement dirty logging on those platforms sometime when it drags down performance, but share the rest of the code with dirty logging capable platforms.

---
kvm-all.c | 13 ++++++++++++-

1 files changed, 12 insertions(+), 1 deletions(-)
kvm-all.c +1 -0
index 60b76cf..72b7935 100644
@@ -300,6 +300,7 @@ int  
 kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr,
    KVMDirtyLog d;
    KVMSlot *mem;
    int ret = 0;
+    int r;
 
    d.dirty_bitmap = NULL;
    while (start_addr < end_addr) {
kvm-all.c +0 -0
@@ -308,6 +309,11 @@ int  
 kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr,
            break;
        }
 
Quoted 8 lines

kvm-all.c int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, · line 316

Quoted 5 lines
kvm-all.c +5 -0
+        /* We didn't activate dirty logging? Don't care then. */
+        if(!(mem->flags & KVM_MEM_LOG_DIRTY_PAGES)) {
+            continue;
+        }
+

Isn't this worth a separate patch? Do we already run into this patch on dirty-logging archs (ie. x86)? I'm still wondering why it's needed.

We don't. I'm just being paranoid.

Quoted 6 lines
Quoted 6 lines
kvm-all.c +0 -0
        size = ((mem->memory_size >> TARGET_PAGE_BITS) + 7) / 8;
        if (!d.dirty_bitmap) {
            d.dirty_bitmap = qemu_malloc(size);
kvm-all.c +0 -0
@@ -319,7 +325,8 @@ int  
 kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr,
 
Quoted 8 lines

kvm-all.c int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, · line 329

Quoted 5 lines
kvm-all.c +2 -1
        d.slot = mem->slot;
 
-        if (kvm_vm_ioctl(s, KVM_GET_DIRTY_LOG, &d) == -1) {
+        r = kvm_vm_ioctl(s, KVM_GET_DIRTY_LOG, &d);
+        if (r == -EINVAL) {

I would say, better exclude 0 and the error you for missing support here than to assume only EINVAL is a "real" error.

Hum, this was the logic that was there before and it makes sense. -EINVAL is used for errors whole -ENOTSUPP is used for N/A. Unfortunately -ENOTSUPP is only exported when __KERNEL__ is defined.

Alex

#17 lore raw

[Qemu-devel] Re: [PATCH 7/7] Fake dirty loggin when it's not there

Quoted 63 lines · Alexander Graf
Quoted 34 lines · Jan Kiszka
Quoted 33 lines · Alexander Graf

Some KVM platforms don't support dirty logging yet, like IA64 and PPC, so in order to still have screen updates on those, we need to fake it.

This patch just tells the getter function for dirty bitmaps, that all pages within a slot are dirty when the slot has dirty logging enabled.

That way we can implement dirty logging on those platforms sometime when it drags down performance, but share the rest of the code with dirty logging capable platforms.

---
kvm-all.c | 13 ++++++++++++-

1 files changed, 12 insertions(+), 1 deletions(-)
kvm-all.c +1 -0
index 60b76cf..72b7935 100644
@@ -300,6 +300,7 @@ int
 kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr,
    KVMDirtyLog d;
    KVMSlot *mem;
    int ret = 0;
+    int r;
 
    d.dirty_bitmap = NULL;
    while (start_addr < end_addr) {
kvm-all.c +0 -0
@@ -308,6 +309,11 @@ int
 kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr,
            break;
        }
 
Quoted 8 lines

kvm-all.c int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, · line 316

Quoted 5 lines
kvm-all.c +5 -0
+        /* We didn't activate dirty logging? Don't care then. */
+        if(!(mem->flags & KVM_MEM_LOG_DIRTY_PAGES)) {
+            continue;
+        }
+

Isn't this worth a separate patch? Do we already run into this patch on dirty-logging archs (ie. x86)? I'm still wondering why it's needed.

We don't. I'm just being paranoid.

Quoted 6 lines
Quoted 6 lines
kvm-all.c +0 -0
        size = ((mem->memory_size >> TARGET_PAGE_BITS) + 7) / 8;
        if (!d.dirty_bitmap) {
            d.dirty_bitmap = qemu_malloc(size);
kvm-all.c +0 -0
@@ -319,7 +325,8 @@ int
 kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr,
 
Quoted 8 lines

kvm-all.c int kvm_physical_sync_dirty_bitmap(target_phys_addr_t start_addr, · line 329

Quoted 5 lines
kvm-all.c +2 -1
        d.slot = mem->slot;
 
-        if (kvm_vm_ioctl(s, KVM_GET_DIRTY_LOG, &d) == -1) {
+        r = kvm_vm_ioctl(s, KVM_GET_DIRTY_LOG, &d);
+        if (r == -EINVAL) {

I would say, better exclude 0 and the error you for missing support here than to assume only EINVAL is a "real" error.

Quoted 3 lines

Hum, this was the logic that was there before and it makes sense. -EINVAL is used for errors whole -ENOTSUPP is used for N/A. Unfortunately -ENOTSUPP is only exported when __KERNEL__ is defined.

Uuh, kvm returns a private error code to user space? That would be a bug. So we need a check for the to-be-fixed error code and >= 512.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
#18 lore raw

[Qemu-devel] Re: [PATCH 6/7] Set slots more carefully

In reply to Alexander Graf
Quoted 18 lines · Alexander Graf

KVM only supports slot sizes of PAGE_SIZE granilarity. On PPC the OS sets the framebuffer to some odd size though, causing the current code to simply abort().

So let's bet graceful here. We can just allocate memory sizes that are of PAGE_SIZE granularity and everything that exceeds that comes in as MMIO and gets handled too - just slower.

---
 kvm-all.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
kvm-all.c +0 -0
index 961fa32..60b76cf 100644

kvm-all.c static int kvm_set_user_memory_region(KVMState *s, KVMSlot *slot) · line 137

Quoted 5 lines
kvm-all.c +1 -1
@@ -134,7 +134,7 @@ static int kvm_set_user_memory_region(KVMState *s, KVMSlot *slot)
 
     mem.slot = slot->slot;
     mem.guest_phys_addr = slot->start_addr;
-    mem.memory_size = slot->memory_size;
+    mem.memory_size = slot->memory_size & ~(TARGET_PAGE_SIZE - 1);

TARGET_PAGE_MASK? And I bet you want to round up here...

Does the caller use the odd size consistently (i.e. also for dirty log enable/disable)? Otherwise we may run into troubles while looking up that slot as it will be registered in user space via its odd end address. Not sure yet, but maybe it's better to round up at the call sites of kvm_set_user_memory_region.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
#19 lore raw

[Qemu-devel] Re: [PATCH 6/7] Set slots more carefully

Quoted 21 lines · Jan Kiszka
Quoted 20 lines · Alexander Graf

KVM only supports slot sizes of PAGE_SIZE granilarity. On PPC the OS sets the framebuffer to some odd size though, causing the current code to simply abort().

So let's bet graceful here. We can just allocate memory sizes that are of PAGE_SIZE granularity and everything that exceeds that comes in as MMIO and gets handled too - just slower.

---
kvm-all.c | 2 +-

1 files changed, 1 insertions(+), 1 deletions(-)
kvm-all.c +0 -0
index 961fa32..60b76cf 100644
@@ -134,7 +134,7 @@ static int kvm_set_user_memory_region(KVMState  
 *s, KVMSlot *slot)
 
Quoted 6 lines

kvm-all.c static int kvm_set_user_memory_region(KVMState *s, KVMSlot *slot) · line 137

Quoted 4 lines
kvm-all.c +1 -1
    mem.slot = slot->slot;
    mem.guest_phys_addr = slot->start_addr;
-    mem.memory_size = slot->memory_size;
+    mem.memory_size = slot->memory_size & ~(TARGET_PAGE_SIZE - 1);

TARGET_PAGE_MASK? And I bet you want to round up here...

Eh - yeah. Same thing, no?

And no, I don't want to round up. After the memory backed slot could easily be real MMIO space. I had such strange configurations with the ESCC overlapping in the same page as graphic memory once.

Better be safe than sorry.

Quoted 5 lines

Does the caller use the odd size consistently (i.e. also for dirty log enable/disable)? Otherwise we may run into troubles while looking up that slot as it will be registered in user space via its odd end address. Not sure yet, but maybe it's better to round up at the call sites of kvm_set_user_memory_region.

Well if the destroyer does not call with the same region it's totally broken, no?

Alex

#20 lore raw

[Qemu-devel] Re: [PATCH 6/7] Set slots more carefully

Quoted 34 lines · Alexander Graf
Quoted 23 lines · Jan Kiszka
Quoted 22 lines · Alexander Graf

KVM only supports slot sizes of PAGE_SIZE granilarity. On PPC the OS sets the framebuffer to some odd size though, causing the current code to simply abort().

So let's bet graceful here. We can just allocate memory sizes that are of
PAGE_SIZE granularity and everything that exceeds that comes in as MMIO and
gets handled too - just slower.

---
kvm-all.c | 2 +-

1 files changed, 1 insertions(+), 1 deletions(-)
kvm-all.c +0 -0
index 961fa32..60b76cf 100644
@@ -134,7 +134,7 @@ static int kvm_set_user_memory_region(KVMState
 *s, KVMSlot *slot)
 
Quoted 6 lines

kvm-all.c static int kvm_set_user_memory_region(KVMState *s, KVMSlot *slot) · line 137

Quoted 4 lines
kvm-all.c +1 -1
    mem.slot = slot->slot;
    mem.guest_phys_addr = slot->start_addr;
-    mem.memory_size = slot->memory_size;
+    mem.memory_size = slot->memory_size & ~(TARGET_PAGE_SIZE - 1);

TARGET_PAGE_MASK? And I bet you want to round up here...

Quoted 1 line

Eh - yeah. Same thing, no?

Just look at its definition...

Quoted 4 lines

And no, I don't want to round up. After the memory backed slot could easily be real MMIO space. I had such strange configurations with the ESCC overlapping in the same page as graphic memory once.

Overlapping is handled by the kvm layer in user space. For sure, if there is a mapping conflict, we are in trouble (kvm-wise) as the second request overwrites the mapping type of the overlapping page(s).

Quoted 2 lines

Better be safe than sorry.

No, something wrong. If you cut off the odd "overhead", you effectively exclude that page. So either the caller should not include it in the first place as it is unused or kvm should try to map the whole page just like the rest of the slot. If there remains an unresolvable conflict, we need to enhance the slot management with some sub-page dispatching mechanism.

Quoted 9 lines
Quoted 6 lines

Does the caller use the odd size consistently (i.e. also for dirty log enable/disable)? Otherwise we may run into troubles while looking up that slot as it will be registered in user space via its odd end address. Not sure yet, but maybe it's better to round up at the call sites of kvm_set_user_memory_region.

Well if the destroyer does not call with the same region it's totally broken, no?

Yes, but I was more concerned about consistency between registration and (potentially partial) logging updates.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
#21 lore raw

[Qemu-devel] Re: [PATCH 6/7] Set slots more carefully

Quoted 49 lines · Jan Kiszka
Quoted 34 lines · Alexander Graf
Quoted 23 lines · Jan Kiszka
Quoted 22 lines · Alexander Graf

KVM only supports slot sizes of PAGE_SIZE granilarity. On PPC the OS sets the framebuffer to some odd size though, causing the current code to simply abort().

So let's bet graceful here. We can just allocate memory sizes that are of
PAGE_SIZE granularity and everything that exceeds that comes in as MMIO and
gets handled too - just slower.

---
kvm-all.c | 2 +-

1 files changed, 1 insertions(+), 1 deletions(-)
kvm-all.c +0 -0
index 961fa32..60b76cf 100644
@@ -134,7 +134,7 @@ static int kvm_set_user_memory_region(KVMState
 *s, KVMSlot *slot)
 
Quoted 6 lines

kvm-all.c static int kvm_set_user_memory_region(KVMState *s, KVMSlot *slot) · line 137

Quoted 4 lines
kvm-all.c +1 -1
   mem.slot = slot->slot;
   mem.guest_phys_addr = slot->start_addr;
-    mem.memory_size = slot->memory_size;
+    mem.memory_size = slot->memory_size & ~(TARGET_PAGE_SIZE - 1);

TARGET_PAGE_MASK? And I bet you want to round up here...

Quoted 1 line

Eh - yeah. Same thing, no?

Just look at its definition...

Quoted 4 lines

And no, I don't want to round up. After the memory backed slot could easily be real MMIO space. I had such strange configurations with the ESCC overlapping in the same page as graphic memory once.

Overlapping is handled by the kvm layer in user space. For sure, if there is a mapping conflict, we are in trouble (kvm-wise) as the second request overwrites the mapping type of the overlapping page(s).

Quoted 1 line
Quoted 8 lines
Quoted 1 line

Better be safe than sorry.

No, something wrong. If you cut off the odd "overhead", you effectively exclude that page. So either the caller should not include it in the first place as it is unused or kvm should try to map the whole page just like the rest of the slot. If there remains an unresolvable conflict, we need to enhance the slot management with some sub-page dispatching mechanism.

Not mapped = MMIO

MMIO then gets passed to userspace which puts it into qemu's read/write functions which then decide if it's real MMIO or just a plain RAM access.

So all we get is a couple more MMIO accesses, right?

Alex

#22 lore raw

[Qemu-devel] Re: [PATCH 6/7] Set slots more carefully

Quoted 54 lines · Alexander Graf
Quoted 51 lines · Jan Kiszka
Quoted 34 lines · Alexander Graf
Quoted 23 lines · Jan Kiszka
Quoted 22 lines · Alexander Graf

KVM only supports slot sizes of PAGE_SIZE granilarity. On PPC the OS sets the framebuffer to some odd size though, causing the current code to simply abort().

So let's bet graceful here. We can just allocate memory sizes that are of
PAGE_SIZE granularity and everything that exceeds that comes in as MMIO and
gets handled too - just slower.

---
kvm-all.c | 2 +-

1 files changed, 1 insertions(+), 1 deletions(-)
kvm-all.c +0 -0
index 961fa32..60b76cf 100644
@@ -134,7 +134,7 @@ static int kvm_set_user_memory_region(KVMState
 *s, KVMSlot *slot)
 
Quoted 6 lines

kvm-all.c static int kvm_set_user_memory_region(KVMState *s, KVMSlot *slot) · line 137

Quoted 4 lines
kvm-all.c +1 -1
   mem.slot = slot->slot;
   mem.guest_phys_addr = slot->start_addr;
-    mem.memory_size = slot->memory_size;
+    mem.memory_size = slot->memory_size & ~(TARGET_PAGE_SIZE - 1);

TARGET_PAGE_MASK? And I bet you want to round up here...

Quoted 1 line

Eh - yeah. Same thing, no?

Just look at its definition...

Quoted 4 lines

And no, I don't want to round up. After the memory backed slot could easily be real MMIO space. I had such strange configurations with the ESCC overlapping in the same page as graphic memory once.

Overlapping is handled by the kvm layer in user space. For sure, if there is a mapping conflict, we are in trouble (kvm-wise) as the second request overwrites the mapping type of the overlapping page(s).

Quoted 2 lines

Better be safe than sorry.

Quoted 8 lines
Quoted 6 lines

No, something wrong. If you cut off the odd "overhead", you effectively exclude that page. So either the caller should not include it in the first place as it is unused or kvm should try to map the whole page just like the rest of the slot. If there remains an unresolvable conflict, we need to enhance the slot management with some sub-page dispatching mechanism.

Not mapped = MMIO

Right.

Quoted 5 lines

MMIO then gets passed to userspace which puts it into qemu's read/write functions which then decide if it's real MMIO or just a plain RAM access.

So all we get is a couple more MMIO accesses, right?

In this particular case. But it only happens to work due to the given setup. You can't derive a generic rounding rule for KVM slots from this.

You may say if slot A is MMIO and slot B RAM, that a overlapping page should be handled as MMIO. But that's not what your code does.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
#23 lore raw

Re: [Qemu-devel] [PATCH 6/7] Set slots more carefully

In reply to Alexander Graf
Quoted 4 lines · Alexander Graf

KVM only supports slot sizes of PAGE_SIZE granilarity. On PPC the OS sets the framebuffer to some odd size though, causing the current code to simply abort().

Why is the caller registering a memory region that isn't PAGE_SIZE? Is it relying on subpage memory registration?

If so, this isn't something we handle in KVM today.

If the PPC framebuffer is just being registered with the wrong size, I'd rather see that fixed.

Regards,

Anthony Liguori

#25 lore raw

Re: [Qemu-devel] [PATCH 3/7] Add mp_state to PPC CPU struct

Quoted 9 lines · Anthony Liguori
Quoted 5 lines · Alexander Graf

Some generic code tries to access env->mp_state. Let's just make it happy and provide it.

Or we can move that code from kvm-all.c to target-i386/kvm.c where it belongs :-)

Common code uses mp_state, and i386 is the only thing calling that code. However, the ioctl used by that code is valid for all architectures; it will just return -EINVAL. Someday we might need it in common code, but then again it's not even clear to me it will be useful outside of x86.

I'm fine with moving it too, as long as it's also renamed at the same time. :)

Signature · 2 lines
Hollis Blanchard
IBM Linux Technology Center
#26 lore raw

Re: [Qemu-devel] [PATCH 3/7] Add mp_state to PPC CPU struct

Quoted 13 lines · Hollis Blanchard
Quoted 10 lines · Anthony Liguori
Quoted 6 lines · Alexander Graf

Some generic code tries to access env->mp_state. Let's just make it happy and provide it.

Or we can move that code from kvm-all.c to target-i386/kvm.c where it belongs :-)

Quoted 8 lines

Common code uses mp_state, and i386 is the only thing calling that code. However, the ioctl used by that code is valid for all architectures; it will just return -EINVAL. Someday we might need it in common code, but then again it's not even clear to me it will be useful outside of x86.

I'm fine with moving it too, as long as it's also renamed at the same time. :)

Why is it a common ioctl instead of an arch specific ioctl?

Regards,

Anthony Liguori

#27 lore raw

Re: [Qemu-devel] [PATCH 3/7] Add mp_state to PPC CPU struct

Quoted 20 lines · Anthony Liguori
Quoted 19 lines · Hollis Blanchard
Quoted 11 lines · Anthony Liguori
Quoted 6 lines · Alexander Graf

Some generic code tries to access env->mp_state. Let's just make it happy and provide it.

Or we can move that code from kvm-all.c to target-i386/kvm.c where it belongs :-)

Common code uses mp_state, and i386 is the only thing calling that code. However, the ioctl used by that code is valid for all architectures; it will just return -EINVAL. Someday we might need it in common code, but then again it's not even clear to me it will be useful outside of x86.

Quoted 5 lines
Quoted 3 lines

I'm fine with moving it too, as long as it's also renamed at the same time. :)

Why is it a common ioctl instead of an arch specific ioctl?

I can only speculate: Marcelo probably thought other architectures may have something useful to put there.

Signature · 2 lines
Hollis Blanchard
IBM Linux Technology Center
#28 lore raw

[Qemu-devel] Re: [PATCH 3/7] Add mp_state to PPC CPU struct

Quoted 20 lines · Hollis Blanchard
Quoted 19 lines · Anthony Liguori
Quoted 18 lines · Hollis Blanchard
Quoted 11 lines · Anthony Liguori
Quoted 6 lines · Alexander Graf

Some generic code tries to access env->mp_state. Let's just make it happy and provide it.

Or we can move that code from kvm-all.c to target-i386/kvm.c where it belongs :-)

Common code uses mp_state, and i386 is the only thing calling that code. However, the ioctl used by that code is valid for all architectures; it will just return -EINVAL. Someday we might need it in common code, but then again it's not even clear to me it will be useful outside of x86.

Quoted 8 lines
Quoted 4 lines
Quoted 3 lines

I'm fine with moving it too, as long as it's also renamed at the same time. :)

Why is it a common ioctl instead of an arch specific ioctl?

I can only speculate: Marcelo probably thought other architectures may have something useful to put there.

mp_state is also used by (yet unsupported) ia64. So I think it's better to keep the related bits in the arch-independent part.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
#29 lore raw

[Qemu-devel] Re: [PATCH 3/7] Add mp_state to PPC CPU struct

Quoted 3 lines · Jan Kiszka

mp_state is also used by (yet unsupported) ia64. So I think it's better to keep the related bits in the arch-independent part.

Then it should go in CPU_COMMON instead of CPUPPCState

Regards,

Anthony Liguori

Quoted 3 lines

Jan

#30 lore raw

[Qemu-devel] Re: [PATCH 3/7] Add mp_state to PPC CPU struct

Quoted 6 lines · Anthony Liguori
Quoted 3 lines · Jan Kiszka

mp_state is also used by (yet unsupported) ia64. So I think it's better to keep the related bits in the arch-independent part.

Then it should go in CPU_COMMON instead of CPUPPCState

That's true. Probably I looked too long at qemu-kvm while porting this. :)

Jan

Acked-by Acked-by Acked-by Acked-by on the v1 series
#31 lore raw
In reply to Alexander Graf
Quoted 2 lines · Alexander Graf

Alexander Graf (7):
Enable PPC KVM for non-embedded

Quoted 1 line

Set PVR in sregs

Quoted 1 line

Add mp_state to PPC CPU struct

Quoted 1 line

Fix warning in kvm-all.c

No comment.

Quoted 1 line

Use correct input constant

Quoted 1 line

Set slots more carefully

No comment.

Quoted 1 line

Fake dirty loggin when it's not there

No comment.

Signature · 2 lines
Hollis Blanchard
IBM Linux Technology Center