Skip to contentSkip to conversation

KVM: PPC: e500: eliminate a trap when entering idle

PATCH In review Scott Wood 2 participants 6 messages Aug 26, 2011 – Sep 19, 2011

Subsystems (2)
lore
Share & tools
lore

Scott Wood

author
#0 lore raw

PATCH 4/5 KVM: PPC: e500: eliminate a trap when entering idle

KVM will respond to setting MSR[WE], no matter what is in HID0.

With KVM we also know it will take effect right away, so no need for a loop, TLF_NAPPING, etc.

KVM paravirt's initcall is moved before SMP init, to avoid any chance of a race when updating the idle loop handler.

---
 arch/powerpc/kernel/kvm.c |   21 ++++++++++++++++++++-
 1 files changed, 20 insertions(+), 1 deletions(-)
arch/powerpc/kernel/kvm.c +20 -1
index eb95a03..5e13500 100644
@@ -30,6 +30,7 @@
 #include <asm/cacheflush.h>
 #include <asm/disassemble.h>
 #include <asm/ppc-opcode.h>
+#include <asm/machdep.h>
 
 #define KVM_MAGIC_PAGE		(-4096L)
 #define magic_var(x) KVM_MAGIC_PAGE + offsetof(struct kvm_vcpu_arch_shared, x)
@@ -774,6 +775,13 @@ static __init void kvm_free_tmp(void)
 	}
 }
 
+#ifdef CONFIG_E500
+static void kvm_msrwe_idle(void)
+{
+	mtmsr(mfmsr() | MSR_WE | MSR_EE);
+}
+#endif
+
 static int __init kvm_guest_init(void)
 {
 	if (!kvm_para_available())
@@ -789,6 +797,17 @@ static int __init kvm_guest_init(void)
 	/* Enable napping */
 	powersave_nap = 1;
 #endif
+#ifdef CONFIG_E500
+	/*
+	 * Skip the overhead of HID0 accesses that KVM ignores --
+	 * just write MSR[WE].
+	 *
+	 * We don't need _TLF_NAPPING, because under KVM we know
+	 * it will take effect right away.
+	 */
+	if (ppc_md.power_save = e500_idle)
+		ppc_md.power_save = kvm_msrwe_idle;
+#endif
 
 free_tmp:
 	kvm_free_tmp();
@@ -796,4 +815,4 @@ free_tmp:
 	return 0;
 }
 
-postcore_initcall(kvm_guest_init);
+early_initcall(kvm_guest_init);
Signature · 1 line
1.7.6
#1 lore raw
Quoted 43 lines · Scott Wood

KVM will respond to setting MSR[WE], no matter what is in HID0.

With KVM we also know it will take effect right away, so no need for a loop, TLF_NAPPING, etc.

KVM paravirt's initcall is moved before SMP init, to avoid any chance of a race when updating the idle loop handler.

---
arch/powerpc/kernel/kvm.c | 21 ++++++++++++++++++++-

1 files changed, 20 insertions(+), 1 deletions(-)
arch/powerpc/kernel/kvm.c +8 -0
index eb95a03..5e13500 100644
@@ -30,6 +30,7 @@
 #include <asm/cacheflush.h>
 #include <asm/disassemble.h>
 #include <asm/ppc-opcode.h>
+#include <asm/machdep.h>
 
 #define KVM_MAGIC_PAGE		(-4096L)
 #define magic_var(x) KVM_MAGIC_PAGE + offsetof(struct kvm_vcpu_arch_shared, x)
@@ -774,6 +775,13 @@ static __init void kvm_free_tmp(void)
 	}
 }
 
+#ifdef CONFIG_E500
+static void kvm_msrwe_idle(void)
+{
+	mtmsr(mfmsr() | MSR_WE | MSR_EE);
+}
+#endif
+
 static int __init kvm_guest_init(void)
 {
 	if (!kvm_para_available())
@@ -789,6 +797,17 @@ static int __init kvm_guest_init(void)
 	/* Enable napping */
 	powersave_nap = 1;
 #endif

arch/powerpc/kernel/kvm.c static int __init kvm_guest_init(void) · line 809

Quoted 10 lines
arch/powerpc/kernel/kvm.c +10 -0
+#ifdef CONFIG_E500
+	/*
+	 * Skip the overhead of HID0 accesses that KVM ignores --
+	 * just write MSR[WE].
+	 *
+	 * We don't need _TLF_NAPPING, because under KVM we know
+	 * it will take effect right away.
+	 */
+	if (ppc_md.power_save = e500_idle)
+		ppc_md.power_save = kvm_msrwe_idle;

Why the if() here?

Alex

author
#2 lore raw
In reply to Scott Wood
Quoted 15 lines · Alexander Graf

arch/powerpc/kernel/kvm.c static int __init kvm_guest_init(void) · line 809

Quoted 10 lines · Scott Wood
arch/powerpc/kernel/kvm.c +10 -0
+#ifdef CONFIG_E500
+	/*
+	 * Skip the overhead of HID0 accesses that KVM ignores --
+	 * just write MSR[WE].
+	 *
+	 * We don't need _TLF_NAPPING, because under KVM we know
+	 * it will take effect right away.
+	 */
+	if (ppc_md.power_save = e500_idle)
+		ppc_md.power_save = kvm_msrwe_idle;

Why the if() here?

To avoid replacing some other power_save() implementation. kvm_msrwe_idle() is a paravirt-optimized version of e500_idle().

However, now that e500_idle has an ifdef for e500mc, we'll need that ifdef here as well. e500mc doesn't use MSR[WE] (and if it did, we couldn't trap on it). For e500mc we'll want to make an hcall for idle (ePAPR EV_IDLE).

-Scott

#3 lore raw
In reply to Scott Wood
Quoted 17 lines · Scott Wood
Quoted 15 lines · Alexander Graf

arch/powerpc/kernel/kvm.c static int __init kvm_guest_init(void) · line 809

Quoted 10 lines · Scott Wood
arch/powerpc/kernel/kvm.c +10 -0
+#ifdef CONFIG_E500
+	/*
+	 * Skip the overhead of HID0 accesses that KVM ignores --
+	 * just write MSR[WE].
+	 *
+	 * We don't need _TLF_NAPPING, because under KVM we know
+	 * it will take effect right away.
+	 */
+	if (ppc_md.power_save = e500_idle)
+		ppc_md.power_save = kvm_msrwe_idle;

Why the if() here?

Quoted 7 lines

To avoid replacing some other power_save() implementation. kvm_msrwe_idle() is a paravirt-optimized version of e500_idle().

However, now that e500_idle has an ifdef for e500mc, we'll need that ifdef here as well. e500mc doesn't use MSR[WE] (and if it did, we couldn't trap on it). For e500mc we'll want to make an hcall for idle (ePAPR EV_IDLE).

Since we're PV'ing here either way, can't we simply define a generic power_save implementation that works across different CPU types? We could for example always use EV_IDLE.

Alex

author
#4 lore raw
In reply to Scott Wood
Quoted 23 lines · Alexander Graf
Quoted 20 lines · Scott Wood
Quoted 15 lines · Alexander Graf

arch/powerpc/kernel/kvm.c static int __init kvm_guest_init(void) · line 809

Quoted 10 lines · Scott Wood
arch/powerpc/kernel/kvm.c +10 -0
+#ifdef CONFIG_E500
+	/*
+	 * Skip the overhead of HID0 accesses that KVM ignores --
+	 * just write MSR[WE].
+	 *
+	 * We don't need _TLF_NAPPING, because under KVM we know
+	 * it will take effect right away.
+	 */
+	if (ppc_md.power_save = e500_idle)
+		ppc_md.power_save = kvm_msrwe_idle;

Why the if() here?

To avoid replacing some other power_save() implementation. kvm_msrwe_idle() is a paravirt-optimized version of e500_idle().

Quoted 6 lines
Quoted 4 lines

However, now that e500_idle has an ifdef for e500mc, we'll need that ifdef here as well. e500mc doesn't use MSR[WE] (and if it did, we couldn't trap on it). For e500mc we'll want to make an hcall for idle (ePAPR EV_IDLE).

Since we're PV'ing here either way, can't we simply define a generic power_save implementation that works across different CPU types? We could for example always use EV_IDLE.

I suppose so, though I'd be more comfortable knowing what it is we're substituting for. E.g. maybe some no-op was put in power_save() because it was determined that the wake latency was too high relative to the workload. Not something you'd usually expect with virtualization, but you never know. There might be one guest that is meant to be prioritized and will be doing a lot of I/O, and the desire is to bunch up the time when other things run.

OTOH, detecting this by what function was put into power_save won't help if for similar reasons someone wants to run an actual wait instruction rather than trap to KVM. So if such a case does come up, it should be via some explicit configuration that tells KVM to leave it alone.

In practice, with the current code, I think e500_idle is the only thing that will show up in power_save if CONFIG_E500 is defined.

-Scott

#5 lore raw
In reply to Scott Wood
Quoted 31 lines · Scott Wood
Quoted 23 lines · Alexander Graf
Quoted 20 lines · Scott Wood
Quoted 15 lines · Alexander Graf

arch/powerpc/kernel/kvm.c static int __init kvm_guest_init(void) · line 809

Quoted 10 lines · Scott Wood
arch/powerpc/kernel/kvm.c +10 -0
+#ifdef CONFIG_E500
+	/*
+	 * Skip the overhead of HID0 accesses that KVM ignores --
+	 * just write MSR[WE].
+	 *
+	 * We don't need _TLF_NAPPING, because under KVM we know
+	 * it will take effect right away.
+	 */
+	if (ppc_md.power_save = e500_idle)
+		ppc_md.power_save = kvm_msrwe_idle;

Why the if() here?

To avoid replacing some other power_save() implementation. kvm_msrwe_idle() is a paravirt-optimized version of e500_idle().

Quoted 6 lines
Quoted 4 lines

However, now that e500_idle has an ifdef for e500mc, we'll need that ifdef here as well. e500mc doesn't use MSR[WE] (and if it did, we couldn't trap on it). For e500mc we'll want to make an hcall for idle (ePAPR EV_IDLE).

Since we're PV'ing here either way, can't we simply define a generic power_save implementation that works across different CPU types? We could for example always use EV_IDLE.

Quoted 7 lines

I suppose so, though I'd be more comfortable knowing what it is we're substituting for. E.g. maybe some no-op was put in power_save() because it was determined that the wake latency was too high relative to the workload. Not something you'd usually expect with virtualization, but you never know. There might be one guest that is meant to be prioritized and will be doing a lot of I/O, and the desire is to bunch up the time when other things run.

But at that point the guest needs to be modified too, no? Or is there a command line parameter that specifies which idle function to use? If so, we maybe want to read that one out to see if it's set manually and only change the default behavior?

Quoted 7 lines

OTOH, detecting this by what function was put into power_save won't help if for similar reasons someone wants to run an actual wait instruction rather than trap to KVM. So if such a case does come up, it should be via some explicit configuration that tells KVM to leave it alone.

In practice, with the current code, I think e500_idle is the only thing that will show up in power_save if CONFIG_E500 is defined.

That's what I assumed, yes. If we need something different, we better make it solid on all edges :)

Alex