Skip to contentSkip to conversation

kvm:ppc:booehv: direct ISI exception to Guest

PATCH In review Tiejun Chen 4 participants 14 messages May 7, 2013 – May 13, 2013

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kvm:ppc:booehv: direct ISI exception to Guest

We also can direct ISI exception to Guest like DSI.

---
 arch/powerpc/kvm/booke_emulate.c |    3 +++
 arch/powerpc/kvm/e500mc.c        |    3 ++-
 2 files changed, 5 insertions(+), 1 deletion(-)
arch/powerpc/kvm/booke_emulate.c +3 -0
index 27a4b28..33b14e9 100644
@@ -197,6 +197,9 @@ int kvmppc_booke_emulate_mtspr(struct kvm_vcpu *vcpu, int sprn, ulong spr_val)
 		break;
 	case SPRN_IVOR3:
 		vcpu->arch.ivor[BOOKE_IRQPRIO_INST_STORAGE] = spr_val;
+#ifdef CONFIG_KVM_BOOKE_HV
+		mtspr(SPRN_GIVOR3, spr_val);
+#endif
 		break;
 	case SPRN_IVOR4:
 		vcpu->arch.ivor[BOOKE_IRQPRIO_EXTERNAL] = spr_val;
arch/powerpc/kvm/e500mc.c +2 -1
index c3bdc0a..acf546a 100644
@@ -123,6 +123,7 @@ void kvmppc_core_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 
 	mtspr(SPRN_GIVPR, vcpu->arch.ivpr);
 	mtspr(SPRN_GIVOR2, vcpu->arch.ivor[BOOKE_IRQPRIO_DATA_STORAGE]);
+	mtspr(SPRN_GIVOR3, vcpu->arch.ivor[BOOKE_IRQPRIO_INST_STORAGE]);
 	mtspr(SPRN_GIVOR8, vcpu->arch.ivor[BOOKE_IRQPRIO_SYSCALL]);
 	mtspr(SPRN_GSPRG0, (unsigned long)vcpu->arch.shared->sprg0);
 	mtspr(SPRN_GSPRG1, (unsigned long)vcpu->arch.shared->sprg1);
@@ -185,7 +186,7 @@ int kvmppc_core_vcpu_setup(struct kvm_vcpu *vcpu)
 	struct kvmppc_vcpu_e500 *vcpu_e500 = to_e500(vcpu);
 
 	vcpu->arch.shadow_epcr = SPRN_EPCR_DSIGS | SPRN_EPCR_DGTMI | \
-				 SPRN_EPCR_DUVD;
+				 SPRN_EPCR_DUVD | SPRN_EPCR_ISIGS;
 #ifdef CONFIG_64BIT
 	vcpu->arch.shadow_epcr |= SPRN_EPCR_ICM;
 #endif
Signature · 1 line
1.7.9.5
#1 lore raw
Quoted 7 lines · Tiejun Chen

We also can direct ISI exception to Guest like DSI.

---
 arch/powerpc/kvm/booke_emulate.c |    3 +++
 arch/powerpc/kvm/e500mc.c        |    3 ++-
 2 files changed, 5 insertions(+), 1 deletion(-)

Are you seeing a real performance improvement from this? This will interfere somewhat with using the VF bit, if we were to ever do so, since VF only affects data accesses (and so the guest would see an ISI storm rather than a machine check if it tries to execute from such an address).

-Scott

#2 lore raw
Quoted 10 lines · Scott Wood
Quoted 7 lines · Tiejun Chen

We also can direct ISI exception to Guest like DSI.

---
 arch/powerpc/kvm/booke_emulate.c |    3 +++
 arch/powerpc/kvm/e500mc.c        |    3 ++-
 2 files changed, 5 insertions(+), 1 deletion(-)

Are you seeing a real performance improvement from this? This will interfere

No. But after we reduce the exit to host, shouldn't this improve performance?

Quoted 1 line

somewhat with using the VF bit, if we were to ever do so, since VF only affects

Sorry, what is the VF you said?

Tiejun

#3 lore raw
Quoted 20 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of tiejun.chen
Sent: Wednesday, May 08, 2013 4:54 AM
To: Wood Scott-B07421
Cc: [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

Quoted 9 lines · Scott Wood
Quoted 7 lines · Tiejun Chen

We also can direct ISI exception to Guest like DSI.

---
 arch/powerpc/kvm/booke_emulate.c |    3 +++
 arch/powerpc/kvm/e500mc.c        |    3 ++-
 2 files changed, 5 insertions(+), 1 deletion(-)
Quoted 5 lines
Quoted 1 line

Are you seeing a real performance improvement from this? This will

interfere

No. But after we reduce the exit to host, shouldn't this improve performance?

We lose some flexibility for this so it make sense only if we gain measurable improvements.

Quoted 5 lines
Quoted 2 lines

somewhat with using the VF bit, if we were to ever do so, since VF only

affects

Sorry, what is the VF you said?

VF stands for virtualization fault see MAS8[VF] and we may use it for virtualized MMIO. The hypervisor should deny execute access on pages marked with VF. Accordingly in this case guest ISI exceptions should be handled by the hypervisor.

-Mike

#4 lore raw
Quoted 20 lines · On 05/08/2013 05:20 PM, Caraman Mihai Claudiu-B02008 wrote:
Quoted 20 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of tiejun.chen
Sent: Wednesday, May 08, 2013 4:54 AM
To: Wood Scott-B07421
Cc: [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

Quoted 9 lines · Scott Wood
Quoted 7 lines · Tiejun Chen

We also can direct ISI exception to Guest like DSI.

---
  arch/powerpc/kvm/booke_emulate.c |    3 +++
  arch/powerpc/kvm/e500mc.c        |    3 ++-
  2 files changed, 5 insertions(+), 1 deletion(-)
Quoted 8 lines
Quoted 5 lines
Quoted 1 line

Are you seeing a real performance improvement from this? This will

interfere

No. But after we reduce the exit to host, shouldn't this improve performance?

We lose some flexibility for this so it make sense only if we gain measurable improvements.

Sounds we have much more works to do.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

somewhat with using the VF bit, if we were to ever do so, since VF only

affects

Sorry, what is the VF you said?

VF stands for virtualization fault see MAS8[VF] and we may use it for virtualized

I almost forget this point :)

Quoted 2 lines

MMIO. The hypervisor should deny execute access on pages marked with VF. Accordingly in this case guest ISI exceptions should be handled by the hypervisor.

Thanks for your information.

Tiejun

#5 lore raw
Quoted 34 lines · tiejun.chen
Quoted 20 lines · On 05/08/2013 05:20 PM, Caraman Mihai Claudiu-B02008 wrote:
Quoted 20 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of tiejun.chen
Sent: Wednesday, May 08, 2013 4:54 AM
To: Wood Scott-B07421
Cc: [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

Quoted 9 lines · Scott Wood
Quoted 7 lines · Tiejun Chen

We also can direct ISI exception to Guest like DSI.

---
  arch/powerpc/kvm/booke_emulate.c |    3 +++
  arch/powerpc/kvm/e500mc.c        |    3 ++-
  2 files changed, 5 insertions(+), 1 deletion(-)
Quoted 8 lines
Quoted 5 lines
Quoted 1 line

Are you seeing a real performance improvement from this? This will

interfere

No. But after we reduce the exit to host, shouldn't this improve performance?

We lose some flexibility for this so it make sense only if we gain measurable improvements.

Sounds we have much more works to do.

Quoted 2 lines
Quoted 2 lines
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line

somewhat with using the VF bit, if we were to ever do so, since VF only

affects

Sorry, what is the VF you said?

VF stands for virtualization fault see MAS8[VF] and we may use it for virtualized

I almost forget this point :)

Looks KVM PPC have no this mechanism currently since I don't find MAS8_VF is used in kernel, right?

If I'm missing something please correct me.

Tiejun

Quoted 5 lines
Quoted 4 lines

MMIO. The hypervisor should deny execute access on pages marked with VF. Accordingly
in this case guest ISI exceptions should be handled by the hypervisor.

#6 lore raw
Quoted 6 lines
Quoted 1 line
Quoted 1 line

VF stands for virtualization fault see MAS8[VF] and we may use it for

virtualized

Looks KVM PPC have no this mechanism currently since I don't find MAS8_VF is
used in kernel, right?

Yes but 'we may use it' in the feature, I have a functional POC with VF. Now we capture virtualized MMIO accesses as TLB misses (we don't write into HW TLB guest translation outside visible memslots).

-Mike

#7 lore raw
Quoted 8 lines · On 05/09/2013 07:34 PM, Caraman Mihai Claudiu-B02008 wrote:
Quoted 6 lines
Quoted 1 line
Quoted 1 line

VF stands for virtualization fault see MAS8[VF] and we may use it for

virtualized

Looks KVM PPC have no this mechanism currently since I don't find MAS8_VF is
used in kernel, right?

Yes but 'we may use it' in the feature, I have a functional POC with VF.

Any IO performance to be improved with this POC?

Quoted 2 lines

Now we capture virtualized MMIO accesses as TLB misses (we don't write into HW TLB guest translation outside visible memslots).

Yes.

Tiejun

#8 lore raw
Quoted 15 lines

-----Original Message-----
From: tiejun.chen [mailto:[email protected]]
Sent: Thursday, May 09, 2013 2:40 PM
To: Caraman Mihai Claudiu-B02008
Cc: Wood Scott-B07421; [email protected]; [email protected]; kvm- [email protected]; [email protected]
Subject: Re: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

Quoted 1 line · On 05/09/2013 07:34 PM, Caraman Mihai Claudiu-B02008 wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line

VF stands for virtualization fault see MAS8[VF] and we may use it

for

Quoted 3 lines
Quoted 3 lines

virtualized

Looks KVM PPC have no this mechanism currently since I don't find

Quoted 8 lines

MAS8_VF

Quoted 4 lines
Quoted 2 lines

is
used in kernel, right?

Yes but 'we may use it' in the feature, I have a functional POC with

VF.

Any IO performance to be improved with this POC?

VF approach puts more stress on HW TLB so I did not advance with performance measurements though it may worth to do it.

-Mike

#9 lore raw
Quoted 19 lines · On 09.05.2013, at 14:36, Caraman Mihai Claudiu-B02008 wrote:
Quoted 19 lines

-----Original Message-----
From: tiejun.chen [mailto:[email protected]]
Sent: Thursday, May 09, 2013 2:40 PM
To: Caraman Mihai Claudiu-B02008
Cc: Wood Scott-B07421; [email protected]; [email protected]; kvm- [email protected]; [email protected]
Subject: Re: [v1][KVM][PATCH 1/1] kvm:ppc:booehv: direct ISI exception to Guest

Quoted 1 line · On 05/09/2013 07:34 PM, Caraman Mihai Claudiu-B02008 wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line

VF stands for virtualization fault see MAS8[VF] and we may use it

for

Quoted 3 lines
Quoted 3 lines

virtualized

Looks KVM PPC have no this mechanism currently since I don't find

MAS8_VF

Quoted 3 lines
Quoted 2 lines

is
used in kernel, right?

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Yes but 'we may use it' in the feature, I have a functional POC with

VF.

Any IO performance to be improved with this POC?

VF approach puts more stress on HW TLB so I did not advance with performance measurements though it may worth to do it.

Could you guys please collect performance data during the next weeks on both guest-directed ISIs as well as VF MMIOs (preferably with in-kernel MMIO), so that we can decide on the direction that's worth going towards?

Thanks!

Alex

#10 lore raw
Quoted 1 line · Alexander Graf

Could you guys please collect performance data during the next weeks on both guest-directed ISIs as well as VF MMIOs (preferably with in-kernel MMIO), so that we can decide on the direction that's worth going towards?

Collecting data on VF MMIO would require implementing it (or at least salvaging and fixing some old code), which is not a high priority at the moment. If we do implement VF in the future we could always undo the direct ISI change, but it would still be nice to know if there's any real benefit in the first place.

FWIW, I doubt that the "more stress on HW TLB" will be significant.

-Scott

#11 lore raw

Am 10.05.2013 um 21:22 schrieb Scott Wood <[email protected]>:

Quoted 4 lines · Alexander Graf
Quoted 1 line

Could you guys please collect performance data during the next weeks on both guest-directed ISIs as well as VF MMIOs (preferably with in-kernel MMIO), so that we can decide on the direction that's worth going towards?

Collecting data on VF MMIO would require implementing it (or at least salvaging and fixing some old code), which is not a high priority at the moment. If we do implement VF in the future we could always undo the direct ISI change, but it would still be nice to know if there's any real benefit in the first place.

Mike sounded like he had an almost working poc, which is good enough to collect rough numbers.

And yes, changes like these should always get at least basic performance numbers along with them, regardless of drawbacks.

Alex

Quoted 4 lines

FWIW, I doubt that the "more stress on HW TLB" will be significant.

-Scott

#12 lore raw
Quoted 10 lines · Alexander Graf

Am 10.05.2013 um 21:22 schrieb Scott Wood <[email protected]>:

Quoted 4 lines · Alexander Graf
Quoted 1 line

Could you guys please collect performance data during the next weeks on both guest-directed ISIs as well as VF MMIOs (preferably with in-kernel MMIO), so that we can decide on the direction that's worth going towards?

Collecting data on VF MMIO would require implementing it (or at least salvaging and fixing some old code), which is not a high priority at the moment. If we do implement VF in the future we could always undo the direct ISI change, but it would still be nice to know if there's any real benefit in the first place.

Mike sounded like he had an almost working poc, which is good enough to collect rough numbers.

Which can the test case be adopted?

Mike,

If you already have a good case for your poc, please share that with me. Then I'd like to run that.

Tiejun

Quoted 11 lines

And yes, changes like these should always get at least basic performance numbers along with them, regardless of drawbacks.

Alex

Quoted 4 lines

FWIW, I doubt that the "more stress on HW TLB" will be significant.

-Scott

#13 lore raw
In reply to tiejun.chen
Quoted 10 lines · tiejun.chen
Quoted 9 lines · Scott Wood
Quoted 7 lines · Tiejun Chen

We also can direct ISI exception to Guest like DSI.

---
 arch/powerpc/kvm/booke_emulate.c |    3 +++
 arch/powerpc/kvm/e500mc.c        |    3 ++-
 2 files changed, 5 insertions(+), 1 deletion(-)
Quoted 3 lines
Quoted 1 line

Are you seeing a real performance improvement from this? This will interfere

No. But after we reduce the exit to host, shouldn't this improve performance?

Not if ISIs are too rare to matter.

-Scott