KVM: PPC: Book3S: Add support for H_IPOLL and H_XIRR_X in XICS emulation
Subsystems (3)
PATCH 1/1 KVM: PPC: Book3S: Add support for H_IPOLL and H_XIRR_X in XICS emulation
This adds the remaining two hypercalls defined by PAPR for manipulating the XICS interrupt controller, H_IPOLL and H_XIRR_X. H_IPOLL returns information about the priority and pending interrupts for a virtual cpu, without changing any state. H_XIRR_X is like H_XIRR in that it reads and acknowledges the highest-priority pending interrupt, but it also returns the timestamp (timebase register value) from when the interrupt was first received by the hypervisor. Currently we just return the current time, since we don't do any software queueing of virtual interrupts inside the XICS emulation code.
These hcalls are not currently used by Linux guests, but may be in future.
---
Unfortunately I missed these two hcalls in the previous submissions. It would be good to get this patch into 3.10 so we don't have a kernel version with these calls missing from the API, in case future guest kernels want to use them.
Alex, given you're on vacation at the moment, are you OK with Ben taking this through his tree?
arch/powerpc/include/asm/hvcall.h | 1 + arch/powerpc/kvm/book3s_hv.c | 2 ++ arch/powerpc/kvm/book3s_pr_papr.c | 2 ++ arch/powerpc/kvm/book3s_xics.c | 29 +++++++++++++++++++++++++++++ 4 files changed, 34 insertions(+)
arch/powerpc/include/asm/hvcall.h +1 -0
@@ -264,6 +264,7 @@#define H_GET_MPP 0x2D4#define H_HOME_NODE_ASSOCIATIVITY 0x2EC#define H_BEST_ENERGY 0x2F4 +#define H_XIRR_X 0x2FC#define H_RANDOM 0x300#define H_COP 0x304#define H_GET_MPP_X 0x314
arch/powerpc/kvm/book3s_hv.c +2 -0
@@ -562,6 +562,8 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu)case H_CPPR :case H_EOI :case H_IPI : +case H_IPOLL : +case H_XIRR_X :if (kvmppc_xics_enabled (vcpu)) { ret =kvmppc_xics_hcall (vcpu, req);break ;
arch/powerpc/kvm/book3s_pr_papr.c +2 -0
@@ -292,6 +292,8 @@ int kvmppc_h_pr(struct kvm_vcpu *vcpu, unsigned long cmd)case H_CPPR :case H_EOI :case H_IPI : +case H_IPOLL : +case H_XIRR_X :if (kvmppc_xics_enabled (vcpu))return kvmppc_h_pr_xics_hcall (vcpu, cmd);break ;
arch/powerpc/kvm/book3s_xics.c +29 -0
@@ -650,6 +650,23 @@ static noinline int kvmppc_h_ipi(struct kvm_vcpu *vcpu, unsigned long server,return H_SUCCESS ; } +static int kvmppc_h_ipoll (struct kvm_vcpu *vcpu,unsigned long server) +{ +union kvmppc_icp_state state; +struct kvmppc_icp *icp; + + icp = vcpu->arch.icp; +if (icp->server_num != server) { + icp =kvmppc_xics_find_server (vcpu->kvm, server); +if (!icp) +return H_PARAMETER ; + } + state =ACCESS_ONCE (icp->state); +kvmppc_set_gpr (vcpu,4 , ((u32 )state.cppr <<24 ) | state.xisr); +kvmppc_set_gpr (vcpu,5 , state.mfrr); +return H_SUCCESS ; +} + static noinline void kvmppc_h_cppr(struct kvm_vcpu *vcpu, unsigned long cppr) {union kvmppc_icp_state old_state, new_state;
@@ -787,6 +804,18 @@ int kvmppc_xics_hcall(struct kvm_vcpu *vcpu, u32 req)if (!xics || !vcpu->arch.icp)return H_HARDWARE ; +/* These requests don't have real-mode implementations at present */ +switch (req) { +case H_XIRR_X : + res =kvmppc_h_xirr (vcpu); +kvmppc_set_gpr (vcpu,4 , res); +kvmppc_set_gpr (vcpu,5 ,get_tb ()); +return rc; +case H_IPOLL : + rc =kvmppc_h_ipoll (vcpu,kvmppc_get_gpr (vcpu,4 )); +return rc; + } +/* Check for real mode returning too hard */ if (xics->real_mode)return kvmppc_xics_rm_complete (vcpu, req);
Signature · 1 line
1.7.10.4
Quoted 14 lines · Paul Mackerras
This adds the remaining two hypercalls defined by PAPR for manipulating the XICS interrupt controller,
H_IPOLLandH_XIRR_X.H_IPOLLreturns information about the priority and pending interrupts for a virtual cpu, without changing any state.H_XIRR_Xis likeH_XIRRin that it reads and acknowledges the highest-priority pending interrupt, but it also returns the timestamp (timebase register value) from when the interrupt was first received by the hypervisor. Currently we just return the current time, since we don't do any software queueing of virtual interrupts inside the XICS emulation code.These hcalls are not currently used by Linux guests, but may be in future.
Signed-off-by Paul Mackerras <[email protected]>
Quoted 8 lines
---
Unfortunately I missed these two hcalls in the previous submissions. It would be good to get this patch into 3.10 so we don't have a kernel version with these calls missing from the API, in case future guest kernels want to use them.Alex, given you're on vacation at the moment, are you OK with Ben taking this through his tree?
I believe Alex is staying far away from e-mail on his vacation. He's asked me to fill in for him while he's gone.
The patch itself seems reasonable (though I don't know much about XICS, and do have one question...), but I'll leave it up to Gleb/Marcelo/Ben if it should go in for 3.10 and via which tree. I understand the desire to not have an incomplete ABI in a released version, but Linus is already grumbling about how much went into rc3, and you say the hcalls aren't currently used... Are they likely to be used in any timeframe in which we'd reasonably care about 3.10? If so, would the effect of not having them implemented be such that it would be worse than not having in-kernel XICS at all?
Quoted 11 lines
arch/powerpc/kvm/book3s_xics.c +1
-0@@ -787,6 +804,18 @@ int kvmppc_xics_hcall(struct kvm_vcpu *vcpu, u32 req)if (!xics || !vcpu->arch.icp)return H_HARDWARE ; +/* These requests don't have real-mode implementations at present */
arch/powerpc/kvm/book3s_xics.c +4
-0+switch (req) { +case H_XIRR_X : + res =kvmppc_h_xirr (vcpu); +kvmppc_set_gpr (vcpu,4 , res);
arch/powerpc/kvm/book3s_xics.c int kvmppc_xics_hcall(struct kvm_vcpu *vcpu, u32 req) · line 821
Quoted 10 lines
arch/powerpc/kvm/book3s_xics.c +7
-0+kvmppc_set_gpr (vcpu,5 ,get_tb ()); +return rc; +case H_IPOLL : + rc =kvmppc_h_ipoll (vcpu,kvmppc_get_gpr (vcpu,4 )); +return rc; + } +/* Check for real mode returning too hard */ if (xics->real_mode)return kvmppc_xics_rm_complete (vcpu, req);
Could you explain what's going on here relative to kvmppc_xics_rm_complete()? What does "returning too hard" mean, and why must rm_action not be checked for these hcalls?
-Scott
Quoted 10 lines · Scott Wood
I believe Alex is staying far away from e-mail on his vacation. He's asked me to fill in for him while he's gone.
The patch itself seems reasonable (though I don't know much about XICS, and do have one question...), but I'll leave it up to Gleb/Marcelo/Ben if it should go in for 3.10 and via which tree. I understand the desire to not have an incomplete ABI in a released version, but Linus is already grumbling about how much went into rc3, and you say the hcalls aren't currently used... Are they likely to be used in any timeframe in which we'd reasonably care about 3.10?
Yes. I'd like to have them in. Their implementation is actually fairly trivial and they cannot be emulated by qemu if the rest of the XICS is in the kernel, so it's a problem.
Quoted 3 lines
If so, would the
effect of not having them implemented be such that it would be worse than not having in-kernel XICS at all?
Well, that would mean that running that potential future kernel that uses them (or some other OS that already uses them) would not work without the user explicitly disable in kernel irq chip.
Quoted 17 lines
Quoted 17 lines
arch/powerpc/kvm/book3s_xics.c +1
-0@@ -787,6 +804,18 @@ int kvmppc_xics_hcall(struct kvm_vcpu *vcpu, u32 req)if (!xics || !vcpu->arch.icp)return H_HARDWARE ; +/* These requests don't have real-mode implementations at present */
arch/powerpc/kvm/book3s_xics.c +10
-0+switch (req) { +case H_XIRR_X : + res =kvmppc_h_xirr (vcpu); +kvmppc_set_gpr (vcpu,4 , res); +kvmppc_set_gpr (vcpu,5 ,get_tb ()); +return rc; +case H_IPOLL : + rc =kvmppc_h_ipoll (vcpu,kvmppc_get_gpr (vcpu,4 )); +return rc; + }
Quoted 8 lines
arch/powerpc/kvm/book3s_xics.c int kvmppc_xics_hcall(struct kvm_vcpu *vcpu, u32 req) · line 821
Quoted 4 lines
arch/powerpc/kvm/book3s_xics.c +1
-0+/* Check for real mode returning too hard */ if (xics->real_mode)return kvmppc_xics_rm_complete (vcpu, req);Could you explain what's going on here relative to
kvmppc_xics_rm_complete()? What does "returning too hard" mean, and why mustrm_actionnot be checked for these hcalls?
This is related to how we handle some hcalls in real mode as a fast path. The real-mode stuff cannot handle cases that require for example a re-emit of the interrupt, a reject, etc... so in some cases, it returns H_TOO_HARD which causes KVM to exit and try to handle the hcall again in kernel virtual mode.
When doing so as the result of a XICS hcall, it sets a bit mask of "tasks" to handle in virtual mode (because it will have already partially done the operation, it cannot just re-play the whole hcall).
So when real-mode is supported we must not just call the normal virtual mode version of the hcalls, we instead go to kvmppc_xics_rm_complete() to handle those "tasks".
However, for those 2 "missing" hcalls, we have no real mode implementation at all (we didn't bother, we will do that later if needed, it's purely a performance issue). So we need to fully handle them in virtual mode, and we know there will be no "tasks" to handle in rm_complete.
We could have done a real mode variant but we wanted to keep the patch as small as possible for 3.10.
Cheers,
Ben.
Quoted 12 lines · Benjamin Herrenschmidt
Quoted 1 line · Scott Wood
I believe Alex is staying far away from e-mail on his vacation.
He's
Quoted 3 lines
asked me to fill in for him while he's gone.
The patch itself seems reasonable (though I don't know much about
XICS,
Quoted 1 line
and do have one question...), but I'll leave it up to
Gleb/Marcelo/Ben
Quoted 2 lines
if it should go in for 3.10 and via which tree. I understand the desire to not have an incomplete ABI in a released version, but
Quoted 8 lines
Linus
Quoted 3 lines
is already grumbling about how much went into rc3, and you say the hcalls aren't currently used... Are they likely to be used in any timeframe in which we'd reasonably care about 3.10?
Yes. I'd like to have them in. Their implementation is actually fairly trivial and they cannot be emulated by qemu if the rest of the XICS is in the kernel, so it's a problem.
OK. Does it make more sense for you to take it as Paul suggested, or for Gleb or Marcelo to pick it up directly?
Quoted 39 lines
Quoted 12 lines
Quoted 12 lines
(quoted diff) +1
-0+ /* These requests don't have real-mode implementations atpresent */
arch/powerpc/kvm/book3s_xics.c +10
-0+switch (req) { +case H_XIRR_X : + res =kvmppc_h_xirr (vcpu); +kvmppc_set_gpr (vcpu,4 , res); +kvmppc_set_gpr (vcpu,5 ,get_tb ()); +return rc; +case H_IPOLL : + rc =kvmppc_h_ipoll (vcpu,kvmppc_get_gpr (vcpu,4 )); +return rc; + }Quoted 8 lines
arch/powerpc/kvm/book3s_xics.c int kvmppc_xics_hcall(struct kvm_vcpu *vcpu, u32 req) · line 821
Quoted 4 lines
arch/powerpc/kvm/book3s_xics.c +1
-0+/* Check for real mode returning too hard */ if (xics->real_mode)return kvmppc_xics_rm_complete (vcpu, req);Could you explain what's going on here relative to
kvmppc_xics_rm_complete()? What does "returning too hard" mean, and why mustrm_actionnot be checked for these hcalls?This is related to how we handle some hcalls in real mode as a fast path. The real-mode stuff cannot handle cases that require for example a
re-emit of the interrupt, a reject, etc... so in some cases, it returnsH_TOO_HARDwhich causes KVM to exit and try to handle the hcall again in
kernel virtual mode.When doing so as the result of a XICS hcall, it sets a bit mask of "tasks" to handle in virtual mode (because it will have already partially done the operation, it cannot just re-play the whole hcall).
So when real-mode is supported we must not just call the normal virtual
mode version of the hcalls, we instead go tokvmppc_xics_rm_complete()to handle those "tasks".
Quoted 6 lines
However, for those 2 "missing" hcalls, we have no real mode implementation at all (we didn't bother, we will do that later if needed, it's purely a performance issue). So we need to fully handle them in virtual mode, and we know there will be no "tasks" to handle in
rm_complete.
Then rm_action should always be 0 for these hcalls, right? So there's no correctness reason to keep the hcalls in separate switch statements. You shave off a few cycles checking rm_action, at the cost of needing to change kvmppc_xics_hcall() if a real-mode version of these hcalls is ever done.
-Scott
Quoted 6 lines · Scott Wood
Quoted 3 lines
Yes. I'd like to have them in. Their implementation is actually fairly trivial and they cannot be emulated by qemu if the rest of the XICS is in the kernel, so it's a problem.
OK. Does it make more sense for you to take it as Paul suggested, or for Gleb or Marcelo to pick it up directly?
I'll take it.
Quoted 5 lines
Then
rm_actionshould always be 0 for these hcalls, right? So there's no correctness reason to keep the hcalls in separate switch statements. You shave off a few cycles checkingrm_action, at the cost of needing to changekvmppc_xics_hcall()if a real-mode version of these hcalls is ever done.
No, because rm_action will also be 0 if the hcall was fully done in real mode (which can happen, that's our fast path), in which case we do *NOT* want to to be re-done in virtual mode.
That's why we always return whether rm_action is 0 or not when real-mode is enabled.
Cheers,
Ben.
Scott Wood
Acked-byQuoted 5 lines · Benjamin Herrenschmidt
Quoted 1 line · Scott Wood
Quoted 1 line
Yes. I'd like to have them in. Their implementation is actually
fairly
Quoted 1 line
Quoted 1 line
trivial and they cannot be emulated by qemu if the rest of the
Quoted 8 lines
XICS is
Quoted 3 lines
Quoted 1 line
in the kernel, so it's a problem.
OK. Does it make more sense for you to take it as Paul suggested,
or
Quoted 1 line
for Gleb or Marcelo to pick it up directly?
I'll take it.
Quoted 8 lines
Quoted 1 line
Then
rm_actionshould always be 0 for these hcalls, right? Sothere's
Quoted 2 lines
no correctness reason to keep the hcalls in separate switch statements. You shave off a few cycles checking
rm_action, at thecost
Quoted 2 lines
of needing to change
kvmppc_xics_hcall()if a real-mode version of these hcalls is ever done.
Quoted 6 lines
No, because
rm_actionwill also be 0 if the hcall was fully done in real mode (which can happen, that's our fast path), in which case we do *NOT* want to to be re-done in virtual mode.That's why we always return whether
rm_actionis 0 or not when real-mode is enabled.
Oh, I misread the code and thought the decision to return was based on the return value of kvmppc_xics_rm_complete. Sorry about that. :-(
-Scott