powerpc: kvm: generic framework and run posix timers in task context
Subsystems (2)
Shrikanth Hegde
reviewercover powerpc: kvm: generic framework and run posix timers in task context
From: Gautam Menghani <[email protected]>
This is an effort to use the generic kvm infra which handles check for need_resched, handling signals etc. i.e xfer_to_guest_mode_handle_work.
kvm guests boots and runs stress-ng CPU stressor on PowerVM and on PowerNV. preempt=full and preempt=lazy was tested on PowerNV and in both cases the KVM guest boots and runs stress-ng CPU stressor.
Please let me know if any specific testing to be done.
Kept the patches separate since they differ functionally, but kept them is a series since 2nd patch depends on functionality of 1st. Also this could help in git bisect.
This is based on tip/master
Shrikanth Hegde (2):
powerpc: kvm: use generic transfer to guest mode work
powerpc: enable to run posix cpu timers in task context
arch/powerpc/Kconfig | 2 ++ arch/powerpc/kvm/book3s_hv.c | 13 +++++++------ arch/powerpc/kvm/powerpc.c | 22 ++++++++-------------- 3 files changed, 17 insertions(+), 20 deletions(-)
Signature · 1 line
2.48.1
Shrikanth Hegde
reviewerPATCH 1/2 powerpc: kvm: use generic transfer to guest mode work
There is generic entry framework to handle signals and check for reschedule etc before entering the guest. Use that framework for powerpc.
Advantages:
- Less code duplication.
- powerpc kvm now understands NR and NR_lazy bits.
- powerpc can enable HAVE_POSIX_CPU_TIMERS_TASK_WORK, currently the powerpc/kvm code doesn't handle TIF_NOTIFY_RESUME.
Testing: No splats seen in below cases.
- Booted KVM guest on PowerVM and PowerNV systems.
- Ran stress-ng CPU stressors in each above case.
- On PowerNV host, tried preempt=lazy/full and run stress-ng CPU stressor in the KVM guest.
--- arch/powerpc/Kconfig | 1 + arch/powerpc/kvm/book3s_hv.c | 13 +++++++------ arch/powerpc/kvm/powerpc.c | 22 ++++++++-------------- 3 files changed, 16 insertions(+), 20 deletions(-)
arch/powerpc/Kconfig +1 -0
@@ -299,6 +299,7 @@ config PPC select IRQ_DOMAIN select IRQ_FORCED_THREADING select KASAN_VMALLOC if KASAN && EXECMEM + select KVM_XFER_TO_GUEST_WORK select LOCK_MM_AND_FIND_VMA select MMU_GATHER_PAGE_SIZE select MMU_GATHER_RCU_TABLE_FREE
arch/powerpc/kvm/book3s_hv.c +7 -6
@@ -80,8 +80,8 @@#include <asm/ultravisor.h> #include <asm/dtl.h> #include <asm/plpar_wrappers.h> -#include <trace/events/ipi.h> +#include <linux/entry-kvm.h> #include "book3s.h" #include "book3s_hv.h"
@@ -4901,7 +4901,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, }if (need_resched ()) -cond_resched (); +schedule ();kvmppc_update_vpas (vcpu);
@@ -5097,10 +5097,11 @@ static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu)return -EINVAL ; } -/* No need to go into the guest when all we'll do is come back out */ -if (signal_pending (current)) { - run->exit_reason =KVM_EXIT_INTR ; -return -EINTR ; +/* use generic frameworks to handle signals, need_resched */ +if (__xfer_to_guest_mode_work_pending ()) { + r =xfer_to_guest_mode_handle_work (vcpu); +if (r) +return r; }#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
arch/powerpc/kvm/powerpc.c +8 -14
@@ -34,6 +34,7 @@#endif #include <asm/ultravisor.h> #include <asm/setup.h> +#include <linux/entry-kvm.h> #include "timing.h" #include "../mm/mmu_decl.h"
@@ -80,24 +81,17 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) {int r; +/* use generic framework to handle need resched and signals */ +if (__xfer_to_guest_mode_work_pending ()) { + r =xfer_to_guest_mode_handle_work (vcpu); +if (r) +return r; + } +WARN_ON (irqs_disabled ());hard_irq_disable ();while (true) { -if (need_resched ()) { -local_irq_enable (); -cond_resched (); -hard_irq_disable (); -continue ; - } - -if (signal_pending (current)) { -kvmppc_account_exit (vcpu,SIGNAL_EXITS ); - vcpu->run->exit_reason =KVM_EXIT_INTR ; - r = -EINTR ; -break ; - } - vcpu->mode =IN_GUEST_MODE ;/*
Signature · 1 line
2.48.1
Re: [PATCH 1/2] powerpc: kvm: use generic transfer to guest mode work
Quoted 15 lines · Shrikanth Hegde
arch/powerpc/kvm/book3s_hv.c +1
-1@@ -80,8 +80,8 @@#include <asm/ultravisor.h> #include <asm/dtl.h> #include <asm/plpar_wrappers.h> -#include <trace/events/ipi.h> +#include <linux/entry-kvm.h> #include "book3s.h" #include "book3s_hv.h"
arch/powerpc/kvm/book3s_hv.c int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, · line 4904
Quoted 5 lines
arch/powerpc/kvm/book3s_hv.c +1
-1@@ -4901,7 +4901,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, }if (need_resched ()) -cond_resched (); +schedule ();
This looks unrelated and odd. I don't why but this should be a cond_resched() so it can be optimized away on PREEMPT kernels.
arch/powerpc/kvm/book3s_hv.c static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu) · line 5102
Quoted 13 lines
arch/powerpc/kvm/book3s_hv.c +3
-4kvmppc_update_vpas (vcpu);@@ -5097,10 +5097,11 @@ static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu)return -EINVAL ; } -/* No need to go into the guest when all we'll do is come back out */ -if (signal_pending (current)) { - run->exit_reason =KVM_EXIT_INTR ; -return -EINTR ; +/* use generic frameworks to handle signals, need_resched */ +if (__xfer_to_guest_mode_work_pending ()) { + r =xfer_to_guest_mode_handle_work (vcpu);
This could be unconditional.
Quoted 18 lines
arch/powerpc/kvm/book3s_hv.c +2
-0+if (r) +return r; }#ifdef CONFIG_PPC_TRANSACTIONAL_MEM arch/powerpc/kvm/powerpc.c +1
-0@@ -34,6 +34,7 @@#endif #include <asm/ultravisor.h> #include <asm/setup.h> +#include <linux/entry-kvm.h> #include "timing.h" #include "../mm/mmu_decl.h"
arch/powerpc/kvm/powerpc.c int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) · line 86
Quoted 6 lines
arch/powerpc/kvm/powerpc.c +3
-0@@ -80,24 +81,17 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) {int r; +/* use generic framework to handle need resched and signals */ +if (__xfer_to_guest_mode_work_pending ()) { + r =xfer_to_guest_mode_handle_work (vcpu);
there is nothing special you do checking and handling the work. Couldn't you invoke xfer_to_guest_mode_handle_work() unconditionally?
Quoted 10 lines
arch/powerpc/kvm/powerpc.c +4
-2+if (r) +return r; + } +WARN_ON (irqs_disabled ());hard_irq_disable ();while (true) { -if (need_resched ()) { -local_irq_enable ();
arch/powerpc/kvm/powerpc.c int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) · line 98
Quoted 10 lines
arch/powerpc/kvm/powerpc.c +0
-10-cond_resched (); -hard_irq_disable (); -continue ; - } - -if (signal_pending (current)) { -kvmppc_account_exit (vcpu,SIGNAL_EXITS ); - vcpu->run->exit_reason =KVM_EXIT_INTR ; - r = -EINTR ; -break ;
I don't how this works but couldn't SIGNAL_EXITS vanish now that it isn't updated anymore? The stat itself moves in kvm_handle_signal_exit() to a different counter so it is not lost. The reader just needs to look somewhere else for it.
arch/powerpc/kvm/powerpc.c int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) · line 97
Quoted 5 lines
arch/powerpc/kvm/powerpc.c +0
-2- } - vcpu->mode =IN_GUEST_MODE ;/*
Sebastian
Shrikanth Hegde
reviewerRe: [PATCH 1/2] powerpc: kvm: use generic transfer to guest mode work
On 4/24/25 20:12, Sebastian Andrzej Siewior wrote:
Thanks Sebastian for taking a look.
Quoted 18 lines · Shrikanth Hegde
Quoted 17 lines
arch/powerpc/kvm/book3s_hv.c +1
-1@@ -80,8 +80,8 @@#include <asm/ultravisor.h> #include <asm/dtl.h> #include <asm/plpar_wrappers.h> -#include <trace/events/ipi.h> +#include <linux/entry-kvm.h> #include "book3s.h" #include "book3s_hv.h" @@ -4901,7 +4901,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, }
Quoted 8 lines
arch/powerpc/kvm/book3s_hv.c int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, · line 4904
Quoted 3 lines
arch/powerpc/kvm/book3s_hv.c +1
-1if (need_resched ()) -cond_resched (); +schedule ();This looks unrelated and odd. I don't why but this should be a
cond_resched()so it can be optimized away on PREEMPT kernels.
This is needed, otherwise KVM on powerVM setup gets stuck on preempt=full/lazy.
Quoted 37 lines
Quoted 4 lines
arch/powerpc/kvm/book3s_hv.c +0
-0kvmppc_update_vpas (vcpu);arch/powerpc/kvm/book3s_hv.c static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu) · line 5102
Quoted 10 lines
arch/powerpc/kvm/book3s_hv.c +3
-4@@ -5097,10 +5097,11 @@ static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu)return -EINVAL ; } -/* No need to go into the guest when all we'll do is come back out */ -if (signal_pending (current)) { - run->exit_reason =KVM_EXIT_INTR ; -return -EINTR ; +/* use generic frameworks to handle signals, need_resched */ +if (__xfer_to_guest_mode_work_pending ()) { + r =xfer_to_guest_mode_handle_work (vcpu);This could be unconditional.
Quoted 21 lines
arch/powerpc/kvm/book3s_hv.c +2
-0+if (r) +return r; }#ifdef CONFIG_PPC_TRANSACTIONAL_MEM arch/powerpc/kvm/powerpc.c +1
-0@@ -34,6 +34,7 @@#endif #include <asm/ultravisor.h> #include <asm/setup.h> +#include <linux/entry-kvm.h> #include "timing.h" #include "../mm/mmu_decl.h" @@ -80,24 +81,17 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) {int r;
Quoted 7 lines
arch/powerpc/kvm/powerpc.c int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) · line 86
Quoted 3 lines
arch/powerpc/kvm/powerpc.c +3
-0+/* use generic framework to handle need resched and signals */ +if (__xfer_to_guest_mode_work_pending ()) { + r =xfer_to_guest_mode_handle_work (vcpu);there is nothing special you do checking and handling the work. Couldn't you invoke
xfer_to_guest_mode_handle_work()unconditionally?
I followed what was in arch/x86/kvm/x86.c. Since xfer_to_guest_mode_handle_work does the same check it makes sense to call it without checks too.
Will update in v2.
Quoted 17 lines
Quoted 17 lines
arch/powerpc/kvm/powerpc.c +4
-9+if (r) +return r; + } +WARN_ON (irqs_disabled ());hard_irq_disable ();while (true) { -if (need_resched ()) { -local_irq_enable (); -cond_resched (); -hard_irq_disable (); -continue ; - } - -if (signal_pending (current)) { -kvmppc_account_exit (vcpu,SIGNAL_EXITS );
Quoted 8 lines
arch/powerpc/kvm/powerpc.c int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) · line 98
Quoted 3 lines
arch/powerpc/kvm/powerpc.c +0
-3- vcpu->run->exit_reason =KVM_EXIT_INTR ; - r = -EINTR ; -break ;I don't how this works but couldn't
SIGNAL_EXITSvanish now that it isn't updated anymore? The stat itself moves inkvm_handle_signal_exit()to a different counter so it is not lost. The reader just needs to look somewhere else for it.
ok. thanks for pointing out.
AFAIU it is updating the stats mostly. But below could keep the stats happy. I will update that in v2.
if (__xfer_to_guest_mode_work_pending()) {
r = xfer_to_guest_mode_handle_work(vcpu);
+ /* generic framework doesn't update ppc specific stats*/
+ if (r == -EINTR)
+ kvmppc_account_exit(vcpu, SIGNAL_EXITS);
if (r)
return r;Quoted 8 lines
arch/powerpc/kvm/powerpc.c int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) · line 97
Quoted 6 lines
arch/powerpc/kvm/powerpc.c +0
-2- } - vcpu->mode =IN_GUEST_MODE ;/* Sebastian
Re: [PATCH 1/2] powerpc: kvm: use generic transfer to guest mode work
Quoted 12 lines · Shrikanth Hegde
Quoted 10 lines
Quoted 9 lines
arch/powerpc/kvm/book3s_hv.c +1
-1@@ -4901,7 +4901,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, }if (need_resched ()) -cond_resched (); +schedule ();
Quoted 4 lines
Quoted 2 lines
This looks unrelated and odd. I don't why but this should be a
cond_resched()so it can be optimized away on PREEMPT kernels.This is needed, otherwise KVM on powerVM setup gets stuck on preempt=full/lazy.
But this makes no sense. On preempt=full the cond_resched() gets patched out while schedule() doesn't. Okay, this explains the stuck.
On preempt=full need_resched() should not return true because the preemption happens right away. Unless you are in a preempt-disabled or interrupt disabled section. But any of the conditions can't be true because in both cases you can't invoke schedule(). So you must have had a wake up on the local CPU which sets need-resched but the schedule() was delayed for some reason. Once that need-resched bit is observed by a remote CPU then it won't send an interrupt for a scheduling request because it should happen any time soon… This should be fixed.
If you replace the above with preempt_disable(); preempt_enable() then it should also work…
…
Quoted 20 lines
Quoted 10 lines
Quoted 10 lines
arch/powerpc/kvm/powerpc.c +1
-0@@ -34,6 +34,7 @@#endif #include <asm/ultravisor.h> #include <asm/setup.h> +#include <linux/entry-kvm.h> #include "timing.h" #include "../mm/mmu_decl.h" @@ -80,24 +81,17 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu)Quoted 9 lines
arch/powerpc/kvm/powerpc.c int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) · line 86
Quoted 5 lines
arch/powerpc/kvm/powerpc.c +3
-0{int r; +/* use generic framework to handle need resched and signals */ +if (__xfer_to_guest_mode_work_pending ()) { + r =xfer_to_guest_mode_handle_work (vcpu);there is nothing special you do checking and handling the work. Couldn't you invoke
xfer_to_guest_mode_handle_work()unconditionally?
Quoted 2 lines
I followed what was in
arch/x86/kvm/x86.c. Sincexfer_to_guest_mode_handle_workdoes the same check it makes sense to call it without checks too.
Yeah but I guess x86 did some other updates, too.
Quoted 2 lines
Will update in v2.
…
Quoted 19 lines
Quoted 11 lines
arch/powerpc/kvm/powerpc.c int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) · line 98
Quoted 6 lines
arch/powerpc/kvm/powerpc.c +0
-6- -if (signal_pending (current)) { -kvmppc_account_exit (vcpu,SIGNAL_EXITS ); - vcpu->run->exit_reason =KVM_EXIT_INTR ; - r = -EINTR ; -break ;I don't how this works but couldn't
SIGNAL_EXITSvanish now that it isn't updated anymore? The stat itself moves inkvm_handle_signal_exit()to a different counter so it is not lost. The reader just needs to look somewhere else for it.ok. thanks for pointing out.
AFAIU it is updating the stats mostly. But below could keep the stats happy. I will update that in v2.
if (__xfer_to_guest_mode_work_pending()) { r = xfer_to_guest_mode_handle_work(vcpu);
Review
Quoted 5 lines
(quoted diff) +3
-0+ /* generic framework doesn't update ppc specific stats*/ + if (r == -EINTR) + kvmppc_account_exit(vcpu, SIGNAL_EXITS); if (r) return r;
Either that or you rip it out entirely but that is not my call.
Sebastian
Shrikanth Hegde
reviewerRe: [PATCH 1/2] powerpc: kvm: use generic transfer to guest mode work
Quoted 13 lines · Sebastian Andrzej Siewior
Quoted 12 lines · Shrikanth Hegde
Quoted 10 lines
Quoted 9 lines
arch/powerpc/kvm/book3s_hv.c +1
-1@@ -4901,7 +4901,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, }if (need_resched ()) -cond_resched (); +schedule ();
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
This looks unrelated and odd. I don't why but this should be a
cond_resched()so it can be optimized away on PREEMPT kernels.This is needed, otherwise KVM on powerVM setup gets stuck on preempt=full/lazy.
But this makes no sense. On preempt=full the
cond_resched()gets patched out whileschedule()doesn't. Okay, this explains the stuck.
cond_resched works. What you said is right about schedule and preemption models. Initially I had some other code changes and they were causing it get stuck. i retested it.
But looking at the semantics of usage of xfer_to_guest_mode_work I think using schedule is probably right over here. Correct me if i got it all wrong.
on x86:kvm_arch_vcpu_ioctl_run
vcpu_run
for () {
.. run guest..
xfer_to_guest_mode_handle_work
schedule
}on Powerpc: ( taking book3s_hv flavour):kvm_arch_vcpu_ioctl_runkvmppc_vcpu_run_hv *1
do while() {
kvmhv_run_single_vcpu or kvmppc_run_vcpu
-- checking for need_resched and signals and bails out *2
}*1 - checks for need resched and signals before entering guest *2 - checks for need resched and signals while running the guest
This patch is addressing only *1 but it needs to address *2 as well using generic framework. I think it is doable for books3s_hv atleast. (though might need rewrite)
__kvmppc_vcpu_run is a block box to me yet. I think it first makes sense
to move it C and then try use the xfer_to_guest_mode_handle_work.
nick, vaibhav, any idea on __kvmppc_vcpu_run on how is it handling signal pending, and need_resched.
So this is going to need more work specially on *2 and doing that is also key for preempt=lazy/full to work for kvm on powepc. will try to figure out.
Re: [PATCH 1/2] powerpc: kvm: use generic transfer to guest mode work
Quoted 17 lines · Shrikanth Hegde
Quoted 16 lines · Sebastian Andrzej Siewior
Quoted 15 lines · Shrikanth Hegde
Quoted 14 lines
arch/powerpc/kvm/book3s_hv.c int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, · line 4903
Quoted 9 lines
arch/powerpc/kvm/book3s_hv.c +1
-1@@ -4901,7 +4901,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, }if (need_resched ()) -cond_resched (); +schedule ();This looks unrelated and odd. I don't why but this should be a
cond_resched()so it can be optimized away on PREEMPT kernels.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
This is needed, otherwise KVM on powerVM setup gets stuck on preempt=full/lazy.
But this makes no sense. On preempt=full the
cond_resched()gets patched out whileschedule()doesn't. Okay, this explains the stuck.
cond_reschedworks. What you said is right about schedule and preemption models. Initially I had some other code changes and they were causing it get stuck. i retested it.
so it is unrelated then ;)
Quoted 3 lines
But looking at the semantics of usage of
xfer_to_guest_mode_workI think using schedule is probably right over here. Correct me if i got it all wrong.
No, if you do xfer_to_guest_mode_work() then it will invoke schedule() when appropriate. It just the thing in kvmhv_run_single_vcpu() looks odd and might have been duct tape or an accident and could probably be removed.
Quoted 13 lines
on x86:
kvm_arch_vcpu_ioctl_runvcpu_run for () { .. run guest.. xfer_to_guest_mode_handle_work schedule }on Powerpc: ( taking
book3s_hvflavour):kvm_arch_vcpu_ioctl_runkvmppc_vcpu_run_hv*1
Quoted 7 lines
do while() { kvmhv_run_single_vcpu or kvmppc_run_vcpu -- checking for need_resched and signals and bails out *2 }*1 - checks for need resched and signals before entering guest
I don't see the need_resched() check here.
Quoted 6 lines
*2 - checks for need resched and signals while running the guest
This patch is addressing only *1 but it needs to address *2 as well using generic framework. I think it is doable for
books3s_hvatleast. (though might need rewrite)
Quoted 7 lines
__kvmppc_vcpu_runis a block box to me yet. I think it first makes sense
to move it C and then try use thexfer_to_guest_mode_handle_work.
nick, vaibhav, any idea on__kvmppc_vcpu_runon how is it handling signal pending, andneed_resched.So this is going to need more work specially on *2 and doing that is also key for preempt=lazy/full to work for kvm on powepc. will try to figure out.
Okay.
Sebastian
Shrikanth Hegde
reviewerRe: [PATCH 1/2] powerpc: kvm: use generic transfer to guest mode work
Quoted 28 lines · Sebastian Andrzej Siewior
Quoted 17 lines · Shrikanth Hegde
Quoted 16 lines · Sebastian Andrzej Siewior
Quoted 15 lines · Shrikanth Hegde
Quoted 14 lines
arch/powerpc/kvm/book3s_hv.c int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, · line 4903
Quoted 9 lines
arch/powerpc/kvm/book3s_hv.c +1
-1@@ -4901,7 +4901,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, }if (need_resched ()) -cond_resched (); +schedule ();This looks unrelated and odd. I don't why but this should be a
cond_resched()so it can be optimized away on PREEMPT kernels.Quoted 7 lines
Quoted 4 lines
Quoted 1 line
This is needed, otherwise KVM on powerVM setup gets stuck on preempt=full/lazy.
But this makes no sense. On preempt=full the
cond_resched()gets patched out whileschedule()doesn't. Okay, this explains the stuck.
cond_reschedworks. What you said is right about schedule and preemption models. Initially I had some other code changes and they were causing it get stuck. i retested it.so it is unrelated then ;)
Quoted 9 lines
Quoted 3 lines
But looking at the semantics of usage of
xfer_to_guest_mode_workI think using schedule is probably right over here. Correct me if i got it all wrong.No, if you do
xfer_to_guest_mode_work()then it will invokeschedule()when appropriate. It just the thing inkvmhv_run_single_vcpu()looks odd and might have been duct tape or an accident and could probably be removed.
I was wondering xfer_to_guest_mode_work could also call cond_resched instead of schedule since for preempt=full/lazy is preemptible as early as possible right?
Quoted 19 lines
Quoted 19 lines
on x86:
kvm_arch_vcpu_ioctl_runvcpu_run for () { .. run guest.. xfer_to_guest_mode_handle_work schedule }on Powerpc: ( taking
book3s_hvflavour):kvm_arch_vcpu_ioctl_runkvmppc_vcpu_run_hv*1do while() { kvmhv_run_single_vcpu or kvmppc_run_vcpu -- checking for need_resched and signals and bails out *2 }
Quoted 3 lines
Quoted 1 line
*1 - checks for need resched and signals before entering guest
I don't see the
need_resched()check here.
right, i think *2 is critical since it is in a loop. *1 is probably an optimization to skip a few cycles.
Quoted 20 lines
Quoted 6 lines
*2 - checks for need resched and signals while running the guest
This patch is addressing only *1 but it needs to address *2 as well using generic framework. I think it is doable for
books3s_hvatleast. (though might need rewrite)Quoted 7 lines
__kvmppc_vcpu_runis a block box to me yet. I think it first makes sense
to move it C and then try use thexfer_to_guest_mode_handle_work.
nick, vaibhav, any idea on__kvmppc_vcpu_runon how is it handling signal pending, andneed_resched.So this is going to need more work specially on *2 and doing that is also key for preempt=lazy/full to work for kvm on powepc. will try to figure out.
Okay.
Sebastian
Re: [PATCH 1/2] powerpc: kvm: use generic transfer to guest mode work
Quoted 10 lines · Shrikanth Hegde
Quoted 9 lines
Quoted 3 lines
But looking at the semantics of usage of
xfer_to_guest_mode_workI think using schedule is probably right over here. Correct me if i got it all wrong.No, if you do
xfer_to_guest_mode_work()then it will invokeschedule()when appropriate. It just the thing inkvmhv_run_single_vcpu()looks odd and might have been duct tape or an accident and could probably be removed.
Quoted 3 lines
I was wondering
xfer_to_guest_mode_workcould also callcond_reschedinstead of schedule since for preempt=full/lazy is preemptible as early as possible right?
No, I think it is okay. For preempt=full you shouldn't observe the flag in xfer_to_guest_mode_work() so it does not matter.
Sebastian
Shrikanth Hegde
reviewerRe: [PATCH 1/2] powerpc: kvm: use generic transfer to guest mode work
In reply to Sebastian Andrzej SiewiorQuoted 15 lines · Sebastian Andrzej Siewior
Quoted 14 lines · Shrikanth Hegde
Quoted 13 lines · Sebastian Andrzej Siewior
Quoted 12 lines · Shrikanth Hegde
Quoted 10 lines
Quoted 9 lines
arch/powerpc/kvm/book3s_hv.c +1
-1@@ -4901,7 +4901,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, }if (need_resched ()) -cond_resched (); +schedule ();
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
Quoted 2 lines
This looks unrelated and odd. I don't why but this should be a
cond_resched()so it can be optimized away on PREEMPT kernels.This is needed, otherwise KVM on powerVM setup gets stuck on preempt=full/lazy.
When i look back, i think below makes sense, since this is what even the generic framework will do and it would work for all preemption models.
Since this happens in a loop, it is necessary to check for reschedule.
if need_resched()
schedule()
Ideal would be call xfer_to_guest_mode_handle_work in the loop, which would handle the above too.
Quoted 56 lines
Quoted 5 lines
Quoted 2 lines
But this makes no sense. On preempt=full the
cond_resched()gets patched out whileschedule()doesn't. Okay, this explains the stuck.
cond_reschedworks. What you said is right about schedule and preemption models. Initially I had some other code changes and they were causing it get stuck. i retested it.so it is unrelated then ;)
Quoted 3 lines
But looking at the semantics of usage of
xfer_to_guest_mode_workI think using schedule is probably right over here. Correct me if i got it all wrong.No, if you do
xfer_to_guest_mode_work()then it will invokeschedule()when appropriate. It just the thing inkvmhv_run_single_vcpu()looks odd and might have been duct tape or an accident and could probably be removed.Quoted 13 lines
on x86:
kvm_arch_vcpu_ioctl_runvcpu_run for () { .. run guest.. xfer_to_guest_mode_handle_work schedule }on Powerpc: ( taking
book3s_hvflavour):kvm_arch_vcpu_ioctl_runkvmppc_vcpu_run_hv*1Quoted 7 lines
do while() { kvmhv_run_single_vcpu or kvmppc_run_vcpu -- checking for need_resched and signals and bails out *2 }*1 - checks for need resched and signals before entering guest
I don't see the
need_resched()check here.Quoted 6 lines
*2 - checks for need resched and signals while running the guest
This patch is addressing only *1 but it needs to address *2 as well using generic framework. I think it is doable for
books3s_hvatleast. (though might need rewrite)Quoted 7 lines
__kvmppc_vcpu_runis a block box to me yet. I think it first makes sense
to move it C and then try use thexfer_to_guest_mode_handle_work.
nick, vaibhav, any idea on__kvmppc_vcpu_runon how is it handling signal pending, andneed_resched.So this is going to need more work specially on *2 and doing that is also key for preempt=lazy/full to work for kvm on powepc. will try to figure out.
Okay.
Sebastian
Shrikanth Hegde
reviewerPATCH 2/2 powerpc: enable to run posix cpu timers in task context
In reply to Shrikanth HegdeNow that all kvm entry to guest paths handle the task work using the generic framework, enable HAVE_POSIX_CPU_TIMERS_TASK_WORK which allows running posix cpu timers in task context instead of running them in hardirq. This would is a necessary step towards enabling PREEMPT_RT on powerNV systems.
--- arch/powerpc/Kconfig | 1 + 1 file changed, 1 insertion(+)
arch/powerpc/Kconfig +1 -0
@@ -277,6 +277,7 @@ config PPC select HAVE_PERF_EVENTS_NMI if PPC64 select HAVE_PERF_REGS select HAVE_PERF_USER_STACK_DUMP + select HAVE_POSIX_CPU_TIMERS_TASK_WORK select HAVE_RETHOOK if KPROBES select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_RELIABLE_STACKTRACE
Signature · 1 line
2.48.1
Shrikanth Hegde
reviewerQuoted 2 lines · Shrikanth Hegde
From: Gautam Menghani <[email protected]>
I made a mistake while generating the patch. Sorry about that. i will fix it up in next version. Please consider the above as:
From: Shrikanth Hegde <[email protected]>
Quoted 24 lines
This is an effort to use the generic kvm infra which handles check for
need_resched, handling signals etc. i.exfer_to_guest_mode_handle_work.kvm guests boots and runs stress-ng CPU stressor on PowerVM and on PowerNV. preempt=full and preempt=lazy was tested on PowerNV and in both cases the KVM guest boots and runs stress-ng CPU stressor.
Please let me know if any specific testing to be done.
Kept the patches separate since they differ functionally, but kept them is a series since 2nd patch depends on functionality of 1st. Also this could help in git bisect.
This is based on tip/master
Shrikanth Hegde (2):
powerpc: kvm: use generic transfer to guest mode work
powerpc: enable to run posix cpu timers in task contextarch/powerpc/Kconfig | 2 ++ arch/powerpc/kvm/book3s_hv.c | 13 +++++++------ arch/powerpc/kvm/powerpc.c | 22 ++++++++-------------- 3 files changed, 17 insertions(+), 20 deletions(-)