Skip to contentSkip to conversation

powerpc: kvm: generic framework and run posix timers in task context

PATCH In review Shrikanth Hegde 2 participants 12 messages Apr 21, 2025 – Apr 29, 2025

Subsystems (2)
lore
Share & tools
lore

Shrikanth Hegde

author reviewer
#0 lore raw

cover powerpc: kvm: generic framework and run posix timers in task context

From: Gautam Menghani <[email protected]>

This is an effort to use the generic kvm infra which handles check for need_resched, handling signals etc. i.e xfer_to_guest_mode_handle_work.

kvm guests boots and runs stress-ng CPU stressor on PowerVM and on PowerNV. preempt=full and preempt=lazy was tested on PowerNV and in both cases the KVM guest boots and runs stress-ng CPU stressor.

Please let me know if any specific testing to be done.

Kept the patches separate since they differ functionally, but kept them is a series since 2nd patch depends on functionality of 1st. Also this could help in git bisect.

This is based on tip/master

Shrikanth Hegde (2):
powerpc: kvm: use generic transfer to guest mode work
powerpc: enable to run posix cpu timers in task context

 arch/powerpc/Kconfig         |  2 ++
 arch/powerpc/kvm/book3s_hv.c | 13 +++++++------
 arch/powerpc/kvm/powerpc.c   | 22 ++++++++--------------
 3 files changed, 17 insertions(+), 20 deletions(-)
Signature · 1 line
2.48.1
author reviewer
#1 lore raw

PATCH 1/2 powerpc: kvm: use generic transfer to guest mode work

There is generic entry framework to handle signals and check for reschedule etc before entering the guest. Use that framework for powerpc.

Advantages:
- Less code duplication.
- powerpc kvm now understands NR and NR_lazy bits.
- powerpc can enable HAVE_POSIX_CPU_TIMERS_TASK_WORK, currently the powerpc/kvm code doesn't handle TIF_NOTIFY_RESUME.

Testing: No splats seen in below cases.
- Booted KVM guest on PowerVM and PowerNV systems.
- Ran stress-ng CPU stressors in each above case.
- On PowerNV host, tried preempt=lazy/full and run stress-ng CPU stressor in the KVM guest.

---
 arch/powerpc/Kconfig         |  1 +
 arch/powerpc/kvm/book3s_hv.c | 13 +++++++------
 arch/powerpc/kvm/powerpc.c   | 22 ++++++++--------------
 3 files changed, 16 insertions(+), 20 deletions(-)
arch/powerpc/Kconfig +1 -0
index 6722625a4..83807ae44 100644
@@ -299,6 +299,7 @@ config PPC
 	select IRQ_DOMAIN
 	select IRQ_FORCED_THREADING
 	select KASAN_VMALLOC			if KASAN && EXECMEM
+	select KVM_XFER_TO_GUEST_WORK
 	select LOCK_MM_AND_FIND_VMA
 	select MMU_GATHER_PAGE_SIZE
 	select MMU_GATHER_RCU_TABLE_FREE
arch/powerpc/kvm/book3s_hv.c +7 -6
index 19f4d298d..123539642 100644
@@ -80,8 +80,8 @@
 #include <asm/ultravisor.h>
 #include <asm/dtl.h>
 #include <asm/plpar_wrappers.h>
-
 #include <trace/events/ipi.h>
+#include <linux/entry-kvm.h>
 
 #include "book3s.h"
 #include "book3s_hv.h"
@@ -4901,7 +4901,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,
 	}
 
 	if (need_resched())
-		cond_resched();
+		schedule();
 
 	kvmppc_update_vpas(vcpu);
 
@@ -5097,10 +5097,11 @@ static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu)
 		return -EINVAL;
 	}
 
-	/* No need to go into the guest when all we'll do is come back out */
-	if (signal_pending(current)) {
-		run->exit_reason = KVM_EXIT_INTR;
-		return -EINTR;
+	/* use generic frameworks to handle signals, need_resched  */
+	if (__xfer_to_guest_mode_work_pending()) {
+		r = xfer_to_guest_mode_handle_work(vcpu);
+		if (r)
+			return r;
 	}
 
 #ifdef CONFIG_PPC_TRANSACTIONAL_MEM
arch/powerpc/kvm/powerpc.c +8 -14
index 153587741..4ff334532 100644
@@ -34,6 +34,7 @@
 #endif
 #include <asm/ultravisor.h>
 #include <asm/setup.h>
+#include <linux/entry-kvm.h>
 
 #include "timing.h"
 #include "../mm/mmu_decl.h"
@@ -80,24 +81,17 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu)
 {
 	int r;
 
+	/* use generic framework to handle need resched and signals */
+	if (__xfer_to_guest_mode_work_pending()) {
+		r = xfer_to_guest_mode_handle_work(vcpu);
+		if (r)
+			return r;
+	}
+
 	WARN_ON(irqs_disabled());
 	hard_irq_disable();
 
 	while (true) {
-		if (need_resched()) {
-			local_irq_enable();
-			cond_resched();
-			hard_irq_disable();
-			continue;
-		}
-
-		if (signal_pending(current)) {
-			kvmppc_account_exit(vcpu, SIGNAL_EXITS);
-			vcpu->run->exit_reason = KVM_EXIT_INTR;
-			r = -EINTR;
-			break;
-		}
-
 		vcpu->mode = IN_GUEST_MODE;
 
 		/*
Signature · 1 line
2.48.1
#2 lore raw

Re: [PATCH 1/2] powerpc: kvm: use generic transfer to guest mode work

Quoted 15 lines · Shrikanth Hegde
arch/powerpc/kvm/book3s_hv.c +1 -1
index 19f4d298d..123539642 100644
@@ -80,8 +80,8 @@
 #include <asm/ultravisor.h>
 #include <asm/dtl.h>
 #include <asm/plpar_wrappers.h>
-
 #include <trace/events/ipi.h>
+#include <linux/entry-kvm.h>
 
 #include "book3s.h"
 #include "book3s_hv.h"

arch/powerpc/kvm/book3s_hv.c int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, · line 4904

Quoted 5 lines
arch/powerpc/kvm/book3s_hv.c +1 -1
@@ -4901,7 +4901,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,
 	}
 
 	if (need_resched())
-		cond_resched();
+		schedule();

This looks unrelated and odd. I don't why but this should be a cond_resched() so it can be optimized away on PREEMPT kernels.

arch/powerpc/kvm/book3s_hv.c static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu) · line 5102

Quoted 13 lines
arch/powerpc/kvm/book3s_hv.c +3 -4
 	kvmppc_update_vpas(vcpu);
 
@@ -5097,10 +5097,11 @@ static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu)
 		return -EINVAL;
 	}
 
-	/* No need to go into the guest when all we'll do is come back out */
-	if (signal_pending(current)) {
-		run->exit_reason = KVM_EXIT_INTR;
-		return -EINTR;
+	/* use generic frameworks to handle signals, need_resched  */
+	if (__xfer_to_guest_mode_work_pending()) {
+		r = xfer_to_guest_mode_handle_work(vcpu);

This could be unconditional.

Quoted 18 lines
arch/powerpc/kvm/book3s_hv.c +2 -0
+		if (r)
+			return r;
 	}
 
 #ifdef CONFIG_PPC_TRANSACTIONAL_MEM
arch/powerpc/kvm/powerpc.c +1 -0
index 153587741..4ff334532 100644
@@ -34,6 +34,7 @@
 #endif
 #include <asm/ultravisor.h>
 #include <asm/setup.h>
+#include <linux/entry-kvm.h>
 
 #include "timing.h"
 #include "../mm/mmu_decl.h"

arch/powerpc/kvm/powerpc.c int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) · line 86

Quoted 6 lines
arch/powerpc/kvm/powerpc.c +3 -0
@@ -80,24 +81,17 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu)
 {
 	int r;
 
+	/* use generic framework to handle need resched and signals */
+	if (__xfer_to_guest_mode_work_pending()) {
+		r = xfer_to_guest_mode_handle_work(vcpu);

there is nothing special you do checking and handling the work. Couldn't you invoke xfer_to_guest_mode_handle_work() unconditionally?

Quoted 10 lines
arch/powerpc/kvm/powerpc.c +4 -2
+		if (r)
+			return r;
+	}
+
 	WARN_ON(irqs_disabled());
 	hard_irq_disable();
 
 	while (true) {
-		if (need_resched()) {
-			local_irq_enable();

arch/powerpc/kvm/powerpc.c int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) · line 98

Quoted 10 lines
arch/powerpc/kvm/powerpc.c +0 -10
-			cond_resched();
-			hard_irq_disable();
-			continue;
-		}
-
-		if (signal_pending(current)) {
-			kvmppc_account_exit(vcpu, SIGNAL_EXITS);
-			vcpu->run->exit_reason = KVM_EXIT_INTR;
-			r = -EINTR;
-			break;

I don't how this works but couldn't SIGNAL_EXITS vanish now that it isn't updated anymore? The stat itself moves in kvm_handle_signal_exit() to a different counter so it is not lost. The reader just needs to look somewhere else for it.

arch/powerpc/kvm/powerpc.c int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) · line 97

Quoted 5 lines
arch/powerpc/kvm/powerpc.c +0 -2
-		}
-
 		vcpu->mode = IN_GUEST_MODE;
 
 		/*

Sebastian

author reviewer
#3 lore raw

Re: [PATCH 1/2] powerpc: kvm: use generic transfer to guest mode work

On 4/24/25 20:12, Sebastian Andrzej Siewior wrote:

Thanks Sebastian for taking a look.

Quoted 18 lines · Shrikanth Hegde
Quoted 17 lines
arch/powerpc/kvm/book3s_hv.c +1 -1
index 19f4d298d..123539642 100644
@@ -80,8 +80,8 @@
  #include <asm/ultravisor.h>
  #include <asm/dtl.h>
  #include <asm/plpar_wrappers.h>
-
  #include <trace/events/ipi.h>
+#include <linux/entry-kvm.h>
 
  #include "book3s.h"
  #include "book3s_hv.h"
@@ -4901,7 +4901,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,
  	}
 
Quoted 8 lines

arch/powerpc/kvm/book3s_hv.c int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, · line 4904

Quoted 3 lines
arch/powerpc/kvm/book3s_hv.c +1 -1
  	if (need_resched())
-		cond_resched();
+		schedule();

This looks unrelated and odd. I don't why but this should be a cond_resched() so it can be optimized away on PREEMPT kernels.

This is needed, otherwise KVM on powerVM setup gets stuck on preempt=full/lazy.

Quoted 37 lines
Quoted 4 lines
arch/powerpc/kvm/book3s_hv.c +0 -0
  	kvmppc_update_vpas(vcpu);
 

arch/powerpc/kvm/book3s_hv.c static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu) · line 5102

Quoted 10 lines
arch/powerpc/kvm/book3s_hv.c +3 -4
@@ -5097,10 +5097,11 @@ static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu)
  		return -EINVAL;
  	}
 
-	/* No need to go into the guest when all we'll do is come back out */
-	if (signal_pending(current)) {
-		run->exit_reason = KVM_EXIT_INTR;
-		return -EINTR;
+	/* use generic frameworks to handle signals, need_resched  */
+	if (__xfer_to_guest_mode_work_pending()) {
+		r = xfer_to_guest_mode_handle_work(vcpu);

This could be unconditional.

Quoted 21 lines
arch/powerpc/kvm/book3s_hv.c +2 -0
+		if (r)
+			return r;
  	}
 
  #ifdef CONFIG_PPC_TRANSACTIONAL_MEM
arch/powerpc/kvm/powerpc.c +1 -0
index 153587741..4ff334532 100644
@@ -34,6 +34,7 @@
  #endif
  #include <asm/ultravisor.h>
  #include <asm/setup.h>
+#include <linux/entry-kvm.h>
 
  #include "timing.h"
  #include "../mm/mmu_decl.h"
@@ -80,24 +81,17 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu)
  {
  	int r;
 
Quoted 7 lines

arch/powerpc/kvm/powerpc.c int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) · line 86

Quoted 3 lines
arch/powerpc/kvm/powerpc.c +3 -0
+	/* use generic framework to handle need resched and signals */
+	if (__xfer_to_guest_mode_work_pending()) {
+		r = xfer_to_guest_mode_handle_work(vcpu);

there is nothing special you do checking and handling the work. Couldn't you invoke xfer_to_guest_mode_handle_work() unconditionally?

I followed what was in arch/x86/kvm/x86.c. Since xfer_to_guest_mode_handle_work does the same check it makes sense to call it without checks too.

Will update in v2.

Quoted 17 lines
Quoted 17 lines
arch/powerpc/kvm/powerpc.c +4 -9
+		if (r)
+			return r;
+	}
+
  	WARN_ON(irqs_disabled());
  	hard_irq_disable();
 
  	while (true) {
-		if (need_resched()) {
-			local_irq_enable();
-			cond_resched();
-			hard_irq_disable();
-			continue;
-		}
-
-		if (signal_pending(current)) {
-			kvmppc_account_exit(vcpu, SIGNAL_EXITS);
Quoted 8 lines

arch/powerpc/kvm/powerpc.c int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) · line 98

Quoted 3 lines
arch/powerpc/kvm/powerpc.c +0 -3
-			vcpu->run->exit_reason = KVM_EXIT_INTR;
-			r = -EINTR;
-			break;

I don't how this works but couldn't SIGNAL_EXITS vanish now that it isn't updated anymore? The stat itself moves in kvm_handle_signal_exit() to a different counter so it is not lost. The reader just needs to look somewhere else for it.

ok. thanks for pointing out.

AFAIU it is updating the stats mostly. But below could keep the stats happy. I will update that in v2.

        if (__xfer_to_guest_mode_work_pending()) {
                r = xfer_to_guest_mode_handle_work(vcpu);
+               /* generic framework doesn't update ppc specific stats*/
+               if (r == -EINTR)
+                       kvmppc_account_exit(vcpu, SIGNAL_EXITS);
                if (r)
                        return r;
Quoted 8 lines

arch/powerpc/kvm/powerpc.c int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) · line 97

Quoted 6 lines
arch/powerpc/kvm/powerpc.c +0 -2
-		}
-
  		vcpu->mode = IN_GUEST_MODE;
 
  		/*

Sebastian

#4 lore raw

Re: [PATCH 1/2] powerpc: kvm: use generic transfer to guest mode work

Quoted 12 lines · Shrikanth Hegde
Quoted 10 lines
Quoted 9 lines
arch/powerpc/kvm/book3s_hv.c +1 -1
index 19f4d298d..123539642 100644
@@ -4901,7 +4901,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,
  	}
  	if (need_resched())
-		cond_resched();
+		schedule();
Quoted 4 lines
Quoted 2 lines

This looks unrelated and odd. I don't why but this should be a cond_resched() so it can be optimized away on PREEMPT kernels.

This is needed, otherwise KVM on powerVM setup gets stuck on preempt=full/lazy.

But this makes no sense. On preempt=full the cond_resched() gets patched out while schedule() doesn't. Okay, this explains the stuck.

On preempt=full need_resched() should not return true because the preemption happens right away. Unless you are in a preempt-disabled or interrupt disabled section. But any of the conditions can't be true because in both cases you can't invoke schedule(). So you must have had a wake up on the local CPU which sets need-resched but the schedule() was delayed for some reason. Once that need-resched bit is observed by a remote CPU then it won't send an interrupt for a scheduling request because it should happen any time soon… This should be fixed.

If you replace the above with preempt_disable(); preempt_enable() then it should also work…

…

Quoted 20 lines
Quoted 10 lines
Quoted 10 lines
arch/powerpc/kvm/powerpc.c +1 -0
@@ -34,6 +34,7 @@
  #endif
  #include <asm/ultravisor.h>
  #include <asm/setup.h>
+#include <linux/entry-kvm.h>
  #include "timing.h"
  #include "../mm/mmu_decl.h"
@@ -80,24 +81,17 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu)
Quoted 9 lines

arch/powerpc/kvm/powerpc.c int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) · line 86

Quoted 5 lines
arch/powerpc/kvm/powerpc.c +3 -0
  {
  	int r;
+	/* use generic framework to handle need resched and signals */
+	if (__xfer_to_guest_mode_work_pending()) {
+		r = xfer_to_guest_mode_handle_work(vcpu);

there is nothing special you do checking and handling the work. Couldn't you invoke xfer_to_guest_mode_handle_work() unconditionally?

Quoted 2 lines

I followed what was in arch/x86/kvm/x86.c. Since xfer_to_guest_mode_handle_work does the same check it makes sense to call it without checks too.

Yeah but I guess x86 did some other updates, too.

Quoted 2 lines

Will update in v2.

…

Quoted 19 lines
Quoted 11 lines

arch/powerpc/kvm/powerpc.c int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) · line 98

Quoted 6 lines
arch/powerpc/kvm/powerpc.c +0 -6
-
-		if (signal_pending(current)) {
-			kvmppc_account_exit(vcpu, SIGNAL_EXITS);
-			vcpu->run->exit_reason = KVM_EXIT_INTR;
-			r = -EINTR;
-			break;

I don't how this works but couldn't SIGNAL_EXITS vanish now that it isn't updated anymore? The stat itself moves in kvm_handle_signal_exit() to a different counter so it is not lost. The reader just needs to look somewhere else for it.

ok. thanks for pointing out.

AFAIU it is updating the stats mostly. But below could keep the stats happy. I will update that in v2.

        if (__xfer_to_guest_mode_work_pending()) {
                r = xfer_to_guest_mode_handle_work(vcpu);

Review

Quoted 5 lines
(quoted diff) +3 -0
+               /* generic framework doesn't update ppc specific stats*/
+               if (r == -EINTR)
+                       kvmppc_account_exit(vcpu, SIGNAL_EXITS);
                if (r)
                        return r;

Either that or you rip it out entirely but that is not my call.

Sebastian

author reviewer
#5 lore raw

Re: [PATCH 1/2] powerpc: kvm: use generic transfer to guest mode work

Quoted 13 lines · Sebastian Andrzej Siewior
Quoted 12 lines · Shrikanth Hegde
Quoted 10 lines
Quoted 9 lines
arch/powerpc/kvm/book3s_hv.c +1 -1
index 19f4d298d..123539642 100644
@@ -4901,7 +4901,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,
   	}
   	if (need_resched())
-		cond_resched();
+		schedule();
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

This looks unrelated and odd. I don't why but this should be a cond_resched() so it can be optimized away on PREEMPT kernels.

This is needed, otherwise KVM on powerVM setup gets stuck on preempt=full/lazy.

But this makes no sense. On preempt=full the cond_resched() gets patched out while schedule() doesn't. Okay, this explains the stuck.

cond_resched works. What you said is right about schedule and preemption models. Initially I had some other code changes and they were causing it get stuck. i retested it.

But looking at the semantics of usage of xfer_to_guest_mode_work I think using schedule is probably right over here. Correct me if i got it all wrong.

on x86:
kvm_arch_vcpu_ioctl_run

	vcpu_run
		for () {
			.. run guest..
			xfer_to_guest_mode_handle_work
				schedule
		}

on Powerpc: ( taking book3s_hv flavour):
kvm_arch_vcpu_ioctl_run
kvmppc_vcpu_run_hv *1

	do while() {
		kvmhv_run_single_vcpu or kvmppc_run_vcpu
			-- checking for need_resched and signals and bails out *2
	}

*1 - checks for need resched and signals before entering guest *2 - checks for need resched and signals while running the guest

This patch is addressing only *1 but it needs to address *2 as well using generic framework. I think it is doable for books3s_hv atleast. (though might need rewrite)

__kvmppc_vcpu_run is a block box to me yet. I think it first makes sense
to move it C and then try use the xfer_to_guest_mode_handle_work.
nick, vaibhav, any idea on __kvmppc_vcpu_run on how is it handling signal pending, and need_resched.

So this is going to need more work specially on *2 and doing that is also key for preempt=lazy/full to work for kvm on powepc. will try to figure out.

#6 lore raw

Re: [PATCH 1/2] powerpc: kvm: use generic transfer to guest mode work

Quoted 17 lines · Shrikanth Hegde
Quoted 16 lines · Sebastian Andrzej Siewior
Quoted 15 lines · Shrikanth Hegde
Quoted 14 lines

arch/powerpc/kvm/book3s_hv.c int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, · line 4903

Quoted 9 lines
arch/powerpc/kvm/book3s_hv.c +1 -1
index 19f4d298d..123539642 100644
@@ -4901,7 +4901,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,
   	}
   	if (need_resched())
-		cond_resched();
+		schedule();

This looks unrelated and odd. I don't why but this should be a cond_resched() so it can be optimized away on PREEMPT kernels.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

This is needed, otherwise KVM on powerVM setup gets stuck on preempt=full/lazy.

But this makes no sense. On preempt=full the cond_resched() gets patched out while schedule() doesn't. Okay, this explains the stuck.

cond_resched works. What you said is right about schedule and preemption models. Initially I had some other code changes and they were causing it get stuck. i retested it.

so it is unrelated then ;)

Quoted 3 lines

But looking at the semantics of usage of xfer_to_guest_mode_work I think using schedule is probably right over here. Correct me if i got it all wrong.

No, if you do xfer_to_guest_mode_work() then it will invoke schedule() when appropriate. It just the thing in kvmhv_run_single_vcpu() looks odd and might have been duct tape or an accident and could probably be removed.

Quoted 13 lines

on x86:
kvm_arch_vcpu_ioctl_run

	vcpu_run
		for () {
			.. run guest..
			xfer_to_guest_mode_handle_work
				schedule
		}

on Powerpc: ( taking book3s_hv flavour):
kvm_arch_vcpu_ioctl_run
kvmppc_vcpu_run_hv *1

Quoted 7 lines
	do while() {
		kvmhv_run_single_vcpu or kvmppc_run_vcpu
			-- checking for need_resched and signals and bails out *2
	}

*1 - checks for need resched and signals before entering guest

I don't see the need_resched() check here.

Quoted 6 lines

*2 - checks for need resched and signals while running the guest

This patch is addressing only *1 but it needs to address *2 as well using generic framework. I think it is doable for books3s_hv atleast. (though might need rewrite)

Quoted 7 lines

__kvmppc_vcpu_run is a block box to me yet. I think it first makes sense
to move it C and then try use the xfer_to_guest_mode_handle_work.
nick, vaibhav, any idea on __kvmppc_vcpu_run on how is it handling signal pending, and need_resched.

So this is going to need more work specially on *2 and doing that is also key for preempt=lazy/full to work for kvm on powepc. will try to figure out.

Okay.

Sebastian

author reviewer
#7 lore raw

Re: [PATCH 1/2] powerpc: kvm: use generic transfer to guest mode work

Quoted 28 lines · Sebastian Andrzej Siewior
Quoted 17 lines · Shrikanth Hegde
Quoted 16 lines · Sebastian Andrzej Siewior
Quoted 15 lines · Shrikanth Hegde
Quoted 14 lines

arch/powerpc/kvm/book3s_hv.c int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, · line 4903

Quoted 9 lines
arch/powerpc/kvm/book3s_hv.c +1 -1
index 19f4d298d..123539642 100644
@@ -4901,7 +4901,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,
    	}
    	if (need_resched())
-		cond_resched();
+		schedule();

This looks unrelated and odd. I don't why but this should be a cond_resched() so it can be optimized away on PREEMPT kernels.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

This is needed, otherwise KVM on powerVM setup gets stuck on preempt=full/lazy.

But this makes no sense. On preempt=full the cond_resched() gets patched out while schedule() doesn't. Okay, this explains the stuck.

cond_resched works. What you said is right about schedule and preemption models. Initially I had some other code changes and they were causing it get stuck. i retested it.

so it is unrelated then ;)

Quoted 9 lines
Quoted 3 lines

But looking at the semantics of usage of xfer_to_guest_mode_work I think using schedule is probably right over here. Correct me if i got it all wrong.

No, if you do xfer_to_guest_mode_work() then it will invoke schedule() when appropriate. It just the thing in kvmhv_run_single_vcpu() looks odd and might have been duct tape or an accident and could probably be removed.

I was wondering xfer_to_guest_mode_work could also call cond_resched instead of schedule since for preempt=full/lazy is preemptible as early as possible right?

Quoted 19 lines
Quoted 19 lines

on x86:
kvm_arch_vcpu_ioctl_run

	vcpu_run
		for () {
			.. run guest..
			xfer_to_guest_mode_handle_work
				schedule
		}

on Powerpc: ( taking book3s_hv flavour):
kvm_arch_vcpu_ioctl_run
kvmppc_vcpu_run_hv *1

	do while() {
		kvmhv_run_single_vcpu or kvmppc_run_vcpu
			-- checking for need_resched and signals and bails out *2
	}
Quoted 3 lines
Quoted 1 line

*1 - checks for need resched and signals before entering guest

I don't see the need_resched() check here.

right, i think *2 is critical since it is in a loop. *1 is probably an optimization to skip a few cycles.

Quoted 20 lines
Quoted 6 lines

*2 - checks for need resched and signals while running the guest

This patch is addressing only *1 but it needs to address *2 as well using generic framework. I think it is doable for books3s_hv atleast. (though might need rewrite)

Quoted 7 lines

__kvmppc_vcpu_run is a block box to me yet. I think it first makes sense
to move it C and then try use the xfer_to_guest_mode_handle_work.
nick, vaibhav, any idea on __kvmppc_vcpu_run on how is it handling signal pending, and need_resched.

So this is going to need more work specially on *2 and doing that is also key for preempt=lazy/full to work for kvm on powepc. will try to figure out.

Okay.

Sebastian

#8 lore raw

Re: [PATCH 1/2] powerpc: kvm: use generic transfer to guest mode work

Quoted 10 lines · Shrikanth Hegde
Quoted 9 lines
Quoted 3 lines

But looking at the semantics of usage of xfer_to_guest_mode_work I think using schedule is probably right over here. Correct me if i got it all wrong.

No, if you do xfer_to_guest_mode_work() then it will invoke schedule() when appropriate. It just the thing in kvmhv_run_single_vcpu() looks odd and might have been duct tape or an accident and could probably be removed.

Quoted 3 lines

I was wondering xfer_to_guest_mode_work could also call cond_resched instead of schedule since for preempt=full/lazy is preemptible as early as possible right?

No, I think it is okay. For preempt=full you shouldn't observe the flag in xfer_to_guest_mode_work() so it does not matter.

Sebastian

author reviewer
#9 lore raw

Re: [PATCH 1/2] powerpc: kvm: use generic transfer to guest mode work

In reply to Sebastian Andrzej Siewior
Quoted 15 lines · Sebastian Andrzej Siewior
Quoted 14 lines · Shrikanth Hegde
Quoted 13 lines · Sebastian Andrzej Siewior
Quoted 12 lines · Shrikanth Hegde
Quoted 10 lines
Quoted 9 lines
arch/powerpc/kvm/book3s_hv.c +1 -1
index 19f4d298d..123539642 100644
@@ -4901,7 +4901,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,
    	}
    	if (need_resched())
-		cond_resched();
+		schedule();
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
Quoted 2 lines

This looks unrelated and odd. I don't why but this should be a cond_resched() so it can be optimized away on PREEMPT kernels.

This is needed, otherwise KVM on powerVM setup gets stuck on preempt=full/lazy.

When i look back, i think below makes sense, since this is what even the generic framework will do and it would work for all preemption models.

Since this happens in a loop, it is necessary to check for reschedule.

if need_resched()
schedule()

Ideal would be call xfer_to_guest_mode_handle_work in the loop, which would handle the above too.

Quoted 56 lines
Quoted 5 lines
Quoted 2 lines

But this makes no sense. On preempt=full the cond_resched() gets patched out while schedule() doesn't. Okay, this explains the stuck.

cond_resched works. What you said is right about schedule and preemption models. Initially I had some other code changes and they were causing it get stuck. i retested it.

so it is unrelated then ;)

Quoted 3 lines

But looking at the semantics of usage of xfer_to_guest_mode_work I think using schedule is probably right over here. Correct me if i got it all wrong.

No, if you do xfer_to_guest_mode_work() then it will invoke schedule() when appropriate. It just the thing in kvmhv_run_single_vcpu() looks odd and might have been duct tape or an accident and could probably be removed.

Quoted 13 lines

on x86:
kvm_arch_vcpu_ioctl_run

	vcpu_run
		for () {
			.. run guest..
			xfer_to_guest_mode_handle_work
				schedule
		}

on Powerpc: ( taking book3s_hv flavour):
kvm_arch_vcpu_ioctl_run
kvmppc_vcpu_run_hv *1

Quoted 7 lines
	do while() {
		kvmhv_run_single_vcpu or kvmppc_run_vcpu
			-- checking for need_resched and signals and bails out *2
	}

*1 - checks for need resched and signals before entering guest

I don't see the need_resched() check here.

Quoted 6 lines

*2 - checks for need resched and signals while running the guest

This patch is addressing only *1 but it needs to address *2 as well using generic framework. I think it is doable for books3s_hv atleast. (though might need rewrite)

Quoted 7 lines

__kvmppc_vcpu_run is a block box to me yet. I think it first makes sense
to move it C and then try use the xfer_to_guest_mode_handle_work.
nick, vaibhav, any idea on __kvmppc_vcpu_run on how is it handling signal pending, and need_resched.

So this is going to need more work specially on *2 and doing that is also key for preempt=lazy/full to work for kvm on powepc. will try to figure out.

Okay.

Sebastian

author reviewer
#10 lore raw

PATCH 2/2 powerpc: enable to run posix cpu timers in task context

In reply to Shrikanth Hegde

Now that all kvm entry to guest paths handle the task work using the generic framework, enable HAVE_POSIX_CPU_TIMERS_TASK_WORK which allows running posix cpu timers in task context instead of running them in hardirq. This would is a necessary step towards enabling PREEMPT_RT on powerNV systems.

---
 arch/powerpc/Kconfig | 1 +
 1 file changed, 1 insertion(+)
arch/powerpc/Kconfig +1 -0
index 83807ae44..f42fa4181 100644
@@ -277,6 +277,7 @@ config PPC
 	select HAVE_PERF_EVENTS_NMI		if PPC64
 	select HAVE_PERF_REGS
 	select HAVE_PERF_USER_STACK_DUMP
+	select HAVE_POSIX_CPU_TIMERS_TASK_WORK
 	select HAVE_RETHOOK			if KPROBES
 	select HAVE_REGS_AND_STACK_ACCESS_API
 	select HAVE_RELIABLE_STACKTRACE
Signature · 1 line
2.48.1
author reviewer
#11 lore raw
In reply to Shrikanth Hegde
Quoted 2 lines · Shrikanth Hegde

From: Gautam Menghani <[email protected]>

I made a mistake while generating the patch. Sorry about that. i will fix it up in next version. Please consider the above as:

From: Shrikanth Hegde <[email protected]>

Quoted 24 lines

This is an effort to use the generic kvm infra which handles check for need_resched, handling signals etc. i.e xfer_to_guest_mode_handle_work.

kvm guests boots and runs stress-ng CPU stressor on PowerVM and on PowerNV. preempt=full and preempt=lazy was tested on PowerNV and in both cases the KVM guest boots and runs stress-ng CPU stressor.

Please let me know if any specific testing to be done.

Kept the patches separate since they differ functionally, but kept them is a series since 2nd patch depends on functionality of 1st. Also this could help in git bisect.

This is based on tip/master

Shrikanth Hegde (2):
powerpc: kvm: use generic transfer to guest mode work
powerpc: enable to run posix cpu timers in task context

  arch/powerpc/Kconfig         |  2 ++
  arch/powerpc/kvm/book3s_hv.c | 13 +++++++------
  arch/powerpc/kvm/powerpc.c   | 22 ++++++++--------------
  3 files changed, 17 insertions(+), 20 deletions(-)