Skip to contentSkip to conversation

KVM: powerpc: Use generic xfer to guest work function

PATCH Superseded Vishal Chourasia 2 participants 4 messages Jul 9

Subsystems (3)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover KVM: powerpc: Use generic xfer to guest work function

This series fixes a KVM scheduling bug on Book3S HV where a guest VM under a cpu.max bandwidth limit can run arbitrarily past its quota and then appear frozen for minutes afterwards.

== Problem ==

Since commit 2cd571245b43 ("sched/fair: Add related data structure for task based throttle"), merged in v6.18, CFS bandwidth throttling no longer dequeues a task directly. Instead it queues a task_work item via task_work_add(..., TWA_RESUME), sets TIF_NOTIFY_RESUME, and relies on that work running on the return path to actually dequeue the task.

The powerpc KVM run loops only test TIF_SIGPENDING and TIF_NEED_RESCHED before re-entering the guest; TIF_NOTIFY_RESUME is never checked. For a CPU-bound guest that generates few KVM exits back to userspace, the vCPU thread never returns to user mode, so the deferred throttle task_work never runs. The guest keeps running unchecked while its runtime_remaining goes increasingly negative, and once it finally does exit to userspace it is legitimately throttled for minutes while the accrued debt is repaid at the bandwidth-timer replenishment rate.

The generic xfer-to-guest-mode infrastructure (commit 935ace2fb5cc, "entry: Provide infrastructure for work before transitioning to guest mode") exists precisely to handle this kind of work before each guest entry. A full trace-backed root-cause analysis was posted with v1 [2].

== Fix ==

Opt powerpc KVM into VIRT_XFER_TO_GUEST_WORK and use the generic xfer_to_guest_mode helpers to check for and handle pending guest-mode work (reschedule, signals, and TIF_NOTIFY_RESUME task_work such as the deferred CFS throttle) on every guest re-entry:

- Book3S HV: both run loops — kvmhv_run_single_vcpu() for POWER9+ and kvmppc_run_vcpu() for pre-POWER9.
- Book3S PR and BookE: the common kvmppc_prepare_to_enter(), which likewise only checked need_resched()/signal_pending().

== Changes from v2 ==
- Add xfer_to_guest_mode_prepare() check for pending rcuog wakeup before entering guest
- While at it patch all exit points for kvmhv_run_single_vcpu() via done label matching the corresponding trace_kvmppc_run_vcpu_enter() with trace_kvmppc_run_vcpu_exit() tracepoint.
- Include posix cpu timer enablement patch by Shrikanth as part of this series

== Changes from v1 ==

- Extend the fix beyond Book3S HV to the shared powerpc KVM entry path:
also convert the common kvmppc_prepare_to_enter() used by Book3S PR and BookE. (Shrikanth Shegde)
- Move "select VIRT_XFER_TO_GUEST_WORK" from KVM_BOOK3S_64_HV up to the common "config KVM" so every powerpc KVM variant gets the infrastructure.
- Drop the redundant signal_pending() recheck and its sigpend label in kvmhv_run_single_vcpu(); xfer_to_guest_mode_work_pending() is a superset of it.
- Preserve the E500 CONFIG_KVM_EXIT_TIMING histogram on the signal path via an explicit kvmppc_set_exit_type(SIGNAL_EXITS).

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/

Shrikanth Hegde (1):
powerpc: enable to run posix cpu timers in task context

Vishal Chourasia (1):
KVM: powerpc: Use generic xfer to guest work function

 arch/powerpc/Kconfig         |  1 +
 arch/powerpc/kvm/Kconfig     |  1 +
 arch/powerpc/kvm/book3s_hv.c | 39 +++++++++++++++++++++++-------------
 arch/powerpc/kvm/powerpc.c   | 37 +++++++++++++++++++++++++---------
 4 files changed, 55 insertions(+), 23 deletions(-)
Signature · 1 line
2.54.0
#1 lore raw

PATCH v3 1/2 KVM: powerpc: Use generic xfer to guest work function

Since commit 2cd571245b43 ("sched/fair: Add related data structure for task based throttle") in v6.18, CFS bandwidth throttling no longer dequeues a task directly; it queues task_work via TWA_RESUME and sets TIF_NOTIFY_RESUME, relying on that work running before the task returns to guest/user mode. The powerpc KVM run loops only checked for reschedule and signals, never TIF_NOTIFY_RESUME, so the deferred throttle never ran while a vCPU stayed in the run loop: a CPU-bound guest that rarely exits to userspace ran far past its cpu.max quota and then appeared frozen for minutes while the accrued throttle debt was repaid.

Use the generic infrastructure to check for and handle pending work before transitioning into guest mode, replacing the open-coded need_resched() and cond_resched() checks in the Book3S HV run loops and in the common kvmppc_prepare_to_enter() used by the Book3S PR and BookE run loops. The redundant signal_pending() recheck (and its sigpend label) in kvmhv_run_single_vcpu() is also dropped, as xfer_to_guest_mode_work_pending() is a superset of it.

This picks up handling for TIF_NOTIFY_RESUME, which was previously ignored, meaning task work will now be correctly handled on every guest re-entry.

Selecting VIRT_XFER_TO_GUEST_WORK disables RCU's last-resort self-IPI fallback for vCPU tasks (see rcu_irq_work_resched()), which on nohz_full CPUs was what forced a reschedule for deferred rcuog wakeups queued right before guest entry. Take over that obligation the same way x86 and s390 do: call xfer_to_guest_mode_prepare() with IRQs disabled immediately before the final xfer_to_guest_mode_work_pending() check at each guest-entry gate (kvmhv_run_single_vcpu(), kvmppc_run_core() and kvmppc_prepare_to_enter()).

In kvmppc_prepare_to_enter(), IRQs are now disabled with local_irq_disable() before hard_irq_disable(): on 32-bit, hard_irq_disable() is a raw MSR[EE] clear that bypasses the lockdep/irq-tracing state, and the strict xfer_to_guest_mode helpers assert that IRQs are seen as disabled. This also allows upgrading the racy __xfer_to_guest_mode_work_pending() check to the asserting variant, as this loop is the terminal gate for the PR and BookE paths.

In kvmhv_run_single_vcpu(), the -EINTR exit and the pre-existing kvmhv_setup_mmu() failure exit now leave via the done label instead of returning directly, keeping the run_vcpu enter/exit tracepoints balanced and vcpu->arch.ret consistent with the returned value.

In kvmppc_prepare_to_enter() the generic helper accounts the signal exit (vcpu->stat.signal_exits and KVM_EXIT_INTR) but does not set the exit type, so kvmppc_set_exit_type(SIGNAL_EXITS) is retained on the signal path to preserve the E500 CONFIG_KVM_EXIT_TIMING histogram; it is a no-op otherwise.

---
 arch/powerpc/kvm/Kconfig     |  1 +
 arch/powerpc/kvm/book3s_hv.c | 39 +++++++++++++++++++++++-------------
 arch/powerpc/kvm/powerpc.c   | 37 +++++++++++++++++++++++++---------
 3 files changed, 54 insertions(+), 23 deletions(-)
arch/powerpc/kvm/Kconfig +1 -0
index 9a0d1c1aca6c..b6bc2fc86dca 100644
@@ -22,6 +22,7 @@ config KVM
 	select KVM_COMMON
 	select KVM_VFIO
 	select HAVE_KVM_IRQ_BYPASS
+	select VIRT_XFER_TO_GUEST_WORK
 
 config KVM_BOOK3S_HANDLER
 	bool
arch/powerpc/kvm/book3s_hv.c +25 -14
index 61dbeea317f3..3cfe9a7be9c6 100644
@@ -3853,7 +3853,8 @@ static noinline void kvmppc_run_core(struct kvmppc_vcore *vc)
 	 */
 	local_irq_disable();
 	hard_irq_disable();
-	if (lazy_irq_pending() || need_resched() ||
+	xfer_to_guest_mode_prepare();
+	if (lazy_irq_pending() || xfer_to_guest_mode_work_pending() ||
 	    recheck_signals_and_mmu(&core_info)) {
 		local_irq_enable();
 		vc->vcore_state = VCORE_INACTIVE;
@@ -4824,10 +4825,16 @@ static int kvmppc_run_vcpu(struct kvm_vcpu *vcpu)
 		vc->runner = vcpu;
 		if (n_ceded == vc->n_runnable) {
 			kvmppc_vcore_blocked(vc);
-		} else if (need_resched()) {
+		} else if (__xfer_to_guest_mode_work_pending()) {
 			kvmppc_vcore_preempt(vc);
-			/* Let something else run */
-			cond_resched_lock(&vc->lock);
+			/*
+			 * Let something else run. The raw helper is used as
+			 * signal exits are accounted by this path already;
+			 * it may schedule(), so drop the vcore lock.
+			 */
+			spin_unlock(&vc->lock);
+			xfer_to_guest_mode_handle_work();
+			spin_lock(&vc->lock);
 			if (vc->vcore_state == VCORE_PREEMPT)
 				kvmppc_vcore_end_preempt(vc);
 		} else {
@@ -4895,12 +4902,16 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,
 			run->exit_reason = KVM_EXIT_FAIL_ENTRY;
 			run->fail_entry.hardware_entry_failure_reason = 0;
 			vcpu->arch.ret = r;
-			return r;
+			goto done;
 		}
 	}
 
-	if (need_resched())
-		cond_resched();
+	r = kvm_xfer_to_guest_mode_handle_work(vcpu);
+	if (r) {
+		/* -EINTR: signal pending, exit to userspace (KVM_EXIT_INTR) */
+		vcpu->arch.ret = r;
+		goto done;
+	}
 
 	kvmppc_update_vpas(vcpu);
 
@@ -4914,9 +4925,13 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,
 
 	vcpu->arch.state = KVMPPC_VCPU_RUNNABLE;
 
-	if (signal_pending(current))
-		goto sigpend;
-	if (need_resched() || !kvm->arch.mmu_ready)
+	xfer_to_guest_mode_prepare();
+
+	/*
+	 * IRQs are disabled here, so on pending work bail to the outer loop,
+	 * which handles it via kvm_xfer_to_guest_mode_handle_work() above.
+	 */
+	if (xfer_to_guest_mode_work_pending() || !kvm->arch.mmu_ready)
 		goto out;
 
 	vcpu->cpu = pcpu;
@@ -5068,10 +5083,6 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit,
 
 	return vcpu->arch.ret;
 
- sigpend:
-	vcpu->stat.signal_exits++;
-	run->exit_reason = KVM_EXIT_INTR;
-	vcpu->arch.ret = -EINTR;
  out:
 	vcpu->cpu = -1;
 	vcpu->arch.thread_cpu = -1;
arch/powerpc/kvm/powerpc.c +28 -9
index 00302399fc37..be5e48ae0c6c 100644
@@ -81,23 +81,41 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu)
 	int r;
 
 	WARN_ON(irqs_disabled());
+	/*
+	 * local_irq_disable() first: on 32-bit, hard_irq_disable() alone is a
+	 * raw MSR[EE] clear that bypasses the lockdep/irq-tracing state, and
+	 * the xfer_to_guest_mode helpers assert IRQs are seen as disabled.
+	 */
+	local_irq_disable();
 	hard_irq_disable();
 
 	while (true) {
-		if (need_resched()) {
+		xfer_to_guest_mode_prepare();
+
+		if (xfer_to_guest_mode_work_pending()) {
+			/*
+			 * The helper must run with IRQs enabled and may
+			 * schedule(). On a pending signal it returns -EINTR
+			 * with run->exit_reason and vcpu->stat.signal_exits
+			 * already set, so just return to userspace.
+			 */
 			local_irq_enable();
-			cond_resched();
+			r = kvm_xfer_to_guest_mode_handle_work(vcpu);
+			local_irq_disable();
 			hard_irq_disable();
+			if (r) {
+				/*
+				 * The generic helper does not set the exit
+				 * type; record it for the E500
+				 * CONFIG_KVM_EXIT_TIMING histogram (a no-op
+				 * otherwise).
+				 */
+				kvmppc_set_exit_type(vcpu, SIGNAL_EXITS);
+				break;
+			}
 			continue;
 		}
 
-		if (signal_pending(current)) {
-			kvmppc_account_exit(vcpu, SIGNAL_EXITS);
-			vcpu->run->exit_reason = KVM_EXIT_INTR;
-			r = -EINTR;
-			break;
-		}
-
 		vcpu->mode = IN_GUEST_MODE;
 
 		/*
@@ -116,6 +134,7 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu)
 			local_irq_enable();
 			trace_kvm_check_requests(vcpu);
 			r = kvmppc_core_check_requests(vcpu);
+			local_irq_disable();
 			hard_irq_disable();
 			if (r > 0)
 				continue;
Signature · 1 line
2.54.0
#2 lore raw
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Lockdep assertion failure in xfer_to_guest_mode_prepare() due to mismatched IRQ state on BookE…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Lockdep assertion failure in xfer_to_guest_mode_prepare() due to mismatched IRQ state on BookE architecture after emulated MTMSRWE. --

--- Patch [1]: [PATCH v3 1/2] KVM: powerpc: Use generic xfer to guest work function --- commit ad7243cc79a6e34d4784ae75e8ea35ce946a8343
Author: Vishal Chourasia <[email protected]>

KVM: powerpc: Use generic xfer to guest work function

This commit updates powerpc KVM run loops to use generic infrastructure for checking and handling pending task_work before guest entry. It replaces open-coded need_resched() checks and adds xfer_to_guest_mode_prepare() to disable IRQs and handle TIF_NOTIFY_RESUME properly.

Quoted 9 lines
arch/powerpc/kvm/powerpc.c +1 -0
index 00302399fc37b..be5e48ae0c6c2 100644
@@ -81,23 +81,41 @@ int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu)
 	int r;
 
 	WARN_ON(irqs_disabled());
+	/*

arch/powerpc/kvm/powerpc.c int kvmppc_prepare_to_enter(struct kvm_vcpu *vcpu) · line 93

Quoted 10 lines
arch/powerpc/kvm/powerpc.c +6 -1
+	 * local_irq_disable() first: on 32-bit, hard_irq_disable() alone is a
+	 * raw MSR[EE] clear that bypasses the lockdep/irq-tracing state, and
+	 * the xfer_to_guest_mode helpers assert IRQs are seen as disabled.
+	 */
+	local_irq_disable();
 	hard_irq_disable();
 
 	while (true) {
-		if (need_resched()) {
+		xfer_to_guest_mode_prepare();

[Severity: High]
Can this trigger a lockdep assertion failure on BookE architecture?

When a BookE guest executes mtmsr to set the MSR_WE bit, it enters an idle state and invokes kvmppc_core_prepare_to_enter(). This function enables lockdep IRQs but disables hardware IRQs using a macro that bypasses lockdep on 32-bit:

arch/powerpc/kvm/booke.c:kvmppc_core_prepare_to_enter() {
    ...
    if (vcpu->arch.shared->msr & MSR_WE) {
        local_irq_enable();
        kvm_vcpu_halt(vcpu);
        hard_irq_disable();
        ...
        r = 1;
    }
}

If kvmppc_core_prepare_to_enter() returns 1, the while loop later in kvmppc_prepare_to_enter() continues:

arch/powerpc/kvm/powerpc.c:kvmppc_prepare_to_enter() {
    ...
    if (kvmppc_core_prepare_to_enter(vcpu)) {
        /* interrupts got enabled in between, so we
           are back at square 1 */
        continue;
    }
}

This continues back to the start of the loop and invokes the newly added xfer_to_guest_mode_prepare() check. Since hard_irq_disable() bypassed the lockdep state update, lockdep still believes IRQs are enabled. Will this cause xfer_to_guest_mode_prepare() to fail its lockdep_assert_irqs_disabled() check and trigger a splat?

[ ... ]

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#3 lore raw

PATCH v3 2/2 powerpc: enable to run posix cpu timers in task context

In reply to Vishal Chourasia

From: Shrikanth Hegde <[email protected]>

From: Shrikanth Hegde <[email protected]>

Now that all kvm entry to guest paths handle the task work using the generic framework, enable HAVE_POSIX_CPU_TIMERS_TASK_WORK which allows running posix cpu timers in task context instead of running them in hardirq. This would is a necessary step towards enabling PREEMPT_RT on powerNV systems.

---
 arch/powerpc/Kconfig | 1 +
 1 file changed, 1 insertion(+)
arch/powerpc/Kconfig +1 -0
index f7ce5fff81f0..51555a0b1a26 100644
@@ -281,6 +281,7 @@ config PPC
 	select HAVE_PERF_REGS
 	select HAVE_PERF_USER_STACK_DUMP
 	select HAVE_PREEMPT_DYNAMIC_KEY
+	select HAVE_POSIX_CPU_TIMERS_TASK_WORK
 	select HAVE_RETHOOK			if KPROBES
 	select HAVE_REGS_AND_STACK_ACCESS_API
 	select HAVE_RELIABLE_STACKTRACE
Signature · 1 line
2.54.0