KVM: PPC: Fixes for Book3S HV HPT locking and paired-single decoding
Subsystems (4)
cover KVM: PPC: Fixes for Book3S HV HPT locking and paired-single decoding
This series addresses a few bug fixes across KVM PPC Book3S HV locking/synchronization and paired-single instruction decoding:
Patches 1 & 2 fix synchronization and preemption issues introduced in commit 6165d5dd99db ("KVM: PPC: Book3S HV: add virtual mode handlers for HPT hcalls and page faults"):
- Patch 1 adds SRCU read lock protection when walking memslots during virtual-mode HPT hcalls to prevent use-after-free races with concurrent memslot updates.
- Patch 2 adds preempt_disable() around virtual-mode HPTE bit-lock holders and raw spinlock usage to prevent CPU stalls and deadlocks on preemption.
Patch 3 fixes paired-single D-form instruction emulation:
- Patch 3 fixes get_d_signext() to correctly extract the full 12-bit D displacement field and perform proper two's-complement sign extension.
Testing:
========
All test kernel builds were compiled with CONFIG_DEBUG_ATOMIC_SLEEP=y. The following scenarios were verified:
1. Power9 PowerNV (L0) in Radix mode:
- Booted L0 host with kernel containing all 3 patches.
- Booted KVM guests in both Radix and Hash modes.
- Ran kernel build workload inside guests — no errors observed.
- Booted L1 guest with the same kernel — booted and ran cleanly.
2. Power9 PowerNV (L0) in Hash mode:
- Booted L0 host with kernel containing all 3 patches.
- Booted KVM guest in Hash mode.
- Ran kernel build workload inside guest — no errors observed.
- Booted L1 nested guest with the same kernel — booted and ran cleanly.
3. Power10 LPAR (L1):
- Booted Power10 LPAR with the patched kernel.
- Booted KVM guest and ran workloads — no errors observed.
Amit Machhiwal (3):
KVM: PPC: Book3S HV: Add SRCU protection for virtual-mode HPT hcalls
KVM: PPC: Book3S HV: Add preempt_disable() around virtual-mode HPTE
bit-lock users
KVM: PPC: Fix get_d_signext() 12-bit displacement for paired-single
D-form
arch/powerpc/kvm/book3s_hv.c | 76 +++++++++++++----------- arch/powerpc/kvm/book3s_paired_singles.c | 7 +-- 2 files changed, 42 insertions(+), 41 deletions(-)
base-commit: 72d3fcf802c45d00b300f25b848a93c3a2bd7c7e
Signature · 1 line
2.54.0 (Apple Git-157)
PATCH RESEND 1/3 KVM: PPC: Book3S HV: Add SRCU protection for virtual-mode HPT hcalls
The virtual-mode HPT hcall handlers (H_ENTER, H_REMOVE, H_BULK_REMOVE, H_CLEAR_REF, H_CLEAR_MOD) call into book3s_hv_rm_mmu.c, which accesses memslots via kvm_memslots_raw(). kvm_memslots_raw() uses rcu_dereference_raw_check() to bypass SRCU lockdep annotation checking. This is safe in real mode because the entire guest entry/exit is wrapped in srcu_read_lock/unlock inside kvmppc_run_core() and kvmhv_run_single_vcpu().
However, since commit 6165d5dd99db ("KVM: PPC: Book3S HV: add virtual mode handlers for HPT hcalls and page faults"), these same handlers are also executed in virtual mode via kvmppc_pseries_do_hcall(), which runs after SRCU has already been released on guest exit.
A concurrent KVM_SET_USER_MEMORY_REGION deletion or move can therefore call synchronize_srcu_expedited() — which does not wait for this thread — and then kfree(slot) and vfree(slot->arch.rmap) while the hcall handler still holds a raw pointer to the memslot. lock_rmap() then writes to freed vmalloc memory, leading to use-after-free and memory corruption. Because kvm_memslots_raw() suppresses lockdep checks, this race is entirely silent.
Fix this by factoring out the 7 HPT hcall handlers (H_REMOVE, H_ENTER, H_READ, H_CLEAR_MOD, H_CLEAR_REF, H_PROTECT, H_BULK_REMOVE) into a helper function, kvmppc_pseries_do_hpt_hcall(), and wrapping its call site in srcu_read_lock(&kvm->srcu) / srcu_read_unlock(&kvm->srcu, idx). Targeting only the HPT hcalls avoids wrapping non-HPT hcalls that sleep (such as H_CONFER, H_REGISTER_VPA, H_PAGE_INIT) or handlers that already acquire SRCU internally (such as H_RTAS).
--- arch/powerpc/kvm/book3s_hv.c | 70 ++++++++++++++++++------------------ 1 file changed, 35 insertions(+), 35 deletions(-)
arch/powerpc/kvm/book3s_hv.c +35 -35
@@ -1159,6 +1159,38 @@ static long kvmppc_h_rpt_invalidate(struct kvm_vcpu *vcpu,return H_SUCCESS ; } +static long kvmppc_pseries_do_hpt_hcall (struct kvm_vcpu *vcpu,unsigned long req) +{ +switch (req) { +case H_REMOVE : +return kvmppc_h_remove (vcpu,kvmppc_get_gpr (vcpu,4 ), +kvmppc_get_gpr (vcpu,5 ), +kvmppc_get_gpr (vcpu,6 )); +case H_ENTER : +return kvmppc_h_enter (vcpu,kvmppc_get_gpr (vcpu,4 ), +kvmppc_get_gpr (vcpu,5 ), +kvmppc_get_gpr (vcpu,6 ), +kvmppc_get_gpr (vcpu,7 )); +case H_READ : +return kvmppc_h_read (vcpu,kvmppc_get_gpr (vcpu,4 ), +kvmppc_get_gpr (vcpu,5 )); +case H_CLEAR_MOD : +return kvmppc_h_clear_mod (vcpu,kvmppc_get_gpr (vcpu,4 ), +kvmppc_get_gpr (vcpu,5 )); +case H_CLEAR_REF : +return kvmppc_h_clear_ref (vcpu,kvmppc_get_gpr (vcpu,4 ), +kvmppc_get_gpr (vcpu,5 )); +case H_PROTECT : +return kvmppc_h_protect (vcpu,kvmppc_get_gpr (vcpu,4 ), +kvmppc_get_gpr (vcpu,5 ), +kvmppc_get_gpr (vcpu,6 )); +case H_BULK_REMOVE : +return kvmppc_h_bulk_remove (vcpu); + } + +return H_FUNCTION ; +} +int kvmppc_pseries_do_hcall (struct kvm_vcpu *vcpu) {struct kvm *kvm = vcpu->kvm;
@@ -1174,47 +1206,15 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu)switch (req) {case H_REMOVE : - ret =kvmppc_h_remove (vcpu,kvmppc_get_gpr (vcpu,4 ), -kvmppc_get_gpr (vcpu,5 ), -kvmppc_get_gpr (vcpu,6 )); -if (ret ==H_TOO_HARD ) -return RESUME_HOST ; -break ;case H_ENTER : - ret =kvmppc_h_enter (vcpu,kvmppc_get_gpr (vcpu,4 ), -kvmppc_get_gpr (vcpu,5 ), -kvmppc_get_gpr (vcpu,6 ), -kvmppc_get_gpr (vcpu,7 )); -if (ret ==H_TOO_HARD ) -return RESUME_HOST ; -break ;case H_READ : - ret =kvmppc_h_read (vcpu,kvmppc_get_gpr (vcpu,4 ), -kvmppc_get_gpr (vcpu,5 )); -if (ret ==H_TOO_HARD ) -return RESUME_HOST ; -break ;case H_CLEAR_MOD : - ret =kvmppc_h_clear_mod (vcpu,kvmppc_get_gpr (vcpu,4 ), -kvmppc_get_gpr (vcpu,5 )); -if (ret ==H_TOO_HARD ) -return RESUME_HOST ; -break ;case H_CLEAR_REF : - ret =kvmppc_h_clear_ref (vcpu,kvmppc_get_gpr (vcpu,4 ), -kvmppc_get_gpr (vcpu,5 )); -if (ret ==H_TOO_HARD ) -return RESUME_HOST ; -break ;case H_PROTECT : - ret =kvmppc_h_protect (vcpu,kvmppc_get_gpr (vcpu,4 ), -kvmppc_get_gpr (vcpu,5 ), -kvmppc_get_gpr (vcpu,6 )); -if (ret ==H_TOO_HARD ) -return RESUME_HOST ; -break ;case H_BULK_REMOVE : - ret =kvmppc_h_bulk_remove (vcpu); + idx =srcu_read_lock (&kvm->srcu); + ret =kvmppc_pseries_do_hpt_hcall (vcpu, req); +srcu_read_unlock (&kvm->srcu, idx);if (ret ==H_TOO_HARD )return RESUME_HOST ;break ;
Signature · 1 line
2.54.0 (Apple Git-157)
Re: [RESEND PATCH 1/3] KVM: PPC: Book3S HV: Add SRCU protection for virtual-mode HPT hcalls
Quoted 29 lines · Amit Machhiwal
The virtual-mode HPT hcall handlers (
H_ENTER,H_REMOVE,H_BULK_REMOVE,H_CLEAR_REF,H_CLEAR_MOD) call intobook3s_hv_rm_mmu.c, which accesses memslots viakvm_memslots_raw().kvm_memslots_raw()usesrcu_dereference_raw_check()to bypass SRCU lockdep annotation checking. This is safe in real mode because the entire guest entry/exit is wrapped in srcu_read_lock/unlock insidekvmppc_run_core()andkvmhv_run_single_vcpu().However, since commit
6165d5dd99db("KVM: PPC: Book3S HV: add virtual mode handlers for HPT hcalls and page faults"), these same handlers are also executed in virtual mode viakvmppc_pseries_do_hcall(), which runs after SRCU has already been released on guest exit.A concurrent
KVM_SET_USER_MEMORY_REGIONdeletion or move can therefore callsynchronize_srcu_expedited()— which does not wait for this thread — and then kfree(slot) andvfree(slot->arch.rmap)while the hcall handler still holds a raw pointer to the memslot.lock_rmap()then writes to freed vmalloc memory, leading to use-after-free and memory corruption. Becausekvm_memslots_raw()suppresses lockdep checks, this race is entirely silent.Fix this by factoring out the 7 HPT hcall handlers (
H_REMOVE,H_ENTER,H_READ,H_CLEAR_MOD,H_CLEAR_REF,H_PROTECT,H_BULK_REMOVE) into a helper function,kvmppc_pseries_do_hpt_hcall(), and wrapping its call site insrcu_read_lock(&kvm->srcu)/srcu_read_unlock(&kvm->srcu, idx). Targeting only the HPT hcalls avoids wrapping non-HPT hcalls that sleep (such asH_CONFER,H_REGISTER_VPA,H_PAGE_INIT) or handlers that already acquire SRCU internally (such asH_RTAS).
Quoted 7 lines
Fixes 6165d5dd99db ("KVM: PPC: Book3S HV: add virtual mode handlers for HPT hcalls and page faults")Cc [email protected] # v5.14+Suggested-by Ritesh Harjani (IBM) <[email protected]>Signed-off-by Amit Machhiwal <[email protected]>--- arch/powerpc/kvm/book3s_hv.c | 70 ++++++++++++++++++------------------ 1 file changed, 35 insertions(+), 35 deletions(-)
I had looked at this path in depth earlier itself. LGTM. Please feel free to add:
PATCH RESEND 2/3 KVM: PPC: Book3S HV: Add preempt_disable() around virtual-mode HPTE bit-lock users
In reply to Amit Machhiwalkvmppc_hv_find_lock_hpte() requires virtual-mode callers to run with preemption disabled, because it can return with HPTE_V_HVLOCK still held until the caller later unlocks the HPTE. Existing virtual-mode callers in book3s_64_mmu_hv.c already follow that rule.
Two virtual-mode call paths do not:
1. kvmppc_handle_exit_hv() calls kvmppc_hpte_hv_fault() for hash-mode data-side and instruction-side faults after guest exit, with preemption already enabled. kvmppc_hpte_hv_fault() calls kvmppc_hv_find_lock_hpte() directly.
2. kvmppc_pseries_do_hcall() executes virtual-mode handlers for HPT hcalls via kvmppc_pseries_do_hpt_hcall() with preemption enabled. The handlers for H_ENTER, H_REMOVE, H_READ, H_CLEAR_MOD, H_CLEAR_REF, H_PROTECT, and H_BULK_REMOVE spin on try_lock_hpte() or lock_rmap(), and H_ENTER also reaches kvmppc_do_h_enter(), which uses arch_spin_lock() on kvm->mmu_lock. That raw lock choice is intentional because kvmppc_do_h_enter() is also used by real-mode callers, so the correct fix is to establish the proper preemption context at the virtual-mode caller boundary.
If a vCPU thread is preempted while holding HPTE_V_HVLOCK, any other thread on the same CPU trying to acquire the same bit-lock can spin indefinitely. The same applies to lock_rmap() users and to the H_ENTER path while holding the raw mmu_lock.
Fix this by adding preempt_disable()/preempt_enable() pairs around the two kvmppc_hpte_hv_fault() call sites in kvmppc_handle_exit_hv(), and around the kvmppc_pseries_do_hpt_hcall() invocation in kvmppc_pseries_do_hcall(). Targeting only the HPT hcall helper avoids wrapping non-HPT hcalls (such as H_RTAS, H_CONFER, H_REGISTER_VPA) that perform page faults, blocking memory allocations, or context switches.
--- arch/powerpc/kvm/book3s_hv.c | 6 ++++++ 1 file changed, 6 insertions(+)
arch/powerpc/kvm/book3s_hv.c +6 -0
@@ -1212,9 +1212,11 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu)case H_CLEAR_REF :case H_PROTECT :case H_BULK_REMOVE : +preempt_disable (); idx =srcu_read_lock (&kvm->srcu); ret =kvmppc_pseries_do_hpt_hcall (vcpu, req);srcu_read_unlock (&kvm->srcu, idx); +preempt_enable ();if (ret ==H_TOO_HARD )return RESUME_HOST ;break ;
@@ -1834,8 +1836,10 @@ static int kvmppc_handle_exit_hv(struct kvm_vcpu *vcpu,else vsid = vcpu->arch.fault_gpa; +preempt_disable (); err =kvmppc_hpte_hv_fault (vcpu, vcpu->arch.fault_dar, vsid, vcpu->arch.fault_dsisr, true); +preempt_enable ();if (err ==0 ) { r =RESUME_GUEST ; }else if (err ==-1 || err ==-2 ) {
@@ -1881,8 +1885,10 @@ static int kvmppc_handle_exit_hv(struct kvm_vcpu *vcpu,else vsid = vcpu->arch.fault_gpa; +preempt_disable (); err =kvmppc_hpte_hv_fault (vcpu, vcpu->arch.fault_dar, vsid, vcpu->arch.fault_dsisr, false); +preempt_enable ();if (err ==0 ) { r =RESUME_GUEST ; }else if (err ==-1 ) {
Signature · 1 line
2.54.0 (Apple Git-157)
Re: [RESEND PATCH 2/3] KVM: PPC: Book3S HV: Add preempt_disable() around virtual-mode HPTE bit-lock users
Quoted 5 lines · Amit Machhiwal
kvmppc_hv_find_lock_hpte()requires virtual-mode callers to run with preemption disabled, because it can return withHPTE_V_HVLOCKstill held until the caller later unlocks the HPTE. Existing virtual-mode callers inbook3s_64_mmu_hv.calready follow that rule.
That make sense. LGTM. Please feel free to add:
Shrikanth Hegde
reviewerRe: [RESEND PATCH 2/3] KVM: PPC: Book3S HV: Add preempt_disable() around virtual-mode HPTE bit-lock users
In reply to Amit MachhiwalQuoted 46 lines · Amit Machhiwal
kvmppc_hv_find_lock_hpte()requires virtual-mode callers to run with preemption disabled, because it can return withHPTE_V_HVLOCKstill held until the caller later unlocks the HPTE. Existing virtual-mode callers inbook3s_64_mmu_hv.calready follow that rule.Two virtual-mode call paths do not:
1.
kvmppc_handle_exit_hv()callskvmppc_hpte_hv_fault()for hash-mode data-side and instruction-side faults after guest exit, with preemption already enabled.kvmppc_hpte_hv_fault()callskvmppc_hv_find_lock_hpte()directly.2.
kvmppc_pseries_do_hcall()executes virtual-mode handlers for HPT hcalls viakvmppc_pseries_do_hpt_hcall()with preemption enabled. The handlers forH_ENTER,H_REMOVE,H_READ,H_CLEAR_MOD,H_CLEAR_REF,H_PROTECT, andH_BULK_REMOVEspin ontry_lock_hpte()orlock_rmap(), andH_ENTERalso reacheskvmppc_do_h_enter(), which usesarch_spin_lock()onkvm->mmu_lock. That raw lock choice is intentional becausekvmppc_do_h_enter()is also used by real-mode callers, so the correct fix is to establish the proper preemption context at the virtual-mode caller boundary.If a vCPU thread is preempted while holding
HPTE_V_HVLOCK, any other thread on the same CPU trying to acquire the same bit-lock can spin indefinitely. The same applies tolock_rmap()users and to theH_ENTERpath while holding the rawmmu_lock.Fix this by adding preempt_disable()/preempt_enable() pairs around the two
kvmppc_hpte_hv_fault()call sites inkvmppc_handle_exit_hv(), and around thekvmppc_pseries_do_hpt_hcall()invocation inkvmppc_pseries_do_hcall(). Targeting only the HPT hcall helper avoids wrapping non-HPT hcalls (such asH_RTAS,H_CONFER,H_REGISTER_VPA) that perform page faults, blocking memory allocations, or context switches.Fixes 6165d5dd99db ("KVM: PPC: Book3S HV: add virtual mode handlers for HPT hcalls and page faults")Cc [email protected] # v5.14+Signed-off-by Amit Machhiwal <[email protected]>--- arch/powerpc/kvm/book3s_hv.c | 6 ++++++ 1 file changed, 6 insertions(+)arch/powerpc/kvm/book3s_hv.c +0
-0
arch/powerpc/kvm/book3s_hv.c int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) · line 1216
Quoted 5 lines
arch/powerpc/kvm/book3s_hv.c +1
-0@@ -1212,9 +1212,11 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu)case H_CLEAR_REF :case H_PROTECT :case H_BULK_REMOVE : +preempt_disable (); idx =srcu_read_lock (&kvm->srcu);
Please add a comment above kvmppc_pseries_do_hpt_hcall, saying preemption is disabled and one cannot make any blocking call.
Quoted 18 lines
arch/powerpc/kvm/book3s_hv.c +3
-0ret =kvmppc_pseries_do_hpt_hcall (vcpu, req);srcu_read_unlock (&kvm->srcu, idx); +preempt_enable ();if (ret ==H_TOO_HARD )return RESUME_HOST ;break ;@@ -1834,8 +1836,10 @@ static int kvmppc_handle_exit_hv(struct kvm_vcpu *vcpu,else vsid = vcpu->arch.fault_gpa; +preempt_disable (); err =kvmppc_hpte_hv_fault (vcpu, vcpu->arch.fault_dar, vsid, vcpu->arch.fault_dsisr, true); +preempt_enable ();if (err ==0 ) { r =RESUME_GUEST ; }else if (err ==-1 || err ==-2 ) {
arch/powerpc/kvm/book3s_hv.c static int kvmppc_handle_exit_hv(struct kvm_vcpu *vcpu, · line 1894
Quoted 10 lines
arch/powerpc/kvm/book3s_hv.c +2
-0@@ -1881,8 +1885,10 @@ static int kvmppc_handle_exit_hv(struct kvm_vcpu *vcpu,else vsid = vcpu->arch.fault_gpa; +preempt_disable (); err =kvmppc_hpte_hv_fault (vcpu, vcpu->arch.fault_dar, vsid, vcpu->arch.fault_dsisr, false); +preempt_enable ();if (err ==0 ) { r =RESUME_GUEST ; }else if (err ==-1 ) {
Please check if below is possible case.
https://sashiko.dev/#/patchset/20260928113704.48912-4-amachhiw%40linux.ibm.com
Re: [RESEND PATCH 2/3] KVM: PPC: Book3S HV: Add preempt_disable() around virtual-mode HPTE bit-lock users
Hi Shrikanth,
Thanks for taking a look at this patch. Please find my responses inline.
Quoted 8 lines · Shrikanth Hegde
Quoted 5 lines · Amit Machhiwal
kvmppc_hv_find_lock_hpte()requires virtual-mode callers to run with preemption disabled, because it can return withHPTE_V_HVLOCKstill held until the caller later unlocks the HPTE. Existing virtual-mode callers inbook3s_64_mmu_hv.calready follow that rule.
<..snip..>
Quoted 5 lines
Quoted 5 lines
arch/powerpc/kvm/book3s_hv.c +0
-0@@ -1212,9 +1212,11 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu)
Quoted 8 lines
arch/powerpc/kvm/book3s_hv.c int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) · line 1216
Quoted 5 lines
arch/powerpc/kvm/book3s_hv.c +1
-0case H_CLEAR_REF :case H_PROTECT :case H_BULK_REMOVE : +preempt_disable (); idx =srcu_read_lock (&kvm->srcu);Please add a comment above
kvmppc_pseries_do_hpt_hcall, saying preemption is disabled and one cannot make any blocking call.
Sure, will do.
Quoted 30 lines
Quoted 18 lines
arch/powerpc/kvm/book3s_hv.c +3
-0ret =kvmppc_pseries_do_hpt_hcall (vcpu, req);srcu_read_unlock (&kvm->srcu, idx); +preempt_enable ();if (ret ==H_TOO_HARD )return RESUME_HOST ;break ;@@ -1834,8 +1836,10 @@ static int kvmppc_handle_exit_hv(struct kvm_vcpu *vcpu,else vsid = vcpu->arch.fault_gpa; +preempt_disable (); err =kvmppc_hpte_hv_fault (vcpu, vcpu->arch.fault_dar, vsid, vcpu->arch.fault_dsisr, true); +preempt_enable ();if (err ==0 ) { r =RESUME_GUEST ; }else if (err ==-1 || err ==-2 ) {arch/powerpc/kvm/book3s_hv.c static int kvmppc_handle_exit_hv(struct kvm_vcpu *vcpu, · line 1893
Quoted 9 lines
arch/powerpc/kvm/book3s_hv.c +2
-0@@ -1881,8 +1885,10 @@ static int kvmppc_handle_exit_hv(struct kvm_vcpu *vcpu,else vsid = vcpu->arch.fault_gpa; +preempt_disable (); err =kvmppc_hpte_hv_fault (vcpu, vcpu->arch.fault_dar, vsid, vcpu->arch.fault_dsisr, false); +preempt_enable ();if (err ==0 ) { r =RESUME_GUEST ; }else if (err ==-1 ) {Please check if below is possible case.
Thank you for raising this point and bringing Sashiko AI's review comment to attention. After conducting a deep-dive audit of the preemption paths and the virtual-mode locking context, it looks like Sashiko's finding is legitimate, and extremely subtle.
The Deadlock Mechanism:
=======================
In this patch we added preempt_disable() / preempt_enable() pairs exclusively around the guest vCPU HPT hcall paths (kvmppc_pseries_do_hpt_hcall) and page fault paths (kvmppc_hpte_hv_fault) in virtual mode to satisfy the HPTE locking preemption contract.
However, several host-side virtual-mode paths still acquire and hold the HPTE bit-lock (HPTE_V_HVLOCK) with preemption enabled. Specifically:
- While MMU notifier paths (unmapping, aging) and the dirty-log path are safe (since they are called with kvm->mmu_lock held, which disables preemption), the memslot flushing path (kvmppc_core_flush_memslot_hv() -> kvm_unmap_rmapp()) runs with only the slots_arch_lock mutex held, leaving preemption fully enabled.
If a host-side thread executing kvm_unmap_rmapp() successfully acquires the HPTE_V_HVLOCK on CPU X and is preempted, a guest vCPU thread subsequently scheduled on CPU X will enter the hcall or fault path, disable preemption (via the current change), and spin indefinitely on try_lock_hpte(). Because preemption is disabled on the spinning vCPU, it will never yield CPU X. The preempted host-side lock owner can never be rescheduled on CPU X to release the lock.
This converts a potential latency/livelock issue into a permanent hard deadlock.
Proposed Solution:
==================
To make this fix completely watertight and eliminate the deadlock window, no thread must ever be preempted while holding HPTE_V_HVLOCK in virtual mode. This means we must disable preemption during the lock-hold window on both the vCPU-side and the host-side paths.
In the next version, I'll be making following changes:
Host-Side Preemption Disabling: Add preempt_disable() / preempt_enable() brackets around the HPTE_V_HVLOCK hold windows inside the four host-side virtual-mode functions in arch/powerpc/kvm/book3s_64_mmu_hv.c:
- kvm_unmap_rmapp()
- kvm_age_rmapp()
- kvm_test_clear_dirty_npages()
- resize_hpt_rehash_hpte()
For example, inside kvm_unmap_rmapp():
diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c index 2ccb3d138f46..59da958e09cb 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
@@ -823,7 +823,9 @@ static void kvm_unmap_rmapp(struct kvm *kvm, struct kvm_memory_slot *memslot,
*/
i = *rmapp & KVMPPC_RMAP_INDEX;
hptep = (__be64 *) (kvm->arch.hpt.virt + (i << 4));
+ preempt_disable();
if (!try_lock_hpte(hptep, HPTE_V_HVLOCK)) {
+ preempt_enable();
/* unlock rmap before spinning on the HPTE lock */
unlock_rmap(rmapp);
while (be64_to_cpu(hptep[0]) & HPTE_V_HVLOCK)
@@ -834,6 +836,7 @@ static void kvm_unmap_rmapp(struct kvm *kvm, struct kvm_memory_slot *memslot,
kvmppc_unmap_hpte(kvm, i, memslot, rmapp, gfn);
unlock_rmap(rmapp);
__unlock_hpte(hptep, be64_to_cpu(hptep[0]));
+ preempt_enable();
}}
To ensure complete consistency and verify there are no other potential preemption deadlock sites, I conducted a full audit of all 8 occurrences of try_lock_hpte() inside arch/powerpc/kvm/book3s_64_mmu_hv.c.
The proposed v2 fixes simply bring the host-side lock users into alignment with the established, pre-existing design pattern used by every other virtual-mode caller of try_lock_hpte() in this file:
- Established safe callers (4) — already wrapped in preempt_disable() / preempt_enable() blocks by design:
- kvmppc_book3s_hv_page_fault()
- read_hpte_for_cma()
- kvmppc_hpte_hv_page_debug()
- Symmetrical alignment in v2 (4):
- kvm_unmap_rmapp()
- kvm_age_rmapp()
- kvm_test_clear_dirty_npages()
- resize_hpt_rehash_hpte()
This ensures 100% preemption-safety symmetry across all virtual-mode lock-holding windows in KVM.
I hope this sounds good to you.
Thanks,
Amit
PATCH RESEND 3/3 KVM: PPC: Fix get_d_signext() 12-bit displacement for paired-single D-form
In reply to Amit Machhiwalget_d_signext() has two compounding bugs since its introduction in 2010:
1. The extraction mask 0x8FF silently drops bits 8-10 of the 12-bit D field (PPC ISA bits 21-23), corrupting any displacement that has any of those bits set.
2. The sign-magnitude idiom "-(d & 0x7ff)" is wrong for two's-complement:
for D=0xFFC (encoding of -4) it returns -252 instead of -4.
Together these errors produce a wrong effective address for psq_l, psq_lu, psq_st, and psq_stu whenever the displacement is negative or is a positive value >= 0x100 with bits 8-10 set — essentially any real-world paired- single stack-relative access.
The D field occupies the bottom 12 bits of the instruction word (confirmed by the adjacent W and I extractions via inst_get_field(inst,16,16) and inst_get_field(inst,17,19)). Replace the open-coded logic with the standard sign_extend32(inst & 0xfff, 11), which correctly performs two's-complement sign extension from 12 bits to 32 bits.
--- arch/powerpc/kvm/book3s_paired_singles.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-)
arch/powerpc/kvm/book3s_paired_singles.c +1 -6
@@ -479,12 +479,7 @@ static bool kvmppc_inst_is_paired_single(struct kvm_vcpu *vcpu, u32 inst)static int get_d_signext (u32 inst) { -int d = inst &0x8ff ; - -if (d &0x800 ) -return -(d &0x7ff ); - -return (d &0x7ff ); +return sign_extend32 (inst &0xfff ,11 ); } static int kvmppc_ps_three_in(struct kvm_vcpu *vcpu, bool rc,
Signature · 1 line
2.54.0 (Apple Git-157)
Quoted 8 lines · Amit Machhiwal
This series addresses a few bug fixes across KVM PPC Book3S HV locking/synchronization and paired-single instruction decoding:
Patches 1 & 2 fix synchronization and preemption issues introduced in commit
6165d5dd99db("KVM: PPC: Book3S HV: add virtual mode handlers for HPT hcalls and page faults"):[...]
Applied to powerpc/next.
[1/3] KVM: PPC: Book3S HV: Add SRCU protection for virtual-mode HPT hcalls
https://git.kernel.org/powerpc/c/61a39797bb136243c48b46e2586d8a9e2cea6a89 [2/3] KVM: PPC: Book3S HV: Add preempt_disable() around virtual-mode HPTE bit-lock users https://git.kernel.org/powerpc/c/abbbe5ff83e7c9ed3a4d61b73011f1df7bf87907 [3/3] KVM: PPC: Fix get_d_signext() 12-bit displacement for paired-single D-form https://git.kernel.org/powerpc/c/31895b0ff96778b524c74b43a108090581271603
cheers