KVM: x86/mmu: Use try_cmpxchg64 in tdp_mmu_set_spte_atomic
PATCH 1/1 KVM: x86/mmu: Use try_cmpxchg64 in tdp_mmu_set_spte_atomic
Use try_cmpxchg64 instead of cmpxchg64 (*ptr, old, new) != old in tdp_mmu_set_spte_atomic. cmpxchg returns success in ZF flag, so this change saves a compare after cmpxchg (and related move instruction in front of cmpxchg). Also, remove explicit assignment to iter->old_spte when cmpxchg fails, this is what try_cmpxchg does implicitly.
---
Patch requires commits 0aa7be05d83cc584da0782405e8007e351dfb6cc and c2df0a6af177b6c06a859806a876f92b072dc624 from tip.git
--- arch/x86/kvm/mmu/tdp_mmu.c | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-)
arch/x86/kvm/mmu/tdp_mmu.c +1 -11
@@ -633,7 +633,6 @@ static inline int tdp_mmu_set_spte_atomic(struct kvm *kvm, u64 new_spte) {u64 *sptep =rcu_dereference (iter->sptep); -u64 old_spte;/* * The caller is responsible for ensuring the old SPTE is not a REMOVED
@@ -649,17 +648,8 @@ static inline int tdp_mmu_set_spte_atomic(struct kvm *kvm,* Note, fast_pf_fix_direct_spte() can also modify TDP MMU SPTEs and * does not hold the mmu_lock. */ - old_spte =cmpxchg64 (sptep, iter->old_spte, new_spte); -if (old_spte != iter->old_spte) { -/* -* The page table entry was modified by a different logical -* CPU. Refresh iter->old_spte with the current value so the -* caller operates on fresh data, e.g. if it retries -* tdp_mmu_set_spte_atomic(). -*/ - iter->old_spte = old_spte; +if (!try_cmpxchg64 (sptep, &iter->old_spte, new_spte))return -EBUSY ; - }__handle_changed_spte (kvm, iter->as_id, iter->gfn, iter->old_spte, new_spte, iter->level, true);
Signature · 1 line
2.35.1
Quoted 10 lines · Uros Bizjak
Use
try_cmpxchg64instead of cmpxchg64 (*ptr, old, new) != old intdp_mmu_set_spte_atomic. cmpxchg returns success in ZF flag, so this change saves a compare after cmpxchg (and related move instruction in front of cmpxchg). Also, remove explicit assignment toiter->old_sptewhen cmpxchg fails, this is whattry_cmpxchgdoes implicitly.Cc Paolo Bonzini <[email protected]>Cc Sean Christopherson <[email protected]>Signed-off-by Uros Bizjak <[email protected]>
Nice cleanup.
Quoted 41 lines
---
Patch requires commits0aa7be05d83cc584da0782405e8007e351dfb6ccandc2df0a6af177b6c06a859806a876f92b072dc624from tip.git--- arch/x86/kvm/mmu/tdp_mmu.c | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-)arch/x86/kvm/mmu/tdp_mmu.c +1
-12@@ -633,7 +633,6 @@ static inline int tdp_mmu_set_spte_atomic(struct kvm *kvm, u64 new_spte) {u64 *sptep =rcu_dereference (iter->sptep); -u64 old_spte;/* * The caller is responsible for ensuring the old SPTE is not a REMOVED @@ -649,17 +648,8 @@ static inline int tdp_mmu_set_spte_atomic(struct kvm *kvm,* Note, fast_pf_fix_direct_spte() can also modify TDP MMU SPTEs and * does not hold the mmu_lock. */ - old_spte =cmpxchg64 (sptep, iter->old_spte, new_spte); -if (old_spte != iter->old_spte) { -/* -* The page table entry was modified by a different logical -* CPU. Refresh iter->old_spte with the current value so the -* caller operates on fresh data, e.g. if it retries -* tdp_mmu_set_spte_atomic(). -*/ - iter->old_spte = old_spte; +if (!try_cmpxchg64 (sptep, &iter->old_spte, new_spte))return -EBUSY ; - }__handle_changed_spte (kvm, iter->as_id, iter->gfn, iter->old_spte, new_spte, iter->level, true); --2.35.1
Queued, thanks.
Paolo