Skip to contentSkip to conversation

KVM: x86/mmu: Use try_cmpxchg64 in tdp_mmu_set_spte_atomic

PATCH Applied Uros Bizjak 3 participants 3 messages May 18, 2022 – Jun 15, 2022

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 KVM: x86/mmu: Use try_cmpxchg64 in tdp_mmu_set_spte_atomic

Use try_cmpxchg64 instead of cmpxchg64 (*ptr, old, new) != old in tdp_mmu_set_spte_atomic. cmpxchg returns success in ZF flag, so this change saves a compare after cmpxchg (and related move instruction in front of cmpxchg). Also, remove explicit assignment to iter->old_spte when cmpxchg fails, this is what try_cmpxchg does implicitly.

Cc Paolo Bonzini <[email protected]>
Cc Sean Christopherson <[email protected]>

---
Patch requires commits 0aa7be05d83cc584da0782405e8007e351dfb6cc and c2df0a6af177b6c06a859806a876f92b072dc624 from tip.git

---
 arch/x86/kvm/mmu/tdp_mmu.c | 12 +-----------
 1 file changed, 1 insertion(+), 11 deletions(-)
arch/x86/kvm/mmu/tdp_mmu.c +1 -11
index 922b06bf4b94..1ccc1a0f8123 100644
@@ -633,7 +633,6 @@ static inline int tdp_mmu_set_spte_atomic(struct kvm *kvm,
 					  u64 new_spte)
 {
 	u64 *sptep = rcu_dereference(iter->sptep);
-	u64 old_spte;
 
 	/*
 	 * The caller is responsible for ensuring the old SPTE is not a REMOVED
@@ -649,17 +648,8 @@ static inline int tdp_mmu_set_spte_atomic(struct kvm *kvm,
 	 * Note, fast_pf_fix_direct_spte() can also modify TDP MMU SPTEs and
 	 * does not hold the mmu_lock.
 	 */
-	old_spte = cmpxchg64(sptep, iter->old_spte, new_spte);
-	if (old_spte != iter->old_spte) {
-		/*
-		 * The page table entry was modified by a different logical
-		 * CPU. Refresh iter->old_spte with the current value so the
-		 * caller operates on fresh data, e.g. if it retries
-		 * tdp_mmu_set_spte_atomic().
-		 */
-		iter->old_spte = old_spte;
+	if (!try_cmpxchg64(sptep, &iter->old_spte, new_spte))
 		return -EBUSY;
-	}
 
 	__handle_changed_spte(kvm, iter->as_id, iter->gfn, iter->old_spte,
 			      new_spte, iter->level, true);
Signature · 1 line
2.35.1
Reviewed-by on v1 1/1
#1 lore raw
Quoted 10 lines · Uros Bizjak

Use try_cmpxchg64 instead of cmpxchg64 (*ptr, old, new) != old in tdp_mmu_set_spte_atomic. cmpxchg returns success in ZF flag, so this change saves a compare after cmpxchg (and related move instruction in front of cmpxchg). Also, remove explicit assignment to iter->old_spte when cmpxchg fails, this is what try_cmpxchg does implicitly.

Cc Paolo Bonzini <[email protected]>
Cc Sean Christopherson <[email protected]>

Nice cleanup.

Quoted 41 lines

---
Patch requires commits 0aa7be05d83cc584da0782405e8007e351dfb6cc and c2df0a6af177b6c06a859806a876f92b072dc624 from tip.git

---
 arch/x86/kvm/mmu/tdp_mmu.c | 12 +-----------
 1 file changed, 1 insertion(+), 11 deletions(-)
arch/x86/kvm/mmu/tdp_mmu.c +1 -12
index 922b06bf4b94..1ccc1a0f8123 100644
@@ -633,7 +633,6 @@ static inline int tdp_mmu_set_spte_atomic(struct kvm *kvm,
                                          u64 new_spte)
 {
        u64 *sptep = rcu_dereference(iter->sptep);
-       u64 old_spte;
 
        /*
         * The caller is responsible for ensuring the old SPTE is not a REMOVED
@@ -649,17 +648,8 @@ static inline int tdp_mmu_set_spte_atomic(struct kvm *kvm,
         * Note, fast_pf_fix_direct_spte() can also modify TDP MMU SPTEs and
         * does not hold the mmu_lock.
         */
-       old_spte = cmpxchg64(sptep, iter->old_spte, new_spte);
-       if (old_spte != iter->old_spte) {
-               /*
-                * The page table entry was modified by a different logical
-                * CPU. Refresh iter->old_spte with the current value so the
-                * caller operates on fresh data, e.g. if it retries
-                * tdp_mmu_set_spte_atomic().
-                */
-               iter->old_spte = old_spte;
+       if (!try_cmpxchg64(sptep, &iter->old_spte, new_spte))
                return -EBUSY;
-       }
 
        __handle_changed_spte(kvm, iter->as_id, iter->gfn, iter->old_spte,
                              new_spte, iter->level, true);
--

2.35.1