KVM: x86: move the can_unsync and prefetch checks outside of the loop
PATCH 1/1 KVM: x86: move the can_unsync and prefetch checks outside of the loop
mmu_try_to_unsync_pages() performs !can_unsync check before attempting to unsync any shadow pages.
This check is peformed inside the loop right now.
It's redundant to perform it every iteration if can_unsync is true, as can_unsync parameter isn't getting updated inside the loop. Move the check outside of the loop.
Same is the case with prefetch.
--- arch/x86/kvm/mmu/mmu.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-)
arch/x86/kvm/mmu/mmu.c +5 -6
@@ -2586,6 +2586,11 @@ int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot,if (kvm_slot_page_track_is_active (kvm, slot, gfn,KVM_PAGE_TRACK_WRITE ))return -EPERM ; +if (!can_unsync) +return -EPERM ; + +if (prefetch) +return -EEXIST ;/* * The page is not write-tracked, mark existing shadow pages unsync * unless KVM is synchronizing an unsync SP (can_unsync = false). In
@@ -2593,15 +2598,9 @@ int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot,* allowing shadow pages to become unsync (writable by the guest). */ for_each_gfn_indirect_valid_sp (kvm, sp, gfn) { -if (!can_unsync) -return -EPERM ; -if (sp->unsync)continue ; -if (prefetch) -return -EEXIST ; -/* * TDP MMU page faults require an additional spinlock as they * run with mmu_lock held for read, not write, and the unsync
Signature · 1 line
2.30.2
Paolo Bonzini
maintainerQuoted 8 lines · Vihas Mak
mmu_try_to_unsync_pages()performs !can_unsync check before attempting to unsync any shadow pages.
This check is peformed inside the loop right now.
It's redundant to perform it every iteration ifcan_unsyncis true, ascan_unsyncparameter isn't getting updated inside the loop. Move the check outside of the loop.Same is the case with prefetch.
The meaning changes if the loop does not execute at all. Is this safe?
Paolo
Quoted 39 lines
Signed-off-by Vihas Mak<[email protected]>Cc Sean Christopherson<[email protected]>Cc Vitaly Kuznetsov<[email protected]>Cc Wanpeng Li<[email protected]>Cc Jim Mattson<[email protected]>Cc Joerg Roedel<[email protected]>--- arch/x86/kvm/mmu/mmu.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-)arch/x86/kvm/mmu/mmu.c +5
-6@@ -2586,6 +2586,11 @@ int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot,if (kvm_slot_page_track_is_active (kvm, slot, gfn,KVM_PAGE_TRACK_WRITE ))return -EPERM ; +if (!can_unsync) +return -EPERM ; + +if (prefetch) +return -EEXIST ;/* * The page is not write-tracked, mark existing shadow pages unsync * unless KVM is synchronizing an unsync SP (can_unsync = false). In @@ -2593,15 +2598,9 @@ int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot,* allowing shadow pages to become unsync (writable by the guest). */ for_each_gfn_indirect_valid_sp (kvm, sp, gfn) { -if (!can_unsync) -return -EPERM ; -if (sp->unsync)continue ; -if (prefetch) -return -EEXIST ; -
Quoted 6 lines · Sean Christopherson
Quoted 6 lines
NAK, this change is functionally wrong. The checks are inside the loop because the flow fails if and only if there is at least one indirect, valid shadow pages at the target gfn. The @prefetch check is even more restrictive as it bails if there is at least one indirect, valid, synchronized shadow page. The
can_unsynccheck could be "optimized" to
Quoted 7 lines
Quoted 7 lines
if (!can_unsync && kvm_gfn_has_indirect_valid_sp())but identifying whether or not there's a valid SP requires walking the list of shadow pages for the gfn, so it's simpler to just handle the check in the loop. And "optimized" in quotes because both checks will be well-predicted single-uop macrofused TEST+Jcc on modern CPUs, whereas walking the list twice would be relatively expensive if there are shadow pages for the gfn.
So this change isn't safe. I will look into the optimization suggested by Sean. Sorry for this patch.
Quoted 51 lines · Paolo Bonzini
Quoted 8 lines · Vihas Mak
mmu_try_to_unsync_pages()performs !can_unsync check before attempting to unsync any shadow pages.
This check is peformed inside the loop right now.
It's redundant to perform it every iteration ifcan_unsyncis true, ascan_unsyncparameter isn't getting updated inside the loop. Move the check outside of the loop.Same is the case with prefetch.
The meaning changes if the loop does not execute at all. Is this safe?
Paolo
Quoted 36 lines
Signed-off-by Vihas Mak<[email protected]>Cc Sean Christopherson<[email protected]>Cc Vitaly Kuznetsov<[email protected]>Cc Wanpeng Li<[email protected]>Cc Jim Mattson<[email protected]>Cc Joerg Roedel<[email protected]>--- arch/x86/kvm/mmu/mmu.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-)arch/x86/kvm/mmu/mmu.c +5
-3@@ -2586,6 +2586,11 @@ int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot,if (kvm_slot_page_track_is_active (kvm, slot, gfn,KVM_PAGE_TRACK_WRITE ))return -EPERM ; +if (!can_unsync) +return -EPERM ; + +if (prefetch) +return -EEXIST ;/* * The page is not write-tracked, mark existing shadow pages unsync * unless KVM is synchronizing an unsync SP (can_unsync = false). In @@ -2593,15 +2598,9 @@ int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot,* allowing shadow pages to become unsync (writable by the guest). */ for_each_gfn_indirect_valid_sp (kvm, sp, gfn) { -if (!can_unsync) -return -EPERM ; -if (sp->unsync)continue ;
Quoted 4 lines
arch/x86/kvm/mmu/mmu.c int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot, · line 2604
Quoted 3 lines
arch/x86/kvm/mmu/mmu.c +0
-3-if (prefetch) -return -EEXIST ; -
--
Thanks,
Vihas
Sean Christopherson
maintainerQuoted 16 lines · Vihas Mak
Quoted 6 lines · Sean Christopherson
Quoted 6 lines
NAK, this change is functionally wrong. The checks are inside the loop because the flow fails if and only if there is at least one indirect, valid shadow pages at the target gfn. The @prefetch check is even more restrictive as it bails if there is at least one indirect, valid, synchronized shadow page. The
can_unsynccheck could be "optimized" toQuoted 7 lines
Quoted 7 lines
if (!can_unsync && kvm_gfn_has_indirect_valid_sp())but identifying whether or not there's a valid SP requires walking the list of shadow pages for the gfn, so it's simpler to just handle the check in the loop. And "optimized" in quotes because both checks will be well-predicted single-uop macrofused TEST+Jcc on modern CPUs, whereas walking the list twice would be relatively expensive if there are shadow pages for the gfn.
Quoted 2 lines
So this change isn't safe. I will look into the optimization suggested by Sean. Sorry for this patch.
Heh, I wasn't actually suggesting we do the "optimization". I was pointing out what the code would look like _if_ we wanted to move the checks out of the loop, but I do not actually think we should make any changes, quite the opposite. The theoretical worst case if there no indirect valid SPs, but lots of direct and/or invalid SPs is far worse than burning a few uops per loop.
The compiler is smart enough to handle the checks out of line, and I'm sure there are other optimizations being made as well. In other words, odds are very good that trying to optimize the code will do more harm than good.