KVM: arm64: Optimize protected mode with FWB and DIC
PATCH v3 1/1 KVM: arm64: Optimize protected mode with FWB and DIC
KVM opportunistically enables FWB if supported by the system for guest VMs, which allows it to elude cache maintenance for data as they are forced to be cacheable from stage-2.
In that case, __clean_dcache_guest_page() will immediately return. However in protected mode, before calling __clean_dcache_guest_page() it loops over the range and fix_map/unmap it, issuing TLB invalidations, dsb() and isb() unnecessarily.
This can be optimized by returning early if FWB is supported, kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation functions issue dsb() for the unmap path.
Similarly for DIC, we can omit the call ininvalidate_icache_guest_page()
No isb() needed as ERET is context synchronization event.
--- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/arm64/kvm/hyp/nvhe/mem_protect.c +7 -0
@@ -261,11 +261,18 @@ static void __apply_guest_page(void *va, size_t size,static void clean_dcache_guest_page (void *va,size_t size) { +/* See comment in __clean_dcache_guest_page() */ +if (cpus_have_final_cap (ARM64_HAS_STAGE2_FWB )) +return ; +__apply_guest_page (va, size, __clean_dcache_guest_page); }static void invalidate_icache_guest_page (void *va,size_t size) { +if (alternative_has_cap_unlikely (ARM64_HAS_CACHE_DIC )) +return ; +__apply_guest_page (va, size, __invalidate_icache_guest_page); }
Signature · 1 line
2.55.0.229.g6434b31f56-goog
Quoted 13 lines · Mostafa Saleh
KVM opportunistically enables FWB if supported by the system for guest VMs, which allows it to elude cache maintenance for data as they are forced to be cacheable from stage-2.
In that case,__clean_dcache_guest_page()will immediately return. However in protected mode, before calling__clean_dcache_guest_page()it loops over the range and fix_map/unmap it, issuing TLB invalidations,dsb()andisb()unnecessarily.This can be optimized by returning early if FWB is supported,
kvm_pgtable_stage2_map()already issuesdsb()and tlb invalidation functions issuedsb()for the unmap path.
Quoted 6 lines
Similarly for DIC, we can omit the call in
invalidate_icache_guest_page()No
isb()needed as ERET is context synchronization event.Signed-off-by Mostafa Saleh <[email protected]>
Cheers,
/fuad
Quoted 30 lines
--- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++ 1 file changed, 7 insertions(+)arch/arm64/kvm/hyp/nvhe/mem_protect.c +7
-1@@ -261,11 +261,18 @@ static void __apply_guest_page(void *va, size_t size,static void clean_dcache_guest_page (void *va,size_t size) { +/* See comment in __clean_dcache_guest_page() */ +if (cpus_have_final_cap (ARM64_HAS_STAGE2_FWB )) +return ; +__apply_guest_page (va, size, __clean_dcache_guest_page); }static void invalidate_icache_guest_page (void *va,size_t size) { +if (alternative_has_cap_unlikely (ARM64_HAS_CACHE_DIC )) +return ; +__apply_guest_page (va, size, __invalidate_icache_guest_page); } --2.55.0.229.g6434b31f56-goog
Quoted 13 lines · Mostafa Saleh
KVM opportunistically enables FWB if supported by the system for guest VMs, which allows it to elude cache maintenance for data as they are forced to be cacheable from stage-2.
In that case,__clean_dcache_guest_page()will immediately return. However in protected mode, before calling__clean_dcache_guest_page()it loops over the range and fix_map/unmap it, issuing TLB invalidations,dsb()andisb()unnecessarily.This can be optimized by returning early if FWB is supported,
kvm_pgtable_stage2_map()already issuesdsb()and tlb invalidation functions issuedsb()for the unmap path.
Quoted 6 lines
Similarly for DIC, we can omit the call in
invalidate_icache_guest_page()No
isb()needed as ERET is context synchronization event.Signed-off-by Mostafa Saleh <[email protected]>
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Quoted 9 lines · Mostafa Saleh
KVM opportunistically enables FWB if supported by the system for guest VMs, which allows it to elude cache maintenance for data as they are forced to be cacheable from stage-2.
In that case,__clean_dcache_guest_page()will immediately return. However in protected mode, before calling__clean_dcache_guest_page()it loops over the range and fix_map/unmap it, issuing TLB invalidations,dsb()andisb()unnecessarily.[...]
Applied to next, thanks!
[1/1] KVM: arm64: Optimize protected mode with FWB and DIC
https://git.kernel.org/kvmarm/kvmarm/c/81b2d03f8c86
--
Best,
Oliver