KVM: arm64: Optimize protected mode and FWB
PATCH v2 1/1 KVM: arm64: Optimize protected mode and FWB
KVM opportunistically enables FWB if supported by the system for guest VMs, which allows it to elude cache maintenance for data as they are forced to be cacheable from stage-2.
In that case, __clean_dcache_guest_page() will immediately return. However in protected mode, before calling __clean_dcache_guest_page() it loops over the range and fix_map/unmap it, issuing TLB invalidations, dsb() and isb() unnecessarily.
This can be optimized by returning early if FWB is supported, kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation functions issue dsb() for the unmap path.
--- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/arm64/kvm/hyp/nvhe/mem_protect.c +7 -0
@@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,static void clean_dcache_guest_page (void *va,size_t size) { +/* +* Guest stage-2 uses FWB if available, making it safe to elide CMOs. +* In contrast, the host stage-2 never has FWB enabled. +*/ +if (cpus_have_final_cap (ARM64_HAS_STAGE2_FWB )) +return ; +__apply_guest_page (va, size, __clean_dcache_guest_page); }
Signature · 1 line
2.55.0.229.g6434b31f56-goog
Marc Zyngier
maintainerQuoted 24 lines · Mostafa Saleh
KVM opportunistically enables FWB if supported by the system for guest VMs, which allows it to elude cache maintenance for data as they are forced to be cacheable from stage-2.
In that case,__clean_dcache_guest_page()will immediately return. However in protected mode, before calling__clean_dcache_guest_page()it loops over the range and fix_map/unmap it, issuing TLB invalidations,dsb()andisb()unnecessarily.This can be optimized by returning early if FWB is supported,
kvm_pgtable_stage2_map()already issuesdsb()and tlb invalidation functions issuedsb()for the unmap path.Signed-off-by Mostafa Saleh <[email protected]>Reviewed-by Fuad Tabba <[email protected]>--- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++ 1 file changed, 7 insertions(+)arch/arm64/kvm/hyp/nvhe/mem_protect.c +0
-0
arch/arm64/kvm/hyp/nvhe/mem_protect.c static void __apply_guest_page(void *va, size_t size, · line 266
Quoted 6 lines
arch/arm64/kvm/hyp/nvhe/mem_protect.c +3
-0@@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,static void clean_dcache_guest_page (void *va,size_t size) { +/* +* Guest stage-2 uses FWB if available, making it safe to elide CMOs. +* In contrast, the host stage-2 never has FWB enabled.
Beeeeeeep!!! Since a373930ec9406 ("KVM: arm64: Switch pKVM host S2 over to KVM_PGTABLE_S2_AS_S1"), FWB is enabled everywhere when available.
Not sure that changes anything, but the comment is factually wrong.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Quoted 37 lines · Marc Zyngier
Quoted 24 lines · Mostafa Saleh
KVM opportunistically enables FWB if supported by the system for guest VMs, which allows it to elude cache maintenance for data as they are forced to be cacheable from stage-2.
In that case,__clean_dcache_guest_page()will immediately return. However in protected mode, before calling__clean_dcache_guest_page()it loops over the range and fix_map/unmap it, issuing TLB invalidations,dsb()andisb()unnecessarily.This can be optimized by returning early if FWB is supported,
kvm_pgtable_stage2_map()already issuesdsb()and tlb invalidation functions issuedsb()for the unmap path.Signed-off-by Mostafa Saleh <[email protected]>Reviewed-by Fuad Tabba <[email protected]>--- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++ 1 file changed, 7 insertions(+)arch/arm64/kvm/hyp/nvhe/mem_protect.c +0
-0arch/arm64/kvm/hyp/nvhe/mem_protect.c static void __apply_guest_page(void *va, size_t size, · line 266
Quoted 6 lines
arch/arm64/kvm/hyp/nvhe/mem_protect.c +3
-0@@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,static void clean_dcache_guest_page (void *va,size_t size) { +/* +* Guest stage-2 uses FWB if available, making it safe to elide CMOs. +* In contrast, the host stage-2 never has FWB enabled. Beeeeeeep!!! Since
a373930ec9406("KVM: arm64: Switch pKVM host S2 over toKVM_PGTABLE_S2_AS_S1"), FWB is enabled everywhere when available.
Quoted 1 line
Not sure that changes anything, but the comment is factually wrong.
Ah, I see it uses KVM_PGTABLE_S2_AS_S1 in that case, that should not change the patch, I will need to update the comment.
Thanks,
Mostafa
Quoted 7 lines
Thanks,
M.Signature · 1 line
Without deviation from the norm, progress is not possible.
Fuad Tabba
reviewerQuoted 39 lines · Mostafa Saleh
Quoted 37 lines · Marc Zyngier
Quoted 24 lines · Mostafa Saleh
KVM opportunistically enables FWB if supported by the system for guest VMs, which allows it to elude cache maintenance for data as they are forced to be cacheable from stage-2.
In that case,__clean_dcache_guest_page()will immediately return. However in protected mode, before calling__clean_dcache_guest_page()it loops over the range and fix_map/unmap it, issuing TLB invalidations,dsb()andisb()unnecessarily.This can be optimized by returning early if FWB is supported,
kvm_pgtable_stage2_map()already issuesdsb()and tlb invalidation functions issuedsb()for the unmap path.Signed-off-by Mostafa Saleh <[email protected]>Reviewed-by Fuad Tabba <[email protected]>--- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++ 1 file changed, 7 insertions(+)arch/arm64/kvm/hyp/nvhe/mem_protect.c +0
-0arch/arm64/kvm/hyp/nvhe/mem_protect.c static void __apply_guest_page(void *va, size_t size, · line 266
Quoted 6 lines
arch/arm64/kvm/hyp/nvhe/mem_protect.c +3
-0@@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,static void clean_dcache_guest_page (void *va,size_t size) { +/* +* Guest stage-2 uses FWB if available, making it safe to elide CMOs. +* In contrast, the host stage-2 never has FWB enabled. Beeeeeeep!!! Since
a373930ec9406("KVM: arm64: Switch pKVM host S2 over toKVM_PGTABLE_S2_AS_S1"), FWB is enabled everywhere when available.
Quoted 4 lines
Quoted 1 line
Not sure that changes anything, but the comment is factually wrong.
Ah, I see it uses
KVM_PGTABLE_S2_AS_S1in that case, that should not change the patch, I will need to update the comment.
Sorry, this confusing is partially my fault. The original comment in v1
Quoted 1 line
/* See
__clean_dcache_guest_page()*/
wasn't very clear to me, since I wasn't sure what I was looking for in __clean_dcache_guest_page(). Talking to Mostafa in person, he's referring to the comment _inside_ __clean_dcache_guest_page. I think the best fix is to rephase as
+ /* See comment in __clean_dcache_guest_page() */
This would save duplication and having to repeat the comment.
Cheers,
/fuad
Quoted 10 lines
Thanks,
MostafaQuoted 7 lines
Thanks,
M.--
Without deviation from the norm, progress is not possible.
Quoted 49 lines · Fuad Tabba
Quoted 39 lines · Mostafa Saleh
Quoted 37 lines · Marc Zyngier
Quoted 24 lines · Mostafa Saleh
KVM opportunistically enables FWB if supported by the system for guest VMs, which allows it to elude cache maintenance for data as they are forced to be cacheable from stage-2.
In that case,__clean_dcache_guest_page()will immediately return. However in protected mode, before calling__clean_dcache_guest_page()it loops over the range and fix_map/unmap it, issuing TLB invalidations,dsb()andisb()unnecessarily.This can be optimized by returning early if FWB is supported,
kvm_pgtable_stage2_map()already issuesdsb()and tlb invalidation functions issuedsb()for the unmap path.Signed-off-by Mostafa Saleh <[email protected]>Reviewed-by Fuad Tabba <[email protected]>--- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++ 1 file changed, 7 insertions(+)arch/arm64/kvm/hyp/nvhe/mem_protect.c +0
-0arch/arm64/kvm/hyp/nvhe/mem_protect.c static void __apply_guest_page(void *va, size_t size, · line 266
Quoted 6 lines
arch/arm64/kvm/hyp/nvhe/mem_protect.c +3
-0@@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,static void clean_dcache_guest_page (void *va,size_t size) { +/* +* Guest stage-2 uses FWB if available, making it safe to elide CMOs. +* In contrast, the host stage-2 never has FWB enabled. Beeeeeeep!!! Since
a373930ec9406("KVM: arm64: Switch pKVM host S2 over toKVM_PGTABLE_S2_AS_S1"), FWB is enabled everywhere when available.Quoted 4 lines
Quoted 1 line
Not sure that changes anything, but the comment is factually wrong.
Ah, I see it uses
KVM_PGTABLE_S2_AS_S1in that case, that should not change the patch, I will need to update the comment.Sorry, this confusing is partially my fault. The original comment in v1
Quoted 1 line
/* See
__clean_dcache_guest_page()*/
Quoted 9 lines
wasn't very clear to me, since I wasn't sure what I was looking for in
__clean_dcache_guest_page(). Talking to Mostafa in person, he's referring to the comment _inside___clean_dcache_guest_page. I think the best fix is to rephase as+ /* See comment in
__clean_dcache_guest_page()*/This would save duplication and having to repeat the comment.
Makes sense, also thinking about this, I think we should do the same for invalidate_icache_guest_page() and ARM64_HAS_CACHE_DIC.
I can include that in the same patch.
Thanks,
Mostafa