vmx/hap: optimize CR4 trapping
v2 cover vmx/hap: optimize CR4 trapping
Hello,
Following two-patch series optimize CR4 access for vmx/hap.
I couldn't figure out a similar approach for AMD, since according to section 15.9 (Instruction Intercepts), you either intercept all access to CR4 or none. In any case a similar optimization for AMD can always be added later.
Thanks, Roger.
Roger Pau Monne (2):
x86/hvm: introduce cr_mask to store trapped bits of CR accesses vmx/hap: optimize CR4 trapping
xen/arch/x86/hvm/svm/vmcb.c | 1 + xen/arch/x86/hvm/vmx/vmcs.c | 1 + xen/arch/x86/hvm/vmx/vmx.c | 39 +++++++++++++++++++++++++++++++++++++++ xen/arch/x86/hvm/vmx/vvmx.c | 2 ++ xen/arch/x86/monitor.c | 5 +++-- xen/include/asm-x86/hvm/vcpu.h | 3 +++ 6 files changed, 49 insertions(+), 2 deletions(-)
Signature · 7 lines
2.16.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 1/2 x86/hvm: introduce cr_mask to store trapped bits of CR accesses
At the moment this is currently set at VMCS creation and not changed, but further patches are going to change the CR4 mask at runtime.
---
Cc: Boris Ostrovsky <[email protected]>
Cc: Suravee Suthikulpanit <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: Jun Nakajima <[email protected]>
Cc: Kevin Tian <[email protected]>
---
Changes since v1:
- New in this version.
--- xen/arch/x86/hvm/svm/vmcb.c | 1 + xen/arch/x86/hvm/vmx/vmcs.c | 1 + xen/include/asm-x86/hvm/vcpu.h | 3 +++ 3 files changed, 5 insertions(+)
xen/arch/x86/hvm/svm/vmcb.c +1 -0
@@ -169,6 +169,7 @@ static int construct_vmcb(struct vcpu *v) vmcb->tr.base =0 ; vmcb->tr.limit =0xff ; + v->arch.hvm_vcpu.mask_cr[0 ] = v->arch.hvm_vcpu.mask_cr[4 ] = ~0UL ; v->arch.hvm_vcpu.guest_cr[0 ] =X86_CR0_PE |X86_CR0_ET ;hvm_update_guest_cr (v,0 );
xen/arch/x86/hvm/vmx/vmcs.c +1 -0
@@ -1165,6 +1165,7 @@ static int construct_vmcs(struct vcpu *v)__vmwrite (CR0_GUEST_HOST_MASK , ~0UL );__vmwrite (CR4_GUEST_HOST_MASK , ~0UL ); + v->arch.hvm_vcpu.mask_cr[0 ] = v->arch.hvm_vcpu.mask_cr[4 ] = ~0UL ;__vmwrite (PAGE_FAULT_ERROR_CODE_MASK ,0 );__vmwrite (PAGE_FAULT_ERROR_CODE_MATCH ,0 );
xen/include/asm-x86/hvm/vcpu.h +3 -0
@@ -156,6 +156,9 @@ struct hvm_vcpu {*/ unsigned long hw_cr[5 ]; +/* Cached copy of the trapped bits of CRs. Used for CR0 and CR4. */ +unsigned long mask_cr[5 ]; +struct vlapic vlapic;s64 cache_tsc_offset;u64 guest_time;
Signature · 7 lines
2.16.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v2 1/2] x86/hvm: introduce cr_mask to store trapped bits of CR accesses
xen/include/asm-x86/hvm/vcpu.h struct hvm_vcpu { · line 160
Quoted 10 lines · Roger Pau Monne
xen/include/asm-x86/hvm/vcpu.h +2
-0@@ -156,6 +156,9 @@ struct hvm_vcpu {*/ unsigned long hw_cr[5 ]; +/* Cached copy of the trapped bits of CRs. Used for CR0 and CR4. */ +unsigned long mask_cr[5 ];
We only need masks for cr0 and cr4, and I don't expect this to change for the foreseeable future (CR4 still has plenty of available bits in it). I'd recommend:
unsigned long cr0_host_mask, cr4_host_mask;
~Andrew
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v2 1/2] x86/hvm: introduce cr_mask to store trapped bits of CR accesses
Quoted 9 lines · Andrew Cooper
Quoted 8 lines · Roger Pau Monne
xen/include/asm-x86/hvm/vcpu.h +0
-0@@ -156,6 +156,9 @@ struct hvm_vcpu {*/ unsigned long hw_cr[5 ];
Quoted 8 lines
xen/include/asm-x86/hvm/vcpu.h struct hvm_vcpu { · line 160
Quoted 2 lines
xen/include/asm-x86/hvm/vcpu.h +2
-0+/* Cached copy of the trapped bits of CRs. Used for CR0 and CR4. */ +unsigned long mask_cr[5 ];We only need masks for cr0 and cr4, and I don't expect this to change for the foreseeable future (CR4 still has plenty of available bits in it). I'd recommend:
unsigned long
cr0_host_mask,cr4_host_mask;
OK, I've used an array to keep consistency with the other CR array fields, but I guess here it makes less sense because only two of them are going to be used.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v2 1/2] x86/hvm: introduce cr_mask to store trapped bits of CR accesses
Quoted 16 lines · Roger Pau Monné
Quoted 15 lines · Andrew Cooper
xen/include/asm-x86/hvm/vcpu.h struct hvm_vcpu { · line 160
Quoted 10 lines · Roger Pau Monne
xen/include/asm-x86/hvm/vcpu.h +2
-0@@ -156,6 +156,9 @@ struct hvm_vcpu {*/ unsigned long hw_cr[5 ]; +/* Cached copy of the trapped bits of CRs. Used for CR0 and CR4. */ +unsigned long mask_cr[5 ];We only need masks for cr0 and cr4, and I don't expect this to change for the foreseeable future (CR4 still has plenty of available bits in it). I'd recommend:
Quoted 4 lines
Quoted 1 line
unsigned long
cr0_host_mask,cr4_host_mask;OK, I've used an array to keep consistency with the other CR array fields, but I guess here it makes less sense because only two of them are going to be used.
I'll be dropping the arrays in due course anyway. Some of the value aren't used at all, and it will simplify code which is common between PV and HVM.
~Andrew
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 2/2 vmx/hap: optimize CR4 trapping
In reply to Roger Pau MonneThere a bunch of bits in CR4 that should be allowed to be set directly by the guest without requiring Xen intervention, currently this is already done by passing through guest writes into the CR4 used when running in non-root mode, but taking an expensive vmexit in order to do so.
xenalyze reports the following when running a PV guest in shim mode:
CR_ACCESS 3885950 6.41s 17.04% 3957 cyc { 2361| 3378| 7920}
cr4 3885940 6.41s 17.04% 3957 cyc { 2361| 3378| 7920}
cr3 1 0.00s 0.00% 3480 cyc { 3480| 3480| 3480}
*[ 0] 1 0.00s 0.00% 3480 cyc { 3480| 3480| 3480} cr0 7 0.00s 0.00% 7112 cyc { 3248| 5960|17480}
clts 2 0.00s 0.00% 4588 cyc { 3456| 5720| 5720}
After this change this turns into:
CR_ACCESS 12 0.00s 0.00% 9972 cyc { 3680|11024|24032}
cr4 2 0.00s 0.00% 17528 cyc {11024|24032|24032}
cr3 1 0.00s 0.00% 3680 cyc { 3680| 3680| 3680}
*[ 0] 1 0.00s 0.00% 3680 cyc { 3680| 3680| 3680} cr0 7 0.00s 0.00% 9209 cyc { 4184| 7848|17488}
clts 2 0.00s 0.00% 8232 cyc { 5352|11112|11112}
Note that this optimized trapping is currently only applied to guests running with HAP on Intel hardware. If using shadow paging more CR4 bits need to be unconditionally trapped, which makes this approach unlikely to yield any important performance improvements.
---
Cc: Jun Nakajima <[email protected]>
Cc: Kevin Tian <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: Razvan Cojocaru <[email protected]>
Cc: Tamas K Lengyel <[email protected]>
---
Changes since v1:
- Use the mask_cr variable in order to cache the cr4 mask.
- Take into account write_ctrlreg_mask when introspection is enabled.
--- xen/arch/x86/hvm/vmx/vmx.c | 39 +++++++++++++++++++++++++++++++++++++++ xen/arch/x86/hvm/vmx/vvmx.c | 2 ++ xen/arch/x86/monitor.c | 5 +++-- 3 files changed, 44 insertions(+), 2 deletions(-)
xen/arch/x86/hvm/vmx/vmx.c +39 -0
@@ -1684,6 +1684,36 @@ static void vmx_update_guest_cr(struct vcpu *v, unsigned int cr) }__vmwrite (GUEST_CR4 , v->arch.hvm_vcpu.hw_cr[4 ]); + +if ( !paging_mode_hap (v->domain) ) +/* +* Shadow path has not been optimized because it requires +* unconditionally trapping more CR4 bits, at which point the +* performance benefit of doing this is quite dubious. +*/ + v->arch.hvm_vcpu.mask_cr[4 ] = ~0UL ; +else + { +/* +* Update CR4 host mask to only trap when the guest tries to set +* bits that are controlled by the hypervisor. +*/ + v->arch.hvm_vcpu.mask_cr[4 ] =HVM_CR4_HOST_MASK |X86_CR4_PKE | + ~hvm_cr4_guest_valid_bits (v,0 ); + v->arch.hvm_vcpu.mask_cr[4 ] |= v->arch.hvm_vmx.vmx_realmode ? +X86_CR4_VME :0 ; + v->arch.hvm_vcpu.mask_cr[4 ] |= !hvm_paging_enabled (v) ? + (X86_CR4_PSE |X86_CR4_SMEP | +X86_CR4_SMAP ) + :0 ; +if ( v->domain->arch.monitor.write_ctrlreg_enabled & +monitor_ctrlreg_bitmask (VM_EVENT_X86_CR4 ) ) + v->arch.hvm_vcpu.mask_cr[4 ] |= + ~v->domain->arch.monitor.write_ctrlreg_mask[4 ]; + + } +__vmwrite (CR4_GUEST_HOST_MASK , v->arch.hvm_vcpu.mask_cr[4 ]); +break ;case 2 :
@@ -3512,6 +3542,15 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)if (paging_mode_hap (v->domain) ) { +/* +* Xen allows the guest to modify some CR4 bits directly, update cached +* values to match. +*/ +__vmread (GUEST_CR4 , &v->arch.hvm_vcpu.hw_cr[4 ]); + v->arch.hvm_vcpu.guest_cr[4 ] &= v->arch.hvm_vcpu.mask_cr[4 ]; + v->arch.hvm_vcpu.guest_cr[4 ] |= v->arch.hvm_vcpu.hw_cr[4 ] & + ~v->arch.hvm_vcpu.mask_cr[4 ]; +__vmread (GUEST_CR3 , &v->arch.hvm_vcpu.hw_cr[3 ]);if (vmx_unrestricted_guest (v) ||hvm_paging_enabled (v) ) v->arch.hvm_vcpu.guest_cr[3 ] = v->arch.hvm_vcpu.hw_cr[3 ];
xen/arch/x86/hvm/vmx/vvmx.c +2 -0
@@ -1100,6 +1100,8 @@ static void load_shadow_guest_state(struct vcpu *v) cr_read_shadow = (get_vvmcs (v,GUEST_CR4 ) & ~cr_gh_mask) | (get_vvmcs (v,CR4_READ_SHADOW ) & cr_gh_mask);__vmwrite (CR4_READ_SHADOW , cr_read_shadow); +/* Add the nested host mask to the one set by vmx_update_guest_cr. */ +__vmwrite (CR4_GUEST_HOST_MASK , cr_gh_mask | v->arch.hvm_vcpu.mask_cr[4 ]);/* TODO: CR3 target control */ }
xen/arch/x86/monitor.c +3 -2
@@ -189,10 +189,11 @@ int arch_monitor_domctl_event(struct domain *d, ad->monitor.write_ctrlreg_enabled &= ~ctrlreg_bitmask; } -if (VM_EVENT_X86_CR3 == mop->u.mov_to_cr.index ) +if (VM_EVENT_X86_CR3 == mop->u.mov_to_cr.index || +VM_EVENT_X86_CR4 == mop->u.mov_to_cr.index ) {struct vcpu *v; -/* Latches new CR3 mask through CR0 code. */ +/* Latches new CR3 or CR4 mask through CR0 code. */ for_each_vcpu ( d, v )hvm_update_guest_cr (v,0 ); }
Signature · 7 lines
2.16.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 8 lines · Roger Pau Monne
xen/arch/x86/monitor.c +0
-0@@ -189,10 +189,11 @@ int arch_monitor_domctl_event(struct domain *d, ad->monitor.write_ctrlreg_enabled &= ~ctrlreg_bitmask; }
xen/arch/x86/monitor.c int arch_monitor_domctl_event(struct domain *d, · line 199
Quoted 10 lines
xen/arch/x86/monitor.c +3
-2-if (VM_EVENT_X86_CR3 == mop->u.mov_to_cr.index ) +if (VM_EVENT_X86_CR3 == mop->u.mov_to_cr.index || +VM_EVENT_X86_CR4 == mop->u.mov_to_cr.index ) {struct vcpu *v; -/* Latches new CR3 mask through CR0 code. */ +/* Latches new CR3 or CR4 mask through CR0 code. */ for_each_vcpu ( d, v )hvm_update_guest_cr (v,0 ); }
Did you, by any chance, test this code with xen-access.c (it already has a test for CR4 for the PGE stuff)? I'm not convinced the hvm_update_guest_cr(v, 0); call suffices to enable CR4 load exits.
Thanks,
Razvan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 9 lines · Razvan Cojocaru
Quoted 8 lines · Roger Pau Monne
xen/arch/x86/monitor.c +0
-0@@ -189,10 +189,11 @@ int arch_monitor_domctl_event(struct domain *d, ad->monitor.write_ctrlreg_enabled &= ~ctrlreg_bitmask; }
Quoted 14 lines
xen/arch/x86/monitor.c int arch_monitor_domctl_event(struct domain *d, · line 199
Quoted 10 lines
xen/arch/x86/monitor.c +3
-2-if (VM_EVENT_X86_CR3 == mop->u.mov_to_cr.index ) +if (VM_EVENT_X86_CR3 == mop->u.mov_to_cr.index || +VM_EVENT_X86_CR4 == mop->u.mov_to_cr.index ) {struct vcpu *v; -/* Latches new CR3 mask through CR0 code. */ +/* Latches new CR3 or CR4 mask through CR0 code. */ for_each_vcpu ( d, v )hvm_update_guest_cr (v,0 ); }Did you, by any chance, test this code with
xen-access.c(it already has a test for CR4 for the PGE stuff)? I'm not convinced thehvm_update_guest_cr(v, 0); call suffices to enable CR4 load exits.
hvm_update_guest_cr is just a wrapper to vmx_update_guest_cr when using vmx, which will unconditionally re-calculate the CR4 mask when called with cr == 0 or cr == 4.
I have not tested it with xen-access, but it seems quite straightforward to me. Are you seeing any other path that could enable CR4 load accesses without calling hvm_update_guest_cr?
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 25 lines · Roger Pau Monné
Quoted 9 lines · Razvan Cojocaru
Quoted 8 lines · Roger Pau Monne
xen/arch/x86/monitor.c +0
-0@@ -189,10 +189,11 @@ int arch_monitor_domctl_event(struct domain *d, ad->monitor.write_ctrlreg_enabled &= ~ctrlreg_bitmask; }Quoted 14 lines
xen/arch/x86/monitor.c int arch_monitor_domctl_event(struct domain *d, · line 199
Quoted 10 lines
xen/arch/x86/monitor.c +3
-2-if (VM_EVENT_X86_CR3 == mop->u.mov_to_cr.index ) +if (VM_EVENT_X86_CR3 == mop->u.mov_to_cr.index || +VM_EVENT_X86_CR4 == mop->u.mov_to_cr.index ) {struct vcpu *v; -/* Latches new CR3 mask through CR0 code. */ +/* Latches new CR3 or CR4 mask through CR0 code. */ for_each_vcpu ( d, v )hvm_update_guest_cr (v,0 ); }Did you, by any chance, test this code with
xen-access.c(it already has a test for CR4 for the PGE stuff)? I'm not convinced thehvm_update_guest_cr(v, 0); call suffices to enable CR4 load exits.
Quoted 7 lines
hvm_update_guest_cris just a wrapper tovmx_update_guest_crwhen using vmx, which will unconditionally re-calculate the CR4 mask when called with cr == 0 or cr == 4.I have not tested it with xen-access, but it seems quite straightforward to me. Are you seeing any other path that could enable CR4 load accesses without calling
hvm_update_guest_cr?
No, I thought I did but as it turns out it wasn't. I'll run a quick test on the patches just to make sure though. They should be alright.
Thanks,
Razvan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Razvan Cojocaru
Acked-byQuoted 30 lines · Razvan Cojocaru
Quoted 29 lines · Roger Pau Monné
Quoted 9 lines · Razvan Cojocaru
Quoted 8 lines · Roger Pau Monne
xen/arch/x86/monitor.c +0
-0@@ -189,10 +189,11 @@ int arch_monitor_domctl_event(struct domain *d, ad->monitor.write_ctrlreg_enabled &= ~ctrlreg_bitmask; }Quoted 14 lines
xen/arch/x86/monitor.c int arch_monitor_domctl_event(struct domain *d, · line 199
Quoted 10 lines
xen/arch/x86/monitor.c +3
-2-if (VM_EVENT_X86_CR3 == mop->u.mov_to_cr.index ) +if (VM_EVENT_X86_CR3 == mop->u.mov_to_cr.index || +VM_EVENT_X86_CR4 == mop->u.mov_to_cr.index ) {struct vcpu *v; -/* Latches new CR3 mask through CR0 code. */ +/* Latches new CR3 or CR4 mask through CR0 code. */ for_each_vcpu ( d, v )hvm_update_guest_cr (v,0 ); }Did you, by any chance, test this code with
xen-access.c(it already has a test for CR4 for the PGE stuff)? I'm not convinced thehvm_update_guest_cr(v, 0); call suffices to enable CR4 load exits.
hvm_update_guest_cris just a wrapper tovmx_update_guest_crwhen using vmx, which will unconditionally re-calculate the CR4 mask when called with cr == 0 or cr == 4.
Quoted 6 lines
Quoted 3 lines
I have not tested it with xen-access, but it seems quite straightforward to me. Are you seeing any other path that could enable CR4 load accesses without calling
hvm_update_guest_cr?No, I thought I did but as it turns out it wasn't. I'll run a quick test on the patches just to make sure though. They should be alright.
Ran a few tests, and everything appears to be in order. For the monitor / vm_event parts:
Thanks,
Razvan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel