KVM: add support for SVM Nested Paging
Hi,
here is the improved patchset which adds support for the Nested Paging feature of the AMD Barcelona and Phenom processors to KVM. The patch set was successfully install- and runtime-tested with various guest operating systems (64 bit, 32 bit legacy and 32 bit PAE Linux, Windows 64 bit and 32 bit versions too).
To the previous post of these patches they were extended with support for KVM on 32 bit PAE hosts. Live migration is also implemented and tested in various situations (migration between 32 bit and 64 bit host, Non-NPT to/from NPT migration, NPT to NPT migration, all with various kinds of guests, including Windows). After all these tests I am pretty sure that these patches don't introduce any regression to KVM and therefore the NPT feature is enabled per default.
Some new performance numbers are also available. For this test I started an Ubuntu 7.10 guest with 4 vcpus and 3G of RAM on a dual socket Opteron box and ran kernbench -M in it with a 2.6.24 kernel as the source tree. The test was done with traditional Shadow Paging and using Nested Paging. The tests ran on the text console with GDM stopped. Here are the results:
-----------------------------------------------------------------------
Average Half load -j 3 Run:
-----------------+-------------------------+---------------------------
| Nested Paging | Shadow Paging
-----------------+-------------------------+---------------------------
| Time (std deviation) | Time (std deviation)
-----------------+-------------------------+---------------------------
Elapsed Time | 147.574 (0.642168) | 211.73 (0.802963) User Time | 382.036 (1.14742) | 416.254 (1.91732) System Time | 48.29 (0.261151) | 221.608 (0.706307) Percent CPU | 291 (0.707107) | 300.8 (0.447214) Context Switches | 33605.4 (148.788) | 34901 (434.318) Sleeps | 47351 (98.2395) | 47017.8 (378.582)
-----------------+-------------------------+---------------------------
Average Half load -j 16 Run:
-----------------+-------------------------+---------------------------
| Nested Paging | Shadow Paging
-----------------+-------------------------+---------------------------
| Time (std deviation) | Time (std deviation)
-----------------+-------------------------+---------------------------
Elapsed Time | 120.156 (0.337757) | 179.162 (1.49999) User Time | 385.318 (3.57206) | 421.624 (5.82785) System Time | 48.959 (0.811411) | 230.8 (9.70806) Percent CPU | 327.6 (38.5982) | 336.2 (37.3655) Context Switches | 42435.5 (9312.09) | 46190.7 (11909.6) Sleeps | 50297.5 (3111.73) | 50688 (3882.49)
-----------------+-------------------------+---------------------------
In other words, Nested Paging gave 30% performance improvement in the 'make -j 3' run and a 33% improvement in the 'make -j 16' run for this benchmark.
I think these patches are ready for merging into the KVM tree. Please consider for inclusion.
Joerg Roedel
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
PATCH 1/8 SVM: move feature detection to hardware setup code
By moving the SVM feature detection from the each_cpu code to the hardware setup code it runs only once. As an additional advance the feature check is now available earlier in the module setup process.
--- arch/x86/kvm/svm.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-)
arch/x86/kvm/svm.c +3 -1
@@ -302,7 +302,6 @@ static void svm_hardware_enable(void *garbage) svm_data->asid_generation =1 ; svm_data->max_asid =cpuid_ebx (SVM_CPUID_FUNC ) -1 ; svm_data->next_asid = svm_data->max_asid +1 ; - svm_features =cpuid_edx (SVM_CPUID_FUNC ); asmvolatile ("sgdt %0" :"=m" (gdt_descr)); gdt = (struct desc_struct *)gdt_descr.address;
@@ -408,6 +407,9 @@ static __init int svm_hardware_setup(void)if (r) goto err_2; } + + svm_features =cpuid_edx (SVM_CPUID_FUNC ); +return 0 ; err_2:
Signature · 9 lines
1.5.3.7 ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
PATCH 2/8 SVM: add detection of Nested Paging feature
In reply to Joerg RoedelLet SVM detect if the Nested Paging feature is available on the hardware. Disable it to keep this patch series bisectable.
--- arch/x86/kvm/svm.c | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-)
arch/x86/kvm/svm.c +8 -0
@@ -47,6 +47,8 @@ MODULE_LICENSE("GPL");#define SVM_FEATURE_LBRV (1 << 1)#define SVM_DEATURE_SVML (1 << 2) +static bool npt_enabled = false; +static void kvm_reput_irq (struct vcpu_svm *svm);static inline struct vcpu_svm *to_svm (struct kvm_vcpu *vcpu)
@@ -410,6 +412,12 @@ static __init int svm_hardware_setup(void) svm_features =cpuid_edx (SVM_CPUID_FUNC ); +if (!svm_has (SVM_FEATURE_NPT )) + npt_enabled = false; + +if (npt_enabled) +printk (KERN_INFO "kvm: Nested Paging enabled\n" ); +return 0 ; err_2:
Signature · 9 lines
1.5.3.7 ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
PATCH 3/8 SVM: add module parameter to disable Nested Paging
In reply to Joerg RoedelTo disable the use of the Nested Paging feature even if it is available in hardware this patch adds a module parameter. Nested Paging can be disabled by passing npt=0 to the kvm_amd module.
--- arch/x86/kvm/svm.c | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-)
arch/x86/kvm/svm.c +8 -0
@@ -48,6 +48,9 @@ MODULE_LICENSE("GPL");#define SVM_DEATURE_SVML (1 << 2)static bool npt_enabled = false; +static int npt =1 ; + +module_param (npt, int,S_IRUGO );static void kvm_reput_irq (struct vcpu_svm *svm);
@@ -415,6 +418,11 @@ static __init int svm_hardware_setup(void)if (!svm_has (SVM_FEATURE_NPT )) npt_enabled = false; +if (npt_enabled && !npt) { +printk (KERN_INFO "kvm: Nested Paging disabled\n" ); + npt_enabled = false; + } +if (npt_enabled)printk (KERN_INFO "kvm: Nested Paging enabled\n" );
Signature · 9 lines
1.5.3.7 ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
PATCH 4/8 X86: export information about NPT to generic x86 code
In reply to Joerg RoedelThe generic x86 code has to know if the specific implementation uses Nested Paging. In the generic code Nested Paging is called Two Dimensional Paging (TDP) to avoid confusion with (future) TDP implementations of other vendors. This patch exports the availability of TDP to the generic x86 code.
--- arch/x86/kvm/mmu.c | 15 +++++++++++++++ arch/x86/kvm/svm.c | 4 +++- include/asm-x86/kvm_host.h | 2 ++ 3 files changed, 20 insertions(+), 1 deletions(-)
arch/x86/kvm/mmu.c +15 -0
@@ -32,6 +32,15 @@#include <asm/cmpxchg.h> #include <asm/io.h> +/* +* When setting this variable to true it enables Two-Dimensional-Paging +* where the hardware walks 2 page tables: +* 1. the guest-virtual to guest-physical +* 2. while doing 1. it walks guest-physical to host-physical +* If the hardware supports that we don't need to do shadow paging. +*/ +static bool tdp_enabled = false; +#undef MMU_DEBUG#undef AUDIT
@@ -1562,6 +1571,12 @@ out: }EXPORT_SYMBOL_GPL (kvm_mmu_page_fault); +void kvm_enable_tdp (void ) +{ + tdp_enabled = true; +} +EXPORT_SYMBOL_GPL (kvm_enable_tdp); +static void free_mmu_pages (struct kvm_vcpu *vcpu) {struct kvm_mmu_page *sp;
arch/x86/kvm/svm.c +3 -1
@@ -423,8 +423,10 @@ static __init int svm_hardware_setup(void) npt_enabled = false; } -if (npt_enabled) +if (npt_enabled) {printk (KERN_INFO "kvm: Nested Paging enabled\n" ); +kvm_enable_tdp (); + }return 0 ;
include/asm-x86/kvm_host.h +2 -0
@@ -491,6 +491,8 @@ int kvm_fix_hypercall(struct kvm_vcpu *vcpu);int kvm_mmu_page_fault (struct kvm_vcpu *vcpu,gva_t gva,u32 error_code); +void kvm_enable_tdp (void ); +int load_pdptrs (struct kvm_vcpu *vcpu,unsigned long cr3);int complete_pio (struct kvm_vcpu *vcpu);
Signature · 9 lines
1.5.3.7 ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
PATCH 5/8 MMU: make the __nonpaging_map function generic
In reply to Joerg RoedelThe mapping function for the nonpaging case in the softmmu does basically the same as required for Nested Paging. Make this function generic so it can be used for both.
--- arch/x86/kvm/mmu.c | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-)
arch/x86/kvm/mmu.c +3 -4
@@ -965,10 +965,9 @@ static void nonpaging_new_cr3(struct kvm_vcpu *vcpu) { } -static int __nonpaging_map (struct kvm_vcpu *vcpu,gva_t v,int write, -gfn_t gfn,struct page *page) +static int __direct_map (struct kvm_vcpu *vcpu,gpa_t v,int write, +gfn_t gfn,struct page *page,int level) { -int level =PT32E_ROOT_LEVEL ;hpa_t table_addr = vcpu->arch.mmu.root_hpa;int pt_write =0 ;
@@ -1026,7 +1025,7 @@ static int nonpaging_map(struct kvm_vcpu *vcpu, gva_t v, int write, gfn_t gfn)spin_lock (&vcpu->kvm->mmu_lock);kvm_mmu_free_some_pages (vcpu); - r =__nonpaging_map (vcpu, v, write, gfn, page); + r =__direct_map (vcpu, v, write, gfn, page,PT32E_ROOT_LEVEL );spin_unlock (&vcpu->kvm->mmu_lock);up_read (¤t->mm->mmap_sem);
Signature · 9 lines
1.5.3.7 ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
PATCH 6/8 X86: export the load_pdptrs() function to modules
In reply to Joerg RoedelThe load_pdptrs() function is required in the SVM module for NPT support.
--- arch/x86/kvm/x86.c | 1 + include/asm-x86/kvm_host.h | 2 ++ 2 files changed, 3 insertions(+), 0 deletions(-)
arch/x86/kvm/x86.c +1 -0
@@ -202,6 +202,7 @@ out:return ret; } +EXPORT_SYMBOL_GPL (load_pdptrs);static bool pdptrs_changed (struct kvm_vcpu *vcpu) {
include/asm-x86/kvm_host.h +2 -0
@@ -410,6 +410,8 @@ void kvm_mmu_zap_all(struct kvm *kvm);unsigned int kvm_mmu_calculate_mmu_pages (struct kvm *kvm);void kvm_mmu_change_mmu_pages (struct kvm *kvm,unsigned int kvm_nr_mmu_pages); +int load_pdptrs (struct kvm_vcpu *vcpu,unsigned long cr3); +enum emulation_result {EMULATE_DONE ,/* no further processing */ EMULATE_DO_MMIO ,/* kvm_run filled with mmio request */
Signature · 1 line
1.5.3.7
PATCH 7/8 MMU: add TDP support to the KVM MMU
In reply to Joerg RoedelThis patch contains the changes to the KVM MMU necessary for support of the Nested Paging feature in AMD Barcelona and Phenom Processors.
--- arch/x86/kvm/mmu.c | 79 ++++++++++++++++++++++++++++++++++++++++++++++++++-- arch/x86/kvm/mmu.h | 6 ++++ 2 files changed, 82 insertions(+), 3 deletions(-)
arch/x86/kvm/mmu.c +76 -3
@@ -1081,6 +1081,7 @@ static void mmu_alloc_roots(struct kvm_vcpu *vcpu)int i;gfn_t root_gfn;struct kvm_mmu_page *sp; +int metaphysical =0 ; root_gfn = vcpu->arch.cr3 >>PAGE_SHIFT ;
@@ -1089,14 +1090,20 @@ static void mmu_alloc_roots(struct kvm_vcpu *vcpu)hpa_t root = vcpu->arch.mmu.root_hpa;ASSERT (!VALID_PAGE (root)); +if (tdp_enabled) + metaphysical =1 ; sp =kvm_mmu_get_page (vcpu, root_gfn,0 , -PT64_ROOT_LEVEL ,0 ,ACC_ALL ,NULL ,NULL ); +PT64_ROOT_LEVEL , metaphysical, +ACC_ALL ,NULL ,NULL ); root =__pa (sp->spt); ++sp->root_count; vcpu->arch.mmu.root_hpa = root; return; }#endif + metaphysical = !is_paging (vcpu); +if (tdp_enabled) + metaphysical =1 ;for (i =0 ; i <4 ; ++i) {hpa_t root = vcpu->arch.mmu.pae_root[i];
@@ -1110,7 +1117,7 @@ static void mmu_alloc_roots(struct kvm_vcpu *vcpu) }else if (vcpu->arch.mmu.root_level ==0 ) root_gfn =0 ; sp =kvm_mmu_get_page (vcpu, root_gfn, i <<30 , -PT32_ROOT_LEVEL , !is_paging (vcpu), +PT32_ROOT_LEVEL , metaphysical,ACC_ALL ,NULL ,NULL ); root =__pa (sp->spt); ++sp->root_count;
@@ -1144,6 +1151,36 @@ static int nonpaging_page_fault(struct kvm_vcpu *vcpu, gva_t gva, error_code & PFERR_WRITE_MASK, gfn); } +static int tdp_page_fault (struct kvm_vcpu *vcpu,gva_t gpa, +u32 error_code) +{ +struct page *page; +int r; + +ASSERT (vcpu); +ASSERT (VALID_PAGE (vcpu->arch.mmu.root_hpa)); + + r =mmu_topup_memory_caches (vcpu); +if (r) +return r; + +down_read (¤t->mm->mmap_sem); + page =gfn_to_page (vcpu->kvm, gpa >>PAGE_SHIFT ); +if (is_error_page (page)) { +kvm_release_page_clean (page); +up_read (¤t->mm->mmap_sem); +return 1 ; + } +spin_lock (&vcpu->kvm->mmu_lock); +kvm_mmu_free_some_pages (vcpu); + r =__direct_map (vcpu, gpa, error_code &PFERR_WRITE_MASK , + gpa >>PAGE_SHIFT , page,TDP_ROOT_LEVEL ); +spin_unlock (&vcpu->kvm->mmu_lock); +up_read (¤t->mm->mmap_sem); + +return r; +} +static void nonpaging_free (struct kvm_vcpu *vcpu) {mmu_free_roots (vcpu);
@@ -1237,7 +1274,35 @@ static int paging32E_init_context(struct kvm_vcpu *vcpu)return paging64_init_context_common (vcpu,PT32E_ROOT_LEVEL ); } -static int init_kvm_mmu (struct kvm_vcpu *vcpu) +static int init_kvm_tdp_mmu (struct kvm_vcpu *vcpu) +{ +struct kvm_mmu *context = &vcpu->arch.mmu; + + context->new_cr3 = nonpaging_new_cr3; + context->page_fault = tdp_page_fault; + context->free = nonpaging_free; + context->prefetch_page = nonpaging_prefetch_page; + context->shadow_root_level =TDP_ROOT_LEVEL ; + context->root_hpa =INVALID_PAGE ; + +if (!is_paging (vcpu)) { + context->gva_to_gpa = nonpaging_gva_to_gpa; + context->root_level =0 ; + }else if (is_long_mode (vcpu)) { + context->gva_to_gpa = paging64_gva_to_gpa; + context->root_level =PT64_ROOT_LEVEL ; + }else if (is_pae (vcpu)) { + context->gva_to_gpa = paging64_gva_to_gpa; + context->root_level =PT32E_ROOT_LEVEL ; + }else { + context->gva_to_gpa = paging32_gva_to_gpa; + context->root_level =PT32_ROOT_LEVEL ; + } + +return 0 ; +} + +static int init_kvm_softmmu (struct kvm_vcpu *vcpu) {ASSERT (vcpu);ASSERT (!VALID_PAGE (vcpu->arch.mmu.root_hpa));
@@ -1252,6 +1317,14 @@ static int init_kvm_mmu(struct kvm_vcpu *vcpu)return paging32_init_context (vcpu); } +static int init_kvm_mmu (struct kvm_vcpu *vcpu) +{ +if (tdp_enabled) +return init_kvm_tdp_mmu (vcpu); +else +return init_kvm_softmmu (vcpu); +} +static void destroy_kvm_mmu (struct kvm_vcpu *vcpu) {ASSERT (vcpu);
arch/x86/kvm/mmu.h +6 -0
@@ -3,6 +3,12 @@#include <linux/kvm_host.h> +#ifdef CONFIG_X86_64 +#define TDP_ROOT_LEVEL PT64_ROOT_LEVEL +#else +#define TDP_ROOT_LEVEL PT32E_ROOT_LEVEL +#endif +static inline void kvm_mmu_free_some_pages (struct kvm_vcpu *vcpu) {if (unlikely (vcpu->kvm->arch.n_free_mmu_pages <KVM_MIN_FREE_MMU_PAGES ))
Signature · 9 lines
1.5.3.7 ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Re: [kvm-devel] [PATCH 7/8] MMU: add TDP support to the KVM MMU
Quoted 3 lines · Joerg Roedel
This patch contains the changes to the KVM MMU necessary for support of the Nested Paging feature in AMD Barcelona and Phenom Processors.
good patch, it look like things will be very fixable with it
Quoted 50 lines
Signed-off-by Joerg Roedel <[email protected]>--- arch/x86/kvm/mmu.c | 79 ++++++++++++++++++++++++++++++++++++++++++++++++++-- arch/x86/kvm/mmu.h | 6 ++++ 2 files changed, 82 insertions(+), 3 deletions(-)arch/x86/kvm/mmu.c +9
-2@@ -1081,6 +1081,7 @@ static void mmu_alloc_roots(struct kvm_vcpu *vcpu)int i;gfn_t root_gfn;struct kvm_mmu_page *sp; +int metaphysical =0 ; root_gfn = vcpu->arch.cr3 >>PAGE_SHIFT ;@@ -1089,14 +1090,20 @@ static void mmu_alloc_roots(struct kvm_vcpu *vcpu)hpa_t root = vcpu->arch.mmu.root_hpa;ASSERT (!VALID_PAGE (root)); +if (tdp_enabled) + metaphysical =1 ; sp =kvm_mmu_get_page (vcpu, root_gfn,0 , -PT64_ROOT_LEVEL ,0 ,ACC_ALL ,NULL ,NULL ); +PT64_ROOT_LEVEL , metaphysical, +ACC_ALL ,NULL ,NULL ); root =__pa (sp->spt); ++sp->root_count; vcpu->arch.mmu.root_hpa = root; return; }#endif + metaphysical = !is_paging (vcpu); +if (tdp_enabled) + metaphysical =1 ;for (i =0 ; i <4 ; ++i) {hpa_t root = vcpu->arch.mmu.pae_root[i];@@ -1110,7 +1117,7 @@ static void mmu_alloc_roots(struct kvm_vcpu *vcpu) }else if (vcpu->arch.mmu.root_level ==0 ) root_gfn =0 ; sp =kvm_mmu_get_page (vcpu, root_gfn, i <<30 , -PT32_ROOT_LEVEL , !is_paging (vcpu), +PT32_ROOT_LEVEL , metaphysical,ACC_ALL ,NULL ,NULL ); root =__pa (sp->spt); ++sp->root_count;
arch/x86/kvm/mmu.c static int nonpaging_page_fault(struct kvm_vcpu *vcpu, gva_t gva, · line 1156
Quoted 6 lines
arch/x86/kvm/mmu.c +2
-0@@ -1144,6 +1151,36 @@ static int nonpaging_page_fault(struct kvm_vcpu *vcpu, gva_t gva, error_code & PFERR_WRITE_MASK, gfn); } +static int tdp_page_fault (struct kvm_vcpu *vcpu,gva_t gpa, +u32 error_code)
you probably mean gpa_t ?
Quoted 9 lines
arch/x86/kvm/mmu.c +9
-0+{ +struct page *page; +int r; + +ASSERT (vcpu); +ASSERT (VALID_PAGE (vcpu->arch.mmu.root_hpa)); + + r =mmu_topup_memory_caches (vcpu); +if (r)
arch/x86/kvm/mmu.c static int nonpaging_page_fault(struct kvm_vcpu *vcpu, gva_t gva,
Quoted 10 lines
arch/x86/kvm/mmu.c +9
-0+return r; + +down_read (¤t->mm->mmap_sem); + page =gfn_to_page (vcpu->kvm, gpa >>PAGE_SHIFT ); +if (is_error_page (page)) { +kvm_release_page_clean (page); +up_read (¤t->mm->mmap_sem); +return 1 ; + }
i dont know if it worth checking it here,
in the worth case we will map the error page and the host will be safe
Quoted 84 lines
arch/x86/kvm/mmu.c +10
-0+spin_lock (&vcpu->kvm->mmu_lock); +kvm_mmu_free_some_pages (vcpu); + r =__direct_map (vcpu, gpa, error_code &PFERR_WRITE_MASK , + gpa >>PAGE_SHIFT , page,TDP_ROOT_LEVEL ); +spin_unlock (&vcpu->kvm->mmu_lock); +up_read (¤t->mm->mmap_sem); + +return r; +} +static void nonpaging_free (struct kvm_vcpu *vcpu){
arch/x86/kvm/mmu.c +0
-1mmu_free_roots (vcpu);@@ -1237,7 +1274,35 @@ static int paging32E_init_context(struct kvm_vcpu *vcpu)return paging64_init_context_common (vcpu,PT32E_ROOT_LEVEL ); } -static int init_kvm_mmu (struct kvm_vcpu *vcpu)tdp_page_fault(struct
arch/x86/kvm/mmu.c +37
-0+static int init_kvm_tdp_mmu (struct kvm_vcpu *vcpu) +{ +struct kvm_mmu *context = &vcpu->arch.mmu; + + context->new_cr3 = nonpaging_new_cr3; + context->page_fault = tdp_page_fault; + context->free = nonpaging_free; + context->prefetch_page = nonpaging_prefetch_page; + context->shadow_root_level =TDP_ROOT_LEVEL ; + context->root_hpa =INVALID_PAGE ; + +if (!is_paging (vcpu)) { + context->gva_to_gpa = nonpaging_gva_to_gpa; + context->root_level =0 ; + }else if (is_long_mode (vcpu)) { + context->gva_to_gpa = paging64_gva_to_gpa; + context->root_level =PT64_ROOT_LEVEL ; + }else if (is_pae (vcpu)) { + context->gva_to_gpa = paging64_gva_to_gpa; + context->root_level =PT32E_ROOT_LEVEL ; + }else { + context->gva_to_gpa = paging32_gva_to_gpa; + context->root_level =PT32_ROOT_LEVEL ; + } + +return 0 ; +} + +static int init_kvm_softmmu (struct kvm_vcpu *vcpu) {ASSERT (vcpu);ASSERT (!VALID_PAGE (vcpu->arch.mmu.root_hpa));@@ -1252,6 +1317,14 @@ static int init_kvm_mmu(struct kvm_vcpu *vcpu)return paging32_init_context (vcpu); } +static int init_kvm_mmu (struct kvm_vcpu *vcpu) +{ +if (tdp_enabled) +return init_kvm_tdp_mmu (vcpu); +else +return init_kvm_softmmu (vcpu); +} +static void destroy_kvm_mmu (struct kvm_vcpu *vcpu) {ASSERT (vcpu);arch/x86/kvm/mmu.h +6
-0@@ -3,6 +3,12 @@#include <linux/kvm_host.h> +#ifdef CONFIG_X86_64 +#define TDP_ROOT_LEVEL PT64_ROOT_LEVEL +#else +#define TDP_ROOT_LEVEL PT32E_ROOT_LEVEL +#endif +static inline void kvm_mmu_free_some_pages (struct kvm_vcpu *vcpu) {if (unlikely (vcpu->kvm->arch.n_free_mmu_pages <KVM_MIN_FREE_MMU_PAGES ))
Signature · 7 lines
woof. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Re: [kvm-devel] [PATCH 7/8] MMU: add TDP support to the KVM MMU
Quoted 52 lines · Izik Eidus
Quoted 3 lines · Joerg Roedel
This patch contains the changes to the KVM MMU necessary for support of the Nested Paging feature in AMD Barcelona and Phenom Processors.
good patch, it look like things will be very fixable with it
Quoted 45 lines
Signed-off-by Joerg Roedel <[email protected]>---
arch/x86/kvm/mmu.c| 79 ++++++++++++++++++++++++++++++++++++++++++++++++++--arch/x86/kvm/mmu.h| 6 ++++2 files changed, 82 insertions(+), 3 deletions(-)arch/x86/kvm/mmu.c +9
-2@@ -1081,6 +1081,7 @@ static void mmu_alloc_roots(struct kvm_vcpu *vcpu)int i;gfn_t root_gfn;struct kvm_mmu_page *sp; +int metaphysical =0 ; root_gfn = vcpu->arch.cr3 >>PAGE_SHIFT ;@@ -1089,14 +1090,20 @@ static void mmu_alloc_roots(struct kvm_vcpu *vcpu)hpa_t root = vcpu->arch.mmu.root_hpa;ASSERT (!VALID_PAGE (root)); +if (tdp_enabled) + metaphysical =1 ; sp =kvm_mmu_get_page (vcpu, root_gfn,0 , -PT64_ROOT_LEVEL ,0 ,ACC_ALL ,NULL ,NULL ); +PT64_ROOT_LEVEL , metaphysical, +ACC_ALL ,NULL ,NULL ); root =__pa (sp->spt); ++sp->root_count; vcpu->arch.mmu.root_hpa = root; return; }#endif + metaphysical = !is_paging (vcpu); +if (tdp_enabled) + metaphysical =1 ;for (i =0 ; i <4 ; ++i) {hpa_t root = vcpu->arch.mmu.pae_root[i];@@ -1110,7 +1117,7 @@ static void mmu_alloc_roots(struct kvm_vcpu *vcpu) }else if (vcpu->arch.mmu.root_level ==0 ) root_gfn =0 ; sp =kvm_mmu_get_page (vcpu, root_gfn, i <<30 , -PT32_ROOT_LEVEL , !is_paging (vcpu), +PT32_ROOT_LEVEL , metaphysical,ACC_ALL ,NULL ,NULL ); root =__pa (sp->spt); ++sp->root_count;
Quoted 8 lines
arch/x86/kvm/mmu.c static int nonpaging_page_fault(struct kvm_vcpu *vcpu, gva_t gva, · line 1155
Quoted 6 lines
arch/x86/kvm/mmu.c +2
-0@@ -1144,6 +1151,36 @@ static int nonpaging_page_fault(struct kvm_vcpu *vcpu, gva_t gva, error_code & PFERR_WRITE_MASK, gfn); } +static int tdp_page_fault (struct kvm_vcpu *vcpu,gva_t gpa, +u32 error_code)you probably mean
gpa_t?
Yes. But the function is assigned to a function pointer. And the type of that pointer expects gva_t there. So I named the parameter gpa to describe that a guest physical address is meant there.
Quoted 14 lines
Quoted 14 lines
arch/x86/kvm/mmu.c +14
-0+{ +struct page *page; +int r; + +ASSERT (vcpu); +ASSERT (VALID_PAGE (vcpu->arch.mmu.root_hpa)); + + r =mmu_topup_memory_caches (vcpu); +if (r) +return r; + +down_read (¤t->mm->mmap_sem); + page =gfn_to_page (vcpu->kvm, gpa >>PAGE_SHIFT ); +if (is_error_page (page)) {
Quoted 8 lines
arch/x86/kvm/mmu.c static int nonpaging_page_fault(struct kvm_vcpu *vcpu, gva_t gva,
Quoted 5 lines
arch/x86/kvm/mmu.c +4
-0+kvm_release_page_clean (page); +up_read (¤t->mm->mmap_sem); +return 1 ; + }i dont know if it worth checking it here,
in the worth case we will map the error page and the host will be safe
Looking at the nonpaging_map function it is the right place to check for the error page.
Joerg Roedel
Signature · 13 lines
| AMD Saxony Limited Liability Company & Co. KG
Operating | Wilschdorfer Landstr. 101, 01109 Dresden, Germany
System | Register Court Dresden: HRA 4896
Research | General Partner authorized to represent:
Center | AMD Saxony LLC (Wilmington, Delaware, US)
| General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/Re: [kvm-devel] [PATCH 7/8] MMU: add TDP support to the KVM MMU
Quoted 95 lines · Joerg Roedel
Quoted 54 lines · Izik Eidus
Quoted 4 lines · Joerg Roedel
This patch contains the changes to the KVM MMU necessary for support of the Nested Paging feature in AMD Barcelona and Phenom Processors.
good patch, it look like things will be very fixable with it
Quoted 45 lines
Signed-off-by Joerg Roedel <[email protected]>---
arch/x86/kvm/mmu.c| 79 ++++++++++++++++++++++++++++++++++++++++++++++++++--arch/x86/kvm/mmu.h| 6 ++++2 files changed, 82 insertions(+), 3 deletions(-)arch/x86/kvm/mmu.c +9
-2@@ -1081,6 +1081,7 @@ static void mmu_alloc_roots(struct kvm_vcpu *vcpu)int i;gfn_t root_gfn;struct kvm_mmu_page *sp; +int metaphysical =0 ; root_gfn = vcpu->arch.cr3 >>PAGE_SHIFT ;@@ -1089,14 +1090,20 @@ static void mmu_alloc_roots(struct kvm_vcpu *vcpu)hpa_t root = vcpu->arch.mmu.root_hpa;ASSERT (!VALID_PAGE (root)); +if (tdp_enabled) + metaphysical =1 ; sp =kvm_mmu_get_page (vcpu, root_gfn,0 , -PT64_ROOT_LEVEL ,0 ,ACC_ALL ,NULL ,NULL ); +PT64_ROOT_LEVEL , metaphysical, +ACC_ALL ,NULL ,NULL ); root =__pa (sp->spt); ++sp->root_count; vcpu->arch.mmu.root_hpa = root; return; }#endif + metaphysical = !is_paging (vcpu); +if (tdp_enabled) + metaphysical =1 ;for (i =0 ; i <4 ; ++i) {hpa_t root = vcpu->arch.mmu.pae_root[i];@@ -1110,7 +1117,7 @@ static void mmu_alloc_roots(struct kvm_vcpu *vcpu) }else if (vcpu->arch.mmu.root_level ==0 ) root_gfn =0 ; sp =kvm_mmu_get_page (vcpu, root_gfn, i <<30 , -PT32_ROOT_LEVEL , !is_paging (vcpu), +PT32_ROOT_LEVEL , metaphysical,ACC_ALL ,NULL ,NULL ); root =__pa (sp->spt); ++sp->root_count;Quoted 9 lines
arch/x86/kvm/mmu.c static int nonpaging_page_fault(struct kvm_vcpu *vcpu, gva_t gva, · line 1156
Quoted 7 lines
arch/x86/kvm/mmu.c +2
-0@@ -1144,6 +1151,36 @@ static int nonpaging_page_fault(struct kvm_vcpu *vcpu, gva_t gva, error_code & PFERR_WRITE_MASK, gfn); } +static int tdp_page_fault (struct kvm_vcpu *vcpu,gva_t gpa, +u32 error_code)you probably mean
gpa_t?Yes. But the function is assigned to a function pointer. And the type of that pointer expects
gva_tthere. So I named the parameter gpa to describe that a guest physical address is meant there.Quoted 14 lines
Quoted 14 lines
arch/x86/kvm/mmu.c +14
-0+{ +struct page *page; +int r; + +ASSERT (vcpu); +ASSERT (VALID_PAGE (vcpu->arch.mmu.root_hpa)); + + r =mmu_topup_memory_caches (vcpu); +if (r) +return r; + +down_read (¤t->mm->mmap_sem); + page =gfn_to_page (vcpu->kvm, gpa >>PAGE_SHIFT ); +if (is_error_page (page)) {Quoted 9 lines
arch/x86/kvm/mmu.c static int nonpaging_page_fault(struct kvm_vcpu *vcpu, gva_t gva,
Quoted 6 lines
arch/x86/kvm/mmu.c +4
-0+kvm_release_page_clean (page); +up_read (¤t->mm->mmap_sem); +return 1 ; + }i dont know if it worth checking it here,
in the worth case we will map the error page and the host will be safe
Quoted 3 lines
Looking at the
nonpaging_mapfunction it is the right place to check for the error page.
thinking about it again you are right, (for some reason i was thinking about old kvm code that was replace already)
the is_error_page should be here.
Quoted 3 lines
Joerg Roedel
Signature · 7 lines
woof. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
PATCH 8/8 SVM: add support for Nested Paging
In reply to Joerg RoedelThis patch contains the SVM architecture dependent changes for KVM to enable support for the Nested Paging feature of AMD Barcelona and Phenom processors.
--- arch/x86/kvm/svm.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++--- 1 files changed, 67 insertions(+), 5 deletions(-)
arch/x86/kvm/svm.c +67 -5
@@ -47,7 +47,12 @@ MODULE_LICENSE("GPL");#define SVM_FEATURE_LBRV (1 << 1)#define SVM_DEATURE_SVML (1 << 2) +/* enable NPT for AMD64 and X86 with PAE */ +#if defined(CONFIG_X86_64 ) || defined(CONFIG_X86_PAE ) +static bool npt_enabled = true; +#else static bool npt_enabled = false; +#endif static int npt =1 ;module_param (npt, int,S_IRUGO );
@@ -187,7 +192,7 @@ static inline void flush_guest_tlb(struct kvm_vcpu *vcpu)static void svm_set_efer (struct kvm_vcpu *vcpu,u64 efer) { -if (!(efer &EFER_LMA )) +if (!npt_enabled && !(efer &EFER_LMA )) efer &= ~EFER_LME ;to_svm (vcpu)->vmcb->save.efer = efer |MSR_EFER_SVME_MASK ;
@@ -570,6 +575,22 @@ static void init_vmcb(struct vmcb *vmcb) save->cr0 =0x00000010 |X86_CR0_PG |X86_CR0_WP ; save->cr4 =X86_CR4_PAE ;/* rdx = ?? */ + +if (npt_enabled) { +/* Setup VMCB for Nested Paging */ + control->nested_ctl =1 ; + control->intercept_exceptions &= ~(1 <<PF_VECTOR ); + control->intercept_cr_read &= ~(INTERCEPT_CR0_MASK | +INTERCEPT_CR3_MASK ); + control->intercept_cr_write &= ~(INTERCEPT_CR0_MASK | +INTERCEPT_CR3_MASK ); + save->g_pat =0x0007040600070406ULL ; +/* enable caching because the QEMU Bios doesn't enable it */ + save->cr0 =X86_CR0_ET ; + save->cr3 =0 ; + save->cr4 =0 ; + } + }static int svm_vcpu_reset (struct kvm_vcpu *vcpu)
@@ -804,6 +825,9 @@ static void svm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) } }#endif +if (npt_enabled) + goto set; +if ((vcpu->arch.cr0 &X86_CR0_TS ) && !(cr0 &X86_CR0_TS )) { svm->vmcb->control.intercept_exceptions &= ~(1 <<NM_VECTOR ); vcpu->fpu_active =1 ;
@@ -811,18 +835,26 @@ static void svm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) vcpu->arch.cr0 = cr0; cr0 |=X86_CR0_PG |X86_CR0_WP ; - cr0 &= ~(X86_CR0_CD |X86_CR0_NW );if (!vcpu->fpu_active) { svm->vmcb->control.intercept_exceptions |= (1 <<NM_VECTOR ); cr0 |=X86_CR0_TS ; } +set: +/* +* re-enable caching here because the QEMU bios +* does not do it - this results in some delay at +* reboot +*/ + cr0 &= ~(X86_CR0_CD |X86_CR0_NW ); svm->vmcb->save.cr0 = cr0; }static void svm_set_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4) { vcpu->arch.cr4 = cr4; -to_svm (vcpu)->vmcb->save.cr4 = cr4 |X86_CR4_PAE ; +if (!npt_enabled) + cr4 |=X86_CR4_PAE ; +to_svm (vcpu)->vmcb->save.cr4 = cr4; } static void svm_set_segment(struct kvm_vcpu *vcpu,
@@ -1288,14 +1320,34 @@ static int (*svm_exit_handlers[])(struct vcpu_svm *svm, [SVM_EXIT_WBINVD] = emulate_on_interception, [SVM_EXIT_MONITOR] = invalid_op_interception, [SVM_EXIT_MWAIT] = invalid_op_interception, + [SVM_EXIT_NPF] = pf_interception, }; -static int handle_exit (struct kvm_run *kvm_run,struct kvm_vcpu *vcpu) {struct vcpu_svm *svm =to_svm (vcpu);u32 exit_code = svm->vmcb->control.exit_code; +if (npt_enabled) { +int mmu_reload =0 ; +if ((vcpu->arch.cr0 ^ svm->vmcb->save.cr0) &X86_CR0_PG ) { +svm_set_cr0 (vcpu, svm->vmcb->save.cr0); + mmu_reload =1 ; + } + vcpu->arch.cr0 = svm->vmcb->save.cr0; + vcpu->arch.cr3 = svm->vmcb->save.cr3; +if (is_paging (vcpu) &&is_pae (vcpu) && !is_long_mode (vcpu)) { +if (!load_pdptrs (vcpu, vcpu->arch.cr3)) { +kvm_inject_gp (vcpu,0 ); +return 1 ; + } + } +if (mmu_reload) { +kvm_mmu_reset_context (vcpu); +kvm_mmu_load (vcpu); + } + } +kvm_reput_irq (svm);if (svm->vmcb->control.exit_code ==SVM_EXIT_ERR ) {
@@ -1306,7 +1358,8 @@ static int handle_exit(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu) }if (is_external_interrupt (svm->vmcb->control.exit_int_info) && - exit_code !=SVM_EXIT_EXCP_BASE +PF_VECTOR ) + exit_code !=SVM_EXIT_EXCP_BASE +PF_VECTOR && + exit_code !=SVM_EXIT_NPF ) printk(KERN_ERR "%s: unexpected exit_ini_info 0x%x " "exit_code 0x%x\n", __FUNCTION__, svm->vmcb->control.exit_int_info,
@@ -1497,6 +1550,9 @@ static void svm_vcpu_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) svm->host_dr6 =read_dr6 (); svm->host_dr7 =read_dr7 (); svm->vmcb->save.cr2 = vcpu->arch.cr2; +/* required for live migration with NPT */ +if (npt_enabled) + svm->vmcb->save.cr3 = vcpu->arch.cr3;if (svm->vmcb->save.dr7 &0xff ) {write_dr7 (0 );
@@ -1640,6 +1696,12 @@ static void svm_set_cr3(struct kvm_vcpu *vcpu, unsigned long root) {struct vcpu_svm *svm =to_svm (vcpu); +if (npt_enabled) { + svm->vmcb->control.nested_cr3 = root; +force_new_asid (vcpu); +return ; + } + svm->vmcb->save.cr3 = root;force_new_asid (vcpu);
Signature · 9 lines
1.5.3.7 ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Avi Kivity
AppliedQuoted 9 lines · Joerg Roedel
Hi,
here is the improved patchset which adds support for the Nested Paging feature of the AMD Barcelona and Phenom processors to KVM. The patch set was successfully install- and runtime-tested with various guest operating systems (64 bit, 32 bit legacy and 32 bit PAE Linux, Windows 64 bit and 32 bit versions too).
Applied all, thanks.
Signature · 1 line
error compiling committee.c: too many arguments to function