KVM: MMU: improve n_max_mmu_pages calculation with TDP
kvm_mmu_calculate_mmu_pages numbers,
maximum number of shadow pages = 2% of mapped guest pages
Does not make sense for TDP guests where mapping all of guest memory with 4k pages cannot exceed "mapped guest pages / 512" (not counting root pages).
Allow that maximum for TDP, forcing the guest to recycle otherwise.
arch/x86/kvm/mmu.c +14 -2
@@ -4293,7 +4293,7 @@ nomem:unsigned int kvm_mmu_calculate_mmu_pages (struct kvm *kvm) {unsigned int nr_mmu_pages; -unsigned int nr_pages =0 ; +unsigned int i, nr_pages =0 ;struct kvm_memslots *slots;struct kvm_memory_slot *memslot;
@@ -4302,7 +4302,19 @@ unsigned int kvm_mmu_calculate_mmu_pages(struct kvm *kvm)kvm_for_each_memslot (memslot, slots) nr_pages += memslot->npages; - nr_mmu_pages = nr_pages *KVM_PERMILLE_MMU_PAGES /1000 ; +if (tdp_enabled) { +/* one root page */ + nr_mmu_pages =1 ; +/* nr_pages / (512^i) per level, due to +* guest RAM map being linear */ +for (i =1 ; i <4 ; i++) { +int nr_pages_round = nr_pages + (1 << (9 *i)); + nr_mmu_pages += nr_pages_round >> (9 *i); + } + }else { + nr_mmu_pages = nr_pages *KVM_PERMILLE_MMU_PAGES /1000 ; + } + nr_mmu_pages =max (nr_mmu_pages, (unsigned int )KVM_MIN_ALLOC_MMU_PAGES );
Quoted 30 lines · Marcelo Tosatti
kvm_mmu_calculate_mmu_pagesnumbers,maximum number of shadow pages = 2% of mapped guest pages
Does not make sense for TDP guests where mapping all of guest memory with 4k pages cannot exceed "mapped guest pages / 512" (not counting root pages).
Allow that maximum for TDP, forcing the guest to recycle otherwise.
Signed-off-by Marcelo Tosatti <[email protected]>arch/x86/kvm/mmu.c +1
-1@@ -4293,7 +4293,7 @@ nomem:unsigned int kvm_mmu_calculate_mmu_pages (struct kvm *kvm) {unsigned int nr_mmu_pages; -unsigned int nr_pages =0 ; +unsigned int i, nr_pages =0 ;struct kvm_memslots *slots;struct kvm_memory_slot *memslot;@@ -4302,7 +4302,19 @@ unsigned int kvm_mmu_calculate_mmu_pages(struct kvm *kvm)kvm_for_each_memslot (memslot, slots) nr_pages += memslot->npages;
arch/x86/kvm/mmu.c unsigned int kvm_mmu_calculate_mmu_pages(struct kvm *kvm) · line 4313
Quoted 10 lines
arch/x86/kvm/mmu.c +9
-1- nr_mmu_pages = nr_pages *KVM_PERMILLE_MMU_PAGES /1000 ; +if (tdp_enabled) { +/* one root page */ + nr_mmu_pages =1 ; +/* nr_pages / (512^i) per level, due to +* guest RAM map being linear */ +for (i =1 ; i <4 ; i++) { +int nr_pages_round = nr_pages + (1 << (9 *i)); + nr_mmu_pages += nr_pages_round >> (9 *i); + }
Marcelo,
Can it work if nested guest is used? Did you see any problem in practice (direct guest uses more memory than your calculation)?
And mmio also can build some page table that looks like not considered in this patch.
Quoted 44 lines · Xiao Guangrong
Quoted 30 lines · Marcelo Tosatti
kvm_mmu_calculate_mmu_pagesnumbers,maximum number of shadow pages = 2% of mapped guest pages
Does not make sense for TDP guests where mapping all of guest memory with 4k pages cannot exceed "mapped guest pages / 512" (not counting root pages).
Allow that maximum for TDP, forcing the guest to recycle otherwise.
Signed-off-by Marcelo Tosatti <[email protected]>arch/x86/kvm/mmu.c +1
-1@@ -4293,7 +4293,7 @@ nomem:unsigned int kvm_mmu_calculate_mmu_pages (struct kvm *kvm) {unsigned int nr_mmu_pages; -unsigned int nr_pages =0 ; +unsigned int i, nr_pages =0 ;struct kvm_memslots *slots;struct kvm_memory_slot *memslot;@@ -4302,7 +4302,19 @@ unsigned int kvm_mmu_calculate_mmu_pages(struct kvm *kvm)kvm_for_each_memslot (memslot, slots) nr_pages += memslot->npages;arch/x86/kvm/mmu.c unsigned int kvm_mmu_calculate_mmu_pages(struct kvm *kvm) · line 4313
Quoted 10 lines
arch/x86/kvm/mmu.c +9
-1- nr_mmu_pages = nr_pages *KVM_PERMILLE_MMU_PAGES /1000 ; +if (tdp_enabled) { +/* one root page */ + nr_mmu_pages =1 ; +/* nr_pages / (512^i) per level, due to +* guest RAM map being linear */ +for (i =1 ; i <4 ; i++) { +int nr_pages_round = nr_pages + (1 << (9 *i)); + nr_mmu_pages += nr_pages_round >> (9 *i); + }Marcelo,
Quoted 2 lines
Can it work if nested guest is used? Did you see any problem in practice (direct guest uses more memory than your calculation)?
Direct guest can use more than the calculation by switching between different paging modes.
About nested guest: at one point in time the working set cannot exceed the number of physical pages visible by the guest.
Allowing an excessively high number of shadow pages is a security concern, also, as unpreemptable long operations are necessary to tear down the pages.
Quoted 2 lines
And mmio also can build some page table that looks like not considered in this patch.
Right, but its only a few pages. Same argument as above: working set at one given time is smaller than total RAM. Do you see any potential problem?
Quoted 45 lines · Marcelo Tosatti
Quoted 44 lines · Xiao Guangrong
Quoted 30 lines · Marcelo Tosatti
kvm_mmu_calculate_mmu_pagesnumbers,maximum number of shadow pages = 2% of mapped guest pages
Does not make sense for TDP guests where mapping all of guest memory with 4k pages cannot exceed "mapped guest pages / 512" (not counting root pages).
Allow that maximum for TDP, forcing the guest to recycle otherwise.
Signed-off-by Marcelo Tosatti <[email protected]>arch/x86/kvm/mmu.c +1
-1@@ -4293,7 +4293,7 @@ nomem:unsigned int kvm_mmu_calculate_mmu_pages (struct kvm *kvm) {unsigned int nr_mmu_pages; -unsigned int nr_pages =0 ; +unsigned int i, nr_pages =0 ;struct kvm_memslots *slots;struct kvm_memory_slot *memslot;@@ -4302,7 +4302,19 @@ unsigned int kvm_mmu_calculate_mmu_pages(struct kvm *kvm)kvm_for_each_memslot (memslot, slots) nr_pages += memslot->npages;arch/x86/kvm/mmu.c unsigned int kvm_mmu_calculate_mmu_pages(struct kvm *kvm) · line 4313
Quoted 10 lines
arch/x86/kvm/mmu.c +9
-1- nr_mmu_pages = nr_pages *KVM_PERMILLE_MMU_PAGES /1000 ; +if (tdp_enabled) { +/* one root page */ + nr_mmu_pages =1 ; +/* nr_pages / (512^i) per level, due to +* guest RAM map being linear */ +for (i =1 ; i <4 ; i++) { +int nr_pages_round = nr_pages + (1 << (9 *i)); + nr_mmu_pages += nr_pages_round >> (9 *i); + }Marcelo,
Quoted 5 lines
Quoted 2 lines
Can it work if nested guest is used? Did you see any problem in practice (direct guest uses more memory than your calculation)?
Direct guest can use more than the calculation by switching between different paging modes.
I mean guest runs on hardmmu (tdp is used but no nested guest). Its only use one page table and seems can not use more memory than your calculation (except some mmio page tables).
So, you calculation is only used to limit memory used if tdp + nested guest?
Quoted 3 lines
About nested guest: at one point in time the working set cannot exceed the number of physical pages visible by the guest.
But it can cause lots of #PF, it is the nightmare for performance, no?
Quoted 2 lines
Allowing an excessively high number of shadow pages is a security
The security concern means "optimization memory usage"? Or something else?
Quoted 2 lines
concern, also, as unpreemptable long operations are necessary to tear down the pages.
You mean limiting the shadow pages to let some patch run faster like remove-write-access and zap-all-sp etc.? If yes, we can directly optimize for these paths, this is more effective i think.
Quoted 7 lines
Quoted 3 lines
And mmio also can build some page table that looks like not considered in this patch.
Right, but its only a few pages. Same argument as above: working set at one given time is smaller than total RAM. Do you see any potential problem?
Marcelo, I just confused whether the limitation is reasonable, as i said, the limitation is not effective enough on hardmmu-only guest (no nested). and it seems too low for nested guests.
Quoted 49 lines · Xiao Guangrong
Quoted 48 lines · Marcelo Tosatti
Quoted 46 lines · Xiao Guangrong
Quoted 30 lines · Marcelo Tosatti
kvm_mmu_calculate_mmu_pagesnumbers,maximum number of shadow pages = 2% of mapped guest pages
Does not make sense for TDP guests where mapping all of guest memory with 4k pages cannot exceed "mapped guest pages / 512" (not counting root pages).
Allow that maximum for TDP, forcing the guest to recycle otherwise.
Signed-off-by Marcelo Tosatti <[email protected]>arch/x86/kvm/mmu.c +1
-1@@ -4293,7 +4293,7 @@ nomem:unsigned int kvm_mmu_calculate_mmu_pages (struct kvm *kvm) {unsigned int nr_mmu_pages; -unsigned int nr_pages =0 ; +unsigned int i, nr_pages =0 ;struct kvm_memslots *slots;struct kvm_memory_slot *memslot;@@ -4302,7 +4302,19 @@ unsigned int kvm_mmu_calculate_mmu_pages(struct kvm *kvm)kvm_for_each_memslot (memslot, slots) nr_pages += memslot->npages;arch/x86/kvm/mmu.c unsigned int kvm_mmu_calculate_mmu_pages(struct kvm *kvm) · line 4313
Quoted 10 lines
arch/x86/kvm/mmu.c +9
-1- nr_mmu_pages = nr_pages *KVM_PERMILLE_MMU_PAGES /1000 ; +if (tdp_enabled) { +/* one root page */ + nr_mmu_pages =1 ; +/* nr_pages / (512^i) per level, due to +* guest RAM map being linear */ +for (i =1 ; i <4 ; i++) { +int nr_pages_round = nr_pages + (1 << (9 *i)); + nr_mmu_pages += nr_pages_round >> (9 *i); + }Marcelo,
Can it work if nested guest is used? Did you see any problem in practice (direct guest uses more memory than your calculation)?
Quoted 8 lines
Quoted 2 lines
Direct guest can use more than the calculation by switching between different paging modes.
I mean guest runs on hardmmu (tdp is used but no nested guest). Its only use one page table and seems can not use more memory than your calculation (except some mmio page tables).
So, you calculation is only used to limit memory used if tdp + nested guest?
Yes, you're right, there is no duplication of shadow pages even with mode switches so the patch is not needed.
Quoted 29 lines
Quoted 2 lines
About nested guest: at one point in time the working set cannot exceed the number of physical pages visible by the guest.
But it can cause lots of #PF, it is the nightmare for performance, no?
Quoted 2 lines
Allowing an excessively high number of shadow pages is a security
The security concern means "optimization memory usage"? Or something else?
Quoted 2 lines
concern, also, as unpreemptable long operations are necessary to tear down the pages.
You mean limiting the shadow pages to let some patch run faster like remove-write-access and zap-all-sp etc.? If yes, we can directly optimize for these paths, this is more effective i think.
Quoted 7 lines
Quoted 3 lines
And mmio also can build some page table that looks like not considered in this patch.
Right, but its only a few pages. Same argument as above: working set at one given time is smaller than total RAM. Do you see any potential problem?
Marcelo, I just confused whether the limitation is reasonable, as i said, the limitation is not effective enough on hardmmu-only guest (no nested). and it seems too low for nested guests.