kvm, mm: account kvm related kmem slabs to kmemcg

PATCH Applied Shakeel Butt 4 participants 6 messages Oct 6, 2017 – Oct 10, 2017

Subsystems (4)

Memory management x86 KVM KVM x86

lore

Share & tools

lore

Shakeel Butt

author

2017-10-06 01:07 UTC #0 lore raw

PATCH 1/1 kvm, mm: account kvm related kmem slabs to kmemcg

The kvm slabs can consume a significant amount of system memory and indeed in our production environment we have observed that a lot of machines are spending significant amount of memory that can not be left as system memory overhead. Also the allocations from these slabs can be triggered directly by user space applications which has access to kvm and thus a buggy application can leak such memory. So, these caches should be accounted to kmemcg.

Signed-off-by Shakeel Butt <[email protected]>

---
 arch/x86/kvm/mmu.c  | 4 ++--
 virt/kvm/kvm_main.c | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

arch/x86/kvm/mmu.c +2 -2

index eca30c1eb1d9..87c5db9e644d 100644
@@ -5475,13 +5475,13 @@ int kvm_mmu_module_init(void)
 
 	pte_list_desc_cache = kmem_cache_create("pte_list_desc",
 					    sizeof(struct pte_list_desc),
-					    0, 0, NULL);
+					    0, SLAB_ACCOUNT, NULL);
 	if (!pte_list_desc_cache)
 		goto nomem;
 
 	mmu_page_header_cache = kmem_cache_create("kvm_mmu_page_header",
 						  sizeof(struct kvm_mmu_page),
-						  0, 0, NULL);
+						  0, SLAB_ACCOUNT, NULL);
 	if (!mmu_page_header_cache)
 		goto nomem;

virt/kvm/kvm_main.c +1 -1

index 9deb5a245b83..3d73299e05f2 100644
@@ -4010,7 +4010,7 @@ int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align,
 	if (!vcpu_align)
 		vcpu_align = __alignof__(struct kvm_vcpu);
 	kvm_vcpu_cache = kmem_cache_create("kvm_vcpu", vcpu_size, vcpu_align,
-					   0, NULL);
+					   SLAB_ACCOUNT, NULL);
 	if (!kvm_vcpu_cache) {
 		r = -ENOMEM;
 		goto out_free_3;

Signature · 7 lines

2.14.2.920.gcf0c67979c-goog

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Anshuman Khandual

2017-10-06 04:28 UTC #1 lore raw

Quoted 7 lines · Shakeel Butt

The kvm slabs can consume a significant amount of system memory and indeed in our production environment we have observed that a lot of machines are spending significant amount of memory that can not be left as system memory overhead. Also the allocations from these slabs can be triggered directly by user space applications which has access to kvm and thus a buggy application can leak such memory. So, these caches should be accounted to kmemcg.

But there may be other situations like this where user space can trigger allocation from various SLAB objects inside the kernel which are accounted as system memory. So how we draw the line which ones should be accounted for memcg. Just being curious.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Shakeel Butt

author

2017-10-06 06:40 UTC #2 lore raw

Quoted 9 lines · Anshuman Khandual

Quoted 7 lines · Shakeel Butt
The kvm slabs can consume a significant amount of system memory and indeed in our production environment we have observed that a lot of machines are spending significant amount of memory that can not be left as system memory overhead. Also the allocations from these slabs can be triggered directly by user space applications which has access to kvm and thus a buggy application can leak such memory. So, these caches should be accounted to kmemcg.

Quoted 5 lines

But there may be other situations like this where user space can trigger allocation from various SLAB objects inside the kernel which are accounted as system memory. So how we draw the line which ones should be accounted for memcg. Just being curious.

Yes, there are indeed other slabs where user space can trigger allocations. IMO selecting which kmem caches to account is kind of workload and user specific decision. The ones I am converting are selected based on the data gathered from our production environment. However I think it would be useful in general.

Michal Hocko

2017-10-06 07:52 UTC #3 lore raw

In reply to Anshuman Khandual

Quoted 9 lines · Anshuman Khandual

Quoted 7 lines · Shakeel Butt
The kvm slabs can consume a significant amount of system memory and indeed in our production environment we have observed that a lot of machines are spending significant amount of memory that can not be left as system memory overhead. Also the allocations from these slabs can be triggered directly by user space applications which has access to kvm and thus a buggy application can leak such memory. So, these caches should be accounted to kmemcg.

Quoted 4 lines

But there may be other situations like this where user space can trigger allocation from various SLAB objects inside the kernel which are accounted as system memory. So how we draw the line which ones should be accounted for memcg. Just being curious.

The thing is that we used to have an opt-out approach for kmem accounting but we decided to go opt-in in a9bb7e620efd ("memcg: only account kmem allocations marked as __GFP_ACCOUNT").

Since then we are adding the flag to caches/allocations which can go wild and consume a lot of or even unbounded amount of memory.

Signature · 8 lines

Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Paolo Bonzini

maintainer Reviewed-by on v1 1/1

2017-10-06 08:55 UTC #4 lore raw

In reply to Shakeel Butt

Quoted 40 lines · Shakeel Butt

The kvm slabs can consume a significant amount of system memory and indeed in our production environment we have observed that a lot of machines are spending significant amount of memory that can not be left as system memory overhead. Also the allocations from these slabs can be triggered directly by user space applications which has access to kvm and thus a buggy application can leak such memory. So, these caches should be accounted to kmemcg.
Signed-off-by Shakeel Butt <[email protected]>
---
 arch/x86/kvm/mmu.c  | 4 ++--
 virt/kvm/kvm_main.c | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)
arch/x86/kvm/mmu.c +2 -2
index eca30c1eb1d9..87c5db9e644d 100644
@@ -5475,13 +5475,13 @@ int kvm_mmu_module_init(void)
 
 	pte_list_desc_cache = kmem_cache_create("pte_list_desc",
 					    sizeof(struct pte_list_desc),
-					    0, 0, NULL);
+					    0, SLAB_ACCOUNT, NULL);
 	if (!pte_list_desc_cache)
 		goto nomem;
 
 	mmu_page_header_cache = kmem_cache_create("kvm_mmu_page_header",
 						  sizeof(struct kvm_mmu_page),
-						  0, 0, NULL);
+						  0, SLAB_ACCOUNT, NULL);
 	if (!mmu_page_header_cache)
 		goto nomem;
 
virt/kvm/kvm_main.c +0 -0
index 9deb5a245b83..3d73299e05f2 100644

virt/kvm/kvm_main.c int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align, · line 4016

Quoted 8 lines

virt/kvm/kvm_main.c +1 -1

@@ -4010,7 +4010,7 @@ int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align,
 	if (!vcpu_align)
 		vcpu_align = __alignof__(struct kvm_vcpu);
 	kvm_vcpu_cache = kmem_cache_create("kvm_vcpu", vcpu_size, vcpu_align,
-					   0, NULL);
+					   SLAB_ACCOUNT, NULL);
 	if (!kvm_vcpu_cache) {
 		r = -ENOMEM;
 		goto out_free_3;

Reviewed-by Paolo Bonzini <[email protected]>

Adding maintainers for other architectures, because they probably want to do something similar.

Paolo

Paolo Bonzini

maintainer Applied

2017-10-10 08:32 UTC #5 lore raw

In reply to Shakeel Butt

Quoted 40 lines · Shakeel Butt

The kvm slabs can consume a significant amount of system memory and indeed in our production environment we have observed that a lot of machines are spending significant amount of memory that can not be left as system memory overhead. Also the allocations from these slabs can be triggered directly by user space applications which has access to kvm and thus a buggy application can leak such memory. So, these caches should be accounted to kmemcg.
Signed-off-by Shakeel Butt <[email protected]>
---
 arch/x86/kvm/mmu.c  | 4 ++--
 virt/kvm/kvm_main.c | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)
arch/x86/kvm/mmu.c +2 -2
index eca30c1eb1d9..87c5db9e644d 100644
@@ -5475,13 +5475,13 @@ int kvm_mmu_module_init(void)
 
 	pte_list_desc_cache = kmem_cache_create("pte_list_desc",
 					    sizeof(struct pte_list_desc),
-					    0, 0, NULL);
+					    0, SLAB_ACCOUNT, NULL);
 	if (!pte_list_desc_cache)
 		goto nomem;
 
 	mmu_page_header_cache = kmem_cache_create("kvm_mmu_page_header",
 						  sizeof(struct kvm_mmu_page),
-						  0, 0, NULL);
+						  0, SLAB_ACCOUNT, NULL);
 	if (!mmu_page_header_cache)
 		goto nomem;
 
virt/kvm/kvm_main.c +0 -0
index 9deb5a245b83..3d73299e05f2 100644

virt/kvm/kvm_main.c int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align, · line 4016

Quoted 8 lines

virt/kvm/kvm_main.c +1 -1

@@ -4010,7 +4010,7 @@ int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align,
 	if (!vcpu_align)
 		vcpu_align = __alignof__(struct kvm_vcpu);
 	kvm_vcpu_cache = kmem_cache_create("kvm_vcpu", vcpu_size, vcpu_align,
-					   0, NULL);
+					   SLAB_ACCOUNT, NULL);
 	if (!kvm_vcpu_cache) {
 		r = -ENOMEM;
 		goto out_free_3;

Queued, thanks.

Paolo