perf, x86: Fall back to vmalloc for BTS buffer allocation
Subsystems (2)
PATCH 1/1 perf, x86: Fall back to vmalloc for BTS buffer allocation
From: Andi Kleen <[email protected]>
On my workstation the order 4 BTS buffer allocation fails regularly after the system has been up for some time due to memory fragmentation.
BTS is virtual memory, so we can just fall back to vmalloc instead of failing.
Do this here.
--- arch/x86/kernel/cpu/perf_event_intel_ds.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_ds.c +6 -3
@@ -312,8 +312,11 @@ static int alloc_bts_buffer(int cpu) buffer =kzalloc_node (BTS_BUFFER_SIZE ,GFP_KERNEL | __GFP_NOWARN, node);if (unlikely (!buffer)) { -WARN_ONCE (1 ,"%s: BTS buffer allocation failure\n" , __func__); -return -ENOMEM ; + buffer =vmalloc_node (BTS_BUFFER_SIZE , node); +if (!buffer) { +WARN_ONCE (1 ,"%s: BTS buffer allocation failure\n" , __func__); +return -ENOMEM ; + } } max =BTS_BUFFER_SIZE /BTS_RECORD_SIZE ;
@@ -336,7 +339,7 @@ static void release_bts_buffer(int cpu)if (!ds || !x86_pmu.bts)return ; -kfree ((void *)(unsigned long )ds->bts_buffer_base); +kvfree ((void *)(unsigned long )ds->bts_buffer_base); ds->bts_buffer_base =0 ; }
Signature · 1 line
1.9.3
Peter Zijlstra
maintainerQuoted 24 lines · Andi Kleen
From: Andi Kleen <[email protected]>
On my workstation the order 4 BTS buffer allocation fails regularly after the system has been up for some time due to memory fragmentation.
BTS is virtual memory, so we can just fall back to vmalloc instead of failing.
Do this here.
Signed-off-by Andi Kleen <[email protected]>--- arch/x86/kernel/cpu/perf_event_intel_ds.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)arch/x86/kernel/cpu/perf_event_intel_ds.c +0
-0@@ -312,8 +312,11 @@ static int alloc_bts_buffer(int cpu) buffer =kzalloc_node (BTS_BUFFER_SIZE ,GFP_KERNEL | __GFP_NOWARN, node);if (unlikely (!buffer)) {
arch/x86/kernel/cpu/perf_event_intel_ds.c static int alloc_bts_buffer(int cpu) · line 322
Quoted 10 lines
arch/x86/kernel/cpu/perf_event_intel_ds.c +5
-2-WARN_ONCE (1 ,"%s: BTS buffer allocation failure\n" , __func__); -return -ENOMEM ; + buffer =vmalloc_node (BTS_BUFFER_SIZE , node); +if (!buffer) { +WARN_ONCE (1 ,"%s: BTS buffer allocation failure\n" , __func__); +return -ENOMEM ; + } } max =BTS_BUFFER_SIZE /BTS_RECORD_SIZE ;
We did this once, and that blew up big time.
Even now, vmalloc_fault() has a very explicit:
WARN_ON_ONCE(in_nmi());
So this isn't going to happen until you fix that.
Quoted 7 lines
We did this once, and that blew up big time.
Even now,
vmalloc_fault()has a very explicit:WARN_ON_ONCE(in_nmi());So this isn't going to happen until you fix that.
Good point.
We just need to call vmalloc_sync_all at allocation time, no need to change vmalloc_fault.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.
Peter Zijlstra
maintainerQuoted 4 lines · Andi Kleen
Quoted 4 lines
We did this once, and that blew up big time.
Even now,
vmalloc_fault()has a very explicit:
Quoted 8 lines
Quoted 3 lines
WARN_ON_ONCE(in_nmi());So this isn't going to happen until you fix that.
Good point.
We just need to call
vmalloc_sync_allat allocation time, no need to changevmalloc_fault.
Somehow that wasn't considered adequate, but lemme try and dig out that thread. It should be somewhere...
Peter Zijlstra
maintainerQuoted 11 lines · Peter Zijlstra
Quoted 10 lines · Andi Kleen
Quoted 7 lines
We did this once, and that blew up big time.
Even now,
vmalloc_fault()has a very explicit:WARN_ON_ONCE(in_nmi());So this isn't going to happen until you fix that.
Good point.
Quoted 7 lines
Quoted 2 lines
We just need to call
vmalloc_sync_allat allocation time, no need to changevmalloc_fault.Somehow that wasn't considered adequate, but lemme try and dig out that thread. It should be somewhere...
Note that since then we have actually fixed the 'cannot fault from NMI context' thing, so we should be able to actually take those faults. I suspect we can simply remove those WARNs from the vmalloc fault path, but it would need double checking to see if there's no other reasons.
Also:
https://lkml.org/lkml/2014/4/10/581
https://patchwork.kernel.org/patch/3048111/
I'm not entirely sure what the final conclusion is, except to note that it didn't happen.
Quoted 4 lines
Note that since then we have actually fixed the 'cannot fault from NMI context' thing, so we should be able to actually take those faults. I suspect we can simply remove those WARNs from the vmalloc fault path, but it would need double checking to see if there's no other reasons.
I think we actually need to use vmalloc_sync_all, because the PEBS/DS microcode may do the first access, and those lose events when the page is not present.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only
Peter Zijlstra
maintainerQuoted 8 lines · Andi Kleen
Quoted 4 lines
Note that since then we have actually fixed the 'cannot fault from NMI context' thing, so we should be able to actually take those faults. I suspect we can simply remove those WARNs from the vmalloc fault path, but it would need double checking to see if there's no other reasons.
I think we actually need to use
vmalloc_sync_all, because the PEBS/DS microcode may do the first access, and those lose events when the page is not present.
You'll need to convince Linus: https://lkml.org/lkml/2010/7/14/478