isolate_freepages_block(): very high intermittent overhead
On my v3.18-rc2 kernel isolate_freepages_block() sometimes shows up very high (>20%) in perf top during the configuration phase of software builds. It increases build time considerably.
Unfortunately the issue is not 100% reproducible, because it appears only intermittently. And the symptoms vanish after a few minutes.
I think the "mm, compaction" series from Vlastimil is to blame, but it's hard to be sure when bisection doesn't work.
Signature · 7 lines
Markus -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 9 lines · Markus Trippelsdorf
On my v3.18-rc2 kernel
isolate_freepages_block()sometimes shows up very high (>20%) in perf top during the configuration phase of software builds. It increases build time considerably.Unfortunately the issue is not 100% reproducible, because it appears only intermittently. And the symptoms vanish after a few minutes.
I think the "mm, compaction" series from Vlastimil is to blame, but it's hard to be sure when bisection doesn't work.
Here is an example:
Overhead Shared Object Symbol 59.12% [kernel] [k] isolate_freepages_block 4.75% [kernel] [k] amd_e400_idle 1.10% [kernel] [k] clear_page_c 0.89% [kernel] [k] get_pfnblock_flags_mask 0.64% ld-2.19.90.so [.] do_lookup_x 0.64% [kernel] [k] unmap_single_vma 0.58% [kernel] [k] copy_page_rep 0.54% libc-2.19.90.so [.] memcpy@@GLIBC_2.14 0.52% [kernel] [k] filemap_map_pages 0.47% [kernel] [k] _cond_resched
Signature · 7 lines
Markus -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 6 lines · Markus Trippelsdorf
On my v3.18-rc2 kernel
isolate_freepages_block()sometimes shows up very high (>20%) in perf top during the configuration phase of software builds. It increases build time considerably.Unfortunately the issue is not 100% reproducible, because it appears only intermittently. And the symptoms vanish after a few minutes.
Does it happen for long enough so you can capture it by perf record -g ?
Vlastimil
Quoted 3 lines
I think the "mm, compaction" series from Vlastimil is to blame, but it's hard to be sure when bisection doesn't work.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 9 lines · Vlastimil Babka
Quoted 6 lines · Markus Trippelsdorf
On my v3.18-rc2 kernel
isolate_freepages_block()sometimes shows up very high (>20%) in perf top during the configuration phase of software builds. It increases build time considerably.Unfortunately the issue is not 100% reproducible, because it appears only intermittently. And the symptoms vanish after a few minutes.
Does it happen for long enough so you can capture it by perf record -g ?
It only happens when I use the "Lockless Allocator":
http://locklessinc.com/downloads/lockless_allocator_src.tgz
I use: LD_PRELOAD=/usr/lib/libllalloc.so.1.3 when building software, because it gives me a ~8% speed boost over glibc's malloc.
Unfortunately, I don't have time to debug this further and have disabled "Transparent Hugepage Support" for now.
Signature · 7 lines
Markus -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 9 lines · Markus Trippelsdorf
Quoted 8 lines · Vlastimil Babka
Quoted 6 lines · Markus Trippelsdorf
On my v3.18-rc2 kernel
isolate_freepages_block()sometimes shows up very high (>20%) in perf top during the configuration phase of software builds. It increases build time considerably.Unfortunately the issue is not 100% reproducible, because it appears only intermittently. And the symptoms vanish after a few minutes.
Quoted 7 lines
Quoted 1 line
Does it happen for long enough so you can capture it by perf record -g ?
It only happens when I use the "Lockless Allocator":
http://locklessinc.com/downloads/lockless_allocator_src.tgzI use: LD_PRELOAD=/usr/lib/libllalloc.so.1.3 when building software, because it gives me a ~8% speed boost over glibc's malloc.
Hm I see. I'll try to test that.
Quoted 2 lines
Unfortunately, I don't have time to debug this further and have disabled "Transparent Hugepage Support" for now.
That's unfortunate indeed. Commit e14c720efdd73c6d69cd8d07fa894bcd11fe1973 "mm, compaction: remember position within pageblock in free pages scanner" would be the most suspicious one here I guess, so testing at least a kernel with this patch reverted would be very useful. Simple git revert seems to apply cleanly here.
Thanks,
Vlastimil
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 9 lines · Markus Trippelsdorf
Quoted 8 lines · Vlastimil Babka
Quoted 6 lines · Markus Trippelsdorf
On my v3.18-rc2 kernel
isolate_freepages_block()sometimes shows up very high (>20%) in perf top during the configuration phase of software builds. It increases build time considerably.Unfortunately the issue is not 100% reproducible, because it appears only intermittently. And the symptoms vanish after a few minutes.
Quoted 7 lines
Quoted 1 line
Does it happen for long enough so you can capture it by perf record -g ?
It only happens when I use the "Lockless Allocator":
http://locklessinc.com/downloads/lockless_allocator_src.tgzI use: LD_PRELOAD=/usr/lib/libllalloc.so.1.3 when building software, because it gives me a ~8% speed boost over glibc's malloc.
I tried the allocator while updating my gentoo desktop with 3.18-rc3 and adding some extra memory pressure, but didn't observe anything like this. It could be system specific. If you do't have the time to debug, can you at least send me output of "cat /proc/zoneinfo"?
Thanks,
Vlastimil
Quoted 3 lines
Unfortunately, I don't have time to debug this further and have disabled "Transparent Hugepage Support" for now.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 15 lines · Vlastimil Babka
Quoted 14 lines · Markus Trippelsdorf
Quoted 9 lines · Vlastimil Babka
Quoted 6 lines · Markus Trippelsdorf
On my v3.18-rc2 kernel
isolate_freepages_block()sometimes shows up very high (>20%) in perf top during the configuration phase of software builds. It increases build time considerably.Unfortunately the issue is not 100% reproducible, because it appears only intermittently. And the symptoms vanish after a few minutes.
Does it happen for long enough so you can capture it by perf record -g ?
It only happens when I use the "Lockless Allocator":
http://locklessinc.com/downloads/lockless_allocator_src.tgz
Quoted 7 lines
Quoted 2 lines
I use: LD_PRELOAD=/usr/lib/libllalloc.so.1.3 when building software, because it gives me a ~8% speed boost over glibc's malloc.
I tried the allocator while updating my gentoo desktop with 3.18-rc3 and adding some extra memory pressure, but didn't observe anything like this. It could be system specific. If you do't have the time to debug, can you at least send me output of "cat
/proc/zoneinfo"?
I will try to debug this further this weekend.
BTW there is an interesting thread on LKML that might be related to this
issue: https://lkml.org/lkml/2014/11/4/904
Signature · 7 lines
Markus -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>