LSFMMBPF proposal [MM]: Eliminate vmap/vmalloc lock contention
Hello, LSF.
Title: Introduce a per-cpu-vmap-cache to eliminate a vmap lock contention
Description:
Currently the vmap code is not scaled to number of CPU cores in a system because a global vmap space is protected by a single spinlock. Such approach has a clear bottleneck if many CPUs simultaneously access to one resource.
In this talk i would like to describe a drawback, show some data related to contentions and places where those occur in a code. Apart of that i would like to share ideas how to eliminate it providing a few approaches and compare them.
Requirements:
* It should be a per-cpu approach;
* Search of freed ptrs should not interfere with other freeing(as much as we can);
* - offload allocated areas(buzy ones) per-cpu;
* Cache ready sized objects or merge them into one big per-cpu-space(split on demand);
* Lazily-freed areas either drained per-cpu individually or by one CPU for all;
* Prefetch a fixed size in front and allocate per-cpu
Goals:
* Implement a per-cpu way of allocation to eliminate a contention.
Thanks!
--
Uladzislau Rezki
Quoted 22 lines · Uladzislau Rezki
Hello, LSF.
Title: Introduce a per-cpu-vmap-cache to eliminate a vmap lock contention
Description:
Currently the vmap code is not scaled to number of CPU cores in a system because a global vmap space is protected by a single spinlock. Such approach has a clear bottleneck if many CPUs simultaneously access to one resource.In this talk i would like to describe a drawback, show some data related to contentions and places where those occur in a code. Apart of that i would like to share ideas how to eliminate it providing a few approaches and compare them.
Requirements:
* It should be a per-cpu approach;
* Search of freed ptrs should not interfere with other freeing(as much as we can);
* - offload allocated areas(buzy ones) per-cpu;
* Cache ready sized objects or merge them into one big per-cpu-space(split on demand);
* Lazily-freed areas either drained per-cpu individually or by one CPU for all;
* Prefetch a fixed size in front and allocate per-cpu
Quoted 8 lines
Goals:
* Implement a per-cpu way of allocation to eliminate a contention.Thanks!
--
Uladzislau Rezki
This is a really interesting topic and I'd really like this to go ahead as there's certainly been a lot of discussion regarding vmalloc locking of late. I for one would certainly attend it.
Quoted 9 lines · Uladzislau Rezki
Hello, LSF.
Title: Introduce a per-cpu-vmap-cache to eliminate a vmap lock contention
Description:
Currently the vmap code is not scaled to number of CPU cores in a system because a global vmap space is protected by a single spinlock. Such approach has a clear bottleneck if many CPUs simultaneously access to one resource.
Quoted 4 lines
In this talk i would like to describe a drawback, show some data related to contentions and places where those occur in a code. Apart of that i would like to share ideas how to eliminate it providing a few approaches and compare them.
It's been some time since you brough this up. Has there been any progress on the topic? Do you still find it important to discuss it at LSFMM?
Quoted 19 lines
Requirements:
* It should be a per-cpu approach;
* Search of freed ptrs should not interfere with other freeing(as much as we can);
* - offload allocated areas(buzy ones) per-cpu;
* Cache ready sized objects or merge them into one big per-cpu-space(split on demand);
* Lazily-freed areas either drained per-cpu individually or by one CPU for all;
* Prefetch a fixed size in front and allocate per-cpuGoals:
* Implement a per-cpu way of allocation to eliminate a contention.Thanks!
--
Uladzislau Rezki
_______________________________________________
Lsf-pc mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/lsf-pc
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 10 lines · Michal Hocko
Quoted 9 lines · Uladzislau Rezki
Hello, LSF.
Title: Introduce a per-cpu-vmap-cache to eliminate a vmap lock contention
Description:
Currently the vmap code is not scaled to number of CPU cores in a system because a global vmap space is protected by a single spinlock. Such approach has a clear bottleneck if many CPUs simultaneously access to one resource.
Quoted 9 lines
Quoted 4 lines
In this talk i would like to describe a drawback, show some data related to contentions and places where those occur in a code. Apart of that i would like to share ideas how to eliminate it providing a few approaches and compare them.
It's been some time since you brough this up. Has there been any progress on the topic? Do you still find it important to discuss it at LSFMM?
The idea about sequence was/is:
1) Give an overview on the proposal;
2) Submit patches to address the problem;
3) Start a discussion over lkml with people who are interested in it;
4) Send out a complete solution.
--
Uladzislau Rezki
Quoted 20 lines · Uladzislau Rezki
Quoted 10 lines · Michal Hocko
Quoted 9 lines · Uladzislau Rezki
Hello, LSF.
Title: Introduce a per-cpu-vmap-cache to eliminate a vmap lock contention
Description:
Currently the vmap code is not scaled to number of CPU cores in a system because a global vmap space is protected by a single spinlock. Such approach has a clear bottleneck if many CPUs simultaneously access to one resource.Quoted 9 lines
Quoted 4 lines
In this talk i would like to describe a drawback, show some data related to contentions and places where those occur in a code. Apart of that i would like to share ideas how to eliminate it providing a few approaches and compare them.
It's been some time since you brough this up. Has there been any progress on the topic? Do you still find it important to discuss it at LSFMM?
Quoted 6 lines
The idea about sequence was/is:
1) Give an overview on the proposal;
2) Submit patches to address the problem;
3) Start a discussion over lkml with people who are interested in it;
4) Send out a complete solution.
Thanks for the clarification. The usual LSFMM format is strongly discussion focused. Long presentations are usually discouraged and they should only introduce people to the underlying problem to kick of a discussion.
That being said, IMO it would be helpful to have some material on the mailing list before any discussion could be productive.
Thanks!
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 28 lines · Michal Hocko
Quoted 20 lines · Uladzislau Rezki
Quoted 10 lines · Michal Hocko
Quoted 9 lines · Uladzislau Rezki
Hello, LSF.
Title: Introduce a per-cpu-vmap-cache to eliminate a vmap lock contention
Description:
Currently the vmap code is not scaled to number of CPU cores in a system because a global vmap space is protected by a single spinlock. Such approach has a clear bottleneck if many CPUs simultaneously access to one resource.Quoted 9 lines
Quoted 4 lines
In this talk i would like to describe a drawback, show some data related to contentions and places where those occur in a code. Apart of that i would like to share ideas how to eliminate it providing a few approaches and compare them.
It's been some time since you brough this up. Has there been any progress on the topic? Do you still find it important to discuss it at LSFMM?
Quoted 6 lines
The idea about sequence was/is:
1) Give an overview on the proposal;
2) Submit patches to address the problem;
3) Start a discussion over lkml with people who are interested in it;
4) Send out a complete solution.
Quoted 5 lines
Thanks for the clarification. The usual LSFMM format is strongly discussion focused. Long presentations are usually discouraged and they should only introduce people to the underlying problem to kick of a discussion.
I have not posted yet any RFC and have not kicked it yet. Though people are aware the problem.
Quoted 4 lines
That being said, IMO it would be helpful to have some material on the mailing list before any discussion could be productive.
This is what i have so far:
wget ftp://vps418301.ovh.net/incoming/Fix_a_vmalloc_lock_contention_in_SMP_env.pdf
I can, of course, move it forward over lkml only. If you are fully booked or there other reason then please just withdraw my proposal from your conference.
--
Uladzislau Rezki
Quoted 41 lines · Uladzislau Rezki
Quoted 28 lines · Michal Hocko
Quoted 20 lines · Uladzislau Rezki
Quoted 10 lines · Michal Hocko
Quoted 9 lines · Uladzislau Rezki
Hello, LSF.
Title: Introduce a per-cpu-vmap-cache to eliminate a vmap lock contention
Description:
Currently the vmap code is not scaled to number of CPU cores in a system because a global vmap space is protected by a single spinlock. Such approach has a clear bottleneck if many CPUs simultaneously access to one resource.Quoted 9 lines
Quoted 4 lines
In this talk i would like to describe a drawback, show some data related to contentions and places where those occur in a code. Apart of that i would like to share ideas how to eliminate it providing a few approaches and compare them.
It's been some time since you brough this up. Has there been any progress on the topic? Do you still find it important to discuss it at LSFMM?
Quoted 6 lines
The idea about sequence was/is:
1) Give an overview on the proposal;
2) Submit patches to address the problem;
3) Start a discussion over lkml with people who are interested in it;
4) Send out a complete solution.Quoted 5 lines
Thanks for the clarification. The usual LSFMM format is strongly discussion focused. Long presentations are usually discouraged and they should only introduce people to the underlying problem to kick of a discussion.
I have not posted yet any RFC and have not kicked it yet. Though people are aware the problem.
Quoted 4 lines
That being said, IMO it would be helpful to have some material on the mailing list before any discussion could be productive.
Quoted 7 lines
This is what i have so far:
wget ftp://vps418301.ovh.net/incoming/Fix_a_vmalloc_lock_contention_in_SMP_env.pdf
I can, of course, move it forward over lkml only. If you are fully booked or there other reason then please just withdraw my proposal from your conference.
For what it's worth I'm definitely interested in attending this session if it goes ahead. I am sure Baoquan, if he's attending LSF/MM, would be too! (cc'd).
Quoted 44 lines · Lorenzo Stoakes
Quoted 43 lines · Uladzislau Rezki
Quoted 28 lines · Michal Hocko
Quoted 20 lines · Uladzislau Rezki
Quoted 10 lines · Michal Hocko
Quoted 9 lines · Uladzislau Rezki
Hello, LSF.
Title: Introduce a per-cpu-vmap-cache to eliminate a vmap lock contention
Description:
Currently the vmap code is not scaled to number of CPU cores in a system because a global vmap space is protected by a single spinlock. Such approach has a clear bottleneck if many CPUs simultaneously access to one resource.Quoted 9 lines
Quoted 4 lines
In this talk i would like to describe a drawback, show some data related to contentions and places where those occur in a code. Apart of that i would like to share ideas how to eliminate it providing a few approaches and compare them.
It's been some time since you brough this up. Has there been any progress on the topic? Do you still find it important to discuss it at LSFMM?
Quoted 6 lines
The idea about sequence was/is:
1) Give an overview on the proposal;
2) Submit patches to address the problem;
3) Start a discussion over lkml with people who are interested in it;
4) Send out a complete solution.Quoted 5 lines
Thanks for the clarification. The usual LSFMM format is strongly discussion focused. Long presentations are usually discouraged and they should only introduce people to the underlying problem to kick of a discussion.
I have not posted yet any RFC and have not kicked it yet. Though people are aware the problem.
Quoted 4 lines
That being said, IMO it would be helpful to have some material on the mailing list before any discussion could be productive.
This is what i have so far:
Quoted 8 lines
Quoted 5 lines
wget ftp://vps418301.ovh.net/incoming/Fix_a_vmalloc_lock_contention_in_SMP_env.pdf
I can, of course, move it forward over lkml only. If you are fully booked or there other reason then please just withdraw my proposal from your conference.
For what it's worth I'm definitely interested in attending this session if it goes ahead. I am sure Baoquan, if he's attending LSF/MM, would be too! (cc'd).
Thanks for CC.
Yes, I am very interested in this topic, have read it from the beginning. Unfortunately, I can't attend LSF/MM this year because of some reasons. I look forward to learning the decision or conclusion of the sessin, and reviewing Uladzislau's RFC or formal patchset later. Wish the session a great success, and you guys a pleasant meeting and discussion.
Thanks
Baoquan
Quoted 23 lines
Hello, LSF.
Title: Introduce a per-cpu-vmap-cache to eliminate a vmap lock contention
Description:
Currently the vmap code is not scaled to number of CPU cores in a system because a global vmap space is protected by a single spinlock. Such approach has a clear bottleneck if many CPUs simultaneously access to one resource.In this talk i would like to describe a drawback, show some data related to contentions and places where those occur in a code. Apart of that i would like to share ideas how to eliminate it providing a few approaches and compare them.
Requirements:
* It should be a per-cpu approach;
* Search of freed ptrs should not interfere with other freeing(as much as we can);
* - offload allocated areas(buzy ones) per-cpu;
* Cache ready sized objects or merge them into one big per-cpu-space(split on demand);
* Lazily-freed areas either drained per-cpu individually or by one CPU for all;
* Prefetch a fixed size in front and allocate per-cpu
Quoted 3 lines
Goals:
* Implement a per-cpu way of allocation to eliminate a contention.
Thank you for giving a time to talk about this topic on the LSFMMBPF. The slides can
be accessed here:
wget ftp://vps418301.ovh.net/incoming/Mitigate_a_vmalloc_lock_contention_in_SMP_env_v1.pdf
There were some issues with a camera, but i was told in the end you could see me.2
Thank you again!
Signature · 1 line
Uladzislau Rezki