Skip to content

epoll: reduce 'epmutex' lock contention

Jason Baron linux-fsdevel lkml 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest2 patches 7 repliesIn review
  2. v13 patches 0 repliesSuperseded

Changes in v2 from the cover letter

-use a union for rcu callback and drop patch #3

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/2epoll: optimize EPOLL_CTL_DEL using rcufs/eventpoll.c+41 -242 replies
2/2epoll: Do not take global 'epmutex' for simple topologiesfs/eventpoll.c+68 -265 replies

Reviews trailers per patch

1 of 2 patches has a Reviewed-by or Acked-by.

PatchNathan ZimmerJason Baron
1/2T-b
2/2T-b R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v2 thread

Cover letter [PATCH 0/2 v2] epoll: reduce 'epmutex' lock contention

Hi,

Updated series, using a 'union' for the rcu callback.

Nathan Zimmer found that once we get over 10+ cpus, the scalability of SPECjbb falls over due to the contention on the global 'epmutex', which is taken in on EPOLL_CTL_ADD and EPOLL_CTL_DEL operations.

Patch #1 removes the 'epmutex' lock completely from the EPOLL_CTL_DEL path by using rcu to guard against any concurrent traversals.

Patch #2 remove the 'epmutex' lock from EPOLL_CTL_ADD operations for simple topologies. IE when adding a link from an epoll file descriptor to a wakeup source, where the epoll file descriptor is not nested.

Performance of SPECjbb improves considerably. From Nathan Zimmer's testing.
Thread: http://marc.info/?l=linux-kernel&m=137908766013329&w=2

"
On the 16 socket run the performance went from 35k jOPS to 125k jOPS. In addition the benchmark when from scaling well on 10 sockets to scaling well on just over 40 sockets.

I should also note there system responsiveness of various commands was quite improved when under the full load of the benchmark.
...

Currently the benchmark stops scaling at around 40-44 sockets but it seems like I found a second unrelated bottleneck.
"

Thanks,

-Jason

Changes in v2:
-use a union for rcu callback and drop patch #3

v1:
http://marc.info/?l=linux-kernel&m=138057995316598&w=2

Jason Baron (2):
epoll: optimize EPOLL_CTL_DEL using rcu
epoll: Do not take global 'epmutex' for simple topologies

 fs/eventpoll.c | 153 +++++++++++++++++++++++++++++++++++++++------------------
 1 file changed, 106 insertions(+), 47 deletions(-)
Signature · 1 line
1.8.2