Skip to content

mm/madvise: remove redundant mmap_lock operations from process_madvise()

SeongJae Park linux-mm lkml 1 version Applied

Status from this message: [merged mm-stable] mm-madvise-remove-redundant-mmap_lock-operations-from-process_madvise.patch removed from -mm tree

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest4 patches 12 repliesApplied 4/4

Changes from RFC v2 from the cover letter

(https://lore.kernel.org/[email protected])
- Release and acquire mmap lock again when a race-caused failure happens (Lorenzo Stoakes)
- Collected Reviewed-by: tags from Shakeel, Lorenzo and Davidlohr.

Patches in v1 All 4 applied

#Subject and filesSizeRepliesApplied
1/4mm/madvise: split out mmap locking operations for madvise()mm/madvise.c+32 -131 replyApplied
2/4mm/madvise: split out madvise input validity checkmm/madvise.c+22 -101 replyApplied
3/4mm/madvise: split out madvise() behavior executionmm/madvise.c+30 -231 replyApplied
4/4mm/madvise: remove redundant mmap_lock operations from process_madvise()mm/madvise.c+24 -27 repliesApplied

Reviews trailers per patch

All 4 patches have a Reviewed-by or Acked-by.

PatchShakeel ButtLorenzo StoakesDavidlohr BuesoLiam R. Howlett
1/4R-b R-b R-b R-b
2/4R-b R-b R-b R-b
3/4R-b R-b R-b
4/4R-b R-b R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 4 reviewers.

Read the v1 thread

Cover letter [PATCH 0/4] mm/madvise: remove redundant mmap_lock operations from process_madvise()

process_madvise() calls do_madvise() for each address range. Then, each do_madvise() invocation holds and releases same mmap_lock. Optimize the redundant lock operations by splitting do_madvise() internal logics including the mmap_lock operations, and calling the small logics directly from process_madvise() in a sequence that removes the redundant locking. As a result of this change, process_madvise() becomes more efficient and less racy in terms of its results and latency.

Note that the potential downside of this series is that other mmap_lock holders may take more time due to the increased length of mmap_lock critical section for process_madvise() calls. But there is maximum limit in the kernel space (IOV_MAX), and the user space can control the critical section length by setting the request size. Hence, the downside would be limited and controllable.

Evaluation
==========

I measured the time to apply MADV_DONTNEED advice to 256 MiB memory using multiple madvise() calls, 4 KiB per each call. I also do the same with process_madvise(), but with varying batch size (vlen) from 1 to 1024. The source code for the measurement is available at GitHub[1]. Because the microbenchmark result is not that stable, I ran each configuration five times and use the average.

The measurement results are as below. 'sz_batches' column shows the batch size of process_madvise() calls. '0' batch size is for madvise() calls case. 'before' and 'after' columns are the measured time to apply MADV_DONTNEED to the 256 MiB memory buffer in nanoseconds, on kernels that built without and with the last patch of this series, respectively. So lower value means better efficiency. 'after/before' column is the ratio of 'after' to 'before'.

    sz_batches  before       after        after/before
    0           146294215.2  121280536.2  0.829017989769427
    1           165851018.8  136305598.2  0.821855658085351
    2           129469321.2  103740383.6  0.801273866569094
    4           110369232.4  87835896.2   0.795836795182785
    8           102906232.4  77420920.2   0.752344327397609
    16          97551017.4   74959714.4   0.768415506038587
    32          94809848.2   71200848.4   0.750985786305689
    64          96087575.6   72593180     0.755489765942227
    128         96154163.8   68517055.4   0.712575022154163
    256         92901257.6   69054216.6   0.743307662177439
    512         93646170.8   67053296.2   0.716028168874151
    1024        92663219.2   70168196.8   0.75723892830177

In despite of the unstable nature of the tet program, the trend is somewhat we can expect. The measurement shows this patch reduces the process_madvise() latency, proportional to the batching size. The latency gain was about 20% with the batch size 2, and it has increased to about 28% with the batch size 512, since more number of mmap locking is reduced with larger batch size.

Note that the standard devitation of the measurements for each sz_batches configuration was ranging from 1.9% to 7.2%. That is, this result is still not very stable. The average of the standard deviations for different batch sizes were 4.62% and 4.70% for the 'before' and 'after' kernel measurements.

Also note that this patch has somehow decreased latencies of madvise() and single batch size process_madvise(). Seems this code path is small enough to significantly be affected by compiler optimizations including inlining of split-out functions. Please focus on only the improvement amount that changed by the batch size.

Changelog
=========

Changes from RFC v2
(https://lore.kernel.org/[email protected])
- Release and acquire mmap lock again when a race-caused failure happens (Lorenzo Stoakes)
- Collected Reviewed-by: tags from Shakeel, Lorenzo and Davidlohr.

Changes from RFC v1
(https://lore.kernel.org/[email protected])
- Split out do_madvise() and use those from vector_madvise(), instead of adding a flag to do_madvise() (Liam R. Howlett)

[1] https://github.com/sjp38/eval_proc_madvise

SeongJae Park (4):
mm/madvise: split out mmap locking operations for madvise() mm/madvise: split out madvise input validity check mm/madvise: split out madvise() behavior execution mm/madvise: remove redundant mmap_lock operations from
process_madvise()

 mm/madvise.c | 154 +++++++++++++++++++++++++++++++++++----------------
 1 file changed, 107 insertions(+), 47 deletions(-)

base-commit: f104b8534d19f31443a4fe6cb701bdb15fd931eb

Signature · 1 line
2.39.5