Skip to content

latched RB-trees and __module_address()

Peter Zijlstra lkml 4 versions – Applied

Status from this message: Applied,

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v6 Latest9 patches 4 repliesApplied
  2. v510 patches 35 repliesSuperseded
  3. v49 patches 12 repliesSuperseded
Earlier versions · 1
VersionPostedPatchesRepliesStatus
v18 patches19 repliesSuperseded

Patches in v6 Applied

#Subject and filesSizeRepliesApplied
1/9rbtree: Make lockless searches non-fatalinclude/linux/rbtree.h include/linux/rbtree_augmented.h lib/rbtree.c+81 -320 replies
2/9seqlock: Better document raw_write_seqcount_latch()include/linux/seqlock.h kernel/time/timekeeping.c+76 -270 replies
3/9rcu: Move lockless_dereference() out of rcupdate.hinclude/linux/compiler.h include/linux/rcupdate.h+15 -150 replies
4/9seqlock: Introduce raw_read_seqcount_latch()include/linux/seqlock.h kernel/time/timekeeping.c+8 -30 replies
5/9rbtree: Implement generic latch_treeinclude/linux/rbtree_latch.h+212 -00 replies
6/9module: Optimize __module_address() using a latched RB-treeinclude/linux/module.h kernel/module.c+136 -80 replies
7/9module: Make the mod_tree stuff conditional on PERF_EVENTS || TRACINGinclude/linux/module.h init/Kconfig kernel/module.c+35 -30 replies
8/9module: Use __module_address() for module_address_lookup()kernel/module.c+7 -100 replies
9/9module: Rework module_addr_{min,max}kernel/module.c+52 -280 replies

Reviews trailers per patch

2 of 9 patches have a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchMichel LespinasseAndrew MortonMathieu DesnoyersRusty Russell
SeriesA-b
1/9R-b
2/9A-b R-b
3/9
4/9
5/9
6/9
7/9Requested-by
8/9
9/9

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 4 reviewers.

Read the v6 thread

Cover letter [PATCH v6 0/9] latched RB-trees and __module_address()

This series is aimed at making __module_address() go fast(er).

The reason for doing so is that most stack unwinders use kernel_text_address() to validate each frame. Perf and ftrace (can) end up doing a lot of stack traces from performance sensitive code.

On the way there it:
- annotates and sanitizes module locking
- introduces the latched RB-tree
- employs it to make __module_address() go fast.

I've build and boot tested this on x86_64 with modules and lockdep enabled. Performance numbers (below) are done with lockdep disabled.

As previously mentioned; the reason for writing the latched RB-tree as generic code is mostly for clarity/documentation purposes; as there are a number of separate and non trivial bits to the complete solution.

As measured on my ivb-ep system with 84 modules loaded; the test module reports (cache hot, performance cpufreq):

avg +- stdev

Before:   611 +- 10 [ns] per __module_address() call
After:     17 +-  5 [ns] per __module_address() call

PMI measurements for a cpu running loops in a module (also [ns]):

Before: Mean: 2719 +- 1, Stdev: 214, Samples: 40036
After: Mean: 947 +- 0, Stdev: 132, Samples: 40037

Note; I have also tested things like: perf record -a -g modprobe mod_test, to make 'sure' to hit some of the more interesting paths.

Changes since last time:

- rebased against Rusty's tree
- raw_read_seqcount_latch() -- (mingo)

Based on rusty/linux.git/pending-rebases; please consider for 4.2

Thanks!