rcu: call_rcu() power improvements
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v9 Latest13 patches 5 repliesIn review
- v813 patches 11 repliesSuperseded
- v711 patches 33 repliesSuperseded
Earlier versions · 1
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v6 | 4 patches | 57 replies | Superseded |
Patches in v9 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/13 | rcu: Fix missing nocb gp wake on rcu_barrier()kernel/ kernel/ kernel/ | +17 | 2 replies | |
| 2/13 | rcu: Make call_rcu() lazy to save powerinclude/ kernel/ kernel/ +5 more | +244 | 0 replies | |
| 3/13 | rcu: Refactor code a bit in rcu_nocb_do_flush_bypass()kernel/ | +9 | 0 replies | |
| 4/13 | rcu: shrinker for lazy rcukernel/ | +52 | 0 replies | |
| 5/13 | rcuscale: Add laziness and kfree testskernel/ | +66 | 0 replies | |
| 6/13 | percpu-refcount: Use call_rcu_flush() for atomic switchlib/ | +2 | 0 replies | |
| 7/13 | rcu/sync: Use call_rcu_flush() instead of call_rcukernel/ | +1 | 0 replies | |
| 8/13 | rcu/rcuscale: Use call_rcu_flush() for async reader testkernel/ | +1 | 0 replies | |
| 9/13 | rcu/rcutorture: Use call_rcu_flush() where neededkernel/ | +8 | 0 replies | |
| 10/13 | scsi/scsi_error: Use call_rcu_flush() instead of call_rcu()drivers/ | +1 | 0 replies | |
| 11/13 | workqueue: Make queue_rcu_work() use call_rcu_flush()kernel/ | +1 | 0 replies | |
| 12/13 | rxrpc: Use call_rcu_flush() instead of call_rcu()net/ | +1 | 0 replies | |
| 13/13 | rcu/debug: Add wake-up debugging for lazy callbackskernel/ kernel/ kernel/ | +170 | 0 replies |
Reviews trailers per patch
1 of 13 patches has a Reviewed-by or Acked-by.
| Patch | Frederic Weisbecker | Joel Fernandes (Google) |
|---|---|---|
| 1/13 | ||
| 2/13 | A-b | |
| 3/13 | ||
| 4/13 | ||
| 5/13 | ||
| 6/13 | ||
| 7/13 | ||
| 8/13 | ||
| 9/13 | ||
| 10/13 | T-b | |
| 11/13 | ||
| 12/13 | ||
| 13/13 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH v9 00/13] rcu: call_rcu() power improvements
v9 version of RCU lazy patches based on rcu/next branch.
Only change since v8 is this discussion:
https://lore.kernel.org/rcu/[email protected]/T/#m8eff15110477f3430b3b02561b66f7b0d34a73b0
To facilitate easier merge, I dropped tracing and other patches and just implemented the new changes. I will post the tracing patches later along with rcutop as I need to add new tracepoints that Frederic suggested.
Main recent changes:
1. rcu_barrier() wake up only for lazy bypass list.
2. Make all call_rcu() default-lazy and add call_rcu_flush() API.
3. Take care of some callers using call_rcu_flush() API.
4. Several refactorings suggested by Paul/Frederic.
5. New call_rcu() to call_rcu_flush() conversions by Joel/Vlad/Paul.
I am seeing good performance and power with these patches on real ChromeOS x86 asymmetric hardware.
Earlier cover letter with lots of details is here:
https://lore.kernel.org/all/[email protected]/
List of recent changes:
[ Frederic Weisbec: Program the lazy timer only if WAKE_NOT, since other
deferral levels wake much earlier so for those it is not needed. ]
[ Frederic Weisbec: Use flush flags to keep bypass API code clean. ]
[ Frederic Weisbec: Make rcu_barrier() wake up only if main list empty. ]
[ Frederic Weisbec: Remove extra 'else if' branch in rcu_nocb_try_bypass(). ]
[ Joel: Fix issue where I was not resetting lazy_len after moving it to rdp ]
[ Paul/Thomas/Joel: Make call_rcu() default lazy so users don't mess up. ]
[ Paul/Frederic : Cosmetic changes, split out wakeup of nocb thread. ]
[ Vlad/Joel : More call_rcu -> flush conversions ]
[ debug code for detecting "wake" in kernel's call_rcu() callbacks. ]
The following 2 scripts can be used to check if any callbacks in the kernel are doing a wake up (it is best effort and may miss some things, but we found issues using it)
1. Script to search for call_rcu() references and dump the callback list to a file:
#!/bin/bash
rm func-list
touch func-list
for f in $(find . \( -name "*.c" -o -name "*.h" \) | grep -v rcu); do
funcs=$(perl -0777 -ne 'while(m/call_rcu\([&]?.+,\s?(.+)\).*;/g){print "$1\n";}' $f)
if [ "x$funcs" != "x" ]; then
for func in $funcs; do
echo "$f $func" >> func-list
echo "$f $func"
done
fidone
cat func-list | sort | uniq | tee func-list-sorted
2. Script to search "wake" after callback references:
#!/bin/bash
while read fl; do
file=$(echo $fl | cut -d " " -f1) func=$(echo $fl | cut -d " " -f2) grep -A 30 $func $file | grep wake > /dev/null if [ $? -eq 0 ]; then echo "keyword wake found after function reference $func in $file" echo "Output:" grep -A 30 $func $file echo "===========================================================" fi
done < func-list-sorted
Frederic Weisbecker (1):
rcu: Fix missing nocb gp wake on rcu_barrier()
Joel Fernandes (Google) (9):
rcu: Make call_rcu() lazy to save power
rcu: Refactor code a bit in rcu_nocb_do_flush_bypass()
rcuscale: Add laziness and kfree tests
percpu-refcount: Use call_rcu_flush() for atomic switch rcu/sync: Use call_rcu_flush() instead of call_rcu rcu/rcuscale: Use call_rcu_flush() for async reader test rcu/rcutorture: Use call_rcu_flush() where needed
rxrpc: Use call_rcu_flush() instead of call_rcu() rcu/debug: Add wake-up debugging for lazy callbacks
Uladzislau Rezki (2):
scsi/scsi_error: Use call_rcu_flush() instead of call_rcu()
workqueue: Make queue_rcu_work() use call_rcu_flush()
Vineeth Pillai (1):
rcu: shrinker for lazy rcu
drivers/scsi/scsi_error.c | 2 +- include/linux/rcupdate.h | 7 ++ kernel/rcu/Kconfig | 15 +++ kernel/rcu/lazy-debug.h | 154 +++++++++++++++++++++++++++ kernel/rcu/rcu.h | 8 ++ kernel/rcu/rcuscale.c | 70 +++++++++++- kernel/rcu/rcutorture.c | 16 +-- kernel/rcu/sync.c | 2 +- kernel/rcu/tiny.c | 2 +- kernel/rcu/tree.c | 149 ++++++++++++++++++-------- kernel/rcu/tree.h | 12 ++- kernel/rcu/tree_exp.h | 2 +- kernel/rcu/tree_nocb.h | 217 ++++++++++++++++++++++++++++++++------ kernel/workqueue.c | 2 +- lib/percpu-refcount.c | 3 +- net/rxrpc/conn_object.c | 2 +- 16 files changed, 565 insertions(+), 98 deletions(-)
create mode 100644 kernel/rcu/lazy-debug.h
--
2.38.0.413.g74048e4d9e-goog