Skip to content

locking/rwsem: Rwsem rearchitecture part 1

Waiman Long lkml 4 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 Latest11 patches 3 repliesIn review
  2. v311 patches 4 repliesSuperseded
  3. v210 patches 6 repliesSuperseded
Earlier versions · 1
VersionPostedPatchesRepliesStatus
v110 patches8 repliesSuperseded

Changes in v4 from the cover letter

- Update the DEBUG_RWSEMS_WARN_ON() macro in patch 6 to call

Patches in v4 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/11locking/rwsem: Relocate rwsem_down_read_failed()kernel/locking/rwsem-xadd.c+86 -860 replies
2/11locking/rwsem: Move owner setting code from rwsem.c to rwsem.hkernel/locking/rwsem-xadd.c kernel/locking/rwsem.c kernel/locking/rwsem.h+20 -220 replies
3/11locking/rwsem: Move rwsem internal function declarations to rwsem-xadd.hinclude/linux/rwsem.h kernel/locking/rwsem.h+7 -70 replies
4/11locking/rwsem: Micro-optimize rwsem_try_read_lock_unqueued()kernel/locking/rwsem-xadd.c+5 -100 replies
5/11locking/rwsem: Add debug check for __down_read*()kernel/locking/rwsem.c kernel/locking/rwsem.h+10 -50 replies
6/11locking/rwsem: Enhance DEBUG_RWSEMS_WARN_ON() macrokernel/locking/rwsem.c kernel/locking/rwsem.h+16 -82 replies
7/11locking/qspinlock_stat: Introduce a generic lockevent counting APIskernel/locking/lock_events.h kernel/locking/lock_events_list.h kernel/locking/qspinlock.c +2 more+181 -1140 replies
8/11locking/lock_events: Make lock_events available for all archs & other locksarch/Kconfig arch/x86/Kconfig kernel/locking/Makefile +3 more+183 -1400 replies
9/11locking/lock_events: Don't show pvqspinlock events on bare metalkernel/locking/lock_events.c+27 -10 replies
10/11locking/rwsem: Enable lock event countingarch/Kconfig kernel/locking/lock_events_list.h kernel/locking/rwsem-xadd.c +1 more+32 -10 replies
11/11locking/rwsem: Optimize rwsem structure for uncontended lock acquisitioninclude/linux/rwsem.h+15 -60 replies

Reviews trailers per patch

10 of 11 patches have a Reviewed-by or Acked-by.

PatchWill DeaconDavidlohr Bueso
1/11A-b A-b
2/11A-b
3/11A-b A-b
4/11A-b A-b
5/11A-b
6/11A-b
7/11A-b
8/11A-b
9/11A-b
10/11A-b
11/11

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v4 thread

Cover letter [PATCH-tip v4 00/11] locking/rwsem: Rwsem rearchitecture part 1

v4:
- Update the DEBUG_RWSEMS_WARN_ON() macro in patch 6 to call

    debug_locks_off().

- Update commit log of patch 11 to include benchmark data.

v3:
- Add patch 11 to move count and owner together as suggested by Linus.
- Reword the commit log of patch 2 to clarify the intent of that patch.

v2:
- Sync up to v4 of the part 0 patch.
- Remove the rwsem.h->rwsem-xadd.h renaming patch & change patches to modify rwsem.h instead of rwsem-xadd.h.
- Add a new patch to micro-optimize rwsem_try_read_lock_unqueued().

This is part 1 of a 3-part (0/1/2) series to rearchitect the internal operation of rwsem.

This part lays the foundation for part 2 without making any functional changes. This part includes the following changes:

1) Move code around and micro-optimize rwsem_try_read_lock_unqueued()

    (patches 1-4).

2) Enhance the DEBUG_RWSEMS_WARN_ON() macro to provide more information and add additional checks (patches 5 & 6).
3) Make the core qspinlock_stat.h code generic (lock event counting) so that it can be used by all the architectures as well as other locking subsystems such as rwsem (patches 7-10). Lock event counting help us visualize how frequently a code path is being used as well as spotting abnormal behavior due to bugs in the code without noticeably affecting kernel performance and hence behavior.
4) Reorganize rwsem structure to optimize for the uncontended case.

Both (2) and (3) are useful debugging aids.

Waiman Long (11):
locking/rwsem: Relocate rwsem_down_read_failed() locking/rwsem: Move owner setting code from rwsem.c to rwsem.h locking/rwsem: Move rwsem internal function declarations to
rwsem-xadd.h
locking/rwsem: Micro-optimize rwsem_try_read_lock_unqueued() locking/rwsem: Add debug check for __down_read*() locking/rwsem: Enhance DEBUG_RWSEMS_WARN_ON() macro locking/qspinlock_stat: Introduce a generic lockevent counting APIs locking/lock_events: Make lock_events available for all archs & other
locks
locking/lock_events: Don't show pvqspinlock events on bare metal locking/rwsem: Enable lock event counting
locking/rwsem: Optimize rwsem structure for uncontended lock
acquisition

 arch/Kconfig                        |   9 ++
 arch/x86/Kconfig                    |   8 -
 include/linux/rwsem.h               |  28 ++--
 kernel/locking/Makefile             |   1 +
 kernel/locking/lock_events.c        | 179 ++++++++++++++++++++
 kernel/locking/lock_events.h        |  59 +++++++
 kernel/locking/lock_events_list.h   |  67 ++++++++
 kernel/locking/qspinlock.c          |   8 +-
 kernel/locking/qspinlock_paravirt.h |  19 +--
 kernel/locking/qspinlock_stat.h     | 242 +++++-----------------------
 kernel/locking/rwsem-xadd.c         | 204 +++++++++++------------
 kernel/locking/rwsem.c              |  25 +--
 kernel/locking/rwsem.h              |  49 +++++-
 13 files changed, 540 insertions(+), 358 deletions(-)
 create mode 100644 kernel/locking/lock_events.c
 create mode 100644 kernel/locking/lock_events.h
 create mode 100644 kernel/locking/lock_events_list.h
Signature · 1 line
2.18.1