locking/rwsem: Rework reader optimistic spinning

v2:
- Update some commit logs to incorporate review comments.
- Patch 2: remove unnecessary comment.
- Patch 3: rename osq_is_empty() to rwsem_no_spinners() as suggested.
- Patch 4: correctly handle HANDOFF clearing.
- Patch 5: fix !CONFIG_RWSEM_SPIN_ON_OWNER compilation errors.

A recent report of SAP certification failure caused by increased system time due to rwsem reader optimistic spinning led me to reexamine the code to see the pro and cons of doing it. This led me to discover a potential lock starvation scenario as explained in patch 2. That patch does reduce reader spinning to avoid this potential problem. Patches 3 and 4 are further optimizations of the current code.

Then there is the issue of reader fragmentation that can potentially reduce performance in some heavily contented cases. Two different approaches are attempted:
1) further reduce reader optimistic spinning
2) disable reader spinning

See the performance shown in patch 5.

This patch series adopts the second approach by dropping reader spinning for now as it simplifies the code. However, writers are still allowed to spin on a reader-owned rwsem for a limited time.

Waiman Long (5):
locking/rwsem: Pass the current atomic count to
rwsem_down_read_slowpath()
locking/rwsem: Prevent potential lock starvation locking/rwsem: Enable reader optimistic lock stealing locking/rwsem: Wake up all waiting readers if RWSEM_WAKE_READ_OWNED locking/rwsem: Remove reader optimistic spinning

 kernel/locking/lock_events_list.h |   6 +-
 kernel/locking/rwsem.c            | 293 ++++++++----------------------
 2 files changed, 82 insertions(+), 217 deletions(-)

2.18.1