locking, sched: The PREEMPT-RT locking infrastructure

Folks,

the following series is an update to V4 which can be found here:

https://lore.kernel.org/r/[email protected]

It contains the bulk of the PREEMPT-RT locking infrastructure. In PREEMPT-RT enabled kernels the following locking primitives are substituted by RT-Mutex based variants:

mutex, ww_mutex, rw_semaphore, spinlock, rwlock

semaphores are not substituted because they do not provide strict owner semantics.

Of course raw_spinlocks are not touched either as they protect low level operations in the scheduler, timers and hardware access.

The most interesting parts of the series which need a lot of eyeballs are:

- the scheduler bits which provide the infrastructure for spinlock and rwlock substitution to ensure that the task state is preserved when blocking on such a lock and a regular wakeup is handled correctly and not lost

- the rtmutex core implementation to handle lock contention on spinlocks and rwlocks correctly vs. the task state

- the rw_semaphore/rwlock substitutions which utilize the same implementation vs. the reader/writer handling

- The new rtmutex based ww_mutex implementation.

- the PI futex related bits to handle the interaction between blocking on the underlying rtmutex and contention on the hash bucket lock which is converted to a 'sleeping spinlock'.

The rest surely needs a thorough review as well, but those parts are pretty straight forward: quite some code restructuring and the actual wrapper functions to replace the existing !RT implementations.

The series survived internal testing in RT kernels and is part of the upcoming v5.14-rc6-rt9 release.

For !RT kernels there is no functional change.

The series is also available from git:

git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git rtmutex

Changes vs. V4

- A few lockdep wait_type fixes in static and runtime initializers. Sebastian noticed while working on getting the lockdep selftests reenabled on RT.

- Missing might_sleep() invocations in RT spin/rwlocks (Sebastian)

- Add explicit owner init for local locks when lockdep is enabled instead of relying on zero initialized memory.

- Add the RT variants for local locks, which is the last lock type getting special treatment on RT.

The lockdep selftest changes are not yet ready and will be posted in a separate submission.

Thanks,

tglx

---
 b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c |    4 
 b/include/linux/debug_locks.h                       |    3 
 b/include/linux/local_lock_internal.h               |   90 +
 b/include/linux/mutex.h                             |   92 +
 b/include/linux/preempt.h                           |    4 
 b/include/linux/rbtree.h                            |   30 
 b/include/linux/rbtree_types.h                      |   34 
 b/include/linux/rtmutex.h                           |   63 -
 b/include/linux/rwbase_rt.h                         |   38 
 b/include/linux/rwlock_rt.h                         |  140 ++
 b/include/linux/rwlock_types.h                      |   53 
 b/include/linux/rwsem.h                             |   78 +
 b/include/linux/sched.h                             |  119 +-
 b/include/linux/sched/wake_q.h                      |    8 
 b/include/linux/spinlock.h                          |   15 
 b/include/linux/spinlock_api_smp.h                  |    3 
 b/include/linux/spinlock_rt.h                       |  159 ++
 b/include/linux/spinlock_types.h                    |   89 -
 b/include/linux/spinlock_types_raw.h                |   73 +
 b/include/linux/ww_mutex.h                          |   50 
 b/kernel/Kconfig.locks                              |    2 
 b/kernel/futex.c                                    |  556 ++++++---
 b/kernel/locking/Makefile                           |    3 
 b/kernel/locking/mutex-debug.c                      |    5 
 b/kernel/locking/mutex.c                            |  431 -------
 b/kernel/locking/mutex.h                            |   48 
 b/kernel/locking/rtmutex.c                          | 1134 +++++++++-----------
 b/kernel/locking/rtmutex_api.c                      |  590 ++++++++++
 b/kernel/locking/rtmutex_common.h                   |  122 +-
 b/kernel/locking/rwbase_rt.c                        |  263 ++++
 b/kernel/locking/rwsem.c                            |  109 +
 b/kernel/locking/spinlock.c                         |    7 
 b/kernel/locking/spinlock_debug.c                   |    5 
 b/kernel/locking/spinlock_rt.c                      |  263 ++++
 b/kernel/locking/ww_mutex.h                         |  569 ++++++++++
 b/kernel/locking/ww_rt_mutex.c                      |   76 +
 b/kernel/rcu/tree_plugin.h                          |    6 
 b/kernel/sched/core.c                               |  109 +
 b/lib/Kconfig.debug                                 |   11 
 b/lib/test_lockup.c                                 |    8 
 kernel/locking/mutex-debug.h                        |   29 
 41 files changed, 3940 insertions(+), 1551 deletions(-)