Skip to contentSkip to conversation

locking changes for v5.3

GIT PULL Merged Ingo Molnar 2 participants 2 messages Jul 8, 2019 – Jul 9, 2019

lore
Share & tools
lore
#0 lore raw

Linus,

Please pull the latest locking-core-for-linus git tree from:

git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git locking-core-for-linus

# HEAD: 9156e545765e467e6268c4814cfa609ebb16237e locking/lockdep: increase size of counters for lockdep statistics

The main changes in this cycle are:

- rwsem scalability improvements, phase #2, by Waiman Long, which are rather impressive:

"On a 2-socket 40-core 80-thread Skylake system with 40 reader and
writer locking threads, the min/mean/max locking operations done in a 5-second testing window before the patchset were:

40 readers, Iterations Min/Mean/Max = 1,807/1,808/1,810 40 writers, Iterations Min/Mean/Max = 1,807/50,344/151,255

After the patchset, they became:

40 readers, Iterations Min/Mean/Max = 30,057/31,359/32,741 40 writers, Iterations Min/Mean/Max = 94,466/95,845/97,098"

There's a lot of changes to the locking implementation that makes it similar to qrwlock, including owner handoff for more fair locking.

Another microbenchmark shows how across the spectrum the improvements are:

    "With a locking microbenchmark running on 5.1 based kernel, the total 
     locking rates (in kops/s) on a 2-socket Skylake system with equal 
     numbers of readers and writers (mixed) before and after this 
     patchset were:

     # of Threads   Before Patch      After Patch
     ------------   ------------      -----------
          2            2,618             4,193
          4            1,202             3,726
          8              802             3,622
         16              729             3,359
         32              319             2,826
         64              102             2,744"

The changes are extensive and the patch-set has been through several iterations addressing various locking workloads. There might be more regressions, but unless they are pathological I believe we want to use this new implementation as the baseline going forward.

- jump-label optimizations by Daniel Bristot de Oliveira: the primary motivation was to remove IPI disturbance of isolated RT-workload CPUs, which resulted in the implementation of batched jump-label updates. Beyond the improvement of the real-time characteristics kernel, in one test this patchset improved static key update overhead from 57 msecs to just 1.4 msecs - which is a nice speedup as well.

- atomic64_t cross-arch type cleanups by Mark Rutland: over the last ~10 years of atomic64_t existence the various types used by the APIs only had to be self-consistent within each architecture - which means they became wildly inconsistent across architectures. Mark puts and end to this by reworking all the atomic64 implementations to use 's64' as the base type for atomic64_t, and to ensure that this type is consistently used for parameters and return values in the API, avoiding further problems in this area.

- A large set of small improvements to lockdep by Yuyang Du: type cleanups, output cleanups, function return type and othr cleanups all around the place.

- A set of percpu ops cleanups and fixes by Peter Zijlstra.

- Misc other changes - please see the Git log for more details.

Thanks,

Ingo

------------------>
Anders Roxell (1):
locking/lockdep: Remove the unused print_lock_trace() function

Arnd Bergmann (1):

      locking/lockdep: Move mark_lock() inside CONFIG_TRACE_IRQFLAGS && CONFIG_PROVE_LOCKING

Daniel Bristot de Oliveira (6):
jump_label: Add a jump_label_can_update() helper x86/jump_label: Add a __jump_label_set_jump_code() helper
jump_label: Sort entries of the same key by the code x86/alternative: Batch of patch operations
jump_label: Batch updates if arch supports it x86/jump_label: Batch jump label updates

Imre Deak (2):
locking/lockdep: Fix OOO unlock when hlocks need merging locking/lockdep: Fix merging of hlocks with non-zero references

Kobe Wu (2):
locking/lockdep: Remove unnecessary DEBUG_LOCKS_WARN_ON() locking/lockdep: increase size of counters for lockdep statistics

Mark Rutland (18):
locking/atomic, crypto/nx: Prepare for atomic64_read() conversion locking/atomic, s390/pci: Prepare for atomic64_read() conversion locking/atomic: Use s64 for atomic64
locking/atomic, alpha: Use s64 for atomic64 locking/atomic, arc: Use s64 for atomic64
locking/atomic, arm: Use s64 for atomic64
locking/atomic, arm64: Use s64 for atomic64 locking/atomic, ia64: Use s64 for atomic64 locking/atomic, mips: Use s64 for atomic64 locking/atomic, powerpc: Use s64 for atomic64 locking/atomic, riscv: Fix atomic64_sub_if_positive() offset argument locking/atomic, riscv: Use s64 for atomic64 locking/atomic, s390: Use s64 for atomic64 locking/atomic, sparc: Use s64 for atomic64 locking/atomic, x86: Use s64 for atomic64
locking/atomic: Use s64 for atomic64_t on 64-bit locking/atomic, crypto/nx: Remove redundant casts locking/atomic, s390/pci: Remove redundant casts

Michael Forney (1):
locking/atomics: Use sed(1) instead of non-standard head(1) option

Nikolay Borisov (1):

      locking/lockdep: Rename lockdep_assert_held_exclusive() -> lockdep_assert_held_write()

Peter Zijlstra (8):
locking/lock_events: Use raw_cpu_{add,inc}() for stats Documentation/atomic_t.txt: Clarify pure non-rmw usage x86/atomic: Fix smp_mb__{before,after}_atomic() x86/percpu: Differentiate this_cpu_{}() and __this_cpu_{}() x86/percpu: Relax smp_processor_id()
x86/percpu, x86/irq: Relax {set,get}_irq_regs() x86/percpu, sched/fair: Avoid local_clock() x86/percpu: Optimize raw_cpu_xchg()

Sebastian Andrzej Siewior (1):
locking/lockdep: Don't complain about incorrect name for no validate class

Waiman Long (17):
futex: Consolidate duplicated timer setup code locking/rwsem: Make owner available even if !CONFIG_RWSEM_SPIN_ON_OWNER locking/rwsem: Remove rwsem_wake() wakeup optimization locking/rwsem: Implement a new locking scheme locking/rwsem: Merge rwsem.h and rwsem-xadd.c into rwsem.c locking/rwsem: Code cleanup after files merging locking/rwsem: Make rwsem_spin_on_owner() return owner state locking/rwsem: Implement lock handoff to prevent lock starvation locking/rwsem: Always release wait_lock before waking up tasks locking/rwsem: More optimal RT task handling of null owner locking/rwsem: Wake up almost all readers in wait queue locking/rwsem: Clarify usage of owner's nonspinaable bit locking/rwsem: Enable readers spinning on writer locking/rwsem: Make rwsem->owner an atomic_long_t locking/rwsem: Enable time-based spinning on reader-owned rwsem locking/rwsem: Adaptive disabling of reader optimistic spinning locking/rwsem: Guard against making count negative

YueHaibing (1):
x86/jump_label: Make tp_vec_nr static

Yuyang Du (23):
locking/lockdep: Change all print_*() return type to void locking/lockdep: Add description and explanation in lockdep design doc locking/lockdep: Adjust lock usage bit character checks locking/lockdep: Remove useless conditional macro
locking/lockdep: Print the right depth for chain key collision locking/lockdep: Update obsolete struct field description locking/lockdep: Use lockdep_init_task for task initiation consistently locking/lockdep: Define INITIAL_CHAIN_KEY for chain keys to start with locking/lockdep: Change the range of class_idx in held_lock struct locking/lockdep: Remove unused argument in validate_chain() and check_deadlock() locking/lockdep: Update comment
locking/lockdep: Change type of the element field in circular_queue locking/lockdep: Change the return type of __cq_dequeue() locking/lockdep: Avoid constant checks in __bfs by using offset reference locking/lockdep: Update comments on dependency search locking/lockdep: Add explanation to lock usage rules in lockdep design doc locking/lockdep: Remove redundant argument in check_deadlock locking/lockdep: Remove unused argument in __lock_release locking/lockdep: Refactorize check_noncircular and check_redundant locking/lockdep: Check redundant dependency only when CONFIG_LOCKDEP_SMALL locking/lockdep: Consolidate lock usage bit initialization locking/lockdep: Adjust new bit cases in mark_lock locking/lockdep: Remove !dir in lock irq usage check

 Documentation/atomic_t.txt               |    9 +-
 Documentation/locking/lockdep-design.txt |  112 ++-
 arch/alpha/include/asm/atomic.h          |   20 +-
 arch/arc/include/asm/atomic.h            |   41 +-
 arch/arm/include/asm/atomic.h            |   50 +-
 arch/arm64/include/asm/atomic_ll_sc.h    |   20 +-
 arch/arm64/include/asm/atomic_lse.h      |   34 +-
 arch/ia64/include/asm/atomic.h           |   20 +-
 arch/mips/include/asm/atomic.h           |   22 +-
 arch/powerpc/include/asm/atomic.h        |   44 +-
 arch/riscv/include/asm/atomic.h          |   44 +-
 arch/s390/include/asm/atomic.h           |   38 +-
 arch/s390/pci/pci_debug.c                |    2 +-
 arch/sparc/include/asm/atomic_64.h       |    8 +-
 arch/x86/events/core.c                   |    2 +-
 arch/x86/include/asm/atomic.h            |    8 +-
 arch/x86/include/asm/atomic64_32.h       |   66 +-
 arch/x86/include/asm/atomic64_64.h       |   46 +-
 arch/x86/include/asm/barrier.h           |    4 +-
 arch/x86/include/asm/irq_regs.h          |    4 +-
 arch/x86/include/asm/jump_label.h        |    2 +
 arch/x86/include/asm/percpu.h            |  236 ++---
 arch/x86/include/asm/smp.h               |    3 +-
 arch/x86/include/asm/text-patching.h     |   15 +
 arch/x86/kernel/alternative.c            |  154 +++-
 arch/x86/kernel/jump_label.c             |  121 ++-
 drivers/crypto/nx/nx-842-pseries.c       |    6 +-
 drivers/infiniband/core/device.c         |    2 +-
 drivers/tty/tty_ldisc.c                  |    8 +-
 fs/dax.c                                 |    2 +-
 include/asm-generic/atomic64.h           |   20 +-
 include/linux/jump_label.h               |    3 +
 include/linux/lockdep.h                  |   36 +-
 include/linux/percpu-rwsem.h             |    4 +-
 include/linux/rwsem.h                    |   16 +-
 include/linux/sched/wake_q.h             |    5 +
 include/linux/smp.h                      |   45 +-
 include/linux/types.h                    |    2 +-
 init/init_task.c                         |    2 +
 kernel/fork.c                            |    3 -
 kernel/futex.c                           |   69 +-
 kernel/jump_label.c                      |   64 +-
 kernel/locking/Makefile                  |    2 +-
 kernel/locking/lock_events.h             |   45 +-
 kernel/locking/lock_events_list.h        |   12 +-
 kernel/locking/lockdep.c                 |  742 ++++++++-------
 kernel/locking/lockdep_internals.h       |   36 +-
 kernel/locking/rwsem-xadd.c              |  745 ---------------
 kernel/locking/rwsem.c                   | 1453 +++++++++++++++++++++++++++++-
 kernel/locking/rwsem.h                   |  306 +------
 kernel/sched/fair.c                      |    5 +-
 lib/Kconfig.debug                        |    8 +-
 lib/atomic64.c                           |   32 +-
 scripts/atomic/check-atomics.sh          |    2 +-
 security/apparmor/label.c                |    8 +-
 55 files changed, 2788 insertions(+), 2020 deletions(-)
 delete mode 100644 kernel/locking/rwsem-xadd.c
merge bot Merged
#1 lore raw
The pull request you sent on Mon, 8 Jul 2019 11:35:16 +0200:

The pull request you sent on Mon, 8 Jul 2019 11:35:16 +0200:

Quoted 1 line

git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git locking-core-for-linus

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/e1928328699a582a540b105e5f4c160832a7fdcb

Thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.wiki.kernel.org/userdoc/prtracker