Skip to content

Add a percpu subsection for cache hot data

Brian Gerst lkml 3 versions – Applied

Status from this message: The following commit has been merged into the x86/core branch of tip:

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest11 patches 12 repliesApplied
  2. v211 patches 9 repliesSuperseded
  3. v1 RFC11 patches 7 repliesSuperseded

Changes in v3 from the cover letter

- Fix typo of CACHE_HOT_DATA()
- Move hardirq_stack_inuse to irq_64.c
- Add __per_cpu_hot_pad to show the end of the actual data

Patches in v3 Applied

#Subject and filesSizeRepliesApplied
1/11percpu: Introduce percpu hot sectioninclude/asm-generic/vmlinux.lds.h include/linux/percpu-defs.h+24 -00 replies
2/11x86/percpu: Move pcpu_hot to percpu hot sectionarch/x86/include/asm/current.h arch/x86/kernel/cpu/common.c arch/x86/kernel/vmlinux.lds.S+15 -180 replies
3/11x86/preempt: Move preempt count to percpu hot sectionarch/x86/include/asm/current.h arch/x86/include/asm/preempt.h arch/x86/kernel/cpu/common.c +1 more+17 -140 replies
4/11x86/smp: Move cpu number to percpu hot sectionarch/x86/include/asm/current.h arch/x86/include/asm/smp.h arch/x86/kernel/setup_percpu.c +1 more+10 -70 replies
5/11x86/retbleed: Move call depth to percpu hot sectionarch/x86/include/asm/current.h arch/x86/include/asm/nospec-branch.h arch/x86/kernel/asm-offsets.c +2 more+15 -120 replies
6/11x86/softirq: Move softirq_pending to percpu hot sectionarch/x86/include/asm/current.h arch/x86/include/asm/hardirq.h arch/x86/kernel/irq.c+5 -30 replies
7/11x86/irq: Move irq stacks to percpu hot sectionarch/x86/include/asm/current.h arch/x86/include/asm/irq_stack.h arch/x86/include/asm/processor.h +6 more+30 -240 replies
8/11x86/percpu: Move top_of_stack to percpu hot sectionarch/x86/entry/entry_32.S arch/x86/entry/entry_64.S arch/x86/entry/entry_64_compat.S +9 more+22 -170 replies
9/11x86/percpu: Move current_task to percpu hot sectionarch/x86/include/asm/current.h arch/x86/include/asm/percpu.h arch/x86/kernel/asm-offsets.c +7 more+17 -250 replies
10/11x86/stackprotector: Move __stack_chk_guard to percpu hot sectionarch/x86/include/asm/stackprotector.h arch/x86/kernel/cpu/common.c+2 -20 replies
11/11x86/smp: Move this_cpu_off to percpu hot sectionarch/x86/include/asm/percpu.h arch/x86/kernel/setup_percpu.c+2 -20 replies

Reviews trailers per patch

None of the 11 patches has a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchUros Bizjak
SeriesA-b
1/11
2/11
3/11
4/11
5/11
6/11
7/11
8/11
9/11
10/11
11/11

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v3 thread

Cover letter [PATCH v3 00/11] Add a percpu subsection for cache hot data

Add a new percpu subsection for data that is frequently accessed and exclusive to each processor. This replaces the pcpu_hot struct on x86, and is available to all architectures and the core kernel.

ffffffff834f5000 D __per_cpu_hot_start
ffffffff834f5000 D hardirq_stack_ptr
ffffffff834f5008 D __ref_stack_chk_guard
ffffffff834f5008 D __stack_chk_guard
ffffffff834f5010 D const_cpu_current_top_of_stack
ffffffff834f5010 D cpu_current_top_of_stack
ffffffff834f5018 D const_current_task
ffffffff834f5018 D current_task
ffffffff834f5020 D __x86_call_depth
ffffffff834f5028 D this_cpu_off
ffffffff834f5030 D __preempt_count
ffffffff834f5034 D cpu_number
ffffffff834f5038 D __softirq_pending
ffffffff834f503a D hardirq_stack_inuse
ffffffff834f503b D __per_cpu_hot_pad
ffffffff834f5040 D __per_cpu_hot_end

This applies to the tip/x86/asm branch.

Changes in v3:
- Fix typo of CACHE_HOT_DATA()
- Move hardirq_stack_inuse to irq_64.c
- Add __per_cpu_hot_pad to show the end of the actual data

Brian Gerst (11):
percpu: Introduce percpu hot section
x86/percpu: Move pcpu_hot to percpu hot section x86/preempt: Move preempt count to percpu hot section x86/smp: Move cpu number to percpu hot section x86/retbleed: Move call depth to percpu hot section x86/softirq: Move softirq_pending to percpu hot section x86/irq: Move irq stacks to percpu hot section x86/percpu: Move top_of_stack to percpu hot section x86/percpu: Move current_task to percpu hot section x86/stackprotector: Move __stack_chk_guard to percpu hot section x86/smp: Move this_cpu_off to percpu hot section

 arch/x86/entry/entry_32.S             |  4 +--
 arch/x86/entry/entry_64.S             |  6 ++---
 arch/x86/entry/entry_64_compat.S      |  4 +--
 arch/x86/include/asm/current.h        | 36 +++++----------------------
 arch/x86/include/asm/hardirq.h        |  4 +--
 arch/x86/include/asm/irq_stack.h      | 12 ++++-----
 arch/x86/include/asm/nospec-branch.h  | 11 ++++----
 arch/x86/include/asm/percpu.h         |  4 +--
 arch/x86/include/asm/preempt.h        | 25 ++++++++++---------
 arch/x86/include/asm/processor.h      | 16 ++++++++++--
 arch/x86/include/asm/smp.h            |  7 +++---
 arch/x86/include/asm/stackprotector.h |  2 +-
 arch/x86/kernel/asm-offsets.c         |  5 ----
 arch/x86/kernel/cpu/common.c          | 25 +++++++++++++------
 arch/x86/kernel/dumpstack_32.c        |  4 +--
 arch/x86/kernel/dumpstack_64.c        |  2 +-
 arch/x86/kernel/head_64.S             |  4 +--
 arch/x86/kernel/irq.c                 |  5 ++++
 arch/x86/kernel/irq_32.c              | 12 +++++----
 arch/x86/kernel/irq_64.c              |  7 +++---
 arch/x86/kernel/process_32.c          |  6 ++---
 arch/x86/kernel/process_64.c          |  6 ++---
 arch/x86/kernel/setup_percpu.c        |  7 ++++--
 arch/x86/kernel/smpboot.c             |  4 +--
 arch/x86/kernel/vmlinux.lds.S         |  6 ++++-
 arch/x86/lib/retpoline.S              |  2 +-
 include/asm-generic/vmlinux.lds.h     | 11 ++++++++
 include/linux/percpu-defs.h           | 13 ++++++++++
 include/linux/preempt.h               |  1 +
 kernel/bpf/verifier.c                 |  4 +--
 scripts/gdb/linux/cpus.py             |  2 +-
 31 files changed, 146 insertions(+), 111 deletions(-)

base-commit: 693c8502970a533363e9ece482c80bb6db0c12a5

Signature · 1 line
2.48.1