Skip to content

percpu: Consistent per cpu operations V1

Christoph Lameter lkml 1 version Applied

Status from this message: Applied to percpu/for-3.14.

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest41 patches 41 repliesApplied

Patches in v1 Applied

#Subject and filesSizeRepliesApplied
1/41mm: Replace __get_cpu_var uses with this_cpu_ptrlib/radix-tree.c mm/memcontrol.c mm/memory-failure.c +5 more+13 -131 reply
2/41tracing: Replace __get_cpu_var uses with this_cpu_ptrinclude/linux/kprobes.h kernel/trace/ftrace.c kernel/trace/trace.c+4 -42 replies
3/41percpu: Replace __get_cpu_var with this_cpu_ptrinclude/linux/percpu.h+1 -11 reply
4/41kernel misc: Replace __get_cpu_var useskernel/printk/printk.c kernel/smp.c+7 -71 reply
5/41drivers/char/random: Replace __get_cpu_var usesdrivers/char/random.c+1 -11 reply
6/41drivers/cpuidle: Replace __get_cpu_var uses for address calculationdrivers/cpuidle/governors/ladder.c drivers/cpuidle/governors/menu.c+5 -51 reply
7/41drivers/oprofile: Replace __get_cpu_var uses for address calculationdrivers/oprofile/cpu_buffer.c drivers/oprofile/timer_int.c+6 -60 replies
8/41drivers/leds: Replace __get_cpu_var use through this_cpu_ptrdrivers/leds/trigger/ledtrig-cpu.c+1 -11 reply
9/41drivers/clocksource: Replace __get_cpu_var used for address calculationdrivers/clocksource/metag_generic.c+1 -11 reply
10/41staging/zsmalloc: Replace instances of using __get_cpu_var for address calculationdrivers/staging/zsmalloc/zsmalloc-main.c+2 -21 reply
11/41parisc: Replace __get_cpu_var uses for address calculationarch/parisc/lib/memcpy.c arch/parisc/mm/fault.c+2 -26 replies
12/41metag: Replace __get_cpu_var uses for address calculationarch/metag/kernel/perf/perf_event.c+7 -70 replies
13/41percpu: Add raw_cpu_opsarch/x86/include/asm/percpu.h include/asm-generic/percpu.h include/linux/percpu.h+207 -14513 replies
14/41x86: Rename __this_cpu_xxx_# operations to raw_cpu_xxx_#arch/x86/include/asm/percpu.h arch/x86/include/asm/preempt.h+57 -570 replies
15/41mm: Use raw_cpu ops for determining current NUMA nodeinclude/linux/topology.h+2 -20 replies
16/41modules: Use raw_cpu_write for initialization of per cpu refcount.kernel/module.c+1 -10 replies
17/41net:i Replace __this_cpu_inc in route.c with raw_cpu_incnet/ipv4/route.c+1 -10 replies
18/41percpu: Add preemption checks to __this_cpu opsinclude/linux/percpu.h lib/smp_processor_id.c+46 -140 replies
19/41time: Replace __get_cpu_var usesdrivers/clocksource/dummy_timer.c kernel/hrtimer.c kernel/irq_work.c +6 more+34 -340 replies
20/41scheduler: Replace __get_cpu_var with this_cpu_ptrinclude/linux/kernel_stat.h kernel/events/callchain.c kernel/events/core.c +5 more+22 -220 replies
21/41block: Replace __this_cpu_ptr with raw_cpu_ptrfs/buffer.c fs/ext4/mballoc.c+2 -23 replies
22/41rcu: Replace __this_cpu_ptr uses with raw_cpu_ptrkernel/rcu/tree.c kernel/rcu/tree_plugin.h+5 -52 replies
23/41watchdog: Replace __raw_get_cpu_var useskernel/watchdog.c+5 -50 replies
24/41net: Replace get_cpu_var through this_cpu_ptrinclude/net/netfilter/nf_conntrack.h include/net/snmp.h net/core/dev.c +8 more+20 -200 replies
25/41drivers/net/ethernet/tile: Replace __get_cpu_var uses for address calculationdrivers/net/ethernet/tile/tilegx.c drivers/net/ethernet/tile/tilepro.c+13 -131 reply
26/41md: Replace __this_cpu_ptr with raw_cpu_ptrdrivers/md/dm-stats.c+1 -10 replies
27/41irqchips: Replace __this_cpu_ptr usesdrivers/irqchip/irq-gic.c kernel/irq/chip.c+6 -60 replies
28/41x86: Replace __get_cpu_var usesarch/x86/include/asm/debugreg.h arch/x86/include/asm/perf_event_p4.h arch/x86/include/asm/uv/uv_hub.h +29 more+142 -1420 replies
29/41arm: Replace __this_cpu_ptr with raw_cpu_ptrarch/arm/kernel/smp_twd.c arch/arm/mach-msm/timer.c+7 -70 replies
30/41mips: Replace __get_cpu_var usesarch/mips/cavium-octeon/octeon-irq.c arch/mips/include/asm/fpu_emulator.h arch/mips/kernel/kprobes.c +2 more+27 -270 replies
31/41s390: Replace __get_cpu_var usesarch/s390/include/asm/cputime.h arch/s390/include/asm/irq.h arch/s390/include/asm/percpu.h +8 more+36 -360 replies
32/41ia64: Replace __get_cpu_var usesarch/alpha/kernel/perf_event.c arch/ia64/include/asm/hw_irq.h arch/ia64/include/asm/sn/arch.h +11 more+40 -400 replies
33/41powerpc: Replace __get_cpu_var usesarch/powerpc/include/asm/cputime.h arch/powerpc/include/asm/hardirq.h arch/powerpc/include/asm/tlbflush.h +28 more+91 -910 replies
34/41sparc: Replace __get_cpu_var usesarch/sparc/include/asm/cpudata_32.h arch/sparc/include/asm/cpudata_64.h arch/sparc/kernel/kprobes.c +7 more+35 -350 replies
35/41tile: Replace __get_cpu_var usesarch/tile/include/asm/irqflags.h arch/tile/include/asm/mmu_context.h arch/tile/kernel/irq.c +9 more+31 -302 replies
36/41blackfin: Replace __get_cpu_var usesarch/blackfin/include/asm/ipipe.h arch/blackfin/kernel/perf_event.c arch/blackfin/mach-common/ints-priority.c +1 more+11 -110 replies
37/41avr32: Replace __get_cpu_var with __this_cpu_writearch/avr32/kernel/kprobes.c+1 -10 replies
38/41alpha: Replace __get_cpu_vararch/alpha/kernel/perf_event.c arch/alpha/kernel/time.c+10 -100 replies
39/41sh: Replace __get_cpu_var usesarch/sh/kernel/hw_breakpoint.c arch/sh/kernel/kprobes.c arch/sh/kernel/localtimer.c +2 more+23 -230 replies
40/41Remove __get_cpu_var and __raw_get_cpu_var macros [only in 3.15]include/asm-generic/percpu.h+0 -50 replies
41/41percpu: Remove __this_cpu_ptrinclude/linux/percpu.h+0 -30 replies

Reviews trailers per patch

11 of 41 patches have a Reviewed-by or Acked-by.

PatchHeiko CarstensJames HoganRafael J. WysockiHans-Christian EgtvedtDavid S. MillerBryan WuMasami HiramatsuChris MetcalfIngo MolnarSteven Rostedt
1/41
2/41A-b A-b
3/41
4/41
5/41
6/41A-b
7/41
8/41A-b
9/41A-b
10/41
11/41
12/41A-b
13/41A-b
14/41
15/41
16/41
17/41
18/41
19/41
20/41
21/41
22/41
23/41
24/41
25/41A-b
26/41
27/41
28/41
29/41
30/41
31/41A-b
32/41
33/41
34/41A-b
35/41A-b
36/41
37/41A-b
38/41
39/41
40/41
41/41

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 10 reviewers.

Read the v1 thread

Cover letter [PATCH 00/41] percpu: Consistent per cpu operations V1

The kernel has never been audited to ensure that this_cpu operations are consistently used throughout the kernel. The code generated in many places can be improved through the use of this_cpu operations (which uses a segment register for relocation of per cpu offsets instead of performing address calculations).

The patchset also addresses various consistency issues in general with the per cpu macros.

A. The semantics of __this_cpu_ptr() differs from this_cpu_ptr only
because checks are skipped. This is typically shown through a raw_ prefix. So this patchset changes the places where __this_cpu_ptr() is used to raw_cpu_ptr().

B. There has been the long term wish by some that __this_cpu operations
would check for preemption. However, there are cases where preemption checks need to be skipped. This patchset adds raw_cpu operations that do not check for preemption and then adds preemption checks to the __this_cpu operations.

C. The use of __get_cpu_var is always a reference to a percpu variable
that can also be handled via a this_cpu operation. This patchset replaces all uses of __get_cpu_var with this_cpu operations.

D. We can then use this_cpu RMW operations in various places replacing
sequences of instructions by a single one.

E. The use of this_cpu operations throughout will allow other arches than
x86 to implement optimized references and RMV operations to work with per cpu local data.

F. The use of this_cpu operations opens up the possibility to
further optimize code that relies on synchronization through per cpu data.

The patchset works in a couple of stages:

I. Patches 1-12 are patches that simply replace uses of __get_cpu_var
with this_cpu_ptr. They do not depend on any changes to the percpu code.

II. Patch 13 adds the additional raw_cpu operations and raw_cpu_ptr().
Patch 14 converts the existing __this_cpu_xx_# primitive in the x86 code to raw_cpu_xx_#.

III. Patch 15-17 use the raw_cpu operations in places that would give
us false positives once they are enabled.

IV. Patch 18 adds preemption checks to __this_cpu operations to allow
checking if preemption is properly disabled when these functions are used.

V. Patches 19-39 are conversion patches that use this_cpu operations
in various kernel subsystems/drivers or arch code.

VI. Patches 41/42 remove no longer used functions (__this_cpu_ptr
and __get_cpu_var). These should only be applied after all the conversion patches have made it and after we have done additonal passes through the kernel to ensure that none of the uses of these functions remain.