powerpc: Detection and scheduler optimization for POWER9 bigcore

From: "Gautham R. Shenoy" <[email protected]>

Hi,

This is the tenth iteration of the patchset to add support for big-core on POWER9. This patch also optimizes the task placement on such big-core systems.

The previous versions can be found here:
v9: https://lkml.org/lkml/2018/10/1/608
v8: https://lkml.org/lkml/2018/9/20/899
v7: https://lkml.org/lkml/2018/8/20/52
v6: https://lkml.org/lkml/2018/8/9/119
v5: https://lkml.org/lkml/2018/8/6/587
v4: https://lkml.org/lkml/2018/7/24/79
v3: https://lkml.org/lkml/2018/7/6/255
v2: https://lkml.org/lkml/2018/7/3/401
v1: https://lkml.org/lkml/2018/5/11/245

Changes :
v9 --> v10:
- Rebased it on v4.19-rc7
- Added a patch to report the correct shared_cpu_map for L1-caches on big-core systems.

Description:
~~~~~~~~~~~~~~~~~~~~

IBM POWER9 SMT8 cores consists of two groups of small-cores where each group has its own L1 cache, translation cache and instruction-data flow. This can be discovered via the "ibm,thread-groups" CPU property in the device tree. Furthermore, on POWER9 the thread-ids of such a big-core is obtained by interleaving the thread-ids of the two small-cores.

Eg: In an SMT8 core with thread ids {0,1,2,3,4,5,6,7}, the thread-ids of the threads in the two small-cores respectively will be {0,2,4,6} and {1,3,5,7} respectively.

-------------------------
| L1 Cache |
----------------------------------
|L2| | | | |
| | 0 | 2 | 4 | 6 |Small Core0
|C | | | | |

Big    |a --------------------------
Core   |c |     |     |     |      |
       |h |  1  |  3  |  5  |  7   | Small Core1
       |e |     |     |     |      |
       -----------------------------
	  |  	    L1 Cache       |
	  --------------------------

On such a big-core system, when multiple tasks are scheduled to run on the big-core, we get the best performance when the tasks are spread across the pair of small-cores.

Eg: Suppose there 4 tasks {p1, p2, p3, p4} are run on a big core, then

An Example of Optimal Task placement:

	   --------------------------
           |     |     |     |      |
           |  0  |  2  |  4  |  6   |   Small Core0
           | (p1)| (p2)|     |      |
Big Core   --------------------------
           |     |     |     |      |
           |  1  |  3  |  5  |  7   |   Small Core1
           |     | (p3)|     | (p4) |
           --------------------------

An example of Suboptimal Task placement:

	   --------------------------
           |     |     |     |      |
           |  0  |  2  |  4  |  6   |   Small Core0
           | (p1)| (p2)|     |  (p4)|
Big Core   --------------------------
           |     |     |     |      |
           |  1  |  3  |  5  |  7   |   Small Core1
           |     | (p3)|     |      |
           --------------------------

Currently on the big-core systems, the sched domain hierarchy is:

SMT : group of CPUs in the SMT8 core.
DIE : groups of CPUs on the same die.
NUMA : all the CPUs in the system.

Thus the scheduler doesn't distinguish between CPUs in the core that share the L1-cache vs the ones that don't resulting in a run-to-run variance when multithreaded applications are run on an SMT8 core.

In this patch-set, we address this by defining the sched-domain on the big-core systems to be:

SMT : group of CPUs sharing the L1 cache
CACHE : group of CPUs in the SMT8 core.
DIE : groups of CPUs on the same die.
NUMA : all the CPUs in the system.

With this, the Linux Kernel load-balancer will ensure that the tasks are spread across all the component small cores in the system, thereby yielding optimum performance.

Furthermore, this solution works correctly across all SMT modes (8,4,2), as the interleaved thread-ids ensures that when we go to lower SMT modes (4,2) the threads are offlined in a descending order, thereby leaving equal number of threads from the component small cores online as illustrated below.

This patchset contains three patches which on detecting the presence of big-cores, defines the SMT level sched domain to correspond to the threads of the small cores.

Patch 1: adds support to detect the presence of
big-cores and parses the output of "ibm,thread-groups" device-tree which using which it updates a per-cpu mask named cpu_smallcore_mask

Patch 2: Defines the SMT level sched domain to correspond to the threads of the small cores.

Patch 3: Added a patch to report the correct shared_cpu_map for L1-caches on big-core systems.

Without patch 3:

       /sys/devices/system/cpu/cpu0/cache/index0/shared_cpu_map : 000000ff
       /sys/devices/system/cpu/cpu0/cache/index1/shared_cpu_map : 000000ff
       /sys/devices/system/cpu/cpu1/cache/index0/shared_cpu_map : 000000ff
       /sys/devices/system/cpu/cpu1/cache/index1/shared_cpu_map : 000000ff

    With patch 3:
       /sys/devices/system/cpu/cpu0/cache/index0/shared_cpu_map : 00000055
       /sys/devices/system/cpu/cpu0/cache/index1/shared_cpu_map : 00000055
       /sys/devices/system/cpu/cpu1/cache/index0/shared_cpu_map : 000000aa
       /sys/devices/system/cpu/cpu1/cache/index1/shared_cpu_map : 000000aa

Results:
~~~~~~~~~~~~~~~~~
1) 2 thread ebizzy
~~~~~~~~~~~~~~~~~~~~~~
Experimental results for ebizzy with 2 threads, bound to a single big-core show a marked improvement with this patchset over the 4.19.0-rc7 vanilla kernel.

The result of 100 such runs for 4.19.0-rc7 kernel and the 4.19.0-rc7 + big-core-patches are as follows

4.19.0-rc7 vanilla
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
records/s : # samples : Histogram
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[0       - 1000000]  :      0      : #
[1000000 - 2000000]  :      2      : #
[2000000 - 3000000]  :      8      : ##
[3000000 - 4000000]  :      19     : ####
[4000000 - 5000000]  :      7      : ##
[5000000 - 6000000]  :      2      : #
[6000000 - 7000000]  :      62     : #############

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

4.19.0-rc7 + big-core-patches
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
records/s : # samples : Histogram
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[0       - 1000000]  :      0      : #
[1000000 - 2000000]  :      0      : #
[2000000 - 3000000]  :      4      : #
[3000000 - 4000000]  :      8      : ##
[4000000 - 5000000]  :      0      : #
[5000000 - 6000000]  :      1      : #
[6000000 - 7000000]  :      87     : ##################

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

2) Hackbench (perf bench sched pipe)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

500 iterations of the hackbench run both on 4.19.0-rc7 vanilla kernel and v4.19.0-rc7 + big-core-patches. There isn't a significant difference between the two.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

			4.19.0-rc7 vanilla

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
N Min Max Median Avg Stddev
500 4.658s 6.293s 6.076s 5.846528s 0.45096266
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

			4.19.0-rc7 + big-core-patches

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
N Min Max Median Avg Stddev
500 4.543s 6.3s 5.75s 5.682208s 0.50767805
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Gautham R. Shenoy (3):
powerpc: Detect the presence of big-cores via "ibm,thread-groups"
powerpc: Use cpu_smallcore_sibling_mask at SMT level on bigcores powerpc/cacheinfo: Report the correct shared_cpu_map on big-cores

 arch/powerpc/include/asm/cputhreads.h |   2 +
 arch/powerpc/include/asm/smp.h        |  11 ++
 arch/powerpc/kernel/cacheinfo.c       |  37 +++++-
 arch/powerpc/kernel/smp.c             | 241 +++++++++++++++++++++++++++++++++-
 4 files changed, 288 insertions(+), 3 deletions(-)

1.9.4