Skip to content

Provide vcpu dispatch statistics

Naveen N. Rao linuxppc-dev 3 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest9 patches 1 replyIn review
  2. v29 patches 0 repliesSuperseded
  3. v18 patches 2 repliesSuperseded

Changes in v3 from the cover letter

- Moved most prototypes to asm/lppaca.h instead of asm/plpar_wrappers.h
- Patch 4/9 changed to use rwlocks for guarding DTL buffer usage across debugfs and the new vcpudispatch_stats.
- Patch 6/9 has been updated to address breakage of the vphn selftest under tools/testing/selftests/powerpc/vphn.
- Patch 7/9 has changes to accommodate use of rwlocks, as well as some simplification for locking while enabling/disabling statistics.

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/9powerpc/pseries: Use macros for referring to the DTL enable maskarch/powerpc/include/asm/lppaca.h arch/powerpc/platforms/pseries/dtl.c arch/powerpc/platforms/pseries/lpar.c +1 more+14 -91 reply
2/9powerpc/pseries: Do not save the previous DTL mask valuearch/powerpc/platforms/pseries/dtl.c+1 -30 replies
3/9powerpc/pseries: Factor out DTL buffer allocation and registration routinesarch/powerpc/include/asm/lppaca.h arch/powerpc/platforms/pseries/lpar.c arch/powerpc/platforms/pseries/setup.c+53 -500 replies
4/9powerpc/pseries: Introduce rwlock to gatekeep DTLB usagearch/powerpc/include/asm/lppaca.h arch/powerpc/platforms/pseries/dtl.c arch/powerpc/platforms/pseries/lpar.c+16 -10 replies
5/9powerpc/pseries: Generalize hcall_vphn()arch/powerpc/mm/book3s64/vphn.h arch/powerpc/mm/numa.c+21 -140 replies
6/9powerpc/pseries: Move mm/book3s64/vphn.c under platforms/pseries/arch/powerpc/include/asm/lppaca.h arch/powerpc/mm/book3s64/Makefile arch/powerpc/mm/book3s64/vphn.h +7 more+46 -440 replies
7/9powerpc/pseries: Provide vcpu dispatch statisticsarch/powerpc/include/asm/topology.h arch/powerpc/mm/numa.c arch/powerpc/platforms/pseries/lpar.c+545 -20 replies
8/9powerpc/pseries: Protect against hogging the cpu while setting up the statsarch/powerpc/include/asm/lppaca.h arch/powerpc/platforms/pseries/lpar.c arch/powerpc/platforms/pseries/setup.c+22 -110 replies
9/9powerpc/pseries: Add documentation for vcpudispatch_statsDocumentation/powerpc/vcpudispatch_stats.txt+68 -00 replies

Reviews trailers per patch

2 of 9 patches have a Reviewed-by or Acked-by.

PatchNathan Lynch
1/9A-b
2/9A-b
3/9
4/9
5/9
6/9
7/9
8/9
9/9

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v3 thread

Cover letter [PATCH v3 0/9] Provide vcpu dispatch statistics

Changes since v2:
- Moved most prototypes to asm/lppaca.h instead of asm/plpar_wrappers.h
- Patch 4/9 changed to use rwlocks for guarding DTL buffer usage across debugfs and the new vcpudispatch_stats.
- Patch 6/9 has been updated to address breakage of the vphn selftest under tools/testing/selftests/powerpc/vphn.
- Patch 7/9 has changes to accommodate use of rwlocks, as well as some simplification for locking while enabling/disabling statistics.

Changes since v1:
- Changes have been done primarily to address a few build issues, by moving changes out of mm/numa.c to other places.
- Patches 1-5,8-9 have been rebased with patches 4 and 5 being interchanged, and patch 4 incorporating #ifdef CONFIG_PPC_SPLPAR.
- Patch 6 is new and moves vphn code under pseries.
- Patch 7 has changes to move most of the code from mm/numa.c into platforms/pseries/lpar.c, along with guarding some of these under CONFIG_PPC_SPLPAR.

Changes since RFC:
- Patches 1/8 to 5/8: no changes, except rebase to powerpc/merge
- Patch 6/8: The mutex guarding the vphn hcall has been dropped. It was only meant to serialize hcalls issued when stats are initially enabled. However, in reality, the various per-cpu workers will be scheduled at slightly different times and chances of hcalls for retrieving the same associativity information at the same time is very less. Even in that case, there are no other side effects.
- Patch 6/8: The third column for vcpu dispatches on the same core, but different thread has been dropped and merged with the second column.
- Patch 7/8: new patch to ensure we don't take too much time while enabling/disabling statistics on large systems with heavy workload.
- Patch 8/8: new patch adding a document describing the fields in the procfs file.

--
This series adds a new procfs file /proc/powerpc/vcpudispatch_stats for providing statistics around how the LPAR processors are dispatched by the POWER Hypervisor, in a shared LPAR environment. Patch 7/9 has more details on how the statistics are gathered.

An example output:

    $ sudo cat /proc/powerpc/vcpudispatch_stats
    cpu0 6839 4126 2683 30 0 6821 18 0
    cpu1 2515 1274 1229 12 0 2509 6 0
    cpu2 2317 1198 1109 10 0 2312 5 0
    cpu3 2259 1165 1088 6 0 2256 3 0
    cpu4 2205 1143 1056 6 0 2202 3 0
    cpu5 2165 1121 1038 6 0 2162 3 0
    cpu6 2183 1127 1050 6 0 2180 3 0
    cpu7 2193 1133 1052 8 0 2187 6 0
    cpu8 2165 1115 1032 18 0 2156 9 0
    cpu9 2301 1252 1033 16 0 2293 8 0
    cpu10 2197 1138 1041 18 0 2187 10 0
    cpu11 2273 1185 1062 26 0 2260 13 0
    cpu12 2186 1125 1043 18 0 2177 9 0
    cpu13 2161 1115 1030 16 0 2153 8 0
    cpu14 2206 1153 1033 20 0 2196 10 0
    cpu15 2163 1115 1032 16 0 2155 8 0

In the output above, for vcpu0, there have been 6839 dispatches since statistics were enabled. 4126 of those dispatches were on the same physical cpu as the last time. 2683 were on a different core, but within the same chip, while 30 dispatches were on a different chip compared to its last dispatch.

Also, out of the total of 6839 dispatches, we see that there have been 6821 dispatches on the vcpu's home node, while 18 dispatches were outside its home node, on a neighbouring chip.

- Naveen

Naveen N. Rao (9):
powerpc/pseries: Use macros for referring to the DTL enable mask powerpc/pseries: Do not save the previous DTL mask value powerpc/pseries: Factor out DTL buffer allocation and registration
routines
powerpc/pseries: Introduce rwlock to gatekeep DTLB usage powerpc/pseries: Generalize hcall_vphn()
powerpc/pseries: Move mm/book3s64/vphn.c under platforms/pseries/ powerpc/pseries: Provide vcpu dispatch statistics powerpc/pseries: Protect against hogging the cpu while setting up the
stats
powerpc/pseries: Add documentation for vcpudispatch_stats

 Documentation/powerpc/vcpudispatch_stats.txt  |  68 ++
 arch/powerpc/include/asm/lppaca.h             |  40 ++
 arch/powerpc/include/asm/topology.h           |   6 +
 arch/powerpc/mm/book3s64/Makefile             |   1 -
 arch/powerpc/mm/book3s64/vphn.h               |  16 -
 arch/powerpc/mm/numa.c                        |  35 +-
 arch/powerpc/platforms/pseries/Makefile       |   1 +
 arch/powerpc/platforms/pseries/dtl.c          |  23 +-
 arch/powerpc/platforms/pseries/lpar.c         | 602 +++++++++++++++++-
 arch/powerpc/platforms/pseries/setup.c        |  34 +-
 .../{mm/book3s64 => platforms/pseries}/vphn.c |  20 +-
 tools/testing/selftests/powerpc/vphn/Makefile |   2 +-
 .../selftests/powerpc/vphn/asm/lppaca.h       |   1 +
 tools/testing/selftests/powerpc/vphn/vphn.c   |   2 +-
 tools/testing/selftests/powerpc/vphn/vphn.h   |   1 -
 15 files changed, 752 insertions(+), 100 deletions(-)
 create mode 100644 Documentation/powerpc/vcpudispatch_stats.txt
 delete mode 100644 arch/powerpc/mm/book3s64/vphn.h
 rename arch/powerpc/{mm/book3s64 => platforms/pseries}/vphn.c (80%)
 create mode 120000 tools/testing/selftests/powerpc/vphn/asm/lppaca.h
 delete mode 120000 tools/testing/selftests/powerpc/vphn/vphn.h
Signature · 1 line
2.22.0