Track and expose idle PURR and SPURR ticks
Subsystems (2)
v2 cover Track and expose idle PURR and SPURR ticks
From: "Gautham R. Shenoy" <[email protected]>
Hi,
This is the second version of the patches to track and expose idle PURR and SPURR ticks. These patches are required by tools such as lparstat to compute system utilization for capacity planning purposes.
v1 can be found here: https://lore.kernel.org/patchwork/cover/1159341/
The key changes from v1 are
- The sysfs reads of idle PURR and SPURR now send an smp_call_function to the target CPU in order to read the most recent value of idle PURR and SPURR. This is required if the target CPU was idle for a long duration, in which case the cycles corresponding to its latest idle duration would not be updated in the variable tracking idle PURR/SPURR. Thus merely reading the variable would not reflect the most accurate idle PURR/SPURR ticks.
- Ensured that even when idle PURR/SPURR values are read in an interrupt context in-between idle_loop_prolog() and idle_loop_epilog(), we return the value that includes the cycles spent in the most recent idle period.
- The sysfs files for idle_purr and idle_spurr are created only when the FW_FEATURE_LPAR is enabled (the earlier version was checking for FW_FEATURE_SPLPAR)
Motivation:
===========
On PSeries LPARs, the data centers planners desire a more accurate view of system utilization per resource such as CPU to plan the system capacity requirements better. Such accuracy can be obtained by reading PURR/SPURR registers for CPU resource utilization.
Tools such as lparstat which are used to compute the utilization need to know [S]PURR ticks when the cpu was busy or idle. The [S]PURR counters are already exposed through sysfs. We already account for PURR ticks when we go to idle so that we can update the VPA area. This patchset extends support to account for SPURR ticks when idle, and expose both via per-cpu sysfs files.
These patches are required for enhancement to the lparstat utility that compute the CPU utilization based on PURR and SPURR which can be found here :
https://groups.google.com/forum/#!topic/powerpc-utils-devel/fYRo69xO9r4
With the patches, when lparstat is run on a LPAR running CPU-Hogs,
=========================================================================
$sudo ./src/lparstat -E 1 3
System Configuration
type=Dedicated mode=Capped smt=8 lcpu=2 mem=4834176 kB cpus=0 ent=2.00
---Actual--- -Normalized-
%busy %idle Frequency %busy %idle ------ ------ ------------- ------ ------ 99.99 0.00 3.35GHz[111%] 110.99 0.00 100.00 0.00 3.35GHz[111%] 111.00 0.00 100.00 0.00 3.35GHz[111%] 111.00 0.00
=========================================================================
When lparstat is run on an LPAR that is idle,
=========================================================================
$ sudo ./src/lparstat -E 1 3 System Configuration type=Dedicated mode=Capped smt=8 lcpu=2 mem=4834176 kB cpus=0 ent=2.00 ---Actual--- -Normalized- %busy %idle Frequency %busy %idle ------ ------ ------------- ------ ------ 0.09 99.91 2.11GHz[ 70%] 0.11 69.90 0.32 99.68 2.17GHz[ 72%] 0.25 71.75 0.56 99.44 2.18GHz[ 72%] 0.42 71.58 =========================================================================
Gautham R. Shenoy (5):
powerpc: Move idle_loop_prolog()/epilog() functions to header file powerpc/idle: Add accessor function to always read latest idle PURR powerpc/pseries: Account for SPURR ticks on idle CPUs powerpc/sysfs: Show idle_purr and idle_spurr for every CPU
Documentation: Document sysfs interfaces purr, spurr, idle_purr,
idle_spurr
Documentation/ABI/testing/sysfs-devices-system-cpu | 39 ++++++++++ arch/powerpc/include/asm/idle.h | 88 ++++++++++++++++++++++ arch/powerpc/kernel/sysfs.c | 54 ++++++++++++- arch/powerpc/platforms/pseries/setup.c | 8 +- drivers/cpuidle/cpuidle-pseries.c | 39 ++-------- 5 files changed, 191 insertions(+), 37 deletions(-) create mode 100644 arch/powerpc/include/asm/idle.h
Signature · 1 line
1.9.4
PATCH v2 1/5 powerpc: Move idle_loop_prolog()/epilog() functions to header file
From: "Gautham R. Shenoy" <[email protected]>
Currently prior to entering an idle state on a Linux Guest, the pseries cpuidle driver implement an idle_loop_prolog() and idle_loop_epilog() functions which ensure that idle_purr is correctly computed, and the hypervisor is informed that the CPU cycles have been donated.
These prolog and epilog functions are also required in the default idle call, i.e pseries_lpar_idle(). Hence move these accessor functions to a common header file and call them from pseries_lpar_idle(). Since the existing header files such as asm/processor.h have enough clutter, create a new header file asm/idle.h.
--- arch/powerpc/include/asm/idle.h | 27 +++++++++++++++++++++++++++ arch/powerpc/platforms/pseries/setup.c | 7 +++++-- drivers/cpuidle/cpuidle-pseries.c | 24 +----------------------- 3 files changed, 33 insertions(+), 25 deletions(-) create mode 100644 arch/powerpc/include/asm/idle.h
arch/powerpc/include/asm/idle.h new file +27 -0
@@ -0,0 +1,27 @@ +#ifndef _ASM_POWERPC_IDLE_H +#define _ASM_POWERPC_IDLE_H +#include <asm/runlatch.h> + +static inline void idle_loop_prolog (unsigned long *in_purr) +{ +ppc64_runlatch_off (); + *in_purr =mfspr (SPRN_PURR ); +/* +* Indicate to the HV that we are idle. Now would be +* a good time to find other work to dispatch. +*/ +get_lppaca ()->idle =1 ; +} + +static inline void idle_loop_epilog (unsigned long in_purr) +{ +u64 wait_cycles; + + wait_cycles =be64_to_cpu (get_lppaca ()->wait_state_cycles); + wait_cycles +=mfspr (SPRN_PURR ) - in_purr; +get_lppaca ()->wait_state_cycles =cpu_to_be64 (wait_cycles); +get_lppaca ()->idle =0 ; + +ppc64_runlatch_on (); +} +#endif
arch/powerpc/platforms/pseries/setup.c +5 -2
@@ -68,6 +68,7 @@#include <asm/isa-bridge.h> #include <asm/security_features.h> #include <asm/asm-const.h> +#include <asm/idle.h> #include <asm/swiotlb.h> #include <asm/svm.h>
@@ -319,6 +320,8 @@ static int alloc_dispatch_log_kmem_cache(void)static void pseries_lpar_idle (void ) { +unsigned long in_purr; +/* * Default handler to go into low thread priority and possibly * low power mode by ceding processor to hypervisor
@@ -328,7 +331,7 @@ static void pseries_lpar_idle(void)return ;/* Indicate to hypervisor that we are idle. */ -get_lppaca ()->idle =1 ; +idle_loop_prolog (&in_purr);/* * Yield the processor to the hypervisor. We return if
@@ -339,7 +342,7 @@ static void pseries_lpar_idle(void)*/ cede_processor (); -get_lppaca ()->idle =0 ; +idle_loop_epilog (in_purr); }/*
drivers/cpuidle/cpuidle-pseries.c +1 -23
@@ -19,6 +19,7 @@#include <asm/machdep.h> #include <asm/firmware.h> #include <asm/runlatch.h> +#include <asm/idle.h> #include <asm/plpar_wrappers.h> struct cpuidle_driver pseries_idle_driver = {
@@ -31,29 +32,6 @@ struct cpuidle_driver pseries_idle_driver = {static u64 snooze_timeout__read_mostly ;static bool snooze_timeout_en__read_mostly ; -static inline void idle_loop_prolog (unsigned long *in_purr) -{ -ppc64_runlatch_off (); - *in_purr =mfspr (SPRN_PURR ); -/* -* Indicate to the HV that we are idle. Now would be -* a good time to find other work to dispatch. -*/ -get_lppaca ()->idle =1 ; -} - -static inline void idle_loop_epilog (unsigned long in_purr) -{ -u64 wait_cycles; - - wait_cycles =be64_to_cpu (get_lppaca ()->wait_state_cycles); - wait_cycles +=mfspr (SPRN_PURR ) - in_purr; -get_lppaca ()->wait_state_cycles =cpu_to_be64 (wait_cycles); -get_lppaca ()->idle =0 ; - -ppc64_runlatch_on (); -} -static int snooze_loop (struct cpuidle_device *dev,struct cpuidle_driver *drv,int index)
Signature · 1 line
1.9.4
Re: [PATCH v2 1/5] powerpc: Move idle_loop_prolog()/epilog() functions to header file
Quoted 46 lines · Gautham R. Shenoy
From: "Gautham R. Shenoy" <[email protected]>
Currently prior to entering an idle state on a Linux Guest, the pseries cpuidle driver implement an
idle_loop_prolog()andidle_loop_epilog()functions which ensure thatidle_purris correctly computed, and the hypervisor is informed that the CPU cycles have been donated.These prolog and epilog functions are also required in the default idle call, i.e
pseries_lpar_idle(). Hence move these accessor functions to a common header file and call them frompseries_lpar_idle(). Since the existing header files such asasm/processor.hhave enough clutter, create a new header fileasm/idle.h.Signed-off-by Gautham R. Shenoy <[email protected]>--- arch/powerpc/include/asm/idle.h | 27 +++++++++++++++++++++++++++ arch/powerpc/platforms/pseries/setup.c | 7 +++++-- drivers/cpuidle/cpuidle-pseries.c | 24 +----------------------- 3 files changed, 33 insertions(+), 25 deletions(-) create mode 100644 arch/powerpc/include/asm/idle.harch/powerpc/include/asm/idle.h new file +17
-0@@ -0,0 +1,27 @@ +#ifndef _ASM_POWERPC_IDLE_H +#define _ASM_POWERPC_IDLE_H +#include <asm/runlatch.h> + +static inline void idle_loop_prolog (unsigned long *in_purr) +{ +ppc64_runlatch_off (); + *in_purr =mfspr (SPRN_PURR ); +/* +* Indicate to the HV that we are idle. Now would be +* a good time to find other work to dispatch. +*/ +get_lppaca ()->idle =1 ; +} + +static inline void idle_loop_epilog (unsigned long in_purr) +{
arch/powerpc/include/asm/idle.h line 27
Quoted 10 lines
arch/powerpc/include/asm/idle.h new file +10
-0+u64 wait_cycles; + + wait_cycles =be64_to_cpu (get_lppaca ()->wait_state_cycles); + wait_cycles +=mfspr (SPRN_PURR ) - in_purr; +get_lppaca ()->wait_state_cycles =cpu_to_be64 (wait_cycles); +get_lppaca ()->idle =0 ; + + ppc64_runlatch_on(); +} +#endif
Looks fine and correct as a cleanup, but asm/include/idle.h and idle_loop_prolog, idle_loop_epilog, strike me as too generic for pseries-specific code.
Re: [PATCH v2 1/5] powerpc: Move idle_loop_prolog()/epilog() functions to header file
Hello Nathan,
Quoted 54 lines · Nathan Lynch
Quoted 52 lines · Gautham R. Shenoy
From: "Gautham R. Shenoy" <[email protected]>
Currently prior to entering an idle state on a Linux Guest, the pseries cpuidle driver implement an
idle_loop_prolog()andidle_loop_epilog()functions which ensure thatidle_purris correctly computed, and the hypervisor is informed that the CPU cycles have been donated.These prolog and epilog functions are also required in the default idle call, i.e
pseries_lpar_idle(). Hence move these accessor functions to a common header file and call them frompseries_lpar_idle(). Since the existing header files such asasm/processor.hhave enough clutter, create a new header fileasm/idle.h.Signed-off-by Gautham R. Shenoy <[email protected]>--- arch/powerpc/include/asm/idle.h | 27 +++++++++++++++++++++++++++ arch/powerpc/platforms/pseries/setup.c | 7 +++++-- drivers/cpuidle/cpuidle-pseries.c | 24 +----------------------- 3 files changed, 33 insertions(+), 25 deletions(-) create mode 100644 arch/powerpc/include/asm/idle.harch/powerpc/include/asm/idle.h new file +23
-0@@ -0,0 +1,27 @@ +#ifndef _ASM_POWERPC_IDLE_H +#define _ASM_POWERPC_IDLE_H +#include <asm/runlatch.h> + +static inline void idle_loop_prolog (unsigned long *in_purr) +{ +ppc64_runlatch_off (); + *in_purr =mfspr (SPRN_PURR ); +/* +* Indicate to the HV that we are idle. Now would be +* a good time to find other work to dispatch. +*/ +get_lppaca ()->idle =1 ; +} + +static inline void idle_loop_epilog (unsigned long in_purr) +{ +u64 wait_cycles; + + wait_cycles =be64_to_cpu (get_lppaca ()->wait_state_cycles); + wait_cycles +=mfspr (SPRN_PURR ) - in_purr; +get_lppaca ()->wait_state_cycles =cpu_to_be64 (wait_cycles); +get_lppaca ()->idle =0 ;
Quoted 8 lines
arch/powerpc/include/asm/idle.h line 27
Quoted 4 lines
arch/powerpc/include/asm/idle.h new file +4
-0+ + ppc64_runlatch_on(); +} +#endif Looks fine and correct as a cleanup, but
asm/include/idle.handidle_loop_prolog,idle_loop_epilog, strike me as too generic for pseries-specific code.
Should it be prefixed with pseries , i.e pseries_idle_prolog() and pseries_idle_epilog() ?
Also, I am planning another round of cleanup to move all the idle-related declaration from asm/include/processor.h to asm/include/idle.h
Re: [PATCH v2 1/5] powerpc: Move idle_loop_prolog()/epilog() functions to header file
Quoted 7 lines · Gautham R Shenoy
Quoted 3 lines · Nathan Lynch
Looks fine and correct as a cleanup, but
asm/include/idle.handidle_loop_prolog,idle_loop_epilog, strike me as too generic for pseries-specific code.Should it be prefixed with pseries , i.e
pseries_idle_prolog()andpseries_idle_epilog()?
Yes that seems appropriate to me.
PATCH v2 2/5 powerpc/idle: Add accessor function to always read latest idle PURR
In reply to Gautham R. ShenoyFrom: "Gautham R. Shenoy" <[email protected]>
Currently when CPU goes idle, we take a snapshot of PURR via idle_loop_prolog() which is used at the CPU idle exit to compute the idle PURR cycles via the function idle_loop_epilog(). Thus, the value of idle PURR cycle thus read before idle_loop_prolog() and after idle_loop_epilog() is always correct.
However, if we were to read the idle PURR cycles from an interrupt context between idle_loop_prolog() and idle_loop_epilog() (this will be done in a future patch), then, the value of the idle PURR thus read will not include the cycles spent in the most recent idle period.
This patch addresses the issue by providing accessor function to read the idle PURR such such that it includes the cycles spent in the most recent idle period, if we read it between idle_loop_prolog() and idle_loop_epilog(). In order to achieve it, the patch saves the snapshot of PURR in idle_loop_prolog() in a per-cpu variable, instead of on the stack, so that it can be accessed from an interrupt context.
--- arch/powerpc/include/asm/idle.h | 46 +++++++++++++++++++++++++++------- arch/powerpc/platforms/pseries/setup.c | 7 +++--- drivers/cpuidle/cpuidle-pseries.c | 15 +++++------ 3 files changed, 46 insertions(+), 22 deletions(-)
arch/powerpc/include/asm/idle.h +37 -9
@@ -2,10 +2,27 @@#define _ASM_POWERPC_IDLE_H#include <asm/runlatch.h> -static inline void idle_loop_prolog (unsigned long *in_purr) +DECLARE_PER_CPU (u64, idle_entry_purr_snap); + +static inline void snapshot_purr_idle_entry (void ) +{ + *this_cpu_ptr (&idle_entry_purr_snap) =mfspr (SPRN_PURR ); +} + +static inline void update_idle_purr_accounting (void ) +{ +u64 wait_cycles; +u64 in_purr = *this_cpu_ptr (&idle_entry_purr_snap); + + wait_cycles =be64_to_cpu (get_lppaca ()->wait_state_cycles); + wait_cycles +=mfspr (SPRN_PURR ) - in_purr; +get_lppaca ()->wait_state_cycles =cpu_to_be64 (wait_cycles); +} + +static inline void idle_loop_prolog (void ) {ppc64_runlatch_off (); - *in_purr =mfspr (SPRN_PURR ); +snapshot_purr_idle_entry ();/* * Indicate to the HV that we are idle. Now would be * a good time to find other work to dispatch.
@@ -13,15 +30,26 @@ static inline void idle_loop_prolog(unsigned long *in_purr) get_lppaca()->idle = 1; } -static inline void idle_loop_epilog (unsigned long in_purr) +static inline void idle_loop_epilog (void ) { -u64 wait_cycles; - - wait_cycles =be64_to_cpu (get_lppaca ()->wait_state_cycles); - wait_cycles +=mfspr (SPRN_PURR ) - in_purr; -get_lppaca ()->wait_state_cycles =cpu_to_be64 (wait_cycles); +update_idle_purr_accounting ();get_lppaca ()->idle =0 ; -ppc64_runlatch_on (); } + +static inline u64 read_this_idle_purr (void ) +{ +/* +* If we are reading from an idle context, update the +* idle-purr cycles corresponding to the last idle period. +* Since the idle context is not yet over, take a fresh +* snapshot of the idle-purr. +*/ +if (unlikely (get_lppaca ()->idle ==1 )) { +update_idle_purr_accounting (); +snapshot_purr_idle_entry (); + } + +return be64_to_cpu (get_lppaca ()->wait_state_cycles); +}#endif
arch/powerpc/platforms/pseries/setup.c +3 -4
@@ -318,10 +318,9 @@ static int alloc_dispatch_log_kmem_cache(void) }machine_early_initcall (pseries, alloc_dispatch_log_kmem_cache); +DEFINE_PER_CPU (u64, idle_entry_purr_snap);static void pseries_lpar_idle (void ) { -unsigned long in_purr; -/* * Default handler to go into low thread priority and possibly * low power mode by ceding processor to hypervisor
@@ -331,7 +330,7 @@ static void pseries_lpar_idle(void)return ;/* Indicate to hypervisor that we are idle. */ -idle_loop_prolog (&in_purr); +idle_loop_prolog ();/* * Yield the processor to the hypervisor. We return if
@@ -342,7 +341,7 @@ static void pseries_lpar_idle(void)*/ cede_processor (); -idle_loop_epilog (in_purr); +idle_loop_epilog (); }/*
drivers/cpuidle/cpuidle-pseries.c +6 -9
@@ -36,12 +36,11 @@ static int snooze_loop(struct cpuidle_device *dev, struct cpuidle_driver *drv, int index) { -unsigned long in_purr;u64 snooze_exit_time;set_thread_flag (TIF_POLLING_NRFLAG ); -idle_loop_prolog (&in_purr); +idle_loop_prolog ();local_irq_enable (); snooze_exit_time =get_tb () + snooze_timeout;
@@ -65,7 +64,7 @@ static int snooze_loop(struct cpuidle_device *dev,local_irq_disable (); -idle_loop_epilog (in_purr); +idle_loop_epilog ();return index; }
@@ -91,9 +90,8 @@ static int dedicated_cede_loop(struct cpuidle_device *dev,struct cpuidle_driver *drv, int index) { -unsigned long in_purr; -idle_loop_prolog (&in_purr); +idle_loop_prolog ();get_lppaca ()->donate_dedicated_cpu =1 ;HMT_medium ();
@@ -102,7 +100,7 @@ static int dedicated_cede_loop(struct cpuidle_device *dev,local_irq_disable ();get_lppaca ()->donate_dedicated_cpu =0 ; -idle_loop_epilog (in_purr); +idle_loop_epilog ();return index; }
@@ -111,9 +109,8 @@ static int shared_cede_loop(struct cpuidle_device *dev,struct cpuidle_driver *drv, int index) { -unsigned long in_purr; -idle_loop_prolog (&in_purr); +idle_loop_prolog ();/* * Yield the processor to the hypervisor. We return if
@@ -125,7 +122,7 @@ static int shared_cede_loop(struct cpuidle_device *dev,check_and_cede_processor ();local_irq_disable (); -idle_loop_epilog (in_purr); +idle_loop_epilog ();return index; }
Signature · 1 line
1.9.4
PATCH v2 3/5 powerpc/pseries: Account for SPURR ticks on idle CPUs
In reply to Gautham R. ShenoyFrom: "Gautham R. Shenoy" <[email protected]>
On Pseries LPARs, to calculate utilization, we need to know the [S]PURR ticks when the CPUs were busy or idle.
Via idle_loop_prolog(), idle_loop_epilog(), we track the idle PURR ticks in the VPA variable "wait_state_cycles". This patch extends the support to account for the idle SPURR ticks. It also provides an accessor function to accurately reads idle SPURR ticks.
--- arch/powerpc/include/asm/idle.h | 33 +++++++++++++++++++++++++++++++++ arch/powerpc/platforms/pseries/setup.c | 2 ++ 2 files changed, 35 insertions(+)
arch/powerpc/include/asm/idle.h +33 -0
@@ -2,13 +2,20 @@#define _ASM_POWERPC_IDLE_H#include <asm/runlatch.h> +DECLARE_PER_CPU (u64, idle_spurr_cycles);DECLARE_PER_CPU (u64, idle_entry_purr_snap); +DECLARE_PER_CPU (u64, idle_entry_spurr_snap);static inline void snapshot_purr_idle_entry (void ) { *this_cpu_ptr (&idle_entry_purr_snap) =mfspr (SPRN_PURR ); } +static inline void snapshot_spurr_idle_entry (void ) +{ + *this_cpu_ptr (&idle_entry_spurr_snap) =mfspr (SPRN_SPURR ); +} +static inline void update_idle_purr_accounting (void ) {u64 wait_cycles;
@@ -19,10 +26,19 @@ static inline void update_idle_purr_accounting(void)get_lppaca ()->wait_state_cycles =cpu_to_be64 (wait_cycles); } +static inline void update_idle_spurr_accounting (void ) +{ +u64 *idle_spurr_cycles_ptr =this_cpu_ptr (&idle_spurr_cycles); +u64 in_spurr = *this_cpu_ptr (&idle_entry_spurr_snap); + + *idle_spurr_cycles_ptr +=mfspr (SPRN_SPURR ) - in_spurr; +} +static inline void idle_loop_prolog (void ) {ppc64_runlatch_off ();snapshot_purr_idle_entry (); +snapshot_spurr_idle_entry ();/* * Indicate to the HV that we are idle. Now would be * a good time to find other work to dispatch.
@@ -33,6 +49,7 @@ static inline void idle_loop_prolog(void)static inline void idle_loop_epilog (void ) {update_idle_purr_accounting (); +update_idle_spurr_accounting ();get_lppaca ()->idle =0 ;ppc64_runlatch_on (); }
@@ -52,4 +69,20 @@ static inline u64 read_this_idle_purr(void) return be64_to_cpu(get_lppaca()->wait_state_cycles); } + +static inline u64 read_this_idle_spurr (void ) +{ +/* +* If we are reading from an idle context, update the +* idle-spurr cycles corresponding to the last idle period. +* Since the idle context is not yet over, take a fresh +* snapshot of the idle-spurr. +*/ +if (get_lppaca ()->idle ==1 ) { +update_idle_spurr_accounting (); +snapshot_spurr_idle_entry (); + } + +return *this_cpu_ptr (&idle_spurr_cycles); +}#endif
arch/powerpc/platforms/pseries/setup.c +2 -0
@@ -318,7 +318,9 @@ static int alloc_dispatch_log_kmem_cache(void) }machine_early_initcall (pseries, alloc_dispatch_log_kmem_cache); +DEFINE_PER_CPU (u64, idle_spurr_cycles);DEFINE_PER_CPU (u64, idle_entry_purr_snap); +DEFINE_PER_CPU (u64, idle_entry_spurr_snap);static void pseries_lpar_idle (void ) {/*
Signature · 1 line
1.9.4
Re: [PATCH v2 3/5] powerpc/pseries: Account for SPURR ticks on idle CPUs
Quoted 5 lines · Gautham R. Shenoy
arch/powerpc/include/asm/idle.h +5
-0+static inline void snapshot_spurr_idle_entry (void ) +{ + *this_cpu_ptr (&idle_entry_spurr_snap) =mfspr (SPRN_SPURR ); +} +
[...]
Quoted 7 lines
arch/powerpc/include/asm/idle.h +7
-0+static inline void update_idle_spurr_accounting (void ) +{ +u64 *idle_spurr_cycles_ptr =this_cpu_ptr (&idle_spurr_cycles); +u64 in_spurr = *this_cpu_ptr (&idle_entry_spurr_snap); + + *idle_spurr_cycles_ptr +=mfspr (SPRN_SPURR ) - in_spurr; +}
[...]
arch/powerpc/include/asm/idle.h static inline u64 read_this_idle_purr(void) · line 83
Quoted 11 lines
arch/powerpc/include/asm/idle.h +11
-0+static inline u64 read_this_idle_spurr (void ) +{ +/* +* If we are reading from an idle context, update the +* idle-spurr cycles corresponding to the last idle period. +* Since the idle context is not yet over, take a fresh +* snapshot of the idle-spurr. +*/ +if (get_lppaca ()->idle ==1 ) { +update_idle_spurr_accounting (); +snapshot_spurr_idle_entry ();
This samples spurr twice when it could do with just one. I don't know the performance implications, but will the results be coherent?
Re: [PATCH v2 3/5] powerpc/pseries: Account for SPURR ticks on idle CPUs
Hello Nathan,
Quoted 19 lines · Nathan Lynch
Quoted 5 lines · Gautham R. Shenoy
arch/powerpc/include/asm/idle.h +5
-0+static inline void snapshot_spurr_idle_entry (void ) +{ + *this_cpu_ptr (&idle_entry_spurr_snap) =mfspr (SPRN_SPURR ); +} +[...]
Quoted 7 lines
arch/powerpc/include/asm/idle.h +7
-0+static inline void update_idle_spurr_accounting (void ) +{ +u64 *idle_spurr_cycles_ptr =this_cpu_ptr (&idle_spurr_cycles); +u64 in_spurr = *this_cpu_ptr (&idle_entry_spurr_snap); + + *idle_spurr_cycles_ptr +=mfspr (SPRN_SPURR ) - in_spurr; +}[...]
Quoted 14 lines
arch/powerpc/include/asm/idle.h static inline u64 read_this_idle_purr(void) · line 83
Quoted 11 lines
arch/powerpc/include/asm/idle.h +11
-0+static inline u64 read_this_idle_spurr (void ) +{ +/* +* If we are reading from an idle context, update the +* idle-spurr cycles corresponding to the last idle period. +* Since the idle context is not yet over, take a fresh +* snapshot of the idle-spurr. +*/ +if (get_lppaca ()->idle ==1 ) { +update_idle_spurr_accounting (); +snapshot_spurr_idle_entry ();This samples spurr twice when it could do with just one. I don't know the performance implications, but will the results be coherent?
We would have taken the snapshot in idle_loop_prolog() just before entering idle. That fact that the "if" condition is true above in read_this_idle_spurr() implies that we are reading the idle_spurr value from an interrupt context and since get_lppaca()->idle == 1, we haven't yet called idle_loop_epilog(), where we would have updated the idle_spurr ticks for the last idle period.
Hence, in this function, we first update the idle_spurr accounting from the time of the last snapshot to now. We update the snapshot to the current SPURR value so that when we eventually call idle_loop_epilog(), we will account for the remaining idle duration, i.e from the read_this_idle_spurr() call to idle_loop_epilog()
The results are therefore coherant, in that we do not perform double accounting the second time we invoke update_idle_spurr_accounting() from idle_loop_epilog(), but only add the spurr ticks from read_this_idle_spurr() to idle_loop_epilog().
--
Thanks and Regards
gautham.
PATCH v2 4/5 powerpc/sysfs: Show idle_purr and idle_spurr for every CPU
In reply to Gautham R. ShenoyFrom: "Gautham R. Shenoy" <[email protected]>
On Pseries LPARs, to calculate utilization, we need to know the [S]PURR ticks when the CPUs were busy or idle.
The total PURR and SPURR ticks are already exposed via the per-cpu sysfs files "purr" and "spurr". This patch adds support for exposing the idle PURR and SPURR ticks via new per-cpu sysfs files named "idle_purr" and "idle_spurr".
--- arch/powerpc/kernel/sysfs.c | 54 ++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 51 insertions(+), 3 deletions(-)
arch/powerpc/kernel/sysfs.c +51 -3
@@ -19,6 +19,7 @@#include <asm/smp.h> #include <asm/pmc.h> #include <asm/firmware.h> +#include <asm/idle.h> #include <asm/svm.h> #include "cacheinfo.h"
@@ -733,6 +734,42 @@ static void create_svm_file(void) }#endif /* CONFIG_PPC_SVM */ +static void read_idle_purr (void *val) +{ +u64 *ret = (u64 *)val; + + *ret =read_this_idle_purr (); +} + +static ssize_t idle_purr_show (struct device *dev, +struct device_attribute *attr,char *buf) +{ + struct cpu *cpu = container_of(dev, struct cpu, dev); +u64 val; + +smp_call_function_single (cpu->dev.id, read_idle_purr, &val,1 ); +return sprintf (buf,"%llx\n" , val); +} +static DEVICE_ATTR(idle_purr, 0400, idle_purr_show, NULL); + +static void read_idle_spurr (void *val) +{ +u64 *ret = (u64 *)val; + + *ret =read_this_idle_spurr (); +} + +static ssize_t idle_spurr_show (struct device *dev, +struct device_attribute *attr,char *buf) +{ + struct cpu *cpu = container_of(dev, struct cpu, dev); +u64 val; + +smp_call_function_single (cpu->dev.id, read_idle_spurr, &val,1 ); +return sprintf (buf,"%llx\n" , val); +} +static DEVICE_ATTR(idle_spurr, 0400, idle_spurr_show, NULL); +static int register_cpu_online (unsigned int cpu) {struct cpu *c = &per_cpu (cpu_devices, cpu);
@@ -794,10 +831,15 @@ static int register_cpu_online(unsigned int cpu)if (!firmware_has_feature (FW_FEATURE_LPAR ))add_write_permission_dev_attr (&dev_attr_purr);device_create_file (s, &dev_attr_purr); +if (firmware_has_feature (FW_FEATURE_LPAR )) +device_create_file (s, &dev_attr_idle_purr); } -if (cpu_has_feature (CPU_FTR_SPURR )) +if (cpu_has_feature (CPU_FTR_SPURR )) {device_create_file (s, &dev_attr_spurr); +if (firmware_has_feature (FW_FEATURE_LPAR )) +device_create_file (s, &dev_attr_idle_spurr); + }if (cpu_has_feature (CPU_FTR_DSCR ))device_create_file (s, &dev_attr_dscr);
@@ -879,11 +921,17 @@ static int unregister_cpu_online(unsigned int cpu)if (cpu_has_feature (CPU_FTR_MMCRA ))device_remove_file (s, &dev_attr_mmcra); -if (cpu_has_feature (CPU_FTR_PURR )) +if (cpu_has_feature (CPU_FTR_PURR )) {device_remove_file (s, &dev_attr_purr); +if (firmware_has_feature (FW_FEATURE_LPAR )) +device_remove_file (s, &dev_attr_idle_purr); + } -if (cpu_has_feature (CPU_FTR_SPURR )) +if (cpu_has_feature (CPU_FTR_SPURR )) {device_remove_file (s, &dev_attr_spurr); +if (firmware_has_feature (FW_FEATURE_LPAR )) +device_remove_file (s, &dev_attr_idle_spurr); + }if (cpu_has_feature (CPU_FTR_DSCR ))device_remove_file (s, &dev_attr_dscr);
Signature · 1 line
1.9.4
Re: [PATCH v2 4/5] powerpc/sysfs: Show idle_purr and idle_spurr for every CPU
Quoted 13 lines · Gautham R. Shenoy
arch/powerpc/kernel/sysfs.c +1
-0@@ -19,6 +19,7 @@#include <asm/smp.h> #include <asm/pmc.h> #include <asm/firmware.h> +#include <asm/idle.h> #include <asm/svm.h> #include "cacheinfo.h"
arch/powerpc/kernel/sysfs.c static void create_svm_file(void) · line 739
Quoted 6 lines
arch/powerpc/kernel/sysfs.c +3
-0@@ -733,6 +734,42 @@ static void create_svm_file(void) }#endif /* CONFIG_PPC_SVM */ +static void read_idle_purr (void *val) +{ +u64 *ret = (u64 *)val;
No cast from void* needed.
Quoted 22 lines
arch/powerpc/kernel/sysfs.c +22
-0+ + *ret =read_this_idle_purr (); +} + +static ssize_t idle_purr_show (struct device *dev, +struct device_attribute *attr,char *buf) +{ + struct cpu *cpu = container_of(dev, struct cpu, dev); +u64 val; + +smp_call_function_single (cpu->dev.id, read_idle_purr, &val,1 ); +return sprintf (buf,"%llx\n" , val); +} +static DEVICE_ATTR(idle_purr, 0400, idle_purr_show, NULL); + +static void read_idle_spurr (void *val) +{ +u64 *ret = (u64 *)val; + + *ret =read_this_idle_spurr (); +} +
arch/powerpc/kernel/sysfs.c static void create_svm_file(void) · line 771
Quoted 10 lines
arch/powerpc/kernel/sysfs.c +10
-0+static ssize_t idle_spurr_show (struct device *dev, +struct device_attribute *attr,char *buf) +{ + struct cpu *cpu = container_of(dev, struct cpu, dev); +u64 val; + +smp_call_function_single (cpu->dev.id, read_idle_spurr, &val,1 ); +return sprintf (buf,"%llx\n" , val); +} +static DEVICE_ATTR(idle_spurr, 0400, idle_spurr_show, NULL);
It's regrettable that we have to wake up potentially idle CPUs in order to derive correct idle statistics for them, but I suppose the main user (lparstat) of these interfaces already is causing this to happen by polling the existing per-cpu purr and spurr attributes.
So now lparstat will incur at minimum four syscalls and four IPIs per CPU per polling interval -- one for each of purr, spurr, idle_purr and idle_spurr. Correct?
At some point it's going to make sense to batch sampling of remote CPUs' SPRs.
Quoted 30 lines
arch/powerpc/kernel/sysfs.c +10
-2static int register_cpu_online (unsigned int cpu) {struct cpu *c = &per_cpu (cpu_devices, cpu);@@ -794,10 +831,15 @@ static int register_cpu_online(unsigned int cpu)if (!firmware_has_feature (FW_FEATURE_LPAR ))add_write_permission_dev_attr (&dev_attr_purr);device_create_file (s, &dev_attr_purr); +if (firmware_has_feature (FW_FEATURE_LPAR )) +device_create_file (s, &dev_attr_idle_purr); } -if (cpu_has_feature (CPU_FTR_SPURR )) +if (cpu_has_feature (CPU_FTR_SPURR )) {device_create_file (s, &dev_attr_spurr); +if (firmware_has_feature (FW_FEATURE_LPAR )) +device_create_file (s, &dev_attr_idle_spurr); + }if (cpu_has_feature (CPU_FTR_DSCR ))device_create_file (s, &dev_attr_dscr);@@ -879,11 +921,17 @@ static int unregister_cpu_online(unsigned int cpu)if (cpu_has_feature (CPU_FTR_MMCRA ))device_remove_file (s, &dev_attr_mmcra); -if (cpu_has_feature (CPU_FTR_PURR )) +if (cpu_has_feature (CPU_FTR_PURR )) {device_remove_file (s, &dev_attr_purr); +if (firmware_has_feature (FW_FEATURE_LPAR )) +device_remove_file (s, &dev_attr_idle_purr); + }
arch/powerpc/kernel/sysfs.c static int unregister_cpu_online(unsigned int cpu) · line 937
Quoted 10 lines
arch/powerpc/kernel/sysfs.c +4
-1-if (cpu_has_feature (CPU_FTR_SPURR )) +if (cpu_has_feature (CPU_FTR_SPURR )) {device_remove_file (s, &dev_attr_spurr); +if (firmware_has_feature (FW_FEATURE_LPAR )) +device_remove_file (s, &dev_attr_idle_spurr); + }if (cpu_has_feature (CPU_FTR_DSCR ))device_remove_file (s, &dev_attr_dscr);
The cpu register/unregister stuff here looks correct.
Re: [PATCH v2 4/5] powerpc/sysfs: Show idle_purr and idle_spurr for every CPU
Quoted 17 lines · Nathan Lynch
Quoted 16 lines · Gautham R. Shenoy
arch/powerpc/kernel/sysfs.c +1
-0@@ -19,6 +19,7 @@#include <asm/smp.h> #include <asm/pmc.h> #include <asm/firmware.h> +#include <asm/idle.h> #include <asm/svm.h> #include "cacheinfo.h" @@ -733,6 +734,42 @@ static void create_svm_file(void) }#endif /* CONFIG_PPC_SVM */
Quoted 5 lines
arch/powerpc/kernel/sysfs.c static void create_svm_file(void) · line 739
Quoted 3 lines
arch/powerpc/kernel/sysfs.c +3
-0+static void read_idle_purr (void *val) +{ +u64 *ret = (u64 *)val;No cast from void* needed.
Will fix this. Thanks.
Quoted 40 lines
Quoted 24 lines
arch/powerpc/kernel/sysfs.c +22
-0+ + *ret =read_this_idle_purr (); +} + +static ssize_t idle_purr_show (struct device *dev, +struct device_attribute *attr,char *buf) +{ + struct cpu *cpu = container_of(dev, struct cpu, dev); +u64 val; + +smp_call_function_single (cpu->dev.id, read_idle_purr, &val,1 ); +return sprintf (buf,"%llx\n" , val); +} +static DEVICE_ATTR(idle_purr, 0400, idle_purr_show, NULL); + +static void read_idle_spurr (void *val) +{ +u64 *ret = (u64 *)val; + + *ret =read_this_idle_spurr (); +} +arch/powerpc/kernel/sysfs.c static void create_svm_file(void) · line 771
Quoted 10 lines
arch/powerpc/kernel/sysfs.c +10
-0+static ssize_t idle_spurr_show (struct device *dev, +struct device_attribute *attr,char *buf) +{ + struct cpu *cpu = container_of(dev, struct cpu, dev); +u64 val; + +smp_call_function_single (cpu->dev.id, read_idle_spurr, &val,1 ); +return sprintf (buf,"%llx\n" , val); +} +static DEVICE_ATTR(idle_spurr, 0400, idle_spurr_show, NULL);It's regrettable that we have to wake up potentially idle CPUs in order to derive correct idle statistics for them, but I suppose the main user (lparstat) of these interfaces already is causing this to happen by polling the existing per-cpu purr and spurr attributes.
Quoted 3 lines
So now lparstat will incur at minimum four syscalls and four IPIs per CPU per polling interval -- one for each of purr, spurr,
idle_purrandidle_spurr. Correct?
Yes, it is unforunate that we will end up making four syscalls and generating IPI noise, and this is something that I discussed with Naveen and Kamalesh. We have the following two constraints:
1) These values of PURR and SPURR required are per-cpu. Hence putting them in lparcfg is not an option.
2) sysfs semantics encourages a single value per key, the key being the sysfs-file. Something like the following would have made far more sense.
cat /sys/devices/system/cpu/cpuX/purr_spurr_accounting purr:A
idle_purr:B
spurr:C
idle_spurr:D
There are some sysfs files which allow something like this. Eg:/sys/devices/system/cpu/cpu0/cpufreq/stats/time_in_state
Thoughts on any other alternatives?
Quoted 43 lines
At some point it's going to make sense to batch sampling of remote CPUs' SPRs.
Quoted 38 lines
arch/powerpc/kernel/sysfs.c +14
-3static int register_cpu_online (unsigned int cpu) {struct cpu *c = &per_cpu (cpu_devices, cpu);@@ -794,10 +831,15 @@ static int register_cpu_online(unsigned int cpu)if (!firmware_has_feature (FW_FEATURE_LPAR ))add_write_permission_dev_attr (&dev_attr_purr);device_create_file (s, &dev_attr_purr); +if (firmware_has_feature (FW_FEATURE_LPAR )) +device_create_file (s, &dev_attr_idle_purr); } -if (cpu_has_feature (CPU_FTR_SPURR )) +if (cpu_has_feature (CPU_FTR_SPURR )) {device_create_file (s, &dev_attr_spurr); +if (firmware_has_feature (FW_FEATURE_LPAR )) +device_create_file (s, &dev_attr_idle_spurr); + }if (cpu_has_feature (CPU_FTR_DSCR ))device_create_file (s, &dev_attr_dscr);@@ -879,11 +921,17 @@ static int unregister_cpu_online(unsigned int cpu)if (cpu_has_feature (CPU_FTR_MMCRA ))device_remove_file (s, &dev_attr_mmcra); -if (cpu_has_feature (CPU_FTR_PURR )) +if (cpu_has_feature (CPU_FTR_PURR )) {device_remove_file (s, &dev_attr_purr); +if (firmware_has_feature (FW_FEATURE_LPAR )) +device_remove_file (s, &dev_attr_idle_purr); + } -if (cpu_has_feature (CPU_FTR_SPURR )) +if (cpu_has_feature (CPU_FTR_SPURR )) {device_remove_file (s, &dev_attr_spurr); +if (firmware_has_feature (FW_FEATURE_LPAR )) +device_remove_file (s, &dev_attr_idle_spurr); + }
Quoted 4 lines
arch/powerpc/kernel/sysfs.c static int unregister_cpu_online(unsigned int cpu) · line 937
Quoted 2 lines
arch/powerpc/kernel/sysfs.c +0
-0if (cpu_has_feature (CPU_FTR_DSCR ))device_remove_file (s, &dev_attr_dscr);The cpu register/unregister stuff here looks correct.
Thanks for reviewing the patch.
--
Thanks and Regards
gautham.
Re: [PATCH v2 4/5] powerpc/sysfs: Show idle_purr and idle_spurr for every CPU
Quoted 87 lines · Gautham R Shenoy
Quoted 17 lines · Nathan Lynch
Quoted 16 lines · Gautham R. Shenoy
arch/powerpc/kernel/sysfs.c +1
-0@@ -19,6 +19,7 @@#include <asm/smp.h> #include <asm/pmc.h> #include <asm/firmware.h> +#include <asm/idle.h> #include <asm/svm.h> #include "cacheinfo.h" @@ -733,6 +734,42 @@ static void create_svm_file(void) }#endif /* CONFIG_PPC_SVM */ Quoted 5 lines
arch/powerpc/kernel/sysfs.c static void create_svm_file(void) · line 739
Quoted 3 lines
arch/powerpc/kernel/sysfs.c +3
-0+static void read_idle_purr (void *val) +{ +u64 *ret = (u64 *)val;No cast from void* needed.
Will fix this. Thanks.
Quoted 40 lines
Quoted 24 lines
arch/powerpc/kernel/sysfs.c +22
-0+ + *ret =read_this_idle_purr (); +} + +static ssize_t idle_purr_show (struct device *dev, +struct device_attribute *attr,char *buf) +{ + struct cpu *cpu = container_of(dev, struct cpu, dev); +u64 val; + +smp_call_function_single (cpu->dev.id, read_idle_purr, &val,1 ); +return sprintf (buf,"%llx\n" , val); +} +static DEVICE_ATTR(idle_purr, 0400, idle_purr_show, NULL); + +static void read_idle_spurr (void *val) +{ +u64 *ret = (u64 *)val; + + *ret =read_this_idle_spurr (); +} +arch/powerpc/kernel/sysfs.c static void create_svm_file(void) · line 771
Quoted 10 lines
arch/powerpc/kernel/sysfs.c +10
-0+static ssize_t idle_spurr_show (struct device *dev, +struct device_attribute *attr,char *buf) +{ + struct cpu *cpu = container_of(dev, struct cpu, dev); +u64 val; + +smp_call_function_single (cpu->dev.id, read_idle_spurr, &val,1 ); +return sprintf (buf,"%llx\n" , val); +} +static DEVICE_ATTR(idle_spurr, 0400, idle_spurr_show, NULL);It's regrettable that we have to wake up potentially idle CPUs in order to derive correct idle statistics for them, but I suppose the main user (lparstat) of these interfaces already is causing this to happen by polling the existing per-cpu purr and spurr attributes.
Quoted 3 lines
So now lparstat will incur at minimum four syscalls and four IPIs per CPU per polling interval -- one for each of purr, spurr,
idle_purrandidle_spurr. Correct?Yes, it is unforunate that we will end up making four syscalls and generating IPI noise, and this is something that I discussed with Naveen and Kamalesh. We have the following two constraints:
1) These values of PURR and SPURR required are per-cpu. Hence putting them in lparcfg is not an option.
2) sysfs semantics encourages a single value per key, the key being the sysfs-file. Something like the following would have made far more sense.
cat
/sys/devices/system/cpu/cpuX/purr_spurr_accountingpurr:A
idle_purr:B
spurr:C
idle_spurr:D
Quoted 3 lines
There are some sysfs files which allow something like this. Eg:
/sys/devices/system/cpu/cpu0/cpufreq/stats/time_in_stateThoughts on any other alternatives?
Umm... procfs?
/me ducks
Quoted 5 lines
Quoted 5 lines
At some point it's going to make sense to batch sampling of remote CPUs' SPRs.
How did you mean this? It looks like we first need to provide a separate user interface, since with the existing sysfs interface providing separate files, I am not sure if we can batch such reads.
- Naveen
Re: [PATCH v2 4/5] powerpc/sysfs: Show idle_purr and idle_spurr for every CPU
Quoted 28 lines · Naveen N. Rao
Quoted 27 lines · Gautham R Shenoy
Quoted 8 lines · Nathan Lynch
It's regrettable that we have to wake up potentially idle CPUs in order to derive correct idle statistics for them, but I suppose the main user (lparstat) of these interfaces already is causing this to happen by polling the existing per-cpu purr and spurr attributes.
So now lparstat will incur at minimum four syscalls and four IPIs per CPU per polling interval -- one for each of purr, spurr,
idle_purrandidle_spurr. Correct?Yes, it is unforunate that we will end up making four syscalls and generating IPI noise, and this is something that I discussed with Naveen and Kamalesh. We have the following two constraints:
1) These values of PURR and SPURR required are per-cpu. Hence putting them in lparcfg is not an option.
2) sysfs semantics encourages a single value per key, the key being the sysfs-file. Something like the following would have made far more sense.
cat
/sys/devices/system/cpu/cpuX/purr_spurr_accountingpurr:A
idle_purr:B
spurr:C
idle_spurr:D
Quoted 7 lines
Quoted 4 lines
There are some sysfs files which allow something like this. Eg:
/sys/devices/system/cpu/cpu0/cpufreq/stats/time_in_stateThoughts on any other alternatives?
Umm... procfs?
/me ducks
I had wondered about perf events but I'm not sure that's any more suitable.
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
At some point it's going to make sense to batch sampling of remote CPUs' SPRs.
How did you mean this? It looks like we first need to provide a separate user interface, since with the existing sysfs interface providing separate files, I am not sure if we can batch such reads.
I mean in order to minimize IPI traffic something like: sample/calculate all of a CPU's purr, idle_purr, spurr, idle_spurr in a single IPI upon a read of any of the attributes, and cache the result for some time, so that the anticipated subsequent reads of the other attributes use the cached results instead of generating more IPIs.
That would keep the current sysfs interface at the cost of imposing a certain coarseness in the results.
Anyway, that's a mitigation that could be considered if the implementation in this patch is found to be too expensive in practice.
Re: [PATCH v2 4/5] powerpc/sysfs: Show idle_purr and idle_spurr for every CPU
Quoted 29 lines · Nathan Lynch
Quoted 28 lines · Naveen N. Rao
Quoted 27 lines · Gautham R Shenoy
Quoted 8 lines · Nathan Lynch
It's regrettable that we have to wake up potentially idle CPUs in order to derive correct idle statistics for them, but I suppose the main user (lparstat) of these interfaces already is causing this to happen by polling the existing per-cpu purr and spurr attributes.
So now lparstat will incur at minimum four syscalls and four IPIs per CPU per polling interval -- one for each of purr, spurr,
idle_purrandidle_spurr. Correct?Yes, it is unforunate that we will end up making four syscalls and generating IPI noise, and this is something that I discussed with Naveen and Kamalesh. We have the following two constraints:
1) These values of PURR and SPURR required are per-cpu. Hence putting them in lparcfg is not an option.
2) sysfs semantics encourages a single value per key, the key being the sysfs-file. Something like the following would have made far more sense.
cat
/sys/devices/system/cpu/cpuX/purr_spurr_accountingpurr:A
idle_purr:B
spurr:C
idle_spurr:D
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
There are some sysfs files which allow something like this. Eg:
/sys/devices/system/cpu/cpu0/cpufreq/stats/time_in_stateThoughts on any other alternatives?
Umm... procfs?
/me ducksI had wondered about perf events but I'm not sure that's any more suitable.
Yes, we considered that, but it looks like the event reads are not "batched" in any manner. So, the IPI overhead will be similar.
Quoted 6 lines
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
At some point it's going to make sense to batch sampling of remote CPUs' SPRs.
How did you mean this? It looks like we first need to provide a separate user interface, since with the existing sysfs interface providing separate files, I am not sure if we can batch such reads.
Quoted 8 lines
I mean in order to minimize IPI traffic something like: sample/calculate all of a CPU's purr,
idle_purr, spurr,idle_spurrin a single IPI upon a read of any of the attributes, and cache the result for some time, so that the anticipated subsequent reads of the other attributes use the cached results instead of generating more IPIs.That would keep the current sysfs interface at the cost of imposing a certain coarseness in the results.
Thanks for clarifying, that makes sense. Though we need to be careful in ensuring the sysfs semantics work as expected.
Quoted 3 lines
Anyway, that's a mitigation that could be considered if the implementation in this patch is found to be too expensive in practice.
That's a good point. We can optimize later if this turns out to be a problem in practice, if we end up using this approach.
- Naveen
PATCH v2 5/5 Documentation: Document sysfs interfaces purr, spurr, idle_purr, idle_spurr
In reply to Gautham R. ShenoyFrom: "Gautham R. Shenoy" <[email protected]>
Add documentation for the following sysfs interfaces:/sys/devices/system/cpu/cpuX/purr /sys/devices/system/cpu/cpuX/spurr /sys/devices/system/cpu/cpuX/idle_purr /sys/devices/system/cpu/cpuX/idle_spurr
--- Documentation/ABI/testing/sysfs-devices-system-cpu | 39 ++++++++++++++++++++++ 1 file changed, 39 insertions(+)
Documentation/ABI/testing/sysfs-devices-system-cpu +39 -0
@@ -580,3 +580,42 @@ Description: Secure Virtual Machine If 1, it means the system is using the Protected Execution Facility in POWER9 and newer processors. i.e., it is a Secure Virtual Machine. + +What: /sys/devices/system/cpu/cpuX/purr +Date: Apr 2005 +Contact: Linux for PowerPC mailing list <[email protected]> +Description: PURR ticks for this CPU since the system boot. + + The Processor Utilization Resources Register (PURR) is + a 64-bit counter which provides an estimate of the + resources used by the CPU thread. The contents of this + register increases monotonically. This sysfs interface + exposes the number of PURR ticks for cpuX. + +What: /sys/devices/system/cpu/cpuX/spurr +Date: Dec 2006 +Contact: Linux for PowerPC mailing list <[email protected]> +Description: SPURR ticks for this CPU since the system boot. + + The Scaled Processor Utilization Resources Register + (SPURR) is a 64-bit counter that provides a frequency + invariant estimate of the resources used by the CPU + thread. The contents of this register increases + monotonically. This sysfs interface exposes the number + of SPURR ticks for cpuX. + +What: /sys/devices/system/cpu/cpuX/idle_purr +Date: Nov 2019 +Contact: Linux for PowerPC mailing list <[email protected]> +Description: PURR ticks for cpuX when it was idle. + + This sysfs interface exposes the number of PURR ticks + for cpuX when it was idle. + +What: /sys/devices/system/cpu/cpuX/spurr +Date: Nov 2019 +Contact: Linux for PowerPC mailing list <[email protected]> +Description: SPURR ticks for cpuX when it was idle. + + This sysfs interface exposes the number of SPURR ticks + for cpuX when it was idle.
Signature · 1 line
1.9.4
Re: [PATCH v2 5/5] Documentation: Document sysfs interfaces purr, spurr, idle_purr, idle_spurr
Quoted 31 lines · Gautham R. Shenoy
Documentation/ABI/testing/sysfs-devices-system-cpu +23
-0@@ -580,3 +580,42 @@ Description: Secure Virtual Machine If 1, it means the system is using the Protected Execution Facility in POWER9 and newer processors. i.e., it is a Secure Virtual Machine. + +What: /sys/devices/system/cpu/cpuX/purr +Date: Apr 2005 +Contact: Linux for PowerPC mailing list <[email protected]> +Description: PURR ticks for this CPU since the system boot. + + The Processor Utilization Resources Register (PURR) is + a 64-bit counter which provides an estimate of the + resources used by the CPU thread. The contents of this + register increases monotonically. This sysfs interface + exposes the number of PURR ticks for cpuX. + +What: /sys/devices/system/cpu/cpuX/spurr +Date: Dec 2006 +Contact: Linux for PowerPC mailing list <[email protected]> +Description: SPURR ticks for this CPU since the system boot. + + The Scaled Processor Utilization Resources Register + (SPURR) is a 64-bit counter that provides a frequency + invariant estimate of the resources used by the CPU + thread. The contents of this register increases + monotonically. This sysfs interface exposes the number + of SPURR ticks for cpuX.
Documentation/ABI/testing/sysfs-devices-system-cpu Description: Secure Virtual Machine · line 615
Quoted 10 lines
Documentation/ABI/testing/sysfs-devices-system-cpu +10
-0+ +What: /sys/devices/system/cpu/cpuX/idle_purr +Date: Nov 2019 +Contact: Linux for PowerPC mailing list <[email protected]> +Description: PURR ticks for cpuX when it was idle. + + This sysfs interface exposes the number of PURR ticks + for cpuX when it was idle. + +What: /sys/devices/system/cpu/cpuX/spurr
Copy-paste error? This should be:
/sys/devices/system/cpu/cpuX/idle_spurr
Documentation/ABI/testing/sysfs-devices-system-cpu Description: Secure Virtual Machine · line 616
Quoted 1 line
Documentation/ABI/testing/sysfs-devices-system-cpu +1
-0+Date: Nov 2019
And I suppose Nov 2019 is no longer accurate.
Quoted 7 lines
Documentation/ABI/testing/sysfs-devices-system-cpu +5
-0+Contact: Linux for PowerPC mailing list <[email protected]> +Description: SPURR ticks for cpuX when it was idle. + + This sysfs interface exposes the number of SPURR ticks + for cpuX when it was idle.Signature · 1 line
1.9.4
Re: [PATCH v2 5/5] Documentation: Document sysfs interfaces purr, spurr, idle_purr, idle_spurr
Quoted 36 lines · Nathan Lynch
Quoted 35 lines · Gautham R. Shenoy
Documentation/ABI/testing/sysfs-devices-system-cpu +27
-0@@ -580,3 +580,42 @@ Description: Secure Virtual Machine If 1, it means the system is using the Protected Execution Facility in POWER9 and newer processors. i.e., it is a Secure Virtual Machine. + +What: /sys/devices/system/cpu/cpuX/purr +Date: Apr 2005 +Contact: Linux for PowerPC mailing list <[email protected]> +Description: PURR ticks for this CPU since the system boot. + + The Processor Utilization Resources Register (PURR) is + a 64-bit counter which provides an estimate of the + resources used by the CPU thread. The contents of this + register increases monotonically. This sysfs interface + exposes the number of PURR ticks for cpuX. + +What: /sys/devices/system/cpu/cpuX/spurr +Date: Dec 2006 +Contact: Linux for PowerPC mailing list <[email protected]> +Description: SPURR ticks for this CPU since the system boot. + + The Scaled Processor Utilization Resources Register + (SPURR) is a 64-bit counter that provides a frequency + invariant estimate of the resources used by the CPU + thread. The contents of this register increases + monotonically. This sysfs interface exposes the number + of SPURR ticks for cpuX. + +What: /sys/devices/system/cpu/cpuX/idle_purr +Date: Nov 2019 +Contact: Linux for PowerPC mailing list <[email protected]>
Quoted 8 lines
Documentation/ABI/testing/sysfs-devices-system-cpu Description: Secure Virtual Machine · line 615
Quoted 6 lines
Documentation/ABI/testing/sysfs-devices-system-cpu +6
-0+Description: PURR ticks for cpuX when it was idle. + + This sysfs interface exposes the number of PURR ticks + for cpuX when it was idle. + +What: /sys/devices/system/cpu/cpuX/spurrCopy-paste error? This should be:
Yes, this should have been idle_spurr. Will fix it in the next version.
Quoted 6 lines
/sys/devices/system/cpu/cpuX/idle_spurrDocumentation/ABI/testing/sysfs-devices-system-cpu Description: Secure Virtual Machine · line 616
Quoted 1 line
Documentation/ABI/testing/sysfs-devices-system-cpu +1
-0+Date: Nov 2019And I suppose Nov 2019 is no longer accurate.
My bad. I will resend this with the updated date.
Quoted 9 lines
Quoted 9 lines
Documentation/ABI/testing/sysfs-devices-system-cpu +5
-0+Contact: Linux for PowerPC mailing list <[email protected]> +Description: SPURR ticks for cpuX when it was idle. + + This sysfs interface exposes the number of SPURR ticks + for cpuX when it was idle.Signature · 1 line
1.9.4
Quoted 1 line · Gautham R. Shenoy
From: "Gautham R. Shenoy" <[email protected]>
[...]
Quoted 9 lines
Gautham R. Shenoy (5):
powerpc: Move idle_loop_prolog()/epilog() functions to header file powerpc/idle: Add accessor function to always read latest idle PURR powerpc/pseries: Account for SPURR ticks on idle CPUs powerpc/sysfs: Showidle_purrandidle_spurrfor every CPU
Documentation: Document sysfs interfaces purr, spurr,idle_purr,
idle_spurr
Quoted 8 lines
Documentation/ABI/testing/sysfs-devices-system-cpu | 39 ++++++++++ arch/powerpc/include/asm/idle.h | 88 ++++++++++++++++++++++ arch/powerpc/kernel/sysfs.c | 54 ++++++++++++- arch/powerpc/platforms/pseries/setup.c | 8 +- drivers/cpuidle/cpuidle-pseries.c | 39 ++-------- 5 files changed, 191 insertions(+), 37 deletions(-) create mode 100644 arch/powerpc/include/asm/idle.h
For the whole series:
Signature · 1 line
Kamalesh