Skip to contentSkip to conversation

powernv:stop: Use psscr_val,mask provided by firmware

PATCH Superseded Gautham R. Shenoy 2 participants 11 messages Dec 9, 2016 – Dec 14, 2016

Subsystems (3)

v4 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v4 cover powernv:stop: Use psscr_val,mask provided by firmware

From: "Gautham R. Shenoy" <[email protected]>

This is the fourth iteration of the patchset to use the psscr_val and psscr_mask provided by the firmware for each of the stop states.

The previous version can be found here:

[v3]: https://lkml.org/lkml/2016/11/10/37
[v2]: https://lkml.org/lkml/2016/10/27/143
[v1]: https://lkml.org/lkml/2016/9/29/45

This version fixes some of the coding style issues pointed out by Michael Ellerman in v3. This version also documents the device-tree bindings defining the properties under the @power-mgt node in the device tree describing the idle states for Linux running on baremetal POWER servers.

Synopsis
==========
In the current implementation, the code for ISA v3.0 stop implementation has a couple of shortcomings.

a) The code hand-codes the values for ESL,EC,TR,MTL bits of PSSCR and uses only the RL field from the firmware. While this is not incorrect, since the hand-coded values are legitimate, it is not a very flexible design since the firmware has the capability to communicate these values via the "ibm,cpu-idle-state-psscr" and "ibm,cpu-idle-state-psscr-mask" properties. In case where the firmware provides values for these fields that is different from the hand-coded values, the current code will not work as intended.

b) Due to issue a), the current code assumes that ESL=EC=1 for all the stop states and hence the wakeup from the stop instruction will happen at 0x100, the system-reset vector. However, the ISA v3.0 allows the ESL=EC=0 behaviour where the corresponding stop-state loses no state and wakes up from the subsequent instruction. The current code doesn't handle this case.

This patch series addresses these issues.

The first patch in the series renames the existing IDLE_STATE_ENTER_SEQ macro to IDLE_STATE_ENTER_SEQ_NORET. It reuses the name IDLE_STATE_ENTER_SEQ for entering into stop-states which wake up at the subsequent instruction.

The second patch adds a helper function in cpuidle-powernv.c for initializing entries of the powernv_states[] table that is passed to the cpu-idle core. This eliminates some of the code duplication in the function that discovers and initializes the stop states.

The third patch in the series fixes issues a) and b) by ensuring that the psscr-value and the psscr-mask provided by the firmware are what will be used to set a particular stop state. It also adds support for handling wake-up from stop states which were entered with ESL=EC=0.

The third patch also handles the older firmware which sets only the Requested Level (RL) field in the psscr and psscr-mask exposed in the device tree. In the presence of such older firmware, this patch will set the default sane values for for remaining PSSCR fields (i.e PSLL, MTL, ESL, EC, and TR).

The fourth patch provides the documentation for the device-tree bindings describing the idle state properties under the @power-mgt node in the device-tree.

The skiboot patch populates all the relevant fields in the PSSCR values and the mask for all the stop states can be found here:
https://lists.ozlabs.org/pipermail/skiboot/2016-September/004869.html

The patches are based on top of
git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git fixes

Gautham R. Shenoy (4):
powernv:idle: Add IDLE_STATE_ENTER_SEQ_NORET macro cpuidle:powernv: Add helper function to populate powernv idle states.
powernv: Pass PSSCR value and mask to power9_idle_stop Documentation:powerpc: Add device-tree bindings for power-mgt

 .../devicetree/bindings/powerpc/opal/power-mgt.txt | 123 +++++++++++++++++++++
 arch/powerpc/include/asm/cpuidle.h                 |  46 +++++++-
 arch/powerpc/include/asm/processor.h               |   3 +-
 arch/powerpc/kernel/exceptions-64s.S               |   6 +-
 arch/powerpc/kernel/idle_book3s.S                  |  41 ++++---
 arch/powerpc/platforms/powernv/idle.c              |  81 +++++++++++---
 arch/powerpc/platforms/powernv/powernv.h           |   3 +-
 arch/powerpc/platforms/powernv/smp.c               |  14 ++-
 drivers/cpuidle/cpuidle-powernv.c                  | 113 ++++++++++++-------
 include/linux/cpuidle.h                            |   1 +
 10 files changed, 348 insertions(+), 83 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/powerpc/opal/power-mgt.txt
Signature · 6 lines
1.9.4

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#1 lore raw

PATCH v4 1/4 powernv:idle: Add IDLE_STATE_ENTER_SEQ_NORET macro

From: "Gautham R. Shenoy" <[email protected]>

Currently all the low-power idle states are expected to wake up at reset vector 0x100. Which is why the macro IDLE_STATE_ENTER_SEQ that puts the CPU to an idle state and never returns.

On ISA_300, when the ESL and EC bits in the PSSCR are zero, the CPU is expected to wake up at the next instruction of the idle instruction.

This patch adds a new macro named IDLE_STATE_ENTER_SEQ_NORET for the no-return variant and reuses the name IDLE_STATE_ENTER_SEQ for a variant that allows resuming operation at the instruction next to the idle-instruction.

---
 arch/powerpc/include/asm/cpuidle.h   |  5 ++++-
 arch/powerpc/kernel/exceptions-64s.S |  6 +++---
 arch/powerpc/kernel/idle_book3s.S    | 10 +++++-----
 3 files changed, 12 insertions(+), 9 deletions(-)
arch/powerpc/include/asm/cpuidle.h +4 -1
index 3919332..0a3255b 100644
@@ -21,7 +21,7 @@
 
 /* Idle state entry routines */
 #ifdef	CONFIG_PPC_P7_NAP
-#define	IDLE_STATE_ENTER_SEQ(IDLE_INST)				\
+#define IDLE_STATE_ENTER_SEQ(IDLE_INST)                         \
 	/* Magic NAP/SLEEP/WINKLE mode enter sequence */	\
 	std	r0,0(r1);					\
 	ptesync;						\
@@ -29,6 +29,9 @@
 1:	cmpd	cr0,r0,r0;					\
 	bne	1b;						\
 	IDLE_INST;						\
+
+#define	IDLE_STATE_ENTER_SEQ_NORET(IDLE_INST)			\
+	IDLE_STATE_ENTER_SEQ(IDLE_INST)                         \
 	b	.
 #endif /* CONFIG_PPC_P7_NAP */
 
arch/powerpc/kernel/exceptions-64s.S +3 -3
index 1ba82ea..7aa8afc 100644
@@ -381,12 +381,12 @@ EXC_COMMON_BEGIN(machine_check_handle_early)
 	lbz	r3,PACA_THREAD_IDLE_STATE(r13)
 	cmpwi	r3,PNV_THREAD_NAP
 	bgt	10f
-	IDLE_STATE_ENTER_SEQ(PPC_NAP)
+	IDLE_STATE_ENTER_SEQ_NORET(PPC_NAP)
 	/* No return */
 10:
 	cmpwi	r3,PNV_THREAD_SLEEP
 	bgt	2f
-	IDLE_STATE_ENTER_SEQ(PPC_SLEEP)
+	IDLE_STATE_ENTER_SEQ_NORET(PPC_SLEEP)
 	/* No return */
 
 2:
@@ -400,7 +400,7 @@ EXC_COMMON_BEGIN(machine_check_handle_early)
 	 */
 	ori	r13,r13,1
 	SET_PACA(r13)
-	IDLE_STATE_ENTER_SEQ(PPC_WINKLE)
+	IDLE_STATE_ENTER_SEQ_NORET(PPC_WINKLE)
 	/* No return */
 4:
 #endif
arch/powerpc/kernel/idle_book3s.S +5 -5
index 72dac0b..be90e2f 100644
@@ -205,7 +205,7 @@ pnv_enter_arch207_idle_mode:
 	stb	r3,PACA_THREAD_IDLE_STATE(r13)
 	cmpwi	cr3,r3,PNV_THREAD_SLEEP
 	bge	cr3,2f
-	IDLE_STATE_ENTER_SEQ(PPC_NAP)
+	IDLE_STATE_ENTER_SEQ_NORET(PPC_NAP)
 	/* No return */
 2:
 	/* Sleep or winkle */
@@ -239,7 +239,7 @@ pnv_fastsleep_workaround_at_entry:
 
 common_enter: /* common code for all the threads entering sleep or winkle */
 	bgt	cr3,enter_winkle
-	IDLE_STATE_ENTER_SEQ(PPC_SLEEP)
+	IDLE_STATE_ENTER_SEQ_NORET(PPC_SLEEP)
 
 fastsleep_workaround_at_entry:
 	ori	r15,r15,PNV_CORE_IDLE_LOCK_BIT
@@ -261,7 +261,7 @@ fastsleep_workaround_at_entry:
 enter_winkle:
 	bl	save_sprs_to_stack
 
-	IDLE_STATE_ENTER_SEQ(PPC_WINKLE)
+	IDLE_STATE_ENTER_SEQ_NORET(PPC_WINKLE)
 
 /*
  * r3 - requested stop state
@@ -280,7 +280,7 @@ power_enter_stop:
 	ld	r4,ADDROFF(pnv_first_deep_stop_state)(r5)
 	cmpd	r3,r4
 	bge	2f
-	IDLE_STATE_ENTER_SEQ(PPC_STOP)
+	IDLE_STATE_ENTER_SEQ_NORET(PPC_STOP)
 2:
 /*
  * Entering deep idle state.
@@ -302,7 +302,7 @@ lwarx_loop_stop:
 
 	bl	save_sprs_to_stack
 
-	IDLE_STATE_ENTER_SEQ(PPC_STOP)
+	IDLE_STATE_ENTER_SEQ_NORET(PPC_STOP)
 
 _GLOBAL(power7_idle)
 	/* Now check if user or arch enabled NAP mode */
Signature · 6 lines
1.9.4

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#2 lore raw

Re: [PATCH v4 1/4] powernv:idle: Add IDLE_STATE_ENTER_SEQ_NORET macro

Quoted 36 lines · Gautham R. Shenoy

From: "Gautham R. Shenoy" <[email protected]>

Currently all the low-power idle states are expected to wake up at reset vector 0x100. Which is why the macro IDLE_STATE_ENTER_SEQ that puts the CPU to an idle state and never returns.

On ISA_300, when the ESL and EC bits in the PSSCR are zero, the CPU is expected to wake up at the next instruction of the idle instruction.

This patch adds a new macro named IDLE_STATE_ENTER_SEQ_NORET for the no-return variant and reuses the name IDLE_STATE_ENTER_SEQ for a variant that allows resuming operation at the instruction next to the idle-instruction.

---
 arch/powerpc/include/asm/cpuidle.h   |  5 ++++-
 arch/powerpc/kernel/exceptions-64s.S |  6 +++---
 arch/powerpc/kernel/idle_book3s.S    | 10 +++++-----
 3 files changed, 12 insertions(+), 9 deletions(-)
arch/powerpc/include/asm/cpuidle.h +1 -1
index 3919332..0a3255b 100644
@@ -21,7 +21,7 @@
 
 /* Idle state entry routines */
 #ifdef	CONFIG_PPC_P7_NAP
-#define	IDLE_STATE_ENTER_SEQ(IDLE_INST)				\
+#define IDLE_STATE_ENTER_SEQ(IDLE_INST)                         \
 	/* Magic NAP/SLEEP/WINKLE mode enter sequence */	\
 	std	r0,0(r1);					\
 	ptesync;						\

arch/powerpc/include/asm/cpuidle.h line 32

Quoted 4 lines
arch/powerpc/include/asm/cpuidle.h +1 -0
@@ -29,6 +29,9 @@
 1:	cmpd	cr0,r0,r0;					\
 	bne	1b;						\
 	IDLE_INST;						\
+

Is the power saving magic sequence the same as before for power 9 as well?

Balbir Singh

#3 lore raw

Re: [PATCH v4 1/4] powernv:idle: Add IDLE_STATE_ENTER_SEQ_NORET macro

Hi Balbir,

Quoted 10 lines · Balbir Singh
Quoted 7 lines · Gautham R. Shenoy

From: "Gautham R. Shenoy" <[email protected]>

arch/powerpc/include/asm/cpuidle.h +0 -0
index 3919332..0a3255b 100644
@@ -21,7 +21,7 @@
 
Quoted 15 lines

arch/powerpc/include/asm/cpuidle.h line 32

Quoted 12 lines
arch/powerpc/include/asm/cpuidle.h +2 -1
 /* Idle state entry routines */
 #ifdef	CONFIG_PPC_P7_NAP
-#define	IDLE_STATE_ENTER_SEQ(IDLE_INST)				\
+#define IDLE_STATE_ENTER_SEQ(IDLE_INST)                         \
 	/* Magic NAP/SLEEP/WINKLE mode enter sequence */	\
 	std	r0,0(r1);					\
 	ptesync;						\
@@ -29,6 +29,9 @@
 1:	cmpd	cr0,r0,r0;					\
 	bne	1b;						\
 	IDLE_INST;						\
+

Is the power saving magic sequence the same as before for power 9 as well?

Yes, this is the same magic sequence for POWER9.

--
Thanks and Regards
gautham.

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw

PATCH v4 2/4 cpuidle:powernv: Add helper function to populate powernv idle states.

In reply to Gautham R. Shenoy

From: "Gautham R. Shenoy" <[email protected]>

In the current code for powernv_add_idle_states, there is a lot of code duplication while initializing an idle state in powernv_states table.

Add an inline helper function to populate the powernv_states[] table for a given idle state. Invoke this for populating the "Nap", "Fastsleep" and the stop states in powernv_add_idle_states.

---
 drivers/cpuidle/cpuidle-powernv.c | 85 ++++++++++++++++++++++-----------------
 include/linux/cpuidle.h           |  1 +
 2 files changed, 50 insertions(+), 36 deletions(-)
drivers/cpuidle/cpuidle-powernv.c +49 -36
index 7fe442c..db18af1 100644
@@ -167,6 +167,24 @@ static int powernv_cpuidle_driver_init(void)
 	return 0;
 }
 
+static inline void add_powernv_state(int index, const char *name,
+				     unsigned int flags,
+				     int (*idle_fn)(struct cpuidle_device *,
+						    struct cpuidle_driver *,
+						    int),
+				     unsigned int target_residency,
+				     unsigned int exit_latency,
+				     u64 psscr_val)
+{
+	strlcpy(powernv_states[index].name, name, CPUIDLE_NAME_LEN);
+	strlcpy(powernv_states[index].desc, name, CPUIDLE_NAME_LEN);
+	powernv_states[index].flags = flags;
+	powernv_states[index].target_residency = target_residency;
+	powernv_states[index].exit_latency = exit_latency;
+	powernv_states[index].enter = idle_fn;
+	stop_psscr_table[index] = psscr_val;
+}
+
 static int powernv_add_idle_states(void)
 {
 	struct device_node *power_mgt;
@@ -236,6 +254,7 @@ static int powernv_add_idle_states(void)
 		"ibm,cpu-idle-state-residency-ns", residency_ns, dt_idle_states);
 
 	for (i = 0; i < dt_idle_states; i++) {
+		unsigned int exit_latency, target_residency;
 		/*
 		 * If an idle state has exit latency beyond
 		 * POWERNV_THRESHOLD_LATENCY_NS then don't use it
@@ -243,28 +262,33 @@ static int powernv_add_idle_states(void)
 		 */
 		if (latency_ns[i] > POWERNV_THRESHOLD_LATENCY_NS)
 			continue;
+		/*
+		 * Firmware passes residency and latency values in ns.
+		 * cpuidle expects it in us.
+		 */
+		exit_latency = ((unsigned int)latency_ns[i]) / 1000;
+		if (!rc)
+			target_residency = residency_ns[i] / 1000;
+		else
+			target_residency = 0;
 
 		/*
-		 * Cpuidle accepts exit_latency and target_residency in us.
-		 * Use default target_residency values if f/w does not expose it.
+		 * For nap and fastsleep, use default target_residency
+		 * values if f/w does not expose it.
 		 */
 		if (flags[i] & OPAL_PM_NAP_ENABLED) {
+			if (!rc)
+				target_residency = 100;
 			/* Add NAP state */
-			strcpy(powernv_states[nr_idle_states].name, "Nap");
-			strcpy(powernv_states[nr_idle_states].desc, "Nap");
-			powernv_states[nr_idle_states].flags = 0;
-			powernv_states[nr_idle_states].target_residency = 100;
-			powernv_states[nr_idle_states].enter = nap_loop;
+			add_powernv_state(nr_idle_states, "Nap",
+					  CPUIDLE_FLAG_NONE, nap_loop,
+					  target_residency, exit_latency, 0);
 		} else if ((flags[i] & OPAL_PM_STOP_INST_FAST) &&
 				!(flags[i] & OPAL_PM_TIMEBASE_STOP)) {
-			strncpy(powernv_states[nr_idle_states].name,
-				names[i], CPUIDLE_NAME_LEN);
-			strncpy(powernv_states[nr_idle_states].desc,
-				names[i], CPUIDLE_NAME_LEN);
-			powernv_states[nr_idle_states].flags = 0;
-
-			powernv_states[nr_idle_states].enter = stop_loop;
-			stop_psscr_table[nr_idle_states] = psscr_val[i];
+			add_powernv_state(nr_idle_states, names[i],
+					  CPUIDLE_FLAG_NONE, stop_loop,
+					  target_residency, exit_latency,
+					  psscr_val[i]);
 		}
 
 		/*
@@ -274,32 +298,21 @@ static int powernv_add_idle_states(void)
 #ifdef CONFIG_TICK_ONESHOT
 		if (flags[i] & OPAL_PM_SLEEP_ENABLED ||
 			flags[i] & OPAL_PM_SLEEP_ENABLED_ER1) {
+			if (!rc)
+				target_residency = 300000;
 			/* Add FASTSLEEP state */
-			strcpy(powernv_states[nr_idle_states].name, "FastSleep");
-			strcpy(powernv_states[nr_idle_states].desc, "FastSleep");
-			powernv_states[nr_idle_states].flags = CPUIDLE_FLAG_TIMER_STOP;
-			powernv_states[nr_idle_states].target_residency = 300000;
-			powernv_states[nr_idle_states].enter = fastsleep_loop;
+			add_powernv_state(nr_idle_states, "FastSleep",
+					  CPUIDLE_FLAG_TIMER_STOP,
+					  fastsleep_loop,
+					  target_residency, exit_latency, 0);
 		} else if ((flags[i] & OPAL_PM_STOP_INST_DEEP) &&
 				(flags[i] & OPAL_PM_TIMEBASE_STOP)) {
-			strncpy(powernv_states[nr_idle_states].name,
-				names[i], CPUIDLE_NAME_LEN);
-			strncpy(powernv_states[nr_idle_states].desc,
-				names[i], CPUIDLE_NAME_LEN);
-
-			powernv_states[nr_idle_states].flags = CPUIDLE_FLAG_TIMER_STOP;
-			powernv_states[nr_idle_states].enter = stop_loop;
-			stop_psscr_table[nr_idle_states] = psscr_val[i];
+			add_powernv_state(nr_idle_states, names[i],
+					  CPUIDLE_FLAG_TIMER_STOP, stop_loop,
+					  target_residency, exit_latency,
+					  psscr_val[i]);
 		}
 #endif
-		powernv_states[nr_idle_states].exit_latency =
-				((unsigned int)latency_ns[i]) / 1000;
-
-		if (!rc) {
-			powernv_states[nr_idle_states].target_residency =
-				((unsigned int)residency_ns[i]) / 1000;
-		}
-
 		nr_idle_states++;
 	}
 out:
include/linux/cpuidle.h +1 -0
index bb31373..c4e10f8 100644
@@ -62,6 +62,7 @@ struct cpuidle_state {
 };
 
 /* Idle State Flags */
+#define CPUIDLE_FLAG_NONE       (0x00)
 #define CPUIDLE_FLAG_COUPLED	(0x02) /* state applies to multiple cpus */
 #define CPUIDLE_FLAG_TIMER_STOP (0x04)  /* timer is stopped on this state */
 
Signature · 1 line
1.9.4
#5 lore raw

Re: [PATCH v4 2/4] cpuidle:powernv: Add helper function to populate powernv idle states.

Quoted 24 lines · Gautham R. Shenoy

From: "Gautham R. Shenoy" <[email protected]>

In the current code for powernv_add_idle_states, there is a lot of code duplication while initializing an idle state in powernv_states table.

Add an inline helper function to populate the powernv_states[] table for a given idle state. Invoke this for populating the "Nap", "Fastsleep" and the stop states in powernv_add_idle_states.

---
 drivers/cpuidle/cpuidle-powernv.c | 85 ++++++++++++++++++++++-----------------
 include/linux/cpuidle.h           |  1 +
 2 files changed, 50 insertions(+), 36 deletions(-)
drivers/cpuidle/cpuidle-powernv.c +1 -0
index 7fe442c..db18af1 100644
@@ -167,6 +167,24 @@ static int powernv_cpuidle_driver_init(void)
 	return 0;
 }
 
+static inline void add_powernv_state(int index, const char *name,

drivers/cpuidle/cpuidle-powernv.c static int powernv_cpuidle_driver_init(void) · line 180

Quoted 10 lines
drivers/cpuidle/cpuidle-powernv.c +10 -0
+				     unsigned int flags,
+				     int (*idle_fn)(struct cpuidle_device *,
+						    struct cpuidle_driver *,
+						    int),
+				     unsigned int target_residency,
+				     unsigned int exit_latency,
+				     u64 psscr_val)
+{
+	strlcpy(powernv_states[index].name, name, CPUIDLE_NAME_LEN);
+	strlcpy(powernv_states[index].desc, name, CPUIDLE_NAME_LEN);

Do name and desc ever diverge?

drivers/cpuidle/cpuidle-powernv.c static int powernv_cpuidle_driver_init(void) · line 184

Quoted 4 lines
drivers/cpuidle/cpuidle-powernv.c +4 -0
+	powernv_states[index].flags = flags;
+	powernv_states[index].target_residency = target_residency;
+	powernv_states[index].exit_latency = exit_latency;
+	powernv_states[index].enter = idle_fn;

Why not call it idle_fn instead of enter?

Quoted 15 lines
drivers/cpuidle/cpuidle-powernv.c +4 -0
+	stop_psscr_table[index] = psscr_val;
+}
+
 static int powernv_add_idle_states(void)
 {
 	struct device_node *power_mgt;
@@ -236,6 +254,7 @@ static int powernv_add_idle_states(void)
 		"ibm,cpu-idle-state-residency-ns", residency_ns, dt_idle_states);
 
 	for (i = 0; i < dt_idle_states; i++) {
+		unsigned int exit_latency, target_residency;
 		/*
 		 * If an idle state has exit latency beyond
 		 * POWERNV_THRESHOLD_LATENCY_NS then don't use it

drivers/cpuidle/cpuidle-powernv.c static int powernv_add_idle_states(void) · line 263

Quoted 2 lines
drivers/cpuidle/cpuidle-powernv.c +0 -0
@@ -243,28 +262,33 @@ static int powernv_add_idle_states(void)
 		 */
 		if (latency_ns[i] > POWERNV_THRESHOLD_LATENCY_NS)

Ideally this should be called POWERNV_MAX_THRESHOLD_LATENCY_NS then

drivers/cpuidle/cpuidle-powernv.c static int powernv_add_idle_states(void) · line 273

Quoted 10 lines
drivers/cpuidle/cpuidle-powernv.c +9 -0
 			continue;
+		/*
+		 * Firmware passes residency and latency values in ns.
+		 * cpuidle expects it in us.
+		 */
+		exit_latency = ((unsigned int)latency_ns[i]) / 1000;
+		if (!rc)
+			target_residency = residency_ns[i] / 1000;
+		else
+			target_residency = 0;

Where do we get rc from? what does target_residency = 0 mean? Balbir Singh
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#6 lore raw

Re: [PATCH v4 2/4] cpuidle:powernv: Add helper function to populate powernv idle states.

Hi Balbir,

Quoted 26 lines · Balbir Singh
Quoted 23 lines · Gautham R. Shenoy

From: "Gautham R. Shenoy" <[email protected]>

In the current code for powernv_add_idle_states, there is a lot of code duplication while initializing an idle state in powernv_states table.

Add an inline helper function to populate the powernv_states[] table for a given idle state. Invoke this for populating the "Nap", "Fastsleep" and the stop states in powernv_add_idle_states.

---
 drivers/cpuidle/cpuidle-powernv.c | 85 ++++++++++++++++++++++-----------------
 include/linux/cpuidle.h           |  1 +
 2 files changed, 50 insertions(+), 36 deletions(-)
drivers/cpuidle/cpuidle-powernv.c +0 -0
index 7fe442c..db18af1 100644
@@ -167,6 +167,24 @@ static int powernv_cpuidle_driver_init(void)
 	return 0;
 }
 
Quoted 13 lines

drivers/cpuidle/cpuidle-powernv.c static int powernv_cpuidle_driver_init(void) · line 180

Quoted 11 lines
drivers/cpuidle/cpuidle-powernv.c +11 -0
+static inline void add_powernv_state(int index, const char *name,
+				     unsigned int flags,
+				     int (*idle_fn)(struct cpuidle_device *,
+						    struct cpuidle_driver *,
+						    int),
+				     unsigned int target_residency,
+				     unsigned int exit_latency,
+				     u64 psscr_val)
+{
+	strlcpy(powernv_states[index].name, name, CPUIDLE_NAME_LEN);
+	strlcpy(powernv_states[index].desc, name, CPUIDLE_NAME_LEN);

Do name and desc ever diverge?

On some other architectures, like kirkwood (see drivers/cpuidle/cpuidle-kirkwood.c) they do. "desc" field is used to provide a more descriptive information regarding the idle state.

On POWER, the names were self-explanatory. So, we have desc same as the name.

Quoted 7 lines

drivers/cpuidle/cpuidle-powernv.c static int powernv_cpuidle_driver_init(void) · line 184

Quoted 5 lines
drivers/cpuidle/cpuidle-powernv.c +4 -0
+	powernv_states[index].flags = flags;
+	powernv_states[index].target_residency = target_residency;
+	powernv_states[index].exit_latency = exit_latency;
+	powernv_states[index].enter = idle_fn;

Why not call it idle_fn instead of enter?

"enter" is a field name in the generic cpuidle_state structure and powernv_states[] is an instance of that structure.

Quoted 10 lines
Quoted 10 lines
drivers/cpuidle/cpuidle-powernv.c +3 -0
+	stop_psscr_table[index] = psscr_val;
+}
+
 static int powernv_add_idle_states(void)
 {
 	struct device_node *power_mgt;
@@ -236,6 +254,7 @@ static int powernv_add_idle_states(void)
 		"ibm,cpu-idle-state-residency-ns", residency_ns, dt_idle_states);
 
Quoted 10 lines

drivers/cpuidle/cpuidle-powernv.c static int powernv_add_idle_states(void) · line 263

Quoted 8 lines
drivers/cpuidle/cpuidle-powernv.c +1 -0
 	for (i = 0; i < dt_idle_states; i++) {
+		unsigned int exit_latency, target_residency;
 		/*
 		 * If an idle state has exit latency beyond
 		 * POWERNV_THRESHOLD_LATENCY_NS then don't use it
@@ -243,28 +262,33 @@ static int powernv_add_idle_states(void)
 		 */
 		if (latency_ns[i] > POWERNV_THRESHOLD_LATENCY_NS)

Ideally this should be called POWERNV_MAX_THRESHOLD_LATENCY_NS then

Yes, it can be called that. But then again, we're only interested in the upper threshold in this code. I will add a comment near the macro definition.

Quoted 13 lines

drivers/cpuidle/cpuidle-powernv.c static int powernv_add_idle_states(void) · line 273

Quoted 11 lines
drivers/cpuidle/cpuidle-powernv.c +9 -0
 			continue;
+		/*
+		 * Firmware passes residency and latency values in ns.
+		 * cpuidle expects it in us.
+		 */
+		exit_latency = ((unsigned int)latency_ns[i]) / 1000;
+		if (!rc)
+			target_residency = residency_ns[i] / 1000;
+		else
+			target_residency = 0;

Where do we get rc from? what does target_residency = 0 mean?

The rc value comes from the
of_property_read_u32_array(power_mgt,

		"ibm,cpu-idle-state-residency-ns", residency_ns,
		dt_idle_states);

just before the for-loop. This tells us whether the firmware has populated the residency information for the idle state or not.

rc != 0 indicates that the firmware has not populated the value.

Since the governor will pick the first idle state whose target_residency matches the predicted residency, setting target_residency = 0 implies that if any stop state is selected at all, it is the earliest state.

Quoted 2 lines

Balbir Singh

#7 lore raw

PATCH v4 3/4 powernv: Pass PSSCR value and mask to power9_idle_stop

In reply to Gautham R. Shenoy

From: "Gautham R. Shenoy" <[email protected]>

The power9_idle_stop method currently takes only the requested stop level as a parameter and picks up the rest of the PSSCR bits from a hand-coded macro. This is not a very flexible design, especially when the firmware has the capability to communicate the psscr value and the mask associated with a particular stop state via device tree.

This patch modifies the power9_idle_stop API to take as parameters the PSSCR value and the PSSCR mask corresponding to the stop state that needs to be set. These PSSCR value and mask are respectively obtained by parsing the "ibm,cpu-idle-state-psscr" and "ibm,cpu-idle-state-psscr-mask" fields from the device tree.

In addition to this, the patch adds support for handling stop states for which ESL and EC bits in the PSSCR are zero. As per the architecture, a wakeup from these stop states resumes execution from the subsequent instruction as opposed to waking up at the System Vector.

The older firmware sets only the Requested Level (RL) field in the psscr and psscr-mask exposed in the device tree. For older firmware where psscr-mask=0xf, this patch will set the default sane values that the set for for remaining PSSCR fields (i.e PSLL, MTL, ESL, EC, and TR).

This skiboot patch that exports fully populated PSSCR values and the mask for all the stop states can be found here:
https://lists.ozlabs.org/pipermail/skiboot/2016-September/004869.html

---
 arch/powerpc/include/asm/cpuidle.h       | 41 ++++++++++++++++
 arch/powerpc/include/asm/processor.h     |  3 +-
 arch/powerpc/kernel/idle_book3s.S        | 31 +++++++-----
 arch/powerpc/platforms/powernv/idle.c    | 81 ++++++++++++++++++++++++++------
 arch/powerpc/platforms/powernv/powernv.h |  3 +-
 arch/powerpc/platforms/powernv/smp.c     | 14 +++---
 drivers/cpuidle/cpuidle-powernv.c        | 40 +++++++++++-----
 7 files changed, 169 insertions(+), 44 deletions(-)
arch/powerpc/include/asm/cpuidle.h +41 -0
index 0a3255b..fa0b6c0 100644
@@ -10,11 +10,52 @@
 #define PNV_CORE_IDLE_LOCK_BIT          0x100
 #define PNV_CORE_IDLE_THREAD_BITS       0x0FF
 
+/*
+ * ============================ NOTE =================================
+ * The older firmware populates only the RL field in the psscr_val and
+ * sets the psscr_mask to 0xf. On such a firmware, the kernel sets the
+ * remaining PSSCR fields to default values as follows:
+ *
+ * - ESL and EC bits are to 1. So wakeup from any stop state will be
+ *   at vector 0x100.
+ *
+ * - MTL and PSLL are set to the maximum allowed value as per the ISA,
+ *    i.e. 15.
+ *
+ * - The Transition Rate, TR is set to the Maximum value 3.
+ */
+#define PSSCR_HV_DEFAULT_VAL    (PSSCR_ESL | PSSCR_EC |		    \
+				PSSCR_PSLL_MASK | PSSCR_TR_MASK |   \
+				PSSCR_MTL_MASK)
+
+#define PSSCR_HV_DEFAULT_MASK   (PSSCR_ESL | PSSCR_EC |		    \
+				PSSCR_PSLL_MASK | PSSCR_TR_MASK |   \
+				PSSCR_MTL_MASK | PSSCR_RL_MASK)
+
 #ifndef __ASSEMBLY__
 extern u32 pnv_fastsleep_workaround_at_entry[];
 extern u32 pnv_fastsleep_workaround_at_exit[];
 
 extern u64 pnv_first_deep_stop_state;
+
+static inline u64 compute_psscr_val(u64 psscr_val, u64 psscr_mask)
+{
+	/*
+	 * psscr_mask == 0xf indicates an older firmware.
+	 * Set remaining fields of psscr to the default values.
+	 * See NOTE above definition of PSSCR_HV_DEFAULT_VAL
+	 */
+	if (psscr_mask == 0xf)
+		return psscr_val | PSSCR_HV_DEFAULT_VAL;
+	return psscr_val;
+}
+
+static inline u64 compute_psscr_mask(u64 psscr_mask)
+{
+	if (psscr_mask == 0xf)
+		return PSSCR_HV_DEFAULT_MASK;
+	return psscr_mask;
+}
 #endif
 
 #endif
arch/powerpc/include/asm/processor.h +2 -1
index c07c31b..422becd 100644
@@ -458,7 +458,8 @@ static inline unsigned long get_clean_sp(unsigned long sp, int is_32)
 extern unsigned long power7_nap(int check_irq);
 extern unsigned long power7_sleep(void);
 extern unsigned long power7_winkle(void);
-extern unsigned long power9_idle_stop(unsigned long stop_level);
+extern unsigned long power9_idle_stop(unsigned long stop_psscr_val,
+				      unsigned long stop_psscr_mask);
 
 extern void flush_instruction_cache(void);
 extern void hard_reset_now(void);
arch/powerpc/kernel/idle_book3s.S +20 -11
index be90e2f..37ee533 100644
@@ -40,9 +40,7 @@
 #define _WORC	GPR11
 #define _PTCR	GPR12
 
-#define PSSCR_HV_TEMPLATE	PSSCR_ESL | PSSCR_EC | \
-				PSSCR_PSLL_MASK | PSSCR_TR_MASK | \
-				PSSCR_MTL_MASK
+#define PSSCR_EC_ESL_MASK_SHIFTED          (PSSCR_EC | PSSCR_ESL) >> 16
 
 	.text
 
@@ -264,7 +262,7 @@ enter_winkle:
 	IDLE_STATE_ENTER_SEQ_NORET(PPC_WINKLE)
 
 /*
- * r3 - requested stop state
+ * r3 - PSSCR value corresponding to the requested stop state.
  */
 power_enter_stop:
 #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
@@ -274,9 +272,19 @@ power_enter_stop:
 	stb	r4,HSTATE_HWTHREAD_STATE(r13)
 #endif
 /*
+ * Check if we are executing the lite variant with ESL=EC=0
+ */
+	andis.   r4, r3, PSSCR_EC_ESL_MASK_SHIFTED
+	andi.    r3, r3, PSSCR_RL_MASK   /* r3 = requested stop state */
+	cmpdi	 r4, 0
+	bne	 1f
+	IDLE_STATE_ENTER_SEQ(PPC_STOP)
+	li	r3,0  /* Since we didn't lose state, return 0 */
+	b 	pnv_wakeup_noloss
+/*
  * Check if the requested state is a deep idle state.
  */
-	LOAD_REG_ADDRBASE(r5,pnv_first_deep_stop_state)
+1:	LOAD_REG_ADDRBASE(r5,pnv_first_deep_stop_state)
 	ld	r4,ADDROFF(pnv_first_deep_stop_state)(r5)
 	cmpd	r3,r4
 	bge	2f
@@ -353,16 +361,17 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66);		\
 	ld	r3,ORIG_GPR3(r1);	/* Restore original r3 */	\
 20:	nop;
 
-
 /*
- * r3 - requested stop state
+ * r3 - The PSSCR value corresponding to the stop state.
+ * r4 - The PSSCR mask corrresonding to the stop state.
  */
 _GLOBAL(power9_idle_stop)
-	LOAD_REG_IMMEDIATE(r4, PSSCR_HV_TEMPLATE)
-	or	r4,r4,r3
-	mtspr	SPRN_PSSCR, r4
-	li	r4, 1
+	mfspr   r5, SPRN_PSSCR
+	andc    r5, r5, r4
+	or      r3, r3, r5
+	mtspr 	SPRN_PSSCR, r3
 	LOAD_REG_ADDR(r5,power_enter_stop)
+	li	r4, 1
 	b	pnv_powersave_common
 	/* No return */
 /*
arch/powerpc/platforms/powernv/idle.c +67 -14

131 lines not on this page · raw message

arch/powerpc/platforms/powernv/powernv.h +2 -1
index da7c843..6130522 100644
@@ -18,7 +18,8 @@ static inline void pnv_pci_shutdown(void) { }
 #endif
 
 extern u32 pnv_get_supported_cpuidle_states(void);
-extern u64 pnv_deepest_stop_state;
+extern u64 pnv_deepest_stop_psscr_val;
+extern u64 pnv_deepest_stop_psscr_mask;
 
 extern void pnv_lpc_init(void);
 
arch/powerpc/platforms/powernv/smp.c +8 -6
index c789258..1c6405f 100644
@@ -182,15 +182,17 @@ static void pnv_smp_cpu_kill_self(void)
 
 		ppc64_runlatch_off();
 
-		if (cpu_has_feature(CPU_FTR_ARCH_300))
-			srr1 = power9_idle_stop(pnv_deepest_stop_state);
-		else if (idle_states & OPAL_PM_WINKLE_ENABLED)
+		if (cpu_has_feature(CPU_FTR_ARCH_300)) {
+			srr1 = power9_idle_stop(pnv_deepest_stop_psscr_val,
+						pnv_deepest_stop_psscr_mask);
+		} else if (idle_states & OPAL_PM_WINKLE_ENABLED) {
 			srr1 = power7_winkle();
-		else if ((idle_states & OPAL_PM_SLEEP_ENABLED) ||
-				(idle_states & OPAL_PM_SLEEP_ENABLED_ER1))
+		} else if ((idle_states & OPAL_PM_SLEEP_ENABLED) ||
+			   (idle_states & OPAL_PM_SLEEP_ENABLED_ER1)) {
 			srr1 = power7_sleep();
-		else
+		} else {
 			srr1 = power7_nap(1);
+		}
 
 		ppc64_runlatch_on();
 
drivers/cpuidle/cpuidle-powernv.c +29 -11
index db18af1..6288189 100644
@@ -19,6 +19,7 @@
 #include <asm/firmware.h>
 #include <asm/opal.h>
 #include <asm/runlatch.h>
+#include <asm/cpuidle.h>
 
 #define POWERNV_THRESHOLD_LATENCY_NS 200000
 
@@ -30,7 +31,12 @@ struct cpuidle_driver powernv_idle_driver = {
 static int max_idle_state;
 static struct cpuidle_state *cpuidle_state_table;
 
-static u64 stop_psscr_table[CPUIDLE_STATE_MAX];
+struct stop_psscr_table {
+	u64 val;
+	u64 mask;
+};
+
+static struct stop_psscr_table stop_psscr_table[CPUIDLE_STATE_MAX];
 
 static u64 snooze_timeout;
 static bool snooze_timeout_en;
@@ -102,7 +108,8 @@ static int stop_loop(struct cpuidle_device *dev,
 		     int index)
 {
 	ppc64_runlatch_off();
-	power9_idle_stop(stop_psscr_table[index]);
+	power9_idle_stop(stop_psscr_table[index].val,
+			 stop_psscr_table[index].mask);
 	ppc64_runlatch_on();
 	return index;
 }
@@ -174,7 +181,7 @@ static inline void add_powernv_state(int index, const char *name,
 						    int),
 				     unsigned int target_residency,
 				     unsigned int exit_latency,
-				     u64 psscr_val)
+				     u64 psscr_val, u64 psscr_mask)
 {
 	strlcpy(powernv_states[index].name, name, CPUIDLE_NAME_LEN);
 	strlcpy(powernv_states[index].desc, name, CPUIDLE_NAME_LEN);
@@ -182,7 +189,9 @@ static inline void add_powernv_state(int index, const char *name,
 	powernv_states[index].target_residency = target_residency;
 	powernv_states[index].exit_latency = exit_latency;
 	powernv_states[index].enter = idle_fn;
-	stop_psscr_table[index] = psscr_val;
+	stop_psscr_table[index].val = compute_psscr_val(psscr_val,
+							psscr_mask);
+	stop_psscr_table[index].mask = compute_psscr_mask(psscr_mask);
 }
 
 static int powernv_add_idle_states(void)
@@ -194,6 +203,7 @@ static int powernv_add_idle_states(void)
 	u32 residency_ns[CPUIDLE_STATE_MAX];
 	u32 flags[CPUIDLE_STATE_MAX];
 	u64 psscr_val[CPUIDLE_STATE_MAX];
+	u64 psscr_mask[CPUIDLE_STATE_MAX];
 	const char *names[CPUIDLE_STATE_MAX];
 	int i, rc;
 
@@ -241,15 +251,23 @@ static int powernv_add_idle_states(void)
 
 	/*
 	 * If the idle states use stop instruction, probe for psscr values
-	 * which are necessary to specify required stop level.
+	 * and psscr mask which are necessary to specify required stop level.
 	 */
-	if (flags[0] & (OPAL_PM_STOP_INST_FAST | OPAL_PM_STOP_INST_DEEP))
+	if (flags[0] & (OPAL_PM_STOP_INST_FAST | OPAL_PM_STOP_INST_DEEP)) {
 		if (of_property_read_u64_array(power_mgt,
 		    "ibm,cpu-idle-state-psscr", psscr_val, dt_idle_states)) {
-			pr_warn("cpuidle-powernv: missing ibm,cpu-idle-states-psscr in DT\n");
+			pr_warn("cpuidle-powernv: missing ibm,cpu-idle-state-psscr in DT\n");
 			goto out;
 		}
 
+		if (of_property_read_u64_array(power_mgt,
+					       "ibm,cpu-idle-state-psscr-mask",
+						psscr_mask, dt_idle_states)) {
+			pr_warn("cpuidle-powernv:Missing ibm,cpu-idle-state-psscr-mask in DT\n");
+			goto out;
+		}
+	}
+
 	rc = of_property_read_u32_array(power_mgt,
 		"ibm,cpu-idle-state-residency-ns", residency_ns, dt_idle_states);
 
@@ -282,13 +300,13 @@ static int powernv_add_idle_states(void)
 			/* Add NAP state */
 			add_powernv_state(nr_idle_states, "Nap",
 					  CPUIDLE_FLAG_NONE, nap_loop,
-					  target_residency, exit_latency, 0);
+					  target_residency, exit_latency, 0, 0);
 		} else if ((flags[i] & OPAL_PM_STOP_INST_FAST) &&
 				!(flags[i] & OPAL_PM_TIMEBASE_STOP)) {
 			add_powernv_state(nr_idle_states, names[i],
 					  CPUIDLE_FLAG_NONE, stop_loop,
 					  target_residency, exit_latency,
-					  psscr_val[i]);
+					  psscr_val[i], psscr_mask[i]);
 		}
 
 		/*
@@ -304,13 +322,13 @@ static int powernv_add_idle_states(void)
 			add_powernv_state(nr_idle_states, "FastSleep",
 					  CPUIDLE_FLAG_TIMER_STOP,
 					  fastsleep_loop,
-					  target_residency, exit_latency, 0);
+					  target_residency, exit_latency, 0, 0);
 		} else if ((flags[i] & OPAL_PM_STOP_INST_DEEP) &&
 				(flags[i] & OPAL_PM_TIMEBASE_STOP)) {
 			add_powernv_state(nr_idle_states, names[i],
 					  CPUIDLE_FLAG_TIMER_STOP, stop_loop,
 					  target_residency, exit_latency,
-					  psscr_val[i]);
+					  psscr_val[i], psscr_mask[i]);
 		}
 #endif
 		nr_idle_states++;
Signature · 1 line
1.9.4
#8 lore raw

Re: [PATCH v4 3/4] powernv: Pass PSSCR value and mask to power9_idle_stop

Quoted 137 lines · Gautham R. Shenoy

From: "Gautham R. Shenoy" <[email protected]>

The power9_idle_stop method currently takes only the requested stop level as a parameter and picks up the rest of the PSSCR bits from a hand-coded macro. This is not a very flexible design, especially when the firmware has the capability to communicate the psscr value and the mask associated with a particular stop state via device tree.

This patch modifies the power9_idle_stop API to take as parameters the PSSCR value and the PSSCR mask corresponding to the stop state that needs to be set. These PSSCR value and mask are respectively obtained by parsing the "ibm,cpu-idle-state-psscr" and "ibm,cpu-idle-state-psscr-mask" fields from the device tree.

In addition to this, the patch adds support for handling stop states for which ESL and EC bits in the PSSCR are zero. As per the architecture, a wakeup from these stop states resumes execution from the subsequent instruction as opposed to waking up at the System Vector.

The older firmware sets only the Requested Level (RL) field in the psscr and psscr-mask exposed in the device tree. For older firmware where psscr-mask=0xf, this patch will set the default sane values that the set for for remaining PSSCR fields (i.e PSLL, MTL, ESL, EC, and TR).

This skiboot patch that exports fully populated PSSCR values and the mask for all the stop states can be found here:
https://lists.ozlabs.org/pipermail/skiboot/2016-September/004869.html

---
 arch/powerpc/include/asm/cpuidle.h       | 41 ++++++++++++++++
 arch/powerpc/include/asm/processor.h     |  3 +-
 arch/powerpc/kernel/idle_book3s.S        | 31 +++++++-----
 arch/powerpc/platforms/powernv/idle.c    | 81 ++++++++++++++++++++++++++------
 arch/powerpc/platforms/powernv/powernv.h |  3 +-
 arch/powerpc/platforms/powernv/smp.c     | 14 +++---
 drivers/cpuidle/cpuidle-powernv.c        | 40 +++++++++++-----
 7 files changed, 169 insertions(+), 44 deletions(-)
arch/powerpc/include/asm/cpuidle.h +41 -0
index 0a3255b..fa0b6c0 100644
@@ -10,11 +10,52 @@
 #define PNV_CORE_IDLE_LOCK_BIT          0x100
 #define PNV_CORE_IDLE_THREAD_BITS       0x0FF
 
+/*
+ * ============================ NOTE =================================
+ * The older firmware populates only the RL field in the psscr_val and
+ * sets the psscr_mask to 0xf. On such a firmware, the kernel sets the
+ * remaining PSSCR fields to default values as follows:
+ *
+ * - ESL and EC bits are to 1. So wakeup from any stop state will be
+ *   at vector 0x100.
+ *
+ * - MTL and PSLL are set to the maximum allowed value as per the ISA,
+ *    i.e. 15.
+ *
+ * - The Transition Rate, TR is set to the Maximum value 3.
+ */
+#define PSSCR_HV_DEFAULT_VAL    (PSSCR_ESL | PSSCR_EC |		    \
+				PSSCR_PSLL_MASK | PSSCR_TR_MASK |   \
+				PSSCR_MTL_MASK)
+
+#define PSSCR_HV_DEFAULT_MASK   (PSSCR_ESL | PSSCR_EC |		    \
+				PSSCR_PSLL_MASK | PSSCR_TR_MASK |   \
+				PSSCR_MTL_MASK | PSSCR_RL_MASK)
+
 #ifndef __ASSEMBLY__
 extern u32 pnv_fastsleep_workaround_at_entry[];
 extern u32 pnv_fastsleep_workaround_at_exit[];
 
 extern u64 pnv_first_deep_stop_state;
+
+static inline u64 compute_psscr_val(u64 psscr_val, u64 psscr_mask)
+{
+	/*
+	 * psscr_mask == 0xf indicates an older firmware.
+	 * Set remaining fields of psscr to the default values.
+	 * See NOTE above definition of PSSCR_HV_DEFAULT_VAL
+	 */
+	if (psscr_mask == 0xf)
+		return psscr_val | PSSCR_HV_DEFAULT_VAL;
+	return psscr_val;
+}
+
+static inline u64 compute_psscr_mask(u64 psscr_mask)
+{
+	if (psscr_mask == 0xf)
+		return PSSCR_HV_DEFAULT_MASK;
+	return psscr_mask;
+}
 #endif
 
 #endif
arch/powerpc/include/asm/processor.h +2 -1
index c07c31b..422becd 100644
@@ -458,7 +458,8 @@ static inline unsigned long get_clean_sp(unsigned long sp, int is_32)
 extern unsigned long power7_nap(int check_irq);
 extern unsigned long power7_sleep(void);
 extern unsigned long power7_winkle(void);
-extern unsigned long power9_idle_stop(unsigned long stop_level);
+extern unsigned long power9_idle_stop(unsigned long stop_psscr_val,
+				      unsigned long stop_psscr_mask);
 
 extern void flush_instruction_cache(void);
 extern void hard_reset_now(void);
arch/powerpc/kernel/idle_book3s.S +2 -4
index be90e2f..37ee533 100644
@@ -40,9 +40,7 @@
 #define _WORC	GPR11
 #define _PTCR	GPR12
 
-#define PSSCR_HV_TEMPLATE	PSSCR_ESL | PSSCR_EC | \
-				PSSCR_PSLL_MASK | PSSCR_TR_MASK | \
-				PSSCR_MTL_MASK
+#define PSSCR_EC_ESL_MASK_SHIFTED          (PSSCR_EC | PSSCR_ESL) >> 16
 
 	.text
 
@@ -264,7 +262,7 @@ enter_winkle:
 	IDLE_STATE_ENTER_SEQ_NORET(PPC_WINKLE)
 
 /*
- * r3 - requested stop state
+ * r3 - PSSCR value corresponding to the requested stop state.
  */
 power_enter_stop:
 #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE

arch/powerpc/kernel/idle_book3s.S power_enter_stop: · line 277

Quoted 6 lines
arch/powerpc/kernel/idle_book3s.S +3 -0
@@ -274,9 +272,19 @@ power_enter_stop:
 	stb	r4,HSTATE_HWTHREAD_STATE(r13)
 #endif
 /*
+ * Check if we are executing the lite variant with ESL=EC=0
+ */
+	andis.   r4, r3, PSSCR_EC_ESL_MASK_SHIFTED

r4 = psscr & (PSSCR_EC | PSSCR_ESL)

arch/powerpc/kernel/idle_book3s.S power_enter_stop: · line 278

Quoted 1 line
arch/powerpc/kernel/idle_book3s.S +1 -0
+	andi.    r3, r3, PSSCR_RL_MASK   /* r3 = requested stop state */

r3 = psscr & RL_MASK (requested mask).

Why do we do and andis. followed by andi. and a compdi below?

arch/powerpc/kernel/idle_book3s.S power_enter_stop: · line 279

Quoted 1 line
arch/powerpc/kernel/idle_book3s.S +1 -0
+	cmpdi	 r4, 0

r4 == 0 implies we either both PSSCR_EC|ESL are cleared. I am not sure if our checks for EC are well defined/implemented. Should we worry about EC at all at this point?

Quoted 13 lines
arch/powerpc/kernel/idle_book3s.S +6 -1
+	bne	 1f
+	IDLE_STATE_ENTER_SEQ(PPC_STOP)
+	li	r3,0  /* Since we didn't lose state, return 0 */
+	b 	pnv_wakeup_noloss
+/*
  * Check if the requested state is a deep idle state.
  */
-	LOAD_REG_ADDRBASE(r5,pnv_first_deep_stop_state)
+1:	LOAD_REG_ADDRBASE(r5,pnv_first_deep_stop_state)
 	ld	r4,ADDROFF(pnv_first_deep_stop_state)(r5)
 	cmpd	r3,r4
 	bge	2f

arch/powerpc/kernel/idle_book3s.S ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66); \ · line 356

Quoted 4 lines
arch/powerpc/kernel/idle_book3s.S +0 -1
@@ -353,16 +361,17 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66);		\
 	ld	r3,ORIG_GPR3(r1);	/* Restore original r3 */	\
 20:	nop;
 
-

Spurious change?

Quoted 62 lines
arch/powerpc/kernel/idle_book3s.S +7 -5
 /*
- * r3 - requested stop state
+ * r3 - The PSSCR value corresponding to the stop state.
+ * r4 - The PSSCR mask corrresonding to the stop state.
  */
 _GLOBAL(power9_idle_stop)
-	LOAD_REG_IMMEDIATE(r4, PSSCR_HV_TEMPLATE)
-	or	r4,r4,r3
-	mtspr	SPRN_PSSCR, r4
-	li	r4, 1
+	mfspr   r5, SPRN_PSSCR
+	andc    r5, r5, r4
+	or      r3, r3, r5
+	mtspr 	SPRN_PSSCR, r3
 	LOAD_REG_ADDR(r5,power_enter_stop)
+	li	r4, 1
 	b	pnv_powersave_common
 	/* No return */
 /*
arch/powerpc/platforms/powernv/idle.c +12 -4
index 479c256..663c6ef 100644
@@ -237,15 +237,21 @@ static DEVICE_ATTR(fastsleep_workaround_applyonce, 0600,
 			show_fastsleep_workaround_applyonce,
 			store_fastsleep_workaround_applyonce);
 
+/*
+ * The default stop state that will be used by ppc_md.power_save
+ * function on platforms that support stop instruction.
+ */
+u64 pnv_default_stop_val;
+u64 pnv_default_stop_mask;
 
 /*
  * Used for ppc_md.power_save which needs a function with no parameters
  */
 static void power9_idle(void)
 {
-	/* Requesting stop state 0 */
-	power9_idle_stop(0);
+	power9_idle_stop(pnv_default_stop_val, pnv_default_stop_mask);
 }
+
 /*
  * First deep stop state. Used to figure out when to save/restore
  * hypervisor context.
@@ -253,9 +259,11 @@ static void power9_idle(void)
 u64 pnv_first_deep_stop_state = MAX_STOP_STATE;
 
 /*
- * Deepest stop idle state. Used when a cpu is offlined
+ * psscr value and mask of the deepest stop idle state.
+ * Used when a cpu is offlined.
  */
-u64 pnv_deepest_stop_state;
+u64 pnv_deepest_stop_psscr_val;
+u64 pnv_deepest_stop_psscr_mask;
 
 /*
  * Power ISA 3.0 idle initialization.

arch/powerpc/platforms/powernv/idle.c static int __init pnv_arch300_idle_init(struct device_node *np, u32 *flags, · line 310

Quoted 1 line
arch/powerpc/platforms/powernv/idle.c +0 -0
@@ -302,28 +310,58 @@ static int __init pnv_arch300_idle_init(struct device_node *np, u32 *flags,
 					int dt_idle_states)

In some cases we say power9 and arch300 in others, not related to this patchset, but just a generic comment

Quoted 69 lines
arch/powerpc/platforms/powernv/idle.c +37 -8
 {
 	u64 *psscr_val = NULL;
+	u64 *psscr_mask = NULL;
+	u32 *residency_ns = NULL;
+	u64 max_residency_ns = 0;
 	int rc = 0, i;
+	bool default_stop_found = false;
 
-	psscr_val = kcalloc(dt_idle_states, sizeof(*psscr_val),
-				GFP_KERNEL);
-	if (!psscr_val) {
+	psscr_val = kcalloc(dt_idle_states, sizeof(*psscr_val), GFP_KERNEL);
+	psscr_mask = kcalloc(dt_idle_states, sizeof(*psscr_mask), GFP_KERNEL);
+	residency_ns = kcalloc(dt_idle_states, sizeof(*residency_ns),
+			       GFP_KERNEL);
+
+	if (!psscr_val || !psscr_mask || !residency_ns) {
 		rc = -1;
 		goto out;
 	}
+
 	if (of_property_read_u64_array(np,
 		"ibm,cpu-idle-state-psscr",
 		psscr_val, dt_idle_states)) {
-		pr_warn("cpuidle-powernv: missing ibm,cpu-idle-states-psscr in DT\n");
+		pr_warn("cpuidle-powernv: missing ibm,cpu-idle-state-psscr in DT\n");
+		rc = -1;
+		goto out;
+	}
+
+	if (of_property_read_u64_array(np,
+				       "ibm,cpu-idle-state-psscr-mask",
+				       psscr_mask, dt_idle_states)) {
+		pr_warn("cpuidle-powernv: missing ibm,cpu-idle-state-psscr-mask in DT\n");
+		rc = -1;
+		goto out;
+	}
+
+	if (of_property_read_u32_array(np,
+				       "ibm,cpu-idle-state-residency-ns",
+					residency_ns, dt_idle_states)) {
+		pr_warn("cpuidle-powernv: missing ibm,cpu-idle-state-residency-ns in DT\n");
 		rc = -1;
 		goto out;
 	}
 
 	/*
-	 * Set pnv_first_deep_stop_state and pnv_deepest_stop_state.
+	 * Set pnv_first_deep_stop_state, pnv_deepest_stop_psscr_{val,mask},
+	 * and the pnv_default_stop_{val,mask}.
+	 *
 	 * pnv_first_deep_stop_state should be set to the first stop
 	 * level to cause hypervisor state loss.
-	 * pnv_deepest_stop_state should be set to the deepest stop
-	 * stop state.
+	 *
+	 * pnv_deepest_stop_{val,mask} should be set to values corresponding to
+	 * the deepest stop state.
+	 *
+	 * pnv_default_stop_{val,mask} should be set to values corresponding to
+	 * the shallowest (OPAL_PM_STOP_INST_FAST) loss-less stop state.
 	 */
 	pnv_first_deep_stop_state = MAX_STOP_STATE;
 	for (i = 0; i < dt_idle_states; i++) {
@@ -333,12 +371,27 @@ static int __init pnv_arch300_idle_init(struct device_node *np, u32 *flags,
 		     (pnv_first_deep_stop_state > psscr_rl))
 			pnv_first_deep_stop_state = psscr_rl;
 
-		if (pnv_deepest_stop_state < psscr_rl)

arch/powerpc/platforms/powernv/idle.c static int __init pnv_arch300_idle_init(struct device_node *np, u32 *flags, · line 381

Quoted 10 lines
arch/powerpc/platforms/powernv/idle.c +7 -2
-			pnv_deepest_stop_state = psscr_rl;
-	}
+		if (max_residency_ns < residency_ns[i]) {
+			max_residency_ns = residency_ns[i];
+			pnv_deepest_stop_psscr_val =
+				compute_psscr_val(psscr_val[i], psscr_mask[i]);
+			pnv_deepest_stop_psscr_mask =
+				compute_psscr_mask(psscr_mask[i]);
+		}
 

Does it make sense to have them sorted and then use the last value from the array?

Balbir Singh

#9 lore raw

Re: [PATCH v4 3/4] powernv: Pass PSSCR value and mask to power9_idle_stop

Hi Balbir,

Thanks for reviewing the patch. Please find my comments inline.

On Wed, Dec 14, 2016 at 11:16:26AM +1100, Balbir Singh wrote:
[..snip..]

Quoted 17 lines
Quoted 8 lines

/*
- * r3 - requested stop state
+ * r3 - PSSCR value corresponding to the requested stop state. */
power_enter_stop:
#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE

arch/powerpc/kernel/idle_book3s.S power_enter_stop: · line 277

Quoted 6 lines
arch/powerpc/kernel/idle_book3s.S +3 -0
@@ -274,9 +272,19 @@ power_enter_stop:
 	stb	r4,HSTATE_HWTHREAD_STATE(r13)
 #endif
 /*
+ * Check if we are executing the lite variant with ESL=EC=0
+ */
+	andis.   r4, r3, PSSCR_EC_ESL_MASK_SHIFTED

r4 = psscr & (PSSCR_EC | PSSCR_ESL)

Quoted 5 lines

arch/powerpc/kernel/idle_book3s.S power_enter_stop: · line 278

Quoted 1 line
arch/powerpc/kernel/idle_book3s.S +1 -0
+	andi.    r3, r3, PSSCR_RL_MASK   /* r3 = requested stop state */

r3 = psscr & RL_MASK (requested mask).

Why do we do and andis. followed by andi. and a compdi below?

Do you mean why are we not using the CR0 value instead of using cmpdi again ? Hmm.. The subsequent code expect r3 to contain only the RL value. So, how about the following?

andi. r4, r3, PSSCR_RL_MASK;
andis. r3, r3, PSSCR_EC_ESL_MASK_SHIFTED;
mr r3, r4;
bne 1f;

Quoted 7 lines

arch/powerpc/kernel/idle_book3s.S power_enter_stop: · line 279

Quoted 2 lines
arch/powerpc/kernel/idle_book3s.S +1 -0
+	cmpdi	 r4, 0

r4 == 0 implies we either both PSSCR_EC|ESL are cleared.

I am not sure if our checks for EC are well defined/implemented. Should we worry about EC at all at this point?

Yes, we need to check the value of EC. Because if EC == 0, that implies that the hardware will wake up from the stop instruction at the subsequent instruction which we need to handle. This behaviour is only available from POWER9 onwards, since on POWER8, the wakeup from nap,sleep and winkle were always at 0x100. Hence the existing code assumes that all the wakeups are at 0x100, which is what this patch modifies.

Quoted 17 lines
Quoted 17 lines
arch/powerpc/kernel/idle_book3s.S +6 -1
+	bne	 1f
+	IDLE_STATE_ENTER_SEQ(PPC_STOP)
+	li	r3,0  /* Since we didn't lose state, return 0 */
+	b 	pnv_wakeup_noloss
+/*
  * Check if the requested state is a deep idle state.
  */
-	LOAD_REG_ADDRBASE(r5,pnv_first_deep_stop_state)
+1:	LOAD_REG_ADDRBASE(r5,pnv_first_deep_stop_state)
 	ld	r4,ADDROFF(pnv_first_deep_stop_state)(r5)
 	cmpd	r3,r4
 	bge	2f
@@ -353,16 +361,17 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66);		\
 	ld	r3,ORIG_GPR3(r1);	/* Restore original r3 */	\
 20:	nop;
 
Quoted 3 lines

arch/powerpc/kernel/idle_book3s.S ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66); \ · line 356

Quoted 1 line
arch/powerpc/kernel/idle_book3s.S +0 -1
-

Spurious change?

There were two empty lines for no particular reason. So got rid of one of them.

Quoted 20 lines
Quoted 20 lines
arch/powerpc/kernel/idle_book3s.S +7 -5
 /*
- * r3 - requested stop state
+ * r3 - The PSSCR value corresponding to the stop state.
+ * r4 - The PSSCR mask corrresonding to the stop state.
  */
 _GLOBAL(power9_idle_stop)
-	LOAD_REG_IMMEDIATE(r4, PSSCR_HV_TEMPLATE)
-	or	r4,r4,r3
-	mtspr	SPRN_PSSCR, r4
-	li	r4, 1
+	mfspr   r5, SPRN_PSSCR
+	andc    r5, r5, r4
+	or      r3, r3, r5
+	mtspr 	SPRN_PSSCR, r3
 	LOAD_REG_ADDR(r5,power_enter_stop)
+	li	r4, 1
 	b	pnv_powersave_common
 	/* No return */
 /*

[..snip..]

Quoted 12 lines
Quoted 12 lines
arch/powerpc/platforms/powernv/idle.c +4 -2
@@ -253,9 +259,11 @@ static void power9_idle(void)
 u64 pnv_first_deep_stop_state = MAX_STOP_STATE;
 
 /*
- * Deepest stop idle state. Used when a cpu is offlined
+ * psscr value and mask of the deepest stop idle state.
+ * Used when a cpu is offlined.
  */
-u64 pnv_deepest_stop_state;
+u64 pnv_deepest_stop_psscr_val;
+u64 pnv_deepest_stop_psscr_mask;
 
Quoted 7 lines

arch/powerpc/platforms/powernv/idle.c static int __init pnv_arch300_idle_init(struct device_node *np, u32 *flags, · line 310

Quoted 4 lines
arch/powerpc/platforms/powernv/idle.c +0 -0
 /*
  * Power ISA 3.0 idle initialization.
@@ -302,28 +310,58 @@ static int __init pnv_arch300_idle_init(struct device_node *np, u32 *flags,
 					int dt_idle_states)

In some cases we say power9 and arch300 in others, not related to

Quoted 1 line
(quoted diff) +0 -0
 this patchset, but just a generic comment

Will see if I can make this consistent.

[..snip..]

Quoted 4 lines
Quoted 4 lines
arch/powerpc/platforms/powernv/idle.c +0 -0
@@ -333,12 +371,27 @@ static int __init pnv_arch300_idle_init(struct device_node *np, u32 *flags,
 		     (pnv_first_deep_stop_state > psscr_rl))
 			pnv_first_deep_stop_state = psscr_rl;
 
Quoted 14 lines

arch/powerpc/platforms/powernv/idle.c static int __init pnv_arch300_idle_init(struct device_node *np, u32 *flags, · line 381

Quoted 11 lines
arch/powerpc/platforms/powernv/idle.c +7 -3
-		if (pnv_deepest_stop_state < psscr_rl)
-			pnv_deepest_stop_state = psscr_rl;
-	}
+		if (max_residency_ns < residency_ns[i]) {
+			max_residency_ns = residency_ns[i];
+			pnv_deepest_stop_psscr_val =
+				compute_psscr_val(psscr_val[i], psscr_mask[i]);
+			pnv_deepest_stop_psscr_mask =
+				compute_psscr_mask(psscr_mask[i]);
+		}
 

Does it make sense to have them sorted and then use the last value from the array?

Yes, if the firmware can be relied upon to do this, we can obtain the deepest_stop_psscr_val and the mask in constant time.

However, this init function is called only once during the boot, and we are anyway iterating over all the idle states to find the first deep stop state and the default stop state. So the optimization for deepest_stop_psscr_val and mask may not gain us much.

Quoted 4 lines

Balbir Singh

--
Thanks and Regards
gautham.

#10 lore raw

PATCH v4 4/4 Documentation:powerpc: Add device-tree bindings for power-mgt

In reply to Gautham R. Shenoy

From: "Gautham R. Shenoy" <[email protected]>

Document the device-tree bindings defining the the properties under the @power-mgt node in the device tree that describe the idle states for Linux running on baremetal POWER servers.

---
 .../devicetree/bindings/powerpc/opal/power-mgt.txt | 123 +++++++++++++++++++++
 1 file changed, 123 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/powerpc/opal/power-mgt.txt
Documentation/devicetree/bindings/powerpc/opal/power-mgt.txt new file +123 -0
new file mode 100644
index 0000000..002b59e
@@ -0,0 +1,123 @@
+IBM Power-Management Bindings
+=============================
+
+Linux running on baremetal POWER machines has access to the processor
+idle states. The description of these idle states is exposed via the
+node @power-mgt in the device-tree by the firmware.
+
+Definitions:
+----------------
+Typically each idle state has the following associated properties:
+
+- name: The name of the idle state as defined by the firmware.
+
+- flags: indicating some aspects of this idle states such as the
+         extent of state-loss, whether timebase is stopped on this
+         idle states and so on. The flag bits are as follows:
+
+- exit-latency: The latency involved in transitioning the state of the
+		CPU from idle to running.
+
+- target-residency: The minimum time that the CPU needs to reside in
+		    this idle state in order to accrue power-savings
+		    benefit.
+
+Properties
+----------------
+The following properties provide details about the idle states. These
+properties are optional unless mentioned otherwise below.
+
+- ibm,cpu-idle-state-names:
+	Array of strings containing the names of the idle states.
+
+- ibm,cpu-idle-state-flags:
+	Array of unsigned 32-bit values containing the values of the
+	flags associated with the the aforementioned idle-states. This
+	property is required on POWER9 whenever
+	ibm,cpu-idle-state-names is defined and the length of this
+	property array should be the same as
+	ibm,-cpu-idle-state-names.The flag bits are as follows:
+		0x00000001 /* Decrementer would stop */
+		0x00000002 /* Needs timebase restore */
+		0x00001000 /* Restore GPRs like nap */
+		0x00002000 /* Restore hypervisor resource from PACA pointer */
+		0x00004000 /* Program PORE to restore PACA pointer */
+		0x00010000 /* This is a nap state */
+		0x00020000 /* This is a fast-sleep state */
+		0x00040000 /* This is a winkle state */
+		0x00080000 /* This is a fast-sleep state which requires a */
+			   /* software workaround for restoring the timebase*/
+		0x00800000 /* This state uses SPR PMICR instruction */
+		0x00100000 /* This is a fast stop state */
+		0x00200000 /* This is a deep-stop state */
+
+- ibm,cpu-idle-state-latencies-ns:
+	Array of unsigned 32-bit values containing the values of the
+	exit-latencies (in ns) for the idle states in
+	ibm,cpu-idle-state-names. This property is required whenever
+	ibm,cpu-idle-state-names is defined and the length of this
+	property array should be the same as
+	ibm,-cpu-idle-state-names.
+
+- ibm,cpu-idle-state-residency-ns:
+	Array of unsigned 32-bit values containing the values of the
+	target-residency (in ns) for the idle states in
+	ibm,cpu-idle-state-names. On POWER8 this is an optional
+	property. If the property is absent, the target residency for
+	the "Nap", "FastSleep" are defined to 10000 and 300000000
+	respectively. On POWER9 this property must be defined if
+	ibm,cpu-idle-state-names is defined and the length should be
+	same as that of ibm,cpu-idle-state-names.
+
+- ibm,cpu-idle-state-psscr:
+	Array of unsigned 64-bit values containing the values for the
+	PSSCR for each of the idle states in ibm,cpu-idle-state-names.
+	This property is required on POWER9 whenever
+	ibm,cpu-idle-state-names is defined and the length of this
+	property array should be the same as
+	ibm,-cpu-idle-state-names.
+
+- ibm,cpu-idle-state-psscr-mask:
+	Array of unsigned 64-bit values containing the masks
+	indicating which psscr fields are set in the corresponding
+	entries of ibm,cpu-idle-state-psscr.  This property is
+	required on POWER9 whenever ibm,cpu-idle-state-names is
+	defined and the length of this property array should be the
+	same as ibm,cpu-idle-state-names.
+
+	Whenever the firmware sets an entry in
+	ibm,cpu-idle-state-psscr-mask value to 0xf, it implies that
+	only the Requested Level (RL) field of the corresponding entry
+	in ibm,cpu-idle-state-psscr should be considered by the
+	kernel. For such idle states, the kernel would set the
+	remaining fields of the psscr to the following sane-default
+	values.
+
+		- ESL and EC bits are to 1. So wakeup from any stop
+		  state will be at vector 0x100.
+
+		- MTL and PSLL are set to the maximum allowed value as
+		  per the ISA, i.e. 15.
+
+		- The Transition Rate, TR is set to the Maximum value
+                  3.
+
+	For all the other values of the entry in
+	ibm,cpu-idle-state-psscr-mask, the Kernel expects all the
+	psscr fields of the corresponding entry in
+	ibm,cpu-idle-state-psscr to be correctly set by the firmware.
+
+- ibm,cpu-idle-state-pmicr:
+	Array of unsigned 64-bit values containing the pmicr values
+	for the idle states in ibm,cpu-idle-state-names. This is an
+	optional property on POWER8 and is absent on POWER9. When
+	present on POWER8, the length of this property array should be
+	the same as ibm,cpu-idle=state-names.
+
+- ibm,cpu-idle-state-pmicr:
+	Array of unsigned 64-bit values containing the mask indicating
+	which of the fields of the PMICR are set in the corresponding
+	entries in ibm,cpu-idle-state-pmicr. This is an optional
+	property on POWER8 and is absent on POWER9. When present on
+	POWER8, the length of this property array should be the same
+	as ibm,cpu-idle=state-names.
Signature · 1 line
1.9.4