Skip to contentSkip to conversation

powernv:stop: Use psscr_val,mask provided by firmware

PATCH Superseded Gautham R. Shenoy 4 participants 8 messages Sep 29, 2016 – Oct 13, 2016

Subsystems (2)

v1 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

cover powernv:stop: Use psscr_val,mask provided by firmware

From: "Gautham R. Shenoy" <[email protected]>

Hi,

In the current implementation, the code for ISA v3.0 stop implementation has a couple of shortcomings.

a) The code hand-codes the values for ESL,EC,TR,MTL bits of PSSCR and uses only the RL field from the firmware. While this is not incorrect, since the hand-coded values are legitimate, it is not a very flexible design since the firmware has the capability to communicate these values via the "ibm,cpu-idle-state-psscr" and "ibm,cpu-idle-state-psscr-mask" properties. In case where the firmware provides values for these fields that is different from the hand-coded values, the current code will not work as intended.

b) Due to issue a), the current code assumes that ESL=EC=1 for all the stop states and hence the wakeup from the stop instruction will happen at 0x100, the system-reset vector. However, the ISA v3.0 allows the ESL=EC=0 behaviour where the corresponding stop-state loses no state and wakes up from the subsequent instruction. The current code doesn't handle this case.

This patch series addresses these issues.

The first patch in the series renames the existing IDLE_STATE_ENTER_SEQ macro to IDLE_STATE_ENTER_SEQ_NORET. It reuses the name IDLE_STATE_ENTER_SEQ for entering into stop-states which wake up at the subsequent instruction.

The second patch in the series fixes issues a) and b) by ensuring that the psscr-value and the psscr-mask provided by the firmware are what will be used to set a particular stop state. It also adds support for handling wake-up from stop states which were entered with ESL=EC=0.

These patches depend on the following skiboot patch that exports the PSSCR values and the mask for all the stop states:
https://lists.ozlabs.org/pipermail/skiboot/2016-September/004869.html

Gautham R. Shenoy (2):
powernv:idle: Add IDLE_STATE_ENTER_SEQ_NORET macro
powernv: Pass PSSCR value and mask to power9_idle_stop

 arch/powerpc/include/asm/cpuidle.h       |  5 ++-
 arch/powerpc/include/asm/processor.h     |  3 +-
 arch/powerpc/kernel/exceptions-64s.S     |  6 +--
 arch/powerpc/kernel/idle_book3s.S        | 41 ++++++++++-------
 arch/powerpc/platforms/powernv/idle.c    | 76 +++++++++++++++++++++++++++-----
 arch/powerpc/platforms/powernv/powernv.h |  3 +-
 arch/powerpc/platforms/powernv/smp.c     |  7 +--
 drivers/cpuidle/cpuidle-powernv.c        | 30 ++++++++++---
 8 files changed, 127 insertions(+), 44 deletions(-)
Signature · 1 line
1.9.4
#1 lore raw

PATCH 1/2 powernv:idle: Add IDLE_STATE_ENTER_SEQ_NORET macro

From: "Gautham R. Shenoy" <[email protected]>

Currently all the low-power idle states are expected to wake up at reset vector 0x100. Which is why the macro IDLE_STATE_ENTER_SEQ that puts the CPU to an idle state and never returns.

On ISA_300, when the ESL and EC bits in the PSSCR are zero, the CPU is expected to wake up at the next instruction of the idle instruction.

This patch adds a new macro named IDLE_STATE_ENTER_SEQ_NORET for the no-return variant and reuses the name IDLE_STATE_ENTER_SEQ for a variant that allows resuming operation at the instruction next to the idle-instruction.

---
 arch/powerpc/include/asm/cpuidle.h   |  5 ++++-
 arch/powerpc/kernel/exceptions-64s.S |  6 +++---
 arch/powerpc/kernel/idle_book3s.S    | 10 +++++-----
 3 files changed, 12 insertions(+), 9 deletions(-)
arch/powerpc/include/asm/cpuidle.h +4 -1
index 01b8a13..9fd23f6 100644
@@ -21,7 +21,7 @@ extern u64 pnv_first_deep_stop_state;
 
 /* Idle state entry routines */
 #ifdef	CONFIG_PPC_P7_NAP
-#define	IDLE_STATE_ENTER_SEQ(IDLE_INST)				\
+#define IDLE_STATE_ENTER_SEQ(IDLE_INST)                         \
 	/* Magic NAP/SLEEP/WINKLE mode enter sequence */	\
 	std	r0,0(r1);					\
 	ptesync;						\
@@ -29,6 +29,9 @@ extern u64 pnv_first_deep_stop_state;
 1:	cmp	cr0,r0,r0;					\
 	bne	1b;						\
 	IDLE_INST;						\
+
+#define	IDLE_STATE_ENTER_SEQ_NORET(IDLE_INST)			\
+	IDLE_STATE_ENTER_SEQ(IDLE_INST)                         \
 	b	.
 #endif /* CONFIG_PPC_P7_NAP */
 
arch/powerpc/kernel/exceptions-64s.S +3 -3
index bffec73..238307d 100644
@@ -1301,12 +1301,12 @@ machine_check_handle_early:
 	lbz	r3,PACA_THREAD_IDLE_STATE(r13)
 	cmpwi	r3,PNV_THREAD_NAP
 	bgt	10f
-	IDLE_STATE_ENTER_SEQ(PPC_NAP)
+	IDLE_STATE_ENTER_SEQ_NORET(PPC_NAP)
 	/* No return */
 10:
 	cmpwi	r3,PNV_THREAD_SLEEP
 	bgt	2f
-	IDLE_STATE_ENTER_SEQ(PPC_SLEEP)
+	IDLE_STATE_ENTER_SEQ_NORET(PPC_SLEEP)
 	/* No return */
 
 2:
@@ -1320,7 +1320,7 @@ machine_check_handle_early:
 	 */
 	ori	r13,r13,1
 	SET_PACA(r13)
-	IDLE_STATE_ENTER_SEQ(PPC_WINKLE)
+	IDLE_STATE_ENTER_SEQ_NORET(PPC_WINKLE)
 	/* No return */
 4:
 #endif
arch/powerpc/kernel/idle_book3s.S +5 -5
index bd739fe..32d666b 100644
@@ -188,7 +188,7 @@ pnv_enter_arch207_idle_mode:
 	stb	r3,PACA_THREAD_IDLE_STATE(r13)
 	cmpwi	cr3,r3,PNV_THREAD_SLEEP
 	bge	cr3,2f
-	IDLE_STATE_ENTER_SEQ(PPC_NAP)
+	IDLE_STATE_ENTER_SEQ_NORET(PPC_NAP)
 	/* No return */
 2:
 	/* Sleep or winkle */
@@ -222,7 +222,7 @@ pnv_fastsleep_workaround_at_entry:
 
 common_enter: /* common code for all the threads entering sleep or winkle */
 	bgt	cr3,enter_winkle
-	IDLE_STATE_ENTER_SEQ(PPC_SLEEP)
+	IDLE_STATE_ENTER_SEQ_NORET(PPC_SLEEP)
 
 fastsleep_workaround_at_entry:
 	ori	r15,r15,PNV_CORE_IDLE_LOCK_BIT
@@ -244,7 +244,7 @@ fastsleep_workaround_at_entry:
 enter_winkle:
 	bl	save_sprs_to_stack
 
-	IDLE_STATE_ENTER_SEQ(PPC_WINKLE)
+	IDLE_STATE_ENTER_SEQ_NORET(PPC_WINKLE)
 
 /*
  * r3 - requested stop state
@@ -257,7 +257,7 @@ power_enter_stop:
 	ld	r4,ADDROFF(pnv_first_deep_stop_state)(r5)
 	cmpd	r3,r4
 	bge	2f
-	IDLE_STATE_ENTER_SEQ(PPC_STOP)
+	IDLE_STATE_ENTER_SEQ_NORET(PPC_STOP)
 2:
 /*
  * Entering deep idle state.
@@ -279,7 +279,7 @@ lwarx_loop_stop:
 
 	bl	save_sprs_to_stack
 
-	IDLE_STATE_ENTER_SEQ(PPC_STOP)
+	IDLE_STATE_ENTER_SEQ_NORET(PPC_STOP)
 
 _GLOBAL(power7_idle)
 	/* Now check if user or arch enabled NAP mode */
Signature · 1 line
1.9.4
#2 lore raw

PATCH 2/2 powernv: Pass PSSCR value and mask to power9_idle_stop

In reply to Gautham R. Shenoy

From: "Gautham R. Shenoy" <[email protected]>

The power9_idle_stop method currently takes only the requested stop level as a parameter and picks up the rest of the PSSCR bits from a hand-coded macro. This is not a very flexible design, especially when the firmware has the capability to communicate the psscr value and the mask associated with a particular stop state via device tree.

This patch modifies the power9_idle_stop API to take as parameters the PSSCR value and the PSSCR mask corresponding to the stop state that needs to be set. These PSSCR value and mask are respectively obtained by parsing the "ibm,cpu-idle-state-psscr" and "ibm,cpu-idle-state-psscr-mask" fields from the device tree.

In addition to this, the patch adds support for handling stop states for which ESL and EC bits in the PSSCR are zero. As per the architecture, a wakeup from these stop states resumes execution from the subsequent instruction as opposed to waking up at the System Vector.

This patch depends on the following skiboot patch that exports the PSSCR values and the mask for all the stop states:
https://lists.ozlabs.org/pipermail/skiboot/2016-September/004869.html

---
 arch/powerpc/include/asm/processor.h     |  3 +-
 arch/powerpc/kernel/idle_book3s.S        | 31 ++++++++-----
 arch/powerpc/platforms/powernv/idle.c    | 76 +++++++++++++++++++++++++++-----
 arch/powerpc/platforms/powernv/powernv.h |  3 +-
 arch/powerpc/platforms/powernv/smp.c     |  7 +--
 drivers/cpuidle/cpuidle-powernv.c        | 30 ++++++++++---
 6 files changed, 115 insertions(+), 35 deletions(-)
arch/powerpc/include/asm/processor.h +2 -1
index 68e3bf5..4ead497 100644
@@ -460,7 +460,8 @@ extern int powersave_nap;	/* set if nap mode can be used in idle loop */
 extern unsigned long power7_nap(int check_irq);
 extern unsigned long power7_sleep(void);
 extern unsigned long power7_winkle(void);
-extern unsigned long power9_idle_stop(unsigned long stop_level);
+extern unsigned long power9_idle_stop(unsigned long stop_psscr_val,
+				unsigned long stop_psscr_mask);
 
 extern void flush_instruction_cache(void);
 extern void hard_reset_now(void);
arch/powerpc/kernel/idle_book3s.S +20 -11
index 32d666b..9f77cd4 100644
@@ -40,9 +40,7 @@
 #define _WORC	GPR11
 #define _PTCR	GPR12
 
-#define PSSCR_HV_TEMPLATE	PSSCR_ESL | PSSCR_EC | \
-				PSSCR_PSLL_MASK | PSSCR_TR_MASK | \
-				PSSCR_MTL_MASK
+#define PSSCR_EC_ESL_MASK_SHIFTED          (PSSCR_EC | PSSCR_ESL) >> 16
 
 	.text
 
@@ -247,13 +245,23 @@ enter_winkle:
 	IDLE_STATE_ENTER_SEQ_NORET(PPC_WINKLE)
 
 /*
- * r3 - requested stop state
+ * r3 - PSSCR value corresponding to the requested stop state.
  */
 power_enter_stop:
 /*
+ * Check if we are executing the lite variant with ESL=EC=0
+ */
+	andis.   r4, r3, PSSCR_EC_ESL_MASK_SHIFTED
+	andi.    r3, r3, PSSCR_RL_MASK   /* r3 = requested stop state */
+	cmpdi	 r4, 0
+	bne	 1f
+	IDLE_STATE_ENTER_SEQ(PPC_STOP)
+	li	r3,0  /* Since we didn't lose state, return 0 */
+	b 	pnv_wakeup_noloss
+/*
  * Check if the requested state is a deep idle state.
  */
-	LOAD_REG_ADDRBASE(r5,pnv_first_deep_stop_state)
+1:	LOAD_REG_ADDRBASE(r5,pnv_first_deep_stop_state)
 	ld	r4,ADDROFF(pnv_first_deep_stop_state)(r5)
 	cmpd	r3,r4
 	bge	2f
@@ -330,16 +338,17 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66);		\
 	ld	r3,ORIG_GPR3(r1);	/* Restore original r3 */	\
 20:	nop;
 
-
 /*
- * r3 - requested stop state
+ * r3 - The PSSCR value corresponding to the stop state.
+ * r4 - The PSSCR mask corrresonding to the stop state.
  */
 _GLOBAL(power9_idle_stop)
-	LOAD_REG_IMMEDIATE(r4, PSSCR_HV_TEMPLATE)
-	or	r4,r4,r3
-	mtspr	SPRN_PSSCR, r4
-	li	r4, 1
+	mfspr   r5, SPRN_PSSCR
+	andc    r5, r5, r4
+	or      r3, r3, r5
+	mtspr 	SPRN_PSSCR, r3
 	LOAD_REG_ADDR(r5,power_enter_stop)
+	li	r4, 1
 	b	pnv_powersave_common
 	/* No return */
 /*
arch/powerpc/platforms/powernv/idle.c +64 -12

128 lines not on this page · raw message

arch/powerpc/platforms/powernv/powernv.h +2 -1
index da7c843..6130522 100644
@@ -18,7 +18,8 @@ static inline void pnv_pci_shutdown(void) { }
 #endif
 
 extern u32 pnv_get_supported_cpuidle_states(void);
-extern u64 pnv_deepest_stop_state;
+extern u64 pnv_deepest_stop_psscr_val;
+extern u64 pnv_deepest_stop_psscr_mask;
 
 extern void pnv_lpc_init(void);
 
arch/powerpc/platforms/powernv/smp.c +4 -3
index c789258..55c4aaa 100644
@@ -182,9 +182,10 @@ static void pnv_smp_cpu_kill_self(void)
 
 		ppc64_runlatch_off();
 
-		if (cpu_has_feature(CPU_FTR_ARCH_300))
-			srr1 = power9_idle_stop(pnv_deepest_stop_state);
-		else if (idle_states & OPAL_PM_WINKLE_ENABLED)
+		if (cpu_has_feature(CPU_FTR_ARCH_300)) {
+			srr1 = power9_idle_stop(pnv_deepest_stop_psscr_val,
+						pnv_deepest_stop_psscr_mask);
+		} else if (idle_states & OPAL_PM_WINKLE_ENABLED)
 			srr1 = power7_winkle();
 		else if ((idle_states & OPAL_PM_SLEEP_ENABLED) ||
 				(idle_states & OPAL_PM_SLEEP_ENABLED_ER1))
drivers/cpuidle/cpuidle-powernv.c +23 -7
index f7ca891..6f2bee0 100644
@@ -30,7 +30,11 @@ struct cpuidle_driver powernv_idle_driver = {
 static int max_idle_state;
 static struct cpuidle_state *cpuidle_state_table;
 
-static u64 stop_psscr_table[CPUIDLE_STATE_MAX];
+struct stop_psscr_table {
+	u64 val;
+	u64 mask;
+};
+static struct stop_psscr_table stop_psscr_table[CPUIDLE_STATE_MAX];
 
 static u64 snooze_timeout;
 static bool snooze_timeout_en;
@@ -102,7 +106,8 @@ static int stop_loop(struct cpuidle_device *dev,
 		     int index)
 {
 	ppc64_runlatch_off();
-	power9_idle_stop(stop_psscr_table[index]);
+	power9_idle_stop(stop_psscr_table[index].val,
+			stop_psscr_table[index].mask);
 	ppc64_runlatch_on();
 	return index;
 }
@@ -186,6 +191,7 @@ static int powernv_add_idle_states(void)
 	u32 residency_ns[CPUIDLE_STATE_MAX];
 	u32 flags[CPUIDLE_STATE_MAX];
 	u64 psscr_val[CPUIDLE_STATE_MAX];
+	u64 psscr_mask[CPUIDLE_STATE_MAX];
 	const char *names[CPUIDLE_STATE_MAX];
 	int i, rc;
 
@@ -233,14 +239,22 @@ static int powernv_add_idle_states(void)
 
 	/*
 	 * If the idle states use stop instruction, probe for psscr values
-	 * which are necessary to specify required stop level.
+	 * and psscr mask which are necessary to specify required stop level.
 	 */
-	if (flags[0] & (OPAL_PM_STOP_INST_FAST | OPAL_PM_STOP_INST_DEEP))
+	if (flags[0] & (OPAL_PM_STOP_INST_FAST | OPAL_PM_STOP_INST_DEEP)) {
 		if (of_property_read_u64_array(power_mgt,
 		    "ibm,cpu-idle-state-psscr", psscr_val, dt_idle_states)) {
-			pr_warn("cpuidle-powernv: missing ibm,cpu-idle-states-psscr in DT\n");
+			pr_warn("cpuidle-powernv: missing ibm,cpu-idle-state-psscr in DT\n");
+			goto out;
+		}
+
+		if (of_property_read_u64_array(power_mgt,
+			"ibm,cpu-idle-state-psscr-mask", psscr_mask,
+						dt_idle_states)) {
+			pr_warn("cpuidle-powernv:Missing ibm,cpu-idle-state-psscr-mask in DT\n");
 			goto out;
 		}
+	}
 
 	rc = of_property_read_u32_array(power_mgt,
 		"ibm,cpu-idle-state-residency-ns", residency_ns, dt_idle_states);
@@ -274,7 +288,8 @@ static int powernv_add_idle_states(void)
 			powernv_states[nr_idle_states].flags = 0;
 
 			powernv_states[nr_idle_states].enter = stop_loop;
-			stop_psscr_table[nr_idle_states] = psscr_val[i];
+			stop_psscr_table[nr_idle_states].val = psscr_val[i];
+			stop_psscr_table[nr_idle_states].mask = psscr_mask[i];
 		}
 
 		/*
@@ -299,7 +314,8 @@ static int powernv_add_idle_states(void)
 
 			powernv_states[nr_idle_states].flags = CPUIDLE_FLAG_TIMER_STOP;
 			powernv_states[nr_idle_states].enter = stop_loop;
-			stop_psscr_table[nr_idle_states] = psscr_val[i];
+			stop_psscr_table[nr_idle_states].val = psscr_val[i];
+			stop_psscr_table[nr_idle_states].mask = psscr_mask[i];
 		}
 #endif
 		powernv_states[nr_idle_states].exit_latency =
Signature · 1 line
1.9.4
#3 lore raw

Re: [PATCH 2/2] powernv: Pass PSSCR value and mask to power9_idle_stop

Quoted 14 lines · Gautham R. Shenoy

From: "Gautham R. Shenoy" <[email protected]>

The power9_idle_stop method currently takes only the requested stop level as a parameter and picks up the rest of the PSSCR bits from a hand-coded macro. This is not a very flexible design, especially when the firmware has the capability to communicate the psscr value and the mask associated with a particular stop state via device tree.

This patch modifies the power9_idle_stop API to take as parameters the PSSCR value and the PSSCR mask corresponding to the stop state that needs to be set. These PSSCR value and mask are respectively obtained by parsing the "ibm,cpu-idle-state-psscr" and "ibm,cpu-idle-state-psscr-mask" fields from the device tree.

Quoted 5 lines

In addition to this, the patch adds support for handling stop states for which ESL and EC bits in the PSSCR are zero. As per the architecture, a wakeup from these stop states resumes execution from the subsequent instruction as opposed to waking up at the System Vector.

That looks good.

Quoted 3 lines

This patch depends on the following skiboot patch that exports the PSSCR values and the mask for all the stop states:
https://lists.ozlabs.org/pipermail/skiboot/2016-September/004869.html

But we can't depend on a skiboot patch. The kernel has to cope with running on an old skiboot.

I realise this is for Power9, and we haven't shipped any of them yet. But skiboot has had Power9 support since May, so I don't think we want to declare that all of those skiboot versions are unsupported because of this.

So is there any way we can make this work on skiboot versions that don't have the patch above?

cheers

#4 lore raw

Re: [PATCH 2/2] powernv: Pass PSSCR value and mask to power9_idle_stop

Quoted 22 lines · Michael Ellerman
Quoted 14 lines · Gautham R. Shenoy

From: "Gautham R. Shenoy" <[email protected]>

The power9_idle_stop method currently takes only the requested stop level as a parameter and picks up the rest of the PSSCR bits from a hand-coded macro. This is not a very flexible design, especially when the firmware has the capability to communicate the psscr value and the mask associated with a particular stop state via device tree.

This patch modifies the power9_idle_stop API to take as parameters the PSSCR value and the PSSCR mask corresponding to the stop state that needs to be set. These PSSCR value and mask are respectively obtained by parsing the "ibm,cpu-idle-state-psscr" and "ibm,cpu-idle-state-psscr-mask" fields from the device tree.

Quoted 5 lines

In addition to this, the patch adds support for handling stop states for which ESL and EC bits in the PSSCR are zero. As per the architecture, a wakeup from these stop states resumes execution from the subsequent instruction as opposed to waking up at the System Vector.

Quoted 8 lines

That looks good.

Quoted 3 lines

This patch depends on the following skiboot patch that exports the PSSCR values and the mask for all the stop states:
https://lists.ozlabs.org/pipermail/skiboot/2016-September/004869.html

But we can't depend on a skiboot patch. The kernel has to cope with running on an old skiboot.

Hi, Michael

I think with an older skiboot the flags don't get exported and the new cpuidle (stop state) does not get discovered. I don't think there is any breakage. Gautham am I missing something?

Balbir Singh.

#5 lore raw

Re: [PATCH 2/2] powernv: Pass PSSCR value and mask to power9_idle_stop

Hi Balbir, Michael,

Quoted 25 lines · Balbir Singh
Quoted 22 lines · Michael Ellerman
Quoted 19 lines · Gautham R. Shenoy

From: "Gautham R. Shenoy" <[email protected]>

The power9_idle_stop method currently takes only the requested stop level as a parameter and picks up the rest of the PSSCR bits from a hand-coded macro. This is not a very flexible design, especially when the firmware has the capability to communicate the psscr value and the mask associated with a particular stop state via device tree.

This patch modifies the power9_idle_stop API to take as parameters the PSSCR value and the PSSCR mask corresponding to the stop state that needs to be set. These PSSCR value and mask are respectively obtained by parsing the "ibm,cpu-idle-state-psscr" and "ibm,cpu-idle-state-psscr-mask" fields from the device tree.

In addition to this, the patch adds support for handling stop states for which ESL and EC bits in the PSSCR are zero. As per the architecture, a wakeup from these stop states resumes execution from the subsequent instruction as opposed to waking up at the System Vector.

Quoted 9 lines
Quoted 8 lines

That looks good.

Quoted 3 lines

This patch depends on the following skiboot patch that exports the PSSCR values and the mask for all the stop states:
https://lists.ozlabs.org/pipermail/skiboot/2016-September/004869.html

But we can't depend on a skiboot patch. The kernel has to cope with running on an old skiboot.

Hmm.. We can still do that. The older skiboot only provides the RL field of the PSSCR value for each stop state and the corresponding PSSCR mask is set to 0xF in the older skiboot for all the stop states.

We can insist that the future skiboot sets the ESL, EC, PSLL, TR, MTL and the the RL fields of the PSSCR for any exported stop state. This should be reflected in the psscr_mask of that stop state. Thus, the psscr_mask of any stop state proposed in the future will have:
(PSSCR_ESL_MASK | PSCCR_EC_MASK | PSCCR_PSLL_MASK | PSSCR_TR_MASK | PSSCR_MTL_MASK | PSSCR_RL_MASK) bits set in the skiboot.

To handle the older firmware, we can do something like the following during the discovery of the stop states to mimic the behaviour present in the 4.8 kernel running on older firmware.

=============== drivers/cpuidle/cpuidle-powernv.c ======================= /*
* By default we set the ESL and EC bits in the PSSCR.
* The MTL and PSLL are set to the maximum value possible as per the
* ISA, i.e 15.
* The Transition Rate is set to the Maximum value 3.
*/

#define DEFAULT_PSSCR_VAL      PSSCR_ESL_MASK |			\
			       PSCCR_EC_MASK | PSCCR_PSLL_MASK |\
			       PSSCR_TR_MASK | PSSCR_MTL_MASK
#define DEFAULT_PSSCR_MASK     PSSCR_ESL_MASK |			\
			       PSCCR_EC_MASK | PSCCR_PSLL_MASK |\
			       PSSCR_TR_MASK | PSSCR_MTL_MASK | \
			       PSSCR_RL_MASK

static int powernv_add_idle_states(void)

{
	.
	.
	.
	for (i = 0; i < dt_idle_states; i++) {
	    u64 val, mask;
	    	.	    
		.
		.
		val = (DEFAULT_PSSCR_VAL & ~psscr_mask[i]) | psscr_val[i];
		mask = DEFAULT_PSSCR_MASK | psscr_mask[i];
		stop_psscr_table[nr_idle_states].val = val;
        	stop_psscr_table[nr_idle_states].mask = mask;
	}
}		

============================================================================

Is this approach ok ?

Quoted 5 lines

Hi, Michael

I think with an older skiboot the flags don't get exported and the new cpuidle (stop state) does not get discovered. I don't think there is any breakage. Gautham am I missing something?

In the second verison of the patch, I am not adding any new flags in the skiboot, but explicitly providing the psscr_value and psscr_mask for each stop state. Thus if you see the v2 of the skiboot patch, the lite versions of stop0,stop1 and stop2 are defined with no additional flags.

The second version of the kernel patch wouldn't then work with the older skiboot since the older skiboot exports only the RL field of the psscr_val while the new kernel patch expects all the relavant bits of the psscr_val be set. This would result in a breakage.

Quoted 3 lines

Balbir Singh.

--
Thanks and Regards
gautham.

#6 lore raw

Re: [PATCH 2/2] powernv: Pass PSSCR value and mask to power9_idle_stop

Quoted 84 lines · Gautham R Shenoy
Quoted 25 lines · Balbir Singh
Quoted 22 lines · Michael Ellerman
Quoted 19 lines · Gautham R. Shenoy

From: "Gautham R. Shenoy" <[email protected]>

The power9_idle_stop method currently takes only the requested stop level as a parameter and picks up the rest of the PSSCR bits from a hand-coded macro. This is not a very flexible design, especially when the firmware has the capability to communicate the psscr value and the mask associated with a particular stop state via device tree.

This patch modifies the power9_idle_stop API to take as parameters the PSSCR value and the PSSCR mask corresponding to the stop state that needs to be set. These PSSCR value and mask are respectively obtained by parsing the "ibm,cpu-idle-state-psscr" and "ibm,cpu-idle-state-psscr-mask" fields from the device tree.

In addition to this, the patch adds support for handling stop states for which ESL and EC bits in the PSSCR are zero. As per the architecture, a wakeup from these stop states resumes execution from the subsequent instruction as opposed to waking up at the System Vector.

Quoted 9 lines
Quoted 8 lines

That looks good.

Quoted 3 lines

This patch depends on the following skiboot patch that exports the PSSCR values and the mask for all the stop states:
https://lists.ozlabs.org/pipermail/skiboot/2016-September/004869.html

But we can't depend on a skiboot patch. The kernel has to cope with running on an old skiboot.

Hmm.. We can still do that. The older skiboot only provides the RL field of the PSSCR value for each stop state and the corresponding PSSCR mask is set to 0xF in the older skiboot for all the stop states.

We can insist that the future skiboot sets the ESL, EC, PSLL, TR, MTL and the the RL fields of the PSSCR for any exported stop state. This should be reflected in the psscr_mask of that stop state. Thus, the psscr_mask of any stop state proposed in the future will have:
(PSSCR_ESL_MASK | PSCCR_EC_MASK | PSCCR_PSLL_MASK | PSSCR_TR_MASK | PSSCR_MTL_MASK | PSSCR_RL_MASK) bits set in the skiboot.

To handle the older firmware, we can do something like the following during the discovery of the stop states to mimic the behaviour present in the 4.8 kernel running on older firmware.

=============== drivers/cpuidle/cpuidle-powernv.c ======================= /*
* By default we set the ESL and EC bits in the PSSCR.
* The MTL and PSLL are set to the maximum value possible as per the
* ISA, i.e 15.
* The Transition Rate is set to the Maximum value 3.
*/

#define DEFAULT_PSSCR_VAL      PSSCR_ESL_MASK |			\
			       PSCCR_EC_MASK | PSCCR_PSLL_MASK |\
			       PSSCR_TR_MASK | PSSCR_MTL_MASK
#define DEFAULT_PSSCR_MASK     PSSCR_ESL_MASK |			\
			       PSCCR_EC_MASK | PSCCR_PSLL_MASK |\
			       PSSCR_TR_MASK | PSSCR_MTL_MASK | \
			       PSSCR_RL_MASK

static int powernv_add_idle_states(void)

{
	.
	.
	.
	for (i = 0; i < dt_idle_states; i++) {
	    u64 val, mask;
	    	.	    
		.
		.
		val = (DEFAULT_PSSCR_VAL & ~psscr_mask[i]) | psscr_val[i];
		mask = DEFAULT_PSSCR_MASK | psscr_mask[i];
		stop_psscr_table[nr_idle_states].val = val;
        	stop_psscr_table[nr_idle_states].mask = mask;
	}
}		
Quoted 4 lines

============================================================================

Is this approach ok ?

What if we just treat the 0xF state from firmware as special and set it to DEFAULT_PSSCR_MASK in that case? That deals with old skiboot, new kernel, and sets a pretty small special case that's easy to track into the future as something we should watch out for.

Additionally, if we make skiboot set sane values in ~DEFAULT_PSSCR_MASK for valid fields in PSSCR on boot/(also kexec?), then
we should end up in a situation where everything works with everything (even if you don't get the best power saving). Specifically, new skiboot, old kernel... but it looks like there's nothing currently missing there

Should this patch also have Fixes: 3005c597ba4 and CC to stable?

Signature · 2 lines
Stewart Smith
OPAL Architect, IBM.
#7 lore raw

Re: [PATCH 2/2] powernv: Pass PSSCR value and mask to power9_idle_stop

Hi Stewart,

Quoted 5 lines · Stewart Smith

What if we just treat the 0xF state from firmware as special and set it to DEFAULT_PSSCR_MASK in that case? That deals with old skiboot, new kernel, and sets a pretty small special case that's easy to track into the future as something we should watch out for.

Yes, that will work. I will use this approach in the next version.

Quoted 7 lines

Additionally, if we make skiboot set sane values in ~DEFAULT_PSSCR_MASK for valid fields in PSSCR on boot/(also kexec?), then
we should end up in a situation where everything works with everything (even if you don't get the best power saving). Specifically, new skiboot, old kernel... but it looks like there's nothing currently missing there

No we're not missing much there. From the code of the old kernel, I see that if we pass the psscr_val instead of just the RL to power9_idle_stop(), we end up saving all the SPRs in power_enter_stop() even if the psscr_val didn't correspond to a deep idle state.

But that should be ok.

Quoted 2 lines

Should this patch also have Fixes: 3005c597ba4 and CC to stable?

Yes, thanks for pointing it out. I shall mark the next version to stable.

Quoted 4 lines
Signature · 3 lines
-- 
Stewart Smith
OPAL Architect, IBM.

--
Thanks and Regards
gautham.