powerpc: inline ppc64_runlatch_off

I'm sick of seeing ppc64_runlatch_off in our profiles, so inline the heavily used part of it into the callers. To avoid a mess of circular includes I didn't add it as an inline function.

Signed-off-by Anton Blanchard <[email protected]>

---

Index: powerpc.git/arch/powerpc/include/asm/reg.h
===================================================================

arch/powerpc/include/asm/reg.h +8 -1

@@ -951,7 +951,14 @@
 #ifdef CONFIG_PPC64
 
 extern void ppc64_runlatch_on(void);
-extern void ppc64_runlatch_off(void);
+extern void __ppc64_runlatch_off(void);
+
+#define ppc64_runlatch_off()					\
+	do {							\
+		if (cpu_has_feature(CPU_FTR_CTRL) &&		\
+		    test_thread_flag(TIF_RUNLATCH))		\
+			__ppc64_runlatch_off();			\
+	} while (0);
 
 extern unsigned long scom970_read(unsigned int address);
 extern void scom970_write(unsigned int address, unsigned long value);

Index: powerpc.git/arch/powerpc/kernel/process.c
===================================================================

arch/powerpc/kernel/process.c +6 -8

@@ -1198,19 +1198,17 @@ void ppc64_runlatch_on(void)
 	}
 }
 
-void ppc64_runlatch_off(void)
+void __ppc64_runlatch_off(void)
 {
 	unsigned long ctrl;
 
-	if (cpu_has_feature(CPU_FTR_CTRL) && test_thread_flag(TIF_RUNLATCH)) {
-		HMT_medium();
+	HMT_medium();
 
-		clear_thread_flag(TIF_RUNLATCH);
+	clear_thread_flag(TIF_RUNLATCH);
 
-		ctrl = mfspr(SPRN_CTRLF);
-		ctrl &= ~CTRL_RUNLATCH;
-		mtspr(SPRN_CTRLT, ctrl);
-	}
+	ctrl = mfspr(SPRN_CTRLF);
+	ctrl &= ~CTRL_RUNLATCH;
+	mtspr(SPRN_CTRLT, ctrl);
 }
 #endif
 

Quoted 3 lines

I'm sick of seeing ppc64_runlatch_off in our profiles, so inline the heavily used part of it into the callers. To avoid a mess of circular includes I didn't add it as an inline function.

Considering that it's just an asm instruction or two, should we make it inline asm and have it NOPed out instead using the feature sections ?

Unfortunately we still need to prevent continual writes to it with a per thread flag because on some CPUs a write to the SPR in low priority mode will stall another SMT thread. So we could get rid of the cpu feature comparison but we would still need the thread bit check (or perhaps replace it with a per cpu variable).

remind me why we need to do that runlatch thing on these CPUs at all ?

I'm sick of seeing ppc64_runlatch_off in our profiles, so inline the heavily used part of it into the callers. To avoid a mess of circular includes I didn't add it as an inline function.

Signed-off-by Anton Blanchard <[email protected]>

---

Index: powerpc.git/arch/powerpc/include/asm/reg.h
===================================================================

arch/powerpc/include/asm/reg.h +0 -0

@@ -951,7 +951,14 @@
 #ifdef CONFIG_PPC64
 

No semicolon here.

Hi Olof,

Quoted 1 line

No semicolon here.

Nice catch!

Anton

I'm sick of seeing ppc64_runlatch_off in our profiles, so inline it into the callers. To avoid a mess of circular includes I didn't add it as an inline function.

Signed-off-by Anton Blanchard <[email protected]>