Skip to contentSkip to conversation

powerpc: Fix buglet with MMU hash management

PATCH In review Benjamin Herrenschmidt 2 participants 3 messages May 30, 2006 PowerPC

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 powerpc: Fix buglet with MMU hash management

Our MMU hash management code would not set the "C" bit (changed bit) in the hardware PTE when updating a RO PTE into a RW PTE. That would cause the hardware to possibly to a write back to the hash table to set it on the first store access, which in addition to being a performance issue, might also hit a bug when running with native hash management (non-HV) as our code is specifically optimized for the case where no write back happens.

Thus there is a very small therocial window were a hash PTE can become corrupted if that HPTE has just been upgraded to read write, a store access happens on it, and that races with another processor evicting that same slot. Since eviction (caused by an almost full hash) is extremely rare, the bug is very unlikely to happen fortunately.

This fixes by allowing the updating of the protection bits in the native hash handling to also set (but not clear) the "C" bit, and, in order to also improve performances in the general case, by always setting that bit on newly inserted hash PTE so that writeback really never happens.

arch/powerpc/mm/hash_low_64.S +3 -0
Index: linux-work/arch/powerpc/mm/hash_low_64.S
===================================================================
@@ -136,6 +136,7 @@ _GLOBAL(__hash_page_4K)
 	and	r0,r0,r4		/* _PAGE_RW & _PAGE_DIRTY ->r0 bit 30*/
 	andc	r0,r30,r0		/* r0 = pte & ~r0 */
 	rlwimi	r3,r0,32-1,31,31	/* Insert result into PP lsb */
+	ori	r3,r3,HPTE_R_C		/* Always add "C" bit for perf. */
 
 	/* We eventually do the icache sync here (maybe inline that
 	 * code rather than call a C function...) 
@@ -400,6 +401,7 @@ _GLOBAL(__hash_page_4K)
 	and	r0,r0,r4		/* _PAGE_RW & _PAGE_DIRTY ->r0 bit 30*/
 	andc	r0,r30,r0		/* r0 = pte & ~r0 */
 	rlwimi	r3,r0,32-1,31,31	/* Insert result into PP lsb */
+	ori	r3,r3,HPTE_R_C		/* Always add "C" bit for perf. */
 
 	/* We eventually do the icache sync here (maybe inline that
 	 * code rather than call a C function...)
@@ -671,6 +673,7 @@ _GLOBAL(__hash_page_64K)
 	and	r0,r0,r4		/* _PAGE_RW & _PAGE_DIRTY ->r0 bit 30*/
 	andc	r0,r30,r0		/* r0 = pte & ~r0 */
 	rlwimi	r3,r0,32-1,31,31	/* Insert result into PP lsb */
+	ori	r3,r3,HPTE_R_C		/* Always add "C" bit for perf. */
 
 	/* We eventually do the icache sync here (maybe inline that
 	 * code rather than call a C function...)
arch/powerpc/mm/hash_native_64.c +1 -1
Index: linux-work/arch/powerpc/mm/hash_native_64.c
===================================================================
@@ -238,7 +238,7 @@ static long native_hpte_updatepp(unsigne
 		DBG_LOW(" -> hit\n");
 		/* Update the HPTE */
 		hptep->r = (hptep->r & ~(HPTE_R_PP | HPTE_R_N)) |
-			(newpp & (HPTE_R_PP | HPTE_R_N));
+			(newpp & (HPTE_R_PP | HPTE_R_N | HPTE_R_C));
 		native_unlock_hpte(hptep);
 	}
 
include/asm-powerpc/mmu.h +2 -0
Index: linux-work/include/asm-powerpc/mmu.h
===================================================================
@@ -96,6 +96,8 @@ extern char initial_stab[];
 #define HPTE_R_FLAGS		ASM_CONST(0x00000000000003ff)
 #define HPTE_R_PP		ASM_CONST(0x0000000000000003)
 #define HPTE_R_N		ASM_CONST(0x0000000000000004)
+#define HPTE_R_C		ASM_CONST(0x0000000000000080)
+#define HPTE_R_R		ASM_CONST(0x0000000000000100)
 
 /* Values for PP (assumes Ks=0, Kp=1) */
 /* pp0 will always be 0 for linux     */
#1 lore raw
Quoted 4 lines · Benjamin Herrenschmidt

This fixes by allowing the updating of the protection bits in the native hash handling to also set (but not clear) the "C" bit, and, in order to also improve performances in the general case, by always setting that bit on newly inserted hash PTE so that writeback really never happens.

Good catch!

However, you'll have a writeback on the R bit too, you should set that together with C.

-Olof

#2 lore raw
Quoted 11 lines · Olof Johansson
Quoted 4 lines · Benjamin Herrenschmidt

This fixes by allowing the updating of the protection bits in the native hash handling to also set (but not clear) the "C" bit, and, in order to also improve performances in the general case, by always setting that bit on newly inserted hash PTE so that writeback really never happens.

Good catch!

However, you'll have a writeback on the R bit too, you should set that together with C.

Nope, R is always set at PTE insertion (it's _PAGE_ACCESSED)

Ben.