Skip to contentSkip to conversation

powerpc/mm: page fault handler cleaning

PATCH In review Laurent Dufour 4 participants 11 messages Feb 14, 2017 – Mar 21, 2017 PowerPC

lore
Share & tools
lore
#0 lore raw

cover powerpc/mm: page fault handler cleaning

This series attempts to clean the page fault handler in the way it has been done previously for the x86 architecture [1].

The goal is to manage the mmap_sem earlier and only in do_page_fault(). This done by handling the retry case earlier, before handling the error case. This way the semaphore can be released earlier and the error path processed without holding it.

The first patch is just moving a unlock to the caller of the service, which as no functional impact.

The second patch is handling the retry case earlier in do_page_fault(). This is where most the change are done, but I was conservative here, not changing the use of mm_fault_error() in the case of the second retry. It may be smarter to handle that case separately but this may create duplicate code.

The last patch is moving up semaphore releasing from mm_fault_error() to do_page_fault().

[1] see commits from Linus Torvalds
26178ec11ef3 ("x86: mm: consolidate VM_FAULT_RETRY handling") 7fb08eca4527 ("x86: mm: move mmap_sem unlock from mm_fault_error() to
caller")

Laurent Dufour (3):
powerpc/mm: move mmap_sem unlock up from do_sigbus powerpc/mm: handle VM_FAULT_RETRY earlier powerpc/mm: move mmap_sem unlocking in do_page_fault()

 arch/powerpc/mm/fault.c | 82 ++++++++++++++++++++++++-------------------------
 1 file changed, 40 insertions(+), 42 deletions(-)
Signature · 1 line
2.7.4
#1 lore raw

PATCH 1/3 powerpc/mm: move mmap_sem unlock up from do_sigbus

Move mmap_sem releasing in the do_sigbus()'s unique caller : mm_fault_error()

No functional changes.

---
 arch/powerpc/mm/fault.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
arch/powerpc/mm/fault.c +3 -3
index 62a50d6d1053..ee09604bbe12 100644
@@ -119,8 +119,6 @@ static int do_sigbus(struct pt_regs *regs, unsigned long address,
 	siginfo_t info;
 	unsigned int lsb = 0;
 
-	up_read(&current->mm->mmap_sem);
-
 	if (!user_mode(regs))
 		return MM_FAULT_ERR(SIGBUS);
 
@@ -184,8 +182,10 @@ static int mm_fault_error(struct pt_regs *regs, unsigned long addr, int fault)
 		return MM_FAULT_RETURN;
 	}
 
-	if (fault & (VM_FAULT_SIGBUS|VM_FAULT_HWPOISON|VM_FAULT_HWPOISON_LARGE))
+	if (fault & (VM_FAULT_SIGBUS|VM_FAULT_HWPOISON|VM_FAULT_HWPOISON_LARGE)) {
+		up_read(&current->mm->mmap_sem);
 		return do_sigbus(regs, addr, fault);
+	}
 
 	/* We don't understand the fault code, this is fatal */
 	BUG();
Signature · 1 line
2.7.4
#4 lore raw

PATCH 2/3 powerpc/mm: handle VM_FAULT_RETRY earlier

In reply to Laurent Dufour

In do_page_fault() if handle_mm_fault() returns VM_FAULT_RETRY, retry the page fault handling before anything else.

This would simplify the handling of the mmap_sem lock in this part of the code.

---
 arch/powerpc/mm/fault.c | 67 ++++++++++++++++++++++++++++---------------------
 1 file changed, 38 insertions(+), 29 deletions(-)
arch/powerpc/mm/fault.c +38 -29
index ee09604bbe12..2a6bc7e6e69a 100644
@@ -434,6 +434,26 @@ int do_page_fault(struct pt_regs *regs, unsigned long address,
 	 * the fault.
 	 */
 	fault = handle_mm_fault(vma, address, flags);
+
+	/*
+	 * Handle the retry right now, the mmap_sem has been released in that
+	 * case.
+	 */
+	if (unlikely(fault & VM_FAULT_RETRY)) {
+		/* We retry only once */
+		if (flags & FAULT_FLAG_ALLOW_RETRY) {
+			/*
+			 * Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk
+			 * of starvation.
+			 */
+			flags &= ~FAULT_FLAG_ALLOW_RETRY;
+			flags |= FAULT_FLAG_TRIED;
+			if (!fatal_signal_pending(current))
+				goto retry;
+		}
+		/* We will enter mm_fault_error() below */
+	}
+
 	if (unlikely(fault & (VM_FAULT_RETRY|VM_FAULT_ERROR))) {
 		if (fault & VM_FAULT_SIGSEGV)
 			goto bad_area;
@@ -445,38 +465,27 @@ int do_page_fault(struct pt_regs *regs, unsigned long address,
 	}
 
 	/*
-	 * Major/minor page fault accounting is only done on the
-	 * initial attempt. If we go through a retry, it is extremely
-	 * likely that the page will be found in page cache at that point.
+	 * Major/minor page fault accounting.
 	 */
-	if (flags & FAULT_FLAG_ALLOW_RETRY) {
-		if (fault & VM_FAULT_MAJOR) {
-			current->maj_flt++;
-			perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MAJ, 1,
-				      regs, address);
+	if (fault & VM_FAULT_MAJOR) {
+		current->maj_flt++;
+		perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MAJ, 1,
+			      regs, address);
 #ifdef CONFIG_PPC_SMLPAR
-			if (firmware_has_feature(FW_FEATURE_CMO)) {
-				u32 page_ins;
-
-				preempt_disable();
-				page_ins = be32_to_cpu(get_lppaca()->page_ins);
-				page_ins += 1 << PAGE_FACTOR;
-				get_lppaca()->page_ins = cpu_to_be32(page_ins);
-				preempt_enable();
-			}
-#endif /* CONFIG_PPC_SMLPAR */
-		} else {
-			current->min_flt++;
-			perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MIN, 1,
-				      regs, address);
-		}
-		if (fault & VM_FAULT_RETRY) {
-			/* Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk
-			 * of starvation. */
-			flags &= ~FAULT_FLAG_ALLOW_RETRY;
-			flags |= FAULT_FLAG_TRIED;
-			goto retry;
+		if (firmware_has_feature(FW_FEATURE_CMO)) {
+			u32 page_ins;
+
+			preempt_disable();
+			page_ins = be32_to_cpu(get_lppaca()->page_ins);
+			page_ins += 1 << PAGE_FACTOR;
+			get_lppaca()->page_ins = cpu_to_be32(page_ins);
+			preempt_enable();
 		}
+#endif /* CONFIG_PPC_SMLPAR */
+	} else {
+		current->min_flt++;
+		perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MIN, 1,
+			      regs, address);
 	}
 
 	up_read(&mm->mmap_sem);
Signature · 1 line
2.7.4
#7 lore raw

PATCH 3/3 powerpc/mm: move mmap_sem unlocking in do_page_fault()

In reply to Laurent Dufour

Since the fault retry is now handled earlier, we can release the mmap_sem lock earlier too and remove later unlocking previously done in mm_fault_error().

---
 arch/powerpc/mm/fault.c | 19 ++++---------------
 1 file changed, 4 insertions(+), 15 deletions(-)
arch/powerpc/mm/fault.c +4 -15
index 2a6bc7e6e69a..21e06cce8984 100644
@@ -151,13 +151,6 @@ static int mm_fault_error(struct pt_regs *regs, unsigned long addr, int fault)
 	 * continue the pagefault.
 	 */
 	if (fatal_signal_pending(current)) {
-		/*
-		 * If we have retry set, the mmap semaphore will have
-		 * alrady been released in __lock_page_or_retry(). Else
-		 * we release it now.
-		 */
-		if (!(fault & VM_FAULT_RETRY))
-			up_read(&current->mm->mmap_sem);
 		/* Coming from kernel, we need to deal with uaccess fixups */
 		if (user_mode(regs))
 			return MM_FAULT_RETURN;
@@ -170,8 +163,6 @@ static int mm_fault_error(struct pt_regs *regs, unsigned long addr, int fault)
 
 	/* Out of memory */
 	if (fault & VM_FAULT_OOM) {
-		up_read(&current->mm->mmap_sem);
-
 		/*
 		 * We ran out of memory, or some other thing happened to us that
 		 * made us unable to handle the page fault gracefully.
@@ -182,10 +173,8 @@ static int mm_fault_error(struct pt_regs *regs, unsigned long addr, int fault)
 		return MM_FAULT_RETURN;
 	}
 
-	if (fault & (VM_FAULT_SIGBUS|VM_FAULT_HWPOISON|VM_FAULT_HWPOISON_LARGE)) {
-		up_read(&current->mm->mmap_sem);
+	if (fault & (VM_FAULT_SIGBUS|VM_FAULT_HWPOISON|VM_FAULT_HWPOISON_LARGE))
 		return do_sigbus(regs, addr, fault);
-	}
 
 	/* We don't understand the fault code, this is fatal */
 	BUG();
@@ -452,11 +441,12 @@ int do_page_fault(struct pt_regs *regs, unsigned long address,
 				goto retry;
 		}
 		/* We will enter mm_fault_error() below */
-	}
+	} else
+		up_read(&current->mm->mmap_sem);
 
 	if (unlikely(fault & (VM_FAULT_RETRY|VM_FAULT_ERROR))) {
 		if (fault & VM_FAULT_SIGSEGV)
-			goto bad_area;
+			goto bad_area_nosemaphore;
 		rc = mm_fault_error(regs, address, fault);
 		if (rc >= MM_FAULT_RETURN)
 			goto bail;
@@ -488,7 +478,6 @@ int do_page_fault(struct pt_regs *regs, unsigned long address,
 			      regs, address);
 	}
 
-	up_read(&mm->mmap_sem);
 	goto bail;
 
 bad_area:
Signature · 1 line
2.7.4
#9 lore raw
In reply to Laurent Dufour

Kindly ping...

Quoted 33 lines · Laurent Dufour

This series attempts to clean the page fault handler in the way it has been done previously for the x86 architecture [1].

The goal is to manage the mmap_sem earlier and only in do_page_fault(). This done by handling the retry case earlier, before handling the error case. This way the semaphore can be released earlier and the error path processed without holding it.

The first patch is just moving a unlock to the caller of the service, which as no functional impact.

The second patch is handling the retry case earlier in do_page_fault(). This is where most the change are done, but I was conservative here, not changing the use of mm_fault_error() in the case of the second retry. It may be smarter to handle that case separately but this may create duplicate code.

The last patch is moving up semaphore releasing from mm_fault_error() to do_page_fault().

[1] see commits from Linus Torvalds
26178ec11ef3 ("x86: mm: consolidate VM_FAULT_RETRY handling") 7fb08eca4527 ("x86: mm: move mmap_sem unlock from mm_fault_error() to
caller")

Laurent Dufour (3):
powerpc/mm: move mmap_sem unlock up from do_sigbus powerpc/mm: handle VM_FAULT_RETRY earlier powerpc/mm: move mmap_sem unlocking in do_page_fault()

 arch/powerpc/mm/fault.c | 82 ++++++++++++++++++++++++-------------------------
 1 file changed, 40 insertions(+), 42 deletions(-)
#10 lore raw
Quoted 1 line · Laurent Dufour

Kindly ping...

On my list for 4.12, but still dealing with the mess that is 4.11.

Will review it in the next few weeks, unless someone else beats me to it ... ;)

cheers

Reviewed-by on v1 1/3
#2 lore raw

Re: [PATCH 1/3] powerpc/mm: move mmap_sem unlock up from do_sigbus

In reply to Laurent Dufour
Quoted 4 lines · Laurent Dufour

Move mmap_sem releasing in the do_sigbus()'s unique caller : mm_fault_error()

No functional changes.

Quoted 32 lines
---
 arch/powerpc/mm/fault.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
arch/powerpc/mm/fault.c +3 -3
index 62a50d6d1053..ee09604bbe12 100644
@@ -119,8 +119,6 @@ static int do_sigbus(struct pt_regs *regs, unsigned long address,
 	siginfo_t info;
 	unsigned int lsb = 0;
 
-	up_read(&current->mm->mmap_sem);
-
 	if (!user_mode(regs))
 		return MM_FAULT_ERR(SIGBUS);
 
@@ -184,8 +182,10 @@ static int mm_fault_error(struct pt_regs *regs, unsigned long addr, int fault)
 		return MM_FAULT_RETURN;
 	}
 
-	if (fault & (VM_FAULT_SIGBUS|VM_FAULT_HWPOISON|VM_FAULT_HWPOISON_LARGE))
+	if (fault & (VM_FAULT_SIGBUS|VM_FAULT_HWPOISON|VM_FAULT_HWPOISON_LARGE)) {
+		up_read(&current->mm->mmap_sem);
 		return do_sigbus(regs, addr, fault);
+	}
 
 	/* We don't understand the fault code, this is fatal */
 	BUG();
Signature · 1 line
2.7.4
Reviewed-by on v1 2/3
#5 lore raw

Re: [PATCH 2/3] powerpc/mm: handle VM_FAULT_RETRY earlier

In reply to Laurent Dufour
Quoted 43 lines · Laurent Dufour

In do_page_fault() if handle_mm_fault() returns VM_FAULT_RETRY, retry the page fault handling before anything else.

This would simplify the handling of the mmap_sem lock in this part of the code.

---
 arch/powerpc/mm/fault.c | 67 ++++++++++++++++++++++++++++---------------------
 1 file changed, 38 insertions(+), 29 deletions(-)
arch/powerpc/mm/fault.c +20 -0
index ee09604bbe12..2a6bc7e6e69a 100644
@@ -434,6 +434,26 @@ int do_page_fault(struct pt_regs *regs, unsigned long address,
 	 * the fault.
 	 */
 	fault = handle_mm_fault(vma, address, flags);
+
+	/*
+	 * Handle the retry right now, the mmap_sem has been released in that
+	 * case.
+	 */
+	if (unlikely(fault & VM_FAULT_RETRY)) {
+		/* We retry only once */
+		if (flags & FAULT_FLAG_ALLOW_RETRY) {
+			/*
+			 * Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk
+			 * of starvation.
+			 */
+			flags &= ~FAULT_FLAG_ALLOW_RETRY;
+			flags |= FAULT_FLAG_TRIED;
+			if (!fatal_signal_pending(current))
+				goto retry;
+		}
+		/* We will enter mm_fault_error() below */
+	}
+
 	if (unlikely(fault & (VM_FAULT_RETRY|VM_FAULT_ERROR))) {
 		if (fault & VM_FAULT_SIGSEGV)
 			goto bad_area;

arch/powerpc/mm/fault.c int do_page_fault(struct pt_regs *regs, unsigned long address, · line 465

Quoted 1 line
arch/powerpc/mm/fault.c +0 -0
@@ -445,38 +465,27 @@ int do_page_fault(struct pt_regs *regs, unsigned long address,
 	}

We could make it further simpler, by handling the FAULT_RETRY without FLAG_ALLOW_RETRY set earlier. But i guess that can be done later ?

Quoted 57 lines
arch/powerpc/mm/fault.c +18 -29
 	/*
-	 * Major/minor page fault accounting is only done on the
-	 * initial attempt. If we go through a retry, it is extremely
-	 * likely that the page will be found in page cache at that point.
+	 * Major/minor page fault accounting.
 	 */
-	if (flags & FAULT_FLAG_ALLOW_RETRY) {
-		if (fault & VM_FAULT_MAJOR) {
-			current->maj_flt++;
-			perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MAJ, 1,
-				      regs, address);
+	if (fault & VM_FAULT_MAJOR) {
+		current->maj_flt++;
+		perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MAJ, 1,
+			      regs, address);
 #ifdef CONFIG_PPC_SMLPAR
-			if (firmware_has_feature(FW_FEATURE_CMO)) {
-				u32 page_ins;
-
-				preempt_disable();
-				page_ins = be32_to_cpu(get_lppaca()->page_ins);
-				page_ins += 1 << PAGE_FACTOR;
-				get_lppaca()->page_ins = cpu_to_be32(page_ins);
-				preempt_enable();
-			}
-#endif /* CONFIG_PPC_SMLPAR */
-		} else {
-			current->min_flt++;
-			perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MIN, 1,
-				      regs, address);
-		}
-		if (fault & VM_FAULT_RETRY) {
-			/* Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk
-			 * of starvation. */
-			flags &= ~FAULT_FLAG_ALLOW_RETRY;
-			flags |= FAULT_FLAG_TRIED;
-			goto retry;
+		if (firmware_has_feature(FW_FEATURE_CMO)) {
+			u32 page_ins;
+
+			preempt_disable();
+			page_ins = be32_to_cpu(get_lppaca()->page_ins);
+			page_ins += 1 << PAGE_FACTOR;
+			get_lppaca()->page_ins = cpu_to_be32(page_ins);
+			preempt_enable();
 		}
+#endif /* CONFIG_PPC_SMLPAR */
+	} else {
+		current->min_flt++;
+		perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MIN, 1,
+			      regs, address);
 	}
 
 	up_read(&mm->mmap_sem);
Signature · 1 line
2.7.4
Reviewed-by on v1 3/3
#8 lore raw

Re: [PATCH 3/3] powerpc/mm: move mmap_sem unlocking in do_page_fault()

In reply to Laurent Dufour
Quoted 4 lines · Laurent Dufour

Since the fault retry is now handled earlier, we can release the mmap_sem lock earlier too and remove later unlocking previously done in mm_fault_error().

Quoted 69 lines
---
 arch/powerpc/mm/fault.c | 19 ++++---------------
 1 file changed, 4 insertions(+), 15 deletions(-)
arch/powerpc/mm/fault.c +4 -15
index 2a6bc7e6e69a..21e06cce8984 100644
@@ -151,13 +151,6 @@ static int mm_fault_error(struct pt_regs *regs, unsigned long addr, int fault)
 	 * continue the pagefault.
 	 */
 	if (fatal_signal_pending(current)) {
-		/*
-		 * If we have retry set, the mmap semaphore will have
-		 * alrady been released in __lock_page_or_retry(). Else
-		 * we release it now.
-		 */
-		if (!(fault & VM_FAULT_RETRY))
-			up_read(&current->mm->mmap_sem);
 		/* Coming from kernel, we need to deal with uaccess fixups */
 		if (user_mode(regs))
 			return MM_FAULT_RETURN;
@@ -170,8 +163,6 @@ static int mm_fault_error(struct pt_regs *regs, unsigned long addr, int fault)
 
 	/* Out of memory */
 	if (fault & VM_FAULT_OOM) {
-		up_read(&current->mm->mmap_sem);
-
 		/*
 		 * We ran out of memory, or some other thing happened to us that
 		 * made us unable to handle the page fault gracefully.
@@ -182,10 +173,8 @@ static int mm_fault_error(struct pt_regs *regs, unsigned long addr, int fault)
 		return MM_FAULT_RETURN;
 	}
 
-	if (fault & (VM_FAULT_SIGBUS|VM_FAULT_HWPOISON|VM_FAULT_HWPOISON_LARGE)) {
-		up_read(&current->mm->mmap_sem);
+	if (fault & (VM_FAULT_SIGBUS|VM_FAULT_HWPOISON|VM_FAULT_HWPOISON_LARGE))
 		return do_sigbus(regs, addr, fault);
-	}
 
 	/* We don't understand the fault code, this is fatal */
 	BUG();
@@ -452,11 +441,12 @@ int do_page_fault(struct pt_regs *regs, unsigned long address,
 				goto retry;
 		}
 		/* We will enter mm_fault_error() below */
-	}
+	} else
+		up_read(&current->mm->mmap_sem);
 
 	if (unlikely(fault & (VM_FAULT_RETRY|VM_FAULT_ERROR))) {
 		if (fault & VM_FAULT_SIGSEGV)
-			goto bad_area;
+			goto bad_area_nosemaphore;
 		rc = mm_fault_error(regs, address, fault);
 		if (rc >= MM_FAULT_RETURN)
 			goto bail;
@@ -488,7 +478,6 @@ int do_page_fault(struct pt_regs *regs, unsigned long address,
 			      regs, address);
 	}
 
-	up_read(&mm->mmap_sem);
 	goto bail;
 
 bad_area:
Signature · 1 line
2.7.4
#6 lore raw

Re: [PATCH 2/3] powerpc/mm: handle VM_FAULT_RETRY earlier

In reply to Aneesh Kumar K.V
Quoted 41 lines · Aneesh Kumar K.V
Quoted 39 lines · Laurent Dufour

In do_page_fault() if handle_mm_fault() returns VM_FAULT_RETRY, retry the page fault handling before anything else.

This would simplify the handling of the mmap_sem lock in this part of the code.

---
 arch/powerpc/mm/fault.c | 67 ++++++++++++++++++++++++++++---------------------
 1 file changed, 38 insertions(+), 29 deletions(-)
arch/powerpc/mm/fault.c +20 -0
index ee09604bbe12..2a6bc7e6e69a 100644
@@ -434,6 +434,26 @@ int do_page_fault(struct pt_regs *regs, unsigned long address,
 	 * the fault.
 	 */
 	fault = handle_mm_fault(vma, address, flags);
+
+	/*
+	 * Handle the retry right now, the mmap_sem has been released in that
+	 * case.
+	 */
+	if (unlikely(fault & VM_FAULT_RETRY)) {
+		/* We retry only once */
+		if (flags & FAULT_FLAG_ALLOW_RETRY) {
+			/*
+			 * Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk
+			 * of starvation.
+			 */
+			flags &= ~FAULT_FLAG_ALLOW_RETRY;
+			flags |= FAULT_FLAG_TRIED;
+			if (!fatal_signal_pending(current))
+				goto retry;
+		}
+		/* We will enter mm_fault_error() below */
+	}
+
Quoted 8 lines

arch/powerpc/mm/fault.c int do_page_fault(struct pt_regs *regs, unsigned long address, · line 465

Quoted 5 lines
arch/powerpc/mm/fault.c +0 -0
 	if (unlikely(fault & (VM_FAULT_RETRY|VM_FAULT_ERROR))) {
 		if (fault & VM_FAULT_SIGSEGV)
 			goto bad_area;
@@ -445,38 +465,27 @@ int do_page_fault(struct pt_regs *regs, unsigned long address,
 	}

We could make it further simpler, by handling the FAULT_RETRY without FLAG_ALLOW_RETRY set earlier. But i guess that can be done later ?

Thanks for the review,

I agree that double checking against VM_FAULT_RETRY is confusing here.

But handling all the retry path in the first if() statement means that we'll have to handle part of the mm_fault_error() code and segv here... Unless we can't identify what is really relevant in that retry path.

It would take time to review all that tricky part, but I agree it should be simplified later.

Quoted 61 lines
Quoted 57 lines
arch/powerpc/mm/fault.c +18 -29
 	/*
-	 * Major/minor page fault accounting is only done on the
-	 * initial attempt. If we go through a retry, it is extremely
-	 * likely that the page will be found in page cache at that point.
+	 * Major/minor page fault accounting.
 	 */
-	if (flags & FAULT_FLAG_ALLOW_RETRY) {
-		if (fault & VM_FAULT_MAJOR) {
-			current->maj_flt++;
-			perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MAJ, 1,
-				      regs, address);
+	if (fault & VM_FAULT_MAJOR) {
+		current->maj_flt++;
+		perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MAJ, 1,
+			      regs, address);
 #ifdef CONFIG_PPC_SMLPAR
-			if (firmware_has_feature(FW_FEATURE_CMO)) {
-				u32 page_ins;
-
-				preempt_disable();
-				page_ins = be32_to_cpu(get_lppaca()->page_ins);
-				page_ins += 1 << PAGE_FACTOR;
-				get_lppaca()->page_ins = cpu_to_be32(page_ins);
-				preempt_enable();
-			}
-#endif /* CONFIG_PPC_SMLPAR */
-		} else {
-			current->min_flt++;
-			perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MIN, 1,
-				      regs, address);
-		}
-		if (fault & VM_FAULT_RETRY) {
-			/* Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk
-			 * of starvation. */
-			flags &= ~FAULT_FLAG_ALLOW_RETRY;
-			flags |= FAULT_FLAG_TRIED;
-			goto retry;
+		if (firmware_has_feature(FW_FEATURE_CMO)) {
+			u32 page_ins;
+
+			preempt_disable();
+			page_ins = be32_to_cpu(get_lppaca()->page_ins);
+			page_ins += 1 << PAGE_FACTOR;
+			get_lppaca()->page_ins = cpu_to_be32(page_ins);
+			preempt_enable();
 		}
+#endif /* CONFIG_PPC_SMLPAR */
+	} else {
+		current->min_flt++;
+		perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MIN, 1,
+			      regs, address);
 	}
 
 	up_read(&mm->mmap_sem);
Signature · 1 line
2.7.4