spurious remap_file_pages() -EINVAL

PATCH William Lee Irwin III 2 participants 2 messages Jul 4, 2004 Memory management

Share & tools

William Lee Irwin III

author

2004-07-04 12:04 UTC #0 lore raw

As ->vm_private_data is used as a cursor for swapout of VM_NONLINEAR vmas, the check for NULL ->vm_private_data or VM_RESERVED is too strict, and should allow VM_NONLINEAR vmas with non-NULL ->vm_private_data.

This fixes an issue on 2.6.7-mm5 where system calls to remap_file_pages() spuriously failed while under memory pressure.

mm/fremap.c +2 -1

Index: mm5-2.6.7/mm/fremap.c
===================================================================
@@ -194,7 +194,8 @@
 	 * or VM_LOCKED, but VM_LOCKED could be revoked later on).
 	 */
 	if (vma && (vma->vm_flags & VM_SHARED) &&
-		(!vma->vm_private_data || (vma->vm_flags & VM_RESERVED)) &&
+		(!vma->vm_private_data ||
+			(vma->vm_flags & (VM_NONLINEAR|VM_RESERVED))) &&
 		vma->vm_ops && vma->vm_ops->populate &&
 			end > start && start >= vma->vm_start &&
 				end <= vma->vm_end) {

Hugh Dickins

2004-07-04 19:22 UTC #1 lore raw

Quoted 6 lines · William Lee Irwin III

As ->vm_private_data is used as a cursor for swapout of VM_NONLINEAR vmas, the check for NULL ->vm_private_data or VM_RESERVED is too strict, and should allow VM_NONLINEAR vmas with non-NULL ->vm_private_data.
This fixes an issue on 2.6.7-mm5 where system calls to remap_file_pages() spuriously failed while under memory pressure.

How embarrassing! Yes, thank you.

Hugh