vm_operation to avoid pagefault/inval race

This patch adds a vm_operations_struct function pointer that allows networked and distributed filesystems to avoid a race between a pagefault on an mmap and an invalidation request from some other node. The race goes as follows:

1. A user process on node A accesses a portion of a mapped
file, resulting in a page fault. The pagefault handler invokes the corresponding nopage function, which reads the page into memory.

2. A user process on node B writes to the same portion of
the file (either via mmap or write()), therefore sending node A an invalidation request to node A.

3. Node A receives this invalidate request, and dutifully
invalidates all mmaps. Except for the one that has not yet been fully mapped by step 1.

4. Node A then executes the rest of do_no_page(), entering
the now-invalid page into the PTEs.

5. One way or another, life is now hard.

One solution would be for the distributed filesystem to hold onto a lock or semaphore upon return from the nopage function. The problem is that there is no way to determine (in a timely fashion) when it safe to release this lock or semaphore.

The attached patch addresses this by adding a nopagedone function for when do_no_page() exits. The filesystem may then drop the lock or semaphore in this nopagedone function.

Thoughts? Is there some other existing way to get this done?

Thanx, Paul

diff -urN -X dontdiff linux-2.5.69/include/linux/mm.h linux-2.5.69.stmmap/include/linux/mm.h
@@ -134,6 +134,7 @@
 	void (*open)(struct vm_area_struct * area);
 	void (*close)(struct vm_area_struct * area);
 	struct page * (*nopage)(struct vm_area_struct * area, unsigned long address, int unused);
+	void (*nopagedone)(struct vm_area_struct * area, unsigned long address, int status);
 	int (*populate)(struct vm_area_struct * area, unsigned long address, unsigned long len, pgprot_t prot, unsigned long pgoff, int nonblock);
 };
 

diff -urN -X dontdiff linux-2.5.69/mm/memory.c linux-2.5.69.stmmap/mm/memory.c
@@ -1426,6 +1487,9 @@
 	ret = VM_FAULT_OOM;
 out:
 	pte_chain_free(pte_chain);
+	if (vma->vm_ops && vma->vm_ops->nopagedone) {
+		vma->vm_ops->nopagedone(vma, address & PAGE_MASK, ret);
+	}
 	return ret;
 }
 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

The race is real although currenly no in-tree filesystem is affected. The patch is uglyh as hell, though. The right fix is to change the ->nopage method to cover what do_no_page is currently, change anonymous vmas to have vm_ops as well and set ->nopage to do_anonymous_page.

The gets of the current do_no_page become a new helper (__finish_nopage?) and EXPORT_SYMBOL_GPL()ed. It would also be nice if you could point to a filesystem that actually needs this, but if you can get rid of the do_anonymous_page special casing a patch might even be acceptable without it.