mm modifications / helpers for emulated GPU coherent memory

Hi,
This is an early RFC to make sure I don't go too far in the wrong direction.

Non-coherent GPUs that can't directly see contents in CPU-visible memory, like VMWare's SVGA device, run into trouble when trying to implement coherent memory requirements of modern graphics APIs. Examples are Vulkan and OpenGL 4.4's ARB_buffer_storage.

To remedy, we need to emulate coherent memory. Typically when it's detected that a buffer object is about to be accessed by the GPU, we need to gather the ranges that have been dirtied by the CPU since the last operation, apply an operation to make the content visible to the GPU and clear the the dirty tracking.

Depending on the size of the buffer object and the access pattern there are two major possibilities:

1) Use page_mkwrite() and pfn_mkwrite(). (GPU buffer objects are backed either by PCI device memory or by driver-alloced pages).
The dirty-tracking needs to be reset by write-protecting the affected ptes and flush tlb. This has a complexity of O(num_dirty_pages), but the write page-fault is of course costly.

2) Use hardware dirty-flags in the ptes. The dirty-tracking needs to be reset by clearing the dirty bits and flush tlb. This has a complexity of O(num_buffer_object_pages) and dirty bits need to be scanned in full before each gpu-access.

So in practice the two methods need to be interleaved for best performance.

So to facilitate this, I propose two new helpers, apply_as_wrprotect() and apply_as_clean() ("as" stands for address-space) both inspired by unmap_mapping_range(). Users of these helpers are in the making, but needs some cleaning-up.

There's also a change to x_mkwrite() to allow dropping the mmap_sem while waiting.

Any comments or suggestions appreciated.

Thanks,
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Driver fault callbacks are allowed to drop the mmap_sem when expecting long hardware waits to avoid blocking other mm users. Allow the mkwrite callbacks to do the same by returning early on VM_FAULT_RETRY.

In particular we want to be able to drop the mmap_sem when waiting for a reservation object lock on a GPU buffer object. These locks may be held while waiting for the GPU.

---
 mm/memory.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

index a52663c0612d..dcd80313cf10 100644
@@ -2144,7 +2144,7 @@ static vm_fault_t do_page_mkwrite(struct vm_fault *vmf)
 	ret = vmf->vma->vm_ops->page_mkwrite(vmf);
 	/* Restore original flags so that caller is not surprised */
 	vmf->flags = old_flags;
-	if (unlikely(ret & (VM_FAULT_ERROR | VM_FAULT_NOPAGE)))
+	if (unlikely(ret & (VM_FAULT_ERROR | VM_FAULT_RETRY | VM_FAULT_NOPAGE)))
 		return ret;
 	if (unlikely(!(ret & VM_FAULT_LOCKED))) {
 		lock_page(page);

@@ -2419,7 +2419,7 @@ static vm_fault_t wp_pfn_shared(struct vm_fault *vmf)
 		pte_unmap_unlock(vmf->pte, vmf->ptl);
 		vmf->flags |= FAULT_FLAG_MKWRITE;
 		ret = vma->vm_ops->pfn_mkwrite(vmf);
-		if (ret & (VM_FAULT_ERROR | VM_FAULT_NOPAGE))
+		if (ret & (VM_FAULT_ERROR | VM_FAULT_RETRY | VM_FAULT_NOPAGE))
 			return ret;
 		return finish_mkwrite_fault(vmf);
 	}

@@ -2440,7 +2440,8 @@ static vm_fault_t wp_page_shared(struct vm_fault *vmf)
 		pte_unmap_unlock(vmf->pte, vmf->ptl);
 		tmp = do_page_mkwrite(vmf);
 		if (unlikely(!tmp || (tmp &
-				      (VM_FAULT_ERROR | VM_FAULT_NOPAGE)))) {
+				      (VM_FAULT_ERROR | VM_FAULT_RETRY |
+				       VM_FAULT_NOPAGE)))) {
 			put_page(vmf->page);
 			return tmp;
 		}

@@ -3472,7 +3473,8 @@ static vm_fault_t do_shared_fault(struct vm_fault *vmf)
 		unlock_page(vmf->page);
 		tmp = do_page_mkwrite(vmf);
 		if (unlikely(!tmp ||
-				(tmp & (VM_FAULT_ERROR | VM_FAULT_NOPAGE)))) {
+				(tmp & (VM_FAULT_ERROR | VM_FAULT_RETRY |
+					VM_FAULT_NOPAGE)))) {
 			put_page(vmf->page);
 			return tmp;
 		}

2.19.0.rc1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

No comment on the patch itself, but please fix your email setup.

All the patches were marked as spam, because you sent them from your vmware.com address, but without going through the proper vmware smtp gateway.

So they lack the proper vmware DKIM hashes and proper mail handling should (and does) consider them spam.

Linus

This is basically apply_to_page_range with added functionality:
Allocating missing parts of the page table becomes optional, which means that the function can be guaranteed not to error if allocation is disabled. Also passing of the closure struct and callback function becomes different and more in line with how things are done elsewhere.

Finally we keep apply_to_page_range as a wrapper around apply_to_pfn_range

---
 include/linux/mm.h |  10 ++++
 mm/memory.c        | 121 +++++++++++++++++++++++++++++++++------------
 2 files changed, 99 insertions(+), 32 deletions(-)

index 80bb6408fe73..b7dd4ddd6efb 100644
@@ -2632,6 +2632,16 @@ typedef int (*pte_fn_t)(pte_t *pte, pgtable_t token, unsigned long addr,
 extern int apply_to_page_range(struct mm_struct *mm, unsigned long address,
 			       unsigned long size, pte_fn_t fn, void *data);
 
+struct pfn_range_apply;
+typedef int (*pter_fn_t)(pte_t *pte, pgtable_t token, unsigned long addr,
+			 struct pfn_range_apply *closure);
+struct pfn_range_apply {
+	struct mm_struct *mm;
+	pter_fn_t ptefn;
+	unsigned int alloc;
+};
+extern int apply_to_pfn_range(struct pfn_range_apply *closure,
+			      unsigned long address, unsigned long size);
 
 #ifdef CONFIG_PAGE_POISONING
 extern bool page_poisoning_enabled(void);

2.19.0.rc1

Add two utilities to a) write-protect and b) clean all ptes pointing into a range of an address space
The utilities are intended to aid in tracking dirty pages (either driver-allocated system memory or pci device memory).
The write-protect utility should be used in conjunction with page_mkwrite() and pfn_mkwrite() to trigger write page-faults on page accesses. Typically one would want to use this on sparse accesses into large memory regions. The clean utility should be used to utilize hardware dirtying functionality and avoid the overhead of page-faults, typically on large accesses into small memory regions.

---
 include/linux/mm.h  |   9 +-
 mm/Makefile         |   2 +-
 mm/apply_as_range.c | 257 ++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 266 insertions(+), 2 deletions(-)
 create mode 100644 mm/apply_as_range.c

index b7dd4ddd6efb..62f24dd0bfa0 100644
@@ -2642,7 +2642,14 @@ struct pfn_range_apply {
 };
 extern int apply_to_pfn_range(struct pfn_range_apply *closure,
 			      unsigned long address, unsigned long size);
-
+unsigned long apply_as_wrprotect(struct address_space *mapping,
+				 pgoff_t first_index, pgoff_t nr);
+unsigned long apply_as_clean(struct address_space *mapping,
+			     pgoff_t first_index, pgoff_t nr,
+			     pgoff_t bitmap_pgoff,
+			     unsigned long *bitmap,
+			     pgoff_t *start,
+			     pgoff_t *end);
 #ifdef CONFIG_PAGE_POISONING
 extern bool page_poisoning_enabled(void);
 extern void kernel_poison_pages(struct page *page, int numpages, int enable);

index d210cc9d6f80..a94b78f12692 100644
@@ -39,7 +39,7 @@ obj-y			:= filemap.o mempool.o oom_kill.o fadvise.o \
 			   mm_init.o mmu_context.o percpu.o slab_common.o \
 			   compaction.o vmacache.o \
 			   interval_tree.o list_lru.o workingset.o \
-			   debug.o $(mmu-y)
+			   debug.o apply_as_range.o $(mmu-y)
 
 obj-y += init-mm.o
 obj-y += memblock.o

2.19.0.rc1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel