mmu notifier provide context informations

From: Jérôme Glisse <[email protected]>

When range of virtual address is updated read only and corresponding user ptr object are already read only it is pointless to do anything. Optimize this case out.

Signed-off-by Jérôme Glisse <[email protected]>

Cc Christian König <[email protected]>

Cc Jan Kara <[email protected]>

Cc Felix Kuehling <[email protected]>

Cc Jason Gunthorpe <[email protected]>

Cc Andrew Morton <[email protected]>

Cc Matthew Wilcox <[email protected]>

Cc Ross Zwisler <[email protected]>

Cc Dan Williams <[email protected]>

Cc Paolo Bonzini <[email protected]>

Cc Radim Krčmář <[email protected]>

Cc Michal Hocko <[email protected]>

Cc Ralph Campbell <[email protected]>

Cc John Hubbard <[email protected]>

Cc [email protected]

Cc [email protected]

Cc [email protected]

Cc [email protected]

Cc Arnd Bergmann <[email protected]>

---
 drivers/gpu/drm/i915/i915_gem_userptr.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

drivers/gpu/drm/i915/i915_gem_userptr.c +16 -0

index 9558582c105e..23330ac3d7ea 100644
@@ -59,6 +59,7 @@ struct i915_mmu_object {
        struct interval_tree_node it;
        struct list_head link;
        struct work_struct work;
+       bool read_only;
        bool attached;
 };
 

@@ -119,6 +120,7 @@ static int i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn,
                container_of(_mn, struct i915_mmu_notifier, mn);
        struct i915_mmu_object *mo;
        struct interval_tree_node *it;
+       bool update_to_read_only;
        LIST_HEAD(cancelled);
        unsigned long end;
 

@@ -128,6 +130,8 @@ static int i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn,
        /* interval ranges are inclusive, but invalidate range is exclusive */
        end = range->end - 1;
 
+       update_to_read_only = mmu_notifier_range_update_to_read_only(range);
+
        spin_lock(&mn->lock);
        it = interval_tree_iter_first(&mn->objects, range->start, end);
        while (it) {

@@ -145,6 +149,17 @@ static int i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn,
                 * object if it is not in the process of being destroyed.
                 */
                mo = container_of(it, struct i915_mmu_object, it);
+
+               /*
+                * If it is already read only and we are updating to
+                * read only then we do not need to change anything.
+                * So save time and skip this one.
+                */
+               if (update_to_read_only && mo->read_only) {
+                       it = interval_tree_iter_next(it, range->start, end);
+                       continue;
+               }
+
                if (kref_get_unless_zero(&mo->obj->base.refcount))
                        queue_work(mn->wq, &mo->work);
 

@@ -270,6 +285,7 @@ i915_gem_userptr_init__mmu_notifier(struct drm_i915_gem_object *obj,
        mo->mn = mn;
        mo->obj = obj;
        mo->it.start = obj->userptr.ptr;
+       mo->read_only = i915_gem_object_is_readonly(obj);
        mo->it.last = obj->userptr.ptr + obj->base.size - 1;
        INIT_WORK(&mo->work, cancel_userptr);
 

Signature · 6 lines

2.17.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Hi Jerome,

This patch seems to have plenty of Cc:s, but none of the right ones :)

For further iterations, I guess you could use git option --cc to make sure everyone gets the whole series, and still keep the Cc:s in the patches themselves relevant to subsystems.

This doesn't seem to be on top of drm-tip, but on top of your previous patches(?) that I had some comments about. Could you take a moment to first address the couple of question I had, before proceeding to discuss what is built on top of that base.

My reply's Message-ID is:
154289518994.19402.3481838548028068213@jlahtine-desk.ger.corp.intel.com

Regards, Joonas

PS. Please keep me Cc:d in the following patches, I'm keen on understanding the motive and benefits.

Quoting [email protected] (2019-01-24 00:23:14)

Quoted 84 lines

From: Jérôme Glisse <[email protected]>

When range of virtual address is updated read only and corresponding user ptr object are already read only it is pointless to do anything. Optimize this case out.

Signed-off-by Jérôme Glisse <[email protected]>

Cc Christian König <[email protected]>

Cc Jan Kara <[email protected]>

Cc Felix Kuehling <[email protected]>

Cc Jason Gunthorpe <[email protected]>

Cc Andrew Morton <[email protected]>

Cc Matthew Wilcox <[email protected]>

Cc Ross Zwisler <[email protected]>

Cc Dan Williams <[email protected]>

Cc Paolo Bonzini <[email protected]>

Cc Radim Krčmář <[email protected]>

Cc Michal Hocko <[email protected]>

Cc Ralph Campbell <[email protected]>

Cc John Hubbard <[email protected]>

Cc [email protected]

Cc [email protected]

Cc [email protected]

Cc [email protected]

Cc Arnd Bergmann <[email protected]>

---
 drivers/gpu/drm/i915/i915_gem_userptr.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

drivers/gpu/drm/i915/i915_gem_userptr.c +16 -0

index 9558582c105e..23330ac3d7ea 100644
@@ -59,6 +59,7 @@ struct i915_mmu_object {
        struct interval_tree_node it;
        struct list_head link;
        struct work_struct work;
+       bool read_only;
        bool attached;
 };
 

@@ -119,6 +120,7 @@ static int i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn,
                container_of(_mn, struct i915_mmu_notifier, mn);
        struct i915_mmu_object *mo;
        struct interval_tree_node *it;
+       bool update_to_read_only;
        LIST_HEAD(cancelled);
        unsigned long end;
 

@@ -128,6 +130,8 @@ static int i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn,
        /* interval ranges are inclusive, but invalidate range is exclusive */
        end = range->end - 1;
 
+       update_to_read_only = mmu_notifier_range_update_to_read_only(range);
+
        spin_lock(&mn->lock);
        it = interval_tree_iter_first(&mn->objects, range->start, end);
        while (it) {

@@ -145,6 +149,17 @@ static int i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn,
                 * object if it is not in the process of being destroyed.
                 */
                mo = container_of(it, struct i915_mmu_object, it);
+
+               /*
+                * If it is already read only and we are updating to
+                * read only then we do not need to change anything.
+                * So save time and skip this one.
+                */
+               if (update_to_read_only && mo->read_only) {
+                       it = interval_tree_iter_next(it, range->start, end);
+                       continue;
+               }
+
                if (kref_get_unless_zero(&mo->obj->base.refcount))
                        queue_work(mn->wq, &mo->work);
 

@@ -270,6 +285,7 @@ i915_gem_userptr_init__mmu_notifier(struct drm_i915_gem_object *obj,
        mo->mn = mn;
        mo->obj = obj;
        mo->it.start = obj->userptr.ptr;
+       mo->read_only = i915_gem_object_is_readonly(obj);
        mo->it.last = obj->userptr.ptr + obj->base.size - 1;
        INIT_WORK(&mo->work, cancel_userptr);
 

Quoted 7 lines

Signature · 6 lines

2.17.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Quoted 3 lines

Hi Jerome,

This patch seems to have plenty of Cc:s, but none of the right ones :)

So sorry, i am bad with git commands.

Quoted 3 lines

For further iterations, I guess you could use git option --cc to make sure everyone gets the whole series, and still keep the Cc:s in the patches themselves relevant to subsystems.

Will do.

Quoted 4 lines

This doesn't seem to be on top of drm-tip, but on top of your previous patches(?) that I had some comments about. Could you take a moment to first address the couple of question I had, before proceeding to discuss what is built on top of that base.

It is on top of Linus tree so roughly ~ rc3 it does not depend on any of the previous patch i posted.

I still intended to propose to remove
GUP from i915 once i get around to implement the equivalent of GUP_fast for HMM and other bonus cookies with it.

The plan is once i have all mm bits properly upstream then i can propose patches to individual driver against the proper driver tree ie following rules of each individual device driver sub-system and Cc only people there to avoid spamming the mm folks :)

Quoted 96 lines

My reply's Message-ID is:
154289518994.19402.3481838548028068213@jlahtine-desk.ger.corp.intel.com

Regards, Joonas

PS. Please keep me Cc:d in the following patches, I'm keen on understanding the motive and benefits.

Quoting [email protected] (2019-01-24 00:23:14)

Quoted 84 lines

From: Jérôme Glisse <[email protected]>

When range of virtual address is updated read only and corresponding user ptr object are already read only it is pointless to do anything. Optimize this case out.

Signed-off-by Jérôme Glisse <[email protected]>

Cc Christian König <[email protected]>

Cc Jan Kara <[email protected]>

Cc Felix Kuehling <[email protected]>

Cc Jason Gunthorpe <[email protected]>

Cc Andrew Morton <[email protected]>

Cc Matthew Wilcox <[email protected]>

Cc Ross Zwisler <[email protected]>

Cc Dan Williams <[email protected]>

Cc Paolo Bonzini <[email protected]>

Cc Radim Krčmář <[email protected]>

Cc Michal Hocko <[email protected]>

Cc Ralph Campbell <[email protected]>

Cc John Hubbard <[email protected]>

Cc [email protected]

Cc [email protected]

Cc [email protected]

Cc [email protected]

Cc Arnd Bergmann <[email protected]>

---
 drivers/gpu/drm/i915/i915_gem_userptr.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

drivers/gpu/drm/i915/i915_gem_userptr.c +16 -0

index 9558582c105e..23330ac3d7ea 100644
@@ -59,6 +59,7 @@ struct i915_mmu_object {
        struct interval_tree_node it;
        struct list_head link;
        struct work_struct work;
+       bool read_only;
        bool attached;
 };
 

@@ -119,6 +120,7 @@ static int i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn,
                container_of(_mn, struct i915_mmu_notifier, mn);
        struct i915_mmu_object *mo;
        struct interval_tree_node *it;
+       bool update_to_read_only;
        LIST_HEAD(cancelled);
        unsigned long end;
 

@@ -128,6 +130,8 @@ static int i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn,
        /* interval ranges are inclusive, but invalidate range is exclusive */
        end = range->end - 1;
 
+       update_to_read_only = mmu_notifier_range_update_to_read_only(range);
+
        spin_lock(&mn->lock);
        it = interval_tree_iter_first(&mn->objects, range->start, end);
        while (it) {

@@ -145,6 +149,17 @@ static int i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn,
                 * object if it is not in the process of being destroyed.
                 */
                mo = container_of(it, struct i915_mmu_object, it);
+
+               /*
+                * If it is already read only and we are updating to
+                * read only then we do not need to change anything.
+                * So save time and skip this one.
+                */
+               if (update_to_read_only && mo->read_only) {
+                       it = interval_tree_iter_next(it, range->start, end);
+                       continue;
+               }
+
                if (kref_get_unless_zero(&mo->obj->base.refcount))
                        queue_work(mn->wq, &mo->work);
 

@@ -270,6 +285,7 @@ i915_gem_userptr_init__mmu_notifier(struct drm_i915_gem_object *obj,
        mo->mn = mn;
        mo->obj = obj;
        mo->it.start = obj->userptr.ptr;
+       mo->read_only = i915_gem_object_is_readonly(obj);
        mo->it.last = obj->userptr.ptr + obj->base.size - 1;
        INIT_WORK(&mo->work, cancel_userptr);
 

Quoted 7 lines

Quoted 7 lines

Signature · 6 lines

2.17.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Quoting Jerome Glisse (2019-01-24 17:30:32)

Quoted 13 lines · Joonas Lahtinen

Quoted 3 lines

Hi Jerome,

This patch seems to have plenty of Cc:s, but none of the right ones :)

So sorry, i am bad with git commands.

Quoted 3 lines

For further iterations, I guess you could use git option --cc to make sure everyone gets the whole series, and still keep the Cc:s in the patches themselves relevant to subsystems.

Will do.

Quoted 7 lines

Quoted 4 lines

This doesn't seem to be on top of drm-tip, but on top of your previous patches(?) that I had some comments about. Could you take a moment to first address the couple of question I had, before proceeding to discuss what is built on top of that base.

It is on top of Linus tree so roughly ~ rc3 it does not depend on any of the previous patch i posted.

You actually managed to race a point in time just when Chris rewrote much of the userptr code in drm-tip, which I didn't remember of. My bad.

Still interested to hearing replies to my questions in the previous thread, if the series is still relevant. Trying to get my head around how the different aspects of HMM pan out for devices without fault handling.

Bit late reply, but here goes :)

We're working quite hard to avoid pinning any pages unless they're in the GPU page tables. And when they are in the GPU page tables, they must be pinned for whole of that duration, for the reason that our GPUs can not take a fault. And to avoid thrashing GPU page tables, we do leave objects in page tables with the expectation that smart userspace recycles buffers.

So what I understand of your proposal, it wouldn't really make a difference for us in the amount of pinned pages (which I agree, we'd love to see going down). When we're unable to take a fault, the first use effectively forces us to pin any pages and keep them pinned to avoid thrashing GPU page tables.

So from i915 perspective, it just seems to be mostly an exchange of an API to an another for getting the pages. You already mentioned the fast path is being worked on, which is an obvious difference. But is there some other improvement one would be expecting, beyond the page pinning?

Also, is the requirement for a single non-file-backed VMA in the plans of being eliminated or is that inherent restriction of the HMM_MIRROR feature? We're currently not imposing such a limitation.

Quoted 8 lines

I still intended to propose to remove
GUP from i915 once i get around to implement the equivalent of GUP_fast for HMM and other bonus cookies with it.

The plan is once i have all mm bits properly upstream then i can propose patches to individual driver against the proper driver tree ie following rules of each individual device driver sub-system and Cc only people there to avoid spamming the mm folks :)

Makes sense, as we're having tons of changes in this field in i915, the churn to rebase on top of them will be substantial.

Regards, Joonas

PS. Are you by any chance attending FOSDEM? Would be nice to chat about this.

From: Jérôme Glisse <[email protected]>

When range of virtual address is updated read only and corresponding user ptr object are already read only it is pointless to do anything. Optimize this case out.

Signed-off-by Jérôme Glisse <[email protected]>

Cc Christian König <[email protected]>

Cc Jan Kara <[email protected]>

Cc Felix Kuehling <[email protected]>

Cc Jason Gunthorpe <[email protected]>

Cc Andrew Morton <[email protected]>

Cc Matthew Wilcox <[email protected]>

Cc Ross Zwisler <[email protected]>

Cc Dan Williams <[email protected]>

Cc Paolo Bonzini <[email protected]>

Cc Radim Krčmář <[email protected]>

Cc Michal Hocko <[email protected]>

Cc Ralph Campbell <[email protected]>

Cc John Hubbard <[email protected]>

Cc [email protected]

Cc [email protected]

Cc [email protected]

Cc [email protected]

Cc Arnd Bergmann <[email protected]>

 drivers/infiniband/core/umem_odp.c | 22 +++++++++++++++++++---
 include/rdma/ib_umem_odp.h         |  1 +
 2 files changed, 20 insertions(+), 3 deletions(-)

drivers/infiniband/core/umem_odp.c +17 -2

index a4ec43093cb3..fa4e7fdcabfc 100644
+++ b/drivers/infiniband/core/umem_odp.c

@@ -140,8 +140,15 @@ static void ib_umem_notifier_release(struct mmu_notifier *mn,
 static int invalidate_range_start_trampoline(struct ib_umem_odp *item,
 					     u64 start, u64 end, void *cookie)
 {
+	bool update_to_read_only = *((bool *)cookie);
+
 	ib_umem_notifier_start_account(item);
-	item->umem.context->invalidate_range(item, start, end);
+	/*
+	 * If it is already read only and we are updating to read only then we
+	 * do not need to change anything. So save time and skip this one.
+	 */
+	if (!update_to_read_only || !item->read_only)
+		item->umem.context->invalidate_range(item, start, end);
 	return 0;
 }
 

@@ -150,6 +157,7 @@ static int ib_umem_notifier_invalidate_range_start(struct mmu_notifier *mn,
 {
 	struct ib_ucontext_per_mm *per_mm =
 		container_of(mn, struct ib_ucontext_per_mm, mn);
+	bool update_to_read_only;
 
 	if (range->blockable)
 		down_read(&per_mm->umem_rwsem);

@@ -166,10 +174,13 @@ static int ib_umem_notifier_invalidate_range_start(struct mmu_notifier *mn,
 		return 0;
 	}
 
+	update_to_read_only = mmu_notifier_range_update_to_read_only(range);
+
 	return rbt_ib_umem_for_each_in_range(&per_mm->umem_tree, range->start,
 					     range->end,
 					     invalidate_range_start_trampoline,
-					     range->blockable, NULL);
+					     range->blockable,
+					     &update_to_read_only);
 }
 
 static int invalidate_range_end_trampoline(struct ib_umem_odp *item, u64 start,

@@ -363,6 +374,9 @@ struct ib_umem_odp *ib_alloc_odp_umem(struct ib_ucontext_per_mm *per_mm,
 		goto out_odp_data;
 	}
 
+	/* Assume read only at first, each time GUP is call this is updated. */
+	odp_data->read_only = true;
+
 	odp_data->dma_list =
 		vzalloc(array_size(pages, sizeof(*odp_data->dma_list)));
 	if (!odp_data->dma_list) {

drivers/infiniband/core/umem_odp.c +1 -0

 		flags |= FOLL_WRITE;
+	}
 
 	start_idx = (user_virt - ib_umem_start(umem)) >> page_shift;
 	k = start_idx;

include/rdma/ib_umem_odp.h +1 -0

index 0b1446fe2fab..8256668c6170 100644
+++ b/include/rdma/ib_umem_odp.h

Quoted 86 lines · On Wed, Jan 23, 2019 at 05:23:15PM -0500, [email protected] wrote:

From: Jérôme Glisse <[email protected]>

When range of virtual address is updated read only and corresponding user ptr object are already read only it is pointless to do anything. Optimize this case out.

Signed-off-by Jérôme Glisse <[email protected]>

Cc Christian König <[email protected]>

Cc Jan Kara <[email protected]>

Cc Felix Kuehling <[email protected]>

Cc Jason Gunthorpe <[email protected]>

Cc Andrew Morton <[email protected]>

Cc Matthew Wilcox <[email protected]>

Cc Ross Zwisler <[email protected]>

Cc Dan Williams <[email protected]>

Cc Paolo Bonzini <[email protected]>

Cc Radim Krčmář <[email protected]>

Cc Michal Hocko <[email protected]>

Cc Ralph Campbell <[email protected]>

Cc John Hubbard <[email protected]>

Cc [email protected]

Cc [email protected]

Cc [email protected]

Cc [email protected]

Cc Arnd Bergmann <[email protected]>

 drivers/infiniband/core/umem_odp.c | 22 +++++++++++++++++++---
 include/rdma/ib_umem_odp.h         |  1 +
 2 files changed, 20 insertions(+), 3 deletions(-)

drivers/infiniband/core/umem_odp.c +17 -2

index a4ec43093cb3..fa4e7fdcabfc 100644
+++ b/drivers/infiniband/core/umem_odp.c

@@ -140,8 +140,15 @@ static void ib_umem_notifier_release(struct mmu_notifier *mn,
 static int invalidate_range_start_trampoline(struct ib_umem_odp *item,
 					     u64 start, u64 end, void *cookie)
 {
+	bool update_to_read_only = *((bool *)cookie);
+
 	ib_umem_notifier_start_account(item);
-	item->umem.context->invalidate_range(item, start, end);
+	/*
+	 * If it is already read only and we are updating to read only then we
+	 * do not need to change anything. So save time and skip this one.
+	 */
+	if (!update_to_read_only || !item->read_only)
+		item->umem.context->invalidate_range(item, start, end);
 	return 0;
 }
 

@@ -150,6 +157,7 @@ static int ib_umem_notifier_invalidate_range_start(struct mmu_notifier *mn,
 {
 	struct ib_ucontext_per_mm *per_mm =
 		container_of(mn, struct ib_ucontext_per_mm, mn);
+	bool update_to_read_only;
 
 	if (range->blockable)
 		down_read(&per_mm->umem_rwsem);

@@ -166,10 +174,13 @@ static int ib_umem_notifier_invalidate_range_start(struct mmu_notifier *mn,
 		return 0;
 	}
 
+	update_to_read_only = mmu_notifier_range_update_to_read_only(range);
+
 	return rbt_ib_umem_for_each_in_range(&per_mm->umem_tree, range->start,
 					     range->end,
 					     invalidate_range_start_trampoline,
-					     range->blockable, NULL);
+					     range->blockable,
+					     &update_to_read_only);
 }
 
 static int invalidate_range_end_trampoline(struct ib_umem_odp *item, u64 start,

@@ -363,6 +374,9 @@ struct ib_umem_odp *ib_alloc_odp_umem(struct ib_ucontext_per_mm *per_mm,
 		goto out_odp_data;
 	}
 
+	/* Assume read only at first, each time GUP is call this is updated. */
+	odp_data->read_only = true;
+
 	odp_data->dma_list =
 		vzalloc(array_size(pages, sizeof(*odp_data->dma_list)));
 	if (!odp_data->dma_list) {

@@ -619,8 +633,10 @@ int ib_umem_odp_map_dma_pages(struct ib_umem_odp *umem_odp, u64 user_virt,
 		goto out_put_task;
 	}
 

drivers/infiniband/core/umem_odp.c int ib_umem_odp_map_dma_pages(struct ib_umem_odp *umem_odp, u64 user_virt, · line 637

Quoted 3 lines

drivers/infiniband/core/umem_odp.c +2 -1

-	if (access_mask & ODP_WRITE_ALLOWED_BIT)
+	if (access_mask & ODP_WRITE_ALLOWED_BIT) {
+		umem_odp->read_only = false;

No locking?

Quoted 4 lines

drivers/infiniband/core/umem_odp.c +1 -0

 		flags |= FOLL_WRITE;
+	}
 

include/rdma/ib_umem_odp.h struct ib_umem_odp { · line 80

Quoted 11 lines

drivers/infiniband/core/umem_odp.c +0 -0

 	start_idx = (user_virt - ib_umem_start(umem)) >> page_shift;
 	k = start_idx;

include/rdma/ib_umem_odp.h +2 -0

index 0b1446fe2fab..8256668c6170 100644
+++ b/include/rdma/ib_umem_odp.h

@@ -76,6 +76,7 @@ struct ib_umem_odp {
 	struct completion	notifier_completion;
 	int			dying;
 	struct work_struct	work;
+	bool read_only;
 };

The ib_umem already has a writeable flag. This reflects if the user asked for write permission to be granted.. The tracking here is if any remote fault thus far has requested write, is this an important difference to justify the new flag?

From: Jérôme Glisse <[email protected]>

Hi Andrew, i see that you still have my event patch in you queue [1]. This patchset replace that single patch and is broken down in further step so that it is easier to review and ascertain that no mistake were made during mechanical changes. Here are the step:

Patch 1 - add the enum values
Patch 2 - coccinelle semantic patch to convert all call site of
mmu_notifier_range_init to default enum value and also to passing down the vma when it is available
Patch 3 - update many call site to more accurate enum values Patch 4 - add the information to the mmu_notifier_range struct Patch 5 - helper to test if a range is updated to read only

All the remaining patches are update to various driver to demonstrate how this new information get use by device driver. I build tested with make all and make all minus everything that enable mmu notifier ie building with MMU_NOTIFIER=no. Also tested with some radeon,amd gpu and intel gpu.

If they are no objections i believe best plan would be to merge the the first 5 patches (all mm changes) through your queue for 5.1 and then to delay driver update to each individual driver tree for 5.2. This will allow each individual device driver maintainer time to more thouroughly test this more then my own testing.

Note that i also intend to use this feature further in nouveau and HMM down the road. I also expect that other user like KVM might be interested into leveraging this new information to optimize some of there secondary page table invalidation.

Here is an explaination on the rational for this patchset:

CPU page table update can happens for many reasons, not only as a result of a syscall (munmap(), mprotect(), mremap(), madvise(), ...) but also as a result of kernel activities (memory compression, reclaim, migration, ...).

This patch introduce a set of enums that can be associated with each of the events triggering a mmu notifier. Latter patches take advantages of those enum values.

- UNMAP: munmap() or mremap()
- CLEAR: page table is cleared (migration, compaction, reclaim, ...)
- PROTECTION_VMA: change in access protections for the range
- PROTECTION_PAGE: change in access protections for page in the range
- SOFT_DIRTY: soft dirtyness tracking

Being able to identify munmap() and mremap() from other reasons why the page table is cleared is important to allow user of mmu notifier to update their own internal tracking structure accordingly (on munmap or mremap it is not longer needed to track range of virtual address as it becomes invalid).

[1] https://www.ozlabs.org/~akpm/mmotm/broken-out/mm-mmu_notifier-contextual-information-for-event-triggering-invalidation-v2.patch

Cc Christian König <[email protected]>

Cc Jan Kara <[email protected]>

Cc Felix Kuehling <[email protected]>

Cc Jason Gunthorpe <[email protected]>

Cc Andrew Morton <[email protected]>

Cc Matthew Wilcox <[email protected]>

Cc Ross Zwisler <[email protected]>

Cc Dan Williams <[email protected]>

Cc Paolo Bonzini <[email protected]>

Cc Radim Krčmář <[email protected]>

Cc Michal Hocko <[email protected]>

Cc Ralph Campbell <[email protected]>

Cc John Hubbard <[email protected]>

Cc [email protected]

Cc [email protected]

Cc [email protected]

Cc [email protected]

Cc Arnd Bergmann <[email protected]>

Jérôme Glisse (9):
mm/mmu_notifier: contextual information for event enums mm/mmu_notifier: contextual information for event triggering
invalidation
mm/mmu_notifier: use correct mmu_notifier events for each invalidation mm/mmu_notifier: pass down vma and reasons why mmu notifier is
happening
mm/mmu_notifier: mmu_notifier_range_update_to_read_only() helper gpu/drm/radeon: optimize out the case when a range is updated to read
only
gpu/drm/amdgpu: optimize out the case when a range is updated to read
only
gpu/drm/i915: optimize out the case when a range is updated to read
only
RDMA/umem_odp: optimize out the case when a range is updated to read
only

 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c  | 13 ++++++++
 drivers/gpu/drm/i915/i915_gem_userptr.c | 16 ++++++++++
 drivers/gpu/drm/radeon/radeon_mn.c      | 13 ++++++++
 drivers/infiniband/core/umem_odp.c      | 22 +++++++++++--
 fs/proc/task_mmu.c                      |  3 +-
 include/linux/mmu_notifier.h            | 42 ++++++++++++++++++++++++-
 include/rdma/ib_umem_odp.h              |  1 +
 kernel/events/uprobes.c                 |  3 +-
 mm/huge_memory.c                        | 14 +++++----
 mm/hugetlb.c                            | 11 ++++---
 mm/khugepaged.c                         |  3 +-
 mm/ksm.c                                |  6 ++--
 mm/madvise.c                            |  3 +-
 mm/memory.c                             | 25 +++++++++------
 mm/migrate.c                            |  5 ++-
 mm/mmu_notifier.c                       | 10 ++++++
 mm/mprotect.c                           |  4 ++-
 mm/mremap.c                             |  3 +-
 mm/oom_kill.c                           |  3 +-
 mm/rmap.c                               |  6 ++--
 20 files changed, 171 insertions(+), 35 deletions(-)

--
2.17.2

Quoted 28 lines · On Wed, Jan 23, 2019 at 2:23 PM <[email protected]> wrote:

From: Jérôme Glisse <[email protected]>

Hi Andrew, i see that you still have my event patch in you queue [1]. This patchset replace that single patch and is broken down in further step so that it is easier to review and ascertain that no mistake were made during mechanical changes. Here are the step:

Patch 1 - add the enum values
Patch 2 - coccinelle semantic patch to convert all call site of
mmu_notifier_range_init to default enum value and also to passing down the vma when it is available
Patch 3 - update many call site to more accurate enum values Patch 4 - add the information to the mmu_notifier_range struct Patch 5 - helper to test if a range is updated to read only

All the remaining patches are update to various driver to demonstrate how this new information get use by device driver. I build tested with make all and make all minus everything that enable mmu notifier ie building with MMU_NOTIFIER=no. Also tested with some radeon,amd gpu and intel gpu.

If they are no objections i believe best plan would be to merge the the first 5 patches (all mm changes) through your queue for 5.1 and then to delay driver update to each individual driver tree for 5.2. This will allow each individual device driver maintainer time to more thouroughly test this more then my own testing.

Quoted 4 lines

Note that i also intend to use this feature further in nouveau and HMM down the road. I also expect that other user like KVM might be interested into leveraging this new information to optimize some of there secondary page table invalidation.

"Down the road" users should introduce the functionality they want to consume. The common concern with preemptively including forward-looking infrastructure is realizing later that the infrastructure is not needed, or needs changing. If it has no current consumer, leave it out.

Quoted 18 lines

Here is an explaination on the rational for this patchset:

CPU page table update can happens for many reasons, not only as a result of a syscall (munmap(), mprotect(), mremap(), madvise(), ...) but also as a result of kernel activities (memory compression, reclaim, migration, ...).

This patch introduce a set of enums that can be associated with each of the events triggering a mmu notifier. Latter patches take advantages of those enum values.

- UNMAP: munmap() or mremap()
- CLEAR: page table is cleared (migration, compaction, reclaim, ...)
- PROTECTION_VMA: change in access protections for the range
- PROTECTION_PAGE: change in access protections for page in the range
- SOFT_DIRTY: soft dirtyness tracking

Quoted 5 lines

Being able to identify munmap() and mremap() from other reasons why the page table is cleared is important to allow user of mmu notifier to update their own internal tracking structure accordingly (on munmap or mremap it is not longer needed to track range of virtual address as it becomes invalid).

The only context information consumed in this patch set is MMU_NOTIFY_PROTECTION_VMA.

What is the practical benefit of these "optimize out the case when a range is updated to read only" optimizations? Any numbers to show this is worth the code thrash?

Quoted 34 lines · Dan Williams

Quoted 28 lines · On Wed, Jan 23, 2019 at 2:23 PM <[email protected]> wrote:

From: Jérôme Glisse <[email protected]>

Hi Andrew, i see that you still have my event patch in you queue [1]. This patchset replace that single patch and is broken down in further step so that it is easier to review and ascertain that no mistake were made during mechanical changes. Here are the step:

Patch 1 - add the enum values
Patch 2 - coccinelle semantic patch to convert all call site of
mmu_notifier_range_init to default enum value and also to passing down the vma when it is available
Patch 3 - update many call site to more accurate enum values Patch 4 - add the information to the mmu_notifier_range struct Patch 5 - helper to test if a range is updated to read only

All the remaining patches are update to various driver to demonstrate how this new information get use by device driver. I build tested with make all and make all minus everything that enable mmu notifier ie building with MMU_NOTIFIER=no. Also tested with some radeon,amd gpu and intel gpu.

If they are no objections i believe best plan would be to merge the the first 5 patches (all mm changes) through your queue for 5.1 and then to delay driver update to each individual driver tree for 5.2. This will allow each individual device driver maintainer time to more thouroughly test this more then my own testing.

Quoted 4 lines

Note that i also intend to use this feature further in nouveau and HMM down the road. I also expect that other user like KVM might be interested into leveraging this new information to optimize some of there secondary page table invalidation.

Quoted 5 lines

"Down the road" users should introduce the functionality they want to consume. The common concern with preemptively including forward-looking infrastructure is realizing later that the infrastructure is not needed, or needs changing. If it has no current consumer, leave it out.

This patchset already show that this is useful, what more can i do ? I know i will use this information, in nouveau for memory policy we allocate our own structure for every vma the GPU ever accessed or that userspace hinted we should set a policy for. Right now with existing mmu notifier i _must_ free those structure because i do not know if the invalidation is an munmap or something else. So i am loosing important informations and unecessarly free struct that i will have to re-allocate just couple jiffies latter. That's one way i am using this.

The other way is to optimize GPU page table update just like i am doing with all the patches to RDMA/ODP and various GPU drivers.

Quoted 26 lines

Quoted 20 lines

Here is an explaination on the rational for this patchset:

CPU page table update can happens for many reasons, not only as a result of a syscall (munmap(), mprotect(), mremap(), madvise(), ...) but also as a result of kernel activities (memory compression, reclaim, migration, ...).

This patch introduce a set of enums that can be associated with each of the events triggering a mmu notifier. Latter patches take advantages of those enum values.

- UNMAP: munmap() or mremap()
- CLEAR: page table is cleared (migration, compaction, reclaim, ...)
- PROTECTION_VMA: change in access protections for the range
- PROTECTION_PAGE: change in access protections for page in the range
- SOFT_DIRTY: soft dirtyness tracking

Quoted 5 lines

Being able to identify munmap() and mremap() from other reasons why the page table is cleared is important to allow user of mmu notifier to update their own internal tracking structure accordingly (on munmap or mremap it is not longer needed to track range of virtual address as it becomes invalid).

Quoted 6 lines

The only context information consumed in this patch set is MMU_NOTIFY_PROTECTION_VMA.

What is the practical benefit of these "optimize out the case when a range is updated to read only" optimizations? Any numbers to show this is worth the code thrash?

It depends on the workload for instance if you map to RDMA a file read only like a log file for export, all write back that would disrupt the RDMA mapping can be optimized out.

See above for more reasons why it is beneficial (knowing when it is an munmap/mremap versus something else).

I would have not thought that passing down information as something that controversial. Hopes this help you see the benefit of this.

From: Jérôme Glisse <[email protected]>

Hi Andrew, i see that you still have my event patch in you queue [1]. This patchset replace that single patch and is broken down in further step so that it is easier to review and ascertain that no mistake were made during mechanical changes. Here are the step:

Patch 1 - add the enum values
Patch 2 - coccinelle semantic patch to convert all call site of
mmu_notifier_range_init to default enum value and also to passing down the vma when it is available
Patch 3 - update many call site to more accurate enum values Patch 4 - add the information to the mmu_notifier_range struct Patch 5 - helper to test if a range is updated to read only

All the remaining patches are update to various driver to demonstrate how this new information get use by device driver. I build tested with make all and make all minus everything that enable mmu notifier ie building with MMU_NOTIFIER=no. Also tested with some radeon,amd gpu and intel gpu.

If they are no objections i believe best plan would be to merge the the first 5 patches (all mm changes) through your queue for 5.1 and then to delay driver update to each individual driver tree for 5.2. This will allow each individual device driver maintainer time to more thouroughly test this more then my own testing.

Note that i also intend to use this feature further in nouveau and HMM down the road. I also expect that other user like KVM might be interested into leveraging this new information to optimize some of there secondary page table invalidation.

Here is an explaination on the rational for this patchset:

CPU page table update can happens for many reasons, not only as a result of a syscall (munmap(), mprotect(), mremap(), madvise(), ...) but also as a result of kernel activities (memory compression, reclaim, migration, ...).

This patch introduce a set of enums that can be associated with each of the events triggering a mmu notifier. Latter patches take advantages of those enum values.

- UNMAP: munmap() or mremap()
- CLEAR: page table is cleared (migration, compaction, reclaim, ...)
- PROTECTION_VMA: change in access protections for the range
- PROTECTION_PAGE: change in access protections for page in the range
- SOFT_DIRTY: soft dirtyness tracking

Being able to identify munmap() and mremap() from other reasons why the page table is cleared is important to allow user of mmu notifier to update their own internal tracking structure accordingly (on munmap or mremap it is not longer needed to track range of virtual address as it becomes invalid).

[1] https://www.ozlabs.org/~akpm/mmotm/broken-out/mm-mmu_notifier-contextual-information-for-event-triggering-invalidation-v2.patch

Cc Christian König <[email protected]>

Cc Jan Kara <[email protected]>

Cc Felix Kuehling <[email protected]>

Cc Jason Gunthorpe <[email protected]>

Cc Andrew Morton <[email protected]>

Cc Matthew Wilcox <[email protected]>

Cc Ross Zwisler <[email protected]>

Cc Dan Williams <[email protected]>

Cc Paolo Bonzini <[email protected]>

Cc Radim Krčmář <[email protected]>

Cc Michal Hocko <[email protected]>

Cc Ralph Campbell <[email protected]>

Cc John Hubbard <[email protected]>

Cc [email protected]

Cc [email protected]

Cc [email protected]

Cc [email protected]

Cc Arnd Bergmann <[email protected]>

Jérôme Glisse (9):
mm/mmu_notifier: contextual information for event enums mm/mmu_notifier: contextual information for event triggering
invalidation
mm/mmu_notifier: use correct mmu_notifier events for each invalidation mm/mmu_notifier: pass down vma and reasons why mmu notifier is
happening
mm/mmu_notifier: mmu_notifier_range_update_to_read_only() helper gpu/drm/radeon: optimize out the case when a range is updated to read
only
gpu/drm/amdgpu: optimize out the case when a range is updated to read
only
gpu/drm/i915: optimize out the case when a range is updated to read
only
RDMA/umem_odp: optimize out the case when a range is updated to read
only

 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c  | 13 ++++++++
 drivers/gpu/drm/i915/i915_gem_userptr.c | 16 ++++++++++
 drivers/gpu/drm/radeon/radeon_mn.c      | 13 ++++++++
 drivers/infiniband/core/umem_odp.c      | 22 +++++++++++--
 fs/proc/task_mmu.c                      |  3 +-
 include/linux/mmu_notifier.h            | 42 ++++++++++++++++++++++++-
 include/rdma/ib_umem_odp.h              |  1 +
 kernel/events/uprobes.c                 |  3 +-
 mm/huge_memory.c                        | 14 +++++----
 mm/hugetlb.c                            | 11 ++++---
 mm/khugepaged.c                         |  3 +-
 mm/ksm.c                                |  6 ++--
 mm/madvise.c                            |  3 +-

 mm/memory.c                             | 25 +++++++++------
 mm/migrate.c                            |  5 ++-
 mm/mmu_notifier.c                       | 10 ++++++
 mm/mprotect.c                           |  4 ++-
 mm/mremap.c                             |  3 +-
 mm/oom_kill.c                           |  3 +-
 mm/rmap.c                               |  6 ++--
 20 files changed, 171 insertions(+), 35 deletions(-)

Signature · 1 line

2.17.2

Andrew what is your plan for this ? I had a discussion with Peter Xu and Andrea about change_pte() and kvm. Today the change_pte() kvm optimization is effectively disabled because of invalidate_range calls. With a minimal couple lines patch on top of this patchset we can bring back the kvm change_pte optimization and we can also optimize some other cases like for instance when write protecting after fork (but i am not sure this is something qemu does often so it might not help for real kvm workload).

I will be posting a the extra patch as an RFC, but in the meantime i wanted to know what was the status for this.

Quoted 11 lines · On Thu, 31 Jan 2019 11:10:06 -0500 Jerome Glisse <[email protected]> wrote:

Andrew what is your plan for this ? I had a discussion with Peter Xu and Andrea about change_pte() and kvm. Today the change_pte() kvm optimization is effectively disabled because of invalidate_range calls. With a minimal couple lines patch on top of this patchset we can bring back the kvm change_pte optimization and we can also optimize some other cases like for instance when write protecting after fork (but i am not sure this is something qemu does often so it might not help for real kvm workload).

I will be posting a the extra patch as an RFC, but in the meantime i wanted to know what was the status for this.

The various drm patches appear to be headed for collisions with drm tree development so we'll need to figure out how to handle that and in what order things happen.

It's quite unclear from the v4 patchset's changelogs that this has anything to do with KVM and "the change_pte() kvm optimization" hasn't been described anywhere(?).

So.. I expect the thing to do here is to get everything finished, get the changelogs completed with this new information and do a resend.

Can we omit the drm and rdma patches for now? Feed them in via the subsystem maintainers when the dust has settled?

Andrew what is your plan for this ? I had a discussion with Peter Xu and Andrea about change_pte() and kvm. Today the change_pte() kvm optimization is effectively disabled because of invalidate_range calls. With a minimal couple lines patch on top of this patchset we can bring back the kvm change_pte optimization and we can also optimize some other cases like for instance when write protecting after fork (but i am not sure this is something qemu does often so it might not help for real kvm workload).

I will be posting a the extra patch as an RFC, but in the meantime i wanted to know what was the status for this.

Jan, Christian does your previous ACK still holds for this ?

Quoted 116 lines

From: Jérôme Glisse <[email protected]>

Hi Andrew, i see that you still have my event patch in you queue [1]. This patchset replace that single patch and is broken down in further step so that it is easier to review and ascertain that no mistake were made during mechanical changes. Here are the step:

Patch 1 - add the enum values
Patch 2 - coccinelle semantic patch to convert all call site of
mmu_notifier_range_init to default enum value and also to passing down the vma when it is available
Patch 3 - update many call site to more accurate enum values Patch 4 - add the information to the mmu_notifier_range struct Patch 5 - helper to test if a range is updated to read only

All the remaining patches are update to various driver to demonstrate how this new information get use by device driver. I build tested with make all and make all minus everything that enable mmu notifier ie building with MMU_NOTIFIER=no. Also tested with some radeon,amd gpu and intel gpu.

If they are no objections i believe best plan would be to merge the the first 5 patches (all mm changes) through your queue for 5.1 and then to delay driver update to each individual driver tree for 5.2. This will allow each individual device driver maintainer time to more thouroughly test this more then my own testing.

Note that i also intend to use this feature further in nouveau and HMM down the road. I also expect that other user like KVM might be interested into leveraging this new information to optimize some of there secondary page table invalidation.

Here is an explaination on the rational for this patchset:

CPU page table update can happens for many reasons, not only as a result of a syscall (munmap(), mprotect(), mremap(), madvise(), ...) but also as a result of kernel activities (memory compression, reclaim, migration, ...).

This patch introduce a set of enums that can be associated with each of the events triggering a mmu notifier. Latter patches take advantages of those enum values.

- UNMAP: munmap() or mremap()
- CLEAR: page table is cleared (migration, compaction, reclaim, ...)
- PROTECTION_VMA: change in access protections for the range
- PROTECTION_PAGE: change in access protections for page in the range
- SOFT_DIRTY: soft dirtyness tracking

Being able to identify munmap() and mremap() from other reasons why the page table is cleared is important to allow user of mmu notifier to update their own internal tracking structure accordingly (on munmap or mremap it is not longer needed to track range of virtual address as it becomes invalid).

[1] https://www.ozlabs.org/~akpm/mmotm/broken-out/mm-mmu_notifier-contextual-information-for-event-triggering-invalidation-v2.patch

Cc Christian König <[email protected]>

Cc Jan Kara <[email protected]>

Cc Felix Kuehling <[email protected]>

Cc Jason Gunthorpe <[email protected]>

Cc Andrew Morton <[email protected]>

Cc Matthew Wilcox <[email protected]>

Cc Ross Zwisler <[email protected]>

Cc Dan Williams <[email protected]>

Cc Paolo Bonzini <[email protected]>

Cc Radim Krčmář <[email protected]>

Cc Michal Hocko <[email protected]>

Cc Ralph Campbell <[email protected]>

Cc John Hubbard <[email protected]>

Cc [email protected]

Cc [email protected]

Cc [email protected]

Cc [email protected]

Cc Arnd Bergmann <[email protected]>

Jérôme Glisse (9):
mm/mmu_notifier: contextual information for event enums mm/mmu_notifier: contextual information for event triggering
invalidation
mm/mmu_notifier: use correct mmu_notifier events for each invalidation mm/mmu_notifier: pass down vma and reasons why mmu notifier is
happening
mm/mmu_notifier: mmu_notifier_range_update_to_read_only() helper gpu/drm/radeon: optimize out the case when a range is updated to read
only
gpu/drm/amdgpu: optimize out the case when a range is updated to read
only
gpu/drm/i915: optimize out the case when a range is updated to read
only
RDMA/umem_odp: optimize out the case when a range is updated to read
only

  drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c  | 13 ++++++++
  drivers/gpu/drm/i915/i915_gem_userptr.c | 16 ++++++++++
  drivers/gpu/drm/radeon/radeon_mn.c      | 13 ++++++++
  drivers/infiniband/core/umem_odp.c      | 22 +++++++++++--
  fs/proc/task_mmu.c                      |  3 +-
  include/linux/mmu_notifier.h            | 42 ++++++++++++++++++++++++-
  include/rdma/ib_umem_odp.h              |  1 +
  kernel/events/uprobes.c                 |  3 +-
  mm/huge_memory.c                        | 14 +++++----
  mm/hugetlb.c                            | 11 ++++---
  mm/khugepaged.c                         |  3 +-
  mm/ksm.c                                |  6 ++--
  mm/madvise.c                            |  3 +-
  mm/memory.c                             | 25 +++++++++------
  mm/migrate.c                            |  5 ++-
  mm/mmu_notifier.c                       | 10 ++++++
  mm/mprotect.c                           |  4 ++-
  mm/mremap.c                             |  3 +-
  mm/oom_kill.c                           |  3 +-
  mm/rmap.c                               |  6 ++--
  20 files changed, 171 insertions(+), 35 deletions(-)

Quoted 3 lines

Signature · 1 line

2.17.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Andrew what is your plan for this ? I had a discussion with Peter Xu and Andrea about change_pte() and kvm. Today the change_pte() kvm optimization is effectively disabled because of invalidate_range calls. With a minimal couple lines patch on top of this patchset we can bring back the kvm change_pte optimization and we can also optimize some other cases like for instance when write protecting after fork (but i am not sure this is something qemu does often so it might not help for real kvm workload).

I will be posting a the extra patch as an RFC, but in the meantime i wanted to know what was the status for this.

Jan, Christian does your previous ACK still holds for this ?

Quoted 10 lines · Jerome Glisse

Andrew what is your plan for this ? I had a discussion with Peter Xu and Andrea about change_pte() and kvm. Today the change_pte() kvm optimization is effectively disabled because of invalidate_range calls. With a minimal couple lines patch on top of this patchset we can bring back the kvm change_pte optimization and we can also optimize some other cases like for instance when write protecting after fork (but i am not sure this is something qemu does often so it might not help for real kvm workload).

Quoted 4 lines

I will be posting a the extra patch as an RFC, but in the meantime i wanted to know what was the status for this.

Jan, Christian does your previous ACK still holds for this ?

Yes, I still think the approach makes sense. Dan's concern about in tree users is valid but it seems you have those just not merged yet, right?