Skip to contentSkip to conversation

DGFX mmap with rpm

PATCH Superseded Anshuman Gupta 4 participants 9 messages Aug 25, 2022 – Aug 30, 2022

Subsystems (2)

v1 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

cover DGFX mmap with rpm

As per PCIe Spec Section 5.3.1.4.1
When pcie function is in d3hot state,
Configuration and Message requests are the only TLPs accepted by a Function in the D3hot state. All other received Requests must be handled as Unsupported Requests, and all received Completions may optionally be handled as Unexpected Completions.

Therefore when gfx endpoint function is in d3 state, all pcie iomem transaction requires to transition the pcie function in D0 state.

This is formal patch extension of
RFC proposal https://patchwork.freedesktop.org/series/107400/

Implementation of handling i915_gem_object_pin_map will be handled in different series.

Anshuman Gupta (1):
drm/i915/dgfx: Release mmap on rpm suspend

 .../gpu/drm/i915/gem/i915_gem_object_types.h  |  3 +-
 drivers/gpu/drm/i915/gem/i915_gem_ttm.c       | 48 ++++++++++++++++---
 drivers/gpu/drm/i915/gt/intel_gt.c            |  2 +
 drivers/gpu/drm/i915/gt/intel_gt_types.h      |  3 ++
 drivers/gpu/drm/i915/i915_gem.c               |  8 ++++
 5 files changed, 57 insertions(+), 7 deletions(-)
Signature · 1 line
2.26.2
#1 lore raw

PATCH 1/1 drm/i915/dgfx: Release mmap on rpm suspend

Release all mmap mapping for all lmem objects which are associated with userfault such that, while pcie function in D3hot, any access to memory mappings will raise a userfault.

Runtime resume the dgpu(when gem object lies in lmem). This will transition the dgpu graphics function to D0 state if it was in D3 in order to access the mmap memory mappings.

v2:
- Squashes the patches. [Matt Auld]
- Add adequate locking for lmem_userfault_list addition. [Matt Auld]
- Reused obj->userfault_count to avoid double addition. [Matt Auld]
- Added i915_gem_object_lock to check i915_gem_object_is_lmem. [Matt Auld]

---
 .../gpu/drm/i915/gem/i915_gem_object_types.h  |  3 +-
 drivers/gpu/drm/i915/gem/i915_gem_ttm.c       | 48 ++++++++++++++++---
 drivers/gpu/drm/i915/gt/intel_gt.c            |  2 +
 drivers/gpu/drm/i915/gt/intel_gt_types.h      |  3 ++
 drivers/gpu/drm/i915/i915_gem.c               |  8 ++++
 5 files changed, 57 insertions(+), 7 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_object_types.h +2 -1
index 9f6b14ec189a..40305e2bcd49 100644
@@ -298,7 +298,8 @@ struct drm_i915_gem_object {
 	};
 
 	/**
-	 * Whether the object is currently in the GGTT mmap.
+	 * Whether the object is currently in the GGTT or any other supported
+	 * fake offset mmap backed by lmem.
 	 */
 	unsigned int userfault_count;
 	struct list_head userfault_link;
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +42 -6

91 lines not on this page · raw message

drivers/gpu/drm/i915/gt/intel_gt.c +2 -0
index e4bac2431e41..63616ea198ca 100644
@@ -39,6 +39,8 @@ static void __intel_gt_init_early(struct intel_gt *gt)
 {
 	spin_lock_init(&gt->irq_lock);
 
+	INIT_LIST_HEAD(&gt->lmem_userfault_list);
+	mutex_init(&gt->lmem_userfault_lock);
 	INIT_LIST_HEAD(&gt->closed_vma);
 	spin_lock_init(&gt->closed_lock);
 
drivers/gpu/drm/i915/gt/intel_gt_types.h +3 -0
index 4d56f7d5a3be..330e7531cf22 100644
@@ -132,6 +132,9 @@ struct intel_gt {
 	struct intel_wakeref wakeref;
 	atomic_t user_wakeref;
 
+	struct mutex lmem_userfault_lock; /* Protects access to usefault list */
+	struct list_head lmem_userfault_list;
+
 	struct list_head closed_vma;
 	spinlock_t closed_lock; /* guards the list of closed_vma */
 
drivers/gpu/drm/i915/i915_gem.c +8 -0
index 702e5b89be22..288dc7e231dc 100644
@@ -842,6 +842,14 @@ void i915_gem_runtime_suspend(struct drm_i915_private *i915)
 				 &to_gt(i915)->ggtt->userfault_list, userfault_link)
 		__i915_gem_object_release_mmap_gtt(obj);
 
+	list_for_each_entry_safe(obj, on,
+				 &to_gt(i915)->lmem_userfault_list, userfault_link) {
+		i915_gem_object_release_mmap_offset(obj);
+
+		if (!--obj->userfault_count)
+			list_del(&obj->userfault_link);
+	}
+
 	/*
 	 * The fence will be lost when the device powers down. If any were
 	 * in use by hardware (i.e. they are pinned), we should not be powering
Signature · 1 line
2.26.2
#2 lore raw

Re: [Intel-gfx] [PATCH 1/1] drm/i915/dgfx: Release mmap on rpm suspend

Quoted 9 lines · Anshuman Gupta

Release all mmap mapping for all lmem objects which are associated with userfault such that, while pcie function in D3hot, any access to memory mappings will raise a userfault.

Runtime resume the dgpu(when gem object lies in lmem). This will transition the dgpu graphics function to D0 state if it was in D3 in order to access the mmap memory mappings.

Quoted 8 lines

v2:
- Squashes the patches. [Matt Auld]
- Add adequate locking for lmem_userfault_list addition. [Matt Auld]
- Reused obj->userfault_count to avoid double addition. [Matt Auld]
- Added i915_gem_object_lock to check i915_gem_object_is_lmem. [Matt Auld]

Just double checking, this is needed for DG1 and DG2, right? Are there any BSpec links we can add here?

Quoted 60 lines
Cc Matthew Auld <[email protected]>
Cc Rodrigo Vivi <[email protected]>
---
  .../gpu/drm/i915/gem/i915_gem_object_types.h  |  3 +-
  drivers/gpu/drm/i915/gem/i915_gem_ttm.c       | 48 ++++++++++++++++---
  drivers/gpu/drm/i915/gt/intel_gt.c            |  2 +
  drivers/gpu/drm/i915/gt/intel_gt_types.h      |  3 ++
  drivers/gpu/drm/i915/i915_gem.c               |  8 ++++
  5 files changed, 57 insertions(+), 7 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_object_types.h +2 -1
index 9f6b14ec189a..40305e2bcd49 100644
@@ -298,7 +298,8 @@ struct drm_i915_gem_object {
  	};
 
  	/**
-	 * Whether the object is currently in the GGTT mmap.
+	 * Whether the object is currently in the GGTT or any other supported
+	 * fake offset mmap backed by lmem.
  	 */
  	unsigned int userfault_count;
  	struct list_head userfault_link;
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +17 -0
index 5a5cf332d8a5..6532a634bd20 100644
@@ -1014,12 +1014,29 @@ static void i915_ttm_delayed_free(struct drm_i915_gem_object *obj)
  	ttm_bo_put(i915_gem_to_ttm(obj));
  }
 
+static intel_wakeref_t
+i915_gem_ttm_get_lmem_obj_wakeref(struct drm_i915_gem_object *obj)
+{
+	intel_wakeref_t wakeref = 0;
+
+	if (i915_gem_object_lock_interruptible(obj, NULL))
+		return 0;
+
+	if (i915_gem_object_is_lmem(obj))
+		wakeref = intel_runtime_pm_get(&to_i915(obj->base.dev)->runtime_pm);
+
+	i915_gem_object_unlock(obj);
+
+	return wakeref;
+}
+
  static vm_fault_t vm_fault_ttm(struct vm_fault *vmf)
  {
  	struct vm_area_struct *area = vmf->vma;
  	struct ttm_buffer_object *bo = area->vm_private_data;
  	struct drm_device *dev = bo->base.dev;
  	struct drm_i915_gem_object *obj;
+	intel_wakeref_t wakeref = 0;
  	vm_fault_t ret;
  	int idx;
 

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault *vmf) · line 1047

Quoted 4 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +1 -0
@@ -1027,18 +1044,23 @@ static vm_fault_t vm_fault_ttm(struct vm_fault *vmf)
  	if (!obj)
  		return VM_FAULT_SIGBUS;
 
+	wakeref = i915_gem_ttm_get_lmem_obj_wakeref(obj);

We shouldn't drop the lock here (also failing to acquire the lock should be fatal), since the object can in thoery transition to/from lmem inbetween dropping the object lock here and re-acquiring it again below, which means we might skip grabbing the wakeref here, but then later touch the list, if say it moves to lmem.

Quoted 33 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +10 -5
+
  	/* Sanity check that we allow writing into this object */
  	if (unlikely(i915_gem_object_is_readonly(obj) &&
-		     area->vm_flags & VM_WRITE))
-		return VM_FAULT_SIGBUS;
+		     area->vm_flags & VM_WRITE)) {
+		ret = VM_FAULT_SIGBUS;
+		goto out_rpm;
+	}
 
  	ret = ttm_bo_vm_reserve(bo, vmf); >   	if (ret)
-		return ret;
+		goto out_rpm;
 
  	if (obj->mm.madv != I915_MADV_WILLNEED) {
  		dma_resv_unlock(bo->base.resv);
-		return VM_FAULT_SIGBUS;
+		ret = VM_FAULT_SIGBUS;
+		goto out_rpm;
  	}
 
  	if (!i915_ttm_resource_mappable(bo->resource)) {
@@ -1062,7 +1084,8 @@ static vm_fault_t vm_fault_ttm(struct vm_fault *vmf)
  		if (err) {
  			drm_dbg(dev, "Unable to make resource CPU accessible\n");
  			dma_resv_unlock(bo->base.resv);
-			return VM_FAULT_SIGBUS;
+			ret = VM_FAULT_SIGBUS;
+			goto out_rpm;
  		}
  	}
 

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault *vmf) · line 1101

Quoted 6 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +3 -0
@@ -1073,12 +1096,25 @@ static vm_fault_t vm_fault_ttm(struct vm_fault *vmf)
  	} else {
  		ret = ttm_bo_vm_dummy_page(vmf, vmf->vma->vm_page_prot);
  	}
+
+	/* ttm_bo_vm_reserve() already has dma_resv_lock */
+	if (!ret && i915_gem_object_is_lmem(obj) && !obj->userfault_count++) {

This might increment userfault_count more than once per mapping/object? Is that intentional? I would have thought that with fault_ttm the user_fault_count shouldn't ever be > 1 (there is no partial mmap here).

Also it look like ret == VM_FAULT_NOPAGE if all went well. Although it also returns that if the wait was interrupted, but I don't think that matters much.

Maybe we can do something like:

ret = ttm_bo_vm_reserve(bo..) /* locks the object for us */ ..

wakeref = 0;
if (i915_ttm_cpu_maps_iomem(bo->resource))
wakeref = intel_runtime_pm_get();

if (drm_dev_enter(dev, &idx)) {
} else {
}

if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT))
goto out_rpm;

if (ret == VM_FAULT_NOPAGE && wakeref && !obj->userfault_count) {
obj->userfault_count++; /* protected by wakeref */ mutex_lock()
list_add();
mutex_unlock();
}

?

AFAICT we are then still missing something for object destruction, to ensure we remove the object from the list. Otherwise we can destroy the object, and then at some later point i915_gem_runtime_suspend() runs and will uaf on the list. I think the trick with GTT mmap is that it will always grab the wakeref when unbinding the vma (like during object destruction), and then it can safely remove the object from the list. Perhaps we will need a similar trick anyway, since destroying the object will likely require unbinding it, which will mean touching the page-tables from the CPU, which are in lmem anyway. Do we maybe just always grab the wakeref for dgpu? What do you think?

Also on second thought it looks like we can't directly call i915_gem_object_release_mmap_offset() from i915_gem_runtime_suspend(), without then also holding the object lock for the ttm_mem_io_free(). But I think it should be OK to just directly call drm_vma_node_unmap(&bo->base.vma_node, ...) instead.

Also, do you know if we need something like CONFIG_DRM_I915_USERFAULT_AUTOSUSPEND here?

Quoted 66 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +11 -1
+		mutex_lock(&to_gt(to_i915(obj->base.dev))->lmem_userfault_lock);
+		list_add(&obj->userfault_link, &to_gt(to_i915(obj->base.dev))->lmem_userfault_list);
+		mutex_unlock(&to_gt(to_i915(obj->base.dev))->lmem_userfault_lock);
+	}
+
  	if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT))
-		return ret;
+		goto out_rpm;
 
  	i915_ttm_adjust_lru(obj);
 
  	dma_resv_unlock(bo->base.resv);
+
+out_rpm:
+	if (wakeref)
+		intel_runtime_pm_put(&to_i915(obj->base.dev)->runtime_pm, wakeref);
+
  	return ret;
  }
 
drivers/gpu/drm/i915/gt/intel_gt.c +2 -0
index e4bac2431e41..63616ea198ca 100644
@@ -39,6 +39,8 @@ static void __intel_gt_init_early(struct intel_gt *gt)
  {
  	spin_lock_init(&gt->irq_lock);
 
+	INIT_LIST_HEAD(&gt->lmem_userfault_list);
+	mutex_init(&gt->lmem_userfault_lock);
  	INIT_LIST_HEAD(&gt->closed_vma);
  	spin_lock_init(&gt->closed_lock);
 
drivers/gpu/drm/i915/gt/intel_gt_types.h +3 -0
index 4d56f7d5a3be..330e7531cf22 100644
@@ -132,6 +132,9 @@ struct intel_gt {
  	struct intel_wakeref wakeref;
  	atomic_t user_wakeref;
 
+	struct mutex lmem_userfault_lock; /* Protects access to usefault list */
+	struct list_head lmem_userfault_list;
+
  	struct list_head closed_vma;
  	spinlock_t closed_lock; /* guards the list of closed_vma */
 
drivers/gpu/drm/i915/i915_gem.c +8 -0
index 702e5b89be22..288dc7e231dc 100644
@@ -842,6 +842,14 @@ void i915_gem_runtime_suspend(struct drm_i915_private *i915)
  				 &to_gt(i915)->ggtt->userfault_list, userfault_link)
  		__i915_gem_object_release_mmap_gtt(obj);
 
+	list_for_each_entry_safe(obj, on,
+				 &to_gt(i915)->lmem_userfault_list, userfault_link) {
+		i915_gem_object_release_mmap_offset(obj);
+
+		if (!--obj->userfault_count)
+			list_del(&obj->userfault_link);
+	}
+
  	/*
  	 * The fence will be lost when the device powers down. If any were
  	 * in use by hardware (i.e. they are pinned), we should not be powering
#3 lore raw

Re: [Intel-gfx] [PATCH 1/1] drm/i915/dgfx: Release mmap on rpm suspend

Quoted 4 lines

-----Original Message-----
From: Auld, Matthew <[email protected]>

Thanks Matt for reviewing this.

Quoted 23 lines

Sent: Friday, August 26, 2022 11:09 PM
To: Gupta, Anshuman <[email protected]>; intel- [email protected]
Cc: [email protected]; Vivi, Rodrigo <[email protected]>; Nilawar, Badal <[email protected]>; [email protected]
Subject: Re: [PATCH 1/1] drm/i915/dgfx: Release mmap on rpm suspend

Quoted 15 lines · Anshuman Gupta

Release all mmap mapping for all lmem objects which are associated with userfault such that, while pcie function in D3hot, any access to memory mappings will raise a userfault.

Runtime resume the dgpu(when gem object lies in lmem). This will transition the dgpu graphics function to D0 state if it was in D3 in order to access the mmap memory mappings.

v2:
- Squashes the patches. [Matt Auld]
- Add adequate locking for lmem_userfault_list addition. [Matt Auld]
- Reused obj->userfault_count to avoid double addition. [Matt Auld]
- Added i915_gem_object_lock to check i915_gem_object_is_lmem. [Matt Auld]

Quoted 4 lines
Quoted 1 line

Just double checking, this is needed for DG1 and DG2, right? Are there any BSpec links we can add here?

This is specific to all discrete products with respect to PCIe Spec Section 5.3.1.4.1 I will add the PCIe specs link here.

Quoted 68 lines

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static void i915_ttm_delayed_free(struct

Quoted 33 lines
Cc Matthew Auld <[email protected]>
Cc Rodrigo Vivi <[email protected]>
---
  .../gpu/drm/i915/gem/i915_gem_object_types.h  |  3 +-
  drivers/gpu/drm/i915/gem/i915_gem_ttm.c       | 48 ++++++++++++++++---
  drivers/gpu/drm/i915/gt/intel_gt.c            |  2 +
  drivers/gpu/drm/i915/gt/intel_gt_types.h      |  3 ++
  drivers/gpu/drm/i915/i915_gem.c               |  8 ++++
  5 files changed, 57 insertions(+), 7 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_object_types.h +2 -1
index 9f6b14ec189a..40305e2bcd49 100644
@@ -298,7 +298,8 @@ struct drm_i915_gem_object {
  	};
 
  	/**
-	 * Whether the object is currently in the GGTT mmap.
+	 * Whether the object is currently in the GGTT or any other supported
+	 * fake offset mmap backed by lmem.
  	 */
  	unsigned int userfault_count;
  	struct list_head userfault_link;
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +0 -0
index 5a5cf332d8a5..6532a634bd20 100644
@@ -1014,12 +1014,29 @@ static void i915_ttm_delayed_free(struct

drm_i915_gem_object *obj)

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault

Quoted 30 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +17 -0
  	ttm_bo_put(i915_gem_to_ttm(obj));
  }
 
+static intel_wakeref_t
+i915_gem_ttm_get_lmem_obj_wakeref(struct drm_i915_gem_object *obj) {
+	intel_wakeref_t wakeref = 0;
+
+	if (i915_gem_object_lock_interruptible(obj, NULL))
+		return 0;
+
+	if (i915_gem_object_is_lmem(obj))
+		wakeref =
+intel_runtime_pm_get(&to_i915(obj->base.dev)->runtime_pm);
+
+	i915_gem_object_unlock(obj);
+
+	return wakeref;
+}
+
  static vm_fault_t vm_fault_ttm(struct vm_fault *vmf)
  {
  	struct vm_area_struct *area = vmf->vma;
  	struct ttm_buffer_object *bo = area->vm_private_data;
  	struct drm_device *dev = bo->base.dev;
  	struct drm_i915_gem_object *obj;
+	intel_wakeref_t wakeref = 0;
  	vm_fault_t ret;
  	int idx;
 
@@ -1027,18 +1044,23 @@ static vm_fault_t vm_fault_ttm(struct vm_fault

*vmf)

Quoted 3 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +0 -0
  	if (!obj)
  		return VM_FAULT_SIGBUS;
 
Quoted 6 lines

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault *vmf) · line 1047

Quoted 1 line
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +1 -0
+	wakeref = i915_gem_ttm_get_lmem_obj_wakeref(obj);

We shouldn't drop the lock here (also failing to acquire the lock should be fatal), since the object can in thoery transition to/from lmem inbetween dropping the object lock here and re-acquiring it again below, which means we might skip grabbing the wakeref here, but then later touch the list, if say it moves to lmem.

By not releasing the lock there was a deadlock from ttm_bo_vm_reserve().

Quoted 46 lines

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault

Quoted 24 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +8 -4
+
  	/* Sanity check that we allow writing into this object */
  	if (unlikely(i915_gem_object_is_readonly(obj) &&
-		     area->vm_flags & VM_WRITE))
-		return VM_FAULT_SIGBUS;
+		     area->vm_flags & VM_WRITE)) {
+		ret = VM_FAULT_SIGBUS;
+		goto out_rpm;
+	}
 
  	ret = ttm_bo_vm_reserve(bo, vmf); >   	if (ret)
-		return ret;
+		goto out_rpm;
 
  	if (obj->mm.madv != I915_MADV_WILLNEED) {
  		dma_resv_unlock(bo->base.resv);
-		return VM_FAULT_SIGBUS;
+		ret = VM_FAULT_SIGBUS;
+		goto out_rpm;
  	}
 
  	if (!i915_ttm_resource_mappable(bo->resource)) {
@@ -1062,7 +1084,8 @@ static vm_fault_t vm_fault_ttm(struct vm_fault

*vmf)

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault · line 1085

Quoted 2 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +0 -0
  		if (err) {
  			drm_dbg(dev, "Unable to make resource CPU

accessible\n");

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault

Quoted 8 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +2 -1
  			dma_resv_unlock(bo->base.resv);
-			return VM_FAULT_SIGBUS;
+			ret = VM_FAULT_SIGBUS;
+			goto out_rpm;
  		}
  	}
 
@@ -1073,12 +1096,25 @@ static vm_fault_t vm_fault_ttm(struct vm_fault

*vmf)

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault *vmf)

Quoted 7 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +3 -0
  	} else {
  		ret = ttm_bo_vm_dummy_page(vmf, vmf->vma-
 vm_page_prot);
  	}
+
+	/* ttm_bo_vm_reserve() already has dma_resv_lock */
+	if (!ret && i915_gem_object_is_lmem(obj) && !obj->userfault_count++)

{

Quoted 3 lines

This might increment userfault_count more than once per mapping/object? Is that intentional? I would have thought that with fault_ttm the user_fault_count shouldn't ever be > 1 (there is no partial mmap here).

I did not get this part, the double addition for the object is possible only when ttm_fault will get called multiple times for an object ? In that case isn't userfault_count would be > 1 ?

Quoted 21 lines

Also it look like ret == VM_FAULT_NOPAGE if all went well. Although it also returns that if the wait was interrupted, but I don't think that matters much.

Maybe we can do something like:

ret = ttm_bo_vm_reserve(bo..) /* locks the object for us */ ..

wakeref = 0;
if (i915_ttm_cpu_maps_iomem(bo->resource))

    wakeref = intel_runtime_pm_get();

if (drm_dev_enter(dev, &idx)) {
} else {
}

if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT))
goto out_rpm;

Quoted 8 lines
if (ret == VM_FAULT_NOPAGE && wakeref && !obj->userfault_count) {
   obj->userfault_count++; /* protected by wakeref */
   mutex_lock()
   list_add();
   mutex_unlock();
}

?

Thanks for suggestion, I will implement it.

Quoted 11 lines

AFAICT we are then still missing something for object destruction, to ensure we remove the object from the list. Otherwise we can destroy the object, and then at some later point i915_gem_runtime_suspend() runs and will uaf on the list. I think the trick with GTT mmap is that it will always grab the wakeref when unbinding the vma (like during object destruction), and then it can safely remove the object from the list. Perhaps we will need a similar trick anyway, since destroying the object will likely require unbinding it, which will mean touching the page-tables from the CPU, which are in lmem anyway. Do we maybe just always grab the wakeref for dgpu? What do you think?

With input from @rodrigo , we can't get permanent wakref as that will
block the d3cold(it will burn the card power).
How about deleting the object from the list in __i915_gem_object_free_mmaps() to handle the object destroy case ?
If this approach is not scalable then probably we need to think over to get the wakref on dGPU whenever any client is connected.
I was also thinking in direct to limit the mmap mapping for lmem object by using a new mmap fake offset to limit the rpm suspend latency due to revoking and re-creation of mmap ?

Quoted 9 lines

Also on second thought it looks like we can't directly call i915_gem_object_release_mmap_offset() from i915_gem_runtime_suspend(), without then also holding the object lock for the ttm_mem_io_free(). But I think it should be OK to just directly call drm_vma_node_unmap(&bo->base.vma_node, ...) instead.

Also, do you know if we need something like
CONFIG_DRM_I915_USERFAULT_AUTOSUSPEND here?

It is to induced additional delay before actual runtime pm triggers. Yeah it make sense to use that , I missed earlier to incorporate this comment. Thanks,
Anshuman Gupta.

Quoted 78 lines

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault · line 1111

Quoted 10 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +5 -0
+		mutex_lock(&to_gt(to_i915(obj->base.dev))-
 lmem_userfault_lock);
+		list_add(&obj->userfault_link, &to_gt(to_i915(obj->base.dev))-
 lmem_userfault_list);
+		mutex_unlock(&to_gt(to_i915(obj->base.dev))-
 lmem_userfault_lock);
+	}
+
  	if (ret == VM_FAULT_RETRY && !(vmf->flags &

FAULT_FLAG_RETRY_NOWAIT))

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault *vmf)

Quoted 10 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +5 -1
-		return ret;
+		goto out_rpm;
 
  	i915_ttm_adjust_lru(obj);
 
  	dma_resv_unlock(bo->base.resv);
+
+out_rpm:
+	if (wakeref)
+		intel_runtime_pm_put(&to_i915(obj->base.dev)->runtime_pm,

wakeref);

Review

Quoted 5 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +1 -0
+
  	return ret;
  }
(quoted diff) +0 -0

b/drivers/gpu/drm/i915/gt/intel_gt.c

Review

Quoted 13 lines

index e4bac2431e41..63616ea198ca 100644

drivers/gpu/drm/i915/gt/intel_gt.c +2 -0
@@ -39,6 +39,8 @@ static void __intel_gt_init_early(struct intel_gt *gt)
  {
  	spin_lock_init(&gt->irq_lock);
 
+	INIT_LIST_HEAD(&gt->lmem_userfault_list);
+	mutex_init(&gt->lmem_userfault_lock);
  	INIT_LIST_HEAD(&gt->closed_vma);
  	spin_lock_init(&gt->closed_lock);
 
(quoted diff) +0 -0

b/drivers/gpu/drm/i915/gt/intel_gt_types.h

Review

Quoted 14 lines

index 4d56f7d5a3be..330e7531cf22 100644

drivers/gpu/drm/i915/gt/intel_gt_types.h +3 -0
@@ -132,6 +132,9 @@ struct intel_gt {
  	struct intel_wakeref wakeref;
  	atomic_t user_wakeref;
 
+	struct mutex lmem_userfault_lock; /* Protects access to usefault list */
+	struct list_head lmem_userfault_list;
+
  	struct list_head closed_vma;
  	spinlock_t closed_lock; /* guards the list of closed_vma */
 
(quoted diff) +0 -0

b/drivers/gpu/drm/i915/i915_gem.c

drivers/gpu/drm/i915/i915_gem.c void i915_gem_runtime_suspend(struct

Quoted 4 lines

index 702e5b89be22..288dc7e231dc 100644

drivers/gpu/drm/i915/i915_gem.c +0 -0
@@ -842,6 +842,14 @@ void i915_gem_runtime_suspend(struct

drm_i915_private *i915)

drivers/gpu/drm/i915/i915_gem.c void i915_gem_runtime_suspend(struct · line 842

Quoted 1 line
drivers/gpu/drm/i915/i915_gem.c +0 -0
  				 &to_gt(i915)->ggtt->userfault_list,

userfault_link)

drivers/gpu/drm/i915/i915_gem.c void i915_gem_runtime_suspend(struct drm_i915_private *i915)

Quoted 4 lines
drivers/gpu/drm/i915/i915_gem.c +2 -0
  		__i915_gem_object_release_mmap_gtt(obj);
 
+	list_for_each_entry_safe(obj, on,
+				 &to_gt(i915)->lmem_userfault_list,

userfault_link) {

Quoted 9 lines
drivers/gpu/drm/i915/i915_gem.c +6 -0
+		i915_gem_object_release_mmap_offset(obj);
+
+		if (!--obj->userfault_count)
+			list_del(&obj->userfault_link);
+	}
+
  	/*
  	 * The fence will be lost when the device powers down. If any were
  	 * in use by hardware (i.e. they are pinned), we should not be powering
#4 lore raw

Re: [Intel-gfx] [PATCH 1/1] drm/i915/dgfx: Release mmap on rpm suspend

Quoted 103 lines · Anshuman
Quoted 5 lines

-----Original Message-----
From: Auld, Matthew <[email protected]>

Thanks Matt for reviewing this.

Quoted 23 lines

Sent: Friday, August 26, 2022 11:09 PM
To: Gupta, Anshuman <[email protected]>; intel- [email protected]
Cc: [email protected]; Vivi, Rodrigo <[email protected]>; Nilawar, Badal <[email protected]>; [email protected]
Subject: Re: [PATCH 1/1] drm/i915/dgfx: Release mmap on rpm suspend

Quoted 15 lines · Anshuman Gupta

Release all mmap mapping for all lmem objects which are associated with userfault such that, while pcie function in D3hot, any access to memory mappings will raise a userfault.

Runtime resume the dgpu(when gem object lies in lmem). This will transition the dgpu graphics function to D0 state if it was in D3 in order to access the mmap memory mappings.

v2:
- Squashes the patches. [Matt Auld]
- Add adequate locking for lmem_userfault_list addition. [Matt Auld]
- Reused obj->userfault_count to avoid double addition. [Matt Auld]
- Added i915_gem_object_lock to check i915_gem_object_is_lmem. [Matt Auld]

Quoted 4 lines
Quoted 1 line

Just double checking, this is needed for DG1 and DG2, right? Are there any BSpec links we can add here?

This is specific to all discrete products with respect to PCIe Spec Section 5.3.1.4.1 I will add the PCIe specs link here.

Quoted 68 lines

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static void i915_ttm_delayed_free(struct

Quoted 33 lines
Cc Matthew Auld <[email protected]>
Cc Rodrigo Vivi <[email protected]>
---
  .../gpu/drm/i915/gem/i915_gem_object_types.h  |  3 +-
  drivers/gpu/drm/i915/gem/i915_gem_ttm.c       | 48 ++++++++++++++++---
  drivers/gpu/drm/i915/gt/intel_gt.c            |  2 +
  drivers/gpu/drm/i915/gt/intel_gt_types.h      |  3 ++
  drivers/gpu/drm/i915/i915_gem.c               |  8 ++++
  5 files changed, 57 insertions(+), 7 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_object_types.h +2 -1
index 9f6b14ec189a..40305e2bcd49 100644
@@ -298,7 +298,8 @@ struct drm_i915_gem_object {
    };
 
    /**
-    * Whether the object is currently in the GGTT mmap.
+    * Whether the object is currently in the GGTT or any other supported
+    * fake offset mmap backed by lmem.
     */
    unsigned int userfault_count;
    struct list_head userfault_link;
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +0 -0
index 5a5cf332d8a5..6532a634bd20 100644
@@ -1014,12 +1014,29 @@ static void i915_ttm_delayed_free(struct

drm_i915_gem_object *obj)

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault

Quoted 30 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +17 -0
    ttm_bo_put(i915_gem_to_ttm(obj));
  }
 
+static intel_wakeref_t
+i915_gem_ttm_get_lmem_obj_wakeref(struct drm_i915_gem_object *obj) {
+   intel_wakeref_t wakeref = 0;
+
+   if (i915_gem_object_lock_interruptible(obj, NULL))
+           return 0;
+
+   if (i915_gem_object_is_lmem(obj))
+           wakeref =
+intel_runtime_pm_get(&to_i915(obj->base.dev)->runtime_pm);
+
+   i915_gem_object_unlock(obj);
+
+   return wakeref;
+}
+
  static vm_fault_t vm_fault_ttm(struct vm_fault *vmf)
  {
    struct vm_area_struct *area = vmf->vma;
    struct ttm_buffer_object *bo = area->vm_private_data;
    struct drm_device *dev = bo->base.dev;
    struct drm_i915_gem_object *obj;
+   intel_wakeref_t wakeref = 0;
    vm_fault_t ret;
    int idx;
 
@@ -1027,18 +1044,23 @@ static vm_fault_t vm_fault_ttm(struct vm_fault

*vmf)

Quoted 3 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +0 -0
    if (!obj)
            return VM_FAULT_SIGBUS;
 
Quoted 7 lines
Quoted 6 lines

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault *vmf) · line 1047

Quoted 1 line
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +1 -0
+   wakeref = i915_gem_ttm_get_lmem_obj_wakeref(obj);

We shouldn't drop the lock here (also failing to acquire the lock should be fatal), since the object can in thoery transition to/from lmem inbetween dropping the object lock here and re-acquiring it again below, which means we might skip grabbing the wakeref here, but then later touch the list, if say it moves to lmem.

By not releasing the lock there was a deadlock from ttm_bo_vm_reserve().

Right, so we should probably move the check under vm_reserve.

Quoted 44 lines
Quoted 44 lines

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault

Quoted 24 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +8 -4
+
    /* Sanity check that we allow writing into this object */
    if (unlikely(i915_gem_object_is_readonly(obj) &&
-                area->vm_flags & VM_WRITE))
-           return VM_FAULT_SIGBUS;
+                area->vm_flags & VM_WRITE)) {
+           ret = VM_FAULT_SIGBUS;
+           goto out_rpm;
+   }
 
    ret = ttm_bo_vm_reserve(bo, vmf); >     if (ret)
-           return ret;
+           goto out_rpm;
 
    if (obj->mm.madv != I915_MADV_WILLNEED) {
            dma_resv_unlock(bo->base.resv);
-           return VM_FAULT_SIGBUS;
+           ret = VM_FAULT_SIGBUS;
+           goto out_rpm;
    }
 
    if (!i915_ttm_resource_mappable(bo->resource)) {
@@ -1062,7 +1084,8 @@ static vm_fault_t vm_fault_ttm(struct vm_fault

*vmf)

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault · line 1085

Quoted 2 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +0 -0
            if (err) {
                    drm_dbg(dev, "Unable to make resource CPU

accessible\n");

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault

Quoted 8 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +2 -1
                    dma_resv_unlock(bo->base.resv);
-                   return VM_FAULT_SIGBUS;
+                   ret = VM_FAULT_SIGBUS;
+                   goto out_rpm;
            }
    }
 
@@ -1073,12 +1096,25 @@ static vm_fault_t vm_fault_ttm(struct vm_fault

*vmf)

Quoted 7 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +3 -0
    } else {
            ret = ttm_bo_vm_dummy_page(vmf, vmf->vma-
 vm_page_prot);
    }
+
+   /* ttm_bo_vm_reserve() already has dma_resv_lock */
+   if (!ret && i915_gem_object_is_lmem(obj) && !obj->userfault_count++)
Quoted 7 lines
Quoted 5 lines

{

This might increment userfault_count more than once per mapping/object? Is that intentional? I would have thought that with fault_ttm the user_fault_count shouldn't ever be > 1 (there is no partial mmap here).

I did not get this part, the double addition for the object is possible only when ttm_fault will get called multiple times for an object ? In that case isn't userfault_count would be > 1 ?

Right, if it gets mapped and then unmapped multiple times, then userfault_count might be > 1. In i915_gem_runtime_suspend() we only remove it from the list once userfault_count
is zero. Just wondering if there is more going on there? Since with ttm_fault we can't partially mmap the object (unlike with GGTT mmap), so there should only be one mapping per object, AFAIK.

Quoted 42 lines
Quoted 22 lines

Also it look like ret == VM_FAULT_NOPAGE if all went well. Although it also returns that if the wait was interrupted, but I don't think that matters much.

Maybe we can do something like:

ret = ttm_bo_vm_reserve(bo..) /* locks the object for us */ ..

wakeref = 0;
if (i915_ttm_cpu_maps_iomem(bo->resource))

    wakeref = intel_runtime_pm_get();

if (drm_dev_enter(dev, &idx)) {
} else {
}

if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT))
goto out_rpm;

Quoted 8 lines
if (ret == VM_FAULT_NOPAGE && wakeref && !obj->userfault_count) {
   obj->userfault_count++; /* protected by wakeref */
   mutex_lock()
   list_add();
   mutex_unlock();
}

?

Thanks for suggestion, I will implement it.

Quoted 11 lines

AFAICT we are then still missing something for object destruction, to ensure we remove the object from the list. Otherwise we can destroy the object, and then at some later point i915_gem_runtime_suspend() runs and will uaf on the list. I think the trick with GTT mmap is that it will always grab the wakeref when unbinding the vma (like during object destruction), and then it can safely remove the object from the list. Perhaps we will need a similar trick anyway, since destroying the object will likely require unbinding it, which will mean touching the page-tables from the CPU, which are in lmem anyway. Do we maybe just always grab the wakeref for dgpu? What do you think?

Quoted 2 lines

With input from @rodrigo , we can't get permanent wakref as that will block the d3cold(it will burn the card power).

I meant just always grabbing during object destruction.

Quoted 2 lines

How about deleting the object from the list in __i915_gem_object_free_mmaps() to handle the object destroy case ?

Yeah, something like that. Also note that if there is a bound vma during object destruction, we will need to grab the runtime pm anyway, since that will touch lmem from the CPU with the page-tables.

Quoted 4 lines

If this approach is not scalable then probably we need to think over to get the wakref on dGPU whenever any client is connected.
I was also thinking in direct to limit the mmap mapping for lmem object by using a new mmap fake offset to limit the rpm suspend latency due to revoking and re-creation of mmap ?

Hmm, not sure I follow with the idea of using a new fake offset. Can you share more info?

Quoted 91 lines
Quoted 9 lines

Also on second thought it looks like we can't directly call i915_gem_object_release_mmap_offset() from i915_gem_runtime_suspend(), without then also holding the object lock for the ttm_mem_io_free(). But I think it should be OK to just directly call drm_vma_node_unmap(&bo->base.vma_node, ...) instead.

Also, do you know if we need something like
CONFIG_DRM_I915_USERFAULT_AUTOSUSPEND here?

It is to induced additional delay before actual runtime pm triggers. Yeah it make sense to use that , I missed earlier to incorporate this comment. Thanks,
Anshuman Gupta.

Quoted 78 lines

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault · line 1111

Quoted 10 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +5 -0
+           mutex_lock(&to_gt(to_i915(obj->base.dev))-
 lmem_userfault_lock);
+           list_add(&obj->userfault_link, &to_gt(to_i915(obj->base.dev))-
 lmem_userfault_list);
+           mutex_unlock(&to_gt(to_i915(obj->base.dev))-
 lmem_userfault_lock);
+   }
+
    if (ret == VM_FAULT_RETRY && !(vmf->flags &

FAULT_FLAG_RETRY_NOWAIT))

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault *vmf)

Quoted 10 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +5 -1
-           return ret;
+           goto out_rpm;
 
    i915_ttm_adjust_lru(obj);
 
    dma_resv_unlock(bo->base.resv);
+
+out_rpm:
+   if (wakeref)
+           intel_runtime_pm_put(&to_i915(obj->base.dev)->runtime_pm,

wakeref);

Review

Quoted 5 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +1 -0
+
    return ret;
  }
(quoted diff) +0 -0

b/drivers/gpu/drm/i915/gt/intel_gt.c

Review

Quoted 13 lines

index e4bac2431e41..63616ea198ca 100644

drivers/gpu/drm/i915/gt/intel_gt.c +2 -0
@@ -39,6 +39,8 @@ static void __intel_gt_init_early(struct intel_gt *gt)
  {
    spin_lock_init(&gt->irq_lock);
 
+   INIT_LIST_HEAD(&gt->lmem_userfault_list);
+   mutex_init(&gt->lmem_userfault_lock);
    INIT_LIST_HEAD(&gt->closed_vma);
    spin_lock_init(&gt->closed_lock);
 
(quoted diff) +0 -0

b/drivers/gpu/drm/i915/gt/intel_gt_types.h

Review

Quoted 14 lines

index 4d56f7d5a3be..330e7531cf22 100644

drivers/gpu/drm/i915/gt/intel_gt_types.h +3 -0
@@ -132,6 +132,9 @@ struct intel_gt {
    struct intel_wakeref wakeref;
    atomic_t user_wakeref;
 
+   struct mutex lmem_userfault_lock; /* Protects access to usefault list */
+   struct list_head lmem_userfault_list;
+
    struct list_head closed_vma;
    spinlock_t closed_lock; /* guards the list of closed_vma */
 
(quoted diff) +0 -0

b/drivers/gpu/drm/i915/i915_gem.c

drivers/gpu/drm/i915/i915_gem.c void i915_gem_runtime_suspend(struct

Quoted 4 lines

index 702e5b89be22..288dc7e231dc 100644

drivers/gpu/drm/i915/i915_gem.c +0 -0
@@ -842,6 +842,14 @@ void i915_gem_runtime_suspend(struct

drm_i915_private *i915)

drivers/gpu/drm/i915/i915_gem.c void i915_gem_runtime_suspend(struct · line 842

Quoted 1 line
drivers/gpu/drm/i915/i915_gem.c +0 -0
                             &to_gt(i915)->ggtt->userfault_list,

userfault_link)

drivers/gpu/drm/i915/i915_gem.c void i915_gem_runtime_suspend(struct drm_i915_private *i915)

Quoted 4 lines
drivers/gpu/drm/i915/i915_gem.c +2 -0
            __i915_gem_object_release_mmap_gtt(obj);
 
+   list_for_each_entry_safe(obj, on,
+                            &to_gt(i915)->lmem_userfault_list,

userfault_link) {

Quoted 9 lines
drivers/gpu/drm/i915/i915_gem.c +6 -0
+           i915_gem_object_release_mmap_offset(obj);
+
+           if (!--obj->userfault_count)
+                   list_del(&obj->userfault_link);
+   }
+
    /*
     * The fence will be lost when the device powers down. If any were
     * in use by hardware (i.e. they are pinned), we should not be powering
#5 lore raw

Re: [Intel-gfx] [PATCH 1/1] drm/i915/dgfx: Release mmap on rpm suspend

Quoted 167 lines · Matthew Auld
Quoted 103 lines · Anshuman
Quoted 5 lines

-----Original Message-----
From: Auld, Matthew <[email protected]>

Thanks Matt for reviewing this.

Quoted 23 lines

Sent: Friday, August 26, 2022 11:09 PM
To: Gupta, Anshuman <[email protected]>; intel- [email protected]
Cc: [email protected]; Vivi, Rodrigo <[email protected]>; Nilawar, Badal <[email protected]>; [email protected]
Subject: Re: [PATCH 1/1] drm/i915/dgfx: Release mmap on rpm suspend

Quoted 15 lines · Anshuman Gupta

Release all mmap mapping for all lmem objects which are associated with userfault such that, while pcie function in D3hot, any access to memory mappings will raise a userfault.

Runtime resume the dgpu(when gem object lies in lmem). This will transition the dgpu graphics function to D0 state if it was in D3 in order to access the mmap memory mappings.

v2:
- Squashes the patches. [Matt Auld]
- Add adequate locking for lmem_userfault_list addition. [Matt Auld]
- Reused obj->userfault_count to avoid double addition. [Matt Auld]
- Added i915_gem_object_lock to check i915_gem_object_is_lmem. [Matt Auld]

Quoted 4 lines
Quoted 1 line

Just double checking, this is needed for DG1 and DG2, right? Are there any BSpec links we can add here?

This is specific to all discrete products with respect to PCIe Spec Section 5.3.1.4.1 I will add the PCIe specs link here.

Quoted 68 lines

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static void i915_ttm_delayed_free(struct

Quoted 33 lines
Cc Matthew Auld <[email protected]>
Cc Rodrigo Vivi <[email protected]>
---
  .../gpu/drm/i915/gem/i915_gem_object_types.h  |  3 +-
  drivers/gpu/drm/i915/gem/i915_gem_ttm.c       | 48 ++++++++++++++++---
  drivers/gpu/drm/i915/gt/intel_gt.c            |  2 +
  drivers/gpu/drm/i915/gt/intel_gt_types.h      |  3 ++
  drivers/gpu/drm/i915/i915_gem.c               |  8 ++++
  5 files changed, 57 insertions(+), 7 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_object_types.h +2 -1
index 9f6b14ec189a..40305e2bcd49 100644
@@ -298,7 +298,8 @@ struct drm_i915_gem_object {
    };
 
    /**
-    * Whether the object is currently in the GGTT mmap.
+    * Whether the object is currently in the GGTT or any other supported
+    * fake offset mmap backed by lmem.
     */
    unsigned int userfault_count;
    struct list_head userfault_link;
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +0 -0
index 5a5cf332d8a5..6532a634bd20 100644
@@ -1014,12 +1014,29 @@ static void i915_ttm_delayed_free(struct

drm_i915_gem_object *obj)

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault

Quoted 30 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +17 -0
    ttm_bo_put(i915_gem_to_ttm(obj));
  }
 
+static intel_wakeref_t
+i915_gem_ttm_get_lmem_obj_wakeref(struct drm_i915_gem_object *obj) {
+   intel_wakeref_t wakeref = 0;
+
+   if (i915_gem_object_lock_interruptible(obj, NULL))
+           return 0;
+
+   if (i915_gem_object_is_lmem(obj))
+           wakeref =
+intel_runtime_pm_get(&to_i915(obj->base.dev)->runtime_pm);
+
+   i915_gem_object_unlock(obj);
+
+   return wakeref;
+}
+
  static vm_fault_t vm_fault_ttm(struct vm_fault *vmf)
  {
    struct vm_area_struct *area = vmf->vma;
    struct ttm_buffer_object *bo = area->vm_private_data;
    struct drm_device *dev = bo->base.dev;
    struct drm_i915_gem_object *obj;
+   intel_wakeref_t wakeref = 0;
    vm_fault_t ret;
    int idx;
 
@@ -1027,18 +1044,23 @@ static vm_fault_t vm_fault_ttm(struct vm_fault

*vmf)

Quoted 3 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +0 -0
    if (!obj)
            return VM_FAULT_SIGBUS;
 
Quoted 7 lines
Quoted 6 lines

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault *vmf) · line 1047

Quoted 1 line
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +1 -0
+   wakeref = i915_gem_ttm_get_lmem_obj_wakeref(obj);

We shouldn't drop the lock here (also failing to acquire the lock should be fatal), since the object can in thoery transition to/from lmem inbetween dropping the object lock here and re-acquiring it again below, which means we might skip grabbing the wakeref here, but then later touch the list, if say it moves to lmem.

By not releasing the lock there was a deadlock from ttm_bo_vm_reserve().

Right, so we should probably move the check under vm_reserve.

Quoted 44 lines
Quoted 44 lines

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault

Quoted 24 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +8 -4
+
    /* Sanity check that we allow writing into this object */
    if (unlikely(i915_gem_object_is_readonly(obj) &&
-                area->vm_flags & VM_WRITE))
-           return VM_FAULT_SIGBUS;
+                area->vm_flags & VM_WRITE)) {
+           ret = VM_FAULT_SIGBUS;
+           goto out_rpm;
+   }
 
    ret = ttm_bo_vm_reserve(bo, vmf); >     if (ret)
-           return ret;
+           goto out_rpm;
 
    if (obj->mm.madv != I915_MADV_WILLNEED) {
            dma_resv_unlock(bo->base.resv);
-           return VM_FAULT_SIGBUS;
+           ret = VM_FAULT_SIGBUS;
+           goto out_rpm;
    }
 
    if (!i915_ttm_resource_mappable(bo->resource)) {
@@ -1062,7 +1084,8 @@ static vm_fault_t vm_fault_ttm(struct vm_fault

*vmf)

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault · line 1085

Quoted 2 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +0 -0
            if (err) {
                    drm_dbg(dev, "Unable to make resource CPU

accessible\n");

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault

Quoted 8 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +2 -1
                    dma_resv_unlock(bo->base.resv);
-                   return VM_FAULT_SIGBUS;
+                   ret = VM_FAULT_SIGBUS;
+                   goto out_rpm;
            }
    }
 
@@ -1073,12 +1096,25 @@ static vm_fault_t vm_fault_ttm(struct vm_fault

*vmf)

Quoted 7 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +3 -0
    } else {
            ret = ttm_bo_vm_dummy_page(vmf, vmf->vma-
 vm_page_prot);
    }
+
+   /* ttm_bo_vm_reserve() already has dma_resv_lock */
+   if (!ret && i915_gem_object_is_lmem(obj) && !obj->userfault_count++)
Quoted 7 lines
Quoted 5 lines

{

This might increment userfault_count more than once per mapping/object? Is that intentional? I would have thought that with fault_ttm the user_fault_count shouldn't ever be > 1 (there is no partial mmap here).

I did not get this part, the double addition for the object is possible only when ttm_fault will get called multiple times for an object ? In that case isn't userfault_count would be > 1 ?

Quoted 6 lines

Right, if it gets mapped and then unmapped multiple times, then userfault_count might be > 1. In i915_gem_runtime_suspend() we only remove it from the list once userfault_count
is zero. Just wondering if there is more going on there? Since with ttm_fault we can't partially mmap the object (unlike with GGTT mmap), so there should only be one mapping per object, AFAIK.

Oh, that's actually not true it seems. It looks like it might only fault some of the pages, as per TTM_BO_VM_NUM_PREFAULT. Although still not sure why we need userfault_count for lmem fault? Or maybe just always set it to one or zero?

Quoted 154 lines
Quoted 43 lines
Quoted 23 lines

Also it look like ret == VM_FAULT_NOPAGE if all went well. Although it also returns that if the wait was interrupted, but I don't think that matters much.

Maybe we can do something like:

ret = ttm_bo_vm_reserve(bo..) /* locks the object for us */ ..

wakeref = 0;
if (i915_ttm_cpu_maps_iomem(bo->resource))

    wakeref = intel_runtime_pm_get();

if (drm_dev_enter(dev, &idx)) {
} else {
}

if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT))
goto out_rpm;

Quoted 8 lines
if (ret == VM_FAULT_NOPAGE && wakeref && !obj->userfault_count) {
   obj->userfault_count++; /* protected by wakeref */
   mutex_lock()
   list_add();
   mutex_unlock();
}

?

Thanks for suggestion, I will implement it.

Quoted 11 lines

AFAICT we are then still missing something for object destruction, to ensure we remove the object from the list. Otherwise we can destroy the object, and then at some later point i915_gem_runtime_suspend() runs and will uaf on the list. I think the trick with GTT mmap is that it will always grab the wakeref when unbinding the vma (like during object destruction), and then it can safely remove the object from the list. Perhaps we will need a similar trick anyway, since destroying the object will likely require unbinding it, which will mean touching the page-tables from the CPU, which are in lmem anyway. Do we maybe just always grab the wakeref for dgpu? What do you think?

Quoted 2 lines

With input from @rodrigo , we can't get permanent wakref as that will block the d3cold(it will burn the card power).

I meant just always grabbing during object destruction.

Quoted 2 lines

How about deleting the object from the list in __i915_gem_object_free_mmaps() to handle the object destroy case ?

Yeah, something like that. Also note that if there is a bound vma during object destruction, we will need to grab the runtime pm anyway, since that will touch lmem from the CPU with the page-tables.

Quoted 4 lines

If this approach is not scalable then probably we need to think over to get the wakref on dGPU whenever any client is connected.
I was also thinking in direct to limit the mmap mapping for lmem object by using a new mmap fake offset to limit the rpm suspend latency due to revoking and re-creation of mmap ?

Hmm, not sure I follow with the idea of using a new fake offset. Can you share more info?

Quoted 91 lines
Quoted 9 lines

Also on second thought it looks like we can't directly call i915_gem_object_release_mmap_offset() from i915_gem_runtime_suspend(), without then also holding the object lock for the ttm_mem_io_free(). But I think it should be OK to just directly call drm_vma_node_unmap(&bo->base.vma_node, ...) instead.

Also, do you know if we need something like
CONFIG_DRM_I915_USERFAULT_AUTOSUSPEND here?

It is to induced additional delay before actual runtime pm triggers. Yeah it make sense to use that , I missed earlier to incorporate this comment. Thanks,
Anshuman Gupta.

Quoted 78 lines

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault · line 1111

Quoted 10 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +5 -0
+           mutex_lock(&to_gt(to_i915(obj->base.dev))-
 lmem_userfault_lock);
+           list_add(&obj->userfault_link, &to_gt(to_i915(obj->base.dev))-
 lmem_userfault_list);
+           mutex_unlock(&to_gt(to_i915(obj->base.dev))-
 lmem_userfault_lock);
+   }
+
    if (ret == VM_FAULT_RETRY && !(vmf->flags &

FAULT_FLAG_RETRY_NOWAIT))

drivers/gpu/drm/i915/gem/i915_gem_ttm.c static vm_fault_t vm_fault_ttm(struct vm_fault *vmf)

Quoted 10 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +5 -1
-           return ret;
+           goto out_rpm;
 
    i915_ttm_adjust_lru(obj);
 
    dma_resv_unlock(bo->base.resv);
+
+out_rpm:
+   if (wakeref)
+           intel_runtime_pm_put(&to_i915(obj->base.dev)->runtime_pm,

wakeref);

Review

Quoted 5 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +1 -0
+
    return ret;
  }
(quoted diff) +0 -0

b/drivers/gpu/drm/i915/gt/intel_gt.c

Review

Quoted 13 lines

index e4bac2431e41..63616ea198ca 100644

drivers/gpu/drm/i915/gt/intel_gt.c +2 -0
@@ -39,6 +39,8 @@ static void __intel_gt_init_early(struct intel_gt *gt)
  {
    spin_lock_init(&gt->irq_lock);
 
+   INIT_LIST_HEAD(&gt->lmem_userfault_list);
+   mutex_init(&gt->lmem_userfault_lock);
    INIT_LIST_HEAD(&gt->closed_vma);
    spin_lock_init(&gt->closed_lock);
 
(quoted diff) +0 -0

b/drivers/gpu/drm/i915/gt/intel_gt_types.h

Review

Quoted 14 lines

index 4d56f7d5a3be..330e7531cf22 100644

drivers/gpu/drm/i915/gt/intel_gt_types.h +3 -0
@@ -132,6 +132,9 @@ struct intel_gt {
    struct intel_wakeref wakeref;
    atomic_t user_wakeref;
 
+   struct mutex lmem_userfault_lock; /* Protects access to usefault list */
+   struct list_head lmem_userfault_list;
+
    struct list_head closed_vma;
    spinlock_t closed_lock; /* guards the list of closed_vma */
 
(quoted diff) +0 -0

b/drivers/gpu/drm/i915/i915_gem.c

drivers/gpu/drm/i915/i915_gem.c void i915_gem_runtime_suspend(struct

Quoted 4 lines

index 702e5b89be22..288dc7e231dc 100644

drivers/gpu/drm/i915/i915_gem.c +0 -0
@@ -842,6 +842,14 @@ void i915_gem_runtime_suspend(struct

drm_i915_private *i915)

drivers/gpu/drm/i915/i915_gem.c void i915_gem_runtime_suspend(struct · line 842

Quoted 1 line
drivers/gpu/drm/i915/i915_gem.c +0 -0
                             &to_gt(i915)->ggtt->userfault_list,

userfault_link)

drivers/gpu/drm/i915/i915_gem.c void i915_gem_runtime_suspend(struct drm_i915_private *i915)

Quoted 4 lines
drivers/gpu/drm/i915/i915_gem.c +2 -0
            __i915_gem_object_release_mmap_gtt(obj);
 
+   list_for_each_entry_safe(obj, on,
+                            &to_gt(i915)->lmem_userfault_list,

userfault_link) {

Quoted 9 lines
drivers/gpu/drm/i915/i915_gem.c +6 -0
+           i915_gem_object_release_mmap_offset(obj);
+
+           if (!--obj->userfault_count)
+                   list_del(&obj->userfault_link);
+   }
+
    /*
     * The fence will be lost when the device powers down. If any were
     * in use by hardware (i.e. they are pinned), we should not be powering
#7 lore raw

[Intel-gfx] ✓ Fi.CI.BAT: success for DGFX mmap with rpm (rev2)

In reply to Anshuman Gupta
== Series Details ==

== Series Details ==

Series: DGFX mmap with rpm (rev2)
URL : https://patchwork.freedesktop.org/series/107400/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_12024 -> Patchwork_107400v2
====================================================

Summary
-------

**SUCCESS**

No regressions found.

External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/index.html

Participating hosts (35 -> 35)
------------------------------

No changes in participating hosts

Possible new issues
-------------------

Here are the unknown changes that may have been introduced in Patchwork_107400v2:

### IGT changes ###

#### Suppressed ####

The following results come from untrusted machines, tests, or statuses. They do not affect the overall result.

* igt@i915_selftest@live@gt_heartbeat:
- {bat-dg2-10}: [PASS][1] -> [DMESG-FAIL][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/bat-dg2-10/igt@i915_selftest@live@gt_heartbeat.html
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/bat-dg2-10/igt@i915_selftest@live@gt_heartbeat.html

Known issues
------------

Here are the changes found in Patchwork_107400v2 that come from known issues:

### IGT changes ###

#### Issues hit ####

* igt@i915_selftest@live@hangcheck:
- fi-hsw-4770: [PASS][3] -> [INCOMPLETE][4] ([i915#4785])
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/fi-hsw-4770/igt@i915_selftest@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/fi-hsw-4770/igt@i915_selftest@[email protected]

* igt@kms_cursor_legacy@basic-busy-flip-before-cursor@atomic-transitions:
- fi-bsw-kefka: [PASS][5] -> [FAIL][6] ([i915#6298])
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/fi-bsw-kefka/igt@kms_cursor_legacy@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/fi-bsw-kefka/igt@kms_cursor_legacy@[email protected]

* igt@runner@aborted:
- fi-hsw-4770: NOTRUN -> [FAIL][7] ([fdo#109271] / [i915#4312] / [i915#5594] / [i915#6246])
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/fi-hsw-4770/igt@[email protected]

{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).

[fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
[i915#2867]: https://gitlab.freedesktop.org/drm/intel/issues/2867
[i915#4312]: https://gitlab.freedesktop.org/drm/intel/issues/4312
[i915#4785]: https://gitlab.freedesktop.org/drm/intel/issues/4785
[i915#5594]: https://gitlab.freedesktop.org/drm/intel/issues/5594
[i915#6246]: https://gitlab.freedesktop.org/drm/intel/issues/6246
[i915#6257]: https://gitlab.freedesktop.org/drm/intel/issues/6257
[i915#6298]: https://gitlab.freedesktop.org/drm/intel/issues/6298
[i915#6380]: https://gitlab.freedesktop.org/drm/intel/issues/6380

Build changes
-------------

* Linux: CI_DRM_12024 -> Patchwork_107400v2

CI-20190529: 20190529
CI_DRM_12024: 656b7e74f416705f11953d30cda518a98f18ba2e @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_6636: 1298b5f0e1b3e010657ffba41d2e775fab028e08 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
Patchwork_107400v2: 656b7e74f416705f11953d30cda518a98f18ba2e @ git://anongit.freedesktop.org/gfx-ci/linux

### Linux commits

0da1a5ee767f drm/i915/dgfx: Release mmap on rpm suspend

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/index.html

#8 lore raw

[Intel-gfx] ✓ Fi.CI.IGT: success for DGFX mmap with rpm (rev2)

In reply to Anshuman Gupta
== Series Details ==

== Series Details ==

Series: DGFX mmap with rpm (rev2)
URL : https://patchwork.freedesktop.org/series/107400/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_12024_full -> Patchwork_107400v2_full
====================================================

Summary
-------

**SUCCESS**

No regressions found.

Participating hosts (11 -> 11)
------------------------------

No changes in participating hosts

Known issues
------------

Here are the changes found in Patchwork_107400v2_full that come from known issues:

### IGT changes ###

#### Issues hit ####

* igt@gem_exec_balancer@parallel-bb-first:
- shard-iclb: [PASS][1] -> [SKIP][2] ([i915#4525]) +2 similar issues
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-iclb2/igt@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-iclb8/igt@[email protected]

* igt@gem_exec_fair@basic-none-vip@rcs0:
- shard-tglb: [PASS][3] -> [FAIL][4] ([i915#2842]) +1 similar issue
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-tglb6/igt@gem_exec_fair@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-tglb7/igt@gem_exec_fair@[email protected]

* igt@gem_exec_fair@basic-none@vcs1:
- shard-iclb: NOTRUN -> [FAIL][5] ([i915#2842])
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-iclb1/igt@gem_exec_fair@[email protected]

* igt@gem_exec_fair@basic-pace@rcs0:
- shard-kbl: [PASS][6] -> [FAIL][7] ([i915#2842]) +2 similar issues
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-kbl4/igt@gem_exec_fair@[email protected]
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-kbl7/igt@gem_exec_fair@[email protected]
- shard-tglb: [PASS][8] -> [FAIL][9] ([i915#2876])
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-tglb6/igt@gem_exec_fair@[email protected]
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-tglb3/igt@gem_exec_fair@[email protected]

* igt@gem_lmem_swapping@verify-random-ccs:
- shard-kbl: NOTRUN -> [SKIP][10] ([fdo#109271] / [i915#4613])
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-kbl1/igt@[email protected]

* igt@gem_userptr_blits@input-checking:
- shard-glk: NOTRUN -> [DMESG-WARN][11] ([i915#4991])
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-glk7/igt@[email protected]

* igt@i915_pm_rpm@dpms-lpsp:
- shard-kbl: NOTRUN -> [SKIP][12] ([fdo#109271]) +27 similar issues
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-kbl7/igt@[email protected]

* igt@kms_big_fb@x-tiled-64bpp-rotate-0:
- shard-snb: [PASS][13] -> [SKIP][14] ([fdo#109271])
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-snb2/igt@[email protected]
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-snb2/igt@[email protected]

* igt@kms_ccs@pipe-a-bad-aux-stride-y_tiled_gen12_rc_ccs_cc:
- shard-apl: NOTRUN -> [SKIP][15] ([fdo#109271] / [i915#3886])
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-apl8/igt@kms_ccs@pipe-a-bad-aux-stride-y_tiled_gen12_rc_ccs_cc.html

* igt@kms_ccs@pipe-a-crc-primary-rotation-180-y_tiled_gen12_rc_ccs_cc:
- shard-kbl: NOTRUN -> [SKIP][16] ([fdo#109271] / [i915#3886])
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-kbl7/igt@kms_ccs@pipe-a-crc-primary-rotation-180-y_tiled_gen12_rc_ccs_cc.html

* igt@kms_ccs@pipe-a-missing-ccs-buffer-y_tiled_gen12_mc_ccs:
- shard-glk: NOTRUN -> [SKIP][17] ([fdo#109271] / [i915#3886]) +4 similar issues
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-glk7/igt@kms_ccs@pipe-a-missing-ccs-buffer-y_tiled_gen12_mc_ccs.html

* igt@kms_chamelium@dp-edid-change-during-suspend:
- shard-glk: NOTRUN -> [SKIP][18] ([fdo#109271] / [fdo#111827]) +4 similar issues
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-glk7/igt@[email protected]

* igt@kms_chamelium@hdmi-mode-timings:
- shard-kbl: NOTRUN -> [SKIP][19] ([fdo#109271] / [fdo#111827]) +1 similar issue
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-kbl7/igt@[email protected]

* igt@kms_flip@2x-plain-flip-fb-recreate-interruptible@ab-hdmi-a1-hdmi-a2:
- shard-glk: [PASS][20] -> [FAIL][21] ([i915#2122])
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-glk6/igt@kms_flip@[email protected]
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-glk2/igt@kms_flip@[email protected]

* igt@kms_flip@flip-vs-suspend-interruptible@a-dp1:
- shard-kbl: [PASS][22] -> [DMESG-WARN][23] ([i915#180]) +1 similar issue
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-kbl1/igt@kms_flip@[email protected]
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-kbl1/igt@kms_flip@[email protected]

* igt@kms_flip@flip-vs-suspend@a-edp1:
- shard-iclb: NOTRUN -> [INCOMPLETE][24] ([i915#6598])
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-iclb5/igt@kms_flip@[email protected]

* igt@kms_flip_scaled_crc@flip-64bpp-yftile-to-32bpp-yftile-downscaling@pipe-a-valid-mode:
- shard-iclb: NOTRUN -> [SKIP][25] ([i915#2672]) +7 similar issues
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-iclb8/igt@kms_flip_scaled_crc@flip-64bpp-yftile-to-32bpp-yftile-downscaling@pipe-a-valid-mode.html

* igt@kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling@pipe-a-default-mode:
- shard-iclb: NOTRUN -> [SKIP][26] ([i915#3555])
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-iclb2/igt@kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling@pipe-a-default-mode.html

* igt@kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilegen12rcccs-upscaling@pipe-a-valid-mode:
- shard-iclb: NOTRUN -> [SKIP][27] ([i915#2672] / [i915#3555])
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-iclb8/igt@kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilegen12rcccs-upscaling@pipe-a-valid-mode.html

* igt@kms_frontbuffer_tracking@fbc-2p-primscrn-spr-indfb-fullscreen:
- shard-apl: NOTRUN -> [SKIP][28] ([fdo#109271]) +13 similar issues
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-apl8/igt@kms_frontbuffer_tracking@fbc-2p-primscrn-spr-indfb-fullscreen.html

* igt@kms_plane@plane-panning-bottom-right-suspend@pipe-b-planes:
- shard-glk: NOTRUN -> [INCOMPLETE][29] ([i915#6598])
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-glk3/igt@kms_plane@[email protected]

* igt@kms_plane_alpha_blend@pipe-b-alpha-transparent-fb:
- shard-glk: NOTRUN -> [FAIL][30] ([i915#265])
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-glk7/igt@[email protected]

* igt@kms_plane_scaling@planes-downscale-factor-0-5@pipe-a-edp-1:
- shard-iclb: [PASS][31] -> [SKIP][32] ([i915#5235]) +2 similar issues
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-iclb3/igt@kms_plane_scaling@[email protected]
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-iclb2/igt@kms_plane_scaling@[email protected]

* igt@kms_plane_scaling@planes-upscale-factor-0-25-downscale-factor-0-5@pipe-c-hdmi-a-1:
- shard-glk: NOTRUN -> [SKIP][33] ([fdo#109271]) +57 similar issues
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-glk3/igt@kms_plane_scaling@planes-upscale-factor-0-25-downscale-factor-0-5@pipe-c-hdmi-a-1.html

* igt@kms_psr2_sf@overlay-plane-update-sf-dmg-area:
- shard-kbl: NOTRUN -> [SKIP][34] ([fdo#109271] / [i915#658])
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-kbl1/igt@[email protected]

* igt@kms_psr@psr2_cursor_blt:
- shard-iclb: [PASS][35] -> [SKIP][36] ([fdo#109441]) +3 similar issues
[35]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-iclb2/igt@kms_psr@psr2_cursor_blt.html
[36]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-iclb1/igt@kms_psr@psr2_cursor_blt.html

* igt@kms_writeback@writeback-fb-id:
- shard-apl: NOTRUN -> [SKIP][37] ([fdo#109271] / [i915#2437])
[37]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-apl8/igt@[email protected]

* igt@kms_writeback@writeback-invalid-parameters:
- shard-kbl: NOTRUN -> [SKIP][38] ([fdo#109271] / [i915#2437])
[38]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-kbl7/igt@[email protected]

* igt@runner@aborted:
- shard-glk: NOTRUN -> [FAIL][39] ([i915#3002] / [i915#4312] / [i915#5257] / [i915#6599])
[39]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-glk7/igt@[email protected]

* igt@sysfs_clients@create:
- shard-glk: NOTRUN -> [SKIP][40] ([fdo#109271] / [i915#2994])
[40]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-glk7/igt@[email protected]

#### Possible fixes ####

* igt@gem_busy@close-race:
- shard-iclb: [TIMEOUT][41] ([i915#6016]) -> [PASS][42]
[41]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-iclb6/igt@[email protected]
[42]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-iclb5/igt@[email protected]

* igt@gem_exec_balancer@parallel-keep-in-fence:
- shard-iclb: [SKIP][43] ([i915#4525]) -> [PASS][44] +2 similar issues
[43]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-iclb8/igt@[email protected]
[44]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-iclb2/igt@[email protected]

* igt@gem_exec_fair@basic-none-solo@rcs0:
- shard-apl: [FAIL][45] ([i915#2842]) -> [PASS][46]
[45]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-apl3/igt@gem_exec_fair@[email protected]
[46]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-apl3/igt@gem_exec_fair@[email protected]

* igt@gem_exec_fair@basic-pace-share@rcs0:
- shard-glk: [FAIL][47] ([i915#2842]) -> [PASS][48]
[47]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-glk7/igt@gem_exec_fair@[email protected]
[48]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-glk7/igt@gem_exec_fair@[email protected]

* igt@gem_exec_fair@basic-pace@vcs0:
- shard-kbl: [FAIL][49] ([i915#2842]) -> [PASS][50]
[49]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-kbl4/igt@gem_exec_fair@[email protected]
[50]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-kbl7/igt@gem_exec_fair@[email protected]

* igt@gen9_exec_parse@allowed-single:
- shard-apl: [DMESG-WARN][51] ([i915#5566] / [i915#716]) -> [PASS][52]
[51]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-apl7/igt@[email protected]
[52]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-apl8/igt@[email protected]

* igt@i915_pm_rc6_residency@rc6-idle@bcs0:
- shard-kbl: [FAIL][53] -> [PASS][54]
[53]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-kbl6/igt@i915_pm_rc6_residency@[email protected]
[54]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-kbl4/igt@i915_pm_rc6_residency@[email protected]

* igt@kms_big_fb@y-tiled-16bpp-rotate-0:
- shard-apl: [DMESG-WARN][55] ([i915#5904] / [i915#62]) -> [PASS][56] +11 similar issues
[55]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-apl6/igt@[email protected]
[56]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-apl1/igt@[email protected]

* igt@kms_cursor_crc@cursor-random@pipe-b-dp-1-64x21:
- shard-apl: [DMESG-WARN][57] ([i915#180] / [i915#5904] / [i915#62]) -> [PASS][58] +38 similar issues
[57]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-apl6/igt@kms_cursor_crc@[email protected]
[58]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-apl1/igt@kms_cursor_crc@[email protected]

* igt@kms_cursor_legacy@flip-vs-cursor@atomic-transitions-varying-size:
- shard-glk: [FAIL][59] ([i915#2346]) -> [PASS][60] +1 similar issue
[59]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-glk9/igt@kms_cursor_legacy@[email protected]
[60]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-glk8/igt@kms_cursor_legacy@[email protected]

* igt@kms_draw_crc@draw-method-rgb565-mmap-cpu-xtiled:
- shard-apl: [DMESG-FAIL][61] ([i915#62]) -> [PASS][62] +1 similar issue
[61]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-apl6/igt@[email protected]
[62]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-apl1/igt@[email protected]

* igt@kms_frontbuffer_tracking@fbc-shrfb-scaledprimary:
- shard-apl: [DMESG-WARN][63] ([i915#62]) -> [PASS][64] +12 similar issues
[63]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-apl6/igt@[email protected]
[64]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-apl1/igt@[email protected]

* igt@kms_frontbuffer_tracking@fbc-suspend:
- shard-kbl: [DMESG-WARN][65] ([i915#180]) -> [PASS][66] +1 similar issue
[65]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-kbl1/igt@[email protected]
[66]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-kbl1/igt@[email protected]

* igt@kms_hdr@bpc-switch-dpms@pipe-a-dp-1:
- shard-kbl: [FAIL][67] ([i915#1188]) -> [PASS][68] +1 similar issue
[67]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-kbl1/igt@kms_hdr@[email protected]
[68]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-kbl7/igt@kms_hdr@[email protected]

* igt@kms_plane_scaling@planes-upscale-factor-0-25-downscale-factor-0-5@pipe-a-edp-1:
- shard-iclb: [SKIP][69] ([i915#5235]) -> [PASS][70] +2 similar issues
[69]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-iclb2/igt@kms_plane_scaling@planes-upscale-factor-0-25-downscale-factor-0-5@pipe-a-edp-1.html
[70]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-iclb8/igt@kms_plane_scaling@planes-upscale-factor-0-25-downscale-factor-0-5@pipe-a-edp-1.html

* igt@kms_psr@psr2_sprite_blt:
- shard-iclb: [SKIP][71] ([fdo#109441]) -> [PASS][72] +1 similar issue
[71]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-iclb3/igt@kms_psr@psr2_sprite_blt.html
[72]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-iclb2/igt@kms_psr@psr2_sprite_blt.html

#### Warnings ####

* igt@gem_eio@unwedge-stress:
- shard-tglb: [TIMEOUT][73] ([i915#3063]) -> [FAIL][74] ([i915#5784])
[73]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-tglb6/igt@[email protected]
[74]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-tglb3/igt@[email protected]

* igt@kms_big_fb@y-tiled-8bpp-rotate-0:
- shard-glk: [FAIL][75] ([i915#5138]) -> [FAIL][76] ([i915#1888] / [i915#5138])
[75]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-glk2/igt@[email protected]
[76]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-glk3/igt@[email protected]

* igt@kms_content_protection@uevent:
- shard-apl: [SKIP][77] ([fdo#109271]) -> [FAIL][78] ([i915#2105])
[77]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-apl6/igt@[email protected]
[78]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-apl1/igt@[email protected]

* igt@kms_psr2_sf@cursor-plane-move-continuous-exceed-fully-sf:
- shard-iclb: [SKIP][79] ([i915#658]) -> [SKIP][80] ([i915#2920])
[79]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-iclb3/igt@[email protected]
[80]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-iclb2/igt@[email protected]

* igt@kms_psr2_sf@overlay-plane-move-continuous-exceed-sf:
- shard-iclb: [SKIP][81] ([i915#2920]) -> [SKIP][82] ([i915#658])
[81]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-iclb2/igt@[email protected]
[82]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-iclb1/igt@[email protected]

* igt@kms_psr2_sf@overlay-primary-update-sf-dmg-area:
- shard-iclb: [SKIP][83] ([i915#2920]) -> [SKIP][84] ([fdo#111068] / [i915#658])
[83]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-iclb2/igt@[email protected]
[84]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-iclb8/igt@[email protected]

* igt@runner@aborted:
- shard-apl: ([FAIL][85], [FAIL][86]) ([fdo#109271] / [i915#3002] / [i915#4312] / [i915#5257] / [i915#6599]) -> [FAIL][87] ([i915#3002] / [i915#4312] / [i915#5257] / [i915#6599])
[85]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-apl2/igt@[email protected]
[86]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-apl7/igt@[email protected]
[87]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-apl6/igt@[email protected]
- shard-kbl: ([FAIL][88], [FAIL][89], [FAIL][90], [FAIL][91], [FAIL][92]) ([fdo#109271] / [i915#180] / [i915#3002] / [i915#4312] / [i915#5257]) -> ([FAIL][93], [FAIL][94], [FAIL][95], [FAIL][96]) ([i915#180] / [i915#3002] / [i915#4312] / [i915#5257])
[88]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-kbl1/igt@[email protected]
[89]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-kbl1/igt@[email protected]
[90]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-kbl1/igt@[email protected]
[91]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-kbl1/igt@[email protected]
[92]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12024/shard-kbl1/igt@[email protected]
[93]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-kbl4/igt@[email protected]
[94]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-kbl1/igt@[email protected]
[95]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-kbl1/igt@[email protected]
[96]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/shard-kbl4/igt@[email protected]

{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).

[fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
[fdo#109274]: https://bugs.freedesktop.org/show_bug.cgi?id=109274
[fdo#109280]: https://bugs.freedesktop.org/show_bug.cgi?id=109280
[fdo#109289]: https://bugs.freedesktop.org/show_bug.cgi?id=109289
[fdo#109291]: https://bugs.freedesktop.org/show_bug.cgi?id=109291
[fdo#109441]: https://bugs.freedesktop.org/show_bug.cgi?id=109441
[fdo#110189]: https://bugs.freedesktop.org/show_bug.cgi?id=110189
[fdo#111068]: https://bugs.freedesktop.org/show_bug.cgi?id=111068
[fdo#111614]: https://bugs.freedesktop.org/show_bug.cgi?id=111614
[fdo#111615]: https://bugs.freedesktop.org/show_bug.cgi?id=111615
[fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
[i915#1063]: https://gitlab.freedesktop.org/drm/intel/issues/1063
[i915#1188]: https://gitlab.freedesktop.org/drm/intel/issues/1188
[i915#180]: https://gitlab.freedesktop.org/drm/intel/issues/180
[i915#1888]: https://gitlab.freedesktop.org/drm/intel/issues/1888
[i915#2105]: https://gitlab.freedesktop.org/drm/intel/issues/2105
[i915#2122]: https://gitlab.freedesktop.org/drm/intel/issues/2122
[i915#2346]: https://gitlab.freedesktop.org/drm/intel/issues/2346
[i915#2437]: https://gitlab.freedesktop.org/drm/intel/issues/2437
[i915#2530]: https://gitlab.freedesktop.org/drm/intel/issues/2530
[i915#265]: https://gitlab.freedesktop.org/drm/intel/issues/265
[i915#2672]: https://gitlab.freedesktop.org/drm/intel/issues/2672
[i915#2842]: https://gitlab.freedesktop.org/drm/intel/issues/2842
[i915#2876]: https://gitlab.freedesktop.org/drm/intel/issues/2876
[i915#2920]: https://gitlab.freedesktop.org/drm/intel/issues/2920
[i915#2994]: https://gitlab.freedesktop.org/drm/intel/issues/2994
[i915#3002]: https://gitlab.freedesktop.org/drm/intel/issues/3002
[i915#3063]: https://gitlab.freedesktop.org/drm/intel/issues/3063
[i915#3555]: https://gitlab.freedesktop.org/drm/intel/issues/3555
[i915#3637]: https://gitlab.freedesktop.org/drm/intel/issues/3637
[i915#3689]: https://gitlab.freedesktop.org/drm/intel/issues/3689
[i915#3886]: https://gitlab.freedesktop.org/drm/intel/issues/3886
[i915#3989]: https://gitlab.freedesktop.org/drm/intel/issues/3989
[i915#426]: https://gitlab.freedesktop.org/drm/intel/issues/426
[i915#4270]: https://gitlab.freedesktop.org/drm/intel/issues/4270
[i915#4312]: https://gitlab.freedesktop.org/drm/intel/issues/4312
[i915#4525]: https://gitlab.freedesktop.org/drm/intel/issues/4525
[i915#454]: https://gitlab.freedesktop.org/drm/intel/issues/454
[i915#4613]: https://gitlab.freedesktop.org/drm/intel/issues/4613
[i915#4991]: https://gitlab.freedesktop.org/drm/intel/issues/4991
[i915#5138]: https://gitlab.freedesktop.org/drm/intel/issues/5138
[i915#5176]: https://gitlab.freedesktop.org/drm/intel/issues/5176
[i915#5235]: https://gitlab.freedesktop.org/drm/intel/issues/5235
[i915#5257]: https://gitlab.freedesktop.org/drm/intel/issues/5257
[i915#5286]: https://gitlab.freedesktop.org/drm/intel/issues/5286
[i915#5289]: https://gitlab.freedesktop.org/drm/intel/issues/5289
[i915#5566]: https://gitlab.freedesktop.org/drm/intel/issues/5566
[i915#5784]: https://gitlab.freedesktop.org/drm/intel/issues/5784
[i915#5904]: https://gitlab.freedesktop.org/drm/intel/issues/5904
[i915#6016]: https://gitlab.freedesktop.org/drm/intel/issues/6016
[i915#6095]: https://gitlab.freedesktop.org/drm/intel/issues/6095
[i915#62]: https://gitlab.freedesktop.org/drm/intel/issues/62
[i915#6335]: https://gitlab.freedesktop.org/drm/intel/issues/6335
[i915#658]: https://gitlab.freedesktop.org/drm/intel/issues/658
[i915#6598]: https://gitlab.freedesktop.org/drm/intel/issues/6598
[i915#6599]: https://gitlab.freedesktop.org/drm/intel/issues/6599
[i915#716]: https://gitlab.freedesktop.org/drm/intel/issues/716

Build changes
-------------

* Linux: CI_DRM_12024 -> Patchwork_107400v2

CI-20190529: 20190529
CI_DRM_12024: 656b7e74f416705f11953d30cda518a98f18ba2e @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_6636: 1298b5f0e1b3e010657ffba41d2e775fab028e08 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
Patchwork_107400v2: 656b7e74f416705f11953d30cda518a98f18ba2e @ git://anongit.freedesktop.org/gfx-ci/linux
piglit_4509: fdc5a4ca11124ab8413c7988896eec4c97336694 @ git://anongit.freedesktop.org/piglit

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_107400v2/index.html