Skip to contentSkip to conversation

drm/i915: Move LMEM (VRAM) management over to TTM

PATCH Superseded Thomas Hellström 4 participants 32 messages May 21, 2021 – May 26, 2021

Subsystems (6)

v3 of 11 Superseded Latest is v11 →

lore
Share & tools
lore

Thomas Hellström

author reviewer
#0 lore raw

v3 cover drm/i915: Move LMEM (VRAM) management over to TTM

This is an initial patch series to move discrete memory management over to TTM. It will be followed up shortly with adding more functionality.

The buddy allocator is temporarily removed along with its selftests and It is replaced with the TTM range manager and some selftests are adjusted to account for introduced fragmentation. Work is ongoing to reintroduce the buddy allocator as a TTM resource manager.

A new memcpy ttm move is introduced that uses kmap_local() functionality rather than vmap(). Among other things stated in the patch commit message it helps us deal with page-pased LMEM memory. It is generic enough to replace the ttm memcpy move with some additional work if so desired. On x86 it also enables prefetching reads from write-combined memory.

Finally the old i915 gem object LMEM backend is replaced with a i915 gem object TTM backend and some additional i915 gem object ops are introduced to support the added functionality.
Currently it is used only to support management and eviction of the LMEM region, but work is underway to extend the support to system memory. In this way we use TTM the way it was originally intended, having the GPU binding taken care of by driver code.

Intention is to follow up with
- System memory support
- Pipelined accelerated moves / migration
- Re-added buddy allocator in the TTM framework

v2:
- Add patches to move pagefaulting over to TTM
- Break out TTM changes to separate patches
- Address various review comments as detailed in the affected patches

v3:
- Drop TTM pagefaulting patches for now due changing approach due to a NAK.
- Address feedback on TTM patches
- Move the new TTM memcpy functionality into TTM.
- Move fast WC memcpy to drm
- Various fixes all over the place as shown in patch commit messages.

Cc Christian König <[email protected]>

Thomas Hellström (12):
drm/i915: Untangle the vma pages_mutex
drm/i915: Don't free shared locks while shared
drm/i915: Fix i915_sg_page_sizes to record dma segments rather than
physical pages
drm/i915/ttm Initialize the ttm device and memory managers drm/i915/ttm: Embed a ttm buffer object in the i915 gem object drm/ttm: Add a generic TTM memcpy move for page-based iomem drm, drm/i915: Move the memcpy_from_wc functionality to core drm drm/ttm: Use drm_memcpy_from_wc_dbm for TTM bo moves drm/ttm: Document and optimize ttm_bo_pipeline_gutting() drm/ttm, drm/amdgpu: Allow the driver some control over swapping drm/i915/ttm: Introduce a TTM i915 gem object backend drm/i915/lmem: Verify checks for lmem residency

 drivers/gpu/drm/Makefile                      |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   4 +
 drivers/gpu/drm/drm_drv.c                     |   2 +
 .../drm/{i915/i915_memcpy.c => drm_memcpy.c}  |  63 +-
 drivers/gpu/drm/i915/Kconfig                  |   1 +
 drivers/gpu/drm/i915/Makefile                 |   4 +-
 drivers/gpu/drm/i915/display/intel_display.c  |   2 +-
 drivers/gpu/drm/i915/gem/i915_gem_create.c    |   9 +-
 drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    |   2 +-
 .../gpu/drm/i915/gem/i915_gem_execbuffer.c    |   4 +-
 drivers/gpu/drm/i915/gem/i915_gem_lmem.c      |  71 +-
 drivers/gpu/drm/i915/gem/i915_gem_lmem.h      |   5 -
 drivers/gpu/drm/i915/gem/i915_gem_object.c    | 154 +++-
 drivers/gpu/drm/i915/gem/i915_gem_object.h    |  13 +
 .../gpu/drm/i915/gem/i915_gem_object_types.h  |  49 +-
 drivers/gpu/drm/i915/gem/i915_gem_pages.c     |   3 +-
 drivers/gpu/drm/i915/gem/i915_gem_phys.c      |   2 +-
 drivers/gpu/drm/i915/gem/i915_gem_region.c    | 126 +--
 drivers/gpu/drm/i915/gem/i915_gem_region.h    |   4 -
 drivers/gpu/drm/i915/gem/i915_gem_shmem.c     |   4 +-
 drivers/gpu/drm/i915/gem/i915_gem_stolen.c    |  10 +-
 drivers/gpu/drm/i915/gem/i915_gem_stolen.h    |   9 +-
 drivers/gpu/drm/i915/gem/i915_gem_ttm.c       | 531 ++++++++++++
 drivers/gpu/drm/i915/gem/i915_gem_ttm.h       |  50 ++
 drivers/gpu/drm/i915/gem/i915_gem_userptr.c   |   2 +-
 drivers/gpu/drm/i915/gt/intel_ggtt.c          |  19 +-
 drivers/gpu/drm/i915/gt/intel_gt.c            |   2 -
 drivers/gpu/drm/i915/gt/intel_gtt.c           |  45 +-
 drivers/gpu/drm/i915/gt/intel_gtt.h           |  28 +-
 drivers/gpu/drm/i915/gt/intel_ppgtt.c         |   2 +-
 drivers/gpu/drm/i915/gt/intel_region_lmem.c   |  30 +-
 drivers/gpu/drm/i915/gt/selftest_reset.c      |   7 +-
 drivers/gpu/drm/i915/gt/uc/intel_guc_log.c    |  11 +-
 drivers/gpu/drm/i915/i915_buddy.c             | 435 ----------
 drivers/gpu/drm/i915/i915_buddy.h             | 131 ---
 drivers/gpu/drm/i915/i915_cmd_parser.c        |   4 +-
 drivers/gpu/drm/i915/i915_drv.c               |  15 +-
 drivers/gpu/drm/i915/i915_drv.h               |   7 +-
 drivers/gpu/drm/i915/i915_gem.c               |   6 +-
 drivers/gpu/drm/i915/i915_globals.c           |   1 -
 drivers/gpu/drm/i915/i915_globals.h           |   1 -
 drivers/gpu/drm/i915/i915_gpu_error.c         |   8 +-
 drivers/gpu/drm/i915/i915_memcpy.h            |  34 -
 drivers/gpu/drm/i915/i915_scatterlist.c       |  70 ++
 drivers/gpu/drm/i915/i915_scatterlist.h       |  20 +-
 drivers/gpu/drm/i915/i915_vma.c               |  29 +-
 drivers/gpu/drm/i915/intel_memory_region.c    | 181 ++--
 drivers/gpu/drm/i915/intel_memory_region.h    |  45 +-
 drivers/gpu/drm/i915/intel_region_ttm.c       | 332 ++++++++
 drivers/gpu/drm/i915/intel_region_ttm.h       |  38 +
 drivers/gpu/drm/i915/selftests/i915_buddy.c   | 789 ------------------
 .../drm/i915/selftests/i915_mock_selftests.h  |   1 -
 .../drm/i915/selftests/intel_memory_region.c  | 140 +---
 drivers/gpu/drm/i915/selftests/mock_region.c  |  52 +-
 drivers/gpu/drm/ttm/ttm_bo.c                  |  63 +-
 drivers/gpu/drm/ttm/ttm_bo_util.c             | 337 ++++----
 drivers/gpu/drm/ttm/ttm_module.c              |  35 +
 drivers/gpu/drm/ttm/ttm_resource.c            | 166 ++++
 drivers/gpu/drm/ttm/ttm_tt.c                  |  50 ++
 include/drm/drm_memcpy.h                      |  47 ++
 include/drm/ttm/ttm_bo_driver.h               |  28 +
 include/drm/ttm/ttm_caching.h                 |   2 +
 include/drm/ttm/ttm_kmap_iter.h               |  61 ++
 include/drm/ttm/ttm_resource.h                |  61 ++
 include/drm/ttm/ttm_tt.h                      |  26 +
 65 files changed, 2322 insertions(+), 2163 deletions(-)
 rename drivers/gpu/drm/{i915/i915_memcpy.c => drm_memcpy.c} (70%)
 create mode 100644 drivers/gpu/drm/i915/gem/i915_gem_ttm.c
 create mode 100644 drivers/gpu/drm/i915/gem/i915_gem_ttm.h
 delete mode 100644 drivers/gpu/drm/i915/i915_buddy.c
 delete mode 100644 drivers/gpu/drm/i915/i915_buddy.h
 delete mode 100644 drivers/gpu/drm/i915/i915_memcpy.h
 create mode 100644 drivers/gpu/drm/i915/intel_region_ttm.c
 create mode 100644 drivers/gpu/drm/i915/intel_region_ttm.h
 delete mode 100644 drivers/gpu/drm/i915/selftests/i915_buddy.c
 create mode 100644 include/drm/drm_memcpy.h
 create mode 100644 include/drm/ttm/ttm_kmap_iter.h
Signature · 1 line
2.31.1
author reviewer
#1 lore raw

PATCH v3 1/12 drm/i915: Untangle the vma pages_mutex

Any sleeping dma_resv lock taken while the vma pages_mutex is held will cause a lockdep splat.
Move the i915_gem_object_pin_pages() call out of the pages_mutex critical section.

---
 drivers/gpu/drm/i915/i915_vma.c | 29 +++++++++++++++++------------
 1 file changed, 17 insertions(+), 12 deletions(-)
drivers/gpu/drm/i915/i915_vma.c +17 -12
index a6cd0fa62847..f2b5912fc542 100644
@@ -800,32 +800,37 @@ static bool try_qad_pin(struct i915_vma *vma, unsigned int flags)
 static int vma_get_pages(struct i915_vma *vma)
 {
 	int err = 0;
+	bool pinned_pages = false;
 
 	if (atomic_add_unless(&vma->pages_count, 1, 0))
 		return 0;
 
+	if (vma->obj) {
+		err = i915_gem_object_pin_pages(vma->obj);
+		if (err)
+			return err;
+		pinned_pages = true;
+	}
+
 	/* Allocations ahoy! */
-	if (mutex_lock_interruptible(&vma->pages_mutex))
-		return -EINTR;
+	if (mutex_lock_interruptible(&vma->pages_mutex)) {
+		err = -EINTR;
+		goto unpin;
+	}
 
 	if (!atomic_read(&vma->pages_count)) {
-		if (vma->obj) {
-			err = i915_gem_object_pin_pages(vma->obj);
-			if (err)
-				goto unlock;
-		}
-
 		err = vma->ops->set_pages(vma);
-		if (err) {
-			if (vma->obj)
-				i915_gem_object_unpin_pages(vma->obj);
+		if (err)
 			goto unlock;
-		}
+		pinned_pages = false;
 	}
 	atomic_inc(&vma->pages_count);
 
 unlock:
 	mutex_unlock(&vma->pages_mutex);
+unpin:
+	if (pinned_pages)
+		__i915_gem_object_unpin_pages(vma->obj);
 
 	return err;
 }
Signature · 6 lines
2.31.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
author reviewer
#3 lore raw

PATCH v3 3/12 drm/i915: Fix i915_sg_page_sizes to record dma segments rather than physical pages

In reply to Thomas Hellström

All users of this function actually want the dma segment sizes, but that's not what's calculated. Fix that and rename the function to i915_sg_dma_sizes to reflect what's calculated.

---
 drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c  |  2 +-
 drivers/gpu/drm/i915/gem/i915_gem_phys.c    |  2 +-
 drivers/gpu/drm/i915/gem/i915_gem_userptr.c |  2 +-
 drivers/gpu/drm/i915/i915_scatterlist.h     | 16 ++++++++++++----
 4 files changed, 15 insertions(+), 7 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c +1 -1
index ccede73c6465..616c3a2f1baf 100644
@@ -209,7 +209,7 @@ static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
 	if (IS_ERR(pages))
 		return PTR_ERR(pages);
 
-	sg_page_sizes = i915_sg_page_sizes(pages->sgl);
+	sg_page_sizes = i915_sg_dma_sizes(pages->sgl);
 
 	__i915_gem_object_set_pages(obj, pages, sg_page_sizes);
 
drivers/gpu/drm/i915/gem/i915_gem_phys.c +1 -1
index 81dc2bf59bc3..36f373dc493c 100644
@@ -208,7 +208,7 @@ static int i915_gem_object_shmem_to_phys(struct drm_i915_gem_object *obj)
 
 err_xfer:
 	if (!IS_ERR_OR_NULL(pages)) {
-		unsigned int sg_page_sizes = i915_sg_page_sizes(pages->sgl);
+		unsigned int sg_page_sizes = i915_sg_dma_sizes(pages->sgl);
 
 		__i915_gem_object_set_pages(obj, pages, sg_page_sizes);
 	}
drivers/gpu/drm/i915/gem/i915_gem_userptr.c +1 -1
index a657b99ec760..602f0ed983ec 100644
@@ -173,7 +173,7 @@ static int i915_gem_userptr_get_pages(struct drm_i915_gem_object *obj)
 		goto err;
 	}
 
-	sg_page_sizes = i915_sg_page_sizes(st->sgl);
+	sg_page_sizes = i915_sg_dma_sizes(st->sgl);
 
 	__i915_gem_object_set_pages(obj, st, sg_page_sizes);
 
drivers/gpu/drm/i915/i915_scatterlist.h +12 -4
index 9cb26a224034..b96baad66a3a 100644
@@ -101,15 +101,23 @@ static inline struct scatterlist *__sg_next(struct scatterlist *sg)
 	     (((__iter).curr += PAGE_SIZE) >= (__iter).max) ?		\
 	     (__iter) = __sgt_iter(__sg_next((__iter).sgp), false), 0 : 0)
 
-static inline unsigned int i915_sg_page_sizes(struct scatterlist *sg)
+/**
+ * i915_sg_dma_sizes - Record the dma segment sizes of a scatterlist
+ * @sg: The scatterlist
+ *
+ * Return: An unsigned int with segment sizes logically or'ed together.
+ * A caller can use this information to determine what hardware page table
+ * entry sizes can be used to map the memory represented by the scatterlist.
+ */
+static inline unsigned int i915_sg_dma_sizes(struct scatterlist *sg)
 {
 	unsigned int page_sizes;
 
 	page_sizes = 0;
-	while (sg) {
+	while (sg && sg_dma_len(sg)) {
 		GEM_BUG_ON(sg->offset);
-		GEM_BUG_ON(!IS_ALIGNED(sg->length, PAGE_SIZE));
-		page_sizes |= sg->length;
+		GEM_BUG_ON(!IS_ALIGNED(sg_dma_len(sg), PAGE_SIZE));
+		page_sizes |= sg_dma_len(sg);
 		sg = __sg_next(sg);
 	}
 
Signature · 1 line
2.31.1
author reviewer
#2 lore raw

PATCH v3 2/12 drm/i915: Don't free shared locks while shared

In reply to Thomas Hellström

We are currently sharing the VM reservation locks across a number of gem objects with page-table memory. Since TTM will individiualize the reservation locks when freeing objects, including accessing the shared locks, make sure that the shared locks are not freed until that is done. For PPGTT we add an additional refcount, for GGTT we take additional measures to make sure objects sharing the GGTT reservation lock are freed at GGTT takedown

---
v2: Try harder to make sure objects sharing the GGTT reservation lock are freed at GGTT takedown.
v3: Use a pointer to the vm to indicate that an object shares a reservation object from that vm, rather than a pointer to the reservation object itself.

---
 drivers/gpu/drm/i915/gem/i915_gem_object.c    |  3 ++
 .../gpu/drm/i915/gem/i915_gem_object_types.h  |  4 ++
 drivers/gpu/drm/i915/gt/intel_ggtt.c          | 19 ++++++--
 drivers/gpu/drm/i915/gt/intel_gtt.c           | 45 +++++++++++++++----
 drivers/gpu/drm/i915/gt/intel_gtt.h           | 28 +++++++++++-
 drivers/gpu/drm/i915/gt/intel_ppgtt.c         |  2 +-
 drivers/gpu/drm/i915/i915_drv.c               |  5 +++
 7 files changed, 93 insertions(+), 13 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_object.c +3 -0
index 28144410df86..2be6109d0093 100644
@@ -252,6 +252,9 @@ static void __i915_gem_free_objects(struct drm_i915_private *i915,
 		if (obj->mm.n_placements > 1)
 			kfree(obj->mm.placements);
 
+		if (obj->shares_resv_from)
+			i915_vm_resv_put(obj->shares_resv_from);
+
 		/* But keep the pointer alive for RCU-protected lookups */
 		call_rcu(&obj->rcu, __i915_gem_free_object_rcu);
 		cond_resched();
drivers/gpu/drm/i915/gem/i915_gem_object_types.h +4 -0
index 0727d0c76aa0..0415f99b6b95 100644
@@ -149,6 +149,10 @@ struct drm_i915_gem_object {
 	 * when i915_gem_ww_ctx_backoff() or i915_gem_ww_ctx_fini() are called.
 	 */
 	struct list_head obj_link;
+	/**
+	 * @shared_resv_from: The object shares the resv from this vm.
+	 */
+	struct i915_address_space *shares_resv_from;
 
 	union {
 		struct rcu_head rcu;
drivers/gpu/drm/i915/gt/intel_ggtt.c +16 -3
index 35069ca5d7de..10c23a749a95 100644
@@ -746,7 +746,6 @@ static void ggtt_cleanup_hw(struct i915_ggtt *ggtt)
 
 	mutex_unlock(&ggtt->vm.mutex);
 	i915_address_space_fini(&ggtt->vm);
-	dma_resv_fini(&ggtt->vm.resv);
 
 	arch_phys_wc_del(ggtt->mtrr);
 
@@ -768,6 +767,19 @@ void i915_ggtt_driver_release(struct drm_i915_private *i915)
 	ggtt_cleanup_hw(ggtt);
 }
 
+/**
+ * i915_ggtt_driver_late_release - Cleanup of GGTT that needs to be done after
+ * all free objects have been drained.
+ * @i915: i915 device
+ */
+void i915_ggtt_driver_late_release(struct drm_i915_private *i915)
+{
+	struct i915_ggtt *ggtt = &i915->ggtt;
+
+	GEM_WARN_ON(kref_read(&ggtt->vm.resv_ref) != 1);
+	dma_resv_fini(&ggtt->vm._resv);
+}
+
 static unsigned int gen6_get_total_gtt_size(u16 snb_gmch_ctl)
 {
 	snb_gmch_ctl >>= SNB_GMCH_GGMS_SHIFT;
@@ -829,6 +841,7 @@ static int ggtt_probe_common(struct i915_ggtt *ggtt, u64 size)
 		return -ENOMEM;
 	}
 
+	kref_init(&ggtt->vm.resv_ref);
 	ret = setup_scratch_page(&ggtt->vm);
 	if (ret) {
 		drm_err(&i915->drm, "Scratch setup failed\n");
@@ -1135,7 +1148,7 @@ static int ggtt_probe_hw(struct i915_ggtt *ggtt, struct intel_gt *gt)
 	ggtt->vm.gt = gt;
 	ggtt->vm.i915 = i915;
 	ggtt->vm.dma = i915->drm.dev;
-	dma_resv_init(&ggtt->vm.resv);
+	dma_resv_init(&ggtt->vm._resv);
 
 	if (INTEL_GEN(i915) <= 5)
 		ret = i915_gmch_probe(ggtt);
@@ -1144,7 +1157,7 @@ static int ggtt_probe_hw(struct i915_ggtt *ggtt, struct intel_gt *gt)
 	else
 		ret = gen8_gmch_probe(ggtt);
 	if (ret) {
-		dma_resv_fini(&ggtt->vm.resv);
+		dma_resv_fini(&ggtt->vm._resv);
 		return ret;
 	}
 
drivers/gpu/drm/i915/gt/intel_gtt.c +37 -8

88 lines not on this page · raw message

drivers/gpu/drm/i915/gt/intel_gtt.h +27 -1
index ca00b45827b7..f39be66e84f6 100644
@@ -245,7 +245,9 @@ struct i915_address_space {
 	atomic_t open;
 
 	struct mutex mutex; /* protects vma and our lists */
-	struct dma_resv resv; /* reservation lock for all pd objects, and buffer pool */
+
+	struct kref resv_ref; /* kref to keep the reservation lock alive. */
+	struct dma_resv _resv; /* reservation lock for all pd objects, and buffer pool */
 #define VM_CLASS_GGTT 0
 #define VM_CLASS_PPGTT 1
 #define VM_CLASS_DPT 2
@@ -404,13 +406,36 @@ i915_vm_get(struct i915_address_space *vm)
 	return vm;
 }
 
+/**
+ * i915_vm_resv_get - Obtain a reference on the vm's reservation lock
+ * @vm: The vm whose reservation lock we want to share.
+ *
+ * Return: A pointer to the vm's reservation lock.
+ */
+static inline struct dma_resv *i915_vm_resv_get(struct i915_address_space *vm)
+{
+	kref_get(&vm->resv_ref);
+	return &vm->_resv;
+}
+
 void i915_vm_release(struct kref *kref);
 
+void i915_vm_resv_release(struct kref *kref);
+
 static inline void i915_vm_put(struct i915_address_space *vm)
 {
 	kref_put(&vm->ref, i915_vm_release);
 }
 
+/**
+ * i915_vm_resv_put - Release a reference on the vm's reservation lock
+ * @resv: Pointer to a reservation lock obtained from i915_vm_resv_get()
+ */
+static inline void i915_vm_resv_put(struct i915_address_space *vm)
+{
+	kref_put(&vm->resv_ref, i915_vm_resv_release);
+}
+
 static inline struct i915_address_space *
 i915_vm_open(struct i915_address_space *vm)
 {
@@ -506,6 +531,7 @@ void i915_ggtt_enable_guc(struct i915_ggtt *ggtt);
 void i915_ggtt_disable_guc(struct i915_ggtt *ggtt);
 int i915_init_ggtt(struct drm_i915_private *i915);
 void i915_ggtt_driver_release(struct drm_i915_private *i915);
+void i915_ggtt_driver_late_release(struct drm_i915_private *i915);
 
 static inline bool i915_ggtt_has_aperture(const struct i915_ggtt *ggtt)
 {
drivers/gpu/drm/i915/gt/intel_ppgtt.c +1 -1
index 4e3d80c2295c..aee3a8929245 100644
@@ -307,7 +307,7 @@ void ppgtt_init(struct i915_ppgtt *ppgtt, struct intel_gt *gt)
 	ppgtt->vm.dma = i915->drm.dev;
 	ppgtt->vm.total = BIT_ULL(INTEL_INFO(i915)->ppgtt_size);
 
-	dma_resv_init(&ppgtt->vm.resv);
+	dma_resv_init(&ppgtt->vm._resv);
 	i915_address_space_init(&ppgtt->vm, VM_CLASS_PPGTT);
 
 	ppgtt->vm.vma_ops.bind_vma    = ppgtt_bind_vma;
drivers/gpu/drm/i915/i915_drv.c +5 -0
index 5118dc8386b2..92bccc5623a8 100644
@@ -631,6 +631,8 @@ static int i915_driver_hw_probe(struct drm_i915_private *dev_priv)
 	intel_memory_regions_driver_release(dev_priv);
 err_ggtt:
 	i915_ggtt_driver_release(dev_priv);
+	i915_gem_drain_freed_objects(dev_priv);
+	i915_ggtt_driver_late_release(dev_priv);
 err_perf:
 	i915_perf_fini(dev_priv);
 	return ret;
@@ -880,6 +882,8 @@ int i915_driver_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 	i915_driver_hw_remove(i915);
 	intel_memory_regions_driver_release(i915);
 	i915_ggtt_driver_release(i915);
+	i915_gem_drain_freed_objects(i915);
+	i915_ggtt_driver_late_release(i915);
 out_cleanup_mmio:
 	i915_driver_mmio_release(i915);
 out_runtime_pm_put:
@@ -936,6 +940,7 @@ static void i915_driver_release(struct drm_device *dev)
 	intel_memory_regions_driver_release(dev_priv);
 	i915_ggtt_driver_release(dev_priv);
 	i915_gem_drain_freed_objects(dev_priv);
+	i915_ggtt_driver_late_release(dev_priv);
 
 	i915_driver_mmio_release(dev_priv);
 
Signature · 6 lines
2.31.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
author reviewer
#4 lore raw

PATCH v3 4/12 drm/i915/ttm Initialize the ttm device and memory managers

In reply to Thomas Hellström

Temporarily remove the buddy allocator and related selftests and hook up the TTM range manager for i915 regions.

Also modify the mock region selftests somewhat to account for a fragmenting manager.

---
v2:
- Fix an error unwind in lmem_get_pages() (Reported by Matthew Auld)
- Break out and modify usage of i915_sg_dma_sizes() (Reported by Mattew Auld)
- Break out TTM changes to a separate patch (Reported by Christian König) v3:
- Fix the same error unwind in mock_region_get_pages()
(Reported by Matthew Auld)
- Don't rely on new TTM functionality, but create a mock TTM device, (Reported by Christian König)

---
 drivers/gpu/drm/i915/Kconfig                  |   1 +
 drivers/gpu/drm/i915/Makefile                 |   2 +-
 drivers/gpu/drm/i915/gem/i915_gem_lmem.c      |  59 +-
 .../gpu/drm/i915/gem/i915_gem_object_types.h  |   6 +-
 drivers/gpu/drm/i915/gem/i915_gem_pages.c     |   3 +-
 drivers/gpu/drm/i915/gem/i915_gem_region.c    | 120 ---
 drivers/gpu/drm/i915/gem/i915_gem_region.h    |   4 -
 drivers/gpu/drm/i915/gem/i915_gem_shmem.c     |   4 +-
 drivers/gpu/drm/i915/gem/i915_gem_stolen.c    |  10 +-
 drivers/gpu/drm/i915/gem/i915_gem_stolen.h    |   9 +-
 drivers/gpu/drm/i915/gt/intel_gt.c            |   2 -
 drivers/gpu/drm/i915/gt/intel_region_lmem.c   |  27 +-
 drivers/gpu/drm/i915/i915_buddy.c             | 435 ----------
 drivers/gpu/drm/i915/i915_buddy.h             | 131 ---
 drivers/gpu/drm/i915/i915_drv.c               |   8 +
 drivers/gpu/drm/i915/i915_drv.h               |   7 +-
 drivers/gpu/drm/i915/i915_gem.c               |   1 +
 drivers/gpu/drm/i915/i915_globals.c           |   1 -
 drivers/gpu/drm/i915/i915_globals.h           |   1 -
 drivers/gpu/drm/i915/i915_scatterlist.c       |  70 ++
 drivers/gpu/drm/i915/i915_scatterlist.h       |   4 +
 drivers/gpu/drm/i915/intel_memory_region.c    | 180 ++--
 drivers/gpu/drm/i915/intel_memory_region.h    |  44 +-
 drivers/gpu/drm/i915/intel_region_ttm.c       | 334 ++++++++
 drivers/gpu/drm/i915/intel_region_ttm.h       |  38 +
 drivers/gpu/drm/i915/selftests/i915_buddy.c   | 789 ------------------
 .../drm/i915/selftests/i915_mock_selftests.h  |   1 -
 .../drm/i915/selftests/intel_memory_region.c  | 133 +--
 drivers/gpu/drm/i915/selftests/mock_region.c  |  52 +-
 29 files changed, 722 insertions(+), 1754 deletions(-)
 delete mode 100644 drivers/gpu/drm/i915/i915_buddy.c
 delete mode 100644 drivers/gpu/drm/i915/i915_buddy.h
 create mode 100644 drivers/gpu/drm/i915/intel_region_ttm.c
 create mode 100644 drivers/gpu/drm/i915/intel_region_ttm.h
 delete mode 100644 drivers/gpu/drm/i915/selftests/i915_buddy.c
drivers/gpu/drm/i915/Kconfig +1 -0
index 1e1cb245fca7..b63d374dff23 100644
@@ -26,6 +26,7 @@ config DRM_I915
 	select SND_HDA_I915 if SND_HDA_CORE
 	select CEC_CORE if CEC_NOTIFIER
 	select VMAP_PFN
+	select DRM_TTM
 	help
 	  Choose this option if you have a system that has "Intel Graphics
 	  Media Accelerator" or "HD Graphics" integrated graphics,
drivers/gpu/drm/i915/Makefile +1 -1
index d0d936d9137b..cb8823570996 100644
@@ -50,6 +50,7 @@ i915-y += i915_drv.o \
 	  intel_memory_region.o \
 	  intel_pch.o \
 	  intel_pm.o \
+	  intel_region_ttm.o \
 	  intel_runtime_pm.o \
 	  intel_sideband.o \
 	  intel_step.o \
@@ -160,7 +161,6 @@ gem-y += \
 i915-y += \
 	  $(gem-y) \
 	  i915_active.o \
-	  i915_buddy.o \
 	  i915_cmd_parser.o \
 	  i915_gem_evict.o \
 	  i915_gem_gtt.o \
drivers/gpu/drm/i915/gem/i915_gem_lmem.c +57 -2

73 lines not on this page · raw message

drivers/gpu/drm/i915/gem/i915_gem_object_types.h +4 -2
index 0415f99b6b95..f5b46d11e6e6 100644
@@ -235,10 +235,12 @@ struct drm_i915_gem_object {
 		 * Memory region for this object.
 		 */
 		struct intel_memory_region *region;
+
 		/**
-		 * List of memory region blocks allocated for this object.
+		 * Memory manager node allocated for this object.
 		 */
-		struct list_head blocks;
+		void *st_mm_node;
+
 		/**
 		 * Element within memory_region->objects or region->purgeable
 		 * if the object is marked as DONTNEED. Access is protected by
drivers/gpu/drm/i915/gem/i915_gem_pages.c +2 -1
index aed8a37ccdc9..62ee2185a41b 100644
@@ -473,7 +473,8 @@ __i915_gem_object_get_sg(struct drm_i915_gem_object *obj,
 
 	might_sleep();
 	GEM_BUG_ON(n >= obj->base.size >> PAGE_SHIFT);
-	GEM_BUG_ON(!i915_gem_object_has_pinned_pages(obj));
+	if (!i915_gem_object_has_pinned_pages(obj))
+		assert_object_held(obj);
 
 	/* As we iterate forward through the sg, we record each entry in a
 	 * radixtree for quick repeated (backwards) lookups. If we have seen
drivers/gpu/drm/i915/gem/i915_gem_region.c +0 -120

129 lines not on this page · raw message

drivers/gpu/drm/i915/gem/i915_gem_region.h +0 -4
index ebddc86d78f7..84fcb3297400 100644
@@ -12,10 +12,6 @@ struct intel_memory_region;
 struct drm_i915_gem_object;
 struct sg_table;
 
-int i915_gem_object_get_pages_buddy(struct drm_i915_gem_object *obj);
-void i915_gem_object_put_pages_buddy(struct drm_i915_gem_object *obj,
-				     struct sg_table *pages);
-
 void i915_gem_object_init_memory_region(struct drm_i915_gem_object *obj,
 					struct intel_memory_region *mem);
 void i915_gem_object_release_memory_region(struct drm_i915_gem_object *obj);
drivers/gpu/drm/i915/gem/i915_gem_shmem.c +3 -1
index a9bfa66c8da1..5d16c4462fda 100644
@@ -628,11 +628,13 @@ static const struct intel_memory_region_ops shmem_region_ops = {
 	.init_object = shmem_object_init,
 };
 
-struct intel_memory_region *i915_gem_shmem_setup(struct drm_i915_private *i915)
+struct intel_memory_region *i915_gem_shmem_setup(struct drm_i915_private *i915,
+						 u16 type, u16 instance)
 {
 	return intel_memory_region_create(i915, 0,
 					  totalram_pages() << PAGE_SHIFT,
 					  PAGE_SIZE, 0,
+					  type, instance,
 					  &shmem_region_ops);
 }
 
drivers/gpu/drm/i915/gem/i915_gem_stolen.c +6 -4
index 293f640faa0a..c42abee206da 100644
@@ -770,7 +770,8 @@ static const struct intel_memory_region_ops i915_region_stolen_lmem_ops = {
 };
 
 struct intel_memory_region *
-i915_gem_stolen_lmem_setup(struct drm_i915_private *i915)
+i915_gem_stolen_lmem_setup(struct drm_i915_private *i915, u16 type,
+			   u16 instance)
 {
 	struct intel_uncore *uncore = &i915->uncore;
 	struct pci_dev *pdev = to_pci_dev(i915->drm.dev);
@@ -788,6 +789,7 @@ i915_gem_stolen_lmem_setup(struct drm_i915_private *i915)
 
 	mem = intel_memory_region_create(i915, lmem_base, lmem_size,
 					 I915_GTT_PAGE_SIZE_4K, io_start,
+					 type, instance,
 					 &i915_region_stolen_lmem_ops);
 	if (IS_ERR(mem))
 		return mem;
@@ -809,14 +811,15 @@ i915_gem_stolen_lmem_setup(struct drm_i915_private *i915)
 }
 
 struct intel_memory_region*
-i915_gem_stolen_smem_setup(struct drm_i915_private *i915)
+i915_gem_stolen_smem_setup(struct drm_i915_private *i915, u16 type,
+			   u16 instance)
 {
 	struct intel_memory_region *mem;
 
 	mem = intel_memory_region_create(i915,
 					 intel_graphics_stolen_res.start,
 					 resource_size(&intel_graphics_stolen_res),
-					 PAGE_SIZE, 0,
+					 PAGE_SIZE, 0, type, instance,
 					 &i915_region_stolen_smem_ops);
 	if (IS_ERR(mem))
 		return mem;
@@ -824,7 +827,6 @@ i915_gem_stolen_smem_setup(struct drm_i915_private *i915)
 	intel_memory_region_set_name(mem, "stolen-system");
 
 	mem->private = true;
-
 	return mem;
 }
 
drivers/gpu/drm/i915/gem/i915_gem_stolen.h +7 -2
index 2bec6c367b9c..ccdf7befc571 100644
@@ -21,8 +21,13 @@ int i915_gem_stolen_insert_node_in_range(struct drm_i915_private *dev_priv,
 					 u64 end);
 void i915_gem_stolen_remove_node(struct drm_i915_private *dev_priv,
 				 struct drm_mm_node *node);
-struct intel_memory_region *i915_gem_stolen_smem_setup(struct drm_i915_private *i915);
-struct intel_memory_region *i915_gem_stolen_lmem_setup(struct drm_i915_private *i915);
+struct intel_memory_region *
+i915_gem_stolen_smem_setup(struct drm_i915_private *i915, u16 type,
+			   u16 instance);
+struct intel_memory_region *
+i915_gem_stolen_lmem_setup(struct drm_i915_private *i915, u16 type,
+			   u16 instance);
+
 struct drm_i915_gem_object *
 i915_gem_object_create_stolen(struct drm_i915_private *dev_priv,
 			      resource_size_t size);
drivers/gpu/drm/i915/gt/intel_gt.c +0 -2
index 8d77dcbad059..3f88ecdee031 100644
@@ -68,8 +68,6 @@ int intel_gt_probe_lmem(struct intel_gt *gt)
 	id = INTEL_REGION_LMEM;
 
 	mem->id = id;
-	mem->type = INTEL_MEMORY_LOCAL;
-	mem->instance = 0;
 
 	intel_memory_region_set_name(mem, "local%u", mem->instance);
 
drivers/gpu/drm/i915/gt/intel_region_lmem.c +21 -6
index 73fceb0c25fc..f7366b054f8e 100644
@@ -5,6 +5,8 @@
 
 #include "i915_drv.h"
 #include "intel_memory_region.h"
+#include "intel_region_lmem.h"
+#include "intel_region_ttm.h"
 #include "gem/i915_gem_lmem.h"
 #include "gem/i915_gem_region.h"
 #include "intel_region_lmem.h"
@@ -66,9 +68,9 @@ static void release_fake_lmem_bar(struct intel_memory_region *mem)
 static void
 region_lmem_release(struct intel_memory_region *mem)
 {
-	release_fake_lmem_bar(mem);
+	intel_region_ttm_fini(mem);
 	io_mapping_fini(&mem->iomap);
-	intel_memory_region_release_buddy(mem);
+	release_fake_lmem_bar(mem);
 }
 
 static int
@@ -83,12 +85,21 @@ region_lmem_init(struct intel_memory_region *mem)
 
 	if (!io_mapping_init_wc(&mem->iomap,
 				mem->io_start,
-				resource_size(&mem->region)))
-		return -EIO;
+				resource_size(&mem->region))) {
+		ret = -EIO;
+		goto out_no_io;
+	}
 
-	ret = intel_memory_region_init_buddy(mem);
+	ret = intel_region_ttm_init(mem);
 	if (ret)
-		io_mapping_fini(&mem->iomap);
+		goto out_no_buddy;
+
+	return 0;
+
+out_no_buddy:
+	io_mapping_fini(&mem->iomap);
+out_no_io:
+	release_fake_lmem_bar(mem);
 
 	return ret;
 }
@@ -127,6 +138,8 @@ intel_gt_setup_fake_lmem(struct intel_gt *gt)
 					 mappable_end,
 					 PAGE_SIZE,
 					 io_start,
+					 INTEL_MEMORY_LOCAL,
+					 0,
 					 &intel_region_lmem_ops);
 	if (!IS_ERR(mem)) {
 		drm_info(&i915->drm, "Intel graphics fake LMEM: %pR\n",
@@ -198,6 +211,8 @@ static struct intel_memory_region *setup_lmem(struct intel_gt *gt)
 					 lmem_size,
 					 I915_GTT_PAGE_SIZE_4K,
 					 io_start,
+					 INTEL_MEMORY_LOCAL,
+					 0,
 					 &intel_region_lmem_ops);
 	if (IS_ERR(mem))
 		return mem;
drivers/gpu/drm/i915/i915_buddy.c deleted +0 -435

435 lines not on this page · raw message

drivers/gpu/drm/i915/i915_buddy.h deleted +0 -131

131 lines not on this page · raw message

drivers/gpu/drm/i915/i915_drv.c +8 -0
index 92bccc5623a8..122dd297b6af 100644
@@ -84,6 +84,7 @@
 #include "intel_gvt.h"
 #include "intel_memory_region.h"
 #include "intel_pm.h"
+#include "intel_region_ttm.h"
 #include "intel_sideband.h"
 #include "vlv_suspend.h"
 
@@ -335,6 +336,10 @@ static int i915_driver_early_probe(struct drm_i915_private *dev_priv)
 	if (ret < 0)
 		goto err_workqueues;
 
+	ret = intel_region_ttm_device_init(dev_priv);
+	if (ret)
+		goto err_ttm;
+
 	intel_wopcm_init_early(&dev_priv->wopcm);
 
 	intel_gt_init_early(&dev_priv->gt, dev_priv);
@@ -359,6 +364,8 @@ static int i915_driver_early_probe(struct drm_i915_private *dev_priv)
 err_gem:
 	i915_gem_cleanup_early(dev_priv);
 	intel_gt_driver_late_release(&dev_priv->gt);
+	intel_region_ttm_device_fini(dev_priv);
+err_ttm:
 	vlv_suspend_cleanup(dev_priv);
 err_workqueues:
 	i915_workqueues_cleanup(dev_priv);
@@ -376,6 +383,7 @@ static void i915_driver_late_release(struct drm_i915_private *dev_priv)
 	intel_power_domains_cleanup(dev_priv);
 	i915_gem_cleanup_early(dev_priv);
 	intel_gt_driver_late_release(&dev_priv->gt);
+	intel_region_ttm_device_fini(dev_priv);
 	vlv_suspend_cleanup(dev_priv);
 	i915_workqueues_cleanup(dev_priv);
 
drivers/gpu/drm/i915/i915_drv.h +6 -1
index 128198e8b4d0..1276a2f60740 100644
@@ -60,6 +60,7 @@
 #include <drm/drm_atomic.h>
 #include <drm/drm_connector.h>
 #include <drm/i915_mei_hdcp_interface.h>
+#include <drm/ttm/ttm_device.h>
 
 #include "i915_params.h"
 #include "i915_reg.h"
@@ -1166,6 +1167,9 @@ struct drm_i915_private {
 	/* Mutex to protect the above hdcp component related values. */
 	struct mutex hdcp_comp_mutex;
 
+	/* The TTM device structure. */
+	struct ttm_device bdev;
+
 	I915_SELFTEST_DECLARE(struct i915_selftest_stash selftest;)
 
 	/*
@@ -1751,7 +1755,8 @@ void i915_gem_cleanup_userptr(struct drm_i915_private *dev_priv);
 void i915_gem_init_early(struct drm_i915_private *dev_priv);
 void i915_gem_cleanup_early(struct drm_i915_private *dev_priv);
 
-struct intel_memory_region *i915_gem_shmem_setup(struct drm_i915_private *i915);
+struct intel_memory_region *i915_gem_shmem_setup(struct drm_i915_private *i915,
+						 u16 type, u16 instance);
 
 static inline void i915_gem_drain_freed_objects(struct drm_i915_private *i915)
 {
drivers/gpu/drm/i915/i915_gem.c +1 -0
index d0018c5f88bd..180f6e9107d4 100644
@@ -1109,6 +1109,7 @@ int i915_gem_init(struct drm_i915_private *dev_priv)
 	}
 
 	i915_gem_drain_freed_objects(dev_priv);
+
 	return ret;
 }
 
drivers/gpu/drm/i915/i915_globals.c +0 -1
index 3aa213684293..77f1911c463b 100644
@@ -87,7 +87,6 @@ static void __i915_globals_cleanup(void)
 
 static __initconst int (* const initfn[])(void) = {
 	i915_global_active_init,
-	i915_global_buddy_init,
 	i915_global_context_init,
 	i915_global_gem_context_init,
 	i915_global_objects_init,
drivers/gpu/drm/i915/i915_globals.h +0 -1
index b2f5cd9b9b1a..2d199f411a4a 100644
@@ -27,7 +27,6 @@ void i915_globals_exit(void);
 
 /* constructors */
 int i915_global_active_init(void);
-int i915_global_buddy_init(void);
 int i915_global_context_init(void);
 int i915_global_gem_context_init(void);
 int i915_global_objects_init(void);
drivers/gpu/drm/i915/i915_scatterlist.c +70 -0

82 lines not on this page · raw message

drivers/gpu/drm/i915/i915_scatterlist.h +4 -0
index b96baad66a3a..5acca45ea981 100644
@@ -13,6 +13,8 @@
 
 #include "i915_gem.h"
 
+struct drm_mm_node;
+
 /*
  * Optimised SGL iterator for GEM objects
  */
@@ -141,4 +143,6 @@ static inline unsigned int i915_sg_segment_size(void)
 
 bool i915_sg_trim(struct sg_table *orig_st);
 
+struct sg_table *i915_sg_from_mm_node(const struct drm_mm_node *node,
+				      u64 region_start);
 #endif
drivers/gpu/drm/i915/intel_memory_region.c +51 -129

245 lines not on this page · raw message

drivers/gpu/drm/i915/intel_memory_region.h +23 -21

94 lines not on this page · raw message

drivers/gpu/drm/i915/intel_region_ttm.c new file +334 -0

334 lines not on this page · raw message

drivers/gpu/drm/i915/intel_region_ttm.h new file +38 -0
new file mode 100644
index 000000000000..23c6c2d4b1cd
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+#ifndef _INTEL_REGION_TTM_H_
+#define _INTEL_REGION_TTM_H_
+
+#include <linux/types.h>
+
+#include "i915_selftest.h"
+
+struct drm_i915_private;
+struct intel_memory_region;
+
+int intel_region_ttm_device_init(struct drm_i915_private *dev_priv);
+
+void intel_region_ttm_device_fini(struct drm_i915_private *dev_priv);
+
+int intel_region_ttm_init(struct intel_memory_region *mem);
+
+void intel_region_ttm_fini(struct intel_memory_region *mem);
+
+struct sg_table *intel_region_ttm_node_to_st(struct intel_memory_region *mem,
+					     void *node);
+
+void *intel_region_ttm_node_alloc(struct intel_memory_region *mem,
+				  resource_size_t size,
+				  unsigned int flags);
+
+void intel_region_ttm_node_free(struct intel_memory_region *mem,
+				void *node);
+
+#ifdef CONFIG_DRM_I915_SELFTEST
+int intel_region_ttm_init_mock(struct intel_memory_region *mem);
+
+void intel_region_ttm_fini_mock(struct intel_memory_region *mem);
+#endif
+#endif /* _INTEL_REGION_TTM_H_ */
drivers/gpu/drm/i915/selftests/i915_buddy.c deleted +0 -789

789 lines not on this page · raw message

drivers/gpu/drm/i915/selftests/i915_mock_selftests.h +0 -1
index 3db34d3eea58..34e5caf38093 100644
@@ -33,5 +33,4 @@ selftest(evict, i915_gem_evict_mock_selftests)
 selftest(gtt, i915_gem_gtt_mock_selftests)
 selftest(hugepages, i915_gem_huge_page_mock_selftests)
 selftest(contexts, i915_gem_context_mock_selftests)
-selftest(buddy, i915_buddy_mock_selftests)
 selftest(memory_region, intel_memory_region_mock_selftests)
drivers/gpu/drm/i915/selftests/intel_memory_region.c +40 -93

213 lines not on this page · raw message

drivers/gpu/drm/i915/selftests/mock_region.c +45 -7

82 lines not on this page · raw message

Signature · 1 line
2.31.1
author reviewer
#5 lore raw

PATCH v3 5/12 drm/i915/ttm: Embed a ttm buffer object in the i915 gem object

In reply to Thomas Hellström

Embed a struct ttm_buffer_object into the i915 gem object, making sure we alias the gem object part. It's a bit unfortunate that the struct ttm_buffer_ojbect embeds a gem object since we otherwise could make the TTM part private to the TTM backend, and use the usual i915 gem object for the other backends.
To make this a bit more storage efficient for the other backends, we'd have to use a pointer for the gem object which would require a lot of changes in the driver. We postpone that for later.

---
 drivers/gpu/drm/i915/gem/i915_gem_object.c       |  7 +++++++
 drivers/gpu/drm/i915/gem/i915_gem_object_types.h | 12 +++++++++++-
 2 files changed, 18 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/gem/i915_gem_object.c +7 -0
index 2be6109d0093..5706d471692d 100644
@@ -62,6 +62,13 @@ void i915_gem_object_init(struct drm_i915_gem_object *obj,
 			  const struct drm_i915_gem_object_ops *ops,
 			  struct lock_class_key *key, unsigned flags)
 {
+	/*
+	 * A gem object is embedded both in a struct ttm_buffer_object :/ and
+	 * in a drm_i915_gem_object. Make sure they are aliased.
+	 */
+	BUILD_BUG_ON(offsetof(typeof(*obj), base) !=
+		     offsetof(typeof(*obj), __do_not_access.base));
+
 	spin_lock_init(&obj->vma.lock);
 	INIT_LIST_HEAD(&obj->vma.list);
 
drivers/gpu/drm/i915/gem/i915_gem_object_types.h +11 -1
index f5b46d11e6e6..d047ea126029 100644
@@ -10,6 +10,7 @@
 #include <linux/mmu_notifier.h>
 
 #include <drm/drm_gem.h>
+#include <drm/ttm/ttm_bo_api.h>
 #include <uapi/drm/i915_drm.h>
 
 #include "i915_active.h"
@@ -99,7 +100,16 @@ struct i915_gem_object_page_iter {
 };
 
 struct drm_i915_gem_object {
-	struct drm_gem_object base;
+	/*
+	 * We might have reason to revisit the below since it wastes
+	 * a lot of space for non-ttm gem objects.
+	 * In any case, always use the accessors for the ttm_buffer_object
+	 * when accessing it.
+	 */
+	union {
+		struct drm_gem_object base;
+		struct ttm_buffer_object __do_not_access;
+	};
 
 	const struct drm_i915_gem_object_ops *ops;
 
Signature · 1 line
2.31.1
author reviewer
#15 lore raw

PATCH v3 7/12 drm, drm/i915: Move the memcpy_from_wc functionality to core drm

In reply to Thomas Hellström

Memcpy from wc will be used as well by TTM memcpy. Move it to core drm, and make the interface do the right thing even on !X86.

Cc Christian König <[email protected]>
Cc Daniel Vetter <[email protected]>
Cc Dave Airlie <[email protected]>
---
 drivers/gpu/drm/Makefile                      |  2 +-
 drivers/gpu/drm/drm_drv.c                     |  2 +
 .../drm/{i915/i915_memcpy.c => drm_memcpy.c}  | 63 ++++++++++++++-----
 drivers/gpu/drm/i915/Makefile                 |  1 -
 .../gpu/drm/i915/gem/i915_gem_execbuffer.c    |  4 +-
 drivers/gpu/drm/i915/gem/i915_gem_object.c    |  5 +-
 drivers/gpu/drm/i915/gt/selftest_reset.c      |  7 ++-
 drivers/gpu/drm/i915/gt/uc/intel_guc_log.c    | 11 ++--
 drivers/gpu/drm/i915/i915_cmd_parser.c        |  4 +-
 drivers/gpu/drm/i915/i915_drv.c               |  2 -
 drivers/gpu/drm/i915/i915_gpu_error.c         |  8 +--
 drivers/gpu/drm/i915/i915_memcpy.h            | 34 ----------
 .../drm/i915/selftests/intel_memory_region.c  |  7 ++-
 include/drm/drm_memcpy.h                      | 47 ++++++++++++++
 14 files changed, 121 insertions(+), 76 deletions(-)
 rename drivers/gpu/drm/{i915/i915_memcpy.c => drm_memcpy.c} (70%)
 delete mode 100644 drivers/gpu/drm/i915/i915_memcpy.h
 create mode 100644 include/drm/drm_memcpy.h
drivers/gpu/drm/Makefile +1 -1
index a91cc7684904..f3ab8586c3d7 100644
@@ -18,7 +18,7 @@ drm-y       :=	drm_aperture.o drm_auth.o drm_cache.o \
 		drm_dumb_buffers.o drm_mode_config.o drm_vblank.o \
 		drm_syncobj.o drm_lease.o drm_writeback.o drm_client.o \
 		drm_client_modeset.o drm_atomic_uapi.o drm_hdcp.o \
-		drm_managed.o drm_vblank_work.o
+		drm_managed.o drm_vblank_work.o drm_memcpy.o \
 
 drm-$(CONFIG_DRM_LEGACY) += drm_agpsupport.o drm_bufs.o drm_context.o drm_dma.o \
 			    drm_legacy_misc.o drm_lock.o drm_memory.o drm_scatter.o \
drivers/gpu/drm/drm_drv.c +2 -0
index 3d8d68a98b95..351cc2900cf1 100644
@@ -40,6 +40,7 @@
 #include <drm/drm_drv.h>
 #include <drm/drm_file.h>
 #include <drm/drm_managed.h>
+#include <drm/drm_memcpy.h>
 #include <drm/drm_mode_object.h>
 #include <drm/drm_print.h>
 
@@ -1041,6 +1042,7 @@ static int __init drm_core_init(void)
 
 	drm_connector_ida_init();
 	idr_init(&drm_minors_idr);
+	drm_memcpy_init_early();
 
 	ret = drm_sysfs_init();
 	if (ret < 0) {
drivers/gpu/drm/drm_memcpy.c renamed from drivers/gpu/drm/i915/i915_memcpy.c +46 -17

123 lines not on this page · raw message

drivers/gpu/drm/i915/Makefile +0 -1
index cb8823570996..998606b7f49f 100644
@@ -61,7 +61,6 @@ i915-y += i915_drv.o \
 # core library code
 i915-y += \
 	dma_resv_utils.o \
-	i915_memcpy.o \
 	i915_mm.o \
 	i915_sw_fence.o \
 	i915_sw_fence_work.o \
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +2 -2
index 297143511f99..77285e421fb8 100644
@@ -10,6 +10,7 @@
 #include <linux/uaccess.h>
 
 #include <drm/drm_syncobj.h>
+#include <drm/drm_memcpy.h>
 
 #include "display/intel_frontbuffer.h"
 
@@ -28,7 +29,6 @@
 #include "i915_sw_fence_work.h"
 #include "i915_trace.h"
 #include "i915_user_extensions.h"
-#include "i915_memcpy.h"
 
 struct eb_vma {
 	struct i915_vma *vma;
@@ -2503,7 +2503,7 @@ static int eb_parse_pipeline(struct i915_execbuffer *eb,
 		!(batch->cache_coherent & I915_BO_CACHE_COHERENT_FOR_READ);
 
 	pw->batch_map = ERR_PTR(-ENODEV);
-	if (needs_clflush && i915_has_memcpy_from_wc())
+	if (needs_clflush && drm_has_memcpy_from_wc())
 		pw->batch_map = i915_gem_object_pin_map(batch, I915_MAP_WC);
 
 	if (IS_ERR(pw->batch_map)) {
drivers/gpu/drm/i915/gem/i915_gem_object.c +3 -2
index 5706d471692d..e9247afb0320 100644
@@ -24,6 +24,8 @@
 
 #include <linux/sched/mm.h>
 
+#include <drm/drm_memcpy.h>
+
 #include "display/intel_frontbuffer.h"
 #include "i915_drv.h"
 #include "i915_gem_clflush.h"
@@ -31,7 +33,6 @@
 #include "i915_gem_mman.h"
 #include "i915_gem_object.h"
 #include "i915_globals.h"
-#include "i915_memcpy.h"
 #include "i915_trace.h"
 
 static struct i915_global_object {
@@ -374,7 +375,7 @@ i915_gem_object_read_from_page_iomap(struct drm_i915_gem_object *obj, u64 offset
 				    PAGE_SIZE);
 
 	src_ptr = src_map + offset_in_page(offset);
-	if (!i915_memcpy_from_wc(dst, (void __force *)src_ptr, size))
+	if (!drm_memcpy_from_wc(dst, (void __force *)src_ptr, size))
 		memcpy_fromio(dst, src_ptr, size);
 
 	io_mapping_unmap(src_map);
drivers/gpu/drm/i915/gt/selftest_reset.c +4 -3
index 8784257ec808..92ada67a3835 100644
@@ -5,9 +5,10 @@
 
 #include <linux/crc32.h>
 
+#include <drm/drm_memcpy.h>
+
 #include "gem/i915_gem_stolen.h"
 
-#include "i915_memcpy.h"
 #include "i915_selftest.h"
 #include "intel_gpu_commands.h"
 #include "selftests/igt_reset.h"
@@ -99,7 +100,7 @@ __igt_reset_stolen(struct intel_gt *gt,
 			memset_io(s, STACK_MAGIC, PAGE_SIZE);
 
 		in = (void __force *)s;
-		if (i915_memcpy_from_wc(tmp, in, PAGE_SIZE))
+		if (drm_memcpy_from_wc(tmp, in, PAGE_SIZE))
 			in = tmp;
 		crc[page] = crc32_le(0, in, PAGE_SIZE);
 
@@ -135,7 +136,7 @@ __igt_reset_stolen(struct intel_gt *gt,
 				      PAGE_SIZE);
 
 		in = (void __force *)s;
-		if (i915_memcpy_from_wc(tmp, in, PAGE_SIZE))
+		if (drm_memcpy_from_wc(tmp, in, PAGE_SIZE))
 			in = tmp;
 		x = crc32_le(0, in, PAGE_SIZE);
 
drivers/gpu/drm/i915/gt/uc/intel_guc_log.c +6 -5
index c36d5eb5bbb9..f045e42be6ca 100644
@@ -5,9 +5,10 @@
 
 #include <linux/debugfs.h>
 
+#include <drm/drm_memcpy.h>
+
 #include "gt/intel_gt.h"
 #include "i915_drv.h"
-#include "i915_memcpy.h"
 #include "intel_guc_log.h"
 
 static void guc_log_capture_logs(struct intel_guc_log *log);
@@ -295,13 +296,13 @@ static void guc_read_update_log_buffer(struct intel_guc_log *log)
 
 		/* Just copy the newly written data */
 		if (read_offset > write_offset) {
-			i915_memcpy_from_wc(dst_data, src_data, write_offset);
+			drm_memcpy_from_wc(dst_data, src_data, write_offset);
 			bytes_to_copy = buffer_size - read_offset;
 		} else {
 			bytes_to_copy = write_offset - read_offset;
 		}
-		i915_memcpy_from_wc(dst_data + read_offset,
-				    src_data + read_offset, bytes_to_copy);
+		drm_memcpy_from_wc(dst_data + read_offset,
+				   src_data + read_offset, bytes_to_copy);
 
 		src_data += buffer_size;
 		dst_data += buffer_size;
@@ -569,7 +570,7 @@ int intel_guc_log_relay_open(struct intel_guc_log *log)
 	 * it should be present on the chipsets supporting GuC based
 	 * submisssions.
 	 */
-	if (!i915_has_memcpy_from_wc()) {
+	if (!drm_has_memcpy_from_wc()) {
 		ret = -ENXIO;
 		goto out_unlock;
 	}
drivers/gpu/drm/i915/i915_cmd_parser.c +2 -2
index 5b4b2bd46e7c..98653f1a2b1d 100644
@@ -24,12 +24,12 @@
  *    Brad Volkin <[email protected]>
  *
  */
+#include <drm/drm_memcpy.h>
 
 #include "gt/intel_engine.h"
 #include "gt/intel_gpu_commands.h"
 
 #include "i915_drv.h"
-#include "i915_memcpy.h"
 
 /**
  * DOC: batch buffer command parser
@@ -1152,7 +1152,7 @@ static u32 *copy_batch(struct drm_i915_gem_object *dst_obj,
 
 	if (src) {
 		GEM_BUG_ON(!needs_clflush);
-		i915_unaligned_memcpy_from_wc(dst, src + offset, length);
+		drm_unaligned_memcpy_from_wc(dst, src + offset, length);
 	} else {
 		struct scatterlist *sg;
 		void *ptr;
drivers/gpu/drm/i915/i915_drv.c +0 -2
index 122dd297b6af..0df9dd62c717 100644
@@ -72,7 +72,6 @@
 #include "i915_drv.h"
 #include "i915_ioc32.h"
 #include "i915_irq.h"
-#include "i915_memcpy.h"
 #include "i915_perf.h"
 #include "i915_query.h"
 #include "i915_suspend.h"
@@ -325,7 +324,6 @@ static int i915_driver_early_probe(struct drm_i915_private *dev_priv)
 	mutex_init(&dev_priv->pps_mutex);
 	mutex_init(&dev_priv->hdcp_comp_mutex);
 
-	i915_memcpy_init_early(dev_priv);
 	intel_runtime_pm_init_early(&dev_priv->runtime_pm);
 
 	ret = i915_workqueues_init(dev_priv);
drivers/gpu/drm/i915/i915_gpu_error.c +4 -4
index 99ca242ec13b..ee11920fbea5 100644
@@ -34,6 +34,7 @@
 #include <linux/utsname.h>
 #include <linux/zlib.h>
 
+#include <drm/drm_memcpy.h>
 #include <drm/drm_print.h>
 
 #include "display/intel_csr.h"
@@ -46,7 +47,6 @@
 
 #include "i915_drv.h"
 #include "i915_gpu_error.h"
-#include "i915_memcpy.h"
 #include "i915_scatterlist.h"
 
 #define ALLOW_FAIL (GFP_KERNEL | __GFP_RETRY_MAYFAIL | __GFP_NOWARN)
@@ -255,7 +255,7 @@ static bool compress_init(struct i915_vma_compress *c)
 	}
 
 	c->tmp = NULL;
-	if (i915_has_memcpy_from_wc())
+	if (drm_has_memcpy_from_wc())
 		c->tmp = pool_alloc(&c->pool, ALLOW_FAIL);
 
 	return true;
@@ -295,7 +295,7 @@ static int compress_page(struct i915_vma_compress *c,
 	struct z_stream_s *zstream = &c->zstream;
 
 	zstream->next_in = src;
-	if (wc && c->tmp && i915_memcpy_from_wc(c->tmp, src, PAGE_SIZE))
+	if (wc && c->tmp && drm_memcpy_from_wc(c->tmp, src, PAGE_SIZE))
 		zstream->next_in = c->tmp;
 	zstream->avail_in = PAGE_SIZE;
 
@@ -395,7 +395,7 @@ static int compress_page(struct i915_vma_compress *c,
 	if (!ptr)
 		return -ENOMEM;
 
-	if (!(wc && i915_memcpy_from_wc(ptr, src, PAGE_SIZE)))
+	if (!(wc && drm_memcpy_from_wc(ptr, src, PAGE_SIZE)))
 		memcpy(ptr, src, PAGE_SIZE);
 	dst->pages[dst->page_count++] = ptr;
 	cond_resched();
drivers/gpu/drm/i915/i915_memcpy.h deleted +0 -34
deleted file mode 100644
index 3df063a3293b..000000000000
@@ -1,34 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-/*
- * Copyright © 2019 Intel Corporation
- */
-
-#ifndef __I915_MEMCPY_H__
-#define __I915_MEMCPY_H__
-
-#include <linux/types.h>
-
-struct drm_i915_private;
-
-void i915_memcpy_init_early(struct drm_i915_private *i915);
-
-bool i915_memcpy_from_wc(void *dst, const void *src, unsigned long len);
-void i915_unaligned_memcpy_from_wc(void *dst, const void *src, unsigned long len);
-
-/* The movntdqa instructions used for memcpy-from-wc require 16-byte alignment,
- * as well as SSE4.1 support. i915_memcpy_from_wc() will report if it cannot
- * perform the operation. To check beforehand, pass in the parameters to
- * to i915_can_memcpy_from_wc() - since we only care about the low 4 bits,
- * you only need to pass in the minor offsets, page-aligned pointers are
- * always valid.
- *
- * For just checking for SSE4.1, in the foreknowledge that the future use
- * will be correctly aligned, just use i915_has_memcpy_from_wc().
- */
-#define i915_can_memcpy_from_wc(dst, src, len) \
-	i915_memcpy_from_wc((void *)((unsigned long)(dst) | (unsigned long)(src) | (len)), NULL, 0)
-
-#define i915_has_memcpy_from_wc() \
-	i915_memcpy_from_wc(NULL, NULL, 0)
-
-#endif /* __I915_MEMCPY_H__ */
drivers/gpu/drm/i915/selftests/intel_memory_region.c +4 -3
index c85d516b85cd..6bb399e9be78 100644
@@ -6,6 +6,8 @@
 #include <linux/prime_numbers.h>
 #include <linux/sort.h>
 
+#include <drm/drm_memcpy.h>
+
 #include "../i915_selftest.h"
 
 #include "mock_drm.h"
@@ -20,7 +22,6 @@
 #include "gem/selftests/mock_context.h"
 #include "gt/intel_engine_user.h"
 #include "gt/intel_gt.h"
-#include "i915_memcpy.h"
 #include "selftests/igt_flush_test.h"
 #include "selftests/i915_random.h"
 
@@ -901,7 +902,7 @@ static inline void igt_memcpy(void *dst, const void *src, size_t size)
 
 static inline void igt_memcpy_from_wc(void *dst, const void *src, size_t size)
 {
-	i915_memcpy_from_wc(dst, src, size);
+	drm_memcpy_from_wc(dst, src, size);
 }
 
 static int _perf_memcpy(struct intel_memory_region *src_mr,
@@ -925,7 +926,7 @@ static int _perf_memcpy(struct intel_memory_region *src_mr,
 		{
 			"memcpy_from_wc",
 			igt_memcpy_from_wc,
-			!i915_has_memcpy_from_wc(),
+			!drm_has_memcpy_from_wc(),
 		},
 	};
 	struct drm_i915_gem_object *src, *dst;
include/drm/drm_memcpy.h new file +47 -0

47 lines not on this page · raw message

Signature · 6 lines
2.31.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
author reviewer
#6 lore raw

PATCH v3 6/12 drm/ttm: Add a generic TTM memcpy move for page-based iomem

In reply to Thomas Hellström

The internal ttm_bo_util memcpy uses ioremap functionality, and while it probably might be possible to use it for copying in- and out of sglist represented io memory, using io_mem_reserve() / io_mem_free() callbacks, that would cause problems with fault().
Instead, implement a method mapping page-by-page using kmap_local() semantics. As an additional benefit we then avoid the occasional global TLB flushes of ioremap() and consuming ioremap space, elimination of a critical point of failure and with a slight change of semantics we could also push the memcpy out async for testing and async driver development purposes.

A special linear iomem iterator is introduced internally to mimic the old ioremap behaviour for code-paths that can't immediately be ported over. This adds to the code size and should be considered a temporary solution.

Looking at the code we have a lot of checks for iomap tagged pointers. Ideally we should extend the core memremap functions to also accept uncached memory and kmap_local functionality. Then we could strip a lot of code.

---
v3:
- Split up in various TTM files and addressed review comments by Christian König. Tested and fixed legacy iomap memcpy path on i915.

---
 drivers/gpu/drm/ttm/ttm_bo_util.c  | 278 ++++++++++-------------------
 drivers/gpu/drm/ttm/ttm_module.c   |  35 ++++
 drivers/gpu/drm/ttm/ttm_resource.c | 166 +++++++++++++++++
 drivers/gpu/drm/ttm/ttm_tt.c       |  42 +++++
 include/drm/ttm/ttm_bo_driver.h    |  28 +++
 include/drm/ttm/ttm_caching.h      |   2 +
 include/drm/ttm/ttm_kmap_iter.h    |  61 +++++++
 include/drm/ttm/ttm_resource.h     |  61 +++++++
 include/drm/ttm/ttm_tt.h           |  16 ++
 9 files changed, 508 insertions(+), 181 deletions(-)
 create mode 100644 include/drm/ttm/ttm_kmap_iter.h
drivers/gpu/drm/ttm/ttm_bo_util.c +97 -181
index ae8b61460724..912cbe8e60a2 100644
@@ -72,190 +72,126 @@ void ttm_mem_io_free(struct ttm_device *bdev,
 	mem->bus.addr = NULL;
 }
 
-static int ttm_resource_ioremap(struct ttm_device *bdev,
-			       struct ttm_resource *mem,
-			       void **virtual)
+/**
+ * ttm_move_memcpy - Helper to perform a memcpy ttm move operation.
+ * @bo: The struct ttm_buffer_object.
+ * @new_mem: The struct ttm_resource we're moving to (copy destination).
+ * @new_iter: A struct ttm_kmap_iter representing the destination resource.
+ * @src_iter: A struct ttm_kmap_iter representing the source resource.
+ *
+ * This function is intended to be able to move out async under a
+ * dma-fence if desired.
+ */
+void ttm_move_memcpy(struct ttm_buffer_object *bo,
+		     struct ttm_resource *dst_mem,
+		     struct ttm_kmap_iter *dst_iter,
+		     struct ttm_kmap_iter *src_iter)
 {
-	int ret;
-	void *addr;
-
-	*virtual = NULL;
-	ret = ttm_mem_io_reserve(bdev, mem);
-	if (ret || !mem->bus.is_iomem)
-		return ret;
+	const struct ttm_kmap_iter_ops *dst_ops = dst_iter->ops;
+	const struct ttm_kmap_iter_ops *src_ops = src_iter->ops;
+	struct ttm_tt *ttm = bo->ttm;
+	struct dma_buf_map src_map, dst_map;
+	pgoff_t i;
 
-	if (mem->bus.addr) {
-		addr = mem->bus.addr;
-	} else {
-		size_t bus_size = (size_t)mem->num_pages << PAGE_SHIFT;
+	/* Single TTM move. NOP */
+	if (dst_ops->maps_tt && src_ops->maps_tt)
+		return;
 
-		if (mem->bus.caching == ttm_write_combined)
-			addr = ioremap_wc(mem->bus.offset, bus_size);
-#ifdef CONFIG_X86
-		else if (mem->bus.caching == ttm_cached)
-			addr = ioremap_cache(mem->bus.offset, bus_size);
-#endif
-		else
-			addr = ioremap(mem->bus.offset, bus_size);
-		if (!addr) {
-			ttm_mem_io_free(bdev, mem);
-			return -ENOMEM;
+	/* Don't move nonexistent data. Clear destination instead. */
+	if (src_ops->maps_tt && (!ttm || !ttm_tt_is_populated(ttm))) {
+		if (ttm && !(ttm->page_flags & TTM_PAGE_FLAG_ZERO_ALLOC))
+			return;
+
+		for (i = 0; i < dst_mem->num_pages; ++i) {
+			dst_ops->map_local(dst_iter, &dst_map, i);
+			if (dst_map.is_iomem)
+				memset_io(dst_map.vaddr_iomem, 0, PAGE_SIZE);
+			else
+				memset(dst_map.vaddr, 0, PAGE_SIZE);
+			if (dst_ops->unmap_local)
+				dst_ops->unmap_local(dst_iter, &dst_map);
 		}
+		return;
 	}
-	*virtual = addr;
-	return 0;
-}
-
-static void ttm_resource_iounmap(struct ttm_device *bdev,
-				struct ttm_resource *mem,
-				void *virtual)
-{
-	if (virtual && mem->bus.addr == NULL)
-		iounmap(virtual);
-	ttm_mem_io_free(bdev, mem);
-}
-
-static int ttm_copy_io_page(void *dst, void *src, unsigned long page)
-{
-	uint32_t *dstP =
-	    (uint32_t *) ((unsigned long)dst + (page << PAGE_SHIFT));
-	uint32_t *srcP =
-	    (uint32_t *) ((unsigned long)src + (page << PAGE_SHIFT));
-
-	int i;
-	for (i = 0; i < PAGE_SIZE / sizeof(uint32_t); ++i)
-		iowrite32(ioread32(srcP++), dstP++);
-	return 0;
-}
-
-static int ttm_copy_io_ttm_page(struct ttm_tt *ttm, void *src,
-				unsigned long page,
-				pgprot_t prot)
-{
-	struct page *d = ttm->pages[page];
-	void *dst;
-
-	if (!d)
-		return -ENOMEM;
-
-	src = (void *)((unsigned long)src + (page << PAGE_SHIFT));
-	dst = kmap_atomic_prot(d, prot);
-	if (!dst)
-		return -ENOMEM;
-
-	memcpy_fromio(dst, src, PAGE_SIZE);
-
-	kunmap_atomic(dst);
-
-	return 0;
-}
-
-static int ttm_copy_ttm_io_page(struct ttm_tt *ttm, void *dst,
-				unsigned long page,
-				pgprot_t prot)
-{
-	struct page *s = ttm->pages[page];
-	void *src;
-
-	if (!s)
-		return -ENOMEM;
-
-	dst = (void *)((unsigned long)dst + (page << PAGE_SHIFT));
-	src = kmap_atomic_prot(s, prot);
-	if (!src)
-		return -ENOMEM;
 
-	memcpy_toio(dst, src, PAGE_SIZE);
-
-	kunmap_atomic(src);
+	for (i = 0; i < dst_mem->num_pages; ++i) {
+		dst_ops->map_local(dst_iter, &dst_map, i);
+		src_ops->map_local(src_iter, &src_map, i);
+
+		if (!src_map.is_iomem && !dst_map.is_iomem) {
+			memcpy(dst_map.vaddr, src_map.vaddr, PAGE_SIZE);
+		} else if (!src_map.is_iomem) {
+			dma_buf_map_memcpy_to(&dst_map, src_map.vaddr,
+					      PAGE_SIZE);
+		} else if (!dst_map.is_iomem) {
+			memcpy_fromio(dst_map.vaddr, src_map.vaddr_iomem,
+				      PAGE_SIZE);
+		} else {
+			int j;
+			u32 __iomem *src = src_map.vaddr_iomem;
+			u32 __iomem *dst = dst_map.vaddr_iomem;
 
-	return 0;
+			for (j = 0; j < (PAGE_SIZE >> 2); ++j)
+				iowrite32(ioread32(src++), dst++);
+		}
+		if (src_ops->unmap_local)
+			src_ops->unmap_local(src_iter, &src_map);
+		if (dst_ops->unmap_local)
+			dst_ops->unmap_local(dst_iter, &dst_map);
+	}
 }
+EXPORT_SYMBOL(ttm_move_memcpy);
 
 int ttm_bo_move_memcpy(struct ttm_buffer_object *bo,
 		       struct ttm_operation_ctx *ctx,
-		       struct ttm_resource *new_mem)
+		       struct ttm_resource *dst_mem)
 {
 	struct ttm_device *bdev = bo->bdev;
-	struct ttm_resource_manager *man = ttm_manager_type(bdev, new_mem->mem_type);
+	struct ttm_resource_manager *dst_man =
+		ttm_manager_type(bo->bdev, dst_mem->mem_type);
 	struct ttm_tt *ttm = bo->ttm;
-	struct ttm_resource *old_mem = &bo->mem;
-	struct ttm_resource old_copy = *old_mem;
-	void *old_iomap;
-	void *new_iomap;
+	struct ttm_resource *src_mem = &bo->mem;
+	struct ttm_resource_manager *src_man =
+		ttm_manager_type(bdev, src_mem->mem_type);
+	struct ttm_resource src_copy = *src_mem;
+	union {
+		struct ttm_kmap_iter_tt tt;
+		struct ttm_kmap_iter_linear_io io;
+	} _dst_iter, _src_iter;
+	struct ttm_kmap_iter *dst_iter, *src_iter;
 	int ret;
-	unsigned long i;
 
-	ret = ttm_bo_wait_ctx(bo, ctx);
-	if (ret)
-		return ret;
-
-	ret = ttm_resource_ioremap(bdev, old_mem, &old_iomap);
-	if (ret)
-		return ret;
-	ret = ttm_resource_ioremap(bdev, new_mem, &new_iomap);
-	if (ret)
-		goto out;
-
-	/*
-	 * Single TTM move. NOP.
-	 */
-	if (old_iomap == NULL && new_iomap == NULL)
-		goto out2;
-
-	/*
-	 * Don't move nonexistent data. Clear destination instead.
-	 */
-	if (old_iomap == NULL &&
-	    (ttm == NULL || (!ttm_tt_is_populated(ttm) &&
-			     !(ttm->page_flags & TTM_PAGE_FLAG_SWAPPED)))) {
-		memset_io(new_iomap, 0, new_mem->num_pages*PAGE_SIZE);
-		goto out2;
-	}
-
-	/*
-	 * TTM might be null for moves within the same region.
-	 */
-	if (ttm) {
+	if (ttm && ((ttm->page_flags & TTM_PAGE_FLAG_SWAPPED) ||
+		    dst_man->use_tt)) {
 		ret = ttm_tt_populate(bdev, ttm, ctx);
 		if (ret)
-			goto out1;
+			return ret;
 	}
 
-	for (i = 0; i < new_mem->num_pages; ++i) {
-		if (old_iomap == NULL) {
-			pgprot_t prot = ttm_io_prot(bo, old_mem, PAGE_KERNEL);
-			ret = ttm_copy_ttm_io_page(ttm, new_iomap, i,
-						   prot);
-		} else if (new_iomap == NULL) {
-			pgprot_t prot = ttm_io_prot(bo, new_mem, PAGE_KERNEL);
-			ret = ttm_copy_io_ttm_page(ttm, old_iomap, i,
-						   prot);
-		} else {
-			ret = ttm_copy_io_page(new_iomap, old_iomap, i);
-		}
-		if (ret)
-			goto out1;
+	dst_iter = ttm_kmap_iter_linear_io_init(&_dst_iter.io, bdev, dst_mem);
+	if (PTR_ERR(dst_iter) == -EINVAL && dst_man->use_tt)
+		dst_iter = ttm_kmap_iter_tt_init(&_dst_iter.tt, bo->ttm);
+	if (IS_ERR(dst_iter))
+		return PTR_ERR(dst_iter);
+
+	src_iter = ttm_kmap_iter_linear_io_init(&_src_iter.io, bdev, src_mem);
+	if (PTR_ERR(src_iter) == -EINVAL && src_man->use_tt)
+		src_iter = ttm_kmap_iter_tt_init(&_src_iter.tt, bo->ttm);
+	if (IS_ERR(src_iter)) {
+		ret = PTR_ERR(src_iter);
+		goto out_src_iter;
 	}
-	mb();
-out2:
-	old_copy = *old_mem;
 
-	ttm_bo_assign_mem(bo, new_mem);
-
-	if (!man->use_tt)
-		ttm_bo_tt_destroy(bo);
+	ttm_move_memcpy(bo, dst_mem, dst_iter, src_iter);
+	src_copy = *src_mem;
+	ttm_bo_move_sync_cleanup(bo, dst_mem);
 
-out1:
-	ttm_resource_iounmap(bdev, old_mem, new_iomap);
-out:
-	ttm_resource_iounmap(bdev, &old_copy, old_iomap);
+	if (!src_iter->ops->maps_tt)
+		ttm_kmap_iter_linear_io_fini(&_src_iter.io, bdev, &src_copy);
+out_src_iter:
+	if (!dst_iter->ops->maps_tt)
+		ttm_kmap_iter_linear_io_fini(&_dst_iter.io, bdev, dst_mem);
 
-	/*
-	 * On error, keep the mm node!
-	 */
-	if (!ret)
-		ttm_resource_free(bo, &old_copy);
 	return ret;
 }
 EXPORT_SYMBOL(ttm_bo_move_memcpy);
@@ -336,27 +272,7 @@ pgprot_t ttm_io_prot(struct ttm_buffer_object *bo, struct ttm_resource *res,
 	man = ttm_manager_type(bo->bdev, res->mem_type);
 	caching = man->use_tt ? bo->ttm->caching : res->bus.caching;
 
-	/* Cached mappings need no adjustment */
-	if (caching == ttm_cached)
-		return tmp;
-
-#if defined(__i386__) || defined(__x86_64__)
-	if (caching == ttm_write_combined)
-		tmp = pgprot_writecombine(tmp);
-	else if (boot_cpu_data.x86 > 3)
-		tmp = pgprot_noncached(tmp);
-#endif
-#if defined(__ia64__) || defined(__arm__) || defined(__aarch64__) || \
-    defined(__powerpc__) || defined(__mips__)
-	if (caching == ttm_write_combined)
-		tmp = pgprot_writecombine(tmp);
-	else
-		tmp = pgprot_noncached(tmp);
-#endif
-#if defined(__sparc__)
-	tmp = pgprot_noncached(tmp);
-#endif
-	return tmp;
+	return ttm_prot_from_caching(caching, tmp);
 }
 EXPORT_SYMBOL(ttm_io_prot);
 
drivers/gpu/drm/ttm/ttm_module.c +35 -0
index 56b0efdba1a9..997c458f68a9 100644
@@ -31,12 +31,47 @@
  */
 #include <linux/module.h>
 #include <linux/device.h>
+#include <linux/pgtable.h>
 #include <linux/sched.h>
 #include <linux/debugfs.h>
 #include <drm/drm_sysfs.h>
+#include <drm/ttm/ttm_caching.h>
 
 #include "ttm_module.h"
 
+/**
+ * ttm_prot_from_caching - Modify the page protection according to the
+ * ttm cacing mode
+ * @caching: The ttm caching mode
+ * @tmp: The original page protection
+ *
+ * Return: The modified page protection
+ */
+pgprot_t ttm_prot_from_caching(enum ttm_caching caching, pgprot_t tmp)
+{
+	/* Cached mappings need no adjustment */
+	if (caching == ttm_cached)
+		return tmp;
+
+#if defined(__i386__) || defined(__x86_64__)
+	if (caching == ttm_write_combined)
+		tmp = pgprot_writecombine(tmp);
+	else if (boot_cpu_data.x86 > 3)
+		tmp = pgprot_noncached(tmp);
+#endif
+#if defined(__ia64__) || defined(__arm__) || defined(__aarch64__) || \
+	defined(__powerpc__) || defined(__mips__)
+	if (caching == ttm_write_combined)
+		tmp = pgprot_writecombine(tmp);
+	else
+		tmp = pgprot_noncached(tmp);
+#endif
+#if defined(__sparc__)
+	tmp = pgprot_noncached(tmp);
+#endif
+	return tmp;
+}
+
 struct dentry *ttm_debugfs_root;
 
 static int __init ttm_init(void)
drivers/gpu/drm/ttm/ttm_resource.c +166 -0

175 lines not on this page · raw message

drivers/gpu/drm/ttm/ttm_tt.c +42 -0

45 lines not on this page · raw message

include/drm/ttm/ttm_bo_driver.h +28 -0
index dbccac957f8f..61fa648ee9e4 100644
@@ -40,6 +40,7 @@
 #include <drm/ttm/ttm_device.h>
 
 #include "ttm_bo_api.h"
+#include "ttm_kmap_iter.h"
 #include "ttm_placement.h"
 #include "ttm_tt.h"
 #include "ttm_pool.h"
@@ -272,6 +273,23 @@ int ttm_bo_move_accel_cleanup(struct ttm_buffer_object *bo,
 			      bool pipeline,
 			      struct ttm_resource *new_mem);
 
+/**
+ * ttm_bo_move_accel_cleanup.
+ *
+ * @bo: A pointer to a struct ttm_buffer_object.
+ * @new_mem: struct ttm_resource indicating where to move.
+ *
+ * Special case of ttm_bo_move_accel_cleanup where the bo is guaranteed
+ * by the caller to be idle. Typically used after memcpy buffer moves.
+ */
+static inline void ttm_bo_move_sync_cleanup(struct ttm_buffer_object *bo,
+					    struct ttm_resource *new_mem)
+{
+	int ret = ttm_bo_move_accel_cleanup(bo, NULL, true, false, new_mem);
+
+	WARN_ON(ret);
+}
+
 /**
  * ttm_bo_pipeline_gutting.
  *
@@ -332,4 +350,14 @@ int ttm_range_man_init(struct ttm_device *bdev,
 int ttm_range_man_fini(struct ttm_device *bdev,
 		       unsigned type);
 
+void ttm_move_memcpy(struct ttm_buffer_object *bo,
+		     struct ttm_resource *new_mem,
+		     struct ttm_kmap_iter *dst_iter,
+		     struct ttm_kmap_iter *src_iter);
+
+struct ttm_kmap_iter *
+ttm_kmap_iter_iomap_init(struct ttm_kmap_iter_iomap *iter_io,
+			 struct io_mapping *iomap,
+			 struct sg_table *st,
+			 resource_size_t start);
 #endif
include/drm/ttm/ttm_caching.h +2 -0
index a0b4a49fa432..3c9dd65f5aaf 100644
@@ -33,4 +33,6 @@ enum ttm_caching {
 	ttm_cached
 };
 
+pgprot_t ttm_prot_from_caching(enum ttm_caching caching, pgprot_t tmp);
+
 #endif
include/drm/ttm/ttm_kmap_iter.h new file +61 -0

61 lines not on this page · raw message

include/drm/ttm/ttm_resource.h +61 -0

86 lines not on this page · raw message

include/drm/ttm/ttm_tt.h +16 -0
index 134d09ef7766..3102059db726 100644
@@ -29,6 +29,7 @@
 
 #include <linux/types.h>
 #include <drm/ttm/ttm_caching.h>
+#include <drm/ttm/ttm_kmap_iter.h>
 
 struct ttm_bo_device;
 struct ttm_tt;
@@ -69,6 +70,18 @@ struct ttm_tt {
 	enum ttm_caching caching;
 };
 
+/**
+ * struct ttm_kmap_iter_tt - Specialization of a mappig iterator for a tt.
+ * @base: Embedded struct ttm_kmap_iter providing the usage interface
+ * @tt: Cached struct ttm_tt.
+ * @prot: Cached page protection for mapping.
+ */
+struct ttm_kmap_iter_tt {
+	struct ttm_kmap_iter base;
+	struct ttm_tt *tt;
+	pgprot_t prot;
+};
+
 static inline bool ttm_tt_is_populated(struct ttm_tt *tt)
 {
 	return tt->page_flags & TTM_PAGE_FLAG_PRIV_POPULATED;
@@ -159,6 +172,9 @@ void ttm_tt_unpopulate(struct ttm_device *bdev, struct ttm_tt *ttm);
 
 void ttm_tt_mgr_init(unsigned long num_pages, unsigned long num_dma32_pages);
 
+struct ttm_kmap_iter *ttm_kmap_iter_tt_init(struct ttm_kmap_iter_tt *iter_tt,
+					    struct ttm_tt *tt);
+
 #if IS_ENABLED(CONFIG_AGP)
 #include <linux/agp_backend.h>
 
Signature · 6 lines
2.31.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
author reviewer
#18 lore raw

PATCH v3 8/12 drm/ttm: Use drm_memcpy_from_wc_dbm for TTM bo moves

In reply to Thomas Hellström

Use fast wc memcpy for reading out of wc memory for TTM bo moves.

Cc Dave Airlie <[email protected]>
Cc Christian König <[email protected]>
Cc Daniel Vetter <[email protected]>
---
 drivers/gpu/drm/ttm/ttm_bo_util.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/gpu/drm/ttm/ttm_bo_util.c +8 -1
index 912cbe8e60a2..4a7d3d672f9a 100644
@@ -31,6 +31,7 @@
 
 #include <drm/ttm/ttm_bo_driver.h>
 #include <drm/ttm/ttm_placement.h>
+#include <drm/drm_memcpy.h>
 #include <drm/drm_vma_manager.h>
 #include <linux/dma-buf-map.h>
 #include <linux/io.h>
@@ -91,6 +92,7 @@ void ttm_move_memcpy(struct ttm_buffer_object *bo,
 	const struct ttm_kmap_iter_ops *src_ops = src_iter->ops;
 	struct ttm_tt *ttm = bo->ttm;
 	struct dma_buf_map src_map, dst_map;
+	bool wc_memcpy;
 	pgoff_t i;
 
 	/* Single TTM move. NOP */
@@ -114,11 +116,16 @@ void ttm_move_memcpy(struct ttm_buffer_object *bo,
 		return;
 	}
 
+	wc_memcpy = ((!src_ops->maps_tt || ttm->caching != ttm_cached) &&
+		     drm_has_memcpy_from_wc());
+
 	for (i = 0; i < dst_mem->num_pages; ++i) {
 		dst_ops->map_local(dst_iter, &dst_map, i);
 		src_ops->map_local(src_iter, &src_map, i);
 
-		if (!src_map.is_iomem && !dst_map.is_iomem) {
+		if (wc_memcpy) {
+			drm_memcpy_from_wc_dbm(&dst_map, &src_map, PAGE_SIZE);
+		} else if (!src_map.is_iomem && !dst_map.is_iomem) {
 			memcpy(dst_map.vaddr, src_map.vaddr, PAGE_SIZE);
 		} else if (!src_map.is_iomem) {
 			dma_buf_map_memcpy_to(&dst_map, src_map.vaddr,
Signature · 1 line
2.31.1
author reviewer
#22 lore raw

PATCH v3 9/12 drm/ttm: Document and optimize ttm_bo_pipeline_gutting()

In reply to Thomas Hellström

If the bo is idle when calling ttm_bo_pipeline_gutting(), we unnecessarily create a ghost object and push it out to delayed destroy.
Fix this by adding a path for idle, and document the function.

Also avoid having the bo end up in a bad state vulnerable to user-space triggered kernel BUGs if the call to ttm_tt_create() fails.

Finally reuse ttm_bo_pipeline_gutting() in ttm_bo_evict().

---
 drivers/gpu/drm/ttm/ttm_bo.c      | 20 ++++++------
 drivers/gpu/drm/ttm/ttm_bo_util.c | 52 ++++++++++++++++++++++++++++---
 drivers/gpu/drm/ttm/ttm_tt.c      |  5 +++
 include/drm/ttm/ttm_tt.h          | 10 ++++++
 4 files changed, 73 insertions(+), 14 deletions(-)
drivers/gpu/drm/ttm/ttm_bo.c +10 -10
index ca1b098b6a56..a8fa3375b8aa 100644
@@ -501,10 +501,15 @@ static int ttm_bo_evict(struct ttm_buffer_object *bo,
 	bdev->funcs->evict_flags(bo, &placement);
 
 	if (!placement.num_placement && !placement.num_busy_placement) {
-		ttm_bo_wait(bo, false, false);
+		ret = ttm_bo_wait(bo, true, false);
+		if (ret)
+			return ret;
 
-		ttm_bo_cleanup_memtype_use(bo);
-		return ttm_tt_create(bo, false);
+		/*
+		 * Since we've already synced, this frees backing store
+		 * immediately.
+		 */
+		return ttm_bo_pipeline_gutting(bo);
 	}
 
 	ret = ttm_bo_mem_space(bo, &placement, &evict_mem, ctx);
@@ -974,13 +979,8 @@ int ttm_bo_validate(struct ttm_buffer_object *bo,
 	/*
 	 * Remove the backing store if no placement is given.
 	 */
-	if (!placement->num_placement && !placement->num_busy_placement) {
-		ret = ttm_bo_pipeline_gutting(bo);
-		if (ret)
-			return ret;
-
-		return ttm_tt_create(bo, false);
-	}
+	if (!placement->num_placement && !placement->num_busy_placement)
+		return ttm_bo_pipeline_gutting(bo);
 
 	/*
 	 * Check whether we need to move buffer.
drivers/gpu/drm/ttm/ttm_bo_util.c +48 -4

74 lines not on this page · raw message

drivers/gpu/drm/ttm/ttm_tt.c +5 -0
index 0e41227116b1..913b330a234b 100644
@@ -134,6 +134,11 @@ void ttm_tt_destroy_common(struct ttm_device *bdev, struct ttm_tt *ttm)
 }
 EXPORT_SYMBOL(ttm_tt_destroy_common);
 
+void ttm_tt_mark_for_clear(struct ttm_tt *ttm)
+{
+	ttm->page_flags |= TTM_PAGE_FLAG_ZERO_ALLOC;
+}
+
 void ttm_tt_destroy(struct ttm_device *bdev, struct ttm_tt *ttm)
 {
 	bdev->funcs->ttm_tt_destroy(bdev, ttm);
include/drm/ttm/ttm_tt.h +10 -0
index 3102059db726..daa9c4cf48bb 100644
@@ -170,6 +170,16 @@ int ttm_tt_populate(struct ttm_device *bdev, struct ttm_tt *ttm, struct ttm_oper
  */
 void ttm_tt_unpopulate(struct ttm_device *bdev, struct ttm_tt *ttm);
 
+/**
+ * ttm_tt_mark_for_clear - Mark pages for clearing on populate.
+ *
+ * @ttm: Pointer to the ttm_tt structure
+ *
+ * Marks pages for clearing so that the next time the page vector is
+ * populated, the pages will be cleared.
+ */
+void ttm_tt_mark_for_clear(struct ttm_tt *ttm);
+
 void ttm_tt_mgr_init(unsigned long num_pages, unsigned long num_dma32_pages);
 
 struct ttm_kmap_iter *ttm_kmap_iter_tt_init(struct ttm_kmap_iter_tt *iter_tt,
Signature · 6 lines
2.31.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
author reviewer
#25 lore raw

PATCH v3 10/12 drm/ttm, drm/amdgpu: Allow the driver some control over swapping

In reply to Thomas Hellström

We are calling the eviction_valuable driver callback at eviction time to determine whether we actually can evict a buffer object.
The upcoming i915 TTM backend needs the same functionality for swapout, and that might actually be beneficial to other drivers as well.

Add an eviction_valuable call also in the swapout path. Try to keep the current behaviour for all drivers by returning true if the buffer object is already in the TTM_PL_SYSTEM placement. We change behaviour for the case where a buffer object is in a TT backed placement when swapped out, in which case the drivers normal eviction_valuable path is run.

Finally make sure we don't try to swapout a bo that was recently purged and therefore unpopulated.

---
v3:
- Don't export ttm_tt_unpopulate
- Fix confusion reading the locked pointer instead of the value pointed to in ttm_bo_evict_swapout_allowable (Reported by Maarten Lankhorst)

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c |  4 +++
 drivers/gpu/drm/ttm/ttm_bo.c            | 43 ++++++++++++++++---------
 drivers/gpu/drm/ttm/ttm_tt.c            |  3 ++
 3 files changed, 34 insertions(+), 16 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +4 -0
index 8c7ec09eb1a4..d5a9d7a88315 100644
@@ -1399,6 +1399,10 @@ static bool amdgpu_ttm_bo_eviction_valuable(struct ttm_buffer_object *bo,
 	struct dma_fence *f;
 	int i;
 
+	/* Swapout? */
+	if (bo->mem.mem_type == TTM_PL_SYSTEM)
+		return true;
+
 	if (bo->type == ttm_bo_type_kernel &&
 	    !amdgpu_vm_evictable(ttm_to_amdgpu_bo(bo)))
 		return false;
drivers/gpu/drm/ttm/ttm_bo.c +27 -16

88 lines not on this page · raw message

drivers/gpu/drm/ttm/ttm_tt.c +3 -0
index 913b330a234b..d9793cbb6d13 100644
@@ -263,6 +263,9 @@ int ttm_tt_swapout(struct ttm_device *bdev, struct ttm_tt *ttm,
 	struct page *to_page;
 	int i, ret;
 
+	if (!ttm_tt_is_populated(ttm))
+		return 0;
+
 	swap_storage = shmem_file_setup("ttm swap", size, 0);
 	if (IS_ERR(swap_storage)) {
 		pr_err("Failed allocating swap storage\n");
Signature · 1 line
2.31.1
author reviewer
#26 lore raw

PATCH v3 11/12 drm/i915/ttm: Introduce a TTM i915 gem object backend

In reply to Thomas Hellström

Most logical place to introduce TTM buffer objects is as an i915 gem object backend. We need to add some ops to account for added functionality like delayed delete and LRU list manipulation.

Initially we support only LMEM and SYSTEM memory, but SYSTEM (which in this case means evicted LMEM objects) is not visible to i915 GEM yet. The plan is to move the i915 gem system region over to the TTM system memory type in upcoming patches.

We set up GPU bindings directly both from LMEM and from the system region, as there is no need to use the legacy TTM_TT memory type. We reserve that for future porting of GGTT bindings to TTM.

Remove the old lmem backend.

---
v2:
- Break out needed TTM functionality to a separate patch (Reported by Christian König).
- Fix an unhandled error (Reported by Matthew Auld and Maarten Lankhorst)
- Remove a stray leftover sg_table allocation (Reported by Matthew Auld)
- Use ttm_tt_unpopulate() rather than ttm_tt_destroy() in the purge path as some TTM functionality relies on having a ttm_tt present for !is_iomem.
v3:
- Use ttm_bo_type_device for userspace visible objects so that TTM can allocate an address space offset for mmap'ing.
- Fix up the destruction path (Reported by Matthew Auld)
- Use ttm_bo_validate() for purging (Reported by Christian König)
- Create ttm_tts write-combined as they are currently for eviction only and we want to maintain consistent write-combined caching for bos that are not in system only. (Suggested by Daniel Vetter)
- Make struct ttm_placements static.
- Add the ttm device funcs/ops to i915_gem_ttm.h for the region code.
- Rename new->dst and old->src. Check for swapin in the move callback.

---
 drivers/gpu/drm/i915/Makefile                 |   1 +
 drivers/gpu/drm/i915/gem/i915_gem_create.c    |   9 +-
 drivers/gpu/drm/i915/gem/i915_gem_lmem.c      |  84 ---
 drivers/gpu/drm/i915/gem/i915_gem_lmem.h      |   5 -
 drivers/gpu/drm/i915/gem/i915_gem_object.c    | 125 +++--
 drivers/gpu/drm/i915/gem/i915_gem_object.h    |   9 +
 .../gpu/drm/i915/gem/i915_gem_object_types.h  |  27 +-
 drivers/gpu/drm/i915/gem/i915_gem_region.c    |   6 +-
 drivers/gpu/drm/i915/gem/i915_gem_ttm.c       | 531 ++++++++++++++++++
 drivers/gpu/drm/i915/gem/i915_gem_ttm.h       |  50 ++
 drivers/gpu/drm/i915/gt/intel_region_lmem.c   |   3 +-
 drivers/gpu/drm/i915/i915_gem.c               |   5 +-
 drivers/gpu/drm/i915/intel_memory_region.c    |   1 -
 drivers/gpu/drm/i915/intel_memory_region.h    |   1 -
 drivers/gpu/drm/i915/intel_region_ttm.c       |   6 +-
 drivers/gpu/drm/i915/intel_region_ttm.h       |   8 +-
 16 files changed, 717 insertions(+), 154 deletions(-)
 create mode 100644 drivers/gpu/drm/i915/gem/i915_gem_ttm.c
 create mode 100644 drivers/gpu/drm/i915/gem/i915_gem_ttm.h
drivers/gpu/drm/i915/Makefile +1 -0
index 998606b7f49f..2022e763f8f7 100644
@@ -154,6 +154,7 @@ gem-y += \
 	gem/i915_gem_stolen.o \
 	gem/i915_gem_throttle.o \
 	gem/i915_gem_tiling.o \
+	gem/i915_gem_ttm.o \
 	gem/i915_gem_userptr.o \
 	gem/i915_gem_wait.o \
 	gem/i915_gemfs.o
drivers/gpu/drm/i915/gem/i915_gem_create.c +3 -6
index 548ddf39d853..93bf63bbaff1 100644
@@ -85,13 +85,10 @@ i915_gem_setup(struct drm_i915_gem_object *obj, u64 size)
 		return -E2BIG;
 
 	/*
-	 * For now resort to CPU based clearing for device local-memory, in the
-	 * near future this will use the blitter engine for accelerated, GPU
-	 * based clearing.
+	 * I915_BO_ALLOC_USER will make sure the object is cleared before
+	 * any user access.
 	 */
-	flags = 0;
-	if (mr->type == INTEL_MEMORY_LOCAL)
-		flags = I915_BO_ALLOC_CPU_CLEAR;
+	flags = I915_BO_ALLOC_USER;
 
 	ret = mr->ops->init_object(mr, obj, size, flags);
 	if (ret)
drivers/gpu/drm/i915/gem/i915_gem_lmem.c +0 -84

97 lines not on this page · raw message

drivers/gpu/drm/i915/gem/i915_gem_lmem.h +0 -5
index fac6bc5a5ebb..ea76fd11ccb0 100644
@@ -26,9 +26,4 @@ i915_gem_object_create_lmem(struct drm_i915_private *i915,
 			    resource_size_t size,
 			    unsigned int flags);
 
-int __i915_gem_lmem_object_init(struct intel_memory_region *mem,
-				struct drm_i915_gem_object *obj,
-				resource_size_t size,
-				unsigned int flags);
-
 #endif /* !__I915_GEM_LMEM_H */
drivers/gpu/drm/i915/gem/i915_gem_object.c +86 -39

164 lines not on this page · raw message

drivers/gpu/drm/i915/gem/i915_gem_object.h +9 -0
index 2ebd79537aea..ae5930e307d5 100644
@@ -200,6 +200,9 @@ static inline bool i915_gem_object_trylock(struct drm_i915_gem_object *obj)
 
 static inline void i915_gem_object_unlock(struct drm_i915_gem_object *obj)
 {
+	if (obj->ops->adjust_lru)
+		obj->ops->adjust_lru(obj);
+
 	dma_resv_unlock(obj->base.resv);
 }
 
@@ -587,6 +590,12 @@ int i915_gem_object_read_from_page(struct drm_i915_gem_object *obj, u64 offset,
 
 bool i915_gem_object_is_shmem(const struct drm_i915_gem_object *obj);
 
+void __i915_gem_free_object_rcu(struct rcu_head *head);
+
+void __i915_gem_free_object(struct drm_i915_gem_object *obj);
+
+bool i915_gem_object_evictable(struct drm_i915_gem_object *obj);
+
 #ifdef CONFIG_MMU_NOTIFIER
 static inline bool
 i915_gem_object_is_userptr(struct drm_i915_gem_object *obj)
drivers/gpu/drm/i915/gem/i915_gem_object_types.h +24 -3
index d047ea126029..68313474e6a6 100644
@@ -63,6 +63,20 @@ struct drm_i915_gem_object_ops {
 		      const struct drm_i915_gem_pwrite *arg);
 
 	int (*dmabuf_export)(struct drm_i915_gem_object *obj);
+
+	/**
+	 * adjust_lru - notify that the madvise value was updated
+	 * @obj: The gem object
+	 *
+	 * The madvise value may have been updated, or object was recently
+	 * referenced so act accordingly (Perhaps changing an LRU list etc).
+	 */
+	void (*adjust_lru)(struct drm_i915_gem_object *obj);
+
+	/**
+	 * delayed_free - Override the default delayed free implementation
+	 */
+	void (*delayed_free)(struct drm_i915_gem_object *obj);
 	void (*release)(struct drm_i915_gem_object *obj);
 
 	const char *name; /* friendly name for debug, e.g. lockdep classes */
@@ -187,12 +201,14 @@ struct drm_i915_gem_object {
 #define I915_BO_ALLOC_VOLATILE   BIT(1)
 #define I915_BO_ALLOC_STRUCT_PAGE BIT(2)
 #define I915_BO_ALLOC_CPU_CLEAR  BIT(3)
+#define I915_BO_ALLOC_USER       BIT(4)
 #define I915_BO_ALLOC_FLAGS (I915_BO_ALLOC_CONTIGUOUS | \
 			     I915_BO_ALLOC_VOLATILE | \
 			     I915_BO_ALLOC_STRUCT_PAGE | \
-			     I915_BO_ALLOC_CPU_CLEAR)
-#define I915_BO_READONLY         BIT(4)
-#define I915_TILING_QUIRK_BIT    5 /* unknown swizzling; do not release! */
+			     I915_BO_ALLOC_CPU_CLEAR | \
+			     I915_BO_ALLOC_USER)
+#define I915_BO_READONLY         BIT(5)
+#define I915_TILING_QUIRK_BIT    6 /* unknown swizzling; do not release! */
 
 	/*
 	 * Is the object to be mapped as read-only to the GPU
@@ -310,6 +326,11 @@ struct drm_i915_gem_object {
 		bool dirty:1;
 	} mm;
 
+	struct {
+		struct sg_table *cached_io_st;
+		bool created:1;
+	} ttm;
+
 	/** Record of address bit 17 of each page at last unbind. */
 	unsigned long *bit_17;
 
drivers/gpu/drm/i915/gem/i915_gem_region.c +1 -5
index f25e6646c5b7..d1f1840540dd 100644
@@ -18,11 +18,7 @@ void i915_gem_object_init_memory_region(struct drm_i915_gem_object *obj,
 
 	mutex_lock(&mem->objects.lock);
 
-	if (obj->flags & I915_BO_ALLOC_VOLATILE)
-		list_add(&obj->mm.region_link, &mem->objects.purgeable);
-	else
-		list_add(&obj->mm.region_link, &mem->objects.list);
-
+	list_add(&obj->mm.region_link, &mem->objects.list);
 	mutex_unlock(&mem->objects.lock);
 }
 
drivers/gpu/drm/i915/gem/i915_gem_ttm.c new file +531 -0

531 lines not on this page · raw message

drivers/gpu/drm/i915/gem/i915_gem_ttm.h new file +50 -0

50 lines not on this page · raw message

drivers/gpu/drm/i915/gt/intel_region_lmem.c +2 -1
index f7366b054f8e..4ae1f717a94c 100644
@@ -9,6 +9,7 @@
 #include "intel_region_ttm.h"
 #include "gem/i915_gem_lmem.h"
 #include "gem/i915_gem_region.h"
+#include "gem/i915_gem_ttm.h"
 #include "intel_region_lmem.h"
 
 static int init_fake_lmem_bar(struct intel_memory_region *mem)
@@ -107,7 +108,7 @@ region_lmem_init(struct intel_memory_region *mem)
 static const struct intel_memory_region_ops intel_region_lmem_ops = {
 	.init = region_lmem_init,
 	.release = region_lmem_release,
-	.init_object = __i915_gem_lmem_object_init,
+	.init_object = __i915_gem_ttm_object_init,
 };
 
 struct intel_memory_region *
drivers/gpu/drm/i915/i915_gem.c +4 -1
index 180f6e9107d4..350283ab9a83 100644
@@ -1005,8 +1005,11 @@ i915_gem_madvise_ioctl(struct drm_device *dev, void *data,
 		}
 	}
 
-	if (obj->mm.madv != __I915_MADV_PURGED)
+	if (obj->mm.madv != __I915_MADV_PURGED) {
 		obj->mm.madv = args->madv;
+		if (obj->ops->adjust_lru)
+			obj->ops->adjust_lru(obj);
+	}
 
 	if (i915_gem_object_has_pages(obj)) {
 		struct list_head *list;
drivers/gpu/drm/i915/intel_memory_region.c +0 -1
index 4092cc987679..bd27e897d4d0 100644
@@ -149,7 +149,6 @@ intel_memory_region_create(struct drm_i915_private *i915,
 
 	mutex_init(&mem->objects.lock);
 	INIT_LIST_HEAD(&mem->objects.list);
-	INIT_LIST_HEAD(&mem->objects.purgeable);
 	INIT_LIST_HEAD(&mem->reserved);
 
 	mutex_init(&mem->mm_lock);
drivers/gpu/drm/i915/intel_memory_region.h +0 -1
index e69cde13daf2..7b5fa97c0b59 100644
@@ -100,7 +100,6 @@ struct intel_memory_region {
 	struct {
 		struct mutex lock; /* Protects access to objects */
 		struct list_head list;
-		struct list_head purgeable;
 	} objects;
 
 	size_t chunk_size;
drivers/gpu/drm/i915/intel_region_ttm.c +2 -4
index 69c98298326b..52352dbe047d 100644
@@ -10,6 +10,7 @@
 
 #include "intel_region_ttm.h"
 
+#include "gem/i915_gem_ttm.h" /* For the funcs/ops export only */
 /**
  * DOC: TTM support structure
  *
@@ -31,9 +32,6 @@ struct intel_region_ttm_private {
 	I915_SELFTEST_DECLARE(struct drm_vma_offset_manager mock_vma);
 };
 
-/* A Zero-initialized driver for now. We don't have a TTM backend yet. */
-static struct ttm_device_funcs i915_ttm_bo_driver;
-
 static struct intel_region_ttm_private *to_ttm_private(void *data)
 {
 	return (struct intel_region_ttm_private *)data;
@@ -219,6 +217,7 @@ struct sg_table *intel_region_ttm_node_to_st(struct intel_memory_region *mem,
 	return i915_sg_from_mm_node(node, mem->region.start);
 }
 
+#ifdef CONFIG_DRM_I915_SELFTEST
 /**
  * intel_region_ttm_node_alloc - Allocate memory resources from a region
  * @mem: The memory region,
@@ -267,7 +266,6 @@ void *intel_region_ttm_node_alloc(struct intel_memory_region *mem,
 	return ret ? ERR_PTR(ret) : res.mm_node;
 }
 
-#ifdef CONFIG_DRM_I915_SELFTEST
 /**
  * intel_region_ttm_init_mock - Initialize a mock memory region for TTM.
  * @mem: The region to initialize.
drivers/gpu/drm/i915/intel_region_ttm.h +4 -4
index 23c6c2d4b1cd..a08998c3a3d8 100644
@@ -23,14 +23,14 @@ void intel_region_ttm_fini(struct intel_memory_region *mem);
 struct sg_table *intel_region_ttm_node_to_st(struct intel_memory_region *mem,
 					     void *node);
 
-void *intel_region_ttm_node_alloc(struct intel_memory_region *mem,
-				  resource_size_t size,
-				  unsigned int flags);
-
 void intel_region_ttm_node_free(struct intel_memory_region *mem,
 				void *node);
 
 #ifdef CONFIG_DRM_I915_SELFTEST
+void *intel_region_ttm_node_alloc(struct intel_memory_region *mem,
+				  resource_size_t size,
+				  unsigned int flags);
+
 int intel_region_ttm_init_mock(struct intel_memory_region *mem);
 
 void intel_region_ttm_fini_mock(struct intel_memory_region *mem);
Signature · 1 line
2.31.1
author reviewer
#27 lore raw

PATCH v3 12/12 drm/i915/lmem: Verify checks for lmem residency

In reply to Thomas Hellström

Since objects can be migrated or evicted when not pinned or locked, update the checks for lmem residency or future residency so that the value returned is not immediately stale.

---
v2: Simplify i915_gem_object_migratable() (Reported by Mattew Auld)

---
 drivers/gpu/drm/i915/display/intel_display.c |  2 +-
 drivers/gpu/drm/i915/gem/i915_gem_lmem.c     | 42 +++++++++++++++++++-
 drivers/gpu/drm/i915/gem/i915_gem_object.c   | 18 +++++++++
 drivers/gpu/drm/i915/gem/i915_gem_object.h   |  4 ++
 4 files changed, 64 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/display/intel_display.c +1 -1
index de1f13d203b5..b95def2d5af3 100644
@@ -11615,7 +11615,7 @@ intel_user_framebuffer_create(struct drm_device *dev,
 
 	/* object is backed with LMEM for discrete */
 	i915 = to_i915(obj->base.dev);
-	if (HAS_LMEM(i915) && !i915_gem_object_is_lmem(obj)) {
+	if (HAS_LMEM(i915) && !i915_gem_object_validates_to_lmem(obj)) {
 		/* object is "remote", not in local memory */
 		i915_gem_object_put(obj);
 		return ERR_PTR(-EREMOTE);
drivers/gpu/drm/i915/gem/i915_gem_lmem.c +41 -1

51 lines not on this page · raw message

drivers/gpu/drm/i915/gem/i915_gem_object.c +18 -0
index df2b4e6b9bcc..c8bb6fb1dba3 100644
@@ -458,6 +458,24 @@ bool i915_gem_object_evictable(struct drm_i915_gem_object *obj)
 	return pin_count == 0;
 }
 
+/**
+ * i915_gem_object_migratable - Whether the object is migratable out of the
+ * current region.
+ * @obj: Pointer to the object.
+ *
+ * Return: Whether the object is allowed to be resident in other
+ * regions than the current while pages are present.
+ */
+bool i915_gem_object_migratable(struct drm_i915_gem_object *obj)
+{
+	struct intel_memory_region *mr = READ_ONCE(obj->mm.region);
+
+	if (!mr)
+		return false;
+
+	return obj->mm.n_placements > 1;
+}
+
 void i915_gem_init__objects(struct drm_i915_private *i915)
 {
 	INIT_WORK(&i915->mm.free_work, __i915_gem_free_work);
drivers/gpu/drm/i915/gem/i915_gem_object.h +4 -0
index ae5930e307d5..a3ad8cf4eefd 100644
@@ -596,6 +596,10 @@ void __i915_gem_free_object(struct drm_i915_gem_object *obj);
 
 bool i915_gem_object_evictable(struct drm_i915_gem_object *obj);
 
+bool i915_gem_object_migratable(struct drm_i915_gem_object *obj);
+
+bool i915_gem_object_validates_to_lmem(struct drm_i915_gem_object *obj);
+
 #ifdef CONFIG_MMU_NOTIFIER
 static inline bool
 i915_gem_object_is_userptr(struct drm_i915_gem_object *obj)
Signature · 6 lines
2.31.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#28 lore raw

[Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for drm/i915: Move LMEM (VRAM) management over to TTM (rev3)

In reply to Thomas Hellström
== Series Details ==

== Series Details ==

Series: drm/i915: Move LMEM (VRAM) management over to TTM (rev3)
URL : https://patchwork.freedesktop.org/series/90022/
State : warning

== Summary ==

$ dim checkpatch origin/drm-tip
1bf39c8cd594 drm/i915: Untangle the vma pages_mutex
650181499469 drm/i915: Don't free shared locks while shared
e0cd60e6a0d3 drm/i915: Fix i915_sg_page_sizes to record dma segments rather than physical pages
4f895c63cc41 drm/i915/ttm Initialize the ttm device and memory managers
-:480: WARNING:FILE_PATH_CHANGES: added, moved or deleted file(s), does MAINTAINERS need updating?
#480: 
deleted file mode 100644

total: 0 errors, 1 warnings, 0 checks, 1578 lines checked fbd2191fd254 drm/i915/ttm: Embed a ttm buffer object in the i915 gem object 67e58a6747b0 drm/ttm: Add a generic TTM memcpy move for page-based iomem -:383: CHECK:ARCH_DEFINES: architecture specific defines should be avoided #383: FILE: drivers/gpu/drm/ttm/ttm_module.c:56:
+#if defined(__i386__) || defined(__x86_64__)

-:699: WARNING:FILE_PATH_CHANGES: added, moved or deleted file(s), does MAINTAINERS need updating? #699:
new file mode 100644

total: 0 errors, 1 warnings, 1 checks, 812 lines checked
fa7a4164f770 drm, drm/i915: Move the memcpy_from_wc functionality to core drm -:54: WARNING:FILE_PATH_CHANGES: added, moved or deleted file(s), does MAINTAINERS need updating? #54:
rename from drivers/gpu/drm/i915/i915_memcpy.c

total: 0 errors, 1 warnings, 0 checks, 410 lines checked
533ef0911275 drm/ttm: Use drm_memcpy_from_wc_dbm for TTM bo moves
655c519cecf1 drm/ttm: Document and optimize ttm_bo_pipeline_gutting()
2f38e8009d75 drm/ttm, drm/amdgpu: Allow the driver some control over swapping 356c346230f1 drm/i915/ttm: Introduce a TTM i915 gem object backend
-:449: WARNING:FILE_PATH_CHANGES: added, moved or deleted file(s), does MAINTAINERS need updating? #449:
new file mode 100644

total: 0 errors, 1 warnings, 0 checks, 1024 lines checked 7ced8a9675bb drm/i915/lmem: Verify checks for lmem residency

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#29 lore raw

[Intel-gfx] ✗ Fi.CI.SPARSE: warning for drm/i915: Move LMEM (VRAM) management over to TTM (rev3)

In reply to Thomas Hellström
== Series Details ==

== Series Details ==

Series: drm/i915: Move LMEM (VRAM) management over to TTM (rev3)
URL : https://patchwork.freedesktop.org/series/90022/
State : warning

== Summary ==

$ dim sparse --fast origin/drm-tip
Sparse version: v0.6.2
Fast mode used, each commit won't be checked separately.
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+./drivers/gpu/drm/amd/amdgpu/../amdgpu/amdgv_sriovmsg.h:316:49: error: static assertion failed: "amd_sriov_msg_pf2vf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c:1345:25: error: incompatible types in comparison expression (different address spaces):
+drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c:1345:25:    struct dma_fence *
+drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c:1345:25:    struct dma_fence [noderef] __rcu *
+drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c:1346:17: error: incompatible types in comparison expression (different address spaces):
+drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c:1346:17:    struct dma_fence *
+drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c:1346:17:    struct dma_fence [noderef] __rcu *
+drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c:1405:17: error: incompatible types in comparison expression (different address spaces):
+drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c:1405:17:    struct dma_fence *
+drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c:1405:17:    struct dma_fence [noderef] __rcu *
+drivers/gpu/drm/amd/amdgpu/amdgpu_device.c:293:16: error: incompatible types in comparison expression (different type sizes):
+drivers/gpu/drm/amd/amdgpu/amdgpu_device.c:293:16:    unsigned long *
+drivers/gpu/drm/amd/amdgpu/amdgpu_device.c:293:16:    unsigned long long *
+drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c:275:25: error: incompatible types in comparison expression (different address spaces):
+drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c:275:25:    struct dma_fence *
+drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c:275:25:    struct dma_fence [noderef] __rcu *
+drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c:276:17: error: incompatible types in comparison expression (different address spaces):
+drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c:276:17:    struct dma_fence *
+drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c:276:17:    struct dma_fence [noderef] __rcu *
+drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c:330:17: error: incompatible types in comparison expression (different address spaces):
+drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c:330:17:    struct dma_fence *
+drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c:330:17:    struct dma_fence [noderef] __rcu *
+drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.h:90:56: error: marked inline, but without a definition
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_info must be 1 KB"
+drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h:312:49: error: static assertion failed: "amd_sriov_msg_vf2pf_in

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#30 lore raw

[Intel-gfx] ✓ Fi.CI.BAT: success for drm/i915: Move LMEM (VRAM) management over to TTM (rev3)

In reply to Thomas Hellström
== Series Details ==

== Series Details ==

Series: drm/i915: Move LMEM (VRAM) management over to TTM (rev3)
URL : https://patchwork.freedesktop.org/series/90022/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_10120 -> Patchwork_20170
====================================================

Summary
-------

**SUCCESS**

No regressions found.

External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/index.html

Known issues
------------

Here are the changes found in Patchwork_20170 that come from known issues:

### IGT changes ###

#### Possible fixes ####

* igt@kms_chamelium@dp-crc-fast:
- fi-kbl-7500u: [FAIL][1] ([i915#1372]) -> [PASS][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/fi-kbl-7500u/igt@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/fi-kbl-7500u/igt@[email protected]

* igt@kms_frontbuffer_tracking@basic:
- fi-icl-u2: [FAIL][3] ([i915#49]) -> [PASS][4]
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/fi-icl-u2/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/fi-icl-u2/igt@[email protected]

#### Warnings ####

* igt@i915_selftest@live@execlists:
- fi-bsw-kefka: [INCOMPLETE][5] ([i915#2782] / [i915#2940] / [i915#3462]) -> [DMESG-FAIL][6] ([i915#3462])
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/fi-bsw-kefka/igt@i915_selftest@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/fi-bsw-kefka/igt@i915_selftest@[email protected]

* igt@runner@aborted:
- fi-skl-6600u: [FAIL][7] ([i915#1436] / [i915#3363]) -> [FAIL][8] ([i915#1436] / [i915#2426] / [i915#3363])
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/fi-skl-6600u/igt@[email protected]
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/fi-skl-6600u/igt@[email protected]
- fi-bdw-5557u: [FAIL][9] ([i915#3462]) -> [FAIL][10] ([i915#1602] / [i915#2029])
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/fi-bdw-5557u/igt@[email protected]
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/fi-bdw-5557u/igt@[email protected]
- fi-kbl-soraka: [FAIL][11] ([i915#1436] / [i915#2426] / [i915#3363]) -> [FAIL][12] ([i915#1436] / [i915#3363])
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/fi-kbl-soraka/igt@[email protected]
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/fi-kbl-soraka/igt@[email protected]
- fi-cml-u2: [FAIL][13] ([i915#2082] / [i915#2426] / [i915#3363] / [i915#3462]) -> [FAIL][14] ([i915#3363] / [i915#3462])
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/fi-cml-u2/igt@[email protected]
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/fi-cml-u2/igt@[email protected]

{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).

[fdo#109285]: https://bugs.freedesktop.org/show_bug.cgi?id=109285
[fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
[i915#1072]: https://gitlab.freedesktop.org/drm/intel/issues/1072
[i915#1372]: https://gitlab.freedesktop.org/drm/intel/issues/1372
[i915#1436]: https://gitlab.freedesktop.org/drm/intel/issues/1436
[i915#1602]: https://gitlab.freedesktop.org/drm/intel/issues/1602
[i915#2029]: https://gitlab.freedesktop.org/drm/intel/issues/2029
[i915#2082]: https://gitlab.freedesktop.org/drm/intel/issues/2082
[i915#2190]: https://gitlab.freedesktop.org/drm/intel/issues/2190
[i915#2426]: https://gitlab.freedesktop.org/drm/intel/issues/2426
[i915#2782]: https://gitlab.freedesktop.org/drm/intel/issues/2782
[i915#2932]: https://gitlab.freedesktop.org/drm/intel/issues/2932
[i915#2940]: https://gitlab.freedesktop.org/drm/intel/issues/2940
[i915#2966]: https://gitlab.freedesktop.org/drm/intel/issues/2966
[i915#3012]: https://gitlab.freedesktop.org/drm/intel/issues/3012
[i915#3276]: https://gitlab.freedesktop.org/drm/intel/issues/3276
[i915#3277]: https://gitlab.freedesktop.org/drm/intel/issues/3277
[i915#3282]: https://gitlab.freedesktop.org/drm/intel/issues/3282
[i915#3283]: https://gitlab.freedesktop.org/drm/intel/issues/3283
[i915#3291]: https://gitlab.freedesktop.org/drm/intel/issues/3291
[i915#3301]: https://gitlab.freedesktop.org/drm/intel/issues/3301
[i915#3363]: https://gitlab.freedesktop.org/drm/intel/issues/3363
[i915#3462]: https://gitlab.freedesktop.org/drm/intel/issues/3462
[i915#49]: https://gitlab.freedesktop.org/drm/intel/issues/49
[i915#533]: https://gitlab.freedesktop.org/drm/intel/issues/533

Participating hosts (41 -> 39)
------------------------------

Additional (1): fi-rkl-11500t
Missing (3): fi-bsw-cyan fi-bdw-samus fi-hsw-4200u

Build changes
-------------

* Linux: CI_DRM_10120 -> Patchwork_20170

CI-20190529: 20190529
CI_DRM_10120: 9221d50d353487d2e10226318d89027037255621 @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_6091: a7016bde81f6e6ee9f2ded3c091c56766a6adc46 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools
Patchwork_20170: 7ced8a9675bbb379bd5287d6bc210960c31f34a6 @ git://anongit.freedesktop.org/gfx-ci/linux

== Linux commits ==

7ced8a9675bb drm/i915/lmem: Verify checks for lmem residency
356c346230f1 drm/i915/ttm: Introduce a TTM i915 gem object backend
2f38e8009d75 drm/ttm, drm/amdgpu: Allow the driver some control over swapping 655c519cecf1 drm/ttm: Document and optimize ttm_bo_pipeline_gutting() 533ef0911275 drm/ttm: Use drm_memcpy_from_wc_dbm for TTM bo moves
fa7a4164f770 drm, drm/i915: Move the memcpy_from_wc functionality to core drm 67e58a6747b0 drm/ttm: Add a generic TTM memcpy move for page-based iomem fbd2191fd254 drm/i915/ttm: Embed a ttm buffer object in the i915 gem object 4f895c63cc41 drm/i915/ttm Initialize the ttm device and memory managers e0cd60e6a0d3 drm/i915: Fix i915_sg_page_sizes to record dma segments rather than physical pages 650181499469 drm/i915: Don't free shared locks while shared
1bf39c8cd594 drm/i915: Untangle the vma pages_mutex

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/index.html

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#31 lore raw

[Intel-gfx] ✗ Fi.CI.IGT: failure for drm/i915: Move LMEM (VRAM) management over to TTM (rev3)

In reply to Thomas Hellström
== Series Details ==

== Series Details ==

Series: drm/i915: Move LMEM (VRAM) management over to TTM (rev3)
URL : https://patchwork.freedesktop.org/series/90022/
State : failure

== Summary ==

CI Bug Log - changes from CI_DRM_10120_full -> Patchwork_20170_full
====================================================

Summary
-------

**FAILURE**

Serious unknown changes coming with Patchwork_20170_full absolutely need to be verified manually.

If you think the reported changes have nothing to do with the changes introduced in Patchwork_20170_full, please notify your bug team to allow them to document this new failure mode, which will reduce false positives in CI.

Possible new issues
-------------------

Here are the unknown changes that may have been introduced in Patchwork_20170_full:

### Piglit changes ###

#### Possible regressions ####

* spec@arb_texture_barrier@arb_texture_barrier-blending-in-shader 512 42 1 8 3:
- pig-glk-j5005: NOTRUN -> [INCOMPLETE][1] +1 similar issue
[1]: None

Known issues
------------

Here are the changes found in Patchwork_20170_full that come from known issues:

### IGT changes ###

#### Issues hit ####

* igt@gem_create@create-clear:
- shard-glk: [PASS][2] -> [DMESG-WARN][3] ([i915#118] / [i915#95])
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-glk1/igt@[email protected]
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-glk9/igt@[email protected]

* igt@gem_create@create-massive:
- shard-snb: NOTRUN -> [DMESG-WARN][4] ([i915#3002])
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-snb7/igt@[email protected]
- shard-skl: NOTRUN -> [DMESG-WARN][5] ([i915#3002])
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl7/igt@[email protected]

* igt@gem_ctx_isolation@preservation-s3@vecs0:
- shard-kbl: [PASS][6] -> [DMESG-WARN][7] ([i915#180]) +5 similar issues
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl4/igt@gem_ctx_isolation@[email protected]
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl1/igt@gem_ctx_isolation@[email protected]

* igt@gem_ctx_persistence@legacy-engines-mixed-process:
- shard-snb: NOTRUN -> [SKIP][8] ([fdo#109271] / [i915#1099]) +4 similar issues
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-snb2/igt@[email protected]

* igt@gem_ctx_ringsize@active@bcs0:
- shard-skl: [PASS][9] -> [INCOMPLETE][10] ([i915#3316])
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-skl9/igt@gem_ctx_ringsize@[email protected]
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl9/igt@gem_ctx_ringsize@[email protected]

* igt@gem_exec_fair@basic-pace-solo@rcs0:
- shard-glk: [PASS][11] -> [FAIL][12] ([i915#2842])
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-glk2/igt@gem_exec_fair@[email protected]
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-glk6/igt@gem_exec_fair@[email protected]

* igt@gem_exec_fair@basic-pace@bcs0:
- shard-tglb: [PASS][13] -> [FAIL][14] ([i915#2842]) +1 similar issue
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-tglb5/igt@gem_exec_fair@[email protected]
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-tglb5/igt@gem_exec_fair@[email protected]

* igt@gem_exec_reloc@basic-wide-active@bcs0:
- shard-apl: NOTRUN -> [FAIL][15] ([i915#2389]) +3 similar issues
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl3/igt@gem_exec_reloc@[email protected]

* igt@gem_mmap_gtt@big-copy-odd:
- shard-glk: [PASS][16] -> [FAIL][17] ([i915#307])
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-glk2/igt@[email protected]
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-glk2/igt@[email protected]

* igt@gem_mmap_gtt@fault-concurrent-x:
- shard-apl: NOTRUN -> [INCOMPLETE][18] ([i915#3468])
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl1/igt@[email protected]

* igt@gem_mmap_gtt@medium-copy-xy:
- shard-snb: [PASS][19] -> [INCOMPLETE][20] ([i915#2055] / [i915#2502] / [i915#3468])
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-snb2/igt@[email protected]
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-snb2/igt@[email protected]

* igt@gem_userptr_blits@set-cache-level:
- shard-apl: NOTRUN -> [FAIL][21] ([i915#3324])
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl3/igt@[email protected]

* igt@gen9_exec_parse@bb-large:
- shard-apl: NOTRUN -> [FAIL][22] ([i915#3296])
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl1/igt@[email protected]

* igt@i915_pm_lpsp@kms-lpsp@kms-lpsp-dp:
- shard-apl: NOTRUN -> [SKIP][23] ([fdo#109271] / [i915#1937])
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl1/igt@i915_pm_lpsp@[email protected]

* igt@i915_selftest@live@execlists:
- shard-apl: NOTRUN -> [DMESG-FAIL][24] ([i915#3462])
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl3/igt@i915_selftest@[email protected]

* igt@kms_big_joiner@invalid-modeset:
- shard-kbl: NOTRUN -> [SKIP][25] ([fdo#109271] / [i915#2705])
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl4/igt@[email protected]
- shard-apl: NOTRUN -> [SKIP][26] ([fdo#109271] / [i915#2705])
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl7/igt@[email protected]

* igt@kms_chamelium@dp-hpd:
- shard-skl: NOTRUN -> [SKIP][27] ([fdo#109271] / [fdo#111827]) +1 similar issue
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl7/igt@[email protected]

* igt@kms_chamelium@dp-mode-timings:
- shard-apl: NOTRUN -> [SKIP][28] ([fdo#109271] / [fdo#111827]) +26 similar issues
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl3/igt@[email protected]

* igt@kms_color@pipe-c-ctm-0-5:
- shard-skl: [PASS][29] -> [DMESG-WARN][30] ([i915#1982])
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-skl4/igt@[email protected]
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl4/igt@[email protected]

* igt@kms_color_chamelium@pipe-a-ctm-0-25:
- shard-snb: NOTRUN -> [SKIP][31] ([fdo#109271] / [fdo#111827]) +19 similar issues
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-snb5/igt@[email protected]

* igt@kms_color_chamelium@pipe-c-ctm-0-5:
- shard-kbl: NOTRUN -> [SKIP][32] ([fdo#109271] / [fdo#111827]) +4 similar issues
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl4/igt@[email protected]

* igt@kms_color_chamelium@pipe-invalid-ctm-matrix-sizes:
- shard-glk: NOTRUN -> [SKIP][33] ([fdo#109271] / [fdo#111827]) +2 similar issues
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-glk5/igt@[email protected]

* igt@kms_content_protection@legacy:
- shard-apl: NOTRUN -> [TIMEOUT][34] ([i915#1319])
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl7/igt@[email protected]

* igt@kms_cursor_crc@pipe-b-cursor-32x32-onscreen:
- shard-skl: NOTRUN -> [SKIP][35] ([fdo#109271]) +35 similar issues
[35]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl7/igt@[email protected]

* igt@kms_cursor_crc@pipe-d-cursor-256x256-onscreen:
- shard-kbl: NOTRUN -> [SKIP][36] ([fdo#109271]) +65 similar issues
[36]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl4/igt@[email protected]

* igt@kms_cursor_crc@pipe-d-cursor-64x21-random:
- shard-glk: NOTRUN -> [SKIP][37] ([fdo#109271]) +32 similar issues
[37]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-glk5/igt@[email protected]

* igt@kms_cursor_edge_walk@pipe-d-128x128-right-edge:
- shard-snb: NOTRUN -> [SKIP][38] ([fdo#109271]) +355 similar issues
[38]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-snb2/igt@[email protected]

* igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size:
- shard-skl: [PASS][39] -> [FAIL][40] ([i915#2346] / [i915#533])
[39]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-skl3/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size.html
[40]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl9/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size.html

* igt@kms_flip@flip-vs-absolute-wf_vblank-interruptible@b-edp1:
- shard-skl: [PASS][41] -> [FAIL][42] ([i915#2122]) +1 similar issue
[41]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-skl2/igt@kms_flip@[email protected]
[42]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl2/igt@kms_flip@[email protected]

* igt@kms_flip@flip-vs-expired-vblank@a-edp1:
- shard-skl: [PASS][43] -> [FAIL][44] ([i915#79])
[43]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-skl7/igt@kms_flip@[email protected]
[44]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl4/igt@kms_flip@[email protected]

* igt@kms_flip_scaled_crc@flip-32bpp-ytile-to-32bpp-ytilegen12rcccs:
- shard-skl: NOTRUN -> [SKIP][45] ([fdo#109271] / [i915#2672])
[45]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl7/igt@kms_flip_scaled_crc@flip-32bpp-ytile-to-32bpp-ytilegen12rcccs.html

* igt@kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile:
- shard-apl: NOTRUN -> [SKIP][46] ([fdo#109271] / [i915#2642])
[46]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl7/igt@[email protected]

* igt@kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs:
- shard-kbl: NOTRUN -> [SKIP][47] ([fdo#109271] / [i915#2672])
[47]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl7/igt@[email protected]
- shard-glk: NOTRUN -> [SKIP][48] ([fdo#109271] / [i915#2672])
[48]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-glk5/igt@[email protected]

* igt@kms_hdr@bpc-switch-dpms:
- shard-skl: NOTRUN -> [FAIL][49] ([i915#1188])
[49]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl7/igt@[email protected]

* igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-d:
- shard-apl: NOTRUN -> [SKIP][50] ([fdo#109271] / [i915#533]) +4 similar issues
[50]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl7/igt@[email protected]
- shard-kbl: NOTRUN -> [SKIP][51] ([fdo#109271] / [i915#533])
[51]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl4/igt@[email protected]

* igt@kms_plane_alpha_blend@pipe-a-alpha-transparent-fb:
- shard-apl: NOTRUN -> [FAIL][52] ([i915#265])
[52]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl3/igt@[email protected]

* igt@kms_plane_alpha_blend@pipe-b-alpha-transparent-fb:
- shard-skl: NOTRUN -> [FAIL][53] ([i915#265])
[53]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl7/igt@[email protected]

* igt@kms_plane_alpha_blend@pipe-b-constant-alpha-max:
- shard-apl: NOTRUN -> [FAIL][54] ([fdo#108145] / [i915#265]) +2 similar issues
[54]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl7/igt@[email protected]

* igt@kms_plane_alpha_blend@pipe-b-coverage-7efc:
- shard-skl: [PASS][55] -> [FAIL][56] ([fdo#108145] / [i915#265])
[55]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-skl3/igt@[email protected]
[56]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl3/igt@[email protected]

* igt@kms_plane_alpha_blend@pipe-c-alpha-7efc:
- shard-kbl: NOTRUN -> [FAIL][57] ([fdo#108145] / [i915#265])
[57]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl7/igt@[email protected]

* igt@kms_plane_scaling@scaler-with-clipping-clamping@pipe-c-scaler-with-clipping-clamping:
- shard-apl: NOTRUN -> [SKIP][58] ([fdo#109271] / [i915#2733])
[58]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl7/igt@kms_plane_scaling@scaler-with-clipping-clamping@pipe-c-scaler-with-clipping-clamping.html

* igt@kms_psr2_sf@overlay-plane-update-sf-dmg-area-3:
- shard-apl: NOTRUN -> [SKIP][59] ([fdo#109271] / [i915#658]) +6 similar issues
[59]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl7/igt@[email protected]

* igt@kms_psr2_sf@overlay-primary-update-sf-dmg-area-2:
- shard-skl: NOTRUN -> [SKIP][60] ([fdo#109271] / [i915#658])
[60]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl3/igt@[email protected]

* igt@kms_psr2_sf@overlay-primary-update-sf-dmg-area-3:
- shard-kbl: NOTRUN -> [SKIP][61] ([fdo#109271] / [i915#658]) +1 similar issue
[61]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl7/igt@[email protected]
- shard-glk: NOTRUN -> [SKIP][62] ([fdo#109271] / [i915#658])
[62]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-glk5/igt@[email protected]

* igt@kms_writeback@writeback-check-output:
- shard-apl: NOTRUN -> [SKIP][63] ([fdo#109271] / [i915#2437])
[63]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl3/igt@[email protected]
- shard-skl: NOTRUN -> [SKIP][64] ([fdo#109271] / [i915#2437])
[64]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl3/igt@[email protected]

* igt@kms_writeback@writeback-pixel-formats:
- shard-glk: NOTRUN -> [SKIP][65] ([fdo#109271] / [i915#2437])
[65]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-glk5/igt@[email protected]
- shard-kbl: NOTRUN -> [SKIP][66] ([fdo#109271] / [i915#2437])
[66]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl7/igt@[email protected]

* igt@perf@polling-parameterized:
- shard-skl: [PASS][67] -> [FAIL][68] ([i915#1542])
[67]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-skl3/igt@[email protected]
[68]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl9/igt@[email protected]

* igt@prime_nv_api@i915_nv_reimport_twice_check_flink_name:
- shard-apl: NOTRUN -> [SKIP][69] ([fdo#109271]) +254 similar issues
[69]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl1/igt@prime_nv_api@i915_nv_reimport_twice_check_flink_name.html

* igt@sysfs_clients@fair-0:
- shard-kbl: NOTRUN -> [SKIP][70] ([fdo#109271] / [i915#2994])
[70]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl4/igt@[email protected]

* igt@sysfs_clients@fair-7:
- shard-apl: NOTRUN -> [SKIP][71] ([fdo#109271] / [i915#2994]) +6 similar issues
[71]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl7/igt@[email protected]

* igt@sysfs_heartbeat_interval@mixed@vcs0:
- shard-skl: [PASS][72] -> [FAIL][73] ([i915#1731])
[72]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-skl3/igt@sysfs_heartbeat_interval@[email protected]
[73]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl9/igt@sysfs_heartbeat_interval@[email protected]

* igt@sysfs_timeslice_duration@timeout@vecs0:
- shard-tglb: [PASS][74] -> [FAIL][75] ([i915#1755])
[74]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-tglb5/igt@sysfs_timeslice_duration@[email protected]
[75]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-tglb8/igt@sysfs_timeslice_duration@[email protected]

#### Possible fixes ####

* igt@gem_exec_fair@basic-deadline:
- shard-glk: [FAIL][76] ([i915#2846]) -> [PASS][77]
[76]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-glk1/igt@[email protected]
[77]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-glk9/igt@[email protected]

* igt@gem_exec_fair@basic-none-share@rcs0:
- shard-tglb: [FAIL][78] ([i915#2842]) -> [PASS][79]
[78]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-tglb1/igt@gem_exec_fair@[email protected]
[79]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-tglb8/igt@gem_exec_fair@[email protected]

* igt@gem_exec_fair@basic-pace-share@rcs0:
- shard-glk: [FAIL][80] ([i915#2842]) -> [PASS][81] +1 similar issue
[80]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-glk4/igt@gem_exec_fair@[email protected]
[81]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-glk2/igt@gem_exec_fair@[email protected]

* igt@gen9_exec_parse@allowed-single:
- shard-skl: [DMESG-WARN][82] ([i915#1436] / [i915#716]) -> [PASS][83]
[82]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-skl4/igt@[email protected]
[83]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl7/igt@[email protected]

* igt@i915_suspend@sysfs-reader:
- shard-kbl: [INCOMPLETE][84] ([i915#155]) -> [PASS][85]
[84]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl2/igt@[email protected]
[85]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl7/igt@[email protected]

* igt@kms_cursor_legacy@2x-long-flip-vs-cursor-legacy:
- shard-glk: [FAIL][86] ([i915#72]) -> [PASS][87]
[86]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-glk7/igt@[email protected]
[87]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-glk5/igt@[email protected]

* igt@kms_draw_crc@draw-method-xrgb2101010-pwrite-ytiled:
- shard-skl: [DMESG-WARN][88] ([i915#1982]) -> [PASS][89]
[88]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-skl3/igt@[email protected]
[89]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl3/igt@[email protected]

* igt@kms_fbcon_fbt@fbc-suspend:
- shard-kbl: [INCOMPLETE][90] ([i915#155] / [i915#180] / [i915#636]) -> [PASS][91]
[90]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl1/igt@[email protected]
[91]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl4/igt@[email protected]

* igt@kms_flip@flip-vs-expired-vblank-interruptible@c-hdmi-a2:
- shard-glk: [FAIL][92] ([i915#79]) -> [PASS][93]
[92]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-glk9/igt@kms_flip@[email protected]
[93]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-glk6/igt@kms_flip@[email protected]

* igt@kms_flip@flip-vs-suspend-interruptible@a-dp1:
- shard-kbl: [DMESG-WARN][94] ([i915#180]) -> [PASS][95]
[94]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl4/igt@kms_flip@[email protected]
[95]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl2/igt@kms_flip@[email protected]
- shard-apl: [DMESG-WARN][96] ([i915#180]) -> [PASS][97]
[96]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-apl1/igt@kms_flip@[email protected]
[97]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl3/igt@kms_flip@[email protected]

* igt@kms_pipe_crc_basic@suspend-read-crc-pipe-b:
- shard-skl: [INCOMPLETE][98] ([i915#198]) -> [PASS][99]
[98]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-skl2/igt@[email protected]
[99]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl7/igt@[email protected]

#### Warnings ####

* igt@gem_mmap_gtt@cpuset-basic-small-copy-xy:
- shard-snb: [INCOMPLETE][100] ([i915#2055]) -> [INCOMPLETE][101] ([i915#2055] / [i915#3468])
[100]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-snb7/igt@[email protected]
[101]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-snb7/igt@[email protected]
- shard-skl: [INCOMPLETE][102] ([i915#198] / [i915#3468]) -> [INCOMPLETE][103] ([i915#198])
[102]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-skl2/igt@[email protected]
[103]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl2/igt@[email protected]

* igt@gem_mmap_gtt@fault-concurrent-x:
- shard-skl: [INCOMPLETE][104] ([i915#198]) -> [INCOMPLETE][105] ([i915#198] / [i915#3468])
[104]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-skl7/igt@[email protected]
[105]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl4/igt@[email protected]

* igt@runner@aborted:
- shard-kbl: ([FAIL][106], [FAIL][107], [FAIL][108], [FAIL][109], [FAIL][110], [FAIL][111], [FAIL][112], [FAIL][113], [FAIL][114], [FAIL][115], [FAIL][116], [FAIL][117], [FAIL][118], [FAIL][119], [FAIL][120], [FAIL][121]) ([fdo#109271] / [i915#1436] / [i915#180] / [i915#1814] / [i915#2292] / [i915#2722] / [i915#3002] / [i915#3363] / [i915#92]) -> ([FAIL][122], [FAIL][123], [FAIL][124], [FAIL][125], [FAIL][126], [FAIL][127], [FAIL][128], [FAIL][129], [FAIL][130], [FAIL][131], [FAIL][132], [FAIL][133], [FAIL][134], [FAIL][135], [FAIL][136], [FAIL][137]) ([fdo#109271] / [i915#1436] / [i915#180] / [i915#1814] / [i915#2292] / [i915#2722] / [i915#3002] / [i915#3363])
[106]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl4/igt@[email protected]
[107]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl4/igt@[email protected]
[108]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl3/igt@[email protected]
[109]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl7/igt@[email protected]
[110]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl4/igt@[email protected]
[111]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl1/igt@[email protected]
[112]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl1/igt@[email protected]
[113]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl1/igt@[email protected]
[114]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl3/igt@[email protected]
[115]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl3/igt@[email protected]
[116]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl7/igt@[email protected]
[117]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl4/igt@[email protected]
[118]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl2/igt@[email protected]
[119]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl2/igt@[email protected]
[120]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl4/igt@[email protected]
[121]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-kbl4/igt@[email protected]
[122]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl7/igt@[email protected]
[123]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl7/igt@[email protected]
[124]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl1/igt@[email protected]
[125]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl1/igt@[email protected]
[126]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl1/igt@[email protected]
[127]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl3/igt@[email protected]
[128]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl2/igt@[email protected]
[129]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl1/igt@[email protected]
[130]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl2/igt@[email protected]
[131]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl1/igt@[email protected]
[132]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl1/igt@[email protected]
[133]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl4/igt@[email protected]
[134]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl4/igt@[email protected]
[135]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl3/igt@[email protected]
[136]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl1/igt@[email protected]
[137]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-kbl1/igt@[email protected]
- shard-apl: ([FAIL][138], [FAIL][139], [FAIL][140], [FAIL][141]) ([i915#180] / [i915#2722] / [i915#3002] / [i915#3363]) -> ([FAIL][142], [FAIL][143], [FAIL][144]) ([fdo#109271] / [i915#2722] / [i915#3363])
[138]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-apl1/igt@[email protected]
[139]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-apl1/igt@[email protected]
[140]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-apl7/igt@[email protected]
[141]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-apl6/igt@[email protected]
[142]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl3/igt@[email protected]
[143]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl6/igt@[email protected]
[144]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-apl1/igt@[email protected]
- shard-skl: ([FAIL][145], [FAIL][146], [FAIL][147], [FAIL][148], [FAIL][149], [FAIL][150]) ([i915#1436] / [i915#2426] / [i915#2722] / [i915#3002] / [i915#3363]) -> ([FAIL][151], [FAIL][152], [FAIL][153], [FAIL][154], [FAIL][155], [FAIL][156], [FAIL][157]) ([i915#1436] / [i915#1814] / [i915#2029] / [i915#2722] / [i915#3002] / [i915#3363])
[145]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-skl4/igt@[email protected]
[146]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-skl9/igt@[email protected]
[147]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-skl4/igt@[email protected]
[148]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-skl7/igt@[email protected]
[149]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-skl2/igt@[email protected]
[150]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10120/shard-skl3/igt@[email protected]
[151]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl4/igt@[email protected]
[152]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl7/igt@[email protected]
[153]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl4/igt@[email protected]
[154]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl7/igt@[email protected]
[155]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl2/igt@[email protected]
[156]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl3/igt@[email protected]
[157]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/shard-skl3/igt@[email protected]

[fdo#108145]: https://bugs.freedesktop.org/show_bug.cgi?id=108145
[fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
[fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
[i915#1099]: https://gitlab.freedesktop.org/drm/intel/issues/1099
[i915#118]: https://gitlab.freedesktop.org/drm/intel/issues/118
[i915#1188]: https://gitlab.freedesktop.org/drm/intel/issues/1188
[i915#1319]: https://gitlab.freedesktop.org/drm/intel/issues/1319
[i915#1436]: https://gitlab.freedesktop.org/drm/intel/issues/1436
[i915#1542]: https://gitlab.freedeskt

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20170/index.html

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Reviewed-by on v3 7/12
#16 lore raw

Re: [Intel-gfx] [PATCH v3 07/12] drm, drm/i915: Move the memcpy_from_wc functionality to core drm

In reply to Thomas Hellström
Quoted 10 lines · Thomas Hellström

Memcpy from wc will be used as well by TTM memcpy. Move it to core drm, and make the interface do the right thing even on !X86.

Cc Christian König <[email protected]>
Cc Daniel Vetter <[email protected]>
Cc Dave Airlie <[email protected]>

---

<snip>

Quoted 22 lines
include/drm/drm_memcpy.h new file +22 -0
+
+#ifdef CONFIG_X86
+bool drm_memcpy_from_wc(void *dst, const void *src, unsigned long len);
+bool drm_memcpy_from_wc_dbm(struct dma_buf_map *dst,
+                           const struct dma_buf_map *src,
+                           unsigned long len);
+void drm_unaligned_memcpy_from_wc(void *dst, const void *src, unsigned long len);
+
+/* The movntdqa instructions used for memcpy-from-wc require 16-byte alignment,
+ * as well as SSE4.1 support. drm_memcpy_from_wc() will report if it cannot
+ * perform the operation. To check beforehand, pass in the parameters to
+ * drm_can_memcpy_from_wc() - since we only care about the low 4 bits,
+ * you only need to pass in the minor offsets, page-aligned pointers are
+ * always valid.
+ *
+ * For just checking for SSE4.1, in the foreknowledge that the future use
+ * will be correctly aligned, just use drm_has_memcpy_from_wc().
+ */
+#define drm_can_memcpy_from_wc(dst, src, len) \
+       drm_memcpy_from_wc((void *)((unsigned long)(dst) | (unsigned long)(src) | (len)), NULL, 0)
+
+#define drm_has_memcpy_from_wc() \

include/drm/drm_memcpy.h line 43

Quoted 10 lines
include/drm/drm_memcpy.h new file +10 -0
+       drm_memcpy_from_wc(NULL, NULL, 0)
+
+void drm_memcpy_init_early(void);
+
+#else
+
+#define drm_memcpy_from_wc(_dst, _src, _len) (false)
+#define drm_memcpy_from_wc_dbm(_dst, _src, _len) (false)
+#define drm_can_memcpy_from_wc(_dst, _src, _len) (false)
+#define drm_has_memcpy_from_wc() (false)

Does the compiler not complain for these on !x86, if called without checking the result of the statement? Maybe just make these function stubs?

Otherwise,

Quoted 11 lines
include/drm/drm_memcpy.h new file +4 -1
+#define drm_unaligned_memcpy_from_wc(_dst, _src, _len) WARN_ON(1)
+#define drm_memcpy_init_early() do {} while (0)
+#endif /* CONFIG_X86 */
+#endif /* __DRM_MEMCPY_H__ */
--

2.31.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

author reviewer
#17 lore raw

Re: [Intel-gfx] [PATCH v3 07/12] drm, drm/i915: Move the memcpy_from_wc functionality to core drm

Quoted 61 lines · Matthew Auld
Quoted 10 lines · Thomas Hellström

Memcpy from wc will be used as well by TTM memcpy. Move it to core drm, and make the interface do the right thing even on !X86.

Cc Christian König <[email protected]>
Cc Daniel Vetter <[email protected]>
Cc Dave Airlie <[email protected]>

---

<snip>

Quoted 31 lines
include/drm/drm_memcpy.h new file +3 -0
+
+#ifdef CONFIG_X86
+bool drm_memcpy_from_wc(void *dst, const void *src, unsigned long

len);

include/drm/drm_memcpy.h new file +4 -0
+bool drm_memcpy_from_wc_dbm(struct dma_buf_map *dst,
+                           const struct dma_buf_map *src,
+                           unsigned long len);
+void drm_unaligned_memcpy_from_wc(void *dst, const void *src,

unsigned long len);

(quoted diff) +2 -0
+
+/* The movntdqa instructions used for memcpy-from-wc require 16-

byte alignment,
+ * as well as SSE4.1 support. drm_memcpy_from_wc() will report if it cannot
+ * perform the operation. To check beforehand, pass in the parameters to
+ * drm_can_memcpy_from_wc() - since we only care about the low 4 bits,
+ * you only need to pass in the minor offsets, page-aligned pointers are
+ * always valid.
+ *
+ * For just checking for SSE4.1, in the foreknowledge that the future use
+ * will be correctly aligned, just use drm_has_memcpy_from_wc().
+ */
+#define drm_can_memcpy_from_wc(dst, src, len) \
+       drm_memcpy_from_wc((void *)((unsigned long)(dst) | (unsigned long)(src) | (len)), NULL, 0)

include/drm/drm_memcpy.h new file +2 -0
+
+#define drm_has_memcpy_from_wc() \

include/drm/drm_memcpy.h line 43

Quoted 10 lines
include/drm/drm_memcpy.h new file +10 -0
+       drm_memcpy_from_wc(NULL, NULL, 0)
+
+void drm_memcpy_init_early(void);
+
+#else
+
+#define drm_memcpy_from_wc(_dst, _src, _len) (false)
+#define drm_memcpy_from_wc_dbm(_dst, _src, _len) (false)
+#define drm_can_memcpy_from_wc(_dst, _src, _len) (false)
+#define drm_has_memcpy_from_wc() (false)

Does the compiler not complain for these on !x86, if called without checking the result of the statement? Maybe just make these function stubs?

Quoted 2 lines

Otherwise,

Hmm, you're right. I'll fix. Thanks for reviewing!

/Thomas

#19 lore raw

Re: [PATCH v3 08/12] drm/ttm: Use drm_memcpy_from_wc_dbm for TTM bo moves

In reply to Thomas Hellström
Quoted 32 lines · Thomas Hellström

Use fast wc memcpy for reading out of wc memory for TTM bo moves.

Cc Dave Airlie <[email protected]>
Cc Christian König <[email protected]>
Cc Daniel Vetter <[email protected]>
---
 drivers/gpu/drm/ttm/ttm_bo_util.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/gpu/drm/ttm/ttm_bo_util.c +2 -0
index 912cbe8e60a2..4a7d3d672f9a 100644
@@ -31,6 +31,7 @@
 
 #include <drm/ttm/ttm_bo_driver.h>
 #include <drm/ttm/ttm_placement.h>
+#include <drm/drm_memcpy.h>
 #include <drm/drm_vma_manager.h>
 #include <linux/dma-buf-map.h>
 #include <linux/io.h>
@@ -91,6 +92,7 @@ void ttm_move_memcpy(struct ttm_buffer_object *bo,
        const struct ttm_kmap_iter_ops *src_ops = src_iter->ops;
        struct ttm_tt *ttm = bo->ttm;
        struct dma_buf_map src_map, dst_map;
+       bool wc_memcpy;
        pgoff_t i;
 
        /* Single TTM move. NOP */

drivers/gpu/drm/ttm/ttm_bo_util.c void ttm_move_memcpy(struct ttm_buffer_object *bo, · line 119

Quoted 4 lines
drivers/gpu/drm/ttm/ttm_bo_util.c +1 -0
@@ -114,11 +116,16 @@ void ttm_move_memcpy(struct ttm_buffer_object *bo,
                return;
        }
 
+       wc_memcpy = ((!src_ops->maps_tt || ttm->caching != ttm_cached) &&

Why do we only consider the caching value for the maps_tt case? Or am I misreading this?

drivers/gpu/drm/ttm/ttm_bo_util.c void ttm_move_memcpy(struct ttm_buffer_object *bo, · line 127

Quoted 9 lines
drivers/gpu/drm/ttm/ttm_bo_util.c +4 -1
+                    drm_has_memcpy_from_wc());
+
        for (i = 0; i < dst_mem->num_pages; ++i) {
                dst_ops->map_local(dst_iter, &dst_map, i);
                src_ops->map_local(src_iter, &src_map, i);
 
-               if (!src_map.is_iomem && !dst_map.is_iomem) {
+               if (wc_memcpy) {
+                       drm_memcpy_from_wc_dbm(&dst_map, &src_map, PAGE_SIZE);

Do we need to check the return value here? memcpy_from_wc expects certain address alignment, or is that always guaranteed here? Maybe throw a warning just for paranoia?

Quoted 7 lines
drivers/gpu/drm/ttm/ttm_bo_util.c +1 -1
+               } else if (!src_map.is_iomem && !dst_map.is_iomem) {
                        memcpy(dst_map.vaddr, src_map.vaddr, PAGE_SIZE);
                } else if (!src_map.is_iomem) {
                        dma_buf_map_memcpy_to(&dst_map, src_map.vaddr,
--

2.31.1

author reviewer
#20 lore raw

Re: [PATCH v3 08/12] drm/ttm: Use drm_memcpy_from_wc_dbm for TTM bo moves

Quoted 40 lines · Matthew Auld
Quoted 38 lines · Thomas Hellström

Use fast wc memcpy for reading out of wc memory for TTM bo moves.

Cc Dave Airlie <[email protected]>
Cc Christian König <[email protected]>
Cc Daniel Vetter <[email protected]>

---
 drivers/gpu/drm/ttm/ttm_bo_util.c | 9 ++++++++-

 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/gpu/drm/ttm/ttm_bo_util.c +2 -0
index 912cbe8e60a2..4a7d3d672f9a 100644
@@ -31,6 +31,7 @@
 
  #include <drm/ttm/ttm_bo_driver.h>
  #include <drm/ttm/ttm_placement.h>
+#include <drm/drm_memcpy.h>
  #include <drm/drm_vma_manager.h>
  #include <linux/dma-buf-map.h>
  #include <linux/io.h>
@@ -91,6 +92,7 @@ void ttm_move_memcpy(struct ttm_buffer_object
 *bo,
         const struct ttm_kmap_iter_ops *src_ops = src_iter->ops;
         struct ttm_tt *ttm = bo->ttm;
         struct dma_buf_map src_map, dst_map;
+       bool wc_memcpy;
         pgoff_t i;
 

        /* Single TTM move. NOP */

drivers/gpu/drm/ttm/ttm_bo_util.c +0 -0
@@ -114,11 +116,16 @@ void ttm_move_memcpy(struct ttm_buffer_object
 *bo,
                 return;
         }
 
Quoted 5 lines
Quoted 2 lines
drivers/gpu/drm/ttm/ttm_bo_util.c +1 -0
+       wc_memcpy = ((!src_ops->maps_tt || ttm->caching !=

ttm_cached) &&

Why do we only consider the caching value for the maps_tt case? Or am I misreading this?

Hmm, you're right we should probably check also the iomem caching or ignore the tt caching. Sometimes these special memcpys tend to be less cpu cache thrashing and should be used wherever possible, but I guess we should start out with only using it when source is WC.

Quoted 7 lines
Quoted 7 lines

+                    drm_has_memcpy_from_wc());
+
        for (i = 0; i < dst_mem->num_pages; ++i) { dst_ops->map_local(dst_iter, &dst_map, i); src_ops->map_local(src_iter, &src_map, i);

Quoted 8 lines
Quoted 4 lines

-               if (!src_map.is_iomem && !dst_map.is_iomem) { +               if (wc_memcpy) {
+                       drm_memcpy_from_wc_dbm(&dst_map, &src_map, PAGE_SIZE);

Do we need to check the return value here? memcpy_from_wc expects certain address alignment, or is that always guaranteed here? Maybe throw a warning just for paranoia?

It should always be PAGE_SIZE aligned. But I guess it doesn't hurt to do
if (wc_memcpy && drm_memcpy_from_wc_dbm())
;

Quoted 11 lines
Quoted 11 lines

+               } else if (!src_map.is_iomem && !dst_map.is_iomem) {
                        memcpy(dst_map.vaddr, src_map.vaddr, PAGE_SIZE);
                } else if (!src_map.is_iomem) { dma_buf_map_memcpy_to(&dst_map, src_map.vaddr,
--
2.31.1

#7 lore raw

Re: [Intel-gfx] [PATCH v3 06/12] drm/ttm: Add a generic TTM memcpy move for page-based iomem

In reply to Thomas Hellström
Quoted 190 lines · Thomas Hellström

The internal ttm_bo_util memcpy uses ioremap functionality, and while it probably might be possible to use it for copying in- and out of sglist represented io memory, using io_mem_reserve() / io_mem_free() callbacks, that would cause problems with fault().
Instead, implement a method mapping page-by-page using kmap_local() semantics. As an additional benefit we then avoid the occasional global TLB flushes of ioremap() and consuming ioremap space, elimination of a critical point of failure and with a slight change of semantics we could also push the memcpy out async for testing and async driver development purposes.

A special linear iomem iterator is introduced internally to mimic the old ioremap behaviour for code-paths that can't immediately be ported over. This adds to the code size and should be considered a temporary solution.

Looking at the code we have a lot of checks for iomap tagged pointers. Ideally we should extend the core memremap functions to also accept uncached memory and kmap_local functionality. Then we could strip a lot of code.

---
v3:
- Split up in various TTM files and addressed review comments by Christian König. Tested and fixed legacy iomap memcpy path on i915.

---
 drivers/gpu/drm/ttm/ttm_bo_util.c  | 278 ++++++++++-------------------
 drivers/gpu/drm/ttm/ttm_module.c   |  35 ++++
 drivers/gpu/drm/ttm/ttm_resource.c | 166 +++++++++++++++++
 drivers/gpu/drm/ttm/ttm_tt.c       |  42 +++++
 include/drm/ttm/ttm_bo_driver.h    |  28 +++
 include/drm/ttm/ttm_caching.h      |   2 +
 include/drm/ttm/ttm_kmap_iter.h    |  61 +++++++
 include/drm/ttm/ttm_resource.h     |  61 +++++++
 include/drm/ttm/ttm_tt.h           |  16 ++
 9 files changed, 508 insertions(+), 181 deletions(-)
 create mode 100644 include/drm/ttm/ttm_kmap_iter.h
drivers/gpu/drm/ttm/ttm_bo_util.c +45 -90
index ae8b61460724..912cbe8e60a2 100644
@@ -72,190 +72,126 @@ void ttm_mem_io_free(struct ttm_device *bdev,
        mem->bus.addr = NULL;
 }
 
-static int ttm_resource_ioremap(struct ttm_device *bdev,
-                              struct ttm_resource *mem,
-                              void **virtual)
+/**
+ * ttm_move_memcpy - Helper to perform a memcpy ttm move operation.
+ * @bo: The struct ttm_buffer_object.
+ * @new_mem: The struct ttm_resource we're moving to (copy destination).
+ * @new_iter: A struct ttm_kmap_iter representing the destination resource.
+ * @src_iter: A struct ttm_kmap_iter representing the source resource.
+ *
+ * This function is intended to be able to move out async under a
+ * dma-fence if desired.
+ */
+void ttm_move_memcpy(struct ttm_buffer_object *bo,
+                    struct ttm_resource *dst_mem,
+                    struct ttm_kmap_iter *dst_iter,
+                    struct ttm_kmap_iter *src_iter)
 {
-       int ret;
-       void *addr;
-
-       *virtual = NULL;
-       ret = ttm_mem_io_reserve(bdev, mem);
-       if (ret || !mem->bus.is_iomem)
-               return ret;
+       const struct ttm_kmap_iter_ops *dst_ops = dst_iter->ops;
+       const struct ttm_kmap_iter_ops *src_ops = src_iter->ops;
+       struct ttm_tt *ttm = bo->ttm;
+       struct dma_buf_map src_map, dst_map;
+       pgoff_t i;
 
-       if (mem->bus.addr) {
-               addr = mem->bus.addr;
-       } else {
-               size_t bus_size = (size_t)mem->num_pages << PAGE_SHIFT;
+       /* Single TTM move. NOP */
+       if (dst_ops->maps_tt && src_ops->maps_tt)
+               return;
 
-               if (mem->bus.caching == ttm_write_combined)
-                       addr = ioremap_wc(mem->bus.offset, bus_size);
-#ifdef CONFIG_X86
-               else if (mem->bus.caching == ttm_cached)
-                       addr = ioremap_cache(mem->bus.offset, bus_size);
-#endif
-               else
-                       addr = ioremap(mem->bus.offset, bus_size);
-               if (!addr) {
-                       ttm_mem_io_free(bdev, mem);
-                       return -ENOMEM;
+       /* Don't move nonexistent data. Clear destination instead. */
+       if (src_ops->maps_tt && (!ttm || !ttm_tt_is_populated(ttm))) {
+               if (ttm && !(ttm->page_flags & TTM_PAGE_FLAG_ZERO_ALLOC))
+                       return;
+
+               for (i = 0; i < dst_mem->num_pages; ++i) {
+                       dst_ops->map_local(dst_iter, &dst_map, i);
+                       if (dst_map.is_iomem)
+                               memset_io(dst_map.vaddr_iomem, 0, PAGE_SIZE);
+                       else
+                               memset(dst_map.vaddr, 0, PAGE_SIZE);
+                       if (dst_ops->unmap_local)
+                               dst_ops->unmap_local(dst_iter, &dst_map);
                }
+               return;
        }
-       *virtual = addr;
-       return 0;
-}
-
-static void ttm_resource_iounmap(struct ttm_device *bdev,
-                               struct ttm_resource *mem,
-                               void *virtual)
-{
-       if (virtual && mem->bus.addr == NULL)
-               iounmap(virtual);
-       ttm_mem_io_free(bdev, mem);
-}
-
-static int ttm_copy_io_page(void *dst, void *src, unsigned long page)
-{
-       uint32_t *dstP =
-           (uint32_t *) ((unsigned long)dst + (page << PAGE_SHIFT));
-       uint32_t *srcP =
-           (uint32_t *) ((unsigned long)src + (page << PAGE_SHIFT));
-
-       int i;
-       for (i = 0; i < PAGE_SIZE / sizeof(uint32_t); ++i)
-               iowrite32(ioread32(srcP++), dstP++);
-       return 0;
-}
-
-static int ttm_copy_io_ttm_page(struct ttm_tt *ttm, void *src,
-                               unsigned long page,
-                               pgprot_t prot)
-{
-       struct page *d = ttm->pages[page];
-       void *dst;
-
-       if (!d)
-               return -ENOMEM;
-
-       src = (void *)((unsigned long)src + (page << PAGE_SHIFT));
-       dst = kmap_atomic_prot(d, prot);
-       if (!dst)
-               return -ENOMEM;
-
-       memcpy_fromio(dst, src, PAGE_SIZE);
-
-       kunmap_atomic(dst);
-
-       return 0;
-}
-
-static int ttm_copy_ttm_io_page(struct ttm_tt *ttm, void *dst,
-                               unsigned long page,
-                               pgprot_t prot)
-{
-       struct page *s = ttm->pages[page];
-       void *src;
-
-       if (!s)
-               return -ENOMEM;
-
-       dst = (void *)((unsigned long)dst + (page << PAGE_SHIFT));
-       src = kmap_atomic_prot(s, prot);
-       if (!src)
-               return -ENOMEM;
 
-       memcpy_toio(dst, src, PAGE_SIZE);
-
-       kunmap_atomic(src);
+       for (i = 0; i < dst_mem->num_pages; ++i) {
+               dst_ops->map_local(dst_iter, &dst_map, i);
+               src_ops->map_local(src_iter, &src_map, i);
+
+               if (!src_map.is_iomem && !dst_map.is_iomem) {
+                       memcpy(dst_map.vaddr, src_map.vaddr, PAGE_SIZE);
+               } else if (!src_map.is_iomem) {
+                       dma_buf_map_memcpy_to(&dst_map, src_map.vaddr,
+                                             PAGE_SIZE);

drivers/gpu/drm/ttm/ttm_bo_util.c void ttm_mem_io_free(struct ttm_device *bdev, · line 134

Quoted 10 lines
drivers/gpu/drm/ttm/ttm_bo_util.c +8 -1
+               } else if (!dst_map.is_iomem) {
+                       memcpy_fromio(dst_map.vaddr, src_map.vaddr_iomem,
+                                     PAGE_SIZE);
+               } else {
+                       int j;
+                       u32 __iomem *src = src_map.vaddr_iomem;
+                       u32 __iomem *dst = dst_map.vaddr_iomem;
 
-       return 0;
+                       for (j = 0; j < (PAGE_SIZE >> 2); ++j)

IMO PAGE_SIZE / sizeof(u32) is easier to understand.

Quoted 351 lines
drivers/gpu/drm/ttm/ttm_bo_util.c +44 -90
+                               iowrite32(ioread32(src++), dst++);
+               }
+               if (src_ops->unmap_local)
+                       src_ops->unmap_local(src_iter, &src_map);
+               if (dst_ops->unmap_local)
+                       dst_ops->unmap_local(dst_iter, &dst_map);
+       }
 }
+EXPORT_SYMBOL(ttm_move_memcpy);
 
 int ttm_bo_move_memcpy(struct ttm_buffer_object *bo,
                       struct ttm_operation_ctx *ctx,
-                      struct ttm_resource *new_mem)
+                      struct ttm_resource *dst_mem)
 {
        struct ttm_device *bdev = bo->bdev;
-       struct ttm_resource_manager *man = ttm_manager_type(bdev, new_mem->mem_type);
+       struct ttm_resource_manager *dst_man =
+               ttm_manager_type(bo->bdev, dst_mem->mem_type);
        struct ttm_tt *ttm = bo->ttm;
-       struct ttm_resource *old_mem = &bo->mem;
-       struct ttm_resource old_copy = *old_mem;
-       void *old_iomap;
-       void *new_iomap;
+       struct ttm_resource *src_mem = &bo->mem;
+       struct ttm_resource_manager *src_man =
+               ttm_manager_type(bdev, src_mem->mem_type);
+       struct ttm_resource src_copy = *src_mem;
+       union {
+               struct ttm_kmap_iter_tt tt;
+               struct ttm_kmap_iter_linear_io io;
+       } _dst_iter, _src_iter;
+       struct ttm_kmap_iter *dst_iter, *src_iter;
        int ret;
-       unsigned long i;
 
-       ret = ttm_bo_wait_ctx(bo, ctx);
-       if (ret)
-               return ret;
-
-       ret = ttm_resource_ioremap(bdev, old_mem, &old_iomap);
-       if (ret)
-               return ret;
-       ret = ttm_resource_ioremap(bdev, new_mem, &new_iomap);
-       if (ret)
-               goto out;
-
-       /*
-        * Single TTM move. NOP.
-        */
-       if (old_iomap == NULL && new_iomap == NULL)
-               goto out2;
-
-       /*
-        * Don't move nonexistent data. Clear destination instead.
-        */
-       if (old_iomap == NULL &&
-           (ttm == NULL || (!ttm_tt_is_populated(ttm) &&
-                            !(ttm->page_flags & TTM_PAGE_FLAG_SWAPPED)))) {
-               memset_io(new_iomap, 0, new_mem->num_pages*PAGE_SIZE);
-               goto out2;
-       }
-
-       /*
-        * TTM might be null for moves within the same region.
-        */
-       if (ttm) {
+       if (ttm && ((ttm->page_flags & TTM_PAGE_FLAG_SWAPPED) ||
+                   dst_man->use_tt)) {
                ret = ttm_tt_populate(bdev, ttm, ctx);
                if (ret)
-                       goto out1;
+                       return ret;
        }
 
-       for (i = 0; i < new_mem->num_pages; ++i) {
-               if (old_iomap == NULL) {
-                       pgprot_t prot = ttm_io_prot(bo, old_mem, PAGE_KERNEL);
-                       ret = ttm_copy_ttm_io_page(ttm, new_iomap, i,
-                                                  prot);
-               } else if (new_iomap == NULL) {
-                       pgprot_t prot = ttm_io_prot(bo, new_mem, PAGE_KERNEL);
-                       ret = ttm_copy_io_ttm_page(ttm, old_iomap, i,
-                                                  prot);
-               } else {
-                       ret = ttm_copy_io_page(new_iomap, old_iomap, i);
-               }
-               if (ret)
-                       goto out1;
+       dst_iter = ttm_kmap_iter_linear_io_init(&_dst_iter.io, bdev, dst_mem);
+       if (PTR_ERR(dst_iter) == -EINVAL && dst_man->use_tt)
+               dst_iter = ttm_kmap_iter_tt_init(&_dst_iter.tt, bo->ttm);
+       if (IS_ERR(dst_iter))
+               return PTR_ERR(dst_iter);
+
+       src_iter = ttm_kmap_iter_linear_io_init(&_src_iter.io, bdev, src_mem);
+       if (PTR_ERR(src_iter) == -EINVAL && src_man->use_tt)
+               src_iter = ttm_kmap_iter_tt_init(&_src_iter.tt, bo->ttm);
+       if (IS_ERR(src_iter)) {
+               ret = PTR_ERR(src_iter);
+               goto out_src_iter;
        }
-       mb();
-out2:
-       old_copy = *old_mem;
 
-       ttm_bo_assign_mem(bo, new_mem);
-
-       if (!man->use_tt)
-               ttm_bo_tt_destroy(bo);
+       ttm_move_memcpy(bo, dst_mem, dst_iter, src_iter);
+       src_copy = *src_mem;
+       ttm_bo_move_sync_cleanup(bo, dst_mem);
 
-out1:
-       ttm_resource_iounmap(bdev, old_mem, new_iomap);
-out:
-       ttm_resource_iounmap(bdev, &old_copy, old_iomap);
+       if (!src_iter->ops->maps_tt)
+               ttm_kmap_iter_linear_io_fini(&_src_iter.io, bdev, &src_copy);
+out_src_iter:
+       if (!dst_iter->ops->maps_tt)
+               ttm_kmap_iter_linear_io_fini(&_dst_iter.io, bdev, dst_mem);
 
-       /*
-        * On error, keep the mm node!
-        */
-       if (!ret)
-               ttm_resource_free(bo, &old_copy);
        return ret;
 }
 EXPORT_SYMBOL(ttm_bo_move_memcpy);
@@ -336,27 +272,7 @@ pgprot_t ttm_io_prot(struct ttm_buffer_object *bo, struct ttm_resource *res,
        man = ttm_manager_type(bo->bdev, res->mem_type);
        caching = man->use_tt ? bo->ttm->caching : res->bus.caching;
 
-       /* Cached mappings need no adjustment */
-       if (caching == ttm_cached)
-               return tmp;
-
-#if defined(__i386__) || defined(__x86_64__)
-       if (caching == ttm_write_combined)
-               tmp = pgprot_writecombine(tmp);
-       else if (boot_cpu_data.x86 > 3)
-               tmp = pgprot_noncached(tmp);
-#endif
-#if defined(__ia64__) || defined(__arm__) || defined(__aarch64__) || \
-    defined(__powerpc__) || defined(__mips__)
-       if (caching == ttm_write_combined)
-               tmp = pgprot_writecombine(tmp);
-       else
-               tmp = pgprot_noncached(tmp);
-#endif
-#if defined(__sparc__)
-       tmp = pgprot_noncached(tmp);
-#endif
-       return tmp;
+       return ttm_prot_from_caching(caching, tmp);
 }
 EXPORT_SYMBOL(ttm_io_prot);
 
drivers/gpu/drm/ttm/ttm_module.c +35 -0
index 56b0efdba1a9..997c458f68a9 100644
@@ -31,12 +31,47 @@
  */
 #include <linux/module.h>
 #include <linux/device.h>
+#include <linux/pgtable.h>
 #include <linux/sched.h>
 #include <linux/debugfs.h>
 #include <drm/drm_sysfs.h>
+#include <drm/ttm/ttm_caching.h>
 
 #include "ttm_module.h"
 
+/**
+ * ttm_prot_from_caching - Modify the page protection according to the
+ * ttm cacing mode
+ * @caching: The ttm caching mode
+ * @tmp: The original page protection
+ *
+ * Return: The modified page protection
+ */
+pgprot_t ttm_prot_from_caching(enum ttm_caching caching, pgprot_t tmp)
+{
+       /* Cached mappings need no adjustment */
+       if (caching == ttm_cached)
+               return tmp;
+
+#if defined(__i386__) || defined(__x86_64__)
+       if (caching == ttm_write_combined)
+               tmp = pgprot_writecombine(tmp);
+       else if (boot_cpu_data.x86 > 3)
+               tmp = pgprot_noncached(tmp);
+#endif
+#if defined(__ia64__) || defined(__arm__) || defined(__aarch64__) || \
+       defined(__powerpc__) || defined(__mips__)
+       if (caching == ttm_write_combined)
+               tmp = pgprot_writecombine(tmp);
+       else
+               tmp = pgprot_noncached(tmp);
+#endif
+#if defined(__sparc__)
+       tmp = pgprot_noncached(tmp);
+#endif
+       return tmp;
+}
+
 struct dentry *ttm_debugfs_root;
 
 static int __init ttm_init(void)
drivers/gpu/drm/ttm/ttm_resource.c +123 -0
index 59e2b7157e41..e05ae7e3d477 100644
@@ -22,6 +22,10 @@
  * Authors: Christian König
  */
 
+#include <linux/dma-buf-map.h>
+#include <linux/io-mapping.h>
+#include <linux/scatterlist.h>
+
 #include <drm/ttm/ttm_resource.h>
 #include <drm/ttm/ttm_bo_driver.h>
 
@@ -147,3 +151,165 @@ void ttm_resource_manager_debug(struct ttm_resource_manager *man,
                man->func->debug(man, p);
 }
 EXPORT_SYMBOL(ttm_resource_manager_debug);
+
+static void ttm_kmap_iter_iomap_map_local(struct ttm_kmap_iter *iter,
+                                         struct dma_buf_map *dmap,
+                                         pgoff_t i)
+{
+       struct ttm_kmap_iter_iomap *iter_io =
+               container_of(iter, typeof(*iter_io), base);
+       void __iomem *addr;
+
+retry:
+       while (i >= iter_io->cache.end) {
+               iter_io->cache.sg = iter_io->cache.sg ?
+                       sg_next(iter_io->cache.sg) : iter_io->st->sgl;
+               iter_io->cache.i = iter_io->cache.end;
+               iter_io->cache.end += sg_dma_len(iter_io->cache.sg) >>
+                       PAGE_SHIFT;
+               iter_io->cache.offs = sg_dma_address(iter_io->cache.sg) -
+                       iter_io->start;
+       }
+
+       if (i < iter_io->cache.i) {
+               iter_io->cache.end = 0;
+               iter_io->cache.sg = NULL;
+               goto retry;
+       }
+
+       addr = io_mapping_map_local_wc(iter_io->iomap, iter_io->cache.offs +
+                                      (((resource_size_t)i - iter_io->cache.i)
+                                       << PAGE_SHIFT));
+       dma_buf_map_set_vaddr_iomem(dmap, addr);
+}
+
+static void ttm_kmap_iter_iomap_unmap_local(struct ttm_kmap_iter *iter,
+                                           struct dma_buf_map *map)
+{
+       io_mapping_unmap_local(map->vaddr_iomem);
+}
+
+static const struct ttm_kmap_iter_ops ttm_kmap_iter_io_ops = {
+       .map_local =  ttm_kmap_iter_iomap_map_local,
+       .unmap_local = ttm_kmap_iter_iomap_unmap_local,
+       .maps_tt = false,
+};
+
+/**
+ * ttm_kmap_iter_iomap_init - Initialize a struct ttm_kmap_iter_iomap
+ * @iter_io: The struct ttm_kmap_iter_iomap to initialize.
+ * @iomap: The struct io_mapping representing the underlying linear io_memory.
+ * @st: sg_table into @iomap, representing the memory of the struct
+ * ttm_resource.
+ * @start: Offset that needs to be subtracted from @st to make
+ * sg_dma_address(st->sgl) - @start == 0 for @iomap start.
+ *
+ * Return: Pointer to the embedded struct ttm_kmap_iter.
+ */
+struct ttm_kmap_iter *
+ttm_kmap_iter_iomap_init(struct ttm_kmap_iter_iomap *iter_io,
+                        struct io_mapping *iomap,
+                        struct sg_table *st,
+                        resource_size_t start)
+{
+       iter_io->base.ops = &ttm_kmap_iter_io_ops;
+       iter_io->iomap = iomap;
+       iter_io->st = st;
+       iter_io->start = start;
+       memset(&iter_io->cache, 0, sizeof(iter_io->cache));
+
+       return &iter_io->base;
+}
+EXPORT_SYMBOL(ttm_kmap_iter_iomap_init);
+
+/**
+ * DOC: Linear io iterator
+ *
+ * This code should die in the not too near future. Best would be if we could
+ * make io-mapping use memremap for all io memory, and have memremap
+ * implement a kmap_local functionality. We could then strip a huge amount of
+ * code. These linear io iterators are implemented to mimic old functionality,
+ * and they don't use kmap_local semantics at all internally. Rather ioremap or
+ * friends, and at least on 32-bit they add global TLB flushes and points
+ * of failure.
+ */
+
+static void ttm_kmap_iter_linear_io_map_local(struct ttm_kmap_iter *iter,
+                                             struct dma_buf_map *dmap,
+                                             pgoff_t i)
+{
+       struct ttm_kmap_iter_linear_io *iter_io =
+               container_of(iter, typeof(*iter_io), base);
+
+       *dmap = iter_io->dmap;
+       dma_buf_map_incr(dmap, i * PAGE_SIZE);
+}
+
+static const struct ttm_kmap_iter_ops ttm_kmap_iter_linear_io_ops = {
+       .map_local =  ttm_kmap_iter_linear_io_map_local,
+       .maps_tt = false,
+};
+
+struct ttm_kmap_iter *
+ttm_kmap_iter_linear_io_init(struct ttm_kmap_iter_linear_io *iter_io,
+                            struct ttm_device *bdev,
+                            struct ttm_resource *mem)
+{
+       int ret;
+
+       ret = ttm_mem_io_reserve(bdev, mem);
+       if (ret)
+               goto out_err;
+       if (!mem->bus.is_iomem) {
+               ret = -EINVAL;
+               goto out_io_free;
+       }
+
+       if (mem->bus.addr) {
+               dma_buf_map_set_vaddr(&iter_io->dmap, mem->bus.addr);
+               iter_io->needs_unmap = false;
+       } else {
+               size_t bus_size = (size_t)mem->num_pages << PAGE_SHIFT;

drivers/gpu/drm/ttm/ttm_resource.c void ttm_resource_manager_debug(struct ttm_resource_manager *man, · line 282

Quoted 10 lines
drivers/gpu/drm/ttm/ttm_resource.c +10 -0
+
+               iter_io->needs_unmap = true;
+               if (mem->bus.caching == ttm_write_combined)
+                       dma_buf_map_set_vaddr_iomem(&iter_io->dmap,
+                                                   ioremap_wc(mem->bus.offset,
+                                                              bus_size));
+               else if (mem->bus.caching == ttm_cached)
+                       dma_buf_map_set_vaddr(&iter_io->dmap,
+                                             memremap(mem->bus.offset, bus_size,
+                                                      MEMREMAP_WB));

The comments in set_vaddr suggest that this is meant for system-memory. Does that actually matter or is it just about not losing the __iomem annotation on platforms where it matters? Apparently cached device local is a thing. Also should this not be wrapped in CONFIG_X86?
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

author reviewer
#8 lore raw

Re: [Intel-gfx] [PATCH v3 06/12] drm/ttm: Add a generic TTM memcpy move for page-based iomem

Quoted 199 lines · Matthew Auld
Quoted 197 lines · Thomas Hellström

The internal ttm_bo_util memcpy uses ioremap functionality, and while it probably might be possible to use it for copying in- and out of sglist represented io memory, using io_mem_reserve() / io_mem_free() callbacks, that would cause problems with fault().
Instead, implement a method mapping page-by-page using kmap_local() semantics. As an additional benefit we then avoid the occasional global TLB flushes of ioremap() and consuming ioremap space, elimination of a critical point of failure and with a slight change of semantics we could also push the memcpy out async for testing and async driver development purposes.

A special linear iomem iterator is introduced internally to mimic the old ioremap behaviour for code-paths that can't immediately be ported over. This adds to the code size and should be considered a temporary solution.

Looking at the code we have a lot of checks for iomap tagged pointers. Ideally we should extend the core memremap functions to also accept uncached memory and kmap_local functionality. Then we could strip a lot of code.

---
v3:
- Split up in various TTM files and addressed review comments by Christian König. Tested and fixed legacy iomap memcpy path on i915.

---
  drivers/gpu/drm/ttm/ttm_bo_util.c  | 278 ++++++++++-------------------
  drivers/gpu/drm/ttm/ttm_module.c   |  35 ++++
  drivers/gpu/drm/ttm/ttm_resource.c | 166 +++++++++++++++++
  drivers/gpu/drm/ttm/ttm_tt.c       |  42 +++++
  include/drm/ttm/ttm_bo_driver.h    |  28 +++
  include/drm/ttm/ttm_caching.h      |   2 +
  include/drm/ttm/ttm_kmap_iter.h    |  61 +++++++
  include/drm/ttm/ttm_resource.h     |  61 +++++++
  include/drm/ttm/ttm_tt.h           |  16 ++
  9 files changed, 508 insertions(+), 181 deletions(-)

create mode 100644 include/drm/ttm/ttm_kmap_iter.h

drivers/gpu/drm/ttm/ttm_bo_util.c +52 -90
index ae8b61460724..912cbe8e60a2 100644
@@ -72,190 +72,126 @@ void ttm_mem_io_free(struct ttm_device *bdev,
         mem->bus.addr = NULL;
  }
 
-static int ttm_resource_ioremap(struct ttm_device *bdev,
-                              struct ttm_resource *mem,
-                              void **virtual)
+/**
+ * ttm_move_memcpy - Helper to perform a memcpy ttm move operation.
+ * @bo: The struct ttm_buffer_object.
+ * @new_mem: The struct ttm_resource we're moving to (copy destination).
+ * @new_iter: A struct ttm_kmap_iter representing the destination resource.
+ * @src_iter: A struct ttm_kmap_iter representing the source resource.
+ *
+ * This function is intended to be able to move out async under a
+ * dma-fence if desired.
+ */
+void ttm_move_memcpy(struct ttm_buffer_object *bo,
+                    struct ttm_resource *dst_mem,
+                    struct ttm_kmap_iter *dst_iter,
+                    struct ttm_kmap_iter *src_iter)
  {
-       int ret;
-       void *addr;
-
-       *virtual = NULL;
-       ret = ttm_mem_io_reserve(bdev, mem);
-       if (ret || !mem->bus.is_iomem)
-               return ret;
+       const struct ttm_kmap_iter_ops *dst_ops = dst_iter->ops;
+       const struct ttm_kmap_iter_ops *src_ops = src_iter->ops;
+       struct ttm_tt *ttm = bo->ttm;
+       struct dma_buf_map src_map, dst_map;
+       pgoff_t i;
 
-       if (mem->bus.addr) {
-               addr = mem->bus.addr;
-       } else {
-               size_t bus_size = (size_t)mem->num_pages << PAGE_SHIFT;
+       /* Single TTM move. NOP */
+       if (dst_ops->maps_tt && src_ops->maps_tt)
+               return;
 
-               if (mem->bus.caching == ttm_write_combined)
-                       addr = ioremap_wc(mem->bus.offset, bus_size);
-#ifdef CONFIG_X86
-               else if (mem->bus.caching == ttm_cached)
-                       addr = ioremap_cache(mem->bus.offset, bus_size);
-#endif
-               else
-                       addr = ioremap(mem->bus.offset, bus_size);
-               if (!addr) {
-                       ttm_mem_io_free(bdev, mem);
-                       return -ENOMEM;
+       /* Don't move nonexistent data. Clear destination instead. */
+       if (src_ops->maps_tt && (!ttm || !ttm_tt_is_populated(ttm))) {
+               if (ttm && !(ttm->page_flags & TTM_PAGE_FLAG_ZERO_ALLOC))
+                       return;
+
+               for (i = 0; i < dst_mem->num_pages; ++i) {
+                       dst_ops->map_local(dst_iter, &dst_map, i);
+                       if (dst_map.is_iomem)
+                               memset_io(dst_map.vaddr_iomem, 0, PAGE_SIZE);
+                       else
+                               memset(dst_map.vaddr, 0, PAGE_SIZE);
+                       if (dst_ops->unmap_local)
+                               dst_ops->unmap_local(dst_iter, &dst_map);
                 }
+               return;
         }
-       *virtual = addr;
-       return 0;
-}
-
-static void ttm_resource_iounmap(struct ttm_device *bdev,
-                               struct ttm_resource *mem,
-                               void *virtual)
-{
-       if (virtual && mem->bus.addr == NULL)
-               iounmap(virtual);
-       ttm_mem_io_free(bdev, mem);
-}
-
-static int ttm_copy_io_page(void *dst, void *src, unsigned long page)
-{
-       uint32_t *dstP =
-           (uint32_t *) ((unsigned long)dst + (page << PAGE_SHIFT));
-       uint32_t *srcP =
-           (uint32_t *) ((unsigned long)src + (page << PAGE_SHIFT));
-
-       int i;
-       for (i = 0; i < PAGE_SIZE / sizeof(uint32_t); ++i)
-               iowrite32(ioread32(srcP++), dstP++);
-       return 0;
-}
-
-static int ttm_copy_io_ttm_page(struct ttm_tt *ttm, void *src,
-                               unsigned long page,
-                               pgprot_t prot)
-{
-       struct page *d = ttm->pages[page];
-       void *dst;
-
-       if (!d)
-               return -ENOMEM;
-
-       src = (void *)((unsigned long)src + (page << PAGE_SHIFT));
-       dst = kmap_atomic_prot(d, prot);
-       if (!dst)
-               return -ENOMEM;
-
-       memcpy_fromio(dst, src, PAGE_SIZE);
-
-       kunmap_atomic(dst);
-
-       return 0;
-}
-
-static int ttm_copy_ttm_io_page(struct ttm_tt *ttm, void *dst,
-                               unsigned long page,
-                               pgprot_t prot)
-{
-       struct page *s = ttm->pages[page];
-       void *src;
-
-       if (!s)
-               return -ENOMEM;
-
-       dst = (void *)((unsigned long)dst + (page << PAGE_SHIFT));
-       src = kmap_atomic_prot(s, prot);
-       if (!src)
-               return -ENOMEM;
 
-       memcpy_toio(dst, src, PAGE_SIZE);
-
-       kunmap_atomic(src);
+       for (i = 0; i < dst_mem->num_pages; ++i) {
+               dst_ops->map_local(dst_iter, &dst_map, i);
+               src_ops->map_local(src_iter, &src_map, i);
+
+               if (!src_map.is_iomem && !dst_map.is_iomem) {
+                       memcpy(dst_map.vaddr, src_map.vaddr, PAGE_SIZE);
+               } else if (!src_map.is_iomem) {
+                       dma_buf_map_memcpy_to(&dst_map, src_map.vaddr,
+                                             PAGE_SIZE);
+               } else if (!dst_map.is_iomem) {
+                       memcpy_fromio(dst_map.vaddr, src_map.vaddr_iomem,
+                                     PAGE_SIZE);
+               } else {
+                       int j;
+                       u32 __iomem *src = src_map.vaddr_iomem;
+                       u32 __iomem *dst = dst_map.vaddr_iomem;
 
Quoted 3 lines

drivers/gpu/drm/ttm/ttm_bo_util.c void ttm_mem_io_free(struct ttm_device *bdev, · line 134

Quoted 2 lines
drivers/gpu/drm/ttm/ttm_bo_util.c +1 -1
-       return 0;
+                       for (j = 0; j < (PAGE_SIZE >> 2); ++j)

IMO PAGE_SIZE / sizeof(u32) is easier to understand.

OK, will fix.

Quoted 357 lines
Quoted 357 lines
drivers/gpu/drm/ttm/ttm_bo_util.c +44 -90
+                               iowrite32(ioread32(src++), dst++);
+               }
+               if (src_ops->unmap_local)
+                       src_ops->unmap_local(src_iter, &src_map);
+               if (dst_ops->unmap_local)
+                       dst_ops->unmap_local(dst_iter, &dst_map);
+       }
  }
+EXPORT_SYMBOL(ttm_move_memcpy);
 
  int ttm_bo_move_memcpy(struct ttm_buffer_object *bo,
                        struct ttm_operation_ctx *ctx,
-                      struct ttm_resource *new_mem)
+                      struct ttm_resource *dst_mem)
  {
         struct ttm_device *bdev = bo->bdev;
-       struct ttm_resource_manager *man = ttm_manager_type(bdev, new_mem->mem_type);
+       struct ttm_resource_manager *dst_man =
+               ttm_manager_type(bo->bdev, dst_mem->mem_type);
         struct ttm_tt *ttm = bo->ttm;
-       struct ttm_resource *old_mem = &bo->mem;
-       struct ttm_resource old_copy = *old_mem;
-       void *old_iomap;
-       void *new_iomap;
+       struct ttm_resource *src_mem = &bo->mem;
+       struct ttm_resource_manager *src_man =
+               ttm_manager_type(bdev, src_mem->mem_type);
+       struct ttm_resource src_copy = *src_mem;
+       union {
+               struct ttm_kmap_iter_tt tt;
+               struct ttm_kmap_iter_linear_io io;
+       } _dst_iter, _src_iter;
+       struct ttm_kmap_iter *dst_iter, *src_iter;
         int ret;
-       unsigned long i;
 
-       ret = ttm_bo_wait_ctx(bo, ctx);
-       if (ret)
-               return ret;
-
-       ret = ttm_resource_ioremap(bdev, old_mem, &old_iomap);
-       if (ret)
-               return ret;
-       ret = ttm_resource_ioremap(bdev, new_mem, &new_iomap);
-       if (ret)
-               goto out;
-
-       /*
-        * Single TTM move. NOP.
-        */
-       if (old_iomap == NULL && new_iomap == NULL)
-               goto out2;
-
-       /*
-        * Don't move nonexistent data. Clear destination instead.
-        */
-       if (old_iomap == NULL &&
-           (ttm == NULL || (!ttm_tt_is_populated(ttm) &&
-                            !(ttm->page_flags & TTM_PAGE_FLAG_SWAPPED)))) {
-               memset_io(new_iomap, 0, new_mem->num_pages*PAGE_SIZE);
-               goto out2;
-       }
-
-       /*
-        * TTM might be null for moves within the same region.
-        */
-       if (ttm) {
+       if (ttm && ((ttm->page_flags & TTM_PAGE_FLAG_SWAPPED) ||
+                   dst_man->use_tt)) {
                 ret = ttm_tt_populate(bdev, ttm, ctx);
                 if (ret)
-                       goto out1;
+                       return ret;
         }
 
-       for (i = 0; i < new_mem->num_pages; ++i) {
-               if (old_iomap == NULL) {
-                       pgprot_t prot = ttm_io_prot(bo, old_mem, PAGE_KERNEL);
-                       ret = ttm_copy_ttm_io_page(ttm, new_iomap, i,
-                                                  prot);
-               } else if (new_iomap == NULL) {
-                       pgprot_t prot = ttm_io_prot(bo, new_mem, PAGE_KERNEL);
-                       ret = ttm_copy_io_ttm_page(ttm, old_iomap, i,
-                                                  prot);
-               } else {
-                       ret = ttm_copy_io_page(new_iomap, old_iomap, i);
-               }
-               if (ret)
-                       goto out1;
+       dst_iter = ttm_kmap_iter_linear_io_init(&_dst_iter.io, bdev, dst_mem);
+       if (PTR_ERR(dst_iter) == -EINVAL && dst_man->use_tt)
+               dst_iter = ttm_kmap_iter_tt_init(&_dst_iter.tt, bo->ttm);
+       if (IS_ERR(dst_iter))
+               return PTR_ERR(dst_iter);
+
+       src_iter = ttm_kmap_iter_linear_io_init(&_src_iter.io, bdev, src_mem);
+       if (PTR_ERR(src_iter) == -EINVAL && src_man->use_tt)
+               src_iter = ttm_kmap_iter_tt_init(&_src_iter.tt, bo->ttm);
+       if (IS_ERR(src_iter)) {
+               ret = PTR_ERR(src_iter);
+               goto out_src_iter;
         }
-       mb();
-out2:
-       old_copy = *old_mem;
 
-       ttm_bo_assign_mem(bo, new_mem);
-
-       if (!man->use_tt)
-               ttm_bo_tt_destroy(bo);
+       ttm_move_memcpy(bo, dst_mem, dst_iter, src_iter);
+       src_copy = *src_mem;
+       ttm_bo_move_sync_cleanup(bo, dst_mem);
 
-out1:
-       ttm_resource_iounmap(bdev, old_mem, new_iomap);
-out:
-       ttm_resource_iounmap(bdev, &old_copy, old_iomap);
+       if (!src_iter->ops->maps_tt)
+               ttm_kmap_iter_linear_io_fini(&_src_iter.io, bdev, &src_copy);
+out_src_iter:
+       if (!dst_iter->ops->maps_tt)
+               ttm_kmap_iter_linear_io_fini(&_dst_iter.io, bdev, dst_mem);
 
-       /*
-        * On error, keep the mm node!
-        */
-       if (!ret)
-               ttm_resource_free(bo, &old_copy);
         return ret;
  }
  EXPORT_SYMBOL(ttm_bo_move_memcpy);
@@ -336,27 +272,7 @@ pgprot_t ttm_io_prot(struct ttm_buffer_object *bo, struct ttm_resource *res,
         man = ttm_manager_type(bo->bdev, res->mem_type);
         caching = man->use_tt ? bo->ttm->caching : res->bus.caching;
 
-       /* Cached mappings need no adjustment */
-       if (caching == ttm_cached)
-               return tmp;
-
-#if defined(__i386__) || defined(__x86_64__)
-       if (caching == ttm_write_combined)
-               tmp = pgprot_writecombine(tmp);
-       else if (boot_cpu_data.x86 > 3)
-               tmp = pgprot_noncached(tmp);
-#endif
-#if defined(__ia64__) || defined(__arm__) || defined(__aarch64__) || \
-    defined(__powerpc__) || defined(__mips__)
-       if (caching == ttm_write_combined)
-               tmp = pgprot_writecombine(tmp);
-       else
-               tmp = pgprot_noncached(tmp);
-#endif
-#if defined(__sparc__)
-       tmp = pgprot_noncached(tmp);
-#endif
-       return tmp;
+       return ttm_prot_from_caching(caching, tmp);
  }
  EXPORT_SYMBOL(ttm_io_prot);
 
drivers/gpu/drm/ttm/ttm_module.c +35 -0
index 56b0efdba1a9..997c458f68a9 100644
@@ -31,12 +31,47 @@
   */
  #include <linux/module.h>
  #include <linux/device.h>
+#include <linux/pgtable.h>
  #include <linux/sched.h>
  #include <linux/debugfs.h>
  #include <drm/drm_sysfs.h>
+#include <drm/ttm/ttm_caching.h>
 
  #include "ttm_module.h"
 
+/**
+ * ttm_prot_from_caching - Modify the page protection according to the
+ * ttm cacing mode
+ * @caching: The ttm caching mode
+ * @tmp: The original page protection
+ *
+ * Return: The modified page protection
+ */
+pgprot_t ttm_prot_from_caching(enum ttm_caching caching, pgprot_t tmp)
+{
+       /* Cached mappings need no adjustment */
+       if (caching == ttm_cached)
+               return tmp;
+
+#if defined(__i386__) || defined(__x86_64__)
+       if (caching == ttm_write_combined)
+               tmp = pgprot_writecombine(tmp);
+       else if (boot_cpu_data.x86 > 3)
+               tmp = pgprot_noncached(tmp);
+#endif
+#if defined(__ia64__) || defined(__arm__) || defined(__aarch64__) || \
+       defined(__powerpc__) || defined(__mips__)
+       if (caching == ttm_write_combined)
+               tmp = pgprot_writecombine(tmp);
+       else
+               tmp = pgprot_noncached(tmp);
+#endif
+#if defined(__sparc__)
+       tmp = pgprot_noncached(tmp);
+#endif
+       return tmp;
+}
+
  struct dentry *ttm_debugfs_root;
 
  static int __init ttm_init(void)
drivers/gpu/drm/ttm/ttm_resource.c +128 -0
index 59e2b7157e41..e05ae7e3d477 100644
@@ -22,6 +22,10 @@
   * Authors: Christian König
   */
 
+#include <linux/dma-buf-map.h>
+#include <linux/io-mapping.h>
+#include <linux/scatterlist.h>
+
  #include <drm/ttm/ttm_resource.h>
  #include <drm/ttm/ttm_bo_driver.h>
 
@@ -147,3 +151,165 @@ void ttm_resource_manager_debug(struct ttm_resource_manager *man,
                 man->func->debug(man, p);
  }
  EXPORT_SYMBOL(ttm_resource_manager_debug);
+
+static void ttm_kmap_iter_iomap_map_local(struct ttm_kmap_iter *iter,
+                                         struct dma_buf_map *dmap,
+                                         pgoff_t i)
+{
+       struct ttm_kmap_iter_iomap *iter_io =
+               container_of(iter, typeof(*iter_io), base);
+       void __iomem *addr;
+
+retry:
+       while (i >= iter_io->cache.end) {
+               iter_io->cache.sg = iter_io->cache.sg ?
+                       sg_next(iter_io->cache.sg) : iter_io->st->sgl;
+               iter_io->cache.i = iter_io->cache.end;
+               iter_io->cache.end += sg_dma_len(iter_io->cache.sg) >>
+                       PAGE_SHIFT;
+               iter_io->cache.offs = sg_dma_address(iter_io->cache.sg) -
+                       iter_io->start;
+       }
+
+       if (i < iter_io->cache.i) {
+               iter_io->cache.end = 0;
+               iter_io->cache.sg = NULL;
+               goto retry;
+       }
+
+       addr = io_mapping_map_local_wc(iter_io->iomap, iter_io->cache.offs +
+                                      (((resource_size_t)i - iter_io->cache.i)
+                                       << PAGE_SHIFT));
+       dma_buf_map_set_vaddr_iomem(dmap, addr);
+}
+
+static void ttm_kmap_iter_iomap_unmap_local(struct ttm_kmap_iter *iter,
+                                           struct dma_buf_map *map)
+{
+       io_mapping_unmap_local(map->vaddr_iomem);
+}
+
+static const struct ttm_kmap_iter_ops ttm_kmap_iter_io_ops = {
+       .map_local =  ttm_kmap_iter_iomap_map_local,
+       .unmap_local = ttm_kmap_iter_iomap_unmap_local,
+       .maps_tt = false,
+};
+
+/**
+ * ttm_kmap_iter_iomap_init - Initialize a struct ttm_kmap_iter_iomap
+ * @iter_io: The struct ttm_kmap_iter_iomap to initialize.
+ * @iomap: The struct io_mapping representing the underlying linear io_memory.
+ * @st: sg_table into @iomap, representing the memory of the struct
+ * ttm_resource.
+ * @start: Offset that needs to be subtracted from @st to make
+ * sg_dma_address(st->sgl) - @start == 0 for @iomap start.
+ *
+ * Return: Pointer to the embedded struct ttm_kmap_iter.
+ */
+struct ttm_kmap_iter *
+ttm_kmap_iter_iomap_init(struct ttm_kmap_iter_iomap *iter_io,
+                        struct io_mapping *iomap,
+                        struct sg_table *st,
+                        resource_size_t start)
+{
+       iter_io->base.ops = &ttm_kmap_iter_io_ops;
+       iter_io->iomap = iomap;
+       iter_io->st = st;
+       iter_io->start = start;
+       memset(&iter_io->cache, 0, sizeof(iter_io->cache));
+
+       return &iter_io->base;
+}
+EXPORT_SYMBOL(ttm_kmap_iter_iomap_init);
+
+/**
+ * DOC: Linear io iterator
+ *
+ * This code should die in the not too near future. Best would be if we could
+ * make io-mapping use memremap for all io memory, and have memremap
+ * implement a kmap_local functionality. We could then strip a huge amount of
+ * code. These linear io iterators are implemented to mimic old functionality,
+ * and they don't use kmap_local semantics at all internally. Rather ioremap or
+ * friends, and at least on 32-bit they add global TLB flushes and points
+ * of failure.
+ */
+
+static void ttm_kmap_iter_linear_io_map_local(struct ttm_kmap_iter *iter,
+                                             struct dma_buf_map *dmap,
+                                             pgoff_t i)
+{
+       struct ttm_kmap_iter_linear_io *iter_io =
+               container_of(iter, typeof(*iter_io), base);
+
+       *dmap = iter_io->dmap;
+       dma_buf_map_incr(dmap, i * PAGE_SIZE);
+}
+
+static const struct ttm_kmap_iter_ops ttm_kmap_iter_linear_io_ops = {
+       .map_local =  ttm_kmap_iter_linear_io_map_local,
+       .maps_tt = false,
+};
+
+struct ttm_kmap_iter *
+ttm_kmap_iter_linear_io_init(struct ttm_kmap_iter_linear_io *iter_io,
+                            struct ttm_device *bdev,
+                            struct ttm_resource *mem)
+{
+       int ret;
+
+       ret = ttm_mem_io_reserve(bdev, mem);
+       if (ret)
+               goto out_err;
+       if (!mem->bus.is_iomem) {
+               ret = -EINVAL;
+               goto out_io_free;
+       }
+
+       if (mem->bus.addr) {
+               dma_buf_map_set_vaddr(&iter_io->dmap, mem->bus.addr);
+               iter_io->needs_unmap = false;
+       } else {
+               size_t bus_size = (size_t)mem->num_pages << PAGE_SHIFT;
+
+               iter_io->needs_unmap = true;
+               if (mem->bus.caching == ttm_write_combined)
+                       dma_buf_map_set_vaddr_iomem(&iter_io->dmap,
+                                                   ioremap_wc(mem->bus.offset,
Quoted 8 lines

drivers/gpu/drm/ttm/ttm_resource.c void ttm_resource_manager_debug(struct ttm_resource_manager *man, · line 282

Quoted 5 lines
drivers/gpu/drm/ttm/ttm_resource.c +5 -0
+                                                              bus_size));
+               else if (mem->bus.caching == ttm_cached)
+                       dma_buf_map_set_vaddr(&iter_io->dmap,
+                                             memremap(mem->bus.offset, bus_size,
+                                                      MEMREMAP_WB));

The comments in set_vaddr suggest that this is meant for system-memory. Does that actually matter or is it just about not losing the __iomem annotation on platforms where it matters?

Yes, it's the latter. dma_buf_map() is relatively new and the author probably didn't think about the case of cached iomem, which is used by, for example, vmwgfx.

Quoted 2 lines

Apparently cached device local is a thing. Also should this not be wrapped in CONFIG_X86?

Both dma_buf_map() and memremap are generic, I think, I guess memremap would return NULL if it's not supported.

/Thomas

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#9 lore raw

Re: [Intel-gfx] [PATCH v3 06/12] drm/ttm: Add a generic TTM memcpy move for page-based iomem

Quoted 579 lines · Thomas Hellström
Quoted 199 lines · Matthew Auld
Quoted 197 lines · Thomas Hellström

The internal ttm_bo_util memcpy uses ioremap functionality, and while it probably might be possible to use it for copying in- and out of sglist represented io memory, using io_mem_reserve() / io_mem_free() callbacks, that would cause problems with fault().
Instead, implement a method mapping page-by-page using kmap_local() semantics. As an additional benefit we then avoid the occasional global TLB flushes of ioremap() and consuming ioremap space, elimination of a critical point of failure and with a slight change of semantics we could also push the memcpy out async for testing and async driver development purposes.

A special linear iomem iterator is introduced internally to mimic the old ioremap behaviour for code-paths that can't immediately be ported over. This adds to the code size and should be considered a temporary solution.

Looking at the code we have a lot of checks for iomap tagged pointers. Ideally we should extend the core memremap functions to also accept uncached memory and kmap_local functionality. Then we could strip a lot of code.

---
v3:
- Split up in various TTM files and addressed review comments by Christian König. Tested and fixed legacy iomap memcpy path on i915.

---
  drivers/gpu/drm/ttm/ttm_bo_util.c  | 278 ++++++++++-------------------
  drivers/gpu/drm/ttm/ttm_module.c   |  35 ++++
  drivers/gpu/drm/ttm/ttm_resource.c | 166 +++++++++++++++++
  drivers/gpu/drm/ttm/ttm_tt.c       |  42 +++++
  include/drm/ttm/ttm_bo_driver.h    |  28 +++
  include/drm/ttm/ttm_caching.h      |   2 +
  include/drm/ttm/ttm_kmap_iter.h    |  61 +++++++
  include/drm/ttm/ttm_resource.h     |  61 +++++++
  include/drm/ttm/ttm_tt.h           |  16 ++
  9 files changed, 508 insertions(+), 181 deletions(-)

create mode 100644 include/drm/ttm/ttm_kmap_iter.h

drivers/gpu/drm/ttm/ttm_bo_util.c +52 -90
index ae8b61460724..912cbe8e60a2 100644
@@ -72,190 +72,126 @@ void ttm_mem_io_free(struct ttm_device *bdev,
         mem->bus.addr = NULL;
  }
 
-static int ttm_resource_ioremap(struct ttm_device *bdev,
-                              struct ttm_resource *mem,
-                              void **virtual)
+/**
+ * ttm_move_memcpy - Helper to perform a memcpy ttm move operation.
+ * @bo: The struct ttm_buffer_object.
+ * @new_mem: The struct ttm_resource we're moving to (copy destination).
+ * @new_iter: A struct ttm_kmap_iter representing the destination resource.
+ * @src_iter: A struct ttm_kmap_iter representing the source resource.
+ *
+ * This function is intended to be able to move out async under a
+ * dma-fence if desired.
+ */
+void ttm_move_memcpy(struct ttm_buffer_object *bo,
+                    struct ttm_resource *dst_mem,
+                    struct ttm_kmap_iter *dst_iter,
+                    struct ttm_kmap_iter *src_iter)
  {
-       int ret;
-       void *addr;
-
-       *virtual = NULL;
-       ret = ttm_mem_io_reserve(bdev, mem);
-       if (ret || !mem->bus.is_iomem)
-               return ret;
+       const struct ttm_kmap_iter_ops *dst_ops = dst_iter->ops;
+       const struct ttm_kmap_iter_ops *src_ops = src_iter->ops;
+       struct ttm_tt *ttm = bo->ttm;
+       struct dma_buf_map src_map, dst_map;
+       pgoff_t i;
 
-       if (mem->bus.addr) {
-               addr = mem->bus.addr;
-       } else {
-               size_t bus_size = (size_t)mem->num_pages << PAGE_SHIFT;
+       /* Single TTM move. NOP */
+       if (dst_ops->maps_tt && src_ops->maps_tt)
+               return;
 
-               if (mem->bus.caching == ttm_write_combined)
-                       addr = ioremap_wc(mem->bus.offset, bus_size);
-#ifdef CONFIG_X86
-               else if (mem->bus.caching == ttm_cached)
-                       addr = ioremap_cache(mem->bus.offset, bus_size);
-#endif
-               else
-                       addr = ioremap(mem->bus.offset, bus_size);
-               if (!addr) {
-                       ttm_mem_io_free(bdev, mem);
-                       return -ENOMEM;
+       /* Don't move nonexistent data. Clear destination instead. */
+       if (src_ops->maps_tt && (!ttm || !ttm_tt_is_populated(ttm))) {
+               if (ttm && !(ttm->page_flags & TTM_PAGE_FLAG_ZERO_ALLOC))
+                       return;
+
+               for (i = 0; i < dst_mem->num_pages; ++i) {
+                       dst_ops->map_local(dst_iter, &dst_map, i);
+                       if (dst_map.is_iomem)
+                               memset_io(dst_map.vaddr_iomem, 0, PAGE_SIZE);
+                       else
+                               memset(dst_map.vaddr, 0, PAGE_SIZE);
+                       if (dst_ops->unmap_local)
+                               dst_ops->unmap_local(dst_iter, &dst_map);
                 }
+               return;
         }
-       *virtual = addr;
-       return 0;
-}
-
-static void ttm_resource_iounmap(struct ttm_device *bdev,
-                               struct ttm_resource *mem,
-                               void *virtual)
-{
-       if (virtual && mem->bus.addr == NULL)
-               iounmap(virtual);
-       ttm_mem_io_free(bdev, mem);
-}
-
-static int ttm_copy_io_page(void *dst, void *src, unsigned long page)
-{
-       uint32_t *dstP =
-           (uint32_t *) ((unsigned long)dst + (page << PAGE_SHIFT));
-       uint32_t *srcP =
-           (uint32_t *) ((unsigned long)src + (page << PAGE_SHIFT));
-
-       int i;
-       for (i = 0; i < PAGE_SIZE / sizeof(uint32_t); ++i)
-               iowrite32(ioread32(srcP++), dstP++);
-       return 0;
-}
-
-static int ttm_copy_io_ttm_page(struct ttm_tt *ttm, void *src,
-                               unsigned long page,
-                               pgprot_t prot)
-{
-       struct page *d = ttm->pages[page];
-       void *dst;
-
-       if (!d)
-               return -ENOMEM;
-
-       src = (void *)((unsigned long)src + (page << PAGE_SHIFT));
-       dst = kmap_atomic_prot(d, prot);
-       if (!dst)
-               return -ENOMEM;
-
-       memcpy_fromio(dst, src, PAGE_SIZE);
-
-       kunmap_atomic(dst);
-
-       return 0;
-}
-
-static int ttm_copy_ttm_io_page(struct ttm_tt *ttm, void *dst,
-                               unsigned long page,
-                               pgprot_t prot)
-{
-       struct page *s = ttm->pages[page];
-       void *src;
-
-       if (!s)
-               return -ENOMEM;
-
-       dst = (void *)((unsigned long)dst + (page << PAGE_SHIFT));
-       src = kmap_atomic_prot(s, prot);
-       if (!src)
-               return -ENOMEM;
 
-       memcpy_toio(dst, src, PAGE_SIZE);
-
-       kunmap_atomic(src);
+       for (i = 0; i < dst_mem->num_pages; ++i) {
+               dst_ops->map_local(dst_iter, &dst_map, i);
+               src_ops->map_local(src_iter, &src_map, i);
+
+               if (!src_map.is_iomem && !dst_map.is_iomem) {
+                       memcpy(dst_map.vaddr, src_map.vaddr, PAGE_SIZE);
+               } else if (!src_map.is_iomem) {
+                       dma_buf_map_memcpy_to(&dst_map, src_map.vaddr,
+                                             PAGE_SIZE);
+               } else if (!dst_map.is_iomem) {
+                       memcpy_fromio(dst_map.vaddr, src_map.vaddr_iomem,
+                                     PAGE_SIZE);
+               } else {
+                       int j;
+                       u32 __iomem *src = src_map.vaddr_iomem;
+                       u32 __iomem *dst = dst_map.vaddr_iomem;
 
Quoted 3 lines

drivers/gpu/drm/ttm/ttm_bo_util.c void ttm_mem_io_free(struct ttm_device *bdev, · line 134

Quoted 2 lines
drivers/gpu/drm/ttm/ttm_bo_util.c +1 -1
-       return 0;
+                       for (j = 0; j < (PAGE_SIZE >> 2); ++j)

IMO PAGE_SIZE / sizeof(u32) is easier to understand.

OK, will fix.

Quoted 357 lines
Quoted 357 lines
drivers/gpu/drm/ttm/ttm_bo_util.c +44 -90
+                               iowrite32(ioread32(src++), dst++);
+               }
+               if (src_ops->unmap_local)
+                       src_ops->unmap_local(src_iter, &src_map);
+               if (dst_ops->unmap_local)
+                       dst_ops->unmap_local(dst_iter, &dst_map);
+       }
  }
+EXPORT_SYMBOL(ttm_move_memcpy);
 
  int ttm_bo_move_memcpy(struct ttm_buffer_object *bo,
                        struct ttm_operation_ctx *ctx,
-                      struct ttm_resource *new_mem)
+                      struct ttm_resource *dst_mem)
  {
         struct ttm_device *bdev = bo->bdev;
-       struct ttm_resource_manager *man = ttm_manager_type(bdev, new_mem->mem_type);
+       struct ttm_resource_manager *dst_man =
+               ttm_manager_type(bo->bdev, dst_mem->mem_type);
         struct ttm_tt *ttm = bo->ttm;
-       struct ttm_resource *old_mem = &bo->mem;
-       struct ttm_resource old_copy = *old_mem;
-       void *old_iomap;
-       void *new_iomap;
+       struct ttm_resource *src_mem = &bo->mem;
+       struct ttm_resource_manager *src_man =
+               ttm_manager_type(bdev, src_mem->mem_type);
+       struct ttm_resource src_copy = *src_mem;
+       union {
+               struct ttm_kmap_iter_tt tt;
+               struct ttm_kmap_iter_linear_io io;
+       } _dst_iter, _src_iter;
+       struct ttm_kmap_iter *dst_iter, *src_iter;
         int ret;
-       unsigned long i;
 
-       ret = ttm_bo_wait_ctx(bo, ctx);
-       if (ret)
-               return ret;
-
-       ret = ttm_resource_ioremap(bdev, old_mem, &old_iomap);
-       if (ret)
-               return ret;
-       ret = ttm_resource_ioremap(bdev, new_mem, &new_iomap);
-       if (ret)
-               goto out;
-
-       /*
-        * Single TTM move. NOP.
-        */
-       if (old_iomap == NULL && new_iomap == NULL)
-               goto out2;
-
-       /*
-        * Don't move nonexistent data. Clear destination instead.
-        */
-       if (old_iomap == NULL &&
-           (ttm == NULL || (!ttm_tt_is_populated(ttm) &&
-                            !(ttm->page_flags & TTM_PAGE_FLAG_SWAPPED)))) {
-               memset_io(new_iomap, 0, new_mem->num_pages*PAGE_SIZE);
-               goto out2;
-       }
-
-       /*
-        * TTM might be null for moves within the same region.
-        */
-       if (ttm) {
+       if (ttm && ((ttm->page_flags & TTM_PAGE_FLAG_SWAPPED) ||
+                   dst_man->use_tt)) {
                 ret = ttm_tt_populate(bdev, ttm, ctx);
                 if (ret)
-                       goto out1;
+                       return ret;
         }
 
-       for (i = 0; i < new_mem->num_pages; ++i) {
-               if (old_iomap == NULL) {
-                       pgprot_t prot = ttm_io_prot(bo, old_mem, PAGE_KERNEL);
-                       ret = ttm_copy_ttm_io_page(ttm, new_iomap, i,
-                                                  prot);
-               } else if (new_iomap == NULL) {
-                       pgprot_t prot = ttm_io_prot(bo, new_mem, PAGE_KERNEL);
-                       ret = ttm_copy_io_ttm_page(ttm, old_iomap, i,
-                                                  prot);
-               } else {
-                       ret = ttm_copy_io_page(new_iomap, old_iomap, i);
-               }
-               if (ret)
-                       goto out1;
+       dst_iter = ttm_kmap_iter_linear_io_init(&_dst_iter.io, bdev, dst_mem);
+       if (PTR_ERR(dst_iter) == -EINVAL && dst_man->use_tt)
+               dst_iter = ttm_kmap_iter_tt_init(&_dst_iter.tt, bo->ttm);
+       if (IS_ERR(dst_iter))
+               return PTR_ERR(dst_iter);
+
+       src_iter = ttm_kmap_iter_linear_io_init(&_src_iter.io, bdev, src_mem);
+       if (PTR_ERR(src_iter) == -EINVAL && src_man->use_tt)
+               src_iter = ttm_kmap_iter_tt_init(&_src_iter.tt, bo->ttm);
+       if (IS_ERR(src_iter)) {
+               ret = PTR_ERR(src_iter);
+               goto out_src_iter;
         }
-       mb();
-out2:
-       old_copy = *old_mem;
 
-       ttm_bo_assign_mem(bo, new_mem);
-
-       if (!man->use_tt)
-               ttm_bo_tt_destroy(bo);
+       ttm_move_memcpy(bo, dst_mem, dst_iter, src_iter);
+       src_copy = *src_mem;
+       ttm_bo_move_sync_cleanup(bo, dst_mem);
 
-out1:
-       ttm_resource_iounmap(bdev, old_mem, new_iomap);
-out:
-       ttm_resource_iounmap(bdev, &old_copy, old_iomap);
+       if (!src_iter->ops->maps_tt)
+               ttm_kmap_iter_linear_io_fini(&_src_iter.io, bdev, &src_copy);
+out_src_iter:
+       if (!dst_iter->ops->maps_tt)
+               ttm_kmap_iter_linear_io_fini(&_dst_iter.io, bdev, dst_mem);
 
-       /*
-        * On error, keep the mm node!
-        */
-       if (!ret)
-               ttm_resource_free(bo, &old_copy);
         return ret;
  }
  EXPORT_SYMBOL(ttm_bo_move_memcpy);
@@ -336,27 +272,7 @@ pgprot_t ttm_io_prot(struct ttm_buffer_object *bo, struct ttm_resource *res,
         man = ttm_manager_type(bo->bdev, res->mem_type);
         caching = man->use_tt ? bo->ttm->caching : res->bus.caching;
 
-       /* Cached mappings need no adjustment */
-       if (caching == ttm_cached)
-               return tmp;
-
-#if defined(__i386__) || defined(__x86_64__)
-       if (caching == ttm_write_combined)
-               tmp = pgprot_writecombine(tmp);
-       else if (boot_cpu_data.x86 > 3)
-               tmp = pgprot_noncached(tmp);
-#endif
-#if defined(__ia64__) || defined(__arm__) || defined(__aarch64__) || \
-    defined(__powerpc__) || defined(__mips__)
-       if (caching == ttm_write_combined)
-               tmp = pgprot_writecombine(tmp);
-       else
-               tmp = pgprot_noncached(tmp);
-#endif
-#if defined(__sparc__)
-       tmp = pgprot_noncached(tmp);
-#endif
-       return tmp;
+       return ttm_prot_from_caching(caching, tmp);
  }
  EXPORT_SYMBOL(ttm_io_prot);
 
drivers/gpu/drm/ttm/ttm_module.c +35 -0
index 56b0efdba1a9..997c458f68a9 100644
@@ -31,12 +31,47 @@
   */
  #include <linux/module.h>
  #include <linux/device.h>
+#include <linux/pgtable.h>
  #include <linux/sched.h>
  #include <linux/debugfs.h>
  #include <drm/drm_sysfs.h>
+#include <drm/ttm/ttm_caching.h>
 
  #include "ttm_module.h"
 
+/**
+ * ttm_prot_from_caching - Modify the page protection according to the
+ * ttm cacing mode
+ * @caching: The ttm caching mode
+ * @tmp: The original page protection
+ *
+ * Return: The modified page protection
+ */
+pgprot_t ttm_prot_from_caching(enum ttm_caching caching, pgprot_t tmp)
+{
+       /* Cached mappings need no adjustment */
+       if (caching == ttm_cached)
+               return tmp;
+
+#if defined(__i386__) || defined(__x86_64__)
+       if (caching == ttm_write_combined)
+               tmp = pgprot_writecombine(tmp);
+       else if (boot_cpu_data.x86 > 3)
+               tmp = pgprot_noncached(tmp);
+#endif
+#if defined(__ia64__) || defined(__arm__) || defined(__aarch64__) || \
+       defined(__powerpc__) || defined(__mips__)
+       if (caching == ttm_write_combined)
+               tmp = pgprot_writecombine(tmp);
+       else
+               tmp = pgprot_noncached(tmp);
+#endif
+#if defined(__sparc__)
+       tmp = pgprot_noncached(tmp);
+#endif
+       return tmp;
+}
+
  struct dentry *ttm_debugfs_root;
 
  static int __init ttm_init(void)
drivers/gpu/drm/ttm/ttm_resource.c +128 -0
index 59e2b7157e41..e05ae7e3d477 100644
@@ -22,6 +22,10 @@
   * Authors: Christian König
   */
 
+#include <linux/dma-buf-map.h>
+#include <linux/io-mapping.h>
+#include <linux/scatterlist.h>
+
  #include <drm/ttm/ttm_resource.h>
  #include <drm/ttm/ttm_bo_driver.h>
 
@@ -147,3 +151,165 @@ void ttm_resource_manager_debug(struct ttm_resource_manager *man,
                 man->func->debug(man, p);
  }
  EXPORT_SYMBOL(ttm_resource_manager_debug);
+
+static void ttm_kmap_iter_iomap_map_local(struct ttm_kmap_iter *iter,
+                                         struct dma_buf_map *dmap,
+                                         pgoff_t i)
+{
+       struct ttm_kmap_iter_iomap *iter_io =
+               container_of(iter, typeof(*iter_io), base);
+       void __iomem *addr;
+
+retry:
+       while (i >= iter_io->cache.end) {
+               iter_io->cache.sg = iter_io->cache.sg ?
+                       sg_next(iter_io->cache.sg) : iter_io->st->sgl;
+               iter_io->cache.i = iter_io->cache.end;
+               iter_io->cache.end += sg_dma_len(iter_io->cache.sg) >>
+                       PAGE_SHIFT;
+               iter_io->cache.offs = sg_dma_address(iter_io->cache.sg) -
+                       iter_io->start;
+       }
+
+       if (i < iter_io->cache.i) {
+               iter_io->cache.end = 0;
+               iter_io->cache.sg = NULL;
+               goto retry;
+       }
+
+       addr = io_mapping_map_local_wc(iter_io->iomap, iter_io->cache.offs +
+                                      (((resource_size_t)i - iter_io->cache.i)
+                                       << PAGE_SHIFT));
+       dma_buf_map_set_vaddr_iomem(dmap, addr);
+}
+
+static void ttm_kmap_iter_iomap_unmap_local(struct ttm_kmap_iter *iter,
+                                           struct dma_buf_map *map)
+{
+       io_mapping_unmap_local(map->vaddr_iomem);
+}
+
+static const struct ttm_kmap_iter_ops ttm_kmap_iter_io_ops = {
+       .map_local =  ttm_kmap_iter_iomap_map_local,
+       .unmap_local = ttm_kmap_iter_iomap_unmap_local,
+       .maps_tt = false,
+};
+
+/**
+ * ttm_kmap_iter_iomap_init - Initialize a struct ttm_kmap_iter_iomap
+ * @iter_io: The struct ttm_kmap_iter_iomap to initialize.
+ * @iomap: The struct io_mapping representing the underlying linear io_memory.
+ * @st: sg_table into @iomap, representing the memory of the struct
+ * ttm_resource.
+ * @start: Offset that needs to be subtracted from @st to make
+ * sg_dma_address(st->sgl) - @start == 0 for @iomap start.
+ *
+ * Return: Pointer to the embedded struct ttm_kmap_iter.
+ */
+struct ttm_kmap_iter *
+ttm_kmap_iter_iomap_init(struct ttm_kmap_iter_iomap *iter_io,
+                        struct io_mapping *iomap,
+                        struct sg_table *st,
+                        resource_size_t start)
+{
+       iter_io->base.ops = &ttm_kmap_iter_io_ops;
+       iter_io->iomap = iomap;
+       iter_io->st = st;
+       iter_io->start = start;
+       memset(&iter_io->cache, 0, sizeof(iter_io->cache));
+
+       return &iter_io->base;
+}
+EXPORT_SYMBOL(ttm_kmap_iter_iomap_init);
+
+/**
+ * DOC: Linear io iterator
+ *
+ * This code should die in the not too near future. Best would be if we could
+ * make io-mapping use memremap for all io memory, and have memremap
+ * implement a kmap_local functionality. We could then strip a huge amount of
+ * code. These linear io iterators are implemented to mimic old functionality,
+ * and they don't use kmap_local semantics at all internally. Rather ioremap or
+ * friends, and at least on 32-bit they add global TLB flushes and points
+ * of failure.
+ */
+
+static void ttm_kmap_iter_linear_io_map_local(struct ttm_kmap_iter *iter,
+                                             struct dma_buf_map *dmap,
+                                             pgoff_t i)
+{
+       struct ttm_kmap_iter_linear_io *iter_io =
+               container_of(iter, typeof(*iter_io), base);
+
+       *dmap = iter_io->dmap;
+       dma_buf_map_incr(dmap, i * PAGE_SIZE);
+}
+
+static const struct ttm_kmap_iter_ops ttm_kmap_iter_linear_io_ops = {
+       .map_local =  ttm_kmap_iter_linear_io_map_local,
+       .maps_tt = false,
+};
+
+struct ttm_kmap_iter *
+ttm_kmap_iter_linear_io_init(struct ttm_kmap_iter_linear_io *iter_io,
+                            struct ttm_device *bdev,
+                            struct ttm_resource *mem)
+{
+       int ret;
+
+       ret = ttm_mem_io_reserve(bdev, mem);
+       if (ret)
+               goto out_err;
+       if (!mem->bus.is_iomem) {
+               ret = -EINVAL;
+               goto out_io_free;
+       }
+
+       if (mem->bus.addr) {
+               dma_buf_map_set_vaddr(&iter_io->dmap, mem->bus.addr);
+               iter_io->needs_unmap = false;
+       } else {
+               size_t bus_size = (size_t)mem->num_pages << PAGE_SHIFT;
+
+               iter_io->needs_unmap = true;
+               if (mem->bus.caching == ttm_write_combined)
+                       dma_buf_map_set_vaddr_iomem(&iter_io->dmap,
+                                                   ioremap_wc(mem->bus.offset,
Quoted 8 lines

drivers/gpu/drm/ttm/ttm_resource.c void ttm_resource_manager_debug(struct ttm_resource_manager *man, · line 282

Quoted 5 lines
drivers/gpu/drm/ttm/ttm_resource.c +5 -0
+                                                              bus_size));
+               else if (mem->bus.caching == ttm_cached)
+                       dma_buf_map_set_vaddr(&iter_io->dmap,
+                                             memremap(mem->bus.offset, bus_size,
+                                                      MEMREMAP_WB));

The comments in set_vaddr suggest that this is meant for system-memory. Does that actually matter or is it just about not losing the __iomem annotation on platforms where it matters?

Yes, it's the latter. dma_buf_map() is relatively new and the author probably didn't think about the case of cached iomem, which is used by, for example, vmwgfx.

Quoted 5 lines
Quoted 2 lines

Apparently cached device local is a thing. Also should this not be wrapped in CONFIG_X86?

Both dma_buf_map() and memremap are generic, I think, I guess memremap would return NULL if it's not supported.

It looks like memremap just wraps ioremap_cache, but since it also discards the __iomem annotation should we be doing that universally? Also not sure if ioremap_cache is universally supported, so wrapping in CONFIG_X86 and falling back to plain ioremap() might be needed? Or at least that looks like roughly what the previous code was doing? Not too sure tbh.

Quoted 4 lines

/Thomas

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

author reviewer
#10 lore raw

Re: [Intel-gfx] [PATCH v3 06/12] drm/ttm: Add a generic TTM memcpy move for page-based iomem

Quoted 693 lines · Matthew Auld
Quoted 683 lines · Thomas Hellström
Quoted 244 lines · Matthew Auld
Quoted 242 lines · Thomas Hellström

The internal ttm_bo_util memcpy uses ioremap functionality, and while it
probably might be possible to use it for copying in- and out of sglist represented io memory, using io_mem_reserve() / io_mem_free()
callbacks, that would cause problems with fault(). Instead, implement a method mapping page-by-page using kmap_local()
semantics. As an additional benefit we then avoid the occasional global
TLB flushes of ioremap() and consuming ioremap space, elimination of a
critical point of failure and with a slight change of semantics we could
also push the memcpy out async for testing and async driver development
purposes.

A special linear iomem iterator is introduced internally to mimic the
old ioremap behaviour for code-paths that can't immediately be ported
over. This adds to the code size and should be considered a temporary
solution.

Looking at the code we have a lot of checks for iomap tagged pointers.
Ideally we should extend the core memremap functions to also accept
uncached memory and kmap_local functionality. Then we could strip a
lot of code.

Cc: Christian König <[email protected]>
Signed-off-by: Thomas Hellström < [email protected]>
---
v3:
- Split up in various TTM files and addressed review comments by
   Christian König. Tested and fixed legacy iomap memcpy path on i915.
---
  drivers/gpu/drm/ttm/ttm_bo_util.c  | 278 ++++++++++----------
---------
  drivers/gpu/drm/ttm/ttm_module.c   |  35 ++++ drivers/gpu/drm/ttm/ttm_resource.c | 166 +++++++++++++++++ drivers/gpu/drm/ttm/ttm_tt.c       |  42 +++++ include/drm/ttm/ttm_bo_driver.h    |  28 +++ include/drm/ttm/ttm_caching.h      |   2 + include/drm/ttm/ttm_kmap_iter.h    |  61 +++++++ include/drm/ttm/ttm_resource.h     |  61 +++++++ include/drm/ttm/ttm_tt.h           |  16 ++

  9 files changed, 508 insertions(+), 181 deletions(-)

  create mode 100644 include/drm/ttm/ttm_kmap_iter.h

drivers/gpu/drm/ttm/ttm_bo_util.c +2 -3
index ae8b61460724..912cbe8e60a2 100644
@@ -72,190 +72,126 @@ void ttm_mem_io_free(struct ttm_device
 *bdev,
          mem->bus.addr = NULL;
   }
 
-static int ttm_resource_ioremap(struct ttm_device *bdev,
-                              struct ttm_resource *mem,
-                              void **virtual)
+/**
+ * ttm_move_memcpy - Helper to perform a memcpy ttm move

operation.
+ * @bo: The struct ttm_buffer_object.
+ * @new_mem: The struct ttm_resource we're moving to (copy destination).
+ * @new_iter: A struct ttm_kmap_iter representing the destination resource.
+ * @src_iter: A struct ttm_kmap_iter representing the source resource.
+ *
+ * This function is intended to be able to move out async under a
+ * dma-fence if desired.
+ */
+void ttm_move_memcpy(struct ttm_buffer_object *bo, +                    struct ttm_resource *dst_mem, +                    struct ttm_kmap_iter *dst_iter, +                    struct ttm_kmap_iter *src_iter) {
-       int ret;
-       void *addr;
-
-       *virtual = NULL;
-       ret = ttm_mem_io_reserve(bdev, mem);
-       if (ret || !mem->bus.is_iomem)
-               return ret;
+       const struct ttm_kmap_iter_ops *dst_ops = dst_iter-

Quoted 1 line

ops;

+       const struct ttm_kmap_iter_ops *src_ops = src_iter-

Quoted 1 line

ops;

+       struct ttm_tt *ttm = bo->ttm;
+       struct dma_buf_map src_map, dst_map;
+       pgoff_t i;

-       if (mem->bus.addr) {
-               addr = mem->bus.addr;
-       } else {
-               size_t bus_size = (size_t)mem->num_pages << PAGE_SHIFT;
+       /* Single TTM move. NOP */
+       if (dst_ops->maps_tt && src_ops->maps_tt) +               return;

-               if (mem->bus.caching == ttm_write_combined) -                       addr = ioremap_wc(mem->bus.offset, bus_size);
-#ifdef CONFIG_X86
-               else if (mem->bus.caching == ttm_cached) -                       addr = ioremap_cache(mem->bus.offset, bus_size);
-#endif
-               else
-                       addr = ioremap(mem->bus.offset, bus_size);
-               if (!addr) {
-                       ttm_mem_io_free(bdev, mem); -                       return -ENOMEM;
+       /* Don't move nonexistent data. Clear destination instead. */
+       if (src_ops->maps_tt && (!ttm || !ttm_tt_is_populated(ttm))) {
+               if (ttm && !(ttm->page_flags & TTM_PAGE_FLAG_ZERO_ALLOC))
+                       return;
+
+               for (i = 0; i < dst_mem->num_pages; ++i) { +                       dst_ops->map_local(dst_iter, &dst_map, i);
+                       if (dst_map.is_iomem)
+                               memset_io(dst_map.vaddr_iomem, 0, PAGE_SIZE);
+                       else
+                               memset(dst_map.vaddr, 0, PAGE_SIZE);
+                       if (dst_ops->unmap_local)
+                               dst_ops->unmap_local(dst_iter, &dst_map);
                 }
+               return;
         }
-       *virtual = addr;
-       return 0;

drivers/gpu/drm/ttm/ttm_bo_util.c +0 -12
-}
-
-static void ttm_resource_iounmap(struct ttm_device *bdev,
-                               struct ttm_resource *mem,
-                               void *virtual)
-{
-       if (virtual && mem->bus.addr == NULL)
-               iounmap(virtual);
-       ttm_mem_io_free(bdev, mem);
-}
-
-static int ttm_copy_io_page(void *dst, void *src, unsigned

long page)
-{
-       uint32_t *dstP =
-           (uint32_t *) ((unsigned long)dst + (page << PAGE_SHIFT));
-       uint32_t *srcP =
-           (uint32_t *) ((unsigned long)src + (page << PAGE_SHIFT));

drivers/gpu/drm/ttm/ttm_bo_util.c +0 -18
-
-       int i;
-       for (i = 0; i < PAGE_SIZE / sizeof(uint32_t); ++i)
-               iowrite32(ioread32(srcP++), dstP++);
-       return 0;
-}
-
-static int ttm_copy_io_ttm_page(struct ttm_tt *ttm, void *src,
-                               unsigned long page,
-                               pgprot_t prot)
-{
-       struct page *d = ttm->pages[page];
-       void *dst;
-
-       if (!d)
-               return -ENOMEM;
-
-       src = (void *)((unsigned long)src + (page <<

PAGE_SHIFT));
-       dst = kmap_atomic_prot(d, prot);
-       if (!dst)
-               return -ENOMEM;

drivers/gpu/drm/ttm/ttm_bo_util.c +0 -19
-
-       memcpy_fromio(dst, src, PAGE_SIZE);
-
-       kunmap_atomic(dst);
-
-       return 0;
-}
-
-static int ttm_copy_ttm_io_page(struct ttm_tt *ttm, void *dst,
-                               unsigned long page,
-                               pgprot_t prot)
-{
-       struct page *s = ttm->pages[page];
-       void *src;
-
-       if (!s)
-               return -ENOMEM;
-
-       dst = (void *)((unsigned long)dst + (page <<

PAGE_SHIFT));
-       src = kmap_atomic_prot(s, prot);
-       if (!src)
-               return -ENOMEM;

-       memcpy_toio(dst, src, PAGE_SIZE);

drivers/gpu/drm/ttm/ttm_bo_util.c +6 -2
-
-       kunmap_atomic(src);
+       for (i = 0; i < dst_mem->num_pages; ++i) {
+               dst_ops->map_local(dst_iter, &dst_map, i);
+               src_ops->map_local(src_iter, &src_map, i);
+
+               if (!src_map.is_iomem && !dst_map.is_iomem) {
+                       memcpy(dst_map.vaddr, src_map.vaddr,

PAGE_SIZE);
+               } else if (!src_map.is_iomem) {
+                       dma_buf_map_memcpy_to(&dst_map, src_map.vaddr,
+                                             PAGE_SIZE); +               } else if (!dst_map.is_iomem) {
+                       memcpy_fromio(dst_map.vaddr, src_map.vaddr_iomem,
+                                     PAGE_SIZE);
+               } else {
+                       int j;
+                       u32 __iomem *src = src_map.vaddr_iomem; +                       u32 __iomem *dst = dst_map.vaddr_iomem;

Quoted 3 lines
Quoted 2 lines

-       return 0;
+                       for (j = 0; j < (PAGE_SIZE >> 2); ++j)

IMO PAGE_SIZE / sizeof(u32) is easier to understand.

OK, will fix.

Quoted 416 lines
Quoted 416 lines

+                               iowrite32(ioread32(src++), dst++);
+               }
+               if (src_ops->unmap_local)
+                       src_ops->unmap_local(src_iter, &src_map);
+               if (dst_ops->unmap_local)
+                       dst_ops->unmap_local(dst_iter, &dst_map);
+       }
  }
+EXPORT_SYMBOL(ttm_move_memcpy);

  int ttm_bo_move_memcpy(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx, -                      struct ttm_resource *new_mem) +                      struct ttm_resource *dst_mem) {
         struct ttm_device *bdev = bo->bdev;
-       struct ttm_resource_manager *man = ttm_manager_type(bdev, new_mem->mem_type);
+       struct ttm_resource_manager *dst_man =
+               ttm_manager_type(bo->bdev, dst_mem->mem_type); struct ttm_tt *ttm = bo->ttm;
-       struct ttm_resource *old_mem = &bo->mem;
-       struct ttm_resource old_copy = *old_mem;
-       void *old_iomap;
-       void *new_iomap;
+       struct ttm_resource *src_mem = &bo->mem;
+       struct ttm_resource_manager *src_man =
+               ttm_manager_type(bdev, src_mem->mem_type); +       struct ttm_resource src_copy = *src_mem;
+       union {
+               struct ttm_kmap_iter_tt tt;
+               struct ttm_kmap_iter_linear_io io;
+       } _dst_iter, _src_iter;
+       struct ttm_kmap_iter *dst_iter, *src_iter; int ret;
-       unsigned long i;

-       ret = ttm_bo_wait_ctx(bo, ctx);
-       if (ret)
-               return ret;

drivers/gpu/drm/ttm/ttm_bo_util.c +0 -16
-
-       ret = ttm_resource_ioremap(bdev, old_mem, &old_iomap);
-       if (ret)
-               return ret;
-       ret = ttm_resource_ioremap(bdev, new_mem, &new_iomap);
-       if (ret)
-               goto out;
-
-       /*
-        * Single TTM move. NOP.
-        */
-       if (old_iomap == NULL && new_iomap == NULL)
-               goto out2;
-
-       /*
-        * Don't move nonexistent data. Clear destination

instead.
-        */
-       if (old_iomap == NULL &&
-           (ttm == NULL || (!ttm_tt_is_populated(ttm) && -                            !(ttm->page_flags & TTM_PAGE_FLAG_SWAPPED)))) {
-               memset_io(new_iomap, 0, new_mem-

Quoted 1 line

num_pages*PAGE_SIZE);

-               goto out2;
-       }
-
-       /*
-        * TTM might be null for moves within the same region. -        */
-       if (ttm) {
+       if (ttm && ((ttm->page_flags & TTM_PAGE_FLAG_SWAPPED) ||
+                   dst_man->use_tt)) {
                 ret = ttm_tt_populate(bdev, ttm, ctx); if (ret)
-                       goto out1;
+                       return ret;
         }

-       for (i = 0; i < new_mem->num_pages; ++i) {
-               if (old_iomap == NULL) {
-                       pgprot_t prot = ttm_io_prot(bo, old_mem, PAGE_KERNEL);
-                       ret = ttm_copy_ttm_io_page(ttm, new_iomap, i,
-                                                  prot); -               } else if (new_iomap == NULL) {
-                       pgprot_t prot = ttm_io_prot(bo, new_mem, PAGE_KERNEL);
-                       ret = ttm_copy_io_ttm_page(ttm, old_iomap, i,
-                                                  prot); -               } else {
-                       ret = ttm_copy_io_page(new_iomap, old_iomap, i);
-               }
-               if (ret)
-                       goto out1;
+       dst_iter = ttm_kmap_iter_linear_io_init(&_dst_iter.io, bdev, dst_mem);
+       if (PTR_ERR(dst_iter) == -EINVAL && dst_man->use_tt) +               dst_iter = ttm_kmap_iter_tt_init(&_dst_iter.tt, bo->ttm);
+       if (IS_ERR(dst_iter))
+               return PTR_ERR(dst_iter);
+
+       src_iter = ttm_kmap_iter_linear_io_init(&_src_iter.io, bdev, src_mem);
+       if (PTR_ERR(src_iter) == -EINVAL && src_man->use_tt) +               src_iter = ttm_kmap_iter_tt_init(&_src_iter.tt, bo->ttm);
+       if (IS_ERR(src_iter)) {
+               ret = PTR_ERR(src_iter);
+               goto out_src_iter;
         }
-       mb();

drivers/gpu/drm/ttm/ttm_bo_util.c +5 -10
-out2:
-       old_copy = *old_mem;
 
-       ttm_bo_assign_mem(bo, new_mem);
-
-       if (!man->use_tt)
-               ttm_bo_tt_destroy(bo);
+       ttm_move_memcpy(bo, dst_mem, dst_iter, src_iter);
+       src_copy = *src_mem;
+       ttm_bo_move_sync_cleanup(bo, dst_mem);
 
-out1:
-       ttm_resource_iounmap(bdev, old_mem, new_iomap);
-out:
-       ttm_resource_iounmap(bdev, &old_copy, old_iomap);
+       if (!src_iter->ops->maps_tt)
+               ttm_kmap_iter_linear_io_fini(&_src_iter.io,

bdev, &src_copy);
+out_src_iter:
+       if (!dst_iter->ops->maps_tt)
+               ttm_kmap_iter_linear_io_fini(&_dst_iter.io, bdev, dst_mem);

-       /*
-        * On error, keep the mm node!
-        */
-       if (!ret)
-               ttm_resource_free(bo, &old_copy);
         return ret;
  }
  EXPORT_SYMBOL(ttm_bo_move_memcpy);

drivers/gpu/drm/ttm/ttm_bo_util.c +0 -0
@@ -336,27 +272,7 @@ pgprot_t ttm_io_prot(struct
 ttm_buffer_object *bo, struct ttm_resource *res,
          man = ttm_manager_type(bo->bdev, res->mem_type);

         caching = man->use_tt ? bo->ttm->caching : res-

Quoted 1 line

bus.caching;

-       /* Cached mappings need no adjustment */
-       if (caching == ttm_cached)
-               return tmp;

drivers/gpu/drm/ttm/ttm_bo_util.c +0 -8
-
-#if defined(__i386__) || defined(__x86_64__)
-       if (caching == ttm_write_combined)
-               tmp = pgprot_writecombine(tmp);
-       else if (boot_cpu_data.x86 > 3)
-               tmp = pgprot_noncached(tmp);
-#endif
-#if defined(__ia64__) || defined(__arm__) ||

defined(__aarch64__) || \
-    defined(__powerpc__) || defined(__mips__)
-       if (caching == ttm_write_combined)
-               tmp = pgprot_writecombine(tmp);
-       else
-               tmp = pgprot_noncached(tmp);

drivers/gpu/drm/ttm/ttm_bo_util.c +1 -5
-#endif
-#if defined(__sparc__)
-       tmp = pgprot_noncached(tmp);
-#endif
-       return tmp;
+       return ttm_prot_from_caching(caching, tmp);

  }
  EXPORT_SYMBOL(ttm_io_prot);

drivers/gpu/drm/ttm/ttm_module.c +4 -0
index 56b0efdba1a9..997c458f68a9 100644
@@ -31,12 +31,47 @@
    */
   #include <linux/module.h>
   #include <linux/device.h>
+#include <linux/pgtable.h>
   #include <linux/sched.h>
   #include <linux/debugfs.h>
   #include <drm/drm_sysfs.h>
+#include <drm/ttm/ttm_caching.h>
 
   #include "ttm_module.h"
 
+/**
+ * ttm_prot_from_caching - Modify the page protection

according to the
+ * ttm cacing mode
+ * @caching: The ttm caching mode
+ * @tmp: The original page protection
+ *
+ * Return: The modified page protection
+ */
+pgprot_t ttm_prot_from_caching(enum ttm_caching caching, pgprot_t tmp)

drivers/gpu/drm/ttm/ttm_module.c +12 -0
+{
+       /* Cached mappings need no adjustment */
+       if (caching == ttm_cached)
+               return tmp;
+
+#if defined(__i386__) || defined(__x86_64__)
+       if (caching == ttm_write_combined)
+               tmp = pgprot_writecombine(tmp);
+       else if (boot_cpu_data.x86 > 3)
+               tmp = pgprot_noncached(tmp);
+#endif
+#if defined(__ia64__) || defined(__arm__) ||

defined(__aarch64__) || \
+       defined(__powerpc__) || defined(__mips__)
+       if (caching == ttm_write_combined)
+               tmp = pgprot_writecombine(tmp);
+       else
+               tmp = pgprot_noncached(tmp);

drivers/gpu/drm/ttm/ttm_module.c +7 -0
+#endif
+#if defined(__sparc__)
+       tmp = pgprot_noncached(tmp);
+#endif
+       return tmp;
+}
+

  struct dentry *ttm_debugfs_root;

  static int __init ttm_init(void)

drivers/gpu/drm/ttm/ttm_resource.c +4 -0
index 59e2b7157e41..e05ae7e3d477 100644
@@ -22,6 +22,10 @@
    * Authors: Christian König
    */
 
+#include <linux/dma-buf-map.h>
+#include <linux/io-mapping.h>
+#include <linux/scatterlist.h>
+
   #include <drm/ttm/ttm_resource.h>
   #include <drm/ttm/ttm_bo_driver.h>
 
@@ -147,3 +151,165 @@ void ttm_resource_manager_debug(struct
 ttm_resource_manager *man,
                  man->func->debug(man, p);
   }

  EXPORT_SYMBOL(ttm_resource_manager_debug);

drivers/gpu/drm/ttm/ttm_resource.c +2 -0
+
+static void ttm_kmap_iter_iomap_map_local(struct ttm_kmap_iter

*iter,
+                                         struct dma_buf_map *dmap,
+                                         pgoff_t i)

drivers/gpu/drm/ttm/ttm_resource.c +9 -0
+{
+       struct ttm_kmap_iter_iomap *iter_io =
+               container_of(iter, typeof(*iter_io), base);
+       void __iomem *addr;
+
+retry:
+       while (i >= iter_io->cache.end) {
+               iter_io->cache.sg = iter_io->cache.sg ?
+                       sg_next(iter_io->cache.sg) : iter_io-
Quoted 1 line

st->sgl;

+               iter_io->cache.i = iter_io->cache.end; +               iter_io->cache.end += sg_dma_len(iter_io-

Quoted 1 line

cache.sg) >>

+                       PAGE_SHIFT;
+               iter_io->cache.offs = sg_dma_address(iter_io-

Quoted 1 line

cache.sg) -

+                       iter_io->start;
+       }

drivers/gpu/drm/ttm/ttm_resource.c +8 -0
+
+       if (i < iter_io->cache.i) {
+               iter_io->cache.end = 0;
+               iter_io->cache.sg = NULL;
+               goto retry;
+       }
+
+       addr = io_mapping_map_local_wc(iter_io->iomap, iter_io-
Quoted 1 line

cache.offs +

+                                      (((resource_size_t)i - iter_io->cache.i)
+                                       << PAGE_SHIFT)); +       dma_buf_map_set_vaddr_iomem(dmap, addr);

drivers/gpu/drm/ttm/ttm_resource.c +3 -0
+}
+
+static void ttm_kmap_iter_iomap_unmap_local(struct

ttm_kmap_iter *iter,
+                                           struct dma_buf_map *map)

drivers/gpu/drm/ttm/ttm_resource.c +12 -0
+{
+       io_mapping_unmap_local(map->vaddr_iomem);
+}
+
+static const struct ttm_kmap_iter_ops ttm_kmap_iter_io_ops = {
+       .map_local =  ttm_kmap_iter_iomap_map_local,
+       .unmap_local = ttm_kmap_iter_iomap_unmap_local,
+       .maps_tt = false,
+};
+
+/**
+ * ttm_kmap_iter_iomap_init - Initialize a struct

ttm_kmap_iter_iomap
+ * @iter_io: The struct ttm_kmap_iter_iomap to initialize.
+ * @iomap: The struct io_mapping representing the underlying linear io_memory.
+ * @st: sg_table into @iomap, representing the memory of the struct
+ * ttm_resource.
+ * @start: Offset that needs to be subtracted from @st to make
+ * sg_dma_address(st->sgl) - @start == 0 for @iomap start.
+ *
+ * Return: Pointer to the embedded struct ttm_kmap_iter.
+ */

drivers/gpu/drm/ttm/ttm_resource.c +20 -0
+struct ttm_kmap_iter *
+ttm_kmap_iter_iomap_init(struct ttm_kmap_iter_iomap *iter_io,
+                        struct io_mapping *iomap,
+                        struct sg_table *st,
+                        resource_size_t start)
+{
+       iter_io->base.ops = &ttm_kmap_iter_io_ops;
+       iter_io->iomap = iomap;
+       iter_io->st = st;
+       iter_io->start = start;
+       memset(&iter_io->cache, 0, sizeof(iter_io->cache));
+
+       return &iter_io->base;
+}
+EXPORT_SYMBOL(ttm_kmap_iter_iomap_init);
+
+/**
+ * DOC: Linear io iterator
+ *
+ * This code should die in the not too near future. Best would

be if we could
+ * make io-mapping use memremap for all io memory, and have memremap
+ * implement a kmap_local functionality. We could then strip a huge amount of
+ * code. These linear io iterators are implemented to mimic old functionality,
+ * and they don't use kmap_local semantics at all internally. Rather ioremap or
+ * friends, and at least on 32-bit they add global TLB flushes and points
+ * of failure.
+ */

drivers/gpu/drm/ttm/ttm_resource.c +2 -0
+
+static void ttm_kmap_iter_linear_io_map_local(struct

ttm_kmap_iter *iter,
+                                             struct dma_buf_map *dmap,
+                                             pgoff_t i)

drivers/gpu/drm/ttm/ttm_resource.c +9 -0
+{
+       struct ttm_kmap_iter_linear_io *iter_io =
+               container_of(iter, typeof(*iter_io), base);
+
+       *dmap = iter_io->dmap;
+       dma_buf_map_incr(dmap, i * PAGE_SIZE);
+}
+
+static const struct ttm_kmap_iter_ops

ttm_kmap_iter_linear_io_ops = {
+       .map_local =  ttm_kmap_iter_linear_io_map_local, +       .maps_tt = false,

drivers/gpu/drm/ttm/ttm_resource.c +4 -0
+};
+
+struct ttm_kmap_iter *
+ttm_kmap_iter_linear_io_init(struct ttm_kmap_iter_linear_io

*iter_io,
+                            struct ttm_device *bdev, +                            struct ttm_resource *mem)

drivers/gpu/drm/ttm/ttm_resource.c +13 -0
+{
+       int ret;
+
+       ret = ttm_mem_io_reserve(bdev, mem);
+       if (ret)
+               goto out_err;
+       if (!mem->bus.is_iomem) {
+               ret = -EINVAL;
+               goto out_io_free;
+       }
+
+       if (mem->bus.addr) {
+               dma_buf_map_set_vaddr(&iter_io->dmap, mem-
Quoted 1 line

bus.addr);

+               iter_io->needs_unmap = false;
+       } else {
+               size_t bus_size = (size_t)mem->num_pages << PAGE_SHIFT;
+
+               iter_io->needs_unmap = true;
+               if (mem->bus.caching == ttm_write_combined) +                       dma_buf_map_set_vaddr_iomem(&iter_io-

Quoted 1 line

dmap,

+
ioremap_wc(mem->bus.offset,
+
bus_size));
+               else if (mem->bus.caching == ttm_cached) +                       dma_buf_map_set_vaddr(&iter_io->dmap, +                                             memremap(mem-

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

bus.offset, bus_size,

+
MEMREMAP_WB));

The comments in set_vaddr suggest that this is meant for system-memory. Does that actually matter or is it just about not losing the __iomem annotation on platforms where it matters?

Yes, it's the latter. dma_buf_map() is relatively new and the author
probably didn't think about the case of cached iomem, which is used by,
for example, vmwgfx.

Quoted 7 lines
Quoted 3 lines

Apparently cached device local is a thing. Also should this not be
wrapped in CONFIG_X86?

Both dma_buf_map() and memremap are generic, I think, I guess memremap
would return NULL if it's not supported.

Quoted 8 lines

It looks like memremap just wraps ioremap_cache, but since it also discards the __iomem annotation should we be doing that universally? Also not sure if ioremap_cache is universally supported, so wrapping in CONFIG_X86 and falling back to plain ioremap() might be needed? Or at least that looks like roughly what the previous code was doing? Not
too sure tbh.

I think the long term goal is to use memremap all over the place, to just not have to bother with the __iomem annotation. But to do that io- mapping.h needs to support memremap. But for now we need to be strict about __iomem unless we're in arch specific code. That's why that dma_buf_map thing was created, but TTM memcpy was never fully adapted.

As for limited arch support for memremap cached, It looks like we only need to or in "backup" mapping modes in the memremap flags, and we'd mimic the previous behaviour.

/Thomas

Quoted 4 lines
Quoted 4 lines

/Thomas

#23 lore raw

Re: [Intel-gfx] [PATCH v3 09/12] drm/ttm: Document and optimize ttm_bo_pipeline_gutting()

In reply to Thomas Hellström
Quoted 80 lines · Thomas Hellström

If the bo is idle when calling ttm_bo_pipeline_gutting(), we unnecessarily create a ghost object and push it out to delayed destroy.
Fix this by adding a path for idle, and document the function.

Also avoid having the bo end up in a bad state vulnerable to user-space triggered kernel BUGs if the call to ttm_tt_create() fails.

Finally reuse ttm_bo_pipeline_gutting() in ttm_bo_evict().

---
 drivers/gpu/drm/ttm/ttm_bo.c      | 20 ++++++------
 drivers/gpu/drm/ttm/ttm_bo_util.c | 52 ++++++++++++++++++++++++++++---
 drivers/gpu/drm/ttm/ttm_tt.c      |  5 +++
 include/drm/ttm/ttm_tt.h          | 10 ++++++
 4 files changed, 73 insertions(+), 14 deletions(-)
drivers/gpu/drm/ttm/ttm_bo.c +10 -10
index ca1b098b6a56..a8fa3375b8aa 100644
@@ -501,10 +501,15 @@ static int ttm_bo_evict(struct ttm_buffer_object *bo,
        bdev->funcs->evict_flags(bo, &placement);
 
        if (!placement.num_placement && !placement.num_busy_placement) {
-               ttm_bo_wait(bo, false, false);
+               ret = ttm_bo_wait(bo, true, false);
+               if (ret)
+                       return ret;
 
-               ttm_bo_cleanup_memtype_use(bo);
-               return ttm_tt_create(bo, false);
+               /*
+                * Since we've already synced, this frees backing store
+                * immediately.
+                */
+               return ttm_bo_pipeline_gutting(bo);
        }
 
        ret = ttm_bo_mem_space(bo, &placement, &evict_mem, ctx);
@@ -974,13 +979,8 @@ int ttm_bo_validate(struct ttm_buffer_object *bo,
        /*
         * Remove the backing store if no placement is given.
         */
-       if (!placement->num_placement && !placement->num_busy_placement) {
-               ret = ttm_bo_pipeline_gutting(bo);
-               if (ret)
-                       return ret;
-
-               return ttm_tt_create(bo, false);
-       }
+       if (!placement->num_placement && !placement->num_busy_placement)
+               return ttm_bo_pipeline_gutting(bo);
 
        /*
         * Check whether we need to move buffer.
drivers/gpu/drm/ttm/ttm_bo_util.c +11 -0
index 4a7d3d672f9a..7fa9b3a852eb 100644
@@ -585,26 +585,70 @@ int ttm_bo_move_accel_cleanup(struct ttm_buffer_object *bo,
 }
 EXPORT_SYMBOL(ttm_bo_move_accel_cleanup);
 
+/**
+ * ttm_bo_pipeline_gutting - purge the contents of a bo
+ * @bo: The buffer object
+ *
+ * Purge the contents of a bo, async if the bo is not idle.
+ * After a successful call, the bo is left unpopulated in
+ * system placement. The function may wait uninterruptible
+ * for idle on OOM.
+ *
+ * Return: 0 if successful, negative error code on failure.
+ */
 int ttm_bo_pipeline_gutting(struct ttm_buffer_object *bo)
 {
        static const struct ttm_place sys_mem = { .mem_type = TTM_PL_SYSTEM };

drivers/gpu/drm/ttm/ttm_bo_util.c int ttm_bo_move_accel_cleanup(struct ttm_buffer_object *bo, · line 610

Quoted 10 lines
drivers/gpu/drm/ttm/ttm_bo_util.c +6 -1
        struct ttm_buffer_object *ghost;
+       struct ttm_tt *ttm;
        int ret;
 
-       ret = ttm_buffer_object_transfer(bo, &ghost);
+       /* If already idle, no need for ghost object dance. */
+       ret = ttm_bo_wait(bo, false, true);
+       if (ret != -EBUSY) {
+               if (!bo->ttm) {
+                       ret = ttm_tt_create(bo, true);

Why do we now unconditionally add clearing? Below also.

Quoted 87 lines
drivers/gpu/drm/ttm/ttm_bo_util.c +31 -3
+                       if (ret)
+                               return ret;
+               } else {
+                       ttm_tt_unpopulate(bo->bdev, bo->ttm);
+                       if (bo->type == ttm_bo_type_device)
+                               ttm_tt_mark_for_clear(bo->ttm);
+               }
+               ttm_resource_free(bo, &bo->mem);
+               ttm_resource_alloc(bo, &sys_mem, &bo->mem);
+
+               return 0;
+       }
+
+       /*
+        * We need an unpopulated ttm_tt after giving our current one,
+        * if any, to the ghost object. And we can't afford to fail
+        * creating one *after* the operation.
+        */
+
+       ttm = bo->ttm;
+       bo->ttm = NULL;
+       ret = ttm_tt_create(bo, true);
+       swap(bo->ttm, ttm);
        if (ret)
                return ret;
 
+       ret = ttm_buffer_object_transfer(bo, &ghost);
+       if (ret) {
+               ttm_tt_destroy(bo->bdev, ttm);
+               return ret;
+       }
+
        ret = dma_resv_copy_fences(&ghost->base._resv, bo->base.resv);
        /* Last resort, wait for the BO to be idle when we are OOM */
        if (ret)
                ttm_bo_wait(bo, false, false);
 
-       ttm_resource_alloc(bo, &sys_mem, &bo->mem);
-       bo->ttm = NULL;
-
        dma_resv_unlock(&ghost->base._resv);
        ttm_bo_put(ghost);
+       bo->ttm = ttm;
+       ttm_resource_alloc(bo, &sys_mem, &bo->mem);
 
        return 0;
 }
drivers/gpu/drm/ttm/ttm_tt.c +5 -0
index 0e41227116b1..913b330a234b 100644
@@ -134,6 +134,11 @@ void ttm_tt_destroy_common(struct ttm_device *bdev, struct ttm_tt *ttm)
 }
 EXPORT_SYMBOL(ttm_tt_destroy_common);
 
+void ttm_tt_mark_for_clear(struct ttm_tt *ttm)
+{
+       ttm->page_flags |= TTM_PAGE_FLAG_ZERO_ALLOC;
+}
+
 void ttm_tt_destroy(struct ttm_device *bdev, struct ttm_tt *ttm)
 {
        bdev->funcs->ttm_tt_destroy(bdev, ttm);
include/drm/ttm/ttm_tt.h +10 -1
index 3102059db726..daa9c4cf48bb 100644
@@ -170,6 +170,16 @@ int ttm_tt_populate(struct ttm_device *bdev, struct ttm_tt *ttm, struct ttm_oper
  */
 void ttm_tt_unpopulate(struct ttm_device *bdev, struct ttm_tt *ttm);
 
+/**
+ * ttm_tt_mark_for_clear - Mark pages for clearing on populate.
+ *
+ * @ttm: Pointer to the ttm_tt structure
+ *
+ * Marks pages for clearing so that the next time the page vector is
+ * populated, the pages will be cleared.
+ */
+void ttm_tt_mark_for_clear(struct ttm_tt *ttm);
+
 void ttm_tt_mgr_init(unsigned long num_pages, unsigned long num_dma32_pages);
 
 struct ttm_kmap_iter *ttm_kmap_iter_tt_init(struct ttm_kmap_iter_tt *iter_tt,
--

2.31.1

Quoted 4 lines

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

author reviewer
#24 lore raw

Re: [Intel-gfx] [PATCH v3 09/12] drm/ttm: Document and optimize ttm_bo_pipeline_gutting()

Quoted 99 lines · Matthew Auld
Quoted 97 lines · Thomas Hellström

If the bo is idle when calling ttm_bo_pipeline_gutting(), we unnecessarily
create a ghost object and push it out to delayed destroy. Fix this by adding a path for idle, and document the function.

Also avoid having the bo end up in a bad state vulnerable to user- space
triggered kernel BUGs if the call to ttm_tt_create() fails.

Finally reuse ttm_bo_pipeline_gutting() in ttm_bo_evict().

---
 drivers/gpu/drm/ttm/ttm_bo.c      | 20 ++++++------ drivers/gpu/drm/ttm/ttm_bo_util.c | 52
++++++++++++++++++++++++++++---
 drivers/gpu/drm/ttm/ttm_tt.c      |  5 +++ include/drm/ttm/ttm_tt.h          | 10 ++++++

 4 files changed, 73 insertions(+), 14 deletions(-)
drivers/gpu/drm/ttm/ttm_bo.c +0 -0
index ca1b098b6a56..a8fa3375b8aa 100644
@@ -501,10 +501,15 @@ static int ttm_bo_evict(struct
 ttm_buffer_object *bo,
         bdev->funcs->evict_flags(bo, &placement);
 

        if (!placement.num_placement && !placement.num_busy_placement) {
-               ttm_bo_wait(bo, false, false); +               ret = ttm_bo_wait(bo, true, false); +               if (ret)
+                       return ret;

-               ttm_bo_cleanup_memtype_use(bo);
-               return ttm_tt_create(bo, false);
+               /*
+                * Since we've already synced, this frees backing store
+                * immediately.
+                */
+               return ttm_bo_pipeline_gutting(bo);
        }

        ret = ttm_bo_mem_space(bo, &placement, &evict_mem, ctx);

drivers/gpu/drm/ttm/ttm_bo.c +2 -7
@@ -974,13 +979,8 @@ int ttm_bo_validate(struct ttm_buffer_object
 *bo,
         /*
          * Remove the backing store if no placement is given.
          */
-       if (!placement->num_placement && !placement-
 >num_busy_placement) {
-               ret = ttm_bo_pipeline_gutting(bo);
-               if (ret)
-                       return ret;
-
-               return ttm_tt_create(bo, false);
-       }
+       if (!placement->num_placement && !placement-
 >num_busy_placement)
+               return ttm_bo_pipeline_gutting(bo);

        /*
         * Check whether we need to move buffer.

drivers/gpu/drm/ttm/ttm_bo_util.c +11 -0
index 4a7d3d672f9a..7fa9b3a852eb 100644
@@ -585,26 +585,70 @@ int ttm_bo_move_accel_cleanup(struct
 ttm_buffer_object *bo,
  }
  EXPORT_SYMBOL(ttm_bo_move_accel_cleanup);
 
+/**
+ * ttm_bo_pipeline_gutting - purge the contents of a bo
+ * @bo: The buffer object
+ *
+ * Purge the contents of a bo, async if the bo is not idle.
+ * After a successful call, the bo is left unpopulated in
+ * system placement. The function may wait uninterruptible
+ * for idle on OOM.
+ *
+ * Return: 0 if successful, negative error code on failure.
+ */
  int ttm_bo_pipeline_gutting(struct ttm_buffer_object *bo)
  {

        static const struct ttm_place sys_mem = { .mem_type = TTM_PL_SYSTEM };
        struct ttm_buffer_object *ghost;
+       struct ttm_tt *ttm;
        int ret;

Quoted 8 lines
Quoted 6 lines

-       ret = ttm_buffer_object_transfer(bo, &ghost); +       /* If already idle, no need for ghost object dance. */ +       ret = ttm_bo_wait(bo, false, true);
+       if (ret != -EBUSY) {
+               if (!bo->ttm) {
+                       ret = ttm_tt_create(bo, true);

Why do we now unconditionally add clearing? Below also.

Here we've dropped the bo content and we add but do not populate a page vector. Now if someone resurrects this object we obtain new pages and those must be cleared, at least that's the intention.

/Thomas

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

maintainer
#11 lore raw

Re: [Intel-gfx] [PATCH v3 06/12] drm/ttm: Add a generic TTM memcpy move for page-based iomem

In reply to Thomas Hellström

Am 25.05.21 um 12:07 schrieb Thomas Hellström:

Quoted 695 lines · Matthew Auld
Quoted 686 lines
Quoted 678 lines · Thomas Hellström
Quoted 244 lines · Matthew Auld
Quoted 242 lines · Thomas Hellström

The internal ttm_bo_util memcpy uses ioremap functionality, and while it
probably might be possible to use it for copying in- and out of sglist represented io memory, using io_mem_reserve() / io_mem_free()
callbacks, that would cause problems with fault(). Instead, implement a method mapping page-by-page using kmap_local()
semantics. As an additional benefit we then avoid the occasional global
TLB flushes of ioremap() and consuming ioremap space, elimination of a
critical point of failure and with a slight change of semantics we could
also push the memcpy out async for testing and async driver development
purposes.

A special linear iomem iterator is introduced internally to mimic the
old ioremap behaviour for code-paths that can't immediately be ported
over. This adds to the code size and should be considered a temporary
solution.

Looking at the code we have a lot of checks for iomap tagged pointers.
Ideally we should extend the core memremap functions to also accept
uncached memory and kmap_local functionality. Then we could strip a
lot of code.

Cc: Christian König <[email protected]>
Signed-off-by: Thomas Hellström < [email protected]>
---
v3:
- Split up in various TTM files and addressed review comments by
   Christian König. Tested and fixed legacy iomap memcpy path
on i915.

---
   drivers/gpu/drm/ttm/ttm_bo_util.c  | 278 ++++++++++----------
---------
   drivers/gpu/drm/ttm/ttm_module.c   |  35 ++++
   drivers/gpu/drm/ttm/ttm_resource.c | 166 +++++++++++++++++
   drivers/gpu/drm/ttm/ttm_tt.c       |  42 +++++
   include/drm/ttm/ttm_bo_driver.h    |  28 +++
   include/drm/ttm/ttm_caching.h      |   2 +
   include/drm/ttm/ttm_kmap_iter.h    |  61 +++++++
   include/drm/ttm/ttm_resource.h     |  61 +++++++
   include/drm/ttm/ttm_tt.h           |  16 ++
   9 files changed, 508 insertions(+), 181 deletions(-)

  create mode 100644 include/drm/ttm/ttm_kmap_iter.h

drivers/gpu/drm/ttm/ttm_bo_util.c +2 -3
index ae8b61460724..912cbe8e60a2 100644
@@ -72,190 +72,126 @@ void ttm_mem_io_free(struct ttm_device
 *bdev,
          mem->bus.addr = NULL;
   }
 
-static int ttm_resource_ioremap(struct ttm_device *bdev,
-                              struct ttm_resource *mem,
-                              void **virtual)
+/**
+ * ttm_move_memcpy - Helper to perform a memcpy ttm move

operation.
+ * @bo: The struct ttm_buffer_object.
+ * @new_mem: The struct ttm_resource we're moving to (copy destination).
+ * @new_iter: A struct ttm_kmap_iter representing the destination resource.
+ * @src_iter: A struct ttm_kmap_iter representing the source resource.
+ *
+ * This function is intended to be able to move out async under a
+ * dma-fence if desired.
+ */

drivers/gpu/drm/ttm/ttm_bo_util.c +5 -7
+void ttm_move_memcpy(struct ttm_buffer_object *bo,
+                    struct ttm_resource *dst_mem,
+                    struct ttm_kmap_iter *dst_iter,
+                    struct ttm_kmap_iter *src_iter)
   {
-       int ret;
-       void *addr;
-
-       *virtual = NULL;
-       ret = ttm_mem_io_reserve(bdev, mem);
-       if (ret || !mem->bus.is_iomem)
-               return ret;
+       const struct ttm_kmap_iter_ops *dst_ops = dst_iter-
Quoted 1 line

ops;

+       const struct ttm_kmap_iter_ops *src_ops = src_iter-

Quoted 1 line

ops;

+       struct ttm_tt *ttm = bo->ttm;
+       struct dma_buf_map src_map, dst_map;
+       pgoff_t i;

-       if (mem->bus.addr) {
-               addr = mem->bus.addr;
-       } else {
-               size_t bus_size = (size_t)mem->num_pages << PAGE_SHIFT;
+       /* Single TTM move. NOP */
+       if (dst_ops->maps_tt && src_ops->maps_tt) +               return;

-               if (mem->bus.caching == ttm_write_combined) -                       addr = ioremap_wc(mem->bus.offset, bus_size);
-#ifdef CONFIG_X86
-               else if (mem->bus.caching == ttm_cached) -                       addr = ioremap_cache(mem->bus.offset, bus_size);
-#endif
-               else
-                       addr = ioremap(mem->bus.offset, bus_size);
-               if (!addr) {
-                       ttm_mem_io_free(bdev, mem); -                       return -ENOMEM;
+       /* Don't move nonexistent data. Clear destination instead. */
+       if (src_ops->maps_tt && (!ttm || !ttm_tt_is_populated(ttm))) {
+               if (ttm && !(ttm->page_flags & TTM_PAGE_FLAG_ZERO_ALLOC))
+                       return;
+
+               for (i = 0; i < dst_mem->num_pages; ++i) { +                       dst_ops->map_local(dst_iter, &dst_map, i);
+                       if (dst_map.is_iomem)
+                               memset_io(dst_map.vaddr_iomem, 0, PAGE_SIZE);
+                       else
+                               memset(dst_map.vaddr, 0, PAGE_SIZE);
+                       if (dst_ops->unmap_local)
+                               dst_ops->unmap_local(dst_iter, &dst_map);
                 }
+               return;
         }
-       *virtual = addr;
-       return 0;

drivers/gpu/drm/ttm/ttm_bo_util.c +0 -12
-}
-
-static void ttm_resource_iounmap(struct ttm_device *bdev,
-                               struct ttm_resource *mem,
-                               void *virtual)
-{
-       if (virtual && mem->bus.addr == NULL)
-               iounmap(virtual);
-       ttm_mem_io_free(bdev, mem);
-}
-
-static int ttm_copy_io_page(void *dst, void *src, unsigned

long page)
-{
-       uint32_t *dstP =
-           (uint32_t *) ((unsigned long)dst + (page << PAGE_SHIFT));
-       uint32_t *srcP =
-           (uint32_t *) ((unsigned long)src + (page << PAGE_SHIFT));

drivers/gpu/drm/ttm/ttm_bo_util.c +0 -18
-
-       int i;
-       for (i = 0; i < PAGE_SIZE / sizeof(uint32_t); ++i)
-               iowrite32(ioread32(srcP++), dstP++);
-       return 0;
-}
-
-static int ttm_copy_io_ttm_page(struct ttm_tt *ttm, void *src,
-                               unsigned long page,
-                               pgprot_t prot)
-{
-       struct page *d = ttm->pages[page];
-       void *dst;
-
-       if (!d)
-               return -ENOMEM;
-
-       src = (void *)((unsigned long)src + (page <<

PAGE_SHIFT));
-       dst = kmap_atomic_prot(d, prot);
-       if (!dst)
-               return -ENOMEM;

drivers/gpu/drm/ttm/ttm_bo_util.c +0 -19
-
-       memcpy_fromio(dst, src, PAGE_SIZE);
-
-       kunmap_atomic(dst);
-
-       return 0;
-}
-
-static int ttm_copy_ttm_io_page(struct ttm_tt *ttm, void *dst,
-                               unsigned long page,
-                               pgprot_t prot)
-{
-       struct page *s = ttm->pages[page];
-       void *src;
-
-       if (!s)
-               return -ENOMEM;
-
-       dst = (void *)((unsigned long)dst + (page <<

PAGE_SHIFT));
-       src = kmap_atomic_prot(s, prot);
-       if (!src)
-               return -ENOMEM;

-       memcpy_toio(dst, src, PAGE_SIZE);

drivers/gpu/drm/ttm/ttm_bo_util.c +6 -2
-
-       kunmap_atomic(src);
+       for (i = 0; i < dst_mem->num_pages; ++i) {
+               dst_ops->map_local(dst_iter, &dst_map, i);
+               src_ops->map_local(src_iter, &src_map, i);
+
+               if (!src_map.is_iomem && !dst_map.is_iomem) {
+                       memcpy(dst_map.vaddr, src_map.vaddr,

PAGE_SIZE);
+               } else if (!src_map.is_iomem) {
+                       dma_buf_map_memcpy_to(&dst_map, src_map.vaddr,
+                                             PAGE_SIZE); +               } else if (!dst_map.is_iomem) {
+                       memcpy_fromio(dst_map.vaddr, src_map.vaddr_iomem,
+                                     PAGE_SIZE);
+               } else {
+                       int j;
+                       u32 __iomem *src = src_map.vaddr_iomem; +                       u32 __iomem *dst = dst_map.vaddr_iomem;

Quoted 3 lines
Quoted 2 lines

-       return 0;
+                       for (j = 0; j < (PAGE_SIZE >> 2); ++j)

IMO PAGE_SIZE / sizeof(u32) is easier to understand.

OK, will fix.

Quoted 414 lines
Quoted 414 lines

+                               iowrite32(ioread32(src++), dst++);
+               }
+               if (src_ops->unmap_local)
+                       src_ops->unmap_local(src_iter, &src_map);
+               if (dst_ops->unmap_local)
+                       dst_ops->unmap_local(dst_iter, &dst_map);
+       }
  }
+EXPORT_SYMBOL(ttm_move_memcpy);

  int ttm_bo_move_memcpy(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx,
-                      struct ttm_resource *new_mem) +                      struct ttm_resource *dst_mem)
  {
         struct ttm_device *bdev = bo->bdev;
-       struct ttm_resource_manager *man = ttm_manager_type(bdev, new_mem->mem_type);
+       struct ttm_resource_manager *dst_man =
+               ttm_manager_type(bo->bdev, dst_mem->mem_type);
         struct ttm_tt *ttm = bo->ttm;
-       struct ttm_resource *old_mem = &bo->mem;
-       struct ttm_resource old_copy = *old_mem;
-       void *old_iomap;
-       void *new_iomap;
+       struct ttm_resource *src_mem = &bo->mem;
+       struct ttm_resource_manager *src_man =
+               ttm_manager_type(bdev, src_mem->mem_type); +       struct ttm_resource src_copy = *src_mem;
+       union {
+               struct ttm_kmap_iter_tt tt;
+               struct ttm_kmap_iter_linear_io io;
+       } _dst_iter, _src_iter;
+       struct ttm_kmap_iter *dst_iter, *src_iter;
         int ret;
-       unsigned long i;

-       ret = ttm_bo_wait_ctx(bo, ctx);
-       if (ret)
-               return ret;

drivers/gpu/drm/ttm/ttm_bo_util.c +0 -16
-
-       ret = ttm_resource_ioremap(bdev, old_mem, &old_iomap);
-       if (ret)
-               return ret;
-       ret = ttm_resource_ioremap(bdev, new_mem, &new_iomap);
-       if (ret)
-               goto out;
-
-       /*
-        * Single TTM move. NOP.
-        */
-       if (old_iomap == NULL && new_iomap == NULL)
-               goto out2;
-
-       /*
-        * Don't move nonexistent data. Clear destination

instead.
-        */
-       if (old_iomap == NULL &&
-           (ttm == NULL || (!ttm_tt_is_populated(ttm) && -                            !(ttm->page_flags & TTM_PAGE_FLAG_SWAPPED)))) {
-               memset_io(new_iomap, 0, new_mem-

Quoted 1 line

num_pages*PAGE_SIZE);

-               goto out2;
-       }
-
-       /*
-        * TTM might be null for moves within the same region. -        */
-       if (ttm) {
+       if (ttm && ((ttm->page_flags & TTM_PAGE_FLAG_SWAPPED) ||
+                   dst_man->use_tt)) {
                 ret = ttm_tt_populate(bdev, ttm, ctx); if (ret)
-                       goto out1;
+                       return ret;
         }

-       for (i = 0; i < new_mem->num_pages; ++i) {
-               if (old_iomap == NULL) {
-                       pgprot_t prot = ttm_io_prot(bo, old_mem, PAGE_KERNEL);
-                       ret = ttm_copy_ttm_io_page(ttm, new_iomap, i,
-                                                  prot); -               } else if (new_iomap == NULL) {
-                       pgprot_t prot = ttm_io_prot(bo, new_mem, PAGE_KERNEL);
-                       ret = ttm_copy_io_ttm_page(ttm, old_iomap, i,
-                                                  prot); -               } else {
-                       ret = ttm_copy_io_page(new_iomap, old_iomap, i);
-               }
-               if (ret)
-                       goto out1;
+       dst_iter = ttm_kmap_iter_linear_io_init(&_dst_iter.io, bdev, dst_mem);
+       if (PTR_ERR(dst_iter) == -EINVAL && dst_man->use_tt) +               dst_iter = ttm_kmap_iter_tt_init(&_dst_iter.tt, bo->ttm);
+       if (IS_ERR(dst_iter))
+               return PTR_ERR(dst_iter);
+
+       src_iter = ttm_kmap_iter_linear_io_init(&_src_iter.io, bdev, src_mem);
+       if (PTR_ERR(src_iter) == -EINVAL && src_man->use_tt) +               src_iter = ttm_kmap_iter_tt_init(&_src_iter.tt, bo->ttm);
+       if (IS_ERR(src_iter)) {
+               ret = PTR_ERR(src_iter);
+               goto out_src_iter;
         }
-       mb();

drivers/gpu/drm/ttm/ttm_bo_util.c +5 -10
-out2:
-       old_copy = *old_mem;
 
-       ttm_bo_assign_mem(bo, new_mem);
-
-       if (!man->use_tt)
-               ttm_bo_tt_destroy(bo);
+       ttm_move_memcpy(bo, dst_mem, dst_iter, src_iter);
+       src_copy = *src_mem;
+       ttm_bo_move_sync_cleanup(bo, dst_mem);
 
-out1:
-       ttm_resource_iounmap(bdev, old_mem, new_iomap);
-out:
-       ttm_resource_iounmap(bdev, &old_copy, old_iomap);
+       if (!src_iter->ops->maps_tt)
+               ttm_kmap_iter_linear_io_fini(&_src_iter.io,

bdev, &src_copy);
+out_src_iter:
+       if (!dst_iter->ops->maps_tt)
+               ttm_kmap_iter_linear_io_fini(&_dst_iter.io, bdev, dst_mem);

-       /*
-        * On error, keep the mm node!
-        */
-       if (!ret)
-               ttm_resource_free(bo, &old_copy);
         return ret;
  }
  EXPORT_SYMBOL(ttm_bo_move_memcpy);

drivers/gpu/drm/ttm/ttm_bo_util.c +1 -21
@@ -336,27 +272,7 @@ pgprot_t ttm_io_prot(struct
 ttm_buffer_object *bo, struct ttm_resource *res,
          man = ttm_manager_type(bo->bdev, res->mem_type);
          caching = man->use_tt ? bo->ttm->caching : res-
 > bus.caching;
-       /* Cached mappings need no adjustment */
-       if (caching == ttm_cached)
-               return tmp;
-
-#if defined(__i386__) || defined(__x86_64__)
-       if (caching == ttm_write_combined)
-               tmp = pgprot_writecombine(tmp);
-       else if (boot_cpu_data.x86 > 3)
-               tmp = pgprot_noncached(tmp);
-#endif
-#if defined(__ia64__) || defined(__arm__) ||
 defined(__aarch64__) || \
-    defined(__powerpc__) || defined(__mips__)
-       if (caching == ttm_write_combined)
-               tmp = pgprot_writecombine(tmp);
-       else
-               tmp = pgprot_noncached(tmp);
-#endif
-#if defined(__sparc__)
-       tmp = pgprot_noncached(tmp);
-#endif
-       return tmp;
+       return ttm_prot_from_caching(caching, tmp);
   }

  EXPORT_SYMBOL(ttm_io_prot);

drivers/gpu/drm/ttm/ttm_module.c +4 -0
index 56b0efdba1a9..997c458f68a9 100644
@@ -31,12 +31,47 @@
    */
   #include <linux/module.h>
   #include <linux/device.h>
+#include <linux/pgtable.h>
   #include <linux/sched.h>
   #include <linux/debugfs.h>
   #include <drm/drm_sysfs.h>
+#include <drm/ttm/ttm_caching.h>
 
   #include "ttm_module.h"
 
+/**
+ * ttm_prot_from_caching - Modify the page protection

according to the
+ * ttm cacing mode
+ * @caching: The ttm caching mode
+ * @tmp: The original page protection
+ *
+ * Return: The modified page protection
+ */
+pgprot_t ttm_prot_from_caching(enum ttm_caching caching, pgprot_t tmp)

drivers/gpu/drm/ttm/ttm_module.c +12 -0
+{
+       /* Cached mappings need no adjustment */
+       if (caching == ttm_cached)
+               return tmp;
+
+#if defined(__i386__) || defined(__x86_64__)
+       if (caching == ttm_write_combined)
+               tmp = pgprot_writecombine(tmp);
+       else if (boot_cpu_data.x86 > 3)
+               tmp = pgprot_noncached(tmp);
+#endif
+#if defined(__ia64__) || defined(__arm__) ||

defined(__aarch64__) || \
+       defined(__powerpc__) || defined(__mips__)
+       if (caching == ttm_write_combined)
+               tmp = pgprot_writecombine(tmp);
+       else
+               tmp = pgprot_noncached(tmp);

drivers/gpu/drm/ttm/ttm_module.c +7 -0
+#endif
+#if defined(__sparc__)
+       tmp = pgprot_noncached(tmp);
+#endif
+       return tmp;
+}
+
   struct dentry *ttm_debugfs_root;
 
   static int __init ttm_init(void)
drivers/gpu/drm/ttm/ttm_resource.c +6 -0
index 59e2b7157e41..e05ae7e3d477 100644
@@ -22,6 +22,10 @@
    * Authors: Christian König
    */
 
+#include <linux/dma-buf-map.h>
+#include <linux/io-mapping.h>
+#include <linux/scatterlist.h>
+
   #include <drm/ttm/ttm_resource.h>
   #include <drm/ttm/ttm_bo_driver.h>
 
@@ -147,3 +151,165 @@ void ttm_resource_manager_debug(struct
 ttm_resource_manager *man,
                  man->func->debug(man, p);
   }
   EXPORT_SYMBOL(ttm_resource_manager_debug);
+
+static void ttm_kmap_iter_iomap_map_local(struct ttm_kmap_iter

*iter,
+                                         struct dma_buf_map *dmap,
+                                         pgoff_t i)

drivers/gpu/drm/ttm/ttm_resource.c +9 -0
+{
+       struct ttm_kmap_iter_iomap *iter_io =
+               container_of(iter, typeof(*iter_io), base);
+       void __iomem *addr;
+
+retry:
+       while (i >= iter_io->cache.end) {
+               iter_io->cache.sg = iter_io->cache.sg ?
+                       sg_next(iter_io->cache.sg) : iter_io-
Quoted 1 line

st->sgl;

+               iter_io->cache.i = iter_io->cache.end; +               iter_io->cache.end += sg_dma_len(iter_io-

Quoted 1 line

cache.sg) >>

+                       PAGE_SHIFT;
+               iter_io->cache.offs = sg_dma_address(iter_io-

Quoted 1 line

cache.sg) -

+                       iter_io->start;
+       }

drivers/gpu/drm/ttm/ttm_resource.c +8 -0
+
+       if (i < iter_io->cache.i) {
+               iter_io->cache.end = 0;
+               iter_io->cache.sg = NULL;
+               goto retry;
+       }
+
+       addr = io_mapping_map_local_wc(iter_io->iomap, iter_io-
Quoted 1 line

cache.offs +

+                                      (((resource_size_t)i - iter_io->cache.i)
+                                       << PAGE_SHIFT)); +       dma_buf_map_set_vaddr_iomem(dmap, addr);

drivers/gpu/drm/ttm/ttm_resource.c +3 -0
+}
+
+static void ttm_kmap_iter_iomap_unmap_local(struct

ttm_kmap_iter *iter,
+                                           struct dma_buf_map *map)

drivers/gpu/drm/ttm/ttm_resource.c +12 -0
+{
+       io_mapping_unmap_local(map->vaddr_iomem);
+}
+
+static const struct ttm_kmap_iter_ops ttm_kmap_iter_io_ops = {
+       .map_local =  ttm_kmap_iter_iomap_map_local,
+       .unmap_local = ttm_kmap_iter_iomap_unmap_local,
+       .maps_tt = false,
+};
+
+/**
+ * ttm_kmap_iter_iomap_init - Initialize a struct

ttm_kmap_iter_iomap
+ * @iter_io: The struct ttm_kmap_iter_iomap to initialize.
+ * @iomap: The struct io_mapping representing the underlying linear io_memory.
+ * @st: sg_table into @iomap, representing the memory of the struct
+ * ttm_resource.
+ * @start: Offset that needs to be subtracted from @st to make
+ * sg_dma_address(st->sgl) - @start == 0 for @iomap start.
+ *
+ * Return: Pointer to the embedded struct ttm_kmap_iter.
+ */

drivers/gpu/drm/ttm/ttm_resource.c +20 -0
+struct ttm_kmap_iter *
+ttm_kmap_iter_iomap_init(struct ttm_kmap_iter_iomap *iter_io,
+                        struct io_mapping *iomap,
+                        struct sg_table *st,
+                        resource_size_t start)
+{
+       iter_io->base.ops = &ttm_kmap_iter_io_ops;
+       iter_io->iomap = iomap;
+       iter_io->st = st;
+       iter_io->start = start;
+       memset(&iter_io->cache, 0, sizeof(iter_io->cache));
+
+       return &iter_io->base;
+}
+EXPORT_SYMBOL(ttm_kmap_iter_iomap_init);
+
+/**
+ * DOC: Linear io iterator
+ *
+ * This code should die in the not too near future. Best would

be if we could
+ * make io-mapping use memremap for all io memory, and have memremap
+ * implement a kmap_local functionality. We could then strip a huge amount of
+ * code. These linear io iterators are implemented to mimic old functionality,
+ * and they don't use kmap_local semantics at all internally. Rather ioremap or
+ * friends, and at least on 32-bit they add global TLB flushes and points
+ * of failure.
+ */

drivers/gpu/drm/ttm/ttm_resource.c +2 -0
+
+static void ttm_kmap_iter_linear_io_map_local(struct

ttm_kmap_iter *iter,
+                                             struct dma_buf_map *dmap,
+                                             pgoff_t i)

drivers/gpu/drm/ttm/ttm_resource.c +9 -0
+{
+       struct ttm_kmap_iter_linear_io *iter_io =
+               container_of(iter, typeof(*iter_io), base);
+
+       *dmap = iter_io->dmap;
+       dma_buf_map_incr(dmap, i * PAGE_SIZE);
+}
+
+static const struct ttm_kmap_iter_ops

ttm_kmap_iter_linear_io_ops = {
+       .map_local =  ttm_kmap_iter_linear_io_map_local, +       .maps_tt = false,

drivers/gpu/drm/ttm/ttm_resource.c +4 -0
+};
+
+struct ttm_kmap_iter *
+ttm_kmap_iter_linear_io_init(struct ttm_kmap_iter_linear_io

*iter_io,
+                            struct ttm_device *bdev, +                            struct ttm_resource *mem)

drivers/gpu/drm/ttm/ttm_resource.c +13 -0
+{
+       int ret;
+
+       ret = ttm_mem_io_reserve(bdev, mem);
+       if (ret)
+               goto out_err;
+       if (!mem->bus.is_iomem) {
+               ret = -EINVAL;
+               goto out_io_free;
+       }
+
+       if (mem->bus.addr) {
+               dma_buf_map_set_vaddr(&iter_io->dmap, mem-
Quoted 1 line

bus.addr);

+               iter_io->needs_unmap = false;
+       } else {
+               size_t bus_size = (size_t)mem->num_pages << PAGE_SHIFT;
+
+               iter_io->needs_unmap = true;
+               if (mem->bus.caching == ttm_write_combined) +                       dma_buf_map_set_vaddr_iomem(&iter_io-

Quoted 1 line

dmap,

+
ioremap_wc(mem->bus.offset,
+
bus_size));
+               else if (mem->bus.caching == ttm_cached) +                       dma_buf_map_set_vaddr(&iter_io->dmap, +                                             memremap(mem-

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

bus.offset, bus_size,

+
MEMREMAP_WB));

The comments in set_vaddr suggest that this is meant for system-memory. Does that actually matter or is it just about not losing the __iomem annotation on platforms where it matters?

Yes, it's the latter. dma_buf_map() is relatively new and the author
probably didn't think about the case of cached iomem, which is used by,
for example, vmwgfx.

Quoted 6 lines
Quoted 3 lines

Apparently cached device local is a thing. Also should this not be
wrapped in CONFIG_X86?

Both dma_buf_map() and memremap are generic, I think, I guess memremap
would return NULL if it's not supported.

Quoted 8 lines

It looks like memremap just wraps ioremap_cache, but since it also discards the __iomem annotation should we be doing that universally? Also not sure if ioremap_cache is universally supported, so wrapping in CONFIG_X86 and falling back to plain ioremap() might be needed? Or at least that looks like roughly what the previous code was doing? Not
too sure tbh.

Quoted 5 lines

I think the long term goal is to use memremap all over the place, to just not have to bother with the __iomem annotation. But to do that io- mapping.h needs to support memremap. But for now we need to be strict about __iomem unless we're in arch specific code. That's why that dma_buf_map thing was created, but TTM memcpy was never fully adapted.

I don't think that this will work. __iomem annotation is there because we have architectures where you need to use special CPU instructions for iomem access.

That won't go away just because we use memremap().

Christian.

Quoted 12 lines

As for limited arch support for memremap cached, It looks like we only need to or in "backup" mapping modes in the memremap flags, and we'd mimic the previous behaviour.

/Thomas

Quoted 3 lines
Quoted 3 lines

/Thomas

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

author reviewer
#12 lore raw

Re: [Intel-gfx] [PATCH v3 06/12] drm/ttm: Add a generic TTM memcpy move for page-based iomem

Quoted 704 lines · Christian König

Am 25.05.21 um 12:07 schrieb Thomas Hellström:

Quoted 695 lines · Matthew Auld
Quoted 686 lines
Quoted 678 lines · Thomas Hellström
Quoted 244 lines · Matthew Auld
Quoted 242 lines · Thomas Hellström

The internal ttm_bo_util memcpy uses ioremap functionality, and while it
probably might be possible to use it for copying in- and out of sglist represented io memory, using io_mem_reserve() / io_mem_free()
callbacks, that would cause problems with fault(). Instead, implement a method mapping page-by-page using kmap_local()
semantics. As an additional benefit we then avoid the occasional global
TLB flushes of ioremap() and consuming ioremap space, elimination of a
critical point of failure and with a slight change of semantics we could
also push the memcpy out async for testing and async driver development
purposes.

A special linear iomem iterator is introduced internally to mimic the
old ioremap behaviour for code-paths that can't immediately be ported
over. This adds to the code size and should be considered a temporary
solution.

Looking at the code we have a lot of checks for iomap tagged pointers.
Ideally we should extend the core memremap functions to also accept
uncached memory and kmap_local functionality. Then we could strip a
lot of code.

Cc: Christian König <[email protected]>
Signed-off-by: Thomas Hellström < [email protected]>
---
v3:
- Split up in various TTM files and addressed review comments by
    Christian König. Tested and fixed legacy iomap memcpy path on i915.
---
   drivers/gpu/drm/ttm/ttm_bo_util.c  | 278 ++++++++++----------
---------
   drivers/gpu/drm/ttm/ttm_module.c   |  35 ++++ drivers/gpu/drm/ttm/ttm_resource.c | 166 +++++++++++++++++ drivers/gpu/drm/ttm/ttm_tt.c       |  42 +++++ include/drm/ttm/ttm_bo_driver.h    |  28 +++ include/drm/ttm/ttm_caching.h      |   2 + include/drm/ttm/ttm_kmap_iter.h    |  61 +++++++ include/drm/ttm/ttm_resource.h     |  61 +++++++ include/drm/ttm/ttm_tt.h           |  16 ++

   9 files changed, 508 insertions(+), 181 deletions(-)

   create mode 100644 include/drm/ttm/ttm_kmap_iter.h

drivers/gpu/drm/ttm/ttm_bo_util.c +2 -3
index ae8b61460724..912cbe8e60a2 100644
@@ -72,190 +72,126 @@ void ttm_mem_io_free(struct ttm_device
 *bdev,
           mem->bus.addr = NULL;
    }
 
-static int ttm_resource_ioremap(struct ttm_device *bdev,
-                              struct ttm_resource *mem,
-                              void **virtual)
+/**
+ * ttm_move_memcpy - Helper to perform a memcpy ttm move

operation.
+ * @bo: The struct ttm_buffer_object.
+ * @new_mem: The struct ttm_resource we're moving to (copy destination).
+ * @new_iter: A struct ttm_kmap_iter representing the destination resource.
+ * @src_iter: A struct ttm_kmap_iter representing the source resource.
+ *
+ * This function is intended to be able to move out async under a
+ * dma-fence if desired.
+ */
+void ttm_move_memcpy(struct ttm_buffer_object *bo, +                    struct ttm_resource *dst_mem, +                    struct ttm_kmap_iter *dst_iter, +                    struct ttm_kmap_iter *src_iter) {
-       int ret;
-       void *addr;
-
-       *virtual = NULL;
-       ret = ttm_mem_io_reserve(bdev, mem);
-       if (ret || !mem->bus.is_iomem)
-               return ret;
+       const struct ttm_kmap_iter_ops *dst_ops = dst_iter-

Quoted 1 line

ops;

+       const struct ttm_kmap_iter_ops *src_ops = src_iter-

Quoted 1 line

ops;

+       struct ttm_tt *ttm = bo->ttm;
+       struct dma_buf_map src_map, dst_map;
+       pgoff_t i;

-       if (mem->bus.addr) {
-               addr = mem->bus.addr;
-       } else {
-               size_t bus_size = (size_t)mem->num_pages << PAGE_SHIFT;
+       /* Single TTM move. NOP */
+       if (dst_ops->maps_tt && src_ops->maps_tt) +               return;

-               if (mem->bus.caching == ttm_write_combined) -                       addr = ioremap_wc(mem->bus.offset, bus_size);
-#ifdef CONFIG_X86
-               else if (mem->bus.caching == ttm_cached) -                       addr = ioremap_cache(mem->bus.offset, bus_size);
-#endif
-               else
-                       addr = ioremap(mem->bus.offset, bus_size);
-               if (!addr) {
-                       ttm_mem_io_free(bdev, mem); -                       return -ENOMEM;
+       /* Don't move nonexistent data. Clear destination instead. */
+       if (src_ops->maps_tt && (!ttm || !ttm_tt_is_populated(ttm))) {
+               if (ttm && !(ttm->page_flags & TTM_PAGE_FLAG_ZERO_ALLOC))
+                       return;
+
+               for (i = 0; i < dst_mem->num_pages; ++i) { +                       dst_ops->map_local(dst_iter, &dst_map, i);
+                       if (dst_map.is_iomem)
+ memset_io(dst_map.vaddr_iomem,
0, PAGE_SIZE);
+                       else
+                               memset(dst_map.vaddr, 0, PAGE_SIZE);
+                       if (dst_ops->unmap_local)
+ dst_ops->unmap_local(dst_iter,
&dst_map);
                  }
+               return;
          }
-       *virtual = addr;
-       return 0;

drivers/gpu/drm/ttm/ttm_bo_util.c +0 -12
-}
-
-static void ttm_resource_iounmap(struct ttm_device *bdev,
-                               struct ttm_resource *mem,
-                               void *virtual)
-{
-       if (virtual && mem->bus.addr == NULL)
-               iounmap(virtual);
-       ttm_mem_io_free(bdev, mem);
-}
-
-static int ttm_copy_io_page(void *dst, void *src, unsigned

long page)
-{
-       uint32_t *dstP =
-           (uint32_t *) ((unsigned long)dst + (page << PAGE_SHIFT));
-       uint32_t *srcP =
-           (uint32_t *) ((unsigned long)src + (page << PAGE_SHIFT));

drivers/gpu/drm/ttm/ttm_bo_util.c +0 -18
-
-       int i;
-       for (i = 0; i < PAGE_SIZE / sizeof(uint32_t); ++i)
-               iowrite32(ioread32(srcP++), dstP++);
-       return 0;
-}
-
-static int ttm_copy_io_ttm_page(struct ttm_tt *ttm, void *src,
-                               unsigned long page,
-                               pgprot_t prot)
-{
-       struct page *d = ttm->pages[page];
-       void *dst;
-
-       if (!d)
-               return -ENOMEM;
-
-       src = (void *)((unsigned long)src + (page <<

PAGE_SHIFT));
-       dst = kmap_atomic_prot(d, prot);
-       if (!dst)
-               return -ENOMEM;

drivers/gpu/drm/ttm/ttm_bo_util.c +0 -19
-
-       memcpy_fromio(dst, src, PAGE_SIZE);
-
-       kunmap_atomic(dst);
-
-       return 0;
-}
-
-static int ttm_copy_ttm_io_page(struct ttm_tt *ttm, void *dst,
-                               unsigned long page,
-                               pgprot_t prot)
-{
-       struct page *s = ttm->pages[page];
-       void *src;
-
-       if (!s)
-               return -ENOMEM;
-
-       dst = (void *)((unsigned long)dst + (page <<

PAGE_SHIFT));
-       src = kmap_atomic_prot(s, prot);
-       if (!src)
-               return -ENOMEM;

-       memcpy_toio(dst, src, PAGE_SIZE);

drivers/gpu/drm/ttm/ttm_bo_util.c +6 -2
-
-       kunmap_atomic(src);
+       for (i = 0; i < dst_mem->num_pages; ++i) {
+               dst_ops->map_local(dst_iter, &dst_map, i);
+               src_ops->map_local(src_iter, &src_map, i);
+
+               if (!src_map.is_iomem && !dst_map.is_iomem) {
+                       memcpy(dst_map.vaddr, src_map.vaddr,

PAGE_SIZE);
+               } else if (!src_map.is_iomem) {
+ dma_buf_map_memcpy_to(&dst_map,
src_map.vaddr,
+ PAGE_SIZE);
+               } else if (!dst_map.is_iomem) {
+                       memcpy_fromio(dst_map.vaddr, src_map.vaddr_iomem,
+                                     PAGE_SIZE);
+               } else {
+                       int j;
+                       u32 __iomem *src = src_map.vaddr_iomem; +                       u32 __iomem *dst = dst_map.vaddr_iomem;

Quoted 3 lines
Quoted 2 lines

-       return 0;
+                       for (j = 0; j < (PAGE_SIZE >> 2); ++j)

IMO PAGE_SIZE / sizeof(u32) is easier to understand.

OK, will fix.

Quoted 414 lines
Quoted 414 lines

+ iowrite32(ioread32(src++),
dst++);
+               }
+               if (src_ops->unmap_local)
+ src_ops->unmap_local(src_iter,
&src_map);
+               if (dst_ops->unmap_local)
+ dst_ops->unmap_local(dst_iter,
&dst_map);
+       }
   }
+EXPORT_SYMBOL(ttm_move_memcpy);

   int ttm_bo_move_memcpy(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx, -                      struct ttm_resource *new_mem) +                      struct ttm_resource *dst_mem) {
          struct ttm_device *bdev = bo->bdev;
-       struct ttm_resource_manager *man = ttm_manager_type(bdev, new_mem->mem_type);
+       struct ttm_resource_manager *dst_man =
+               ttm_manager_type(bo->bdev, dst_mem->mem_type); struct ttm_tt *ttm = bo->ttm;
-       struct ttm_resource *old_mem = &bo->mem;
-       struct ttm_resource old_copy = *old_mem;
-       void *old_iomap;
-       void *new_iomap;
+       struct ttm_resource *src_mem = &bo->mem;
+       struct ttm_resource_manager *src_man =
+               ttm_manager_type(bdev, src_mem->mem_type); +       struct ttm_resource src_copy = *src_mem;
+       union {
+               struct ttm_kmap_iter_tt tt;
+               struct ttm_kmap_iter_linear_io io;
+       } _dst_iter, _src_iter;
+       struct ttm_kmap_iter *dst_iter, *src_iter; int ret;
-       unsigned long i;

-       ret = ttm_bo_wait_ctx(bo, ctx);
-       if (ret)
-               return ret;

drivers/gpu/drm/ttm/ttm_bo_util.c +0 -16
-
-       ret = ttm_resource_ioremap(bdev, old_mem, &old_iomap);
-       if (ret)
-               return ret;
-       ret = ttm_resource_ioremap(bdev, new_mem, &new_iomap);
-       if (ret)
-               goto out;
-
-       /*
-        * Single TTM move. NOP.
-        */
-       if (old_iomap == NULL && new_iomap == NULL)
-               goto out2;
-
-       /*
-        * Don't move nonexistent data. Clear destination

instead.
-        */
-       if (old_iomap == NULL &&
-           (ttm == NULL || (!ttm_tt_is_populated(ttm) && -                            !(ttm->page_flags & TTM_PAGE_FLAG_SWAPPED)))) {
-               memset_io(new_iomap, 0, new_mem-

Quoted 1 line

num_pages*PAGE_SIZE);

-               goto out2;
-       }
-
-       /*
-        * TTM might be null for moves within the same region. -        */
-       if (ttm) {
+       if (ttm && ((ttm->page_flags & TTM_PAGE_FLAG_SWAPPED) ||
+                   dst_man->use_tt)) {
                  ret = ttm_tt_populate(bdev, ttm, ctx); if (ret)
-                       goto out1;
+                       return ret;
          }

-       for (i = 0; i < new_mem->num_pages; ++i) {
-               if (old_iomap == NULL) {
-                       pgprot_t prot = ttm_io_prot(bo, old_mem, PAGE_KERNEL);
-                       ret = ttm_copy_ttm_io_page(ttm, new_iomap, i,
- prot);
-               } else if (new_iomap == NULL) {
-                       pgprot_t prot = ttm_io_prot(bo, new_mem, PAGE_KERNEL);
-                       ret = ttm_copy_io_ttm_page(ttm, old_iomap, i,
- prot);
-               } else {
-                       ret = ttm_copy_io_page(new_iomap, old_iomap, i);
-               }
-               if (ret)
-                       goto out1;
+       dst_iter = ttm_kmap_iter_linear_io_init(&_dst_iter.io, bdev, dst_mem);
+       if (PTR_ERR(dst_iter) == -EINVAL && dst_man->use_tt) +               dst_iter = ttm_kmap_iter_tt_init(&_dst_iter.tt, bo->ttm);
+       if (IS_ERR(dst_iter))
+               return PTR_ERR(dst_iter);
+
+       src_iter = ttm_kmap_iter_linear_io_init(&_src_iter.io, bdev, src_mem);
+       if (PTR_ERR(src_iter) == -EINVAL && src_man->use_tt) +               src_iter = ttm_kmap_iter_tt_init(&_src_iter.tt, bo->ttm);
+       if (IS_ERR(src_iter)) {
+               ret = PTR_ERR(src_iter);
+               goto out_src_iter;
          }
-       mb();

drivers/gpu/drm/ttm/ttm_bo_util.c +5 -10
-out2:
-       old_copy = *old_mem;
 
-       ttm_bo_assign_mem(bo, new_mem);
-
-       if (!man->use_tt)
-               ttm_bo_tt_destroy(bo);
+       ttm_move_memcpy(bo, dst_mem, dst_iter, src_iter);
+       src_copy = *src_mem;
+       ttm_bo_move_sync_cleanup(bo, dst_mem);
 
-out1:
-       ttm_resource_iounmap(bdev, old_mem, new_iomap);
-out:
-       ttm_resource_iounmap(bdev, &old_copy, old_iomap);
+       if (!src_iter->ops->maps_tt)
+ ttm_kmap_iter_linear_io_fini(&_src_iter.io,

bdev, &src_copy);
+out_src_iter:
+       if (!dst_iter->ops->maps_tt)
+ ttm_kmap_iter_linear_io_fini(&_dst_iter.io,
bdev, dst_mem);

-       /*
-        * On error, keep the mm node!
-        */
-       if (!ret)
-               ttm_resource_free(bo, &old_copy);
          return ret;
   }
   EXPORT_SYMBOL(ttm_bo_move_memcpy);

drivers/gpu/drm/ttm/ttm_bo_util.c +0 -0
@@ -336,27 +272,7 @@ pgprot_t ttm_io_prot(struct
 ttm_buffer_object *bo, struct ttm_resource *res,
           man = ttm_manager_type(bo->bdev, res->mem_type);

          caching = man->use_tt ? bo->ttm->caching : res-

Quoted 1 line

bus.caching;

-       /* Cached mappings need no adjustment */
-       if (caching == ttm_cached)
-               return tmp;

drivers/gpu/drm/ttm/ttm_bo_util.c +0 -8
-
-#if defined(__i386__) || defined(__x86_64__)
-       if (caching == ttm_write_combined)
-               tmp = pgprot_writecombine(tmp);
-       else if (boot_cpu_data.x86 > 3)
-               tmp = pgprot_noncached(tmp);
-#endif
-#if defined(__ia64__) || defined(__arm__) ||

defined(__aarch64__) || \
-    defined(__powerpc__) || defined(__mips__)
-       if (caching == ttm_write_combined)
-               tmp = pgprot_writecombine(tmp);
-       else
-               tmp = pgprot_noncached(tmp);

drivers/gpu/drm/ttm/ttm_bo_util.c +1 -5
-#endif
-#if defined(__sparc__)
-       tmp = pgprot_noncached(tmp);
-#endif
-       return tmp;
+       return ttm_prot_from_caching(caching, tmp);

   }
   EXPORT_SYMBOL(ttm_io_prot);

drivers/gpu/drm/ttm/ttm_module.c +4 -0
index 56b0efdba1a9..997c458f68a9 100644
@@ -31,12 +31,47 @@
     */
    #include <linux/module.h>
    #include <linux/device.h>
+#include <linux/pgtable.h>
    #include <linux/sched.h>
    #include <linux/debugfs.h>
    #include <drm/drm_sysfs.h>
+#include <drm/ttm/ttm_caching.h>
 
    #include "ttm_module.h"
 
+/**
+ * ttm_prot_from_caching - Modify the page protection

according to the
+ * ttm cacing mode
+ * @caching: The ttm caching mode
+ * @tmp: The original page protection
+ *
+ * Return: The modified page protection
+ */
+pgprot_t ttm_prot_from_caching(enum ttm_caching caching, pgprot_t tmp)

drivers/gpu/drm/ttm/ttm_module.c +12 -0
+{
+       /* Cached mappings need no adjustment */
+       if (caching == ttm_cached)
+               return tmp;
+
+#if defined(__i386__) || defined(__x86_64__)
+       if (caching == ttm_write_combined)
+               tmp = pgprot_writecombine(tmp);
+       else if (boot_cpu_data.x86 > 3)
+               tmp = pgprot_noncached(tmp);
+#endif
+#if defined(__ia64__) || defined(__arm__) ||

defined(__aarch64__) || \
+       defined(__powerpc__) || defined(__mips__)
+       if (caching == ttm_write_combined)
+               tmp = pgprot_writecombine(tmp);
+       else
+               tmp = pgprot_noncached(tmp);

drivers/gpu/drm/ttm/ttm_module.c +7 -0
+#endif
+#if defined(__sparc__)
+       tmp = pgprot_noncached(tmp);
+#endif
+       return tmp;
+}
+

   struct dentry *ttm_debugfs_root;

   static int __init ttm_init(void)

drivers/gpu/drm/ttm/ttm_resource.c +4 -0
index 59e2b7157e41..e05ae7e3d477 100644
@@ -22,6 +22,10 @@
     * Authors: Christian König
     */
 
+#include <linux/dma-buf-map.h>
+#include <linux/io-mapping.h>
+#include <linux/scatterlist.h>
+
    #include <drm/ttm/ttm_resource.h>
    #include <drm/ttm/ttm_bo_driver.h>
 
@@ -147,3 +151,165 @@ void ttm_resource_manager_debug(struct
 ttm_resource_manager *man,
                   man->func->debug(man, p);
    }

   EXPORT_SYMBOL(ttm_resource_manager_debug);

drivers/gpu/drm/ttm/ttm_resource.c +2 -0
+
+static void ttm_kmap_iter_iomap_map_local(struct ttm_kmap_iter

*iter,
+                                         struct dma_buf_map *dmap,
+                                         pgoff_t i)

drivers/gpu/drm/ttm/ttm_resource.c +9 -0
+{
+       struct ttm_kmap_iter_iomap *iter_io =
+               container_of(iter, typeof(*iter_io), base);
+       void __iomem *addr;
+
+retry:
+       while (i >= iter_io->cache.end) {
+               iter_io->cache.sg = iter_io->cache.sg ?
+                       sg_next(iter_io->cache.sg) : iter_io-
Quoted 1 line

st->sgl;

+               iter_io->cache.i = iter_io->cache.end; +               iter_io->cache.end += sg_dma_len(iter_io-

Quoted 1 line

cache.sg) >>

+                       PAGE_SHIFT;
+               iter_io->cache.offs = sg_dma_address(iter_io-

Quoted 1 line

cache.sg) -

+                       iter_io->start;
+       }

drivers/gpu/drm/ttm/ttm_resource.c +8 -0
+
+       if (i < iter_io->cache.i) {
+               iter_io->cache.end = 0;
+               iter_io->cache.sg = NULL;
+               goto retry;
+       }
+
+       addr = io_mapping_map_local_wc(iter_io->iomap, iter_io-
Quoted 1 line

cache.offs +

+ (((resource_size_t)i -
iter_io->cache.i)
+                                       << PAGE_SHIFT)); +       dma_buf_map_set_vaddr_iomem(dmap, addr);

drivers/gpu/drm/ttm/ttm_resource.c +3 -0
+}
+
+static void ttm_kmap_iter_iomap_unmap_local(struct

ttm_kmap_iter *iter,
+                                           struct dma_buf_map *map)

drivers/gpu/drm/ttm/ttm_resource.c +12 -0
+{
+       io_mapping_unmap_local(map->vaddr_iomem);
+}
+
+static const struct ttm_kmap_iter_ops ttm_kmap_iter_io_ops = {
+       .map_local =  ttm_kmap_iter_iomap_map_local,
+       .unmap_local = ttm_kmap_iter_iomap_unmap_local,
+       .maps_tt = false,
+};
+
+/**
+ * ttm_kmap_iter_iomap_init - Initialize a struct

ttm_kmap_iter_iomap
+ * @iter_io: The struct ttm_kmap_iter_iomap to initialize.
+ * @iomap: The struct io_mapping representing the underlying linear io_memory.
+ * @st: sg_table into @iomap, representing the memory of the struct
+ * ttm_resource.
+ * @start: Offset that needs to be subtracted from @st to make
+ * sg_dma_address(st->sgl) - @start == 0 for @iomap start.
+ *
+ * Return: Pointer to the embedded struct ttm_kmap_iter.
+ */

drivers/gpu/drm/ttm/ttm_resource.c +20 -0
+struct ttm_kmap_iter *
+ttm_kmap_iter_iomap_init(struct ttm_kmap_iter_iomap *iter_io,
+                        struct io_mapping *iomap,
+                        struct sg_table *st,
+                        resource_size_t start)
+{
+       iter_io->base.ops = &ttm_kmap_iter_io_ops;
+       iter_io->iomap = iomap;
+       iter_io->st = st;
+       iter_io->start = start;
+       memset(&iter_io->cache, 0, sizeof(iter_io->cache));
+
+       return &iter_io->base;
+}
+EXPORT_SYMBOL(ttm_kmap_iter_iomap_init);
+
+/**
+ * DOC: Linear io iterator
+ *
+ * This code should die in the not too near future. Best would

be if we could
+ * make io-mapping use memremap for all io memory, and have memremap
+ * implement a kmap_local functionality. We could then strip a huge amount of
+ * code. These linear io iterators are implemented to mimic old functionality,
+ * and they don't use kmap_local semantics at all internally. Rather ioremap or
+ * friends, and at least on 32-bit they add global TLB flushes and points
+ * of failure.
+ */

drivers/gpu/drm/ttm/ttm_resource.c +2 -0
+
+static void ttm_kmap_iter_linear_io_map_local(struct

ttm_kmap_iter *iter,
+                                             struct dma_buf_map *dmap,
+                                             pgoff_t i)

drivers/gpu/drm/ttm/ttm_resource.c +9 -0
+{
+       struct ttm_kmap_iter_linear_io *iter_io =
+               container_of(iter, typeof(*iter_io), base);
+
+       *dmap = iter_io->dmap;
+       dma_buf_map_incr(dmap, i * PAGE_SIZE);
+}
+
+static const struct ttm_kmap_iter_ops

ttm_kmap_iter_linear_io_ops = {
+       .map_local =  ttm_kmap_iter_linear_io_map_local, +       .maps_tt = false,

drivers/gpu/drm/ttm/ttm_resource.c +4 -0
+};
+
+struct ttm_kmap_iter *
+ttm_kmap_iter_linear_io_init(struct ttm_kmap_iter_linear_io

*iter_io,
+                            struct ttm_device *bdev, +                            struct ttm_resource *mem)

drivers/gpu/drm/ttm/ttm_resource.c +13 -0
+{
+       int ret;
+
+       ret = ttm_mem_io_reserve(bdev, mem);
+       if (ret)
+               goto out_err;
+       if (!mem->bus.is_iomem) {
+               ret = -EINVAL;
+               goto out_io_free;
+       }
+
+       if (mem->bus.addr) {
+ dma_buf_map_set_vaddr(&iter_io->dmap, mem-
Quoted 1 line

bus.addr);

+               iter_io->needs_unmap = false;
+       } else {
+               size_t bus_size = (size_t)mem->num_pages << PAGE_SHIFT;
+
+               iter_io->needs_unmap = true;
+               if (mem->bus.caching == ttm_write_combined)
+ dma_buf_map_set_vaddr_iomem(&iter_io-

Quoted 1 line

dmap,

+
ioremap_wc(mem->bus.offset,
+
bus_size));
+               else if (mem->bus.caching == ttm_cached)
+ dma_buf_map_set_vaddr(&iter_io->dmap,
+ memremap(mem-

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

bus.offset, bus_size,

+
MEMREMAP_WB));

The comments in set_vaddr suggest that this is meant for system-memory. Does that actually matter or is it just about not losing the __iomem annotation on platforms where it matters?

Yes, it's the latter. dma_buf_map() is relatively new and the author
probably didn't think about the case of cached iomem, which is used by,
for example, vmwgfx.

Quoted 6 lines
Quoted 3 lines

Apparently cached device local is a thing. Also should this not be
wrapped in CONFIG_X86?

Both dma_buf_map() and memremap are generic, I think, I guess memremap
would return NULL if it's not supported.

Quoted 8 lines

It looks like memremap just wraps ioremap_cache, but since it also discards the __iomem annotation should we be doing that universally? Also not sure if ioremap_cache is universally supported, so wrapping in CONFIG_X86 and falling back to plain ioremap() might be needed? Or at least that looks like roughly what the previous code was doing? Not
too sure tbh.

Quoted 5 lines

I think the long term goal is to use memremap all over the place, to just not have to bother with the __iomem annotation. But to do that io- mapping.h needs to support memremap. But for now we need to be strict about __iomem unless we're in arch specific code. That's why that dma_buf_map thing was created, but TTM memcpy was never fully adapted.

Quoted 3 lines

I don't think that this will work. __iomem annotation is there because we have architectures where you need to use special CPU instructions for iomem access.

That won't go away just because we use memremap().

That's true, but can we ever support those with TTM, given that we allow user-space mmaping that transparently may change to an iomap? Given that, and what's written here

https://lwn.net/Articles/653585/

To me it sounds like if an architecture can't support memremap, we can't support it with TTM either.

In any case for this particular patch, to avoid potential regressions, OK if I just add an ioremap() in case the memremap fails?

/Thomas

Quoted 16 lines

Christian.

Quoted 12 lines

As for limited arch support for memremap cached, It looks like we only need to or in "backup" mapping modes in the memremap flags, and we'd mimic the previous behaviour.

/Thomas

Quoted 3 lines
Quoted 3 lines

/Thomas

maintainer
#13 lore raw

Re: [Intel-gfx] [PATCH v3 06/12] drm/ttm: Add a generic TTM memcpy move for page-based iomem

Am 26.05.21 um 09:39 schrieb Thomas Hellström:

Quoted 9 lines

[SNIP]

Quoted 8 lines
Quoted 5 lines

I think the long term goal is to use memremap all over the place, to just not have to bother with the __iomem annotation. But to do that io- mapping.h needs to support memremap. But for now we need to be strict about __iomem unless we're in arch specific code. That's why that dma_buf_map thing was created, but TTM memcpy was never fully adapted.

I don't think that this will work. __iomem annotation is there because we have architectures where you need to use special CPU instructions for iomem access.

Quoted 8 lines
Quoted 1 line

That won't go away just because we use memremap().

That's true, but can we ever support those with TTM, given that we allow user-space mmaping that transparently may change to an iomap? Given that, and what's written here

https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flwn.net%2FArticles%2F653585%2F&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C1cdcfe9d20e740308c9e08d92019785b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637576116034492654%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=e2BFGQJcElwVxrvHcnALDWscHN7ernLekGvXHqWBcwY%3D&amp;reserved=0

To me it sounds like if an architecture can't support memremap, we can't support it with TTM either.

That was also my argument, but this is unfortunately not true.

We already have architectures where the __iomem approach is mandatory for kernel mappings, but work fine for userspace (don't ask me how that works, idk).

That's the reason why we had to fix the UVD firmware upload in the kernel:

commit ba0b2275a6781b2f3919d931d63329b5548f6d5f
Author: Christian König <[email protected]> Date:   Tue Aug 23 11:00:17 2016 +0200

    drm/amdgpu: use memcpy_to/fromio for UVD fw upload

Quoted 2 lines

In any case for this particular patch, to avoid potential regressions, OK if I just add an ioremap() in case the memremap fails?

Well because of the issues outlined above I would actually prefer if we can keep the __iomem annotation around.

Christian.

Quoted 20 lines

/Thomas

Quoted 16 lines

Christian.

Quoted 12 lines

As for limited arch support for memremap cached, It looks like we only need to or in "backup" mapping modes in the memremap flags, and we'd mimic the previous behaviour.

/Thomas

Quoted 3 lines
Quoted 3 lines

/Thomas

author reviewer
#14 lore raw

Re: [Intel-gfx] [PATCH v3 06/12] drm/ttm: Add a generic TTM memcpy move for page-based iomem

Quoted 31 lines · Christian König

Am 26.05.21 um 09:39 schrieb Thomas Hellström:

Quoted 23 lines

[SNIP]

Quoted 11 lines
Quoted 10 lines

I think the long term goal is to use memremap all over the place, to
just not have to bother with the __iomem annotation. But to do that io-
mapping.h needs to support memremap. But for now we need to be strict
about __iomem unless we're in arch specific code. That's why that
dma_buf_map thing was created, but TTM memcpy was never fully adapted.

Quoted 5 lines

I don't think that this will work. __iomem annotation is there because we have architectures where you need to use special CPU instructions for iomem access.

That won't go away just because we use memremap().

That's true, but can we ever support those with TTM, given that we allow user-space mmaping that transparently may change to an iomap? Given that, and what's written here

Quoted 6 lines

https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flwn.net%2FArticles%2F653585%2F&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C1cdcfe9d20e740308c9e08d92019785b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637576116034492654%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=e2BFGQJcElwVxrvHcnALDWscHN7ernLekGvXHqWBcwY%3D&amp;reserved=0

To me it sounds like if an architecture can't support memremap, we can't support it with TTM either.

Quoted 6 lines

That was also my argument, but this is unfortunately not true.

We already have architectures where the __iomem approach is mandatory for kernel mappings, but work fine for userspace (don't ask me how that
works, idk).

Ugh. :/

Quoted 11 lines

That's the reason why we had to fix the UVD firmware upload in the kernel:

commit ba0b2275a6781b2f3919d931d63329b5548f6d5f
Author: Christian König <[email protected]> Date:   Tue Aug 23 11:00:17 2016 +0200

     drm/amdgpu: use memcpy_to/fromio for UVD fw upload

Quoted 1 line
Quoted 7 lines
Quoted 3 lines

In any case for this particular patch, to avoid potential regressions,
OK if I just add an ioremap() in case the memremap fails?

Well because of the issues outlined above I would actually prefer if we
can keep the __iomem annotation around.

Well, we'd do that. Since we use the dma_buf_map unconditionally.

So what would happen in the above case, would be:

- memremap would fail. (Otherwise I'd be terribly confused)
- we retry with ioremap and the dma-buf-map member is_iomem would thus be set
- memcpy would do the right thing, based on is_iomem.

/Thomas

Quoted 26 lines

Christian.

Quoted 22 lines

/Thomas

Quoted 18 lines

Christian.

Quoted 14 lines

As for limited arch support for memremap cached, It looks like we only
need to or in "backup" mapping modes in the memremap flags, and we'd
mimic the previous behaviour.

/Thomas

Quoted 3 lines
Quoted 3 lines

/Thomas

maintainer Reviewed-by on v3 8/12
#21 lore raw

Re: [PATCH v3 08/12] drm/ttm: Use drm_memcpy_from_wc_dbm for TTM bo moves

In reply to Thomas Hellström

Am 21.05.21 um 17:32 schrieb Thomas Hellström:

Quoted 6 lines

Use fast wc memcpy for reading out of wc memory for TTM bo moves.

Cc Dave Airlie <[email protected]>
Cc Christian König <[email protected]>
Cc Daniel Vetter <[email protected]>
Quoted 42 lines
---
  drivers/gpu/drm/ttm/ttm_bo_util.c | 9 ++++++++-
  1 file changed, 8 insertions(+), 1 deletion(-)
drivers/gpu/drm/ttm/ttm_bo_util.c +8 -1
index 912cbe8e60a2..4a7d3d672f9a 100644
@@ -31,6 +31,7 @@
 
  #include <drm/ttm/ttm_bo_driver.h>
  #include <drm/ttm/ttm_placement.h>
+#include <drm/drm_memcpy.h>
  #include <drm/drm_vma_manager.h>
  #include <linux/dma-buf-map.h>
  #include <linux/io.h>
@@ -91,6 +92,7 @@ void ttm_move_memcpy(struct ttm_buffer_object *bo,
  	const struct ttm_kmap_iter_ops *src_ops = src_iter->ops;
  	struct ttm_tt *ttm = bo->ttm;
  	struct dma_buf_map src_map, dst_map;
+	bool wc_memcpy;
  	pgoff_t i;
 
  	/* Single TTM move. NOP */
@@ -114,11 +116,16 @@ void ttm_move_memcpy(struct ttm_buffer_object *bo,
  		return;
  	}
 
+	wc_memcpy = ((!src_ops->maps_tt || ttm->caching != ttm_cached) &&
+		     drm_has_memcpy_from_wc());
+
  	for (i = 0; i < dst_mem->num_pages; ++i) {
  		dst_ops->map_local(dst_iter, &dst_map, i);
  		src_ops->map_local(src_iter, &src_map, i);
 
-		if (!src_map.is_iomem && !dst_map.is_iomem) {
+		if (wc_memcpy) {
+			drm_memcpy_from_wc_dbm(&dst_map, &src_map, PAGE_SIZE);
+		} else if (!src_map.is_iomem && !dst_map.is_iomem) {
  			memcpy(dst_map.vaddr, src_map.vaddr, PAGE_SIZE);
  		} else if (!src_map.is_iomem) {
  			dma_buf_map_memcpy_to(&dst_map, src_map.vaddr,