Skip to contentSkip to conversation

Introduce drm evictable lru

PATCH In review Oak Zeng 4 participants 20 messages Nov 2, 2023 – Dec 21, 2023

Subsystems (10)
lore
Share & tools
lore

Oak Zeng

author
#0 lore raw

cover Introduce drm evictable lru

We plan to implement xe driver's shared virtual memory manager (aka SVM) without buffer object concept. This means we won't build our shared virtual memory manager upon TTM infrastructure like amdgpu does.

Even though this approach is more efficient, it does create a problem for memory eviction when there is memory pressure: memory allocated by SVM and memory allocated by TTM should be able to mutually evict from each other. TTM's resource manager maintains a LRU list for each memory type and this list is used to pick up the memory eviction victim. Since we don't use TTM for SVM implementation, SVM allocated memory can't be added to TTM resource manager's LRU list. Thus SVM allocated memory and TTM allocated memory are not mutually evictable.

See more discussion on this topic here:
https://www.spinics.net/lists/dri-devel/msg410740.html

This series solve this problem by creating a shared LRU list b/t SVM and TTM, or any other resource manager.

The basic idea is, abstract a drm_lru_entity structure which is supposed to be embedded in ttm_resource structure, or any other resource manager. The resource LRU list is a list of drm_lru_entity. drm_lru_entity has eviction function pointers which can be used to call back drivers' specific eviction function to evict a memory resource.

Introduce global drm_lru_manager to struct drm_device to manage LRU lists. Each memory type or memory region can have a LRU list. TTM resource manager's LRU list functions including bulk move functions are moved to drm lru manager. drm lru manager provides a evict_first function to evict the first memory resource from LRU list. This function can be called from TTM, SVM or any other resource manager, so all the memory allocated in the drm sub-system can be mutually evicted.

The lru_lock is also moved from struct ttm_device to struct drm_device.

Opens:
1) memory accounting: currently the ttm resource manager's memory accounting functions is kept at ttm resource manager. Since memory accounting should be cross TTM and SVM, it should be ideally in the drm lru manager layer. This will be polished in the future.

2) eviction callback function interface: The current eviction function interface is designed to meet TTM memory eviction requirements. When SVM is in the picture, this interface need to be futher tunned to meet SVM requirement also.

This series is not tested and it is only compiled for xe driver. Some minor changes are needed for other driver such as amdgpu, nouveau etc. I intended to send this out as a request for comment series to get some early feedback, to see whether this is the right direction to go. I will futher polish this series after a direction is agreed.

Oak Zeng (11):
drm/ttm: re-parameter ttm_device_init
drm: move lru_lock from ttm_device to drm_device
drm: introduce drm evictable LRU
drm: Add evict function pointer to drm lru entity
drm: Replace ttm macros with drm macros
drm/ttm: Set lru manager to ttm resource manager drm/ttm: re-parameterize a few ttm functions
drm: Initialize drm lru manager
drm/ttm: Use drm LRU manager iterator
drm/ttm: Implement ttm memory evict functions drm/ttm: Write ttm functions using drm lru manager functions

 drivers/gpu/drm/Makefile                      |   1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c   |   6 +
 .../gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.c   |   6 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |  10 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c        |   6 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h        |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c  |  10 +-
 drivers/gpu/drm/drm_drv.c                     |   1 +
 drivers/gpu/drm/drm_evictable_lru.c           | 266 ++++++++++++++++++
 drivers/gpu/drm/drm_gem_vram_helper.c         |  10 +-
 drivers/gpu/drm/i915/gem/i915_gem_ttm.c       |   6 +-
 drivers/gpu/drm/i915/i915_ttm_buddy_manager.c |  10 +
 drivers/gpu/drm/i915/intel_region_ttm.c       |   4 +-
 drivers/gpu/drm/i915/selftests/mock_region.c  |   2 +-
 drivers/gpu/drm/loongson/lsdc_ttm.c           |  10 +-
 drivers/gpu/drm/nouveau/nouveau_ttm.c         |  22 +-
 drivers/gpu/drm/qxl/qxl_ttm.c                 |   6 +-
 drivers/gpu/drm/radeon/radeon_ttm.c           |  10 +-
 drivers/gpu/drm/ttm/tests/ttm_device_test.c   |   2 +-
 drivers/gpu/drm/ttm/tests/ttm_kunit_helpers.c |   2 +-
 drivers/gpu/drm/ttm/ttm_bo.c                  | 247 ++++++++++++----
 drivers/gpu/drm/ttm/ttm_bo_util.c             |  20 +-
 drivers/gpu/drm/ttm/ttm_bo_vm.c               |   2 +-
 drivers/gpu/drm/ttm/ttm_device.c              |  55 ++--
 drivers/gpu/drm/ttm/ttm_module.h              |   3 +-
 drivers/gpu/drm/ttm/ttm_range_manager.c       |  14 +-
 drivers/gpu/drm/ttm/ttm_resource.c            | 242 +++-------------
 drivers/gpu/drm/ttm/ttm_sys_manager.c         |   8 +-
 drivers/gpu/drm/vmwgfx/vmwgfx_bo.c            |   2 +-
 drivers/gpu/drm/vmwgfx/vmwgfx_bo.h            |   2 +-
 drivers/gpu/drm/vmwgfx/vmwgfx_drv.c           |   6 +-
 .../gpu/drm/vmwgfx/vmwgfx_system_manager.c    |   6 +
 drivers/gpu/drm/xe/xe_bo.c                    |  48 ++--
 drivers/gpu/drm/xe/xe_bo.h                    |   5 +-
 drivers/gpu/drm/xe/xe_device.c                |   2 +-
 drivers/gpu/drm/xe/xe_dma_buf.c               |   4 +-
 drivers/gpu/drm/xe/xe_exec.c                  |   6 +-
 drivers/gpu/drm/xe/xe_migrate.c               |   6 +-
 drivers/gpu/drm/xe/xe_res_cursor.h            |  10 +-
 drivers/gpu/drm/xe/xe_ttm_sys_mgr.c           |   8 +-
 drivers/gpu/drm/xe/xe_ttm_vram_mgr.c          |  18 +-
 drivers/gpu/drm/xe/xe_vm.c                    |   6 +-
 drivers/gpu/drm/xe/xe_vm_types.h              |   2 +-
 include/drm/drm_device.h                      |  12 +
 include/drm/drm_evictable_lru.h               | 260 +++++++++++++++++
 include/drm/ttm/ttm_bo.h                      |  10 +-
 include/drm/ttm/ttm_device.h                  |  13 +-
 include/drm/ttm/ttm_range_manager.h           |  17 +-
 include/drm/ttm/ttm_resource.h                | 117 +++-----
 49 files changed, 1042 insertions(+), 501 deletions(-)
 create mode 100644 drivers/gpu/drm/drm_evictable_lru.c
 create mode 100644 include/drm/drm_evictable_lru.h
Signature · 1 line
2.26.3
author
#1 lore raw

RFC 1/11 drm/ttm: re-parameter ttm_device_init

Change the 3rd parameter of ttm_device_init from struct device * to struct drm_device *. This is a prepare work for moving lru_lock from ttm_device to drm_device.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       | 2 +-
 drivers/gpu/drm/drm_gem_vram_helper.c         | 2 +-
 drivers/gpu/drm/i915/intel_region_ttm.c       | 2 +-
 drivers/gpu/drm/loongson/lsdc_ttm.c           | 2 +-
 drivers/gpu/drm/nouveau/nouveau_ttm.c         | 2 +-
 drivers/gpu/drm/radeon/radeon_ttm.c           | 2 +-
 drivers/gpu/drm/ttm/tests/ttm_kunit_helpers.c | 2 +-
 drivers/gpu/drm/ttm/ttm_device.c              | 7 ++++---
 drivers/gpu/drm/vmwgfx/vmwgfx_drv.c           | 2 +-
 drivers/gpu/drm/xe/xe_device.c                | 2 +-
 include/drm/ttm/ttm_device.h                  | 3 ++-
 11 files changed, 15 insertions(+), 13 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +1 -1
index 4e51dce3aab5..5cdbc901cbe2 100644
@@ -1817,7 +1817,7 @@ int amdgpu_ttm_init(struct amdgpu_device *adev)
 	mutex_init(&adev->mman.gtt_window_lock);
 
 	/* No others user of address space so set it to 0 */
-	r = ttm_device_init(&adev->mman.bdev, &amdgpu_bo_driver, adev->dev,
+	r = ttm_device_init(&adev->mman.bdev, &amdgpu_bo_driver, adev_to_drm(adev),
 			       adev_to_drm(adev)->anon_inode->i_mapping,
 			       adev_to_drm(adev)->vma_offset_manager,
 			       adev->need_swiotlb,
drivers/gpu/drm/drm_gem_vram_helper.c +1 -1
index b67eafa55715..56749e40459f 100644
@@ -1002,7 +1002,7 @@ static int drm_vram_mm_init(struct drm_vram_mm *vmm, struct drm_device *dev,
 	vmm->vram_base = vram_base;
 	vmm->vram_size = vram_size;
 
-	ret = ttm_device_init(&vmm->bdev, &bo_driver, dev->dev,
+	ret = ttm_device_init(&vmm->bdev, &bo_driver, dev,
 				 dev->anon_inode->i_mapping,
 				 dev->vma_offset_manager,
 				 false, true);
drivers/gpu/drm/i915/intel_region_ttm.c +1 -1
index bf6097e7433d..b845782c9859 100644
@@ -33,7 +33,7 @@ int intel_region_ttm_device_init(struct drm_i915_private *dev_priv)
 	struct drm_device *drm = &dev_priv->drm;
 
 	return ttm_device_init(&dev_priv->bdev, i915_ttm_driver(),
-			       drm->dev, drm->anon_inode->i_mapping,
+			       drm, drm->anon_inode->i_mapping,
 			       drm->vma_offset_manager, false, false);
 }
 
drivers/gpu/drm/loongson/lsdc_ttm.c +1 -1
index bf79dc55afa4..bd68cb9366b5 100644
@@ -548,7 +548,7 @@ int lsdc_ttm_init(struct lsdc_device *ldev)
 	unsigned long num_gtt_pages;
 	int ret;
 
-	ret = ttm_device_init(&ldev->bdev, &lsdc_bo_driver, ddev->dev,
+	ret = ttm_device_init(&ldev->bdev, &lsdc_bo_driver, ddev,
 			      ddev->anon_inode->i_mapping,
 			      ddev->vma_offset_manager, false, true);
 	if (ret)
drivers/gpu/drm/nouveau/nouveau_ttm.c +1 -1
index 486f39f31a38..831918437850 100644
@@ -299,7 +299,7 @@ nouveau_ttm_init(struct nouveau_drm *drm)
 		drm->agp.cma = pci->agp.cma;
 	}
 
-	ret = ttm_device_init(&drm->ttm.bdev, &nouveau_bo_driver, drm->dev->dev,
+	ret = ttm_device_init(&drm->ttm.bdev, &nouveau_bo_driver, dev,
 				  dev->anon_inode->i_mapping,
 				  dev->vma_offset_manager,
 				  drm_need_swiotlb(drm->client.mmu.dmabits),
drivers/gpu/drm/radeon/radeon_ttm.c +1 -1
index 4eb83ccc4906..77ca50187162 100644
@@ -688,7 +688,7 @@ int radeon_ttm_init(struct radeon_device *rdev)
 	int r;
 
 	/* No others user of address space so set it to 0 */
-	r = ttm_device_init(&rdev->mman.bdev, &radeon_bo_driver, rdev->dev,
+	r = ttm_device_init(&rdev->mman.bdev, &radeon_bo_driver, rdev->ddev,
 			       rdev->ddev->anon_inode->i_mapping,
 			       rdev->ddev->vma_offset_manager,
 			       rdev->need_swiotlb,
drivers/gpu/drm/ttm/tests/ttm_kunit_helpers.c +1 -1
index 81661d8827aa..63eb6fdc3460 100644
@@ -16,7 +16,7 @@ int ttm_device_kunit_init(struct ttm_test_devices *priv,
 	struct drm_device *drm = priv->drm;
 	int err;
 
-	err = ttm_device_init(ttm, &ttm_dev_funcs, drm->dev,
+	err = ttm_device_init(ttm, &ttm_dev_funcs, drm,
 			      drm->anon_inode->i_mapping,
 			      drm->vma_offset_manager,
 			      use_dma_alloc, use_dma32);
drivers/gpu/drm/ttm/ttm_device.c +4 -3
index 7726a72befc5..12014788b595 100644
@@ -179,7 +179,7 @@ EXPORT_SYMBOL(ttm_device_swapout);
  *
  * @bdev: A pointer to a struct ttm_device to initialize.
  * @funcs: Function table for the device.
- * @dev: The core kernel device pointer for DMA mappings and allocations.
+ * @drm: drm_device pointer
  * @mapping: The address space to use for this bo.
  * @vma_manager: A pointer to a vma manager.
  * @use_dma_alloc: If coherent DMA allocation API should be used.
@@ -190,7 +190,7 @@ EXPORT_SYMBOL(ttm_device_swapout);
  * !0: Failure.
  */
 int ttm_device_init(struct ttm_device *bdev, const struct ttm_device_funcs *funcs,
-		    struct device *dev, struct address_space *mapping,
+		    struct drm_device *drm, struct address_space *mapping,
 		    struct drm_vma_offset_manager *vma_manager,
 		    bool use_dma_alloc, bool use_dma32)
 {
@@ -213,7 +213,8 @@ int ttm_device_init(struct ttm_device *bdev, const struct ttm_device_funcs *func
 	bdev->funcs = funcs;
 
 	ttm_sys_man_init(bdev);
-	ttm_pool_init(&bdev->pool, dev, NUMA_NO_NODE, use_dma_alloc, use_dma32);
+	ttm_pool_init(&bdev->pool, drm?drm->dev:NULL, NUMA_NO_NODE,
+				use_dma_alloc, use_dma32);
 
 	bdev->vma_manager = vma_manager;
 	spin_lock_init(&bdev->lru_lock);
drivers/gpu/drm/vmwgfx/vmwgfx_drv.c +1 -1
index 8b24ecf60e3e..cf1c1f16102a 100644
@@ -1047,7 +1047,7 @@ static int vmw_driver_load(struct vmw_private *dev_priv, u32 pci_id)
 	}
 
 	ret = ttm_device_init(&dev_priv->bdev, &vmw_bo_driver,
-			      dev_priv->drm.dev,
+			      &dev_priv->drm,
 			      dev_priv->drm.anon_inode->i_mapping,
 			      dev_priv->drm.vma_offset_manager,
 			      dev_priv->map_mode == vmw_dma_alloc_coherent,
drivers/gpu/drm/xe/xe_device.c +1 -1
index a964dc205da9..3377bfd5b1a1 100644
@@ -191,7 +191,7 @@ struct xe_device *xe_device_create(struct pci_dev *pdev,
 	if (IS_ERR(xe))
 		return xe;
 
-	err = ttm_device_init(&xe->ttm, &xe_ttm_funcs, xe->drm.dev,
+	err = ttm_device_init(&xe->ttm, &xe_ttm_funcs, &xe->drm,
 			      xe->drm.anon_inode->i_mapping,
 			      xe->drm.vma_offset_manager, false, false);
 	if (WARN_ON(err))
include/drm/ttm/ttm_device.h +2 -1
index c22f30535c84..bab868d55383 100644
@@ -29,6 +29,7 @@
 #include <linux/workqueue.h>
 #include <drm/ttm/ttm_resource.h>
 #include <drm/ttm/ttm_pool.h>
+#include <drm/drm_device.h>
 
 struct ttm_device;
 struct ttm_placement;
@@ -288,7 +289,7 @@ static inline void ttm_set_driver_manager(struct ttm_device *bdev, int type,
 }
 
 int ttm_device_init(struct ttm_device *bdev, const struct ttm_device_funcs *funcs,
-		    struct device *dev, struct address_space *mapping,
+		    struct drm_device *drm, struct address_space *mapping,
 		    struct drm_vma_offset_manager *vma_manager,
 		    bool use_dma_alloc, bool use_dma32);
 void ttm_device_fini(struct ttm_device *bdev);
Signature · 1 line
2.26.3
author
#2 lore raw

RFC 2/11 drm: move lru_lock from ttm_device to drm_device

In reply to Oak Zeng

In the coming patches, we will share the lru list b/t ttm bo based memory allocator and hmm/svm based memory allocator. Thus lru_lock (which is used mainly to protect the lru list) is moved from struct ttm_device to struct drm_device, so this lock can be shared b/t those two memory allocators.

To minimize code change, struct ttm_device still hold a weak reference of lru_lock, so ttm layer can still reference to this lock easily.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c       |  4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c |  4 +-
 drivers/gpu/drm/drm_drv.c                    |  1 +
 drivers/gpu/drm/i915/gem/i915_gem_ttm.c      |  4 +-
 drivers/gpu/drm/ttm/ttm_bo.c                 | 40 +++++++++----------
 drivers/gpu/drm/ttm/ttm_device.c             | 18 ++++-----
 drivers/gpu/drm/ttm/ttm_resource.c           | 42 ++++++++++----------
 drivers/gpu/drm/xe/xe_bo.c                   |  4 +-
 drivers/gpu/drm/xe/xe_exec.c                 |  4 +-
 drivers/gpu/drm/xe/xe_vm.c                   |  4 +-
 include/drm/drm_device.h                     |  5 +++
 include/drm/ttm/ttm_bo.h                     |  4 +-
 include/drm/ttm/ttm_device.h                 |  4 +-
 13 files changed, 72 insertions(+), 66 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +2 -2
index f5daadcec865..747bcad86d5d 100644
@@ -368,9 +368,9 @@ int amdgpu_vm_lock_pd(struct amdgpu_vm *vm, struct drm_exec *exec,
 void amdgpu_vm_move_to_lru_tail(struct amdgpu_device *adev,
 				struct amdgpu_vm *vm)
 {
-	spin_lock(&adev->mman.bdev.lru_lock);
+	spin_lock(adev->mman.bdev.lru_lock);
 	ttm_lru_bulk_move_tail(&vm->lru_bulk_move);
-	spin_unlock(&adev->mman.bdev.lru_lock);
+	spin_unlock(adev->mman.bdev.lru_lock);
 }
 
 /* Create scheduler entities for page table updates */
drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +2 -2
index c7085a747b03..b83e1741905e 100644
@@ -290,9 +290,9 @@ static void amdgpu_vram_mgr_do_reserve(struct ttm_resource_manager *man)
 
 		vis_usage = amdgpu_vram_mgr_vis_size(adev, block);
 		atomic64_add(vis_usage, &mgr->vis_usage);
-		spin_lock(&man->bdev->lru_lock);
+		spin_lock(man->bdev->lru_lock);
 		man->usage += rsv->size;
-		spin_unlock(&man->bdev->lru_lock);
+		spin_unlock(man->bdev->lru_lock);
 		list_move(&rsv->blocks, &mgr->reserved_pages);
 	}
 }
drivers/gpu/drm/drm_drv.c +1 -0
index 3eda026ffac6..1943c38815aa 100644
@@ -623,6 +623,7 @@ static int drm_dev_init(struct drm_device *dev,
 
 	INIT_LIST_HEAD(&dev->managed.resources);
 	spin_lock_init(&dev->managed.lock);
+	spin_lock_init(&dev->lru_lock);
 
 	/* no per-device feature limits by default */
 	dev->driver_features = ~0u;
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +2 -2
index 9227f8146a58..c46f54f83f54 100644
@@ -984,7 +984,7 @@ void i915_ttm_adjust_lru(struct drm_i915_gem_object *obj)
 	/*
 	 * Put on the correct LRU list depending on the MADV status
 	 */
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
 	if (shrinkable) {
 		/* Try to keep shmem_tt from being considered for shrinking. */
 		bo->priority = TTM_MAX_BO_PRIORITY - 1;
@@ -1013,7 +1013,7 @@ void i915_ttm_adjust_lru(struct drm_i915_gem_object *obj)
 	}
 
 	ttm_bo_move_to_lru_tail(bo);
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
 }
 
 /*
drivers/gpu/drm/ttm/ttm_bo.c +20 -20
index e58b7e249816..26e0555bad0c 100644
@@ -68,7 +68,7 @@ static void ttm_bo_mem_space_debug(struct ttm_buffer_object *bo,
  * @bo: The buffer object.
  *
  * Move this BO to the tail of all lru lists used to lookup and reserve an
- * object. This function must be called with struct ttm_global::lru_lock
+ * object. This function must be called with struct drm_device::lru_lock
  * held, and is used to make a BO less likely to be considered for eviction.
  */
 void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo)
@@ -102,13 +102,13 @@ void ttm_bo_set_bulk_move(struct ttm_buffer_object *bo,
 	if (bo->bulk_move == bulk)
 		return;
 
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
 	if (bo->resource)
 		ttm_resource_del_bulk_move(bo->resource, bo);
 	bo->bulk_move = bulk;
 	if (bo->resource)
 		ttm_resource_add_bulk_move(bo->resource, bo);
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
 }
 EXPORT_SYMBOL(ttm_bo_set_bulk_move);
 
@@ -202,9 +202,9 @@ static int ttm_bo_individualize_resv(struct ttm_buffer_object *bo)
 		 * reference it any more. The only tricky case is the trylock on
 		 * the resv object while holding the lru_lock.
 		 */
-		spin_lock(&bo->bdev->lru_lock);
+		spin_lock(bo->bdev->lru_lock);
 		bo->base.resv = &bo->base._resv;
-		spin_unlock(&bo->bdev->lru_lock);
+		spin_unlock(bo->bdev->lru_lock);
 	}
 
 	return r;
@@ -255,7 +255,7 @@ static int ttm_bo_cleanup_refs(struct ttm_buffer_object *bo,
 
 		if (unlock_resv)
 			dma_resv_unlock(bo->base.resv);
-		spin_unlock(&bo->bdev->lru_lock);
+		spin_unlock(bo->bdev->lru_lock);
 
 		lret = dma_resv_wait_timeout(resv, DMA_RESV_USAGE_BOOKKEEP,
 					     interruptible,
@@ -266,7 +266,7 @@ static int ttm_bo_cleanup_refs(struct ttm_buffer_object *bo,
 		else if (lret == 0)
 			return -EBUSY;
 
-		spin_lock(&bo->bdev->lru_lock);
+		spin_lock(bo->bdev->lru_lock);
 		if (unlock_resv && !dma_resv_trylock(bo->base.resv)) {
 			/*
 			 * We raced, and lost, someone else holds the reservation now,
@@ -276,7 +276,7 @@ static int ttm_bo_cleanup_refs(struct ttm_buffer_object *bo,
 			 * delayed destruction would succeed, so just return success
 			 * here.
 			 */
-			spin_unlock(&bo->bdev->lru_lock);
+			spin_unlock(bo->bdev->lru_lock);
 			return 0;
 		}
 		ret = 0;
@@ -285,11 +285,11 @@ static int ttm_bo_cleanup_refs(struct ttm_buffer_object *bo,
 	if (ret) {
 		if (unlock_resv)
 			dma_resv_unlock(bo->base.resv);
-		spin_unlock(&bo->bdev->lru_lock);
+		spin_unlock(bo->bdev->lru_lock);
 		return ret;
 	}
 
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
 	ttm_bo_cleanup_memtype_use(bo);
 
 	if (unlock_resv)
@@ -351,7 +351,7 @@ static void ttm_bo_release(struct kref *kref)
 			ttm_bo_flush_all_fences(bo);
 			bo->deleted = true;
 
-			spin_lock(&bo->bdev->lru_lock);
+			spin_lock(bo->bdev->lru_lock);
 
 			/*
 			 * Make pinned bos immediately available to
@@ -367,7 +367,7 @@ static void ttm_bo_release(struct kref *kref)
 			}
 
 			kref_init(&bo->kref);
-			spin_unlock(&bo->bdev->lru_lock);
+			spin_unlock(bo->bdev->lru_lock);
 
 			INIT_WORK(&bo->delayed_delete, ttm_bo_delayed_delete);
 			queue_work(bdev->wq, &bo->delayed_delete);
@@ -598,7 +598,7 @@ int ttm_mem_evict_first(struct ttm_device *bdev,
 	bool locked = false;
 	int ret;
 
-	spin_lock(&bdev->lru_lock);
+	spin_lock(bdev->lru_lock);
 	ttm_resource_manager_for_each_res(man, &cursor, res) {
 		bool busy;
 
@@ -621,7 +621,7 @@ int ttm_mem_evict_first(struct ttm_device *bdev,
 	if (!bo) {
 		if (busy_bo && !ttm_bo_get_unless_zero(busy_bo))
 			busy_bo = NULL;
-		spin_unlock(&bdev->lru_lock);
+		spin_unlock(bdev->lru_lock);
 		ret = ttm_mem_evict_wait_busy(busy_bo, ctx, ticket);
 		if (busy_bo)
 			ttm_bo_put(busy_bo);
@@ -635,7 +635,7 @@ int ttm_mem_evict_first(struct ttm_device *bdev,
 		return ret;
 	}
 
-	spin_unlock(&bdev->lru_lock);
+	spin_unlock(bdev->lru_lock);
 
 	ret = ttm_bo_evict(bo, ctx);
 	if (locked)
@@ -658,11 +658,11 @@ void ttm_bo_pin(struct ttm_buffer_object *bo)
 {
 	dma_resv_assert_held(bo->base.resv);
 	WARN_ON_ONCE(!kref_read(&bo->kref));
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
 	if (bo->resource)
 		ttm_resource_del_bulk_move(bo->resource, bo);
 	++bo->pin_count;
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
 }
 EXPORT_SYMBOL(ttm_bo_pin);
 
@@ -679,11 +679,11 @@ void ttm_bo_unpin(struct ttm_buffer_object *bo)
 	if (WARN_ON_ONCE(!bo->pin_count))
 		return;
 
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
 	--bo->pin_count;
 	if (bo->resource)
 		ttm_resource_add_bulk_move(bo->resource, bo);
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
 }
 EXPORT_SYMBOL(ttm_bo_unpin);
 
@@ -1156,7 +1156,7 @@ int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx,
 	}
 
 	/* TODO: Cleanup the locking */
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
 
 	/*
 	 * Move to system cached
drivers/gpu/drm/ttm/ttm_device.c +9 -9
index 12014788b595..d18eca86ebd6 100644
@@ -147,7 +147,7 @@ int ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx,
 	unsigned i;
 	int ret;
 
-	spin_lock(&bdev->lru_lock);
+	spin_lock(bdev->lru_lock);
 	for (i = TTM_PL_SYSTEM; i < TTM_NUM_MEM_TYPES; ++i) {
 		man = ttm_manager_type(bdev, i);
 		if (!man || !man->use_tt)
@@ -169,7 +169,7 @@ int ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx,
 				return ret;
 		}
 	}
-	spin_unlock(&bdev->lru_lock);
+	spin_unlock(bdev->lru_lock);
 	return 0;
 }
 EXPORT_SYMBOL(ttm_device_swapout);
@@ -217,7 +217,7 @@ int ttm_device_init(struct ttm_device *bdev, const struct ttm_device_funcs *func
 				use_dma_alloc, use_dma32);
 
 	bdev->vma_manager = vma_manager;
-	spin_lock_init(&bdev->lru_lock);
+	bdev->lru_lock = &drm->lru_lock;
 	INIT_LIST_HEAD(&bdev->pinned);
 	bdev->dev_mapping = mapping;
 	mutex_lock(&ttm_global_mutex);
@@ -244,11 +244,11 @@ void ttm_device_fini(struct ttm_device *bdev)
 	drain_workqueue(bdev->wq);
 	destroy_workqueue(bdev->wq);
 
-	spin_lock(&bdev->lru_lock);
+	spin_lock(bdev->lru_lock);
 	for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i)
 		if (list_empty(&man->lru[0]))
 			pr_debug("Swap list %d was clean\n", i);
-	spin_unlock(&bdev->lru_lock);
+	spin_unlock(bdev->lru_lock);
 
 	ttm_pool_fini(&bdev->pool);
 	ttm_global_release();
@@ -260,7 +260,7 @@ static void ttm_device_clear_lru_dma_mappings(struct ttm_device *bdev,
 {
 	struct ttm_resource *res;
 
-	spin_lock(&bdev->lru_lock);
+	spin_lock(bdev->lru_lock);
 	while ((res = list_first_entry_or_null(list, typeof(*res), lru))) {
 		struct ttm_buffer_object *bo = res->bo;
 
@@ -269,15 +269,15 @@ static void ttm_device_clear_lru_dma_mappings(struct ttm_device *bdev,
 			continue;
 
 		list_del_init(&res->lru);
-		spin_unlock(&bdev->lru_lock);
+		spin_unlock(bdev->lru_lock);
 
 		if (bo->ttm)
 			ttm_tt_unpopulate(bo->bdev, bo->ttm);
 
 		ttm_bo_put(bo);
-		spin_lock(&bdev->lru_lock);
+		spin_lock(bdev->lru_lock);
 	}
-	spin_unlock(&bdev->lru_lock);
+	spin_unlock(bdev->lru_lock);
 }
 
 void ttm_device_clear_dma_mappings(struct ttm_device *bdev)
drivers/gpu/drm/ttm/ttm_resource.c +21 -21

133 lines not on this page · raw message

drivers/gpu/drm/xe/xe_bo.c +2 -2
index 25fdc04627ca..827f798cccc0 100644
@@ -946,9 +946,9 @@ static bool xe_ttm_bo_lock_in_destructor(struct ttm_buffer_object *ttm_bo)
 	 * the ttm_bo refcount is zero at this point. So trylocking *should*
 	 * always succeed here, as long as we hold the lru lock.
 	 */
-	spin_lock(&ttm_bo->bdev->lru_lock);
+	spin_lock(ttm_bo->bdev->lru_lock);
 	locked = dma_resv_trylock(ttm_bo->base.resv);
-	spin_unlock(&ttm_bo->bdev->lru_lock);
+	spin_unlock(ttm_bo->bdev->lru_lock);
 	XE_WARN_ON(!locked);
 
 	return locked;
drivers/gpu/drm/xe/xe_exec.c +2 -2
index 890fadb0a93e..dafebdfb2368 100644
@@ -370,9 +370,9 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 	xe_vm_reactivate_rebind(vm);
 
 	if (!err && !xe_vm_no_dma_fences(vm)) {
-		spin_lock(&xe->ttm.lru_lock);
+		spin_lock(xe->ttm.lru_lock);
 		ttm_lru_bulk_move_tail(&vm->lru_bulk_move);
-		spin_unlock(&xe->ttm.lru_lock);
+		spin_unlock(xe->ttm.lru_lock);
 	}
 
 err_repin:
drivers/gpu/drm/xe/xe_vm.c +2 -2
index a6a0f17fec1d..44e038276d41 100644
@@ -651,9 +651,9 @@ static void preempt_rebind_work_func(struct work_struct *w)
 
 #undef retry_required
 
-	spin_lock(&vm->xe->ttm.lru_lock);
+	spin_lock(vm->xe->ttm.lru_lock);
 	ttm_lru_bulk_move_tail(&vm->lru_bulk_move);
-	spin_unlock(&vm->xe->ttm.lru_lock);
+	spin_unlock(vm->xe->ttm.lru_lock);
 
 	/* Point of no return. */
 	arm_preempt_fences(vm, &preempt_fences);
include/drm/drm_device.h +5 -0
index 7cf4afae2e79..d0b5f42786be 100644
@@ -326,6 +326,11 @@ struct drm_device {
 	 */
 	struct list_head debugfs_list;
 
+	/**
+	 * @lru_lock: Protection for the per manager LRU and destroy lists.
+	 */
+	spinlock_t lru_lock;
+
 	/* Everything below here is for legacy driver, never use! */
 	/* private: */
 #if IS_ENABLED(CONFIG_DRM_LEGACY)
include/drm/ttm/ttm_bo.h +2 -2
index 0223a41a64b2..49f32df32204 100644
@@ -290,9 +290,9 @@ void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo);
 static inline void
 ttm_bo_move_to_lru_tail_unlocked(struct ttm_buffer_object *bo)
 {
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
 	ttm_bo_move_to_lru_tail(bo);
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
 }
 
 static inline void ttm_bo_assign_mem(struct ttm_buffer_object *bo,
include/drm/ttm/ttm_device.h +2 -2
index bab868d55383..4d29e96bd892 100644
@@ -248,9 +248,9 @@ struct ttm_device {
 	struct ttm_pool pool;
 
 	/**
-	 * @lru_lock: Protection for the per manager LRU and ddestroy lists.
+	 * @lru_lock: Weak reference to drm_device::lru_lock.
 	 */
-	spinlock_t lru_lock;
+	spinlock_t *lru_lock;
 
 	/**
 	 * @pinned: Buffer objects which are pinned and so not on any LRU list.
Signature · 1 line
2.26.3
maintainer
#3 lore raw

Re: [RFC 02/11] drm: move lru_lock from ttm_device to drm_device

Am 02.11.23 um 05:32 schrieb Oak Zeng:

Quoted 10 lines

In the coming patches, we will share the lru list b/t ttm bo based memory allocator and hmm/svm based memory allocator. Thus lru_lock (which is used mainly to protect the lru list) is moved from struct ttm_device to struct drm_device, so this lock can be shared b/t those two memory allocators.

To minimize code change, struct ttm_device still hold a weak reference of lru_lock, so ttm layer can still reference to this lock easily.

I would rather like to see drm_device to become the base class of ttm_device.

Similar to how drm_gem_object is the base class of ttm_buffer_object.

That is probably a bit more work, but would also eliminate some of the duplicate house keeping we currently have (e.g. bdev pointer in ttm_buffer_object etc...).

Moving then stuff from the ttm_device into the drm_device becomes trivial.

Regards,
Christian.

Quoted 566 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c       |  4 +-
  drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c |  4 +-
  drivers/gpu/drm/drm_drv.c                    |  1 +
  drivers/gpu/drm/i915/gem/i915_gem_ttm.c      |  4 +-
  drivers/gpu/drm/ttm/ttm_bo.c                 | 40 +++++++++----------
  drivers/gpu/drm/ttm/ttm_device.c             | 18 ++++-----
  drivers/gpu/drm/ttm/ttm_resource.c           | 42 ++++++++++----------
  drivers/gpu/drm/xe/xe_bo.c                   |  4 +-
  drivers/gpu/drm/xe/xe_exec.c                 |  4 +-
  drivers/gpu/drm/xe/xe_vm.c                   |  4 +-
  include/drm/drm_device.h                     |  5 +++
  include/drm/ttm/ttm_bo.h                     |  4 +-
  include/drm/ttm/ttm_device.h                 |  4 +-
  13 files changed, 72 insertions(+), 66 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +2 -2
index f5daadcec865..747bcad86d5d 100644
@@ -368,9 +368,9 @@ int amdgpu_vm_lock_pd(struct amdgpu_vm *vm, struct drm_exec *exec,
  void amdgpu_vm_move_to_lru_tail(struct amdgpu_device *adev,
  				struct amdgpu_vm *vm)
  {
-	spin_lock(&adev->mman.bdev.lru_lock);
+	spin_lock(adev->mman.bdev.lru_lock);
  	ttm_lru_bulk_move_tail(&vm->lru_bulk_move);
-	spin_unlock(&adev->mman.bdev.lru_lock);
+	spin_unlock(adev->mman.bdev.lru_lock);
  }
 
  /* Create scheduler entities for page table updates */
drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +2 -2
index c7085a747b03..b83e1741905e 100644
@@ -290,9 +290,9 @@ static void amdgpu_vram_mgr_do_reserve(struct ttm_resource_manager *man)
 
  		vis_usage = amdgpu_vram_mgr_vis_size(adev, block);
  		atomic64_add(vis_usage, &mgr->vis_usage);
-		spin_lock(&man->bdev->lru_lock);
+		spin_lock(man->bdev->lru_lock);
  		man->usage += rsv->size;
-		spin_unlock(&man->bdev->lru_lock);
+		spin_unlock(man->bdev->lru_lock);
  		list_move(&rsv->blocks, &mgr->reserved_pages);
  	}
  }
drivers/gpu/drm/drm_drv.c +1 -0
index 3eda026ffac6..1943c38815aa 100644
@@ -623,6 +623,7 @@ static int drm_dev_init(struct drm_device *dev,
 
  	INIT_LIST_HEAD(&dev->managed.resources);
  	spin_lock_init(&dev->managed.lock);
+	spin_lock_init(&dev->lru_lock);
 
  	/* no per-device feature limits by default */
  	dev->driver_features = ~0u;
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +2 -2
index 9227f8146a58..c46f54f83f54 100644
@@ -984,7 +984,7 @@ void i915_ttm_adjust_lru(struct drm_i915_gem_object *obj)
  	/*
  	 * Put on the correct LRU list depending on the MADV status
  	 */
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
  	if (shrinkable) {
  		/* Try to keep shmem_tt from being considered for shrinking. */
  		bo->priority = TTM_MAX_BO_PRIORITY - 1;
@@ -1013,7 +1013,7 @@ void i915_ttm_adjust_lru(struct drm_i915_gem_object *obj)
  	}
 
  	ttm_bo_move_to_lru_tail(bo);
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  }
 
  /*
drivers/gpu/drm/ttm/ttm_bo.c +20 -20
index e58b7e249816..26e0555bad0c 100644
@@ -68,7 +68,7 @@ static void ttm_bo_mem_space_debug(struct ttm_buffer_object *bo,
   * @bo: The buffer object.
   *
   * Move this BO to the tail of all lru lists used to lookup and reserve an
- * object. This function must be called with struct ttm_global::lru_lock
+ * object. This function must be called with struct drm_device::lru_lock
   * held, and is used to make a BO less likely to be considered for eviction.
   */
  void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo)
@@ -102,13 +102,13 @@ void ttm_bo_set_bulk_move(struct ttm_buffer_object *bo,
  	if (bo->bulk_move == bulk)
  		return;
 
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
  	if (bo->resource)
  		ttm_resource_del_bulk_move(bo->resource, bo);
  	bo->bulk_move = bulk;
  	if (bo->resource)
  		ttm_resource_add_bulk_move(bo->resource, bo);
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  }
  EXPORT_SYMBOL(ttm_bo_set_bulk_move);
 
@@ -202,9 +202,9 @@ static int ttm_bo_individualize_resv(struct ttm_buffer_object *bo)
  		 * reference it any more. The only tricky case is the trylock on
  		 * the resv object while holding the lru_lock.
  		 */
-		spin_lock(&bo->bdev->lru_lock);
+		spin_lock(bo->bdev->lru_lock);
  		bo->base.resv = &bo->base._resv;
-		spin_unlock(&bo->bdev->lru_lock);
+		spin_unlock(bo->bdev->lru_lock);
  	}
 
  	return r;
@@ -255,7 +255,7 @@ static int ttm_bo_cleanup_refs(struct ttm_buffer_object *bo,
 
  		if (unlock_resv)
  			dma_resv_unlock(bo->base.resv);
-		spin_unlock(&bo->bdev->lru_lock);
+		spin_unlock(bo->bdev->lru_lock);
 
  		lret = dma_resv_wait_timeout(resv, DMA_RESV_USAGE_BOOKKEEP,
  					     interruptible,
@@ -266,7 +266,7 @@ static int ttm_bo_cleanup_refs(struct ttm_buffer_object *bo,
  		else if (lret == 0)
  			return -EBUSY;
 
-		spin_lock(&bo->bdev->lru_lock);
+		spin_lock(bo->bdev->lru_lock);
  		if (unlock_resv && !dma_resv_trylock(bo->base.resv)) {
  			/*
  			 * We raced, and lost, someone else holds the reservation now,
@@ -276,7 +276,7 @@ static int ttm_bo_cleanup_refs(struct ttm_buffer_object *bo,
  			 * delayed destruction would succeed, so just return success
  			 * here.
  			 */
-			spin_unlock(&bo->bdev->lru_lock);
+			spin_unlock(bo->bdev->lru_lock);
  			return 0;
  		}
  		ret = 0;
@@ -285,11 +285,11 @@ static int ttm_bo_cleanup_refs(struct ttm_buffer_object *bo,
  	if (ret) {
  		if (unlock_resv)
  			dma_resv_unlock(bo->base.resv);
-		spin_unlock(&bo->bdev->lru_lock);
+		spin_unlock(bo->bdev->lru_lock);
  		return ret;
  	}
 
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  	ttm_bo_cleanup_memtype_use(bo);
 
  	if (unlock_resv)
@@ -351,7 +351,7 @@ static void ttm_bo_release(struct kref *kref)
  			ttm_bo_flush_all_fences(bo);
  			bo->deleted = true;
 
-			spin_lock(&bo->bdev->lru_lock);
+			spin_lock(bo->bdev->lru_lock);
 
  			/*
  			 * Make pinned bos immediately available to
@@ -367,7 +367,7 @@ static void ttm_bo_release(struct kref *kref)
  			}
 
  			kref_init(&bo->kref);
-			spin_unlock(&bo->bdev->lru_lock);
+			spin_unlock(bo->bdev->lru_lock);
 
  			INIT_WORK(&bo->delayed_delete, ttm_bo_delayed_delete);
  			queue_work(bdev->wq, &bo->delayed_delete);
@@ -598,7 +598,7 @@ int ttm_mem_evict_first(struct ttm_device *bdev,
  	bool locked = false;
  	int ret;
 
-	spin_lock(&bdev->lru_lock);
+	spin_lock(bdev->lru_lock);
  	ttm_resource_manager_for_each_res(man, &cursor, res) {
  		bool busy;
 
@@ -621,7 +621,7 @@ int ttm_mem_evict_first(struct ttm_device *bdev,
  	if (!bo) {
  		if (busy_bo && !ttm_bo_get_unless_zero(busy_bo))
  			busy_bo = NULL;
-		spin_unlock(&bdev->lru_lock);
+		spin_unlock(bdev->lru_lock);
  		ret = ttm_mem_evict_wait_busy(busy_bo, ctx, ticket);
  		if (busy_bo)
  			ttm_bo_put(busy_bo);
@@ -635,7 +635,7 @@ int ttm_mem_evict_first(struct ttm_device *bdev,
  		return ret;
  	}
 
-	spin_unlock(&bdev->lru_lock);
+	spin_unlock(bdev->lru_lock);
 
  	ret = ttm_bo_evict(bo, ctx);
  	if (locked)
@@ -658,11 +658,11 @@ void ttm_bo_pin(struct ttm_buffer_object *bo)
  {
  	dma_resv_assert_held(bo->base.resv);
  	WARN_ON_ONCE(!kref_read(&bo->kref));
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
  	if (bo->resource)
  		ttm_resource_del_bulk_move(bo->resource, bo);
  	++bo->pin_count;
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  }
  EXPORT_SYMBOL(ttm_bo_pin);
 
@@ -679,11 +679,11 @@ void ttm_bo_unpin(struct ttm_buffer_object *bo)
  	if (WARN_ON_ONCE(!bo->pin_count))
  		return;
 
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
  	--bo->pin_count;
  	if (bo->resource)
  		ttm_resource_add_bulk_move(bo->resource, bo);
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  }
  EXPORT_SYMBOL(ttm_bo_unpin);
 
@@ -1156,7 +1156,7 @@ int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx,
  	}
 
  	/* TODO: Cleanup the locking */
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
 
  	/*
  	 * Move to system cached
drivers/gpu/drm/ttm/ttm_device.c +9 -9
index 12014788b595..d18eca86ebd6 100644
@@ -147,7 +147,7 @@ int ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx,
  	unsigned i;
  	int ret;
 
-	spin_lock(&bdev->lru_lock);
+	spin_lock(bdev->lru_lock);
  	for (i = TTM_PL_SYSTEM; i < TTM_NUM_MEM_TYPES; ++i) {
  		man = ttm_manager_type(bdev, i);
  		if (!man || !man->use_tt)
@@ -169,7 +169,7 @@ int ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx,
  				return ret;
  		}
  	}
-	spin_unlock(&bdev->lru_lock);
+	spin_unlock(bdev->lru_lock);
  	return 0;
  }
  EXPORT_SYMBOL(ttm_device_swapout);
@@ -217,7 +217,7 @@ int ttm_device_init(struct ttm_device *bdev, const struct ttm_device_funcs *func
  				use_dma_alloc, use_dma32);
 
  	bdev->vma_manager = vma_manager;
-	spin_lock_init(&bdev->lru_lock);
+	bdev->lru_lock = &drm->lru_lock;
  	INIT_LIST_HEAD(&bdev->pinned);
  	bdev->dev_mapping = mapping;
  	mutex_lock(&ttm_global_mutex);
@@ -244,11 +244,11 @@ void ttm_device_fini(struct ttm_device *bdev)
  	drain_workqueue(bdev->wq);
  	destroy_workqueue(bdev->wq);
 
-	spin_lock(&bdev->lru_lock);
+	spin_lock(bdev->lru_lock);
  	for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i)
  		if (list_empty(&man->lru[0]))
  			pr_debug("Swap list %d was clean\n", i);
-	spin_unlock(&bdev->lru_lock);
+	spin_unlock(bdev->lru_lock);
 
  	ttm_pool_fini(&bdev->pool);
  	ttm_global_release();
@@ -260,7 +260,7 @@ static void ttm_device_clear_lru_dma_mappings(struct ttm_device *bdev,
  {
  	struct ttm_resource *res;
 
-	spin_lock(&bdev->lru_lock);
+	spin_lock(bdev->lru_lock);
  	while ((res = list_first_entry_or_null(list, typeof(*res), lru))) {
  		struct ttm_buffer_object *bo = res->bo;
 
@@ -269,15 +269,15 @@ static void ttm_device_clear_lru_dma_mappings(struct ttm_device *bdev,
  			continue;
 
  		list_del_init(&res->lru);
-		spin_unlock(&bdev->lru_lock);
+		spin_unlock(bdev->lru_lock);
 
  		if (bo->ttm)
  			ttm_tt_unpopulate(bo->bdev, bo->ttm);
 
  		ttm_bo_put(bo);
-		spin_lock(&bdev->lru_lock);
+		spin_lock(bdev->lru_lock);
  	}
-	spin_unlock(&bdev->lru_lock);
+	spin_unlock(bdev->lru_lock);
  }
 
  void ttm_device_clear_dma_mappings(struct ttm_device *bdev)
drivers/gpu/drm/ttm/ttm_resource.c +21 -21
index 46ff9c75bb12..6ada77f51fba 100644
@@ -48,7 +48,7 @@ EXPORT_SYMBOL(ttm_lru_bulk_move_init);
   * @bulk: bulk move structure
   *
   * Bulk move BOs to the LRU tail, only valid to use when driver makes sure that
- * resource order never changes. Should be called with &ttm_device.lru_lock held.
+ * resource order never changes. Should be called with &drm_device.lru_lock held.
   */
  void ttm_lru_bulk_move_tail(struct ttm_lru_bulk_move *bulk)
  {
@@ -62,7 +62,7 @@ void ttm_lru_bulk_move_tail(struct ttm_lru_bulk_move *bulk)
  			if (!pos->first)
  				continue;
 
-			lockdep_assert_held(&pos->first->bo->bdev->lru_lock);
+			lockdep_assert_held(pos->first->bo->bdev->lru_lock);
  			dma_resv_assert_held(pos->first->bo->base.resv);
  			dma_resv_assert_held(pos->last->bo->base.resv);
 
@@ -148,7 +148,7 @@ void ttm_resource_move_to_lru_tail(struct ttm_resource *res)
  	struct ttm_buffer_object *bo = res->bo;
  	struct ttm_device *bdev = bo->bdev;
 
-	lockdep_assert_held(&bo->bdev->lru_lock);
+	lockdep_assert_held(bo->bdev->lru_lock);
 
  	if (bo->pin_count) {
  		list_move_tail(&res->lru, &bdev->pinned);
@@ -191,13 +191,13 @@ void ttm_resource_init(struct ttm_buffer_object *bo,
  	res->bo = bo;
 
  	man = ttm_manager_type(bo->bdev, place->mem_type);
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
  	if (bo->pin_count)
  		list_add_tail(&res->lru, &bo->bdev->pinned);
  	else
  		list_add_tail(&res->lru, &man->lru[bo->priority]);
  	man->usage += res->size;
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  }
  EXPORT_SYMBOL(ttm_resource_init);
 
@@ -216,10 +216,10 @@ void ttm_resource_fini(struct ttm_resource_manager *man,
  {
  	struct ttm_device *bdev = man->bdev;
 
-	spin_lock(&bdev->lru_lock);
+	spin_lock(bdev->lru_lock);
  	list_del_init(&res->lru);
  	man->usage -= res->size;
-	spin_unlock(&bdev->lru_lock);
+	spin_unlock(bdev->lru_lock);
  }
  EXPORT_SYMBOL(ttm_resource_fini);
 
@@ -235,9 +235,9 @@ int ttm_resource_alloc(struct ttm_buffer_object *bo,
  	if (ret)
  		return ret;
 
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
  	ttm_resource_add_bulk_move(*res_ptr, bo);
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  	return 0;
  }
 
@@ -248,9 +248,9 @@ void ttm_resource_free(struct ttm_buffer_object *bo, struct ttm_resource **res)
  	if (!*res)
  		return;
 
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
  	ttm_resource_del_bulk_move(*res, bo);
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  	man = ttm_manager_type(bo->bdev, (*res)->mem_type);
  	man->func->free(man, *res);
  	*res = NULL;
@@ -368,9 +368,9 @@ bool ttm_resource_compat(struct ttm_resource *res,
  void ttm_resource_set_bo(struct ttm_resource *res,
  			 struct ttm_buffer_object *bo)
  {
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
  	res->bo = bo;
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  }
 
  /**
@@ -424,18 +424,18 @@ int ttm_resource_manager_evict_all(struct ttm_device *bdev,
  	 * Can't use standard list traversal since we're unlocking.
  	 */
 
-	spin_lock(&bdev->lru_lock);
+	spin_lock(bdev->lru_lock);
  	for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) {
  		while (!list_empty(&man->lru[i])) {
-			spin_unlock(&bdev->lru_lock);
+			spin_unlock(bdev->lru_lock);
  			ret = ttm_mem_evict_first(bdev, man, NULL, &ctx,
  						  NULL);
  			if (ret)
  				return ret;
-			spin_lock(&bdev->lru_lock);
+			spin_lock(bdev->lru_lock);
  		}
  	}
-	spin_unlock(&bdev->lru_lock);
+	spin_unlock(bdev->lru_lock);
 
  	spin_lock(&man->move_lock);
  	fence = dma_fence_get(man->move);
@@ -463,9 +463,9 @@ uint64_t ttm_resource_manager_usage(struct ttm_resource_manager *man)
  {
  	uint64_t usage;
 
-	spin_lock(&man->bdev->lru_lock);
+	spin_lock(man->bdev->lru_lock);
  	usage = man->usage;
-	spin_unlock(&man->bdev->lru_lock);
+	spin_unlock(man->bdev->lru_lock);
  	return usage;
  }
  EXPORT_SYMBOL(ttm_resource_manager_usage);
@@ -502,7 +502,7 @@ ttm_resource_manager_first(struct ttm_resource_manager *man,
  {
  	struct ttm_resource *res;
 
-	lockdep_assert_held(&man->bdev->lru_lock);
+	lockdep_assert_held(man->bdev->lru_lock);
 
  	for (cursor->priority = 0; cursor->priority < TTM_MAX_BO_PRIORITY;
  	     ++cursor->priority)
@@ -526,7 +526,7 @@ ttm_resource_manager_next(struct ttm_resource_manager *man,
  			  struct ttm_resource_cursor *cursor,
  			  struct ttm_resource *res)
  {
-	lockdep_assert_held(&man->bdev->lru_lock);
+	lockdep_assert_held(man->bdev->lru_lock);
 
  	list_for_each_entry_continue(res, &man->lru[cursor->priority], lru)
  		return res;
drivers/gpu/drm/xe/xe_bo.c +2 -2
index 25fdc04627ca..827f798cccc0 100644
@@ -946,9 +946,9 @@ static bool xe_ttm_bo_lock_in_destructor(struct ttm_buffer_object *ttm_bo)
  	 * the ttm_bo refcount is zero at this point. So trylocking *should*
  	 * always succeed here, as long as we hold the lru lock.
  	 */
-	spin_lock(&ttm_bo->bdev->lru_lock);
+	spin_lock(ttm_bo->bdev->lru_lock);
  	locked = dma_resv_trylock(ttm_bo->base.resv);
-	spin_unlock(&ttm_bo->bdev->lru_lock);
+	spin_unlock(ttm_bo->bdev->lru_lock);
  	XE_WARN_ON(!locked);
 
  	return locked;
drivers/gpu/drm/xe/xe_exec.c +2 -2
index 890fadb0a93e..dafebdfb2368 100644
@@ -370,9 +370,9 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
  	xe_vm_reactivate_rebind(vm);
 
  	if (!err && !xe_vm_no_dma_fences(vm)) {
-		spin_lock(&xe->ttm.lru_lock);
+		spin_lock(xe->ttm.lru_lock);
  		ttm_lru_bulk_move_tail(&vm->lru_bulk_move);
-		spin_unlock(&xe->ttm.lru_lock);
+		spin_unlock(xe->ttm.lru_lock);
  	}
 
  err_repin:
drivers/gpu/drm/xe/xe_vm.c +2 -2
index a6a0f17fec1d..44e038276d41 100644
@@ -651,9 +651,9 @@ static void preempt_rebind_work_func(struct work_struct *w)
 
  #undef retry_required
 
-	spin_lock(&vm->xe->ttm.lru_lock);
+	spin_lock(vm->xe->ttm.lru_lock);
  	ttm_lru_bulk_move_tail(&vm->lru_bulk_move);
-	spin_unlock(&vm->xe->ttm.lru_lock);
+	spin_unlock(vm->xe->ttm.lru_lock);
 
  	/* Point of no return. */
  	arm_preempt_fences(vm, &preempt_fences);
include/drm/drm_device.h +5 -0
index 7cf4afae2e79..d0b5f42786be 100644
@@ -326,6 +326,11 @@ struct drm_device {
  	 */
  	struct list_head debugfs_list;
 
+	/**
+	 * @lru_lock: Protection for the per manager LRU and destroy lists.
+	 */
+	spinlock_t lru_lock;
+
  	/* Everything below here is for legacy driver, never use! */
  	/* private: */
  #if IS_ENABLED(CONFIG_DRM_LEGACY)
include/drm/ttm/ttm_bo.h +2 -2
index 0223a41a64b2..49f32df32204 100644
@@ -290,9 +290,9 @@ void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo);
  static inline void
  ttm_bo_move_to_lru_tail_unlocked(struct ttm_buffer_object *bo)
  {
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
  	ttm_bo_move_to_lru_tail(bo);
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  }
 
  static inline void ttm_bo_assign_mem(struct ttm_buffer_object *bo,
include/drm/ttm/ttm_device.h +2 -2
index bab868d55383..4d29e96bd892 100644
@@ -248,9 +248,9 @@ struct ttm_device {
  	struct ttm_pool pool;
 
  	/**
-	 * @lru_lock: Protection for the per manager LRU and ddestroy lists.
+	 * @lru_lock: Weak reference to drm_device::lru_lock.
  	 */
-	spinlock_t lru_lock;
+	spinlock_t *lru_lock;
 
  	/**
  	 * @pinned: Buffer objects which are pinned and so not on any LRU list.
author
#4 lore raw

RE: [RFC 02/11] drm: move lru_lock from ttm_device to drm_device

Quoted 19 lines

-----Original Message-----
From: Christian König <[email protected]>
Sent: Thursday, November 2, 2023 8:53 AM
To: Zeng, Oak <[email protected]>; [email protected]; intel- [email protected]
Cc: [email protected]; [email protected]; [email protected]; Welty, Brian <[email protected]>
Subject: Re: [RFC 02/11] drm: move lru_lock from ttm_device to drm_device

Am 02.11.23 um 05:32 schrieb Oak Zeng:

Quoted 7 lines

In the coming patches, we will share the lru list b/t ttm bo based memory allocator and hmm/svm based memory allocator. Thus lru_lock (which is used mainly to protect the lru list) is moved from struct ttm_device to struct drm_device, so this lock can be shared b/t those two memory allocators.

Quoted 6 lines
Quoted 3 lines

To minimize code change, struct ttm_device still hold a weak reference of lru_lock, so ttm layer can still reference to this lock easily.

I would rather like to see drm_device to become the base class of ttm_device.

Yah...so drm_dev is the base of ttm_device, and ttm_device is the base of amdgpu_device or xe_device...

Quoted 2 lines

Similar to how drm_gem_object is the base class of ttm_buffer_object.

And ttm_buffer_object is base of amdgpu_bo

Pretty uniformed structure 😊

Quoted 4 lines

That is probably a bit more work, but would also eliminate some of the duplicate house keeping we currently have (e.g. bdev pointer in ttm_buffer_object etc...).

Right, if we do that, we can cast a ttm buffer object to amdgpu_bo, then get amdgpu_device of this amdgpu_bo, then get the ttm_device. Need to write a function to get this. Yes, this way we only need to keep a amdgpu_device pointer in amdgpu_bo. Cleaner than keep bdev pointer in tbo.

Quoted 2 lines

Moving then stuff from the ttm_device into the drm_device becomes trivial.

Agree.

Oak

Quoted 604 lines

Regards,
Christian.

Quoted 18 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c       |  4 +-
  drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c |  4 +-
  drivers/gpu/drm/drm_drv.c                    |  1 +
  drivers/gpu/drm/i915/gem/i915_gem_ttm.c      |  4 +-
  drivers/gpu/drm/ttm/ttm_bo.c                 | 40 +++++++++----------
  drivers/gpu/drm/ttm/ttm_device.c             | 18 ++++-----
  drivers/gpu/drm/ttm/ttm_resource.c           | 42 ++++++++++----------
  drivers/gpu/drm/xe/xe_bo.c                   |  4 +-
  drivers/gpu/drm/xe/xe_exec.c                 |  4 +-
  drivers/gpu/drm/xe/xe_vm.c                   |  4 +-
  include/drm/drm_device.h                     |  5 +++
  include/drm/ttm/ttm_bo.h                     |  4 +-
  include/drm/ttm/ttm_device.h                 |  4 +-
  13 files changed, 72 insertions(+), 66 deletions(-)

Review

Quoted 1 line
(quoted diff) +0 -0

b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_lock_pd(struct amdgpu_vm *vm, struct

Quoted 4 lines

index f5daadcec865..747bcad86d5d 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0 -0
@@ -368,9 +368,9 @@ int amdgpu_vm_lock_pd(struct amdgpu_vm *vm, struct

drm_exec *exec,

Review

Quoted 12 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +2 -2
  void amdgpu_vm_move_to_lru_tail(struct amdgpu_device *adev,
  				struct amdgpu_vm *vm)
  {
-	spin_lock(&adev->mman.bdev.lru_lock);
+	spin_lock(adev->mman.bdev.lru_lock);
  	ttm_lru_bulk_move_tail(&vm->lru_bulk_move);
-	spin_unlock(&adev->mman.bdev.lru_lock);
+	spin_unlock(adev->mman.bdev.lru_lock);
  }
 
  /* Create scheduler entities for page table updates */
(quoted diff) +0 -0

b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c

drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c static void amdgpu_vram_mgr_do_reserve(struct

Quoted 4 lines

index c7085a747b03..b83e1741905e 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +0 -0
@@ -290,9 +290,9 @@ static void amdgpu_vram_mgr_do_reserve(struct

ttm_resource_manager *man)

Review

Quoted 24 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +2 -2
  		vis_usage = amdgpu_vram_mgr_vis_size(adev, block);
  		atomic64_add(vis_usage, &mgr->vis_usage);
-		spin_lock(&man->bdev->lru_lock);
+		spin_lock(man->bdev->lru_lock);
  		man->usage += rsv->size;
-		spin_unlock(&man->bdev->lru_lock);
+		spin_unlock(man->bdev->lru_lock);
  		list_move(&rsv->blocks, &mgr->reserved_pages);
  	}
  }
drivers/gpu/drm/drm_drv.c +1 -0
index 3eda026ffac6..1943c38815aa 100644
@@ -623,6 +623,7 @@ static int drm_dev_init(struct drm_device *dev,
 
  	INIT_LIST_HEAD(&dev->managed.resources);
  	spin_lock_init(&dev->managed.lock);
+	spin_lock_init(&dev->lru_lock);
 
  	/* no per-device feature limits by default */
  	dev->driver_features = ~0u;
(quoted diff) +0 -0

b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c

drivers/gpu/drm/i915/gem/i915_gem_ttm.c void i915_ttm_adjust_lru(struct drm_i915_gem_object

Quoted 4 lines

index 9227f8146a58..c46f54f83f54 100644

drivers/gpu/drm/i915/gem/i915_gem_ttm.c +0 -0
@@ -984,7 +984,7 @@ void i915_ttm_adjust_lru(struct drm_i915_gem_object

*obj)

drivers/gpu/drm/i915/gem/i915_gem_ttm.c void i915_ttm_adjust_lru(struct drm_i915_gem_object

Quoted 9 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +1 -1
  	/*
  	 * Put on the correct LRU list depending on the MADV status
  	 */
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
  	if (shrinkable) {
  		/* Try to keep shmem_tt from being considered for shrinking. */
  		bo->priority = TTM_MAX_BO_PRIORITY - 1;
@@ -1013,7 +1013,7 @@ void i915_ttm_adjust_lru(struct drm_i915_gem_object

*obj)

drivers/gpu/drm/ttm/ttm_bo.c static void ttm_bo_mem_space_debug(struct

Quoted 13 lines
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +1 -1
  	}
 
  	ttm_bo_move_to_lru_tail(bo);
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  }
 
  /*
drivers/gpu/drm/ttm/ttm_bo.c +0 -0
index e58b7e249816..26e0555bad0c 100644
@@ -68,7 +68,7 @@ static void ttm_bo_mem_space_debug(struct

ttm_buffer_object *bo,

drivers/gpu/drm/ttm/ttm_bo.c void ttm_bo_set_bulk_move(struct ttm_buffer_object

Quoted 9 lines
drivers/gpu/drm/ttm/ttm_bo.c +1 -1
   * @bo: The buffer object.
   *
   * Move this BO to the tail of all lru lists used to lookup and reserve an
- * object. This function must be called with struct ttm_global::lru_lock
+ * object. This function must be called with struct drm_device::lru_lock
   * held, and is used to make a BO less likely to be considered for eviction.
   */
  void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo)
@@ -102,13 +102,13 @@ void ttm_bo_set_bulk_move(struct ttm_buffer_object

*bo,

drivers/gpu/drm/ttm/ttm_bo.c static int ttm_bo_individualize_resv(struct

Quoted 16 lines
drivers/gpu/drm/ttm/ttm_bo.c +2 -2
  	if (bo->bulk_move == bulk)
  		return;
 
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
  	if (bo->resource)
  		ttm_resource_del_bulk_move(bo->resource, bo);
  	bo->bulk_move = bulk;
  	if (bo->resource)
  		ttm_resource_add_bulk_move(bo->resource, bo);
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  }
  EXPORT_SYMBOL(ttm_bo_set_bulk_move);
 
@@ -202,9 +202,9 @@ static int ttm_bo_individualize_resv(struct

ttm_buffer_object *bo)

drivers/gpu/drm/ttm/ttm_bo.c static int ttm_bo_cleanup_refs(struct ttm_buffer_object

Quoted 12 lines
drivers/gpu/drm/ttm/ttm_bo.c +2 -2
  		 * reference it any more. The only tricky case is the trylock on
  		 * the resv object while holding the lru_lock.
  		 */
-		spin_lock(&bo->bdev->lru_lock);
+		spin_lock(bo->bdev->lru_lock);
  		bo->base.resv = &bo->base._resv;
-		spin_unlock(&bo->bdev->lru_lock);
+		spin_unlock(bo->bdev->lru_lock);
  	}
 
  	return r;
@@ -255,7 +255,7 @@ static int ttm_bo_cleanup_refs(struct ttm_buffer_object

*bo,

drivers/gpu/drm/ttm/ttm_bo.c static int ttm_bo_cleanup_refs(struct ttm_buffer_object

Quoted 9 lines
drivers/gpu/drm/ttm/ttm_bo.c +1 -1
  		if (unlock_resv)
  			dma_resv_unlock(bo->base.resv);
-		spin_unlock(&bo->bdev->lru_lock);
+		spin_unlock(bo->bdev->lru_lock);
 
  		lret = dma_resv_wait_timeout(resv, DMA_RESV_USAGE_BOOKKEEP,
  					     interruptible,
@@ -266,7 +266,7 @@ static int ttm_bo_cleanup_refs(struct ttm_buffer_object

*bo,

drivers/gpu/drm/ttm/ttm_bo.c static int ttm_bo_cleanup_refs(struct ttm_buffer_object *bo,

Quoted 8 lines
drivers/gpu/drm/ttm/ttm_bo.c +1 -1
  		else if (lret == 0)
  			return -EBUSY;
 
-		spin_lock(&bo->bdev->lru_lock);
+		spin_lock(bo->bdev->lru_lock);
  		if (unlock_resv && !dma_resv_trylock(bo->base.resv)) {
  			/*
  			 * We raced, and lost, someone else holds the reservation

now,

drivers/gpu/drm/ttm/ttm_bo.c static int ttm_bo_cleanup_refs(struct ttm_buffer_object

Quoted 1 line
drivers/gpu/drm/ttm/ttm_bo.c +0 -0
@@ -276,7 +276,7 @@ static int ttm_bo_cleanup_refs(struct ttm_buffer_object

*bo,

drivers/gpu/drm/ttm/ttm_bo.c static int ttm_bo_cleanup_refs(struct ttm_buffer_object

Quoted 9 lines
drivers/gpu/drm/ttm/ttm_bo.c +1 -1
  			 * delayed destruction would succeed, so just return success
  			 * here.
  			 */
-			spin_unlock(&bo->bdev->lru_lock);
+			spin_unlock(bo->bdev->lru_lock);
  			return 0;
  		}
  		ret = 0;
@@ -285,11 +285,11 @@ static int ttm_bo_cleanup_refs(struct ttm_buffer_object

*bo,

drivers/gpu/drm/ttm/ttm_bo.c int ttm_bo_swapout(struct ttm_buffer_object *bo,

Quoted 87 lines
drivers/gpu/drm/ttm/ttm_bo.c +11 -11
  	if (ret) {
  		if (unlock_resv)
  			dma_resv_unlock(bo->base.resv);
-		spin_unlock(&bo->bdev->lru_lock);
+		spin_unlock(bo->bdev->lru_lock);
  		return ret;
  	}
 
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  	ttm_bo_cleanup_memtype_use(bo);
 
  	if (unlock_resv)
@@ -351,7 +351,7 @@ static void ttm_bo_release(struct kref *kref)
  			ttm_bo_flush_all_fences(bo);
  			bo->deleted = true;
 
-			spin_lock(&bo->bdev->lru_lock);
+			spin_lock(bo->bdev->lru_lock);
 
  			/*
  			 * Make pinned bos immediately available to
@@ -367,7 +367,7 @@ static void ttm_bo_release(struct kref *kref)
  			}
 
  			kref_init(&bo->kref);
-			spin_unlock(&bo->bdev->lru_lock);
+			spin_unlock(bo->bdev->lru_lock);
 
  			INIT_WORK(&bo->delayed_delete, ttm_bo_delayed_delete);
  			queue_work(bdev->wq, &bo->delayed_delete);
@@ -598,7 +598,7 @@ int ttm_mem_evict_first(struct ttm_device *bdev,
  	bool locked = false;
  	int ret;
 
-	spin_lock(&bdev->lru_lock);
+	spin_lock(bdev->lru_lock);
  	ttm_resource_manager_for_each_res(man, &cursor, res) {
  		bool busy;
 
@@ -621,7 +621,7 @@ int ttm_mem_evict_first(struct ttm_device *bdev,
  	if (!bo) {
  		if (busy_bo && !ttm_bo_get_unless_zero(busy_bo))
  			busy_bo = NULL;
-		spin_unlock(&bdev->lru_lock);
+		spin_unlock(bdev->lru_lock);
  		ret = ttm_mem_evict_wait_busy(busy_bo, ctx, ticket);
  		if (busy_bo)
  			ttm_bo_put(busy_bo);
@@ -635,7 +635,7 @@ int ttm_mem_evict_first(struct ttm_device *bdev,
  		return ret;
  	}
 
-	spin_unlock(&bdev->lru_lock);
+	spin_unlock(bdev->lru_lock);
 
  	ret = ttm_bo_evict(bo, ctx);
  	if (locked)
@@ -658,11 +658,11 @@ void ttm_bo_pin(struct ttm_buffer_object *bo)
  {
  	dma_resv_assert_held(bo->base.resv);
  	WARN_ON_ONCE(!kref_read(&bo->kref));
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
  	if (bo->resource)
  		ttm_resource_del_bulk_move(bo->resource, bo);
  	++bo->pin_count;
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  }
  EXPORT_SYMBOL(ttm_bo_pin);
 
@@ -679,11 +679,11 @@ void ttm_bo_unpin(struct ttm_buffer_object *bo)
  	if (WARN_ON_ONCE(!bo->pin_count))
  		return;
 
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
  	--bo->pin_count;
  	if (bo->resource)
  		ttm_resource_add_bulk_move(bo->resource, bo);
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  }
  EXPORT_SYMBOL(ttm_bo_unpin);
 
@@ -1156,7 +1156,7 @@ int ttm_bo_swapout(struct ttm_buffer_object *bo,

struct ttm_operation_ctx *ctx,

Review

Quoted 9 lines
drivers/gpu/drm/ttm/ttm_bo.c +1 -1
  	}
 
  	/* TODO: Cleanup the locking */
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
 
  	/*
  	 * Move to system cached
(quoted diff) +0 -0

b/drivers/gpu/drm/ttm/ttm_device.c

drivers/gpu/drm/ttm/ttm_device.c int ttm_device_swapout(struct ttm_device *bdev, struct

Quoted 4 lines

index 12014788b595..d18eca86ebd6 100644

drivers/gpu/drm/ttm/ttm_device.c +0 -0
@@ -147,7 +147,7 @@ int ttm_device_swapout(struct ttm_device *bdev, struct

ttm_operation_ctx *ctx,

drivers/gpu/drm/ttm/ttm_device.c int ttm_device_swapout(struct ttm_device *bdev, struct

Quoted 9 lines
drivers/gpu/drm/ttm/ttm_device.c +1 -1
  	unsigned i;
  	int ret;
 
-	spin_lock(&bdev->lru_lock);
+	spin_lock(bdev->lru_lock);
  	for (i = TTM_PL_SYSTEM; i < TTM_NUM_MEM_TYPES; ++i) {
  		man = ttm_manager_type(bdev, i);
  		if (!man || !man->use_tt)
@@ -169,7 +169,7 @@ int ttm_device_swapout(struct ttm_device *bdev, struct

ttm_operation_ctx *ctx,

drivers/gpu/drm/ttm/ttm_device.c int ttm_device_init(struct ttm_device *bdev, const struct

Quoted 9 lines
drivers/gpu/drm/ttm/ttm_device.c +1 -1
  				return ret;
  		}
  	}
-	spin_unlock(&bdev->lru_lock);
+	spin_unlock(bdev->lru_lock);
  	return 0;
  }
  EXPORT_SYMBOL(ttm_device_swapout);
@@ -217,7 +217,7 @@ int ttm_device_init(struct ttm_device *bdev, const struct

ttm_device_funcs *func

drivers/gpu/drm/ttm/ttm_device.c static void ttm_device_clear_lru_dma_mappings(struct

Quoted 23 lines
drivers/gpu/drm/ttm/ttm_device.c +3 -3
  				use_dma_alloc, use_dma32);
 
  	bdev->vma_manager = vma_manager;
-	spin_lock_init(&bdev->lru_lock);
+	bdev->lru_lock = &drm->lru_lock;
  	INIT_LIST_HEAD(&bdev->pinned);
  	bdev->dev_mapping = mapping;
  	mutex_lock(&ttm_global_mutex);
@@ -244,11 +244,11 @@ void ttm_device_fini(struct ttm_device *bdev)
  	drain_workqueue(bdev->wq);
  	destroy_workqueue(bdev->wq);
 
-	spin_lock(&bdev->lru_lock);
+	spin_lock(bdev->lru_lock);
  	for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i)
  		if (list_empty(&man->lru[0]))
  			pr_debug("Swap list %d was clean\n", i);
-	spin_unlock(&bdev->lru_lock);
+	spin_unlock(bdev->lru_lock);
 
  	ttm_pool_fini(&bdev->pool);
  	ttm_global_release();
@@ -260,7 +260,7 @@ static void ttm_device_clear_lru_dma_mappings(struct

ttm_device *bdev,

drivers/gpu/drm/ttm/ttm_device.c static void ttm_device_clear_lru_dma_mappings(struct

Quoted 9 lines
drivers/gpu/drm/ttm/ttm_device.c +1 -1
  {
  	struct ttm_resource *res;
 
-	spin_lock(&bdev->lru_lock);
+	spin_lock(bdev->lru_lock);
  	while ((res = list_first_entry_or_null(list, typeof(*res), lru))) {
  		struct ttm_buffer_object *bo = res->bo;
 
@@ -269,15 +269,15 @@ static void ttm_device_clear_lru_dma_mappings(struct

ttm_device *bdev,

Review

Quoted 19 lines
drivers/gpu/drm/ttm/ttm_device.c +3 -3
  			continue;
 
  		list_del_init(&res->lru);
-		spin_unlock(&bdev->lru_lock);
+		spin_unlock(bdev->lru_lock);
 
  		if (bo->ttm)
  			ttm_tt_unpopulate(bo->bdev, bo->ttm);
 
  		ttm_bo_put(bo);
-		spin_lock(&bdev->lru_lock);
+		spin_lock(bdev->lru_lock);
  	}
-	spin_unlock(&bdev->lru_lock);
+	spin_unlock(bdev->lru_lock);
  }
 
  void ttm_device_clear_dma_mappings(struct ttm_device *bdev)
(quoted diff) +0 -0

b/drivers/gpu/drm/ttm/ttm_resource.c

drivers/gpu/drm/ttm/ttm_resource.c void ttm_resource_move_to_lru_tail(struct ttm_resource

Quoted 22 lines

index 46ff9c75bb12..6ada77f51fba 100644

drivers/gpu/drm/ttm/ttm_resource.c +2 -2
@@ -48,7 +48,7 @@ EXPORT_SYMBOL(ttm_lru_bulk_move_init);
   * @bulk: bulk move structure
   *
   * Bulk move BOs to the LRU tail, only valid to use when driver makes sure that
- * resource order never changes. Should be called with &ttm_device.lru_lock held.
+ * resource order never changes. Should be called with &drm_device.lru_lock held.
   */
  void ttm_lru_bulk_move_tail(struct ttm_lru_bulk_move *bulk)
  {
@@ -62,7 +62,7 @@ void ttm_lru_bulk_move_tail(struct ttm_lru_bulk_move *bulk)
  			if (!pos->first)
  				continue;
 
-			lockdep_assert_held(&pos->first->bo->bdev->lru_lock);
+			lockdep_assert_held(pos->first->bo->bdev->lru_lock);
  			dma_resv_assert_held(pos->first->bo->base.resv);
  			dma_resv_assert_held(pos->last->bo->base.resv);
 
@@ -148,7 +148,7 @@ void ttm_resource_move_to_lru_tail(struct ttm_resource

*res)

drivers/gpu/drm/ttm/ttm_resource.c void ttm_resource_fini(struct ttm_resource_manager

Quoted 25 lines
drivers/gpu/drm/ttm/ttm_resource.c +3 -3
  	struct ttm_buffer_object *bo = res->bo;
  	struct ttm_device *bdev = bo->bdev;
 
-	lockdep_assert_held(&bo->bdev->lru_lock);
+	lockdep_assert_held(bo->bdev->lru_lock);
 
  	if (bo->pin_count) {
  		list_move_tail(&res->lru, &bdev->pinned);
@@ -191,13 +191,13 @@ void ttm_resource_init(struct ttm_buffer_object *bo,
  	res->bo = bo;
 
  	man = ttm_manager_type(bo->bdev, place->mem_type);
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
  	if (bo->pin_count)
  		list_add_tail(&res->lru, &bo->bdev->pinned);
  	else
  		list_add_tail(&res->lru, &man->lru[bo->priority]);
  	man->usage += res->size;
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  }
  EXPORT_SYMBOL(ttm_resource_init);
 
@@ -216,10 +216,10 @@ void ttm_resource_fini(struct ttm_resource_manager

*man,

drivers/gpu/drm/ttm/ttm_resource.c void ttm_resource_free(struct ttm_buffer_object *bo,

Quoted 25 lines
drivers/gpu/drm/ttm/ttm_resource.c +4 -4
  {
  	struct ttm_device *bdev = man->bdev;
 
-	spin_lock(&bdev->lru_lock);
+	spin_lock(bdev->lru_lock);
  	list_del_init(&res->lru);
  	man->usage -= res->size;
-	spin_unlock(&bdev->lru_lock);
+	spin_unlock(bdev->lru_lock);
  }
  EXPORT_SYMBOL(ttm_resource_fini);
 
@@ -235,9 +235,9 @@ int ttm_resource_alloc(struct ttm_buffer_object *bo,
  	if (ret)
  		return ret;
 
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
  	ttm_resource_add_bulk_move(*res_ptr, bo);
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  	return 0;
  }
 
@@ -248,9 +248,9 @@ void ttm_resource_free(struct ttm_buffer_object *bo,

struct ttm_resource **res)

drivers/gpu/drm/ttm/ttm_resource.c int ttm_resource_manager_evict_all(struct ttm_device

Quoted 24 lines
drivers/gpu/drm/ttm/ttm_resource.c +4 -4
  	if (!*res)
  		return;
 
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
  	ttm_resource_del_bulk_move(*res, bo);
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  	man = ttm_manager_type(bo->bdev, (*res)->mem_type);
  	man->func->free(man, *res);
  	*res = NULL;
@@ -368,9 +368,9 @@ bool ttm_resource_compat(struct ttm_resource *res,
  void ttm_resource_set_bo(struct ttm_resource *res,
  			 struct ttm_buffer_object *bo)
  {
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
  	res->bo = bo;
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  }
 
  /**
@@ -424,18 +424,18 @@ int ttm_resource_manager_evict_all(struct ttm_device

*bdev,

drivers/gpu/drm/ttm/ttm_resource.c uint64_t ttm_resource_manager_usage(struct

Quoted 23 lines
drivers/gpu/drm/ttm/ttm_resource.c +4 -4
  	 * Can't use standard list traversal since we're unlocking.
  	 */
 
-	spin_lock(&bdev->lru_lock);
+	spin_lock(bdev->lru_lock);
  	for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) {
  		while (!list_empty(&man->lru[i])) {
-			spin_unlock(&bdev->lru_lock);
+			spin_unlock(bdev->lru_lock);
  			ret = ttm_mem_evict_first(bdev, man, NULL, &ctx,
  						  NULL);
  			if (ret)
  				return ret;
-			spin_lock(&bdev->lru_lock);
+			spin_lock(bdev->lru_lock);
  		}
  	}
-	spin_unlock(&bdev->lru_lock);
+	spin_unlock(bdev->lru_lock);
 
  	spin_lock(&man->move_lock);
  	fence = dma_fence_get(man->move);
@@ -463,9 +463,9 @@ uint64_t ttm_resource_manager_usage(struct

ttm_resource_manager *man)

drivers/gpu/drm/ttm/ttm_resource.c ttm_resource_manager_first(struct

Quoted 12 lines
drivers/gpu/drm/ttm/ttm_resource.c +2 -2
  {
  	uint64_t usage;
 
-	spin_lock(&man->bdev->lru_lock);
+	spin_lock(man->bdev->lru_lock);
  	usage = man->usage;
-	spin_unlock(&man->bdev->lru_lock);
+	spin_unlock(man->bdev->lru_lock);
  	return usage;
  }
  EXPORT_SYMBOL(ttm_resource_manager_usage);
@@ -502,7 +502,7 @@ ttm_resource_manager_first(struct

ttm_resource_manager *man,

drivers/gpu/drm/ttm/ttm_resource.c ttm_resource_manager_next(struct

Quoted 9 lines
drivers/gpu/drm/ttm/ttm_resource.c +1 -1
  {
  	struct ttm_resource *res;
 
-	lockdep_assert_held(&man->bdev->lru_lock);
+	lockdep_assert_held(man->bdev->lru_lock);
 
  	for (cursor->priority = 0; cursor->priority < TTM_MAX_BO_PRIORITY;
  	     ++cursor->priority)
@@ -526,7 +526,7 @@ ttm_resource_manager_next(struct

ttm_resource_manager *man,

drivers/gpu/drm/xe/xe_bo.c static bool xe_ttm_bo_lock_in_destructor(struct

Quoted 13 lines
drivers/gpu/drm/ttm/ttm_resource.c +1 -1
  			  struct ttm_resource_cursor *cursor,
  			  struct ttm_resource *res)
  {
-	lockdep_assert_held(&man->bdev->lru_lock);
+	lockdep_assert_held(man->bdev->lru_lock);
 
  	list_for_each_entry_continue(res, &man->lru[cursor->priority], lru)
  		return res;
drivers/gpu/drm/xe/xe_bo.c +0 -0
index 25fdc04627ca..827f798cccc0 100644
@@ -946,9 +946,9 @@ static bool xe_ttm_bo_lock_in_destructor(struct

ttm_buffer_object *ttm_bo)

drivers/gpu/drm/xe/xe_exec.c int xe_exec_ioctl(struct drm_device *dev, void *data,

Quoted 16 lines
drivers/gpu/drm/xe/xe_bo.c +2 -2
  	 * the ttm_bo refcount is zero at this point. So trylocking *should*
  	 * always succeed here, as long as we hold the lru lock.
  	 */
-	spin_lock(&ttm_bo->bdev->lru_lock);
+	spin_lock(ttm_bo->bdev->lru_lock);
  	locked = dma_resv_trylock(ttm_bo->base.resv);
-	spin_unlock(&ttm_bo->bdev->lru_lock);
+	spin_unlock(ttm_bo->bdev->lru_lock);
  	XE_WARN_ON(!locked);
 
  	return locked;
drivers/gpu/drm/xe/xe_exec.c +0 -0
index 890fadb0a93e..dafebdfb2368 100644
@@ -370,9 +370,9 @@ int xe_exec_ioctl(struct drm_device *dev, void *data,

struct drm_file *file)

drivers/gpu/drm/xe/xe_vm.c static void preempt_rebind_work_func(struct work_struct

Quoted 16 lines
drivers/gpu/drm/xe/xe_exec.c +2 -2
  	xe_vm_reactivate_rebind(vm);
 
  	if (!err && !xe_vm_no_dma_fences(vm)) {
-		spin_lock(&xe->ttm.lru_lock);
+		spin_lock(xe->ttm.lru_lock);
  		ttm_lru_bulk_move_tail(&vm->lru_bulk_move);
-		spin_unlock(&xe->ttm.lru_lock);
+		spin_unlock(xe->ttm.lru_lock);
  	}
 
  err_repin:
drivers/gpu/drm/xe/xe_vm.c +0 -0
index a6a0f17fec1d..44e038276d41 100644
@@ -651,9 +651,9 @@ static void preempt_rebind_work_func(struct work_struct

*w)

include/drm/ttm/ttm_bo.h void ttm_bo_move_to_lru_tail(struct ttm_buffer_object

Quoted 32 lines
drivers/gpu/drm/xe/xe_vm.c +2 -2
  #undef retry_required
 
-	spin_lock(&vm->xe->ttm.lru_lock);
+	spin_lock(vm->xe->ttm.lru_lock);
  	ttm_lru_bulk_move_tail(&vm->lru_bulk_move);
-	spin_unlock(&vm->xe->ttm.lru_lock);
+	spin_unlock(vm->xe->ttm.lru_lock);
 
  	/* Point of no return. */
  	arm_preempt_fences(vm, &preempt_fences);
include/drm/drm_device.h +5 -0
index 7cf4afae2e79..d0b5f42786be 100644
@@ -326,6 +326,11 @@ struct drm_device {
  	 */
  	struct list_head debugfs_list;
 
+	/**
+	 * @lru_lock: Protection for the per manager LRU and destroy lists.
+	 */
+	spinlock_t lru_lock;
+
  	/* Everything below here is for legacy driver, never use! */
  	/* private: */
  #if IS_ENABLED(CONFIG_DRM_LEGACY)
include/drm/ttm/ttm_bo.h +0 -0
index 0223a41a64b2..49f32df32204 100644
@@ -290,9 +290,9 @@ void ttm_bo_move_to_lru_tail(struct ttm_buffer_object

*bo);

Quoted 27 lines
include/drm/ttm/ttm_bo.h +2 -2
  static inline void
  ttm_bo_move_to_lru_tail_unlocked(struct ttm_buffer_object *bo)
  {
-	spin_lock(&bo->bdev->lru_lock);
+	spin_lock(bo->bdev->lru_lock);
  	ttm_bo_move_to_lru_tail(bo);
-	spin_unlock(&bo->bdev->lru_lock);
+	spin_unlock(bo->bdev->lru_lock);
  }
 
  static inline void ttm_bo_assign_mem(struct ttm_buffer_object *bo,
include/drm/ttm/ttm_device.h +2 -2
index bab868d55383..4d29e96bd892 100644
@@ -248,9 +248,9 @@ struct ttm_device {
  	struct ttm_pool pool;
 
  	/**
-	 * @lru_lock: Protection for the per manager LRU and ddestroy lists.
+	 * @lru_lock: Weak reference to drm_device::lru_lock.
  	 */
-	spinlock_t lru_lock;
+	spinlock_t *lru_lock;
 
  	/**
  	 * @pinned: Buffer objects which are pinned and so not on any LRU list.
author
#5 lore raw

RFC 3/11 drm: introduce drm evictable LRU

In reply to Oak Zeng

drm LRU manager is introuced for resource eviction purpose. It maintains a LRU list per resource type. It provides functions to add or remove resource to or from the list. It also provides function to retrieve the first entity on the LRU list.

drm LRU manager also provides functions for bulk moving resources on the LRU lists.

drm LRU manager also does very basic memory accounting function, i.e., LRU manager keeps a size of this resource type and a usage member for how much of resource has been added to this LRU manager's LRU list. TTM resource manager memory accounting functoins such as struct ttm_resource_manager::size and struct ttm_resource_manger::usage are still kept. In the future, when SVM codes are in the picture, those memory accounting functions need some rework to consider the memory used by both TTM and SVM.

For one device, a global drm LRU manager per resource type should be created/initialized at device initialization time. Drm LRU manager instances are embedded in struct drm_device.

It is pretty much moving some of the ttm resource manager functions to the drm layer. The reason of this code refactory is, we want to create a single LRU list for memory allocated from BO(buffer object) based driver and hmm/svm(shared virtual memory) based driver, thus BO driver and svm driver can evict memory from each other.

Previously the LRU list in TTM resource manager (lru field in struct ttm_reource_manager) is coupled with ttm_buffer_object concept, i.e., each ttm resource is backed by a ttm_buffer_object and the LRU list is essentially a list of ttm_buffer_object. Due to this behavior, the TTM resource manager can't be used by hmm/svm driver as we don't plan to have the BO concept for the hmm/svm implemenation. So we decouple the evictable LRU list from the BO concept in this series.

The design goal of drm lru manager is to make it as lean as possible. So each lru entity only has a list node member used to link this entity to the evictable LRU list, and the basic resource size/type/priority of this entity. It doesn't have any driver specify information. A lru entity also has a function pointer of evict function. This is used to implement ttm or svm specific eviction function. A lru entity is supposed to be embedded in a driver specific structure such as struct ttm_resource, see the usage in the next patch of this series.

The ttm resource manager, and some of the ttm_bo functions such as ttm_mem_evict_first will be rewriten using the new drm lru manager library, see the next patch in this series.

The future hmm/svm implemenation will call lru manager function to add hmm/svm allocations to the shared evictable lru list.

Lock design: previously ttm_resource LRU list is protected by a device global ttm_device::lru_lock (bdev->lru_lock in codes). This lock also protects ttm_buffer_object::pin_count, ttm_resource_manager::usage, ttm_resource::bo, ttm_device::pinned list etc. With this refactory, lru_lock is moved out of ttm_device and is added to struct drm_deive, so it can be shared b/t ttm code and svm code.

---
 drivers/gpu/drm/Makefile            |   1 +
 drivers/gpu/drm/drm_evictable_lru.c | 232 ++++++++++++++++++++++++++++
 include/drm/drm_device.h            |   7 +
 include/drm/drm_evictable_lru.h     | 188 ++++++++++++++++++++++
 4 files changed, 428 insertions(+)
 create mode 100644 drivers/gpu/drm/drm_evictable_lru.c
 create mode 100644 include/drm/drm_evictable_lru.h
drivers/gpu/drm/Makefile +1 -0
index 1ad88efb1752..13953b0d271b 100644
@@ -46,6 +46,7 @@ drm-y := \
 	drm_vblank_work.o \
 	drm_vma_manager.o \
 	drm_gpuva_mgr.o \
+	drm_evictable_lru.o \
 	drm_writeback.o
 drm-$(CONFIG_DRM_LEGACY) += \
 	drm_agpsupport.o \
drivers/gpu/drm/drm_evictable_lru.c new file +232 -0

232 lines not on this page · raw message

include/drm/drm_device.h +7 -0
index d0b5f42786be..1bdcd34d3f6b 100644
@@ -8,6 +8,7 @@
 
 #include <drm/drm_legacy.h>
 #include <drm/drm_mode_config.h>
+#include <drm/drm_evictable_lru.h>
 
 struct drm_driver;
 struct drm_minor;
@@ -331,6 +332,12 @@ struct drm_device {
 	 */
 	spinlock_t lru_lock;
 
+	/**
+	 * @lru_mgr: Device global lru managers per memory type or memory
+	 * region. Each lru manager manages a lru list of this memory type.
+	 */
+	struct drm_lru_mgr lru_mgr[DRM_NUM_MEM_TYPES];
+
 	/* Everything below here is for legacy driver, never use! */
 	/* private: */
 #if IS_ENABLED(CONFIG_DRM_LEGACY)
include/drm/drm_evictable_lru.h new file +188 -0

188 lines not on this page · raw message

Signature · 1 line
2.26.3
maintainer
#6 lore raw

Re: [RFC 03/11] drm: introduce drm evictable LRU

Am 02.11.23 um 05:32 schrieb Oak Zeng:

Quoted 2 lines

drm LRU manager is introuced for resource eviction purpose. It maintains a LRU list per resource type.

Shouldn't we first add the possible resource types in a separate patch?

Quoted 3 lines

It provides functions to add or remove
resource to or from the list. It also provides function to retrieve the first entity on the LRU list.

+ functions to iterate over them.

Quoted 4 lines

drm LRU manager also provides functions for bulk moving resources on the LRU lists.

Quoted 8 lines

drm LRU manager also does very basic memory accounting function, i.e., LRU manager keeps a size of this resource type and a usage member for how much of resource has been added to this LRU manager's LRU list. TTM resource manager memory accounting functoins such as struct ttm_resource_manager::size and struct ttm_resource_manger::usage are still kept. In the future, when SVM codes are in the picture, those memory accounting functions need some rework to consider the memory used by both TTM and SVM.

Please keep in mind that this structure needs to extremely small to be usable for SVM. E.g. struct page size small :)

At least HMM based implementations ideally wants to have one for each page or something like that.

Quoted 10 lines

For one device, a global drm LRU manager per resource type should be created/initialized at device initialization time. Drm LRU manager instances are embedded in struct drm_device.

It is pretty much moving some of the ttm resource manager functions to the drm layer. The reason of this code refactory is, we want to create a single LRU list for memory allocated from BO(buffer object) based driver and hmm/svm(shared virtual memory) based driver, thus BO driver and svm driver can evict memory from each other.

Quoted 4 lines

Previously the LRU list in TTM resource manager (lru field in struct ttm_reource_manager) is coupled with ttm_buffer_object concept, i.e., each ttm resource is backed by a ttm_buffer_object and the LRU list is essentially a list of ttm_buffer_object.

Actually it's the other way around. The resource provides the backing of the BO.

And when a BO moves around it can temporary be that multiple resource point to the same BO.

I also want to have a more advanced iterator at some point where we grab the BO lock for keeping a reference into the LRU list. Not sure how to do this if we don't have the BO here any more.

Need to think about that further,
Christian.

Quoted 507 lines

Due to this behavior, the
TTM resource manager can't be used by hmm/svm driver as we don't plan to have the BO concept for the hmm/svm implemenation. So we decouple the evictable LRU list from the BO concept in this series.

The design goal of drm lru manager is to make it as lean as possible. So each lru entity only has a list node member used to link this entity to the evictable LRU list, and the basic resource size/type/priority of this entity. It doesn't have any driver specify information. A lru entity also has a function pointer of evict function. This is used to implement ttm or svm specific eviction function. A lru entity is supposed to be embedded in a driver specific structure such as struct ttm_resource, see the usage in the next patch of this series.

The ttm resource manager, and some of the ttm_bo functions such as ttm_mem_evict_first will be rewriten using the new drm lru manager library, see the next patch in this series.

The future hmm/svm implemenation will call lru manager function to add hmm/svm allocations to the shared evictable lru list.

Lock design: previously ttm_resource LRU list is protected by a device global ttm_device::lru_lock (bdev->lru_lock in codes). This lock also protects ttm_buffer_object::pin_count, ttm_resource_manager::usage, ttm_resource::bo, ttm_device::pinned list etc. With this refactory, lru_lock is moved out of ttm_device and is added to struct drm_deive, so it can be shared b/t ttm code and svm code.

---
  drivers/gpu/drm/Makefile            |   1 +
  drivers/gpu/drm/drm_evictable_lru.c | 232 ++++++++++++++++++++++++++++
  include/drm/drm_device.h            |   7 +
  include/drm/drm_evictable_lru.h     | 188 ++++++++++++++++++++++
  4 files changed, 428 insertions(+)

create mode 100644 drivers/gpu/drm/drm_evictable_lru.c create mode 100644 include/drm/drm_evictable_lru.h

drivers/gpu/drm/Makefile +1 -0
index 1ad88efb1752..13953b0d271b 100644
@@ -46,6 +46,7 @@ drm-y := \
  	drm_vblank_work.o \
  	drm_vma_manager.o \
  	drm_gpuva_mgr.o \
+	drm_evictable_lru.o \
  	drm_writeback.o
  drm-$(CONFIG_DRM_LEGACY) += \
  	drm_agpsupport.o \
drivers/gpu/drm/drm_evictable_lru.c new file +232 -0
new file mode 100644
index 000000000000..2ba9105cca03
@@ -0,0 +1,232 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#include <linux/lockdep.h>
+#include <linux/container_of.h>
+#include <drm/drm_evictable_lru.h>
+#include <drm/drm_device.h>
+
+static inline struct drm_lru_mgr *entity_to_mgr(struct drm_lru_entity *entity)
+{
+	struct drm_lru_mgr *mgr;
+
+	mgr = &entity->drm->lru_mgr[entity->mem_type];
+	BUG_ON(!mgr->used);
+
+	return mgr;
+}
+
+void drm_lru_entity_init(struct drm_lru_entity *entity, struct drm_device *drm,
+			uint32_t mem_type, uint64_t size, uint32_t priority)
+{
+	entity->drm = drm;
+	entity->mem_type = mem_type;
+	entity->size = size;
+	entity->priority = priority;
+	INIT_LIST_HEAD(&entity->lru);
+}
+
+/**
+ * drm_lru_mgr_init
+ *
+ * @mgr: drm lru manager to init
+ * @size: size of the resource managed by this manager
+ * @lock: pointer of the global lru_lock
+ *
+ * Initialize a drm lru manager
+ */
+void drm_lru_mgr_init(struct drm_lru_mgr *mgr, uint64_t size, spinlock_t *lock)
+{
+	unsigned j;
+
+	mgr->used = true;
+	mgr->size = size;
+	mgr->usage = 0;
+	mgr->lru_lock = lock;
+
+	for(j = 0; j < DRM_MAX_LRU_PRIORITY; j++)
+		INIT_LIST_HEAD(&mgr->lru[j]);
+}
+
+void drm_lru_bulk_move_init(struct drm_lru_bulk_move *bulk_move)
+{
+	memset(bulk_move, 0, sizeof(*bulk_move));
+}
+
+/**
+ * drm_lru_first
+ *
+ * @mgr: drm lru manager to iterate over
+ * @cursor: cursor of the current position
+ *
+ * Returns the first entity in drm lru manager
+ */
+struct drm_lru_entity *
+drm_lru_first(struct drm_lru_mgr *mgr, struct drm_lru_cursor *cursor)
+{
+	struct drm_lru_entity *entity;
+
+	lockdep_assert_held(mgr->lru_lock);
+
+	for(cursor->priority = 0; cursor->priority < DRM_MAX_LRU_PRIORITY; ++cursor->priority)
+		list_for_each_entry(entity, &mgr->lru[cursor->priority], lru)
+			return entity;
+
+	return NULL;
+}
+
+/**
+ * drm_lru_next
+ *
+ * @mgr: drm lru manager to iterate over
+ * @cursor: cursor of the current position
+ * @entity: the current lru entity pointer
+ *
+ * Returns the next entity from drm lru manager
+ */
+struct drm_lru_entity *
+drm_lru_next(struct drm_lru_mgr *mgr, struct drm_lru_cursor *cursor,
+		struct drm_lru_entity *entity)
+{
+	lockdep_assert_held(mgr->lru_lock);
+
+	list_for_each_entry_continue(entity, &mgr->lru[cursor->priority], lru)
+		return entity;
+
+	for(++cursor->priority; cursor->priority < DRM_MAX_LRU_PRIORITY; ++cursor->priority)
+		list_for_each_entry(entity, &mgr->lru[cursor->priority], lru)
+			return entity;
+
+	return NULL;
+}
+
+/**
+ * drm_lru_move_to_tail
+ *
+ * @entity: the lru entity to move to lru tail
+ *
+ * Move a lru entity to lru tail
+ */
+void drm_lru_move_to_tail(struct drm_lru_entity * entity)
+{
+	struct list_head *lru;
+	struct drm_lru_mgr *mgr;
+
+	mgr = entity_to_mgr(entity);
+	lockdep_assert_held(mgr->lru_lock);
+	lru = &mgr->lru[entity->priority];
+	list_move_tail(&entity->lru, lru);
+}
+
+/**
+ * drm_lru_bulk_move_range_tail
+ *
+ * @range: bulk move range
+ * @entity: lru_entity to move
+ *
+ * Move a lru_entity to the tail of a bulk move range
+ */
+void drm_lru_bulk_move_range_tail(struct drm_lru_bulk_move_range *range,
+									struct drm_lru_entity *entity)
+{
+	if (entity == range->last)
+		return;
+
+	if (entity == range->first)
+		range->first = container_of(entity->lru.next, struct drm_lru_entity, lru);
+
+	if (range->last)
+		list_move(&entity->lru, &range->last->lru);
+
+	range->last = entity;
+}
+EXPORT_SYMBOL(drm_lru_bulk_move_range_tail);
+
+/**
+ * drm_lru_bulk_move_tail - bulk move range of entities to the LRU tail.
+ *
+ * @bulk: bulk_move structure
+ *
+ * Bulk move entities to the LRU tail, only valid to use when driver makes sure that
+ * resource order never changes.
+ */
+void drm_lru_bulk_move_tail(struct drm_lru_bulk_move *bulk)
+{
+
+	unsigned i, j;
+
+	for (i = 0; i < DRM_NUM_MEM_TYPES; ++i) {
+		for (j = 0; j < DRM_MAX_LRU_PRIORITY; ++j) {
+			struct drm_lru_bulk_move_range *range = &bulk->range[i][j];
+			struct drm_lru_mgr *mgr;
+
+			if (!range->first)
+				continue;
+
+			mgr = entity_to_mgr(range->first);
+			lockdep_assert_held(mgr->lru_lock);
+			list_bulk_move_tail(&mgr->lru[range->first->priority], &range->first->lru,
+					&range->last->lru);
+		}
+	}
+}
+EXPORT_SYMBOL(drm_lru_bulk_move_tail);
+
+/**
+ * drm_lru_add_bulk_move
+ *
+ * @entity: the lru entity to add to the bulk move range
+ * @bulk_move: the bulk move ranges to add the entity
+ *
+ * Add a lru entity to the tail of a bulk move range
+ */
+void drm_lru_add_bulk_move(struct drm_lru_entity *entity,
+						struct drm_lru_bulk_move *bulk_move)
+{
+	struct drm_lru_bulk_move_range *range;
+
+	range = &bulk_move->range[entity->mem_type][entity->priority];
+
+	if (!range->first) {
+		range->first = entity;
+		range->last = entity;
+		return;
+	}
+
+	drm_lru_bulk_move_range_tail(range, entity);
+}
+
+EXPORT_SYMBOL(drm_lru_add_bulk_move);
+/**
+ * drm_lru_del_bulk_move
+ *
+ * @entity: the lru entity to move from the bulk move range
+ * @bulk_move: the bulk move ranges to move the entity out of
+ *
+ * Move a lru entity out of bulk move range. This doesn't
+ * delete entity from lru manager's lru list.
+ */
+void drm_lru_del_bulk_move(struct drm_lru_entity *entity,
+					struct drm_lru_bulk_move *bulk_move)
+{
+	struct drm_lru_bulk_move_range *range;
+
+	range = &bulk_move->range[entity->mem_type][entity->priority];
+
+	if (unlikely(WARN_ON(!range->first || !range->last) ||
+			(range->first == entity && range->last == entity))) {
+		range->first = NULL;
+		range->last = NULL;
+	} else if (range->first == entity) {
+		range->first = container_of(entity->lru.next,
+				struct drm_lru_entity, lru);
+	} else if (range->last == entity) {
+		range->last = container_of(entity->lru.prev,
+				struct drm_lru_entity, lru);
+	} else {
+		list_move(&entity->lru, &range->last->lru);
+	}
+}
+EXPORT_SYMBOL(drm_lru_del_bulk_move);
include/drm/drm_device.h +7 -0
index d0b5f42786be..1bdcd34d3f6b 100644
@@ -8,6 +8,7 @@
 
  #include <drm/drm_legacy.h>
  #include <drm/drm_mode_config.h>
+#include <drm/drm_evictable_lru.h>
 
  struct drm_driver;
  struct drm_minor;
@@ -331,6 +332,12 @@ struct drm_device {
  	 */
  	spinlock_t lru_lock;
 
+	/**
+	 * @lru_mgr: Device global lru managers per memory type or memory
+	 * region. Each lru manager manages a lru list of this memory type.
+	 */
+	struct drm_lru_mgr lru_mgr[DRM_NUM_MEM_TYPES];
+
  	/* Everything below here is for legacy driver, never use! */
  	/* private: */
  #if IS_ENABLED(CONFIG_DRM_LEGACY)
include/drm/drm_evictable_lru.h new file +188 -0
new file mode 100644
index 000000000000..3fd6bd2475d9
@@ -0,0 +1,188 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#ifndef _DRM_EVICTABLE_LRU_H_
+#define _DRM_EVICTABLE_LRU_H_
+
+#include <linux/list.h>
+#include <linux/spinlock_types.h>
+#include <linux/spinlock.h>
+
+struct drm_device;
+
+#define DRM_MAX_LRU_PRIORITY 4
+#define DRM_NUM_MEM_TYPES 8
+
+/**
+ * struct drm_lru_entity
+ *
+ * @drm: drm device that this entity belongs to
+ * @mem_type: The memory type that this entity belongs to
+ * @size: resource size of this entity
+ * @priority: The priority of this entity
+ * @lru: least recent used list node, see &drm_lru_mgr.lru
+ *
+ * This structure represents an entity in drm_lru_mgr's
+ * list. This structure is supposed to be embedded in
+ * user's data structure.
+ */
+struct drm_lru_entity {
+	struct drm_device *drm;
+	uint32_t mem_type;
+	uint64_t size;
+	uint32_t priority;
+	struct list_head lru;
+};
+
+/**
+ * struct drm_lru_mgr
+ *
+ * @used: whether this lru manager is used or not
+ * @size: size of the resource
+ * @usage: how much resource has been used
+ * @lru_lock: a weak reference to the global lru_lock
+ * @lru: least recent used list, per priority
+ *
+ * This structure maintains all the buffer allocations
+ * in a least recent used list, so a victim for eviction
+ * can be easily found.
+ */
+struct drm_lru_mgr {
+	bool used;
+	uint64_t size;
+	uint64_t usage;
+	spinlock_t *lru_lock;
+	struct list_head lru[DRM_MAX_LRU_PRIORITY];
+};
+
+/**
+ * struct drm_lru_cursor
+ *
+ * @priority: the current priority
+ *
+ * Cursor to iterate over all entities in lru manager.
+ */
+struct drm_lru_cursor {
+	unsigned priority;
+};
+
+/**
+ * struct drm_lru_bulk_move_range
+ *
+ * @first: the first entity in the range
+ * @last: the last entity in the range
+ *
+ * Range of entities on a lru list.
+ */
+struct drm_lru_bulk_move_range
+{
+	struct drm_lru_entity *first;
+	struct drm_lru_entity *last;
+};
+
+/**
+ * struct drm_lru_bulk_move
+ *
+ * @range: An array of bulk move range, each corelates to the drm_lru_mgr's
+ * lru list of the same memory type and same priority.
+ *
+ * A collection of bulk_move range which can be used to move drm_lru_entity
+ * on the lru list in a bulk way. It should be initialized through
+ * drm_lru_bulk_move_init. Add/delete a drm_lru_entity to bulk move should call
+ * drm_lru_add_bulk_move/drm_lru_del_bulk_move.
+ */
+struct drm_lru_bulk_move {
+	struct drm_lru_bulk_move_range range[DRM_NUM_MEM_TYPES][DRM_MAX_LRU_PRIORITY];
+};
+
+
+
+/**
+ * drm_lru_add_entity
+ *
+ * @entity: the lru entity to add
+ * @mgr: the drm lru manager
+ * @priority: specify which priority list to add
+ *
+ * Add an entity to lru list
+ */
+static inline void drm_lru_add_entity(struct drm_lru_entity *entity,
+		struct drm_lru_mgr *mgr, unsigned priority)
+{
+	lockdep_assert_held(mgr->lru_lock);
+	list_add_tail(&entity->lru, &mgr->lru[priority]);
+	mgr->usage += entity->size;
+}
+
+/**
+ * drm_lru_remove_entity
+ *
+ * @entity: the lru entity to remove
+ * @mgr: the drm lru manager
+ *
+ * Remove an entity from lru list
+ */
+static inline void drm_lru_remove_entity(struct drm_lru_entity *entity,
+		struct drm_lru_mgr *mgr)
+{
+	lockdep_assert_held(mgr->lru_lock);
+	list_del_init(&entity->lru);
+	mgr->usage -= entity->size;
+}
+
+/**
+ * drm_lru_mgr_fini
+ *
+ * @mgr: the drm lru manager
+ *
+ * de-initialize a lru manager
+ */
+static inline void drm_lru_mgr_fini(struct drm_lru_mgr *mgr)
+{
+	mgr->used = false;
+}
+
+void drm_lru_entity_init(struct drm_lru_entity *entity, struct drm_device *drm,
+			uint32_t mem_type, uint64_t size, uint32_t priority);
+
+struct drm_lru_entity *
+drm_lru_first(struct drm_lru_mgr *mgr, struct drm_lru_cursor *cursor);
+
+struct drm_lru_entity *
+drm_lru_next(struct drm_lru_mgr *mgr, struct drm_lru_cursor *cursor,
+		struct drm_lru_entity *entity);
+
+void drm_lru_mgr_init(struct drm_lru_mgr *mgr, uint64_t size,
+		spinlock_t *lru_lock);
+
+void drm_lru_move_to_tail(struct drm_lru_entity * entity);
+
+void drm_lru_bulk_move_init(struct drm_lru_bulk_move *bulk_move);
+
+
+void drm_lru_bulk_move_tail(struct drm_lru_bulk_move *bulk);
+
+void drm_lru_bulk_move_range_tail(struct drm_lru_bulk_move_range *range,
+		struct drm_lru_entity *entity);
+
+void drm_lru_add_bulk_move(struct drm_lru_entity *entity,
+		struct drm_lru_bulk_move *bulk_move);
+
+void drm_lru_del_bulk_move(struct drm_lru_entity *entity,
+		struct drm_lru_bulk_move *bulk_move);
+/**
+ * drm_lru_for_each_entity
+ *
+ * @mgr: the drm lru manager
+ * @cursor: cursor for the current position
+ * @entity: the current drm_lru_entity
+ *
+ * Iterate over all entities in drm lru manager
+ */
+#define drm_lru_for_each_entity(mgr, cursor, entity)		\
+	for (entity = drm_lru_first(mgr, cursor); entity;	\
+	     entity = drm_lru_next(mgr, cursor, entity))
+
+#endif
author
#7 lore raw

RE: [RFC 03/11] drm: introduce drm evictable LRU

Quoted 11 lines

-----Original Message-----
From: Christian König <[email protected]>
Sent: Thursday, November 2, 2023 9:24 AM
To: Zeng, Oak <[email protected]>; [email protected]; intel- [email protected]
Cc: [email protected]; [email protected]; [email protected]; Welty, Brian <[email protected]>
Subject: Re: [RFC 03/11] drm: introduce drm evictable LRU

Quoted 5 lines

Am 02.11.23 um 05:32 schrieb Oak Zeng:

Quoted 2 lines

drm LRU manager is introuced for resource eviction purpose. It maintains a LRU list per resource type.

Shouldn't we first add the possible resource types in a separate patch?

Resource type in my description message is not a good name. for resource type we have:
System memory
Gpu stolen memory
Normal gpu vram

Some device such as Intel's pvc has sub-device concept (it is called tile in xe driver). For such device, we create multiple vram type ttm resource manager and multiple lru manager, one for each sub-device...So currently we only defined a DRM_NUM_MEM_TYPES in lru manager, but not defining each resource (memory) type.

Quoted 6 lines
Quoted 4 lines

It provides functions to add or remove
resource to or from the list. It also provides function to retrieve the first entity on the LRU list.

+ functions to iterate over them.

Yes basic iterate functions are implemented in this patch. Will add it to description message.

Quoted 14 lines
Quoted 5 lines

drm LRU manager also provides functions for bulk moving resources on the LRU lists.

Quoted 8 lines

drm LRU manager also does very basic memory accounting function, i.e., LRU manager keeps a size of this resource type and a usage member for how much of resource has been added to this LRU manager's LRU list. TTM resource manager memory accounting functoins such as struct ttm_resource_manager::size and struct ttm_resource_manger::usage are still kept. In the future, when SVM codes are in the picture, those memory accounting functions need some rework to consider the memory used by both TTM and SVM.

Quoted 5 lines

Please keep in mind that this structure needs to extremely small to be usable for SVM. E.g. struct page size small :)

At least HMM based implementations ideally wants to have one for each page or something like that.

Very good point. List node and eviction function pointer are necessary for drm_lru_entity. I will look whether we can remove other members. At least we can remove the drm_device pointer if we make drm_device base class of ttm_device as you suggested in previous patch comment.

And mem_type and priority can use bitfield, so a dword is enough.

Quoted 11 lines
Quoted 11 lines

For one device, a global drm LRU manager per resource type should be created/initialized at device initialization time. Drm LRU manager instances are embedded in struct drm_device.

It is pretty much moving some of the ttm resource manager functions to the drm layer. The reason of this code refactory is, we want to create a single LRU list for memory allocated from BO(buffer object) based driver and hmm/svm(shared virtual memory) based driver, thus BO driver and svm driver can evict memory from each other.

Quoted 7 lines
Quoted 4 lines

Previously the LRU list in TTM resource manager (lru field in struct ttm_reource_manager) is coupled with ttm_buffer_object concept, i.e., each ttm resource is backed by a ttm_buffer_object and the LRU list is essentially a list of ttm_buffer_object.

Actually it's the other way around. The resource provides the backing of the BO.

You are right. Will fix this description.

Quoted 9 lines

And when a BO moves around it can temporary be that multiple resource point to the same BO.

I also want to have a more advanced iterator at some point where we grab the BO lock for keeping a reference into the LRU list. Not sure how to do this if we don't have the BO here any more.

Need to think about that further,

Don't quite get the what you want to do with the advanced iterator. But with this work, the lru entity is a base class of ttm_resource or any other resource struct in hmm/svm. Lru is decoupled from bo concept - this is why this lru can be shared with svm code which is bo-less.

Oak

Quoted 518 lines

Christian.

Review

Quoted 51 lines

Due to this behavior, the
TTM resource manager can't be used by hmm/svm driver as we don't plan to have the BO concept for the hmm/svm implemenation. So we decouple the evictable LRU list from the BO concept in this series.

The design goal of drm lru manager is to make it as lean as possible. So each lru entity only has a list node member used to link this entity to the evictable LRU list, and the basic resource size/type/priority of this entity. It doesn't have any driver specify information. A lru entity also has a function pointer of evict function. This is used to implement ttm or svm specific eviction function. A lru entity is supposed to be embedded in a driver specific structure such as struct ttm_resource, see the usage in the next patch of this series.

The ttm resource manager, and some of the ttm_bo functions such as ttm_mem_evict_first will be rewriten using the new drm lru manager library, see the next patch in this series.

The future hmm/svm implemenation will call lru manager function to add hmm/svm allocations to the shared evictable lru list.

Lock design: previously ttm_resource LRU list is protected by a device global ttm_device::lru_lock (bdev->lru_lock in codes). This lock also protects ttm_buffer_object::pin_count, ttm_resource_manager::usage, ttm_resource::bo, ttm_device::pinned list etc. With this refactory, lru_lock is moved out of ttm_device and is added to struct drm_deive, so it can be shared b/t ttm code and svm code.

---
  drivers/gpu/drm/Makefile            |   1 +
  drivers/gpu/drm/drm_evictable_lru.c | 232 ++++++++++++++++++++++++++++
  include/drm/drm_device.h            |   7 +
  include/drm/drm_evictable_lru.h     | 188 ++++++++++++++++++++++
  4 files changed, 428 insertions(+)

create mode 100644 drivers/gpu/drm/drm_evictable_lru.c create mode 100644 include/drm/drm_evictable_lru.h

drivers/gpu/drm/Makefile +1 -0
index 1ad88efb1752..13953b0d271b 100644
@@ -46,6 +46,7 @@ drm-y := \
  	drm_vblank_work.o \
  	drm_vma_manager.o \
  	drm_gpuva_mgr.o \
+	drm_evictable_lru.o \
  	drm_writeback.o
  drm-$(CONFIG_DRM_LEGACY) += \
  	drm_agpsupport.o \
(quoted diff) +0 -0

b/drivers/gpu/drm/drm_evictable_lru.c

Quoted 68 lines

new file mode 100644
index 000000000000..2ba9105cca03

drivers/gpu/drm/drm_evictable_lru.c new file +63 -0
@@ -0,0 +1,232 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#include <linux/lockdep.h>
+#include <linux/container_of.h>
+#include <drm/drm_evictable_lru.h>
+#include <drm/drm_device.h>
+
+static inline struct drm_lru_mgr *entity_to_mgr(struct drm_lru_entity *entity)
+{
+	struct drm_lru_mgr *mgr;
+
+	mgr = &entity->drm->lru_mgr[entity->mem_type];
+	BUG_ON(!mgr->used);
+
+	return mgr;
+}
+
+void drm_lru_entity_init(struct drm_lru_entity *entity, struct drm_device *drm,
+			uint32_t mem_type, uint64_t size, uint32_t priority)
+{
+	entity->drm = drm;
+	entity->mem_type = mem_type;
+	entity->size = size;
+	entity->priority = priority;
+	INIT_LIST_HEAD(&entity->lru);
+}
+
+/**
+ * drm_lru_mgr_init
+ *
+ * @mgr: drm lru manager to init
+ * @size: size of the resource managed by this manager
+ * @lock: pointer of the global lru_lock
+ *
+ * Initialize a drm lru manager
+ */
+void drm_lru_mgr_init(struct drm_lru_mgr *mgr, uint64_t size, spinlock_t *lock)
+{
+	unsigned j;
+
+	mgr->used = true;
+	mgr->size = size;
+	mgr->usage = 0;
+	mgr->lru_lock = lock;
+
+	for(j = 0; j < DRM_MAX_LRU_PRIORITY; j++)
+		INIT_LIST_HEAD(&mgr->lru[j]);
+}
+
+void drm_lru_bulk_move_init(struct drm_lru_bulk_move *bulk_move)
+{
+	memset(bulk_move, 0, sizeof(*bulk_move));
+}
+
+/**
+ * drm_lru_first
+ *
+ * @mgr: drm lru manager to iterate over
+ * @cursor: cursor of the current position
+ *

drivers/gpu/drm/drm_evictable_lru.c line 73

Quoted 10 lines
drivers/gpu/drm/drm_evictable_lru.c new file +10 -0
+ * Returns the first entity in drm lru manager
+ */
+struct drm_lru_entity *
+drm_lru_first(struct drm_lru_mgr *mgr, struct drm_lru_cursor *cursor)
+{
+	struct drm_lru_entity *entity;
+
+	lockdep_assert_held(mgr->lru_lock);
+
+	for(cursor->priority = 0; cursor->priority < DRM_MAX_LRU_PRIORITY;

++cursor->priority)

Quoted 15 lines
drivers/gpu/drm/drm_evictable_lru.c new file +15 -0
+		list_for_each_entry(entity, &mgr->lru[cursor->priority], lru)
+			return entity;
+
+	return NULL;
+}
+
+/**
+ * drm_lru_next
+ *
+ * @mgr: drm lru manager to iterate over
+ * @cursor: cursor of the current position
+ * @entity: the current lru entity pointer
+ *
+ * Returns the next entity from drm lru manager
+ */

drivers/gpu/drm/drm_evictable_lru.c

Quoted 10 lines
drivers/gpu/drm/drm_evictable_lru.c new file +10 -0
+struct drm_lru_entity *
+drm_lru_next(struct drm_lru_mgr *mgr, struct drm_lru_cursor *cursor,
+		struct drm_lru_entity *entity)
+{
+	lockdep_assert_held(mgr->lru_lock);
+
+	list_for_each_entry_continue(entity, &mgr->lru[cursor->priority], lru)
+		return entity;
+
+	for(++cursor->priority; cursor->priority < DRM_MAX_LRU_PRIORITY;

++cursor->priority)

Quoted 24 lines
drivers/gpu/drm/drm_evictable_lru.c new file +24 -0
+		list_for_each_entry(entity, &mgr->lru[cursor->priority], lru)
+			return entity;
+
+	return NULL;
+}
+
+/**
+ * drm_lru_move_to_tail
+ *
+ * @entity: the lru entity to move to lru tail
+ *
+ * Move a lru entity to lru tail
+ */
+void drm_lru_move_to_tail(struct drm_lru_entity * entity)
+{
+	struct list_head *lru;
+	struct drm_lru_mgr *mgr;
+
+	mgr = entity_to_mgr(entity);
+	lockdep_assert_held(mgr->lru_lock);
+	lru = &mgr->lru[entity->priority];
+	list_move_tail(&entity->lru, lru);
+}
+

drivers/gpu/drm/drm_evictable_lru.c

Quoted 10 lines
drivers/gpu/drm/drm_evictable_lru.c new file +10 -0
+/**
+ * drm_lru_bulk_move_range_tail
+ *
+ * @range: bulk move range
+ * @entity: lru_entity to move
+ *
+ * Move a lru_entity to the tail of a bulk move range
+ */
+void drm_lru_bulk_move_range_tail(struct drm_lru_bulk_move_range *range,
+									struct

drm_lru_entity *entity)

drivers/gpu/drm/drm_evictable_lru.c

Quoted 6 lines
drivers/gpu/drm/drm_evictable_lru.c new file +6 -0
+{
+	if (entity == range->last)
+		return;
+
+	if (entity == range->first)
+		range->first = container_of(entity->lru.next, struct drm_lru_entity,

lru);

Quoted 25 lines
drivers/gpu/drm/drm_evictable_lru.c new file +24 -0
+
+	if (range->last)
+		list_move(&entity->lru, &range->last->lru);
+
+	range->last = entity;
+}
+EXPORT_SYMBOL(drm_lru_bulk_move_range_tail);
+
+/**
+ * drm_lru_bulk_move_tail - bulk move range of entities to the LRU tail.
+ *
+ * @bulk: bulk_move structure
+ *
+ * Bulk move entities to the LRU tail, only valid to use when driver makes sure that
+ * resource order never changes.
+ */
+void drm_lru_bulk_move_tail(struct drm_lru_bulk_move *bulk)
+{
+
+	unsigned i, j;
+
+	for (i = 0; i < DRM_NUM_MEM_TYPES; ++i) {
+		for (j = 0; j < DRM_MAX_LRU_PRIORITY; ++j) {
+			struct drm_lru_bulk_move_range *range = &bulk-

range[i][j];

drivers/gpu/drm/drm_evictable_lru.c

Quoted 8 lines
drivers/gpu/drm/drm_evictable_lru.c new file +8 -0
+			struct drm_lru_mgr *mgr;
+
+			if (!range->first)
+				continue;
+
+			mgr = entity_to_mgr(range->first);
+			lockdep_assert_held(mgr->lru_lock);
+			list_bulk_move_tail(&mgr->lru[range->first->priority],

&range->first->lru,

Quoted 6 lines
drivers/gpu/drm/drm_evictable_lru.c new file +6 -0
+					&range->last->lru);
+		}
+	}
+}
+EXPORT_SYMBOL(drm_lru_bulk_move_tail);
+

drivers/gpu/drm/drm_evictable_lru.c

Quoted 10 lines
drivers/gpu/drm/drm_evictable_lru.c new file +10 -0
+/**
+ * drm_lru_add_bulk_move
+ *
+ * @entity: the lru entity to add to the bulk move range
+ * @bulk_move: the bulk move ranges to add the entity
+ *
+ * Add a lru entity to the tail of a bulk move range
+ */
+void drm_lru_add_bulk_move(struct drm_lru_entity *entity,
+						struct drm_lru_bulk_move

*bulk_move)

Quoted 164 lines
drivers/gpu/drm/drm_evictable_lru.c new file +46 -0
+{
+	struct drm_lru_bulk_move_range *range;
+
+	range = &bulk_move->range[entity->mem_type][entity->priority];
+
+	if (!range->first) {
+		range->first = entity;
+		range->last = entity;
+		return;
+	}
+
+	drm_lru_bulk_move_range_tail(range, entity);
+}
+
+EXPORT_SYMBOL(drm_lru_add_bulk_move);
+/**
+ * drm_lru_del_bulk_move
+ *
+ * @entity: the lru entity to move from the bulk move range
+ * @bulk_move: the bulk move ranges to move the entity out of
+ *
+ * Move a lru entity out of bulk move range. This doesn't
+ * delete entity from lru manager's lru list.
+ */
+void drm_lru_del_bulk_move(struct drm_lru_entity *entity,
+					struct drm_lru_bulk_move *bulk_move)
+{
+	struct drm_lru_bulk_move_range *range;
+
+	range = &bulk_move->range[entity->mem_type][entity->priority];
+
+	if (unlikely(WARN_ON(!range->first || !range->last) ||
+			(range->first == entity && range->last == entity))) {
+		range->first = NULL;
+		range->last = NULL;
+	} else if (range->first == entity) {
+		range->first = container_of(entity->lru.next,
+				struct drm_lru_entity, lru);
+	} else if (range->last == entity) {
+		range->last = container_of(entity->lru.prev,
+				struct drm_lru_entity, lru);
+	} else {
+		list_move(&entity->lru, &range->last->lru);
+	}
+}
+EXPORT_SYMBOL(drm_lru_del_bulk_move);
include/drm/drm_device.h +7 -0
index d0b5f42786be..1bdcd34d3f6b 100644
@@ -8,6 +8,7 @@
 
  #include <drm/drm_legacy.h>
  #include <drm/drm_mode_config.h>
+#include <drm/drm_evictable_lru.h>
 
  struct drm_driver;
  struct drm_minor;
@@ -331,6 +332,12 @@ struct drm_device {
  	 */
  	spinlock_t lru_lock;
 
+	/**
+	 * @lru_mgr: Device global lru managers per memory type or memory
+	 * region. Each lru manager manages a lru list of this memory type.
+	 */
+	struct drm_lru_mgr lru_mgr[DRM_NUM_MEM_TYPES];
+
  	/* Everything below here is for legacy driver, never use! */
  	/* private: */
  #if IS_ENABLED(CONFIG_DRM_LEGACY)
include/drm/drm_evictable_lru.h new file +87 -0
new file mode 100644
index 000000000000..3fd6bd2475d9
@@ -0,0 +1,188 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#ifndef _DRM_EVICTABLE_LRU_H_
+#define _DRM_EVICTABLE_LRU_H_
+
+#include <linux/list.h>
+#include <linux/spinlock_types.h>
+#include <linux/spinlock.h>
+
+struct drm_device;
+
+#define DRM_MAX_LRU_PRIORITY 4
+#define DRM_NUM_MEM_TYPES 8
+
+/**
+ * struct drm_lru_entity
+ *
+ * @drm: drm device that this entity belongs to
+ * @mem_type: The memory type that this entity belongs to
+ * @size: resource size of this entity
+ * @priority: The priority of this entity
+ * @lru: least recent used list node, see &drm_lru_mgr.lru
+ *
+ * This structure represents an entity in drm_lru_mgr's
+ * list. This structure is supposed to be embedded in
+ * user's data structure.
+ */
+struct drm_lru_entity {
+	struct drm_device *drm;
+	uint32_t mem_type;
+	uint64_t size;
+	uint32_t priority;
+	struct list_head lru;
+};
+
+/**
+ * struct drm_lru_mgr
+ *
+ * @used: whether this lru manager is used or not
+ * @size: size of the resource
+ * @usage: how much resource has been used
+ * @lru_lock: a weak reference to the global lru_lock
+ * @lru: least recent used list, per priority
+ *
+ * This structure maintains all the buffer allocations
+ * in a least recent used list, so a victim for eviction
+ * can be easily found.
+ */
+struct drm_lru_mgr {
+	bool used;
+	uint64_t size;
+	uint64_t usage;
+	spinlock_t *lru_lock;
+	struct list_head lru[DRM_MAX_LRU_PRIORITY];
+};
+
+/**
+ * struct drm_lru_cursor
+ *
+ * @priority: the current priority
+ *
+ * Cursor to iterate over all entities in lru manager.
+ */
+struct drm_lru_cursor {
+	unsigned priority;
+};
+
+/**
+ * struct drm_lru_bulk_move_range
+ *
+ * @first: the first entity in the range
+ * @last: the last entity in the range
+ *
+ * Range of entities on a lru list.
+ */
+struct drm_lru_bulk_move_range
+{
+	struct drm_lru_entity *first;
+	struct drm_lru_entity *last;
+};
+
+/**
+ * struct drm_lru_bulk_move
+ *

include/drm/drm_evictable_lru.h line 97

Quoted 10 lines
include/drm/drm_evictable_lru.h new file +10 -0
+ * @range: An array of bulk move range, each corelates to the drm_lru_mgr's
+ * lru list of the same memory type and same priority.
+ *
+ * A collection of bulk_move range which can be used to move drm_lru_entity
+ * on the lru list in a bulk way. It should be initialized through
+ * drm_lru_bulk_move_init. Add/delete a drm_lru_entity to bulk move should call
+ * drm_lru_add_bulk_move/drm_lru_del_bulk_move.
+ */
+struct drm_lru_bulk_move {
+	struct drm_lru_bulk_move_range

range[DRM_NUM_MEM_TYPES][DRM_MAX_LRU_PRIORITY];

Quoted 91 lines
include/drm/drm_evictable_lru.h new file +91 -0
+};
+
+
+
+/**
+ * drm_lru_add_entity
+ *
+ * @entity: the lru entity to add
+ * @mgr: the drm lru manager
+ * @priority: specify which priority list to add
+ *
+ * Add an entity to lru list
+ */
+static inline void drm_lru_add_entity(struct drm_lru_entity *entity,
+		struct drm_lru_mgr *mgr, unsigned priority)
+{
+	lockdep_assert_held(mgr->lru_lock);
+	list_add_tail(&entity->lru, &mgr->lru[priority]);
+	mgr->usage += entity->size;
+}
+
+/**
+ * drm_lru_remove_entity
+ *
+ * @entity: the lru entity to remove
+ * @mgr: the drm lru manager
+ *
+ * Remove an entity from lru list
+ */
+static inline void drm_lru_remove_entity(struct drm_lru_entity *entity,
+		struct drm_lru_mgr *mgr)
+{
+	lockdep_assert_held(mgr->lru_lock);
+	list_del_init(&entity->lru);
+	mgr->usage -= entity->size;
+}
+
+/**
+ * drm_lru_mgr_fini
+ *
+ * @mgr: the drm lru manager
+ *
+ * de-initialize a lru manager
+ */
+static inline void drm_lru_mgr_fini(struct drm_lru_mgr *mgr)
+{
+	mgr->used = false;
+}
+
+void drm_lru_entity_init(struct drm_lru_entity *entity, struct drm_device *drm,
+			uint32_t mem_type, uint64_t size, uint32_t priority);
+
+struct drm_lru_entity *
+drm_lru_first(struct drm_lru_mgr *mgr, struct drm_lru_cursor *cursor);
+
+struct drm_lru_entity *
+drm_lru_next(struct drm_lru_mgr *mgr, struct drm_lru_cursor *cursor,
+		struct drm_lru_entity *entity);
+
+void drm_lru_mgr_init(struct drm_lru_mgr *mgr, uint64_t size,
+		spinlock_t *lru_lock);
+
+void drm_lru_move_to_tail(struct drm_lru_entity * entity);
+
+void drm_lru_bulk_move_init(struct drm_lru_bulk_move *bulk_move);
+
+
+void drm_lru_bulk_move_tail(struct drm_lru_bulk_move *bulk);
+
+void drm_lru_bulk_move_range_tail(struct drm_lru_bulk_move_range *range,
+		struct drm_lru_entity *entity);
+
+void drm_lru_add_bulk_move(struct drm_lru_entity *entity,
+		struct drm_lru_bulk_move *bulk_move);
+
+void drm_lru_del_bulk_move(struct drm_lru_entity *entity,
+		struct drm_lru_bulk_move *bulk_move);
+/**
+ * drm_lru_for_each_entity
+ *
+ * @mgr: the drm lru manager
+ * @cursor: cursor for the current position
+ * @entity: the current drm_lru_entity
+ *
+ * Iterate over all entities in drm lru manager
+ */
+#define drm_lru_for_each_entity(mgr, cursor, entity)		\
+	for (entity = drm_lru_first(mgr, cursor); entity;	\
+	     entity = drm_lru_next(mgr, cursor, entity))
+
+#endif
maintainer
#8 lore raw

Re: [RFC 03/11] drm: introduce drm evictable LRU

Am 03.11.23 um 05:04 schrieb Zeng, Oak:[SNIP]

Quoted 6 lines

I also want to have a more advanced iterator at some point where we grab the BO lock for keeping a reference into the LRU list. Not sure how to do this if we don't have the BO here any more.

Need to think about that further,
Don't quite get the what you want to do with the advanced iterator. But with this work, the lru entity is a base class of ttm_resource or any other resource struct in hmm/svm. Lru is decoupled from bo concept - this is why this lru can be shared with svm code which is bo-less.

This is just a crazy idea I had because TTM tends to perform bad on certain tasks.

When we start to evict something we use a callback which indicates if an eviction is valuable or not. So it can happen that we have to skip quite a bunch of BOs on the LRU until we found one which is worth evicting.

Not it can be that the first eviction doesn't make enough room to fulfill the allocation requirement, in this case we currently start over at the beginning searching for some BO to evict.

I want to avoid this by being able to have cursors into the LRU, e.g. the next BO which can't move until we have evicted the current one.

BTW: How do you handle eviction here? I mean we can't call the evict callback with the spinlock held easily?

Christian.

Quoted 3 lines

Oak

author
#9 lore raw

RE: [RFC 03/11] drm: introduce drm evictable LRU

From: Christian König <[email protected]>
Sent: Friday, November 3, 2023 5:36 AM
To: Zeng, Oak <[email protected]>; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Welty, Brian <[email protected]>
Subject: Re: [RFC 03/11] drm: introduce drm evictable LRU

Am 03.11.23 um 05:04 schrieb Zeng, Oak:[SNIP]

I also want to have a more advanced iterator at some point where we grab

the BO lock for keeping a reference into the LRU list. Not sure how to

do this if we don't have the BO here any more.

Need to think about that further,

Don't quite get the what you want to do with the advanced iterator. But with this work, the lru entity is a base class of ttm_resource or any other resource struct in hmm/svm. Lru is decoupled from bo concept - this is why this lru can be shared with svm code which is bo-less.

This is just a crazy idea I had because TTM tends to perform bad on certain tasks.

When we start to evict something we use a callback which indicates if an eviction is valuable or not. So it can happen that we have to skip quite a bunch of BOs on the LRU until we found one which is worth evicting.

Not it can be that the first eviction doesn't make enough room to fulfill the allocation requirement, in this case we currently start over at the beginning searching for some BO to evict.

I want to avoid this by being able to have cursors into the LRU, e.g. the next BO which can't move until we have evicted the current one.

Got you now. I didn’t know this problem so I didn’t try to fix this efficiency problem in this series. Theoretically I think we can fix this issue this way: change ttm_mem_evict_first to ttm_mem_evict_first_n and add a parameter to this function to specify how much room we want to yield; then we evict the first n objects to make enough room before return, or fail if we can’t make enough room. This scheme would need the caller of ttm_mem_evict_first to tell how much room he need – I think reasonable.

BTW: How do you handle eviction here? I mean we can't call the evict callback with the spinlock held easily?

I was actually struggling when I refactored ttm_mem_evict_first function. I moved this function to lru manager and abstracted 3 callback functions (evict_allowable/valuable, evict_entity, evict_busy_entity) – those need to relook when hmm/svm codes come in picture. I tried not to change any logic of this function – I know people worked on this function in the past 15 years so better to be very careful.

So in my current implementation, spinlock is held calling the evict_entity callback. Spinlock is unlocked before calling ttm_bo_evict in the evict_entity callback and re-held if we need to move entity in lru list. See details in patch 4 and patch 10. So it keeps exactly the original call sequence but does look awkward.

But I think you are right. We can release the spinlock in the drm_lru_evict_first function before calling evict callback.

Oak

Christian.

Oak

author
#10 lore raw

RFC 4/11 drm: Add evict function pointer to drm lru entity

In reply to Oak Zeng

Drm lru manager provides generic functions to manage lru list, and function to evict a lru entity. But how to evict an entity is implemented in an entity's sub-class. This patch introduces a few function pointers to drm lru entity for this purpose. Those functions are abstracted from the current ttm resource eviction process. They need to be tunned in the future when svm code comes into the picture.

Also implemented a drm_lru_evict_first function to evict the first lru entity from lru manager. Both ttm and svm codes are supposed to call this function to evict the first resource from lru list. This way ttm and svm codes can mutually evict resources from each other.

---
 drivers/gpu/drm/drm_evictable_lru.c | 36 +++++++++++++-
 include/drm/drm_evictable_lru.h     | 74 ++++++++++++++++++++++++++++-
 2 files changed, 108 insertions(+), 2 deletions(-)
drivers/gpu/drm/drm_evictable_lru.c +35 -1
index 2ba9105cca03..7b62cae2dfea 100644
@@ -19,13 +19,15 @@ static inline struct drm_lru_mgr *entity_to_mgr(struct drm_lru_entity *entity)
 }
 
 void drm_lru_entity_init(struct drm_lru_entity *entity, struct drm_device *drm,
-			uint32_t mem_type, uint64_t size, uint32_t priority)
+			uint32_t mem_type, uint64_t size, uint32_t priority,
+			struct drm_lru_evict_func *evict_func)
 {
 	entity->drm = drm;
 	entity->mem_type = mem_type;
 	entity->size = size;
 	entity->priority = priority;
 	INIT_LIST_HEAD(&entity->lru);
+	entity->evict_func = evict_func;
 }
 
 /**
@@ -230,3 +232,35 @@ void drm_lru_del_bulk_move(struct drm_lru_entity *entity,
 	}
 }
 EXPORT_SYMBOL(drm_lru_del_bulk_move);
+
+int drm_lru_evict_first(struct drm_lru_mgr *mgr,
+			const struct drm_lru_evict_ctx *evict_ctx)
+{
+	struct drm_lru_entity *entity, *busy_entity = NULL;
+	struct drm_lru_cursor cursor;
+	bool locked = false, busy = false, found = false;
+
+	spin_lock(mgr->lru_lock);
+
+	/* First find a victim to evict*/
+	drm_lru_for_each_entity(mgr, &cursor, entity) {
+		if (!entity->evict_func->evict_allowable(entity,
+			evict_ctx, &busy, &locked)) {
+			if (!busy_entity && busy)
+				busy_entity = entity;
+			continue;
+		}
+		found = true;
+		break;
+	}
+
+	/* We didn't find a victim, but we found a busy entity, i.e.,
+	 * other clients hold a reservation lock of this entity. Try
+	 * to wait and evict this busy entity.
+	 */
+	if (!found && busy_entity)
+		return busy_entity->evict_func->evict_busy_entity(busy_entity, evict_ctx);
+
+	/* If here, we found a victim to evict*/
+	return entity->evict_func->evict_entity(entity, evict_ctx, locked);
+}
include/drm/drm_evictable_lru.h +73 -1

111 lines not on this page · raw message

Signature · 1 line
2.26.3
author
#11 lore raw

RFC 5/11 drm: Replace ttm macros with drm macros

In reply to Oak Zeng

TTM_MAX_BO_PRIORITY and TTM_NUM_MEM_TYPES are move from ttm to drm, so:
s/TTM_MAX_BO_PRIORITY/DRM_MAX_LRU_PRIORITY s/TTM_NUM_MEM_TYPES/DRM_NUM_MEM_TYPES

---
 drivers/gpu/drm/i915/gem/i915_gem_ttm.c      |  2 +-
 drivers/gpu/drm/i915/intel_region_ttm.c      |  2 +-
 drivers/gpu/drm/i915/selftests/mock_region.c |  2 +-
 drivers/gpu/drm/ttm/ttm_device.c             |  8 ++++----
 drivers/gpu/drm/ttm/ttm_resource.c           | 12 ++++++------
 drivers/gpu/drm/vmwgfx/vmwgfx_bo.c           |  2 +-
 drivers/gpu/drm/vmwgfx/vmwgfx_bo.h           |  2 +-
 drivers/gpu/drm/xe/xe_bo.h                   |  2 +-
 include/drm/ttm/ttm_device.h                 |  6 +++---
 include/drm/ttm/ttm_range_manager.h          |  4 ++--
 include/drm/ttm/ttm_resource.h               | 10 ++++------
 11 files changed, 25 insertions(+), 27 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +1 -1
index c46f54f83f54..228dbea60949 100644
@@ -987,7 +987,7 @@ void i915_ttm_adjust_lru(struct drm_i915_gem_object *obj)
 	spin_lock(bo->bdev->lru_lock);
 	if (shrinkable) {
 		/* Try to keep shmem_tt from being considered for shrinking. */
-		bo->priority = TTM_MAX_BO_PRIORITY - 1;
+		bo->priority = DRM_MAX_LRU_PRIORITY - 1;
 	} else if (obj->mm.madv != I915_MADV_WILLNEED) {
 		bo->priority = I915_TTM_PRIO_PURGE;
 	} else if (!i915_gem_object_has_pages(obj)) {
drivers/gpu/drm/i915/intel_region_ttm.c +1 -1
index b845782c9859..f75520c2ba59 100644
@@ -63,7 +63,7 @@ int intel_region_to_ttm_type(const struct intel_memory_region *mem)
 		return TTM_PL_SYSTEM;
 
 	type = mem->instance + TTM_PL_PRIV;
-	GEM_BUG_ON(type >= TTM_NUM_MEM_TYPES);
+	GEM_BUG_ON(type >= DRM_NUM_MEM_TYPES);
 
 	return type;
 }
drivers/gpu/drm/i915/selftests/mock_region.c +1 -1
index 6324eb32d4dd..6ea0e6bec812 100644
@@ -111,7 +111,7 @@ mock_region_create(struct drm_i915_private *i915,
 		   resource_size_t io_size)
 {
 	int instance = ida_alloc_max(&i915->selftest.mock_region_instances,
-				     TTM_NUM_MEM_TYPES - TTM_PL_PRIV - 1,
+				     DRM_NUM_MEM_TYPES - TTM_PL_PRIV - 1,
 				     GFP_KERNEL);
 
 	if (instance < 0)
drivers/gpu/drm/ttm/ttm_device.c +4 -4
index d18eca86ebd6..e8c8006ba748 100644
@@ -148,7 +148,7 @@ int ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx,
 	int ret;
 
 	spin_lock(bdev->lru_lock);
-	for (i = TTM_PL_SYSTEM; i < TTM_NUM_MEM_TYPES; ++i) {
+	for (i = TTM_PL_SYSTEM; i < DRM_NUM_MEM_TYPES; ++i) {
 		man = ttm_manager_type(bdev, i);
 		if (!man || !man->use_tt)
 			continue;
@@ -245,7 +245,7 @@ void ttm_device_fini(struct ttm_device *bdev)
 	destroy_workqueue(bdev->wq);
 
 	spin_lock(bdev->lru_lock);
-	for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i)
+	for (i = 0; i < DRM_MAX_LRU_PRIORITY; ++i)
 		if (list_empty(&man->lru[0]))
 			pr_debug("Swap list %d was clean\n", i);
 	spin_unlock(bdev->lru_lock);
@@ -287,12 +287,12 @@ void ttm_device_clear_dma_mappings(struct ttm_device *bdev)
 
 	ttm_device_clear_lru_dma_mappings(bdev, &bdev->pinned);
 
-	for (i = TTM_PL_SYSTEM; i < TTM_NUM_MEM_TYPES; ++i) {
+	for (i = TTM_PL_SYSTEM; i < DRM_NUM_MEM_TYPES; ++i) {
 		man = ttm_manager_type(bdev, i);
 		if (!man || !man->use_tt)
 			continue;
 
-		for (j = 0; j < TTM_MAX_BO_PRIORITY; ++j)
+		for (j = 0; j < DRM_MAX_LRU_PRIORITY; ++j)
 			ttm_device_clear_lru_dma_mappings(bdev, &man->lru[j]);
 	}
 }
drivers/gpu/drm/ttm/ttm_resource.c +6 -6
index 6ada77f51fba..05eef866065e 100644
@@ -54,8 +54,8 @@ void ttm_lru_bulk_move_tail(struct ttm_lru_bulk_move *bulk)
 {
 	unsigned i, j;
 
-	for (i = 0; i < TTM_NUM_MEM_TYPES; ++i) {
-		for (j = 0; j < TTM_MAX_BO_PRIORITY; ++j) {
+	for (i = 0; i < DRM_NUM_MEM_TYPES; ++i) {
+		for (j = 0; j < DRM_MAX_LRU_PRIORITY; ++j) {
 			struct ttm_lru_bulk_move_pos *pos = &bulk->pos[i][j];
 			struct ttm_resource_manager *man;
 
@@ -393,7 +393,7 @@ void ttm_resource_manager_init(struct ttm_resource_manager *man,
 	man->size = size;
 	man->usage = 0;
 
-	for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i)
+	for (i = 0; i < DRM_MAX_LRU_PRIORITY; ++i)
 		INIT_LIST_HEAD(&man->lru[i]);
 	man->move = NULL;
 }
@@ -425,7 +425,7 @@ int ttm_resource_manager_evict_all(struct ttm_device *bdev,
 	 */
 
 	spin_lock(bdev->lru_lock);
-	for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) {
+	for (i = 0; i < DRM_MAX_LRU_PRIORITY; ++i) {
 		while (!list_empty(&man->lru[i])) {
 			spin_unlock(bdev->lru_lock);
 			ret = ttm_mem_evict_first(bdev, man, NULL, &ctx,
@@ -504,7 +504,7 @@ ttm_resource_manager_first(struct ttm_resource_manager *man,
 
 	lockdep_assert_held(man->bdev->lru_lock);
 
-	for (cursor->priority = 0; cursor->priority < TTM_MAX_BO_PRIORITY;
+	for (cursor->priority = 0; cursor->priority < DRM_MAX_LRU_PRIORITY;
 	     ++cursor->priority)
 		list_for_each_entry(res, &man->lru[cursor->priority], lru)
 			return res;
@@ -531,7 +531,7 @@ ttm_resource_manager_next(struct ttm_resource_manager *man,
 	list_for_each_entry_continue(res, &man->lru[cursor->priority], lru)
 		return res;
 
-	for (++cursor->priority; cursor->priority < TTM_MAX_BO_PRIORITY;
+	for (++cursor->priority; cursor->priority < DRM_MAX_LRU_PRIORITY;
 	     ++cursor->priority)
 		list_for_each_entry(res, &man->lru[cursor->priority], lru)
 			return res;
drivers/gpu/drm/vmwgfx/vmwgfx_bo.c +1 -1
index c43853597776..9efde9ba9fe2 100644
@@ -383,7 +383,7 @@ static int vmw_bo_init(struct vmw_private *dev_priv,
 
 	memset(vmw_bo, 0, sizeof(*vmw_bo));
 
-	BUILD_BUG_ON(TTM_MAX_BO_PRIORITY <= 3);
+	BUILD_BUG_ON(DRM_MAX_LRU_PRIORITY <= 3);
 	vmw_bo->tbo.priority = 3;
 	vmw_bo->res_tree = RB_ROOT;
 
drivers/gpu/drm/vmwgfx/vmwgfx_bo.h +1 -1
index 1d433fceed3d..c43a82c8afb9 100644
@@ -82,7 +82,7 @@ struct vmw_bo {
 	struct ttm_bo_kmap_obj map;
 
 	struct rb_root res_tree;
-	u32 res_prios[TTM_MAX_BO_PRIORITY];
+	u32 res_prios[DRM_MAX_LRU_PRIORITY];
 
 	atomic_t cpu_writers;
 	/* Not ref-counted.  Protected by binding_mutex */
drivers/gpu/drm/xe/xe_bo.h +1 -1
index 9097bcc13209..9918b2d630e1 100644
@@ -72,7 +72,7 @@
 #define XE_PL_TT		TTM_PL_TT
 #define XE_PL_VRAM0		TTM_PL_VRAM
 #define XE_PL_VRAM1		(XE_PL_VRAM0 + 1)
-#define XE_PL_STOLEN		(TTM_NUM_MEM_TYPES - 1)
+#define XE_PL_STOLEN		(DRM_NUM_MEM_TYPES - 1)
 
 #define XE_BO_PROPS_INVALID	(-1)
 
include/drm/ttm/ttm_device.h +3 -3
index 4d29e96bd892..a75e820dc671 100644
@@ -235,7 +235,7 @@ struct ttm_device {
 	/**
 	 * @man_drv: An array of resource_managers, one per resource type.
 	 */
-	struct ttm_resource_manager *man_drv[TTM_NUM_MEM_TYPES];
+	struct ttm_resource_manager *man_drv[DRM_NUM_MEM_TYPES];
 
 	/**
 	 * @vma_manager: Address space manager for finding BOs to mmap.
@@ -277,14 +277,14 @@ static inline struct ttm_resource_manager *
 ttm_manager_type(struct ttm_device *bdev, int mem_type)
 {
 	BUILD_BUG_ON(__builtin_constant_p(mem_type)
-		     && mem_type >= TTM_NUM_MEM_TYPES);
+		     && mem_type >= DRM_NUM_MEM_TYPES);
 	return bdev->man_drv[mem_type];
 }
 
 static inline void ttm_set_driver_manager(struct ttm_device *bdev, int type,
 					  struct ttm_resource_manager *manager)
 {
-	BUILD_BUG_ON(__builtin_constant_p(type) && type >= TTM_NUM_MEM_TYPES);
+	BUILD_BUG_ON(__builtin_constant_p(type) && type >= DRM_NUM_MEM_TYPES);
 	bdev->man_drv[type] = manager;
 }
 
include/drm/ttm/ttm_range_manager.h +2 -2
index 7963b957e9ef..becdb88c4d84 100644
@@ -43,14 +43,14 @@ static __always_inline int ttm_range_man_init(struct ttm_device *bdev,
 		       unsigned int type, bool use_tt,
 		       unsigned long p_size)
 {
-	BUILD_BUG_ON(__builtin_constant_p(type) && type >= TTM_NUM_MEM_TYPES);
+	BUILD_BUG_ON(__builtin_constant_p(type) && type >= DRM_NUM_MEM_TYPES);
 	return ttm_range_man_init_nocheck(bdev, type, use_tt, p_size);
 }
 
 static __always_inline int ttm_range_man_fini(struct ttm_device *bdev,
 		       unsigned int type)
 {
-	BUILD_BUG_ON(__builtin_constant_p(type) && type >= TTM_NUM_MEM_TYPES);
+	BUILD_BUG_ON(__builtin_constant_p(type) && type >= DRM_NUM_MEM_TYPES);
 	return ttm_range_man_fini_nocheck(bdev, type);
 }
 #endif
include/drm/ttm/ttm_resource.h +4 -6
index 78a226eba953..92241c2374fa 100644
@@ -34,9 +34,7 @@
 #include <drm/drm_print.h>
 #include <drm/ttm/ttm_caching.h>
 #include <drm/ttm/ttm_kmap_iter.h>
-
-#define TTM_MAX_BO_PRIORITY	4U
-#define TTM_NUM_MEM_TYPES 8
+#include <drm/drm_evictable_lru.h>
 
 struct ttm_device;
 struct ttm_resource_manager;
@@ -167,7 +165,7 @@ struct ttm_resource_manager {
 	/*
 	 * Protected by the bdev->lru_lock.
 	 */
-	struct list_head lru[TTM_MAX_BO_PRIORITY];
+	struct list_head lru[DRM_MAX_LRU_PRIORITY];
 
 	/**
 	 * @usage: How much of the resources are used, protected by the
@@ -253,7 +251,7 @@ struct ttm_lru_bulk_move_pos {
  * ttm_lru_bulk_move_init() and ttm_bo_set_bulk_move().
  */
 struct ttm_lru_bulk_move {
-	struct ttm_lru_bulk_move_pos pos[TTM_NUM_MEM_TYPES][TTM_MAX_BO_PRIORITY];
+	struct ttm_lru_bulk_move_pos pos[DRM_NUM_MEM_TYPES][DRM_MAX_LRU_PRIORITY];
 };
 
 /**
@@ -309,7 +307,7 @@ ttm_resource_manager_set_used(struct ttm_resource_manager *man, bool used)
 {
 	int i;
 
-	for (i = 0; i < TTM_MAX_BO_PRIORITY; i++)
+	for (i = 0; i < DRM_MAX_LRU_PRIORITY; i++)
 		WARN_ON(!list_empty(&man->lru[i]));
 	man->use_type = used;
 }
Signature · 1 line
2.26.3
author
#12 lore raw

RFC 6/11 drm/ttm: Set lru manager to ttm resource manager

In reply to Oak Zeng

Add a weak reference of lru manager to ttm resource manager, and add a function to set lru manager for ttm resource manager.

---
 include/drm/ttm/ttm_resource.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
include/drm/ttm/ttm_resource.h +19 -0
index 92241c2374fa..e4fc1ada5236 100644
@@ -46,6 +46,7 @@ struct iosys_map;
 struct io_mapping;
 struct sg_table;
 struct scatterlist;
+struct drm_lru_mgr;
 
 struct ttm_resource_manager_func {
 	/**
@@ -172,6 +173,12 @@ struct ttm_resource_manager {
 	 * bdev->lru_lock.
 	 */
 	uint64_t usage;
+
+	/**
+	 * @lru_mgr: weak reference of the lru manager that manages lru
+	 * list for this ttm resource manager.
+	 */
+	struct drm_lru_mgr *lru_mgr;
 };
 
 /**
@@ -326,6 +333,18 @@ static inline bool ttm_resource_manager_used(struct ttm_resource_manager *man)
 	return man->use_type;
 }
 
+/**
+ * ttm_resource_manager_set_lru_mgr
+ *
+ * @man: ttm resource manager
+ * @mgr: pointing to lru manager
+ */
+static inline void
+ttm_resource_manager_set_lru_mgr(struct ttm_resource_manager *man, struct drm_lru_mgr *mgr)
+{
+	man->lru_mgr = mgr;
+}
+
 /**
  * ttm_resource_manager_cleanup
  *
Signature · 1 line
2.26.3
author
#13 lore raw

RFC 7/11 drm/ttm: re-parameterize a few ttm functions

In reply to Oak Zeng

Add a struct drm_device *drm parameter to function ttm_range_man_init, ttm_range_man_fini, ttm_sys_man_init, and ttm_sys_man_free. This drm parameter will be used in the coming patches to retrieve and initialize drm lru manager.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c |  8 ++++----
 drivers/gpu/drm/drm_gem_vram_helper.c   |  8 ++++----
 drivers/gpu/drm/loongson/lsdc_ttm.c     |  8 ++++----
 drivers/gpu/drm/nouveau/nouveau_ttm.c   |  8 ++++----
 drivers/gpu/drm/qxl/qxl_ttm.c           |  6 +++---
 drivers/gpu/drm/radeon/radeon_ttm.c     |  8 ++++----
 drivers/gpu/drm/ttm/ttm_device.c        |  2 +-
 drivers/gpu/drm/ttm/ttm_module.h        |  3 ++-
 drivers/gpu/drm/ttm/ttm_range_manager.c |  6 ++++--
 drivers/gpu/drm/ttm/ttm_sys_manager.c   |  6 +++++-
 drivers/gpu/drm/vmwgfx/vmwgfx_drv.c     |  4 ++--
 include/drm/ttm/ttm_range_manager.h     | 13 +++++++------
 12 files changed, 44 insertions(+), 36 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +4 -4
index 5cdbc901cbe2..cc0736f82a80 100644
@@ -75,7 +75,7 @@ static int amdgpu_ttm_init_on_chip(struct amdgpu_device *adev,
 				    unsigned int type,
 				    uint64_t size_in_page)
 {
-	return ttm_range_man_init(&adev->mman.bdev, type,
+	return ttm_range_man_init(adev_to_drm(adev), &adev->mman.bdev, type,
 				  false, size_in_page);
 }
 
@@ -2026,9 +2026,9 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev)
 	amdgpu_vram_mgr_fini(adev);
 	amdgpu_gtt_mgr_fini(adev);
 	amdgpu_preempt_mgr_fini(adev);
-	ttm_range_man_fini(&adev->mman.bdev, AMDGPU_PL_GDS);
-	ttm_range_man_fini(&adev->mman.bdev, AMDGPU_PL_GWS);
-	ttm_range_man_fini(&adev->mman.bdev, AMDGPU_PL_OA);
+	ttm_range_man_fini(adev_to_drm(adev), &adev->mman.bdev, AMDGPU_PL_GDS);
+	ttm_range_man_fini(adev_to_drm(adev), &adev->mman.bdev, AMDGPU_PL_GWS);
+	ttm_range_man_fini(adev_to_drm(adev), &adev->mman.bdev, AMDGPU_PL_OA);
 	ttm_device_fini(&adev->mman.bdev);
 	adev->mman.initialized = false;
 	DRM_INFO("amdgpu: ttm finalized\n");
drivers/gpu/drm/drm_gem_vram_helper.c +4 -4
index 56749e40459f..5b18db72cc96 100644
@@ -1009,7 +1009,7 @@ static int drm_vram_mm_init(struct drm_vram_mm *vmm, struct drm_device *dev,
 	if (ret)
 		return ret;
 
-	ret = ttm_range_man_init(&vmm->bdev, TTM_PL_VRAM,
+	ret = ttm_range_man_init(dev, &vmm->bdev, TTM_PL_VRAM,
 				 false, vram_size >> PAGE_SHIFT);
 	if (ret)
 		return ret;
@@ -1017,9 +1017,9 @@ static int drm_vram_mm_init(struct drm_vram_mm *vmm, struct drm_device *dev,
 	return 0;
 }
 
-static void drm_vram_mm_cleanup(struct drm_vram_mm *vmm)
+static void drm_vram_mm_cleanup(struct drm_device *drm, struct drm_vram_mm *vmm)
 {
-	ttm_range_man_fini(&vmm->bdev, TTM_PL_VRAM);
+	ttm_range_man_fini(drm, &vmm->bdev, TTM_PL_VRAM);
 	ttm_device_fini(&vmm->bdev);
 }
 
@@ -1056,7 +1056,7 @@ static void drm_vram_helper_release_mm(struct drm_device *dev)
 	if (!dev->vram_mm)
 		return;
 
-	drm_vram_mm_cleanup(dev->vram_mm);
+	drm_vram_mm_cleanup(dev, dev->vram_mm);
 	kfree(dev->vram_mm);
 	dev->vram_mm = NULL;
 }
drivers/gpu/drm/loongson/lsdc_ttm.c +4 -4
index bd68cb9366b5..f7f226314a09 100644
@@ -533,8 +533,8 @@ static void lsdc_ttm_fini(struct drm_device *ddev, void *data)
 {
 	struct lsdc_device *ldev = (struct lsdc_device *)data;
 
-	ttm_range_man_fini(&ldev->bdev, TTM_PL_VRAM);
-	ttm_range_man_fini(&ldev->bdev, TTM_PL_TT);
+	ttm_range_man_fini(ddev, &ldev->bdev, TTM_PL_VRAM);
+	ttm_range_man_fini(ddev, &ldev->bdev, TTM_PL_TT);
 
 	ttm_device_fini(&ldev->bdev);
 
@@ -556,7 +556,7 @@ int lsdc_ttm_init(struct lsdc_device *ldev)
 
 	num_vram_pages = ldev->vram_size >> PAGE_SHIFT;
 
-	ret = ttm_range_man_init(&ldev->bdev, TTM_PL_VRAM, false, num_vram_pages);
+	ret = ttm_range_man_init(&ldev->base, &ldev->bdev, TTM_PL_VRAM, false, num_vram_pages);
 	if (unlikely(ret))
 		return ret;
 
@@ -567,7 +567,7 @@ int lsdc_ttm_init(struct lsdc_device *ldev)
 
 	num_gtt_pages = ldev->gtt_size >> PAGE_SHIFT;
 
-	ret = ttm_range_man_init(&ldev->bdev, TTM_PL_TT, true, num_gtt_pages);
+	ret = ttm_range_man_init(&ldev->base, &ldev->bdev, TTM_PL_TT, true, num_gtt_pages);
 	if (unlikely(ret))
 		return ret;
 
drivers/gpu/drm/nouveau/nouveau_ttm.c +4 -4
index 831918437850..1898f27f0510 100644
@@ -194,7 +194,7 @@ nouveau_ttm_init_vram(struct nouveau_drm *drm)
 		ttm_resource_manager_set_used(man, true);
 		return 0;
 	} else {
-		return ttm_range_man_init(&drm->ttm.bdev, TTM_PL_VRAM, false,
+		return ttm_range_man_init(drm->dev, &drm->ttm.bdev, TTM_PL_VRAM, false,
 					  drm->gem.vram_available >> PAGE_SHIFT);
 	}
 }
@@ -211,7 +211,7 @@ nouveau_ttm_fini_vram(struct nouveau_drm *drm)
 		ttm_set_driver_manager(&drm->ttm.bdev, TTM_PL_VRAM, NULL);
 		kfree(man);
 	} else
-		ttm_range_man_fini(&drm->ttm.bdev, TTM_PL_VRAM);
+		ttm_range_man_fini(drm->dev, &drm->ttm.bdev, TTM_PL_VRAM);
 }
 
 static int
@@ -226,7 +226,7 @@ nouveau_ttm_init_gtt(struct nouveau_drm *drm)
 	else if (!drm->agp.bridge)
 		func = &nv04_gart_manager;
 	else
-		return ttm_range_man_init(&drm->ttm.bdev, TTM_PL_TT, true,
+		return ttm_range_man_init(drm->dev, &drm->ttm.bdev, TTM_PL_TT, true,
 					  size_pages);
 
 	man = kzalloc(sizeof(*man), GFP_KERNEL);
@@ -248,7 +248,7 @@ nouveau_ttm_fini_gtt(struct nouveau_drm *drm)
 
 	if (drm->client.device.info.family < NV_DEVICE_INFO_V0_TESLA &&
 	    drm->agp.bridge)
-		ttm_range_man_fini(&drm->ttm.bdev, TTM_PL_TT);
+		ttm_range_man_fini(drm->dev, &drm->ttm.bdev, TTM_PL_TT);
 	else {
 		ttm_resource_manager_set_used(man, false);
 		ttm_resource_manager_evict_all(&drm->ttm.bdev, man);
drivers/gpu/drm/qxl/qxl_ttm.c +3 -3
index 1a82629bce3f..9f984afca831 100644
@@ -186,7 +186,7 @@ static int qxl_ttm_init_mem_type(struct qxl_device *qdev,
 				 unsigned int type,
 				 uint64_t size)
 {
-	return ttm_range_man_init(&qdev->mman.bdev, type, false, size);
+	return ttm_range_man_init(&qdev->ddev, &qdev->mman.bdev, type, false, size);
 }
 
 int qxl_ttm_init(struct qxl_device *qdev)
@@ -227,8 +227,8 @@ int qxl_ttm_init(struct qxl_device *qdev)
 
 void qxl_ttm_fini(struct qxl_device *qdev)
 {
-	ttm_range_man_fini(&qdev->mman.bdev, TTM_PL_VRAM);
-	ttm_range_man_fini(&qdev->mman.bdev, TTM_PL_PRIV);
+	ttm_range_man_fini(&qdev->ddev, &qdev->mman.bdev, TTM_PL_VRAM);
+	ttm_range_man_fini(&qdev->ddev, &qdev->mman.bdev, TTM_PL_PRIV);
 	ttm_device_fini(&qdev->mman.bdev);
 	DRM_INFO("qxl: ttm finalized\n");
 }
drivers/gpu/drm/radeon/radeon_ttm.c +4 -4
index 77ca50187162..5ab3f229082e 100644
@@ -68,13 +68,13 @@ struct radeon_device *radeon_get_rdev(struct ttm_device *bdev)
 
 static int radeon_ttm_init_vram(struct radeon_device *rdev)
 {
-	return ttm_range_man_init(&rdev->mman.bdev, TTM_PL_VRAM,
+	return ttm_range_man_init(rdev->ddev, &rdev->mman.bdev, TTM_PL_VRAM,
 				  false, rdev->mc.real_vram_size >> PAGE_SHIFT);
 }
 
 static int radeon_ttm_init_gtt(struct radeon_device *rdev)
 {
-	return ttm_range_man_init(&rdev->mman.bdev, TTM_PL_TT,
+	return ttm_range_man_init(rdev->ddev, &rdev->mman.bdev, TTM_PL_TT,
 				  true, rdev->mc.gtt_size >> PAGE_SHIFT);
 }
 
@@ -753,8 +753,8 @@ void radeon_ttm_fini(struct radeon_device *rdev)
 		}
 		radeon_bo_unref(&rdev->stolen_vga_memory);
 	}
-	ttm_range_man_fini(&rdev->mman.bdev, TTM_PL_VRAM);
-	ttm_range_man_fini(&rdev->mman.bdev, TTM_PL_TT);
+	ttm_range_man_fini(rdev->ddev, &rdev->mman.bdev, TTM_PL_VRAM);
+	ttm_range_man_fini(rdev->ddev, &rdev->mman.bdev, TTM_PL_TT);
 	ttm_device_fini(&rdev->mman.bdev);
 	radeon_gart_fini(rdev);
 	rdev->mman.initialized = false;
drivers/gpu/drm/ttm/ttm_device.c +1 -1
index e8c8006ba748..393c3e27016e 100644
@@ -212,7 +212,7 @@ int ttm_device_init(struct ttm_device *bdev, const struct ttm_device_funcs *func
 
 	bdev->funcs = funcs;
 
-	ttm_sys_man_init(bdev);
+	ttm_sys_man_init(drm, bdev);
 	ttm_pool_init(&bdev->pool, drm?drm->dev:NULL, NUMA_NO_NODE,
 				use_dma_alloc, use_dma32);
 
drivers/gpu/drm/ttm/ttm_module.h +2 -1
index 767fe22aed48..6c1443704a35 100644
@@ -35,9 +35,10 @@
 
 struct dentry;
 struct ttm_device;
+struct drm_device;
 
 extern struct dentry *ttm_debugfs_root;
 
-void ttm_sys_man_init(struct ttm_device *bdev);
+void ttm_sys_man_init(struct drm_device *drm, struct ttm_device *bdev);
 
 #endif /* _TTM_MODULE_H_ */
drivers/gpu/drm/ttm/ttm_range_manager.c +4 -2
index ae11d07eb63a..db1ae370580d 100644
@@ -166,6 +166,7 @@ static const struct ttm_resource_manager_func ttm_range_manager_func = {
  * ttm_range_man_init_nocheck - Initialise a generic range manager for the
  * selected memory type.
  *
+ * @drm: drm device
  * @bdev: ttm device
  * @type: memory manager type
  * @use_tt: if the memory manager uses tt
@@ -175,7 +176,7 @@ static const struct ttm_resource_manager_func ttm_range_manager_func = {
  *
  * Return: %0 on success or a negative error code on failure
  */
-int ttm_range_man_init_nocheck(struct ttm_device *bdev,
+int ttm_range_man_init_nocheck(struct drm_device *drm, struct ttm_device *bdev,
 		       unsigned type, bool use_tt,
 		       unsigned long p_size)
 {
@@ -206,12 +207,13 @@ EXPORT_SYMBOL(ttm_range_man_init_nocheck);
  * ttm_range_man_fini_nocheck - Remove the generic range manager from a slot
  * and tear it down.
  *
+ * @drm: drm device
  * @bdev: ttm device
  * @type: memory manager type
  *
  * Return: %0 on success or a negative error code on failure
  */
-int ttm_range_man_fini_nocheck(struct ttm_device *bdev,
+int ttm_range_man_fini_nocheck(struct drm_device *drm, struct ttm_device *bdev,
 		       unsigned type)
 {
 	struct ttm_resource_manager *man = ttm_manager_type(bdev, type);
drivers/gpu/drm/ttm/ttm_sys_manager.c +5 -1
index 2ced169513cb..f0f026d40a69 100644
@@ -20,6 +20,10 @@ static int ttm_sys_man_alloc(struct ttm_resource_manager *man,
 	return 0;
 }
 
+/* FIXME: Need to call drm_lru_mgr_fini. This requires a
+ * struct drm_device *drm parameter which need the change
+ * the definition of struct ttm_resource_manager_func. A
+ * very intrusive change. Leave it for now */
 static void ttm_sys_man_free(struct ttm_resource_manager *man,
 			     struct ttm_resource *res)
 {
@@ -32,7 +36,7 @@ static const struct ttm_resource_manager_func ttm_sys_manager_func = {
 	.free = ttm_sys_man_free,
 };
 
-void ttm_sys_man_init(struct ttm_device *bdev)
+void ttm_sys_man_init(struct drm_device *drm, struct ttm_device *bdev)
 {
 	struct ttm_resource_manager *man = &bdev->sysman;
 
drivers/gpu/drm/vmwgfx/vmwgfx_drv.c +2 -2
index cf1c1f16102a..1a0c161e9977 100644
@@ -708,7 +708,7 @@ static int vmw_dma_masks(struct vmw_private *dev_priv)
 static int vmw_vram_manager_init(struct vmw_private *dev_priv)
 {
 	int ret;
-	ret = ttm_range_man_init(&dev_priv->bdev, TTM_PL_VRAM, false,
+	ret = ttm_range_man_init(&dev_priv->drm, &dev_priv->bdev, TTM_PL_VRAM, false,
 				 dev_priv->vram_size >> PAGE_SHIFT);
 	ttm_resource_manager_set_used(ttm_manager_type(&dev_priv->bdev, TTM_PL_VRAM), false);
 	return ret;
@@ -716,7 +716,7 @@ static int vmw_vram_manager_init(struct vmw_private *dev_priv)
 
 static void vmw_vram_manager_fini(struct vmw_private *dev_priv)
 {
-	ttm_range_man_fini(&dev_priv->bdev, TTM_PL_VRAM);
+	ttm_range_man_fini(&dev_priv->drm, &dev_priv->bdev, TTM_PL_VRAM);
 }
 
 static int vmw_setup_pci_resources(struct vmw_private *dev,
include/drm/ttm/ttm_range_manager.h +7 -6
index becdb88c4d84..33cb5016bde6 100644
@@ -34,23 +34,24 @@ to_ttm_range_mgr_node(struct ttm_resource *res)
 	return container_of(res, struct ttm_range_mgr_node, base);
 }
 
-int ttm_range_man_init_nocheck(struct ttm_device *bdev,
+int ttm_range_man_init_nocheck(struct drm_device *drm, struct ttm_device *bdev,
 		       unsigned type, bool use_tt,
 		       unsigned long p_size);
-int ttm_range_man_fini_nocheck(struct ttm_device *bdev,
+int ttm_range_man_fini_nocheck(struct drm_device *drm, struct ttm_device *bdev,
 		       unsigned type);
-static __always_inline int ttm_range_man_init(struct ttm_device *bdev,
+static __always_inline int ttm_range_man_init(struct drm_device *drm,
+			   struct ttm_device *bdev,
 		       unsigned int type, bool use_tt,
 		       unsigned long p_size)
 {
 	BUILD_BUG_ON(__builtin_constant_p(type) && type >= DRM_NUM_MEM_TYPES);
-	return ttm_range_man_init_nocheck(bdev, type, use_tt, p_size);
+	return ttm_range_man_init_nocheck(drm, bdev, type, use_tt, p_size);
 }
 
-static __always_inline int ttm_range_man_fini(struct ttm_device *bdev,
+static __always_inline int ttm_range_man_fini(struct drm_device *drm, struct ttm_device *bdev,
 		       unsigned int type)
 {
 	BUILD_BUG_ON(__builtin_constant_p(type) && type >= DRM_NUM_MEM_TYPES);
-	return ttm_range_man_fini_nocheck(bdev, type);
+	return ttm_range_man_fini_nocheck(drm, bdev, type);
 }
 #endif
Signature · 1 line
2.26.3
author
#14 lore raw

RFC 8/11 drm: Initialize drm lru manager

In reply to Oak Zeng

Initialize lru_mgr for each memory type or memory region. Also set ttm_resource_manager's weak reference to drm lru manager.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c     |  6 ++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.c |  6 ++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c    |  6 ++++++
 drivers/gpu/drm/i915/i915_ttm_buddy_manager.c   | 10 ++++++++++
 drivers/gpu/drm/nouveau/nouveau_ttm.c           | 12 ++++++++++++
 drivers/gpu/drm/ttm/ttm_range_manager.c         |  6 ++++++
 drivers/gpu/drm/ttm/ttm_sys_manager.c           |  2 ++
 drivers/gpu/drm/vmwgfx/vmwgfx_system_manager.c  |  6 ++++++
 drivers/gpu/drm/xe/xe_ttm_sys_mgr.c             |  6 ++++++
 drivers/gpu/drm/xe/xe_ttm_vram_mgr.c            |  6 ++++++
 10 files changed, 66 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +6 -0
index 44367f03316f..57e8b1688977 100644
@@ -278,6 +278,7 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size)
 {
 	struct amdgpu_gtt_mgr *mgr = &adev->mman.gtt_mgr;
 	struct ttm_resource_manager *man = &mgr->manager;
+	struct drm_device *drm = adev_to_drm(adev);
 	uint64_t start, size;
 
 	man->use_tt = true;
@@ -292,6 +293,9 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size)
 
 	ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, &mgr->manager);
 	ttm_resource_manager_set_used(man, true);
+
+	drm_lru_mgr_init(&drm->lru_mgr[TTM_PL_TT], gtt_size, &drm->lru_lock);
+	ttm_resource_manager_set_lru_mgr(man, &drm->lru_mgr[TTM_PL_TT]);
 	return 0;
 }
 
@@ -307,6 +311,7 @@ void amdgpu_gtt_mgr_fini(struct amdgpu_device *adev)
 {
 	struct amdgpu_gtt_mgr *mgr = &adev->mman.gtt_mgr;
 	struct ttm_resource_manager *man = &mgr->manager;
+	struct drm_device *drm = adev_to_drm(adev);
 	int ret;
 
 	ttm_resource_manager_set_used(man, false);
@@ -321,4 +326,5 @@ void amdgpu_gtt_mgr_fini(struct amdgpu_device *adev)
 
 	ttm_resource_manager_cleanup(man);
 	ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, NULL);
+	drm_lru_mgr_fini(&drm->lru_mgr[TTM_PL_TT]);
 }
drivers/gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.c +6 -0
index e8adfd0a570a..f989aca2bfc4 100644
@@ -100,6 +100,7 @@ static const struct ttm_resource_manager_func amdgpu_preempt_mgr_func = {
 int amdgpu_preempt_mgr_init(struct amdgpu_device *adev)
 {
 	struct ttm_resource_manager *man = &adev->mman.preempt_mgr;
+	struct drm_device *drm = adev_to_drm(adev);
 	int ret;
 
 	man->use_tt = true;
@@ -115,6 +116,9 @@ int amdgpu_preempt_mgr_init(struct amdgpu_device *adev)
 
 	ttm_set_driver_manager(&adev->mman.bdev, AMDGPU_PL_PREEMPT, man);
 	ttm_resource_manager_set_used(man, true);
+
+	drm_lru_mgr_init(&drm->lru_mgr[AMDGPU_PL_PREEMPT], (1 << 30), &drm->lru_lock);
+	ttm_resource_manager_set_lru_mgr(man, &drm->lru_mgr[VMW_PL_SYSTEM]);
 	return 0;
 }
 
@@ -129,6 +133,7 @@ int amdgpu_preempt_mgr_init(struct amdgpu_device *adev)
 void amdgpu_preempt_mgr_fini(struct amdgpu_device *adev)
 {
 	struct ttm_resource_manager *man = &adev->mman.preempt_mgr;
+	struct drm_device *drm = adev_to_drm(adev);
 	int ret;
 
 	ttm_resource_manager_set_used(man, false);
@@ -141,4 +146,5 @@ void amdgpu_preempt_mgr_fini(struct amdgpu_device *adev)
 
 	ttm_resource_manager_cleanup(man);
 	ttm_set_driver_manager(&adev->mman.bdev, AMDGPU_PL_PREEMPT, NULL);
+	drm_lru_mgr_fini(&drm->lru_mgr[AMDGPU_PL_PREEMPT]);
 }
drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +6 -0
index b83e1741905e..0792d22508c9 100644
@@ -884,6 +884,7 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev)
 {
 	struct amdgpu_vram_mgr *mgr = &adev->mman.vram_mgr;
 	struct ttm_resource_manager *man = &mgr->manager;
+	struct drm_device *drm = adev_to_drm(adev);
 	int err;
 
 	ttm_resource_manager_init(man, &adev->mman.bdev,
@@ -907,6 +908,9 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev)
 
 	ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, &mgr->manager);
 	ttm_resource_manager_set_used(man, true);
+	drm_lru_mgr_init(&drm->lru_mgr[TTM_PL_VRAM], adev->gmc.real_vram_size,
+						&drm->lru_lock);
+	ttm_resource_manager_set_lru_mgr(man, &drm->lru_mgr[TTM_PL_VRAM]);
 	return 0;
 }
 
@@ -922,6 +926,7 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev)
 {
 	struct amdgpu_vram_mgr *mgr = &adev->mman.vram_mgr;
 	struct ttm_resource_manager *man = &mgr->manager;
+	struct drm_device *drm = adev_to_drm(adev);
 	int ret;
 	struct amdgpu_vram_reservation *rsv, *temp;
 
@@ -945,4 +950,5 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev)
 
 	ttm_resource_manager_cleanup(man);
 	ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, NULL);
+	drm_lru_mgr_fini(&drm->lru_mgr[TTM_PL_VRAM]);
 }
drivers/gpu/drm/i915/i915_ttm_buddy_manager.c +10 -0
index a1bc804cfa15..968f9c01152e 100644
@@ -304,6 +304,9 @@ int i915_ttm_buddy_man_init(struct ttm_device *bdev,
 {
 	struct ttm_resource_manager *man;
 	struct i915_ttm_buddy_manager *bman;
+	struct drm_i915_private *i915 = container_of(bdev,
+							struct drm_i915_private, bdev);
+	struct drm_device *drm = &i915->drm;
 	int err;
 
 	bman = kzalloc(sizeof(*bman), GFP_KERNEL);
@@ -329,6 +332,9 @@ int i915_ttm_buddy_man_init(struct ttm_device *bdev,
 	ttm_resource_manager_set_used(man, true);
 	ttm_set_driver_manager(bdev, type, man);
 
+	drm_lru_mgr_init(&drm->lru_mgr[type], bman->mm.size >> PAGE_SHIFT, &drm->lru_lock);
+	ttm_resource_manager_set_lru_mgr(man, &drm->lru_mgr[type]);
+
 	return 0;
 
 err_free_bman:
@@ -350,6 +356,9 @@ int i915_ttm_buddy_man_fini(struct ttm_device *bdev, unsigned int type)
 {
 	struct ttm_resource_manager *man = ttm_manager_type(bdev, type);
 	struct i915_ttm_buddy_manager *bman = to_buddy_manager(man);
+	struct drm_i915_private *i915 = container_of(bdev,
+							struct drm_i915_private, bdev);
+	struct drm_device *drm = &i915->drm;
 	struct drm_buddy *mm = &bman->mm;
 	int ret;
 
@@ -369,6 +378,7 @@ int i915_ttm_buddy_man_fini(struct ttm_device *bdev, unsigned int type)
 	mutex_unlock(&bman->lock);
 
 	ttm_resource_manager_cleanup(man);
+	drm_lru_mgr_fini(&drm->lru_mgr[type]);
 	kfree(bman);
 
 	return 0;
drivers/gpu/drm/nouveau/nouveau_ttm.c +12 -0
index 1898f27f0510..b0936c235ff6 100644
@@ -182,6 +182,7 @@ nouveau_ttm_init_vram(struct nouveau_drm *drm)
 {
 	if (drm->client.device.info.family >= NV_DEVICE_INFO_V0_TESLA) {
 		struct ttm_resource_manager *man = kzalloc(sizeof(*man), GFP_KERNEL);
+		struct drm_device *drm_dev = drm->dev;
 
 		if (!man)
 			return -ENOMEM;
@@ -192,6 +193,9 @@ nouveau_ttm_init_vram(struct nouveau_drm *drm)
 					  drm->gem.vram_available >> PAGE_SHIFT);
 		ttm_set_driver_manager(&drm->ttm.bdev, TTM_PL_VRAM, man);
 		ttm_resource_manager_set_used(man, true);
+		drm_lru_mgr_init(&drm_dev->lru_mgr[TTM_PL_VRAM],
+			drm->gem.vram_available >> PAGE_SHIFT, &drm_dev->lru_lock);
+		ttm_resource_manager_set_lru_mgr(man, &drm_dev->lru_mgr[TTM_PL_VRAM]);
 		return 0;
 	} else {
 		return ttm_range_man_init(drm->dev, &drm->ttm.bdev, TTM_PL_VRAM, false,
@@ -205,10 +209,13 @@ nouveau_ttm_fini_vram(struct nouveau_drm *drm)
 	struct ttm_resource_manager *man = ttm_manager_type(&drm->ttm.bdev, TTM_PL_VRAM);
 
 	if (drm->client.device.info.family >= NV_DEVICE_INFO_V0_TESLA) {
+		struct drm_device *drm_dev = drm->dev;
+
 		ttm_resource_manager_set_used(man, false);
 		ttm_resource_manager_evict_all(&drm->ttm.bdev, man);
 		ttm_resource_manager_cleanup(man);
 		ttm_set_driver_manager(&drm->ttm.bdev, TTM_PL_VRAM, NULL);
+		drm_lru_mgr_fini(&drm_dev->lru_mgr[TTM_PL_VRAM]);
 		kfree(man);
 	} else
 		ttm_range_man_fini(drm->dev, &drm->ttm.bdev, TTM_PL_VRAM);
@@ -220,6 +227,7 @@ nouveau_ttm_init_gtt(struct nouveau_drm *drm)
 	struct ttm_resource_manager *man;
 	unsigned long size_pages = drm->gem.gart_available >> PAGE_SHIFT;
 	const struct ttm_resource_manager_func *func = NULL;
+	struct drm_device *drm_dev = drm->dev;
 
 	if (drm->client.device.info.family >= NV_DEVICE_INFO_V0_TESLA)
 		func = &nouveau_gart_manager;
@@ -238,6 +246,8 @@ nouveau_ttm_init_gtt(struct nouveau_drm *drm)
 	ttm_resource_manager_init(man, &drm->ttm.bdev, size_pages);
 	ttm_set_driver_manager(&drm->ttm.bdev, TTM_PL_TT, man);
 	ttm_resource_manager_set_used(man, true);
+	drm_lru_mgr_init(&drm_dev->lru_mgr[TTM_PL_TT], size_pages, &drm_dev->lru_lock);
+	ttm_resource_manager_set_lru_mgr(man, &drm_dev->lru_mgr[TTM_PL_TT]);
 	return 0;
 }
 
@@ -245,6 +255,7 @@ static void
 nouveau_ttm_fini_gtt(struct nouveau_drm *drm)
 {
 	struct ttm_resource_manager *man = ttm_manager_type(&drm->ttm.bdev, TTM_PL_TT);
+	struct drm_device *drm_dev = drm->dev;
 
 	if (drm->client.device.info.family < NV_DEVICE_INFO_V0_TESLA &&
 	    drm->agp.bridge)
@@ -254,6 +265,7 @@ nouveau_ttm_fini_gtt(struct nouveau_drm *drm)
 		ttm_resource_manager_evict_all(&drm->ttm.bdev, man);
 		ttm_resource_manager_cleanup(man);
 		ttm_set_driver_manager(&drm->ttm.bdev, TTM_PL_TT, NULL);
+		drm_lru_mgr_fini(&drm_dev->lru_mgr[TTM_PL_TT]);
 		kfree(man);
 	}
 }
drivers/gpu/drm/ttm/ttm_range_manager.c +6 -0
index db1ae370580d..898ede7d25c4 100644
@@ -36,6 +36,7 @@
 #include <drm/drm_mm.h>
 #include <linux/slab.h>
 #include <linux/spinlock.h>
+#include <drm/drm_evictable_lru.h>
 
 /*
  * Currently we use a spinlock for the lock, but a mutex *may* be
@@ -199,6 +200,10 @@ int ttm_range_man_init_nocheck(struct drm_device *drm, struct ttm_device *bdev,
 
 	ttm_set_driver_manager(bdev, type, &rman->manager);
 	ttm_resource_manager_set_used(man, true);
+
+	drm_lru_mgr_init(&drm->lru_mgr[type], p_size, &drm->lru_lock);
+	ttm_resource_manager_set_lru_mgr(man, &drm->lru_mgr[type]);
+
 	return 0;
 }
 EXPORT_SYMBOL(ttm_range_man_init_nocheck);
@@ -236,6 +241,7 @@ int ttm_range_man_fini_nocheck(struct drm_device *drm, struct ttm_device *bdev,
 
 	ttm_resource_manager_cleanup(man);
 	ttm_set_driver_manager(bdev, type, NULL);
+	drm_lru_mgr_fini(&drm->lru_mgr[type]);
 	kfree(rman);
 	return 0;
 }
drivers/gpu/drm/ttm/ttm_sys_manager.c +2 -0
index f0f026d40a69..db410c7f73fe 100644
@@ -50,4 +50,6 @@ void ttm_sys_man_init(struct drm_device *drm, struct ttm_device *bdev)
 	ttm_resource_manager_init(man, bdev, 0);
 	ttm_set_driver_manager(bdev, TTM_PL_SYSTEM, man);
 	ttm_resource_manager_set_used(man, true);
+	drm_lru_mgr_init(&drm->lru_mgr[TTM_PL_SYSTEM], 0, &drm->lru_lock);
+	ttm_resource_manager_set_lru_mgr(man, &drm->lru_mgr[TTM_PL_SYSTEM]);
 }
drivers/gpu/drm/vmwgfx/vmwgfx_system_manager.c +6 -0
index ee7964cbdaca..102296399e00 100644
@@ -62,6 +62,7 @@ int vmw_sys_man_init(struct vmw_private *dev_priv)
 	struct ttm_device *bdev = &dev_priv->bdev;
 	struct ttm_resource_manager *man =
 			kzalloc(sizeof(*man), GFP_KERNEL);
+	struct drm_device *drm = &dev_priv->drm;
 
 	if (!man)
 		return -ENOMEM;
@@ -72,6 +73,9 @@ int vmw_sys_man_init(struct vmw_private *dev_priv)
 	ttm_resource_manager_init(man, bdev, 0);
 	ttm_set_driver_manager(bdev, VMW_PL_SYSTEM, man);
 	ttm_resource_manager_set_used(man, true);
+
+	drm_lru_mgr_init(&drm->lru_mgr[WMW_PL_SYSTEM], 0, &drm->lru_lock);
+	ttm_resource_manager_set_lru_mgr(man, &drm->lru_mgr[VMW_PL_SYSTEM]);
 	return 0;
 }
 
@@ -79,6 +83,7 @@ void vmw_sys_man_fini(struct vmw_private *dev_priv)
 {
 	struct ttm_resource_manager *man = ttm_manager_type(&dev_priv->bdev,
 							    VMW_PL_SYSTEM);
+	struct drm_device *drm = &dev_priv->drm;
 
 	ttm_resource_manager_evict_all(&dev_priv->bdev, man);
 
@@ -86,5 +91,6 @@ void vmw_sys_man_fini(struct vmw_private *dev_priv)
 	ttm_resource_manager_cleanup(man);
 
 	ttm_set_driver_manager(&dev_priv->bdev, VMW_PL_SYSTEM, NULL);
+	drm_lru_mgr_fini(&drm->lru_mgr[VMW_PL_SYSTEM]);
 	kfree(man);
 }
drivers/gpu/drm/xe/xe_ttm_sys_mgr.c +6 -0
index 3e1fa0c832ca..ace42852a419 100644
@@ -96,11 +96,13 @@ static void ttm_sys_mgr_fini(struct drm_device *drm, void *arg)
 
 	ttm_resource_manager_cleanup(man);
 	ttm_set_driver_manager(&xe->ttm, XE_PL_TT, NULL);
+	drm_lru_mgr_fini(&drm->lru_mgr[XE_PL_TT]);
 }
 
 int xe_ttm_sys_mgr_init(struct xe_device *xe)
 {
 	struct ttm_resource_manager *man = &xe->mem.sys_mgr;
+	struct drm_device *drm = &xe->drm;
 	struct sysinfo si;
 	u64 gtt_size;
 
@@ -114,5 +116,9 @@ int xe_ttm_sys_mgr_init(struct xe_device *xe)
 	ttm_resource_manager_init(man, &xe->ttm, gtt_size >> PAGE_SHIFT);
 	ttm_set_driver_manager(&xe->ttm, XE_PL_TT, man);
 	ttm_resource_manager_set_used(man, true);
+
+	drm_lru_mgr_init(&drm->lru_mgr[XE_PL_TT], gtt_size >> PAGE_SHIFT, &drm->lru_lock);
+	ttm_resource_manager_set_lru_mgr(man, &drm->lru_mgr[XE_PL_TT]);
+
 	return drmm_add_action_or_reset(&xe->drm, ttm_sys_mgr_fini, xe);
 }
drivers/gpu/drm/xe/xe_ttm_vram_mgr.c +6 -0
index 06a54c8bd46f..a3c1bf555c06 100644
@@ -328,6 +328,8 @@ static void ttm_vram_mgr_fini(struct drm_device *dev, void *arg)
 	ttm_resource_manager_cleanup(&mgr->manager);
 
 	ttm_set_driver_manager(&xe->ttm, mgr->mem_type, NULL);
+
+	drm_lru_mgr_fini(&dev->lru_mgr[mgr->mem_type]);
 }
 
 int __xe_ttm_vram_mgr_init(struct xe_device *xe, struct xe_ttm_vram_mgr *mgr,
@@ -335,6 +337,7 @@ int __xe_ttm_vram_mgr_init(struct xe_device *xe, struct xe_ttm_vram_mgr *mgr,
 			   u64 default_page_size)
 {
 	struct ttm_resource_manager *man = &mgr->manager;
+	struct drm_device *drm = &xe->drm;
 	int err;
 
 	man->func = &xe_ttm_vram_mgr_func;
@@ -350,6 +353,9 @@ int __xe_ttm_vram_mgr_init(struct xe_device *xe, struct xe_ttm_vram_mgr *mgr,
 	ttm_set_driver_manager(&xe->ttm, mem_type, &mgr->manager);
 	ttm_resource_manager_set_used(&mgr->manager, true);
 
+	drm_lru_mgr_init(&drm->lru_mgr[mem_type], size, &drm->lru_lock);
+	ttm_resource_manager_set_lru_mgr(man, &drm->lru_mgr[mem_type]);
+
 	return drmm_add_action_or_reset(&xe->drm, ttm_vram_mgr_fini, mgr);
 }
 
Signature · 1 line
2.26.3
author
#15 lore raw

RFC 9/11 drm/ttm: Use drm LRU manager iterator

In reply to Oak Zeng

Since TTM resource LRU list is moved to drm LRU manager layer, use drm lru manager iterator instead of TTM resource manager iterator. TTM resource manager iterator is deleted. No function change.

---
 drivers/gpu/drm/ttm/ttm_bo.c       |  7 ++--
 drivers/gpu/drm/ttm/ttm_device.c   | 10 ++++--
 drivers/gpu/drm/ttm/ttm_resource.c | 51 ------------------------------
 include/drm/ttm/ttm_resource.h     | 33 ++-----------------
 4 files changed, 14 insertions(+), 87 deletions(-)
drivers/gpu/drm/ttm/ttm_bo.c +5 -2
index 26e0555bad0c..4a5ffa920665 100644
@@ -43,6 +43,7 @@
 #include <linux/module.h>
 #include <linux/atomic.h>
 #include <linux/dma-resv.h>
+#include <drm/drm_evictable_lru.h>
 
 #include "ttm_module.h"
 
@@ -593,15 +594,17 @@ int ttm_mem_evict_first(struct ttm_device *bdev,
 			struct ww_acquire_ctx *ticket)
 {
 	struct ttm_buffer_object *bo = NULL, *busy_bo = NULL;
-	struct ttm_resource_cursor cursor;
+	struct drm_lru_cursor cursor;
 	struct ttm_resource *res;
+	struct drm_lru_entity *entity;
 	bool locked = false;
 	int ret;
 
 	spin_lock(bdev->lru_lock);
-	ttm_resource_manager_for_each_res(man, &cursor, res) {
+	drm_lru_for_each_entity(man->lru_mgr, &cursor, entity) {
 		bool busy;
 
+		res = container_of(entity, struct ttm_resource, lru_entity);
 		if (!ttm_bo_evict_swapout_allowable(res->bo, ctx, place,
 						    &locked, &busy)) {
 			if (busy && !busy_bo && ticket !=
drivers/gpu/drm/ttm/ttm_device.c +7 -3
index 393c3e27016e..881662d69aba 100644
@@ -33,6 +33,7 @@
 #include <drm/ttm/ttm_device.h>
 #include <drm/ttm/ttm_tt.h>
 #include <drm/ttm/ttm_placement.h>
+#include <drm/drm_evictable_lru.h>
 
 #include "ttm_module.h"
 
@@ -141,7 +142,8 @@ int ttm_global_swapout(struct ttm_operation_ctx *ctx, gfp_t gfp_flags)
 int ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx,
 		       gfp_t gfp_flags)
 {
-	struct ttm_resource_cursor cursor;
+	struct drm_lru_cursor cursor;
+	struct drm_lru_entity *entity;
 	struct ttm_resource_manager *man;
 	struct ttm_resource *res;
 	unsigned i;
@@ -153,10 +155,12 @@ int ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx,
 		if (!man || !man->use_tt)
 			continue;
 
-		ttm_resource_manager_for_each_res(man, &cursor, res) {
-			struct ttm_buffer_object *bo = res->bo;
+		drm_lru_for_each_entity(man->lru_mgr, &cursor, entity) {
+			struct ttm_buffer_object *bo;
 			uint32_t num_pages;
 
+			res = container_of(entity, struct ttm_resource, lru_entity);
+			bo = res->bo;
 			if (!bo || bo->resource != res)
 				continue;
 
drivers/gpu/drm/ttm/ttm_resource.c +0 -51

57 lines not on this page · raw message

include/drm/ttm/ttm_resource.h +2 -31
index e4fc1ada5236..c2528cec12e6 100644
@@ -207,6 +207,7 @@ struct ttm_bus_placement {
  * @placement: Placement flags.
  * @bus: Placement on io bus accessible to the CPU
  * @bo: weak reference to the BO, protected by ttm_device::lru_lock
+ * @lru_entity: lru entity for this ttm resource.
  *
  * Structure indicating the placement and space resources used by a
  * buffer object.
@@ -223,17 +224,7 @@ struct ttm_resource {
 	 * @lru: Least recently used list, see &ttm_resource_manager.lru
 	 */
 	struct list_head lru;
-};
-
-/**
- * struct ttm_resource_cursor
- *
- * @priority: the current priority
- *
- * Cursor to iterate over the resources in a manager.
- */
-struct ttm_resource_cursor {
-	unsigned int priority;
+	struct drm_lru_entity lru_entity;
 };
 
 /**
@@ -402,26 +393,6 @@ uint64_t ttm_resource_manager_usage(struct ttm_resource_manager *man);
 void ttm_resource_manager_debug(struct ttm_resource_manager *man,
 				struct drm_printer *p);
 
-struct ttm_resource *
-ttm_resource_manager_first(struct ttm_resource_manager *man,
-			   struct ttm_resource_cursor *cursor);
-struct ttm_resource *
-ttm_resource_manager_next(struct ttm_resource_manager *man,
-			  struct ttm_resource_cursor *cursor,
-			  struct ttm_resource *res);
-
-/**
- * ttm_resource_manager_for_each_res - iterate over all resources
- * @man: the resource manager
- * @cursor: struct ttm_resource_cursor for the current position
- * @res: the current resource
- *
- * Iterate over all the evictable resources in a resource manager.
- */
-#define ttm_resource_manager_for_each_res(man, cursor, res)		\
-	for (res = ttm_resource_manager_first(man, cursor); res;	\
-	     res = ttm_resource_manager_next(man, cursor, res))
-
 struct ttm_kmap_iter *
 ttm_kmap_iter_iomap_init(struct ttm_kmap_iter_iomap *iter_io,
 			 struct io_mapping *iomap,
Signature · 1 line
2.26.3
author
#16 lore raw

RFC 10/11 drm/ttm: Implement ttm memory evict functions

In reply to Oak Zeng

Implement ttm_mem_evict_valuable, ttm_mem_evict_entity and ttm_mem_evict_busy_entity. Those are callback functions from drm lru manager. Register those functions during drm lru entity initialization. Those 3 functions are splitted from original ttm_mem_evict_first function.

Reimplemented ttm_mem_evict_first function using drm_lru_evict_first function. For now, drm_lru_evict_first just calls back to above 3 functions which are splitted from ttm_mem_evict_first function, so there is no function change. In the future, when SVM code is added, drm_lru_evict_first function can also calls into SVM resource eviction functions, thus TTM and SVM can mutually evict resources from each other.

---
 drivers/gpu/drm/ttm/ttm_bo.c | 192 ++++++++++++++++++++++++++++-------
 include/drm/ttm/ttm_bo.h     |   2 +
 2 files changed, 158 insertions(+), 36 deletions(-)
drivers/gpu/drm/ttm/ttm_bo.c +156 -36
index 4a5ffa920665..9ec7a246e2ad 100644
@@ -587,50 +587,148 @@ static int ttm_mem_evict_wait_busy(struct ttm_buffer_object *busy_bo,
 	return r == -EDEADLK ? -EBUSY : r;
 }
 
-int ttm_mem_evict_first(struct ttm_device *bdev,
-			struct ttm_resource_manager *man,
-			const struct ttm_place *place,
-			struct ttm_operation_ctx *ctx,
-			struct ww_acquire_ctx *ticket)
+struct ttm_mem_evict_ctx {
+	const struct ttm_place *place;
+	struct ttm_operation_ctx *ctx;
+	struct ww_acquire_ctx *ticket;
+};
+
+/**
+ * ttm_mem_evict_allowable
+ *
+ * @lru_entity: The struct ttm_resource::lru_entity when this resource is
+ * added to drm lru list.
+ * @place: The preferred ttm placement where we want to evict memory for
+ * more memory space. If the current ttm_resource doesn't match the preferred
+ * placement, then there is no need to evict the current resource.
+ * @ctx: ttm operation context
+ * @ticket: dma reservation's context used to lock resource
+ * @busy: used to return whether the current resource is busy (i.e., locked
+ * by other clients)
+ * @locked: used to return whether this resource is locked during this check,
+ * i.e., successfully trylocked bo's dma reservation object
+ *
+ * Check whether we are allowed to evict a memory resource. Return true if we
+ * are allowed to evict resource; otherwise false.
+ *
+ * When this function returns true, a resource reference counter (bo's reference)
+ * is hold. This reference counter need to be released after evict operation later
+ * on.
+ *
+ * This function should be called with lru_lock hold.
+ */
+bool ttm_mem_evict_allowable(struct drm_lru_entity *lru_entity,
+			const struct drm_lru_evict_ctx *lru_evict_ctx,
+			bool *busy, bool *locked)
 {
-	struct ttm_buffer_object *bo = NULL, *busy_bo = NULL;
-	struct drm_lru_cursor cursor;
 	struct ttm_resource *res;
-	struct drm_lru_entity *entity;
-	bool locked = false;
-	int ret;
+	struct ttm_buffer_object *bo = NULL;
+	struct ttm_device *bdev;
+	const struct ttm_place *place;
+	struct ttm_operation_ctx *ctx;
+	struct ww_acquire_ctx *ticket;
+	struct ttm_mem_evict_ctx *evict_ctx;
 
-	spin_lock(bdev->lru_lock);
-	drm_lru_for_each_entity(man->lru_mgr, &cursor, entity) {
-		bool busy;
+	evict_ctx = (struct ttm_mem_evict_ctx *)lru_evict_ctx;
+	place = evict_ctx->place;
+	ctx = evict_ctx->ctx;
+	ticket = evict_ctx->ticket;
 
-		res = container_of(entity, struct ttm_resource, lru_entity);
-		if (!ttm_bo_evict_swapout_allowable(res->bo, ctx, place,
-						    &locked, &busy)) {
-			if (busy && !busy_bo && ticket !=
-			    dma_resv_locking_ctx(res->bo->base.resv))
-				busy_bo = res->bo;
-			continue;
-		}
+	res = container_of(lru_entity, struct ttm_resource, lru_entity);
+	bo = res->bo;
+	bdev = bo->bdev;
 
-		if (ttm_bo_get_unless_zero(res->bo)) {
-			bo = res->bo;
-			break;
-		}
-		if (locked)
-			dma_resv_unlock(res->bo->base.resv);
-	}
+	if (!ttm_bo_evict_swapout_allowable(bo, ctx, place, locked, busy)) {
+		if (busy && ticket != dma_resv_locking_ctx(bo->base.resv))
+			*busy = true;
 
-	if (!bo) {
-		if (busy_bo && !ttm_bo_get_unless_zero(busy_bo))
-			busy_bo = NULL;
-		spin_unlock(bdev->lru_lock);
-		ret = ttm_mem_evict_wait_busy(busy_bo, ctx, ticket);
-		if (busy_bo)
-			ttm_bo_put(busy_bo);
-		return ret;
+		return false;
 	}
 
+	if (ttm_bo_get_unless_zero(bo))
+		return true;
+
+	if (locked)
+		dma_resv_unlock(bo->base.resv);
+
+	return false;
+}
+
+/**
+ * ttm_mem_evict_busy_entity
+ *
+ * @lru_entity: The struct ttm_resource::lru_entity when this resource is
+ * added to drm lru list.
+ * @ctx: ttm operation context
+ * @ticket: dma reservation's context used to lock resource
+ *
+ * Evict a busy memory resource.
+ * This function should be called with lru_lock hold.
+ */
+int ttm_mem_evict_busy_entity(struct drm_lru_entity *lru_entity,
+			const struct drm_lru_evict_ctx *lru_evict_ctx)
+{
+	struct ttm_resource *res;
+	struct ttm_buffer_object *bo = NULL;
+	struct ttm_device *bdev;
+	int ret;
+	struct ttm_operation_ctx *ctx;
+	struct ww_acquire_ctx *ticket;
+	struct ttm_mem_evict_ctx *evict_ctx;
+
+	evict_ctx = (struct ttm_mem_evict_ctx *)lru_evict_ctx;
+	ctx = evict_ctx->ctx;
+	ticket = evict_ctx->ticket;
+
+	res = container_of(lru_entity, struct ttm_resource, lru_entity);
+	bo = res->bo;
+	bdev = bo->bdev;
+
+	if (bo && !ttm_bo_get_unless_zero(bo))
+		bo = NULL;
+	spin_unlock(bdev->lru_lock);
+	ret = ttm_mem_evict_wait_busy(bo, ctx, ticket);
+	/* FIXME: this is code copied originally from ttm_mem_evict_first.
+	 * 1) Shouldn't we ttm_bo_evict this bo also? Otherwise how can we
+	 * make any memory space?
+	 * 2) We also need to check whether this busy entity is in the same
+	 * ttm_place as specified in lru_evict_ctx::place; if not, there is
+	 * no help to wait this busy entity.
+     */
+	if (bo)
+		ttm_bo_put(bo);
+
+	return ret;
+}
+
+/**
+ * @lru_entity: The struct ttm_resource::lru_entity when this resource is
+ * added to drm lru list.
+ * @ctx: ttm operation context
+ * @locked: whether this resource is dma-reserved (if reserved, we need to
+ * unreserve it in this function)
+ *
+ * Evict a memory resource corresponding to a lru_entity. This should be
+ * called holding lru_lock
+ *
+ */
+int ttm_mem_evict_entity(struct drm_lru_entity *lru_entity,
+			const struct drm_lru_evict_ctx *lru_evict_ctx, bool locked)
+{
+	struct ttm_resource *res;
+	struct ttm_buffer_object *bo = NULL;
+	struct ttm_device *bdev;
+	int ret;
+	struct ttm_operation_ctx *ctx;
+	struct ttm_mem_evict_ctx *evict_ctx;
+
+	evict_ctx = (struct ttm_mem_evict_ctx *)lru_evict_ctx;
+	ctx = evict_ctx->ctx;
+
+	res = container_of(lru_entity, struct ttm_resource, lru_entity);
+	bo = res->bo;
+	bdev = bo->bdev;
+
 	if (bo->deleted) {
 		ret = ttm_bo_cleanup_refs(bo, ctx->interruptible,
 					  ctx->no_wait_gpu, locked);
@@ -650,6 +748,28 @@ int ttm_mem_evict_first(struct ttm_device *bdev,
 	return ret;
 }
 
+struct drm_lru_evict_func ttm_evict_func = {
+	.evict_allowable = ttm_mem_evict_allowable,
+	.evict_busy_entity = ttm_mem_evict_busy_entity,
+	.evict_entity = ttm_mem_evict_entity
+};
+EXPORT_SYMBOL(ttm_evict_func);
+
+int ttm_mem_evict_first(struct ttm_device *bdev,
+			struct ttm_resource_manager *man,
+			const struct ttm_place *place,
+			struct ttm_operation_ctx *ctx,
+			struct ww_acquire_ctx *ticket)
+{
+	struct drm_lru_evict_ctx evict_ctx = {
+			.data1 = place,
+			.data2 = ctx,
+			.data3 = ticket
+	};
+
+	return drm_lru_evict_first(man->lru_mgr, &evict_ctx);
+}
+
 /**
  * ttm_bo_pin - Pin the buffer object.
  * @bo: The buffer object to pin
include/drm/ttm/ttm_bo.h +2 -0
index 49f32df32204..223b198fe371 100644
@@ -50,6 +50,7 @@ struct ttm_place;
 struct ttm_resource;
 struct ttm_resource_manager;
 struct ttm_tt;
+struct drm_lru_evict_func;
 
 /**
  * enum ttm_bo_type
@@ -424,4 +425,5 @@ pgprot_t ttm_io_prot(struct ttm_buffer_object *bo, struct ttm_resource *res,
 		     pgprot_t tmp);
 void ttm_bo_tt_destroy(struct ttm_buffer_object *bo);
 
+extern struct drm_lru_evict_func ttm_evict_func;
 #endif
Signature · 1 line
2.26.3
author
#17 lore raw

RFC 11/11 drm/ttm: Write ttm functions using drm lru manager functions

In reply to Oak Zeng

Replace struct ttm_resource::lru with drm lru entity. Replace struct ttm_resource_manager::lru[] with drm lru manager. Remove ttm_lru_bulk_move functions and definitions as those are moved to drm lru manager.

Some of ttm resource, ttm bo and ttm device functions are re-written using drm lru manager functions.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c      |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h      |   2 +-
 drivers/gpu/drm/ttm/tests/ttm_device_test.c |   2 +-
 drivers/gpu/drm/ttm/ttm_bo.c                |  20 +--
 drivers/gpu/drm/ttm/ttm_bo_util.c           |  20 +--
 drivers/gpu/drm/ttm/ttm_bo_vm.c             |   2 +-
 drivers/gpu/drm/ttm/ttm_device.c            |  10 +-
 drivers/gpu/drm/ttm/ttm_range_manager.c     |   2 +-
 drivers/gpu/drm/ttm/ttm_resource.c          | 155 ++++----------------
 drivers/gpu/drm/xe/xe_bo.c                  |  44 +++---
 drivers/gpu/drm/xe/xe_bo.h                  |   3 +-
 drivers/gpu/drm/xe/xe_dma_buf.c             |   4 +-
 drivers/gpu/drm/xe/xe_exec.c                |   2 +-
 drivers/gpu/drm/xe/xe_migrate.c             |   6 +-
 drivers/gpu/drm/xe/xe_res_cursor.h          |  10 +-
 drivers/gpu/drm/xe/xe_ttm_sys_mgr.c         |   2 +-
 drivers/gpu/drm/xe/xe_ttm_vram_mgr.c        |  12 +-
 drivers/gpu/drm/xe/xe_vm.c                  |   2 +-
 drivers/gpu/drm/xe/xe_vm_types.h            |   2 +-
 include/drm/ttm/ttm_bo.h                    |   4 +-
 include/drm/ttm/ttm_resource.h              |  59 ++------
 21 files changed, 112 insertions(+), 253 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +1 -1
index 747bcad86d5d..c977c00e986a 100644
@@ -369,7 +369,7 @@ void amdgpu_vm_move_to_lru_tail(struct amdgpu_device *adev,
 				struct amdgpu_vm *vm)
 {
 	spin_lock(adev->mman.bdev.lru_lock);
-	ttm_lru_bulk_move_tail(&vm->lru_bulk_move);
+	drm_lru_bulk_move_tail(&vm->lru_bulk_move);
 	spin_unlock(adev->mman.bdev.lru_lock);
 }
 
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +1 -1
index 204ab13184ed..fec545b5d154 100644
@@ -337,7 +337,7 @@ struct amdgpu_vm {
 	struct amdgpu_task_info task_info;
 
 	/* Store positions of group of BOs */
-	struct ttm_lru_bulk_move lru_bulk_move;
+	struct drm_lru_bulk_move lru_bulk_move;
 	/* Flag to indicate if VM is used for compute */
 	bool			is_compute_context;
 
drivers/gpu/drm/ttm/tests/ttm_device_test.c +1 -1
index b1b423b68cdf..a62ca31b55df 100644
@@ -90,7 +90,7 @@ static void ttm_device_fini_basic(struct kunit *test)
 	ttm_device_fini(ttm_dev);
 
 	KUNIT_ASSERT_FALSE(test, man->use_type);
-	KUNIT_ASSERT_TRUE(test, list_empty(&man->lru[0]));
+	KUNIT_ASSERT_TRUE(test, list_empty(&man->lru_mgr->lru[0]));
 	KUNIT_ASSERT_NULL(test, ttm_dev->man_drv[TTM_PL_SYSTEM]);
 }
 
drivers/gpu/drm/ttm/ttm_bo.c +10 -10
index 9ec7a246e2ad..d44ca5e51dff 100644
@@ -92,11 +92,11 @@ EXPORT_SYMBOL(ttm_bo_move_to_lru_tail);
  * resulting in much less overhead of maintaining the LRU.
  * The only requirement is that the resources stay together on the LRU and are
  * never separated. This is enforces by setting the bulk_move structure on a BO.
- * ttm_lru_bulk_move_tail() should be used to move all resources to the tail of
+ * drm_lru_bulk_move_tail() should be used to move all resources to the tail of
  * their LRU list.
  */
 void ttm_bo_set_bulk_move(struct ttm_buffer_object *bo,
-			  struct ttm_lru_bulk_move *bulk)
+			  struct drm_lru_bulk_move *bulk)
 {
 	dma_resv_assert_held(bo->base.resv);
 
@@ -122,8 +122,8 @@ static int ttm_bo_handle_move_mem(struct ttm_buffer_object *bo,
 	bool old_use_tt, new_use_tt;
 	int ret;
 
-	old_use_tt = !bo->resource || ttm_manager_type(bdev, bo->resource->mem_type)->use_tt;
-	new_use_tt = ttm_manager_type(bdev, mem->mem_type)->use_tt;
+	old_use_tt = !bo->resource || ttm_manager_type(bdev, bo->resource->lru_entity.mem_type)->use_tt;
+	new_use_tt = ttm_manager_type(bdev, mem->lru_entity.mem_type)->use_tt;
 
 	ttm_bo_unmap_virtual(bo);
 
@@ -139,7 +139,7 @@ static int ttm_bo_handle_move_mem(struct ttm_buffer_object *bo,
 		if (ret)
 			goto out_err;
 
-		if (mem->mem_type != TTM_PL_SYSTEM) {
+		if (mem->lru_entity.mem_type != TTM_PL_SYSTEM) {
 			ret = ttm_tt_populate(bo->bdev, bo->ttm, ctx);
 			if (ret)
 				goto out_err;
@@ -492,7 +492,7 @@ bool ttm_bo_eviction_valuable(struct ttm_buffer_object *bo,
 	struct ttm_device *bdev = bo->bdev;
 
 	dma_resv_assert_held(bo->base.resv);
-	if (bo->resource->mem_type == TTM_PL_SYSTEM)
+	if (bo->resource->lru_entity.mem_type == TTM_PL_SYSTEM)
 		return true;
 
 	/* Don't evict this BO if it's outside of the
@@ -540,7 +540,7 @@ static bool ttm_bo_evict_swapout_allowable(struct ttm_buffer_object *bo,
 			*busy = !ret;
 	}
 
-	if (ret && place && (bo->resource->mem_type != place->mem_type ||
+	if (ret && place && (bo->resource->lru_entity.mem_type != place->mem_type ||
 		!bo->bdev->funcs->eviction_valuable(bo, place))) {
 		ret = false;
 		if (*locked) {
@@ -1039,7 +1039,7 @@ int ttm_bo_validate(struct ttm_buffer_object *bo,
 	/*
 	 * We might need to add a TTM.
 	 */
-	if (!bo->resource || bo->resource->mem_type == TTM_PL_SYSTEM) {
+	if (!bo->resource || bo->resource->lru_entity.mem_type == TTM_PL_SYSTEM) {
 		ret = ttm_tt_create(bo, true);
 		if (ret)
 			return ret;
@@ -1259,7 +1259,7 @@ int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx,
 	 * as an indication that we're about to swap out.
 	 */
 	memset(&place, 0, sizeof(place));
-	place.mem_type = bo->resource->mem_type;
+	place.mem_type = bo->resource->lru_entity.mem_type;
 	if (!ttm_bo_evict_swapout_allowable(bo, ctx, &place, &locked, NULL))
 		return -EBUSY;
 
@@ -1284,7 +1284,7 @@ int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx,
 	/*
 	 * Move to system cached
 	 */
-	if (bo->resource->mem_type != TTM_PL_SYSTEM) {
+	if (bo->resource->lru_entity.mem_type != TTM_PL_SYSTEM) {
 		struct ttm_resource *evict_mem;
 		struct ttm_place hop;
 
drivers/gpu/drm/ttm/ttm_bo_util.c +10 -10
index fd9fd3d15101..7176fbfca5eb 100644
@@ -145,7 +145,7 @@ int ttm_bo_move_memcpy(struct ttm_buffer_object *bo,
 {
 	struct ttm_device *bdev = bo->bdev;
 	struct ttm_resource_manager *dst_man =
-		ttm_manager_type(bo->bdev, dst_mem->mem_type);
+		ttm_manager_type(bo->bdev, dst_mem->lru_entity.mem_type);
 	struct ttm_tt *ttm = bo->ttm;
 	struct ttm_resource *src_mem = bo->resource;
 	struct ttm_resource_manager *src_man;
@@ -160,7 +160,7 @@ int ttm_bo_move_memcpy(struct ttm_buffer_object *bo,
 	if (WARN_ON(!src_mem))
 		return -EINVAL;
 
-	src_man = ttm_manager_type(bdev, src_mem->mem_type);
+	src_man = ttm_manager_type(bdev, src_mem->lru_entity.mem_type);
 	if (ttm && ((ttm->page_flags & TTM_TT_FLAG_SWAPPED) ||
 		    dst_man->use_tt)) {
 		ret = ttm_tt_populate(bdev, ttm, ctx);
@@ -184,7 +184,7 @@ int ttm_bo_move_memcpy(struct ttm_buffer_object *bo,
 
 	clear = src_iter->ops->maps_tt && (!ttm || !ttm_tt_is_populated(ttm));
 	if (!(clear && ttm && !(ttm->page_flags & TTM_TT_FLAG_ZERO_ALLOC)))
-		ttm_move_memcpy(clear, PFN_UP(dst_mem->size), dst_iter, src_iter);
+		ttm_move_memcpy(clear, PFN_UP(dst_mem->lru_entity.size), dst_iter, src_iter);
 
 	if (!src_iter->ops->maps_tt)
 		ttm_kmap_iter_linear_io_fini(&_src_iter.io, bdev, src_mem);
@@ -293,7 +293,7 @@ pgprot_t ttm_io_prot(struct ttm_buffer_object *bo, struct ttm_resource *res,
 	struct ttm_resource_manager *man;
 	enum ttm_caching caching;
 
-	man = ttm_manager_type(bo->bdev, res->mem_type);
+	man = ttm_manager_type(bo->bdev, res->lru_entity.mem_type);
 	caching = man->use_tt ? bo->ttm->caching : res->bus.caching;
 
 	return ttm_prot_from_caching(caching, tmp);
@@ -393,9 +393,9 @@ int ttm_bo_kmap(struct ttm_buffer_object *bo,
 
 	map->virtual = NULL;
 	map->bo = bo;
-	if (num_pages > PFN_UP(bo->resource->size))
+	if (num_pages > PFN_UP(bo->resource->lru_entity.size))
 		return -EINVAL;
-	if ((start_page + num_pages) > PFN_UP(bo->resource->size))
+	if ((start_page + num_pages) > PFN_UP(bo->resource->lru_entity.size))
 		return -EINVAL;
 
 	ret = ttm_mem_io_reserve(bo->bdev, bo->resource);
@@ -607,7 +607,7 @@ static void ttm_bo_move_pipeline_evict(struct ttm_buffer_object *bo,
 	struct ttm_device *bdev = bo->bdev;
 	struct ttm_resource_manager *from;
 
-	from = ttm_manager_type(bdev, bo->resource->mem_type);
+	from = ttm_manager_type(bdev, bo->resource->lru_entity.mem_type);
 
 	/**
 	 * BO doesn't have a TTM we need to bind/unbind. Just remember
@@ -646,8 +646,8 @@ int ttm_bo_move_accel_cleanup(struct ttm_buffer_object *bo,
 			      struct ttm_resource *new_mem)
 {
 	struct ttm_device *bdev = bo->bdev;
-	struct ttm_resource_manager *from = ttm_manager_type(bdev, bo->resource->mem_type);
-	struct ttm_resource_manager *man = ttm_manager_type(bdev, new_mem->mem_type);
+	struct ttm_resource_manager *from = ttm_manager_type(bdev, bo->resource->lru_entity.mem_type);
+	struct ttm_resource_manager *man = ttm_manager_type(bdev, new_mem->lru_entity.mem_type);
 	int ret = 0;
 
 	dma_resv_add_fence(bo->base.resv, fence, DMA_RESV_USAGE_KERNEL);
@@ -680,7 +680,7 @@ void ttm_bo_move_sync_cleanup(struct ttm_buffer_object *bo,
 			      struct ttm_resource *new_mem)
 {
 	struct ttm_device *bdev = bo->bdev;
-	struct ttm_resource_manager *man = ttm_manager_type(bdev, new_mem->mem_type);
+	struct ttm_resource_manager *man = ttm_manager_type(bdev, new_mem->lru_entity.mem_type);
 	int ret;
 
 	ret = ttm_bo_wait_free_node(bo, man->use_tt);
drivers/gpu/drm/ttm/ttm_bo_vm.c +1 -1
index 4212b8c91dd4..0e550430fa85 100644
@@ -421,7 +421,7 @@ int ttm_bo_vm_access(struct vm_area_struct *vma, unsigned long addr,
 	if (ret)
 		return ret;
 
-	switch (bo->resource->mem_type) {
+	switch (bo->resource->lru_entity.mem_type) {
 	case TTM_PL_SYSTEM:
 		fallthrough;
 	case TTM_PL_TT:
drivers/gpu/drm/ttm/ttm_device.c +6 -4
index 881662d69aba..7e5bfdffc08d 100644
@@ -250,7 +250,7 @@ void ttm_device_fini(struct ttm_device *bdev)
 
 	spin_lock(bdev->lru_lock);
 	for (i = 0; i < DRM_MAX_LRU_PRIORITY; ++i)
-		if (list_empty(&man->lru[0]))
+		if (list_empty(&man->lru_mgr->lru[0]))
 			pr_debug("Swap list %d was clean\n", i);
 	spin_unlock(bdev->lru_lock);
 
@@ -263,16 +263,18 @@ static void ttm_device_clear_lru_dma_mappings(struct ttm_device *bdev,
 					      struct list_head *list)
 {
 	struct ttm_resource *res;
+	struct drm_lru_entity *entity;
 
 	spin_lock(bdev->lru_lock);
-	while ((res = list_first_entry_or_null(list, typeof(*res), lru))) {
+	while ((entity = list_first_entry_or_null(list, typeof(*entity), lru))){
+		res = container_of(entity, struct ttm_resource, lru_entity);
 		struct ttm_buffer_object *bo = res->bo;
 
 		/* Take ref against racing releases once lru_lock is unlocked */
 		if (!ttm_bo_get_unless_zero(bo))
 			continue;
 
-		list_del_init(&res->lru);
+		list_del_init(&entity->lru);
 		spin_unlock(bdev->lru_lock);
 
 		if (bo->ttm)
@@ -297,7 +299,7 @@ void ttm_device_clear_dma_mappings(struct ttm_device *bdev)
 			continue;
 
 		for (j = 0; j < DRM_MAX_LRU_PRIORITY; ++j)
-			ttm_device_clear_lru_dma_mappings(bdev, &man->lru[j]);
+			ttm_device_clear_lru_dma_mappings(bdev, &man->lru_mgr->lru[j]);
 	}
 }
 EXPORT_SYMBOL(ttm_device_clear_dma_mappings);
drivers/gpu/drm/ttm/ttm_range_manager.c +1 -1
index 898ede7d25c4..afdea64ffc10 100644
@@ -84,7 +84,7 @@ static int ttm_range_man_alloc(struct ttm_resource_manager *man,
 
 	spin_lock(&rman->lock);
 	ret = drm_mm_insert_node_in_range(mm, &node->mm_nodes[0],
-					  PFN_UP(node->base.size),
+					  PFN_UP(node->base.lru_entity.size),
 					  bo->page_alignment, 0,
 					  place->fpfn, lpfn, mode);
 	spin_unlock(&rman->lock);
drivers/gpu/drm/ttm/ttm_resource.c +25 -130

257 lines not on this page · raw message

drivers/gpu/drm/xe/xe_bo.c +22 -22

161 lines not on this page · raw message

drivers/gpu/drm/xe/xe_bo.h +2 -1
index 9918b2d630e1..3ab17c81fe6e 100644
@@ -9,6 +9,7 @@
 #include "xe_bo_types.h"
 #include "xe_macros.h"
 #include "xe_vm_types.h"
+#include <drm/drm_evictable_lru.h>
 
 #define XE_DEFAULT_GTT_SIZE_MB          3072ULL /* 3GB by default */
 
@@ -83,7 +84,7 @@ void xe_bo_free(struct xe_bo *bo);
 
 struct xe_bo *__xe_bo_create_locked(struct xe_device *xe, struct xe_bo *bo,
 				    struct xe_tile *tile, struct dma_resv *resv,
-				    struct ttm_lru_bulk_move *bulk, size_t size,
+				    struct drm_lru_bulk_move *bulk, size_t size,
 				    enum ttm_bo_type type, u32 flags);
 struct xe_bo *
 xe_bo_create_locked_range(struct xe_device *xe,
drivers/gpu/drm/xe/xe_dma_buf.c +2 -2
index 09343b8b3e96..f22a4e0388d4 100644
@@ -82,7 +82,7 @@ static struct sg_table *xe_dma_buf_map(struct dma_buf_attachment *attach,
 
 	if (!xe_bo_is_pinned(bo)) {
 		if (!attach->peer2peer ||
-		    bo->ttm.resource->mem_type == XE_PL_SYSTEM) {
+		    bo->ttm.resource->lru_entity.mem_type == XE_PL_SYSTEM) {
 			if (xe_bo_can_migrate(bo, XE_PL_TT))
 				r = xe_bo_migrate(bo, XE_PL_TT);
 			else
@@ -92,7 +92,7 @@ static struct sg_table *xe_dma_buf_map(struct dma_buf_attachment *attach,
 			return ERR_PTR(r);
 	}
 
-	switch (bo->ttm.resource->mem_type) {
+	switch (bo->ttm.resource->lru_entity.mem_type) {
 	case XE_PL_TT:
 		sgt = drm_prime_pages_to_sg(obj->dev,
 					    bo->ttm.ttm->pages,
drivers/gpu/drm/xe/xe_exec.c +1 -1
index dafebdfb2368..100385e71a94 100644
@@ -371,7 +371,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 
 	if (!err && !xe_vm_no_dma_fences(vm)) {
 		spin_lock(xe->ttm.lru_lock);
-		ttm_lru_bulk_move_tail(&vm->lru_bulk_move);
+		drm_lru_bulk_move_tail(&vm->lru_bulk_move);
 		spin_unlock(xe->ttm.lru_lock);
 	}
 
drivers/gpu/drm/xe/xe_migrate.c +3 -3
index ee8bc5f3ba3d..fdf2a448457d 100644
@@ -643,8 +643,8 @@ struct dma_fence *xe_migrate_copy(struct xe_migrate *m,
 	u64 src_L0, dst_L0;
 	int pass = 0;
 	int err;
-	bool src_is_vram = mem_type_is_vram(src->mem_type);
-	bool dst_is_vram = mem_type_is_vram(dst->mem_type);
+	bool src_is_vram = mem_type_is_vram(src->lru_entity.mem_type);
+	bool dst_is_vram = mem_type_is_vram(dst->lru_entity.mem_type);
 	bool copy_ccs = xe_device_has_flat_ccs(xe) &&
 		xe_bo_needs_ccs_pages(src_bo) && xe_bo_needs_ccs_pages(dst_bo);
 	bool copy_system_ccs = copy_ccs && (!src_is_vram || !dst_is_vram);
@@ -895,7 +895,7 @@ struct dma_fence *xe_migrate_clear(struct xe_migrate *m,
 				   struct xe_bo *bo,
 				   struct ttm_resource *dst)
 {
-	bool clear_vram = mem_type_is_vram(dst->mem_type);
+	bool clear_vram = mem_type_is_vram(dst->lru_entity.mem_type);
 	struct xe_gt *gt = m->tile->primary_gt;
 	struct xe_device *xe = gt_to_xe(gt);
 	struct dma_fence *fence = NULL;
drivers/gpu/drm/xe/xe_res_cursor.h +5 -5
index 5cb4b66a5d74..64c5549f4d44 100644
@@ -53,8 +53,8 @@ static struct drm_buddy *xe_res_get_buddy(struct ttm_resource *res)
 	struct xe_device *xe = ttm_to_xe_device(res->bo->bdev);
 	struct ttm_resource_manager *mgr;
 
-	if (res->mem_type != XE_PL_STOLEN)
-		return &xe->tiles[res->mem_type - XE_PL_VRAM0].mem.vram_mgr->mm;
+	if (res->lru_entity.mem_type != XE_PL_STOLEN)
+		return &xe->tiles[res->lru_entity.mem_type - XE_PL_VRAM0].mem.vram_mgr->mm;
 
 	mgr = ttm_manager_type(&xe->ttm, XE_PL_STOLEN);
 
@@ -79,9 +79,9 @@ static inline void xe_res_first(struct ttm_resource *res,
 	if (!res)
 		goto fallback;
 
-	XE_WARN_ON(start + size > res->size);
+	XE_WARN_ON(start + size > res->lru_entity.size);
 
-	cur->mem_type = res->mem_type;
+	cur->mem_type = res->lru_entity.mem_type;
 
 	switch (cur->mem_type) {
 	case XE_PL_STOLEN:
@@ -128,7 +128,7 @@ static inline void xe_res_first(struct ttm_resource *res,
 	cur->remaining = size;
 	cur->node = NULL;
 	cur->mem_type = XE_PL_TT;
-	XE_WARN_ON(res && start + size > res->size);
+	XE_WARN_ON(res && start + size > res->lru_entity.size);
 }
 
 static inline void __xe_res_sg_next(struct xe_res_cursor *cur)
drivers/gpu/drm/xe/xe_ttm_sys_mgr.c +1 -1
index ace42852a419..7aa179e73d50 100644
@@ -48,7 +48,7 @@ static int xe_ttm_sys_mgr_new(struct ttm_resource_manager *man,
 	}
 
 	node->base.mm_nodes[0].start = 0;
-	node->base.mm_nodes[0].size = PFN_UP(node->base.base.size);
+	node->base.mm_nodes[0].size = PFN_UP(node->base.base.lru_entity.size);
 	node->base.base.start = XE_BO_INVALID_OFFSET;
 
 	*res = &node->base.base;
drivers/gpu/drm/xe/xe_ttm_vram_mgr.c +6 -6
index a3c1bf555c06..06a469cc8490 100644
@@ -83,11 +83,11 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man,
 	if (place->fpfn || lpfn != man->size >> PAGE_SHIFT)
 		vres->flags |= DRM_BUDDY_RANGE_ALLOCATION;
 
-	if (WARN_ON(!vres->base.size)) {
+	if (WARN_ON(!vres->base.lru_entity.size)) {
 		err = -EINVAL;
 		goto error_fini;
 	}
-	size = vres->base.size;
+	size = vres->base.lru_entity.size;
 
 	min_page_size = mgr->default_page_size;
 	if (tbo->page_alignment)
@@ -150,8 +150,8 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man,
 	} while (remaining_size);
 
 	if (place->flags & TTM_PL_FLAG_CONTIGUOUS) {
-		if (!drm_buddy_block_trim(mm, vres->base.size, &vres->blocks))
-			size = vres->base.size;
+		if (!drm_buddy_block_trim(mm, vres->base.lru_entity.size, &vres->blocks))
+			size = vres->base.lru_entity.size;
 	}
 
 	if (lpfn <= mgr->visible_size >> PAGE_SHIFT) {
@@ -378,14 +378,14 @@ int xe_ttm_vram_mgr_alloc_sgt(struct xe_device *xe,
 			      enum dma_data_direction dir,
 			      struct sg_table **sgt)
 {
-	struct xe_tile *tile = &xe->tiles[res->mem_type - XE_PL_VRAM0];
+	struct xe_tile *tile = &xe->tiles[res->lru_entity.mem_type - XE_PL_VRAM0];
 	struct xe_ttm_vram_mgr_resource *vres = to_xe_ttm_vram_mgr_resource(res);
 	struct xe_res_cursor cursor;
 	struct scatterlist *sg;
 	int num_entries = 0;
 	int i, r;
 
-	if (vres->used_visible_size < res->size)
+	if (vres->used_visible_size < res->lru_entity.size)
 		return -EOPNOTSUPP;
 
 	*sgt = kmalloc(sizeof(**sgt), GFP_KERNEL);
drivers/gpu/drm/xe/xe_vm.c +1 -1
index 44e038276d41..f96009b02800 100644
@@ -652,7 +652,7 @@ static void preempt_rebind_work_func(struct work_struct *w)
 #undef retry_required
 
 	spin_lock(vm->xe->ttm.lru_lock);
-	ttm_lru_bulk_move_tail(&vm->lru_bulk_move);
+	drm_lru_bulk_move_tail(&vm->lru_bulk_move);
 	spin_unlock(vm->xe->ttm.lru_lock);
 
 	/* Point of no return. */
drivers/gpu/drm/xe/xe_vm_types.h +1 -1
index 52e5eaed91c3..784b07660fff 100644
@@ -149,7 +149,7 @@ struct xe_vm {
 	struct dma_resv resv;
 
 	/** @lru_bulk_move: Bulk LRU move list for this VM's BOs */
-	struct ttm_lru_bulk_move lru_bulk_move;
+	struct drm_lru_bulk_move lru_bulk_move;
 
 	u64 size;
 
include/drm/ttm/ttm_bo.h +2 -2
index 223b198fe371..f4d939ee174c 100644
@@ -118,7 +118,7 @@ struct ttm_buffer_object {
 	struct ttm_resource *resource;
 	struct ttm_tt *ttm;
 	bool deleted;
-	struct ttm_lru_bulk_move *bulk_move;
+	struct drm_lru_bulk_move *bulk_move;
 	unsigned priority;
 	unsigned pin_count;
 
@@ -355,7 +355,7 @@ int ttm_bo_validate(struct ttm_buffer_object *bo,
 		    struct ttm_operation_ctx *ctx);
 void ttm_bo_put(struct ttm_buffer_object *bo);
 void ttm_bo_set_bulk_move(struct ttm_buffer_object *bo,
-			  struct ttm_lru_bulk_move *bulk);
+			  struct drm_lru_bulk_move *bulk);
 bool ttm_bo_eviction_valuable(struct ttm_buffer_object *bo,
 			      const struct ttm_place *place);
 int ttm_bo_init_reserved(struct ttm_device *bdev, struct ttm_buffer_object *bo,
include/drm/ttm/ttm_resource.h +10 -49

115 lines not on this page · raw message

Signature · 1 line
2.26.3
#18 lore raw

[Intel-xe] ✗ CI.Patch_applied: failure for Introduce drm evictable lru

In reply to Oak Zeng
== Series Details ==

== Series Details ==

Series: Introduce drm evictable lru
URL : https://patchwork.freedesktop.org/series/125879/
State : failure

== Summary ==

=== Applying kernel patches on branch 'drm-xe-next' with base: === Base commit: ff616f1d0 drm/xe/xe2: Program correct MOCS registers === git am output follows ===
error: patch failed: drivers/gpu/drm/xe/xe_bo.c:946
error: drivers/gpu/drm/xe/xe_bo.c: patch does not apply
error: patch failed: include/drm/drm_device.h:326
error: include/drm/drm_device.h: patch does not apply
hint: Use 'git am --show-current-patch' to see the failed patch
Applying: drm/ttm: re-parameter ttm_device_init
Applying: drm: move lru_lock from ttm_device to drm_device Patch failed at 0002 drm: move lru_lock from ttm_device to drm_device When you have resolved this problem, run "git am --continue". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort".

#19 lore raw
In reply to Oak Zeng

Hi Oak, Christian

Quoted 125 lines · Oak Zeng

We plan to implement xe driver's shared virtual memory manager (aka SVM) without buffer object concept. This means we won't build our shared virtual memory manager upon TTM infrastructure like amdgpu does.

Even though this approach is more efficient, it does create a problem for memory eviction when there is memory pressure: memory allocated by SVM and memory allocated by TTM should be able to mutually evict from each other. TTM's resource manager maintains a LRU list for each memory type and this list is used to pick up the memory eviction victim. Since we don't use TTM for SVM implementation, SVM allocated memory can't be added to TTM resource manager's LRU list. Thus SVM allocated memory and TTM allocated memory are not mutually evictable.

See more discussion on this topic here:
https://www.spinics.net/lists/dri-devel/msg410740.html

This series solve this problem by creating a shared LRU list b/t SVM and TTM, or any other resource manager.

The basic idea is, abstract a drm_lru_entity structure which is supposed to be embedded in ttm_resource structure, or any other resource manager. The resource LRU list is a list of drm_lru_entity. drm_lru_entity has eviction function pointers which can be used to call back drivers' specific eviction function to evict a memory resource.

Introduce global drm_lru_manager to struct drm_device to manage LRU lists. Each memory type or memory region can have a LRU list. TTM resource manager's LRU list functions including bulk move functions are moved to drm lru manager. drm lru manager provides a evict_first function to evict the first memory resource from LRU list. This function can be called from TTM, SVM or any other resource manager, so all the memory allocated in the drm sub-system can be mutually evicted.

The lru_lock is also moved from struct ttm_device to struct drm_device.

Opens:
1) memory accounting: currently the ttm resource manager's memory accounting functions is kept at ttm resource manager. Since memory accounting should be cross TTM and SVM, it should be ideally in the drm lru manager layer. This will be polished in the future.

2) eviction callback function interface: The current eviction function interface is designed to meet TTM memory eviction requirements. When SVM is in the picture, this interface need to be futher tunned to meet SVM requirement also.

This series is not tested and it is only compiled for xe driver. Some minor changes are needed for other driver such as amdgpu, nouveau etc. I intended to send this out as a request for comment series to get some early feedback, to see whether this is the right direction to go. I will futher polish this series after a direction is agreed.

Oak Zeng (11):
drm/ttm: re-parameter ttm_device_init
drm: move lru_lock from ttm_device to drm_device
drm: introduce drm evictable LRU
drm: Add evict function pointer to drm lru entity
drm: Replace ttm macros with drm macros
drm/ttm: Set lru manager to ttm resource manager drm/ttm: re-parameterize a few ttm functions
drm: Initialize drm lru manager
drm/ttm: Use drm LRU manager iterator
drm/ttm: Implement ttm memory evict functions drm/ttm: Write ttm functions using drm lru manager functions

  drivers/gpu/drm/Makefile                      |   1 +
  drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c   |   6 +
  .../gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.c   |   6 +
  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |  10 +-
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c        |   6 +-
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h        |   2 +-
  drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c  |  10 +-
  drivers/gpu/drm/drm_drv.c                     |   1 +
  drivers/gpu/drm/drm_evictable_lru.c           | 266 ++++++++++++++++++
  drivers/gpu/drm/drm_gem_vram_helper.c         |  10 +-
  drivers/gpu/drm/i915/gem/i915_gem_ttm.c       |   6 +-
  drivers/gpu/drm/i915/i915_ttm_buddy_manager.c |  10 +
  drivers/gpu/drm/i915/intel_region_ttm.c       |   4 +-
  drivers/gpu/drm/i915/selftests/mock_region.c  |   2 +-
  drivers/gpu/drm/loongson/lsdc_ttm.c           |  10 +-
  drivers/gpu/drm/nouveau/nouveau_ttm.c         |  22 +-
  drivers/gpu/drm/qxl/qxl_ttm.c                 |   6 +-
  drivers/gpu/drm/radeon/radeon_ttm.c           |  10 +-
  drivers/gpu/drm/ttm/tests/ttm_device_test.c   |   2 +-
  drivers/gpu/drm/ttm/tests/ttm_kunit_helpers.c |   2 +-
  drivers/gpu/drm/ttm/ttm_bo.c                  | 247 ++++++++++++----
  drivers/gpu/drm/ttm/ttm_bo_util.c             |  20 +-
  drivers/gpu/drm/ttm/ttm_bo_vm.c               |   2 +-
  drivers/gpu/drm/ttm/ttm_device.c              |  55 ++--
  drivers/gpu/drm/ttm/ttm_module.h              |   3 +-
  drivers/gpu/drm/ttm/ttm_range_manager.c       |  14 +-
  drivers/gpu/drm/ttm/ttm_resource.c            | 242 +++-------------
  drivers/gpu/drm/ttm/ttm_sys_manager.c         |   8 +-
  drivers/gpu/drm/vmwgfx/vmwgfx_bo.c            |   2 +-
  drivers/gpu/drm/vmwgfx/vmwgfx_bo.h            |   2 +-
  drivers/gpu/drm/vmwgfx/vmwgfx_drv.c           |   6 +-
  .../gpu/drm/vmwgfx/vmwgfx_system_manager.c    |   6 +
  drivers/gpu/drm/xe/xe_bo.c                    |  48 ++--
  drivers/gpu/drm/xe/xe_bo.h                    |   5 +-
  drivers/gpu/drm/xe/xe_device.c                |   2 +-
  drivers/gpu/drm/xe/xe_dma_buf.c               |   4 +-
  drivers/gpu/drm/xe/xe_exec.c                  |   6 +-
  drivers/gpu/drm/xe/xe_migrate.c               |   6 +-
  drivers/gpu/drm/xe/xe_res_cursor.h            |  10 +-
  drivers/gpu/drm/xe/xe_ttm_sys_mgr.c           |   8 +-
  drivers/gpu/drm/xe/xe_ttm_vram_mgr.c          |  18 +-
  drivers/gpu/drm/xe/xe_vm.c                    |   6 +-
  drivers/gpu/drm/xe/xe_vm_types.h              |   2 +-
  include/drm/drm_device.h                      |  12 +
  include/drm/drm_evictable_lru.h               | 260 +++++++++++++++++
  include/drm/ttm/ttm_bo.h                      |  10 +-
  include/drm/ttm/ttm_device.h                  |  13 +-
  include/drm/ttm/ttm_range_manager.h           |  17 +-
  include/drm/ttm/ttm_resource.h                | 117 +++-----
  49 files changed, 1042 insertions(+), 501 deletions(-)
Quoted 3 lines

create mode 100644 drivers/gpu/drm/drm_evictable_lru.c create mode 100644 include/drm/drm_evictable_lru.h

Some additional comments after looking through this patchset and the comments.

1) General: IMO a good start.

2) Where should this reside? I'm not a big fan of *prescribing* that a component should be part of a specific device structure. IMO that leads to dumping the whole drm namespace into this component rather than to keep it isolated with as few dependencies as possible. I would make the part that should reside in the drm device a "struct drm_lru_device", and the resource base class a "struct drm_lru_resource". Whether the drm device then wants to embed the struct drm_lru_device (nice if you need this component) or have a pointer to it (nice if you don't need this component) and whether the ttm_device should subclass the drm device becomes pretty orthogonal to the actual implementation, and we'd avoid hard to follow code cross-references all over the place. The drm_gpuvm IMO did a good job with this, but I know there are other opinions on this, and I don't want it to become a blocker.

3) Christian's comment about cursors into the LRU for LRU traversal restart
This is quickly becoming a must for the Xe driver, and a very-nice to have for a working TTM shrinker, but I think we should start to bring what we have in TTM currently over and do the cursors as a separate task. I have it pretty much on top of my priority list currently. Both downstream i915 and drm_gpuvm have a way to handle this nicely by permutating the LRU list just before the lock needs to be released, so I was thinking something similar. The complication would be the bulk move tracking.

4) The struct drm_lru_evict_ctx - This one needs to be common across all users since it's set up by the evictor and used by the evictee ops.

/Thomas