Add a TTM shrinker
Subsystems (8)
Thomas Hellström
reviewercover Add a TTM shrinker
This series introduces a TTM shrinker.
Currently the TTM subsystem allows a certain watermark fraction of system memory to be pinned by GPUs. Any allocation beyond that will cause TTM to attempt to copy memory to shmem objects for possible later swapout so that that fraction is fulfilled. That unnecessarily happens also on systems where swapping is not available, but still works reasonably well in many cases.
However there is no way for the system to swap out all of graphics memory even in situatons where graphics processes are suspended.
So add a TTM shrinker capable of moving graphics memory pages to the swap cache for later laundring and free, and, in the case there is no swap available, freeing graphics memory that is kept around for caching purposes.
For devices where the shrinker is active, the watermark fraction is disabled, but for devices not (yet) supporting shrinking or using dma_alloced memory which we can't insert into the swap-cache, keep it around.
Each driver needs to implement a callback to enable the shrinker for its devices. Enable it for i915 as a POC. Will also be used by the new Intel xe driver if accepted.
The parts of the series mostly needing consideration and feecback is
*) The mm part, inserting pages into the swap-cache. Is it acceptable and,
if so, correct? It *might* be possible we can do without this part, but then we'd have to be able to call read_mapping_page() and trylock_page() on non-isolated shmem pages from reclaim context, and need to be able to recover from failures.
*) The TTM driver callback for shrinking
*) The additional TTM functions to mark buffer-objects as not needed, but
good to have around for caching purposes.
*) Swapin doesn't lose content on error and is also interruptible or at
least killable ATM. This complicates helpers. Should we drop this and just drop content on error, and wait for swapin uninterruptible? The TTM pool code could indeed do without additional complication...
*) Is there a better way to do shrink throttling to avoid filling the
swap-cache completely.
*) Is it good enough for real-world workloads?
The series has been tested using the i915 driver with a 4GiB VRAM DG1 on a system with 14GiB system memory and 16GiB SSD Swap, and using an old igt-gpu-tools version, 8c0bb07b7b4d, of gem_lmem_swapping which overcommits system memory quite extensively
Patch walkthrough:
Initial bugfixes, could be decoupled from the series. drm/ttm: Fix a NULL pointer dereference. drm/ttm/pool: Fix ttm_pool_alloc error path.
Cleanups and restructuring:
drm/ttm: Use the BIT macro for the TTM_TT_FLAGs drm/ttm, drm/vmwgfx: Update the TTM swapout interface drm/ttm: Unexport ttm_global_swapout()
Adding shrinker without enabling it:
drm/ttm: Don't use watermark accounting on shrinkable pools drm/ttm: Reduce the number of used allocation orders for TTM pages drm/ttm: Add a shrinker and shrinker accounting drm/ttm: Introduce shrink throttling
drm/ttm: Remove pinned bos from shrinkable accounting drm/ttm: Add a simple api to set/ clear purgeable ttm_tt content
Adding the core mm part to insert and read-back pages from the swap-cache:
mm: Add interfaces to back up and recover folio contents using swap.
TTM helpers for shrinking:
drm/ttm: Make the call to ttm_tt_populate() interruptible when faulting. drm/ttm: Provide helpers for shrinking.
drm/ttm: Use fault-injection to test error paths.
Enable i915:
drm/i915, drm/ttm: Use the TTM shrinker rather than the external shmem pool
Any feedback greatly appreciated.
Thomas
Thomas Hellström (16):
drm/ttm: Fix a NULL pointer dereference
drm/ttm/pool: Fix ttm_pool_alloc error path
drm/ttm: Use the BIT macro for the TTM_TT_FLAGs
drm/ttm, drm/vmwgfx: Update the TTM swapout interface drm/ttm: Unexport ttm_global_swapout()
drm/ttm: Don't use watermark accounting on shrinkable pools drm/ttm: Reduce the number of used allocation orders for TTM pages drm/ttm: Add a shrinker and shrinker accounting
drm/ttm: Introduce shrink throttling.
drm/ttm: Remove pinned bos from shrinkable accounting drm/ttm: Add a simple api to set / clear purgeable ttm_tt content
mm: Add interfaces to back up and recover folio contents using swap drm/ttm: Make the call to ttm_tt_populate() interruptible when
faulting
drm/ttm: Provide helpers for shrinking
drm/ttm: Use fault-injection to test error paths
drm/i915, drm/ttm: Use the TTM shrinker rather than the external shmem
pool
drivers/gpu/drm/Kconfig | 11 + drivers/gpu/drm/i915/gem/i915_gem_object.h | 6 - .../gpu/drm/i915/gem/i915_gem_object_types.h | 6 - drivers/gpu/drm/i915/gem/i915_gem_pages.c | 5 +- drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 273 ++------- drivers/gpu/drm/i915/i915_gem.c | 3 +- drivers/gpu/drm/ttm/ttm_bo.c | 45 +- drivers/gpu/drm/ttm/ttm_bo_vm.c | 19 +- drivers/gpu/drm/ttm/ttm_device.c | 85 ++- drivers/gpu/drm/ttm/ttm_pool.c | 522 ++++++++++++++++-- drivers/gpu/drm/ttm/ttm_tt.c | 336 +++++++++-- drivers/gpu/drm/vmwgfx/vmwgfx_drv.c | 3 +- include/drm/ttm/ttm_bo.h | 4 +- include/drm/ttm/ttm_device.h | 36 +- include/drm/ttm/ttm_pool.h | 19 + include/drm/ttm/ttm_tt.h | 57 +- include/linux/swap.h | 10 + mm/Kconfig | 18 + mm/Makefile | 2 + mm/swap_backup_folio.c | 178 ++++++ mm/swap_backup_folio_test.c | 111 ++++ 21 files changed, 1361 insertions(+), 388 deletions(-) create mode 100644 mm/swap_backup_folio.c create mode 100644 mm/swap_backup_folio_test.c
Signature · 1 line
2.34.1
Thomas Hellström
reviewerRFC PATCH 1/16 drm/ttm: Fix a NULL pointer dereference
The LRU mechanism may look up a resource in the process of being removed from an object. The locking rules here are a bit unclear but it looks currently like res->bo assignment is protected by the LRU lock, whereas bo->resource is protected by the object lock, while *clearing* of bo->resource is also protected by the LRU lock. This means that if we check that bo->resource points to the LRU resource under the LRU lock we should be safe.
So perform that check before deciding to swap out a bo. That avoids dereferencing a NULL bo->resource in ttm_bo_swapout().
--- drivers/gpu/drm/ttm/ttm_device.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/ttm/ttm_device.c +1 -1
@@ -158,7 +158,7 @@ int ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx,struct ttm_buffer_object *bo = res->bo;uint32_t num_pages; -if (!bo) +if (!bo || bo->resource != res)continue ; num_pages =PFN_UP (bo->base.size);
Signature · 1 line
2.34.1
Thomas Hellström
reviewerRFC PATCH 2/16 drm/ttm/pool: Fix ttm_pool_alloc error path
In reply to Thomas HellströmWhen hitting an error, the error path forgot to unmap dma mappings and could call set_pages_wb() on already uncached pages.
Fix this by introducing a common __ttm_pool_free() function that does the right thing.
--- drivers/gpu/drm/ttm/ttm_pool.c | 74 +++++++++++++++++++++------------- 1 file changed, 45 insertions(+), 29 deletions(-)
drivers/gpu/drm/ttm/ttm_pool.c +45 -29
@@ -367,6 +367,39 @@ static int ttm_pool_page_allocated(struct ttm_pool *pool, unsigned int order,return 0 ; } +static void __ttm_pool_free (struct ttm_pool *pool,struct ttm_tt *tt, +struct page **caching_divide, +enum ttm_caching initial_caching, +enum ttm_caching subseq_caching, +pgoff_t num_pages) +{ +enum ttm_caching caching = subseq_caching; +struct page **pages = tt->pages; +unsigned int order; +pgoff_t i, nr; + +if (pool && caching_divide) + caching = initial_caching; + +for (i =0 ; i < num_pages; i += nr, pages += nr) { +struct ttm_pool_type *pt =NULL ; + +if (unlikely (caching_divide == pages)) + caching = subseq_caching; + + order =ttm_pool_page_order (pool, *pages); + nr = (1UL << order); +if (tt->dma_address) +ttm_pool_unmap (pool, tt->dma_address[i], nr); + + pt =ttm_pool_select_type (pool, caching, order); +if (pt) +ttm_pool_type_give (pt, *pages); +else +ttm_pool_free_page (pool, caching, order, *pages); + } +} +/** * ttm_pool_alloc - Fill a ttm_tt object *
@@ -386,8 +419,9 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,dma_addr_t *dma_addr = tt->dma_address;struct page **caching = tt->pages;struct page **pages = tt->pages; +enum ttm_caching page_caching;gfp_t gfp_flags =GFP_USER ; -unsigned int i, order; +unsigned int order;struct page *p;int r;
@@ -410,6 +444,7 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt, order = min_t(unsigned int, order, __fls(num_pages))) {struct ttm_pool_type *pt; + page_caching = tt->caching; pt =ttm_pool_select_type (pool, tt->caching, order); p = pt ?ttm_pool_type_take (pt) :NULL ;if (p) {
@@ -418,6 +453,7 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,if (r) goto error_free_page; + caching = pages; do { r = ttm_pool_page_allocated(pool, order, p, &dma_addr,
@@ -426,14 +462,15 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,if (r) goto error_free_page; + caching = pages;if (num_pages < (1 << order))break ; p =ttm_pool_type_take (pt); }while (p); - caching = pages; } + page_caching = ttm_cached;while (num_pages >= (1 << order) && (p =ttm_pool_alloc_page (pool, gfp_flags, order))) {
@@ -442,6 +479,7 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt, tt->caching);if (r) goto error_free_page; + caching = pages; } r =ttm_pool_page_allocated (pool, order, p, &dma_addr, &num_pages, &pages);
@@ -468,15 +506,12 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,return 0 ; error_free_page: -ttm_pool_free_page (pool, tt->caching, order, p); +ttm_pool_free_page (pool, page_caching, order, p); error_free_all: num_pages = tt->num_pages - num_pages; -for (i =0 ; i < num_pages; ) { - order =ttm_pool_page_order (pool, tt->pages[i]); -ttm_pool_free_page (pool, tt->caching, order, tt->pages[i]); - i +=1 << order; - } +__ttm_pool_free (pool, tt, caching, tt->caching, ttm_cached, + num_pages);return r; }
@@ -492,27 +527,8 @@ EXPORT_SYMBOL(ttm_pool_alloc);*/ void ttm_pool_free (struct ttm_pool *pool,struct ttm_tt *tt) { -unsigned int i; - -for (i =0 ; i < tt->num_pages; ) { -struct page *p = tt->pages[i]; -unsigned int order, num_pages; -struct ttm_pool_type *pt; - - order =ttm_pool_page_order (pool, p); - num_pages =1ULL << order; -if (tt->dma_address) -ttm_pool_unmap (pool, tt->dma_address[i], num_pages); - - pt =ttm_pool_select_type (pool, tt->caching, order); -if (pt) -ttm_pool_type_give (pt, tt->pages[i]); -else -ttm_pool_free_page (pool, tt->caching, order, - tt->pages[i]); - - i += num_pages; - } +__ttm_pool_free (pool, tt,NULL , tt->caching, tt->caching, + tt->num_pages);while (atomic_long_read (&allocated_pages) > page_pool_size)ttm_pool_shrink ();
Signature · 1 line
2.34.1
Thomas Hellström
reviewerRFC PATCH 3/16 drm/ttm: Use the BIT macro for the TTM_TT_FLAGs
In reply to Thomas HellströmNew code is recommended to use the BIT macro instead of the explicit shifts. Change the older defines so that we can keep the style consistent with upcoming changes.
--- include/drm/ttm/ttm_tt.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
include/drm/ttm/ttm_tt.h +5 -5
@@ -83,12 +83,12 @@ struct ttm_tt {* set by TTM after ttm_tt_populate() has successfully returned, and is * then unset when TTM calls ttm_tt_unpopulate(). */ -#define TTM_TT_FLAG_SWAPPED (1 << 0) -#define TTM_TT_FLAG_ZERO_ALLOC (1 << 1) -#define TTM_TT_FLAG_EXTERNAL (1 << 2) -#define TTM_TT_FLAG_EXTERNAL_MAPPABLE (1 << 3) +#define TTM_TT_FLAG_SWAPPED BIT(0) +#define TTM_TT_FLAG_ZERO_ALLOC BIT(1) +#define TTM_TT_FLAG_EXTERNAL BIT(2) +#define TTM_TT_FLAG_EXTERNAL_MAPPABLE BIT(3) -#define TTM_TT_FLAG_PRIV_POPULATED (1U << 31) +#define TTM_TT_FLAG_PRIV_POPULATED BIT(31)uint32_t page_flags;/** @num_pages: Number of pages in the page array. */ uint32_t num_pages;
Signature · 1 line
2.34.1
Thomas Hellström
reviewerRFC PATCH 4/16 drm/ttm, drm/vmwgfx: Update the TTM swapout interface
In reply to Thomas HellströmUpdate the TTM swapout interfaces for better compatibility with a shrinker.
- Replace number-of-pages int return with a long to better match the kernel's shrinker interface.
- The gfp_flags parameter to ttm_xx_swapout() currently only takes the GFP_KERNEL value and shouldn't really be needed since the shrinker we hook up in upcoming patches sets a allocation context to match reclaim.
- Introduce a shrink reason enumeration and a driver callback to shrink buffer objects.
The TTM_SHRINK_WATERMARK reason is going to still be handled using the existing shmem copy, and will be used by pool types that don't lend themselves well to shinking (dma_alloc pool) and when drivers explicitly requests swapout.
The TTM_SHRINK_SWAP and TTM_SHRINK_PURGE reasons originate from a shrinker and is to be handled by a new driver callback, bo_shrink(). Helpers for the new driver callback are provided in upcoming patches.
--- drivers/gpu/drm/ttm/ttm_bo.c | 38 ++++++++++++++++---- drivers/gpu/drm/ttm/ttm_device.c | 55 +++++++++++++++++++++-------- drivers/gpu/drm/ttm/ttm_tt.c | 23 ++++++------ drivers/gpu/drm/vmwgfx/vmwgfx_drv.c | 3 +- include/drm/ttm/ttm_bo.h | 4 +-- include/drm/ttm/ttm_device.h | 36 +++++++++++++++++-- include/drm/ttm/ttm_tt.h | 17 +++++++-- 7 files changed, 136 insertions(+), 40 deletions(-)
drivers/gpu/drm/ttm/ttm_bo.c +31 -7
@@ -1114,13 +1114,29 @@ int ttm_bo_wait_ctx(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx) }EXPORT_SYMBOL (ttm_bo_wait_ctx); -int ttm_bo_swapout (struct ttm_buffer_object *bo,struct ttm_operation_ctx *ctx, -gfp_t gfp_flags) +/** +* ttm_bo_swapout() - Swap out or purge a buffer object +* @bo: The buffer object. +* @ctx: The ttm operation context. +* @reason: The swapout reason. +* +* Try to swap out or purge the contents of a system memory backed buffer +* object. The function needs to be called with the device's LRU lock held. +* +* Return: -EBUSY if the bo lock could not be grabbed or the object was +* otherwise busy. Otherwise the number of pages swapped out or negative +* error code on error. Iff the function didn't return -EBUSY, the +* LRU lock was dropped, and LRU traversal needs to restart. +*/ +long ttm_bo_swapout (struct ttm_buffer_object *bo,struct ttm_operation_ctx *ctx, +enum ttm_shrink_reason reason) {struct ttm_place place;bool locked;long ret; +lockdep_assert_held (&bo->bdev->lru_lock); +/* * While the bo may already reside in SYSTEM placement, set * SYSTEM as new placement to cover also the move further below.
@@ -1142,8 +1158,12 @@ int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx, }if (bo->deleted) { +long num_pages = bo->ttm->num_pages; + ret =ttm_bo_cleanup_refs (bo, false, false, locked);ttm_bo_put (bo); +if (!ret) +return num_pages;return ret == -EBUSY ? -ENOSPC : ret; }
@@ -1184,13 +1204,17 @@ int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx,* Swap out. Buffer will be swapped in again as soon as * anyone tries to access a ttm page. */ -if (bo->bdev->funcs->swap_notify) - bo->bdev->funcs->swap_notify (bo); +if (bo->bdev->funcs->bo_shrink && reason !=TTM_SHRINK_WATERMARK ) { + ret = bo->bdev->funcs->bo_shrink (bo, ctx); + }else { +if (bo->bdev->funcs->swap_notify) + bo->bdev->funcs->swap_notify (bo); + ret =ttm_tt_swapout (bo->bdev, bo->ttm); +if (!ret) + ret = bo->ttm->num_pages; + } -if (ttm_tt_is_populated (bo->ttm)) - ret =ttm_tt_swapout (bo->bdev, bo->ttm, gfp_flags); out: -/* * Unreserve without putting on LRU to avoid swapping out an * already swapped buffer.
drivers/gpu/drm/ttm/ttm_device.c +41 -14
90 lines not on this page · raw message
drivers/gpu/drm/ttm/ttm_tt.c +12 -11
@@ -239,22 +239,21 @@ int ttm_tt_swapin(struct ttm_tt *ttm)/** * ttm_tt_swapout - swap out tt object -* * @bdev: TTM device structure. * @ttm: The struct ttm_tt. -* @gfp_flags: Flags to use for memory allocation. * -* Swapout a TT object to a shmem_file, return number of pages swapped out or -* negative error code. +* Swapout a TT object to a shmem_file. +* +* Return: number of pages swapped out or negative error code on error. */ -int ttm_tt_swapout (struct ttm_device *bdev,struct ttm_tt *ttm, -gfp_t gfp_flags) +int ttm_tt_swapout (struct ttm_device *bdev,struct ttm_tt *ttm) {loff_t size = (loff_t )ttm->num_pages <<PAGE_SHIFT ;struct address_space *swap_space;struct file *swap_storage;struct page *from_page;struct page *to_page; +gfp_t gfp_flags;int i, ret; swap_storage =shmem_file_setup ("ttm swap" , size,0 );
@@ -264,7 +263,7 @@ int ttm_tt_swapout(struct ttm_device *bdev, struct ttm_tt *ttm, } swap_space = swap_storage->f_mapping; - gfp_flags &=mapping_gfp_mask (swap_space); + gfp_flags =GFP_KERNEL &mapping_gfp_mask (swap_space);for (i =0 ; i < ttm->num_pages; ++i) { from_page = ttm->pages[i];
@@ -315,12 +314,14 @@ int ttm_tt_populate(struct ttm_device *bdev,while (atomic_long_read (&ttm_pages_allocated) > ttm_pages_limit ||atomic_long_read (&ttm_dma32_pages_allocated) > ttm_dma32_pages_limit) { +long r =ttm_global_swapout (ctx,TTM_SHRINK_WATERMARK ); - ret =ttm_global_swapout (ctx,GFP_KERNEL ); -if (ret ==0 ) +if (!r)break ; -if (ret <0 ) +if (r <0 ) { + ret = r; goto error; + } }if (bdev->funcs->ttm_tt_populate)
@@ -379,7 +380,7 @@ static int ttm_tt_debugfs_shrink_show(struct seq_file *m, void *data) {struct ttm_operation_ctx ctx = { false, false }; -seq_printf (m,"%d\n" ,ttm_global_swapout (&ctx,GFP_KERNEL )); +seq_printf (m,"%ld\n" ,ttm_global_swapout (&ctx,TTM_SHRINK_SWAP ));return 0 ; }DEFINE_SHOW_ATTRIBUTE (ttm_tt_debugfs_shrink);
drivers/gpu/drm/vmwgfx/vmwgfx_drv.c +2 -1
@@ -1514,7 +1514,8 @@ static int vmw_pm_freeze(struct device *kdev)vmw_execbuf_release_pinned_bo (dev_priv);vmw_resource_evict_all (dev_priv);vmw_release_device_early (dev_priv); -while (ttm_device_swapout (&dev_priv->bdev, &ctx,GFP_KERNEL ) >0 ); +while (ttm_device_swapout (&dev_priv->bdev, &ctx,TTM_SHRINK_WATERMARK ) >0 ) + ;vmw_fifo_resource_dec (dev_priv);if (atomic_read (&dev_priv->num_fifo_resources) !=0 ) {DRM_ERROR ("Can't hibernate while 3D resources are active.\n" );
include/drm/ttm/ttm_bo.h +2 -2
@@ -375,8 +375,8 @@ void ttm_bo_kunmap(struct ttm_bo_kmap_obj *map);int ttm_bo_vmap (struct ttm_buffer_object *bo,struct iosys_map *map);void ttm_bo_vunmap (struct ttm_buffer_object *bo,struct iosys_map *map);int ttm_bo_mmap_obj (struct vm_area_struct *vma,struct ttm_buffer_object *bo); -int ttm_bo_swapout (struct ttm_buffer_object *bo,struct ttm_operation_ctx *ctx, -gfp_t gfp_flags); +long ttm_bo_swapout (struct ttm_buffer_object *bo,struct ttm_operation_ctx *ctx, +enum ttm_shrink_reason reason);void ttm_bo_pin (struct ttm_buffer_object *bo);void ttm_bo_unpin (struct ttm_buffer_object *bo); int ttm_mem_evict_first(struct ttm_device *bdev,
include/drm/ttm/ttm_device.h +33 -3
@@ -35,6 +35,21 @@ struct ttm_placement;struct ttm_buffer_object ;struct ttm_operation_ctx ; +/** +* enum ttm_shrink_reason - Reason for shrinking system memory +* @TTM_SHRINK_WATERMARK - A watermark limit was reached. Not from reclaim. +* @TTM_SHRINK_PURGE - A request for shrinking only purged objects. +* @TTM_SHRINK_SWAP - A request for shrinking any object. +* +* This enum is intended for the buffer object- and shrink method selection +* algorithms. It's not intended to leak to or be used by TTM drivers. +*/ +enum ttm_shrink_reason { +TTM_SHRINK_WATERMARK , +TTM_SHRINK_PURGE , +TTM_SHRINK_SWAP , +}; +/** * struct ttm_global - Buffer object driver global data. */
@@ -207,6 +222,19 @@ struct ttm_device_funcs {* adding fences that may force a delayed delete */ void (*release_notify)(struct ttm_buffer_object *bo); + +/** +* Shrink the bo's system pages, Either by swapping or by purging. +* @bo: Bo the system pages of which are to be shrunken. +* @ctx: Operation ctx. In particular the driver callback should +* adhere to the no_wait_gpu and interruptible fields. +* +* This is also notifying the driver that the bo is about to be +* shrunken and the driver should take care to unbind any GPU bindings +* and to note that the content is purged if @bo->ttm is purgeable. +*/ +long (*bo_shrink)(struct ttm_buffer_object *bo, +struct ttm_operation_ctx *ctx); };/**
@@ -268,9 +296,11 @@ struct ttm_device {struct workqueue_struct *wq; }; -int ttm_global_swapout (struct ttm_operation_ctx *ctx,gfp_t gfp_flags); -int ttm_device_swapout (struct ttm_device *bdev,struct ttm_operation_ctx *ctx, -gfp_t gfp_flags); +long ttm_global_swapout (struct ttm_operation_ctx *ctx, +enum ttm_shrink_reason reason); + +long ttm_device_swapout (struct ttm_device *bdev,struct ttm_operation_ctx *ctx, +enum ttm_shrink_reason reason);static inline struct ttm_resource_manager *ttm_manager_type (struct ttm_device *bdev,int mem_type)
include/drm/ttm/ttm_tt.h +15 -2
@@ -87,6 +87,7 @@ struct ttm_tt {#define TTM_TT_FLAG_ZERO_ALLOC BIT(1)#define TTM_TT_FLAG_EXTERNAL BIT(2)#define TTM_TT_FLAG_EXTERNAL_MAPPABLE BIT(3) +#define TTM_TT_FLAG_DONTNEED BIT(4)#define TTM_TT_FLAG_PRIV_POPULATED BIT(31)uint32_t page_flags;
@@ -180,8 +181,8 @@ void ttm_tt_destroy(struct ttm_device *bdev, struct ttm_tt *ttm);* Swap in a previously swap out ttm_tt. */ int ttm_tt_swapin (struct ttm_tt *ttm); -int ttm_tt_swapout (struct ttm_device *bdev,struct ttm_tt *ttm, -gfp_t gfp_flags); + +int ttm_tt_swapout (struct ttm_device *bdev,struct ttm_tt *ttm);/** * ttm_tt_populate - allocate pages for a ttm
@@ -223,6 +224,18 @@ void ttm_tt_mgr_init(unsigned long num_pages, unsigned long num_dma32_pages);struct ttm_kmap_iter *ttm_kmap_iter_tt_init (struct ttm_kmap_iter_tt *iter_tt,struct ttm_tt *tt); +/** +* ttm_tt_purgeable() - Whether a struct ttm_tt's contents is purgeable +* @tt: The struct ttm_tt to consider. +* +* Return: Whether the contents is purgeable in the sence that the owner +* doesn't mind losing it as long as it gets notified. +*/ +static inline bool ttm_tt_purgeable (struct ttm_tt *tt) +{ +return tt->page_flags &TTM_TT_FLAG_DONTNEED ; +} +#if IS_ENABLED (CONFIG_AGP )#include <linux/agp_backend.h>
Signature · 1 line
2.34.1
Thomas Hellström
reviewerRFC PATCH 5/16 drm/ttm: Unexport ttm_global_swapout()
In reply to Thomas HellströmUnexport ttm_global_swapout() since it is not used outside of TTM.
--- drivers/gpu/drm/ttm/ttm_device.c | 1 - 1 file changed, 1 deletion(-)
drivers/gpu/drm/ttm/ttm_device.c +0 -1
@@ -146,7 +146,6 @@ long ttm_global_swapout(struct ttm_operation_ctx *ctx,mutex_unlock (&ttm_global_mutex);return ret; } -EXPORT_SYMBOL (ttm_global_swapout);/** * ttm_device_swapout() - Select and swap out a system-memory-backed bo.
Signature · 1 line
2.34.1
Thomas Hellström
reviewerRFC PATCH 6/16 drm/ttm: Don't use watermark accounting on shrinkable pools
In reply to Thomas HellströmClarify the meaning of the ttm_tt pages_limit watermarks as the max number of pages not accessible by shrinkers, and update accordingly so that memory allocated by TTM devices that support shrinking is not accounted against those limits. In particular this means that devices using the dma_alloc pool will still be using the watermark method.
--- drivers/gpu/drm/ttm/ttm_device.c | 3 ++- drivers/gpu/drm/ttm/ttm_tt.c | 43 +++++++++++++++++++------------- include/drm/ttm/ttm_pool.h | 15 +++++++++++ 3 files changed, 42 insertions(+), 19 deletions(-)
drivers/gpu/drm/ttm/ttm_device.c +2 -1
@@ -168,7 +168,8 @@ long ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx,unsigned i;long ret; -if (reason !=TTM_SHRINK_WATERMARK && !bdev->funcs->bo_shrink) +if (reason !=TTM_SHRINK_WATERMARK && + (!bdev->funcs->bo_shrink || !ttm_pool_can_shrink (&bdev->pool)))return 0 ;spin_lock (&bdev->lru_lock);
drivers/gpu/drm/ttm/ttm_tt.c +25 -18
67 lines not on this page · raw message
include/drm/ttm/ttm_pool.h +15 -0
@@ -89,4 +89,19 @@ int ttm_pool_debugfs(struct ttm_pool *pool, struct seq_file *m);int ttm_pool_mgr_init (unsigned long num_pages);void ttm_pool_mgr_fini (void ); +/** +* ttm_pool_can_shrink - Whether page allocations from this pool are shrinkable +* @pool: The pool. +* +* Return: true if shrinkable, false if not. +*/ +static inline bool ttm_pool_can_shrink (const struct ttm_pool *pool) +{ +/* +* The dma_alloc pool pages can't be inserted into the +* swap cache. Nor can they be split. +*/ +return !pool->use_dma_alloc; +} +#endif
Signature · 1 line
2.34.1
Thomas Hellström
reviewerRFC PATCH 7/16 drm/ttm: Reduce the number of used allocation orders for TTM pages
In reply to Thomas HellströmWhen swapping out, we will split multi-order pages both in order to move them to the swap-cache and to be able to return memory to the swap cache as soon as possible on a page-by-page basis.
By reducing the page max order to the system PMD size, we can be nicer to the system and avoid splitting gigantic pages. On top of this we also include the 64K page size in the page sizes tried, since that appears to be a common size for GPU applications.
Looking forward to when we might be able to swap out PMD size folios without splitting, this will also be a benefit.
--- drivers/gpu/drm/ttm/ttm_pool.c | 58 ++++++++++++++++++++++++++-------- 1 file changed, 45 insertions(+), 13 deletions(-)
drivers/gpu/drm/ttm/ttm_pool.c +45 -13
@@ -31,6 +31,8 @@* cause they are rather slow compared to alloc_pages+map. */ +#define pr_fmt (fmt) "[TTM POOL] " fmt +#include <linux/module.h> #include <linux/dma-mapping.h> #include <linux/debugfs.h>
@@ -47,6 +49,18 @@#include "ttm_module.h" +#define TTM_MAX_ORDER (PMD_SHIFT - PAGE_SHIFT) +#define TTM_64K_ORDER (16 - PAGE_SHIFT) +#if (TTM_MAX_ORDER <TTM_64K_ORDER ) +#undef TTM_MAX_ORDER +#define TTM_MAX_ORDER TTM_64K_ORDER +#endif +#if ((MAX_ORDER -1 ) <TTM_MAX_ORDER ) +#undef TTM_MAX_ORDER +#define TTM_MAX_ORDER (MAX_ORDER - 1) +#endif +#define TTM_DIM_ORDER (TTM_MAX_ORDER + 1) +/** * struct ttm_pool_dma - Helper object for coherent DMA mappings *
@@ -65,16 +79,18 @@ module_param(page_pool_size, ulong, 0644);static atomic_long_t allocated_pages; -static struct ttm_pool_type global_write_combined[MAX_ORDER ]; -static struct ttm_pool_type global_uncached[MAX_ORDER ]; +static struct ttm_pool_type global_write_combined[TTM_DIM_ORDER ]; +static struct ttm_pool_type global_uncached[TTM_DIM_ORDER ]; -static struct ttm_pool_type global_dma32_write_combined[MAX_ORDER ]; -static struct ttm_pool_type global_dma32_uncached[MAX_ORDER ]; +static struct ttm_pool_type global_dma32_write_combined[TTM_DIM_ORDER ]; +static struct ttm_pool_type global_dma32_uncached[TTM_DIM_ORDER ];static spinlock_t shrinker_lock;static struct list_head shrinker_list;static struct shrinker mm_shrinker; +static unsigned int ttm_pool_orders[] = {TTM_MAX_ORDER ,0 ,0 }; +/* Allocate pages of size 1 << order with the given gfp_flags */ static struct page *ttm_pool_alloc_page (struct ttm_pool *pool,gfp_t gfp_flags,unsigned int order)
@@ -400,6 +416,17 @@ static void __ttm_pool_free(struct ttm_pool *pool, struct ttm_tt *tt, } } +static unsigned int ttm_pool_select_order (unsigned int order,pgoff_t num_pages) +{ +unsigned int *cur_order = ttm_pool_orders; + + order = min_t(unsigned int, __fls(num_pages), order); +while (order < *cur_order) + ++cur_order; + +return *cur_order; +} +/** * ttm_pool_alloc - Fill a ttm_tt object *
@@ -439,9 +466,8 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,else gfp_flags |=GFP_HIGHUSER ; - for (order = min_t(unsigned int, MAX_ORDER - 1, __fls(num_pages)); - num_pages; - order = min_t(unsigned int, order, __fls(num_pages))) { + order =ttm_pool_select_order (ttm_pool_orders[0 ], num_pages); +for (; num_pages; order =ttm_pool_select_order (order, num_pages)) {struct ttm_pool_type *pt; page_caching = tt->caching;
@@ -558,7 +584,7 @@ void ttm_pool_init(struct ttm_pool *pool, struct device *dev,if (use_dma_alloc) {for (i =0 ; i <TTM_NUM_CACHING_TYPES ; ++i) -for (j =0 ; j <MAX_ORDER ; ++j) +for (j =0 ; j <TTM_DIM_ORDER ; ++j)ttm_pool_type_init (&pool->caching[i].orders[j], pool, i, j); }
@@ -578,7 +604,7 @@ void ttm_pool_fini(struct ttm_pool *pool)if (pool->use_dma_alloc) {for (i =0 ; i <TTM_NUM_CACHING_TYPES ; ++i) -for (j =0 ; j <MAX_ORDER ; ++j) +for (j =0 ; j <TTM_DIM_ORDER ; ++j)ttm_pool_type_fini (&pool->caching[i].orders[j]); }
@@ -632,7 +658,7 @@ static void ttm_pool_debugfs_header(struct seq_file *m)unsigned int i;seq_puts (m,"\t " ); -for (i =0 ; i <MAX_ORDER ; ++i) +for (i =0 ; i <TTM_DIM_ORDER ; ++i)seq_printf (m," ---%2u---" , i);seq_puts (m,"\n" ); }
@@ -643,7 +669,7 @@ static void ttm_pool_debugfs_orders(struct ttm_pool_type *pt, {unsigned int i; -for (i =0 ; i <MAX_ORDER ; ++i) +for (i =0 ; i <TTM_DIM_ORDER ; ++i)seq_printf (m," %8u" ,ttm_pool_type_count (&pt[i]));seq_puts (m,"\n" ); }
@@ -749,10 +775,16 @@ int ttm_pool_mgr_init(unsigned long num_pages)if (!page_pool_size) page_pool_size = num_pages; +if (TTM_64K_ORDER <TTM_MAX_ORDER ) + ttm_pool_orders[1 ] =TTM_64K_ORDER ; + +pr_debug ("Used orders are %u %u %u\n" , ttm_pool_orders[0 ], + ttm_pool_orders[1 ], ttm_pool_orders[2 ]); +spin_lock_init (&shrinker_lock);INIT_LIST_HEAD (&shrinker_list); -for (i =0 ; i <MAX_ORDER ; ++i) { +for (i =0 ; i <TTM_DIM_ORDER ; ++i) {ttm_pool_type_init (&global_write_combined[i],NULL , ttm_write_combined, i);ttm_pool_type_init (&global_uncached[i],NULL , ttm_uncached, i);
@@ -785,7 +817,7 @@ void ttm_pool_mgr_fini(void) {unsigned int i; -for (i =0 ; i <MAX_ORDER ; ++i) { +for (i =0 ; i <TTM_DIM_ORDER ; ++i) {ttm_pool_type_fini (&global_write_combined[i]);ttm_pool_type_fini (&global_uncached[i]);
Signature · 1 line
2.34.1
Thomas Hellström
reviewerRFC PATCH 8/16 drm/ttm: Add a shrinker and shrinker accounting
In reply to Thomas HellströmRegister a TTM system memory-backed object shrinker and add accounting for shrinkable and purgeable pages. For the shrinker to work, the driver needs to register the bo_shrink callback which is responsible for unbinding from GPU and the dma layer if needed. Helpers for that callback to actually perform shrinking will be introduced in upcoming patches.
Note that we can't lock the ttm_global_mutex from within the shrinker scan() function as that might cause a deadlock issue. To fix that, add and use a mutex which is used for global device list manipulation only and make sure it isn't held when registering the shrinker.
--- drivers/gpu/drm/ttm/ttm_device.c | 26 ++++--- drivers/gpu/drm/ttm/ttm_tt.c | 112 +++++++++++++++++++++++++++++-- include/drm/ttm/ttm_tt.h | 2 + 3 files changed, 125 insertions(+), 15 deletions(-)
drivers/gpu/drm/ttm/ttm_device.c +15 -11
@@ -36,10 +36,10 @@#include "ttm_module.h" -/* -* ttm_global_mutex - protecting the global state -*/ +/* ttm_global_mutex - protects the global state init and fini. */ static DEFINE_MUTEX (ttm_global_mutex); +/* ttm_global_list_mutex - protects the device list. */ +static DEFINE_MUTEX (ttm_global_list_mutex);static unsigned ttm_glob_use_count;struct ttm_global ttm_glob;EXPORT_SYMBOL (ttm_glob);
@@ -54,6 +54,7 @@ static void ttm_global_release(void)if (--ttm_glob_use_count >0 ) goto out; +ttm_tt_mgr_fini ();ttm_pool_mgr_fini ();debugfs_remove (ttm_debugfs_root);
@@ -102,7 +103,10 @@ static int ttm_global_init(void) goto out; } +mutex_lock (&ttm_global_list_mutex);INIT_LIST_HEAD (&glob->device_list); +mutex_unlock (&ttm_global_list_mutex); +atomic_set (&glob->bo_count,0 ); debugfs_create_atomic_t("buffer_objects", 0444, ttm_debugfs_root,
@@ -135,7 +139,7 @@ long ttm_global_swapout(struct ttm_operation_ctx *ctx,struct ttm_device *bdev;long ret =0 ; -mutex_lock (&ttm_global_mutex); +mutex_lock (&ttm_global_list_mutex);list_for_each_entry (bdev, &glob->device_list, device_list) { ret =ttm_device_swapout (bdev, ctx, reason);if (ret >0 ) {
@@ -143,7 +147,7 @@ long ttm_global_swapout(struct ttm_operation_ctx *ctx,break ; } } -mutex_unlock (&ttm_global_mutex); +mutex_unlock (&ttm_global_list_mutex);return ret; }
@@ -247,9 +251,9 @@ int ttm_device_init(struct ttm_device *bdev, struct ttm_device_funcs *funcs,spin_lock_init (&bdev->lru_lock);INIT_LIST_HEAD (&bdev->pinned); bdev->dev_mapping = mapping; -mutex_lock (&ttm_global_mutex); +mutex_lock (&ttm_global_list_mutex);list_add_tail (&bdev->device_list, &glob->device_list); -mutex_unlock (&ttm_global_mutex); +mutex_unlock (&ttm_global_list_mutex);return 0 ; }
@@ -260,14 +264,14 @@ void ttm_device_fini(struct ttm_device *bdev)struct ttm_resource_manager *man;unsigned i; +mutex_lock (&ttm_global_list_mutex); +list_del (&bdev->device_list); +mutex_unlock (&ttm_global_list_mutex); + man =ttm_manager_type (bdev,TTM_PL_SYSTEM );ttm_resource_manager_set_used (man, false);ttm_set_driver_manager (bdev,TTM_PL_SYSTEM ,NULL ); -mutex_lock (&ttm_global_mutex); -list_del (&bdev->device_list); -mutex_unlock (&ttm_global_mutex); -drain_workqueue (bdev->wq);destroy_workqueue (bdev->wq);
drivers/gpu/drm/ttm/ttm_tt.c +108 -4
155 lines not on this page · raw message
include/drm/ttm/ttm_tt.h +2 -0
@@ -221,6 +221,8 @@ static inline void ttm_tt_mark_for_clear(struct ttm_tt *ttm)void ttm_tt_mgr_init (unsigned long num_pages,unsigned long num_dma32_pages); +void ttm_tt_mgr_fini (void ); +struct ttm_kmap_iter *ttm_kmap_iter_tt_init (struct ttm_kmap_iter_tt *iter_tt,struct ttm_tt *tt);
Signature · 1 line
2.34.1
Thomas Hellström
reviewerRFC PATCH 9/16 drm/ttm: Introduce shrink throttling.
In reply to Thomas HellströmSince pages are not immediately freed by the TTM shrinker but rather inserted into the swap cache, the system will keep on calling the shrinker rapidly filling the swap cache which has a negative impact on system performance.
When shrinking, throttle on the number of pages present in the swap cache.
--- drivers/gpu/drm/ttm/ttm_tt.c | 40 ++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+)
drivers/gpu/drm/ttm/ttm_tt.c +40 -0
@@ -432,6 +432,42 @@ static unsigned long ttm_tt_shrinker_count(struct shrinker *shrink,return num_pages ? num_pages :SHRINK_EMPTY ; } +#define TTM_SWAP_MIN_SWAP_PAGES (SZ_128M >> PAGE_SHIFT) +#define TTM_SWAP_MAX_SWAPCACHE_PAGES (SZ_1G >> PAGE_SHIFT) +static unsigned long ttm_tt_shrinker_throttle (unsigned long pages) +{ +unsigned long + tmp =get_nr_swap_pages (); + +/* +* Draining available swap space too far will trigger +* systemd-oomd even if there are a huge number of dirty pages +* available for laundry and free in the swap cache. Don't drain +* the available swap-space too far. +*/ +if (tmp >TTM_SWAP_MIN_SWAP_PAGES ) + tmp -=TTM_SWAP_MIN_SWAP_PAGES ; +else + tmp =0 ; + + pages =min (tmp, pages); + +/* +* Our shrinker doesn't immediately free pages unless they belong +* to purgeable objects. Rather they are inserted into the swap-cache. +* But the system doesn't really get this and continues to call our +* shrinker thinking it's still out of memory, when it could just +* laundry pages in the swap cache and free them. So throttle on the +* number of pages in the swap cache. +*/ + + tmp =total_swapcache_pages (); +if (tmp >TTM_SWAP_MAX_SWAPCACHE_PAGES ) + pages =0 ; + +return pages; +} +static unsigned long ttm_tt_shrinker_scan (struct shrinker *shrink,struct shrink_control *sc) {
@@ -459,6 +495,10 @@ static unsigned long ttm_tt_shrinker_scan(struct shrinker *shrink, nr_to_scan -= freed; else nr_to_scan =0 ; + +if (nr_to_scan) + nr_to_scan =ttm_tt_shrinker_throttle (nr_to_scan); +if (!nr_to_scan)return freed ? freed :SHRINK_STOP ;
Signature · 1 line
2.34.1
Thomas Hellström
reviewerRFC PATCH 10/16 drm/ttm: Remove pinned bos from shrinkable accounting
In reply to Thomas HellströmPinned bos aren't shinkable and needs to be removed from the shrinkable accounting. Do that, and in the process constify the tt argument to ttm_tt_is_populated.
--- drivers/gpu/drm/ttm/ttm_bo.c | 7 +++++++ drivers/gpu/drm/ttm/ttm_tt.c | 22 ++++++++++++++++++++++ include/drm/ttm/ttm_tt.h | 6 +++++- 3 files changed, 34 insertions(+), 1 deletion(-)
drivers/gpu/drm/ttm/ttm_bo.c +7 -0
@@ -650,6 +650,10 @@ void ttm_bo_pin(struct ttm_buffer_object *bo) {dma_resv_assert_held (bo->base.resv);WARN_ON_ONCE (!kref_read (&bo->kref)); + +if (!bo->pin_count && bo->ttm) +ttm_tt_set_pinned (bo->bdev, bo->ttm); +spin_lock (&bo->bdev->lru_lock);if (bo->resource)ttm_resource_del_bulk_move (bo->resource, bo);
@@ -671,6 +675,9 @@ void ttm_bo_unpin(struct ttm_buffer_object *bo)if (WARN_ON_ONCE (!bo->pin_count))return ; +if (bo->pin_count ==1 && bo->ttm) +ttm_tt_set_unpinned (bo->bdev, bo->ttm); +spin_lock (&bo->bdev->lru_lock); --bo->pin_count;if (bo->resource)
drivers/gpu/drm/ttm/ttm_tt.c +22 -0
@@ -83,6 +83,28 @@ static void ttm_tt_mod_shrinkable_pages(long shrinkable, long purgeable)write_unlock (&shrinkable_lock); } +/** +* ttm_tt_set_pinned() - Modify the shinkable accounting when pinning a bo. +* @bdev: The TTM device. +* @tt: The struct tt_tt used by the pinned bo. +*/ +void ttm_tt_set_pinned (const struct ttm_device *bdev,const struct ttm_tt *tt) +{ +if (ttm_tt_shrinkable (bdev, tt) &&ttm_tt_is_populated (tt)) +ttm_tt_mod_shrinkable_pages (-(long )tt->num_pages,0 ); +} + +/** +* ttm_tt_set_unpinned() - Modify the shinkable accounting when unpinning a bo. +* @bdev: The TTM device. +* @tt: The struct tt_tt used by the no longer pinned bo. +*/ +void ttm_tt_set_unpinned (const struct ttm_device *bdev,const struct ttm_tt *tt) +{ +if (ttm_tt_shrinkable (bdev, tt) &&ttm_tt_is_populated (tt)) +ttm_tt_mod_shrinkable_pages (tt->num_pages,0 ); +} +/* * Allocates a ttm structure for the given BO. */
include/drm/ttm/ttm_tt.h +5 -1
@@ -118,7 +118,7 @@ struct ttm_kmap_iter_tt {pgprot_t prot; }; -static inline bool ttm_tt_is_populated (struct ttm_tt *tt) +static inline bool ttm_tt_is_populated (const struct ttm_tt *tt) {return tt->page_flags &TTM_TT_FLAG_PRIV_POPULATED ; }
@@ -238,6 +238,10 @@ static inline bool ttm_tt_purgeable(struct ttm_tt *tt)return tt->page_flags &TTM_TT_FLAG_DONTNEED ; } +void ttm_tt_set_pinned (const struct ttm_device *bdev,const struct ttm_tt *tt); + +void ttm_tt_set_unpinned (const struct ttm_device *bdev,const struct ttm_tt *tt); +#if IS_ENABLED (CONFIG_AGP )#include <linux/agp_backend.h>
Signature · 1 line
2.34.1
Thomas Hellström
reviewerRFC PATCH 11/16 drm/ttm: Add a simple api to set / clear purgeable ttm_tt content
In reply to Thomas HellströmIn the absence of free swap space, a shrinker could still efficiently free memory the content of which is no longer needed, and graphics drivers typically has an interface to mark buffer object content as no longer needed.
Add a possibility to propagate this to TTM, so that the shrinker accounting and shrinker actions can be updated accordingly.
Moving forward, we will probably want this interface on the bo level and have bo move support for it, but for now we strictly only need it for the shrinker. Another option would be to have the drivers do the purgeable vs shrinkable accounting.
This still leaves the responsibility to the driver to assign proper LRU priority to purgeable buffer object so that the shrinker finds those objects early during LRU traversal.
--- drivers/gpu/drm/ttm/ttm_tt.c | 59 ++++++++++++++++++++++++++++++++++++ include/drm/ttm/ttm_tt.h | 3 ++ 2 files changed, 62 insertions(+)
drivers/gpu/drm/ttm/ttm_tt.c +59 -0
@@ -105,6 +105,65 @@ void ttm_tt_set_unpinned(const struct ttm_device *bdev, const struct ttm_tt *tt)ttm_tt_mod_shrinkable_pages (tt->num_pages,0 ); } +/** +* ttm_tt_set_dontneed() - Mark ttm_tt content as not needed. +* @bdev: The ttm device. +* @tt: The struct ttm_tt. +* +* Mark the ttm_tt content as not needed for the shrinker accounting. +* This also means that the content will not be backed up on shrinking, +* but rather freed immediately. +* +* Return: 0 if successful, -EALREADY if content was never present or +* already backed up and was purged by this call. +*/ +int ttm_tt_set_dontneed (const struct ttm_device *bdev,struct ttm_tt *tt) +{ +if (ttm_tt_is_populated (tt)) { +if (!ttm_tt_purgeable (tt)) { + tt->page_flags |=TTM_TT_FLAG_DONTNEED ; +if (ttm_tt_shrinkable (bdev, tt)) +ttm_tt_mod_shrinkable_pages (-(long )tt->num_pages, + tt->num_pages); + } +return 0 ; + } + +if (tt->swap_storage) +fput (tt->swap_storage); + tt->swap_storage =NULL ; + +return -EALREADY ; +} +EXPORT_SYMBOL (ttm_tt_set_dontneed); + +/** +* ttm_tt_set_willneed() - Mark tt_tt content as needed. +* @bdev: The ttm device. +* @tt: The struct ttm_tt. +* +* Mark the ttm_tt content as needed and update the shrinker accounting +* accordingly. +* +* Return: 0 if successful, -EALREADY if content was never present or +* was already purged. +*/ +int ttm_tt_set_willneed (const struct ttm_device *bdev,struct ttm_tt *tt) +{ +if (ttm_tt_is_populated (tt)) { +if (ttm_tt_purgeable (tt)) { + tt->page_flags &= ~TTM_TT_FLAG_DONTNEED ; +if (ttm_tt_shrinkable (bdev, tt)) +ttm_tt_mod_shrinkable_pages (tt->num_pages, + -(long )tt->num_pages); + } +return 0 ; + } + +return -EALREADY ; +} +EXPORT_SYMBOL (ttm_tt_set_willneed); +/* * Allocates a ttm structure for the given BO. */
include/drm/ttm/ttm_tt.h +3 -0
@@ -241,6 +241,9 @@ static inline bool ttm_tt_purgeable(struct ttm_tt *tt)void ttm_tt_set_pinned (const struct ttm_device *bdev,const struct ttm_tt *tt);void ttm_tt_set_unpinned (const struct ttm_device *bdev,const struct ttm_tt *tt); +int ttm_tt_set_dontneed (const struct ttm_device *bdev,struct ttm_tt *tt); + +int ttm_tt_set_willneed (const struct ttm_device *bdev,struct ttm_tt *tt);#if IS_ENABLED (CONFIG_AGP )#include <linux/agp_backend.h>
Signature · 1 line
2.34.1
Thomas Hellström
reviewerRFC PATCH 12/16 mm: Add interfaces to back up and recover folio contents using swap
In reply to Thomas HellströmGPU drivers have traditionally used shmem to back up GPU buffer contents for swap on physical memory shortage. Some integrated GPU drivers use shmem files as the backing storage for their GPU buffers, other drivers, in particular drivers that need a Write-Combining caching strategy on system pages, (but also drivers for discrete gpus in general) need to copy to shmem on anticipated memory shortage.
The latter strategy does not lend itself very well to shrinker usage, since shmem memory needs to be allocated and page trylocking of pagecache pages need to be performed from reclaim context and both are prone to failures. That makes the approach very fragile at best.
Add interfaces for GPU drivers to directly insert pages into the swap-cache, thereby bypassing shmem and avoiding the shmem page allocation and locking at shrink time completely, as well as the content copy.
Also add a kunit test for experimenting with the interface functionality, currently it seems PMD size folios doesn't work properly. Needs further investigation if this is a viable approach.
--- include/linux/swap.h | 10 ++ mm/Kconfig | 18 ++++ mm/Makefile | 2 + mm/swap_backup_folio.c | 178 ++++++++++++++++++++++++++++++++++++ mm/swap_backup_folio_test.c | 111 ++++++++++++++++++++++ 5 files changed, 319 insertions(+) create mode 100644 mm/swap_backup_folio.c create mode 100644 mm/swap_backup_folio_test.c
include/linux/swap.h +10 -0
@@ -706,5 +706,15 @@ static inline bool mem_cgroup_swap_full(struct folio *folio) }#endif +#ifdef CONFIG_SWAP_BACKUP_FOLIO +swp_entry_t swap_backup_folio (struct folio *folio,bool writeback, +gfp_t folio_gfp,gfp_t alloc_gfp); + +int swap_copy_folio (swp_entry_t swap,struct page *page,unsigned long index, +bool killable); + +void swap_drop_folio (swp_entry_t swap); +#endif +#endif /* __KERNEL__*/ #endif /* _LINUX_SWAP_H */
mm/Kconfig +18 -0
@@ -191,6 +191,10 @@ config ZSMALLOC_STAT information to userspace via debugfs. If unsure, say N. +config SWAP_BACKUP_FOLIO + bool + default n + menu "SLAB allocator options" choice
@@ -1183,6 +1187,20 @@ config LRU_GEN_STATS This option has a per-memcg and per-node memory overhead. # } +config SWAP_BACKUP_FOLIO_KUNIT_TEST + tristate "KUnit tests for swap_backup_folio() functionality" if !KUNIT_ALL_TESTS + depends on SWAP && KUNIT && SWAP_BACKUP_FOLIO + help + This builds unit tests for the swap_backup_folio_functionality(). + This option is not useful for distributions or general kernels, + but only for kernel developers working on MM swap functionality. + + For more information on KUnit and unit tests in general, + please refer to the KUnit documentation in + Documentation/dev-tools/kunit/. + + If in doubt, say "N". + source "mm/damon/Kconfig" endmenu
mm/Makefile +2 -0
@@ -138,3 +138,5 @@ obj-$(CONFIG_IO_MAPPING) += io-mapping.o obj-$(CONFIG_HAVE_BOOTMEM_INFO_NODE) += bootmem_info.o obj-$(CONFIG_GENERIC_IOREMAP) += ioremap.o obj-$(CONFIG_SHRINKER_DEBUG) += shrinker_debug.o +obj-$(CONFIG_SWAP_BACKUP_FOLIO) += swap_backup_folio.o +obj-$(CONFIG_SWAP_BACKUP_FOLIO_KUNIT_TEST) += swap_backup_folio_test.o
mm/swap_backup_folio.c new file +178 -0
178 lines not on this page · raw message
mm/swap_backup_folio_test.c new file +111 -0
111 lines not on this page · raw message
Signature · 1 line
2.34.1
Thomas Hellström
reviewerRFC PATCH 13/16 drm/ttm: Make the call to ttm_tt_populate() interruptible when faulting
In reply to Thomas HellströmWhen swapping in, or under memory pressure ttm_tt_populate() may sleep for a substantiable amount of time. Allow interrupts during the sleep. This will also allow us to inject -EINTR errors during swapin in upcoming patches.
Also avoid returning VM_FAULT_OOM, since that will confuse the core mm, making it print out a confused message and retrying the fault. Return VM_FAULT_SIGBUS also under OOM conditions.
--- drivers/gpu/drm/ttm/ttm_bo_vm.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)
drivers/gpu/drm/ttm/ttm_bo_vm.c +10 -3
@@ -218,14 +218,21 @@ vm_fault_t ttm_bo_vm_fault_reserved(struct vm_fault *vmf, prot =ttm_io_prot (bo, bo->resource, prot);if (!bo->resource->bus.is_iomem) {struct ttm_operation_ctx ctx = { - .interruptible = false, + .interruptible = true, .no_wait_gpu = false, .force_alloc = true }; ttm = bo->ttm; -if (ttm_tt_populate (bdev, bo->ttm, &ctx)) -return VM_FAULT_OOM ; + err =ttm_tt_populate (bdev, bo->ttm, &ctx); +if (err) { +if (err == -EINTR || err == -ERESTARTSYS || + err == -EAGAIN ) +return VM_FAULT_NOPAGE ; + +pr_debug ("TTM fault hit %pe.\n" ,ERR_PTR (err)); +return VM_FAULT_SIGBUS ; + } }else {/* Iomem should not be marked encrypted */ prot =pgprot_decrypted (prot);
Signature · 1 line
2.34.1
Thomas Hellström
reviewerRFC PATCH 14/16 drm/ttm: Provide helpers for shrinking
In reply to Thomas HellströmProvide a helper to be used by the driver bo_shrink() callback to either insert the pages of a struct ttm_tt into the swap-cache or to purge them if the struct ttm_tt is purgeable. For pages with write-combined or uncached linear kernel map, that linear kernel map is first changed to cached.
Release pages with as little intermediate memory allocation as possible, however some memory might be allocated during swapout for the swap space radix tree.
Due to swapout- or swapin errors, allow partially swapped out struct ttm_tt's, although mark them as swapped out stopping them from being swapped out a second time. More details in the ttm_pool.c DOC section.
--- drivers/gpu/drm/Kconfig | 1 + drivers/gpu/drm/ttm/ttm_pool.c | 403 +++++++++++++++++++++++++++++++-- drivers/gpu/drm/ttm/ttm_tt.c | 34 +++ include/drm/ttm/ttm_pool.h | 4 + include/drm/ttm/ttm_tt.h | 10 + 5 files changed, 437 insertions(+), 15 deletions(-)
drivers/gpu/drm/Kconfig +1 -0
@@ -196,6 +196,7 @@ source "drivers/gpu/drm/display/Kconfig" config DRM_TTM tristate depends on DRM && MMU + select SWAP_BACKUP_FOLIO help GPU memory management subsystem for devices with multiple GPU memory types. Will be enabled automatically if a device driver
drivers/gpu/drm/ttm/ttm_pool.c +388 -15
509 lines not on this page · raw message
drivers/gpu/drm/ttm/ttm_tt.c +34 -0
@@ -133,6 +133,8 @@ int ttm_tt_set_dontneed(const struct ttm_device *bdev, struct ttm_tt *tt)fput (tt->swap_storage); tt->swap_storage =NULL ; +ttm_pool_release_shrunken (tt); +return -EALREADY ; }EXPORT_SYMBOL (ttm_tt_set_dontneed);
@@ -253,6 +255,7 @@ static void ttm_tt_init_fields(struct ttm_tt *ttm, ttm->swap_storage =NULL ; ttm->sg = bo->sg; ttm->caching = caching; + ttm->restore =NULL ; } int ttm_tt_init(struct ttm_tt *ttm, struct ttm_buffer_object *bo,
@@ -277,6 +280,8 @@ void ttm_tt_fini(struct ttm_tt *ttm)fput (ttm->swap_storage); ttm->swap_storage =NULL ; +ttm_pool_release_shrunken (ttm); +if (ttm->pages)kvfree (ttm->pages);else
@@ -347,6 +352,35 @@ int ttm_tt_swapin(struct ttm_tt *ttm)return ret; } +/** +* ttm_tt_shrink() - Helper for the driver bo_shrink() method. +* @bdev: The TTM device. +* @tt: The struct ttm_tt. +* +* Helper for a TTM driver to use from the bo_shrink() method to shrink +* a struct ttm_tt, after it has done the necessary unbinding. This function +* will update the page accounting and call ttm_pool_shrink_tt to free pages +* or move them to the swap cache. +* +* Return: Number of pages freed or swapped out, or negative error code on +* error. +*/ +long ttm_tt_shrink (struct ttm_device *bdev,struct ttm_tt *tt) +{ +long ret =ttm_pool_shrink_tt (&bdev->pool, tt); + +if (ret >0 ) { + tt->page_flags &= ~TTM_TT_FLAG_PRIV_POPULATED ; +if (ttm_tt_purgeable (tt)) +ttm_tt_mod_shrinkable_pages (0 , -(long )tt->num_pages); +else +ttm_tt_mod_shrinkable_pages (-(long )tt->num_pages,0 ); + } + +return ret; +} +EXPORT_SYMBOL (ttm_tt_shrink); +/** * ttm_tt_swapout - swap out tt object * @bdev: TTM device structure.
include/drm/ttm/ttm_pool.h +4 -0
@@ -86,6 +86,10 @@ void ttm_pool_fini(struct ttm_pool *pool);int ttm_pool_debugfs (struct ttm_pool *pool,struct seq_file *m); +void ttm_pool_release_shrunken (struct ttm_tt *tt); + +long ttm_pool_shrink_tt (struct ttm_pool *pool,struct ttm_tt *ttm); +int ttm_pool_mgr_init (unsigned long num_pages);void ttm_pool_mgr_fini (void );
include/drm/ttm/ttm_tt.h +10 -0
@@ -37,6 +37,7 @@ struct ttm_tt;struct ttm_resource ;struct ttm_buffer_object ;struct ttm_operation_ctx ; +struct ttm_pool_tt_restore ;/** * struct ttm_tt - This is a structure holding the pages, caching- and aperture
@@ -79,6 +80,10 @@ struct ttm_tt {* page_flags = TTM_TT_FLAG_EXTERNAL | * TTM_TT_FLAG_EXTERNAL_MAPPABLE; * +* TTM_TT_FLAG_PRIV_SHRUNKEN: TTM internal only. This is set if the +* struct ttm_tt has been (possibly partially) swapped out to the +* swap cache. +* * TTM_TT_FLAG_PRIV_POPULATED: TTM internal only. DO NOT USE. This is * set by TTM after ttm_tt_populate() has successfully returned, and is * then unset when TTM calls ttm_tt_unpopulate().
@@ -89,6 +94,7 @@ struct ttm_tt {#define TTM_TT_FLAG_EXTERNAL_MAPPABLE BIT(3)#define TTM_TT_FLAG_DONTNEED BIT(4) +#define TTM_TT_FLAG_PRIV_SHRUNKEN BIT(30)#define TTM_TT_FLAG_PRIV_POPULATED BIT(31)uint32_t page_flags;/** @num_pages: Number of pages in the page array. */
@@ -104,6 +110,8 @@ struct ttm_tt {* ttm_caching. */ enum ttm_caching caching; +/** @restore: Swap restore state. Drivers keep off. */ +struct ttm_pool_tt_restore *restore; };/**
@@ -226,6 +234,8 @@ void ttm_tt_mgr_fini(void);struct ttm_kmap_iter *ttm_kmap_iter_tt_init (struct ttm_kmap_iter_tt *iter_tt,struct ttm_tt *tt); +long ttm_tt_shrink (struct ttm_device *bdev,struct ttm_tt *tt); +/** * ttm_tt_purgeable() - Whether a struct ttm_tt's contents is purgeable * @tt: The struct ttm_tt to consider.
Signature · 1 line
2.34.1
Thomas Hellström
reviewerRFC PATCH 15/16 drm/ttm: Use fault-injection to test error paths
In reply to Thomas HellströmUse fault-injection to test partial TTM swapout and interrupted swapin. Return -EINTR for swapin to test the callers ability to handle and restart the swapin, and on swapout perform a partial swapout to test that the swapin and release_shrunken functionality.
--- drivers/gpu/drm/Kconfig | 10 ++++++++++ drivers/gpu/drm/ttm/ttm_pool.c | 17 ++++++++++++++++- 2 files changed, 26 insertions(+), 1 deletion(-)
drivers/gpu/drm/Kconfig +10 -0
@@ -202,6 +202,16 @@ config DRM_TTM GPU memory types. Will be enabled automatically if a device driver uses it. +config DRM_TTM_SHRINK_FAULT_INJECT + bool "Enable fault injection during TTM shrinking" + depends on DRM_TTM + default n + help + Inject recoverable failures during TTM shrinking and recovery of + shrunken objects. For DRM driver developers only. + + If in doubt, choose N. + config DRM_BUDDY tristate depends on DRM
drivers/gpu/drm/ttm/ttm_pool.c +16 -1
@@ -453,6 +453,7 @@ static bool ttm_pool_restore_valid(const struct ttm_pool_tt_restore *restore)static int ttm_pool_swapin (struct ttm_pool_tt_restore *restore,struct ttm_operation_ctx *ctx) { +static unsigned long __maybe_unused swappedin;unsigned int i, nr =1 << restore->order;int ret =0 ;
@@ -468,6 +469,13 @@ static int ttm_pool_swapin(struct ttm_pool_tt_restore *restore,if (swap.val ==0 )continue ; +if (IS_ENABLED (CONFIG_DRM_TTM_SHRINK_FAULT_INJECT ) && + ctx->interruptible && + ++swappedin %100 ==0 ) { + ret = -EINTR ; +break ; + } + ret =swap_copy_folio (swap, restore->first_page[i],0 , ctx->interruptible);if (ret)
@@ -905,7 +913,14 @@ long ttm_pool_shrink_tt(struct ttm_pool *pool, struct ttm_tt *ttm)if (current_is_kswapd ()) alloc_gfp |= __GFP_NOMEMALLOC; -for (i =0 ; i < ttm->num_pages; ++i) { + num_pages = ttm->num_pages; + +/* Pretend doing fault injection by shrinking only half of the pages. */ + +if (IS_ENABLED (CONFIG_DRM_TTM_SHRINK_FAULT_INJECT )) + num_pages =DIV_ROUND_UP (num_pages,2 ); + +for (i =0 ; i < num_pages; ++i) { page = ttm->pages[i];if (unlikely (!page))continue ;
Signature · 1 line
2.34.1
Thomas Hellström
reviewerRFC PATCH 16/16 drm/i915, drm/ttm: Use the TTM shrinker rather than the external shmem pool
In reply to Thomas HellströmRemove the external i915 TTM shmem pool and replace it with the normal TTM page allocation. Also provide a callback for the TTM shrinker functionality.
--- drivers/gpu/drm/i915/gem/i915_gem_object.h | 6 - .../gpu/drm/i915/gem/i915_gem_object_types.h | 6 - drivers/gpu/drm/i915/gem/i915_gem_pages.c | 5 +- drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 273 +++--------------- drivers/gpu/drm/i915/i915_gem.c | 3 +- drivers/gpu/drm/ttm/ttm_bo_vm.c | 6 +- drivers/gpu/drm/ttm/ttm_tt.c | 3 - include/drm/ttm/ttm_tt.h | 15 +- 8 files changed, 53 insertions(+), 264 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_object.h +0 -6
@@ -282,12 +282,6 @@ i915_gem_object_is_shrinkable(const struct drm_i915_gem_object *obj)return i915_gem_object_type_has (obj,I915_GEM_OBJECT_IS_SHRINKABLE ); } -static inline bool -i915_gem_object_has_self_managed_shrink_list (const struct drm_i915_gem_object *obj) -{ -return i915_gem_object_type_has (obj,I915_GEM_OBJECT_SELF_MANAGED_SHRINK_LIST ); -} -static inline bool i915_gem_object_is_proxy (const struct drm_i915_gem_object *obj) {
drivers/gpu/drm/i915/gem/i915_gem_object_types.h +0 -6
@@ -544,12 +544,6 @@ struct drm_i915_gem_object {*/ atomic_t shrink_pin; -/** -* @ttm_shrinkable: True when the object is using shmem pages -* underneath. Protected by the object lock. -*/ -bool ttm_shrinkable; -/** * @unknown_state: Indicate that the object is effectively * borked. This is write-once and set if we somehow encounter a
drivers/gpu/drm/i915/gem/i915_gem_pages.c +2 -3
@@ -73,7 +73,7 @@ void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj, shrinkable = false; } -if (shrinkable && !i915_gem_object_has_self_managed_shrink_list (obj)) { +if (shrinkable) {struct list_head *list;unsigned long flags;
@@ -216,8 +216,7 @@ __i915_gem_object_unset_pages(struct drm_i915_gem_object *obj)if (i915_gem_object_is_volatile (obj)) obj->mm.madv =I915_MADV_WILLNEED ; -if (!i915_gem_object_has_self_managed_shrink_list (obj)) -i915_gem_object_make_unshrinkable (obj); +i915_gem_object_make_unshrinkable (obj);if (obj->mm.mapping) {unmap_object (obj,page_mask_bits (obj->mm.mapping));
drivers/gpu/drm/i915/gem/i915_gem_ttm.c +47 -226
386 lines not on this page · raw message
drivers/gpu/drm/i915/i915_gem.c +1 -2
@@ -1068,8 +1068,7 @@ i915_gem_madvise_ioctl(struct drm_device *dev, void *data, obj->ops->adjust_lru (obj); } -if (i915_gem_object_has_pages (obj) || -i915_gem_object_has_self_managed_shrink_list (obj)) { +if (i915_gem_object_has_pages (obj)) {unsigned long flags;spin_lock_irqsave (&i915->mm.obj_lock, flags);
drivers/gpu/drm/ttm/ttm_bo_vm.c +2 -4
@@ -150,10 +150,8 @@ vm_fault_t ttm_bo_vm_reserve(struct ttm_buffer_object *bo,* (if at all) by redirecting mmap to the exporter. */ if (bo->ttm && (bo->ttm->page_flags &TTM_TT_FLAG_EXTERNAL )) { -if (!(bo->ttm->page_flags &TTM_TT_FLAG_EXTERNAL_MAPPABLE )) { -dma_resv_unlock (bo->base.resv); -return VM_FAULT_SIGBUS ; - } +dma_resv_unlock (bo->base.resv); +return VM_FAULT_SIGBUS ; }return 0 ;
drivers/gpu/drm/ttm/ttm_tt.c +0 -3
@@ -198,9 +198,6 @@ int ttm_tt_create(struct ttm_buffer_object *bo, bool zero_alloc)if (unlikely (bo->ttm ==NULL ))return -ENOMEM ; -WARN_ON (bo->ttm->page_flags &TTM_TT_FLAG_EXTERNAL_MAPPABLE && - !(bo->ttm->page_flags &TTM_TT_FLAG_EXTERNAL )); -return 0 ; }
include/drm/ttm/ttm_tt.h +1 -14
@@ -68,18 +68,6 @@ struct ttm_tt {* Note that enum ttm_bo_type.ttm_bo_type_sg objects will always enable * this flag. * -* TTM_TT_FLAG_EXTERNAL_MAPPABLE: Same behaviour as -* TTM_TT_FLAG_EXTERNAL, but with the reduced restriction that it is -* still valid to use TTM to map the pages directly. This is useful when -* implementing a ttm_tt backend which still allocates driver owned -* pages underneath(say with shmem). -* -* Note that since this also implies TTM_TT_FLAG_EXTERNAL, the usage -* here should always be: -* -* page_flags = TTM_TT_FLAG_EXTERNAL | -* TTM_TT_FLAG_EXTERNAL_MAPPABLE; -* * TTM_TT_FLAG_PRIV_SHRUNKEN: TTM internal only. This is set if the * struct ttm_tt has been (possibly partially) swapped out to the * swap cache.
@@ -91,8 +79,7 @@ struct ttm_tt {#define TTM_TT_FLAG_SWAPPED BIT(0)#define TTM_TT_FLAG_ZERO_ALLOC BIT(1)#define TTM_TT_FLAG_EXTERNAL BIT(2) -#define TTM_TT_FLAG_EXTERNAL_MAPPABLE BIT(3) -#define TTM_TT_FLAG_DONTNEED BIT(4) +#define TTM_TT_FLAG_DONTNEED BIT(3)#define TTM_TT_FLAG_PRIV_SHRUNKEN BIT(30)#define TTM_TT_FLAG_PRIV_POPULATED BIT(31)
Signature · 1 line
2.34.1
Re: [RFC PATCH 01/16] drm/ttm: Fix a NULL pointer dereference
In reply to Thomas HellströmAm 15.02.23 um 17:13 schrieb Thomas Hellström:
Quoted 20 lines
The LRU mechanism may look up a resource in the process of being removed from an object. The locking rules here are a bit unclear but it looks currently like
res->boassignment is protected by the LRU lock, whereasbo->resourceis protected by the object lock, while *clearing* ofbo->resourceis also protected by the LRU lock. This means that if we check thatbo->resourcepoints to the LRU resource under the LRU lock we should be safe.
So perform that check before deciding to swap out a bo. That avoids dereferencing a NULLbo->resourceinttm_bo_swapout().Fixes 6a9b02899402 ("drm/ttm: move the LRU into resource handling v4")Cc Christian König <[email protected]>Cc Daniel Vetter <[email protected]>Cc Christian Koenig <[email protected]>Cc Huang Rui <[email protected]>Cc Alex Deucher <[email protected]>Cc Felix Kuehling <[email protected]>Cc Philip Yang <[email protected]>Cc Qiang Yu <[email protected]>Cc Matthew Auld <[email protected]>
Quoted 8 lines
Cc Nirmoy Das <[email protected]>Cc Tvrtko Ursulin <[email protected]>Cc "Thomas Hellström" <[email protected]>Cc Anshuman Gupta <[email protected]>Cc Ramalingam C <[email protected]>Cc Arunpravin Paneer Selvam <[email protected]>Signed-off-by Thomas Hellström <[email protected]>
Quoted 17 lines
--- drivers/gpu/drm/ttm/ttm_device.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/ttm/ttm_device.c +1
-1@@ -158,7 +158,7 @@ int ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx,struct ttm_buffer_object *bo = res->bo;uint32_t num_pages; -if (!bo) +if (!bo || bo->resource != res)continue ; num_pages =PFN_UP (bo->base.size);
Re: [RFC PATCH 02/16] drm/ttm/pool: Fix ttm_pool_alloc error path
In reply to Thomas HellströmAm 15.02.23 um 17:13 schrieb Thomas Hellström:
Quoted 1 line
When hitting an error, the error path forgot to unmap dma mappings and
I don't see where this happens?
Quoted 1 line
could call
set_pages_wb()on already uncached pages.
Yeah, but what's the problem?
Regards,
Christian.
Quoted 161 lines
Fix this by introducing a common
__ttm_pool_free()function that does the right thing.Fixes d099fc8f540a ("drm/ttm: new TT backend allocation pool v3")Cc Christian König <[email protected]>Cc Dave Airlie <[email protected]>Cc Madhav Chauhan <[email protected]>Cc Christian Koenig <[email protected]>Cc Huang Rui <[email protected]>Signed-off-by Thomas Hellström <[email protected]>--- drivers/gpu/drm/ttm/ttm_pool.c | 74 +++++++++++++++++++++------------- 1 file changed, 45 insertions(+), 29 deletions(-)drivers/gpu/drm/ttm/ttm_pool.c +45
-29@@ -367,6 +367,39 @@ static int ttm_pool_page_allocated(struct ttm_pool *pool, unsigned int order,return 0 ; } +static void __ttm_pool_free (struct ttm_pool *pool,struct ttm_tt *tt, +struct page **caching_divide, +enum ttm_caching initial_caching, +enum ttm_caching subseq_caching, +pgoff_t num_pages) +{ +enum ttm_caching caching = subseq_caching; +struct page **pages = tt->pages; +unsigned int order; +pgoff_t i, nr; + +if (pool && caching_divide) + caching = initial_caching; + +for (i =0 ; i < num_pages; i += nr, pages += nr) { +struct ttm_pool_type *pt =NULL ; + +if (unlikely (caching_divide == pages)) + caching = subseq_caching; + + order =ttm_pool_page_order (pool, *pages); + nr = (1UL << order); +if (tt->dma_address) +ttm_pool_unmap (pool, tt->dma_address[i], nr); + + pt =ttm_pool_select_type (pool, caching, order); +if (pt) +ttm_pool_type_give (pt, *pages); +else +ttm_pool_free_page (pool, caching, order, *pages); + } +} +/** * ttm_pool_alloc - Fill a ttm_tt object * @@ -386,8 +419,9 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,dma_addr_t *dma_addr = tt->dma_address;struct page **caching = tt->pages;struct page **pages = tt->pages; +enum ttm_caching page_caching;gfp_t gfp_flags =GFP_USER ; -unsigned int i, order; +unsigned int order;struct page *p;int r;@@ -410,6 +444,7 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt, order = min_t(unsigned int, order, __fls(num_pages))) {struct ttm_pool_type *pt; + page_caching = tt->caching; pt =ttm_pool_select_type (pool, tt->caching, order); p = pt ?ttm_pool_type_take (pt) :NULL ;if (p) {@@ -418,6 +453,7 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,if (r) goto error_free_page; + caching = pages; do { r = ttm_pool_page_allocated(pool, order, p, &dma_addr,@@ -426,14 +462,15 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,if (r) goto error_free_page; + caching = pages;if (num_pages < (1 << order))break ; p =ttm_pool_type_take (pt); }while (p); - caching = pages; } + page_caching = ttm_cached;while (num_pages >= (1 << order) && (p =ttm_pool_alloc_page (pool, gfp_flags, order))) {@@ -442,6 +479,7 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt, tt->caching);if (r) goto error_free_page; + caching = pages; } r =ttm_pool_page_allocated (pool, order, p, &dma_addr, &num_pages, &pages);@@ -468,15 +506,12 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,return 0 ; error_free_page: -ttm_pool_free_page (pool, tt->caching, order, p); +ttm_pool_free_page (pool, page_caching, order, p); error_free_all: num_pages = tt->num_pages - num_pages; -for (i =0 ; i < num_pages; ) { - order =ttm_pool_page_order (pool, tt->pages[i]); -ttm_pool_free_page (pool, tt->caching, order, tt->pages[i]); - i +=1 << order; - } +__ttm_pool_free (pool, tt, caching, tt->caching, ttm_cached, + num_pages);return r; }@@ -492,27 +527,8 @@ EXPORT_SYMBOL(ttm_pool_alloc);*/ void ttm_pool_free (struct ttm_pool *pool,struct ttm_tt *tt) { -unsigned int i; - -for (i =0 ; i < tt->num_pages; ) { -struct page *p = tt->pages[i]; -unsigned int order, num_pages; -struct ttm_pool_type *pt; - - order =ttm_pool_page_order (pool, p); - num_pages =1ULL << order; -if (tt->dma_address) -ttm_pool_unmap (pool, tt->dma_address[i], num_pages); - - pt =ttm_pool_select_type (pool, tt->caching, order); -if (pt) -ttm_pool_type_give (pt, tt->pages[i]); -else -ttm_pool_free_page (pool, tt->caching, order, - tt->pages[i]); - - i += num_pages; - } +__ttm_pool_free (pool, tt,NULL , tt->caching, tt->caching, + tt->num_pages);while (atomic_long_read (&allocated_pages) > page_pool_size)ttm_pool_shrink ();
Re: [RFC PATCH 03/16] drm/ttm: Use the BIT macro for the TTM_TT_FLAGs
In reply to Thomas HellströmAm 15.02.23 um 17:13 schrieb Thomas Hellström:
Quoted 18 lines
New code is recommended to use the BIT macro instead of the explicit shifts. Change the older defines so that we can keep the style consistent with upcoming changes.
Signed-off-by Thomas Hellström <[email protected]>--- include/drm/ttm/ttm_tt.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)include/drm/ttm/ttm_tt.h +0
-1@@ -83,12 +83,12 @@ struct ttm_tt {* set by TTM after ttm_tt_populate() has successfully returned, and is * then unset when TTM calls ttm_tt_unpopulate(). */ -#define TTM_TT_FLAG_SWAPPED (1 << 0)
include/drm/ttm/ttm_tt.h struct ttm_tt { · line 91
Quoted 10 lines
include/drm/ttm/ttm_tt.h +5
-4-#define TTM_TT_FLAG_ZERO_ALLOC (1 << 1) -#define TTM_TT_FLAG_EXTERNAL (1 << 2) -#define TTM_TT_FLAG_EXTERNAL_MAPPABLE (1 << 3) +#define TTM_TT_FLAG_SWAPPED BIT(0) +#define TTM_TT_FLAG_ZERO_ALLOC BIT(1) +#define TTM_TT_FLAG_EXTERNAL BIT(2) +#define TTM_TT_FLAG_EXTERNAL_MAPPABLE BIT(3) -#define TTM_TT_FLAG_PRIV_POPULATED (1U << 31) +#define TTM_TT_FLAG_PRIV_POPULATED BIT(31)
While at it please just use BIT(4) for this, there is actually nothing special about it.
Christian.
Quoted 3 lines
include/drm/ttm/ttm_tt.h +0
-0uint32_t page_flags;/** @num_pages: Number of pages in the page array. */ uint32_t num_pages;
Re: [RFC PATCH 04/16] drm/ttm, drm/vmwgfx: Update the TTM swapout interface
In reply to Thomas HellströmAm 15.02.23 um 17:13 schrieb Thomas Hellström:
Quoted 9 lines
Update the TTM swapout interfaces for better compatibility with a shrinker.
- Replace number-of-pages int return with a long to better match the kernel's shrinker interface.
- Thegfp_flagsparameter tottm_xx_swapout()currently only takes theGFP_KERNELvalue and shouldn't really be needed since the shrinker we hook up in upcoming patches sets a allocation context to match reclaim.- Introduce a shrink reason enumeration and a driver callback to shrink buffer objects.
Is that really necessary? This is mid-layering once more.
If drivers want to implement driver specific shrinking they should register their own shrinker callback.
Christian.
Quoted 389 lines
The
TTM_SHRINK_WATERMARKreason is going to still be handled using the existing shmem copy, and will be used by pool types that don't lend themselves well to shinking (dma_allocpool) and when drivers explicitly requests swapout.
TheTTM_SHRINK_SWAPandTTM_SHRINK_PURGEreasons originate from a shrinker and is to be handled by a new driver callback,bo_shrink(). Helpers for the new driver callback are provided in upcoming patches.Signed-off-by Thomas Hellström <[email protected]>--- drivers/gpu/drm/ttm/ttm_bo.c | 38 ++++++++++++++++---- drivers/gpu/drm/ttm/ttm_device.c | 55 +++++++++++++++++++++-------- drivers/gpu/drm/ttm/ttm_tt.c | 23 ++++++------ drivers/gpu/drm/vmwgfx/vmwgfx_drv.c | 3 +- include/drm/ttm/ttm_bo.h | 4 +-- include/drm/ttm/ttm_device.h | 36 +++++++++++++++++-- include/drm/ttm/ttm_tt.h | 17 +++++++-- 7 files changed, 136 insertions(+), 40 deletions(-)drivers/gpu/drm/ttm/ttm_bo.c +31
-7@@ -1114,13 +1114,29 @@ int ttm_bo_wait_ctx(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx) }EXPORT_SYMBOL (ttm_bo_wait_ctx); -int ttm_bo_swapout (struct ttm_buffer_object *bo,struct ttm_operation_ctx *ctx, -gfp_t gfp_flags) +/** +* ttm_bo_swapout() - Swap out or purge a buffer object +* @bo: The buffer object. +* @ctx: The ttm operation context. +* @reason: The swapout reason. +* +* Try to swap out or purge the contents of a system memory backed buffer +* object. The function needs to be called with the device's LRU lock held. +* +* Return: -EBUSY if the bo lock could not be grabbed or the object was +* otherwise busy. Otherwise the number of pages swapped out or negative +* error code on error. Iff the function didn't return -EBUSY, the +* LRU lock was dropped, and LRU traversal needs to restart. +*/ +long ttm_bo_swapout (struct ttm_buffer_object *bo,struct ttm_operation_ctx *ctx, +enum ttm_shrink_reason reason) {struct ttm_place place;bool locked;long ret; +lockdep_assert_held (&bo->bdev->lru_lock); +/* * While the bo may already reside in SYSTEM placement, set * SYSTEM as new placement to cover also the move further below. @@ -1142,8 +1158,12 @@ int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx, }if (bo->deleted) { +long num_pages = bo->ttm->num_pages; + ret =ttm_bo_cleanup_refs (bo, false, false, locked);ttm_bo_put (bo); +if (!ret) +return num_pages;return ret == -EBUSY ? -ENOSPC : ret; }@@ -1184,13 +1204,17 @@ int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx,* Swap out. Buffer will be swapped in again as soon as * anyone tries to access a ttm page. */ -if (bo->bdev->funcs->swap_notify) - bo->bdev->funcs->swap_notify (bo); +if (bo->bdev->funcs->bo_shrink && reason !=TTM_SHRINK_WATERMARK ) { + ret = bo->bdev->funcs->bo_shrink (bo, ctx); + }else { +if (bo->bdev->funcs->swap_notify) + bo->bdev->funcs->swap_notify (bo); + ret =ttm_tt_swapout (bo->bdev, bo->ttm); +if (!ret) + ret = bo->ttm->num_pages; + } -if (ttm_tt_is_populated (bo->ttm)) - ret =ttm_tt_swapout (bo->bdev, bo->ttm, gfp_flags); out: -/* * Unreserve without putting on LRU to avoid swapping out an * already swapped buffer. drivers/gpu/drm/ttm/ttm_device.c +41
-14@@ -116,19 +116,28 @@ static int ttm_global_init(void)return ret; } -/* -* A buffer object shrink method that tries to swap out the first -* buffer object on the global::swap_lru list. +/** +* ttm_global_swapout() - Select and swap out a system-memory-backed bo. +* @ctx: The operation context. +* @reason: The reason for swapout. +* +* Select, based on round-robin a TTM device and traverse the LRUs of +* that specific device until a suitable bo backed by system memory is found +* and swapped-out or purged. +* +* Return: Positive value or zero indicating the size in pages of the +* bo swapped out. Negative error code on error. */ -int ttm_global_swapout (struct ttm_operation_ctx *ctx,gfp_t gfp_flags) +long ttm_global_swapout (struct ttm_operation_ctx *ctx, +enum ttm_shrink_reason reason) {struct ttm_global *glob = &ttm_glob;struct ttm_device *bdev; -int ret =0 ; +long ret =0 ;mutex_lock (&ttm_global_mutex);list_for_each_entry (bdev, &glob->device_list, device_list) { - ret =ttm_device_swapout (bdev, ctx, gfp_flags); + ret =ttm_device_swapout (bdev, ctx, reason);if (ret >0 ) {list_move_tail (&bdev->device_list, &glob->device_list);break ;@@ -139,14 +148,29 @@ int ttm_global_swapout(struct ttm_operation_ctx *ctx, gfp_t gfp_flags) }EXPORT_SYMBOL (ttm_global_swapout); -int ttm_device_swapout (struct ttm_device *bdev,struct ttm_operation_ctx *ctx, -gfp_t gfp_flags) +/** +* ttm_device_swapout() - Select and swap out a system-memory-backed bo. +* @bdev: The device whos bos are considered for swapout. +* @ctx: The operation context. +* @reason: The reason for swapout. +* +* Traverse the LRUs of a specific device until a suitable bo backed by +* system memory is found and swapped-out or purged. +* +* Return: Positive value or zero indicating the size in pages of the +* bo swapped out. Negative error code on error. +*/ +long ttm_device_swapout (struct ttm_device *bdev,struct ttm_operation_ctx *ctx, +enum ttm_shrink_reason reason) {struct ttm_resource_cursor cursor;struct ttm_resource_manager *man;struct ttm_resource *res;unsigned i; -int ret; +long ret; + +if (reason !=TTM_SHRINK_WATERMARK && !bdev->funcs->bo_shrink) +return 0 ;spin_lock (&bdev->lru_lock);for (i =TTM_PL_SYSTEM ; i <TTM_NUM_MEM_TYPES ; ++i) {@@ -156,16 +180,19 @@ int ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx,ttm_resource_manager_for_each_res (man, &cursor, res) {struct ttm_buffer_object *bo = res->bo; -uint32_t num_pages; +struct ttm_tt *tt;if (!bo || bo->resource != res)continue ; - num_pages =PFN_UP (bo->base.size); - ret =ttm_bo_swapout (bo, ctx, gfp_flags); + tt = bo->ttm; +if (!tt || (reason ==TTM_SHRINK_PURGE && + !ttm_tt_purgeable (tt))) +continue ; + ret =ttm_bo_swapout (bo, ctx, reason);/* ttm_bo_swapout has dropped the lru_lock */ -if (!ret) -return num_pages; +if (ret >=0 ) +return ret;if (ret != -EBUSY )return ret; }drivers/gpu/drm/ttm/ttm_tt.c +12
-11@@ -239,22 +239,21 @@ int ttm_tt_swapin(struct ttm_tt *ttm)/** * ttm_tt_swapout - swap out tt object -* * @bdev: TTM device structure. * @ttm: The struct ttm_tt. -* @gfp_flags: Flags to use for memory allocation. * -* Swapout a TT object to a shmem_file, return number of pages swapped out or -* negative error code. +* Swapout a TT object to a shmem_file. +* +* Return: number of pages swapped out or negative error code on error. */ -int ttm_tt_swapout (struct ttm_device *bdev,struct ttm_tt *ttm, -gfp_t gfp_flags) +int ttm_tt_swapout (struct ttm_device *bdev,struct ttm_tt *ttm) {loff_t size = (loff_t )ttm->num_pages <<PAGE_SHIFT ;struct address_space *swap_space;struct file *swap_storage;struct page *from_page;struct page *to_page; +gfp_t gfp_flags;int i, ret; swap_storage =shmem_file_setup ("ttm swap" , size,0 );@@ -264,7 +263,7 @@ int ttm_tt_swapout(struct ttm_device *bdev, struct ttm_tt *ttm, } swap_space = swap_storage->f_mapping; - gfp_flags &=mapping_gfp_mask (swap_space); + gfp_flags =GFP_KERNEL &mapping_gfp_mask (swap_space);for (i =0 ; i < ttm->num_pages; ++i) { from_page = ttm->pages[i];@@ -315,12 +314,14 @@ int ttm_tt_populate(struct ttm_device *bdev,while (atomic_long_read (&ttm_pages_allocated) > ttm_pages_limit ||atomic_long_read (&ttm_dma32_pages_allocated) > ttm_dma32_pages_limit) { +long r =ttm_global_swapout (ctx,TTM_SHRINK_WATERMARK ); - ret =ttm_global_swapout (ctx,GFP_KERNEL ); -if (ret ==0 ) +if (!r)break ; -if (ret <0 ) +if (r <0 ) { + ret = r; goto error; + } }if (bdev->funcs->ttm_tt_populate)@@ -379,7 +380,7 @@ static int ttm_tt_debugfs_shrink_show(struct seq_file *m, void *data) {struct ttm_operation_ctx ctx = { false, false }; -seq_printf (m,"%d\n" ,ttm_global_swapout (&ctx,GFP_KERNEL )); +seq_printf (m,"%ld\n" ,ttm_global_swapout (&ctx,TTM_SHRINK_SWAP ));return 0 ; }DEFINE_SHOW_ATTRIBUTE (ttm_tt_debugfs_shrink);drivers/gpu/drm/vmwgfx/vmwgfx_drv.c +2
-1@@ -1514,7 +1514,8 @@ static int vmw_pm_freeze(struct device *kdev)vmw_execbuf_release_pinned_bo (dev_priv);vmw_resource_evict_all (dev_priv);vmw_release_device_early (dev_priv); -while (ttm_device_swapout (&dev_priv->bdev, &ctx,GFP_KERNEL ) >0 ); +while (ttm_device_swapout (&dev_priv->bdev, &ctx,TTM_SHRINK_WATERMARK ) >0 ) + ;vmw_fifo_resource_dec (dev_priv);if (atomic_read (&dev_priv->num_fifo_resources) !=0 ) {DRM_ERROR ("Can't hibernate while 3D resources are active.\n" );include/drm/ttm/ttm_bo.h +2
-2@@ -375,8 +375,8 @@ void ttm_bo_kunmap(struct ttm_bo_kmap_obj *map);int ttm_bo_vmap (struct ttm_buffer_object *bo,struct iosys_map *map);void ttm_bo_vunmap (struct ttm_buffer_object *bo,struct iosys_map *map);int ttm_bo_mmap_obj (struct vm_area_struct *vma,struct ttm_buffer_object *bo); -int ttm_bo_swapout (struct ttm_buffer_object *bo,struct ttm_operation_ctx *ctx, -gfp_t gfp_flags); +long ttm_bo_swapout (struct ttm_buffer_object *bo,struct ttm_operation_ctx *ctx, +enum ttm_shrink_reason reason);void ttm_bo_pin (struct ttm_buffer_object *bo);void ttm_bo_unpin (struct ttm_buffer_object *bo); int ttm_mem_evict_first(struct ttm_device *bdev,include/drm/ttm/ttm_device.h +33
-3@@ -35,6 +35,21 @@ struct ttm_placement;struct ttm_buffer_object ;struct ttm_operation_ctx ; +/** +* enum ttm_shrink_reason - Reason for shrinking system memory +* @TTM_SHRINK_WATERMARK - A watermark limit was reached. Not from reclaim. +* @TTM_SHRINK_PURGE - A request for shrinking only purged objects. +* @TTM_SHRINK_SWAP - A request for shrinking any object. +* +* This enum is intended for the buffer object- and shrink method selection +* algorithms. It's not intended to leak to or be used by TTM drivers. +*/ +enum ttm_shrink_reason { +TTM_SHRINK_WATERMARK , +TTM_SHRINK_PURGE , +TTM_SHRINK_SWAP , +}; +/** * struct ttm_global - Buffer object driver global data. */ @@ -207,6 +222,19 @@ struct ttm_device_funcs {* adding fences that may force a delayed delete */ void (*release_notify)(struct ttm_buffer_object *bo); + +/** +* Shrink the bo's system pages, Either by swapping or by purging. +* @bo: Bo the system pages of which are to be shrunken. +* @ctx: Operation ctx. In particular the driver callback should +* adhere to the no_wait_gpu and interruptible fields. +* +* This is also notifying the driver that the bo is about to be +* shrunken and the driver should take care to unbind any GPU bindings +* and to note that the content is purged if @bo->ttm is purgeable. +*/ +long (*bo_shrink)(struct ttm_buffer_object *bo, +struct ttm_operation_ctx *ctx); };/** @@ -268,9 +296,11 @@ struct ttm_device {struct workqueue_struct *wq; }; -int ttm_global_swapout (struct ttm_operation_ctx *ctx,gfp_t gfp_flags); -int ttm_device_swapout (struct ttm_device *bdev,struct ttm_operation_ctx *ctx, -gfp_t gfp_flags); +long ttm_global_swapout (struct ttm_operation_ctx *ctx, +enum ttm_shrink_reason reason); + +long ttm_device_swapout (struct ttm_device *bdev,struct ttm_operation_ctx *ctx, +enum ttm_shrink_reason reason);static inline struct ttm_resource_manager *ttm_manager_type (struct ttm_device *bdev,int mem_type)include/drm/ttm/ttm_tt.h +15
-2@@ -87,6 +87,7 @@ struct ttm_tt {#define TTM_TT_FLAG_ZERO_ALLOC BIT(1)#define TTM_TT_FLAG_EXTERNAL BIT(2)#define TTM_TT_FLAG_EXTERNAL_MAPPABLE BIT(3) +#define TTM_TT_FLAG_DONTNEED BIT(4)#define TTM_TT_FLAG_PRIV_POPULATED BIT(31)uint32_t page_flags;@@ -180,8 +181,8 @@ void ttm_tt_destroy(struct ttm_device *bdev, struct ttm_tt *ttm);* Swap in a previously swap out ttm_tt. */ int ttm_tt_swapin (struct ttm_tt *ttm); -int ttm_tt_swapout (struct ttm_device *bdev,struct ttm_tt *ttm, -gfp_t gfp_flags); + +int ttm_tt_swapout (struct ttm_device *bdev,struct ttm_tt *ttm);/** * ttm_tt_populate - allocate pages for a ttm @@ -223,6 +224,18 @@ void ttm_tt_mgr_init(unsigned long num_pages, unsigned long num_dma32_pages);struct ttm_kmap_iter *ttm_kmap_iter_tt_init (struct ttm_kmap_iter_tt *iter_tt,struct ttm_tt *tt); +/** +* ttm_tt_purgeable() - Whether a struct ttm_tt's contents is purgeable +* @tt: The struct ttm_tt to consider. +* +* Return: Whether the contents is purgeable in the sence that the owner +* doesn't mind losing it as long as it gets notified. +*/ +static inline bool ttm_tt_purgeable (struct ttm_tt *tt) +{ +return tt->page_flags &TTM_TT_FLAG_DONTNEED ; +} +#if IS_ENABLED (CONFIG_AGP )#include <linux/agp_backend.h>
Re: [RFC PATCH 07/16] drm/ttm: Reduce the number of used allocation orders for TTM pages
In reply to Thomas HellströmAm 15.02.23 um 17:13 schrieb Thomas Hellström:
Quoted 10 lines
When swapping out, we will split multi-order pages both in order to move them to the swap-cache and to be able to return memory to the swap cache as soon as possible on a page-by-page basis.
By reducing the page max order to the system PMD size, we can be nicer to the system and avoid splitting gigantic pages.On top of this we also
include the 64K page size in the page sizes tried, since that appears to be a common size for GPU applications.
Please completely drop that. This is just nonsense spilling in from the Windows drivers.
Christian.
Quoted 157 lines
Looking forward to when we might be able to swap out PMD size folios without splitting, this will also be a benefit.
Signed-off-by Thomas Hellström <[email protected]>--- drivers/gpu/drm/ttm/ttm_pool.c | 58 ++++++++++++++++++++++++++-------- 1 file changed, 45 insertions(+), 13 deletions(-)drivers/gpu/drm/ttm/ttm_pool.c +45
-13@@ -31,6 +31,8 @@* cause they are rather slow compared to alloc_pages+map. */ +#define pr_fmt (fmt) "[TTM POOL] " fmt +#include <linux/module.h> #include <linux/dma-mapping.h> #include <linux/debugfs.h> @@ -47,6 +49,18 @@#include "ttm_module.h" +#define TTM_MAX_ORDER (PMD_SHIFT - PAGE_SHIFT) +#define TTM_64K_ORDER (16 - PAGE_SHIFT) +#if (TTM_MAX_ORDER <TTM_64K_ORDER ) +#undef TTM_MAX_ORDER +#define TTM_MAX_ORDER TTM_64K_ORDER +#endif +#if ((MAX_ORDER -1 ) <TTM_MAX_ORDER ) +#undef TTM_MAX_ORDER +#define TTM_MAX_ORDER (MAX_ORDER - 1) +#endif +#define TTM_DIM_ORDER (TTM_MAX_ORDER + 1) +/** * struct ttm_pool_dma - Helper object for coherent DMA mappings * @@ -65,16 +79,18 @@ module_param(page_pool_size, ulong, 0644);static atomic_long_t allocated_pages; -static struct ttm_pool_type global_write_combined[MAX_ORDER ]; -static struct ttm_pool_type global_uncached[MAX_ORDER ]; +static struct ttm_pool_type global_write_combined[TTM_DIM_ORDER ]; +static struct ttm_pool_type global_uncached[TTM_DIM_ORDER ]; -static struct ttm_pool_type global_dma32_write_combined[MAX_ORDER ]; -static struct ttm_pool_type global_dma32_uncached[MAX_ORDER ]; +static struct ttm_pool_type global_dma32_write_combined[TTM_DIM_ORDER ]; +static struct ttm_pool_type global_dma32_uncached[TTM_DIM_ORDER ];static spinlock_t shrinker_lock;static struct list_head shrinker_list;static struct shrinker mm_shrinker; +static unsigned int ttm_pool_orders[] = {TTM_MAX_ORDER ,0 ,0 }; +/* Allocate pages of size 1 << order with the given gfp_flags */ static struct page *ttm_pool_alloc_page (struct ttm_pool *pool,gfp_t gfp_flags,unsigned int order)@@ -400,6 +416,17 @@ static void __ttm_pool_free(struct ttm_pool *pool, struct ttm_tt *tt, } } +static unsigned int ttm_pool_select_order (unsigned int order,pgoff_t num_pages) +{ +unsigned int *cur_order = ttm_pool_orders; + + order = min_t(unsigned int, __fls(num_pages), order); +while (order < *cur_order) + ++cur_order; + +return *cur_order; +} +/** * ttm_pool_alloc - Fill a ttm_tt object * @@ -439,9 +466,8 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,else gfp_flags |=GFP_HIGHUSER ; - for (order = min_t(unsigned int, MAX_ORDER - 1, __fls(num_pages)); - num_pages; - order = min_t(unsigned int, order, __fls(num_pages))) { + order =ttm_pool_select_order (ttm_pool_orders[0 ], num_pages); +for (; num_pages; order =ttm_pool_select_order (order, num_pages)) {struct ttm_pool_type *pt; page_caching = tt->caching;@@ -558,7 +584,7 @@ void ttm_pool_init(struct ttm_pool *pool, struct device *dev,if (use_dma_alloc) {for (i =0 ; i <TTM_NUM_CACHING_TYPES ; ++i) -for (j =0 ; j <MAX_ORDER ; ++j) +for (j =0 ; j <TTM_DIM_ORDER ; ++j)ttm_pool_type_init (&pool->caching[i].orders[j], pool, i, j); }@@ -578,7 +604,7 @@ void ttm_pool_fini(struct ttm_pool *pool)if (pool->use_dma_alloc) {for (i =0 ; i <TTM_NUM_CACHING_TYPES ; ++i) -for (j =0 ; j <MAX_ORDER ; ++j) +for (j =0 ; j <TTM_DIM_ORDER ; ++j)ttm_pool_type_fini (&pool->caching[i].orders[j]); }@@ -632,7 +658,7 @@ static void ttm_pool_debugfs_header(struct seq_file *m)unsigned int i;seq_puts (m,"\t " ); -for (i =0 ; i <MAX_ORDER ; ++i) +for (i =0 ; i <TTM_DIM_ORDER ; ++i)seq_printf (m," ---%2u---" , i);seq_puts (m,"\n" ); }@@ -643,7 +669,7 @@ static void ttm_pool_debugfs_orders(struct ttm_pool_type *pt, {unsigned int i; -for (i =0 ; i <MAX_ORDER ; ++i) +for (i =0 ; i <TTM_DIM_ORDER ; ++i)seq_printf (m," %8u" ,ttm_pool_type_count (&pt[i]));seq_puts (m,"\n" ); }@@ -749,10 +775,16 @@ int ttm_pool_mgr_init(unsigned long num_pages)if (!page_pool_size) page_pool_size = num_pages; +if (TTM_64K_ORDER <TTM_MAX_ORDER ) + ttm_pool_orders[1 ] =TTM_64K_ORDER ; + +pr_debug ("Used orders are %u %u %u\n" , ttm_pool_orders[0 ], + ttm_pool_orders[1 ], ttm_pool_orders[2 ]); +spin_lock_init (&shrinker_lock);INIT_LIST_HEAD (&shrinker_list); -for (i =0 ; i <MAX_ORDER ; ++i) { +for (i =0 ; i <TTM_DIM_ORDER ; ++i) {ttm_pool_type_init (&global_write_combined[i],NULL , ttm_write_combined, i);ttm_pool_type_init (&global_uncached[i],NULL , ttm_uncached, i);@@ -785,7 +817,7 @@ void ttm_pool_mgr_fini(void) {unsigned int i; -for (i =0 ; i <MAX_ORDER ; ++i) { +for (i =0 ; i <TTM_DIM_ORDER ; ++i) {ttm_pool_type_fini (&global_write_combined[i]);ttm_pool_type_fini (&global_uncached[i]);
Thomas Hellström
reviewerRe: [RFC PATCH 02/16] drm/ttm/pool: Fix ttm_pool_alloc error path
In reply to Christian KönigQuoted 5 lines · Christian König
Am 15.02.23 um 17:13 schrieb Thomas Hellström:
Quoted 2 lines
When hitting an error, the error path forgot to unmap dma mappings and
I don't see where this happens?
From what I can tell, ttm_pool_page_allocated() maps the page for dma, If we later hit an error, ttm_pool_free_page() will leak the mapping.
Quoted 4 lines
Quoted 2 lines
could call
set_pages_wb()on already uncached pages.Yeah, but what's the problem?
Umm, at least if you try to set WC on an already WC'd page, the set_pages_ code will spam dmesg with warnings.
Not sure if set_pages_wb() on WB pages does the same, nor if it issues unnecessary global cache / tlb flushes or whether that will change in the future.
The point of avoiding the set_pages_wb() when already WB is you don't have to check, and you don't have to care.
That said, the __ttm_pool_free() is used also in upcoming patches.
/Thomas
Quoted 190 lines
Regards,
Christian.Quoted 185 lines
Fix this by introducing a common
__ttm_pool_free()function that does the right thing.Fixes d099fc8f540a ("drm/ttm: new TT backend allocation pool v3")Cc Christian König <[email protected]>Cc Dave Airlie <[email protected]>Cc Madhav Chauhan <[email protected]>Cc Christian Koenig <[email protected]>Cc Huang Rui <[email protected]>Signed-off-by Thomas Hellström <[email protected]>---
drivers/gpu/drm/ttm/ttm_pool.c| 74 +++++++++++++++++++++--------
-----1 file changed, 45 insertions(+), 29 deletions(-)drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -367,6 +367,39 @@ static int ttm_pool_page_allocated(struct ttm_pool *pool, unsigned int order, return 0; }
+static void __ttm_pool_free(structttm_pool*pool, structttm_tt*tt,
+ struct page **caching_divide, + enumttm_cachinginitial_caching, + enumttm_cachingsubseq_caching, +pgoff_tnum_pages)drivers/gpu/drm/ttm/ttm_pool.c +19
-0+{ + enum ttm_caching caching = subseq_caching; + struct page **pages = tt->pages; + unsigned int order; + pgoff_t i, nr; + + if (pool && caching_divide) + caching = initial_caching; + + for (i = 0; i < num_pages; i += nr, pages += nr) { + struct ttm_pool_type *pt = NULL; + + if (unlikely(caching_divide == pages)) + caching = subseq_caching; + + order = ttm_pool_page_order(pool, *pages); + nr = (1UL << order); + if (tt->dma_address) + ttm_pool_unmap(pool, tt->dma_address[i],nr);
+
+ pt =ttm_pool_select_type(pool, caching, order); + if (pt)
+ ttm_pool_type_give(pt, *pages); + else
+ ttm_pool_free_page(pool, caching, order, *pages);
+ }drivers/gpu/drm/ttm/ttm_pool.c +2
-0+} +/**
*ttm_pool_alloc- Fill attm_ttobject
*drivers/gpu/drm/ttm/ttm_pool.c +2
-1@@ -386,8 +419,9 @@ int ttm_pool_alloc(struct ttm_pool *pool,struct ttm_tt *tt, dma_addr_t *dma_addr = tt->dma_address; struct page **caching = tt->pages; struct page **pages = tt->pages; + enum ttm_caching page_caching; gfp_t gfp_flags = GFP_USER; - unsigned int i, order; + unsigned int order; struct page *p; int r;drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -410,6 +444,7 @@ int ttm_pool_alloc(struct ttm_pool *pool,struct ttm_tt *tt, order = min_t(unsigned int, order, __fls(num_pages))) { struct ttm_pool_type *pt;
+ page_caching =tt->caching;
pt =ttm_pool_select_type(pool, tt->caching, order);
p = pt ?ttm_pool_type_take(pt): NULL; if (p) {drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -418,6 +453,7 @@ int ttm_pool_alloc(struct ttm_pool *pool,struct ttm_tt *tt, if (r) goto error_free_page;
+ caching = pages;
do {
r = ttm_pool_page_allocated(pool, order, p,&dma_addr,drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -426,14 +462,15 @@ int ttm_pool_alloc(struct ttm_pool *pool,struct ttm_tt *tt, if (r) goto error_free_page;
+ caching = pages;
if (num_pages< (1 << order)) break;
p =ttm_pool_type_take(pt); } while (p);
- caching = pages;
}
+ page_caching =ttm_cached;
while (num_pages>= (1 << order) &&
(p =ttm_pool_alloc_page(pool, gfp_flags, order))) {drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -442,6 +479,7 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,tt-
Quoted 1 line
caching);
if (r)
gotoerror_free_page; + caching = pages;
}
r = ttm_pool_page_allocated(pool, order, p,&dma_addr,
&num_pages, &pages);drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -468,15 +506,12 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt, return 0;
error_free_page:
- ttm_pool_free_page(pool,tt->caching, order, p); + ttm_pool_free_page(pool,page_caching, order, p);
error_free_all:
num_pages =tt->num_pages-num_pages;
- for (i = 0; i <num_pages; ) {
- order =ttm_pool_page_order(pool, tt->pages[i]); - ttm_pool_free_page(pool,tt->caching, order, tt-Quoted 1 line
pages[i]);
- i += 1 << order;
- }
+ __ttm_pool_free(pool, tt, caching,tt->caching,ttm_cached, + num_pages);
return r;
}drivers/gpu/drm/ttm/ttm_pool.c +2
-21@@ -492,27 +527,8 @@ EXPORT_SYMBOL(ttm_pool_alloc);*/ void ttm_pool_free(struct ttm_pool *pool, struct ttm_tt *tt) { - unsigned int i; - - for (i = 0; i < tt->num_pages; ) { - struct page *p = tt->pages[i]; - unsigned int order, num_pages; - struct ttm_pool_type *pt; - - order = ttm_pool_page_order(pool, p); - num_pages = 1ULL << order; - if (tt->dma_address) - ttm_pool_unmap(pool, tt->dma_address[i], num_pages); - - pt = ttm_pool_select_type(pool, tt->caching, order); - if (pt) - ttm_pool_type_give(pt, tt->pages[i]); - else - ttm_pool_free_page(pool, tt->caching, order, - tt->pages[i]); - - i += num_pages; - } + __ttm_pool_free(pool, tt, NULL, tt->caching, tt->caching, + tt->num_pages);
while (atomic_long_read(&allocated_pages)>page_pool_size)ttm_pool_shrink();
Thomas Hellström
reviewerRe: [RFC PATCH 07/16] drm/ttm: Reduce the number of used allocation orders for TTM pages
In reply to Christian KönigQuoted 9 lines · Christian König
Am 15.02.23 um 17:13 schrieb Thomas Hellström:
Quoted 6 lines
When swapping out, we will split multi-order pages both in order to move them to the swap-cache and to be able to return memory to the swap cache as soon as possible on a page-by-page basis. By reducing the page max order to the system PMD size, we can be nicer
to the system and avoid splitting gigantic pages.
Quoted 6 lines
Quoted 4 lines
On top of this we also
include the 64K page size in the page sizes tried, since that appears to
be a common size for GPU applications.Please completely drop that.
You mean the 64K page size, or the whole patch?
Quoted 2 lines
This is just nonsense spilling in from the
Windows drivers.
Agreed, but IIRC on the last RFC you asked me not to drop the 64K pages, so that's why they are here. I can remove them if needed.
The only reason for keeping them from a performance point of view is better efficiency on GPUs with 64K page size if not using a coalescing IOMMU for dma-mapping.
Let me know what you think is best and I'll adjust accordingly.
/Thomas
Quoted 181 lines
Christian.
Quoted 177 lines
Looking forward to when we might be able to swap out PMD size folios
without splitting, this will also be a benefit.Signed-off-by Thomas Hellström <[email protected]>---
drivers/gpu/drm/ttm/ttm_pool.c| 58 ++++++++++++++++++++++++++---
-----1 file changed, 45 insertions(+), 13 deletions(-)drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -31,6 +31,8 @@* cause they are rather slow compared to alloc_pages+map. */ drivers/gpu/drm/ttm/ttm_pool.c +2
-0+#define pr_fmt (fmt) "[TTM POOL] " fmt +#include <
linux/module.h>
#include <linux/dma-mapping.h>
#include <linux/debugfs.h>drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -47,6 +49,18 @@
#include "ttm_module.h"drivers/gpu/drm/ttm/ttm_pool.c +12
-0+#define TTM_MAX_ORDER (PMD_SHIFT - PAGE_SHIFT) +#define TTM_64K_ORDER (16 - PAGE_SHIFT) +#if (TTM_MAX_ORDER <TTM_64K_ORDER ) +#undef TTM_MAX_ORDER +#define TTM_MAX_ORDER TTM_64K_ORDER +#endif +#if ((MAX_ORDER -1 ) <TTM_MAX_ORDER ) +#undef TTM_MAX_ORDER +#define TTM_MAX_ORDER (MAX_ORDER - 1) +#endif +#define TTM_DIM_ORDER (TTM_MAX_ORDER + 1) +/**
* structttm_pool_dma- Helper object for coherent DMA mappings *drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -65,16 +79,18 @@ module_param(page_pool_size, ulong, 0644);
staticatomic_long_tallocated_pages;drivers/gpu/drm/ttm/ttm_pool.c +2
-2-static struct ttm_pool_type global_write_combined[MAX_ORDER ]; -static struct ttm_pool_type global_uncached[MAX_ORDER ]; +static struct ttm_pool_type global_write_combined[TTM_DIM_ORDER ]; +static struct ttm_pool_type global_uncached[TTM_DIM_ORDER ];
-static structttm_pool_type
global_dma32_write_combined[MAX_ORDER];drivers/gpu/drm/ttm/ttm_pool.c +1
-1-static struct ttm_pool_type global_dma32_uncached[MAX_ORDER ]; +static struct ttm_pool_typeglobal_dma32_write_combined[TTM_DIM_ORDER];
+static structttm_pool_typeglobal_dma32_uncached[TTM_DIM_ORDER];
staticspinlock_tshrinker_lock;
static structlist_headshrinker_list;
static struct shrinkermm_shrinker;drivers/gpu/drm/ttm/ttm_pool.c +2
-0+static unsigned int ttm_pool_orders[] = {TTM_MAX_ORDER ,0 ,0 }; +/* Allocate pages of size 1 << order with the given
gfp_flags*/ static struct page *ttm_pool_alloc_page(structttm_pool*pool,gfp_tgfp_flags,
unsigned int order)drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -400,6 +416,17 @@ static void __ttm_pool_free(struct ttm_pool *pool, struct ttm_tt *tt, } }
+static unsigned intttm_pool_select_order(unsigned int order, pgoff_t num_pages)drivers/gpu/drm/ttm/ttm_pool.c +10
-0+{ + unsigned int *cur_order = ttm_pool_orders; + + order = min_t(unsigned int, __fls(num_pages), order); + while (order < *cur_order) + ++cur_order; + + return *cur_order; +} +/**
*ttm_pool_alloc- Fill attm_ttobject
*drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -439,9 +466,8 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,else
gfp_flags |=GFP_HIGHUSER;
- for (order =min_t(unsigned int, MAX_ORDER - 1, __fls(num_pages));
-num_pages;
- order =min_t(unsigned int, order, __fls(num_pages))) {
+ order =ttm_pool_select_order(ttm_pool_orders[0], num_pages);
+ for (;num_pages; order =ttm_pool_select_order(order, num_pages)) {
structttm_pool_type*pt;
page_caching =tt->caching;drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -558,7 +584,7 @@ void ttm_pool_init(struct ttm_pool *pool, struct device *dev,
if (use_dma_alloc) {
for (i = 0; i <TTM_NUM_CACHING_TYPES; ++i) - for (j = 0; j <MAX_ORDER; ++j) + for (j = 0; j <TTM_DIM_ORDER; ++j) ttm_pool_type_init(&pool-Quoted 1 line
caching[i].orders[j],
pool, i, j); }
drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -578,7 +604,7 @@ void ttm_pool_fini(struct ttm_pool *pool)
if (pool->use_dma_alloc) {
for (i = 0; i <TTM_NUM_CACHING_TYPES; ++i) - for (j = 0; j <MAX_ORDER; ++j) + for (j = 0; j <TTM_DIM_ORDER; ++j) ttm_pool_type_fini(&pool-Quoted 1 line
caching[i].orders[j]);
}
drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -632,7 +658,7 @@ static void ttm_pool_debugfs_header(struct seq_file *m) unsigned int i;
seq_puts(m, "\t ");
- for (i = 0; i <MAX_ORDER; ++i)
+ for (i = 0; i <TTM_DIM_ORDER; ++i)
seq_printf(m, " ---%2u---", i);
seq_puts(m, "\n");
}drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -643,7 +669,7 @@ static void ttm_pool_debugfs_orders(structttm_pool_type *pt, { unsigned int i;
- for (i = 0; i <MAX_ORDER; ++i)
+ for (i = 0; i <TTM_DIM_ORDER; ++i)seq_printf(m, " %8u", ttm_pool_type_count(&pt[i]));seq_puts(m, "\n");
}drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -749,10 +775,16 @@ int ttm_pool_mgr_init(unsigned long num_pages) if (!page_pool_size) page_pool_size = num_pages;
+ if (TTM_64K_ORDER<TTM_MAX_ORDER) + ttm_pool_orders[1] =TTM_64K_ORDER;drivers/gpu/drm/ttm/ttm_pool.c +4
-0+ + pr_debug("Used orders are %u %u %u\n", ttm_pool_orders[0], + ttm_pool_orders[1], ttm_pool_orders[2]); +spin_lock_init(&shrinker_lock);
INIT_LIST_HEAD(&shrinker_list);
- for (i = 0; i <MAX_ORDER; ++i) {
+ for (i = 0; i <TTM_DIM_ORDER; ++i) {ttm_pool_type_init(&global_write_combined[i], NULL, ttm_write_combined, i);ttm_pool_type_init(&global_uncached[i], NULL, ttm_uncached, i);drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -785,7 +817,7 @@ void ttm_pool_mgr_fini(void) { unsigned int i;
- for (i = 0; i <MAX_ORDER; ++i) {
+ for (i = 0; i <TTM_DIM_ORDER; ++i) {ttm_pool_type_fini(&global_write_combined[i]);ttm_pool_type_fini(&global_uncached[i]);
kernel test robot
build botRe: [RFC PATCH 07/16] drm/ttm: Reduce the number of used allocation orders for TTM pages
In reply to Thomas Hellström[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in…
Hi Thomas,
[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on drm-misc/drm-misc-next]
[also build test WARNING on drm-intel/for-linux-next] [cannot apply to drm-intel/for-linux-next-fixes linus/master v6.2-rc8] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Thomas-Hellstr-m/drm-ttm-Fix-a-NULL-pointer-dereference/20230216-001727 base: git://anongit.freedesktop.org/drm/drm-misc drm-misc-next patch link: https://lore.kernel.org/r/20230215161405.187368-8-thomas.hellstrom%40linux.intel.com patch subject: [RFC PATCH 07/16] drm/ttm: Reduce the number of used allocation orders for TTM pages config: powerpc-allyesconfig (https://download.01.org/0day-ci/archive/20230216/[email protected]/config) compiler: powerpc-linux-gcc (GCC) 12.1.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/4925cefe2ad6e10e768cebed1a78bfde13199195
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Thomas-Hellstr-m/drm-ttm-Fix-a-NULL-pointer-dereference/20230216-001727 git checkout 4925cefe2ad6e10e768cebed1a78bfde13199195
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=powerpc olddefconfig COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=powerpc SHELL=/bin/bash drivers/gpu/drm/ttm/
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Link: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
In file included from arch/powerpc/include/asm/page.h:256,
from arch/powerpc/include/asm/thread_info.h:13,
from include/linux/thread_info.h:60,
from arch/powerpc/include/asm/ptrace.h:342,
from arch/powerpc/include/asm/hw_irq.h:12,
from arch/powerpc/include/asm/irqflags.h:12,
from include/linux/irqflags.h:16,
from include/asm-generic/cmpxchg-local.h:6,
from arch/powerpc/include/asm/cmpxchg.h:755,
from arch/powerpc/include/asm/atomic.h:11,
from include/linux/atomic.h:7,
from include/linux/mm_types_task.h:13,
from include/linux/mm_types.h:5,
from include/linux/buildid.h:5,
from include/linux/module.h:14,
from drivers/gpu/drm/ttm/ttm_pool.c:36:
>> arch/powerpc/include/asm/page_32.h:58:26: warning: "__builtin_ffs" is not defined, evaluates to 0 [-Wundef]
58 | #define PTE_T_LOG2 (__builtin_ffs(sizeof(pte_t)) - 1)
| ^~~~~~~~~~~~~
arch/powerpc/include/asm/page_32.h:23:39: note: in expansion of macro 'PTE_T_LOG2'
23 | #define PTE_SHIFT (PAGE_SHIFT - PTE_T_LOG2) /* full page */
| ^~~~~~~~~~
arch/powerpc/include/asm/book3s/32/pgtable.h:120:25: note: in expansion of macro 'PTE_SHIFT'
120 | #define PTE_INDEX_SIZE PTE_SHIFT
| ^~~~~~~~~
arch/powerpc/include/asm/book3s/32/pgtable.h:152:39: note: in expansion of macro 'PTE_INDEX_SIZE'
152 | #define PGDIR_SHIFT (PAGE_SHIFT + PTE_INDEX_SIZE)
| ^~~~~~~~~~~~~~
include/asm-generic/pgtable-nop4d.h:11:33: note: in expansion of macro 'PGDIR_SHIFT'
11 | #define P4D_SHIFT PGDIR_SHIFT
| ^~~~~~~~~~~
include/asm-generic/pgtable-nopud.h:18:25: note: in expansion of macro 'P4D_SHIFT'
18 | #define PUD_SHIFT P4D_SHIFT
| ^~~~~~~~~
include/asm-generic/pgtable-nopmd.h:20:25: note: in expansion of macro 'PUD_SHIFT'
20 | #define PMD_SHIFT PUD_SHIFT
| ^~~~~~~~~
drivers/gpu/drm/ttm/ttm_pool.c:52:24: note: in expansion of macro 'PMD_SHIFT'
52 | #define TTM_MAX_ORDER (PMD_SHIFT - PAGE_SHIFT)
| ^~~~~~~~~
drivers/gpu/drm/ttm/ttm_pool.c:54:6: note: in expansion of macro 'TTM_MAX_ORDER'
54 | #if (TTM_MAX_ORDER < TTM_64K_ORDER)
| ^~~~~~~~~~~~~
arch/powerpc/include/asm/page_32.h:58:39: error: missing binary operator before token "("
58 | #define PTE_T_LOG2 (__builtin_ffs(sizeof(pte_t)) - 1)
| ^
arch/powerpc/include/asm/page_32.h:23:39: note: in expansion of macro 'PTE_T_LOG2'
23 | #define PTE_SHIFT (PAGE_SHIFT - PTE_T_LOG2) /* full page */
| ^~~~~~~~~~
arch/powerpc/include/asm/book3s/32/pgtable.h:120:25: note: in expansion of macro 'PTE_SHIFT'
120 | #define PTE_INDEX_SIZE PTE_SHIFT
| ^~~~~~~~~
arch/powerpc/include/asm/book3s/32/pgtable.h:152:39: note: in expansion of macro 'PTE_INDEX_SIZE'
152 | #define PGDIR_SHIFT (PAGE_SHIFT + PTE_INDEX_SIZE)
| ^~~~~~~~~~~~~~
include/asm-generic/pgtable-nop4d.h:11:33: note: in expansion of macro 'PGDIR_SHIFT'
11 | #define P4D_SHIFT PGDIR_SHIFT
| ^~~~~~~~~~~
include/asm-generic/pgtable-nopud.h:18:25: note: in expansion of macro 'P4D_SHIFT'
18 | #define PUD_SHIFT P4D_SHIFT
| ^~~~~~~~~
include/asm-generic/pgtable-nopmd.h:20:25: note: in expansion of macro 'PUD_SHIFT'
20 | #define PMD_SHIFT PUD_SHIFT
| ^~~~~~~~~
drivers/gpu/drm/ttm/ttm_pool.c:52:24: note: in expansion of macro 'PMD_SHIFT'
52 | #define TTM_MAX_ORDER (PMD_SHIFT - PAGE_SHIFT)
| ^~~~~~~~~
drivers/gpu/drm/ttm/ttm_pool.c:54:6: note: in expansion of macro 'TTM_MAX_ORDER'
54 | #if (TTM_MAX_ORDER < TTM_64K_ORDER)
| ^~~~~~~~~~~~~
>> arch/powerpc/include/asm/page_32.h:58:26: warning: "__builtin_ffs" is not defined, evaluates to 0 [-Wundef]
58 | #define PTE_T_LOG2 (__builtin_ffs(sizeof(pte_t)) - 1)
| ^~~~~~~~~~~~~
arch/powerpc/include/asm/page_32.h:23:39: note: in expansion of macro 'PTE_T_LOG2'
23 | #define PTE_SHIFT (PAGE_SHIFT - PTE_T_LOG2) /* full page */
| ^~~~~~~~~~
arch/powerpc/include/asm/book3s/32/pgtable.h:120:25: note: in expansion of macro 'PTE_SHIFT'
120 | #define PTE_INDEX_SIZE PTE_SHIFT
| ^~~~~~~~~
arch/powerpc/include/asm/book3s/32/pgtable.h:152:39: note: in expansion of macro 'PTE_INDEX_SIZE'
152 | #define PGDIR_SHIFT (PAGE_SHIFT + PTE_INDEX_SIZE)
| ^~~~~~~~~~~~~~
include/asm-generic/pgtable-nop4d.h:11:33: note: in expansion of macro 'PGDIR_SHIFT'
11 | #define P4D_SHIFT PGDIR_SHIFT
| ^~~~~~~~~~~
include/asm-generic/pgtable-nopud.h:18:25: note: in expansion of macro 'P4D_SHIFT'
18 | #define PUD_SHIFT P4D_SHIFT
| ^~~~~~~~~
include/asm-generic/pgtable-nopmd.h:20:25: note: in expansion of macro 'PUD_SHIFT'
20 | #define PMD_SHIFT PUD_SHIFT
| ^~~~~~~~~
drivers/gpu/drm/ttm/ttm_pool.c:52:24: note: in expansion of macro 'PMD_SHIFT'
52 | #define TTM_MAX_ORDER (PMD_SHIFT - PAGE_SHIFT)
| ^~~~~~~~~
drivers/gpu/drm/ttm/ttm_pool.c:58:24: note: in expansion of macro 'TTM_MAX_ORDER'
58 | #if ((MAX_ORDER - 1) < TTM_MAX_ORDER)
| ^~~~~~~~~~~~~
arch/powerpc/include/asm/page_32.h:58:39: error: missing binary operator before token "("
58 | #define PTE_T_LOG2 (__builtin_ffs(sizeof(pte_t)) - 1)
| ^
arch/powerpc/include/asm/page_32.h:23:39: note: in expansion of macro 'PTE_T_LOG2'
23 | #define PTE_SHIFT (PAGE_SHIFT - PTE_T_LOG2) /* full page */
| ^~~~~~~~~~
arch/powerpc/include/asm/book3s/32/pgtable.h:120:25: note: in expansion of macro 'PTE_SHIFT'
120 | #define PTE_INDEX_SIZE PTE_SHIFT
| ^~~~~~~~~
arch/powerpc/include/asm/book3s/32/pgtable.h:152:39: note: in expansion of macro 'PTE_INDEX_SIZE'
152 | #define PGDIR_SHIFT (PAGE_SHIFT + PTE_INDEX_SIZE)
| ^~~~~~~~~~~~~~
include/asm-generic/pgtable-nop4d.h:11:33: note: in expansion of macro 'PGDIR_SHIFT'
11 | #define P4D_SHIFT PGDIR_SHIFT
| ^~~~~~~~~~~
include/asm-generic/pgtable-nopud.h:18:25: note: in expansion of macro 'P4D_SHIFT'
18 | #define PUD_SHIFT P4D_SHIFT
| ^~~~~~~~~
include/asm-generic/pgtable-nopmd.h:20:25: note: in expansion of macro 'PUD_SHIFT'
20 | #define PMD_SHIFT PUD_SHIFT
| ^~~~~~~~~
drivers/gpu/drm/ttm/ttm_pool.c:52:24: note: in expansion of macro 'PMD_SHIFT'
52 | #define TTM_MAX_ORDER (PMD_SHIFT - PAGE_SHIFT)
| ^~~~~~~~~
drivers/gpu/drm/ttm/ttm_pool.c:58:24: note: in expansion of macro 'TTM_MAX_ORDER'
58 | #if ((MAX_ORDER - 1) < TTM_MAX_ORDER)
| ^~~~~~~~~~~~~vim +/__builtin_ffs +58 arch/powerpc/include/asm/page_32.h
5cd16ee934eafc include/asm-powerpc/page_32.h Michael Ellerman 2005-11-11 56
ca9153a3a2a755 arch/powerpc/include/asm/page_32.h Ilya Yanok 2008-12-11 57 #define PGD_T_LOG2 (__builtin_ffs(sizeof(pgd_t)) - 1) ca9153a3a2a755 arch/powerpc/include/asm/page_32.h Ilya Yanok 2008-12-11 @58 #define PTE_T_LOG2 (__builtin_ffs(sizeof(pte_t)) - 1) ca9153a3a2a755 arch/powerpc/include/asm/page_32.h Ilya Yanok 2008-12-11 59
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests
Thomas Hellström
reviewerRe: [RFC PATCH 04/16] drm/ttm, drm/vmwgfx: Update the TTM swapout interface
In reply to Christian KönigQuoted 13 lines · Christian König
Am 15.02.23 um 17:13 schrieb Thomas Hellström:
Quoted 11 lines
Update the TTM swapout interfaces for better compatibility with a shrinker.
- Replace number-of-pages int return with a long to better match the
kernel's shrinker interface.
- Thegfp_flagsparameter tottm_xx_swapout()currently only takes the
GFP_KERNELvalue and shouldn't really be needed since the shrinker we
hook up in upcoming patches sets a allocation context to match reclaim.
Quoted 8 lines
Quoted 3 lines
- Introduce a shrink reason enumeration and a driver callback to shrink
buffer objects.Is that really necessary? This is mid-layering once more.
If drivers want to implement driver specific shrinking they should register their own shrinker callback.
Yes, a choice needs to be made here. If TTM registers the shrinker, the driver needs to be called at least to unbind and to remove dma- mappings.
If the driver registers the shrinker it can still (I think) use the pool helpers, but needs TTM for LRU traversal and accounting.
I can have a look at the latter if yout think that will be a better solution.
/Thomas
Quoted 471 lines
Christian.
Quoted 466 lines
The
TTM_SHRINK_WATERMARKreason is going to still be handled using the
existing shmem copy, and will be used by pool types that don't lend
themselves well to shinking (dma_allocpool) and when drivers explicitly
requests swapout.
TheTTM_SHRINK_SWAPandTTM_SHRINK_PURGEreasons originate from a
shrinker and is to be handled by a new driver callback,bo_shrink().
Helpers for the new driver callback are provided in upcoming patches.Signed-off-by Thomas Hellström <[email protected]>---
drivers/gpu/drm/ttm/ttm_bo.c | 38 ++++++++++++++++---- drivers/gpu/drm/ttm/ttm_device.c | 55 +++++++++++++++++++++---
-----
drivers/gpu/drm/ttm/ttm_tt.c | 23 ++++++------drivers/gpu/drm/vmwgfx/vmwgfx_drv.c| 3 +- include/drm/ttm/ttm_bo.h | 4 +-- include/drm/ttm/ttm_device.h | 36 +++++++++++++++++-- include/drm/ttm/ttm_tt.h | 17 +++++++--7 files changed, 136 insertions(+), 40 deletions(-)drivers/gpu/drm/ttm/ttm_bo.c +0
-0@@ -1114,13 +1114,29 @@ int ttm_bo_wait_ctx(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx) } EXPORT_SYMBOL(ttm_bo_wait_ctx);
-int ttm_bo_swapout(structttm_buffer_object*bo, structttm_operation_ctx*ctx,
-gfp_tgfp_flags)
+/**
+ *ttm_bo_swapout()- Swap out or purge a buffer object
+ * @bo: The buffer object.
+ * @ctx: The ttm operation context.
+ * @reason: The swapout reason.
+ *
+ * Try to swap out or purge the contents of a system memory backed buffer
+ * object. The function needs to be called with the device's LRU lock held.
+ *
+ * Return:-EBUSYif the bo lock could not be grabbed or the object was
+ * otherwise busy. Otherwise the number of pages swapped out or negative
+ * error code on error. Iff the function didn't return-EBUSY, the
+ * LRU lock was dropped, and LRU traversal needs to restart.
+ */
+long ttm_bo_swapout(structttm_buffer_object*bo, structttm_operation_ctx*ctx,
+ enumttm_shrink_reasonreason)
{
structttm_placeplace;
bool locked;
long ret;
+ lockdep_assert_held(&bo->bdev->lru_lock);
+
/*
* While the bo may already reside in SYSTEM placement, set
* SYSTEM as new placement to cover also the move further below.drivers/gpu/drm/ttm/ttm_bo.c +0
-0@@ -1142,8 +1158,12 @@ int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx, }
if (bo->deleted) {
+ longnum_pages=bo->ttm->num_pages; +
ret =ttm_bo_cleanup_refs(bo, false, false, locked);
ttm_bo_put(bo);
+ if (!ret)
+ returnnum_pages;
return ret ==-EBUSY?-ENOSPC: ret; }drivers/gpu/drm/ttm/ttm_bo.c +9
-2@@ -1184,13 +1204,17 @@ int ttm_bo_swapout(struct ttm_buffer_object*bo, struct ttm_operation_ctx *ctx, * Swap out. Buffer will be swapped in again as soon as * anyone tries to access a ttm page. */ - if (bo->bdev->funcs->swap_notify) - bo->bdev->funcs->swap_notify(bo); + if (bo->bdev->funcs->bo_shrink && reason !=TTM_SHRINK_WATERMARK ) { + ret = bo->bdev->funcs->bo_shrink(bo, ctx); + } else { + if (bo->bdev->funcs->swap_notify) + bo->bdev->funcs->swap_notify(bo); + ret = ttm_tt_swapout(bo->bdev, bo->ttm); + if (!ret) + ret = bo->ttm->num_pages; + }
- if (ttm_tt_is_populated(bo->ttm))
- ret =ttm_tt_swapout(bo->bdev, bo->ttm, gfp_flags); out:
-
/*
* Unreserve without putting on LRU to avoid swapping out an
* already swapped buffer.drivers/gpu/drm/ttm/ttm_device.c +0
-0@@ -116,19 +116,28 @@ static int ttm_global_init(void) return ret; }drivers/gpu/drm/ttm/ttm_device.c +2
-3-/* -* A buffer object shrink method that tries to swap out the first -* buffer object on the global::swap_lru list. +/** +* ttm_global_swapout() - Select and swap out a system-memory- backed bo.
+ * @ctx: The operation context.
+ * @reason: The reason for swapout.
+ *
+ * Select, based on round-robin a TTM device and traverse the LRUs of
+ * that specific device until a suitable bo backed by system memory is found
+ * and swapped-out or purged.
+ *
+ * Return: Positive value or zero indicating the size in pages of the
+ * bo swapped out. Negative error code on error.
*/
-intttm_global_swapout(struct ttm_operation_ctx *ctx, gfp_t gfp_flags)
+long ttm_global_swapout(structttm_operation_ctx*ctx, + enumttm_shrink_reasonreason)
{
structttm_global*glob =&ttm_glob;
structttm_device*bdev;
- int ret = 0;
+ long ret = 0;
mutex_lock(&ttm_global_mutex);list_for_each_entry(bdev, &glob->device_list, device_list){
- ret =ttm_device_swapout(bdev, ctx, gfp_flags); + ret =ttm_device_swapout(bdev, ctx, reason); if (ret > 0) { list_move_tail(&bdev->device_list, &glob-Quoted 1 line
device_list);break;
drivers/gpu/drm/ttm/ttm_device.c +0
-0@@ -139,14 +148,29 @@ int ttm_global_swapout(struct ttm_operation_ctx *ctx, gfp_t gfp_flags) } EXPORT_SYMBOL(ttm_global_swapout);
-int ttm_device_swapout(structttm_device*bdev, structttm_operation_ctx*ctx,
-gfp_tgfp_flags)
+/**
+ *ttm_device_swapout()- Select and swap out a system-memory- backed bo.
+ * @bdev: The device whos bos are considered for swapout.
+ * @ctx: The operation context.
+ * @reason: The reason for swapout.
+ *
+ * Traverse the LRUs of a specific device until a suitable bo backed by
+ * system memory is found and swapped-out or purged.
+ *
+ * Return: Positive value or zero indicating the size in pages of the
+ * bo swapped out. Negative error code on error.
+ */
+long ttm_device_swapout(structttm_device*bdev, structttm_operation_ctx*ctx,
+ enumttm_shrink_reasonreason) {
structttm_resource_cursorcursor;
structttm_resource_manager*man;
structttm_resource*res;
unsigned i;
- int ret;
+ long ret;
+
+ if (reason !=TTM_SHRINK_WATERMARK&& !bdev->funcs-Quoted 1 line
bo_shrink)+ return 0;
spin_lock(&bdev->lru_lock);
for (i =TTM_PL_SYSTEM; i <TTM_NUM_MEM_TYPES; ++i) {drivers/gpu/drm/ttm/ttm_device.c +0
-0@@ -156,16 +180,19 @@ int ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx,
ttm_resource_manager_for_each_res(man, &cursor, res) {
structttm_buffer_object*bo =res->bo; - uint32_tnum_pages;
+ structttm_tt*tt;
if (!bo ||bo->resource!= res) continue;
- num_pages =PFN_UP(bo->base.size); - ret =ttm_bo_swapout(bo, ctx, gfp_flags); + tt =bo->ttm;
+ if (!tt || (reason ==TTM_SHRINK_PURGE&& + !ttm_tt_purgeable(tt))) + continue;
+ ret =ttm_bo_swapout(bo, ctx, reason); /*ttm_bo_swapouthas dropped thelru_lock*/
- if (!ret)
- returnnum_pages;
+ if (ret >= 0)
+ return ret;
if (ret !=-EBUSY)
return ret;
}drivers/gpu/drm/ttm/ttm_tt.c +0
-0@@ -239,22 +239,21 @@ int ttm_tt_swapin(struct ttm_tt *ttm)
/**
*ttm_tt_swapout- swap out tt object
- *
* @bdev: TTM device structure.
* @ttm: The structttm_tt.
- * @gfp_flags: Flags to use for memory allocation.
*
- * Swapout a TT object to ashmem_file, return number of pages swapped out or
- * negative error code.
+ * Swapout a TT object to ashmem_file.
+ *
+ * Return: number of pages swapped out or negative error code on error.
*/drivers/gpu/drm/ttm/ttm_tt.c +1
-2-int ttm_tt_swapout (struct ttm_device *bdev,struct ttm_tt *ttm, - gfp_t gfp_flags) +int ttm_tt_swapout (struct ttm_device *bdev,struct ttm_tt *ttm){
loff_t size = (loff_t)ttm->num_pages <<PAGE_SHIFT; structaddress_space*swap_space;
struct file*swap_storage;
struct page*from_page;
struct page*to_page;
+ gfp_tgfp_flags;
int i, ret;
swap_storage =shmem_file_setup("ttm swap", size, 0);drivers/gpu/drm/ttm/ttm_tt.c +0
-0@@ -264,7 +263,7 @@ int ttm_tt_swapout(struct ttm_device *bdev, struct ttm_tt *ttm, }
swap_space =swap_storage->f_mapping;
- gfp_flags &=mapping_gfp_mask(swap_space);
+ gfp_flags =GFP_KERNEL&mapping_gfp_mask(swap_space);
for (i = 0; i <ttm->num_pages; ++i) {from_page= ttm->pages[i];drivers/gpu/drm/ttm/ttm_tt.c +0
-0@@ -315,12 +314,14 @@ int ttm_tt_populate(struct ttm_device *bdev,while (
atomic_long_read(&ttm_pages_allocated)>ttm_pages_limit||atomic_long_read(&ttm_dma32_pages_allocated)>ttm_dma32_pages_limit) {
+ long r =ttm_global_swapout(ctx, TTM_SHRINK_WATERMARK);
- ret =ttm_global_swapout(ctx, GFP_KERNEL); - if (ret == 0)
+ if (!r)
break;
- if (ret < 0)
+ if (r < 0) {
+ ret = r;
goto error;
+ }
}
if (bdev->funcs->ttm_tt_populate)drivers/gpu/drm/ttm/ttm_tt.c +0
-0@@ -379,7 +380,7 @@ static int ttm_tt_debugfs_shrink_show(struct seq_file *m, void *data) { struct ttm_operation_ctx ctx = { false, false };
- seq_printf(m, "%d\n",ttm_global_swapout(&ctx, GFP_KERNEL));
+ seq_printf(m, "%ld\n",ttm_global_swapout(&ctx, TTM_SHRINK_SWAP));
return 0;
}DEFINE_SHOW_ATTRIBUTE(ttm_tt_debugfs_shrink);drivers/gpu/drm/vmwgfx/vmwgfx_drv.c +2
-1@@ -1514,7 +1514,8 @@ static int vmw_pm_freeze(struct device *kdev) vmw_execbuf_release_pinned_bo(dev_priv); vmw_resource_evict_all(dev_priv); vmw_release_device_early(dev_priv); - while (ttm_device_swapout(&dev_priv->bdev, &ctx, GFP_KERNEL) > 0); + while (ttm_device_swapout(&dev_priv->bdev, &ctx,TTM_SHRINK_WATERMARK ) >0 ) + ; vmw_fifo_resource_dec(dev_priv);if (
atomic_read(&dev_priv->num_fifo_resources)!= 0) {DRM_ERROR("Can't hibernate while 3D resources are active.\n");include/drm/ttm/ttm_bo.h +0
-0@@ -375,8 +375,8 @@ void ttm_bo_kunmap(struct ttm_bo_kmap_obj *map);int
ttm_bo_vmap(struct ttm_buffer_object *bo, struct iosys_map *map);
voidttm_bo_vunmap(struct ttm_buffer_object *bo, struct iosys_map *map);
intttm_bo_mmap_obj(struct vm_area_struct *vma, struct ttm_buffer_object *bo);
-int ttm_bo_swapout(structttm_buffer_object*bo, structttm_operation_ctx*ctx,
-gfp_tgfp_flags);
+long ttm_bo_swapout(structttm_buffer_object*bo, structttm_operation_ctx*ctx,
+ enumttm_shrink_reasonreason);
voidttm_bo_pin(struct ttm_buffer_object *bo);
voidttm_bo_unpin(struct ttm_buffer_object *bo);
int ttm_mem_evict_first(structttm_device*bdev,include/drm/ttm/ttm_device.h +0
-0@@ -35,6 +35,21 @@ struct ttm_placement; struct ttm_buffer_object; struct ttm_operation_ctx;
+/**
+ * enumttm_shrink_reason- Reason for shrinking system memory
+ * @TTM_SHRINK_WATERMARK - A watermark limit was reached. Not from reclaim.
+ * @TTM_SHRINK_PURGE - A request for shrinking only purged objects.
+ * @TTM_SHRINK_SWAP - A request for shrinking any object.
+ *
+ * This enum is intended for the buffer object- and shrink method selection
+ * algorithms. It's not intended to leak to or be used by TTM drivers.
+ */include/drm/ttm/ttm_device.h +6
-0+enum ttm_shrink_reason { + TTM_SHRINK_WATERMARK, + TTM_SHRINK_PURGE, + TTM_SHRINK_SWAP, +}; +/**
* structttm_global- Buffer object driver global data. */include/drm/ttm/ttm_device.h +3
-0@@ -207,6 +222,19 @@ struct ttm_device_funcs {* adding fences that may force a delayed delete */ void (*release_notify)(struct ttm_buffer_object *bo); + + /** +* Shrink the bo's system pages, Either by swapping or by purging.
+ * @bo: Bo the system pages of which are to be shrunken. + * @ctx: Operation ctx. In particular the driver callback should
+ * adhere to theno_wait_gpuand interruptible fields.
+ *
+ * This is also notifying the driver that the bo is about to be
+ * shrunken and the driver should take care to unbind any GPU bindings
+ * and to note that the content is purged if @bo->ttm is purgeable.
+ */
+ long (*bo_shrink)(structttm_buffer_object*bo, + structttm_operation_ctx*ctx); };
/**include/drm/ttm/ttm_device.h +0
-0@@ -268,9 +296,11 @@ struct ttm_device { struct workqueue_struct *wq; };
-intttm_global_swapout(struct ttm_operation_ctx *ctx, gfp_t gfp_flags);
-int ttm_device_swapout(structttm_device*bdev, structttm_operation_ctx*ctx,
-gfp_tgfp_flags);include/drm/ttm/ttm_device.h +4
-0+long ttm_global_swapout (struct ttm_operation_ctx *ctx, + enum ttm_shrink_reason reason); + +long ttm_device_swapout(struct ttm_device *bdev, struct
ttm_operation_ctx*ctx,
+ enumttm_shrink_reasonreason);
static inline structttm_resource_manager*ttm_manager_type(struct ttm_device *bdev, int mem_type)include/drm/ttm/ttm_tt.h +1
-0@@ -87,6 +87,7 @@ struct ttm_tt { #define TTM_TT_FLAG_ZERO_ALLOC BIT(1) #define TTM_TT_FLAG_EXTERNAL BIT(2) #define TTM_TT_FLAG_EXTERNAL_MAPPABLE BIT(3) +#define TTM_TT_FLAG_DONTNEED BIT(4)
#define TTM_TT_FLAG_PRIV_POPULATED BIT(31)
uint32_tpage_flags;include/drm/ttm/ttm_tt.h +2
-2@@ -180,8 +181,8 @@ void ttm_tt_destroy(struct ttm_device *bdev,struct ttm_tt *ttm); * Swap in a previously swap out ttm_tt. */ int ttm_tt_swapin(struct ttm_tt *ttm); -int ttm_tt_swapout (struct ttm_device *bdev,struct ttm_tt *ttm, - gfp_t gfp_flags); + +int ttm_tt_swapout (struct ttm_device *bdev,struct ttm_tt *ttm);
/**
*ttm_tt_populate- allocate pages for a ttminclude/drm/ttm/ttm_tt.h +0
-0@@ -223,6 +224,18 @@ void ttm_tt_mgr_init(unsigned long num_pages, unsigned long num_dma32_pages);struct
ttm_kmap_iter*ttm_kmap_iter_tt_init(structttm_kmap_iter_tt*iter_tt,
structttm_tt*tt);
+/**
+ *ttm_tt_purgeable()- Whether a struct ttm_tt's contents is purgeable
+ * @tt: The structttm_ttto consider.
+ *
+ * Return: Whether the contents is purgeable in the sence that the owner
+ * doesn't mind losing it as long as it gets notified.
+ */include/drm/ttm/ttm_tt.h +5
-0+static inline bool ttm_tt_purgeable (struct ttm_tt *tt) +{ + return tt->page_flags & TTM_TT_FLAG_DONTNEED; +} +#if
IS_ENABLED(CONFIG_AGP)
#include <linux/agp_backend.h>
Re: [RFC PATCH 02/16] drm/ttm/pool: Fix ttm_pool_alloc error path
In reply to Thomas HellströmAm 15.02.23 um 19:02 schrieb Thomas Hellström:
Quoted 7 lines · Christian König
Quoted 4 lines
Am 15.02.23 um 17:13 schrieb Thomas Hellström:
Quoted 2 lines
When hitting an error, the error path forgot to unmap dma mappings and
I don't see where this happens?
From what I can tell,
ttm_pool_page_allocated()maps the page for dma,
If we later hit an error,ttm_pool_free_page()will leak the mapping.
Ah, I see. Good point.
Quoted 10 lines
Quoted 3 lines
Quoted 2 lines
could call
set_pages_wb()on already uncached pages.Yeah, but what's the problem?
Umm, at least if you try to set WC on an already WC'd page, the
set_pages_code will spam dmesg with warnings.
Not sure ifset_pages_wb()on WB pages does the same, nor if it issues unnecessary global cache / tlb flushes or whether that will change in the future.
The point of avoiding theset_pages_wb()when already WB is you don't have to check, and you don't have to care.
Please just open code the error handling then. That helper function looks horrible complicated to me.
Alternatively we could have a free function for a range of pages.
Regards,
Christian.
Quoted 193 lines
That said, the
__ttm_pool_free()is used also in upcoming patches./Thomas
Quoted 187 lines
Regards,
Christian.Quoted 184 lines
Fix this by introducing a common
__ttm_pool_free()function that does the right thing.Fixes d099fc8f540a ("drm/ttm: new TT backend allocation pool v3")Cc Christian König <[email protected]>Cc Dave Airlie <[email protected]>Cc Madhav Chauhan <[email protected]>Cc Christian Koenig <[email protected]>Cc Huang Rui <[email protected]>Signed-off-by Thomas Hellström <[email protected]>--- drivers/gpu/drm/ttm/ttm_pool.c | 74 +++++++++++++++++++++-------- ----- 1 file changed, 45 insertions(+), 29 deletions(-)drivers/gpu/drm/ttm/ttm_pool.c +30
-0@@ -367,6 +367,39 @@ static int ttm_pool_page_allocated(struct ttm_pool *pool, unsigned int order, return 0; } +static void __ttm_pool_free (struct ttm_pool *pool,struct ttm_tt *tt, + struct page **caching_divide, + enum ttm_caching initial_caching, + enum ttm_caching subseq_caching, + pgoff_t num_pages) +{ + enum ttm_caching caching = subseq_caching; + struct page **pages = tt->pages; + unsigned int order; + pgoff_t i, nr; + + if (pool && caching_divide) + caching = initial_caching; + + for (i = 0; i < num_pages; i += nr, pages += nr) { + struct ttm_pool_type *pt = NULL; + + if (unlikely(caching_divide == pages)) + caching = subseq_caching; + + order = ttm_pool_page_order(pool, *pages); + nr = (1UL << order); + if (tt->dma_address) + ttm_pool_unmap(pool, tt->dma_address[i], nr); + + pt = ttm_pool_select_type(pool, caching, order); + if (pt) + ttm_pool_type_give(pt, *pages); + else + ttm_pool_free_page(pool, caching, order,*pages);
+ }drivers/gpu/drm/ttm/ttm_pool.c +5
-1+} + /*** ttm_pool_alloc - Fill a ttm_tt object * @@ -386,8 +419,9 @@ int ttm_pool_alloc(struct ttm_pool *pool,struct ttm_tt *tt, dma_addr_t *dma_addr = tt->dma_address; struct page **caching = tt->pages; struct page **pages = tt->pages; + enum ttm_caching page_caching; gfp_t gfp_flags = GFP_USER; - unsigned int i, order; + unsigned int order; struct page *p; int r;@@ -410,6 +444,7 @@ int ttm_pool_alloc(struct ttm_pool *pool,struct ttm_tt *tt, order = min_t(unsigned int, order, __fls(num_pages))) { struct ttm_pool_type *pt; + page_caching = tt->caching; pt = ttm_pool_select_type(pool, tt->caching,order);
p = pt ?ttm_pool_type_take(pt): NULL; if (p) {drivers/gpu/drm/ttm/ttm_pool.c +1
-0@@ -418,6 +453,7 @@ int ttm_pool_alloc(struct ttm_pool *pool,struct ttm_tt *tt, if (r) goto error_free_page; + caching = pages; do { r = ttm_pool_page_allocated(pool,order, p,
&dma_addr,drivers/gpu/drm/ttm/ttm_pool.c +2
-1@@ -426,14 +462,15 @@ int ttm_pool_alloc(struct ttm_pool *pool,struct ttm_tt *tt, if (r) goto error_free_page; + caching = pages; if (num_pages < (1 << order)) break; p = ttm_pool_type_take(pt); } while (p); - caching = pages; } + page_caching = ttm_cached; while (num_pages >= (1 << order) && (p = ttm_pool_alloc_page(pool, gfp_flags,order))) {
drivers/gpu/drm/ttm/ttm_pool.c +1
-0@@ -442,6 +479,7 @@ int ttm_pool_alloc(struct ttm_pool *pool,struct ttm_tt *tt, tt- > caching); if (r) goto error_free_page; + caching = pages; }r = ttm_pool_page_allocated(pool, order, p,
&dma_addr,
&num_pages,
&pages);drivers/gpu/drm/ttm/ttm_pool.c +3
-6@@ -468,15 +506,12 @@ int ttm_pool_alloc(struct ttm_pool *pool,struct ttm_tt *tt, return 0; error_free_page: - ttm_pool_free_page(pool, tt->caching, order, p); + ttm_pool_free_page(pool, page_caching, order, p); error_free_all: num_pages = tt->num_pages - num_pages; - for (i = 0; i < num_pages; ) { - order = ttm_pool_page_order(pool, tt->pages[i]); - ttm_pool_free_page(pool, tt->caching, order, tt- > pages[i]); - i += 1 << order; - } + __ttm_pool_free(pool, tt, caching, tt->caching, ttm_cached, + num_pages);return r;
}drivers/gpu/drm/ttm/ttm_pool.c +2
-21@@ -492,27 +527,8 @@ EXPORT_SYMBOL(ttm_pool_alloc);*/ void ttm_pool_free(struct ttm_pool *pool, struct ttm_tt *tt) { - unsigned int i; - - for (i = 0; i < tt->num_pages; ) { - struct page *p = tt->pages[i]; - unsigned int order, num_pages; - struct ttm_pool_type *pt; - - order = ttm_pool_page_order(pool, p); - num_pages = 1ULL << order; - if (tt->dma_address) - ttm_pool_unmap(pool, tt->dma_address[i], num_pages); - - pt = ttm_pool_select_type(pool, tt->caching, order); - if (pt) - ttm_pool_type_give(pt, tt->pages[i]); - else - ttm_pool_free_page(pool, tt->caching, order, - tt->pages[i]); - - i += num_pages; - } + __ttm_pool_free(pool, tt, NULL, tt->caching, tt->caching, + tt->num_pages);while (
atomic_long_read(&allocated_pages)>page_pool_size)ttm_pool_shrink();
Re: [RFC PATCH 07/16] drm/ttm: Reduce the number of used allocation orders for TTM pages
In reply to Thomas HellströmAm 15.02.23 um 19:12 schrieb Thomas Hellström:
Quoted 9 lines · Christian König
Quoted 8 lines
Am 15.02.23 um 17:13 schrieb Thomas Hellström:
Quoted 6 lines
When swapping out, we will split multi-order pages both in order to move them to the swap-cache and to be able to return memory to the swap cache as soon as possible on a page-by-page basis. By reducing the page max order to the system PMD size, we can be nicer
to the system and avoid splitting gigantic pages.
Quoted 6 lines
Quoted 5 lines
Quoted 4 lines
On top of this we also
include the 64K page size in the page sizes tried, since that appears to
be a common size for GPU applications.Please completely drop that.
You mean the 64K page size, or the whole patch?
The 64K page size. This was an invention from Microsoft to standardize GPU handling ~15-20years ago.
It turned out to be a complete shipwreck and by now 2MiB and 1GiB pages or just flexible hardware which can handle everything seem to become standard.
Quoted 4 lines
Quoted 2 lines
This is just nonsense spilling in from the
Windows drivers.Agreed, but IIRC on the last RFC you asked me not to drop the 64K pages, so that's why they are here. I can remove them if needed.
We could keep it if it's in any way beneficial, but I'm pretty sure I must have been drunk to ask for that.
Quoted 3 lines
The only reason for keeping them from a performance point of view is better efficiency on GPUs with 64K page size if not using a coalescing IOMMU for dma-mapping.
Are any of those still produced? As far as I know neither NVidia, Intel nor AMD still assumes that page size in their hardware for quite a while now.
Regards,
Christian.
Quoted 184 lines
Let me know what you think is best and I'll adjust accordingly.
/Thomas
Quoted 178 lines
Christian.
Quoted 176 lines
Looking forward to when we might be able to swap out PMD size folios
without splitting, this will also be a benefit.Signed-off-by Thomas Hellström <[email protected]>--- drivers/gpu/drm/ttm/ttm_pool.c | 58 ++++++++++++++++++++++++++--- ----- 1 file changed, 45 insertions(+), 13 deletions(-)drivers/gpu/drm/ttm/ttm_pool.c +20
-4@@ -31,6 +31,8 @@* cause they are rather slow compared to alloc_pages+map. */ +#define pr_fmt (fmt) "[TTM POOL] " fmt + #include <linux/module.h> #include <linux/dma-mapping.h> #include <linux/debugfs.h>@@ -47,6 +49,18 @@ #include "ttm_module.h" +#define TTM_MAX_ORDER (PMD_SHIFT - PAGE_SHIFT) +#define TTM_64K_ORDER (16 - PAGE_SHIFT) +#if (TTM_MAX_ORDER <TTM_64K_ORDER ) +#undef TTM_MAX_ORDER +#define TTM_MAX_ORDER TTM_64K_ORDER +#endif +#if ((MAX_ORDER -1 ) <TTM_MAX_ORDER ) +#undef TTM_MAX_ORDER +#define TTM_MAX_ORDER (MAX_ORDER - 1) +#endif +#define TTM_DIM_ORDER (TTM_MAX_ORDER + 1) + /*** struct ttm_pool_dma - Helper object for coherent DMA mappings * @@ -65,16 +79,18 @@ module_param(page_pool_size, ulong, 0644); static atomic_long_t allocated_pages; -static struct ttm_pool_type global_write_combined[MAX_ORDER ]; -static struct ttm_pool_type global_uncached[MAX_ORDER ]; +static struct ttm_pool_type global_write_combined[TTM_DIM_ORDER ]; +static struct ttm_pool_type global_uncached[TTM_DIM_ORDER ]; -static struct ttm_pool_type global_dma32_write_combined[MAX_ORDER ]; -static struct ttm_pool_type global_dma32_uncached[MAX_ORDER ]; +static struct ttm_pool_type global_dma32_write_combined[TTM_DIM_ORDER ]; +static struct ttm_pool_type global_dma32_uncached[TTM_DIM_ORDER ]; static spinlock_t shrinker_lock; static struct list_head shrinker_list; static struct shrinker mm_shrinker; +static unsigned int ttm_pool_orders[] = {TTM_MAX_ORDER ,0 ,0 }; + /* Allocate pages of size 1 << order with the given gfp_flags */static struct page *ttm_pool_alloc_page(struct
ttm_pool*pool,gfp_tgfp_flags,
unsigned int order)drivers/gpu/drm/ttm/ttm_pool.c +11
-0@@ -400,6 +416,17 @@ static void __ttm_pool_free(struct ttm_pool *pool, struct ttm_tt *tt, } } +static unsigned int ttm_pool_select_order (unsigned int order,pgoff_t num_pages) +{ + unsigned int *cur_order = ttm_pool_orders; + + order = min_t(unsigned int, __fls(num_pages), order); + while (order < *cur_order) + ++cur_order; + + return *cur_order; +} + /***
ttm_pool_alloc- Fill attm_ttobject
*drivers/gpu/drm/ttm/ttm_pool.c +1
-3@@ -439,9 +466,8 @@ int ttm_pool_alloc(struct ttm_pool *pool,struct ttm_tt *tt, else gfp_flags |= GFP_HIGHUSER; - for (order = min_t(unsigned int, MAX_ORDER - 1, __fls(num_pages)); - num_pages; - order = min_t(unsigned int, order, __fls(num_pages))) { + order = ttm_pool_select_order(ttm_pool_orders[0],
num_pages);
+ for (;num_pages; order =ttm_pool_select_order(order, num_pages)) {
structttm_pool_type*pt;page_caching =
tt->caching;drivers/gpu/drm/ttm/ttm_pool.c +1
-1@@ -558,7 +584,7 @@ void ttm_pool_init(struct ttm_pool *pool,struct device *dev, if (use_dma_alloc) { for (i = 0; i < TTM_NUM_CACHING_TYPES; ++i) - for (j = 0; j < MAX_ORDER; ++j) + for (j = 0; j < TTM_DIM_ORDER; ++j) ttm_pool_type_init(&pool- > caching[i].orders[j],pool, i, j); }
drivers/gpu/drm/ttm/ttm_pool.c +2
-2@@ -578,7 +604,7 @@ void ttm_pool_fini(struct ttm_pool *pool) if (pool->use_dma_alloc) { for (i = 0; i < TTM_NUM_CACHING_TYPES; ++i) - for (j = 0; j < MAX_ORDER; ++j) + for (j = 0; j < TTM_DIM_ORDER; ++j) ttm_pool_type_fini(&pool- > caching[i].orders[j]); }@@ -632,7 +658,7 @@ static void ttm_pool_debugfs_header(struct seq_file *m) unsigned int i; seq_puts(m, "\t "); - for (i = 0; i < MAX_ORDER; ++i) + for (i = 0; i < TTM_DIM_ORDER; ++i) seq_printf(m, " ---%2u---", i); seq_puts(m, "\n");}
drivers/gpu/drm/ttm/ttm_pool.c +1
-1@@ -643,7 +669,7 @@ static void ttm_pool_debugfs_orders(structttm_pool_type *pt, { unsigned int i; - for (i = 0; i < MAX_ORDER; ++i) + for (i = 0; i < TTM_DIM_ORDER; ++i) seq_printf(m, " %8u", ttm_pool_type_count(&pt[i])); seq_puts(m, "\n");}
drivers/gpu/drm/ttm/ttm_pool.c +7
-1@@ -749,10 +775,16 @@ int ttm_pool_mgr_init(unsigned long num_pages) if (!page_pool_size) page_pool_size = num_pages; + if (TTM_64K_ORDER < TTM_MAX_ORDER) + ttm_pool_orders[1] = TTM_64K_ORDER; + + pr_debug("Used orders are %u %u %u\n", ttm_pool_orders[0], + ttm_pool_orders[1], ttm_pool_orders[2]); + spin_lock_init(&shrinker_lock); INIT_LIST_HEAD(&shrinker_list); - for (i = 0; i < MAX_ORDER; ++i) { + for (i = 0; i < TTM_DIM_ORDER; ++i) { ttm_pool_type_init(&global_write_combined[i], NULL, ttm_write_combined, i);ttm_pool_type_init(&global_uncached[i], NULL,
ttm_uncached, i);drivers/gpu/drm/ttm/ttm_pool.c +1
-1@@ -785,7 +817,7 @@ void ttm_pool_mgr_fini(void) { unsigned int i; - for (i = 0; i < MAX_ORDER; ++i) { + for (i = 0; i < TTM_DIM_ORDER; ++i) { ttm_pool_type_fini(&global_write_combined[i]); ttm_pool_type_fini(&global_uncached[i]);
Re: [RFC PATCH 04/16] drm/ttm, drm/vmwgfx: Update the TTM swapout interface
In reply to Thomas HellströmAm 15.02.23 um 19:19 schrieb Thomas Hellström:
Quoted 24 lines · Christian König
Quoted 15 lines
Am 15.02.23 um 17:13 schrieb Thomas Hellström:
Quoted 6 lines
Update the TTM swapout interfaces for better compatibility with a shrinker.
- Replace number-of-pages int return with a long to better match the
kernel's shrinker interface.
- Thegfp_flagsparameter tottm_xx_swapout()currently only takesQuoted 8 lines
the
GFP_KERNELvalue and shouldn't really be needed since the
shrinker we
hook up in upcoming patches sets a allocation context to match
reclaim.
- Introduce a shrink reason enumeration and a driver callback to shrink
buffer objects.Quoted 4 lines
Is that really necessary? This is mid-layering once more.
If drivers want to implement driver specific shrinking they should register their own shrinker callback.
Yes, a choice needs to be made here. If TTM registers the shrinker, the driver needs to be called at least to unbind and to remove dma- mappings.
Quoted 5 lines
If the driver registers the shrinker it can still (I think) use the pool helpers, but needs TTM for LRU traversal and accounting.
I can have a look at the latter if yout think that will be a better solution.
Yeah, that's what I had in mind as well. Something like the drivers registers the shrinker and TTM provides the function to give a candidate for eviction.
Christian.
Quoted 473 lines
/Thomas
Quoted 469 lines
Christian.
Quoted 466 lines
The
TTM_SHRINK_WATERMARKreason is going to still be handled
using the
existing shmem copy, and will be used by pool types that don't
lend
themselves well to shinking (dma_allocpool) and when drivers
explicitly
requests swapout.
TheTTM_SHRINK_SWAPandTTM_SHRINK_PURGEreasons originate from
a
shrinker and is to be handled by a new driver callback,bo_shrink().
Helpers for the new driver callback are provided in upcoming
patches.Signed-off-by Thomas Hellström <[email protected]>--- drivers/gpu/drm/ttm/ttm_bo.c | 38 ++++++++++++++++---- drivers/gpu/drm/ttm/ttm_device.c | 55 +++++++++++++++++++++--- ----- drivers/gpu/drm/ttm/ttm_tt.c | 23 ++++++------ drivers/gpu/drm/vmwgfx/vmwgfx_drv.c | 3 +- include/drm/ttm/ttm_bo.h | 4 +-- include/drm/ttm/ttm_device.h | 36 +++++++++++++++++-- include/drm/ttm/ttm_tt.h | 17 +++++++-- 7 files changed, 136 insertions(+), 40 deletions(-)drivers/gpu/drm/ttm/ttm_bo.c +7
-2@@ -1114,13 +1114,29 @@ int ttm_bo_wait_ctx(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx) } EXPORT_SYMBOL(ttm_bo_wait_ctx); -int ttm_bo_swapout (struct ttm_buffer_object *bo,struct ttm_operation_ctx *ctx, - gfp_t gfp_flags) +/** +* ttm_bo_swapout() - Swap out or purge a buffer object +* @bo: The buffer object. +* @ctx: The ttm operation context. +* @reason: The swapout reason. +* +* Try to swap out or purge the contents of a system memory backed buffer
+ * object. The function needs to be called with the device's LRU lock held.
+ *
+ * Return:-EBUSYif the bo lock could not be grabbed or the object was
+ * otherwise busy. Otherwise the number of pages swapped out or negative
+ * error code on error. Iff the function didn't return-EBUSY, the
+ * LRU lock was dropped, and LRU traversal needs to restart.
+ */
+long ttm_bo_swapout(structttm_buffer_object*bo, structttm_operation_ctx*ctx,
+ enumttm_shrink_reasonreason)
{
structttm_placeplace;
bool locked;
long ret;+ lockdep_assert_held(&bo->bdev->lru_lock);
+
/*
* While the bo may already reside in SYSTEM placement, set
* SYSTEM as new placement to cover also the move further
below.drivers/gpu/drm/ttm/ttm_bo.c +4
-0@@ -1142,8 +1158,12 @@ int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx, } if (bo->deleted) { + long num_pages = bo->ttm->num_pages; + ret = ttm_bo_cleanup_refs(bo, false, false, locked); ttm_bo_put(bo); + if (!ret) + return num_pages; return ret == -EBUSY ? -ENOSPC : ret;}
drivers/gpu/drm/ttm/ttm_bo.c +9
-5@@ -1184,13 +1204,17 @@ int ttm_bo_swapout(struct ttm_buffer_object*bo, struct ttm_operation_ctx *ctx, * Swap out. Buffer will be swapped in again as soon as * anyone tries to access a ttm page. */ - if (bo->bdev->funcs->swap_notify) - bo->bdev->funcs->swap_notify(bo); + if (bo->bdev->funcs->bo_shrink && reason !=TTM_SHRINK_WATERMARK ) { + ret = bo->bdev->funcs->bo_shrink(bo, ctx); + } else { + if (bo->bdev->funcs->swap_notify) + bo->bdev->funcs->swap_notify(bo); + ret = ttm_tt_swapout(bo->bdev, bo->ttm); + if (!ret) + ret = bo->ttm->num_pages; + } - if (ttm_tt_is_populated(bo->ttm)) - ret = ttm_tt_swapout(bo->bdev, bo->ttm, gfp_flags); out: - /** Unreserve without putting on LRU to avoid swapping out
an
* already swapped buffer.drivers/gpu/drm/ttm/ttm_device.c +2
-3@@ -116,19 +116,28 @@ static int ttm_global_init(void) return ret; } -/* -* A buffer object shrink method that tries to swap out the first -* buffer object on the global::swap_lru list. +/** +* ttm_global_swapout() - Select and swap out a system-memory- backed bo.
+ * @ctx: The operation context.
+ * @reason: The reason for swapout.
+ *
+ * Select, based on round-robin a TTM device and traverse the LRUs of
+ * that specific device until a suitable bo backed by system memory is found
+ * and swapped-out or purged.
+ *
+ * Return: Positive value or zero indicating the size in pages of the
+ * bo swapped out. Negative error code on error.
*/
-intttm_global_swapout(struct ttm_operation_ctx *ctx, gfp_t gfp_flags)
+long ttm_global_swapout(structttm_operation_ctx*ctx, + enumttm_shrink_reasonreason)
{
structttm_global*glob =&ttm_glob;
structttm_device*bdev;
- int ret = 0;
+ long ret = 0;mutex_lock(&ttm_global_mutex);
list_for_each_entry(bdev, &glob->device_list, device_list)
{
- ret =ttm_device_swapout(bdev, ctx, gfp_flags); + ret =ttm_device_swapout(bdev, ctx, reason);
if (ret > 0) { list_move_tail(&bdev->device_list, &glob-Quoted 1 line
device_list);break;
drivers/gpu/drm/ttm/ttm_device.c +2
-2@@ -139,14 +148,29 @@ int ttm_global_swapout(struct ttm_operation_ctx *ctx, gfp_t gfp_flags) } EXPORT_SYMBOL(ttm_global_swapout); -int ttm_device_swapout (struct ttm_device *bdev,struct ttm_operation_ctx *ctx, - gfp_t gfp_flags) +/** +* ttm_device_swapout() - Select and swap out a system-memory- backed bo.
+ * @bdev: The device whos bos are considered for swapout.
+ * @ctx: The operation context.
+ * @reason: The reason for swapout.
+ *
+ * Traverse the LRUs of a specific device until a suitable bo backed by
+ * system memory is found and swapped-out or purged.
+ *
+ * Return: Positive value or zero indicating the size in pages of the
+ * bo swapped out. Negative error code on error.
+ */
+long ttm_device_swapout(structttm_device*bdev, structttm_operation_ctx*ctx,
+ enumttm_shrink_reasonreason)
{
structttm_resource_cursorcursor;
structttm_resource_manager*man;
structttm_resource*res;
unsigned i;
- int ret;
+ long ret;
+
+ if (reason !=TTM_SHRINK_WATERMARK&& !bdev->funcs-Quoted 1 line
bo_shrink)+ return 0;
spin_lock(&bdev->lru_lock);
for (i =TTM_PL_SYSTEM; i <TTM_NUM_MEM_TYPES; ++i) {drivers/gpu/drm/ttm/ttm_device.c +8
-5@@ -156,16 +180,19 @@ int ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx, ttm_resource_manager_for_each_res(man, &cursor, res) { struct ttm_buffer_object *bo = res->bo; - uint32_t num_pages; + struct ttm_tt *tt; if (!bo || bo->resource != res) continue; - num_pages = PFN_UP(bo->base.size); - ret = ttm_bo_swapout(bo, ctx, gfp_flags); + tt = bo->ttm; + if (!tt || (reason == TTM_SHRINK_PURGE && + !ttm_tt_purgeable(tt))) + continue; + ret = ttm_bo_swapout(bo, ctx, reason); /* ttm_bo_swapout has dropped the lru_lock */ - if (!ret) - return num_pages; + if (ret >= 0) + return ret;if (ret !=
-EBUSY)
return ret;
}drivers/gpu/drm/ttm/ttm_tt.c +0
-3@@ -239,22 +239,21 @@ int ttm_tt_swapin(struct ttm_tt *ttm) /*** ttm_tt_swapout - swap out tt object -* * @bdev: TTM device structure. * @ttm: The struct ttm_tt. -* @gfp_flags: Flags to use for memory allocation. * -* Swapout a TT object to a shmem_file, return number of pages swapped out or
- * negative error code.
+ * Swapout a TT object to ashmem_file.
+ *
+ * Return: number of pages swapped out or negative error code on error.
*/drivers/gpu/drm/ttm/ttm_tt.c +3
-3-int ttm_tt_swapout (struct ttm_device *bdev,struct ttm_tt *ttm, - gfp_t gfp_flags) +int ttm_tt_swapout (struct ttm_device *bdev,struct ttm_tt *ttm) { loff_t size = (loff_t)ttm->num_pages << PAGE_SHIFT; struct address_space *swap_space; struct file *swap_storage; struct page *from_page; struct page *to_page; + gfp_t gfp_flags; int i, ret; swap_storage = shmem_file_setup("ttm swap", size, 0);@@ -264,7 +263,7 @@ int ttm_tt_swapout(struct ttm_device *bdev,struct ttm_tt *ttm, } swap_space = swap_storage->f_mapping; - gfp_flags &= mapping_gfp_mask(swap_space); + gfp_flags = GFP_KERNEL & mapping_gfp_mask(swap_space); for (i = 0; i < ttm->num_pages; ++i) {from_page = ttm->pages[i];
drivers/gpu/drm/ttm/ttm_tt.c +0
-0@@ -315,12 +314,14 @@ int ttm_tt_populate(struct ttm_device *bdev, while (atomic_long_read(&ttm_pages_allocated) >
ttm_pages_limit||
atomic_long_read(&ttm_dma32_pages_allocated)>ttm_dma32_pages_limit) {
+ long r =ttm_global_swapout(ctx, TTM_SHRINK_WATERMARK);- ret =
ttm_global_swapout(ctx, GFP_KERNEL); - if (ret == 0)
+ if (!r)
break;
- if (ret < 0)
+ if (r < 0) {
+ ret = r;
goto error;
+ }
}if (
bdev->funcs->ttm_tt_populate)drivers/gpu/drm/ttm/ttm_tt.c +1
-1@@ -379,7 +380,7 @@ static int ttm_tt_debugfs_shrink_show(struct seq_file *m, void *data) { struct ttm_operation_ctx ctx = { false, false }; - seq_printf(m, "%d\n", ttm_global_swapout(&ctx, GFP_KERNEL)); + seq_printf(m, "%ld\n", ttm_global_swapout(&ctx,TTM_SHRINK_SWAP ));return 0;
}
DEFINE_SHOW_ATTRIBUTE(ttm_tt_debugfs_shrink);drivers/gpu/drm/vmwgfx/vmwgfx_drv.c +2
-1@@ -1514,7 +1514,8 @@ static int vmw_pm_freeze(struct device *kdev) vmw_execbuf_release_pinned_bo(dev_priv); vmw_resource_evict_all(dev_priv); vmw_release_device_early(dev_priv); - while (ttm_device_swapout(&dev_priv->bdev, &ctx, GFP_KERNEL) > 0); + while (ttm_device_swapout(&dev_priv->bdev, &ctx,TTM_SHRINK_WATERMARK ) >0 ) + ; vmw_fifo_resource_dec(dev_priv);if (
atomic_read(&dev_priv->num_fifo_resources)!= 0) { DRM_ERROR("Can't hibernate while 3D resources are
active.\n");include/drm/ttm/ttm_bo.h +0
-1@@ -375,8 +375,8 @@ void ttm_bo_kunmap(struct ttm_bo_kmap_obj *map); int ttm_bo_vmap(struct ttm_buffer_object *bo, struct iosys_map *map); void ttm_bo_vunmap(struct ttm_buffer_object *bo, struct iosys_map *map); int ttm_bo_mmap_obj(struct vm_area_struct *vma, structttm_buffer_object *bo); -int ttm_bo_swapout(struct ttm_buffer_object *bo, struct
ttm_operation_ctx*ctx,
-gfp_tgfp_flags);
+long ttm_bo_swapout(structttm_buffer_object*bo, structttm_operation_ctx*ctx,
+ enumttm_shrink_reasonreason);
voidttm_bo_pin(struct ttm_buffer_object *bo); voidttm_bo_unpin(struct ttm_buffer_object *bo); int ttm_mem_evict_first(structttm_device*bdev,include/drm/ttm/ttm_device.h +3
-0@@ -35,6 +35,21 @@ struct ttm_placement; struct ttm_buffer_object; struct ttm_operation_ctx; +/** +* enum ttm_shrink_reason - Reason for shrinking system memory +* @TTM_SHRINK_WATERMARK - A watermark limit was reached. Not from reclaim.
+ * @TTM_SHRINK_PURGE - A request for shrinking only purged objects.
+ * @TTM_SHRINK_SWAP - A request for shrinking any object.
+ *
+ * This enum is intended for the buffer object- and shrink method selection
+ * algorithms. It's not intended to leak to or be used by TTM drivers.
+ */include/drm/ttm/ttm_device.h +9
-0+enum ttm_shrink_reason { + TTM_SHRINK_WATERMARK, + TTM_SHRINK_PURGE, + TTM_SHRINK_SWAP, +}; + /*** struct ttm_global - Buffer object driver global data. */ @@ -207,6 +222,19 @@ struct ttm_device_funcs {* adding fences that may force a delayed delete */ void (*release_notify)(struct ttm_buffer_object *bo); + + /** +* Shrink the bo's system pages, Either by swapping or by purging.
+ * @bo: Bo the system pages of which are to be shrunken. + * @ctx: Operation ctx. In particular the driver callback should
+ * adhere to theno_wait_gpuand interruptible fields.
+ *
+ * This is also notifying the driver that the bo is about to be
+ * shrunken and the driver should take care to unbind any GPU bindings
+ * and to note that the content is purged if @bo->ttm is purgeable.
+ */
+ long (*bo_shrink)(structttm_buffer_object*bo, + structttm_operation_ctx*ctx);
};/**
include/drm/ttm/ttm_device.h +5
-3@@ -268,9 +296,11 @@ struct ttm_device { struct workqueue_struct *wq; }; -int ttm_global_swapout (struct ttm_operation_ctx *ctx,gfp_t gfp_flags); -int ttm_device_swapout (struct ttm_device *bdev,struct ttm_operation_ctx *ctx, - gfp_t gfp_flags); +long ttm_global_swapout(struct ttm_operation_ctx *ctx, + enum ttm_shrink_reason reason); + +long ttm_device_swapout (struct ttm_device *bdev,struct ttm_operation_ctx *ctx, + enum ttm_shrink_reason reason);static inline struct
ttm_resource_manager*ttm_manager_type(struct ttm_device *bdev, int mem_type)include/drm/ttm/ttm_tt.h +3
-2@@ -87,6 +87,7 @@ struct ttm_tt { #define TTM_TT_FLAG_ZERO_ALLOC BIT(1) #define TTM_TT_FLAG_EXTERNAL BIT(2) #define TTM_TT_FLAG_EXTERNAL_MAPPABLE BIT(3) +#define TTM_TT_FLAG_DONTNEED BIT(4) #define TTM_TT_FLAG_PRIV_POPULATED BIT(31) uint32_t page_flags;@@ -180,8 +181,8 @@ void ttm_tt_destroy(struct ttm_device *bdev,struct ttm_tt *ttm); * Swap in a previously swap out ttm_tt. */ int ttm_tt_swapin(struct ttm_tt *ttm); -int ttm_tt_swapout (struct ttm_device *bdev,struct ttm_tt *ttm, - gfp_t gfp_flags); + +int ttm_tt_swapout (struct ttm_device *bdev,struct ttm_tt *ttm); /***
ttm_tt_populate- allocate pages for a ttminclude/drm/ttm/ttm_tt.h +2
-0@@ -223,6 +224,18 @@ void ttm_tt_mgr_init(unsigned long num_pages, unsigned long num_dma32_pages); struct ttm_kmap_iter *ttm_kmap_iter_tt_init(structttm_kmap_iter_tt *iter_tt, struct ttm_tt *tt); +/** +* ttm_tt_purgeable() - Whether a struct ttm_tt's contents is purgeable
+ * @tt: The structttm_ttto consider.
+ *
+ * Return: Whether the contents is purgeable in the sence that the owner
+ * doesn't mind losing it as long as it gets notified.
+ */include/drm/ttm/ttm_tt.h +5
-0+static inline bool ttm_tt_purgeable (struct ttm_tt *tt) +{ + return tt->page_flags & TTM_TT_FLAG_DONTNEED; +} + #if IS_ENABLED(CONFIG_AGP) #include <linux/agp_backend.h>
Thomas Hellström
reviewerRe: [RFC PATCH 02/16] drm/ttm/pool: Fix ttm_pool_alloc error path
In reply to Christian KönigQuoted 26 lines · Christian König
Am 15.02.23 um 19:02 schrieb Thomas Hellström:
Quoted 10 lines · Christian König
Quoted 5 lines
Am 15.02.23 um 17:13 schrieb Thomas Hellström:
Quoted 3 lines
When hitting an error, the error path forgot to unmap dma mappings
andI don't see where this happens?
From what I can tell,
ttm_pool_page_allocated()maps the page for dma,
If we later hit an error,ttm_pool_free_page()will leak the mapping.Ah, I see. Good point.
Quoted 11 lines
Quoted 3 lines
Quoted 2 lines
could call
set_pages_wb()on already uncached pages.Yeah, but what's the problem?
Umm, at least if you try to set WC on an already WC'd page, the
set_pages_code will spam dmesg with warnings.
Not sure ifset_pages_wb()on WB pages does the same, nor if it issues unnecessary global cache / tlb flushes or whether that will change in the future.
The point of avoiding theset_pages_wb()when already WB is you don't
have to check, and you don't have to care.
Quoted 4 lines
Please just open code the error handling then. That helper function looks horrible complicated to me.
Alternatively we could have a free function for a range of pages.
OK, I'll see if this is doable without adding a tremendous amount of code.
/Thomas
Quoted 218 lines
Regards,
Christian.Quoted 212 lines
That said, the
__ttm_pool_free()is used also in upcoming patches./Thomas
Quoted 206 lines
Regards,
Christian.Quoted 203 lines
Fix this by introducing a common
__ttm_pool_free()function that
does the right thing.Fixes d099fc8f540a ("drm/ttm: new TT backend allocation poolv3")
Cc: Christian König <[email protected]>
Cc: Dave Airlie <[email protected]>
Cc: Madhav Chauhan <[email protected]>
Cc: Christian Koenig <[email protected]>
Cc: Huang Rui <[email protected]>
Cc: [email protected]
Signed-off-by: Thomas Hellström <[email protected]>
---
drivers/gpu/drm/ttm/ttm_pool.c| 74 +++++++++++++++++++++---
-----
-----1 file changed, 45 insertions(+), 29 deletions(-)drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -367,6 +367,39 @@ static int ttm_pool_page_allocated(struct ttm_pool *pool, unsigned int order, return 0; }
+static void __ttm_pool_free(structttm_pool*pool, structttm_tt
*tt,
+ struct page **caching_divide, + enumttm_cachinginitial_caching, + enumttm_cachingsubseq_caching, +pgoff_tnum_pages)drivers/gpu/drm/ttm/ttm_pool.c +19
-0+{ + enum ttm_caching caching = subseq_caching; + struct page **pages = tt->pages; + unsigned int order; + pgoff_t i, nr; + + if (pool && caching_divide) + caching = initial_caching; + + for (i = 0; i < num_pages; i += nr, pages += nr) { + struct ttm_pool_type *pt = NULL; + + if (unlikely(caching_divide == pages)) + caching = subseq_caching; + + order = ttm_pool_page_order(pool, *pages); + nr = (1UL << order); + if (tt->dma_address) + ttm_pool_unmap(pool, tt-Quoted 1 line
dma_address[i],
nr);
+
+ pt =ttm_pool_select_type(pool, caching, order);
+ if (pt)
+ ttm_pool_type_give(pt, *pages); + else
+ ttm_pool_free_page(pool, caching, order,
*pages);
+ }drivers/gpu/drm/ttm/ttm_pool.c +2
-0+} +/**
*ttm_pool_alloc- Fill attm_ttobject *drivers/gpu/drm/ttm/ttm_pool.c +2
-1@@ -386,8 +419,9 @@ int ttm_pool_alloc(struct ttm_pool *pool,struct ttm_tt *tt, dma_addr_t *dma_addr = tt->dma_address; struct page **caching = tt->pages; struct page **pages = tt->pages; + enum ttm_caching page_caching; gfp_t gfp_flags = GFP_USER; - unsigned int i, order; + unsigned int order; struct page *p; int r;drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -410,6 +444,7 @@ int ttm_pool_alloc(struct ttm_pool *pool,struct ttm_tt *tt, order = min_t(unsigned int, order, __fls(num_pages))) { struct ttm_pool_type *pt;
+ page_caching =tt->caching;
pt =ttm_pool_select_type(pool, tt->caching, order);
p = pt ?ttm_pool_type_take(pt): NULL; if (p) {drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -418,6 +453,7 @@ int ttm_pool_alloc(struct ttm_pool *pool,struct ttm_tt *tt, if (r) goto error_free_page;
+ caching = pages; do {
r = ttm_pool_page_allocated(pool,
order, p,&dma_addr,drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -426,14 +462,15 @@ int ttm_pool_alloc(struct ttm_pool *pool,struct ttm_tt *tt, if (r) goto error_free_page;
+ caching = pages;
if (num_pages< (1 << order)) break;
p =ttm_pool_type_take(pt); } while (p);
- caching = pages;
}
+ page_caching =ttm_cached;
while (num_pages>= (1 << order) && (p = ttm_pool_alloc_page(pool,gfp_flags,
order))) {drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -442,6 +479,7 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,tt-
Quoted 1 line
caching);
if (r)
gotoerror_free_page; + caching = pages;
}
r = ttm_pool_page_allocated(pool, order, p,&dma_addr,&num_pages,
&pages);drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -468,15 +506,12 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt, return 0;
error_free_page:
- ttm_pool_free_page(pool,tt->caching, order, p); + ttm_pool_free_page(pool,page_caching, order, p);
error_free_all:
num_pages =tt->num_pages-num_pages; - for (i = 0; i <num_pages; ) {
- order = ttm_pool_page_order(pool, tt-Quoted 1 line
pages[i]);
- ttm_pool_free_page(pool,
tt->caching, order, tt-Quoted 1 line
pages[i]);
- i += 1 << order;
- }
+ __ttm_pool_free(pool, tt, caching,tt->caching,ttm_cached,
+ num_pages);
return r;
}drivers/gpu/drm/ttm/ttm_pool.c +1
-21@@ -492,27 +527,8 @@ EXPORT_SYMBOL(ttm_pool_alloc);*/ void ttm_pool_free(struct ttm_pool *pool, struct ttm_tt *tt) { - unsigned int i; - - for (i = 0; i < tt->num_pages; ) { - struct page *p = tt->pages[i]; - unsigned int order, num_pages; - struct ttm_pool_type *pt; - - order = ttm_pool_page_order(pool, p); - num_pages = 1ULL << order; - if (tt->dma_address) - ttm_pool_unmap(pool, tt- >dma_address[i], num_pages); - - pt = ttm_pool_select_type(pool, tt->caching, order); - if (pt) - ttm_pool_type_give(pt, tt->pages[i]); - else - ttm_pool_free_page(pool, tt->caching, order, - tt->pages[i]); - - i += num_pages; - } + __ttm_pool_free(pool, tt, NULL, tt->caching, tt-Quoted 1 line
caching,
+ tt->num_pages);
while (atomic_long_read(&allocated_pages)>page_pool_size)
ttm_pool_shrink();
Thomas Hellström
reviewerRe: [RFC PATCH 07/16] drm/ttm: Reduce the number of used allocation orders for TTM pages
In reply to Christian KönigQuoted 43 lines · Christian König
Am 15.02.23 um 19:12 schrieb Thomas Hellström:
Quoted 12 lines · Christian König
Quoted 11 lines
Am 15.02.23 um 17:13 schrieb Thomas Hellström:
Quoted 9 lines
When swapping out, we will split multi-order pages both in order to
move them to the swap-cache and to be able to return memory to the
swap cache as soon as possible on a page-by-page basis. By reducing the page max order to the system PMD size, we can be
nicer
to the system and avoid splitting gigantic pages.Quoted 6 lines
Quoted 5 lines
Quoted 4 lines
On top of this we also
include the 64K page size in the page sizes tried, since that appears to
be a common size for GPU applications.Please completely drop that.
You mean the 64K page size, or the whole patch?
The 64K page size. This was an invention from Microsoft to standardize
GPU handling ~15-20years ago.It turned out to be a complete shipwreck and by now 2MiB and 1GiB pages
or just flexible hardware which can handle everything seem to become standard.Quoted 4 lines
Quoted 2 lines
This is just nonsense spilling in from the
Windows drivers.Agreed, but IIRC on the last RFC you asked me not to drop the 64K pages, so that's why they are here. I can remove them if needed.
We could keep it if it's in any way beneficial, but I'm pretty sure I must have been drunk to ask for that.
Quoted 5 lines
The only reason for keeping them from a performance point of view is
better efficiency on GPUs with 64K page size if not using a coalescing
IOMMU for dma-mapping.
Quoted 5 lines
Are any of those still produced? As far as I know neither NVidia, Intel
nor AMD still assumes that page size in their hardware for quite a while
now.
Intel still supports 64K PTEs, so we use them where possible, otherwise falling back to 4K. Typically we have coalescing IOMMU enabled when testing, so can't really see the impact, but TBH I was surprised by the number of 64K page allocations TTM spat out with this patch series, so I definitely think there is a performance impact with !IOMMU, although I can't quantify it ATM.
So then if it's OK with you I'll keep that size for now.
/Thomas
Quoted 202 lines
Regards,
Christian.Quoted 197 lines
Let me know what you think is best and I'll adjust accordingly.
/Thomas
Quoted 191 lines
Christian.
Quoted 189 lines
Looking forward to when we might be able to swap out PMD size folios
without splitting, this will also be a benefit.Signed-off-by: Thomas Hellström
<[email protected]>
---
drivers/gpu/drm/ttm/ttm_pool.c| 58
++++++++++++++++++++++++++---
-----1 file changed, 45 insertions(+), 13 deletions(-)drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -31,6 +31,8 @@* cause they are rather slow compared to alloc_pages+map. */ drivers/gpu/drm/ttm/ttm_pool.c +2
-0+#define pr_fmt (fmt) "[TTM POOL] " fmt +#include <
linux/module.h>
#include <linux/dma-mapping.h>
#include <linux/debugfs.h>drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -47,6 +49,18 @@
#include "ttm_module.h"drivers/gpu/drm/ttm/ttm_pool.c +12
-0+#define TTM_MAX_ORDER (PMD_SHIFT - PAGE_SHIFT) +#define TTM_64K_ORDER (16 - PAGE_SHIFT) +#if (TTM_MAX_ORDER <TTM_64K_ORDER ) +#undef TTM_MAX_ORDER +#define TTM_MAX_ORDER TTM_64K_ORDER +#endif +#if ((MAX_ORDER -1 ) <TTM_MAX_ORDER ) +#undef TTM_MAX_ORDER +#define TTM_MAX_ORDER (MAX_ORDER - 1) +#endif +#define TTM_DIM_ORDER (TTM_MAX_ORDER + 1) +/**
* structttm_pool_dma- Helper object for coherent DMA mappings
*drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -65,16 +79,18 @@ module_param(page_pool_size, ulong, 0644);
staticatomic_long_tallocated_pages;drivers/gpu/drm/ttm/ttm_pool.c +1
-2-static struct ttm_pool_type global_write_combined[MAX_ORDER ]; -static struct ttm_pool_type global_uncached[MAX_ORDER ]; +static struct ttm_pool_typeglobal_write_combined[TTM_DIM_ORDER];
+static structttm_pool_typeglobal_uncached[TTM_DIM_ORDER];
-static structttm_pool_typeglobal_dma32_write_combined[MAX_ORDER];drivers/gpu/drm/ttm/ttm_pool.c +1
-1-static struct ttm_pool_type global_dma32_uncached[MAX_ORDER ]; +static struct ttm_pool_typeglobal_dma32_write_combined[TTM_DIM_ORDER];
+static structttm_pool_type
global_dma32_uncached[TTM_DIM_ORDER];
staticspinlock_tshrinker_lock;
static structlist_headshrinker_list;
static struct shrinkermm_shrinker;drivers/gpu/drm/ttm/ttm_pool.c +2
-0+static unsigned int ttm_pool_orders[] = {TTM_MAX_ORDER ,0 ,0 }; +/* Allocate pages of size 1 << order with the given
gfp_flags*/
static struct page *ttm_pool_alloc_page(structttm_pool*pool,gfp_tgfp_flags,
unsigned int order)drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -400,6 +416,17 @@ static void __ttm_pool_free(struct
ttm_pool
*pool, structttm_tt*tt,
}
}
+static unsigned intttm_pool_select_order(unsigned int order, pgoff_t num_pages)drivers/gpu/drm/ttm/ttm_pool.c +10
-0+{ + unsigned int *cur_order = ttm_pool_orders; + + order = min_t(unsigned int, __fls(num_pages), order); + while (order < *cur_order) + ++cur_order; + + return *cur_order; +} +/**
*ttm_pool_alloc- Fill attm_ttobject
*drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -439,9 +466,8 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,else
gfp_flags |=GFP_HIGHUSER;
- for (order =min_t(unsigned int, MAX_ORDER - 1, __fls(num_pages));
-num_pages;
- order =min_t(unsigned int, order, __fls(num_pages)))
{
+ order =ttm_pool_select_order(ttm_pool_orders[0], num_pages);
+ for (;num_pages; order =ttm_pool_select_order(order, num_pages)) {
structttm_pool_type*pt;
page_caching =tt->caching;drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -558,7 +584,7 @@ void ttm_pool_init(struct ttm_pool *pool, struct device *dev,
if (use_dma_alloc) {
for (i = 0; i <TTM_NUM_CACHING_TYPES; ++i) - for (j = 0; j <MAX_ORDER; ++j) + for (j = 0; j <TTM_DIM_ORDER; ++j) ttm_pool_type_init(&pool-Quoted 1 line
caching[i].orders[j],
pool, i, j);
}drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -578,7 +604,7 @@ void ttm_pool_fini(struct ttm_pool *pool)
if (pool->use_dma_alloc) {
for (i = 0; i <TTM_NUM_CACHING_TYPES; ++i) - for (j = 0; j <MAX_ORDER; ++j) + for (j = 0; j <TTM_DIM_ORDER; ++j) ttm_pool_type_fini(&pool-Quoted 1 line
caching[i].orders[j]);
}
drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -632,7 +658,7 @@ static void ttm_pool_debugfs_header(struct seq_file *m) unsigned int i;
seq_puts(m, "\t ");
- for (i = 0; i <MAX_ORDER; ++i)
+ for (i = 0; i <TTM_DIM_ORDER; ++i)
seq_printf(m, " ---%2u---", i);
seq_puts(m, "\n");
}drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -643,7 +669,7 @@ static void ttm_pool_debugfs_orders(structttm_pool_type *pt, { unsigned int i;
- for (i = 0; i <MAX_ORDER; ++i)
+ for (i = 0; i <TTM_DIM_ORDER; ++i)
seq_printf(m, " %8u",ttm_pool_type_count(&pt[i]));
seq_puts(m, "\n");
}drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -749,10 +775,16 @@ int ttm_pool_mgr_init(unsigned long num_pages) if (!page_pool_size) page_pool_size = num_pages;
+ if (TTM_64K_ORDER<TTM_MAX_ORDER)
+ ttm_pool_orders[1] =TTM_64K_ORDER; +
+ pr_debug("Used orders are %u %u %u\n", ttm_pool_orders[0],
+ ttm_pool_orders[1], ttm_pool_orders[2]); +
spin_lock_init(&shrinker_lock);INIT_LIST_HEAD(&shrinker_list);
- for (i = 0; i <MAX_ORDER; ++i) {
+ for (i = 0; i <TTM_DIM_ORDER; ++i) { ttm_pool_type_init(&global_write_combined[i], NULL,
ttm_write_combined, i);ttm_pool_type_init(&global_uncached[i], NULL, ttm_uncached, i);drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -785,7 +817,7 @@ void ttm_pool_mgr_fini(void) { unsigned int i;
- for (i = 0; i <MAX_ORDER; ++i) {
+ for (i = 0; i <TTM_DIM_ORDER; ++i) {ttm_pool_type_fini(&global_write_combined[i]);ttm_pool_type_fini(&global_uncached[i]);
Patchwork
CI[Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for Add a TTM shrinker
In reply to Thomas Hellström== Series Details ==
== Series Details ==
Series: Add a TTM shrinker
URL : https://patchwork.freedesktop.org/series/114060/
State : warning
== Summary ==
Error: dim checkpatch failed868fc3be5310 drm/ttm: Fix a NULL pointer dereference98296903c17c drm/ttm/pool: Fix ttm_pool_alloc error pathf7427cf5528b drm/ttm: Use the BIT macro for the TTM_TT_FLAGsc43883970a8f drm/ttm, drm/vmwgfx: Update the TTM swapout interface71e9cc6ac605 drm/ttm: Unexport ttm_global_swapout()28c5436faefb drm/ttm: Don't use watermark accounting on shrinkable pools 0cbaa4650e8b drm/ttm: Reduce the number of used allocation orders for TTM pages 0ffa5f3b05c3 drm/ttm: Add a shrinker and shrinker accountingd5139754ecca drm/ttm: Introduce shrink throttling.eb622f9e593b drm/ttm: Remove pinned bos from shrinkable accounting5f42659a9f7f drm/ttm: Add a simple api to set / clear purgeable ttm_tt content 2f92445365a0 mm: Add interfaces to back up and recover folio contents using swap Traceback (most recent call last):
File "scripts/spdxcheck.py", line 6, in <module>
from ply import lex, yacc
ModuleNotFoundError: No module named 'ply'
Traceback (most recent call last):
File "scripts/spdxcheck.py", line 6, in <module>
from ply import lex, yacc
ModuleNotFoundError: No module named 'ply'
-:109: WARNING:FILE_PATH_CHANGES: added, moved or deleted file(s), does MAINTAINERS need updating? #109:
new file mode 100644
total: 0 errors, 1 warnings, 0 checks, 339 lines checkede76abab95b0c drm/ttm: Make the call to ttm_tt_populate() interruptible when faulting fdd0475a9c94 drm/ttm: Provide helpers for shrinking629412555d1a drm/ttm: Use fault-injection to test error paths
-:24: WARNING:CONFIG_DESCRIPTION: please write a help paragraph that fully describes the config symbol #24: FILE: drivers/gpu/drm/Kconfig:205:
+config DRM_TTM_SHRINK_FAULT_INJECT
+ bool "Enable fault injection during TTM shrinking"
+ depends on DRM_TTM
+ default n
+ help
+ Inject recoverable failures during TTM shrinking and recovery of
+ shrunken objects. For DRM driver developers only.
+
+ If in doubt, choose N.
+
total: 0 errors, 1 warnings, 0 checks, 51 lines checked714409e46b58 drm/i915, drm/ttm: Use the TTM shrinker rather than the external shmem pool
Patchwork
CI[Intel-gfx] ✓ Fi.CI.BAT: success for Add a TTM shrinker
In reply to Thomas Hellström== Series Details ==
== Series Details ==
Series: Add a TTM shrinker
URL : https://patchwork.freedesktop.org/series/114060/
State : success
== Summary ==
CI Bug Log - changes from CI_DRM_12744 -> Patchwork_114060v1
====================================================
Summary
-------
**SUCCESS**
No regressions found.
External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/index.html
Participating hosts (40 -> 39)
------------------------------
Missing (1): fi-snb-2520m
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in Patchwork_114060v1:
### IGT changes ###
#### Suppressed ####
The following results come from untrusted machines, tests, or statuses. They do not affect the overall result.
* igt@i915_suspend@basic-s2idle-without-i915:
- {bat-rpls-1}: [PASS][1] -> [ABORT][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/bat-rpls-1/igt@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/bat-rpls-1/igt@[email protected]
Known issues
------------
Here are the changes found in Patchwork_114060v1 that come from known issues:
### IGT changes ###
#### Possible fixes ####
* igt@i915_selftest@live@hangcheck:
- fi-skl-guc: [DMESG-WARN][3] ([i915#8073]) -> [PASS][4]
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/fi-skl-guc/igt@i915_selftest@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/fi-skl-guc/igt@i915_selftest@[email protected]
* igt@i915_selftest@live@mman:
- {bat-rpls-1}: [TIMEOUT][5] ([i915#6794]) -> [PASS][6]
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/bat-rpls-1/igt@i915_selftest@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/bat-rpls-1/igt@i915_selftest@[email protected]
* igt@i915_selftest@live@reset:
- {bat-rpls-2}: [ABORT][7] ([i915#4983]) -> [PASS][8]
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/bat-rpls-2/igt@i915_selftest@[email protected]
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/bat-rpls-2/igt@i915_selftest@[email protected]
{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).
[i915#1845]: https://gitlab.freedesktop.org/drm/intel/issues/1845
[i915#3546]: https://gitlab.freedesktop.org/drm/intel/issues/3546
[i915#4983]: https://gitlab.freedesktop.org/drm/intel/issues/4983
[i915#6367]: https://gitlab.freedesktop.org/drm/intel/issues/6367
[i915#6794]: https://gitlab.freedesktop.org/drm/intel/issues/6794
[i915#7828]: https://gitlab.freedesktop.org/drm/intel/issues/7828
[i915#7996]: https://gitlab.freedesktop.org/drm/intel/issues/7996
[i915#8073]: https://gitlab.freedesktop.org/drm/intel/issues/8073
Build changes
-------------
* Linux: CI_DRM_12744 -> Patchwork_114060v1
CI-20190529: 20190529
CI_DRM_12744: 2e4cac06e4de01a35626de676b9eeb474ae7ab5d @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_7160: 45da871dd2684227e93a2fc002b87dfc58bd5fd9 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
Patchwork_114060v1: 2e4cac06e4de01a35626de676b9eeb474ae7ab5d @ git://anongit.freedesktop.org/gfx-ci/linux
### Linux commits
973a53557c1e drm/i915, drm/ttm: Use the TTM shrinker rather than the external shmem pool d74baa34a517 drm/ttm: Use fault-injection to test error paths7cb4a7ddbf61 drm/ttm: Provide helpers for shrinkingb95280e8b385 drm/ttm: Make the call to ttm_tt_populate() interruptible when faulting 2df6e6537d76 mm: Add interfaces to back up and recover folio contents using swap a8d1b7591b5a drm/ttm: Add a simple api to set / clear purgeable ttm_tt content 7ded50ee63b5 drm/ttm: Remove pinned bos from shrinkable accounting 9d6623a5d104 drm/ttm: Introduce shrink throttling.0c8485e9f550 drm/ttm: Add a shrinker and shrinker accounting275c2e2830c2 drm/ttm: Reduce the number of used allocation orders for TTM pages ca10fb200301 drm/ttm: Don't use watermark accounting on shrinkable pools 655187c03a8d drm/ttm: Unexport ttm_global_swapout()9f946fa42819 drm/ttm, drm/vmwgfx: Update the TTM swapout interface 20a099fea0fd drm/ttm: Use the BIT macro for the TTM_TT_FLAGs8ef0e28f28fc drm/ttm/pool: Fix ttm_pool_alloc error path03dd353ce759 drm/ttm: Fix a NULL pointer dereference
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/index.html
kernel test robot
build botRe: [RFC PATCH 07/16] drm/ttm: Reduce the number of used allocation orders for TTM pages
In reply to Thomas Hellström[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in…
Hi Thomas,
[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on drm-misc/drm-misc-next]
[also build test ERROR on drm-intel/for-linux-next drm-tip/drm-tip] [cannot apply to drm-intel/for-linux-next-fixes linus/master v6.2-rc8] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Thomas-Hellstr-m/drm-ttm-Fix-a-NULL-pointer-dereference/20230216-001727 base: git://anongit.freedesktop.org/drm/drm-misc drm-misc-next patch link: https://lore.kernel.org/r/20230215161405.187368-8-thomas.hellstrom%40linux.intel.com patch subject: [RFC PATCH 07/16] drm/ttm: Reduce the number of used allocation orders for TTM pages config: mips-allmodconfig (https://download.01.org/0day-ci/archive/20230216/[email protected]/config) compiler: mips-linux-gcc (GCC) 12.1.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/4925cefe2ad6e10e768cebed1a78bfde13199195
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Thomas-Hellstr-m/drm-ttm-Fix-a-NULL-pointer-dereference/20230216-001727 git checkout 4925cefe2ad6e10e768cebed1a78bfde13199195
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=mips olddefconfig COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=mips SHELL=/bin/bash drivers/gpu/drm/ttm/
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Link: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All error/warnings (new ones prefixed by >>):
In file included from include/linux/pgtable.h:6,
from include/linux/mm.h:29,
from include/linux/scatterlist.h:8,
from include/linux/dma-mapping.h:10,
from drivers/gpu/drm/ttm/ttm_pool.c:37:
>> arch/mips/include/asm/pgtable.h:238:26: warning: "__builtin_ffs" is not defined, evaluates to 0 [-Wundef]
238 | #define PTE_T_LOG2 (__builtin_ffs(sizeof(pte_t)) - 1)
| ^~~~~~~~~~~~~
arch/mips/include/asm/pgtable-32.h:67:43: note: in expansion of macro 'PTE_T_LOG2'
67 | # define PGDIR_SHIFT (2 * PAGE_SHIFT - PTE_T_LOG2)
| ^~~~~~~~~~
include/asm-generic/pgtable-nop4d.h:11:33: note: in expansion of macro 'PGDIR_SHIFT'
11 | #define P4D_SHIFT PGDIR_SHIFT
| ^~~~~~~~~~~
include/asm-generic/pgtable-nopud.h:18:25: note: in expansion of macro 'P4D_SHIFT'
18 | #define PUD_SHIFT P4D_SHIFT
| ^~~~~~~~~
include/asm-generic/pgtable-nopmd.h:20:25: note: in expansion of macro 'PUD_SHIFT'
20 | #define PMD_SHIFT PUD_SHIFT
| ^~~~~~~~~
drivers/gpu/drm/ttm/ttm_pool.c:52:24: note: in expansion of macro 'PMD_SHIFT'
52 | #define TTM_MAX_ORDER (PMD_SHIFT - PAGE_SHIFT)
| ^~~~~~~~~
drivers/gpu/drm/ttm/ttm_pool.c:54:6: note: in expansion of macro 'TTM_MAX_ORDER'
54 | #if (TTM_MAX_ORDER < TTM_64K_ORDER)
| ^~~~~~~~~~~~~
>> arch/mips/include/asm/pgtable.h:238:39: error: missing binary operator before token "("
238 | #define PTE_T_LOG2 (__builtin_ffs(sizeof(pte_t)) - 1)
| ^
arch/mips/include/asm/pgtable-32.h:67:43: note: in expansion of macro 'PTE_T_LOG2'
67 | # define PGDIR_SHIFT (2 * PAGE_SHIFT - PTE_T_LOG2)
| ^~~~~~~~~~
include/asm-generic/pgtable-nop4d.h:11:33: note: in expansion of macro 'PGDIR_SHIFT'
11 | #define P4D_SHIFT PGDIR_SHIFT
| ^~~~~~~~~~~
include/asm-generic/pgtable-nopud.h:18:25: note: in expansion of macro 'P4D_SHIFT'
18 | #define PUD_SHIFT P4D_SHIFT
| ^~~~~~~~~
include/asm-generic/pgtable-nopmd.h:20:25: note: in expansion of macro 'PUD_SHIFT'
20 | #define PMD_SHIFT PUD_SHIFT
| ^~~~~~~~~
drivers/gpu/drm/ttm/ttm_pool.c:52:24: note: in expansion of macro 'PMD_SHIFT'
52 | #define TTM_MAX_ORDER (PMD_SHIFT - PAGE_SHIFT)
| ^~~~~~~~~
drivers/gpu/drm/ttm/ttm_pool.c:54:6: note: in expansion of macro 'TTM_MAX_ORDER'
54 | #if (TTM_MAX_ORDER < TTM_64K_ORDER)
| ^~~~~~~~~~~~~
>> arch/mips/include/asm/pgtable.h:238:26: warning: "__builtin_ffs" is not defined, evaluates to 0 [-Wundef]
238 | #define PTE_T_LOG2 (__builtin_ffs(sizeof(pte_t)) - 1)
| ^~~~~~~~~~~~~
arch/mips/include/asm/pgtable-32.h:67:43: note: in expansion of macro 'PTE_T_LOG2'
67 | # define PGDIR_SHIFT (2 * PAGE_SHIFT - PTE_T_LOG2)
| ^~~~~~~~~~
include/asm-generic/pgtable-nop4d.h:11:33: note: in expansion of macro 'PGDIR_SHIFT'
11 | #define P4D_SHIFT PGDIR_SHIFT
| ^~~~~~~~~~~
include/asm-generic/pgtable-nopud.h:18:25: note: in expansion of macro 'P4D_SHIFT'
18 | #define PUD_SHIFT P4D_SHIFT
| ^~~~~~~~~
include/asm-generic/pgtable-nopmd.h:20:25: note: in expansion of macro 'PUD_SHIFT'
20 | #define PMD_SHIFT PUD_SHIFT
| ^~~~~~~~~
drivers/gpu/drm/ttm/ttm_pool.c:52:24: note: in expansion of macro 'PMD_SHIFT'
52 | #define TTM_MAX_ORDER (PMD_SHIFT - PAGE_SHIFT)
| ^~~~~~~~~
drivers/gpu/drm/ttm/ttm_pool.c:58:24: note: in expansion of macro 'TTM_MAX_ORDER'
58 | #if ((MAX_ORDER - 1) < TTM_MAX_ORDER)
| ^~~~~~~~~~~~~
>> arch/mips/include/asm/pgtable.h:238:39: error: missing binary operator before token "("
238 | #define PTE_T_LOG2 (__builtin_ffs(sizeof(pte_t)) - 1)
| ^
arch/mips/include/asm/pgtable-32.h:67:43: note: in expansion of macro 'PTE_T_LOG2'
67 | # define PGDIR_SHIFT (2 * PAGE_SHIFT - PTE_T_LOG2)
| ^~~~~~~~~~
include/asm-generic/pgtable-nop4d.h:11:33: note: in expansion of macro 'PGDIR_SHIFT'
11 | #define P4D_SHIFT PGDIR_SHIFT
| ^~~~~~~~~~~
include/asm-generic/pgtable-nopud.h:18:25: note: in expansion of macro 'P4D_SHIFT'
18 | #define PUD_SHIFT P4D_SHIFT
| ^~~~~~~~~
include/asm-generic/pgtable-nopmd.h:20:25: note: in expansion of macro 'PUD_SHIFT'
20 | #define PMD_SHIFT PUD_SHIFT
| ^~~~~~~~~
drivers/gpu/drm/ttm/ttm_pool.c:52:24: note: in expansion of macro 'PMD_SHIFT'
52 | #define TTM_MAX_ORDER (PMD_SHIFT - PAGE_SHIFT)
| ^~~~~~~~~
drivers/gpu/drm/ttm/ttm_pool.c:58:24: note: in expansion of macro 'TTM_MAX_ORDER'
58 | #if ((MAX_ORDER - 1) < TTM_MAX_ORDER)
| ^~~~~~~~~~~~~vim +238 arch/mips/include/asm/pgtable.h
^1da177e4c3f41 include/asm-mips/pgtable.h Linus Torvalds 2005-04-16 235 5ff974720abec2 include/asm-mips/pgtable.h Ralf Baechle 2007-08-01 236 #define PGD_T_LOG2 (__builtin_ffs(sizeof(pgd_t)) - 1) 5ff974720abec2 include/asm-mips/pgtable.h Ralf Baechle 2007-08-01 237 #define PMD_T_LOG2 (__builtin_ffs(sizeof(pmd_t)) - 1)
5ff974720abec2 include/asm-mips/pgtable.h Ralf Baechle 2007-08-01 @238 #define PTE_T_LOG2 (__builtin_ffs(sizeof(pte_t)) - 1)
^1da177e4c3f41 include/asm-mips/pgtable.h Linus Torvalds 2005-04-16 239
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests
kernel test robot
build botRe: [Intel-gfx] [RFC PATCH 14/16] drm/ttm: Provide helpers for shrinking
In reply to Thomas Hellström[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in…
Hi Thomas,
[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on drm-misc/drm-misc-next]
[also build test ERROR on drm-intel/for-linux-next drm-tip/drm-tip] [cannot apply to drm-intel/for-linux-next-fixes linus/master v6.2-rc8 next-20230215] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Thomas-Hellstr-m/drm-ttm-Fix-a-NULL-pointer-dereference/20230216-001727 base: git://anongit.freedesktop.org/drm/drm-misc drm-misc-next patch link: https://lore.kernel.org/r/20230215161405.187368-15-thomas.hellstrom%40linux.intel.com patch subject: [Intel-gfx] [RFC PATCH 14/16] drm/ttm: Provide helpers for shrinking config: parisc64-defconfig (https://download.01.org/0day-ci/archive/20230216/[email protected]/config) compiler: hppa64-linux-gcc (GCC) 12.1.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/888719f1ca5b28696b77e89e4d365aba16053df3
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Thomas-Hellstr-m/drm-ttm-Fix-a-NULL-pointer-dereference/20230216-001727 git checkout 888719f1ca5b28696b77e89e4d365aba16053df3
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=parisc64 olddefconfig COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=parisc64 SHELL=/bin/bash
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Link: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
In file included from include/linux/init.h:5,
from arch/parisc/include/asm/alternative.h:18, from arch/parisc/include/asm/barrier.h:5, from arch/parisc/include/asm/atomic.h:11, from include/linux/atomic.h:7,
from include/linux/mm_types_task.h:13, from include/linux/mm_types.h:5, from mm/swap_backup_folio.c:3:
mm/swap_backup_folio.c: In function 'swap_backup_folio':
>> include/linux/build_bug.h:30:33: error: void value not ignored as it ought to be
30 | #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
| ^
include/linux/mmdebug.h:92:45: note: in expansion of macro 'BUILD_BUG_ON_INVALID'
92 | #define VM_WARN_ON_ONCE_FOLIO(cond, folio) BUILD_BUG_ON_INVALID(cond)
| ^~~~~~~~~~~~~~~~~~~~
mm/swap_backup_folio.c:36:13: note: in expansion of macro 'VM_WARN_ON_ONCE_FOLIO'
36 | if (VM_WARN_ON_ONCE_FOLIO(folio_nr_pages(folio) != 1 &&
| ^~~~~~~~~~~~~~~~~~~~~
>> include/linux/build_bug.h:30:33: error: void value not ignored as it ought to be
30 | #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
| ^
include/linux/mmdebug.h:92:45: note: in expansion of macro 'BUILD_BUG_ON_INVALID'
92 | #define VM_WARN_ON_ONCE_FOLIO(cond, folio) BUILD_BUG_ON_INVALID(cond)
| ^~~~~~~~~~~~~~~~~~~~
mm/swap_backup_folio.c:42:13: note: in expansion of macro 'VM_WARN_ON_ONCE_FOLIO'
42 | if (VM_WARN_ON_ONCE_FOLIO(folio_ref_count(folio) != 1 ||
| ^~~~~~~~~~~~~~~~~~~~~vim +30 include/linux/build_bug.h
527edbc18a70e7 Masahiro Yamada 2019-01-03 18 527edbc18a70e7 Masahiro Yamada 2019-01-03 19 /* Force a compilation error if a constant expression is not a power of 2 */ 527edbc18a70e7 Masahiro Yamada 2019-01-03 20 #define __BUILD_BUG_ON_NOT_POWER_OF_2(n) \ 527edbc18a70e7 Masahiro Yamada 2019-01-03 21 BUILD_BUG_ON(((n) & ((n) - 1)) != 0) 527edbc18a70e7 Masahiro Yamada 2019-01-03 22 #define BUILD_BUG_ON_NOT_POWER_OF_2(n) \ 527edbc18a70e7 Masahiro Yamada 2019-01-03 23 BUILD_BUG_ON((n) == 0 || (((n) & ((n) - 1)) != 0)) bc6245e5efd70c Ian Abbott 2017-07-10 24 bc6245e5efd70c Ian Abbott 2017-07-10 25 /* bc6245e5efd70c Ian Abbott 2017-07-10 26 * BUILD_BUG_ON_INVALID() permits the compiler to check the validity of the bc6245e5efd70c Ian Abbott 2017-07-10 27 * expression but avoids the generation of any code, even if that expression bc6245e5efd70c Ian Abbott 2017-07-10 28 * has side-effects. bc6245e5efd70c Ian Abbott 2017-07-10 29 */
bc6245e5efd70c Ian Abbott 2017-07-10 @30 #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))bc6245e5efd70c Ian Abbott 2017-07-10 31
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests
kernel test robot
build botRe: [Intel-gfx] [RFC PATCH 14/16] drm/ttm: Provide helpers for shrinking
In reply to Thomas Hellström[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in…
Hi Thomas,
[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on drm-misc/drm-misc-next]
[also build test ERROR on drm-intel/for-linux-next drm-tip/drm-tip] [cannot apply to drm-intel/for-linux-next-fixes linus/master v6.2-rc8 next-20230215] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Thomas-Hellstr-m/drm-ttm-Fix-a-NULL-pointer-dereference/20230216-001727 base: git://anongit.freedesktop.org/drm/drm-misc drm-misc-next patch link: https://lore.kernel.org/r/20230215161405.187368-15-thomas.hellstrom%40linux.intel.com patch subject: [Intel-gfx] [RFC PATCH 14/16] drm/ttm: Provide helpers for shrinking config: x86_64-randconfig-a001-20230213 (https://download.01.org/0day-ci/archive/20230216/[email protected]/config) compiler: clang version 14.0.6 (https://github.com/llvm/llvm-project f28c006a5895fc0e329fe15fead81e37457cb1d1)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/888719f1ca5b28696b77e89e4d365aba16053df3
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Thomas-Hellstr-m/drm-ttm-Fix-a-NULL-pointer-dereference/20230216-001727 git checkout 888719f1ca5b28696b77e89e4d365aba16053df3
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=x86_64 olddefconfig COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=x86_64 SHELL=/bin/bash
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Link: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
>> mm/swap_backup_folio.c:36:2: error: statement requires expression of scalar type ('void' invalid)
if (VM_WARN_ON_ONCE_FOLIO(folio_nr_pages(folio) != 1 &&
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mm/swap_backup_folio.c:42:2: error: statement requires expression of scalar type ('void' invalid)
if (VM_WARN_ON_ONCE_FOLIO(folio_ref_count(folio) != 1 ||
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~2 errors generated.
vim +36 mm/swap_backup_folio.c
337546c404a4a7 Thomas Hellström 2023-02-15 10
337546c404a4a7 Thomas Hellström 2023-02-15 11 /**
337546c404a4a7 Thomas Hellström 2023-02-15 12 * swap_backup_folio() - Insert an isolated folio into the swap-cache.
337546c404a4a7 Thomas Hellström 2023-02-15 13 * @folio: The folio to insert.
337546c404a4a7 Thomas Hellström 2023-02-15 14 * @writeback: Whether to perform immediate writeback.
337546c404a4a7 Thomas Hellström 2023-02-15 15 * @folio_gfp: The gfp value used when the folio was allocated. Used for
337546c404a4a7 Thomas Hellström 2023-02-15 16 * cgroup charging only.
337546c404a4a7 Thomas Hellström 2023-02-15 17 * @alloc_fgp: The gfp value used for swap cache radix tree memory allocations.
337546c404a4a7 Thomas Hellström 2023-02-15 18 *
337546c404a4a7 Thomas Hellström 2023-02-15 19 * Insert a folio into the swap cache and get a swp_entry_t back as a reference.
337546c404a4a7 Thomas Hellström 2023-02-15 20 * If the swap cache folio should be subject of immediate writeback to
337546c404a4a7 Thomas Hellström 2023-02-15 21 * a swap device, @writeback should be set to true.
337546c404a4a7 Thomas Hellström 2023-02-15 22 * After a call to swap_backup_folio() the caller can
337546c404a4a7 Thomas Hellström 2023-02-15 23 * drop its folio reference and use swap_copy_folio() to get the folio
337546c404a4a7 Thomas Hellström 2023-02-15 24 * content back, or swap_drop_folio() to drop it completely.
337546c404a4a7 Thomas Hellström 2023-02-15 25 * Currently only PAGE_SIZE folios work, or if CONFIG_THP_SWAP is
337546c404a4a7 Thomas Hellström 2023-02-15 26 * enabled, HPAGE_PMD_NR*PAGE_SIZE may work as well, although that
337546c404a4a7 Thomas Hellström 2023-02-15 27 * needs further testing.
337546c404a4a7 Thomas Hellström 2023-02-15 28 *
337546c404a4a7 Thomas Hellström 2023-02-15 29 * Return: A swp_entry_t. If its .val field is zero, an error occurred.
337546c404a4a7 Thomas Hellström 2023-02-15 30 */
337546c404a4a7 Thomas Hellström 2023-02-15 31 swp_entry_t swap_backup_folio(struct folio *folio, bool writeback,
337546c404a4a7 Thomas Hellström 2023-02-15 32 gfp_t folio_gfp, gfp_t alloc_gfp)
337546c404a4a7 Thomas Hellström 2023-02-15 33 {
337546c404a4a7 Thomas Hellström 2023-02-15 34 swp_entry_t swap = {};
337546c404a4a7 Thomas Hellström 2023-02-15 35
337546c404a4a7 Thomas Hellström 2023-02-15 @36 if (VM_WARN_ON_ONCE_FOLIO(folio_nr_pages(folio) != 1 &&
337546c404a4a7 Thomas Hellström 2023-02-15 37 !(IS_ENABLED(CONFIG_THP_SWAP) &&
337546c404a4a7 Thomas Hellström 2023-02-15 38 folio_nr_pages(folio) == HPAGE_PMD_NR),
337546c404a4a7 Thomas Hellström 2023-02-15 39 folio))
337546c404a4a7 Thomas Hellström 2023-02-15 40 return swap;
337546c404a4a7 Thomas Hellström 2023-02-15 41
337546c404a4a7 Thomas Hellström 2023-02-15 42 if (VM_WARN_ON_ONCE_FOLIO(folio_ref_count(folio) != 1 ||
337546c404a4a7 Thomas Hellström 2023-02-15 43 folio_test_lru(folio) ||
337546c404a4a7 Thomas Hellström 2023-02-15 44 folio_test_locked(folio), folio))
337546c404a4a7 Thomas Hellström 2023-02-15 45 return swap;
337546c404a4a7 Thomas Hellström 2023-02-15 46
337546c404a4a7 Thomas Hellström 2023-02-15 47 /*
337546c404a4a7 Thomas Hellström 2023-02-15 48 * Typically called from reclaim so use folio_trylock. If the folio
337546c404a4a7 Thomas Hellström 2023-02-15 49 * is isolated with refcount == 1, then this trylock should always
337546c404a4a7 Thomas Hellström 2023-02-15 50 * succeed.
337546c404a4a7 Thomas Hellström 2023-02-15 51 */
337546c404a4a7 Thomas Hellström 2023-02-15 52 if (!folio_trylock(folio))
337546c404a4a7 Thomas Hellström 2023-02-15 53 return swap;
337546c404a4a7 Thomas Hellström 2023-02-15 54
337546c404a4a7 Thomas Hellström 2023-02-15 55 __folio_mark_uptodate(folio);
337546c404a4a7 Thomas Hellström 2023-02-15 56 __folio_set_swapbacked(folio);
337546c404a4a7 Thomas Hellström 2023-02-15 57
337546c404a4a7 Thomas Hellström 2023-02-15 58 mem_cgroup_charge(folio, NULL, folio_gfp);
337546c404a4a7 Thomas Hellström 2023-02-15 59
337546c404a4a7 Thomas Hellström 2023-02-15 60 swap = folio_alloc_swap(folio);
337546c404a4a7 Thomas Hellström 2023-02-15 61 if (!swap.val)
337546c404a4a7 Thomas Hellström 2023-02-15 62 goto out;
337546c404a4a7 Thomas Hellström 2023-02-15 63
337546c404a4a7 Thomas Hellström 2023-02-15 64 if (add_to_swap_cache(folio, swap, alloc_gfp, NULL) == 0) {
337546c404a4a7 Thomas Hellström 2023-02-15 65 int ret = -EINVAL;
337546c404a4a7 Thomas Hellström 2023-02-15 66
337546c404a4a7 Thomas Hellström 2023-02-15 67 swap_shmem_alloc(swap);
337546c404a4a7 Thomas Hellström 2023-02-15 68 folio_add_lru(folio);
337546c404a4a7 Thomas Hellström 2023-02-15 69 lru_add_drain();
337546c404a4a7 Thomas Hellström 2023-02-15 70
337546c404a4a7 Thomas Hellström 2023-02-15 71 /* Stolen from pageout(). */
337546c404a4a7 Thomas Hellström 2023-02-15 72 if (writeback && folio_clear_dirty_for_io(folio)) {
337546c404a4a7 Thomas Hellström 2023-02-15 73 struct writeback_control wbc = {
337546c404a4a7 Thomas Hellström 2023-02-15 74 .sync_mode = WB_SYNC_NONE,
337546c404a4a7 Thomas Hellström 2023-02-15 75 .nr_to_write = SWAP_CLUSTER_MAX,
337546c404a4a7 Thomas Hellström 2023-02-15 76 .range_start = 0,
337546c404a4a7 Thomas Hellström 2023-02-15 77 .range_end = LLONG_MAX,
337546c404a4a7 Thomas Hellström 2023-02-15 78 .for_reclaim = 1,
337546c404a4a7 Thomas Hellström 2023-02-15 79 };
337546c404a4a7 Thomas Hellström 2023-02-15 80
337546c404a4a7 Thomas Hellström 2023-02-15 81 folio_set_reclaim(folio);
337546c404a4a7 Thomas Hellström 2023-02-15 82 ret = swap_writepage(folio_page(folio, 0), &wbc);
337546c404a4a7 Thomas Hellström 2023-02-15 83 if (!folio_test_writeback(folio))
337546c404a4a7 Thomas Hellström 2023-02-15 84 folio_clear_reclaim(folio);
337546c404a4a7 Thomas Hellström 2023-02-15 85 }
337546c404a4a7 Thomas Hellström 2023-02-15 86
337546c404a4a7 Thomas Hellström 2023-02-15 87 if (ret)
337546c404a4a7 Thomas Hellström 2023-02-15 88 folio_unlock(folio);
337546c404a4a7 Thomas Hellström 2023-02-15 89 return swap;
337546c404a4a7 Thomas Hellström 2023-02-15 90 }
337546c404a4a7 Thomas Hellström 2023-02-15 91
337546c404a4a7 Thomas Hellström 2023-02-15 92 put_swap_folio(folio, swap);
337546c404a4a7 Thomas Hellström 2023-02-15 93 out:
337546c404a4a7 Thomas Hellström 2023-02-15 94 folio_clear_swapbacked(folio);
337546c404a4a7 Thomas Hellström 2023-02-15 95 folio_mark_dirty(folio);
337546c404a4a7 Thomas Hellström 2023-02-15 96 folio_unlock(folio);
337546c404a4a7 Thomas Hellström 2023-02-15 97 mem_cgroup_uncharge(folio);
337546c404a4a7 Thomas Hellström 2023-02-15 98
337546c404a4a7 Thomas Hellström 2023-02-15 99 return swap;
337546c404a4a7 Thomas Hellström 2023-02-15 100 }
337546c404a4a7 Thomas Hellström 2023-02-15 101 EXPORT_SYMBOL(swap_backup_folio);
337546c404a4a7 Thomas Hellström 2023-02-15 102 Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests
Re: [RFC PATCH 07/16] drm/ttm: Reduce the number of used allocation orders for TTM pages
In reply to Thomas HellströmAm 15.02.23 um 20:00 schrieb Thomas Hellström:
Quoted 45 lines · Christian König
Quoted 39 lines
Am 15.02.23 um 19:12 schrieb Thomas Hellström:
Quoted 15 lines · Christian König
Quoted 14 lines
Am 15.02.23 um 17:13 schrieb Thomas Hellström:
Quoted 13 lines
When swapping out, we will split multi-order pages both in order to
move them to the swap-cache and to be able to return memory to the
swap cache as soon as possible on a page-by-page basis. By reducing the page max order to the system PMD size, we can be
nicer
to the system and avoid splitting gigantic pages.
On top of this we also
include the 64K page size in the page sizes tried, since that appears to
be a common size for GPU applications.Quoted 2 lines
Quoted 1 line
Please completely drop that.
You mean the 64K page size, or the whole patch?
The 64K page size. This was an invention from Microsoft to standardize
GPU handling ~15-20years ago.It turned out to be a complete shipwreck and by now 2MiB and 1GiB pages
or just flexible hardware which can handle everything seem to become standard.Quoted 4 lines
Quoted 2 lines
This is just nonsense spilling in from the
Windows drivers.Agreed, but IIRC on the last RFC you asked me not to drop the 64K pages, so that's why they are here. I can remove them if needed.
We could keep it if it's in any way beneficial, but I'm pretty sure I must have been drunk to ask for that.
Quoted 5 lines
The only reason for keeping them from a performance point of view is
better efficiency on GPUs with 64K page size if not using a coalescing
IOMMU for dma-mapping.Quoted 5 lines
Are any of those still produced? As far as I know neither NVidia, Intel
nor AMD still assumes that page size in their hardware for quite a while
now.
Quoted 8 lines
Intel still supports 64K PTEs, so we use them where possible, otherwise falling back to 4K. Typically we have coalescing IOMMU enabled when testing, so can't really see the impact, but TBH I was surprised by the number of 64K page allocations TTM spat out with this patch series, so I definitely think there is a performance impact with !IOMMU, although I can't quantify it ATM.
So then if it's OK with you I'll keep that size for now.
If it makes 64K pages preferred then this is a pretty clear NAK.
What we can do is to support any page size up to at least 2MiB here.
Christian.
Quoted 204 lines
/Thomas
Quoted 199 lines
Regards,
Christian.Quoted 196 lines
Let me know what you think is best and I'll adjust accordingly.
/Thomas
Quoted 191 lines
Christian.
Quoted 189 lines
Looking forward to when we might be able to swap out PMD size folios
without splitting, this will also be a benefit.Signed-off-by: Thomas Hellström
<[email protected]>--- drivers/gpu/drm/ttm/ttm_pool.c | 58 ++++++++++++++++++++++++++--- ----- 1 file changed, 45 insertions(+), 13 deletions(-)drivers/gpu/drm/ttm/ttm_pool.c +14
-0@@ -31,6 +31,8 @@* cause they are rather slow compared to alloc_pages+map. */ +#define pr_fmt (fmt) "[TTM POOL] " fmt + #include <linux/module.h> #include <linux/dma-mapping.h> #include <linux/debugfs.h>@@ -47,6 +49,18 @@ #include "ttm_module.h" +#define TTM_MAX_ORDER (PMD_SHIFT - PAGE_SHIFT) +#define TTM_64K_ORDER (16 - PAGE_SHIFT) +#if (TTM_MAX_ORDER <TTM_64K_ORDER ) +#undef TTM_MAX_ORDER +#define TTM_MAX_ORDER TTM_64K_ORDER +#endif +#if ((MAX_ORDER -1 ) <TTM_MAX_ORDER ) +#undef TTM_MAX_ORDER +#define TTM_MAX_ORDER (MAX_ORDER - 1) +#endif +#define TTM_DIM_ORDER (TTM_MAX_ORDER + 1) + /*** struct ttm_pool_dma - Helper object for coherent DMA mappings
*drivers/gpu/drm/ttm/ttm_pool.c +6
-4@@ -65,16 +79,18 @@ module_param(page_pool_size, ulong, 0644); static atomic_long_t allocated_pages; -static struct ttm_pool_type global_write_combined[MAX_ORDER ]; -static struct ttm_pool_type global_uncached[MAX_ORDER ]; +static struct ttm_pool_type global_write_combined[TTM_DIM_ORDER ]; +static struct ttm_pool_type global_uncached[TTM_DIM_ORDER ]; -static struct ttm_pool_type global_dma32_write_combined[MAX_ORDER ]; -static struct ttm_pool_type global_dma32_uncached[MAX_ORDER ]; +static struct ttm_pool_type global_dma32_write_combined[TTM_DIM_ORDER ]; +static struct ttm_pool_type global_dma32_uncached[TTM_DIM_ORDER ]; static spinlock_t shrinker_lock; static struct list_head shrinker_list; static struct shrinker mm_shrinker; +static unsigned int ttm_pool_orders[] = {TTM_MAX_ORDER ,0 ,0 }; + /* Allocate pages of size 1 << order with the given
gfp_flags*/
static struct page *ttm_pool_alloc_page(structttm_pool
*pool,gfp_tgfp_flags,
unsigned int order)drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -400,6 +416,17 @@ static void __ttm_pool_free(struct
ttm_pool
*pool, structttm_tt*tt,
}
}+static unsigned int
ttm_pool_select_order(unsigned int order, pgoff_t num_pages)drivers/gpu/drm/ttm/ttm_pool.c +10
-3+{ + unsigned int *cur_order = ttm_pool_orders; + + order = min_t(unsigned int, __fls(num_pages), order); + while (order < *cur_order) + ++cur_order; + + return *cur_order; +} + /*** ttm_pool_alloc - Fill a ttm_tt object * @@ -439,9 +466,8 @@ int ttm_pool_alloc(struct ttm_pool *pool,struct ttm_tt *tt, else gfp_flags |= GFP_HIGHUSER; - for (order = min_t(unsigned int, MAX_ORDER - 1, __fls(num_pages)); - num_pages; - order = min_t(unsigned int, order, __fls(num_pages))){
+ order =ttm_pool_select_order(ttm_pool_orders[0], num_pages);
+ for (;num_pages; order =ttm_pool_select_order(order, num_pages)) {
structttm_pool_type*pt;page_caching =
tt->caching;drivers/gpu/drm/ttm/ttm_pool.c +1
-1@@ -558,7 +584,7 @@ void ttm_pool_init(struct ttm_pool *pool,struct device *dev, if (use_dma_alloc) { for (i = 0; i < TTM_NUM_CACHING_TYPES; ++i) - for (j = 0; j < MAX_ORDER; ++j) + for (j = 0; j < TTM_DIM_ORDER; ++j) ttm_pool_type_init(&pool- > caching[i].orders[j],pool, i,
j);
}drivers/gpu/drm/ttm/ttm_pool.c +2
-2@@ -578,7 +604,7 @@ void ttm_pool_fini(struct ttm_pool *pool) if (pool->use_dma_alloc) { for (i = 0; i < TTM_NUM_CACHING_TYPES; ++i) - for (j = 0; j < MAX_ORDER; ++j) + for (j = 0; j < TTM_DIM_ORDER; ++j) ttm_pool_type_fini(&pool- > caching[i].orders[j]); }@@ -632,7 +658,7 @@ static void ttm_pool_debugfs_header(struct seq_file *m) unsigned int i; seq_puts(m, "\t "); - for (i = 0; i < MAX_ORDER; ++i) + for (i = 0; i < TTM_DIM_ORDER; ++i) seq_printf(m, " ---%2u---", i); seq_puts(m, "\n");}
drivers/gpu/drm/ttm/ttm_pool.c +1
-1@@ -643,7 +669,7 @@ static void ttm_pool_debugfs_orders(structttm_pool_type *pt, { unsigned int i; - for (i = 0; i < MAX_ORDER; ++i) + for (i = 0; i < TTM_DIM_ORDER; ++i) seq_printf(m, " %8u",ttm_pool_type_count (&pt[i]));seq_puts(m, "\n");
}drivers/gpu/drm/ttm/ttm_pool.c +7
-1@@ -749,10 +775,16 @@ int ttm_pool_mgr_init(unsigned long num_pages) if (!page_pool_size) page_pool_size = num_pages; + if (TTM_64K_ORDER < TTM_MAX_ORDER) + ttm_pool_orders[1] = TTM_64K_ORDER; + + pr_debug("Used orders are %u %u %u\n", ttm_pool_orders[0 ], + ttm_pool_orders[1], ttm_pool_orders[2]); + spin_lock_init(&shrinker_lock); INIT_LIST_HEAD(&shrinker_list); - for (i = 0; i < MAX_ORDER; ++i) { + for (i = 0; i < TTM_DIM_ORDER; ++i) { ttm_pool_type_init(&global_write_combined[i],NULL,
ttm_write_combined, i); ttm_pool_type_init(&global_uncached[i], NULL,ttm_uncached, i);drivers/gpu/drm/ttm/ttm_pool.c +1
-1@@ -785,7 +817,7 @@ void ttm_pool_mgr_fini(void) { unsigned int i; - for (i = 0; i < MAX_ORDER; ++i) { + for (i = 0; i < TTM_DIM_ORDER; ++i) { ttm_pool_type_fini(&global_write_combined[i]); ttm_pool_type_fini(&global_uncached[i]);
Thomas Hellström
reviewerRe: [RFC PATCH 07/16] drm/ttm: Reduce the number of used allocation orders for TTM pages
Quoted 53 lines · Christian König
Am 15.02.23 um 20:00 schrieb Thomas Hellström:
Quoted 52 lines · Christian König
Quoted 39 lines
Am 15.02.23 um 19:12 schrieb Thomas Hellström:
Quoted 15 lines · Christian König
Quoted 14 lines
Am 15.02.23 um 17:13 schrieb Thomas Hellström:
Quoted 13 lines
When swapping out, we will split multi-order pages both in order to
move them to the swap-cache and to be able to return memory to the
swap cache as soon as possible on a page-by-page basis. By reducing the page max order to the system PMD size, we can be
nicer
to the system and avoid splitting gigantic pages.
On top of this we also
include the 64K page size in the page sizes tried, since that appears to
be a common size for GPU applications.Quoted 2 lines
Quoted 1 line
Please completely drop that.
You mean the 64K page size, or the whole patch?
The 64K page size. This was an invention from Microsoft to standardize
GPU handling ~15-20years ago.It turned out to be a complete shipwreck and by now 2MiB and 1GiB pages
or just flexible hardware which can handle everything seem to become standard.Quoted 4 lines
Quoted 2 lines
This is just nonsense spilling in from the
Windows drivers.Agreed, but IIRC on the last RFC you asked me not to drop the 64K pages, so that's why they are here. I can remove them if needed.
We could keep it if it's in any way beneficial, but I'm pretty sure I must have been drunk to ask for that.
Quoted 5 lines
The only reason for keeping them from a performance point of view is
better efficiency on GPUs with 64K page size if not using a coalescing
IOMMU for dma-mapping.Quoted 5 lines
Are any of those still produced? As far as I know neither NVidia, Intel
nor AMD still assumes that page size in their hardware for quite a while
now.Intel still supports 64K PTEs, so we use them where possible, otherwise falling back to 4K. Typically we have coalescing IOMMU enabled when testing, so can't really see the impact, but TBH I was surprised by the number of 64K page allocations TTM spat out with this patch series, so I definitely think there is a performance impact with !IOMMU, although I can't quantify it ATM.
Quoted 5 lines
Quoted 1 line
So then if it's OK with you I'll keep that size for now.
If it makes 64K pages preferred then this is a pretty clear NAK.
What we can do is to support any page size up to at least 2MiB here.
OK, I'll use that latter approach then. I don't have any strong preferences here except the swapin helper wants to keep the max pagesize as low as possible since it needs to store one page worth of 4K swap entries.
/Thomas
Quoted 188 lines
Christian.
Quoted 184 lines
/Thomas
Quoted 179 lines
Regards,
Christian.Quoted 176 lines
Let me know what you think is best and I'll adjust accordingly.
/Thomas
Quoted 171 lines
Christian.
Quoted 169 lines
Looking forward to when we might be able to swap out PMD size folios
without splitting, this will also be a benefit.Signed-off-by: Thomas Hellström
<[email protected]>
---
drivers/gpu/drm/ttm/ttm_pool.c| 58
++++++++++++++++++++++++++---
-----1 file changed, 45 insertions(+), 13 deletions(-)drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -31,6 +31,8 @@* cause they are rather slow compared to alloc_pages+map. */ +#define
pr_fmt(fmt)"[TTM POOL] " fmt
+
#include <linux/module.h>
#include <linux/dma-mapping.h>
#include <linux/debugfs.h>drivers/gpu/drm/ttm/ttm_pool.c +11
-0@@ -47,6 +49,18 @@ #include "ttm_module.h" +#define TTM_MAX_ORDER (PMD_SHIFT - PAGE_SHIFT) +#define TTM_64K_ORDER (16 - PAGE_SHIFT) +#if (TTM_MAX_ORDER <TTM_64K_ORDER ) +#undef TTM_MAX_ORDER +#define TTM_MAX_ORDER TTM_64K_ORDER +#endif +#if ((MAX_ORDER -1 ) <TTM_MAX_ORDER ) +#undef TTM_MAX_ORDER +#define TTM_MAX_ORDER (MAX_ORDER - 1) +#endif +#define TTM_DIM_ORDER (TTM_MAX_ORDER + 1) + /*** struct ttm_pool_dma - Helper object for coherent DMA mappings
*drivers/gpu/drm/ttm/ttm_pool.c +2
-1@@ -65,16 +79,18 @@ module_param(page_pool_size, ulong, 0644); static atomic_long_t allocated_pages; -static struct ttm_pool_type global_write_combined[MAX_ORDER]; -static struct ttm_pool_type global_uncached[MAX_ORDER ]; +static struct ttm_pool_type global_write_combined[TTM_DIM_ORDER ]; +static struct ttm_pool_type global_uncached[TTM_DIM_ORDER ];-static struct
ttm_pool_type
global_dma32_write_combined[MAX_ORDER];drivers/gpu/drm/ttm/ttm_pool.c +1
-1-static struct ttm_pool_type global_dma32_uncached[MAX_ORDER ]; +static struct ttm_pool_typeglobal_dma32_write_combined[TTM_DIM_ORDER];
+static structttm_pool_typeglobal_dma32_uncached[TTM_DIM_ORDER];
staticspinlock_tshrinker_lock;
static structlist_headshrinker_list;
static struct shrinkermm_shrinker;
+static unsigned int ttm_pool_orders[] = {TTM_MAX_ORDER, 0, 0}; +
/* Allocate pages of size 1 << order with the givengfp_flags*/
static struct page *ttm_pool_alloc_page(structttm_pool*pool,gfp_tgfp_flags,
unsigned int order)drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -400,6 +416,17 @@ static void __ttm_pool_free(struct
ttm_pool
*pool, structttm_tt*tt,
}
}
+static unsigned intttm_pool_select_order(unsigned int order, pgoff_t num_pages)drivers/gpu/drm/ttm/ttm_pool.c +10
-0+{ + unsigned int *cur_order = ttm_pool_orders; + + order = min_t(unsigned int, __fls(num_pages), order); + while (order < *cur_order) + ++cur_order; + + return *cur_order; +} +/**
*ttm_pool_alloc- Fill attm_ttobject
*drivers/gpu/drm/ttm/ttm_pool.c +0
-0@@ -439,9 +466,8 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,else
gfp_flags |=GFP_HIGHUSER;
- for (order =min_t(unsigned int, MAX_ORDER - 1, __fls(num_pages));
-num_pages;
- order =min_t(unsigned int, order, __fls(num_pages)))
{
+ order =ttm_pool_select_order(ttm_pool_orders[0], num_pages);
+ for (;num_pages; order =ttm_pool_select_order(order, num_pages)) {
structttm_pool_type*pt;page_caching=tt->caching;drivers/gpu/drm/ttm/ttm_pool.c +1
-1@@ -558,7 +584,7 @@ void ttm_pool_init(struct ttm_pool *pool, struct device *dev, if (use_dma_alloc) { for (i = 0; i < TTM_NUM_CACHING_TYPES; ++i) - for (j = 0; j < MAX_ORDER; ++j) + for (j = 0; j < TTM_DIM_ORDER; ++j)ttm_pool_type_init(&pool-
Quoted 1 line
caching[i].orders[j],
pool, i,
j);
}drivers/gpu/drm/ttm/ttm_pool.c +1
-1@@ -578,7 +604,7 @@ void ttm_pool_fini(struct ttm_pool *pool) if (pool->use_dma_alloc) { for (i = 0; i < TTM_NUM_CACHING_TYPES; ++i) - for (j = 0; j < MAX_ORDER; ++j) + for (j = 0; j < TTM_DIM_ORDER; ++j)ttm_pool_type_fini(&pool-
Quoted 1 line
caching[i].orders[j]);
}
@@ -632,7 +658,7 @@ static voidttm_pool_debugfs_header(struct seq_file *m)
unsigned int i;
seq_puts(m, "\t ");
- for (i = 0; i <MAX_ORDER; ++i)
+ for (i = 0; i <TTM_DIM_ORDER; ++i)seq_printf(m, " ---%2u---", i);seq_puts(m, "\n");
}drivers/gpu/drm/ttm/ttm_pool.c +1
-0@@ -643,7 +669,7 @@ static void ttm_pool_debugfs_orders(structttm_pool_type *pt, { unsigned int i; - for (i = 0; i < MAX_ORDER; ++i) + for (i = 0; i < TTM_DIM_ORDER; ++i) seq_printf(m, " %8u",ttm_pool_type_count (&pt[i]));seq_puts(m, "\n");
}drivers/gpu/drm/ttm/ttm_pool.c +6
-0@@ -749,10 +775,16 @@ int ttm_pool_mgr_init(unsigned long num_pages) if (!page_pool_size) page_pool_size = num_pages; + if (TTM_64K_ORDER < TTM_MAX_ORDER) + ttm_pool_orders[1] = TTM_64K_ORDER; + + pr_debug("Used orders are %u %u %u\n", ttm_pool_orders[0 ], + ttm_pool_orders[1], ttm_pool_orders[2]); + spin_lock_init(&shrinker_lock); INIT_LIST_HEAD(&shrinker_list); - for (i = 0; i < MAX_ORDER; ++i) { + for (i = 0; i < TTM_DIM_ORDER; ++i) { ttm_pool_type_init(&global_write_combined[i], NULL,
ttm_write_combined, i);ttm_pool_type_init(&global_uncached[i], NULL, ttm_uncached, i);drivers/gpu/drm/ttm/ttm_pool.c +1
-0@@ -785,7 +817,7 @@ void ttm_pool_mgr_fini(void) { unsigned int i; - for (i = 0; i < MAX_ORDER; ++i) { + for (i = 0; i < TTM_DIM_ORDER; ++i) { ttm_pool_type_fini(&global_write_combined[i]); ttm_pool_type_fini(&global_uncached[i]);
Patchwork
CI[Intel-gfx] ✗ Fi.CI.IGT: failure for Add a TTM shrinker
In reply to Thomas Hellström== Series Details ==
== Series Details ==
Series: Add a TTM shrinker
URL : https://patchwork.freedesktop.org/series/114060/
State : failure
== Summary ==
CI Bug Log - changes from CI_DRM_12744_full -> Patchwork_114060v1_full
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with Patchwork_114060v1_full absolutely need to be verified manually.
If you think the reported changes have nothing to do with the changes introduced in Patchwork_114060v1_full, please notify your bug team to allow them to document this new failure mode, which will reduce false positives in CI.
External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/index.html
Participating hosts (10 -> 11)
------------------------------
Additional (1): shard-rkl0
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in Patchwork_114060v1_full:
### IGT changes ###
#### Possible regressions ####
* igt@gem_eio@in-flight-suspend:
- shard-apl: [PASS][1] -> [INCOMPLETE][2] +5 similar issues
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-apl1/igt@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-apl7/igt@[email protected]
* igt@kms_vblank@pipe-a-ts-continuation-suspend:
- shard-snb: [PASS][3] -> [INCOMPLETE][4] +1 similar issue
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-snb6/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-snb6/igt@[email protected]
#### Warnings ####
* igt@kms_content_protection@lic@pipe-a-dp-1:
- shard-apl: [TIMEOUT][5] ([i915#7173]) -> [INCOMPLETE][6]
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-apl1/igt@kms_content_protection@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-apl1/igt@kms_content_protection@[email protected]
New tests
---------
New tests have been introduced between CI_DRM_12744_full and Patchwork_114060v1_full:
### New IGT tests (2) ###
* igt@gem_tiled_swapping:
- Statuses :
- Exec time: [None] s
* igt@perf_pmu:
- Statuses :
- Exec time: [None] s
Known issues
------------
Here are the changes found in Patchwork_114060v1_full that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@gem_eio@suspend:
- shard-snb: [PASS][7] -> [INCOMPLETE][8] ([i915#7850])
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-snb2/igt@[email protected]
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-snb1/igt@[email protected]
* igt@gem_userptr_blits@vma-merge:
- shard-apl: NOTRUN -> [FAIL][9] ([i915#3318])
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-apl4/igt@[email protected]
* igt@i915_pm_lpsp@kms-lpsp@kms-lpsp-vga:
- shard-snb: NOTRUN -> [SKIP][10] ([fdo#109271]) +22 similar issues
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-snb6/igt@i915_pm_lpsp@[email protected]
* igt@i915_selftest@live@gt_heartbeat:
- shard-apl: [PASS][11] -> [DMESG-FAIL][12] ([i915#5334])
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-apl4/igt@i915_selftest@live@gt_heartbeat.html
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-apl2/igt@i915_selftest@live@gt_heartbeat.html
* igt@kms_flip@flip-vs-expired-vblank-interruptible@a-hdmi-a1:
- shard-glk: [PASS][13] -> [FAIL][14] ([i915#79])
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-glk1/igt@kms_flip@[email protected]
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-glk1/igt@kms_flip@[email protected]
* igt@kms_frontbuffer_tracking@fbcpsr-2p-scndscrn-pri-indfb-draw-mmap-gtt:
- shard-glk: NOTRUN -> [SKIP][15] ([fdo#109271]) +7 similar issues
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-glk8/igt@kms_frontbuffer_tracking@fbcpsr-2p-scndscrn-pri-indfb-draw-mmap-gtt.html
* igt@kms_plane_alpha_blend@alpha-transparent-fb@pipe-c-dp-1:
- shard-apl: NOTRUN -> [FAIL][16] ([i915#4573]) +1 similar issue
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-apl4/igt@kms_plane_alpha_blend@[email protected]
* igt@kms_plane_scaling@plane-downscale-with-rotation-factor-0-75@pipe-b-dp-1:
- shard-apl: NOTRUN -> [SKIP][17] ([fdo#109271]) +32 similar issues
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-apl4/igt@kms_plane_scaling@[email protected]
#### Possible fixes ####
* igt@drm_fdinfo@virtual-idle:
- {shard-rkl}: [FAIL][18] ([i915#7742]) -> [PASS][19] +2 similar issues
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-rkl-1/igt@[email protected]
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-rkl-4/igt@[email protected]
* igt@fbdev@eof:
- {shard-tglu}: [SKIP][20] ([i915#2582]) -> [PASS][21]
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-tglu-6/igt@[email protected]
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-tglu-3/igt@[email protected]
* igt@fbdev@pan:
- {shard-rkl}: [SKIP][22] ([i915#2582]) -> [PASS][23]
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-rkl-5/igt@[email protected]
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-rkl-6/igt@[email protected]
* igt@gem_ctx_isolation@preservation-s3@vcs0:
- shard-apl: [ABORT][24] -> [PASS][25]
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-apl1/igt@gem_ctx_isolation@[email protected]
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-apl4/igt@gem_ctx_isolation@[email protected]
* igt@gem_ctx_persistence@hang:
- {shard-rkl}: [SKIP][26] ([i915#6252]) -> [PASS][27]
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-rkl-5/igt@[email protected]
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-rkl-3/igt@[email protected]
* igt@gem_eio@in-flight-external:
- {shard-rkl}: [ABORT][28] ([i915#7811]) -> [PASS][29]
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-rkl-1/igt@[email protected]
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-rkl-4/igt@[email protected]
* igt@gem_exec_fair@basic-none@vcs0:
- {shard-rkl}: [FAIL][30] ([i915#2842]) -> [PASS][31] +3 similar issues
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-rkl-3/igt@gem_exec_fair@[email protected]
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-rkl-5/igt@gem_exec_fair@[email protected]
* igt@gem_exec_fair@basic-pace-share@rcs0:
- shard-glk: [FAIL][32] ([i915#2842]) -> [PASS][33]
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-glk1/igt@gem_exec_fair@[email protected]
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-glk2/igt@gem_exec_fair@[email protected]
* igt@gem_exec_reloc@basic-gtt-read-noreloc:
- {shard-rkl}: [SKIP][34] ([i915#3281]) -> [PASS][35] +9 similar issues
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-rkl-2/igt@[email protected]
[35]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-rkl-5/igt@[email protected]
* igt@gem_exec_suspend@basic-s3@smem:
- {shard-rkl}: [ABORT][36] ([i915#5122]) -> [PASS][37]
[36]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-rkl-4/igt@gem_exec_suspend@[email protected]
[37]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-rkl-2/igt@gem_exec_suspend@[email protected]
* igt@gem_mmap_gtt@coherency:
- {shard-rkl}: [SKIP][38] ([fdo#111656]) -> [PASS][39]
[38]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-rkl-3/igt@[email protected]
[39]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-rkl-5/igt@[email protected]
* igt@gem_userptr_blits@forbidden-operations:
- {shard-rkl}: [SKIP][40] ([i915#3282]) -> [PASS][41] +6 similar issues
[40]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-rkl-3/igt@[email protected]
[41]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-rkl-5/igt@[email protected]
* igt@gen9_exec_parse@allowed-single:
- shard-glk: [ABORT][42] ([i915#5566]) -> [PASS][43]
[42]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-glk6/igt@[email protected]
[43]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-glk9/igt@[email protected]
* igt@gen9_exec_parse@bb-chained:
- {shard-rkl}: [SKIP][44] ([i915#2527]) -> [PASS][45] +2 similar issues
[44]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-rkl-3/igt@[email protected]
[45]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-rkl-5/igt@[email protected]
* igt@i915_pm_dc@dc6-dpms:
- {shard-tglu}: [FAIL][46] ([i915#3989] / [i915#454]) -> [PASS][47]
[46]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-tglu-6/igt@[email protected]
[47]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-tglu-4/igt@[email protected]
* igt@i915_pm_rpm@drm-resources-equal:
- {shard-rkl}: [SKIP][48] ([fdo#109308]) -> [PASS][49]
[48]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-rkl-5/igt@[email protected]
[49]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-rkl-6/igt@[email protected]
* igt@i915_pm_rpm@modeset-lpsp-stress:
- {shard-rkl}: [SKIP][50] ([i915#1397]) -> [PASS][51]
[50]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-rkl-5/igt@[email protected]
[51]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-rkl-6/igt@[email protected]
- {shard-dg1}: [SKIP][52] ([i915#1397]) -> [PASS][53]
[52]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-dg1-16/igt@[email protected]
[53]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-dg1-14/igt@[email protected]
* igt@kms_ccs@pipe-c-bad-pixel-format-y_tiled_gen12_rc_ccs:
- {shard-tglu}: [SKIP][54] ([i915#1845] / [i915#7651]) -> [PASS][55] +28 similar issues
[54]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-tglu-6/igt@kms_ccs@pipe-c-bad-pixel-format-y_tiled_gen12_rc_ccs.html
[55]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-tglu-3/igt@kms_ccs@pipe-c-bad-pixel-format-y_tiled_gen12_rc_ccs.html
* igt@kms_cursor_legacy@flip-vs-cursor@atomic-transitions:
- shard-glk: [FAIL][56] ([i915#2346]) -> [PASS][57]
[56]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-glk4/igt@kms_cursor_legacy@[email protected]
[57]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-glk8/igt@kms_cursor_legacy@[email protected]
* igt@kms_frontbuffer_tracking@fbc-1p-indfb-fliptrack-mmap-gtt:
- {shard-tglu}: [SKIP][58] ([i915#1849]) -> [PASS][59] +2 similar issues
[58]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-tglu-6/igt@[email protected]
[59]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-tglu-4/igt@[email protected]
* igt@kms_frontbuffer_tracking@psr-1p-offscren-pri-shrfb-draw-pwrite:
- {shard-rkl}: [SKIP][60] ([i915#1849] / [i915#4098]) -> [PASS][61] +6 similar issues
[60]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-rkl-5/igt@kms_frontbuffer_tracking@psr-1p-offscren-pri-shrfb-draw-pwrite.html
[61]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-rkl-6/igt@kms_frontbuffer_tracking@psr-1p-offscren-pri-shrfb-draw-pwrite.html
* igt@kms_properties@crtc-properties-atomic:
- {shard-rkl}: [SKIP][62] ([i915#1849]) -> [PASS][63]
[62]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-rkl-5/igt@[email protected]
[63]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-rkl-6/igt@[email protected]
* igt@kms_psr@cursor_plane_onoff:
- {shard-rkl}: [SKIP][64] ([i915#1072]) -> [PASS][65] +1 similar issue
[64]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-rkl-5/igt@kms_psr@cursor_plane_onoff.html
[65]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-rkl-6/igt@kms_psr@cursor_plane_onoff.html
* igt@kms_rotation_crc@primary-rotation-90:
- {shard-rkl}: [SKIP][66] ([i915#1845] / [i915#4098]) -> [PASS][67] +7 similar issues
[66]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-rkl-5/igt@[email protected]
[67]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-rkl-6/igt@[email protected]
* igt@perf@gen8-unprivileged-single-ctx-counters:
- {shard-rkl}: [SKIP][68] ([i915#2436]) -> [PASS][69]
[68]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-rkl-3/igt@[email protected]
[69]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-rkl-5/igt@[email protected]
* igt@perf@polling-small-buf:
- {shard-rkl}: [FAIL][70] ([i915#1722]) -> [PASS][71]
[70]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12744/shard-rkl-3/igt@[email protected]
[71]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/shard-rkl-5/igt@[email protected]
{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).
[fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
[fdo#109274]: https://bugs.freedesktop.org/show_bug.cgi?id=109274
[fdo#109280]: https://bugs.freedesktop.org/show_bug.cgi?id=109280
[fdo#109283]: https://bugs.freedesktop.org/show_bug.cgi?id=109283
[fdo#109285]: https://bugs.freedesktop.org/show_bug.cgi?id=109285
[fdo#109289]: https://bugs.freedesktop.org/show_bug.cgi?id=109289
[fdo#109291]: https://bugs.freedesktop.org/show_bug.cgi?id=109291
[fdo#109295]: https://bugs.freedesktop.org/show_bug.cgi?id=109295
[fdo#109308]: https://bugs.freedesktop.org/show_bug.cgi?id=109308
[fdo#109313]: https://bugs.freedesktop.org/show_bug.cgi?id=109313
[fdo#109315]: https://bugs.freedesktop.org/show_bug.cgi?id=109315
[fdo#109506]: https://bugs.freedesktop.org/show_bug.cgi?id=109506
[fdo#110189]: https://bugs.freedesktop.org/show_bug.cgi?id=110189
[fdo#110723]: https://bugs.freedesktop.org/show_bug.cgi?id=110723
[fdo#111068]: https://bugs.freedesktop.org/show_bug.cgi?id=111068
[fdo#111614]: https://bugs.freedesktop.org/show_bug.cgi?id=111614
[fdo#111615]: https://bugs.freedesktop.org/show_bug.cgi?id=111615
[fdo#111644]: https://bugs.freedesktop.org/show_bug.cgi?id=111644
[fdo#111656]: https://bugs.freedesktop.org/show_bug.cgi?id=111656
[fdo#111825]: https://bugs.freedesktop.org/show_bug.cgi?id=111825
[fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
[fdo#112283]: https://bugs.freedesktop.org/show_bug.cgi?id=112283
[i915#1072]: https://gitlab.freedesktop.org/drm/intel/issues/1072
[i915#132]: https://gitlab.freedesktop.org/drm/intel/issues/132
[i915#1397]: https://gitlab.freedesktop.org/drm/intel/issues/1397
[i915#1722]: https://gitlab.freedesktop.org/drm/intel/issues/1722
[i915#1755]: https://gitlab.freedesktop.org/drm/intel/issues/1755
[i915#1825]: https://gitlab.freedesktop.org/drm/intel/issues/1825
[i915#1839]: https://gitlab.freedesktop.org/drm/intel/issues/1839
[i915#1845]: https://gitlab.freedesktop.org/drm/intel/issues/1845
[i915#1849]: https://gitlab.freedesktop.org/drm/intel/issues/1849
[i915#1902]: https://gitlab.freedesktop.org/drm/intel/issues/1902
[i915#2346]: https://gitlab.freedesktop.org/drm/intel/issues/2346
[i915#2434]: https://gitlab.freedesktop.org/drm/intel/issues/2434
[i915#2436]: https://gitlab.freedesktop.org/drm/intel/issues/2436
[i915#2527]: https://gitlab.freedesktop.org/drm/intel/issues/2527
[i915#2575]: https://gitlab.freedesktop.org/drm/intel/issues/2575
[i915#2582]: https://gitlab.freedesktop.org/drm/intel/issues/2582
[i915#2587]: https://gitlab.freedesktop.org/drm/intel/issues/2587
[i915#2658]: https://gitlab.freedesktop.org/drm/intel/issues/2658
[i915#2672]: https://gitlab.freedesktop.org/drm/intel/issues/2672
[i915#2705]: https://gitlab.freedesktop.org/drm/intel/issues/2705
[i915#280]: https://gitlab.freedesktop.org/drm/intel/issues/280
[i915#2842]: https://gitlab.freedesktop.org/drm/intel/issues/2842
[i915#2846]: https://gitlab.freedesktop.org/drm/intel/issues/2846
[i915#2856]: https://gitlab.freedesktop.org/drm/intel/issues/2856
[i915#2920]: https://gitlab.freedesktop.org/drm/intel/issues/2920
[i915#3116]: https://gitlab.freedesktop.org/drm/intel/issues/3116
[i915#3281]: https://gitlab.freedesktop.org/drm/intel/issues/3281
[i915#3282]: https://gitlab.freedesktop.org/drm/intel/issues/3282
[i915#3291]: https://gitlab.freedesktop.org/drm/intel/issues/3291
[i915#3297]: https://gitlab.freedesktop.org/drm/intel/issues/3297
[i915#3299]: https://gitlab.freedesktop.org/drm/intel/issues/3299
[i915#3318]: https://gitlab.freedesktop.org/drm/intel/issues/3318
[i915#3359]: https://gitlab.freedesktop.org/drm/intel/issues/3359
[i915#3458]: https://gitlab.freedesktop.org/drm/intel/issues/3458
[i915#3469]: https://gitlab.freedesktop.org/drm/intel/issues/3469
[i915#3539]: https://gitlab.freedesktop.org/drm/intel/issues/3539
[i915#3546]: https://gitlab.freedesktop.org/drm/intel/issues/3546
[i915#3547]: https://gitlab.freedesktop.org/drm/intel/issues/3547
[i915#3555]: https://gitlab.freedesktop.org/drm/intel/issues/3555
[i915#3558]: https://gitlab.freedesktop.org/drm/intel/issues/3558
[i915#3637]: https://gitlab.freedesktop.org/drm/intel/issues/3637
[i915#3638]: https://gitlab.freedesktop.org/drm/intel/issues/3638
[i915#3689]: https://gitlab.freedesktop.org/drm/intel/issues/3689
[i915#3708]: https://gitlab.freedesktop.org/drm/intel/issues/3708
[i915#3734]: https://gitlab.freedesktop.org/drm/intel/issues/3734
[i915#3742]: https://gitlab.freedesktop.org/drm/intel/issues/3742
[i915#3804]: https://gitlab.freedesktop.org/drm/intel/issues/3804
[i915#3840]: https://gitlab.freedesktop.org/drm/intel/issues/3840
[i915#3886]: https://gitlab.freedesktop.org/drm/intel/issues/3886
[i915#3952]: https://gitlab.freedesktop.org/drm/intel/issues/3952
[i915#3955]: https://gitlab.freedesktop.org/drm/intel/issues/3955
[i915#3989]: https://gitlab.freedesktop.org/drm/intel/issues/3989
[i915#404]: https://gitlab.freedesktop.org/drm/intel/issues/404
[i915#4070]: https://gitlab.freedesktop.org/drm/intel/issues/4070
[i915#4077]: https://gitlab.freedesktop.org/drm/intel/issues/4077
[i915#4079]: https://gitlab.freedesktop.org/drm/intel/issues/4079
[i915#4083]: https://gitlab.freedesktop.org/drm/intel/issues/4083
[i915#4098]: https://gitlab.freedesktop.org/drm/intel/issues/4098
[i915#4103]: https://gitlab.freedesktop.org/drm/intel/issues/4103
[i915#4212]: https://gitlab.freedesktop.org/drm/intel/issues/4212
[i915#4270]: https://gitlab.freedesktop.org/drm/intel/issues/4270
[i915#4387]: https://gitlab.freedesktop.org/drm/intel/issues/4387
[i915#4391]: https://gitlab.freedesktop.org/drm/intel/issues/4391
[i915#4538]: https://gitlab.freedesktop.org/drm/intel/issues/4538
[i915#454]: https://gitlab.freedesktop.org/drm/intel/issues/454
[i915#4565]: https://gitlab.freedesktop.org/drm/intel/issues/4565
[i915#4573]: https://gitlab.freedesktop.org/drm/intel/issues/4573
[i915#4613]: https://gitlab.freedesktop.org/drm/intel/issues/4613
[i915#4771]: https://gitlab.freedesktop.org/drm/intel/issues/4771
[i915#4812]: https://gitlab.freedesktop.org/drm/intel/issues/4812
[i915#4833]: https://gitlab.freedesktop.org/drm/intel/issues/4833
[i915#4852]: https://gitlab.freedesktop.org/drm/intel/issues/4852
[i915#4860]: https://gitlab.freedesktop.org/drm/intel/issues/4860
[i915#4881]: https://gitlab.freedesktop.org/drm/intel/issues/4881
[i915#4936]: https://gitlab.freedesktop.org/drm/intel/issues/4936
[i915#5030]: https://gitlab.freedesktop.org/drm/intel/issues/5030
[i915#5115]: https://gitlab.freedesktop.org/drm/intel/issues/5115
[i915#5122]: https://gitlab.freedesktop.org/drm/intel/issues/5122
[i915#5176]: https://gitlab.freedesktop.org/drm/intel/issues/5176
[i915#5286]: https://gitlab.freedesktop.org/drm/intel/issues/5286
[i915#5289]: https://gitlab.freedesktop.org/drm/intel/issues/5289
[i915#5325]: https://gitlab.freedesktop.org/drm/intel/issues/5325
[i915#533]: https://gitlab.freedesktop.org/drm/intel/issues/533
[i915#5334]: https://gitlab.freedesktop.org/drm/intel/issues/5334
[i915#5439]: https://gitlab.freedesktop.org/drm/intel/issues/5439
[i915#5461]: https://gitlab.freedesktop.org/drm/intel/issues/5461
[i915#5563]: https://gitlab.freedesktop.org/drm/intel/issues/5563
[i915#5566]: https://gitlab.freedesktop.org/drm/intel/issues/5566
[i915#5723]: https://gitlab.freedesktop.org/drm/intel/issues/5723
[i915#6095]: https://gitlab.freedesktop.org/drm/intel/issues/6095
[i915#6227]: https://gitlab.freedesktop.org/drm/intel/issues/6227
[i915#6247]: https://gitlab.freedesktop.org/drm/intel/issues/6247
[i915#6248]: https://gitlab.freedesktop.org/drm/intel/issues/6248
[i915#6252]: https://gitlab.freedesktop.org/drm/intel/issues/6252
[i915#6301]: https://gitlab.freedesktop.org/drm/intel/issues/6301
[i915#6412]: https://gitlab.freedesktop.org/drm/intel/issues/6412
[i915#6433]: https://gitlab.freedesktop.org/drm/intel/issues/6433
[i915#6497]: https://gitlab.freedesktop.org/drm/intel/issues/6497
[i915#6524]: https://gitlab.freedesktop.org/drm/intel/issues/6524
[i915#658]: https://gitlab.freedesktop.org/drm/intel/issues/658
[i915#6768]: https://gitlab.freedesktop.org/drm/intel/issues/6768
[i915#6944]: https://gitlab.freedesktop.org/drm/intel/issues/6944
[i915#6953]: https://gitlab.freedesktop.org/drm/intel/issues/6953
[i915#7052]: https://gitlab.freedesktop.org/drm/intel/issues/7052
[i915#7116]: https://gitlab.freedesktop.org/drm/intel/issues/7116
[i915#7118]: https://gitlab.freedesktop.org/drm/intel/issues/7118
[i915#7128]: https://gitlab.freedesktop.org/drm/intel/issues/7128
[i915#7173]: https://gitlab.freedesktop.org/drm/intel/issues/7173
[i915#7294]: https://gitlab.freedesktop.org/drm/intel/issues/7294
[i915#7330]: https://gitlab.freedesktop.org/drm/intel/issues/7330
[i915#7456]: https://gitlab.freedesktop.org/drm/intel/issues/7456
[i915#7561]: https://gitlab.freedesktop.org/drm/intel/issues/7561
[i915#7582]: https://gitlab.freedesktop.org/drm/intel/issues/7582
[i915#7651]: https://gitlab.freedesktop.org/drm/intel/issues/7651
[i915#7697]: https://gitlab.freedesktop.org/drm/intel/issues/7697
[i915#7701]: https://gitlab.freedesktop.org/drm/intel/issues/7701
[i915#7711]: https://gitlab.freedesktop.org/drm/intel/issues/7711
[i915#7742]: https://gitlab.freedesktop.org/drm/intel/issues/7742
[i915#7811]: https://gitlab.freedesktop.org/drm/intel/issues/7811
[i915#7828]: https://gitlab.freedesktop.org/drm/intel/issues/7828
[i915#7850]: https://gitlab.freedesktop.org/drm/intel/issues/7850
[i915#79]: https://gitlab.freedesktop.org/drm/intel/issues/79
[i915#7949]: https://gitlab.freedesktop.org/drm/intel/issues/7949
[i915#7957]: https://gitlab.freedesktop.org/drm/intel/issues/7957
[i915#7984]: https://gitlab.freedesktop.org/drm/intel/issues/7984
[i915#8018]: https://gitlab.freedesktop.org/drm/intel/issues/8018
[i915#8152]: https://gitlab.freedesktop.org/drm/intel/issues/8152
Build changes
-------------
* Linux: CI_DRM_12744 -> Patchwork_114060v1
CI-20190529: 20190529
CI_DRM_12744: 2e4cac06e4de01a35626de676b9eeb474ae7ab5d @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_7160: 45da871dd2684227e93a2fc002b87dfc58bd5fd9 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
Patchwork_114060v1: 2e4cac06e4de01a35626de676b9eeb474ae7ab5d @ git://anongit.freedesktop.org/gfx-ci/linux
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_114060v1/index.html
kernel test robot
build botRe: [Intel-gfx] [RFC PATCH 14/16] drm/ttm: Provide helpers for shrinking
In reply to Thomas Hellström[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in…
Hi Thomas,
[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on drm-misc/drm-misc-next]
[also build test ERROR on drm-intel/for-linux-next drm-tip/drm-tip] [cannot apply to drm-intel/for-linux-next-fixes linus/master v6.2-rc8 next-20230216] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Thomas-Hellstr-m/drm-ttm-Fix-a-NULL-pointer-dereference/20230216-001727 base: git://anongit.freedesktop.org/drm/drm-misc drm-misc-next patch link: https://lore.kernel.org/r/20230215161405.187368-15-thomas.hellstrom%40linux.intel.com patch subject: [Intel-gfx] [RFC PATCH 14/16] drm/ttm: Provide helpers for shrinking config: sparc-randconfig-r011-20230213 (https://download.01.org/0day-ci/archive/20230217/[email protected]/config) compiler: sparc64-linux-gcc (GCC) 12.1.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/888719f1ca5b28696b77e89e4d365aba16053df3
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Thomas-Hellstr-m/drm-ttm-Fix-a-NULL-pointer-dereference/20230216-001727 git checkout 888719f1ca5b28696b77e89e4d365aba16053df3
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=sparc olddefconfig COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=sparc SHELL=/bin/bash
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Link: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
In file included from arch/sparc/include/asm/percpu_64.h:5,
from arch/sparc/include/asm/percpu.h:5,
from include/linux/irqflags.h:17,
from include/asm-generic/cmpxchg-local.h:6,
from arch/sparc/include/asm/cmpxchg_64.h:111,
from arch/sparc/include/asm/cmpxchg.h:5,
from arch/sparc/include/asm/atomic_64.h:12,
from arch/sparc/include/asm/atomic.h:5,
from include/linux/atomic.h:7,
from include/linux/mm_types_task.h:13,
from include/linux/mm_types.h:5,
from mm/swap_backup_folio.c:3:
mm/swap_backup_folio.c: In function 'swap_backup_folio':
>> include/linux/compiler.h:56:45: error: invalid use of void expression
56 | #define if(cond, ...) if ( __trace_if_var( !!(cond , ## __VA_ARGS__) ) )
| ^
include/linux/compiler.h:58:52: note: in definition of macro '__trace_if_var'
58 | #define __trace_if_var(cond) (__builtin_constant_p(cond) ? (cond) : __trace_if_value(cond))
| ^~~~
mm/swap_backup_folio.c:36:9: note: in expansion of macro 'if'
36 | if (VM_WARN_ON_ONCE_FOLIO(folio_nr_pages(folio) != 1 &&
| ^~
>> include/linux/compiler.h:56:45: error: invalid use of void expression
56 | #define if(cond, ...) if ( __trace_if_var( !!(cond , ## __VA_ARGS__) ) )
| ^
include/linux/compiler.h:58:61: note: in definition of macro '__trace_if_var'
58 | #define __trace_if_var(cond) (__builtin_constant_p(cond) ? (cond) : __trace_if_value(cond))
| ^~~~
mm/swap_backup_folio.c:36:9: note: in expansion of macro 'if'
36 | if (VM_WARN_ON_ONCE_FOLIO(folio_nr_pages(folio) != 1 &&
| ^~
>> include/linux/compiler.h:56:45: error: invalid use of void expression
56 | #define if(cond, ...) if ( __trace_if_var( !!(cond , ## __VA_ARGS__) ) )
| ^
include/linux/compiler.h:69:10: note: in definition of macro '__trace_if_value'
69 | (cond) ? \
| ^~~~
include/linux/compiler.h:56:28: note: in expansion of macro '__trace_if_var'
56 | #define if(cond, ...) if ( __trace_if_var( !!(cond , ## __VA_ARGS__) ) )
| ^~~~~~~~~~~~~~
mm/swap_backup_folio.c:36:9: note: in expansion of macro 'if'
36 | if (VM_WARN_ON_ONCE_FOLIO(folio_nr_pages(folio) != 1 &&
| ^~
>> include/linux/compiler.h:56:45: error: invalid use of void expression
56 | #define if(cond, ...) if ( __trace_if_var( !!(cond , ## __VA_ARGS__) ) )
| ^
include/linux/compiler.h:58:52: note: in definition of macro '__trace_if_var'
58 | #define __trace_if_var(cond) (__builtin_constant_p(cond) ? (cond) : __trace_if_value(cond))
| ^~~~
mm/swap_backup_folio.c:42:9: note: in expansion of macro 'if'
42 | if (VM_WARN_ON_ONCE_FOLIO(folio_ref_count(folio) != 1 ||
| ^~
>> include/linux/compiler.h:56:45: error: invalid use of void expression
56 | #define if(cond, ...) if ( __trace_if_var( !!(cond , ## __VA_ARGS__) ) )
| ^
include/linux/compiler.h:58:61: note: in definition of macro '__trace_if_var'
58 | #define __trace_if_var(cond) (__builtin_constant_p(cond) ? (cond) : __trace_if_value(cond))
| ^~~~
mm/swap_backup_folio.c:42:9: note: in expansion of macro 'if'
42 | if (VM_WARN_ON_ONCE_FOLIO(folio_ref_count(folio) != 1 ||
| ^~
>> include/linux/compiler.h:56:45: error: invalid use of void expression
56 | #define if(cond, ...) if ( __trace_if_var( !!(cond , ## __VA_ARGS__) ) )
| ^
include/linux/compiler.h:69:10: note: in definition of macro '__trace_if_value'
69 | (cond) ? \
| ^~~~
include/linux/compiler.h:56:28: note: in expansion of macro '__trace_if_var'
56 | #define if(cond, ...) if ( __trace_if_var( !!(cond , ## __VA_ARGS__) ) )
| ^~~~~~~~~~~~~~
mm/swap_backup_folio.c:42:9: note: in expansion of macro 'if'
42 | if (VM_WARN_ON_ONCE_FOLIO(folio_ref_count(folio) != 1 ||
| ^~vim +56 include/linux/compiler.h
2bcd521a684cc9 Steven Rostedt 2008-11-21 50 2bcd521a684cc9 Steven Rostedt 2008-11-21 51 #ifdef CONFIG_PROFILE_ALL_BRANCHES 2bcd521a684cc9 Steven Rostedt 2008-11-21 52 /* 2bcd521a684cc9 Steven Rostedt 2008-11-21 53 * "Define 'is'", Bill Clinton 2bcd521a684cc9 Steven Rostedt 2008-11-21 54 * "Define 'if'", Steven Rostedt 2bcd521a684cc9 Steven Rostedt 2008-11-21 55 */
a15fd609ad53a6 Linus Torvalds 2019-03-20 @56 #define if(cond, ...) if ( __trace_if_var( !!(cond , ## __VA_ARGS__) ) ) a15fd609ad53a6 Linus Torvalds 2019-03-20 57
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests