Skip to contentSkip to conversation

cgroups: Add support for pinned device memory

RFC In review Maarten Lankhorst 7 participants 20 messages Aug 19, 2025 – Sep 4, 2025

Subsystems (8)
lore
Share & tools
lore

Maarten Lankhorst

author maintainer
#0 lore raw

cover cgroups: Add support for pinned device memory

When exporting dma-bufs to other devices, even when it is allowed to use move_notify in some drivers, performance will degrade severely when eviction happens.

A perticular example where this can happen is in a multi-card setup, where PCI-E peer-to-peer is used to prevent using access to system memory.

If the buffer is evicted to system memory, not only the evicting GPU wher the buffer resided is affected, but it will also stall the GPU that is waiting on the buffer.

It also makes sense for long running jobs not to be preempted by having its buffers evicted, so it will make sense to have the ability to pin from system memory too.

This is dependant on patches by Dave Airlie, so it's not part of this series yet. But I'm planning on extending pinning to the memory cgroup controller in the future to handle this case.

Implementation details:

For each cgroup up until the root cgroup, the 'min' limit is checked against currently effectively pinned value. If the value will go above 'min', the pinning attempt is rejected.

Pinned memory is handled slightly different and affects calculating effective min/low values. Pinned memory is subtracted from both, and needs to be added afterwards when calculating.

This is because increasing the amount of pinned memory, the amount of free min/low memory decreases for all cgroups that are part of the hierarchy.

Maarten Lankhorst (3):
page_counter: Allow for pinning some amount of memory cgroup/dmem: Implement pinning device memory
drm/xe: Add DRM_XE_GEM_CREATE_FLAG_PINNED flag and implementation

 drivers/gpu/drm/xe/xe_bo.c      | 66 +++++++++++++++++++++-
 drivers/gpu/drm/xe/xe_dma_buf.c | 10 +++-
 include/linux/cgroup_dmem.h     |  2 +
 include/linux/page_counter.h    |  8 +++
 include/uapi/drm/xe_drm.h       | 10 +++-
 kernel/cgroup/dmem.c            | 57 ++++++++++++++++++-
 mm/page_counter.c               | 98 ++++++++++++++++++++++++++++++---
 7 files changed, 237 insertions(+), 14 deletions(-)
Signature · 1 line
2.50.0
author maintainer
#1 lore raw

RFC 1/3 page_counter: Allow for pinning some amount of memory

Add a pinned member, and use it for implementing pinning accounting. Memory to be pinned has to already be accounted for as normally used, and only memory up to the 'min' limit is allowed to be pinned.

This limit is chosen because cgroups already guarantees that memory up to that limit will not evicted.

Pinned memory affects min and low calculations, so alter those slightly.

---
 include/linux/page_counter.h |  8 +++
 mm/page_counter.c            | 98 +++++++++++++++++++++++++++++++++---
 2 files changed, 99 insertions(+), 7 deletions(-)
include/linux/page_counter.h +8 -0
index d649b6bbbc871..5836c6dfb3c76 100644
@@ -13,6 +13,7 @@ struct page_counter {
 	 * v2. The memcg->memory.usage is a hot member of struct mem_cgroup.
 	 */
 	atomic_long_t usage;
+	atomic_long_t pinned;
 	unsigned long failcnt; /* v1-only field */
 
 	CACHELINE_PADDING(_pad1_);
@@ -68,11 +69,18 @@ static inline unsigned long page_counter_read(struct page_counter *counter)
 	return atomic_long_read(&counter->usage);
 }
 
+static inline unsigned long page_counter_pinned(struct page_counter *counter)
+{
+	return atomic_long_read(&counter->pinned);
+}
+
 void page_counter_cancel(struct page_counter *counter, unsigned long nr_pages);
 void page_counter_charge(struct page_counter *counter, unsigned long nr_pages);
 bool page_counter_try_charge(struct page_counter *counter,
 			     unsigned long nr_pages,
 			     struct page_counter **fail);
+bool page_counter_try_pin(struct page_counter *counter, unsigned long nr_pages);
+void page_counter_unpin(struct page_counter *counter, unsigned long nr_pages);
 void page_counter_uncharge(struct page_counter *counter, unsigned long nr_pages);
 void page_counter_set_min(struct page_counter *counter, unsigned long nr_pages);
 void page_counter_set_low(struct page_counter *counter, unsigned long nr_pages);
mm/page_counter.c +91 -7

127 lines not on this page · raw message

Signature · 1 line
2.50.0
author maintainer
#2 lore raw

RFC 2/3 cgroup/dmem: Implement pinning device memory

In reply to Maarten Lankhorst

Add a function to pin, and to unipn memory and adjust the calculations in dmem_cgroup_state_evict_valuable().

---
 include/linux/cgroup_dmem.h |  2 ++
 kernel/cgroup/dmem.c        | 57 +++++++++++++++++++++++++++++++++++--
 2 files changed, 56 insertions(+), 3 deletions(-)
include/linux/cgroup_dmem.h +2 -0
index dd4869f1d736e..a981bb692ba22 100644
@@ -21,6 +21,8 @@ int dmem_cgroup_try_charge(struct dmem_cgroup_region *region, u64 size,
 			   struct dmem_cgroup_pool_state **ret_pool,
 			   struct dmem_cgroup_pool_state **ret_limit_pool);
 void dmem_cgroup_uncharge(struct dmem_cgroup_pool_state *pool, u64 size);
+int dmem_cgroup_try_pin(struct dmem_cgroup_pool_state *pool, u64 size);
+void dmem_cgroup_unpin(struct dmem_cgroup_pool_state *pool, u64 size);
 bool dmem_cgroup_state_evict_valuable(struct dmem_cgroup_pool_state *limit_pool,
 				      struct dmem_cgroup_pool_state *test_pool,
 				      bool ignore_low, bool *ret_hit_low);
kernel/cgroup/dmem.c +54 -3

101 lines not on this page · raw message

Signature · 1 line
2.50.0
author maintainer
#3 lore raw

RFC 3/3 drm/xe: Add DRM_XE_GEM_CREATE_FLAG_PINNED flag and implementation

In reply to Maarten Lankhorst

Add an option to pin memory through the science of cgroup accounting. A bo will be pinned for its entire lifetime, and this allows buffers that are pinned for dma-buf export without requiring the pinning to be done at the dma-buf layer for all devices.

For now only implement VRAM pinning. Dave Airlie has a series to implement memcg accounting for the GPU but that is not ready yet.

---
 drivers/gpu/drm/xe/xe_bo.c      | 66 ++++++++++++++++++++++++++++++++-
 drivers/gpu/drm/xe/xe_dma_buf.c | 10 ++++-
 include/uapi/drm/xe_drm.h       | 10 ++++-
 3 files changed, 82 insertions(+), 4 deletions(-)
drivers/gpu/drm/xe/xe_bo.c +64 -2
index 6fea39842e1e6..4095e6bd04ea9 100644
@@ -5,6 +5,7 @@
 
 #include "xe_bo.h"
 
+#include <linux/cgroup_dmem.h>
 #include <linux/dma-buf.h>
 #include <linux/nospec.h>
 
@@ -208,7 +209,8 @@ static bool force_contiguous(u32 bo_flags)
 	 * must be contiguous, also only contiguous BOs support xe_bo_vmap.
 	 */
 	return bo_flags & XE_BO_FLAG_NEEDS_CPU_ACCESS &&
-	       bo_flags & XE_BO_FLAG_PINNED;
+	       bo_flags & XE_BO_FLAG_PINNED &&
+	       !(bo_flags & XE_BO_FLAG_USER);
 }
 
 static void add_vram(struct xe_device *xe, struct xe_bo *bo,
@@ -1697,6 +1699,16 @@ static void xe_gem_object_free(struct drm_gem_object *obj)
 	ttm_bo_put(container_of(obj, struct ttm_buffer_object, base));
 }
 
+static void xe_bo_unpin_user(struct xe_bo *bo)
+{
+	xe_bo_unpin_external(bo);
+
+	if (bo->flags & XE_BO_FLAG_SYSTEM)
+		WARN_ON(1);
+	else
+		dmem_cgroup_unpin(bo->ttm.resource->css, xe_bo_size(bo));
+}
+
 static void xe_gem_object_close(struct drm_gem_object *obj,
 				struct drm_file *file_priv)
 {
@@ -1708,6 +1720,10 @@ static void xe_gem_object_close(struct drm_gem_object *obj,
 		xe_bo_lock(bo, false);
 		ttm_bo_set_bulk_move(&bo->ttm, NULL);
 		xe_bo_unlock(bo);
+	} else if (bo->flags & XE_BO_FLAG_PINNED) {
+		xe_bo_lock(bo, false);
+		xe_bo_unpin_user(bo);
+		xe_bo_unlock(bo);
 	}
 }
 
@@ -2128,8 +2144,27 @@ struct xe_bo *xe_bo_create_user(struct xe_device *xe, struct xe_tile *tile,
 	struct xe_bo *bo = __xe_bo_create_locked(xe, tile, vm, size, 0, ~0ULL,
 						 cpu_caching, ttm_bo_type_device,
 						 flags | XE_BO_FLAG_USER, 0);
-	if (!IS_ERR(bo))
+	if (!IS_ERR(bo)) {
+		int ret = 0;
+
+		if (bo->flags & XE_BO_FLAG_PINNED) {
+			if (bo->flags & XE_BO_FLAG_SYSTEM) {
+				ret = -ENOSYS; // TODO
+			} else {
+				ret = dmem_cgroup_try_pin(bo->ttm.resource->css, size);
+			}
+			if (!ret)
+				ret = xe_bo_pin_external(bo);
+			else if (ret == -EAGAIN)
+				ret = -ENOSPC;
+		}
+
 		xe_bo_unlock_vm_held(bo);
+		if (ret) {
+			xe_bo_put(bo);
+			return ERR_PTR(ret);
+		}
+	}
 
 	return bo;
 }
@@ -2745,6 +2780,28 @@ int xe_gem_create_ioctl(struct drm_device *dev, void *data,
 			 args->cpu_caching == DRM_XE_GEM_CPU_CACHING_WB))
 		return -EINVAL;
 
+	if (XE_IOCTL_DBG(xe, args->flags & DRM_XE_GEM_CREATE_FLAG_PINNED)) {
+		bool pinned_flag = true;
+		/* Only allow a single placement for pinning */
+		if (XE_IOCTL_DBG(xe, pinned_flag && hweight32(args->placement) != 1))
+			return -EINVAL;
+
+		/* Meant for exporting, do not allow a VM-local BO */
+		if (XE_IOCTL_DBG(xe, pinned_flag && args->vm_id))
+			return -EINVAL;
+
+		/* Similarly, force fail at creation time for now. We may relax this requirement later */
+		if (XE_IOCTL_DBG(xe, pinned_flag && args->flags & DRM_XE_GEM_CREATE_FLAG_DEFER_BACKING))
+			return -EINVAL;
+
+		/* Require the appropriate cgroups to be enabled. */
+		if (XE_IOCTL_DBG(xe, pinned_flag && !IS_ENABLED(CONFIG_CGROUP_DMEM) && bo_flags & XE_BO_FLAG_VRAM_MASK) ||
+		    XE_IOCTL_DBG(xe, pinned_flag && !IS_ENABLED(CONFIG_MEMCG) && bo_flags & XE_BO_FLAG_SYSTEM))
+			return -EINVAL;
+
+		bo_flags |= XE_BO_FLAG_PINNED;
+	}
+
 	if (args->vm_id) {
 		vm = xe_vm_lookup(xef, args->vm_id);
 		if (XE_IOCTL_DBG(xe, !vm))
@@ -2790,6 +2847,11 @@ int xe_gem_create_ioctl(struct drm_device *dev, void *data,
 		__xe_bo_unset_bulk_move(bo);
 		xe_vm_unlock(vm);
 	}
+	if (bo->flags & XE_BO_FLAG_PINNED) {
+		xe_bo_lock(bo, false);
+		xe_bo_unpin_user(bo);
+		xe_bo_unlock(bo);
+	}
 out_put:
 	xe_bo_put(bo);
 out_vm:
drivers/gpu/drm/xe/xe_dma_buf.c +9 -1
index 346f857f38374..6719f4552ad37 100644
@@ -53,6 +53,11 @@ static int xe_dma_buf_pin(struct dma_buf_attachment *attach)
 	struct xe_device *xe = xe_bo_device(bo);
 	int ret;
 
+	if (bo->flags & XE_BO_FLAG_PINNED) {
+		ttm_bo_pin(&bo->ttm);
+		return 0;
+	}
+
 	/*
 	 * For now only support pinning in TT memory, for two reasons:
 	 * 1) Avoid pinning in a placement not accessible to some importers.
@@ -83,7 +88,10 @@ static void xe_dma_buf_unpin(struct dma_buf_attachment *attach)
 	struct drm_gem_object *obj = attach->dmabuf->priv;
 	struct xe_bo *bo = gem_to_xe_bo(obj);
 
-	xe_bo_unpin_external(bo);
+	if (bo->flags & XE_BO_FLAG_PINNED)
+		ttm_bo_unpin(&bo->ttm);
+	else
+		xe_bo_unpin_external(bo);
 }
 
 static struct sg_table *xe_dma_buf_map(struct dma_buf_attachment *attach,
include/uapi/drm/xe_drm.h +9 -1
index c721e130c1d2d..3184fa38ce17e 100644
@@ -765,12 +765,15 @@ struct drm_xe_device_query {
  *    until the object is either bound to a virtual memory region via
  *    VM_BIND or accessed by the CPU. As a result, no backing memory is
  *    reserved at the time of GEM object creation.
- *  - %DRM_XE_GEM_CREATE_FLAG_SCANOUT
+ *  - %DRM_XE_GEM_CREATE_FLAG_SCANOUT - GEM object will be used
+ *    display framebuffer.
  *  - %DRM_XE_GEM_CREATE_FLAG_NEEDS_VISIBLE_VRAM - When using VRAM as a
  *    possible placement, ensure that the corresponding VRAM allocation
  *    will always use the CPU accessible part of VRAM. This is important
  *    for small-bar systems (on full-bar systems this gets turned into a
  *    noop).
+ *  - %DRM_XE_GEM_CREATE_FLAG_PINNED - Pin the backing memory permanently
+ *    on allocation, if withing cgroups limits.
  *    Note1: System memory can be used as an extra placement if the kernel
  *    should spill the allocation to system memory, if space can't be made
  *    available in the CPU accessible part of VRAM (giving the same
@@ -781,6 +784,10 @@ struct drm_xe_device_query {
  *    need to use VRAM for display surfaces, therefore the kernel requires
  *    setting this flag for such objects, otherwise an error is thrown on
  *    small-bar systems.
+ *    Note3: %DRM_XE_GEM_CREATE_FLAG_PINNED requires the BO to have only
+ *    a single placement, no vm_id, requires (device) memory cgroups enabled,
+ *    and is incompatible with the %DEFER_BACKING and %NEEDS_VISIBLE_VRAM
+ *    flags.
  *
  * @cpu_caching supports the following values:
  *  - %DRM_XE_GEM_CPU_CACHING_WB - Allocate the pages with write-back
@@ -827,6 +834,7 @@ struct drm_xe_gem_create {
 #define DRM_XE_GEM_CREATE_FLAG_DEFER_BACKING		(1 << 0)
 #define DRM_XE_GEM_CREATE_FLAG_SCANOUT			(1 << 1)
 #define DRM_XE_GEM_CREATE_FLAG_NEEDS_VISIBLE_VRAM	(1 << 2)
+#define DRM_XE_GEM_CREATE_FLAG_PINNED			(1 << 3)
 	/**
 	 * @flags: Flags, currently a mask of memory instances of where BO can
 	 * be placed
Signature · 1 line
2.50.0
#4 lore raw

Re: [RFC 3/3] drm/xe: Add DRM_XE_GEM_CREATE_FLAG_PINNED flag and implementation

Hi, Maarten,

Quoted 9 lines · Maarten Lankhorst

Add an option to pin memory through the science of cgroup accounting. A bo will be pinned for its entire lifetime, and this allows buffers that are pinned for dma-buf export without requiring the pinning to be
done at the dma-buf layer for all devices.

For now only implement VRAM pinning. Dave Airlie has a series to implement
memcg accounting for the GPU but that is not ready yet.

Previous discussions around this have favoured a UAPI where we pin a gpu-vm range, with a pin at mapping time, or dma-buf pin time where required, this allows for dynamic pinning and unpinning, and would avoid having separate pinning interfaces for bos and userptr.

In particular if we don't know at bo creation time which buffer objects will be exported with a method requiring pinning, how would UMD deduce what buffer objects to pin?

Thanks,
Thomas

Quoted 240 lines

---
 drivers/gpu/drm/xe/xe_bo.c      | 66
++++++++++++++++++++++++++++++++-
 drivers/gpu/drm/xe/xe_dma_buf.c | 10 ++++-
 include/uapi/drm/xe_drm.h       | 10 ++++-

 3 files changed, 82 insertions(+), 4 deletions(-)
drivers/gpu/drm/xe/xe_bo.c +0 -0
index 6fea39842e1e6..4095e6bd04ea9 100644
@@ -5,6 +5,7 @@


 #include "xe_bo.h"

+#include <linux/cgroup_dmem.h>
 #include <linux/dma-buf.h>
 #include <linux/nospec.h>

drivers/gpu/drm/xe/xe_bo.c +0 -0
@@ -208,7 +209,8 @@ static bool force_contiguous(u32 bo_flags)
  	 * must be contiguous, also only contiguous BOs support

xe_bo_vmap.
  */
  return bo_flags & XE_BO_FLAG_NEEDS_CPU_ACCESS &&

drivers/gpu/drm/xe/xe_bo.c +2 -1
-	       bo_flags & XE_BO_FLAG_PINNED;
+	       bo_flags & XE_BO_FLAG_PINNED &&
+	       !(bo_flags & XE_BO_FLAG_USER);

 }

 static void add_vram(struct xe_device *xe, struct xe_bo *bo,

drivers/gpu/drm/xe/xe_bo.c +0 -0
@@ -1697,6 +1699,16 @@ static void xe_gem_object_free(struct
 drm_gem_object *obj)
  	ttm_bo_put(container_of(obj, struct ttm_buffer_object,
 base));
  }

drivers/gpu/drm/xe/xe_bo.c +8 -0
+static void xe_bo_unpin_user(struct xe_bo *bo)
+{
+	xe_bo_unpin_external(bo);
+
+	if (bo->flags & XE_BO_FLAG_SYSTEM)
+		WARN_ON(1);
+	else
+		dmem_cgroup_unpin(bo->ttm.resource->css,

xe_bo_size(bo));

drivers/gpu/drm/xe/xe_bo.c +2 -0
+}
+

 static void xe_gem_object_close(struct drm_gem_object *obj, struct drm_file *file_priv) {

drivers/gpu/drm/xe/xe_bo.c +4 -0
@@ -1708,6 +1720,10 @@ static void xe_gem_object_close(struct
 drm_gem_object *obj,
  		xe_bo_lock(bo, false);
  		ttm_bo_set_bulk_move(&bo->ttm, NULL);
  		xe_bo_unlock(bo);
+	} else if (bo->flags & XE_BO_FLAG_PINNED) {
+		xe_bo_lock(bo, false);
+		xe_bo_unpin_user(bo);
+		xe_bo_unlock(bo);
  	}
  }

drivers/gpu/drm/xe/xe_bo.c +0 -0
@@ -2128,8 +2144,27 @@ struct xe_bo *xe_bo_create_user(struct
 xe_device *xe, struct xe_tile *tile,
  	struct xe_bo *bo = __xe_bo_create_locked(xe, tile, vm, size,
 0, ~0ULL,
  						 cpu_caching,
 ttm_bo_type_device,

  flags | XE_BO_FLAG_USER, 0);

drivers/gpu/drm/xe/xe_bo.c +8 -1
-	if (!IS_ERR(bo))
+	if (!IS_ERR(bo)) {
+		int ret = 0;
+
+		if (bo->flags & XE_BO_FLAG_PINNED) {
+			if (bo->flags & XE_BO_FLAG_SYSTEM) {
+				ret = -ENOSYS; // TODO
+			} else {
+				ret = dmem_cgroup_try_pin(bo-
Quoted 1 line

ttm.resource->css, size);

drivers/gpu/drm/xe/xe_bo.c +7 -0
+			}
+			if (!ret)
+				ret = xe_bo_pin_external(bo);
+			else if (ret == -EAGAIN)
+				ret = -ENOSPC;
+		}
+

  xe_bo_unlock_vm_held(bo);

drivers/gpu/drm/xe/xe_bo.c +5 -0
+		if (ret) {
+			xe_bo_put(bo);
+			return ERR_PTR(ret);
+		}
+	}


  return bo;
 }

drivers/gpu/drm/xe/xe_bo.c +0 -0
@@ -2745,6 +2780,28 @@ int xe_gem_create_ioctl(struct drm_device
 *dev, void *data,

  args->cpu_caching ==
DRM_XE_GEM_CPU_CACHING_WB))
  return -EINVAL;

drivers/gpu/drm/xe/xe_bo.c +1 -0
+	if (XE_IOCTL_DBG(xe, args->flags &

DRM_XE_GEM_CREATE_FLAG_PINNED)) {

drivers/gpu/drm/xe/xe_bo.c +3 -0
+		bool pinned_flag = true;
+		/* Only allow a single placement for pinning */
+		if (XE_IOCTL_DBG(xe, pinned_flag && hweight32(args-
Quoted 1 line

placement) != 1))

drivers/gpu/drm/xe/xe_bo.c +3 -0
+			return -EINVAL;
+
+		/* Meant for exporting, do not allow a VM-local BO

*/

drivers/gpu/drm/xe/xe_bo.c +4 -0
+		if (XE_IOCTL_DBG(xe, pinned_flag && args->vm_id))
+			return -EINVAL;
+
+		/* Similarly, force fail at creation time for now.

We may relax this requirement later */

drivers/gpu/drm/xe/xe_bo.c +1 -0
+		if (XE_IOCTL_DBG(xe, pinned_flag && args->flags &

DRM_XE_GEM_CREATE_FLAG_DEFER_BACKING))

drivers/gpu/drm/xe/xe_bo.c +4 -0
+			return -EINVAL;
+
+		/* Require the appropriate cgroups to be enabled. */
+		if (XE_IOCTL_DBG(xe, pinned_flag &&

!IS_ENABLED(CONFIG_CGROUP_DMEM) && bo_flags & XE_BO_FLAG_VRAM_MASK) ||

drivers/gpu/drm/xe/xe_bo.c +1 -0
+		    XE_IOCTL_DBG(xe, pinned_flag &&

!IS_ENABLED(CONFIG_MEMCG) && bo_flags & XE_BO_FLAG_SYSTEM))

drivers/gpu/drm/xe/xe_bo.c +5 -0
+			return -EINVAL;
+
+		bo_flags |= XE_BO_FLAG_PINNED;
+	}
+

  if (args->vm_id) {
  vm = xe_vm_lookup(xef, args->vm_id); if (XE_IOCTL_DBG(xe, !vm))

drivers/gpu/drm/xe/xe_bo.c +5 -0
@@ -2790,6 +2847,11 @@ int xe_gem_create_ioctl(struct drm_device
 *dev, void *data,
  		__xe_bo_unset_bulk_move(bo);
  		xe_vm_unlock(vm);
  	}
+	if (bo->flags & XE_BO_FLAG_PINNED) {
+		xe_bo_lock(bo, false);
+		xe_bo_unpin_user(bo);
+		xe_bo_unlock(bo);
+	}

 out_put:
  xe_bo_put(bo);
 out_vm:

drivers/gpu/drm/xe/xe_dma_buf.c +0 -0
index 346f857f38374..6719f4552ad37 100644
@@ -53,6 +53,11 @@ static int xe_dma_buf_pin(struct
 dma_buf_attachment *attach)
  	struct xe_device *xe = xe_bo_device(bo);
  	int ret;

drivers/gpu/drm/xe/xe_dma_buf.c +5 -0
+	if (bo->flags & XE_BO_FLAG_PINNED) {
+		ttm_bo_pin(&bo->ttm);
+		return 0;
+	}
+

  /*
  * For now only support pinning in TT memory, for two reasons:
  * 1) Avoid pinning in a placement not accessible to some importers.

drivers/gpu/drm/xe/xe_dma_buf.c +0 -0
@@ -83,7 +88,10 @@ static void xe_dma_buf_unpin(struct
 dma_buf_attachment *attach)
  	struct drm_gem_object *obj = attach->dmabuf->priv;
  	struct xe_bo *bo = gem_to_xe_bo(obj);

drivers/gpu/drm/xe/xe_dma_buf.c +4 -1
-	xe_bo_unpin_external(bo);
+	if (bo->flags & XE_BO_FLAG_PINNED)
+		ttm_bo_unpin(&bo->ttm);
+	else
+		xe_bo_unpin_external(bo);

 }

 static struct sg_table *xe_dma_buf_map(struct dma_buf_attachment *attach,

include/uapi/drm/xe_drm.h +0 -0
index c721e130c1d2d..3184fa38ce17e 100644
@@ -765,12 +765,15 @@ struct drm_xe_device_query {
   *    until the object is either bound to a virtual memory region

via
  *    VM_BIND or accessed by the CPU. As a result, no backing memory is
  *    reserved at the time of GEM object creation.
- *  - %DRM_XE_GEM_CREATE_FLAG_SCANOUT
+ *  - %DRM_XE_GEM_CREATE_FLAG_SCANOUT - GEM object will be used
+ *    display framebuffer.
  *  - %DRM_XE_GEM_CREATE_FLAG_NEEDS_VISIBLE_VRAM - When using VRAM as a
  *    possible placement, ensure that the corresponding VRAM allocation
  *    will always use the CPU accessible part of VRAM. This is important
  *    for small-bar systems (on full-bar systems this gets turned into a
  *    noop).
+ *  - %DRM_XE_GEM_CREATE_FLAG_PINNED - Pin the backing memory permanently
+ *    on allocation, if withing cgroups limits.
  *    Note1: System memory can be used as an extra placement if the kernel
  *    should spill the allocation to system memory, if space can't be made
  *    available in the CPU accessible part of VRAM (giving the same

include/uapi/drm/xe_drm.h +0 -0
@@ -781,6 +784,10 @@ struct drm_xe_device_query {
   *    need to use VRAM for display surfaces, therefore the kernel

requires
  *    setting this flag for such objects, otherwise an error is thrown on
  *    small-bar systems.
+ *    Note3: %DRM_XE_GEM_CREATE_FLAG_PINNED requires the BO to have only
+ *    a single placement, no vm_id, requires (device) memory cgroups enabled,
+ *    and is incompatible with the %DEFER_BACKING and %NEEDS_VISIBLE_VRAM
+ *    flags.
  *
  * @cpu_caching supports the following values:
  *  - %DRM_XE_GEM_CPU_CACHING_WB - Allocate the pages with write- back

include/uapi/drm/xe_drm.h +1 -0
@@ -827,6 +834,7 @@ struct drm_xe_gem_create {
  #define DRM_XE_GEM_CREATE_FLAG_DEFER_BACKING		(1 << 0)
  #define DRM_XE_GEM_CREATE_FLAG_SCANOUT			(1 << 1)
  #define DRM_XE_GEM_CREATE_FLAG_NEEDS_VISIBLE_VRAM	(1 << 2)
+#define DRM_XE_GEM_CREATE_FLAG_PINNED			(1 << 3)
  	/**
  	 * @flags: Flags, currently a mask of memory instances of

where BO can
  * be placed

author maintainer
#5 lore raw

Re: [RFC 3/3] drm/xe: Add DRM_XE_GEM_CREATE_FLAG_PINNED flag and implementation

Hey,

Den 2025-08-19 kl. 18:22, skrev Thomas Hellström:

Quoted 18 lines

Hi, Maarten,

Quoted 9 lines · Maarten Lankhorst

Add an option to pin memory through the science of cgroup accounting. A bo will be pinned for its entire lifetime, and this allows buffers that are pinned for dma-buf export without requiring the pinning to be
done at the dma-buf layer for all devices.

For now only implement VRAM pinning. Dave Airlie has a series to implement
memcg accounting for the GPU but that is not ready yet.

Previous discussions around this have favoured a UAPI where we pin a gpu-vm range, with a pin at mapping time, or dma-buf pin time where required, this allows for dynamic pinning and unpinning, and would avoid having separate pinning interfaces for bos and userptr.

Quoted 7 lines

In particular if we don't know at bo creation time which buffer objects will be exported with a method requiring pinning, how would UMD deduce what buffer objects to pin?

Thanks,
Thomas
For discussion purposes, it seems compute preferred pinning at allocation time,

so I wanted to propose that solution since it's easiest to implement.

I will change it for the next version, but I'm very interested in feedback on the adjustments I made in cgroups. Those are the ones that will have the most impact, and changes how effective min/low is calculated when memory pinned is involved.

Kind regards,
~Maarten

#6 lore raw

Re: [RFC 3/3] drm/xe: Add DRM_XE_GEM_CREATE_FLAG_PINNED flag and implementation

In reply to Maarten Lankhorst
kernel test robot noticed the following build errors:

Hi Maarten,

[This is a private test report for your RFC patch.] kernel test robot noticed the following build errors:

[auto build test ERROR on drm-xe/drm-xe-next]
[also build test ERROR on next-20250819]
[cannot apply to tj-cgroup/for-next akpm-mm/mm-everything linus/master v6.17-rc2] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Maarten-Lankhorst/page_counter-Allow-for-pinning-some-amount-of-memory/20250819-195117
base:   https://gitlab.freedesktop.org/drm/xe/kernel.git drm-xe-next
patch link:    https://lore.kernel.org/r/20250819114932.597600-8-dev%40lankhorst.se
patch subject: [RFC 3/3] drm/xe: Add DRM_XE_GEM_CREATE_FLAG_PINNED flag and implementation
config: i386-buildonly-randconfig-003-20250820 (https://download.01.org/0day-ci/archive/20250820/[email protected]/config)
compiler: gcc-12 (Debian 12.2.0-14+deb12u1) 12.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250820/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

drivers/gpu/drm/xe/xe_bo.c: In function 'xe_bo_unpin_user':

>> drivers/gpu/drm/xe/xe_bo.c:1709:17: error: implicit declaration of function 'dmem_cgroup_unpin'; did you mean 'mem_cgroup_init'? [-Werror=implicit-function-declaration]
    1709 |                 dmem_cgroup_unpin(bo->ttm.resource->css, xe_bo_size(bo));
         |                 ^~~~~~~~~~~~~~~~~
         |                 mem_cgroup_init
   drivers/gpu/drm/xe/xe_bo.c: In function 'xe_bo_create_user':
>> drivers/gpu/drm/xe/xe_bo.c:2154:39: error: implicit declaration of function 'dmem_cgroup_try_pin'; did you mean 'mem_cgroup_tryget'? [-Werror=implicit-function-declaration]
    2154 |                                 ret = dmem_cgroup_try_pin(bo->ttm.resource->css, size);
         |                                       ^~~~~~~~~~~~~~~~~~~
         |                                       mem_cgroup_tryget

cc1: some warnings being treated as errors

vim +1709 drivers/gpu/drm/xe/xe_bo.c

1701
1702 static void xe_bo_unpin_user(struct xe_bo *bo)
1703 {
1704 xe_bo_unpin_external(bo);
1705
1706 if (bo->flags & XE_BO_FLAG_SYSTEM)
1707 WARN_ON(1);
1708 else

> 1709			dmem_cgroup_unpin(bo->ttm.resource->css, xe_bo_size(bo));

1710 }
1711

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#7 lore raw

✗ CI.checkpatch: warning for cgroups: Add support for pinned device memory

In reply to Maarten Lankhorst
== Series Details ==

== Series Details ==

Series: cgroups: Add support for pinned device memory
URL : https://patchwork.freedesktop.org/series/153134/
State : warning

== Summary ==

+ KERNEL=/kernel
+ git clone https://gitlab.freedesktop.org/drm/maintainer-tools mt Cloning into 'mt'...
warning: redirecting to https://gitlab.freedesktop.org/drm/maintainer-tools.git/
+ git -C mt rev-list -n1 origin/master 553439844b6500767ce8aef522cfe9fbb7ece541
+ cd /kernel
+ git config --global --add safe.directory /kernel
+ git log -n1
commit 6adbd62f5794fdbd7fcbd26b53705d8e75f5b88b

Author: Maarten Lankhorst <[email protected]>
Date:   Tue Aug 19 13:49:36 2025 +0200

drm/xe: Add DRM_XE_GEM_CREATE_FLAG_PINNED flag and implementation

Add an option to pin memory through the science of cgroup accounting. A bo will be pinned for its entire lifetime, and this allows buffers that are pinned for dma-buf export without requiring the pinning to be done at the dma-buf layer for all devices.

For now only implement VRAM pinning. Dave Airlie has a series to implement memcg accounting for the GPU but that is not ready yet.

+ /mt/dim checkpatch 8093b67375779a975c13b9a23c8fcc25cb1e9c83 drm-intel d77dfce28451 page_counter: Allow for pinning some amount of memory -:77: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis #77: FILE: mm/page_counter.c:208:
+bool page_counter_try_pin(struct page_counter *counter,
+ unsigned long nr_pages)

-:107: WARNING:DATA_RACE: data_race without comment #107: FILE: mm/page_counter.c:238:
+ data_race(c->failcnt++);

total: 0 errors, 1 warnings, 1 checks, 152 lines checked
33d0ce53c33f cgroup/dmem: Implement pinning device memory
-:97: WARNING:TYPO_SPELLING: 'succesfully' may be misspelled - perhaps 'successfully'? #97: FILE: kernel/cgroup/dmem.c:674:
+ * If the function succeeds, the memory is succesfully accounted as being pinned.

                                            ^^^^^^^^^^^

-:106: CHECK:BRACES: Blank lines aren't necessary before a close brace '}' #106: FILE: kernel/cgroup/dmem.c:683:
+
+}

total: 0 errors, 1 warnings, 1 checks, 109 lines checked 6adbd62f5794 drm/xe: Add DRM_XE_GEM_CREATE_FLAG_PINNED flag and implementation -:76: WARNING:BRACES: braces {} are not necessary for any arm of this statement #76: FILE: drivers/gpu/drm/xe/xe_bo.c:2151:
+ if (bo->flags & XE_BO_FLAG_SYSTEM) {
[...]
+ } else {
[...]

-:77: WARNING:ENOSYS: ENOSYS means 'invalid syscall nr' and nothing else #77: FILE: drivers/gpu/drm/xe/xe_bo.c:2152:
+ ret = -ENOSYS; // TODO

-:110: WARNING:LONG_LINE_COMMENT: line length of 105 exceeds 100 columns #110: FILE: drivers/gpu/drm/xe/xe_bo.c:2793:
+ /* Similarly, force fail at creation time for now. We may relax this requirement later */

-:111: WARNING:LONG_LINE: line length of 104 exceeds 100 columns #111: FILE: drivers/gpu/drm/xe/xe_bo.c:2794:
+ if (XE_IOCTL_DBG(xe, pinned_flag && args->flags & DRM_XE_GEM_CREATE_FLAG_DEFER_BACKING))

-:115: WARNING:LONG_LINE: line length of 122 exceeds 100 columns #115: FILE: drivers/gpu/drm/xe/xe_bo.c:2798:
+ if (XE_IOCTL_DBG(xe, pinned_flag && !IS_ENABLED(CONFIG_CGROUP_DMEM) && bo_flags & XE_BO_FLAG_VRAM_MASK) ||

-:116: WARNING:LONG_LINE: line length of 111 exceeds 100 columns #116: FILE: drivers/gpu/drm/xe/xe_bo.c:2799:
+ XE_IOCTL_DBG(xe, pinned_flag && !IS_ENABLED(CONFIG_MEMCG) && bo_flags & XE_BO_FLAG_SYSTEM))

-:182: WARNING:TYPO_SPELLING: 'withing' may be misspelled - perhaps 'within'? #182: FILE: include/uapi/drm/xe_drm.h:776:
+ * on allocation, if withing cgroups limits.

                         ^^^^^^^

total: 0 errors, 7 warnings, 0 checks, 164 lines checked

#8 lore raw

✗ CI.KUnit: failure for cgroups: Add support for pinned device memory

In reply to Maarten Lankhorst
== Series Details ==

== Series Details ==

Series: cgroups: Add support for pinned device memory
URL : https://patchwork.freedesktop.org/series/153134/
State : failure

== Summary ==

+ trap cleanup EXIT
+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/xe/.kunitconfig
ERROR:root:../drivers/gpu/drm/xe/xe_bo.c: In function ‘xe_bo_unpin_user’:
../drivers/gpu/drm/xe/xe_bo.c:1709:17: error: implicit declaration of function ‘dmem_cgroup_unpin’; did you mean ‘mem_cgroup_init’? [-Werror=implicit-function-declaration]

 1709 |                 dmem_cgroup_unpin(bo->ttm.resource->css, xe_bo_size(bo));
      |                 ^~~~~~~~~~~~~~~~~
      |                 mem_cgroup_init
../drivers/gpu/drm/xe/xe_bo.c: In function ‘xe_bo_create_user’:
../drivers/gpu/drm/xe/xe_bo.c:2154:39: error: implicit declaration of function ‘dmem_cgroup_try_pin’; did you mean ‘mem_cgroup_tryget’? [-Werror=implicit-function-declaration]
 2154 |                                 ret = dmem_cgroup_try_pin(bo->ttm.resource->css, size);
      |                                       ^~~~~~~~~~~~~~~~~~~
      |                                       mem_cgroup_tryget

cc1: some warnings being treated as errors
make[7]: *** [../scripts/Makefile.build:287: drivers/gpu/drm/xe/xe_bo.o] Error 1 make[7]: *** Waiting for unfinished jobs....
make[6]: *** [../scripts/Makefile.build:556: drivers/gpu/drm/xe] Error 2 make[6]: *** Waiting for unfinished jobs....
make[5]: *** [../scripts/Makefile.build:556: drivers/gpu/drm] Error 2 make[4]: *** [../scripts/Makefile.build:556: drivers/gpu] Error 2 make[4]: *** Waiting for unfinished jobs....
make[3]: *** [../scripts/Makefile.build:556: drivers] Error 2 make[3]: *** Waiting for unfinished jobs....
make[2]: *** [/kernel/Makefile:2011: .] Error 2
make[1]: *** [/kernel/Makefile:248: __sub-make] Error 2
make: *** [Makefile:248: __sub-make] Error 2

[16:35:23] Configuring KUnit Kernel ...
Generating .config ...
Populating config with:

$ make ARCH=um O=.kunit olddefconfig
[16:35:27] Building KUnit Kernel ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
Building with:
$ make all compile_commands.json scripts_gdb ARCH=um O=.kunit --jobs=48
+ cleanup
++ stat -c %u:%g /kernel
+ chown -R 1003:1003 /kernel
maintainer
#9 lore raw
In reply to Maarten Lankhorst

Hello Maarten.

Quoted 5 lines · Maarten Lankhorst

Implementation details:

For each cgroup up until the root cgroup, the 'min' limit is checked against currently effectively pinned value. If the value will go above 'min', the pinning attempt is rejected.

How is pinning different from setting a 'min' limit (from a user perspective)?

Quoted 8 lines

Pinned memory is handled slightly different and affects calculating effective min/low values. Pinned memory is subtracted from both, and needs to be added afterwards when calculating.

This is because increasing the amount of pinned memory, the amount of free min/low memory decreases for all cgroups that are part of the hierarchy.

What is supposed to happen with pinned memory after cgroup removal? I find the page_counter changes little bit complex without understanding of the difference between min and pinned. Should this be conceptually similar to memory.stat:unevictable? Or rather mlock(2)? So far neither of those needed interaction with min/low values (in memcg).

Thanks,
Michal

author maintainer
#10 lore raw

Hey,

Den 2025-08-26 kl. 16:20, skrev Michal Koutný:

Quoted 11 lines

Hello Maarten.

Quoted 5 lines · Maarten Lankhorst

Implementation details:

For each cgroup up until the root cgroup, the 'min' limit is checked against currently effectively pinned value. If the value will go above 'min', the pinning attempt is rejected.

How is pinning different from setting a 'min' limit (from a user perspective)?

It's related, in fact you have to set the 'min' limit first.

The 'pinned' allows you to pick /which/ memory falls under the 'min' limit.

Quoted 10 lines
Quoted 8 lines

Pinned memory is handled slightly different and affects calculating effective min/low values. Pinned memory is subtracted from both, and needs to be added afterwards when calculating.

This is because increasing the amount of pinned memory, the amount of free min/low memory decreases for all cgroups that are part of the hierarchy.

What is supposed to happen with pinned memory after cgroup removal?

I think for accounting purposes pinned memory stays pinned, otherwise the idea of pinning is lost. However when you kill all processes in the cgroup, that should solve itself eventually.

Quoted 4 lines

I find the page_counter changes little bit complex without understanding of the difference between min and pinned. Should this be conceptually similar to memory.stat:unevictable? Or rather mlock(2)? So far neither of those needed interaction with min/low values (in memcg).

You could in theory implement mlockall using the 'min' values too.

The page counter changes implement the following:

Lets say you have this tree with 'min' values.

      / '5' A
X'6' -- '5' B
      \ '5' C

Effective min without pinned pages:

      / '2' A
X'6' -- '2' B
      \ '2' C

Now 'B' pins 3 pages:

Effective min:

         / '1' A
X'3+3p' -- '1' B (1 + 3 pinned pages makes effective min 4)
         \ '1' C

Same for applies to effective 'low' calculations.

Kind regards,
~Maarten

#11 lore raw
In reply to Maarten Lankhorst
Quoted 21 lines · Maarten Lankhorst

When exporting dma-bufs to other devices, even when it is allowed to use move_notify in some drivers, performance will degrade severely when eviction happens.

A perticular example where this can happen is in a multi-card setup, where PCI-E peer-to-peer is used to prevent using access to system memory.

If the buffer is evicted to system memory, not only the evicting GPU wher the buffer resided is affected, but it will also stall the GPU that is waiting on the buffer.

It also makes sense for long running jobs not to be preempted by having its buffers evicted, so it will make sense to have the ability to pin from system memory too.

This is dependant on patches by Dave Airlie, so it's not part of this series yet. But I'm planning on extending pinning to the memory cgroup controller in the future to handle this case.

Implementation details:

Quoted 7 lines

For each cgroup up until the root cgroup, the 'min' limit is checked against currently effectively pinned value. If the value will go above 'min', the pinning attempt is rejected.

Pinned memory is handled slightly different and affects calculating effective min/low values. Pinned memory is subtracted from both, and needs to be added afterwards when calculating.

The term "pinning" is overloaded, and frequently we refer to pin_user_pages() and friends.

So I'm wondering if there is an alternative term to describe what you want to achieve.

Is it something like "unevictable" ?

Signature · 3 lines
Cheers

David / dhildenb
author maintainer
#12 lore raw

Hello David,

Den 2025-09-01 kl. 14:25, skrev David Hildenbrand:

Quoted 30 lines · Maarten Lankhorst
Quoted 21 lines

When exporting dma-bufs to other devices, even when it is allowed to use move_notify in some drivers, performance will degrade severely when eviction happens.

A perticular example where this can happen is in a multi-card setup, where PCI-E peer-to-peer is used to prevent using access to system memory.

If the buffer is evicted to system memory, not only the evicting GPU wher the buffer resided is affected, but it will also stall the GPU that is waiting on the buffer.

It also makes sense for long running jobs not to be preempted by having its buffers evicted, so it will make sense to have the ability to pin from system memory too.

This is dependant on patches by Dave Airlie, so it's not part of this series yet. But I'm planning on extending pinning to the memory cgroup controller in the future to handle this case.

Implementation details:

Quoted 7 lines

For each cgroup up until the root cgroup, the 'min' limit is checked against currently effectively pinned value. If the value will go above 'min', the pinning attempt is rejected.

Pinned memory is handled slightly different and affects calculating effective min/low values. Pinned memory is subtracted from both, and needs to be added afterwards when calculating.

Quoted 5 lines

The term "pinning" is overloaded, and frequently we refer to pin_user_pages() and friends.

So I'm wondering if there is an alternative term to describe what you want to achieve.

Is it something like "unevictable" ?

It could be required to include a call pin_user_pages(), in case a process wants to pin from a user's address space to the gpu.

It's not done yet, but it wouldn't surprise me if we want to include it in the future. Functionally it's similar to mlock() and related functions.

Perhaps call it mlocked instead?

Kind regards,
~Maarten Lankhorst

#13 lore raw

Hi,

Quoted 57 lines · Maarten Lankhorst

Hello David,

Den 2025-09-01 kl. 14:25, skrev David Hildenbrand:

Quoted 43 lines · Maarten Lankhorst
Quoted 37 lines

When exporting dma-bufs to other devices, even when it is allowed to use
move_notify in some drivers, performance will degrade severely when
eviction happens.

A perticular example where this can happen is in a multi-card setup,
where PCI-E peer-to-peer is used to prevent using access to system memory.

If the buffer is evicted to system memory, not only the evicting GPU wher
the buffer resided is affected, but it will also stall the GPU that is
waiting on the buffer.

It also makes sense for long running jobs not to be preempted by having
its buffers evicted, so it will make sense to have the ability to pin
from system memory too.

This is dependant on patches by Dave Airlie, so it's not part of this
series yet. But I'm planning on extending pinning to the memory cgroup
controller in the future to handle this case.

Implementation details:

For each cgroup up until the root cgroup, the 'min' limit is checked
against currently effectively pinned value. If the value will go above
'min', the pinning attempt is rejected.

Quoted 4 lines

Pinned memory is handled slightly different and affects calculating
effective min/low values. Pinned memory is subtracted from both, and needs to be added afterwards when calculating.

Quoted 7 lines

The term "pinning" is overloaded, and frequently we refer to pin_user_pages() and friends.

So I'm wondering if there is an alternative term to describe what you want to achieve.

Is it something like "unevictable" ?

It could be required to include a call pin_user_pages(), in case a process wants to pin
from a user's address space to the gpu.

Quoted 5 lines

It's not done yet, but it wouldn't surprise me if we want to include it in the future.
Functionally it's similar to mlock() and related functions.

Perhaps call it mlocked instead?

I was under the impression that mlocked() memory can be migrated to other physical memory but not to swap? whereas pinned memory needs to remain the exact same physical memory.

IMO "pinned" is pretty established within GPU drivers (dma-buf, TTM) and essentially means the same as "pin" in "pin_user_pages", so inventing a new name would probably cause even more confusion?

Thanks,
Thomas

Quoted 3 lines

Kind regards,
~Maarten Lankhorst

#14 lore raw
Quoted 57 lines · Thomas Hellström

Hi,

Quoted 54 lines · Maarten Lankhorst

Hello David,

Den 2025-09-01 kl. 14:25, skrev David Hildenbrand:

Quoted 43 lines · Maarten Lankhorst
Quoted 37 lines

When exporting dma-bufs to other devices, even when it is allowed to use
move_notify in some drivers, performance will degrade severely when
eviction happens.

A perticular example where this can happen is in a multi-card setup,
where PCI-E peer-to-peer is used to prevent using access to system memory.

If the buffer is evicted to system memory, not only the evicting GPU wher
the buffer resided is affected, but it will also stall the GPU that is
waiting on the buffer.

It also makes sense for long running jobs not to be preempted by having
its buffers evicted, so it will make sense to have the ability to pin
from system memory too.

This is dependant on patches by Dave Airlie, so it's not part of this
series yet. But I'm planning on extending pinning to the memory cgroup
controller in the future to handle this case.

Implementation details:

For each cgroup up until the root cgroup, the 'min' limit is checked
against currently effectively pinned value. If the value will go above
'min', the pinning attempt is rejected.

Quoted 4 lines

Pinned memory is handled slightly different and affects calculating
effective min/low values. Pinned memory is subtracted from both, and needs to be added afterwards when calculating.

Quoted 7 lines

The term "pinning" is overloaded, and frequently we refer to pin_user_pages() and friends.

So I'm wondering if there is an alternative term to describe what you want to achieve.

Is it something like "unevictable" ?

It could be required to include a call pin_user_pages(), in case a

We'll only care about long-term pinnings (i.e., FOLL_LONGTERM). Ordinary short-term pinning is just fine.

(see how even "pinning" is overloaded? :) )

Quoted 6 lines
Quoted 6 lines

process wants to pin
from a user's address space to the gpu.

It's not done yet, but it wouldn't surprise me if we want to include it in the future.
Functionally it's similar to mlock() and related functions.

Traditionally, vfio, io_uring and rdma do exactly that: they use GUP to longterm pin and then account that memory towards RLIMIT_MEMLOCK.

If you grep for "rlimit(RLIMIT_MEMLOCK)", you'll see what I mean.

There are known issues with that: imagine long-term pinning the same folio through GUP with 2 interfaces (e.g., vfio, io_uring, rdma), or within the same interface.

You'd account the memory multiple times, which is horrible. And so far there is no easy way out.

Quoted 6 lines
Quoted 2 lines

Perhaps call it mlocked instead?

I was under the impression that mlocked() memory can be migrated to other physical memory but not to swap? whereas pinned memory needs to remain the exact same physical memory.

Yes, exactly.

Quoted 4 lines

IMO "pinned" is pretty established within GPU drivers (dma-buf, TTM) and essentially means the same as "pin" in "pin_user_pages", so inventing a new name would probably cause even more confusion?

If it's the same thing, absolutely. But Marteen said "It's not done yet, but it wouldn't surprise me if we want to include it in the future".

So how is the memory we are talking about in this series "pinned" ?

Signature · 3 lines
Cheers

David / dhildenb
#15 lore raw
Quoted 115 lines · David Hildenbrand
Quoted 69 lines · Thomas Hellström

Hi,

Quoted 66 lines · Maarten Lankhorst

Hello David,

Den 2025-09-01 kl. 14:25, skrev David Hildenbrand:

Quoted 53 lines · Maarten Lankhorst
Quoted 46 lines

When exporting dma-bufs to other devices, even when it is allowed
to use
move_notify in some drivers, performance will degrade severely
when
eviction happens.

A perticular example where this can happen is in a multi-card setup,
where PCI-E peer-to-peer is used to prevent using access to system memory.

If the buffer is evicted to system memory, not only the evicting
GPU wher
the buffer resided is affected, but it will also stall the GPU
that is
waiting on the buffer.

It also makes sense for long running jobs not to be preempted by
having
its buffers evicted, so it will make sense to have the ability to
pin
from system memory too.

This is dependant on patches by Dave Airlie, so it's not part of
this
series yet. But I'm planning on extending pinning to the memory
cgroup
controller in the future to handle this case.

Implementation details:

For each cgroup up until the root cgroup, the 'min' limit is checked
against currently effectively pinned value. If the value will go
above
'min', the pinning attempt is rejected.

Quoted 5 lines

Pinned memory is handled slightly different and affects calculating
effective min/low values. Pinned memory is subtracted from both,
and needs to be added afterwards when calculating.

Quoted 8 lines

The term "pinning" is overloaded, and frequently we refer to pin_user_pages() and friends.

So I'm wondering if there is an alternative term to describe what
you want to achieve.

Is it something like "unevictable" ?

It could be required to include a call pin_user_pages(), in case a

We'll only care about long-term pinnings (i.e., FOLL_LONGTERM). Ordinary
short-term pinning is just fine.

(see how even "pinning" is overloaded? :) )

Quoted 7 lines
Quoted 7 lines

process wants to pin
from a user's address space to the gpu.

It's not done yet, but it wouldn't surprise me if we want to include
it in the future.
Functionally it's similar to mlock() and related functions.

Traditionally, vfio, io_uring and rdma do exactly that: they use GUP to
longterm pin and then account that memory towards RLIMIT_MEMLOCK.

If you grep for "rlimit(RLIMIT_MEMLOCK)", you'll see what I mean.

There are known issues with that: imagine long-term pinning the same folio through GUP with 2 interfaces (e.g., vfio, io_uring, rdma), or within the same interface.

You'd account the memory multiple times, which is horrible. And so far
there is no easy way out.

Quoted 7 lines
Quoted 2 lines

Perhaps call it mlocked instead?

I was under the impression that mlocked() memory can be migrated to other physical memory but not to swap? whereas pinned memory needs to
remain the exact same physical memory.

Yes, exactly.

Quoted 5 lines

IMO "pinned" is pretty established within GPU drivers (dma-buf, TTM)
and essentially means the same as "pin" in "pin_user_pages", so inventing a new name would probably cause even more confusion?

Quoted 5 lines

If it's the same thing, absolutely. But Marteen said "It's not done yet,
but it wouldn't surprise me if we want to include it in the future".

So how is the memory we are talking about in this series "pinned" ?

Reading the cover-letter from Maarten, he only talks about pinning affecting performance, which would be similar to user-space calling mlock(), although I doubt that moving content to other physical pages within the same memory type will be a near-term use-case.

However what's more important are situation where a device (like RDMA) needs to pin, because it can't handle the case where access is interrupted and content transferred to another physical location.

Perhaps Maarten could you elaborate whether this series is intended for both these use-cases?

/Thomas

Quoted 2 lines
author maintainer
#16 lore raw

Hey,

Den 2025-09-02 kl. 15:42, skrev Thomas Hellström:

Quoted 127 lines · David Hildenbrand
Quoted 115 lines · Thomas Hellström
Quoted 69 lines

Hi,

Quoted 66 lines · Maarten Lankhorst

Hello David,

Den 2025-09-01 kl. 14:25, skrev David Hildenbrand:

Quoted 53 lines · Maarten Lankhorst
Quoted 46 lines

When exporting dma-bufs to other devices, even when it is allowed
to use
move_notify in some drivers, performance will degrade severely
when
eviction happens.

A perticular example where this can happen is in a multi-card setup,
where PCI-E peer-to-peer is used to prevent using access to system memory.

If the buffer is evicted to system memory, not only the evicting
GPU wher
the buffer resided is affected, but it will also stall the GPU
that is
waiting on the buffer.

It also makes sense for long running jobs not to be preempted by
having
its buffers evicted, so it will make sense to have the ability to
pin
from system memory too.

This is dependant on patches by Dave Airlie, so it's not part of
this
series yet. But I'm planning on extending pinning to the memory
cgroup
controller in the future to handle this case.

Implementation details:

For each cgroup up until the root cgroup, the 'min' limit is checked
against currently effectively pinned value. If the value will go
above
'min', the pinning attempt is rejected.

Quoted 5 lines

Pinned memory is handled slightly different and affects calculating
effective min/low values. Pinned memory is subtracted from both,
and needs to be added afterwards when calculating.

Quoted 8 lines

The term "pinning" is overloaded, and frequently we refer to pin_user_pages() and friends.

So I'm wondering if there is an alternative term to describe what
you want to achieve.

Is it something like "unevictable" ?

It could be required to include a call pin_user_pages(), in case a

We'll only care about long-term pinnings (i.e., FOLL_LONGTERM). Ordinary
short-term pinning is just fine.

(see how even "pinning" is overloaded? :) )

Quoted 7 lines
Quoted 7 lines

process wants to pin
from a user's address space to the gpu.

It's not done yet, but it wouldn't surprise me if we want to include
it in the future.
Functionally it's similar to mlock() and related functions.

Traditionally, vfio, io_uring and rdma do exactly that: they use GUP to
longterm pin and then account that memory towards RLIMIT_MEMLOCK.

If you grep for "rlimit(RLIMIT_MEMLOCK)", you'll see what I mean.

There are known issues with that: imagine long-term pinning the same folio through GUP with 2 interfaces (e.g., vfio, io_uring, rdma), or within the same interface.

You'd account the memory multiple times, which is horrible. And so far
there is no easy way out.

Quoted 7 lines
Quoted 2 lines

Perhaps call it mlocked instead?

I was under the impression that mlocked() memory can be migrated to other physical memory but not to swap? whereas pinned memory needs to
remain the exact same physical memory.

Yes, exactly.

Quoted 5 lines

IMO "pinned" is pretty established within GPU drivers (dma-buf, TTM)
and essentially means the same as "pin" in "pin_user_pages", so inventing a new name would probably cause even more confusion?

Quoted 5 lines

If it's the same thing, absolutely. But Marteen said "It's not done yet,
but it wouldn't surprise me if we want to include it in the future".

So how is the memory we are talking about in this series "pinned" ?

Reading the cover-letter from Maarten, he only talks about pinning affecting performance, which would be similar to user-space calling mlock(), although I doubt that moving content to other physical pages within the same memory type will be a near-term use-case.

Quoted 6 lines

However what's more important are situation where a device (like RDMA) needs to pin, because it can't handle the case where access is interrupted and content transferred to another physical location.

Perhaps Maarten could you elaborate whether this series is intended for both these use-cases?

Yeah, this is definitely for the latter case too.

It's a performance optimization for the generic case, and very nice to have for the second case, to prevent unlimited vram pinning. With cgroups, we would be able to limit the amounts of used memory there.

Kind regards,
~Maarten

#17 lore raw
In reply to Maarten Lankhorst

Hi,

Quoted 19 lines · Maarten Lankhorst

When exporting dma-bufs to other devices, even when it is allowed to use move_notify in some drivers, performance will degrade severely when eviction happens.

A perticular example where this can happen is in a multi-card setup, where PCI-E peer-to-peer is used to prevent using access to system memory.

If the buffer is evicted to system memory, not only the evicting GPU wher the buffer resided is affected, but it will also stall the GPU that is waiting on the buffer.

It also makes sense for long running jobs not to be preempted by having its buffers evicted, so it will make sense to have the ability to pin from system memory too.

This is dependant on patches by Dave Airlie, so it's not part of this series yet. But I'm planning on extending pinning to the memory cgroup controller in the future to handle this case.

Quoted 5 lines

Implementation details:

For each cgroup up until the root cgroup, the 'min' limit is checked against currently effectively pinned value. If the value will go above 'min', the pinning attempt is rejected.

Why do you want to reject pins in this case? What happens in desktop usecases (e.g. PRIME buffer sharing)? AFAIU, you kind of need to be able to pin buffers and export them to other devices for that whole thing to work, right? If the user doesn't explicitly set a min value, wouldn't the value being zero mean any pins will be rejected (and thus PRIME would break)?

If your objective is to prevent pinned buffers from being evicted, perhaps you could instead make TTM try to avoid evicting pinned buffers and prefer unpinned buffers as long as there are unpinned buffers to evict? As long as the total amount of pinned memory stays below min, no pinned buffers should get evicted with that either.

Best,
Natalie

Quoted 23 lines

Pinned memory is handled slightly different and affects calculating effective min/low values. Pinned memory is subtracted from both, and needs to be added afterwards when calculating.

This is because increasing the amount of pinned memory, the amount of free min/low memory decreases for all cgroups that are part of the hierarchy.

Maarten Lankhorst (3):
page_counter: Allow for pinning some amount of memory cgroup/dmem: Implement pinning device memory
drm/xe: Add DRM_XE_GEM_CREATE_FLAG_PINNED flag and implementation

  drivers/gpu/drm/xe/xe_bo.c      | 66 +++++++++++++++++++++-
  drivers/gpu/drm/xe/xe_dma_buf.c | 10 +++-
  include/linux/cgroup_dmem.h     |  2 +
  include/linux/page_counter.h    |  8 +++
  include/uapi/drm/xe_drm.h       | 10 +++-
  kernel/cgroup/dmem.c            | 57 ++++++++++++++++++-
  mm/page_counter.c               | 98 ++++++++++++++++++++++++++++++---
  7 files changed, 237 insertions(+), 14 deletions(-)
#18 lore raw

Hi,

Quoted 39 lines · Natalie Vock

Hi,

Quoted 28 lines · Maarten Lankhorst

When exporting dma-bufs to other devices, even when it is allowed to use
move_notify in some drivers, performance will degrade severely when eviction happens.

A perticular example where this can happen is in a multi-card setup,
where PCI-E peer-to-peer is used to prevent using access to system memory.

If the buffer is evicted to system memory, not only the evicting GPU wher
the buffer resided is affected, but it will also stall the GPU that is
waiting on the buffer.

It also makes sense for long running jobs not to be preempted by having
its buffers evicted, so it will make sense to have the ability to pin
from system memory too.

This is dependant on patches by Dave Airlie, so it's not part of this
series yet. But I'm planning on extending pinning to the memory cgroup
controller in the future to handle this case.

Quoted 7 lines

Implementation details:

For each cgroup up until the root cgroup, the 'min' limit is checked
against currently effectively pinned value. If the value will go above
'min', the pinning attempt is rejected.

Quoted 8 lines

Why do you want to reject pins in this case? What happens in desktop usecases (e.g. PRIME buffer sharing)? AFAIU, you kind of need to be able
to pin buffers and export them to other devices for that whole thing to
work, right? If the user doesn't explicitly set a min value, wouldn't the value being zero mean any pins will be rejected (and thus PRIME would break)?

That's really the point. If an unprivileged malicious process is allowed to pin arbitrary amounts of memory, thats a DOS vector.

However drivers that allow unlimited pinning today need to take care when implementing restrictions to avoid regressions. Like perhaps adding this behind a config option.

That said, IMO dma-buf clients should implement move_notify() whenever possible to provide an option to avoid pinning unless necessary.

/Thomas

Quoted 41 lines

If your objective is to prevent pinned buffers from being evicted, perhaps you could instead make TTM try to avoid evicting pinned buffers
and prefer unpinned buffers as long as there are unpinned buffers to evict? As long as the total amount of pinned memory stays below min, no
pinned buffers should get evicted with that either.

Best,
Natalie

Quoted 26 lines

Pinned memory is handled slightly different and affects calculating effective min/low values. Pinned memory is subtracted from both, and needs to be added afterwards when calculating.

This is because increasing the amount of pinned memory, the amount of
free min/low memory decreases for all cgroups that are part of the hierarchy.

Maarten Lankhorst (3):
   page_counter: Allow for pinning some amount of memory cgroup/dmem: Implement pinning device memory drm/xe: Add DRM_XE_GEM_CREATE_FLAG_PINNED flag and implementation

  drivers/gpu/drm/xe/xe_bo.c      | 66 +++++++++++++++++++++- drivers/gpu/drm/xe/xe_dma_buf.c | 10 +++- include/linux/cgroup_dmem.h     |  2 + include/linux/page_counter.h    |  8 +++ include/uapi/drm/xe_drm.h       | 10 +++- kernel/cgroup/dmem.c            | 57 ++++++++++++++++++- mm/page_counter.c               | 98
++++++++++++++++++++++++++++++---

  7 files changed, 237 insertions(+), 14 deletions(-)
author maintainer
#19 lore raw
In reply to Natalie Vock

Hello Nathalie,

Den 2025-09-01 kl. 14:45, skrev Natalie Vock:

Quoted 24 lines

Hi,

Quoted 21 lines · Maarten Lankhorst

When exporting dma-bufs to other devices, even when it is allowed to use move_notify in some drivers, performance will degrade severely when eviction happens.

A perticular example where this can happen is in a multi-card setup, where PCI-E peer-to-peer is used to prevent using access to system memory.

If the buffer is evicted to system memory, not only the evicting GPU wher the buffer resided is affected, but it will also stall the GPU that is waiting on the buffer.

It also makes sense for long running jobs not to be preempted by having its buffers evicted, so it will make sense to have the ability to pin from system memory too.

This is dependant on patches by Dave Airlie, so it's not part of this series yet. But I'm planning on extending pinning to the memory cgroup controller in the future to handle this case.

Implementation details:

Quoted 7 lines
Quoted 3 lines

For each cgroup up until the root cgroup, the 'min' limit is checked against currently effectively pinned value. If the value will go above 'min', the pinning attempt is rejected.

Why do you want to reject pins in this case? What happens in desktop usecases (e.g. PRIME buffer sharing)? AFAIU, you kind of need to be able to pin buffers and export them to other devices for that whole thing to work, right? If the user doesn't explicitly set a min value, wouldn't the value being zero mean any pins will be rejected (and thus PRIME would break)?

If your objective is to prevent pinned buffers from being evicted, perhaps you could instead make TTM try to avoid evicting pinned buffers and prefer unpinned buffers as long as there are unpinned buffers to evict? As long as the total amount of pinned memory stays below min, no pinned buffers should get evicted with that either.

That would be setting an eviction priority, that can be done but that gives no guarantee memory will not be evicted.

Kind regards,
~Maarten