Skip to contentSkip to conversation

drm/amdgpu: always allocate cleared VRAM for GEM allocations

PATCH In review Alex Deucher 4 participants 5 messages Jul 26, 2024 – Jul 29, 2024

Subsystems (2)
lore
Share & tools
lore

Alex Deucher

author maintainer
#0 lore raw

PATCH 1/1 drm/amdgpu: always allocate cleared VRAM for GEM allocations

This adds allocation latency, but aligns better with user expectations. The latency should improve with the drm buddy clearing patches that Arun has been working on.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +4 -0
index ebb3f87ef4f6..768840e8937d 100644
@@ -347,6 +347,10 @@ int amdgpu_gem_create_ioctl(struct drm_device *dev, void *data,
 		return -EINVAL;
 	}
 
+	/* always clear VRAM */
+	if (args->in.domains & AMDGPU_GEM_DOMAIN_VRAM)
+		flags |= AMDGPU_GEM_CREATE_VRAM_CLEARED;
+
 	/* create a gem object to contain this object in */
 	if (args->in.domains & (AMDGPU_GEM_DOMAIN_GDS |
 	    AMDGPU_GEM_DOMAIN_GWS | AMDGPU_GEM_DOMAIN_OA)) {
Signature · 1 line
2.45.2
#1 lore raw
Quoted 15 lines · Alex Deucher

This adds allocation latency, but aligns better with user expectations. The latency should improve with the drm buddy clearing patches that Arun has been working on.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +0 -0
index ebb3f87ef4f6..768840e8937d 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c int amdgpu_gem_create_ioctl(struct drm_device *dev, void *data, · line 352

Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +3 -0
@@ -347,6 +347,10 @@ int amdgpu_gem_create_ioctl(struct drm_device *dev, void *data,
                return -EINVAL;
        }
 
+       /* always clear VRAM */
+       if (args->in.domains & AMDGPU_GEM_DOMAIN_VRAM)
+               flags |= AMDGPU_GEM_CREATE_VRAM_CLEARED;

Alternatively, we can set AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE here, although for cases like reboots, we probably want clear on create.

Alex

Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +1 -1
+
        /* create a gem object to contain this object in */
        if (args->in.domains & (AMDGPU_GEM_DOMAIN_GDS |
            AMDGPU_GEM_DOMAIN_GWS | AMDGPU_GEM_DOMAIN_OA)) {
--

2.45.2

Reviewed-by
#2 lore raw
Quoted 19 lines · Alex Deucher
Quoted 18 lines · Alex Deucher

This adds allocation latency, but aligns better with user expectations. The latency should improve with the drm buddy clearing patches that Arun has been working on.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +0 -0
index ebb3f87ef4f6..768840e8937d 100644
@@ -347,6 +347,10 @@ int amdgpu_gem_create_ioctl(struct drm_device *dev, void *data,
                return -EINVAL;
        }
 
Quoted 6 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c int amdgpu_gem_create_ioctl(struct drm_device *dev, void *data, · line 352

Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +3 -0
+       /* always clear VRAM */
+       if (args->in.domains & AMDGPU_GEM_DOMAIN_VRAM)
+               flags |= AMDGPU_GEM_CREATE_VRAM_CLEARED;

Alternatively, we can set AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE here, although for cases like reboots, we probably want clear on create.

I agree. Thanks for doing this!

BTW, maybe this allows reducing the use of AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE, or even eliminating it altogether? Does this leave any way for user space to access uncleared VRAM?

Signature · 2 lines
Earthling Michel Dänzer            |                  https://redhat.com
Libre software enthusiast          |         Mesa and Xwayland developer
#3 lore raw

Am 29.07.24 um 12:42 schrieb Michel Dänzer:

Quoted 19 lines · Alex Deucher
Quoted 18 lines · Alex Deucher
Quoted 17 lines

This adds allocation latency, but aligns better with user expectations. The latency should improve with the drm buddy clearing patches that Arun has been working on.

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 4 ++++
  1 file changed, 4 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +0 -0
index ebb3f87ef4f6..768840e8937d 100644
@@ -347,6 +347,10 @@ int amdgpu_gem_create_ioctl(struct drm_device *dev, void *data,
                 return -EINVAL;
         }
 
Quoted 6 lines
Quoted 5 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c int amdgpu_gem_create_ioctl(struct drm_device *dev, void *data, · line 352

Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +3 -0
+       /* always clear VRAM */
+       if (args->in.domains & AMDGPU_GEM_DOMAIN_VRAM)
+               flags |= AMDGPU_GEM_CREATE_VRAM_CLEARED;

Alternatively, we can set AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE here, although for cases like reboots, we probably want clear on create.

I agree. Thanks for doing this!

I strongly disagree. We have use cases were this will immediately become a problem.

Especially we won't be able to hold latency requirements on boot for certain projects any more.

Regards,
Christian.

Quoted 7 lines

BTW, maybe this allows reducing the use of AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE, or even eliminating it altogether? Does this leave any way for user space to access uncleared VRAM?

#4 lore raw
Quoted 25 lines · Christian König

Am 29.07.24 um 12:42 schrieb Michel Dänzer:

Quoted 24 lines · Alex Deucher
Quoted 23 lines · Alex Deucher
Quoted 14 lines

This adds allocation latency, but aligns better with user expectations.  The latency should improve with the drm buddy clearing patches that Arun has been working on.

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 4 ++++

  1 file changed, 4 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +0 -0
index ebb3f87ef4f6..768840e8937d 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c int amdgpu_gem_create_ioctl(struct drm_device *dev, void *data, · line 352

Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +3 -0
@@ -347,6 +347,10 @@ int amdgpu_gem_create_ioctl(struct drm_device *dev, void *data,
                  return -EINVAL;
          }
 
+       /* always clear VRAM */
+       if (args->in.domains & AMDGPU_GEM_DOMAIN_VRAM)
+               flags |= AMDGPU_GEM_CREATE_VRAM_CLEARED;

Alternatively, we can set AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE here, although for cases like reboots, we probably want clear on create.

Quoted 5 lines
Quoted 1 line

I agree. Thanks for doing this!

I strongly disagree. We have use cases were this will immediately become a problem.

Especially we won't be able to hold latency requirements on boot for certain projects any more.

Then it was only possible to achieve them by essentially cheating, leaking uncleared VRAM contents to user space.

If that's acceptable for those use cases, you could maybe add some kind of opt-in mechanism for them.

Signature · 2 lines
Earthling Michel Dänzer            |                  https://redhat.com
Libre software enthusiast          |         Mesa and Xwayland developer