Skip to contentSkip to conversation

drm/amdkfd: register HMM dev memory to DMA-able range first

PATCH In review francisco_flynn 2 participants 3 messages Jun 9, 2025 – Jun 10, 2025

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/amdkfd: register HMM dev memory to DMA-able range first

HMM device memory is allocated at the top of iomem_resource, when iomem_resource is larger than GPU device's dma mask, after devm_memremap_pages, max_pfn will also be update and exceed device's dma mask, when there are multiple card on system need to be init, ttm_device_init would be called with use_dma32=true, and this is not necessary at all. let's request dev memory region at DMA-able range first.

---
 drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdkfd/kfd_migrate.c +5 -1
index 79251f22b702..3856b9fd2a70 100644
@@ -1020,6 +1020,7 @@ int kgd2kfd_init_zone_device(struct amdgpu_device *adev)
 	struct amdgpu_kfd_dev *kfddev = &adev->kfd;
 	struct dev_pagemap *pgmap;
 	struct resource *res = NULL;
+	struct resource temp_res = iomem_resource;
 	unsigned long size;
 	void *r;
 
@@ -1042,7 +1043,10 @@ int kgd2kfd_init_zone_device(struct amdgpu_device *adev)
 		pgmap->range.end = adev->gmc.aper_base + adev->gmc.aper_size - 1;
 		pgmap->type = MEMORY_DEVICE_COHERENT;
 	} else {
-		res = devm_request_free_mem_region(adev->dev, &iomem_resource, size);
+		temp_res.end = dma_get_mask(adev->dev);
+		res = devm_request_free_mem_region(adev->dev, &temp_res, size);
+		if (IS_ERR(res))
+			res = devm_request_free_mem_region(adev->dev, &iomem_resource, size);
 		if (IS_ERR(res))
 			return PTR_ERR(res);
 		pgmap->range.start = res->start;
Signature · 1 line
2.25.1
maintainer
#1 lore raw
Quoted 9 lines · francisco_flynn

HMM device memory is allocated at the top of iomem_resource, when iomem_resource is larger than GPU device's dma mask, after devm_memremap_pages, max_pfn will also be update and exceed device's dma mask, when there are multiple card on system need to be init, ttm_device_init would be called with use_dma32=true, and this is not necessary at all. let's request dev memory region at DMA-able range first.

That doesn't make sense to me. The addresses allocated here are not DMA addresses. They cannot be accessed by the GPU via DMA. They are purely fictional addresses for the purposes of creating struct pages for device-private memory. There should be no need to limit them by the GPU's DMA mask.

Regards,
  Felix

Quoted 30 lines
---
 drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdkfd/kfd_migrate.c +5 -1
index 79251f22b702..3856b9fd2a70 100644
@@ -1020,6 +1020,7 @@ int kgd2kfd_init_zone_device(struct amdgpu_device *adev)
 	struct amdgpu_kfd_dev *kfddev = &adev->kfd;
 	struct dev_pagemap *pgmap;
 	struct resource *res = NULL;
+	struct resource temp_res = iomem_resource;
 	unsigned long size;
 	void *r;
 
@@ -1042,7 +1043,10 @@ int kgd2kfd_init_zone_device(struct amdgpu_device *adev)
 		pgmap->range.end = adev->gmc.aper_base + adev->gmc.aper_size - 1;
 		pgmap->type = MEMORY_DEVICE_COHERENT;
 	} else {
-		res = devm_request_free_mem_region(adev->dev, &iomem_resource, size);
+		temp_res.end = dma_get_mask(adev->dev);
+		res = devm_request_free_mem_region(adev->dev, &temp_res, size);
+		if (IS_ERR(res))
+			res = devm_request_free_mem_region(adev->dev, &iomem_resource, size);
 		if (IS_ERR(res))
 			return PTR_ERR(res);
 		pgmap->range.start = res->start;
#2 lore raw
Quoted 11 lines · Felix Kuehling
Quoted 9 lines · francisco_flynn

HMM device memory is allocated at the top of iomem_resource, when iomem_resource is larger than GPU device's dma mask, after devm_memremap_pages, max_pfn will also be update and exceed device's dma mask, when there are multiple card on system need to be init, ttm_device_init would be called with use_dma32=true, and this is not necessary at all. let's request dev memory region at DMA-able range first.

That doesn't make sense to me. The addresses allocated here are not DMA addresses. They cannot be accessed by the GPU via DMA. They are purely fictional addresses for the purposes of creating struct pages for device-private memory. There should be no need to limit them by the GPU's DMA mask.

yes, this address is used by CPU to access VRAM. The patch is intended to cope with a special case, after checking the latest kernel code, i found this problem has been solved by this commit,

https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/commit/?id=7170130e4c72ce0caa0cb42a1627c635cc262821

thanks for you reply.

Best regards,
flynn

Quoted 33 lines

Regards,
  Felix

Quoted 29 lines
---
  drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 6 +++++-
  1 file changed, 5 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdkfd/kfd_migrate.c +5 -1
index 79251f22b702..3856b9fd2a70 100644
@@ -1020,6 +1020,7 @@ int kgd2kfd_init_zone_device(struct amdgpu_device *adev)
  	struct amdgpu_kfd_dev *kfddev = &adev->kfd;
  	struct dev_pagemap *pgmap;
  	struct resource *res = NULL;
+	struct resource temp_res = iomem_resource;
  	unsigned long size;
  	void *r;
 
@@ -1042,7 +1043,10 @@ int kgd2kfd_init_zone_device(struct amdgpu_device *adev)
  		pgmap->range.end = adev->gmc.aper_base + adev->gmc.aper_size - 1;
  		pgmap->type = MEMORY_DEVICE_COHERENT;
  	} else {
-		res = devm_request_free_mem_region(adev->dev, &iomem_resource, size);
+		temp_res.end = dma_get_mask(adev->dev);
+		res = devm_request_free_mem_region(adev->dev, &temp_res, size);
+		if (IS_ERR(res))
+			res = devm_request_free_mem_region(adev->dev, &iomem_resource, size);
  		if (IS_ERR(res))
  			return PTR_ERR(res);
  		pgmap->range.start = res->start;