drm/amd: Add support for non-4K page size systems
Subsystems (4)
v3 cover drm/amd: Add support for non-4K page size systems
This is v3 of the patch series enabling 64 KB system page size support in AMDGPU. v2, part 1 of this series [1] has already been merged upstream and provides the minimal infrastructure required for 64 KB page support.
This series addresses additional issues uncovered in AMDGPU when running rccl unit tests and rocr-debug-agent tessts on 64KB page-size systems.
With this series applied, all RCCL unit tests and rocr-debug-agent tests pass on systems using a 64 KB system page size, across multi-GPU configurations, with XNACK both enabled and disabled.
Patch 1 in this series (drm/amdgpu: Change AMDGPU_VA_RESERVED_TRAP_SIZE to 2 * PAGE_SIZE) fixes a kernel crash observed when running rocminfo on systems with a 64 KB page size. This patch is required to enable minimal support for 64 KB system page sizes.
Since RFC v2, we observed AQL queue creation failures while running certain workloads on 64K page-size systems due to an expected queue size mismatch. This issue is addressed in patch 2 of this series.
The questions we had in this seres are:
=======================================
1 When the control stack size is aligned to 64 KB, we consistently
observe queue preemption or eviction failures on gfx9, on both 4 KB and 64 KB system page-size configurations.
The control stack size is calculated based on the number of CUs and waves and is then aligned to PAGE_SIZE. On systems with a 64 KB system page size, this alignment always results in a 64 KB-aligned control stack size, after which queue preemption fails.
Is there any hardware-imposed limitation on gfx9 that prevents the control stack size from being 64 KB? For gfx10, I see explicit hardware limitations on the control stack size in the code [2]. Is there anything similar for gfx9?
What is the correct or recommended control stack size for gfx9? With a 4 KB system page size, I observe a control stack size of around 44 KB—can it grow beyond this? If the control stack size is fixed for a given gfx version, do you see any issues with aligning the control stack size to the GPU page size?
This series has 5 patches
=========================
1. AMDGPU_VA_RESERVED_TRAP_SIZE was hard-coded to 8 KB while KFD_CWSR_TBA_TMA_SIZE is defined as 2 * PAGE_SIZE, which matches on 4 KB page-size systems but results in a size mismatch on 64 KB systems, leading to kernel crashes when running rocminfo or RCCL unit tests.
This patch updates AMDGPU_VA_RESERVED_TRAP_SIZE to 2 * PAGE_SIZE so that the reserved trap area matches the allocation size across all system page sizes. This is a must needed patch to enable minimal support for 64 KB system page sizes.
2. Aligned expected_queue_size to PAGE_SIZE to fix AQL queue creation failure.
3. Fix amdgpu page fault handler (for xnack) to pass the corresponding system pfn (instead of gpu pfn) for restoring SVM range mapping.
4. Updated AMDGPU_GTT_MAX_TRANSFER_SIZE to always match the PMD size across all page sizes.
5. On systems where the CPU page size is larger than the GPU’s 4 KB page size, the MQD and control stack were aligned to the CPU PAGE_SIZE, causing multiple GPU pages to incorrectly inherit the UC attribute. This change aligns both regions to the GPU page size, ensuring that the MQD is mapped as UC and the control stack as NC, restoring the correct behavior.
6. Queue preemption fails when the control stack size is aligned to 64 KB. This patch fixes this issue by aligning the control stack size to gpu page size.
Setup details:
============
System details: Power10 LPAR using 64K pagesize.
AMD GPU:
Name: gfx90a Marketing Name: AMD Instinct MI210
[1] https://lore.kernel.org/all/[email protected]/
[2] https://elixir.bootlin.com/linux/v6.19-rc5/source/drivers/gpu/drm/amd/amdkfd/kfd_queue.c#L457
RFC V3 - https://lore.kernel.org/all/[email protected]/
RFC V2 - https://lore.kernel.org/all/[email protected]/
RFC V1 - https://lore.kernel.org/all/[email protected]/
Donet Tom (6):
drm/amdgpu: Change AMDGPU_VA_RESERVED_TRAP_SIZE to 2 PAGE_SIZE pages drm/amdkfd: Align expected_queue_size to PAGE_SIZE drm/amdgpu: Handle GPU page faults correctly on non-4K page systems drm/amdgpu: Fix AMDGPU_GTT_MAX_TRANSFER_SIZE for non-4K page size drm/amd: Fix MQD and control stack alignment for non-4K drm/amdkfd: Fix queue preemption/eviction failures by aligning control
stack size to GPU page size
drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 44 +++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 24 ++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 6 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 2 +- drivers/gpu/drm/amd/amdgpu/vce_v1_0.c | 3 +- .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c | 23 ++++++---- drivers/gpu/drm/amd/amdkfd/kfd_queue.c | 11 ++--- 9 files changed, 82 insertions(+), 35 deletions(-)
Signature · 1 line
2.52.0
RFC PATCH v3 1/6 drm/amdgpu: Change AMDGPU_VA_RESERVED_TRAP_SIZE to 2 PAGE_SIZE pages
Currently, AMDGPU_VA_RESERVED_TRAP_SIZE is hardcoded to 8KB, while KFD_CWSR_TBA_TMA_SIZE is defined as 2 * PAGE_SIZE. On systems with 4K pages, both values match (8KB), so allocation and reserved space are consistent.
However, on 64K page-size systems, KFD_CWSR_TBA_TMA_SIZE becomes 128KB, while the reserved trap area remains 8KB. This mismatch causes the kernel to crash when running rocminfo or rccl unit tests.
Kernel attempted to read user page (2) - exploit attempt? (uid: 1001)
BUG: Kernel NULL pointer dereference on read at 0x00000002 Faulting instruction address: 0xc0000000002c8a64 Oops: Kernel access of bad area, sig: 11 [#1] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA pSeries CPU: 34 UID: 1001 PID: 9379 Comm: rocminfo Tainted: G E 6.19.0-rc4-amdgpu-00320-gf23176405700 #56 VOLUNTARY Tainted: [E]=UNSIGNED_MODULE Hardware name: IBM,9105-42A POWER10 (architected) 0x800200 0xf000006 of:IBM,FW1060.30 (ML1060_896) hv:phyp pSeries NIP: c0000000002c8a64 LR: c00000000125dbc8 CTR: c00000000125e730 REGS: c0000001e0957580 TRAP: 0300 Tainted: G E MSR: 8000000000009033 <SF,EE,ME,IR,DR,RI,LE> CR: 24008268 XER: 00000036 CFAR: c00000000125dbc4 DAR: 0000000000000002 DSISR: 40000000 IRQMASK: 1 GPR00: c00000000125d908 c0000001e0957820 c0000000016e8100
c00000013d814540
GPR04: 0000000000000002 c00000013d814550 0000000000000045 0000000000000000
GPR08: c00000013444d000 c00000013d814538 c00000013d814538 0000000084002268
GPR12: c00000000125e730 c000007e2ffd5f00 ffffffffffffffff 0000000000020000
GPR16: 0000000000000000 0000000000000002 c00000015f653000 0000000000000000
GPR20: c000000138662400 c00000013d814540 0000000000000000 c00000013d814500
GPR24: 0000000000000000 0000000000000002 c0000001e0957888 c0000001e0957878
GPR28: c00000013d814548 0000000000000000 c00000013d814540 c0000001e0957888
NIP [c0000000002c8a64] __mutex_add_waiter+0x24/0xc0 LR [c00000000125dbc8] __mutex_lock.constprop.0+0x318/0xd00
Call Trace: 0xc0000001e0957890 (unreliable) __mutex_lock.constprop.0+0x58/0xd00 amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu+0x6fc/0xb60 [amdgpu] kfd_process_alloc_gpuvm+0x54/0x1f0 [amdgpu] kfd_process_device_init_cwsr_dgpu+0xa4/0x1a0 [amdgpu] kfd_process_device_init_vm+0xd8/0x2e0 [amdgpu] kfd_ioctl_acquire_vm+0xd0/0x130 [amdgpu] kfd_ioctl+0x514/0x670 [amdgpu] sys_ioctl+0x134/0x180 system_call_exception+0x114/0x300 system_call_vectored_common+0x15c/0x2ec
This patch changes AMDGPU_VA_RESERVED_TRAP_SIZE to 2 * PAGE_SIZE, ensuring that the reserved trap area matches the allocation size across all page sizes.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +1 -1
@@ -173,7 +173,7 @@ struct amdgpu_bo_vm;#define AMDGPU_VA_RESERVED_SEQ64_SIZE (2ULL << 20)#define AMDGPU_VA_RESERVED_SEQ64_START (adev) (AMDGPU_VA_RESERVED_CSA_START(adev) \ - AMDGPU_VA_RESERVED_SEQ64_SIZE) -#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << 12) +#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << PAGE_SHIFT)#define AMDGPU_VA_RESERVED_TRAP_START (adev) (AMDGPU_VA_RESERVED_SEQ64_START(adev) \ - AMDGPU_VA_RESERVED_TRAP_SIZE)#define AMDGPU_VA_RESERVED_BOTTOM (1ULL << 16)
Signature · 1 line
2.52.0
Christian König
maintainerRe: [RESEND RFC PATCH v3 1/6] drm/amdgpu: Change AMDGPU_VA_RESERVED_TRAP_SIZE to 2 PAGE_SIZE pages
Quoted 73 lines · Donet Tom
Currently,
AMDGPU_VA_RESERVED_TRAP_SIZEis hardcoded to 8KB, whileKFD_CWSR_TBA_TMA_SIZEis defined as 2 *PAGE_SIZE. On systems with 4K pages, both values match (8KB), so allocation and reserved space are consistent.However, on 64K page-size systems,
KFD_CWSR_TBA_TMA_SIZEbecomes 128KB, while the reserved trap area remains 8KB. This mismatch causes the kernel to crash when running rocminfo or rccl unit tests.Kernel attempted to read user page (2) - exploit attempt? (uid: 1001)
BUG: Kernel NULL pointer dereference on read at 0x00000002 Faulting instruction address: 0xc0000000002c8a64 Oops: Kernel access of bad area, sig: 11 [#1] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA pSeries CPU: 34 UID: 1001 PID: 9379 Comm: rocminfo Tainted: G E 6.19.0-rc4-amdgpu-00320-gf23176405700 #56 VOLUNTARY Tainted: [E]=UNSIGNED_MODULE Hardware name: IBM,9105-42A POWER10 (architected) 0x800200 0xf000006 of:IBM,FW1060.30 (ML1060_896) hv:phyp pSeries NIP: c0000000002c8a64 LR: c00000000125dbc8 CTR: c00000000125e730 REGS: c0000001e0957580 TRAP: 0300 Tainted: G E MSR: 8000000000009033 <SF,EE,ME,IR,DR,RI,LE> CR: 24008268 XER: 00000036 CFAR: c00000000125dbc4 DAR: 0000000000000002 DSISR: 40000000 IRQMASK: 1 GPR00: c00000000125d908 c0000001e0957820 c0000000016e8100
c00000013d814540
GPR04: 0000000000000002c00000013d8145500000000000000045 0000000000000000
GPR08:c00000013444d000c00000013d814538c00000013d8145380000000084002268
GPR12:c00000000125e730c000007e2ffd5f00ffffffffffffffff 0000000000020000
GPR16: 0000000000000000 0000000000000002c00000015f6530000000000000000000
GPR20:c000000138662400c00000013d8145400000000000000000c00000013d814500
GPR24: 0000000000000000 0000000000000002c0000001e0957888c0000001e0957878
GPR28:c00000013d8145480000000000000000c00000013d814540c0000001e0957888
NIP [c0000000002c8a64]__mutex_add_waiter+0x24/0xc0LR [c00000000125dbc8] __mutex_lock.constprop.0+0x318/0xd00Call Trace: 0xc0000001e0957890 (unreliable) __mutex_lock.constprop.0+0x58/0xd00 amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu+0x6fc/0xb60 [amdgpu] kfd_process_alloc_gpuvm+0x54/0x1f0 [amdgpu] kfd_process_device_init_cwsr_dgpu+0xa4/0x1a0 [amdgpu] kfd_process_device_init_vm+0xd8/0x2e0 [amdgpu] kfd_ioctl_acquire_vm+0xd0/0x130 [amdgpu] kfd_ioctl+0x514/0x670 [amdgpu] sys_ioctl+0x134/0x180 system_call_exception+0x114/0x300 system_call_vectored_common+0x15c/0x2ecThis patch changes
AMDGPU_VA_RESERVED_TRAP_SIZEto 2 *PAGE_SIZE, ensuring that the reserved trap area matches the allocation size across all page sizes.Fixes 34a1de0f7935 ("drm/amdkfd: Relocate TBA/TMA to opposite side of VM hole")Reviewed-by Ritesh Harjani (IBM) <[email protected]>Signed-off-by Donet Tom <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0
-0
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h struct amdgpu_bo_vm; · line 176
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +1
-1@@ -173,7 +173,7 @@ struct amdgpu_bo_vm;#define AMDGPU_VA_RESERVED_SEQ64_SIZE (2ULL << 20)#define AMDGPU_VA_RESERVED_SEQ64_START (adev) (AMDGPU_VA_RESERVED_CSA_START(adev) \ - AMDGPU_VA_RESERVED_SEQ64_SIZE) -#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << 12) +#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << PAGE_SHIFT)
Well using PAGE_SHIFT in amdgpu_vm.h looks quite broken to me.
That makes the GPU VA reservation depend on the CPU page size and that is clearly not something we want to have.
Where is KFD_CWSR_TBA_TMA_SIZE defined?
Regards,
Christian.
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0
-0#define AMDGPU_VA_RESERVED_TRAP_START (adev) (AMDGPU_VA_RESERVED_SEQ64_START(adev) \ - AMDGPU_VA_RESERVED_TRAP_SIZE)#define AMDGPU_VA_RESERVED_BOTTOM (1ULL << 16)
Re: [RESEND RFC PATCH v3 1/6] drm/amdgpu: Change AMDGPU_VA_RESERVED_TRAP_SIZE to 2 PAGE_SIZE pages
On 3/23/26 3:41 PM, Christian König wrote:
Hi Christian
Quoted 81 lines · Donet Tom
Quoted 73 lines
Currently,
AMDGPU_VA_RESERVED_TRAP_SIZEis hardcoded to 8KB, whileKFD_CWSR_TBA_TMA_SIZEis defined as 2 *PAGE_SIZE. On systems with 4K pages, both values match (8KB), so allocation and reserved space are consistent.However, on 64K page-size systems,
KFD_CWSR_TBA_TMA_SIZEbecomes 128KB, while the reserved trap area remains 8KB. This mismatch causes the kernel to crash when running rocminfo or rccl unit tests.Kernel attempted to read user page (2) - exploit attempt? (uid: 1001)
BUG: Kernel NULL pointer dereference on read at 0x00000002 Faulting instruction address: 0xc0000000002c8a64 Oops: Kernel access of bad area, sig: 11 [#1] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA pSeries CPU: 34 UID: 1001 PID: 9379 Comm: rocminfo Tainted: G E 6.19.0-rc4-amdgpu-00320-gf23176405700 #56 VOLUNTARY Tainted: [E]=UNSIGNED_MODULE Hardware name: IBM,9105-42A POWER10 (architected) 0x800200 0xf000006 of:IBM,FW1060.30 (ML1060_896) hv:phyp pSeries NIP: c0000000002c8a64 LR: c00000000125dbc8 CTR: c00000000125e730 REGS: c0000001e0957580 TRAP: 0300 Tainted: G E MSR: 8000000000009033 <SF,EE,ME,IR,DR,RI,LE> CR: 24008268 XER: 00000036 CFAR: c00000000125dbc4 DAR: 0000000000000002 DSISR: 40000000 IRQMASK: 1 GPR00: c00000000125d908 c0000001e0957820 c0000000016e8100
c00000013d814540
GPR04: 0000000000000002c00000013d8145500000000000000045 0000000000000000
GPR08:c00000013444d000c00000013d814538c00000013d8145380000000084002268
GPR12:c00000000125e730c000007e2ffd5f00ffffffffffffffff 0000000000020000
GPR16: 0000000000000000 0000000000000002c00000015f6530000000000000000000
GPR20:c000000138662400c00000013d8145400000000000000000c00000013d814500
GPR24: 0000000000000000 0000000000000002c0000001e0957888c0000001e0957878
GPR28:c00000013d8145480000000000000000c00000013d814540c0000001e0957888
NIP [c0000000002c8a64]__mutex_add_waiter+0x24/0xc0LR [c00000000125dbc8] __mutex_lock.constprop.0+0x318/0xd00Call Trace: 0xc0000001e0957890 (unreliable) __mutex_lock.constprop.0+0x58/0xd00 amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu+0x6fc/0xb60 [amdgpu] kfd_process_alloc_gpuvm+0x54/0x1f0 [amdgpu] kfd_process_device_init_cwsr_dgpu+0xa4/0x1a0 [amdgpu] kfd_process_device_init_vm+0xd8/0x2e0 [amdgpu] kfd_ioctl_acquire_vm+0xd0/0x130 [amdgpu] kfd_ioctl+0x514/0x670 [amdgpu] sys_ioctl+0x134/0x180 system_call_exception+0x114/0x300 system_call_vectored_common+0x15c/0x2ecThis patch changes
AMDGPU_VA_RESERVED_TRAP_SIZEto 2 *PAGE_SIZE, ensuring that the reserved trap area matches the allocation size across all page sizes.Fixes 34a1de0f7935 ("drm/amdkfd: Relocate TBA/TMA to opposite side of VM hole")Reviewed-by Ritesh Harjani (IBM)<[email protected]>Signed-off-by Donet Tom<[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0
-0drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h struct amdgpu_bo_vm; · line 176
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +1
-1@@ -173,7 +173,7 @@ struct amdgpu_bo_vm;#define AMDGPU_VA_RESERVED_SEQ64_SIZE (2ULL << 20)#define AMDGPU_VA_RESERVED_SEQ64_START (adev) (AMDGPU_VA_RESERVED_CSA_START(adev) \ - AMDGPU_VA_RESERVED_SEQ64_SIZE) -#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << 12) +#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << PAGE_SHIFT)Well using
PAGE_SHIFTinamdgpu_vm.hlooks quite broken to me.
Quoted 4 lines
That makes the GPU VA reservation depend on the CPU page size and that is clearly not something we want to have.
Where is
KFD_CWSR_TBA_TMA_SIZEdefined?
Thanks Christian for reviewing this patch.
It is defined in kfd_priv.h.
/* * Size of the per-process TBA+TMA buffer: 2 pages * * The first chunk is the TBA used for the CWSR ISA code. The second * chunk is used as TMA for user-mode trap handler setup in daisy-chain mode. */ #define KFD_CWSR_TBA_TMA_SIZE (PAGE_SIZE * 2)
Could you please suggest the correct way to fix this issue? -Donet
Quoted 6 lines
Regards,
Christian.Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0
-0#define AMDGPU_VA_RESERVED_TRAP_START (adev) (AMDGPU_VA_RESERVED_SEQ64_START(adev) \ - AMDGPU_VA_RESERVED_TRAP_SIZE)#define AMDGPU_VA_RESERVED_BOTTOM (1ULL << 16)
Christian König
maintainerRe: [RESEND RFC PATCH v3 1/6] drm/amdgpu: Change AMDGPU_VA_RESERVED_TRAP_SIZE to 2 PAGE_SIZE pages
Quoted 105 lines · Donet Tom
On 3/23/26 3:41 PM, Christian König wrote:
Hi Christian
Quoted 81 lines · Donet Tom
Quoted 73 lines
Currently,
AMDGPU_VA_RESERVED_TRAP_SIZEis hardcoded to 8KB, whileKFD_CWSR_TBA_TMA_SIZEis defined as 2 *PAGE_SIZE. On systems with 4K pages, both values match (8KB), so allocation and reserved space are consistent.However, on 64K page-size systems,
KFD_CWSR_TBA_TMA_SIZEbecomes 128KB, while the reserved trap area remains 8KB. This mismatch causes the kernel to crash when running rocminfo or rccl unit tests.Kernel attempted to read user page (2) - exploit attempt? (uid: 1001)
BUG: Kernel NULL pointer dereference on read at 0x00000002 Faulting instruction address: 0xc0000000002c8a64 Oops: Kernel access of bad area, sig: 11 [#1] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA pSeries CPU: 34 UID: 1001 PID: 9379 Comm: rocminfo Tainted: G E 6.19.0-rc4-amdgpu-00320-gf23176405700 #56 VOLUNTARY Tainted: [E]=UNSIGNED_MODULE Hardware name: IBM,9105-42A POWER10 (architected) 0x800200 0xf000006 of:IBM,FW1060.30 (ML1060_896) hv:phyp pSeries NIP: c0000000002c8a64 LR: c00000000125dbc8 CTR: c00000000125e730 REGS: c0000001e0957580 TRAP: 0300 Tainted: G E MSR: 8000000000009033 <SF,EE,ME,IR,DR,RI,LE> CR: 24008268 XER: 00000036 CFAR: c00000000125dbc4 DAR: 0000000000000002 DSISR: 40000000 IRQMASK: 1 GPR00: c00000000125d908 c0000001e0957820 c0000000016e8100
c00000013d814540
GPR04: 0000000000000002c00000013d8145500000000000000045 0000000000000000
GPR08:c00000013444d000c00000013d814538c00000013d8145380000000084002268
GPR12:c00000000125e730c000007e2ffd5f00ffffffffffffffff 0000000000020000
GPR16: 0000000000000000 0000000000000002c00000015f6530000000000000000000
GPR20:c000000138662400c00000013d8145400000000000000000c00000013d814500
GPR24: 0000000000000000 0000000000000002c0000001e0957888c0000001e0957878
GPR28:c00000013d8145480000000000000000c00000013d814540c0000001e0957888
NIP [c0000000002c8a64]__mutex_add_waiter+0x24/0xc0LR [c00000000125dbc8] __mutex_lock.constprop.0+0x318/0xd00Call Trace: 0xc0000001e0957890 (unreliable) __mutex_lock.constprop.0+0x58/0xd00 amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu+0x6fc/0xb60 [amdgpu] kfd_process_alloc_gpuvm+0x54/0x1f0 [amdgpu] kfd_process_device_init_cwsr_dgpu+0xa4/0x1a0 [amdgpu] kfd_process_device_init_vm+0xd8/0x2e0 [amdgpu] kfd_ioctl_acquire_vm+0xd0/0x130 [amdgpu] kfd_ioctl+0x514/0x670 [amdgpu] sys_ioctl+0x134/0x180 system_call_exception+0x114/0x300 system_call_vectored_common+0x15c/0x2ecThis patch changes
AMDGPU_VA_RESERVED_TRAP_SIZEto 2 *PAGE_SIZE, ensuring that the reserved trap area matches the allocation size across all page sizes.Fixes 34a1de0f7935 ("drm/amdkfd: Relocate TBA/TMA to opposite side of VM hole")Reviewed-by Ritesh Harjani (IBM) <[email protected]>Signed-off-by Donet Tom <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0
-0drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h struct amdgpu_bo_vm; · line 176
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +1
-1@@ -173,7 +173,7 @@ struct amdgpu_bo_vm;#define AMDGPU_VA_RESERVED_SEQ64_SIZE (2ULL << 20)#define AMDGPU_VA_RESERVED_SEQ64_START (adev) (AMDGPU_VA_RESERVED_CSA_START(adev) \ - AMDGPU_VA_RESERVED_SEQ64_SIZE) -#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << 12) +#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << PAGE_SHIFT)Well using
PAGE_SHIFTinamdgpu_vm.hlooks quite broken to me.Quoted 4 lines
That makes the GPU VA reservation depend on the CPU page size and that is clearly not something we want to have.
Where is
KFD_CWSR_TBA_TMA_SIZEdefined?Thanks Christian for reviewing this patch.
It is defined in
kfd_priv.h./*
* Size of the per-process TBA+TMA buffer: 2 pages
*
* The first chunk is the TBA used for the CWSR ISA code. The second
* chunk is used as TMA for user-mode trap handler setup in daisy-chain mode. */
#defineKFD_CWSR_TBA_TMA_SIZE(PAGE_SIZE* 2)
Quoted 1 line
Could you please suggest the correct way to fix this issue?
I'm only looking from the POV of the VM code on this, but my educated guess is that KFD_CWSR_TBA_TMA_SIZE should be 8k independent of the CPU page size.
Background is that this is written by the shader trap handler and that byte code doesn't care what CPU architecture you have.
But I think only the engineers working on that trap handler can really answer this. @Felix / @Philip?
Regards,
Christian.
Quoted 9 lines
-Donet
Quoted 6 lines
Regards,
Christian.Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0
-0#define AMDGPU_VA_RESERVED_TRAP_START (adev) (AMDGPU_VA_RESERVED_SEQ64_START(adev) \ - AMDGPU_VA_RESERVED_TRAP_SIZE)#define AMDGPU_VA_RESERVED_BOTTOM (1ULL << 16)
Re: [RESEND RFC PATCH v3 1/6] drm/amdgpu: Change AMDGPU_VA_RESERVED_TRAP_SIZE to 2 PAGE_SIZE pages
Quoted 104 lines · Christian König
Quoted 103 lines · Donet Tom
On 3/23/26 3:41 PM, Christian König wrote:
Hi Christian
Quoted 81 lines · Donet Tom
Quoted 73 lines
Currently,
AMDGPU_VA_RESERVED_TRAP_SIZEis hardcoded to 8KB, whileKFD_CWSR_TBA_TMA_SIZEis defined as 2 *PAGE_SIZE. On systems with 4K pages, both values match (8KB), so allocation and reserved space are consistent.However, on 64K page-size systems,
KFD_CWSR_TBA_TMA_SIZEbecomes 128KB, while the reserved trap area remains 8KB. This mismatch causes the kernel to crash when running rocminfo or rccl unit tests.Kernel attempted to read user page (2) - exploit attempt? (uid: 1001)
BUG: Kernel NULL pointer dereference on read at 0x00000002 Faulting instruction address: 0xc0000000002c8a64 Oops: Kernel access of bad area, sig: 11 [#1] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA pSeries CPU: 34 UID: 1001 PID: 9379 Comm: rocminfo Tainted: G E 6.19.0-rc4-amdgpu-00320-gf23176405700 #56 VOLUNTARY Tainted: [E]=UNSIGNED_MODULE Hardware name: IBM,9105-42A POWER10 (architected) 0x800200 0xf000006 of:IBM,FW1060.30 (ML1060_896) hv:phyp pSeries NIP: c0000000002c8a64 LR: c00000000125dbc8 CTR: c00000000125e730 REGS: c0000001e0957580 TRAP: 0300 Tainted: G E MSR: 8000000000009033 <SF,EE,ME,IR,DR,RI,LE> CR: 24008268 XER: 00000036 CFAR: c00000000125dbc4 DAR: 0000000000000002 DSISR: 40000000 IRQMASK: 1 GPR00: c00000000125d908 c0000001e0957820 c0000000016e8100
c00000013d814540
GPR04: 0000000000000002c00000013d8145500000000000000045 0000000000000000
GPR08:c00000013444d000c00000013d814538c00000013d8145380000000084002268
GPR12:c00000000125e730c000007e2ffd5f00ffffffffffffffff 0000000000020000
GPR16: 0000000000000000 0000000000000002c00000015f6530000000000000000000
GPR20:c000000138662400c00000013d8145400000000000000000c00000013d814500
GPR24: 0000000000000000 0000000000000002c0000001e0957888c0000001e0957878
GPR28:c00000013d8145480000000000000000c00000013d814540c0000001e0957888
NIP [c0000000002c8a64]__mutex_add_waiter+0x24/0xc0LR [c00000000125dbc8] __mutex_lock.constprop.0+0x318/0xd00Call Trace: 0xc0000001e0957890 (unreliable) __mutex_lock.constprop.0+0x58/0xd00 amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu+0x6fc/0xb60 [amdgpu] kfd_process_alloc_gpuvm+0x54/0x1f0 [amdgpu] kfd_process_device_init_cwsr_dgpu+0xa4/0x1a0 [amdgpu] kfd_process_device_init_vm+0xd8/0x2e0 [amdgpu] kfd_ioctl_acquire_vm+0xd0/0x130 [amdgpu] kfd_ioctl+0x514/0x670 [amdgpu] sys_ioctl+0x134/0x180 system_call_exception+0x114/0x300 system_call_vectored_common+0x15c/0x2ecThis patch changes
AMDGPU_VA_RESERVED_TRAP_SIZEto 2 *PAGE_SIZE, ensuring that the reserved trap area matches the allocation size across all page sizes.Fixes 34a1de0f7935 ("drm/amdkfd: Relocate TBA/TMA to opposite side of VM hole")Reviewed-by Ritesh Harjani (IBM) <[email protected]>Signed-off-by Donet Tom <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0
-0drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h struct amdgpu_bo_vm; · line 176
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +1
-1@@ -173,7 +173,7 @@ struct amdgpu_bo_vm;#define AMDGPU_VA_RESERVED_SEQ64_SIZE (2ULL << 20)#define AMDGPU_VA_RESERVED_SEQ64_START (adev) (AMDGPU_VA_RESERVED_CSA_START(adev) \ - AMDGPU_VA_RESERVED_SEQ64_SIZE) -#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << 12) +#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << PAGE_SHIFT)Well using
PAGE_SHIFTinamdgpu_vm.hlooks quite broken to me.Quoted 4 lines
That makes the GPU VA reservation depend on the CPU page size and that is clearly not something we want to have.
Where is
KFD_CWSR_TBA_TMA_SIZEdefined?Thanks Christian for reviewing this patch.
It is defined in
kfd_priv.h./*
* Size of the per-process TBA+TMA buffer: 2 pages
*
* The first chunk is the TBA used for the CWSR ISA code. The second
* chunk is used as TMA for user-mode trap handler setup in daisy-chain mode. */
#defineKFD_CWSR_TBA_TMA_SIZE(PAGE_SIZE* 2)
Quoted 6 lines
Quoted 1 line
Could you please suggest the correct way to fix this issue?
I'm only looking from the POV of the VM code on this, but my educated guess is that
KFD_CWSR_TBA_TMA_SIZEshould be 8k independent of the CPU page size.Background is that this is written by the shader trap handler and that byte code doesn't care what CPU architecture you have.
But I think only the engineers working on that trap handler can really answer this. @Felix / @Philip?
Hi @christian @Felix @Philip
To remove the dependency on CPU page size, can we use
+#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << 16)
During reservation, we reserve 128 bytes, but during allocation, we use 2 * PAGE_SIZE.
-Donet
Quoted 12 lines
Regards,
Christian.Quoted 8 lines
-Donet
Quoted 6 lines
Regards,
Christian.Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0
-0#define AMDGPU_VA_RESERVED_TRAP_START (adev) (AMDGPU_VA_RESERVED_SEQ64_START(adev) \ - AMDGPU_VA_RESERVED_TRAP_SIZE)#define AMDGPU_VA_RESERVED_BOTTOM (1ULL << 16)
Kuehling, Felix
maintainerRe: [RESEND RFC PATCH v3 1/6] drm/amdgpu: Change AMDGPU_VA_RESERVED_TRAP_SIZE to 2 PAGE_SIZE pages
Quoted 114 lines · Donet Tom
Quoted 104 lines · Christian König
Quoted 103 lines · Donet Tom
On 3/23/26 3:41 PM, Christian König wrote:
Hi Christian
Quoted 81 lines · Donet Tom
Quoted 73 lines
Currently,
AMDGPU_VA_RESERVED_TRAP_SIZEis hardcoded to 8KB, whileKFD_CWSR_TBA_TMA_SIZEis defined as 2 *PAGE_SIZE. On systems with 4K pages, both values match (8KB), so allocation and reserved space are consistent.However, on 64K page-size systems,
KFD_CWSR_TBA_TMA_SIZEbecomes 128KB, while the reserved trap area remains 8KB. This mismatch causes the kernel to crash when running rocminfo or rccl unit tests.Kernel attempted to read user page (2) - exploit attempt? (uid: 1001)
BUG: Kernel NULL pointer dereference on read at 0x00000002 Faulting instruction address: 0xc0000000002c8a64 Oops: Kernel access of bad area, sig: 11 [#1] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA pSeries CPU: 34 UID: 1001 PID: 9379 Comm: rocminfo Tainted: G E 6.19.0-rc4-amdgpu-00320-gf23176405700 #56 VOLUNTARY Tainted: [E]=UNSIGNED_MODULE Hardware name: IBM,9105-42A POWER10 (architected) 0x800200 0xf000006of:IBM,FW1060.30 (
ML1060_896) hv:phyp pSeries
NIP:c0000000002c8a64LR:c00000000125dbc8CTR:c00000000125e730
REGS:c0000001e0957580TRAP: 0300 Tainted: G E
MSR: 8000000000009033 <SF,EE,ME,IR,DR,RI,LE> CR: 24008268
XER: 00000036
CFAR:c00000000125dbc4DAR: 0000000000000002 DSISR: 40000000
IRQMASK: 1
GPR00:c00000000125d908c0000001e0957820c0000000016e8100c00000013d814540
GPR04: 0000000000000002c00000013d8145500000000000000045 0000000000000000
GPR08:c00000013444d000c00000013d814538c00000013d8145380000000084002268
GPR12:c00000000125e730c000007e2ffd5f00ffffffffffffffff 0000000000020000
GPR16: 0000000000000000 0000000000000002c00000015f6530000000000000000000
GPR20:c000000138662400c00000013d8145400000000000000000c00000013d814500
GPR24: 0000000000000000 0000000000000002c0000001e0957888c0000001e0957878
GPR28:c00000013d8145480000000000000000c00000013d814540c0000001e0957888
NIP [c0000000002c8a64]__mutex_add_waiter+0x24/0xc0
LR [c00000000125dbc8] __mutex_lock.constprop.0+0x318/0xd00Call Trace: 0xc0000001e0957890 (unreliable) __mutex_lock.constprop.0+0x58/0xd00 amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu+0x6fc/0xb60 [amdgpu] kfd_process_alloc_gpuvm+0x54/0x1f0 [amdgpu] kfd_process_device_init_cwsr_dgpu+0xa4/0x1a0 [amdgpu] kfd_process_device_init_vm+0xd8/0x2e0 [amdgpu] kfd_ioctl_acquire_vm+0xd0/0x130 [amdgpu] kfd_ioctl+0x514/0x670 [amdgpu] sys_ioctl+0x134/0x180 system_call_exception+0x114/0x300 system_call_vectored_common+0x15c/0x2ecThis patch changes
AMDGPU_VA_RESERVED_TRAP_SIZEto 2 *PAGE_SIZE, ensuring that the reserved trap area matches the allocation size across all page sizes.Fixes 34a1de0f7935 ("drm/amdkfd: Relocate TBA/TMA to opposite side of VM hole")Reviewed-by Ritesh Harjani (IBM) <[email protected]>Signed-off-by Donet Tom <[email protected]>---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0
-0drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h struct amdgpu_bo_vm; · line 176
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +1
-1@@ -173,7 +173,7 @@ struct amdgpu_bo_vm; #define AMDGPU_VA_RESERVED_SEQ64_SIZE (2ULL << 20) #define AMDGPU_VA_RESERVED_SEQ64_START(adev) (AMDGPU_VA_RESERVED_CSA_START(adev) \ - AMDGPU_VA_RESERVED_SEQ64_SIZE) -#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << 12) +#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << PAGE_SHIFT)Well using
PAGE_SHIFTinamdgpu_vm.hlooks quite broken to me.Quoted 4 lines
That makes the GPU VA reservation depend on the CPU page size and that is clearly not something we want to have.
Where is
KFD_CWSR_TBA_TMA_SIZEdefined?Thanks Christian for reviewing this patch.
It is defined in
kfd_priv.h./*
* Size of the per-process TBA+TMA buffer: 2 pages
*
* The first chunk is the TBA used for the CWSR ISA code. The second
* chunk is used as TMA for user-mode trap handler setup in daisy-chain mode. */
#defineKFD_CWSR_TBA_TMA_SIZE(PAGE_SIZE* 2)Quoted 6 lines
Quoted 1 line
Could you please suggest the correct way to fix this issue?
I'm only looking from the POV of the VM code on this, but my educated guess is that
KFD_CWSR_TBA_TMA_SIZEshould be 8k independent of the CPU page size.Background is that this is written by the shader trap handler and that byte code doesn't care what CPU architecture you have.
But I think only the engineers working on that trap handler can really answer this. @Felix / @Philip?
Quoted 8 lines
Hi @christian @Felix @Philip
To remove the dependency on CPU page size, can we use
+#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << 16)
During reservation, we reserve 128 bytes, but during allocation, we use 2 *
PAGE_SIZE.
We only need two GPU pages here. I think what Christian is objecting to is, that the GPU VM layout should not depend on the CPU page size. @Christian, it sounds like the BO allocations happen with 64KB granularity, but the mapping is still using 4KB granularity. Is the right solution to GPU-map only the first 8KB of the trap handler BO to keep the layout the same across CPU architectures?
I guess then the "correct" solution would be to change amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu and amdgpu_amdkfd_gpuvm_map_memory_to_gpu to support mapping of the requested size with GPU page size granularity regardless of the CPU page size. But that would increase complexity for a very niche uses case.
An easier solution would be to PAGE_ALIGN 8KB to the system page size. But that results in the virtual address space layout to depend on the system page size.
If that's objectionable, then the next best solution is to round up the trap handler size to 64KB byte unconditionally, so its the same with 4KB or 64KB system page size. But that would mean unnecessarily wasting a little memory per process/GPU on x86.
Regards,
Felix
Quoted 16 lines
-Donet
Quoted 12 lines
Regards,
Christian.Quoted 8 lines
-Donet
Quoted 6 lines
Regards,
Christian.Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0
-0#define AMDGPU_VA_RESERVED_TRAP_START(adev) (AMDGPU_VA_RESERVED_SEQ64_START(adev) \ - AMDGPU_VA_RESERVED_TRAP_SIZE) #define AMDGPU_VA_RESERVED_BOTTOM (1ULL << 16)
Christian König
maintainerRe: [RESEND RFC PATCH v3 1/6] drm/amdgpu: Change AMDGPU_VA_RESERVED_TRAP_SIZE to 2 PAGE_SIZE pages
Quoted 118 lines · Felix
Quoted 116 lines · Donet Tom
Quoted 104 lines · Christian König
Quoted 103 lines · Donet Tom
On 3/23/26 3:41 PM, Christian König wrote:
Hi Christian
Quoted 81 lines · Donet Tom
Quoted 73 lines
Currently,
AMDGPU_VA_RESERVED_TRAP_SIZEis hardcoded to 8KB, whileKFD_CWSR_TBA_TMA_SIZEis defined as 2 *PAGE_SIZE. On systems with 4K pages, both values match (8KB), so allocation and reserved space are consistent.However, on 64K page-size systems,
KFD_CWSR_TBA_TMA_SIZEbecomes 128KB, while the reserved trap area remains 8KB. This mismatch causes the kernel to crash when running rocminfo or rccl unit tests.Kernel attempted to read user page (2) - exploit attempt? (uid: 1001)
BUG: Kernel NULL pointer dereference on read at 0x00000002 Faulting instruction address: 0xc0000000002c8a64 Oops: Kernel access of bad area, sig: 11 [#1] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA pSeries CPU: 34 UID: 1001 PID: 9379 Comm: rocminfo Tainted: G E 6.19.0-rc4-amdgpu-00320-gf23176405700 #56 VOLUNTARY Tainted: [E]=UNSIGNED_MODULE Hardware name: IBM,9105-42A POWER10 (architected) 0x800200 0xf000006of:IBM,FW1060.30 (
ML1060_896) hv:phyp pSeries
NIP:c0000000002c8a64LR:c00000000125dbc8CTR:c00000000125e730
REGS:c0000001e0957580TRAP: 0300 Tainted: G E
MSR: 8000000000009033 <SF,EE,ME,IR,DR,RI,LE> CR: 24008268
XER: 00000036
CFAR:c00000000125dbc4DAR: 0000000000000002 DSISR: 40000000
IRQMASK: 1
GPR00:c00000000125d908c0000001e0957820c0000000016e8100c00000013d814540
GPR04: 0000000000000002c00000013d8145500000000000000045 0000000000000000
GPR08:c00000013444d000c00000013d814538c00000013d8145380000000084002268
GPR12:c00000000125e730c000007e2ffd5f00ffffffffffffffff 0000000000020000
GPR16: 0000000000000000 0000000000000002c00000015f6530000000000000000000
GPR20:c000000138662400c00000013d8145400000000000000000c00000013d814500
GPR24: 0000000000000000 0000000000000002c0000001e0957888c0000001e0957878
GPR28:c00000013d8145480000000000000000c00000013d814540c0000001e0957888
NIP [c0000000002c8a64]__mutex_add_waiter+0x24/0xc0
LR [c00000000125dbc8] __mutex_lock.constprop.0+0x318/0xd00Call Trace: 0xc0000001e0957890 (unreliable) __mutex_lock.constprop.0+0x58/0xd00 amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu+0x6fc/0xb60 [amdgpu] kfd_process_alloc_gpuvm+0x54/0x1f0 [amdgpu] kfd_process_device_init_cwsr_dgpu+0xa4/0x1a0 [amdgpu] kfd_process_device_init_vm+0xd8/0x2e0 [amdgpu] kfd_ioctl_acquire_vm+0xd0/0x130 [amdgpu] kfd_ioctl+0x514/0x670 [amdgpu] sys_ioctl+0x134/0x180 system_call_exception+0x114/0x300 system_call_vectored_common+0x15c/0x2ecThis patch changes
AMDGPU_VA_RESERVED_TRAP_SIZEto 2 *PAGE_SIZE, ensuring that the reserved trap area matches the allocation size across all page sizes.Fixes 34a1de0f7935 ("drm/amdkfd: Relocate TBA/TMA to opposite side of VM hole")Reviewed-by Ritesh Harjani (IBM) <[email protected]>Signed-off-by Donet Tom <[email protected]>---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0
-0drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h struct amdgpu_bo_vm; · line 176
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +1
-1@@ -173,7 +173,7 @@ struct amdgpu_bo_vm; #define AMDGPU_VA_RESERVED_SEQ64_SIZE (2ULL << 20) #define AMDGPU_VA_RESERVED_SEQ64_START(adev) (AMDGPU_VA_RESERVED_CSA_START(adev) \ - AMDGPU_VA_RESERVED_SEQ64_SIZE) -#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << 12) +#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << PAGE_SHIFT)Well using
PAGE_SHIFTinamdgpu_vm.hlooks quite broken to me.Quoted 4 lines
That makes the GPU VA reservation depend on the CPU page size and that is clearly not something we want to have.
Where is
KFD_CWSR_TBA_TMA_SIZEdefined?Thanks Christian for reviewing this patch.
It is defined in
kfd_priv.h./*
* Size of the per-process TBA+TMA buffer: 2 pages
*
* The first chunk is the TBA used for the CWSR ISA code. The second
* chunk is used as TMA for user-mode trap handler setup in daisy-chain mode. */
#defineKFD_CWSR_TBA_TMA_SIZE(PAGE_SIZE* 2)Quoted 6 lines
Quoted 1 line
Could you please suggest the correct way to fix this issue?
I'm only looking from the POV of the VM code on this, but my educated guess is that
KFD_CWSR_TBA_TMA_SIZEshould be 8k independent of the CPU page size.Background is that this is written by the shader trap handler and that byte code doesn't care what CPU architecture you have.
But I think only the engineers working on that trap handler can really answer this. @Felix / @Philip?
Hi @christian @Felix @Philip
Quoted 8 lines
Quoted 6 lines
To remove the dependency on CPU page size, can we use
+#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << 16)
During reservation, we reserve 128 bytes, but during allocation, we use 2 *
PAGE_SIZE.We only need two GPU pages here. I think what Christian is objecting to is, that the GPU VM layout should not depend on the CPU page size.
Yes, exactly that was my concern.
Quoted 1 line
@Christian, it sounds like the BO allocations happen with 64KB granularity, but the mapping is still using 4KB granularity. Is the right solution to GPU-map only the first 8KB of the trap handler BO to keep the layout the same across CPU architectures?
Well that would work technically, but I agree that it also sounds a bit questionable as well.
Quoted 3 lines
I guess then the "correct" solution would be to change
amdgpu_amdkfd_gpuvm_alloc_memory_of_gpuandamdgpu_amdkfd_gpuvm_map_memory_to_gputo support mapping of the requested size with GPU page size granularity regardless of the CPU page size. But that would increase complexity for a very niche uses case.An easier solution would be to
PAGE_ALIGN8KB to the system page size. But that results in the virtual address space layout to depend on the system page size.
Yeah, that dependency is certainly undesirable. We could easily end up with issues which can only be reproduced on systems with 64k page size.
Quoted 1 line
If that's objectionable, then the next best solution is to round up the trap handler size to 64KB byte unconditionally, so its the same with 4KB or 64KB system page size. But that would mean unnecessarily wasting a little memory per process/GPU on x86.
How about we always reserve 64KiB address space (or maybe even more, if you reserve 2MiB or 64KiB doesn't matter), but only map as large as the allocated buffer actually is?
I think that this would be my preferred solution.
Regards,
Christian.
Quoted 21 lines
Regards,
FelixQuoted 16 lines
-Donet
Quoted 12 lines
Regards,
Christian.Quoted 8 lines
-Donet
Quoted 6 lines
Regards,
Christian.Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0
-0#define AMDGPU_VA_RESERVED_TRAP_START(adev) (AMDGPU_VA_RESERVED_SEQ64_START(adev) \ - AMDGPU_VA_RESERVED_TRAP_SIZE) #define AMDGPU_VA_RESERVED_BOTTOM (1ULL << 16)
Re: [RESEND RFC PATCH v3 1/6] drm/amdgpu: Change AMDGPU_VA_RESERVED_TRAP_SIZE to 2 PAGE_SIZE pages
Quoted 130 lines · Christian König
Quoted 115 lines · Felix
Quoted 114 lines · Donet Tom
Quoted 104 lines · Christian König
Quoted 103 lines · Donet Tom
On 3/23/26 3:41 PM, Christian König wrote:
Hi Christian
Quoted 81 lines · Donet Tom
Quoted 73 lines
Currently,
AMDGPU_VA_RESERVED_TRAP_SIZEis hardcoded to 8KB, whileKFD_CWSR_TBA_TMA_SIZEis defined as 2 *PAGE_SIZE. On systems with 4K pages, both values match (8KB), so allocation and reserved space are consistent.However, on 64K page-size systems,
KFD_CWSR_TBA_TMA_SIZEbecomes 128KB, while the reserved trap area remains 8KB. This mismatch causes the kernel to crash when running rocminfo or rccl unit tests.Kernel attempted to read user page (2) - exploit attempt? (uid: 1001)
BUG: Kernel NULL pointer dereference on read at 0x00000002 Faulting instruction address: 0xc0000000002c8a64 Oops: Kernel access of bad area, sig: 11 [#1] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA pSeries CPU: 34 UID: 1001 PID: 9379 Comm: rocminfo Tainted: G E 6.19.0-rc4-amdgpu-00320-gf23176405700 #56 VOLUNTARY Tainted: [E]=UNSIGNED_MODULE Hardware name: IBM,9105-42A POWER10 (architected) 0x800200 0xf000006of:IBM,FW1060.30 (
ML1060_896) hv:phyp pSeries
NIP:c0000000002c8a64LR:c00000000125dbc8CTR:c00000000125e730
REGS:c0000001e0957580TRAP: 0300 Tainted: G E
MSR: 8000000000009033 <SF,EE,ME,IR,DR,RI,LE> CR: 24008268
XER: 00000036
CFAR:c00000000125dbc4DAR: 0000000000000002 DSISR: 40000000
IRQMASK: 1
GPR00:c00000000125d908c0000001e0957820c0000000016e8100c00000013d814540
GPR04: 0000000000000002c00000013d8145500000000000000045 0000000000000000
GPR08:c00000013444d000c00000013d814538c00000013d8145380000000084002268
GPR12:c00000000125e730c000007e2ffd5f00ffffffffffffffff 0000000000020000
GPR16: 0000000000000000 0000000000000002c00000015f6530000000000000000000
GPR20:c000000138662400c00000013d8145400000000000000000c00000013d814500
GPR24: 0000000000000000 0000000000000002c0000001e0957888c0000001e0957878
GPR28:c00000013d8145480000000000000000c00000013d814540c0000001e0957888
NIP [c0000000002c8a64]__mutex_add_waiter+0x24/0xc0
LR [c00000000125dbc8] __mutex_lock.constprop.0+0x318/0xd00Call Trace: 0xc0000001e0957890 (unreliable) __mutex_lock.constprop.0+0x58/0xd00 amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu+0x6fc/0xb60 [amdgpu] kfd_process_alloc_gpuvm+0x54/0x1f0 [amdgpu] kfd_process_device_init_cwsr_dgpu+0xa4/0x1a0 [amdgpu] kfd_process_device_init_vm+0xd8/0x2e0 [amdgpu] kfd_ioctl_acquire_vm+0xd0/0x130 [amdgpu] kfd_ioctl+0x514/0x670 [amdgpu] sys_ioctl+0x134/0x180 system_call_exception+0x114/0x300 system_call_vectored_common+0x15c/0x2ecThis patch changes
AMDGPU_VA_RESERVED_TRAP_SIZEto 2 *PAGE_SIZE, ensuring that the reserved trap area matches the allocation size across all page sizes.Fixes 34a1de0f7935 ("drm/amdkfd: Relocate TBA/TMA to opposite side of VM hole")Reviewed-by Ritesh Harjani (IBM) <[email protected]>Signed-off-by Donet Tom <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0
-0drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h struct amdgpu_bo_vm; · line 176
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +1
-1@@ -173,7 +173,7 @@ struct amdgpu_bo_vm; #define AMDGPU_VA_RESERVED_SEQ64_SIZE (2ULL << 20) #define AMDGPU_VA_RESERVED_SEQ64_START(adev) (AMDGPU_VA_RESERVED_CSA_START(adev) \ - AMDGPU_VA_RESERVED_SEQ64_SIZE) -#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << 12) +#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << PAGE_SHIFT)Well using
PAGE_SHIFTinamdgpu_vm.hlooks quite broken to me.Quoted 4 lines
That makes the GPU VA reservation depend on the CPU page size and that is clearly not something we want to have.
Where is
KFD_CWSR_TBA_TMA_SIZEdefined?Thanks Christian for reviewing this patch.
It is defined in
kfd_priv.h./*
* Size of the per-process TBA+TMA buffer: 2 pages
*
* The first chunk is the TBA used for the CWSR ISA code. The second
* chunk is used as TMA for user-mode trap handler setup in daisy-chain mode. */
#defineKFD_CWSR_TBA_TMA_SIZE(PAGE_SIZE* 2)Quoted 6 lines
Quoted 1 line
Could you please suggest the correct way to fix this issue?
I'm only looking from the POV of the VM code on this, but my educated guess is that
KFD_CWSR_TBA_TMA_SIZEshould be 8k independent of the CPU page size.Background is that this is written by the shader trap handler and that byte code doesn't care what CPU architecture you have.
But I think only the engineers working on that trap handler can really answer this. @Felix / @Philip?
Hi @christian @Felix @Philip
Quoted 7 lines
Quoted 6 lines
To remove the dependency on CPU page size, can we use
+#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << 16)
During reservation, we reserve 128 bytes, but during allocation, we use 2 *
PAGE_SIZE.We only need two GPU pages here. I think what Christian is objecting to is, that the GPU VM layout should not depend on the CPU page size.
Yes, exactly that was my concern.
Quoted 1 line
@Christian, it sounds like the BO allocations happen with 64KB granularity, but the mapping is still using 4KB granularity. Is the right solution to GPU-map only the first 8KB of the trap handler BO to keep the layout the same across CPU architectures?
Well that would work technically, but I agree that it also sounds a bit questionable as well.
Quoted 2 lines
I guess then the "correct" solution would be to change
amdgpu_amdkfd_gpuvm_alloc_memory_of_gpuandamdgpu_amdkfd_gpuvm_map_memory_to_gputo support mapping of the requested size with GPU page size granularity regardless of the CPU page size. But that would increase complexity for a very niche uses case.
Quoted 7 lines
Quoted 1 line
An easier solution would be to
PAGE_ALIGN8KB to the system page size. But that results in the virtual address space layout to depend on the system page size.Yeah, that dependency is certainly undesirable. We could easily end up with issues which can only be reproduced on systems with 64k page size.
Quoted 1 line
If that's objectionable, then the next best solution is to round up the trap handler size to 64KB byte unconditionally, so its the same with 4KB or 64KB system page size. But that would mean unnecessarily wasting a little memory per process/GPU on x86.
How about we always reserve 64KiB address space (or maybe even more, if you reserve 2MiB or 64KiB doesn't matter), but only map as large as the allocated buffer actually is?
I think that this would be my preferred solution.
Hi @Christian @Felix
Thanks for the review.
I have made the suggested change. I am now reserving 64 KB in the address space for the trap, while allocating only 8 KB for both 4K and 64K page sizes. With this change, I am no longer seeing crashes on either 4K or 64K systems.
Does this approach look reasonable to you?
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0 -0
@@ -173,7 +173,7 @@ struct amdgpu_bo_vm;
#define AMDGPU_VA_RESERVED_SEQ64_SIZE (2ULL << 20)
#define AMDGPU_VA_RESERVED_SEQ64_START(adev) (AMDGPU_VA_RESERVED_CSA_START(adev) \
- AMDGPU_VA_RESERVED_SEQ64_SIZE) -#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << 12)
+#define AMDGPU_VA_RESERVED_TRAP_SIZE (1ULL << 16)
#define AMDGPU_VA_RESERVED_TRAP_START(adev) (AMDGPU_VA_RESERVED_SEQ64_START(adev) \
- AMDGPU_VA_RESERVED_TRAP_SIZE) #define AMDGPU_VA_RESERVED_BOTTOM (1ULL << 16)
drivers/gpu/drm/amd/amdkfd/kfd_priv.h +0 -0
@@ -102,8 +102,8 @@
* The first chunk is the TBA used for the CWSR ISA code. The second
* chunk is used as TMA for user-mode trap handler setup in daisy-chain mode. */
-#define KFD_CWSR_TBA_TMA_SIZE (PAGE_SIZE * 2)
-#define KFD_CWSR_TMA_OFFSET (PAGE_SIZE + 2048)
+#define KFD_CWSR_TBA_TMA_SIZE (AMDGPU_GPU_PAGE_SIZE * 2) +#define KFD_CWSR_TMA_OFFSET (AMDGPU_GPU_PAGE_SIZE + 2048)
#define KFD_MAX_NUM_OF_QUEUES_PER_DEVICE \
(KFD_MAX_NUM_OF_PROCESSES *
Quoted 22 lines
Regards,
Christian.Quoted 18 lines
Regards,
FelixQuoted 14 lines
-Donet
Quoted 11 lines
Regards,
Christian.Quoted 8 lines
-Donet
Quoted 6 lines
Regards,
Christian.Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0
-0#define AMDGPU_VA_RESERVED_TRAP_START(adev) (AMDGPU_VA_RESERVED_SEQ64_START(adev) \ - AMDGPU_VA_RESERVED_TRAP_SIZE) #define AMDGPU_VA_RESERVED_BOTTOM (1ULL << 16)
Christian König
maintainerRe: [RESEND RFC PATCH v3 1/6] drm/amdgpu: Change AMDGPU_VA_RESERVED_TRAP_SIZE to 2 PAGE_SIZE pages
Quoted 143 lines · Donet Tom
Quoted 130 lines · Christian König
Quoted 115 lines · Felix
Quoted 114 lines · Donet Tom
Quoted 104 lines · Christian König
Quoted 103 lines · Donet Tom
On 3/23/26 3:41 PM, Christian König wrote:
Hi Christian
Quoted 81 lines · Donet Tom
Quoted 73 lines
Currently,
AMDGPU_VA_RESERVED_TRAP_SIZEis hardcoded to 8KB, whileKFD_CWSR_TBA_TMA_SIZEis defined as 2 *PAGE_SIZE. On systems with 4K pages, both values match (8KB), so allocation and reserved space are consistent.However, on 64K page-size systems,
KFD_CWSR_TBA_TMA_SIZEbecomes 128KB, while the reserved trap area remains 8KB. This mismatch causes the kernel to crash when running rocminfo or rccl unit tests.Kernel attempted to read user page (2) - exploit attempt? (uid: 1001)
BUG: Kernel NULL pointer dereference on read at 0x00000002 Faulting instruction address: 0xc0000000002c8a64 Oops: Kernel access of bad area, sig: 11 [#1] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA pSeries CPU: 34 UID: 1001 PID: 9379 Comm: rocminfo Tainted: G E 6.19.0-rc4-amdgpu-00320-gf23176405700 #56 VOLUNTARY Tainted: [E]=UNSIGNED_MODULE Hardware name: IBM,9105-42A POWER10 (architected) 0x800200 0xf000006of:IBM,FW1060.30 (
ML1060_896) hv:phyp pSeries
NIP:c0000000002c8a64LR:c00000000125dbc8CTR:c00000000125e730
REGS:c0000001e0957580TRAP: 0300 Tainted: G E
MSR: 8000000000009033 <SF,EE,ME,IR,DR,RI,LE> CR: 24008268
XER: 00000036
CFAR:c00000000125dbc4DAR: 0000000000000002 DSISR: 40000000
IRQMASK: 1
GPR00:c00000000125d908c0000001e0957820c0000000016e8100c00000013d814540
GPR04: 0000000000000002c00000013d8145500000000000000045 0000000000000000
GPR08:c00000013444d000c00000013d814538c00000013d8145380000000084002268
GPR12:c00000000125e730c000007e2ffd5f00ffffffffffffffff 0000000000020000
GPR16: 0000000000000000 0000000000000002c00000015f6530000000000000000000
GPR20:c000000138662400c00000013d8145400000000000000000c00000013d814500
GPR24: 0000000000000000 0000000000000002c0000001e0957888c0000001e0957878
GPR28:c00000013d8145480000000000000000c00000013d814540c0000001e0957888
NIP [c0000000002c8a64]__mutex_add_waiter+0x24/0xc0
LR [c00000000125dbc8] __mutex_lock.constprop.0+0x318/0xd00Call Trace: 0xc0000001e0957890 (unreliable) __mutex_lock.constprop.0+0x58/0xd00 amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu+0x6fc/0xb60 [amdgpu] kfd_process_alloc_gpuvm+0x54/0x1f0 [amdgpu] kfd_process_device_init_cwsr_dgpu+0xa4/0x1a0 [amdgpu] kfd_process_device_init_vm+0xd8/0x2e0 [amdgpu] kfd_ioctl_acquire_vm+0xd0/0x130 [amdgpu] kfd_ioctl+0x514/0x670 [amdgpu] sys_ioctl+0x134/0x180 system_call_exception+0x114/0x300 system_call_vectored_common+0x15c/0x2ecThis patch changes
AMDGPU_VA_RESERVED_TRAP_SIZEto 2 *PAGE_SIZE, ensuring that the reserved trap area matches the allocation size across all page sizes.Fixes 34a1de0f7935 ("drm/amdkfd: Relocate TBA/TMA to opposite side of VM hole")Reviewed-by Ritesh Harjani (IBM) <[email protected]>Signed-off-by Donet Tom <[email protected]>---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0
-0drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h struct amdgpu_bo_vm; · line 176
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +1
-1@@ -173,7 +173,7 @@ struct amdgpu_bo_vm; #define AMDGPU_VA_RESERVED_SEQ64_SIZE (2ULL << 20) #define AMDGPU_VA_RESERVED_SEQ64_START(adev) (AMDGPU_VA_RESERVED_CSA_START(adev) \ - AMDGPU_VA_RESERVED_SEQ64_SIZE) -#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << 12) +#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << PAGE_SHIFT)Well using
PAGE_SHIFTinamdgpu_vm.hlooks quite broken to me.Quoted 4 lines
That makes the GPU VA reservation depend on the CPU page size and that is clearly not something we want to have.
Where is
KFD_CWSR_TBA_TMA_SIZEdefined?Thanks Christian for reviewing this patch.
It is defined in
kfd_priv.h./*
* Size of the per-process TBA+TMA buffer: 2 pages
*
* The first chunk is the TBA used for the CWSR ISA code. The second
* chunk is used as TMA for user-mode trap handler setup in daisy-chain mode. */
#defineKFD_CWSR_TBA_TMA_SIZE(PAGE_SIZE* 2)Quoted 6 lines
Quoted 1 line
Could you please suggest the correct way to fix this issue?
I'm only looking from the POV of the VM code on this, but my educated guess is that
KFD_CWSR_TBA_TMA_SIZEshould be 8k independent of the CPU page size.Background is that this is written by the shader trap handler and that byte code doesn't care what CPU architecture you have.
But I think only the engineers working on that trap handler can really answer this. @Felix / @Philip?
Hi @christian @Felix @Philip
Quoted 7 lines
Quoted 6 lines
To remove the dependency on CPU page size, can we use
+#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << 16)
During reservation, we reserve 128 bytes, but during allocation, we use 2 *
PAGE_SIZE.We only need two GPU pages here. I think what Christian is objecting to is, that the GPU VM layout should not depend on the CPU page size.
Yes, exactly that was my concern.
Quoted 1 line
@Christian, it sounds like the BO allocations happen with 64KB granularity, but the mapping is still using 4KB granularity. Is the right solution to GPU-map only the first 8KB of the trap handler BO to keep the layout the same across CPU architectures?
Well that would work technically, but I agree that it also sounds a bit questionable as well.
Quoted 2 lines
I guess then the "correct" solution would be to change
amdgpu_amdkfd_gpuvm_alloc_memory_of_gpuandamdgpu_amdkfd_gpuvm_map_memory_to_gputo support mapping of the requested size with GPU page size granularity regardless of the CPU page size. But that would increase complexity for a very niche uses case.Quoted 7 lines
Quoted 1 line
An easier solution would be to
PAGE_ALIGN8KB to the system page size. But that results in the virtual address space layout to depend on the system page size.Yeah, that dependency is certainly undesirable. We could easily end up with issues which can only be reproduced on systems with 64k page size.
Quoted 1 line
If that's objectionable, then the next best solution is to round up the trap handler size to 64KB byte unconditionally, so its the same with 4KB or 64KB system page size. But that would mean unnecessarily wasting a little memory per process/GPU on x86.
How about we always reserve 64KiB address space (or maybe even more, if you reserve 2MiB or 64KiB doesn't matter), but only map as large as the allocated buffer actually is?
I think that this would be my preferred solution.
Hi @Christian @Felix
Quoted 8 lines
Thanks for the review.
I have made the suggested change. I am now reserving 64 KB in the address space for the trap, while allocating only 8 KB for both 4K and 64K page sizes. With this change, I am no longer seeing crashes on either 4K or 64K systems.
Does this approach look reasonable to you?
Looks correct to me, but Felix clearly has the last word on that.
Regards,
Christian.
Quoted 54 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +1
-1@@ -173,7 +173,7 @@ struct amdgpu_bo_vm; #define AMDGPU_VA_RESERVED_SEQ64_SIZE (2ULL << 20) #define AMDGPU_VA_RESERVED_SEQ64_START(adev) (AMDGPU_VA_RESERVED_CSA_START(adev) \ - AMDGPU_VA_RESERVED_SEQ64_SIZE) -#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << 12) +#define AMDGPU_VA_RESERVED_TRAP_SIZE (1ULL << 16) #define AMDGPU_VA_RESERVED_TRAP_START(adev) (AMDGPU_VA_RESERVED_SEQ64_START(adev) \ - AMDGPU_VA_RESERVED_TRAP_SIZE) #define AMDGPU_VA_RESERVED_BOTTOM (1ULL << 16)drivers/gpu/drm/amd/amdkfd/kfd_priv.h +2
-2@@ -102,8 +102,8 @@* The first chunk is the TBA used for the CWSR ISA code. The second * chunk is used as TMA for user-mode trap handler setup in daisy-chain mode. */ -#define KFD_CWSR_TBA_TMA_SIZE (PAGE_SIZE * 2) -#define KFD_CWSR_TMA_OFFSET (PAGE_SIZE + 2048) +#define KFD_CWSR_TBA_TMA_SIZE (AMDGPU_GPU_PAGE_SIZE * 2) +#define KFD_CWSR_TMA_OFFSET (AMDGPU_GPU_PAGE_SIZE + 2048) #define KFD_MAX_NUM_OF_QUEUES_PER_DEVICE \(
KFD_MAX_NUM_OF_PROCESSES*Quoted 22 lines
Regards,
Christian.Quoted 18 lines
Regards,
FelixQuoted 14 lines
-Donet
Quoted 11 lines
Regards,
Christian.Quoted 8 lines
-Donet
Quoted 6 lines
Regards,
Christian.Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0
-0#define AMDGPU_VA_RESERVED_TRAP_START(adev) (AMDGPU_VA_RESERVED_SEQ64_START(adev) \ - AMDGPU_VA_RESERVED_TRAP_SIZE) #define AMDGPU_VA_RESERVED_BOTTOM (1ULL << 16)
Kuehling, Felix
maintainerRe: [RESEND RFC PATCH v3 1/6] drm/amdgpu: Change AMDGPU_VA_RESERVED_TRAP_SIZE to 2 PAGE_SIZE pages
Quoted 11 lines · Christian König
Quoted 10 lines
Hi @Christian @Felix
Thanks for the review.
I have made the suggested change. I am now reserving 64 KB in the address space for the trap, while allocating only 8 KB for both 4K and 64K page sizes. With this change, I am no longer seeing crashes on either 4K or 64K systems.
Does this approach look reasonable to you?
Looks correct to me, but Felix clearly has the last word on that.
That works for me as well.
Thanks,
Felix
Quoted 32 lines
Regards,
Christian.Quoted 28 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +1
-1@@ -173,7 +173,7 @@ struct amdgpu_bo_vm; #define AMDGPU_VA_RESERVED_SEQ64_SIZE (2ULL << 20) #define AMDGPU_VA_RESERVED_SEQ64_START(adev) (AMDGPU_VA_RESERVED_CSA_START(adev) \ - AMDGPU_VA_RESERVED_SEQ64_SIZE) -#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << 12) +#define AMDGPU_VA_RESERVED_TRAP_SIZE (1ULL << 16) #define AMDGPU_VA_RESERVED_TRAP_START(adev) (AMDGPU_VA_RESERVED_SEQ64_START(adev) \ - AMDGPU_VA_RESERVED_TRAP_SIZE) #define AMDGPU_VA_RESERVED_BOTTOM (1ULL << 16)drivers/gpu/drm/amd/amdkfd/kfd_priv.h +2
-2@@ -102,8 +102,8 @@* The first chunk is the TBA used for the CWSR ISA code. The second * chunk is used as TMA for user-mode trap handler setup in daisy-chain mode. */ -#define KFD_CWSR_TBA_TMA_SIZE (PAGE_SIZE * 2) -#define KFD_CWSR_TMA_OFFSET (PAGE_SIZE + 2048) +#define KFD_CWSR_TBA_TMA_SIZE (AMDGPU_GPU_PAGE_SIZE * 2) +#define KFD_CWSR_TMA_OFFSET (AMDGPU_GPU_PAGE_SIZE + 2048) #define KFD_MAX_NUM_OF_QUEUES_PER_DEVICE \ (KFD_MAX_NUM_OF_PROCESSES *
Re: [RESEND RFC PATCH v3 1/6] drm/amdgpu: Change AMDGPU_VA_RESERVED_TRAP_SIZE to 2 PAGE_SIZE pages
Quoted 11 lines · Felix
Quoted 9 lines · Christian König
Quoted 9 lines
Hi @Christian @Felix
Thanks for the review.
I have made the suggested change. I am now reserving 64 KB in the address space for the trap, while allocating only 8 KB for both 4K and 64K page sizes. With this change, I am no longer seeing crashes on either 4K or 64K systems.
Quoted 4 lines
Quoted 2 lines
Quoted 1 line
Does this approach look reasonable to you?
Looks correct to me, but Felix clearly has the last word on that.
That works for me as well.
Thank you , Felix. I will incorporate this change and post an updated version.
-Donet
Quoted 37 lines
Thanks,
FelixQuoted 32 lines
Regards,
Christian.Quoted 28 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +1
-1@@ -173,7 +173,7 @@ struct amdgpu_bo_vm; #define AMDGPU_VA_RESERVED_SEQ64_SIZE (2ULL << 20) #define AMDGPU_VA_RESERVED_SEQ64_START(adev) (AMDGPU_VA_RESERVED_CSA_START(adev) \ - AMDGPU_VA_RESERVED_SEQ64_SIZE) -#define AMDGPU_VA_RESERVED_TRAP_SIZE (2ULL << 12) +#define AMDGPU_VA_RESERVED_TRAP_SIZE (1ULL << 16) #define AMDGPU_VA_RESERVED_TRAP_START(adev) (AMDGPU_VA_RESERVED_SEQ64_START(adev) \ - AMDGPU_VA_RESERVED_TRAP_SIZE) #define AMDGPU_VA_RESERVED_BOTTOM (1ULL << 16)drivers/gpu/drm/amd/amdkfd/kfd_priv.h +2
-2@@ -102,8 +102,8 @@* The first chunk is the TBA used for the CWSR ISA code. The second * chunk is used as TMA for user-mode trap handler setup in daisy-chain mode. */ -#define KFD_CWSR_TBA_TMA_SIZE (PAGE_SIZE * 2) -#define KFD_CWSR_TMA_OFFSET (PAGE_SIZE + 2048) +#define KFD_CWSR_TBA_TMA_SIZE (AMDGPU_GPU_PAGE_SIZE * 2) +#define KFD_CWSR_TMA_OFFSET (AMDGPU_GPU_PAGE_SIZE + 2048) #define KFD_MAX_NUM_OF_QUEUES_PER_DEVICE \(
KFD_MAX_NUM_OF_PROCESSES*
RFC PATCH v3 2/6 drm/amdkfd: Align expected_queue_size to PAGE_SIZE
In reply to Donet TomThe AQL queue size can be 4K, but the minimum buffer object (BO) allocation size is PAGE_SIZE. On systems with a page size larger than 4K, the expected queue size does not match the allocated BO size, causing queue creation to fail.
Align the expected queue size to PAGE_SIZE so that it matches the allocated BO size and allows queue creation to succeed.
--- drivers/gpu/drm/amd/amdkfd/kfd_queue.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_queue.c +2 -2
@@ -249,10 +249,10 @@ int kfd_queue_acquire_buffers(struct kfd_process_device *pdd, struct queue_prope topo_dev->node_props.gfx_target_version < 90000)/* metadata_queue_size not supported on GFX7/GFX8 */ expected_queue_size = - properties->queue_size /2 ; +PAGE_ALIGN (properties->queue_size /2 ); else expected_queue_size = - properties->queue_size + properties->metadata_queue_size; +PAGE_ALIGN (properties->queue_size + properties->metadata_queue_size); vm =drm_priv_to_vm (pdd->drm_priv); err =amdgpu_bo_reserve (vm->root.bo, false);
Signature · 1 line
2.52.0
Re: [RESEND RFC PATCH v3 2/6] drm/amdkfd: Align expected_queue_size to PAGE_SIZE
Quoted 9 lines · Donet Tom
The AQL queue size can be 4K, but the minimum buffer object (BO) allocation size is
PAGE_SIZE. On systems with a page size larger than 4K, the expected queue size does not match the allocated BO size, causing queue creation to fail.Align the expected queue size to
PAGE_SIZEso that it matches the allocated BO size and allows queue creation to succeed.Signed-off-by Donet Tom <[email protected]>
Quoted 21 lines
--- drivers/gpu/drm/amd/amdkfd/kfd_queue.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/gpu/drm/amd/amdkfd/kfd_queue.c +2
-2@@ -249,10 +249,10 @@ int kfd_queue_acquire_buffers(struct kfd_process_device *pdd, struct queue_prope topo_dev->node_props.gfx_target_version < 90000)/* metadata_queue_size not supported on GFX7/GFX8 */ expected_queue_size = - properties->queue_size /2 ; +PAGE_ALIGN (properties->queue_size /2 ); else expected_queue_size = - properties->queue_size + properties->metadata_queue_size; +PAGE_ALIGN (properties->queue_size + properties->metadata_queue_size); vm =drm_priv_to_vm (pdd->drm_priv); err =amdgpu_bo_reserve (vm->root.bo, false);
Alex Deucher
AppliedRe: [RESEND RFC PATCH v3 2/6] drm/amdkfd: Align expected_queue_size to PAGE_SIZE
Applied. Thanks!
Quoted 36 lines · Felix
Quoted 9 lines · Donet Tom
The AQL queue size can be 4K, but the minimum buffer object (BO) allocation size is
PAGE_SIZE. On systems with a page size larger than 4K, the expected queue size does not match the allocated BO size, causing queue creation to fail.Align the expected queue size to
PAGE_SIZEso that it matches the allocated BO size and allows queue creation to succeed.Signed-off-by Donet Tom <[email protected]>Reviewed-by Felix Kuehling <[email protected]>Quoted 21 lines
--- drivers/gpu/drm/amd/amdkfd/kfd_queue.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/gpu/drm/amd/amdkfd/kfd_queue.c +2
-2@@ -249,10 +249,10 @@ int kfd_queue_acquire_buffers(struct kfd_process_device *pdd, struct queue_prope topo_dev->node_props.gfx_target_version < 90000)/* metadata_queue_size not supported on GFX7/GFX8 */ expected_queue_size = - properties->queue_size /2 ; +PAGE_ALIGN (properties->queue_size /2 ); else expected_queue_size = - properties->queue_size + properties->metadata_queue_size; +PAGE_ALIGN (properties->queue_size + properties->metadata_queue_size); vm =drm_priv_to_vm (pdd->drm_priv); err =amdgpu_bo_reserve (vm->root.bo, false);
RFC PATCH v3 3/6 drm/amdgpu: Handle GPU page faults correctly on non-4K page systems
In reply to Donet TomDuring a GPU page fault, the driver restores the SVM range and then maps it into the GPU page tables. The current implementation passes a GPU-page-size (4K-based) PFN to svm_range_restore_pages() to restore the range.
SVM ranges are tracked using system-page-size PFNs. On systems where the system page size is larger than 4K, using GPU-page-size PFNs to restore the range causes two problems:
Range lookup fails:
Because the restore function receives PFNs in GPU (4K) units, the SVM range lookup does not find the existing range. This will result in a duplicate SVM range being created.
VMA lookup failure:
The restore function also tries to locate the VMA for the faulting address. It converts the GPU-page-size PFN into an address using the system page size, which results in an incorrect address on non-4K page-size systems. As a result, the VMA lookup fails with the message: "address 0xxxx VMA is removed".
This patch passes the system-page-size PFN to svm_range_restore_pages() so that the SVM range is restored correctly on non-4K page systems.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +3 -3
@@ -2985,14 +2985,14 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,if (!root)return false; - addr /=AMDGPU_GPU_PAGE_SIZE ; -if (is_compute_context && !svm_range_restore_pages (adev, pasid, vmid, - node_id, addr, ts, write_fault)) { + node_id, addr >>PAGE_SHIFT , ts, write_fault)) {amdgpu_bo_unref (&root);return true; } + addr /=AMDGPU_GPU_PAGE_SIZE ; + r =amdgpu_bo_reserve (root, true);if (r) goto error_unref;
Signature · 1 line
2.52.0
Re: [RESEND RFC PATCH v3 3/6] drm/amdgpu: Handle GPU page faults correctly on non-4K page systems
Quoted 20 lines · Donet Tom
During a GPU page fault, the driver restores the SVM range and then maps it into the GPU page tables. The current implementation passes a GPU-page-size (4K-based) PFN to
svm_range_restore_pages()to restore the range.SVM ranges are tracked using system-page-size PFNs. On systems where the system page size is larger than 4K, using GPU-page-size PFNs to restore the range causes two problems:
Range lookup fails:
Because the restore function receives PFNs in GPU (4K) units, the SVM range lookup does not find the existing range. This will result in a duplicate SVM range being created.VMA lookup failure:
The restore function also tries to locate the VMA for the faulting address. It converts the GPU-page-size PFN into an address using the system page size, which results in an incorrect address on non-4K page-size systems. As a result, the VMA lookup fails with the message: "address 0xxxx VMA is removed".
Quoted 4 lines
This patch passes the system-page-size PFN to
svm_range_restore_pages()so that the SVM range is restored correctly on non-4K page systems.Signed-off-by Donet Tom <[email protected]>
Quoted 26 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +3
-3@@ -2985,14 +2985,14 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,if (!root)return false; - addr /=AMDGPU_GPU_PAGE_SIZE ; -if (is_compute_context && !svm_range_restore_pages (adev, pasid, vmid, - node_id, addr, ts, write_fault)) { + node_id, addr >>PAGE_SHIFT , ts, write_fault)) {amdgpu_bo_unref (&root);return true; } + addr /=AMDGPU_GPU_PAGE_SIZE ; + r =amdgpu_bo_reserve (root, true);if (r) goto error_unref;
Alex Deucher
AppliedRe: [RESEND RFC PATCH v3 3/6] drm/amdgpu: Handle GPU page faults correctly on non-4K page systems
Applied. Thanks!
Alex
Quoted 56 lines · Christian König
Quoted 20 lines · Donet Tom
During a GPU page fault, the driver restores the SVM range and then maps it into the GPU page tables. The current implementation passes a GPU-page-size (4K-based) PFN to
svm_range_restore_pages()to restore the range.SVM ranges are tracked using system-page-size PFNs. On systems where the system page size is larger than 4K, using GPU-page-size PFNs to restore the range causes two problems:
Range lookup fails:
Because the restore function receives PFNs in GPU (4K) units, the SVM range lookup does not find the existing range. This will result in a duplicate SVM range being created.VMA lookup failure:
The restore function also tries to locate the VMA for the faulting address. It converts the GPU-page-size PFN into an address using the system page size, which results in an incorrect address on non-4K page-size systems. As a result, the VMA lookup fails with the message: "address 0xxxx VMA is removed".Quoted 4 lines
This patch passes the system-page-size PFN to
svm_range_restore_pages()so that the SVM range is restored correctly on non-4K page systems.Signed-off-by Donet Tom <[email protected]>Acked-by Christian König <[email protected]>Quoted 26 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +3
-3@@ -2985,14 +2985,14 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,if (!root)return false; - addr /=AMDGPU_GPU_PAGE_SIZE ; -if (is_compute_context && !svm_range_restore_pages (adev, pasid, vmid, - node_id, addr, ts, write_fault)) { + node_id, addr >>PAGE_SHIFT , ts, write_fault)) {amdgpu_bo_unref (&root);return true; } + addr /=AMDGPU_GPU_PAGE_SIZE ; + r =amdgpu_bo_reserve (root, true);if (r) goto error_unref;
Re: [RESEND RFC PATCH v3 3/6] drm/amdgpu: Handle GPU page faults correctly on non-4K page systems
Quoted 1 line · Alex Deucher
Applied. Thanks!
Hi @Alex
Thank you for applying this patch.
I am planning to send the next version for PATCH 1/6. For the other patches that have already received Reviewed-by tags, would you prefer to pick them from this series, or should I include them again in the next version?
-Donet
Quoted 58 lines
Alex
Quoted 53 lines · Christian König
Quoted 20 lines · Donet Tom
During a GPU page fault, the driver restores the SVM range and then maps it into the GPU page tables. The current implementation passes a GPU-page-size (4K-based) PFN to
svm_range_restore_pages()to restore the range.SVM ranges are tracked using system-page-size PFNs. On systems where the system page size is larger than 4K, using GPU-page-size PFNs to restore the range causes two problems:
Range lookup fails:
Because the restore function receives PFNs in GPU (4K) units, the SVM range lookup does not find the existing range. This will result in a duplicate SVM range being created.VMA lookup failure:
The restore function also tries to locate the VMA for the faulting address. It converts the GPU-page-size PFN into an address using the system page size, which results in an incorrect address on non-4K page-size systems. As a result, the VMA lookup fails with the message: "address 0xxxx VMA is removed".Quoted 4 lines
This patch passes the system-page-size PFN to
svm_range_restore_pages()so that the SVM range is restored correctly on non-4K page systems.Signed-off-by Donet Tom <[email protected]>Acked-by Christian König <[email protected]>Quoted 26 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +3
-3@@ -2985,14 +2985,14 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,if (!root)return false; - addr /=AMDGPU_GPU_PAGE_SIZE ; -if (is_compute_context && !svm_range_restore_pages (adev, pasid, vmid, - node_id, addr, ts, write_fault)) { + node_id, addr >>PAGE_SHIFT , ts, write_fault)) {amdgpu_bo_unref (&root);return true; } + addr /=AMDGPU_GPU_PAGE_SIZE ; + r =amdgpu_bo_reserve (root, true);if (r) goto error_unref;
Re: [RESEND RFC PATCH v3 3/6] drm/amdgpu: Handle GPU page faults correctly on non-4K page systems
Quoted 7 lines · Donet Tom
Quoted 1 line · Alex Deucher
Applied. Thanks!
Hi @Alex
Quoted 7 lines
Thank you for applying this patch.
I am planning to send the next version for PATCH 1/6. For the other patches that have already received Reviewed-by tags, would you prefer to pick them from this series, or should I include them again in the next version?
I'll pick up the reviewed patches. Feel free to include them in your resend if that's easier for you. I'll pick up whatever the delta is once those are reviewed.
Alex
Quoted 62 lines
-Donet
Quoted 58 lines
Alex
Quoted 53 lines · Christian König
Quoted 20 lines · Donet Tom
During a GPU page fault, the driver restores the SVM range and then maps it into the GPU page tables. The current implementation passes a GPU-page-size (4K-based) PFN to
svm_range_restore_pages()to restore the range.SVM ranges are tracked using system-page-size PFNs. On systems where the system page size is larger than 4K, using GPU-page-size PFNs to restore the range causes two problems:
Range lookup fails:
Because the restore function receives PFNs in GPU (4K) units, the SVM range lookup does not find the existing range. This will result in a duplicate SVM range being created.VMA lookup failure:
The restore function also tries to locate the VMA for the faulting address. It converts the GPU-page-size PFN into an address using the system page size, which results in an incorrect address on non-4K page-size systems. As a result, the VMA lookup fails with the message: "address 0xxxx VMA is removed".Quoted 4 lines
This patch passes the system-page-size PFN to
svm_range_restore_pages()so that the SVM range is restored correctly on non-4K page systems.Signed-off-by Donet Tom <[email protected]>Acked-by Christian König <[email protected]>Quoted 26 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +3
-3@@ -2985,14 +2985,14 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,if (!root)return false; - addr /=AMDGPU_GPU_PAGE_SIZE ; -if (is_compute_context && !svm_range_restore_pages (adev, pasid, vmid, - node_id, addr, ts, write_fault)) { + node_id, addr >>PAGE_SHIFT , ts, write_fault)) {amdgpu_bo_unref (&root);return true; } + addr /=AMDGPU_GPU_PAGE_SIZE ; + r =amdgpu_bo_reserve (root, true);if (r) goto error_unref;
Re: [RESEND RFC PATCH v3 3/6] drm/amdgpu: Handle GPU page faults correctly on non-4K page systems
In reply to Donet TomQuoted 20 lines · Donet Tom
During a GPU page fault, the driver restores the SVM range and then maps it into the GPU page tables. The current implementation passes a GPU-page-size (4K-based) PFN to
svm_range_restore_pages()to restore the range.SVM ranges are tracked using system-page-size PFNs. On systems where the system page size is larger than 4K, using GPU-page-size PFNs to restore the range causes two problems:
Range lookup fails:
Because the restore function receives PFNs in GPU (4K) units, the SVM range lookup does not find the existing range. This will result in a duplicate SVM range being created.VMA lookup failure:
The restore function also tries to locate the VMA for the faulting address. It converts the GPU-page-size PFN into an address using the system page size, which results in an incorrect address on non-4K page-size systems. As a result, the VMA lookup fails with the message: "address 0xxxx VMA is removed".
Quoted 4 lines
This patch passes the system-page-size PFN to
svm_range_restore_pages()so that the SVM range is restored correctly on non-4K page systems.Signed-off-by Donet Tom <[email protected]>
Quoted 26 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +3
-3@@ -2985,14 +2985,14 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,if (!root)return false; - addr /=AMDGPU_GPU_PAGE_SIZE ; -if (is_compute_context && !svm_range_restore_pages (adev, pasid, vmid, - node_id, addr, ts, write_fault)) { + node_id, addr >>PAGE_SHIFT , ts, write_fault)) {amdgpu_bo_unref (&root);return true; } + addr /=AMDGPU_GPU_PAGE_SIZE ; + r =amdgpu_bo_reserve (root, true);if (r) goto error_unref;
RFC PATCH v3 4/6 drm/amdgpu: Fix AMDGPU_GTT_MAX_TRANSFER_SIZE for non-4K page size
In reply to Donet TomAMDGPU_GTT_MAX_TRANSFER_SIZE represented the maximum number of system-page-sized pages that could be transferred in a single operation. The effective maximum transfer size was intended to be one PMD-sized mapping.
In the existing code, AMDGPU_GTT_MAX_TRANSFER_SIZE was hard-coded to 512 pages. This corresponded to 2 MB on 4 KB page-size systems, matching the PMD size. However, on systems with a non-4 KB page size, this value no longer matched the PMD size.
This patch changed the calculation of AMDGPU_GTT_MAX_TRANSFER_SIZE to derive it from PMD_SHIFT and PAGE_SHIFT, ensuring that the maximum transfer size remained PMD-sized across all system page sizes.
Additionally, in some places, AMDGPU_GTT_MAX_TRANSFER_SIZE was implicitly assumed to be based on 4 KB pages. This resulted in incorrect address offset calculations. This patch updated the address calculations to correctly handle non-4 KB system page sizes as well.
amdgpu_ttm_map_buffer() can create both GTT GART entries and VRAM GART entries. For GTT mappings, amdgpu_gart_map() takes system page–sized PFNs, and the mappings are created correctly.
However, for VRAM GART mappings, amdgpu_gart_map_vram_range() expects GPU page–sized PFNs, but CPU page–sized PFNs were being passed, resulting in incorrect mappings.
This patch updates the code to pass GPU page–sized PFNs to amdgpu_gart_map_vram_range(), ensuring that VRAM GART mappings are created correctly.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 8 +++++--- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h | 2 +- drivers/gpu/drm/amd/amdgpu/vce_v1_0.c | 3 ++- 3 files changed, 8 insertions(+), 5 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +5 -3
@@ -204,7 +204,7 @@ static int amdgpu_ttm_map_buffer(struct amdgpu_ttm_buffer_entity *entity,int r;BUG_ON (adev->mman.buffer_funcs->copy_max_bytes < -AMDGPU_GTT_MAX_TRANSFER_SIZE *8 ); +AMDGPU_GTT_MAX_TRANSFER_SIZE *AMDGPU_GPU_PAGES_IN_CPU_PAGE *8 );if (WARN_ON (mem->mem_type ==AMDGPU_PL_PREEMPT ))return -EINVAL ;
@@ -230,7 +230,7 @@ static int amdgpu_ttm_map_buffer(struct amdgpu_ttm_buffer_entity *entity, *addr = adev->gmc.gart_start; *addr += (u64 )window *AMDGPU_GTT_MAX_TRANSFER_SIZE * -AMDGPU_GPU_PAGE_SIZE ; +AMDGPU_GPU_PAGES_IN_CPU_PAGE *AMDGPU_GPU_PAGE_SIZE ; *addr += offset; num_dw =ALIGN (adev->mman.buffer_funcs->copy_num_dw,8 );
@@ -248,7 +248,8 @@ static int amdgpu_ttm_map_buffer(struct amdgpu_ttm_buffer_entity *entity, src_addr += job->ibs[0 ].gpu_addr; dst_addr =amdgpu_bo_gpu_offset (adev->gart.bo); - dst_addr += window *AMDGPU_GTT_MAX_TRANSFER_SIZE *8 ; + dst_addr += window *AMDGPU_GTT_MAX_TRANSFER_SIZE * +AMDGPU_GPU_PAGES_IN_CPU_PAGE *8 ;amdgpu_emit_copy_buffer (adev, &job->ibs[0 ], src_addr, dst_addr, num_bytes,0 );
@@ -266,6 +267,7 @@ static int amdgpu_ttm_map_buffer(struct amdgpu_ttm_buffer_entity *entity, }else {u64 pa = mm_cur->start + adev->vm_manager.vram_base_offset; + num_pages *=AMDGPU_GPU_PAGES_IN_CPU_PAGE ;amdgpu_gart_map_vram_range (adev, pa,0 , num_pages, flags, cpu_addr); }
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h +1 -1
@@ -38,7 +38,7 @@#define AMDGPU_PL_MMIO_REMAP (TTM_PL_PRIV + 5)#define __AMDGPU_PL_NUM (TTM_PL_PRIV + 6) -#define AMDGPU_GTT_MAX_TRANSFER_SIZE 512 +#define AMDGPU_GTT_MAX_TRANSFER_SIZE (1 << (PMD_SHIFT - PAGE_SHIFT))#define AMDGPU_GTT_NUM_TRANSFER_WINDOWS 2extern const struct attribute_group amdgpu_vram_mgr_attr_group;
drivers/gpu/drm/amd/amdgpu/vce_v1_0.c +2 -1
@@ -48,7 +48,8 @@#define VCE_STATUS_VCPU_REPORT_FW_LOADED_MASK 0x02#define VCE_V1_0_GART_PAGE_START \ - (AMDGPU_GTT_MAX_TRANSFER_SIZE * AMDGPU_GTT_NUM_TRANSFER_WINDOWS) + (AMDGPU_GTT_MAX_TRANSFER_SIZE * AMDGPU_GPU_PAGES_IN_CPU_PAGE * \ + AMDGPU_GTT_NUM_TRANSFER_WINDOWS)#define VCE_V1_0_GART_ADDR_START \ (VCE_V1_0_GART_PAGE_START * AMDGPU_GPU_PAGE_SIZE)
Signature · 1 line
2.52.0
RFC PATCH v3 5/6 drm/amd: Fix MQD and control stack alignment for non-4K
In reply to Donet TomFor gfxV9, due to a hardware bug ("based on the comments in the code here [1]"), the control stack of a user-mode compute queue must be allocated immediately after the page boundary of its regular MQD buffer. To handle this, we allocate an enlarged MQD buffer where the first page is used as the MQD and the remaining pages store the control stack. Although these regions share the same BO, they require different memory
types: the MQD must be UC (uncached), while the control stack must be NC (non-coherent), matching the behavior when the control stack is allocated in user space.
This logic works correctly on systems where the CPU page size matches the GPU page size (4K). However, the current implementation aligns both the MQD and the control stack to the CPU PAGE_SIZE. On systems with a larger CPU page size, the entire first CPU page is marked UC—even though that page may contain multiple GPU pages. The GPU treats the second 4K GPU page inside that CPU page as part of the control stack, but it is incorrectly mapped as UC.
This patch fixes the issue by aligning both the MQD and control stack sizes to the GPU page size (4K). The first 4K page is correctly marked as UC for the MQD, and the remaining GPU pages are marked NC for the control stack. This ensures proper memory type assignment on systems with larger CPU page sizes.
[1]: https://elixir.bootlin.com/linux/v6.18/source/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c#L118
--- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 44 +++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 16 ++----- .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c | 23 ++++++---- 4 files changed, 64 insertions(+), 21 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +44 -0
@@ -403,6 +403,50 @@ void amdgpu_gart_map_vram_range(struct amdgpu_device *adev, uint64_t pa,drm_dev_exit (idx); } +/** +* amdgpu_gart_map_gfx9_mqd - map mqd and ctrl_stack dma_addresses into GART entries +* +* @adev: amdgpu_device pointer +* @offset: offset into the GPU's gart aperture +* @pages: number of pages to bind +* @dma_addr: DMA addresses of pages +* @flags: page table entry flags +* +* Map the MQD and control stack addresses into GART entries with the correct +* memory types on gfxv9. The MQD occupies the first 4KB and is followed by +* the control stack. The MQD uses UC (uncached) memory, while the control stack +* uses NC (non-coherent) memory. +*/ +void amdgpu_gart_map_gfx9_mqd (struct amdgpu_device *adev,uint64_t offset, +int pages,dma_addr_t *dma_addr,uint64_t flags) +{ +uint64_t page_base; +unsigned int i, j, t; +int idx; +uint64_t ctrl_flags =AMDGPU_PTE_MTYPE_VG10 (flags,AMDGPU_MTYPE_NC ); +void *dst; + +if (!adev->gart.ptr) +return ; + +if (!drm_dev_enter (adev_to_drm (adev), &idx)) +return ; + + t = offset /AMDGPU_GPU_PAGE_SIZE ; + dst = adev->gart.ptr; +for (i =0 ; i < pages; i++) { + page_base = dma_addr[i]; +for (j =0 ; j <AMDGPU_GPU_PAGES_IN_CPU_PAGE ; j++, t++) { +if ((i ==0 ) && (j ==0 )) +amdgpu_gmc_set_pte_pde (adev, dst, t, page_base, flags); +else +amdgpu_gmc_set_pte_pde (adev, dst, t, page_base, ctrl_flags); + page_base +=AMDGPU_GPU_PAGE_SIZE ; + } + } +drm_dev_exit (idx); +} +/** * amdgpu_gart_bind - bind pages into the gart page table *
drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h +2 -0
@@ -62,6 +62,8 @@ void amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,void amdgpu_gart_map (struct amdgpu_device *adev,uint64_t offset,int pages,dma_addr_t *dma_addr,uint64_t flags,void *dst); +void amdgpu_gart_map_gfx9_mqd (struct amdgpu_device *adev,uint64_t offset, +int pages,dma_addr_t *dma_addr,uint64_t flags);void amdgpu_gart_bind (struct amdgpu_device *adev,uint64_t offset,int pages,dma_addr_t *dma_addr,uint64_t flags); void amdgpu_gart_map_vram_range(struct amdgpu_device *adev, uint64_t pa,
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +3 -13
@@ -855,25 +855,15 @@ static void amdgpu_ttm_gart_bind_gfx9_mqd(struct amdgpu_device *adev,int num_xcc =max (1U , adev->gfx.num_xcc_per_xcp);uint64_t page_idx, pages_per_xcc;int i; -uint64_t ctrl_flags =AMDGPU_PTE_MTYPE_VG10 (flags,AMDGPU_MTYPE_NC ); pages_per_xcc = total_pages;do_div (pages_per_xcc, num_xcc);for (i =0 , page_idx =0 ; i < num_xcc; i++, page_idx += pages_per_xcc) { -/* MQD page: use default flags */ -amdgpu_gart_bind (adev, +amdgpu_gart_map_gfx9_mqd (adev, gtt->offset + (page_idx <<PAGE_SHIFT ), -1 , >t->ttm.dma_address[page_idx], flags); -/* -* Ctrl pages - modify the memory type to NC (ctrl_flags) from -* the second page of the BO onward. -*/ -amdgpu_gart_bind (adev, - gtt->offset + ((page_idx +1 ) <<PAGE_SHIFT ), - pages_per_xcc -1 , - >t->ttm.dma_address[page_idx +1 ], - ctrl_flags); + pages_per_xcc, >t->ttm.dma_address[page_idx], + flags); } }
drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c +15 -8
@@ -42,9 +42,16 @@ static uint64_t mqd_stride_v9(struct mqd_manager *mm, struct queue_properties *q) {if (mm->dev->kfd->cwsr_enabled && - q->type ==KFD_QUEUE_TYPE_COMPUTE ) -return ALIGN (q->ctl_stack_size,PAGE_SIZE ) + -ALIGN (sizeof (struct v9_mqd ),PAGE_SIZE ); + q->type ==KFD_QUEUE_TYPE_COMPUTE ) { + +/* On gfxv9, the MQD resides in the first 4K page, +* followed by the control stack. Align both to +* AMDGPU_GPU_PAGE_SIZE to maintain the required 4K boundary. +*/ + +return ALIGN (ALIGN (q->ctl_stack_size,AMDGPU_GPU_PAGE_SIZE ) + +ALIGN (sizeof (struct v9_mqd ),AMDGPU_GPU_PAGE_SIZE ),PAGE_SIZE ); + }return mm->mqd_size; }
@@ -148,8 +155,8 @@ static struct kfd_mem_obj *allocate_mqd(struct mqd_manager *mm,if (!mqd_mem_obj)return NULL ; retval = amdgpu_amdkfd_alloc_kernel_mem(node->adev, - (ALIGN(q->ctl_stack_size, PAGE_SIZE) + - ALIGN(sizeof(struct v9_mqd), PAGE_SIZE)) * + (ALIGN(ALIGN(q->ctl_stack_size, AMDGPU_GPU_PAGE_SIZE) + + ALIGN(sizeof(struct v9_mqd), AMDGPU_GPU_PAGE_SIZE), PAGE_SIZE)) * NUM_XCC(node->xcc_mask), mqd_on_vram(node->adev) ? AMDGPU_GEM_DOMAIN_VRAM : AMDGPU_GEM_DOMAIN_GTT,
@@ -357,7 +364,7 @@ static int get_wave_state(struct mqd_manager *mm, void *mqd,struct kfd_context_save_area_header header;/* Control stack is located one page after MQD. */ -void *mqd_ctl_stack = (void *)((uintptr_t )mqd +PAGE_SIZE ); +void *mqd_ctl_stack = (void *)((uintptr_t )mqd +AMDGPU_GPU_PAGE_SIZE ); m =get_mqd (mqd);
@@ -394,7 +401,7 @@ static void checkpoint_mqd(struct mqd_manager *mm, void *mqd, void *mqd_dst, voi {struct v9_mqd *m;/* Control stack is located one page after MQD. */ -void *ctl_stack = (void *)((uintptr_t )mqd +PAGE_SIZE ); +void *ctl_stack = (void *)((uintptr_t )mqd +AMDGPU_GPU_PAGE_SIZE ); m =get_mqd (mqd);
@@ -440,7 +447,7 @@ static void restore_mqd(struct mqd_manager *mm, void **mqd, *gart_addr = addr;/* Control stack is located one page after MQD. */ - ctl_stack = (void *)((uintptr_t )*mqd +PAGE_SIZE ); + ctl_stack = (void *)((uintptr_t )*mqd +AMDGPU_GPU_PAGE_SIZE );memcpy (ctl_stack, ctl_stack_src, ctl_stack_size); m->cp_hqd_pq_doorbell_control =
Signature · 1 line
2.52.0
Re: [RESEND RFC PATCH v3 5/6] drm/amd: Fix MQD and control stack alignment for non-4K
Quoted 24 lines · Donet Tom
For gfxV9, due to a hardware bug ("based on the comments in the code here [1]"), the control stack of a user-mode compute queue must be allocated immediately after the page boundary of its regular MQD buffer. To handle this, we allocate an enlarged MQD buffer where the first page is used as the MQD and the remaining pages store the control stack. Although these regions share the same BO, they require different memory
types: the MQD must be UC (uncached), while the control stack must be NC (non-coherent), matching the behavior when the control stack is allocated in user space.This logic works correctly on systems where the CPU page size matches the GPU page size (4K). However, the current implementation aligns both the MQD and the control stack to the CPU
PAGE_SIZE. On systems with a larger CPU page size, the entire first CPU page is marked UC—even though that page may contain multiple GPU pages. The GPU treats the second 4K GPU page inside that CPU page as part of the control stack, but it is incorrectly mapped as UC.This patch fixes the issue by aligning both the MQD and control stack sizes to the GPU page size (4K). The first 4K page is correctly marked as UC for the MQD, and the remaining GPU pages are marked NC for the control stack. This ensures proper memory type assignment on systems with larger CPU page sizes.
Quoted 3 lines
[1]: https://elixir.bootlin.com/linux/v6.18/source/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c#L118
Signed-off-by Donet Tom <[email protected]>
Quoted 170 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 44 +++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 16 ++----- .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c | 23 ++++++---- 4 files changed, 64 insertions(+), 21 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +44
-0@@ -403,6 +403,50 @@ void amdgpu_gart_map_vram_range(struct amdgpu_device *adev, uint64_t pa,drm_dev_exit (idx); } +/** +* amdgpu_gart_map_gfx9_mqd - map mqd and ctrl_stack dma_addresses into GART entries +* +* @adev: amdgpu_device pointer +* @offset: offset into the GPU's gart aperture +* @pages: number of pages to bind +* @dma_addr: DMA addresses of pages +* @flags: page table entry flags +* +* Map the MQD and control stack addresses into GART entries with the correct +* memory types on gfxv9. The MQD occupies the first 4KB and is followed by +* the control stack. The MQD uses UC (uncached) memory, while the control stack +* uses NC (non-coherent) memory. +*/ +void amdgpu_gart_map_gfx9_mqd (struct amdgpu_device *adev,uint64_t offset, +int pages,dma_addr_t *dma_addr,uint64_t flags) +{ +uint64_t page_base; +unsigned int i, j, t; +int idx; +uint64_t ctrl_flags =AMDGPU_PTE_MTYPE_VG10 (flags,AMDGPU_MTYPE_NC ); +void *dst; + +if (!adev->gart.ptr) +return ; + +if (!drm_dev_enter (adev_to_drm (adev), &idx)) +return ; + + t = offset /AMDGPU_GPU_PAGE_SIZE ; + dst = adev->gart.ptr; +for (i =0 ; i < pages; i++) { + page_base = dma_addr[i]; +for (j =0 ; j <AMDGPU_GPU_PAGES_IN_CPU_PAGE ; j++, t++) { +if ((i ==0 ) && (j ==0 )) +amdgpu_gmc_set_pte_pde (adev, dst, t, page_base, flags); +else +amdgpu_gmc_set_pte_pde (adev, dst, t, page_base, ctrl_flags); + page_base +=AMDGPU_GPU_PAGE_SIZE ; + } + } +drm_dev_exit (idx); +} +/** * amdgpu_gart_bind - bind pages into the gart page table * drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h +2
-0@@ -62,6 +62,8 @@ void amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,void amdgpu_gart_map (struct amdgpu_device *adev,uint64_t offset,int pages,dma_addr_t *dma_addr,uint64_t flags,void *dst); +void amdgpu_gart_map_gfx9_mqd (struct amdgpu_device *adev,uint64_t offset, +int pages,dma_addr_t *dma_addr,uint64_t flags);void amdgpu_gart_bind (struct amdgpu_device *adev,uint64_t offset,int pages,dma_addr_t *dma_addr,uint64_t flags); void amdgpu_gart_map_vram_range(struct amdgpu_device *adev, uint64_t pa,drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +3
-13@@ -855,25 +855,15 @@ static void amdgpu_ttm_gart_bind_gfx9_mqd(struct amdgpu_device *adev,int num_xcc =max (1U , adev->gfx.num_xcc_per_xcp);uint64_t page_idx, pages_per_xcc;int i; -uint64_t ctrl_flags =AMDGPU_PTE_MTYPE_VG10 (flags,AMDGPU_MTYPE_NC ); pages_per_xcc = total_pages;do_div (pages_per_xcc, num_xcc);for (i =0 , page_idx =0 ; i < num_xcc; i++, page_idx += pages_per_xcc) { -/* MQD page: use default flags */ -amdgpu_gart_bind (adev, +amdgpu_gart_map_gfx9_mqd (adev, gtt->offset + (page_idx <<PAGE_SHIFT ), -1 , >t->ttm.dma_address[page_idx], flags); -/* -* Ctrl pages - modify the memory type to NC (ctrl_flags) from -* the second page of the BO onward. -*/ -amdgpu_gart_bind (adev, - gtt->offset + ((page_idx +1 ) <<PAGE_SHIFT ), - pages_per_xcc -1 , - >t->ttm.dma_address[page_idx +1 ], - ctrl_flags); + pages_per_xcc, >t->ttm.dma_address[page_idx], + flags); } }drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c +15
-8@@ -42,9 +42,16 @@ static uint64_t mqd_stride_v9(struct mqd_manager *mm, struct queue_properties *q) {if (mm->dev->kfd->cwsr_enabled && - q->type ==KFD_QUEUE_TYPE_COMPUTE ) -return ALIGN (q->ctl_stack_size,PAGE_SIZE ) + -ALIGN (sizeof (struct v9_mqd ),PAGE_SIZE ); + q->type ==KFD_QUEUE_TYPE_COMPUTE ) { + +/* On gfxv9, the MQD resides in the first 4K page, +* followed by the control stack. Align both to +* AMDGPU_GPU_PAGE_SIZE to maintain the required 4K boundary. +*/ + +return ALIGN (ALIGN (q->ctl_stack_size,AMDGPU_GPU_PAGE_SIZE ) + +ALIGN (sizeof (struct v9_mqd ),AMDGPU_GPU_PAGE_SIZE ),PAGE_SIZE ); + }return mm->mqd_size; }@@ -148,8 +155,8 @@ static struct kfd_mem_obj *allocate_mqd(struct mqd_manager *mm,if (!mqd_mem_obj)return NULL ; retval = amdgpu_amdkfd_alloc_kernel_mem(node->adev, - (ALIGN(q->ctl_stack_size, PAGE_SIZE) + - ALIGN(sizeof(struct v9_mqd), PAGE_SIZE)) * + (ALIGN(ALIGN(q->ctl_stack_size, AMDGPU_GPU_PAGE_SIZE) + + ALIGN(sizeof(struct v9_mqd), AMDGPU_GPU_PAGE_SIZE), PAGE_SIZE)) * NUM_XCC(node->xcc_mask), mqd_on_vram(node->adev) ? AMDGPU_GEM_DOMAIN_VRAM : AMDGPU_GEM_DOMAIN_GTT,@@ -357,7 +364,7 @@ static int get_wave_state(struct mqd_manager *mm, void *mqd,struct kfd_context_save_area_header header;/* Control stack is located one page after MQD. */ -void *mqd_ctl_stack = (void *)((uintptr_t )mqd +PAGE_SIZE ); +void *mqd_ctl_stack = (void *)((uintptr_t )mqd +AMDGPU_GPU_PAGE_SIZE ); m =get_mqd (mqd);@@ -394,7 +401,7 @@ static void checkpoint_mqd(struct mqd_manager *mm, void *mqd, void *mqd_dst, voi {struct v9_mqd *m;/* Control stack is located one page after MQD. */ -void *ctl_stack = (void *)((uintptr_t )mqd +PAGE_SIZE ); +void *ctl_stack = (void *)((uintptr_t )mqd +AMDGPU_GPU_PAGE_SIZE ); m =get_mqd (mqd);@@ -440,7 +447,7 @@ static void restore_mqd(struct mqd_manager *mm, void **mqd, *gart_addr = addr;/* Control stack is located one page after MQD. */ - ctl_stack = (void *)((uintptr_t )*mqd +PAGE_SIZE ); + ctl_stack = (void *)((uintptr_t )*mqd +AMDGPU_GPU_PAGE_SIZE );memcpy (ctl_stack, ctl_stack_src, ctl_stack_size); m->cp_hqd_pq_doorbell_control =
Alex Deucher
AppliedRe: [RESEND RFC PATCH v3 5/6] drm/amd: Fix MQD and control stack alignment for non-4K
Applied. Thanks!
Alex
Quoted 204 lines · Felix
Quoted 24 lines · Donet Tom
For gfxV9, due to a hardware bug ("based on the comments in the code here [1]"), the control stack of a user-mode compute queue must be allocated immediately after the page boundary of its regular MQD buffer. To handle this, we allocate an enlarged MQD buffer where the first page is used as the MQD and the remaining pages store the control stack. Although these regions share the same BO, they require different memory
types: the MQD must be UC (uncached), while the control stack must be NC (non-coherent), matching the behavior when the control stack is allocated in user space.This logic works correctly on systems where the CPU page size matches the GPU page size (4K). However, the current implementation aligns both the MQD and the control stack to the CPU
PAGE_SIZE. On systems with a larger CPU page size, the entire first CPU page is marked UC—even though that page may contain multiple GPU pages. The GPU treats the second 4K GPU page inside that CPU page as part of the control stack, but it is incorrectly mapped as UC.This patch fixes the issue by aligning both the MQD and control stack sizes to the GPU page size (4K). The first 4K page is correctly marked as UC for the MQD, and the remaining GPU pages are marked NC for the control stack. This ensures proper memory type assignment on systems with larger CPU page sizes.
Quoted 3 lines
[1]: https://elixir.bootlin.com/linux/v6.18/source/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c#L118
Signed-off-by Donet Tom <[email protected]>Acked-by Felix Kuehling <[email protected]>Quoted 170 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 44 +++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 16 ++----- .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c | 23 ++++++---- 4 files changed, 64 insertions(+), 21 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +44
-0@@ -403,6 +403,50 @@ void amdgpu_gart_map_vram_range(struct amdgpu_device *adev, uint64_t pa,drm_dev_exit (idx); } +/** +* amdgpu_gart_map_gfx9_mqd - map mqd and ctrl_stack dma_addresses into GART entries +* +* @adev: amdgpu_device pointer +* @offset: offset into the GPU's gart aperture +* @pages: number of pages to bind +* @dma_addr: DMA addresses of pages +* @flags: page table entry flags +* +* Map the MQD and control stack addresses into GART entries with the correct +* memory types on gfxv9. The MQD occupies the first 4KB and is followed by +* the control stack. The MQD uses UC (uncached) memory, while the control stack +* uses NC (non-coherent) memory. +*/ +void amdgpu_gart_map_gfx9_mqd (struct amdgpu_device *adev,uint64_t offset, +int pages,dma_addr_t *dma_addr,uint64_t flags) +{ +uint64_t page_base; +unsigned int i, j, t; +int idx; +uint64_t ctrl_flags =AMDGPU_PTE_MTYPE_VG10 (flags,AMDGPU_MTYPE_NC ); +void *dst; + +if (!adev->gart.ptr) +return ; + +if (!drm_dev_enter (adev_to_drm (adev), &idx)) +return ; + + t = offset /AMDGPU_GPU_PAGE_SIZE ; + dst = adev->gart.ptr; +for (i =0 ; i < pages; i++) { + page_base = dma_addr[i]; +for (j =0 ; j <AMDGPU_GPU_PAGES_IN_CPU_PAGE ; j++, t++) { +if ((i ==0 ) && (j ==0 )) +amdgpu_gmc_set_pte_pde (adev, dst, t, page_base, flags); +else +amdgpu_gmc_set_pte_pde (adev, dst, t, page_base, ctrl_flags); + page_base +=AMDGPU_GPU_PAGE_SIZE ; + } + } +drm_dev_exit (idx); +} +/** * amdgpu_gart_bind - bind pages into the gart page table * drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h +2
-0@@ -62,6 +62,8 @@ void amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,void amdgpu_gart_map (struct amdgpu_device *adev,uint64_t offset,int pages,dma_addr_t *dma_addr,uint64_t flags,void *dst); +void amdgpu_gart_map_gfx9_mqd (struct amdgpu_device *adev,uint64_t offset, +int pages,dma_addr_t *dma_addr,uint64_t flags);void amdgpu_gart_bind (struct amdgpu_device *adev,uint64_t offset,int pages,dma_addr_t *dma_addr,uint64_t flags); void amdgpu_gart_map_vram_range(struct amdgpu_device *adev, uint64_t pa,drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +3
-13@@ -855,25 +855,15 @@ static void amdgpu_ttm_gart_bind_gfx9_mqd(struct amdgpu_device *adev,int num_xcc =max (1U , adev->gfx.num_xcc_per_xcp);uint64_t page_idx, pages_per_xcc;int i; -uint64_t ctrl_flags =AMDGPU_PTE_MTYPE_VG10 (flags,AMDGPU_MTYPE_NC ); pages_per_xcc = total_pages;do_div (pages_per_xcc, num_xcc);for (i =0 , page_idx =0 ; i < num_xcc; i++, page_idx += pages_per_xcc) { -/* MQD page: use default flags */ -amdgpu_gart_bind (adev, +amdgpu_gart_map_gfx9_mqd (adev, gtt->offset + (page_idx <<PAGE_SHIFT ), -1 , >t->ttm.dma_address[page_idx], flags); -/* -* Ctrl pages - modify the memory type to NC (ctrl_flags) from -* the second page of the BO onward. -*/ -amdgpu_gart_bind (adev, - gtt->offset + ((page_idx +1 ) <<PAGE_SHIFT ), - pages_per_xcc -1 , - >t->ttm.dma_address[page_idx +1 ], - ctrl_flags); + pages_per_xcc, >t->ttm.dma_address[page_idx], + flags); } }drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c +15
-8@@ -42,9 +42,16 @@ static uint64_t mqd_stride_v9(struct mqd_manager *mm, struct queue_properties *q) {if (mm->dev->kfd->cwsr_enabled && - q->type ==KFD_QUEUE_TYPE_COMPUTE ) -return ALIGN (q->ctl_stack_size,PAGE_SIZE ) + -ALIGN (sizeof (struct v9_mqd ),PAGE_SIZE ); + q->type ==KFD_QUEUE_TYPE_COMPUTE ) { + +/* On gfxv9, the MQD resides in the first 4K page, +* followed by the control stack. Align both to +* AMDGPU_GPU_PAGE_SIZE to maintain the required 4K boundary. +*/ + +return ALIGN (ALIGN (q->ctl_stack_size,AMDGPU_GPU_PAGE_SIZE ) + +ALIGN (sizeof (struct v9_mqd ),AMDGPU_GPU_PAGE_SIZE ),PAGE_SIZE ); + }return mm->mqd_size; }@@ -148,8 +155,8 @@ static struct kfd_mem_obj *allocate_mqd(struct mqd_manager *mm,if (!mqd_mem_obj)return NULL ; retval = amdgpu_amdkfd_alloc_kernel_mem(node->adev, - (ALIGN(q->ctl_stack_size, PAGE_SIZE) + - ALIGN(sizeof(struct v9_mqd), PAGE_SIZE)) * + (ALIGN(ALIGN(q->ctl_stack_size, AMDGPU_GPU_PAGE_SIZE) + + ALIGN(sizeof(struct v9_mqd), AMDGPU_GPU_PAGE_SIZE), PAGE_SIZE)) * NUM_XCC(node->xcc_mask), mqd_on_vram(node->adev) ? AMDGPU_GEM_DOMAIN_VRAM : AMDGPU_GEM_DOMAIN_GTT,@@ -357,7 +364,7 @@ static int get_wave_state(struct mqd_manager *mm, void *mqd,struct kfd_context_save_area_header header;/* Control stack is located one page after MQD. */ -void *mqd_ctl_stack = (void *)((uintptr_t )mqd +PAGE_SIZE ); +void *mqd_ctl_stack = (void *)((uintptr_t )mqd +AMDGPU_GPU_PAGE_SIZE ); m =get_mqd (mqd);@@ -394,7 +401,7 @@ static void checkpoint_mqd(struct mqd_manager *mm, void *mqd, void *mqd_dst, voi {struct v9_mqd *m;/* Control stack is located one page after MQD. */ -void *ctl_stack = (void *)((uintptr_t )mqd +PAGE_SIZE ); +void *ctl_stack = (void *)((uintptr_t )mqd +AMDGPU_GPU_PAGE_SIZE ); m =get_mqd (mqd);@@ -440,7 +447,7 @@ static void restore_mqd(struct mqd_manager *mm, void **mqd, *gart_addr = addr;/* Control stack is located one page after MQD. */ - ctl_stack = (void *)((uintptr_t )*mqd +PAGE_SIZE ); + ctl_stack = (void *)((uintptr_t )*mqd +AMDGPU_GPU_PAGE_SIZE );memcpy (ctl_stack, ctl_stack_src, ctl_stack_size); m->cp_hqd_pq_doorbell_control =
RFC PATCH v3 6/6 drm/amdkfd: Fix queue preemption/eviction failures by aligning control stack size to GPU page size
In reply to Donet TomThe control stack size is calculated based on the number of CUs and waves, and is then aligned to PAGE_SIZE. When the resulting control stack size is aligned to 64 KB, GPU hangs and queue preemption failures are observed while running RCCL unit tests on systems with more than two GPUs.
amdgpu 0048:0f:00.0: amdgpu: Queue preemption failed for queue withdoorbell_id: 80030008
amdgpu 0048:0f:00.0: amdgpu: Failed to evict process queues amdgpu 0048:0f:00.0: amdgpu: GPU reset begin!. Source: 4 amdgpu 0048:0f:00.0: amdgpu: Queue preemption failed for queue withdoorbell_id: 80030008
amdgpu 0048:0f:00.0: amdgpu: Failed to evict process queues amdgpu 0048:0f:00.0: amdgpu: Failed to restore process queues
This issue is observed on both 4 KB and 64 KB system page-size configurations.
This patch fixes the issue by aligning the control stack size to AMDGPU_GPU_PAGE_SIZE instead of PAGE_SIZE, so the control stack size will not be 64 KB on systems with a 64 KB page size and queue preemption works correctly.
Additionally, In the current code, wg_data_size is aligned to PAGE_SIZE, which can waste memory if the system page size is large. In this patch, wg_data_size is aligned to AMDGPU_GPU_PAGE_SIZE. The cwsr_size, calculated from wg_data_size and the control stack size, is aligned to PAGE_SIZE.
--- drivers/gpu/drm/amd/amdkfd/kfd_queue.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_queue.c +4 -3
@@ -492,10 +492,11 @@ void kfd_queue_ctx_save_restore_size(struct kfd_topology_device *dev) cu_num = props->simd_count / props->simd_per_cu /NUM_XCC (dev->gpu->xcc_mask); wave_num =get_num_waves (props, gfxv, cu_num); - wg_data_size =ALIGN (cu_num *WG_CONTEXT_DATA_SIZE_PER_CU (gfxv, props),PAGE_SIZE ); + wg_data_size =ALIGN (cu_num *WG_CONTEXT_DATA_SIZE_PER_CU (gfxv, props), +AMDGPU_GPU_PAGE_SIZE ); ctl_stack_size = wave_num *CNTL_STACK_BYTES_PER_WAVE (gfxv) +8 ; ctl_stack_size =ALIGN (SIZEOF_HSA_USER_CONTEXT_SAVE_AREA_HEADER + ctl_stack_size, -PAGE_SIZE ); +AMDGPU_GPU_PAGE_SIZE );if ((gfxv /10000 *10000 ) ==100000 ) {/* HW design limits control stack size to 0x7000.
@@ -507,7 +508,7 @@ void kfd_queue_ctx_save_restore_size(struct kfd_topology_device *dev) props->ctl_stack_size = ctl_stack_size; props->debug_memory_size =ALIGN (wave_num *DEBUGGER_BYTES_PER_WAVE ,DEBUGGER_BYTES_ALIGN ); - props->cwsr_size = ctl_stack_size + wg_data_size; + props->cwsr_size =ALIGN (ctl_stack_size + wg_data_size,PAGE_SIZE );if (gfxv ==80002 )/* GFX_VERSION_TONGA */ props->eop_buffer_size =0x8000 ;
Signature · 1 line
2.52.0
Re: [RESEND RFC PATCH v3 6/6] drm/amdkfd: Fix queue preemption/eviction failures by aligning control stack size to GPU page size
Quoted 23 lines · Donet Tom
The control stack size is calculated based on the number of CUs and waves, and is then aligned to
PAGE_SIZE. When the resulting control stack size is aligned to 64 KB, GPU hangs and queue preemption failures are observed while running RCCL unit tests on systems with more than two GPUs.amdgpu 0048:0f:00.0: amdgpu: Queue preemption failed for queue with
doorbell_id: 80030008
amdgpu 0048:0f:00.0: amdgpu: Failed to evict process queues amdgpu 0048:0f:00.0: amdgpu: GPU reset begin!. Source: 4 amdgpu 0048:0f:00.0: amdgpu: Queue preemption failed for queue withdoorbell_id: 80030008
amdgpu 0048:0f:00.0: amdgpu: Failed to evict process queues amdgpu 0048:0f:00.0: amdgpu: Failed to restore process queuesThis issue is observed on both 4 KB and 64 KB system page-size configurations.
This patch fixes the issue by aligning the control stack size to
AMDGPU_GPU_PAGE_SIZEinstead ofPAGE_SIZE, so the control stack size will not be 64 KB on systems with a 64 KB page size and queue preemption works correctly.
Quoted 6 lines
Additionally, In the current code,
wg_data_sizeis aligned toPAGE_SIZE, which can waste memory if the system page size is large. In this patch,wg_data_sizeis aligned toAMDGPU_GPU_PAGE_SIZE. Thecwsr_size, calculated fromwg_data_sizeand the control stack size, is aligned toPAGE_SIZE.Signed-off-by Donet Tom <[email protected]>
Quoted 31 lines
--- drivers/gpu/drm/amd/amdkfd/kfd_queue.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)drivers/gpu/drm/amd/amdkfd/kfd_queue.c +4
-3@@ -492,10 +492,11 @@ void kfd_queue_ctx_save_restore_size(struct kfd_topology_device *dev) cu_num = props->simd_count / props->simd_per_cu /NUM_XCC (dev->gpu->xcc_mask); wave_num =get_num_waves (props, gfxv, cu_num); - wg_data_size =ALIGN (cu_num *WG_CONTEXT_DATA_SIZE_PER_CU (gfxv, props),PAGE_SIZE ); + wg_data_size =ALIGN (cu_num *WG_CONTEXT_DATA_SIZE_PER_CU (gfxv, props), +AMDGPU_GPU_PAGE_SIZE ); ctl_stack_size = wave_num *CNTL_STACK_BYTES_PER_WAVE (gfxv) +8 ; ctl_stack_size =ALIGN (SIZEOF_HSA_USER_CONTEXT_SAVE_AREA_HEADER + ctl_stack_size, -PAGE_SIZE ); +AMDGPU_GPU_PAGE_SIZE );if ((gfxv /10000 *10000 ) ==100000 ) {/* HW design limits control stack size to 0x7000. @@ -507,7 +508,7 @@ void kfd_queue_ctx_save_restore_size(struct kfd_topology_device *dev) props->ctl_stack_size = ctl_stack_size; props->debug_memory_size =ALIGN (wave_num *DEBUGGER_BYTES_PER_WAVE ,DEBUGGER_BYTES_ALIGN ); - props->cwsr_size = ctl_stack_size + wg_data_size; + props->cwsr_size =ALIGN (ctl_stack_size + wg_data_size,PAGE_SIZE );if (gfxv ==80002 )/* GFX_VERSION_TONGA */ props->eop_buffer_size =0x8000 ;
Alex Deucher
AppliedRe: [RESEND RFC PATCH v3 6/6] drm/amdkfd: Fix queue preemption/eviction failures by aligning control stack size to GPU page size
Applied. Thanks!
Alex
Quoted 67 lines · Felix
Quoted 23 lines · Donet Tom
The control stack size is calculated based on the number of CUs and waves, and is then aligned to
PAGE_SIZE. When the resulting control stack size is aligned to 64 KB, GPU hangs and queue preemption failures are observed while running RCCL unit tests on systems with more than two GPUs.amdgpu 0048:0f:00.0: amdgpu: Queue preemption failed for queue with
doorbell_id: 80030008
amdgpu 0048:0f:00.0: amdgpu: Failed to evict process queues amdgpu 0048:0f:00.0: amdgpu: GPU reset begin!. Source: 4 amdgpu 0048:0f:00.0: amdgpu: Queue preemption failed for queue withdoorbell_id: 80030008
amdgpu 0048:0f:00.0: amdgpu: Failed to evict process queues amdgpu 0048:0f:00.0: amdgpu: Failed to restore process queuesThis issue is observed on both 4 KB and 64 KB system page-size configurations.
This patch fixes the issue by aligning the control stack size to
AMDGPU_GPU_PAGE_SIZEinstead ofPAGE_SIZE, so the control stack size will not be 64 KB on systems with a 64 KB page size and queue preemption works correctly.Quoted 6 lines
Additionally, In the current code,
wg_data_sizeis aligned toPAGE_SIZE, which can waste memory if the system page size is large. In this patch,wg_data_sizeis aligned toAMDGPU_GPU_PAGE_SIZE. Thecwsr_size, calculated fromwg_data_sizeand the control stack size, is aligned toPAGE_SIZE.Signed-off-by Donet Tom <[email protected]>Reviewed-by Felix Kuehling <[email protected]>Quoted 31 lines
--- drivers/gpu/drm/amd/amdkfd/kfd_queue.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)drivers/gpu/drm/amd/amdkfd/kfd_queue.c +4
-3@@ -492,10 +492,11 @@ void kfd_queue_ctx_save_restore_size(struct kfd_topology_device *dev) cu_num = props->simd_count / props->simd_per_cu /NUM_XCC (dev->gpu->xcc_mask); wave_num =get_num_waves (props, gfxv, cu_num); - wg_data_size =ALIGN (cu_num *WG_CONTEXT_DATA_SIZE_PER_CU (gfxv, props),PAGE_SIZE ); + wg_data_size =ALIGN (cu_num *WG_CONTEXT_DATA_SIZE_PER_CU (gfxv, props), +AMDGPU_GPU_PAGE_SIZE ); ctl_stack_size = wave_num *CNTL_STACK_BYTES_PER_WAVE (gfxv) +8 ; ctl_stack_size =ALIGN (SIZEOF_HSA_USER_CONTEXT_SAVE_AREA_HEADER + ctl_stack_size, -PAGE_SIZE ); +AMDGPU_GPU_PAGE_SIZE );if ((gfxv /10000 *10000 ) ==100000 ) {/* HW design limits control stack size to 0x7000. @@ -507,7 +508,7 @@ void kfd_queue_ctx_save_restore_size(struct kfd_topology_device *dev) props->ctl_stack_size = ctl_stack_size; props->debug_memory_size =ALIGN (wave_num *DEBUGGER_BYTES_PER_WAVE ,DEBUGGER_BYTES_ALIGN ); - props->cwsr_size = ctl_stack_size + wg_data_size; + props->cwsr_size =ALIGN (ctl_stack_size + wg_data_size,PAGE_SIZE );if (gfxv ==80002 )/* GFX_VERSION_TONGA */ props->eop_buffer_size =0x8000 ;
Kuehling, Felix
maintainerQuoted 38 lines · Donet Tom
This is v3 of the patch series enabling 64 KB system page size support in AMDGPU. v2, part 1 of this series [1] has already been merged upstream and provides the minimal infrastructure required for 64 KB page support.
This series addresses additional issues uncovered in AMDGPU when running rccl unit tests and rocr-debug-agent tessts on 64KB page-size systems.
With this series applied, all RCCL unit tests and rocr-debug-agent tests pass on systems using a 64 KB system page size, across multi-GPU configurations, with XNACK both enabled and disabled.
Patch 1 in this series (drm/amdgpu: Change
AMDGPU_VA_RESERVED_TRAP_SIZEto 2 *PAGE_SIZE) fixes a kernel crash observed when running rocminfo on systems with a 64 KB page size. This patch is required to enable minimal support for 64 KB system page sizes.Since RFC v2, we observed AQL queue creation failures while running certain workloads on 64K page-size systems due to an expected queue size mismatch. This issue is addressed in patch 2 of this series.
The questions we had in this seres are:
=======================================
1 When the control stack size is aligned to 64 KB, we consistently
observe queue preemption or eviction failures on gfx9, on both 4 KB and 64 KB system page-size configurations.The control stack size is calculated based on the number of CUs and waves and is then aligned to
PAGE_SIZE. On systems with a 64 KB system page size, this alignment always results in a 64 KB-aligned control stack size, after which queue preemption fails.Is there any hardware-imposed limitation on gfx9 that prevents the control stack size from being 64 KB? For gfx10, I see explicit hardware limitations on the control stack size in the code [2]. Is there anything similar for gfx9?
Quoted 5 lines
What is the correct or recommended control stack size for gfx9? With a 4 KB system page size, I observe a control stack size of around 44 KB—can it grow beyond this? If the control stack size is fixed for a given gfx version, do you see any issues with aligning the control stack size to the GPU page size?
I think there is a bug in user mode that uses its own calculation of the ctl_stack_size to calculate the total context save area size. If kernel mode increases the ctl_stack_size, the context save are allocated by user mode will be too small.
/* Keep calculating it in case we are using an older kernel, but if we have
* the CtlStackSize and CwsrSize from KFD, use that as the definitive value
*/
q->ctx_save_restore_size = node.CwsrSize > 0 ? node.CwsrSize :
q->ctl_stack_size + PAGE_ALIGN_UP(wg_data_size);
q->ctl_stack_size = node.CtlStackSize > 0 ? node.CtlStackSize : q->ctl_stack_size;ctx_save_restore_size should be calculated after correcting ctl_stack_size with the one from the kernel mode driver.
Regards,
Felix
Quoted 68 lines
This series has 5 patches
=========================
1.AMDGPU_VA_RESERVED_TRAP_SIZEwas hard-coded to 8 KB whileKFD_CWSR_TBA_TMA_SIZEis defined as 2 *PAGE_SIZE, which matches on 4 KB page-size systems but results in a size mismatch on 64 KB systems, leading to kernel crashes when running rocminfo or RCCL unit tests.
This patch updatesAMDGPU_VA_RESERVED_TRAP_SIZEto 2 *PAGE_SIZEso that the reserved trap area matches the allocation size across all system page sizes. This is a must needed patch to enable minimal support for 64 KB system page sizes.2. Aligned
expected_queue_sizetoPAGE_SIZEto fix AQL queue creation failure.3. Fix amdgpu page fault handler (for xnack) to pass the corresponding system pfn (instead of gpu pfn) for restoring SVM range mapping.
4. Updated
AMDGPU_GTT_MAX_TRANSFER_SIZEto always match the PMD size across all page sizes.5. On systems where the CPU page size is larger than the GPU’s 4 KB page size, the MQD and control stack were aligned to the CPU
PAGE_SIZE, causing multiple GPU pages to incorrectly inherit the UC attribute. This change aligns both regions to the GPU page size, ensuring that the MQD is mapped as UC and the control stack as NC, restoring the correct behavior.6. Queue preemption fails when the control stack size is aligned to 64 KB. This patch fixes this issue by aligning the control stack size to gpu page size.
Setup details:
============
System details: Power10 LPAR using 64K pagesize.
AMD GPU:Name: gfx90a Marketing Name: AMD Instinct MI210[1]https://lore.kernel.org/all/[email protected]/ [2]https://elixir.bootlin.com/linux/v6.19-rc5/source/drivers/gpu/drm/amd/amdkfd/kfd_queue.c#L457
RFC V3 -https://lore.kernel.org/all/[email protected]/ RFC V2 -https://lore.kernel.org/all/[email protected]/ RFC V1 -https://lore.kernel.org/all/[email protected]/
Donet Tom (6):
drm/amdgpu: ChangeAMDGPU_VA_RESERVED_TRAP_SIZEto 2PAGE_SIZEpages drm/amdkfd: Alignexpected_queue_sizetoPAGE_SIZEdrm/amdgpu: Handle GPU page faults correctly on non-4K page systems drm/amdgpu: FixAMDGPU_GTT_MAX_TRANSFER_SIZEfor non-4K page size drm/amd: Fix MQD and control stack alignment for non-4K drm/amdkfd: Fix queue preemption/eviction failures by aligning control
stack size to GPU page sizedrivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 44 +++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 24 ++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 6 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 2 +- drivers/gpu/drm/amd/amdgpu/vce_v1_0.c | 3 +- .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c | 23 ++++++---- drivers/gpu/drm/amd/amdkfd/kfd_queue.c | 11 ++--- 9 files changed, 82 insertions(+), 35 deletions(-)
Quoted 4 lines · Felix
On 2026-03-23 00: 28, Donet Tom wrote: This is v3 of the patch series enabling 64 KB system page size support in AMDGPU. v2, part 1 of this series [1] has already been merged upstream and provides the minimal infrastructure required for 64 KB
Quoted 44 lines
Quoted 43 lines · Donet Tom
This is v3 of the patch series enabling 64 KB system page size support in AMDGPU. v2, part 1 of this series [1] has already been merged upstream and provides the minimal infrastructure required for 64 KB page support.
This series addresses additional issues uncovered in AMDGPU when running rccl unit tests and rocr-debug-agent tessts on 64KB page-size systems.
With this series applied, all RCCL unit tests and rocr-debug-agent tests pass on systems using a 64 KB system page size, across multi-GPU configurations, with XNACK both enabled and disabled.
Patch 1 in this series (drm/amdgpu: Change
AMDGPU_VA_RESERVED_TRAP_SIZEto 2 *PAGE_SIZE) fixes a kernel crash observed when running rocminfo on systems with a 64 KB page size. This patch is required to enable minimal support for 64 KB system page sizes.Since RFC v2, we observed AQL queue creation failures while running certain workloads on 64K page-size systems due to an expected queue size mismatch. This issue is addressed in patch 2 of this series.
The questions we had in this seres are:
=======================================
1 When the control stack size is aligned to 64 KB, we consistently
observe queue preemption or eviction failures on gfx9, on both 4 KB and 64 KB system page-size configurations.The control stack size is calculated based on the number of CUs and waves and is then aligned to
PAGE_SIZE. On systems with a 64 KB system page size, this alignment always results in a 64 KB-aligned control stack size, after which queue preemption fails.Is there any hardware-imposed limitation on gfx9 that prevents the control stack size from being 64 KB? For gfx10, I see explicit hardware limitations on the control stack size in the code [2]. Is there anything similar for gfx9?
What is the correct or recommended control stack size for gfx9? With a 4 KB system page size, I observe a control stack size of around 44 KB—can it grow beyond this? If the control stack size is fixed for a given gfx version, do you see any issues with aligning the control stack size to the GPU page size?
Thank you, Felix, for your time and for reviewing this patch
Quoted 4 lines
I think there is a bug in user mode that uses its own calculation of the
ctl_stack_sizeto calculate the total context save area size. If kernel mode increases thectl_stack_size, the context save are allocated by user mode will be too small.
Quoted 9 lines
/* Keep calculating it in case we are using an older kernel, but if we have * the CtlStackSize and CwsrSize from KFD, use that as the definitive value */ q->ctx_save_restore_size = node.CwsrSize > 0 ? node.CwsrSize : q->ctl_stack_size + PAGE_ALIGN_UP(wg_data_size); q->ctl_stack_size = node.CtlStackSize > 0 ? node.CtlStackSize : q->ctl_stack_size;
ctx_save_restore_sizeshould be calculated after correctingctl_stack_sizewith the one from the kernel mode driver.
Yes, we also need a fix in rocr-runtime. In rocr-runtime, I used the same approach as in the kernel (patch 6/6) to calculate ctl_stack_size and ctx_save_restore_size. Without this library change, I was encountering queue creation failures. With the library change and with this series all rccl tests are passing on both 4K and 64K page sizes.
-Donet
Quoted 70 lines
Regards,
FelixQuoted 67 lines
This series has 5 patches
=========================
1.AMDGPU_VA_RESERVED_TRAP_SIZEwas hard-coded to 8 KB whileKFD_CWSR_TBA_TMA_SIZEis defined as 2 *PAGE_SIZE, which matches on 4 KB page-size systems but results in a size mismatch on 64 KB systems, leading to kernel crashes when running rocminfo or RCCL unit tests.
This patch updatesAMDGPU_VA_RESERVED_TRAP_SIZEto 2 *PAGE_SIZEso that the reserved trap area matches the allocation size across all system page sizes. This is a must needed patch to enable minimal support for 64 KB system page sizes.2. Aligned
expected_queue_sizetoPAGE_SIZEto fix AQL queue creation failure.3. Fix amdgpu page fault handler (for xnack) to pass the corresponding system pfn (instead of gpu pfn) for restoring SVM range mapping.
4. Updated
AMDGPU_GTT_MAX_TRANSFER_SIZEto always match the PMD size across all page sizes.5. On systems where the CPU page size is larger than the GPU’s 4 KB page size, the MQD and control stack were aligned to the CPU
PAGE_SIZE, causing multiple GPU pages to incorrectly inherit the UC attribute. This change aligns both regions to the GPU page size, ensuring that the MQD is mapped as UC and the control stack as NC, restoring the correct behavior.6. Queue preemption fails when the control stack size is aligned to 64 KB. This patch fixes this issue by aligning the control stack size to gpu page size.
Setup details:
============
System details: Power10 LPAR using 64K pagesize.
AMD GPU:Name: gfx90a Marketing Name: AMD Instinct MI210[1]https://lore.kernel.org/all/[email protected]/ [2]https://elixir.bootlin.com/linux/v6.19-rc5/source/drivers/gpu/drm/amd/amdkfd/kfd_queue.c#L457
RFC V3 -https://lore.kernel.org/all/[email protected]/ RFC V2 -https://lore.kernel.org/all/[email protected]/ RFC V1 -https://lore.kernel.org/all/[email protected]/
Donet Tom (6):
drm/amdgpu: ChangeAMDGPU_VA_RESERVED_TRAP_SIZEto 2PAGE_SIZEpages drm/amdkfd: Alignexpected_queue_sizetoPAGE_SIZEdrm/amdgpu: Handle GPU page faults correctly on non-4K page systems drm/amdgpu: FixAMDGPU_GTT_MAX_TRANSFER_SIZEfor non-4K page size drm/amd: Fix MQD and control stack alignment for non-4K drm/amdkfd: Fix queue preemption/eviction failures by aligning control
stack size to GPU page sizedrivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 44 +++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 24 ++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 6 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 2 +- drivers/gpu/drm/amd/amdgpu/vce_v1_0.c | 3 +- .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c | 23 ++++++---- drivers/gpu/drm/amd/amdkfd/kfd_queue.c | 11 ++--- 9 files changed, 82 insertions(+), 35 deletions(-)