Skip to contentSkip to conversation

drm/amdgpu: Fix default page access routing

PATCH Applied Jay Cornwall 3 participants 4 messages Nov 5, 2015 – Nov 10, 2015

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/amdgpu: Fix default page access routing

The VM default page (used when a VM translation fails) is allocated in system memory. The VM is misconfigured to interpret the physical address as referencing a VRAM physical page.

Route default page accesses to system memory.

---
 drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 1 +
 drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 1 +
 2 files changed, 2 insertions(+)
drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +1 -0
index fab5471..b9836f6 100644
@@ -474,6 +474,7 @@ static int gmc_v7_0_gart_enable(struct amdgpu_device *adev)
 	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_PDE0_CACHE_LRU_UPDATE_BY_WRITE, 1);
 	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, EFFECTIVE_L2_QUEUE_SIZE, 7);
 	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, CONTEXT1_IDENTITY_ACCESS_MODE, 1);
+	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_DEFAULT_PAGE_OUT_TO_SYSTEM_MEMORY, 1);
 	WREG32(mmVM_L2_CNTL, tmp);
 	tmp = REG_SET_FIELD(0, VM_L2_CNTL2, INVALIDATE_ALL_L1_TLBS, 1);
 	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL2, INVALIDATE_L2_CACHE, 1);
drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c +1 -0
index 7bc9e9f..cb4e2bb 100644
@@ -588,6 +588,7 @@ static int gmc_v8_0_gart_enable(struct amdgpu_device *adev)
 	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_PDE0_CACHE_LRU_UPDATE_BY_WRITE, 1);
 	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, EFFECTIVE_L2_QUEUE_SIZE, 7);
 	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, CONTEXT1_IDENTITY_ACCESS_MODE, 1);
+	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_DEFAULT_PAGE_OUT_TO_SYSTEM_MEMORY, 1);
 	WREG32(mmVM_L2_CNTL, tmp);
 	tmp = RREG32(mmVM_L2_CNTL2);
 	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL2, INVALIDATE_ALL_L1_TLBS, 1);
Signature · 1 line
1.9.1
#1 lore raw
Quoted 8 lines · Jay Cornwall

The VM default page (used when a VM translation fails) is allocated in system memory. The VM is misconfigured to interpret the physical address as referencing a VRAM physical page.

Route default page accesses to system memory.

Nice catch, patch is Reviewed-by: Christian König <[email protected]>

Do we also need this for Radeon?

Regards,
Christian.

Quoted 22 lines
---
  drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 1 +
  drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 1 +
  2 files changed, 2 insertions(+)
drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +1 -0
index fab5471..b9836f6 100644
@@ -474,6 +474,7 @@ static int gmc_v7_0_gart_enable(struct amdgpu_device *adev)
  	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_PDE0_CACHE_LRU_UPDATE_BY_WRITE, 1);
  	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, EFFECTIVE_L2_QUEUE_SIZE, 7);
  	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, CONTEXT1_IDENTITY_ACCESS_MODE, 1);
+	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_DEFAULT_PAGE_OUT_TO_SYSTEM_MEMORY, 1);
  	WREG32(mmVM_L2_CNTL, tmp);
  	tmp = REG_SET_FIELD(0, VM_L2_CNTL2, INVALIDATE_ALL_L1_TLBS, 1);
  	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL2, INVALIDATE_L2_CACHE, 1);
drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c +0 -0
index 7bc9e9f..cb4e2bb 100644

drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c static int gmc_v8_0_gart_enable(struct amdgpu_device *adev) · line 594

Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c +1 -0
@@ -588,6 +588,7 @@ static int gmc_v8_0_gart_enable(struct amdgpu_device *adev)
  	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_PDE0_CACHE_LRU_UPDATE_BY_WRITE, 1);
  	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, EFFECTIVE_L2_QUEUE_SIZE, 7);
  	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, CONTEXT1_IDENTITY_ACCESS_MODE, 1);
+	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_DEFAULT_PAGE_OUT_TO_SYSTEM_MEMORY, 1);
  	WREG32(mmVM_L2_CNTL, tmp);
  	tmp = RREG32(mmVM_L2_CNTL2);
  	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL2, INVALIDATE_ALL_L1_TLBS, 1);

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel

#2 lore raw
Quoted 5 lines · Christian König
Quoted 4 lines · Jay Cornwall

The VM default page (used when a VM translation fails) is allocated in system memory. The VM is misconfigured to interpret the physical address as referencing a VRAM physical page.

Quoted 8 lines
Quoted 4 lines

Route default page accesses to system memory.

Nice catch, patch is Reviewed-by: Christian König <[email protected]>

Do we also need this for Radeon?

It looks like this field was introduced in CIK. This change could be replicated in radeon on those parts, but I haven't got one to test with.

I'm not sure if this misconfiguration is fatal. The default page should be used for loads only. I haven't been able to get anything besides 0 from a faulting load on my Fiji with a system or VRAM default page. A physical address > VRAM doesn't seem to cause problems, either.

My primary motivation for this change is to allow HSA to handle faulting atomic operations. The KFD exposes a MTYPE=UC aperture to the user which allows atomics to be forwarded to system memory, via MC/BIF/PCI. When one of these atomic operations faults the MC is unable to handle the VRAM atomic (atomics are implemented by TC) and the TC hangs waiting for a response.

If the default page is directed towards BIF then the atomic is correctly discarded.

Signature · 1 line
Jay Cornwall
Applied
#3 lore raw
In reply to Jay Cornwall
Quoted 8 lines · Jay Cornwall

The VM default page (used when a VM translation fails) is allocated in system memory. The VM is misconfigured to interpret the physical address as referencing a VRAM physical page.

Route default page accesses to system memory.

Applied. thanks!

Alex

Quoted 36 lines
---
 drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 1 +
 drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 1 +
 2 files changed, 2 insertions(+)
drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +1 -0
index fab5471..b9836f6 100644
@@ -474,6 +474,7 @@ static int gmc_v7_0_gart_enable(struct amdgpu_device *adev)
        tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_PDE0_CACHE_LRU_UPDATE_BY_WRITE, 1);
        tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, EFFECTIVE_L2_QUEUE_SIZE, 7);
        tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, CONTEXT1_IDENTITY_ACCESS_MODE, 1);
+       tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_DEFAULT_PAGE_OUT_TO_SYSTEM_MEMORY, 1);
        WREG32(mmVM_L2_CNTL, tmp);
        tmp = REG_SET_FIELD(0, VM_L2_CNTL2, INVALIDATE_ALL_L1_TLBS, 1);
        tmp = REG_SET_FIELD(tmp, VM_L2_CNTL2, INVALIDATE_L2_CACHE, 1);
drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c +1 -1
index 7bc9e9f..cb4e2bb 100644
@@ -588,6 +588,7 @@ static int gmc_v8_0_gart_enable(struct amdgpu_device *adev)
        tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_PDE0_CACHE_LRU_UPDATE_BY_WRITE, 1);
        tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, EFFECTIVE_L2_QUEUE_SIZE, 7);
        tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, CONTEXT1_IDENTITY_ACCESS_MODE, 1);
+       tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_DEFAULT_PAGE_OUT_TO_SYSTEM_MEMORY, 1);
        WREG32(mmVM_L2_CNTL, tmp);
        tmp = RREG32(mmVM_L2_CNTL2);
        tmp = REG_SET_FIELD(tmp, VM_L2_CNTL2, INVALIDATE_ALL_L1_TLBS, 1);
--

1.9.1

_______________________________________________
dri-devel mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/dri-devel