Skip to contentSkip to conversation

drm/amdgpu: use only the lower address space on GMC9

PATCH In review Christian König 7 participants 44 messages Aug 27, 2018 – Aug 30, 2018 27 messages not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/10 drm/amdgpu: use only the lower address space on GMC9

Only use the lower address space on GMC9 for the system domain. Otherwise we would need to sign extend GMC addresses.

---
 drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +3 -4
index e44b5191735d..d982956c8329 100644
@@ -938,11 +938,10 @@ static int gmc_v9_0_sw_init(void *handle)
 	if (r)
 		return r;
 
-	/* Set the internal MC address mask
-	 * This is the max address of the GPU's
-	 * internal address space.
+	/* Use only the lower range for the internal MC address mask. This is
+	 * the max address of the GPU's internal address space.
 	 */
-	adev->gmc.mc_mask = 0xffffffffffffULL; /* 48 bit MC */
+	adev->gmc.mc_mask = 0x7fffffffffffULL;
 
 	/* set DMA mask + need_dma32 flags.
 	 * PCIE - can handle 44-bits.
Signature · 6 lines
2.17.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20180827165337.1651-1-christian.koenig-5C7GfCeVMHo@public.gmane.org

#2 lore raw

PATCH 2/10 drm/amdgpu: move amdgpu_device_(vram|gtt)_location

Move that into amdgpu_gmc.c since we are really deadling with GMC address space here.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  4 --
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 65 ----------------------
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c    | 64 +++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h    |  4 ++
 drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c      |  4 +-
 drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c      |  4 +-
 drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c      |  4 +-
 drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c      |  4 +-
 8 files changed, 76 insertions(+), 77 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +0 -4
index 340e40d03d54..09bdedfc91c7 100644
@@ -1166,10 +1166,6 @@ bool amdgpu_device_need_post(struct amdgpu_device *adev);
 
 void amdgpu_cs_report_moved_bytes(struct amdgpu_device *adev, u64 num_bytes,
 				  u64 num_vis_bytes);
-void amdgpu_device_vram_location(struct amdgpu_device *adev,
-				 struct amdgpu_gmc *mc, u64 base);
-void amdgpu_device_gart_location(struct amdgpu_device *adev,
-				 struct amdgpu_gmc *mc);
 int amdgpu_device_resize_fb_bar(struct amdgpu_device *adev);
 void amdgpu_device_program_register_sequence(struct amdgpu_device *adev,
 					     const u32 *registers,
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -65

71 lines not on this page · raw message

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +64 -0

67 lines not on this page · raw message

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h +4 -0
index 0d2c9f65ca13..72fcc9338f5e 100644
@@ -134,5 +134,9 @@ static inline bool amdgpu_gmc_vram_full_visible(struct amdgpu_gmc *gmc)
 void amdgpu_gmc_get_pde_for_bo(struct amdgpu_bo *bo, int level,
 			       uint64_t *addr, uint64_t *flags);
 uint64_t amdgpu_gmc_pd_addr(struct amdgpu_bo *bo);
+void amdgpu_gmc_vram_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc,
+			      u64 base);
+void amdgpu_gmc_gart_location(struct amdgpu_device *adev,
+			      struct amdgpu_gmc *mc);
 
 #endif
drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c +2 -2
index 543287e5d67b..c2866e1acbef 100644
@@ -224,8 +224,8 @@ static void gmc_v6_0_vram_gtt_location(struct amdgpu_device *adev,
 	u64 base = RREG32(mmMC_VM_FB_LOCATION) & 0xFFFF;
 	base <<= 24;
 
-	amdgpu_device_vram_location(adev, &adev->gmc, base);
-	amdgpu_device_gart_location(adev, mc);
+	amdgpu_gmc_vram_location(adev, &adev->gmc, base);
+	amdgpu_gmc_gart_location(adev, mc);
 }
 
 static void gmc_v6_0_mc_program(struct amdgpu_device *adev)
drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +2 -2
index 060c79afef80..cbbbb6bbf782 100644
@@ -244,8 +244,8 @@ static void gmc_v7_0_vram_gtt_location(struct amdgpu_device *adev,
 	u64 base = RREG32(mmMC_VM_FB_LOCATION) & 0xFFFF;
 	base <<= 24;
 
-	amdgpu_device_vram_location(adev, &adev->gmc, base);
-	amdgpu_device_gart_location(adev, mc);
+	amdgpu_gmc_vram_location(adev, &adev->gmc, base);
+	amdgpu_gmc_gart_location(adev, mc);
 }
 
 /**
drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c +2 -2
index 3fe9b9755cf7..fe26bd08b593 100644
@@ -410,8 +410,8 @@ static void gmc_v8_0_vram_gtt_location(struct amdgpu_device *adev,
 		base = RREG32(mmMC_VM_FB_LOCATION) & 0xFFFF;
 	base <<= 24;
 
-	amdgpu_device_vram_location(adev, &adev->gmc, base);
-	amdgpu_device_gart_location(adev, mc);
+	amdgpu_gmc_vram_location(adev, &adev->gmc, base);
+	amdgpu_gmc_gart_location(adev, mc);
 }
 
 /**
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +2 -2
index d982956c8329..60c8f848c102 100644
@@ -749,8 +749,8 @@ static void gmc_v9_0_vram_gtt_location(struct amdgpu_device *adev,
 	u64 base = 0;
 	if (!amdgpu_sriov_vf(adev))
 		base = mmhub_v1_0_get_fb_location(adev);
-	amdgpu_device_vram_location(adev, &adev->gmc, base);
-	amdgpu_device_gart_location(adev, mc);
+	amdgpu_gmc_vram_location(adev, &adev->gmc, base);
+	amdgpu_gmc_gart_location(adev, mc);
 	/* base offset of vram pages */
 	adev->vm_manager.vram_base_offset = gfxhub_v1_0_get_mc_fb_offset(adev);
 }
Signature · 6 lines
2.17.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20180827165337.1651-2-christian.koenig-5C7GfCeVMHo@public.gmane.org

Reviewed-by
#4 lore raw

Re: [PATCH 02/10] drm/amdgpu: move amdgpu_device_(vram|gtt)_location

Quoted 5 lines · Christian König

Move that into amdgpu_gmc.c since we are really deadling with GMC address space here.

Quoted 251 lines
---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  4 --
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 65 ----------------------
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c    | 64 +++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h    |  4 ++
 drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c      |  4 +-
 drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c      |  4 +-
 drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c      |  4 +-
 drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c      |  4 +-
 8 files changed, 76 insertions(+), 77 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +0 -4
index 340e40d03d54..09bdedfc91c7 100644
@@ -1166,10 +1166,6 @@ bool amdgpu_device_need_post(struct amdgpu_device *adev);
 
 void amdgpu_cs_report_moved_bytes(struct amdgpu_device *adev, u64 num_bytes,
                                  u64 num_vis_bytes);
-void amdgpu_device_vram_location(struct amdgpu_device *adev,
-                                struct amdgpu_gmc *mc, u64 base);
-void amdgpu_device_gart_location(struct amdgpu_device *adev,
-                                struct amdgpu_gmc *mc);
 int amdgpu_device_resize_fb_bar(struct amdgpu_device *adev);
 void amdgpu_device_program_register_sequence(struct amdgpu_device *adev,
                                             const u32 *registers,
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -65
index 9f5e4be76d5e..0afc5e599683 100644
@@ -651,71 +651,6 @@ void amdgpu_device_wb_free(struct amdgpu_device *adev, u32 wb)
                __clear_bit(wb, adev->wb.used);
 }
 
-/**
- * amdgpu_device_vram_location - try to find VRAM location
- *
- * @adev: amdgpu device structure holding all necessary informations
- * @mc: memory controller structure holding memory informations
- * @base: base address at which to put VRAM
- *
- * Function will try to place VRAM at base address provided
- * as parameter.
- */
-void amdgpu_device_vram_location(struct amdgpu_device *adev,
-                                struct amdgpu_gmc *mc, u64 base)
-{
-       uint64_t limit = (uint64_t)amdgpu_vram_limit << 20;
-
-       mc->vram_start = base;
-       mc->vram_end = mc->vram_start + mc->mc_vram_size - 1;
-       if (limit && limit < mc->real_vram_size)
-               mc->real_vram_size = limit;
-       dev_info(adev->dev, "VRAM: %lluM 0x%016llX - 0x%016llX (%lluM used)\n",
-                       mc->mc_vram_size >> 20, mc->vram_start,
-                       mc->vram_end, mc->real_vram_size >> 20);
-}
-
-/**
- * amdgpu_device_gart_location - try to find GART location
- *
- * @adev: amdgpu device structure holding all necessary informations
- * @mc: memory controller structure holding memory informations
- *
- * Function will place try to place GART before or after VRAM.
- *
- * If GART size is bigger than space left then we ajust GART size.
- * Thus function will never fails.
- */
-void amdgpu_device_gart_location(struct amdgpu_device *adev,
-                                struct amdgpu_gmc *mc)
-{
-       u64 size_af, size_bf;
-
-       mc->gart_size += adev->pm.smu_prv_buffer_size;
-
-       size_af = adev->gmc.mc_mask - mc->vram_end;
-       size_bf = mc->vram_start;
-       if (size_bf > size_af) {
-               if (mc->gart_size > size_bf) {
-                       dev_warn(adev->dev, "limiting GART\n");
-                       mc->gart_size = size_bf;
-               }
-               mc->gart_start = 0;
-       } else {
-               if (mc->gart_size > size_af) {
-                       dev_warn(adev->dev, "limiting GART\n");
-                       mc->gart_size = size_af;
-               }
-               /* VCE doesn't like it when BOs cross a 4GB segment, so align
-                * the GART base on a 4GB boundary as well.
-                */
-               mc->gart_start = ALIGN(mc->vram_end + 1, 0x100000000ULL);
-       }
-       mc->gart_end = mc->gart_start + mc->gart_size - 1;
-       dev_info(adev->dev, "GART: %lluM 0x%016llX - 0x%016llX\n",
-                       mc->gart_size >> 20, mc->gart_start, mc->gart_end);
-}
-
 /**
  * amdgpu_device_resize_fb_bar - try to resize FB BAR
  *
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +64 -0
index a249931ef512..72dffa3fd194 100644
@@ -78,3 +78,67 @@ uint64_t amdgpu_gmc_pd_addr(struct amdgpu_bo *bo)
        }
        return pd_addr;
 }
+
+/**
+ * amdgpu_gmc_vram_location - try to find VRAM location
+ *
+ * @adev: amdgpu device structure holding all necessary informations
+ * @mc: memory controller structure holding memory informations
+ * @base: base address at which to put VRAM
+ *
+ * Function will try to place VRAM at base address provided
+ * as parameter.
+ */
+void amdgpu_gmc_vram_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc,
+                             u64 base)
+{
+       uint64_t limit = (uint64_t)amdgpu_vram_limit << 20;
+
+       mc->vram_start = base;
+       mc->vram_end = mc->vram_start + mc->mc_vram_size - 1;
+       if (limit && limit < mc->real_vram_size)
+               mc->real_vram_size = limit;
+       dev_info(adev->dev, "VRAM: %lluM 0x%016llX - 0x%016llX (%lluM used)\n",
+                       mc->mc_vram_size >> 20, mc->vram_start,
+                       mc->vram_end, mc->real_vram_size >> 20);
+}
+
+/**
+ * amdgpu_gmc_gart_location - try to find GART location
+ *
+ * @adev: amdgpu device structure holding all necessary informations
+ * @mc: memory controller structure holding memory informations
+ *
+ * Function will place try to place GART before or after VRAM.
+ *
+ * If GART size is bigger than space left then we ajust GART size.
+ * Thus function will never fails.
+ */
+void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
+{
+       u64 size_af, size_bf;
+
+       mc->gart_size += adev->pm.smu_prv_buffer_size;
+
+       size_af = adev->gmc.mc_mask - mc->vram_end;
+       size_bf = mc->vram_start;
+       if (size_bf > size_af) {
+               if (mc->gart_size > size_bf) {
+                       dev_warn(adev->dev, "limiting GART\n");
+                       mc->gart_size = size_bf;
+               }
+               mc->gart_start = 0;
+       } else {
+               if (mc->gart_size > size_af) {
+                       dev_warn(adev->dev, "limiting GART\n");
+                       mc->gart_size = size_af;
+               }
+               /* VCE doesn't like it when BOs cross a 4GB segment, so align
+                * the GART base on a 4GB boundary as well.
+                */
+               mc->gart_start = ALIGN(mc->vram_end + 1, 0x100000000ULL);
+       }
+       mc->gart_end = mc->gart_start + mc->gart_size - 1;
+       dev_info(adev->dev, "GART: %lluM 0x%016llX - 0x%016llX\n",
+                       mc->gart_size >> 20, mc->gart_start, mc->gart_end);
+}
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h +4 -0
index 0d2c9f65ca13..72fcc9338f5e 100644
@@ -134,5 +134,9 @@ static inline bool amdgpu_gmc_vram_full_visible(struct amdgpu_gmc *gmc)
 void amdgpu_gmc_get_pde_for_bo(struct amdgpu_bo *bo, int level,
                               uint64_t *addr, uint64_t *flags);
 uint64_t amdgpu_gmc_pd_addr(struct amdgpu_bo *bo);
+void amdgpu_gmc_vram_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc,
+                             u64 base);
+void amdgpu_gmc_gart_location(struct amdgpu_device *adev,
+                             struct amdgpu_gmc *mc);
 
 #endif
drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c +2 -2
index 543287e5d67b..c2866e1acbef 100644
@@ -224,8 +224,8 @@ static void gmc_v6_0_vram_gtt_location(struct amdgpu_device *adev,
        u64 base = RREG32(mmMC_VM_FB_LOCATION) & 0xFFFF;
        base <<= 24;
 
-       amdgpu_device_vram_location(adev, &adev->gmc, base);
-       amdgpu_device_gart_location(adev, mc);
+       amdgpu_gmc_vram_location(adev, &adev->gmc, base);
+       amdgpu_gmc_gart_location(adev, mc);
 }
 
 static void gmc_v6_0_mc_program(struct amdgpu_device *adev)
drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +2 -2
index 060c79afef80..cbbbb6bbf782 100644
@@ -244,8 +244,8 @@ static void gmc_v7_0_vram_gtt_location(struct amdgpu_device *adev,
        u64 base = RREG32(mmMC_VM_FB_LOCATION) & 0xFFFF;
        base <<= 24;
 
-       amdgpu_device_vram_location(adev, &adev->gmc, base);
-       amdgpu_device_gart_location(adev, mc);
+       amdgpu_gmc_vram_location(adev, &adev->gmc, base);
+       amdgpu_gmc_gart_location(adev, mc);
 }
 
 /**
drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c +2 -2
index 3fe9b9755cf7..fe26bd08b593 100644
@@ -410,8 +410,8 @@ static void gmc_v8_0_vram_gtt_location(struct amdgpu_device *adev,
                base = RREG32(mmMC_VM_FB_LOCATION) & 0xFFFF;
        base <<= 24;
 
-       amdgpu_device_vram_location(adev, &adev->gmc, base);
-       amdgpu_device_gart_location(adev, mc);
+       amdgpu_gmc_vram_location(adev, &adev->gmc, base);
+       amdgpu_gmc_gart_location(adev, mc);
 }
 
 /**
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +2 -3
index d982956c8329..60c8f848c102 100644
@@ -749,8 +749,8 @@ static void gmc_v9_0_vram_gtt_location(struct amdgpu_device *adev,
        u64 base = 0;
        if (!amdgpu_sriov_vf(adev))
                base = mmhub_v1_0_get_fb_location(adev);
-       amdgpu_device_vram_location(adev, &adev->gmc, base);
-       amdgpu_device_gart_location(adev, mc);
+       amdgpu_gmc_vram_location(adev, &adev->gmc, base);
+       amdgpu_gmc_gart_location(adev, mc);
        /* base offset of vram pages */
        adev->vm_manager.vram_base_offset = gfxhub_v1_0_get_mc_fb_offset(adev);
 }
--

2.17.1

Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): CADnq5_OPp_EgorzVizo4LpcdJj38daZbRrF0aSRg8emouKLzEg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

Reviewed-by Junwei Zhang
#6 lore raw

Re: [PATCH 02/10] drm/amdgpu: move amdgpu_device_(vram|gtt)_location

Quoted 9 lines · Alex Deucher
Quoted 5 lines · Christian König

Move that into amdgpu_gmc.c since we are really deadling with GMC address space here.

Quoted 252 lines
Quoted 252 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  4 --
  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 65 ----------------------
  drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c    | 64 +++++++++++++++++++++
  drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h    |  4 ++
  drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c      |  4 +-
  drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c      |  4 +-
  drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c      |  4 +-
  drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c      |  4 +-
  8 files changed, 76 insertions(+), 77 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +0 -4
index 340e40d03d54..09bdedfc91c7 100644
@@ -1166,10 +1166,6 @@ bool amdgpu_device_need_post(struct amdgpu_device *adev);
 
  void amdgpu_cs_report_moved_bytes(struct amdgpu_device *adev, u64 num_bytes,
                                   u64 num_vis_bytes);
-void amdgpu_device_vram_location(struct amdgpu_device *adev,
-                                struct amdgpu_gmc *mc, u64 base);
-void amdgpu_device_gart_location(struct amdgpu_device *adev,
-                                struct amdgpu_gmc *mc);
  int amdgpu_device_resize_fb_bar(struct amdgpu_device *adev);
  void amdgpu_device_program_register_sequence(struct amdgpu_device *adev,
                                              const u32 *registers,
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -65
index 9f5e4be76d5e..0afc5e599683 100644
@@ -651,71 +651,6 @@ void amdgpu_device_wb_free(struct amdgpu_device *adev, u32 wb)
                 __clear_bit(wb, adev->wb.used);
  }
 
-/**
- * amdgpu_device_vram_location - try to find VRAM location
- *
- * @adev: amdgpu device structure holding all necessary informations
- * @mc: memory controller structure holding memory informations
- * @base: base address at which to put VRAM
- *
- * Function will try to place VRAM at base address provided
- * as parameter.
- */
-void amdgpu_device_vram_location(struct amdgpu_device *adev,
-                                struct amdgpu_gmc *mc, u64 base)
-{
-       uint64_t limit = (uint64_t)amdgpu_vram_limit << 20;
-
-       mc->vram_start = base;
-       mc->vram_end = mc->vram_start + mc->mc_vram_size - 1;
-       if (limit && limit < mc->real_vram_size)
-               mc->real_vram_size = limit;
-       dev_info(adev->dev, "VRAM: %lluM 0x%016llX - 0x%016llX (%lluM used)\n",
-                       mc->mc_vram_size >> 20, mc->vram_start,
-                       mc->vram_end, mc->real_vram_size >> 20);
-}
-
-/**
- * amdgpu_device_gart_location - try to find GART location
- *
- * @adev: amdgpu device structure holding all necessary informations
- * @mc: memory controller structure holding memory informations
- *
- * Function will place try to place GART before or after VRAM.
- *
- * If GART size is bigger than space left then we ajust GART size.
- * Thus function will never fails.
- */
-void amdgpu_device_gart_location(struct amdgpu_device *adev,
-                                struct amdgpu_gmc *mc)
-{
-       u64 size_af, size_bf;
-
-       mc->gart_size += adev->pm.smu_prv_buffer_size;
-
-       size_af = adev->gmc.mc_mask - mc->vram_end;
-       size_bf = mc->vram_start;
-       if (size_bf > size_af) {
-               if (mc->gart_size > size_bf) {
-                       dev_warn(adev->dev, "limiting GART\n");
-                       mc->gart_size = size_bf;
-               }
-               mc->gart_start = 0;
-       } else {
-               if (mc->gart_size > size_af) {
-                       dev_warn(adev->dev, "limiting GART\n");
-                       mc->gart_size = size_af;
-               }
-               /* VCE doesn't like it when BOs cross a 4GB segment, so align
-                * the GART base on a 4GB boundary as well.
-                */
-               mc->gart_start = ALIGN(mc->vram_end + 1, 0x100000000ULL);
-       }
-       mc->gart_end = mc->gart_start + mc->gart_size - 1;
-       dev_info(adev->dev, "GART: %lluM 0x%016llX - 0x%016llX\n",
-                       mc->gart_size >> 20, mc->gart_start, mc->gart_end);
-}
-
  /**
   * amdgpu_device_resize_fb_bar - try to resize FB BAR
   *
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +64 -0
index a249931ef512..72dffa3fd194 100644
@@ -78,3 +78,67 @@ uint64_t amdgpu_gmc_pd_addr(struct amdgpu_bo *bo)
         }
         return pd_addr;
  }
+
+/**
+ * amdgpu_gmc_vram_location - try to find VRAM location
+ *
+ * @adev: amdgpu device structure holding all necessary informations
+ * @mc: memory controller structure holding memory informations
+ * @base: base address at which to put VRAM
+ *
+ * Function will try to place VRAM at base address provided
+ * as parameter.
+ */
+void amdgpu_gmc_vram_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc,
+                             u64 base)
+{
+       uint64_t limit = (uint64_t)amdgpu_vram_limit << 20;
+
+       mc->vram_start = base;
+       mc->vram_end = mc->vram_start + mc->mc_vram_size - 1;
+       if (limit && limit < mc->real_vram_size)
+               mc->real_vram_size = limit;
+       dev_info(adev->dev, "VRAM: %lluM 0x%016llX - 0x%016llX (%lluM used)\n",
+                       mc->mc_vram_size >> 20, mc->vram_start,
+                       mc->vram_end, mc->real_vram_size >> 20);
+}
+
+/**
+ * amdgpu_gmc_gart_location - try to find GART location
+ *
+ * @adev: amdgpu device structure holding all necessary informations
+ * @mc: memory controller structure holding memory informations
+ *
+ * Function will place try to place GART before or after VRAM.
+ *
+ * If GART size is bigger than space left then we ajust GART size.
+ * Thus function will never fails.
+ */
+void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
+{
+       u64 size_af, size_bf;
+
+       mc->gart_size += adev->pm.smu_prv_buffer_size;
+
+       size_af = adev->gmc.mc_mask - mc->vram_end;
+       size_bf = mc->vram_start;
+       if (size_bf > size_af) {
+               if (mc->gart_size > size_bf) {
+                       dev_warn(adev->dev, "limiting GART\n");
+                       mc->gart_size = size_bf;
+               }
+               mc->gart_start = 0;
+       } else {
+               if (mc->gart_size > size_af) {
+                       dev_warn(adev->dev, "limiting GART\n");
+                       mc->gart_size = size_af;
+               }
+               /* VCE doesn't like it when BOs cross a 4GB segment, so align
+                * the GART base on a 4GB boundary as well.
+                */
+               mc->gart_start = ALIGN(mc->vram_end + 1, 0x100000000ULL);
+       }
+       mc->gart_end = mc->gart_start + mc->gart_size - 1;
+       dev_info(adev->dev, "GART: %lluM 0x%016llX - 0x%016llX\n",
+                       mc->gart_size >> 20, mc->gart_start, mc->gart_end);
+}
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h +4 -0
index 0d2c9f65ca13..72fcc9338f5e 100644
@@ -134,5 +134,9 @@ static inline bool amdgpu_gmc_vram_full_visible(struct amdgpu_gmc *gmc)
  void amdgpu_gmc_get_pde_for_bo(struct amdgpu_bo *bo, int level,
                                uint64_t *addr, uint64_t *flags);
  uint64_t amdgpu_gmc_pd_addr(struct amdgpu_bo *bo);
+void amdgpu_gmc_vram_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc,
+                             u64 base);
+void amdgpu_gmc_gart_location(struct amdgpu_device *adev,
+                             struct amdgpu_gmc *mc);
 
  #endif
drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c +2 -2
index 543287e5d67b..c2866e1acbef 100644
@@ -224,8 +224,8 @@ static void gmc_v6_0_vram_gtt_location(struct amdgpu_device *adev,
         u64 base = RREG32(mmMC_VM_FB_LOCATION) & 0xFFFF;
         base <<= 24;
 
-       amdgpu_device_vram_location(adev, &adev->gmc, base);
-       amdgpu_device_gart_location(adev, mc);
+       amdgpu_gmc_vram_location(adev, &adev->gmc, base);
+       amdgpu_gmc_gart_location(adev, mc);
  }
 
  static void gmc_v6_0_mc_program(struct amdgpu_device *adev)
drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +2 -2
index 060c79afef80..cbbbb6bbf782 100644
@@ -244,8 +244,8 @@ static void gmc_v7_0_vram_gtt_location(struct amdgpu_device *adev,
         u64 base = RREG32(mmMC_VM_FB_LOCATION) & 0xFFFF;
         base <<= 24;
 
-       amdgpu_device_vram_location(adev, &adev->gmc, base);
-       amdgpu_device_gart_location(adev, mc);
+       amdgpu_gmc_vram_location(adev, &adev->gmc, base);
+       amdgpu_gmc_gart_location(adev, mc);
  }
 
  /**
drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c +2 -2
index 3fe9b9755cf7..fe26bd08b593 100644
@@ -410,8 +410,8 @@ static void gmc_v8_0_vram_gtt_location(struct amdgpu_device *adev,
                 base = RREG32(mmMC_VM_FB_LOCATION) & 0xFFFF;
         base <<= 24;
 
-       amdgpu_device_vram_location(adev, &adev->gmc, base);
-       amdgpu_device_gart_location(adev, mc);
+       amdgpu_gmc_vram_location(adev, &adev->gmc, base);
+       amdgpu_gmc_gart_location(adev, mc);
  }
 
  /**
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +2 -3
index d982956c8329..60c8f848c102 100644
@@ -749,8 +749,8 @@ static void gmc_v9_0_vram_gtt_location(struct amdgpu_device *adev,
         u64 base = 0;
         if (!amdgpu_sriov_vf(adev))
                 base = mmhub_v1_0_get_fb_location(adev);
-       amdgpu_device_vram_location(adev, &adev->gmc, base);
-       amdgpu_device_gart_location(adev, mc);
+       amdgpu_gmc_vram_location(adev, &adev->gmc, base);
+       amdgpu_gmc_gart_location(adev, mc);
         /* base offset of vram pages */
         adev->vm_manager.vram_base_offset = gfxhub_v1_0_get_mc_fb_offset(adev);
  }
--

2.17.1

Quoted 9 lines
Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#7 lore raw

PATCH 3/10 drm/amdgpu: fix amdgpu_gmc_gart_location a little bit

In reply to a message not in the archive

Improve the VCE limitation handling.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 28 ++++++++++++-------------
 1 file changed, 13 insertions(+), 15 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +13 -15
index 72dffa3fd194..8269197df8e0 100644
@@ -120,24 +120,22 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
 
 	mc->gart_size += adev->pm.smu_prv_buffer_size;
 
-	size_af = adev->gmc.mc_mask - mc->vram_end;
+	/* VCE doesn't like it when BOs cross a 4GB segment, so align
+	 * the GART base on a 4GB boundary as well.
+	 */
 	size_bf = mc->vram_start;
-	if (size_bf > size_af) {
-		if (mc->gart_size > size_bf) {
-			dev_warn(adev->dev, "limiting GART\n");
-			mc->gart_size = size_bf;
-		}
+	size_af = adev->gmc.mc_mask + 1 -
+		ALIGN(mc->vram_end + 1, 0x100000000ULL);
+
+	if (mc->gart_size > max(size_bf, size_af)) {
+		dev_warn(adev->dev, "limiting GART\n");
+		mc->gart_size = max(size_bf, size_af);
+	}
+
+	if (size_bf > size_af)
 		mc->gart_start = 0;
-	} else {
-		if (mc->gart_size > size_af) {
-			dev_warn(adev->dev, "limiting GART\n");
-			mc->gart_size = size_af;
-		}
-		/* VCE doesn't like it when BOs cross a 4GB segment, so align
-		 * the GART base on a 4GB boundary as well.
-		 */
+	else
 		mc->gart_start = ALIGN(mc->vram_end + 1, 0x100000000ULL);
-	}
 	mc->gart_end = mc->gart_start + mc->gart_size - 1;
 	dev_info(adev->dev, "GART: %lluM 0x%016llX - 0x%016llX\n",
 			mc->gart_size >> 20, mc->gart_start, mc->gart_end);
Signature · 6 lines
2.17.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20180827165337.1651-3-christian.koenig-5C7GfCeVMHo@public.gmane.org

Reviewed-by
#9 lore raw

Re: [PATCH 03/10] drm/amdgpu: fix amdgpu_gmc_gart_location a little bit

Quoted 18 lines · Christian König

Improve the VCE limitation handling.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 28 ++++++++++++-------------
 1 file changed, 13 insertions(+), 15 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +1 -1
index 72dffa3fd194..8269197df8e0 100644
@@ -120,24 +120,22 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
 
        mc->gart_size += adev->pm.smu_prv_buffer_size;
 
-       size_af = adev->gmc.mc_mask - mc->vram_end;
+       /* VCE doesn't like it when BOs cross a 4GB segment, so align

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc) · line 128

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +4 -5
+        * the GART base on a 4GB boundary as well.
+        */
        size_bf = mc->vram_start;
-       if (size_bf > size_af) {
-               if (mc->gart_size > size_bf) {
-                       dev_warn(adev->dev, "limiting GART\n");
-                       mc->gart_size = size_bf;
-               }
+       size_af = adev->gmc.mc_mask + 1 -
+               ALIGN(mc->vram_end + 1, 0x100000000ULL);

Is it worth limiting this to asics with VCE support? Probably not a big deal either way.

Quoted 25 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +8 -10
+
+       if (mc->gart_size > max(size_bf, size_af)) {
+               dev_warn(adev->dev, "limiting GART\n");
+               mc->gart_size = max(size_bf, size_af);
+       }
+
+       if (size_bf > size_af)
                mc->gart_start = 0;
-       } else {
-               if (mc->gart_size > size_af) {
-                       dev_warn(adev->dev, "limiting GART\n");
-                       mc->gart_size = size_af;
-               }
-               /* VCE doesn't like it when BOs cross a 4GB segment, so align
-                * the GART base on a 4GB boundary as well.
-                */
+       else
                mc->gart_start = ALIGN(mc->vram_end + 1, 0x100000000ULL);
-       }
        mc->gart_end = mc->gart_start + mc->gart_size - 1;
        dev_info(adev->dev, "GART: %lluM 0x%016llX - 0x%016llX\n",
                        mc->gart_size >> 20, mc->gart_start, mc->gart_end);
--

2.17.1

Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): CADnq5_OmwbjRufiL30T9EZOrhu4suRDy417xSy=BfdGmb94XJA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#11 lore raw

Re: [PATCH 03/10] drm/amdgpu: fix amdgpu_gmc_gart_location a little bit

Am 27.08.2018 um 21:03 schrieb Alex Deucher:

Quoted 17 lines
Quoted 15 lines · Christian König

Improve the VCE limitation handling.

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 28 ++++++++++++-------------
  1 file changed, 13 insertions(+), 15 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +0 -0
index 72dffa3fd194..8269197df8e0 100644
@@ -120,24 +120,22 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
 
         mc->gart_size += adev->pm.smu_prv_buffer_size;
 
Quoted 13 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc) · line 128

Quoted 12 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +5 -6
-       size_af = adev->gmc.mc_mask - mc->vram_end;
+       /* VCE doesn't like it when BOs cross a 4GB segment, so align
+        * the GART base on a 4GB boundary as well.
+        */
         size_bf = mc->vram_start;
-       if (size_bf > size_af) {
-               if (mc->gart_size > size_bf) {
-                       dev_warn(adev->dev, "limiting GART\n");
-                       mc->gart_size = size_bf;
-               }
+       size_af = adev->gmc.mc_mask + 1 -
+               ALIGN(mc->vram_end + 1, 0x100000000ULL);

Is it worth limiting this to asics with VCE support?

Actually I think that a couple of blocks might have a problem with that. E.g. DCE/DCN seem to use a similar approach as VCE/VCN. Not 100% sure about UVD.

Only GFX/Compute/SDMA seems to have a true 48bit interface to the MC.

Regards,
Christian.

Quoted 29 lines

Probably not a
big deal either way.

Quoted 25 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +8 -10
+
+       if (mc->gart_size > max(size_bf, size_af)) {
+               dev_warn(adev->dev, "limiting GART\n");
+               mc->gart_size = max(size_bf, size_af);
+       }
+
+       if (size_bf > size_af)
                 mc->gart_start = 0;
-       } else {
-               if (mc->gart_size > size_af) {
-                       dev_warn(adev->dev, "limiting GART\n");
-                       mc->gart_size = size_af;
-               }
-               /* VCE doesn't like it when BOs cross a 4GB segment, so align
-                * the GART base on a 4GB boundary as well.
-                */
+       else
                 mc->gart_start = ALIGN(mc->vram_end + 1, 0x100000000ULL);
-       }
         mc->gart_end = mc->gart_start + mc->gart_size - 1;
         dev_info(adev->dev, "GART: %lluM 0x%016llX - 0x%016llX\n",
                         mc->gart_size >> 20, mc->gart_start, mc->gart_end);
--

2.17.1

Quoted 4 lines
Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Reviewed-by Junwei Zhang
#12 lore raw

Re: [PATCH 03/10] drm/amdgpu: fix amdgpu_gmc_gart_location a little bit

In reply to a message not in the archive
Quoted 3 lines · Christian König

Improve the VCE limitation handling.

Quoted 37 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 28 ++++++++++++-------------
  1 file changed, 13 insertions(+), 15 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +12 -10
index 72dffa3fd194..8269197df8e0 100644
@@ -120,24 +120,22 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
 
  	mc->gart_size += adev->pm.smu_prv_buffer_size;
 
-	size_af = adev->gmc.mc_mask - mc->vram_end;
+	/* VCE doesn't like it when BOs cross a 4GB segment, so align
+	 * the GART base on a 4GB boundary as well.
+	 */
  	size_bf = mc->vram_start;
-	if (size_bf > size_af) {
-		if (mc->gart_size > size_bf) {
-			dev_warn(adev->dev, "limiting GART\n");
-			mc->gart_size = size_bf;
-		}
+	size_af = adev->gmc.mc_mask + 1 -
+		ALIGN(mc->vram_end + 1, 0x100000000ULL);
+
+	if (mc->gart_size > max(size_bf, size_af)) {
+		dev_warn(adev->dev, "limiting GART\n");
+		mc->gart_size = max(size_bf, size_af);
+	}
+
+	if (size_bf > size_af)
  		mc->gart_start = 0;
-	} else {
-		if (mc->gart_size > size_af) {
-			dev_warn(adev->dev, "limiting GART\n");
-			mc->gart_size = size_af;

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc) · line 141

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +1 -5
-		}
-		/* VCE doesn't like it when BOs cross a 4GB segment, so align
-		 * the GART base on a 4GB boundary as well.
-		 */
+	else
  		mc->gart_start = ALIGN(mc->vram_end + 1, 0x100000000ULL);
-	}
  	mc->gart_end = mc->gart_start + mc->gart_size - 1;
  	dev_info(adev->dev, "GART: %lluM 0x%016llX - 0x%016llX\n",
  			mc->gart_size >> 20, mc->gart_start, mc->gart_end);

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#13 lore raw

PATCH 4/10 drm/amdgpu: use the smaller hole for GART

In reply to a message not in the archive

Instead of the larger one use the smaller hole in the MC address space for the GART mappings.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +2 -1
index 8269197df8e0..265ec6807130 100644
@@ -132,7 +132,8 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
 		mc->gart_size = max(size_bf, size_af);
 	}
 
-	if (size_bf > size_af)
+	if ((size_bf >= mc->gart_size && size_bf < size_af) ||
+	    (size_af < mc->gart_size))
 		mc->gart_start = 0;
 	else
 		mc->gart_start = ALIGN(mc->vram_end + 1, 0x100000000ULL);
Signature · 6 lines
2.17.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20180827165337.1651-4-christian.koenig-5C7GfCeVMHo@public.gmane.org

Reviewed-by
#15 lore raw

Re: [PATCH 04/10] drm/amdgpu: use the smaller hole for GART

Quoted 5 lines · Christian König

Instead of the larger one use the smaller hole in the MC address space for the GART mappings.

Quoted 21 lines
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +2 -2
index 8269197df8e0..265ec6807130 100644
@@ -132,7 +132,8 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
                mc->gart_size = max(size_bf, size_af);
        }
 
-       if (size_bf > size_af)
+       if ((size_bf >= mc->gart_size && size_bf < size_af) ||
+           (size_af < mc->gart_size))
                mc->gart_start = 0;
        else
                mc->gart_start = ALIGN(mc->vram_end + 1, 0x100000000ULL);
--

2.17.1

Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): CADnq5_OB2HV16p4_idyqowKDTZxZkJF5GmKGnHG5+DPm2=4qEQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

Reviewed-by Junwei Zhang
#17 lore raw

Re: [PATCH 04/10] drm/amdgpu: use the smaller hole for GART

Quoted 9 lines · Alex Deucher
Quoted 5 lines · Christian König

Instead of the larger one use the smaller hole in the MC address space for the GART mappings.

Quoted 22 lines
Quoted 22 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +2 -2
index 8269197df8e0..265ec6807130 100644
@@ -132,7 +132,8 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
                 mc->gart_size = max(size_bf, size_af);
         }
 
-       if (size_bf > size_af)
+       if ((size_bf >= mc->gart_size && size_bf < size_af) ||
+           (size_af < mc->gart_size))
                 mc->gart_start = 0;
         else
                 mc->gart_start = ALIGN(mc->vram_end + 1, 0x100000000ULL);
--

2.17.1

Quoted 9 lines
Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#18 lore raw

PATCH 5/10 drm/amdgpu: put GART away from VRAM

In reply to a message not in the archive

Always try to put the GART away from where VRAM is.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +1 -1
index 265ec6807130..4331a0e25cdc 100644
@@ -136,7 +136,7 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
 	    (size_af < mc->gart_size))
 		mc->gart_start = 0;
 	else
-		mc->gart_start = ALIGN(mc->vram_end + 1, 0x100000000ULL);
+		mc->gart_start = mc->mc_mask - mc->gart_size + 1;
 	mc->gart_end = mc->gart_start + mc->gart_size - 1;
 	dev_info(adev->dev, "GART: %lluM 0x%016llX - 0x%016llX\n",
 			mc->gart_size >> 20, mc->gart_start, mc->gart_end);
Signature · 6 lines
2.17.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20180827165337.1651-5-christian.koenig-5C7GfCeVMHo@public.gmane.org

#20 lore raw

Re: [PATCH 05/10] drm/amdgpu: put GART away from VRAM

Quoted 12 lines · Christian König

Always try to put the GART away from where VRAM is.

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +0 -0
index 265ec6807130..4331a0e25cdc 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc) · line 139

Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +1 -1
@@ -136,7 +136,7 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
  	    (size_af < mc->gart_size))
  		mc->gart_start = 0;
  	else
-		mc->gart_start = ALIGN(mc->vram_end + 1, 0x100000000ULL);
+		mc->gart_start = mc->mc_mask - mc->gart_size + 1;

it seems to break the VCE limitation about 4G alignment?

Jerry

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc) · line 142

Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +0 -0
  	mc->gart_end = mc->gart_start + mc->gart_size - 1;
  	dev_info(adev->dev, "GART: %lluM 0x%016llX - 0x%016llX\n",
  			mc->gart_size >> 20, mc->gart_start, mc->gart_end);

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#21 lore raw

PATCH 6/10 drm/amdgpu: add amdgpu_gmc_agp_location

In reply to a message not in the archive

Helper to figure out the location of the AGP BAR.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 42 +++++++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h |  5 +++
 2 files changed, 47 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +42 -0
index 4331a0e25cdc..eed5352f3136 100644
@@ -141,3 +141,45 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
 	dev_info(adev->dev, "GART: %lluM 0x%016llX - 0x%016llX\n",
 			mc->gart_size >> 20, mc->gart_start, mc->gart_end);
 }
+
+/**
+ * amdgpu_gmc_agp_location - try to find AGP location
+ * @adev: amdgpu device structure holding all necessary informations
+ * @mc: memory controller structure holding memory informations
+ *
+ * Function will place try to fina a place for the AGP BAR in the MC address
+ * space.
+ *
+ * AGP BAR will be assigned the largest available hole in the address space.
+ */
+void amdgpu_gmc_agp_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
+{
+	const uint64_t sixteen_gb = 1ULL << 34;
+	u64 size_af, size_bf;
+
+	if (mc->vram_start > mc->gart_start) {
+		size_bf = mc->vram_start - mc->gart_end + 1;
+		size_af = mc->mc_mask - mc->vram_end;
+	} else {
+		size_bf = mc->vram_start;
+		size_af = mc->mc_mask - mc->gart_end;
+	}
+
+	size_bf &= ~(sixteen_gb - 1);
+	size_af &= ~(sixteen_gb - 1);
+
+	if (size_bf > size_af) {
+		mc->agp_start = mc->vram_start > mc->gart_start ?
+			mc->gart_start : 0;
+		mc->agp_size = size_bf;
+	} else {
+		mc->agp_start = (mc->vram_start > mc->gart_start ?
+			mc->vram_end : mc->gart_end) + 1,
+		mc->agp_size = size_af;
+	}
+
+	mc->agp_start = ALIGN(mc->agp_start, sixteen_gb);
+	mc->agp_end = mc->agp_start + mc->agp_size - 1;
+	dev_info(adev->dev, "AGP: %lluM 0x%016llX - 0x%016llX\n",
+			mc->agp_size >> 20, mc->agp_start, mc->agp_end);
+}
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h +5 -0
index 72fcc9338f5e..163110fe375d 100644
@@ -81,6 +81,9 @@ struct amdgpu_gmc {
 	 * about vram size near mc fb location */
 	u64			mc_vram_size;
 	u64			visible_vram_size;
+	u64			agp_size;
+	u64			agp_start;
+	u64			agp_end;
 	u64			gart_size;
 	u64			gart_start;
 	u64			gart_end;
@@ -138,5 +141,7 @@ void amdgpu_gmc_vram_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc,
 			      u64 base);
 void amdgpu_gmc_gart_location(struct amdgpu_device *adev,
 			      struct amdgpu_gmc *mc);
+void amdgpu_gmc_agp_location(struct amdgpu_device *adev,
+			     struct amdgpu_gmc *mc);
 
 #endif
Signature · 6 lines
2.17.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20180827165337.1651-6-christian.koenig-5C7GfCeVMHo@public.gmane.org

#23 lore raw

Re: [PATCH 06/10] drm/amdgpu: add amdgpu_gmc_agp_location

Quoted 13 lines · Christian König

Helper to figure out the location of the AGP BAR.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 42 +++++++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h |  5 +++
 2 files changed, 47 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +0 -0
index 4331a0e25cdc..eed5352f3136 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc) · line 150

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +7 -0
@@ -141,3 +141,45 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
 	dev_info(adev->dev, "GART: %lluM 0x%016llX - 0x%016llX\n",
 			mc->gart_size >> 20, mc->gart_start, mc->gart_end);
 }
+
+/**
+ * amdgpu_gmc_agp_location - try to find AGP location
+ * @adev: amdgpu device structure holding all necessary informations
+ * @mc: memory controller structure holding memory informations
+ *
+ * Function will place try to fina a place for the AGP BAR in the MC address

s/fina/find

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc) · line 153

Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +3 -0
+ * space.
+ *
+ * AGP BAR will be assigned the largest available hole in the address space.

I'd add a comment that this function must be called after amdgpu_gmc_vram_location and amdgpu_gmc_gart_location.

Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +6 -0
+ */
+void amdgpu_gmc_agp_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
+{
+	const uint64_t sixteen_gb = 1ULL << 34;
+	u64 size_af, size_bf;
+

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc) · line 169

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +10 -0
+	if (mc->vram_start > mc->gart_start) {
+		size_bf = mc->vram_start - mc->gart_end + 1;
+		size_af = mc->mc_mask - mc->vram_end;
+	} else {
+		size_bf = mc->vram_start;
+		size_af = mc->mc_mask - mc->gart_end;
+	}
+
+	size_bf &= ~(sixteen_gb - 1);
+	size_af &= ~(sixteen_gb - 1);

This is not correct. E.g. vram_end  = 12GB, gart_start = 28GB, size = 16GB. agp_start will be rounded up to 16GB and AGP will run into the GART aperture.

You need to align the addresses before calculating the sizes.

Regards,
  Felix

Quoted 31 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +16 -0
+
+	if (size_bf > size_af) {
+		mc->agp_start = mc->vram_start > mc->gart_start ?
+			mc->gart_start : 0;
+		mc->agp_size = size_bf;
+	} else {
+		mc->agp_start = (mc->vram_start > mc->gart_start ?
+			mc->vram_end : mc->gart_end) + 1,
+		mc->agp_size = size_af;
+	}
+
+	mc->agp_start = ALIGN(mc->agp_start, sixteen_gb);
+	mc->agp_end = mc->agp_start + mc->agp_size - 1;
+	dev_info(adev->dev, "AGP: %lluM 0x%016llX - 0x%016llX\n",
+			mc->agp_size >> 20, mc->agp_start, mc->agp_end);
+}
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h +3 -0
index 72fcc9338f5e..163110fe375d 100644
@@ -81,6 +81,9 @@ struct amdgpu_gmc {
 	 * about vram size near mc fb location */
 	u64			mc_vram_size;
 	u64			visible_vram_size;
+	u64			agp_size;
+	u64			agp_start;
+	u64			agp_end;
 	u64			gart_size;
 	u64			gart_start;
 	u64			gart_end;

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h void amdgpu_gmc_vram_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc, · line 147

Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h +2 -0
@@ -138,5 +141,7 @@ void amdgpu_gmc_vram_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc,
 			      u64 base);
 void amdgpu_gmc_gart_location(struct amdgpu_device *adev,
 			      struct amdgpu_gmc *mc);
+void amdgpu_gmc_agp_location(struct amdgpu_device *adev,
+			     struct amdgpu_gmc *mc);
 
 #endif

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 995a274e-8478-79ac-bed3-61fe6655a393-5C7GfCeVMHo@public.gmane.org

#25 lore raw

Re: [PATCH 06/10] drm/amdgpu: add amdgpu_gmc_agp_location

Am 27.08.2018 um 20:50 schrieb Felix Kuehling:

Quoted 52 lines · Christian König
Quoted 13 lines

Helper to figure out the location of the AGP BAR.

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 42 +++++++++++++++++++++++++
  drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h |  5 +++
  2 files changed, 47 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +0 -0
index 4331a0e25cdc..eed5352f3136 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc) · line 150

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +7 -0
@@ -141,3 +141,45 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
  	dev_info(adev->dev, "GART: %lluM 0x%016llX - 0x%016llX\n",
  			mc->gart_size >> 20, mc->gart_start, mc->gart_end);
  }
+
+/**
+ * amdgpu_gmc_agp_location - try to find AGP location
+ * @adev: amdgpu device structure holding all necessary informations
+ * @mc: memory controller structure holding memory informations
+ *
+ * Function will place try to fina a place for the AGP BAR in the MC address

s/fina/find

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc) · line 153

Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +3 -0
+ * space.
+ *
+ * AGP BAR will be assigned the largest available hole in the address space.

I'd add a comment that this function must be called after amdgpu_gmc_vram_location and amdgpu_gmc_gart_location.

Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +6 -0
+ */
+void amdgpu_gmc_agp_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
+{
+	const uint64_t sixteen_gb = 1ULL << 34;
+	u64 size_af, size_bf;
+

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc) · line 169

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +10 -0
+	if (mc->vram_start > mc->gart_start) {
+		size_bf = mc->vram_start - mc->gart_end + 1;
+		size_af = mc->mc_mask - mc->vram_end;
+	} else {
+		size_bf = mc->vram_start;
+		size_af = mc->mc_mask - mc->gart_end;
+	}
+
+	size_bf &= ~(sixteen_gb - 1);
+	size_af &= ~(sixteen_gb - 1);

This is not correct. E.g. vram_end  = 12GB, gart_start = 28GB, size = 16GB. agp_start will be rounded up to 16GB and AGP will run into the GART aperture.

Quoted 1 line

You need to align the addresses before calculating the sizes.

Ah, crap you are right. Going to fix this.

I'm really wondering if we shouldn't use "struct resource" for that handling?

This will only get more complicated in the future when even more apertures are enabled.

Alex what do you think?

Regards,
Christian.

Quoted 41 lines

Regards,
  Felix

Quoted 37 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +16 -0
+
+	if (size_bf > size_af) {
+		mc->agp_start = mc->vram_start > mc->gart_start ?
+			mc->gart_start : 0;
+		mc->agp_size = size_bf;
+	} else {
+		mc->agp_start = (mc->vram_start > mc->gart_start ?
+			mc->vram_end : mc->gart_end) + 1,
+		mc->agp_size = size_af;
+	}
+
+	mc->agp_start = ALIGN(mc->agp_start, sixteen_gb);
+	mc->agp_end = mc->agp_start + mc->agp_size - 1;
+	dev_info(adev->dev, "AGP: %lluM 0x%016llX - 0x%016llX\n",
+			mc->agp_size >> 20, mc->agp_start, mc->agp_end);
+}
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h +5 -0
index 72fcc9338f5e..163110fe375d 100644
@@ -81,6 +81,9 @@ struct amdgpu_gmc {
  	 * about vram size near mc fb location */
  	u64			mc_vram_size;
  	u64			visible_vram_size;
+	u64			agp_size;
+	u64			agp_start;
+	u64			agp_end;
  	u64			gart_size;
  	u64			gart_start;
  	u64			gart_end;
@@ -138,5 +141,7 @@ void amdgpu_gmc_vram_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc,
  			      u64 base);
  void amdgpu_gmc_gart_location(struct amdgpu_device *adev,
  			      struct amdgpu_gmc *mc);
+void amdgpu_gmc_agp_location(struct amdgpu_device *adev,
+			     struct amdgpu_gmc *mc);
 
Quoted 5 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h void amdgpu_gmc_vram_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc, · line 147

Quoted 1 line
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h +0 -0
  #endif

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#26 lore raw

Re: [PATCH 06/10] drm/amdgpu: add amdgpu_gmc_agp_location

In reply to a message not in the archive
Quoted 36 lines · Christian König

Helper to figure out the location of the AGP BAR.

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 42 +++++++++++++++++++++++++
  drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h |  5 +++
  2 files changed, 47 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +20 -0
index 4331a0e25cdc..eed5352f3136 100644
@@ -141,3 +141,45 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
  	dev_info(adev->dev, "GART: %lluM 0x%016llX - 0x%016llX\n",
  			mc->gart_size >> 20, mc->gart_start, mc->gart_end);
  }
+
+/**
+ * amdgpu_gmc_agp_location - try to find AGP location
+ * @adev: amdgpu device structure holding all necessary informations
+ * @mc: memory controller structure holding memory informations
+ *
+ * Function will place try to fina a place for the AGP BAR in the MC address
+ * space.
+ *
+ * AGP BAR will be assigned the largest available hole in the address space.
+ */
+void amdgpu_gmc_agp_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
+{
+	const uint64_t sixteen_gb = 1ULL << 34;
+	u64 size_af, size_bf;
+
+	if (mc->vram_start > mc->gart_start) {
+		size_bf = mc->vram_start - mc->gart_end + 1;
+		size_af = mc->mc_mask - mc->vram_end;
+	} else {

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc) · line 173

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +10 -0
+		size_bf = mc->vram_start;
+		size_af = mc->mc_mask - mc->gart_end;
+	}
+
+	size_bf &= ~(sixteen_gb - 1);
+	size_af &= ~(sixteen_gb - 1);
+
+	if (size_bf > size_af) {
+		mc->agp_start = mc->vram_start > mc->gart_start ?
+			mc->gart_start : 0;

Here looks mc->gart_end?

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc) · line 181

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +8 -0
+		mc->agp_size = size_bf;
+	} else {
+		mc->agp_start = (mc->vram_start > mc->gart_start ?
+			mc->vram_end : mc->gart_end) + 1,
+		mc->agp_size = size_af;
+	}
+
+	mc->agp_start = ALIGN(mc->agp_start, sixteen_gb);

still needs mc->apg_start + 1 for alignment?

Quoted 20 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +4 -0
+	mc->agp_end = mc->agp_start + mc->agp_size - 1;
+	dev_info(adev->dev, "AGP: %lluM 0x%016llX - 0x%016llX\n",
+			mc->agp_size >> 20, mc->agp_start, mc->agp_end);
+}
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h +3 -0
index 72fcc9338f5e..163110fe375d 100644
@@ -81,6 +81,9 @@ struct amdgpu_gmc {
  	 * about vram size near mc fb location */
  	u64			mc_vram_size;
  	u64			visible_vram_size;
+	u64			agp_size;
+	u64			agp_start;
+	u64			agp_end;
  	u64			gart_size;
  	u64			gart_start;
  	u64			gart_end;

drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h void amdgpu_gmc_vram_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc, · line 147

Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h +2 -0
@@ -138,5 +141,7 @@ void amdgpu_gmc_vram_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc,
  			      u64 base);
  void amdgpu_gmc_gart_location(struct amdgpu_device *adev,
  			      struct amdgpu_gmc *mc);
+void amdgpu_gmc_agp_location(struct amdgpu_device *adev,
+			     struct amdgpu_gmc *mc);
 
  #endif

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#27 lore raw

PATCH 7/10 drm/amdgpu: stop using gart_start as offset for the GTT domain

In reply to a message not in the archive

Further separate GART and GTT domain.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | 3 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c     | 6 +++---
 2 files changed, 5 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +2 -1
index da7b1b92d9cf..c2539f6821c0 100644
@@ -143,7 +143,8 @@ static int amdgpu_gtt_mgr_alloc(struct ttm_mem_type_manager *man,
 	spin_unlock(&mgr->lock);
 
 	if (!r)
-		mem->start = node->node.start;
+		mem->start = node->node.start +
+			(adev->gmc.gart_start >> PAGE_SHIFT);
 
 	return r;
 }
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +3 -3
index 2f304f9dd543..5cadf4f1ee2c 100644
@@ -188,7 +188,7 @@ static int amdgpu_init_mem_type(struct ttm_bo_device *bdev, uint32_t type,
 	case TTM_PL_TT:
 		/* GTT memory  */
 		man->func = &amdgpu_gtt_mgr_func;
-		man->gpu_offset = adev->gmc.gart_start;
+		man->gpu_offset = 0;
 		man->available_caching = TTM_PL_MASK_CACHING;
 		man->default_caching = TTM_PL_FLAG_CACHED;
 		man->flags = TTM_MEMTYPE_FLAG_MAPPABLE | TTM_MEMTYPE_FLAG_CMA;
@@ -1062,7 +1062,7 @@ static int amdgpu_ttm_backend_bind(struct ttm_tt *ttm,
 	flags = amdgpu_ttm_tt_pte_flags(adev, ttm, bo_mem);
 
 	/* bind pages into GART page tables */
-	gtt->offset = (u64)bo_mem->start << PAGE_SHIFT;
+	gtt->offset = ((u64)bo_mem->start << PAGE_SHIFT) - adev->gmc.gart_start;
 	r = amdgpu_gart_bind(adev, gtt->offset, ttm->num_pages,
 		ttm->pages, gtt->ttm.dma_address, flags);
 
@@ -1110,7 +1110,7 @@ int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo)
 	flags = amdgpu_ttm_tt_pte_flags(adev, bo->ttm, &tmp);
 
 	/* Bind pages */
-	gtt->offset = (u64)tmp.start << PAGE_SHIFT;
+	gtt->offset = ((u64)tmp.start << PAGE_SHIFT) - adev->gmc.gart_start;
 	r = amdgpu_ttm_gart_bind(adev, bo, flags);
 	if (unlikely(r)) {
 		ttm_bo_mem_put(bo, &tmp);
Signature · 6 lines
2.17.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20180827165337.1651-7-christian.koenig-5C7GfCeVMHo@public.gmane.org

Reviewed-by
#29 lore raw

Re: [PATCH 07/10] drm/amdgpu: stop using gart_start as offset for the GTT domain

Quoted 4 lines · Christian König

Further separate GART and GTT domain.

Quoted 53 lines
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | 3 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c     | 6 +++---
 2 files changed, 5 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +2 -1
index da7b1b92d9cf..c2539f6821c0 100644
@@ -143,7 +143,8 @@ static int amdgpu_gtt_mgr_alloc(struct ttm_mem_type_manager *man,
        spin_unlock(&mgr->lock);
 
        if (!r)
-               mem->start = node->node.start;
+               mem->start = node->node.start +
+                       (adev->gmc.gart_start >> PAGE_SHIFT);
 
        return r;
 }
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +3 -4
index 2f304f9dd543..5cadf4f1ee2c 100644
@@ -188,7 +188,7 @@ static int amdgpu_init_mem_type(struct ttm_bo_device *bdev, uint32_t type,
        case TTM_PL_TT:
                /* GTT memory  */
                man->func = &amdgpu_gtt_mgr_func;
-               man->gpu_offset = adev->gmc.gart_start;
+               man->gpu_offset = 0;
                man->available_caching = TTM_PL_MASK_CACHING;
                man->default_caching = TTM_PL_FLAG_CACHED;
                man->flags = TTM_MEMTYPE_FLAG_MAPPABLE | TTM_MEMTYPE_FLAG_CMA;
@@ -1062,7 +1062,7 @@ static int amdgpu_ttm_backend_bind(struct ttm_tt *ttm,
        flags = amdgpu_ttm_tt_pte_flags(adev, ttm, bo_mem);
 
        /* bind pages into GART page tables */
-       gtt->offset = (u64)bo_mem->start << PAGE_SHIFT;
+       gtt->offset = ((u64)bo_mem->start << PAGE_SHIFT) - adev->gmc.gart_start;
        r = amdgpu_gart_bind(adev, gtt->offset, ttm->num_pages,
                ttm->pages, gtt->ttm.dma_address, flags);
 
@@ -1110,7 +1110,7 @@ int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo)
        flags = amdgpu_ttm_tt_pte_flags(adev, bo->ttm, &tmp);
 
        /* Bind pages */
-       gtt->offset = (u64)tmp.start << PAGE_SHIFT;
+       gtt->offset = ((u64)tmp.start << PAGE_SHIFT) - adev->gmc.gart_start;
        r = amdgpu_ttm_gart_bind(adev, bo, flags);
        if (unlikely(r)) {
                ttm_bo_mem_put(bo, &tmp);
--

2.17.1

Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): CADnq5_M-ppsYV3dsUZEeHaLKyVOkVU1LsWJyyyWve0_eMD-7qw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

Reviewed-by Junwei Zhang
#31 lore raw

Re: [PATCH 07/10] drm/amdgpu: stop using gart_start as offset for the GTT domain

Quoted 8 lines · Alex Deucher
Quoted 4 lines · Christian König

Further separate GART and GTT domain.

Quoted 54 lines
Quoted 54 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | 3 ++-
  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c     | 6 +++---
  2 files changed, 5 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +2 -1
index da7b1b92d9cf..c2539f6821c0 100644
@@ -143,7 +143,8 @@ static int amdgpu_gtt_mgr_alloc(struct ttm_mem_type_manager *man,
         spin_unlock(&mgr->lock);
 
         if (!r)
-               mem->start = node->node.start;
+               mem->start = node->node.start +
+                       (adev->gmc.gart_start >> PAGE_SHIFT);
 
         return r;
  }
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +3 -4
index 2f304f9dd543..5cadf4f1ee2c 100644
@@ -188,7 +188,7 @@ static int amdgpu_init_mem_type(struct ttm_bo_device *bdev, uint32_t type,
         case TTM_PL_TT:
                 /* GTT memory  */
                 man->func = &amdgpu_gtt_mgr_func;
-               man->gpu_offset = adev->gmc.gart_start;
+               man->gpu_offset = 0;
                 man->available_caching = TTM_PL_MASK_CACHING;
                 man->default_caching = TTM_PL_FLAG_CACHED;
                 man->flags = TTM_MEMTYPE_FLAG_MAPPABLE | TTM_MEMTYPE_FLAG_CMA;
@@ -1062,7 +1062,7 @@ static int amdgpu_ttm_backend_bind(struct ttm_tt *ttm,
         flags = amdgpu_ttm_tt_pte_flags(adev, ttm, bo_mem);
 
         /* bind pages into GART page tables */
-       gtt->offset = (u64)bo_mem->start << PAGE_SHIFT;
+       gtt->offset = ((u64)bo_mem->start << PAGE_SHIFT) - adev->gmc.gart_start;
         r = amdgpu_gart_bind(adev, gtt->offset, ttm->num_pages,
                 ttm->pages, gtt->ttm.dma_address, flags);
 
@@ -1110,7 +1110,7 @@ int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo)
         flags = amdgpu_ttm_tt_pte_flags(adev, bo->ttm, &tmp);
 
         /* Bind pages */
-       gtt->offset = (u64)tmp.start << PAGE_SHIFT;
+       gtt->offset = ((u64)tmp.start << PAGE_SHIFT) - adev->gmc.gart_start;
         r = amdgpu_ttm_gart_bind(adev, bo, flags);
         if (unlikely(r)) {
                 ttm_bo_mem_put(bo, &tmp);
--

2.17.1

Quoted 9 lines
Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#32 lore raw

PATCH 8/10 drm/amdgpu: distinct between allocated GART space and GMC addr

In reply to a message not in the archive

Most of the time we only need to know if the BO has a valid GMC addr.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c |  2 --
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.h |  2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c    | 13 +++++--------
 3 files changed, 6 insertions(+), 11 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0 -2
index 5ddd4e87480b..b5f20b42439e 100644
@@ -1362,8 +1362,6 @@ void amdgpu_bo_fence(struct amdgpu_bo *bo, struct dma_fence *fence,
 u64 amdgpu_bo_gpu_offset(struct amdgpu_bo *bo)
 {
 	WARN_ON_ONCE(bo->tbo.mem.mem_type == TTM_PL_SYSTEM);
-	WARN_ON_ONCE(bo->tbo.mem.mem_type == TTM_PL_TT &&
-		     !amdgpu_gtt_mgr_has_gart_addr(&bo->tbo.mem));
 	WARN_ON_ONCE(!ww_mutex_is_locked(&bo->tbo.resv->lock) &&
 		     !bo->pin_count);
 	WARN_ON_ONCE(bo->tbo.mem.start == AMDGPU_BO_INVALID_OFFSET);
drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +1 -1
index 18945dd6982d..37c79ae3574e 100644
@@ -200,7 +200,7 @@ static inline u64 amdgpu_bo_mmap_offset(struct amdgpu_bo *bo)
 static inline bool amdgpu_bo_gpu_accessible(struct amdgpu_bo *bo)
 {
 	switch (bo->tbo.mem.mem_type) {
-	case TTM_PL_TT: return amdgpu_gtt_mgr_has_gart_addr(&bo->tbo.mem);
+	case TTM_PL_TT: return bo->tbo.mem.start != AMDGPU_BO_INVALID_OFFSET;
 	case TTM_PL_VRAM: return true;
 	default: return false;
 	}
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +5 -8
index 5cadf4f1ee2c..d9f3201c9e5c 100644
@@ -345,7 +345,7 @@ static uint64_t amdgpu_mm_node_addr(struct ttm_buffer_object *bo,
 {
 	uint64_t addr = 0;
 
-	if (mem->mem_type != TTM_PL_TT || amdgpu_gtt_mgr_has_gart_addr(mem)) {
+	if (mm_node->start != AMDGPU_BO_INVALID_OFFSET) {
 		addr = mm_node->start << PAGE_SHIFT;
 		addr += bo->bdev->man[mem->mem_type].gpu_offset;
 	}
@@ -433,8 +433,7 @@ int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
 		/* Map only what needs to be accessed. Map src to window 0 and
 		 * dst to window 1
 		 */
-		if (src->mem->mem_type == TTM_PL_TT &&
-		    !amdgpu_gtt_mgr_has_gart_addr(src->mem)) {
+		if (src->mem->start == AMDGPU_BO_INVALID_OFFSET) {
 			r = amdgpu_map_buffer(src->bo, src->mem,
 					PFN_UP(cur_size + src_page_offset),
 					src_node_start, 0, ring,
@@ -447,8 +446,7 @@ int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
 			from += src_page_offset;
 		}
 
-		if (dst->mem->mem_type == TTM_PL_TT &&
-		    !amdgpu_gtt_mgr_has_gart_addr(dst->mem)) {
+		if (dst->mem->start == AMDGPU_BO_INVALID_OFFSET) {
 			r = amdgpu_map_buffer(dst->bo, dst->mem,
 					PFN_UP(cur_size + dst_page_offset),
 					dst_node_start, 1, ring,
@@ -1086,11 +1084,10 @@ int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo)
 	uint64_t flags;
 	int r;
 
-	if (bo->mem.mem_type != TTM_PL_TT ||
-	    amdgpu_gtt_mgr_has_gart_addr(&bo->mem))
+	if (bo->mem.start != AMDGPU_BO_INVALID_OFFSET)
 		return 0;
 
-	/* allocate GTT space */
+	/* allocate GART space */
 	tmp = bo->mem;
 	tmp.mm_node = NULL;
 	placement.num_placement = 1;
Signature · 6 lines
2.17.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20180827165337.1651-8-christian.koenig-5C7GfCeVMHo@public.gmane.org

#34 lore raw

Re: [PATCH 08/10] drm/amdgpu: distinct between allocated GART space and GMC addr

Quoted 27 lines · Christian König

Most of the time we only need to know if the BO has a valid GMC addr.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c |  2 --
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.h |  2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c    | 13 +++++--------
 3 files changed, 6 insertions(+), 11 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0 -2
index 5ddd4e87480b..b5f20b42439e 100644
@@ -1362,8 +1362,6 @@ void amdgpu_bo_fence(struct amdgpu_bo *bo, struct dma_fence *fence,
 u64 amdgpu_bo_gpu_offset(struct amdgpu_bo *bo)
 {
 	WARN_ON_ONCE(bo->tbo.mem.mem_type == TTM_PL_SYSTEM);
-	WARN_ON_ONCE(bo->tbo.mem.mem_type == TTM_PL_TT &&
-		     !amdgpu_gtt_mgr_has_gart_addr(&bo->tbo.mem));
 	WARN_ON_ONCE(!ww_mutex_is_locked(&bo->tbo.resv->lock) &&
 		     !bo->pin_count);
 	WARN_ON_ONCE(bo->tbo.mem.start == AMDGPU_BO_INVALID_OFFSET);
drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +0 -0
index 18945dd6982d..37c79ae3574e 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_object.h static inline u64 amdgpu_bo_mmap_offset(struct amdgpu_bo *bo) · line 200

Quoted 1 line
drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +0 -0
@@ -200,7 +200,7 @@ static inline u64 amdgpu_bo_mmap_offset(struct amdgpu_bo *bo)
 static inline bool amdgpu_bo_gpu_accessible(struct amdgpu_bo *bo)

As far as I can tell, this function is completely unused. So your change here has no effect.

Other than that Acked-by: Felix Kuehling <[email protected]>

Regards,
  Felix

Quoted 44 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +1 -1
 {
 	switch (bo->tbo.mem.mem_type) {
-	case TTM_PL_TT: return amdgpu_gtt_mgr_has_gart_addr(&bo->tbo.mem);
+	case TTM_PL_TT: return bo->tbo.mem.start != AMDGPU_BO_INVALID_OFFSET;
 	case TTM_PL_VRAM: return true;
 	default: return false;
 	}
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +3 -5
index 5cadf4f1ee2c..d9f3201c9e5c 100644
@@ -345,7 +345,7 @@ static uint64_t amdgpu_mm_node_addr(struct ttm_buffer_object *bo,
 {
 	uint64_t addr = 0;
 
-	if (mem->mem_type != TTM_PL_TT || amdgpu_gtt_mgr_has_gart_addr(mem)) {
+	if (mm_node->start != AMDGPU_BO_INVALID_OFFSET) {
 		addr = mm_node->start << PAGE_SHIFT;
 		addr += bo->bdev->man[mem->mem_type].gpu_offset;
 	}
@@ -433,8 +433,7 @@ int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
 		/* Map only what needs to be accessed. Map src to window 0 and
 		 * dst to window 1
 		 */
-		if (src->mem->mem_type == TTM_PL_TT &&
-		    !amdgpu_gtt_mgr_has_gart_addr(src->mem)) {
+		if (src->mem->start == AMDGPU_BO_INVALID_OFFSET) {
 			r = amdgpu_map_buffer(src->bo, src->mem,
 					PFN_UP(cur_size + src_page_offset),
 					src_node_start, 0, ring,
@@ -447,8 +446,7 @@ int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
 			from += src_page_offset;
 		}
 
-		if (dst->mem->mem_type == TTM_PL_TT &&
-		    !amdgpu_gtt_mgr_has_gart_addr(dst->mem)) {
+		if (dst->mem->start == AMDGPU_BO_INVALID_OFFSET) {
 			r = amdgpu_map_buffer(dst->bo, dst->mem,
 					PFN_UP(cur_size + dst_page_offset),
 					dst_node_start, 1, ring,
@@ -1086,11 +1084,10 @@ int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo)
 	uint64_t flags;
 	int r;
 

drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo) · line 1093

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +2 -3
-	if (bo->mem.mem_type != TTM_PL_TT ||
-	    amdgpu_gtt_mgr_has_gart_addr(&bo->mem))
+	if (bo->mem.start != AMDGPU_BO_INVALID_OFFSET)
 		return 0;
 
-	/* allocate GTT space */
+	/* allocate GART space */
 	tmp = bo->mem;
 	tmp.mm_node = NULL;
 	placement.num_placement = 1;

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Reviewed-by Junwei Zhang
#35 lore raw

Re: [PATCH 08/10] drm/amdgpu: distinct between allocated GART space and GMC addr

In reply to a message not in the archive
Quoted 3 lines · Christian König

Most of the time we only need to know if the BO has a valid GMC addr.

good to see this cleanup :)

Quoted 70 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_object.c |  2 --
  drivers/gpu/drm/amd/amdgpu/amdgpu_object.h |  2 +-
  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c    | 13 +++++--------
  3 files changed, 6 insertions(+), 11 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0 -2
index 5ddd4e87480b..b5f20b42439e 100644
@@ -1362,8 +1362,6 @@ void amdgpu_bo_fence(struct amdgpu_bo *bo, struct dma_fence *fence,
  u64 amdgpu_bo_gpu_offset(struct amdgpu_bo *bo)
  {
  	WARN_ON_ONCE(bo->tbo.mem.mem_type == TTM_PL_SYSTEM);
-	WARN_ON_ONCE(bo->tbo.mem.mem_type == TTM_PL_TT &&
-		     !amdgpu_gtt_mgr_has_gart_addr(&bo->tbo.mem));
  	WARN_ON_ONCE(!ww_mutex_is_locked(&bo->tbo.resv->lock) &&
  		     !bo->pin_count);
  	WARN_ON_ONCE(bo->tbo.mem.start == AMDGPU_BO_INVALID_OFFSET);
drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +1 -1
index 18945dd6982d..37c79ae3574e 100644
@@ -200,7 +200,7 @@ static inline u64 amdgpu_bo_mmap_offset(struct amdgpu_bo *bo)
  static inline bool amdgpu_bo_gpu_accessible(struct amdgpu_bo *bo)
  {
  	switch (bo->tbo.mem.mem_type) {
-	case TTM_PL_TT: return amdgpu_gtt_mgr_has_gart_addr(&bo->tbo.mem);
+	case TTM_PL_TT: return bo->tbo.mem.start != AMDGPU_BO_INVALID_OFFSET;
  	case TTM_PL_VRAM: return true;
  	default: return false;
  	}
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +3 -5
index 5cadf4f1ee2c..d9f3201c9e5c 100644
@@ -345,7 +345,7 @@ static uint64_t amdgpu_mm_node_addr(struct ttm_buffer_object *bo,
  {
  	uint64_t addr = 0;
 
-	if (mem->mem_type != TTM_PL_TT || amdgpu_gtt_mgr_has_gart_addr(mem)) {
+	if (mm_node->start != AMDGPU_BO_INVALID_OFFSET) {
  		addr = mm_node->start << PAGE_SHIFT;
  		addr += bo->bdev->man[mem->mem_type].gpu_offset;
  	}
@@ -433,8 +433,7 @@ int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
  		/* Map only what needs to be accessed. Map src to window 0 and
  		 * dst to window 1
  		 */
-		if (src->mem->mem_type == TTM_PL_TT &&
-		    !amdgpu_gtt_mgr_has_gart_addr(src->mem)) {
+		if (src->mem->start == AMDGPU_BO_INVALID_OFFSET) {
  			r = amdgpu_map_buffer(src->bo, src->mem,
  					PFN_UP(cur_size + src_page_offset),
  					src_node_start, 0, ring,
@@ -447,8 +446,7 @@ int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
  			from += src_page_offset;
  		}
 
-		if (dst->mem->mem_type == TTM_PL_TT &&
-		    !amdgpu_gtt_mgr_has_gart_addr(dst->mem)) {
+		if (dst->mem->start == AMDGPU_BO_INVALID_OFFSET) {
  			r = amdgpu_map_buffer(dst->bo, dst->mem,
  					PFN_UP(cur_size + dst_page_offset),
  					dst_node_start, 1, ring,
@@ -1086,11 +1084,10 @@ int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo)
  	uint64_t flags;
  	int r;
 

drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo) · line 1093

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +2 -3
-	if (bo->mem.mem_type != TTM_PL_TT ||
-	    amdgpu_gtt_mgr_has_gart_addr(&bo->mem))
+	if (bo->mem.start != AMDGPU_BO_INVALID_OFFSET)
  		return 0;
 
-	/* allocate GTT space */
+	/* allocate GART space */
  	tmp = bo->mem;
  	tmp.mm_node = NULL;
  	placement.num_placement = 1;

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#36 lore raw

PATCH 9/10 drm/amdgpu: use the AGP aperture for system memory access

In reply to a message not in the archive

Start to use the old AGP aperture for system memory access.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c     | 24 +++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h     |  1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c |  9 ++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c     | 21 ++++++++++--------
 4 files changed, 46 insertions(+), 9 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +24 -0
index eed5352f3136..54d353951e21 100644
@@ -79,6 +79,30 @@ uint64_t amdgpu_gmc_pd_addr(struct amdgpu_bo *bo)
 	return pd_addr;
 }
 
+/**
+ * amdgpu_gmc_agp_addr - return the address in the AGP address space
+ *
+ * @tbo: TTM BO which needs the address, must be in GTT domain
+ *
+ * Tries to figure out how to access the BO through the AGP aperture. Returns
+ * AMDGPU_BO_INVALID_OFFSET if that is not possible.
+ */
+uint64_t amdgpu_gmc_agp_addr(struct ttm_buffer_object *tbo)
+{
+	struct amdgpu_device *adev = amdgpu_ttm_adev(tbo->bdev);
+	struct ttm_dma_tt *ttm;
+
+	if (tbo->num_pages != 1 || !tbo->ttm ||
+	    tbo->ttm->caching_state == tt_cached)
+		return AMDGPU_BO_INVALID_OFFSET;
+
+	ttm = container_of(tbo->ttm, struct ttm_dma_tt, ttm);
+	if (ttm->dma_address[0] + PAGE_SIZE >= adev->gmc.agp_size)
+		return AMDGPU_BO_INVALID_OFFSET;
+
+	return adev->gmc.agp_start + ttm->dma_address[0];
+}
+
 /**
  * amdgpu_gmc_vram_location - try to find VRAM location
  *
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h +1 -0
index 163110fe375d..6e8432fd3309 100644
@@ -137,6 +137,7 @@ static inline bool amdgpu_gmc_vram_full_visible(struct amdgpu_gmc *gmc)
 void amdgpu_gmc_get_pde_for_bo(struct amdgpu_bo *bo, int level,
 			       uint64_t *addr, uint64_t *flags);
 uint64_t amdgpu_gmc_pd_addr(struct amdgpu_bo *bo);
+uint64_t amdgpu_gmc_agp_addr(struct ttm_buffer_object *tbo);
 void amdgpu_gmc_vram_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc,
 			      u64 base);
 void amdgpu_gmc_gart_location(struct amdgpu_device *adev,
drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +9 -0
index c2539f6821c0..deaea11eb39a 100644
@@ -132,6 +132,15 @@ static int amdgpu_gtt_mgr_alloc(struct ttm_mem_type_manager *man,
 	else
 		lpfn = adev->gart.num_cpu_pages;
 
+	if (fpfn == 0 && lpfn == adev->gart.num_cpu_pages) {
+		uint64_t addr = amdgpu_gmc_agp_addr(tbo);
+
+		if (addr != AMDGPU_BO_INVALID_OFFSET) {
+			mem->start = addr >> PAGE_SHIFT;
+			return 0;
+		}
+	}
+
 	mode = DRM_MM_INSERT_BEST;
 	if (place && place->flags & TTM_PL_FLAG_TOPDOWN)
 		mode = DRM_MM_INSERT_HIGH;
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +12 -9
index d9f3201c9e5c..281611f6bcd4 100644
@@ -1103,15 +1103,18 @@ int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo)
 	if (unlikely(r))
 		return r;
 
-	/* compute PTE flags for this buffer object */
-	flags = amdgpu_ttm_tt_pte_flags(adev, bo->ttm, &tmp);
-
-	/* Bind pages */
-	gtt->offset = ((u64)tmp.start << PAGE_SHIFT) - adev->gmc.gart_start;
-	r = amdgpu_ttm_gart_bind(adev, bo, flags);
-	if (unlikely(r)) {
-		ttm_bo_mem_put(bo, &tmp);
-		return r;
+	if (amdgpu_gtt_mgr_has_gart_addr(&tmp)) {
+		/* compute PTE flags for this buffer object */
+		flags = amdgpu_ttm_tt_pte_flags(adev, bo->ttm, &tmp);
+
+		/* Bind pages */
+		gtt->offset = ((u64)tmp.start << PAGE_SHIFT) -
+			adev->gmc.gart_start;
+		r = amdgpu_ttm_gart_bind(adev, bo, flags);
+		if (unlikely(r)) {
+			ttm_bo_mem_put(bo, &tmp);
+			return r;
+		}
 	}
 
 	ttm_bo_mem_put(bo, &bo->mem);
Signature · 6 lines
2.17.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20180827165337.1651-9-christian.koenig-5C7GfCeVMHo@public.gmane.org

Reviewed-by Junwei Zhang
#38 lore raw

Re: [PATCH 09/10] drm/amdgpu: use the AGP aperture for system memory access

Quoted 3 lines · Christian König

Start to use the old AGP aperture for system memory access.

Quoted 97 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c     | 24 +++++++++++++++++++++
  drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h     |  1 +
  drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c |  9 ++++++++
  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c     | 21 ++++++++++--------
  4 files changed, 46 insertions(+), 9 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +24 -0
index eed5352f3136..54d353951e21 100644
@@ -79,6 +79,30 @@ uint64_t amdgpu_gmc_pd_addr(struct amdgpu_bo *bo)
  	return pd_addr;
  }
 
+/**
+ * amdgpu_gmc_agp_addr - return the address in the AGP address space
+ *
+ * @tbo: TTM BO which needs the address, must be in GTT domain
+ *
+ * Tries to figure out how to access the BO through the AGP aperture. Returns
+ * AMDGPU_BO_INVALID_OFFSET if that is not possible.
+ */
+uint64_t amdgpu_gmc_agp_addr(struct ttm_buffer_object *tbo)
+{
+	struct amdgpu_device *adev = amdgpu_ttm_adev(tbo->bdev);
+	struct ttm_dma_tt *ttm;
+
+	if (tbo->num_pages != 1 || !tbo->ttm ||
+	    tbo->ttm->caching_state == tt_cached)
+		return AMDGPU_BO_INVALID_OFFSET;
+
+	ttm = container_of(tbo->ttm, struct ttm_dma_tt, ttm);
+	if (ttm->dma_address[0] + PAGE_SIZE >= adev->gmc.agp_size)
+		return AMDGPU_BO_INVALID_OFFSET;
+
+	return adev->gmc.agp_start + ttm->dma_address[0];
+}
+
  /**
   * amdgpu_gmc_vram_location - try to find VRAM location
   *
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h +1 -0
index 163110fe375d..6e8432fd3309 100644
@@ -137,6 +137,7 @@ static inline bool amdgpu_gmc_vram_full_visible(struct amdgpu_gmc *gmc)
  void amdgpu_gmc_get_pde_for_bo(struct amdgpu_bo *bo, int level,
  			       uint64_t *addr, uint64_t *flags);
  uint64_t amdgpu_gmc_pd_addr(struct amdgpu_bo *bo);
+uint64_t amdgpu_gmc_agp_addr(struct ttm_buffer_object *tbo);
  void amdgpu_gmc_vram_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc,
  			      u64 base);
  void amdgpu_gmc_gart_location(struct amdgpu_device *adev,
drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +9 -0
index c2539f6821c0..deaea11eb39a 100644
@@ -132,6 +132,15 @@ static int amdgpu_gtt_mgr_alloc(struct ttm_mem_type_manager *man,
  	else
  		lpfn = adev->gart.num_cpu_pages;
 
+	if (fpfn == 0 && lpfn == adev->gart.num_cpu_pages) {
+		uint64_t addr = amdgpu_gmc_agp_addr(tbo);
+
+		if (addr != AMDGPU_BO_INVALID_OFFSET) {
+			mem->start = addr >> PAGE_SHIFT;
+			return 0;
+		}
+	}
+
  	mode = DRM_MM_INSERT_BEST;
  	if (place && place->flags & TTM_PL_FLAG_TOPDOWN)
  		mode = DRM_MM_INSERT_HIGH;
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +5 -9
index d9f3201c9e5c..281611f6bcd4 100644
@@ -1103,15 +1103,18 @@ int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo)
  	if (unlikely(r))
  		return r;
 
-	/* compute PTE flags for this buffer object */
-	flags = amdgpu_ttm_tt_pte_flags(adev, bo->ttm, &tmp);
-
-	/* Bind pages */
-	gtt->offset = ((u64)tmp.start << PAGE_SHIFT) - adev->gmc.gart_start;
-	r = amdgpu_ttm_gart_bind(adev, bo, flags);
-	if (unlikely(r)) {
-		ttm_bo_mem_put(bo, &tmp);
-		return r;
+	if (amdgpu_gtt_mgr_has_gart_addr(&tmp)) {
+		/* compute PTE flags for this buffer object */
+		flags = amdgpu_ttm_tt_pte_flags(adev, bo->ttm, &tmp);
+
+		/* Bind pages */

drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo) · line 1120

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +7 -0
+		gtt->offset = ((u64)tmp.start << PAGE_SHIFT) -
+			adev->gmc.gart_start;
+		r = amdgpu_ttm_gart_bind(adev, bo, flags);
+		if (unlikely(r)) {
+			ttm_bo_mem_put(bo, &tmp);
+			return r;
+		}
  	}
 
  	ttm_bo_mem_put(bo, &bo->mem);

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#39 lore raw

PATCH 10/10 drm/amdgpu: enable AGP aperture for GMC9

In reply to a message not in the archive

Enable the old AGP aperture to avoid GART mappings.

---
 drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c | 10 +++++-----
 drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c    |  1 +
 drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c  | 10 +++++-----
 3 files changed, 11 insertions(+), 10 deletions(-)
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +5 -5
index 3403ded39d13..ffd0ec9586d1 100644
@@ -65,16 +65,16 @@ static void gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)
 {
 	uint64_t value;
 
-	/* Disable AGP. */
+	/* Program the AGP BAR */
 	WREG32_SOC15(GC, 0, mmMC_VM_AGP_BASE, 0);
-	WREG32_SOC15(GC, 0, mmMC_VM_AGP_TOP, 0);
-	WREG32_SOC15(GC, 0, mmMC_VM_AGP_BOT, 0xFFFFFFFF);
+	WREG32_SOC15(GC, 0, mmMC_VM_AGP_BOT, adev->gmc.agp_start >> 24);
+	WREG32_SOC15(GC, 0, mmMC_VM_AGP_TOP, adev->gmc.agp_end >> 24);
 
 	/* Program the system aperture low logical page number. */
 	WREG32_SOC15(GC, 0, mmMC_VM_SYSTEM_APERTURE_LOW_ADDR,
-		     adev->gmc.vram_start >> 18);
+		     min(adev->gmc.vram_start, adev->gmc.agp_start) >> 18);
 	WREG32_SOC15(GC, 0, mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR,
-		     adev->gmc.vram_end >> 18);
+		     max(adev->gmc.vram_end, adev->gmc.agp_end) >> 18);
 
 	/* Set default page address. */
 	value = adev->vram_scratch.gpu_addr - adev->gmc.vram_start
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +1 -0
index 60c8f848c102..3f40c153039f 100644
@@ -751,6 +751,7 @@ static void gmc_v9_0_vram_gtt_location(struct amdgpu_device *adev,
 		base = mmhub_v1_0_get_fb_location(adev);
 	amdgpu_gmc_vram_location(adev, &adev->gmc, base);
 	amdgpu_gmc_gart_location(adev, mc);
+	amdgpu_gmc_agp_location(adev, mc);
 	/* base offset of vram pages */
 	adev->vm_manager.vram_base_offset = gfxhub_v1_0_get_mc_fb_offset(adev);
 }
drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c +5 -5
index 5f6a9c85488f..73d7c075dd33 100644
@@ -76,16 +76,16 @@ static void mmhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)
 	uint64_t value;
 	uint32_t tmp;
 
-	/* Disable AGP. */
+	/* Program the AGP BAR */
 	WREG32_SOC15(MMHUB, 0, mmMC_VM_AGP_BASE, 0);
-	WREG32_SOC15(MMHUB, 0, mmMC_VM_AGP_TOP, 0);
-	WREG32_SOC15(MMHUB, 0, mmMC_VM_AGP_BOT, 0x00FFFFFF);
+	WREG32_SOC15(MMHUB, 0, mmMC_VM_AGP_BOT, adev->gmc.agp_start >> 24);
+	WREG32_SOC15(MMHUB, 0, mmMC_VM_AGP_TOP, adev->gmc.agp_end >> 24);
 
 	/* Program the system aperture low logical page number. */
 	WREG32_SOC15(MMHUB, 0, mmMC_VM_SYSTEM_APERTURE_LOW_ADDR,
-		     adev->gmc.vram_start >> 18);
+		     min(adev->gmc.vram_start, adev->gmc.agp_start) >> 18);
 	WREG32_SOC15(MMHUB, 0, mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR,
-		     adev->gmc.vram_end >> 18);
+		     max(adev->gmc.vram_end, adev->gmc.agp_end) >> 18);
 
 	/* Set default page address. */
 	value = adev->vram_scratch.gpu_addr - adev->gmc.vram_start +
Signature · 6 lines
2.17.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20180827165337.1651-10-christian.koenig-5C7GfCeVMHo@public.gmane.org

#41 lore raw

Re: [PATCH 10/10] drm/amdgpu: enable AGP aperture for GMC9

Quoted 22 lines · Christian König

Enable the old AGP aperture to avoid GART mappings.

---
  drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c | 10 +++++-----
  drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c    |  1 +
  drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c  | 10 +++++-----
  3 files changed, 11 insertions(+), 10 deletions(-)
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +1 -3
index 3403ded39d13..ffd0ec9586d1 100644
@@ -65,16 +65,16 @@ static void gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)
  {
  	uint64_t value;
 
-	/* Disable AGP. */
+	/* Program the AGP BAR */
  	WREG32_SOC15(GC, 0, mmMC_VM_AGP_BASE, 0);
-	WREG32_SOC15(GC, 0, mmMC_VM_AGP_TOP, 0);
-	WREG32_SOC15(GC, 0, mmMC_VM_AGP_BOT, 0xFFFFFFFF);

drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c static void gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev) · line 77

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +4 -2
+	WREG32_SOC15(GC, 0, mmMC_VM_AGP_BOT, adev->gmc.agp_start >> 24);
+	WREG32_SOC15(GC, 0, mmMC_VM_AGP_TOP, adev->gmc.agp_end >> 24);
 
  	/* Program the system aperture low logical page number. */
  	WREG32_SOC15(GC, 0, mmMC_VM_SYSTEM_APERTURE_LOW_ADDR,
-		     adev->gmc.vram_start >> 18);
+		     min(adev->gmc.vram_start, adev->gmc.agp_start) >> 18);
  	WREG32_SOC15(GC, 0, mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR,
-		     adev->gmc.vram_end >> 18);
+		     max(adev->gmc.vram_end, adev->gmc.agp_end) >> 18);

It may not work for AGP after GART range.

|--- vram ---|--- GART ---|--- AGP ---|

in this case, we may have to replace AGP and GART range.

Regards,
Jerry

Quoted 32 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +0 -0
  	/* Set default page address. */
  	value = adev->vram_scratch.gpu_addr - adev->gmc.vram_start
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +1 -0
index 60c8f848c102..3f40c153039f 100644
@@ -751,6 +751,7 @@ static void gmc_v9_0_vram_gtt_location(struct amdgpu_device *adev,
  		base = mmhub_v1_0_get_fb_location(adev);
  	amdgpu_gmc_vram_location(adev, &adev->gmc, base);
  	amdgpu_gmc_gart_location(adev, mc);
+	amdgpu_gmc_agp_location(adev, mc);
  	/* base offset of vram pages */
  	adev->vm_manager.vram_base_offset = gfxhub_v1_0_get_mc_fb_offset(adev);
  }
drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c +3 -3
index 5f6a9c85488f..73d7c075dd33 100644
@@ -76,16 +76,16 @@ static void mmhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)
  	uint64_t value;
  	uint32_t tmp;
 
-	/* Disable AGP. */
+	/* Program the AGP BAR */
  	WREG32_SOC15(MMHUB, 0, mmMC_VM_AGP_BASE, 0);
-	WREG32_SOC15(MMHUB, 0, mmMC_VM_AGP_TOP, 0);
-	WREG32_SOC15(MMHUB, 0, mmMC_VM_AGP_BOT, 0x00FFFFFF);
+	WREG32_SOC15(MMHUB, 0, mmMC_VM_AGP_BOT, adev->gmc.agp_start >> 24);
+	WREG32_SOC15(MMHUB, 0, mmMC_VM_AGP_TOP, adev->gmc.agp_end >> 24);
 

drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c static void mmhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev) · line 91

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c +2 -2
  	/* Program the system aperture low logical page number. */
  	WREG32_SOC15(MMHUB, 0, mmMC_VM_SYSTEM_APERTURE_LOW_ADDR,
-		     adev->gmc.vram_start >> 18);
+		     min(adev->gmc.vram_start, adev->gmc.agp_start) >> 18);
  	WREG32_SOC15(MMHUB, 0, mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR,
-		     adev->gmc.vram_end >> 18);
+		     max(adev->gmc.vram_end, adev->gmc.agp_end) >> 18);
 
  	/* Set default page address. */
  	value = adev->vram_scratch.gpu_addr - adev->gmc.vram_start +

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): [email protected]

#43 lore raw

Re: [PATCH 10/10] drm/amdgpu: enable AGP aperture for GMC9

Am 28.08.2018 um 04:52 schrieb Zhang, Jerry (Junwei):

Quoted 37 lines · Christian König
Quoted 23 lines

Enable the old AGP aperture to avoid GART mappings.

---
  drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c | 10 +++++----- drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c    |  1 + drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c  | 10 +++++-----

  3 files changed, 11 insertions(+), 10 deletions(-)
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +1 -3
index 3403ded39d13..ffd0ec9586d1 100644
@@ -65,16 +65,16 @@ static void 
 gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)
   {
       uint64_t value;
 
-    /* Disable AGP. */
+    /* Program the AGP BAR */
       WREG32_SOC15(GC, 0, mmMC_VM_AGP_BASE, 0);
-    WREG32_SOC15(GC, 0, mmMC_VM_AGP_TOP, 0);
-    WREG32_SOC15(GC, 0, mmMC_VM_AGP_BOT, 0xFFFFFFFF);

drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c static void · line 78

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +4 -2
+    WREG32_SOC15(GC, 0, mmMC_VM_AGP_BOT, adev->gmc.agp_start >> 24);
+    WREG32_SOC15(GC, 0, mmMC_VM_AGP_TOP, adev->gmc.agp_end >> 24);
 
       /* Program the system aperture low logical page number. */
       WREG32_SOC15(GC, 0, mmMC_VM_SYSTEM_APERTURE_LOW_ADDR,
-             adev->gmc.vram_start >> 18);
+             min(adev->gmc.vram_start, adev->gmc.agp_start) >> 18);
       WREG32_SOC15(GC, 0, mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR,
-             adev->gmc.vram_end >> 18);
+             max(adev->gmc.vram_end, adev->gmc.agp_end) >> 18);

It may not work for AGP after GART range.

Quoted 3 lines

|--- vram ---|--- GART ---|--- AGP ---|

in this case, we may have to replace AGP and GART range.

Yeah, that is exactly the reason why I've came up with "drm/amdgpu: put GART away from VRAM v2".

With this patch that scenario should never happen.

Regards,
Christian.

Quoted 45 lines

Regards,
Jerry

Quoted 41 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +0 -0
       /* Set default page address. */

      value = adev->vram_scratch.gpu_addr - adev->gmc.vram_start

drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +1 -0
index 60c8f848c102..3f40c153039f 100644
@@ -751,6 +751,7 @@ static void gmc_v9_0_vram_gtt_location(struct 
 amdgpu_device *adev,
           base = mmhub_v1_0_get_fb_location(adev);
       amdgpu_gmc_vram_location(adev, &adev->gmc, base);
       amdgpu_gmc_gart_location(adev, mc);
+    amdgpu_gmc_agp_location(adev, mc);
       /* base offset of vram pages */
       adev->vm_manager.vram_base_offset = gfxhub_v1_0_get_mc_fb_offset(adev);

  }

drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c +5 -5
index 5f6a9c85488f..73d7c075dd33 100644
@@ -76,16 +76,16 @@ static void 
 mmhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)
       uint64_t value;
       uint32_t tmp;
 
-    /* Disable AGP. */
+    /* Program the AGP BAR */
       WREG32_SOC15(MMHUB, 0, mmMC_VM_AGP_BASE, 0);
-    WREG32_SOC15(MMHUB, 0, mmMC_VM_AGP_TOP, 0);
-    WREG32_SOC15(MMHUB, 0, mmMC_VM_AGP_BOT, 0x00FFFFFF);
+    WREG32_SOC15(MMHUB, 0, mmMC_VM_AGP_BOT, adev->gmc.agp_start >> 24);
+    WREG32_SOC15(MMHUB, 0, mmMC_VM_AGP_TOP, adev->gmc.agp_end >> 24);
 
       /* Program the system aperture low logical page number. */
       WREG32_SOC15(MMHUB, 0, mmMC_VM_SYSTEM_APERTURE_LOW_ADDR,
-             adev->gmc.vram_start >> 18);
+             min(adev->gmc.vram_start, adev->gmc.agp_start) >> 18);
       WREG32_SOC15(MMHUB, 0, mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR,
-             adev->gmc.vram_end >> 18);
+             max(adev->gmc.vram_end, adev->gmc.agp_end) >> 18);
 
Quoted 3 lines
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c +0 -0
       /* Set default page address. */

      value = adev->vram_scratch.gpu_addr - adev->gmc.vram_start +

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Reviewed-by
#44 lore raw
In reply to a message not in the archive
Quoted 14 lines · Christian König

Only use the lower address space on GMC9 for the system domain. Otherwise we would need to sign extend GMC addresses.

---
 drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +0 -0
index e44b5191735d..d982956c8329 100644

drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c static int gmc_v9_0_sw_init(void *handle) · line 943

Quoted 9 lines
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +2 -3
@@ -938,11 +938,10 @@ static int gmc_v9_0_sw_init(void *handle)
        if (r)
                return r;
 
-       /* Set the internal MC address mask
-        * This is the max address of the GPU's
-        * internal address space.
+       /* Use only the lower range for the internal MC address mask. This is
+        * the max address of the GPU's internal address space.
         */

Update this comment to note the sign extension requirements and note that this max address is only for the system context (vmid 0). With that fixed:

Alex

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +1 -2
-       adev->gmc.mc_mask = 0xffffffffffffULL; /* 48 bit MC */
+       adev->gmc.mc_mask = 0x7fffffffffffULL;
 
        /* set DMA mask + need_dma32 flags.
         * PCIE - can handle 44-bits.
--

2.17.1

Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#45 lore raw
In reply to a message not in the archive

The point of this series seems to be to allow access to small system memory BOs (one page) without a GART mapping. I'm guessing that reduces pressure on the GART and removes the need for HDP and TLB flushes. Why does Patch 10 only enable that on GFXv9? Is there less benefit on older chips?

Is this related to your recent changes to allow page tables in system memory?

See my replies to patch 6 and 8. Other than that, the series is

Regards,
  Felix

Quoted 17 lines · Christian König

Only use the lower address space on GMC9 for the system domain. Otherwise we would need to sign extend GMC addresses.

---
 drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +0 -1
index e44b5191735d..d982956c8329 100644
@@ -938,11 +938,10 @@ static int gmc_v9_0_sw_init(void *handle)
 	if (r)
 		return r;
 
-	/* Set the internal MC address mask

drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c static int gmc_v9_0_sw_init(void *handle) · line 947

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +3 -3
-	 * This is the max address of the GPU's
-	 * internal address space.
+	/* Use only the lower range for the internal MC address mask. This is
+	 * the max address of the GPU's internal address space.
 	 */
-	adev->gmc.mc_mask = 0xffffffffffffULL; /* 48 bit MC */
+	adev->gmc.mc_mask = 0x7fffffffffffULL;
 
 	/* set DMA mask + need_dma32 flags.
 	 * PCIE - can handle 44-bits.

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): e5ef79ee-caef-9df4-aadc-bbd15b5eb9b3-5C7GfCeVMHo@public.gmane.org

#47 lore raw
Quoted 3 lines · Felix Kuehling

The point of this series seems to be to allow access to small system memory BOs (one page) without a GART mapping. I'm guessing that reduces pressure on the GART and removes the need for HDP and TLB flushes.

I think if adding these explain/reason to comments are better to enable AGP apperture. If that's true, it's really a clever idea.

Regards,
David Zhou

Quoted 40 lines

Why
does Patch 10 only enable that on GFXv9? Is there less benefit on older chips?

Is this related to your recent changes to allow page tables in system memory?

See my replies to patch 6 and 8. Other than that, the series is

Regards,
  Felix

Quoted 25 lines · Christian König

Only use the lower address space on GMC9 for the system domain. Otherwise we would need to sign extend GMC addresses.

---
  drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 7 +++----
  1 file changed, 3 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +3 -4
index e44b5191735d..d982956c8329 100644
@@ -938,11 +938,10 @@ static int gmc_v9_0_sw_init(void *handle)
  	if (r)
  		return r;
 
-	/* Set the internal MC address mask
-	 * This is the max address of the GPU's
-	 * internal address space.
+	/* Use only the lower range for the internal MC address mask. This is
+	 * the max address of the GPU's internal address space.
  	 */
-	adev->gmc.mc_mask = 0xffffffffffffULL; /* 48 bit MC */
+	adev->gmc.mc_mask = 0x7fffffffffffULL;
 
Quoted 6 lines

drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c static int gmc_v9_0_sw_init(void *handle) · line 947

Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +0 -0
  	/* set DMA mask + need_dma32 flags.
  	 * PCIE - can handle 44-bits.

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 75818784-27b6-21bc-274f-f29b0dd479e0-5C7GfCeVMHo@public.gmane.org

#49 lore raw

Am 28.08.2018 um 06:21 schrieb zhoucm1:

Quoted 7 lines · Felix Kuehling
Quoted 3 lines

The point of this series seems to be to allow access to small system memory BOs (one page) without a GART mapping. I'm guessing that reduces pressure on the GART and removes the need for HDP and TLB flushes.

I think if adding these explain/reason to comments are better to enable AGP apperture. If that's true, it's really a clever idea.

Well that is one use case. Another one is that we can enable dGPU without any VRAM.

That is really important for emulation and early ASIC testing.

Quoted 7 lines

Regards,
David Zhou

Quoted 3 lines

Why
does Patch 10 only enable that on GFXv9? Is there less benefit on older chips?

Can easily be enabled on older chips as well, but I haven't yet had time to test it.

Quoted 3 lines
Quoted 3 lines

Is this related to your recent changes to allow page tables in system memory?

Yes, as it turned out that otherwise results in a huge bunch of extra GART mappings.

Regards,
Christian.

Quoted 31 lines
Quoted 31 lines

See my replies to patch 6 and 8. Other than that, the series is

Regards,
   Felix

Quoted 22 lines · Christian König

Only use the lower address space on GMC9 for the system domain. Otherwise we would need to sign extend GMC addresses.

---
  drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 7 +++----

  1 file changed, 3 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +0 -0
index e44b5191735d..d982956c8329 100644
@@ -938,11 +938,10 @@ static int gmc_v9_0_sw_init(void *handle)
       if (r)
           return r;

  -    /* Set the internal MC address mask
-     * This is the max address of the GPU's
-     * internal address space.
+    /* Use only the lower range for the internal MC address mask. This is
+     * the max address of the GPU's internal address space. */

Quoted 9 lines
Quoted 8 lines
Quoted 4 lines

-    adev->gmc.mc_mask = 0xffffffffffffULL; /* 48 bit MC */ +    adev->gmc.mc_mask = 0x7fffffffffffULL;
        /* set DMA mask + need_dma32 flags.
       * PCIE - can handle 44-bits.

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): e21d83cd-fee8-fd99-ac02-39b5f29df92e-5C7GfCeVMHo@public.gmane.org

#51 lore raw
Quoted 19 lines · Christian König

Am 28.08.2018 um 06:21 schrieb zhoucm1:

Quoted 8 lines · Felix Kuehling
Quoted 3 lines

The point of this series seems to be to allow access to small system memory BOs (one page) without a GART mapping. I'm guessing that reduces pressure on the GART and removes the need for HDP and TLB flushes.

I think if adding these explain/reason to comments are better to enable AGP apperture. If that's true, it's really a clever idea.

Well that is one use case. Another one is that we can enable dGPU without any VRAM.

That is really important for emulation and early ASIC testing.

Quoted 4 lines

Regards,
David Zhou

Quoted 6 lines
Quoted 3 lines
Quoted 3 lines

Why
does Patch 10 only enable that on GFXv9? Is there less benefit on older chips?

Can easily be enabled on older chips as well, but I haven't yet had time to test it.

But on older ASICs, page tables are typically bigger than one page. So you'd still get GART mappings for them. BTW, how do you make sure those page tables are physically contiguous in the first place?

Quoted 6 lines
Quoted 3 lines
Quoted 3 lines

Is this related to your recent changes to allow page tables in system memory?

Yes, as it turned out that otherwise results in a huge bunch of extra GART mappings.

Oh, so the point is to avoid GART mappings for page tables, which are a single page on GFXv9. Now it makes sense. I was wondering what the killer use case was.

Thanks,
  Felix

Quoted 46 lines

Regards,
Christian.

Quoted 41 lines
Quoted 40 lines

See my replies to patch 6 and 8. Other than that, the series is

Regards,
   Felix

Quoted 19 lines · Christian König

Only use the lower address space on GMC9 for the system domain. Otherwise we would need to sign extend GMC addresses.

---
  drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 7 +++----

  1 file changed, 3 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +0 -0
index e44b5191735d..d982956c8329 100644
@@ -938,11 +938,10 @@ static int gmc_v9_0_sw_init(void *handle)
       if (r)
           return r;

  -    /* Set the internal MC address mask
-     * This is the max address of the GPU's
-     * internal address space.

Quoted 8 lines

+    /* Use only the lower range for the internal MC address mask. This is
+     * the max address of the GPU's internal address space. */
-    adev->gmc.mc_mask = 0xffffffffffffULL; /* 48 bit MC */ +    adev->gmc.mc_mask = 0x7fffffffffffULL;
        /* set DMA mask + need_dma32 flags.
       * PCIE - can handle 44-bits.

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Acked-by Felix Kuehling
#52 lore raw
In reply to a message not in the archive

This series patches seems to make AGP aperture allowed to access any system memory (16GB) bypass GPU VM protection. If someone made a wrong logic requesting an illegal address which occasionally was located inside AGP aperture, but without any VM protection, the illegal address would be finally translated into a system memory address; if GPU read/wrote such system memory address, the system memory address might belong to kernel or any user application, the r/w operation would result in any unpredictable issue.
The most important is that such kind of issue is so hard to be addressed.
Is it worth doing this, but exposing risk?

Regards,
Jack

-----Original Message-----
From: amd-gfx <[email protected]> On Behalf Of Felix Kuehling
Sent: Tuesday, August 28, 2018 3:03 AM
To: Christian König <[email protected]>; [email protected]; Koenig, Christian <[email protected]>
Subject: Re: [PATCH 01/10] drm/amdgpu: use only the lower address space on GMC9

The point of this series seems to be to allow access to small system memory BOs (one page) without a GART mapping. I'm guessing that reduces pressure on the GART and removes the need for HDP and TLB flushes. Why does Patch 10 only enable that on GFXv9? Is there less benefit on older chips?

Is this related to your recent changes to allow page tables in system memory?

See my replies to patch 6 and 8. Other than that, the series is

Regards,
  Felix

Quoted 18 lines · Christian König

Only use the lower address space on GMC9 for the system domain. Otherwise we would need to sign extend GMC addresses.

---
 drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +0 -1
index e44b5191735d..d982956c8329 100644
@@ -938,11 +938,10 @@ static int gmc_v9_0_sw_init(void *handle)
 	if (r)
 		return r;
 
-	/* Set the internal MC address mask

drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c static int gmc_v9_0_sw_init(void *handle) · line 947

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +3 -3
-	 * This is the max address of the GPU's
-	 * internal address space.
+	/* Use only the lower range for the internal MC address mask. This is
+	 * the max address of the GPU's internal address space.
 	 */
-	adev->gmc.mc_mask = 0xffffffffffffULL; /* 48 bit MC */
+	adev->gmc.mc_mask = 0x7fffffffffffULL;
 
 	/* set DMA mask + need_dma32 flags.
 	 * PCIE - can handle 44-bits.

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): BYAPR12MB2662148F853F560ED08FDBCFEF0A0-ZGDeBxoHBPm6habpV69vmAdYzm3356FpvxpqHgZTriW3zl9H0oFU5g@public.gmane.org

#54 lore raw
Quoted 1 line

This series patches seems to make AGP aperture allowed to access any system memory (16GB) bypass GPU VM protection.

The system aperture should only be active in the system domain, or otherwise applications would have access to local memory as well.

There are some bits in the VM registers to enable/disable that, but when we would have that setting incorrect we would see quite a bunch of other problems.

Might still be a good idea to double check if all the bits are setup correctly.

Regards,
Christian.

Am 28.08.2018 um 07:31 schrieb Xiao, Jack:

Quoted 55 lines

This series patches seems to make AGP aperture allowed to access any system memory (16GB) bypass GPU VM protection. If someone made a wrong logic requesting an illegal address which occasionally was located inside AGP aperture, but without any VM protection, the illegal address would be finally translated into a system memory address; if GPU read/wrote such system memory address, the system memory address might belong to kernel or any user application, the r/w operation would result in any unpredictable issue.
The most important is that such kind of issue is so hard to be addressed.
Is it worth doing this, but exposing risk?

Regards,
Jack

-----Original Message-----
From: amd-gfx <[email protected]> On Behalf Of Felix Kuehling
Sent: Tuesday, August 28, 2018 3:03 AM
To: Christian König <[email protected]>; [email protected]; Koenig, Christian <[email protected]>
Subject: Re: [PATCH 01/10] drm/amdgpu: use only the lower address space on GMC9

The point of this series seems to be to allow access to small system memory BOs (one page) without a GART mapping. I'm guessing that reduces pressure on the GART and removes the need for HDP and TLB flushes. Why does Patch 10 only enable that on GFXv9? Is there less benefit on older chips?

Is this related to your recent changes to allow page tables in system memory?

See my replies to patch 6 and 8. Other than that, the series is

Regards,
  Felix

Quoted 26 lines · Christian König

Only use the lower address space on GMC9 for the system domain. Otherwise we would need to sign extend GMC addresses.

---
  drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 7 +++----
  1 file changed, 3 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +3 -4
index e44b5191735d..d982956c8329 100644
@@ -938,11 +938,10 @@ static int gmc_v9_0_sw_init(void *handle)
  	if (r)
  		return r;
 
-	/* Set the internal MC address mask
-	 * This is the max address of the GPU's
-	 * internal address space.
+	/* Use only the lower range for the internal MC address mask. This is
+	 * the max address of the GPU's internal address space.
  	 */
-	adev->gmc.mc_mask = 0xffffffffffffULL; /* 48 bit MC */
+	adev->gmc.mc_mask = 0x7fffffffffffULL;
 
Quoted 6 lines

drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c static int gmc_v9_0_sw_init(void *handle) · line 947

Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +0 -0
  	/* set DMA mask + need_dma32 flags.
  	 * PCIE - can handle 44-bits.

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 5812c9d9-00bb-9065-fefe-49d661ab9439-5C7GfCeVMHo@public.gmane.org

#56 lore raw

I mean it has risk to make GPU allowed to access to most system memory without explicit claiming, it's easier to mask problem.

Regards,
Jack

-----Original Message-----
From: Koenig, Christian
Sent: Tuesday, August 28, 2018 2:46 PM
To: Xiao, Jack <[email protected]>; Kuehling, Felix <[email protected]>; Christian König <[email protected]>; [email protected]
Subject: Re: [PATCH 01/10] drm/amdgpu: use only the lower address space on GMC9

Quoted 1 line

This series patches seems to make AGP aperture allowed to access any system memory (16GB) bypass GPU VM protection.

The system aperture should only be active in the system domain, or otherwise applications would have access to local memory as well.

There are some bits in the VM registers to enable/disable that, but when we would have that setting incorrect we would see quite a bunch of other problems.

Might still be a good idea to double check if all the bits are setup correctly.

Regards,
Christian.

Am 28.08.2018 um 07:31 schrieb Xiao, Jack:

Quoted 59 lines

This series patches seems to make AGP aperture allowed to access any system memory (16GB) bypass GPU VM protection.
If someone made a wrong logic requesting an illegal address which
occasionally was located inside AGP aperture, but without any VM
protection, the illegal address would be finally translated into a
system memory address; if GPU read/wrote such system memory address, the system memory address might belong to kernel or any user application, the r/w operation would result in any unpredictable issue. The most important is that such kind of issue is so hard to be addressed.
Is it worth doing this, but exposing risk?

Regards,
Jack

-----Original Message-----
From: amd-gfx <[email protected]> On Behalf Of Felix Kuehling
Sent: Tuesday, August 28, 2018 3:03 AM
To: Christian König <[email protected]>; [email protected]; Koenig, Christian <[email protected]>
Subject: Re: [PATCH 01/10] drm/amdgpu: use only the lower address space on GMC9

The point of this series seems to be to allow access to small system memory BOs (one page) without a GART mapping. I'm guessing that reduces pressure on the GART and removes the need for HDP and TLB flushes. Why does Patch 10 only enable that on GFXv9? Is there less benefit on older chips?

Is this related to your recent changes to allow page tables in system memory?

See my replies to patch 6 and 8. Other than that, the series is

Regards,
  Felix

Quoted 26 lines · Christian König

Only use the lower address space on GMC9 for the system domain. Otherwise we would need to sign extend GMC addresses.

---
  drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 7 +++----
  1 file changed, 3 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +3 -4
index e44b5191735d..d982956c8329 100644
@@ -938,11 +938,10 @@ static int gmc_v9_0_sw_init(void *handle)
  	if (r)
  		return r;
 
-	/* Set the internal MC address mask
-	 * This is the max address of the GPU's
-	 * internal address space.
+	/* Use only the lower range for the internal MC address mask. This is
+	 * the max address of the GPU's internal address space.
  	 */
-	adev->gmc.mc_mask = 0xffffffffffffULL; /* 48 bit MC */
+	adev->gmc.mc_mask = 0x7fffffffffffULL;
 
Quoted 6 lines

drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c static int gmc_v9_0_sw_init(void *handle) · line 947

Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +0 -0
  	/* set DMA mask + need_dma32 flags.
  	 * PCIE - can handle 44-bits.

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): DM6PR12MB2666F87E35885284AB6B29A7EF0A0-lmeGfMZKVrEbyIpG/Z/[email protected]

#58 lore raw

Well that is indeed true, but we still have IOMMU between the GPU and the system memory.

So we should still catch issues when something goes terrible wrong.

Additional to that only the system domain, e.g. kernel copies, page table updates etc are allowed to use it.

Regards,
Christian.

Am 28.08.2018 um 09:06 schrieb Xiao, Jack:

Quoted 14 lines

I mean it has risk to make GPU allowed to access to most system memory without explicit claiming, it's easier to mask problem.

Regards,
Jack

-----Original Message-----
From: Koenig, Christian
Sent: Tuesday, August 28, 2018 2:46 PM
To: Xiao, Jack <[email protected]>; Kuehling, Felix <[email protected]>; Christian König <[email protected]>; [email protected]
Subject: Re: [PATCH 01/10] drm/amdgpu: use only the lower address space on GMC9

Quoted 1 line

This series patches seems to make AGP aperture allowed to access any system memory (16GB) bypass GPU VM protection.

The system aperture should only be active in the system domain, or otherwise applications would have access to local memory as well.

Quoted 73 lines

There are some bits in the VM registers to enable/disable that, but when we would have that setting incorrect we would see quite a bunch of other problems.

Might still be a good idea to double check if all the bits are setup correctly.

Regards,
Christian.

Am 28.08.2018 um 07:31 schrieb Xiao, Jack:

Quoted 65 lines

This series patches seems to make AGP aperture allowed to access any system memory (16GB) bypass GPU VM protection.
If someone made a wrong logic requesting an illegal address which
occasionally was located inside AGP aperture, but without any VM
protection, the illegal address would be finally translated into a
system memory address; if GPU read/wrote such system memory address, the system memory address might belong to kernel or any user application, the r/w operation would result in any unpredictable issue. The most important is that such kind of issue is so hard to be addressed.
Is it worth doing this, but exposing risk?

Regards,
Jack

-----Original Message-----
From: amd-gfx <[email protected]> On Behalf Of Felix Kuehling
Sent: Tuesday, August 28, 2018 3:03 AM
To: Christian König <[email protected]>; [email protected]; Koenig, Christian <[email protected]>
Subject: Re: [PATCH 01/10] drm/amdgpu: use only the lower address space on GMC9

The point of this series seems to be to allow access to small system memory BOs (one page) without a GART mapping. I'm guessing that reduces pressure on the GART and removes the need for HDP and TLB flushes. Why does Patch 10 only enable that on GFXv9? Is there less benefit on older chips?

Is this related to your recent changes to allow page tables in system memory?

See my replies to patch 6 and 8. Other than that, the series is

Regards,
  Felix

Quoted 18 lines · Christian König

Only use the lower address space on GMC9 for the system domain. Otherwise we would need to sign extend GMC addresses.

---
   drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 7 +++----
   1 file changed, 3 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +0 -1
index e44b5191735d..d982956c8329 100644
@@ -938,11 +938,10 @@ static int gmc_v9_0_sw_init(void *handle)
   	if (r)
   		return r;
 
-	/* Set the internal MC address mask

drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c static int gmc_v9_0_sw_init(void *handle) · line 947

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +3 -3
-	 * This is the max address of the GPU's
-	 * internal address space.
+	/* Use only the lower range for the internal MC address mask. This is
+	 * the max address of the GPU's internal address space.
   	 */
-	adev->gmc.mc_mask = 0xffffffffffffULL; /* 48 bit MC */
+	adev->gmc.mc_mask = 0x7fffffffffffULL;
 
   	/* set DMA mask + need_dma32 flags.
   	 * PCIE - can handle 44-bits.

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): b845f750-d6a7-c8ae-5235-7d961a3db428-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#60 lore raw

The GPU always has access to the entire guest physical address space. You can just program arbitrary addresses into page table entries and set the system bit. That's how GART and GPUVM mappings work. They will not go through the AGP aperture. And there is no mechanism (other than an IOMMU) to protect system memory from GPU access.

Regards,
  Felix

Quoted 5 lines · Christian König

Well that is indeed true, but we still have IOMMU between the GPU and the system memory.

So we should still catch issues when something goes terrible wrong.

Quoted 112 lines

Additional to that only the system domain, e.g. kernel copies, page table updates etc are allowed to use it.

Regards,
Christian.

Am 28.08.2018 um 09:06 schrieb Xiao, Jack:

Quoted 104 lines

I mean it has risk to make GPU allowed to access to most system memory without explicit claiming, it's easier to mask problem.

Regards,
Jack

-----Original Message-----
From: Koenig, Christian
Sent: Tuesday, August 28, 2018 2:46 PM
To: Xiao, Jack <[email protected]>; Kuehling, Felix <[email protected]>; Christian König <[email protected]>; [email protected]
Subject: Re: [PATCH 01/10] drm/amdgpu: use only the lower address space on GMC9

Quoted 2 lines

This series patches seems to make AGP aperture allowed to access any system memory (16GB) bypass GPU VM protection.

The system aperture should only be active in the system domain, or otherwise applications would have access to local memory as well.

There are some bits in the VM registers to enable/disable that, but when we would have that setting incorrect we would see quite a bunch of other problems.

Might still be a good idea to double check if all the bits are setup correctly.

Regards,
Christian.

Am 28.08.2018 um 07:31 schrieb Xiao, Jack:

Quoted 73 lines

This series patches seems to make AGP aperture allowed to access any system memory (16GB) bypass GPU VM protection.
If someone made a wrong logic requesting an illegal address which occasionally was located inside AGP aperture, but without any VM protection, the illegal address would be finally translated into a system memory address; if GPU read/wrote such system memory address, the system memory address might belong to kernel or any user application, the r/w operation would result in any unpredictable issue. The most important is that such kind of issue is so hard to be addressed.
Is it worth doing this, but exposing risk?

Regards,
Jack

-----Original Message-----
From: amd-gfx <[email protected]> On Behalf Of Felix Kuehling
Sent: Tuesday, August 28, 2018 3:03 AM
To: Christian König <[email protected]>; [email protected]; Koenig, Christian <[email protected]>
Subject: Re: [PATCH 01/10] drm/amdgpu: use only the lower address space on GMC9

The point of this series seems to be to allow access to small system memory BOs (one page) without a GART mapping. I'm guessing that reduces pressure on the GART and removes the need for HDP and TLB flushes. Why does Patch 10 only enable that on GFXv9? Is there less benefit on older chips?

Is this related to your recent changes to allow page tables in system memory?

See my replies to patch 6 and 8. Other than that, the series is

Regards,
    Felix

Quoted 19 lines · Christian König

Only use the lower address space on GMC9 for the system domain. Otherwise we would need to sign extend GMC addresses.

---
   drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 7 +++----

   1 file changed, 3 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +0 -0
index e44b5191735d..d982956c8329 100644
@@ -938,11 +938,10 @@ static int gmc_v9_0_sw_init(void *handle)
        if (r)
            return r;

   -    /* Set the internal MC address mask
-     * This is the max address of the GPU's
-     * internal address space.

Quoted 8 lines

+    /* Use only the lower range for the internal MC address mask. This is
+     * the max address of the GPU's internal address space. */
-    adev->gmc.mc_mask = 0xffffffffffffULL; /* 48 bit MC */ +    adev->gmc.mc_mask = 0x7fffffffffffULL;
          /* set DMA mask + need_dma32 flags.
        * PCIE - can handle 44-bits.

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 491b519e-6f02-1c4c-3cc8-c0ed147a82cd-5C7GfCeVMHo@public.gmane.org

#62 lore raw

Completely agree with Felix here.

It makes system memory access slightly simpler, but I would say that you accidentally corrupt the GART table and that you accidentally corrupt the the AGP aperture is equally likely.

Regards,
Christian.

Am 28.08.2018 um 20:12 schrieb Felix Kuehling:

Quoted 127 lines

The GPU always has access to the entire guest physical address space. You can just program arbitrary addresses into page table entries and set the system bit. That's how GART and GPUVM mappings work. They will not go through the AGP aperture. And there is no mechanism (other than an IOMMU) to protect system memory from GPU access.

Regards,
  Felix

Quoted 5 lines · Christian König

Well that is indeed true, but we still have IOMMU between the GPU and the system memory.

So we should still catch issues when something goes terrible wrong.

Quoted 111 lines

Additional to that only the system domain, e.g. kernel copies, page table updates etc are allowed to use it.

Regards,
Christian.

Am 28.08.2018 um 09:06 schrieb Xiao, Jack:

Quoted 104 lines

I mean it has risk to make GPU allowed to access to most system memory without explicit claiming, it's easier to mask problem.

Regards,
Jack

-----Original Message-----
From: Koenig, Christian
Sent: Tuesday, August 28, 2018 2:46 PM
To: Xiao, Jack <[email protected]>; Kuehling, Felix <[email protected]>; Christian König <[email protected]>; [email protected]
Subject: Re: [PATCH 01/10] drm/amdgpu: use only the lower address space on GMC9

Quoted 2 lines

This series patches seems to make AGP aperture allowed to access any system memory (16GB) bypass GPU VM protection.

The system aperture should only be active in the system domain, or otherwise applications would have access to local memory as well.

There are some bits in the VM registers to enable/disable that, but when we would have that setting incorrect we would see quite a bunch of other problems.

Might still be a good idea to double check if all the bits are setup correctly.

Regards,
Christian.

Am 28.08.2018 um 07:31 schrieb Xiao, Jack:

Quoted 73 lines

This series patches seems to make AGP aperture allowed to access any system memory (16GB) bypass GPU VM protection.
If someone made a wrong logic requesting an illegal address which occasionally was located inside AGP aperture, but without any VM protection, the illegal address would be finally translated into a system memory address; if GPU read/wrote such system memory address, the system memory address might belong to kernel or any user application, the r/w operation would result in any unpredictable issue. The most important is that such kind of issue is so hard to be addressed.
Is it worth doing this, but exposing risk?

Regards,
Jack

-----Original Message-----
From: amd-gfx <[email protected]> On Behalf Of Felix Kuehling
Sent: Tuesday, August 28, 2018 3:03 AM
To: Christian König <[email protected]>; [email protected]; Koenig, Christian <[email protected]>
Subject: Re: [PATCH 01/10] drm/amdgpu: use only the lower address space on GMC9

The point of this series seems to be to allow access to small system memory BOs (one page) without a GART mapping. I'm guessing that reduces pressure on the GART and removes the need for HDP and TLB flushes. Why does Patch 10 only enable that on GFXv9? Is there less benefit on older chips?

Is this related to your recent changes to allow page tables in system memory?

See my replies to patch 6 and 8. Other than that, the series is

Regards,
    Felix

Quoted 20 lines · Christian König

Only use the lower address space on GMC9 for the system domain. Otherwise we would need to sign extend GMC addresses.

---
    drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 7 +++----
    1 file changed, 3 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +1 -2
index e44b5191735d..d982956c8329 100644
@@ -938,11 +938,10 @@ static int gmc_v9_0_sw_init(void *handle)
        if (r)
            return r;
    -    /* Set the internal MC address mask
-     * This is the max address of the GPU's
-     * internal address space.
+    /* Use only the lower range for the internal MC address mask.
Quoted 7 lines

This is
+     * the max address of the GPU's internal address space.
        */
-    adev->gmc.mc_mask = 0xffffffffffffULL; /* 48 bit MC */ +    adev->gmc.mc_mask = 0x7fffffffffffULL;
          /* set DMA mask + need_dma32 flags.
        * PCIE - can handle 44-bits.

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): a2680c89-493a-2df8-f767-5d0f25e9f7ef-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#64 lore raw

Can you explain what the benefits can be gained from AGP aperture enablement? Otherwise, it would increase our maintenance workload.

Regards,
Jack

-----Original Message-----
From: Christian König <[email protected]>
Sent: Wednesday, August 29, 2018 4:47 PM
To: Kuehling, Felix <[email protected]>; Koenig, Christian <[email protected]>; Xiao, Jack <[email protected]>; [email protected]
Subject: Re: [PATCH 01/10] drm/amdgpu: use only the lower address space on GMC9

Completely agree with Felix here.

It makes system memory access slightly simpler, but I would say that you accidentally corrupt the GART table and that you accidentally corrupt the the AGP aperture is equally likely.

Regards,
Christian.

Am 28.08.2018 um 20:12 schrieb Felix Kuehling:

Quoted 128 lines

The GPU always has access to the entire guest physical address space. You can just program arbitrary addresses into page table entries and set the system bit. That's how GART and GPUVM mappings work. They will not go through the AGP aperture. And there is no mechanism (other than an
IOMMU) to protect system memory from GPU access.

Regards,
  Felix

Quoted 5 lines · Christian König

Well that is indeed true, but we still have IOMMU between the GPU and the system memory.

So we should still catch issues when something goes terrible wrong.

Quoted 111 lines

Additional to that only the system domain, e.g. kernel copies, page table updates etc are allowed to use it.

Regards,
Christian.

Am 28.08.2018 um 09:06 schrieb Xiao, Jack:

Quoted 104 lines

I mean it has risk to make GPU allowed to access to most system memory without explicit claiming, it's easier to mask problem.

Regards,
Jack

-----Original Message-----
From: Koenig, Christian
Sent: Tuesday, August 28, 2018 2:46 PM
To: Xiao, Jack <[email protected]>; Kuehling, Felix <[email protected]>; Christian König <[email protected]>; [email protected]
Subject: Re: [PATCH 01/10] drm/amdgpu: use only the lower address space on GMC9

Quoted 2 lines

This series patches seems to make AGP aperture allowed to access any system memory (16GB) bypass GPU VM protection.

The system aperture should only be active in the system domain, or otherwise applications would have access to local memory as well.

There are some bits in the VM registers to enable/disable that, but when we would have that setting incorrect we would see quite a bunch of other problems.

Might still be a good idea to double check if all the bits are setup correctly.

Regards,
Christian.

Am 28.08.2018 um 07:31 schrieb Xiao, Jack:

Quoted 73 lines

This series patches seems to make AGP aperture allowed to access any system memory (16GB) bypass GPU VM protection.
If someone made a wrong logic requesting an illegal address which occasionally was located inside AGP aperture, but without any VM protection, the illegal address would be finally translated into a system memory address; if GPU read/wrote such system memory address, the system memory address might belong to kernel or any user application, the r/w operation would result in any unpredictable issue. The most important is that such kind of issue is so hard to be addressed.
Is it worth doing this, but exposing risk?

Regards,
Jack

-----Original Message-----
From: amd-gfx <[email protected]> On Behalf Of Felix Kuehling
Sent: Tuesday, August 28, 2018 3:03 AM
To: Christian König <[email protected]>; [email protected]; Koenig, Christian <[email protected]>
Subject: Re: [PATCH 01/10] drm/amdgpu: use only the lower address space on GMC9

The point of this series seems to be to allow access to small system memory BOs (one page) without a GART mapping. I'm guessing that reduces pressure on the GART and removes the need for HDP and TLB flushes. Why does Patch 10 only enable that on GFXv9? Is there less benefit on older chips?

Is this related to your recent changes to allow page tables in system memory?

See my replies to patch 6 and 8. Other than that, the series is

Regards,
    Felix

Quoted 20 lines · Christian König

Only use the lower address space on GMC9 for the system domain. Otherwise we would need to sign extend GMC addresses.

---
    drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 7 +++----
    1 file changed, 3 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +1 -2
index e44b5191735d..d982956c8329 100644
@@ -938,11 +938,10 @@ static int gmc_v9_0_sw_init(void *handle)
        if (r)
            return r;
    -    /* Set the internal MC address mask
-     * This is the max address of the GPU's
-     * internal address space.
+    /* Use only the lower range for the internal MC address mask.
Quoted 7 lines

This is
+     * the max address of the GPU's internal address space.
        */
-    adev->gmc.mc_mask = 0xffffffffffffULL; /* 48 bit MC */ +    adev->gmc.mc_mask = 0x7fffffffffffULL;
          /* set DMA mask + need_dma32 flags.
        * PCIE - can handle 44-bits.

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): BYAPR12MB2662C7816E74F1C90E2E73BAEF080-ZGDeBxoHBPm6habpV69vmAdYzm3356FpvxpqHgZTriW3zl9H0oFU5g@public.gmane.org

#66 lore raw

As explained before we don't need a GART mapping any more for small allocations.

Additional to that we want it to support early bringup where no VRAM is attached to the ASIC.

Regards,
Christian.

Am 30.08.2018 um 05:48 schrieb Xiao, Jack:

Quoted 147 lines

Can you explain what the benefits can be gained from AGP aperture enablement? Otherwise, it would increase our maintenance workload.

Regards,
Jack

-----Original Message-----
From: Christian König <[email protected]>
Sent: Wednesday, August 29, 2018 4:47 PM
To: Kuehling, Felix <[email protected]>; Koenig, Christian <[email protected]>; Xiao, Jack <[email protected]>; [email protected]
Subject: Re: [PATCH 01/10] drm/amdgpu: use only the lower address space on GMC9

Completely agree with Felix here.

It makes system memory access slightly simpler, but I would say that you accidentally corrupt the GART table and that you accidentally corrupt the the AGP aperture is equally likely.

Regards,
Christian.

Am 28.08.2018 um 20:12 schrieb Felix Kuehling:

Quoted 128 lines

The GPU always has access to the entire guest physical address space. You can just program arbitrary addresses into page table entries and set the system bit. That's how GART and GPUVM mappings work. They will not go through the AGP aperture. And there is no mechanism (other than an
IOMMU) to protect system memory from GPU access.

Regards,
  Felix

Quoted 116 lines · Christian König

Well that is indeed true, but we still have IOMMU between the GPU and the system memory.

So we should still catch issues when something goes terrible wrong.

Additional to that only the system domain, e.g. kernel copies, page table updates etc are allowed to use it.

Regards,
Christian.

Am 28.08.2018 um 09:06 schrieb Xiao, Jack:

Quoted 104 lines

I mean it has risk to make GPU allowed to access to most system memory without explicit claiming, it's easier to mask problem.

Regards,
Jack

-----Original Message-----
From: Koenig, Christian
Sent: Tuesday, August 28, 2018 2:46 PM
To: Xiao, Jack <[email protected]>; Kuehling, Felix <[email protected]>; Christian König <[email protected]>; [email protected]
Subject: Re: [PATCH 01/10] drm/amdgpu: use only the lower address space on GMC9

Quoted 2 lines

This series patches seems to make AGP aperture allowed to access any system memory (16GB) bypass GPU VM protection.

The system aperture should only be active in the system domain, or otherwise applications would have access to local memory as well.

There are some bits in the VM registers to enable/disable that, but when we would have that setting incorrect we would see quite a bunch of other problems.

Might still be a good idea to double check if all the bits are setup correctly.

Regards,
Christian.

Am 28.08.2018 um 07:31 schrieb Xiao, Jack:

Quoted 73 lines

This series patches seems to make AGP aperture allowed to access any system memory (16GB) bypass GPU VM protection.
If someone made a wrong logic requesting an illegal address which occasionally was located inside AGP aperture, but without any VM protection, the illegal address would be finally translated into a system memory address; if GPU read/wrote such system memory address, the system memory address might belong to kernel or any user application, the r/w operation would result in any unpredictable issue. The most important is that such kind of issue is so hard to be addressed.
Is it worth doing this, but exposing risk?

Regards,
Jack

-----Original Message-----
From: amd-gfx <[email protected]> On Behalf Of Felix Kuehling
Sent: Tuesday, August 28, 2018 3:03 AM
To: Christian König <[email protected]>; [email protected]; Koenig, Christian <[email protected]>
Subject: Re: [PATCH 01/10] drm/amdgpu: use only the lower address space on GMC9

The point of this series seems to be to allow access to small system memory BOs (one page) without a GART mapping. I'm guessing that reduces pressure on the GART and removes the need for HDP and TLB flushes. Why does Patch 10 only enable that on GFXv9? Is there less benefit on older chips?

Is this related to your recent changes to allow page tables in system memory?

See my replies to patch 6 and 8. Other than that, the series is

Regards,
    Felix

Quoted 20 lines · Christian König

Only use the lower address space on GMC9 for the system domain. Otherwise we would need to sign extend GMC addresses.

---
     drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 7 +++----
     1 file changed, 3 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +1 -2
index e44b5191735d..d982956c8329 100644
@@ -938,11 +938,10 @@ static int gmc_v9_0_sw_init(void *handle)
         if (r)
             return r;
     -    /* Set the internal MC address mask
-     * This is the max address of the GPU's
-     * internal address space.
+    /* Use only the lower range for the internal MC address mask.
Quoted 7 lines

This is
+     * the max address of the GPU's internal address space.
        */
-    adev->gmc.mc_mask = 0xffffffffffffULL; /* 48 bit MC */ +    adev->gmc.mc_mask = 0x7fffffffffffULL;
          /* set DMA mask + need_dma32 flags.
        * PCIE - can handle 44-bits.

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Quoted 8 lines
Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#67 lore raw
In reply to a message not in the archive
Quoted 1 line

Well that is indeed true, but we still have IOMMU between the GPU and the system memory.

Some platform hasn't IOMMU, or IOMMU isn’t enabled by default.
If such kind of issue was reported by customer, it was so hard to narrow down.

Quoted 1 line

Additional to that only the system domain, e.g. kernel copies, page table updates etc are allowed to use it.

What was the motivation of this series patch?
I guess it is for performance improvement, right? But AGP aperture memory is translated as mtype=UC same with GART (mtype=UC). The more overhead on GART is gpuVM pagetable walker, but I guess the overhead is little.

Regards,
Jack

-----Original Message-----
From: Christian König <[email protected]>
Sent: Tuesday, August 28, 2018 7:42 PM
To: Xiao, Jack <[email protected]>; Koenig, Christian <[email protected]>; Kuehling, Felix <[email protected]>; [email protected]
Subject: Re: [PATCH 01/10] drm/amdgpu: use only the lower address space on GMC9

Well that is indeed true, but we still have IOMMU between the GPU and the system memory.

So we should still catch issues when something goes terrible wrong.

Additional to that only the system domain, e.g. kernel copies, page table updates etc are allowed to use it.

Regards,
Christian.

Am 28.08.2018 um 09:06 schrieb Xiao, Jack:

Quoted 17 lines

I mean it has risk to make GPU allowed to access to most system memory without explicit claiming, it's easier to mask problem.

Regards,
Jack

-----Original Message-----
From: Koenig, Christian
Sent: Tuesday, August 28, 2018 2:46 PM
To: Xiao, Jack <[email protected]>; Kuehling, Felix <[email protected]>; Christian König <[email protected]>; [email protected]
Subject: Re: [PATCH 01/10] drm/amdgpu: use only the lower address space on GMC9

Quoted 1 line

This series patches seems to make AGP aperture allowed to access any system memory (16GB) bypass GPU VM protection.

The system aperture should only be active in the system domain, or otherwise applications would have access to local memory as well.

Quoted 73 lines

There are some bits in the VM registers to enable/disable that, but when we would have that setting incorrect we would see quite a bunch of other problems.

Might still be a good idea to double check if all the bits are setup correctly.

Regards,
Christian.

Am 28.08.2018 um 07:31 schrieb Xiao, Jack:

Quoted 65 lines

This series patches seems to make AGP aperture allowed to access any system memory (16GB) bypass GPU VM protection.
If someone made a wrong logic requesting an illegal address which
occasionally was located inside AGP aperture, but without any VM
protection, the illegal address would be finally translated into a
system memory address; if GPU read/wrote such system memory address, the system memory address might belong to kernel or any user application, the r/w operation would result in any unpredictable issue. The most important is that such kind of issue is so hard to be addressed.
Is it worth doing this, but exposing risk?

Regards,
Jack

-----Original Message-----
From: amd-gfx <[email protected]> On Behalf Of Felix Kuehling
Sent: Tuesday, August 28, 2018 3:03 AM
To: Christian König <[email protected]>; [email protected]; Koenig, Christian <[email protected]>
Subject: Re: [PATCH 01/10] drm/amdgpu: use only the lower address space on GMC9

The point of this series seems to be to allow access to small system memory BOs (one page) without a GART mapping. I'm guessing that reduces pressure on the GART and removes the need for HDP and TLB flushes. Why does Patch 10 only enable that on GFXv9? Is there less benefit on older chips?

Is this related to your recent changes to allow page tables in system memory?

See my replies to patch 6 and 8. Other than that, the series is

Regards,
  Felix

Quoted 18 lines · Christian König

Only use the lower address space on GMC9 for the system domain. Otherwise we would need to sign extend GMC addresses.

---
   drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 7 +++----
   1 file changed, 3 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +0 -1
index e44b5191735d..d982956c8329 100644
@@ -938,11 +938,10 @@ static int gmc_v9_0_sw_init(void *handle)
   	if (r)
   		return r;
 
-	/* Set the internal MC address mask

drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c static int gmc_v9_0_sw_init(void *handle) · line 947

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +3 -3
-	 * This is the max address of the GPU's
-	 * internal address space.
+	/* Use only the lower range for the internal MC address mask. This is
+	 * the max address of the GPU's internal address space.
   	 */
-	adev->gmc.mc_mask = 0xffffffffffffULL; /* 48 bit MC */
+	adev->gmc.mc_mask = 0x7fffffffffffULL;
 
   	/* set DMA mask + need_dma32 flags.
   	 * PCIE - can handle 44-bits.

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#68 lore raw
In reply to a message not in the archive
Quoted 14 lines · Christian König

Only use the lower address space on GMC9 for the system domain. Otherwise we would need to sign extend GMC addresses.

---
  drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 7 +++----
  1 file changed, 3 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +0 -0
index e44b5191735d..d982956c8329 100644
@@ -938,11 +938,10 @@ static int gmc_v9_0_sw_init(void *handle)
  	if (r)

drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c static int gmc_v9_0_sw_init(void *handle) · line 944

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +3 -4
  		return r;
 
-	/* Set the internal MC address mask
-	 * This is the max address of the GPU's
-	 * internal address space.
+	/* Use only the lower range for the internal MC address mask. This is
+	 * the max address of the GPU's internal address space.
  	 */
-	adev->gmc.mc_mask = 0xffffffffffffULL; /* 48 bit MC */
+	adev->gmc.mc_mask = 0x7fffffffffffULL;

do we need to update vm_size as 128T at the same time?

Jerry

drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c static int gmc_v9_0_sw_init(void *handle)

Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +0 -0
  	/* set DMA mask + need_dma32 flags.
  	 * PCIE - can handle 44-bits.
 

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): [email protected]

#70 lore raw
Quoted 17 lines · Jerry (Junwei)
Quoted 16 lines · Christian König

Only use the lower address space on GMC9 for the system domain. Otherwise we would need to sign extend GMC addresses.

---
  drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 7 +++----
  1 file changed, 3 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +0 -0
index e44b5191735d..d982956c8329 100644
@@ -938,11 +938,10 @@ static int gmc_v9_0_sw_init(void *handle)
      if (r)
          return r;
 
Quoted 10 lines

drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c static int gmc_v9_0_sw_init(void *handle) · line 944

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +3 -4
-    /* Set the internal MC address mask
-     * This is the max address of the GPU's
-     * internal address space.
+    /* Use only the lower range for the internal MC address mask. This is
+     * the max address of the GPU's internal address space.
       */
-    adev->gmc.mc_mask = 0xffffffffffffULL; /* 48 bit MC */
+    adev->gmc.mc_mask = 0x7fffffffffffULL;

do we need to update vm_size as 128T at the same time?

Likely no, since we use that for system domain only.

BTW, how do we decide it's size limitation.
looks we always use that less than 40bit?

Jerry

Quoted 11 lines

Jerry

drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c static int gmc_v9_0_sw_init(void *handle)

Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +0 -0
      /* set DMA mask + need_dma32 flags.
       * PCIE - can handle 44-bits.
 

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx