Skip to contentSkip to conversation

drm/amdgpu: support get_cu_info for Arcturus

PATCH In review Alex Deucher 2 participants 4 messages Aug 1, 2019 3 messages not in the archive

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/2 drm/amdgpu: support get_cu_info for Arcturus

From: Le Ma <[email protected]>

This change is because SE/SH layout on Arcturus is 8*1, different from 4*2(or 4*1) on Vega ASICs.

Currently the cu bitmap array is 4x4 size, and besides the bitmap is used widely across SW stack. To mostly reduce the scale of impact, we make the cu bitmap array compatible with SE/SH layout on Arcturus. Then the store of cu bits of each shader array for Arcturus will be like below:

    SE0,SH0 --> bitmap[0][0]
    SE1,SH0 --> bitmap[1][0]
    SE2,SH0 --> bitmap[2][0]
    SE3,SH0 --> bitmap[3][0]
    SE4,SH0 --> bitmap[0][1]
    SE5,SH0 --> bitmap[1][1]
    SE6,SH0 --> bitmap[2][1]
    SE7,SH0 --> bitmap[3][1]
---
 drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c | 35 +++++++++++++++++++++------
 1 file changed, 28 insertions(+), 7 deletions(-)
drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c +28 -7
index 2dbd88b2ce87..a0c64d47760f 100644
@@ -6349,12 +6349,21 @@ static int gfx_v9_0_get_cu_info(struct amdgpu_device *adev,
 {
 	int i, j, k, counter, active_cu_number = 0;
 	u32 mask, bitmap, ao_bitmap, ao_cu_mask = 0;
-	unsigned disable_masks[4 * 2];
+	unsigned disable_masks[4 * 4];
 
 	if (!adev || !cu_info)
 		return -EINVAL;
 
-	amdgpu_gfx_parse_disable_cu(disable_masks, 4, 2);
+	/*
+	 * 16 comes from bitmap array size 4*4, and it can cover all gfx9 ASICs
+	 */
+	if (adev->gfx.config.max_shader_engines *
+		adev->gfx.config.max_sh_per_se > 16)
+		return -EINVAL;
+
+	amdgpu_gfx_parse_disable_cu(disable_masks,
+				    adev->gfx.config.max_shader_engines,
+				    adev->gfx.config.max_sh_per_se);
 
 	mutex_lock(&adev->grbm_idx_mutex);
 	for (i = 0; i < adev->gfx.config.max_shader_engines; i++) {
@@ -6363,11 +6372,23 @@ static int gfx_v9_0_get_cu_info(struct amdgpu_device *adev,
 			ao_bitmap = 0;
 			counter = 0;
 			gfx_v9_0_select_se_sh(adev, i, j, 0xffffffff);
-			if (i < 4 && j < 2)
-				gfx_v9_0_set_user_cu_inactive_bitmap(
-					adev, disable_masks[i * 2 + j]);
+			gfx_v9_0_set_user_cu_inactive_bitmap(
+				adev, disable_masks[i * adev->gfx.config.max_sh_per_se + j]);
 			bitmap = gfx_v9_0_get_cu_active_bitmap(adev);
-			cu_info->bitmap[i][j] = bitmap;
+
+			/*
+			 * The bitmap(and ao_cu_bitmap) in cu_info structure is
+			 * 4x4 size array, and it's usually suitable for Vega
+			 * ASICs which has 4*2 SE/SH layout.
+			 * But for Arcturus, SE/SH layout is changed to 8*1.
+			 * To mostly reduce the impact, we make it compatible
+			 * with current bitmap array as below:
+			 *    SE4,SH0 --> bitmap[0][1]
+			 *    SE5,SH0 --> bitmap[1][1]
+			 *    SE6,SH0 --> bitmap[2][1]
+			 *    SE7,SH0 --> bitmap[3][1]
+			 */
+			cu_info->bitmap[i % 4][j + i / 4] = bitmap;
 
 			for (k = 0; k < adev->gfx.config.max_cu_per_sh; k ++) {
 				if (bitmap & mask) {
@@ -6380,7 +6401,7 @@ static int gfx_v9_0_get_cu_info(struct amdgpu_device *adev,
 			active_cu_number += counter;
 			if (i < 2 && j < 2)
 				ao_cu_mask |= (ao_bitmap << (i * 16 + j * 8));
-			cu_info->ao_cu_bitmap[i][j] = ao_bitmap;
+			cu_info->ao_cu_bitmap[i % 4][j + i / 4] = ao_bitmap;
 		}
 	}
 	gfx_v9_0_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
Signature · 6 lines
2.20.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20190801184741.20064-1-alexander.deucher-5C7GfCeVMHo@public.gmane.org

#2 lore raw

PATCH 2/2 drm/amdkfd: Extend CU mask to 8 SEs (v2)

From: Jay Cornwall <[email protected]>

Following bitmap layout logic introduced by:
"drm/amdgpu: support get_cu_info for Arcturus".

v2: squash in fixup for gfx_v9_0.c (Alex)

---
 drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c          |  4 ++++
 drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c   | 10 +++++-----
 drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h   |  2 ++
 .../gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c    | 18 +++++++++++++++---
 drivers/gpu/drm/amd/include/v9_structs.h       |  8 ++++----
 5 files changed, 30 insertions(+), 12 deletions(-)
drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c +4 -0
index a0c64d47760f..4dee5d502a1e 100644
@@ -3337,6 +3337,10 @@ static int gfx_v9_0_mqd_init(struct amdgpu_ring *ring)
 	mqd->compute_static_thread_mgmt_se1 = 0xffffffff;
 	mqd->compute_static_thread_mgmt_se2 = 0xffffffff;
 	mqd->compute_static_thread_mgmt_se3 = 0xffffffff;
+	mqd->compute_static_thread_mgmt_se4 = 0xffffffff;
+	mqd->compute_static_thread_mgmt_se5 = 0xffffffff;
+	mqd->compute_static_thread_mgmt_se6 = 0xffffffff;
+	mqd->compute_static_thread_mgmt_se7 = 0xffffffff;
 	mqd->compute_misc_reserved = 0x00000003;
 
 	mqd->dynamic_cu_mask_addr_lo =
drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c +5 -5
index d6cf391da591..88813dad731f 100644
@@ -98,8 +98,8 @@ void mqd_symmetrically_map_cu_mask(struct mqd_manager *mm,
 		uint32_t *se_mask)
 {
 	struct kfd_cu_info cu_info;
-	uint32_t cu_per_sh[4] = {0};
-	int i, se, cu = 0;
+	uint32_t cu_per_se[KFD_MAX_NUM_SE] = {0};
+	int i, se, sh, cu = 0;
 
 	amdgpu_amdkfd_get_cu_info(mm->dev->kgd, &cu_info);
 
@@ -107,8 +107,8 @@ void mqd_symmetrically_map_cu_mask(struct mqd_manager *mm,
 		cu_mask_count = cu_info.cu_active_number;
 
 	for (se = 0; se < cu_info.num_shader_engines; se++)
-		for (i = 0; i < 4; i++)
-			cu_per_sh[se] += hweight32(cu_info.cu_bitmap[se][i]);
+		for (sh = 0; sh < cu_info.num_shader_arrays_per_engine; sh++)
+			cu_per_se[se] += hweight32(cu_info.cu_bitmap[se % 4][sh + (se / 4)]);
 
 	/* Symmetrically map cu_mask to all SEs:
 	 * cu_mask[0] bit0 -> se_mask[0] bit0;
@@ -128,6 +128,6 @@ void mqd_symmetrically_map_cu_mask(struct mqd_manager *mm,
 				se = 0;
 				cu++;
 			}
-		} while (cu >= cu_per_sh[se] && cu < 32);
+		} while (cu >= cu_per_se[se] && cu < 32);
 	}
 }
drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h +2 -0
index 550b61e81015..fbdb16418847 100644
@@ -26,6 +26,8 @@
 
 #include "kfd_priv.h"
 
+#define KFD_MAX_NUM_SE 8
+
 /**
  * struct mqd_manager
  *
drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c +15 -3
index 0c58f91b3ff3..1d7a8a8fcdc7 100644
@@ -46,7 +46,7 @@ static void update_cu_mask(struct mqd_manager *mm, void *mqd,
 			struct queue_properties *q)
 {
 	struct v9_mqd *m;
-	uint32_t se_mask[4] = {0}; /* 4 is the max # of SEs */
+	uint32_t se_mask[KFD_MAX_NUM_SE] = {0};
 
 	if (q->cu_mask_count == 0)
 		return;
@@ -59,12 +59,20 @@ static void update_cu_mask(struct mqd_manager *mm, void *mqd,
 	m->compute_static_thread_mgmt_se1 = se_mask[1];
 	m->compute_static_thread_mgmt_se2 = se_mask[2];
 	m->compute_static_thread_mgmt_se3 = se_mask[3];
+	m->compute_static_thread_mgmt_se4 = se_mask[4];
+	m->compute_static_thread_mgmt_se5 = se_mask[5];
+	m->compute_static_thread_mgmt_se6 = se_mask[6];
+	m->compute_static_thread_mgmt_se7 = se_mask[7];
 
-	pr_debug("update cu mask to %#x %#x %#x %#x\n",
+	pr_debug("update cu mask to #%x #%x #%x #%x #%x #%x #%x #%x\n",
 		m->compute_static_thread_mgmt_se0,
 		m->compute_static_thread_mgmt_se1,
 		m->compute_static_thread_mgmt_se2,
-		m->compute_static_thread_mgmt_se3);
+		m->compute_static_thread_mgmt_se3,
+		m->compute_static_thread_mgmt_se4,
+		m->compute_static_thread_mgmt_se5,
+		m->compute_static_thread_mgmt_se6,
+		m->compute_static_thread_mgmt_se7);
 }
 
 static void set_priority(struct v9_mqd *m, struct queue_properties *q)
@@ -125,6 +133,10 @@ static void init_mqd(struct mqd_manager *mm, void **mqd,
 	m->compute_static_thread_mgmt_se1 = 0xFFFFFFFF;
 	m->compute_static_thread_mgmt_se2 = 0xFFFFFFFF;
 	m->compute_static_thread_mgmt_se3 = 0xFFFFFFFF;
+	m->compute_static_thread_mgmt_se4 = 0xFFFFFFFF;
+	m->compute_static_thread_mgmt_se5 = 0xFFFFFFFF;
+	m->compute_static_thread_mgmt_se6 = 0xFFFFFFFF;
+	m->compute_static_thread_mgmt_se7 = 0xFFFFFFFF;
 
 	m->cp_hqd_persistent_state = CP_HQD_PERSISTENT_STATE__PRELOAD_REQ_MASK |
 			0x53 << CP_HQD_PERSISTENT_STATE__PRELOAD_SIZE__SHIFT;
drivers/gpu/drm/amd/include/v9_structs.h +4 -4
index 8b383dbe1cda..a0c672889fe4 100644
@@ -196,10 +196,10 @@ struct v9_mqd {
 	uint32_t compute_wave_restore_addr_lo;
 	uint32_t compute_wave_restore_addr_hi;
 	uint32_t compute_wave_restore_control;
-	uint32_t reserved_39;
-	uint32_t reserved_40;
-	uint32_t reserved_41;
-	uint32_t reserved_42;
+	uint32_t compute_static_thread_mgmt_se4;
+	uint32_t compute_static_thread_mgmt_se5;
+	uint32_t compute_static_thread_mgmt_se6;
+	uint32_t compute_static_thread_mgmt_se7;
 	uint32_t reserved_43;
 	uint32_t reserved_44;
 	uint32_t reserved_45;
Signature · 6 lines
2.20.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20190801184741.20064-2-alexander.deucher-5C7GfCeVMHo@public.gmane.org

#4 lore raw

Re: [PATCH 2/2] drm/amdkfd: Extend CU mask to 8 SEs (v2)

Quoted 6 lines · Alex Deucher

From: Jay Cornwall <[email protected]>

Following bitmap layout logic introduced by:
"drm/amdgpu: support get_cu_info for Arcturus".

v2: squash in fixup for gfx_v9_0.c (Alex)

There's a second patch to squash, which fixed breakage here (%# swapped):

drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c static void update_cu_mask(struct mqd_manager *mm, void *mqd, · line 67

Quoted 2 lines
drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c +1 -1
-	pr_debug("update cu mask to %#x %#x %#x %#x\n",
+	pr_debug("update cu mask to #%x #%x #%x #%x #%x #%x #%x #%x\n",

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 98bcc2b1-43ae-4a7d-8220-90861d881c2e-jFIJ+Wc5/Vo7lZ9V/[email protected]

#6 lore raw

Re: [PATCH 2/2] drm/amdkfd: Extend CU mask to 8 SEs (v2)

Whoops. Done.

Alex

Quoted 13 lines · Jay Cornwall
Quoted 6 lines · Alex Deucher

From: Jay Cornwall <[email protected]>

Following bitmap layout logic introduced by:
"drm/amdgpu: support get_cu_info for Arcturus".

v2: squash in fixup for gfx_v9_0.c (Alex)

There's a second patch to squash, which fixed breakage here (%# swapped):

Quoted 2 lines
drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c +1 -1
-     pr_debug("update cu mask to %#x %#x %#x %#x\n",
+     pr_debug("update cu mask to #%x #%x #%x #%x #%x #%x #%x #%x\n",

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx