Skip to contentSkip to conversation

drm/amdkfd: Fix a bug in SDMA RLC queue counting under HWS mode

PATCH In review Yong Zhao 2 participants 3 messages Jan 30, 2020

Subsystems (3)
lore
Share & tools
lore

Yong Zhao

author
#0 lore raw

PATCH 1/1 drm/amdkfd: Fix a bug in SDMA RLC queue counting under HWS mode

The sdma_queue_count increment should be done before execute_queues_cpsch(), which calls pm_calc_rlib_size() where sdma_queue_count is used to calculate whether over_subscription is triggered.

With the previous code, when a SDMA queue is created, compute_queue_count in pm_calc_rlib_size() is one more than the actual compute queue number, because the queue_count has been incremented while sdma_queue_count has not. This patch fixes that.

Change-Id I20353e657efd505353d0dd9f7eb2fab5085e7202
---
 drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c +6 -4
index 2870553a2ce0..80d22bf702e8 100644
@@ -1237,16 +1237,18 @@ static int create_queue_cpsch(struct device_queue_manager *dqm, struct queue *q,
 
 	list_add(&q->list, &qpd->queues_list);
 	qpd->queue_count++;
+
+	if (q->properties.type == KFD_QUEUE_TYPE_SDMA)
+		dqm->sdma_queue_count++;
+	else if (q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI)
+		dqm->xgmi_sdma_queue_count++;
+
 	if (q->properties.is_active) {
 		dqm->queue_count++;
 		retval = execute_queues_cpsch(dqm,
 				KFD_UNMAP_QUEUES_FILTER_DYNAMIC_QUEUES, 0);
 	}
 
-	if (q->properties.type == KFD_QUEUE_TYPE_SDMA)
-		dqm->sdma_queue_count++;
-	else if (q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI)
-		dqm->xgmi_sdma_queue_count++;
 	/*
 	 * Unconditionally increment this counter, regardless of the queue's
 	 * type or whether the queue is active.
Signature · 6 lines
2.17.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
maintainer Reviewed-by on v1 1/1
#1 lore raw
Quoted 5 lines · Yong Zhao

The sdma_queue_count increment should be done before execute_queues_cpsch(), which calls pm_calc_rlib_size() where sdma_queue_count is used to calculate whether over_subscription is triggered.

Quoted 7 lines

With the previous code, when a SDMA queue is created, compute_queue_count in pm_calc_rlib_size() is one more than the actual compute queue number, because the queue_count has been incremented while sdma_queue_count has not. This patch fixes that.

Change-Id I20353e657efd505353d0dd9f7eb2fab5085e7202

But I took a look at pm_calc_rlib_size. I don't think subtracting dqm->sdma_queue_count from dqm->queue_count is not quite correct, because sdma_queue_count counts all SDMA queues, while queue_count only counts active queues. So an application that creates inactive SDMA queues will also create errors here. We probably need to count active compute and active SDMA queues separately in DQM to fix this properly.

Regards,
  Felix

Quoted 21 lines
---
  drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c | 10 ++++++----
  1 file changed, 6 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c +6 -0
index 2870553a2ce0..80d22bf702e8 100644
@@ -1237,16 +1237,18 @@ static int create_queue_cpsch(struct device_queue_manager *dqm, struct queue *q,
 
  	list_add(&q->list, &qpd->queues_list);
  	qpd->queue_count++;
+
+	if (q->properties.type == KFD_QUEUE_TYPE_SDMA)
+		dqm->sdma_queue_count++;
+	else if (q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI)
+		dqm->xgmi_sdma_queue_count++;
+
  	if (q->properties.is_active) {
  		dqm->queue_count++;
  		retval = execute_queues_cpsch(dqm,

drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c static int create_queue_cpsch(struct device_queue_manager *dqm, struct queue *q, · line 1254

Quoted 10 lines
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c +0 -4
  				KFD_UNMAP_QUEUES_FILTER_DYNAMIC_QUEUES, 0);
  	}
 
-	if (q->properties.type == KFD_QUEUE_TYPE_SDMA)
-		dqm->sdma_queue_count++;
-	else if (q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI)
-		dqm->xgmi_sdma_queue_count++;
  	/*
  	 * Unconditionally increment this counter, regardless of the queue's
  	 * type or whether the queue is active.

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

author
#2 lore raw

True. It is a bug too. I am looking into it.

Yong

Quoted 44 lines · Felix Kuehling
Quoted 5 lines · Yong Zhao

The sdma_queue_count increment should be done before execute_queues_cpsch(), which calls pm_calc_rlib_size() where sdma_queue_count is used to calculate whether over_subscription is triggered.

Quoted 7 lines

With the previous code, when a SDMA queue is created, compute_queue_count in pm_calc_rlib_size() is one more than the actual compute queue number, because the queue_count has been incremented while sdma_queue_count has not. This patch fixes that.

Change-Id I20353e657efd505353d0dd9f7eb2fab5085e7202

But I took a look at pm_calc_rlib_size. I don't think subtracting dqm->sdma_queue_count from dqm->queue_count is not quite correct, because sdma_queue_count counts all SDMA queues, while queue_count only counts active queues. So an application that creates inactive SDMA queues will also create errors here. We probably need to count active compute and active SDMA queues separately in DQM to fix this properly.

Regards,
  Felix

Quoted 22 lines

---
  drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c | 10 ++++++----

  1 file changed, 6 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c +6 -0
index 2870553a2ce0..80d22bf702e8 100644
@@ -1237,16 +1237,18 @@ static int create_queue_cpsch(struct 
 device_queue_manager *dqm, struct queue *q,
         list_add(&q->list, &qpd->queues_list);
       qpd->queue_count++;
+
+    if (q->properties.type == KFD_QUEUE_TYPE_SDMA)
+        dqm->sdma_queue_count++;
+    else if (q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI)
+        dqm->xgmi_sdma_queue_count++;
+
       if (q->properties.is_active) {
           dqm->queue_count++;
           retval = execute_queues_cpsch(dqm,
                   KFD_UNMAP_QUEUES_FILTER_DYNAMIC_QUEUES, 0);
Quoted 8 lines

drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c static int create_queue_cpsch(struct device_queue_manager *dqm, struct queue *q, · line 1254

Quoted 8 lines
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c +0 -3
       }
   -    if (q->properties.type == KFD_QUEUE_TYPE_SDMA)
-        dqm->sdma_queue_count++;
-    else if (q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI)
-        dqm->xgmi_sdma_queue_count++;
       /*
        * Unconditionally increment this counter, regardless of the queue's
        * type or whether the queue is active.

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx