drm/amdgpu: Fix KFD oversubscription by tracking queues correctly
Subsystems (3)
PATCH 1/1 drm/amdgpu: Fix KFD oversubscription by tracking queues correctly
The number of compute queues available to the KFD was erroneously calculated as 64. Only the first MEC can execute compute queues and it has 32 queue slots.
This caused the oversubscription limit to be calculated incorrectly, leading to a missing chained runlist command at the end of an oversubscribed runlist.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +1 -1
@@ -140,7 +140,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev)/* According to linux/bitmap.h we shouldn't use bitmap_clear if * nbits is not compile time constant */ - last_valid_bit = adev->gfx.mec.num_mec + last_valid_bit =1 /* only first MEC can have compute queues */ * adev->gfx.mec.num_pipe_per_mec * adev->gfx.mec.num_queue_per_pipe;for (i = last_valid_bit; i <KGD_MAX_QUEUES ; ++i)
Signature · 6 lines
2.7.4 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 1499884012-7669-1-git-send-email-Jay.Cornwall-5C7GfCeVMHo@public.gmane.org
Quoted 19 lines · Jay Cornwall
The number of compute queues available to the KFD was erroneously calculated as 64. Only the first MEC can execute compute queues and it has 32 queue slots.
This caused the oversubscription limit to be calculated incorrectly, leading to a missing chained runlist command at the end of an oversubscribed runlist.
Change-Id Ic4a139c04b8a6d025fbb831a0a67e98728bfe461Signed-off-by Jay Cornwall <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +0
-0
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) · line 143
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +1
-1@@ -140,7 +140,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev)/* According to linux/bitmap.h we shouldn't use bitmap_clear if * nbits is not compile time constant */ - last_valid_bit = adev->gfx.mec.num_mec + last_valid_bit =1 /* only first MEC can have compute queues */
Hey Jay,
Minor nitpick. We already have some similar resource patching in kgd2kfd_device_init(), and I think it would be good to keep all of these together.
Otherwise, looks good to me.
Regards,
Andres
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) · line 146
Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +0
-0* adev->gfx.mec.num_pipe_per_mec * adev->gfx.mec.num_queue_per_pipe; for (i = last_valid_bit; i < KGD_MAX_QUEUES; ++i)
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 0d68e81c-28dc-a221-e37a-f9f10fd4ee49-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Quoted 29 lines · Andres Rodriguez
Quoted 19 lines · Jay Cornwall
The number of compute queues available to the KFD was erroneously calculated as 64. Only the first MEC can execute compute queues and it has 32 queue slots.
This caused the oversubscription limit to be calculated incorrectly, leading to a missing chained runlist command at the end of an oversubscribed runlist.
Change-Id Ic4a139c04b8a6d025fbb831a0a67e98728bfe461Signed-off-by Jay Cornwall <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +0
-0drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c void amdgpu_amdkfd_device_init(struct · line 144
Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +1
-1@@ -140,7 +140,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev)/* According to linux/bitmap.h we shouldn't use bitmap_clear if * nbits is not compile time constant */ - last_valid_bit = adev->gfx.mec.num_mec + last_valid_bit =1 /* only first MEC can have compute queues */ Hey Jay,
Quoted 3 lines
Minor nitpick. We already have some similar resource patching in
kgd2kfd_device_init(), and I think it would be good to keep all of these together.Otherwise, looks good to me.
Just re-read my reply and wanted to clarify. I don't really have a strong opining on which side does the resource availability patched. Whether it happens here or on the KFD side it is fine.
I just don't think it is good to keep it in different two places.
Regards,
Andres
Quoted 8 lines
Regards,
AndresQuoted 4 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +0
-0* adev->gfx.mec.num_pipe_per_mec * adev->gfx.mec.num_queue_per_pipe; for (i =
last_valid_bit; i <KGD_MAX_QUEUES; ++i)
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Quoted 28 lines · Andres Rodriguez
Quoted 19 lines · Jay Cornwall
The number of compute queues available to the KFD was erroneously calculated as 64. Only the first MEC can execute compute queues and it has 32 queue slots.
This caused the oversubscription limit to be calculated incorrectly, leading to a missing chained runlist command at the end of an oversubscribed runlist.
Change-Id Ic4a139c04b8a6d025fbb831a0a67e98728bfe461Signed-off-by Jay Cornwall <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +0
-0drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) · line 143
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +1
-1@@ -140,7 +140,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev)/* According to linux/bitmap.h we shouldn't use bitmap_clear if * nbits is not compile time constant */ - last_valid_bit = adev->gfx.mec.num_mec + last_valid_bit =1 /* only first MEC can have compute queues */ Hey Jay,
Quoted 3 lines
Minor nitpick. We already have some similar resource patching in
kgd2kfd_device_init(), and I think it would be good to keep all of these together.
OK. I see shared_resources.num_mec is set to 1 in kgd2kfd_device_init. That's not very clear (the number of MECs doesn't change) and num_mec doesn't appear to be used anywhere except in dead code in kfd_device.c. That code also runs after the queue bitmap setup.
How about I remove that field entirely?
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 1499973356.2889480.1040107984.4839CCCF-2RFepEojUI2N1INw9kWLP6GC3tUn3ZHUQQ4Iyu8u01E@public.gmane.org
Quoted 31 lines · Jay Cornwall
Quoted 27 lines · Andres Rodriguez
Quoted 19 lines · Jay Cornwall
The number of compute queues available to the KFD was erroneously calculated as 64. Only the first MEC can execute compute queues and it has 32 queue slots.
This caused the oversubscription limit to be calculated incorrectly, leading to a missing chained runlist command at the end of an oversubscribed runlist.
Change-Id Ic4a139c04b8a6d025fbb831a0a67e98728bfe461Signed-off-by Jay Cornwall <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +0
-0drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) · line 143
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +1
-1@@ -140,7 +140,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev)/* According to linux/bitmap.h we shouldn't use bitmap_clear if * nbits is not compile time constant */ - last_valid_bit = adev->gfx.mec.num_mec + last_valid_bit =1 /* only first MEC can have compute queues */ Hey Jay,
Quoted 3 lines
Minor nitpick. We already have some similar resource patching in
kgd2kfd_device_init(), and I think it would be good to keep all of these together.
Quoted 6 lines
OK. I see shared_resources.num_mec is set to 1 in
kgd2kfd_device_init. That's not very clear (the number of MECs doesn't change) andnum_mecdoesn't appear to be used anywhere except in dead code inkfd_device.c. That code also runs after the queue bitmap setup.How about I remove that field entirely?
Yeah, that's fine with me.
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 7ada7ccb-ed8b-0624-81fe-69cd529ea9c1-5C7GfCeVMHo@public.gmane.org
Quoted 32 lines · Felix Kuehling
Quoted 31 lines · Jay Cornwall
Quoted 30 lines · Andres Rodriguez
Quoted 19 lines · Jay Cornwall
The number of compute queues available to the KFD was erroneously calculated as 64. Only the first MEC can execute compute queues and it has 32 queue slots.
This caused the oversubscription limit to be calculated incorrectly, leading to a missing chained runlist command at the end of an oversubscribed runlist.
Change-Id Ic4a139c04b8a6d025fbb831a0a67e98728bfe461Signed-off-by Jay Cornwall <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +0
-0drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) · line 143
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +1
-1@@ -140,7 +140,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev)/* According to linux/bitmap.h we shouldn't use bitmap_clear if * nbits is not compile time constant */ - last_valid_bit = adev->gfx.mec.num_mec + last_valid_bit =1 /* only first MEC can have compute queues */ Hey Jay,
Minor nitpick. We already have some similar resource patching in
kgd2kfd_device_init(), and I think it would be good to keep all of these together.
Quoted 8 lines
Quoted 6 lines
OK. I see shared_resources.num_mec is set to 1 in
kgd2kfd_device_init. That's not very clear (the number of MECs doesn't change) andnum_mecdoesn't appear to be used anywhere except in dead code inkfd_device.c. That code also runs after the queue bitmap setup.How about I remove that field entirely?
Yeah, that's fine with me.
Good with me as well.
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 214a6fc1-d99e-bfe9-6f86-418cefd6283c-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
There is a function get_mec_num use the field , but seems no one call it , maybe remove it as well.
Regards
Shaoyun.liu
-----Original Message-----
From: amd-gfx [mailto:[email protected]] On Behalf Of Andres Rodriguez
Sent: Thursday, July 13, 2017 3:54 PM
To: Kuehling, Felix; Jay Cornwall; [email protected]
Subject: Re: [PATCH] drm/amdgpu: Fix KFD oversubscription by tracking queues correctly
Quoted 33 lines · Felix Kuehling
Quoted 32 lines · Jay Cornwall
Quoted 31 lines · Andres Rodriguez
Quoted 20 lines · Jay Cornwall
The number of compute queues available to the KFD was erroneously calculated as 64. Only the first MEC can execute compute queues and it has 32 queue slots.
This caused the oversubscription limit to be calculated incorrectly, leading to a missing chained runlist command at the end of an oversubscribed runlist.
Change-Id Ic4a139c04b8a6d025fbb831a0a67e98728bfe461Signed-off-by Jay Cornwall <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +0
-0drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) · line 143
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +1
-1@@ -140,7 +140,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev)/* According to linux/bitmap.h we shouldn't use bitmap_clear if * nbits is not compile time constant */ - last_valid_bit = adev->gfx.mec.num_mec + last_valid_bit =1 /* only first MEC can have compute queues */ Hey Jay,
Minor nitpick. We already have some similar resource patching in
kgd2kfd_device_init(), and I think it would be good to keep all of these together.
Quoted 8 lines
Quoted 6 lines
OK. I see shared_resources.num_mec is set to 1 in
kgd2kfd_device_init. That's not very clear (the number of MECs doesn't change) andnum_mecdoesn't appear to be used anywhere except in dead code inkfd_device.c. That code also runs after the queue bitmap setup.How about I remove that field entirely?
Yeah, that's fine with me.
Good with me as well.
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx