drm/amdgpu: Fix KFD oversubscription by tracking queues correctly
Subsystems (3)
PATCH v3 1/4 drm/amdgpu: Fix KFD oversubscription by tracking queues correctly
The number of compute queues available to the KFD was erroneously calculated as 64. Only the first MEC can execute compute queues and it has 32 queue slots.
This caused the oversubscription limit to be calculated incorrectly, leading to a missing chained runlist command at the end of an oversubscribed runlist.
v2: Remove unused num_mec field to avoid duplicate logic
v3: Separate num_mec removal into separate patches
--- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +1 -1
@@ -140,7 +140,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev)/* According to linux/bitmap.h we shouldn't use bitmap_clear if * nbits is not compile time constant */ - last_valid_bit = adev->gfx.mec.num_mec + last_valid_bit =1 /* only first MEC can have compute queues */ * adev->gfx.mec.num_pipe_per_mec * adev->gfx.mec.num_queue_per_pipe;for (i = last_valid_bit; i <KGD_MAX_QUEUES ; ++i)
Signature · 6 lines
2.7.4 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 2/4 drm/amdkfd: Remove unused references to shared_resources.num_mec
In reply to a message not in the archiveDead code.
--- drivers/gpu/drm/amd/amdkfd/kfd_device.c | 4 ---- drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c | 7 ------- 2 files changed, 11 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_device.c +0 -4
@@ -494,10 +494,6 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd, }else kfd->max_proc_per_quantum = hws_max_conc_proc; -/* We only use the first MEC */ -if (kfd->shared_resources.num_mec >1 ) - kfd->shared_resources.num_mec =1 ; -/* calculate max size of mqds needed for queues */ size = max_num_of_queues_per_device * kfd->device_info->mqd_size_aligned;
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c +0 -7
@@ -82,13 +82,6 @@ static bool is_pipe_enabled(struct device_queue_manager *dqm, int mec, int pipe)return false; } -unsigned int get_mec_num (struct device_queue_manager *dqm) -{ -BUG_ON (!dqm || !dqm->dev); - -return dqm->dev->shared_resources.num_mec; -} -unsigned int get_queues_num (struct device_queue_manager *dqm) {BUG_ON (!dqm || !dqm->dev);
Signature · 6 lines
2.7.4 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 3/4 drm/radeon: Remove initialization of shared_resources.num_mec
In reply to a message not in the archiveDead code.
--- drivers/gpu/drm/radeon/radeon_kfd.c | 1 - 1 file changed, 1 deletion(-)
drivers/gpu/drm/radeon/radeon_kfd.c +0 -1
@@ -251,7 +251,6 @@ void radeon_kfd_device_init(struct radeon_device *rdev)if (rdev->kfd) {struct kgd2kfd_shared_resources gpu_resources = { .compute_vmid_bitmap =0xFF00 , - .num_mec =1 , .num_pipe_per_mec =4 , .num_queue_per_pipe =8 , .gpuvm_size = (uint64_t )radeon_vm_size <<30
Signature · 6 lines
2.7.4 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 4/4 drm/amdgpu: Remove unused field kgd2kfd_shared_resources.num_mec
In reply to a message not in the archiveDead code.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 1 - drivers/gpu/drm/amd/include/kgd_kfd_interface.h | 3 --- 2 files changed, 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +0 -1
@@ -116,7 +116,6 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev)if (adev->kfd) {struct kgd2kfd_shared_resources gpu_resources = { .compute_vmid_bitmap = global_compute_vmid_bitmap, - .num_mec = adev->gfx.mec.num_mec, .num_pipe_per_mec = adev->gfx.mec.num_pipe_per_mec, .num_queue_per_pipe = adev->gfx.mec.num_queue_per_pipe, .gpuvm_size = (uint64_t )amdgpu_vm_size <<30
drivers/gpu/drm/amd/include/kgd_kfd_interface.h +0 -3
@@ -107,9 +107,6 @@ struct kgd2kfd_shared_resources {/* Bit n == 1 means VMID n is available for KFD. */ unsigned int compute_vmid_bitmap; -/* number of mec available from the hardware */ -uint32_t num_mec; -/* number of pipes per mec */ uint32_t num_pipe_per_mec;
Signature · 6 lines
2.7.4 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Alex Deucher
Reviewed-byQuoted 8 lines · Jay Cornwall
The number of compute queues available to the KFD was erroneously calculated as 64. Only the first MEC can execute compute queues and it has 32 queue slots.
This caused the oversubscription limit to be calculated incorrectly, leading to a missing chained runlist command at the end of an oversubscribed runlist.
Quoted 5 lines
v2: Remove unused
num_mecfield to avoid duplicate logic
v3: Separatenum_mecremoval into separate patchesChange-Id I9e7bba2cc1928b624e3eeb1edb06fdb602e5294fSigned-off-by Jay Cornwall <[email protected]>
Series is:
Quoted 20 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +1
-2@@ -140,7 +140,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev)/* According to linux/bitmap.h we shouldn't use bitmap_clear if * nbits is not compile time constant */ - last_valid_bit = adev->gfx.mec.num_mec + last_valid_bit =1 /* only first MEC can have compute queues */ * adev->gfx.mec.num_pipe_per_mec * adev->gfx.mec.num_queue_per_pipe;for (i = last_valid_bit; i <KGD_MAX_QUEUES ; ++i) --2.7.4
Quoted 4 lines
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Quoted 9 lines · Alex Deucher
Quoted 8 lines · Jay Cornwall
The number of compute queues available to the KFD was erroneously calculated as 64. Only the first MEC can execute compute queues and it has 32 queue slots.
This caused the oversubscription limit to be calculated incorrectly, leading to a missing chained runlist command at the end of an oversubscribed runlist.
Quoted 9 lines
Quoted 5 lines
v2: Remove unused
num_mecfield to avoid duplicate logic
v3: Separatenum_mecremoval into separate patchesChange-Id I9e7bba2cc1928b624e3eeb1edb06fdb602e5294fSigned-off-by Jay Cornwall <[email protected]>Series is:
Reviewed-by Alex Deucher <[email protected]>
Hi Jay,
Thanks for the patches, I applied them to amdkfd-fixes (after rebasing them over 4.13-rc1)
Oded
Quoted 20 lines
Quoted 20 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +1
-2@@ -140,7 +140,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev)/* According to linux/bitmap.h we shouldn't use bitmap_clear if * nbits is not compile time constant */ - last_valid_bit = adev->gfx.mec.num_mec + last_valid_bit =1 /* only first MEC can have compute queues */ * adev->gfx.mec.num_pipe_per_mec * adev->gfx.mec.num_queue_per_pipe;for (i = last_valid_bit; i <KGD_MAX_QUEUES ; ++i) --2.7.4
Quoted 8 lines
Quoted 4 lines
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Re: [PATCH 2/4] drm/amdkfd: Remove unused references to shared_resources.num_mec
In reply to a message not in the archiveQuoted 29 lines · Jay Cornwall
Dead code.
Change-Id Ic0bb1bcca87e96bc5e8fa9894727b0de152e8818Signed-off-by Jay Cornwall <[email protected]>--- drivers/gpu/drm/amd/amdkfd/kfd_device.c | 4 ---- drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c | 7 ------- 2 files changed, 11 deletions(-)drivers/gpu/drm/amd/amdkfd/kfd_device.c +0
-4@@ -494,10 +494,6 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd, }else kfd->max_proc_per_quantum = hws_max_conc_proc; -/* We only use the first MEC */ -if (kfd->shared_resources.num_mec >1 ) - kfd->shared_resources.num_mec =1 ; -/* calculate max size of mqds needed for queues */ size = max_num_of_queues_per_device * kfd->device_info->mqd_size_aligned;drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c +0
-0
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c static bool is_pipe_enabled(struct device_queue_manager *dqm, int mec, int pipe) · line 91
Quoted 10 lines
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c +0
-7@@ -82,13 +82,6 @@ static bool is_pipe_enabled(struct device_queue_manager *dqm, int mec, int pipe)return false; } -unsigned int get_mec_num (struct device_queue_manager *dqm) -{ -BUG_ON (!dqm || !dqm->dev); - -return dqm->dev->shared_resources.num_mec; -} -
FYI, I removed also the declaration of get_mec_num() in the header file Oded
Quoted 10 lines
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c +0
-1unsigned int get_queues_num (struct device_queue_manager *dqm) {BUG_ON (!dqm || !dqm->dev); --2.7.4
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Hi Alex,
This patch series went into amd-kfd-staging. I'd like to also push it into amd-staging-4.11 as I'm just working to minimize any unnecessary differences between the branches before the big KFD history rework.
I rebased it, resolved some contlicts, and removed the declaration of get_mec_num from kfd_device_queue_manager.h. Do you want me to push that rebased patch series?
Thanks,
Felix
Quoted 52 lines · Oded Gabbay
Quoted 9 lines · Alex Deucher
Quoted 8 lines · Jay Cornwall
The number of compute queues available to the KFD was erroneously calculated as 64. Only the first MEC can execute compute queues and it has 32 queue slots.
This caused the oversubscription limit to be calculated incorrectly, leading to a missing chained runlist command at the end of an oversubscribed runlist.
Quoted 8 lines
Quoted 5 lines
v2: Remove unused
num_mecfield to avoid duplicate logic
v3: Separatenum_mecremoval into separate patchesChange-Id I9e7bba2cc1928b624e3eeb1edb06fdb602e5294fSigned-off-by Jay Cornwall <[email protected]>Series is:
Reviewed-by Alex Deucher <[email protected]>Hi Jay,
Thanks for the patches, I applied them to amdkfd-fixes (after rebasing them over 4.13-rc1)Oded
Quoted 20 lines
Quoted 20 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +1
-2@@ -140,7 +140,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev)/* According to linux/bitmap.h we shouldn't use bitmap_clear if * nbits is not compile time constant */ - last_valid_bit = adev->gfx.mec.num_mec + last_valid_bit =1 /* only first MEC can have compute queues */ * adev->gfx.mec.num_pipe_per_mec * adev->gfx.mec.num_queue_per_pipe;for (i = last_valid_bit; i <KGD_MAX_QUEUES ; ++i) --2.7.4
Quoted 8 lines
Quoted 4 lines
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Quoted 4 lines
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Quoted 9 lines
-----Original Message-----
From: Kuehling, Felix
Sent: Tuesday, July 18, 2017 12:41 AM
To: [email protected]; Deucher, Alexander
Subject: Re: [PATCH v3 1/4] drm/amdgpu: Fix KFD oversubscription by tracking queues correctlyHi Alex,
Quoted 8 lines
This patch series went into amd-kfd-staging. I'd like to also push it into amd-staging-4.11 as I'm just working to minimize any unnecessary differences between the branches before the big KFD history rework.
I rebased it, resolved some contlicts, and removed the declaration of
get_mec_numfromkfd_device_queue_manager.h. Do you want me to push that
rebased patch series?
Sure. Sounds good.
Alex
Quoted 36 lines
Thanks,
FelixQuoted 1 line · Oded Gabbay
On Fri, Jul 14, 2017 at 7:24 PM, Alex Deucher <[email protected]>
Quoted 1 line · wrote:
Quoted 1 line
On Thu, Jul 13, 2017 at 9:21 PM, Jay Cornwall <[email protected]>
Quoted 26 lines · wrote:
Quoted 8 lines
Quoted 8 lines
The number of compute queues available to the KFD was erroneously calculated as 64. Only the first MEC can execute compute queues and it has 32 queue slots.
This caused the oversubscription limit to be calculated incorrectly, leading to a missing chained runlist command at the end of an oversubscribed runlist.
Quoted 8 lines
Quoted 5 lines
v2: Remove unused
num_mecfield to avoid duplicate logic
v3: Separatenum_mecremoval into separate patchesChange-Id I9e7bba2cc1928b624e3eeb1edb06fdb602e5294fSigned-off-by Jay Cornwall <[email protected]>Series is:
Reviewed-by Alex Deucher <[email protected]>Hi Jay,
Thanks for the patches, I applied them to amdkfd-fixes (after rebasing them over 4.13-rc1)Oded
Quoted 4 lines
Quoted 4 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 30 lines
Quoted 1 line
Quoted 1 line
Review
Quoted 1 line
(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.cQuoted 4 lines
Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c void amdgpu_amdkfd_device_init(struct
Quoted 4 lines
index 7060daf..aa4006a 100644
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +0
-0@@ -140,7 +140,7 @@ void amdgpu_amdkfd_device_init(struct
amdgpu_device*adev)Quoted 23 lines
Quoted 11 lines
Quoted 11 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +1
-2/* According to linux/bitmap.h we shouldn't use bitmap_clear if * nbits is not compile time constant */ - last_valid_bit = adev->gfx.mec.num_mec + last_valid_bit =1 /* only first MEC can have compute queues */ * adev->gfx.mec.num_pipe_per_mec * adev->gfx.mec.num_queue_per_pipe;for (i = last_valid_bit; i <KGD_MAX_QUEUES ; ++i) --2.7.4
Quoted 8 lines
Quoted 4 lines
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx