drm/amdkfd: new event wait support
Subsystems (3)
v2 cover drm/amdkfd: new event wait support
In kernel amdgpu driver, kfd_wait_on_events is used to support user space signal event wait function. For multiple threads waiting on same event scenery, race condition could occur since some threads after checking signal condition, before calling kfd_wait_on_events, the event interrupt could be fired and wake up other thread which are sleeping on this event. Then those threads could fall into sleep without waking up again. Adding event age tracking in both kernel and user mode, will help avoiding this race condition.
The changes for The user space ROCT-Thunk-Interface/ROCR-Runtime are listed below for review togehter with kernel mode changes.
ROCT-Thunk-Interface:
https://github.com/RadeonOpenCompute/ROCT-Thunk-Interface/commit/efdbf6cfbc026bd68ac3c35d00dacf84370eb81e
https://github.com/RadeonOpenCompute/ROCT-Thunk-Interface/commit/1820ae0a2db85b6f584611dc0cde1a00e7c22915
ROCR-Runtime:
https://github.com/RadeonOpenCompute/ROCR-Runtime/compare/master...zhums:ROCR-Runtime:new_event_wait_review
https://github.com/RadeonOpenCompute/ROCR-Runtime/commit/e1f5bdb88eb882ac798aeca2c00ea3fbb2dba459
https://github.com/RadeonOpenCompute/ROCR-Runtime/commit/7d26afd14107b5c2a754c1a3f415d89f3aabb503
-v2: remove unnecessay link
James Zhu (3):
drm/amdkfd: add event age tracking
drm/amdkfd: add event_age tracking when receiving interrupt drm/amdkfd: don't sleep when event age unmatch
drivers/gpu/drm/amd/amdkfd/kfd_events.c | 21 +++++++++++++++++++++ drivers/gpu/drm/amd/amdkfd/kfd_events.h | 1 + include/uapi/linux/kfd_ioctl.h | 13 +++++++++++-- 3 files changed, 33 insertions(+), 2 deletions(-)
Signature · 1 line
2.34.1
PATCH v2 1/3 drm/amdkfd: add event age tracking
Add event age tracking
--- include/uapi/linux/kfd_ioctl.h | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)
include/uapi/linux/kfd_ioctl.h +11 -2
@@ -39,9 +39,10 @@* - 1.11 - Add unified memory for ctx save/restore area * - 1.12 - Add DMA buf export ioctl * - 1.13 - Add debugger API +* - 1.14 - Update kfd_event_data */ #define KFD_IOCTL_MAJOR_VERSION 1 -#define KFD_IOCTL_MINOR_VERSION 13 +#define KFD_IOCTL_MINOR_VERSION 14struct kfd_ioctl_get_version_args {__u32 major_version;/* from KFD */
@@ -320,12 +321,20 @@ struct kfd_hsa_hw_exception_data {__u32 gpu_id; }; +/* hsa signal event data */ +struct kfd_hsa_signal_event_data { +__u64 last_event_age;/* to and from KFD */ +}; +/* Event data */ struct kfd_event_data {union { +/* From KFD */ struct kfd_hsa_memory_exception_data memory_exception_data;struct kfd_hsa_hw_exception_data hw_exception_data; - };/* From KFD */ +/* To and From KFD */ +struct kfd_hsa_signal_event_data signal_event_data; + };__u64 kfd_event_data_ext;/* pointer to an extension structure for future exception types */ __u32 event_id;/* to KFD */
Signature · 1 line
2.34.1
Felix Kuehling
maintainerRe: [PATCH v2 1/3] drm/amdkfd: add event age tracking
Quoted 12 lines · James Zhu
Add event age tracking
Signed-off-by James Zhu <[email protected]>--- include/uapi/linux/kfd_ioctl.h | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)include/uapi/linux/kfd_ioctl.h +0
-0
include/uapi/linux/kfd_ioctl.h line 45
Quoted 8 lines
include/uapi/linux/kfd_ioctl.h +2
-1@@ -39,9 +39,10 @@* - 1.11 - Add unified memory for ctx save/restore area * - 1.12 - Add DMA buf export ioctl * - 1.13 - Add debugger API +* - 1.14 - Update kfd_event_data */ #define KFD_IOCTL_MAJOR_VERSION 1 -#define KFD_IOCTL_MINOR_VERSION 13 +#define KFD_IOCTL_MINOR_VERSION 14
Bumping the version number should be done in the last patch in the series, once the feature is fully enabled.
Regards,
Felix
Quoted 25 lines
include/uapi/linux/kfd_ioctl.h +9
-1struct kfd_ioctl_get_version_args {__u32 major_version;/* from KFD */ @@ -320,12 +321,20 @@ struct kfd_hsa_hw_exception_data {__u32 gpu_id; }; +/* hsa signal event data */ +struct kfd_hsa_signal_event_data { +__u64 last_event_age;/* to and from KFD */ +}; +/* Event data */ struct kfd_event_data {union { +/* From KFD */ struct kfd_hsa_memory_exception_data memory_exception_data;struct kfd_hsa_hw_exception_data hw_exception_data; - };/* From KFD */ +/* To and From KFD */ +struct kfd_hsa_signal_event_data signal_event_data; + };__u64 kfd_event_data_ext;/* pointer to an extension structure for future exception types */ __u32 event_id;/* to KFD */
Re: [PATCH v2 1/3] drm/amdkfd: add event age tracking
Quoted 8 lines · Felix Kuehling
Quoted 7 lines · James Zhu
Add event age tracking
Signed-off-by James Zhu <[email protected]>---
include/uapi/linux/kfd_ioctl.h| 13 +++++++++++--1 file changed, 11 insertions(+), 2 deletions(-)
Quoted 15 lines
include/uapi/linux/kfd_ioctl.h line 45
Quoted 13 lines
include/uapi/linux/kfd_ioctl.h +2
-1@@ -39,9 +39,10 @@* - 1.11 - Add unified memory for ctx save/restore area * - 1.12 - Add DMA buf export ioctl * - 1.13 - Add debugger API +* - 1.14 - Update kfd_event_data */ #define KFD_IOCTL_MAJOR_VERSION 1 -#define KFD_IOCTL_MINOR_VERSION 13 +#define KFD_IOCTL_MINOR_VERSION 14Bumping the version number should be done in the last patch in the series, once the feature is fully enabled.
[JZ] Noted, Thanks!
Quoted 28 lines
Regards,
FelixQuoted 23 lines
include/uapi/linux/kfd_ioctl.h +0
-0struct kfd_ioctl_get_version_args {
__u32major_version; /* from KFD */include/uapi/linux/kfd_ioctl.h +0
-0@@ -320,12 +321,20 @@ struct kfd_hsa_hw_exception_data { __u32 gpu_id; };+/* hsa signal event data */
include/uapi/linux/kfd_ioctl.h +4
-0+struct kfd_hsa_signal_event_data { + __u64 last_event_age; /* to and from KFD */ +}; +/* Event data */
structkfd_event_data{
union {
+ /* From KFD */
structkfd_hsa_memory_exception_datamemory_exception_data; structkfd_hsa_hw_exception_datahw_exception_data; - }; /* From KFD */
+ /* To and From KFD */
+ structkfd_hsa_signal_event_datasignal_event_data; + };
__u64kfd_event_data_ext; /* pointer to an extension structure for future exception types */
__u32event_id; /* to KFD */
PATCH v2 2/3 drm/amdkfd: add event_age tracking when receiving interrupt
In reply to James ZhuAdd event_age tracking when receiving interrupt.
--- drivers/gpu/drm/amd/amdkfd/kfd_events.c | 6 ++++++ drivers/gpu/drm/amd/amdkfd/kfd_events.h | 1 + 2 files changed, 7 insertions(+)
drivers/gpu/drm/amd/amdkfd/kfd_events.c +6 -0
@@ -431,6 +431,7 @@ int kfd_event_create(struct file *devkfd, struct kfd_process *p,if (!ret) { *event_id = ev->event_id; *event_trigger_data = ev->event_id; + ev->event_age =1 ; }else {kfree (ev); }
@@ -629,6 +630,11 @@ static void set_event(struct kfd_event *ev)* updating the wait queues in kfd_wait_on_events. */ ev->signaled = !ev->auto_reset || !waitqueue_active (&ev->wq); +if (!(++ev->event_age)) { +/* Never wrap back to reserved/default event age 0/1 */ + ev->event_age =2 ; +WARN_ONCE (1 ,"event_age wrap back!" ); + }list_for_each_entry (waiter, &ev->wq.head, wait.entry)WRITE_ONCE (waiter->activated, true);
drivers/gpu/drm/amd/amdkfd/kfd_events.h +1 -0
@@ -53,6 +53,7 @@ struct signal_page;struct kfd_event {u32 event_id; +u64 event_age;bool signaled;bool auto_reset;
Signature · 1 line
2.34.1
PATCH v2 3/3 drm/amdkfd: don't sleep when event age unmatch
In reply to James ZhuDon't sleep when event age unmatch, and update last_event_age. It is only for KFD_EVENT_TYPE_SIGNAL which is checked by user space.
--- drivers/gpu/drm/amd/amdkfd/kfd_events.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
drivers/gpu/drm/amd/amdkfd/kfd_events.c +15 -0
@@ -952,6 +952,21 @@ int kfd_wait_on_events(struct kfd_process *p, event_data.event_id);if (ret) goto out_unlock; + +/* last_event_age = 0 reserved for backward compatible */ +if (event_data.signal_event_data.last_event_age && + event_waiters[i].event->event_age != + event_data.signal_event_data.last_event_age) { + event_data.signal_event_data.last_event_age = + event_waiters[i].event->event_age; +WRITE_ONCE (event_waiters[i].activated, true); + +if (copy_to_user (&events[i], &event_data, +sizeof (struct kfd_event_data ))) { + ret = -EFAULT ; + goto out_unlock; + } + } }/* Check condition once. */
Signature · 1 line
2.34.1
Felix Kuehling
maintainerRe: [PATCH v2 3/3] drm/amdkfd: don't sleep when event age unmatch
Quoted 13 lines · James Zhu
Don't sleep when event age unmatch, and update
last_event_age. It is only forKFD_EVENT_TYPE_SIGNALwhich is checked by user space.Signed-off-by James Zhu <[email protected]>--- drivers/gpu/drm/amd/amdkfd/kfd_events.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)drivers/gpu/drm/amd/amdkfd/kfd_events.c +0
-0
drivers/gpu/drm/amd/amdkfd/kfd_events.c int kfd_wait_on_events(struct kfd_process *p, · line 961
Quoted 10 lines
drivers/gpu/drm/amd/amdkfd/kfd_events.c +7
-0@@ -952,6 +952,21 @@ int kfd_wait_on_events(struct kfd_process *p, event_data.event_id);if (ret) goto out_unlock; + +/* last_event_age = 0 reserved for backward compatible */ +if (event_data.signal_event_data.last_event_age && + event_waiters[i].event->event_age != + event_data.signal_event_data.last_event_age) { + event_data.signal_event_data.last_event_age = + event_waiters[i].event->event_age;
The event_age is updated in set_event under the event->spin_lock. You need to take that lock for this check here as well.
I think the easiest way to do this would be to move the check into init_event_waiter. That way you can initialize the waiter as activated if the event age is not up to date.
drivers/gpu/drm/amd/amdkfd/kfd_events.c int kfd_wait_on_events(struct kfd_process *p, · line 969
Quoted 8 lines
drivers/gpu/drm/amd/amdkfd/kfd_events.c +8
-0+WRITE_ONCE (event_waiters[i].activated, true); + +if (copy_to_user (&events[i], &event_data, +sizeof (struct kfd_event_data ))) { + ret = -EFAULT ; + goto out_unlock; + } + }
I think we also need to update the event age in event data after an event has signaled. You should probably move updating and copying of the event age to user mode into copy_signaled_event_data. That way it would handle all the cases.
Regards,
Felix
Quoted 3 lines
drivers/gpu/drm/amd/amdkfd/kfd_events.c +0
-0}/* Check condition once. */
Re: [PATCH v2 3/3] drm/amdkfd: don't sleep when event age unmatch
Quoted 27 lines · Felix Kuehling
Quoted 13 lines · James Zhu
Don't sleep when event age unmatch, and update
last_event_age. It is only forKFD_EVENT_TYPE_SIGNALwhich is checked by user space.Signed-off-by James Zhu <[email protected]>---
drivers/gpu/drm/amd/amdkfd/kfd_events.c| 15 +++++++++++++++1 file changed, 15 insertions(+)drivers/gpu/drm/amd/amdkfd/kfd_events.c +0
-0drivers/gpu/drm/amd/amdkfd/kfd_events.c int kfd_wait_on_events(struct kfd_process *p, · line 961
Quoted 10 lines
drivers/gpu/drm/amd/amdkfd/kfd_events.c +7
-0@@ -952,6 +952,21 @@ int kfd_wait_on_events(struct kfd_process *p, event_data.event_id); if (ret) goto out_unlock; + + /* last_event_age = 0 reserved for backward compatible */ + if (event_data.signal_event_data.last_event_age && + event_waiters[i].event->event_age != + event_data.signal_event_data.last_event_age) { + event_data.signal_event_data.last_event_age = + event_waiters[i].event->event_age;The
event_ageis updated inset_eventunder theevent->spin_lock. You need to take that lock for this check here as well.
Quoted 1 line
I think the easiest way to do this would be to move the check into
init_event_waiter. That way you can initialize the waiter as activated if the event age is not up to date.
[JZ] Sure
Quoted 11 lines
drivers/gpu/drm/amd/amdkfd/kfd_events.c int kfd_wait_on_events(struct kfd_process *p, · line 969
Quoted 10 lines
drivers/gpu/drm/amd/amdkfd/kfd_events.c +8
-0+WRITE_ONCE (event_waiters[i].activated, true); + + if (copy_to_user(&events[i], &event_data, + sizeof(struct kfd_event_data))) { + ret = -EFAULT; + goto out_unlock; + } + }
Quoted 1 line
I think we also need to update the event age in event data after an event has signaled. You should probably move updating and copying of the event age to user mode into
copy_signaled_event_data. That way it would handle all the cases.
[JZ] Sure
Quoted 7 lines
Regards,
FelixQuoted 2 lines
drivers/gpu/drm/amd/amdkfd/kfd_events.c +0
-0} /* Check condition once. */