eventdev: fix race condition on timer list counter
PATCH 1/3 eventdev: fix race condition on timer list counter
The n_poll_lcores counter and poll_lcore array are shared between lcores and the update of these variables are out of the protection of spinlock on each lcore timer list. The read-modify-write operations of the counter are not atomic, so it has the potential of race condition between lcores.
Use c11 atomics with RELAXED ordering to prevent confliction.
--- lib/librte_eventdev/rte_event_timer_adapter.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +12 -4
@@ -583,6 +583,7 @@ swtim_callback(struct rte_timer *tim)uint16_t nb_evs_invalid =0 ;uint64_t opaque;int ret; +int n_lcores; opaque = evtim->impl_opaque[1 ]; adapter = (struct rte_event_timer_adapter *)(uintptr_t )opaque;
@@ -605,8 +606,12 @@ swtim_callback(struct rte_timer *tim) "with immediate expiry value"); } -if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore].v))) - sw->poll_lcores[sw->n_poll_lcores++] = lcore; +if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore].v))) { + n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , + __ATOMIC_RELAXED); +__atomic_store_n (&sw->poll_lcores[n_lcores], lcore, + __ATOMIC_RELAXED); + } }else {EVTIM_BUF_LOG_DBG ("buffered an event timer expiry event" );
@@ -1011,6 +1016,7 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,uint32_t lcore_id =rte_lcore_id ();struct rte_timer *tim, *tims[nb_evtims];uint64_t cycles; +int n_lcores;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */
@@ -1033,8 +1039,10 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore_id].v))) {EVTIM_LOG_DBG ("Adding lcore id = %u to list of lcores to poll" , lcore_id); - sw->poll_lcores[sw->n_poll_lcores] = lcore_id; - ++sw->n_poll_lcores; + n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , + __ATOMIC_RELAXED); +__atomic_store_n (&sw->poll_lcores[n_lcores], lcore_id, + __ATOMIC_RELAXED); } ret = rte_mempool_get_bulk(sw->tim_pool, (void **)tims,
Signature · 1 line
2.7.4
PATCH 2/3 eventdev: use c11 atomics for lcore timer armed flag
The in_use flag is a per core variable which is not shared between lcores in the normal case and the access of this variable should be ordered on the same core. However, if non-EAL thread pick the highest lcore to insert timers into, there is the possibility of conflicts on this flag between threads. Then the atomic CAS operation is needed.
Use the c11 atomic CAS instead of the generic rte_atomic operations to avoid the unnecessary barrier on aarch64.
--- lib/librte_eventdev/rte_event_timer_adapter.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +11 -4
@@ -554,7 +554,7 @@ struct swtim {uint32_t timer_data_id;/* Track which cores have actually armed a timer */ struct { -rte_atomic16_t v; +int16_t v; } __rte_cache_aligned in_use[RTE_MAX_LCORE];/* Track which cores' timer lists should be polled */ unsigned int poll_lcores[RTE_MAX_LCORE ];
@@ -606,7 +606,8 @@ swtim_callback(struct rte_timer *tim) "with immediate expiry value"); } -if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore].v))) { +if (unlikely (sw->in_use[lcore].v ==0 )) { + sw->in_use[lcore].v =1 ; n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , __ATOMIC_RELAXED); __atomic_store_n(&sw->poll_lcores[n_lcores], lcore,
@@ -834,7 +835,7 @@ swtim_init(struct rte_event_timer_adapter *adapter)/* Initialize the variables that track in-use timer lists */ for (i =0 ; i <RTE_MAX_LCORE ; i++) -rte_atomic16_init (&sw->in_use[i].v); + sw->in_use[i].v =0 ;/* Initialize the timer subsystem and allocate timer data instance */ ret =rte_timer_subsystem_init ();
@@ -1017,6 +1018,8 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,struct rte_timer *tim, *tims[nb_evtims];uint64_t cycles;int n_lcores; +/* Timer is not armed state */ +int16_t exp_state =0 ;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */
@@ -1035,8 +1038,12 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,/* If this is the first time we're arming an event timer on this lcore, * mark this lcore as "in use"; this will cause the service * function to process the timer list that corresponds to this lcore. +* The atomic CAS operation can prevent the race condition on in_use +* flag between multiple non-EAL threads. */ -if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore_id].v))) { +if (unlikely (__atomic_compare_exchange_n (&sw->in_use[lcore_id].v, + &exp_state,1 ,0 , + __ATOMIC_RELAXED, __ATOMIC_RELAXED))) {EVTIM_LOG_DBG ("Adding lcore id = %u to list of lcores to poll" , lcore_id); n_lcores = __atomic_fetch_add(&sw->n_poll_lcores, 1,
Signature · 1 line
2.7.4
Re: [dpdk-dev] [PATCH 2/3] eventdev: use c11 atomics for lcore timer armed flag
Hi Phil,
Comment in-line:
Quoted 61 lines
-----Original Message-----
From: Phil Yang <[email protected]>
Sent: Friday, June 12, 2020 6:20 AM
To: [email protected]; Carrillo, Erik G <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH 2/3] eventdev: use c11 atomics for lcore timer armed flagThe
in_useflag is a per core variable which is not shared between lcores in the normal case and the access of this variable should be ordered on the same core. However, if non-EAL thread pick the highest lcore to insert timers into, there is the possibility of conflicts on this flag between threads. Then the atomic CAS operation is needed.Use the c11 atomic CAS instead of the generic
rte_atomicoperations to avoid the unnecessary barrier on aarch64.Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>--- lib/librte_eventdev/rte_event_timer_adapter.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)lib/librte_eventdev/rte_event_timer_adapter.c +1
-1@@ -554,7 +554,7 @@ struct swtim {uint32_t timer_data_id;/* Track which cores have actually armed a timer */ struct { -rte_atomic16_t v; +int16_t v; } __rte_cache_aligned in_use[RTE_MAX_LCORE];/* Track which cores' timer lists should be polled */ unsigned int poll_lcores[RTE_MAX_LCORE]; @@ -606,7 +606,8 @@
swtim_callback(struct rte_timer *tim)lib/librte_eventdev/rte_event_timer_adapter.c +0
-1"with immediate expiry value"); } - if (unlikely(rte_atomic16_test_and_set(&sw-Quoted 1 line
in_use[lcore].v))) {
lib/librte_eventdev/rte_event_timer_adapter.c +2
-0+if (unlikely (sw->in_use[lcore].v ==0 )) { + sw->in_use[lcore].v =1 ; n_lcores = __atomic_fetch_add(&sw->n_poll_lcores,1,
lib/librte_eventdev/rte_event_timer_adapter.c +0
-0__ATOMIC_RELAXED); __atomic_store_n(&sw->poll_lcores[n_lcores],lcore, @@ -834,7 +835,7 @@
swtim_init(struct rte_event_timer_adapter *adapter)lib/librte_eventdev/rte_event_timer_adapter.c +1
-1/* Initialize the variables that track in-use timer lists */ for (i =0 ; i <RTE_MAX_LCORE ; i++) -rte_atomic16_init (&sw->in_use[i].v); + sw->in_use[i].v =0 ;/* Initialize the timer subsystem and allocate timer data instance */ ret =rte_timer_subsystem_init ();
lib/librte_eventdev/rte_event_timer_adapter.c __swtim_arm_burst(const struct · line 1022
Quoted 5 lines
lib/librte_eventdev/rte_event_timer_adapter.c +1
-0@@ -1017,6 +1018,8 @@ __swtim_arm_burst(const structrte_event_timer_adapter *adapter, struct rte_timer *tim, *tims[nb_evtims];uint64_t cycles;int n_lcores; +/* Timer is not armed state */
A more accurate comment would be something like "Timer list for this lcore is not in use".
With that change, it looks good to me:
Quoted 23 lines
lib/librte_eventdev/rte_event_timer_adapter.c +1
-0+int16_t exp_state =0 ;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */ @@ -1035,8 +1038,12 @@
__swtim_arm_burst(const structrte_event_timer_adapter*adapter,lib/librte_eventdev/rte_event_timer_adapter.c +1
-0/* If this is the first time we're arming an event timer on this lcore, * mark this lcore as "in use"; this will cause the service * function to process the timer list that corresponds to this lcore. +* The atomic CAS operation can prevent the race condition on
in_uselib/librte_eventdev/rte_event_timer_adapter.c +2
-1+* flag between multiple non-EAL threads. */ -if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore_id].v))) { + if (unlikely(__atomic_compare_exchange_n(&sw-Quoted 1 line
in_use[lcore_id].v,
lib/librte_eventdev/rte_event_timer_adapter.c +2
-0+ &exp_state, 1, 0, + __ATOMIC_RELAXED, __ATOMIC_RELAXED))) { EVTIM_LOG_DBG("Adding lcore id = %u to list of lcores topoll",
lib/librte_eventdev/rte_event_timer_adapter.c +0
-1lcore_id); n_lcores = __atomic_fetch_add(&sw->n_poll_lcores, 1, --2.7.4
Re: [dpdk-dev] [PATCH 2/3] eventdev: use c11 atomics for lcore timer armed flag
Hi Erik,
Sorry, I was on vacation.
Thanks for your feedback. I will update it in the next version.
Thanks,
Phil
Quoted 113 lines
-----Original Message-----
From: Carrillo, Erik G <[email protected]>
Sent: Wednesday, June 24, 2020 5:02 AM
To: Phil Yang <[email protected]>; [email protected]
Cc: [email protected]; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>; Dharmik Thakkar <[email protected]>; nd <[email protected]>
Subject: RE: [PATCH 2/3] eventdev: use c11 atomics for lcore timer armed flagHi Phil,
Comment in-line:
Quoted 11 lines
-----Original Message-----
From: Phil Yang <[email protected]>
Sent: Friday, June 12, 2020 6:20 AM
To: [email protected]; Carrillo, Erik G <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH 2/3] eventdev: use c11 atomics for lcore timer armed flagThe
in_useflag is a per core variable which is not shared between lcores in the normal case and the access of this variable should be ordered on the same core. However, if non-EAL thread pick the highest lcore to inserttimers
Quoted 4 lines
into, there is the possibility of conflicts on this flag between threads. Then the atomic CAS operation is needed.
Use the c11 atomic CAS instead of the generic
rte_atomicoperations toavoid
Quoted 46 lines
the unnecessary barrier on aarch64.
Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>--- lib/librte_eventdev/rte_event_timer_adapter.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)lib/librte_eventdev/rte_event_timer_adapter.c +1
-1@@ -554,7 +554,7 @@ struct swtim {uint32_t timer_data_id;/* Track which cores have actually armed a timer */ struct { -rte_atomic16_t v; +int16_t v; } __rte_cache_aligned in_use[RTE_MAX_LCORE];/* Track which cores' timer lists should be polled */ unsigned int poll_lcores[RTE_MAX_LCORE]; @@ -606,7 +606,8 @@
swtim_callback(struct rte_timer *tim)lib/librte_eventdev/rte_event_timer_adapter.c +0
-1"with immediate expiry value"); } - if (unlikely(rte_atomic16_test_and_set(&sw-Quoted 1 line
in_use[lcore].v))) {
lib/librte_eventdev/rte_event_timer_adapter.c +2
-0+if (unlikely (sw->in_use[lcore].v ==0 )) { + sw->in_use[lcore].v =1 ; n_lcores = __atomic_fetch_add(&sw->n_poll_lcores,1,
lib/librte_eventdev/rte_event_timer_adapter.c +0
-0__ATOMIC_RELAXED); __atomic_store_n(&sw->poll_lcores[n_lcores],lcore, @@ -834,7 +835,7 @@
swtim_init(struct rte_event_timer_adapter *adapter)lib/librte_eventdev/rte_event_timer_adapter.c +1
-1/* Initialize the variables that track in-use timer lists */ for (i =0 ; i <RTE_MAX_LCORE ; i++) -rte_atomic16_init (&sw->in_use[i].v); + sw->in_use[i].v =0 ;/* Initialize the timer subsystem and allocate timer data instance */ ret =rte_timer_subsystem_init ();lib/librte_eventdev/rte_event_timer_adapter.c __swtim_arm_burst(const struct · line 1022
Quoted 5 lines
lib/librte_eventdev/rte_event_timer_adapter.c +1
-0@@ -1017,6 +1018,8 @@ __swtim_arm_burst(const structrte_event_timer_adapter *adapter, struct rte_timer *tim, *tims[nb_evtims];uint64_t cycles;int n_lcores; +/* Timer is not armed state */ A more accurate comment would be something like "Timer list for this lcore is not in use".
With that change, it looks good to me:
Quoted 23 lines
lib/librte_eventdev/rte_event_timer_adapter.c +1
-0+int16_t exp_state =0 ;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */ @@ -1035,8 +1038,12 @@
__swtim_arm_burst(const structrte_event_timer_adapter*adapter,lib/librte_eventdev/rte_event_timer_adapter.c +1
-0/* If this is the first time we're arming an event timer on this lcore, * mark this lcore as "in use"; this will cause the service * function to process the timer list that corresponds to this lcore. +* The atomic CAS operation can prevent the race condition on
in_uselib/librte_eventdev/rte_event_timer_adapter.c +2
-1+* flag between multiple non-EAL threads. */ -if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore_id].v))) { + if (unlikely(__atomic_compare_exchange_n(&sw-Quoted 1 line
in_use[lcore_id].v,
lib/librte_eventdev/rte_event_timer_adapter.c +2
-0+ &exp_state, 1, 0, + __ATOMIC_RELAXED, __ATOMIC_RELAXED))) { EVTIM_LOG_DBG("Adding lcore id = %u to list of lcores topoll",
lib/librte_eventdev/rte_event_timer_adapter.c +0
-1lcore_id); n_lcores = __atomic_fetch_add(&sw->n_poll_lcores, 1, --2.7.4
Re: [dpdk-dev] [PATCH 2/3] eventdev: use c11 atomics for lcore timer armed flag
In reply to Phil Yanglib/librte_eventdev/rte_event_timer_adapter.c struct swtim { · line 558
Quoted 5 lines · On Fri, 12 Jun 2020 19:19:57 +0800 Phil Yang <[email protected]> wrote:
lib/librte_eventdev/rte_event_timer_adapter.c +1
-1/* Track which cores have actually armed a timer */ struct { -rte_atomic16_t v; +int16_t v; } __rte_cache_aligned in_use[RTE_MAX_LCORE];
Do you really need this to be cache aligned (ie one per line)? Why have a signed value for a reference count? Shouldn't it be unsigned?
Re: [dpdk-dev] [PATCH 2/3] eventdev: use c11 atomics for lcore timer armed flag
Quoted 13 lines
-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Tuesday, June 23, 2020 4:20 PM
To: Phil Yang <[email protected]>
Cc: [email protected]; Carrillo, Erik G <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [dpdk-dev] [PATCH 2/3] eventdev: use c11 atomics for lcore timer armed flagOn Fri, 12 Jun 2020 19:19:57 +0800
Phil Yang <[email protected]> wrote:
Quoted 7 lines
lib/librte_eventdev/rte_event_timer_adapter.c struct swtim { · line 558
Quoted 5 lines
lib/librte_eventdev/rte_event_timer_adapter.c +1
-1/* Track which cores have actually armed a timer */ struct { -rte_atomic16_t v; +int16_t v; } __rte_cache_aligned in_use[RTE_MAX_LCORE];Do you really need this to be cache aligned (ie one per line)?
I believe I did this originally to keep a cache line from bouncing when two different cores are arming timers, so it's not strictly necessary.
Quoted 1 line
Why have a signed value for a reference count? Shouldn't it be unsigned?
Re: [dpdk-dev] [PATCH 2/3] eventdev: use c11 atomics for lcore timer armed flag
Quoted 33 lines
-----Original Message-----
From: Carrillo, Erik G <[email protected]>
Sent: Wednesday, June 24, 2020 5:32 AM
To: Stephen Hemminger <[email protected]>; Phil Yang <[email protected]>
Cc: [email protected]; [email protected]; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>; Dharmik Thakkar <[email protected]>; nd <[email protected]>
Subject: RE: [dpdk-dev] [PATCH 2/3] eventdev: use c11 atomics for lcore timer armed flagQuoted 13 lines
-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Tuesday, June 23, 2020 4:20 PM
To: Phil Yang <[email protected]>
Cc: [email protected]; Carrillo, Erik G <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [dpdk-dev] [PATCH 2/3] eventdev: use c11 atomics for lcore timer armed flagOn Fri, 12 Jun 2020 19:19:57 +0800
Phil Yang <[email protected]> wrote:Quoted 7 lines
lib/librte_eventdev/rte_event_timer_adapter.c struct swtim { · line 558
Quoted 5 lines
lib/librte_eventdev/rte_event_timer_adapter.c +1
-1/* Track which cores have actually armed a timer */ struct { -rte_atomic16_t v; +int16_t v; } __rte_cache_aligned in_use[RTE_MAX_LCORE];Do you really need this to be cache aligned (ie one per line)?
Quoted 2 lines
I believe I did this originally to keep a cache line from bouncing when two different cores are arming timers, so it's not strictly necessary.
Yeah, if we remove it, these per core variables might cause a false sharing issue between threads. That will hurt performance.
Quoted 2 lines
Quoted 2 lines
Why have a signed value for a reference count? Shouldn't it be unsigned?
Yes. It should be unsigned in the new code.
I will update it in the next version.
Thanks,
Phil
PATCH 3/3 eventdev: relax smp barriers with c11 atomics
In reply to Phil YangThe implementation-specific opaque data is shared between arm and cancel operations. The state flag acts as a guard variable to make sure the update of opaque data is synchronized. This patch uses c11 atomics with explicit one way memory barrier instead of full barriers rte_smp_w/rmb() to synchronize the opaque data between timer arm and cancel threads.
--- lib/librte_eventdev/rte_event_timer_adapter.c | 55 ++++++++++++++++++--------- lib/librte_eventdev/rte_event_timer_adapter.h | 2 +- 2 files changed, 38 insertions(+), 19 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +37 -18
@@ -629,7 +629,8 @@ swtim_callback(struct rte_timer *tim) sw->expired_timers[sw->n_expired_timers++] = tim; sw->stats.evtim_exp_count++; - evtim->state =RTE_EVENT_TIMER_NOT_ARMED ; +__atomic_store_n (&evtim->state,RTE_EVENT_TIMER_NOT_ARMED , + __ATOMIC_RELEASE); }if (event_buffer_batch_ready (&sw->buffer)) {
@@ -1020,6 +1021,7 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,int n_lcores;/* Timer is not armed state */ int16_t exp_state =0 ; +enum rte_event_timer_state n_state;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */
@@ -1060,30 +1062,36 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter, }for (i =0 ; i < nb_evtims; i++) { -/* Don't modify the event timer state in these cases */ -if (evtims[i]->state ==RTE_EVENT_TIMER_ARMED ) { + n_state =__atomic_load_n (&evtims[i]->state, __ATOMIC_RELAXED); +if (n_state ==RTE_EVENT_TIMER_ARMED ) { rte_errno =EALREADY ;break ; - }else if (!(evtims[i]->state ==RTE_EVENT_TIMER_NOT_ARMED || - evtims[i]->state ==RTE_EVENT_TIMER_CANCELED )) { + }else if (!(n_state ==RTE_EVENT_TIMER_NOT_ARMED || + n_state ==RTE_EVENT_TIMER_CANCELED )) { rte_errno =EINVAL ;break ; } ret =check_timeout (evtims[i], adapter);if (unlikely (ret ==-1 )) { - evtims[i]->state =RTE_EVENT_TIMER_ERROR_TOOLATE ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR_TOOLATE , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }else if (unlikely (ret ==-2 )) { - evtims[i]->state =RTE_EVENT_TIMER_ERROR_TOOEARLY ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR_TOOEARLY , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }if (unlikely (check_destination_event_queue (evtims[i], adapter) <0 )) { - evtims[i]->state =RTE_EVENT_TIMER_ERROR ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }
@@ -1099,13 +1107,18 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter, SINGLE, lcore_id, NULL, evtims[i]);if (ret <0 ) {/* tim was in RUNNING or CONFIG state */ - evtims[i]->state =RTE_EVENT_TIMER_ERROR ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR , + __ATOMIC_RELEASE);break ; } -rte_smp_wmb ();EVTIM_LOG_DBG ("armed an event timer" ); - evtims[i]->state =RTE_EVENT_TIMER_ARMED ; +/* RELEASE ordering guarantees the adapter specific value +* changes observed before the update of state. +*/ +__atomic_store_n (&evtims[i]->state,RTE_EVENT_TIMER_ARMED , + __ATOMIC_RELEASE); }if (i < nb_evtims)
@@ -1132,6 +1145,7 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,struct rte_timer *timp;uint64_t opaque;struct swtim *sw =swtim_pmd_priv (adapter); +enum rte_event_timer_state n_state;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */
@@ -1143,16 +1157,18 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,for (i =0 ; i < nb_evtims; i++) {/* Don't modify the event timer state in these cases */ -if (evtims[i]->state ==RTE_EVENT_TIMER_CANCELED ) { +/* ACQUIRE ordering guarantees the access of implementation +* specific opague data under the correct state. +*/ + n_state =__atomic_load_n (&evtims[i]->state, __ATOMIC_ACQUIRE); +if (n_state ==RTE_EVENT_TIMER_CANCELED ) { rte_errno =EALREADY ;break ; - }else if (evtims[i]->state !=RTE_EVENT_TIMER_ARMED ) { + }else if (n_state !=RTE_EVENT_TIMER_ARMED ) { rte_errno =EINVAL ;break ; } -rte_smp_rmb (); - opaque = evtims[i]->impl_opaque[0 ]; timp = (struct rte_timer *)(uintptr_t )opaque;RTE_ASSERT (timp !=NULL );
@@ -1166,11 +1182,14 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,rte_mempool_put (sw->tim_pool, (void **)timp); - evtims[i]->state =RTE_EVENT_TIMER_CANCELED ; +__atomic_store_n (&evtims[i]->state,RTE_EVENT_TIMER_CANCELED , + __ATOMIC_RELAXED); evtims[i]->impl_opaque[0 ] =0 ; evtims[i]->impl_opaque[1 ] =0 ; - -rte_smp_wmb (); +/* The RELEASE fence make sure the clean up +* of opaque data observed between threads. +*/ +__atomic_thread_fence (__ATOMIC_RELEASE); }return i;
lib/librte_eventdev/rte_event_timer_adapter.h +1 -1
@@ -467,7 +467,7 @@ struct rte_event_timer {* - op: RTE_EVENT_OP_NEW * - event_type: RTE_EVENT_TYPE_TIMER */ -volatile enum rte_event_timer_state state; +enum rte_event_timer_state state;/**< State of the event timer. */ uint64_t timeout_ticks;/**< Expiry timer ticks expressed in number of *timer_ticks_ns* from
Signature · 1 line
2.7.4
Re: [dpdk-dev] [PATCH 3/3] eventdev: relax smp barriers with c11 atomics
Quoted 162 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of Phil Yang
Sent: Friday, June 12, 2020 7:20 PM
To: [email protected]; [email protected]
Cc: [email protected]; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>; Dharmik Thakkar <[email protected]>; nd <[email protected]>
Subject: [dpdk-dev] [PATCH 3/3] eventdev: relax smp barriers with c11 atomicsThe implementation-specific opaque data is shared between arm and cancel operations. The state flag acts as a guard variable to make sure the update of opaque data is synchronized. This patch uses c11 atomics with explicit one way memory barrier instead of full barriers rte_smp_w/rmb() to synchronize the opaque data between timer arm and cancel threads.
Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>--- lib/librte_eventdev/rte_event_timer_adapter.c | 55 ++++++++++++++++++--------- lib/librte_eventdev/rte_event_timer_adapter.h | 2 +- 2 files changed, 38 insertions(+), 19 deletions(-)lib/librte_eventdev/rte_event_timer_adapter.c +2
-1@@ -629,7 +629,8 @@ swtim_callback(struct rte_timer *tim) sw->expired_timers[sw->n_expired_timers++] = tim; sw->stats.evtim_exp_count++; - evtim->state =RTE_EVENT_TIMER_NOT_ARMED ; +__atomic_store_n (&evtim->state,RTE_EVENT_TIMER_NOT_ARMED , + __ATOMIC_RELEASE); }lib/librte_eventdev/rte_event_timer_adapter.c +1
-0if (event_buffer_batch_ready (&sw->buffer)) {@@ -1020,6 +1021,7 @@ __swtim_arm_burst(const structrte_event_timer_adapter *adapter, int n_lcores;/* Timer is not armed state */ int16_t exp_state =0 ; +enum rte_event_timer_state n_state;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG lib/librte_eventdev/rte_event_timer_adapter.c +2
-3/* Check that the service is running. */ @@ -1060,30 +1062,36 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter, }for (i =0 ; i < nb_evtims; i++) { -/* Don't modify the event timer state in these cases */ -if (evtims[i]->state ==RTE_EVENT_TIMER_ARMED ) { + n_state =__atomic_load_n (&evtims[i]->state, __ATOMIC_RELAXED); +if (n_state ==RTE_EVENT_TIMER_ARMED ) { rte_errno =EALREADY ;break ; - } else if (!(evtims[i]->state ==
RTE_EVENT_TIMER_NOT_ARMED||lib/librte_eventdev/rte_event_timer_adapter.c +0
-1- evtims[i]->state ==
RTE_EVENT_TIMER_CANCELED)) {lib/librte_eventdev/rte_event_timer_adapter.c +2
-1+ }else if (!(n_state ==RTE_EVENT_TIMER_NOT_ARMED || + n_state ==RTE_EVENT_TIMER_CANCELED )) { rte_errno =EINVAL ;break ; } ret =check_timeout (evtims[i], adapter);if (unlikely (ret ==-1 )) { - evtims[i]->state =
RTE_EVENT_TIMER_ERROR_TOOLATE;lib/librte_eventdev/rte_event_timer_adapter.c +3
-1+__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR_TOOLATE , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }else if (unlikely (ret ==-2 )) { - evtims[i]->state =
RTE_EVENT_TIMER_ERROR_TOOEARLY;lib/librte_eventdev/rte_event_timer_adapter.c +2
-0+ __atomic_store_n(&evtims[i]->state, +
RTE_EVENT_TIMER_ERROR_TOOEARLY,lib/librte_eventdev/rte_event_timer_adapter.c +12
-4+ __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }if (unlikely (check_destination_event_queue (evtims[i], adapter) <0 )) { - evtims[i]->state =RTE_EVENT_TIMER_ERROR ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }@@ -1099,13 +1107,18 @@ __swtim_arm_burst(const structrte_event_timer_adapter *adapter, SINGLE, lcore_id, NULL, evtims[i]);if (ret <0 ) {/* tim was in RUNNING or CONFIG state */ - evtims[i]->state =RTE_EVENT_TIMER_ERROR ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR , + __ATOMIC_RELEASE);break ; } -rte_smp_wmb ();EVTIM_LOG_DBG ("armed an event timer" ); - evtims[i]->state =RTE_EVENT_TIMER_ARMED ; +/* RELEASE ordering guarantees the adapter specific value +* changes observed before the update of state. +*/ +__atomic_store_n (&evtims[i]->state,RTE_EVENT_TIMER_ARMED , + __ATOMIC_RELEASE); }lib/librte_eventdev/rte_event_timer_adapter.c +1
-0if (i < nb_evtims)@@ -1132,6 +1145,7 @@ swtim_cancel_burst(const structrte_event_timer_adapter *adapter, struct rte_timer *timp;uint64_t opaque;struct swtim *sw =swtim_pmd_priv (adapter); +enum rte_event_timer_state n_state;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG lib/librte_eventdev/rte_event_timer_adapter.c +1
-1/* Check that the service is running. */ @@ -1143,16 +1157,18 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter, for (i = 0; i < nb_evtims; i++) {/* Don't modify the event timer state in these cases */ -if (evtims[i]->state ==RTE_EVENT_TIMER_CANCELED ) { +/* ACQUIRE ordering guarantees the access of implementation
lib/librte_eventdev/rte_event_timer_adapter.c +3
-0+* specific opague data under the correct state. +*/ + n_state = __atomic_load_n(&evtims[i]->state,
__ATOMIC_ACQUIRE);lib/librte_eventdev/rte_event_timer_adapter.c +2
-3+if (n_state ==RTE_EVENT_TIMER_CANCELED ) { rte_errno =EALREADY ;break ; - }else if (evtims[i]->state !=RTE_EVENT_TIMER_ARMED ) { + }else if (n_state !=RTE_EVENT_TIMER_ARMED ) { rte_errno =EINVAL ;break ; } -rte_smp_rmb (); - opaque = evtims[i]->impl_opaque[0 ]; timp = (struct rte_timer *)(uintptr_t )opaque;RTE_ASSERT (timp !=NULL );
lib/librte_eventdev/rte_event_timer_adapter.c swtim_cancel_burst(const struct · line 1190
Quoted 10 lines
lib/librte_eventdev/rte_event_timer_adapter.c +2
-1@@ -1166,11 +1182,14 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,rte_mempool_put (sw->tim_pool, (void **)timp); - evtims[i]->state =RTE_EVENT_TIMER_CANCELED ; +__atomic_store_n (&evtims[i]->state,RTE_EVENT_TIMER_CANCELED , + __ATOMIC_RELAXED); evtims[i]->impl_opaque[0 ] =0 ; evtims[i]->impl_opaque[1 ] =0 ;
Is that safe to remove impl_opaque cleanup above?
Once the soft timer canceled, the __swtim_arm_burst cannot access these two fields under the RTE_EVENT_TIMER_CANCELED state. After new timer armed, it refills these two fields in the __swtim_arm_burst thread, which is the only producer of these two fields. I think the only risk is that the values of these two field might be unknow after swtim_cancel_burst.
So it should be safe to remove them if no other thread access them after canceling the timer.
Any comments on this?
If we remove these two instructions, we can also remove the __atomic_thread_fence below to save performance penalty.
Thanks,
Phil
Quoted 26 lines
lib/librte_eventdev/rte_event_timer_adapter.c +4
-2- -rte_smp_wmb (); +/* The RELEASE fence make sure the clean up +* of opaque data observed between threads. +*/ +__atomic_thread_fence (__ATOMIC_RELEASE); }lib/librte_eventdev/rte_event_timer_adapter.c +0
-0return i;lib/librte_eventdev/rte_event_timer_adapter.h +1
-1@@ -467,7 +467,7 @@ struct rte_event_timer {* - op: RTE_EVENT_OP_NEW * - event_type: RTE_EVENT_TYPE_TIMER */ -volatile enum rte_event_timer_state state; +enum rte_event_timer_state state;/**< State of the event timer. */ uint64_t timeout_ticks;/**< Expiry timer ticks expressed in number of *timer_ticks_ns* from
--
2.7.4
Re: [dpdk-dev] [PATCH 3/3] eventdev: relax smp barriers with c11 atomics
Hi Phil,
Comment in-line:
Quoted 196 lines
-----Original Message-----
From: Phil Yang <[email protected]>
Sent: Monday, June 22, 2020 5:12 AM
To: Phil Yang <[email protected]>; [email protected]; Carrillo, Erik G <[email protected]>
Cc: [email protected]; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>; Dharmik Thakkar <[email protected]>; nd <[email protected]>
Subject: RE: [dpdk-dev] [PATCH 3/3] eventdev: relax smp barriers with c11 atomicsQuoted 6 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of Phil Yang
Sent: Friday, June 12, 2020 7:20 PM
To: [email protected]; [email protected]
Cc: [email protected]; Honnappa Nagarahalli <[email protected]>; Ruifeng WangQuoted 9 lines
Dharmik Thakkar <[email protected]>; nd <[email protected]>
Subject: [dpdk-dev] [PATCH 3/3] eventdev: relax smp barriers with c11 atomicsThe implementation-specific opaque data is shared between arm and cancel operations. The state flag acts as a guard variable to make sure the update of opaque data is synchronized. This patch uses c11 atomics with explicit one way memory barrier instead of full barriers rte_smp_w/rmb() to synchronize the opaque data between timer arm and
cancel threads.
Quoted 25 lines
Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>--- lib/librte_eventdev/rte_event_timer_adapter.c | 55 ++++++++++++++++++--------- lib/librte_eventdev/rte_event_timer_adapter.h | 2 +- 2 files changed, 38 insertions(+), 19 deletions(-)lib/librte_eventdev/rte_event_timer_adapter.c +2
-1@@ -629,7 +629,8 @@ swtim_callback(struct rte_timer *tim) sw->expired_timers[sw->n_expired_timers++] = tim; sw->stats.evtim_exp_count++; - evtim->state =RTE_EVENT_TIMER_NOT_ARMED ; +__atomic_store_n (&evtim->state,RTE_EVENT_TIMER_NOT_ARMED , + __ATOMIC_RELEASE); }Quoted 1 line
if (
event_buffer_batch_ready(&sw->buffer)) { @@ -1020,6 +1021,7@@
Quoted 33 lines
__swtim_arm_burst(const struct
rte_event_timer_adapter*adapter,lib/librte_eventdev/rte_event_timer_adapter.c +1
-0int n_lcores;/* Timer is not armed state */ int16_t exp_state =0 ; +enum rte_event_timer_state n_state;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */ @@ -1060,30 +1062,36 @@__swtim_arm_burst(const struct
rte_event_timer_adapter*adapter,lib/librte_eventdev/rte_event_timer_adapter.c +1
-2}for (i =0 ; i < nb_evtims; i++) { -/* Don't modify the event timer state in these cases */ -if (evtims[i]->state ==RTE_EVENT_TIMER_ARMED ) { + n_state = __atomic_load_n(&evtims[i]->state,
__ATOMIC_RELAXED);lib/librte_eventdev/rte_event_timer_adapter.c +1
-1+if (n_state ==RTE_EVENT_TIMER_ARMED ) { rte_errno =EALREADY ;break ; - } else if (!(evtims[i]->state ==
RTE_EVENT_TIMER_NOT_ARMED||lib/librte_eventdev/rte_event_timer_adapter.c +0
-1- evtims[i]->state ==
RTE_EVENT_TIMER_CANCELED)) {lib/librte_eventdev/rte_event_timer_adapter.c +2
-1+ }else if (!(n_state ==RTE_EVENT_TIMER_NOT_ARMED || + n_state ==RTE_EVENT_TIMER_CANCELED )) { rte_errno =EINVAL ;break ; } ret =check_timeout (evtims[i], adapter);if (unlikely (ret ==-1 )) { - evtims[i]->state =
RTE_EVENT_TIMER_ERROR_TOOLATE;lib/librte_eventdev/rte_event_timer_adapter.c
Quoted 2 lines
lib/librte_eventdev/rte_event_timer_adapter.c +2
-0+ __atomic_store_n(&evtims[i]->state, +
RTE_EVENT_TIMER_ERROR_TOOLATE,Quoted 82 lines
lib/librte_eventdev/rte_event_timer_adapter.c +1
-1+ __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }else if (unlikely (ret ==-2 )) { - evtims[i]->state =
RTE_EVENT_TIMER_ERROR_TOOEARLY;lib/librte_eventdev/rte_event_timer_adapter.c +2
-0+ __atomic_store_n(&evtims[i]->state, +
RTE_EVENT_TIMER_ERROR_TOOEARLY,lib/librte_eventdev/rte_event_timer_adapter.c +12
-4+ __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }if (unlikely (check_destination_event_queue (evtims[i], adapter) <0 )) { - evtims[i]->state =RTE_EVENT_TIMER_ERROR ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }@@ -1099,13 +1107,18 @@ __swtim_arm_burst(const structrte_event_timer_adapter *adapter, SINGLE, lcore_id, NULL, evtims[i]);if (ret <0 ) {/* tim was in RUNNING or CONFIG state */ - evtims[i]->state =RTE_EVENT_TIMER_ERROR ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR , + __ATOMIC_RELEASE);break ; } -rte_smp_wmb ();EVTIM_LOG_DBG ("armed an event timer" ); - evtims[i]->state =RTE_EVENT_TIMER_ARMED ; +/* RELEASE ordering guarantees the adapter specific value +* changes observed before the update of state. +*/ +__atomic_store_n (&evtims[i]->state,RTE_EVENT_TIMER_ARMED , + __ATOMIC_RELEASE); }lib/librte_eventdev/rte_event_timer_adapter.c +1
-0if (i < nb_evtims)@@ -1132,6 +1145,7 @@ swtim_cancel_burst(const structrte_event_timer_adapter *adapter, struct rte_timer *timp;uint64_t opaque;struct swtim *sw =swtim_pmd_priv (adapter); +enum rte_event_timer_state n_state;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */ @@ -1143,16 +1157,18 @@
swtim_cancel_burst(const structrte_event_timer_adapter*adapter,lib/librte_eventdev/rte_event_timer_adapter.c +1
-1for (i =0 ; i < nb_evtims; i++) {/* Don't modify the event timer state in these cases */ -if (evtims[i]->state ==RTE_EVENT_TIMER_CANCELED ) { +/* ACQUIRE ordering guarantees the access of implementation
lib/librte_eventdev/rte_event_timer_adapter.c +3
-0+* specific opague data under the correct state. +*/ + n_state = __atomic_load_n(&evtims[i]->state,
__ATOMIC_ACQUIRE);lib/librte_eventdev/rte_event_timer_adapter.c +2
-3+if (n_state ==RTE_EVENT_TIMER_CANCELED ) { rte_errno =EALREADY ;break ; - }else if (evtims[i]->state !=RTE_EVENT_TIMER_ARMED ) { + }else if (n_state !=RTE_EVENT_TIMER_ARMED ) { rte_errno =EINVAL ;break ; } -rte_smp_rmb (); - opaque = evtims[i]->impl_opaque[0 ]; timp = (struct rte_timer *)(uintptr_t )opaque;RTE_ASSERT (timp !=NULL );lib/librte_eventdev/rte_event_timer_adapter.c swtim_cancel_burst(const struct · line 1190
Quoted 10 lines
lib/librte_eventdev/rte_event_timer_adapter.c +2
-1@@ -1166,11 +1182,14 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,rte_mempool_put (sw->tim_pool, (void **)timp); - evtims[i]->state =RTE_EVENT_TIMER_CANCELED ; +__atomic_store_n (&evtims[i]->state,RTE_EVENT_TIMER_CANCELED , + __ATOMIC_RELAXED); evtims[i]->impl_opaque[0 ] =0 ; evtims[i]->impl_opaque[1 ] =0 ;Is that safe to remove
impl_opaquecleanup above?Once the soft timer canceled, the
__swtim_arm_burstcannot access these two fields under theRTE_EVENT_TIMER_CANCELEDstate.
After new timer armed, it refills these two fields in the__swtim_arm_burstthread, which is the only producer of these two fields.
I think the only risk is that the values of these two field might be unknow afterswtim_cancel_burst.
So it should be safe to remove them if no other thread access them after canceling the timer.
Quoted 7 lines
Any comments on this?
If we remove these two instructions, we can also remove the__atomic_thread_fencebelow to save performance penalty.Thanks,
Phil
In this case, I see the fence as (more importantly) ensuring the state update is visible to other threads... do I misunderstand? I suppose we could also update the state with an __atomic_store(..., __ATOMIC_RELEASE), but perhaps that roughly equivalent?
Quoted 26 lines
Quoted 15 lines
lib/librte_eventdev/rte_event_timer_adapter.c +4
-2- -rte_smp_wmb (); +/* The RELEASE fence make sure the clean up +* of opaque data observed between threads. +*/ +__atomic_thread_fence (__ATOMIC_RELEASE); }lib/librte_eventdev/rte_event_timer_adapter.c +0
-0return i;lib/librte_eventdev/rte_event_timer_adapter.h +0
-0lib/librte_eventdev/rte_event_timer_adapter.h struct rte_event_timer { · line 473
Quoted 8 lines
lib/librte_eventdev/rte_event_timer_adapter.h +1
-1@@ -467,7 +467,7 @@ struct rte_event_timer {* - op: RTE_EVENT_OP_NEW * - event_type: RTE_EVENT_TYPE_TIMER */ -volatile enum rte_event_timer_state state; +enum rte_event_timer_state state;/**< State of the event timer. */ uint64_t timeout_ticks;/**< Expiry timer ticks expressed in number of *timer_ticks_ns* from
Quoted 2 lines
--
2.7.4
Re: [dpdk-dev] [PATCH 3/3] eventdev: relax smp barriers with c11 atomics
Hi Erick,
Quoted 216 lines
-----Original Message-----
From: Carrillo, Erik G <[email protected]>
Sent: Wednesday, June 24, 2020 3:39 AM
To: Phil Yang <[email protected]>; [email protected]
Cc: [email protected]; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>; Dharmik Thakkar <[email protected]>; nd <[email protected]>
Subject: RE: [dpdk-dev] [PATCH 3/3] eventdev: relax smp barriers with c11 atomicsHi Phil,
Comment in-line:
Quoted 12 lines
-----Original Message-----
From: Phil Yang <[email protected]>
Sent: Monday, June 22, 2020 5:12 AM
To: Phil Yang <[email protected]>; [email protected]; Carrillo, Erik G <[email protected]>
Cc: [email protected]; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>; Dharmik Thakkar <[email protected]>; nd <[email protected]>
Subject: RE: [dpdk-dev] [PATCH 3/3] eventdev: relax smp barriers with c11 atomicsQuoted 16 lines
Quoted 6 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of Phil Yang
Sent: Friday, June 12, 2020 7:20 PM
To: [email protected]; [email protected]
Cc: [email protected]; Honnappa Nagarahalli <[email protected]>; Ruifeng WangQuoted 9 lines
Dharmik Thakkar <[email protected]>; nd <[email protected]>
Subject: [dpdk-dev] [PATCH 3/3] eventdev: relax smp barriers with c11 atomicsThe implementation-specific opaque data is shared between arm and cancel operations. The state flag acts as a guard variable to make sure the update of opaque data is synchronized. This patch uses c11 atomics with explicit one way memory barrier instead of full barriers rte_smp_w/rmb() to synchronize the opaque data between timer arm
and
Quoted 172 lines
cancel threads.
Quoted 25 lines
Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>--- lib/librte_eventdev/rte_event_timer_adapter.c | 55 ++++++++++++++++++--------- lib/librte_eventdev/rte_event_timer_adapter.h | 2 +- 2 files changed, 38 insertions(+), 19 deletions(-)lib/librte_eventdev/rte_event_timer_adapter.c +2
-1@@ -629,7 +629,8 @@ swtim_callback(struct rte_timer *tim) sw->expired_timers[sw->n_expired_timers++] = tim; sw->stats.evtim_exp_count++; - evtim->state =RTE_EVENT_TIMER_NOT_ARMED ; +__atomic_store_n (&evtim->state,RTE_EVENT_TIMER_NOT_ARMED , + __ATOMIC_RELEASE); }Quoted 1 line
if (
event_buffer_batch_ready(&sw->buffer)) { @@ -1020,6 +1021,7@@
Quoted 33 lines
__swtim_arm_burst(const struct
rte_event_timer_adapter*adapter,lib/librte_eventdev/rte_event_timer_adapter.c +1
-0int n_lcores;/* Timer is not armed state */ int16_t exp_state =0 ; +enum rte_event_timer_state n_state;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */ @@ -1060,30 +1062,36 @@__swtim_arm_burst(const struct
rte_event_timer_adapter*adapter,lib/librte_eventdev/rte_event_timer_adapter.c +1
-2}for (i =0 ; i < nb_evtims; i++) { -/* Don't modify the event timer state in these cases */ -if (evtims[i]->state ==RTE_EVENT_TIMER_ARMED ) { + n_state = __atomic_load_n(&evtims[i]->state,
__ATOMIC_RELAXED);lib/librte_eventdev/rte_event_timer_adapter.c +1
-1+if (n_state ==RTE_EVENT_TIMER_ARMED ) { rte_errno =EALREADY ;break ; - } else if (!(evtims[i]->state ==
RTE_EVENT_TIMER_NOT_ARMED||lib/librte_eventdev/rte_event_timer_adapter.c +0
-1- evtims[i]->state ==
RTE_EVENT_TIMER_CANCELED)) {lib/librte_eventdev/rte_event_timer_adapter.c +2
-1+ }else if (!(n_state ==RTE_EVENT_TIMER_NOT_ARMED || + n_state ==RTE_EVENT_TIMER_CANCELED )) { rte_errno =EINVAL ;break ; } ret =check_timeout (evtims[i], adapter);if (unlikely (ret ==-1 )) { - evtims[i]->state =
RTE_EVENT_TIMER_ERROR_TOOLATE;lib/librte_eventdev/rte_event_timer_adapter.c
Quoted 2 lines
lib/librte_eventdev/rte_event_timer_adapter.c +2
-0+ __atomic_store_n(&evtims[i]->state, +
RTE_EVENT_TIMER_ERROR_TOOLATE,Quoted 82 lines
lib/librte_eventdev/rte_event_timer_adapter.c +1
-1+ __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }else if (unlikely (ret ==-2 )) { - evtims[i]->state =
RTE_EVENT_TIMER_ERROR_TOOEARLY;lib/librte_eventdev/rte_event_timer_adapter.c +2
-0+ __atomic_store_n(&evtims[i]->state, +
RTE_EVENT_TIMER_ERROR_TOOEARLY,lib/librte_eventdev/rte_event_timer_adapter.c +12
-4+ __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }if (unlikely (check_destination_event_queue (evtims[i], adapter) <0 )) { - evtims[i]->state =RTE_EVENT_TIMER_ERROR ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }@@ -1099,13 +1107,18 @@ __swtim_arm_burst(const structrte_event_timer_adapter *adapter, SINGLE, lcore_id, NULL, evtims[i]);if (ret <0 ) {/* tim was in RUNNING or CONFIG state */ - evtims[i]->state =RTE_EVENT_TIMER_ERROR ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR , + __ATOMIC_RELEASE);break ; } -rte_smp_wmb ();EVTIM_LOG_DBG ("armed an event timer" ); - evtims[i]->state =RTE_EVENT_TIMER_ARMED ; +/* RELEASE ordering guarantees the adapter specific value +* changes observed before the update of state. +*/ +__atomic_store_n (&evtims[i]->state,RTE_EVENT_TIMER_ARMED , + __ATOMIC_RELEASE); }lib/librte_eventdev/rte_event_timer_adapter.c +1
-0if (i < nb_evtims)@@ -1132,6 +1145,7 @@ swtim_cancel_burst(const structrte_event_timer_adapter *adapter, struct rte_timer *timp;uint64_t opaque;struct swtim *sw =swtim_pmd_priv (adapter); +enum rte_event_timer_state n_state;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */ @@ -1143,16 +1157,18 @@
swtim_cancel_burst(const structrte_event_timer_adapter*adapter,lib/librte_eventdev/rte_event_timer_adapter.c +1
-1for (i =0 ; i < nb_evtims; i++) {/* Don't modify the event timer state in these cases */ -if (evtims[i]->state ==RTE_EVENT_TIMER_CANCELED ) { +/* ACQUIRE ordering guarantees the access of implementation
lib/librte_eventdev/rte_event_timer_adapter.c +3
-0+* specific opague data under the correct state. +*/ + n_state = __atomic_load_n(&evtims[i]->state,
__ATOMIC_ACQUIRE);lib/librte_eventdev/rte_event_timer_adapter.c +2
-3+if (n_state ==RTE_EVENT_TIMER_CANCELED ) { rte_errno =EALREADY ;break ; - }else if (evtims[i]->state !=RTE_EVENT_TIMER_ARMED ) { + }else if (n_state !=RTE_EVENT_TIMER_ARMED ) { rte_errno =EINVAL ;break ; } -rte_smp_rmb (); - opaque = evtims[i]->impl_opaque[0 ]; timp = (struct rte_timer *)(uintptr_t )opaque;RTE_ASSERT (timp !=NULL );lib/librte_eventdev/rte_event_timer_adapter.c swtim_cancel_burst(const struct · line 1190
Quoted 10 lines
lib/librte_eventdev/rte_event_timer_adapter.c +2
-1@@ -1166,11 +1182,14 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,rte_mempool_put (sw->tim_pool, (void **)timp); - evtims[i]->state =RTE_EVENT_TIMER_CANCELED ; +__atomic_store_n (&evtims[i]->state,RTE_EVENT_TIMER_CANCELED , + __ATOMIC_RELAXED); evtims[i]->impl_opaque[0 ] =0 ; evtims[i]->impl_opaque[1 ] =0 ;Is that safe to remove
impl_opaquecleanup above?Once the soft timer canceled, the
__swtim_arm_burstcannot access these two fields under theRTE_EVENT_TIMER_CANCELEDstate.
After new timer armed, it refills these two fields in the__swtim_arm_burstthread, which is the only producer of these two fields.
I think the only risk is that the values of these two field might be unknow afterswtim_cancel_burst.
So it should be safe to remove them if no other thread access them after canceling the timer.Any comments on this?
If we remove these two instructions, we can also remove the__atomic_thread_fencebelow to save performance penalty.
Quoted 8 lines
Quoted 3 lines
Thanks,
PhilIn this case, I see the fence as (more importantly) ensuring the state update
is visible to other threads... do I misunderstand? I suppose we could also
update the state with an__atomic_store(..., __ATOMIC_RELEASE), but perhaps that roughly equivalent?
Yeah. In my understanding, the fence ensures the state and the implementation-specific opaque data update are visible between other timer arm and cancel threads. Actually, we only care about the state's value here.
The atomic RELEASE can also make sure all writes in the current thread are visible in other threads that acquire the same atomic variable. So I think we can remove the fence and update the state with RELEASE then load the state with ACQUIRE in the timer arm and the cancel threads to achieve the same goal.
Quoted 27 lines
Quoted 27 lines
Quoted 16 lines
lib/librte_eventdev/rte_event_timer_adapter.c +4
-2- -rte_smp_wmb (); +/* The RELEASE fence make sure the clean up +* of opaque data observed between threads. +*/ +__atomic_thread_fence (__ATOMIC_RELEASE); }lib/librte_eventdev/rte_event_timer_adapter.c +0
-0return i;lib/librte_eventdev/rte_event_timer_adapter.h +0
-0lib/librte_eventdev/rte_event_timer_adapter.h struct rte_event_timer { · line 473
Quoted 8 lines
lib/librte_eventdev/rte_event_timer_adapter.h +1
-1@@ -467,7 +467,7 @@ struct rte_event_timer {* - op: RTE_EVENT_OP_NEW * - event_type: RTE_EVENT_TYPE_TIMER */ -volatile enum rte_event_timer_state state; +enum rte_event_timer_state state;/**< State of the event timer. */ uint64_t timeout_ticks;/**< Expiry timer ticks expressed in number of *timer_ticks_ns* from
Quoted 2 lines
--
2.7.4
Re: [dpdk-dev] [PATCH 3/3] eventdev: relax smp barriers with c11 atomics
Quoted 241 lines
-----Original Message-----
From: Phil Yang <[email protected]>
Sent: Sunday, June 28, 2020 12:34 PM
To: Carrillo, Erik G <[email protected]>; [email protected]
Cc: [email protected]; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>; Dharmik Thakkar <[email protected]>; nd <[email protected]>
Subject: RE: [dpdk-dev] [PATCH 3/3] eventdev: relax smp barriers with c11 atomicsHi Erick,
Quoted 6 lines
-----Original Message-----
From: Carrillo, Erik G <[email protected]>
Sent: Wednesday, June 24, 2020 3:39 AM
To: Phil Yang <[email protected]>; [email protected]
Cc: [email protected]; Honnappa Nagarahalli <[email protected]>; Ruifeng WangQuoted 4 lines
Dharmik Thakkar <[email protected]>; nd <[email protected]>
Subject: RE: [dpdk-dev] [PATCH 3/3] eventdev: relax smp barriers with c11 atomicsQuoted 11 lines
Hi Phil,
Comment in-line:
Quoted 7 lines
-----Original Message-----
From: Phil Yang <[email protected]>
Sent: Monday, June 22, 2020 5:12 AM
To: Phil Yang <[email protected]>; [email protected]; Carrillo, Erik G <[email protected]>
Cc: [email protected]; Honnappa Nagarahalli <[email protected]>; Ruifeng WangQuoted 15 lines
Quoted 15 lines
Dharmik Thakkar <[email protected]>; nd <[email protected]>
Subject: RE: [dpdk-dev] [PATCH 3/3] eventdev: relax smp barriers with c11 atomicsQuoted 6 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of Phil Yang
Sent: Friday, June 12, 2020 7:20 PM
To: [email protected]; [email protected]
Cc: [email protected]; Honnappa Nagarahalli <[email protected]>; Ruifeng WangQuoted 4 lines
Dharmik Thakkar <[email protected]>; nd <[email protected]>
Subject: [dpdk-dev] [PATCH 3/3] eventdev: relax smp barriers with c11 atomicsQuoted 170 lines
Quoted 6 lines
Quoted 6 lines
The implementation-specific opaque data is shared between arm and cancel operations. The state flag acts as a guard variable to make sure the update of opaque data is synchronized. This patch uses c11 atomics with explicit one way memory barrier instead of full barriers
rte_smp_w/rmb() to synchronize the opaque data between timer armand
Quoted 163 lines
cancel threads.
Quoted 25 lines
Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>--- lib/librte_eventdev/rte_event_timer_adapter.c | 55 ++++++++++++++++++--------- lib/librte_eventdev/rte_event_timer_adapter.h | 2 +- 2 files changed, 38 insertions(+), 19 deletions(-)lib/librte_eventdev/rte_event_timer_adapter.c +2
-1@@ -629,7 +629,8 @@ swtim_callback(struct rte_timer *tim) sw->expired_timers[sw->n_expired_timers++] = tim; sw->stats.evtim_exp_count++; - evtim->state =RTE_EVENT_TIMER_NOT_ARMED ; +__atomic_store_n (&evtim->state,RTE_EVENT_TIMER_NOT_ARMED , + __ATOMIC_RELEASE); }Quoted 1 line
if (
event_buffer_batch_ready(&sw->buffer)) { @@ -1020,6 +1021,7@@
Quoted 33 lines
__swtim_arm_burst(const struct
rte_event_timer_adapter*adapter,lib/librte_eventdev/rte_event_timer_adapter.c +1
-0int n_lcores;/* Timer is not armed state */ int16_t exp_state =0 ; +enum rte_event_timer_state n_state;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */ @@ -1060,30 +1062,36 @@__swtim_arm_burst(const struct
rte_event_timer_adapter*adapter,lib/librte_eventdev/rte_event_timer_adapter.c +1
-2}for (i =0 ; i < nb_evtims; i++) { -/* Don't modify the event timer state in these cases */ -if (evtims[i]->state ==RTE_EVENT_TIMER_ARMED ) { + n_state = __atomic_load_n(&evtims[i]->state,
__ATOMIC_RELAXED);lib/librte_eventdev/rte_event_timer_adapter.c +1
-1+if (n_state ==RTE_EVENT_TIMER_ARMED ) { rte_errno =EALREADY ;break ; - } else if (!(evtims[i]->state ==
RTE_EVENT_TIMER_NOT_ARMED||lib/librte_eventdev/rte_event_timer_adapter.c +0
-1- evtims[i]->state ==
RTE_EVENT_TIMER_CANCELED)) {lib/librte_eventdev/rte_event_timer_adapter.c +2
-1+ }else if (!(n_state ==RTE_EVENT_TIMER_NOT_ARMED || + n_state ==RTE_EVENT_TIMER_CANCELED )) { rte_errno =EINVAL ;break ; } ret =check_timeout (evtims[i], adapter);if (unlikely (ret ==-1 )) { - evtims[i]->state =
RTE_EVENT_TIMER_ERROR_TOOLATE;lib/librte_eventdev/rte_event_timer_adapter.c
Quoted 2 lines
lib/librte_eventdev/rte_event_timer_adapter.c +2
-0+ __atomic_store_n(&evtims[i]->state, +
RTE_EVENT_TIMER_ERROR_TOOLATE,Quoted 82 lines
lib/librte_eventdev/rte_event_timer_adapter.c +1
-1+ __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }else if (unlikely (ret ==-2 )) { - evtims[i]->state =
RTE_EVENT_TIMER_ERROR_TOOEARLY;lib/librte_eventdev/rte_event_timer_adapter.c +2
-0+ __atomic_store_n(&evtims[i]->state, +
RTE_EVENT_TIMER_ERROR_TOOEARLY,lib/librte_eventdev/rte_event_timer_adapter.c +12
-4+ __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }if (unlikely (check_destination_event_queue (evtims[i], adapter) <0 )) { - evtims[i]->state =RTE_EVENT_TIMER_ERROR ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }@@ -1099,13 +1107,18 @@ __swtim_arm_burst(const structrte_event_timer_adapter *adapter, SINGLE, lcore_id, NULL, evtims[i]);if (ret <0 ) {/* tim was in RUNNING or CONFIG state */ - evtims[i]->state =RTE_EVENT_TIMER_ERROR ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR , + __ATOMIC_RELEASE);break ; } -rte_smp_wmb ();EVTIM_LOG_DBG ("armed an event timer" ); - evtims[i]->state =RTE_EVENT_TIMER_ARMED ; +/* RELEASE ordering guarantees the adapter specific value +* changes observed before the update of state. +*/ +__atomic_store_n (&evtims[i]->state,RTE_EVENT_TIMER_ARMED , + __ATOMIC_RELEASE); }lib/librte_eventdev/rte_event_timer_adapter.c +1
-0if (i < nb_evtims)@@ -1132,6 +1145,7 @@ swtim_cancel_burst(const structrte_event_timer_adapter *adapter, struct rte_timer *timp;uint64_t opaque;struct swtim *sw =swtim_pmd_priv (adapter); +enum rte_event_timer_state n_state;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */ @@ -1143,16 +1157,18 @@
swtim_cancel_burst(const structrte_event_timer_adapter*adapter,lib/librte_eventdev/rte_event_timer_adapter.c +1
-1for (i =0 ; i < nb_evtims; i++) {/* Don't modify the event timer state in these cases */ -if (evtims[i]->state ==RTE_EVENT_TIMER_CANCELED ) { +/* ACQUIRE ordering guarantees the access of implementation
lib/librte_eventdev/rte_event_timer_adapter.c +3
-0+* specific opague data under the correct state. +*/ + n_state = __atomic_load_n(&evtims[i]->state,
__ATOMIC_ACQUIRE);lib/librte_eventdev/rte_event_timer_adapter.c +2
-3+if (n_state ==RTE_EVENT_TIMER_CANCELED ) { rte_errno =EALREADY ;break ; - }else if (evtims[i]->state !=RTE_EVENT_TIMER_ARMED ) { + }else if (n_state !=RTE_EVENT_TIMER_ARMED ) { rte_errno =EINVAL ;break ; } -rte_smp_rmb (); - opaque = evtims[i]->impl_opaque[0 ]; timp = (struct rte_timer *)(uintptr_t )opaque;RTE_ASSERT (timp !=NULL );lib/librte_eventdev/rte_event_timer_adapter.c swtim_cancel_burst(const struct · line 1190
Quoted 10 lines
lib/librte_eventdev/rte_event_timer_adapter.c +2
-1@@ -1166,11 +1182,14 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,rte_mempool_put (sw->tim_pool, (void **)timp); - evtims[i]->state =RTE_EVENT_TIMER_CANCELED ; +__atomic_store_n (&evtims[i]->state,RTE_EVENT_TIMER_CANCELED , + __ATOMIC_RELAXED); evtims[i]->impl_opaque[0 ] =0 ; evtims[i]->impl_opaque[1 ] =0 ;Is that safe to remove
impl_opaquecleanup above?Once the soft timer canceled, the
__swtim_arm_burstcannot access these two fields under theRTE_EVENT_TIMER_CANCELEDstate. After new timer armed, it refills these two fields in the__swtim_arm_burstthread, which is the only producer of these twofields.
Quoted 9 lines
Quoted 9 lines
I think the only risk is that the values of these two field might be unknow after
swtim_cancel_burst.
So it should be safe to remove them if no other thread access them after canceling the timer.Any comments on this?
If we remove these two instructions, we can also remove the__atomic_thread_fencebelow to save performance penalty.Quoted 5 lines
Quoted 3 lines
Thanks,
PhilIn this case, I see the fence as (more importantly) ensuring the state
update
Quoted 3 lines
is visible to other threads... do I misunderstand? I suppose we could also
update the state with an__atomic_store(..., __ATOMIC_RELEASE), but perhaps that roughly equivalent?
Quoted 9 lines
Yeah. In my understanding, the fence ensures the state and the implementation-specific opaque data update are visible between other timer arm and cancel threads.
Actually, we only care about the state's value here.
The atomic RELEASE can also make sure all writes in the current thread are visible in other threads that acquire the same atomic variable. So I think we can remove the fence and update the state with RELEASE then load the state with ACQUIRE in the timer arm and the cancel threads to achieve the same goal.
Ok, that sounds good to me.
Thanks,
Erik
Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Quoted 17 lines
lib/librte_eventdev/rte_event_timer_adapter.c +4
-2- -rte_smp_wmb (); +/* The RELEASE fence make sure the clean up +* of opaque data observed between threads. +*/ +__atomic_thread_fence (__ATOMIC_RELEASE); }lib/librte_eventdev/rte_event_timer_adapter.c +0
-0return i;lib/librte_eventdev/rte_event_timer_adapter.h +0
-0lib/librte_eventdev/rte_event_timer_adapter.h struct rte_event_timer { · line 473
Quoted 8 lines
lib/librte_eventdev/rte_event_timer_adapter.h +1
-1@@ -467,7 +467,7 @@ struct rte_event_timer {* - op: RTE_EVENT_OP_NEW * - event_type: RTE_EVENT_TYPE_TIMER */ -volatile enum rte_event_timer_state state; +enum rte_event_timer_state state;/**< State of the event timer. */ uint64_t timeout_ticks;/**< Expiry timer ticks expressed in number of *timer_ticks_ns* from
Quoted 2 lines
--
2.7.4
Hi Phil,
Good catch - thanks for the fix. I've commented in-line:
Quoted 42 lines
-----Original Message-----
From: Phil Yang <[email protected]>
Sent: Friday, June 12, 2020 6:20 AM
To: [email protected]; Carrillo, Erik G <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH 1/3] eventdev: fix race condition on timer list counterThe
n_poll_lcorescounter andpoll_lcorearray are shared between lcores and the update of these variables are out of the protection of spinlock on each lcore timer list. The read-modify-write operations of the counter are not atomic, so it has the potential of race condition between lcores.Use c11 atomics with RELAXED ordering to prevent confliction.
Fixes cc7b73ea9e3b ("eventdev: add new software timer adapter")Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>--- lib/librte_eventdev/rte_event_timer_adapter.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)lib/librte_eventdev/rte_event_timer_adapter.c +1
-0@@ -583,6 +583,7 @@ swtim_callback(struct rte_timer *tim)uint16_t nb_evs_invalid =0 ;uint64_t opaque;int ret; +int n_lcores; opaque = evtim->impl_opaque[1 ]; adapter = (struct rte_event_timer_adapter *)(uintptr_t )opaque;@@ -605,8 +606,12 @@ swtim_callback(struct rte_timer *tim) "with immediate expiry value");
lib/librte_eventdev/rte_event_timer_adapter.c swtim_callback(struct rte_timer *tim) · line 614
Quoted 10 lines
lib/librte_eventdev/rte_event_timer_adapter.c +3
-2} -if (unlikely (rte_atomic16_test_and_set (&sw- >in_use[lcore].v))) - sw->poll_lcores[sw->n_poll_lcores++] = lcore; +if (unlikely (rte_atomic16_test_and_set (&sw- >in_use[lcore].v))) { + n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , + __ATOMIC_RELAXED);
Just a nit, but let's align the continued line with the opening parentheses in this location and below. With these changes:
Quoted 33 lines
lib/librte_eventdev/rte_event_timer_adapter.c +1
-0+ __atomic_store_n(&sw->poll_lcores[n_lcores],lcore,
lib/librte_eventdev/rte_event_timer_adapter.c +2
-0+ __ATOMIC_RELAXED); + } }else { EVTIM_BUF_LOG_DBG("buffered an event timer expiryevent");
lib/librte_eventdev/rte_event_timer_adapter.c +1
-0@@ -1011,6 +1016,7 @@ __swtim_arm_burst(const structrte_event_timer_adapter *adapter, uint32_t lcore_id = rte_lcore_id();struct rte_timer *tim, *tims[nb_evtims];uint64_t cycles; +int n_lcores;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */ @@ -1033,8 +1039,10 @@
__swtim_arm_burst(const structrte_event_timer_adapter*adapter,lib/librte_eventdev/rte_event_timer_adapter.c +0
-0if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore_id].v))) { EVTIM_LOG_DBG("Adding lcore id = %u to list of lcores topoll",
lib/librte_eventdev/rte_event_timer_adapter.c +4
-3lcore_id); - sw->poll_lcores[sw->n_poll_lcores] = lcore_id; - ++sw->n_poll_lcores; + n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , + __ATOMIC_RELAXED); +__atomic_store_n (&sw->poll_lcores[n_lcores], lcore_id, + __ATOMIC_RELAXED); } ret = rte_mempool_get_bulk(sw->tim_pool, (void **)tims, --2.7.4
<snip>
Quoted 51 lines
Hi Phil,
Good catch - thanks for the fix. I've commented in-line:
Quoted 10 lines
-----Original Message-----
From: Phil Yang <[email protected]>
Sent: Friday, June 12, 2020 6:20 AM
To: [email protected]; Carrillo, Erik G <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH 1/3] eventdev: fix race condition on timer list counterQuoted 4 lines
The
n_poll_lcorescounter andpoll_lcorearray are shared between lcores and the update of these variables are out of the protection of spinlock on each lcore timer list. The read-modify-write operations of the counter are not atomic, so it has the potential of race conditionbetween lcores.
Quoted 31 lines
Use c11 atomics with RELAXED ordering to prevent confliction.
Fixes cc7b73ea9e3b ("eventdev: add new software timer adapter")Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>--- lib/librte_eventdev/rte_event_timer_adapter.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)lib/librte_eventdev/rte_event_timer_adapter.c +1
-1@@ -583,6 +583,7 @@ swtim_callback(struct rte_timer *tim)uint16_t nb_evs_invalid =0 ;uint64_t opaque;int ret; +int n_lcores; opaque = evtim->impl_opaque[1 ]; adapter = (struct rte_event_timer_adapter *)(uintptr_t)opaque; @@ -605,8 +606,12 @@ swtim_callback(struct rte_timer *tim) "with immediate expiry value"); }
Quoted 8 lines
lib/librte_eventdev/rte_event_timer_adapter.c
Quoted 8 lines
lib/librte_eventdev/rte_event_timer_adapter.c +0
-1- if (unlikely(rte_atomic16_test_and_set(&sw-Quoted 1 line
in_use[lcore].v)))
lib/librte_eventdev/rte_event_timer_adapter.c +1
-1- sw->poll_lcores[sw->n_poll_lcores++] = lcore; + if (unlikely(rte_atomic16_test_and_set(&sw-Quoted 1 line
in_use[lcore].v))) {
lib/librte_eventdev/rte_event_timer_adapter.c +1
-0+ n_lcores = __atomic_fetch_add(&sw->n_poll_lcores,1,
lib/librte_eventdev/rte_event_timer_adapter.c +1
-0+ __ATOMIC_RELAXED);
Since this commit will be back ported, we should prefer to use rte_atomic APIs for this commit. Otherwise, we will have a mix of rte_atomic and C11 APIs.
My suggestion is to fix this bug using rte_atomic so that backported code will have only rte_atomic APIs. Add another commit (if required) in this series to make the bug fix use C11 APIs (this commit will not be backported).
Quoted 38 lines
Just a nit, but let's align the continued line with the opening parentheses in this location and below. With these changes:
Quoted 17 lines
lib/librte_eventdev/rte_event_timer_adapter.c +1
-0+ __atomic_store_n(&sw->poll_lcores[n_lcores],lcore,
lib/librte_eventdev/rte_event_timer_adapter.c +3
-0+ __ATOMIC_RELAXED); + } }else {EVTIM_BUF_LOG_DBG ("buffered an event timer expiry event" );@@ -1011,6 +1016,7 @@ __swtim_arm_burst(const structrte_event_timer_adapter *adapter, uint32_t lcore_id = rte_lcore_id();struct rte_timer *tim, *tims[nb_evtims];uint64_t cycles; +int n_lcores;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */ @@ -1033,8 +1039,10 @@
__swtim_arm_burst(const structrte_event_timer_adapter*adapter,lib/librte_eventdev/rte_event_timer_adapter.c
Quoted 2 lines
lib/librte_eventdev/rte_event_timer_adapter.c +0
-0if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore_id].v))) { EVTIM_LOG_DBG("Adding lcore id = %u to list of lcores topoll",
Quoted 12 lines
lib/librte_eventdev/rte_event_timer_adapter.c +4
-3lcore_id); - sw->poll_lcores[sw->n_poll_lcores] = lcore_id; - ++sw->n_poll_lcores; + n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , + __ATOMIC_RELAXED); +__atomic_store_n (&sw->poll_lcores[n_lcores], lcore_id, + __ATOMIC_RELAXED); } ret = rte_mempool_get_bulk(sw->tim_pool, (void **)tims, --2.7.4
Quoted 69 lines
-----Original Message-----
From: Honnappa Nagarahalli <[email protected]>
Sent: Friday, June 19, 2020 2:26 AM
To: Carrillo, Erik G <[email protected]>; Phil Yang <[email protected]>; [email protected]
Cc: [email protected]; Ruifeng Wang <[email protected]>; Dharmik Thakkar <[email protected]>; nd <[email protected]>; [email protected]; Honnappa Nagarahalli <[email protected]>; nd <[email protected]>
Subject: RE: [PATCH 1/3] eventdev: fix race condition on timer list counter<snip>
Quoted 56 lines
Hi Phil,
Good catch - thanks for the fix. I've commented in-line:
Quoted 10 lines
-----Original Message-----
From: Phil Yang <[email protected]>
Sent: Friday, June 12, 2020 6:20 AM
To: [email protected]; Carrillo, Erik G <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH 1/3] eventdev: fix race condition on timer list counterQuoted 4 lines
The
n_poll_lcorescounter andpoll_lcorearray are shared between lcores and the update of these variables are out of the protection of spinlock on each lcore timer list. The read-modify-write operations of the counter are not atomic, so it has the potential of race conditionbetween lcores.
Quoted 36 lines
Use c11 atomics with RELAXED ordering to prevent confliction.
Fixes cc7b73ea9e3b ("eventdev: add new software timer adapter")Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>--- lib/librte_eventdev/rte_event_timer_adapter.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)lib/librte_eventdev/rte_event_timer_adapter.c +1
-2@@ -583,6 +583,7 @@ swtim_callback(struct rte_timer *tim)uint16_t nb_evs_invalid =0 ;uint64_t opaque;int ret; +int n_lcores; opaque = evtim->impl_opaque[1 ]; adapter = (struct rte_event_timer_adapter *)(uintptr_t)opaque; @@ -605,8 +606,12 @@ swtim_callback(struct rte_timer *tim) "with immediate expiry value"); } -if (unlikely(rte_atomic16_test_and_set(&sw-Quoted 1 line
in_use[lcore].v)))
lib/librte_eventdev/rte_event_timer_adapter.c +1
-1-sw->poll_lcores[sw->n_poll_lcores++] = lcore; +if (unlikely(rte_atomic16_test_and_set(&sw-Quoted 1 line
in_use[lcore].v))) {
Quoted 9 lines
Quoted 3 lines
Quoted 3 lines
+n_lcores = __atomic_fetch_add(&sw->n_poll_lcores, 1,
+__ATOMIC_RELAXED);Since this commit will be back ported, we should prefer to use
rte_atomicAPIs for this commit. Otherwise, we will have a mix ofrte_atomicand C11 APIs.
My suggestion is to fix this bug usingrte_atomicso that backported code will have onlyrte_atomicAPIs. Add another commit (if required) in this series to make the bug fix use C11 APIs (this commit will not be backported).
Agree.
I will change this patch to the rte_atomic version in the next version.
Thanks,
Phil
Quoted 40 lines
Quoted 39 lines
Just a nit, but let's align the continued line with the opening parentheses in this location and below. With these changes:
Quoted 15 lines
+__atomic_store_n(&sw->poll_lcores[n_lcores],
lcore,lib/librte_eventdev/rte_event_timer_adapter.c +3
-0+__ATOMIC_RELAXED); +} }else {EVTIM_BUF_LOG_DBG ("buffered an event timer expiry event" );@@ -1011,6 +1016,7 @@ __swtim_arm_burst(const structrte_event_timer_adapter *adapter, uint32_t lcore_id = rte_lcore_id();struct rte_timer *tim, *tims[nb_evtims];uint64_t cycles; +int n_lcores;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG Quoted 4 lines
/* Check that the service is running. */ @@ -1033,8 +1039,10 @@
__swtim_arm_burst(const structrte_event_timer_adapter*adapter,
if (unlikely(rte_atomic16_test_and_set(&sw->in_use[lcore_id].v))) { EVTIM_LOG_DBG("Adding lcore id = %u to list of lcores topoll",
Quoted 12 lines
lcore_id);lib/librte_eventdev/rte_event_timer_adapter.c +4
-3-sw->poll_lcores[sw->n_poll_lcores] = lcore_id; -++sw->n_poll_lcores; +n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , +__ATOMIC_RELAXED); +__atomic_store_n (&sw->poll_lcores[n_lcores], lcore_id, +__ATOMIC_RELAXED); } ret = rte_mempool_get_bulk(sw->tim_pool, (void **)tims, --2.7.4
Quoted 20 lines · Phil Yang
Quoted 13 lines
-----Original Message-----
From: Honnappa Nagarahalli <[email protected]>
Sent: Friday, June 19, 2020 2:26 AM
To: Carrillo, Erik G <[email protected]>; Phil Yang <[email protected]>; [email protected]
Cc: [email protected]; Ruifeng Wang <[email protected]>; Dharmik Thakkar <[email protected]>; nd <[email protected]>; [email protected]; Honnappa Nagarahalli <[email protected]>; nd <[email protected]>
Subject: RE: [PATCH 1/3] eventdev: fix race condition on timer list counterQuoted 6 lines
Since this commit will be back ported, we should prefer to use
rte_atomicAPIs for this commit. Otherwise, we will have a mix ofrte_atomicand C11 APIs.
My suggestion is to fix this bug usingrte_atomicso that backported code will have onlyrte_atomicAPIs. Add another commit (if required) in this series to make the bug fix use C11 APIs (this commit will not be backported).
Quoted 2 lines
Agree.
I will change this patch to therte_atomicversion in the next version.
Hi Phil,
Could you send the next version? I would like to take this series for RC1(next-eventdev tree)
Quoted 4 lines
Thanks,
Phil
Quoted 35 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Wednesday, July 1, 2020 7:22 PM
To: Phil Yang <[email protected]>
Cc: Honnappa Nagarahalli <[email protected]>; Carrillo, Erik G <[email protected]>; [email protected]; [email protected]; Ruifeng Wang <[email protected]>; Dharmik Thakkar <[email protected]>; nd <[email protected]>; [email protected]
Subject: Re: [dpdk-dev] [PATCH 1/3] eventdev: fix race condition on timer list counterQuoted 9 lines · Phil Yang
Quoted 9 lines
-----Original Message-----
From: Honnappa Nagarahalli <[email protected]>
Sent: Friday, June 19, 2020 2:26 AM
To: Carrillo, Erik G <[email protected]>; Phil Yang <[email protected]>; [email protected]
Cc: [email protected]; Ruifeng Wang <[email protected]>; Dharmik Thakkar <[email protected]>; nd <[email protected]>; [email protected]; Honnappa NagarahalliQuoted 8 lines
Quoted 8 lines
nd <[email protected]>
Subject: RE: [PATCH 1/3] eventdev: fix race condition on timer list counterSince this commit will be back ported, we should prefer to use
rte_atomicAPIs for this commit. Otherwise, we will have a mix ofrte_atomicand C11 APIs.
My suggestion is to fix this bug usingrte_atomicso that backported codewill
Quoted 1 line
Quoted 1 line
have only
rte_atomicAPIs. Add another commit (if required) in this seriesto
Quoted 2 lines
Quoted 1 line
make the bug fix use C11 APIs (this commit will not be backported).
Quoted 7 lines
Quoted 2 lines
Agree.
I will change this patch to therte_atomicversion in the next version.Hi Phil,
Could you send the next version? I would like to take this series for RC1(next-eventdev tree)
Thanks, Jerin.
I will upstream the new patch series soon.
Quoted 6 lines
Quoted 6 lines
Thanks,
Phil
Quoted 69 lines
-----Original Message-----
From: Honnappa Nagarahalli <[email protected]>
Sent: Friday, June 19, 2020 2:26 AM
To: Carrillo, Erik G <[email protected]>; Phil Yang <[email protected]>; [email protected]
Cc: [email protected]; Ruifeng Wang <[email protected]>; Dharmik Thakkar <[email protected]>; nd <[email protected]>; [email protected]; Honnappa Nagarahalli <[email protected]>; nd <[email protected]>
Subject: RE: [PATCH 1/3] eventdev: fix race condition on timer list counter<snip>
Quoted 56 lines
Hi Phil,
Good catch - thanks for the fix. I've commented in-line:
Quoted 10 lines
-----Original Message-----
From: Phil Yang <[email protected]>
Sent: Friday, June 12, 2020 6:20 AM
To: [email protected]; Carrillo, Erik G <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH 1/3] eventdev: fix race condition on timer list counterQuoted 4 lines
The
n_poll_lcorescounter andpoll_lcorearray are shared between lcores and the update of these variables are out of the protection of spinlock on each lcore timer list. The read-modify-write operations of the counter are not atomic, so it has the potential of race conditionbetween lcores.
Quoted 36 lines
Use c11 atomics with RELAXED ordering to prevent confliction.
Fixes cc7b73ea9e3b ("eventdev: add new software timer adapter")Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>--- lib/librte_eventdev/rte_event_timer_adapter.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)lib/librte_eventdev/rte_event_timer_adapter.c +1
-2@@ -583,6 +583,7 @@ swtim_callback(struct rte_timer *tim)uint16_t nb_evs_invalid =0 ;uint64_t opaque;int ret; +int n_lcores; opaque = evtim->impl_opaque[1 ]; adapter = (struct rte_event_timer_adapter *)(uintptr_t)opaque; @@ -605,8 +606,12 @@ swtim_callback(struct rte_timer *tim) "with immediate expiry value"); } -if (unlikely(rte_atomic16_test_and_set(&sw-Quoted 1 line
in_use[lcore].v)))
lib/librte_eventdev/rte_event_timer_adapter.c +1
-1-sw->poll_lcores[sw->n_poll_lcores++] = lcore; +if (unlikely(rte_atomic16_test_and_set(&sw-Quoted 1 line
in_use[lcore].v))) {
Quoted 9 lines
Quoted 3 lines
Quoted 3 lines
+n_lcores = __atomic_fetch_add(&sw->n_poll_lcores, 1,
+__ATOMIC_RELAXED);Since this commit will be back ported, we should prefer to use
rte_atomicAPIs for this commit. Otherwise, we will have a mix ofrte_atomicand C11 APIs.
My suggestion is to fix this bug usingrte_atomicso that backported code will have onlyrte_atomicAPIs. Add another commit (if required) in this series to make the bug fix use C11 APIs (this commit will not be backported).
Hi Honnappa,
It doesn't have an applicable rte_atomic_XXX API to fix this issue.
The rte_atomic32_inc doesn't return the original value of the input parameter and rte_atomic32_add_return can only return the new value.
Meanwhile, the rte_timer_alt_manage & rte_timer_stop_all API not support rte_atomic type parameters. We might need to rewrite these two APIs if we want to use rte_atomic operations for n_pol_lcores and poll_lcores array.
So, a better solution could be to backport the entire c11 solution to stable releases.
Thanks,
Phil
Quoted 40 lines
Quoted 39 lines
Just a nit, but let's align the continued line with the opening parentheses in this location and below. With these changes:
Quoted 15 lines
+__atomic_store_n(&sw->poll_lcores[n_lcores],
lcore,lib/librte_eventdev/rte_event_timer_adapter.c +3
-0+__ATOMIC_RELAXED); +} }else {EVTIM_BUF_LOG_DBG ("buffered an event timer expiry event" );@@ -1011,6 +1016,7 @@ __swtim_arm_burst(const structrte_event_timer_adapter *adapter, uint32_t lcore_id = rte_lcore_id();struct rte_timer *tim, *tims[nb_evtims];uint64_t cycles; +int n_lcores;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG Quoted 4 lines
/* Check that the service is running. */ @@ -1033,8 +1039,10 @@
__swtim_arm_burst(const structrte_event_timer_adapter*adapter,
if (unlikely(rte_atomic16_test_and_set(&sw->in_use[lcore_id].v))) { EVTIM_LOG_DBG("Adding lcore id = %u to list of lcores topoll",
Quoted 12 lines
lcore_id);lib/librte_eventdev/rte_event_timer_adapter.c +4
-3-sw->poll_lcores[sw->n_poll_lcores] = lcore_id; -++sw->n_poll_lcores; +n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , +__ATOMIC_RELAXED); +__atomic_store_n (&sw->poll_lcores[n_lcores], lcore_id, +__ATOMIC_RELAXED); } ret = rte_mempool_get_bulk(sw->tim_pool, (void **)tims, --2.7.4
<snip>
Quoted 65 lines
Quoted 56 lines
Quoted 56 lines
Hi Phil,
Good catch - thanks for the fix. I've commented in-line:
Quoted 10 lines
-----Original Message-----
From: Phil Yang <[email protected]>
Sent: Friday, June 12, 2020 6:20 AM
To: [email protected]; Carrillo, Erik G <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH 1/3] eventdev: fix race condition on timer list counterQuoted 5 lines
The
n_poll_lcorescounter andpoll_lcorearray are shared between lcores and the update of these variables are out of the protection of spinlock on each lcore timer list. The read-modify-write operations of the counter are not atomic, so it has the potential of race conditionbetween lcores.
Quoted 34 lines
Use c11 atomics with RELAXED ordering to prevent confliction.
Fixes cc7b73ea9e3b ("eventdev: add new software timer adapter")Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>--- lib/librte_eventdev/rte_event_timer_adapter.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)lib/librte_eventdev/rte_event_timer_adapter.c +1
-2@@ -583,6 +583,7 @@ swtim_callback(struct rte_timer *tim)uint16_t nb_evs_invalid =0 ;uint64_t opaque;int ret; +int n_lcores; opaque = evtim->impl_opaque[1 ]; adapter = (struct rte_event_timer_adapter *)(uintptr_t)opaque; @@ -605,8 +606,12 @@ swtim_callback(struct rte_timer *tim) "with immediate expiry value"); } -if (unlikely(rte_atomic16_test_and_set(&sw-Quoted 1 line
in_use[lcore].v)))
lib/librte_eventdev/rte_event_timer_adapter.c +1
-1-sw->poll_lcores[sw->n_poll_lcores++] = lcore; +if (unlikely(rte_atomic16_test_and_set(&sw-Quoted 1 line
in_use[lcore].v))) {
Quoted 9 lines
Quoted 3 lines
Quoted 3 lines
+n_lcores = __atomic_fetch_add(&sw->n_poll_lcores, 1,
+__ATOMIC_RELAXED);Since this commit will be back ported, we should prefer to use
rte_atomicAPIs for this commit. Otherwise, we will have a mix ofrte_atomicand C11 APIs.
My suggestion is to fix this bug usingrte_atomicso that backported code will have onlyrte_atomicAPIs. Add another commit (if required) in this series to make the bug fix use C11 APIs (this commit will not be
Quoted 7 lines
backported).
Hi Honnappa,
It doesn't have an applicable
rte_atomic_XXXAPI to fix this issue. Therte_atomic32_incdoesn't return the original value of the input parameter andrte_atomic32_add_returncan only return the new value.
Ok, understood.
Quoted 7 lines
Meanwhile, the
rte_timer_alt_manage&rte_timer_stop_allAPI not supportrte_atomictype parameters. We might need to rewrite these two APIs if we want to userte_atomicoperations forn_pol_lcoresandpoll_lcoresarray.So, a better solution could be to backport the entire c11 solution to stable releases.
I am ok with the approach.
Erik, are you ok with this?
Quoted 42 lines
Thanks,
PhilQuoted 38 lines
Quoted 37 lines
Just a nit, but let's align the continued line with the opening parentheses in this location and below. With these changes:
Quoted 13 lines
+__atomic_store_n(&sw->poll_lcores[n_lcores],
lcore,lib/librte_eventdev/rte_event_timer_adapter.c +2
-0+__ATOMIC_RELAXED); +} }else {EVTIM_BUF_LOG_DBG ("buffered an event timer expiry event" );@@ -1011,6 +1016,7 @@ __swtim_arm_burst(const struct
rte_event_timer_adapter*adapter,uint32_tlcore_id=rte_lcore_id(); structrte_timer*tim, *tims[nb_evtims];uint64_tcycles;
+intn_lcores;Quoted 5 lines
#ifdef
RTE_LIBRTE_EVENTDEV_DEBUG
/* Check that the service is running. */ @@ -1033,8 +1039,10 @@
__swtim_arm_burst(const structrte_event_timer_adapter*adapter, if (unlikely(rte_atomic16_test_and_set(&sw->in_use[lcore_id].v))) { EVTIM_LOG_DBG("Adding lcore id = %u to list of lcores topoll",
Quoted 11 lines
lcore_id);lib/librte_eventdev/rte_event_timer_adapter.c +3
-3-sw->poll_lcores[sw->n_poll_lcores] = lcore_id; -++sw->n_poll_lcores; +n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , +__ATOMIC_RELAXED);__atomic_store_n (&sw->poll_lcores[n_lcores], +lcore_id, __ATOMIC_RELAXED); } ret = rte_mempool_get_bulk(sw->tim_pool, (void **)tims, --2.7.4
Quoted 93 lines
-----Original Message-----
From: Honnappa Nagarahalli <[email protected]>
Sent: Wednesday, July 1, 2020 10:56 PM
To: Phil Yang <[email protected]>; Carrillo, Erik G <[email protected]>; [email protected]
Cc: [email protected]; Ruifeng Wang <[email protected]>; Dharmik Thakkar <[email protected]>; [email protected]; nd <[email protected]>; [email protected]; Honnappa Nagarahalli <[email protected]>; nd <[email protected]>
Subject: RE: [PATCH 1/3] eventdev: fix race condition on timer list counter<snip>
Quoted 66 lines
Quoted 56 lines
Quoted 56 lines
Hi Phil,
Good catch - thanks for the fix. I've commented in-line:
Quoted 10 lines
-----Original Message-----
From: Phil Yang <[email protected]>
Sent: Friday, June 12, 2020 6:20 AM
To: [email protected]; Carrillo, Erik G <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH 1/3] eventdev: fix race condition on timer list counterQuoted 5 lines
The
n_poll_lcorescounter andpoll_lcorearray are shared between lcores and the update of these variables are out of the protection of spinlock on each lcore timer list. The read-modify-write operations of the counter are not atomic, so it has the potential of race conditionbetween lcores.
Quoted 34 lines
Use c11 atomics with RELAXED ordering to prevent confliction.
Fixes cc7b73ea9e3b ("eventdev: add new software timer adapter")Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>--- lib/librte_eventdev/rte_event_timer_adapter.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)lib/librte_eventdev/rte_event_timer_adapter.c +1
-0@@ -583,6 +583,7 @@ swtim_callback(struct rte_timer *tim)uint16_t nb_evs_invalid =0 ;uint64_t opaque;int ret; +int n_lcores; opaque = evtim->impl_opaque[1]; adapter = (struct
rte_event_timer_adapter*)(uintptr_t)opaque; @@ -605,8 +606,12 @@swtim_callback(struct rte_timer *tim)"with immediate expiry value"); }-if (unlikely(rte_atomic16_test_and_set(&sw-
Quoted 1 line
in_use[lcore].v)))
lib/librte_eventdev/rte_event_timer_adapter.c +1
-1-sw->poll_lcores[sw->n_poll_lcores++] = lcore; +if (unlikely(rte_atomic16_test_and_set(&sw-Quoted 1 line
in_use[lcore].v))) {
Quoted 10 lines
Quoted 3 lines
Quoted 3 lines
+n_lcores = __atomic_fetch_add(&sw->n_poll_lcores, 1,
+__ATOMIC_RELAXED);Since this commit will be back ported, we should prefer to use
rte_atomicAPIs for this commit. Otherwise, we will have a mix ofrte_atomicand C11 APIs.
My suggestion is to fix this bug usingrte_atomicso that backported code will have onlyrte_atomicAPIs. Add another commit (if required) in this series to make the bug fix use C11 APIs (this commit will not beQuoted 7 lines
backported).
Hi Honnappa,
It doesn't have an applicable
rte_atomic_XXXAPI to fix this issue. Therte_atomic32_incdoesn't return the original value of the input parameter andrte_atomic32_add_returncan only return the new value.Ok, understood.
Quoted 4 lines
Meanwhile, the
rte_timer_alt_manage&rte_timer_stop_allAPI not supportrte_atomictype parameters. We might need to rewrite these two APIs if we want to userte_atomicoperations forn_pol_lcoresand
Quoted 7 lines
poll_lcoresarray.Quoted 3 lines
So, a better solution could be to backport the entire c11 solution to stable releases.
I am ok with the approach.
Erik, are you ok with this?
Yes, I'm OK with that as well.
Thanks,
Erik
Quoted 44 lines
Quoted 44 lines
Thanks,
PhilQuoted 27 lines
Quoted 27 lines
Just a nit, but let's align the continued line with the opening parentheses in this location and below. With these changes:
Quoted 13 lines
+__atomic_store_n(&sw->poll_lcores[n_lcores],
lcore,lib/librte_eventdev/rte_event_timer_adapter.c +2
-0+__ATOMIC_RELAXED); +} }else {EVTIM_BUF_LOG_DBG ("buffered an event timer expiry event" );@@ -1011,6 +1016,7 @@ __swtim_arm_burst(const struct
rte_event_timer_adapter*adapter,uint32_tlcore_id=rte_lcore_id(); structrte_timer*tim, *tims[nb_evtims];uint64_tcycles;
+intn_lcores;Quoted 6 lines
#ifdef
RTE_LIBRTE_EVENTDEV_DEBUG
/* Check that the service is running. */ @@ -1033,8 +1039,10 @@
__swtim_arm_burst(const structrte_event_timer_adapter*adapter, if (unlikely(rte_atomic16_test_and_set(&sw->in_use[lcore_id].v))) { EVTIM_LOG_DBG("Adding lcore id = %u to list of lcores topoll",
Quoted 5 lines
Quoted 5 lines
lib/librte_eventdev/rte_event_timer_adapter.c __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,
Quoted 5 lines
lcore_id);lib/librte_eventdev/rte_event_timer_adapter.c +2
-2-sw->poll_lcores[sw->n_poll_lcores] = lcore_id; -++sw->n_poll_lcores; +n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , +__ATOMIC_RELAXED); __atomic_store_n(&sw-poll_lcores[n_lcores],
Quoted 7 lines
Quoted 6 lines
Quoted 6 lines
+lcore_id,
__ATOMIC_RELAXED);
}ret = rte_mempool_get_bulk(sw->tim_pool, (void **)tims,
--
2.7.4
<snip>
Quoted 84 lines
Quoted 84 lines
Quoted 68 lines
Quoted 57 lines
Quoted 57 lines
Hi Phil,
Good catch - thanks for the fix. I've commented in-line:
Quoted 10 lines
-----Original Message-----
From: Phil Yang <[email protected]>
Sent: Friday, June 12, 2020 6:20 AM
To: [email protected]; Carrillo, Erik G <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH 1/3] eventdev: fix race condition on timer list counterQuoted 5 lines
The
n_poll_lcorescounter andpoll_lcorearray are shared between lcores and the update of these variables are out of the protection of spinlock on each lcore timer list. The read-modify-write operations of the counter are not atomic, so it has the potential of race conditionbetween lcores.
Quoted 35 lines
Use c11 atomics with RELAXED ordering to prevent confliction.
Fixes cc7b73ea9e3b ("eventdev: add new software timeradapter")
Cc: [email protected]
Cc: [email protected]Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>--- lib/librte_eventdev/rte_event_timer_adapter.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)lib/librte_eventdev/rte_event_timer_adapter.c +1
-0@@ -583,6 +583,7 @@ swtim_callback(struct rte_timer *tim)uint16_t nb_evs_invalid =0 ;uint64_t opaque;int ret; +int n_lcores; opaque = evtim->impl_opaque[1]; adapter = (struct
rte_event_timer_adapter*)(uintptr_t)opaque; @@ -605,8 +606,12 @@swtim_callback(struct rte_timer *tim)"with immediate expiry value"); }-if (unlikely(rte_atomic16_test_and_set(&sw-
Quoted 1 line
in_use[lcore].v)))
lib/librte_eventdev/rte_event_timer_adapter.c +1
-1-sw->poll_lcores[sw->n_poll_lcores++] = lcore; +if (unlikely(rte_atomic16_test_and_set(&sw-Quoted 1 line
in_use[lcore].v))) {
Quoted 11 lines
Quoted 3 lines
Quoted 3 lines
+n_lcores = __atomic_fetch_add(&sw->n_poll_lcores, 1,
+__ATOMIC_RELAXED);Since this commit will be back ported, we should prefer to use
rte_atomicAPIs for this commit. Otherwise, we will have a mix ofrte_atomicand C11 APIs.
My suggestion is to fix this bug usingrte_atomicso that backported code will have onlyrte_atomicAPIs. Add another commit (if
required) in this series to make the bug fix use C11 APIs (this commit will not beQuoted 7 lines
backported).
Hi Honnappa,
It doesn't have an applicable
rte_atomic_XXXAPI to fix this issue. Therte_atomic32_incdoesn't return the original value of the input parameter andrte_atomic32_add_returncan only return the new value.Ok, understood.
Quoted 5 lines
Meanwhile, the
rte_timer_alt_manage&rte_timer_stop_allAPI not supportrte_atomictype parameters. We might need to rewrite these two APIs if we want to userte_atomicoperations forn_pol_lcoresand
poll_lcoresarray.Quoted 1 line
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
So, a better solution could be to backport the entire c11 solution to stable releases.
I am ok with the approach.
Erik, are you ok with this?Yes, I'm OK with that as well.
Thanks Erik. I think we need to add [email protected] to Cc for all the commits in the series.
Quoted 48 lines
Thanks,
ErikQuoted 44 lines
Quoted 44 lines
Thanks,
PhilQuoted 27 lines
Quoted 27 lines
Just a nit, but let's align the continued line with the opening parentheses in this location and below. With these changes:
Quoted 13 lines
+__atomic_store_n(&sw->poll_lcores[n_lcores],
lcore,lib/librte_eventdev/rte_event_timer_adapter.c +2
-0+__ATOMIC_RELAXED); +} }else {EVTIM_BUF_LOG_DBG ("buffered an event timer expiry event" );@@ -1011,6 +1016,7 @@ __swtim_arm_burst(const struct
rte_event_timer_adapter*adapter,uint32_tlcore_id=rte_lcore_id(); structrte_timer*tim, *tims[nb_evtims];uint64_tcycles;
+intn_lcores;Quoted 6 lines
#ifdef
RTE_LIBRTE_EVENTDEV_DEBUG
/* Check that the service is running. */ @@ -1033,8 +1039,10
@@ __swtim_arm_burst(const structrte_event_timer_adapter*adapter, if (unlikely(rte_atomic16_test_and_set(&sw->in_use[lcore_id].v))) { EVTIM_LOG_DBG("Adding lcore id = %u to list of lcores topoll",
Quoted 5 lines
Quoted 5 lines
lib/librte_eventdev/rte_event_timer_adapter.c __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,
Quoted 5 lines
lcore_id);lib/librte_eventdev/rte_event_timer_adapter.c +2
-2-sw->poll_lcores[sw->n_poll_lcores] = lcore_id; -++sw->n_poll_lcores; +n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , +__ATOMIC_RELAXED); __atomic_store_n(&sw-poll_lcores[n_lcores],
Quoted 7 lines
Quoted 6 lines
Quoted 6 lines
+lcore_id,
__ATOMIC_RELAXED);
}ret = rte_mempool_get_bulk(sw->tim_pool, (void **)tims,
--
2.7.4
Quoted 62 lines
-----Original Message-----
From: Carrillo, Erik G <[email protected]>
Sent: Thursday, June 18, 2020 11:18 PM
To: Phil Yang <[email protected]>; [email protected]
Cc: [email protected]; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>; Dharmik Thakkar <[email protected]>; nd <[email protected]>; [email protected]
Subject: RE: [PATCH 1/3] eventdev: fix race condition on timer list counterHi Phil,
Good catch - thanks for the fix. I've commented in-line:
Quoted 48 lines
-----Original Message-----
From: Phil Yang <[email protected]>
Sent: Friday, June 12, 2020 6:20 AM
To: [email protected]; Carrillo, Erik G <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH 1/3] eventdev: fix race condition on timer list counterThe
n_poll_lcorescounter andpoll_lcorearray are shared between lcores and the update of these variables are out of the protection of spinlock on each lcore timer list. The read-modify-write operations of the counter are not atomic, so it has the potential of race condition between lcores.Use c11 atomics with RELAXED ordering to prevent confliction.
Fixes cc7b73ea9e3b ("eventdev: add new software timer adapter")Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>--- lib/librte_eventdev/rte_event_timer_adapter.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)lib/librte_eventdev/rte_event_timer_adapter.c +2
-2@@ -583,6 +583,7 @@ swtim_callback(struct rte_timer *tim)uint16_t nb_evs_invalid =0 ;uint64_t opaque;int ret; +int n_lcores; opaque = evtim->impl_opaque[1 ]; adapter = (struct rte_event_timer_adapter *)(uintptr_t )opaque;@@ -605,8 +606,12 @@ swtim_callback(struct rte_timer *tim) "with immediate expiry value"); } -if (unlikely (rte_atomic16_test_and_set (&sw- >in_use[lcore].v))) - sw->poll_lcores[sw->n_poll_lcores++] = lcore; + if (unlikely(rte_atomic16_test_and_set(&sw-
Quoted 7 lines
lib/librte_eventdev/rte_event_timer_adapter.c swtim_callback(struct rte_timer *tim) · line 614
Quoted 4 lines
lib/librte_eventdev/rte_event_timer_adapter.c +2
-0>in_use[lcore].v))) { + n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , + __ATOMIC_RELAXED);Just a nit, but let's align the continued line with the opening parentheses in this location and below. With these changes:
Thanks Erik.
I will do it in the new version.
Quoted 36 lines
Quoted 33 lines
lib/librte_eventdev/rte_event_timer_adapter.c +1
-0+ __atomic_store_n(&sw->poll_lcores[n_lcores],lcore,
lib/librte_eventdev/rte_event_timer_adapter.c +2
-0+ __ATOMIC_RELAXED); + } }else { EVTIM_BUF_LOG_DBG("buffered an event timer expiryevent");
lib/librte_eventdev/rte_event_timer_adapter.c +1
-0@@ -1011,6 +1016,7 @@ __swtim_arm_burst(const structrte_event_timer_adapter *adapter, uint32_t lcore_id = rte_lcore_id();struct rte_timer *tim, *tims[nb_evtims];uint64_t cycles; +int n_lcores;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */ @@ -1033,8 +1039,10 @@
__swtim_arm_burst(const structrte_event_timer_adapter*adapter,lib/librte_eventdev/rte_event_timer_adapter.c +0
-0if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore_id].v))) { EVTIM_LOG_DBG("Adding lcore id = %u to list of lcores topoll",
lib/librte_eventdev/rte_event_timer_adapter.c +4
-3lcore_id); - sw->poll_lcores[sw->n_poll_lcores] = lcore_id; - ++sw->n_poll_lcores; + n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , + __ATOMIC_RELAXED); +__atomic_store_n (&sw->poll_lcores[n_lcores], lcore_id, + __ATOMIC_RELAXED); } ret = rte_mempool_get_bulk(sw->tim_pool, (void **)tims, --2.7.4
PATCH v2 1/4 eventdev: fix race condition on timer list counter
In reply to Phil YangThe n_poll_lcores counter and poll_lcore array are shared between lcores and the update of these variables are out of the protection of spinlock on each lcore timer list. The read-modify-write operations of the counter are not atomic, so it has the potential of race condition between lcores.
Use c11 atomics with RELAXED ordering to prevent confliction.
---
v2:
Align the code. (Erik)
lib/librte_eventdev/rte_event_timer_adapter.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +12 -4
@@ -583,6 +583,7 @@ swtim_callback(struct rte_timer *tim)uint16_t nb_evs_invalid =0 ;uint64_t opaque;int ret; +int n_lcores; opaque = evtim->impl_opaque[1 ]; adapter = (struct rte_event_timer_adapter *)(uintptr_t )opaque;
@@ -605,8 +606,12 @@ swtim_callback(struct rte_timer *tim) "with immediate expiry value"); } -if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore].v))) - sw->poll_lcores[sw->n_poll_lcores++] = lcore; +if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore].v))) { + n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , + __ATOMIC_RELAXED); +__atomic_store_n (&sw->poll_lcores[n_lcores], lcore, + __ATOMIC_RELAXED); + } }else {EVTIM_BUF_LOG_DBG ("buffered an event timer expiry event" );
@@ -1011,6 +1016,7 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,uint32_t lcore_id =rte_lcore_id ();struct rte_timer *tim, *tims[nb_evtims];uint64_t cycles; +int n_lcores;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */
@@ -1033,8 +1039,10 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore_id].v))) {EVTIM_LOG_DBG ("Adding lcore id = %u to list of lcores to poll" , lcore_id); - sw->poll_lcores[sw->n_poll_lcores] = lcore_id; - ++sw->n_poll_lcores; + n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , + __ATOMIC_RELAXED); +__atomic_store_n (&sw->poll_lcores[n_lcores], lcore_id, + __ATOMIC_RELAXED); } ret = rte_mempool_get_bulk(sw->tim_pool, (void **)tims,
Signature · 1 line
2.7.4
PATCH v2 2/4 eventdev: use c11 atomics for lcore timer armed flag
The in_use flag is a per core variable which is not shared between lcores in the normal case and the access of this variable should be ordered on the same core. However, if non-EAL thread pick the highest lcore to insert timers into, there is the possibility of conflicts on this flag between threads. Then the atomic CAS operation is needed.
Use the c11 atomic CAS instead of the generic rte_atomic operations to avoid the unnecessary barrier on aarch64.
---
v2:
1. Make the code comments more accurate. (Erik)
2. Define the in_use flag as an unsigned type. (Stephen)
lib/librte_eventdev/rte_event_timer_adapter.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +11 -4
@@ -554,7 +554,7 @@ struct swtim {uint32_t timer_data_id;/* Track which cores have actually armed a timer */ struct { -rte_atomic16_t v; +uint16_t v; } __rte_cache_aligned in_use[RTE_MAX_LCORE];/* Track which cores' timer lists should be polled */ unsigned int poll_lcores[RTE_MAX_LCORE ];
@@ -606,7 +606,8 @@ swtim_callback(struct rte_timer *tim) "with immediate expiry value"); } -if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore].v))) { +if (unlikely (sw->in_use[lcore].v ==0 )) { + sw->in_use[lcore].v =1 ; n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , __ATOMIC_RELAXED); __atomic_store_n(&sw->poll_lcores[n_lcores], lcore,
@@ -834,7 +835,7 @@ swtim_init(struct rte_event_timer_adapter *adapter)/* Initialize the variables that track in-use timer lists */ for (i =0 ; i <RTE_MAX_LCORE ; i++) -rte_atomic16_init (&sw->in_use[i].v); + sw->in_use[i].v =0 ;/* Initialize the timer subsystem and allocate timer data instance */ ret =rte_timer_subsystem_init ();
@@ -1017,6 +1018,8 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,struct rte_timer *tim, *tims[nb_evtims];uint64_t cycles;int n_lcores; +/* Timer list for this lcore is not in use. */ +uint16_t exp_state =0 ;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */
@@ -1035,8 +1038,12 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,/* If this is the first time we're arming an event timer on this lcore, * mark this lcore as "in use"; this will cause the service * function to process the timer list that corresponds to this lcore. +* The atomic CAS operation can prevent the race condition on in_use +* flag between multiple non-EAL threads. */ -if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore_id].v))) { +if (unlikely (__atomic_compare_exchange_n (&sw->in_use[lcore_id].v, + &exp_state,1 ,0 , + __ATOMIC_RELAXED, __ATOMIC_RELAXED))) {EVTIM_LOG_DBG ("Adding lcore id = %u to list of lcores to poll" , lcore_id); n_lcores = __atomic_fetch_add(&sw->n_poll_lcores, 1,
Signature · 1 line
2.7.4
Re: [dpdk-dev] [PATCH v2 2/4] eventdev: use c11 atomics for lcore timer armed flag
Quoted 15 lines
-----Original Message-----
From: Phil Yang <[email protected]>
Sent: Thursday, July 2, 2020 12:27 AM
To: Carrillo, Erik G <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH v2 2/4] eventdev: use c11 atomics for lcore timer armed flagThe
in_useflag is a per core variable which is not shared between lcores in the normal case and the access of this variable should be ordered on the same core. However, if non-EAL thread pick the highest lcore to insert timers into, there is the possibility of conflicts on this flag between threads. Then the atomic CAS operation is needed.
Quoted 6 lines
Use the c11 atomic CAS instead of the generic
rte_atomicoperations to avoid the unnecessary barrier on aarch64.Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>
PATCH v2 3/4 eventdev: remove redundant code
In reply to Phil YangThere is no thread will access these impl_opaque data after timer canceled. When new timer armed, it got refilled. So the cleanup process is unnecessary.
--- lib/librte_eventdev/rte_event_timer_adapter.c | 2 -- 1 file changed, 2 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +0 -2
@@ -1167,8 +1167,6 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,rte_mempool_put (sw->tim_pool, (void **)timp); evtims[i]->state =RTE_EVENT_TIMER_CANCELED ; - evtims[i]->impl_opaque[0 ] =0 ; - evtims[i]->impl_opaque[1 ] =0 ;rte_smp_wmb (); }
Signature · 1 line
2.7.4
Re: [dpdk-dev] [PATCH v2 3/4] eventdev: remove redundant code
Hi Phil,
Looks good.
Quoted 24 lines · Phil Yang
There is no thread will access these
impl_opaquedata after timer canceled. When new timer armed, it got refilled. So the cleanup process is unnecessary.Signed-off-by Phil Yang <[email protected]>---
lib/librte_eventdev/rte_event_timer_adapter.c| 2 --1 file changed, 2 deletions(-)lib/librte_eventdev/rte_event_timer_adapter.c +0
-2@@ -1167,8 +1167,6 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,rte_mempool_put (sw->tim_pool, (void **)timp); evtims[i]->state =RTE_EVENT_TIMER_CANCELED ; - evtims[i]->impl_opaque[0 ] =0 ; - evtims[i]->impl_opaque[1 ] =0 ;rte_smp_wmb (); }
Quoted 3 lines
—
2.7.4
PATCH v2 4/4 eventdev: relax smp barriers with c11 atomics
In reply to Phil YangThe implementation-specific opaque data is shared between arm and cancel operations. The state flag acts as a guard variable to make sure the update of opaque data is synchronized. This patch uses c11 atomics with explicit one way memory barrier instead of full barriers rte_smp_w/rmb() to synchronize the opaque data between timer arm and cancel threads.
---
v2:
1. Removed implementation-specific opaque data cleanup code.
2. Replaced thread fence with atomic ACQURE/RELEASE ordering on state access.
lib/librte_eventdev/rte_event_timer_adapter.c | 55 ++++++++++++++++++--------- lib/librte_eventdev/rte_event_timer_adapter.h | 2 +- 2 files changed, 38 insertions(+), 19 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +37 -18
@@ -629,7 +629,8 @@ swtim_callback(struct rte_timer *tim) sw->expired_timers[sw->n_expired_timers++] = tim; sw->stats.evtim_exp_count++; - evtim->state =RTE_EVENT_TIMER_NOT_ARMED ; +__atomic_store_n (&evtim->state,RTE_EVENT_TIMER_NOT_ARMED , + __ATOMIC_RELEASE); }if (event_buffer_batch_ready (&sw->buffer)) {
@@ -1020,6 +1021,7 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,int n_lcores;/* Timer list for this lcore is not in use. */ uint16_t exp_state =0 ; +enum rte_event_timer_state n_state;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */
@@ -1060,30 +1062,36 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter, }for (i =0 ; i < nb_evtims; i++) { -/* Don't modify the event timer state in these cases */ -if (evtims[i]->state ==RTE_EVENT_TIMER_ARMED ) { + n_state =__atomic_load_n (&evtims[i]->state, __ATOMIC_ACQUIRE); +if (n_state ==RTE_EVENT_TIMER_ARMED ) { rte_errno =EALREADY ;break ; - }else if (!(evtims[i]->state ==RTE_EVENT_TIMER_NOT_ARMED || - evtims[i]->state ==RTE_EVENT_TIMER_CANCELED )) { + }else if (!(n_state ==RTE_EVENT_TIMER_NOT_ARMED || + n_state ==RTE_EVENT_TIMER_CANCELED )) { rte_errno =EINVAL ;break ; } ret =check_timeout (evtims[i], adapter);if (unlikely (ret ==-1 )) { - evtims[i]->state =RTE_EVENT_TIMER_ERROR_TOOLATE ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR_TOOLATE , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }else if (unlikely (ret ==-2 )) { - evtims[i]->state =RTE_EVENT_TIMER_ERROR_TOOEARLY ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR_TOOEARLY , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }if (unlikely (check_destination_event_queue (evtims[i], adapter) <0 )) { - evtims[i]->state =RTE_EVENT_TIMER_ERROR ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }
@@ -1099,13 +1107,18 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter, SINGLE, lcore_id, NULL, evtims[i]);if (ret <0 ) {/* tim was in RUNNING or CONFIG state */ - evtims[i]->state =RTE_EVENT_TIMER_ERROR ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR , + __ATOMIC_RELEASE);break ; } -rte_smp_wmb ();EVTIM_LOG_DBG ("armed an event timer" ); - evtims[i]->state =RTE_EVENT_TIMER_ARMED ; +/* RELEASE ordering guarantees the adapter specific value +* changes observed before the update of state. +*/ +__atomic_store_n (&evtims[i]->state,RTE_EVENT_TIMER_ARMED , + __ATOMIC_RELEASE); }if (i < nb_evtims)
@@ -1132,6 +1145,7 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,struct rte_timer *timp;uint64_t opaque;struct swtim *sw =swtim_pmd_priv (adapter); +enum rte_event_timer_state n_state;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */
@@ -1143,16 +1157,18 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,for (i =0 ; i < nb_evtims; i++) {/* Don't modify the event timer state in these cases */ -if (evtims[i]->state ==RTE_EVENT_TIMER_CANCELED ) { +/* ACQUIRE ordering guarantees the access of implementation +* specific opague data under the correct state. +*/ + n_state =__atomic_load_n (&evtims[i]->state, __ATOMIC_ACQUIRE); +if (n_state ==RTE_EVENT_TIMER_CANCELED ) { rte_errno =EALREADY ;break ; - }else if (evtims[i]->state !=RTE_EVENT_TIMER_ARMED ) { + }else if (n_state !=RTE_EVENT_TIMER_ARMED ) { rte_errno =EINVAL ;break ; } -rte_smp_rmb (); - opaque = evtims[i]->impl_opaque[0 ]; timp = (struct rte_timer *)(uintptr_t )opaque;RTE_ASSERT (timp !=NULL );
@@ -1166,9 +1182,12 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,rte_mempool_put (sw->tim_pool, (void **)timp); - evtims[i]->state =RTE_EVENT_TIMER_CANCELED ; - -rte_smp_wmb (); +/* The RELEASE ordering here pairs with atomic ordering +* to make sure the state update data observed between +* threads. +*/ +__atomic_store_n (&evtims[i]->state,RTE_EVENT_TIMER_CANCELED , + __ATOMIC_RELEASE); }return i;
lib/librte_eventdev/rte_event_timer_adapter.h +1 -1
@@ -467,7 +467,7 @@ struct rte_event_timer {* - op: RTE_EVENT_OP_NEW * - event_type: RTE_EVENT_TYPE_TIMER */ -volatile enum rte_event_timer_state state; +enum rte_event_timer_state state;/**< State of the event timer. */ uint64_t timeout_ticks;/**< Expiry timer ticks expressed in number of *timer_ticks_ns* from
Signature · 1 line
2.7.4
Re: [dpdk-dev] [PATCH v2 4/4] eventdev: relax smp barriers with c11 atomics
Quoted 15 lines
-----Original Message-----
From: Phil Yang <[email protected]>
Sent: Thursday, July 2, 2020 12:27 AM
To: Carrillo, Erik G <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH v2 4/4] eventdev: relax smp barriers with c11 atomicsThe implementation-specific opaque data is shared between arm and cancel operations. The state flag acts as a guard variable to make sure the update of opaque data is synchronized. This patch uses c11 atomics with explicit one way memory barrier instead of full barriers rte_smp_w/rmb() to synchronize the opaque data between timer arm and cancel threads.
Quoted 3 lines
Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>
Re: [dpdk-dev] [PATCH v2 4/4] eventdev: relax smp barriers with c11 atomics
Quoted 16 lines · Erik G
Quoted 16 lines
-----Original Message-----
From: Phil Yang <[email protected]>
Sent: Thursday, July 2, 2020 12:27 AM
To: Carrillo, Erik G <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [PATCH v2 4/4] eventdev: relax smp barriers with c11 atomicsThe implementation-specific opaque data is shared between arm and cancel operations. The state flag acts as a guard variable to make sure the update of opaque data is synchronized. This patch uses c11 atomics with explicit one way memory barrier instead of full barriers rte_smp_w/rmb() to synchronize the opaque data between timer arm and cancel threads.
Quoted 4 lines
Quoted 3 lines
Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>
Series applied to dpdk-next-eventdev/master. Thanks.
Re: [dpdk-dev] [PATCH v2 4/4] eventdev: relax smp barriers with c11 atomics
In reply to Phil Yang02/07/2020 07:26, Phil Yang:
Quoted 5 lines
The implementation-specific opaque data is shared between arm and cancel operations. The state flag acts as a guard variable to make sure the update of opaque data is synchronized. This patch uses c11 atomics with explicit one way memory barrier instead of full barriers rte_smp_w/rmb() to synchronize the opaque data between timer arm and cancel threads.
I think we should write C11 (uppercase).
Please, in your explanations, try to be more specific. Naming fields may help to make things clear.
[...]
lib/librte_eventdev/rte_event_timer_adapter.h struct rte_event_timer { · line 471
Quoted 9 lines
lib/librte_eventdev/rte_event_timer_adapter.h +1
-1@@ -467,7 +467,7 @@ struct rte_event_timer {* - op: RTE_EVENT_OP_NEW * - event_type: RTE_EVENT_TYPE_TIMER */ -volatile enum rte_event_timer_state state; +enum rte_event_timer_state state;/**< State of the event timer. */
Why do you remove the volatile keyword?
It is not explained in the commit log.
This change is triggering a warning in the ABI check:
http://mails.dpdk.org/archives/test-report/2020-July/140440.html
Moving from volatile to non-volatile is probably not an issue. I expect the code generated for the volatile case to work the same in non-volatile case. Do you confirm?
In any case, we need an explanation and an ABI check exception.
Re: [dpdk-dev] [PATCH v2 4/4] eventdev: relax smp barriers with c11 atomics
Quoted 14 lines
-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Monday, July 6, 2020 6:04 PM
To: Phil Yang <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; Honnappa Nagarahalli <[email protected]>; [email protected]; Ruifeng Wang <[email protected]>; Dharmik Thakkar <[email protected]>; nd <[email protected]>; [email protected]; [email protected]; Neil Horman <[email protected]>; Dodji Seketeli <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v2 4/4] eventdev: relax smp barriers with c11 atomics02/07/2020 07:26, Phil Yang:
Quoted 8 lines
Quoted 1 line
The implementation-specific opaque data is shared between arm and
cancel
Quoted 4 lines
operations. The state flag acts as a guard variable to make sure the update of opaque data is synchronized. This patch uses c11 atomics with explicit one way memory barrier instead of full barriers rte_smp_w/rmb() to synchronize the opaque data between timer arm and cancel threads.
I think we should write C11 (uppercase).
Agreed.
I will change it in the next version.
Quoted 3 lines
Please, in your explanations, try to be more specific. Naming fields may help to make things clear.
OK. Thanks.
Quoted 14 lines
[...]
lib/librte_eventdev/rte_event_timer_adapter.h struct rte_event_timer { · line 471
Quoted 9 lines
lib/librte_eventdev/rte_event_timer_adapter.h +1
-1@@ -467,7 +467,7 @@ struct rte_event_timer {* - op: RTE_EVENT_OP_NEW * - event_type: RTE_EVENT_TYPE_TIMER */ -volatile enum rte_event_timer_state state; +enum rte_event_timer_state state;/**< State of the event timer. */ Why do you remove the volatile keyword?
It is not explained in the commit log.
By using the C11 atomic operations, it will generate the same instructions for non-volatile and volatile version. Please check the sample code here: https://gcc.godbolt.org/z/8x5rWs
Quoted 6 lines
This change is triggering a warning in the ABI check:
http://mails.dpdk.org/archives/test-report/2020-July/140440.html
Moving from volatile to non-volatile is probably not an issue. I expect the code generated for the volatile case to work the same in non-volatile case. Do you confirm?
They generate the same instructions, so either way will work. Do I need to revert it to the volatile version?
Thanks,
Phil
Quoted 3 lines
In any case, we need an explanation and an ABI check exception.
Re: [dpdk-dev] [PATCH v2 4/4] eventdev: relax smp barriers with c11 atomics
06/07/2020 17:32, Phil Yang:
Quoted 17 lines
From: Thomas Monjalon <[email protected]>
Quoted 13 lines
02/07/2020 07:26, Phil Yang:
lib/librte_eventdev/rte_event_timer_adapter.h struct rte_event_timer { · line 471
Quoted 9 lines
lib/librte_eventdev/rte_event_timer_adapter.h +1
-1@@ -467,7 +467,7 @@ struct rte_event_timer {* - op: RTE_EVENT_OP_NEW * - event_type: RTE_EVENT_TYPE_TIMER */ -volatile enum rte_event_timer_state state; +enum rte_event_timer_state state;/**< State of the event timer. */ Why do you remove the volatile keyword?
It is not explained in the commit log.By using the C11 atomic operations, it will generate the same instructions for non-volatile and volatile version. Please check the sample code here: https://gcc.godbolt.org/z/8x5rWs
Quoted 7 lines
Quoted 5 lines
This change is triggering a warning in the ABI check:
http://mails.dpdk.org/archives/test-report/2020-July/140440.html
Moving from volatile to non-volatile is probably not an issue. I expect the code generated for the volatile case to work the same in non-volatile case. Do you confirm?They generate the same instructions, so either way will work. Do I need to revert it to the volatile version?
Either you revert, or you add explanation in the commit log
+ exception in libabigail.abignore
PATCH v3 1/4 eventdev: fix race condition on timer list counter
In reply to Phil YangThe n_poll_lcores counter and poll_lcore array are shared between lcores and the update of these variables are out of the protection of spinlock on each lcore timer list. The read-modify-write operations of the counter are not atomic, so it has the potential of race condition between lcores.
Use c11 atomics with RELAXED ordering to prevent confliction.
---
v2:
Align the code. (Erik)
lib/librte_eventdev/rte_event_timer_adapter.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +12 -4
@@ -583,6 +583,7 @@ swtim_callback(struct rte_timer *tim)uint16_t nb_evs_invalid =0 ;uint64_t opaque;int ret; +int n_lcores; opaque = evtim->impl_opaque[1 ]; adapter = (struct rte_event_timer_adapter *)(uintptr_t )opaque;
@@ -605,8 +606,12 @@ swtim_callback(struct rte_timer *tim) "with immediate expiry value"); } -if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore].v))) - sw->poll_lcores[sw->n_poll_lcores++] = lcore; +if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore].v))) { + n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , + __ATOMIC_RELAXED); +__atomic_store_n (&sw->poll_lcores[n_lcores], lcore, + __ATOMIC_RELAXED); + } }else {EVTIM_BUF_LOG_DBG ("buffered an event timer expiry event" );
@@ -1011,6 +1016,7 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,uint32_t lcore_id =rte_lcore_id ();struct rte_timer *tim, *tims[nb_evtims];uint64_t cycles; +int n_lcores;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */
@@ -1033,8 +1039,10 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore_id].v))) {EVTIM_LOG_DBG ("Adding lcore id = %u to list of lcores to poll" , lcore_id); - sw->poll_lcores[sw->n_poll_lcores] = lcore_id; - ++sw->n_poll_lcores; + n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , + __ATOMIC_RELAXED); +__atomic_store_n (&sw->poll_lcores[n_lcores], lcore_id, + __ATOMIC_RELAXED); } ret = rte_mempool_get_bulk(sw->tim_pool, (void **)tims,
Signature · 1 line
2.7.4
PATCH v3 2/4 eventdev: use C11 atomics for lcore timer armed flag
The in_use flag is a per core variable which is not shared between lcores in the normal case and the access of this variable should be ordered on the same core. However, if non-EAL thread pick the highest lcore to insert timers into, there is the possibility of conflicts on this flag between threads. Then the atomic CAS operation is needed.
Use the C11 atomic CAS instead of the generic rte_atomic operations to avoid the unnecessary barrier on aarch64.
---
v2:
1. Make the code comments more accurate. (Erik)
2. Define the in_use flag as an unsigned type. (Stephen)
lib/librte_eventdev/rte_event_timer_adapter.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +11 -4
@@ -554,7 +554,7 @@ struct swtim {uint32_t timer_data_id;/* Track which cores have actually armed a timer */ struct { -rte_atomic16_t v; +uint16_t v; } __rte_cache_aligned in_use[RTE_MAX_LCORE];/* Track which cores' timer lists should be polled */ unsigned int poll_lcores[RTE_MAX_LCORE ];
@@ -606,7 +606,8 @@ swtim_callback(struct rte_timer *tim) "with immediate expiry value"); } -if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore].v))) { +if (unlikely (sw->in_use[lcore].v ==0 )) { + sw->in_use[lcore].v =1 ; n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , __ATOMIC_RELAXED); __atomic_store_n(&sw->poll_lcores[n_lcores], lcore,
@@ -834,7 +835,7 @@ swtim_init(struct rte_event_timer_adapter *adapter)/* Initialize the variables that track in-use timer lists */ for (i =0 ; i <RTE_MAX_LCORE ; i++) -rte_atomic16_init (&sw->in_use[i].v); + sw->in_use[i].v =0 ;/* Initialize the timer subsystem and allocate timer data instance */ ret =rte_timer_subsystem_init ();
@@ -1017,6 +1018,8 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,struct rte_timer *tim, *tims[nb_evtims];uint64_t cycles;int n_lcores; +/* Timer list for this lcore is not in use. */ +uint16_t exp_state =0 ;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */
@@ -1035,8 +1038,12 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,/* If this is the first time we're arming an event timer on this lcore, * mark this lcore as "in use"; this will cause the service * function to process the timer list that corresponds to this lcore. +* The atomic CAS operation can prevent the race condition on in_use +* flag between multiple non-EAL threads. */ -if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore_id].v))) { +if (unlikely (__atomic_compare_exchange_n (&sw->in_use[lcore_id].v, + &exp_state,1 ,0 , + __ATOMIC_RELAXED, __ATOMIC_RELAXED))) {EVTIM_LOG_DBG ("Adding lcore id = %u to list of lcores to poll" , lcore_id); n_lcores = __atomic_fetch_add(&sw->n_poll_lcores, 1,
Signature · 1 line
2.7.4
PATCH v3 3/4 eventdev: remove redundant code
In reply to Phil YangThere is no thread will access these impl_opaque data after timer canceled. When new timer armed, it got refilled. So the cleanup process is unnecessary.
--- lib/librte_eventdev/rte_event_timer_adapter.c | 2 -- 1 file changed, 2 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +0 -2
@@ -1167,8 +1167,6 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,rte_mempool_put (sw->tim_pool, (void **)timp); evtims[i]->state =RTE_EVENT_TIMER_CANCELED ; - evtims[i]->impl_opaque[0 ] =0 ; - evtims[i]->impl_opaque[1 ] =0 ;rte_smp_wmb (); }
Signature · 1 line
2.7.4
PATCH v3 4/4 eventdev: relax smp barriers with C11 atomics
In reply to Phil YangThe impl_opaque field is shared between the timer arm and cancel operations. Meanwhile, the state flag acts as a guard variable to make sure the update of impl_opaque is synchronized. The original code uses rte_smp barriers to achieve that. This patch uses C11 atomics with an explicit one-way memory barrier instead of full barriers rte_smp_w/rmb() to avoid the unnecessary barrier on aarch64.
Since compilers can generate the same instructions for volatile and non-volatile variable in C11 __atomics built-ins, so remain the volatile keyword in front of state enum to avoid the ABI break issue.
---
v3:
Fix ABI issue: revert to 'volatile enum rte_event_timer_state type state'.
v2:
1. Removed implementation-specific opaque data cleanup code.
2. Replaced thread fence with atomic ACQURE/RELEASE ordering on state access.
lib/librte_eventdev/rte_event_timer_adapter.c | 55 ++++++++++++++++++--------- 1 file changed, 37 insertions(+), 18 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +37 -18
@@ -629,7 +629,8 @@ swtim_callback(struct rte_timer *tim) sw->expired_timers[sw->n_expired_timers++] = tim; sw->stats.evtim_exp_count++; - evtim->state =RTE_EVENT_TIMER_NOT_ARMED ; +__atomic_store_n (&evtim->state,RTE_EVENT_TIMER_NOT_ARMED , + __ATOMIC_RELEASE); }if (event_buffer_batch_ready (&sw->buffer)) {
@@ -1020,6 +1021,7 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,int n_lcores;/* Timer list for this lcore is not in use. */ uint16_t exp_state =0 ; +enum rte_event_timer_state n_state;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */
@@ -1060,30 +1062,36 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter, }for (i =0 ; i < nb_evtims; i++) { -/* Don't modify the event timer state in these cases */ -if (evtims[i]->state ==RTE_EVENT_TIMER_ARMED ) { + n_state =__atomic_load_n (&evtims[i]->state, __ATOMIC_ACQUIRE); +if (n_state ==RTE_EVENT_TIMER_ARMED ) { rte_errno =EALREADY ;break ; - }else if (!(evtims[i]->state ==RTE_EVENT_TIMER_NOT_ARMED || - evtims[i]->state ==RTE_EVENT_TIMER_CANCELED )) { + }else if (!(n_state ==RTE_EVENT_TIMER_NOT_ARMED || + n_state ==RTE_EVENT_TIMER_CANCELED )) { rte_errno =EINVAL ;break ; } ret =check_timeout (evtims[i], adapter);if (unlikely (ret ==-1 )) { - evtims[i]->state =RTE_EVENT_TIMER_ERROR_TOOLATE ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR_TOOLATE , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }else if (unlikely (ret ==-2 )) { - evtims[i]->state =RTE_EVENT_TIMER_ERROR_TOOEARLY ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR_TOOEARLY , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }if (unlikely (check_destination_event_queue (evtims[i], adapter) <0 )) { - evtims[i]->state =RTE_EVENT_TIMER_ERROR ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }
@@ -1099,13 +1107,18 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter, SINGLE, lcore_id, NULL, evtims[i]);if (ret <0 ) {/* tim was in RUNNING or CONFIG state */ - evtims[i]->state =RTE_EVENT_TIMER_ERROR ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR , + __ATOMIC_RELEASE);break ; } -rte_smp_wmb ();EVTIM_LOG_DBG ("armed an event timer" ); - evtims[i]->state =RTE_EVENT_TIMER_ARMED ; +/* RELEASE ordering guarantees the adapter specific value +* changes observed before the update of state. +*/ +__atomic_store_n (&evtims[i]->state,RTE_EVENT_TIMER_ARMED , + __ATOMIC_RELEASE); }if (i < nb_evtims)
@@ -1132,6 +1145,7 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,struct rte_timer *timp;uint64_t opaque;struct swtim *sw =swtim_pmd_priv (adapter); +enum rte_event_timer_state n_state;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */
@@ -1143,16 +1157,18 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,for (i =0 ; i < nb_evtims; i++) {/* Don't modify the event timer state in these cases */ -if (evtims[i]->state ==RTE_EVENT_TIMER_CANCELED ) { +/* ACQUIRE ordering guarantees the access of implementation +* specific opague data under the correct state. +*/ + n_state =__atomic_load_n (&evtims[i]->state, __ATOMIC_ACQUIRE); +if (n_state ==RTE_EVENT_TIMER_CANCELED ) { rte_errno =EALREADY ;break ; - }else if (evtims[i]->state !=RTE_EVENT_TIMER_ARMED ) { + }else if (n_state !=RTE_EVENT_TIMER_ARMED ) { rte_errno =EINVAL ;break ; } -rte_smp_rmb (); - opaque = evtims[i]->impl_opaque[0 ]; timp = (struct rte_timer *)(uintptr_t )opaque;RTE_ASSERT (timp !=NULL );
@@ -1166,9 +1182,12 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,rte_mempool_put (sw->tim_pool, (void **)timp); - evtims[i]->state =RTE_EVENT_TIMER_CANCELED ; - -rte_smp_wmb (); +/* The RELEASE ordering here pairs with atomic ordering +* to make sure the state update data observed between +* threads. +*/ +__atomic_store_n (&evtims[i]->state,RTE_EVENT_TIMER_CANCELED , + __ATOMIC_RELEASE); }return i;
Signature · 1 line
2.7.4
Re: [dpdk-dev] [PATCH v3 4/4] eventdev: relax smp barriers with C11 atomics
Quoted 8 lines · Phil Yang
The
impl_opaquefield is shared between the timer arm and cancel operations. Meanwhile, the state flag acts as a guard variable to make sure the update ofimpl_opaqueis synchronized. The original code usesrte_smpbarriers to achieve that. This patch uses C11 atomics with an explicit one-way memory barrier instead of full barriers rte_smp_w/rmb() to avoid the unnecessary barrier on aarch64.
Quoted 8 lines
Since compilers can generate the same instructions for volatile and non-volatile variable in C11
__atomicsbuilt-ins, so remain the volatile keyword in front of state enum to avoid the ABI break issue.Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>
Could you fix the following:
WARNING:TYPO_SPELLING: 'opague' may be misspelled - perhaps 'opaque'? #184: FILE: lib/librte_eventdev/rte_event_timer_adapter.c:1161:
+ * specific opague data under the correct state.
total: 0 errors, 1 warnings, 124 lines checked
Quoted 149 lines
---
v3:
Fix ABI issue: revert to 'volatile enumrte_event_timer_statetype state'.v2:
1. Removed implementation-specific opaque data cleanup code.
2. Replaced thread fence with atomic ACQURE/RELEASE ordering on state access.lib/librte_eventdev/rte_event_timer_adapter.c | 55 ++++++++++++++++++--------- 1 file changed, 37 insertions(+), 18 deletions(-)lib/librte_eventdev/rte_event_timer_adapter.c +37
-19@@ -629,7 +629,8 @@ swtim_callback(struct rte_timer *tim) sw->expired_timers[sw->n_expired_timers++] = tim; sw->stats.evtim_exp_count++; - evtim->state =RTE_EVENT_TIMER_NOT_ARMED ; +__atomic_store_n (&evtim->state,RTE_EVENT_TIMER_NOT_ARMED , + __ATOMIC_RELEASE); }if (event_buffer_batch_ready (&sw->buffer)) {@@ -1020,6 +1021,7 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,int n_lcores;/* Timer list for this lcore is not in use. */ uint16_t exp_state =0 ; +enum rte_event_timer_state n_state;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */ @@ -1060,30 +1062,36 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter, }for (i =0 ; i < nb_evtims; i++) { -/* Don't modify the event timer state in these cases */ -if (evtims[i]->state ==RTE_EVENT_TIMER_ARMED ) { + n_state =__atomic_load_n (&evtims[i]->state, __ATOMIC_ACQUIRE); +if (n_state ==RTE_EVENT_TIMER_ARMED ) { rte_errno =EALREADY ;break ; - }else if (!(evtims[i]->state ==RTE_EVENT_TIMER_NOT_ARMED || - evtims[i]->state ==RTE_EVENT_TIMER_CANCELED )) { + }else if (!(n_state ==RTE_EVENT_TIMER_NOT_ARMED || + n_state ==RTE_EVENT_TIMER_CANCELED )) { rte_errno =EINVAL ;break ; } ret =check_timeout (evtims[i], adapter);if (unlikely (ret ==-1 )) { - evtims[i]->state =RTE_EVENT_TIMER_ERROR_TOOLATE ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR_TOOLATE , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }else if (unlikely (ret ==-2 )) { - evtims[i]->state =RTE_EVENT_TIMER_ERROR_TOOEARLY ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR_TOOEARLY , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }if (unlikely (check_destination_event_queue (evtims[i], adapter) <0 )) { - evtims[i]->state =RTE_EVENT_TIMER_ERROR ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }@@ -1099,13 +1107,18 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter, SINGLE, lcore_id, NULL, evtims[i]);if (ret <0 ) {/* tim was in RUNNING or CONFIG state */ - evtims[i]->state =RTE_EVENT_TIMER_ERROR ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR , + __ATOMIC_RELEASE);break ; } -rte_smp_wmb ();EVTIM_LOG_DBG ("armed an event timer" ); - evtims[i]->state =RTE_EVENT_TIMER_ARMED ; +/* RELEASE ordering guarantees the adapter specific value +* changes observed before the update of state. +*/ +__atomic_store_n (&evtims[i]->state,RTE_EVENT_TIMER_ARMED , + __ATOMIC_RELEASE); }if (i < nb_evtims)@@ -1132,6 +1145,7 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,struct rte_timer *timp;uint64_t opaque;struct swtim *sw =swtim_pmd_priv (adapter); +enum rte_event_timer_state n_state;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */ @@ -1143,16 +1157,18 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,for (i =0 ; i < nb_evtims; i++) {/* Don't modify the event timer state in these cases */ -if (evtims[i]->state ==RTE_EVENT_TIMER_CANCELED ) { +/* ACQUIRE ordering guarantees the access of implementation +* specific opague data under the correct state. +*/ + n_state =__atomic_load_n (&evtims[i]->state, __ATOMIC_ACQUIRE); +if (n_state ==RTE_EVENT_TIMER_CANCELED ) { rte_errno =EALREADY ;break ; - }else if (evtims[i]->state !=RTE_EVENT_TIMER_ARMED ) { + }else if (n_state !=RTE_EVENT_TIMER_ARMED ) { rte_errno =EINVAL ;break ; } -rte_smp_rmb (); - opaque = evtims[i]->impl_opaque[0 ]; timp = (struct rte_timer *)(uintptr_t )opaque;RTE_ASSERT (timp !=NULL );@@ -1166,9 +1182,12 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,rte_mempool_put (sw->tim_pool, (void **)timp); - evtims[i]->state =RTE_EVENT_TIMER_CANCELED ; - -rte_smp_wmb (); +/* The RELEASE ordering here pairs with atomic ordering +* to make sure the state update data observed between +* threads. +*/ +__atomic_store_n (&evtims[i]->state,RTE_EVENT_TIMER_CANCELED , + __ATOMIC_RELEASE); }return i; --2.7.4
Re: [dpdk-dev] [PATCH v3 4/4] eventdev: relax smp barriers with C11 atomics
Quoted 33 lines
-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Tuesday, July 7, 2020 10:30 PM
To: Phil Yang <[email protected]>
Cc: [email protected]; Erik Gabriel Carrillo <[email protected]>; dpdk-dev <[email protected]>; [email protected]; Honnappa Nagarahalli <[email protected]>; David Christensen <[email protected]>; Ruifeng Wang <[email protected]>; Dharmik Thakkar <[email protected]>; nd <[email protected]>; David Marchand <[email protected]>; Ray Kinsella <[email protected]>; Neil Horman <[email protected]>; [email protected]
Subject: Re: [dpdk-dev] [PATCH v3 4/4] eventdev: relax smp barriers with C11 atomicsQuoted 8 lines · Phil Yang
The
impl_opaquefield is shared between the timer arm and cancel operations. Meanwhile, the state flag acts as a guard variable to make sure the update ofimpl_opaqueis synchronized. The original code usesrte_smpbarriers to achieve that. This patch uses C11 atomics with an explicit one-way memory barrier instead of full barriers rte_smp_w/rmb() to avoid the unnecessary barrier on aarch64.Quoted 8 lines
Since compilers can generate the same instructions for volatile and non-volatile variable in C11
__atomicsbuilt-ins, so remain the volatile keyword in front of state enum to avoid the ABI break issue.Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>
Quoted 5 lines
Could you fix the following:
WARNING:TYPO_SPELLING: 'opague' may be misspelled - perhaps 'opaque'? #184: FILE:
lib/librte_eventdev/rte_event_timer_adapter.c:1161:
+ * specific opague data under the correct state.
Done.
Thanks,
Phil
PATCH v4 1/4 eventdev: fix race condition on timer list counter
In reply to Phil YangThe n_poll_lcores counter and poll_lcore array are shared between lcores and the update of these variables are out of the protection of spinlock on each lcore timer list. The read-modify-write operations of the counter are not atomic, so it has the potential of race condition between lcores.
Use c11 atomics with RELAXED ordering to prevent confliction.
---
v2:
Align the code. (Erik)
lib/librte_eventdev/rte_event_timer_adapter.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +12 -4
@@ -583,6 +583,7 @@ swtim_callback(struct rte_timer *tim)uint16_t nb_evs_invalid =0 ;uint64_t opaque;int ret; +int n_lcores; opaque = evtim->impl_opaque[1 ]; adapter = (struct rte_event_timer_adapter *)(uintptr_t )opaque;
@@ -605,8 +606,12 @@ swtim_callback(struct rte_timer *tim) "with immediate expiry value"); } -if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore].v))) - sw->poll_lcores[sw->n_poll_lcores++] = lcore; +if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore].v))) { + n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , + __ATOMIC_RELAXED); +__atomic_store_n (&sw->poll_lcores[n_lcores], lcore, + __ATOMIC_RELAXED); + } }else {EVTIM_BUF_LOG_DBG ("buffered an event timer expiry event" );
@@ -1011,6 +1016,7 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,uint32_t lcore_id =rte_lcore_id ();struct rte_timer *tim, *tims[nb_evtims];uint64_t cycles; +int n_lcores;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */
@@ -1033,8 +1039,10 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore_id].v))) {EVTIM_LOG_DBG ("Adding lcore id = %u to list of lcores to poll" , lcore_id); - sw->poll_lcores[sw->n_poll_lcores] = lcore_id; - ++sw->n_poll_lcores; + n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , + __ATOMIC_RELAXED); +__atomic_store_n (&sw->poll_lcores[n_lcores], lcore_id, + __ATOMIC_RELAXED); } ret = rte_mempool_get_bulk(sw->tim_pool, (void **)tims,
Signature · 1 line
2.7.4
PATCH v4 2/4 eventdev: use C11 atomics for lcore timer armed flag
The in_use flag is a per core variable which is not shared between lcores in the normal case and the access of this variable should be ordered on the same core. However, if non-EAL thread pick the highest lcore to insert timers into, there is the possibility of conflicts on this flag between threads. Then the atomic compare-and-swap operation is needed.
Use the C11 atomics instead of the generic rte_atomic operations to avoid the unnecessary barrier on aarch64.
---
v4:
1.Change 'CAS' to 'compare-and-swap' to pass the coding style check. 2.Cc to stable release. (Honnappa)
v2:
1. Make the code comments more accurate. (Erik)
2. Define the in_use flag as an unsigned type. (Stephen)
lib/librte_eventdev/rte_event_timer_adapter.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +11 -4
@@ -554,7 +554,7 @@ struct swtim {uint32_t timer_data_id;/* Track which cores have actually armed a timer */ struct { -rte_atomic16_t v; +uint16_t v; } __rte_cache_aligned in_use[RTE_MAX_LCORE];/* Track which cores' timer lists should be polled */ unsigned int poll_lcores[RTE_MAX_LCORE ];
@@ -606,7 +606,8 @@ swtim_callback(struct rte_timer *tim) "with immediate expiry value"); } -if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore].v))) { +if (unlikely (sw->in_use[lcore].v ==0 )) { + sw->in_use[lcore].v =1 ; n_lcores =__atomic_fetch_add (&sw->n_poll_lcores,1 , __ATOMIC_RELAXED); __atomic_store_n(&sw->poll_lcores[n_lcores], lcore,
@@ -834,7 +835,7 @@ swtim_init(struct rte_event_timer_adapter *adapter)/* Initialize the variables that track in-use timer lists */ for (i =0 ; i <RTE_MAX_LCORE ; i++) -rte_atomic16_init (&sw->in_use[i].v); + sw->in_use[i].v =0 ;/* Initialize the timer subsystem and allocate timer data instance */ ret =rte_timer_subsystem_init ();
@@ -1017,6 +1018,8 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,struct rte_timer *tim, *tims[nb_evtims];uint64_t cycles;int n_lcores; +/* Timer list for this lcore is not in use. */ +uint16_t exp_state =0 ;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */
@@ -1035,8 +1038,12 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,/* If this is the first time we're arming an event timer on this lcore, * mark this lcore as "in use"; this will cause the service * function to process the timer list that corresponds to this lcore. +* The atomic compare-and-swap operation can prevent the race condition +* on in_use flag between multiple non-EAL threads. */ -if (unlikely (rte_atomic16_test_and_set (&sw->in_use[lcore_id].v))) { +if (unlikely (__atomic_compare_exchange_n (&sw->in_use[lcore_id].v, + &exp_state,1 ,0 , + __ATOMIC_RELAXED, __ATOMIC_RELAXED))) {EVTIM_LOG_DBG ("Adding lcore id = %u to list of lcores to poll" , lcore_id); n_lcores = __atomic_fetch_add(&sw->n_poll_lcores, 1,
Signature · 1 line
2.7.4
PATCH v4 3/4 eventdev: remove redundant code
In reply to Phil YangThere is no thread will access these impl_opaque data after timer canceled. When new timer armed, it got refilled. So the cleanup process is unnecessary.
---
v4:
Cc to stable release. (Honnappa)
lib/librte_eventdev/rte_event_timer_adapter.c | 2 -- 1 file changed, 2 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +0 -2
@@ -1167,8 +1167,6 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,rte_mempool_put (sw->tim_pool, (void **)timp); evtims[i]->state =RTE_EVENT_TIMER_CANCELED ; - evtims[i]->impl_opaque[0 ] =0 ; - evtims[i]->impl_opaque[1 ] =0 ;rte_smp_wmb (); }
Signature · 1 line
2.7.4
PATCH v4 4/4 eventdev: relax smp barriers with C11 atomics
In reply to Phil YangThe impl_opaque field is shared between the timer arm and cancel operations. Meanwhile, the state flag acts as a guard variable to make sure the update of impl_opaque is synchronized. The original code uses rte_smp barriers to achieve that. This patch uses C11 atomics with an explicit one-way memory barrier instead of full barriers rte_smp_w/rmb() to avoid the unnecessary barrier on aarch64.
Since compilers can generate the same instructions for volatile and non-volatile variable in C11 __atomics built-ins, so remain the volatile keyword in front of state enum to avoid the ABI break issue.
---
v4:
1. Fix typo.
2. Cc to stable release. (Honnappa)
v3:
Fix ABI issue: revert to 'volatile enum rte_event_timer_state type state'.
v2:
1. Removed implementation-specific opaque data cleanup code.
2. Replaced thread fence with atomic ACQURE/RELEASE ordering on state access.
lib/librte_eventdev/rte_event_timer_adapter.c | 55 ++++++++++++++++++--------- 1 file changed, 37 insertions(+), 18 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +37 -18
@@ -629,7 +629,8 @@ swtim_callback(struct rte_timer *tim) sw->expired_timers[sw->n_expired_timers++] = tim; sw->stats.evtim_exp_count++; - evtim->state =RTE_EVENT_TIMER_NOT_ARMED ; +__atomic_store_n (&evtim->state,RTE_EVENT_TIMER_NOT_ARMED , + __ATOMIC_RELEASE); }if (event_buffer_batch_ready (&sw->buffer)) {
@@ -1020,6 +1021,7 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,int n_lcores;/* Timer list for this lcore is not in use. */ uint16_t exp_state =0 ; +enum rte_event_timer_state n_state;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */
@@ -1060,30 +1062,36 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter, }for (i =0 ; i < nb_evtims; i++) { -/* Don't modify the event timer state in these cases */ -if (evtims[i]->state ==RTE_EVENT_TIMER_ARMED ) { + n_state =__atomic_load_n (&evtims[i]->state, __ATOMIC_ACQUIRE); +if (n_state ==RTE_EVENT_TIMER_ARMED ) { rte_errno =EALREADY ;break ; - }else if (!(evtims[i]->state ==RTE_EVENT_TIMER_NOT_ARMED || - evtims[i]->state ==RTE_EVENT_TIMER_CANCELED )) { + }else if (!(n_state ==RTE_EVENT_TIMER_NOT_ARMED || + n_state ==RTE_EVENT_TIMER_CANCELED )) { rte_errno =EINVAL ;break ; } ret =check_timeout (evtims[i], adapter);if (unlikely (ret ==-1 )) { - evtims[i]->state =RTE_EVENT_TIMER_ERROR_TOOLATE ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR_TOOLATE , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }else if (unlikely (ret ==-2 )) { - evtims[i]->state =RTE_EVENT_TIMER_ERROR_TOOEARLY ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR_TOOEARLY , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }if (unlikely (check_destination_event_queue (evtims[i], adapter) <0 )) { - evtims[i]->state =RTE_EVENT_TIMER_ERROR ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR , + __ATOMIC_RELAXED); rte_errno =EINVAL ;break ; }
@@ -1099,13 +1107,18 @@ __swtim_arm_burst(const struct rte_event_timer_adapter *adapter, SINGLE, lcore_id, NULL, evtims[i]);if (ret <0 ) {/* tim was in RUNNING or CONFIG state */ - evtims[i]->state =RTE_EVENT_TIMER_ERROR ; +__atomic_store_n (&evtims[i]->state, +RTE_EVENT_TIMER_ERROR , + __ATOMIC_RELEASE);break ; } -rte_smp_wmb ();EVTIM_LOG_DBG ("armed an event timer" ); - evtims[i]->state =RTE_EVENT_TIMER_ARMED ; +/* RELEASE ordering guarantees the adapter specific value +* changes observed before the update of state. +*/ +__atomic_store_n (&evtims[i]->state,RTE_EVENT_TIMER_ARMED , + __ATOMIC_RELEASE); }if (i < nb_evtims)
@@ -1132,6 +1145,7 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,struct rte_timer *timp;uint64_t opaque;struct swtim *sw =swtim_pmd_priv (adapter); +enum rte_event_timer_state n_state;#ifdef RTE_LIBRTE_EVENTDEV_DEBUG /* Check that the service is running. */
@@ -1143,16 +1157,18 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,for (i =0 ; i < nb_evtims; i++) {/* Don't modify the event timer state in these cases */ -if (evtims[i]->state ==RTE_EVENT_TIMER_CANCELED ) { +/* ACQUIRE ordering guarantees the access of implementation +* specific opaque data under the correct state. +*/ + n_state =__atomic_load_n (&evtims[i]->state, __ATOMIC_ACQUIRE); +if (n_state ==RTE_EVENT_TIMER_CANCELED ) { rte_errno =EALREADY ;break ; - }else if (evtims[i]->state !=RTE_EVENT_TIMER_ARMED ) { + }else if (n_state !=RTE_EVENT_TIMER_ARMED ) { rte_errno =EINVAL ;break ; } -rte_smp_rmb (); - opaque = evtims[i]->impl_opaque[0 ]; timp = (struct rte_timer *)(uintptr_t )opaque;RTE_ASSERT (timp !=NULL );
@@ -1166,9 +1182,12 @@ swtim_cancel_burst(const struct rte_event_timer_adapter *adapter,rte_mempool_put (sw->tim_pool, (void **)timp); - evtims[i]->state =RTE_EVENT_TIMER_CANCELED ; - -rte_smp_wmb (); +/* The RELEASE ordering here pairs with atomic ordering +* to make sure the state update data observed between +* threads. +*/ +__atomic_store_n (&evtims[i]->state,RTE_EVENT_TIMER_CANCELED , + __ATOMIC_RELEASE); }return i;
Signature · 1 line
2.7.4
Quoted 8 lines · Phil Yang
The
n_poll_lcorescounter andpoll_lcorearray are shared between lcores and the update of these variables are out of the protection of spinlock on each lcore timer list. The read-modify-write operations of the counter are not atomic, so it has the potential of race condition between lcores.Use c11 atomics with RELAXED ordering to prevent confliction.
Quoted 8 lines
Fixes cc7b73ea9e3b ("eventdev: add new software timer adapter")Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>
Hi Thomas,
The latest version does not have ABI breakage issue.
I have added the ABI verifier in my local patch verification setup.
Series applied to dpdk-next-eventdev/master.
Please pull this series from dpdk-next-eventdev/master. Thanks.
I am marking this patch series as "Awaiting Upstream" in patchwork status to reflect the actual status.
08/07/2020 15:30, Jerin Jacob:
Quoted 22 lines · Phil Yang
Quoted 8 lines
The
n_poll_lcorescounter andpoll_lcorearray are shared between lcores and the update of these variables are out of the protection of spinlock on each lcore timer list. The read-modify-write operations of the counter are not atomic, so it has the potential of race condition between lcores.Use c11 atomics with RELAXED ordering to prevent confliction.
Quoted 8 lines
Fixes cc7b73ea9e3b ("eventdev: add new software timer adapter")Signed-off-by Phil Yang <[email protected]>Reviewed-by Dharmik Thakkar <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Hi Thomas,
The latest version does not have ABI breakage issue.
Quoted 8 lines
I have added the ABI verifier in my local patch verification setup.
Series applied to dpdk-next-eventdev/master.
Please pull this series from dpdk-next-eventdev/master. Thanks.
I am marking this patch series as "Awaiting Upstream" in patchwork status to reflect the actual status.
OK, pulled and marked as Accepted in patchwork.