vdpa/mlx5: fix polling threads scheduling
Subsystems (2)
PATCH 1/1 vdpa/mlx5: fix polling threads scheduling
When the event mode is with 0 fixed delay, the polling-thread will never give-up CPU.
So, when multi-polling-threads are active, the context-switch between them will be managed by the system which may affect latency according to the time-out decided by the system.
In order to fix multi-devices polling thread scheduling, this patch forces rescheduling for each CQ poll iteration.
Move the polling thread to SCHED_RR mode with maximum priority to complete the fairness.
--- drivers/vdpa/mlx5/mlx5_vdpa_event.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
drivers/vdpa/mlx5/mlx5_vdpa_event.c +16 -0
@@ -232,6 +232,9 @@ }if (priv->timer_delay_us)usleep (priv->timer_delay_us); +else +/* Give-up CPU to improve polling threads scheduling. */ +pthread_yield (); } static void *
@@ -500,6 +503,9 @@rte_cpuset_t cpuset;pthread_attr_t attr;char name[16 ]; +const struct sched_param sp = { + .sched_priority =sched_get_priority_max (SCHED_RR ), + };if (!priv->eventc)/* All virtqs are in poll mode. */
@@ -520,6 +526,16 @@DRV_LOG (ERR ,"Failed to set thread affinity." );return -1 ; } + ret =pthread_attr_setschedpolicy (&attr,SCHED_RR ); +if (ret) { +DRV_LOG (ERR ,"Failed to set thread sched policy = RR." ); +return -1 ; + } + ret =pthread_attr_setschedparam (&attr, &sp); +if (ret) { +DRV_LOG (ERR ,"Failed to set thread priority." ); +return -1 ; + } ret =pthread_create (&priv->timer_tid, &attr, mlx5_vdpa_poll_handle, (void *)priv);if (ret) {
Signature · 1 line
1.8.3.1
Quoted 52 lines · Matan Azrad
When the event mode is with 0 fixed delay, the polling-thread will never give-up CPU.
So, when multi-polling-threads are active, the context-switch between them will be managed by the system which may affect latency according to the time-out decided by the system.
In order to fix multi-devices polling thread scheduling, this patch forces rescheduling for each CQ poll iteration.
Move the polling thread to
SCHED_RRmode with maximum priority to complete the fairness.Fixes 6956a48cabbb ("vdpa/mlx5: set polling mode default delay to zero")Signed-off-by Matan Azrad <[email protected]>--- drivers/vdpa/mlx5/mlx5_vdpa_event.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)drivers/vdpa/mlx5/mlx5_vdpa_event.c +9
-0@@ -232,6 +232,9 @@ }if (priv->timer_delay_us)usleep (priv->timer_delay_us); +else +/* Give-up CPU to improve polling threads scheduling. */ +pthread_yield (); } static void *@@ -500,6 +503,9 @@rte_cpuset_t cpuset;pthread_attr_t attr;char name[16 ]; +const struct sched_param sp = { + .sched_priority =sched_get_priority_max (SCHED_RR ), + };if (!priv->eventc)/* All virtqs are in poll mode. */ @@ -520,6 +526,16 @@DRV_LOG (ERR ,"Failed to set thread affinity." );return -1 ; } + ret =pthread_attr_setschedpolicy (&attr,SCHED_RR ); +if (ret) { +DRV_LOG (ERR ,"Failed to set thread sched policy = RR." );
drivers/vdpa/mlx5/mlx5_vdpa_event.c line 541
Quoted 10 lines
drivers/vdpa/mlx5/mlx5_vdpa_event.c +7
-0+return -1 ; + } + ret =pthread_attr_setschedparam (&attr, &sp); +if (ret) { +DRV_LOG (ERR ,"Failed to set thread priority." ); +return -1 ; + } ret =pthread_create (&priv->timer_tid, &attr, mlx5_vdpa_poll_handle, (void *)priv);if (ret) {
Xueming(Steven) Li
Acked-by Xueming LiQuoted 67 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of Maxime Coquelin
Sent: Monday, February 8, 2021 7:17 PM
To: Matan Azrad <[email protected]>; [email protected]
Subject: Re: [dpdk-dev] [PATCH] vdpa/mlx5: fix polling threads schedulingQuoted 58 lines · Matan Azrad
When the event mode is with 0 fixed delay, the polling-thread will never give-up CPU.
So, when multi-polling-threads are active, the context-switch between them will be managed by the system which may affect latency according to the time-out decided by the system.
In order to fix multi-devices polling thread scheduling, this patch forces rescheduling for each CQ poll iteration.
Move the polling thread to
SCHED_RRmode with maximum priority to complete the fairness.Fixes 6956a48cabbb ("vdpa/mlx5: set polling mode default delay tozero")
Signed-off-by Matan Azrad <[email protected]>--- drivers/vdpa/mlx5/mlx5_vdpa_event.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)drivers/vdpa/mlx5/mlx5_vdpa_event.c +14
-0@@ -232,6 +232,9 @@ }if (priv->timer_delay_us)usleep (priv->timer_delay_us); +else +/* Give-up CPU to improve polling threads scheduling. */ +pthread_yield (); } static void *@@ -500,6 +503,9 @@rte_cpuset_t cpuset;pthread_attr_t attr;char name[16 ]; +const struct sched_param sp = { + .sched_priority =sched_get_priority_max (SCHED_RR ), + };if (!priv->eventc)/* All virtqs are in poll mode. */ @@ -520,6 +526,16 @@DRV_LOG (ERR ,"Failed to set thread affinity." );return -1 ; } + ret =pthread_attr_setschedpolicy (&attr,SCHED_RR ); +if (ret) { +DRV_LOG (ERR ,"Failed to set thread sched policy = RR." ); +return -1 ; + } + ret =pthread_attr_setschedparam (&attr, &sp); +if (ret) { +DRV_LOG (ERR ,"Failed to set thread priority." );
Quoted 8 lines
drivers/vdpa/mlx5/mlx5_vdpa_event.c line 541
Quoted 6 lines
drivers/vdpa/mlx5/mlx5_vdpa_event.c +2
-0+return -1 ; + } ret =pthread_create (&priv->timer_tid, &attr, mlx5_vdpa_poll_handle, (void *)priv);if (ret) {Reviewed-by Maxime Coquelin <[email protected]>
Thomas Monjalon
AppliedQuoted 13 lines
Quoted 13 lines
Quoted 13 lines
When the event mode is with 0 fixed delay, the polling-thread will never give-up CPU.
So, when multi-polling-threads are active, the context-switch between them will be managed by the system which may affect latency according to the time-out decided by the system.
In order to fix multi-devices polling thread scheduling, this patch forces rescheduling for each CQ poll iteration.
Move the polling thread to
SCHED_RRmode with maximum priority to complete the fairness.
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Fixes 6956a48cabbb ("vdpa/mlx5: set polling mode default delay tozero")
Signed-off-by Matan Azrad <[email protected]>Reviewed-by Maxime Coquelin <[email protected]>Acked-by Xueming Li <[email protected]>
converted to nvidia.com
Applied, thanks