Skip to contentSkip to conversation

vdpa/mlx5: fix configuration mutex cleanup

PATCH Applied Matan Azrad 3 participants 17 messages Jan 6, 2021 – Jan 27, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 vdpa/mlx5: fix configuration mutex cleanup

When the vDPA device is closed, the driver polling thread is canceled. The polling thread locks the configuration mutex while it polls the CQs.

When the cancellation happens, it may terminate the thread inside the critical section what remains the configuration mutex locked.

After device close, the driver may be configured again, in this case, for example, when the first queue state is updated, the driver tries to lock the mutex again and deadlock appears.

Initialize the mutex after the polling thread cancellation.

Fixes 99abbd62c272 ("vdpa/mlx5: fix queue update synchronization")
---
 drivers/vdpa/mlx5/mlx5_vdpa.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/vdpa/mlx5/mlx5_vdpa.c +2 -0
index b64f364..0b2f1ab 100644
@@ -295,6 +295,8 @@
 	}
 	priv->configured = 0;
 	priv->vid = 0;
+	/* The mutex may stay locked after event thread cancel - initiate it. */
+	pthread_mutex_init(&priv->vq_config_lock, NULL);
 	DRV_LOG(INFO, "vDPA device %d was closed.", vid);
 	return ret;
 }
Signature · 1 line
1.8.3.1
#1 lore raw
Quoted 27 lines · Matan Azrad

When the vDPA device is closed, the driver polling thread is canceled. The polling thread locks the configuration mutex while it polls the CQs.

When the cancellation happens, it may terminate the thread inside the critical section what remains the configuration mutex locked.

After device close, the driver may be configured again, in this case, for example, when the first queue state is updated, the driver tries to lock the mutex again and deadlock appears.

Initialize the mutex after the polling thread cancellation.

Fixes 99abbd62c272 ("vdpa/mlx5: fix queue update synchronization")
---
 drivers/vdpa/mlx5/mlx5_vdpa.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/vdpa/mlx5/mlx5_vdpa.c +0 -0
index b64f364..0b2f1ab 100644

drivers/vdpa/mlx5/mlx5_vdpa.c line 302

Quoted 8 lines
drivers/vdpa/mlx5/mlx5_vdpa.c +2 -0
@@ -295,6 +295,8 @@
 	}
 	priv->configured = 0;
 	priv->vid = 0;
+	/* The mutex may stay locked after event thread cancel - initiate it. */
+	pthread_mutex_init(&priv->vq_config_lock, NULL);
 	DRV_LOG(INFO, "vDPA device %d was closed.", vid);
 	return ret;
 }

I wonder if it would be possible and cleaner to disable cancellation on the thread while the mutex is held?

Regards,
Maxime

#2 lore raw
Quoted 31 lines · Maxime Coquelin
Quoted 30 lines · Matan Azrad

When the vDPA device is closed, the driver polling thread is canceled. The polling thread locks the configuration mutex while it polls the CQs.

When the cancellation happens, it may terminate the thread inside the critical section what remains the configuration mutex locked.

After device close, the driver may be configured again, in this case, for example, when the first queue state is updated, the driver tries to lock the mutex again and deadlock appears.

Initialize the mutex after the polling thread cancellation.

Fixes 99abbd62c272 ("vdpa/mlx5: fix queue update synchronization")
---
 drivers/vdpa/mlx5/mlx5_vdpa.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/vdpa/mlx5/mlx5_vdpa.c +1 -0
index b64f364..0b2f1ab 100644
@@ -295,6 +295,8 @@
      }
      priv->configured = 0;
      priv->vid = 0;
+     /* The mutex may stay locked after event thread cancel - initiate it. */
Quoted 8 lines

drivers/vdpa/mlx5/mlx5_vdpa.c line 302

Quoted 5 lines
drivers/vdpa/mlx5/mlx5_vdpa.c +1 -0
+     pthread_mutex_init(&priv->vq_config_lock, NULL);
      DRV_LOG(INFO, "vDPA device %d was closed.", vid);
      return ret;
 }

I wonder if it would be possible and cleaner to disable cancellation on the thread while the mutex is held?

+1

Signature · 1 line
David Marchand
#3 lore raw
Quoted 4 lines · David Marchand
Quoted 2 lines

I wonder if it would be possible and cleaner to disable cancellation on the thread while the mutex is held?

+1

IEEE Std 1003.1-2001/Cor 2-2004, item XBD/TC2/D6/26 is applied, adding pthread_t to the list of types that are not required to be arithmetic types, thus allowing pthread_t to be defined as a structure.

It would be better to leave pthread_t alone and not interpret it:

if (priv->timer_tid) {
    pthread_cancel(priv->timer_tid);
    pthread_join(priv->timer_tid, &status);
}

priv->timer_tid = 0;

Signature · 1 line
David Marchand
#4 lore raw

Hi Maxime and David

Thank you for Review.

From: David Marchand

Quoted 4 lines
Quoted 2 lines · David Marchand
Quoted 2 lines

I wonder if it would be possible and cleaner to disable cancellation on the thread while the mutex is held?

Yes, we can cause thread to return by some global variable sync. It is the same logic.

Quoted 6 lines
Quoted 1 line

+1

IEEE Std 1003.1-2001/Cor 2-2004, item XBD/TC2/D6/26 is applied, adding pthread_t to the list of types that are not required to be arithmetic types, thus allowing pthread_t to be defined as a structure.

Quoted 6 lines

It would be better to leave pthread_t alone and not interpret it:

if (priv->timer_tid) {
    pthread_cancel(priv->timer_tid);
    pthread_join(priv->timer_tid, &status); }

priv->timer_tid = 0;

I'm not sure why you think it is better in this specific case.
The cancellation will close the thread in faster way, no need to wait for the thread to close itself.

Quoted 3 lines

--
David Marchand

#5 lore raw

Hi Matan,

Quoted 4 lines · Matan Azrad

Hi Maxime and David

Thank you for Review.

Quoted 8 lines

From: David Marchand

Quoted 4 lines
Quoted 2 lines · David Marchand
Quoted 2 lines

I wonder if it would be possible and cleaner to disable cancellation on the thread while the mutex is held?

Yes, we can cause thread to return by some global variable sync. It is the same logic.

No, that was not my suggestion. My suggestion is to block the thread cancellation while in the critical section, using pthread_setcancelstate().

Quoted 22 lines
Quoted 6 lines
Quoted 1 line

+1

IEEE Std 1003.1-2001/Cor 2-2004, item XBD/TC2/D6/26 is applied, adding pthread_t to the list of types that are not required to be arithmetic types, thus allowing pthread_t to be defined as a structure.

Quoted 6 lines

It would be better to leave pthread_t alone and not interpret it:

if (priv->timer_tid) {
    pthread_cancel(priv->timer_tid);
    pthread_join(priv->timer_tid, &status); }

priv->timer_tid = 0;

I'm not sure why you think it is better in this specific case.
The cancellation will close the thread in faster way, no need to wait for the thread to close itself.

Quoted 3 lines

--
David Marchand

#6 lore raw

From: Maxime Coquelin

Quoted 10 lines

Hi Matan,

Quoted 7 lines · Matan Azrad

Hi Maxime and David

Thank you for Review.

From: David Marchand

Quoted 2 lines

On Fri, Jan 8, 2021 at 9:48 AM David Marchand
<[email protected]> wrote:

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

I wonder if it would be possible and cleaner to disable cancellation on the thread while the mutex is held?

Yes, we can cause thread to return by some global variable sync. It is the same logic.

No, that was not my suggestion. My suggestion is to block the thread cancellation while in the critical section, using pthread_setcancelstate().

Yes, Generally it is better to let the thread control his cancellation, either cancel itself or enabling\disabling cancellations.

I don't see a reason to wait for the thread in current logic - the critical section is not important to be completed here.

We just want to close the thread and to clean the mutex.

Quoted 23 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line

+1

IEEE Std 1003.1-2001/Cor 2-2004, item XBD/TC2/D6/26 is applied, adding pthread_t to the list of types that are not required to be arithmetic types, thus allowing pthread_t to be defined as a structure.

It would be better to leave pthread_t alone and not interpret it:

Quoted 8 lines
Quoted 4 lines
if (priv->timer_tid) {
    pthread_cancel(priv->timer_tid);
    pthread_join(priv->timer_tid, &status); }

priv->timer_tid = 0;

I'm not sure why you think it is better in this specific case. The cancellation will close the thread in faster way, no need to wait for the

thread to close itself.

Quoted 6 lines
Quoted 5 lines

--
David Marchand

#7 lore raw
Quoted 19 lines · Matan Azrad

From: Maxime Coquelin

Quoted 16 lines

Hi Matan,

Quoted 12 lines · Matan Azrad

Hi Maxime and David

Thank you for Review.

From: David Marchand

Quoted 4 lines
Quoted 2 lines · David Marchand
Quoted 2 lines

I wonder if it would be possible and cleaner to disable cancellation on the thread while the mutex is held?

Yes, we can cause thread to return by some global variable sync. It is the same logic.

Quoted 6 lines
Quoted 2 lines

No, that was not my suggestion. My suggestion is to block the thread cancellation while in the critical section, using pthread_setcancelstate().

Yes, Generally it is better to let the thread control his cancellation, either cancel itself or enabling\disabling cancellations.

I don't see a reason to wait for the thread in current logic - the critical section is not important to be completed here.

The reason I see is there are quite a few things done in this critical section. And if tomorrow someone add new things in it, he may not know the thread can be cancelled at any time, which could cause hard to debug issues.

Quoted 26 lines

We just want to close the thread and to clean the mutex.

Quoted 23 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line

+1

IEEE Std 1003.1-2001/Cor 2-2004, item XBD/TC2/D6/26 is applied, adding pthread_t to the list of types that are not required to be arithmetic types, thus allowing pthread_t to be defined as a structure.

It would be better to leave pthread_t alone and not interpret it:

Quoted 8 lines
Quoted 4 lines
if (priv->timer_tid) {
    pthread_cancel(priv->timer_tid);
    pthread_join(priv->timer_tid, &status); }

priv->timer_tid = 0;

I'm not sure why you think it is better in this specific case. The cancellation will close the thread in faster way, no need to wait for the

thread to close itself.

Quoted 6 lines
Quoted 5 lines

--
David Marchand

#8 lore raw

From: Maxime Coquelin

Quoted 24 lines · Matan Azrad
Quoted 16 lines

From: Maxime Coquelin

Quoted 13 lines

Hi Matan,

Quoted 10 lines · Matan Azrad

Hi Maxime and David

Thank you for Review.

From: David Marchand

Quoted 4 lines
Quoted 2 lines · David Marchand
Quoted 2 lines

I wonder if it would be possible and cleaner to disable cancellation on the thread while the mutex is held?

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Yes, we can cause thread to return by some global variable sync. It is the same logic.

No, that was not my suggestion. My suggestion is to block the thread cancellation while in the critical section, using pthread_setcancelstate().

Yes, Generally it is better to let the thread control his cancellation, either

Quoted 8 lines

cancel itself or enabling\disabling cancellations.

Quoted 2 lines

I don't see a reason to wait for the thread in current logic - the critical section

is not important to be completed here.

The reason I see is there are quite a few things done in this critical section. And if tomorrow someone add new things in it, he may not know the thread can be cancelled at any time, which could cause hard to debug issues.

As I said, here it is not needed, this thread designed just to cause guest notifications.

The optional future developer mistake can be done also outside the critical section in in any other place - we cannot protect it.

The design choice is to close the thread fast.

Quoted 27 lines
Quoted 27 lines

We just want to close the thread and to clean the mutex.

Quoted 24 lines
Quoted 14 lines
Quoted 6 lines
Quoted 1 line

+1

IEEE Std 1003.1-2001/Cor 2-2004, item XBD/TC2/D6/26 is applied, adding pthread_t to the list of types that are not required to be arithmetic types, thus allowing pthread_t to be defined as a structure.

Quoted 6 lines

It would be better to leave pthread_t alone and not interpret it:

if (priv->timer_tid) {
    pthread_cancel(priv->timer_tid);
    pthread_join(priv->timer_tid, &status); }

priv->timer_tid = 0;

Quoted 3 lines

I'm not sure why you think it is better in this specific case. The cancellation will close the thread in faster way, no need to wait for the

thread to close itself.

Quoted 6 lines
Quoted 5 lines

--
David Marchand

#9 lore raw
Quoted 36 lines · Matan Azrad

From: Maxime Coquelin

Quoted 24 lines · Matan Azrad
Quoted 16 lines

From: Maxime Coquelin

Quoted 13 lines

Hi Matan,

Quoted 10 lines · Matan Azrad

Hi Maxime and David

Thank you for Review.

From: David Marchand

Quoted 4 lines
Quoted 2 lines · David Marchand
Quoted 2 lines

I wonder if it would be possible and cleaner to disable cancellation on the thread while the mutex is held?

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Yes, we can cause thread to return by some global variable sync. It is the same logic.

No, that was not my suggestion. My suggestion is to block the thread cancellation while in the critical section, using pthread_setcancelstate().

Yes, Generally it is better to let the thread control his cancellation, either

Quoted 8 lines

cancel itself or enabling\disabling cancellations.

Quoted 2 lines

I don't see a reason to wait for the thread in current logic - the critical section

is not important to be completed here.

The reason I see is there are quite a few things done in this critical section. And if tomorrow someone add new things in it, he may not know the thread can be cancelled at any time, which could cause hard to debug issues.

Quoted 5 lines

As I said, here it is not needed, this thread designed just to cause guest notifications.

The optional future developer mistake can be done also outside the critical section in in any other place - we cannot protect it.

The design choice is to close the thread fast.

But why is it so urgent that it cannot been stopped cleanly? I don't think it would add seconds delay by doing it in a clean way.

Thanks,
Maxime

Quoted 28 lines
Quoted 27 lines
Quoted 27 lines

We just want to close the thread and to clean the mutex.

Quoted 24 lines
Quoted 14 lines
Quoted 6 lines
Quoted 1 line

+1

IEEE Std 1003.1-2001/Cor 2-2004, item XBD/TC2/D6/26 is applied, adding pthread_t to the list of types that are not required to be arithmetic types, thus allowing pthread_t to be defined as a structure.

Quoted 6 lines

It would be better to leave pthread_t alone and not interpret it:

if (priv->timer_tid) {
    pthread_cancel(priv->timer_tid);
    pthread_join(priv->timer_tid, &status); }

priv->timer_tid = 0;

Quoted 3 lines

I'm not sure why you think it is better in this specific case. The cancellation will close the thread in faster way, no need to wait for the

thread to close itself.

Quoted 6 lines
Quoted 5 lines

--
David Marchand

#10 lore raw

From: Maxime Coquelin

Quoted 44 lines · Matan Azrad
Quoted 25 lines

From: Maxime Coquelin

Quoted 22 lines · Matan Azrad
Quoted 21 lines

From: Maxime Coquelin

Quoted 18 lines

Hi Matan,

Quoted 4 lines · Matan Azrad

Hi Maxime and David

Thank you for Review.

Quoted 8 lines

From: David Marchand

Quoted 4 lines
Quoted 2 lines · David Marchand
Quoted 2 lines

I wonder if it would be possible and cleaner to disable cancellation on the thread while the mutex is held?

Yes, we can cause thread to return by some global variable sync. It is the same logic.

No, that was not my suggestion. My suggestion is to block the thread cancellation while in the critical section, using

pthread_setcancelstate().

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

Yes, Generally it is better to let the thread control his cancellation, either

cancel itself or enabling\disabling cancellations.

Quoted 1 line
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines

I don't see a reason to wait for the thread in current logic - the critical section

is not important to be completed here.

The reason I see is there are quite a few things done in this critical section. And if tomorrow someone add new things in it, he may not know the thread can be cancelled at any time, which could cause

hard to debug issues.

Quoted 2 lines

As I said, here it is not needed, this thread designed just to cause guest

notifications.

Quoted 1 line
Quoted 7 lines
Quoted 1 line

The optional future developer mistake can be done also outside the critical

section in in any other place - we cannot protect it.

Quoted 2 lines

The design choice is to close the thread fast.

But why is it so urgent that it cannot been stopped cleanly? I don't think it would add seconds delay by doing it in a clean way.

We have system calls there per queue.
No need this optional delay just because of mutex cleaning.

Quoted 31 lines

Thanks,
Maxime

Quoted 28 lines
Quoted 27 lines
Quoted 27 lines

We just want to close the thread and to clean the mutex.

Quoted 24 lines
Quoted 14 lines
Quoted 6 lines
Quoted 1 line

+1

IEEE Std 1003.1-2001/Cor 2-2004, item XBD/TC2/D6/26 is applied, adding pthread_t to the list of types that are not required to be arithmetic types, thus allowing pthread_t to be defined as a structure.

Quoted 6 lines

It would be better to leave pthread_t alone and not interpret it:

if (priv->timer_tid) {
    pthread_cancel(priv->timer_tid);
    pthread_join(priv->timer_tid, &status); }

priv->timer_tid = 0;

Quoted 3 lines

I'm not sure why you think it is better in this specific case. The cancellation will close the thread in faster way, no need to wait for the

thread to close itself.

Quoted 6 lines
Quoted 5 lines

--
David Marchand

#11 lore raw
Quoted 50 lines · Matan Azrad

From: Maxime Coquelin

Quoted 47 lines · Matan Azrad
Quoted 25 lines

From: Maxime Coquelin

Quoted 22 lines · Matan Azrad
Quoted 21 lines

From: Maxime Coquelin

Quoted 18 lines

Hi Matan,

Quoted 4 lines · Matan Azrad

Hi Maxime and David

Thank you for Review.

Quoted 8 lines

From: David Marchand

Quoted 4 lines
Quoted 2 lines · David Marchand
Quoted 2 lines

I wonder if it would be possible and cleaner to disable cancellation on the thread while the mutex is held?

Yes, we can cause thread to return by some global variable sync. It is the same logic.

No, that was not my suggestion. My suggestion is to block the thread cancellation while in the critical section, using

pthread_setcancelstate().

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

Yes, Generally it is better to let the thread control his cancellation, either

cancel itself or enabling\disabling cancellations.

Quoted 1 line
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines

I don't see a reason to wait for the thread in current logic - the critical section

is not important to be completed here.

The reason I see is there are quite a few things done in this critical section. And if tomorrow someone add new things in it, he may not know the thread can be cancelled at any time, which could cause

hard to debug issues.

Quoted 2 lines

As I said, here it is not needed, this thread designed just to cause guest

notifications.

Quoted 2 lines

The optional future developer mistake can be done also outside the critical

section in in any other place - we cannot protect it.

Quoted 1 line
Quoted 7 lines
Quoted 4 lines
Quoted 1 line

The design choice is to close the thread fast.

But why is it so urgent that it cannot been stopped cleanly? I don't think it would add seconds delay by doing it in a clean way.

We have system calls there per queue.
No need this optional delay just because of mutex cleaning.

OK, up to you...

And what about the timer lock?

Quoted 34 lines
Quoted 33 lines

Thanks,
Maxime

Quoted 28 lines
Quoted 27 lines
Quoted 27 lines

We just want to close the thread and to clean the mutex.

Quoted 24 lines
Quoted 14 lines
Quoted 6 lines
Quoted 1 line

+1

IEEE Std 1003.1-2001/Cor 2-2004, item XBD/TC2/D6/26 is applied, adding pthread_t to the list of types that are not required to be arithmetic types, thus allowing pthread_t to be defined as a structure.

Quoted 6 lines

It would be better to leave pthread_t alone and not interpret it:

if (priv->timer_tid) {
    pthread_cancel(priv->timer_tid);
    pthread_join(priv->timer_tid, &status); }

priv->timer_tid = 0;

Quoted 3 lines

I'm not sure why you think it is better in this specific case. The cancellation will close the thread in faster way, no need to wait for the

thread to close itself.

Quoted 6 lines
Quoted 5 lines

--
David Marchand

#12 lore raw

From: Maxime Coquelin

Quoted 56 lines
Quoted 56 lines

From: Maxime Coquelin

Quoted 54 lines · Matan Azrad
Quoted 25 lines

From: Maxime Coquelin

Quoted 22 lines · Matan Azrad
Quoted 21 lines

From: Maxime Coquelin

Quoted 18 lines

Hi Matan,

Quoted 4 lines · Matan Azrad

Hi Maxime and David

Thank you for Review.

Quoted 8 lines

From: David Marchand

Quoted 4 lines
Quoted 2 lines · David Marchand
Quoted 2 lines

I wonder if it would be possible and cleaner to disable cancellation on the thread while the mutex is held?

Yes, we can cause thread to return by some global variable sync. It is the same logic.

No, that was not my suggestion. My suggestion is to block the thread cancellation while in the critical section, using

pthread_setcancelstate().

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

Yes, Generally it is better to let the thread control his cancellation, either

cancel itself or enabling\disabling cancellations.

Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines

I don't see a reason to wait for the thread in current logic - the critical section

is not important to be completed here.

The reason I see is there are quite a few things done in this critical section. And if tomorrow someone add new things in it, he may not know the thread can be cancelled at any time, which could cause

hard to debug issues.

Quoted 3 lines

As I said, here it is not needed, this thread designed just to cause guest

notifications.

Quoted 3 lines

The optional future developer mistake can be done also outside the critical

section in in any other place - we cannot protect it.

Quoted 2 lines

The design choice is to close the thread fast.

But why is it so urgent that it cannot been stopped cleanly? I don't think it would add seconds delay by doing it in a clean way.

Quoted 6 lines
Quoted 2 lines

We have system calls there per queue.
No need this optional delay just because of mutex cleaning.

OK, up to you...

And what about the timer lock?

Existing code initiates it before reusing...

Thanks.

Quoted 36 lines
Quoted 6 lines
Quoted 6 lines

Thanks,
Maxime

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

We just want to close the thread and to clean the mutex.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line

+1

IEEE Std 1003.1-2001/Cor 2-2004, item XBD/TC2/D6/26 is applied, adding pthread_t to the list of types that are not required to be arithmetic types, thus allowing pthread_t to be defined as a

structure.

Quoted 22 lines
Quoted 21 lines
Quoted 21 lines
Quoted 20 lines
Quoted 20 lines
Quoted 19 lines
Quoted 9 lines
Quoted 7 lines

It would be better to leave pthread_t alone and not interpret it:

if (priv->timer_tid) {
    pthread_cancel(priv->timer_tid);
    pthread_join(priv->timer_tid, &status); }

priv->timer_tid = 0;

Quoted 3 lines

I'm not sure why you think it is better in this specific case. The cancellation will close the thread in faster way, no need to wait for the

thread to close itself.

Quoted 6 lines
Quoted 5 lines

--
David Marchand

#13 lore raw
Quoted 59 lines · Matan Azrad

From: Maxime Coquelin

Quoted 56 lines
Quoted 56 lines

From: Maxime Coquelin

Quoted 54 lines · Matan Azrad
Quoted 25 lines

From: Maxime Coquelin

Quoted 22 lines · Matan Azrad
Quoted 21 lines

From: Maxime Coquelin

Quoted 18 lines

Hi Matan,

Quoted 4 lines · Matan Azrad

Hi Maxime and David

Thank you for Review.

Quoted 8 lines

From: David Marchand

Quoted 4 lines
Quoted 2 lines · David Marchand
Quoted 2 lines

I wonder if it would be possible and cleaner to disable cancellation on the thread while the mutex is held?

Yes, we can cause thread to return by some global variable sync. It is the same logic.

No, that was not my suggestion. My suggestion is to block the thread cancellation while in the critical section, using

pthread_setcancelstate().

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

Yes, Generally it is better to let the thread control his cancellation, either

cancel itself or enabling\disabling cancellations.

Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines

I don't see a reason to wait for the thread in current logic - the critical section

is not important to be completed here.

The reason I see is there are quite a few things done in this critical section. And if tomorrow someone add new things in it, he may not know the thread can be cancelled at any time, which could cause

hard to debug issues.

Quoted 3 lines

As I said, here it is not needed, this thread designed just to cause guest

notifications.

Quoted 3 lines

The optional future developer mistake can be done also outside the critical

section in in any other place - we cannot protect it.

Quoted 2 lines

The design choice is to close the thread fast.

But why is it so urgent that it cannot been stopped cleanly? I don't think it would add seconds delay by doing it in a clean way.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

We have system calls there per queue.
No need this optional delay just because of mutex cleaning.

OK, up to you...

And what about the timer lock?

Existing code initiates it before reusing...

Ok, so why not applying same logic for both mutexes?

Quoted 39 lines

Thanks.

Quoted 36 lines
Quoted 6 lines
Quoted 6 lines

Thanks,
Maxime

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

We just want to close the thread and to clean the mutex.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line

+1

IEEE Std 1003.1-2001/Cor 2-2004, item XBD/TC2/D6/26 is applied, adding pthread_t to the list of types that are not required to be arithmetic types, thus allowing pthread_t to be defined as a

structure.

Quoted 22 lines
Quoted 21 lines
Quoted 21 lines
Quoted 20 lines
Quoted 20 lines
Quoted 19 lines
Quoted 9 lines
Quoted 7 lines

It would be better to leave pthread_t alone and not interpret it:

if (priv->timer_tid) {
    pthread_cancel(priv->timer_tid);
    pthread_join(priv->timer_tid, &status); }

priv->timer_tid = 0;

Quoted 3 lines

I'm not sure why you think it is better in this specific case. The cancellation will close the thread in faster way, no need to wait for the

thread to close itself.

Quoted 6 lines
Quoted 5 lines

--
David Marchand

#14 lore raw

From: Maxime Coquelin

Quoted 63 lines · Matan Azrad
Quoted 62 lines

From: Maxime Coquelin

Quoted 59 lines
Quoted 58 lines

From: Maxime Coquelin

Quoted 46 lines · Matan Azrad
Quoted 25 lines

From: Maxime Coquelin

Quoted 22 lines · Matan Azrad
Quoted 21 lines

From: Maxime Coquelin

Quoted 18 lines

Hi Matan,

Quoted 4 lines · Matan Azrad

Hi Maxime and David

Thank you for Review.

Quoted 8 lines

From: David Marchand

Quoted 4 lines
Quoted 2 lines · David Marchand
Quoted 2 lines

I wonder if it would be possible and cleaner to disable
cancellation on the thread while the mutex is held?

Yes, we can cause thread to return by some global variable sync. It is the same logic.

No, that was not my suggestion. My suggestion is to block the thread cancellation while in the critical section, using

pthread_setcancelstate().

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

Yes, Generally it is better to let the thread control his cancellation, either

cancel itself or enabling\disabling cancellations.

Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines

I don't see a reason to wait for the thread in current logic - the critical section

is not important to be completed here.

The reason I see is there are quite a few things done in this critical section. And if tomorrow someone add new things in it, he may not know the thread can be cancelled at any time, which could cause

hard to debug issues.

Quoted 3 lines

As I said, here it is not needed, this thread designed just to cause guest

notifications.

Quoted 1 line
Quoted 8 lines
Quoted 2 lines

The optional future developer mistake can be done also outside the critical

section in in any other place - we cannot protect it.

Quoted 2 lines

The design choice is to close the thread fast.

But why is it so urgent that it cannot been stopped cleanly? I don't think it would add seconds delay by doing it in a clean way.

We have system calls there per queue.
No need this optional delay just because of mutex cleaning.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

OK, up to you...

And what about the timer lock?

Existing code initiates it before reusing...

Ok, so why not applying same logic for both mutexes?

Different dependencies, different usage.

Timer timer lock is more tied to the poll thread usage, this patch mutex has more usage, not only for the poll thread management.

Quoted 41 lines
Quoted 41 lines

Thanks.

Quoted 37 lines
Quoted 6 lines
Quoted 6 lines

Thanks,
Maxime

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

We just want to close the thread and to clean the mutex.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line

+1

IEEE Std 1003.1-2001/Cor 2-2004, item XBD/TC2/D6/26 is applied, adding pthread_t to the list of types that are not required to be arithmetic types, thus allowing pthread_t to be defined as a

structure.

Quoted 22 lines
Quoted 21 lines
Quoted 21 lines
Quoted 20 lines
Quoted 20 lines
Quoted 19 lines
Quoted 9 lines
Quoted 7 lines

It would be better to leave pthread_t alone and not interpret it:

if (priv->timer_tid) {
    pthread_cancel(priv->timer_tid);
    pthread_join(priv->timer_tid, &status); }

priv->timer_tid = 0;

Quoted 3 lines

I'm not sure why you think it is better in this specific case. The cancellation will close the thread in faster way, no need to wait for the

thread to close itself.

Quoted 6 lines
Quoted 5 lines

--
David Marchand

Acked-by on v1 1/1
#15 lore raw
In reply to Matan Azrad
Quoted 27 lines · Matan Azrad

When the vDPA device is closed, the driver polling thread is canceled. The polling thread locks the configuration mutex while it polls the CQs.

When the cancellation happens, it may terminate the thread inside the critical section what remains the configuration mutex locked.

After device close, the driver may be configured again, in this case, for example, when the first queue state is updated, the driver tries to lock the mutex again and deadlock appears.

Initialize the mutex after the polling thread cancellation.

Fixes 99abbd62c272 ("vdpa/mlx5: fix queue update synchronization")
---
 drivers/vdpa/mlx5/mlx5_vdpa.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/vdpa/mlx5/mlx5_vdpa.c +0 -0
index b64f364..0b2f1ab 100644

drivers/vdpa/mlx5/mlx5_vdpa.c line 302

Quoted 8 lines
drivers/vdpa/mlx5/mlx5_vdpa.c +2 -0
@@ -295,6 +295,8 @@
 	}
 	priv->configured = 0;
 	priv->vid = 0;
+	/* The mutex may stay locked after event thread cancel - initiate it. */
+	pthread_mutex_init(&priv->vq_config_lock, NULL);
 	DRV_LOG(INFO, "vDPA device %d was closed.", vid);
 	return ret;
 }
#16 lore raw
In reply to Matan Azrad
Quoted 12 lines · Matan Azrad

When the vDPA device is closed, the driver polling thread is canceled. The polling thread locks the configuration mutex while it polls the CQs.

When the cancellation happens, it may terminate the thread inside the critical section what remains the configuration mutex locked.

After device close, the driver may be configured again, in this case, for example, when the first queue state is updated, the driver tries to lock the mutex again and deadlock appears.

Initialize the mutex after the polling thread cancellation.

Quoted 9 lines
Fixes 99abbd62c272 ("vdpa/mlx5: fix queue update synchronization")
---
 drivers/vdpa/mlx5/mlx5_vdpa.c | 2 ++
 1 file changed, 2 insertions(+)

Applied to dpdk-next-virtio/main.

Thanks,
Maxime