vhost: fix deadlock when handling user messages
PATCH 1/1 vhost: fix deadlock when handling user messages
In vhost_user_msg_handler(), if vhost message handling failed, we should check whether the queue is locked and release the lock before returning. Or, it will cause a deadlock later.
--- lib/vhost/vhost_user.c | 2 ++ 1 file changed, 2 insertions(+)
lib/vhost/vhost_user.c +2 -0
@@ -3113,6 +3113,8 @@ vhost_user_msg_handler(int vid, int fd)send_vhost_reply (dev, fd, &ctx); }else if (ret ==RTE_VHOST_MSG_RESULT_ERR ) {VHOST_LOG_CONFIG (ERR ,"(%s) vhost message handling failed.\n" , dev->ifname); +if (unlock_required) +vhost_user_unlock_all_queue_pairs (dev);return -1 ; }
Signature · 1 line
2.25.1
Hello,
Quoted 5 lines · Wenwu Ma
In
vhost_user_msg_handler(), if vhost message handling failed, we should check whether the queue is locked and release the lock before returning. Or, it will cause a deadlock later.
Quoted 11 lines
Signed-off-by Wenwu Ma <[email protected]>--- lib/vhost/vhost_user.c | 2 ++ 1 file changed, 2 insertions(+)lib/vhost/vhost_user.c +0
-0
lib/vhost/vhost_user.c vhost_user_msg_handler(int vid, int fd) · line 3120
Quoted 8 lines
lib/vhost/vhost_user.c +2
-0@@ -3113,6 +3113,8 @@ vhost_user_msg_handler(int vid, int fd)send_vhost_reply (dev, fd, &ctx); }else if (ret ==RTE_VHOST_MSG_RESULT_ERR ) {VHOST_LOG_CONFIG (ERR ,"(%s) vhost message handling failed.\n" , dev->ifname); +if (unlock_required) +vhost_user_unlock_all_queue_pairs (dev);return -1 ; }
This fixes the issue, but my concern is that changes in the future might introduce a new return statement (forgetting to unlock). I suggest having a single return statement, like:
lib/vhost/vhost_user.c +0 -1
@@ -2976,7 +2976,6 @@ vhost_user_msg_handler(int vid, int fd)return -1 ; } - ret =0 ; request = ctx.msg.request.master;if (request >VHOST_USER_NONE && request <VHOST_USER_MAX && vhost_message_str[request]) {
@@ -3113,9 +3112,11 @@ vhost_user_msg_handler(int vid, int fd)send_vhost_reply (dev, fd, &ctx); }else if (ret ==RTE_VHOST_MSG_RESULT_ERR ) { VHOST_LOG_CONFIG(ERR, "(%s) vhost message handling
failed.\n", dev->ifname);
- return -1;
+ ret = -1;
+ goto unlock;
}+ ret = 0;
for (i = 0; i < dev->nr_vring; i++) {
struct vhost_virtqueue *vq = dev->virtqueue[i];
bool cur_ready = vq_is_ready(dev, vq);
@@ -3126,10 +3127,11 @@ vhost_user_msg_handler(int vid, int fd)
}
}+unlock:
if (unlock_required)
vhost_user_unlock_all_queue_pairs(dev);- if (!virtio_is_ready(dev))
+ if (ret != 0 || !virtio_is_ready(dev))
goto out;
/*
@@ -3156,7 +3158,7 @@ vhost_user_msg_handler(int vid, int fd)
} out:
- return 0;
+ return ret;
}
static int process_slave_message_reply(struct virtio_net *dev,
Signature · 1 line
David Marchand
PATCH v2 1/1 vhost: fix deadlock when handling user messages
In reply to Wenwu MaIn vhost_user_msg_handler(), if vhost message handling failed, we should check whether the queue is locked and release the lock before returning. Or, it will cause a deadlock later.
--- lib/vhost/vhost_user.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
lib/vhost/vhost_user.c +6 -4
@@ -2976,7 +2976,6 @@ vhost_user_msg_handler(int vid, int fd)return -1 ; } - ret =0 ; request = ctx.msg.request.master;if (request >VHOST_USER_NONE && request <VHOST_USER_MAX && vhost_message_str[request]) {
@@ -3113,9 +3112,11 @@ vhost_user_msg_handler(int vid, int fd)send_vhost_reply (dev, fd, &ctx); }else if (ret ==RTE_VHOST_MSG_RESULT_ERR ) {VHOST_LOG_CONFIG (ERR ,"(%s) vhost message handling failed.\n" , dev->ifname); -return -1 ; + ret =-1 ; + goto unlock; } + ret =0 ;for (i =0 ; i < dev->nr_vring; i++) {struct vhost_virtqueue *vq = dev->virtqueue[i];bool cur_ready =vq_is_ready (dev, vq);
@@ -3126,10 +3127,11 @@ vhost_user_msg_handler(int vid, int fd) } } +unlock:if (unlock_required)vhost_user_unlock_all_queue_pairs (dev); -if (!virtio_is_ready (dev)) +if (ret !=0 || !virtio_is_ready (dev)) goto out;/*
@@ -3156,7 +3158,7 @@ vhost_user_msg_handler(int vid, int fd) } out: -return 0 ; +return ret; } static int process_slave_message_reply(struct virtio_net *dev,
Signature · 1 line
2.25.1
Quoted 72 lines
-----Original Message-----
From: Ma, WenwuX <[email protected]>
Sent: Friday, May 6, 2022 12:17 AM
To: [email protected]; Xia, Chenbo <[email protected]>; [email protected]
Cc: Hu, Jiayu <[email protected]>; Wang, Yinan <[email protected]>; He, Xingguang <[email protected]>; Ma, WenwuX <[email protected]>; [email protected]
Subject: [PATCH v2] vhost: fix deadlock when handling user messagesIn
vhost_user_msg_handler(), if vhost message handling failed, we should check whether the queue is locked and release the lock before returning. Or, it will cause a deadlock later.Fixes 7f31d4ea05ca ("vhost: fix lock on device readiness notification")Signed-off-by Wenwu Ma <[email protected]>--- lib/vhost/vhost_user.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)lib/vhost/vhost_user.c +3
-2@@ -2976,7 +2976,6 @@ vhost_user_msg_handler(int vid, int fd)return -1 ; } - ret =0 ; request = ctx.msg.request.master;if (request >VHOST_USER_NONE && request <VHOST_USER_MAX && vhost_message_str[request]) {@@ -3113,9 +3112,11 @@ vhost_user_msg_handler(int vid, int fd)send_vhost_reply (dev, fd, &ctx); }else if (ret ==RTE_VHOST_MSG_RESULT_ERR ) {VHOST_LOG_CONFIG (ERR ,"(%s) vhost message handling failed.\n" , dev->ifname); -return -1 ; + ret =-1 ; + goto unlock; } + ret =0 ;for (i =0 ; i < dev->nr_vring; i++) {struct vhost_virtqueue *vq = dev->virtqueue[i];lib/vhost/vhost_user.c +3
-3bool cur_ready =vq_is_ready (dev, vq);@@ -3126,10 +3127,11 @@ vhost_user_msg_handler(int vid, int fd) } } +unlock:if (unlock_required)vhost_user_unlock_all_queue_pairs (dev); -if (!virtio_is_ready (dev)) +if (ret !=0 || !virtio_is_ready (dev)) goto out;/* @@ -3156,7 +3158,7 @@ vhost_user_msg_handler(int vid, int fd) } out: -return 0 ; +return ret; } static int process_slave_message_reply(struct virtio_net *dev, --
Quoted 1 line
2.25.1
'vhost: fix deadlock when message handling failed' may be a better title.
PATCH v3 1/1 vhost: fix deadlock when message handling failed
In reply to Wenwu MaIn vhost_user_msg_handler(), if vhost message handling failed, we should check whether the queue is locked and release the lock before returning. Or, it will cause a deadlock later.
--- lib/vhost/vhost_user.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
lib/vhost/vhost_user.c +6 -4
@@ -2976,7 +2976,6 @@ vhost_user_msg_handler(int vid, int fd)return -1 ; } - ret =0 ; request = ctx.msg.request.master;if (request >VHOST_USER_NONE && request <VHOST_USER_MAX && vhost_message_str[request]) {
@@ -3113,9 +3112,11 @@ vhost_user_msg_handler(int vid, int fd)send_vhost_reply (dev, fd, &ctx); }else if (ret ==RTE_VHOST_MSG_RESULT_ERR ) {VHOST_LOG_CONFIG (ERR ,"(%s) vhost message handling failed.\n" , dev->ifname); -return -1 ; + ret =-1 ; + goto unlock; } + ret =0 ;for (i =0 ; i < dev->nr_vring; i++) {struct vhost_virtqueue *vq = dev->virtqueue[i];bool cur_ready =vq_is_ready (dev, vq);
@@ -3126,10 +3127,11 @@ vhost_user_msg_handler(int vid, int fd) } } +unlock:if (unlock_required)vhost_user_unlock_all_queue_pairs (dev); -if (!virtio_is_ready (dev)) +if (ret !=0 || !virtio_is_ready (dev)) goto out;/*
@@ -3156,7 +3158,7 @@ vhost_user_msg_handler(int vid, int fd) } out: -return 0 ; +return ret; } static int process_slave_message_reply(struct virtio_net *dev,
Signature · 1 line
2.25.1
RE: [PATCH v3] vhost: fix deadlock when message handling failed
Quoted 72 lines
-----Original Message-----
From: Ma, WenwuX <[email protected]>
Sent: Saturday, May 7, 2022 9:28 PM
To: [email protected]; Xia, Chenbo <[email protected]>; [email protected]
Cc: Hu, Jiayu <[email protected]>; Wang, Yinan <[email protected]>; He, Xingguang <[email protected]>; Ma, WenwuX <[email protected]>; [email protected]
Subject: [PATCH v3] vhost: fix deadlock when message handling failedIn
vhost_user_msg_handler(), if vhost message handling failed, we should check whether the queue is locked and release the lock before returning. Or, it will cause a deadlock later.Fixes 7f31d4ea05ca ("vhost: fix lock on device readiness notification")Signed-off-by Wenwu Ma <[email protected]>--- lib/vhost/vhost_user.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)lib/vhost/vhost_user.c +3
-2@@ -2976,7 +2976,6 @@ vhost_user_msg_handler(int vid, int fd)return -1 ; } - ret =0 ; request = ctx.msg.request.master;if (request >VHOST_USER_NONE && request <VHOST_USER_MAX && vhost_message_str[request]) {@@ -3113,9 +3112,11 @@ vhost_user_msg_handler(int vid, int fd)send_vhost_reply (dev, fd, &ctx); }else if (ret ==RTE_VHOST_MSG_RESULT_ERR ) {VHOST_LOG_CONFIG (ERR ,"(%s) vhost message handling failed.\n" , dev->ifname); -return -1 ; + ret =-1 ; + goto unlock; } + ret =0 ;for (i =0 ; i < dev->nr_vring; i++) {struct vhost_virtqueue *vq = dev->virtqueue[i];lib/vhost/vhost_user.c +3
-3bool cur_ready =vq_is_ready (dev, vq);@@ -3126,10 +3127,11 @@ vhost_user_msg_handler(int vid, int fd) } } +unlock:if (unlock_required)vhost_user_unlock_all_queue_pairs (dev); -if (!virtio_is_ready (dev)) +if (ret !=0 || !virtio_is_ready (dev)) goto out;/* @@ -3156,7 +3158,7 @@ vhost_user_msg_handler(int vid, int fd) } out: -return 0 ; +return ret; } static int process_slave_message_reply(struct virtio_net *dev, --
Quoted 1 line
2.25.1
RE: [PATCH v3] vhost: fix deadlock when message handling failed
In reply to Wenwu MaQuoted 10 lines
-----Original Message-----
From: Wenwu Ma <[email protected]>
Sent: Saturday, May 7, 2022 9:28 PM
To: [email protected]; Xia, Chenbo <[email protected]>; [email protected]
Cc: Hu, Jiayu <[email protected]>; Wang, Yinan <[email protected]>; He, Xingguang <[email protected]>; Ma, WenwuX <[email protected]>; [email protected]
Subject: [PATCH v3] vhost: fix deadlock when message handling failed
Re: [PATCH v3] vhost: fix deadlock when message handling failed
In reply to Wenwu MaQuoted 10 lines · Wenwu Ma
In
vhost_user_msg_handler(), if vhost message handling failed, we should check whether the queue is locked and release the lock before returning. Or, it will cause a deadlock later.Fixes 7f31d4ea05ca ("vhost: fix lock on device readiness notification")Signed-off-by Wenwu Ma <[email protected]>
Thanks, lgtm.
Signature · 1 line
David Marchand
Maxime Coquelin
AppliedRe: [PATCH v3] vhost: fix deadlock when message handling failed
In reply to Wenwu MaQuoted 5 lines · Wenwu Ma
In
vhost_user_msg_handler(), if vhost message handling failed, we should check whether the queue is locked and release the lock before returning. Or, it will cause a deadlock later.
Quoted 8 lines
Fixes 7f31d4ea05ca ("vhost: fix lock on device readiness notification")Signed-off-by Wenwu Ma <[email protected]>--- lib/vhost/vhost_user.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
Applied to dpdk-next-virtio/main.
Thanks,
Maxime
RE: [PATCH v3] vhost: fix deadlock when message handling failed
HI ALL,
I see that in function vhost_user_msg_handler.
We use "ret" to store both vhost msg return code like "RTE_VHOST_MSG_RESULT_XXX" and function return value. I wonder if it is better to use two different variable to make it easy to read.
Quoted 30 lines
-----Original Message-----
From: Maxime Coquelin <[email protected]>
Sent: Tuesday, May 17, 2022 9:24 PM
To: Ma, WenwuX <[email protected]>; Xia, Chenbo <[email protected]>; [email protected]
Cc: Hu, Jiayu <[email protected]>; Wang, Yinan <[email protected]>; He, Xingguang <[email protected]>; [email protected]
Subject: Re: [PATCH v3] vhost: fix deadlock when message handling failedQuoted 4 lines · Wenwu Ma
In
vhost_user_msg_handler(), if vhost message handling failed, we should check whether the queue is locked and release the lock before returning. Or, it will cause a deadlock later.Quoted 9 lines
Fixes 7f31d4ea05ca ("vhost: fix lock on device readinessnotification")
Cc: [email protected]Signed-off-by Wenwu Ma <[email protected]>--- lib/vhost/vhost_user.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)Applied to dpdk-next-virtio/main.
Thanks,
Maxime
RE: [PATCH v3] vhost: fix deadlock when message handling failed
Quoted 9 lines
-----Original Message-----
From: Pei, Andy <[email protected]>
Sent: Monday, July 11, 2022 4:42 PM
To: Maxime Coquelin <[email protected]>; Ma, WenwuX <[email protected]>; Xia, Chenbo <[email protected]>; [email protected]
Cc: Hu, Jiayu <[email protected]>; Wang, Yinan <[email protected]>; He, Xingguang <[email protected]>; [email protected]
Subject: RE: [PATCH v3] vhost: fix deadlock when message handling failed
Quoted 7 lines
HI ALL,
I see that in function
vhost_user_msg_handler.
We use "ret" to store both vhost msg return code like "RTE_VHOST_MSG_RESULT_XXX" and function return value.
I wonder if it is better to use two different variable to make it easy to read.
By saying 'function return value', you mean which function? Can you elaborate more?
Thanks,
Chenbo
Quoted 32 lines
Quoted 7 lines
-----Original Message-----
From: Maxime Coquelin <[email protected]>
Sent: Tuesday, May 17, 2022 9:24 PM
To: Ma, WenwuX <[email protected]>; Xia, Chenbo <[email protected]>; [email protected]
Cc: Hu, Jiayu <[email protected]>; Wang, Yinan <[email protected]>;He,
Quoted 24 lines
Xingguang <[email protected]>; [email protected]
Subject: Re: [PATCH v3] vhost: fix deadlock when message handling failedQuoted 4 lines · Wenwu Ma
In
vhost_user_msg_handler(), if vhost message handling failed, we should check whether the queue is locked and release the lock before returning. Or, it will cause a deadlock later.Quoted 9 lines
Fixes 7f31d4ea05ca ("vhost: fix lock on device readinessnotification")
Cc: [email protected]Signed-off-by Wenwu Ma <[email protected]>--- lib/vhost/vhost_user.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)Applied to dpdk-next-virtio/main.
Thanks,
Maxime
RE: [PATCH v3] vhost: fix deadlock when message handling failed
HI Chenbo,
See below example
1.
ret = read_vhost_message(dev, fd, &ctx);
if (ret <= 0) {
if (ret < 0)
VHOST_LOG_CONFIG(dev->ifname, ERR, "vhost read message failed\n");
else
VHOST_LOG_CONFIG(dev->ifname, INFO, "vhost peer closed\n");
return -1;
}2.
ret = vhost_user_check_and_alloc_queue_pair(dev, &ctx);
if (ret < 0) {
VHOST_LOG_CONFIG(dev->ifname, ERR, "failed to alloc queue\n");
return -1;
}3. ret = -1
} else if (ret == RTE_VHOST_MSG_RESULT_ERR) {
VHOST_LOG_CONFIG(dev->ifname, ERR, "vhost message handling failed.\n");
ret = -1;
goto unlock;
}I mean ret is used to check the function is returned successfully or not while it is also used to store vhost msg return code like "RTE_VHOST_MSG_RESULT_XXX".
Quoted 70 lines
-----Original Message-----
From: Xia, Chenbo <[email protected]>
Sent: Monday, July 11, 2022 4:54 PM
To: Pei, Andy <[email protected]>; Maxime Coquelin <[email protected]>; Ma, WenwuX <[email protected]>; [email protected]
Cc: Hu, Jiayu <[email protected]>; Wang, Yinan <[email protected]>; He, Xingguang <[email protected]>; [email protected]
Subject: RE: [PATCH v3] vhost: fix deadlock when message handling failedQuoted 12 lines
-----Original Message-----
From: Pei, Andy <[email protected]>
Sent: Monday, July 11, 2022 4:42 PM
To: Maxime Coquelin <[email protected]>; Ma, WenwuX <[email protected]>; Xia, Chenbo <[email protected]>; [email protected]
Cc: Hu, Jiayu <[email protected]>; Wang, Yinan <[email protected]>; He, Xingguang <[email protected]>; [email protected]
Subject: RE: [PATCH v3] vhost: fix deadlock when message handling failedQuoted 7 lines
HI ALL,
I see that in function
vhost_user_msg_handler.
We use "ret" to store both vhost msg return code like "RTE_VHOST_MSG_RESULT_XXX" and function return value.
I wonder if it is better to use two different variable to make it easy to read.By saying 'function return value', you mean which function? Can you elaborate more?
Thanks,
ChenboQuoted 34 lines
Quoted 8 lines
-----Original Message-----
From: Maxime Coquelin <[email protected]>
Sent: Tuesday, May 17, 2022 9:24 PM
To: Ma, WenwuX <[email protected]>; Xia, Chenbo <[email protected]>; [email protected]
Cc: Hu, Jiayu <[email protected]>; Wang, Yinan <[email protected]>;He,
Quoted 25 lines
Xingguang <[email protected]>; [email protected]
Subject: Re: [PATCH v3] vhost: fix deadlock when message handling failedQuoted 4 lines · Wenwu Ma
In
vhost_user_msg_handler(), if vhost message handling failed, we should check whether the queue is locked and release the lock before returning. Or, it will cause a deadlock later.Quoted 9 lines
Fixes 7f31d4ea05ca ("vhost: fix lock on device readinessnotification")
Cc: [email protected]Signed-off-by Wenwu Ma <[email protected]>--- lib/vhost/vhost_user.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)Applied to dpdk-next-virtio/main.
Thanks,
Maxime
RE: [PATCH v3] vhost: fix deadlock when message handling failed
Hi Andy,
Quoted 43 lines
-----Original Message-----
From: Pei, Andy <[email protected]>
Sent: Monday, July 11, 2022 5:03 PM
To: Xia, Chenbo <[email protected]>; Maxime Coquelin <[email protected]>; Ma, WenwuX <[email protected]>; [email protected]
Cc: Hu, Jiayu <[email protected]>; Wang, Yinan <[email protected]>; He, Xingguang <[email protected]>; [email protected]
Subject: RE: [PATCH v3] vhost: fix deadlock when message handling failedHI Chenbo,
See below example
1.
ret = read_vhost_message(dev, fd, &ctx); if (ret <= 0) { if (ret < 0) VHOST_LOG_CONFIG(dev->ifname, ERR, "vhost read message failed\n"); else VHOST_LOG_CONFIG(dev->ifname, INFO, "vhost peerclosed\n");
return -1; }2.
ret = vhost_user_check_and_alloc_queue_pair(dev, &ctx); if (ret < 0) { VHOST_LOG_CONFIG(dev->ifname, ERR, "failed to alloc queue\n"); return -1; }3. ret = -1
} else if (ret == RTE_VHOST_MSG_RESULT_ERR) { VHOST_LOG_CONFIG(dev->ifname, ERR, "vhost message handling failed.\n"); ret = -1; goto unlock; }
Quoted 3 lines
I mean ret is used to check the function is returned successfully or not while it is also used to store vhost msg return code like "
RTE_VHOST_MSG_RESULT_XXX".
It makes sense to make it two, one int for return value, one enum rte_vhost_msg_result only for message handling. IMHO, this could improve readability.
Thanks,
Chenbo
Quoted 76 lines
Quoted 11 lines
-----Original Message-----
From: Xia, Chenbo <[email protected]>
Sent: Monday, July 11, 2022 4:54 PM
To: Pei, Andy <[email protected]>; Maxime Coquelin <[email protected]>; Ma, WenwuX <[email protected]>; [email protected]
Cc: Hu, Jiayu <[email protected]>; Wang, Yinan <[email protected]>;He,
Quoted 17 lines
Xingguang <[email protected]>; [email protected]
Subject: RE: [PATCH v3] vhost: fix deadlock when message handling failedQuoted 14 lines
-----Original Message-----
From: Pei, Andy <[email protected]>
Sent: Monday, July 11, 2022 4:42 PM
To: Maxime Coquelin <[email protected]>; Ma, WenwuX <[email protected]>; Xia, Chenbo <[email protected]>; [email protected]
Cc: Hu, Jiayu <[email protected]>; Wang, Yinan <[email protected]>; He, Xingguang <[email protected]>; [email protected]
Subject: RE: [PATCH v3] vhost: fix deadlock when message handling failedHI ALL,
Quoted 7 lines
Quoted 5 lines
I see that in function
vhost_user_msg_handler.
We use "ret" to store both vhost msg return code like "RTE_VHOST_MSG_RESULT_XXX" and function return value.
I wonder if it is better to use two different variable to make it easy to read.By saying 'function return value', you mean which function? Can you
elaborate
Quoted 39 lines
more?
Thanks,
ChenboQuoted 34 lines
Quoted 8 lines
-----Original Message-----
From: Maxime Coquelin <[email protected]>
Sent: Tuesday, May 17, 2022 9:24 PM
To: Ma, WenwuX <[email protected]>; Xia, Chenbo <[email protected]>; [email protected]
Cc: Hu, Jiayu <[email protected]>; Wang, Yinan <[email protected]>;He,
Quoted 25 lines
Xingguang <[email protected]>; [email protected]
Subject: Re: [PATCH v3] vhost: fix deadlock when message handling failedQuoted 4 lines · Wenwu Ma
In
vhost_user_msg_handler(), if vhost message handling failed, we should check whether the queue is locked and release the lock before returning. Or, it will cause a deadlock later.Quoted 9 lines
Fixes 7f31d4ea05ca ("vhost: fix lock on device readinessnotification")
Cc: [email protected]Signed-off-by Wenwu Ma <[email protected]>--- lib/vhost/vhost_user.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)Applied to dpdk-next-virtio/main.
Thanks,
Maxime