vhost: vsock: kick send_pkt worker once device is started
Subsystems (6)
PATCH v2 1/1 vhost: vsock: kick send_pkt worker once device is started
Ning Bo reported an abnormal 2-second gap when booting Kata container [1]. The unconditional timeout was caused by VSOCK_DEFAULT_CONNECT_TIMEOUT of connecting from the client side. The vhost vsock client tries to connect an initializing virtio vsock server.
The abnormal flow looks like:
host-userspace vhost vsock guest vsock
============== =========== ============
connect() --------> vhost_transport_send_pkt_work() initializing
| vq->private_data==NULL
| will not be queued
V
schedule_timeout(2s)
vhost_vsock_start() <--------- device ready
set vq->private_datawait for 2s and failedconnect() again vq->private_data!=NULL recv connecting pkt
Details:
1. Host userspace sends a connect pkt, at that time, guest vsock is under initializing, hence the vhost_vsock_start has not been called. So vq->private_data==NULL, and the pkt is not been queued to send to guest
2. Then it sleeps for 2s
3. After guest vsock finishes initializing, vq->private_data is set
4. When host userspace wakes up after 2s, send connecting pkt again, everything is fine.
As suggested by Stefano Garzarella, this fixes it by additional kicking the send_pkt worker in vhost_vsock_start once the virtio device is started. This makes the pending pkt sent again.
After this patch, kata-runtime (with vsock enabled) boot time is reduced from 3s to 1s on a ThunderX2 arm64 server.
[1] https://github.com/kata-containers/runtime/issues/1917
---
v2: new solution suggested by Stefano Garzarella
drivers/vhost/vsock.c | 5 +++++ 1 file changed, 5 insertions(+)
drivers/vhost/vsock.c +5 -0
@@ -543,6 +543,11 @@ static int vhost_vsock_start(struct vhost_vsock *vsock)mutex_unlock (&vq->mutex); } +/* Some packets may have been queued before the device was started, +* let's kick the send worker to send them. +*/ +vhost_work_queue (&vsock->dev, &vsock->send_pkt_work); +mutex_unlock (&vsock->dev.mutex);return 0 ;
Signature · 1 line
2.17.1
Quoted 37 lines · Jia He
Ning Bo reported an abnormal 2-second gap when booting Kata container [1]. The unconditional timeout was caused by
VSOCK_DEFAULT_CONNECT_TIMEOUTof connecting from the client side. The vhost vsock client tries to connect an initializing virtio vsock server.The abnormal flow looks like:
host-userspace vhost vsock guest vsock ============== =========== ============ connect() --------> vhost_transport_send_pkt_work() initializing | vq->private_data==NULL | will not be queued V schedule_timeout(2s) vhost_vsock_start() <--------- device ready set vq->private_datawait for 2s and failed
connect()again vq->private_data!=NULL recv connecting pktDetails:
1. Host userspace sends a connect pkt, at that time, guest vsock is under initializing, hence thevhost_vsock_starthas not been called. So vq->private_data==NULL, and the pkt is not been queued to send to guest
2. Then it sleeps for 2s
3. After guest vsock finishes initializing,vq->private_datais set
4. When host userspace wakes up after 2s, send connecting pkt again, everything is fine.As suggested by Stefano Garzarella, this fixes it by additional kicking the
send_pktworker invhost_vsock_startonce the virtio device is started. This makes the pending pkt sent again.After this patch, kata-runtime (with vsock enabled) boot time is reduced from 3s to 1s on a ThunderX2 arm64 server.
Quoted 8 lines
Reported-by Ning Bo <[email protected]>Suggested-by Stefano Garzarella <[email protected]>Signed-off-by Jia He <[email protected]>---
v2: new solution suggested by Stefano Garzarelladrivers/vhost/vsock.c | 5 +++++ 1 file changed, 5 insertions(+)
Thanks,
Stefano
Quoted 20 lines
drivers/vhost/vsock.c +5
-0@@ -543,6 +543,11 @@ static int vhost_vsock_start(struct vhost_vsock *vsock)mutex_unlock (&vq->mutex); } +/* Some packets may have been queued before the device was started, +* let's kick the send worker to send them. +*/ +vhost_work_queue (&vsock->dev, &vsock->send_pkt_work); +mutex_unlock (&vsock->dev.mutex);return 0 ;Signature · 1 line
2.17.1