vhost/vsock: reset only the h2g connections upon release
Subsystems (7)
PATCH 1/1 vhost/vsock: reset only the h2g connections upon release
Filtering non-h2g connections out when determining orphaned connections. Otherwise, in a nested VM configuration, destroying the nested VM (which often involves the closing of /dev/vhost-vsock if there was h2g connections to the nested VM) kills not only the h2g connections, but also all existing g2h connections to the (outmost) host which are totally unrelated.
Tested: Executed the following steps on Cuttlefish (Android running on a VM) [1]: (1) Enter into an adb shell session - to have a g2h connection inside the VM, (2) open and then close /dev/vhost-vsock by exec 3< /dev/vhost-vsock && exec 3<&-, (3) observe that the adb session is not reset.
[1] https://android.googlesource.com/device/google/cuttlefish/
--- drivers/vhost/vsock.c | 4 ++++ 1 file changed, 4 insertions(+)
drivers/vhost/vsock.c +4 -0
@@ -722,6 +722,10 @@ static void vhost_vsock_reset_orphans(struct sock *sk)* executing. */ +/* Only the h2g connections are reset */ +if (vsk->transport != &vhost_transport.transport) +return ; +/* If the peer is still valid, no need to reset connection */ if (vhost_vsock_get (vsk->remote_addr.svm_cid))return ;
Signature · 1 line
2.35.1.723.g4982287a31-goog
Stefano Garzarella
maintainerHi Jiyong,
Quoted 38 lines · Jiyong Park
Filtering non-h2g connections out when determining orphaned connections. Otherwise, in a nested VM configuration, destroying the nested VM (which often involves the closing of
/dev/vhost-vsockif there was h2g connections to the nested VM) kills not only the h2g connections, but also all existing g2h connections to the (outmost) host which are totally unrelated.Tested: Executed the following steps on Cuttlefish (Android running on a VM) [1]: (1) Enter into an
adb shellsession - to have a g2h connection inside the VM, (2) open and then close/dev/vhost-vsockbyexec 3< /dev/vhost-vsock && exec 3<&-, (3) observe that the adb session is not reset.[1] https://android.googlesource.com/device/google/cuttlefish/
Signed-off-by Jiyong Park <[email protected]>---
drivers/vhost/vsock.c| 4 ++++1 file changed, 4 insertions(+)drivers/vhost/vsock.c +4
-0@@ -722,6 +722,10 @@ static void vhost_vsock_reset_orphans(struct sock *sk)* executing. */ +/* Only the h2g connections are reset */ +if (vsk->transport != &vhost_transport.transport) +return ; +/* If the peer is still valid, no need to reset connection */ if (vhost_vsock_get (vsk->remote_addr.svm_cid))return ;Signature · 1 line
2.35.1.723.g4982287a31-goog
Thanks for your patch!
Yes, I see the problem and I think I introduced it with the multi-transports support (ooops).
So we should add this fixes tag:
IIUC the problem is for all transports that should only cycle on their own sockets. Indeed I think there is the same problem if the g2h driver will be unloaded (or a reset event is received after a VM migration), it will close all sockets of the nested h2g.
So I suggest a more generic solution, modifying vsock_for_each_connected_socket() like this (not tested):
net/vmw_vsock/af_vsock.c +7 -2
@@ -334,7 +334,8 @@ void vsock_remove_sock(struct vsock_sock *vsk) }EXPORT_SYMBOL_GPL (vsock_remove_sock); -void vsock_for_each_connected_socket (void (*fn)(struct sock *sk)) +void vsock_for_each_connected_socket (struct vsock_transport *transport, +void (*fn)(struct sock *sk)) {int i;
@@ -343,8 +344,12 @@ void vsock_for_each_connected_socket(void (*fn)(struct sock *sk))for (i =0 ; i <ARRAY_SIZE (vsock_connected_table); i++) {struct vsock_sock *vsk;list_for_each_entry (vsk, &vsock_connected_table[i], - connected_table) + connected_table) { +if (vsk->transport != transport) +continue ; +fn (sk_vsock (vsk)); + } }
And all transports that call it.
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Hi Stefano,
Quoted 91 lines · Stefano Garzarella
Hi Jiyong,
Quoted 36 lines · Jiyong Park
Filtering non-h2g connections out when determining orphaned connections. Otherwise, in a nested VM configuration, destroying the nested VM (which often involves the closing of
/dev/vhost-vsockif there was h2g connections to the nested VM) kills not only the h2g connections, but also all existing g2h connections to the (outmost) host which are totally unrelated.Tested: Executed the following steps on Cuttlefish (Android running on a VM) [1]: (1) Enter into an
adb shellsession - to have a g2h connection inside the VM, (2) open and then close/dev/vhost-vsockbyexec 3< /dev/vhost-vsock && exec 3<&-, (3) observe that the adb session is not reset.[1] https://android.googlesource.com/device/google/cuttlefish/
Signed-off-by Jiyong Park <[email protected]>---
drivers/vhost/vsock.c| 4 ++++1 file changed, 4 insertions(+)drivers/vhost/vsock.c +4
-1@@ -722,6 +722,10 @@ static void vhost_vsock_reset_orphans(struct sock *sk)* executing. */ +/* Only the h2g connections are reset */ +if (vsk->transport != &vhost_transport.transport) +return ; +/* If the peer is still valid, no need to reset connection */ if (vhost_vsock_get (vsk->remote_addr.svm_cid))return ; --Quoted 2 lines
2.35.1.723.g4982287a31-goog
Thanks for your patch!
Yes, I see the problem and I think I introduced it with the multi-transports support (ooops).
So we should add this fixes tag:
Fixes c0cfa2d8a788 ("vsock: add multi-transports support")IIUC the problem is for all transports that should only cycle on their own sockets. Indeed I think there is the same problem if the g2h driver will be unloaded (or a reset event is received after a VM migration), it will close all sockets of the nested h2g.
So I suggest a more generic solution, modifying
vsock_for_each_connected_socket()like this (not tested):net/vmw_vsock/af_vsock.c +7
-2@@ -334,7 +334,8 @@ void vsock_remove_sock(struct vsock_sock *vsk) }EXPORT_SYMBOL_GPL (vsock_remove_sock); -void vsock_for_each_connected_socket (void (*fn)(struct sock *sk)) +void vsock_for_each_connected_socket (struct vsock_transport *transport, +void (*fn)(struct sock *sk)) {int i;@@ -343,8 +344,12 @@ void vsock_for_each_connected_socket(void (*fn)(struct sock *sk))for (i =0 ; i <ARRAY_SIZE (vsock_connected_table); i++) {struct vsock_sock *vsk;list_for_each_entry (vsk, &vsock_connected_table[i], - connected_table) + connected_table) { +if (vsk->transport != transport) +continue ; +fn (sk_vsock (vsk)); + } }And all transports that call it.
Quoted 3 lines
Thanks,
Stefano
Thanks for the suggestion, which looks much better. It actually worked well.
By the way, the suggested change will alter the kernel-module interface (KMI), which will make it difficult to land the change on older releases where we'd like to keep the KMI stable [1]. Would it be OK if we let the supplied function (fn) be responsible for checking the transport? I think that there, in the future,
might be a case where one needs to cycle over all sockets for inspection or so. I admit that this would be prone to error, though.
Please let me know what you think. I don't have a strong preference. I will submit a revision as you want.
[1] https://source.android.com/devices/architecture/kernel/generic-kernel-image#kmi-stability
Stefano Garzarella
maintainerQuoted 92 lines · Jiyong Park
Hi Stefano,
Quoted 89 lines · Stefano Garzarella
Hi Jiyong,
Quoted 36 lines · Jiyong Park
Filtering non-h2g connections out when determining orphaned connections. Otherwise, in a nested VM configuration, destroying the nested VM (which often involves the closing of
/dev/vhost-vsockif there was h2g connections to the nested VM) kills not only the h2g connections, but also all existing g2h connections to the (outmost) host which are totally unrelated.Tested: Executed the following steps on Cuttlefish (Android running on a VM) [1]: (1) Enter into an
adb shellsession - to have a g2h connection inside the VM, (2) open and then close/dev/vhost-vsockbyexec 3< /dev/vhost-vsock && exec 3<&-, (3) observe that the adb session is not reset.[1] https://android.googlesource.com/device/google/cuttlefish/
Signed-off-by Jiyong Park <[email protected]>---
drivers/vhost/vsock.c| 4 ++++1 file changed, 4 insertions(+)drivers/vhost/vsock.c +4
-1@@ -722,6 +722,10 @@ static void vhost_vsock_reset_orphans(struct sock *sk)* executing. */ +/* Only the h2g connections are reset */ +if (vsk->transport != &vhost_transport.transport) +return ; +/* If the peer is still valid, no need to reset connection */ if (vhost_vsock_get (vsk->remote_addr.svm_cid))return ; --Quoted 2 lines
2.35.1.723.g4982287a31-goog
Thanks for your patch!
Yes, I see the problem and I think I introduced it with the multi-transports support (ooops).
So we should add this fixes tag:
Fixes c0cfa2d8a788 ("vsock: add multi-transports support")IIUC the problem is for all transports that should only cycle on their own sockets. Indeed I think there is the same problem if the g2h driver will be unloaded (or a reset event is received after a VM migration), it will close all sockets of the nested h2g.
So I suggest a more generic solution, modifying
vsock_for_each_connected_socket()like this (not tested):net/vmw_vsock/af_vsock.c +7
-2@@ -334,7 +334,8 @@ void vsock_remove_sock(struct vsock_sock *vsk) }EXPORT_SYMBOL_GPL (vsock_remove_sock); -void vsock_for_each_connected_socket (void (*fn)(struct sock *sk)) +void vsock_for_each_connected_socket (struct vsock_transport *transport, +void (*fn)(struct sock *sk)) {int i;@@ -343,8 +344,12 @@ void vsock_for_each_connected_socket(void (*fn)(struct sock *sk))for (i =0 ; i <ARRAY_SIZE (vsock_connected_table); i++) {struct vsock_sock *vsk;list_for_each_entry (vsk, &vsock_connected_table[i], - connected_table) + connected_table) { +if (vsk->transport != transport) +continue ; +fn (sk_vsock (vsk)); + } }
Quoted 7 lines
Quoted 5 lines
And all transports that call it.
Thanks,
StefanoThanks for the suggestion, which looks much better. It actually worked well.
Thanks for trying this!
Quoted 11 lines
By the way, the suggested change will alter the kernel-module interface (KMI), which will make it difficult to land the change on older releases where we'd like to keep the KMI stable [1]. Would it be OK if we let the supplied function (fn) be responsible for checking the transport? I think that there, in the future,
might be a case where one needs to cycle over all sockets for inspection or so. I admit that this would be prone to error, though.Please let me know what you think. I don't have a strong preference. I will submit a revision as you want.
I see your point, and it makes sense to keep KMI on stable branches, but mainline I would like to have the proposed approach since all transports use it to cycle on their sockets.
So we could do a series with 2 patches:
- Patch 1 fixes the problem in all transports by checking the transport in the callback (here we insert the fixes tag so we backport it to the stable branches)
- Patch 2 moves the check in vsock_for_each_connected_socket() removing it from callbacks so it is less prone to errors and we merge it only mainline
What do you think?
Thanks,
Stefano
Quoted 12 lines
I see your point, and it makes sense to keep KMI on stable branches, but mainline I would like to have the proposed approach since all transports use it to cycle on their sockets.
So we could do a series with 2 patches:
- Patch 1 fixes the problem in all transports by checking the transport in the callback (here we insert the fixes tag so we backport it to the stable branches)
- Patch 2 moves the check invsock_for_each_connected_socket()removing it from callbacks so it is less prone to errors and we merge it only mainline
Quoted 5 lines
What do you think?
Thanks,
Stefano
That sounds like a plan. Will submit a new series soon.