vsock: Add flags field in the vsock address
Subsystems (3)
v4 cover vsock: Add flags field in the vsock address
vsock enables communication between virtual machines and the host they are running on. Nested VMs can be setup to use vsock channels, as the multi transport support has been available in the mainline since the v5.5 Linux kernel has been released.
Implicitly, if no host->guest vsock transport is loaded, all the vsock packets are forwarded to the host. This behavior can be used to setup communication channels between sibling VMs that are running on the same host. One example can be the vsock channels that can be established within AWS Nitro Enclaves (see Documentation/virt/ne_overview.rst).
To be able to explicitly mark a connection as being used for a certain use case, add a flags field in the vsock address data structure. The value of the flags field is taken into consideration when the vsock transport is assigned. This way can distinguish between different use cases, such as nested VMs / local communication and sibling VMs.
The flags field can be set in the user space application connect logic. On the listen path, the field can be set in the kernel space logic.
Thank you.
Andra
---
Patch Series Changelog
The patch series is built on top of v5.10.
GitHub repo branch for the latest version of the patch series:
* https://github.com/andraprs/linux/tree/vsock-flag-sibling-comm-v4
v3 -> v4
* Rebase on top of v5.10.
* Add check for supported flag values.
* Update the "svm_flags" field to be 1 byte instead of 2 bytes.
* v3: https://lore.kernel.org/lkml/[email protected]/
v2 -> v3
* Rebase on top of v5.10-rc7.
* Add "svm_flags" as a new field, not reusing "svm_reserved1".
* Update comments to mention when the "VMADDR_FLAG_TO_HOST" flag is set in the connect and listen paths.
* Update bitwise check logic to not compare result to the flag value.
* v2: https://lore.kernel.org/lkml/[email protected]/
v1 -> v2
* Update the vsock flag naming to "VMADDR_FLAG_TO_HOST".
* Use bitwise operators to setup and check the vsock flag.
* Set the vsock flag on the receive path in the vsock transport assignment logic.
* Merge the checks for the g2h transport assignment in one "if" block.
* v1: https://lore.kernel.org/lkml/[email protected]/
---
Andra Paraschiv (5):
vm_sockets: Add flags field in the vsock address data structure
vm_sockets: Add VMADDR_FLAG_TO_HOST vsock flag
vsock_addr: Check for supported flag values
af_vsock: Set VMADDR_FLAG_TO_HOST flag on the receive path
af_vsock: Assign the vsock transport considering the vsock address
flags
include/uapi/linux/vm_sockets.h | 26 +++++++++++++++++++++++++- net/vmw_vsock/af_vsock.c | 21 +++++++++++++++++++-- net/vmw_vsock/vsock_addr.c | 4 +++- 3 files changed, 47 insertions(+), 4 deletions(-)
Signature · 6 lines
2.20.1 (Apple Git-117) Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
PATCH v4 1/5 vm_sockets: Add flags field in the vsock address data structure
vsock enables communication between virtual machines and the host they are running on. With the multi transport support (guest->host and host->guest), nested VMs can also use vsock channels for communication.
In addition to this, by default, all the vsock packets are forwarded to the host, if no host->guest transport is loaded. This behavior can be implicitly used for enabling vsock communication between sibling VMs.
Add a flags field in the vsock address data structure that can be used to explicitly mark the vsock connection as being targeted for a certain type of communication. This way, can distinguish between different use cases such as nested VMs and sibling VMs.
This field can be set when initializing the vsock address variable used for the connect() call.
Changelog
v3 -> v4
* Update the size of "svm_flags" field to be 1 byte instead of 2 bytes.
v2 -> v3
* Add "svm_flags" as a new field, not reusing "svm_reserved1".
v1 -> v2
* Update the field name to "svm_flags".
* Split the current patch in 2 patches.
--- include/uapi/linux/vm_sockets.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
include/uapi/linux/vm_sockets.h +5 -1
@@ -18,6 +18,7 @@#define _UAPI_VM_SOCKETS_H#include <linux/socket.h> +#include <linux/types.h> /* Option name for STREAM socket buffer size. Use as the option name in * setsockopt(3) or getsockopt(3) to set or get an unsigned long long that
@@ -148,10 +149,13 @@ struct sockaddr_vm {unsigned short svm_reserved1;unsigned int svm_port;unsigned int svm_cid; +__u8 svm_flags;unsigned char svm_zero[sizeof (struct sockaddr ) -sizeof (sa_family_t) -sizeof (unsigned short ) - -sizeof (unsigned int ) -sizeof (unsigned int )]; +sizeof (unsigned int ) - +sizeof (unsigned int ) - +sizeof (__u8)]; };#define IOCTL_VM_SOCKETS_GET_LOCAL_CID _IO(7, 0xb9)
Signature · 6 lines
2.20.1 (Apple Git-117) Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
PATCH v4 2/5 vm_sockets: Add VMADDR_FLAG_TO_HOST vsock flag
In reply to Andra ParaschivAdd VMADDR_FLAG_TO_HOST vsock flag that is used to setup a vsock connection where all the packets are forwarded to the host.
Then, using this type of vsock channel, vsock communication between sibling VMs can be built on top of it.
Changelog
v3 -> v4
* Update the "VMADDR_FLAG_TO_HOST" value, as the size of the field has been updated to 1 byte.
v2 -> v3
* Update comments to mention when the flag is set in the connect and listen paths.
v1 -> v2
* New patch in v2, it was split from the first patch in the series.
* Remove the default value for the vsock flags field.
* Update the naming for the vsock flag to "VMADDR_FLAG_TO_HOST".
--- include/uapi/linux/vm_sockets.h | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
include/uapi/linux/vm_sockets.h +20 -0
@@ -115,6 +115,26 @@#define VMADDR_CID_HOST 2 +/* The current default use case for the vsock channel is the following: +* local vsock communication between guest and host and nested VMs setup. +* In addition to this, implicitly, the vsock packets are forwarded to the host +* if no host->guest vsock transport is set. +* +* Set this flag value in the sockaddr_vm corresponding field if the vsock +* packets need to be always forwarded to the host. Using this behavior, +* vsock communication between sibling VMs can be setup. +* +* This way can explicitly distinguish between vsock channels created for +* different use cases, such as nested VMs (or local communication between +* guest and host) and sibling VMs. +* +* The flag can be set in the connect logic in the user space application flow. +* In the listen logic (from kernel space) the flag is set on the remote peer +* address. This happens for an incoming connection when it is routed from the +* host and comes from the guest (local CID and remote CID > VMADDR_CID_HOST). +*/ +#define VMADDR_FLAG_TO_HOST 0x01 +/* Invalid vSockets version. */ #define VM_SOCKETS_INVALID_VERSION -1U
Signature · 6 lines
2.20.1 (Apple Git-117) Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
PATCH v4 3/5 vsock_addr: Check for supported flag values
In reply to Andra ParaschivCheck if the provided flags value from the vsock address data structure includes the supported flags in the corresponding kernel version.
The first byte of the "svm_zero" field is used as "svm_flags", so add the flags check instead.
Changelog
v3 -> v4
* New patch in v4.
--- net/vmw_vsock/vsock_addr.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
net/vmw_vsock/vsock_addr.c +3 -1
@@ -22,13 +22,15 @@ EXPORT_SYMBOL_GPL(vsock_addr_init);int vsock_addr_validate (const struct sockaddr_vm *addr) { +__u8 svm_valid_flags =VMADDR_FLAG_TO_HOST ; +if (!addr)return -EFAULT ;if (addr->svm_family !=AF_VSOCK )return -EAFNOSUPPORT ; -if (addr->svm_zero[0 ] !=0 ) +if (addr->svm_flags & ~svm_valid_flags)return -EINVAL ;return 0 ;
Signature · 6 lines
2.20.1 (Apple Git-117) Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
Re: [PATCH net-next v4 3/5] vsock_addr: Check for supported flag values
Quoted 8 lines · Andra Paraschiv
Check if the provided flags value from the vsock address data structure includes the supported flags in the corresponding kernel version.
The first byte of the "
svm_zero" field is used as "svm_flags", so add the flags check instead.Changelog
Quoted 8 lines
v3 -> v4
* New patch in v4.
Signed-off-by Andra Paraschiv <[email protected]>---
net/vmw_vsock/vsock_addr.c| 4 +++-1 file changed, 3 insertions(+), 1 deletion(-)
Quoted 30 lines
net/vmw_vsock/vsock_addr.c +3
-1@@ -22,13 +22,15 @@ EXPORT_SYMBOL_GPL(vsock_addr_init);int vsock_addr_validate (const struct sockaddr_vm *addr) { +__u8 svm_valid_flags =VMADDR_FLAG_TO_HOST ; +if (!addr)return -EFAULT ;if (addr->svm_family !=AF_VSOCK )return -EAFNOSUPPORT ; -if (addr->svm_zero[0 ] !=0 ) +if (addr->svm_flags & ~svm_valid_flags)return -EINVAL ;return 0 ;Signature · 6 lines
2.20.1 (Apple Git-117) Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
PATCH v4 4/5 af_vsock: Set VMADDR_FLAG_TO_HOST flag on the receive path
In reply to Andra ParaschivThe vsock flags can be set during the connect() setup logic, when initializing the vsock address data structure variable. Then the vsock transport is assigned, also considering this flags field.
The vsock transport is also assigned on the (listen) receive path. The flags field needs to be set considering the use case.
Set the value of the vsock flags of the remote address to the one targeted for packets forwarding to the host, if the following conditions are met:
* The source CID of the packet is higher than VMADDR_CID_HOST.
* The destination CID of the packet is higher than VMADDR_CID_HOST.
Changelog
v3 -> v4
* No changes.
v2 -> v3
* No changes.
v1 -> v2
* Set the vsock flag on the receive path in the vsock transport assignment logic.
* Use bitwise operator for the vsock flag setup.
* Use the updated "VMADDR_FLAG_TO_HOST" flag naming.
--- net/vmw_vsock/af_vsock.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
net/vmw_vsock/af_vsock.c +12 -0
@@ -431,6 +431,18 @@ int vsock_assign_transport(struct vsock_sock *vsk, struct vsock_sock *psk)unsigned int remote_cid = vsk->remote_addr.svm_cid;int ret; +/* If the packet is coming with the source and destination CIDs higher +* than VMADDR_CID_HOST, then a vsock channel where all the packets are +* forwarded to the host should be established. Then the host will +* need to forward the packets to the guest. +* +* The flag is set on the (listen) receive path (psk is not NULL). On +* the connect path the flag can be set by the user space application. +*/ +if (psk && vsk->local_addr.svm_cid >VMADDR_CID_HOST && + vsk->remote_addr.svm_cid >VMADDR_CID_HOST ) + vsk->remote_addr.svm_flags |=VMADDR_FLAG_TO_HOST ; +switch (sk->sk_type) {case SOCK_DGRAM : new_transport = transport_dgram;
Signature · 6 lines
2.20.1 (Apple Git-117) Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
PATCH v4 5/5 af_vsock: Assign the vsock transport considering the vsock address flags
In reply to Andra ParaschivThe vsock flags field can be set in the connect path (user space app) and the (listen) receive path (kernel space logic).
When the vsock transport is assigned, the remote CID is used to distinguish between types of connection.
Use the vsock flags value (in addition to the CID) from the remote address to decide which vsock transport to assign. For the sibling VMs use case, all the vsock packets need to be forwarded to the host, so always assign the guest->host transport if the VMADDR_FLAG_TO_HOST flag is set. For the other use cases, the vsock transport assignment logic is not changed.
Changelog
v3 -> v4
* Update the "remote_flags" local variable type to reflect the change of the "svm_flags" field to be 1 byte in size.
v2 -> v3
* Update bitwise check logic to not compare result to the flag value.
v1 -> v2
* Use bitwise operator to check the vsock flag.
* Use the updated "VMADDR_FLAG_TO_HOST" flag naming.
* Merge the checks for the g2h transport assignment in one "if" block.
--- net/vmw_vsock/af_vsock.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
net/vmw_vsock/af_vsock.c +7 -2
@@ -421,7 +421,8 @@ static void vsock_deassign_transport(struct vsock_sock *vsk)* The vsk->remote_addr is used to decide which transport to use: * - remote CID == VMADDR_CID_LOCAL or g2h->local_cid or VMADDR_CID_HOST if * g2h is not loaded, will use local transport; -* - remote CID <= VMADDR_CID_HOST will use guest->host transport; +* - remote CID <= VMADDR_CID_HOST or h2g is not loaded or remote flags field +* includes VMADDR_FLAG_TO_HOST flag value, will use guest->host transport; * - remote CID > VMADDR_CID_HOST will use host->guest transport; */ int vsock_assign_transport (struct vsock_sock *vsk,struct vsock_sock *psk)
@@ -429,6 +430,7 @@ int vsock_assign_transport(struct vsock_sock *vsk, struct vsock_sock *psk)const struct vsock_transport *new_transport;struct sock *sk =sk_vsock (vsk);unsigned int remote_cid = vsk->remote_addr.svm_cid; +__u8 remote_flags;int ret;/* If the packet is coming with the source and destination CIDs higher
@@ -443,6 +445,8 @@ int vsock_assign_transport(struct vsock_sock *vsk, struct vsock_sock *psk) vsk->remote_addr.svm_cid > VMADDR_CID_HOST) vsk->remote_addr.svm_flags |= VMADDR_FLAG_TO_HOST; + remote_flags = vsk->remote_addr.svm_flags; +switch (sk->sk_type) {case SOCK_DGRAM : new_transport = transport_dgram;
@@ -450,7 +454,8 @@ int vsock_assign_transport(struct vsock_sock *vsk, struct vsock_sock *psk)case SOCK_STREAM :if (vsock_use_local_transport (remote_cid)) new_transport = transport_local; -else if (remote_cid <=VMADDR_CID_HOST || !transport_h2g) +else if (remote_cid <=VMADDR_CID_HOST || !transport_h2g || + (remote_flags &VMADDR_FLAG_TO_HOST )) new_transport = transport_g2h;else new_transport = transport_h2g;
Signature · 6 lines
2.20.1 (Apple Git-117) Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
Stefano Garzarella
maintainerQuoted 11 lines · Andra Paraschiv
vsock enables communication between virtual machines and the host they are running on. Nested VMs can be setup to use vsock channels, as the multi transport support has been available in the mainline since the v5.5 Linux kernel has been released.
Implicitly, if no
host->guestvsock transport is loaded, all the vsock packets are forwarded to the host. This behavior can be used to setup communication channels between sibling VMs that are running on the same host. One example can be the vsock channels that can be established within AWS Nitro Enclaves (seeDocumentation/virt/ne_overview.rst).
Quoted 8 lines
To be able to explicitly mark a connection as being used for a certain use case, add a flags field in the vsock address data structure. The value of the flags field is taken into consideration when the vsock transport is assigned. This way can distinguish between different use cases, such as nested VMs / local communication and sibling VMs.
The flags field can be set in the user space application connect logic. On the listen path, the field can be set in the kernel space logic.
I reviewed and tested all the patches, great job!
Thanks,
Stefano
Quoted 63 lines
Thank you.
Andra
---
Patch Series Changelog
The patch series is built on top of v5.10.
GitHub repo branch for the latest version of the patch series:
* https://github.com/andraprs/linux/tree/vsock-flag-sibling-comm-v4
v3 -> v4
* Rebase on top of v5.10.
* Add check for supported flag values.
* Update the "svm_flags" field to be 1 byte instead of 2 bytes.
* v3: https://lore.kernel.org/lkml/[email protected]/v2 -> v3
* Rebase on top of v5.10-rc7.
* Add "svm_flags" as a new field, not reusing "svm_reserved1".
* Update comments to mention when the "VMADDR_FLAG_TO_HOST" flag is set in the connect and listen paths.
* Update bitwise check logic to not compare result to the flag value.
* v2: https://lore.kernel.org/lkml/[email protected]/v1 -> v2
* Update the vsock flag naming to "
VMADDR_FLAG_TO_HOST".
* Use bitwise operators to setup and check the vsock flag.
* Set the vsock flag on the receive path in the vsock transport assignment logic.
* Merge the checks for the g2h transport assignment in one "if" block.
* v1: https://lore.kernel.org/lkml/[email protected]/---
Andra Paraschiv (5):
vm_sockets: Add flags field in the vsock address data structure
vm_sockets: AddVMADDR_FLAG_TO_HOSTvsock flag
vsock_addr: Check for supported flag values
af_vsock: SetVMADDR_FLAG_TO_HOSTflag on the receive path
af_vsock: Assign the vsock transport considering the vsock address
flagsinclude/uapi/linux/vm_sockets.h | 26 +++++++++++++++++++++++++- net/vmw_vsock/af_vsock.c | 21 +++++++++++++++++++-- net/vmw_vsock/vsock_addr.c | 4 +++- 3 files changed, 47 insertions(+), 4 deletions(-)Signature · 6 lines
2.20.1 (Apple Git-117) Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
Quoted 19 lines · Stefano Garzarella
Quoted 17 lines · Andra Paraschiv
vsock enables communication between virtual machines and the host they are running on. Nested VMs can be setup to use vsock channels, as the multi transport support has been available in the mainline since the v5.5 Linux kernel has been released.
Implicitly, if no
host->guestvsock transport is loaded, all the vsock packets are forwarded to the host. This behavior can be used to setup communication channels between sibling VMs that are running on the same host. One example can be the vsock channels that can be established within AWS Nitro Enclaves (seeDocumentation/virt/ne_overview.rst).To be able to explicitly mark a connection as being used for a certain use case, add a flags field in the vsock address data structure. The value of the flags field is taken into consideration when the vsock transport is assigned. This way can distinguish between different use cases, such as nested VMs / local communication and sibling VMs.
Quoted 5 lines
Quoted 2 lines
The flags field can be set in the user space application connect logic. On the listen path, the field can be set in the kernel space logic.
I reviewed and tested all the patches, great job!
Thanks for checking it out.
Andra
Quoted 56 lines
Quoted 56 lines
Thank you.
Andra
---
Patch Series Changelog
The patch series is built on top of v5.10.
GitHub repo branch for the latest version of the patch series:
* https://github.com/andraprs/linux/tree/vsock-flag-sibling-comm-v4
v3 -> v4
* Rebase on top of v5.10.
* Add check for supported flag values.
* Update the "svm_flags" field to be 1 byte instead of 2 bytes.
* v3: https://lore.kernel.org/lkml/[email protected]/v2 -> v3
* Rebase on top of v5.10-rc7.
* Add "svm_flags" as a new field, not reusing "svm_reserved1".
* Update comments to mention when the "VMADDR_FLAG_TO_HOST" flag is set in the connect and listen paths.
* Update bitwise check logic to not compare result to the flag value.
* v2: https://lore.kernel.org/lkml/[email protected]/v1 -> v2
* Update the vsock flag naming to "
VMADDR_FLAG_TO_HOST".
* Use bitwise operators to setup and check the vsock flag.
* Set the vsock flag on the receive path in the vsock transport assignment logic.
* Merge the checks for the g2h transport assignment in one "if" block.
* v1: https://lore.kernel.org/lkml/[email protected]/---
Andra Paraschiv (5):
vm_sockets: Add flags field in the vsock address data structure
vm_sockets: AddVMADDR_FLAG_TO_HOSTvsock flag
vsock_addr: Check for supported flag values
af_vsock: SetVMADDR_FLAG_TO_HOSTflag on the receive path
af_vsock: Assign the vsock transport considering the vsock address flags
include/uapi/linux/vm_sockets.h| 26 +++++++++++++++++++++++++- net/vmw_vsock/af_vsock.c | 21 +++++++++++++++++++-- net/vmw_vsock/vsock_addr.c | 4 +++-3 files changed, 47 insertions(+), 4 deletions(-)
Quoted 9 lines
Quoted 8 lines
Signature · 6 lines
2.20.1 (Apple Git-117) Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
This series was applied to netdev/net-next.git (refs/heads/master):
Hello:
This series was applied to netdev/net-next.git (refs/heads/master):
Quoted 5 lines · On Mon, 14 Dec 2020 18:11:17 +0200 you wrote:
vsock enables communication between virtual machines and the host they are running on. Nested VMs can be setup to use vsock channels, as the multi transport support has been available in the mainline since the v5.5 Linux kernel has been released.
Quoted 7 lines
Implicitly, if no
host->guestvsock transport is loaded, all the vsock packets are forwarded to the host. This behavior can be used to setup communication channels between sibling VMs that are running on the same host. One example can be the vsock channels that can be established within AWS Nitro Enclaves (seeDocumentation/virt/ne_overview.rst).[...]
Here is the summary with links:
- [net-next,v4,1/5] vm_sockets: Add flags field in the vsock address data structure
https://git.kernel.org/netdev/net-next/c/dc8eeef73b63 - [net-next,v4,2/5] vm_sockets: Add VMADDR_FLAG_TO_HOST vsock flag https://git.kernel.org/netdev/net-next/c/caaf95e0f23f - [net-next,v4,3/5] vsock_addr: Check for supported flag values https://git.kernel.org/netdev/net-next/c/cada7ccd9dc7 - [net-next,v4,4/5] af_vsock: Set VMADDR_FLAG_TO_HOST flag on the receive path https://git.kernel.org/netdev/net-next/c/1b5f2ab98e7f - [net-next,v4,5/5] af_vsock: Assign the vsock transport considering the vsock address flags https://git.kernel.org/netdev/net-next/c/7f816984f439
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html